From 6832cac3f38d325e619c2be791aed9fe68c5882a Mon Sep 17 00:00:00 2001 From: divya-mohan0209 Date: Sun, 8 Mar 2020 01:35:34 +0530 Subject: [PATCH 001/140] Update content in intermediate.md (#19524) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Update content in intermediate.md Replaced “If your feature is moving out of Alpha, make sure to remove it from that file.” with “If your feature is moving to Beta or to General Availability, update the feature gates file.” * Revised update intermediate.md Preserved the wrapping per Tim's suggestion --- content/en/docs/contribute/intermediate.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/en/docs/contribute/intermediate.md b/content/en/docs/contribute/intermediate.md index 2da6ccf523..9e477a90a4 100644 --- a/content/en/docs/contribute/intermediate.md +++ b/content/en/docs/contribute/intermediate.md @@ -911,8 +911,8 @@ deadlines. Some deadlines related to documentation are: If your feature is an Alpha feature and is behind a feature gate, make sure you add it to [Feature gates](/docs/reference/command-line-tools-reference/feature-gates/) -as part of your pull request. If your feature is moving out of Alpha, make sure to -remove it from that file. +as part of your pull request. If your feature is moving to Beta +or to General Availability, update the feature gates file. ## Contribute to other repos From c81f153cfd2f78c05fdb1c97bed136d98b691d77 Mon Sep 17 00:00:00 2001 From: Jie Shen Date: Sun, 8 Mar 2020 11:31:34 +0800 Subject: [PATCH 002/140] Fix rendering issue for blogs (#19536) --- .../2015-03-00-Kubernetes-Gathering-Videos.md | 1 - ...-00-Weekly-Kubernetes-Community-Hangout.md | 5 +-- .../2015-03-00-Welcome-To-Kubernetes-Blog.md | 3 +- .../2015-04-00-Kubernetes-Release-0150.md | 1 - ...-Weekly-Kubernetes-Community-Hangout_17.md | 15 ++++---- ...-Weekly-Kubernetes-Community-Hangout_29.md | 5 +-- .../2015-05-00-Kubernetes-On-Openstack.md | 1 - ...-00-Weekly-Kubernetes-Community-Hangout.md | 17 ++++----- ...15-06-00-Slides-Cluster-Management-With.md | 1 - ...-Announcing-First-Kubernetes-Enterprise.md | 1 - ...And-Replication-Controllers-With-Puppet.md | 29 +++++++-------- ...-Simple-Leader-Election-With-Kubernetes.md | 29 +++++++-------- ...00-Why-Kubernetes-Doesnt-Use-Libnetwork.md | 1 - ...Kubecon-Eu-2016-Kubernetes-Community-In.md | 21 +++++------ ...2-00-Kubernetes-Community-Meeting-Notes.md | 11 +++--- ...6-07-00-Citrix-Netscaler-And-Kubernetes.md | 21 +++++------ ...-Dashboard-Web-Interface-For-Kubernetes.md | 11 +++--- .../2016-07-00-Oh-The-Places-You-Will-Go.md | 37 +++++++++---------- .../2017-10-00-Five-Days-Of-Kubernetes-18.md | 1 - .../2017-11-00-Autoscaling-In-Kubernetes.md | 5 +-- 20 files changed, 98 insertions(+), 118 deletions(-) diff --git a/content/zh/blog/_posts/2015-03-00-Kubernetes-Gathering-Videos.md b/content/zh/blog/_posts/2015-03-00-Kubernetes-Gathering-Videos.md index 2576296c64..c53ea68585 100644 --- a/content/zh/blog/_posts/2015-03-00-Kubernetes-Gathering-Videos.md +++ b/content/zh/blog/_posts/2015-03-00-Kubernetes-Gathering-Videos.md @@ -3,7 +3,6 @@ title: " Kubernetes 采集视频 " date: 2015-03-23 slug: kubernetes-gathering-videos -url: /blog/2015/03/Kubernetes-Gathering-Videos --- * kubectl exec -p $POD -- $CMD @@ -116,7 +115,7 @@ Notes from meeting: * want to inject a binary under control of the host, similar to pre-start hooks * socat, nsenter, whatever the pre-start hook needs - + --> * 想要在主机的控制下注入二进制文件，类似于预启动钩子 diff --git a/content/zh/blog/_posts/2015-03-00-Welcome-To-Kubernetes-Blog.md b/content/zh/blog/_posts/2015-03-00-Welcome-To-Kubernetes-Blog.md index 3e3b5f6d3d..3b35b0b21a 100644 --- a/content/zh/blog/_posts/2015-03-00-Welcome-To-Kubernetes-Blog.md +++ b/content/zh/blog/_posts/2015-03-00-Welcome-To-Kubernetes-Blog.md @@ -2,12 +2,11 @@ title: 欢迎来到 Kubernetes 博客! date: 2015-03-20 slug: welcome-to-kubernetes-blog -url: /blog/2015/03/Welcome-To-Kubernetes-Blog --- 每个星期，Kubernetes 贡献者社区几乎都会在谷歌 Hangouts 上聚会。我们希望任何对此感兴趣的人都能了解这个论坛的讨论内容。 -议程 +议程 * Mesos 集成 * 高可用性（HA） @@ -36,7 +35,7 @@ Agenda * 客户端版本化 笔记 @@ -71,7 +70,7 @@ Notes * Load-balance apiserver. * Cold standby for controller manager and other master components. - + --> * HA @@ -95,7 +94,7 @@ Notes * See * Justin working on multi-platform e2e dashboard - + --> * 向 e2e 添加性能和分析详细信息以跟踪回归 @@ -123,7 +122,7 @@ Notes * Structured types are useful in the client. Versioned structs would be ok. * If start with json/yaml (kubectl), shouldn’t convert to structured types. Use swagger. - + --> * 客户端版本化 diff --git a/content/zh/blog/_posts/2015-04-00-Weekly-Kubernetes-Community-Hangout_29.md b/content/zh/blog/_posts/2015-04-00-Weekly-Kubernetes-Community-Hangout_29.md index c451022671..399d926792 100644 --- a/content/zh/blog/_posts/2015-04-00-Weekly-Kubernetes-Community-Hangout_29.md +++ b/content/zh/blog/_posts/2015-04-00-Weekly-Kubernetes-Community-Hangout_29.md @@ -2,7 +2,6 @@ title: " Kubernetes 社区每周聚会笔记- 2015年4月24日 " date: 2015-04-30 slug: weekly-kubernetes-community-hangout_29 -url: /blog/2015/04/Weekly-Kubernetes-Community-Hangout_29 --- 每个星期，Kubernetes 贡献者社区几乎都会在谷歌 Hangouts 上聚会。我们希望任何对此感兴趣的人都能了解这个论坛的讨论内容。 @@ -85,7 +84,7 @@ Notes: * Brendan: 请求，它如何查找重复请求？Cassandra 希望在底层复制数据。向上和向下扩缩是有效的。根据负载动态地创建存储。它的步骤不仅仅是快照——通过编程使用预分配创建副本。 * Tim: 帮助自动配置。 - + +--> * 简单的滚动更新 - Brendan @@ -58,7 +57,7 @@ Every week the Kubernetes contributing community meet virtually over Google Hang * Can run AppContainer and docker containers in same pod. * Changes are close to merged. - + --> * Rocket 演示 - CoreOS 的伙计们 @@ -88,7 +87,7 @@ Every week the Kubernetes contributing community meet virtually over Google Hang * * Can create new service account with ServiceAccountToken. Controller will create token for it. * Can create a pod with service account, pods will have service account secret mounted at /var/run/secrets/kubernetes.io/… - + --> * 演示 service accounts 和 secrets 被添加到 pod - Jordan @@ -106,16 +105,16 @@ Every week the Kubernetes contributing community meet virtually over Google Hang * * 可以使用 ServiceAccountToken 创建新的 service account。控制器将为它创建令牌。 * 可以创建一个带有 service account 的 pod, pod 将在 /var/run/secrets/kubernets.io/… - - - + * Kubelet 在容器中运行 - Paul * Kubelet 成功地运行了带有 secret 的 pod。 - + diff --git a/content/zh/blog/_posts/2015-06-00-Slides-Cluster-Management-With.md b/content/zh/blog/_posts/2015-06-00-Slides-Cluster-Management-With.md index 8e1c001d65..e2937cd16e 100644 --- a/content/zh/blog/_posts/2015-06-00-Slides-Cluster-Management-With.md +++ b/content/zh/blog/_posts/2015-06-00-Slides-Cluster-Management-With.md @@ -2,7 +2,6 @@ title: "幻灯片：Kubernetes 集群管理，爱丁堡大学演讲" date: 2015-06-26 slug: slides-cluster-management-with -url: /blog/2015/06/Slides-Cluster-Management-With --- _今天的嘉宾帖子是由 IT 自动化领域的领导者 Puppet Labs 的高级软件工程师 Gareth Rushgrove 撰写的。Gareth告诉我们一个新的 Puppet 模块，它帮助管理 Kubernetes 中的资源。_ @@ -27,7 +26,7 @@ _今天的嘉宾帖子是由 IT 自动化领域的领导者 Puppet Labs 的高 ### Puppet Kubernetes 模块 @@ -48,7 +47,7 @@ kubernetes_pod { 'sample-pod': }] }, ``` -} +} --> ``` @@ -63,10 +62,10 @@ kubernetes_pod { 'sample-pod': image => 'nginx', }] }, -} +} ``` @@ -91,7 +90,7 @@ Kubernetes has several resources, from Pods and Services to Replication Controll Kubernetes 有很多资源，来自 Pods、 Services、 Replication Controllers 和 Service Accounts。您可以在[Puppet 中的 kubernetes 留言簿示例](https://puppetlabs.com/blog/kubernetes-guestbook-example-puppet)文章中看到管理这些资源的模块示例。这演示了如何将规范的 hello-world 示例转换为使用 Puppet代码。 @@ -113,17 +112,17 @@ guestbook { 'myguestbook': frontend_replicas => 3, redis_master_image => 'redis', redis_slave_image => 'gcr.io/google_samples/gb-redisslave:v1', - frontend_image => 'gcr.io/google_samples/gb-frontend:v3', + frontend_image => 'gcr.io/google_samples/gb-frontend:v3', } ``` 您可以在Puppet博客文章[在 Puppet 中为 Kubernetes 构建自己的抽象](https://puppetlabs.com/blog/building-your-own-abstractions-kubernetes-puppet)中阅读更多关于使用 Puppet 定义的类型的信息，并看到更多的代码示例。 @@ -146,13 +145,13 @@ The advantages of using Puppet rather than just the standard YAML files and kube - 能够针对 Kubernetes API 重复运行相同的代码，以检测任何更改或修正配置。 值得注意的是，大多数大型组织都将拥有非常异构的环境，运行各种各样的软件和操作系统。拥有统一这些离散系统的单一工具链可以使采用 Kubernetes 等新技术变得更加容易。 diff --git a/content/zh/blog/_posts/2016-01-00-Simple-Leader-Election-With-Kubernetes.md b/content/zh/blog/_posts/2016-01-00-Simple-Leader-Election-With-Kubernetes.md index bcea1c026e..a1c9bb3c3b 100644 --- a/content/zh/blog/_posts/2016-01-00-Simple-Leader-Election-With-Kubernetes.md +++ b/content/zh/blog/_posts/2016-01-00-Simple-Leader-Election-With-Kubernetes.md @@ -3,7 +3,6 @@ title: " Simple leader election with Kubernetes and Docker " date: 2016-01-11 slug: simple-leader-election-with-kubernetes -url: /blog/2016/01/Simple-Leader-Election-With-Kubernetes --- #### Overview @@ -59,13 +58,13 @@ Given these primitives, the code to use master election is relatively straightfo 给定这些原语，使用 master election 的代码相对简单，您可以在这里找到[here][1]。我们自己来做吧。 -``` +``` $ kubectl run leader-elector --image=gcr.io/google_containers/leader-elector:0.4 --replicas=3 -- --election=example ``` 这将创建一个包含3个副本的 leader election 集合： -``` +``` $ kubectl get pods NAME READY STATUS RESTARTS AGE leader-elector-inmr1 1/1 Running 0 13s @@ -91,7 +90,7 @@ leader-elector-sgwcq 1/1 Running 0 13s @@ -127,7 +126,7 @@ _'example' 是上面 kubectl run … 命令_中候选集的名称 $ kubectl get endpoints example -o yaml ``` 现在，要验证 leader election 是否实际有效，请在另一个终端运行： -``` +``` $ kubectl delete pods (leader-pod-name) ``` @@ -142,7 +141,7 @@ The leader-election container provides a simple webserver that can serve on any Leader-election container 提供了一个简单的 web 服务器，可以服务于任何地址(e.g. http://localhost:4040)。您可以通过删除现有的 leader election 组并创建一个新的 leader elector 组来测试这一点，在该组中，您还可以向 leader elector 映像传递--http=(host):(port) 规范。这将导致集合中的每个成员通过 webhook 提供有关领导者的信息。 -``` +``` # delete the old leader elector group $ kubectl delete rc leader-elector @@ -174,7 +173,7 @@ http://localhost:8001/api/v1/proxy/namespaces/default/pods/(leader-pod-name):404 And you will see: -``` +``` {"name":"(name-of-leader-here)"} ``` #### Leader election with sidecars @@ -192,7 +191,7 @@ http://localhost:8001/api/v1/proxy/namespaces/default/pods/(leader-pod-name):404 你会看到： -``` +``` {"name":"(name-of-leader-here)"} ``` #### 有副手的 leader election @@ -209,7 +208,7 @@ Leader-election container 可以作为一个 sidecar，您可以从自己的应 -* “Kubernetes 硬件黑客：通过旋钮、推杆和滑块探索 Kubernetes API” 演讲者 Ian Lewis 和 Brian Dorsey，谷歌开发布道师* [http://sched.co/6Bl3](http://sched.co/6Bl3) +* “Kubernetes 硬件黑客：通过旋钮、推杆和滑块探索 Kubernetes API” 演讲者 Ian Lewis 和 Brian Dorsey，谷歌开发布道师* [http://sched.co/6Bl3](http://sched.co/6Bl3) * “rktnetes: 容器运行时和 Kubernetes 的新功能” 演讲者 Jonathan Boulle, CoreOS 的主程 -* [http://sched.co/6BY7](http://sched.co/6BY7) * “Kubernetes 文档：贡献、修复问题、收集奖金” 作者：John Mulhausen，首席技术作家，谷歌 -* [http://sched.co/6BUP](http://sched.co/6BUP)  * “[OpenStack 在 Kubernetes 的世界中扮演什么角色？](https://kubeconeurope2016.sched.org/event/6BYC/what-is-openstacks-role-in-a-kubernetes-world?iframe=yes&w=i:0;&sidebar=yes&bg=no#?iframe=yes&w=i:100;&sidebar=yes&bg=no)” 作者：Thierry carez, OpenStack 基金会工程总监 -* http://sched.co/6BYC -* “容器调度的实用指南” 作者：Mandy Waite，开发者倡导者，谷歌 -* [http://sched.co/6BZa](http://sched.co/6BZa) +* “容器调度的实用指南” 作者：Mandy Waite，开发者倡导者，谷歌 -* [http://sched.co/6BZa](http://sched.co/6BZa) * “[《纽约时报》编辑部正在制作 Kubernetes](https://kubeconeurope2016.sched.org/event/67f2/kubernetes-in-production-in-the-new-york-times-newsroom?iframe=yes&w=i:0;&sidebar=yes&bg=no#?iframe=yes&w=i:100;&sidebar=yes&bg=no)” Eric Lewis，《纽约时报》网站开发人员 -* [http://sched.co/67f2](http://sched.co/67f2) * “[使用 NGINX 为 Kubernetes 创建一个高级负载均衡解决方案](https://kubeconeurope2016.sched.org/event/6Bc9/creating-an-advanced-load-balancing-solution-for-kubernetes-with-nginx?iframe=yes&w=i:0;&sidebar=yes&bg=no#?iframe=yes&w=i:100;&sidebar=yes&bg=no)” 作者：Andrew Hutchings, NGINX 技术产品经理 -* http://sched.co/6Bc9 @@ -61,11 +60,11 @@ Get your KubeCon EU [tickets here](https://ti.to/kubecon/kubecon-eu-2016). [在这里](https://ti.to/kubecon/kubecon-eu-2016)获取您的 KubeCon EU 门票。 会场地址：CodeNode * 英国伦敦南广场 10 号 酒店住宿：[酒店](https://skillsmatter.com/contact-us) @@ -74,9 +73,9 @@ Google is a proud Diamond sponsor of KubeCon EU 2016. Come to London next month, 谷歌是 KubeCon EU 2016 的钻石赞助商。下个月 3 月 10 - 11 号来伦敦，参观 13 号展位，了解 Kubernetes，Google Container Engine（GKE），Google Cloud Platform 的所有信息! _KubeCon 是由 KubeAcademy、LLC 组织的，这是一个由社区驱动的开发者团体，专注于开发人员的教育和 kubernet.com 的推广 diff --git a/content/zh/blog/_posts/2016-02-00-Kubernetes-Community-Meeting-Notes.md b/content/zh/blog/_posts/2016-02-00-Kubernetes-Community-Meeting-Notes.md index 5d1037f747..608a5bbd33 100644 --- a/content/zh/blog/_posts/2016-02-00-Kubernetes-Community-Meeting-Notes.md +++ b/content/zh/blog/_posts/2016-02-00-Kubernetes-Community-Meeting-Notes.md @@ -2,7 +2,6 @@ title: " Kubernetes 社区会议记录 - 20160204 " date: 2016-02-09 slug: kubernetes-community-meeting-notes -url: /blog/2016/02/Kubernetes-Community-Meeting-Notes --- * 书记员：Rob Hirschfeld * 演示视频（20分钟）：CoreOS rkt + Kubernetes[Shaya Potter] - * 期待在未来几个月内看到与rkt和k8s的整合（“rkt-netes”）。 还没有集成到 v1.2版本中。 + * 期待在未来几个月内看到与rkt和k8s的整合（“rkt-netes”）。 还没有集成到 v1.2版本中。 * Shaya 做了一个演示（8分钟的会议视频参考） * rkt的CLI显示了旋转容器 * [注意：音频在点数上是乱码] @@ -46,11 +45,11 @@ Kubernetes 贡献社区在每周四 10:00 PT 开会,通过视频会议讨论项 * Dawn Chen: * 将 rkt 与 kubernetes 集成的其余问题：1）cadivsor 2） DNS 3）与日志记录相关的错误 * 但是需要在 e2e 测试套件上做更多的工作 - * 用例（10分钟）：在 OpenStack 上的 eBay k8s 和 k8s 上的 OpenStack [Ashwin Raveendran] @@ -90,7 +89,7 @@ Kubernetes 贡献社区在每周四 10:00 PT 开会,通过视频会议讨论项 * 我们希望在多个平台上进行测试的共识。 * 为测试报告提供一个全面转储会很有帮助 * 可以使用"phone-home"收集异常 - + 要参与 Kubernetes 社区，请考虑加入我们的[Slack 频道][2]，查看 GitHub上的 [Kubernetes 项目][3]，或加入[Kubernetes-dev Google 小组][4]。如果你真的很兴奋，你可以完成上述所有工作并加入我们的下一次社区对话-2016年2月11日。请将您自己或您想要了解的主题添加到[议程][5]并通过加入[此组][6]来获取日历邀请。 diff --git a/content/zh/blog/_posts/2016-07-00-Citrix-Netscaler-And-Kubernetes.md b/content/zh/blog/_posts/2016-07-00-Citrix-Netscaler-And-Kubernetes.md index b0a27044dc..45ab59e3af 100644 --- a/content/zh/blog/_posts/2016-07-00-Citrix-Netscaler-And-Kubernetes.md +++ b/content/zh/blog/_posts/2016-07-00-Citrix-Netscaler-And-Kubernetes.md @@ -2,7 +2,6 @@ title: " Citrix + Kubernetes = 全垒打 " date: 2016-07-14 slug: citrix-netscaler-and-kubernetes -url: /blog/2016/07/Citrix-Netscaler-And-Kubernetes --- -编者按：今天的客座文章来自 Citrix Systems 的产品管理总监 Mikko Disini，他分享了他们在 Kubernetes 集成上的合作经验。 _ +编者按：今天的客座文章来自 Citrix Systems 的产品管理总监 Mikko Disini，他分享了他们在 Kubernetes 集成上的合作经验。 _ 技术合作就像体育运动。如果你能像一个团队一样合作，你就能在最后关头取得胜利。这就是我们对谷歌云平台团队的经验。 -最近，我们与 Google 云平台（GCP）联系，代表 Citrix 客户以及更广泛的企业市场，希望就工作负载的迁移进行协作。此迁移需要将 [NetScaler Docker 负载均衡器]https://www.citrix.com/blogs/2016/06/20/the-best-docker-load-balancer-at-dockercon-in-seattle-this-week/) CPX 包含到 Kubernetes 节点中，并解决将流量引入 CPX 代理的任何问题。 +最近，我们与 Google 云平台（GCP）联系，代表 Citrix 客户以及更广泛的企业市场，希望就工作负载的迁移进行协作。此迁移需要将 [NetScaler Docker 负载均衡器]https://www.citrix.com/blogs/2016/06/20/the-best-docker-load-balancer-at-dockercon-in-seattle-this-week/) CPX 包含到 Kubernetes 节点中，并解决将流量引入 CPX 代理的任何问题。 **为什么是 NetScaler 和 Kubernetes** @@ -42,16 +41,16 @@ Recently, we approached Google Cloud Platform (GCP) to collaborate on behalf of --> 1. Citrix 的客户希望他们开始使用 Kubernetes 部署他们的容器和微服务体系结构时，能够像当初迁移到云计算时一样，享有 NetScaler 所提供的第 4 层到第 7 层能力  -2. Kubernetes 提供了一套经过验证的基础设施，可用来运行容器和虚拟机，并自动交付工作负载； +2. Kubernetes 提供了一套经过验证的基础设施，可用来运行容器和虚拟机，并自动交付工作负载； 3. NetScaler CPX 提供第 4 层到第 7 层的服务，并为日志和分析平台 [NetScaler 管理和分析系统](https://www.citrix.com/blogs/2016/05/24/introducing-the-next-generation-netscaler-management-and-analytics-system/) 提供高效的度量数据。 我希望我们所有与技术合作伙伴一起工作的经验都能像与 GCP 一起工作一样好。我们有一个列表，包含支持我们的用例所需要解决的问题。我们能够快速协作形成解决方案。为了解决这些问题，GCP 团队提供了深入的技术支持，与 Citrix 合作，从而使得 NetScaler CPX 能够在每台主机上作为客户端代理启动运行。 接下来，需要在 GCP 入口负载均衡器的数据路径中插入 NetScaler CPX，使 NetScaler CPX 能够将流量分散到前端 web 服务器。NetScaler 团队进行了修改，以便 NetScaler CPX 监听 API 服务器事件，并配置自己来创建 VIP、IP 表规则和服务器规则，以便跨前端应用程序接收流量和负载均衡。谷歌云平台团队提供反馈和帮助，验证为克服技术障碍所做的修改。完成了! @@ -61,7 +60,7 @@ NetScaler CPX use case is supported in [Kubernetes 1.3](https://kubernetes.io/bl NetScaler CPX 用例在 [Kubernetes 1.3](https://kubernetes.io/blog/2016/07/kubernetes-1-3-bridging-cloud-native-and-enterprise-workloads/) 中提供支持。Citrix 的客户和更广泛的企业市场将有机会基于 Kubernetes 享用 NetScaler 服务，从而降低将工作负载转移到云平台的阻力。  您可以在[此处](https://www.citrix.com/networking/microservices.html)了解有关 NetScaler CPX 的更多信息。 diff --git a/content/zh/blog/_posts/2016-07-00-Dashboard-Web-Interface-For-Kubernetes.md b/content/zh/blog/_posts/2016-07-00-Dashboard-Web-Interface-For-Kubernetes.md index 935cbcf325..dd0533def3 100644 --- a/content/zh/blog/_posts/2016-07-00-Dashboard-Web-Interface-For-Kubernetes.md +++ b/content/zh/blog/_posts/2016-07-00-Dashboard-Web-Interface-For-Kubernetes.md @@ -2,7 +2,6 @@ title: " Dashboard - Kubernetes 的全功能 Web 界面 " date: 2016-07-15 slug: dashboard-web-interface-for-kubernetes -url: /blog/2016/07/Dashboard-Web-Interface-For-Kubernetes --- _编者按：这篇文章是[一系列深入的文章](https://kubernetes.io/blog/2016/07/five-days-of-kubernetes-1-3) 中关于Kubernetes 1.3的新内容的一部分_ [Kubernetes Dashboard](http://github.com/kubernetes/dashboard)是一个旨在为 Kubernetes 世界带来通用监控和操作 Web 界面的项目。三个月前，我们[发布](https://kubernetes.io/blog/2016/04/building-awesome-user-interfaces-for-kubernetes)第一个面向生产的版本，从那时起 dashboard 已经做了大量的改进。在一个 UI 中，您可以在不离开浏览器的情况下，与 Kubernetes 集群执行大多数可能的交互。这篇博客文章分解了最新版本中引入的新功能，并概述了未来的路线图。 -**全功能的 Dashboard** +**全功能的 Dashboard** 由于社区和项目成员的大量贡献，我们能够为[Kubernetes 1.3发行版](https://kubernetes.io/blog/2016/07/kubernetes-1-3-bridging-cloud-native-and-enterprise-workloads/)提供许多新功能。我们一直在认真听取用户的反馈(参见[摘要信息图表](http://static.lwy.io/img/kubernetes_dashboard_infographic.png))，并解决了最高优先级的请求和难点。 --> -_编者按，今天的嘉宾帖子来自一位独立的 kubernetes 撰稿人 Justin Santa Barbara，分享了他对项目从一开始到未来发展的思考。_ +_编者按，今天的嘉宾帖子来自一位独立的 kubernetes 撰稿人 Justin Santa Barbara，分享了他对项目从一开始到未来发展的思考。_ -**亲爱的 K8s,** +**亲爱的 K8s,** -_很难相信你是唯一的一个 - 成长这么快的。在你一岁生日的时候，我想我可以写一个小纸条，告诉你为什么我在你出生的时候那么兴奋，为什么我觉得很幸运能成为抚养你长大的一员，为什么我渴望看到你继续成长！_ +_很难相信你是唯一的一个 - 成长这么快的。在你一岁生日的时候，我想我可以写一个小纸条，告诉你为什么我在你出生的时候那么兴奋，为什么我觉得很幸运能成为抚养你长大的一员，为什么我渴望看到你继续成长！_ -_--Justin_ +_--Justin_ 你从一个优秀的基础 - 良好的声明性功能开始，它是围绕一个具有良好定义的模式和机制的坚实的 API 构建的，这样我们就可以向前发展了。果然，在你的第一年里，你增长得如此之快：autoscaling、HTTP load-balancing support (Ingress)、support for persistent workloads including clustered databases (PetSets)。你已经和更多的云交了朋友(欢迎 azure 和 openstack 加入家庭)，甚至开始跨越区域和集群(Federation)。这些只是一些最明显的变化 - 在你的大脑里发生了太多的变化！ 我觉得你一直保持开放的态度真是太好了 - 你好像把所有的东西都写在 github 上 - 不管是好是坏。我想我们在这方面都学到了很多，比如让工程师做缩放声明的风险，然后在没有完全相同的精确性和严谨性框架的情况下，将这些声明与索赔进行权衡。但我很自豪你选择了不降低你的标准，而是上升到挑战，只是跑得更快 - 这可能不是最现实的办法，但这是唯一的方式能移动山！ 然而，不知何故，你已经设法避免了许多其他开源软件陷入的共同死胡同，特别是当那些项目越来越大，开发人员最终要做的比直接使用它更多的时候。你是怎么做到的？有一个很可能是虚构的故事，讲的是 IBM 的一名员工犯了一个巨大的错误，被传唤去见大老板，希望被解雇，却被告知“我们刚刚花了几百万美元培训你。我们为什么要解雇你？“。尽管谷歌对你进行了大量的投资(包括 redhat 和其他公司)，但我有时想知道，我们正在避免的错误是否更有价值。有一个非常开放的开发过程，但也有一个“oracle”，它有时会通过告诉我们两年后如果我们做一个特定的设计决策会发生什么来纠正错误。这是你应该听的父母！ 所以，尽管你只有一岁，你真的有一个[旧灵魂](http://queue.acm.org/detail.cfm?ID=2898444)。我只是[很多人抚养你](https://kubernetes.io/blog/2016/07/happy-k8sbday-1)中的一员，但对我来说，能够与那些建立了这些令人难以置信的系统并拥有所有这些领域知识的人一起工作是一次极好的学习经历。然而，因为我们是白手起家(而不是采用现有的 Borg 代码)，我们处于同一水平，仍然可以就如何培养你进行真正的讨论。好吧，至少和我们的水平一样接近，但值得称赞的是，他们都太好了，从来没提过！ @@ -67,9 +66,9 @@ If I would pick just two of the wise decisions those brilliant people made: - 控制器是状态同步器：我们指定目标，您的控制器将不遗余力地工作，使系统达到该状态。它们工作在强类型 API 基础上，并且贯穿整个代码，因此 Kubernetes 比一个大的程序多一百个小程序。仅仅从技术上扩展到数千个节点是不够的；这个项目还必须扩展到数千个开发人员和特性；控制器帮助我们达到目的。 等等我们就走！我们将取代那些控制器，建立更多，API 基金会让我们构建任何我们可以用这种方式表达的东西 - 大多数东西只是标签或注释远离！但你的思想不会由语言来定义：有了第三方资源，你可以表达任何你选择的东西。现在我们可以不用在 Kubernetes 建造Kubernetes 了，创造出与其他任何东西一样感觉是 Kubernetes 的一部分的东西。最近添加的许多功能，如ingress、DNS integration、autoscaling and network policies ，都已经完成或可以通过这种方式完成。最终，在这些事情发生之前很难想象你会是怎样的一个人，但是明天的标准功能可以从今天开始，没有任何障碍或看门人，甚至对一个听众来说也是这样。 @@ -77,13 +76,13 @@ So I’m looking forward to seeing more and more growth happen further and furth 所以我期待着看到越来越多的增长发生在离 Kubernetes 核心越来越远的地方。我们必须通过这些阶段来工作；从需要在 kubernetes 内核中发生的事情开始——比如用部署替换复制控制器。现在我们开始构建不需要核心更改的东西。但我们仍然在讨论基础设施和应用程序。接下来真正有趣的是：当我们开始构建依赖于 kubernetes api 的应用程序时。我们一直有使用 kubernetes api 进行自组装的 cassandra 示例，但我们还没有真正开始更广泛地探讨这个问题。正如 S3 APIs 改变了我们构建记忆事物的方式一样，我认为 k8s APIs 也将改变我们构建思考事物的方式。 所以我很期待你的二岁生日：我可以试着预测你那时的样子，但我知道你会超越我所能想象的最大胆的东西。哦，这是你要去的地方！ -_-- Justin Santa Barbara, 独立的 Kubernetes 贡献者_ +_-- Justin Santa Barbara, 独立的 Kubernetes 贡献者_ diff --git a/content/zh/blog/_posts/2017-10-00-Five-Days-Of-Kubernetes-18.md b/content/zh/blog/_posts/2017-10-00-Five-Days-Of-Kubernetes-18.md index 979d5cd931..eee16d7a49 100644 --- a/content/zh/blog/_posts/2017-10-00-Five-Days-Of-Kubernetes-18.md +++ b/content/zh/blog/_posts/2017-10-00-Five-Days-Of-Kubernetes-18.md @@ -2,7 +2,6 @@ title: " Kubernetes 1.8 的五天 " date: 2017-10-24 slug: five-days-of-kubernetes-18 -url: /blog/2017/10/Five-Days-Of-Kubernetes-18 --- Kubernetes 允许开发人员根据当前的流量和负载自动调整集群大小和 pod 副本的数量。这些调整减少了未使用节点的数量，节省了资金和资源。 在这次演讲中，谷歌的 Marcin Wielgus 将带领您了解 Kubernetes 中 pod 和 node 自动调焦的当前状态：它是如何工作的，以及如何使用它，包括在生产应用程序中部署的最佳实践。 喜欢这个演讲吗？ 12 月 6 日至 8 日，在 Austin 参加 KubeCon 关于扩展和自动化您的 Kubernetes 集群的更令人兴奋的会议。[现在注册](https://www.eventbrite.com/e/kubecon-cloudnativecon-north-america-registration-37824050754?_ga=2.9666039.317115486.1510003873-1623727562.1496428006)。 From e31e989756e357238eab96af46f3ad426c715bf0 Mon Sep 17 00:00:00 2001 From: Qiming Teng Date: Sun, 8 Mar 2020 11:37:34 +0800 Subject: [PATCH 003/140] [zh] fix object overview page (#19529) --- .../kubernetes-objects.md | 26 ++++++++++++++----- 1 file changed, 19 insertions(+), 7 deletions(-) diff --git a/content/zh/docs/concepts/overview/working-with-objects/kubernetes-objects.md b/content/zh/docs/concepts/overview/working-with-objects/kubernetes-objects.md index 0015553467..156fa61d56 100644 --- a/content/zh/docs/concepts/overview/working-with-objects/kubernetes-objects.md +++ b/content/zh/docs/concepts/overview/working-with-objects/kubernetes-objects.md @@ -1,12 +1,21 @@ --- title: 理解 Kubernetes 对象 - -redirect_from: -- "/docs/concepts/abstractions/overview/" -- "/docs/concepts/abstractions/overview.html" content_template: templates/concept +weight: 10 +card: + name: 概念 + weight: 40 --- + + {{% capture overview %}} -也需要提供对象的 `spec` 字段。对象 `spec` 的精确格式对每个 Kubernetes 对象来说是不同的，包含了特定于该对象的嵌套字段。[Kubernetes API 参考](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/)能够帮助我们找到任何我们想创建的对象的 spec 格式。 +您也需要提供对象的 `spec` 字段。对象 `spec` 的精确格式对每个 Kubernetes 对象来说是不同的，包含了特定于该对象的嵌套字段。[Kubernetes API 参考](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/)能够帮助我们找到任何我们想创建的对象的 spec 格式。 例如，可以从 [这里](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#podspec-v1-core) 查看 `Pod` 的 `spec` 格式， @@ -150,10 +159,13 @@ and the `spec` format for a `Deployment` can be found {{% capture whatsnext %}} - -* 了解最重要的基本 Kubernetes 对象，例如 [Pod](/docs/concepts/workloads/pods/pod-overview/)。 +* [Kubernetes API 概述](/docs/reference/using-api/api-overview/) 提供关于 API 概念的进一步阐述 +* 了解最重要的 Kubernetes 基本对象，例如 [Pod](/docs/concepts/workloads/pods/pod-overview/)。 +* 了解 Kubernetes 中的[控制器](/docs/concepts/architecture/controller/)。 {{% /capture %}} From 44acf0428ab65c3ccad503f1c64c5f5fb89da52b Mon Sep 17 00:00:00 2001 From: megaredfan Date: Sun, 8 Mar 2020 20:27:34 +0800 Subject: [PATCH 004/140] improve understandability (#19544) --- content/zh/docs/concepts/architecture/controller.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/zh/docs/concepts/architecture/controller.md b/content/zh/docs/concepts/architecture/controller.md index a28f2a577d..ecbd1f3726 100644 --- a/content/zh/docs/concepts/architecture/controller.md +++ b/content/zh/docs/concepts/architecture/controller.md @@ -51,7 +51,7 @@ detail. --> ## 控制器模式 {#controller-pattern} -一个控制器至少追踪一种类型的 Kubernetes 资源。这些[对象](/docs/concepts/overview/working-with-objects/kubernetes-objects/)有一个代表期望状态的指定字段。正对这种资源的控制器就是要使他的当前状态接近与期望状态。 +一个控制器至少追踪一种类型的 Kubernetes 资源。这些[对象](/docs/concepts/overview/working-with-objects/kubernetes-objects/)有一个代表期望状态的指定字段。控制器负责确保其追踪的资源对象的当前状态接近期望状态。 控制器可能会自行执行操作；在 Kubernetes 中更常见的是一个控制器会发送信息给 {{< glossary_tooltip text="API 服务器" term_id="kube-apiserver" >}}，这会有副作用。看下面这个例子。 From 5f9f8ea6fcbef4dda9920464e251dc3965996fbc Mon Sep 17 00:00:00 2001 From: 2BFL Date: Sun, 8 Mar 2020 23:03:34 +0800 Subject: [PATCH 005/140] update zh-translation docs/concepts/extend-kubernetes/operator.md (#19533) Accept sftim's suggestion Accept xichengliudui's suggestion Accept tengqm's suggestion --- .../concepts/extend-kubernetes/operator.md | 232 ++++++++++++++++++ 1 file changed, 232 insertions(+) create mode 100644 content/zh/docs/concepts/extend-kubernetes/operator.md diff --git a/content/zh/docs/concepts/extend-kubernetes/operator.md b/content/zh/docs/concepts/extend-kubernetes/operator.md new file mode 100644 index 0000000000..da14235dff --- /dev/null +++ b/content/zh/docs/concepts/extend-kubernetes/operator.md @@ -0,0 +1,232 @@ +--- +title: Operator 模式 +content_template: templates/concept +weight: 30 +--- + + + +{{% capture overview %}} + + + +Operator 是 Kubernetes 的扩展软件，它利用[自定义资源](/docs/concepts/extend-kubernetes/api-extension/custom-resources/)管理应用及其组件。 +Operator 遵循 Kubernetes 的理念，特别是在[控制环](/docs/concepts/#kubernetes-control-plane)方面。 + +{{% /capture %}} + +{{% capture body %}} + + + +## 初衷 + +Operator 模式旨在捕获（正在管理一个或一组服务的）运维人员的关键目标。 +负责特定应用和 service 的运维人员，在系统应该如何运行、如何部署以及出现问题时如何处理等方面有深入的了解。 + +在 Kubernetes 上运行工作负载的人们都喜欢通过自动化来处理重复的任务。Operator 模式会封装您编写的（Kubernetes 本身提供功能以外的）任务自动化代码。 + + + +## Kubernetes 上的 Operator + +Kubernetes 为自动化而生。无需任何修改，您即可以从 Kubernetes 核心中获得许多内置的自动化功能。 +您可以使用 Kubernetes 自动化部署和运行工作负载， *甚至* 可以自动化 Kubernetes 自身。 + +Kubernetes {{< glossary_tooltip text="控制器" term_id="controller" >}} 使您无需修改 Kubernetes 自身的代码，即可以扩展集群的行为。 +Operator 是 Kubernetes API 的客户端，充当[自定义资源](/docs/concepts/api-extension/custom-resources/)的控制器。 + + + +## Operator 示例 {#example} + +使用 Operator 可以自动化的事情包括： + +* 按需部署应用 +* 获取/还原应用状态的备份 +* 处理应用代码的升级以及相关改动。例如，数据库 schema 或额外的配置设置 +* 发布一个 service，要求不支持 Kubernetes API 的应用也能发现它 +* 模拟整个或部分集群中的故障以测试其稳定性 +* 在没有内部成员选举程序的情况下，为分布式应用选择首领角色 + + + +想要更详细的了解 Operator？这儿有一个详细的示例： + +1. 有一个名为 SampleDB 的自定义资源，您可以将其配置到集群中。 +2. 一个包含 Operator 控制器部分的 Deployment，用来确保 Pod 处于运行状态。 +3. Operator 代码的容器镜像。 +4. 控制器代码，负责查询控制平面以找出已配置的 SampleDB 资源。 +5. Operator 的核心是告诉 API 服务器，如何使现实与代码里配置的资源匹配。 + * 如果添加新的 SampleDB，Operator 将设置 PersistentVolumeClaims 以提供持久化的数据库存储，设置 StatefulSet 以运行 SampleDB，并设置 Job 来处理初始配置。 + * 如果您删除它，Operator 将建立快照，然后确保 StatefulSet 和 Volume 已被删除。 +6. Operator 也可以管理常规数据库的备份。对于每个 SampleDB 资源，Operator 会确定何时创建（可以连接到数据库并进行备份的）Pod。这些 Pod 将依赖于 ConfigMap 和/或 具有数据库连接详细信息和凭据的 Secret。 +7. 由于 Operator 旨在为其管理的资源提供强大的自动化功能，因此它还需要一些额外的支持性代码。在这个示例中，代码将检查数据库是否正运行在旧版本上，如果是，则创建 Job 对象为您升级数据库。 + + + +## 部署 Operator + +部署 Operator 最常见的方法是将自定义资源及其关联的控制器添加到您的集群中。跟运行容器化应用一样，Controller 通常会运行在 {{< glossary_tooltip text="控制平面" term_id="control-plane" >}} 之外。例如，您可以在集群中将控制器作为 Deployment 运行。 + + + +## 使用 Operator {#using-operators} + +部署 Operator 后，您可以对 Operator 所使用的资源执行添加、修改或删除操作。按照上面的示例，您将为 Operator 本身建立一个 Deployment，然后： + +```shell +kubectl get SampleDB # 查找所配置的数据库 + +kubectl edit SampleDB/example-database # 手动修改某些配置 +``` + + + +可以了！Operator 会负责应用所作的更改并保持现有服务处于良好的状态 + +## 编写你自己的 Operator {#writing-operator} + + + +如果生态系统中没可以实现您目标的 Operator，您可以自己编写代码。在[接下来](#what-s-next)一节中，您会找到编写自己的云原生 Operator 需要的库和工具的链接。 + +您还可以使用任何支持 [Kubernetes API 客户端](/docs/reference/using-api/client-libraries/)的语言或运行时来实现 Operator（即控制器）。 + +{{% /capture %}} + +{{% capture whatsnext %}} + + + +* 详细了解[自定义资源](/docs/concepts/extend-kubernetes/api-extension/custom-resources/) +* 在 [OperatorHub.io](https://operatorhub.io/) 上找到现成的、适合您的 Operator +* 借助已有的工具来编写您自己的 Operator，例如： + * [KUDO](https://kudo.dev/) (Kubernetes 通用声明式 Operator) + * [kubebuilder](https://book.kubebuilder.io/) + * [Metacontroller](https://metacontroller.app/)，可与 Webhook 结合使用，以实现自己的功能。 + * [Operator 框架](https://github.com/operator-framework/getting-started) +* [发布](https://operatorhub.io/)您的 Operator，让别人也可以使用 +* 阅读 [CoreOS 原文](https://coreos.com/blog/introducing-operators.html)，其介绍了 Operator 介绍 +* 阅读这篇来自谷歌云的关于构建 Operator 最佳实践的[文章](https://cloud.google.com/blog/products/containers-kubernetes/best-practices-for-building-kubernetes-operators-and-stateful-apps) + +{{% /capture %}} From e7a84f9bc2be4e245e6b06d06c5fdb9da187c485 Mon Sep 17 00:00:00 2001 From: divya-mohan0209 Date: Sun, 8 Mar 2020 20:47:35 +0530 Subject: [PATCH 006/140] Updated service.md (#19539) --- .../en/docs/concepts/services-networking/service.md | 13 ------------- 1 file changed, 13 deletions(-) diff --git a/content/en/docs/concepts/services-networking/service.md b/content/en/docs/concepts/services-networking/service.md index fbecdb3316..4e8326b4ae 100644 --- a/content/en/docs/concepts/services-networking/service.md +++ b/content/en/docs/concepts/services-networking/service.md @@ -1178,19 +1178,6 @@ SCTP is not supported on Windows based nodes. The kube-proxy does not support the management of SCTP associations when it is in userspace mode. {{< /warning >}} -## Future work - -In the future, the proxy policy for Services can become more nuanced than -simple round-robin balancing, for example master-elected or sharded. We also -envision that some Services will have "real" load balancers, in which case the -virtual IP address will simply transport the packets there. - -The Kubernetes project intends to improve support for L7 (HTTP) Services. - -The Kubernetes project intends to have more flexible ingress modes for Services -that encompass the current ClusterIP, NodePort, and LoadBalancer modes and more. - - {{% /capture %}} {{% capture whatsnext %}} From ead3e36150870bb9826ff6dcd17f53d266f7f173 Mon Sep 17 00:00:00 2001 From: Sujay Pillai Date: Mon, 9 Mar 2020 01:23:34 +0800 Subject: [PATCH 007/140] =?UTF-8?q?Fix=20branch=20names=20in=20=E2=80=9CSt?= =?UTF-8?q?art=20Contributing=E2=80=9D=20(#19493)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Replaced shortcodes with hard-coded values 1.17 * Updated with shortcode as per PR reveiew * Update with proper shortcode after rebase --- content/en/docs/contribute/start.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/content/en/docs/contribute/start.md b/content/en/docs/contribute/start.md index acd5a5bfdf..181e359682 100644 --- a/content/en/docs/contribute/start.md +++ b/content/en/docs/contribute/start.md @@ -209,7 +209,7 @@ to base your work on. Use these guidelines to make the decision: - Some localization teams work with a series of long-lived branches, and periodically merge these to `master`. This kind of branch has a name like dev-\-\.\; for example: - `dev-{{< release-branch >}}-ja.1`. + `dev-{{< latest-semver >}}-ja.1` - If you're writing or updating documentation for a feature change release, then you need to know the major and minor version of Kubernetes that the change will first appear in. @@ -217,8 +217,8 @@ to base your work on. Use these guidelines to make the decision: to beta in the next minor version, you need to know what the next minor version number is. - Find the release branch named for that version. For example, features that - changed in the v{{< release-branch >}} release got documented in the branch - named `dev-{{< release-branch >}}`. + changed in the {{< latest-version >}} release got documented in the branch + named `dev-{{< latest-semver >}}`. If you're still not sure which branch to choose, ask in `#sig-docs` on Slack or attend a weekly SIG Docs meeting to get clarity. From 4144a95da38d751d05779c929d29548f68995b07 Mon Sep 17 00:00:00 2001 From: jCOTINEAU Date: Sun, 8 Mar 2020 19:11:34 +0100 Subject: [PATCH 008/140] Update start.md, fix wrong traduction (#19313) fix wrong traduction, should be "demandes" instead of "demander" --- content/fr/docs/contribute/start.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/fr/docs/contribute/start.md b/content/fr/docs/contribute/start.md index 59cb54cfc2..39eee2a39d 100644 --- a/content/fr/docs/contribute/start.md +++ b/content/fr/docs/contribute/start.md @@ -70,7 +70,7 @@ Pour plus d'informations sur la contribution à la documentation dans plusieurs Si vous souhaitez démarrer une nouvelle traduction, voir ["Traduction"](/docs/contribute/localization/). -## Créer des demander recevables +## Créer des demandes recevables Toute personne possédant un compte GitHub peut soumettre un problème (rapport de bogue) à la documentation de Kubernetes. Si vous voyez quelque chose qui ne va pas, même si vous ne savez pas comment le réparer, [ouvrez un ticket](#how-to-file-an-issue). From ecc7f31964a17c09a3b5514a9d57f48ea272c88c Mon Sep 17 00:00:00 2001 From: Jhon Mike Date: Sun, 8 Mar 2020 18:25:34 -0300 Subject: [PATCH 009/140] Translate content/pt/docs/concepts/cluster-administration/kubelet-garbage-collection.md into Portugues (#19434) * translating doc kubelet garbage collection * code review corrections in the garbage collection doc * Update content/pt/docs/concepts/cluster-administration/kubelet-garbage-collection.md Update content/pt/docs/concepts/cluster-administration/kubelet-garbage-collection.md Co-Authored-By: Diego W. Antunes Update content/pt/docs/concepts/cluster-administration/kubelet-garbage-collection.md Co-Authored-By: Diego W. Antunes Update content/pt/docs/concepts/cluster-administration/kubelet-garbage-collection.md Co-Authored-By: Diego W. Antunes Update content/pt/docs/concepts/cluster-administration/kubelet-garbage-collection.md Co-Authored-By: Diego W. Antunes Update content/pt/docs/concepts/cluster-administration/kubelet-garbage-collection.md Update content/pt/docs/concepts/cluster-administration/kubelet-garbage-collection.md Co-Authored-By: Diego W. Antunes Update content/pt/docs/concepts/cluster-administration/kubelet-garbage-collection.md Co-Authored-By: Diego W. Antunes Co-authored-by: Diego W. Antunes --- .../kubelet-garbage-collection.md | 71 +++++++++++++++++++ 1 file changed, 71 insertions(+) create mode 100644 content/pt/docs/concepts/cluster-administration/kubelet-garbage-collection.md diff --git a/content/pt/docs/concepts/cluster-administration/kubelet-garbage-collection.md b/content/pt/docs/concepts/cluster-administration/kubelet-garbage-collection.md new file mode 100644 index 0000000000..78270eedcc --- /dev/null +++ b/content/pt/docs/concepts/cluster-administration/kubelet-garbage-collection.md @@ -0,0 +1,71 @@ +--- +reviewers: +title: Configurando o Garbage Collection do kubelet +content_template: templates/concept +weight: 70 +--- + +{{% capture overview %}} + +O Garbage collection(Coleta de lixo) é uma função útil do kubelet que limpa imagens e contêineres não utilizados. O kubelet executará o garbage collection para contêineres a cada minuto e para imagens a cada cinco minutos. + +Ferramentas externas de garbage collection não são recomendadas, pois podem potencialmente interromper o comportamento do kubelet removendo os contêineres que existem. + +{{% /capture %}} + +{{% capture body %}} + +## Coleta de imagens + +O Kubernetes gerencia o ciclo de vida de todas as imagens através do imageManager, com a cooperação do cadvisor. + +A política para o garbage collection de imagens leva dois fatores em consideração: +`HighThresholdPercent` e `LowThresholdPercent`. Uso do disco acima do limite acionará o garbage collection. O garbage collection excluirá as imagens que foram menos usadas recentemente até que o nível fique abaixo do limite. + +## Coleta de container + +A política para o garbage collection de contêineres considera três variáveis definidas pelo usuário. `MinAge` é a idade mínima em que um contêiner pode ser coletado. `MaxPerPodContainer` é o número máximo de contêineres mortos que todo par de pod (UID, container name) pode ter. `MaxContainers` é o número máximo de contêineres mortos totais. Essas variáveis podem ser desabilitadas individualmente, definindo `MinAge` como zero e definindo `MaxPerPodContainer` e `MaxContainers` respectivamente para menor que zero. + +O Kubelet atuará em contêineres não identificados, excluídos ou fora dos limites definidos pelos sinalizadores mencionados. Os contêineres mais antigos geralmente serão removidos primeiro. `MaxPerPodContainer` e `MaxContainer` podem potencialmente conflitar entre si em situações em que a retenção do número máximo de contêineres por pod (`MaxPerPodContainer`) estaria fora do intervalo permitido de contêineres globais mortos (`MaxContainers`). O `MaxPerPodContainer` seria ajustado nesta situação: O pior cenário seria fazer o downgrade do `MaxPerPodContainer` para 1 e remover os contêineres mais antigos. Além disso, os contêineres pertencentes a pods que foram excluídos são removidos assim que se tornem mais antigos que `MinAge`. + +Os contêineres que não são gerenciados pelo kubelet não estão sujeitos ao garbage collection de contêiner. + +## Configurações do usuário + +Os usuários podem ajustar os seguintes limites para ajustar o garbage collection da imagem com os seguintes sinalizadores do kubelet: + +1. `image-gh-high-threshold`, a porcentagem de uso de disco que aciona o garbage collection da imagem. O padrão é 85%. +2. `image-gc-low-threshold`, a porcentagem de uso de disco com o qual o garbage collection da imagem tenta liberar. O padrão é 80%. + +Também permitimos que os usuários personalizem a política do garbagem collection através dos seguintes sinalizadores do kubelet: + +1. `minimum-container-ttl-duration`, idade mínima para um contêiner finalizado antes de ser colectado. O padrão é 0 minuto, o que significa que todo contêiner finalizado será coletado como lixo. +2. `maximum-dead-containers-per-container`, número máximo de instâncias antigas a serem retidas por contêiner. O padrão é 1. +3. `maximum-dead-containers`, número máximo de instâncias antigas de contêineres para retenção global. O padrão é -1, o que significa que não há limite global. + +Os contêineres podem ser potencialmente coletados como lixo antes que sua utilidade expire. Esses contêineres podem conter logs e outros dados que podem ser úteis para solucionar problemas. Um valor suficientemente grande para `maximum-dead-containers-per-container` é altamente recomendado para permitir que pelo menos 1 contêiner morto seja retido por contêiner esperado. Um valor maior para `maximum-dead-containers` também é recomendados por um motivo semelhante. +Consulte [esta issue](https://github.com/kubernetes/kubernetes/issues/13287) para obter mais detalhes. + +## Descontinuado + +Alguns recursos do Garbage Collection neste documento serão substituídos pelo kubelet eviction no futuro. + +Incluindo: + +| Flag Existente | Nova Flag | Fundamentação | +| ----------------------------------------- | --------------------------------------- | ------------------------------------------------------------------------------------ | +| `--image-gc-high-threshold` | `--eviction-hard` ou `--eviction-soft` | os sinais existentes de despejo podem acionar o garbage collection da imagem | +| `--image-gc-low-threshold` | `--eviction-minimum-reclaim` | recuperações de despejo atinge o mesmo comportamento | +| `--maximum-dead-containers` | | descontinuado quando os logs antigos forem armazenados fora do contexto do contêiner | +| `--maximum-dead-containers-per-container` | | descontinuado quando os logs antigos forem armazenados fora do contexto do contêiner | +| `--minimum-container-ttl-duration` | | descontinuado quando os logs antigos forem armazenados fora do contexto do contêiner | +| `--low-diskspace-threshold-mb` | `--eviction-hard` ou `eviction-soft` | O despejo generaliza os limites do disco para outros recursos | +| `--outofdisk-transition-frequency` | `--eviction-pressure-transition-period` | O despejo generaliza a transição da pressão do disco para outros recursos | + +{{% /capture %}} + +{{% capture whatsnext %}} + +Consulte [Configurando a Manipulação de Recursos Insuficientes](/docs/tasks/administer-cluster/out-of-resource/) para mais detalhes. + +{{% /capture %}} From 978a55ea7b628cc4a0f51b33df29730bb48c9687 Mon Sep 17 00:00:00 2001 From: Yong Zhang Date: Mon, 9 Mar 2020 07:03:34 +0800 Subject: [PATCH 010/140] =?UTF-8?q?Localize=20=E2=80=9Cemail=20address?= =?UTF-8?q?=E2=80=9D=20placeholder=20on=20home=20page=20(#19412)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- i18n/fr.toml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/i18n/fr.toml b/i18n/fr.toml index 2cf29aafda..13d2bade6e 100644 --- a/i18n/fr.toml +++ b/i18n/fr.toml @@ -134,3 +134,6 @@ other = "Calendrier" # UI elements [ui_search_placeholder] other = "Recherche" + +[input_placeholder_email_address] +other = "adresse email" \ No newline at end of file From 0d3905efeb352aa905d1688bbaeb657b828cbc70 Mon Sep 17 00:00:00 2001 From: Yong Zhang Date: Mon, 9 Mar 2020 07:05:34 +0800 Subject: [PATCH 011/140] =?UTF-8?q?Localize=20=E2=80=9Cemail=20address?= =?UTF-8?q?=E2=80=9D=20placeholder=20on=20home=20page=20(#19414)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- i18n/id.toml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/i18n/id.toml b/i18n/id.toml index 8406981b78..aa196a6758 100644 --- a/i18n/id.toml +++ b/i18n/id.toml @@ -137,3 +137,6 @@ other = "Kalender acara" # UI elements [ui_search_placeholder] other = "Cari" + +[input_placeholder_email_address] +other = "Alamat email" From d29ede7f47d3209bd1162c729f8d6d28e5f2d6f2 Mon Sep 17 00:00:00 2001 From: RA489 Date: Mon, 9 Mar 2020 08:37:34 +0530 Subject: [PATCH 012/140] Document minikube Podman support (#19514) --- content/en/docs/tasks/tools/install-minikube.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/content/en/docs/tasks/tools/install-minikube.md b/content/en/docs/tasks/tools/install-minikube.md index 03c3b07cd5..b106c23ef6 100644 --- a/content/en/docs/tasks/tools/install-minikube.md +++ b/content/en/docs/tasks/tools/install-minikube.md @@ -86,6 +86,12 @@ The `none` VM driver can result in security and data loss issues. Before using `--vm-driver=none`, consult [this documentation](https://minikube.sigs.k8s.io/docs/reference/drivers/none/) for more information. {{< /caution >}} +Minikube also supports a `vm-driver=podman` similar to the Docker driver. Podman run as superuser privilege (root user) is the best way to ensure that your containers have full access to any feature available on your system. + +{{< caution >}} +The `podman` driver requires running the containers as root because regular user accounts don’t have full access to all operating system features that their containers might need to run. +{{< /caution >}} + ### Install Minikube using a package There are *experimental* packages for Minikube available; you can find Linux (AMD64) packages From c2bae8c6e4e072dca06faa480c6844df5901a484 Mon Sep 17 00:00:00 2001 From: KimJiWon Date: Mon, 9 Mar 2020 14:03:34 +0900 Subject: [PATCH 013/140] Fix pod.md (ko) (#19187) delete incorrectly duplicated phrases. --- content/ko/docs/concepts/workloads/pods/pod.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/ko/docs/concepts/workloads/pods/pod.md b/content/ko/docs/concepts/workloads/pods/pod.md index 16f2e4149b..b3a070af28 100644 --- a/content/ko/docs/concepts/workloads/pods/pod.md +++ b/content/ko/docs/concepts/workloads/pods/pod.md @@ -189,7 +189,7 @@ API에서 파드를 즉시 제거하므로 동일한 이름으로 새 파드를 Kubernetes v1.1부터, 파드의 모든 컨테이너는 컨테이너 스펙의 `SecurityContext`의 `privileged` 플래그를 사용하여 특권 모드를 사용할 수 있다. 이것은 네트워크 스택을 조작하고 장치에 액세스하는 것과 같은 Linux 기능을 사용하려는 컨테이너에 유용하다. 컨테이너 내의 프로세스는 컨테이너 외부의 프로세스에서 사용할 수 있는 거의 동일한 권한을 갖는다. 특권 모드를 사용하면 네트워크 및 볼륨 플러그인을 kubelet에 컴파일 할 필요가 없는 별도의 파드로 쉽게 만들 수 있다. 마스터가 Kubernetes v1.1 이상에서 실행 중이고, 노드가 v1.1 보다 낮은 버전을 실행중인 경우 새 권한이 부여 된 파드는 api-server에 의해 승인되지만 시작되지는 않는다. 이것들은 pending 상태가 될 것이다. -사용자가 `kubectl describe pod FooPodName` 을 호출하면 사용자는 파드가 사용자가 `kubectl describe pod FooPodName` 을 호출하면 사용자는 파드가 pending 상태에 있는 이유를 볼 수 있다. describe 명령 출력의 이벤트 테이블은 다음과 같다. +사용자가 `kubectl describe pod FooPodName` 을 호출하면 사용자는 파드가 pending 상태에 있는 이유를 볼 수 있다. describe 명령 출력의 이벤트 테이블은 다음과 같다. `Error validating pod "FooPodName"."FooPodNamespace" from api, ignoring: spec.containers[0].securityContext.privileged: forbidden '<*>(0xc2089d3248)true'` 마스터가 v1.1보다 낮은 버전에서 실행중인 경우 특권을 갖는 파드를 만들 수 없다. 유저가 특권을 갖는 컨테이너가 있는 파드를 만들려고 하면 다음과 같은 오류가 발생한다. From 16fedade58b270fea7cdb49decde4019232e275c Mon Sep 17 00:00:00 2001 From: Cria Hu Date: Mon, 9 Mar 2020 14:49:34 +0800 Subject: [PATCH 014/140] Modify sentences with poor translation (#19562) --- content/zh/docs/tutorials/services/source-ip.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/zh/docs/tutorials/services/source-ip.md b/content/zh/docs/tutorials/services/source-ip.md index f4369268ce..d8811a57ce 100644 --- a/content/zh/docs/tutorials/services/source-ip.md +++ b/content/zh/docs/tutorials/services/source-ip.md @@ -278,7 +278,7 @@ client_address=10.240.0.5 ``` -然而，如果你的集群运行在 Google Kubernetes Engine/GCE 上，设置 `service.spec.externalTrafficPolicy` 字段值为 `Local` 可以强制使*没有* endpoints 的节点把他们自己从负载均衡流量的可选节点名单中删除。这是通过故意使它们健康检查失败达到的。 +然而，如果你的集群运行在 Google Kubernetes Engine/GCE 上，可以通过设置 service.spec.externalTrafficPolicy 字段值为 Local ，故意导致健康检查失败来强制使没有 endpoints 的节点把自己从负载均衡流量的可选节点列表中删除。 用图表示： From 9e2c426ac4a065ed1da0a270208e9352dfc1bb72 Mon Sep 17 00:00:00 2001 From: gorquan Date: Mon, 9 Mar 2020 15:03:34 +0800 Subject: [PATCH 015/140] Update setup-ha-etcd-with-kubeadm.md Content (#19559) Update docs/setup/production-environment/tools/kubeadm/setup-ha-etcd-with-kubeadm.md content and fix create CA file command --- .../kubeadm/setup-ha-etcd-with-kubeadm.md | 39 ++++++++++++++----- 1 file changed, 30 insertions(+), 9 deletions(-) diff --git a/content/zh/docs/setup/production-environment/tools/kubeadm/setup-ha-etcd-with-kubeadm.md b/content/zh/docs/setup/production-environment/tools/kubeadm/setup-ha-etcd-with-kubeadm.md index c8d4fe2da3..a3e4420e35 100644 --- a/content/zh/docs/setup/production-environment/tools/kubeadm/setup-ha-etcd-with-kubeadm.md +++ b/content/zh/docs/setup/production-environment/tools/kubeadm/setup-ha-etcd-with-kubeadm.md @@ -1,7 +1,7 @@ --- title: 使用 kubeadm 创建一个高可用 etcd 集群 content_template: templates/task -weight: 60 +weight: 70 --- {{% capture overview %}} +{{< note >}} + +在本指南中，当 kubeadm 用作为外部 etcd 节点管理工具，请注意 kubeadm 不计划支持此类节点的证书更换或升级。对于长期规划是使用 [etcdadm](https://github.com/kubernetes-sigs/etcdadm) 增强工具来管理这方面。 +{{< /note >}} + + - 将 `${ETCD_TAG}` 设置为你的 etcd 镜像的版本标签，例如 `3.4.3-0`。要查看 kubeadm 使用的 etcd 镜像和标签，请执行 `kubeadm config images list --kubernetes-version ${K8S_VERSION}`，其中 `${K8S_VERSION}` 是 `v1.17.0` 作为例子。 + + - 将 `${HOST0}` 设置为要测试的主机的 IP 地址 {{% /capture %}} From 853090125a843cb10f97714cce01773be3782d5b Mon Sep 17 00:00:00 2001 From: 2BFL Date: Mon, 9 Mar 2020 15:07:35 +0800 Subject: [PATCH 016/140] zh-translation: configure-runasusername.md (#19547) Accept tengqm's suggestion --- .../configure-runasusername.md | 234 ++++++++++++++++++ 1 file changed, 234 insertions(+) create mode 100644 content/zh/docs/tasks/configure-pod-container/configure-runasusername.md diff --git a/content/zh/docs/tasks/configure-pod-container/configure-runasusername.md b/content/zh/docs/tasks/configure-pod-container/configure-runasusername.md new file mode 100644 index 0000000000..65a9e88cb5 --- /dev/null +++ b/content/zh/docs/tasks/configure-pod-container/configure-runasusername.md @@ -0,0 +1,234 @@ +--- +title: 为 Windows 的 pod 和容器配置 RunAsUserName +content_template: templates/task +weight: 20 +--- + + + +{{% capture overview %}} + +{{< feature-state for_k8s_version="v1.17" state="beta" >}} + + + +本页展示如何为运行在 Windows 节点上的 pod 和容器启用并使用 `RunAsUserName` 功能。此功能旨在成为 Windows 版的 `runAsUser`（Linux），允许用户使用与默认用户名不同的用户名运行容器 entrypoint。 + +{{< note >}} + +该功能目前处于 beta 状态。 `RunAsUserName` 的整体功能不会出现变更，但是关于用户名验证的部分可能会有所更改。 +{{< /note >}} + +{{% /capture %}} + +{{% capture prerequisites %}} + + + +你必须有一个 Kubernetes 集群，并且 kubectl 必须能和集群通信。集群应该要有 Windows 工作节点，将在其中调度运行 Windows 工作负载的 pod 和容器。 + + + +## 为 Pod 设置 Username + +要指定运行 Pod 容器时所使用的用户名，请在 Pod 声明中包含 `securityContext` （[PodSecurityContext](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#podsecuritycontext-v1-core)）字段，并在其内部包含 `windowsOptions` （[WindowsSecurityContextOptions](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#windowssecuritycontextoptions-v1-core)）字段的 `runAsUserName` 字段。 + + + +您为 Pod 指定的 Windows SecurityContext 选项适用于该 Pod 中（包括 init 容器）的所有容器。 + +这儿有一个已经设置了 `runAsUserName` 字段的 Windows Pod 的配置文件： + +{{< codenew file="windows/run-as-username-pod.yaml" >}} + + + +创建 Pod： + +```shell +kubectl apply -f https://k8s.io/examples/windows/run-as-username-pod.yaml +``` + + + +验证 Pod 容器是否在运行： + +```shell +kubectl get pod run-as-username-pod-demo +``` + + + +获取该容器的 shell： + +```shell +kubectl exec -it run-as-username-pod-demo -- powershell +``` + + + +检查运行 shell 的用户的用户名是否正确： + +```powershell +echo $env:USERNAME +``` + + + +输出结果应该是这样： + +```shell +ContainerUser +``` + + + +## 为容器设置 Username + +要指定运行容器时所使用的用户名，请在容器清单中包含 `securityContext` （[SecurityContext](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#securitycontext-v1-core)）字段，并在其内部包含 `windowsOptions` （[WindowsSecurityContextOptions](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#windowssecuritycontextoptions-v1-core)）字段的 `runAsUserName` 字段。 + + + +您为容器指定的 Windows SecurityContext 选项仅适用于该容器，并且它会覆盖 Pod 级别设置。 + +这儿有一个 Pod 的配置文件，其只有一个容器，并且在 Pod 级别和容器级别都设置了 `runAsUserName`： + +{{< codenew file="windows/run-as-username-container.yaml" >}} + + + +创建 Pod： + +```shell +kubectl apply -f https://k8s.io/examples/windows/run-as-username-container.yaml +``` + + + +验证 Pod 容器是否在运行： + +```shell +kubectl get pod run-as-username-container-demo +``` + + + +获取该容器的 shell： + +```shell +kubectl exec -it run-as-username-container-demo -- powershell +``` + + + +检查运行 shell 的用户的用户名是否正确（应该是容器级别设置的那个）： + +```powershell +echo $env:USERNAME +``` + + + +输出结果应该是这样： + +```shell +ContainerAdministrator +``` + + + +## Windows Username 的局限性 + +想要使用此功能，在 `runAsUserName` 字段中设置的值必须是有效的用户名。它必须是 `DOMAIN\USER` 这种格式，其中 `DOMAIN\` 是可选的。Windows 用户名不区分大小写。此外，关于 `DOMAIN` 和 `USER` 还有一些限制： + + + +- `runAsUserName` 字段不能为空，并且不能包含控制字符（ASCII 值：`0x00-0x1F`、`0x7F`） +- `DOMAIN` 必须是 NetBios 名称或 DNS 名称，每种名称都有各自的局限性： + - NetBios 名称：最多 15 个字符，不能以 `.`（点）开头，并且不能包含以下字符：`\ / : * ? " < > |` + - DNS 名称：最多 255 个字符，只能包含字母、数字、点和中划线，并且不能以 `.`（点）或 `-`（中划线）开头和结尾。 +- `USER` 最多不超过 20 个字符，不能 **只** 包含点或空格，并且不能包含以下字符：`" / \ [ ] : ; | = , + * ? < > @` + + + +`runAsUserName` 字段接受的值的一些示例：`ContainerAdministrator`、`ContainerUser`、`NT AUTHORITY\NETWORK SERVICE`、`NT AUTHORITY\LOCAL SERVICE`。 + +关于这些限制的更多信息，可以查看[这里](https://support.microsoft.com/en-us/help/909264/naming-conventions-in-active-directory-for-computers-domains-sites-and)和[这里](https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.localaccounts/new-localuser?view=powershell-5.1)。 + +{{% /capture %}} + +{{% capture whatsnext %}} + + + +* [Kubernetes 中调度 Windows 容器的指南](/docs/setup/production-environment/windows/user-guide-windows-containers/) +* [使用组托管服务帐户（GMSA）管理工作负载身份](/docs/setup/production-environment/windows/user-guide-windows-containers/#managing-workload-identity-with-group-managed-service-accounts) +* [Windows 下 pod 和容器的 GMSA 配置](/docs/tasks/configure-pod-container/configure-gmsa/) + +{{% /capture %}} From 610a910aaf4b112da5bc7da515c085de086d8454 Mon Sep 17 00:00:00 2001 From: Yong Zhang Date: Mon, 9 Mar 2020 20:01:36 +0800 Subject: [PATCH 017/140] =?UTF-8?q?Localize=20=E2=80=9Cemail=20address?= =?UTF-8?q?=E2=80=9D=20placeholder=20on=20home=20page=20(#19410)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- i18n/de.toml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/i18n/de.toml b/i18n/de.toml index ea0ad27f74..4155902ec7 100644 --- a/i18n/de.toml +++ b/i18n/de.toml @@ -195,3 +195,6 @@ other = "Veranstaltungskalender" # UI elements [ui_search_placeholder] other = "Suchen" + +[input_placeholder_email_address] +other = "E-Mail-Addresse" \ No newline at end of file From 73b6f80f36deb8c8b725ae27412ad068d706b605 Mon Sep 17 00:00:00 2001 From: Rajesh Deshpande Date: Mon, 9 Mar 2020 17:33:36 +0530 Subject: [PATCH 018/140] Adding 'whats next' section to the PersistentVolume page (#19557) * Add whats next section to PersistentVolume page Add whats next section to PersistentVolume page * Adding references to PersistentVolume page Adding references to PersistentVolume page --- .../en/docs/concepts/storage/persistent-volumes.md | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/content/en/docs/concepts/storage/persistent-volumes.md b/content/en/docs/concepts/storage/persistent-volumes.md index 003b1836c5..5ddd72d251 100644 --- a/content/en/docs/concepts/storage/persistent-volumes.md +++ b/content/en/docs/concepts/storage/persistent-volumes.md @@ -743,5 +743,17 @@ and need persistent storage, it is recommended that you use the following patter dynamic storage support (in which case the user should create a matching PV) or the cluster has no storage system (in which case the user cannot deploy config requiring PVCs). - +{{% /capture %}} + {{% capture whatsnext %}} + +* Learn more about [Creating a Persistent Volume](/docs/tasks/configure-pod-container/configure-persistent-volume-storage/#create-a-persistentvolume). +* Learn more about [Creating a Persistent Volume Claim](/docs/tasks/configure-pod-container/configure-persistent-volume-storage/#create-a-persistentvolumeclaim). +* Read the [Persistent Storage design document](https://git.k8s.io/community/contributors/design-proposals/storage/persistent-storage.md). + +### Reference + +* [PersistentVolume](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#persistentvolume-v1-core) +* [PersistentVolumeSpec](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#persistentvolumespec-v1-core) +* [PersistentVolumeClaim](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#persistentvolumeclaim-v1-core) +* [PersistentVolumeClaimSpec](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#persistentvolumeclaimspec-v1-core) {{% /capture %}} From cc594a19a916ca3fbf4f0ed885db67182ac6eea8 Mon Sep 17 00:00:00 2001 From: Morten Lied Johansen Date: Mon, 9 Mar 2020 17:17:36 +0100 Subject: [PATCH 019/140] Added FIAAS Python client library (#19224) --- content/en/docs/reference/using-api/client-libraries.md | 1 + 1 file changed, 1 insertion(+) diff --git a/content/en/docs/reference/using-api/client-libraries.md b/content/en/docs/reference/using-api/client-libraries.md index 093490b345..4f76e16352 100644 --- a/content/en/docs/reference/using-api/client-libraries.md +++ b/content/en/docs/reference/using-api/client-libraries.md @@ -60,6 +60,7 @@ their authors, not the Kubernetes team. | PHP | [github.com/allansun/kubernetes-php-client](https://github.com/allansun/kubernetes-php-client) | | PHP | [github.com/travisghansen/kubernetes-client-php](https://github.com/travisghansen/kubernetes-client-php) | | Python | [github.com/eldarion-gondor/pykube](https://github.com/eldarion-gondor/pykube) | +| Python | [github.com/fiaas/k8s](https://github.com/fiaas/k8s) | | Python | [github.com/mnubo/kubernetes-py](https://github.com/mnubo/kubernetes-py) | | Python | [github.com/tomplus/kubernetes_asyncio](https://github.com/tomplus/kubernetes_asyncio) | | Ruby | [github.com/Ch00k/kuber](https://github.com/Ch00k/kuber) | From c2b7c035f34b5ed72bc8396786b4b2dd075ad391 Mon Sep 17 00:00:00 2001 From: Raghu Katti Date: Mon, 9 Mar 2020 12:29:38 -0400 Subject: [PATCH 020/140] Added cluster-health check instructions (#18958) * Added cluster-health check instructions * Added cluster info command Added cluster info command * Fixed Cluster health check instruction * Fixed typo in documentation --- .../en/docs/tasks/debug-application-cluster/debug-cluster.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/content/en/docs/tasks/debug-application-cluster/debug-cluster.md b/content/en/docs/tasks/debug-application-cluster/debug-cluster.md index ae56c42411..495545ee1a 100644 --- a/content/en/docs/tasks/debug-application-cluster/debug-cluster.md +++ b/content/en/docs/tasks/debug-application-cluster/debug-cluster.md @@ -29,6 +29,11 @@ kubectl get nodes And verify that all of the nodes you expect to see are present and that they are all in the `Ready` state. +To get detailed information about the overall health of your cluster, you can run: + +```shell +kubectl cluster-info dump +``` ## Looking at logs For now, digging deeper into the cluster requires logging into the relevant machines. Here are the locations From 6cfb6b76ff194d08efb477b2ecfb3e72b05851ec Mon Sep 17 00:00:00 2001 From: Jordan Liggitt Date: Mon, 9 Mar 2020 13:05:37 -0400 Subject: [PATCH 021/140] Update deprecated API details (#19388) --- ...19-07-18-some-apis-are-being-deprecated.md | 24 +++++++++---------- 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/content/en/blog/_posts/2019-07-18-some-apis-are-being-deprecated.md b/content/en/blog/_posts/2019-07-18-some-apis-are-being-deprecated.md index 9639e87a53..587080617c 100644 --- a/content/en/blog/_posts/2019-07-18-some-apis-are-being-deprecated.md +++ b/content/en/blog/_posts/2019-07-18-some-apis-are-being-deprecated.md @@ -23,30 +23,30 @@ The **v1.16** release will stop serving the following deprecated API versions in Existing persisted data can be retrieved/updated via the new version. * Notable changes: * `spec.templateGeneration` is removed - * `spec.selector` is now required and immutable after creation - * `spec.updateStrategy.type` now defaults to `RollingUpdate` + * `spec.selector` is now required and immutable after creation; use the existing template labels as the selector for seamless upgrades + * `spec.updateStrategy.type` now defaults to `RollingUpdate` (the default in `extensions/v1beta1` was `OnDelete`) * Deployment in the **extensions/v1beta1**, **apps/v1beta1**, and **apps/v1beta2** API versions is no longer served * Migrate to use the **apps/v1** API version, available since v1.9. Existing persisted data can be retrieved/updated via the new version. * Notable changes: * `spec.rollbackTo` is removed - * `spec.selector` is now required and immutable after creation - * `spec.progressDeadlineSeconds` now defaults to `600` seconds - * `spec.revisionHistoryLimit` now defaults to `10` - * `maxSurge` and `maxUnavailable` now default to `25%` + * `spec.selector` is now required and immutable after creation; use the existing template labels as the selector for seamless upgrades + * `spec.progressDeadlineSeconds` now defaults to `600` seconds (the default in `extensions/v1beta1` was no deadline) + * `spec.revisionHistoryLimit` now defaults to `10` (the default in `apps/v1beta1` was `2`, the default in `extensions/v1beta1` was to retain all) + * `maxSurge` and `maxUnavailable` now default to `25%` (the default in `extensions/v1beta1` was `1`) * StatefulSet in the **apps/v1beta1** and **apps/v1beta2** API versions is no longer served * Migrate to use the **apps/v1** API version, available since v1.9. Existing persisted data can be retrieved/updated via the new version. * Notable changes: - * `spec.selector` is now required and immutable after creation - * `spec.updateStrategy.type` now defaults to `RollingUpdate` + * `spec.selector` is now required and immutable after creation; use the existing template labels as the selector for seamless upgrades + * `spec.updateStrategy.type` now defaults to `RollingUpdate` (the default in `apps/v1beta1` was `OnDelete`) * ReplicaSet in the **extensions/v1beta1**, **apps/v1beta1**, and **apps/v1beta2** API versions is no longer served * Migrate to use the **apps/v1** API version, available since v1.9. Existing persisted data can be retrieved/updated via the new version. * Notable changes: - * `spec.selector` is now required and immutable after creation + * `spec.selector` is now required and immutable after creation; use the existing template labels as the selector for seamless upgrades -The **v1.20** release will stop serving the following deprecated API versions in favor of newer and more stable API versions: +The **v1.22** release will stop serving the following deprecated API versions in favor of newer and more stable API versions: * Ingress in the **extensions/v1beta1** API version will no longer be served * Migrate to use the **networking.k8s.io/v1beta1** API version, available since v1.14. @@ -84,8 +84,8 @@ apiserver startup arguments: Deprecations are announced in the Kubernetes release notes. You can see these announcements in -[1.14](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.14.md#deprecations) -and [1.15](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.15.md#deprecations-and-removals). +[1.14](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.14.md#deprecations) +and [1.15](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.15.md#deprecations-and-removals). You can read more [in our deprecation policy document](https://kubernetes.io/docs/reference/using-api/deprecation-policy/#deprecating-parts-of-the-api) about the deprecation policies for Kubernetes APIs, and other Kubernetes components. From 05bf8be2051737abd92c666bea545bc9a40140ec Mon Sep 17 00:00:00 2001 From: Jacky Wu Date: Tue, 10 Mar 2020 08:03:37 +0800 Subject: [PATCH 022/140] Federation Cleanup (1) doc: clean some outdate redirects and its associated content. (#19572) * doc: clean some outdate redirects and its associated content. * doc: also cleanup tasks.yml content. --- .../connect-applications-service.md | 10 ------- .../contribute/style/content-organization.md | 1 - .../federated-task-tutorial-prereqs.md | 5 ---- data/tasks.yml | 19 -------------- data/tools.yml | 4 +-- static/_redirects | 26 ------------------- 6 files changed, 1 insertion(+), 64 deletions(-) delete mode 100644 content/en/includes/federated-task-tutorial-prereqs.md diff --git a/content/en/docs/concepts/services-networking/connect-applications-service.md b/content/en/docs/concepts/services-networking/connect-applications-service.md index 6a958ea31e..44ff5c331d 100644 --- a/content/en/docs/concepts/services-networking/connect-applications-service.md +++ b/content/en/docs/concepts/services-networking/connect-applications-service.md @@ -419,13 +419,3 @@ LoadBalancer Ingress: a320587ffd19711e5a37606cf4a74574-1142138393.us-east-1.el ``` {{% /capture %}} - -{{% capture whatsnext %}} - -Kubernetes also supports Federated Services, which can span multiple -clusters and cloud providers, to provide increased availability, -better fault tolerance and greater scalability for your services. See -the [Federated Services User Guide](/docs/concepts/cluster-administration/federation-service-discovery/) -for further information. - -{{% /capture %}} diff --git a/content/en/docs/contribute/style/content-organization.md b/content/en/docs/contribute/style/content-organization.md index 55997dcaf5..e93cf8126e 100644 --- a/content/en/docs/contribute/style/content-organization.md +++ b/content/en/docs/contribute/style/content-organization.md @@ -107,7 +107,6 @@ Another widely used example is the `includes` bundle. It sets `headless: true` i ```bash en/includes ├── default-storage-class-prereqs.md -├── federated-task-tutorial-prereqs.md ├── index.md ├── partner-script.js ├── partner-style.css diff --git a/content/en/includes/federated-task-tutorial-prereqs.md b/content/en/includes/federated-task-tutorial-prereqs.md deleted file mode 100644 index b254407a67..0000000000 --- a/content/en/includes/federated-task-tutorial-prereqs.md +++ /dev/null @@ -1,5 +0,0 @@ -This guide assumes that you have a running Kubernetes Cluster Federation installation. -If not, then head over to the [federation admin guide](/docs/tutorials/federation/set-up-cluster-federation-kubefed/) to learn how to -bring up a cluster federation (or have your cluster administrator do this for you). -Other tutorials, such as Kelsey Hightower's [Federated Kubernetes Tutorial](https://github.com/kelseyhightower/kubernetes-cluster-federation), -might also help you create a Federated Kubernetes cluster. diff --git a/data/tasks.yml b/data/tasks.yml index 71ca4c957d..4220e059cc 100644 --- a/data/tasks.yml +++ b/data/tasks.yml @@ -196,25 +196,6 @@ toc: - docs/tasks/administer-cluster/storage-object-in-use-protection.md - docs/tasks/administer-cluster/endpoint-slices.md -- title: Federation - Run an App on Multiple Clusters - landing_page: /docs/tasks/federation/set-up-cluster-federation-kubefed/ - section: - - docs/tasks/federation/federation-service-discovery.md - - docs/tasks/federation/set-up-cluster-federation-kubefed.md - - docs/tasks/federation/set-up-coredns-provider-federation.md - - docs/tasks/federation/set-up-placement-policies-federation.md - - docs/tasks/administer-federation/cluster.md - - docs/tasks/administer-federation/configmap.md - - docs/tasks/administer-federation/daemonset.md - - docs/tasks/administer-federation/deployment.md - - docs/tasks/administer-federation/events.md - - docs/tasks/administer-federation/hpa.md - - docs/tasks/administer-federation/ingress.md - - docs/tasks/administer-federation/job.md - - docs/tasks/administer-federation/namespaces.md - - docs/tasks/administer-federation/replicaset.md - - docs/tasks/administer-federation/secret.md - - title: Manage Cluster Daemons landing_page: /docs/tasks/manage-daemon/update-daemon-set/ section: diff --git a/data/tools.yml b/data/tools.yml index 53a9e5ef43..0f5a970d3c 100644 --- a/data/tools.yml +++ b/data/tools.yml @@ -9,11 +9,9 @@ toc: path: /docs/reference/kubectl/overview/ - title: Kubeadm path: /docs/getting-started-guides/kubeadm - - title: Kubefed - path: /docs/admin/federation/kubefed/ - title: Kubernetes Dashboard path: /docs/user-guide/ui/ - + - title: Third-Party Tools section: - docs/tools/kompose/index.md diff --git a/static/_redirects b/static/_redirects index 64dd0e9041..e032dcbdf9 100644 --- a/static/_redirects +++ b/static/_redirects @@ -37,8 +37,6 @@ /docs/admin/etcd/ /docs/tasks/administer-cluster/configure-upgrade-etcd/ 301 /docs/admin/etcd_upgrade/ /docs/tasks/administer-cluster/configure-upgrade-etcd/ 301 /docs/admin/extensible-admission-controllers.md /docs/admin/extensible-admission-controllers/ 301 -/docs/admin/federation/kubefed/ /docs/tasks/federation/set-up-cluster-federation-kubefed/ 301 -/docs/admin/federation/kubefed.md /docs/tasks/federation/set-up-cluster-federation-kubefed/ 301 /docs/admin/garbage-collection/ /docs/concepts/cluster-administration/kubelet-garbage-collection/ 301 /docs/admin/ha-master-gce/ /docs/tasks/administer-cluster/highly-available-master/ 301 /docs/admin/ha-master-gce.md/ /docs/tasks/administer-cluster/highly-available-master/ 301 @@ -49,7 +47,6 @@ /docs/admin/limitrange/ /docs/tasks/administer-cluster/cpu-memory-limit/ 301 /docs/admin/limitrange/Limits/ /docs/tasks/administer-cluster/limit-storage-consumption/#limitrange-to-limit-requests-for-storage/ 301 /docs/admin/master-node-communication/ /docs/concepts/architecture/master-node-communication/ 301 -/docs/admin/multi-cluster/ /docs/concepts/cluster-administration/federation/ 301 /docs/admin/multiple-schedulers/ /docs/tasks/administer-cluster/configure-multiple-schedulers/ 301 /docs/admin/namespaces/ /docs/tasks/administer-cluster/namespaces/ 301 /docs/admin/namespaces/walkthrough/ /docs/tasks/administer-cluster/namespaces-walkthrough/ 301 @@ -96,10 +93,8 @@ /docs/concepts/cluster-administration/configure-etcd/ /docs/tasks/administer-cluster/configure-upgrade-etcd/ 301 /docs/concepts/cluster-administration/device-plugins/ /docs/concepts/extend-kubernetes/compute-storage-net/device-plugins/ 301 /docs/concepts/cluster-administration/etcd-upgrade/ /docs/tasks/administer-cluster/configure-upgrade-etcd/ 301 -/docs/concepts/cluster-administration/federation-service-discovery/ /docs/tasks/federation/federation-service-discovery/ 301 /docs/concepts/cluster-administration/guaranteed-scheduling-critical-addon-pods/ /docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods/ 301 /docs/concepts/cluster-administration/master-node-communication/ /docs/concepts/architecture/master-node-communication/ 301 -/docs/concepts/cluster-administration/multiple-clusters/ /docs/concepts/cluster-administration/federation/ 301 /docs/concepts/cluster-administration/network-plugins/ /docs/concepts/extend-kubernetes/compute-storage-net/network-plugins/ 301 /docs/concepts/cluster-administration/out-of-resource/ /docs/tasks/administer-cluster/out-of-resource/ 301 /docs/concepts/cluster-administration/resource-usage-monitoring /docs/tasks/debug-application-cluster/resource-usage-monitoring/ 301 @@ -286,7 +281,6 @@ /docs/tasks/configure-pod-container/romana-network-policy/ /docs/tasks/administer-cluster/romana-network-policy/ 301 /docs/tasks/configure-pod-container/weave-network-policy/ /docs/tasks/administer-cluster/weave-network-policy/ 301 /docs/tasks/debug-application-cluster/sematext-logging-monitoring/ https://sematext.com/kubernetes/ 301 -/docs/tasks/federation/set-up-cluster-federation-kubefed.md /docs/tasks/federation/set-up-cluster-federation-kubefed/ 301 /docs/tasks/job/work-queue-1/ /docs/concepts/workloads/controllers/jobs-run-to-completion/ 301 /docs/tasks/kubectl/get-shell-running-container/ /docs/tasks/debug-application-cluster/get-shell-running-container/ 301 /docs/tasks/kubectl/install/ /docs/tasks/tools/install-kubectl/ 301 @@ -309,10 +303,6 @@ /docs/tutorials/clusters/multiple-schedulers/ /docs/tasks/administer-cluster/configure-multiple-schedulers/ 301 /docs/tutorials/connecting-apps/connecting-frontend-backend/ /docs/tasks/access-application-cluster/connecting-frontend-backend/ 301 -/docs/tutorials/federation/set-up-cluster-federation-kubefed/ /docs/tasks/federation/set-up-cluster-federation-kubefed/ 301 -/docs/tutorials/federation/set-up-cluster-federation-kubefed.md /docs/tasks/federation/set-up-cluster-federation-kubefed/ 301 -/docs/tutorials/federation/set-up-coredns-provider-federation/ /docs/tasks/federation/set-up-coredns-provider-federation/ 301 -/docs/tutorials/federation/set-up-placement-policies-federation/ /docs/tasks/federation/set-up-placement-policies-federation/ 301 /docs/tutorials/kubernetes-basics/cluster-interactive/ /docs/tutorials/kubernetes-basics/create-cluster/cluster-interactive/ 301 /docs/tutorials/kubernetes-basics/cluster-intro/ /docs/tutorials/kubernetes-basics/create-cluster/cluster-intro/ 301 /docs/tutorials/kubernetes-basics/deploy-interactive/ /docs/tutorials/kubernetes-basics/deploy-app/deploy-interactive/ 301 @@ -342,7 +332,6 @@ /docs/tutorials/stateless-application/hello-minikube/ /docs/tutorials/hello-minikube/ 301 /docs/tutorials/stateless-application/run-stateless-ap-replication-controller/ /docs/tasks/run-application/run-stateless-application-deployment/ 301 /docs/tutorials/stateless-application/run-stateless-application-deployment/ /docs/tasks/run-application/run-stateless-application-deployment/ 301 -/docs/tutorials/tasks/set-up-cluster-federation-kubefed/ /docs/tasks/federation/set-up-cluster-federation-kubefed/ 301 /docs/user-guide/ /docs/home/ 301 /docs/user-guide/accessing-the-cluster/ /docs/tasks/access-application-cluster/access-cluster/ 301 @@ -369,21 +358,6 @@ /docs/user-guide/downward-api/README/ /docs/tasks/inject-data-application/downward-api-volume-expose-pod-information/ 301 /docs/user-guide/downward-api/volume/ /docs/tasks/inject-data-application/downward-api-volume-expose-pod-information/ 301 /docs/user-guide/environment-guide/ /docs/tasks/inject-data-application/environment-variable-expose-pod-information/ 301 -/docs/user-guide/federation/ /docs/concepts/cluster-administration/federation/ 301 -/docs/user-guide/federation/cluster/ /docs/tasks/administer-federation/cluster/ 301 -/docs/user-guide/federation/configmap/ /docs/tasks/administer-federation/configmap/ 301 -/docs/user-guide/federation/configmap.md /docs/tasks/administer-federation/configmap/ 301 -/docs/user-guide/federation/daemonsets/ /docs/tasks/administer-federation/daemonset/ 301 -/docs/user-guide/federation/daemonsets.md /docs/tasks/administer-federation/daemonset/ 301 -/docs/user-guide/federation/deployment/ /docs/tasks/administer-federation/deployment/ 301 -/docs/user-guide/federation/deployment.md /docs/tasks/administer-federation/deployment/ 301 -/docs/user-guide/federation/events/ /docs/tasks/administer-federation/events/ 301 -/docs/user-guide/federation/federated-ingress/ /docs/tasks/administer-federation/ingress/ 301 -/docs/user-guide/federation/federated-services/ /docs/tasks/federation/federation-service-discovery/ 301 -/docs/user-guide/federation/index.md /docs/concepts/cluster-administration/federation/ 301 -/docs/user-guide/federation/namespaces/ /docs/tasks/administer-federation/namespaces/ 301 -/docs/user-guide/federation/replicasets/ /docs/tasks/administer-federation/replicaset/ 301 -/docs/user-guide/federation/secrets/ /docs/tasks/administer-federation/secret/ 301 /docs/user-guide/garbage-collection/ /docs/concepts/workloads/controllers/garbage-collection/ 301 /docs/user-guide/garbage-collector/* /docs/concepts/workloads/controllers/garbage-collection/ 301 /docs/user-guide/gpus/ /docs/tasks/manage-gpus/scheduling-gpus/ 301 From dade10e4548da0381f41889a4910c39c797c2160 Mon Sep 17 00:00:00 2001 From: Tim Bannister Date: Tue, 10 Mar 2020 13:53:36 +0000 Subject: [PATCH 023/140] Tidy page about Poseidon/Firmament scheduler (#18626) * Reword Poseidon/Firmament concept * Tidy feature state information * Fix whitespace * Use caution shortcode for note * Improve table accessibility * Tidy page --- .../poseidon-firmament-alternate-scheduler.md | 116 +++++++++--------- 1 file changed, 55 insertions(+), 61 deletions(-) diff --git a/content/en/docs/concepts/extend-kubernetes/poseidon-firmament-alternate-scheduler.md b/content/en/docs/concepts/extend-kubernetes/poseidon-firmament-alternate-scheduler.md index 1afbc17c09..4c5ab12c03 100644 --- a/content/en/docs/concepts/extend-kubernetes/poseidon-firmament-alternate-scheduler.md +++ b/content/en/docs/concepts/extend-kubernetes/poseidon-firmament-alternate-scheduler.md @@ -1,117 +1,111 @@ --- -title: Poseidon-Firmament - An alternate scheduler +title: Poseidon-Firmament Scheduler content_template: templates/concept weight: 80 --- {{% capture overview %}} -**Current release of Poseidon-Firmament scheduler is an alpha release.** +{{< feature-state for_k8s_version="v1.6" state="alpha" >}} -Poseidon-Firmament scheduler is an alternate scheduler that can be deployed alongside the default Kubernetes scheduler. +The Poseidon-Firmament scheduler is an alternate scheduler that can be deployed alongside the default Kubernetes scheduler. {{% /capture %}} {{% capture body %}} -## Introduction +## Introduction -Poseidon is a service that acts as the integration glue for the [Firmament scheduler](https://github.com/Huawei-PaaS/firmament) with Kubernetes. Poseidon-Firmament scheduler augments the current Kubernetes scheduling capabilities. It incorporates novel flow network graph based scheduling capabilities alongside the default Kubernetes Scheduler. Firmament scheduler models workloads and clusters as flow networks and runs min-cost flow optimizations over these networks to make scheduling decisions. +Poseidon is a service that acts as the integration glue between the [Firmament scheduler](https://github.com/Huawei-PaaS/firmament) and Kubernetes. Poseidon-Firmament augments the current Kubernetes scheduling capabilities. It incorporates novel flow network graph based scheduling capabilities alongside the default Kubernetes scheduler. The Firmament scheduler models workloads and clusters as flow networks and runs min-cost flow optimizations over these networks to make scheduling decisions. -It models the scheduling problem as a constraint-based optimization over a flow network graph. This is achieved by reducing scheduling to a min-cost max-flow optimization problem. The Poseidon-Firmament scheduler dynamically refines the workload placements. +Firmament models the scheduling problem as a constraint-based optimization over a flow network graph. This is achieved by reducing scheduling to a min-cost max-flow optimization problem. The Poseidon-Firmament scheduler dynamically refines the workload placements. -Poseidon-Firmament scheduler runs alongside the default Kubernetes Scheduler as an alternate scheduler, so multiple schedulers run simultaneously. +Poseidon-Firmament scheduler runs alongside the default Kubernetes scheduler as an alternate scheduler. You can simultaneously run multiple, different schedulers. -## Key Advantages +Flow graph scheduling with the Poseidon-Firmament scheduler provides the following advantages: -### Flow graph scheduling based Poseidon-Firmament scheduler provides the following key advantages: -- Workloads (pods) are bulk scheduled to enable scheduling at massive scale.. -- Based on the extensive performance test results, Poseidon-Firmament scales much better than the Kubernetes default scheduler as the number of nodes increase in a cluster. This is due to the fact that Poseidon-Firmament is able to amortize more and more work across workloads. -- Poseidon-Firmament Scheduler outperforms the Kubernetes default scheduler by a wide margin when it comes to throughput performance numbers for scenarios where compute resource requirements are somewhat uniform across jobs (Replicasets/Deployments/Jobs). Poseidon-Firmament scheduler end-to-end throughput performance numbers, including bind time, consistently get better as the number of nodes in a cluster increase. For example, for a 2,700 node cluster (shown in the graphs [here](https://github.com/kubernetes-sigs/poseidon/blob/master/docs/benchmark/README.md)), Poseidon-Firmament scheduler achieves a 7X or greater end-to-end throughput than the Kubernetes default scheduler, which includes bind time. +- Workloads (Pods) are bulk scheduled to enable scheduling at massive scale. + The Poseidon-Firmament scheduler outperforms the Kubernetes default scheduler by a wide margin when it comes to throughput performance for scenarios where compute resource requirements are somewhat uniform across your workload (Deployments, ReplicaSets, Jobs). +- The Poseidon-Firmament's scheduler's end-to-end throughput performance and bind time improves as the number of nodes in a cluster increases. As you scale out, Poseidon-Firmament scheduler is able to amortize more and more work across workloads. +- Scheduling in Poseidon-Firmament is dynamic; it keeps cluster resources in a global optimal state during every scheduling run. +- The Poseidon-Firmament scheduler supports scheduling complex rule constraints. -- Availability of complex rule constraints. -- Scheduling in Poseidon-Firmament is dynamic; it keeps cluster resources in a global optimal state during every scheduling run. -- Highly efficient resource utilizations. +## How the Poseidon-Firmament scheduler works -## Poseidon-Firmament Scheduler - How it works +Kubernetes supports [using multiple schedulers](/docs/tasks/administer-cluster/configure-multiple-schedulers/). You can specify, for a particular Pod, that it is scheduled by a custom scheduler (“poseidon” for this case), by setting the `schedulerName` field in the PodSpec at the time of pod creation. The default scheduler will ignore that Pod and allow Poseidon-Firmament scheduler to schedule the Pod on a relevant node. -As part of the Kubernetes multiple schedulers support, each new pod is typically scheduled by the default scheduler. Kubernetes can be instructed to use another scheduler by specifying the name of another custom scheduler (“poseidon” in our case) in the **schedulerName** field of the PodSpec at the time of pod creation. In this case, the default scheduler will ignore that Pod and allow Poseidon scheduler to schedule the Pod on a relevant node. +For example: ```yaml apiVersion: v1 kind: Pod - ... spec: - schedulerName: poseidon -``` + schedulerName: poseidon +... +``` - -{{< note >}} -For details about the design of this project see the [design document](https://github.com/kubernetes-sigs/poseidon/blob/master/docs/design/README.md). -{{< /note >}} - -## Possible Use Case Scenarios - When to use it +## Batch scheduling As mentioned earlier, Poseidon-Firmament scheduler enables an extremely high throughput scheduling environment at scale due to its bulk scheduling approach versus Kubernetes pod-at-a-time approach. In our extensive tests, we have observed substantial throughput benefits as long as resource requirements (CPU/Memory) for incoming Pods are uniform across jobs (Replicasets/Deployments/Jobs), mainly due to efficient amortization of work across jobs. Although, Poseidon-Firmament scheduler is capable of scheduling various types of workloads, such as service, batch, etc., the following are a few use cases where it excels the most: -1. For “Big Data/AI” jobs consisting of large number of tasks, throughput benefits are tremendous. -2. Service or batch jobs where workload resource requirements are uniform across jobs (Replicasets/Deployments/Jobs). +1. For “Big Data/AI” jobs consisting of large number of tasks, throughput benefits are tremendous. +2. Service or batch jobs where workload resource requirements are uniform across jobs (Replicasets/Deployments/Jobs). -## Current Project Stage +## Feature state -- **Alpha Release - Incubation repo.** at https://github.com/kubernetes-sigs/poseidon. -- Currently, Poseidon-Firmament scheduler **does not provide support for high availability**, our implementation assumes that the scheduler cannot fail. The [design document](https://github.com/kubernetes-sigs/poseidon/blob/master/docs/design/README.md) describes possible ways to enable high availability, but we leave this to future work. -- We are **not aware of any production deployment** of Poseidon-Firmament scheduler at this time. -- Poseidon-Firmament is supported from Kubernetes release 1.6 and works with all subsequent releases. -- Release process for Poseidon and Firmament repos are in lock step. The current Poseidon release can be found [here](https://github.com/kubernetes-sigs/poseidon/releases) and the corresponding Firmament release can be found [here](https://github.com/Huawei-PaaS/firmament/releases). +Poseidon-Firmament is designed to work with Kubernetes release 1.6 and all subsequent releases. -## Features Comparison Matrix +{{< caution >}} +Poseidon-Firmament scheduler does not provide support for high availability; its implementation assumes that the scheduler cannot fail. +{{< /caution >}} +## Feature comparison {#feature-comparison-matrix} +{{< table caption="Feature comparison of Kubernetes and Poseidon-Firmament schedulers." >}} |Feature|Kubernetes Default Scheduler|Poseidon-Firmament Scheduler|Notes| |--- |--- |--- |--- | |Node Affinity/Anti-Affinity|Y|Y|| -|Pod Affinity/Anti-Affinity - including support for pod anti-affinity symmetry|Y|Y|Currently, the default scheduler outperforms the Poseidon-Firmament scheduler pod affinity/anti-affinity functionality. We are working towards resolving this.| +|Pod Affinity/Anti-Affinity - including support for pod anti-affinity symmetry|Y|Y|The default scheduler outperforms the Poseidon-Firmament scheduler pod affinity/anti-affinity functionality.| |Taints & Tolerations|Y|Y|| -|Baseline Scheduling capability in accordance to available compute resources (CPU & Memory) on a node|Y|Y**|Not all Predicates & Priorities are supported at this time.| -|Extreme Throughput at scale|Y**|Y|Bulk scheduling approach scales or increases workload placement. Substantial throughput benefits using Firmament scheduler as long as resource requirements (CPU/Memory) for incoming Pods is uniform across Replicasets/Deployments/Jobs. This is mainly due to efficient amortization of work across Replicasets/Deployments/Jobs . 1) For “Big Data/AI” jobs consisting of large no. of tasks, throughput benefits are tremendous. 2) Substantial throughput benefits also for service or batch job scenarios where workload resource requirements are uniform across Replicasets/Deployments/Jobs.| -|Optimal Scheduling|Pod-by-Pod scheduler, processes one pod at a time (may result into sub-optimal scheduling)|Bulk Scheduling (Optimal scheduling)|Pod-by-Pod Kubernetes default scheduler may assign tasks to a sub-optimal machine. By contrast, Firmament considers all unscheduled tasks at the same time together with their soft and hard constraints.| -|Colocation Interference Avoidance|N|N**|Planned in Poseidon-Firmament.| -|Priority Pre-emption|Y|N**|Partially exists in Poseidon-Firmament versus extensive support in Kubernetes default scheduler.| -|Inherent Re-Scheduling|N|Y**|Poseidon-Firmament scheduler supports workload re-scheduling. In each scheduling run it considers all the pods, including running pods, and as a result can migrate or evict pods – a globally optimal scheduling environment.| +|Baseline Scheduling capability in accordance to available compute resources (CPU & Memory) on a node|Y|Y†|**†** Not all Predicates & Priorities are supported with Poseidon-Firmament.| +|Extreme Throughput at scale|Y†|Y|**†** Bulk scheduling approach scales or increases workload placement. Firmament scheduler offers high throughput when resource requirements (CPU/Memory) for incoming Pods are uniform across ReplicaSets/Deployments/Jobs.| +|Colocation Interference Avoidance|N|N|| +|Priority Preemption|Y|N†|**†** Partially exists in Poseidon-Firmament versus extensive support in Kubernetes default scheduler.| +|Inherent Rescheduling|N|Y†|**†** Poseidon-Firmament scheduler supports workload re-scheduling. In each scheduling run, Poseidon-Firmament considers all Pods, including running Pods, and as a result can migrate or evict Pods – a globally optimal scheduling environment.| |Gang Scheduling|N|Y|| |Support for Pre-bound Persistence Volume Scheduling|Y|Y|| -|Support for Local Volume & Dynamic Persistence Volume Binding Scheduling|Y|N**|Planned.| -|High Availability|Y|N**|Planned.| -|Real-time metrics based scheduling|N|Y**|Initially supported using Heapster (now deprecated) for placing pods using actual cluster utilization statistics rather than reservations. Plans to switch over to "metric server".| +|Support for Local Volume & Dynamic Persistence Volume Binding Scheduling|Y|N|| +|High Availability|Y|N|| +|Real-time metrics based scheduling|N|Y†|**†** Partially supported in Poseidon-Firmament using Heapster (now deprecated) for placing Pods using actual cluster utilization statistics rather than reservations.| |Support for Max-Pod per node|Y|Y|Poseidon-Firmament scheduler seamlessly co-exists with Kubernetes default scheduler.| |Support for Ephemeral Storage, in addition to CPU/Memory|Y|Y|| +{{< /table >}} +## Installation -## Installation +The [Poseidon-Firmament installation guide](https://github.com/kubernetes-sigs/poseidon/blob/master/docs/install/README.md#Installation) explains how to deploy Poseidon-Firmament to your cluster. -For in-cluster installation of Poseidon, please start at the [Installation instructions](https://github.com/kubernetes-sigs/poseidon/blob/master/docs/install/README.md). - - -## Development - -For developers, please refer to the [Developer Setup instructions](https://github.com/kubernetes-sigs/poseidon/blob/master/docs/devel/README.md). - -## Latest Throughput Performance Testing Results - -Pod-by-pod schedulers, such as the Kubernetes default scheduler, typically process one pod at a time. These schedulers have the following crucial drawbacks: - -1. The scheduler commits to a pod placement early and restricts the choices for other pods that wait to be placed. -2. There is limited opportunities for amortizing work across pods because they are considered for placement individually. - -These downsides of pod-by-pod schedulers are addressed by batching or bulk scheduling in Poseidon-Firmament scheduler. Processing several pods in a batch allows the scheduler to jointly consider their placement, and thus to find the best trade-off for the whole batch instead of one pod. At the same time it amortizes work across pods resulting in much higher throughput. +## Performance comparison {{< note >}} Please refer to the [latest benchmark results](https://github.com/kubernetes-sigs/poseidon/blob/master/docs/benchmark/README.md) for detailed throughput performance comparison test results between Poseidon-Firmament scheduler and the Kubernetes default scheduler. {{< /note >}} +Pod-by-pod schedulers, such as the Kubernetes default scheduler, process Pods in small batches (typically one at a time). These schedulers have the following crucial drawbacks: + +1. The scheduler commits to a pod placement early and restricts the choices for other pods that wait to be placed. +2. There is limited opportunities for amortizing work across pods because they are considered for placement individually. + +These downsides of pod-by-pod schedulers are addressed by batching or bulk scheduling in Poseidon-Firmament scheduler. Processing several pods in a batch allows the scheduler to jointly consider their placement, and thus to find the best trade-off for the whole batch instead of one pod. At the same time it amortizes work across pods resulting in much higher throughput. + +{{% /capture %}} +{{% capture whatsnext %}} +* See [Poseidon-Firmament](https://github.com/kubernetes-sigs/poseidon#readme) on GitHub for more information. +* See the [design document](https://github.com/kubernetes-sigs/poseidon/blob/master/docs/design/README.md) for Poseidon. +* Read [Firmament: Fast, Centralized Cluster Scheduling at Scale](https://www.usenix.org/system/files/conference/osdi16/osdi16-gog.pdf), the academic paper on the Firmament scheduling design. +* If you'd like to contribute to Poseidon-Firmament, refer to the [developer setup instructions](https://github.com/kubernetes-sigs/poseidon/blob/master/docs/devel/README.md). {{% /capture %}} From 4a024808673b958d93ecb60bb018d05f03d62719 Mon Sep 17 00:00:00 2001 From: Tim Hockin Date: Tue, 10 Mar 2020 07:17:37 -0700 Subject: [PATCH 024/140] Do a top-to-bottom review of "debug-services" (#19152) * Do a top-to-bottom review of "debug-services" I frequently refer people to this doc, so I thought it would make sense to review it. Overall ot was pretty up to date. I did a cleanup pass and moved a few things around. Mostly cosmetic. * Fix backticked resources in "debug-services" * Fix code vs output blocks in "debug-services" * Remove prompts from code blocks * s/we/you/ * Apply feedback edits --- .../debug-service.md | 507 ++++++++++-------- 1 file changed, 296 insertions(+), 211 deletions(-) diff --git a/content/en/docs/tasks/debug-application-cluster/debug-service.md b/content/en/docs/tasks/debug-application-cluster/debug-service.md index e0683fbd02..a065c9fa85 100644 --- a/content/en/docs/tasks/debug-application-cluster/debug-service.md +++ b/content/en/docs/tasks/debug-application-cluster/debug-service.md @@ -8,57 +8,30 @@ title: Debug Services {{% capture overview %}} An issue that comes up rather frequently for new installations of Kubernetes is -that a `Service` is not working properly. You've run your `Deployment` and -created a `Service`, but you get no response when you try to access it. -This document will hopefully help you to figure out what's going wrong. +that a Service is not working properly. You've run your Pods through a +Deployment (or other workload controller) and created a Service, but you +get no response when you try to access it. This document will hopefully help +you to figure out what's going wrong. {{% /capture %}} {{% capture body %}} -## Conventions - -Throughout this doc you will see various commands that you can run. Some -commands need to be run within a `Pod`, others on a Kubernetes `Node`, and others -can run anywhere you have `kubectl` and credentials for the cluster. To make it -clear what is expected, this document will use the following conventions. - -If the command "COMMAND" is expected to run in a `Pod` and produce "OUTPUT": - -```shell -u@pod$ COMMAND -OUTPUT -``` - -If the command "COMMAND" is expected to run on a `Node` and produce "OUTPUT": - -```shell -u@node$ COMMAND -OUTPUT -``` - -If the command is "kubectl ARGS": - -```shell -kubectl ARGS -OUTPUT -``` - ## Running commands in a Pod -For many steps here you will want to see what a `Pod` running in the cluster -sees. The simplest way to do this is to run an interactive alpine `Pod`: +For many steps here you will want to see what a Pod running in the cluster +sees. The simplest way to do this is to run an interactive alpine Pod: ```none kubectl run -it --rm --restart=Never alpine --image=alpine sh -/ # ``` + {{< note >}} If you don't see a command prompt, try pressing enter. {{< /note >}} -If you already have a running `Pod` that you prefer to use, you can run a +If you already have a running Pod that you prefer to use, you can run a command in it using: ```shell @@ -67,21 +40,23 @@ kubectl exec -c -- ## Setup -For the purposes of this walk-through, let's run some `Pods`. Since you're -probably debugging your own `Service` you can substitute your own details, or you +For the purposes of this walk-through, let's run some Pods. Since you're +probably debugging your own Service you can substitute your own details, or you can follow along and get a second data point. ```shell kubectl run hostnames --image=k8s.gcr.io/serve_hostname \ - --labels=app=hostnames \ - --port=9376 \ - --replicas=3 + --replicas=3 +``` +```none deployment.apps/hostnames created ``` `kubectl` commands will print the type and name of the resource created or mutated, which can then be used in subsequent commands. + {{< note >}} -This is the same as if you started the `Deployment` with the following YAML: +This is the same as if you had started the Deployment with the following +YAML: ```yaml apiVersion: apps/v1 @@ -91,61 +66,111 @@ metadata: spec: selector: matchLabels: - app: hostnames + run: hostnames replicas: 3 template: metadata: labels: - app: hostnames + run: hostnames spec: containers: - name: hostnames image: k8s.gcr.io/serve_hostname - ports: - - containerPort: 9376 - protocol: TCP ``` + +The label "run" is automatically set by `kubectl run` to the name of the +Deployment. {{< /note >}} -Confirm your `Pods` are running: +You can confirm your Pods are running: ```shell -kubectl get pods -l app=hostnames +kubectl get pods -l run=hostnames +``` +```none NAME READY STATUS RESTARTS AGE hostnames-632524106-bbpiw 1/1 Running 0 2m hostnames-632524106-ly40y 1/1 Running 0 2m hostnames-632524106-tlaok 1/1 Running 0 2m ``` +You can also confirm that your Pods are serving. You can get the list of +Pod IP addresses and test them directly. + +```shell +kubectl get pods -l run=hostnames \ + -o go-template='{{range .items}}{{.status.podIP}}{{"\n"}}{{end}}' +``` +```none +10.244.0.5 +10.244.0.6 +10.244.0.7 +``` + +The example container used for this walk-through simply serves its own hostname +via HTTP on port 9376, but if you are debugging your own app, you'll want to +use whatever port number your Pods are listening on. + +From within a pod: + +```shell +for ep in 10.244.0.5:9376 10.244.0.6:9376 10.244.0.7:9376; do + wget -qO- $ep +done +``` + +This should produce something like: + +``` +hostnames-0uton +hostnames-bvc05 +hostnames-yp2kp +``` + +If you are not getting the responses you expect at this point, your Pods +might not be healthy or might not be listening on the port you think they are. +You might find `kubectl logs` to be useful for seeing what is happening, or +perhaps you need to `kubectl exec` directly into your Pods and debug from +there. + +Assuming everything has gone to plan so far, you can start to investigate why +your Service doesn't work. + ## Does the Service exist? -The astute reader will have noticed that we did not actually create a `Service` +The astute reader will have noticed that you did not actually create a Service yet - that is intentional. This is a step that sometimes gets forgotten, and is the first thing to check. -So what would happen if I tried to access a non-existent `Service`? Assuming you -have another `Pod` that consumes this `Service` by name you would get something -like: +What would happen if you tried to access a non-existent Service? If +you have another Pod that consumes this Service by name you would get +something like: ```shell -u@pod$ wget -O- hostnames +wget -O- hostnames +``` +```none Resolving hostnames (hostnames)... failed: Name or service not known. wget: unable to resolve host address 'hostnames' ``` -So the first thing to check is whether that `Service` actually exists: +The first thing to check is whether that Service actually exists: ```shell kubectl get svc hostnames +``` +```none No resources found. Error from server (NotFound): services "hostnames" not found ``` -So we have a culprit, let's create the `Service`. As before, this is for the -walk-through - you can use your own `Service`'s details here. +Let's create the Service. As before, this is for the walk-through - you can +use your own Service's details here. ```shell kubectl expose deployment hostnames --port=80 --target-port=9376 +``` +```none service/hostnames exposed ``` @@ -153,11 +178,16 @@ And read it back, just to be sure: ```shell kubectl get svc hostnames +``` +```none NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE hostnames ClusterIP 10.0.1.175 80/TCP 5s ``` -As before, this is the same as if you had started the `Service` with YAML: +Now you know that the Service exists. + +{{< note >}} +As before, this is the same as if you had started the Service with YAML: ```yaml apiVersion: v1 @@ -166,7 +196,7 @@ metadata: name: hostnames spec: selector: - app: hostnames + run: hostnames ports: - name: default protocol: TCP @@ -174,25 +204,35 @@ spec: targetPort: 9376 ``` -Now you can confirm that the `Service` exists. +In order to highlight the full range of configuration, the Service you created +here uses a different port number than the Pods. For many real-world +Services, these values might be the same. +{{< /note >}} -## Does the Service work by DNS? +## Does the Service work by DNS name? -From a `Pod` in the same `Namespace`: +One of the most common ways that clients consume a Service is through a DNS +name. + +From a Pod in the same Namespace: ```shell -u@pod$ nslookup hostnames +nslookup hostnames +``` +```none Address 1: 10.0.0.10 kube-dns.kube-system.svc.cluster.local Name: hostnames Address 1: 10.0.1.175 hostnames.default.svc.cluster.local ``` -If this fails, perhaps your `Pod` and `Service` are in different -`Namespaces`, try a namespace-qualified name: +If this fails, perhaps your Pod and Service are in different +Namespaces, try a namespace-qualified name (again, from within a Pod): ```shell -u@pod$ nslookup hostnames.default +nslookup hostnames.default +``` +```none Address 1: 10.0.0.10 kube-dns.kube-system.svc.cluster.local Name: hostnames.default @@ -200,11 +240,13 @@ Address 1: 10.0.1.175 hostnames.default.svc.cluster.local ``` If this works, you'll need to adjust your app to use a cross-namespace name, or -run your app and `Service` in the same `Namespace`. If this still fails, try a +run your app and Service in the same Namespace. If this still fails, try a fully-qualified name: ```shell -u@pod$ nslookup hostnames.default.svc.cluster.local +nslookup hostnames.default.svc.cluster.local +``` +```none Address 1: 10.0.0.10 kube-dns.kube-system.svc.cluster.local Name: hostnames.default.svc.cluster.local @@ -212,18 +254,20 @@ Address 1: 10.0.1.175 hostnames.default.svc.cluster.local ``` Note the suffix here: "default.svc.cluster.local". The "default" is the -`Namespace` we're operating in. The "svc" denotes that this is a `Service`. +Namespace you're operating in. The "svc" denotes that this is a Service. The "cluster.local" is your cluster domain, which COULD be different in your own cluster. -You can also try this from a `Node` in the cluster: +You can also try this from a Node in the cluster: {{< note >}} -10.0.0.10 is my DNS `Service`, yours might be different. +10.0.0.10 is the cluster's DNS Service IP, yours might be different. {{< /note >}} ```shell -u@node$ nslookup hostnames.default.svc.cluster.local 10.0.0.10 +nslookup hostnames.default.svc.cluster.local 10.0.0.10 +``` +```none Server: 10.0.0.10 Address: 10.0.0.10#53 @@ -232,39 +276,49 @@ Address: 10.0.1.175 ``` If you are able to do a fully-qualified name lookup but not a relative one, you -need to check that your `/etc/resolv.conf` file is correct. +need to check that your `/etc/resolv.conf` file in your Pod is correct. From +within a Pod: ```shell -u@pod$ cat /etc/resolv.conf +cat /etc/resolv.conf +``` + +You should see something like: + +``` nameserver 10.0.0.10 search default.svc.cluster.local svc.cluster.local cluster.local example.com options ndots:5 ``` -The `nameserver` line must indicate your cluster's DNS `Service`. This is +The `nameserver` line must indicate your cluster's DNS Service. This is passed into `kubelet` with the `--cluster-dns` flag. The `search` line must include an appropriate suffix for you to find the -`Service` name. In this case it is looking for `Services` in the local -`Namespace` (`default.svc.cluster.local`), `Services` in all `Namespaces` -(`svc.cluster.local`), and the cluster (`cluster.local`). Depending on your own -install you might have additional records after that (up to 6 total). The -cluster suffix is passed into `kubelet` with the `--cluster-domain` flag. We -assume that is "cluster.local" in this document, but yours might be different, -in which case you should change that in all of the commands above. +Service name. In this case it is looking for Services in the local +Namespace ("default.svc.cluster.local"), Services in all Namespaces +("svc.cluster.local"), and lastly for names in the cluster ("cluster.local"). +Depending on your own install you might have additional records after that (up +to 6 total). The cluster suffix is passed into `kubelet` with the +`--cluster-domain` flag. Throughout this document, the cluster suffix is +assumed to be "cluster.local". Your own clusters might be configured +differently, in which case you should change that in all of the previous +commands. The `options` line must set `ndots` high enough that your DNS client library considers search paths at all. Kubernetes sets this to 5 by default, which is high enough to cover all of the DNS names it generates. -### Does any Service exist in DNS? +### Does any Service work by DNS name? {#does-any-service-exist-in-dns} -If the above still fails - DNS lookups are not working for your `Service` - we +If the above still fails, DNS lookups are not working for your Service. You can take a step back and see what else is not working. The Kubernetes master -`Service` should always work: +Service should always work. From within a Pod: ```shell -u@pod$ nslookup kubernetes.default +nslookup kubernetes.default +``` +```none Server: 10.0.0.10 Address 1: 10.0.0.10 kube-dns.kube-system.svc.cluster.local @@ -272,34 +326,37 @@ Name: kubernetes.default Address 1: 10.0.0.1 kubernetes.default.svc.cluster.local ``` -If this fails, you might need to go to the kube-proxy section of this doc, or -even go back to the top of this document and start over, but instead of -debugging your own `Service`, debug DNS. +If this fails, please see the [kube-proxy](#is-the-kube-proxy-working) section +of this document, or even go back to the top of this document and start over, +but instead of debugging your own Service, debug the DNS Service. ## Does the Service work by IP? -Assuming we can confirm that DNS works, the next thing to test is whether your -`Service` works at all. From a node in your cluster, access the `Service`'s -IP (from `kubectl get` above). +Assuming you have confirmed that DNS works, the next thing to test is whether your +Service works by its IP address. From a Pod in your cluster, access the +Service's IP (from `kubectl get` above). ```shell -u@node$ curl 10.0.1.175:80 -hostnames-0uton - -u@node$ curl 10.0.1.175:80 -hostnames-yp2kp - -u@node$ curl 10.0.1.175:80 -hostnames-bvc05 +for i in $(seq 1 3); do + wget -qO- 10.0.1.175:80 +done ``` -If your `Service` is working, you should get correct responses. If not, there +This should produce something like: + +``` +hostnames-0uton +hostnames-bvc05 +hostnames-yp2kp +``` + +If your Service is working, you should get correct responses. If not, there are a number of things that could be going wrong. Read on. -## Is the Service correct? +## Is the Service defined correctly? It might sound silly, but you should really double and triple check that your -`Service` is correct and matches your `Pod`'s port. Read back your `Service` +Service is correct and matches your Pod's port. Read back your Service and verify it: ```shell @@ -316,7 +373,7 @@ kubectl get service hostnames -o json "resourceVersion": "347189", "creationTimestamp": "2015-07-07T15:24:29Z", "labels": { - "app": "hostnames" + "run": "hostnames" } }, "spec": { @@ -330,7 +387,7 @@ kubectl get service hostnames -o json } ], "selector": { - "app": "hostnames" + "run": "hostnames" }, "clusterIP": "10.0.1.175", "type": "ClusterIP", @@ -342,110 +399,116 @@ kubectl get service hostnames -o json } ``` -* Is the port you are trying to access in `spec.ports[]`? -* Is the `targetPort` correct for your `Pods` (many `Pods` choose to use a different port than the `Service`)? -* If you meant it to be a numeric port, is it a number (9376) or a -string "9376"? -* If you meant it to be a named port, do your `Pods` expose a port -with the same name? -* Is the port's `protocol` the same as the `Pod`'s? +* Is the Service port you are trying to access listed in `spec.ports[]`? +* Is the `targetPort` correct for your Pods (some Pods use a different port than the Service)? +* If you meant to use a numeric port, is it a number (9376) or a string "9376"? +* If you meant to use a named port, do your Pods expose a port with the same name? +* Is the port's `protocol` correct for your Pods? ## Does the Service have any Endpoints? -If you got this far, we assume that you have confirmed that your `Service` -exists and is resolved by DNS. Now let's check that the `Pods` you ran are -actually being selected by the `Service`. +If you got this far, you have confirmed that your Service is correctly +defined and is resolved by DNS. Now let's check that the Pods you ran are +actually being selected by the Service. -Earlier we saw that the `Pods` were running. We can re-check that: +Earlier you saw that the Pods were running. You can re-check that: ```shell -kubectl get pods -l app=hostnames +kubectl get pods -l run=hostnames +``` +```none NAME READY STATUS RESTARTS AGE hostnames-0uton 1/1 Running 0 1h hostnames-bvc05 1/1 Running 0 1h hostnames-yp2kp 1/1 Running 0 1h ``` -The "AGE" column says that these `Pods` are about an hour old, which implies that +The `-l run=hostnames` argument is a label selector - just like our Service +has. + +The "AGE" column says that these Pods are about an hour old, which implies that they are running fine and not crashing. -The `-l app=hostnames` argument is a label selector - just like our `Service` -has. Inside the Kubernetes system is a control loop which evaluates the -selector of every `Service` and saves the results into an `Endpoints` object. +The "RESTARTS" column says that these pods are not crashing frequently or being +restarted. Frequent restarts could lead to intermittent connectivity issues. +If the restart count is high, read more about how to [debug pods](/docs/tasks/debug-application-cluster/debug-pod-replication-controller/#debugging-pods). + +Inside the Kubernetes system is a control loop which evaluates the selector of +every Service and saves the results into a corresponding Endpoints object. ```shell kubectl get endpoints hostnames + NAME ENDPOINTS hostnames 10.244.0.5:9376,10.244.0.6:9376,10.244.0.7:9376 ``` -This confirms that the endpoints controller has found the correct `Pods` for -your `Service`. If the `hostnames` row is blank, you should check that the -`spec.selector` field of your `Service` actually selects for `metadata.labels` -values on your `Pods`. A common mistake is to have a typo or other error, such -as the `Service` selecting for `run=hostnames`, but the `Deployment` specifying -`app=hostnames`. +This confirms that the endpoints controller has found the correct Pods for +your Service. If the `ENDPOINTS` column is ``, you should check that +the `spec.selector` field of your Service actually selects for +`metadata.labels` values on your Pods. A common mistake is to have a typo or +other error, such as the Service selecting for `app=hostnames`, but the +Deployment specifying `run=hostnames`. ## Are the Pods working? -At this point, we know that your `Service` exists and has selected your `Pods`. -Let's check that the `Pods` are actually working - we can bypass the `Service` -mechanism and go straight to the `Pods`. +At this point, you know that your Service exists and has selected your Pods. +At the beginning of this walk-through, you verified the Pods themselves. +Let's check again that the Pods are actually working - you can bypass the +Service mechanism and go straight to the Pods, as listed by the Endpoints +above. {{< note >}} -These commands use the `Pod` port (9376), rather than the `Service` port (80). +These commands use the Pod port (9376), rather than the Service port (80). {{< /note >}} +From within a Pod: + ```shell -u@pod$ wget -qO- 10.244.0.5:9376 +for ep in 10.244.0.5:9376 10.244.0.6:9376 10.244.0.7:9376; do + wget -qO- $ep +done +``` + +This should produce something like: + +``` hostnames-0uton - -pod $ wget -qO- 10.244.0.6:9376 hostnames-bvc05 - -u@pod$ wget -qO- 10.244.0.7:9376 hostnames-yp2kp ``` -We expect each `Pod` in the `Endpoints` list to return its own hostname. If +You expect each Pod in the Endpoints list to return its own hostname. If this is not what happens (or whatever the correct behavior is for your own -`Pods`), you should investigate what's happening there. You might find -`kubectl logs` to be useful or `kubectl exec` directly to your `Pods` and check -service from there. - -Another thing to check is that your `Pods` are not crashing or being restarted. -Frequent restarts could lead to intermittent connectivity issues. - -```shell -kubectl get pods -l app=hostnames -NAME READY STATUS RESTARTS AGE -hostnames-632524106-bbpiw 1/1 Running 0 2m -hostnames-632524106-ly40y 1/1 Running 0 2m -hostnames-632524106-tlaok 1/1 Running 0 2m -``` - -If the restart count is high, read more about how to [debug -pods](/docs/tasks/debug-application-cluster/debug-pod-replication-controller/#debugging-pods). +Pods), you should investigate what's happening there. ## Is the kube-proxy working? -If you get here, your `Service` is running, has `Endpoints`, and your `Pods` -are actually serving. At this point, the whole `Service` proxy mechanism is +If you get here, your Service is running, has Endpoints, and your Pods +are actually serving. At this point, the whole Service proxy mechanism is suspect. Let's confirm it, piece by piece. +The default implementation of Services, and the one used on most clusters, is +kube-proxy. This is a program that runs on every node and configures one of a +small set of mechanisms for providing the Service abstraction. If your +cluster does not use kube-proxy, the following sections will not apply, and you +will have to investigate whatever implementation of Services you are using. + ### Is kube-proxy running? -Confirm that `kube-proxy` is running on your `Nodes`. You should get something -like the below: +Confirm that `kube-proxy` is running on your Nodes. Running directly on a +Node, you should get something like the below: ```shell -u@node$ ps auxw | grep kube-proxy +ps auxw | grep kube-proxy +``` +```none root 4194 0.4 0.1 101864 17696 ? Sl Jul04 25:43 /usr/local/bin/kube-proxy --master=https://kubernetes-master --kubeconfig=/var/lib/kube-proxy/kubeconfig --v=2 ``` Next, confirm that it is not failing something obvious, like contacting the master. To do this, you'll have to look at the logs. Accessing the logs -depends on your `Node` OS. On some OSes it is a file, such as +depends on your Node OS. On some OSes it is a file, such as /var/log/kube-proxy.log, while other OSes use `journalctl` to access logs. You should see something like: @@ -463,7 +526,7 @@ I1027 22:14:54.040223 5063 proxier.go:294] Adding new service "kube-system/ku ``` If you see error messages about not being able to contact the master, you -should double-check your `Node` configuration and installation steps. +should double-check your Node configuration and installation steps. One of the possible reasons that `kube-proxy` cannot run correctly is that the required `conntrack` binary cannot be found. This may happen on some Linux @@ -472,36 +535,19 @@ installing Kubernetes from scratch. If this is the case, you need to manually install the `conntrack` package (e.g. `sudo apt install conntrack` on Ubuntu) and then retry. -### Is kube-proxy writing iptables rules? +Kube-proxy can run in one of a few modes. In the log listed above, the +line `Using iptables Proxier` indicates that kube-proxy is running in +"iptables" mode. The most common other mode is "ipvs". The older "userspace" +mode has largely been replaced by these. -One of the main responsibilities of `kube-proxy` is to write the `iptables` -rules which implement `Services`. Let's check that those rules are getting -written. +#### Iptables mode -The kube-proxy can run in "userspace" mode, "iptables" mode or "ipvs" mode. -Hopefully you are using the "iptables" mode or "ipvs" mode. You -should see one of the following cases. - -#### Userspace +In "iptables" mode, you should see something like the following on a Node: ```shell -u@node$ iptables-save | grep hostnames --A KUBE-PORTALS-CONTAINER -d 10.0.1.175/32 -p tcp -m comment --comment "default/hostnames:default" -m tcp --dport 80 -j REDIRECT --to-ports 48577 --A KUBE-PORTALS-HOST -d 10.0.1.175/32 -p tcp -m comment --comment "default/hostnames:default" -m tcp --dport 80 -j DNAT --to-destination 10.240.115.247:48577 +iptables-save | grep hostnames ``` - -There should be 2 rules for each port on your `Service` (just one in this -example) - a "KUBE-PORTALS-CONTAINER" and a "KUBE-PORTALS-HOST". If you do -not see these, try restarting `kube-proxy` with the `-v` flag set to 4, and -then look at the logs again. - -Almost nobody should be using the "userspace" mode any more, so we won't spend -more time on it here. - -#### Iptables - -```shell -u@node$ iptables-save | grep hostnames +```none -A KUBE-SEP-57KPRZ3JQVENLNBR -s 10.244.3.6/32 -m comment --comment "default/hostnames:" -j MARK --set-xmark 0x00004000/0x00004000 -A KUBE-SEP-57KPRZ3JQVENLNBR -p tcp -m comment --comment "default/hostnames:" -m tcp -j DNAT --to-destination 10.244.3.6:9376 -A KUBE-SEP-WNBA2IHDGP2BOBGZ -s 10.244.1.7/32 -m comment --comment "default/hostnames:" -j MARK --set-xmark 0x00004000/0x00004000 @@ -514,15 +560,20 @@ u@node$ iptables-save | grep hostnames -A KUBE-SVC-NWV5X2332I4OT4T3 -m comment --comment "default/hostnames:" -j KUBE-SEP-57KPRZ3JQVENLNBR ``` -There should be 1 rule in `KUBE-SERVICES`, 1 or 2 rules per endpoint in -`KUBE-SVC-(hash)` (depending on `SessionAffinity`), one `KUBE-SEP-(hash)` chain -per endpoint, and a few rules in each `KUBE-SEP-(hash)` chain. The exact rules -will vary based on your exact config (including node-ports and load-balancers). +For each port of each Service, there should be 1 rule in `KUBE-SERVICES` and +one `KUBE-SVC-` chain. For each Pod endpoint, there should be a small +number of rules in that `KUBE-SVC-` and one `KUBE-SEP-` chain with +a small number of rules in it. The exact rules will vary based on your exact +config (including node-ports and load-balancers). -#### IPVS +#### IPVS mode + +In "ipvs" mode, you should see something like the following on a Node: ```shell -u@node$ ipvsadm -ln +ipvsadm -ln +``` +```none Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn ... @@ -533,14 +584,39 @@ TCP 10.0.1.175:80 rr ... ``` -IPVS proxy will create a virtual server for each service address(e.g. Cluster IP, External IP, NodePort IP, Load Balancer IP etc.) and some corresponding real servers for endpoints of the service, if any. In this example, service hostnames(`10.0.1.175:80`) has 3 endpoints(`10.244.0.5:9376`, `10.244.0.6:9376`, `10.244.0.7:9376`) and you'll get results similar to above. +For each port of each Service, plus any NodePorts, external IPs, and +load-balancer IPs, kube-proxy will create a virtual server. For each Pod +endpoint, it will create corresponding real servers. In this example, service +hostnames(`10.0.1.175:80`) has 3 endpoints(`10.244.0.5:9376`, +`10.244.0.6:9376`, `10.244.0.7:9376`). + +#### Userspace mode + +In rare cases, you may be using "userspace" mode. From your Node: + +```shell +iptables-save | grep hostnames +``` +```none +-A KUBE-PORTALS-CONTAINER -d 10.0.1.175/32 -p tcp -m comment --comment "default/hostnames:default" -m tcp --dport 80 -j REDIRECT --to-ports 48577 +-A KUBE-PORTALS-HOST -d 10.0.1.175/32 -p tcp -m comment --comment "default/hostnames:default" -m tcp --dport 80 -j DNAT --to-destination 10.240.115.247:48577 +``` + +There should be 2 rules for each port of your Service (just one in this +example) - a "KUBE-PORTALS-CONTAINER" and a "KUBE-PORTALS-HOST". + +Almost nobody should be using the "userspace" mode any more, so you won't spend +more time on it here. ### Is kube-proxy proxying? -Assuming you do see the above rules, try again to access your `Service` by IP: +Assuming you do see one the above cases, try again to access your Service by +IP from one of your Nodes: ```shell -u@node$ curl 10.0.1.175:80 +curl 10.0.1.175:80 +``` +```none hostnames-0uton ``` @@ -548,31 +624,36 @@ If this fails and you are using the userspace proxy, you can try accessing the proxy directly. If you are using the iptables proxy, skip this section. Look back at the `iptables-save` output above, and extract the -port number that `kube-proxy` is using for your `Service`. In the above +port number that `kube-proxy` is using for your Service. In the above examples it is "48577". Now connect to that: ```shell -u@node$ curl localhost:48577 +curl localhost:48577 +``` +```none hostnames-yp2kp ``` If this still fails, look at the `kube-proxy` logs for specific lines like: -```shell +```none Setting endpoints for default/hostnames:default to [10.244.0.5:9376 10.244.0.6:9376 10.244.0.7:9376] ``` If you don't see those, try restarting `kube-proxy` with the `-v` flag set to 4, and then look at the logs again. -### A Pod cannot reach itself via Service IP +### Edge case: A Pod fails to reach itself via the Service IP {#a-pod-fails-to-reach-itself-via-the-service-ip} + +This might sound unlikely, but it does happen and it is supposed to work. This can happen when the network is not properly configured for "hairpin" traffic, usually when `kube-proxy` is running in `iptables` mode and Pods are connected with bridge network. The `Kubelet` exposes a `hairpin-mode` -[flag](/docs/admin/kubelet/) that allows endpoints of a Service to loadbalance back to themselves -if they try to access their own Service VIP. The `hairpin-mode` flag must either be -set to `hairpin-veth` or `promiscuous-bridge`. +[flag](/docs/admin/kubelet/) that allows endpoints of a Service to loadbalance +back to themselves if they try to access their own Service VIP. The +`hairpin-mode` flag must either be set to `hairpin-veth` or +`promiscuous-bridge`. The common steps to trouble shoot this are as follows: @@ -581,9 +662,10 @@ You should see something like the below. `hairpin-mode` is set to `promiscuous-bridge` in the following example. ```shell -u@node$ ps auxw|grep kubelet +ps auxw | grep kubelet +``` +```none root 3392 1.1 0.8 186804 65208 ? Sl 00:51 11:11 /usr/local/bin/kubelet --enable-debugging-handlers=true --config=/etc/kubernetes/manifests --allow-privileged=True --v=4 --cluster-dns=10.0.0.10 --cluster-domain=cluster.local --configure-cbr0=true --cgroup-root=/ --system-cgroups=/system --hairpin-mode=promiscuous-bridge --runtime-cgroups=/docker-daemon --kubelet-cgroups=/kubelet --babysit-daemons=true --max-pods=110 --serialize-image-pulls=false --outofdisk-transition-frequency=0 - ``` * Confirm the effective `hairpin-mode`. To do this, you'll have to look at @@ -594,7 +676,7 @@ match `--hairpin-mode` flag due to compatibility. Check if there is any log lines with key word `hairpin` in kubelet.log. There should be log lines indicating the effective hairpin mode, like something below. -```shell +```none I0629 00:51:43.648698 3252 kubelet.go:380] Hairpin mode set to "promiscuous-bridge" ``` @@ -604,6 +686,8 @@ you should see something like: ```shell for intf in /sys/devices/virtual/net/cbr0/brif/*; do cat $intf/hairpin_mode; done +``` +```none 1 1 1 @@ -615,20 +699,21 @@ has the permission to manipulate linux bridge on node. If `cbr0` bridge is used and configured properly, you should see: ```shell -u@node$ ifconfig cbr0 |grep PROMISC +ifconfig cbr0 |grep PROMISC +``` +```none UP BROADCAST RUNNING PROMISC MULTICAST MTU:1460 Metric:1 - ``` * Seek help if none of above works out. ## Seek help -If you get this far, something very strange is happening. Your `Service` is -running, has `Endpoints`, and your `Pods` are actually serving. You have DNS -working, `iptables` rules installed, and `kube-proxy` does not seem to be -misbehaving. And yet your `Service` is not working. You should probably let -us know, so we can help investigate! +If you get this far, something very strange is happening. Your Service is +running, has Endpoints, and your Pods are actually serving. You have DNS +working, and `kube-proxy` does not seem to be misbehaving. And yet your +Service is not working. Please let us know what is going on, so we can help +investigate! Contact us on [Slack](/docs/troubleshooting/#slack) or From 4d09c4efac1266f462e173c1fd2ece1b020d4d02 Mon Sep 17 00:00:00 2001 From: prameshj Date: Tue, 10 Mar 2020 16:31:36 -0700 Subject: [PATCH 025/140] Update nodelocaldns.md with config steps. (#18716) Remove references to 169.254.20.10 as the default. Update content/en/docs/tasks/administer-cluster/nodelocaldns.md Co-Authored-By: Tim Bannister --- .../tasks/administer-cluster/nodelocaldns.md | 41 +++++++++++++++---- 1 file changed, 34 insertions(+), 7 deletions(-) diff --git a/content/en/docs/tasks/administer-cluster/nodelocaldns.md b/content/en/docs/tasks/administer-cluster/nodelocaldns.md index 7d15596112..f23525e377 100644 --- a/content/en/docs/tasks/administer-cluster/nodelocaldns.md +++ b/content/en/docs/tasks/administer-cluster/nodelocaldns.md @@ -2,6 +2,7 @@ reviewers: - bowei - zihongz +- sftim title: Using NodeLocal DNSCache in Kubernetes clusters content_template: templates/task --- @@ -47,18 +48,44 @@ This is the path followed by DNS Queries after NodeLocal DNSCache is enabled: {{< figure src="/images/docs/nodelocaldns.jpg" alt="NodeLocal DNSCache flow" title="Nodelocal DNSCache flow" caption="This image shows how NodeLocal DNSCache handles DNS queries." >}} ## Configuration +{{< note >}} The local listen IP address for NodeLocal DNSCache can be any IP in the 169.254.20.0/16 space or any other IP address that can be guaranteed to not collide with any existing IP. This document uses 169.254.20.10 as an example. +{{< /note >}} -This feature can be enabled using the command: +This feature can be enabled using the following steps: -`KUBE_ENABLE_NODELOCAL_DNS=true kubetest --up` +* Prepare a manifest similar to the sample [`nodelocaldns.yaml`](https://github.com/kubernetes/kubernetes/blob/master/cluster/addons/dns/nodelocaldns/nodelocaldns.yaml) and save it as `nodelocaldns.yaml.` +* Substitute the variables in the manifest with the right values: -This works for e2e clusters created on GCE. On all other environments, the following steps will setup NodeLocal DNSCache: + * kubedns=`kubectl get svc kube-dns -n kube-system -o jsonpath={.spec.clusterIP}` + + * domain=`` + + * localdns=`` + + `` is "cluster.local" by default. `` is the local listen IP address chosen for NodeLocal DNSCache. -* A yaml similar to [this](https://github.com/kubernetes/kubernetes/blob/master/cluster/addons/dns/nodelocaldns/nodelocaldns.yaml) can be applied using `kubectl create -f` command. -* No need to modify the --cluster-dns flag since NodeLocal DNSCache listens on both the kube-dns service IP as well as a link-local IP (169.254.20.10 by default) + * If kube-proxy is running in IPTABLES mode: + + ``` bash + sed -i "s/__PILLAR__LOCAL__DNS__/$localdns/g; s/__PILLAR__DNS__DOMAIN__/$domain/g; s/__PILLAR__DNS__SERVER__/$kubedns/g" nodelocaldns.yaml + ``` + + `__PILLAR__CLUSTER__DNS__` and `__PILLAR__UPSTREAM__SERVERS__` will be populated by the node-local-dns pods. + In this mode, node-local-dns pods listen on both the kube-dns service IP as well as ``, so pods can lookup DNS records using either IP address. + + * If kube-proxy is running in IPVS mode: + + ``` bash + sed -i "s/__PILLAR__LOCAL__DNS__/$localdns/g; s/__PILLAR__DNS__DOMAIN__/$domain/g; s/__PILLAR__DNS__SERVER__//g; s/__PILLAR__CLUSTER__DNS__/$kubedns/g" nodelocaldns.yaml + ``` + In this mode, node-local-dns pods listen only on ``. The node-local-dns interface cannot bind the kube-dns cluster IP since the interface used for IPVS loadbalancing already uses this address. + `__PILLAR__UPSTREAM__SERVERS__` will be populated by the node-local-dns pods. + +* Run `kubectl create -f nodelocaldns.yaml` +* If using kube-proxy in IPVS mode, `--cluster-dns` flag to kubelet needs to be modified to use `` that NodeLocal DNSCache is listening on. + Otherwise, there is no need to modify the value of the `--cluster-dns` flag, since NodeLocal DNSCache listens on both the kube-dns service IP as well as ``. Once enabled, node-local-dns Pods will run in the kube-system namespace on each of the cluster nodes. This Pod runs [CoreDNS](https://github.com/coredns/coredns) in cache mode, so all CoreDNS metrics exposed by the different plugins will be available on a per-node basis. -The feature can be disabled by removing the daemonset, using `kubectl delete -f` command. On e2e clusters created on GCE, the daemonset can be removed by deleting the node-local-dns yaml from `/etc/kubernetes/addons/0-dns/nodelocaldns.yaml` - +You can disable this feature by removing the DaemonSet, using `kubectl delete -f ` . You should also revert any changes you made to the kubelet configuration. {{% /capture %}} From 20da6a930999fcf83aaca0630e91f07bb542ceec Mon Sep 17 00:00:00 2001 From: Xing Yang Date: Tue, 10 Mar 2020 21:41:36 -0400 Subject: [PATCH 026/140] Change restore volume from snapshot feature status to beta (#19484) --- content/en/docs/concepts/storage/persistent-volumes.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/content/en/docs/concepts/storage/persistent-volumes.md b/content/en/docs/concepts/storage/persistent-volumes.md index 5ddd72d251..a4451f2867 100644 --- a/content/en/docs/concepts/storage/persistent-volumes.md +++ b/content/en/docs/concepts/storage/persistent-volumes.md @@ -4,6 +4,7 @@ reviewers: - saad-ali - thockin - msau42 +- xing-yang title: Persistent Volumes feature: title: Storage orchestration @@ -663,7 +664,7 @@ Only statically provisioned volumes are supported for alpha release. Administrat ## Volume Snapshot and Restore Volume from Snapshot Support -{{< feature-state for_k8s_version="v1.12" state="alpha" >}} +{{< feature-state for_k8s_version="v1.17" state="beta" >}} Volume snapshot feature was added to support CSI Volume Plugins only. For details, see [volume snapshots](/docs/concepts/storage/volume-snapshots/). From cbd3f895107b5201b9053aa80f62c69aed6f74c1 Mon Sep 17 00:00:00 2001 From: 2BFL Date: Wed, 11 Mar 2020 11:11:35 +0800 Subject: [PATCH 027/140] zh-translation: feature-gates.md (#19571) --- .../feature-gates.md | 377 +++++++++++------- 1 file changed, 242 insertions(+), 135 deletions(-) diff --git a/content/zh/docs/reference/command-line-tools-reference/feature-gates.md b/content/zh/docs/reference/command-line-tools-reference/feature-gates.md index dae2db093b..945c5dac7e 100644 --- a/content/zh/docs/reference/command-line-tools-reference/feature-gates.md +++ b/content/zh/docs/reference/command-line-tools-reference/feature-gates.md @@ -1,46 +1,57 @@ +--- +weight: 10 +title: 特性门控 +content_template: templates/concept +--- + ---- -title: 功能开关 -weight: 10 -title: 功能开关 -content_template: templates/concept ---- {{% capture overview %}} -本页详述了管理员可以在不同的 Kubernetes 组件上指定的各种功能开关。 + +本页详述了管理员可以在不同的 Kubernetes 组件上指定的各种特性门控。 + +关于特性各个阶段的说明，请参见[特性阶段](#feature-stages)。 + {{% /capture %}} {{% capture body %}} + + ## 概述 -功能开关是一组描述 alpha 或实验功能的键值对。 + +特性门控是描述 Kubernetes 特性的一组键值对。您可以在 Kubernetes 的每一个组件中使用 `--feature-gates` flag 来启用或禁用这些特性。 + -管理员可以在每个组件上使用 `--feature-gates` 命令行参数来打开或关闭功能。每个组件都支持该组件唯一的一组功能开关。 -使用 `-h` 参数来查看所有组件的完整功能开关。 -要为诸如 kubelet 之类的组件设置功能开关，请使用分配给功能键值对列表的 `--feature-gates` 参数： + +每个 Kubernetes 组件都支持启用或禁用与该组件相关的一组特性门控。 +使用 `-h` 参数来查看所有组件支持的完整特性门控。 +要为诸如 kubelet 之类的组件设置特性门控，请使用 `--feature-gates` 参数，并向其传递一组特性： ```shell --feature-gates="...,DynamicKubeletConfig=true" @@ -50,7 +61,8 @@ To set feature gates for a component, such as kubelet, use the `--feature-gates` The following tables are a summary of the feature gates that you can set on different Kubernetes components. --> -下表总结了可以在不同的 Kubernetes 组件上设置的功能开关。 + +下表总结了在不同的 Kubernetes 组件上可以设置的特性门控。 -- 引入功能或更改其发布阶段后，"Since" 列将包含 Kubernetes 版本。 -- "Until" 列（如果不为空）包含最后一个 Kubernetes 版本，您仍可以在其中使用功能开关。 -- 如果某个功能处于 Alpha 或 Beta 状态，则可以在 Alpha 或者 Beta 功能开关表中找到该功能。 -- 如果某个功能稳定（GA）或已弃用，则可以在 GA 或者不推荐使用的功能开关表中找到该功能的所有阶段。 + +- 引入特性或更改其发布阶段后，"Since" 列将包含 Kubernetes 版本。 +- "Until" 列（如果不为空）包含最后一个 Kubernetes 版本，您仍可以在其中使用特性门控。 +- 如果某个特性处于 Alpha 或 Beta 状态，您可以在 [Alpha 和 Beta 特性门控表](#feature-gates-for-alpha-or-beta-features)中找到该特性。 +- 如果某个特性处于稳定状态，您可以在[毕业和废弃特性门控表](#feature-gates-for-graduated-or-deprecated-features).中找到该特性的所有阶段。 +- [毕业和废弃特性门控表](#feature-gates-for-graduated-or-deprecated-features) 还列出了废弃的和已被移除的特性。 -### Alpha 或 Beta 功能的功能开关 - -{{< table caption="Alpha 或 Beta 状态中功能的功能开关" >}} +### Alpha 和 Beta 的特性门控 -| 功能 | 默认值 | 状态 | 开始(Since) | 结束(Until) | +{{< table caption="处于 Alpha 或 Beta 状态的特性门控" >}} + +| 特性 | 默认值 | 状态 | 开始(Since) | 结束(Until) | |---------|---------|-------|-------|-------| | `APIListChunking` | `false` | Alpha | 1.8 | 1.8 | | `APIListChunking` | `true` | Beta | 1.9 | | +| `APIPriorityAndFairness` | `false` | Alpha | 1.17 | | | `APIResponseCompression` | `false` | Alpha | 1.7 | | | `AppArmor` | `true` | Beta | 1.4 | | -| `AttachVolumeLimit` | `true` | Alpha | 1.11 | 1.11 | -| `AttachVolumeLimit` | `true` | Beta | 1.12 | | | `BalanceAttachedNodeVolumes` | `false` | Alpha | 1.11 | | | `BlockVolume` | `false` | Alpha | 1.9 | 1.12 | | `BlockVolume` | `true` | Beta | 1.13 | - | @@ -102,14 +118,20 @@ Alpha 或 Beta 状态中功能的功能开关 | `CSIDriverRegistry` | `true` | Beta | 1.14 | | | `CSIInlineVolume` | `false` | Alpha | 1.15 | 1.15 | | `CSIInlineVolume` | `true` | Beta | 1.16 | - | -| `CSIMigration` | `false` | Alpha | 1.14 | | +| `CSIMigration` | `false` | Alpha | 1.14 | 1.16 | +| `CSIMigration` | `true` | Beta | 1.17 | | | `CSIMigrationAWS` | `false` | Alpha | 1.14 | | +| `CSIMigrationAWS` | `false` | Beta | 1.17 | | +| `CSIMigrationAWSComplete` | `false` | Alpha | 1.17 | | | `CSIMigrationAzureDisk` | `false` | Alpha | 1.15 | | +| `CSIMigrationAzureDiskComplete` | `false` | Alpha | 1.17 | | | `CSIMigrationAzureFile` | `false` | Alpha | 1.15 | | -| `CSIMigrationGCE` | `false` | Alpha | 1.14 | | +| `CSIMigrationAzureFileComplete` | `false` | Alpha | 1.17 | | +| `CSIMigrationGCE` | `false` | Alpha | 1.14 | 1.16 | +| `CSIMigrationGCE` | `false` | Beta | 1.17 | | +| `CSIMigrationGCEComplete` | `false` | Alpha | 1.17 | | | `CSIMigrationOpenStack` | `false` | Alpha | 1.14 | | -| `CSINodeInfo` | `false` | Alpha | 1.12 | 1.13 | -| `CSINodeInfo` | `true` | Beta | 1.14 | | +| `CSIMigrationOpenStackComplete` | `false` | Alpha | 1.17 | | | `CustomCPUCFSQuotaPeriod` | `false` | Alpha | 1.12 | | | `CustomResourceDefaulting` | `false` | Alpha| 1.15 | 1.15 | | `CustomResourceDefaulting` | `true` | Beta | 1.16 | | @@ -120,7 +142,8 @@ Alpha 或 Beta 状态中功能的功能开关 | `DynamicAuditing` | `false` | Alpha | 1.13 | | | `DynamicKubeletConfig` | `false` | Alpha | 1.4 | 1.10 | | `DynamicKubeletConfig` | `true` | Beta | 1.11 | | -| `EndpointSlice` | `false` | Alpha | 1.16 | | +| `EndpointSlice` | `false` | Alpha | 1.16 | 1.16 | +| `EndpointSlice` | `false` | Beta | 1.17 | | | `EphemeralContainers` | `false` | Alpha | 1.16 | | | `ExpandCSIVolumes` | `false` | Alpha | 1.14 | 1.15 | | `ExpandCSIVolumes` | `true` | Beta | 1.16 | | @@ -140,31 +163,21 @@ Alpha 或 Beta 状态中功能的功能开关 | `LocalStorageCapacityIsolationFSQuotaMonitoring` | `false` | Alpha | 1.15 | | | `MountContainers` | `false` | Alpha | 1.9 | | | `NodeDisruptionExclusion` | `false` | Alpha | 1.16 | | -| `NodeLease` | `false` | Alpha | 1.12 | 1.13 | -| `NodeLease` | `true` | Beta | 1.14 | | | `NonPreemptingPriority` | `false` | Alpha | 1.15 | | | `PodOverhead` | `false` | Alpha | 1.16 | - | -| `PodShareProcessNamespace` | `false` | Alpha | 1.10 | 1.11 | -| `PodShareProcessNamespace` | `true` | Beta | 1.12 | | | `ProcMountType` | `false` | Alpha | 1.12 | | | `QOSReserved` | `false` | Alpha | 1.11 | | | `RemainingItemCount` | `false` | Alpha | 1.15 | | -| `RequestManagement` | `false` | Alpha | 1.15 | | | `ResourceLimitsPriorityFunction` | `false` | Alpha | 1.9 | | -| `ResourceQuotaScopeSelectors` | `false` | Alpha | 1.11 | 1.11 | -| `ResourceQuotaScopeSelectors` | `true` | Beta | 1.12 | | | `RotateKubeletClientCertificate` | `true` | Beta | 1.8 | | | `RotateKubeletServerCertificate` | `false` | Alpha | 1.7 | 1.11 | | `RotateKubeletServerCertificate` | `true` | Beta | 1.12 | | | `RunAsGroup` | `true` | Beta | 1.14 | | | `RuntimeClass` | `false` | Alpha | 1.12 | 1.13 | | `RuntimeClass` | `true` | Beta | 1.14 | | -| `ScheduleDaemonSetPods` | `false` | Alpha | 1.11 | 1.11 | -| `ScheduleDaemonSetPods` | `true` | Beta | 1.12 | | | `SCTPSupport` | `false` | Alpha | 1.12 | | | `ServerSideApply` | `false` | Alpha | 1.14 | 1.15 | | `ServerSideApply` | `true` | Beta | 1.16 | | -| `ServiceLoadBalancerFinalizer` | `false` | Alpha | 1.15 | | | `ServiceNodeExclusion` | `false` | Alpha | 1.8 | | | `StartupProbe` | `false` | Alpha | 1.16 | | | `StorageVersionHash` | `false` | Alpha | 1.14 | 1.14 | @@ -178,43 +191,40 @@ Alpha 或 Beta 状态中功能的功能开关 | `Sysctls` | `true` | Beta | 1.11 | | | `TaintBasedEvictions` | `false` | Alpha | 1.6 | 1.12 | | `TaintBasedEvictions` | `true` | Beta | 1.13 | | -| `TaintNodesByCondition` | `false` | Alpha | 1.8 | 1.11 | -| `TaintNodesByCondition` | `true` | Beta | 1.12 | | | `TokenRequest` | `false` | Alpha | 1.10 | 1.11 | | `TokenRequest` | `true` | Beta | 1.12 | | | `TokenRequestProjection` | `false` | Alpha | 1.11 | 1.11 | | `TokenRequestProjection` | `true` | Beta | 1.12 | | | `TTLAfterFinished` | `false` | Alpha | 1.12 | | | `TopologyManager` | `false` | Alpha | 1.16 | | -| `ValidateProxyRedirects` | `false` | Alpha | 1.10 | 1.13 | +| `ValidateProxyRedirects` | `false` | Alpha | 1.12 | 1.13 | | `ValidateProxyRedirects` | `true` | Beta | 1.14 | | | `VolumePVCDataSource` | `false` | Alpha | 1.15 | 1.15 | | `VolumePVCDataSource` | `true` | Beta | 1.16 | | -| `VolumeSubpathEnvExpansion` | `false` | Alpha | 1.14 | 1.14 | -| `VolumeSubpathEnvExpansion` | `true` | Beta | 1.15 | | -| `VolumeSnapshotDataSource` | `false` | Alpha | 1.12 | - | -| `WatchBookmark` | `false` | Alpha | 1.15 | 1.15 | -| `WatchBookmark` | `true` | Beta | 1.16 | | +| `VolumeSnapshotDataSource` | `false` | Alpha | 1.12 | 1.16 | +| `VolumeSnapshotDataSource` | `true` | Beta | 1.17 | - | | `WindowsGMSA` | `false` | Alpha | 1.14 | | | `WindowsGMSA` | `true` | Beta | 1.16 | | | `WinDSR` | `false` | Alpha | 1.14 | | | `WinOverlay` | `false` | Alpha | 1.14 | | + {{< /table >}} -### 已完成或不推荐使用的功能的功能开关 - -{{< table caption="已完成或不推荐使用的功能的功能开关" >}} +### 已毕业和不推荐使用的特性门控 -| 功能 | 默认值 | 状态 | 开始(Since) | 结束(Until) | +{{< table caption="已毕业或不推荐使用的特性门控" >}} + +| 特性 | 默认值 | 状态 | 开始(Since) | 结束(Until) | |---------|---------|-------|-------|-------| | `Accelerators` | `false` | Alpha | 1.6 | 1.10 | | `Accelerators` | - | Deprecated | 1.11 | - | @@ -225,6 +235,12 @@ Feature Gates for Graduated or Deprecated Features | `AffinityInAnnotations` | - | Deprecated | 1.8 | - | | `AllowExtTrafficLocalEndpoints` | `false` | Beta | 1.4 | 1.6 | | `AllowExtTrafficLocalEndpoints` | `true` | GA | 1.7 | - | +| `CSINodeInfo` | `false` | Alpha | 1.12 | 1.13 | +| `CSINodeInfo` | `true` | Beta | 1.14 | 1.16 | +| `CSINodeInfo` | `true` | GA | 1.17 | | +| `AttachVolumeLimit` | `false` | Alpha | 1.11 | 1.11 | +| `AttachVolumeLimit` | `true` | Beta | 1.12 | 1.16 | +| `AttachVolumeLimit` | `true` | GA | 1.17 | - | | `CSIPersistentVolume` | `false` | Alpha | 1.9 | 1.9 | | `CSIPersistentVolume` | `true` | Beta | 1.10 | 1.12 | | `CSIPersistentVolume` | `true` | GA | 1.13 | - | @@ -266,6 +282,9 @@ Feature Gates for Graduated or Deprecated Features | `MountPropagation` | `false` | Alpha | 1.8 | 1.9 | | `MountPropagation` | `true` | Beta | 1.10 | 1.11 | | `MountPropagation` | `true` | GA | 1.12 | - | +| `NodeLease` | `false` | Alpha | 1.12 | 1.13 | +| `NodeLease` | `true` | Beta | 1.14 | 1.16 | +| `NodeLease` | `true` | GA | 1.17 | - | | `PersistentLocalVolumes` | `false` | Alpha | 1.7 | 1.9 | | `PersistentLocalVolumes` | `true` | Beta | 1.10 | 1.13 | | `PersistentLocalVolumes` | `true` | GA | 1.14 | - | @@ -275,39 +294,61 @@ Feature Gates for Graduated or Deprecated Features | `PodReadinessGates` | `false` | Alpha | 1.11 | 1.11 | | `PodReadinessGates` | `true` | Beta | 1.12 | 1.13 | | `PodReadinessGates` | `true` | GA | 1.14 | - | +| `PodShareProcessNamespace` | `false` | Alpha | 1.10 | 1.11 | +| `PodShareProcessNamespace` | `true` | Beta | 1.12 | 1.16 | +| `PodShareProcessNamespace` | `true` | GA | 1.17 | - | | `PVCProtection` | `false` | Alpha | 1.9 | 1.9 | | `PVCProtection` | - | Deprecated | 1.10 | - | +| `RequestManagement` | `false` | Alpha | 1.15 | 1.16 | +| `ResourceQuotaScopeSelectors` | `false` | Alpha | 1.11 | 1.11 | +| `ResourceQuotaScopeSelectors` | `true` | Beta | 1.12 | 1.16 | +| `ResourceQuotaScopeSelectors` | `true` | GA | 1.17 | - | +| `ScheduleDaemonSetPods` | `false` | Alpha | 1.11 | 1.11 | +| `ScheduleDaemonSetPods` | `true` | Beta | 1.12 | 1.16 | +| `ScheduleDaemonSetPods` | `true` | GA | 1.17 | - | +| `ServiceLoadBalancerFinalizer` | `false` | Alpha | 1.15 | 1.15 | +| `ServiceLoadBalancerFinalizer` | `true` | Beta | 1.16 | 1.16 | +| `ServiceLoadBalancerFinalizer` | `true` | GA | 1.17 | - | | `StorageObjectInUseProtection` | `true` | Beta | 1.10 | 1.10 | | `StorageObjectInUseProtection` | `true` | GA | 1.11 | - | | `SupportIPVSProxyMode` | `false` | Alpha | 1.8 | 1.8 | | `SupportIPVSProxyMode` | `false` | Beta | 1.9 | 1.9 | | `SupportIPVSProxyMode` | `true` | Beta | 1.10 | 1.10 | | `SupportIPVSProxyMode` | `true` | GA | 1.11 | - | +| `TaintNodesByCondition` | `false` | Alpha | 1.8 | 1.11 | +| `TaintNodesByCondition` | `true` | Beta | 1.12 | 1.16 | +| `TaintNodesByCondition` | `true` | GA | 1.17 | - | | `VolumeScheduling` | `false` | Alpha | 1.9 | 1.9 | | `VolumeScheduling` | `true` | Beta | 1.10 | 1.12 | | `VolumeScheduling` | `true` | GA | 1.13 | - | | `VolumeSubpath` | `true` | GA | 1.13 | - | +| `VolumeSubpathEnvExpansion` | `false` | Alpha | 1.14 | 1.14 | +| `VolumeSubpathEnvExpansion` | `true` | Beta | 1.15 | 1.16 | +| `VolumeSubpathEnvExpansion` | `true` | GA | 1.17 | - | +| `WatchBookmark` | `false` | Alpha | 1.15 | 1.15 | +| `WatchBookmark` | `true` | Beta | 1.16 | 1.16 | +| `WatchBookmark` | `true` | GA | 1.17 | - | + {{< /table >}} -## 使用功能 +## Using a feature - -### 功能阶段 + +## 使用特性 + +### 特性阶段 -处于 *Alpha*、*Beta*、*GA* 阶段的功能 - -*Alpha* 功能代表： + +处于 *Alpha* 、*Beta* 、 *GA* 阶段的特性。 + +*Alpha* 特性代表： + * 默认禁用。 -* 可能有错误，启用此功能可能会导致错误。 -* 随时可能删除对此功能的支持，恕不另行通知。 +* 可能有错误，启用此特性可能会导致错误。 +* 随时可能删除对此特性的支持，恕不另行通知。 * 在以后的软件版本中，API 可能会以不兼容的方式更改，恕不另行通知。 -* 建议将其仅用于短期测试集群中，因为开启功能会增加错误的风险，并且缺乏长期支持。 +* 建议将其仅用于短期测试中，因为开启特性会增加错误的风险，并且缺乏长期支持。 -*Beta* 功能代表： + +*Beta* 特性代表： + * 默认禁用。 -* 该功能已经经过良好测试。启用该功能是安全的。 -* 尽管详细信息可能会更改，但不会放弃对整体功能的支持。 -* 对象的架构或语义可能会在随后的 Beta 或稳定版本中以不兼容的方式更改。当发生这种情况时，我们将提供迁移到下一版本的说明。此功能可能需要删除、编辑和重新创建 API 对象。编辑过程可能需要慎重操作，因为这可能会导致依赖该功能的应用程序停机。 -* 推荐仅用于非关键业务用途，因为在后续版本中可能会发生不兼容的更改。如果您具有多个可以独立升级的集群，则可以放宽此限制。 +* 该特性已经经过良好测试。启用该特性是安全的。 +* 尽管详细信息可能会更改，但不会放弃对整体特性的支持。 +* 对象的架构或语义可能会在随后的 Beta 或稳定版本中以不兼容的方式更改。当发生这种情况时，我们将提供迁移到下一版本的说明。此特性可能需要删除、编辑和重新创建 API 对象。编辑过程可能需要慎重操作，因为这可能会导致依赖该特性的应用程序停机。 +* 推荐仅用于非关键业务用途，因为在后续版本中可能会发生不兼容的更改。如果您具有多个可以独立升级的，则可以放宽此限制。 {{< note >}} -请尝试使用 *Beta* 功能并提供有关它们的反馈！ -功能退出 Beta 后，对我们进行更多更改可能不切实际。 +请试用 *Beta* 特性并提供相关反馈！ +一旦特性结束 Beta 状态，我们就不太可能再对特性进行大幅修改。 {{< /note >}} -*GA* 功能也称为 *稳定* 功能，*GA* 功能代表着： + +*General Availability* (GA) 特性也称为 *稳定* 特性，*GA* 特性代表着： -* 不再需要相应的功能开关。 -* 对于许多后续版本，功能的稳定版本将出现在发行的软件中。 + +* 此特性会一直启用；你不能禁用它。 +* 不再需要相应的特性门控。 +* 对于许多后续版本，特性的稳定版本将出现在发行的软件中。 -### 功能开关 +## List of feature gates {#feature-gates} - -每个功能开关均设计用于启用或禁用特定功能： + +### 特性门控列表 + +每个特性门控均用于启用或禁用某个特定的特性： + - `Accelerators`：使用 Docker 时启用 Nvidia GPU 支持。 - `AdvancedAuditing`：启用[高级审查功能](/docs/tasks/debug-application-cluster/audit/#advanced-audit)。 -- `AffinityInAnnotations`（*已弃用*）：启用 [Pod 亲和力或反亲和力](/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity)。 +- `AffinityInAnnotations`（ *已弃用* ）：启用 [Pod 亲和力或反亲和力](/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity)。 - `AllowExtTrafficLocalEndpoints`：启用服务用于将外部请求路由到节点本地终端。 - `APIListChunking`：启用 API 客户端以块的形式从 API 服务器检索（“LIST” 或 “GET”）资源。 +- `APIPriorityAndFairness`: Enable managing request concurrency with prioritization and fairness at each server. (Renamed from `RequestManagement`) +- `APIPriorityAndFairness`: 在每个服务器上启用优先级和公平性来管理请求并发。（由 `RequestManagement` 重命名而来） - `APIResponseCompression`：压缩 “LIST” 或 “GET” 请求的 API 响应。 -- `AppArmor`：使用 Docker 时，在 Linux 节点上启用基于 AppArmor 机制的强制访问控制。有关更多详细信息，请参见 [AppArmor 教程](/docs/tutorials/clusters/apparmor/)。 +- `AppArmor`：使用 Docker 时，在 Linux 节点上启用基于 AppArmor 机制的强制访问控制。请参见 [AppArmor 教程](/docs/tutorials/clusters/apparmor/) 获取详细信息。 + + - `AttachVolumeLimit`：启用卷插件用于报告可连接到节点的卷数限制。有关更多详细信息，请参见[动态卷限制](/docs/concepts/storage/storage-limits/#dynamic-volume-limits)。 - `BalanceAttachedNodeVolumes`：包括要在调度时进行平衡资源分配的节点上的卷数。scheduler 在决策时会优先考虑 CPU、内存利用率和卷数更近的节点。 - `BlockVolume`：在 Pod 中启用原始块设备的定义和使用。有关更多详细信息，请参见[原始块卷支持](/docs/concepts/storage/persistent-volumes/#raw-block-volume-support)。 - `BoundServiceAccountTokenVolume`：迁移 ServiceAccount 卷以使用由 ServiceAccountTokenVolumeProjection 组成的预计卷。有关更多详细信息，请参见 [Service Account Token 卷](https://git.k8s.io/community/contributors/design-proposals/storage/svcacct-token-volume-source.md)。 - `CPUManager`：启用容器级别的 CPU 亲和力支持，有关更多详细信息，请参见 [CPU 管理策略](/docs/tasks/administer-cluster/cpu-management-policies/)。 + + - `CRIContainerLogRotation`：为 cri 容器运行时启用容器日志轮换。 - `CSIBlockVolume`：启用外部 CSI 卷驱动程序用于支持块存储。有关更多详细信息，请参见 [`csi` 原始块卷支持](/docs/concepts/storage/volumes/#csi-raw-block-volume-support)。 - `CSIDriverRegistry`：在 csi.storage.k8s.io 中启用与 CSIDriver API 对象有关的所有逻辑。 - `CSIInlineVolume`：为 Pod 启用 CSI 内联卷支持。 - `CSIMigration`：确保填充和转换逻辑能够将卷操作从内嵌插件路由到相应的预安装 CSI 插件。 -- `CSIMigrationAWS`：确保填充和转换逻辑能够将卷操作从 AWS-EBS 内嵌插件路由到 EBS CSI 插件。 -- `CSIMigrationAzureDisk`：确保填充和转换逻辑能够将卷操作从 Azure 磁盘内嵌插件路由到 Azure 磁盘 CSI 插件。 -- `CSIMigrationAzureFile`：确保填充和转换逻辑能够将卷操作从 Azure 文件内嵌插件路由到 Azure 文件 CSI 插件。 +- `CSIMigrationAWS`：确保填充和转换逻辑能够将卷操作从 AWS-EBS 内嵌插件路由到 EBS CSI 插件。如果节点未安装和配置 EBS CSI 插件，则支持回退到内嵌 EBS 插件。这需要启用 CSIMigration 特性标志。 +- `CSIMigrationAWSComplete`：停止在 kubelet 和卷控制器中注册 EBS 内嵌插件，并启用 shims 和转换逻辑将卷操作从AWS-EBS 内嵌插件路由到 EBS CSI 插件。这需要启用 CSIMigration 和 CSIMigrationAWS 特性标志，并在群集中的所有节点上安装和配置 EBS CSI 插件。 +- `CSIMigrationAzureDisk`：确保填充和转换逻辑能够将卷操作从 Azure 磁盘内嵌插件路由到 Azure 磁盘 CSI 插件。如果节点未安装和配置 AzureDisk CSI 插件，支持回退到内建 AzureDisk 插件。这需要启用 CSIMigration 特性标志。 +- `CSIMigrationAzureDiskComplete`：停止在 kubelet 和卷控制器中注册 Azure 磁盘内嵌插件，并启用 shims 和转换逻辑以将卷操作从 Azure 磁盘内嵌插件路由到 AzureDisk CSI 插件。这需要启用 CSIMigration 和 CSIMigrationAzureDisk 特性标志，并在群集中的所有节点上安装和配置 AzureDisk CSI 插件。 +- `CSIMigrationAzureFile`：确保填充和转换逻辑能够将卷操作从 Azure 文件内嵌插件路由到 Azure 文件 CSI 插件。如果节点未安装和配置 AzureFile CSI 插件，支持回退到内嵌 AzureFile 插件。这需要启用 CSIMigration 特性标志。 +- `CSIMigrationAzureFileComplete`：停止在 kubelet 和卷控制器中注册 Azure-File 内嵌插件，并启用 shims 和转换逻辑以将卷操作从 Azure-File 内嵌插件路由到 AzureFile CSI 插件。这需要启用 CSIMigration 和 CSIMigrationAzureFile 特性标志，并在群集中的所有节点上安装和配置 AzureFile CSI 插件。 + -- `CSIMigrationGCE`：确保填充和转换逻辑能够将卷操作从 GCE-PD 内嵌插件路由到 PD CSI 插件 -- `CSIMigrationOpenStack`：确保填充和转换逻辑能够将卷操作从 Cinder 内嵌插件路由到 Cinder CSI 插件。 + +- `CSIMigrationGCE`：使 shims 和转换逻辑能够将卷操作从 GCE-PD 内嵌插件路由到 PD CSI 插件。如果节点未安装和配置 PD CSI 插件，支持回退到内嵌 GCE 插件。这需要启用 CSIMigration 特性标志。 +- `CSIMigrationGCEComplete`：停止在 kubelet 和卷控制器中注册 GCE-PD 内嵌插件，并启用 shims 和转换逻辑以将卷操作从 GCE-PD 内嵌插件路由到 PD CSI 插件。这需要启用 CSIMigration 和 CSIMigrationGCE 特性标志，并在群集中的所有节点上安装和配置 PD CSI 插件。 +- `CSIMigrationOpenStack`：确保填充和转换逻辑能够将卷操作从 Cinder 内嵌插件路由到 Cinder CSI 插件。如果节点未安装和配置 Cinder CSI 插件，支持回退到内嵌 Cinder 插件。这需要启用 CSIMigration 特性标志。 +- `CSIMigrationOpenStackComplete`：停止在 kubelet 和卷控制器中注册 Cinder 内嵌插件，并启用 shims 和转换逻辑将卷操作从 Cinder 内嵌插件路由到 Cinder CSI 插件。这需要启用 CSIMigration 和 CSIMigrationOpenStack 特性标志，并在群集中的所有节点上安装和配置 Cinder CSI 插件。 - `CSINodeInfo`：在 csi.storage.k8s.io 中启用与 CSINodeInfo API 对象有关的所有逻辑。 - `CSIPersistentVolume`：启用发现并挂载通过 [CSI（容器存储接口）](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/storage/container-storage-interface.md)兼容卷插件配置的卷。有关更多详细信息，请参见 [`csi` 卷类型](/docs/concepts/storage/volumes/#csi)。 + + - `CustomCPUCFSQuotaPeriod`：使节点能够更改 CPUCFSQuotaPeriod。 - `CustomPodDNS`：使用其 `dnsConfig` 属性启用 Pod 的自定义 DNS 设置。有关更多详细信息，请参见 [Pod 的 DNS 配置](/docs/concepts/services-networking/dns-pod-service/#pods-dns-config)。 - `CustomResourceDefaulting`：为 OpenAPI v3 验证架构中的默认值启用 CRD 支持。 @@ -470,6 +540,7 @@ Each feature gate is designed for enabling/disabling a specific feature: - `CustomResourceValidation`：对于从 [CustomResourceDefinition](/docs/concepts/api-extension/custom-resources/) 中创建的资源启用基于架构的验证。 - `CustomResourceWebhookConversion`：对于从 [CustomResourceDefinition](/docs/concepts/api-extension/custom-resources/) 中创建的资源启用基于 Webhook 的转换。 对正在运行的 Pod 进行故障排除。 + + - `DevicePlugins`：在节点上启用基于 [device-plugins](/docs/concepts/cluster-administration/device-plugins/) 的资源供应。 - `DryRun`：启用服务器端 [dry run](/docs/reference/using-api/api-concepts/#dry-run) 请求，以便无需提交即可测试验证、合并和差异化。 - `DynamicAuditing`：确保[动态审查](/docs/tasks/debug-application-cluster/audit/#dynamic-backend)。 - `DynamicKubeletConfig`：启用 kubelet 的动态配置。请参阅[重新配置 kubelet](/docs/tasks/administer-cluster/reconfigure-kubelet/)。 -- `DynamicProvisioningScheduling`：扩展默认 scheduler 以了解卷拓扑并处理 PV 配置。此功能已在 v1.12 中完全由 `VolumeScheduling` 功能取代。 -- `DynamicVolumeProvisioning`（*已弃用*）：启用持久化卷到 Pod 的[动态预配置](/docs/concepts/storage/dynamic-provisioning/)。 +- `DynamicProvisioningScheduling`：扩展默认 scheduler 以了解卷拓扑并处理 PV 配置。此特性已在 v1.12 中完全被 `VolumeScheduling` 特性取代。 +- `DynamicVolumeProvisioning`（ *已弃用* ）：启用持久化卷到 Pod 的[动态预配置](/docs/concepts/storage/dynamic-provisioning/)。 + -- `EnableAggregatedDiscoveryTimeout` （*已弃用*）：对聚集的发现调用启用五秒钟超时设置。 + +- `EnableAggregatedDiscoveryTimeout` （ *已弃用* ）：对聚集的发现调用启用五秒钟超时设置。 - `EnableEquivalenceClassCache`：调度 Pod 时，使 scheduler 缓存节点的等效项。 -- `EphemeralContainers`：启用添加 {{< glossary_tooltip text="临时容器" term_id="ephemeral-container" >}} 到正在运行的 Pod 的功能。 -- `EvenPodsSpread`：使 Pod 能够在拓扑域之间平衡调度。请参阅 [Even Pods Spread](/docs/concepts/configuration/even-pods-spread)。 +- `EphemeralContainers`：启用添加 {{< glossary_tooltip text="临时容器" term_id="ephemeral-container" >}} 到正在运行的 Pod 的特性。 +- `EvenPodsSpread`：使 Pod 能够在拓扑域之间平衡调度。请参阅 [Pod 拓扑扩展约束](/docs/concepts/workloads/pods/pod-topology-spread-constraints/)。 - `ExpandInUsePersistentVolumes`：启用扩展使用中的 PVC。请查阅 [调整使用中的 PersistentVolumeClaim 的大小](/docs/concepts/storage/persistent-volumes/#resizing-an-in-use-persistentvolumeclaim)。 - `ExpandPersistentVolumes`：启用持久卷的扩展。请查阅[扩展永久卷声明](/docs/concepts/storage/persistent-volumes/#expanding-persistent-volumes-claims)。 -- `ExperimentalCriticalPodAnnotation`：启用将特定 Pod 注解为 *critical* 的方式，用于[确保其调度](/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods/)。从 v1.13 开始，Pod 优先级和抢占功能已弃用此功能。 +- `ExperimentalCriticalPodAnnotation`：启用将特定 Pod 注解为 *critical* 的方式，用于[确保其调度](/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods/)。从 v1.13 开始，Pod 优先级和抢占功能已弃用此特性。 + -- `ExperimentalHostUserNamespaceDefaultingGate`：启用默认的用户命名空间进行托管。这适用于使用其他主机命名空间、主机安装的容器，或具有特权或使用特定的非命名空间功能（例如MKNODE、SYS_MODULE等）的容器。如果在 Docker 守护程序中启用了用户命名空间重新映射，则启用此选项。 -- `EndpointSlice`：启用端点切片以实现更多可扩展的网络端点。需要启用相应的 API 和控制器，请参阅[启用端点切片](/docs/tasks/administer-cluster/enabling-endpoint-slices/)。 -- `GCERegionalPersistentDisk`：在 GCE 上启用区域 PD 功能。 + +- `ExperimentalHostUserNamespaceDefaultingGate`：启用主机默认的用户命名空间。这适用于使用其他主机命名空间、主机安装的容器，或具有特权或使用特定的非命名空间功能（例如MKNODE、SYS_MODULE等）的容器。如果在 Docker 守护程序中启用了用户命名空间重新映射，则启用此选项。 +- `EndpointSlice`：启用端点切片以实现更多可扩展的网络端点。需要启用相应的 API 和控制器，请参阅[启用端点切片](/docs/tasks/administer-cluster/enabling-endpointslices/)。 +- `GCERegionalPersistentDisk`：在 GCE 上启用区域 PD 特性。 - `HugePages`: 启用分配和使用预分配的 [huge pages](/docs/tasks/manage-hugepages/scheduling-hugepages/)。 + + - `HyperVContainer`：为 Windows 容器启用[Hyper-V 隔离](https://docs.microsoft.com/en-us/virtualization/windowscontainers/manage-containers/hyperv-container)。 - `HPAScaleToZero`：使用自定义指标或外部指标时，可将 `HorizontalPodAutoscaler` 资源的 `minReplicas` 设置为 0。 - `KubeletConfigFile`：启用从使用配置文件指定的文件中加载 kubelet 配置。有关更多详细信息，请参见[通过配置文件设置 kubelet 参数](/docs/tasks/administer-cluster/kubelet-config-file/)。 - `KubeletPluginsWatcher`：启用基于探针的插件监视应用程序，使 kubelet 能够发现插件，例如 [CSI 卷驱动程序](/docs/concepts/storage/volumes/#csi)。 - `KubeletPodResources`：启用 kubelet 的 pod 资源 grpc 端点。有关更多详细信息，请参见[支持设备监控](https://git.k8s.io/community/keps/sig-node/compute-device-assignment.md)。 -- `LegacyNodeRoleBehavior`：禁用此选项后，服务负载均衡器中的旧版操作和节点中断将忽略 `node-role.kubernetes.io/master` 标签，而使用特定于功能的标签。 +- `LegacyNodeRoleBehavior`：禁用此选项后，服务负载均衡中的旧版操作和节点中断将忽略 `node-role.kubernetes.io/master` 标签，而使用特性指定的标签。 + + - `LocalStorageCapacityIsolation`：启用[本地临时存储](/docs/concepts/configuration/manage-compute-resources-container/)的消耗，以及 [emptyDir 卷](/docs/concepts/storage/volumes/#emptydir) 的 `sizeLimit` 属性。 - `LocalStorageCapacityIsolationFSQuotaMonitoring`：如果为[本地临时存储](/docs/concepts/configuration/manage-compute-resources-container/)启用了 `LocalStorageCapacityIsolation`，并且 [emptyDir 卷](/docs/concepts/storage/volumes/#emptydir) 的后备文件系统支持项目配额，并且启用了这些配额，请使用项目配额来监视 [emptyDir 卷](/docs/concepts/storage/volumes/#emptydir)的存储消耗而不是遍历文件系统，以此获得更好的性能和准确性。 - `MountContainers`：在主机上启用将应用程序容器用作卷安装程序。 - `MountPropagation`：启用将一个容器安装的共享卷共享到其他容器或 Pod。有关更多详细信息，请参见 [mount propagation](/docs/concepts/storage/volumes/#mount-propagation)。 - `NodeDisruptionExclusion`：启用节点标签 `node.kubernetes.io/exclude-disruption`，以防止在区域故障期间驱逐节点。 + + - `NodeLease`：启用新的租赁 API 以报告节点心跳，可用作节点运行状况信号。 - `NonPreemptingPriority`：为 PriorityClass 和 Pod 启用 NonPreempting 选项。 - `PersistentLocalVolumes`：在 Pod 中启用 “本地” 卷类型的使用。如果请求 “本地” 卷，则必须指定 Pod 亲和力。 -- `PodOverhead`：启用 [PodOverhead](/docs/concepts/configuration/pod-overhead/) 功能以解决 Pod 开销。 +- `PodOverhead`：启用 [PodOverhead](/docs/concepts/configuration/pod-overhead/) 特性以解决 Pod 开销。 - `PodPriority`：根据[优先级](/docs/concepts/configuration/pod-priority-preemption/)启用 Pod 的调度和抢占。 -- `PodReadinessGates`：启用 `PodReadinessGate` 字段的设置以扩展 Pod 准备状态评估。有关更多详细信息，请参见 [Pod readiness 功能开关](/docs/concepts/workloads/pods/pod-lifecycle/#pod-readiness-gate)。 +- `PodReadinessGates`：启用 `PodReadinessGate` 字段的设置以扩展 Pod 准备状态评估。有关更多详细信息，请参见 [Pod readiness 特性门控](/docs/concepts/workloads/pods/pod-lifecycle/#pod-readiness-gate)。 + + - `PodShareProcessNamespace`：在 Pod 中启用 `shareProcessNamespace` 的设置，以便在 Pod 中运行的容器之间共享单个进程命名空间。更多详细信息，请参见[在 Pod 中的容器之间共享进程命名空间](/docs/tasks/configure-pod-container/share-process-namespace/)。 - `ProcMountType`：启用对容器的 ProcMountType 的控制。 -- `PVCProtection`：启用防止任何 Pod 仍使用 PersistentVolumeClaim(PVC) 删除的功能。可以在[此处](/docs/tasks/administer-cluster/storage-object-in-use-protection/)中找到更多详细信息。 +- `PVCProtection`：启用防止任何 Pod 仍使用 PersistentVolumeClaim(PVC) 删除的特性。可以在[此处](/docs/tasks/administer-cluster/storage-object-in-use-protection/)中找到更多详细信息。 - `QOSReserved`：允许在 QoS 级别进行资源预留，以防止处于较低 QoS 级别的 Pod 突发进入处于较高 QoS 级别的请求资源（仅适用于内存）。 -- `ResourceLimitsPriorityFunction`：启用 scheduler 优先级功能，该功能将最低可能得 1 分配给至少满足输入 Pod 的 cpu 和内存限制之一的节点，目的是打破得分相同的节点之间的联系。 +- `ResourceLimitsPriorityFunction`：启用 scheduler 优先级特性，该特性将最低可能得 1 分配给至少满足输入 Pod 的 cpu 和内存限制之一的节点，目的是打破得分相同的节点之间的联系。 + + - `RequestManagement`：在每个服务器上启用具有优先级和公平性的管理请求并发性。 - `ResourceQuotaScopeSelectors`：启用资源配额范围选择器。 - `RotateKubeletClientCertificate`：在 kubelet 上启用客户端 TLS 证书的轮换。有关更多详细信息，请参见 [kubelet 配置](/docs/reference/command-line-tools-reference/kubelet-tls-bootstrapping/#kubelet-configuration)。 - `RotateKubeletServerCertificate`：在 kubelet 上启用服务器 TLS 证书的轮换。有关更多详细信息，请参见 [kubelet 配置](/docs/reference/command-line-tools-reference/kubelet-tls-bootstrapping/#kubelet-configuration)。 - `RunAsGroup`：启用对容器初始化过程中设置的主要组 ID 的控制。 -- `RuntimeClass`：启用[RuntimeClass](/docs/concepts/containers/runtime-class/) 功能用于选择容器运行时配置。 +- `RuntimeClass`：启用 [RuntimeClass](/docs/concepts/containers/runtime-class/) 特性用于选择容器运行时配置。 - `ScheduleDaemonSetPods`：启用 DaemonSet Pods 由默认调度程序而不是 DaemonSet 控制器进行调度。 + + - `SCTPSupport`：在 “服务”、“端点”、“NetworkPolicy” 和 “Pod” 定义中，将 SCTP 用作 “协议” 值。 - `ServerSideApply`：在 API 服务器上启用[服务器端应用（SSA）](/docs/reference/using-api/api-concepts/#server-side-apply) 路径。 -- `ServiceLoadBalancerFinalizer`：为服务负载均衡器启用终结器保护。 -- `ServiceNodeExclusion`：启用从云提供商创建的负载均衡器中排除节点。如果节点标记有 `alpha.service-controller.kubernetes.io/exclude-balancer` 键（启用 `LegacyNodeRoleBehavior` 时）或 `node.kubernetes.io/exclude-from-external-load-balancers`，则可以排除节点。 +- `ServiceLoadBalancerFinalizer`：为服务负载均衡启用终结器保护。 +- `ServiceNodeExclusion`：启用从云提供商创建的负载均衡中排除节点。如果节点标记有 `alpha.service-controller.kubernetes.io/exclude-balancer` 键或 `node.kubernetes.io/exclude-from-external-load-balancers`，则可以排除节点。 - `StartupProbe`：在 kubelet 中启用 [startup](/docs/concepts/workloads/pods/pod-lifecycle/#when-should-you-use-a-startup-probe) 探针。 - `StorageObjectInUseProtection`：如果仍在使用 PersistentVolume 或 PersistentVolumeClaim 对象，则将其推迟。 + + - `StorageVersionHash`：允许 apiserver 在发现中公开存储版本的哈希值。 - `StreamingProxyRedirects`：指示 API 服务器拦截（并遵循）从后端（kubelet）进行重定向以处理流请求。流请求的例子包括 `exec`、`attach` 和 `port-forward` 请求。 -- `SupportIPVSProxyMode`：启用使用 IPVS 提供集群内服务负载平衡。有关更多详细信息，请参见[服务代理](/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies)。 +- `SupportIPVSProxyMode`：启用使用 IPVS 提供内服务负载平衡。有关更多详细信息，请参见[服务代理](/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies)。 - `SupportPodPidsLimit`：启用支持限制 Pod 中的进程 PID。 - `Sysctls`：启用对可以为每个 Pod 设置的命名空间内核参数（sysctls）的支持。有关更多详细信息，请参见 [sysctls](/docs/tasks/administer-cluster/sysctl-cluster/)。 + -- `TaintBasedEvictions`：根据节点上的污点和 Pod 上的容忍度启用从节点驱逐 Pod 的功能。有关更多详细信息，请参见[污点和容忍度](/docs/concepts/configuration/taint-and-toleration/)。 + +- `TaintBasedEvictions`：根据节点上的污点和 Pod 上的容忍度启用从节点驱逐 Pod 的特性。有关更多详细信息，请参见[污点和容忍度](/docs/concepts/configuration/taint-and-toleration/)。 - `TaintNodesByCondition`：根据[节点条件](/docs/concepts/configuration/taint-and-toleration/)启用自动在节点标记污点。 - `TokenRequest`：在服务帐户资源上启用 `TokenRequest` 端点。 -- `TokenRequestProjection`：启用通过 [`projected` 卷](/docs/concepts/storage/volumes/#projected) 将服务帐户令牌注入到 Pod 中的功能。 +- `TokenRequestProjection`：启用通过 [`projected` 卷](/docs/concepts/storage/volumes/#projected) 将服务帐户令牌注入到 Pod 中的特性。 +- `TopologyManager`：启用一种机制来协调 Kubernetes 不同组件的细粒度硬件资源分配。详见 [控制节点上的拓扑管理策略](/docs/tasks/administer-cluster/topology-manager/)。 - `TTLAfterFinished`：完成执行后，允许 [TTL 控制器](/docs/concepts/workloads/controllers/ttlafterfinished/)清理资源。 + + - `VolumePVCDataSource`：启用对将现有 PVC 指定数据源的支持。 -- `VolumeScheduling`：启用卷拓扑感知调度，并使 PersistentVolumeClaim（PVC）绑定调度决策；当与 PersistentLocalVolumes 功能开关一起使用时，还可以使用 `PersistentLocalVolumes` 卷类型。 +- `VolumeScheduling`：启用卷拓扑感知调度，并使 PersistentVolumeClaim（PVC）绑定调度决策；当与 PersistentLocalVolumes 特性门控一起使用时，还可以使用 `PersistentLocalVolumes` 卷类型。 - `VolumeSnapshotDataSource`：启用卷快照数据源支持。 + + - `VolumeSubpathEnvExpansion`：启用 `subPathExpr` 字段用于将环境变量扩展为 `subPath`。 - `WatchBookmark`：启用对监测 bookmark 事件的支持。 - `WindowsGMSA`：允许将 GMSA 凭据规范从 Pod 传递到容器运行时。 -- `WinDSR`：允许 kube-proxy 为 Windows 创建 DSR 负载均衡器。 +- `WinDSR`：允许 kube-proxy 为 Windows 创建 DSR 负载均衡。 - `WinOverlay`：允许 kube-proxy 在 Windows 的 overlay 模式下运行。 {{% /capture %}} +{{% capture whatsnext %}} + + + +* Kubernetes 的 [弃用策略](/docs/reference/using-api/deprecation-policy/) 介绍了项目已移除的特性部件和组件的方法。 + +{{% /capture %}} From 7ac1614f74f71b2ed9c9be30c0c4a3bc302632e2 Mon Sep 17 00:00:00 2001 From: tom1299 Date: Wed, 11 Mar 2020 09:33:36 +0100 Subject: [PATCH 028/140] Harmonize Pod Preset tasks (#18128) --- .../inject-data-application/podpreset.md | 286 +++++++++++++----- .../examples/podpreset/allow-db-merged.yaml | 6 - content/en/examples/podpreset/allow-db.yaml | 6 - 3 files changed, 210 insertions(+), 88 deletions(-) diff --git a/content/en/docs/tasks/inject-data-application/podpreset.md b/content/en/docs/tasks/inject-data-application/podpreset.md index beb57754c3..73ee4bbb42 100644 --- a/content/en/docs/tasks/inject-data-application/podpreset.md +++ b/content/en/docs/tasks/inject-data-application/podpreset.md @@ -2,23 +2,19 @@ reviewers: - jessfraz title: Inject Information into Pods Using a PodPreset +min-kubernetes-server-version: v1.10 content_template: templates/task weight: 60 --- {{% capture overview %}} -You can use a `PodPreset` object to inject information like secrets, volume -mounts, and environment variables etc into pods at creation time. -This task shows some examples on using the `PodPreset` resource. +This page shows how to use PodPreset objects to inject information like {{< glossary_tooltip text="Secrets" term_id="secret" >}}, volume mounts, and {{< glossary_tooltip text="environment variables" term_id="container-env-variables" >}} into Pods at creation time. {{% /capture %}} {{% capture prerequisites %}} -Get an overview of PodPresets at -[Understanding Pod Presets](/docs/concepts/workloads/pods/podpreset/). - {{< include "task-tutorial-prereqs.md" >}} {{< version-check >}} {{% /capture %}} @@ -26,157 +22,295 @@ Get an overview of PodPresets at {{% capture steps %}} -## Simple Pod Spec Example +## Use Pod presets to inject environment variables and volumes -This is a simple example to show how a Pod spec is modified by the Pod -Preset. +In this step, you create a preset that has a volume mount and one environment variable. +Here is the manifest for the PodPreset: {{< codenew file="podpreset/preset.yaml" >}} +In the manifest, you can see that the preset has an environment variable definition called `DB_PORT` +and a volume mount definition called `cache-volume` which is mounted under `/cache`. The {{< glossary_tooltip text="selector" term_id="selector" >}} specifies that +the preset will act upon any Pod that is labeled `role:frontend`. + Create the PodPreset: ```shell kubectl apply -f https://k8s.io/examples/podpreset/preset.yaml ``` -Examine the created PodPreset: +Verify that the PodPreset has been created: ```shell kubectl get podpreset ``` ``` -NAME AGE -allow-database 1m +NAME CREATED AT +allow-database 2020-01-24T08:54:29Z ``` -The new PodPreset will act upon any pod that has label `role: frontend`. +This manifest defines a Pod labelled `role: frontend` (matching the PodPreset's selector): {{< codenew file="podpreset/pod.yaml" >}} -Create a pod: +Create the Pod: ```shell kubectl create -f https://k8s.io/examples/podpreset/pod.yaml ``` -List the running Pods: +Verify that the Pod is running: ```shell kubectl get pods ``` + +The output shows that the Pod is running: + ``` NAME READY STATUS RESTARTS AGE website 1/1 Running 0 4m ``` -**Pod spec after admission controller:** - -{{< codenew file="podpreset/merged.yaml" >}} - -To see above output, run the following command: +View the Pod spec altered by the admission controller in order to see the effects of the preset +having been applied: ```shell kubectl get pod website -o yaml ``` -## Pod Spec with ConfigMap Example +{{< codenew file="podpreset/merged.yaml" >}} -This is an example to show how a Pod spec is modified by the Pod Preset -that defines a `ConfigMap` for Environment Variables. +The `DB_PORT` environment variable, the `volumeMount` and the `podpreset.admission.kubernetes.io` annotation +of the Pod verify that the preset has been applied. -**User submitted pod spec:** +## Pod spec with ConfigMap example -{{< codenew file="podpreset/pod.yaml" >}} +This is an example to show how a Pod spec is modified by a Pod preset +that references a ConfigMap containing environment variables. -**User submitted `ConfigMap`:** +Here is the manifest containing the definition of the ConfigMap: {{< codenew file="podpreset/configmap.yaml" >}} -**Example Pod Preset:** +Create the ConfigMap: + +```shell +kubectl create -f https://k8s.io/examples/podpreset/configmap.yaml +``` + +Here is a PodPreset manifest referencing that ConfigMap: {{< codenew file="podpreset/allow-db.yaml" >}} -**Pod spec after admission controller:** +Create the preset that references the ConfigMap: -{{< codenew file="podpreset/allow-db-merged.yaml" >}} +```shell +kubectl create -f https://k8s.io/examples/podpreset/allow-db.yaml +``` -## ReplicaSet with Pod Spec Example - -The following example shows that only the pod spec is modified by the Pod -Preset. - -**User submitted ReplicaSet:** - -{{< codenew file="podpreset/replicaset.yaml" >}} - -**Example Pod Preset:** - -{{< codenew file="podpreset/preset.yaml" >}} - -**Pod spec after admission controller:** - -Note that the ReplicaSet spec was not changed, users have to check individual pods -to validate that the PodPreset has been applied. - -{{< codenew file="podpreset/replicaset-merged.yaml" >}} - -## Multiple PodPreset Example - -This is an example to show how a Pod spec is modified by multiple Pod -Injection Policies. - -**User submitted pod spec:** +The following manifest defines a Pod matching the PodPreset for this example: {{< codenew file="podpreset/pod.yaml" >}} -**Example Pod Preset:** +Create the Pod: + +```shell +kubectl create -f https://k8s.io/examples/podpreset/pod.yaml +``` + +View the Pod spec altered by the admission controller in order to see the effects of the preset +having been applied: + +```shell +kubectl get pod website -o yaml +``` + +{{< codenew file="podpreset/allow-db-merged.yaml" >}} + +The `DB_PORT` environment variable and the `podpreset.admission.kubernetes.io` annotation of the Pod +verify that the preset has been applied. + +## ReplicaSet with Pod spec example + +This is an example to show that only Pod specs are modified by Pod presets. Other workload types +like ReplicaSets or Deployments are unaffected. + +Here is the manifest for the PodPreset for this example: {{< codenew file="podpreset/preset.yaml" >}} -**Another Pod Preset:** +Create the preset: + +```shell +kubectl apply -f https://k8s.io/examples/podpreset/preset.yaml +``` + +This manifest defines a ReplicaSet that manages three application Pods: + +{{< codenew file="podpreset/replicaset.yaml" >}} + +Create the ReplicaSet: + +```shell +kubectl create -f https://k8s.io/examples/podpreset/replicaset.yaml +``` + +Verify that the Pods created by the ReplicaSet are running: + +```shell +kubectl get pods +``` + +The output shows that the Pods are running: + +``` +NAME READY STATUS RESTARTS AGE +frontend-2l94q 1/1 Running 0 2m18s +frontend-6vdgn 1/1 Running 0 2m18s +frontend-jzt4p 1/1 Running 0 2m18s +``` + +View the `spec` of the ReplicaSet: + +```shell +kubectl get replicasets frontend -o yaml +``` + +{{< note >}} +The ReplicaSet object's `spec` was not changed, nor does the ReplicaSet contain a +`podpreset.admission.kubernetes.io` annotation. This is because a PodPreset only +applies to Pod objects. + +To see the effects of the preset having been applied, you need to look at individual Pods. +{{< /note >}} + +The command to view the specs of the affected Pods is: + +```shell +kubectl get pod --selector=role=frontend -o yaml +``` + +{{< codenew file="podpreset/replicaset-merged.yaml" >}} + +Again the `podpreset.admission.kubernetes.io` annotation of the Pods +verifies that the preset has been applied. + +## Multiple Pod presets example + +This is an example to show how a Pod spec is modified by multiple Pod presets. + + +Here is the manifest for the first PodPreset: + +{{< codenew file="podpreset/preset.yaml" >}} + +Create the first PodPreset for this example: + +```shell +kubectl apply -f https://k8s.io/examples/podpreset/preset.yaml +``` + +Here is the manifest for the second PodPreset: {{< codenew file="podpreset/proxy.yaml" >}} -**Pod spec after admission controller:** +Create the second preset: + +```shell +kubectl apply -f https://k8s.io/examples/podpreset/proxy.yaml +``` + +Here's a manifest containing the definition of an applicable Pod (matched by two PodPresets): + +{{< codenew file="podpreset/pod.yaml" >}} + +Create the Pod: + +```shell +kubectl create -f https://k8s.io/examples/podpreset/pod.yaml +``` + +View the Pod spec altered by the admission controller in order to see the effects of both presets +having been applied: + +```shell +kubectl get pod website -o yaml +``` {{< codenew file="podpreset/multi-merged.yaml" >}} -## Conflict Example +The `DB_PORT` environment variable, the `proxy-volume` VolumeMount and the two `podpreset.admission.kubernetes.io` +annotations of the Pod verify that both presets have been applied. -This is an example to show how a Pod spec is not modified by the Pod Preset -when there is a conflict. +## Conflict example -**User submitted pod spec:** +This is an example to show how a Pod spec is not modified by a Pod preset when there is a conflict. +The conflict in this example consists of a `VolumeMount` in the PodPreset conflicting with a Pod that defines the same `mountPath`. -{{< codenew file="podpreset/conflict-pod.yaml" >}} - -**Example Pod Preset:** +Here is the manifest for the PodPreset: {{< codenew file="podpreset/conflict-preset.yaml" >}} -**Pod spec after admission controller will not change because of the conflict:** +Note the `mountPath` value of `/cache`. + +Create the preset: + +```shell +kubectl apply -f https://k8s.io/examples/podpreset/conflict-preset.yaml +``` + +Here is the manifest for the Pod: {{< codenew file="podpreset/conflict-pod.yaml" >}} -**If we run `kubectl describe...` we can see the event:** +Note the volumeMount element with the same path as in the PodPreset. + +Create the Pod: ```shell -kubectl describe ... -``` -``` -.... -Events: - FirstSeen LastSeen Count From SubobjectPath Reason Message - Tue, 07 Feb 2017 16:56:12 -0700 Tue, 07 Feb 2017 16:56:12 -0700 1 {podpreset.admission.kubernetes.io/podpreset-allow-database } conflict Conflict on pod preset. Duplicate mountPath /cache. +kubectl create -f https://k8s.io/examples/podpreset/conflict-pod.yaml ``` -## Deleting a Pod Preset +View the Pod spec: -Once you don't need a pod preset anymore, you can delete it with `kubectl`: +```shell +kubectl get pod website -o yaml +``` + +{{< codenew file="podpreset/conflict-pod.yaml" >}} + +You can see there is no preset annotation (`podpreset.admission.kubernetes.io`). Seeing no annotation tells you that no preset has not been applied to the Pod. + +However, the +[PodPreset admission controller](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#podpreset) +logs a warning containing details of the conflict. +You can view the warning using `kubectl`: + +```shell +kubectl -n kube-system logs -l=component=kube-apiserver +``` + +The output should look similar to: + +``` +W1214 13:00:12.987884 1 admission.go:147] conflict occurred while applying podpresets: allow-database on pod: err: merging volume mounts for allow-database has a conflict on mount path /cache: +v1.VolumeMount{Name:"other-volume", ReadOnly:false, MountPath:"/cache", SubPath:"", MountPropagation:(*v1.MountPropagationMode)(nil), SubPathExpr:""} +does not match +core.VolumeMount{Name:"cache-volume", ReadOnly:false, MountPath:"/cache", SubPath:"", MountPropagation:(*core.MountPropagationMode)(nil), SubPathExpr:""} + in container +``` + +Note the conflict message on the path for the VolumeMount. + +## Deleting a PodPreset + +Once you don't need a PodPreset anymore, you can delete it with `kubectl`: ```shell kubectl delete podpreset allow-database ``` +The output shows that the PodPreset was deleted: ``` podpreset "allow-database" deleted ``` diff --git a/content/en/examples/podpreset/allow-db-merged.yaml b/content/en/examples/podpreset/allow-db-merged.yaml index 8a0ad101d7..7f52cc1fa4 100644 --- a/content/en/examples/podpreset/allow-db-merged.yaml +++ b/content/en/examples/podpreset/allow-db-merged.yaml @@ -14,9 +14,6 @@ spec: volumeMounts: - mountPath: /cache name: cache-volume - - mountPath: /etc/app/config.json - readOnly: true - name: secret-volume ports: - containerPort: 80 env: @@ -32,6 +29,3 @@ spec: volumes: - name: cache-volume emptyDir: {} - - name: secret-volume - secret: - secretName: config-details diff --git a/content/en/examples/podpreset/allow-db.yaml b/content/en/examples/podpreset/allow-db.yaml index 0cca13bab2..2c511e650d 100644 --- a/content/en/examples/podpreset/allow-db.yaml +++ b/content/en/examples/podpreset/allow-db.yaml @@ -19,12 +19,6 @@ spec: volumeMounts: - mountPath: /cache name: cache-volume - - mountPath: /etc/app/config.json - readOnly: true - name: secret-volume volumes: - name: cache-volume emptyDir: {} - - name: secret-volume - secret: - secretName: config-details From 29075d47cfc325ecd8df2f2d7973e2e5282d2b28 Mon Sep 17 00:00:00 2001 From: Ihor Sychevskyi <26163841+Arhell@users.noreply.github.com> Date: Wed, 11 Mar 2020 15:43:36 +0200 Subject: [PATCH 029/140] fix readme markdown (#19589) --- README-vi.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README-vi.md b/README-vi.md index 454d687d02..b06b6df368 100644 --- a/README-vi.md +++ b/README-vi.md @@ -26,7 +26,7 @@ Cách được đề xuất để chạy trang web Kubernetes cục bộ là dù > Nếu bạn làm việc trên môi trường Windows, bạn sẽ cần thêm môt vài công cụ mà bạn có thể cài đặt với [Chocolatey](https://chocolatey.org). `choco install make` -> Nếu bạn không muốn dùng Docker để chạy trang web cục bộ, hãy xem [Chạy website cục bộ dùng Hugo](#Chạy website cục bộ dùng Hugo) dưới đây. +> Nếu bạn không muốn dùng Docker để chạy trang web cục bộ, hãy xem [Chạy website cục bộ dùng Hugo](#chạy-website-cục-bộ-dùng-hugo) dưới đây. Nếu bạn có Docker đang [up và running](https://www.docker.com/get-started), build `kubernetes-hugo` Docker image cục bộ: From c331ff47a7da09be0eb0705b1086033fbd24a6d2 Mon Sep 17 00:00:00 2001 From: Marshall Ford Date: Wed, 11 Mar 2020 10:13:37 -0500 Subject: [PATCH 030/140] fixes troubleshooting-kubeadm references to taint node-role.kubernetes.io/master (#19293) * fixes troubleshooting-kubeadm references to taint node-role.kubernetes.io/master Signed-off-by: Marshall Ford * revert ja,zsh langs Signed-off-by: Marshall Ford --- .../tools/kubeadm/troubleshooting-kubeadm.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/content/en/docs/setup/production-environment/tools/kubeadm/troubleshooting-kubeadm.md b/content/en/docs/setup/production-environment/tools/kubeadm/troubleshooting-kubeadm.md index 31f94ef137..c0c1e2b0b2 100644 --- a/content/en/docs/setup/production-environment/tools/kubeadm/troubleshooting-kubeadm.md +++ b/content/en/docs/setup/production-environment/tools/kubeadm/troubleshooting-kubeadm.md @@ -307,17 +307,17 @@ The tracking issue for this problem is [here](https://github.com/kubernetes/kube *Note: This [issue](https://github.com/kubernetes/kubeadm/issues/1358) only applies to tools that marshal kubeadm types (e.g. to a YAML configuration file). It will be fixed in kubeadm API v1beta2.* -By default, kubeadm applies the `role.kubernetes.io/master:NoSchedule` taint to control-plane nodes. +By default, kubeadm applies the `node-role.kubernetes.io/master:NoSchedule` taint to control-plane nodes. If you prefer kubeadm to not taint the control-plane node, and set `InitConfiguration.NodeRegistration.Taints` to an empty slice, the field will be omitted when marshalling. When the field is omitted, kubeadm applies the default taint. There are at least two workarounds: -1. Use the `role.kubernetes.io/master:PreferNoSchedule` taint instead of an empty slice. [Pods will get scheduled on masters](/docs/concepts/configuration/taint-and-toleration/), unless other nodes have capacity. +1. Use the `node-role.kubernetes.io/master:PreferNoSchedule` taint instead of an empty slice. [Pods will get scheduled on masters](/docs/concepts/configuration/taint-and-toleration/), unless other nodes have capacity. 2. Remove the taint after kubeadm init exits: ```bash -kubectl taint nodes NODE_NAME role.kubernetes.io/master:NoSchedule- +kubectl taint nodes NODE_NAME node-role.kubernetes.io/master:NoSchedule- ``` ## `/usr` is mounted read-only on nodes {#usr-mounted-read-only} From 0b00e8e27f4f1e3a676b5d9076f16d4eea6a65e8 Mon Sep 17 00:00:00 2001 From: June Yi Date: Thu, 12 Mar 2020 00:21:37 +0900 Subject: [PATCH 031/140] Remove gochist from sig-docs-en-owners (#19600) --- OWNERS_ALIASES | 2 -- 1 file changed, 2 deletions(-) diff --git a/OWNERS_ALIASES b/OWNERS_ALIASES index 291db36bf2..16434ddbce 100644 --- a/OWNERS_ALIASES +++ b/OWNERS_ALIASES @@ -41,7 +41,6 @@ aliases: sig-docs-en-owners: # Admins for English content - bradtopol - daminisatya - - gochist - jaredbhatti - jimangel - kbarnard10 @@ -58,7 +57,6 @@ aliases: sig-docs-en-reviews: # PR reviews for English content - bradtopol - daminisatya - - gochist - jaredbhatti - jimangel - kbarnard10 From 96c6ac61c56fcb3db7141ec3111648d1997ac5b9 Mon Sep 17 00:00:00 2001 From: Ihor Sychevskyi <26163841+Arhell@users.noreply.github.com> Date: Wed, 11 Mar 2020 21:07:41 +0200 Subject: [PATCH 032/140] Fix hudo anchor in readme (id) (#19329) --- README-id.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/README-id.md b/README-id.md index b178b220bc..1d6b830f2e 100644 --- a/README-id.md +++ b/README-id.md @@ -9,7 +9,7 @@ Selamat datang! Repositori ini merupakan wadah bagi semua komponen yang dibutuhk Pertama, kamu dapat menekan tombol **Fork** yang berada pada bagian atas layar, untuk menyalin repositori pada akun Github-mu. Salinan ini disebut sebagai **fork**. Kamu dapat menambahkan konten pada **fork** yang kamu miliki, setelah kamu merasa cukup untuk menambahkan konten yang kamu miliki dan ingin memberikan konten tersebut pada kami, kamu dapat melihat **fork** yang telah kamu buat dan membuat **pull request** untuk memberi tahu kami bahwa kamu ingin menambahkan konten yang telah kamu buat. -Setelah kamu membuat sebuah **pull request**, seorang **reviewer** akan memberikan masukan terhadap konten yang kamu sediakan serta beberapa hal yang dapat kamu lakukan apabila perbaikan diperlukan terhadap konten yang telah kamu sediakan. Sebagai seorang yang membuat **pull request**, **sudah menjadi kewajiban kamu untuk melakukan modifikasi terhadap konten yang kamu berikan sesuai dengan masukan yang diberikan oleh seorang reviewer Kubernetes**. Perlu kamu ketahui bahwa kamu dapat saja memiliki lebih dari satu orang **reviewer Kubernetes** atau dalam kasus kamu bisa saja mendapatkan **reviewer Kubernetes** yang berbeda dengan **reviewer Kubernetes** awal yang ditugaskan untuk memberikan masukan terhadap konten yang kamu sediakan. Selain itu, seorang **reviewer Kubernetes** bisa saja meminta masukan teknis dari [reviewer teknis Kubernetes](https://github.com/kubernetes/website/wiki/Tech-reviewers) jika diperlukan. +Setelah kamu membuat sebuah **pull request**, seorang **reviewer** akan memberikan masukan terhadap konten yang kamu sediakan serta beberapa hal yang dapat kamu lakukan apabila perbaikan diperlukan terhadap konten yang telah kamu sediakan. Sebagai seorang yang membuat **pull request**, **sudah menjadi kewajiban kamu untuk melakukan modifikasi terhadap konten yang kamu berikan sesuai dengan masukan yang diberikan oleh seorang reviewer Kubernetes**. Perlu kamu ketahui bahwa kamu dapat saja memiliki lebih dari satu orang **reviewer Kubernetes** atau dalam kasus kamu bisa saja mendapatkan **reviewer Kubernetes** yang berbeda dengan **reviewer Kubernetes** awal yang ditugaskan untuk memberikan masukan terhadap konten yang kamu sediakan. Selain itu, seorang **reviewer Kubernetes** bisa saja meminta masukan teknis dari [reviewer teknis Kubernetes](https://github.com/kubernetes/website/wiki/Tech-reviewers) jika diperlukan. Untuk informasi lebih lanjut mengenai tata cara melakukan kontribusi, kamu dapat melihat tautan di bawah ini: @@ -21,11 +21,11 @@ Untuk informasi lebih lanjut mengenai tata cara melakukan kontribusi, kamu dapat ## Menjalankan Dokumentasi Kubernetes pada Mesin Lokal Kamu -Petunjuk yang disarankan untuk menjalankan Dokumentasi Kubernetes pada mesin lokal kamus adalah dengan menggunakan [Docker](https://docker.com) **image** yang memiliki **package** [Hugo](https://gohugo.io), **Hugo** sendiri merupakan generator website statis. +Petunjuk yang disarankan untuk menjalankan Dokumentasi Kubernetes pada mesin lokal kamus adalah dengan menggunakan [Docker](https://docker.com) **image** yang memiliki **package** [Hugo](https://gohugo.io), **Hugo** sendiri merupakan generator website statis. > Jika kamu menggunakan Windows, kamu mungkin membutuhkan beberapa langkah tambahan untuk melakukan instalasi perangkat lunak yang dibutuhkan. Instalasi ini dapat dilakukan dengan menggunakan [Chocolatey](https://chocolatey.org). `choco install make` -> Jika kamu ingin menjalankan **website** tanpa menggunakan **Docker**, kamu dapat melihat tautan berikut [Petunjuk untuk menjalankan website pada mesin lokal dengan menggunakan Hugo](#petunjuk-untuk-menjalankan-website-pada-mesin-lokal-denga-menggunakan-hugo) di bagian bawah. +> Jika kamu ingin menjalankan **website** tanpa menggunakan **Docker**, kamu dapat melihat tautan berikut [Petunjuk untuk menjalankan website pada mesin lokal dengan menggunakan Hugo](#petunjuk-untuk-menjalankan-website-pada-mesin-lokal-dengan-menggunakan-hugo) di bagian bawah. Jika kamu sudah memiliki **Docker** [yang sudah dapat digunakan](https://www.docker.com/get-started), kamu dapat melakukan **build** `kubernetes-hugo` **Docker image** secara lokal: @@ -44,7 +44,7 @@ Buka **browser** kamu ke http://localhost:1313 untuk melihat laman dokumentasi. ## Petunjuk untuk menjalankan website pada mesin lokal dengan menggunakan Hugo -Kamu dapat melihat [dokumentasi resmi Hugo](https://gohugo.io/getting-started/installing/) untuk mengetahui langkah yang diperlukan untuk melakukan instalasi **Hugo**. Pastikan kamu melakukan instalasi versi **Hugo** sesuai dengan versi yang tersedia pada **environment variable** `HUGO_VERSION` pada **file**[`netlify.toml`](netlify.toml#L9). +Kamu dapat melihat [dokumentasi resmi Hugo](https://gohugo.io/getting-started/installing/) untuk mengetahui langkah yang diperlukan untuk melakukan instalasi **Hugo**. Pastikan kamu melakukan instalasi versi **Hugo** sesuai dengan versi yang tersedia pada **environment variable** `HUGO_VERSION` pada **file**[`netlify.toml`](netlify.toml#L9). Untuk menjalankan laman pada mesin lokal setelah instalasi **Hugo**, kamu dapat menjalankan perintah berikut: From 9e91d16c6e55db6911a4a4ad912a681e1f2338d6 Mon Sep 17 00:00:00 2001 From: Jhon Mike Date: Wed, 11 Mar 2020 22:34:37 -0300 Subject: [PATCH 033/140] add new owner alias in pt lang (#19607) --- OWNERS_ALIASES | 2 ++ 1 file changed, 2 insertions(+) diff --git a/OWNERS_ALIASES b/OWNERS_ALIASES index 16434ddbce..43f27a88b2 100644 --- a/OWNERS_ALIASES +++ b/OWNERS_ALIASES @@ -190,10 +190,12 @@ aliases: - femrtnz - jcjesus - devlware + - jhonmike sig-docs-pt-reviews: # PR reviews for Portugese content - femrtnz - jcjesus - devlware + - jhonmike sig-docs-vi-owners: # Admins for Vietnamese content - huynguyennovem - ngtuna From 8195f2baff7c63c28edbe0e4e97286c7e58c34b0 Mon Sep 17 00:00:00 2001 From: chentanjun <2799194073@qq.com> Date: Thu, 12 Mar 2020 09:38:37 +0800 Subject: [PATCH 034/140] sync zh content/zh/docs/setup/production-environment/tools/kubeadm/troubleshooting-kubeadm.md (#19602) --- .../tools/kubeadm/troubleshooting-kubeadm.md | 1300 ++++++++--------- 1 file changed, 650 insertions(+), 650 deletions(-) diff --git a/content/zh/docs/setup/production-environment/tools/kubeadm/troubleshooting-kubeadm.md b/content/zh/docs/setup/production-environment/tools/kubeadm/troubleshooting-kubeadm.md index 8b514ad40b..1e3ab441b4 100644 --- a/content/zh/docs/setup/production-environment/tools/kubeadm/troubleshooting-kubeadm.md +++ b/content/zh/docs/setup/production-environment/tools/kubeadm/troubleshooting-kubeadm.md @@ -1,650 +1,650 @@ ---- -title: 对 kubeadm 进行故障排查 -content_template: templates/concept -weight: 20 ---- - - -{{% capture overview %}} - - -与任何程序一样，您可能会在安装或者运行 kubeadm 时遇到错误。 -本文列举了一些常见的故障场景，并提供可帮助您理解和解决这些问题的步骤。 - -如果您的问题未在下面列出，请执行以下步骤： - -- 如果您认为问题是 kubeadm 的错误： - - 转到 [github.com/kubernetes/kubeadm](https://github.com/kubernetes/kubeadm/issues) 并搜索存在的问题。 - - 如果没有问题，请 [打开](https://github.com/kubernetes/kubeadm/issues/new) 并遵循问题模板。 - -- 如果您对 kubeadm 的工作方式有疑问，可以在 [Slack](http://slack.k8s.io/) 上的 #kubeadm 频道提问， -或者在 [StackOverflow](https://stackoverflow.com/questions/tagged/kubernetes) 上提问。 -请加入相关标签，例如 `#kubernetes` 和 `#kubeadm`，这样其他人可以帮助您。 - -{{% /capture %}} - -{{% capture body %}} - - -## 在安装过程中没有找到 `ebtables` 或者其他类似的可执行文件 - -如果在运行 `kubeadm init` 命令时，遇到以下的警告 - -```sh -[preflight] WARNING: ebtables not found in system path -[preflight] WARNING: ethtool not found in system path -``` - -那么或许在您的节点上缺失 `ebtables`、`ethtool` 或者类似的可执行文件。 -您可以使用以下命令安装它们： - -- 对于 Ubuntu/Debian 用户，运行 `apt install ebtables ethtool` 命令。 -- 对于 CentOS/Fedora 用户，运行 `yum install ebtables ethtool` 命令。 - - -## 在安装过程中，kubeadm 一直等待控制平面就绪 - -如果您注意到 `kubeadm init` 在打印以下行后挂起： - -```sh -[apiclient] Created API client, waiting for the control plane to become ready -``` - - -这可能是由许多问题引起的。最常见的是： - -- 网络连接问题。在继续之前，请检查您的计算机是否具有全部联通的网络连接。 -- kubelet 的默认 cgroup 驱动程序配置不同于 Docker 使用的配置。 - 检查系统日志文件 (例如 `/var/log/message`) 或检查 `journalctl -u kubelet` 的输出。 如果您看见以下内容： - - ```shell - error: failed to run Kubelet: failed to create kubelet: - misconfiguration: kubelet cgroup driver: "systemd" is different from docker cgroup driver: "cgroupfs" - ``` - - 有两种常见方法可解决 cgroup 驱动程序问题： - - 1. 按照 [此处](/docs/setup/production-environment/container-runtimes/#docker) 的说明再次安装 Docker。 - - 1. 更改 kubelet 配置以手动匹配 Docker cgroup 驱动程序，您可以参考 - [在主节点上配置 kubelet 要使用的 cgroup 驱动程序](/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#configure-cgroup-driver-used-by-kubelet-on-master-node) - -- 控制平面上的 Docker 容器持续进入崩溃状态或（因其他原因）挂起。您可以运行 `docker ps` 命令来检查以及 `docker logs` 命令来检视每个容器的运行日志。 - - -## 当删除托管容器时 kubeadm 阻塞 - -如果 Docker 停止并且不删除 Kubernetes 所管理的所有容器，可能发生以下情况： - -```bash -sudo kubeadm reset -[preflight] Running pre-flight checks -[reset] Stopping the kubelet service -[reset] Unmounting mounted directories in "/var/lib/kubelet" -[reset] Removing kubernetes-managed containers -(block) -``` - -一个可行的解决方案是重新启动 Docker 服务，然后重新运行 `kubeadm reset`： - -```bash -sudo systemctl restart docker.service -sudo kubeadm reset -``` - -检查 docker 的日志也可能有用： - -```sh -journalctl -ul docker -``` - - -## Pods 处于 `RunContainerError`、`CrashLoopBackOff` 或者 `Error` 状态 - -在 `kubeadm init` 命令运行后，系统中不应该有 pods 处于这类状态。 - -- 在 `kubeadm init` 命令执行完后，如果有 pods 处于这些状态之一，请在 kubeadm - 仓库提起一个 issue。`coredns` (或者 `kube-dns`) 应该处于 `Pending` 状态， - 直到您部署了网络解决方案为止。 -- 如果在部署完网络解决方案之后，有 Pods 处于 `RunContainerError`、`CrashLoopBackOff` - 或 `Error` 状态之一，并且`coredns` （或者 `kube-dns`）仍处于 `Pending` 状态， - 那很可能是您安装的网络解决方案由于某种原因无法工作。您或许需要授予它更多的 - RBAC 特权或使用较新的版本。请在 Pod Network 提供商的问题跟踪器中提交问题， - 然后在此处分类问题。 -- 如果您安装的 Docker 版本早于 1.12.1，请在使用 `systemd` 来启动 `dockerd` 和重启 `docker` 时， - 删除 `MountFlags=slave` 选项。 - 您可以在 `/usr/lib/systemd/system/docker.service` 中看到 MountFlags。 - MountFlags 可能会干扰 Kubernetes 挂载的卷， 并使 Pods 处于 `CrashLoopBackOff` 状态。 - 当 Kubernetes 不能找到 `var/run/secrets/kubernetes.io/serviceaccount` 文件时会发生错误。 - - -## `coredns` （或 `kube-dns`）停滞在 `Pending` 状态 - -这一行为是 **预期之中** 的，因为系统就是这么设计的。 -kubeadm 的网络供应商是中立的，因此管理员应该选择 [安装 pod 的网络解决方案](/docs/concepts/cluster-administration/addons/)。 -您必须完成 Pod 的网络配置，然后才能完全部署 CoreDNS。 -在网络被配置好之前，DNS 组件会一直处于 `Pending` 状态。 - - -## `HostPort` 服务无法工作 - -此 `HostPort` 和 `HostIP` 功能是否可用取决于您的 Pod 网络配置。请联系 Pod 解决方案的作者， -以确认 `HostPort` 和 `HostIP` 功能是否可用。 - -已验证 Calico、Canal 和 Flannel CNI 驱动程序支持 HostPort。 - -有关更多信息，请参考 [CNI portmap 文档](https://github.com/containernetworking/plugins/blob/master/plugins/meta/portmap/README.md). - -如果您的网络提供商不支持 portmap CNI 插件，您或许需要使用 [NodePort 服务的功能](/docs/concepts/services-networking/service/#nodeport) 或者使用 `HostNetwork=true`。 - - -## 无法通过其服务 IP 访问 Pod - -- 许多网络附加组件尚未启用 [hairpin 模式](/docs/tasks/debug-application-cluster/debug-service/#a-pod-cannot-reach-itself-via-service-ip) - 该模式允许 Pod 通过其服务 IP 进行访问。这是与 [CNI](https://github.com/containernetworking/cni/issues/476) 有关的问题。请与网络附加组件提供商联系，以获取他们所提供的 hairpin 模式的最新状态。 - -- 如果您正在使用 VirtualBox (直接使用或者通过 Vagrant 使用)，您需要 - 确保 `hostname -i` 返回一个可路由的 IP 地址。默认情况下，第一个接口连接不能路由的仅主机网络。 - 解决方法是修改 `/etc/hosts`，请参考示例 [Vagrantfile](https://github.com/errordeveloper/k8s-playground/blob/22dd39dfc06111235620e6c4404a96ae146f26fd/Vagrantfile#L11)。 - - -## TLS 证书错误 - -以下错误指出证书可能不匹配。 - -```none -# kubectl get pods -Unable to connect to the server: x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubernetes") -``` - -- 验证 `$HOME/.kube/config` 文件是否包含有效证书，并 - 在必要时重新生成证书。在 kubeconfig 文件中的证书是 base64 编码的。 - 该 `base64 -d` 命令可以用来解码证书，`openssl x509 -text -noout` 命令 - 可以用于查看证书信息。 -- 使用如下方法取消设置 `KUBECONFIG` 环境变量的值： - - ```sh - unset KUBECONFIG - ``` - - 或者将其设置为默认的 `KUBECONFIG` 位置： - - ```sh - export KUBECONFIG=/etc/kubernetes/admin.conf - ``` - -- 另一个方法是覆盖 `kubeconfig` 的现有用户 "管理员" ： - - ```sh - mv $HOME/.kube $HOME/.kube.bak - mkdir $HOME/.kube - sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config - sudo chown $(id -u):$(id -g) $HOME/.kube/config - ``` - - -## 在 Vagrant 中使用 flannel 作为 pod 网络时的默认 NIC - -以下错误可能表明 Pod 网络中出现问题： - -```sh -Error from server (NotFound): the server could not find the requested resource -``` - -- 如果你正在 Vagrant 中使用 flannel 作为 pod 网络，则必须指定 flannel 的默认接口名称。 - - Vagrant 通常为所有 VM 分配两个接口。第一个为所有主机分配了 IP 地址 `10.0.2.15`，用于获得 NATed 的外部流量。 - - 这可能会导致 flannel 出现问题，它默认为主机上的第一个接口。这导致所有主机认为它们具有 - 相同的公共 IP 地址。为防止这种情况，传递 `--iface eth1` 标志给 flannel 以便选择第二个接口。 - - -## 容器使用的非公共 IP - -在某些情况下 `kubectl logs` 和 `kubectl run` 命令或许会返回以下错误，即便除此之外集群一切功能正常： - -```sh -Error from server: Get https://10.19.0.41:10250/containerLogs/default/mysql-ddc65b868-glc5m/mysql: dial tcp 10.19.0.41:10250: getsockopt: no route to host -``` - -- 这或许是由于 Kubernetes 使用的 IP 无法与看似相同的子网上的其他 IP 进行通信的缘故， -可能是由机器提供商的政策所导致的。 -- Digital Ocean 既分配一个共有 IP 给 `eth0`，也分配一个私有 IP 在内部用作其浮动 IP 功能的锚点， -然而 `kubelet` 将选择后者作为节点的 `InternalIP` 而不是公共 IP - - 使用 `ip addr show` 命令代替 `ifconfig` 命令去检查这种情况，因为 `ifconfig` 命令 - 不会显示有问题的别名 IP 地址。或者指定的 Digital Ocean 的 API 端口允许从 droplet 中 - 查询 anchor IP： - - ```sh - curl http://169.254.169.254/metadata/v1/interfaces/public/0/anchor_ipv4/address - ``` - - 解决方法是通知 `kubelet` 使用哪个 `--node-ip`。当使用 Digital Ocean 时，可以是公网IP（分配给 `eth0`的）， - 或者是私网IP（分配给 `eth1` 的）。私网 IP 是可选的。 - 这个 [`KubeletExtraArgs` section of the kubeadm `NodeRegistrationOptions` structure](https://github.com/kubernetes/kubernetes/blob/release-1.13/cmd/kubeadm/app/apis/kubeadm/v1beta1/types.go) 被用来处理这种情况。 - - 然后重启 `kubelet`： - - ```sh - systemctl daemon-reload - systemctl restart kubelet - ``` - -## `coredns` pods 有 `CrashLoopBackOff` 或者 `Error` 状态 - -如果有些节点运行的是旧版本的 Docker，同时启用了 SELinux，您或许会遇到 `coredns` pods 无法启动的情况。 -要解决此问题，您可以尝试以下选项之一： - -- 升级到 [Docker 的较新版本](/docs/setup/production-environment/container-runtimes/#docker)。 - -- [禁用 SELinux](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/security-enhanced_linux/sect-security-enhanced_linux-enabling_and_disabling_selinux-disabling_selinux). -- 修改 `coredns` 部署以设置 `allowPrivilegeEscalation` 为 `true`： - -```bash -kubectl -n kube-system get deployment coredns -o yaml | \ - sed 's/allowPrivilegeEscalation: false/allowPrivilegeEscalation: true/g' | \ - kubectl apply -f - -``` - -CoreDNS 处于 `CrashLoopBackOff` 时的另一个原因是当 Kubernetes 中部署的 CoreDNS Pod 检测 -到环路时。[有许多解决方法](https://github.com/coredns/coredns/tree/master/plugin/loop#troubleshooting-loops-in-kubernetes-clusters) -可以避免在每次 CoreDNS 监测到循环并退出时，Kubernetes 尝试重启 CoreDNS Pod 的情况。 - -{{< warning >}} - -**警告**：禁用 SELinux 或设置 `allowPrivilegeEscalation` 为 `true` 可能会损害集群的安全性。 -{{< /warning >}} - - -## etcd pods 持续重启 - -如果您遇到以下错误： - -``` -rpc error: code = 2 desc = oci runtime error: exec failed: container_linux.go:247: starting container process caused "process_linux.go:110: decoding init error from pipe caused \"read parent: connection reset by peer\"" -``` - -如果您使用 Docker 1.13.1.84 运行 CentOS 7 就会出现这种问题。 -此版本的 Docker 会阻止 kubelet 在 etcd 容器中执行。 - -为解决此问题，请选择以下选项之一： - -- 回滚到早期版本的 Docker，例如 1.13.1-75 -``` -yum downgrade docker-1.13.1-75.git8633870.el7.centos.x86_64 docker-client-1.13.1-75.git8633870.el7.centos.x86_64 docker-common-1.13.1-75.git8633870.el7.centos.x86_64 -``` - -- 安装较新的推荐版本之一，例如 18.06: -```bash -sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo -yum install docker-ce-18.06.1.ce-3.el7.x86_64 -``` - - -## 无法将以逗号分隔的值列表传递给 `--component-extra-args` 标志内的参数 - -`kubeadm init` 标志例如 `--component-extra-args` 允许您将自定义参数传递给像 kube-apiserver 这样的控制平面组件。然而，由于解析 (`mapStringString`) 的基础类型值，此机制将受到限制。 - -如果您决定传递一个支持多个逗号分隔值（例如 `--apiserver-extra-args "enable-admission-plugins=LimitRanger,NamespaceExists"`）参数，将出现 `flag: malformed pair, expect string=string` 错误。 -发生这种问题是因为参数列表 `--apiserver-extra-args` 预期的是 `key=value` 形式，而这里的 `NamespacesExists` 被误认为是缺少取值的键名。 - -一种解决方法是尝试分离 `key=value` 对，像这样： -`--apiserver-extra-args "enable-admission-plugins=LimitRanger,enable-admission-plugins=NamespaceExists"` -但这将导致键 `enable-admission-plugins` 仅有值 `NamespaceExists`。 - -已知的解决方法是使用 kubeadm [配置文件](/docs/setup/production-environment/tools/kubeadm/control-plane-flags/#apiserver-flags)。 - - -## 在节点被云控制管理器初始化之前，kube-proxy 就被调度了 - -在云环境场景中，可能出现在云控制管理器完成节点地址初始化之前，kube-proxy 就被调度到新节点了。 -这会导致 kube-proxy 无法正确获取节点的 IP 地址，并对管理负载平衡器的代理功能产生连锁反应。 - -在 kube-proxy Pod 中可以看到以下错误： -``` -server.go:610] Failed to retrieve node IP: host IP unknown; known addresses: [] -proxier.go:340] invalid nodeIP, initializing kube-proxy with 127.0.0.1 as nodeIP -``` - -一种已知的解决方案是修补 kube-proxy DaemonSet，以允许在控制平面节点上调度它，而不管它们的条件如何，将其与其他节点保持隔离，直到它们的初始保护条件消除： -``` -kubectl -n kube-system patch ds kube-proxy -p='{ "spec": { "template": { "spec": { "tolerations": [ { "key": "CriticalAddonsOnly", "operator": "Exists" }, { "effect": "NoSchedule", "key": "node-role.kubernetes.io/master" } ] } } } }' -``` - -此问题的跟踪 [在这里](https://github.com/kubernetes/kubeadm/issues/1027)。 - - -## NodeRegistration.Taints 字段在编组 kubeadm 配置时丢失 - -*注意：这个 [问题](https://github.com/kubernetes/kubeadm/issues/1358) 仅适用于操控 kubeadm 数据类型的工具（例如，YAML 配置文件）。它将在 kubeadm API v1beta2 修复。* - -默认情况下，kubeadm 将 `role.kubernetes.io/master:NoSchedule` 污点应用于控制平面节点。 -如果您希望 kubeadm 不污染控制平面节点，并将 `InitConfiguration.NodeRegistration.Taints` 设置成空切片，则应在编组时省略该字段。 -如果省略该字段，则 kubeadm 将应用默认污点。 - -至少有两种解决方法： - -1. 使用 `role.kubernetes.io/master:PreferNoSchedule` 污点代替空切片。 -除非其他节点具有容量，[否则将在主节点上调度 Pods](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/)。 - -2. 在 kubeadm init 退出后删除污点： -```bash -kubectl taint nodes NODE_NAME role.kubernetes.io/master:NoSchedule- -``` - -{{% /capture %}} +--- +title: 对 kubeadm 进行故障排查 +content_template: templates/concept +weight: 20 +--- + + +{{% capture overview %}} + + +与任何程序一样，您可能会在安装或者运行 kubeadm 时遇到错误。 +本文列举了一些常见的故障场景，并提供可帮助您理解和解决这些问题的步骤。 + +如果您的问题未在下面列出，请执行以下步骤： + +- 如果您认为问题是 kubeadm 的错误： + - 转到 [github.com/kubernetes/kubeadm](https://github.com/kubernetes/kubeadm/issues) 并搜索存在的问题。 + - 如果没有问题，请 [打开](https://github.com/kubernetes/kubeadm/issues/new) 并遵循问题模板。 + +- 如果您对 kubeadm 的工作方式有疑问，可以在 [Slack](http://slack.k8s.io/) 上的 #kubeadm 频道提问， +或者在 [StackOverflow](https://stackoverflow.com/questions/tagged/kubernetes) 上提问。 +请加入相关标签，例如 `#kubernetes` 和 `#kubeadm`，这样其他人可以帮助您。 + +{{% /capture %}} + +{{% capture body %}} + + +## 在安装过程中没有找到 `ebtables` 或者其他类似的可执行文件 + +如果在运行 `kubeadm init` 命令时，遇到以下的警告 + +```sh +[preflight] WARNING: ebtables not found in system path +[preflight] WARNING: ethtool not found in system path +``` + +那么或许在您的节点上缺失 `ebtables`、`ethtool` 或者类似的可执行文件。 +您可以使用以下命令安装它们： + +- 对于 Ubuntu/Debian 用户，运行 `apt install ebtables ethtool` 命令。 +- 对于 CentOS/Fedora 用户，运行 `yum install ebtables ethtool` 命令。 + + +## 在安装过程中，kubeadm 一直等待控制平面就绪 + +如果您注意到 `kubeadm init` 在打印以下行后挂起： + +```sh +[apiclient] Created API client, waiting for the control plane to become ready +``` + + +这可能是由许多问题引起的。最常见的是： + +- 网络连接问题。在继续之前，请检查您的计算机是否具有全部联通的网络连接。 +- kubelet 的默认 cgroup 驱动程序配置不同于 Docker 使用的配置。 + 检查系统日志文件 (例如 `/var/log/message`) 或检查 `journalctl -u kubelet` 的输出。 如果您看见以下内容： + + ```shell + error: failed to run Kubelet: failed to create kubelet: + misconfiguration: kubelet cgroup driver: "systemd" is different from docker cgroup driver: "cgroupfs" + ``` + + 有两种常见方法可解决 cgroup 驱动程序问题： + + 1. 按照 [此处](/docs/setup/production-environment/container-runtimes/#docker) 的说明再次安装 Docker。 + + 1. 更改 kubelet 配置以手动匹配 Docker cgroup 驱动程序，您可以参考 + [在主节点上配置 kubelet 要使用的 cgroup 驱动程序](/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#configure-cgroup-driver-used-by-kubelet-on-master-node) + +- 控制平面上的 Docker 容器持续进入崩溃状态或（因其他原因）挂起。您可以运行 `docker ps` 命令来检查以及 `docker logs` 命令来检视每个容器的运行日志。 + + +## 当删除托管容器时 kubeadm 阻塞 + +如果 Docker 停止并且不删除 Kubernetes 所管理的所有容器，可能发生以下情况： + +```bash +sudo kubeadm reset +[preflight] Running pre-flight checks +[reset] Stopping the kubelet service +[reset] Unmounting mounted directories in "/var/lib/kubelet" +[reset] Removing kubernetes-managed containers +(block) +``` + +一个可行的解决方案是重新启动 Docker 服务，然后重新运行 `kubeadm reset`： + +```bash +sudo systemctl restart docker.service +sudo kubeadm reset +``` + +检查 docker 的日志也可能有用： + +```sh +journalctl -ul docker +``` + + +## Pods 处于 `RunContainerError`、`CrashLoopBackOff` 或者 `Error` 状态 + +在 `kubeadm init` 命令运行后，系统中不应该有 pods 处于这类状态。 + +- 在 `kubeadm init` 命令执行完后，如果有 pods 处于这些状态之一，请在 kubeadm + 仓库提起一个 issue。`coredns` (或者 `kube-dns`) 应该处于 `Pending` 状态， + 直到您部署了网络解决方案为止。 +- 如果在部署完网络解决方案之后，有 Pods 处于 `RunContainerError`、`CrashLoopBackOff` + 或 `Error` 状态之一，并且`coredns` （或者 `kube-dns`）仍处于 `Pending` 状态， + 那很可能是您安装的网络解决方案由于某种原因无法工作。您或许需要授予它更多的 + RBAC 特权或使用较新的版本。请在 Pod Network 提供商的问题跟踪器中提交问题， + 然后在此处分类问题。 +- 如果您安装的 Docker 版本早于 1.12.1，请在使用 `systemd` 来启动 `dockerd` 和重启 `docker` 时， + 删除 `MountFlags=slave` 选项。 + 您可以在 `/usr/lib/systemd/system/docker.service` 中看到 MountFlags。 + MountFlags 可能会干扰 Kubernetes 挂载的卷， 并使 Pods 处于 `CrashLoopBackOff` 状态。 + 当 Kubernetes 不能找到 `var/run/secrets/kubernetes.io/serviceaccount` 文件时会发生错误。 + + +## `coredns` （或 `kube-dns`）停滞在 `Pending` 状态 + +这一行为是 **预期之中** 的，因为系统就是这么设计的。 +kubeadm 的网络供应商是中立的，因此管理员应该选择 [安装 pod 的网络解决方案](/docs/concepts/cluster-administration/addons/)。 +您必须完成 Pod 的网络配置，然后才能完全部署 CoreDNS。 +在网络被配置好之前，DNS 组件会一直处于 `Pending` 状态。 + + +## `HostPort` 服务无法工作 + +此 `HostPort` 和 `HostIP` 功能是否可用取决于您的 Pod 网络配置。请联系 Pod 解决方案的作者， +以确认 `HostPort` 和 `HostIP` 功能是否可用。 + +已验证 Calico、Canal 和 Flannel CNI 驱动程序支持 HostPort。 + +有关更多信息，请参考 [CNI portmap 文档](https://github.com/containernetworking/plugins/blob/master/plugins/meta/portmap/README.md). + +如果您的网络提供商不支持 portmap CNI 插件，您或许需要使用 [NodePort 服务的功能](/docs/concepts/services-networking/service/#nodeport) 或者使用 `HostNetwork=true`。 + + +## 无法通过其服务 IP 访问 Pod + +- 许多网络附加组件尚未启用 [hairpin 模式](/docs/tasks/debug-application-cluster/debug-service/#a-pod-cannot-reach-itself-via-service-ip) + 该模式允许 Pod 通过其服务 IP 进行访问。这是与 [CNI](https://github.com/containernetworking/cni/issues/476) 有关的问题。请与网络附加组件提供商联系，以获取他们所提供的 hairpin 模式的最新状态。 + +- 如果您正在使用 VirtualBox (直接使用或者通过 Vagrant 使用)，您需要 + 确保 `hostname -i` 返回一个可路由的 IP 地址。默认情况下，第一个接口连接不能路由的仅主机网络。 + 解决方法是修改 `/etc/hosts`，请参考示例 [Vagrantfile](https://github.com/errordeveloper/k8s-playground/blob/22dd39dfc06111235620e6c4404a96ae146f26fd/Vagrantfile#L11)。 + + +## TLS 证书错误 + +以下错误指出证书可能不匹配。 + +```none +# kubectl get pods +Unable to connect to the server: x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubernetes") +``` + +- 验证 `$HOME/.kube/config` 文件是否包含有效证书，并 + 在必要时重新生成证书。在 kubeconfig 文件中的证书是 base64 编码的。 + 该 `base64 -d` 命令可以用来解码证书，`openssl x509 -text -noout` 命令 + 可以用于查看证书信息。 +- 使用如下方法取消设置 `KUBECONFIG` 环境变量的值： + + ```sh + unset KUBECONFIG + ``` + + 或者将其设置为默认的 `KUBECONFIG` 位置： + + ```sh + export KUBECONFIG=/etc/kubernetes/admin.conf + ``` + +- 另一个方法是覆盖 `kubeconfig` 的现有用户 "管理员" ： + + ```sh + mv $HOME/.kube $HOME/.kube.bak + mkdir $HOME/.kube + sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config + sudo chown $(id -u):$(id -g) $HOME/.kube/config + ``` + + +## 在 Vagrant 中使用 flannel 作为 pod 网络时的默认 NIC + +以下错误可能表明 Pod 网络中出现问题： + +```sh +Error from server (NotFound): the server could not find the requested resource +``` + +- 如果你正在 Vagrant 中使用 flannel 作为 pod 网络，则必须指定 flannel 的默认接口名称。 + + Vagrant 通常为所有 VM 分配两个接口。第一个为所有主机分配了 IP 地址 `10.0.2.15`，用于获得 NATed 的外部流量。 + + 这可能会导致 flannel 出现问题，它默认为主机上的第一个接口。这导致所有主机认为它们具有 + 相同的公共 IP 地址。为防止这种情况，传递 `--iface eth1` 标志给 flannel 以便选择第二个接口。 + + +## 容器使用的非公共 IP + +在某些情况下 `kubectl logs` 和 `kubectl run` 命令或许会返回以下错误，即便除此之外集群一切功能正常： + +```sh +Error from server: Get https://10.19.0.41:10250/containerLogs/default/mysql-ddc65b868-glc5m/mysql: dial tcp 10.19.0.41:10250: getsockopt: no route to host +``` + +- 这或许是由于 Kubernetes 使用的 IP 无法与看似相同的子网上的其他 IP 进行通信的缘故， +可能是由机器提供商的政策所导致的。 +- Digital Ocean 既分配一个共有 IP 给 `eth0`，也分配一个私有 IP 在内部用作其浮动 IP 功能的锚点， +然而 `kubelet` 将选择后者作为节点的 `InternalIP` 而不是公共 IP + + 使用 `ip addr show` 命令代替 `ifconfig` 命令去检查这种情况，因为 `ifconfig` 命令 + 不会显示有问题的别名 IP 地址。或者指定的 Digital Ocean 的 API 端口允许从 droplet 中 + 查询 anchor IP： + + ```sh + curl http://169.254.169.254/metadata/v1/interfaces/public/0/anchor_ipv4/address + ``` + + 解决方法是通知 `kubelet` 使用哪个 `--node-ip`。当使用 Digital Ocean 时，可以是公网IP（分配给 `eth0`的）， + 或者是私网IP（分配给 `eth1` 的）。私网 IP 是可选的。 + 这个 [`KubeletExtraArgs` section of the kubeadm `NodeRegistrationOptions` structure](https://github.com/kubernetes/kubernetes/blob/release-1.13/cmd/kubeadm/app/apis/kubeadm/v1beta1/types.go) 被用来处理这种情况。 + + 然后重启 `kubelet`： + + ```sh + systemctl daemon-reload + systemctl restart kubelet + ``` + +## `coredns` pods 有 `CrashLoopBackOff` 或者 `Error` 状态 + +如果有些节点运行的是旧版本的 Docker，同时启用了 SELinux，您或许会遇到 `coredns` pods 无法启动的情况。 +要解决此问题，您可以尝试以下选项之一： + +- 升级到 [Docker 的较新版本](/docs/setup/production-environment/container-runtimes/#docker)。 + +- [禁用 SELinux](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/security-enhanced_linux/sect-security-enhanced_linux-enabling_and_disabling_selinux-disabling_selinux). +- 修改 `coredns` 部署以设置 `allowPrivilegeEscalation` 为 `true`： + +```bash +kubectl -n kube-system get deployment coredns -o yaml | \ + sed 's/allowPrivilegeEscalation: false/allowPrivilegeEscalation: true/g' | \ + kubectl apply -f - +``` + +CoreDNS 处于 `CrashLoopBackOff` 时的另一个原因是当 Kubernetes 中部署的 CoreDNS Pod 检测 +到环路时。[有许多解决方法](https://github.com/coredns/coredns/tree/master/plugin/loop#troubleshooting-loops-in-kubernetes-clusters) +可以避免在每次 CoreDNS 监测到循环并退出时，Kubernetes 尝试重启 CoreDNS Pod 的情况。 + +{{< warning >}} + +**警告**：禁用 SELinux 或设置 `allowPrivilegeEscalation` 为 `true` 可能会损害集群的安全性。 +{{< /warning >}} + + +## etcd pods 持续重启 + +如果您遇到以下错误： + +``` +rpc error: code = 2 desc = oci runtime error: exec failed: container_linux.go:247: starting container process caused "process_linux.go:110: decoding init error from pipe caused \"read parent: connection reset by peer\"" +``` + +如果您使用 Docker 1.13.1.84 运行 CentOS 7 就会出现这种问题。 +此版本的 Docker 会阻止 kubelet 在 etcd 容器中执行。 + +为解决此问题，请选择以下选项之一： + +- 回滚到早期版本的 Docker，例如 1.13.1-75 +``` +yum downgrade docker-1.13.1-75.git8633870.el7.centos.x86_64 docker-client-1.13.1-75.git8633870.el7.centos.x86_64 docker-common-1.13.1-75.git8633870.el7.centos.x86_64 +``` + +- 安装较新的推荐版本之一，例如 18.06: +```bash +sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo +yum install docker-ce-18.06.1.ce-3.el7.x86_64 +``` + + +## 无法将以逗号分隔的值列表传递给 `--component-extra-args` 标志内的参数 + +`kubeadm init` 标志例如 `--component-extra-args` 允许您将自定义参数传递给像 kube-apiserver 这样的控制平面组件。然而，由于解析 (`mapStringString`) 的基础类型值，此机制将受到限制。 + +如果您决定传递一个支持多个逗号分隔值（例如 `--apiserver-extra-args "enable-admission-plugins=LimitRanger,NamespaceExists"`）参数，将出现 `flag: malformed pair, expect string=string` 错误。 +发生这种问题是因为参数列表 `--apiserver-extra-args` 预期的是 `key=value` 形式，而这里的 `NamespacesExists` 被误认为是缺少取值的键名。 + +一种解决方法是尝试分离 `key=value` 对，像这样： +`--apiserver-extra-args "enable-admission-plugins=LimitRanger,enable-admission-plugins=NamespaceExists"` +但这将导致键 `enable-admission-plugins` 仅有值 `NamespaceExists`。 + +已知的解决方法是使用 kubeadm [配置文件](/docs/setup/production-environment/tools/kubeadm/control-plane-flags/#apiserver-flags)。 + + +## 在节点被云控制管理器初始化之前，kube-proxy 就被调度了 + +在云环境场景中，可能出现在云控制管理器完成节点地址初始化之前，kube-proxy 就被调度到新节点了。 +这会导致 kube-proxy 无法正确获取节点的 IP 地址，并对管理负载平衡器的代理功能产生连锁反应。 + +在 kube-proxy Pod 中可以看到以下错误： +``` +server.go:610] Failed to retrieve node IP: host IP unknown; known addresses: [] +proxier.go:340] invalid nodeIP, initializing kube-proxy with 127.0.0.1 as nodeIP +``` + +一种已知的解决方案是修补 kube-proxy DaemonSet，以允许在控制平面节点上调度它，而不管它们的条件如何，将其与其他节点保持隔离，直到它们的初始保护条件消除： +``` +kubectl -n kube-system patch ds kube-proxy -p='{ "spec": { "template": { "spec": { "tolerations": [ { "key": "CriticalAddonsOnly", "operator": "Exists" }, { "effect": "NoSchedule", "key": "node-role.kubernetes.io/master" } ] } } } }' +``` + +此问题的跟踪 [在这里](https://github.com/kubernetes/kubeadm/issues/1027)。 + + +## NodeRegistration.Taints 字段在编组 kubeadm 配置时丢失 + +*注意：这个 [问题](https://github.com/kubernetes/kubeadm/issues/1358) 仅适用于操控 kubeadm 数据类型的工具（例如，YAML 配置文件）。它将在 kubeadm API v1beta2 修复。* + +默认情况下，kubeadm 将 `node-role.kubernetes.io/master:NoSchedule` 污点应用于控制平面节点。 +如果您希望 kubeadm 不污染控制平面节点，并将 `InitConfiguration.NodeRegistration.Taints` 设置成空切片，则应在编组时省略该字段。 +如果省略该字段，则 kubeadm 将应用默认污点。 + +至少有两种解决方法： + +1. 使用 `node-role.kubernetes.io/master:PreferNoSchedule` 污点代替空切片。 +除非其他节点具有容量，[否则将在主节点上调度 Pods](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/)。 + +2. 在 kubeadm init 退出后删除污点： +```bash +kubectl taint nodes NODE_NAME node-role.kubernetes.io/master:NoSchedule- +``` + +{{% /capture %}} From 16ff8f55a0bd3b815967f82f29eb6c85c86b708e Mon Sep 17 00:00:00 2001 From: Sophy417 <53026875+Sophy417@users.noreply.github.com> Date: Thu, 12 Mar 2020 17:54:38 +0800 Subject: [PATCH 035/140] Create api-group.md (#19537) * Create api-group.md * Update api-group.md --- .../zh/docs/reference/glossary/api-group.md | 45 +++++++++++++++++++ 1 file changed, 45 insertions(+) create mode 100644 content/zh/docs/reference/glossary/api-group.md diff --git a/content/zh/docs/reference/glossary/api-group.md b/content/zh/docs/reference/glossary/api-group.md new file mode 100644 index 0000000000..4687391f62 --- /dev/null +++ b/content/zh/docs/reference/glossary/api-group.md @@ -0,0 +1,45 @@ +--- +title: API Group +id: api-group +date: 2019-09-02 +full_link: /docs/concepts/overview/kubernetes-api/#api-groups +short_description: > + Kubernetes API 中的一组相关路径 + +aka: +tags: +- fundamental +- architecture +--- + + + +Kubernetes API 中的一组相关路径。 + + + + +通过更改 API server 的配置，可以启用或禁用每个 API Group。你还可以禁用或启用指向特定资源的路径。API group 使扩展 Kubernetes API 更加的容易。API group 在 REST 路径和序列化对象的 `apiVersion` 字段中指定。 + + +* 阅读 [API Group](/docs/concepts/overview/kubernetes-api/#api-groups) 了解更多信息。 From f9e2dbbed92ee63bf9ad5f84f8e6712e5a00238d Mon Sep 17 00:00:00 2001 From: Franco Giovanolli Date: Thu, 12 Mar 2020 10:58:39 -0300 Subject: [PATCH 036/140] Delete insecure external link. (#19567) * Delete insecure external link. Delete insecure external link. The website hasn't an HTTPS version. Also, the project (https://github.com/kubernetes-retired/contrib/tree/master/node-perf-dash) will be archived 2 years ago. * Update reserve-compute-resources.md * Remove link to blog for the same reason. --- .../tasks/administer-cluster/reserve-compute-resources.md | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/content/en/docs/tasks/administer-cluster/reserve-compute-resources.md b/content/en/docs/tasks/administer-cluster/reserve-compute-resources.md index e4fe4a5ac9..39d0e825b8 100644 --- a/content/en/docs/tasks/administer-cluster/reserve-compute-resources.md +++ b/content/en/docs/tasks/administer-cluster/reserve-compute-resources.md @@ -94,13 +94,7 @@ be configured to use the `systemd` cgroup driver. `kube-reserved` is meant to capture resource reservation for kubernetes system daemons like the `kubelet`, `container runtime`, `node problem detector`, etc. It is not meant to reserve resources for system daemons that are run as pods. -`kube-reserved` is typically a function of `pod density` on the nodes. [This -performance dashboard](http://node-perf-dash.k8s.io/#/builds) exposes `cpu` and -`memory` usage profiles of `kubelet` and `docker engine` at multiple levels of -pod density. [This blog -post](https://kubernetes.io/blog/2016/11/visualize-kubelet-performance-with-node-dashboard) -explains how the dashboard can be interpreted to come up with a suitable -`kube-reserved` reservation. +`kube-reserved` is typically a function of `pod density` on the nodes. In addition to `cpu`, `memory`, and `ephemeral-storage`, `pid` may be specified to reserve the specified number of process IDs for From 41cec8d757471a3cbb5d8708c7ca1f1e22464e9e Mon Sep 17 00:00:00 2001 From: Gasmi Christophe Date: Thu, 12 Mar 2020 17:30:40 +0100 Subject: [PATCH 037/140] Update home in French (#19601) --- content/fr/_index.html | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/content/fr/_index.html b/content/fr/_index.html index 026a0f910f..16bfc5c82f 100644 --- a/content/fr/_index.html +++ b/content/fr/_index.html @@ -3,12 +3,12 @@ title: "Solution professionnelle d’orchestration de conteneurs" abstract: "Déploiement, mise à l'échelle et gestion automatisée des conteneurs" cid: home --- +{{< announcement >}} {{< deprecationwarning >}} {{< blocks/section id="oceanNodes" >}} {{% blocks/feature image="flower" %}} - ### [Kubernetes (K8s)]({{< relref "/docs/concepts/overview/what-is-kubernetes" >}}) est un système open-source permettant d'automatiser le déploiement, la mise à l'échelle et la gestion des applications conteneurisées. Les conteneurs qui composent une application sont regroupés dans des unités logiques pour en faciliter la gestion et la découverte. Kubernetes s’appuie sur [15 années d’expérience dans la gestion de charges de travail de production (workloads) chez Google](http://queue.acm.org/detail.cfm?id=2898444), associé aux meilleures idées et pratiques de la communauté. @@ -18,6 +18,7 @@ Les conteneurs qui composent une application sont regroupés dans des unités lo #### Quel que soit le nombre Conçu selon les mêmes principes qui permettent à Google de gérer des milliards de conteneurs par semaine, Kubernetes peut évoluer sans augmenter votre équipe d'opérations. + {{% /blocks/feature %}} {{% blocks/feature image="blocks" %}} @@ -28,18 +29,15 @@ Qu'il s'agisse de tester localement ou d'une implémentation globale, Kubernetes {{% /blocks/feature %}} {{% blocks/feature image="suitcase" %}} - #### Quel que soit l'endroit -Kubernetes est une solution open-source qui vous permet de tirer parti de vos infrastructures qu'elles soient sur site (on-premises), hybride ou en Cloud publique. -Vous pourrez ainsi répartir sans effort vos workloads là où vous le souhaitez. +Kubernetes est une solution open-source qui vous permet de tirer parti de vos infrastructures qu'elles soient sur site (on-premises), hybride ou en Cloud publique. Vous pourrez ainsi répartir sans effort vos workloads là où vous le souhaitez. {{% /blocks/feature %}} {{< /blocks/section >}} {{< blocks/section id="video" background-image="kub_video_banner_homepage" >}} -

Les défis de la migration de plus de 150 microservices vers Kubernetes

Par Sarah Wells, directrice technique des opérations et de la fiabilité, Financial Times

@@ -47,12 +45,12 @@ Vous pourrez ainsi répartir sans effort vos workloads là où vous le souhaitez


- Venez au KubeCon San Diego du 18 au 21 Nov 2019 + Venez au KubeCon Amsterdam en Juillet/Aout (à définir)



- Venez au KubeCon Amsterdam du 30 Mars au 2 Avril 2020 + Venez au KubeCon Boston du 17 au 20 Novembre 2020
From 83119121cfb1c4ae49f9d405a276b70cbb33c6e5 Mon Sep 17 00:00:00 2001 From: Mendy Danzinger Date: Thu, 12 Mar 2020 17:56:38 -0400 Subject: [PATCH 038/140] fix broken link (#18812) --- .../configuration/manage-compute-resources-container.md | 2 +- .../configuration/manage-compute-resources-container.md | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/content/en/docs/concepts/configuration/manage-compute-resources-container.md b/content/en/docs/concepts/configuration/manage-compute-resources-container.md index 43e71b15f7..576a008ba9 100644 --- a/content/en/docs/concepts/configuration/manage-compute-resources-container.md +++ b/content/en/docs/concepts/configuration/manage-compute-resources-container.md @@ -191,7 +191,7 @@ resource limits, see the The resource usage of a Pod is reported as part of the Pod status. -If [optional monitoring](http://releases.k8s.io/{{< param "githubbranch" >}}/cluster/addons/cluster-monitoring/README.md) +If [optional monitoring](/docs/tasks/debug-application-cluster/resource-metrics-pipeline/) is configured for your cluster, then Pod resource usage can be retrieved from the monitoring system. diff --git a/content/zh/docs/concepts/configuration/manage-compute-resources-container.md b/content/zh/docs/concepts/configuration/manage-compute-resources-container.md index 627e27a2ad..c3e6b6af4d 100644 --- a/content/zh/docs/concepts/configuration/manage-compute-resources-container.md +++ b/content/zh/docs/concepts/configuration/manage-compute-resources-container.md @@ -275,7 +275,7 @@ resource limits, see the The resource usage of a Pod is reported as part of the Pod status. -If [optional monitoring](http://releases.k8s.io/{{< param "githubbranch" >}}/cluster/addons/cluster-monitoring/README.md) +If [optional monitoring](/docs/tasks/debug-application-cluster/resource-metrics-pipeline/) is configured for your cluster, then Pod resource usage can be retrieved from the monitoring system. --> @@ -284,7 +284,7 @@ the monitoring system. Pod 的资源使用情况被报告为 Pod 状态的一部分。 -如果为集群配置了 [可选监控](http://releases.k8s.io/{{< param "githubbranch" >}}/cluster/addons/cluster-monitoring/README.md)，则可以从监控系统检索 Pod 资源的使用情况。 +如果为集群配置了 [可选监控](/docs/tasks/debug-application-cluster/resource-metrics-pipeline/)，则可以从监控系统检索 Pod 资源的使用情况。 +Các worker node chứa các pod (là những thành phần của ứng dụng). Control Plane quản lý các worker node và pod trong cluster. +Trong môi trường sản phẩm (production environment), Control Plane thường chạy trên nhiều máy tính và một cluster thường chạy trên nhiều node, cung cấp khả năng chịu lỗi (fault-tolerance) và tính sẵn sàng cao (high availability). \ No newline at end of file diff --git a/content/vi/docs/reference/glossary/containerd.md b/content/vi/docs/reference/glossary/containerd.md new file mode 100644 index 0000000000..b14971032a --- /dev/null +++ b/content/vi/docs/reference/glossary/containerd.md @@ -0,0 +1,15 @@ +--- +title: containerd +id: containerd +date: 2020-03-04 +full_link: https://containerd.io/docs/ +short_description: > + Một container runtime tập trung vào sự đơn giản, mạnh mẽ và linh động. +aka: +tags: +- tool +--- + Một container runtime tập trung vào sự đơn giản, mạnh mẽ và linh động. + + +containerd là một {{< glossary_tooltip text="container" term_id="container" >}} runtime cái mà chạy như một daemon trên Linux hoặc Windows. containerd quan tâm tới việc lấy và lưu trữ các container image, thực thi các container, cung cấp truy cập mạng, và nhiều hơn nữa. \ No newline at end of file diff --git a/content/vi/docs/reference/glossary/control-plane.md b/content/vi/docs/reference/glossary/control-plane.md new file mode 100644 index 0000000000..b8f9b0e7f8 --- /dev/null +++ b/content/vi/docs/reference/glossary/control-plane.md @@ -0,0 +1,15 @@ +--- +title: Control Plane +id: control-plane +date: 2020-03-04 +full_link: +short_description: > + Tầng điều khiển container, được dùng để đưa ra API và các interface để định nghĩa, triển khai, và quản lý vòng đời của các container. + +aka: +tags: +- fundamental +--- + Tầng điều khiển container, được dùng để đưa ra API và các interface để định nghĩa, triển khai, và quản lý vòng đời của các container. + + diff --git a/content/vi/docs/reference/glossary/cri-o.md b/content/vi/docs/reference/glossary/cri-o.md new file mode 100644 index 0000000000..be8dde10ac --- /dev/null +++ b/content/vi/docs/reference/glossary/cri-o.md @@ -0,0 +1,19 @@ +--- +title: CRI-O +id: cri-o +date: 2020-03-05 +full_link: https://cri-o.io/#what-is-cri-o +short_description: > + Một container runtime nhẹ dành riêng cho Kubernetes +aka: +tags: +- tool +--- +Một công cụ giúp bạn sử dụng các OCI container runtime với Kubernetes CRI. + + + +CRI-O là một thực thi của {{< glossary_tooltip term_id="cri" >}} để cho phép sử dụng các {{< glossary_tooltip text="container" term_id="container" >}} runtime cái mà tương thích với Open Container Initiative (OCI) +[runtime spec](http://www.github.com/opencontainers/runtime-spec). + +Triển khai CRI-O cho phép Kuberentes sử dụng bất kì OCI-compliant runtime như container runtime để chạy {{< glossary_tooltip text="Pods" term_id="pod" >}}, và để lấy CRI container image từ các remote registry. \ No newline at end of file diff --git a/content/vi/docs/reference/glossary/daemonset.md b/content/vi/docs/reference/glossary/daemonset.md new file mode 100644 index 0000000000..1039514a62 --- /dev/null +++ b/content/vi/docs/reference/glossary/daemonset.md @@ -0,0 +1,19 @@ +--- +title: DaemonSet +id: daemonset +date: 2020-03-05 +full_link: /docs/concepts/workloads/controllers/daemonset +short_description: > + Đảm bảo một bản sao của Pod đang chạy trên một tập các node của cluster. +aka: +tags: +- fundamental +- core-object +- workload +--- + Đảm bảo một bản sao của {{< glossary_tooltip text="Pod" term_id="pod" >}} đang chạy trên một tập các node của {{< glossary_tooltip text="cluster" term_id="cluster" >}}. + + + +Được sử dụng để deploy những system daemon ví dụ như log collector, monitoring agent, những cái thường phải chạy trên mọi {{< glossary_tooltip term_id="node" >}}. + diff --git a/content/vi/docs/reference/glossary/etcd.md b/content/vi/docs/reference/glossary/etcd.md new file mode 100644 index 0000000000..d52a91fdbd --- /dev/null +++ b/content/vi/docs/reference/glossary/etcd.md @@ -0,0 +1,20 @@ +--- +title: etcd +id: etcd +date: 2020-27-02 +full_link: /docs/tasks/administer-cluster/configure-upgrade-etcd/ +short_description: > + Key value store nhất quán (consistent) và sẵn sàng cao (highly-available) được sử dụng như một kho lưu trữ của Kubernetes cho tất cả dữ liệu của cluster. + +aka: +tags: +- architecture +- storage +--- + Key value store nhất quán (consistent) và sẵn sàng cao (highly-available) được sử dụng như một kho lưu trữ của Kubernetes cho tất cả dữ liệu của cluster. + + + +Nếu Kubernetes cluster của bạn sử dụng etcd như kho lưu trữ của nó, chắc chắn bạn có một kế hoạch [back up](/docs/tasks/administer-cluster/configure-upgrade-etcd/#backing-up-an-etcd-cluster) cho những dữ liệu này. + +Bạn có thể tìm thêm thông tin chi tiết về etcd tại [documentation](https://etcd.io/docs/). \ No newline at end of file diff --git a/content/vi/docs/reference/glossary/kube-apiserver.md b/content/vi/docs/reference/glossary/kube-apiserver.md new file mode 100644 index 0000000000..94b0aa2183 --- /dev/null +++ b/content/vi/docs/reference/glossary/kube-apiserver.md @@ -0,0 +1,22 @@ +--- +title: API server +id: kube-apiserver +date: 2020-02-26 +full_link: /docs/reference/generated/kube-apiserver/ +short_description: > + Thành phần tầng điểu khiển (control plane), được dùng để phục vụ Kubernetes API. + +aka: +- kube-apiserver +tags: +- architecture +- fundamental +--- + API server là một thành phần của Kubernetes {{< glossary_tooltip text="control plane" term_id="control-plane" >}}, được dùng để đưa ra Kubernetes API. +API server là front end của Kubernetes control plane. + + + +Thực thi chính của API server là [kube-apiserver](/docs/reference/generated/kube-apiserver/). +kube-apiserver được thiết kế để co giãn theo chiều ngang — có nghĩa là nó co giãn bằng cách triển khai thêm các thực thể. +Bạn có thể chạy một vài thực thể của kube-apiserver và cân bằng lưu lượng giữa các thực thể này. \ No newline at end of file diff --git a/content/vi/docs/reference/glossary/kube-scheduler.md b/content/vi/docs/reference/glossary/kube-scheduler.md new file mode 100644 index 0000000000..dc7c63a31f --- /dev/null +++ b/content/vi/docs/reference/glossary/kube-scheduler.md @@ -0,0 +1,17 @@ +--- +title: kube-scheduler +id: kube-scheduler +date: 2020-03-05 +full_link: /docs/reference/generated/kube-scheduler/ +short_description: > + Thành phần của Control Plane, được dùng để giám sát việc tạo những pod mới mà chưa được chỉ định vào node nào, và chọn một node để chúng chạy trên đó. + +aka: +tags: +- architecture +--- + Thành phần của Control Plane, được dùng để giám sát việc tạo những pod mới mà chưa được chỉ định vào node nào, và chọn một node để chúng chạy trên đó. + + + +Những yếu tố trong những quyết định lập lịch bao gồm những yêu cầu về tài nguyên, những đòi hỏi về phần cứng/phần mềm/chính sách, những thông số về affinity và anti-affinity, dữ liệu tại chỗ (data locality), nhiễu inter-workload và thời hạn (deadline). From 6cf5ce0f3534f95286b8f9917e2ab6844709e0af Mon Sep 17 00:00:00 2001 From: Weidong Cai Date: Fri, 13 Mar 2020 19:54:39 +0800 Subject: [PATCH 042/140] fix bad format (#19489) --- .../configure-pod-configmap.md | 23 ------------------- 1 file changed, 23 deletions(-) diff --git a/content/zh/docs/tasks/configure-pod-container/configure-pod-configmap.md b/content/zh/docs/tasks/configure-pod-container/configure-pod-configmap.md index fcaaf4f940..a5cdbea9d8 100644 --- a/content/zh/docs/tasks/configure-pod-container/configure-pod-configmap.md +++ b/content/zh/docs/tasks/configure-pod-container/configure-pod-configmap.md @@ -73,17 +73,6 @@ For example: --> 你可以使用 `kubectl create configmap` 从同一目录中的多个文件创建 ConfigMap。 例如： - - - ```shell # 创建本地目录 mkdir -p configure-pod-container/configmap/ @@ -534,18 +523,6 @@ configmap/game-config-5-m67dt67794 created 要从文字 `special.type=charm` 和 `special.how=very` 生成 ConfigMap，可以在 `kusotmization.yaml` 中将 ConfigMap 生成器指定。 - - - ```shell # 使用 ConfigMapGenerator 创建 kustomization.yaml 文件 cat <./kustomization.yaml From 3e117cd6b7cf3301ed29ea08cfaa4dcdad4b6bdc Mon Sep 17 00:00:00 2001 From: Gasmi Christophe Date: Fri, 13 Mar 2020 17:06:41 +0100 Subject: [PATCH 043/140] Update from EN docs for install-kubectl task (#19616) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Update from EN docs for install-kubectl task * add missing option "--client" * typo alternatif=> alternatifs * Update content/fr/docs/tasks/tools/install-kubectl.md Co-Authored-By: Tim Bannister * Update content/fr/docs/tasks/tools/install-kubectl.md Co-Authored-By: Rémy Léone * Update content/fr/docs/tasks/tools/install-kubectl.md Co-Authored-By: Rémy Léone Co-authored-by: Tim Bannister Co-authored-by: Rémy Léone --- .../fr/docs/tasks/tools/install-kubectl.md | 115 ++++++++++++------ 1 file changed, 81 insertions(+), 34 deletions(-) diff --git a/content/fr/docs/tasks/tools/install-kubectl.md b/content/fr/docs/tasks/tools/install-kubectl.md index 091cb63c30..2d1f50fccd 100644 --- a/content/fr/docs/tasks/tools/install-kubectl.md +++ b/content/fr/docs/tasks/tools/install-kubectl.md @@ -54,7 +54,7 @@ Vous devez utiliser une version de kubectl qui différe seulement d'une version 4. Testez pour vous assurer que la version que vous avez installée est à jour: ``` - kubectl version + kubectl version --client ``` ### Installation à l'aide des gestionnaires des paquets natifs @@ -80,28 +80,33 @@ yum install -y kubectl {{< /tab >}} {{< /tabs >}} +### Installation avec des gestionnaires de paquets alternatifs -### Installer avec snap - +{{< tabs name="other_kubectl_install" >}} +{{% tab name="Snap" %}} Si vous êtes sur Ubuntu ou une autre distribution Linux qui supporte le gestionnaire de paquets [snap](https://snapcraft.io/docs/core/install), kubectl est disponible comme application [snap](https://snapcraft.io/). -1. Passez à l'utilisateur snap et exécutez la commande d'installation : +```shell +snap install kubectl --classic - ``` - sudo snap install kubectl --classic - ``` +kubectl version --client +``` +{{% /tab %}} +{{% tab name="Homebrew" %}} +Si vous êtes sur Linux et que vous utiliser [Homebrew](https://docs.brew.sh/Homebrew-on-Linux) comme gestionnaire de paquets, kubectl est disponible. [installation](https://docs.brew.sh/Homebrew-on-Linux#install) +```shell +brew install kubectl -2. Testez pour vous assurer que la version que vous avez installée est à jour : - - ``` - kubectl version - ``` +kubectl version --client +``` +{{% /tab %}} +{{< /tabs >}} ## Installer kubectl sur macOS ### Installer le binaire kubectl avec curl sur macOS -1. Téléchargez la dernière release: +1. Téléchargez la dernière version: ``` curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/darwin/amd64/kubectl @@ -129,7 +134,7 @@ Si vous êtes sur Ubuntu ou une autre distribution Linux qui supporte le gestion 4. Testez pour vous assurer que la version que vous avez installée est à jour: ``` - kubectl version + kubectl version --client ``` ### Installer avec Homebrew sur macOS @@ -138,6 +143,11 @@ Si vous êtes sur MacOS et que vous utilisez le gestionnaire de paquets [Homebre 1. Exécutez la commande d'installation: + ``` + brew install kubectl + ``` + ou + ``` brew install kubernetes-cli ``` @@ -145,7 +155,7 @@ Si vous êtes sur MacOS et que vous utilisez le gestionnaire de paquets [Homebre 2. Testez pour vous assurer que la version que vous avez installée est à jour: ``` - kubectl version + kubectl version --client ``` ### Installer avec Macports sur macOS @@ -162,14 +172,14 @@ Si vous êtes sur MacOS et que vous utilisez le gestionnaire de paquets [Macport 2. Testez pour vous assurer que la version que vous avez installée est à jour: ``` - kubectl version + kubectl version --client ``` ## Installer kubectl sur Windows ### Installer le binaire kubectl avec curl sur Windows -1. Téléchargez la dernière release {{< param "fullversion" >}} depuis [ce lien](https://storage.googleapis.com/kubernetes-release/release/{{< param "fullversion" >}}/bin/windows/amd64/kubectl.exe). +1. Téléchargez la dernière version {{< param "fullversion" >}} depuis [ce lien](https://storage.googleapis.com/kubernetes-release/release/{{< param "fullversion" >}}/bin/windows/amd64/kubectl.exe). Ou si vous avez `curl` installé, utilisez cette commande: @@ -183,8 +193,12 @@ Si vous êtes sur MacOS et que vous utilisez le gestionnaire de paquets [Macport 3. Testez pour vous assurer que la version que vous avez installée est à jour: ``` - kubectl version + kubectl version --client ``` +{{< note >}} +[Docker Desktop pour Windows](https://docs.docker.com/docker-for-windows/#kubernetes) ajoute sa propre version de `kubectl` au $PATH. +Si vous avez déjà installé Docker Desktop, vous devrez peut-être placer votre entrée PATH avant celle ajoutée par le programme d'installation de Docker Desktop ou supprimer le `kubectl` de Docker Desktop. +{{< /note >}} ### Installer avec Powershell de PSGallery @@ -204,7 +218,7 @@ Si vous êtes sous Windows et que vous utilisez le gestionnaire de paquets [Powe 2. Testez pour vous assurer que la version que vous avez installée est à jour: ``` - kubectl version + kubectl version --client ``` {{< note >}}La mise à jour de l'installation s'effectue en réexécutant les deux commandes listées à l'étape 1.{{< /note >}} @@ -227,7 +241,7 @@ Pour installer kubectl sur Windows, vous pouvez utiliser le gestionnaire de paqu 2. Testez pour vous assurer que la version que vous avez installée est à jour: ``` - kubectl version + kubectl version --client ``` 3. Accédez à votre répertoire personnel: @@ -269,7 +283,7 @@ Vous pouvez installer kubectl en tant qu'élément du SDK Google Cloud. 3. Testez pour vous assurer que la version que vous avez installée est à jour: ``` - kubectl version + kubectl version --client ``` ## Vérification de la configuration de kubectl @@ -307,7 +321,7 @@ Vous trouverez ci-dessous les étapes à suivre pour configurer l'auto-compléti {{< tabs name="kubectl_autocompletion" >}} -{{% tab name="Bash on Linux" %}} +{{% tab name="Bash sur Linux" %}} ### Introduction @@ -344,6 +358,12 @@ Vous devez maintenant vérifier que le script de completion de kubectl est bien ```shell kubectl completion bash >/etc/bash_completion.d/kubectl ``` +- Si vous avez un alias pour kubectl, vous pouvez étendre la completion de votre shell pour fonctionner avec cet alias: + + ```shell + echo 'alias k=kubectl' >>~/.bashrc + echo 'complete -F __start_kubectl k' >>~/.bashrc + ``` {{< note >}} bash-completion source tous les scripts de completion dans `/etc/bash_completion.d`. @@ -354,49 +374,69 @@ Les deux approches sont équivalentes. Après avoir rechargé votre shell, l'aut {{% /tab %}} -{{% tab name="Bash on macOS" %}} +{{% tab name="Bash sur macOS" %}} -{{< warning>}} -macOS inclut Bash 3.2 par défaut. Le script de complétion kubectl nécessite Bash 4.1+ et ne fonctionne pas avec Bash 3.2. Une des solutions possibles est d'installer une version plus récente de Bash sous macOS (voir instructions [ici](https://itnext.io/upgrading-bash-on-macos-7138bd1066ba)). Les instructions ci-dessous ne fonctionnent que si vous utilisez Bash 4.1+. -{{< /warning >}} ### Introduction Le script de complétion kubectl pour Bash peut être généré avec la commande `kubectl completion bash`. Sourcer le script de completion dans votre shell permet l'auto-complétion de kubectl. -En revanche, le script de complétion dépend de [**bash-completion**](https://github.com/scop/bash-completion), ce qui implique que vous devez d'abord installer ce logiciel (vous pouvez tester si vous avez déjà installé bash-completion en utilisant `type _init_completion`). +En revanche, le script de complétion dépend de [**bash-completion**](https://github.com/scop/bash-completion), ce qui implique que vous devez d'abord installer ce logiciel. + +{{< warning>}} +macOS inclut Bash 3.2 par défaut. Le script de complétion kubectl nécessite Bash 4.1+ et ne fonctionne pas avec Bash 3.2. Une des solutions possibles est d'installer une version plus récente de Bash sous macOS (voir instructions [ici](https://itnext.io/upgrading-bash-on-macos-7138bd1066ba)). Les instructions ci-dessous ne fonctionnent que si vous utilisez Bash 4.1+. +{{< /warning >}} + ### Installer bash-completion -Vous pouvez installer bash-completion avec Homebrew: +{{< note >}} +Comme mentionné, ces instructions supposent que vous utilisez Bash 4.1+, ce qui signifie que vous installerez bash-completion v2 (contrairement à Bash 3.2 et bash-completion v1, auquel cas la complétion pour kubectl ne fonctionnera pas). +{{< /note >}} + +Vous pouvez tester si vous avez déjà installé bash-completion en utilisant `type _init_completion`. Si il n'est pas installé, vous pouvez installer bash-completion avec Homebrew: ```shell -brew install bash-completion +brew install bash-completion@2 ``` Comme indiqué dans la sortie de `brew install` (section "Caveats"), ajoutez les lignes suivantes à votre fichier `~/.bashrc` ou `~/.bash_profile` : ```shell -[ -f /usr/local/etc/bash_completion ] && . /usr/local/etc/bash_completion +export BASH_COMPLETION_COMPAT_DIR="/usr/local/etc/bash_completion.d" +[[ -r "/usr/local/etc/profile.d/bash_completion.sh" ]] && . "/usr/local/etc/profile.d/bash_completion.sh" ``` -Rechargez votre shell. +Rechargez votre shell et vérifiez que bash-completion v2 est correctement installé avec `type _init_completion`. ### Activer l'auto-complétion de kubectl -Si vous avez installé kubectl avec Homebrew (comme expliqué [ici](#installer-avec-homebrew-sur-macos)), alors le script de complétion a été automatiquement installé dans `/usr/local/etc/bash_completion.d/kubectl`. Dans ce cas, vous n'avez rien à faire. - Si vous n'avez pas installé via Homebrew, vous devez maintenant vous assurer que le script de complétion kubectl est bien sourcé dans toutes vos sessions shell comme suit: +- Sourcer le script de completion dans votre fichier `~/.bashrc`: + + ```shell + echo 'source <(kubectl completion bash)' >>~/.bashrc + + ``` + - Ajoutez le script de complétion dans le répertoire `/usr/local/etc/bash_completion.d`: ```shell kubectl completion bash >/usr/local/etc/bash_completion.d/kubectl ``` +- Si vous avez un alias pour kubectl, vous pouvez étendre la completion de votre shell pour fonctionner avec cet alias: + + ```shell + echo 'alias k=kubectl' >>~/.bashrc + echo 'complete -F __start_kubectl k' >>~/.bashrc + ``` + +Si vous avez installé kubectl avec Homebrew (comme expliqué [ici](#installer-avec-homebrew-sur-macos)), alors le script de complétion a été automatiquement installé dans `/usr/local/etc/bash_completion.d/kubectl`. Dans ce cas, vous n'avez rien à faire. {{< note >}} -bash-completion (en cas d'installation avec Homebrew) source tous les scripts de complétion dans le répertoire. +L'installation Homebrew de bash-complétion v2 source tous les fichiers du répertoire `BASH_COMPLETION_COMPAT_DIR`, c'est pourquoi les deux dernières méthodes fonctionnent. {{< /note >}} Après avoir rechargé votre shell, l'auto-complétion de kubectl devrait fonctionner. @@ -412,6 +452,13 @@ Pour faire ainsi dans toutes vos sessions shell, ajoutez ce qui suit à votre fi source <(kubectl completion zsh) ``` +Si vous avez un alias pour kubectl, vous pouvez étendre la completion de votre shell pour fonctionner avec cet alias: + +```shell +echo 'alias k=kubectl' >>~/.zshrc +echo 'complete -F __start_kubectl k' >>~/.zshrc +``` + Après avoir rechargé votre shell, l'auto-complétion de kubectl devrait fonctionner. Si vous rencontrez une erreur comme `complete:13: command not found: compdef`, alors ajoutez ce qui suit au début de votre fichier `~/.zshrc`: From 8ccc83fad62c10632a66ff9a49383ee22b1905d8 Mon Sep 17 00:00:00 2001 From: Andrey Zhelnin Date: Sat, 14 Mar 2020 01:50:34 +0100 Subject: [PATCH 044/140] Update kubectl explain usage case (#19612) You can't specify more then one resources at one for explain. See error below: $ kubectl explain pods,svc the server doesn't have a resource type "pods,svc" error: We accept only this format: explain RESOURCE --- content/en/docs/reference/kubectl/cheatsheet.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/reference/kubectl/cheatsheet.md b/content/en/docs/reference/kubectl/cheatsheet.md index adb7fb8b6f..83a553e5e1 100644 --- a/content/en/docs/reference/kubectl/cheatsheet.md +++ b/content/en/docs/reference/kubectl/cheatsheet.md @@ -95,7 +95,7 @@ kubectl apply -f ./my1.yaml -f ./my2.yaml # create from multiple files kubectl apply -f ./dir # create resource(s) in all manifest files in dir kubectl apply -f https://git.io/vPieo # create resource(s) from url kubectl create deployment nginx --image=nginx # start a single instance of nginx -kubectl explain pods,svc # get the documentation for pod and svc manifests +kubectl explain pods # get the documentation for pod manifests # Create multiple YAML objects from stdin cat < Date: Sun, 15 Mar 2020 13:46:35 +0530 Subject: [PATCH 045/140] Improve kops macOS installation (#19425) --- content/en/docs/setup/production-environment/tools/kops.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/content/en/docs/setup/production-environment/tools/kops.md b/content/en/docs/setup/production-environment/tools/kops.md index 03d45f7827..10ae6dfa65 100644 --- a/content/en/docs/setup/production-environment/tools/kops.md +++ b/content/en/docs/setup/production-environment/tools/kops.md @@ -49,14 +49,12 @@ Download the latest release with the command: curl -LO https://github.com/kubernetes/kops/releases/download/$(curl -s https://api.github.com/repos/kubernetes/kops/releases/latest | grep tag_name | cut -d '"' -f 4)/kops-darwin-amd64 ``` -To download a specific version, replace the +To download a specific version, replace the following portion of the command with the specific kops version. ```shell $(curl -s https://api.github.com/repos/kubernetes/kops/releases/latest | grep tag_name | cut -d '"' -f 4) ``` -portion of the command with the specific version. - For example, to download kops version v1.15.0 type: ```shell From 740e53d1e1d15cbbd574152b074d008c673b09ad Mon Sep 17 00:00:00 2001 From: Vinicius Barbosa Date: Sun, 15 Mar 2020 05:32:34 -0300 Subject: [PATCH 046/140] Update expose-intro.html (#19637) --- .../tutorials/kubernetes-basics/expose/expose-intro.html | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/content/en/docs/tutorials/kubernetes-basics/expose/expose-intro.html b/content/en/docs/tutorials/kubernetes-basics/expose/expose-intro.html index 8adf05965b..8a7d60dd87 100644 --- a/content/en/docs/tutorials/kubernetes-basics/expose/expose-intro.html +++ b/content/en/docs/tutorials/kubernetes-basics/expose/expose-intro.html @@ -79,13 +79,8 @@ weight: 10
  • Embed version tags
  • Classify an object using tags
    • - -
    -
    -
    -

    You can create a Service at the same time you create a Deployment by using
    --expose in kubectl.

    -
    +
    From cd0cf819598af5db25642f670f2a1551c52a4aa0 Mon Sep 17 00:00:00 2001 From: abhinav-goyall <47552229+abhinav-goyall@users.noreply.github.com> Date: Sun, 15 Mar 2020 14:36:35 +0530 Subject: [PATCH 047/140] Typo:Comma Instead of Semi-Colon (#19623) --- content/en/docs/setup/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/setup/_index.md b/content/en/docs/setup/_index.md index 6c903d4e60..3cf2afa968 100644 --- a/content/en/docs/setup/_index.md +++ b/content/en/docs/setup/_index.md @@ -24,7 +24,7 @@ This section covers different options to set up and run Kubernetes. Different Kubernetes solutions meet different requirements: ease of maintenance, security, control, available resources, and expertise required to operate and manage a cluster. -You can deploy a Kubernetes cluster on a local machine, cloud, on-prem datacenter; or choose a managed Kubernetes cluster. You can also create custom solutions across a wide range of cloud providers, or bare metal environments. +You can deploy a Kubernetes cluster on a local machine, cloud, on-prem datacenter, or choose a managed Kubernetes cluster. You can also create custom solutions across a wide range of cloud providers, or bare metal environments. More simply, you can create a Kubernetes cluster in learning and production environments. From 24f8ec395bbc8575b15061c42dfa93e3548fbe20 Mon Sep 17 00:00:00 2001 From: Rajesh Deshpande Date: Sun, 15 Mar 2020 14:54:35 +0530 Subject: [PATCH 048/140] Correcting heading syntax (#19609) Correcting heading syntax --- .../connecting-frontend-backend.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/content/en/docs/tasks/access-application-cluster/connecting-frontend-backend.md b/content/en/docs/tasks/access-application-cluster/connecting-frontend-backend.md index 3cb90383c7..123271be44 100644 --- a/content/en/docs/tasks/access-application-cluster/connecting-frontend-backend.md +++ b/content/en/docs/tasks/access-application-cluster/connecting-frontend-backend.md @@ -39,7 +39,7 @@ frontend and backend are connected using a Kubernetes {{% capture lessoncontent %}} -### Creating the backend using a Deployment +## Creating the backend using a Deployment The backend is a simple hello greeter microservice. Here is the configuration file for the backend Deployment: @@ -95,7 +95,7 @@ Events: ... ``` -### Creating the backend Service object +## Creating the backend Service object The key to connecting a frontend to a backend is the backend Service. A Service creates a persistent IP address and DNS name entry @@ -119,7 +119,7 @@ kubectl apply -f https://k8s.io/examples/service/access/hello-service.yaml At this point, you have a backend Deployment running, and you have a Service that can route traffic to it. -### Creating the frontend +## Creating the frontend Now that you have your backend, you can create a frontend that connects to the backend. The frontend connects to the backend worker Pods by using the DNS name @@ -158,7 +158,7 @@ be to use a so that you can change the configuration more easily. {{< /note >}} -### Interact with the frontend Service +## Interact with the frontend Service Once you’ve created a Service of type LoadBalancer, you can use this command to find the external IP: @@ -186,7 +186,7 @@ frontend LoadBalancer 10.51.252.116 XXX.XXX.XXX.XXX 80/TCP 1m That IP can now be used to interact with the `frontend` service from outside the cluster. -### Send traffic through the frontend +## Send traffic through the frontend The frontend and backends are now connected. You can hit the endpoint by using the curl command on the external IP of your frontend Service. From d4167faa2ce59612b36e4edcc7357c1a63131bbc Mon Sep 17 00:00:00 2001 From: Vinicius Barbosa Date: Sun, 15 Mar 2020 06:30:35 -0300 Subject: [PATCH 049/140] Adding Docker website reference (#19598) --- content/en/docs/setup/learning-environment/minikube.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/setup/learning-environment/minikube.md b/content/en/docs/setup/learning-environment/minikube.md index 7233f98c0b..439e4a10a2 100644 --- a/content/en/docs/setup/learning-environment/minikube.md +++ b/content/en/docs/setup/learning-environment/minikube.md @@ -24,7 +24,7 @@ Minikube supports the following Kubernetes features: * NodePorts * ConfigMaps and Secrets * Dashboards -* Container Runtime: Docker, [CRI-O](https://cri-o.io/), and [containerd](https://github.com/containerd/containerd) +* Container Runtime: [Docker](https://www.docker.com/), [CRI-O](https://cri-o.io/), and [containerd](https://github.com/containerd/containerd) * Enabling CNI (Container Network Interface) * Ingress From 6aa392535ccc4216a21ae9f941175d24d2b299f6 Mon Sep 17 00:00:00 2001 From: Jhon Mike Date: Sun, 15 Mar 2020 18:44:35 -0300 Subject: [PATCH 050/140] Translate content/pt/docs/concepts/cluster-administration/logging.md into Portugues (#19573) * translating doc kubelet logging * adjustments to the logging doc in pt recommended in the CR --- .../cluster-administration/logging.md | 206 ++++++++++++++++++ .../admin/logging/fluentd-sidecar-config.yaml | 25 +++ .../two-files-counter-pod-agent-sidecar.yaml | 39 ++++ ...o-files-counter-pod-streaming-sidecar.yaml | 38 ++++ .../admin/logging/two-files-counter-pod.yaml | 26 +++ content/pt/examples/debug/counter-pod.yaml | 10 + 6 files changed, 344 insertions(+) create mode 100644 content/pt/docs/concepts/cluster-administration/logging.md create mode 100644 content/pt/examples/admin/logging/fluentd-sidecar-config.yaml create mode 100644 content/pt/examples/admin/logging/two-files-counter-pod-agent-sidecar.yaml create mode 100644 content/pt/examples/admin/logging/two-files-counter-pod-streaming-sidecar.yaml create mode 100644 content/pt/examples/admin/logging/two-files-counter-pod.yaml create mode 100644 content/pt/examples/debug/counter-pod.yaml diff --git a/content/pt/docs/concepts/cluster-administration/logging.md b/content/pt/docs/concepts/cluster-administration/logging.md new file mode 100644 index 0000000000..f605a3e875 --- /dev/null +++ b/content/pt/docs/concepts/cluster-administration/logging.md @@ -0,0 +1,206 @@ +--- +reviewers: + - piosz + - x13n +title: Arquitetura de Log +content_template: templates/concept +weight: 60 +--- + +{{% capture overview %}} + +Os logs de aplicativos e sistemas podem ajudá-lo a entender o que está acontecendo dentro do seu cluster. Os logs são particularmente úteis para depurar problemas e monitorar a atividade do cluster. A maioria das aplicações modernas possui algum tipo de mecanismo de logs; como tal, a maioria dos mecanismos de contêineres também é projetada para suportar algum tipo de log. O método de log mais fácil e abrangente para aplicações em contêiner é gravar nos fluxos de saída e erro padrão. + +No entanto, a funcionalidade nativa fornecida por um mecanismo de contêiner ou tempo de execução geralmente não é suficiente para uma solução completa de log. Por exemplo, se um contêiner travar, um pod for despejado ou um nó morrer, geralmente você ainda desejará acessar os logs do aplicativo. Dessa forma, os logs devem ter armazenamento e ciclo de vida separados, independentemente de nós, pods ou contêineres. Este conceito é chamado _cluster-level-logging_. O log no nível de cluster requer um back-end separado para armazenar, analisar e consultar logs. O kubernetes não fornece uma solução de armazenamento nativa para dados de log, mas você pode integrar muitas soluções de log existentes no cluster do Kubernetes. + +{{% /capture %}} + +{{% capture body %}} + +As arquiteturas de log no nível de cluster são descritas no pressuposto de que um back-end de log esteja presente dentro ou fora do cluster. Se você não estiver interessado em ter o log no nível do cluster, ainda poderá encontrar a descrição de como os logs são armazenados e manipulados no nó para serem úteis. + +## Log básico no Kubernentes + +Nesta seção, você pode ver um exemplo de log básico no Kubernetes que gera dados para o fluxo de saída padrão(standard output stream). Esta demostração usa uma [especificação de pod](/examples/debug/counter-pod.yaml) com um contêiner que grava algum texto na saída padrão uma vez por segundo. + +{{< codenew file="debug/counter-pod.yaml" >}} + +Para executar este pod, use o seguinte comando: + +```shell +kubectl apply -f https://k8s.io/examples/debug/counter-pod.yaml +``` + +A saída será: + +``` +pod/counter created +``` + +Para buscar os logs, use o comando `kubectl logs`, da seguinte maneira: + +```shell +kubectl logs counter +``` + +A saída será: + +``` +0: Mon Jan 1 00:00:00 UTC 2001 +1: Mon Jan 1 00:00:01 UTC 2001 +2: Mon Jan 1 00:00:02 UTC 2001 +... +``` + +Você pode usar `kubectl logs` para recuperar logs de uma instanciação anterior de um contêiner com o sinalizador `--previous`, caso o contêiner tenha falhado. Se o seu pod tiver vários contêineres, você deverá especificar quais logs do contêiner você deseja acessar anexando um nome de contêiner ao comando. Veja a [documentação do `kubectl logs`](/docs/reference/generated/kubectl/kubectl-commands#logs) para mais destalhes. + +## Logs no nível do Nó + +![Log no nível do nó](/images/docs/user-guide/logging/logging-node-level.png) + +Tudo o que um aplicativo em contêiner grava no `stdout` e `stderr` é tratado e redirecionado para algum lugar por dentro do mecanismo de contêiner. Por exemplo, o mecanismo de contêiner do Docker redireciona esses dois fluxos para [um driver de log](https://docs.docker.com/engine/admin/logging/overview), configurado no Kubernetes para gravar em um arquivo no formato json. + +{{< note >}} +O driver de log json do Docker trata cada linha como uma mensagem separada. Ao usar o driver de log do Docker, não há suporte direto para mensagens de várias linhas. Você precisa lidar com mensagens de várias linhas no nível do agente de log ou superior. +{{< /note >}} + +Por padrão, se um contêiner reiniciar, o kubelet manterá um contêiner terminado com seus logs. Se um pod for despejado do nó, todos os contêineres correspondentes também serão despejados, juntamente com seus logs. + +Uma consideração importante no log no nível do nó está implementado a rotação de log, para que os logs não consumam todo o armazenamento disponível no nó. Atualmente, o Kubernentes não é responsável pela rotação de logs, mas uma ferramenta de deployment deve configurar uma solução para resolver isso. +Por exemplo, nos clusters do Kubernetes, implementados pelo script `kube-up.sh`, existe uma ferramenta [`logrotate`](https://linux.die.net/man/8/logrotate) configurada para executar a cada hora. Você pode configurar um tempo de execução do contêiner para girar os logs do aplicativo automaticamente, por exemplo, usando o `log-opt` do Docker. +No script `kube-up.sh`, a última abordagem é usada para imagem COS no GCP, e a anterior é usada em qualquer outro ambiente. Nos dois casos por padrão, a rotação é configurada para ocorrer quando o arquivo de log exceder 10MB. + +Como exemplo, você pode encontrar informações detalhadas sobre como o `kube-up.sh` define o log da imagem COS no GCP no [script][cosconfigurehelper] correspondente. + +Quando você executa [`kubectl logs`](/docs/reference/generated/kubectl/kubectl-commands#logs) como no exemplo de log básico acima, o kubelet no nó lida com a solicitação e lê diretamente do arquivo de log, retornando o conteúdo na resposta. + +{{< note >}} +Atualmente, se algum sistema externo executou a rotação, apenas o conteúdo do arquivo de log mais recente estará disponível através de `kubectl logs`. Por exemplo, se houver um arquivo de 10MB, o `logrotate` executa a rotação e existem dois arquivos, um com 10MB de tamanho e um vazio, o `kubectl logs` retornará uma resposta vazia. +{{< /note >}} + +[cosConfigureHelper]: https://github.com/kubernetes/kubernetes/blob/{{< param "githubbranch" >}}/cluster/gce/gci/configure-helper.sh + +### Logs de componentes do sistema + +Existem dois tipos de componentes do sistema: aqueles que são executados em um contêiner e aqueles que não são executados em um contêiner. Por exemplo: + +- O scheduler Kubernetes e o kube-proxy são executados em um contêiner. +- O tempo de execução do kubelet e do contêiner, por exemplo, Docker, não é executado em contêineres. + +Nas máquinas com systemd, o tempo de execução do kubelet e do container é gravado no journald. Se systemd não estiver presente, eles gravam em arquivos `.log` no diretório `/var/log`. +Os componentes do sistema dentro dos contêineres sempre gravam no diretório `/var/log`, ignorando o mecanismo de log padrão. Eles usam a biblioteca de logs [klog][klog]. Você pode encontrar as convenções para a gravidade do log desses componentes nos [documentos de desenvolvimento sobre log](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-instrumentation/logging.md). + +Da mesma forma que os logs de contêiner, os logs de componentes do sistema no diretório `/var/log` devem ser rotacionados. Nos clusters do Kubernetes criados pelo script `kube-up.sh`, esses logs são configurados para serem rotacionados pela ferramenta `logrotate` diariamente ou quando o tamanho exceder 100MB. + +[klog]: https://github.com/kubernetes/klog + +## Arquiteturas de log no nível de cluster + +Embora o Kubernetes não forneça uma solução nativa para o log em nível de cluster, há várias abordagens comuns que você pode considerar. Aqui estão algumas opções: + +- Use um agente de log no nível do nó que seja executado em todos os nós. +- Inclua um contêiner sidecar dedicado para efetuar logging em um pod de aplicativo. +- Envie logs diretamente para um back-end de dentro de um aplicativo. + +### Usando um agente de log de nó + +![Usando um agente de log no nível do nó](/images/docs/user-guide/logging/logging-with-node-agent.png) + +Você pode implementar o log em nível de cluster incluindo um _agente de log em nível de nó_ em cada nó. O agente de log é uma ferramenta dedicada que expõe logs ou envia logs para um back-end. Geralmente, o agente de log é um contêiner que tem acesso a um diretório com arquivos de log de todos os contêineres de aplicativos nesse nó. + +Como o agente de log deve ser executado em todos os nós, é comum implementá-lo como uma réplica do DaemonSet, um pod de manifesto ou um processo nativo dedicado no nó. No entanto, as duas últimas abordagens são obsoletas e altamente desencorajadas. + +O uso de um agente de log no nível do nó é a abordagem mais comum e incentivada para um cluster Kubernetes, porque ele cria apenas um agente por nó e não requer alterações nos aplicativos em execução no nó. No entanto, o log no nível do nó _funciona apenas para a saída padrão dos aplicativos e o erro padrão_. + +O Kubernetes não especifica um agente de log, mas dois agentes de log opcionais são fornecidos com a versão Kubernetes: [Stackdriver Logging](/docs/user-guide/logging/stackdriver) para uso com o Google Cloud Platform e [Elasticsearch](/docs/user-guide/logging/elasticsearch). Você pode encontrar mais informações e instruções nos documentos dedicados. Ambos usam [fluentd](http://www.fluentd.org/) com configuração customizada como um agente no nó. + +### Usando um contêiner sidecar com o agente de log + +Você pode usar um contêiner sidecar de uma das seguintes maneiras: + +- O container sidecar transmite os logs do aplicativo para seu próprio `stdout`. +- O contêiner do sidecar executa um agente de log, configurado para selecionar logs de um contêiner de aplicativo. + +#### Streaming sidecar conteiner + +![Conteiner sidecar com um streaming container](/images/docs/user-guide/logging/logging-with-streaming-sidecar.png) + +Fazendo com que seus contêineres de sidecar fluam para seus próprios `stdout` e `stderr`, você pode tirar proveito do kubelet e do agente de log que já executam em cada nó. Os contêineres sidecar lêem logs de um arquivo, socket ou journald. Cada contêiner sidecar individual imprime o log em seu próprio `stdout` ou `stderr` stream. + +Essa abordagem permite separar vários fluxos de logs de diferentes partes do seu aplicativo, algumas das quais podem não ter suporte para gravar em `stdout` ou `stderr`. A lógica por trás do redirecionamento de logs é mínima, portanto dificilmente representa uma sobrecarga significativa. Além disso, como `stdout` e `stderr` são manipulados pelo kubelet, você pode usar ferramentas internas como o `kubectl logs`. + +Considere o seguinte exemplo. Um pod executa um único contêiner e grava em dois arquivos de log diferentes, usando dois formatos diferentes. Aqui está um arquivo de configuração para o Pod: + +{{< codenew file="admin/logging/two-files-counter-pod.yaml" >}} + +Seria uma bagunça ter entradas de log de diferentes formatos no mesmo fluxo de logs, mesmo se você conseguisse redirecionar os dois componentes para o fluxo `stdout` do contêiner. Em vez disso, você pode introduzir dois contêineres sidecar. Cada contêiner sidecar pode direcionar um arquivo de log específico de um volume compartilhado e depois redirecionar os logs para seu próprio fluxo `stdout`. + +Aqui está um arquivo de configuração para um pod que possui dois contêineres sidecar: + +{{< codenew file="admin/logging/two-files-counter-pod-streaming-sidecar.yaml" >}} + +Agora, quando você executa este pod, é possível acessar cada fluxo de log separadamente, executando os seguintes comandos: + +```shell +kubectl logs counter count-log-1 +``` + +``` +0: Mon Jan 1 00:00:00 UTC 2001 +1: Mon Jan 1 00:00:01 UTC 2001 +2: Mon Jan 1 00:00:02 UTC 2001 +... +``` + +```shell +kubectl logs counter count-log-2 +``` + +``` +Mon Jan 1 00:00:00 UTC 2001 INFO 0 +Mon Jan 1 00:00:01 UTC 2001 INFO 1 +Mon Jan 1 00:00:02 UTC 2001 INFO 2 +... +``` + +O agente no nível do nó instalado em seu cluster coleta esses fluxos de logs automaticamente sem nenhuma configuração adicional. Se desejar, você pode configurar o agente para analisar as linhas de log, dependendo do contêiner de origem. + +Observe que, apesar do baixo uso da CPU e da memória (ordem de alguns milicores por CPU e ordem de vários megabytes de memória), gravar logs em um arquivo e depois transmiti-los para o `stdout` pode duplicar o uso do disco. Se você tem um aplicativo que grava em um único arquivo, geralmente é melhor definir `/dev/stdout` como destino, em vez de implementar a abordagem de contêiner de transmissão no sidecar. + +Os contêineres sidecar também podem ser usados para rotacionar arquivos de log que não podem ser rotacionados pelo próprio aplicativo. Um exemplo dessa abordagem é um pequeno contêiner executando `logrotate` periodicamente. +No entanto, é recomendável usar o `stdout` e o `stderr` diretamente e deixar as políticas de rotação e retenção no kubelet. + +#### Contêiner sidecar com um agente de log + +![Contêiner sidecar com um agente de log](/images/docs/user-guide/logging/logging-with-sidecar-agent.png) + +Se o agente de log no nível do nó não for flexível o suficiente para sua situação, você poderá criar um contêiner secundário com um agente de log separado que você configurou especificamente para executar com seu aplicativo. + +{{< note >}} +O uso de um agente de log em um contêiner sidecar pode levar a um consumo significativo de recursos. Além disso, você não poderá acessar esses logs usando o comando `kubectl logs`, porque eles não são controlados pelo kubelet. +{{< /note >}} + +Como exemplo, você pode usar o [Stackdriver](/docs/tasks/debug-application-cluster/logging-stackdriver/), que usa fluentd como um agente de log. Aqui estão dois arquivos de configuração que você pode usar para implementar essa abordagem. O primeiro arquivo contém um [ConfigMap](/docs/tasks/configure-pod-container/configure-pod-configmap/) para configurar o fluentd. + +{{< codenew file="admin/logging/fluentd-sidecar-config.yaml" >}} + +{{< note >}} +A configuração do fluentd está além do escopo deste artigo. Para obter informações sobre como configurar o fluentd, consulte a [documentação oficial do fluentd](http://docs.fluentd.org/). +{{< /note >}} + +O segundo arquivo descreve um pod que possui um contêiner sidecar rodando fluentemente. +O pod monta um volume onde o fluentd pode coletar seus dados de configuração. + +{{< codenew file="admin/logging/two-files-counter-pod-agent-sidecar.yaml" >}} + +Depois de algum tempo, você pode encontrar mensagens de log na interface do Stackdriver. + +Lembre-se de que este é apenas um exemplo e você pode realmente substituir o fluentd por qualquer agente de log, lendo de qualquer fonte dentro de um contêiner de aplicativo. + +### Expondo logs diretamente do aplicativo + +![Expondo logs diretamente do aplicativo](/images/docs/user-guide/logging/logging-from-application.png) + +Você pode implementar o log no nível do cluster, expondo ou enviando logs diretamente de todos os aplicativos; no entanto, a implementação desse mecanismo de log está fora do escopo do Kubernetes. + +{{% /capture %}} diff --git a/content/pt/examples/admin/logging/fluentd-sidecar-config.yaml b/content/pt/examples/admin/logging/fluentd-sidecar-config.yaml new file mode 100644 index 0000000000..eea1849b03 --- /dev/null +++ b/content/pt/examples/admin/logging/fluentd-sidecar-config.yaml @@ -0,0 +1,25 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: fluentd-config +data: + fluentd.conf: | + + type tail + format none + path /var/log/1.log + pos_file /var/log/1.log.pos + tag count.format1 + + + + type tail + format none + path /var/log/2.log + pos_file /var/log/2.log.pos + tag count.format2 + + + + type google_cloud + diff --git a/content/pt/examples/admin/logging/two-files-counter-pod-agent-sidecar.yaml b/content/pt/examples/admin/logging/two-files-counter-pod-agent-sidecar.yaml new file mode 100644 index 0000000000..b37b616e6f --- /dev/null +++ b/content/pt/examples/admin/logging/two-files-counter-pod-agent-sidecar.yaml @@ -0,0 +1,39 @@ +apiVersion: v1 +kind: Pod +metadata: + name: counter +spec: + containers: + - name: count + image: busybox + args: + - /bin/sh + - -c + - > + i=0; + while true; + do + echo "$i: $(date)" >> /var/log/1.log; + echo "$(date) INFO $i" >> /var/log/2.log; + i=$((i+1)); + sleep 1; + done + volumeMounts: + - name: varlog + mountPath: /var/log + - name: count-agent + image: k8s.gcr.io/fluentd-gcp:1.30 + env: + - name: FLUENTD_ARGS + value: -c /etc/fluentd-config/fluentd.conf + volumeMounts: + - name: varlog + mountPath: /var/log + - name: config-volume + mountPath: /etc/fluentd-config + volumes: + - name: varlog + emptyDir: {} + - name: config-volume + configMap: + name: fluentd-config diff --git a/content/pt/examples/admin/logging/two-files-counter-pod-streaming-sidecar.yaml b/content/pt/examples/admin/logging/two-files-counter-pod-streaming-sidecar.yaml new file mode 100644 index 0000000000..87bd198cfd --- /dev/null +++ b/content/pt/examples/admin/logging/two-files-counter-pod-streaming-sidecar.yaml @@ -0,0 +1,38 @@ +apiVersion: v1 +kind: Pod +metadata: + name: counter +spec: + containers: + - name: count + image: busybox + args: + - /bin/sh + - -c + - > + i=0; + while true; + do + echo "$i: $(date)" >> /var/log/1.log; + echo "$(date) INFO $i" >> /var/log/2.log; + i=$((i+1)); + sleep 1; + done + volumeMounts: + - name: varlog + mountPath: /var/log + - name: count-log-1 + image: busybox + args: [/bin/sh, -c, 'tail -n+1 -f /var/log/1.log'] + volumeMounts: + - name: varlog + mountPath: /var/log + - name: count-log-2 + image: busybox + args: [/bin/sh, -c, 'tail -n+1 -f /var/log/2.log'] + volumeMounts: + - name: varlog + mountPath: /var/log + volumes: + - name: varlog + emptyDir: {} diff --git a/content/pt/examples/admin/logging/two-files-counter-pod.yaml b/content/pt/examples/admin/logging/two-files-counter-pod.yaml new file mode 100644 index 0000000000..6ebeb717a1 --- /dev/null +++ b/content/pt/examples/admin/logging/two-files-counter-pod.yaml @@ -0,0 +1,26 @@ +apiVersion: v1 +kind: Pod +metadata: + name: counter +spec: + containers: + - name: count + image: busybox + args: + - /bin/sh + - -c + - > + i=0; + while true; + do + echo "$i: $(date)" >> /var/log/1.log; + echo "$(date) INFO $i" >> /var/log/2.log; + i=$((i+1)); + sleep 1; + done + volumeMounts: + - name: varlog + mountPath: /var/log + volumes: + - name: varlog + emptyDir: {} diff --git a/content/pt/examples/debug/counter-pod.yaml b/content/pt/examples/debug/counter-pod.yaml new file mode 100644 index 0000000000..f997886386 --- /dev/null +++ b/content/pt/examples/debug/counter-pod.yaml @@ -0,0 +1,10 @@ +apiVersion: v1 +kind: Pod +metadata: + name: counter +spec: + containers: + - name: count + image: busybox + args: [/bin/sh, -c, + 'i=0; while true; do echo "$i: $(date)"; i=$((i+1)); sleep 1; done'] From 957444d1b3b69dfc964ed81dbfcac4728f92317f Mon Sep 17 00:00:00 2001 From: Michael Michael Date: Sun, 15 Mar 2020 18:02:35 -0500 Subject: [PATCH 051/140] Update user-guide-windows-containers.md (#19209) * Update user-guide-windows-containers.md * Update user-guide-windows-containers.md * Update content/en/docs/setup/production-environment/windows/user-guide-windows-containers.md Co-Authored-By: Tim Bannister Co-authored-by: Tim Bannister --- .../windows/user-guide-windows-containers.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/content/en/docs/setup/production-environment/windows/user-guide-windows-containers.md b/content/en/docs/setup/production-environment/windows/user-guide-windows-containers.md index 2366f61018..a4f177b364 100644 --- a/content/en/docs/setup/production-environment/windows/user-guide-windows-containers.md +++ b/content/en/docs/setup/production-environment/windows/user-guide-windows-containers.md @@ -107,6 +107,14 @@ Port mapping is also supported, but for simplicity in this example the container Windows container hosts are not able to access the IP of services scheduled on them due to current platform limitations of the Windows networking stack. Only Windows pods are able to access service IPs. {{< /note >}} +## Observability + +### Capturing logs from workloads + +Logs are an important element of observability; they enable users to gain insights into the operational aspect of workloads and are a key ingredient to troubleshooting issues. Because Windows containers and workloads inside Windows containers behave differently from Linux containers, users had a hard time collecting logs, limiting operational visibility. Windows workloads for example are usually configured to log to ETW (Event Tracing for Windows) or push entries to the application event log. [LogMonitor](https://github.com/microsoft/windows-container-tools/tree/master/LogMonitor), an open source tool by Microsoft, is the recommended way to monitor configured log sources inside a Windows container. LogMonitor supports monitoring event logs, ETW providers, and custom application logs, piping them to STDOUT for consumption by `kubectl logs `. + +Follow the instructions in the LogMonitor GitHub page to copy its binaries and configuration files to all your containers and add the necessary entrypoints for LogMonitor to push your logs to STDOUT. + ## Using configurable Container usernames Starting with Kubernetes v1.16, Windows containers can be configured to run their entrypoints and processes with different usernames than the image defaults. The way this is achieved is a bit different from the way it is done for Linux containers. Learn more about it [here](/docs/tasks/configure-pod-container/configure-runasusername/). From 83460e04a272c5f76822a3c20f65b5fdfd08124b Mon Sep 17 00:00:00 2001 From: Rajesh Deshpande Date: Mon, 16 Mar 2020 08:58:35 +0530 Subject: [PATCH 052/140] Adding references to relevant tasks. (#19608) Adding references to relevant tasks. --- .../services-networking/connect-applications-service.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/content/en/docs/concepts/services-networking/connect-applications-service.md b/content/en/docs/concepts/services-networking/connect-applications-service.md index 44ff5c331d..bc17b74d15 100644 --- a/content/en/docs/concepts/services-networking/connect-applications-service.md +++ b/content/en/docs/concepts/services-networking/connect-applications-service.md @@ -419,3 +419,11 @@ LoadBalancer Ingress: a320587ffd19711e5a37606cf4a74574-1142138393.us-east-1.el ``` {{% /capture %}} + +{{% capture whatsnext %}} + +* Learn more about [Using a Service to Access an Application in a Cluster](/docs/tasks/access-application-cluster/service-access-application-cluster/) +* Learn more about [Connecting a Front End to a Back End Using a Service](/docs/tasks/access-application-cluster/connecting-frontend-backend/) +* Learn more about [Creating an External Load Balancer](/docs/tasks/access-application-cluster/create-external-load-balancer/) + +{{% /capture %}} From 04b20e1dcee880f6f7c0b45c445c02ebe3cd8f99 Mon Sep 17 00:00:00 2001 From: Enshin Andrey Date: Mon, 16 Mar 2020 12:38:35 +0900 Subject: [PATCH 053/140] Update custom-resources.md (#19597) --- .../extend-kubernetes/api-extension/custom-resources.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/concepts/extend-kubernetes/api-extension/custom-resources.md b/content/en/docs/concepts/extend-kubernetes/api-extension/custom-resources.md index d0b990e0da..91c41796e6 100644 --- a/content/en/docs/concepts/extend-kubernetes/api-extension/custom-resources.md +++ b/content/en/docs/concepts/extend-kubernetes/api-extension/custom-resources.md @@ -167,7 +167,7 @@ CRDs are easier to create than Aggregated APIs. | CRDs | Aggregated API | | --------------------------- | -------------- | -| Do not require programming. Users can choose any language for a CRD controller. | Requires programming in Go and building binary and image. Users can choose any language for a CRD controller. | +| Do not require programming. Users can choose any language for a CRD controller. | Requires programming in Go and building binary and image. | | No additional service to run; CRs are handled by API Server. | An additional service to create and that could fail. | | No ongoing support once the CRD is created. Any bug fixes are picked up as part of normal Kubernetes Master upgrades. | May need to periodically pickup bug fixes from upstream and rebuild and update the Aggregated APIserver. | | No need to handle multiple versions of your API. For example: when you control the client for this resource, you can upgrade it in sync with the API. | You need to handle multiple versions of your API, for example: when developing an extension to share with the world. | From d089656267cc989107423b49c18f4a570ea976ae Mon Sep 17 00:00:00 2001 From: Jacky Wu Date: Mon, 16 Mar 2020 11:46:35 +0800 Subject: [PATCH 054/140] doc: remove federation tests in example and some federation content. (#19581) --- .../dns-debugging-resolution.md | 9 ----- content/en/examples/examples_test.go | 14 ++----- .../federation/policy-engine-deployment.yaml | 37 ------------------- .../federation/policy-engine-service.yaml | 13 ------- .../federation/replicaset-example-policy.yaml | 21 ----------- .../scheduling-policy-admission.yaml | 29 --------------- 6 files changed, 3 insertions(+), 120 deletions(-) delete mode 100644 content/en/examples/federation/policy-engine-deployment.yaml delete mode 100644 content/en/examples/federation/policy-engine-service.yaml delete mode 100644 content/en/examples/federation/replicaset-example-policy.yaml delete mode 100644 content/en/examples/federation/scheduling-policy-admission.yaml diff --git a/content/en/docs/tasks/administer-cluster/dns-debugging-resolution.md b/content/en/docs/tasks/administer-cluster/dns-debugging-resolution.md index 352a709386..8c494935b6 100644 --- a/content/en/docs/tasks/administer-cluster/dns-debugging-resolution.md +++ b/content/en/docs/tasks/administer-cluster/dns-debugging-resolution.md @@ -265,15 +265,6 @@ work properly owing to a known issue with Alpine. Check [here](https://github.com/kubernetes/kubernetes/issues/30215) for more information. -## Kubernetes Federation (Multiple Zone support) - -Release 1.3 introduced Cluster Federation support for multi-site Kubernetes -installations. This required some minor (backward-compatible) changes to the -way the Kubernetes cluster DNS server processes DNS queries, to facilitate -the lookup of federated services (which span multiple Kubernetes clusters). -See the [Cluster Federation Administrators' Guide](/docs/concepts/cluster-administration/federation/) -for more details on Cluster Federation and multi-site support. - ## References - [DNS for Services and Pods](/docs/concepts/services-networking/dns-pod-service/) diff --git a/content/en/examples/examples_test.go b/content/en/examples/examples_test.go index ee29c3a68b..7c9664b64c 100644 --- a/content/en/examples/examples_test.go +++ b/content/en/examples/examples_test.go @@ -34,8 +34,6 @@ import ( utilfeature "k8s.io/apiserver/pkg/util/feature" "k8s.io/kubernetes/pkg/api/legacyscheme" "k8s.io/kubernetes/pkg/api/testapi" - "k8s.io/kubernetes/pkg/apis/admissionregistration" - ar_validation "k8s.io/kubernetes/pkg/apis/admissionregistration/validation" "k8s.io/kubernetes/pkg/apis/apps" apps_validation "k8s.io/kubernetes/pkg/apis/apps/validation" "k8s.io/kubernetes/pkg/apis/autoscaling" @@ -434,12 +432,6 @@ func TestExampleObjectSchemas(t *testing.T) { "node-problem-detector-configmap": {&apps.DaemonSet{}}, "termination": {&api.Pod{}}, }, - "federation": { - "policy-engine-deployment": {&apps.Deployment{}}, - "policy-engine-service": {&api.Service{}}, - "replicaset-example-policy": {&apps.ReplicaSet{}}, - "scheduling-policy-admission": {&api.ConfigMap{}}, - }, "podpreset": { "allow-db": {&settings.PodPreset{}}, "allow-db-merged": {&api.Pod{}}, @@ -525,9 +517,9 @@ func TestExampleObjectSchemas(t *testing.T) { "redis": {&api.Pod{}}, }, "policy": { - "privileged-psp": {&policy.PodSecurityPolicy{}}, - "restricted-psp": {&policy.PodSecurityPolicy{}}, - "example-psp": {&policy.PodSecurityPolicy{}}, + "privileged-psp": {&policy.PodSecurityPolicy{}}, + "restricted-psp": {&policy.PodSecurityPolicy{}}, + "example-psp": {&policy.PodSecurityPolicy{}}, "zookeeper-pod-disruption-budget-maxunavailable": {&policy.PodDisruptionBudget{}}, "zookeeper-pod-disruption-budget-minunavailable": {&policy.PodDisruptionBudget{}}, }, diff --git a/content/en/examples/federation/policy-engine-deployment.yaml b/content/en/examples/federation/policy-engine-deployment.yaml deleted file mode 100644 index 168af7ba4c..0000000000 --- a/content/en/examples/federation/policy-engine-deployment.yaml +++ /dev/null @@ -1,37 +0,0 @@ -apiVersion: apps/v1 -kind: Deployment -metadata: - labels: - app: opa - name: opa - namespace: federation-system -spec: - replicas: 1 - selector: - matchLabels: - app: opa - template: - metadata: - labels: - app: opa - name: opa - spec: - containers: - - name: opa - image: openpolicyagent/opa:0.4.10 - args: - - "run" - - "--server" - - name: kube-mgmt - image: openpolicyagent/kube-mgmt:0.2 - args: - - "-kubeconfig=/srv/kubernetes/kubeconfig" - - "-cluster=federation/v1beta1/clusters" - volumeMounts: - - name: federation-kubeconfig - mountPath: /srv/kubernetes - readOnly: true - volumes: - - name: federation-kubeconfig - secret: - secretName: federation-controller-manager-kubeconfig diff --git a/content/en/examples/federation/policy-engine-service.yaml b/content/en/examples/federation/policy-engine-service.yaml deleted file mode 100644 index 982870b06b..0000000000 --- a/content/en/examples/federation/policy-engine-service.yaml +++ /dev/null @@ -1,13 +0,0 @@ -apiVersion: v1 -kind: Service -metadata: - name: opa - namespace: federation-system -spec: - selector: - app: opa - ports: - - name: http - protocol: TCP - port: 8181 - targetPort: 8181 diff --git a/content/en/examples/federation/replicaset-example-policy.yaml b/content/en/examples/federation/replicaset-example-policy.yaml deleted file mode 100644 index 43dc83b18b..0000000000 --- a/content/en/examples/federation/replicaset-example-policy.yaml +++ /dev/null @@ -1,21 +0,0 @@ -apiVersion: apps/v1 -kind: ReplicaSet -metadata: - labels: - app: nginx-pci - name: nginx-pci - annotations: - requires-pci: "true" -spec: - replicas: 3 - selector: - matchLabels: - app: nginx-pci - template: - metadata: - labels: - app: nginx-pci - spec: - containers: - - image: nginx - name: nginx-pci diff --git a/content/en/examples/federation/scheduling-policy-admission.yaml b/content/en/examples/federation/scheduling-policy-admission.yaml deleted file mode 100644 index a164722425..0000000000 --- a/content/en/examples/federation/scheduling-policy-admission.yaml +++ /dev/null @@ -1,29 +0,0 @@ -apiVersion: v1 -kind: ConfigMap -metadata: - name: admission - namespace: federation-system -data: - config.yml: | - apiVersion: apiserver.k8s.io/v1alpha1 - kind: AdmissionConfiguration - plugins: - - name: SchedulingPolicy - path: /etc/kubernetes/admission/scheduling-policy-config.yml - scheduling-policy-config.yml: | - kubeconfig: /etc/kubernetes/admission/opa-kubeconfig - opa-kubeconfig: | - clusters: - - name: opa-api - cluster: - server: http://opa.federation-system.svc.cluster.local:8181/v0/data/kubernetes/placement - users: - - name: scheduling-policy - user: - token: deadbeefsecret - contexts: - - name: default - context: - cluster: opa-api - user: scheduling-policy - current-context: default From e2c9be736a7014e38b8c1e700d2a24cbef1cc434 Mon Sep 17 00:00:00 2001 From: divya-mohan0209 Date: Mon, 16 Mar 2020 09:18:35 +0530 Subject: [PATCH 055/140] Signposting added in /docs/home to redirect users to /training (#19592) * Update _index.md Signpost readers from /docs/home/ to /training/ #19554 * Added the link to reroute users from /docs/home to /training * Signposting for users added * Updated _index.md --- content/en/docs/home/_index.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/content/en/docs/home/_index.md b/content/en/docs/home/_index.md index 31f37880ff..6ce04e800b 100644 --- a/content/en/docs/home/_index.md +++ b/content/en/docs/home/_index.md @@ -36,9 +36,14 @@ cards: button_path: "/docs/setup" - name: tasks title: "Learn how to use Kubernetes" - description: "Look up common tasks and how to perform them using a short sequence of steps." + description: "Look up common tasks and how to perform them using a short sequence of steps." button: "View Tasks" button_path: "/docs/tasks" +- name: training + title: "Training" + description: "Get certified in Kubernetes and make your cloud native projects successful!" + button: "View training" + button_path: "/training" - name: reference title: Look up reference information description: Browse terminology, command line syntax, API resource types, and setup tool documentation. From c667b22ef2b231d615480752c0ee956fd18a8e90 Mon Sep 17 00:00:00 2001 From: Hanyu Cui Date: Sun, 15 Mar 2020 21:04:35 -0700 Subject: [PATCH 056/140] Instructions to upgrade Bash for shell autocompletion (#19561) * Instructions to upgrade Bash for shell autocompletion * Tweak * Clarification --- .../en/docs/tasks/tools/install-kubectl.md | 21 +++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/content/en/docs/tasks/tools/install-kubectl.md b/content/en/docs/tasks/tools/install-kubectl.md index 4a799e5f01..83c9d1761b 100644 --- a/content/en/docs/tasks/tools/install-kubectl.md +++ b/content/en/docs/tasks/tools/install-kubectl.md @@ -385,6 +385,27 @@ However, the kubectl completion script depends on [**bash-completion**](https:// there are two versions of bash-completion, v1 and v2. V1 is for Bash 3.2 (which is the default on macOS), and v2 is for Bash 4.1+. The kubectl completion script **doesn't work** correctly with bash-completion v1 and Bash 3.2. It requires **bash-completion v2** and **Bash 4.1+**. Thus, to be able to correctly use kubectl completion on macOS, you have to install and use Bash 4.1+ ([*instructions*](https://itnext.io/upgrading-bash-on-macos-7138bd1066ba)). The following instructions assume that you use Bash 4.1+ (that is, any Bash version of 4.1 or newer). {{< /warning >}} +### Upgrade Bash + +The instructions here assume you use Bash 4.1+. You can check your Bash's version by running: + +```shell +echo $BASH_VERSION +``` + +If it is too old, you can install/upgrade it using Homebrew: + +```shell +brew install bash +``` + +Reload your shell and verify that the desired version is being used: + +```shell +echo $BASH_VERSION $SHELL +``` + +Homebrew usually installs it at `/usr/local/bin/bash`. ### Install bash-completion From 588dfad48921955ebbf61442bf03460b39d60286 Mon Sep 17 00:00:00 2001 From: vergissmiNET <45730016+vergissmiNET@users.noreply.github.com> Date: Mon, 16 Mar 2020 05:14:35 +0100 Subject: [PATCH 057/140] Update cluster-intro.html (#19446) --- .../kubernetes-basics/create-cluster/cluster-intro.html | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/tutorials/kubernetes-basics/create-cluster/cluster-intro.html b/content/en/docs/tutorials/kubernetes-basics/create-cluster/cluster-intro.html index 2a6af0af4c..13d3d99758 100644 --- a/content/en/docs/tutorials/kubernetes-basics/create-cluster/cluster-intro.html +++ b/content/en/docs/tutorials/kubernetes-basics/create-cluster/cluster-intro.html @@ -77,7 +77,7 @@ weight: 10
    -

    Masters manage the cluster and the nodes are used to host the running applications.

    +

    Masters manage the cluster and the nodes that are used to host the running applications.

    From 8d1f0dec7b06b556c87fbf1dfb3b58b9702619cc Mon Sep 17 00:00:00 2001 From: Chris McInerney <40616093+c-mcinerney@users.noreply.github.com> Date: Mon, 16 Mar 2020 00:20:35 -0400 Subject: [PATCH 058/140] Update D2iQ Kommander link (#19353) Update the table with the proper link for D2iQ's Kommander. --- content/en/docs/setup/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/setup/_index.md b/content/en/docs/setup/_index.md index 3cf2afa968..ba241d61ab 100644 --- a/content/en/docs/setup/_index.md +++ b/content/en/docs/setup/_index.md @@ -76,7 +76,7 @@ The following production environment solutions table lists the providers and the | [CloudStack](https://cloudstack.apache.org/) | | | | | ✔| | [Canonical](https://ubuntu.com/kubernetes) | ✔ | ✔ | ✔ | ✔ |✔ | ✔ | [Containership](https://containership.io) | ✔ |✔ | | | | -| [D2iQ](https://d2iq.com/) | | [Kommander](https://d2iq.com/solutions/ksphere) | [Konvoy](https://d2iq.com/solutions/ksphere/konvoy) | [Konvoy](https://d2iq.com/solutions/ksphere/konvoy) | [Konvoy](https://d2iq.com/solutions/ksphere/konvoy) | [Konvoy](https://d2iq.com/solutions/ksphere/konvoy) | +| [D2iQ](https://d2iq.com/) | | [Kommander](https://docs.d2iq.com/ksphere/kommander/) | [Konvoy](https://d2iq.com/solutions/ksphere/konvoy) | [Konvoy](https://d2iq.com/solutions/ksphere/konvoy) | [Konvoy](https://d2iq.com/solutions/ksphere/konvoy) | [Konvoy](https://d2iq.com/solutions/ksphere/konvoy) | | [Digital Rebar](https://provision.readthedocs.io/en/tip/README.html) | | | | | | ✔ | [DigitalOcean](https://www.digitalocean.com/products/kubernetes/) | ✔ | | | | | | [Docker Enterprise](https://www.docker.com/products/docker-enterprise) | |✔ | ✔ | | | ✔ From 186b9e0186f4771da1242bb7362ba289875aafdd Mon Sep 17 00:00:00 2001 From: Stephen Cprek Date: Sun, 15 Mar 2020 23:22:35 -0500 Subject: [PATCH 059/140] Clarify 2 second extended grace period for preStop (#19348) --- content/en/docs/concepts/workloads/pods/pod.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/concepts/workloads/pods/pod.md b/content/en/docs/concepts/workloads/pods/pod.md index 55e71a5c72..d64227be48 100644 --- a/content/en/docs/concepts/workloads/pods/pod.md +++ b/content/en/docs/concepts/workloads/pods/pod.md @@ -175,7 +175,7 @@ An example flow: 1. The Pod in the API server is updated with the time beyond which the Pod is considered "dead" along with the grace period. 1. Pod shows up as "Terminating" when listed in client commands 1. (simultaneous with 3) When the Kubelet sees that a Pod has been marked as terminating because the time in 2 has been set, it begins the Pod shutdown process. - 1. If one of the Pod's containers has defined a [preStop hook](/docs/concepts/containers/container-lifecycle-hooks/#hook-details), it is invoked inside of the container. If the `preStop` hook is still running after the grace period expires, step 2 is then invoked with a small (2 second) extended grace period. + 1. If one of the Pod's containers has defined a [preStop hook](/docs/concepts/containers/container-lifecycle-hooks/#hook-details), it is invoked inside of the container. If the `preStop` hook is still running after the grace period expires, step 2 is then invoked with a small (2 second) one-time extended grace period. You must modify `terminationGracePeriodSeconds` if the `preStop` hook needs longer to complete. 1. The container is sent the TERM signal. Note that not all containers in the Pod will receive the TERM signal at the same time and may each require a `preStop` hook if the order in which they shut down matters. 1. (simultaneous with 3) Pod is removed from endpoints list for service, and are no longer considered part of the set of running Pods for replication controllers. Pods that shutdown slowly cannot continue to serve traffic as load balancers (like the service proxy) remove them from their rotations. 1. When the grace period expires, any processes still running in the Pod are killed with SIGKILL. From e37b117a3f4d1fe2d63ca9061e41c5950da9697c Mon Sep 17 00:00:00 2001 From: Naoki Oketani Date: Mon, 16 Mar 2020 13:24:35 +0900 Subject: [PATCH 060/140] remove example of deprecated option: kubectl get --export (#19336) --- content/en/docs/reference/kubectl/cheatsheet.md | 1 - 1 file changed, 1 deletion(-) diff --git a/content/en/docs/reference/kubectl/cheatsheet.md b/content/en/docs/reference/kubectl/cheatsheet.md index 83a553e5e1..6269c5c362 100644 --- a/content/en/docs/reference/kubectl/cheatsheet.md +++ b/content/en/docs/reference/kubectl/cheatsheet.md @@ -148,7 +148,6 @@ kubectl get pods -o wide # List all pods in the current nam kubectl get deployment my-dep # List a particular deployment kubectl get pods # List all pods in the namespace kubectl get pod my-pod -o yaml # Get a pod's YAML -kubectl get pod my-pod -o yaml --export # Get a pod's YAML without cluster specific information # Describe commands with verbose output kubectl describe nodes my-node From a9719704e934ac50923d4523f11d3b999dae0160 Mon Sep 17 00:00:00 2001 From: Rajesh Deshpande Date: Mon, 16 Mar 2020 09:56:36 +0530 Subject: [PATCH 061/140] Removing non required details about the context (#19334) Removing non-required details about the context --- .../tasks/administer-cluster/namespaces.md | 86 ++----------------- 1 file changed, 8 insertions(+), 78 deletions(-) diff --git a/content/en/docs/tasks/administer-cluster/namespaces.md b/content/en/docs/tasks/administer-cluster/namespaces.md index 9ccb429b46..de2230a6a4 100644 --- a/content/en/docs/tasks/administer-cluster/namespaces.md +++ b/content/en/docs/tasks/administer-cluster/namespaces.md @@ -188,88 +188,22 @@ This delete is asynchronous, so for a time you will see the namespace in the `Te To demonstrate this, let's spin up a simple Deployment and Pods in the `development` namespace. - We first check what is the current context: - ```shell - kubectl config view - ``` - ```yaml - apiVersion: v1 - clusters: - cluster: - certificate-authority-data: REDACTED - server: https://130.211.122.180 - name: lithe-cocoa-92103_kubernetes - contexts: - context: - cluster: lithe-cocoa-92103_kubernetes - user: lithe-cocoa-92103_kubernetes - name: lithe-cocoa-92103_kubernetes - current-context: lithe-cocoa-92103_kubernetes - kind: Config - preferences: {} - users: - name: lithe-cocoa-92103_kubernetes - user: - client-certificate-data: REDACTED - client-key-data: REDACTED - token: 65rZW78y8HbwXXtSXuUw9DbP4FLjHi4b - name: lithe-cocoa-92103_kubernetes-basic-auth - user: - password: h5M0FtUUIflBSdI7 - username: admin - ``` - - ```shell - kubectl config current-context - ``` - ``` - lithe-cocoa-92103_kubernetes - ``` - - The next step is to define a context for the kubectl client to work in each namespace. The values of "cluster" and "user" fields are copied from the current context. - - ```shell - kubectl config set-context dev --namespace=development --cluster=lithe-cocoa-92103_kubernetes --user=lithe-cocoa-92103_kubernetes - kubectl config set-context prod --namespace=production --cluster=lithe-cocoa-92103_kubernetes --user=lithe-cocoa-92103_kubernetes - ``` - - The above commands provided two request contexts you can alternate against depending on what namespace you - wish to work against. - - Let's switch to operate in the `development` namespace. - - ```shell - kubectl config use-context dev - ``` - - You can verify your current context by doing the following: - - ```shell - kubectl config current-context - dev - ``` - - At this point, all requests we make to the Kubernetes cluster from the command line are scoped to the `development` namespace. - - Let's create some contents. - - ```shell - kubectl run snowflake --image=k8s.gcr.io/serve_hostname --replicas=2 + kubectl run snowflake --image=k8s.gcr.io/serve_hostname --replicas=2 -n=development ``` We have just created a deployment whose replica size is 2 that is running the pod called `snowflake` with a basic container that just serves the hostname. Note that `kubectl run` creates deployments only on Kubernetes cluster >= v1.2. If you are running older versions, it creates replication controllers instead. If you want to obtain the old behavior, use `--generator=run/v1` to create replication controllers. See [`kubectl run`](/docs/reference/generated/kubectl/kubectl-commands/#run) for more details. ```shell - kubectl get deployment + kubectl get deployment -n=development ``` ``` NAME READY UP-TO-DATE AVAILABLE AGE snowflake 2/2 2 2 2m ``` ```shell - kubectl get pods -l run=snowflake + kubectl get pods -l run=snowflake -n=development ``` ``` NAME READY STATUS RESTARTS AGE @@ -281,23 +215,19 @@ This delete is asynchronous, so for a time you will see the namespace in the `Te Let's switch to the `production` namespace and show how resources in one namespace are hidden from the other. - ```shell - kubectl config use-context prod - ``` - The `production` namespace should be empty, and the following commands should return nothing. ```shell - kubectl get deployment - kubectl get pods + kubectl get deployment -n=production + kubectl get pods -n=production ``` Production likes to run cattle, so let's create some cattle pods. ```shell - kubectl run cattle --image=k8s.gcr.io/serve_hostname --replicas=5 + kubectl run cattle --image=k8s.gcr.io/serve_hostname --replicas=5 -n=production - kubectl get deployment + kubectl get deployment -n=production ``` ``` NAME READY UP-TO-DATE AVAILABLE AGE @@ -305,7 +235,7 @@ This delete is asynchronous, so for a time you will see the namespace in the `Te ``` ```shell - kubectl get pods -l run=cattle + kubectl get pods -l run=cattle -n=production ``` ``` NAME READY STATUS RESTARTS AGE From 5f496a6ce641975f9b44091aaefa5915877a8888 Mon Sep 17 00:00:00 2001 From: Tsahi Duek Date: Mon, 16 Mar 2020 06:28:35 +0200 Subject: [PATCH 062/140] Changed 'caBundle' description (#19341) It is not clear that the 'caBundle' field in the Mutating/Validating webhook object is used for client-side validation from the admission controller itself. The comment which appears at the source code is much more clear --> https://github.com/kubernetes/api/blob/master/admissionregistration/v1beta1/types.go#L534-L537 My suggestion is to change the description to match the documentation which is IMO much more clear --- .../access-authn-authz/extensible-admission-controllers.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/content/en/docs/reference/access-authn-authz/extensible-admission-controllers.md b/content/en/docs/reference/access-authn-authz/extensible-admission-controllers.md index 4da9bc951c..c57bcdaf34 100644 --- a/content/en/docs/reference/access-authn-authz/extensible-admission-controllers.md +++ b/content/en/docs/reference/access-authn-authz/extensible-admission-controllers.md @@ -114,7 +114,7 @@ webhooks: service: namespace: "example-namespace" name: "example-service" - caBundle: "Ci0tLS0tQk......tLS0K" + caBundle: "Ci0tLS0tQk...<`caBundle` is a PEM encoded CA bundle which will be used to validate the webhook's server certificate.>...tLS0K" admissionReviewVersions: ["v1", "v1beta1"] sideEffects: None timeoutSeconds: 5 @@ -139,7 +139,7 @@ webhooks: service: namespace: "example-namespace" name: "example-service" - caBundle: "Ci0tLS0tQk......tLS0K" + caBundle: "Ci0tLS0tQk...<`caBundle` is a PEM encoded CA bundle which will be used to validate the webhook's server certificate>...tLS0K" admissionReviewVersions: ["v1beta1"] timeoutSeconds: 5 ``` @@ -1122,7 +1122,7 @@ kind: MutatingWebhookConfiguration webhooks: - name: my-webhook.example.com clientConfig: - caBundle: "Ci0tLS0tQk......tLS0K" + caBundle: "Ci0tLS0tQk...<`caBundle` is a PEM encoded CA bundle which will be used to validate the webhook's server certificate>...tLS0K" service: namespace: my-service-namespace name: my-service-name From c68f3883c29e64af0ffc99ff9f6b693c3a00650c Mon Sep 17 00:00:00 2001 From: Rajesh Deshpande Date: Mon, 16 Mar 2020 10:00:35 +0530 Subject: [PATCH 063/140] Correcting inconsistency of storage class name (#19295) Correcting inconsistency of storage class name --- .../administer-cluster/change-default-storage-class.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/content/en/docs/tasks/administer-cluster/change-default-storage-class.md b/content/en/docs/tasks/administer-cluster/change-default-storage-class.md index 4c3351956d..a2070bcfe3 100644 --- a/content/en/docs/tasks/administer-cluster/change-default-storage-class.md +++ b/content/en/docs/tasks/administer-cluster/change-default-storage-class.md @@ -62,10 +62,10 @@ for details about addon manager and how to disable individual addons. To mark a StorageClass as non-default, you need to change its value to `false`: ```bash - kubectl patch storageclass -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"false"}}}' + kubectl patch storageclass standard -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"false"}}}' ``` - where `` is the name of your chosen StorageClass. + where `standard` is the name of your chosen StorageClass. 1. Mark a StorageClass as default: @@ -73,7 +73,7 @@ for details about addon manager and how to disable individual addons. `storageclass.kubernetes.io/is-default-class=true`. ```bash - kubectl patch storageclass -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}' + kubectl patch storageclass gold -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}' ``` Please note that at most one StorageClass can be marked as default. If two From 7b4ad25f138c35d0a0a7d8e35269934b8ae395b5 Mon Sep 17 00:00:00 2001 From: Rajesh Deshpande Date: Mon, 16 Mar 2020 10:02:35 +0530 Subject: [PATCH 064/140] Correcting example from namespaces-walkthrough task (#19294) * Correcting deployment example As 'kubectl run' command is deprecated, creating deployment with deployment manifest and apply command. * Deployment manifest for snowflake example Deployment manifest for snowflake example --- .../namespaces-walkthrough.md | 8 +++++--- .../examples/admin/snowflake-deployment.yaml | 20 +++++++++++++++++++ 2 files changed, 25 insertions(+), 3 deletions(-) create mode 100644 content/en/examples/admin/snowflake-deployment.yaml diff --git a/content/en/docs/tasks/administer-cluster/namespaces-walkthrough.md b/content/en/docs/tasks/administer-cluster/namespaces-walkthrough.md index bd136e3ae2..9a69058ceb 100644 --- a/content/en/docs/tasks/administer-cluster/namespaces-walkthrough.md +++ b/content/en/docs/tasks/administer-cluster/namespaces-walkthrough.md @@ -224,12 +224,14 @@ At this point, all requests we make to the Kubernetes cluster from the command l Let's create some contents. +{{< codenew file="admin/snowflake-deployment.yaml" >}} + +Apply the manifest to create a Deployment + ```shell -kubectl run snowflake --image=k8s.gcr.io/serve_hostname --replicas=2 +kubectl apply -f https://k8s.io/examples/admin/snowflake-deployment.yaml ``` We have just created a deployment whose replica size is 2 that is running the pod called `snowflake` with a basic container that just serves the hostname. -Note that `kubectl run` creates deployments only on Kubernetes cluster >= v1.2. If you are running older versions, it creates replication controllers instead. -If you want to obtain the old behavior, use `--generator=run/v1` to create replication controllers. See [`kubectl run`](/docs/reference/generated/kubectl/kubectl-commands/#run) for more details. ```shell kubectl get deployment diff --git a/content/en/examples/admin/snowflake-deployment.yaml b/content/en/examples/admin/snowflake-deployment.yaml new file mode 100644 index 0000000000..2f4f267916 --- /dev/null +++ b/content/en/examples/admin/snowflake-deployment.yaml @@ -0,0 +1,20 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + labels: + app: snowflake + name: snowflake +spec: + replicas: 2 + selector: + matchLabels: + app: snowflake + template: + metadata: + labels: + app: snowflake + spec: + containers: + - image: k8s.gcr.io/serve_hostname + imagePullPolicy: Always + name: snowflake From 7007e8156fb04d1ec3ce292ae5c9f2bdfd94cc34 Mon Sep 17 00:00:00 2001 From: davidgfolch Date: Mon, 16 Mar 2020 05:40:35 +0100 Subject: [PATCH 065/140] Config elasticsearch installed in minikube host (#19200) * config elasticsearch installed in minikube host Link to how to config an elasticsearch server in minikube host machine. * config elasticsearch installed in minikube host Link to how to config an elasticsearch server in minikube host machine. --- .../stateless-application/guestbook-logs-metrics-with-elk.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/tutorials/stateless-application/guestbook-logs-metrics-with-elk.md b/content/en/docs/tutorials/stateless-application/guestbook-logs-metrics-with-elk.md index af24c72999..94008289ee 100644 --- a/content/en/docs/tutorials/stateless-application/guestbook-logs-metrics-with-elk.md +++ b/content/en/docs/tutorials/stateless-application/guestbook-logs-metrics-with-elk.md @@ -111,7 +111,7 @@ There are four files to edit to create a k8s secret when you are connecting to s 1. ELASTICSEARCH_USERNAME 1. KIBANA_HOST -Set these with the information for your Elasticsearch cluster and your Kibana host. Here are some examples +Set these with the information for your Elasticsearch cluster and your Kibana host. Here are some examples (also see [*this configuration*](https://stackoverflow.com/questions/59892896/how-to-connect-from-minikube-to-elasticsearch-installed-on-host-local-developme/59892897#59892897)) #### `ELASTICSEARCH_HOSTS` 1. A nodeGroup from the Elastic Elasticsearch Helm Chart: From c5d7d3215153b97afe9f46ea7d055b9cbef4a446 Mon Sep 17 00:00:00 2001 From: Tim Bannister Date: Mon, 16 Mar 2020 04:46:35 +0000 Subject: [PATCH 066/140] Reword API Server aggregation (#19141) * Reword API Server aggregation * Document custom resources ahead of APIService CustomResourceDefinition is the newer, shinier and often more appropriate resource to help with cases where APIService was not a good fit. Switch order to mention the custom resources page first, which introduces both APIService and CustomResourceDefinition. --- .../api-extension/apiserver-aggregation.md | 29 ++++++++++--------- .../api-extension/custom-resources.md | 2 +- 2 files changed, 17 insertions(+), 14 deletions(-) diff --git a/content/en/docs/concepts/extend-kubernetes/api-extension/apiserver-aggregation.md b/content/en/docs/concepts/extend-kubernetes/api-extension/apiserver-aggregation.md index bf2d550192..8bc6e22861 100644 --- a/content/en/docs/concepts/extend-kubernetes/api-extension/apiserver-aggregation.md +++ b/content/en/docs/concepts/extend-kubernetes/api-extension/apiserver-aggregation.md @@ -5,30 +5,34 @@ reviewers: - cheftako - chenopis content_template: templates/concept -weight: 10 +weight: 20 --- {{% capture overview %}} -The aggregation layer allows Kubernetes to be extended with additional APIs, beyond what is offered by the core Kubernetes APIs. +The aggregation layer allows Kubernetes to be extended with additional APIs, beyond what is offered by the core Kubernetes APIs. +The additional APIs can either be ready-made solutions such as [service-catalog](/docs/concepts/extend-kubernetes/service-catalog/), or APIs that you develop yourself. + +The aggregation layer is different from [Custom Resources](/docs/concepts/extend-kubernetes/api-extension/custom-resources/), which are a way to make the {{< glossary_tooltip term_id="kube-apiserver" text="kube-apiserver" >}} recognise new kinds of object. {{% /capture %}} {{% capture body %}} -## Overview +## Aggregation layer -The aggregation layer enables installing additional Kubernetes-style APIs in your cluster. These can either be pre-built, existing 3rd party solutions, such as [service-catalog](https://github.com/kubernetes-incubator/service-catalog/blob/master/README.md), or user-created APIs like [apiserver-builder](https://github.com/kubernetes-incubator/apiserver-builder/blob/master/README.md), which can get you started. +The aggregation layer runs in-process with the kube-apiserver. Until an extension resource is registered, the aggregation layer will do nothing. To register an API, you add an _APIService_ object, which "claims" the URL path in the Kubernetes API. At that point, the aggregation layer will proxy anything sent to that API path (e.g. `/apis/myextension.mycompany.io/v1/…`) to the registered APIService. -The aggregation layer runs in-process with the kube-apiserver. Until an extension resource is registered, the aggregation layer will do nothing. To register an API, users must add an APIService object, which "claims" the URL path in the Kubernetes API. At that point, the aggregation layer will proxy anything sent to that API path (e.g. /apis/myextension.mycompany.io/v1/…) to the registered APIService. +The most common way to implement the APIService is to run an *extension API server* in Pod(s) that run in your cluster. If you're using the extension API server to manage resources in your cluster, the extension API server (also written as "extension-apiserver") is typically paired with one or more {{< glossary_tooltip text="controllers" term_id="controller" >}}. The apiserver-builder library provides a skeleton for both extension API servers and the associated controller(s). -Ordinarily, the APIService will be implemented by an *extension-apiserver* in a pod running in the cluster. This extension-apiserver will normally need to be paired with one or more controllers if active management of the added resources is needed. As a result, the apiserver-builder will actually provide a skeleton for both. As another example, when the service-catalog is installed, it provides both the extension-apiserver and controller for the services it provides. +### Response latency -Extension-apiservers should have low latency connections to and from the kube-apiserver. -In particular, discovery requests are required to round-trip from the kube-apiserver in five seconds or less. -If your deployment cannot achieve this, you should consider how to change it. For now, setting the -`EnableAggregatedDiscoveryTimeout=false` feature gate on the kube-apiserver -will disable the timeout restriction. It will be removed in a future release. +Extension API servers should have low latency networking to and from the kube-apiserver. +Discovery requests are required to round-trip from the kube-apiserver in five seconds or less. + +If your extension API server cannot achieve that latency requirement, consider making changes that let you meet it. You can also set the +`EnableAggregatedDiscoveryTimeout=false` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/) on the kube-apiserver +to disable the timeout restriction. This deprecated feature gate will be removed in a future release. {{% /capture %}} @@ -37,7 +41,6 @@ will disable the timeout restriction. It will be removed in a future release. * To get the aggregator working in your environment, [configure the aggregation layer](/docs/tasks/access-kubernetes-api/configure-aggregation-layer/). * Then, [setup an extension api-server](/docs/tasks/access-kubernetes-api/setup-extension-api-server/) to work with the aggregation layer. * Also, learn how to [extend the Kubernetes API using Custom Resource Definitions](/docs/tasks/access-kubernetes-api/extend-api-custom-resource-definitions/). +* Read the specification for [APIService](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#apiservice-v1-apiregistration-k8s-io) {{% /capture %}} - - diff --git a/content/en/docs/concepts/extend-kubernetes/api-extension/custom-resources.md b/content/en/docs/concepts/extend-kubernetes/api-extension/custom-resources.md index 91c41796e6..f2d4b814ad 100644 --- a/content/en/docs/concepts/extend-kubernetes/api-extension/custom-resources.md +++ b/content/en/docs/concepts/extend-kubernetes/api-extension/custom-resources.md @@ -4,7 +4,7 @@ reviewers: - enisoc - deads2k content_template: templates/concept -weight: 20 +weight: 10 --- {{% capture overview %}} From 9cc2aa225a03678b93f3873ae0189e9ff86f8ed1 Mon Sep 17 00:00:00 2001 From: nirroz93 <32959392+nirroz93@users.noreply.github.com> Date: Mon, 16 Mar 2020 06:48:35 +0200 Subject: [PATCH 067/140] correct example (#19137) if you will try this it will give an error: "found character that cannot start any token" same for all versions --- .../run-application/horizontal-pod-autoscale-walkthrough.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/en/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough.md b/content/en/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough.md index 6a9fe4de32..800abfea69 100644 --- a/content/en/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough.md +++ b/content/en/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough.md @@ -369,8 +369,8 @@ label, you can specify the following metric block to scale only on GET requests: type: Object object: metric: - name: `http_requests` - selector: `verb=GET` + name: http_requests + selector: {matchLabels: {verb: GET}} ``` This selector uses the same syntax as the full Kubernetes label selectors. The monitoring pipeline From 01e7453cdfc7639d686d998be813340ca01bb112 Mon Sep 17 00:00:00 2001 From: Chris Chinchilla Date: Mon, 16 Mar 2020 05:50:35 +0100 Subject: [PATCH 068/140] Update "what is kubernetes" subheading (#19115) * Update what-is-kubernetes.md The sub heading felt awkward as it was asking a question, but there was no question mark. As question marks in headings are not ideal in technical documentation anyway, I thought swapping the word order slightly made it sounds _slightly_ better. * Update content/en/docs/concepts/overview/what-is-kubernetes.md Co-Authored-By: Tim Bannister Co-authored-by: Tim Bannister --- content/en/docs/concepts/overview/what-is-kubernetes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/concepts/overview/what-is-kubernetes.md b/content/en/docs/concepts/overview/what-is-kubernetes.md index 34e1ba2f8f..6849083ccf 100644 --- a/content/en/docs/concepts/overview/what-is-kubernetes.md +++ b/content/en/docs/concepts/overview/what-is-kubernetes.md @@ -48,7 +48,7 @@ Containers have become popular because they provide extra benefits, such as: * Resource isolation: predictable application performance. * Resource utilization: high efficiency and density. -## Why you need Kubernetes and what can it do +## Why you need Kubernetes and what it can do {#why-you-need-kubernetes-and-what-can-it-do} Containers are a good way to bundle and run your applications. In a production environment, you need to manage the containers that run the applications and ensure that there is no downtime. For example, if a container goes down, another container needs to start. Wouldn't it be easier if this behavior was handled by a system? From e0a03c0ebdfd153e1ce581770cf8597e32e5e05e Mon Sep 17 00:00:00 2001 From: Qiming Teng Date: Mon, 16 Mar 2020 12:52:36 +0800 Subject: [PATCH 069/140] Resource name constraints (5) (#19122) xref: #17969, #19099, #18746 --- .../configuration/pod-priority-preemption.md | 3 +++ content/en/docs/concepts/containers/runtime-class.md | 3 +++ .../en/docs/concepts/policy/pod-security-policy.md | 2 ++ .../en/docs/concepts/storage/dynamic-provisioning.md | 3 +++ content/en/docs/reference/access-authn-authz/rbac.md | 12 ++++++++++-- .../docs/tasks/inject-data-application/podpreset.md | 3 +++ 6 files changed, 24 insertions(+), 2 deletions(-) diff --git a/content/en/docs/concepts/configuration/pod-priority-preemption.md b/content/en/docs/concepts/configuration/pod-priority-preemption.md index 5c2edb1a0a..77fbd93c58 100644 --- a/content/en/docs/concepts/configuration/pod-priority-preemption.md +++ b/content/en/docs/concepts/configuration/pod-priority-preemption.md @@ -117,6 +117,9 @@ priority class name to the integer value of the priority. The name is specified in the `name` field of the PriorityClass object's metadata. The value is specified in the required `value` field. The higher the value, the higher the priority. +The name of a PriorityClass object must be a valid +[DNS subdomain name](/docs/concepts/overview/working-with-objects/names#dns-subdomain-names), +and it cannot be prefixed with `system-`. A PriorityClass object can have any 32-bit integer value smaller than or equal to 1 billion. Larger numbers are reserved for critical system Pods that should diff --git a/content/en/docs/concepts/containers/runtime-class.md b/content/en/docs/concepts/containers/runtime-class.md index 00bd9fae34..9ea21fa6b0 100644 --- a/content/en/docs/concepts/containers/runtime-class.md +++ b/content/en/docs/concepts/containers/runtime-class.md @@ -82,6 +82,9 @@ metadata: handler: myconfiguration # The name of the corresponding CRI configuration ``` +The name of a RuntimeClass object must be a valid +[DNS subdomain name](/docs/concepts/overview/working-with-objects/names#dns-subdomain-names). + {{< note >}} It is recommended that RuntimeClass write operations (create/update/patch/delete) be restricted to the cluster administrator. This is typically the default. See [Authorization diff --git a/content/en/docs/concepts/policy/pod-security-policy.md b/content/en/docs/concepts/policy/pod-security-policy.md index 45b48f62ae..44c4a41e7a 100644 --- a/content/en/docs/concepts/policy/pod-security-policy.md +++ b/content/en/docs/concepts/policy/pod-security-policy.md @@ -197,6 +197,8 @@ alias kubectl-user='kubectl --as=system:serviceaccount:psp-example:fake-user -n Define the example PodSecurityPolicy object in a file. This is a policy that simply prevents the creation of privileged pods. +The name of a PodSecurityPolicy object must be a valid +[DNS subdomain name](/docs/concepts/overview/working-with-objects/names#dns-subdomain-names). {{< codenew file="policy/example-psp.yaml" >}} diff --git a/content/en/docs/concepts/storage/dynamic-provisioning.md b/content/en/docs/concepts/storage/dynamic-provisioning.md index 6bdca6b8af..77885981f7 100644 --- a/content/en/docs/concepts/storage/dynamic-provisioning.md +++ b/content/en/docs/concepts/storage/dynamic-provisioning.md @@ -46,6 +46,9 @@ To enable dynamic provisioning, a cluster administrator needs to pre-create one or more StorageClass objects for users. StorageClass objects define which provisioner should be used and what parameters should be passed to that provisioner when dynamic provisioning is invoked. +The name of a StorageClass object must be a valid +[DNS subdomain name](/docs/concepts/overview/working-with-objects/names#dns-subdomain-names). + The following manifest creates a storage class "slow" which provisions standard disk-like persistent disks. diff --git a/content/en/docs/reference/access-authn-authz/rbac.md b/content/en/docs/reference/access-authn-authz/rbac.md index 852e73fd79..3e18ae283e 100644 --- a/content/en/docs/reference/access-authn-authz/rbac.md +++ b/content/en/docs/reference/access-authn-authz/rbac.md @@ -74,6 +74,9 @@ rules: verbs: ["get", "watch", "list"] ``` +The name of a Role or a ClusterRole object must be a valid +[path segment name](/docs/concepts/overview/working-with-objects/names#path-segment-names). + ### RoleBinding and ClusterRoleBinding A role binding grants the permissions defined in a role to a user or set of users. @@ -81,6 +84,9 @@ It holds a list of subjects (users, groups, or service accounts), and a referenc Permissions can be granted within a namespace with a `RoleBinding`, or cluster-wide with a `ClusterRoleBinding`. A `RoleBinding` may reference a `Role` in the same namespace. +The name of a `RoleBinding` object must be a valid +[path segment name](/docs/concepts/overview/working-with-objects/names#path-segment-names). + The following `RoleBinding` grants the "pod-reader" role to the user "jane" within the "default" namespace. This allows "jane" to read pods in the "default" namespace. @@ -129,8 +135,10 @@ roleRef: apiGroup: rbac.authorization.k8s.io ``` -Finally, a `ClusterRoleBinding` may be used to grant permission at the cluster level and in all -namespaces. The following `ClusterRoleBinding` allows any user in the group "manager" to read +Finally, a `ClusterRoleBinding` may be used to grant permission at the cluster level and in all namespaces. + The name of a `ClusterRoleBinding` object must be a valid +[path segment name](/docs/concepts/overview/working-with-objects/names#path-segment-names). +The following `ClusterRoleBinding` allows any user in the group "manager" to read secrets in any namespace. ```yaml diff --git a/content/en/docs/tasks/inject-data-application/podpreset.md b/content/en/docs/tasks/inject-data-application/podpreset.md index 73ee4bbb42..de41c0f73a 100644 --- a/content/en/docs/tasks/inject-data-application/podpreset.md +++ b/content/en/docs/tasks/inject-data-application/podpreset.md @@ -29,6 +29,9 @@ Here is the manifest for the PodPreset: {{< codenew file="podpreset/preset.yaml" >}} +The name of a PodPreset object must be a valid +[DNS subdomain name](/docs/concepts/overview/working-with-objects/names#dns-subdomain-names). + In the manifest, you can see that the preset has an environment variable definition called `DB_PORT` and a volume mount definition called `cache-volume` which is mounted under `/cache`. The {{< glossary_tooltip text="selector" term_id="selector" >}} specifies that the preset will act upon any Pod that is labeled `role:frontend`. From a869f00509cea5e00ccbd3b770f47e060c4d1cd4 Mon Sep 17 00:00:00 2001 From: Louis Raymond Date: Mon, 16 Mar 2020 11:58:35 +0700 Subject: [PATCH 070/140] Update cheatsheet.md (#19039) Replaced 'if' ZSH kubectl autocomplete with a more compact form --- content/en/docs/reference/kubectl/cheatsheet.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/reference/kubectl/cheatsheet.md b/content/en/docs/reference/kubectl/cheatsheet.md index 6269c5c362..38d723e981 100644 --- a/content/en/docs/reference/kubectl/cheatsheet.md +++ b/content/en/docs/reference/kubectl/cheatsheet.md @@ -42,7 +42,7 @@ complete -F __start_kubectl k ```bash source <(kubectl completion zsh) # setup autocomplete in zsh into the current shell -echo "if [ $commands[kubectl] ]; then source <(kubectl completion zsh); fi" >> ~/.zshrc # add autocomplete permanently to your zsh shell +echo "[[ $commands[kubectl] ]] && source <(kubectl completion zsh)" >> ~/.zshrc # add autocomplete permanently to your zsh shell ``` ## Kubectl Context and Configuration From be7d845c410333ed1dfe348660f13fdfe67c25fc Mon Sep 17 00:00:00 2001 From: Tim Bannister Date: Mon, 16 Mar 2020 05:00:36 +0000 Subject: [PATCH 071/140] =?UTF-8?q?Revise=20=E2=80=9CUsing=20Source=20IP?= =?UTF-8?q?=E2=80=9D=20task=20(#19007)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Revise tutorial prerequisites - Set and use min-kubernetes-server-version metadata - Combine existing prerequisite sections * Switch to documentation IPv4 addresses Use documentation IPv4 addresses as per https://tools.ietf.org/html/rfc5737 * Tidy Source IP tutorial * Migrate to definition list Also reword some definitions --- .../en/docs/tutorials/services/source-ip.md | 210 +++++++++++------- 1 file changed, 125 insertions(+), 85 deletions(-) diff --git a/content/en/docs/tutorials/services/source-ip.md b/content/en/docs/tutorials/services/source-ip.md index e1b4876a24..ca3a2bb409 100644 --- a/content/en/docs/tutorials/services/source-ip.md +++ b/content/en/docs/tutorials/services/source-ip.md @@ -1,6 +1,7 @@ --- title: Using Source IP content_template: templates/tutorial +min-kubernetes-server-version: v1.5 --- {{% capture overview %}} @@ -14,26 +15,38 @@ of Services, and how you can toggle this behavior according to your needs. {{% capture prerequisites %}} -{{< include "task-tutorial-prereqs.md" >}} {{< version-check >}} - -## Terminology +### Terminology This document makes use of the following terms: -* [NAT](https://en.wikipedia.org/wiki/Network_address_translation): network address translation -* [Source NAT](https://en.wikipedia.org/wiki/Network_address_translation#SNAT): replacing the source IP on a packet, usually with a node's IP -* [Destination NAT](https://en.wikipedia.org/wiki/Network_address_translation#DNAT): replacing the destination IP on a packet, usually with a pod IP -* [VIP](/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies): a virtual IP, such as the one assigned to every Kubernetes Service -* [Kube-proxy](/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies): a network daemon that orchestrates Service VIP management on every node +{{< comment >}} +If localizing this section, link to the equivalent Wikipedia pages for +the target localization. +{{< /comment >}} +[NAT](https://en.wikipedia.org/wiki/Network_address_translation) +: network address translation -## Prerequisites +[Source NAT](https://en.wikipedia.org/wiki/Network_address_translation#SNAT) +: replacing the source IP on a packet; in this page, that usually means replacing with the IP address of a node. -You must have a working Kubernetes 1.5 cluster to run the examples in this -document. The examples use a small nginx webserver that echoes back the source +[Destination NAT](https://en.wikipedia.org/wiki/Network_address_translation#DNAT) +: replacing the destination IP on a packet; in this page, that usually means replacing with the IP address of a {{< glossary_tooltip term_id="pod" >}} + +[VIP](/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies) +: a virtual IP address, such as the one assigned to every {{< glossary_tooltip text="Service" term_id="service" >}} in Kubernetes + +[kube-proxy](/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies) +: a network daemon that orchestrates Service VIP management on every node + +### Prerequisites + +{{< include "task-tutorial-prereqs.md" >}} + +The examples use a small nginx webserver that echoes back the source IP of requests it receives through an HTTP header. You can create it as follows: -```console +```shell kubectl create deployment source-ip-app --image=k8s.gcr.io/echoserver:1.4 ``` The output is: @@ -54,12 +67,13 @@ deployment.apps/source-ip-app created {{% capture lessoncontent %}} -## Source IP for Services with Type=ClusterIP +## Source IP for Services with `Type=ClusterIP` Packets sent to ClusterIP from within the cluster are never source NAT'd if -you're running kube-proxy in [iptables mode](/docs/concepts/services-networking/service/#proxy-mode-iptables), -which is the default since Kubernetes 1.2. Kube-proxy exposes its mode through -a `proxyMode` endpoint: +you're running kube-proxy in +[iptables mode](/docs/concepts/services-networking/service/#proxy-mode-iptables), +(the default). You can query the kube-proxy mode by fetching +`http://localhost:10249/proxyMode` on the node where kube-proxy is running. ```console kubectl get nodes @@ -71,9 +85,11 @@ kubernetes-node-6jst Ready 2h v1.13.0 kubernetes-node-cx31 Ready 2h v1.13.0 kubernetes-node-jj1t Ready 2h v1.13.0 ``` -Get the proxy mode on one of the node -```console -kubernetes-node-6jst $ curl localhost:10249/proxyMode + +Get the proxy mode on one of the nodes (kube-proxy listens on port 10249): +```shell +# Run this in a shell on the node you want to query. +curl http://localhost:10249/proxyMode ``` The output is: ``` @@ -82,14 +98,14 @@ iptables You can test source IP preservation by creating a Service over the source IP app: -```console +```shell kubectl expose deployment source-ip-app --name=clusterip --port=80 --target-port=8080 ``` The output is: ``` service/clusterip exposed ``` -```console +```shell kubectl get svc clusterip ``` The output is similar to: @@ -100,7 +116,7 @@ clusterip ClusterIP 10.0.170.92 80/TCP 51s And hitting the `ClusterIP` from a pod in the same cluster: -```console +```shell kubectl run busybox -it --image=busybox --restart=Never --rm ``` The output is similar to this: @@ -108,7 +124,14 @@ The output is similar to this: Waiting for pod default/busybox to be running, status is Pending, pod ready: false If you don't see a command prompt, try pressing enter. -# ip addr +``` +You can then run a command inside that Pod: + +```shell +# Run this inside the terminal from "kubectl run" +ip addr +``` +``` 1: lo: mtu 65536 qdisc noqueue link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo @@ -121,21 +144,28 @@ If you don't see a command prompt, try pressing enter. valid_lft forever preferred_lft forever inet6 fe80::188a:84ff:feb0:26a5/64 scope link valid_lft forever preferred_lft forever +``` -# wget -qO - 10.0.170.92 +…then use `wget` to query the local webserver +```shell +# Replace 10.0.170.92 with the Pod's IPv4 address +wget -qO - 10.0.170.92 +``` +``` CLIENT VALUES: client_address=10.244.3.8 command=GET ... ``` -The client_address is always the client pod's IP address, whether the client pod and server pod are in the same node or in different nodes. +The `client_address` is always the client pod's IP address, whether the client pod and server pod are in the same node or in different nodes. -## Source IP for Services with Type=NodePort +## Source IP for Services with `Type=NodePort` -As of Kubernetes 1.5, packets sent to Services with [Type=NodePort](/docs/concepts/services-networking/service/#nodeport) +Packets sent to Services with +[`Type=NodePort`](/docs/concepts/services-networking/service/#nodeport) are source NAT'd by default. You can test this by creating a `NodePort` Service: -```console +```shell kubectl expose deployment source-ip-app --name=nodeport --port=80 --target-port=8080 --type=NodePort ``` The output is: @@ -143,17 +173,17 @@ The output is: service/nodeport exposed ``` -```console +```shell NODEPORT=$(kubectl get -o jsonpath="{.spec.ports[0].nodePort}" services nodeport) NODES=$(kubectl get nodes -o jsonpath='{ $.items[*].status.addresses[?(@.type=="ExternalIP")].address }') ``` -If you're running on a cloudprovider, you may need to open up a firewall-rule +If you're running on a cloud provider, you may need to open up a firewall-rule for the `nodes:nodeport` reported above. Now you can try reaching the Service from outside the cluster through the node port allocated above. -```console +```shell for node in $NODES; do curl -s $node:$NODEPORT | grep -i client_address; done ``` The output is similar to: @@ -187,18 +217,19 @@ Visually: ``` -To avoid this, Kubernetes has a feature to preserve the client source IP -[(check here for feature availability)](/docs/tasks/access-application-cluster/create-external-load-balancer/#preserving-the-client-source-ip). -Setting `service.spec.externalTrafficPolicy` to the value `Local` will only -proxy requests to local endpoints, never forwarding traffic to other nodes -and thereby preserving the original source IP address. If there are no -local endpoints, packets sent to the node are dropped, so you can rely -on the correct source-ip in any packet processing rules you might apply a -packet that make it through to the endpoint. +To avoid this, Kubernetes has a feature to +[preserve the client source IP](/docs/tasks/access-application-cluster/create-external-load-balancer/#preserving-the-client-source-ip). +If you set `service.spec.externalTrafficPolicy` to the value `Local`, +kube-proxy only proxies proxy requests to local endpoints, and does not +forward traffic to other nodes. This approach preserves the original +source IP address. If there are no local endpoints, packets sent to the +node are dropped, so you can rely on the correct source-ip in any packet +processing rules you might apply a packet that make it through to the +endpoint. Set the `service.spec.externalTrafficPolicy` field as follows: -```console +```shell kubectl patch svc nodeport -p '{"spec":{"externalTrafficPolicy":"Local"}}' ``` The output is: @@ -208,12 +239,12 @@ service/nodeport patched Now, re-run the test: -```console +```shell for node in $NODES; do curl --connect-timeout 1 -s $node:$NODEPORT | grep -i client_address; done ``` -The output is: +The output is similar to: ``` -client_address=104.132.1.79 +client_address=198.51.100.79 ``` Note that you only got one reply, with the *right* client IP, from the one node on which the endpoint pod @@ -242,18 +273,19 @@ Visually: -## Source IP for Services with Type=LoadBalancer +## Source IP for Services with `Type=LoadBalancer` -As of Kubernetes 1.5, packets sent to Services with [Type=LoadBalancer](/docs/concepts/services-networking/service/#loadbalancer) are -source NAT'd by default, because all schedulable Kubernetes nodes in the -`Ready` state are eligible for loadbalanced traffic. So if packets arrive +Packets sent to Services with +[`Type=LoadBalancer`](/docs/concepts/services-networking/service/#loadbalancer) +are source NAT'd by default, because all schedulable Kubernetes nodes in the +`Ready` state are eligible for load-balanced traffic. So if packets arrive at a node without an endpoint, the system proxies it to a node *with* an endpoint, replacing the source IP on the packet with the IP of the node (as described in the previous section). -You can test this by exposing the source-ip-app through a loadbalancer +You can test this by exposing the source-ip-app through a load balancer: -```console +```shell kubectl expose deployment source-ip-app --name=loadbalancer --port=80 --target-port=8080 --type=LoadBalancer ``` The output is: @@ -261,18 +293,20 @@ The output is: service/loadbalancer exposed ``` -Print IPs of the Service: +Print out the IP addresses of the Service: ```console kubectl get svc loadbalancer ``` The output is similar to this: ``` NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE -loadbalancer LoadBalancer 10.0.65.118 104.198.149.140 80/TCP 5m +loadbalancer LoadBalancer 10.0.65.118 203.0.113.140 80/TCP 5m ``` -```console -curl 104.198.149.140 +Next, send a request to this Service's external-ip: + +```shell +curl 203.0.113.140 ``` The output is similar to this: ``` @@ -302,25 +336,25 @@ health check ---> node 1 node 2 <--- health check You can test this by setting the annotation: -```console +```shell kubectl patch svc loadbalancer -p '{"spec":{"externalTrafficPolicy":"Local"}}' ``` You should immediately see the `service.spec.healthCheckNodePort` field allocated by Kubernetes: -```console +```shell kubectl get svc loadbalancer -o yaml | grep -i healthCheckNodePort ``` The output is similar to this: -``` +```yaml healthCheckNodePort: 32122 ``` The `service.spec.healthCheckNodePort` field points to a port on every node serving the health check at `/healthz`. You can test this: -```console +```shell kubectl get pod -o wide -l run=source-ip-app ``` The output is similar to this: @@ -328,43 +362,48 @@ The output is similar to this: NAME READY STATUS RESTARTS AGE IP NODE source-ip-app-826191075-qehz4 1/1 Running 0 20h 10.180.1.136 kubernetes-node-6jst ``` -Curl the `/healthz` endpoint on different nodes. -```console -kubernetes-node-6jst $ curl localhost:32122/healthz + +Use `curl` to fetch the `/healthz` endpoint on various nodes: +```shell +# Run this locally on a node you choose +curl localhost:32122/healthz ``` -The output is similar to this: ``` 1 Service Endpoints found ``` -```console -kubernetes-node-jj1t $ curl localhost:32122/healthz + +On a different node you might get a different result: +```shell +# Run this locally on a node you choose +curl localhost:32122/healthz ``` -The output is similar to this: ``` No Service Endpoints Found ``` -A service controller running on the master is responsible for allocating the cloud -loadbalancer, and when it does so, it also allocates HTTP health checks -pointing to this port/path on each node. Wait about 10 seconds for the 2 nodes -without endpoints to fail health checks, then curl the lb ip: +A controller running on the +{{< glossary_tooltip text="control plane" term_id="control-plane" >}} is +responsible for allocating the cloud load balancer. The same controller also +allocates HTTP health checks pointing to this port/path on each node. Wait +about 10 seconds for the 2 nodes without endpoints to fail health checks, +then use `curl` to query the IPv4 address of the load balancer: -```console -curl 104.198.149.140 +```shell +curl 203.0.113.140 ``` The output is similar to this: ``` CLIENT VALUES: -client_address=104.132.1.79 +client_address=198.51.100.79 ... ``` -__Cross platform support__ +## Cross-platform support -As of Kubernetes 1.5, support for source IP preservation through Services -with Type=LoadBalancer is only implemented in a subset of cloudproviders -(GCP and Azure). The cloudprovider you're running on might fulfill the -request for a loadbalancer in a few different ways: +Only some cloud providers offer support for source IP preservation through +Services with `Type=LoadBalancer`. +The cloud provider you're running on might fulfill the request for a loadbalancer +in a few different ways: 1. With a proxy that terminates the client connection and opens a new connection to your nodes/endpoints. In such cases the source IP will always be that of the @@ -374,12 +413,14 @@ cloud LB, not that of the client. loadbalancer VIP end up at the node with the source IP of the client, not an intermediate proxy. -Loadbalancers in the first category must use an agreed upon +Load balancers in the first category must use an agreed upon protocol between the loadbalancer and backend to communicate the true client IP -such as the HTTP [X-FORWARDED-FOR](https://en.wikipedia.org/wiki/X-Forwarded-For) -header, or the [proxy protocol](http://www.haproxy.org/download/1.5/doc/proxy-protocol.txt). -Loadbalancers in the second category can leverage the feature described above -by simply creating an HTTP health check pointing at the port stored in +such as the HTTP [Forwarded](https://tools.ietf.org/html/rfc7239#section-5.2) +or [X-FORWARDED-FOR](https://en.wikipedia.org/wiki/X-Forwarded-For) +headers, or the +[proxy protocol](http://www.haproxy.org/download/1.5/doc/proxy-protocol.txt). +Load balancers in the second category can leverage the feature described above +by creating an HTTP health check pointing at the port stored in the `service.spec.healthCheckNodePort` field on the Service. {{% /capture %}} @@ -388,13 +429,13 @@ the `service.spec.healthCheckNodePort` field on the Service. Delete the Services: -```console +```shell kubectl delete svc -l run=source-ip-app ``` Delete the Deployment, ReplicaSet and Pod: -```console +```shell kubectl delete deployment source-ip-app ``` @@ -402,7 +443,6 @@ kubectl delete deployment source-ip-app {{% capture whatsnext %}} * Learn more about [connecting applications via services](/docs/concepts/services-networking/connect-applications-service/) -* Learn more about [loadbalancing](/docs/user-guide/load-balancer) +* Read how to [Create an External Load Balancer](https://kubernetes.io/docs/tasks/access-application-cluster/create-external-load-balancer/) {{% /capture %}} - From 9e289296c21919c13f19caec4220c6bab2b28c46 Mon Sep 17 00:00:00 2001 From: Daniel Helfand Date: Mon, 16 Mar 2020 01:02:35 -0400 Subject: [PATCH 072/140] fix typos and styling for limitrange doc (#18961) --- .../en/docs/concepts/policy/limit-range.md | 106 +++++++++--------- 1 file changed, 54 insertions(+), 52 deletions(-) diff --git a/content/en/docs/concepts/policy/limit-range.md b/content/en/docs/concepts/policy/limit-range.md index 3493983bde..dfc941681b 100644 --- a/content/en/docs/concepts/policy/limit-range.md +++ b/content/en/docs/concepts/policy/limit-range.md @@ -9,55 +9,58 @@ weight: 10 {{% capture overview %}} By default, containers run with unbounded [compute resources](/docs/user-guide/compute-resources) on a Kubernetes cluster. -With Resource quotas, cluster administrators can restrict the resource consumption and creation on a namespace basis. -Within a namespace, a Pod or Container can consume as much CPU and memory as defined by the namespace's resource quota. There is a concern that one Pod or Container could monopolize all of the resources. Limit Range is a policy to constrain resource by Pod or Container in a namespace. +With resource quotas, cluster administrators can restrict resource consumption and creation on a namespace basis. +Within a namespace, a Pod or Container can consume as much CPU and memory as defined by the namespace's resource quota. There is a concern that one Pod or Container could monopolize all available resources. A LimitRange is a policy to constrain resource allocations (to Pods or Containers) in a namespace. {{% /capture %}} {{% capture body %}} -A limit range, defined by a `LimitRange` object, provides constraints that can: +A _LimitRange_ provides constraints that can: - Enforce minimum and maximum compute resources usage per Pod or Container in a namespace. - Enforce minimum and maximum storage request per PersistentVolumeClaim in a namespace. - Enforce a ratio between request and limit for a resource in a namespace. - Set default request/limit for compute resources in a namespace and automatically inject them to Containers at runtime. -## Enabling Limit Range +## Enabling LimitRange -Limit Range support is enabled by default for many Kubernetes distributions. It is +LimitRange support is enabled by default for many Kubernetes distributions. It is enabled when the apiserver `--enable-admission-plugins=` flag has `LimitRanger` admission controller as one of its arguments. -A limit range is enforced in a particular namespace when there is a -`LimitRange` object in that namespace. +A LimitRange is enforced in a particular namespace when there is a +LimitRange object in that namespace. + +The name of a LimitRange object must be a valid +[DNS subdomain name](/docs/concepts/overview/working-with-objects/names#dns-subdomain-names). ### Overview of Limit Range - The administrator creates one `LimitRange` in one namespace. - Users create resources like Pods, Containers, and PersistentVolumeClaims in the namespace. -- The `LimitRanger` admission controller enforces defaults limits for all Pods and Container that do not set compute resource requirements and tracks usage to ensure it does not exceed resource minimum , maximum and ratio defined in any `LimitRange` present in the namespace. -- If creating or updating a resource (Pod, Container, PersistentVolumeClaim) violates a limit range constraint, the request to the API server will fail with HTTP status code `403 FORBIDDEN` and a message explaining the constraint that would have been violated. -- If limit range is activated in a namespace for compute resources like `cpu` and `memory`, users must specify - requests or limits for those values; otherwise, the system may reject pod creation. -- LimitRange validations occurs only at Pod Admission stage, not on Running pods. +- The `LimitRanger` admission controller enforces defaults and limits for all Pods and Containers that do not set compute resource requirements and tracks usage to ensure it does not exceed resource minimum, maximum and ratio defined in any LimitRange present in the namespace. +- If creating or updating a resource (Pod, Container, PersistentVolumeClaim) that violates a LimitRange constraint, the request to the API server will fail with an HTTP status code `403 FORBIDDEN` and a message explaining the constraint that have been violated. +- If a LimitRange is activated in a namespace for compute resources like `cpu` and `memory`, users must specify + requests or limits for those values. Otherwise, the system may reject Pod creation. +- LimitRange validations occurs only at Pod Admission stage, not on Running Pods. Examples of policies that could be created using limit range are: -- In a 2 node cluster with a capacity of 8 GiB RAM, and 16 cores, constrain Pods in a namespace to request 100m and not exceeds 500m for CPU , request 200Mi and not exceed 600Mi -- Define default CPU limits and request to 150m and Memory default request to 300Mi for containers started with no cpu and memory requests in their spec. +- In a 2 node cluster with a capacity of 8 GiB RAM and 16 cores, constrain Pods in a namespace to request 100m of CPU with a max limit of 500m for CPU and request 200Mi for Memory with a max limit of 600Mi for Memory. +- Define default CPU limit and request to 150m and memory default request to 300Mi for Containers started with no cpu and memory requests in their specs. In the case where the total limits of the namespace is less than the sum of the limits of the Pods/Containers, -there may be contention for resources; The Containers or Pods will not be created. +there may be contention for resources. In this case, the Containers or Pods will not be created. -Neither contention nor changes to limitrange will affect already created resources. +Neither contention nor changes to a LimitRange will affect already created resources. ## Limiting Container compute resources The following section discusses the creation of a LimitRange acting at Container Level. -A Pod with 04 containers is first created; each container within the Pod has a specific `spec.resource` configuration -each container within the pod is handled differently by the LimitRanger admission controller. +A Pod with 04 Containers is first created. Each Container within the Pod has a specific `spec.resource` configuration. +Each Container within the Pod is handled differently by the `LimitRanger` admission controller. Create a namespace `limitrange-demo` using the following kubectl command: @@ -74,9 +77,7 @@ kubectl config set-context --current --namespace=limitrange-demo Here is the configuration file for a LimitRange object: {{< codenew file="admin/resource/limit-mem-cpu-container.yaml" >}} -This object defines minimum and maximum Memory/CPU limits, default cpu/Memory requests and default limits for CPU/Memory resources to be apply to containers. -The name of a LimitRange object must be a valid -[DNS subdomain name](/docs/concepts/overview/working-with-objects/names#dns-subdomain-names). +This object defines minimum and maximum CPU/Memory limits, default CPU/Memory requests, and default limits for CPU/Memory resources to be apply to containers. Create the `limit-mem-cpu-per-container` LimitRange in the `limitrange-demo` namespace with the following kubectl command: @@ -95,7 +96,7 @@ Container cpu 100m 800m 110m 700m - Container memory 99Mi 1Gi 111Mi 900Mi - ``` -Here is the configuration file for a Pod with 04 containers to demonstrate LimitRange features : +Here is the configuration file for a Pod with 04 Containers to demonstrate LimitRange features: {{< codenew file="admin/resource/limit-range-pod-1.yaml" >}} Create the `busybox1` Pod: @@ -126,9 +127,9 @@ kubectl get po/busybox1 -n limitrange-demo -o json | jq ".spec.containers[0].res ``` - The `busybox-cnt01` Container inside `busybox` Pod defined `requests.cpu=100m` and `requests.memory=100Mi`. -- `100m <= 500m <= 800m` , The container cpu limit (500m) falls inside the authorized CPU limit range. -- `99Mi <= 200Mi <= 1Gi` , The container memory limit (200Mi) falls inside the authorized Memory limit range. -- No request/limits ratio validation for CPU/Memory , thus the container is valid and created. +- `100m <= 500m <= 800m` , The Container cpu limit (500m) falls inside the authorized CPU LimitRange. +- `99Mi <= 200Mi <= 1Gi` , The Container memory limit (200Mi) falls inside the authorized Memory LimitRange. +- No request/limits ratio validation for CPU/Memory, so the Container is valid and created. ### Container spec with a valid CPU/Memory requests but no limits @@ -152,14 +153,15 @@ kubectl get po/busybox1 -n limitrange-demo -o json | jq ".spec.containers[1].res } ``` - The `busybox-cnt02` Container inside `busybox1` Pod defined `requests.cpu=100m` and `requests.memory=100Mi` but not limits for cpu and memory. -- The container do not have a limits section, the default limits defined in the limit-mem-cpu-per-container LimitRange object are injected to this container `limits.cpu=700mi` and `limits.memory=900Mi`. -- `100m <= 700m <= 800m` , The container cpu limit (700m) falls inside the authorized CPU limit range. -- `99Mi <= 900Mi <= 1Gi` , The container memory limit (900Mi) falls inside the authorized Memory limit range. -- No request/limits ratio set , thus the container is valid and created. +- The Container does not have a limits section. The default limits defined in the `limit-mem-cpu-per-container` LimitRange object are injected in to this Container: `limits.cpu=700mi` and `limits.memory=900Mi`. +- `100m <= 700m <= 800m` , The Container cpu limit (700m) falls inside the authorized CPU limit range. +- `99Mi <= 900Mi <= 1Gi` , The Container memory limit (900Mi) falls inside the authorized Memory limit range. +- No request/limits ratio set, so the Container is valid and created. -### Container spec with a valid CPU/Memory limits but no requests -View the `busybox-cnt03` resource configuration +### Container spec with a valid CPU/Memory limits but no requests + +View the `busybox-cnt03` resource configuration: ```shell kubectl get po/busybox1 -n limitrange-demo -o json | jq ".spec.containers[2].resources" @@ -178,10 +180,10 @@ kubectl get po/busybox1 -n limitrange-demo -o json | jq ".spec.containers[2].res ``` - The `busybox-cnt03` Container inside `busybox1` Pod defined `limits.cpu=500m` and `limits.memory=200Mi` but no `requests` for cpu and memory. -- The container do not define a request section, the defaultRequest defined in the limit-mem-cpu-per-container LimitRange is not used to fill its limits section but the limits defined by the container are set as requests `limits.cpu=500m` and `limits.memory=200Mi`. -- `100m <= 500m <= 800m` , The container cpu limit (500m) falls inside the authorized CPU limit range. -- `99Mi <= 200Mi <= 1Gi` , The container memory limit (200Mi) falls inside the authorized Memory limit range. -- No request/limits ratio set , thus the container is valid and created. +- The Container does not define a request section. The default request defined in the limit-mem-cpu-per-container LimitRange is not used to fill its limits section, but the limits defined by the Container are set as requests `limits.cpu=500m` and `limits.memory=200Mi`. +- `100m <= 500m <= 800m` , The Container cpu limit (500m) falls inside the authorized CPU limit range. +- `99Mi <= 200Mi <= 1Gi` , The Container memory limit (200Mi) falls inside the authorized Memory limit range. +- No request/limits ratio set, so the Container is valid and created. ### Container spec with no CPU/Memory requests/limits @@ -205,27 +207,27 @@ kubectl get po/busybox1 -n limitrange-demo -o json | jq ".spec.containers[3].res ``` - The `busybox-cnt04` Container inside `busybox1` define neither `limits` nor `requests`. -- The container do not define a limit section, the default limit defined in the limit-mem-cpu-per-container LimitRange is used to fill its request +- The Container do not define a limit section, the default limit defined in the limit-mem-cpu-per-container LimitRange is used to fill its request `limits.cpu=700m and` `limits.memory=900Mi` . -- The container do not define a request section, the defaultRequest defined in the limit-mem-cpu-per-container LimitRange is used to fill its request section requests.cpu=110m and requests.memory=111Mi -- `100m <= 700m <= 800m` , The container cpu limit (700m) falls inside the authorized CPU limit range. -- `99Mi <= 900Mi <= 1Gi` , The container memory limit (900Mi) falls inside the authorized Memory limitrange . -- No request/limits ratio set , thus the container is valid and created. +- The Container do not define a request section, the defaultRequest defined in the `limit-mem-cpu-per-container` LimitRange is used to fill its request section requests.cpu=110m and requests.memory=111Mi +- `100m <= 700m <= 800m` , The Container cpu limit (700m) falls inside the authorized CPU limit range. +- `99Mi <= 900Mi <= 1Gi` , The Container memory limit (900Mi) falls inside the authorized Memory limit range . +- No request/limits ratio set, so the Container is valid and created. -All containers defined in the `busybox` Pod passed LimitRange validations, this the Pod is valid and create in the namespace. +All Containers defined in the `busybox` Pod passed LimitRange validations, so this the Pod is valid and created in the namespace. ## Limiting Pod compute resources -The following section discusses how to constrain resources at Pod level. +The following section discusses how to constrain resources at the Pod level. {{< codenew file="admin/resource/limit-mem-cpu-pod.yaml" >}} -Without having to delete `busybox1` Pod, create the `limit-mem-cpu-pod` LimitRange in the `limitrange-demo` namespace: +Without having to delete the `busybox1` Pod, create the `limit-mem-cpu-pod` LimitRange in the `limitrange-demo` namespace: ```shell kubectl apply -f https://k8s.io/examples/admin/resource/limit-mem-cpu-pod.yaml -n limitrange-demo ``` -The limitrange is created and limits CPU to 2 Core and Memory to 2Gi per Pod: +The LimitRange is created and limits CPU to 2 Core and Memory to 2Gi per Pod: ```shell limitrange/limit-mem-cpu-per-pod created @@ -254,7 +256,7 @@ Now create the `busybox2` Pod: kubectl apply -f https://k8s.io/examples/admin/resource/limit-range-pod-2.yaml -n limitrange-demo ``` -The `busybox2` Pod definition is identical to `busybox1` but an error is reported since Pod's resources are now limited: +The `busybox2` Pod definition is identical to `busybox1`, but an error is reported since the Pod's resources are now limited: ```shell Error from server (Forbidden): error when creating "limit-range-pod-2.yaml": pods "busybox2" is forbidden: [maximum cpu usage per Pod is 2, but limit is 2400m., maximum memory usage per Pod is 2Gi, but limit is 2306867200.] @@ -268,12 +270,12 @@ kubectl get po/busybox1 -n limitrange-demo -o json | jq ".spec.containers[].reso "900Mi" ``` -`busybox2` Pod will not be admitted on the cluster since the total memory limit of its container is greater than the limit defined in the LimitRange. +`busybox2` Pod will not be admitted on the cluster since the total memory limit of its Container is greater than the limit defined in the LimitRange. `busybox1` will not be evicted since it was created and admitted on the cluster before the LimitRange creation. ## Limiting Storage resources -You can enforce minimum and maximum size of [storage resources](/docs/concepts/storage/persistent-volumes/) that can be requested by each PersistentVolumeClaim in a namespace using a LimitRange: +You can enforce minimum and maximum size of [storage resources](/docs/concepts/storage/persistent-volumes/) that can be requested by each PersistentVolumeClaim in a namespace using a LimitRange: {{< codenew file="admin/resource/storagelimits.yaml" >}} @@ -329,9 +331,9 @@ Error from server (Forbidden): error when creating "pvc-limit-greater.yaml": per ## Limits/Requests Ratio -If `LimitRangeItem.maxLimitRequestRatio` is specified in the `LimitRangeSpec`, the named resource must have a request and limit that are both non-zero where limit divided by request is less than or equal to the enumerated value +If `LimitRangeItem.maxLimitRequestRatio` is specified in the `LimitRangeSpec`, the named resource must have a request and limit that are both non-zero where limit divided by request is less than or equal to the enumerated value. -The following `LimitRange` enforces memory limit to be at most twice the amount of the memory request for any pod in the namespace. +The following LimitRange enforces memory limit to be at most twice the amount of the memory request for any Pod in the namespace: {{< codenew file="admin/resource/limit-memory-ratio-pod.yaml" >}} @@ -353,7 +355,7 @@ Type Resource Min Max Default Request Default Limit Max Limit/Reques Pod memory - - - - 2 ``` -Let's create a pod with `requests.memory=100Mi` and `limits.memory=300Mi`: +Create a pod with `requests.memory=100Mi` and `limits.memory=300Mi`: {{< codenew file="admin/resource/limit-range-pod-3.yaml" >}} @@ -361,9 +363,9 @@ Let's create a pod with `requests.memory=100Mi` and `limits.memory=300Mi`: kubectl apply -f https://k8s.io/examples/admin/resource/limit-range-pod-3.yaml ``` -The pod creation failed as the ratio here (`3`) is greater than the enforced limit (`2`) in `limit-memory-ratio-pod` LimitRange +The pod creation failed as the ratio here (`3`) is greater than the enforced limit (`2`) in `limit-memory-ratio-pod` LimitRange: -```shell +``` Error from server (Forbidden): error when creating "limit-range-pod-3.yaml": pods "busybox3" is forbidden: memory max limit to request ratio per Pod is 2, but provided ratio is 3.000000. ``` From 1d08290c55cff8236b44e01450cfe19afbd7df5f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Manuel=20R=C3=BCger?= Date: Mon, 16 Mar 2020 06:04:35 +0100 Subject: [PATCH 073/140] AdmissionControllers: Rewrite PodTolerationRestriction (#18849) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Personally the explanation was quite difficult to understand without looking into the code. I hope I managed to simplify it a bit and added an example how to add annotations to a namespace so they can be consumed by the PodTolerationRestriction admission controller. Signed-off-by: Manuel Rüger --- .../admission-controllers.md | 35 ++++++++++++------- 1 file changed, 22 insertions(+), 13 deletions(-) diff --git a/content/en/docs/reference/access-authn-authz/admission-controllers.md b/content/en/docs/reference/access-authn-authz/admission-controllers.md index 3d85194ed7..7160c25f45 100644 --- a/content/en/docs/reference/access-authn-authz/admission-controllers.md +++ b/content/en/docs/reference/access-authn-authz/admission-controllers.md @@ -645,21 +645,30 @@ for more information. ### PodTolerationRestriction {#podtolerationrestriction} -This admission controller first verifies any conflict between a pod's tolerations and its -namespace's tolerations, and rejects the pod request if there is a conflict. -It then merges the namespace's tolerations into the pod's tolerations. -The resulting tolerations are checked against the namespace's whitelist of -tolerations. If the check succeeds, the pod request is admitted otherwise -rejected. +The PodTolerationRestriction admission controller verifies any conflict between tolerations of a pod and the tolerations of its namespace. +It rejects the pod request if there is a conflict. +It then merges the tolerations annotated on the namespace into the tolerations of the pod. +The resulting tolerations are checked against a whitelist of tolerations annotated to the namespace. +If the check succeeds, the pod request is admitted otherwise it is rejected. -If the pod's namespace does not have any associated default or whitelist of -tolerations, then the cluster-level default or whitelist of tolerations are used -instead if specified. +If the namespace of the pod does not have any associated default tolerations or a whitelist of +tolerations annotated, the cluster-level default tolerations or cluster-level whitelist of tolerations are used +instead if they are specified. -Tolerations to a namespace are assigned via the -`scheduler.alpha.kubernetes.io/defaultTolerations` and -`scheduler.alpha.kubernetes.io/tolerationsWhitelist` -annotation keys. +Tolerations to a namespace are assigned via the `scheduler.alpha.kubernetes.io/defaultTolerations` annotation key. +The whitelist can be added via the `scheduler.alpha.kubernetes.io/tolerationsWhitelist` annotation key. + +Example for namespace annotations: + +```yaml +apiVersion: v1 +kind: Namespace +metadata: + name: apps-that-need-nodes-exclusively + annotations: + scheduler.alpha.kubernetes.io/defaultTolerations: '{"operator": "Exists", "effect": "NoSchedule", "key": "dedicated-node"}' + scheduler.alpha.kubernetes.io/tolerationsWhitelist: '{"operator": "Exists", "effect": "NoSchedule", "key": "dedicated-node"}' +``` ### Priority {#priority} From 08e21a1e90364e00cc9b974cf4d2839120c235a5 Mon Sep 17 00:00:00 2001 From: makocchi Date: Mon, 16 Mar 2020 14:06:36 +0900 Subject: [PATCH 074/140] Clean up extensions/v1beta1 in docs [reference/using-api/api-overview.md] (#18840) * Clean up extensions/v1beta1 in docs * Mention about resources in the extensions/v1beta1 api group --- .../en/docs/reference/using-api/api-overview.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/content/en/docs/reference/using-api/api-overview.md b/content/en/docs/reference/using-api/api-overview.md index a004c86353..3820085e6b 100644 --- a/content/en/docs/reference/using-api/api-overview.md +++ b/content/en/docs/reference/using-api/api-overview.md @@ -91,12 +91,14 @@ The two paths that support extending the API with [custom resources](/docs/conce - [aggregator](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/api-machinery/aggregated-api-servers.md) for a full set of Kubernetes API semantics to implement their own apiserver. -## Enabling API groups +## Enabling or disabling API groups Certain resources and API groups are enabled by default. You can enable or disable them by setting `--runtime-config` on the apiserver. `--runtime-config` accepts comma separated values. For example: + - to disable batch/v1, set `--runtime-config=batch/v1=false` - to enable batch/v2alpha1, set `--runtime-config=batch/v2alpha1` + The flag accepts comma separated set of key=value pairs describing runtime configuration of the apiserver. {{< note >}} @@ -104,12 +106,10 @@ When you enable or disable groups or resources, you need to restart the apiserve to pick up the `--runtime-config` changes. {{< /note >}} -## Enabling resources in the groups - -DaemonSets, Deployments, HorizontalPodAutoscalers, Ingress, Jobs and ReplicaSets are enabled by default. -You can enable other extensions resources by setting `--runtime-config` on -apiserver. `--runtime-config` accepts comma separated values. For example, to disable deployments and jobs, set -`--runtime-config=extensions/v1beta1/deployments=false,extensions/v1beta1/jobs=false` -{{% /capture %}} +## Enabling specific resources in the extensions/v1beta1 group +DaemonSets, Deployments, StatefulSet, NetworkPolicies, PodSecurityPolicies and ReplicaSets in the `extensions/v1beta1` API group are disabled by default. +For example: to enable deployments and daemonsets, set +`--runtime-config=extensions/v1beta1/deployments=true,extensions/v1beta1/daemonsets=true`. +{{< note >}}Individual resource enablement/disablement is only supported in the `extensions/v1beta1` API group for legacy reasons.{{< /note >}} From 83637c1ed2fe8f1ff46abdc8c97e28a1696b62e3 Mon Sep 17 00:00:00 2001 From: Tim Bannister Date: Mon, 16 Mar 2020 05:08:35 +0000 Subject: [PATCH 075/140] Correct meaning of scheduling affinity benefits (#18764) Commit f5bfa04aa44f02cf987a64aafc7155089152c440 incorrectly reworded the benefits of using affinity / anti-affinity for Pod scheduling. Revise this with new wording. --- content/en/docs/concepts/configuration/assign-pod-node.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/content/en/docs/concepts/configuration/assign-pod-node.md b/content/en/docs/concepts/configuration/assign-pod-node.md index 19f33c47e8..d1dc2bfbf2 100644 --- a/content/en/docs/concepts/configuration/assign-pod-node.md +++ b/content/en/docs/concepts/configuration/assign-pod-node.md @@ -111,9 +111,10 @@ For example, `example.com.node-restriction.kubernetes.io/fips=true` or `example. `nodeSelector` provides a very simple way to constrain pods to nodes with particular labels. The affinity/anti-affinity feature, greatly expands the types of constraints you can express. The key enhancements are -1. the language is more expressive (not just "AND or exact match") +1. The affinity/anti-affinity language is more expressive. The language offers more matching rules + besides exact matches created with a logical AND operation; 2. you can indicate that the rule is "soft"/"preference" rather than a hard requirement, so if the scheduler - can't satisfy it, the pod will still be scheduled + can't satisfy it, the pod will still be scheduled; 3. you can constrain against labels on other pods running on the node (or other topological domain), rather than against labels on the node itself, which allows rules about which pods can and cannot be co-located From d9a74ae5fcc7b5d6f86872ccd27bc12b4ab28433 Mon Sep 17 00:00:00 2001 From: Tim Bannister Date: Mon, 16 Mar 2020 05:10:35 +0000 Subject: [PATCH 076/140] Tidy port forward task (#18765) * Reword to follow style guidelines * Highlight commands as shell code * Specify minimum version This metadata alters the behavior of the {{< version-check >}} shortcode. * Downgrade warning to note I don't see any adverse consequences if the reader is unaware that port forwarding for UDP is not yet implemented. --- ...port-forward-access-application-cluster.md | 155 ++++++++++++------ 1 file changed, 103 insertions(+), 52 deletions(-) diff --git a/content/en/docs/tasks/access-application-cluster/port-forward-access-application-cluster.md b/content/en/docs/tasks/access-application-cluster/port-forward-access-application-cluster.md index 4a3b201ff3..fc24022d0c 100644 --- a/content/en/docs/tasks/access-application-cluster/port-forward-access-application-cluster.md +++ b/content/en/docs/tasks/access-application-cluster/port-forward-access-application-cluster.md @@ -2,6 +2,7 @@ title: Use Port Forwarding to Access Applications in a Cluster content_template: templates/task weight: 40 +min-kubernetes-server-version: v1.10 --- {{% capture overview %}} @@ -26,104 +27,157 @@ for database debugging. ## Creating Redis deployment and service -1. Create a Redis deployment: +1. Create a Deployment that runs Redis: - kubectl apply -f https://k8s.io/examples/application/guestbook/redis-master-deployment.yaml + ```shell + kubectl apply -f https://k8s.io/examples/application/guestbook/redis-master-deployment.yaml + ``` The output of a successful command verifies that the deployment was created: - deployment.apps/redis-master created + ``` + deployment.apps/redis-master created + ``` View the pod status to check that it is ready: - kubectl get pods + ```shell + kubectl get pods + ``` The output displays the pod created: - NAME READY STATUS RESTARTS AGE - redis-master-765d459796-258hz 1/1 Running 0 50s + ``` + NAME READY STATUS RESTARTS AGE + redis-master-765d459796-258hz 1/1 Running 0 50s + ``` - View the deployment status: + View the Deployment's status: - kubectl get deployment + ```shell + kubectl get deployment + ``` - The output displays that the deployment was created: + The output displays that the Deployment was created: - NAME READY UP-TO-DATE AVAILABLE AGE - redis-master 1/1 1 1 55s + ``` + NAME READY UP-TO-DATE AVAILABLE AGE + redis-master 1/1 1 1 55s + ``` - View the replicaset status using: + The Deployment automatically manages a ReplicaSet. + View the ReplicaSet status using: - kubectl get rs + ```shell + kubectl get replicaset + ``` - The output displays that the replicaset was created: + The output displays that the ReplicaSet was created: - NAME DESIRED CURRENT READY AGE - redis-master-765d459796 1 1 1 1m + ``` + NAME DESIRED CURRENT READY AGE + redis-master-765d459796 1 1 1 1m + ``` -2. Create a Redis service: +2. Create a Service to expose Redis on the network: - kubectl apply -f https://k8s.io/examples/application/guestbook/redis-master-service.yaml + ```shell + kubectl apply -f https://k8s.io/examples/application/guestbook/redis-master-service.yaml + ``` - The output of a successful command verifies that the service was created: + The output of a successful command verifies that the Service was created: - service/redis-master created + ``` + service/redis-master created + ``` - Check the service created: + Check the Service created: - kubectl get svc | grep redis + ```shell + kubectl get service redis-master + ``` The output displays the service created: - NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE - redis-master ClusterIP 10.0.0.213 6379/TCP 27s + ``` + NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE + redis-master ClusterIP 10.0.0.213 6379/TCP 27s + ``` -3. Verify that the Redis server is running in the pod and listening on port 6379: +3. Verify that the Redis server is running in the Pod, and listening on port 6379: - kubectl get pods redis-master-765d459796-258hz --template='{{(index (index .spec.containers 0).ports 0).containerPort}}{{"\n"}}' + ```shell + # Change redis-master-765d459796-258hz to the name of the Pod + kubectl get pod redis-master-765d459796-258hz --template='{{(index (index .spec.containers 0).ports 0).containerPort}}{{"\n"}}' + ``` - The output displays the port: + The output displays the port for Redis in that Pod: - 6379 + ``` + 6379 + ``` + + (this is the TCP port allocated to Redis on the internet). + +## Forward a local port to a port on the Pod + +1. `kubectl port-forward` allows using resource name, such as a pod name, to select a matching pod to port forward to. -## Forward a local port to a port on the pod - -1. `kubectl port-forward` allows using resource name, such as a pod name, to select a matching pod to port forward to since Kubernetes v1.10. - - kubectl port-forward redis-master-765d459796-258hz 7000:6379 + ```shell + # Change redis-master-765d459796-258hz to the name of the Pod + kubectl port-forward redis-master-765d459796-258hz 7000:6379 + ``` which is the same as - kubectl port-forward pods/redis-master-765d459796-258hz 7000:6379 + ```shell + kubectl port-forward pods/redis-master-765d459796-258hz 7000:6379 + ``` or - kubectl port-forward deployment/redis-master 7000:6379 + ```shell + kubectl port-forward deployment/redis-master 7000:6379 + ``` or - kubectl port-forward rs/redis-master 7000:6379 + ```shell + kubectl port-forward replicaset/redis-master 7000:6379 + ``` or - kubectl port-forward svc/redis-master 7000:6379 + ```shell + kubectl port-forward service/redis-master 7000:6379 + ``` Any of the above commands works. The output is similar to this: - I0710 14:43:38.274550 3655 portforward.go:225] Forwarding from 127.0.0.1:7000 -> 6379 - I0710 14:43:38.274797 3655 portforward.go:225] Forwarding from [::1]:7000 -> 6379 + ``` + I0710 14:43:38.274550 3655 portforward.go:225] Forwarding from 127.0.0.1:7000 -> 6379 + I0710 14:43:38.274797 3655 portforward.go:225] Forwarding from [::1]:7000 -> 6379 + ``` 2. Start the Redis command line interface: - redis-cli -p 7000 + ```shell + redis-cli -p 7000 + ``` 3. At the Redis command line prompt, enter the `ping` command: - 127.0.0.1:7000>ping + ``` + ping + ``` - A successful ping request returns PONG. + A successful ping request returns: + + ``` + PONG + ``` {{% /capture %}} @@ -132,15 +186,15 @@ for database debugging. ## Discussion -Connections made to local port 7000 are forwarded to port 6379 of the pod that -is running the Redis server. With this connection in place you can use your -local workstation to debug the database that is running in the pod. +Connections made to local port 7000 are forwarded to port 6379 of the Pod that +is running the Redis server. With this connection in place, you can use your +local workstation to debug the database that is running in the Pod. -{{< warning >}} -Due to known limitations, port forward today only works for TCP protocol. -The support to UDP protocol is being tracked in +{{< note >}} +`kubectl port-forward` is implemented for TCP ports only. +The support for UDP protocol is tracked in [issue 47862](https://github.com/kubernetes/kubernetes/issues/47862). -{{< /warning >}} +{{< /note >}} {{% /capture %}} @@ -148,6 +202,3 @@ The support to UDP protocol is being tracked in {{% capture whatsnext %}} Learn more about [kubectl port-forward](/docs/reference/generated/kubectl/kubectl-commands/#port-forward). {{% /capture %}} - - - From 6c0e47e6527f4476b03e07450beb416892fba1c1 Mon Sep 17 00:00:00 2001 From: makocchi Date: Mon, 16 Mar 2020 14:12:35 +0900 Subject: [PATCH 077/140] Add defails about enabling or disabling extensions/v1beta1 api group (#18842) --- .../docs/concepts/overview/kubernetes-api.md | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/content/en/docs/concepts/overview/kubernetes-api.md b/content/en/docs/concepts/overview/kubernetes-api.md index 1c6066f4db..ad9569ede6 100644 --- a/content/en/docs/concepts/overview/kubernetes-api.md +++ b/content/en/docs/concepts/overview/kubernetes-api.md @@ -121,21 +121,22 @@ There are two supported paths to extending the API with [custom resources](/docs to make it seamless for clients. -## Enabling API groups +## Enabling or disabling API groups Certain resources and API groups are enabled by default. They can be enabled or disabled by setting `--runtime-config` -on apiserver. `--runtime-config` accepts comma separated values. For ex: to disable batch/v1, set +on apiserver. `--runtime-config` accepts comma separated values. For example: to disable batch/v1, set `--runtime-config=batch/v1=false`, to enable batch/v2alpha1, set `--runtime-config=batch/v2alpha1`. The flag accepts comma separated set of key=value pairs describing runtime configuration of the apiserver. -IMPORTANT: Enabling or disabling groups or resources requires restarting apiserver and controller-manager -to pick up the `--runtime-config` changes. +{{< note >}}Enabling or disabling groups or resources requires restarting apiserver and controller-manager +to pick up the `--runtime-config` changes.{{< /note >}} -## Enabling resources in the groups +## Enabling specific resources in the extensions/v1beta1 group -DaemonSets, Deployments, HorizontalPodAutoscalers, Ingresses, Jobs and ReplicaSets are enabled by default. -Other extensions resources can be enabled by setting `--runtime-config` on -apiserver. `--runtime-config` accepts comma separated values. For example: to disable deployments and ingress, set -`--runtime-config=extensions/v1beta1/deployments=false,extensions/v1beta1/ingresses=false` +DaemonSets, Deployments, StatefulSet, NetworkPolicies, PodSecurityPolicies and ReplicaSets in the `extensions/v1beta1` API group are disabled by default. +For example: to enable deployments and daemonsets, set +`--runtime-config=extensions/v1beta1/deployments=true,extensions/v1beta1/daemonsets=true`. + +{{< note >}}Individual resource enablement/disablement is only supported in the `extensions/v1beta1` API group for legacy reasons.{{< /note >}} {{% /capture %}} From 1e5d05293b26b9c941bc8b08fa3f6d71cacc569f Mon Sep 17 00:00:00 2001 From: neo502721 Date: Mon, 16 Mar 2020 13:14:35 +0800 Subject: [PATCH 078/140] change the job example k8s.io to kubernetes.io (#18699) k8s.io No longer accessible --- .../concepts/workloads/controllers/jobs-run-to-completion.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/concepts/workloads/controllers/jobs-run-to-completion.md b/content/en/docs/concepts/workloads/controllers/jobs-run-to-completion.md index 8024984f48..8848774103 100644 --- a/content/en/docs/concepts/workloads/controllers/jobs-run-to-completion.md +++ b/content/en/docs/concepts/workloads/controllers/jobs-run-to-completion.md @@ -39,7 +39,7 @@ It takes around 10s to complete. You can run the example with this command: ```shell -kubectl apply -f https://k8s.io/examples/controllers/job.yaml +kubectl apply -f https://kubernetes.io/examples/controllers/job.yaml ``` ``` job.batch/pi created From 8af9b075c678380773ffcc86c6cb81f63d4ce45d Mon Sep 17 00:00:00 2001 From: Larbi Aburomaissae Date: Mon, 16 Mar 2020 06:16:35 +0100 Subject: [PATCH 079/140] Appending the kustomization.yaml issue (#18572) Running the second part of filling the `kustomization.yaml` using `cat` an issue arrises where the command breaks due to the additional white spaces. This could cause further issues for the reader might mistake the command as to replace the file content, and this will break the proper functioning of the cluster From 8999de523849c6f30282ab28f794ca128fc83185 Mon Sep 17 00:00:00 2001 From: Tim Bannister Date: Mon, 16 Mar 2020 05:18:35 +0000 Subject: [PATCH 080/140] Fix glossary definitions for cluster infrastructure & operations (#18317) - Fix category typo - Reword --- .../reference/glossary/cluster-infrastructure.md | 4 ++-- .../docs/reference/glossary/cluster-operations.md | 14 +++++++++++--- 2 files changed, 13 insertions(+), 5 deletions(-) diff --git a/content/en/docs/reference/glossary/cluster-infrastructure.md b/content/en/docs/reference/glossary/cluster-infrastructure.md index 0b8a5377b4..d8a6fe7280 100644 --- a/content/en/docs/reference/glossary/cluster-infrastructure.md +++ b/content/en/docs/reference/glossary/cluster-infrastructure.md @@ -8,6 +8,6 @@ short_description: > aka: tags: -- operations +- operation --- -The infrastructure layer provides and maintains VMs, networking, security groups and others. + The infrastructure layer provides and maintains VMs, networking, security groups and others. diff --git a/content/en/docs/reference/glossary/cluster-operations.md b/content/en/docs/reference/glossary/cluster-operations.md index 424e6579fc..cf4831d935 100644 --- a/content/en/docs/reference/glossary/cluster-operations.md +++ b/content/en/docs/reference/glossary/cluster-operations.md @@ -4,10 +4,18 @@ id: cluster-operations date: 2019-05-12 full_link: short_description: > - Activities such as upgrading the clusters, implementing security, storage, ingress, networking, logging and monitoring, and other operations involved in managing a Kubernetes cluster. + The work involved in managing a Kubernetes cluster. aka: tags: -- operations +- operation --- - Activities such as upgrading the clusters, implementing security, storage, ingress, networking, logging and monitoring, and other operations involved in managing a Kubernetes cluster. + The work involved in managing a Kubernetes cluster: managing +day-to-day operations, and co-ordinating upgrades. + + + + Examples of cluster operations work include: deploying new Nodes to +scale the cluster; performing software upgrades; implementing security +controls; adding or removing storage; configuring cluster networking; +managing cluster-wide observability; and responding to events. From b2e7dee806e2223cb72947a7b7019162df0ad435 Mon Sep 17 00:00:00 2001 From: steadysupply Date: Mon, 16 Mar 2020 05:20:35 +0000 Subject: [PATCH 081/140] Grammar correction (#19083) * Grammar correction * Update ingress.md --- content/en/docs/concepts/services-networking/ingress.md | 2 +- content/en/docs/reference/glossary/ingress.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/content/en/docs/concepts/services-networking/ingress.md b/content/en/docs/concepts/services-networking/ingress.md index f69cce446f..a46981d28b 100644 --- a/content/en/docs/concepts/services-networking/ingress.md +++ b/content/en/docs/concepts/services-networking/ingress.md @@ -37,7 +37,7 @@ Traffic routing is controlled by rules defined on the Ingress resource. [ Services ] ``` -An Ingress can be configured to give Services externally-reachable URLs, load balance traffic, terminate SSL / TLS, and offer name based virtual hosting. An [Ingress controller](/docs/concepts/services-networking/ingress-controllers) is responsible for fulfilling the Ingress, usually with a load balancer, though it may also configure your edge router or additional frontends to help handle the traffic. +An Ingress may be configured to give Services externally-reachable URLs, load balance traffic, terminate SSL / TLS, and offer name based virtual hosting. An [Ingress controller](/docs/concepts/services-networking/ingress-controllers) is responsible for fulfilling the Ingress, usually with a load balancer, though it may also configure your edge router or additional frontends to help handle the traffic. An Ingress does not expose arbitrary ports or protocols. Exposing services other than HTTP and HTTPS to the internet typically uses a service of type [Service.Type=NodePort](/docs/concepts/services-networking/service/#nodeport) or diff --git a/content/en/docs/reference/glossary/ingress.md b/content/en/docs/reference/glossary/ingress.md index 0a83428002..6fe935fb4e 100755 --- a/content/en/docs/reference/glossary/ingress.md +++ b/content/en/docs/reference/glossary/ingress.md @@ -16,5 +16,5 @@ tags: -Ingress can provide load balancing, SSL termination and name-based virtual hosting. +Ingress may provide load balancing, SSL termination and name-based virtual hosting. From 6f48108c95c0d6c8f89b991f61fe68c46b9c8a5f Mon Sep 17 00:00:00 2001 From: Tim Bannister Date: Mon, 16 Mar 2020 05:24:35 +0000 Subject: [PATCH 082/140] Cleanup container probes task (#18233) * Fix markdown for sample output Don't highlight this sample output as if it's Bourne shell code. * Tweak capitalization - Pod is an API object type - Container exists in the Kubernetes API, but you can't: kubectl get container - Socket isn't an API object type * Pin source to relevant release Handle case where sample Golang server changes in master branch post release. * Tweak page wording - Reword a heading in What's Next to explain the links - explain context for a probe --- ...igure-liveness-readiness-startup-probes.md | 62 +++++++++---------- 1 file changed, 31 insertions(+), 31 deletions(-) diff --git a/content/en/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes.md b/content/en/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes.md index dfa0a1b414..d4b306f02b 100644 --- a/content/en/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes.md +++ b/content/en/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes.md @@ -6,20 +6,20 @@ weight: 110 {{% capture overview %}} -This page shows how to configure liveness, readiness and startup probes for Containers. +This page shows how to configure liveness, readiness and startup probes for containers. The [kubelet](/docs/admin/kubelet/) uses liveness probes to know when to -restart a Container. For example, liveness probes could catch a deadlock, +restart a container. For example, liveness probes could catch a deadlock, where an application is running, but unable to make progress. Restarting a -Container in such a state can help to make the application more available +container in such a state can help to make the application more available despite bugs. -The kubelet uses readiness probes to know when a Container is ready to start -accepting traffic. A Pod is considered ready when all of its Containers are ready. +The kubelet uses readiness probes to know when a container is ready to start +accepting traffic. A Pod is considered ready when all of its containers are ready. One use of this signal is to control which Pods are used as backends for Services. When a Pod is not ready, it is removed from Service load balancers. -The kubelet uses startup probes to know when a Container application has started. +The kubelet uses startup probes to know when a container application has started. If such a probe is configured, it disables liveness and readiness checks until it succeeds, making sure those probes don't interfere with the application startup. This can be used to adopt liveness checks on slow starting containers, avoiding them @@ -41,27 +41,27 @@ Many applications running for long periods of time eventually transition to broken states, and cannot recover except by being restarted. Kubernetes provides liveness probes to detect and remedy such situations. -In this exercise, you create a Pod that runs a Container based on the +In this exercise, you create a Pod that runs a container based on the `k8s.gcr.io/busybox` image. Here is the configuration file for the Pod: {{< codenew file="pods/probe/exec-liveness.yaml" >}} -In the configuration file, you can see that the Pod has a single Container. +In the configuration file, you can see that the Pod has a single `Container`. The `periodSeconds` field specifies that the kubelet should perform a liveness probe every 5 seconds. The `initialDelaySeconds` field tells the kubelet that it should wait 5 second before performing the first probe. To perform a probe, the -kubelet executes the command `cat /tmp/healthy` in the Container. If the -command succeeds, it returns 0, and the kubelet considers the Container to be alive and -healthy. If the command returns a non-zero value, the kubelet kills the Container +kubelet executes the command `cat /tmp/healthy` in the target container. If the +command succeeds, it returns 0, and the kubelet considers the container to be alive and +healthy. If the command returns a non-zero value, the kubelet kills the container and restarts it. -When the Container starts, it executes this command: +When the container starts, it executes this command: ```shell /bin/sh -c "touch /tmp/healthy; sleep 30; rm -rf /tmp/healthy; sleep 600" ``` -For the first 30 seconds of the Container's life, there is a `/tmp/healthy` file. +For the first 30 seconds of the container's life, there is a `/tmp/healthy` file. So during the first 30 seconds, the command `cat /tmp/healthy` returns a success code. After 30 seconds, `cat /tmp/healthy` returns a failure code. @@ -79,7 +79,7 @@ kubectl describe pod liveness-exec The output indicates that no liveness probes have failed yet: -```shell +``` FirstSeen LastSeen Count From SubobjectPath Type Reason Message --------- -------- ----- ---- ------------- -------- ------ ------- 24s 24s 1 {default-scheduler } Normal Scheduled Successfully assigned liveness-exec to worker0 @@ -98,7 +98,7 @@ kubectl describe pod liveness-exec At the bottom of the output, there are messages indicating that the liveness probes have failed, and the containers have been killed and recreated. -```shell +``` FirstSeen LastSeen Count From SubobjectPath Type Reason Message --------- -------- ----- ---- ------------- -------- ------ ------- 37s 37s 1 {default-scheduler } Normal Scheduled Successfully assigned liveness-exec to worker0 @@ -109,7 +109,7 @@ FirstSeen LastSeen Count From SubobjectPath Type 2s 2s 1 {kubelet worker0} spec.containers{liveness} Warning Unhealthy Liveness probe failed: cat: can't open '/tmp/healthy': No such file or directory ``` -Wait another 30 seconds, and verify that the Container has been restarted: +Wait another 30 seconds, and verify that the container has been restarted: ```shell kubectl get pod liveness-exec @@ -117,7 +117,7 @@ kubectl get pod liveness-exec The output shows that `RESTARTS` has been incremented: -```shell +``` NAME READY STATUS RESTARTS AGE liveness-exec 1/1 Running 1 1m ``` @@ -130,23 +130,23 @@ image. {{< codenew file="pods/probe/http-liveness.yaml" >}} -In the configuration file, you can see that the Pod has a single Container. +In the configuration file, you can see that the Pod has a single container. The `periodSeconds` field specifies that the kubelet should perform a liveness probe every 3 seconds. The `initialDelaySeconds` field tells the kubelet that it should wait 3 seconds before performing the first probe. To perform a probe, the -kubelet sends an HTTP GET request to the server that is running in the Container +kubelet sends an HTTP GET request to the server that is running in the container and listening on port 8080. If the handler for the server's `/healthz` path -returns a success code, the kubelet considers the Container to be alive and -healthy. If the handler returns a failure code, the kubelet kills the Container +returns a success code, the kubelet considers the container to be alive and +healthy. If the handler returns a failure code, the kubelet kills the container and restarts it. Any code greater than or equal to 200 and less than 400 indicates success. Any other code indicates failure. You can see the source code for the server in -[server.go](https://github.com/kubernetes/kubernetes/blob/master/test/images/agnhost/liveness/server.go). +[server.go](https://github.com/kubernetes/kubernetes/blob/{{< param "githubbranch" >}}/test/images/agnhost/liveness/server.go). -For the first 10 seconds that the Container is alive, the `/healthz` handler +For the first 10 seconds that the container is alive, the `/healthz` handler returns a status of 200. After that, the handler returns a status of 500. ```go @@ -162,9 +162,9 @@ http.HandleFunc("/healthz", func(w http.ResponseWriter, r *http.Request) { }) ``` -The kubelet starts performing health checks 3 seconds after the Container starts. +The kubelet starts performing health checks 3 seconds after the container starts. So the first couple of health checks will succeed. But after 10 seconds, the health -checks will fail, and the kubelet will kill and restart the Container. +checks will fail, and the kubelet will kill and restart the container. To try the HTTP liveness check, create a Pod: @@ -173,21 +173,21 @@ kubectl apply -f https://k8s.io/examples/pods/probe/http-liveness.yaml ``` After 10 seconds, view Pod events to verify that liveness probes have failed and -the Container has been restarted: +the container has been restarted: ```shell kubectl describe pod liveness-http ``` In releases prior to v1.13 (including v1.13), if the environment variable -`http_proxy` (or `HTTP_PROXY`) is set on the node where a pod is running, +`http_proxy` (or `HTTP_PROXY`) is set on the node where a Pod is running, the HTTP liveness probe uses that proxy. In releases after v1.13, local HTTP proxy environment variable settings do not affect the HTTP liveness probe. ## Define a TCP liveness probe -A third type of liveness probe uses a TCP Socket. With this configuration, the +A third type of liveness probe uses a TCP socket. With this configuration, the kubelet will attempt to open a socket to your container on the specified port. If it can establish a connection, the container is considered healthy, if it can’t it is considered a failure. @@ -197,7 +197,7 @@ can’t it is considered a failure. As you can see, configuration for a TCP check is quite similar to an HTTP check. This example uses both readiness and liveness probes. The kubelet will send the first readiness probe 5 seconds after the container starts. This will attempt to -connect to the `goproxy` container on port 8080. If the probe succeeds, the pod +connect to the `goproxy` container on port 8080. If the probe succeeds, the Pod will be marked as ready. The kubelet will continue to run this check every 10 seconds. @@ -351,7 +351,7 @@ port to perform the check. The kubelet sends the probe to the pod’s IP address unless the address is overridden by the optional `host` field in `httpGet`. If `scheme` field is set to `HTTPS`, the kubelet sends an HTTPS request skipping the certificate verification. In most scenarios, you do not want to set the `host` field. -Here's one scenario where you would set it. Suppose the Container listens on 127.0.0.1 +Here's one scenario where you would set it. Suppose the container listens on 127.0.0.1 and the Pod's `hostNetwork` field is true. Then `host`, under `httpGet`, should be set to 127.0.0.1. If your pod relies on virtual hosts, which is probably the more common case, you should not use `host`, but rather set the `Host` header in `httpHeaders`. @@ -367,7 +367,7 @@ to resolve it. * Learn more about [Container Probes](/docs/concepts/workloads/pods/pod-lifecycle/#container-probes). -### Reference +You can also read the API references for: * [Pod](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#pod-v1-core) * [Container](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#container-v1-core) From ec909fd010efe9d6480582c3582c617e7699ada6 Mon Sep 17 00:00:00 2001 From: Tim Bannister Date: Mon, 16 Mar 2020 05:28:35 +0000 Subject: [PATCH 083/140] Add overview for container concept (#18090) --- .../en/docs/concepts/containers/overview.md | 45 +++++++++++++++++++ 1 file changed, 45 insertions(+) create mode 100644 content/en/docs/concepts/containers/overview.md diff --git a/content/en/docs/concepts/containers/overview.md b/content/en/docs/concepts/containers/overview.md new file mode 100644 index 0000000000..968ba7104a --- /dev/null +++ b/content/en/docs/concepts/containers/overview.md @@ -0,0 +1,45 @@ +--- +reviewers: +- erictune +- thockin +title: Containers overview +content_template: templates/concept +weight: 1 +--- + +{{% capture overview %}} + +Containers are a technnology for packaging the (compiled) code for an +application along with the dependencies it needs at run time. Each +container that you run is repeatable; the standardisation from having +dependencies included means that you get the same behavior wherever you +run it. + +Containers decouple applications from underlying host infrastructure. +This makes deployment easier in different cloud or OS environments. + +{{% /capture %}} + + +{{% capture body %}} + +## Container images +A [container image](/docs/concepts/containers/images/) is a ready-to-run +software package, containing everything needed to run an application: +the code and any runtime it requires, application and system libraries, +and default values for any essential settings. + +By design, a container is immutable: you cannot change the code of a +container that is already running. If you have a containerized application +and want to make changes, you need to build a new container that includes +the change, then recreate the container to start from the updated image. + +## Container runtimes + +{{< glossary_definition term_id="container-runtime" length="all" >}} + +{{% /capture %}} +{{% capture whatsnext %}} +* Read about [container images](/docs/concepts/containers/images/) +* Read about [Pods](/docs/concepts/workloads/pods/) +{{% /capture %}} From d341eee62ec75d1a92db87712c4648e88e522d62 Mon Sep 17 00:00:00 2001 From: Tim Bannister Date: Mon, 16 Mar 2020 05:30:35 +0000 Subject: [PATCH 084/140] Add cross-references to fundamental concepts (#18092) --- content/en/docs/reference/glossary/annotation.md | 2 +- content/en/docs/reference/glossary/cluster.md | 10 ++++++++-- .../reference/glossary/container-env-variables.md | 4 ++-- content/en/docs/reference/glossary/deployment.md | 2 +- .../glossary/horizontal-pod-autoscaler.md | 4 ++-- .../en/docs/reference/glossary/host-aliases.md | 2 +- content/en/docs/reference/glossary/image.md | 4 ++-- .../en/docs/reference/glossary/init-container.md | 7 +++---- content/en/docs/reference/glossary/kube-proxy.md | 14 ++++++++------ .../en/docs/reference/glossary/kube-scheduler.md | 15 ++++++++++----- content/en/docs/reference/glossary/kubeadm.md | 2 +- content/en/docs/reference/glossary/kubelet.md | 3 +-- .../reference/glossary/persistent-volume-claim.md | 5 ++--- .../en/docs/reference/glossary/pod-priority.md | 2 +- content/en/docs/reference/glossary/podpreset.md | 4 ++-- content/en/docs/reference/glossary/preemption.md | 2 +- .../reference/glossary/replication-controller.md | 2 +- .../docs/reference/glossary/security-context.md | 14 ++++++++++---- content/en/docs/reference/glossary/selector.md | 4 ++-- content/en/docs/reference/glossary/taint.md | 4 ++-- content/en/docs/reference/glossary/volume.md | 5 +++-- 21 files changed, 64 insertions(+), 47 deletions(-) diff --git a/content/en/docs/reference/glossary/annotation.md b/content/en/docs/reference/glossary/annotation.md index 03feb69a05..474573ef16 100755 --- a/content/en/docs/reference/glossary/annotation.md +++ b/content/en/docs/reference/glossary/annotation.md @@ -14,5 +14,5 @@ tags: -The metadata in an annotation can be small or large, structured or unstructured, and can include characters not permitted by labels. Clients such as tools and libraries can retrieve this metadata. +The metadata in an annotation can be small or large, structured or unstructured, and can include characters not permitted by {{< glossary_tooltip text="labels" term_id="label" >}}. Clients such as tools and libraries can retrieve this metadata. diff --git a/content/en/docs/reference/glossary/cluster.md b/content/en/docs/reference/glossary/cluster.md index 2e8aecb23e..8f2659459b 100755 --- a/content/en/docs/reference/glossary/cluster.md +++ b/content/en/docs/reference/glossary/cluster.md @@ -11,7 +11,13 @@ tags: - fundamental - operation --- -A set of worker machines, called nodes, that run containerized applications. Every cluster has at least one worker node. +A set of worker machines, called {{< glossary_tooltip text="nodes" term_id="node" >}}, +that run containerized applications. Every cluster has at least one worker node. -The worker node(s) host the pods that are the components of the application. The Control Plane manages the worker nodes and the pods in the cluster. In production environments, the Control Plane usually runs across multiple computers and a cluster usually runs multiple nodes, providing fault-tolerance and high availability. +The worker node(s) host the {{< glossary_tooltip text="Pods" term_id="pod" >}} that are +the components of the application workload. The +{{< glossary_tooltip text="control plane" term_id="control-plane" >}} manages the worker +nodes and the Pods in the cluster. In production environments, the control plane usually +runs across multiple computers and a cluster usually runs multiple nodes, providing +fault-tolerance and high availability. diff --git a/content/en/docs/reference/glossary/container-env-variables.md b/content/en/docs/reference/glossary/container-env-variables.md index a4682f5719..5e19a1dfa2 100755 --- a/content/en/docs/reference/glossary/container-env-variables.md +++ b/content/en/docs/reference/glossary/container-env-variables.md @@ -10,8 +10,8 @@ aka: tags: - fundamental --- - Container environment variables are name=value pairs that provide useful information into containers running in a Pod. + Container environment variables are name=value pairs that provide useful information into containers running in a {{< glossary_tooltip text="pod" term_id="pod" >}} -Container environment variables provide information that is required by the running containerized applications along with information about important resources to the {{< glossary_tooltip text="Containers" term_id="container" >}}. For example, file system details, information about the container itself, and other cluster resources such as service endpoints. \ No newline at end of file +Container environment variables provide information that is required by the running containerized applications along with information about important resources to the {{< glossary_tooltip text="containers" term_id="container" >}}. For example, file system details, information about the container itself, and other cluster resources such as service endpoints. diff --git a/content/en/docs/reference/glossary/deployment.md b/content/en/docs/reference/glossary/deployment.md index 89a4f7b9b2..b1ea465746 100755 --- a/content/en/docs/reference/glossary/deployment.md +++ b/content/en/docs/reference/glossary/deployment.md @@ -16,5 +16,5 @@ tags: -Each replica is represented by a {{< glossary_tooltip term_id="pod" >}}, and the Pods are distributed among the nodes of a cluster. +Each replica is represented by a {{< glossary_tooltip term_id="pod" >}}, and the Pods are distributed among the {{< glossary_tooltip text="nodes" term_id="node" >}} of a cluster. diff --git a/content/en/docs/reference/glossary/horizontal-pod-autoscaler.md b/content/en/docs/reference/glossary/horizontal-pod-autoscaler.md index 4e3ace729e..d90009583c 100755 --- a/content/en/docs/reference/glossary/horizontal-pod-autoscaler.md +++ b/content/en/docs/reference/glossary/horizontal-pod-autoscaler.md @@ -11,9 +11,9 @@ aka: tags: - operation --- - An API resource that automatically scales the number of pod replicas based on targeted CPU utilization or custom metric targets. + An API resource that automatically scales the number of {{< glossary_tooltip term_id="pod" >}} replicas based on targeted CPU utilization or custom metric targets. -HPA is typically used with {{< glossary_tooltip text="Replication Controllers" term_id="replication-controller" >}}, {{< glossary_tooltip text="Deployments" term_id="deployment" >}}, or Replica Sets. It cannot be applied to objects that cannot be scaled, for example {{< glossary_tooltip text="DaemonSets" term_id="daemonset" >}}. +HPA is typically used with {{< glossary_tooltip text="ReplicationControllers" term_id="replication-controller" >}}, {{< glossary_tooltip text="Deployments" term_id="deployment" >}}, or {{< glossary_tooltip text="ReplicaSets" term_id="replica-set" >}}. It cannot be applied to objects that cannot be scaled, for example {{< glossary_tooltip text="DaemonSets" term_id="daemonset" >}}. diff --git a/content/en/docs/reference/glossary/host-aliases.md b/content/en/docs/reference/glossary/host-aliases.md index fa92be21f4..47bd22d433 100644 --- a/content/en/docs/reference/glossary/host-aliases.md +++ b/content/en/docs/reference/glossary/host-aliases.md @@ -10,7 +10,7 @@ aka: tags: - operation --- - A HostAliases is a mapping between the IP address and hostname to be injected into a Pod's hosts file. + A HostAliases is a mapping between the IP address and hostname to be injected into a {{< glossary_tooltip text="Pod" term_id="pod" >}}'s hosts file. diff --git a/content/en/docs/reference/glossary/image.md b/content/en/docs/reference/glossary/image.md index cdb2c7e02c..d185fc2eed 100755 --- a/content/en/docs/reference/glossary/image.md +++ b/content/en/docs/reference/glossary/image.md @@ -10,9 +10,9 @@ aka: tags: - fundamental --- - Stored instance of a container that holds a set of software needed to run an application. + Stored instance of a {{< glossary_tooltip term_id="container" >}} that holds a set of software needed to run an application. - + A way of packaging software that allows it to be stored in a container registry, pulled to a local system, and run as an application. Meta data is included in the image that can indicate what executable to run, who built it, and other information. diff --git a/content/en/docs/reference/glossary/init-container.md b/content/en/docs/reference/glossary/init-container.md index 2ea793661c..a999042e30 100755 --- a/content/en/docs/reference/glossary/init-container.md +++ b/content/en/docs/reference/glossary/init-container.md @@ -4,15 +4,14 @@ id: init-container date: 2018-04-12 full_link: short_description: > - One or more initialization containers that must run to completion before any app containers run. + One or more initialization containers that must run to completion before any app containers run. aka: tags: - fundamental --- - One or more initialization containers that must run to completion before any app containers run. + One or more initialization {{< glossary_tooltip text="containers" term_id="container" >}} that must run to completion before any app containers run. -Initialization (init) containers are like regular app containers, with one difference: init containers must run to completion before any app containers can start. Init containers run in series: each init container must run to completion before the next init container begins. - +Initialization (init) containers are like regular app containers, with one difference: init containers must run to completion before any app containers can start. Init containers run in series: each init container must run to completion before the next init container begins. diff --git a/content/en/docs/reference/glossary/kube-proxy.md b/content/en/docs/reference/glossary/kube-proxy.md index 87be7e68ac..3b2a572504 100755 --- a/content/en/docs/reference/glossary/kube-proxy.md +++ b/content/en/docs/reference/glossary/kube-proxy.md @@ -11,15 +11,17 @@ tags: - fundamental - networking --- - [kube-proxy](/docs/reference/command-line-tools-reference/kube-proxy/) is a -network proxy that runs on each node in your cluster, implementing part of -the Kubernetes {{< glossary_tooltip term_id="service">}} concept. + kube-proxy is a network proxy that runs on each +{{< glossary_tooltip text="node" term_id="node" >}} in your cluster, +implementing part of the Kubernetes +{{< glossary_tooltip term_id="service">}} concept. -kube-proxy maintains network rules on nodes. These network rules allow -network communication to your Pods from network sessions inside or outside -of your cluster. +[kube-proxy](/docs/reference/command-line-tools-reference/kube-proxy/) +maintains network rules on nodes. These network rules allow network +communication to your Pods from network sessions inside or outside of +your cluster. kube-proxy uses the operating system packet filtering layer if there is one and it's available. Otherwise, kube-proxy forwards the traffic itself. diff --git a/content/en/docs/reference/glossary/kube-scheduler.md b/content/en/docs/reference/glossary/kube-scheduler.md index 7094a6982a..e3babcab3f 100755 --- a/content/en/docs/reference/glossary/kube-scheduler.md +++ b/content/en/docs/reference/glossary/kube-scheduler.md @@ -4,15 +4,20 @@ id: kube-scheduler date: 2018-04-12 full_link: /docs/reference/generated/kube-scheduler/ short_description: > - Control Plane component that watches for newly created pods with no assigned node, and selects a node for them to run on. + Control plane component that watches for newly created pods with no assigned node, and selects a node for them to run on. aka: tags: - architecture --- - Control Plane component that watches for newly created pods with no assigned node, and selects a node for them to run on. +Control plane component that watches for newly created +{{< glossary_tooltip term_id="node" >}} with no assigned +{{< glossary_tooltip term_id="node" text="node">}}, and selects a node for them +to run on. - - -Factors taken into account for scheduling decisions include individual and collective resource requirements, hardware/software/policy constraints, affinity and anti-affinity specifications, data locality, inter-workload interference and deadlines. + +Factors taken into account for scheduling decisions include: +individual and collective resource requirements, hardware/software/policy +constraints, affinity and anti-affinity specifications, data locality, +inter-workload interference, and deadlines. diff --git a/content/en/docs/reference/glossary/kubeadm.md b/content/en/docs/reference/glossary/kubeadm.md index ce26062270..74cc0d1f6a 100755 --- a/content/en/docs/reference/glossary/kubeadm.md +++ b/content/en/docs/reference/glossary/kubeadm.md @@ -15,5 +15,5 @@ tags: -You can use kubeadm to install both the control plane and the worker node components. +You can use kubeadm to install both the control plane and the {{< glossary_tooltip text="worker node" term_id="node" >}} components. diff --git a/content/en/docs/reference/glossary/kubelet.md b/content/en/docs/reference/glossary/kubelet.md index 0c4ea9425a..0c95ac846e 100755 --- a/content/en/docs/reference/glossary/kubelet.md +++ b/content/en/docs/reference/glossary/kubelet.md @@ -11,9 +11,8 @@ tags: - fundamental - core-object --- - An agent that runs on each node in the cluster. It makes sure that containers are running in a pod. + An agent that runs on each {{< glossary_tooltip text="node" term_id="node" >}} in the cluster. It makes sure that {{< glossary_tooltip text="containers" term_id="container" >}} are running in a {{< glossary_tooltip text="Pod" term_id="pod" >}}. The kubelet takes a set of PodSpecs that are provided through various mechanisms and ensures that the containers described in those PodSpecs are running and healthy. The kubelet doesn’t manage containers which were not created by Kubernetes. - diff --git a/content/en/docs/reference/glossary/persistent-volume-claim.md b/content/en/docs/reference/glossary/persistent-volume-claim.md index dc6314d887..6af54d60f5 100755 --- a/content/en/docs/reference/glossary/persistent-volume-claim.md +++ b/content/en/docs/reference/glossary/persistent-volume-claim.md @@ -11,9 +11,8 @@ tags: - core-object - storage --- - Claims storage resources defined in a PersistentVolume so that it can be mounted as a volume in a container. + Claims storage resources defined in a {{< glossary_tooltip text="PersistentVolume" term_id="persistent-volume" >}} so that it can be mounted as a volume in a {{< glossary_tooltip text="container" term_id="container" >}}. -Specifies the amount of storage, how the storage will be accessed (read-only, read-write and/or exclusive) and how it is reclaimed (retained, recycled or deleted). Details of the storage itself are in the PersistentVolume specification. - +Specifies the amount of storage, how the storage will be accessed (read-only, read-write and/or exclusive) and how it is reclaimed (retained, recycled or deleted). Details of the storage itself are described in the PersistentVolume object. diff --git a/content/en/docs/reference/glossary/pod-priority.md b/content/en/docs/reference/glossary/pod-priority.md index 0b80602b35..994f8bc4d8 100644 --- a/content/en/docs/reference/glossary/pod-priority.md +++ b/content/en/docs/reference/glossary/pod-priority.md @@ -10,7 +10,7 @@ aka: tags: - operation --- - Pod Priority indicates the importance of a Pod relative to other Pods. + Pod Priority indicates the importance of a {{< glossary_tooltip term_id="pod" >}} relative to other Pods. diff --git a/content/en/docs/reference/glossary/podpreset.md b/content/en/docs/reference/glossary/podpreset.md index c60f03cd95..f63187ff71 100755 --- a/content/en/docs/reference/glossary/podpreset.md +++ b/content/en/docs/reference/glossary/podpreset.md @@ -10,9 +10,9 @@ aka: tags: - operation --- - An API object that injects information such as secrets, volume mounts, and environment variables into pods at creation time. + An API object that injects information such as secrets, volume mounts, and environment variables into {{< glossary_tooltip text="Pods" term_id="pod" >}} at creation time. -This object chooses the pods to inject information into using standard selectors. This allows the podspec definitions to be nonspecific, decoupling the podspec from environment specific configuration. +This object chooses the Pods to inject information into using standard selectors. This allows the podspec definitions to be nonspecific, decoupling the podspec from environment specific configuration. diff --git a/content/en/docs/reference/glossary/preemption.md b/content/en/docs/reference/glossary/preemption.md index 0810acfcfc..f27e36c66f 100644 --- a/content/en/docs/reference/glossary/preemption.md +++ b/content/en/docs/reference/glossary/preemption.md @@ -10,7 +10,7 @@ aka: tags: - operation --- - Preemption logic in Kubernetes helps a pending Pod to find a suitable Node by evicting low priority Pods existing on that Node. + Preemption logic in Kubernetes helps a pending {{< glossary_tooltip term_id="pod" >}} to find a suitable {{< glossary_tooltip term_id="node" >}} by evicting low priority Pods existing on that Node. diff --git a/content/en/docs/reference/glossary/replication-controller.md b/content/en/docs/reference/glossary/replication-controller.md index b564c29691..9f4523e0f5 100755 --- a/content/en/docs/reference/glossary/replication-controller.md +++ b/content/en/docs/reference/glossary/replication-controller.md @@ -11,7 +11,7 @@ tags: - workload - core-object --- - Kubernetes service that ensures a specific number of instances of a pod are always running. + Kubernetes service that ensures a specific number of instances of a {{< glossary_tooltip text="Pod" term_id="pod" >}} are always running. diff --git a/content/en/docs/reference/glossary/security-context.md b/content/en/docs/reference/glossary/security-context.md index 9812304e4d..c53e96aa19 100755 --- a/content/en/docs/reference/glossary/security-context.md +++ b/content/en/docs/reference/glossary/security-context.md @@ -4,14 +4,20 @@ id: security-context date: 2018-04-12 full_link: /docs/tasks/configure-pod-container/security-context/ short_description: > - The securityContext field defines privilege and access control settings for a Pod or Container, including the runtime UID and GID. + The securityContext field defines privilege and access control settings for a Pod or container. aka: tags: - security --- - The securityContext field defines privilege and access control settings for a Pod or Container, including the runtime UID and GID. + The `securityContext` field defines privilege and access control settings for +a {{< glossary_tooltip text="Pod" term_id="pod" >}} or +{{< glossary_tooltip text="container" term_id="container" >}}. - + -The securityContext field in a {{< glossary_tooltip term_id="pod" >}} (applying to all containers) or container is used to set the user, groups, capabilities, privilege settings, and security policies (SELinux/AppArmor/Seccomp) and more that container processes use. +In a `securityContext`, you can define: the user that processes run as, +the group that processes run as, and privilege settings. +You can also configure security policies (for example: SELinux, AppArmor or seccomp). + +The `PodSpec.securityContext` setting applies to all containers in a Pod. diff --git a/content/en/docs/reference/glossary/selector.md b/content/en/docs/reference/glossary/selector.md index 9b8a020252..622b24694d 100755 --- a/content/en/docs/reference/glossary/selector.md +++ b/content/en/docs/reference/glossary/selector.md @@ -10,9 +10,9 @@ aka: tags: - fundamental --- - Allows users to filter a list of resources based on labels. + Allows users to filter a list of resources based on {{< glossary_tooltip text="labels" term_id="label" >}}. -Selectors are applied when querying lists of resources to filter them by {{< glossary_tooltip text="Labels" term_id="label" >}}. +Selectors are applied when querying lists of resources to filter them by labels. diff --git a/content/en/docs/reference/glossary/taint.md b/content/en/docs/reference/glossary/taint.md index 9faaee579b..c1aaaf5f65 100644 --- a/content/en/docs/reference/glossary/taint.md +++ b/content/en/docs/reference/glossary/taint.md @@ -11,8 +11,8 @@ tags: - core-object - fundamental --- - A core object consisting of three required properties: key, value, and effect. Taints prevent the scheduling of pods on nodes or node groups. + A core object consisting of three required properties: key, value, and effect. Taints prevent the scheduling of {{< glossary_tooltip text="Pods" term_id="pod" >}} on {{< glossary_tooltip text="nodes" term_id="node" >}} or node groups. -Taints and {{< glossary_tooltip text="tolerations" term_id="toleration" >}} work together to ensure that pods are not scheduled onto inappropriate nodes. One or more taints are applied to a {{< glossary_tooltip text="node" term_id="node" >}}. A node should only schedule a pod with the matching tolerations for the configured taints. +Taints and {{< glossary_tooltip text="tolerations" term_id="toleration" >}} work together to ensure that pods are not scheduled onto inappropriate nodes. One or more taints are applied to a node. A node should only schedule a Pod with the matching tolerations for the configured taints. diff --git a/content/en/docs/reference/glossary/volume.md b/content/en/docs/reference/glossary/volume.md index f162ce0010..2076378bb3 100755 --- a/content/en/docs/reference/glossary/volume.md +++ b/content/en/docs/reference/glossary/volume.md @@ -11,9 +11,10 @@ tags: - core-object - fundamental --- - A directory containing data, accessible to the containers in a {{< glossary_tooltip text="pod" term_id="pod" >}}. + A directory containing data, accessible to the {{< glossary_tooltip text="containers" term_id="container" >}} in a {{< glossary_tooltip term_id="pod" >}}. -A Kubernetes volume lives as long as the {{< glossary_tooltip text="pod" term_id="pod" >}} that encloses it. Consequently, a volume outlives any {{< glossary_tooltip text="containers" term_id="container" >}} that run within the {{< glossary_tooltip text="pod" term_id="pod" >}}, and data is preserved across {{< glossary_tooltip text="container" term_id="container" >}} restarts. +A Kubernetes volume lives as long as the Pod that encloses it. Consequently, a volume outlives any containers that run within the Pod, and data in the volume is preserved across container restarts. +See [storage](https://kubernetes.io/docs/concepts/storage/) for more information. From f29221f4c40d06d2c91719dc4a6ca2d5c1cc0d49 Mon Sep 17 00:00:00 2001 From: Sagar Velankar Date: Mon, 16 Mar 2020 11:10:35 +0530 Subject: [PATCH 085/140] Issue #17556 (#17557) Issue #17556 Problem: Below Lines are incorrect If you specify multiple nodeSelectorTerms associated with nodeAffinity types, then the pod can be scheduled onto a node if one of the nodeSelectorTerms is satisfied. If you specify multiple matchExpressions associated with nodeSelectorTerms, then the pod can be scheduled onto a node only if all matchExpressions can be satisfied. Proposed Solution: They need to be corrected to below If you specify multiple nodeSelectorTerms associated with nodeAffinity types, then the pod can be scheduled onto a node only if all nodeSelectorTerms can be satisfied. If you specify multiple matchExpressions associated with nodeSelectorTerms, then the pod can be scheduled onto a node if one of the matchExpressions is satisfied. --- content/en/docs/concepts/configuration/assign-pod-node.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/en/docs/concepts/configuration/assign-pod-node.md b/content/en/docs/concepts/configuration/assign-pod-node.md index d1dc2bfbf2..9224a1be11 100644 --- a/content/en/docs/concepts/configuration/assign-pod-node.md +++ b/content/en/docs/concepts/configuration/assign-pod-node.md @@ -160,9 +160,9 @@ You can use `NotIn` and `DoesNotExist` to achieve node anti-affinity behavior, o If you specify both `nodeSelector` and `nodeAffinity`, *both* must be satisfied for the pod to be scheduled onto a candidate node. -If you specify multiple `nodeSelectorTerms` associated with `nodeAffinity` types, then the pod can be scheduled onto a node **if one of** the `nodeSelectorTerms` is satisfied. +If you specify multiple `nodeSelectorTerms` associated with `nodeAffinity` types, then the pod can be scheduled onto a node **only if all** `nodeSelectorTerms` can be satisfied. -If you specify multiple `matchExpressions` associated with `nodeSelectorTerms`, then the pod can be scheduled onto a node **only if all** `matchExpressions` can be satisfied. +If you specify multiple `matchExpressions` associated with `nodeSelectorTerms`, then the pod can be scheduled onto a node **if one of** the `matchExpressions` is satisfied. If you remove or change the label of the node where the pod is scheduled, the pod won't be removed. In other words, the affinity selection works only at the time of scheduling the pod. From 43212d6bc7526e9e78eaf07aac4d016921bb5739 Mon Sep 17 00:00:00 2001 From: Tim Bannister Date: Mon, 16 Mar 2020 05:42:35 +0000 Subject: [PATCH 086/140] Reword pod priority and preemption concept (#17508) * Remove mentions of unsupported Kubernetes versions No need to mention supported-since-this-version details for Kubernetes releases that are now out of support. * Warn about disabling preemption * Tweak wording * Add What's next section * Tweak Pod priority troubleshooting advice Reshape the advice about pod priority troubleshooting to explain user-induced issues (and define expected behavior). If the reader detects behavior that does not match the documentation, they have observed a bug and can report that via the usual routes. * Rewords notes on Pod priority vs. QoS * Move warning into page body --- .../configuration/pod-priority-preemption.md | 156 ++++++++---------- 1 file changed, 69 insertions(+), 87 deletions(-) diff --git a/content/en/docs/concepts/configuration/pod-priority-preemption.md b/content/en/docs/concepts/configuration/pod-priority-preemption.md index 77fbd93c58..1d24c2f094 100644 --- a/content/en/docs/concepts/configuration/pod-priority-preemption.md +++ b/content/en/docs/concepts/configuration/pod-priority-preemption.md @@ -16,42 +16,25 @@ importance of a Pod relative to other Pods. If a Pod cannot be scheduled, the scheduler tries to preempt (evict) lower priority Pods to make scheduling of the pending Pod possible. -In Kubernetes 1.9 and later, Priority also affects scheduling order of Pods and -out-of-resource eviction ordering on the Node. - -Pod priority and preemption graduated to beta in Kubernetes 1.11 and to GA in -Kubernetes 1.14. They have been enabled by default since 1.11. - -In Kubernetes versions where Pod priority and preemption is still an alpha-level -feature, you need to explicitly enable it. To use these features in the older -versions of Kubernetes, follow the instructions in the documentation for your -Kubernetes version, by going to the documentation archive version for your -Kubernetes version. - -Kubernetes Version | Priority and Preemption State | Enabled by default ------------------- | :---------------------------: | :----------------: -1.8 | alpha | no -1.9 | alpha | no -1.10 | alpha | no -1.11 | beta | yes -1.14 | stable | yes - -{{< warning >}}In a cluster where not all users are trusted, a -malicious user could create pods at the highest possible priorities, causing -other pods to be evicted/not get scheduled. To resolve this issue, -[ResourceQuota](/docs/concepts/policy/resource-quotas/) is -augmented to support Pod priority. An admin can create ResourceQuota for users -at specific priority levels, preventing them from creating pods at high -priorities. This feature is in beta since Kubernetes 1.12. -{{< /warning >}} - {{% /capture %}} {{% capture body %}} + +{{< warning >}} +In a cluster where not all users are trusted, a malicious user could create Pods +at the highest possible priorities, causing other Pods to be evicted/not get +scheduled. +An administrator can use ResourceQuota to prevent users from creating pods at +high priorities. + +See [limit Priority Class consumption by default](/docs/concepts/policy/resource-quotas/#limit-priority-class-consumption-by-default) +for details. +{{< /warning >}} + ## How to use priority and preemption -To use priority and preemption in Kubernetes 1.11 and later, follow these steps: +To use priority and preemption: 1. Add one or more [PriorityClasses](#priorityclass). @@ -77,21 +60,20 @@ Pods. ## How to disable preemption -{{< note >}} -In Kubernetes 1.12+, critical pods rely on scheduler preemption to be scheduled -when a cluster is under resource pressure. For this reason, it is not -recommended to disable preemption. -{{< /note >}} +{{< caution >}} +Critical pods rely on scheduler preemption to be scheduled when a cluster +is under resource pressure. For this reason, it is not recommended to +disable preemption. +{{< /caution >}} {{< note >}} -In Kubernetes 1.15 and later, -if the feature `NonPreemptingPriority` is enabled, +In Kubernetes 1.15 and later, if the feature `NonPreemptingPriority` is enabled, PriorityClasses have the option to set `preemptionPolicy: Never`. This will prevent pods of that PriorityClass from preempting other pods. {{< /note >}} -In Kubernetes 1.11 and later, preemption is controlled by a kube-scheduler flag -`disablePreemption`, which is set to `false` by default. +Preemption is controlled by a kube-scheduler flag `disablePreemption`, which is +set to `false` by default. If you want to disable preemption despite the above note, you can set `disablePreemption` to `true`. @@ -240,12 +222,12 @@ spec: ### Effect of Pod priority on scheduling order -In Kubernetes 1.9 and later, when Pod priority is enabled, scheduler orders -pending Pods by their priority and a pending Pod is placed ahead of other -pending Pods with lower priority in the scheduling queue. As a result, the -higher priority Pod may be scheduled sooner than Pods with lower priority if its -scheduling requirements are met. If such Pod cannot be scheduled, scheduler will -continue and tries to schedule other lower priority Pods. +When Pod priority is enabled, the scheduler orders pending Pods by +their priority and a pending Pod is placed ahead of other pending Pods +with lower priority in the scheduling queue. As a result, the higher +priority Pod may be scheduled sooner than Pods with lower priority if +its scheduling requirements are met. If such Pod cannot be scheduled, +scheduler will continue and tries to schedule other lower priority Pods. ## Preemption @@ -291,12 +273,12 @@ point that scheduler preempts victims and the time that Pod P is scheduled. In order to minimize this gap, one can set graceful termination period of lower priority Pods to zero or a small number. -#### PodDisruptionBudget is supported, but not guaranteed! +#### PodDisruptionBudget is supported, but not guaranteed A [Pod Disruption Budget (PDB)](/docs/concepts/workloads/pods/disruptions/) allows application owners to limit the number of Pods of a replicated application -that are down simultaneously from voluntary disruptions. Kubernetes 1.9 supports -PDB when preempting Pods, but respecting PDB is best effort. The Scheduler tries +that are down simultaneously from voluntary disruptions. Kubernetes supports +PDB when preempting Pods, but respecting PDB is best effort. The scheduler tries to find victims whose PDB are not violated by preemption, but if no such victims are found, preemption will still happen, and lower priority Pods will be removed despite their PDBs being violated. @@ -347,28 +329,23 @@ gone, and Pod P could possibly be scheduled on Node N. We may consider adding cross Node preemption in future versions if there is enough demand and if we find an algorithm with reasonable performance. -## Debugging Pod Priority and Preemption +## Troubleshooting -Pod Priority and Preemption is a major feature that could potentially disrupt -Pod scheduling if it has bugs. +Pod priority and pre-emption can have unwanted side effects. Here are some +examples of potential problems and ways to deal with them. -### Potential problems caused by Priority and Preemption - -The followings are some of the potential problems that could be caused by bugs -in the implementation of the feature. This list is not exhaustive. - -#### Pods are preempted unnecessarily +### Pods are preempted unnecessarily Preemption removes existing Pods from a cluster under resource pressure to make -room for higher priority pending Pods. If a user gives high priorities to -certain Pods by mistake, these unintentional high priority Pods may cause -preemption in the cluster. As mentioned above, Pod priority is specified by -setting the `priorityClassName` field of `podSpec`. The integer value of +room for higher priority pending Pods. If you give high priorities to +certain Pods by mistake, these unintentionally high priority Pods may cause +preemption in your cluster. Pod priority is specified by setting the +`priorityClassName` field in the Pod's specification. The integer value for priority is then resolved and populated to the `priority` field of `podSpec`. -To resolve the problem, `priorityClassName` of the Pods must be changed to use -lower priority classes or should be left empty. Empty `priorityClassName` is -resolved to zero by default. +To address the problem, you can change the `priorityClassName` for those Pods +to use lower priority classes, or leave that field empty. An empty +`priorityClassName` is resolved to zero by default. When a Pod is preempted, there will be events recorded for the preempted Pod. Preemption should happen only when a cluster does not have enough resources for @@ -377,29 +354,31 @@ Pod (preemptor) is higher than the victim Pods. Preemption must not happen when there is no pending Pod, or when the pending Pods have equal or lower priority than the victims. If preemption happens in such scenarios, please file an issue. -#### Pods are preempted, but the preemptor is not scheduled +### Pods are preempted, but the preemptor is not scheduled When pods are preempted, they receive their requested graceful termination -period, which is by default 30 seconds, but it can be any different value as -specified in the PodSpec. If the victim Pods do not terminate within this period, -they are force-terminated. Once all the victims go away, the preemptor Pod can -be scheduled. +period, which is by default 30 seconds. If the victim Pods do not terminate within +this period, they are forcibly terminated. Once all the victims go away, the +preemptor Pod can be scheduled. While the preemptor Pod is waiting for the victims to go away, a higher priority -Pod may be created that fits on the same node. In this case, the scheduler will +Pod may be created that fits on the same Node. In this case, the scheduler will schedule the higher priority Pod instead of the preemptor. -In the absence of such a higher priority Pod, we expect the preemptor Pod to be -scheduled after the graceful termination period of the victims is over. +This is expected behavior: the Pod with the higher priority should take the place +of a Pod with a lower priority. Other controller actions, such as +[cluster autoscaling](/docs/tasks/administer-cluster/cluster-management/#cluster-autoscaling), +may eventually provide capacity to schedule the pending Pods. -#### Higher priority Pods are preempted before lower priority pods +### Higher priority Pods are preempted before lower priority pods -The scheduler tries to find nodes that can run a pending Pod and if no node is -found, it tries to remove Pods with lower priority from one node to make room -for the pending pod. If a node with low priority Pods is not feasible to run the -pending Pod, the scheduler may choose another node with higher priority Pods -(compared to the Pods on the other node) for preemption. The victims must still -have lower priority than the preemptor Pod. +The scheduler tries to find nodes that can run a pending Pod. If no node is +found, the scheduler tries to remove Pods with lower priority from an arbitrary +node in order to make room for the pending pod. +If a node with low priority Pods is not feasible to run the pending Pod, the scheduler +may choose another node with higher priority Pods (compared to the Pods on the +other node) for preemption. The victims must still have lower priority than the +preemptor Pod. When there are multiple nodes available for preemption, the scheduler tries to choose the node with a set of Pods with lowest priority. However, if such Pods @@ -407,13 +386,11 @@ have PodDisruptionBudget that would be violated if they are preempted then the scheduler may choose another node with higher priority Pods. When multiple nodes exist for preemption and none of the above scenarios apply, -we expect the scheduler to choose a node with the lowest priority. If that is -not the case, it may indicate a bug in the scheduler. +the scheduler chooses a node with the lowest priority. -## Interactions of Pod priority and QoS +## Interactions between Pod priority and quality of service {#interactions-of-pod-priority-and-qos} -Pod priority and -[QoS](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/node/resource-qos.md) +Pod priority and {{< glossary_tooltip text="QoS class" term_id="qos-class" >}} are two orthogonal features with few interactions and no default restrictions on setting the priority of a Pod based on its QoS classes. The scheduler's preemption logic does not consider QoS when choosing preemption targets. @@ -424,15 +401,20 @@ to schedule the preemptor Pod, or if the lowest priority Pods are protected by `PodDisruptionBudget`. The only component that considers both QoS and Pod priority is -[Kubelet out-of-resource eviction](/docs/tasks/administer-cluster/out-of-resource/). +[kubelet out-of-resource eviction](/docs/tasks/administer-cluster/out-of-resource/). The kubelet ranks Pods for eviction first by whether or not their usage of the starved resource exceeds requests, then by Priority, and then by the consumption of the starved compute resource relative to the Pods’ scheduling requests. See -[Evicting end-user pods](/docs/tasks/administer-cluster/out-of-resource/#evicting-end-user-pods) -for more details. Kubelet out-of-resource eviction does not evict Pods whose +[evicting end-user pods](/docs/tasks/administer-cluster/out-of-resource/#evicting-end-user-pods) +for more details. + +kubelet out-of-resource eviction does not evict Pods wheir their usage does not exceed their requests. If a Pod with lower priority is not exceeding its requests, it won't be evicted. Another Pod with higher priority that exceeds its requests may be evicted. {{% /capture %}} +{{% capture whatsnext %}} +* Read about using ResourceQuotas in connection with PriorityClasses: [limit Priority Class consumption by default](/docs/concepts/policy/resource-quotas/#limit-priority-class-consumption-by-default) +{{% /capture %}} From 8ea63bde2f2198726dcc63622d257ac79299786c Mon Sep 17 00:00:00 2001 From: Qiming Teng Date: Mon, 16 Mar 2020 13:48:35 +0800 Subject: [PATCH 087/140] Drop 'federation' from cluster admin concepts (#19525) Almost all links on the federation page are gone. Federation related contents should be cleanly removed. --- .../cluster-administration/federation.md | 186 ------------------ 1 file changed, 186 deletions(-) delete mode 100644 content/en/docs/concepts/cluster-administration/federation.md diff --git a/content/en/docs/concepts/cluster-administration/federation.md b/content/en/docs/concepts/cluster-administration/federation.md deleted file mode 100644 index e26aef3fa8..0000000000 --- a/content/en/docs/concepts/cluster-administration/federation.md +++ /dev/null @@ -1,186 +0,0 @@ ---- -title: Federation -content_template: templates/concept -weight: 80 ---- - -{{% capture overview %}} - -{{< deprecationfilewarning >}} -{{< include "federation-deprecation-warning-note.md" >}} -{{< /deprecationfilewarning >}} - -This page explains why and how to manage multiple Kubernetes clusters using -federation. -{{% /capture %}} - -{{% capture body %}} -## Why federation - -Federation makes it easy to manage multiple clusters. It does so by providing 2 -major building blocks: - - * Sync resources across clusters: Federation provides the ability to keep - resources in multiple clusters in sync. For example, you can ensure that the same deployment exists in multiple clusters. - * Cross cluster discovery: Federation provides the ability to auto-configure DNS servers and load balancers with backends from all clusters. For example, you can ensure that a global VIP or DNS record can be used to access backends from multiple clusters. - -Some other use cases that federation enables are: - -* High Availability: By spreading load across clusters and auto configuring DNS - servers and load balancers, federation minimises the impact of cluster - failure. -* Avoiding provider lock-in: By making it easier to migrate applications across - clusters, federation prevents cluster provider lock-in. - - -Federation is not helpful unless you have multiple clusters. Some of the reasons -why you might want multiple clusters are: - -* Low latency: Having clusters in multiple regions minimises latency by serving - users from the cluster that is closest to them. -* Fault isolation: It might be better to have multiple small clusters rather - than a single large cluster for fault isolation (for example: multiple - clusters in different availability zones of a cloud provider). -* Scalability: There are scalability limits to a single kubernetes cluster (this - should not be the case for most users. For more details: - [Kubernetes Scaling and Performance Goals](https://git.k8s.io/community/sig-scalability/goals.md)). -* [Hybrid cloud](#hybrid-cloud-capabilities): You can have multiple clusters on different cloud providers or - on-premises data centers. - -### Caveats - -While there are a lot of attractive use cases for federation, there are also -some caveats: - -* Increased network bandwidth and cost: The federation control plane watches all - clusters to ensure that the current state is as expected. This can lead to - significant network cost if the clusters are running in different regions on - a cloud provider or on different cloud providers. -* Reduced cross cluster isolation: A bug in the federation control plane can - impact all clusters. This is mitigated by keeping the logic in federation - control plane to a minimum. It mostly delegates to the control plane in - kubernetes clusters whenever it can. The design and implementation also errs - on the side of safety and avoiding multi-cluster outage. -* Maturity: The federation project is relatively new and is not very mature. - Not all resources are available and many are still alpha. [Issue - 88](https://github.com/kubernetes/federation/issues/88) enumerates - known issues with the system that the team is busy solving. - -### Hybrid cloud capabilities - -Federations of Kubernetes Clusters can include clusters running in -different cloud providers (e.g. Google Cloud, AWS), and on-premises -(e.g. on OpenStack). [Kubefed](/docs/tasks/federation/set-up-cluster-federation-kubefed/) is the recommended way to deploy federated clusters. - -Thereafter, your [API resources](#api-resources) can span different clusters -and cloud providers. - -## Setting up federation - -To be able to federate multiple clusters, you first need to set up a federation -control plane. -Follow the [setup guide](/docs/tutorials/federation/set-up-cluster-federation-kubefed/) to set up the -federation control plane. - -## API resources - -Once you have the control plane set up, you can start creating federation API -resources. -The following guides explain some of the resources in detail: - -* [Cluster](/docs/tasks/federation/administer-federation/cluster/) -* [ConfigMap](/docs/tasks/federation/administer-federation/configmap/) -* [DaemonSets](/docs/tasks/federation/administer-federation/daemonset/) -* [Deployment](/docs/tasks/federation/administer-federation/deployment/) -* [Events](/docs/tasks/federation/administer-federation/events/) -* [Hpa](/docs/tasks/federation/administer-federation/hpa/) -* [Ingress](/docs/tasks/federation/administer-federation/ingress/) -* [Jobs](/docs/tasks/federation/administer-federation/job/) -* [Namespaces](/docs/tasks/federation/administer-federation/namespaces/) -* [ReplicaSets](/docs/tasks/federation/administer-federation/replicaset/) -* [Secrets](/docs/tasks/federation/administer-federation/secret/) -* [Services](/docs/concepts/cluster-administration/federation-service-discovery/) - - -The [API reference docs](/docs/reference/federation/) list all the -resources supported by federation apiserver. - -## Cascading deletion - -Kubernetes version 1.6 includes support for cascading deletion of federated -resources. With cascading deletion, when you delete a resource from the -federation control plane, you also delete the corresponding resources in all underlying clusters. - -Cascading deletion is not enabled by default when using the REST API. To enable -it, set the option `DeleteOptions.orphanDependents=false` when you delete a -resource from the federation control plane using the REST API. Using `kubectl -delete` -enables cascading deletion by default. You can disable it by running `kubectl -delete --cascade=false` - -Note: Kubernetes version 1.5 included cascading deletion support for a subset of -federation resources. - -## Scope of a single cluster - -On IaaS providers such as Google Compute Engine or Amazon Web Services, a VM exists in a -[zone](https://cloud.google.com/compute/docs/zones) or [availability -zone](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html). -We suggest that all the VMs in a Kubernetes cluster should be in the same availability zone, because: - - - compared to having a single global Kubernetes cluster, there are fewer single-points of failure. - - compared to a cluster that spans availability zones, it is easier to reason about the availability properties of a - single-zone cluster. - - when the Kubernetes developers are designing the system (e.g. making assumptions about latency, bandwidth, or - correlated failures) they are assuming all the machines are in a single data center, or otherwise closely connected. - -It is recommended to run fewer clusters with more VMs per availability zone; but it is possible to run multiple clusters per availability zones. - -Reasons to prefer fewer clusters per availability zone are: - - - improved bin packing of Pods in some cases with more nodes in one cluster (less resource fragmentation). - - reduced operational overhead (though the advantage is diminished as ops tooling and processes mature). - - reduced costs for per-cluster fixed resource costs, for example apiserver VMs (but small as a percentage - of overall cluster cost for medium to large clusters). - -Reasons to have multiple clusters include: - - - strict security policies requiring isolation of one class of work from another (but, see Partitioning Clusters - below). - - test clusters to canary new Kubernetes releases or other cluster software. - -## Selecting the right number of clusters - -The selection of the number of Kubernetes clusters may be a relatively static choice, only revisited occasionally. -By contrast, the number of nodes in a cluster and the number of pods in a service may change frequently according to -load and growth. - -To pick the number of clusters, first, decide which regions you need to be in to have adequate latency to all your end users, for services that will run -on Kubernetes (if you use a Content Distribution Network, the latency requirements for the CDN-hosted content need not -be considered). Legal issues might influence this as well. For example, a company with a global customer base might decide to have clusters in US, EU, AP, and SA regions. -Call the number of regions to be in `R`. - -Second, decide how many clusters should be able to be unavailable at the same time, while still being available. Call -the number that can be unavailable `U`. If you are not sure, then 1 is a fine choice. - -If it is allowable for load-balancing to direct traffic to any region in the event of a cluster failure, then -you need at least the larger of `R` or `U + 1` clusters. If it is not (e.g. you want to ensure low latency for all -users in the event of a cluster failure), then you need to have `R * (U + 1)` clusters -(`U + 1` in each of `R` regions). In any case, try to put each cluster in a different zone. - -Finally, if any of your clusters would need more than the maximum recommended number of nodes for a Kubernetes cluster, then -you may need even more clusters. Kubernetes v1.3 supports clusters up to 1000 nodes in size. Kubernetes v1.8 supports -clusters up to 5000 nodes. See [Building Large Clusters](/docs/setup/best-practices/cluster-large/) for more guidance. - -{{% /capture %}} - -{{% capture whatsnext %}} -* Learn more about the [Federation - proposal](https://github.com/kubernetes/community/blob/{{< param "githubbranch" >}}/contributors/design-proposals/multicluster/federation.md). -* See this [setup guide](/docs/tutorials/federation/set-up-cluster-federation-kubefed/) for cluster federation. -* See this [Kubecon2016 talk on federation](https://www.youtube.com/watch?v=pq9lbkmxpS8) -* See this [Kubecon2017 Europe update on federation](https://www.youtube.com/watch?v=kwOvOLnFYck) -* See this [Kubecon2018 Europe update on sig-multicluster](https://www.youtube.com/watch?v=vGZo5DaThQU) -* See this [Kubecon2018 Europe Federation-v2 prototype presentation](https://youtu.be/q27rbaX5Jis?t=7m20s) -* See this [Federation-v2 Userguide](https://github.com/kubernetes-sigs/federation-v2/blob/master/docs/userguide.md) -{{% /capture %}} From 6d37f8ccfc779aedda2b4c632823e83e4ba8fe55 Mon Sep 17 00:00:00 2001 From: RA489 Date: Mon, 16 Mar 2020 11:20:35 +0530 Subject: [PATCH 088/140] improvements to the br_netfilter documentation for kubeadm (#19610) --- .../tools/kubeadm/create-cluster-kubeadm.md | 10 ------- .../tools/kubeadm/install-kubeadm.md | 29 ++++++++++++------- 2 files changed, 18 insertions(+), 21 deletions(-) diff --git a/content/en/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm.md b/content/en/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm.md index 889b85f64d..9d35fa2c5a 100644 --- a/content/en/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm.md +++ b/content/en/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm.md @@ -344,10 +344,6 @@ Please refer to this installation guide: [Contiv-VPP Manual Installation](https: For `flannel` to work correctly, you must pass `--pod-network-cidr=10.244.0.0/16` to `kubeadm init`. -Set `/proc/sys/net/bridge/bridge-nf-call-iptables` to `1` by running `sysctl net.bridge.bridge-nf-call-iptables=1` -to pass bridged IPv4 traffic to iptables' chains. This is a requirement for some CNI plugins to work, for more information -please see [Network Plugin Requirements](/docs/concepts/cluster-administration/network-plugins/#network-plugin-requirements). - Make sure that your firewall rules allow UDP ports 8285 and 8472 traffic for all hosts participating in the overlay network. The [Firewall](https://coreos.com/flannel/docs/latest/troubleshooting.html#firewalls) section of Flannel's troubleshooting guide explains about this in more detail. Flannel works on `amd64`, `arm`, `arm64`, `ppc64le` and `s390x` architectures under Linux. @@ -362,9 +358,6 @@ For more information about `flannel`, see [the CoreOS flannel repository on GitH {{% /tab %}} {{% tab name="Kube-router" %}} -Set `/proc/sys/net/bridge/bridge-nf-call-iptables` to `1` by running `sysctl net.bridge.bridge-nf-call-iptables=1` -to pass bridged IPv4 traffic to iptables' chains. This is a requirement for some CNI plugins to work, for more information -please see [Network Plugin Requirements](/docs/concepts/cluster-administration/network-plugins/#network-plugin-requirements). Kube-router relies on kube-controller-manager to allocate Pod CIDR for the nodes. Therefore, use `kubeadm init` with the `--pod-network-cidr` flag. @@ -374,9 +367,6 @@ For information on using the `kubeadm` tool to set up a Kubernetes cluster with {{% /tab %}} {{% tab name="Weave Net" %}} -Set `/proc/sys/net/bridge/bridge-nf-call-iptables` to `1` by running `sysctl net.bridge.bridge-nf-call-iptables=1` -to pass bridged IPv4 traffic to iptables' chains. This is a requirement for some CNI plugins to work, for more information -please see [Network Plugin Requirements](/docs/concepts/cluster-administration/network-plugins/#network-plugin-requirements). For more information on setting up your Kubernetes cluster with Weave Net, please see [Integrating Kubernetes via the Addon]((https://www.weave.works/docs/net/latest/kube-addon/). diff --git a/content/en/docs/setup/production-environment/tools/kubeadm/install-kubeadm.md b/content/en/docs/setup/production-environment/tools/kubeadm/install-kubeadm.md index 09aa01de0c..8ff967e217 100644 --- a/content/en/docs/setup/production-environment/tools/kubeadm/install-kubeadm.md +++ b/content/en/docs/setup/production-environment/tools/kubeadm/install-kubeadm.md @@ -51,6 +51,23 @@ may [fail](https://github.com/kubernetes/kubeadm/issues/31). If you have more than one network adapter, and your Kubernetes components are not reachable on the default route, we recommend you add IP route(s) so Kubernetes cluster addresses go via the appropriate adapter. +## Letting iptables see bridged traffic + +As a requirement for your Linux Node's iptables to correctly see bridged traffic, you should ensure `net.bridge.bridge-nf-call-iptables` is set to 1 in your `sysctl` config, e.g. + +```bash +cat < /etc/sysctl.d/k8s.conf +net.bridge.bridge-nf-call-ip6tables = 1 +net.bridge.bridge-nf-call-iptables = 1 +EOF +sysctl --system +``` + +Make sure that the `br_netfilter` module is loaded before this step. This can be done by running `lsmod | grep br_netfilter`. To load it explicitly call `modprobe br_netfilter`. + +For more details please see the [Network Plugin Requirements](https://kubernetes.io/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins/#network-plugin-requirements) page. + + ## Ensure iptables tooling does not use the nftables backend In Linux, nftables is available as a modern replacement for the kernel's iptables subsystem. The @@ -230,17 +247,7 @@ systemctl enable --now kubelet - Setting SELinux in permissive mode by running `setenforce 0` and `sed ...` effectively disables it. This is required to allow containers to access the host filesystem, which is needed by pod networks for example. You have to do this until SELinux support is improved in the kubelet. - - Some users on RHEL/CentOS 7 have reported issues with traffic being routed incorrectly due to iptables being bypassed. You should ensure - `net.bridge.bridge-nf-call-iptables` is set to 1 in your `sysctl` config, e.g. - - ```bash - cat < /etc/sysctl.d/k8s.conf - net.bridge.bridge-nf-call-ip6tables = 1 - net.bridge.bridge-nf-call-iptables = 1 - EOF - sysctl --system - ``` - - Make sure that the `br_netfilter` module is loaded before this step. This can be done by running `lsmod | grep br_netfilter`. To load it explicitly call `modprobe br_netfilter`. + {{% /tab %}} {{% tab name="Container Linux" %}} Install CNI plugins (required for most pod network): From 26d8e0c539a327f086c1840ef31c3b90335ad814 Mon Sep 17 00:00:00 2001 From: Jie Shen Date: Mon, 16 Mar 2020 13:54:35 +0800 Subject: [PATCH 089/140] Change languge type for sample in multiple-zones.md (#19436) --- content/en/docs/setup/best-practices/multiple-zones.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/setup/best-practices/multiple-zones.md b/content/en/docs/setup/best-practices/multiple-zones.md index b01cbceedf..ba58df028f 100644 --- a/content/en/docs/setup/best-practices/multiple-zones.md +++ b/content/en/docs/setup/best-practices/multiple-zones.md @@ -188,7 +188,7 @@ kubernetes-minion-wf8i Ready 2m v1.13.0 Create a volume using the dynamic volume creation (only PersistentVolumes are supported for zone affinity): -```json +```bash kubectl apply -f - < Date: Mon, 16 Mar 2020 13:56:36 +0800 Subject: [PATCH 090/140] Enhanced FileOrCreate description (#19442) Signed-off-by: wawa0210 --- content/en/docs/concepts/storage/volumes.md | 32 +++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/content/en/docs/concepts/storage/volumes.md b/content/en/docs/concepts/storage/volumes.md index cbace11392..ac864327e9 100644 --- a/content/en/docs/concepts/storage/volumes.md +++ b/content/en/docs/concepts/storage/volumes.md @@ -605,6 +605,38 @@ spec: type: Directory ``` +{{< caution >}} +It should be noted that the `FileOrCreate` mode does not create the parent directory of the file. If the parent directory of the mounted file does not exist, the pod fails to start. To ensure that this mode works, you can try to mount directories and files separately, as shown below. +{{< /caution >}} + +#### Example Pod FileOrCreate + +```yaml +apiVersion: v1 +kind: Pod +metadata: + name: test-webserver +spec: + containers: + - name: test-webserver + image: k8s.gcr.io/test-webserver:latest + volumeMounts: + - mountPath: /var/local/aaa + name: mydir + - mountPath: /var/local/aaa/1.txt + name: myfile + volumes: + - name: mydir + hostPath: + # Ensure the file directory is created. + path: /var/local/aaa + type: DirectoryOrCreate + - name: myfile + hostPath: + path: /var/local/aaa/1.txt + type: FileOrCreate +``` + ### iscsi {#iscsi} An `iscsi` volume allows an existing iSCSI (SCSI over IP) volume to be mounted From 19c883dd827c8023553d462ff55d7ab1706d63a8 Mon Sep 17 00:00:00 2001 From: Tim Bannister Date: Mon, 16 Mar 2020 05:58:35 +0000 Subject: [PATCH 091/140] Admit that some objects don't have a spec (#18050) Whilst most objects have a spec, it will confuse readers who come to this page from a page about an object that doesn't (eg Secret, Namespace, Role). --- .../kubernetes-objects.md | 21 +++++++++++++++++-- 1 file changed, 19 insertions(+), 2 deletions(-) diff --git a/content/en/docs/concepts/overview/working-with-objects/kubernetes-objects.md b/content/en/docs/concepts/overview/working-with-objects/kubernetes-objects.md index 0ce8d81bf6..7364596306 100644 --- a/content/en/docs/concepts/overview/working-with-objects/kubernetes-objects.md +++ b/content/en/docs/concepts/overview/working-with-objects/kubernetes-objects.md @@ -26,10 +26,27 @@ To work with Kubernetes objects--whether to create, modify, or delete them--you' ### Object Spec and Status -Every Kubernetes object includes two nested object fields that govern the object's configuration: the object *spec* and the object *status*. The *spec*, which you must provide, describes your desired state for the object--the characteristics that you want the object to have. The *status* describes the *actual state* of the object, and is supplied and updated by the Kubernetes system. At any given time, the Kubernetes Control Plane actively manages an object's actual state to match the desired state you supplied. +Almost every Kubernetes object includes two nested object fields that govern +the object's configuration: the object *`spec`* and the object *`status`*. +For objects that have a `spec`, you have to set this when you create the object, +providing a description of the characteristics you want the resource to have: +its _desired state_. +The `status` describes the _current state_ of the object, supplied and updated +by the Kubernetes and its components. The Kubernetes +{{< glossary_tooltip text="control plane" term_id="control-plane" >}} continually +and actively manages every object's actual state to match the desired state you +supplied. -For example, a Kubernetes Deployment is an object that can represent an application running on your cluster. When you create the Deployment, you might set the Deployment spec to specify that you want three replicas of the application to be running. The Kubernetes system reads the Deployment spec and starts three instances of your desired application--updating the status to match your spec. If any of those instances should fail (a status change), the Kubernetes system responds to the difference between spec and status by making a correction--in this case, starting a replacement instance. +For example: in Kubernetes, a Deployment is an object that can represent an +application running on your cluster. When you create the Deployment, you +might set the Deployment `spec` to specify that you want three replicas of +the application to be running. The Kubernetes system reads the Deployment +spec and starts three instances of your desired application--updating +the status to match your spec. If any of those instances should fail +(a status change), the Kubernetes system responds to the difference +between spec and status by making a correction--in this case, starting +a replacement instance. For more information on the object spec, status, and metadata, see the [Kubernetes API Conventions](https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md). From a794297373da9a023ccbc93bcfe81947c23e04f5 Mon Sep 17 00:00:00 2001 From: zhouya0 <50729202+zhouya0@users.noreply.github.com> Date: Mon, 16 Mar 2020 14:00:35 +0800 Subject: [PATCH 092/140] Fix pod anti-affinity topology key (#19587) --- content/en/docs/concepts/configuration/assign-pod-node.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/concepts/configuration/assign-pod-node.md b/content/en/docs/concepts/configuration/assign-pod-node.md index 9224a1be11..5f86b37025 100644 --- a/content/en/docs/concepts/configuration/assign-pod-node.md +++ b/content/en/docs/concepts/configuration/assign-pod-node.md @@ -229,7 +229,7 @@ for performance and security reasons, there are some constraints on topologyKey: 1. For affinity and for `requiredDuringSchedulingIgnoredDuringExecution` pod anti-affinity, empty `topologyKey` is not allowed. 2. For `requiredDuringSchedulingIgnoredDuringExecution` pod anti-affinity, the admission controller `LimitPodHardAntiAffinityTopology` was introduced to limit `topologyKey` to `kubernetes.io/hostname`. If you want to make it available for custom topologies, you may modify the admission controller, or simply disable it. -3. For `preferredDuringSchedulingIgnoredDuringExecution` pod anti-affinity, empty `topologyKey` is interpreted as "all topologies" ("all topologies" here is now limited to the combination of `kubernetes.io/hostname`, `failure-domain.beta.kubernetes.io/zone` and `failure-domain.beta.kubernetes.io/region`). +3. For `preferredDuringSchedulingIgnoredDuringExecution` pod anti-affinity, empty `topologyKey` is not allowed. 4. Except for the above cases, the `topologyKey` can be any legal label-key. In addition to `labelSelector` and `topologyKey`, you can optionally specify a list `namespaces` From fa286e842ee4f7fbc58fdd63bcaf785de354199f Mon Sep 17 00:00:00 2001 From: Johnny Lim Date: Mon, 16 Mar 2020 15:02:35 +0900 Subject: [PATCH 093/140] Fix typo (#19651) --- content/en/docs/tutorials/stateless-application/guestbook.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/tutorials/stateless-application/guestbook.md b/content/en/docs/tutorials/stateless-application/guestbook.md index 979a14198a..f1f93dd988 100644 --- a/content/en/docs/tutorials/stateless-application/guestbook.md +++ b/content/en/docs/tutorials/stateless-application/guestbook.md @@ -79,7 +79,7 @@ Replace POD-NAME with the name of your Pod. ### Creating the Redis Master Service -The guestbook applications needs to communicate to the Redis master to write its data. You need to apply a [Service](/docs/concepts/services-networking/service/) to proxy the traffic to the Redis master Pod. A Service defines a policy to access the Pods. +The guestbook application needs to communicate to the Redis master to write its data. You need to apply a [Service](/docs/concepts/services-networking/service/) to proxy the traffic to the Redis master Pod. A Service defines a policy to access the Pods. {{< codenew file="application/guestbook/redis-master-service.yaml" >}} From 08c535577b1119966e6e730620bd093c04c3e370 Mon Sep 17 00:00:00 2001 From: CJ Cullen Date: Sun, 15 Mar 2020 23:04:35 -0700 Subject: [PATCH 094/140] Add to the PSP warning about PVs that hostPath PVs cannot be made read-only. (#19504) --- content/en/docs/concepts/policy/pod-security-policy.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/content/en/docs/concepts/policy/pod-security-policy.md b/content/en/docs/concepts/policy/pod-security-policy.md index 44c4a41e7a..f482c5efb2 100644 --- a/content/en/docs/concepts/policy/pod-security-policy.md +++ b/content/en/docs/concepts/policy/pod-security-policy.md @@ -421,8 +421,10 @@ The **recommended minimum set** of allowed volumes for new PSPs are: - projected {{< warning >}} -PodSecurityPolicy does not limit the types of `PersistentVolume` objects that may be referenced by a `PersistentVolumeClaim`. -Only trusted users should be granted permission to create `PersistentVolume` objects. +PodSecurityPolicy does not limit the types of `PersistentVolume` objects that +may be referenced by a `PersistentVolumeClaim`, and hostPath type +`PersistentVolumes` do not support read-only access mode. Only trusted users +should be granted permission to create `PersistentVolume` objects. {{< /warning >}} **FSGroup** - Controls the supplemental group applied to some volumes. From f9aa0bc0e539bc895a771c06f6c3d49627d4cc27 Mon Sep 17 00:00:00 2001 From: Tim Bannister Date: Mon, 16 Mar 2020 06:06:35 +0000 Subject: [PATCH 095/140] =?UTF-8?q?Revise=20=E2=80=9CWhat=20is=20Kubernete?= =?UTF-8?q?s=E2=80=9D=20(#19534)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Expand list of places Kubernetes can run As well as GKE, there are AKS, EKS, and more. * Add page description * Link to blog post about Borg The blog post has (a better) link to Google's paper about Borg. Because of the historical relevance of that paper and that blog article to the history of Kubernetes, I think it's OK to link there. Linking to the blog article provides a less academic introduction to the history of Kubernetes, with the option to click through and learn things in detail. --- content/en/docs/concepts/overview/what-is-kubernetes.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/content/en/docs/concepts/overview/what-is-kubernetes.md b/content/en/docs/concepts/overview/what-is-kubernetes.md index 6849083ccf..4dbb673382 100644 --- a/content/en/docs/concepts/overview/what-is-kubernetes.md +++ b/content/en/docs/concepts/overview/what-is-kubernetes.md @@ -3,6 +3,8 @@ reviewers: - bgrant0607 - mikedanese title: What is Kubernetes +description: > + Kubernetes is a portable, extensible, open-source platform for managing containerized workloads and services, that facilitates both declarative configuration and automation. It has a large, rapidly growing ecosystem. Kubernetes services, support, and tools are widely available. content_template: templates/concept weight: 10 card: @@ -17,9 +19,10 @@ This page is an overview of Kubernetes. {{% capture body %}} Kubernetes is a portable, extensible, open-source platform for managing containerized workloads and services, that facilitates both declarative configuration and automation. It has a large, rapidly growing ecosystem. Kubernetes services, support, and tools are widely available. -The name Kubernetes originates from Greek, meaning helmsman or pilot. Google open-sourced the Kubernetes project in 2014. Kubernetes builds upon a [decade and a half of experience that Google has with running production workloads at scale](https://ai.google/research/pubs/pub43438), combined with best-of-breed ideas and practices from the community. +The name Kubernetes originates from Greek, meaning helmsman or pilot. Google open-sourced the Kubernetes project in 2014. Kubernetes combines [over 15 years of Google's experience](/blog/2015/04/borg-predecessor-to-kubernetes/) running production workloads at scale with best-of-breed ideas and practices from the community. ## Going back in time + Let's take a look at why Kubernetes is so useful by going back in time. ![Deployment evolution](/images/docs/Container_Evolution.svg) @@ -42,7 +45,7 @@ Containers have become popular because they provide extra benefits, such as: * Dev and Ops separation of concerns: create application container images at build/release time rather than deployment time, thereby decoupling applications from infrastructure. * Observability not only surfaces OS-level information and metrics, but also application health and other signals. * Environmental consistency across development, testing, and production: Runs the same on a laptop as it does in the cloud. -* Cloud and OS distribution portability: Runs on Ubuntu, RHEL, CoreOS, on-prem, Google Kubernetes Engine, and anywhere else. +* Cloud and OS distribution portability: Runs on Ubuntu, RHEL, CoreOS, on-premises, on major public clouds, and anywhere else. * Application-centric management: Raises the level of abstraction from running an OS on virtual hardware to running an application on an OS using logical resources. * Loosely coupled, distributed, elastic, liberated micro-services: applications are broken into smaller, independent pieces and can be deployed and managed dynamically – not a monolithic stack running on one big single-purpose machine. * Resource isolation: predictable application performance. From 9e5b981f800cc22e34a5fc4ab0bc604e6f10152a Mon Sep 17 00:00:00 2001 From: Tim Bannister Date: Mon, 16 Mar 2020 06:08:35 +0000 Subject: [PATCH 096/140] Add description (#19538) --- content/en/docs/home/_index.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/content/en/docs/home/_index.md b/content/en/docs/home/_index.md index 6ce04e800b..692f10dbef 100644 --- a/content/en/docs/home/_index.md +++ b/content/en/docs/home/_index.md @@ -16,6 +16,8 @@ menu: weight: 20 post: >

    Learn how to use Kubernetes with conceptual, tutorial, and reference documentation. You can even help contribute to the docs!

    +description: > + Kubernetes is an open source container orchestration engine for automating deployment, scaling, and management of containerized applications. The open source project is hosted by the Cloud Native Computing Foundation. overview: > Kubernetes is an open source container orchestration engine for automating deployment, scaling, and management of containerized applications. The open source project is hosted by the Cloud Native Computing Foundation (CNCF). cards: From 1f958e0744a3b56f5c4daa6d29029dea8b3ca7a7 Mon Sep 17 00:00:00 2001 From: Anastas Dancha Date: Mon, 16 Mar 2020 09:12:35 +0300 Subject: [PATCH 097/140] updating cheatsheet.md with command to get containerIDs of all initContainer (#19243) * adding cheatsheet.md adding kubectl command to get containerIDs of all initContainers for every Pod in the cluster helps to identify stopped container which should not be removed, when running manual cleanup of stopped containers on K8s cluster nodes * adding more context for finding initContainer IDs * removing reference to github issue Signed-off-by: Anastas Dancha --- content/en/docs/reference/kubectl/cheatsheet.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/content/en/docs/reference/kubectl/cheatsheet.md b/content/en/docs/reference/kubectl/cheatsheet.md index 38d723e981..8401d00ecf 100644 --- a/content/en/docs/reference/kubectl/cheatsheet.md +++ b/content/en/docs/reference/kubectl/cheatsheet.md @@ -191,6 +191,10 @@ JSONPATH='{range .items[*]}{@.metadata.name}:{range @.status.conditions[*]}{@.ty # List all Secrets currently in use by a pod kubectl get pods -o json | jq '.items[].spec.containers[].env[]?.valueFrom.secretKeyRef.name' | grep -v null | sort | uniq +# List all containerIDs of initContainer of all pods +# Helpful when cleaning up stopped containers, while avoiding removal of initContainers. +kubectl get pods --all-namespaces -o jsonpath='{range .items[*].status.initContainerStatuses[*]}{.containerID}{"\n"}{end}' | cut -d/ -f3 + # List Events sorted by timestamp kubectl get events --sort-by=.metadata.creationTimestamp From 933975324ce6836c61c1263ad76a5dec89c447ca Mon Sep 17 00:00:00 2001 From: Nguyen Quang Huy Date: Mon, 16 Mar 2020 13:22:35 +0700 Subject: [PATCH 098/140] Add Vietnamese code of conduct (#19020) * Add Vietnamese code of conduct Add Vietnamese code of conduct page to website * Add CNCF code of conduct Add CNCF code of conduct translation --- content/vi/community/_index.html | 236 ++++++++++++++++++ content/vi/community/code-of-conduct.md | 26 ++ content/vi/community/static/README.md | 1 + .../community/static/cncf-code-of-conduct.md | 31 +++ 4 files changed, 294 insertions(+) create mode 100644 content/vi/community/_index.html create mode 100644 content/vi/community/code-of-conduct.md create mode 100644 content/vi/community/static/README.md create mode 100644 content/vi/community/static/cncf-code-of-conduct.md diff --git a/content/vi/community/_index.html b/content/vi/community/_index.html new file mode 100644 index 0000000000..142ee6477a --- /dev/null +++ b/content/vi/community/_index.html @@ -0,0 +1,236 @@ +--- +title: Community +layout: basic +cid: community +--- + +
    +
    + Thư viện ảnh hội nghị Kubernetes + Thư viện ảnh hội nghị Kubernetes +
    + +
    +
    +

    Cộng đồng Kubernetes -- người dùng, người đóng góp, và văn hóa chúng ta xây dựng cùng nhau -- là một trong những lý do lớn nhất cho sự gia tăng nhanh chóng của dự án nguồn mở này. Văn hóa và giá trị của chúng ta tiếp tục phát triển và thay đổi khi dự án thay đổi và phát triển. Chúng ta làm việc cùng nhau hướng tới việc cải tiến liên tục dự án và cách chúng ta làm việc với nó. +

    Chúng ta là những người phát hiện các vấn đề, và xử lý các yêu cầu, tham dự các buổi họp SIG, các Kubernetes meetup, và KubeCon, ủng hộ việc áp dụng và đổi mới, chạy kubectl get pods, và đóng góp trong một ngàn cách quan trọng khác nhau. Đọc để tìm hiểu làm thế nào bạn có thể tham gia và trở thành một phần của cộng đồng tuyệt vời này.

    +
    +
    + +
    + +Quy tắc ứng xử      +Videos      +Thảo luận      +Sự kiện và gặp gỡ      +Tin tức + +
    +

    +
    +
    +
    + Thư viện ảnh hội nghị Kubernetes +
    + +
    + Thư viện ảnh hội nghị Kubernetes +
    + +
    + Thư viện ảnh hội nghị Kubernetes +
    + Thư viện ảnh hội nghị Kubernetes + + +
    + + + +
    +
    +

    +

    +

    Quy tắc ứng xử

    +Cộng đồng Kubernetes coi trọng sự tôn trọng và thực thi Quy tắc ứng xử trong tất cả các tương tác. Nếu bạn nhận thấy vi phạm Quy tắc ứng xử tại một sự kiện hoặc cuộc họp, trong Slack hoặc trong một cơ chế giao tiếp khác, hãy liên hệ với Ủy ban Quy tắc ứng xử Kubernetes tại conduct@kubernetes.io. Tất cả các báo cáo được giữ bí mật. Bạn có thể đọc về ủy ban ở đây. +
    + +

    + + +ĐỌC TIẾP + +     +
    +
    +
    + + + +
    +

    +

    +

    Videos

    + +
    Chúng tôi đang ở trên YouTube, rất nhiều video. Đăng ký cho một loạt các chủ đề.
    + +
    + +
    + + + + +
    +Xem những giờ làm việc hàng tháng ▶
    +     +
    + +
    + + +
    +Xem cuộc họp cộng đồng hàng tuần ▶ +
    +     +
    + +
    + + + + +
    +Xem cuộc nói chuyện của thành viên cộng đồng ▶ +
    + +     + +
    +
    +
    + + +
    +

    +

    +

    Thảo luận

    + +
    Chúng ta thảo luận rất nhiều. Tìm chúng tôi và tham gia cuộc trò chuyện trên bất kỳ  những nền tảng này.
    + +
    + +
    +Forum" + +forum ▶ + +
    +Các cuộc thảo luận kỹ thuật theo chủ đề có tài liệu kết nối, StackOverflow và  hơn nữa +
    +
    + +
    +Twitter + +twitter ▶ + +
    Thông báo thời gian thực của bài viết trên blog, sự kiện, tin tức, ý tưởng +
    +
    + +
    +GitHub + +github ▶ + +
    +Tất cả các dự án và theo dõi vấn đề, thêm cả mã khóa học +
    +
    + +
    +Stack Overflow + +stack overflow ▶ + +
    + Xử lý kỹ thuật for mọi  tình huống + +
    +
    + + + +
    +
    +
    +

    +

    +
    +

    Các sự kiện sắp tới

    + {{< upcoming-events >}} +
    +
    + +
    +
    +
    +

    Cộng đồng toàn cầu

    +Với hơn 150 meetup trên thế giới và đang phát triển, hãy đi tìm tại địa phương của bạn. Nếu không ở gần đó, hãy nhận chi phí và tạo của riêng bạn. +
    + +
    +TÌM MEETUP +
    +     + +
    +
    + + + + +
    +

    +

    +

    Bản tin gần đây

    + +
    +
    + +
    + +
    +



    +
    + +
    diff --git a/content/vi/community/code-of-conduct.md b/content/vi/community/code-of-conduct.md new file mode 100644 index 0000000000..7dfdf3364e --- /dev/null +++ b/content/vi/community/code-of-conduct.md @@ -0,0 +1,26 @@ +--- +title: Community +layout: basic +cid: community +css: /css/community.css +--- + +
    +

    Quy tắc ứng xử cộng đồng Kubernetes

    + +Kubernetes tuân theo +quy tắc ứng xử CNCF. +Văn bản của CNCF CoC được sao chép dưới đây, kể từ +commit 214585e. +Nếu bạn nhận thấy rằng điều này đã lỗi thời, xin vui lòng +gửi một vấn đề. + +Nếu bạn nhận thấy vi phạm Quy tắc ứng xử tại một sự kiện hoặc cuộc họp, trong +Slack, hoặc trong một cơ chế giao tiếp khác, hãy liên lạc với Ủy ban Quy tắc ứng xử Kubernetes. +Bạn có thể liên hệ với chúng tôi qua email tại conduct@kubernetes.io. +Sự ẩn danh của bạn sẽ được bảo vệ. + +
    +{{< include "/static/cncf-code-of-conduct.md" >}} +
    +
    diff --git a/content/vi/community/static/README.md b/content/vi/community/static/README.md new file mode 100644 index 0000000000..94f742e750 --- /dev/null +++ b/content/vi/community/static/README.md @@ -0,0 +1 @@ +Tài liệu trong thư mục này được lấy từ nguồn khác. Vui lòng không trực tiếp sửa chúng, ngoại trừ việc thay thế chúng với phiên bản khác. \ No newline at end of file diff --git a/content/vi/community/static/cncf-code-of-conduct.md b/content/vi/community/static/cncf-code-of-conduct.md new file mode 100644 index 0000000000..9d7008e902 --- /dev/null +++ b/content/vi/community/static/cncf-code-of-conduct.md @@ -0,0 +1,31 @@ + +## Quy tắc ứng xử cộng đồng CNCF v1.0 + +### Quy tắc ứng xử người đóng góp + +Là người đóng góp và duy trì dự án này, và vì lợi ích thúc đẩy một cộng đồng mở, chúng tôi cam kết tôn trọng tất cả những người đóng góp thông qua việc báo cáo các vấn đề, các tính năng yêu cầu, cập nhật tài liệu, gửi PRs hay patches, và các hoạt động khác. + +Chúng tôi cam kết tham gia dự án này là một trải nghiệm tốt cho mọi người, bất kể mức độ kinh nghiệm, giới tính, khuynh hướng tình dục, khuyết tật, ngoại hình cá nhân, chủng tộc, dân tộc, tuổi tác, tôn giáo, hoặc quốc tịch. + +Ví dụ về hành vi không được chấp nhận của người tham gia bao gồm: + +* Việc sử dụng ngôn ngữ hoặc hình ảnh tình dục +* Tấn công cá nhân +* Trêu đùa hoặc bình luận lăng mạ, xúc phạm +* Quấy rối công khai hoặc riêng tư +* Xuất bản thông tin cá nhân của người khác, chẳng hạn như địa chỉ vật lý hoặc điện tử, mà không có sự cho phép rõ ràng +* Hành vi phi đạo đức hoặc không chuyên nghiệp khác. + +Người duy trì dự án có quyền và trách nhiệm xóa, chỉnh sửa hoặc từ chối comments, commits, code, chỉnh sửa wiki, issues và các đóng góp khác không phù hợp với Quy tắc ứng xử này. Bằng cách áp dụng Quy tắc ứng xử này, những người duy trì dự án cam kết áp dụng một cách công bằng và nhất quán các nguyên tắc này cho mọi khía cạnh của việc quản lý dự án này. Những người duy trì dự án không tuân theo hoặc thực thi Quy tắc ứng xử có thể bị xóa vĩnh viễn khỏi nhóm dự án. + +Quy tắc ứng xử này áp dụng cả trong không gian dự án và trong không gian công cộng khi một cá nhân đại diện cho dự án hoặc cộng đồng của dự án. + +Các trường hợp lạm dụng, quấy rối hoặc hành vi không thể chấp nhận được trong Kubernetes có thể được báo cáo bằng cách liên hệ với [Ủy ban Quy tắc ứng xử Kubernetes](https://git.k8s.io/community/committee-code-of-conduct) thông qua . Đối với các dự án khác, vui lòng liên hệ với người bảo trì dự án CNCF hoặc hòa giải viên của chúng tôi, Mishi Choudhary . + +Quy tắc ứng xử này được điều chỉnh từ Giao ước cộng tác viên (http://contributor-covenant.org), phiên bản 1.2.0, có sẵn tại +http://contributor-covenant.org/version/1/2/0/ + +### Quy tắc ứng xử sự kiện CNCF + +Các sự kiện CNCF được điều hành bởi Linux Foundation [Quy tắc ứng xử] (https://events.linuxfoundation.org/code-of-conduct/) có sẵn trên trang sự kiện. Điều này được thiết kế để tương thích với chính sách trên và cũng bao gồm nhiều chi tiết hơn về cách ứng phó với các sự cố. \ No newline at end of file From b13dc090e5ae187618b501034bbf8b930c97e79b Mon Sep 17 00:00:00 2001 From: tiendc Date: Mon, 16 Mar 2020 13:30:35 +0700 Subject: [PATCH 099/140] Adds doc vi/partners (#18203) * Adds doc vi/partners Signed-off-by: Dao Cong Tien * Update _index.html --- content/vi/partners/_index.html | 91 +++++++++++++++++++++++++++++++++ 1 file changed, 91 insertions(+) create mode 100644 content/vi/partners/_index.html diff --git a/content/vi/partners/_index.html b/content/vi/partners/_index.html new file mode 100644 index 0000000000..f5b44d5e94 --- /dev/null +++ b/content/vi/partners/_index.html @@ -0,0 +1,91 @@ +--- +title: Các đối tác +bigheader: Các đối tác của Kubernetes +abstract: Phát triển hệ sinh thái Kubernetes. +class: gridPage +cid: partners +--- + +
    +
    +
    Kubernetes phối hợp làm việc với các đối tác để tạo ra một codebase mạnh mẽ hỗ trợ một loạt các nền tảng bổ sung.
    +
    +
    +
    +
    + Các nhà cung cấp dịch vụ được chứng nhận bởi Kubernetes (KCSP) +
    +
    Các nhà cung cấp dịch vụ được chứng nhận với bề dày kinh nghiệm sẽ trợ giúp các tổ chức kinh doanh, các công ty ứng dụng Kubernetes nhanh chóng. +


    + +

    Bạn muốn trở thành một KCSP? +
    +
    +
    +
    +
    + Các nhà phân phối Kubernetes, dịch vụ hosting, dịch vụ cài đặt +
    Tiêu chuẩn tương thích về phần mềm bảo đảm rằng các phiên bản Kubernetes từ các nhà cung cấp sẽ hỗ trợ các bộ API được yêu cầu bởi khách hàng. +


    + +

    Bạn muốn trở thành một Kubernetes Certified? +
    +
    +
    +
    +
    Các đối tác đào tạo Kubernetes (KTP)
    +
    Các đối tác đào tạo được chứng nhận đã và đang sở hữu bề dày kinh nghiệm trong lĩnh vực đám mây. +



    + +

    Bạn muốn trở thành một KTP? +
    +
    +
    + + + +
    + + +
    + +
    +
    + + + + From 8aafa4ab9d63ec066b9e61238202382df288497a Mon Sep 17 00:00:00 2001 From: Rajesh Deshpande Date: Mon, 16 Mar 2020 21:38:44 +0530 Subject: [PATCH 100/140] Correcting name of concept 'Container Enviorment Variables' (#19279) * Correcting name of concept 'Container Enviorment Variables' This page discusses the container environment and container environment variables are part of it. So removing a 'variable' word to make it more correct. * Correcting references Correcting references * Correcting reference links Correcting reference links --- .../cluster-administration/cluster-administration-overview.md | 2 +- ...tainer-environment-variables.md => container-environment.md} | 2 +- .../en/docs/concepts/containers/container-lifecycle-hooks.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) rename content/en/docs/concepts/containers/{container-environment-variables.md => container-environment.md} (98%) diff --git a/content/en/docs/concepts/cluster-administration/cluster-administration-overview.md b/content/en/docs/concepts/cluster-administration/cluster-administration-overview.md index a825335304..0512cabc90 100644 --- a/content/en/docs/concepts/cluster-administration/cluster-administration-overview.md +++ b/content/en/docs/concepts/cluster-administration/cluster-administration-overview.md @@ -44,7 +44,7 @@ Note: Not all distros are actively maintained. Choose distros which have been te * [Certificates](/docs/concepts/cluster-administration/certificates/) describes the steps to generate certificates using different tool chains. -* [Kubernetes Container Environment](/docs/concepts/containers/container-environment-variables/) describes the environment for Kubelet managed containers on a Kubernetes node. +* [Kubernetes Container Environment](/docs/concepts/containers/container-environment/) describes the environment for Kubelet managed containers on a Kubernetes node. * [Controlling Access to the Kubernetes API](/docs/reference/access-authn-authz/controlling-access/) describes how to set up permissions for users and service accounts. diff --git a/content/en/docs/concepts/containers/container-environment-variables.md b/content/en/docs/concepts/containers/container-environment.md similarity index 98% rename from content/en/docs/concepts/containers/container-environment-variables.md rename to content/en/docs/concepts/containers/container-environment.md index b8b3c28a6b..86b595661d 100644 --- a/content/en/docs/concepts/containers/container-environment-variables.md +++ b/content/en/docs/concepts/containers/container-environment.md @@ -2,7 +2,7 @@ reviewers: - mikedanese - thockin -title: Container Environment Variables +title: Container Environment content_template: templates/concept weight: 20 --- diff --git a/content/en/docs/concepts/containers/container-lifecycle-hooks.md b/content/en/docs/concepts/containers/container-lifecycle-hooks.md index 3d4f81152d..fe810d23c5 100644 --- a/content/en/docs/concepts/containers/container-lifecycle-hooks.md +++ b/content/en/docs/concepts/containers/container-lifecycle-hooks.md @@ -116,7 +116,7 @@ Events: {{% capture whatsnext %}} -* Learn more about the [Container environment](/docs/concepts/containers/container-environment-variables/). +* Learn more about the [Container environment](/docs/concepts/containers/container-environment/). * Get hands-on experience [attaching handlers to Container lifecycle events](/docs/tasks/configure-pod-container/attach-handler-lifecycle-event/). From 246c052f5dfd74cbc357826618830744f376c7df Mon Sep 17 00:00:00 2001 From: Daniel Date: Mon, 16 Mar 2020 16:10:43 +0000 Subject: [PATCH 101/140] Updating release note link (#19585) --- content/en/docs/tasks/administer-cluster/cluster-management.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/tasks/administer-cluster/cluster-management.md b/content/en/docs/tasks/administer-cluster/cluster-management.md index bceb4fef96..65728ec4ee 100644 --- a/content/en/docs/tasks/administer-cluster/cluster-management.md +++ b/content/en/docs/tasks/administer-cluster/cluster-management.md @@ -24,7 +24,7 @@ To install Kubernetes on a set of machines, consult one of the existing [Getting ## Upgrading a cluster -The current state of cluster upgrades is provider dependent, and some releases may require special care when upgrading. It is recommended that administrators consult both the [release notes](https://git.k8s.io/kubernetes/CHANGELOG.md), as well as the version specific upgrade notes prior to upgrading their clusters. +The current state of cluster upgrades is provider dependent, and some releases may require special care when upgrading. It is recommended that administrators consult both the [release notes](https://git.k8s.io/kubernetes/CHANGELOG/README.md), as well as the version specific upgrade notes prior to upgrading their clusters. ### Upgrading an Azure Kubernetes Service (AKS) cluster From 403ef9c84e198c88737d1eee6b8224c6edf226be Mon Sep 17 00:00:00 2001 From: Alexey Pyltsyn Date: Mon, 16 Mar 2020 19:12:43 +0300 Subject: [PATCH 102/140] Translate Learning environment section into Russian (#19658) * Translate Learning environment section into Russian * Apply suggestions from code review Co-Authored-By: Nikita Potapenko * Update content/ru/docs/setup/learning-environment/minikube.md Co-Authored-By: Nikita Potapenko Co-authored-by: Nikita Potapenko --- .../docs/setup/learning-environment/_index.md | 4 + .../docs/setup/learning-environment/kind.md | 19 + .../setup/learning-environment/minikube.md | 512 ++++++++++++++++++ 3 files changed, 535 insertions(+) create mode 100644 content/ru/docs/setup/learning-environment/_index.md create mode 100644 content/ru/docs/setup/learning-environment/kind.md create mode 100644 content/ru/docs/setup/learning-environment/minikube.md diff --git a/content/ru/docs/setup/learning-environment/_index.md b/content/ru/docs/setup/learning-environment/_index.md new file mode 100644 index 0000000000..0670a18c50 --- /dev/null +++ b/content/ru/docs/setup/learning-environment/_index.md @@ -0,0 +1,4 @@ +--- +title: Среда обучения +weight: 20 +--- diff --git a/content/ru/docs/setup/learning-environment/kind.md b/content/ru/docs/setup/learning-environment/kind.md new file mode 100644 index 0000000000..d0f775dd22 --- /dev/null +++ b/content/ru/docs/setup/learning-environment/kind.md @@ -0,0 +1,19 @@ +--- +title: Установка Kubernetes с помощью Kind +weight: 40 +content_template: templates/concept +--- + +{{% capture overview %}} + +Kind — это инструмент для запуска локальных кластеров Kubernetes с помощью "узлов" контейнера Docker. + +{{% /capture %}} + +{{% capture body %}} + +## Установка + +Смотрите страницу [по установке Kind](https://kind.sigs.k8s.io/docs/user/quick-start/). + +{{% /capture %}} diff --git a/content/ru/docs/setup/learning-environment/minikube.md b/content/ru/docs/setup/learning-environment/minikube.md new file mode 100644 index 0000000000..1e0cb02673 --- /dev/null +++ b/content/ru/docs/setup/learning-environment/minikube.md @@ -0,0 +1,512 @@ +--- +reviewers: +- dlorenc +- balopat +- aaron-prindle +title: Установка Kubernetes с помощью Minikube +weight: 30 +content_template: templates/concept +--- + +{{% capture overview %}} + +Minikube — это инструмент, позволяющий легко запускать Kubernetes на локальной машине. Для тех, кто хочет попробовать Kubernetes или рассмотреть возможность его использования в повседневной разработке, Minikube станет отличным вариантом, потому что он запускает одноузловой кластер Kubernetes внутри виртуальной машины (VM) на компьютере пользователя. + +{{% /capture %}} + +{{% capture body %}} + +## Возможности Minikube + +Minikube поддерживает следующие возможности Kubernetes: + +* DNS +* Сервисы NodePort +* Словари конфигурации (ConfigMaps) и секреты (Secrets) +* Панель управления (Dashboard) +* Среда выполнения контейнера: Docker, [CRI-O](https://cri-o.io/) и [containerd](https://github.com/containerd/containerd) +* Поддержка CNI (Container Network Interface) +* Ingress + +## Установке + +Посмотрите страницу [Установка Minikube](/ru/docs/tasks/tools/install-minikube/). + +## Краткое руководство + +Эта простая демонстрация поможет запустить, использовать и удалить Minikube на локальной машине. Следуйте перечисленным ниже шагам, чтобы начать знакомство с Minikube. + +1. Запустите Minikube и создайте кластер: + + ```shell + minikube start + ``` + + Вывод будет примерно следующим: + + ``` + Starting local Kubernetes cluster... + Running pre-create checks... + Creating machine... + Starting local Kubernetes cluster... + ``` + + Дополнительную информацию о запуске кластера в определенной версии Kubernetes, виртуальной машине или среде выполнения контейнера смотрите в разделе [Запуск кластера](#запуск-кластера). + +2. Теперь вы можете работать со своим кластером через CLI-инструмент kubectl. Для получения дополнительной информации смотрите раздел [Работа с кластером](#работа-с-кластером). + + Давайте создадим развёртывание (Deployment) в Kubernetes, используя существующий образ `echoserver`, представляющий простой HTTP-сервер, и сделаем его доступным на порту 8080 с помощью `--port`. + + ```shell + kubectl create deployment hello-minikube --image=k8s.gcr.io/echoserver:1.10 + ``` + + Вывод будет примерно следующим: + + ``` + deployment.apps/hello-minikube created + ``` + +3. Чтобы получить доступ к объекту Deployment `hello-minikube` извне, создайте объект сервиса (Service): + + ```shell + kubectl expose deployment hello-minikube --type=NodePort --port=8080 + ``` + + Опция `--type=NodePort` определяет тип сервиса. + + Вывод будет примерно следующим: + + ``` + service/hello-minikube exposed + ``` + +4. Под (Pod) `hello-minikube` теперь запущен, но нужно подождать, пока он начнёт функционировать, прежде чем обращаться к нему. + + Проверьте, что под работает: + + ```shell + kubectl get pod + ``` + + Если в столбце вывода `STATUS` выводится `ContainerCreating`, значит под все еще создается: + + ``` + NAME READY STATUS RESTARTS AGE + hello-minikube-3383150820-vctvh 0/1 ContainerCreating 0 3s + ``` + + Если в столбце `STATUS` указано `Running`, то под теперь в рабочем состоянии: + + ``` + NAME READY STATUS RESTARTS AGE + hello-minikube-3383150820-vctvh 1/1 Running 0 13s + ``` +5. Узнайте URL-адрес открытого (exposed) сервиса, чтобы просмотреть подробные сведения о сервисе: + + ```shell + minikube service hello-minikube --url + ``` +6. Чтобы ознакомиться с подробной информацией о локальном кластере, скопируйте и откройте полученный из вывода команды на предыдущем шаге URL-адрес в браузере. + + Вывод будет примерно следующим: + + ``` + Hostname: hello-minikube-7c77b68cff-8wdzq + + Pod Information: + -no pod information available- + + Server values: + server_version=nginx: 1.13.3 - lua: 10008 + + Request Information: + client_address=172.17.0.1 + method=GET + real path=/ + query= + request_version=1.1 + request_scheme=http + request_uri=http://192.168.99.100:8080/ + + Request Headers: + accept=*/* + host=192.168.99.100:30674 + user-agent=curl/7.47.0 + + Request Body: + -no body in request- + ``` + + Если сервис и кластер вам больше не нужны, их можно удалить. +7. Удалите сервис `hello-minikube`: + + ```shell + kubectl delete services hello-minikube + ``` + + Вывод будет примерно следующим: + + ``` + service "hello-minikube" deleted + ``` +8. Удалите развёртывание `hello-minikube`: + + ```shell + kubectl delete deployment hello-minikube + ``` + + Вывод будет примерно следующим: + + ``` + deployment.extensions "hello-minikube" deleted + ``` +9. Остановите локальный кластер Minikube: + ```shell + minikube stop + ``` + Вывод будет примерно следующим: + ``` + Stopping "minikube"... + "minikube" stopped. + ``` + Подробности смотрите в разделе [Остановка кластера](#остановка-кластера). +10. Удалите локальный кластер Minikube: + ```shell + minikube delete + ``` + + Вывод будет примерно следующим: + + ``` + Deleting "minikube" ... + The "minikube" cluster has been deleted. + ``` + Подробности смотрите в разделе [Удаление кластера](#удаление-кластера). + +## Управление кластером + +### Запуск кластера + +Команда `minikube start` используется для запуска кластера. +Эта команда создаёт и конфигурирует виртуальную машину, которая запускает одноузловой кластер Kubernetes. +Эта команда также настраивает вашу установку [kubectl](/docs/user-guide/kubectl-overview/) для взаимодействия с этим кластером. + +{{< note >}} +Если вы работаете из-под веб-прокси, вам нужно указать данные прокси в команде `minikube start`: + +```shell +https_proxy= minikube start --docker-env http_proxy= --docker-env https_proxy= --docker-env no_proxy=192.168.99.0/24 +``` + +К сожалению, установка переменных окружения не cработает. + +Minikube также создает контекст "minikube" и устанавливает его по умолчанию в kubectl. +Чтобы вернуться к этому контексту, выполните следующую команду: `kubectl config use-context minikube`. +{{< /note >}} + +#### Указание версии Kubernetes + +Вы можете указать используемую версию Kubernetes в Minikube, добавив параметр `--kubernetes-version` в команду `minikube start`. Например, чтобы запустить Minikube из-под версии {{< param "fullversion" >}}, вам нужно выполнить следующую команду: + +```shell +minikube start --kubernetes-version {{< param "fullversion" >}} +``` + +#### Указание драйвера виртуальной машины + +Вы можете изменить драйвер виртуальной машины, добавив флаг `--vm-driver=` в команду `minikube start`. + +Тогда команда будет выглядеть так: + +```shell +minikube start --vm-driver= +``` + +Minikube поддерживает следующие драйверы: + +{{< note >}} +Смотрите файл [DRIVERS](https://git.k8s.io/minikube/docs/drivers.md) для получения подробной информации о поддерживаемых драйверах и как устанавливать плагины. +{{< /note >}} + +* virtualbox +* vmwarefusion +* kvm2 ([установка драйвера](https://git.k8s.io/minikube/docs/drivers.md#kvm2-driver)) +* hyperkit ([установка драйвера](https://git.k8s.io/minikube/docs/drivers.md#hyperkit-driver)) +* hyperv ([установка драйвера](https://github.com/kubernetes/minikube/blob/master/docs/drivers.md#hyperv-driver)) +Обратите внимание, что указанный IP-адрес на этой странице является динамическим и может изменяться. Его можно получить с помощью `minikube ip`. +* vmware ([установка драйвера](https://github.com/kubernetes/minikube/blob/master/docs/drivers.md#vmware-unified-driver)) (VMware unified driver) +* none (Запускает компоненты Kubernetes на хосте, а не на виртуальной машине. Использование этого драйвера требует использование Linux и установленного {{< glossary_tooltip term_id="docker" >}}.) + +{{< caution >}} +Если вы используете драйвер `none`, некоторые компоненты Kubernetes запускаются как привилегированные контейнеры, которые имеют побочные эффекты вне окружения Minikube. Эти побочные эффекты означают, что драйвер `none` не рекомендуется использовать в личных рабочих станций. +{{< /caution >}} + +#### Запуск кластера в других средах выполнения контейнеров + +Вы можете запустить Minikube в следующих средах выполнения контейнеров. + +{{< tabs name="container_runtimes" >}} +{{% tab name="containerd" %}} +Чтобы использовать [containerd](https://github.com/containerd/containerd) в качестве среды выполнения контейнера, выполните команду ниже: + +```bash +minikube start \ + --network-plugin=cni \ + --enable-default-cni \ + --container-runtime=containerd \ + --bootstrapper=kubeadm +``` + +Также можете использовать расширенную вариант команды: + +```bash +minikube start \ + --network-plugin=cni \ + --enable-default-cni \ + --extra-config=kubelet.container-runtime=remote \ + --extra-config=kubelet.container-runtime-endpoint=unix:///run/containerd/containerd.sock \ + --extra-config=kubelet.image-service-endpoint=unix:///run/containerd/containerd.sock \ + --bootstrapper=kubeadm +``` +{{% /tab %}} +{{% tab name="CRI-O" %}} +Чтобы использовать [CRI-O](https://cri-o.io/) в качестве среды выполнения контейнера, выполните команду ниже: + +```bash +minikube start \ + --network-plugin=cni \ + --enable-default-cni \ + --container-runtime=cri-o \ + --bootstrapper=kubeadm +``` + +Также можете использовать расширенную вариант команды: + +```bash +minikube start \ + --network-plugin=cni \ + --enable-default-cni \ + --extra-config=kubelet.container-runtime=remote \ + --extra-config=kubelet.container-runtime-endpoint=/var/run/crio.sock \ + --extra-config=kubelet.image-service-endpoint=/var/run/crio.sock \ + --bootstrapper=kubeadm +``` +{{% /tab %}} +{{< /tabs >}} + +#### Использование локальных образов путём повторного использования демона Docker + +При использовании одной виртуальной машины для Kubernetes легко повторно использовать демон Docker, встроенный в Minikube. В этом случае нет необходимости создавать реестр Docker на вашей хост-машине и отправлять образ туда. Вместо этого вы можете создать реестр внутри того же демона Docker, который использует Minikube, что позволит ускорить локальные запуски. + +{{< note >}} +Обязательно пометьте собственным тегом Docker-образ, и затем при получении образа всегда указывайте его. Так как `:latest` — это тег по умолчанию, поэтому наряду с соответствующей стандартной политикой получения образа, равной `Always`, в конечном итоге возникнет ошибка при получении образа (`ErrImagePull`), если Docker-образ не найден в базовом реестре Docker (как правило, в DockerHub). +{{< /note >}} + +Для работы с Docker-демоном на вашем хосте под управлением Mac/Linux, запустите последнюю строку из вывода команды `minikube docker-env`. + +Теперь вы можете использовать Docker в командной строке вашего хост-компьютера на Mac/Linux для взаимодействия с демоном Docker внутри виртуальной машины Minikube: + +```shell +docker ps +``` + +{{< note >}} +На Centos 7 Docker может возникнуть следующая ошибка: + +``` +Could not read CA certificate "/etc/docker/ca.pem": open /etc/docker/ca.pem: no such file or directory +``` + +Для исправления этой ошибки обновите файл `/etc/sysconfig/docker`, чтобы учитывались изменения в среде Minikube: + +```shell +< DOCKER_CERT_PATH=/etc/docker +--- +> if [ -z "${DOCKER_CERT_PATH}" ]; then +> DOCKER_CERT_PATH=/etc/docker +> fi +``` +{{< /note >}} + +### Конфигурация Kubernetes + +Minikube имеет такую возможность как "конфигуратор" ("configurator"), позволяющая пользователям настраивать компоненты Kubernetes произвольными значениями. +Чтобы использовать эту возможность, используйте флаг `--extra-config` в команде `minikube start`. + +Этот флаг можно дублировать, поэтому вы можете указать его несколько раз с несколькими разными значениями, чтобы установить несколько опций. + +Этот флаг принимает строку вида `component.key=value`, где `component` — это одно из значение в приведённом ниже списка, `key` — ключ из структуры конфигурации, а `value` — значение, которое нужно установить. + +Допустимые ключи можно найти в документации по `componentconfigs` в Kubernetes каждого компонента. +Ниже вы найдете документации по каждой поддерживаемой конфигурации: + +* [kubelet](https://godoc.org/k8s.io/kubernetes/pkg/kubelet/apis/config#KubeletConfiguration) +* [apiserver](https://godoc.org/k8s.io/kubernetes/cmd/kube-apiserver/app/options#ServerRunOptions) +* [proxy](https://godoc.org/k8s.io/kubernetes/pkg/proxy/apis/config#KubeProxyConfiguration) +* [controller-manager](https://godoc.org/k8s.io/kubernetes/pkg/controller/apis/config#KubeControllerManagerConfiguration) +* [etcd](https://godoc.org/github.com/coreos/etcd/etcdserver#ServerConfig) +* [scheduler](https://godoc.org/k8s.io/kubernetes/pkg/scheduler/apis/config#KubeSchedulerConfiguration) + +#### Примеры + +Чтобы изменить настройку `MaxPods` на значение 5 в Kubelet, передайте этот флаг `--extra-config=kubelet.MaxPods=5`. + +Эта возможность также поддерживает вложенные структуры. Для изменения настройки `LeaderElection.LeaderElect` на значение `true` в планировщике, передайте флаг `--extra-config=scheduler.LeaderElection.LeaderElect=true`. + +Чтобы изменить настройку `AuthorizationMode` в `apiserver` на значение `RBAC`, используйте флаг `--extra-config=apiserver.authorization-mode=RBAC`. + +### Остановка кластера +Команда `minikube stop` используется для остановки кластера. +Эта команда выключает виртуальную машины Minikube, но сохраняет всё состояние кластера и данные. +Повторный запуск кластера вернет его в прежнее состояние. + +### Удаление кластера +Команда `minikube delete` используется для удаления кластера. +Эта команда выключает и удаляет виртуальную машину Minikube. +Данные или состояние не сохраняются. + +### Обновление minikube +Смотрите [инструкцию по обновлению minikube](https://minikube.sigs.k8s.io/docs/start/macos/). + +## Работа с кластером + +### Kubectl + +Команда `minikube start` создает [контекст kubectl](/docs/reference/generated/kubectl/kubectl-commands#-em-set-context-em-) под именем "minikube". +Этот контекст содержит конфигурацию для взаимодействия с кластером Minikube. + +Minikube автоматически устанавливает этот контекст, но если вам потребуется явно использовать его в будущем, выполните команду ниже: + +```shell +kubectl config use-context minikube +``` + +Либо передайте контекст при выполнении команды следующим образом: `kubectl get pods --context=minikube`. + +### Панель управления + +Чтобы получить доступ к [панели управления Kubernetes](/docs/tasks/access-application-cluster/web-ui-dashboard/), запустите эту команду в командной оболочке после запуска Minikube, чтобы получить адрес: + +```shell +minikube dashboard +``` + +### Сервисы + +Чтобы получить доступ к сервису, открытой через порт узла, выполните команду в командной оболочке после запуска Minikube, чтобы получить адрес: + +```shell +minikube service [-n NAMESPACE] [--url] NAME +``` + +## Организация сети + +Виртуальная машина Minikube доступна только хост-системе через IP-адрес, который можно получить с помощью команды `minikube ip`. +Вы можете использовать IP-адрес для доступа к любому сервису типа `NodePort`. + +Чтобы определить NodePort для вашего сервиса, вы можете использовать такую команду `kubectl`: + +```shell +kubectl get service $SERVICE --output='jsonpath="{.spec.ports[0].nodePort}"' +``` + +## Постоянные тома + +Minikube поддерживает [PersistentVolumes](/docs/concepts/storage/persistent-volumes/) типа `hostPath`. +Эти постоянные тома монтируются в виртуальную машину Minikube. + +Виртуальная машина Minikube загружается в файловую систему tmpfs, поэтому большинство директорий не будет сохранено при перезагрузках (`minikube stop`). +Однако Minikube сконфигурирован на сохранение файлов, хранящихся в перечисленных ниже директорий хоста. + +* `/data` +* `/var/lib/minikube` +* `/var/lib/docker` + +Пример конфигурации PersistentVolume для сохранения данных в директории `/data`: + +```yaml +apiVersion: v1 +kind: PersistentVolume +metadata: + name: pv0001 +spec: + accessModes: + - ReadWriteOnce + capacity: + storage: 5Gi + hostPath: + path: /data/pv0001/ +``` + +## Смонтированные директории хоста +Некоторые драйверы монтируют директорию хоста в виртуальную машину, чтобы можно было легко обмениваться файлами между виртуальной машиной и хостом. В настоящее время это не настраивается и отличается от используемого драйвера и ОС. + +{{< note >}} +Совместное использование директории хоста еще не реализовано в драйвере KVM. +{{< /note >}} + +| Driver | OS | HostFolder | VM | +|---------------|---------|------------|-----------------| +| VirtualBox | Linux | /home | /hosthome | +| VirtualBox | macOS | /Users | /Users | +| VirtualBox | Windows | C://Users | /c/Users | +| VMware Fusion | macOS | /Users | /mnt/hgfs/Users | +| Xhyve | macOS | /Users | /Users | + +## Приватные реестры контейнеров + +Для доступа к реестру приватных контейнеров, выполните шаги, описанные на [этой странице](/docs/concepts/containers/images/). + +Мы рекомендуем использовать `ImagePullSecrets`, но если вам нужно обратиться к нему из виртуальной машины Minikube, нужно поместить файл `.dockercfg` в директорию `/home/docker` или `config.json` в директорию `/home/docker/.docker`. + +## Дополнения + +Для того, чтобы Minikube смог запустить или перезапустить пользовательские дополнения, поместите дополнения, которые вы хотите запускать с помощью Minikube, в директорию `~/.minikube/addons`. Дополнения в этой директории будут перемещены в виртуальную машину Minikube и запускаться каждый раз при запуске или перезапуске Minikube. + +## Использование Minikube с помощью HTTP-прокси + +Minikube создаёт виртуальную машину, включающая в себя Kubernetes и демон Docker. +Когда Kubernetes планирует выполнение контейнеров с использованием Docker, демону Docker может потребоваться доступ к внешней сети для получения контейнеров. + +Если вы работаете через HTTP-прокси, вам нужно сконфигурировать настройки прокси для Docker. +Для этого нужно передать необходимые переменные окружения в флаги перед выполнением команды `minikube start`. + +Например: + +```shell +minikube start --docker-env http_proxy=http://$YOURPROXY:PORT \ + --docker-env https_proxy=https://$YOURPROXY:PORT +``` + +Если адрес вашей виртуальной машины 192.168.99.100, то, скорее всего, настройки прокси помешают `kubectl` обратиться к ней. +Чтобы прокси игнорировал этот IP-адрес, нужно скорректировать настройки no_proxy следующим образом: + +```shell +export no_proxy=$no_proxy,$(minikube ip) +``` + +## Известные проблемы + +Функциональность, для которой требуется несколько узлов, не будет работать в Minikube. + +## Реализация + +Minikube использует [libmachine](https://github.com/docker/machine/tree/master/libmachine) для подготовки виртуальных машин и [kubeadm](https://github.com/kubernetes/kubeadm) для инициализации кластера Kubernetes. + +Для получения дополнительной информации о Minikube посмотрите [статью](https://git.k8s.io/community/contributors/design-proposals/cluster-lifecycle/local-cluster-ux.md). + +## Дополнительные ссылки + +* **Цели**: цели проекта Minikube смотрите в [дорожной карте](https://git.k8s.io/minikube/docs/contributors/roadmap.md). +* **Руководство по разработке**: посмотрите [CONTRIBUTING.md](https://git.k8s.io/minikube/CONTRIBUTING.md), чтобы ознакомиться с тем, как отправлять пулрексты. +* **Сборка Minikube**: инструкции по сборке/тестированию Minikube из исходного кода смотрите в [руководстве по сборке](https://git.k8s.io/minikube/docs/contributors/build_guide.md). +* **Добавление новой зависимости**: инструкции по добавлению новой зависимости в Minikube смотрите в [руководстве по добавлению зависимостей](https://git.k8s.io/minikube/docs/contributors/adding_a_dependency.md). +* **Добавление нового дополнения**: инструкции по добавлению нового дополнения для Minikube смотрите в [руководстве по добавлению дополнений](https://git.k8s.io/minikube/docs/contributors/adding_an_addon.md). +* **MicroK8**: пользователи Linux, которые не хотят использовать виртуальную машину, могут в качестве альтернативы посмотреть в сторону [MicroK8s](https://microk8s.io/). + +## Сообщество + +Помощь, вопросы и комментарии приветствуются и поощряются! Разработчики Minikube проводят время на [Slack](https://kubernetes.slack.com) в канале #minikube (получить приглашение можно [здесь](http://slack.kubernetes.io/)). У нас также есть [список рассылки kubernetes-dev на Google Groups](https://groups.google.com/forum/#!forum/kubernetes-dev). Если вы отправляете сообщение в список, пожалуйста, начните вашу тему с "minikube: ". + +{{% /capture %}} From 915fe136f3be7a41a8cf5126a7b909b972a8416a Mon Sep 17 00:00:00 2001 From: Alexey Pyltsyn Date: Mon, 16 Mar 2020 19:14:43 +0300 Subject: [PATCH 103/140] Translate Install Tools subsection in Tasks section into Russian (#19659) --- content/ru/docs/tasks/tools/_index.md | 5 + .../ru/docs/tasks/tools/install-kubectl.md | 485 ++++++++++++++++++ .../ru/docs/tasks/tools/install-minikube.md | 255 +++++++++ 3 files changed, 745 insertions(+) create mode 100755 content/ru/docs/tasks/tools/_index.md create mode 100644 content/ru/docs/tasks/tools/install-kubectl.md create mode 100644 content/ru/docs/tasks/tools/install-minikube.md diff --git a/content/ru/docs/tasks/tools/_index.md b/content/ru/docs/tasks/tools/_index.md new file mode 100755 index 0000000000..976aeed7a8 --- /dev/null +++ b/content/ru/docs/tasks/tools/_index.md @@ -0,0 +1,5 @@ +--- +title: "Установка инструментов" +weight: 10 +--- + diff --git a/content/ru/docs/tasks/tools/install-kubectl.md b/content/ru/docs/tasks/tools/install-kubectl.md new file mode 100644 index 0000000000..0e40eec3b3 --- /dev/null +++ b/content/ru/docs/tasks/tools/install-kubectl.md @@ -0,0 +1,485 @@ +--- +reviewers: +- mikedanese +title: Установка и настройка kubectl +content_template: templates/task +weight: 10 +card: + name: tasks + weight: 20 + title: Установка kubectl +--- + +{{% capture overview %}} +Инструмент командной строки Kubernetes [kubectl](/docs/user-guide/kubectl/) позволяет запускать команды для кластеров Kubernetes. Вы можете использовать kubectl для развертывания приложений, проверки и управления ресурсов кластера, а также для просмотра логов. Полный список операций kubectl смотрите в [Overview of kubectl](/docs/reference/kubectl/overview/). +{{% /capture %}} + +{{% capture prerequisites %}} +Используемая вами мажорная версия kubectl не должна отличаться от той, которая используется в кластере. Например, версия v1.2 может работать с версиями v1.1, v1.2 и v1.3. Использование последней версии kubectl поможет избежать непредвиденных проблем. +{{% /capture %}} + +{{% capture steps %}} + +## Установка kubectl в Linux + +### Установка двоичного файла kubectl с помощью curl в Linux + +1. Загрузите последнюю версию с помощью команды: + + ``` + curl -LO https://storage.googleapis.com/kubernetes-release/release/`curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt`/bin/linux/amd64/kubectl + ``` + + Чтобы загрузить определенную версию, вставьте в фрагмент команды `$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)` нужную версию. + + Например, команда загрузки версии {{< param "fullversion" >}} для Linux будет выглядеть следующим образом: + + ``` + curl -LO https://storage.googleapis.com/kubernetes-release/release/{{< param "fullversion" >}}/bin/linux/amd64/kubectl + ``` + +2. Сделайте двоичный файл kubectl исполняемым: + + ``` + chmod +x ./kubectl + ``` + +3. Переместите двоичный файл в директорию из переменной окружения PATH: + + ``` + sudo mv ./kubectl /usr/local/bin/kubectl + ``` +4. Убедитесь, что установлена последняя версия: + + ``` + kubectl version --client + ``` + +### Установка с помощью встроенного пакетного менеджера + +{{< tabs name="kubectl_install" >}} +{{< tab name="Ubuntu, Debian или HypriotOS" codelang="bash" >}}sudo apt-get update && sudo apt-get install -y apt-transport-https +curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add - +echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee -a /etc/apt/sources.list.d/kubernetes.list +sudo apt-get update +sudo apt-get install -y kubectl +{{< /tab >}} +{{< tab name="CentOS, RHEL или Fedora" codelang="bash" >}}cat < /etc/yum.repos.d/kubernetes.repo +[kubernetes] +name=Kubernetes +baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64 +enabled=1 +gpgcheck=1 +repo_gpgcheck=1 +gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg +EOF +yum install -y kubectl +{{< /tab >}} +{{< /tabs >}} + +### Установка с помощью стороннего пакетного менеджера + +{{< tabs name="other_kubectl_install" >}} +{{% tab name="Snap" %}} +Если вы используйте Ubuntu или другой Linux-дистрибутив, в котором есть пакетный менеджер [snap](https://snapcraft.io/docs/core/install), kubectl доступен в виде приложения [snap](https://snapcraft.io/). + +```shell +snap install kubectl --classic + +kubectl version +``` +{{% /tab %}} +{{% tab name="Homebrew" %}} +Если вы работаете в Linux и используете пакетный менеджер [Homebrew](https://docs.brew.sh/Homebrew-on-Linux), то kubectl можно [установить](https://docs.brew.sh/Homebrew-on-Linux#install) через него. + +```shell +brew install kubectl + +kubectl version +``` +{{% /tab %}} +{{< /tabs >}} + +## Установка kubectl в macOS + +### Установка двоичного файла kubectl с помощью curl в macOS + +1. Загрузите последнюю версию: + + ``` + curl -LO "https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/darwin/amd64/kubectl" + ``` + + Чтобы загрузить определенную версию, вставьте в фрагмент команды `$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)` нужную версию. + + Например, команда загрузки версии {{< param "fullversion" >}} для macOS будет выглядеть следующим образом: + + ``` + curl -LO https://storage.googleapis.com/kubernetes-release/release/{{< param "fullversion" >}}/bin/darwin/amd64/kubectl + ``` + +2. Сделайте двоичный файл kubectl исполняемым: + + ``` + chmod +x ./kubectl + ``` + +3. Переместите двоичный файл в директорию из переменной окружения PATH: + + ``` + sudo mv ./kubectl /usr/local/bin/kubectl + ``` + +4. Убедитесь, что установлена последняя версия: + + ``` + kubectl version --client + ``` + +### Установка с помощью Homebrew в macOS + +Если вы используете macOS и [Homebrew](https://brew.sh/), то kubectl можно установить с помощью пакетного менеджера Homebrew. + +1. Выполните команду установки: + + ``` + brew install kubectl + ``` + + Или: + + ``` + brew install kubernetes-cli + ``` + +2. Убедитесь, что установлена последняя версия: + + ``` + kubectl version --client + ``` + +### Установка с помощью Macports в macOS + +Если вы используйте macOS и [Macports](https://macports.org/), то kubectl можно установить с помощью пакетного менеджера Macports. + +1. Выполните команду установки: + + ``` + sudo port selfupdate + sudo port install kubectl + ``` + +2. Убедитесь, что установлена последняя версия: + + ``` + kubectl version --client + ``` + +## Установка kubectl в Windows + +### Установка двоичного файла kubectl с помощью curl в Windows + +1. Загрузите последнюю версию {{< param "fullversion" >}} по [этой ссылке](https://storage.googleapis.com/kubernetes-release/release/{{< param "fullversion" >}}/bin/windows/amd64/kubectl.exe). + + Либо, если у вас установлен `curl`, выполните команду ниже: + + ``` + curl -LO https://storage.googleapis.com/kubernetes-release/release/{{< param "fullversion" >}}/bin/windows/amd64/kubectl.exe + ``` + + Последнюю стабильную версию (например, при написании скриптов) вы можете узнать из файла по ссылке [https://storage.googleapis.com/kubernetes-release/release/stable.txt](https://storage.googleapis.com/kubernetes-release/release/stable.txt). + +2. Переместите двоичный файл в директорию из переменной окружения PATH: +3. Убедитесь, что версия `kubectl` совпадает загружённой: + + ``` + kubectl version --client + ``` +{{< note >}} + +[Docker Desktop for Windows](https://docs.docker.com/docker-for-windows/#kubernetes) добавляет собственную версию `kubectl` в переменную окружения `PATH`. +Если у вас установлен Docker Desktop, вам придётся поместить путь к установленному двоичному файлу перед записью, добавленной установщиком Docker Desktop, либо же удалить вовсе `kubectl`, поставляемый вместе с Docker Desktop. +{{< /note >}} + +### Установка с помощью Powershell из PSGallery + +Если вы работаете в Windows и используете менеджер пакетов [Powershell Gallery](https://www.powershellgallery.com/), вы можете установить и обновить kubectl с помощью Powershell. + +1. Выполните команды по установке (обязательно укажите `DownloadLocation`): + + ``` + Install-Script -Name install-kubectl -Scope CurrentUser -Force + install-kubectl.ps1 [-DownloadLocation ] + ``` + + {{< note >}}Если вы не укажете `DownloadLocation`, то `kubectl` будет установлен во временную директорию пользователя.{{< /note >}} + + Установщик создаст `$HOME/.kube` вместе с конфигурационным файлом. + +2. Убедитесь, что установлена последняя версия: + + ``` + kubectl version --client + ``` + + {{< note >}}Обновить kubectl можно путём выполнения двух команд, перечисленных в шаге 1.{{< /note >}} + +### Установка в Windows с помощью Chocolatey или Scoop + +Для установки kubectl в Windows вы можете использовать либо менеджер пакетов [Chocolatey](https://chocolatey.org) , либо установщик в командной строке [Scoop](https://scoop.sh). + +{{< tabs name="kubectl_win_install" >}} +{{% tab name="choco" %}} + + choco install kubernetes-cli + +{{% /tab %}} +{{% tab name="scoop" %}} + + scoop install kubectl + +{{% /tab %}} +{{< /tabs >}} +2. Убедитесь, что установлена последняя версия: + + ``` + kubectl version --client + ``` + +3. Перейдите в домашнюю директорию: + + ``` + cd %USERPROFILE% + ``` + +4. Создайте директорию `.kube`: + + ``` + mkdir .kube + ``` + +5. Перейдите в созданную только что директорию `.kube`: + + ``` + cd .kube + ``` + +6. Настройте kubectl, чтобы возможно было использовать удаленный кластер Kubernetes: + + ``` + New-Item config -type file + ``` + + {{< note >}}Отредактируйте конфигурационный файл, используя ваш любимый текстовый редактор или обычный Notepad.{{< /note >}} + +## Установка kubectl из SDK Google Cloud + +Вы можете использовать kubectl из SDK Google Cloud, который использует этот CLI-инструмент. + +1. Установите [Google Cloud SDK](https://cloud.google.com/sdk/). +2. Выполните команду для установки `kubectl`: + + ``` + gcloud components install kubectl + ``` + +3. Убедитесь, что установлена последняя версия: + + ``` + kubectl version --client + ``` + +## Проверка конфигурации kubectl + +Чтобы kubectl мог найти и получить доступ к кластеру Kubernetes, нужен [файл kubeconfig](/docs/tasks/access-application-cluster/configure-access-multiple-clusters/), который создаётся автоматически при создании кластера с помощью скрипта [kube-up.sh](https://github.com/kubernetes/kubernetes/blob/master/cluster/kube-up.sh) или при успешном развертывании кластера Minikube. По умолчанию конфигурация kubectl находится в `~/.kube/config`. + +Посмотрите на состояние кластера, чтобы убедиться, что kubectl правильно сконфигурирован: + +```shell +kubectl cluster-info +``` + +Если вы видите URL-ответ, значит kubectl корректно настроен для работы с вашим кластером. + +Если вы видите сообщение следующего содержания, то значит kubectl настроен некорректно или не может подключиться к кластеру Kubernetes: + +```shell +The connection to the server was refused - did you specify the right host or port? +``` + +Например, если вы собираетесь запустить кластер Kubernetes на своем ноутбуке (локально), вам потребуется сначала установить специальный для этого инструмент, например Minikube, а затем снова выполнить указанные выше команды. + +Если команда `kubectl cluster-info` возвращает URL-ответ, но вы не можете подключиться к своему кластеру, чтобы убедиться, что он правильно настроен, воспользуйтесь этой командой: + +```shell +kubectl cluster-info dump +``` + +## Дополнительная конфигурация kubectl + +### Включение автодополнения ввода shell + +kubectl поддерживает автодополнение (автозаполнение) ввода в Bash и Zsh, которое сэкономит вам много времени на набор команд. + +Ниже приведены инструкции по настройке автодополнения для Bash (для Linux и macOS) и Zsh. + +{{< tabs name="kubectl_autocompletion" >}} + +{{% tab name="Bash в Linux" %}} + +### Основные сведения + +Скрипт дополнения ввода kubectl для Bash может быть сгенерирован с помощью команды `kubectl completion bash`. Подключение скрипта дополнения ввода в вашу оболочку включает поддержку автозаполнения ввода для kubectl. + +Однако скрипт дополнения ввода зависит от [**bash-completion**](https://github.com/scop/bash-completion), поэтому вам нужно сначала установить этот пакет (вы можете выполнить команду `type _init_completion`, чтобы проверить, установлен ли у вас уже bash-completion). + +### Установка bash-completion + +bash-completion можно установить через многие менеджеры пакеты (см. [здесь](https://github.com/scop/bash-completion#installation)). Вы можете установить его с помощью `apt-get install bash-completion` или `yum install bash-completion` и т.д. + +Приведенные выше команды создадут файл `/usr/share/bash-completion/bash_completion`, который является основным скриптом bash-completion. В зависимости от используемого менеджера пакетов, вы можете подключить этот файл в файле `~/.bashrc`. + +Чтобы убедиться, что этот скрипт выполняется, перезагрузите оболочку и выполните команду `type _init_completion`. Если команда отработала успешно, установка сделана правильно, в противном случае добавьте следующее содержимое в файл `~/.bashrc`: + +```shell +source /usr/share/bash-completion/bash_completion +``` + +Перезагрузите вашу оболочку и убедитесь, что bash-completion правильно установлен, напечатав в терминале `type _init_completion`. + +### Включение автодополнения ввода kubectl + +Теперь нужно убедиться, что скрипт дополнения ввода kubectl выполняется во всех сессиях командной оболочки. Есть два способа сделать это: + +- Добавьте запуск скрипта дополнения ввода в файл `~/.bashrc`: + + ```shell + echo 'source <(kubectl completion bash)' >>~/.bashrc + ``` + +- Добавьте скрипт дополнения ввода в директорию `/etc/bash_completion.d`: + + ```shell + kubectl completion bash >/etc/bash_completion.d/kubectl + ``` + +- Если у вас определён псевдоним для kubectl, вы можете интегрировать его с автодополнением оболочки: + + ```shell + echo 'alias k=kubectl' >>~/.bashrc + echo 'complete -F __start_kubectl k' >>~/.bashrc + ``` + +{{< note >}} +Все скрипты дополнения ввода bash-completion находятся в `/etc/bash_completion.d`. +{{< /note >}} + +Оба подхода эквивалентны. После перезагрузки вашей оболочки, должны появляться дополнения ввода kubectl. + +{{% /tab %}} + + +{{% tab name="Bash в macOS" %}} + + +### Основные сведения + +Скрипт дополнения ввода kubectl для Bash может быть сгенерирован с помощью команды `kubectl completion bash`. Подключение скрипта дополнения ввода в вашей оболочке включает поддержку автозаполнения ввода для kubectl. + +Однако скрипт дополнения ввода kubectl зависит от пакета [**bash-completion**](https://github.com/scop/bash-completion), который первым делом нужно установить. + +{{< warning>}} +Есть две версии bash-completion: первая (v1) и вторая (v2). Первая предназначена для Bash 3.2 (который используется по умолчанию в macOS), а вторая — для Bash 4.1+. Скрипт дополнения ввода kubectl **не работает** корректно с bash-completion v1 и Bash 3.2. Требуется **bash-completion v2** и **Bash 4.1+**. Таким образом, чтобы правильно использовать дополнение kubectl в macOS, вам нужно установить и использовать Bash 4.1+ ([*инструкции по обновлению*](https://itnext.io/upgrading-bash-on-macos-7138bd1066ba)). Последующие шаги предполагают, что вы используете Bash 4.1+ (то есть любую версию Bash 4.1 или более новую). +{{< /warning >}} + + +### Установка bash-completion + +{{< note >}} +Как уже упоминалось, в этих инструкциях предполагается, что вы используете Bash 4.1+, поэтому вы устанавливаете bash-completion v2 (а не Bash 3.2 и bash-completion v1, в таком случае дополнение ввода kubectl не будет работать). +{{< /note >}} + +Вы можете проверить, установлен ли у вас bash-completion v2, набрав команду `type _init_completion`. Если он не установлен, вы можете сделать это с помощью Homebrew: + +```shell +brew install bash-completion@2 +``` + +Как указано в выводе этой команды, добавьте следующий код в файл `~/.bashrc`: + +```shell +export BASH_COMPLETION_COMPAT_DIR="/usr/local/etc/bash_completion.d" +[[ -r "/usr/local/etc/profile.d/bash_completion.sh" ]] && . "/usr/local/etc/profile.d/bash_completion.sh" +``` + +Перезагрузите вашу командную оболочку и убедитесь, что bash-completion v2 корректно установлен, напечатав в терминале `type _init_completion`. + +### Включение автодополнения ввода kubectl + +Теперь нужно убедиться, что скрипт дополнения ввода kubectl выполняется во всех сессиях командной оболочки. Есть два способа сделать это: + +- Добавьте запуск скрипта дополнения ввода в файл `~/.bashrc`: + + ```shell + echo 'source <(kubectl completion bash)' >>~/.bashrc + ``` + +- Добавьте скрипт дополнения ввода в директорию `/etc/bash_completion.d`: + + ```shell + kubectl completion bash >/usr/local/etc/bash_completion.d/kubectl + ``` + +- Если у вас определён псевдоним для kubectl, вы можете интегрировать его с автодополнением оболочки: + + ```shell + echo 'alias k=kubectl' >>~/.bashrc + echo 'complete -F __start_kubectl k' >>~/.bashrc + ``` + +Если вы установили kubectl с помощью Homebrew (как описано [выше](#install-with-homebrew-on-macos)), то скрипт дополнения ввода kubectl уже должен быть находится в `/usr/local/etc/bash_completion.d/kubectl`. В этом случае вам не нужно ничего делать. + +{{< note >}} +Homebrew устанавливает bash-completion v2 в директорию `BASH_COMPLETION_COMPAT_DIR`, что делает рабочими два метода. +{{< /note >}} + +Какой вариант бы вы не выбрали, после перезагрузки командной оболочки, дополнение ввода kubectl должно заработать. +{{% /tab %}} + +{{% tab name="Zsh" %}} + +Скрипт дополнения ввода kubectl для Zsh может быть сгенерирован с помощью команды `kubectl completion zsh`. Подключение скрипта дополнения ввода в вашу оболочку включает поддержку автозаполнения ввода для kubectl. + +Чтобы подключить его во все сессии командной оболочки, добавьте следующую строчку в файл `~/.zshrc`: + +```shell +source <(kubectl completion zsh) +``` + +Если у вас определён псевдоним для kubectl, вы можете интегрировать его с автодополнением оболочки: + +```shell +echo 'alias k=kubectl' >>~/.zshrc +echo 'complete -F __start_kubectl k' >>~/.zshrc +``` + +После перезагрузки командной оболочки должны появляться дополнения ввода kubectl. + +Если появляется такая ошибка как `complete:13: command not found: compdef`, то добавьте следующее содержимое в начало вашего файла `~/.zshrc`: + +```shell +autoload -Uz compinit +compinit +``` +{{% /tab %}} +{{< /tabs >}} + +{{% /capture %}} + +{{% capture whatsnext %}} +* [Установка Minikube](/ru/docs/tasks/tools/install-minikube/) +* Смотрите [руководства по установке](/docs/setup/), чтобы узнать больше про создание кластеров. +* [Learn how to launch and expose your application.](/docs/tasks/access-application-cluster/service-access-application-cluster/) +* Если у вас нет доступа к кластеру, который не создавали, посмотрите страницу [Совместный доступ к кластеру](/docs/tasks/access-application-cluster/configure-access-multiple-clusters/). +* Read the [kubectl reference docs](/docs/reference/kubectl/kubectl/) +{{% /capture %}} diff --git a/content/ru/docs/tasks/tools/install-minikube.md b/content/ru/docs/tasks/tools/install-minikube.md new file mode 100644 index 0000000000..64fca93399 --- /dev/null +++ b/content/ru/docs/tasks/tools/install-minikube.md @@ -0,0 +1,255 @@ +--- +title: Установка Minikube +content_template: templates/task +weight: 20 +card: + name: tasks + weight: 10 +--- + +{{% capture overview %}} + +На этой странице рассказано, как установить [Minikube](/ru/docs/tutorials/hello-minikube), инструмент для запуска одноузлового кластера Kubernetes на виртуальной машине в персональном компьютере. + +{{% /capture %}} + +{{% capture prerequisites %}} + +{{< tabs name="minikube_before_you_begin" >}} +{{% tab name="Linux" %}} +Чтобы проверить, поддерживается ли виртуализация в Linux, выполните следующую команду и проверьте, что вывод не пустой: +``` +grep -E --color 'vmx|svm' /proc/cpuinfo +``` +{{% /tab %}} + +{{% tab name="macOS" %}} +Чтобы проверить, поддерживается ли виртуализация в macOS, выполните следующую команду в терминале: +``` +sysctl -a | grep -E --color 'machdep.cpu.features|VMX' +``` +Если вы видите `VMX` в выводе (должен быть окрашенным), значит в вашем компьютере поддерживается виртуализация VT-x. +{{% /tab %}} + +{{% tab name="Windows" %}} +Чтобы проверить, поддерживается ли виртуализация в Windows 8 и выше, выполните следующую команду в Windows Terminal или в командной строке. +``` +systeminfo +``` +Если вы видите следующий вывод, значит виртуализация поддерживается в Windows. +``` +Hyper-V Requirements: VM Monitor Mode Extensions: Yes + Virtualization Enabled In Firmware: Yes + Second Level Address Translation: Yes + Data Execution Prevention Available: Yes +``` +Если вы видите следующий вывод, значит системе уже установлен гипервизор, значит вы можете пропустить следующий шаг установке гипервизора. +``` +Hyper-V Requirements: A hypervisor has been detected. Features required for Hyper-V will not be displayed. +``` + +{{% /tab %}} +{{< /tabs >}} + +{{% /capture %}} + +{{% capture steps %}} + +# Установка minikube + +{{< tabs name="tab_with_md" >}} +{{% tab name="Linux" %}} + +### Установка kubectl + +Убедитесь, что у вас установлен kubectl. Вы можете установить kubectl согласно инструкциям в разделе [Установка и настройка kubectl](/docs/tasks/tools/install-kubectl/#install-kubectl-on-linux). + +### Установка Hypervisor + +Если у вас ещё не установлен гипервизор, установите один из них: + +• [KVM](https://www.linux-kvm.org/), который также использует QEMU + +• [VirtualBox](https://www.virtualbox.org/wiki/Downloads) + +Minikube также поддерживает опцию `--vm-driver=none`, которая запускает компоненты Kubernetes на хосте, а не на виртуальной машине. +Для использования этого драйвера требуется только [Docker](https://www.docker.com/products/docker-desktop) и Linux, но не гипервизор. + +Если вы используете драйвер `none` в Debian и его производных, используйте пакеты `.deb` для Docker, а не snap-пакет, который не работает с Minikube. +Вы можете скачать `.deb`-пакеты с сайта [Docker](https://www.docker.com/products/docker-desktop). + +{{< caution >}} +Драйвера виртуальной машины `none` может привести к проблемам безопасности и потери данных. Перед использованием `--vm-driver=none` обратитесь к [этой документации](https://minikube.sigs.k8s.io/docs/reference/drivers/none/) для получения дополнительной информации. +{{< /caution >}} + +Minikube также поддерживает `vm-driver=podman`, похожий на драйвер Docker. Podman, работающий с правами суперпользователя (пользователь root) — это лучший способ гарантировать вашим контейнерам полный доступ ко всем возможностям в системе. + +{{< caution >}} +Драйвер `podman` должен запускать контейнеры от имени суперпользователя, поскольку у обычных аккаунтов нет полного доступа ко всем возможностям операционной системы, которые могут понадобиться контейнерам для работы. +{{< /caution >}} + +### Установка Minikube через пакет + +Доступны *экспериментальные* пакеты для Minikube; Вы можете загрузить пакеты для Linux (AMD64) со страницы [релизов](https://github.com/kubernetes/minikube/releases) Minikube на GitHub. + +Используйте пакетный менеджер в вашем дистрибутиве Linux для установки нужного пакета. + +### Установка Minikube с помощью прямой ссылки + +Вы также можете загрузить двоичный файл и использовать его вместо установки пакета: + +```shell +curl -Lo minikube https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64 \ + && chmod +x minikube +``` + +Чтобы исполняемый файл Minikube был доступен из любой директории выполните следующие команды: + +```shell +sudo mkdir -p /usr/local/bin/ +sudo install minikube /usr/local/bin/ +``` + +### Установка Minikube через Homebrew + +В качестве ещё одной альтернативы вы можете установить Minikube с помощью Linux [Homebrew](https://docs.brew.sh/Homebrew-on-Linux): + +```shell +brew install minikube +``` + +{{% /tab %}} +{{% tab name="macOS" %}} +### Установка kubectl + +Убедитесь, что у вас установлен kubectl. Вы можете установить kubectl согласно инструкциям в разделе [Установка и настройка kubectl](/docs/tasks/tools/install-kubectl/#install-kubectl-on-macos). + +### Установка Hypervisor + +Если у вас ещё не установлен гипервизор, установите один из них: + +• [HyperKit](https://github.com/moby/hyperkit) + +• [VirtualBox](https://www.virtualbox.org/wiki/Downloads) + +• [VMware Fusion](https://www.vmware.com/products/fusion) + +### Установка Minikube +Простейший способ установить Minikube в macOS — использовать [Homebrew](https://brew.sh): + +```shell +brew install minikube +``` + +Вы также можете установить его в macOS, загрузив двоичный файл: + +```shell +curl -Lo minikube https://storage.googleapis.com/minikube/releases/latest/minikube-darwin-amd64 \ + && chmod +x minikube +``` + +Чтобы исполняемый файл Minikube был доступен из любой директории выполните следующие команды: + +```shell +sudo mv minikube /usr/local/bin +``` + +{{% /tab %}} +{{% tab name="Windows" %}} +### Установка kubectl + +Убедитесь, что у вас установлен kubectl. Вы можете установить kubectl согласно инструкциям в разделе [Установка и настройка kubectl](/docs/tasks/tools/install-kubectl/#install-kubectl-on-windows). + +### Установка Hypervisor + +Если у вас ещё не установлен гипервизор, установите один из них: + +• [Hyper-V](https://msdn.microsoft.com/en-us/virtualization/hyperv_on_windows/quick_start/walkthrough_install) + +• [VirtualBox](https://www.virtualbox.org/wiki/Downloads) + +{{< note >}} +Hyper-V может работать в трёх версиях Windows 10: Windows 10 Enterprise, Windows 10 Professional и Windows 10 Education. +{{< /note >}} + +### Установка Minikube с помощью Chocolatey + +Простейший способ установить Minikube в Windows — использовать [Chocolatey](https://chocolatey.org/) (запущенный с правами администратора): + +```shell +choco install minikube +``` + +После установки Minikube нужно перезапустить терминал. Minikube должен быть автоматически добавлен в директорию с исполняемыми файлами. + +### Установка Minikube с помощью исполняемого файла установки + +Для ручной установки Minikube в Windows с помощью [установщика Windows](https://docs.microsoft.com/en-us/windows/desktop/msi/windows-installer-portal), загрузите и запустите установщик [`minikube-installer.exe`](https://github.com/kubernetes/minikube/releases/latest/download/minikube-installer.exe). + +### Установка Minikube с помощью прямой ссылки + +Для ручной установки Minikube в Windows, загрузите [`minikube-windows-amd64`](https://github.com/kubernetes/minikube/releases/latest), переименуйте его в `minikube.exe` и добавьте его в директорию исполняемых файлов. + +{{% /tab %}} +{{< /tabs >}} + + +{{% /capture %}} + +{{% capture whatsnext %}} + +* [Running Kubernetes Locally via Minikube](/docs/setup/learning-environment/minikube/) + +{{% /capture %}} + +## Проверка установки + +Чтобы убедиться в том, что гипервизор и Minikube были установлены корректно, выполните следующую команду, которая запускает локальный кластер Kubernetes: + +{{< note >}} + +Для использования опции `--vm-driver` с командой `minikube start` укажите имя установленного вами гипервизора в нижнем регистре в заполнителе `` команды ниже. Полный список значений для опции `--vm-driver` перечислен в разделе по [указанию драйвера виртуальной машины](/docs/setup/learning-environment/minikube/#specifying-the-vm-driver). + +{{< /note >}} + +```shell +minikube start --vm-driver= +``` + +После того, как команда `minikube start` отработала успешно, выполните команду для проверки состояния кластера: + +```shell +minikube status +``` + +Если ваш кластер запущен, то в выводе команды `minikube status` должно быть что-то вроде этого: + +``` +host: Running +kubelet: Running +apiserver: Running +kubeconfig: Configured +``` + +Теперь, когда вы убедились, что Minikube работает с выбранным вами гипервизором, вы можете продолжить использовать Minikube или остановить кластер. Чтобы остановить кластер выполните команду ниже: + +```shell +minikube stop +``` + +## Очистка локального состояния {#cleanup-local-state} + +Если вы уже установили Minikube, то выполните следующую команду: +```shell +minikube start +``` + +В таком случае команда `minikube start` вернёт ошибку: +``` +machine does not exist +``` + +Чтобы исправить это, нужно очистить локальное состояние: +```shell +minikube delete +``` From b76520526d5ed4e002606a7887c83433be4911ed Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?J=C3=A9r=C3=A9my=20THERIN=20/=20jtherincom?= Date: Mon, 16 Mar 2020 17:16:43 +0100 Subject: [PATCH 104/140] Add link to Scaleway CCM (fr) (#19669) --- content/fr/docs/concepts/architecture/cloud-controller.md | 1 + .../fr/docs/tasks/administer-cluster/running-cloud-controller.md | 1 + 2 files changed, 2 insertions(+) diff --git a/content/fr/docs/concepts/architecture/cloud-controller.md b/content/fr/docs/concepts/architecture/cloud-controller.md index e8a513e42d..ca0542a2c3 100644 --- a/content/fr/docs/concepts/architecture/cloud-controller.md +++ b/content/fr/docs/concepts/architecture/cloud-controller.md @@ -253,6 +253,7 @@ Les fournisseurs de cloud suivants ont implémenté leur CCM: * [AWS](https://github.com/kubernetes/cloud-provider-aws) * [BaiduCloud](https://github.com/baidu/cloud-provider-baiducloud) * [Linode](https://github.com/linode/linode-cloud-controller-manager) +* [Scaleway](https://github.com/scaleway/scaleway-cloud-controller-manager) ## Administration de cluster diff --git a/content/fr/docs/tasks/administer-cluster/running-cloud-controller.md b/content/fr/docs/tasks/administer-cluster/running-cloud-controller.md index a6b6a5b820..631eda2baf 100644 --- a/content/fr/docs/tasks/administer-cluster/running-cloud-controller.md +++ b/content/fr/docs/tasks/administer-cluster/running-cloud-controller.md @@ -69,6 +69,7 @@ Pour les cloud-controller-manager ne faisant pas partie de Kubernetes, vous pouv * [keepalived](https://github.com/munnerz/keepalived-cloud-provider) * [Oracle Cloud Infrastructure](https://github.com/oracle/oci-cloud-controller-manager) * [Rancher](https://github.com/rancher/rancher-cloud-controller-manager) +* [Scaleway](https://github.com/scaleway/scaleway-cloud-controller-manager) Pour les fournisseurs qui se trouvent déjà dans Kubernetes, vous pouvez exécuter le cloud-controller-manager dans l'arborescence en tant que Daemonset dans votre cluster. Utilisez ce qui suit comme guide: From 9e0f7b4eb3039a28d64addbfc6c21fe5dddcfa32 Mon Sep 17 00:00:00 2001 From: James Spurin Date: Mon, 16 Mar 2020 21:34:30 +0000 Subject: [PATCH 105/140] update command nslookup target to include fully qualified service name, namespace picked up from running pod (#19603) --- content/en/docs/concepts/workloads/pods/init-containers.md | 4 ++-- content/fr/docs/concepts/workloads/pods/init-containers.md | 4 ++-- content/id/docs/concepts/workloads/pods/init-containers.md | 4 ++-- content/ja/docs/concepts/workloads/pods/init-containers.md | 4 ++-- content/ko/docs/concepts/workloads/pods/init-containers.md | 4 ++-- content/zh/docs/concepts/workloads/pods/init-containers.md | 4 ++-- 6 files changed, 12 insertions(+), 12 deletions(-) diff --git a/content/en/docs/concepts/workloads/pods/init-containers.md b/content/en/docs/concepts/workloads/pods/init-containers.md index ef4c5eeed9..14e7054a86 100644 --- a/content/en/docs/concepts/workloads/pods/init-containers.md +++ b/content/en/docs/concepts/workloads/pods/init-containers.md @@ -122,10 +122,10 @@ spec: initContainers: - name: init-myservice image: busybox:1.28 - command: ['sh', '-c', 'until nslookup myservice; do echo waiting for myservice; sleep 2; done;'] + command: ['sh', '-c', "until nslookup myservice.$(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace).svc.cluster.local; do echo waiting for myservice; sleep 2; done"] - name: init-mydb image: busybox:1.28 - command: ['sh', '-c', 'until nslookup mydb; do echo waiting for mydb; sleep 2; done;'] + command: ['sh', '-c', "until nslookup mydb.$(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace).svc.cluster.local; do echo waiting for mydb; sleep 2; done"] ``` You can start this Pod by running: diff --git a/content/fr/docs/concepts/workloads/pods/init-containers.md b/content/fr/docs/concepts/workloads/pods/init-containers.md index 905b9275db..c2ac521df4 100644 --- a/content/fr/docs/concepts/workloads/pods/init-containers.md +++ b/content/fr/docs/concepts/workloads/pods/init-containers.md @@ -111,10 +111,10 @@ spec: initContainers: - name: init-myservice image: busybox:1.28 - command: ['sh', '-c', 'until nslookup myservice; do echo "En attente de myservice"; sleep 2; done;'] + command: ['sh', '-c', "until nslookup myservice.$(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace).svc.cluster.local; do echo en attente de myservice; sleep 2; done"] - name: init-mydb image: busybox:1.28 - command: ['sh', '-c', 'until nslookup mydb; do echo "En attente de mydb"; sleep 2; done;'] + command: ['sh', '-c', "until nslookup mydb.$(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace).svc.cluster.local; do echo en attente de mydb; sleep 2; done"] ``` Les fichiers YAML suivants résument les services `mydb` et `myservice` : diff --git a/content/id/docs/concepts/workloads/pods/init-containers.md b/content/id/docs/concepts/workloads/pods/init-containers.md index f792e2b1df..60ce9d31ce 100644 --- a/content/id/docs/concepts/workloads/pods/init-containers.md +++ b/content/id/docs/concepts/workloads/pods/init-containers.md @@ -78,12 +78,12 @@ metadata: { "name": "init-myservice", "image": "busybox:1.28", - "command": ["sh", "-c", "until nslookup myservice; do echo waiting for myservice; sleep 2; done;"] + "command": ['sh', '-c', "until nslookup myservice.$(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace).svc.cluster.local; do echo waiting for myservice; sleep 2; done"] }, { "name": "init-mydb", "image": "busybox:1.28", - "command": ["sh", "-c", "until nslookup mydb; do echo waiting for mydb; sleep 2; done;"] + "command": ['sh', '-c', "until nslookup mydb.$(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace).svc.cluster.local; do echo waiting for mydb; sleep 2; done"] } ]' spec: diff --git a/content/ja/docs/concepts/workloads/pods/init-containers.md b/content/ja/docs/concepts/workloads/pods/init-containers.md index 9dde5bc7eb..8ba075b32c 100644 --- a/content/ja/docs/concepts/workloads/pods/init-containers.md +++ b/content/ja/docs/concepts/workloads/pods/init-containers.md @@ -76,12 +76,12 @@ metadata: { "name": "init-myservice", "image": "busybox:1.28", - "command": ["sh", "-c", "until nslookup myservice; do echo waiting for myservice; sleep 2; done;"] + "command": ['sh', '-c', "until nslookup myservice.$(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace).svc.cluster.local; do echo waiting for myservice; sleep 2; done"] }, { "name": "init-mydb", "image": "busybox:1.28", - "command": ["sh", "-c", "until nslookup mydb; do echo waiting for mydb; sleep 2; done;"] + "command": ['sh', '-c', "until nslookup mydb.$(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace).svc.cluster.local; do echo waiting for mydb; sleep 2; done"] } ]' spec: diff --git a/content/ko/docs/concepts/workloads/pods/init-containers.md b/content/ko/docs/concepts/workloads/pods/init-containers.md index 19dabdd03c..9e90cb0d77 100644 --- a/content/ko/docs/concepts/workloads/pods/init-containers.md +++ b/content/ko/docs/concepts/workloads/pods/init-containers.md @@ -120,10 +120,10 @@ spec: initContainers: - name: init-myservice image: busybox:1.28 - command: ['sh', '-c', 'until nslookup myservice; do echo waiting for myservice; sleep 2; done;'] + command: ['sh', '-c', "until nslookup myservice.$(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace).svc.cluster.local; do echo waiting for myservice; sleep 2; done"] - name: init-mydb image: busybox:1.28 - command: ['sh', '-c', 'until nslookup mydb; do echo waiting for mydb; sleep 2; done;'] + command: ['sh', '-c', "until nslookup mydb.$(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace).svc.cluster.local; do echo waiting for mydb; sleep 2; done"] ``` 다음 커맨드들을 이용하여 파드를 시작하거나 디버깅할 수 있다. diff --git a/content/zh/docs/concepts/workloads/pods/init-containers.md b/content/zh/docs/concepts/workloads/pods/init-containers.md index 31ff8dd202..2ea79d197b 100644 --- a/content/zh/docs/concepts/workloads/pods/init-containers.md +++ b/content/zh/docs/concepts/workloads/pods/init-containers.md @@ -186,10 +186,10 @@ spec: initContainers: - name: init-myservice image: busybox:1.28 - command: ['sh', '-c', 'until nslookup myservice; do echo waiting for myservice; sleep 2; done;'] + command: ['sh', '-c', "until nslookup myservice.$(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace).svc.cluster.local; do echo waiting for myservice; sleep 2; done"] - name: init-mydb image: busybox:1.28 - command: ['sh', '-c', 'until nslookup mydb; do echo waiting for mydb; sleep 2; done;'] + command: ['sh', '-c', "until nslookup mydb.$(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace).svc.cluster.local; do echo waiting for mydb; sleep 2; done"] ``` 下面的 yaml 文件展示了 `mydb` 和 `myservice` 两个 Service： From 763c5a4216adff646d9b7dc5c57781334f0c8f76 Mon Sep 17 00:00:00 2001 From: Dian Zhou <1104127978@qq.com> Date: Tue, 17 Mar 2020 11:02:22 +0800 Subject: [PATCH 106/140] fix chinese version kubeadm-join.md (#19648) fix a small bug during translation. create--print-join-command > create --print-join-command --- content/zh/docs/reference/setup-tools/kubeadm/kubeadm-join.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/zh/docs/reference/setup-tools/kubeadm/kubeadm-join.md b/content/zh/docs/reference/setup-tools/kubeadm/kubeadm-join.md index 00ba1ff7e1..6e6fd76fa4 100644 --- a/content/zh/docs/reference/setup-tools/kubeadm/kubeadm-join.md +++ b/content/zh/docs/reference/setup-tools/kubeadm/kubeadm-join.md @@ -101,7 +101,7 @@ The CA key hash has the format `sha256:`. By default, the hash --> CA 键哈希格式为 `sha256:`。 -默认情况下，在 `kubeadm init` 最后打印的 `kubeadm join` 命令或者 `kubeadm token create--print-join-command` 的输出信息中返回哈希值。 +默认情况下，在 `kubeadm init` 最后打印的 `kubeadm join` 命令或者 `kubeadm token create --print-join-command` 的输出信息中返回哈希值。 它使用标准格式 (请参考 [RFC7469](https://tools.ietf.org/html/rfc7469#section-2.4)) 并且也能通过第三方工具或者驱动系统进行计算。 例如，使用 OpenSSL CLI： From a97e5ef28715a74d93a96f9c3ead15df50ec148c Mon Sep 17 00:00:00 2001 From: Alexey Pyltsyn Date: Tue, 17 Mar 2020 12:28:22 +0300 Subject: [PATCH 107/140] Translate Kubernetes Components page into Russian (#19657) * Translate Kubernetes Components page into Russian * Apply suggestions from code review Co-Authored-By: Nikita Potapenko * Update service.md * Update kube-scheduler.md * Update control-plane.md * Update kube-scheduler.md Co-authored-by: Nikita Potapenko --- .../ru/docs/concepts/overview/components.md | 118 ++++++++++++++++++ content/ru/docs/reference/glossary/cluster.md | 10 +- .../reference/glossary/container-runtime.md | 21 ++++ .../ru/docs/reference/glossary/container.md | 18 +++ .../ru/docs/reference/glossary/containerd.md | 17 +++ .../docs/reference/glossary/control-plane.md | 13 ++ content/ru/docs/reference/glossary/cri-o.md | 19 +++ content/ru/docs/reference/glossary/cri.md | 17 +++ .../ru/docs/reference/glossary/daemonset.md | 19 +++ .../ru/docs/reference/glossary/deployment.md | 20 +++ content/ru/docs/reference/glossary/docker.md | 17 +++ content/ru/docs/reference/glossary/etcd.md | 20 +++ .../docs/reference/glossary/kube-apiserver.md | 23 ++++ .../glossary/kube-controller-manager.md | 18 +++ .../ru/docs/reference/glossary/kube-proxy.md | 20 +++ .../docs/reference/glossary/kube-scheduler.md | 19 +++ content/ru/docs/reference/glossary/kubelet.md | 18 +++ content/ru/docs/reference/glossary/label.md | 17 +++ content/ru/docs/reference/glossary/node.md | 17 +++ content/ru/docs/reference/glossary/pod.md | 18 +++ .../ru/docs/reference/glossary/selector.md | 18 +++ content/ru/docs/reference/glossary/service.md | 18 +++ 22 files changed, 490 insertions(+), 5 deletions(-) create mode 100644 content/ru/docs/concepts/overview/components.md create mode 100644 content/ru/docs/reference/glossary/container-runtime.md create mode 100755 content/ru/docs/reference/glossary/container.md create mode 100644 content/ru/docs/reference/glossary/containerd.md create mode 100644 content/ru/docs/reference/glossary/control-plane.md create mode 100644 content/ru/docs/reference/glossary/cri-o.md create mode 100644 content/ru/docs/reference/glossary/cri.md create mode 100755 content/ru/docs/reference/glossary/daemonset.md create mode 100755 content/ru/docs/reference/glossary/deployment.md create mode 100755 content/ru/docs/reference/glossary/docker.md create mode 100755 content/ru/docs/reference/glossary/etcd.md create mode 100755 content/ru/docs/reference/glossary/kube-apiserver.md create mode 100755 content/ru/docs/reference/glossary/kube-controller-manager.md create mode 100755 content/ru/docs/reference/glossary/kube-proxy.md create mode 100755 content/ru/docs/reference/glossary/kube-scheduler.md create mode 100755 content/ru/docs/reference/glossary/kubelet.md create mode 100755 content/ru/docs/reference/glossary/label.md create mode 100755 content/ru/docs/reference/glossary/node.md create mode 100755 content/ru/docs/reference/glossary/pod.md create mode 100755 content/ru/docs/reference/glossary/selector.md create mode 100755 content/ru/docs/reference/glossary/service.md diff --git a/content/ru/docs/concepts/overview/components.md b/content/ru/docs/concepts/overview/components.md new file mode 100644 index 0000000000..9689c2e1fd --- /dev/null +++ b/content/ru/docs/concepts/overview/components.md @@ -0,0 +1,118 @@ +--- +reviewers: +- lavalamp +title: Компоненты Kubernetes +content_template: templates/concept +weight: 20 +card: + name: concepts + weight: 20 +--- + +{{% capture overview %}} +При развёртывании Kubernetes вы имеете дело с кластером. +{{< glossary_definition term_id="cluster" length="all" prepend="Кластер Kubernetes cluster состоит из">}} + +На этой странице в общих чертах описывается различные компоненты, необходимые для работы кластера Kubernetes. + +Ниже показана диаграмма кластера Kubernetes со всеми связанными компонентами. + +![Компоненты Kubernetes](/images/docs/components-of-kubernetes.png) + +{{% /capture %}} + +{{% capture body %}} + +## Панель управления компонентами + +Компоненты панели управления отвечают за основные операции кластера (например, планирование), а также обрабатывают события кластера (например, запускают новый {{< glossary_tooltip text="под" term_id="pod">}}, когда поле `replicas` развертывания не соответствует требуемому количеству реплик). + +Компоненты панели управления могут быть запущены на любой машине в кластере. Однако для простоты сценарии настройки обычно запускают все компоненты панели управления на одном компьютере и в то же время не позволяют запускать пользовательские контейнеры на этом компьютере. Смотрите страницу [Создание высоконадёжных кластеров](/docs/admin/high-availability/) для примера настройки нескольких ведущих виртуальных машин. + +### kube-apiserver + +{{< glossary_definition term_id="kube-apiserver" length="all" >}} + +### etcd + +{{< glossary_definition term_id="etcd" length="all" >}} + +### kube-scheduler + +{{< glossary_definition term_id="kube-scheduler" length="all" >}} + +### kube-controller-manager + +{{< glossary_definition term_id="kube-controller-manager" length="all" >}} + +Эти контроллеры включают: + + * Контроллер узла (Node Controller): уведомляет и реагирует на сбои узла. + * Контроллер репликации (Replication Controller): поддерживает правильное количество подов для каждого объекта контроллера репликации в системе. + * Контроллер конечных точек (Endpoints Controller): заполняет объект конечных точек (Endpoints), то есть связывает сервисы (Services) и поды (Pods). + * Контроллеры учетных записей и токенов (Account & Token Controllers): создают стандартные учетные записи и токены доступа API для новых пространств имен. + +### cloud-controller-manager + +[cloud-controller-manager](/docs/tasks/administer-cluster/running-cloud-controller/) запускает контроллеры, которые взаимодействуют с основными облачными провайдерами. Двоичный файл cloud-controller-manager — это альфа-функциональность, появившиеся в Kubernetes 1.6. + +cloud-controller-manager запускает только циклы контроллера, относящиеся к облачному провайдеру. Вам нужно отключить эти циклы контроллера в kube-controller-manager. Вы можете отключить циклы контроллера, установив флаг `--cloud-provider` со значением `external` при запуске kube-controller-manager. + +С помощью cloud-controller-manager код как облачных провайдеров, так и самого Kubernetes может разрабатываться независимо друг от друга. В предыдущих версиях код ядра Kubernetes зависел от кода, предназначенного для функциональности облачных провайдеров. В будущих выпусках код, специфичный для облачных провайдеров, должен поддерживаться самим облачным провайдером и компоноваться с cloud-controller-manager во время запуска Kubernetes. + +Следующие контроллеры зависят от облачных провайдеров: + + * Контроллер узла (Node Controller): проверяет облачный провайдер, чтобы определить, был ли удален узел в облаке после того, как он перестал работать + * Контроллер маршрутов (Route Controller): настраивает маршруты в основной инфраструктуре облака + * Контроллер сервисов (Service Controller): создаёт, обновляет и удаляет балансировщики нагрузки облачного провайдера. + * Контроллер тома (Volume Controller): создаёт, присоединяет и монтирует тома, а также взаимодействует с облачным провайдером для оркестрации томов. + +## Компоненты узла + +Компоненты узла работают на каждом узле, поддерживая работу подов и среды выполнения Kubernetes. + +### kubelet + +{{< glossary_definition term_id="kubelet" length="all" >}} + +### kube-proxy + +{{< glossary_definition term_id="kube-proxy" length="all" >}} + +### Среда выполнения контейнера + +{{< glossary_definition term_id="container-runtime" length="all" >}} + +## Дополнения + +Дополнения используют ресурсы Kubernetes ({{< glossary_tooltip term_id="daemonset" >}}, {{< glossary_tooltip term_id="deployment" >}} и т.д.) для расширения функциональности кластера. Поскольку дополнения охватывают весь кластер, ресурсы относятся к пространству имен `kube-system`. + +Некоторые из дополнений описаны ниже; более подробный список доступных расширений вы можете найти на странице [Дополнения](/docs/concepts/cluster-administration/addons/). + +### DNS + +Хотя прочие дополнения не являются строго обязательными, однако при этом у всех Kubernetes-кластеров должен быть [кластерный DNS](/docs/concepts/services-networking/dns-pod-service/), так как многие примеры предполагают его наличие. + +Кластерный DNS — это DNS-сервер наряду с другими DNS-серверами в вашем окружении, который обновляет DNS-записи для сервисов Kubernetes. + +Контейнеры, запущенные посредством Kubernetes, автоматически включают этот DNS-сервер в свои DNS. + +### Веб-интерфейс (Dashboard) + +[Dashboard](/docs/tasks/access-application-cluster/web-ui-dashboard/) — это универсальный веб-интерфейс для кластеров Kubernetes. С помощью этой панели, пользователи могут управлять и устранять неполадки кластера и приложений, работающих в кластере. + +### Мониторинг ресурсов контейнера + +[Мониторинг ресурсов контейнера](/docs/tasks/debug-application-cluster/resource-usage-monitoring/) записывает общие метрики о контейнерах в виде временных рядов в центральной базе данных и предлагает пользовательский интерфейс для просмотра этих данных. + +### Логирование кластера + +Механизм [логирования кластера](/docs/concepts/cluster-administration/logging/) отвечает за сохранение логов контейнера в централизованном хранилище логов с возможностью их поиска/просмотра. + +{{% /capture %}} +{{% capture whatsnext %}} +* Подробнее про [узлы](/docs/concepts/architecture/nodes/) +* Подробнее про [контроллеры](/docs/concepts/architecture/controller/) +* Подробнее про [kube-scheduler](/docs/concepts/scheduling/kube-scheduler/) +* Официальная [документация](https://etcd.io/docs/) etcd +{{% /capture %}} diff --git a/content/ru/docs/reference/glossary/cluster.md b/content/ru/docs/reference/glossary/cluster.md index 614b42dd98..79e10c8fcc 100644 --- a/content/ru/docs/reference/glossary/cluster.md +++ b/content/ru/docs/reference/glossary/cluster.md @@ -2,16 +2,16 @@ title: Кластер id: cluster date: 2019-06-15 -full_link: +full_link: short_description: > - Набор машин, называемых нодами, которые запускают контейнерные приложения, управляемые Kubernetes. Кластер имеет как минимум одну рабочую ноду и хотя бы одну главную ноду. + Набор машин, так называемые узлы, которые запускают контейнеризированные приложения. Кластер имеет как минимум один рабочий узел. -aka: +aka: tags: - fundamental - operation --- -Набор машин, называемых нодами, которые запускают контейнерные приложения, управляемые Kubernetes. Кластер имеет как минимум одну рабочую ноду и хотя бы одну главную ноду. +Набор машин, так называемые узлы, которые запускают контейнеризированные приложения. Кластер имеет как минимум один рабочий узел. -Родительские ноды содержат дочерние ноды, являющиеся компонентами приложения. Главные ноды управляют родительскими нодами и модулями в кластере. Несколько главных нод используются для обеспечения отказоустойчивости кластера и высокой доступности. +В рабочих узлах размещены поды, являющиеся компонентами приложения. Панель управления управляет рабочими узлами и подами в кластере. В промышленных средах панель управления обычно запускается на нескольких компьютерах, а кластер, как правило, развёртывается на нескольких узлах, гарантируя отказоустойчивость и высокую надёжность. diff --git a/content/ru/docs/reference/glossary/container-runtime.md b/content/ru/docs/reference/glossary/container-runtime.md new file mode 100644 index 0000000000..cea63dfa94 --- /dev/null +++ b/content/ru/docs/reference/glossary/container-runtime.md @@ -0,0 +1,21 @@ +--- +title: Среда выполнения контейнера +id: container-runtime +date: 2019-06-05 +full_link: /docs/reference/generated/container-runtime +short_description: > + Среда выполнения контейнера — это программа, предназначенная для выполнения контейнеров. + +aka: +tags: +- fundamental +- workload +--- + Среда выполнения контейнера — это программа, предназначенная для выполнения контейнеров. + + + +Kubernetes поддерживает несколько сред для запуска контейнеров: {{< glossary_tooltip term_id="docker">}}, +{{< glossary_tooltip term_id="containerd" >}}, {{< glossary_tooltip term_id="cri-o" >}}, +и любая реализация [Kubernetes CRI (Container Runtime +Interface)](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-node/container-runtime-interface.md). diff --git a/content/ru/docs/reference/glossary/container.md b/content/ru/docs/reference/glossary/container.md new file mode 100755 index 0000000000..7c2e66e4d1 --- /dev/null +++ b/content/ru/docs/reference/glossary/container.md @@ -0,0 +1,18 @@ +--- +title: Контейнер +id: container +date: 2018-04-12 +full_link: /docs/concepts/overview/what-is-kubernetes/#why-containers +short_description: > + Переносимый и не требовательный к ресурсам исполняемый экземпляр образа, содержащий приложение вместе со всеми его зависимостями. + +aka: +tags: +- fundamental +- workload +--- + Переносимый и не требовательный к ресурсам исполняемый экземпляр образа, содержащий приложение вместе со всеми его зависимостями. + + + +Контейнеры изолирует приложения от инфраструктуры хост-машины, чтобы обеспечить простое масштабирование и упростить развёртывание в различных средах облачных платформ или операционных систем. diff --git a/content/ru/docs/reference/glossary/containerd.md b/content/ru/docs/reference/glossary/containerd.md new file mode 100644 index 0000000000..251bebcd20 --- /dev/null +++ b/content/ru/docs/reference/glossary/containerd.md @@ -0,0 +1,17 @@ +--- +title: containerd +id: containerd +date: 2019-05-14 +full_link: https://containerd.io/docs/ +short_description: > + Среда выполнения контейнера с упором на простоту, надежность и переносимость + +aka: +tags: +- tool +--- + Среда выполнения контейнера с упором на простоту, надежность и переносимость + + + +containerd — среда выполнения {{< glossary_tooltip text="контейнера" term_id="container" >}}, который представляет собой демон для Linux или Windows. containerd заботится о получении и хранении образов контейнеров, запуске контейнеров, осуществлять доступ по сети и т.д. diff --git a/content/ru/docs/reference/glossary/control-plane.md b/content/ru/docs/reference/glossary/control-plane.md new file mode 100644 index 0000000000..a4565af381 --- /dev/null +++ b/content/ru/docs/reference/glossary/control-plane.md @@ -0,0 +1,13 @@ +--- +title: Плоскость управления (Control Plane) +id: control-plane +date: 2019-05-12 +full_link: +short_description: > + Уровень оркестрации контейнеров с API и интерфейсами для определения, развёртывания и управления жизненным циклом контейнеров. + +aka: +tags: +- fundamental +--- + Уровень оркестрации контейнеров с API и интерфейсами для определения, развёртывания и управления жизненным циклом контейнеров. diff --git a/content/ru/docs/reference/glossary/cri-o.md b/content/ru/docs/reference/glossary/cri-o.md new file mode 100644 index 0000000000..1b99ac947c --- /dev/null +++ b/content/ru/docs/reference/glossary/cri-o.md @@ -0,0 +1,19 @@ +--- +title: CRI-O +id: cri-o +date: 2019-05-14 +full_link: https://cri-o.io/#what-is-cri-o +short_description: > + Оптимизированная среда выполнения контейнеров, разработанная специально для Kubernetes + +aka: +tags: +- tool +--- +Инструмент, позволяющий использовать среды выполнения контейнеров формата OCI с помощью технологии Kubernetes CRI. + + + +CRI-O — это реализация {{< glossary_tooltip term_id="cri" >}}, позволяющая использовать среды выполнения {{< glossary_tooltip text="контейнеров" term_id="container" >}}, совместимые со [спецификацией исполняемой среды контейнеров](http://www.github.com/opencontainers/runtime-spec) Open Container Initiative (OCI). + +Развертывание CRI-O позволяет Kubernetes использовать любую OCI-совместимую среду выполнения в качестве контейнерной среды выполнения для выполнения {{< glossary_tooltip text="подов" term_id="pod" >}} и загружать образы OCI-контейнера из удаленных реестров. diff --git a/content/ru/docs/reference/glossary/cri.md b/content/ru/docs/reference/glossary/cri.md new file mode 100644 index 0000000000..2f47ddc915 --- /dev/null +++ b/content/ru/docs/reference/glossary/cri.md @@ -0,0 +1,17 @@ +--- +title: Container runtime interface (CRI) +id: cri +date: 2019-03-07 +full_link: /docs/concepts/overview/components/#container-runtime +short_description: > + API сред выполнения контейнеров для интеграции с kubelet + +aka: +tags: +- fundamental +--- +Интерфейс среды выполнения контейнера (Container Runtime Interface, CRI) — это API сред выполнения контейнера, которая интегрируется с kubelet на узле. + + + +Для получения дополнительной информации смотрите API и спефикации [CRI](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-node/container-runtime-interface.md). diff --git a/content/ru/docs/reference/glossary/daemonset.md b/content/ru/docs/reference/glossary/daemonset.md new file mode 100755 index 0000000000..b0e4ca6195 --- /dev/null +++ b/content/ru/docs/reference/glossary/daemonset.md @@ -0,0 +1,19 @@ +--- +title: DaemonSet +id: daemonset +date: 2018-04-12 +full_link: /docs/concepts/workloads/controllers/daemonset +short_description: > + Гарантирует, что копия Pod выполняется в наборе узлов кластера. + +aka: +tags: +- fundamental +- core-object +- workload +--- + Гарантирует, что копия {{< glossary_tooltip text="Pod" term_id="pod" >}} выполняется в наборе узлов {{< glossary_tooltip text="кластера" term_id="cluster" >}}. + + + +Используется для развертывания системных демонов, таких как сборщики логов и агенты мониторинга, которые, как правило, должны работать на каждом {{< glossary_tooltip text="узла" term_id="node" >}}. diff --git a/content/ru/docs/reference/glossary/deployment.md b/content/ru/docs/reference/glossary/deployment.md new file mode 100755 index 0000000000..26c9aedcd8 --- /dev/null +++ b/content/ru/docs/reference/glossary/deployment.md @@ -0,0 +1,20 @@ +--- +title: Deployment +id: deployment +date: 2018-04-12 +full_link: /docs/concepts/workloads/controllers/deployment/ +short_description: > + API-объект, управляющий реплицированным приложением. + +aka: +tags: +- fundamental +- core-object +- workload +--- + API-объект, управляющий реплицированным приложением. + + + +Каждая реплика представляет {{< glossary_tooltip term_id="pod" >}}, а все Pod-объекты распределяются по узлам кластера. + diff --git a/content/ru/docs/reference/glossary/docker.md b/content/ru/docs/reference/glossary/docker.md new file mode 100755 index 0000000000..290b6d3c60 --- /dev/null +++ b/content/ru/docs/reference/glossary/docker.md @@ -0,0 +1,17 @@ +--- +title: Docker +id: docker +date: 2018-04-12 +full_link: https://docs.docker.com/engine/ +short_description: > + Docker — это программное обеспечение для виртуализации на уровне операционной системы, которая известна как контейнеризация. + +aka: +tags: +- fundamental +--- +Docker (в частности, Docker Engine) — это программное обеспечение для виртуализации на уровне операционной системы, которая также известна как {{< glossary_tooltip text="контейнеризация" term_id="container" >}}. + + + +Docker использует возможности изоляции ресурсов ядра Linux, такие как cgroups и пространства имен ядра, а также каскадно-объединённую файловую систему, например, OverlayFS и другие, чтобы независимые друг от друга контейнеры могли работать в одном экземпляре Linux без накладных расходов на запуск и поддержку работы виртуальных машин (VM). diff --git a/content/ru/docs/reference/glossary/etcd.md b/content/ru/docs/reference/glossary/etcd.md new file mode 100755 index 0000000000..cb3ac286d7 --- /dev/null +++ b/content/ru/docs/reference/glossary/etcd.md @@ -0,0 +1,20 @@ +--- +title: etcd +id: etcd +date: 2018-04-12 +full_link: /docs/tasks/administer-cluster/configure-upgrade-etcd/ +short_description: > + Распределённое и высоконадёжное хранилище данных в формате "ключ-значение", которое используется как основное хранилище всех данных кластера в Kubernetes. + +aka: +tags: +- architecture +- storage +--- +Распределённое и высоконадёжное хранилище данных в формате "ключ-значение", которое используется как основное хранилище всех данных кластера в Kubernetes. + + + +Если ваш кластер Kubernetes использует etcd в качестве основного хранилища, убедитесь, что у вас [настроено резервное копирование](/docs/tasks/administer-cluster/configure-upgrade-etcd/#backing-up-an-etcd-cluster) данных. + +Вы можете найти подробную информацию о etcd в [официальной документации](https://etcd.io/docs/). diff --git a/content/ru/docs/reference/glossary/kube-apiserver.md b/content/ru/docs/reference/glossary/kube-apiserver.md new file mode 100755 index 0000000000..24d3a33587 --- /dev/null +++ b/content/ru/docs/reference/glossary/kube-apiserver.md @@ -0,0 +1,23 @@ +--- +title: API-сервер +id: kube-apiserver +date: 2018-04-12 +full_link: /docs/reference/generated/kube-apiserver/ +short_description: > + Компонент панели управления, обслуживающий API Kubernetes. + +aka: +- kube-apiserver +tags: +- architecture +- fundamental +--- + Сервер API — компонент Kubernetes +{{< glossary_tooltip text="панели управления" term_id="control-plane" >}}, который представляет API Kubernetes. +API-сервер — это клиентская часть панели управления Kubernetes + + + +Основной реализацией API-сервера Kubernetes является [kube-apiserver](/docs/reference/generated/kube-apiserver/). +kube-apiserver предназначен для горизонтального масштабирования, то есть развёртывание на несколько экземпляров. +Вы можете запустить несколько экземпляров kube-apiserver и сбалансировать трафик между этими экземплярами. diff --git a/content/ru/docs/reference/glossary/kube-controller-manager.md b/content/ru/docs/reference/glossary/kube-controller-manager.md new file mode 100755 index 0000000000..d1cb711164 --- /dev/null +++ b/content/ru/docs/reference/glossary/kube-controller-manager.md @@ -0,0 +1,18 @@ +--- +title: kube-controller-manager +id: kube-controller-manager +date: 2018-04-12 +full_link: /docs/reference/command-line-tools-reference/kube-controller-manager/ +short_description: > + Компонент Control Plane запускает процессы контроллера. + +aka: +tags: +- architecture +- fundamental +--- + Компонент Control Plane запускает процессы {{< glossary_tooltip text="контроллера" term_id="controller" >}}. + + + +Вполне логично, что каждый {{< glossary_tooltip text="контроллер" term_id="controller" >}} в свою очередь представляет собой отдельный процесс, и для упрощения все такие процессы скомпилированы в один двоичный файл и выполняются в одном процессе. diff --git a/content/ru/docs/reference/glossary/kube-proxy.md b/content/ru/docs/reference/glossary/kube-proxy.md new file mode 100755 index 0000000000..2858f60403 --- /dev/null +++ b/content/ru/docs/reference/glossary/kube-proxy.md @@ -0,0 +1,20 @@ +--- +title: kube-proxy +id: kube-proxy +date: 2018-04-12 +full_link: /docs/reference/command-line-tools-reference/kube-proxy/ +short_description: > + `kube-proxy` — сетевой прокси, работающий на каждом узле в кластере. + +aka: +tags: +- fundamental +- networking +--- + [kube-proxy](/docs/reference/command-line-tools-reference/kube-proxy/) — сетевой прокси, работающий на каждом узле в кластере, и реализующий часть концепции {{< glossary_tooltip text="сервис" term_id="service">}}. + + + +kube-proxy конфигурирует правила сети на узлах. При помощи них разрешаются сетевые подключения к вашими подам изнутри и снаружи кластера. + +kube-proxy использует уровень фильтрации пакетов в операционной системы, если он доступен. В противном случае, kube-proxy сам обрабатывает передачу сетевого трафика. \ No newline at end of file diff --git a/content/ru/docs/reference/glossary/kube-scheduler.md b/content/ru/docs/reference/glossary/kube-scheduler.md new file mode 100755 index 0000000000..4cf084ea0f --- /dev/null +++ b/content/ru/docs/reference/glossary/kube-scheduler.md @@ -0,0 +1,19 @@ +--- +title: kube-scheduler +id: kube-scheduler +date: 2018-04-12 +full_link: /docs/reference/generated/kube-scheduler/ +short_description: > + Компонент плоскости управления, который отслеживает созданные поды без привязанного узла и выбирает узел, на котором они должны работать. + +aka: +tags: +- architecture +--- + Компонент плоскости управления, который отслеживает созданные поды без привязанного узла и выбирает узел, на котором они должны работать. + + + + + +При планировании развёртывания подов на узлах учитываются множество факторов, включая требования к ресурсам, ограничения, связанные с аппаратными/программными политиками, принадлежности (affinity) и непринадлежности (anti-affinity) узлов/подов, местонахождения данных, предельных сроков. diff --git a/content/ru/docs/reference/glossary/kubelet.md b/content/ru/docs/reference/glossary/kubelet.md new file mode 100755 index 0000000000..0aadf8ed29 --- /dev/null +++ b/content/ru/docs/reference/glossary/kubelet.md @@ -0,0 +1,18 @@ +--- +title: Kubelet +id: kubelet +date: 2018-04-12 +full_link: /docs/reference/generated/kubelet +short_description: > + Агент, работающий на каждом узле в кластере. Он следит за тем, чтобы контейнеры были запущены в поде. + +aka: +tags: +- fundamental +- core-object +--- + Агент, работающий на каждом узле в кластере. Он следит за тем, чтобы контейнеры были запущены в поде. + + + +Утилита kubelet принимает набор PodSpecs, и гарантирует работоспособность и исправность определённых в них контейнеров. Агент kubelet не отвечает за контейнеры, не созданные Kubernetes. diff --git a/content/ru/docs/reference/glossary/label.md b/content/ru/docs/reference/glossary/label.md new file mode 100755 index 0000000000..287fd32528 --- /dev/null +++ b/content/ru/docs/reference/glossary/label.md @@ -0,0 +1,17 @@ +--- +title: Метка +id: label +date: 2018-04-12 +full_link: /docs/concepts/overview/working-with-objects/labels +short_description: > + Группирует объекты на основе произвольных критериев, по которым пользователи могут их идентифицировать. + +aka: +tags: +- fundamental +--- + Группирует объекты на основе произвольных критериев, по которым пользователи могут их идентифицировать. + + + +Метки — это пары "ключ-значение", которые прикрепляются к таким объектам, как {{< glossary_tooltip text="Pod" term_id="pod" >}}. Они используются для организации и получения подмножеств объектов. diff --git a/content/ru/docs/reference/glossary/node.md b/content/ru/docs/reference/glossary/node.md new file mode 100755 index 0000000000..0a2cc77e62 --- /dev/null +++ b/content/ru/docs/reference/glossary/node.md @@ -0,0 +1,17 @@ +--- +title: Node +id: node +date: 2018-04-12 +full_link: /docs/concepts/architecture/nodes/ +short_description: > + Узел — рабочая машина в Kubernetes. + +aka: +tags: +- fundamental +--- + Узел — рабочая машина в Kubernetes. + + + +Рабочий узел может быть как виртуальной, так и физической машиной, в зависимости от кластера. У него есть локальные демоны или сервисы, необходимые для запуска {{< glossary_tooltip text="подов" term_id="pod" >}}, а сам он управляется панелью управления. Демоны на узле включают в себя {{< glossary_tooltip text="kubelet" term_id="kubelet" >}}, {{< glossary_tooltip text="kube-proxy" term_id="kube-proxy" >}} и среду выполнения контейнера, основанную на {{< glossary_tooltip text="CRI" term_id="cri" >}}, например {{< glossary_tooltip term_id="docker" >}}. diff --git a/content/ru/docs/reference/glossary/pod.md b/content/ru/docs/reference/glossary/pod.md new file mode 100755 index 0000000000..d00f3225a4 --- /dev/null +++ b/content/ru/docs/reference/glossary/pod.md @@ -0,0 +1,18 @@ +--- +title: Pod +id: pod +date: 2018-04-12 +full_link: /docs/concepts/workloads/pods/pod-overview/ +short_description: > + Самый маленький и простой объект в Kubernetes. Под — это набор запущенных контейнеров в кластере. + +aka: +tags: +- core-object +- fundamental +--- + Самый маленький и простой объект в Kubernetes. Объект Pod — набор запущенных {{< glossary_tooltip text="контейнеров" term_id="container" >}} в кластере. + + + +Как правило, один под предназначен для выполнения одного основного контейнера. Он также может запускать дополнительные "прицепные" (sidecar) контейнеры, добавляющие новые функциональные возможности, например, логирование. Контейнеры обычно управляются {{< glossary_tooltip term_id="deployment" >}}. diff --git a/content/ru/docs/reference/glossary/selector.md b/content/ru/docs/reference/glossary/selector.md new file mode 100755 index 0000000000..4eef4299f0 --- /dev/null +++ b/content/ru/docs/reference/glossary/selector.md @@ -0,0 +1,18 @@ +--- +title: Селектор +id: selector +date: 2018-04-12 +full_link: /docs/concepts/overview/working-with-objects/labels/ +short_description: > + Позволяет пользователям фильтровать список ресурсов по меткам. + +aka: +tags: +- fundamental +--- + Позволяет пользователям фильтровать список ресурсов по меткам. + + + +Селекторы применяются при создании запросов для фильтрации списков ресурсов по {{< glossary_tooltip text="меткам" term_id="label" >}}. + diff --git a/content/ru/docs/reference/glossary/service.md b/content/ru/docs/reference/glossary/service.md new file mode 100755 index 0000000000..1f4d4736c1 --- /dev/null +++ b/content/ru/docs/reference/glossary/service.md @@ -0,0 +1,18 @@ +--- +title: Сервис (Service) +id: service +date: 2018-04-12 +full_link: /docs/concepts/services-networking/service/ +short_description: > + Способ представления приложения, запущенного в наборе подов, в виде сетевого сервиса. + +aka: +tags: +- fundamental +- core-object +--- +Абстрактный способ представления приложения, запущенного в наборе {{< glossary_tooltip text="подов" term_id="pod" >}}, в виде сетевого сервиса. + + + +Набор подов, из которых состоит сервис, определяется (как правило) {{< glossary_tooltip text="селектором" term_id="selector" >}}. При добавлении или удалении подов, набор подов, соответствующий селектору, изменится. Сервис обеспечивает, что сетевой трафик может быть направлен на текущий набор подов для планирования рабочей нагрузки. From 64f9b39d33377cb9522138dc86e78bafab3f2aab Mon Sep 17 00:00:00 2001 From: Jacky Wu Date: Tue, 17 Mar 2020 21:32:22 +0800 Subject: [PATCH 108/140] fix: remove the redirect_from settings. (#19645) --- content/zh/docs/concepts/policy/pod-security-policy.md | 3 --- content/zh/docs/concepts/workloads/pods/init-containers.md | 5 ----- content/zh/docs/concepts/workloads/pods/pod-lifecycle.md | 3 --- .../communicate-containers-same-pod-shared-volume.md | 5 ----- .../docs/tasks/administer-cluster/access-cluster-services.md | 3 --- 5 files changed, 19 deletions(-) diff --git a/content/zh/docs/concepts/policy/pod-security-policy.md b/content/zh/docs/concepts/policy/pod-security-policy.md index 3f2717ac44..1573ff7af2 100644 --- a/content/zh/docs/concepts/policy/pod-security-policy.md +++ b/content/zh/docs/concepts/policy/pod-security-policy.md @@ -2,9 +2,6 @@ approvers: - pweil- title: Pod 安全策略 -redirect_from: -- "/docs/user-guide/pod-security-policy/" -- "/docs/user-guide/pod-security-policy/index.html" --- diff --git a/content/zh/docs/concepts/workloads/pods/init-containers.md b/content/zh/docs/concepts/workloads/pods/init-containers.md index 2ea79d197b..b9cc7cd9d9 100644 --- a/content/zh/docs/concepts/workloads/pods/init-containers.md +++ b/content/zh/docs/concepts/workloads/pods/init-containers.md @@ -2,11 +2,6 @@ approvers: - erictune title: Init 容器 -redirect_from: -- "/docs/concepts/abstractions/init-containers/" -- "/docs/concepts/abstractions/init-containers.html" -- "/docs/user-guide/pods/init-container/" -- "/docs/user-guide/pods/init-container.html" content_template: templates/concept --- diff --git a/content/zh/docs/concepts/workloads/pods/pod-lifecycle.md b/content/zh/docs/concepts/workloads/pods/pod-lifecycle.md index afbe139e2e..614d62cf0e 100644 --- a/content/zh/docs/concepts/workloads/pods/pod-lifecycle.md +++ b/content/zh/docs/concepts/workloads/pods/pod-lifecycle.md @@ -1,8 +1,5 @@ --- title: Pod 的生命周期 -redirect_from: -- "/docs/user-guide/pod-states/" -- "/docs/user-guide/pod-states.html" content_template: templates/concept --- diff --git a/content/zh/docs/tasks/access-application-cluster/communicate-containers-same-pod-shared-volume.md b/content/zh/docs/tasks/access-application-cluster/communicate-containers-same-pod-shared-volume.md index b08a6dda2e..a858c266c5 100644 --- a/content/zh/docs/tasks/access-application-cluster/communicate-containers-same-pod-shared-volume.md +++ b/content/zh/docs/tasks/access-application-cluster/communicate-containers-same-pod-shared-volume.md @@ -1,10 +1,5 @@ --- title: 同 Pod 内的容器使用共享卷通信 -redirect_from: -- "/docs/user-guide/pods/multi-container/" -- "/docs/user-guide/pods/multi-container.html" -- "/docs/tasks/configure-pod-container/communicate-containers-same-pod/" -- "/docs/tasks/configure-pod-container/communicate-containers-same-pod.html" content_template: templates/task --- diff --git a/content/zh/docs/tasks/administer-cluster/access-cluster-services.md b/content/zh/docs/tasks/administer-cluster/access-cluster-services.md index 2035d498b4..82253e0c32 100644 --- a/content/zh/docs/tasks/administer-cluster/access-cluster-services.md +++ b/content/zh/docs/tasks/administer-cluster/access-cluster-services.md @@ -1,9 +1,6 @@ --- title: 访问集群上运行的服务 -redirect_from: -- "/docs/user-guide/accessing-the-cluster/" -- "/docs/user-guide/accessing-the-cluster.html" content_template: templates/task --- From c618e97a9ea368f330f81d7742845828ace49d6b Mon Sep 17 00:00:00 2001 From: wawa Date: Tue, 17 Mar 2020 21:44:23 +0800 Subject: [PATCH 109/140] Enhanced FileOrCreate description (#19465) --- content/zh/docs/concepts/storage/volumes.md | 33 +++++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/content/zh/docs/concepts/storage/volumes.md b/content/zh/docs/concepts/storage/volumes.md index 6a46296cbb..797352e61b 100644 --- a/content/zh/docs/concepts/storage/volumes.md +++ b/content/zh/docs/concepts/storage/volumes.md @@ -988,6 +988,39 @@ spec: type: Directory ``` +{{< caution >}} + +应当注意,`FileOrCreate` 类型不会负责创建文件的父目录。如果挂载挂载文件的父目录不存在，pod 启动会失败。为了确保这种 `type` 能够工作，可以尝试把文件和它对应的目录分开挂载，如下所示： +{{< /caution >}} + +#### FileOrCreate pod 示例 + +```yaml +apiVersion: v1 +kind: Pod +metadata: + name: test-webserver +spec: + containers: + - name: test-webserver + image: k8s.gcr.io/test-webserver:latest + volumeMounts: + - mountPath: /var/local/aaa + name: mydir + - mountPath: /var/local/aaa/1.txt + name: myfile + volumes: + - name: mydir + hostPath: + # 确保文件所在目录成功创建。 + path: /var/local/aaa + type: DirectoryOrCreate + - name: myfile + hostPath: + path: /var/local/aaa/1.txt + type: FileOrCreate +``` + ### iscsi {#iscsi} -* TOC -{:toc} + +{{% capture overview %}} + +此页面概述了准入控制器。 +{{% /capture %}} + +{{% capture body %}} + ## 什么是准入控制插件？ -一个准入控制插件是一段代码，它会在请求通过认证和授权之后、对象被持久化之前拦截到达 API server 的请求。插件代码运行在 API server 进程中，必须将其编译为二进制文件，以便在此时使用。 + +准入控制器是一段代码，它会在请求通过认证和授权之后、对象被持久化之前拦截到达 API 服务器的请求。控制器由下面的[列表](#what-does-each-admission-controller-do)组成，并编译进 `kube-apiserver` 二进制文件，并且只能由集群管理员配置。在该列表中，有两个特殊的控制器：MutatingAdmissionWebhook 和 ValidatingAdmissionWebhook。它们根据 API 中的配置，分别执行变更和验证[准入控制 webhook](/docs/reference/access-authn-authz/extensible-admission-controllers/#admission-webhooks)。 + -在每个请求被集群接受之前，准入控制插件依次执行。如果插件序列中任何一个拒绝了该请求，则整个请求将立即被拒绝并且返回一个错误给终端用户。 + +准入控制器可以执行 “验证” 和/或 “变更” 操作。变更（mutating）控制器可以修改被其接受的对象；验证（validating）控制器则不行。 + +准入控制过程分为两个阶段。第一阶段，运行变更准入控制器。第二阶段，运行验证准入控制器。 +再次提醒，某些控制器既是变更准入控制器又是验证准入控制器。 + +如果任何一个阶段的任何控制器拒绝了该请求，则整个请求将立即被拒绝，并向终端用户返回一个错误。 -准入控制插件可能会在某些情况下改变传入的对象，从而应用系统配置的默认值。另外，作为请求处理的一部分，准入控制插件可能会对相关的资源进行变更，以实现类似增加配额使用量这样的功能。 + +最后，除了对对象进行变更外，准入控制器还可以有其它作用：将相关资源作为请求处理的一部分进行变更。 +增加使用配额就是一个典型的示例，说明了这样做的必要性。 +此类用法都需要相应的回收或回调过程，因为任一准入控制器都无法确定某个请能否通过所有其它准入控制器。 -## 为什么需要准入控制插件？ +## 为什么需要准入控制器？ -Kubernetes 的许多高级功能都要求启用一个准入控制插件，以便正确地支持该特性。因此，一个没有正确配置准入控制插件的 Kubernetes API server 是不完整的，它不会支持您所期望的所有特性。 +Kubernetes 的许多高级功能都要求启用一个准入控制器，以便正确地支持该特性。因此，没有正确配置准入控制器的 Kubernetes API 服务器是不完整的，它无法支持您期望的所有特性。 -## 如何启用一个准入控制插件？ + +## 如何启用一个准入控制器？ -Kubernetes API server 支持一个标志参数 `admission-control` ，它指定了一个用于在集群修改对象之前调用的以逗号分隔的准入控制插件顺序列表。 +Kubernetes API 服务器的 `enable-admission-plugins` 标志，它指定了一个用于在集群修改对象之前调用的（以逗号分隔的）准入控制插件顺序列表。 +例如，下面的命令就启用了 `NamespaceLifecycle` 和 `LimitRanger` 准入控制插件： + +```shell +kube-apiserver --enable-admission-plugins=NamespaceLifecycle,LimitRanger ... +``` + +{{< note >}} -## 每个插件的功能是什么？ - -### AlwaysAdmit +根据您 Kubernetes 集群的部署方式以及 API 服务器的启动方式的不同，您可能需要以不同的方式应用设置。 +例如，如果将 API 服务器部署为 systemd 服务，你可能需要修改 systemd 单元文件； +如果以自托管方式部署 Kubernetes，你可能需要修改 API 服务器的清单文件。 +{{< /note >}} -使用这个插件自行通过所有的请求。 -### AlwaysPullImages +## 怎么关闭准入控制器？ + +Kubernetes API 服务器的 `disable-admission-plugins` 标志，会将传入的（以逗号分隔的）准入控制插件列表禁用，即使是默认启用的插件也会被禁用。 + +```shell +kube-apiserver --disable-admission-plugins=PodNodeSelector,AlwaysDeny ... +``` + +## 哪些插件是默认启用的？ + +下面的命令可以查看哪些插件是默认启用的： + +```shell +kube-apiserver -h | grep enable-admission-plugins +``` + + + +在 1.16 中，它们是： + +```shell +NamespaceLifecycle, LimitRanger, ServiceAccount, TaintNodesByCondition, Priority, DefaultTolerationSeconds, DefaultStorageClass, StorageObjectInUseProtection, PersistentVolumeClaimResize, MutatingAdmissionWebhook, ValidatingAdmissionWebhook, RuntimeClass, ResourceQuota +``` + + + +## 每个准入控制器的作用是什么？ + +### AlwaysAdmit {#alwaysadmit} {{< feature-state for_k8s_version="v1.13" state="deprecated" >}} + +该准入控制器会允许所有的 pod 接入集群。已废弃，因为它的行为根本就和没有准入控制器一样。 + +### AlwaysPullImages {#alwayspullimages} + + -这个插件修改每一个新创建的 Pod 的镜像拉取策略为 Always 。这在多租户集群中是有用的，这样用户就可以放心，他们的私有镜像只能被那些有凭证的人使用。没有这个插件，一旦镜像被拉取到节点上，任何用户的 pod 都可以通过已了解到的镜像的名称（假设 pod 被调度到正确的节点上）来使用它，而不需要对镜像进行任何授权检查。当启用这个插件时，总是在启动容器之前拉取镜像，这意味着需要有效的凭证。 +该准入控制器会修改每一个新创建的 Pod 的镜像拉取策略为 Always 。 +这在多租户集群中是有用的，这样用户就可以放心，他们的私有镜像只能被那些有凭证的人使用。 +如果没有这个准入控制器，一旦镜像被拉取到节点上，任何用户的 pod 都可以通过已了解到的镜像的名称（假设 pod 被调度到正确的节点上）来使用它，而不需要对镜像进行任何授权检查。 +当启用这个准入控制器时，总是在启动容器之前拉取镜像，这意味着需要有效的凭证。 -### AlwaysDeny +### AlwaysDeny {#alwaysdeny} {{< feature-state for_k8s_version="v1.13" state="deprecated" >}} -拒绝所有的请求。用于测试。 +拒绝所有的请求。由于没有实际意义，已废弃。 + +### DefaultStorageClass {#defaultstorageclass} -### DenyExecOnPrivileged （已废弃） + +该准入控制器监测没有请求任何特定存储类的 `PersistentVolumeClaim` 对象的创建，并自动向其添加默认存储类。 +这样，没有任何特殊存储类需求的用户根本不需要关心它们，它们将获得默认存储类。 -如果一个 pod 拥有一个特权容器，这个插件将拦截所有在该 pod 中执行 exec 命令的请求。 + +当未配置默认存储类时，此准入控制器不执行任何操作。如果将多个存储类标记为默认存储类，它将拒绝任何创建 `PersistentVolumeClaim` 的操作，并显示错误。此时准入控制器会忽略任何 `PersistentVolumeClaim` 更新操作，仅响应创建操作。要修复此错误，管理员必须重新访问其 `StorageClass` 对象，并仅将其中一个标记为默认。 -如果集群支持特权容器，并且希望限制最终用户在这些容器中执行 exec 命令的能力，我们强烈建议启用这个插件。 + +关于持久化卷和存储类，以及如何将存储类标记为默认，请参见[持久化卷](/docs/concepts/storage/persistent-volumes/)。 + +### DefaultTolerationSeconds {#defaulttolerationseconds} + + + +该准入控制器为 Pod 设置默认的容忍度，在 5 分钟内容忍 `notready:NoExecute` 和 `unreachable:NoExecute` 污点。（如果 Pod 尚未容忍 `node.kubernetes.io/not-ready：NoExecute` 和 `node.alpha.kubernetes.io/unreachable：NoExecute` 污点的话） + +### DenyExecOnPrivileged {#denyexeconprivileged} {{< feature-state for_k8s_version="v1.13" state="deprecated" >}} + + +如果一个 pod 拥有一个特权容器，该准入控制器将拦截所有在该 pod 中执行 exec 命令的请求。 -此功能已合并到 [DenyEscalatingExec](#denyescalatingexec)。 -### DenyEscalatingExec +此功能已合并至 [DenyEscalatingExec](#denyescalatingexec)。 +而 DenyExecOnPrivileged 准入插件已被废弃，并将在 v1.18 被移除。 + +建议使用基于策略的准入插件（例如 [PodSecurityPolicy](#podsecuritypolicy) 和自定义准入插件）， +该插件可以针对特定用户或命名空间，还可以防止创建权限过高的 Pod。 + +### DenyEscalatingExec {#denyescalatingexec} {{< feature-state for_k8s_version="v1.13" state="deprecated" >}} + + -这个插件将拒绝在拥有衍生特权而具备访问宿主机能力的 pod 中执行 exec 和 attach 命令。这包括在特权模式运行的 pod ，可以访问主机 IPC 命名空间的 pod ，和访问主机 PID 命名空间的 pod 。 + +该准入控制器将拒绝在由于拥有升级特权，而具备访问宿主机能力的 pod 中执行 exec 和 attach 命令。这包括在特权模式运行的 pod ，可以访问主机 IPC 命名空间的 pod ，和访问主机 PID 命名空间的 pod 。 -如果集群支持使用以衍生特权运行的容器，并且希望限制最终用户在这些容器中执行 exec 命令的能力，我们强烈建议启用这个插件。 +The DenyEscalatingExec admission plugin is deprecated and will be removed in v1.18. -### ImagePolicyWebhook +Use of a policy-based admission plugin (like [PodSecurityPolicy](#podsecuritypolicy) or a custom admission plugin) +which can be targeted at specific users or Namespaces and also protects against creation of overly privileged Pods +is recommended instead. +--> + +DenyExecOnPrivileged 准入插件已被废弃，并将在 v1.18 被移除。 + +建议使用基于策略的准入插件（例如 [PodSecurityPolicy](#podsecuritypolicy) 和自定义准入插件）， +该插件可以针对特定用户或命名空间，还可以防止创建权限过高的 Pod。 + +### EventRateLimit {#eventratelimit} {{< feature-state for_k8s_version="v1.13" state="alpha" >}} -ImagePolicyWebhook 插件允许使用一个后端的 webhook 做出准入决策。您可以按照如下配置 admission-control 选项来启用这个插件: -```shell ---admission-control=ImagePolicyWebhook +该准入控制器缓解了事件请求淹没 API 服务器的问题。集群管理员可以通过以下方式指定事件速率限制： + + + + * 确保 API 服务器的 `--runtime-config` 标志中包含了 `eventratelimit.admission.k8s.io/v1alpha1=true`； + * 启用 `EventRateLimit` 准入控制器； + * 从文件中引用 `EventRateLimit` 配置文件，并提供给 API 服务器命令的 `--admission-control-config-file` 标志： + +{{< tabs name="eventratelimit_example" >}} +{{% tab name="apiserver.config.k8s.io/v1" %}} +```yaml +apiVersion: apiserver.config.k8s.io/v1 +kind: AdmissionConfiguration +plugins: +- name: EventRateLimit + path: eventconfig.yaml +... ``` +{{% /tab %}} +{{% tab name="apiserver.k8s.io/v1alpha1" %}} +```yaml +# Deprecated in v1.17 in favor of apiserver.config.k8s.io/v1 +apiVersion: apiserver.k8s.io/v1alpha1 +kind: AdmissionConfiguration +plugins: +- name: EventRateLimit + path: eventconfig.yaml +... +``` +{{% /tab %}} +{{< /tabs >}} + + + +可以在配置中指定四种类型的限制： + + + + * `Server`: API 服务器收到的所有事件请求共享一个桶。 + * `Namespace`: 每个命名空间都有一个专用的桶。 + * `User`: 给每个用户都分配一个桶。 + * `SourceAndObject`: 根据事件的源和涉及对象的每种组合分配桶。 + + + +下面是一个配置示例 `eventconfig.yaml`： + +```yaml +apiVersion: eventratelimit.admission.k8s.io/v1alpha1 +kind: Configuration +limits: +- type: Namespace + qps: 50 + burst: 100 + cacheSize: 2000 +- type: User + qps: 10 + burst: 50 +``` + + + +详情请参见[事件速率限制提案](https://git.k8s.io/community/contributors/design-proposals/api-machinery/admission_control_event_rate_limit.md)。 + +### ExtendedResourceToleration {#extendedresourcetoleration} + + + +该插件有助于创建可扩展资源的专用节点。 +如果运营商想创建可扩展资源的专用节点（如 GPU、FPGA 等）， +那他们应该以扩展资源名称作为键名，[为节点设置污点](/docs/concepts/configuration/taint-and-toleration/#example-use-cases)。 +如果启用了该准入控制器，会将此类污点的容忍自动添加到请求扩展资源的 Pod 中，用户不必再手动添加这些容忍。 + +### ImagePolicyWebhook {#imagepolicywebhook} + + + +ImagePolicyWebhook 准入控制器允许使用一个后端的 webhook 做出准入决策。 -#### 配置文件格式 - -ImagePolicyWebhook 插件使用了admission config 文件 `--admission-control-config-file` 来为后端行为设置配置选项。该文件可以是 json 或 yaml ，并具有以下格式: -```javascript -{ - "imagePolicy": { - "kubeConfigFile": "path/to/kubeconfig/for/backend", - "allowTTL": 50, // time in s to cache approval - "denyTTL": 50, // time in s to cache denial - "retryBackoff": 500, // time in ms to wait between retries - "defaultAllow": true // determines behavior if the webhook backend fails - } -} +#### 配置文件格式 + + + +ImagePolicyWebhook 使用配置文件来为后端行为设置配置选项。该文件可以是 json 或 yaml ，并具有以下格式: + +```yaml +imagePolicy: + kubeConfigFile: /path/to/kubeconfig/for/backend + # time in s to cache approval + allowTTL: 50 + # time in s to cache denial + denyTTL: 50 + # time in ms to wait between retries + retryBackoff: 500 + # determines behavior if the webhook backend fails + defaultAllow: true ``` -这个配置文件必须引用一个 [kubeconfig](/docs/concepts/cluster-administration/authenticate-across-clusters-kubeconfig/) 格式的文件，并在其中配置指向后端的连接。且需要在 TLS 上与后端进行通信。 +从文件中引用 ImagePolicyWebhook 的配置文件，并将其提供给 API 服务器命令 `--admission-control-config-file` 标志： + +{{< tabs name="imagepolicywebhook_example1" >}} +{{% tab name="apiserver.config.k8s.io/v1" %}} +```yaml +apiVersion: apiserver.config.k8s.io/v1 +kind: AdmissionConfiguration +plugins: +- name: ImagePolicyWebhook + path: imagepolicyconfig.yaml +... +``` +{{% /tab %}} +{{% tab name="apiserver.k8s.io/v1alpha1" %}} +```yaml +# Deprecated in v1.17 in favor of apiserver.config.k8s.io/v1 +apiVersion: apiserver.k8s.io/v1alpha1 +kind: AdmissionConfiguration +plugins: +- name: ImagePolicyWebhook + path: imagepolicyconfig.yaml +... +``` +{{% /tab %}} +{{< /tabs >}} + + + +或者，您也可以直接将配置嵌入到文件中： + +{{< tabs name="imagepolicywebhook_example2" >}} +{{% tab name="apiserver.config.k8s.io/v1" %}} +```yaml +apiVersion: apiserver.config.k8s.io/v1 +kind: AdmissionConfiguration +plugins: +- name: ImagePolicyWebhook + configuration: + imagePolicy: + kubeConfigFile: + allowTTL: 50 + denyTTL: 50 + retryBackoff: 500 + defaultAllow: true +``` +{{% /tab %}} +{{% tab name="apiserver.k8s.io/v1alpha1" %}} +```yaml +# Deprecated in v1.17 in favor of apiserver.config.k8s.io/v1 +apiVersion: apiserver.k8s.io/v1alpha1 +kind: AdmissionConfiguration +plugins: +- name: ImagePolicyWebhook + configuration: + imagePolicy: + kubeConfigFile: + allowTTL: 50 + denyTTL: 50 + retryBackoff: 500 + defaultAllow: true +``` +{{% /tab %}} +{{< /tabs >}} + + + +ImagePolicyWebhook 的配置文件必须引用 [kubeconfig](/docs/concepts/cluster-administration/authenticate-across-clusters-kubeconfig/) 格式的文件，该文件设置了到后端的连接，要求后端使用 TLS 进行通信。 kubeconfig 文件的 cluster 字段需要指向远端服务，user 字段需要包含已返回的授权者。 + + +```yaml +# clusters 指的是远程服务。 +clusters: +- name: name-of-remote-imagepolicy-service + cluster: + certificate-authority: /path/to/ca.pem # CA 用于验证远程服务 + server: https://images.example.com/policy # 要查询的远程服务的 URL。必须是 'https' 。 + +# users 指的是 API 服务器的 Webhook 配置。 +users: +- name: name-of-api-server + user: + client-certificate: /path/to/cert.pem # webhook 准入控制器使用的证书 + client-key: /path/to/key.pem # 证书匹配的密钥 +``` + -对于更多的 HTTP 配置，请参阅 [kubeconfig](/docs/concepts/cluster-administration/authenticate-across-clusters-kubeconfig/) 文档。 + +HTTP 更多的配置，请参阅 [kubeconfig](/docs/concepts/cluster-administration/authenticate-across-clusters-kubeconfig/) 文档。 -当面对一个准入决策时，API server 发送一个描述操作的 JSON 序列化的 api.imagepolicy.v1alpha1.ImageReview 对象。该对象包含描述被审核容器的字段，以及所有匹配 `*.image-policy.k8s.io/*` 的 pod 注释。 +当面对一个准入决策时，API server 发送一个描述操作的 JSON 序列化的 `imagepolicy.k8s.io/v1alpha1` `ImageReview` 对象。该对象包含描述被审核容器的字段，以及所有匹配 `*.image-policy.k8s.io/*` 的 pod 注释。 -请求载荷例子： +请求载荷示例： -``` +```json { "apiVersion":"imagepolicy.k8s.io/v1alpha1", "kind":"ImageReview", @@ -241,20 +585,20 @@ An example request body: "image":"myrepo/myimage@sha256:beb6bd6a68f114c1dc2ea4b28db81bdf91de202a9014972bec5e4d9171d90ed" } ], - "annotations":[ + "annotations":{ "mycluster.image-policy.k8s.io/ticket-1234": "break-glass" - ], + }, "namespace":"mynamespace" } } ``` -远程服务将填充请求的 ImageReviewStatus 字段，并返回允许或不允许访问。响应主体的 "spec" 字段会被忽略，并且可以省略。一个允许访问应答会返回： +远程服务将填充请求的 `ImageReviewStatus` 字段，并返回允许或不允许访问的响应。响应体的 "spec" 字段会被忽略，并且可以省略。一个允许访问应答会返回： -``` +```json { "apiVersion": "imagepolicy.k8s.io/v1alpha1", "kind": "ImageReview", @@ -269,7 +613,7 @@ To disallow access, the service would return: --> 不允许访问，服务将返回： -``` +```json { "apiVersion": "imagepolicy.k8s.io/v1alpha1", "kind": "ImageReview", @@ -288,12 +632,12 @@ For further documentation refer to the `imagepolicy.v1alpha1` API objects and `p -使用注解进行扩展 +#### 使用注解进行扩展 -一个 pod 中匹配 `*.image-policy.k8s.io/*` 的注解都会被发送给 webhook。这允许了解镜像策略后端的用户向它发送额外的信息，并为不同的后端实现接收不同的信息。 +一个 pod 中匹配 `*.image-policy.k8s.io/*` 的注解都会被发送给 webhook。这允许了解后端镜像策略的用户向它发送额外的信息，并为不同的后端实现接收不同的信息。 - * 在紧急情况下，请求 "break glass" 覆盖一个策略。 - - * 从一个记录了 break-glass 的请求的票证系统得到的一个票证编号 - - * 向策略服务器提供一个提示，用于提供镜像的 imageID，以方便它进行查找 + + * 在紧急情况下，请求 "break glass" 覆盖一个策略。 + * 从一个记录了 break-glass 的请求的 ticket 系统得到的一个 ticket 号号码。 + * 向策略服务器提供一个提示，用于提供镜像的 imageID，以方便它进行查找。 在任何情况下，注解都是由用户提供的，并不会被 Kubernetes 以任何方式进行验证。在将来，如果一个注解确定将被广泛使用，它可能会被提升为 ImageReviewSpec 的一个命名字段。 -### ServiceAccount +### LimitPodHardAntiAffinityTopology {#limitpodhardantiaffinitytopology} -这个插件实现了 [serviceAccounts](/docs/user-guide/service-accounts) 的自动化。 -如果您打算使用 Kubernetes 的 ServiceAccount 对象，我们强烈建议您使用这个插件。 +该准入控制器拒绝（定义了 `Anti Affinity` 拓扑键的）任何 Pod（`requiredDuringSchedulingRequiredDuringExecution` 中的 `kubernetes.io/hostname` 除外） -### SecurityContextDeny +### LimitRanger {#limitranger} -该插件将拒绝任何试图设置特定扩展 [SecurityContext](/docs/user-guide/security-context) 字段的 pod。如果集群没有使用 [ pod 安全策略](/docs/user-guide/pod-security-policy) 来限制安全上下文所能获取的值集，那么应该启用这个功能。 - -### ResourceQuota - - -此插件将观察传入的请求，并确保它不违反任何一个 `Namespace` 中的 `ResourceQuota` 对象中枚举出来的约束。如果您在 Kubernetes 部署中使用了 `ResourceQuota` -，您必须使用这个插件来强制执行配额限制。 - - -请查看 [resourceQuota 设计文档](https://git.k8s.io/community/contributors/design-proposals/admission_control_resource_quota.md) 和 [Resource Quota 例子](/docs/concepts/policy/resource-quotas/) 了解更多细节。 - - -强烈建议将这个插件配置在准入控制插件序列的末尾。这样配额就不会过早地增加，只会在稍后的准入控制中被拒绝。 - -### LimitRanger - - -这个插件将观察传入的请求，并确保它不会违反 `Namespace` 中 `LimitRange` 对象枚举的任何约束。如果您在 Kubernetes 部署中使用了 `LimitRange` 对象，则必须使用此插件来执行这些约束。LimitRanger 插件还可以用于将默认资源请求应用到没有指定任何内容的 Pod ；当前，默认的 LimitRanger 对 `default` 命名空间中的所有 pod 应用了0.1 CPU 的需求。 + +该准入控制器会观察传入的请求，并确保它不会违反 `Namespace` 中 `LimitRange` 对象枚举的任何约束。如果您在 Kubernetes 部署中使用了 `LimitRange` 对象，则必须使用此准入控制器来执行这些约束。LimitRanger 还可以用于将默认资源请求应用到没有指定任何内容的 Pod；当前，默认的 LimitRanger 对 `default` 命名空间中的所有 pod 都应用了 0.1 CPU 的需求。 -请查看 [limitRange 设计文档](https://git.k8s.io/community/contributors/design-proposals/admission_control_limit_range.md) 和 [Limit Range 例子](/docs/tasks/configure-pod-container/limit-range/) 了解更多细节。 + +请查看 [limitRange 设计文档](https://git.k8s.io/community/contributors/design-proposals/resource-management/admission_control_limit_range.md) 和 [Limit Range 例子](/docs/tasks/configure-pod-container/limit-range/)了解更多细节。 + +### MutatingAdmissionWebhook {#mutatingadmissionwebhook} {{< feature-state for_k8s_version="v1.13" state="beta" >}} -### InitialResources （试验） +该准入控制器调用任何与请求匹配的变更 webhook。匹配的 webhook 将被串行调用。每一个 webhook 都可以根据需要修改对象。 + +`MutatingAdmissionWebhook` ，顾名思义，仅在变更阶段运行。 -此插件观察 pod 创建请求。如果容器忽略了 requests 和 limits 计算资源，那么插件就会根据运行相同镜像的容器的历史使用记录来自动填充计算资源请求。如果没有足够的数据进行决策，则请求将保持不变。当插件设置了一个计算资源请求时，它会用它自动填充的计算资源对 pod 进行注解。 +如果由此准入控制器调用的 Webhook 有副作用（如降低配额）， +则它 *必须* 具有协调系统，因为不能保证后续的 Webhook 和验证准入控制器都会允许完成请求。 -请查看 [InitialResouces 建议书](https://git.k8s.io/community/contributors/design-proposals/initial-resources.md) 了解更多细节。 - -### NamespaceLifecycle +如果你禁用了 MutatingAdmissionWebhook，那么还必须使用 `--runtime-config` 标志禁止 `admissionregistration.k8s.io/v1beta1` 组/版本中的 `MutatingWebhookConfiguration` 对象（版本 >=1.9 时，这两个对象都是默认启用的）。 -这个插件强制不能在一个正在被终止的 `Namespace` 中创建新对象，和确保使用不存在 `Namespace` 的请求被拒绝。 +#### 谨慎编写和安装变更 webhook + + + + * 当用户尝试创建的对象与返回的对象不同时，用户可能会感到困惑。 + * 当它们回读的对象与尝试创建的对象不同，内建的控制环可能会出问题。 + * 与覆盖原始请求中设置的字段相比，使用原始请求未设置的字段会引起问题的可能性较小。应尽量避免前面那种方式。 + * 这是一个 beta 特性。Kubernetes 未来的版本可能会限制这些 webhook 可以进行的变更类型。 + * 内建资源和第三方资源的控制环，未来可能会受到破坏性的更改，使现在运行良好的 Webhook 无法再正常运行。即使完成了 webhook API 安装，也不代表会为该 webhook 提供无限期的支持。 + +### NamespaceAutoProvision {#namespaceautoprovision} + + +该准入控制器会检查命名空间资源上的所有传入请求，并检查所引用的命名空间是否确实存在。如果找不到，它将创建一个命名空间。 +此准入控制器对于不想要求命名空间必须先创建后使用的集群部署中很有用。 + +### NamespaceExists {#namespaceexists} + + +该准入控制器检查除自身 `Namespace` 以外的命名空间资源上的所有请求。如果请求引用的命名空间不存在，则拒绝该请求。 + +### NamespaceLifecycle {#namespacelifecycle} + + +该准入控制器禁止在一个正在被终止的 `Namespace` 中创建新对象，并确保使用不存在的 `Namespace` 的请求被拒绝。 +该准入控制器还会禁止删除三个系统保留的命名空间，即 `default`、`kube-system` 和 `kube-public`。 -删除 `Namespace` 触发了在该命名空间中删除所有对象（ pod 、 services 等）的一系列操作。为了确保这个过程的完整性，我们强烈建议启用这个插件。 +删除 `Namespace` 会触发删除该命名空间中所有对象（pod、services 等）的一系列操作。为了确保这个过程的完整性，我们强烈建议启用这个准入控制器。 -### DefaultStorageClass +### NodeRestriction {#noderestriction} -这个插件观察不指定 storage class 字段的 `PersistentVolumeClaim` 对象的创建，并自动向它们添加默认的 storage class 。这样，不指定 storage class 字段的用户根本无需关心它们，它们将得到默认的 storage class 。 +该准入控制器限制了 kubelet 可以修改的 `Node` 和 `Pod` 对象。 为了受到这个准入控制器的限制，kubelet 必须使用在 `system:nodes` 组中的凭证，并使用 `system:node:` 形式的用户名。这样，kubelet 只可修改自己的 `Node` API 对象，只能修改绑定到节点本身的 `Pod` 对象。 -当没有配置默认 storage class 时，这个插件不会执行任何操作。当一个以上的 storage class 被标记为默认时，它拒绝 `PersistentVolumeClaim` 创建并返回一个错误，管理员必须重新检查 `StorageClass` 对象，并且只标记一个作为默认值。这个插件忽略了任何 `PersistentVolumeClaim` 更新，它只对创建起作用。 +在 Kubernetes 1.11+ 的版本中，不允许 kubelet 从 `Node` API 对象中更新或删除污点。 + +在 Kubernetes 1.13+ 的版本中，`NodeRestriction` 准入插件可防止 kubelet 删除 `Node` API 对象，并对 `kubernetes.io/` 或 `k8s.io/` 前缀标签的 kubelet 强制进行如下修改： -查看 [persistent volume](/docs/user-guide/persistent-volumes) 文档了解 persistent volume claims 和 storage classes 并了解如何将一个 storage classes 标志为默认。 -### DefaultTolerationSeconds +* **防止** kubelets 添加/删除/更新带有 `node-restriction.kubernetes.io/` 前缀的标签。保留此前缀的标签，供管理员用来标记 `Node` 对象以隔离工作负载，并且不允许 kubelet 修改带有该前缀的标签。 +* **允许** kubelet 添加/删除/更新这些和这些前缀的标签： + * `kubernetes.io/hostname` + * `kubernetes.io/arch` + * `kubernetes.io/os` + * `beta.kubernetes.io/instance-type` + * `node.kubernetes.io/instance-type` + * `failure-domain.beta.kubernetes.io/region` + * `failure-domain.beta.kubernetes.io/zone` + * `topology.kubernetes.io/region` + * `topology.kubernetes.io/zone` + * `kubelet.kubernetes.io/`-prefixed labels + * `node.kubernetes.io/`-prefixed labels -这个插件设置了 pod 默认的宽恕容忍时间，对于那些没有设置宽恕容忍时间的 pod ，可以容忍 `notready:NoExecute` 和 `unreachable:NoExecute` 这些 taint 5分钟。 +Use of any other labels under the `kubernetes.io` or `k8s.io` prefixes by kubelets is reserved, and may be disallowed or allowed by the `NodeRestriction` admission plugin in the future. -### PodNodeSelector +Future versions may add additional restrictions to ensure kubelets have the minimal set of permissions required to operate correctly. +--> +kubelet 保留 `kubernetes.io` 或 `k8s.io` 前缀的所有标签，并且将来可能会被 `NodeRestriction` 准入插件允许或禁止。 + +将来的版本可能会增加其他限制，以确保 kubelet 具有正常运行所需的最小权限集。 + +### OwnerReferencesPermissionEnforcement {#ownerreferencespermissionenforcement} -通过读取命名空间注释和全局配置,这个插件默认并限制了在一个命名空间中使用什么节点选择器。 + +该准入控制器保护对 `metadata.ownerReferences` 对象的访问，以便只有对该对象具有 “删除” 权限的用户才能对其进行更改。该准入控制器还保护对 `metadata.ownerReferences[x].blockOwnerDeletion` 对象的访问，以便只有对所引用的 **属主（owner）** 的 `finalizers` 子资源具有 “更新” 权限的用户才能对其进行更改。 + +### PersistentVolumeLabel {#persistentvolumelabel} {{< feature-state for_k8s_version="v1.13" state="deprecated" >}} + + +该准入控制器会自动将区（region）或区域（zone）标签附加到由云提供商（如 GCE、AWS）定义的 PersistentVolumes 中。 +这有助于确保 Pod 和 PersistentVolume 位于相同的区或区域。 +如果准入控制器不支持为 PersistentVolumes 自动添加标签，那你可能需要手动添加标签，以防止 Pod 挂载其他区域的卷。 +PersistentVolumeLabel 已被废弃，标记持久卷已由[云管理控制器](/docs/tasks/administer-cluster/running-cloud-controller/)接管。 +从 1.11 开始，默认情况下禁用此准入控制器。 + +### PodNodeSelector {#podnodeselector} + + +该准入控制器通过读取命名空间注解和全局配置，来为命名空间中可以可以使用的节点选择器设置默认值并实施限制。 -#### 配置文件格式 - -PodNodeSelector 插件使用准入配置文件 `--admission-control-config-file` 来设置后端行为的配置选项。 - -请注意，配置文件格式将在未来版本中移至版本化文件。 - - -这个文件可能是 json 或 yaml ，格式如下： + +#### 配置文件格式 + +`PodNodeSelector` 使用配置文件来设置后端行为的选项。 +请注意，配置文件格式将在将来某个版本中迁移为版本化文件。 +该文件可以是 json 或 yaml，格式如下： ```yaml podNodeSelectorPluginConfig: - clusterDefaultNodeSelector: - namespace1: - namespace2: + clusterDefaultNodeSelector: name-of-node-selector + namespace1: name-of-node-selector + namespace2: name-of-node-selector ``` + +从文件中引用 `PodNodeSelector` 配置文件，提供给 API 服务器命令行标志 `--admission-control-config-file`： + +{{< tabs name="podnodeselector_example1" >}} +{{% tab name="apiserver.config.k8s.io/v1" %}} +```yaml +apiVersion: apiserver.config.k8s.io/v1 +kind: AdmissionConfiguration +plugins: +- name: PodNodeSelector + path: podnodeselector.yaml +... +``` +{{% /tab %}} +{{% tab name="apiserver.k8s.io/v1alpha1" %}} +```yaml +# Deprecated in v1.17 in favor of apiserver.config.k8s.io/v1 +apiVersion: apiserver.k8s.io/v1alpha1 +kind: AdmissionConfiguration +plugins: +- name: PodNodeSelector + path: podnodeselector.yaml +... +``` +{{% /tab %}} +{{< /tabs >}} + #### 配置注解格式 - -PodNodeSelector 插件使用键为 `scheduler.alpha.kubernetes.io/node-selector` 的注解将节点选择器分配给 namespace 。 + +`PodNodeSelector` 使用键为 `scheduler.alpha.kubernetes.io/node-selector` 的注解将节点选择器分配给命名空间。 ```yaml apiVersion: v1 kind: Namespace metadata: annotations: - scheduler.alpha.kubernetes.io/node-selector: + scheduler.alpha.kubernetes.io/node-selector: name-of-node-selector name: namespace3 ``` -### PodSecurityPolicy + + +#### 内部行为 + +该准入控制器行为如下： +1. 如果 `Namespace` 的注解带有键 `scheduler.alpha.kubernetes.io/node-selector` ，则将其值用作节点选择器。 +2. 如果命名空间缺少此类注解，则使用 `PodNodeSelector` 插件配置文件中定义的 `clusterDefaultNodeSelector` 作为节点选择器。 +3. 评估 pod 节点选择器和命名空间节点选择器是否存在冲突。存在冲突将导致拒绝。 +4. 评估 pod 节点选择器和命名空间的白名单定义的插件配置文件是否存在冲突。存在冲突将导致拒绝。 + +{{< note >}} + +PodNodeSelector 允许 Pod 强制在特定标签的节点上运行。另请参阅 PodTolerationRestriction 准入插件，该插件可防止 Pod 在特定污点的节点上运行。 +{{< /note >}} + +### PersistentVolumeClaimResize {#persistentvolumeclaimresize} + + +该准入控制器检查传入的 `PersistentVolumeClaim` 调整大小请求，对其执行额外的验证操作。 + +{{< note >}} + +对调整卷大小的支持是一种 Alpha 特性。管理员必须将特性门控 `ExpandPersistentVolumes` 设置为 `true` 才能启用调整大小。 +{{< /note >}} + + +启用 `ExpandPersistentVolumes` 特性门控之后，建议将 `PersistentVolumeClaimResize` 准入控制器也启用。除非 PVC 的 `StorageClass` 明确地将 `allowVolumeExpansion` 设置为 `true` 来显式启用调整大小。否则，默认情况下该准入控制器会阻止所有对 PVC 大小的调整。 + +例如：由以下 `StorageClass` 创建的所有 `PersistentVolumeClaim` 都支持卷容量扩充： + +```yaml +apiVersion: storage.k8s.io/v1 +kind: StorageClass +metadata: + name: gluster-vol-default +provisioner: kubernetes.io/glusterfs +parameters: + resturl: "http://192.168.10.100:8080" + restuser: "" + secretNamespace: "" + secretName: "" +allowVolumeExpansion: true +``` + + + +关于持久化卷申领的更多信息，请参见 [PersistentVolumeClaims](/docs/concepts/storage/persistent-volumes/#persistentvolumeclaims)。 + +### PodPreset {#podpreset} + + + +该准入控制器根据与 PodPreset 中条件的匹配情况，将指定字段注入一个 pod。 +另请参见 [PodPreset 概念](/docs/concepts/workloads/pods/podpreset/)和[使用 PodPreset 将信息注入 Pod](/docs/tasks/inject-data-application/podpreset) 获取详情。 + +### PodSecurityPolicy {#podsecuritypolicy} + + -此插件负责在创建和修改 pod 时根据请求的安全上下文和可用的 pod 安全策略确定是否应该通过 pod。 +此准入控制器负责在创建和修改 pod 时根据请求的安全上下文和可用的 pod 安全策略确定是否可以执行请求。 -查看 [Pod 安全策略文档](/docs/concepts/policy/pod-security-policy/) 了解更多细节。 +查看 [Pod 安全策略文档](/docs/concepts/policy/pod-security-policy/)了解更多细节。 -### NodeRestriction +### PodTolerationRestriction {#podtolerationrestriction} -这个插件限制了 kubelet 可以修改的 `Node` 和 `Pod` 对象。 为了受到这个入场插件的限制，kubelet 必须在 `system：nodes` 组中使用凭证，并使用 `system：node：` 形式的用户名。这样的 kubelet 只允许修改自己的 `Node` API 对象，只能修改绑定到节点本身的 `Pod` 对象。 - -未来的版本可能会添加额外的限制，以确保 kubelet 具有正确操作所需的最小权限集。 + +该准入控制器首先验证 Pod 的容忍度与其命名空间的容忍度之间的冲突。如果存在冲突，则拒绝 Pod 请求。 +然后，它将命名空间的容忍度合并到 pod 的容忍度中，之后根据命名空间的容忍度白名单检查所得到的容忍度结果。 +如果检查成功，则将接受 pod 请求，否则拒绝该请求。 -## 是否有推荐的一组插件可供使用？ + +如果 pod 的命名空间没有任何关联的默认容忍度或容忍度白名单，则使用集群级别的默认容忍度或容忍度白名单（如果有的话）。 -有。 -对于 Kubernetes >= 1.6.0 版本，我们强烈建议运行以下一系列准入控制插件（顺序也很重要） +命名空间的容忍度通过注解健 `scheduler.alpha.kubernetes.io/defaultTolerations` 和 `scheduler.alpha.kubernetes.io/tolerationsWhitelist` 设置。 -```shell ---admission-control=NamespaceLifecycle,LimitRanger,ServiceAccount,PersistentVolumeLabel,DefaultStorageClass,ResourceQuota,DefaultTolerationSeconds -``` -对于 Kubernetes >= 1.4.0 版本，我们强烈建议运行以下一系列准入控制插件（顺序也很重要） +### Priority {#priority} -```shell ---admission-control=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,ResourceQuota -``` +The priority admission controller uses the `priorityClassName` field and populates the integer value of the priority. If the priority class is not found, the Pod is rejected. +--> + +### 优先级 {#priority} + +优先级准入控制器使用 `priorityClassName` 字段并用整型值填充优先级。如果找不到优先级，则拒绝 Pod。 + +### ResourceQuota {#resourcequota} -对于 Kubernetes >= 1.2.0 版本，我们强烈建议运行以下一系列准入控制插件（顺序也很重要） - -```shell ---admission-control=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota -``` +该准入控制器会监测传入的请求，并确保它不违反任何一个 `Namespace` 中的 `ResourceQuota` 对象中枚举出来的约束。 +如果您在 Kubernetes 部署中使用了 `ResourceQuota` ，您必须使用这个准入控制器来强制执行配额限制。 -对于 Kubernetes >= 1.0.0 版本，我们强烈建议运行以下一系列准入控制插件（顺序也很重要） +请查看 [resourceQuota 设计文档](https://git.k8s.io/community/contributors/design-proposals/admission_control_resource_quota.md)和 [Resource Quota 例子](/docs/concepts/policy/resource-quotas/)了解更多细节。 -```shell ---admission-control=NamespaceLifecycle,LimitRanger,SecurityContextDeny,ServiceAccount,PersistentVolumeLabel,ResourceQuota -``` + + + +### 容器运行时类 {#runtimeclass} {{< feature-state for_k8s_version="v1.16" state="alpha" >}} + +[容器运行时类](/docs/concepts/containers/runtime-class/)定义描述了与运行 Pod 相关的开销。此准入控制器将相应地设置 pod.Spec.Overhead 字段。 + +详情请参见 [Pod 开销](/docs/concepts/configuration/pod-overhead/)。 + +### SecurityContextDeny {#securitycontextdeny} + + +该准入控制器将拒绝任何试图设置特定提升 [SecurityContext](/docs/user-guide/security-context) 字段的 pod。 +如果集群没有使用 [pod 安全策略](/docs/user-guide/pod-security-policy)来限制安全上下文所能获取的值集，那么应该启用这个功能。 + +### ServiceAccount {#serviceaccount} + + +该准入控制器实现了 [serviceAccounts](/docs/user-guide/service-accounts) 的自动化。 +如果您打算使用 Kubernetes 的 ServiceAccount 对象，我们强烈建议您使用这个准入控制器。 + +### StorageObjectInUseProtection + + +`StorageObjectInUseProtection` 插件将 `kubernetes.io/pvc-protection` 或 `kubernetes.io/pv-protection` finalizers 添加到新创建的持久化卷声明（PVC）或持久化卷（PV）中。 如果用户尝试删除 PVC/PV，除非 PVC/PV 的保护控制器移除 finalizers，否则 PVC/PV 不会被删除。有关更多详细信息，请参考[保护使用中的存储对象](/docs/concepts/storage/persistent-volumes/#storage-object-in-use-protection)。 + +### TaintNodesByCondition {#taintnodesbycondition} {{< feature-state for_k8s_version="v1.12" state="beta" >}} + + +该准入控制器 {{< glossary_tooltip text="污点" term_id="taint" >}} 新创建的 `NotReady` 和 `NoSchedule` 节点。 +避免了可能导致 Pod 在更新其污点以准确反映其所报告状况之前，就安排了在新节点上的竞争条件的情况。 + +### ValidatingAdmissionWebhook {#validatingadmissionwebhook} {{< feature-state for_k8s_version="v1.13" state="beta" >}} + + +该准入控制器调用与请求匹配的所有验证 webhook。匹配的 webhook 将被并行调用。如果其中任何一个拒绝请求，则整个请求将失败。 +该准入控制器仅在验证阶段运行；与 `MutatingAdmissionWebhook` 准入控制器所调用的 webhook 相反，它调用的 webhook 应该不会使对象出现变更。 + + +如果以此方式调用的 webhook 有其它作用（如，配额递减），则它必须具有协调系统，因为不能保证后续的 webhook 或其他有效的准入控制器都允许请求完成。 + + +如果您禁用了 ValidatingAdmissionWebhook，还必须在 `admissionregistration.k8s.io/v1beta1` 组/版本中使用 `--runtime-config` 标志来禁用 `ValidatingWebhookConfiguration` 对象（默认情况下在 1.9 版和更高版本中均处于启用状态）。 + + + +## 有推荐的准入控制器吗？ + +有，对于 Kubernetes 1.10 以上的版本，推荐使用的准入控制器默认情况下都处于启用状态（查看[这里](/docs/reference/command-line-tools-reference/kube-apiserver/#options)）。 +因此您无需显式指定它们。您可以使用 `--enable-admission-plugins` 标志（ **顺序不重要** ）来启用默认设置以外的其他准入控制器。 + +{{< note >}} + +`--admission-control` 在 1.10 中已废弃，已由 `--enable-admission-plugins` 取代。 +{{< /note >}} + + + +对于 Kubernetes 1.9 及更早版本，我们建议使用 `--admission-control` 标志（**顺序很重要**）运行下面的一组准入控制器。 + +* v1.9 + + ```shell + --admission-control=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota + ``` + + + +* 需要重申的是，在 1.9 中，它们都发生在变更阶段和验证阶段，例如 `ResourceQuota` 在验证阶段运行，因此是最后一个运行的准入控制器。 + `MutatingAdmissionWebhook` 出现在此列表的前面，因为它在变更阶段运行。 + + + 对于更早期版本，没有验证和变更的概念，并且准入控制器按照指定的确切顺序运行。 + +{{% /capture %}} From ec43d8f08e572b79d686be4c9175f37f2fa4af9c Mon Sep 17 00:00:00 2001 From: 2BFL Date: Tue, 17 Mar 2020 21:49:24 +0800 Subject: [PATCH 111/140] zh-translation: install-minikube.md (#19626) Accept tengqm's suggestion --- .../zh/docs/tasks/tools/install-minikube.md | 481 ++++++++++++++++-- 1 file changed, 427 insertions(+), 54 deletions(-) diff --git a/content/zh/docs/tasks/tools/install-minikube.md b/content/zh/docs/tasks/tools/install-minikube.md index 9c93d35cb5..db9657876d 100644 --- a/content/zh/docs/tasks/tools/install-minikube.md +++ b/content/zh/docs/tasks/tools/install-minikube.md @@ -1,7 +1,10 @@ --- -title: Install Minikube +title: 安装 Minikube content_template: templates/task weight: 20 +card: + name: tasks + weight: 10 --- {{% capture overview %}} -本页面讲述如何安装 Minikube。 +本页面讲述如何安装 [Minikube](/docs/tutorials/hello-minikube)，该工具用于在您电脑中的虚拟机上运行一个单节点的 Kubernetes 集群。 + {{% /capture %}} {{% capture prerequisites %}} -您的计算机必须在 BIOS 中启用 VT-x 或 AMD-v 虚拟化。 +{{< tabs name="minikube_before_you_begin" >}} +{{% tab name="Linux" %}} + +若要检查您的 Linux 是否支持虚拟化技术，请运行下面的命令并验证输出结果是否不为空： + +``` +grep -E --color 'vmx|svm' /proc/cpuinfo +``` + +{{% /tab %}} + +{{% tab name="macOS" %}} + + + +若要检查您的 macOS 是否支持虚拟化技术，请运行下面的命令： + +``` +sysctl -a | grep -E --color 'machdep.cpu.features|VMX' +``` + + + +如果你在输出结果中看到了 `VMX` （应该会高亮显示）的字眼，说明您的电脑已启用 VT-x 特性。 + +{{% /tab %}} + +{{% tab name="Windows" %}} + + +若要检查您的 Windows8 及以上的系统是否支持虚拟化技术，请终端或者 cmd 中运行以下命令： + +``` +systeminfo +``` + + +如果您看到下面的输出，则表示该 Windows 支持虚拟化技术。 + +``` +Hyper-V Requirements: VM Monitor Mode Extensions: Yes + Virtualization Enabled In Firmware: Yes + Second Level Address Translation: Yes + Data Execution Prevention Available: Yes +``` + + + +如果您看到下面的输出，则表示您的操作系统已经安装了 Hypervisor，您可以跳过安装 Hypervisor 的步骤。 +``` +Hyper-V Requirements: A hypervisor has been detected. Features required for Hyper-V will not be displayed. +``` + + +{{% /tab %}} +{{< /tabs >}} + {{% /capture %}} {{% capture steps %}} -## 安装 Hypervisor -如果还没有装过 hypervisor，以下是一些不错的选择： - - - +# 安装 minikube -* macOS：[VirtualBox](https://www.virtualbox.org/wiki/Downloads) 或者 -[VMware Fusion](https://www.vmware.com/products/fusion)，或者 -[HyperKit](https://github.com/moby/hyperkit)。 -* Linux：[VirtualBox](https://www.virtualbox.org/wiki/Downloads) 或者 -[KVM](http://www.linux-kvm.org/)。 - - - {{< note >}} - - Minikube 也支持 `-\-vm-driver=none` 选项，该选项在主机而非 VM 上运行 Kubernetes 组件。 - 使用这个驱动程序需要 Docker 和 linux 环境，而不需要 hypervisor。 - - - - - {{< /note >}} - -* Windows：[VirtualBox](https://www.virtualbox.org/wiki/Downloads) 或者 -[Hyper-V](https://msdn.microsoft.com/en-us/virtualization/hyperv_on_windows/quick_start/walkthrough_install)。 - -## 安装 kubectl +{{< tabs name="tab_with_md" >}} +{{% tab name="Linux" %}} - -* 请参照 [安装与设置 kubectl](/docs/tasks/tools/install-kubectl/) 中的说明安装 kubectl。 +### 安装 kubectl -## 安装 Minikube +请确保你已正确安装 kubectl。您可以根据[安装并设置 kubectl](/docs/tasks/tools/install-kubectl/#install-kubectl-on-linux) 的说明来安装 kubectl。 + +### 安装 Hypervisor + +如果还没有装过 hypervisor，请选择以下方式之一进行安装： + -* 请参照[最新发行](https://github.com/kubernetes/minikube/releases)指导安装 Minikube。 +• [KVM](https://www.linux-kvm.org/)，也使用了 QEMU +• [VirtualBox](https://www.virtualbox.org/wiki/Downloads) + + + +Minikube 还支持使用一个 `--vm-driver=none` 选项，让 Kubernetes 组件运行在主机中，而不是在 VM 中。 +使用这种驱动方式需要 [Docker](https://www.docker.com/products/docker-desktop) 和 Linux 环境，但不需要 hypervisor。 + +如果你在 Debian 系的 OS 中使用了 `none` 这种驱动方式，请使用 `.deb` 包安装 Docker，不要使用 snap 包的方式，Minikube 不支持这种方式。 +你可以从 [Docker](https://www.docker.com/products/docker-desktop) 下载 `.deb` 包。 + +{{< caution >}} + + +`none` VM 驱动方式存在导致安全和数据丢失的问题。 +使用 `--vm-driver=none` 之前，请参考[这个文档](https://minikube.sigs.k8s.io/docs/reference/drivers/none/)获取详细信息。 +{{< /caution >}} + + + +Minikube 还支持另外一个类似于 Docker 驱动的方式 `vm-driver=podman`。 +使用超级用户权限（root 用户）运行 Podman 可以最好的确保容器具有足够的权限使用你操作系统上的所有特性。 + +{{< caution >}} + +`Podman` 驱动方式需要以 root 用户身份运行容器，因为普通用户帐户没有足够的权限使用容器运行可能需要的操作系统上的所有特性。 +{{< /caution >}} + + + +### 使用包安装 Minikube + +Minikube 有 *实验性* 的安装包。你可以在 Minikube 在 GitHub 上的 [releases](https://github.com/kubernetes/minikube/releases) 找到 Linux (AMD64) 的包。 + +根据您的 Linux 发行版选择安装合适的包。 + + + +### 直接下载并安装 Minikube + +如果你不想通过包安装，你也可以下载并使用一个单节点二进制文件。 + +```shell +curl -Lo minikube https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64 \ + && chmod +x minikube +``` + + +将 Minikube 可执行文件添加至 path： + +```shell +sudo mkdir -p /usr/local/bin/ +sudo install minikube /usr/local/bin/ +``` + + + +### 使用 Homebrew 安装 Minikube + +你还可以使用 Linux [Homebrew](https://docs.brew.sh/Homebrew-on-Linux) 安装 Minikube： + +```shell +brew install minikube +``` + +{{% /tab %}} +{{% tab name="macOS" %}} + + + +### 安装 kubectl + +请确保你已正确安装 kubectl。您可以根据[安装并设置 kubectl](/docs/tasks/tools/install-kubectl/#install-kubectl-on-linux) 的说明来安装 kubectl。 + + + +### 安装 Hypervisor + +如果你还没有安装 hypervisor，请选择以下方式之一进行安装： + +• [HyperKit](https://github.com/moby/hyperkit) + +• [VirtualBox](https://www.virtualbox.org/wiki/Downloads) + +• [VMware Fusion](https://www.vmware.com/products/fusion) + + + +### 安装 Minikube + +macOS 安装 Minikube 最简单的方法是使用 [Homebrew](https://brew.sh)： + +```shell +brew install minikube +``` + + + +你也可以通过下载单节点二进制文件进行安装： + +```shell +curl -Lo minikube https://storage.googleapis.com/minikube/releases/latest/minikube-darwin-amd64 \ + && chmod +x minikube +``` + + + +这是一个简单的将 Minikube 可执行文件添加至 path 的方法： + +```shell +sudo mv minikube /usr/local/bin +``` + +{{% /tab %}} +{{% tab name="Windows" %}} + + + +### 安装 kubectl + +请确保你已正确安装 kubectl。您可以根据[安装并设置 kubectl](/docs/tasks/tools/install-kubectl/#install-kubectl-on-windows) 的说明来安装 kubectl。 + + + +### 安装 Hypervisor + +如果你还没有安装 hypervisor，请选择以下方式之一进行安装： + +• [Hyper-V](https://msdn.microsoft.com/en-us/virtualization/hyperv_on_windows/quick_start/walkthrough_install) + +• [VirtualBox](https://www.virtualbox.org/wiki/Downloads) + +{{< note >}} + +Hyper-V 可以运行在三个版本的 Windows 10 上：企业版、专业版和教育版（Enterprise, Professional, Education）。 +{{< /note >}} + + + +### 使用 Chocolatey 安装 Minikube + +Windows 安装 Minikube 最简单的方法是使用 [Chocolatey](https://chocolatey.org/) （以管理员身份运行）： + +```shell +choco install minikube +``` + + + +完成 Minikube 的安装后，关闭当前 CLI 界面再重新打开。 +Minikube 应该已经自动添加至 path 中。 + + + +### 使用安装程序安装 Minikube + +在 Windows 上使用 [Windows Installer](https://docs.microsoft.com/en-us/windows/desktop/msi/windows-installer-portal) 手动安装 Minikube，下载并运行 [`minikube-installer.exe`](https://github.com/kubernetes/minikube/releases/latest/download/minikube-installer.exe) 即可。 + + + +### 直接下载并安装 Minikube + +想在 Windows 上手动安装 Minikube，下载 [`minikube-windows-amd64`](https://github.com/kubernetes/minikube/releases/latest) 并将其重命名为 `minikube.exe`，然后将其添加至 path 即可。 + +{{% /tab %}} +{{< /tabs >}} {{% /capture %}} {{% capture whatsnext %}} -* [使用 Minikube 在本地运行 Kubernetes](/docs/getting-started-guides/minikube/) +* [使用 Minikube 在本地运行 Kubernetes](/docs/setup/learning-environment/minikube/) + {{% /capture %}} + + + +## 安装确认 + +要确认 hypervisor 和 Minikube 均已成功安装，可以运行以下命令来启动本地 Kubernetes 集群： + +{{< note >}} + + +通过 `minikube start` 设置 `--vm-driver`。在下面提到 `` 的地方，用小写字母，输入你安装的 hypervisor 的名称。 +[指定 VM 驱动程序](https://kubernetes.io/docs/setup/learning-environment/minikube/#specifying-the-vm-driver) 列举了 `--vm-driver` 值的完整列表 + +{{< /note >}} + +```shell +minikube start --vm-driver= +``` + + + +一旦 `minikube start` 完成，你可以运行下面的命令来检查集群的状态： + +```shell +minikube status +``` + + + +如果你的集群正在运行，`minikube status` 的输出结果应该类似于这样： + +``` +host: Running +kubelet: Running +apiserver: Running +kubeconfig: Configured +``` + + + +在确认 Minikube 与 hypervisor 均正常工作后，您可以继续使用 Minikube 或停止集群。要停止集群，请运行： + +```shell +minikube stop +``` + + + +## 清理本地状态{#cleanup-local-state} + +如果您之前安装过 Minikube，并运行了： + +```shell +minikube start +``` + + + +并且 `minikube start` 返回了一个错误： +``` +machine does not exist +``` + + + +那么，你需要清理 minikube 的本地状态： +```shell +minikube delete +``` From 67c3e0bc5daedcc56fbf5fd5b99e7350ddbb0218 Mon Sep 17 00:00:00 2001 From: Rudy Steiner Date: Tue, 17 Mar 2020 21:50:23 +0800 Subject: [PATCH 112/140] Trans zh blog 2019-12-09-kubernetes-1.17-release-announcement.md (#19640) --- ...09-kubernetes-1.17-release-announcement.md | 274 ++++++++++++++++++ 1 file changed, 274 insertions(+) create mode 100644 content/zh/blog/_posts/2019-12-09-kubernetes-1.17-release-announcement.md diff --git a/content/zh/blog/_posts/2019-12-09-kubernetes-1.17-release-announcement.md b/content/zh/blog/_posts/2019-12-09-kubernetes-1.17-release-announcement.md new file mode 100644 index 0000000000..0505d24542 --- /dev/null +++ b/content/zh/blog/_posts/2019-12-09-kubernetes-1.17-release-announcement.md @@ -0,0 +1,274 @@ +--- +layout: blog +title: "Kubernetes 1.17：稳定" +date: 2019-12-09T13：00：00-08：00 +slug: kubernetes-1-17-release-announcement +--- + + +**作者:** [Kubernetes 1.17发布团队](https://github.com/kubernetes/sig-release/blob/master/releases/release-1.17/release_team.md) + + +我们高兴的宣布Kubernetes 1.17版本的交付，它是我们2019年的第四个也是最后一个发布版本。Kubernetes v1.17包含22个增强功能：有14个增强已经逐步稳定(stable)，4个增强功能已经进入公开测试版(beta)，4个增强功能刚刚进入内部测试版(alpha)。 + +## 主要的主题 + +### 云服务提供商标签基本可用 + +作为公开测试版特性添加到v1.2，v1.7中可以看到云提供商标签达到基本可用。 + +### 卷快照进入公开测试版 + +在v1.7中，Kubernetes卷快照特性是公开测试版。这个特性是在v1.12中以内部测试版引入的，第二个有重大变化的内部测试版是v1.13。 + +## 容器存储接口迁移公开测试版 + +在v1.7中，Kubernetes树内存储插件到容器存储接口(CSI)的迁移基础架构是公开测试版。容器存储接口迁移最初是在Kubernetes v1.14中以内部测试版引入的。 + +## 云服务提供商标签基本可用 + +当节点和卷被创建，会基于基础云提供商的Kubernetes集群打上一系列标准标签。节点会获得一个实例类型标签。节点和卷都会得到两个描述资源在云提供商拓扑的位置标签,通常是以区域和地区的方式组织。 + + +Kubernetes组件使用标准标签来支持一些特性。例如，调度者会保证pods和它们所声明的卷放置在相同的区域；当调度部署的pods时，调度器会优先将它们分布在不同的区域。你还可以在自己的pods标准中利用标签来配置，如节点亲和性，之类的事。标准标签使得你写的pod规范在不同的云提供商之间是可移植的。 + + +在这个版本中，标签已经达到基本可用。Kubernetes组件都已经更新，可以填充基本可用和公开测试版标签，并对两者做出反应。然而，如果你的pod规范或自定义的控制器正在使用公开测试版标签，如节点亲和性，我们建议你可以将它们迁移到新的基本可用标签中。你可以从如下地方找到新标签的文档： + + +- [实例类型](https://kubernetes.io/docs/reference/kubernetes-api/labels-annotations-taints/#nodekubernetesioinstance-type) +- [地区](https://kubernetes.io/docs/reference/kubernetes-api/labels-annotations-taints/#topologykubernetesioregion) +- [区域](https://kubernetes.io/docs/reference/kubernetes-api/labels-annotations-taints/#topologykubernetesiozone) + + +## 卷快照进入公开测试版 + +在v1.7中，Kubernetes卷快照是是公开测试版。最初是在v1.12中以内部测试版引入的，第二个有重大变化的内部测试版是v1.13。这篇文章总结它在公开版本中的变化。 + +### 卷快照是什么？ + +许多的存储系统(如谷歌云持久化磁盘，亚马逊弹性块存储和许多的内部存储系统)支持为持久卷创建快照。快照代表卷在一个时间点的复制。它可用于配置新卷(使用快照数据提前填充)或恢复卷到一个之前的状态(用快照表示)。 + +### 为什么给Kubernetes加入卷快照？ + +Kubernetes卷插件系统已经提供了功能强大的抽象用于自动配置、附加和挂载块文件系统。 + + +支持所有这些特性是Kubernets负载可移植的目标：Kubernetes旨在分布式系统应用和底层集群之间创建一个抽象层,使得应用可以不感知其运行集群的具体信息并且部署也不需特定集群的知识。 + + +Kubernetes存储特别兴趣组(SIG)将快照操作确定为对很多有状态负载的关键功能。如数据库管理员希望在操作数据库前保存数据库卷快照。 + + +在Kubernetes接口中提供一种标准的方式触发快照操作，Kubernetes用户可以处理这种用户场景，而不必使用Kubernetes API(并手动执行存储系统的具体操作)。 + + +取而代之的是，Kubernetes用户现在被授权以与集群无关的方式将快照操作放进他们的工具和策略中，并且确信它将对任意的Kubernetes集群有效，而与底层存储无关。 + + +此外，Kubernetes 快照原语作为基础构建能力解锁了为Kubernetes开发高级、企业级、存储管理特性的能力:包括应用或集群级别的备份方案。 + + +你可以阅读更多关于[发布容器存储接口卷快照公开测试版](https://kubernetes.io/blog/2019/12/09/kubernetes-1-17-feature-cis-volume-snapshot-beta/) + +## 容器存储接口迁移公测版 + +### 为什么我们迁移内建树插件到容器存储接口？ + +在容器存储接口之前，Kubernetes提供功能强大的卷插件系统。这些卷插件是树内的意味着它们的代码是核心Kubernetes代码的一部分并附带在核心Kubernetes二进制中。然而，为Kubernetes添加插件支持新卷是非常有挑战的。希望在Kubernetes上为自己存储系统添加支持(或修复现有卷插件的bug)的供应商被迫与Kubernetes发行进程对齐。此外，第三方存储代码在核心Kubernetes二进制中会造成可靠性和安全问题，并且这些代码对于Kubernetes的维护者来说是难以(一些场景是不可能)测试和维护的。在Kubernetes上采用容器存储接口可以解决大部分问题。 + + +随着更多容器存储接口驱动变成生产环境可用，我们希望所有的Kubernetes用户从容器存储接口模型中获益。然而，我们不希望强制用户以破坏现有基本可用的存储接口的方式去改变负载和配置。道路很明确，我们将不得不用CSI替换树内插件接口。什么是容器存储接口迁移？ + + +在容器存储接口迁移上所做的努力使得替换现有的树内存储插件，如`kubernetes.io/gce-pd`或`kubernetes.io/aws-ebs`，为相应的容器存储接口驱动成为可能。如果容器存储接口迁移正常工作，Kubernetes终端用户不会注意到任何差别。迁移过后，Kubernetes用户可以继续使用现有接口来依赖树内存储插件的功能。 + + +当Kubernetes集群管理者更新集群使得CSI迁移可用，现有的有状态部署和工作负载照常工作；然而，在幕后Kubernetes将存储管理操作交给了(以前是交给树内驱动)CSI驱动。 + + +Kubernetes组非常努力地保证存储接口的稳定性和平滑升级体验的承诺。这需要细致的考虑现有特性和行为来确保后向兼容和接口稳定性。你可以想像成在加速行驶的直线上给赛车换轮胎。 + + +你可以在这篇博客中阅读更多关于[容器存储接口迁移成为公开测试版](https://kubernetes.io/blog/2019/12/09/kubernetes-1-17-feature-csi-migration-beta/). + +## 其它更新 + +### 稳定💯 + +- [按条件污染节点](https://github.com/kubernetes/enhancements/issues/382) +- [可配置的Pod进程共享命名空间](https://github.com/kubernetes/enhancements/issues/495) +- [采用kube-scheduler调度DaemonSet Pods](https://github.com/kubernetes/enhancements/issues/548) +- [动态卷最大值](https://github.com/kubernetes/enhancements/issues/554) +- [Kubernetes容器存储接口支持拓扑](https://github.com/kubernetes/enhancements/issues/557) +- [在SubPath挂载提供环境变量扩展](https://github.com/kubernetes/enhancements/issues/559) +- [为Custom Resources提供默认值](https://github.com/kubernetes/enhancements/issues/575) +- [从频繁的Kublet心跳到租约接口](https://github.com/kubernetes/enhancements/issues/589) +- [拆分Kubernetes测试Tarball](https://github.com/kubernetes/enhancements/issues/714) +- [添加Watch书签支持](https://github.com/kubernetes/enhancements/issues/956) +- [行为驱动一致性测试](https://github.com/kubernetes/enhancements/issues/960) +- [服务负载均衡终结保护](https://github.com/kubernetes/enhancements/issues/980) +- [避免每一个Watcher独立序列化相同的对象](https://github.com/kubernetes/enhancements/issues/1152) + + +### 主要变化 + +- [添加IPv4/IPv6双栈支持](https://github.com/kubernetes/enhancements/issues/563) + + +### 其它显著特性 + +- [拓扑感知路由服务(内部测试版)](https://github.com/kubernetes/enhancements/issues/536) +- [为Windows添加RunAsUserName](https://github.com/kubernetes/enhancements/issues/1043) + + +### 可用性 + +Kubernetes 1.17 可以[在GitHub下载](https://github.com/kubernetes/kubernetes/releases/tag/v1.17.0)。开始使用Kubernetes，看看这些[交互教学](https://kubernetes.io/docs/tutorials/)。你可以非常容易使用[kubeadm](https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/)安装1.17。 + +### 发布团队 + +正是因为有上千人参与技术或非技术内容的贡献才使这个版本成为可能。特别感谢由Guinevere Saenger领导的[发布团队](https://github.com/kubernetes/sig-release/blob/master/releases/release-1.17/release_team.md)。发布团队的35名成员在发布版本的多方面进行了协调，从文档到测试，校验和特性的完善。 + +随着Kubernetes社区的成长，我们的发布流程是在开源软件协作方面惊人的示例。Kubernetes快速并持续获得新用户。这一成长产生了良性的反馈循环，更多的贡献者贡献代码创造了更加活跃的生态。Kubernetes已经有超过[39000位贡献者](https://k8s.devstats.cncf.io/d/24/overall-project-statistics?orgId=1)和一个超过66000人的活跃社区。 + +### 网络研讨会 + +2020年1月7号，加入Kubernetes 1.17发布团队，学习关于这次发布的主要特性。[这里](https://zoom.us/webinar/register/9315759188139/WN_kPOZA_6RTjeGdXTG7YFO3A)注册。 + +### 参与其中 + +最简单的参与Kubernetes的方式是加入其中一个与你兴趣相同的[特别兴趣组](https://github.com/kubernetes/community/blob/master/sig-list.md)（SIGs)。有什么想要广播到Kubernetes社区吗？通过如下的频道，在每周的[社区会议](https://github.com/kubernetes/community/tree/master/communication)分享你的声音。感谢你的贡献和支持。 + + +- 在Twitter上关注我们[@Kubernetesio](https://twitter.com/kubernetesio)获取最新的更新 +- 在[Discuss](https://discuss.kubernetes.io/)参与社区的讨论 +- 在[Slack](http://slack.k8s.io/)加入社区 +- 在[Stack Overflow](http://stackoverflow.com/questions/tagged/kubernetes)发布问题(或回答问题) +- 分享你的Kubernetes[故事](https://docs.google.com/a/linuxfoundation.org/forms/d/e/1FAIpQLScuI7Ye3VQHQTwBASrgkjQDSS5TP0g3AXfFhwSM9YpHgxRKFA/viewform) + + From c5f549984de2408755fc5621938ce88f2abd3ad0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=EC=A0=95=EC=A4=80=EC=98=81?= <17703627+bcc829@users.noreply.github.com> Date: Tue, 17 Mar 2020 22:54:22 +0900 Subject: [PATCH 113/140] Update pod.md (#19652) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 부트스트랩과 같이 컨트롤러의 생에와 파드의 생애 분리 -> 부트스트랩과 같이 컨트롤러의 생애와 파드의 생애 분리 오타 수정 --- content/ko/docs/concepts/workloads/pods/pod.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/ko/docs/concepts/workloads/pods/pod.md b/content/ko/docs/concepts/workloads/pods/pod.md index 4a1b3645e0..4196a5fbb7 100644 --- a/content/ko/docs/concepts/workloads/pods/pod.md +++ b/content/ko/docs/concepts/workloads/pods/pod.md @@ -146,7 +146,7 @@ _컨테이너의 어피니티(affinity) 기반 공동 스케줄링을 지원하 * 스케줄러 및 컨트롤러 연결 가능 * "프록시" 없이 컨트롤러 API를 통한 파드-레벨 수준의 동작 지원 -* 부트스트랩과 같이 컨트롤러의 생에와 파드의 생애 분리 +* 부트스트랩과 같이 컨트롤러의 생애와 파드의 생애 분리 * 컨트롤러와 서비스의 분리 — 파드를 감시하는 엔드 포인트 컨트롤러 * 클러스터 레벨과 kubelet 레벨 기능의 깔끔한 구성 — Kubelet은 효과적인 "파드 컨트롤러" 이다. * 계획된 삭제 또는 이미지 프리페칭과 같이 파드가 종료되기 전에 교체가 될 것이고, From d5f741b03e3b4e234bb1a04062b2a39c48b516bf Mon Sep 17 00:00:00 2001 From: Samuel Almeida Date: Tue, 17 Mar 2020 18:46:22 +0100 Subject: [PATCH 114/140] Add comment about DaemonSet toleration to run on masters (#19530) Fixes issue #19512 --- content/en/examples/controllers/daemonset.yaml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/content/en/examples/controllers/daemonset.yaml b/content/en/examples/controllers/daemonset.yaml index 1bfa082833..f291b750c1 100644 --- a/content/en/examples/controllers/daemonset.yaml +++ b/content/en/examples/controllers/daemonset.yaml @@ -15,6 +15,8 @@ spec: name: fluentd-elasticsearch spec: tolerations: + # this toleration is to have the daemonset runnable on master nodes + # remove it if your masters can't run pods - key: node-role.kubernetes.io/master effect: NoSchedule containers: From 4297f3b29c170b56f316a08e338e576527dc22b3 Mon Sep 17 00:00:00 2001 From: inductor Date: Wed, 18 Mar 2020 02:48:23 +0900 Subject: [PATCH 115/140] Update nginx to modern versions (#19198) * update nginx versions * update * update examples --- ...2020-01-21-csi-ephemeral-inline-volumes.md | 2 +- .../manage-deployment.md | 6 +-- .../concepts/configuration/assign-pod-node.md | 2 +- .../working-with-objects/annotations.md | 2 +- .../overview/working-with-objects/labels.md | 2 +- .../overview/working-with-objects/names.md | 2 +- .../workloads/controllers/deployment.md | 44 +++++++++--------- .../en/docs/contribute/style/style-guide.md | 2 +- .../access-authn-authz/authentication.md | 2 +- .../declarative-config.md | 46 +++++++++---------- .../rolling-update-replication-controller.md | 10 ++-- .../run-stateless-application-deployment.md | 4 +- .../application/deployment-scale.yaml | 2 +- .../application/deployment-update.yaml | 2 +- .../en/examples/application/deployment.yaml | 2 +- .../en/examples/application/nginx-app.yaml | 2 +- .../application/nginx/nginx-deployment.yaml | 2 +- .../application/simple_deployment.yaml | 2 +- .../application/update_deployment.yaml | 2 +- .../controllers/nginx-deployment.yaml | 2 +- ...7.9.yaml => replication-nginx-1.14.2.yaml} | 2 +- ...9.2.yaml => replication-nginx-1.16.1.yaml} | 2 +- content/en/examples/pods/simple-pod.yaml | 2 +- 23 files changed, 73 insertions(+), 73 deletions(-) rename content/en/examples/controllers/{replication-nginx-1.7.9.yaml => replication-nginx-1.14.2.yaml} (89%) rename content/en/examples/controllers/{replication-nginx-1.9.2.yaml => replication-nginx-1.16.1.yaml} (92%) diff --git a/content/en/blog/_posts/2020-01-21-csi-ephemeral-inline-volumes.md b/content/en/blog/_posts/2020-01-21-csi-ephemeral-inline-volumes.md index 46570a3e5a..b1574e341d 100644 --- a/content/en/blog/_posts/2020-01-21-csi-ephemeral-inline-volumes.md +++ b/content/en/blog/_posts/2020-01-21-csi-ephemeral-inline-volumes.md @@ -186,7 +186,7 @@ metadata: spec: containers: - name: nginx - image: nginx:1.13-alpine + image: nginx:1.16-alpine ports: - containerPort: 80 volumeMounts: diff --git a/content/en/docs/concepts/cluster-administration/manage-deployment.md b/content/en/docs/concepts/cluster-administration/manage-deployment.md index 39e9695062..eedafce1a3 100644 --- a/content/en/docs/concepts/cluster-administration/manage-deployment.md +++ b/content/en/docs/concepts/cluster-administration/manage-deployment.md @@ -424,16 +424,16 @@ At some point, you'll eventually need to update your deployed application, typic We'll guide you through how to create and update applications with Deployments. -Let's say you were running version 1.7.9 of nginx: +Let's say you were running version 1.14.2 of nginx: ```shell -kubectl run my-nginx --image=nginx:1.7.9 --replicas=3 +kubectl run my-nginx --image=nginx:1.14.2 --replicas=3 ``` ```shell deployment.apps/my-nginx created ``` -To update to version 1.9.1, simply change `.spec.template.spec.containers[0].image` from `nginx:1.7.9` to `nginx:1.9.1`, with the kubectl commands we learned above. +To update to version 1.16.1, simply change `.spec.template.spec.containers[0].image` from `nginx:1.14.2` to `nginx:1.16.1`, with the kubectl commands we learned above. ```shell kubectl edit deployment/my-nginx diff --git a/content/en/docs/concepts/configuration/assign-pod-node.md b/content/en/docs/concepts/configuration/assign-pod-node.md index 5f86b37025..2323cd76f4 100644 --- a/content/en/docs/concepts/configuration/assign-pod-node.md +++ b/content/en/docs/concepts/configuration/assign-pod-node.md @@ -319,7 +319,7 @@ spec: topologyKey: "kubernetes.io/hostname" containers: - name: web-app - image: nginx:1.12-alpine + image: nginx:1.16-alpine ``` If we create the above two deployments, our three node cluster should look like below. diff --git a/content/en/docs/concepts/overview/working-with-objects/annotations.md b/content/en/docs/concepts/overview/working-with-objects/annotations.md index 7e2446ba5a..f88c6a0003 100644 --- a/content/en/docs/concepts/overview/working-with-objects/annotations.md +++ b/content/en/docs/concepts/overview/working-with-objects/annotations.md @@ -82,7 +82,7 @@ metadata: spec: containers: - name: nginx - image: nginx:1.7.9 + image: nginx:1.14.2 ports: - containerPort: 80 diff --git a/content/en/docs/concepts/overview/working-with-objects/labels.md b/content/en/docs/concepts/overview/working-with-objects/labels.md index a74e219103..0de7f04a8c 100644 --- a/content/en/docs/concepts/overview/working-with-objects/labels.md +++ b/content/en/docs/concepts/overview/working-with-objects/labels.md @@ -69,7 +69,7 @@ metadata: spec: containers: - name: nginx - image: nginx:1.7.9 + image: nginx:1.14.2 ports: - containerPort: 80 diff --git a/content/en/docs/concepts/overview/working-with-objects/names.md b/content/en/docs/concepts/overview/working-with-objects/names.md index 60c07391a5..01bb53b56d 100644 --- a/content/en/docs/concepts/overview/working-with-objects/names.md +++ b/content/en/docs/concepts/overview/working-with-objects/names.md @@ -64,7 +64,7 @@ metadata: spec: containers: - name: nginx - image: nginx:1.7.9 + image: nginx:1.14.2 ports: - containerPort: 80 ``` diff --git a/content/en/docs/concepts/workloads/controllers/deployment.md b/content/en/docs/concepts/workloads/controllers/deployment.md index 03c58c6525..4f4fa57819 100644 --- a/content/en/docs/concepts/workloads/controllers/deployment.md +++ b/content/en/docs/concepts/workloads/controllers/deployment.md @@ -64,7 +64,7 @@ In this example: * The Pods are labeled `app: nginx`using the `labels` field. * The Pod template's specification, or `.template.spec` field, indicates that the Pods run one container, `nginx`, which runs the `nginx` - [Docker Hub](https://hub.docker.com/) image at version 1.7.9. + [Docker Hub](https://hub.docker.com/) image at version 1.14.2. * Create one container and name it `nginx` using the `name` field. Follow the steps given below to create the above Deployment: @@ -153,15 +153,15 @@ is changed, for example if the labels or container images of the template are up Follow the steps given below to update your Deployment: -1. Let's update the nginx Pods to use the `nginx:1.9.1` image instead of the `nginx:1.7.9` image. +1. Let's update the nginx Pods to use the `nginx:1.16.1` image instead of the `nginx:1.14.2` image. ```shell - kubectl --record deployment.apps/nginx-deployment set image deployment.v1.apps/nginx-deployment nginx=nginx:1.9.1 + kubectl --record deployment.apps/nginx-deployment set image deployment.v1.apps/nginx-deployment nginx=nginx:1.16.1 ``` or simply use the following command: ```shell - kubectl set image deployment/nginx-deployment nginx=nginx:1.9.1 --record + kubectl set image deployment/nginx-deployment nginx=nginx:1.16.1 --record ``` The output is similar to this: @@ -169,7 +169,7 @@ Follow the steps given below to update your Deployment: deployment.apps/nginx-deployment image updated ``` - Alternatively, you can `edit` the Deployment and change `.spec.template.spec.containers[0].image` from `nginx:1.7.9` to `nginx:1.9.1`: + Alternatively, you can `edit` the Deployment and change `.spec.template.spec.containers[0].image` from `nginx:1.14.2` to `nginx:1.16.1`: ```shell kubectl edit deployment.v1.apps/nginx-deployment @@ -265,7 +265,7 @@ up to 3 replicas, as well as scaling down the old ReplicaSet to 0 replicas. Labels: app=nginx Containers: nginx: - Image: nginx:1.9.1 + Image: nginx:1.16.1 Port: 80/TCP Environment: Mounts: @@ -306,11 +306,11 @@ If you update a Deployment while an existing rollout is in progress, the Deploym as per the update and start scaling that up, and rolls over the ReplicaSet that it was scaling up previously -- it will add it to its list of old ReplicaSets and start scaling it down. -For example, suppose you create a Deployment to create 5 replicas of `nginx:1.7.9`, -but then update the Deployment to create 5 replicas of `nginx:1.9.1`, when only 3 -replicas of `nginx:1.7.9` had been created. In that case, the Deployment immediately starts -killing the 3 `nginx:1.7.9` Pods that it had created, and starts creating -`nginx:1.9.1` Pods. It does not wait for the 5 replicas of `nginx:1.7.9` to be created +For example, suppose you create a Deployment to create 5 replicas of `nginx:1.14.2`, +but then update the Deployment to create 5 replicas of `nginx:1.16.1`, when only 3 +replicas of `nginx:1.14.2` had been created. In that case, the Deployment immediately starts +killing the 3 `nginx:1.14.2` Pods that it had created, and starts creating +`nginx:1.16.1` Pods. It does not wait for the 5 replicas of `nginx:1.14.2` to be created before changing course. ### Label selector updates @@ -347,10 +347,10 @@ This means that when you roll back to an earlier revision, only the Deployment's rolled back. {{< /note >}} -* Suppose that you made a typo while updating the Deployment, by putting the image name as `nginx:1.91` instead of `nginx:1.9.1`: +* Suppose that you made a typo while updating the Deployment, by putting the image name as `nginx:1.161` instead of `nginx:1.16.1`: ```shell - kubectl set image deployment.v1.apps/nginx-deployment nginx=nginx:1.91 --record=true + kubectl set image deployment.v1.apps/nginx-deployment nginx=nginx:1.161 --record=true ``` The output is similar to this: @@ -427,7 +427,7 @@ rolled back. Labels: app=nginx Containers: nginx: - Image: nginx:1.91 + Image: nginx:1.161 Port: 80/TCP Host Port: 0/TCP Environment: @@ -468,13 +468,13 @@ Follow the steps given below to check the rollout history: deployments "nginx-deployment" REVISION CHANGE-CAUSE 1 kubectl apply --filename=https://k8s.io/examples/controllers/nginx-deployment.yaml --record=true - 2 kubectl set image deployment.v1.apps/nginx-deployment nginx=nginx:1.9.1 --record=true - 3 kubectl set image deployment.v1.apps/nginx-deployment nginx=nginx:1.91 --record=true + 2 kubectl set image deployment.v1.apps/nginx-deployment nginx=nginx:1.16.1 --record=true + 3 kubectl set image deployment.v1.apps/nginx-deployment nginx=nginx:1.161 --record=true ``` `CHANGE-CAUSE` is copied from the Deployment annotation `kubernetes.io/change-cause` to its revisions upon creation. You can specify the`CHANGE-CAUSE` message by: - * Annotating the Deployment with `kubectl annotate deployment.v1.apps/nginx-deployment kubernetes.io/change-cause="image updated to 1.9.1"` + * Annotating the Deployment with `kubectl annotate deployment.v1.apps/nginx-deployment kubernetes.io/change-cause="image updated to 1.16.1"` * Append the `--record` flag to save the `kubectl` command that is making changes to the resource. * Manually editing the manifest of the resource. @@ -488,10 +488,10 @@ Follow the steps given below to check the rollout history: deployments "nginx-deployment" revision 2 Labels: app=nginx pod-template-hash=1159050644 - Annotations: kubernetes.io/change-cause=kubectl set image deployment.v1.apps/nginx-deployment nginx=nginx:1.9.1 --record=true + Annotations: kubernetes.io/change-cause=kubectl set image deployment.v1.apps/nginx-deployment nginx=nginx:1.16.1 --record=true Containers: nginx: - Image: nginx:1.9.1 + Image: nginx:1.16.1 Port: 80/TCP QoS Tier: cpu: BestEffort @@ -549,7 +549,7 @@ Follow the steps given below to rollback the Deployment from the current version CreationTimestamp: Sun, 02 Sep 2018 18:17:55 -0500 Labels: app=nginx Annotations: deployment.kubernetes.io/revision=4 - kubernetes.io/change-cause=kubectl set image deployment.v1.apps/nginx-deployment nginx=nginx:1.9.1 --record=true + kubernetes.io/change-cause=kubectl set image deployment.v1.apps/nginx-deployment nginx=nginx:1.16.1 --record=true Selector: app=nginx Replicas: 3 desired | 3 updated | 3 total | 3 available | 0 unavailable StrategyType: RollingUpdate @@ -559,7 +559,7 @@ Follow the steps given below to rollback the Deployment from the current version Labels: app=nginx Containers: nginx: - Image: nginx:1.9.1 + Image: nginx:1.16.1 Port: 80/TCP Host Port: 0/TCP Environment: @@ -722,7 +722,7 @@ apply multiple fixes in between pausing and resuming without triggering unnecess * Then update the image of the Deployment: ```shell - kubectl set image deployment.v1.apps/nginx-deployment nginx=nginx:1.9.1 + kubectl set image deployment.v1.apps/nginx-deployment nginx=nginx:1.16.1 ``` The output is similar to this: diff --git a/content/en/docs/contribute/style/style-guide.md b/content/en/docs/contribute/style/style-guide.md index 30865c49e8..12a6c66839 100644 --- a/content/en/docs/contribute/style/style-guide.md +++ b/content/en/docs/contribute/style/style-guide.md @@ -183,7 +183,7 @@ For field values of type string or integer, use normal style without quotation m Do | Don't :--| :----- Set the value of `imagePullPolicy` to Always. | Set the value of `imagePullPolicy` to "Always". -Set the value of `image` to nginx:1.8. | Set the value of `image` to `nginx:1.8`. +Set the value of `image` to nginx:1.16. | Set the value of `image` to `nginx:1.16`. Set the value of the `replicas` field to 2. | Set the value of the `replicas` field to `2`. {{< /table >}} diff --git a/content/en/docs/reference/access-authn-authz/authentication.md b/content/en/docs/reference/access-authn-authz/authentication.md index 0065bf7abe..0089c08a91 100644 --- a/content/en/docs/reference/access-authn-authz/authentication.md +++ b/content/en/docs/reference/access-authn-authz/authentication.md @@ -208,7 +208,7 @@ spec: serviceAccountName: bob-the-bot containers: - name: nginx - image: nginx:1.7.9 + image: nginx:1.14.2 ``` Service account bearer tokens are perfectly valid to use outside the cluster and diff --git a/content/en/docs/tasks/manage-kubernetes-objects/declarative-config.md b/content/en/docs/tasks/manage-kubernetes-objects/declarative-config.md index 0dee8eb60a..70b5af9df1 100644 --- a/content/en/docs/tasks/manage-kubernetes-objects/declarative-config.md +++ b/content/en/docs/tasks/manage-kubernetes-objects/declarative-config.md @@ -119,7 +119,7 @@ metadata: {"apiVersion":"apps/v1","kind":"Deployment", "metadata":{"annotations":{},"name":"nginx-deployment","namespace":"default"}, "spec":{"minReadySeconds":5,"selector":{"matchLabels":{"app":nginx}},"template":{"metadata":{"labels":{"app":"nginx"}}, - "spec":{"containers":[{"image":"nginx:1.7.9","name":"nginx", + "spec":{"containers":[{"image":"nginx:1.14.2","name":"nginx", "ports":[{"containerPort":80}]}]}}}} # ... spec: @@ -136,7 +136,7 @@ spec: app: nginx spec: containers: - - image: nginx:1.7.9 + - image: nginx:1.14.2 # ... name: nginx ports: @@ -199,7 +199,7 @@ metadata: {"apiVersion":"apps/v1","kind":"Deployment", "metadata":{"annotations":{},"name":"nginx-deployment","namespace":"default"}, "spec":{"minReadySeconds":5,"selector":{"matchLabels":{"app":nginx}},"template":{"metadata":{"labels":{"app":"nginx"}}, - "spec":{"containers":[{"image":"nginx:1.7.9","name":"nginx", + "spec":{"containers":[{"image":"nginx:1.14.2","name":"nginx", "ports":[{"containerPort":80}]}]}}}} # ... spec: @@ -216,7 +216,7 @@ spec: app: nginx spec: containers: - - image: nginx:1.7.9 + - image: nginx:1.14.2 # ... name: nginx ports: @@ -255,7 +255,7 @@ metadata: {"apiVersion":"apps/v1","kind":"Deployment", "metadata":{"annotations":{},"name":"nginx-deployment","namespace":"default"}, "spec":{"minReadySeconds":5,"selector":{"matchLabels":{"app":nginx}},"template":{"metadata":{"labels":{"app":"nginx"}}, - "spec":{"containers":[{"image":"nginx:1.7.9","name":"nginx", + "spec":{"containers":[{"image":"nginx:1.14.2","name":"nginx", "ports":[{"containerPort":80}]}]}}}} # ... spec: @@ -273,7 +273,7 @@ spec: app: nginx spec: containers: - - image: nginx:1.7.9 + - image: nginx:1.14.2 # ... name: nginx ports: @@ -282,7 +282,7 @@ spec: ``` Update the `simple_deployment.yaml` configuration file to change the image from -`nginx:1.7.9` to `nginx:1.11.9`, and delete the `minReadySeconds` field: +`nginx:1.14.2` to `nginx:1.16.1`, and delete the `minReadySeconds` field: {{< codenew file="application/update_deployment.yaml" >}} @@ -303,7 +303,7 @@ The output shows the following changes to the live configuration: * The `replicas` field retains the value of 2 set by `kubectl scale`. This is possible because it is omitted from the configuration file. -* The `image` field has been updated to `nginx:1.11.9` from `nginx:1.7.9`. +* The `image` field has been updated to `nginx:1.16.1` from `nginx:1.14.2`. * The `last-applied-configuration` annotation has been updated with the new image. * The `minReadySeconds` field has been cleared. * The `last-applied-configuration` annotation no longer contains the `minReadySeconds` field. @@ -320,7 +320,7 @@ metadata: {"apiVersion":"apps/v1","kind":"Deployment", "metadata":{"annotations":{},"name":"nginx-deployment","namespace":"default"}, "spec":{"selector":{"matchLabels":{"app":nginx}},"template":{"metadata":{"labels":{"app":"nginx"}}, - "spec":{"containers":[{"image":"nginx:1.11.9","name":"nginx", + "spec":{"containers":[{"image":"nginx:1.16.1","name":"nginx", "ports":[{"containerPort":80}]}]}}}} # ... spec: @@ -338,7 +338,7 @@ spec: app: nginx spec: containers: - - image: nginx:1.11.9 # Set by `kubectl apply` + - image: nginx:1.16.1 # Set by `kubectl apply` # ... name: nginx ports: @@ -460,7 +460,7 @@ metadata: {"apiVersion":"apps/v1","kind":"Deployment", "metadata":{"annotations":{},"name":"nginx-deployment","namespace":"default"}, "spec":{"minReadySeconds":5,"selector":{"matchLabels":{"app":nginx}},"template":{"metadata":{"labels":{"app":"nginx"}}, - "spec":{"containers":[{"image":"nginx:1.7.9","name":"nginx", + "spec":{"containers":[{"image":"nginx:1.14.2","name":"nginx", "ports":[{"containerPort":80}]}]}}}} # ... spec: @@ -478,7 +478,7 @@ spec: app: nginx spec: containers: - - image: nginx:1.7.9 + - image: nginx:1.14.2 # ... name: nginx ports: @@ -518,7 +518,7 @@ metadata: {"apiVersion":"apps/v1","kind":"Deployment", "metadata":{"annotations":{},"name":"nginx-deployment","namespace":"default"}, "spec":{"selector":{"matchLabels":{"app":nginx}},"template":{"metadata":{"labels":{"app":"nginx"}}, - "spec":{"containers":[{"image":"nginx:1.11.9","name":"nginx", + "spec":{"containers":[{"image":"nginx:1.16.1","name":"nginx", "ports":[{"containerPort":80}]}]}}}} # ... spec: @@ -536,7 +536,7 @@ spec: app: nginx spec: containers: - - image: nginx:1.11.9 # Set by `kubectl apply` + - image: nginx:1.16.1 # Set by `kubectl apply` # ... name: nginx ports: @@ -654,7 +654,7 @@ by `name`. # last-applied-configuration value containers: - name: nginx - image: nginx:1.10 + image: nginx:1.16 - name: nginx-helper-a # key: nginx-helper-a; will be deleted in result image: helper:1.3 - name: nginx-helper-b # key: nginx-helper-b; will be retained @@ -663,7 +663,7 @@ by `name`. # configuration file value containers: - name: nginx - image: nginx:1.10 + image: nginx:1.16 - name: nginx-helper-b image: helper:1.3 - name: nginx-helper-c # key: nginx-helper-c; will be added in result @@ -672,7 +672,7 @@ by `name`. # live configuration containers: - name: nginx - image: nginx:1.10 + image: nginx:1.16 - name: nginx-helper-a image: helper:1.3 - name: nginx-helper-b @@ -684,7 +684,7 @@ by `name`. # result after merge containers: - name: nginx - image: nginx:1.10 + image: nginx:1.16 # Element nginx-helper-a was deleted - name: nginx-helper-b image: helper:1.3 @@ -779,7 +779,7 @@ spec: app: nginx spec: containers: - - image: nginx:1.7.9 + - image: nginx:1.14.2 imagePullPolicy: IfNotPresent # defaulted by apiserver name: nginx ports: @@ -819,7 +819,7 @@ spec: spec: containers: - name: nginx - image: nginx:1.7.9 + image: nginx:1.14.2 ports: - containerPort: 80 @@ -834,7 +834,7 @@ spec: spec: containers: - name: nginx - image: nginx:1.7.9 + image: nginx:1.14.2 ports: - containerPort: 80 @@ -852,7 +852,7 @@ spec: spec: containers: - name: nginx - image: nginx:1.7.9 + image: nginx:1.14.2 ports: - containerPort: 80 @@ -870,7 +870,7 @@ spec: spec: containers: - name: nginx - image: nginx:1.7.9 + image: nginx:1.14.2 ports: - containerPort: 80 ``` diff --git a/content/en/docs/tasks/run-application/rolling-update-replication-controller.md b/content/en/docs/tasks/run-application/rolling-update-replication-controller.md index e70877a9d4..11df107b24 100644 --- a/content/en/docs/tasks/run-application/rolling-update-replication-controller.md +++ b/content/en/docs/tasks/run-application/rolling-update-replication-controller.md @@ -153,14 +153,14 @@ from the [`kubectl` reference](/docs/reference/generated/kubectl/kubectl-command ## Walkthrough -Let's say you were running version 1.7.9 of nginx: +Let's say you were running version 1.14.2 of nginx: -{{< codenew file="controllers/replication-nginx-1.7.9.yaml" >}} +{{< codenew file="controllers/replication-nginx-1.14.2.yaml" >}} -To update to version 1.9.1, you can use [`kubectl rolling-update --image`](https://git.k8s.io/community/contributors/design-proposals/cli/simple-rolling-update.md) to specify the new image: +To update to version 1.16.1, you can use [`kubectl rolling-update --image`](https://git.k8s.io/community/contributors/design-proposals/cli/simple-rolling-update.md) to specify the new image: ```shell -kubectl rolling-update my-nginx --image=nginx:1.9.1 +kubectl rolling-update my-nginx --image=nginx:1.16.1 ``` ``` Created my-nginx-ccba8fbd8cc8160970f63f9a2696fc46 @@ -213,7 +213,7 @@ This is one example where the immutability of containers is a huge asset. If you need to update more than just the image (e.g., command arguments, environment variables), you can create a new replication controller, with a new name and distinguishing label value, such as: -{{< codenew file="controllers/replication-nginx-1.9.2.yaml" >}} +{{< codenew file="controllers/replication-nginx-1.16.1.yaml" >}} and roll it out: diff --git a/content/en/docs/tasks/run-application/run-stateless-application-deployment.md b/content/en/docs/tasks/run-application/run-stateless-application-deployment.md index 5da19f0c25..c9e0aebd51 100644 --- a/content/en/docs/tasks/run-application/run-stateless-application-deployment.md +++ b/content/en/docs/tasks/run-application/run-stateless-application-deployment.md @@ -34,7 +34,7 @@ This page shows how to run an application using a Kubernetes Deployment object. You can run an application by creating a Kubernetes Deployment object, and you can describe a Deployment in a YAML file. For example, this YAML file describes -a Deployment that runs the nginx:1.7.9 Docker image: +a Deployment that runs the nginx:1.14.2 Docker image: {{< codenew file="application/deployment.yaml" >}} @@ -64,7 +64,7 @@ a Deployment that runs the nginx:1.7.9 Docker image: Labels: app=nginx Containers: nginx: - Image: nginx:1.7.9 + Image: nginx:1.14.2 Port: 80/TCP Environment: Mounts: diff --git a/content/en/examples/application/deployment-scale.yaml b/content/en/examples/application/deployment-scale.yaml index 3bdc7b6f5b..68801c971d 100644 --- a/content/en/examples/application/deployment-scale.yaml +++ b/content/en/examples/application/deployment-scale.yaml @@ -14,6 +14,6 @@ spec: spec: containers: - name: nginx - image: nginx:1.8 + image: nginx:1.14.2 ports: - containerPort: 80 diff --git a/content/en/examples/application/deployment-update.yaml b/content/en/examples/application/deployment-update.yaml index 8c683d6dc7..18e8be65fb 100644 --- a/content/en/examples/application/deployment-update.yaml +++ b/content/en/examples/application/deployment-update.yaml @@ -14,6 +14,6 @@ spec: spec: containers: - name: nginx - image: nginx:1.8 # Update the version of nginx from 1.7.9 to 1.8 + image: nginx:1.16.1 # Update the version of nginx from 1.14.2 to 1.16.1 ports: - containerPort: 80 diff --git a/content/en/examples/application/deployment.yaml b/content/en/examples/application/deployment.yaml index 0f526b16c0..2cd599218d 100644 --- a/content/en/examples/application/deployment.yaml +++ b/content/en/examples/application/deployment.yaml @@ -14,6 +14,6 @@ spec: spec: containers: - name: nginx - image: nginx:1.7.9 + image: nginx:1.14.2 ports: - containerPort: 80 diff --git a/content/en/examples/application/nginx-app.yaml b/content/en/examples/application/nginx-app.yaml index c3f926b74e..d00682e1fc 100644 --- a/content/en/examples/application/nginx-app.yaml +++ b/content/en/examples/application/nginx-app.yaml @@ -29,6 +29,6 @@ spec: spec: containers: - name: nginx - image: nginx:1.7.9 + image: nginx:1.14.2 ports: - containerPort: 80 diff --git a/content/en/examples/application/nginx/nginx-deployment.yaml b/content/en/examples/application/nginx/nginx-deployment.yaml index f05bfa3c5f..7f608bc47f 100644 --- a/content/en/examples/application/nginx/nginx-deployment.yaml +++ b/content/en/examples/application/nginx/nginx-deployment.yaml @@ -14,6 +14,6 @@ spec: spec: containers: - name: nginx - image: nginx:1.7.9 + image: nginx:1.14.2 ports: - containerPort: 80 diff --git a/content/en/examples/application/simple_deployment.yaml b/content/en/examples/application/simple_deployment.yaml index 10fa1ddf29..d9c74af8c5 100644 --- a/content/en/examples/application/simple_deployment.yaml +++ b/content/en/examples/application/simple_deployment.yaml @@ -14,6 +14,6 @@ spec: spec: containers: - name: nginx - image: nginx:1.7.9 + image: nginx:1.14.2 ports: - containerPort: 80 diff --git a/content/en/examples/application/update_deployment.yaml b/content/en/examples/application/update_deployment.yaml index d53aa3e6d2..2d7603acb9 100644 --- a/content/en/examples/application/update_deployment.yaml +++ b/content/en/examples/application/update_deployment.yaml @@ -13,6 +13,6 @@ spec: spec: containers: - name: nginx - image: nginx:1.11.9 # update the image + image: nginx:1.16.1 # update the image ports: - containerPort: 80 diff --git a/content/en/examples/controllers/nginx-deployment.yaml b/content/en/examples/controllers/nginx-deployment.yaml index f7f95deebb..685c17aa68 100644 --- a/content/en/examples/controllers/nginx-deployment.yaml +++ b/content/en/examples/controllers/nginx-deployment.yaml @@ -16,6 +16,6 @@ spec: spec: containers: - name: nginx - image: nginx:1.7.9 + image: nginx:1.14.2 ports: - containerPort: 80 diff --git a/content/en/examples/controllers/replication-nginx-1.7.9.yaml b/content/en/examples/controllers/replication-nginx-1.14.2.yaml similarity index 89% rename from content/en/examples/controllers/replication-nginx-1.7.9.yaml rename to content/en/examples/controllers/replication-nginx-1.14.2.yaml index 768ab92ca7..b74bc81547 100644 --- a/content/en/examples/controllers/replication-nginx-1.7.9.yaml +++ b/content/en/examples/controllers/replication-nginx-1.14.2.yaml @@ -11,6 +11,6 @@ spec: spec: containers: - name: nginx - image: nginx:1.7.9 + image: nginx:1.14.2 ports: - containerPort: 80 diff --git a/content/en/examples/controllers/replication-nginx-1.9.2.yaml b/content/en/examples/controllers/replication-nginx-1.16.1.yaml similarity index 92% rename from content/en/examples/controllers/replication-nginx-1.9.2.yaml rename to content/en/examples/controllers/replication-nginx-1.16.1.yaml index f92f2657ed..0708cae4b5 100644 --- a/content/en/examples/controllers/replication-nginx-1.9.2.yaml +++ b/content/en/examples/controllers/replication-nginx-1.16.1.yaml @@ -15,7 +15,7 @@ spec: spec: containers: - name: nginx - image: nginx:1.9.2 + image: nginx:1.16.1 args: ["nginx", "-T"] ports: - containerPort: 80 diff --git a/content/en/examples/pods/simple-pod.yaml b/content/en/examples/pods/simple-pod.yaml index 4208f4b365..0e79d8a3c6 100644 --- a/content/en/examples/pods/simple-pod.yaml +++ b/content/en/examples/pods/simple-pod.yaml @@ -5,6 +5,6 @@ metadata: spec: containers: - name: nginx - image: nginx:1.7.9 + image: nginx:1.14.2 ports: - containerPort: 80 From f9f70c12658f4e423c367590c9b8644c07bc72a1 Mon Sep 17 00:00:00 2001 From: Felipe Date: Tue, 17 Mar 2020 18:55:26 +0000 Subject: [PATCH 116/140] Adding fix from #19646 (#19691) --- content/pt/docs/home/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/pt/docs/home/_index.md b/content/pt/docs/home/_index.md index 2fb80a8781..61c2921ce9 100644 --- a/content/pt/docs/home/_index.md +++ b/content/pt/docs/home/_index.md @@ -5,7 +5,7 @@ title: Kubernetes noedit: true cid: docsHome layout: docsportal_home -class: gridPage +class: gridPage gridPageHome linkTitle: "Home" main_menu: true weight: 10 From b1050541c7ff719857eae5daffa09252bc654faf Mon Sep 17 00:00:00 2001 From: Taylor Dolezal Date: Tue, 17 Mar 2020 12:13:27 -0700 Subject: [PATCH 117/140] Convert SIG Scalability, Kubernetes The Hard Way blog post to Markdown and add in community feedback (#19628) --- .../2020-03-16-kubernetes-the-hard-way.md | 42 +++++++++++++++++++ 1 file changed, 42 insertions(+) create mode 100644 content/en/blog/_posts/2020-03-16-kubernetes-the-hard-way.md diff --git a/content/en/blog/_posts/2020-03-16-kubernetes-the-hard-way.md b/content/en/blog/_posts/2020-03-16-kubernetes-the-hard-way.md new file mode 100644 index 0000000000..422ee78527 --- /dev/null +++ b/content/en/blog/_posts/2020-03-16-kubernetes-the-hard-way.md @@ -0,0 +1,42 @@ +--- +layout: blog +title: Join SIG Scalability and Learn Kubernetes the Hard Way +date: 2020-03-19 +slug: join-sig-scalability +--- + +**Authors:** Alex Handy + +Contributing to SIG Scalability is a great way to learn Kubernetes in all its depth and breadth, and the team would love to have you [join as a contributor](https://github.com/kubernetes/community/tree/master/sig-scalability#scalability-special-interest-group). I took a look at the value of learning the hard way and interviewed the current SIG chairs to give you an idea of what contribution feels like. + +## The value of Learning The Hard Way + +There is a belief in the software development community that pushes for the most challenging and rigorous possible method of learning a new language or system. These tend to go by the moniker of "Learn \_\_ the Hard Way." Examples abound: Learn Code the Hard Way, Learn Python the Hard Way, and many others originating with Zed Shaw's courses in the topic. + +While there are folks out there who offer you a "Learn Kubernetes the Hard Way" type experience (most notably [Kelsey Hightower's](https://github.com/kelseyhightower/kubernetes-the-hard-way)), any "Hard Way" project should attempt to cover every aspect of the core topic's principles. + +Therefore, the real way to "Learn Kubernetes the Hard Way," is to join the CNCF and get involved in the project itself. And there is only one SIG that could genuinely offer a full-stack learning experience for Kubernetes: SIG Scalability. + +The team behind SIG Scalability is responsible for detecting and dealing with issues that arise when Kubernetes clusters are working with upwards of a thousand nodes. Said [Wojiciech Tyczynski](https://github.com/wojtek-t), a staff software engineer at Google and a member of SIG Scalability, the standard size for a test cluster for this SIG is over 5,000 nodes. + +And yet, this SIG is not composed of Ph.D.'s in highly scalable systems designs. Many of the folks working with Tyczynski, for example, joined the SIG knowing very little about these types of issues, and often, very little about Kubernetes. + +Working on SIG Scalability is like jumping into the deep end of the pool to learn to swim, and the SIG is inherently concerned with the entire Kubernetes project. SIG Scalability focuses on how Kubernetes functions as a whole and at scale. The SIG Scalability team members have an impetus to learn about every system and to understand how all systems interact with one another. + +## A complex and rewarding contributor experience + +While that may sound complicated (and it is!), that doesn't mean it's outside the reach of an average developer, tester, or administrator. Google software developer Matt Matejczyk has only been on the team since the beginning of 2019, and he's been a valued member of the team since then, ferreting out bugs. + +"I am new here," said Matejczyk. "I joined the team in January [2019]. Before that, I worked on AdWords at Google in New York. Why did I join? I knew some people there, so that was one of the decisions for me to move. I thought at that time that Kubernetes is a unique, cutting edge technology. I thought it'd be cool to work on that." + +Matejczyk was correct about the coolness. "It's cool," he said. "So actually, ramping up on scalability is not easy. There are many things you need to understand. You need to understand Kubernetes very well. It can use every part of Kubernetes. I am still ramping up after these 8 months. I think it took me maybe 3 months to get up to decent speed." + +When Matejczyk spoke to what he had worked on during those 8 months, he answered, "An interesting example is a regression I have been working on recently. We noticed the overall slowness of Kubernetes control plane in specific scenarios, and we couldn't attribute it to any particular component. In the end, we realized that everything boiled down to the memory allocation on the golang level. It was very counterintuitive to have two completely separate pieces of code (running as a part of the same binary) affecting the performance of each other only because one of them was allocating memory too fast. But connecting all the dots and getting to the bottom of regression like this gives great satisfaction." + +Tyczynski said that "It's not only debugging regressions, but it's also debugging and finding bottlenecks. In general, those can be regressions, but those can be things we can improve. The other significant area is extending what we want to guarantee to users. Extending SLA and SLO coverage of the system so users can rely on what they can expect from the system in terms of performance and scalability. Matt is doing much work in extending our tests to be more representative and cover more Kubernetes concepts." + +## Give SIG Scalability a try + +The SIG Scalability team is always in need of new members, and if you're the sort of developer or tester who loves taking on new complex challenges, and perhaps loves learning things the hard way, consider joining this SIG. As the team points out, adding Kubernetes expertise to your resume is never a bad idea, and this is the one SIG where you can learn it all from top to bottom. + +See [the SIG's documentation](https://github.com/kubernetes/community/tree/master/sig-scalability#scalability-special-interest-group) to learn about upcoming meetings, its charter, and more. You can also join the [#sig-scalability Slack channel](https://kubernetes.slack.com/archives/C09QZTRH7) to see what it's like. We hope to see you join in to take advantage of this great opportunity to learn Kubernetes and contribute back at the same time. From e1397c9e2d2435f0034453b8696f184a1b69b111 Mon Sep 17 00:00:00 2001 From: Brian Pursley Date: Tue, 17 Mar 2020 16:05:27 -0400 Subject: [PATCH 118/140] Fixed incorrect commands for generating docs (#19693) I followed the process for generating docs, but encountered and fixed some problems along the way due to incorrect commands in the documentation: * Fixed incorrect go get source for `github.com/kubernetes-sigs/reference-docs` * Fixed syntax error in shell commands by changing `$(GOPATH)` to `$GOPATH` * Fixed wrong variable name, changed `WEB_ROOT` to `K8S_WEBROOT` to match what Makefile expects --- content/en/docs/contribute/generate-ref-docs/kubectl.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/content/en/docs/contribute/generate-ref-docs/kubectl.md b/content/en/docs/contribute/generate-ref-docs/kubectl.md index 797a0f5371..5930a1f452 100644 --- a/content/en/docs/contribute/generate-ref-docs/kubectl.md +++ b/content/en/docs/contribute/generate-ref-docs/kubectl.md @@ -47,7 +47,7 @@ Get a local clone of the following repositories: go get -u github.com/spf13/pflag go get -u github.com/spf13/cobra go get -u gopkg.in/yaml.v2 -go get -u kubernetes-sigs/reference-docs +go get -u github.com/kubernetes-sigs/reference-docs ``` If you don't already have the kubernetes/website repository, get it now: @@ -140,15 +140,15 @@ need to work with someone who can set the label and milestone for you. Go to ``. On you command line, set the following environment variables. * Set `K8S_ROOT` to ``. -* Set `WEB_ROOT` to ``. +* Set `K8S_WEBROOT` to ``. * Set `K8S_RELEASE` to the version of the docs you want to build. For example, if you want to build docs for Kubernetes 1.17, set `K8S_RELEASE` to 1.17. For example: ```shell -export WEB_ROOT=$(GOPATH)/src/github.com//website -export K8S_ROOT=$(GOPATH)/src/k8s.io/kubernetes +export K8S_WEBROOT=$GOPATH/src/github.com//website +export K8S_ROOT=$GOPATH/src/k8s.io/kubernetes export K8S_RELEASE=1.17 ``` From a2844f7390c671bc075ab0250e50598fe340816c Mon Sep 17 00:00:00 2001 From: Kevin Chen Date: Tue, 17 Mar 2020 14:49:28 -0700 Subject: [PATCH 119/140] blog: Kong Ingress Controller and Istio (#19451) * blog: Kong Ingress Controller and Istio * blog: update kong ingress controller blog * update date * chore: update date --- ...ong-Ingress-Controller-and-Service-Mesh.md | 198 ++++++++++++++++++ .../KIC-gRPC.png | Bin 0 -> 17766 bytes .../k4k8s.png | Bin 0 -> 129088 bytes 3 files changed, 198 insertions(+) create mode 100644 content/en/blog/_posts/Kong-Ingress-Controller-and-Service-Mesh.md create mode 100644 static/images/blog/Kong-Ingress-Controller-and-Service-Mesh/KIC-gRPC.png create mode 100644 static/images/blog/Kong-Ingress-Controller-and-Service-Mesh/k4k8s.png diff --git a/content/en/blog/_posts/Kong-Ingress-Controller-and-Service-Mesh.md b/content/en/blog/_posts/Kong-Ingress-Controller-and-Service-Mesh.md new file mode 100644 index 0000000000..9e9a4a5dd4 --- /dev/null +++ b/content/en/blog/_posts/Kong-Ingress-Controller-and-Service-Mesh.md @@ -0,0 +1,198 @@ +--- +layout: blog +title: 'Kong Ingress Controller and Service Mesh: Setting up Ingress to Istio on Kubernetes' +date: 2020-03-18 +slug: kong-ingress-controller-and-istio-service-mesh +--- + +**Author:** Kevin Chen, Kong + +Kubernetes has become the de facto way to orchestrate containers and the services within services. But how do we give services outside our cluster access to what is within? Kubernetes comes with the Ingress API object that manages external access to services within a cluster. + +Ingress is a group of rules that will proxy inbound connections to endpoints defined by a backend. However, Kubernetes does not know what to do with Ingress resources without an Ingress controller, which is where an open source controller can come into play. In this post, we are going to use one option for this: the Kong Ingress Controller. The Kong Ingress Controller was open-sourced a year ago and recently reached one million downloads. In the recent 0.7 release, service mesh support was also added. Other features of this release include: + +* **Built-In Kubernetes Admission Controller,** which validates Custom Resource Definitions (CRD) as they are created or updated and rejects any invalid configurations. +* **n-Memory Mode** - Each pod’s controller actively configures the Kong container in its pod, which limits the blast radius of failure of a single container of Kong or controller container to that pod only. +* **Native gRPC Routing** - gRPC traffic can now be routed via Kong Ingress Controller natively with support for method-based routing. + +![K4K-gRPC](/images/blog/Kong-Ingress-Controller-and-Service-Mesh/KIC-gRPC.png) + +If you would like a deeper dive into Kong Ingress Controller 0.7, please check out the [GitHub repository](https://github.com/Kong/kubernetes-ingress-controller). + +But let’s get back to the service mesh support since that will be the main focal point of this blog post. Service mesh allows organizations to address microservices challenges related to security, reliability, and observability by abstracting inter-service communication into a mesh layer. But what if our mesh layer sits within Kubernetes and we still need to expose certain services beyond our cluster? Then you need an Ingress controller such as the Kong Ingress Controller. In this blog post, we’ll cover how to deploy Kong Ingress Controller as your Ingress layer to an Istio mesh. Let’s dive right in: + +![Kong Kubernetes Ingress Controller](/images/blog/Kong-Ingress-Controller-and-Service-Mesh/k4k8s.png) + +### Part 0: Set up Istio on Kubernetes + +This blog will assume you have Istio set up on Kubernetes. If you need to catch up to this point, please check out the [Istio documentation](https://istio.io/docs/setup/). It will walk you through setting up Istio on Kubernetes. + +### 1. Install the Bookinfo Application + +First, we need to label the namespaces that will host our application and Kong proxy. To label our default namespace where the bookinfo app sits, run this command: + +``` +$ kubectl label namespace default istio-injection=enabled +namespace/default labeled +``` + +Then create a new namespace that will be hosting our Kong gateway and the Ingress controller: + +``` +$ kubectl create namespace kong +namespace/kong created +``` + +Because Kong will be sitting outside the default namespace, be sure you also label the Kong namespace with istio-injection enabled as well: + +``` +$ kubectl label namespace kong istio-injection=enabled +namespace/kong labeled +``` + +Having both namespaces labeled istio-injection=enabled is necessary. Or else the default configuration will not inject a sidecar into the pods of your namespaces. + +Now deploy your BookInfo application with the following command: + +``` +$ kubectl apply -f http://bit.ly/bookinfoapp +service/details created +serviceaccount/bookinfo-details created +deployment.apps/details-v1 created +service/ratings created +serviceaccount/bookinfo-ratings created +deployment.apps/ratings-v1 created +service/reviews created +serviceaccount/bookinfo-reviews created +deployment.apps/reviews-v1 created +deployment.apps/reviews-v2 created +deployment.apps/reviews-v3 created +service/productpage created +serviceaccount/bookinfo-productpage created +deployment.apps/productpage-v1 created +``` + +Let’s double-check our services and pods to make sure that we have it all set up correctly: + +``` +$ kubectl get services +NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +details ClusterIP 10.97.125.254 9080/TCP 29s +kubernetes ClusterIP 10.96.0.1 443/TCP 29h +productpage ClusterIP 10.97.62.68 9080/TCP 28s +ratings ClusterIP 10.96.15.180 9080/TCP 28s +reviews ClusterIP 10.104.207.136 9080/TCP 28s +``` + +You should see four new services: details, productpage, ratings, and reviews. None of them have an external IP so we will use the [Kong gateway](https://github.com/Kong/kong) to expose the necessary services. And to check pods, run the following command: + +``` +NAME READY STATUS RESTARTS AGE +details-v1-c5b5f496d-9wm29 2/2 Running 0 101s +productpage-v1-7d6cfb7dfd-5mc96 2/2 Running 0 100s +ratings-v1-f745cf57b-hmkwf 2/2 Running 0 101s +reviews-v1-85c474d9b8-kqcpt 2/2 Running 0 101s +reviews-v2-ccffdd984-9jnsj 2/2 Running 0 101s +reviews-v3-98dc67b68-nzw97 2/2 Running 0 101s +``` + +is command outputs useful data, so let’s take a second to understand it. If you examine the READY column, each pod has two containers running: the service and an Envoy sidecar injected alongside it. Another thing to highlight is that there are three review pods but only 1 review service. The Envoy sidecar will load balance the traffic to three different review pods that contain different versions, giving us the ability to A/B test our changes. With that said, you should now be able to access your product page! + +``` +$ kubectl exec -it $(kubectl get pod -l app=ratings -o jsonpath='{.items[0].metadata.name}') -c ratings -- curl productpage:9080/productpage | grep -o ".*" +Simple Bookstore App +``` + +### 2. Kong Kubernetes Ingress Controller Without Database + +To expose your services to the world, we will deploy Kong as the north-south traffic gateway. [Kong 1.1](https://github.com/Kong/kong/releases/tag/1.1.2) released with declarative configuration and DB-less mode. Declarative configuration allows you to specify the desired system state through a YAML or JSON file instead of a sequence of API calls. Using declarative config provides several key benefits to reduce complexity, increase automation and enhance system performance. And with the Kong Ingress Controller, any Ingress rules you apply to the cluster will automatically be configured on the Kong proxy. Let’s set up the Kong Ingress Controller and the actual Kong proxy first like this: + +``` +$ kubectl apply -f https://bit.ly/k4k8s +namespace/kong configured +customresourcedefinition.apiextensions.k8s.io/kongconsumers.configuration.konghq.com created +customresourcedefinition.apiextensions.k8s.io/kongcredentials.configuration.konghq.com created +customresourcedefinition.apiextensions.k8s.io/kongingresses.configuration.konghq.com created +customresourcedefinition.apiextensions.k8s.io/kongplugins.configuration.konghq.com created +serviceaccount/kong-serviceaccount created +clusterrole.rbac.authorization.k8s.io/kong-ingress-clusterrole created +clusterrolebinding.rbac.authorization.k8s.io/kong-ingress-clusterrole-nisa-binding created +configmap/kong-server-blocks created +service/kong-proxy created +service/kong-validation-webhook created +deployment.apps/ingress-kong created +``` + +To check if the Kong pod is up and running, run: + +``` +$ kubectl get pods -n kong +NAME READY STATUS RESTARTS AGE +pod/ingress-kong-8b44c9856-9s42v 3/3 Running 0 2m26s +``` + +There will be three containers within this pod. The first container is the Kong Gateway that will be the Ingress point to your cluster. The second container is the Ingress controller. It uses Ingress resources and updates the proxy to follow rules defined in the resource. And lastly, the third container is the Envoy proxy injected by Istio. Kong will route traffic through the Envoy sidecar proxy to the appropriate service. To send requests into the cluster via our newly deployed Kong Gateway, setup an environment variable with the IP address at which Kong is accessible. + +``` +$ export PROXY_IP=$(minikube service -n kong kong-proxy --url | head -1) +$ echo $PROXY_IP +http://192.168.99.100:32728 +``` + +Next, we need to change some configuration so that the side-car Envoy process can route the request correctly based on the host/authority header of the request. Run the following to stop the route from preserving host: + +``` +$ echo " +apiVersion: configuration.konghq.com/v1 +kind: KongIngress +metadata: + name: do-not-preserve-host +route: + preserve_host: false +" | kubectl apply -f - +kongingress.configuration.konghq.com/do-not-preserve-host created +``` + +And annotate the existing productpage service to set service-upstream as true: + +``` +$ kubectl annotate svc productpage Ingress.kubernetes.io/service-upstream="true" +service/productpage annotated +``` + +Now that we have everything set up, we can look at how to use the Ingress resource to help route external traffic to the services within your Istio mesh. We’ll create an Ingress rule that routes all traffic with the path of `/` to our productpage service: + +``` +$ echo " +apiVersion: extensions/v1beta1 +kind: Ingress +metadata: + name: productpage + annotations: + configuration.konghq.com: do-not-preserve-host +spec: + rules: + - http: + paths: + - path: / + backend: + serviceName: productpage + servicePort: 9080 +" | kubectl apply -f - +ingress.extensions/productpage created +``` + +And just like that, the Kong Ingress Controller is able to understand the rules you defined in the Ingress resource and routes it to the productpage service! To view the product page service’s GUI, go to [http://](http://{Your)$PROXY_IP/productpage. Or to test it in your command line, try: + +``` +$ curl $PROXY_IP/productpage +``` + +That is all I have for this walk-through. If you enjoyed the technologies used in this post, please check out their repositories since they are all open source and would love to have more contributors! Here are their links for your convenience: + +* Kong: [[GitHub](https://github.com/Kong/kubernetes-ingress-controller)] [[Twitter](https://twitter.com/thekonginc)] +* Kubernetes: [[GitHub](https://github.com/kubernetes/kubernetes)] [[Twitter](https://twitter.com/kubernetesio)] +* Istio: [[GitHub](https://github.com/istio/istio)] [[Twitter](https://twitter.com/IstioMesh)] +* Envoy: [[GitHub](https://github.com/envoyproxy/envoy)] [[Twitter](https://twitter.com/EnvoyProxy)] + +Thank you for following along! diff --git a/static/images/blog/Kong-Ingress-Controller-and-Service-Mesh/KIC-gRPC.png b/static/images/blog/Kong-Ingress-Controller-and-Service-Mesh/KIC-gRPC.png new file mode 100644 index 0000000000000000000000000000000000000000..9ff3e9bc21a2122e899e6791f9e7d4921443898b GIT binary patch literal 17766 zcmc$GWl&sC5a-)vfyG^eyGw9iJOmBFLU0J~?kpDE39dmB65I*y5S#>ecXz-1@9L`V z%YD15tNt)E{p&Z=({H+F-b~MnP*ah|KqW;5002Ybjf^?~ApFaM450rdh%({0|B|0- zN*c1aw>NwH`*(NuFE3BGH}@A87f;X6FE1}D-wdvvLGRm9>q^H$CH^-=j+xUa5bJnm1bh zZlnwpM|Wn%mn;-lmzfuNw)PA&Oa9mZ04hL1MpDCb{xBQEq2L$p=lsZsEZcWnOeiRH z*y+;v*pvykF9AoRvvkS}%|5zEAC6M|`WN_7whN|OE{2IS-L|=A4Q(->AzV=4x?)>Q zOiZZKYAy*G*0VnEj)DNjTy%K*C3FuI=}|1jo@Xb zhL07&!e9Qp{x63M1iA=gFpqybjDP?TB~$I{t$({>d5rgW>DeS^7>Mqk|4siU;to`1*atc*Ip5iW1on zaf6dhu`@=nBi|t?B?6QFR$->%0_Oom>(|~fXt>}gX&x->a7j2bF1*>aFfSHF*Hay=;V>}YD3a3F~muIwm?koXytd|&yn^YA1-Rt7Pz|IZRhbnxl>6tskz zpN*8~4z>>t0w`+ic}+50S0GX-7aur{RENaop*(-}Wh5iNW4!M-P6i^}=%1#QNJj5V z;p=q|NiQ{97z%l-f$#nqFKwME-fTsyclUY4z&4-&P=w$8Pw6ToM{|bz%-=Wf009zW z)YI?ZrFfA*3^G7jHupo^+>_zm3oH9+X0a*C>31d=09FbCM4wF;SE7ALRok0<|Fpaf zD}{I$1L?ezlHGcFoJs&sG3{M>s#}2jP1vnO?aa2Duo?3>;ae8@jd}ojT89El$VGNz zx@kCmVK^Cks6JZOM9@H2qmhBchYwM8q%y!Y+_y8Wn=>Cy9Dr3KHaJ57C)_XRrVwBg z1VC-PAX_B5EjEgC?)wu;e3<_-kzGwdCc@cGiXY61!5whIz(jzo5MX2ZF%@+3iN zLeueV%yhXBgXdL2_it?ZI)P^gv-dN0PI#U~9Wrd@FuiglqNA1@Fhe0A7`#FS*m#i* zc=g<(4c>gxUume)?wj$qWJ4MrseDEGi|{5y63(d#$Xge!b5Jg|y;6G3JNn4n>U{j= zNAyBljle)-w;fk4yOSa@o;UT*iSsw3&pzzJv%FWVzkTr)AFFL^fscAPnN{xsA^( z<)@rgc#8slWQW9SfYI-UHgof{StD<`KRqR*_U7!n@Ze-nJUVfP|Mp zoZ=Aw&z|v;{nLFD91W}NaBj>vyg1jy z&x5KVu!|*(SW1lXNkdI5(qiM*r3BJT4vgHFPlitDzWpH3bH+O=`6Y+oqY)TWkyws= z`7$x+d$ofnHH>mhOgi|X)hH#>+oNR_R|`$KP7*uRYTM>cI0F}?C6NGm)oGsk=PbB4 zBzMZ#JL0P%sTT6RQ7SITzeQ0^v%ppo-o7RdPpNJcUp%1K)?Rk5&q|+j(&yG@FM?^1 zvhDoNj4Pd8iG<%A1P2R8>v$bAE2);(Ge0PLwxX>1G|*-Bqi@phcu=6)RAF#B?1!H` zg=2;6gbzAN)*z`gq( zJJHO~_RE%fhxuyW8o86&#x_=PEj?Tnr(r(+!O5UE=Tf2?hG4z!&=b~Q%-qu zhrAvuSV^67|8!HaY;Q#vV=Tx9cpHehb2vaw=xT>{zgIE#uFkL6276ElwR`{aa5C52 zNus1sKL0E`NSEI7i?8j9;lSbeDAiQV&eW?WTa58m$Ol&Iy2aYkG=Y+H8?^-W{>c7z*njvA9s6o2_k)lp~-$TPWdf^MJfiBcog zzqTAifGE7g5d+3*-mhDJoRFAClRhH^C8De{q@Gsu?}cJU15Bjv&ub<%5JIe4#q4a{ zjpti79CPeSUPjlLvWrt$OhJWiZo-n%`326FN}@4A1JT?gVd2&%En{>EO|h zTIqfwQTQr6AGV0&_g#1PY@9iF#ZBMJ2A~C=|IFGXl}@|JO*Y$JzJY)Fso=U7Dz5Tp z_j_-8ZM@bLh_9mu%9T`uoV&fyEc?MRBS}E}2SSR|E7n&pceG!Fvjv2;I+fYBl*C0j z{>FmoHM|)f3!4}Z%!&yqP+%Y*c3wpPx0{2q%(<$!WrLbJz=u^dWn>i9KzjoM*dzSmHnMD zj?mVl`E-vmhz<8S08-e-iD?f0yV-`HOdn?KXV%Z#goy?hibuJYJs3_d%=PnTRIws= z^saI9qV@4L@(#U{*7)~tQ?OOhta(6(uw3^(RwgquJlH2FegVjBLsB#|O zk8Ns$=ovdB!)&I_lXL3l9aa2WOwt{xRvP;|FleSj>ES}FHUHLvsV5w0$H|a3!ZyyU z+o6djPFLcbN;ASXp$cpGD-huVmp(sfncs>p(FgVds0kbM4h^hF;;WtAR^H}!5 z*sGDyRTJ0*Vh27K>1MM&ZzSw8tiE^6^L-NQut;m@vD=`yckB_Z8kY8X;oHgJU4+Ha z2d-u0ETrOh#9km(fT#HrxYAF2KR)$sThW36QYyU6(kyHAXL1PN{0RV%BdkLBG?WCahz~T zlLo|ZmQIW4)4$d)12+P9GAKZFFdqBFfxAIo-W3tBP6e>Z6_#XfTU{E4^xSDmP!alS zGJTlRK=1|o9B}bRDW8+G?hKW@bEE;v4^@$-?$6wxG(5p+ebOiQ|1_e97gH~NkZ6_` zGhiDy-Aj?!e|l~diS;3Vd`HsIeK|n-TQY;2Vn!raV40$k44S=_h*e3=%@U7NP&F$E zCYFcO+yWU(D?xh3xh0?aZ$p3%B({R*&u%PPnp_}k?NCGA#^7QJvoKF}vE^m_Wfb^a zZ*Q(E<7SU91-wWo3XJhm^zkz6r@V84e)a6LX$Dr2;&hV4BC;yg7QAhKfb~n@4nFue{Bz|SlF_0emD)8z`Cm)*Zvtg;_U9bAPL^U!h zoNValN1~iafs8Ei@iswvdAX&e9~bTH-{2dj0Swr%i%Sq@iv2PBgqirS4G}3~_7AtG zJdw|T!sU)ZZ$01O3`S=V@&6r3n2tHNqS|#??%V;vNPl%bex$BJqBoPxXrwv_d6#b# zXZKmqrmd+a9E?Ga>gopRa3&J%^~kttD@)BnNu$BjqNwjba~9ejbPI|{H*h5 zXQ`` z$0HN3@z)5sPJ{*+Xw_g2`g-ra%vvW}kK53}()rp<@W=Boo8;G|iRv5{o~onvxW@`n z1`SZ13j8`--@VPY2znq6EMtYfeu^BjG2j=+xgMYUx*A z7XCBb-f;WSWBBYKL~rwST(kcdQ~WS;qcGTVhM;Qh;ZDtDp9tv@_bJF7*2SM_l%WCi z;@F@4T?I4YW`=zT^sAmlobxi??rbW6gK;=g77(fZ!kg zUeJ}df{Qt{&stc&CZNT8*7k?UB&vq{Bt}AQT{oya8|zHuXq>s=u8_;>T#S3QoOEIX znQ5fJEmtGuxT)h-v^|fm6HP~aGpr(PhWYes!G5fQ=7`9|1P)kF6@Q?y zt#M>@EbYZd@cFcNZ0upH!8zc4l5}Nm@gCRvWU0eCj+puqIouCxUH7s#Y$6CuFDI2) zuQmkjSF$l@>JdnIj?q1m+jgWZ+YtS2!@5N`iD7l0eG%^KchTn}w%^}NHzWV|i@#e=G` zlo5s5{XzBK1~(o%x2%{fh*=#bJX*9$3PE_?f1<&j-_F!YHb)rDkNrMZ&|xf#7fYcJ z&DF`@W+-JU>b_eGjv~5MwHWbC*lxpC*=mtHD^h#)>7#G}v-rO5+mJrlxmRn{u;BXV zxp_{KW8mv#sd-T^aw9(_fC&C1v@GDWk+`Ia32Rt5c=>*C->-Ayy^>Prkdd|-5LVAr z^wCgIE-(lx`!0Ca%S)Zj>~cdm>-I!**Bwj}OJkWO4&i(E%+Er|4;iDKGA$Sraizv1 zT^hbeWLYJ*yWlbL}Fa74sXn z`I6NszwUE}rj=^zlWlw<3dkO5^zh!qvP|->`Yy4;r2;+u15vB!V@Kz_dUMJ-L9*MF z4=j+z;PflHhJ4JErD289>VyKZ>NL|2TgTZzEhW%eG|@PS$BmXV5Y80l4?d)l;n{i!jRU<@v|>Z19fh!kBqQGQY8wt*sC;~rgc9fxq$RAfQ( z4%aMeC|~t{6R~+$H_u15O%qjx*yQP#Ur{1vqWuPx@&*fLUj-nhPc-M=a^!Sv7c ztl}LD5Vay>#5V_<`D6J3;3r~1BWuT6Aknql=L?RR;`B?(56e+>V*tpq=&uTD)U`7qP5h( zc`0qcf<+_>pW%Zx%6MaSDDXk%E!1%j$JGy_gk(}p zNym`fZ_KMnbK>?-o`&F~{rxZZUf>q@AN=gOzslMqp;QZDxE*6OH>4d4yw3@kwsCwj z?Y&LR5+?{PPCvx&G*a~eYPc;W31ZilO5g6dzMys6|(feu9{eL}34mmX~=cr4)J?!^0owO}crRwOF z{P<}Xi%M~R9io6vO%LuH09CdMcP0(A;%TND4dkB(FC~HChDLAJLd=lXGu&c8PP8R-9G@if4Fv`$Eq0@`8JNSsb1 zsHe^#O_EVBNG@?T-179&ez>&cc%gp9M4{?~%S8vOTR#?W&eV|x{!c0ZKL4X)VlXi{ z%>-%$HHxH$Q^T46ugw2||2uiG<;?J*gU~_G1Yplj^tY?;Q1W2NKH^svDdJb%C|oV2 zQHV%_VNS;!d4JkD4A~YDt{1UaKjOi+NRq275yr4`UPom621<>Y8#5QMZ7;V=!_`a% zQC}_4yRF*CzYeL2UbSx&k>P^eZbQ<<=px>PSibEw$DN;zSSPs%YI(b_3vDuz5JTEM za%Wq{WBSUH{7K-%(ISu5ZdH=$tdGuZnx+$b)dCw{#WJ`qm{P}tz>ox;mrd4i?^u@` zm>p%MWd%*XUY@VfDHd-@Z zZ>!5rm&rG!8w1qF4C=_Lp^D33U@2e&z?9$@?>8Gu81gM+Pk*FekykXvy`M{+DOQduMmT!h%04lwol z2^d{`Q0s~yi|+`6S8*9|T}z_?3+PL)-gS<=LDhh(H7&-x0qyHB%d^5RbMhZq1!AQ? z3H38z{dryxKHPT!Vk6~Gx{I#pYm|T z2PBw!R7b&*Ksecwyw%w{)+2U{s0e(bjn&X84v)?79!(EVfY?aL73WVDIYX zxrJ4TnEddrIMAiTv4k+ezgN9c`wTu?MQ-aKO-D8jNCrmDAdYO89Vi^EEswoy=z=xXav5xy`j^K_%VYQ2?+Tq_mUCsWqacL=m`24Q}XW?DEjf5!( z9}C4Z`)qS0&}qxfu|`=^LcpQtIERq@@!Aa?-sBez3ep0ZTz=kXF!LkD*PJQa$2Azn zIt%)P3LgruMa|*>F#$_E??1k@R-8`w-JG(QSV3V|?&xq-Y`K$3L~uOVh~Y0{wxtPv z98HOo+aF?Ji&CQOc~^f)^Pb~U;5+3S-rvjWFup4f=qmD>1-qqe;2XJS;;9GP{(jY| zUlB#jO+@gO^ExnBM{r~ay8bY@4$^_c^{flpl?EvHg*`NxsAV> zoN=6MLrq&%ey}x8I*xr9nQ?+G-e_`6!&--8SP|9iG`co6%lG7iDM?3N!c#$zjuCiwQT63a2J0d zz$Yl5gbi=TMVEiiCi9q1+e(%$!9LkaR{edn4;z;u2s6H#SR|udxZI(DAuI5CmnAks z?(OdGMMEnrnOo*A(*>+U${FGTr9y%mo3yG@ z!0Qu|QHbfWM7df==Su~PNc0b4TWW(qqPc2taD1pGg##hND%rxfaD&{O9E#kW`u%>! z%viNJ4`I6#eLn>|iMDMl8tIN7tWYkSA4Kh`iVi0?^vgu#C($;sY!2i58(tMO-AJ#V z+P2gfC`z-UE)3EzrtBrm{VT;r51CSNrM|5^AITP5{ZbkudKR+%EBHj5193QIGjzol zozkVc{ngD?D}8j{T<-pG1}Es9Po{iVYXlwB{pV8~Ez>wyq9-cvWsIjpzxEkQ!eOFt z>BFZ8LoM=qDB77q39<8a`{i5|bT>Aq9McOq!9rwE*?pP?+Be+}llqfC{O9g?nZgP| zKQKEPeqGRiZN0d5W~0p9!Efxz4VvGwwRBs0(^t-()K?sWB`mi2U@HIZEqj>9Uf!AR zZiFd-{H9x5R^8L5UDCufH$|92MZLbj_Q&A24xgb!?OW;c2$1ev%m*;2wvB|L&e7GpQ|5plD%EVu5KQ=!90lF~tkea8j&puA*g!c^FSGnA3qB*(B8 z?^sZqCs`o#f5!s!17*9CS#igH`tk5!{A<$H_=c_3<_|}^&nE-Lbb#**PKM}2$%N~MYmm;xX8H1lIQT#S~Ra>3BkQqcJC z8|{yr3m1}oPL?$;WE`>%R!n)|u><$|&xQ(sH%791IB2{zP9%6GyE4mvE5M?R+(uNw zBbW`_@XL4Lp=*xhv83neRO!b_F!=I7jDrDY5p=TJ9ix|L>QDFU@oo0GV@u884+5h$ z>XIYT92Mt3RgILB(52bvxJ*pKd)J0wq(|NM549{K&gC(nQVtwcVVH`9Ahk8!mf6k1 zlC$e#zIM>^XJFTY8U)VJXps(_Er1Y(7tl8fBI!(FHPQqwSt-P$p~8OoUSbg|4e<^| z`KrP1%x9M4JmX_Ua}zRoQqC)4$w?~Zh%D{O7<4d%>$`+Ej}imQ{;d3d*xfNKyFID+ zm>CIp!oZ6*Vk9c2<_fVt#Mv6|{lHK)VC7=gPXo}W+V)|@@0j7uJ{^OajACqk} zh`+2B4hf(>dpgMct6(>ZL0$;e$2gX82Cb%UN?fWvcuLrPj;Yj4o} zTfu6*i-9gr&S=A9#^^MskAmvMtx?kx?yoMkc+(Fv=7D`HhN*WX3ufXOES2?yYb*Rn zx3Bc_PE>#w1Bgzs>E1CdT%4GK)RfUu2-4%{5FXuNjsVA41&=slaM3e+WK9jB_%(+m#G7j4Vd%B?5{16u@wlO)i!_Eha)v z|Ajvdaq@Z4ILa3VgtXEA^tze5;|Rs^hS&%d9R6>;@QS8mcJwB&1Dnw1B}eaNRC!)A z%2nLid;Y98j&hzY{z}4U=4;lEY8lDF0T>D@&D%g-QeR-YM0k+>>4md-=>ksjSEeS3 zd<11*%b4odQO?vD-<9i<8T$(ar1O18=K6&nyt?D*OS#$}35ciMG6kx#?~*8_&T5xs z_kG+x!Jl&ge`~WQF~)yWr}_Pi1WL8D;`8qeh;r2!r(;Iy=_`c8ngyPm7l3zv`7yuX zO_guYU&Eh4(O?vNii5|$J|f_(J&%_Is8P5m?B*4Y(e*3DU-5z|?|T-+;bkFbS6MLj zy~*!qUbW%5J>;LY#s{>{4cE)o5nA}J{Lw+toN{o&8pJvkLh@T#_=5~kGLqV+j)yq8 z|8LUGb1|mU+Ob_|@zf2CeiP>-_f`RW*}#JZoMqwT1A#!nKc`K%#y1`S32s!4fON$b zg>8^;8H8hUKj=`4feQRZ5^>cF;bYqE{_2}lf8VGGu7W`wF)r|K-;1ggd~i$#kR?#z zSfYY$Hn39~q9Dv!R}&qPIK{i6t;V*NL4tjkP!l7#!f4@Vt5B+DXWc6Ti4J6)XCunh zL?k9C6@#l#jQlUUnF*?_c8mjLp{+4&gxDA`pZyWUBL{`D?(MibyUXU9Lb zq@3f-6YepkDA8e0^1hAzcYXjE?Ah;n1d;q%aV~)B+7Im@FdCO&5Mrb}Y9mL)omH7kkGS0pn{jOl%8={yAi<*imx%XFrPrsRq}`6Io?&4Jc_$A#9ua$0pzD= zBhi!ErX3APxUTHWJG|$(j6M#}QgfmAaQmA%WAkzUc8946KBDhYTa8$!@m>Hu#^GAP zCb#=8mJIyB7m!TWt6{!Lvnk0fba_d_qUo?GyHp{*Frlp97+$y>rG>BT-^J8IDmf25 zDL~%J9yLoybGxg#ORQ07CG>c>Ko25~0oD(*AQHYtf{rnQQ-RNUXziZgMl!dxl47E8 z2x%4hd(fO@Fe~S0@{U;ln1A!GzLsaEf>G`y?j;qmPCSUilgZ2scJ;Z>Rcr!@i^L5l zf?StW!pdwUFQHKkFWQq^bvSuE(GpII_r4;1f!9tMx>RMz1~r?s?_Sz>?pG@fh?&`)NKVY<-wW#xy0+U-54m`c^y};UF{vbrmOSTYS51c zpg0jtXPAT~9y?Ct;>~2f+Qcm&`z_Ig1Da)YM0%>2MdI=1M>VsFW=bG@#95;P;je;D z++Tu_xFEA|3_6ZH#-OWZE-kdb#--RuruGFJ&%q8GC2so&@gCc5`{_AriRKw%tYm$Q zE)KHrc1^U9Z5d(aA%*=;d8HcbL+_;5Zk%ibvck14JYX(wD#jx$@im5AYs5FvJYIm- zJc-pB#X^dH*W^7B*)L+y(&on0kbSTOT_Wr8g0tZc{yq}77RGOr-GLAGh#+lv=hDi& zLuPtU^As|bYYY3(gvl=dpTiGpZ=#BBFrbc__~DJ+3WVm4lw~Li-L{9LQMMCJjSZ96FcCm#5u9)2WMQaYsfbir|>|^_;2>SOB4f= z%~!&^Wj5One=Xhy7?X~VSRlJ2im21d`$tIzU4YM+;^{RyxWH9yV-HwW5e--RUX#cr zjI0RrVV5o{&u?0rpM$3_2gKkdnl&1RQNXbv+K}WgrY`3IsT@CQavKnjL&ijhv9XJ zZ4(zA;kOfs9){&tR2mrw z6#hZWZ`Ra4ij5TsGk?K4q+)3HNZU9~6YWM0()jkkfhpWHqS94)X*pCBPI)(h(xgx* zcU3hFBKl=CK`AG(wA5Peo`+aUEc^pak{(;eEyA*J3ar4-7gdJeF^fi$dy$FNvH?z7 zr*D2$#>oBooDVvR6(7b{Uufx)=nqnq82SFf&;o`C_E8j7GhBK@s5mltP|_VzdGsd0 zy`R=_uoIotH3zFnJ2MNJ)IHvPabZ^D0$<1eC}ruNmPNc}g?3zv6jY|s8cYxudwvEE zSQbfp5dM5;>MC`BC*W(zehJ0f(Fv+iBa;XL$S|4%P=Up7?vp(WP~?pswy zM0(ux*WnoYaJWNB9folV+d61n+5$(5+xmCBpj2Lcat|UZes|nSDWZwi0=_Njx6#0S zyhUlmU7b8MKbJ=IC}S?vnqFZfPnbAdC)UQyTD@BIJ* zphwhzM`rbMEE0wVfP#=-gjUKnyI%VgrM)rku|79_E1yQ+7`UsfRL;oiGUZcJe{FE_ zSem<7{%c(IhxWKyxe|t-B&M%-h6u>dn^1Ifh87r2r-1rtahT)J)Btn2W0I4mrYQDK z!UazM_8*Wpdz~1mgjc!EqV9ZO5u$_|m!#vTE)BNhA(}iWtHH@GNCd^2MNvagjo>Ye z#VEPLeqCcL3zyw&4DWBHgqjFIzrL~U7Cy)`VE@r}t}d0!B;lC+F!uCopubNiA;2@e z^(&Gi`f9JEk{8V~-rPF$i!w-$)iIQK6wS=ZU2sm@xx1n$6(GrYeD&?EV{uwo4wvxv zDe1Quh*MgR<#Cmpe7$^~Y}t{It)c+goSx+H^occ^Ct>G%&;}*))V}mq*q% zSUV$Pt~oiR$Q2@qX_T#)zyfi)njUWR>N=|GB$Mf{5Q5PpH{Q zT|SC{?cq%p_)+i-%*?R{#?K?}`a6U8iiyH(3Y#ELC%8=PtEKgyQ8qMIV!hD0Qx0Fa zb?h`Grc>X70h7zM=vvHCs~j_S|8Ev-X#WnBit&u{1@bzqZ{?i^kgYHV*!| z!uecem;@qH3z;V&Y=~0cn>(cJ4j6$bUcI@GfvWNF=-7Ro$D#!^8D4k@b%S}bLUkV; zPac4B3nZgx1|#Y?Nsf@lOBWcN@3+3*rzZu${NaxV-QlkRNtX|++@_{cSK9Z$iF}}A z0966ZN;Aeseh)z>Z*ogeWnIv{1BGLRL&*Gt6gH?_6Miw3$5*9rGK6h(@>3j!*J4&h zSq!FnE=5M#cz*ecG|*eguY+Z`fvd>Kf^X&5`BnnHtirSo(_U23xl^51y4QnPvVQw5 zw4@$?gAM;UmV`U}sv6Y%4a?ue$!vZ7!>+(rNxQ%;wNLV?T$J$Gw3;~*%-jxh!N32wnIG3OE-8=>6W;$&f~$5()({ol|8tgR9zFuoa!a3q z7GjR-j|ADafdk>05@D&l1Y$rMg2B-|>FRGtH1y@Anzd}XS^)*LsJ3XW~lX1MGmO*XBn6v zk_1E&I?@MR%_xEA@DXi_Ju>jMM4N}q=SHHn*MlEXLAQKg7@?ffZE~KN;|^lyZ25Uf zsWT`NXIDf;Rt{bh;vETkdB{1x+sm~oz6O25ft%zMwwF>gOBm+%%-i)j$x{J_dlD9S z-ooJB&N8BmW$U559O_!94)A@P8a!QJl};N*ZPhA@LHwk(Jb8@zyCoV-1@j4>UkzHP z!5Q94at0=;IyWnZwt=4LMj1kiSSf(y{pA@lR>I!l&P}SgLECAEC)uO#T}l)gOwZf& zdG;oK_8JPouJBAR_xydQaY}z7oZa@U$7fvzE|K|cv9Th%<=!{LLIu|WgbZgN`*-Gk zC!ey9qZtBV5ZBQu(U|TxD)@0iL&iwd2xSAF$!iB>(_zU^4<5~26NVjq0hqmN3ZD`5 z3;6<7(&l77*g&n~RO~3>wJ;+U=@*l<9-seRdARg`Pbx)OE*hQpY^!90$br~t(R^-M zN@Hur6;tfR^N$66Wo)^BcTo^TXYko)?LGd*lMc+&D?eo8w57?6U=Ch1j(N+NeZ9>M zT`^o`1sd~osJl3ZM7#BJ-<1&T$ezx`>thi@{&;lU^om!fGYgk z(oAP9Clvc~!F62!ih^ltE%9y4lqEkwZMx-fd!(z^6uIcNY}Z`QaLgj{g1>qt4A(!gH#AUE>l72ka~3K1)R6HJ(zJ{0~21+;vCuq%rB zCyQz>e(l{P6+W4b8Fs{qBlR4ouD#BtmtuVu6Fmm$)G*@fN60ht2IXI|uPv_z<=&T4 zZb&d{jxc)H(p5(TWwngNyGg;$nHMy$ZHE=(cc-_x;cA>TCELMbh23ZteaM zwHMHjbxE0Kb%o!IkR@2i&)qKy#NjGG!Dp>n0qAu8@7$k6*=yfgu>6FjGa$@;@c5D4 z>u_Q_MM>nyc3Ym?ZnVsk4oVJ6JL~03R!b3_q7tL->rs37sLJvs?2cQ^I1#xM6{``A z4y*3)+3T9wPAyd%Wo3#-W-m0mVdf-WJ6(gU1O-XCESktVNKWooH2vJwnxyoWF!ott z5NdlEko1$eYnaQ6fFPkooK+UivS8HPE3wSRDFfs@T2HO|zs`srb>~&gSdID019|XB z2Rl%M@+#*j?bdF^R%{bT%-cB36dS_}#olhCupU`fDJh3@FXuT9Dlw9mmEC9bH^d2d zE4UZ5i2u$@bp~!v$?$2eh_Byp-<}fwt|j68f?q%Qm^|Gdnk*)VMrZi8GYhX%;_V)yYSV2 zS6KmVPUci|)gW8W;>7M#W_@*%b+N1cOUkT>4(6QY@p0k>Hx<$UOK#lSWCNi>#co1` z$#-zr6-V#{_XjS%Vdt;p4r*NuqI=&ro^O8eHSTC!=IFDFHu(hScz^_L)xt;g(7aYA zdCBYF`w0rfiGMy?GiiJiW;JiW`j!K`hPHJPhXfZSBWu$7sSBq{C!Ed*`D!A2Ts~6F z=&VJv8VIl^t|?Och!`Y+D}^;wHNW;mDlUc*CVgfV`|Vu$&2OnjF)T)ZHo37YM9OG&KY-!SwyPQUV!5r3P+ z0$gOlFX_&*MM|j-CWvg#`E3M$$H@Y(r<)LRo7&ht@%AYvv5+Z8(VYEWJMR53h)SLo z@EE&R9agMJ^(ZV&UmW*^xA&3H{|$t&6+N;Db?!WBe&OqUC4hYTIXHsM2vnpR{u1+x zQO~K@OLTSWj|n6Fx4d@_}rimUl?Y38@!^O3gyK4OgCG{+UrG%o`s% zJlQEc7hbO_R34d~^OICiBVyMIAE4o&r#LO<7r8MOb}><^cr&l4grEm74bV;rXyru8 zw8P#{ZpPb6NAgHFUgfz(lmOl=+YmZg3W3z1~#adEJF0bAesft z*ga{HU%W(2JX>!oUIF6RTGte0%e1geLU6T`#}~UpG{bizNSekWo;_&B>?`ji&?%h< zV=hckg}P)muWb{|K7Ac0&#-h0Y(_eCE;*O>*zCoqlM25ni|CJlZMRDCkBcD-ihzkX zxJugMM$Z+62arAu*Hpb$=;$klpM$=r?aU73REuyfq{tGy(zxT|g`qJJ!WV518hJDj z@k0y;uUdF1J8~T09TNRbItvxL(>9m<1KJRueESosZYYfaFQw6`HZ?pEr01!ir5X6z zf_J!&2Hv*1K%d|hMXhj|28i&!HkD(woy{Aoyu#_lzeQCF-Ff{*-l{r_wUt6Cl4S=0 z;sWU1rN|qJzu|Niav%N5tT53P>M0MupzXNwcdAG2Ddi^P2|oLKTku;<4c7=>5MtKG zS?l>H<_JMF^ScwiR(|13>;1g%`(xkeR|w1)Zq!rKCIf*^?VrEs(%_`mEIM`MAu@n) zuEc2%X|h*Y$q6!y#;7@**2z;MCKbQ|f6c+^*i1-X_{g%^R@BQW9KacH_)~5!`9r0g3>aK=;w}f(?$MaF38C`n zQGG#R;%BmkD13@YQj%H;WBGg`gp>DfTmIlgHLI7ToQD$B_MEjEKnOD1{tc!K>v?s5 zT#DX%`OcTp;j?=vF4e>zzG#ND z`D$fz@|>q0csgWmO|*HVv&(A%v+64iGVFvFzVqNH2KoNFn}zxJG1O5S3Mg=)wMs6n zbvzUH(;+LYTLp}mt&LL)T<~^INiDKKV1+Hh zZKigi0LCEZ!R~leI3rlN4~DAc>$lT(w4%D=aKt;H^&F0SZVNM1s~7z}AQcqHZhQB2 z1KmRu9I|!YPWqq5oY4Tug&QzE9)`G~o=wfgcJ|OG_l}Y<7>` zKiEgO7605BZ`1#J05i4$)7d_7Kt0dz68l=J3=exXY74l!1|@HFln4PUV1=6m-S}u) zIKNZ{HS*~X3HulcwTN`Tn3^4rO@xRAE*?$#lCm`+nEycZPVB$9BtUe}Kgk7IA{fAz zpNZ|j2CeHh!qQXQ%=;)uKiyRJFu#~l%+1yNw#thci$*K>^~W9!(ZaHYVCc|ss$F^%rKSzHZiL(a1#7#}!VZ%B;J>h8;cbFoBjR;pV_!SS z09*D-h?vXI{Nc*rAZnxG12Sajs#|0rn(GjhiV!Hil4(zCp4ay${MRDW*T&C?Ky|+> zf1foWdfU23$}uJzni38kXRq6Q=kOfkmq;-chWI-asXlvQWGYBKjCe)^d25B35U}#) zK(_3p_@06Ydmjr-e!9s(gIhyR^_`M>3gjAXDSfsq>K9M4w)qn9KEiOzvF}E-0-;nu z>tDK`5>F(wFrW=nVhY8eY@T=~3BgGB5?Rbj#BYjS?UIfV{;8`fZ|}gt1~->*)1{#M zZzn4fcOcM^i4YD#E;we2J(vf{HVWt@TAZho0jfwF4AxUF@msF^$>nx#JHYTvaLICt zqN7V`p_@Z=Q@!&s;qO{d^-9#vt78Q0S~6E?nFU&p2lmlkY00)jP9$(L*I*B~O6fIkAm{JUzF_zX6TO_@=xl4( z6Q6Qod&ri=@--I^1h_}!ZS)on?H{zRyv*?n0VwOd zdAJOZXjaV&Jms5;L5YkIOYg4@!ap7A#AEu8&SVw4#q^>tXF|na9`9$Ayt(bFEdHn- zoj*!%Z@y2eQVaTwURg-vdz9m<()wbj{^~n3 zu>2;|`Jf>8x9Y1uFK$Xh&#>_l=r~2cVgyu~O8fZIzw#SoV55h5`6ga@!Msm#EDNZq zcRZ>#WctLy?3Ah$5e@c|_tKk!9|P?!B7ijLyb-4en8j?)Ir03xVFSASd%#e%F!gZd ze^qSI0DhxS$1OAm01$)!{tNJN5MMs9tDCq?=LIbw@}VghHt_&BiGcO1GA~buAdU^= z$^l|SU)3W}Rw*~UpSgz7Ox{hYQW{Oz^RRb#O8oS%^&25tJE#(hRL6rw5(5Kg-4-wU z;79s=#POJP`^Rr>uh2+K-E`!!SO#kE0go}KPhdhZLGUns9yu-GB@{7nekca zKs2jBta{Fq=FY$OTf^fni65AMs5F;BAVHUr?e@+e|8#FnWmwI}+?(&q65-mAZ@)mV zd+%4qq_-Rn4INB1{abX7wQB~6DqLz}kQQ2S|B!OAk1_+(8*he2296Ekj4VtF41)^= z0FOst$# z+TY&o)4l)i`(1zX<8oQMEj#K3!d= z&HnuDd*$t7hf}^A2e*{JxcjzQUE>{>b3yBapB|PKiT6~K(!PG!6tU|3YL*+G|DNC6 zI_tAocO}!7o9|Np3NaqqEqL#8ghs>vnx^&}Jq$s|L?7&!Zcy9so$~?XhaQGT<@Ev% zJY8~3Zw!A%Gd`KkB5)~twZ8C!cl+WL8p_+F#SZARW^)zX3Gc{O?c{Ca$eI<&Vx#JC z!_K45Qj6=rH15VJMX`*BzWDEFPnhMPx^mmqFY(b03V%N@>+-QX;vRZx(v2XV-Nvsu zA8fBJnk;`(ym1-7!?9eg-@dgfPanPeF3j{JcfPI4e!i<51!`~2!nqEmy|iNb@m7BN zzsOGu?f9M-SE`u_H!E#OU?^S4keBf1#2>N`eMwF9Zi@2zOZKcQq}`y46}a4r{;cppKxTY_4GN5 z=HL3a&d+gfPi3&`>xDCC`p!~iJy6Q9seA4J?-TwWar@5lT4+JE(lU-4=bxN8{o+64 ZMdgI711`tgfd{8Bc)I$ztaD0e0sxx5g315@ literal 0 HcmV?d00001 diff --git a/static/images/blog/Kong-Ingress-Controller-and-Service-Mesh/k4k8s.png b/static/images/blog/Kong-Ingress-Controller-and-Service-Mesh/k4k8s.png new file mode 100644 index 0000000000000000000000000000000000000000..8d712f5783ba0c1ee38e4995705c51ef8c760d91 GIT binary patch literal 129088 zcmeFZWn7e7+dfQ+!bVgWkw%d&C8e=I=^BQR5b5q18b!9$NSB0k58W`JA}!6(C@{oO z5<|!T0&X|l&%M9DU)~S$Q=GZhTIYG3eXQ$)l7cibAtfOe78dc#7td6%u<#nNuyB*E zUOD~E;z$`57CqL>XOGofv6m;zT&R(W^Lu(yt0LP`_pw%kw!11UIXT9RTt*7nT%0gG z4s#m9ylY>#-?}}p(NF3M6P@331Vdu-bex?=z|Q?;@w>&&M-HQ zMJlI6e>I%`V~3#SBL<#(*5e1&H~ZQUYexYscGS`wSLuDQaQ^QPzBr#j!R1$>16 za%P`K@|}O#wvVJL;f-DMcIyn~f7=J3DuRC#(f`M)GW`oXOC^jt>mi#g=gm`5P$~e> z`RYUglDg1AiW^>ghpbk;qq4KWm}p2U{g3&G(&ixs68v;BYsXxqn&B!`HAy zl4Df4a!cwsYBk0`++)Q(Q;n|^ps6LY{7ZZAX|yJaoHGVLY~l|4=~cN>O$#jBLkNId z>vN15!=*i7Sbjcg_w>+p7GSUuz1y({Z_!d=K3&!K%54DEjfv8OXEHh0o2jAi;jWX> z$?tg4P)CK5+0S{KV>;nR@QcYh?aTS~b<>?)29bUHg0 z%`N?J$m@?){<qnEwp4TTFa4fkz0DtzVZD7mOY(AdVHb0n5W_SGkkprRUT%6q*FkiMmm#~5NHc<4q3dhs4voz3qMF^jS+BQoFS~uyAcw}<#O!)L|GqDew|J!#rUcaUmF`T7) zkf`pp@4^-u(mpl$c@}X53rzM>ZrHvKE}_WqI-)Z2I+7j9qhM^&zN zl^t$+tF3lTb&L&t-qkzSZcg@+YIx@7gS@_j-+bx=3%K;lg|SwoFKaFl^a8Pyq;j&w z3;JHzitNq#=5nG^Y?2%$7^dXr_LBW%COJ}U6Tb0nTIP6Ak9{2lzp*ts!j-~zhAOJl zI0U*ZU;P%mK!;M9lW7w+-Rk@7x4>I_aQ5x3{td>l`w0UeTATCd)TD3@ojbvC>ax|h=nzlnlC}Ie$ zZIdl3nAZ6F?sGafwT5TmDM-%`4m@W_RR=a#(f69}zYwq2Xs1?ZuNgI%&*81)`&1f4 zedi-;r`Cw<`*$^J1XGq;I4Gziu*GnF?)%Dt{r$5C=@0>sx8PB@94~z3@kMDhbrQqf z*x%68pZrY8e_`nP3?dW}me}3qY#C)@0{)Z`p~GgSMW%NSm?BbOKh96WcrJ^p5%j_X zBeXXvg0iD_F^r)$(fLju)JA&sm)osh6oTOEeLL~;#>|- zLwHXZ7y?B%L2ifGQH8q7LQcFzpmBs=gDgOyh^@i$?)iW4u94sdA|8%>X*BTGEYjMtsw19jwUWROTV^M3+*49!eah zMpRG>$om-IpAP!WA11DLJjySG3LIC~V^(B6{H9uB@L>q+bJUsnxibL*?_Q{Wa0y28 z9JUO*QHMWVP)jSSzcyvrvLNGEv_F4*!mQ9bh2$=!_5Af59|}#l^EUHr;pHeF8#q^| z)1{Ci2}D-Q7Ug2g4(D{A?(pu7s*ug{a@O@O8&ga_Q3(W+@l2(J!(}v0kWBq&b$_7v zes;dCt7@=ZLw=6h2QM z%!td9fuET}oL{D@kUSU4=`~w{ecKSj0V#C-8LVcy31cx1zHSQH;Gv&uC>mW3tUOJmUEAssq%%}n?FA;^XZkMUy~sUuD8W|p)E(pj1a2k_?(^R5OMy-|QAuz3o`J5B z97qs7d*9hyVCIITKB;f`%0j0}&_1T6wB|uWy2s~yy1ECcaGDhr=>hJO5Z2J45VKIK zzy}O9oyjw;yXADYrf zzZ}%x&xiiEv-3un6BUgYDbF@zmD*ro^TH-_1@n@ zcSruCaWZmgB$h5;>*H)FrMrvc%OG|)jPrY^GHxbP(KnCJJ)Se*3apvG{wLKu&$)i% zW5ZF(dJZ4?qSHg|1wV(nTu9+r(QaT{{T=8KHj|VP z_xmYpVnca9vh^mdDMWZH?T&lP`9(Fj%4$J?J(ijsM!b-X9|P1f^qcUXiEC**`gizF7ur%mEjj)&=jw4Ixk+YUiiBSzY7+aK*S$~|6Z z1g?ZqCcOHwNY~xfT0(P#QuP!bCxt*bQc`R-zwPXdP%E4nmFW{~*qh_7uYQ-K&eid^ zJ`@=KWG+)nJ8+CiDP>}Zxj}JCoSN>4$IGZbt&bgKD;Vv-cSq*1{lSE*Im{GldOX=l zu99MHMKy1*8J~Fh<3=a6dKs`X^L7<;EYp`+ z>?_ju9CeY=LFf;Am|>*^=qIz;pWCQN&IL;6guzzcRjzJ81tWi12 z;3dfF_10d=>MYR{zD55VES>lme92*FLww6220|r_QS6~|Xs!rohmG3+^7hcleiSYn zINgPd)6^@2-eg1_!>?tLQ!AHLSni=W=guqEH?!V4zta{jU2k=lW(~$pyD=fKP+nXI zjY`!$I|AmuWGEbpCPrTZizE$U>^DrDhi1;JmqB^YJp~ebzsWq-QARY_m1rNe6#*y^ z0QYQ|zl6+{?l0^IemCZ6H|$^=Rv9Utg*+FBlMx*ZAa4$ZH2@KQ=|D7_MU(MQh3uS>E}6GzQOkLLl=2GTwTg)_edF|4icM7IeMGlPj}0s37VGRz&4=G> zTa!`|be5K0V2E))bXKyqW(J7OCFiGYT)~S)yNXbpZ{$2PNNL88E@3?SDnhzHSk2l6 zpT!LHS|?Kfy{Mu0B<$xLHalq};U1^|A@)mOs(zY8st9T^>)#eu=$S-2t1;!zmKG1S zdr;N+-{;h$t|sJsOrk;+;Ph{l%jt}dkTKcUO z3Sr-))yl-o6*)|+7}8ablxdbjb2yh>q^+#fU-2W5>I^k<4e&rtPLxc%Jj%)GH5fAI zpw%YAHY*$G&q0GGw5Vdv{q^JT@Yl)F>ay*(+tH!;6Ov=b$T^X7?-+}2{qI&+0%87b zHmHrI@;npIz9mn=hrQswk?{5kNcc)a@x$BO>||SQ^P6n5o7Mey>g{JYq2u>4w6K}b z4Ic6b79e-dhhELw*&23g9Aj({ncUcSd&c7F4ETQ0`yx+-`(l*5zrYJ!C6C%Zij?;) zg?VkS2Q%F!+Amd|0Gt=&6f(2ECFV@`Ql~v6>`S@iRZBMg$yAN};r1v?7e{b|hH|D- zACRpz%e@?VrM$VAI#>FfbKcS@2vjf^Si)Y>p3%OqO=Cf{v(F*;;`Uz$QCD3DL7TM< zRnS|Hofw~>0|}z89)O74K-cy`MRGo!L*MLmZFF>d=(xC9DYnq?B#MDI2V-Q{W9g}h z&bN^OlsBW}{x+AhQ+-nG2*b%bonkfT$aYZF4>)5{Wmi>U`AK_lI}g>|aQWI0ldzWY zNr>m%)sFDo*X$9rQ}@4fC+KUB480PDw?(BI$EfpHe?rk28plsOv*ursGLY#@kyKoA0h0PuI82$q_xiI`SqloI$*#JBt{ss5*4ogFriK}cXeEy> z3|GjNKsV_ky4fN5RSaz#8G-ixfrQN^_WmUsAXc5CC~h@F_w?h6l+dkbd;>IKy?fcA zrb@k3A25~5^4HJ3xZSB2_rmS*xKw1Uf*b4X9%^T^=yBBc--O6ksO*LFREMinmy|GS z8$;9EP$)I(xN4Zi$$D%0W;jZ@r;B(_SyaE;#xnT*;pz*nP0yY+r?;<)Qxsn#kX@+q z3Bf0m7733?P9sH1H6GBxgn0K+URu!^+$g8WXJeEfy2{f}FP{qC=|7sZ+o|2WV`1U^ zracY0(YoO}o)Qfd2QSyVD&lUNX(nO5?UrYigq~-;&Op{{1qqzFEWtlD6QxWeQmH(n z!-HvDiiT9{>;o&M6OD_ygOeP*McJQOHQ$JK14wuRoC9yFxIy zeS}ciZB=&jIE)<56aQ6n_TVw{pHu2%mD4XS9ZO3QsIil%A;0>vN{HN|N;O@!ViHpG zFnp#DPXQL%e}^yTHoaG2cc-56%}r>r49OXua9Eet1kw?_Jr!lir>3%jk@ z3S6QIw$HQH@1zO^vUV9=c*mRVj*1CogXlQg#cHGc*qK(jEWRdwSPKJ8XzlyMxmp>f zG#T;v=#`-FhW=kSG@{)@uYHSi+S%3a$Z$B9-p6M@gU0>{@9bBrS}AoUse>>HmDO>f zD5jQ-L9p9w*uA^>p|p@`swuT1xL)%TGE)3N8^rt_MMtT=k>H&x>C(NAA&ejeisWvv zrl!h5F*DgoUkpmEy}fQTl#pB;?B=Z1PCnik!%VBXJCGLWeY8{}`y5oL6Cp^ZV(a>> z!PP@dze4xXd6?(q22_Kx!N$6z;wpFFEySS=`^gq8m!q>Sc!g)CiY+6REi`Qq_9JI^ zkI|{&(7RQJNv+O9b7W^SdcHuAOxQ!s_eUW{@abHkE_j;+-l3}*?V_K&Z16Zo@3@rb zWTQKmgR|X2$}uiX4;9X4A9?Uo3}liyczm?R4n2IQ9V$r8wBWR((@JrwL0KR{F#lkk z8FmR=%lSTQjo8l->zw#kcG{+TR~4+vAP!o`@RR3xbPZXD%ybr*FIh4k7>kWmIwlYACc8 z41gSV=-ElT#g*gvmfQ=|vE+|Ao41(qVNarmO~s@< zFFDg}+*4l~^P+WTr}o3*ii)N}&V6qiICaFg29@Xt?y8pyU!`HKF{TkS!)#vL9|HUP zRWedH+DqCaNUd!<(W^kmw2Rm=u2+B3wJvkHPb(B|_(YwBqvL)fchvZ|eJ9Ds=N&k- zhR;nsr*R>*Gl`r-*%x}dDCl{?C$YS~s1BaAO6uBF8MX`=Va6pXYc5r9J@O4bgWXgpCCYmuO!T88RQ)56haIsahs zqcVG&V_nqZOF<=L6CO6*+PN!jaySPI#cwvgldhVLkaU}&$R!}hZ(SzR_tahBBK&;{ zS>uA@@z3`E({#0z2I7Zv*?5I);Te)P%lT9nWMm1b}DwDJ)-L~Wu(G_DdKdUBVKhHAs3BA5Y=7nYjI*xHARr_-mY`S~nG%Fx?H0$2j zDC3yAo6++qIcLr|Pf<2SmW61mjSnO29jgh{7J){fGim#B;@?>>XCx81gmcNp9JO&l z9;hg7oP>?GN|OGI&>7`bm<{{H6t}ltvP@~Ime?khP)-^3Qji4ymzGf09+8ysL!qee zE$Q}}VceeJ<6|$uDkB^{V{|%*ux)0zy*)||zRD<`f zY`2Z|GrQYO(lmiZ=6jQ0v`dJlerZ-Tf*Yegd?$08W8ZLbtce8L%nLss z5(c_jeQjADi+yimt!+vo<#0Y`oIV<)$YO|NKq8j^Ww)xmUzLzn{GQFDn(M}X#=Y}N zF`%HfAE;TYHKnaK=5d}p0{og9o5%_Hmj#eebNE5U3sOrWI*G!=D0Vg&jNN1U9o4(M zq!#9LG$dRDQad^SwafBp1hv9+z>M|x_B=P7oPqfcY8?s^esJF#QfbgF#td}8+kP2k zGR7Y@{kkcH3g__Ff2t%KozeOsf{>V`Hmyli93fKP>!GJJ{z^aKI{PpZE@M4NT-6py z6I!K8|CE5FjTL0{{7uOVM?Iq%(o;?S6gX|Y;C@A33KwN1KB`>o62iHAkYhR!GrS|l z60)$)JS$sK&A}vzlD$70)MCTqCc3&0l|+Ac#$X9bXguPuVJX`l+AEaew+oe*V#cIG>hZPApDCjA{fIj>=c~Mv)aLE6XsZW24d~ zKV&Pntr<+fpZ=fckDsQ-G#?&a%13-}gNjzl-;@3}@1^oFK5glRoaP4W2On==KI)%} z%krM>W_?q_@+|u$AYQlS_Eq7ZulgE;s-T$MT<C>mH^FiDV!<58Xrw#K)DGLTAfcxY=YFNlWJ8A0g^B zd6EY+gn7Pu#H0Mlb#0|v)pj;NNHP$95_2CaaCYK*vm$2W9E%lOjqGARO>6&m>f4iw zBXPVA!=UrvHBH=4_5EHMqmd+Dzj1d;pKF>r2VJui=<5|Fg4b;jc0Uf9AmStURX=4o z1lB7g{5gy-6L}RTpkXhpTMf>%<{0lH?W>(kdy5ax+0RFO=Je64TiePUh;-_a#+qN zckevK&=yQ0Tn;o7Z(FXq`02UiwX6fVLicH|Q0Y+|fiucHU+tR7O{`#1!n$6$P8VNc5p>f0xX&H}DL#IKKuX$tTsJb|y z9^dyM+y6Av&RQI%O1=oi!fO+!O{L0WH`h?FW^ZP@_#RXv^126Qx$D!>EJaWoZp>)`W6do2bIF;< z1NO&6>6MT;gLK=*foX1}6ZIU%&{&~ld|zX-?0SR+va(+t)-pr;U7NNtt@+`Zc&?iH!Z| z7IbOJ#O9;xfCtJJ1RNP=h+*Ku9sLZ&G2RN?du;7BeDae(qH1P+5Vv!^yvYZ><8iNN z8E38mex>c9hHUOEAtZ!)N zHU<{hH{>6_(yERN7S(0t8O~IK$VE?$sgUtGr}?axproBJf3T<*nQ=6OA}My@`oPKAcquD4{!*gEqF_#87bHc^Y+Q zJhoK6=07b&(v&*M;OK>-+BO2`1X;mVWa#X0#j=NHhNW`zNLgQu@dOqG-51Oj=hq&} z*|A}|>Jk%4r>B0<|22?qNx4e!u1zbmrp2{xDstcW1e2CSa2Ag>qY7XMenw}ult%bq zuvpnW6#9?gIN9{Q-X*YV%t`M#PtBSB$T|MoMo56SCeJeO6K8HwEZFN_#$Yi%3eIJY zLVs#s^kXYA+T+;s!)^;_+Y7Iwm2KM2o@PE6I3C9KmmZz4a@l8Chre$KS)FssZkXt` z`}E$LH%c1IcL=2a4YIT?#`%wMDOFO0_h+qfG+i7lpQc)>o3$usC@BjiLadoGmWRXD z;Xl}2JT>=5B9{kK+3aVpxqm)Cn)xBd$s~E~LA;bK9e9bfIAslu{C&E zlSaF*F8~=O;MOo#((WOicDNiEh=h>2yGpK;dRp|;eLeg%+|@AAW?{86^58MgX6Hl~ zOL1a8v(c&Yci0;dIjfM5n-^y74A!@_?5v-3mMTztYgnt*R9_c6Q*~`gbv~=y{1WGmRcJ*m^Bi}o@N!_ffg0oCGJ3$ZDC&y!UMM|+sKJq30)bb2`i+@&_V1q2vv5v zP0@PXv>i~t*TOF!QPUwA`R`{^*b${VEfq#`- z>>Vx1cs|M*9|%-u1`<<^{r0+BBs+Oaac>ECn8zD6OAiE84}N?S>X+;Kd>RNEt$18iCK)OWizk;>R@Hi%%* zxlXb7SfDwehVNt>Y$Tox_7cjQHdXK+i42S|V@w){Z>t_zS4Vd*DP+`b;y*g)JoplD zOG>p9*mPdvooq`s2LC1ExnCyo+=7G)!5qW4Hc@J&4XV8dlO}xjR7PAlJ84ZceU%)> z;#Okg{un+ws!h8$_SM`Q?Q`XL`XF_D+(=H-^>Kdrqv@yeM`+RvuC5xJ zcllz8n0+$S=J_;ncR#JWK8VcQ> z@U2duqtryopbq%U2Fu64I(BRkex48U_it(K&F~j|N>+NPt@N6!K!uIlgx*Y-x=p18 zr{K58z(WZzuooR5MJf*(esw{Y)OW#;=U)J`sz)lKB9mmy>>L#~A@iu>jL8{KG{CHJ zLozu(s$j@^rJkWSgl^C6bLf!i&WMbAr81Rs6i9NP>~H)vF6J0%>nDe;k}Ah9l(bk` zTwY5tX3d6f@gT>482H)G5r?l%_h#C>cZBtJc~Z3(3e6AA2-G$R8JAWnPUg_Mu+`Q~ zy>cks5X_UU_dwDFr&xGpZ?Zkj8N?iryh%DtDrD9{vw$W4dIF^uZ!*N z{8pzP&9ecmM=!t3wJB2VefEu*HcnrKD*`#*8Sdc)7p@tNR50TA^i+s*ROr=sf&0`G zy{wR}#)ry?>IRW1ejga3h%uU|t6*hM_+jP+RrXbqTuFTCpY}LP7L*ZP8c&<-(7glY ztfZ|6H2qy(Ed5Dgc5}56n5|+|_?GpNd53^etc8h+W^;tH({`EF=p6L0gd!ORb<9+_ zN?)`CWNh0FB-s};i$>gA4}^l{dh#f6^q71~Ya$0Rm2mUz8CI0KI+%@Y;<4(nsZv`* z`Yqg*5*fjVNU0Y>ftY5#ai9 zsi#`PBUkZ9i23Y7O2b}EH`^_u0=+UdmZfQF?eR?6@Qt=DNvgvQ}@c*t{CXqzcDjA6?qlgXX!5X^co6lox( zQ@`^)Yro~hLrN8C!e@Wz6DsN`xi{!n=qaAzT`gKLprSQ(oCnXTMLgZ??|{yiZ`!kk zQl)EsQp_n}D3pAV{h^p5#8Z4R^p5*_ZFdzz<&SzNWd~AO=(22!(@)!z{np~WIL7+* zgdNOr0y&wic&1EsD#@|!W5oWVMUBU=8iNnn;WWyXKbVG{_r6qRGT+`Z@jm)AT1%>> z`C}mwDQ-6v?Y12D+CS~C3tudSnX{*(md)|pClSh(Y8x@l{?UCstD5<=fToyx7k8%}Vy~<5G6zFuHe|J~B zoNh(Ej{CFZeR^=`0{fp_*W}a*<>rIYGFvk#_6XJtrkF@=U9d4* zMw0YS&d#%BkEWF)HCq$#(3R04QkRKHK~)@Xy{b&_{jt(`%H)0`@9oH>lc9d}f|=?= zy#W(CoxwQ0x(t%z-*Hd^5lp#+XCIB*-1;Le-IqC)eV^A31LpbcEfFHm*V81tGN+C< zXmqL{6uu!LF|$pF;2wzZ^H!%g*yt&p_*&!pdF>1@K};Nv>tzGW@tuATdvS86J8nbP zOg${sVcw{BNnWe3owk|4Igcn2JUNwK2Vs-$N2G3?u!e(V-JYH>Y$wMSem5%xq_oNS zO=IJIBY_CyU=WgDur~WEGl}VLYh%|H!B(wOQ99x;^87a?^Kl^{x3>Q6bvQ#;+h1BCoK@5P$>IlZe>J6(9a}|UTq=*MSq778Nf1a=7bX(V zN=?M>mZJLEP@~!LG3v=k@cL+v8Yzq?4I%KZsgAa}W@dC};N$ybiPR?MZPv+TH6h5r zmOpFPCpwqSzpYqIJ3`9$EuCIK8QDqm_YeE?Mg_;yOc3ku_IF&^?`+jp7l(?ArmHGa zt5m-lS}=*wir2Htl8xsfwNe(tl&kK4PnK8;or7+SL_pU?x)*lS0syZX)iF1rKhV1V z*oarN{v;qs>yx%!UPRX{pR?JMxqVy$=!DO(30)|9AH_;cM=_A>b&>~N9aBXA$_hmn zn}jZIxghf4Cwnm<#rIp23CRnG?lJt22GW{-=?dy^@3lapw7Y7B5bC#8YP^iJyby{o zQKx6U8m{bqHDk!Xx}_&RtK0>mm8|c|(bOX+O((XC%Mv{@ID_;Wb5{28amIc;T6tR9!c9BWjmp8JzViQa3Q<+wC$g~Rc60&tb$)C~V%5&zy`TV9)&%gbgi zxT@$t9>&n=2+n^#c_DT7n@&;Oo7tE_iWqT-AJ)(+WC&`Qr-j z4L^RKde>z@{jT1?=QjtAKa~)2DhK6$8K$Dkp_9V4u1Yp1s|+E!pL+Vc6Z^tc2>_j_ z#BeK^I@>EBym#k!^He}`b!VHT$hOc5v3RGfduj2bH;ac}A94dYEfVwnevq}fJ=%Qf zs)jt*Z=fmEei$bxs7^YZAoxf8?&NDTUv%;l` zzplU*(bg)U%C-{1&|=;_Qq>?Z+JTpy`?gDNx#Lx>Gb>28J>Sqsswg9xcYpS+vY>vs zS1?0VkWsk{^+232ky`BVA+Ep>N@{ZS8Eh94Fe_nji}9iL;>lzi~XFYV;Tev-n4 z0nz}cmG&|wxI&&uiYiRhq0=fMF4c70?AK2VSz1*=1Wu}wJE}Y!1g|bRwwA}Ybvb$S!8?ibSp2at6DI!SB*7}t9&ntC=>~3xxvTdGL=tsk-&_14rOuRkq z6;>|hX>CYQa6sAbk^E^fJA4wCY)i;^W5_lAxZXF;sP9*cx`aFA{LAvo8$(KO%kj=z zx;%8M4x6^%j2YINwfMTI3x_bJ7~)tR9n@eBE0fil2Nu77urP8R_kpBx&V|m&a2k#C z$SGNs3d)c2r*&|2%qY0HHL0FwFfsf=t7L7jN>vkpTQK4YXb&PzIkb4C$}?dhZ?26p%sb>Y~M@$ zv_Xz5T8RTU!O3g&jF>Jsu&G?Ja|i`{tB(s0S*+wgbAQDUKmhuEF!AXP?$ZhA_7ZGy zM8a+0$bHT~z-ThI$~zqrptuTgH~Eq{=eikBCH-;Y@OW-?l&|?NW@TyM+i)4+&s>}q z5BUWHhG+DIa+L08Yw+YA5gd?R58&J0E1SzAM>>3n3n#2jE!0Ws8_uwcTMY9&s3jvL z7&JzETPCOW`fz!PW|B-E4kT_odNct4mG(9vl)6^4bpMsb08mEp=YVjo4wOn_rj#(R zq|XP5XCHxu)u3_^^H`%CUu;by=^iDph7Il`JeZKJEg4wEo zx>W{IfSiBoBa)8xXW9#HJFUXV`JCOJ97yjEIk+`P+@ulWPn~Sy(mCAC_1c^Af-6c? z#W9C1`9 zxQ=O`y)|a}Y;*Cz+x#2uPgP7nYq$bv&QaB#WQ+ZY&OIeK#xgUf4R67Patgm| zf5%A zN8ux_#8>LSOcBedR32IO)$L9jE*6OhmHCLLfi!4!`kVH+rsXTS3J}M)`hy{(xYnhs z>sqTXcwr1~RxnZH_*qWeG7qohP;ovWJT}K*I!?B0CihpQ zO;3RLpjC&US}zKt1pE5EW+6^o~9oK$ZTNt8SKz`d--y`Y~~-MpC(! zpsn2Ke$&U?sy4jP)iW3vrc%hbXIq1>AWZsm7?uGF)q@ym!@c^x zgE#Hn8Bqy3%6naPfuyXAbQ4(>E!(W)$sdukwYi#|8FIj=-3-QmFa+t+zSB|BKH^8n zYgQ1-nfSt^^{2ebV{&%wSmSJ7wIg|WA4(} z%172F53zY1^4vT1B;n!xWz?vkMaki0hiwimlXbsjV@QZn%tuRZR zAA-Z%G6xCL(rjxRL4HmO{hNn8GqOQ6YkDJBT|8OcTD)MqoUvTYO|q9E(K@9wIxVop z?yb^bn3|sL166j}3PM@U0$puqT-`j#?Z^ZT^@kDKd3iY-A+ra1kEUiBmR_R$y*Gma zl%@i83(Dbn9Nva8p@E5FLc)Ofjg4CD`p*ow`%5QB350l_G8N4NWaJR*%vO1t~Fm*%oSS0RSXfZ962o zmP$g^E}AYW{&)gJ&Jz}!U9AsDx6IQUW3E}c9qEF{D<6(7wJnGE#sMUL^SDKSe>q); z7-KTIcLR1+vcH}U({M)XwOLG0*u@Dxxh^^i4~5ez&q`*gi&H|jM^iV5SiwB_pt-(4 z7$)ba*R#Y=x17ctq6ymR3Z$yDl8$EE0WxYDR25B0@fA!4#wNEXuqMJt35KM@ z!5vm*-BKRnlwGm=_k0>BPZ^B|;we`y8f5%9z(68rq-F<9DrwP5tv56p*=*mdTy2Sz zGm6Cdg9BV_fZv8Pc*mH z_j-W`->+koiK>iVqN>-!w87%%&qFcU7FDXn4e2s@jBdEm2TU(`6CVmzr*@)FJNIlN z4GW=n*bC+|0%xxL^jQD+#q;HJ5+_zSd!Pxdv$~^=`E*Ux%~N;BCU3_1O)^C@sLrU} z1XJVOtZrSNGH3~$9Md!wqt+SYI6L5l`OC3}p)t2bVALcHu(dfkfN`Y12DD>WJn58h zFKmtAzP{JE;M}y`RHG?*2VfcIRGrzmgf+a?5mr1mnv`;T z6Kz~RN_-Hfr8!s&_sn;&@yyKdFDulIx0QAm+d#bu9DflzRTL6Ix!Oz;VktjydOE<9 ztR#v-E=Ja_J~*eU1|RG)%#Xc6D%lEm7s6cBGK5@jGx&o+L!AO?n?dE!)xJ2`!82A1**- z^i=I?!^%U%NA>lIx+gP1fx(VdHldw5I^h%)>g{tzQsQyncEo}Eb`?dbPZMQ6EvRmo z!nHHGZ)$ZoIq^mf;VzF%u#VR8rf9IBM^UWxM+dOkO-;(-sl5IvHyLTg>(w^U@x*x1 z*gQV;=DI!~3qZ1o*90c?ygOW0|BxTGUZvc0xYO)DL+R>)V4AGU-O`XrKAdUMs&2?{ zGI>AXR@6`I7bmG}Ym6gvq>x0$Rx<@Cx>T#aA$xqQM9%L>rf^}7d=t2D(`>Ne{G5zt zRc%&3#(*J_(4_?xM3At3o!7U)lp{YNN%$UODo=74{)^$jX=^}9HF%qc2KjflbpNC+ z-(SG2Fc#>_5J<^3U09?$YfrXNuz{Ez&C!ruvv69xhxa(Ksm@*(Bj+p4o(7Za**l-T z7zx6yu{drJvkXx|TyhlGlhRdYg0v{LRiHc>%PQ(QIiT_t8<)*@Mkc&6z(l5#3`M*< z;FTipnRR6l!z>M=wT5g*Q0!R2s+-GZi&4b;Mwz_t2}a2z`qFwD1pKc=5Z?c4eb0Tc zj@6F)VDGW62sJVKGxPPO-yd*Sufmlcv3U2K&~sN?7zqM3q=n?GtAJtv3OA(Rk&*cP zUePIQq6!RX#$(S<*Xb`8mhsyv2H86OYl^1ZGiF_h$;!rT1^(Uh{JY=8n+d{K)xPh^ zp6)e@T8>R;O zqd4#tI2~7jfN_PsC+o$HB%pgkDHITfsw7(sHh(~=WI+FUJM6lSw}l)L`JCelMwwGb zM=4zd9XiQlN?06?zi-ysev$*_ey z(`kVY3iN$;Rg`P^s8_t2ZGiidQ!mklw7UfwvC2~X!sHpyh!#gM!i=PB3ntjqd* z2%VoXN?YGM8a*0U>S6}2h0HCfY5`aG^?*4B<~eD@g;Gs%e2v>)&#r3`0<3zm$%1Ek zax?)ojw>O`^4_93`uEbCT6Xe|Uc`uq@$}t{l9+XPs}A(Han?TyUw5wZy=gI3EptrQP!rj}w$|2Kt=yF9M!e-F3sABnBp8 z+fdP@a!zje9c6iJK+MfQQ9wwX?&QUH2;4dtUXw8Uy|$0Ih%8Pgvb!Z%Z=_0hR*| zcBb%ZC@b>G=guVIZclKqw!b=Uf0u zpgY-ZS!HP&vT*aPhy@Dk1OLJUL=n9J|KW!SeR|Itz-v>X#tZ) zcVt?BuU1IR`6x9%pXj$ozG@LbbCKt2{2Sml>Bj?^=D9(Zm0&%F?=Kvg_vDSz4IbKJ zlxv3yo|j8#L42Yad?eXV$>9FFnIZp}FA*W2O>yCRfdwO7DciYDib?}K#CCYxuqh>qI9)0oxm+KAU6UC zt2>^O#1{~u{fH$LOnYOQCUAlc_;NYrA%niUz$t~EJSbZeJonqG_lC!EgAn8ymTx_9Ewfy3_0>5~_7orbsw1v-q z37C(4&6QB6I_b=2h$djr|I{^A(8S3b-r<2rY{t_USYlB9Zaig^B^NrP$qXs7*Jt?@**s5wHo~o3zv`vB@T0Cgp5swmBFzs_KtQ|m=&eh-QEwzo) zN*CGh_gfDnRy6??IAABix^#AL1Kv;;IG_WDx!~e}Rd@C6?<9(QLXQf38v#~>Jn-R! z+xd5WBprZ4!hH2O!|9K1BxQ+WsilDCyOkL(B@DP)=PG`Fr?^KBXZF7da#%P^0H(FF z^znVUum1M%@O{S(nBhwRhJ)!evZWV?dS>8mje`fU$Xk%NA1yxtUaSN_Xpm3m<Oj|6Wf_pFoiA$Y1{hZPAtzSjZ5&ZL z?Vm-bpZed-CmTc`pj>ce>8d3eptpsl+d$_D4*2c46>Y7v+X1IdJs{Qz1#2DO#sZc~ zf^pgo1G|@J08pQ~C_{O1B~ytV*uEC=(9L0%#?9j1x4=DTQ(b@z#&0v#$MZ+0DJVb^ z0~?iCSlWP4Eq%Xo0iU7-fG^)4#diVx6m=hJsna)!0GG_>-=q;}pV3Q!M5EHijN(ry z>)8MlWWJ}3<--Z+vEqzF&ldt*tBucOH@o`hQ$CUf!1m4gHKg^^fZ%j?VJ*e&RE>3> zvK@a^R`-DCM`0b(6AFA8mx0z?r)3y}CF zH%~87U#7iEb`Sv?I=%H%aRZPlPJj-ud%jbA#TTaE{R2fToN{3Mr}xZ#LjV;~i46nL znL8zTZEkaN{^R2v-f`9Ge7YkKn~mq43~lT^;>|t zYI)^xKB-afCGkU}KGWysY+J$KXpT7nzW%{U;({{(b&5Zr-a_{G|9h+ppj5yj>lY>#CYk3%n0-i61Mi#lM~W&+QF!#+E2 zfGwTRuuS~v2mKU$B7bBGkbZv-c=1|qI+|Gd8N6FdMlML{nO=HdbP-GBr7Dn|4b7Nv ze&;i}=vnm;9=~U!y(tW-hvg{$<~k?4 z4b#)L)9Im1x1_zz&2!RBkZ=_qeW=JMX<6LRc9@uc9+#1##zyVFtJ@)>-?WSsmwWOX4$N$8@|| zaX4z>e_pnIAaP%#sB*qu?#~Du$8tk+G%l9%Bi9n+7K)mi=hS7DoYC&V$vm(>rP*YgF$9Q zgGDir>&j>W-v7HWBOL4ot2U^=e#^M=ipD{7d>4NVf14HBM+tey1>BsRtS?V%60Q&* z9V|BaKQB#sILb2CKB(w;ON%q-v0QUYH%@e>etn+}#Rms3Mvz*a3>Jwrdg#spi8bhtN<7FIx6HuC~D zh_^sC?EH`1d_gb6d|mOsnMJWM38_hnWLIiQ8E%eboUU+kE$ve63Ofh`8_^Q;?bMi( zr$H3`N6&!5Su@%_gC=FnQ(XHiV%lG0$L!j+mhE1DFnjJ+Ve_z;uXuo{>nXceAKwa* zj4%l9`1@?73F0f8MSdHT6qe{TH}(*;br9tGf?K!nJQgRuLj=%{DK2n15vLwU(mzTD zwEfc=DI_>4ME~@;XK3=UXPpb*s?owc4LLL;MD0`y-`H#TvzKM~!v!1ufSFup_rH34 zv)`k9k)E8Ut}^T+Nc&6wL`Smqnuq1jlm%p?0&>;pi}=V)EcfUhZTUcJLjq3$y$plC z3clOuYys|NMgowA^EWyvvB#fG^bH8KnyCsz-gc9SX-9qdxNoQ4gYaa4kE<&(AWA&? zI5~(qc)Zo?7RetZZHNG;OU5p|%nT^AgsPp?K#MD%6!VXD#`t0yqEqQ3QnT^_x}2Fg zLIS+c87sT*|Jp>1xGzscoS|(zr~vEdO$tH)iwxi|aeyF7($&$!9?22-9R3CIRlLwO zPQO15JF=~p=|=PT2Ve(tbi&KK7pcdnkIe~{BIX-UM5fQZ+t&FF5t+-97gwg&g;vPd zN@bc@X<_h;sy-0baR~?*3T8OUW_shk?V8lFIS2t8J-`CO#C)1VRLY;_?fr!7Hd((1 ze75mEO`HB|w>psy7V;*gQ%r9k$>5_99U0G)QfsX)LOHEDF17Kp$kf)pdDq+5cl_Ds z=JRx|1px{QN|oYRTBvPZd30*;uoocah6gcQkMKz;{1G#c2sr_ecATJk)6H(D6Q0&H zRT9WtX|Jqa>l;vR@L$e5yy>e&+qX_8d1WbVWRz3Kw{Di9$Yrs>XXQ~BRO?JaOuV5s zd>ia@g-C-5=CKzKwE6IB_?=YU-e}-?ER#cz9pJOW88BkcHb zy`{T8Yvr4ZaN%U#PhOQ0VWc>GwSql@$eXx_b!myYxw+3h;&+kbO?5K8Z{CygIa?OD ztu@XxW(@68>1-yfQo>@VqykD$ZLno|WJbdHnEmUdk}sW~59kZT5y*?;{ysRoN3YbB z7ckYi7+`zub0*~zaB*=#B2I&xmT)v!s#9O=u3eb#4ofd0*U1n+ci+p^nln2kUOMV^ z;4rrLeIy>b3oXyGYeRbdF`ITXDk|zlTDfk6pDTY^z^l@X)XX0BN*&;M@~oz5$!luW zvv*+2fz5Y6oSDm`Ek8;{1p@xO0mR3Jbd2c7!ZW3ZWeOgTddBzC5#+$G!|&#NxERUe zx%||YH)FcaYED(m=l7*gc>u$4BX*7yxQYobmR8a0{xnPDycRk;K#4EoF>7bl#vfim zeV(7OR!7tQT9lb4K{+hIG%PuZRy?%yyZMUJNIgp>X5;7pSK4sDfj$Q#J`Jc8^DtANG#s_90PMz9V`2D zet)Shu_MDr>G_^AW%|W0q%>WY+1l7$Qf_a+hxcfOqs#X)4WgnR`CO832uU1{>?wFq z9TQvo>SE4DNkIGvyAr6>1fc6B-Zm6b&6W|~B003i>ENi6!vj}wEIoG>VM8!SEZN>c`=JXHRK zaU~{%`1g9-T{2Pdi<~TNZOqg2N2(D{w$bnw)WHQQ!x+J@rH|R3=XAt=8)_cpfOf@fbmzf?b)z#+K=7qXY z`pHOd=pL9B)uu9yHdF{!y)ENL2V~fQ10>@4nPd6Kagc+Sd{H9qlK`}{LHsT*M*O%n zE~dxSNTxteoDJEkSq-a0TAXs=2hAvD;Wu@TUhFW}FLgcOfl8W{q!8VKi}yw`-=pgc zcHdZN1PEd^ihqp@xO)lSHs}?26KfJ@CLh4UqqMVR$;e(C8+jWb$6OX|nJyO<`j?w* z9XmM)@&lRbU#u0=0pbv$^$%vH`yo*h7t^_BEBs!XKSR9xJAavJR)`X|Absu|`|-|a zwM_zxjKeB8$<#&nXLV_5X`c@WpYL0MHNTg+#=g(g0B{yPjs81I)%Uj&?eSy1Yt9m& zAEE?rQcWuLdYSyvY{xC7!8Xj+pF5ce8f=Gw9sK{Lz!3VCC)CTj4eH_?*yXgxTjK!? zM&!5RSFxBPZXv9+xf(h;Y1ln+JfWC0PMo7`3L&y(@y$#*Ap=%JdNdMeahS5l{Bq1E z+Czu=`^gs)NH~xeS}EA$#jUOSGtXbi#d%a}i8CUX%IBN!hZYuC(Phy=LzBqlpGHpV z4>`O_($y!M;m#t(0rK@8h;WCSj|5Ed4_pHPNxv-|rF*C^yMQHgS8)C?w3j0dI>J86 zpBF$;KUdnEC$1US#4MZZij|?5&Yd%R{SfT}p7XqF2~nm!e}^4j;NdF+F59*eI-hv6 zGrw&m*Xf4Lm%do$S{!PoapGOc*PwTN7AJnWbbnD_F1IMOaIpsc-s{qO`pikAf7@A3 z;5Yz>rCOI^rdiP5ikU|ru(BCs<*SbwXwm;dQQ(a!D=bz(6;H6r3%|v)BZ*W+WE@-- zd2sy;t}k5iz6l@Zs%Mz}G^)1pXOicwDPzCB1+(opuWjzn)zh^#TYD5#>S|CtH88!s z>F<2L8uI{!q{;byId3BQfBsLbB7n>^=TuoFG;k7Qb!ct`!$AA)Y)u&e^=>Nwxzv3W zqm=5A^irq*ko=K~J2MFf05|r=A9+4m2hX}ng1t7Bv*qQHcEK$d}>GnSiFIImy{iOjTQit4LpaWoPQGH z8VIg{{GT`wK+YSivWd96>y!Wl(@l4ZN(I@+0~L(D z!y*rWn*I?}Kk~!UI6sC(Mq&YRl6D+(07{Oa6kcY?D+up>oa87{>ULlSQ}dt z2_g#$xyGmXjt!0+bccIdDOd$2Mg;O)j2)1n-_9qD8~(XXa)>Ijd5$ih^Pcp1g4XW- zd_%k+w8YWXuo7uFS(7u%!lbfy#QR}ihRnd#-vWt8|Ko>9a7`|Uy=8h$6;8WTeCSHJ z9Y(F7pgVcp-q9abG5lb~&{t?X}~c?Jk04r`1> zYjp1sG6nXG5AmhWO|D=@b~B}b=+cqz8Ub}>45t5Hu&%{I#hHO>OaH0O#FQAb{m&HO zi|wNyjL~L<$M}zy`nBM+%UG^gSoB#Ksbd&GfMJw@43E7{A9MlHeK5-8cWEXxe{7AE zuEL}b-Ey`jG*=_a0I`Al4k(~m%Qcu4`TOMIs9}P5PbmMj^@o5;1h=wH!xQ!Ch{*;| z{~s>Efin;LJ`c~>HniTA`vcTl5pFsAVT(xsBu&q7M^FD_vC`z%2hPL4dv+;)Aop{o z>&BHIuKk*Su>Swo{s34@NxoEgcF2)saon1b_f}Br@v+TO?g@!Q)W=cxym85?J0) zy_))XfyeLP%LBokSP5O>ad3aE)f7G;RyvsVvHvmHM?nUVC(yeIEa%81{V4gLyZEyR zs|(*nlP8^Yd^-KGw)FlB1)eA+${I&9lMXf&$vTg(i*tRf`kp&0P`Z z*A3q5yl?*wBVQ$bJE5HT^FX2V@5cL$;3x?#ms-7K?><;ss&a5pK*|Tim_r;}z}>GJ zyt91nuSxZLNO>RP-n_JMvNSa|*TDw(&ldpS^pywpm^qeZ7VyU|pb)jucLPU%SV=g0 z?loF#E(!%Ik_xxecasqjGIgA>7G+ka{Mec(0H&~7XjIc{W7k`9sC?f#HK%*C!Sf&t zRx%)u{a?XW5<7;q_rcs|m_>;t%Ab?Y_p`!LuFx8^{U|>~B|-o8;E+IzD29urv;ylt z`|F2+TX{l-@$Ji(mX=?Y@mj??E&VuIl@ikvstu%Qx67^HPxD*5M-{H;T;W7*Mk>fw9N*Enyy5@cS;L{{V zq%~G_bb(-`hsv3ULmNHuL3?Fb*+)*1PwIiU|GuiyeFL~Zgzi;)78aqg#0pa;33n3& zwe>wQfD&RO2%u;W8Gd;l&|wPW*f=q3Tv|g*e%+?|a5Xu>gMI&%4gb|Ggp`uJA(FaV z?m)56K2ew-qRAT84#fMu#_`+lc4v26NR2+8pcWy)h}L99!!tufzb?VRU@0=S_|b_JCh)crc0YWoe;;p9S}x#v-o_9?AeD+-JHIxIdt*$8n>0CXgW zq-e{n>neUHf&pR$28!G>YlNpmGybNS(@+M;{iUGt~|T>^SHQ z0Uq^4!KG z1TscWOcC6>paPzHEh$iqS65)H2@KiHxE5wz2Y$ch>{Qo5l8Z&V=2X-~ zcQU?HC4=){MM7)9vb)6sUvyNq66aq|P_-3cy2n$uwK<)Zi2N@%go_Bb5oT0i{3noU zTEfM(6&4lA1oARFFUd_BzzLHgpJyq4_U|#0SnkzMu-1q8ajI3Q;}y`l(_jdLSeuZe z*4GPN?Kz7xwS(h4J{q#Hs8S?k)={ADH~21UC>igWbLB4Dp3KZEzJY7_D)`;;g*iR| z@!)W>`=D^}?@k}zHZWiq+NCi#s&0`EG3h6HEt;TM?hl-nQPf!RddpZvku=ufaQY&-G^~h9wrLBFPxnJdCM1DJC1`U!&925Z)60A6BWNM z+W7C<(skyrQm%@0!x>RyM>sbNRLH&In}ha!SAJU#-1Rn)EHxUxA?-hg;N*}rENPf= zW!%Lag1glF-7>Xbm-#+TqAmg*-6@GvL6XJcvpg;S`l z5uKZatx5Czm(|7^_9*ikBS92(HS+K5u%ofo0oZIID~TaX!&er4E5~0O{iz+}Gj`)~ z_6$mCFXUn?6sUc_axgbJ#@|kOOV`VsmpR!jBu$L((2-%lYB7 zMX}0+rXT;Ql{aThlh6hcCOd47CWsdvGrA`>L=nZXy6F9~GN|j9iFwml_%cL=Fw(8i zx={GX2{CRWKE0i6H+B6`_+k`n%CMyBR8j}$UT@-40YWmTE-h8xw-&_8dImwttPwe_ zDpy`2xQV;#w^TU52elhm7 z8DD}1oyPZ?7LIU>NRk6z?7akAnB3Dc3!UDV6I?-dP*9jG%EWcJx5{v3mnBegi+4Pf z+*RrAR;^8MdC<_KAEzV>BvT!9DlA*iE}SiX+b3e12v`@vn9-83E#M?k64{SK%HLn3 zcZr)0LKv{38J7Fhjr~ijV4`+J&z|jfAFUfjTt;>X9r%B?flJ)AP_Au7co+PHS8V6N z0Qngv*A^xYVzoG2r6HjT-VXiZ8?GeeWCFxRo+yEGFHrPcxH9((y3P-rc}`9^V2>Ta zlG(A`i`tCCUb&{lpoRBhyC1bl-402E5PLC?5DhC+#=n-JrwMwoXsb`IOm{rxJ~_}* zI}D%8@?A(3nslSb?A`9tINFF_5?dXV^}LabrX&tsIcoZR)KRXOUG9M-SWbrE4|Wbl zQ()LR4jmQnt-Y-2JVg6B*2>Q9aQ5}l5~9QFuOIsMSemp_8g(VHpcCtJEvw*0$2}MNaxK|`My46bp#kJ$ul-ev zNV?ECWR)!QjbFlyv8a~=o@XY^EXyV37Dn!u77dy`;J{bjj-&oT*SNo#>Xu37IQc9Y zf_rcxfyVFEE_0^BtP@g_Eus#OHeSE73-6L?vtfU#U?H zdeMuGv`hVF($wAq@0>feLDuOlE0MCY9+DgcuMUMSX!NpcQV?aS5b+nC`%jYNoSib> ziFrN5H{M|@$*zSaG2;V2Ay@3w6F*>AA!V?>X9L5%LQba>LkravdURtm&b| zvgmm%pv%7ywTk#cwLf=l_4_t9QefAMh@$}(Z_>!p$>RHMDuUxvvVAEZC}z;Dl**iH zXyb-#jowy=)U9GpSrJN-Cl2K@y*qEP?Yb@jirmR~MF~QSo*4Jhhzw1M)%CEK>s7o< zmx{RdD^Xani7UJV`Kwx)mWFR3eu& z+3*za3|bR%x%{0#7zH2$F5UU{{`h|q%0pwP89Rnl5-5JBf!0bh@T{xIo;)^CA3UK2 zS=Uw~n>f5mqK7PqDSiedCnAVl3F3#x2?0^Qky(t_D=VOLOkUf^{;t_t1*bq}Bk@a& ziT=Y2HGR3VOqB>-u(08<3rQC)JwuZ2QHQ#0T=ls6ud%6b|>=a;3*16T0(n3S>f zVM8btmTDP;)su5<5y#`9MU~PACAVO`U=K2_JWyHC+sUuH9g~{1m3Xnrs5!z&J^6|p zOyxnq!y#Qt6avL%55e_h`*RrwhbZI6(Eeofz zy=*ipx6SsnqCu5}SG6t;2*8(Qv$sZSk!n(@Y()VQM@)z$mV$U}jH}qSkl?jvx7A(zUE8o zN+^g_kuRy-s#EG#DetobrdgD}sG1j%c>0H?F~neYvfV?iAbu0@O=EDPFs)G%_C|^) zmke@s$$NS!FAJK+*_%W}&DMo5JPQ)LbzO|I^uxda#g970<8wR#nROxw(1e`#5=_)< zFXLrt%cJF2* zxyi1PJtmbzShC*C@m&#j#v~E6c_&q20~tgn?1dWa_1SUL3?38D8)c3ywBuL?lJQQ0 zRc~XZQBCr|cqb{In^GBy(Y)q4w2`v`=1H&#*vz+43LADZN&pQ zr|Q*%>ZBxJNG8q3a#_zjpCEL3zin)9J^XWrkJooLIV9e+xa09p8DI3TUz zk1X1cs0BPC!krbF0Mmw39jmh&ULFiIQ~Pg@qidsACX|#YS=ZD=lg2GYw+HPf=5-J) za-KAHf6~Q`mK$Not|%gu_(nr$Mb_*3`KfcbK23DiS*t8LR;b9%VhR1SEU_H5YS|Z8 z(|0+R1+Ug>EGQ=?iGMh!@U1WiH@yj0W*i$G2Nk(BRfLhHYF|C;o|;AJspQ015!@QqYK@I)=LIxAH+gAk z0x;N5NnVnu2xm3T}u(t##Coa_9qxnji0I4EI@1f%h^|a^slPF>mLDY9DV9KEP z6x#K~g&}4-b(RoMyUggZ0iAO$gFgQQm|niV1iC~d{mAH=Y)>^Bl;WqDwzq)&qBup! zX&S-Nl;&$*w#`T0Q!R|`o`p!P>@bJtN`}^JP-9BI5Ka2jtpOk`?|fcxhCar*GM%*V zI>+QwVWT1(26e8LiC}_dHIopN^WLN)X6lY#TQN`dqsi9QdRS8fbW z{%5o-8T+sG_3Gdlh_$~MageC|tO0!PS$dDzxOXe!tF$L&ImjI4C1O9qh!e{#8wIxS zwROl+YhHcNc)kxb@uZdDxNa^AK4;YYhv5Ojg}v@%mS|R@cKZeN)P<;s6v%(+O<3&*lB?7w)f@+;{ic zQxb!tFA=bc!MA(ALL^tXT~1IC+=G;tUKe$946M^!Hv%v)ub>i6W5j7C(55Za-$Ti` zbG^rK6zQVnowE;`)t*+uGwgb)$fxUE>EdJmQhZRvF^W zqU-*&7les1F^yz8^OJI=msXw>xcLnvudmtm%MTlUYW|EGZ{+jN0UUv!H@Hh<6hg^C zgc@YAsH}1NVLK8b@v|KEOj?Ay_~CoM#VlVBLy%E2LD!*yWVQ`y51EDJ_zkh9tflY$ z$FDRxJBffn&6IzAKt89DudgecC>Ywp_y4XmJpxoq$b2=+p9jL2DoNKO-n#hPU`zsW6N2<@l7E?;ZY z3U_r5G(n2I#oWvA-JFN%v~K%e(0wZNsn8;BwpN{*Y^v{7xtLKUeDFn(^m=OfUhMdW z?Y#CpsU{P1lOTHWA#ALw3SIJa4GOMQvUgtiRhdZ7Shu=$Yc}rOszLL8^i9LWPNlmT z&Vi5PvkxC%)E-P!_q>x_Jt0q)f!UF}LUJV@{ZuQ}?U4uUBd z7Atv!U1?0WrH_M;+EI?bO&lp+xnF+hIG?}u!@`Qyb+_t`ss|3DSPB>%we-ETWYIOd!6ZIu=i-ivnM3s1Bn&$&mFl?pp!p>u zgenSYMV1>!shFgCa)^l%C*(;U61p3!-Fw1&hOVRfXiAP1iEc7g`{Zf7EPJxFD)$d! zqG-^CCiy_>Fc&pe%-6=i4?tshK+v3nCDw=H>_RZrRrE^q`A|+j&&HsoEkxYDD(z^z zP44(;<45!RBP*@w3|*!-CWiM4^IGbOAxi~qXmI?BVJj6G1C%STtbw={nXDPdd$M(! zoTu$%KS}18tqAJM809MzAkFRf{lvIrb~<;5R&AhAL5nOk1)e33{0A^^hSZP)<1VHG zh~yvCEWCj|*8F9|0<r%(&Gl`LVY2S^whh~kxxQYdE$Xo*Dt2oj zUQ0)|{Rf$h7&_maYWRNkjT}z}niIJ_*%D;8t}US{#KUaF+RMqDoqO8pk3+2Ph47i- z{zxA1M>S}aya!U*4yY2B9G%KZ(P5E^V_8t8Ahi$A!WQx5$iX4xtlT*#fo?459Y1*d zLg;+WS>d{I#)!^XwG~*Qcpcgrh5B@yb_)gL3BhsppHZmf3!4Yd`mTgj-i zM5l&D7{rYZnYd%wur;w82o#Y0(3jIN4tDk83Ae;lkqfIyvkX=@$n;u-Vb0p_7G9z! z)-E1zZkDW#)uoHCV|Vg0#WtCJ;6Z|c+?(IWKZf$CVF#IeXzs?Hx_xtWr`=%14rY+) z21oX(wH7^JmP1Uje22@K*zrRMRs%{%d!rX})p>L#pY5tf^g>-#F-UY{n{7(aF91xh zrx#jvOB`#U1G+5IkMNp+wYj9f>#_z3c)K?B8X9OpR+@HxmN5GvnITk*EFiyJLGPSn zLpj{`2Jz}1VMJ1KFH12J`wPOZtrHp2*R-n}z}Q4>K*xJ{qEy?!MMu-!W=H@ zL+v;#tVUEf_Bo2xGkH*ghNi|GOcYsjb|*J-IS}UNSSn!M3Q0Ptc8qY2~R9(r-5!_k!|~;ZqV{2 zv&o93kf@2itA`|}-8JTcbCP_S;e!&Pdx3Q{5bP}OD zZ1uU?qd|E>3@acS15;HR;4U@HGFKll3^?e>jg!}Q;?ng7c)G=}Wb;iwpi}v?VYsQ| z+~o;I=o3pG=2+wTnK4v9&{@Y8!;( zW$8s~QlmRS+)hTq-Jdzq!pl1a7J7bow#UH0c5(kU@sMHVX@^q-**(Ngo^;=@!&JNw zS{k<^)oGOYbuk-CHTog2ayZHl56{-RP(s9l=uACtknNR;euXOSi^$z>W45n)V8`0? z4ognsbAfNJD^D@L9z99VfotSpodVKD|P z?c$>;*zIs8`e#{NflZ@mW>z`#2-vMgES^;Q-RsOTz5G|fErl(gVQJEmI;LJDjE3r8 zF{5?yZTSfnh-EII$fd$xqxIO9l|u4+ksXSjAy_D{_C5KB3jjWg!`anlzPbKbR%0Or zJgB~;L~6B)czv0sp`F^zyDmXqmu#jpbM>L2%MT03nOE7-hBrx z?gYN^PggQ`!JXc%L?d0WK=e2RRbf*b?8pse0IeS(KQua2kV`gAPNgwd6xq-uIr!%e zshD9q{ODNsCrLax;qo6{=InXdgQ1;`lxrH(Mx`%=y&K@`ZYbr`$^6duU{J?I4BiE{ z)RIo!h-!vej*+1GQXo#0eIknMzI2oHccPT?bYXrt4xe1S zksDEN7Ic#%c_G~es(yChu{y(2Dqd{qku>y$OunVl1vg{O^=sA<8!MjNP}O4Uiy%*& zlf8FN$NfP)9p0;d6TkgL5On+FX=%rUSr<$=A^@Df(5LP+&g zXq6hC`H}-sml@(uqBCK?gEMz{(ptqDORcD4v|8H)9>d{LG4EKks990K>V*-X?E#aa z7tdJvN@AMCcGgT~O!M*l!3b$YQ=UK8z&r__x?lP>OUPr6cc~cOLHr-|iTa>V|ChIa z#;Yv#%8 z36}!sXvdn>pZCaFH;DVzFVHnh@fmHbg9a@91ZRd`@dc$O<^lGX_+wDyF%1L>_<|KD zfE$GakpCY@h3$@TR(cCp(8GH~1jOKq^>L}AVY5teJtxkS6Vx#kn6xN{QvEkqvR}5d z+5VK;k9s=WvhAD)z2QQ)Jm{XTNLg+xcAhu!I?CFwQCm9NRm0x%z6WS%1vp~=89^UJ zw4aemdU|<{u%D1emkdE%%zR$DzCxbIhun8ic;0#}_5Hq{H&`V&-1DCkfE|GEjq`p0 z9ZTtBl#i{}jhs~=7|Vd`MKPtT311 z4mC1je(JiUSsH}>KxN>fsQ`%0MXP)+2lAGS%{=3ai)9zboNkQk=j6~FXC_6xPxB8Z z36Y>CKlF#e24=E4-TFAi4330KvcY}~@6*7j5`zSQg`BswZXnjbg|kmR}* zI{NKu`rhYGW@W)Y_@!HjmIa?cE53}ODOai(`L#y5UX$}*q%pJSKtt?Jy*PB|h|>*V zv$*UefA47pKQQ!=RmRGs_IVuCNtpf+u>PhzKmuv9ab#?anhbZTV|YN~+?W(b4-Bye zhtJpz4`(oHO9SCeN6gLN*WqA}TLFxYotuQp!P+Lb(~TnNjG%WEFzhkhM()QDqwKUe zObdS-3ouxQJZjGbbYd@^u3_bN29~6f&dA9R4br3Yv%*!OQpLMgtx`3rzeX;(r}Zx| zxgGDQ@7*?C?Cv}Bi4Xyz^pjEVUte6;ARcD?BN=j1zF-5nzD$A7$gpldfGNRi@%cE6 zv4?XN{lcjE55|73x zjne`Gk+`?-0d6BBlg|ZkN~yvXbb?>ba3K!2EhEg|xF)9Q;n9SR78e+BR+FM03!KD> zE7v;)c^ous7L@I#j{XFAv;eom8}8(OBn}MXz!0gF^mfEOmh(0=)#3#hpbt9*3U@{@ zz^Y>cTU;a*fSmWHb41B&3|hWV$bXc8tCtXN#_#cng`Qg(nY{hW1|d%#D9%C+Z7mi;bzuhIsQ819=V6skKN~W z9o0mW<43^`qo!OW2L=dy02|H{JVo~Thkr-zbGi0QU+#U3oQcxsN=1T}H7TiYKcqdP zW$8Q|dhh|BY3lo^ctBpA1m%!)YI2vQcu@-S_cu7gl*Ce|{Jx>#Vq4zV+>LFQSU^`M zywXIE&~1|vc=i%HG>18$&$HP2`^>z&1Dt*7sk_ptOs*Sn3#W?0r|#=ZZEjL&_Wtal z`u0`BXg%c#Y|PRUWzkPM1|@-f_B{sR>E`zQcJGsb5CzIjj?%S+hn zS`BP(;E|B5HCO;yL#&%b(y%>J7|P9`jDWS{OvSn*GEE{*i61;t7(8j@-(e)efhjqG z6Hsr43sGU|Y_Xn(LURqBco^58u%?{CEKn-FXlk;8vu8&vgUt2z;{2T`tmE5D#aT*u z!VHL!1^;gFFyaCS<(P4`>e8G+YyOGUZ}5&8lEc@wP|)3$dB=kxT%)w{K@hQ0y#!p3EMGvpfTr3fQYqcN5#zniGfDDQlrN& z!ek5}Ik>@=I0f1=e(^1>B%G3^Lsh3*S*G8Mq_NG|>*x;E2Lb#XQb(!{dhfq5SE zbDL1obU0c#`02&@Bz;30Y!Wt~7}<}DFqpM)RNr6G8(nD&l;V6ll^Ze9;s!#*)4n7oK=q$60NdMUC#W%nMeZ*To@EJr zI;}AK>KTSOW+OMs=rzSvpMZ4tD(em^>zYpNk|tc=M_Jsm^1%Fg*cKWL_dHnMj05*& zS;Q>`gcP}x*v&$b))SO3-CG{X%NAr1pJD&(J`9&WOt}3#dbdKND7^LlA+CI;WL-}A zR(XKB_{`qr&Egh?Mmb#6UktL1km^cl0xGSO>lJ=ts|<_7q7%r5QbTq0)D6#D?p z276{x1Gin*FtD}}CJ&L1)w_g*BALJ_S!w<0>3O5FqXE3^p@s6mp@h82$!){P&$~=I z7q9 zmyw2@5sj9G;tM(BX*_K{h7NY_uij{K*QHLJG$9K4%?Us}4rhG_PpTxi1d>8b#@J!v zI|5$oZ0BBP%^UG&wpYe~X65Gh`xea~;`xR1`Y6{jIwETM=oqIWg=zZisE;-S3qOBz z?;rp;kvYmD<&r2|7VJoKCajA{?I@hN2&1EbB}?``FuxAT%{l&LP*&zlwlx=MC0f#Q zCnYm1Gq&a9-miKQ^38>h1&}R{UrJbfpP z&QVg?8lN+$7rdF5!BL@7$mJBH6~gY@3&(&y(y!)Zv5vFw`L#jZh* z8Q2FEy~(O0ryB>k`Nuu}V#5`$ZYx`N9`b}H@zGh3L0?dB6MpmW?PtxW2Z&eDg(b1y z?c^(!-1l>Bo+L_MJJ5<>UkvndVTZ|(Ry1xGyM%ttx%|WcWka_s?6nxYJ$9kChrQ>x z?FCBHR;?TLV{@S1om45|6G?#k{hZEc8_i7=$qf!uh#Ba4wMzGy&YAlzCIlbXv4NVs zaXFH*KQ(}Qc>-u;b&_7Eo$R4-CUz?Cw|5WdUwQOmf=`WBbcISdCHl(&iYKGK!QIO& z<}dWg&|c$j9V|;iQS}(4VtQ&sCrt0s`Ipdb$T=)7^?fjX&RJM~HvB$Qn@~$K-3v|! ze(13IbcF9ide*N!vt>tKvuErWG3}x_z$uVG+TjcKR2Zlf?damX<>AhDPn*}}Ew}9! zl3xe8I4FMQ^fqzpq;p89tJ>fejsM~}7+?G-_I@Nfy=Qk*zaugBX-H3AJUknWnTO)a z*0$Jm^Rk`icFvNpg5CZ+f*XlhMvE=-lE*R~_t7|(I2D%scNl(s7Vona2G_&(;H&SO zS)Ff`1ieX03-mb&QI;3r$(9YAfAa+)1Y?HxO%9wKramP&_7V6E{c5)AerKc87Bp2b z-z=lE!(mn~vuxq~ia(A9)NQ3df9UYMn_>^yomoFGAyj!KA3dP-D-z!v+Jjc5y zWKa*h0DG3T-={psO6$&`+>(P(_fqNuMf3nPSTUGC4H+REOKn8Wb|TsED)EPHEr##` zEzv2!3@xrQz7zVr8qDp` zptGMgfPa1N;5r;xcL1xJz-{!3$gDW&;+9$WtFK94R8D^W+Z}kPSrP^K|JA$X;m8pU zy~m?X_s?#ZgkZ9k5RrG&^t<19+1eugJ`}pc@ji+tIKCaV?&&*dA#rm%r6SQKY6{_E zX%2cSW`-FRK)xgw7hXM|(>%I(ccj+r^NtMMcKgHirqxVYMKe{`O23Y;(c>sDS-)%U z_M+?Xa`ehgf`7Tu>GQ9GumBF~wX@Fm5idm-Q|lSsnQCyI-ZJTLT#7*5zcCM1nEQlbY63 z<<1#&&{&MHar3j;S$V;JD{M`1;B-<=NX3j=FK|4(|LgO#_zwLxG9mY4v#V>C#m=KG zLUG#$wb;w!kpum^&9*y|`4;bIbw2wHnoS?p(;Z^b-4KtR=hC?6y)av|%`Qd}dD|^H zJ8Zsp?GyEdAKoh?8;Yb?3qh*9O$uTf+<1O&SV zWi+|+)Ut4LRf=B8@hx3Pll>eU$0_#NZ|buAJYZKo8eMBU!QM&I=GE_lp(IdTPHBHh<$58vsK3vwP`*lINvQJ#>zT5L+TX^p*3cu zJ{z24u#5k+xD(wl7cFPTXSZz1*Q*D{#pkQA7h_K<{oq^LUJRl>r}eBP_|-yHXjKyM zEjF5Av(fI@kxvwY`7sG<{0shzvZ`_4Rt!mtp1-W8g?W*r-FsN^_V2Qc#9iwSdW6rM z^O~y&Ya{Vkb>xpc;J{TucfJ1whoDVg-4N2_VHD@ zRCBY(ob?20FH;B>_J1ASikfR_g?|sDES;-yAGxTloTylDOkx;WJzxEOzhPawe}VAs z{6ejZv(VO{*Y`JH``OzMUQ=ZM$}mCCQ#G$tEPF7G z1ES>7Z$jSdo36)`Stf?&+%yoH)DX|8*jLo!Hx1zgc9%it(>EjrorCM81C!Q`0$BDY zqlxvl*$zUqaAifTt!Wvi48u~~gb)vN(HVZ$*+@d7(Yu}Zq=@gZGN|>yCTK&Jx`wFt+)*q>dndPONeo5+7C5<#mgH+cE3i`+~Oi6 z_CEp5*<3Sv*8N}}=J5;C5W~^k;G=sMrJwf-7OWgUJM-C}9o@po4zs6y z)AVv-XstMBqj7CH{~lbt&mwTkEqLE8utPxo?V>Uk`EDf;!)|#PkNIv7v9bq5noT@V??Y{uw4Ufw5XmYa9tU;2-k7A2L|3 z7e*~TsTf|OpAS$eL)Ry>u0h#y(w!d9fD)>BO07#vC!`ExE@4x%9nbfR2kK&S3!x5Q z#7@)hatVVUTTj!=FozjaS8~;RUvX+$pB$U7(`yQwCo_t7w?eH^4xgD#CWGG1|K8HZ z)xgcI_re5tvZ4PURc{&82HS*<7AwKsB@o=*S_tm$6nCe%6oN}}E$-Ul#T|;fTe0Hq z?(pS#&-u>GnaK}+Bs0t{yL;~?D}J66Ja5^b)b+-WjC=|}=yi&TF`GO_H1S-nv3G!D zERW*&d*0Kso$96)dkC9{KjJ^BwnfP<4;eT|&k@Bc&@7R9losL<4_Y+c5B@)}r zvZ-~U5N-e87(daCLOT9LBAzV=%(&3?`5ce5utJ*msl@`>J0DKAe-r2!v;5AhwgSAa z^#UHm^-)2vg^I>vHT0Pul-X(elTVv8>}^-t6dQSkRYi-XPK!%rJMl_BM8Eva<=O5(QaKI1n=8=DtkBZWryZu?HvYuUrKk;N5`9A|DbCgeB=51 z+4scendalFFF-noZ>LhOpS(~2L#j^I_v_*S#bmb==Kkl2jq|WKd`~tXl1ae}nu;nt zF0-VO6mjW`*WC8#>r~cqU`0ctrJxCnM{FD2#ogQeMUu9m`1+rX_Q(ZY}oUJsI@UV2@V`s zAWjZ{uR_$y?`$Auds)epi`H#yg(*L?)#3~NQ6pFTrYv~-f!FuQ4-cPVYxY=pIm*-U zb4cfe$?CH+8I&{F_n36bfAi0#qO%R5-$IE{@msuKb=Gm9RnK>{d$! z;efWS62cY#nGETw#Dw|0+oSt3$UhQ{CG^eR;0y-J6+cz4JCOfAhOc(g_ za{70Nxv4zl9ES>LF`IUpl028>JZCj{_LS~2S*-Ck;Nn7Vg}f8Q-;ig7uOJqy;fx4Pmw34_PMR?@>p3owA%ObI!b7@{yeTnds!I^ zjXLlcX$aV4O%WL|8W+)tq!V>TcRu36Q`!ce{6LNE6k&6PR8+-Ze^c75Arc4HOr==uV4x4rj|WcXGkhoHXlE{r%zv9 z@yI7}An=*12HNrRQmtEdA6z_WY==S<9k1rbRXRE9U2LB7x8xCTw#~%R1@6txxN31t zgY=PVn#aa7KQFWiQXB1WFx1j&Q0@z(t-SL|@HLdDhEvYV%XC@(vjgax9sFBILC1HI zGo)8lpMMR>u9FA@+(-_kWORRluzcGS!l>+4Ifv4ppPUcY>Y!XoMm+B?hN>Z(ZyYzC zarE5^xcz5LuY_czgJriY*1B47g&eQ%VW6rsW@wS1XqNU}rHr!+V{`PuA=}DV7v%DPnGshdbzdy+^ zZ5c_;6WT`_@u<-Q)8pgf8bhVR4|6}kp^1r!Bt67WPfr(hxv!|$-iIzN8fv~odKMNm ze$P$*&mVp2J0G%0;Oo2OJ)ihk`K+CPv`9h4|815pf25_S_}HEFVC4;SV4x3p*|~X* z?U<&21I3LVw?bQ5=5zN6I;cQ0fk?#jHfbcIY#g!!!nPohznYrGJUG&H-Lm;$x z?i{bwmm(yYgx%>AIL$vF4d;><6g7#+$@=@5v?)AmarM+$w4x9&gq(7>E+^jZ=g7z$ z_g{AWmOr;$ymbd3xyoL>={fK%EN;#!3&tiSsJ6c>9|NS-o*@i^)J0cQ31+yRmH}CB z>qD088ErC9tA(W>y)thHVYFgS_stw`Uq2M?hi$q(PSsivU7dV9xV{Oe4D=EVv!)0e zbZl_0?h_x$yWUx}>Kh2_cU>qnv}9&__?ab$=u1unbEO zVLs{K=ucH`c(P;HMYgK&c(^c0_OXLN^pW(T8m6#$TS;wgf`5L`4nyrxA`Otot5Wv< z>-|YHb90H^jkHo#0ov1u0NUc#R$>+w7D5sdb+5Qq*`l1AM?U{+z9X&G zD+GD$4S6F(bW#y3D>}dPdq0fb-Cgy3)b+;+Pp6Z=91Z8AMsJ+B37;&OE8}{m$_n-~Z0aKdh{*EIq8> zS3nGEW8q?6ATTEIpXC@%W4+*^$5{!pfXsQmtaN!WIh!npJU{^Pij zOEWcI4MPrO|GvnK%V8ug*t1eEc>U#e(rmjGI>Br@DR(t!aot<`G9Ghsg~c@ZLbmk4 zjh>aUFDdtL5S#xS11@wJ!+rXH-Nf_v9U9VygWpda7g`R(_mYp(@w-w`!1}9skI)wh z5f?(6f6U9(y6KIp7O|g_djkPW{UKYX?f~LLzq$30~L3Vv61Mpl7wH79zOaK z3a-cf`GxVRLJ~wMA;|IgU8S!kZD*}@58i_e|6AQi9kEG$8m%}D6_SjOMOl~EFH)n# zefiXz!f8}TEs$44TfLW@tWN&0ef#ntk7Er~Ey13t$XMFG#K|BNF=SCo=-6I-b$kA8 zB)Htd9S*8}n9C+?TX`~WB%EKP3J#?BkF6bfgni?IDym;Vx86*5-see%`N6VI^_yda z4q;5cvFBc)%{Me9h#QuWgSZn~+x#P_Ao&MgerJpBt#C)Sn|Ea;22@rXJ63FcY{$B9 z*(>0%2FGab+&Kt>(gR3Dt$pFw48jM+mPe0sBvGMUb$`_A#i*`_s4fu)hf3^{qM3mh zl1(T6i)Fj&&FA%Z@Y@B(yVD4X+=55hv7Vj0M2Y%OB?jeMx?G4_15J>I3l`SV+w=LL z75JJ67Y~miP~QG!wChFQ&C|2IO-$}HeiTA=*ITP?b!jOQwDGL@&tPU@&2v45z|afk zyyHH{`z?F@k<48|u%k-9r9guJ@&_)u6RVa5Pb9HPK;*_y5K5 zcaUKoQ9CSmT_CSaMtuYBb_tFR3cgD8tjv9l>rgMI^%r_W$SqoXhodmHwB`s647?x= zjaqv3zZ-^KSF2L`b($U4xCCQDu;)gY?#DuytLD4$%`is3HOgGQg-YEBXu*Nf?78iI z>nrihtgfdSVb%47_g@F{V$05PE8F*3Gx0r>m;ZY(0{Uli-v>&_?zM7$Rio}}JnJ>J z@ULpU2H16Iz8Wa4+pl>>WZT$DcXd4o{0V^K67A@<838-&BPlbFek80N&gjtYM>q@u zkJ21?F1)PGb`y9llA89C#Fq2rV-(DITtg4nNtmZGY#@g)Y689K)vi|U*V0wv8fwiW zDLJ|J)>b_~i3sT(f^3_SnV6r7rwP||brFiEttP#jZuHJWrW|IYR%iVgKB?tK5%rN* z(njCh4&4}0)goY$(3G`W=G(H+--Uh7VMv#pvt#^{T*-H`@c(|WN2liFWKP{l#AQ(X z0#J)WG&qM)*4v3 zWLU-+SjI}^O{b7jH54vwF#+q(Gs5WK*ZN0(F8s=$El+RMac)7#Ui6;&*73a(EDo>Z zsz+8)EG1LBI_AtXFc|0mViw=1o&Rw!4Sa2qfTb|^E>|HHP6{@yon7Vf50fxNdQDZp zu#wURaOXgui&0Pm#ME@*PuWUR6iBc+m~2Tb#WXMUex?-@v>gX8DJ*&ybLz?*Gx9R2 zGF4Ac-YRcJHX?oa<4QTg{^WjL{`j)AMXylnks8_Pv$LMUVA&+(SI3hb@mh}W_>hhlM6~}Vf`OIvu#ZUcG%PO-I4HFad#4k z(-KG4YW^J-c(wf^G{tx*XCQXXB!+W8D+;2SksmR~++0;_bm!507^$1w_?!QnuR%Yv zO_@}XUxn}g^1R+%IEcP&^nPuT)dit(Rjfilmgrm9Q&fD|)6N4KSyK1#5rsOT~~2Mn=PE$h!;l zCM=JMFEzhV{Fa9D?TNGmLnc8Yx=h?%CH0uy#pHupC@#m7O7c`$tg{5Hs04)aE)a5F z=k>Mta8o`v@y3j`XU5uLyd%mN0gn4w%rcx~x;=w>yt3ifFNWTOKh$51&iflX#((`{ z$&sDeN6Tw-{ovj4@d%rWi!!SvZ4>dwmsTwA+cNN%&JqsT?qz+e1RMWM5K*9qgXU0paiLepMN~x-8S>HzdA+k@Oj_(+!fk; zf1W!{pHUr)tIe8ZvKY%ti0b`v-sOWtuu?ri;CuHi0U+uaKfEY+YQIQXN5y(7Gg{vX za+;;)u2f>nFw8b|ah$PZV3?s^&3``cjrbnR$;W`Q#*{Nq=x}Taxhzfktt>G8ZU0QH z$=uCBew6lQa{bkTn{QzJ(EFmRfI*^`ZzW5JywNKO|&GiA3voC|L|JXDT*-vdxcEY>`5$ zQHa!G4@zSj;~qE@xRBe?(EmEnxQ5p3TS)@!a1YEDyj-Q!&`1rCspfk>#>w*DCLJk{ zyf}%9gAT{enH`y{o$z6XjTNc^HZA*HbldH-+hmG;aYwp-FIF4I0d8u+@ixqFS^DsS zy|zh9NXfHaMzAy%Q%&-$a2;Q-a3F8aJw?Yg`xu2bc4(uqV`ReuZ}YAO*23!%GThM) zZ5hT4Je=xd-$sAX5M0?n%#n(&@q^)0h~BJn0*lnkonJPfb({*~%6ykp-+2~aN3uR9 zpeTxfZEN#8&`^ELuG57!hvmL1F$M9Fmh$^eX;CU>l#GRUV?3PCS(lhds3VtXz%%;@ zd~xz1rAs9KPZ`=SPMV5D#j_;E1C)9Hoe@2%Yuty2A=2{IScV`mRx4tI^o3w^f|-zl z;h)7JI^4)zpND88EJljgJ{y5|Ms-S)sWuX*Z~-;-#V!mK8RZUOBjVwGSz4;DKR2V? z@u*^*Q}FLS>HqkQ1WP4G^o6KLg#w*EG=({CZ0iBVKq*_(jf2xVGku8^OA*{xzsUAc z##9HQ{|FdW^ayr+5~^XZ*>0GC`R`;FdRz z=&uaHc>LD+SN#c=tHBf;`5{6E)kYuI8FCgCLB0{^e+Zyz0YkP0l{%csft}8Q0!efz z)iT^cwTw)6WH#as>NGnp4A(x?C8?z4`zO&1LJ8r&(>)A%9k5DNhE*5UkhYi5|Ax`2 zuB$;suGlFU#d&I@HOHOf7cIUI8x7Ex_Y*BE#D&bX27X#5q(eQT$meY(&UAg#flHVA zhmlPvo;yo%^XxR-7yY1=g-;$ziTz5a2y@U=0dOB_WJ%w8jjI-nM7(zlRU$P}HC~%K$y4m8#TwD3I{0n)vpld$&64zJVa(9NeA$++cqiui_ycR*A+YUj3Nfd%E zX{aA{bwaK%a@LJ5Dvw__)o*fuIqmb$6lzk`U_Z6TdITBU5DU_z>S`HwbL!mHuG zrC;#9?Q~GF0yR?}w zWj192-9-kCU4c9Gi{8Sy1B1F|4OurdrbutO!jGSB-r}6^vvVo%(X2GP^7dsL;VXGL zEa7Jd-FC6H$_tu@>k1FIu3SlBU|>LM&2elnF}F*!Nk%)>Q5TaTFIz>6PxZMc&Ih6ed6L>OmFMD-ByC2@aHXQb| z>OdQ9njKo69sngH?O-IfRAaHL7BFrA^D<v9fMiTU~IES+cb0w&8 zdmDvfD5Rh2Kha>221;X`1heG9-|@+M*z%>Q#u}mnO4*khMx@!eO_xOvzts+X6RNbwCy3Y}_bfP1 zY6Cor(c3ZvF8?xhILW;!xDRGUg)DqmjBCo<>C>_%kuw(dB{(C8x-5aJ!>|ZlL37&y zn<6aHxA+>8y=ml3Q(_Vfh^(7`dtqY@`4hb2qfo)| zLYu~t?6?w&W(*bmC0{bkqt?0-7O!m}_N1|{&&0{0+-w-9|6nKAgr&JRmN93TvL7h~ zzGHk0RxmpXL?jQ8Dm0l=?VZn+|x^mgG&K;^;X>#P+ zzu)oF0u-HE)GZNSN;0WiqL|SAh;ytXfjR57He04szlusSB$)+^RY4#CnTu0fkO=w# zRX1Su4bwU(6F*5enF4mm2FyY)8RfUDKE-frUHP>)G)M7(r~{=zE%;dveYd1-(_Hm~-{KY*&!nv=2L!z)I zJx=w>cpmRqwYC!0#2-+FMMvvBfSGaA>*s@t_iSrY)gWS{AztyrxWWwg$(u`VWkl7b z2u>2$pDbk88e(2{F~fXtpAJ=moJrS2Q6@nq0oGSf3HvD855!{ zy$MVYTHm#ZEsbzWjHJOVEmBE7$Y0U=I6FaXT|ORfvs?G_-NNDLaGo)Y5vF?5S1*0T zETQ?>oC(Ugxzv1TJDwjI^LpL1!37oca^RRaJeZoG%v)VeT{R}+7;(xw3hIwqzF7o2` zhM@cg)G%Kj6?_|_FO9!l+4J#RYAi}jn?7FTzCWJUpz= zpFcyHSti`Xm*b9pIF@zaN-)vK4wOAE;?(Ywv znd6}ZDzE3$rb~?X3k;~AI|)^_+b6OpdGi#Jig9!94)G9t1NqKQ_hYRo=_^DaBp~-? zhwg8u?`Nj_?9ZO;f`X{6elHKs@o{X(hchx0(#=YGKKRj$00T)ZwLD} zl`{y$9zwEXhGbg9U$9A^S@HAfaSi5|o9NJdea6$#T$%zoRX*Svr9BI`cN0KMrk>`~ zYRpYd3m}lEBSbcFnSqI0Y15gDu39K9#ZlTwl|hh$3v6j z-Pk5)wq@;uZ1GQwLmlcM8UE;6TP}3+^%z8D@_^|I`N)1x6d0IL{V*$F3Qvh;7| z7MzSU=c_2bMfqNRK?(VppdwtnC(blGV3&u&n?GFnwjF{*x|MypUe$PuzzvoONaR<$Qg-qFVkJ3CfoB;IzAN3JJcOwDH0&2Egt5NKZXDp)lWC4$X1yeV5PI zE^u+=sW1K%Z9vk~SLy6Eke$}fg?|4Fg+@>4CpHg@=n5QWwDsz!596i5jgZ4f-d2$ z$@gPZZ`~J}yb0rDAwCVB)~^Vev6^`0Z8NHT|AWp`dM1wT2K9Xt&t}3$yjZF9;OWN& zQqZ_)%RbUnspQ@be&Th~|j~ab}`m9TNEBHqB;0QE+r@^zPj%UpJ93ysh2!H2-Bk5hL8I5Qsqv zO}O-*pF|iRC5xd2>p&A45Ju>Qdla3eonN)QaqeW>pY}dKob#=Rt!!K1xX6!Nbw2Ej zbe~>sB+Xhm-##gxtkj{bKkjL(sA+Jov%I_@V?s9$`OkKfrSN~bcu2+OPi*yRkFhJ3 zxA0-ub`@C%a5c1JHY_99{nxTKoOm*lS_}N92%(8TVVEsbeX*3nKeF!~qv&0|!Ebd! zX(yQ>14U9pz_7tRslPZ-@<*WKSm-ad=rXuldY>}9R8jOJxtJ7VTu$nlNqru zRo|-`NS{iu>E&e;1uy5g@TL$|BYWKGQo;fdyZo+s+r z4vy@z1jI(+R>gWGfE(QAM$`7!J;7^x{_n6S0hUoc&3Ku{6;r7$B$et9G$AsFi*P>n z%T^=gZxIN<3Vi=C%)7I~Bp2mW3=irNOd@aGYM)4C-1qvYb+LP!d(3m@4oG`Y zwt3meDEiu7T&IUC^vW|!GUJ=l4q>-^v=ztW&3FJ&#*t3oY)WS6nA;9TRHs{Re-;S% zaBuR%H%4NIz=jCP(ql=xr-uOx2n?NFI98qYqUIe%bhgbYLfcDshTsYJ%s?1lHetkL>JJr|EKf6tdGQsgjQ5HWAy3G?x8CGt7TRBPI>pBSlV z9b?QcT!2AJ!bO}6^}AAO$IR?=8Tm}pqWPL*{6z!|)GFG9dG*$qNFnLMpa{8-5&E^% z>k^NFg4GDp#;2Gnna3u-{>DXAd178IFR&J(s?pB-V0T*M&+T7tIzo}crMpl9!+>G) z5efwIP^x$#0AL5%cjNlMKTZ>cXc)&50jGZkrQe$TiK~%`G<7Sr#As7#cHShpa~ieJ z`?^`AISy-k!4~rJgb2PSmd>@reFT+3Iu=#E|1zFcD_C+gfk~oqNpWEKroosv;m>d` z3c0d?@4Ze6iWb=JTBI3M71gB!b;{sCgB87{29|Nfha=a?tz!RYj2jH}pZ#W9pbw^( z{IJ~=XLmPSj{Z-RUdJ66X?Kg_(&#S!#ihn9muWag^6=5plPvH<->3hB39>%9Gge#T zFP(&`MjZLh|DH?dxcJ7=y512yZ1`EM8}o+?0q{Jj7y~ovEu}~NR9u}r#^LCfb zKfQ7}Z6X3fySwGg4@(jOJ+}1;2iNU&WH4ZGZAYk;0MI4&NnR3^iah?(b7)60A)f%J_l>3*lk7T zPYjs_j$bkS%MbKt)nNDr1vpT5A{>@#j62W68}NlZ7Wg^Z=;es7`L7=j)fIz4cK=-| zv!oV^5)^LE@jl_d`NXm_Th79mj7-|{{$aWC>%cS6H?r$}sf!YKB08W@R)({?-F=_+ z?dkpbbGz#i)yo9R`k#^k@9@-7Q&z2X!LEsjYC*>hv1;;4OQ}H1+K?}mkRAzUJaO%k z!4jNI)%Idk1E5>aw4L-9JA{#Y4N;>qd1f-^j(g5{J3RWY45Aeb11~NKh2CFVe<*i= zu3LAWp)Jmch6o1DjC>8fjay|XGsu5F-+C%4&k7a06yI<~kpe$Z!6Hs@l!#Vd2EAplDX6Z=6r02+LC9N@!WmgZFA&LP)! z)+>!fjJq=grUl&4MGASoKOJB*e06NUoOnjDm0+-7)JS8xkXH5sPLg|(ck#cT^ZOrj zgEoEOb*OPAr$~a-3Uz7=eKG<*>%3FM&I=(2&wL0q_e+wEkbuqB3hrq0Ft(HDf`%B> zx9!xfo{gb{GEC!;{a)#~jTIvwa_?+(xn!4EM8X#RFw=5`$s7nO2I=3Zd+-zP`_=V3 ztWAmq_K@I@tRtgl7rdtPH^LR(rVR}L{gq$zOuOrj=^xT?tq#@PXU3sfCH@rfa`9al z{`E>o22V{Vv*ILG*1%?0QZ}c$GYttnl>Dd4z@StEmyJX6f+Gt1|K$SILx_U4{UhV8 zP%jCQd|h7^`nIudvL5ASG(YzB_#H$xikR7k$ke*!qEqLct#<*TI$FI(k()jo~>4HBMavkNxR7RvjJ@5$2i$4V`@bhpRc*D<3Mu|R^( zD?#Xq!+PoRe^-9k8Eg4AOqHRs$mnieW!-(B`y}jaHmLdXlH@NfecnK%B8ZN$!zfU{ zH&)*%VajngPufQnLuWjXE6SkBVbsrs#QIy@o$tV%|8h!owaJms?{+!!q3In$RHWHZ zL4m#5+0jOnuLISw$LMzCjY)dN-)Xx`3Uof5%<}*Cp)>S-Wof@WdnU3{X0rQSlEv?{ z34(xy{q{qv3*F`A2kx)K|6ECbkdFIb#&&||hb0X}z?#BqDk!+sOFQue8cGvv)jV1~ z$S{qm`M~!p8U-j=7_zVRvsOt&CE*xG<&ReHW5WhD-rGL0KV9@h3)h}}!4}_swcnNgIgU^x0pB3X<4Ek5UulX# zZsU!M7e@T6s7$~z2IXIt8`K7_P{*X3uj@uKOYjZz?YYwP63Esy0?DA2)N}H*I_8C3 zj~Ypgme7_cO5wz&f3yYmR2P?Gt-p4x4Z{)sh3r{;gE0xHyTfKe^_WNR4EL}1sn}ya+;%5p8~vEGjl=jSLh}93`_r2zR)nMpCt=!`Z8_TRy>f(No2f&mgHGQu zPrS-7nb;iyWX=dwsTicVGcQy`-nH-AlN~{xteqd@CSzpyGyXYu%7kDW5Ruyp1{x&$ zff^Xqt!A=1-aV(43@Wskn zeC9zd{`aihd^{UtkMa-Cxvwa+WD_?_aM4q%^9y0;kYl^CF=qkG@@rZ{3IhI1O$s!F zU-LWQXyk_7Zyit&CoWDw<-fQOjXNUV@ZoMcXj_!NM=#9M6hg2?&d~W7{II0}T!Jr4 zr41DAYS$pdGf8|dF3&M;;bH}((0urH{wo&2#%t&Hr^nHUodLuPImlGnnT;E|JP+w#0RZ9B06PKE7}}$)>4hJ zb9Ls~_E4{)Y;E(JjcGIKG})uWNw$siw?m_JbstJ}2R=vJ9w202a&AF`!^kv|#yXwm zr~1Kj)&t`WdHnwEJ~;ZSJ@DJDR>%h+kSQo`eGFR1?tFl?JM0aYsTRAS%77c!caqd+ z*|K99lPnUcq=pS2}@^R=d9tli_IRsUBa7tf`V# z2%y$2j{w-FZSi>(b&Gr@V-GP-mWJ!Ap_SPn6O^BnpTHsL^+1s$@GU4kIx*$1E&^va zGZVwW@HR%U0>k?20nW=-@_Jrv1+J0b9s}v$Iuxlw7_~DWcS*%R5!HCzgN}R9)AzsK z_@$tp*_rSSe8ZO#jFlJp3LuXN?kbEHMq3N9GlIC^$R~1 z?N@Cy?PrXA4lYDb)>_CmWkS-a61SgvH>_;ldLs#y?=j$i9fR80*;rLm(%xq#hkf18dWDd6h;TPcllt;m)<$afa$SK^HP3=R|covfWu<& zb16&GE}h@l<;SCFpZ2jfw#|Gq<5w%Eu8jcH7z82IvQ(TM8*-4RXE04?W&d0Xo|Xa?H=j7d>w4=X9h5_Fn2JLq_bkUe^o9q@@Ultr+7c#KXkIB~l{`Slz{OPNm zof+_-L_j6pNB`GFf3&Avaf)%CMA`o+mJ=IZ4;Py}bQpZdZDNGL;|P`*jzoh40j0o; z-K6v5C0N1gTM3F5|DPkL$6sOm%ZnB(Q)qUh_3`?wi@di zRd+ryz8%E9S$BeVPaegHfXCA!H!rjf-mF~}h|7X;_F@zS<^)wxKD|&_s>L`ag<9zM z6ETYPn}Wmy5!SJI3vJo!F773MF#I(eSiM!|Jqn8UL6!Xty&hE3L7;5_MKAa=R7y-^F_&$_Y{toG?$^2tkZx5+{7#Vb#yxR`3vI z_E#uUw>5D3-a^p?jdUBW7ABhf+7LoKt#WFJT>O+AEZsO}yc9Wtl3%5x@=)WY=xZQ1 zx4N+R0zoqAkb`bG>Ss8o`@bg76(r(aw$hXGx%tnvjFDX&YOSbO*( zX5OX9q-^&pp8=`gS7&&@AmskCfe={?D7PS%Z^e>MmndM)O>7~&{L|Gf2G$0gz^lSo zDv_aFBzxC`!uH!MsI*uG9aMj?Ei7;feAz%`cG!G;gOb$74_7plILC2En2Mwam4#oq zsdBhr-Sk%yd+{{?pjZ)z`dIJYAH^FzcD ze5^PMYy4Ns`7_S<2?@A1Di^ZOEpCbniq)|4&H+E4)Ugwyaxq1yHBlsiLBZigpi1_q zX>c~q5JTVrR&3~*ggI6*oj$?DY)UvyWNrqp(8}=66`k-SQk2rUFK7-P6geU%%t?bg zM#WGd5~ON~n=8ZHnp*$+*O-Jx$JGdv*W+$#QWS?(F0D7c%2EKz}ie{bqEO(0`v+pR5od02eO(AYq`?a0?q+>>0owSSwQR-qq&|sr zLg=+RxaeH~_BW))rMB~?c}~V0VryjuEUoS@zUWj9B-@h$=jh-?Av{NE*EaDU+SDc! zgjqLGAL^j$$ykwd>Z}f4iB(YvtY96+FtL2tJ`rxrur^mz%1p%0e`;wS8seH(K|Mu* zIpLa4Q%Ay z@J_;r$ZqlV&Y(O+cDNH2sI0m=E;(jGy)_CUQu+|{k|y3*LuU1vp6N54%>$sMZ^JKt$kX_7!4U_=b!WJn>gu<(o z*Z@lY!|sPtC9k$q_S4Ex4|NU=3vU?>&hS2cmEn{|10{jl+qO&`qfr}0KjjGu3c$CZ z56N5zlf6M|BxBOvNb|cpsR4aY_I1t>fQ2p)zLr1Li*(u;ikp9U;~;4FMSt+7+W@AC z8Lx(>v4zlYVGh%%9C53p9_2G~1?8t1^uCn)`enIn7K9w+&t@V|jr9uO5+Oq~Q;RJF0{se4lcW7RL!%_|BfjJ} z-VFYh`f^Yhq0M)sZ!;;&Q2@qPrOOZz2iavF^zU#4Y=(VJowqC=%<%>qRrl-u{tL>@ z+A`7<=iwSuNeZCJM7Uj9^y~8Sa#0CAPMb7RHsBqK4WvTS2kFXzQqU;zlB$P^`!M*p z3awGOjpI!VQOdAwM4yGf3WIX%f(jHVYBM9IZ3KVp3xz2v@GO*@^0#WVG{6mHlG2&^ z4$NU1sTJp zSPDRn%1GyhvhBa1isbvMRC=fbxF$m%qVqlJ)2S2h@Y0!SapXg|OvFlJq7XpQ;nW~u zgL)bLTFuJK3NN5FolV5nQf0aZa%I1?xwK_K`jn^k3L zNE-^QxB>f>D^X(jt|Fw+Zz7CwuH;e;!C5%V<(Pwmwhh2rB9ICSP5hU_pDVG$OmKjk z&l{%=R7Yr_9gM^#1QEuts=u%ann-IUru0mTQyQ6h>}BahR4sb(nHnlOJW~rg8_law zry%+lDy2QRF}_VH<0&KqJ%sv9`MgRFE>HB(Aji(Rq#TAGHoc=hp3d508ZUtbc$kd* zq9xHcF(Vu=0>vLrbTB^}sr4N_*R1`mU2ah9N_R;qjP6Um${$Mk!3S7u*Z)N}tn%%0jLsHqyuUz8qq6(9FqGI46$g zw6disV{b||11vDNu#|$7>mhU&;IqaS`{9 zW@Ww~z?nEE{+QV5xEaVO%x$(RS2w&Ge$I4qagJdYo_KQ>A7$oheZW%vprIJXv_|c` z6KH4vPa~__(e&90^yGXf`HF6~Q+Z4X+2tfnMZ~Iq{>zxK6p6&Ps1JW2kmE3kUvLw{C?Qg#zVEi8PWBt>ivj3;@pc@!*>V!N2T1?eC5DnRsPsU3Sl8rhj*n6nRLstDlsIX*`T@FSSRWvj6*VK?5IuMTJX1CCf zYAnJ7lZo>I(T2KdOhsWUQY)W6g=cC;BeM?X)bT#7m< z0zo^DmNlI)TS9I^bEgl>Q8+lX5~+4bO3@=-8_t6g2z)LP>IlfL;kw74Io z!e+urg~tKP4<`gk-xrZd_F+F(f_lPbe%{@Ac#LBBUBF366kpp&N)YwS&&L;#sMFQT z%z}ANq1vg$wag4&fh;;$CLV(s4JuRCN5SOJ0tRI$>D15`=^NG12}hEM4Q7pWNm}V{ z$@8I*hD<&tlvLXIGY{d5%xf6;YEKPdD_k6eEv{}OHhsww{h)?tLpjTMX5(Q20NYgB z9aqL@&QC4ib-#VX2!d00ic|j2-1|HPCorEO?S8I%PHUpRpZ_tLvYCXCWn}=7u9NpS z*F}W@>|?aZCznVCAp8|$QmZ=gA^bP&xtZ)qYDIFo&sY?-N#K7W7|kclyURZqr7{cy z*{`<|A^^}Q1}OY*?K-0_jO@>zxjYhNWy~i|ICJ91L==#gTz+|-h{?2wo}Iw+sr7mhv>;&A&yO3c~ z$Hi>fU!~FXuPzeo`YlAY$_BGF>~v&F0(rjj$;kwd<`M_m;t?;6nChrn+9_0#y?zC2 zlgho}e^<6VN{9xbDrK+DNqyzO409NWp05&*kUOliX9{iJfp3<8NHmSK=Nm5z6NLg? z9hW`1XXN@B#4W9pyt!TF|;$*hEm{UmD zn9V5z%0@vcQ|RgF($GmV`L)etI>rIT(ohz^Q3<}g4xT=e@$8#7$O;fZS&9-f{a}FF zc8kxpJ2Z*xcG|u@z;lMw2no0$o#qVDfF$qbRYDspy(LYdR4T|*XUt+nYI}E#Z(~_E z83bKJf>`s5OE5KZu@9`oqSJ2Uq{t$|$ZvT495+k%TH5#U-1A*+uw0M5$I`7;Nx*gg zsC=~#kgxufBl=>se&d?tNp?<0%0WK6hamAP&0#giEokr*r zYmDrC@&n#D+`@Sp<9*nHeU_Zyx*jw~hoP*n61F((+pd$I~7h7D|T}$K^5yC zj`QWumyDme0Y%>-K`3H7pHd`|$ip%+i)aeQC4!L0Wu0@u-{>H~g@gqTnIQGRJZ*)v zvZIW))eyf%#!V+MRTRzrs1%E8(+OjF#l*m1#eyhgO}av#6bFhjz|unNh2Z)EI37Zx zlTRC}wjx%cQ25eANk5@0LcXzE8K!8f&+% z_ryRQTw`Tq{bbP6tXXDDfhrcH*ZxMBan``vKFEJ;gyAZii%wSWxj`vzTp+Qak0S?~ zQ~#Ww>)OxB7kbNI1j`p(hX~a4@Z{aF$JFmiY%J%+1feO#A}CBTvQT3rF^ne!QYqU> z_YjOt9l={vV3OHgp*bfiqYP=;M8u+BfM7+Le2h~4MG2Nh$||Nf zT`G&$;JKqHReC}B5Mrh=S15Y)2+NXim?Z^tkpxP&3O{*x4Dtr6kg*gGW|&5;sJ0v0 zl!oA97I|+?@}Z*Vwotj7HSRRu*iqx+6OUg;6TkK=owcl%*GYx>9)-TnoiWqu%p&;l6a?9Ii_3dAy+_hh;3J$QK0^EcIzfpod=f- ziR)HMR5ycV854%pA`NgJ1^k+U-7-;Z3oBLL&;XMYM>DlvPx7@!?Z(0wMZ=cRE}b_7 znxrD&G|x}W{fLC^^{xg^iCaue`;gNY2c_wfmm{V(FY2#vxYRgPSofQ1ZEzhvesZ{a zbK4V7ew|Z8B=`Hx?{Cr@O-C>GVX&Hvifng8h-9(KgjDc_p*io%H68IZcY{gFEu! zVJmZC@m2`TcY%N;_{o?P;gWAv-7w#~zhx?13Eo$SlP*_;*f#nD(LET$KzrBB6HGHm zq~(Q0M~3jFn;;N{6BGIW^#U}D8I~LdhX}r@pb~)h-V7kfJuOQFdF=dtiGlztBTg2? zu;ogD6mFJ*v8%@}#b=l2)WD^Wf!%f!5vYz<0w*wbl0H@~ms?WKT$;naVDa`#fJ+rs z`eec#wqM`AkLNP5p*K-8sn!tFfjB8i7A8$6$v_~%foqO@Wg(B7tV&v4C0E9Oll_f9 zG%=8MsadVaN|!TR{Z}u4kVl8)zH^(?K{Sktn)%4k877mjH-u@D2*vVINc6Pn1QtBv zV{F0o70V(nzc;Y`M#fHf6>c*Dg8!*CzoXt((#6(zDsb`IVB{FsR>yfM!ht@V+Sq#Y zR4XL0S`EhKW6ffdu}+z2_q7$8mCtnaaF#fzLK@{X1qMf`dt77E*1jrKw#c-xuGore zjy;O~x|Q9oNj&~avC_`{7xvJLmdO}`=zVtH@R5{DQ3P||Grq?GASL@B*y~TX6N=H$ zQOa_3dDviC=I-Z6QR72ul7yIYGsmTy0@gbQ>d^*gSd~(jj;&FAB~bpUuH0N%P;YHW z!)Me`69L;X{FP=4s$D>CB;`fXTgWNrJ#d2qLh z=i~R?QR-_w5ifN(>#PY9O-`K6ew0wrq(p{>MZ|Qvo=^$-JhM$v7S~e_ z0unPjUdQZd)RE37WP(i^oLk7u(W^cd5mqDD!W&_2G@|y~-Rt2ezaHySa7n+rd+j@W zI-Wluxz8pW2(pnA*EHp>Ui`KFObpa^YST_%ZDGO3*|i7$hGB+SOrVX4L58r363Y+# zm5q`)O25&i18c#5tu4N^@nS~PqQD)=6pjfg?Om}I8J>~N-~xK$E5r2|VG4n|@dYo{ zh3>rnfmUf+QCkUS78#OjiR9XKOh4~skj)y{M+!=n+T zF~fvsgoCki9{HSf%DZ|{;oG6{J3RsYJx-Th4(v}v)}bXs0hSzb{Z~dgzi;rF{M>;e zG%R(mHzcRRF)*ygvgI%+GK;&&LVhn-OC7K?1GY9oJ>OP>->h(aFPegNdhg|H3f1SO z49iAauy%v^b4ysBvvyM&`6orBsr2t{Xx@n>wow&9OFTac$ly6aORY=k}8<18Fcv+etEQO-8TM+S}jB#9CsMgLqGS4=zJQw z+OMkMBn^N0Q&;*DF-$2C6Do*fRKiUDjx`H2o$74oy*A$I=i|NQx_)%B0dLfs0#*X8 zX$G#_-qOYC&3Wgqm!s-vd5!sV&KSt(k-@WWcG#>>n_jybmTq)0C`t#t-~8dlx4HEG zymJxxzkLF7QU3D`%S=v3P+E%)W@@(fE3E;bD`C$~#Ipn4EE%b^+dej<9i_nI=}T>$ zO^54s;@al)I;D`~|3Rn!n*N;6ZKI-Q4YG5Avj2KMvi;f5a!XIp?uur>@q_1T>oQof z7Ie{LF3Crwo_WoL7U;w0E!UMObqad~4jzO;zVUyxPasN)&piQEf3e?HH8pXs%bCoI zcY>JVzBdUhSxSNO&|-P00_65>VP7Lc3#U+^;hX?XgfMG_6lg55D-yDPSUkk3N}=)K z;yxG3m_J zN7Z&~Ey77{9&Y>C|2m}^CPr$~Sd^_}Q|G1$0np{%_9%m_i8m6WwCAz-=2SN~U!K|R zYPO#p`sb;i3e0@S!T+MZ?|dRJ`l&;=7ISgjm}^^_diCi2e%8qjInmqI5rSsiTKi;} zZxaHSIQre|Q@G-#` z>$~&DSKKqAPgxW(BdnMg$_7Gyh_`6@PNio>VD&^nn6nc8nehXZO z(BA17ccoe347^AW$N!X1c73=ivIyP_r{UD68g;#vy_%(nZ;lhc3wKz6@tLIvy7~lC zMKcEO%!y(KMy5LFMh>u~4xVWlWYWkn8DzX-JYaR>tqav=<-%JbS;4V18)}FZs?bus zgb+X>6W5U?_D23<lKkDr!3dC<5TAehWLfI7;Ror5eSK5`1+8fPwv-sl4_34Z>@X^Ru7&mTn{Di@DAS(#s-X)R_M;=`e_4E4(9`ID zokdq9BJ8CIP;Z|mQ$7{7MVGWH>#w`UPC?5=5cHaf&q{eXT|6Qq=3GdueKzl5jM#}9 zHHm?3^w^ktJzApj9X9;Z?Q*z)Xy&jA$UL*W_mJ>0x$RdQ_+`Cxztal&{YDS_J(k1I163#p#nm61-9xjL zD)S;7N^ofwETw5O<7VhRqSSQQ*hBw1aP!<~brhP)@n>^mqiO1AoGUW)sDWR1gIvZ^ z6a?m0Q6S?>fx1kEGq@`2WOGjiab8n4Jl*CbHj?(Dh8jd6rGQN0BnZDGYUwZ0N5v;A z-b-9*M!uFomBzWbOHJG{{Tmhz^v!=*1ORa1e>1@q7<1PgyywA*ed$X)L^Xuv20}3+qh@LF7x`3{7V-kdQ4Nl<$FSJ*YFmXX*Noc z{Ke0+?2un8JeMAqjt^`5#JAuJJao?*?7X3{`JNOrIpd3+-_Bfx!_>%LMtsZpS3Vg~ zMBq;A&y(@a;~F0_)$- zkibxT^3L>MKidyi-?=s&S!t@iu~x|jxCZ64lH&AOr(fO3df#5*$-9DPK4n+1tKl2f zw6ZZW_TvYH2pMOO8KjGqh5UKqaUJHP!Jm#8vaGm*43yX~p-`UroNfm-s^%8ioj!*ZDNu31ITTuecnmffz)uQ~g5@vNO4F;q3IqclppN*dk_5QT_E z!51E<5N}CQ@kEB*w58Q7NC|c<>q#Uy^Ayb4y6?I!=+V#EolLy0rUYMVjr^nw57&}* zto^IWp?5uS{>kWNv^(O3Tp4ef#+u0}?tJiXAEGvp9o1+9&Ay;{&$=6(9ek42nM8}C zYAPw$tCxo4kV(%F%|@d!$I@m%qnvs;b7qLsN=U2gp@x}?$~h59$Q>)fMX<6jk>QBc zLaY~yff&^(P~O!g&AKA>3ZdGz#5$`CQys&EJAFu@7$=Q}S^(d4`@rt{eZf9OeZG9( zYDmXc*lUAprao-Bp))wf;QUv~9y22coyyDJej@qsi*2ZMrrv}+p_?Qpe(TEKeSA&u z6yazXbp{u1yElo{yLlDs=@h}~iud6xz?~g#2BaGzp`rTXRvTdpSjCCVkku{R)^DMF zeG|zi_M1Y$H{AXE)S6keZpy}h6Zojm#_H||Suc$k1`jdeu2#_0jJQHYKOm|TNka;j zXpRjvd17qqRf83b15KJHhXyx_YN4nn5g}C+0Vx(Cb%HKXu{Pxi9&K#p&rVVNNa-m^ zBG}k2>ab8yQx9%RB_d|xrGWNj8OJyY8rv!;aYMNJOu8|n(Sm2Ogl;Q$jcN+hMp?(I zUwM~C$m=(muC<=U9Lp{)hg=n>?bhy6p*^%V`3B()KXmH8d~n>fIti0I(jtk`1w4VXd__5kKJ`O+eH?h#kE?1-r#}q7`O4@4E9a-7^8PB#(%-9D z!#hE2PduR3N(dhdj`h1rWkVzGpt@vE6L;_Id&aq5QybFj2^p`1c@q2S14w6HHU#tR z7ZF?t8o6*ubk|R)5=Jzu$XRjNY15phL8`for?dpCx0tad<*L=b;~O@GJ3*M%8D|qz z#1Om!Qzo_s_0sXtGgXLoEs`J`-xqYnUJs~}RE?A5==bP_AYo9P`ci6PVQEUc21#ad zLklE&g)tTVW<+o%OVaH8#t1akW`envf(ftgWw>cIz!^E-%Q83VFuIK9vzIR_lFd+L zmGA@PeS_}C8Z;F*M1Lc`2)!Zy8CY*wenU zn5tT>DaMPD*a;))AS&}J=AbD88TTU3Tig_H%F64i;EJ87I_i+M;&k4=GQ2TI5=4hq z!Y2ejc{2m;;@qMRr3|)$gcj*3bE?CxIU%?Lpg3txC!IqanWPW z|1Y*09n{7X*X+w3S%0mS4WYf>PxI1G|EUoPyd3$ToaI64uq75}uHCZ2WVNpB2ZnpF ziK-N6jo-ZQ%9<&dg#8iE5VJ&3JmE0(V(m#a-v7jc39x1eh_Vfrnsbtt4 zqfdy>;=$8<>BCu>9j4_KR0O^u077RzKOdL*PP}%-ox53j(+_`M!ue5bj!N+*xw)Zg z>ClB>BpPzSiJGR*UA0U4&L0t2oE}H5DFeNygS@(Q5O>$(x4j9!6 zir1%@rxG?=BA+YNWNypm;lMmJ%SY=b+gb|@JM;?3<;UYGuRp(gc_$}Cg}#orck5J} z(wlsHIPu!*@=dRvds$0I7Il!8-@{7Jn(ydD%hmk7KCyaqX!FGeBKwBim;pz;B{u%m z+nmXGa&ihSBe+l24|iH7;X6>TMFXeDBhG{$r%*!>CTp`4CTy2Dg#;5wCp8DpRh!CL zqvwP#N0gPuO2RWrnu4N5W`B&PmIlT@;a#fX>MaR+_GuxxQ%Sk7f=2gEM@_DG#SEvQ zCa>@Oe0b*)q{%z)W7WAq+_rz&>;Pui|9Mh$|BJSgcE5;XB zRK71GUgmuH%awIjx=4$2g|KYA(-CP#AM@2QrkywENC20{9^O=77{d75?sK!r;t4)* zQ!$A_E&4sC!u)!>(NvAEIlDXJxXY8Jd!gJ|0Su; zB-)Y*CM)8N&YVAUvcf>J*M?}Oba)(+NT$|!}dD;f^kXi4AYcFkhOYbI$^ zlDJ3`bB!e0Aox$Q`mtrb{)k^Nm=GW0_Jlw78FM~f@E1Ql`-lU)YxX!zcFRg#M=~5) zYN%EBtO65jE)V0E`QlPkE{9qR$jx<>n!bIv{jtN}z@7=yD_U&wE^w0rP&KrRJvPqOLc(8iSvvinpJ2CNkZ1 znB?c+UML*e<7)vS8X+meSTt1q42pcX-!?gvN_pf7?69yV#+)NCfYKp0S=$MO6wRdX zs>`te95opmF-2nJQ?fKoR&(*X2p{#!O%__IG-^r&FgdISEQN^XfZET>?d#_b>H-^alOcir zvjF7``|UCkV(#Zso9`8uto6ZG z9dV2|)wN9pjZ81wE=!^J(V{X9n*n0`d&p-GItdkIX!b`2G2pLzEZVMX9;PrHd`L{W zJ#ae5_4hg`oiz*P8EH7F!8wQ2FsYK`Q-qUMQF|`GGqLer|DfL~#%0I;eq|GCK-;vX z|H4NWoi~3zAXaMdV5IctzaICQ>hm`YzcBz@hC(Mqk)zFNrZniNj59wdB~O{#gak$BMrIRM&>Yt?&7)f8wMhV>E;mYhy04YSYQUz=RS5nimVISj!XvzduUdryA zMmrCTp1ZrsPJ|zPkB%~GLaER)I)IVvczKzavo4rl@bw4L zex__`LrtuLX(E>+6b~fvRb&?Xi`t~e=9q_g-|+_*_l86-FM2=^AxyD(PM1m}j%gMHw9YaBYZd|pSE8k=Fpwul4>5T2*tuu#4m&XUbr(UY3gRzm zd<0XPq4RUGOr#W3!qA5QAFdx-n!$N9{+@#VtTvOB5niF|0~La~9SiZ=%Xg|<*oa(OX{^)XTNi<8pknGmh2BSiajhbR**E^&p!^i}EbSyYk6}viH&;Y}V+- zXJMJ0{lZ7(S8P}yk9%|(zLU_Yh($QfUH#d_cXV`WIQ(XPbYxZ%!j`Fc$6BM5A zIzEc?y~LMDlDxni0WaNj8Z!mOrw7G!J^4?`bja?|@5W(Y<2TdqWm&!y5X2Hx#-yD~ zCxbK^zC3W$Vj0lqCM)t?rOSyK;9~nMUBUX00{*y{BP!8UJ%@z3;y0n@K88S>DiDJ%?FA91(0Y>7IrVGPZ+`+MrZ z?EUy9lD`b6*(r`{FxUi&780grD`ILsFH{%vSo1#SLquvrgGWZVpw%%>)H+xIjTBwa z+3fn^=_xYPrD4L;{}{th)xh4{Cho+iYD8{_(k7QSiGeFe z^ed|o+n-zY&qWMoNx=1m3DR=Q`CP9V z`FN#tKN@z>!6E}dB4i2jkS~dtvwiUbCwSMshuWiBcVzV?%Qk4}ovg_5_u`}sJu|b# zZ(&_$HNQew7eltpURhjPRFc8218}MxngjU7((D;TugoTmDy9be=CRwde((Jv1V!kY zdGH%K&`HjWI9tZ(m@-d`q86wOyEfGf8in|ZzP#x2wK`ku$|iHad5TcT@xHL1qx5q{)D4DWVqdMx_Y z$BS9R99lhxEJ)otVDzy1oPQi^j#Z+7ooBlA0pj&jyCM zbc@_)a%{h>I6hd$k?4uL5dXe6RbgDp-!nKO(|xnD&sMUgLpQt%pj4bfiaW-F$Y-dh>hLqoesGTl`Xo4*2rDyp1DqbOSnE z%m(;KaYG@Y7<5hg%tnw1A#AyXP`PR4dicDpcQicCcx}FLjG?2aEfpsOO9s*y-E3fY zJIc%x7*Cdu6lV96=n=!Dl%jfG#xDeK@H3vRr~u)BADi4omZR`mbS|upP3rGmUxOXF z+y^(gAw9kjGteU>8sx(0I$?ufeQJV^U4u)S#1piGz^%LpZLLSCNfR<+#dh@JO{F(C zbaTg;8rH63nwDD0Vp~f)m2%4`js%OOrc1y`9UJ2r=PTvRa2iNbDzOD-H(OEc(=}fn z`5w%x*Ytf~H{dbTPGXtlln<&NzwwQaiW%iZrR}~aln%pHFQcPLS1LT#GfgTAokX4; zYTe6Ayhb$JeHlJ+ZJByax`+q(2Q)u5~x^H(xghBXU(UZI4-6>vn3>ria# zgxSY)MF5|4ro(x9Jl66)SLNXk&z~h&KV~^X>RY`IyRb#+Lh{2UyItN7@R*CO6XrO8 z+|3W9Y&+k zj7gHR1qx#I|S77U8==-w7n1OagC4Jidmd)c0vUfh~xnb3KaD89c zR0$MIk2f|1wPiCWtc45r)>mXm{;w>xbfCW-Y?h4PZL^Pq+M7_bnfCRGD1KSczCuQS zN{*eS8^Kx*2r{gl6S{i{!i@PDpxhJFdPWvV=kTldUFq8v5A#i6EtGN~BI^*C4Pplt zwo+Q2R82ml59DII$u2OtAu)#;?eQjJD8Q{k0(6dohjalnaZnShQ-nPDXAqZXHNFOe za4yr8#X>h^(dSDVoJ5tJ;dComBM(e2^#<$Q0)mtY^Hw+0(aI62gEpE*w+-VvlQ6CU zPd2Bu(O4^)jW`@4lm?Gd%b?R<&f2f}&5Mu^MrL_Nzq+NPAh#Qir{FXruyoC-lZDt2 z#oBPDO+*{@JV*KL=0fSCib;WM9!>>V<|vx4X)k{M9IMTT)0%N&Ws5);60Ad~6N1K% zimana@oD2+MIdPNVDzfp-EyLlqpRQQyp@SvHhxdHL^~&(~!5Vwvh9x=76$kDC;iQ!GzI3+)90m zX7z1()x$Gl)U-GB8_e*Cpk~h@xb0>Sn8s(qtE%Rc`eix0JJQJ7wKs*4Bv@%k3UKs5 zNU{Vbp_oev_Z0gYk&BC1M=qp;A%i%hy&MR$&B}yU3Z(=?ih%~%#!yu&=5f&IEf*rk z%s>-y!I*pM87nLXcs;V>J_^efL)o~Y zK*u)?EU&}|3?hw+ayONANIs+(!VdZx=uW|IJ{TLv$>FnWVh-pfL4B#gnk(h`&8|>3 z=LWJ;2ix(Vq`YbCrlFf2t&EjZR#jnHx47VVEq_e=ml3TAX&dyNL`4{dVh$E?Gb2GH zGygK8XIXSSGvBwbPTO>AcRwuo*T`V2-*Bwwb$6rh>{4lTG=qLDFo*`={eWJJx}<@{ z9MsPU?N1YW;C>uj=YrMjMF{={F;+8%=0BkKRou}XaX|Ped?jC8kOD zty@Xm4O?dH+~=~phJu!j_+{$cBaFr54olALJOuN-ov$?W{101{H{G|8!F3}8RM@a#HreEDnw&{h)~5MPTPtjVR*mO~BWp3UmNULC0^8kZ^jt!ppjALVdfGb(cL{B2>U z!XAv{)YaAN%?QL9wA~R&hVky#WK-%X!0X9HxZ$yIGzaw)V=sT&x)NW3M#tTk1U;a; z-2`cNbs`I!DF-A54XYrk!*zzP7QH6#R60uVgQxw@?9KvfCqqyACHF6dj!@vjA0Bkg zrYA$@&4X5cgje(->uLj8U;BpZiQL2hf=!RKuQHy>8_QSPvh6xsox0tv~NQ zW;i#PknNr9Jik%+WRvuw>Vq==HjMyg7Uryk~vl}KHT1|a3;7Md(z9D-&>E923}RSb8r<0mQI{WxnAB;3|*p2pP-%>-Lt{; zu;gx1*GLTLZB3PKRI)*drfKIv=oTovAT4S?jMbeLG~yf5G?3cZMQR*G76yr+FH`3Hm$@4QIA8_!v9L`w=lul`)YDc`PUh9R$l> zwwF(bt4wjuetN(bytBSDoNgrh=7q-Kur^=c>tuL_Zx=R1fn2W<)=Mu3?-e zfbgI}?&^)i#?iEgf36y<0`vBZZGl)4*Eo&quE&~RUxR@P68QPX)~-2Zv@6aoq;>~t zp>5_xzS2`vUzlvlzB55ZuNTM0&`-vJOx_MXjfDJY?|Z9aB3%>XU&+}z34jZe`Lc_%Q z$0TC!BC~IjxyDgcB(F)ScAM+mocD!5L2AJ?YTSjL-2L#(s%v}x{#A_CNu7U-U&74o zG8J*si|pw<)h2W^(4T4hwI61?=4&Svr%y|iItOL8?8aA*ZB7R$f7N5`eW>O!nBN>< zeSH-OB)6P5Q-A@IN^n!`&4f-4;^|t>{6nqJF_}~;b~Pe1cfR!k3rRMiJH;5qsm*i9 z!P_BQlI$K={ylY!j(f_(hjjVbW_s^|Ar zI|Y`N>3U=G-j)Y~DNCRU;KI)5iSUS!(@Z z+)=~znlQ9#jPDm+SDnbNJ#kAGLVB)ng3gC-K%sGhK~-u&tm>5R$4jg>x1bF!)+h5#DLtlf&7zNAB8m{V-gHjfL2tP#_tW0y_{* z&aHbba6{h~rv@hsVzBN+dkB=HQH$r)QFBIi>h{?sPU4Jq3DBfv3I>T(!&)xsuxc|$ ziU^=8`{LgAB92QK0t&2$=IIqF_xRrc-RtiyYPBx69%3`0GP*Ppd6DW55}1M#N%}vpH%(WS|A8~p_)pq-AVV}# zR2hGjbB(d8%n0cu(|1lz&Pc}1RM(^j+1-Be1^SI}^|uzS7I`vMesEPG<56p4`5s~1 zm%#5oNu8Yoev$QK*=6$Vrc(5!w9x;Gldsi_e*9*tqwPLOVT%#y^}~<-yXOS149sf= z3bKoWTzdv{g}`cq8DC_;2AXoVFM#oXkxQ;P=)>Na4dKDQihak4xtAr7@t+>43~z)~ z`&p5a2$@{G+SNg7ZTz2$WDq+v#22|IceRb7CET%ofc2ds?2=ayl#kLJVT?lVRxb*& zV@2;sU({OmaGUgl(vS+GDKPG}-YE5*7DNHWHf(!O+AsfZ`d{dxA_q7nz&MYDk~9}! z!RW%@Wq@%Hez%sRbc*hzMFZ<3lobhRfNCP)1AcqqVM9g@U2&@MF=z5|Q&6mNK6vIVQWc$LmyIz2kA_0Qp5Db`%?=Xy1X0$AtvblhXx`x_DOjEhWkAMGnbsMk? z^u@%LfIGbd($T?;+0Mwjgf42(TMB3(+`mVa*O4IMA!}yBrt(Qb=pu90HkiTZsYl{8 z`qlQgWzB7cOG$_ARP(P}X{q;@FF^w7o@2Kx%>p#e4Tc|sZ2v>s6l9Li+aBSp4Igrz z&sW=rvOU1+yrFJDsfZ=sOp5>((D;t<*L>5v?arZweWfEXx*JExA7VSTl(jy# zDepCXB6OU64Utwm!h$C3CFCz4BS1kkzJgThqiEQXeU}F!Sw>UV&p@aEba}%f%XroAEC1tz} zD3f2JJU*NHS48)T#q@?IaZ4zv(BGzZ6iKh>k=4DR&M{_GWHM-xB|t)A5=+X^$YQf1 zK(<{b22Luq6-O$DdrkdKio3J`I}#|Wlo6tOV@Wza^vnD2QSUK5GWGtw!hgpI7TH%Q zA%boY-Ex5=HHbxw5M9(B2#<%#_F)1N3-_<(BeveLR|ee3}}P@A-`USPMce)5^gOti=2G_Leu){3d;`9!}` zo)%3S33BOQ+K6=`w*mf?gG`A>p)*`qpl9slIgxZiRDMl^_nb-?kBEi^qb0t-wjH)s zeHH#7kcwb_ibVu?(Y^yvz?~AAD)7{eA7)i$QX>Dn{rtDwj)D`UrOXgsbyyg5eGJoZ zc5>P>_3N-d*)65GGn#U=?~SJG5Q@!q5>Q7oOObkXC;=+$3fLHrV%-NKs4`@1zP$naTV| z@baO~l^5YRV50{Pk>pQVj>m@$?a{j^*g-Cl5l3AUwwp?CAJP)}n=NgNxveNpNWK3K zCH{Eca*4m0$NJbZi$e!{BomQy7`~44w#nqwu@l@5lPhkMW-=PJk=2cF@3+iL?p6&_CC%quiI|Jy*K4YQqydx4P@ zv7o8*ev_l5=o_|}lpg3EMbUc_Z0@^I`QNEyefLjoFUJIX4brq49kZ?g2jd(MR~u4i zrJzzLOL|cC9gx1WL6?W?B{?Q{LdDQcGK?y6jPrW!PvPk@n~9VPI7~mxFgkcq^gVBY zj1;*;i=gPn{3?-TFn$fwoK6$T-29yIW48-6q6~%HY$b+$@Q#{QtDLp(*!&j$HYNUb zDY8Lg(KQP+nNI|{#ysJ9$WYlJ5W|^rq6uU>_aC1C7Ws)pn)&=tPEcI*HNzyfT$*lDStjj{JPP>w_yuzkqN-3*F!v~ra8?Ry zt=6Ur74DH&bD!uz!OU&nOe8^XVdK&F5Yj%2#wJt++%9%=fy4*Zyv>}kgrn7%8%lwo zpGV{j!61@Yw6%5@3tW-^qUgQ7nYeiHSXD=d_m0;%A_aJ(1PR(9bKV{6=97HP?9(`4 z3n}#@7N>>Nur%zNL3U-++a79!orT#_y=gjlML8lP+NKEZ7+TDL!5WfzW-Jj>+p^CblU5J0>7`xuItK_bI}^PqCld zDS80apfbOU<%2<*Ux8^z@-qLq6oqbad~GPPSbdVdtODh02}$DG2Vj@cwNqdJMZ_sA zFo2KBvf)>y{raOgeSu2GpH`aUqxT<~Fis570Qy&xAwkOs2zW5!0860wK9lyKT1h?5 z3>7)_H~lVA%h&Ac-xC=!!Isa2&+FkQidmVn2oR{x6A$c$1*Yc-KoLcEI;4l>HXk zZnn`}%H_uTDwWwLhf-`xUn&kHnZ*L~>sWCTtZ`-`*s|Y_pI1e^$k4P*|KnsZp~5zm zmCCvp8m?qyT`J@i)!x>K*|mKnG~)cQV>6718idIcvo`Bf;@kJBzWnb8gUI_lXdoVO z@X&)l?FPP6ifQgc?x^v}8|yT-Ijd*;j+Mkk+q_SAHk*#V-Xx0t54T`|B~hq}m`XE& zH%&x6Wh*e2b4X5#C+RneI=!{23YCsh&wEh+6uk58BBHrf#vzJKgUS5B1^4Bm*O)QU zJXR3^MEZ3&2u)f{4}rT47Uy?M43FC%AI3BP;{%5U@K5L5>A0KTWmEgo6CZ9<>AEQ| zGHw4eKQ%tU{7j#F7?{<843YSrcfjCwG|E8zr(BLk6o4s90z@zM6+|;IHD1zgT9tr^BLDLR2P(p6cdroQW9Acp+qT>$9;LdlwSHGc0vyzxncrvBRmRl zc9J?E*RcfL0BC0b2?=Q*08;b!cuA$pb}hL)b$_f83-iOj zwYSKANfsqbh=O&zlE98Q5~#P=VVz8&O=II>PS%E2Co<&1hKP}JaJBAhq6mh4zkDZ;%M+llk#VH} zuL%0|oqAzZv+^PBheZQ$CW(4O6&KQSnMcMCY#hr2G&pH`PjsgE+KrSrb6F7IS}xKec$?!@_n}Gbi8)>8v<;W zZuPuM?RNC(Vm&4$XH@*}ZTpr=6B|UkAWmq^#sJQe$(kjHDD;6@cIleK;1Q6A5XGiI zM}o@R!+S!hg3+u01>jg;n&nR@&@zO<|5xSdRl+et&XG}N=siOauVz*;TcIbk=mDm} z!}RPg{q2PRURZSA>=A$O>)S)!A|n2KH)a2R8t-v1U*=KN`t4H;eK5$74E+C(@YztKyCB!bC8V>m0A;k$8CL`G?2xsF-BPUGSX(KT8*)|GwO>kIHm9_}^#W&gz}epeeJ0yLu`iuuKFX-b)1>wNuQU z#urVI3jXinL@zk1y`WP<6VH&qFIagysIEPcz@dz;H+OTxvcczb(>JKJ&o8_k+WkA8 zBuiA2g(s%*ar$MyyLV9O`Wx==_?)4eI=~SfpNsXLfWRum5BOioXQ74x88IMo`Ub@L z?j$}WrMaTDA)}BVA@?~I5gH~5E6iX zoq|;vZYRdZa36nR_2zuxD09Re3`Fe$SZ4w&sqH_t1zxM!XTGn91X4QluSY5zAAcDU zbAF1=or`NJ=9O4VN!4;an$)Z%7+t{veA@h$F4-4Z$b`cMI;2 z7N>ZjxD= zDpc<1+lau%D$hKh?-f*aHL%saiJATWJ9~<@yy82=*!7Z+|0}Iku$)r~tMyMkLxXye zz2GlhF=-a}WF|TgJ=31KX3x5+0D{|u8i{Zo6I4H+UoJg_t4je_gE-OfWXxCGf;SmljMsv;Jos4cjWE+%=obL-=Pq?-=Li^B={o=My*MDS;PsZ{C61;{db35d5#`L%^zhH z>6IBTQU+IoL!f!(hrU~Qx)eg;$zdB37lAV94YwwXq*9@VXg2-?r{5r)W@#Ua`E$_RQ{Qr6n@mgqfV(hb9xrr{;7c$`aUmzv$AcD}30Fgsr#f=R(`vw8KZt z3aP#M*;zc7dAii!doeGSh|>|27)P9~B4n7oT$_CnJa>yVUTap_kG}q7B5G){6F_gf zlJ`%K!VA-Z8!tzd+Ktx4)+`EzzgQ{(IRC3^+U5GL82?))P72tT*}3|2Mx5=^r|LAq zxpG@f0o=uaB^dUo`eB8RyY|ejB7q*T+mx|-0;zb3i6@EpI1kKwYUr?*N{9cBaCNUu z#|BCsY}%d%I&?*6y+%9oy7#CXVLtCWFQyQDvgCV5TpTmGUH)d05)HW*&1UUW#RZK}v_~$!H zX?L9F+S`S}OOV*FHAxf(#fwUfe}csCQhw0GM@Ydg;_APBDe!f^lPq#Y&!95O{3vi>gh*&uhYia0-kTZEJA~0QD zIb{_vXY%VgfRYm>7l$C#+z z8+`?RP)85N;2XhdX{7OJZFqh(Ne}_?@<$aW-5^XI*k>F=!EQf3dT$g3-B^*nNE&)h z)SLbBF~DV7J4l%jdihxUI7SPS78I3XiC1iamW((a*j@Cb0IxpZ z-icH_l;_Qy-^v*oS&s96U+7iY9?5QdFAZsSB1WYYM+6T7Yl|&< zOXB$BYvB#?_#=9F-2d>kkMYf!fM87S#7Oi+CCk$n)1<0JW7oc}(RpVYp6i{)j< z^1KiD_8jZju(evZo8mGUEt};Hr=?#yOR=TFC{ zY&jb!>LXsj(rclKr5Z0(800B%B#2+vuW#KK9)l9+IheSEIK)SzjFXyuDmFT=uZ~RNKwy`tDb(sHz)?+u_T7^%6C7 zb-z<4b!ZUKG3wmFiuE`>xOnW~)%vM-MBtwKarAAwO#9 zbDSFc5|(_ZG&b|>Q)Ztm1!#OEnOW9Ba9`}#f_v`fCXd}NvAkfaM5z&4q{)}U_l}KN z?|Mx&S8F*G801LcBh|#W5jYqhi#rGZ178(o{txjQ@}sNlBZf;bs(Un1O%@+$4ri`a zPsGY&W)&L4$OZ;1JWzHgPIQQxvn3eWof$kr0ieiU#_`Q|0fWfajb{!s%auaKUQ*q1 zL`2D`+V+`!-o|wMSi*_U7exQ7uIykC_r63Bpmi*&6wryB-L$jDNnCi3#^3nWyhQH9 zEFER!7zu3+DKCm2!cCfwsi0VWc<%gLxWF}ZXWW5DX&1mE2LmX5@iN!^^6Y3odUOx)J#K@6 z0X5q{i@t4L%L!j!P&($qs%|kg|MnO-bU~OMmX!`a*Ch%26wM6xSlkGEE^P8WUYr}G zTR0V#3i>5VJA#aX)w^?-Gr`4_3}S&)jf-a+%|_V4L<3Rtflf^y_;mefZ(-Wy?wf?A}#M3>!|_+#DQa{ zQha97ccts8s)%-J)wE~A;8UU)hecan|a}ys$E3YNj-FyK~f~ck{cC$ zyEbgQ-|zn8d3pq#C$FCNFTv>XV@|f61^ymH!;~q8$M}>*M>+WFiK!gm2LOL%txxSbT-j zs+|Nhr`=thQ=qSYoEGXwK2fsJep@NE&lJg;THkM4l6nZ2tYB_CM^)M#UWbpw)x7nM zAZ+nYqtO=F@K1L@J|BLQ=Z$ckoBPfYp%Jd8{b*3fsCCJ(;xb1vtSJ*G{QoK2 z+=`2kMrW0EZJWI_4#n3apaT@H{I1leXw-$ml)9OkZ3;4?mZ4l^Bm<6|!Cj8n{*74e zZU#iBu3c+f!MRHLFfu<#MqmVTi5^Z)g&{<+n7o%K>#@#gDHs**=UHHf$XGi#%B-$2 ztIoiou=kt@>{uZlm=2cK-)q59N$rc#KHse(nF$;s(a?*)YqBA{JZMtBOfoD)cx{kS>S zwAXT`mL!t`jyy2*fB*J+v$3gf6Lv|@U&5@ymXWUhgZn!#akcy2S6^t}1sFrifTMzo zC87&<8uWVj*SYZKFPZeXIke(2B*Fn7_ar?{H;m){nf_%_?B(LXs@EP}Z1G};Ud?!H zZ==met4ASaaELzL+E@G5w%6)C^5=1vt7Y8V@D&wTbcJoW1Qk6sf?O%<4la%`y!HL| z_*Wh0%;jl2U|_=UF^$BjsX$sRI(QVBPs89BI8EQEgbcZ`E_Y9Iij0+&ce0W|L}9Ya z=JuX=F@Z3(@RX0+43cp&bnUkmVl05tKp7<04EwN!Etws?yTc#nGE-?d8U{5V3fOLt zZ+J-m4&o7WZ=y$OD<$5dLJAunY2fBIc*0GeZJhQ!Y1=4(Us$fIomRfgmNA8$w}D_; z6u^5;b}~Q7#VR*?c<8&BHZPL9!iflvr>FALs-xVb$)kEJyJHC(bw=A%f--Hr_lZlK z73^>zY$QV<+_}9$eH~stKEgKi(*|}7Wl`@Dt=naj9}~w2cpy?M$N2|J6hlx~J#<#+ z))nLLUqPfK1(}N23R;B8rRB(i7WW)3+ZL-NIRR2+LT-DQu7taK*W-WG&?ndb+mFB2 zRT3)_9!;pc@QFKK_5eFbz{k(K^u9+5M)2(5-(7pbbY3*_4C{pm#P4A?)a;Y7;!j0XU2@0FeM=hVtZv~4ZxOp%G$9gK#=pP(Bdjiv zo*r8E&JbGRwK+7f2T=GuNwbx5^tLw7>=d4V;yes<8KjjrAoEPbRxrP(x(u; zI;B{B-lQdR_K}X+^>cQqaU8+W+L}M9{He?~Mr5QNC-vQc2l@~w>v+1mbR|$IyY>BxAR3d0*XcrP!ghuoNc4&M$1~8mJ}^=sWhs->eaDl?)I$K4K-` zi$NSZ`2UyVv<++DG4r_HMP9$Xn*?hfIPc0Tvd8&7Z@i~_j z2L~sS?L#Y*969OA{#+;T)3(we=^%~g9L?zpwjxe$OrtqelEw^(v-yCn%~JM0+5B0H z88+@1jn*~u3{<6h_x4v$`_GTwC#PN9YPusgs|)6r)3;}_P{h^2$>|={%a&Kqm+lm& z*RzyO14>WQ(^So-u(rpYc97)G5oX)Lj@DK`<9#kn7+E=ig#I2N+JQxAP)|Ym6y}jr zUrdU%@l1w@nDLl$0YaT){gPuma3HzTY%hAdWBC!)=Q6&s0|_E6B=|Uh@&f5NaH{Bd zZy!raQjSP$i~;q1i}VDarEC?5RQwbhiE1dI5WVbWRBfokrhL6-FVfXy+2*#viT5!Q z{RyV@Mmgco|HVg|Qh;}rWPhD=+xP;2Rcvw}Xg(78kV+XQffKP+*Y#0D>4iD-lI?qoLpQG(k*WFhJllMV<)YQbH8bn?KvZc5m>Rs?r~GX6stdI z->>}pJ@ovAG;fZ2eF4V$YS$I%RL4 z=aAax!YmNNi537X(E0S^W#-^~0O-HpE6><2v%D*9BP(FI=79_ZZK@CvxkmF3S@TEa zr@Y-RwZ>=SuTj6njRc8RUVx!NV40bzTvpbZ(6-ew`A)@FH&+yDk4RD1(>lk0z%;>O zn6&egfg}$mnYa)$a9*L1+(&=fq^zMs$Cl-}}<(WuuqUCXt!sJQa;Lv!+Z%Rfm z?Fx4q)esU4qf5Af;k5}OMg%F`$ZVRB@Qk{22h03)EZUBPASe|)1toWA?;y*l^xQoC z;!B|2qW44opive;j#w8jEF>1du`_~wg3_F!=D_#&IM>>?`2^>;Lj_D!%feRNGU2}6 z_3H5VJVf2G0c&&^BDOKK8QVvY@(nRpq@hr`>g|H@vbPc#90>ia?>)xgd;j)I)p1+i zdU=jz@IFW_$db|vw^Fp05;mKli0IYse|vYgHHDOgn>?FhSo=``bZMvEX)d%%L3n>A z6q=H~gyd5|OwbDVnyEZXVe_erkc?co`x7$AXBR)iMmVge23~`mxxafsWgDy06H)2T zj;Uf4EVAw*WdC}}x7c7RhHk;0n=OLrE(1mdu!;EnxMxc@gJN0;jp276t5?dCW&zQz8p!8f?>ba zZ|{XJcW)|@!5wXKT67^HUQK&Xr84C`zXt=qV;7F|Zx{b?KhfA`xb95b4Wbjwjvsd6 zjz3HGilyp;Y7KPxavD=iF(^N-Duq~H_%3Ia)xmnSBHF3;4|g2LGB;lQj!^|8+UO8j zUp)&eUgVH4dg^ezZiJ}mFA6*1sR*gxM z#I8gnt~={61J9t!ESwQ^d11T77BbkJLT@6-lUN^7=W8lpd3}K=Ks_uSdmP!ccix=n z(8FY6=6P~`Iy%zbw#x_TO@95Wrk`7_6A&O^C>#Y`dtl3Y>1Aay`WmH_QZ`T$J?DAn zfaG*pzoXsJfFlG+l@PEIS%}!LJuA}I4Ia*yme@#Jb-pu{XN1V{5Bu`t0*`iBod$7@ zpbVi_3k(AhqJ^#0#idd-36G}#V`Ug_AsX|tbs4$v1x(6xjdE0K@v6yaEIL;tA@L7M zd(mHn>BswA_MQ~qOxaV+2u~CfqVA|a%fd%5;M3R^)ZqAK6UCi=WfL1j1wZ%aV8>ZM zP)gN;o;sdZ&KIsIfgLKR9DlM?qSMzq$c-SgVU81^H~70~_S$2d1Sz+>gUih9Qz=7b z29C9wk8~ah4fx^7;s?58cpFZG_Y3L^(Q~d&udV$*xI_o|@u__k`cF zg;#=M+nPkycgl#Ee1WGbMmdGF-fb*zsHrWtri{o4vWukl^$*ZH6A9M1$BNaGs)#)d z|2+53CxC3$emdh8u6RGQ8{EX&VX!|#*Fg%2L?6BpyG|3$B889C5ie3;Cb;c0uAuwF zJq{^T?(gzO3Vk6ub~mn?3tXl58Wg+;&l1kktb|#|NICdn)s#4YAF0psm=)*O=|jmx zkJ915MJnK1R$|)t8c}x5wurej*>XX)8pMI#c$?GNjMd=Vs{M#f)*dQs%6n81UZv{4jsqZ*st$jv&;q-Lb za^1o%2(@@GYF~K?skc?axJKof(pZUUQ9n>sO?cn_IZQq5cxq-T?4^fH-*ZWKHq*L$ zX-BC@|Ld<(hZZ9GB!oQwenkRW_th#4H^#K9TiVL)C0QddLP|ydFSl>7TmJ{_U!)yU z-s?2GdxnY%y3~ENFC<)8^Cot_sqIggUcCG&TLv3i36bLZn0Sx2M8TI#4lt1RN|9<% z^Xil_Oa#NIcsm%QTeG0M9i*Uot9ezoN`uBE5n

    |WIMV1;PXcqGJs(sI8+&Vb)b4k#n*-O8Z zFKT_sm~#ChRC_cy*ykjw2_u7aWOk67k88GlBvO69?=50g-y>u%47Wyq2;qJnz8w{q z_til9h001-O?FjX;@dPz5UX)#-1iXEx)1^#@OP#7A$f6R=(%QS3mNS1N%aZU4BEQn zERy}OMoqTdjHNT;TDte5k|l(M zD+qriRb4>>*0>xqgSl^~lK010I$QA-2C_kyofIJS7&chOqhm#MHnuitReyMN3Z9kqvl(EgrE{|6CY8NSyV;A+&~0f!9dpZEd|mKcbiHB;{z;u7s(}?d5?Q+5 zjG-gs?plAJTWN`{Ft35i9knNo7^D_!Zd3Z*_gKl*R_;i}f{Z{Va$7=oUM~+ko?KK$70E$O#+9QXf zI3x#=QynjSQkfZzpiXv{pN+rGjje24_eL{%>>&&`g?U1@Sev+`c`Ik&0riK=)Z19_ zQa#$Pqb$YsuiY;niC=6NdLPeK+)<_@A&FJOFxU+AdSSHLm+HNGUt!}2wiBQMIz;(K z*HhFrErTwbw5_#Q969FAP-4Li3w&9Ww9R%mXefXt7HUKsMy2Hfi^0mk2MTtO%wm&}AsR3EJlm-qnHWnET04WHG8o3$sJ)BGkwX>wK8Lw9EoUg8 zy9CLD$byOsKaBy}7bSNW^--J_^F4zS5ghX4i%CL+=7vS2+G(5I^)E2Yp4_69xKsep z4LepOoV;@Wgp+K|a(}NOVrZrzN|z*|PPf8nFdlK%K9$K3n~#b~w&$H~*qoh0G={hx z&87bpI7UDr!gK=fADU6ESaPR-81MzVq#U&M*s>YvaV+cjl<8o5Dp{^f!Q2w^OH2Cf z<6L91t*O85Nn!}HPEhjA(yyI-T9cUiwx}tsrPe`RXbr0KoK>Sc;=?z&YW+{5PGH1y z%MKjqcgyt3h_uowAku@6aDHb$hZMoPB0=CnCkBtmhG`X znA~V{3QsX1NtM3RWloycU$;4CK@G7#yFayU>i`fe^C<~RqH#y^bN8>jujbF=s(<2e91na6zt(;_)BRx1*7jJyw%D zMjf|KzfUA-iY!f@0gj>8Pm*!Y493no;S_=4kzx8T=HHbMQG!o2bO`wp1qqQFbqdiXqe7GjJ12&FizYFqOS4vMu z1}`0kFthw!Wcd*q7)Mn+Kx(cOuc#lx>zdZHiPjv_Y$Bi2{k;oG+^wBzndUX{2QNvr zSv;|`7WfmDbB}e2KCN|9d|+hdabk&$jV0XZOK;zv#y?^!h>>f-_$M}TnLh8r)0=VZ#B)+A zX0(Svks@L;1GMteg-`LY(?{q=evi5a28fRE6VK+pyN2sS(#oZf>dVBuO?{I<(nUxJ z*sfgQ3Mlk=Bj0#|oiYx97)WlKw(PmwXlCQqaw!LH2N%OY=6S{w zna5Y()1$S=y9Jv#On#Vfb~$aI=5vnR^{uO;Q_E04tzw!oSGvh8>rY(&CPi2FleaP^4qwVi%vT2DpP?;HvfGlJ#9N zHYu-Zc2PzbrJkn0!n5Zxj14mjFuoMl`bB>hu!!%EC}0wulU@;^5@AF;r^dQ(ch-2^ z%rye+S=OejfeNGr!v$9M8*C@D%=?o^7Ep}zD#bv^fLAxlHhbWIW0RX=!UIP2KZrFe z^nNbm9ymUZssELH9(3H-#*H~G#cwsR?=Yih)6A<9FD+v3uBqR<6nyoB9KH5*YDs@9 zrkb;=CKxH19CV&_%qY3eK|Fn2Po(?e>;pT0Xx!qCth+pY2`@XZ#WOiN+6ey#5>Icl zhIZsTuyJ_#KD2JIvn6_tcD>OhH@U|B!w-o-C2Z91xafG6aVdb8;n}8FNDlYJYr`Pr zS6uFn7m-Miqjef+lMCQl?=v7bWqwH3v;R0uaYKVUE~>4C`3c8zG&}I>FeBS7Pda_N zVQQH1lIl9J|EbFU3e)<^Sc>uR87uRXx=k3B>|yE2`UaFZdd!Z8f9dRM3ovmiQ?T|6 z0NTd%-5JULq>}x<@vl8-~LSUaRfP(kOKf$(Y_JRZr}__Z<|p^xc< zZI)K684jh=;*0P0Qh5&s27{eG2TT~Vg!`sk0Vndt3S@lzN5rEP2&Km8@T<(pcXU)@ zU~y?U+E_7qd&FST+~}R=-ON$mTLJl=@dO9S?qd_PBy$w*F#jrEZ*n>Q_>5d13|%r( zMVa45+7a7s^vFnITy>qwk88Qv0Eb3g46L@m%}tA4yYk?9>+of~5UO27b`!0^D;ZU_ znaf<6!a(X7_}GEZxON}H;MEv|SW=P#q^370AR^GZ8`vcF-Mt%P=W%7x~*IM_cm&k?!$I-+1)Mf@a=dCL|K}yFhfjqk`{3zP-E~T0S^FLRPPlUSr78p7Z$l%RKiF9We-|dW4DSVY+la+UwP{Xm$bw$Ga^4La#2t0Sh4#*RkJ3V8;=75 z9|G}?kh6DjcH(DEmPIbw)ZcZcb>2fEej{xCq zn6qcxap>~R_nyP|wNqcvb9Xk}&3{%S5iorn z*-GpGaDdfiQM+y6&jQ_Gy6;F}n9rM(lmaG<{q9gdx&q6`k7U}WK_ zd(^e^`sS5FpYI)yYR-GDa8xM;>`a_q#AA}`7Qx>QGG#OjOuotxjN(%b+q{v4gvTm^ zhrk->B(H-$Ds85`bh!iCOuC-&8St44VLWjk3tYUYB-7#A``)q5Tig8F+O#E>!#gW`Zu*gv(K-BH|E9l5R*t~cGi$8jBXqndU| z9ApMwWRQiE#Wb<~Ld!R56i?BKFOeL#@=HTvIr?+Ql79@ukOFY-Jnxp?dtN{ITrPwe z;Va-5Hxerfvl*#}On`1MclTqJ24MkT_~u21--6FmTH^G62PqL&28zvaSAYhrbD^}- z;``R-#ZUrDLTLSMBaWc;tznB^;rCl$us{s&}?Klrdw2CPy$nQWrP z{UZX+AdKh6nWZtSeqzKi8AfB~wqx7lj!B@z|Cl9^%keW+%zh;c*f4^WQJ}F%2igOv%SN$=IBdT)YMSY7C>7N2IIDw;P37fd<7S zx??wvAsgX^P=~j}1t#WEa6e(0$}(F%3@;1i8@WnSSqrT6J3+F#I3+JPRdE199|U5* z_b0Hx_yU~OJ2e#DXheoYHKEKqP>f%uGFozWcvtLuoN#@^j4#GHd$c&L2~5?ZVYD-Y zJAKsH#j4k0JudgL5$B;Cp|XHJR5A+m{hEwc1>~RRr#DK{S*z>`Ob1@DW_cr&ApY zU+d3q$^~Eys|t?MjkL2pj2KIjyN>jeSffv_7Hdo}FVV(^4HeSuvvLy9nPvQ=;lw{m zn*EXiu%$VY&gWAL)HKiF+8+^J`T{6#5pMBux{!?BJ~}YW^uU$ZuCH_31o}VPm@Soi zuH@@+ac=g3UJk`wU86tBRpUu*=+cc^LGI2#P%=%n;*5Wb{RtEL+uDjU`qg$k>wY_P zO&2-fzOSzB5x09+PVhfY3<5|L1V)H&F@p(1r5$`pu+=r{vayk zMlh{Kt1QppV8E1B@D!2X53p665QSqiNwWZ+sk&MDJN-4T92#aV_#8SI%H$N&}L~xYr#Wi4N9nf0j;!q>;|k z`q&Az7*&ScXfVFm176C-E@8Q(p2~pF`wc9+wj5yte?tJL2zA5~jH+dMG$Z(4ND|2C zo>Lx02QKSjC=nAx}&|@ZYUxno394$wz-w zU-jBE{lB4X-AAtKBI~Xt8uKG~%P1Uq?~mI3ow7CZfV7pF?0M6P(}CB{XSAO;i}9S! z0dX(ZR@U8(pf=nMXlFLq4p!=%t+hW6Re}KSzEM#kGhE~co!3u#KVO~>CO~*2G~orQ zumlW2*fL|A7_I) zg><6*1<{%`Z9r%eBUAz=yMgZnjjEL!7E0SElUwYpmJi6F#X5hA1UxCQvJDc`Q$qr! zS^4dmIHw#^f!JMJxb-{7-5k$i0~OvwRodw*aKq=ZS8&w)ZhDm>)7I9B(L;fm7T{Hj_R2zk?KU zeptNL?GoP1AGr+LVK*2NEj`1Ppd)jc90)B!%HN=dV^Sx#>#xSr*d;c&P^z=2s zL*UyUtrzl8!fZ^JM$53PW|XxPj?qpH$EGEvg;nWX}Zgo zDVF!&2lrCH#ng#q}GW;$kTY@7%*61^;BQs9!Y7{CX=>EC~E zT&c2~S@wdUq!*V9Eu}6x9chw@=rH=GM6)s5fkCgXTC-22=x9SwuDq?|qHZcaJxDL42bdS}7a zVy;3KLi6Q(s761>@UP~fD`%w)u3cPxT25Z5>HO@d3CIyaF%r8$T6qV@*PJZ5&+na z1>1r}sF`#Lm&R z(0Pl}BxWdjcgkuqv}rp`t8)+T6h+cC_B-yq_^7E_D-T>~k_n=Ki3|GHF8mlfo@Q@ELWJ=SSD zZo$5UU|Zef+;sa*mXAsvjirBQ87ogEIz&Z})$QJ|;45I=65xUWBl(1S?c7MV;zDH|3`mFV1Tu zVex$*v*fu_7%=M~SuVX7W@k_GXntchND6=kC_u`ZqKQ)Y2zRCGXGk*@h z>862H^{ZfmMgq^QOuN6n7&`%vqB>g$-F~09eruVF&6 zHvdNnI5qJI-oz<*v(&ie0;1-07MYizBQXEJR8VX&07>`+3n8dy*sebmb=C1XY0-6x z<#77%+J8V95j~x*nJOXtS1mOa5KNam(6Kxl6kjG!Lj5<+aM<>14W-P`t+7-{OKBli z=gV34RnLeI?tCPFDNK+J3vu!S|3Sc&PvK*=zKpK)#W89>c!o7=f;QKsd9|b)kfB%3 zbo8d9<+DVx`iWU8j?Su;vL<&hJ#6X&V&$4(m_A=YB$n-|8kZgo09^+S*6A_M3Bkk| z^D*|HeRakd+D^{2zRQ~nt&6%UVn^xU6Q4m`^KEDAzCO|HO>hWVFTyuVreFAVd^Yl(xhC&MoAqh@wrkHZ_v|~5H>r=RyD|MR zM^QkE2&eF>5J0($ys;m!+UDp-TuiVj^G|z<>Pup za%UG8>|q`%7XxBjm^EcH9i^^<34e5|Q7l-1&+QjqddjB4n>!UC6C&c>Vm4zVP1F)72smV9WC1)ro*9 zTAJ_BjfsOJz@EhDszt~w)`c?(NHqaqtj@dG8G`C`1nN;cJ%praignfyebCxzChn(}RdZ}8_bkW1W&t$M$( zoIlMdo$TuLa+7yv!(dk>)hzsJ@PaGTkWVE3fAb<25KoTu43M44MBu;K22d0FSf#*U zCl#?!-Auu&gSMleK`9^6xDCmA2K{@8Q3gt zbi0!YRoRZp_Uj_@`FeZn0_1~G`s*$OsU8!|09&{kVQ9He*`%Any-Bb^)fUM%h@!3f z9raDzr&wY!3z?haj%=go@uGqPpO(Vbme!IdpIc@(G389~!)&qeP3ap%>!D3~CZP{| z0s1kdAv_H$kBKt}9oz9k?mUo&+V_X5Y`A#fQk#?edp9NaNw0ctHw~e>L#&rCVWy1A ziv^Xw?f-0V=eiX*geSy(z@2v=*p+}81Z~B^%zBe6J38D-BKG^4KCgU~Ridw-|sd0LJ{ z_&w;X&9A#e+-0?Pt-ls_hq2Qp2y76{>Hc%6N?i=W(xYBE0`=F| zTHLf?J%(G$?D{_bmwCpvVqc_n@ohg6Q3v&ZqwBsWah3-3K+W#I%z-DBt;>dfXG*OX z>2E7fvC4wcSOX)C>>KJ6?Uyx-+pj|xYj$e}F4xlLAWZn{oBQeY1=|}f%E9LxIKOvW z^#YiRzNt_R5}fu~_a`#ndI`Q~u%g@aE43n}5lQedR5IWbJIY}T=1#uC;%i>AqIHtT zsvi;G#E*vj(($S+^R;O`Zd#eglIV%JM~7S1oE`a%NLh3F&@F;|Na(#x zdaX%TJ^N|h)s~t9AD_0VkIOdM{KBzrlu3uv1!cXN?&*efJsYi;kq)^9f$l0n}6f*YYCgN_=1+_ToR++_JY9gaiyQ`VaO~WYF+mQtbJ8w`#uIHyYtYGyN6~5Caa`0lQ=ZiI zmnJW}ygk-oP>TIH@VkW9d#RwGpP#c=I#WvgnQRzZ@7}nSkfgxQlU46(x;)MEh{F!g zsD>j$CI6bYtc2TumG^w!TDkm<3W%&o{g+Et`ysgM8T|Waa+R9;R%!6H^Os~`Dxtp^oe@;x0&*)!py!#{H63}3r$Yu4%ZrPioVw$7LK#Lao`LKc3yC*^B9;uC zh3#V%kS-swk@iln!}vCJE%Ww;^?7T;tkzZWYN&AQ?u(0svfyr>+A$^0VlTn=Z^Qad zI#Orccztw~L!F`%p;6Tc=tm1kZdUTSn|y7WjkQA^ed+jRvdLp1DJk1nQ74gx8zE!F zYo2a&kx%Vo2+WVglu&2uBbBNWJj@ZTu_|QY%WbUlpG#!6Fp2|ZoJpk(CH90z{QdXe zxf5|Tb72M~+OB@hR~h)0%OL0}c?0b2xwuS_zBvU#pI6QxVm@a{S{uk&+;-24SGf22 zZ;;Niy!JW!=S7TpSe7$;%!e`$L(i<&ZqN&P>&*i+UF>a}^_=$S%=VK9FCX91uUba3 z{ilaoL(#x-A@chx(#mS1jwhCht=C%~^{o6XaofK?VsIo%$#bWHw@5$!h38wKk=vrir=TB16aF1MfVt-eZZz8&kbq$yx)akNR#Md{uG(H(S-QW6&Ujm?Nh9j_t5cSW< zw1qu{StFo6=-mefWQ~g(V{8>VxCq1Xx0H2dss8Da?&N02-@|m4_XY+ocCT(Hr}IYb z{>y|f`>u6Ml^fS1?nnI>oc-*017M!}RaoT1+f|6tddi2J{uDs9Ud8iYyT@AxZJQC- zk*ZCv8VVn<&`410k}^>s>=pE&jF;zygU9{@`$AkvvdnsFWZA#JVvPrWqDP(teko+3 zlo^jzJ1TZ$_>dq^W&Qeoi#2sY=bQB&;?C3D$|2L!wAxbcEx0aY+NgH_%QIG|ODUg# z65(Gn`Z}{M-)F$vf~B)?*~DB`s^UB*g|NvAr4RSQ){r00T*ceuv+Y(7B=Om~T+Y$P z!KM~%!XWvC^BG5q>dUL-s|FMS89SiM6}C149rT`z2etik5m?auAjd&ece}z*Q*iq5T2O^n&xEIzOc3W?)5`vcxx5;Z5YGC z#6H{Ul5}uq*Sl65_(DtcV1+eZ{>!q7NnVupd_3E?Vo> zWC)>QtNSjl6WHY6<6sPbZHzb98Hk#aMTA|%7Fcu_dxROtq6hk_@(zyCTfToTK-4@*q zJMOJ<-@ToBhdTzA=inV2YzqW@%0)3l!DjmOeWk%phi3$40X84m?6f1}-~+kdTWa@t zAP^K03j3zzq}mR<@Aw1hj6fyn=YW=l;ExTqOWSvID-NAvQM+>Q;cIEI@9zlR-~N0v z{}rxZ>XZ1?m^HMm6!7i?i$VZ2IlU6fFpVRF3~k4kN&5#EXo8}#{e)Hv!qbaQ-R7F( znyRsQOiD}?en^sn-2Qsfv(hOX;*Nd?hpfnmoe$0N{}o$#WXHQAM#^u?$sTO>@h|B2 zLe`w4hpHjU$%*ZN`INQ8lkCx1jtw!TQT`3ujpCTCF*+W@)b-3S-M9aTy|)aDGFsb) z0S5&U!2v|6L8Q|l1SCYHyL;#kLAq1AySrng8B&x+>5vprq@=rsca3}R=h@Hm?f>6# zyvO@bx#zxDU2$ILb*;68PnkPpH|eo;TfIkF?wXD&?ir}?km8a=lA!egcWpoRo*4|@ z%wsug|5mfwFld#)9_-asMRF&|7w&KD>~Jv3i3Pn@)mLYVr#*~j&K*dX9Vu2{4527_ zff)+fZ`)|c(c8PzpF<6%`IV^KV^MG#?4tkFnsED5zaFV_bwt)7sjk#s*u(gQIQG>* zakNsw;6LhZ777Z6sd`@>Dh2e}b=Q~-(%7Y{@sJXbL=yWk9z~aEN8g`&o6c=hqgt-! zHhOelmnH!C%nEV-SNms_cAUYc)#h{}EK z(Ok7rZ^gCRc-zl>*j0(iGotpy#NoD+#dKopiF_;wVLLMhSHGkbGCE%MJxU`+K^&}b z5mAWLb&PO<(0fO_(C`w$%->y9p?aNsYwb)^E(=cTaz6$&&3QY&6mAuHS83MvY^5ES zzl)|4AS?Tb@N8*rF1n+XSk1CpC-UptQ|-IWH;=dGRtq`D2XD9iSZMQTtkgk{HMf|Z z@>R_gIB7k#f2Z>3Sxel6kR^7ay>Mk;_Rutk`+8Oh_TDCEwpP_BFku{|Pj1_v&y^g` zCV88_m09YMOIQ-Rn=-!4^}NM>To}@JdDz)san0Nddzwt;jl?@ z6DH#%b525Pc(qLLUb@lTZ=wI@AA?l7RuFO$89t2b@rZ_y;h%TSVdJ0I^}&L^TxG)9 z@tbL5*1r~p=R0j3-0B{8(;}8vW`5VU?znzBJlV`Y+yTFlkd;NnAwa2~Cih2)$`&J@ zL)J$NIrYw7LBRMZw^hfhK1;~P+~8le0Ojprs{_R^&R@@F{v4KHaWt8*&m@u=h6&x> zl-!0*F751iO!h^;EP~>M>7CzDAD<)1IORIJ)9}|I@!9BhG`Hgx-g~RIi?aHFK-kST z2@l)z+>D(>lE<6t4tqor%zIP5d)3R?K88fuzwdcL=F_y)qY~8fz|yMlXOBW>b)0nc zcU&wmAvkcI)HnA==5lhNVAC&G6fp@*jH^fZn6OWhn)&MWl!E=qULw&4yW)x(|5x|r z$cX!NzK5!vPv({SlXY2)HXpCA1mKbNg{BUvQs5KAJa=CWlMQr_+O;>PZAZ(Is#~-q zld}u7^-t(4-4IC}FD(2y%T5R3Wt8s~UbjRZ4Dq*}mkF|$w1*Y1=Y`e2Tb1LG1B6t$ z_#3V6Okcrr!J=$uO+1p(=EqBW!mm+fM zHSe69)fPRK^*1Ht-_4oxi+)PAjt#yD)#NLB8am|^)o_3v=`<;(MXU}pQ|9yTRI;UT zoL^b*c`yWBD6DL!*?b;}Oh_6U)sS2zDk373e;Otwd>SStE!srU{BWKAUT+S$2}Qf$ zL-B?LUKAv6LTEmR0i{wrt$*ms#u>(wi}7hJnzbCNt?741#VcUV6X!#By6)>FU) zt;?2r)G}Ey7q0jOls27Jtm$}%<-g<1Q909Tmy$P3AArh?CmJ+#AYZH%hKVw(6BIJ- zNp)(bTj&Z3tK2_v)|i&AI%g)Rb~qYAYN;<=UbfybJcP$xh;yaA+S;iqxBMkeMFg1; zKMKzCKfMLd*=Tn5A%5gNxm}j)GZE$0_WMtdP}=X3z4ukGu+4Ih>JeTx4mvqr+a^gs zS@6j|o4@>-m2Yc_@IV&TtfBI+Y+3V~n$6DGM(0yR?Nv;%yOsH#kX&Glu`BOVY3s6{-yrIPLWXml`9;!1Hj`Mi>-^Xz38g zRlehp?YXYP1e-wjuS}-F63_dY?oqe!iRf|I9_XdhE1Rvs_@6=%(~h|cwSxu##=1FU;AXy@f8 z%PS(S^1)K!7Yt&dU*K->miSJm7u1y4|Ke)(?3OQb+X~fswMtSesNCM2u@kM#dh5!* z;#C3Btt*Qrr`~Rx3!iuRdWl6zI$)S9FS>?nO8dA7CY?d^68M+?aehx=OR_+fA|-_7!nApc9XCRf`JJj(4sV|Y{? z?*kxg_};2_7xHy^6OwKYbO-7jjbNypTonY;9z|=_hq6j>VNAnx0tcWem1`?fKdauc zc2LA_T#X4ynMFDo+W+Bn-iubJrO@>{n#@Yau58Jv{z&OE|Dz3qSeu7^1odu+xAzUp zxjM&>Pj{N-k)&2+&n*^^_A&4$pH$Fh?DRBkR?wO*!uNN?Em?CSwKruq{Niv$OMPfx z9Wo;y$z`tH3_`MIWPB@pm19H~I|%ci}C%PQ7v`F{-gsHv3q=>yU2l~oM zNt1RT*;%=d`2-A&_?Zom8Mms%i$yPP%)HQ`!p2|tH-5x$`BhrZdT4Fhi7k8$w#f$=HZnK zCQ6Ar75OBHtN+9(qDk!iFf#LMRC>>8tJ$RJK$qg`{Kj%}w%E$lI;^mE@*U%JZQ~*F z=1-YW?Pi8-uj84lpk(s=H8=J_oomu`ldTm&xu45Hl9E4ZBcE~}?-#?Y)kAfd8+|f< zaRif(pPo!!xc>$HZSYuJM8{1YXuPTftGS_zf2PkF<_!2j9*o zX>td5DB#O6g2h`Ew1?%Lx!+e4OV-2R%bhCnL47$(9H$hcDum#q@;C0cIXT&9zAb3I zgv&Zwd^uJR5Iqwf*B_r$T^WZNlm*Wr9Gzh5$g88T>I9jaO%}JK$B4-pO1?H^D2B*i zGa^X*{iWPW?VZ`NYL*m)@Yrk5j1;^$UE3>;aTDnP+nI^j{(70INY()`z_5DKc`H#W z=iA^5z~F`TlhH+*!UTo3M0#l%u=2XdGY+>bA0UL<^>a&HnM`K863$c9f!Q^=Nu*-bn_bXRZbV3_AE^q|$L>IjeP5-u~K2kqElTL;e*Me}?ftba+k)mucA<0E6H^e3z2 z>7H9`R^UT+`b3X~CNL%tS&w_&b zngfC(tj~ve-s?!;y&X&NI1-MtN1sD#4t+xBTqy|93WD>mih~268<5##Xbfrz7iEJF+vki z;TP8JF&A%7!Q>i;>7 zJm3>yOwaA971+?90J5Mq7%wl2BVm!WODEo!QZ-e8+sF9{-I)oV6=gp`vLE3=d6N+s zSNWcY`vwJ=O_vRXU|CARvM9TKF_jjK0Q`xt?LYd1oC|HkeMD_GV>MOrgJ2`Dj$slf z8QrBk^(Gr>%YGVQln??S6s8gylH&#fR5O=6`w9b#2$HhVml%Uq9!wZK)Ygz${U+M4 z$ku-BRey-zUmA)|-gLNuHw9lSNwA|u&xV`}CRR8d`1}!}ARsyjSld6@87V~&VqSIT ztmB8k?^zOuK_6lPVsoQ7Ev_O69oPe-m@;70pxr!ez{R6&SsRY9s+<2{_g&EL%YB*P zRgoBpra3UtZX3;{LYgB)f5!=u1cdEH4FuY_j|o8xg8Ny@2`YjPL40D=KJ_Nw%=et9 zc|>XzQC97BajYP%U5dJA`RH-h1Hp*0ap9VIHU{hYCi~g-+2CC+eCZPYHCW-#2#`Y? zv-*@uh?D&fLg~~-CJX>KAWru_i6~Mpp>?Drkn8;-XOKbA#}zC;Eus?9>k_BTvQ#0# z2xfWhAka@4hG&ePV zjzEMZ5kp5AbVNVQB1-j*oas|EP<-SkmGsQuGM1>&y?f0er{odQNU-WB9E0P?Cj&&i zoDXiP6i+&Y!n<4droG_ds9h~URj6x-A~xz$9ddmVhP0;tT%CO1@sbz}Z7eM2wl5l+ zIFk1j&cId4vr8Df@uOVFKLcigwc&W4?-%S|kRJMJG6cv&@Xe^Sq!ONCXw?8Nndf^h z47Os53_QOlIBQ=!&MhD0y?w9}YDBiIzU@U9byW2x^=dh}2VR50B6g`n{>XhYulq|& z2Z*wa5SwZ=iAfH*4z~RmYW3 z9KRI8Z-6KHHGxPW=5^@q3RZMVH*Q!a%~N-pV@M&tdMr_JYNNYpaK<Cl>lMgV4e#F@4YFm!m%v8Kshmrf~5SZA?^9-BY-C^($ zy;?eQZ%;t9$vkXH=#8$8m!`J#0^})jnvCoD0V<~Ih*KPwwO2mlL#H1mISeLm+)|*# zWMymH!29mEhO||Ap-5RFYs+O`p>_wLcLesNurx+kbpp)2pt=St1Bp`$3j&3l&>_>? z+u_#8q0IHx7gXanN~f+q56jvVGetkEjZ#8hnix}JP159`sHN}VPHZo#SGL9{5uC)& z)|o%r!y7H<-hzHfMBSr(c3~4K&gIQys*{Lya0`yo=Jto$CP$H#>6_36P_y7x;#O; zn<-gd8E7yg_lL(1CPFcrvli50lk}&D%-MDim_~#Kic6A+%2D42DI=W7sHoX_$y+glZ+?>-8Pk^0y#XCC7AQJ!l(|8Cts)-EcCzS!u&^u~QyggsL{wE(F)eG- zZK`h7~02*EcpZW5LG87E0XyEHwh3fWSkV zuK3-CO`0f6s4}$R^7q+nacBK~Qd+%!cL~%-TF(`{bRN57igJDct?Q~RV)=w#WIluQ z+0Vg(Q1^^~QzdMFE$xU5u%CG^48>iOEj`D~Q+KTlC&=H`NJ>*uxK!c`(JLIgbMo?B zyy>MA!77T9U0Eqy5c9feA#x>O%`y;CQjfEOo9N4@S9$%`Iq3~JR6CToZ9;$DaaLQz z=4EX^)4fi|6`#Q?`?V;^Z)3}zj>yLTZP#{qsCF8CsYTSb4ypQubaHhBN|#%95662_QB7n2F})%Sqo^d#_=ISv z%6Xhw^{YJ+$Mz(hPTnq;c1azI>pH1M4!I7+ZJ1)6Sh(_>($pz*#OsY$JRue4-h_UJ zf(O=lKnT-Tz1=Mz+%=dt*$P5>B9Vj=2&Jg-bSuTIxay&j_R@TNR#hKgMR|o-#*RN< zh4-IwRy_6v;f1`?aIzfiVk*n*Sxuh#_I5$}2k|eG{m32>2+VWv=v*3=?cxC zTj>!gUrp^=wi>KmTpE+3<`8x2m0F2M#p_X%%Z(G@HZzk-$1Lpz>%Ccpj@yQ{&Req; zEqdSHi%%eK2}s*^TpdffB$gGxK^o#TG{1#Fr4ZO~7ldLv>ywfduB9xO$>wdV$?s*V z9hNKkRz24)T|JTHwOTqL+oae|HSZ-neD&fcYdZ~xtkOpZ9Q}t%ciYUfSO!^S-E=y< z+jv$KH`5##JM?NGAgP+Yx@rY3o?4uyFu|q|4&TS87tgODjwkX+&HC;*E~}>{_1e3Y zGgHoDT{dIU(0tZ7E;GKT-luO&dh1W)M`aI-xh*5mY#WuM?p%qdO8?}ZnFg=1~( z)_8dJxkb}N=V9cY^-}$({bhYVC$s+c4m#IViBphZ z-#y9o%okjkA;C307yrZ)lZj0eS8^rmtJ!a-#c$tIPE4ChOH0LQnX|WYUkAUGE?->K z8y5k%wRHNsOd(FK&zA&(UH}DuK})*YN*`!B^i>k1^!ewFJlESZHu+J(9409Qr#Q)t zZ1S1pfnR-t*3ND!uMDXMClNqGcjCmib=2;(GKX664B< z4C*Y5frU%^@ddSCq*S8FW8rHYd^%i!iSNKIlm9C*K_yd@DeHABv`;VDOPEC}_JM9>N|K?CT8F?8ew zb)pnqt;57^EE6aQvT$R@A7S`u5bBp1v8IKJBuNRQULLNno)k7JjsJLwLmP{6^*FRr zDigd+jp-E{W-lCbdyX)LTKIZN3gjm?{4hSL3)DGKPTzxxInAG{wEZI;B?_KFg)snF zHWw`eN7x@mhvm{4fKH15oc|!VCx`I|QOtKN*;N`afN?N@qUQz#U;r<%PM?(Nq%S&6 z>lV>jyOp;rf zP6OK_qq#Djj%im=*XP*eK3k;mbDfDua#R2p+;#NVc78glt4Gh%+kwAnY;S#7W7A|O zYu@Y72bCJR+xoz=Ea536XeCW>vI@dQ?iRa=fUcXzrGq=Hj{tx{51V-n5y>6kk40g zpFaa9>1Tn6YF&A)-)3h)91AjJUs#>`TlR~1({F%m=5F41kDaz!UN*CNr_HIEfkYOY zqPnoq_~&~Mb|OTsL!JtT-NnS|3KWj>dLZdKMq@8VlZ@Bmyj8Q%;xd=Srv0b$7Oakv z-&sZUa<3Ghs(Tuu+2iVQ(R%-lh82q8JAucbKm7EP7}cpFT&$)gN4q$CwzJv z$o899G$g@aIDSYj;5_krINfahrsQkn>z|rtvb-b5$m{VQ0$5tjaL1@;CpJpf>Us-Q zx1e~6{~XwH!ktrn#%FCW5+W;GOa}z7#Wp<#c64T$8fi1vP{vDsw>@@BU#;j0B?R@iFCuyA$DcPM1Pw`KMyuOn1p>hcHH4nP zN@`g$uX|PhvaWwr#rLC2{P$aP40`74D+(?yF1v3#w^G}aG;L?kp1G{EYd5fRK9x7Z z3e|G~elejm{hU$6&JJDpg z0QQWWyf(^Y5F8g)x@PF8O{bXNp)Dk<$ zl1_O0%mO?{lyW&`$XwiG1;3b`{=(74=v4?Vhm(_URYult`RnPgQ(T9$x_r4hK0I{I z%L!!jA&dvaxLFNrS^4sLz})fbx=>c&k^2@*(i|Mdj#g&sj?cNv`misD}Rm&SuSAxBEZB^3Gfw7 zOl6HPQ+%(r@&g>y_e1`~J5&ySxlC%~={c`mSLMZ5L_Jsx>It-Vk;TbkHMHZ0FR&{w z*os!x^u}4PECzd1#-^khg2?S!?Pr^0Ry?O$78Bofh;rk~Lw6mw!3l$H&pIym2b21Z zTd(v!xuKLbRkw3oS2psRnJYA^4bvvmiEf#kB~B;|Up+3hk!k}*avdbAfi08w#{s_f z#s^vg4d{f9a#m@Fu5O%_J=;W<@Aq9#$#e>YlT5=@sATsJ z!~OZHJiBFwovY)_SI2g$F>U=z}Cw?(eT z5@itgYD%`ix8gAhSCcQj)*oJsye2@hwqDH+F<>#-VB;#Y%bu{r4p6$}r4lt9vS3PG zWa1P#*1=b%ImK}Gp!2jN>&Trkr8b=fdOR|X(KD%D4$pN@h6e0BkBfy`xD4mY^Zs0_ zbMd&j=w_f`-N2$TM-f#UN(l#$nKi)1nH8*InD{+M$CJEEP62lKNtthwZ#Iz$w_w)q z9BLw@`r`Ul_TRzN9B?IV?pDOB_Mw0sDoC4|$ld2+#KK5XFD-U&a)TtOgAzjZ^uM(0r zs*)`Sh6S##gt{)r|5XdH7gwW4aP^qU=8XUl?J-iw#o*T$-?fcT#C^$G&#y}qvXD2&R+0*# zs|=-}UWC9x1Ouy|8f|qCDZn7vpxK8J11B_M_uslBdmvO{$tL=N}55)mq+DK(;>{1CqnW^GHRNRGHO${mPPMgPmJ$PXQ=eNFY|lV6xRJH z)N5zsrEkBSt66r|H>&aGR*A_oT?Mg%s>pfFPcoM)oBisDdi;CyYcdN^_ehEoZUFC1 zg^5cMw7ojY;^aoBs>)hg*7G*cXtV(_{pnSF(jxZu4kUAYhhV3B2AzUlv96~q870zfe=(#40 zt|ctBl{kQq2m!osY5gJ&1B+~p_*p-~kqhR!>BP_X(pZ4oa=qu1?lwIGfWJm z0vA?`474T0jy$?2oflub7}xZ8kSX*vG(m@uUz4AAusT(P&t7tmWH`H_h@Kgq998>k ztf8V$XNHz4!KVxcmxIE2GFgq9ZLPAb=#0N9RewhwCScA%;Wn5hcvvK$+BL?* z&1YoKvHquq;EY_JKl2Rk=h>ti&l*_70G0jFW-CTbE3Ex5=Iq z`26f3?g8U>gMrU0B*ZrscqPv&r%|y9$N5xXj_o10)3b?_?^c5g;jo}>^V+YkKbEBh z=}xxwMKxsBEe4qlneQDP#@^u85VNA_SNG7PeyWpo$p{losE${Po5@5q{(V9m$IL+X3_3SJQ%B$49`;5I`kZuCOgR( zpK$4(tkxfSF1zP%7`u!e^epjlN&}=y$70Z+H7mc<(FFVc7Y+^6$v5j#75m!0*n`{O z)yTp)av3yi)ky`ucfVO@3CZ;MeK9UI<_?NH;$}me!Q`%>4Pi9QOgsY(?JBqbROiOt z(;jR2NS2x2xI~BPi zA+&{0$9dVKJx#LL=ToDQj%Mdy)%9GW0$V&=butRB=#f6Om6atR!cgD?YBM%tZ+ZvE z97EYWhA6<2wk#y*=GwOFhXwyR|GeWw*L=7SP7i)htVZEGT76!7<_8Q-TVn*<>Uzwz zi-awW{ZiV7DGm7)Dnr!ejLFG0zzq;G24S2al}ha8+6Y0SM_O809%q6Fv~+HJ-8HTE z`_KLgb+tbsKGxad?oDY@iV*BZ68I-s_RSCa^E&qac8&q|Ajjz0c~$dgV{7^?@h5?^@rk~Js z-WMBiJRfR=Jxujyr?xX++(uS;wrYKg?ST-}J=!v`J816La~YHvZJ;DYJdvJke~3el zk0{a!EkcC%!AFAL#5u-E_@ZLF)C%Q5voRpTi@})-rJ+_75zLrBM2f0bNhQwM*9~V0 zw07v*J{G^QIX$)4e&*w+tNg^-fyO*6pY_Y7nrZXz2HOR)GpR8ln(v~-gjj?)e!^p3 z4=CLYgT*aSE6G8^R?d*}LiH@vOoC|);2(%HM_t%!!G*4|rWb2cUGC7E+sLXoWuy{EpHu9BBZA#;u(Jru$5 z5*@@4%YM8z%v*~$B;MfgG=tfo^A{flK^l|4VDV{-t}3ckHN7s z9Oncx9AV~RBwLivZ^H-q`L=Gf;q4bqWLwt!TsPHT9LwU78zboY8hq#K@wz(g5h$k? zrMBguGaa|Qad9$N?sKJ1RD8L3Es(A}7$v=Es%G-rghwk15;rg8ppvW$hL{PJn6_vu zb>}>WEbS`9iiRf9Vjay3flcxgz}}~%q656Py@g-~7Q~UIoC~d8t>yD<>pz6*psylb zqDr8zUvTyJNDgimoIVGWGy267u&|%s;}@AU7& z`sf_ONvEo;=sy@n!XjgO3gXQ)C0v_ZC$ga*#r}~E4FQn@t*_9p22sQ&8-Lm_=obrGT&1pu_;>2Eva*r!$ZZC8+c&)B?2NiUAas->MBQhc6^??T<9qEzz7 zfW(Pl{9s{XlN4}(#+VF&q#=x@xM7&W{kfSgbK5)k^`mj<15uIWb z7vlI&V(EY}dxzCLy`DvR^mZ+jlDqC&c+v1Yn)0-G9RBW3Rt^XL`Ogj4xw!65^F`;K zmGVlXe(2+!4A-K!chcRqDat!|*st=>jr)LDOF)63($>>ErHTm)w$n;U5ZfmSj?=s` z?tP{(K&X=*+3leJRN6sc&nzW=#xd(z=&C_PJGa`ys?yWZK(D^Mw|*8uYXu}r+=uG%CU2#)Q7GFXZ}>`8OOvr=juXBFo(B(u^x2!g^K4j!qu zaU~=T%I+Y$*17rmJ{;*ku*8wR{0lngkV)R{WxoMa{3V_WoQ^AD$lwd%Bz5)6ne5r& zj(skVBgA!C-rHoKeKr2;aTl52uNEiG1aUihSnStfTs@-4f9(jhXLlVbfgFwxxl+JGoW48;TfuhI z5;^*R~ulEQ4l-UXB%U`2l z_qlAIs<#DP(7Lx>_*fr%x?@M5HTJihf4RCWZ=>o(DcXqrx&>&mY+*i>YR*lnwmPV>=%oCx{ybV{9tieG0&ci zCA&c&KbXW8 z58q@VO#J+mA*VE^#bxKXi(Q@M3=fb14(a3GPEnBOn%Gjb?_T*U2tdxo1?rwT1!6#c zeU{+pTqfhZ+O3g=Gnby$?B6~=NRt)$B|xr4WIt#kctJ79|9JZ$ zztThF^BvN!{;NpQgDscd{7b+XIBY6#13$nD>;TDyH7?6=Nx}2O9#md2&wE|d{k!Sr zCS&ERD8+|)1-CJAL}LNYA=VCh79=LDJVonSp;=3>61<*E5)(*|E8*#|JpF6vp_&7( z9C8e&2hY~-d;7xq`MJz}BX)Dbk{HJy;iw;48F0{CKje_+bvww*P^`W+?sdG55862B zae;AbS|^hCwYd5lJmnDV-8ZB!1m%l=H z1q8EAtT3$#&4S1K-tl~$v+$#Ga{y-c+6^XJd%Y|~hxnpBf#5A8Lxo0u1y| zx9Crhk7Q%x=HxTH(Hk`*AuYXF({ttuNsJ8L-%pTDB83>O7H0>QA z4E21ll4jX-8Q&;(X87|~gZH2}DiJY*Ms~_%VyjtBpE=}bc+&Cr1!c>%zR0g?oYjLp zIs?iyxBlXe?o*FQY-#k4*jl0bY1EIF<4f3ebt2zFZ0kGiDb+Z)syY@V2e46b))M5vKi|d7 zm}4krE0qKr)eY+GfA%z=iNNnd{`d>)fy7g*`3p2^3<21*d+RgH-Hsx%D2f+LydS1H zduDE)9mgF_kDzYU3QS&*(YmQ8pH+}E1rjX-_*)_<36{c11Z17~EY{pB=>;P;>Q_o# zo`S-+ZUvgV0Ajdc*&iFw4zlr9axlEuiK)T=K{tU#sKL?SORP99;Kl^g;P~C^ z6@C@x$#8721bCGPdR#r*8;uJa~lFob>q)`XZ-F z?ZzR1kl$mwrszxJ0gpogdfmr`i+wvDbn#Pxq?RPePfBQoE5U%IlVXNq8~98ra!le7 zN?`eYxDbP;D4X>7ofnv_Z$D27M6sMg0iqWCoT(0;_p=YoUcFq1oUS)on2- zQ@pT-p=!0)V{7T!XXV4|e*=A4j2~80eJ>u#K@M}K9N9Jht>W+lHW2OLk7-<1FV%0R z*=qbb?0&-95%!fuke11!4a!(36iC-J$?|Rk9}{OH3S(%?sy8d6T=?VJlQ@DK|0^9z zG}3FURGP3(;Ee^@$n?EjB)Pz3oHhe4@2V^pMHs??$=bd==bX@r140O9EU4l}5-5YT zP0tE@O??cRhb`5*RTo->@eOB^d*G;GV3`npYqB>|G?PAbm@+ub%!>hsHxPzU`>AlJ ze--mOf`wK0vOjLxP7WC_q^E-rM0Gt@(80w!)7J z4OQ&C%-SYc^?!bpWHfmQIgFWp-7?=gk7DU%!u?e>~<{OJ8a9~=Za_yjImMlJ@xx?*`?MS0_T1EW|93uDVi2=S(Q$>iO}> z4siaX;v0u#d9y&!0BxHRs8(L0AIh5AyRYk?|OUpF7x z0VYl_|8k(QA^d*8aP^k^U*1n7`?WcE9mFrY6~~o|*O_c)Ew9nnY346HQFTkV#nwGf z>OpI}ZVSn|YMV}n!&SuyF9jO86KZ0emw~D?Nrn%=DplM^OW!N~t=ak+5P|vv+SpfE z6LP%0P>u*NOXh&Cxhp-fxob`XW3kv=egyL8rXD4w@48_NwvgdCc z9gS#eq+ZD;!Nd>A?C{sjlrYAj4aD%GeuOW{1xC#TL2m-K*`!*faF7rEcoRqzF=k}^ zWU%qm-{-1|3{aqQzQTbZS@EXHo>8j_a)>d#I^-4-JeWAK{6^Tzx3O{0^0^cqpoZwF z2Z=-VZf{OizP@CRhZK94-!jMVk^UZF%jI9a*Y1*`KOIP3VI~#?XWi7#z;rl-@%; zuj=Dz?y5;Z4fvMd^b5;5%uWYU)#`#dkylr%miU?wO(mKPo;J-nHWb_vun zSnf$;ok*__a?sX!*nacLkKfc1AvQSQ3WO`XE9^g<-_9dd2eTI=iiU30s-?=Wu-u#Y ziEK4LHz8h;yo|*m48g_3)h$)A92>^ZQxZ{ zhBD=#m;{z24K+-{{i3@rgtp$&VRM-jsUofEYOz!%bO&MrkD-A=5c~< z;&Hw=dm9&=(|Itas5R}nevymbxPcG$W}e+PAg8KtrJ^0PBxK2zzM+x$#;^2tY-_y& zBMxnxZ&rXkVLOHA$^4MnLL|m-i9K>NtmdJ5NI;bx=wn7X9GC(Ozvf3VbYO+uaINPW z*nl$5{#k7EXnWE4-P>d8OU*3eBlk;7H-R<22`&k=<=m||v(kk9|?t$#wIJz7A?4Lj#g{O8BmU~D89T?YT5Sv17~ z1tfx5i=yR!evICeej8)?$$!nu|3G3df%J6OE--Ta=f{{pJ+LwCAN=pp{_Qt)7Mix4 zu}vWRkB@UOfX~A6eeti^|HmZ%Yvu^$!Ke4Ed87dL`Pbk6ZDJLC!CGJnQ@sD5Z~gNZ z|3%3D=f{k~7(dGY-!lLAk_!J{zs!8+c}oAP1^9OX|G!VOf6>K!HI>CfpH2b>r?3Md z+oZ%eLSwjpsgLj&PJvQsfokaaKNU-g24Hc?Z!gerCqJz#xG&=jUSo}%D?ejEo(S7Kmf{C{=sf3%+`2aM(ho9Nkpehm7( zs|w}*>m2?yxc`1X1f3r(X(U_$G|+!|pD`3zkpDO6e_yo!UspQk=>~Rz=l_r}VH`+} z=3qqB?6;8^6Y*d+enHb2nVy=|3X__L(p>dg@63w1>g(N7O=>lz3zZ%j%Rv9}y5B5F zC6`fpIQ+Z$7hb`USRDBNP7Vyj9VHM>m3|rv7XG0L%11h}T#=%k#@! zq5Yi5)`san8rfWCJr$+-<7w=kjV80Jv!N)XhU>}ctu3Qoo43~UJvExOUKf9klHoOt zaFEhE=;ycI{uDIH{Os(X-(?1wrg9n$N#cLky-9{A+%NQlol#%(E_yp^iCSQd*6?hu zh-mG26WJTrNkswNF1C$MIv4i*oZxqGw;8dX=Sgyf6;DmLxdlyc*tJ^>TYG)vyU$j0 z;=Rl*U1*P|bCpCZ{&z4q;Xp4mXeDp`dy8DOPd;4Zd#vxY+jg7|HCq1~c_bYfq493E z;>{;jW|LIM0c*!Evp3~5lwG1TFGAAmJx^myzOu$k&A4(K1uxFqT{!R5 zx#)H*&bG8zX|UA%bAb}X!6?D8UjOdv{nDU)QPFchNVDpb+nuu8s<#Zt1;)%Q2DJhh z3=6GUt&#>z1q@gcxydEDzU`*lrBJ@qKbW1h{*&i23ogdAV2Glul~Gfsdd?m9l$?^P z;mc`?dShteT7ObHZ`Dt~Pkg!lcTt2R(3q%ckCE@c&5Cd)QJ?Xjky`ngT5VJd<>?OsR*(Gx zu9Wtk7oBfxTY8>7taY4y_K-=RiJY9g-mf^jE>`_S!_=|s_no`5V^jA%-Zgd0-MO1a zTjS_FpO%(Zk&B#>Ui;M_e>|ZqKNCP8Xg(}--J1Er{?p;4ICi&{g~eVwMryyYOn4NV zPSeQZHR@yz6q=hQ>$#Vbn&qkdxZ>zU@2WUHLij9D`CQubJFdkfpA-uoF$E%apTt8! z*ZnlmrrGI0(e>Yi*qx4A8qw%1U@&Zb=CieAbZwZL`^ZYBnEnlieJm$e7T2Jwck7ol zQjUhh)ibHTy2!RXY`!u&@@l=v2#)G4?ZD*tVN zW$?94l?t;-I9gU)R|Oi(hN(Q3Lm6I%KTiz!r_CqC4CJV~@!x|0NIgQBJH9$#_)mk` z&c$>7-Je@Y4+SkEhrVp8=)TO>1;^~vEslJ%W?R*@{HeQeJoOWyC8qq4&Qtb*t(&2` zxY4Y88kvow|)+8TFbB5Lf=6$2psd8*0UCfV5w^QMASel(m!TpY}{qryiV_N z`XhL25RR&05VJ@tsHhdZm>G6A)YNDiU$0}+tm&Gn(%sa|yIQ&>gu`Mn28awW{eb}T zOG&&|!vF7{`<*`;X*{9x*iJv0vl5y$Ws`*l zO1F~IC;|e~og1augmkBYh=6o=cgLnN=#cJEx_i^`t&O+f^S;kG;~VGCGmd{Sws7sW z*0tuGzu9XA7<5EeJFj+NPZ;O5M~o%86%23nSn;v(TZoG_t!-{bk`j}ZT%#3V+iR3} z*_qAsPbJCiq%}MYdae~btxVSu!`;nNe`3wIKW-yX53%MwFG)D2Esg)`_QWr!9N)M0Xgy531$UWAR#$eIs`%SkZtA3)_^EeSNx9cjIr>|in)KLfO8Idv z6dgoM<5O8LVSdK24lghXK0-;wYzC+XguHdqd4;*6eTUlq>~KL-RW)LP={!tTr$`mz zX186L+X)&Q3^yE``Eq~sH@&%mJ=!NLWG*RkxUv$=TToJB!zOS@#rJyJliuQCOV)XB znpCaz%J9}qv-DY;RvbsC_13TN@o!G2VP+Set}q9}cTORtL%RYzBvcHXyRN6oB#wqY z2IAso<0XS^QYJcjO*D0jj&0lRQkk$;XBKU9sV<%=v3Tb%GV%N-BS}x7Kn$~#Mw{4X zbcn5=3$3cVv>vtR=G`}lTgw|%NZ(9jWjs4s5ZWgQ6GsV37m)O{ezIcyvEJ?2{y4+q zh!*rh?ADPv?N2zke2EBXrLQ}A>vn!TRY3@G?b(^LTgj^!T@c)d)qswY#Y_SCupWG% zz1y;I<1w9`4rSz0Zqi;sWf?rQ_Ghoyn;yD9Ov zX&YspJe>p$u&ZNJXXUiRCJYOfr-PRFU18VfK)sKrFe*?1qJ{Ew2pqk%0jz-mVB5lb zbW7b*s$?(j9-OJPH|vwXeqa2DfkaluXmzMjMqJVMM7i;#E6LblrGLWXmsv%3r2GRl zj=ouSKYh`6kJ{NkUW3Cx0rK9=^0}rIw1l%X&R#7chv^+$g5s#lp>9T9LgMk!4;#3wT(gk{{BviG|F(T`ys84 zuOhy&sR!5Y#BJ;{RmD3m)4Kr8RPPp%W@n^30*nB+dtVP7mHibPkc%0nv$x@+AGgjkm|r6fxjf*_6(;*Kv_Ahqw|NE4x+8%>0qb$LY7a9#5nAcGMmBY3brMn{zCF?A zn?2CkemwC*G$`~TbTHyhpLDndzj>5h)os`ie@@ZQmn$0^`){e$HBTp|PUB8@Gla@O zvs?9iG;}p2;UtwAQs*j(Qe4@qO3}@?!LM2L>U*f(=#Q=PaxT7|4))r|l2RxGFP?-O zUb}W}Qk8mYzi!_tISMYpal9jRZqxm0#bxR^p0-i~FNCf_w-KcpJ_;3RFBWitJ9(L<)XB_i7xHqmwXGo4x>Qtn$hZbv^HIrbq-kFa}pAWs7@DXQb7qrM)t>n%t+Usp!`@}eD z*%@_bp?wDk@!ytAayQ z54y5+vXd*((i=g-NxJ1r8^1{)TDzF+$1kM8J?WT2OUFa5?i75}uHs&AyYmYbEty?~ z>o{402aVXHGl4=DQz!G>wM!pm1rJk={L^Q${nEKj8B*RBEF^eLmfYIGgop>++lrl& z@~j3I0cNC{-3-zYIlYx#nk2p^qOKQ+5lQi<7y13lKPcD?vqauU+lDrIM7GsDhGm#@>u+M%pKI zF2Xx@SZY$bHB`@o(~u4*@No3R%?lenq{V8+?4K3LhBA(rKY{ zk**=o}z9-$T(;QEM}j~3O2uk} z72nyB4#_g#E9a6pX=ybF6OEbYGa(P# zjb2Ea-)6>~B>_)-#9Tz|m-FwClXnZVXNa(p`T&C3l80ckN;%1_iq8%~`SqHc zngnR!=1ZkzsfzK#X-^+-F)|84{5!)1=kV$>L9XSMNuSbFb+_81wd!GXV} z>sN$P!9D(3q)0-UDgGOSF9aM=X%bm{wteHK-u4ctn>#8f2WP8xXAH4-^Up{pxc+=c z$Paz!>vWM0yZzw?h{rgV;--Pxtv`YT&H_rIl^vb-#pPn-vY-_Y81ag5w^$R18~jXE zwt5hW^-thvPvc7xAx>&m%9r8+tJgZjN`lK2xVYoWVgA9*9q;v}xdiVhLYz9(+IE$j z`=Ldo3 zi-&A|c+YRT{QD-@AnDnhkw*s-pqD5XhhuE9Q5c?Y(VkB8Hh+dP#s2l{;p<{etKV88 zvHXqrt)v92QYBg;5&F3|&vl6UIGW+_OZPZ9HN?NtpX*q^n)+NG@cz2y_~0u)4b2yV z4e#d&bDs#-Sz|)<^z`Dthep3bv@?2lB98AE(s9W{oG2R~STNHZI0k&&MRoVJsd}Ni z6S!fswmK-0sO)Y6J><9UJ=O&!f#txAbw14chuzom%;t0@+WYVuu54u6{@H%LPzL(K zuLO^?L%3A^uGFOJgSmbk!NL5RZOH;bmRt1~R>^HQ5T)#~q?AOzgfj}c1ytg(vAmrV zzdXAo_4o$P$MHp{Un${(t@5(F#6$@P8Z+O{Gv2aFYj()EHWrmwed*0nR?%7VW#uh* z6s+qV9Q~m?!Q5G%Y!eVSHekT)FU=KeQE2A;iXoM}_H8~m>ZO}O>ewJEt>D2tPEr?e zBdSl~V{0?y=amb0&+;lJi|bFv>h}oFBk;lkPE46~YKC!BK@J`2alY@dp{^nEy4Ut= zf%{$#{FfiAbKMsXY551k*zTLxWYPN$aXL6U6;$*@L`3WlHeYTGLF{i@OdSnYtY!$E zW=%NG_?Lq&$^xyk$7goZouSIZ_^Mhz-H;W+bvA%qL&My6Vx@i&6ZTrfF$R?$Gfj&G zfx(EZv{GvWsr}dgZ*OrTZ1_8(IyV2UW22U&lnrBSD z>d|d<8@o7lrJzyNy}Ai_*tigA=s*P1*Y`~&G(XK=GeJe7a^vnL6yhk7ndjvVj_?dQ zv#Oh5x-UsWjZLCv9ae`QxvUm7uf|NBq^}JZebu*1%X)+JB|0X?{Osf~-g7RUX<7AG zPb`lOKkYtFss3RZhy#S65BBxxf!@y@rfe!Li66bC&u?nsWKqcpY`0LtPd7ZK{6JSE zfhNw^*&L}?BRmVSdLWFj{fyPux){%8E=-jsuPHPk_uCop0=^?M@#-#avy|jTvTn4RNk@-@7u=)IT}cbm9MTT`)NN=8Rrv?4uJf?GpGFNG7xkDgu# zXBmg4BKRK{r@EqzUs+kHvy_x-V5fR<|LB6P^8T&{S2Q%?^LH8DL z;P*mQbj;1o5pg1?TRp387QBJH)6ZL?hTew#A5~Cx9qM)vA%CaU~hk&(7_v_O{Q<) zM5m>jx&^iH=!F^`!RlGaYI=+X+u>5M8du2TTt4@JvUhyCK(;H+!9~iN;x;QvmKjU+ ztmA-J?}btP8}$1;}Q^9Lb(^9-NUH>A=Wi;ukdDNvG zPDIW}t;^|LFO#{=l&=tdnt4w1EvFUY;Vqgk^*9(iU*SzUpD8__hK2?|BdfYI+8d_! z(hfYTItj;i>bg!_ojSeZdm#-u1xjt(e!sDM3t^MibESOyBfcO~X(vN5@IzM`?O7tP zxRU+e+}Mhb&`HvQ?2Eofeo@5g_UTl=zncWYfEc9oH~*p%__o!9!~(@O*{IlI=<+%m zp_Z=D*;e3MD{Vq4w(In@$-2h_KE)T`XfOfDnxS5J{R&vYxc&E$dHT*ngbr~L?qUsn z)HXfMqZhY!h6*4;jP<&99|9Q=2F(W!U=uv&#(wvoIT{9O;ahwAt0OaRH+$#ofXW^L zo~-=FwOi|qVe5DQMRde51Y0JIlAz-G-(Wdn)0I3Mz!?y06&}YG0tq?@cmxrWG4SRu zLk_5Y4*ykttQU1`O%(^B_bw4n7+j*WTUL$FsN#4Q2w+$<=hbIcaA3(pE-pjyK(aDQ z@qoxd&Vizeii&Afs`jqP--D-RRvC>&$LW>_q|PvL7fYS;?u*V!qVr=>DE(l>x^9 zHFiNAQwKQO$3D=n9Df}|wtqYhx{aDGD*SXDoEJ}9Fg3lWooD?SbPynU(SZot$KT!g z|7wKWlqeFn>!`-*r&E~32+2g5qmhOjdNMx%bxIY91hISZdPV_sD*e$P;P3#dVc>y} z^A&+3eEHv~a)Cw=Mj1?l#lflNWXMoiTuw|W6v{=V>84e)Oi7pOAbYw!FhtG8d%csuC~ zbzBCuQFjIFD$)O<&e=J9eU~SHrDo@#VBdlR9!K#UX&hBDweoq9K`X{h)*{-F(6y*m zDt1NsVC5ad?D=;)gr5&FgKIh%6|$mz+rvK2eTy-ZpSV;pV#ZgcouBdi@r85@H=Ruz(y3|*rwL{d43OtSlSH3Bn&wC zxKf7SJffzZ`M8=bR|-?8D}-!2Zl}ceqadT+7Dn>2Lw`5}B{kOaMnXZS@I?HOQ$s25 zV2oI-E;OwZE$ft&GzyK69(b`)Np_m8SZnE?>Jl;8vsPc198X?T&Cx z!yF!eplj(;`l#_e7zghSQtO&uQiKWR<-oHjs`*;<)?Td939SrqFBmzaE9m5QsM)&(c>>x2ur# zOx{emt$`p&CB*Z*+}eY@?lD9>#Vi7A&CsPvrfF@W2$3~i~!(mCyGWoO6*esrAP>HH@E z>a72?^^9J0Y(>e*HBfSYpbbK12v_wIR06e@D&pckAI;71j2-07QEv}>B6$^HO~KrrLD7HPMJsk=clnm}13s&Z9=R?=H<86YR;hOj-I5?kCi2qq=4fYts&hD#5i< zzErp4PiSWeJL#`82LyZ1y?mFQ!Z4wkR|FLxJ}-r^*QP`-lMZ&Yq~g?ud?YnjTH2{r zs;gHaBZt*vY#kOCT`*Xsw&y*F#m37~pFgZ?79?!J6?QrU0}*IH$)Dv={O1A9oSR<* z2hJy%(`F>nf+Y_51)W6PMPst@M>X@SLd>@ZMDdHL{a&7HO`DpG=9{UE*s5&(q?aDQ z*FS7o#bE^%Zf&isaW6o@zw-{RWIB{(A4Zm|n3?`;o2j-{1KNG~TWHMkk=X0FhC-_E zUx?7fkHQp`sNq~gav6G_Fgb=0l|-u4?Qo5-(*DfV*C~=3IZC~1XS)<(&6I&tDbGb< zPY2RK8Gq9Ly2&q%1R58gu)FaJX2wsWepV*4a%FenOmF1H$8d^mACtOWr@Mz>gXtnb ziDm*UV zN9`n=45J!(y2{Ve^6ZGu)&!`lnm8tfa+hv=jV2>gO}5lhX06BuSB0SEn88(xJ`h^) z`B}N*Sy;;3px#z10F&>vS&?3`!6c^Ij~umamrj{(27wp_G4;a}LE6pPF!7{O1yQ(? z$?91o&HHra5tTgSJ>GC?=Tu>XXUeQ=@|i_STh)5KtQK@zuN8cYO#IzWgj=_W-sL23 zN4%fI3f^a&y7Byxf1v9I<}#qQCav%^bj1wDemQf)Elj+xFA8RD74+OvqE{2-xZ>nb z8=VhN+>O43b)L1QvP*tfB(CMlEozik-XF-WbBkcuS7|5LITpHM29Kbw4eXC-S0(BF zR;Gh>zFPijD-!2xUsu9_$b-$YuYu=^eys1ewv?U^`V#MIUSaN*PrXSX+R=KvMIT*ZPv^Eao3<@Wl zDtTe;%12vqzQt*?Fb~%>vDw!+Bg-MJVCaBAX;~Bl4`A=r&^tb?gx+U-D%?N zC%>pUC93p=C=zdw^leTcQ_~u9TGwmrtx0DKsO1Rjufgi|kqe7I<3y$Q`N=qCm5K(d ztSa--a^VlX$@g(S&>gN7+gBE%` z(+L#sH1{8^_K2tv3>5RB;pF1amwTMrG_Qf??ROp(B4gC=R;?QKLX2}yIXjod21Sd# z&uY&eJNx|Xb8tyXb|y<-yj!)BB81+fVIh=fm^(MCHL{ICQdtpl-$Qq+wpl!#ga0uD z)T;eVTlOC0&I9aQmFBLq$n?rU3G(Bqog|DpsXfVld$~<=S}5TTi@QsGF}=n^a}V3} z_comisyysui-PFWew9e3^>yXuJa>AAwN^cgeIwRHiYu_q`_YJ!MqQ$;bhATFX_8W{ zYBlHUnIsyrk<|ivX?Os%xL#hGbxpaHfg#YtTnevbGS{${enda>1w-EK?x$iR&EI`Pv+Q=CeI- zCr@G+CDU<_-Utb%8LNNGYOWTlO&?2uFTOmdq;0`xX6(rqz;a$AY2NFFb~EB(WUi>x zyTh?JOdK@x>P{g#s&0z@HJFAi+8*Zt`6unirWSdMGq5Sf%!Y`Srsj~qdoBoVb?4yM z{VR>x!srEZV3{c?B6)dvye{zEt^}d_^&=RwPY^TZy=tLc-^n|))A)HzicWUV%8sGR zb=4Yn=62gRYPOvgyj9_b&9KD?JO=En{NyJ=*&i69q5V5gGSaWpa`teiFZD#y+@R^V zH7m&0$idG>?`CCnC9Z33J9-dRhgPRhGyRl+Jo);oivJ7hgKC99=&NJN?LE+D6^rNDqq)$1(r#A zZfs}`>7IXXW01n9znN9xA{@tkygSM*lUbf*>OL6J33uMu<~)VB(V9*|3XDyjX;--H zg^+9Y_|zXMdCAFPNh@Q7z2)HI$jaGpO|M%=rha8r3-1gmjx}@;+K}Hk_2E5hdpW?N z)Vi|AV@^yO{o}a)=o$IRwjwF-HhEp7$v#KX3D0`@R*+C}KPocnwuEPcs^H#ryC;|0 zAd;}K;*n0}>&C;ePZYx^;K~9inlLMs+s>{!Y3xL3r%yO*3pLBciIwYep8ov!p~RJy zJ;XX3g;!{P!Ck<@)l`r`coqjz(giThT1j*F7t4$qlKvZp@Z%*6`-rs22-SIdoV>zF_ zR(smLkz*noCn-s}eaJ6qq_Dh3{DZsxI2+fom?DDN;pABAJU_nrTGiPZ^XcTi&Oj2S zOTSyjRFvDm*)Kump$NXl>ej8u$!b?^cHhW=no`JFqO8z)Yr)=6KJL+~@IPjk#&R_&i5v z>2dl{{a)~iV7-_)4gNo91MaJy4B07F@O5dUrIgOy;=Gl?YkT|uWY@4|;}($FhpV~o zPBMq7hCY5$0KQlpFJvWrBi^#7wcy?$`H6(jXkf*X`+~^nYV!s?j(_HLb*@D@Y2+TG=yZ0*kpOsX5M=S;g>K^t}b*nnL{0K2;HUMsr-2S+il zId9!^OIV(q&0X~d#z#@F4fX6!xAQG(4x=p2W)-CcV#8UdLzr1hZP=D_4MCgaj`)z> z!a@bdAbYwkTi%GRg1TFfQM(Es%TBszOq#NWYI;+Ow5iMKX9iP-up(cXP_Wg`8lc{kHi zElYnC*1xi~G&dBls;#$o(!o|gl`%gkMtfi+vAHJw=|;lT3RQZbbk`!crAO|FqeAO5 z$pkk?@sOE5-tFG|3&fb-Si84!>R>WAAja}7t-f5UQtT$9ngxV1t2Tz76IGL0BaTnI zYlr7MWLDN(GxNvXBs18E$>V?QAKX$kgs}8lVD&ZM=uFN^t6+`q@zWouGLyNboophNWPTN=zOx>`Hd*NG^_JB;o z^32qDKe*&vtNvVEPe#DPm7fsrbeUV0;oeOK!8o}Gf;qY^3#b+In9+<~hxG$?%Ei;s zaS^f7Sthczw?-A(s@=M*#id-QT+V~Y#@8IerPeL76m?t2$Oy+-Eb9rD4_Y+k7=Wkh zL@zwe$XfkQ^3ryHNBw$z97bBO*-H^g{k>xL;9cd!Nk%uvrd7hPuaDAjiNXfMvVRV< zg!PN!lf->iwP`11Z~b&~r=B>Fb7_**GUr^QAR026^NeN=)s4wMR3bgLf|=Kw(K{orOOiu9=R*zlYf2f=6Bf~Hms)SK#7?Bd0Q><_lS~!7Eba?$cG*GRXV3Qv*IH|sVFbiXixOx+!s@`jam@g0s?yW;;k z7?xmf6GQU)f=jWf?C2)u|HnJ_A~7rjKZFs2`~FN)e2LZaF&l?PPv2+wvNN$*NOZY*x~9E$P^ zl`#st8#OrHZP`k)nPcmb=gL^)^kXIp^Gt1eMIA+l2UpXchz^UQ!+CZ6LQ-j)ppxRmvNoraXZC^gbf`(z8l84 zVv=y?DLZva4(ZDv2N9p{s)4?djOZ$BQr^*Q-G|dunN!u45%EG!!a}E8)ki+($Kk*b zcV5a7$M;1Y=-ZP6<7&H|Z3%=uqde+LaF-5Z_14&{Hnm-q^08f+IX)T@Iun~o331+% z@+qq>v*=kZ%@}wD?l%CENh`7p0#b*pKr+vdKM-H?Av)6?HRj>6&Hkq`OJORPXF!>d`v(@(qtuU|6tj0P~oROlo<3VFJVF4aU2#UH>pjQCRxJ zjlqCYmb`=~#}A6myti+Wne;R+XzotV1#$|8{9q()R@PSKsmCiy&Q>aLpQ(I*T`Uf1 zWJ>$Uq1NRP7www6KR>Z5q&rcvJ>}@@xxJ(~gm;mtw)FaqugIkCRtfeSc>@`$mN}(t z*$V;d%1#DL-mZFnCg7s@q<}zIhctI|iqD{&ux>j?C==_-S!P2aSS%LOtgC$Jbjp@d z^-7gx)1ai#TIzP(dnf*Fv{Mxo{h=T*$1F>IqX-G zf<^kGm+q-_<{T@7fK}smA^RJ%`_W-WUIoe(FI2MB6nuZue|KrZCy!U4iIH!ZQEg}y zE@5!|kK`1CPE4Cz0f#g`n67 zUPDfHg1*%kPtSs32CiaejfFusW}9&B(yiEmep% zyq8ol^UDhCA($$h90SW?0eaD~A9yT3myYPp;mdP!4ACjD?Ta-(=SV$2psmkwG*gjg zF56)*x2_nU&?pc(giI;W;m8?Ck0j5~Lfmv*M(ye2WZEa;cmF}Nj^IgvyLcSy7|{$f zW&}cQ9iNu?rb;HiR31`({3K8A@xim;l5rB&YhH;V=Q%|%nL29r1p@rg1tAPO*I6FM z0TJLs1LzAgB7JK2Z;%qn>lI<)x7~0)v)ab|ckb_^zqXKFhMw zzH!>sk*Y+K)N_NM_ zo=_azM+Ug`8TON&vS)1y?=-XL%8nM&bvvgF*8MsqbHvoRS9mAx(m7V*5S181+tay7*4IztrsX9Ow6A-dk0?EYx!&q{qDCMzMCFM^$U+O=h z3M}q2%Y-!y#eV3FWSCOUA3ZH|-gVL_39+&}FIQw2VP}Bm$@LWpV2Fueygi8~zB`l$ zt_QDB$zBWN5{+h1axT=Fsb3mgTyyXI^nyWPfcwVVx}5jT zmT8GeFZRGyuEfEIbA&zlV7d~g&&oiNh&ChQi+fmqIAd|Wn6i&UPEDF zrQ3CQ)jUumsSSi36hqi0JE&r*qv2dFBI26;#q~X{qT$$k5KB_P4Y?JyS|CXGo&GcV zxwK*|yp$MWQT+Bvz3@@z)1}gk6eVfRRQd^VGx-%Hq_g00ys5?SB?T%+^Md{OOi9lW9tv9jgU( znnGvz0BmeL7zlno@IW#a{6orVLoYx@Jf@p%D|;q}bm^1(8;nYK6uz%x9C`~nUDFBU zR%TF;PYym(!193036M{yx|)dzklR619fqS02TtZ{vrP&jgk~MWXPo$hg^1;<>RyyV zEL9JG(qNtavXP!5QgG%?3UWEc5Fj3UvGC!6^c)EFtG?2G>f36vARSar^=NvhU6DwB zDc&@!oICSqm<%mj3NQC#$J}u7Q>{{ilHPBNrTHF?n(Td#!c@ao9O_TAb(~K#zfd=N zTbckS_4dg?2A*ynN__5V_g%AplR*~=2)RTMvC-DnUS3^g${d`ib`>W-pU!@ToiT^y zq;%~1Mx(Abd!Dt7L5U!AE~A)PAO@ru_S|-bzgCc&Y&S+4>RG*GlvFCJP}u9-1q)q;MGk#@Vd$y!HsN3?Xbj4Br9s^I9QYAxfQ$%#x zemirvfLul)u5GZwR_OZ;EPBq6-u~&z_k!g8yVI#9RIFO~@)I^#7|&3;J{4|FT77SR zwXH9Jq5O8hwOF*woHLpKw3fK?NY0I1=yUUmvM4kS_r&xuPzGOnj=wISZh9qi$`6oR*oKS9 z*kZ5r#66*=#ceyF4xD0iVNanoz$7SPC@4^pF!v=6Shq*fD}UQN+55PUGaDo$TKPhh z%fICWpO$?r@29J(kaX8{H?f57Ryn*^ZR96c!n;UBL;-(6fE}Oquf;_RCoMc|OJ|pS zK?B_L{NoM#?EP9Y>D@p^1yU6!Xc=g-OTR(G!TB)jHAij}W%b^;gN~ep-80Jl_ngiv zK}Qz5Eel(1D?3?cs%dR8j7lbrxkn13Ig)sESe>1^l19L7;OrAhdcjXE+mazCt#oGm z93C!lXM>@%`8;cnd3S58|GG5_5KjuF2&HplBcM4PBU0KIf(ocIdk0Mo*hXM)4+H~qY#i~IZHm&BZurFNu+N(JSK7wcvo?ew>^XG#XX=sQYEPa=^%7ONrQ2Gf=@?XLKqaMklLNe%FX-~)6mm0n z$#k1qzgCD+8jYI3XToYH-(-n-J?Z3IUE4enRrB5XJO8p|_C*f35DVqq`K?KF;binH z;QoSFcux#;`@B;>#5Y~razeyl;lg>#VSE{Le64w_X79`hZRozIrwiax zScRnt(Vt1=b}ybhF0QJonoq#w-&8x`?-07{ikL`?N;rU((2hp>zs>5%)bQy)GjCTb zdL4x1^*_F-##G4RGw>_r*swBezTw`u$0Qt!^f`hWVJ@v!epXrDAf^K z$gya^Uc1`Oow}L_vZr0`!>T6|&nJ5%w*);ho)l=U zp}WsfmG6ga71d6L0;=;-3I`O#S!F2wwXa&#pD~@)Q^3Z?8GM4*rllF29b03S^FzRi zMqT%hCkF=RTxl1EtTk3I5J)WtKt|a|t4mV`v1%q9wY$3-wz?8z1!>Kz4xj+QY`({u zq_JR5Y4{@DUPyop4u7B5`fFYci3Sul0wpI<6pfKJil#g?{!bnzAq!J>#aacCR(I8i zwRjAD`Rn#hRL#o3jTqX7gJFKDb&zm7WCU)$$MO5<%K#j@nH=C$5>(gs?aXR+pg@zh zt2A*)0|jE#725+MPYIRSeC0BpTSa!|<3u2s3_Lc~vt2aKD_I@ZM>cdeXQQjI2}7Fs zsyrc~2f|ut$dk*JDF0Od#k7}li=hd|c)3qrCjJ{X0e`BAHdx+zK)_#vDNUT;A2b}$g~zAIp@Xq_>ONMVn+W{jt7%iPWLTAp$@p~u zNF=DU)VZz zDI7yY4*7V{t|Zqi0>ST6(bgF9o?G%CYQ^HfV|ggE|97JSLAt^%4 zM55UKED;&i{u$UIzArLKXoy$;9FQChr13t0L6ToHuP$H8%awq_M_zFMmfe^M*hrau zq&Z5@27C0+Gf=)z`fHbD)fGojjB*uEg97DKAj%S5wl5M22@U|V_Xm}CFCV+ycwq_% zc=XJBR8woF`{A*#{ zH*X(UD_ZsvT>jBZ6lM0I#fV(!qdRw&Y+mRV-D@cE;6Aa5hCQv1S6m`I ziJ^9jQnTxONBG)Ovest1w>Fm|^rhU231?wqgtlb@YHfUa8Ba42dCW|UHrVZYwC5HH zQ{W=ozjm7fvD?#mGYXgwH459kRiWy@BY~0*$GpDzm*K)*xBWj1mv_{K;ZYFOT8tda zEM~yCa>ghf2MncTDRoy{ggy&vDCs(Ruo$*LDzS%{ln@;d>j`Zy>J_o((3pIJIt5>i zxqD@x)uTI`dn7P12W=H8y`4g*m=*(UWU;C|dR4M8jSiU2i2QAsH<;bN9AcfuE@+M8 z`oPmO1~Ajy{A8N>-z)HH_!0{NdNHDtqj;Lo(z4@CR}uzzz_T4;DAl|%0gC=o>JBMB zW4g~RKmEZ-+enM}eS-={VZRll9gO?(sfGXyumJHxLi~75xG~>NXXfOQBJxjxq$wG? zzyT9OKIaM-mA@Jj-7E)z4PnPw@6sN4h$i>X$-H>gR% zlAMzx6~UrCy6IMlj6#C{*af~SX8BXAK59l!k>Q&(*a0u*+rVT)%N)n~^K#&p8`Sa# zJ_z+_H{TI;VoXs+M~o=9Fms-^V{8)du|>xcMe_1Sn4J%`MMbanY>=Ulh`lyEd}MUA zO`#->?qrg{t25rJ5`RWdlY$P^7Ti{QgJfx^b&Vk73f3H6CF{#%tpz!d-h595>KWYP-6bT`hlK)xt>mtAb2q-iE3P2WtZ9owrA%mQX zFs;1kl_=~X%pf-lE#-UQv0sZu8ot6sm=9-ih7*uRFE!-Vnkqq+c7zmPxZ#ya@qG5- zFM45J0|5DLT=#&bu#XTpSx7I=)hyHqbVO0`6skWz-Dp3vj;YPn{kbvjE;z)}aG+LA zq)Oo_{1f0ELycfc(-j|g)9Lbj;^pH@*joK1xc228=C&ih-`VKs=q_l9mG#(D^++o! zDzdvaQU>Z@)%W)>Sxa?0!L0`b>%We90phQ`onZs7O$Rs;Z9rQIn5BPs99ZTDe)R{K z>cZd7x?*`!%(zxM3X3U*V=U`n`Cou-bhNuk>1}OoqCToI zNe27t>+AMiko8efZjCwd{FCr-0&t2+zBP@HH6TMQY65^ELtr()U=AiEe_%jv7VP3^ z?2QH#3BbeF7KTy}2rWF|MjrD_fu$5tr^?F8t?)UnF(YJxF*AAtmPo^Pa%M%hWe)m%|h|@QD|9aM9Iv7rPM)iqbYL;i>Ep<(ev8`=CEO5@YwV(s;= zq%8NLl(S`H_B%`U(v9*Bf9^@OGC}*v?|y^}MxGA`PA}k1;i?u$&L|Dlk0`kD)H#jnvkIHLME4 z(mqRdJB3fFyl2iSr4EOi*p)mxfxDz7j+Km8-0gbliYLO_Cx!|D={x%sC{Al2PtI35 zX+x2-!D-1YQOaeT`9RqubT5>>J~y3WzH7o$$z&8cH`|7mX;x320ckIPxK#%#Y7z~l z9$K~OMP7h+1VOO zb#XD(@mdhZ*3LB}*WC55@~F38`+&&CL{DJp?{a&DWa|fME6@Jpg%Lvelk%Qd2z*f8QK+Y=gEZ6_?g=Qm|GUSCyIpZfPgu1pO*~*)wB}p@Of|P*`K4-u~~wC!j(K?hWC;L7GXq z>m0=eR3^{=%}9o>b8dw4PcEtSmlD_mB-98b9=co_Iv$9>e8NbM$(+3f?09?h_*cNG z(nR=PjNI?8be>rTc&uD8rWivS_+G{W;M!+lT>*se_5WrxyeiMmU>=8ctt9R@M}(2W zs;a791Ch%Jo8S;YBcsrfm&Vl+&AdF(Fqr~-qS%dWftHe{Z(Mg;>P?e|5pqA#%|I=G zWwxrMNk<4?Q%xE~JxqD;U77xL#$H;Vv&+itzIw4@#%k9%79|Q~y8gr*E~z1k>C`Gk zTF+9<;F_@?M|f*3n#i)?%;b@6i>vaD*kiKL*R|6;4a-&(g@z7n%qq0aC5sZT{dFa4em5PKI(dY%)$E5hO!4K?Mqh%9q775mb1hxiK%= z!B$nXY3N;uFoH!oCzL<%Y>)&as|3Fkx*XVZ1r5>3!W_Fc zFgTbFaQM((Q-{iZCdh?i`_!qBNC*DU0*a1Q!iE;#0V2xrQKA9z!aptV3MSBVX@F;m zR#sZtk?@Kq$odKDSgR%f_MsD~yh+~q6kKoNj(!n{q<{%@UzyxjpFH`{)Rq^!+|!i* z8uVXI7Vbf4j3aR*q|5!gl+1=W@G|2g1*I65zq&LG=$wI_B6^OFbVa7iPnC+mI^{bK zQ@N7VOSusRH7_}fWnBcFW9Y74m+Ni#l;eRhBaS@^5PbrbE-s Date: Tue, 17 Mar 2020 15:45:25 -0700 Subject: [PATCH 120/140] KEP 1327: Apply feedback for third party and dual sourced content (#19486) * Apply feedback for third party and dual sourced content Add feedback from sftim clarify which slack * Limit third-party content to OSS See https://github.com/kubernetes/enhancements/pull/1327#discussion_r392966251 for more discussion --- .../en/docs/contribute/style/content-guide.md | 129 ++++++------------ content/en/docs/setup/_index.md | 59 +------- 2 files changed, 46 insertions(+), 142 deletions(-) diff --git a/content/en/docs/contribute/style/content-guide.md b/content/en/docs/contribute/style/content-guide.md index 5d3f5790c7..b96f28eac7 100644 --- a/content/en/docs/contribute/style/content-guide.md +++ b/content/en/docs/contribute/style/content-guide.md @@ -10,107 +10,68 @@ card: --- {{% capture overview %}} -This page contains guidelines for adding content to the Kubernetes documentation. -If you have questions about allowed content, join the [Kubernetes Slack](http://slack.k8s.io/) #sig-docs channel and ask! Use your best judgment, and feel free to -propose changes to this document in a pull request. -For additional information on creating new content for the Kubernetes -docs, follow the instructions in the [Style guide](/docs/contribute/style/style-guide). +This page contains guidelines for Kubernetes documentation. + +If you have questions about what's allowed, join the #sig-docs channel in +[Kubernetes Slack](http://slack.k8s.io/) and ask! + +You can register for Kubernetes Slack at http://slack.k8s.io/. + +For information on creating new content for the Kubernetes +docs, follow the [style guide](/docs/contribute/style/style-guide). + {{% /capture %}} {{% capture body %}} -## Contributing content -The Kubernetes documentation comprises the content of the -[kubernetes/website](https://github.com/kubernetes/website) source repository. +## Overview + +Source for the Kubernetes website, including the docs, resides in the +[kubernetes/website](https://github.com/kubernetes/website) repository. + Located in the `kubernetes/website/content//docs` folder, the -majority of the Kubernetes documentation is specific to the [Kubernetes -project](https://github.com/kubernetes/kubernetes). The Kubernetes -documentation may also include content from projects in the -[kubernetes](https://github.com/kubernetes) and -[kubernetes-sigs](https://github.com/kubernetes-sigs) GitHub organizations if -those projects do not have their own documentation. Linking to active kubernetes, -kubernetes-sigs, and ({{< glossary_tooltip text="CNCF" term_id="cncf" >}}) projects from the Kubernetes documentation is always -allowed, but linking to vendor-specific products is not. Check the CNCF project lists -([Graduated/Incubating](https://www.cncf.io/projects/), -[Sandbox](https://www.cncf.io/sandbox-projects/), -[Archived](https://www.cncf.io/archived-projects/)) if you are unsure of a -project's CNCF status. +majority of Kubernetes documentation is specific to the [Kubernetes +project](https://github.com/kubernetes/kubernetes). -### Dual-sourced content +## What's allowed -Kubernetes documentation does not include duplicate content sourced from multiple -locations (*dual-sourced* content). Dual-sourced content requires duplicated -effort from project maintainers and tends to become outdated more quickly. -Before adding content, ask yourself this: +Kubernetes docs permit only some kinds of content. -- Is the content about an active CNCF project OR a project in the kubernetes or kubernetes-sigs GitHub organizations? - - If yes, then: - - Does the project have its own documentation? - - if yes, link to the project's documentation from the Kubernetes documentation - - if no, add the content to the project's repository if possible and then link to it from the Kubernetes documentation - - If no, then: - - Stop! - - Adding content about vendor-specific products is not allowed - - Linking to vendor-specific documentation and websites is not allowed +### Third party content +Kubernetes documentation includes applied examples of projects in the Kubernetes project&emdash;projects that live in the [kubernetes](https://github.com/kubernetes) and +[kubernetes-sigs](https://github.com/kubernetes-sigs) GitHub organizations. -### What is and isn't allowed +Links to active content in the Kubernetes project are always allowed. -There are some scenarios in which the Kubernetes documentation includes content from non-Kubernetes projects. -Below are general categories of non-Kubernetes project content along with guidelines of what is and is not allowed: +Kubernetes requires some third party content to function. Examples include container runtimes (containerd, CRI-O, Docker), +[networking policy](/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins/) (CNI plugins), [Ingress controllers](https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/), and [logging](https://kubernetes.io/docs/concepts/cluster-administration/logging/). -1. Instructional content involving non-Kubernetes projects during setup or operation of Kubernetes - - Allowed: - - Referring to or linking to existing documentation about a CNCF project or a project in the kubernetes or kubernetes-sigs GitHub organizations - - Example: for installating Kubernetes in a learning environment, including a prerequisite stating that successful installation and configuration of minikube is required and linking to the relevant minikube documentation - - Adding content for kubernetes or kubernetes-sigs projects that don't have their own instructional content - - Example: including [kubeadm](https://github.com/kubernetes/kubeadm) installation and troubleshooting instructions - - Not Allowed: - - Adding content that duplicates documentation in another repository - - Examples: - - Including minikube installation and configuration instructions; minikube has its own [documentation](https://minikube.sigs.k8s.io/docs/) that provides those instructions - - Including instructions for installing Docker, CRI-O, containerd, and other container runtimes on various operating systems - - Including instructions for installing Kubernetes on production environments using various projects: - - Kubernetes Rebar Integrated Bootstrap (KRIB) is a vendor-specific project and content belongs in the vendor's documentation - - [Kubernetes Operations (kops)](https://github.com/kubernetes/kops) has installation instructions and tutorials in its GitHub repository - - [Kubespray](https://kubespray.io) has its own documentation - - Adding a tutorial that explains how to perform a task using a vendor-specific product or an open source project that is not a CNCF project or a project in the kubernetes or kubnetes-sigs GitHub organizations - - Adding a tutorial on how to use a CNCF project or a project in the kubernetes or kubnetes-sigs GitHub organizations if the project has its own documentation -1. Detailed technical content about how to use a non-Kubernetes project or how that project is designed +Docs can link to third-party open source software (OSS) outside the Kubernetes project if it's necessary for Kubernetes to function. - Adding this type of content to the Kubernetes documentation is not allowed. -1. Content that describes a non-Kubernetes project - - Allowed: - - Adding a brief introductory paragraph about a CNCF project or a project in the kubernetes or kubernetes-sigs GitHub organizations; the paragraph may contain links to the project - - Not Allowed: - - Adding content describing a vendor-specific product - - Adding content describing an open source project that is not a CNCF project or a project in the kubernetes or kubnetes-sigs GitHub organizations - - Adding content that duplicates documentation from another project, regardless of source repository - - Example: adding [Kubernetes in Docker (KinD)](https://kind.sigs.k8s.io) documentation to the Kubernetes documentation -1. Content that simply links to information about a non-Kubernetes project - - Allowed: - - Linking to projects in the kubernetes and kubernetes-sigs GitHub organizations - - Example: linking to Kubernetes in Docker (KinD) [documentation](https://kind.sigs.k8s.io/docs/user/quick-start), which resides in the kubernetes-sigs GitHub organization - - Linking to active CNCF projects - - Example: linking to the Prometheus [documentation](https://prometheus.io/docs/introduction/overview/); Prometheus is an active CNCF project - - Not Allowed: - - Linking to vendor-specific products - - Linking to archived CNCF projects - - Linking to inactive projects in the kubernetes and kubernetes-sigs GitHub organizations - - Linking to open source projects that are not CNCF projects or do not reside in the kubernetes or kubernetes-sigs GitHub organizations -1. Content about training courses - - Allowed: - - Linking to vendor-neutral Kubernetes training courses offered by the [CNCF](https://www.cncf.io/), the [Linux Foundation](https://www.linuxfoundation.org/), and the [Linux Academy](https://linuxacademy.com/), which is a partner of the Linux Foundation - - Example: linking to Linux Academy courses such as [Kubernetes Quick Start](https://linuxacademy.com/course/kubernetes-quick-start/) and [Kubernetes Security](https://linuxacademy.com/course/kubernetes-security/) - - Not Allowed: - - Linking to online training outside of the CNCF, the Linux Foundation, or the Linux Academy; the Kubernetes documentation does not link to third-party content - - Example: linking to Kubernetes tutorials or courses on Medium, KodeKloud, Udacity, Coursera, learnk8s, and similar websites - - Linking to vendor-specific tutorials regardless of the training provider - - Example: linking to Linux Academy courses such as [Google Kubernetes Engine Deep Dive](https://linuxacademy.com/google-cloud-platform/training/course/name/google-kubernetes-engine-deep-dive) and [Amazon EKS Deep Dive](https://linuxacademy.com/course/amazon-eks-deep-dive/) +### Dual sourced content + +Wherever possible, Kubernetes docs link to canonical sources instead of hosting +dual-sourced content. + +Dual-sourced content requires double the effort (or more!) to maintain +and grows stale more quickly. + +{{< note >}} + +If you're a maintainer for a Kubernetes project and need help hosting your own docs, +ask for help in [#sig-docs on Kubernetes Slack](https://kubernetes.slack.com/messages/C1J0BPD2M/). + +{{< /note >}} + +### More information If you have questions about allowed content, join the [Kubernetes Slack](http://slack.k8s.io/) #sig-docs channel and ask! + {{% /capture %}} {{% capture whatsnext %}} + * Read the [Style guide](/docs/contribute/style/style-guide). + {{% /capture %}} diff --git a/content/en/docs/setup/_index.md b/content/en/docs/setup/_index.md index ba241d61ab..880dd46024 100644 --- a/content/en/docs/setup/_index.md +++ b/content/en/docs/setup/_index.md @@ -53,63 +53,6 @@ If you're learning Kubernetes, use the Docker-based solutions: tools supported b When evaluating a solution for a production environment, consider which aspects of operating a Kubernetes cluster (or _abstractions_) you want to manage yourself or offload to a provider. -Some possible abstractions of a Kubernetes cluster are {{< glossary_tooltip text="applications" term_id="applications" >}}, {{< glossary_tooltip text="data plane" term_id="data-plane" >}}, {{< glossary_tooltip text="control plane" term_id="control-plane" >}}, {{< glossary_tooltip text="cluster infrastructure" term_id="cluster-infrastructure" >}}, and {{< glossary_tooltip text="cluster operations" term_id="cluster-operations" >}}. - -The following diagram lists the possible abstractions of a Kubernetes cluster and whether an abstraction is self-managed or managed by a provider. - -Production environment solutions![Production environment solutions](/images/docs/KubernetesSolutions.svg) - -{{< table caption="Production environment solutions table lists the providers and the solutions." >}} -The following production environment solutions table lists the providers and the solutions that they offer. - -|Providers | Managed | Turnkey cloud | On-prem datacenter | Custom (cloud) | Custom (On-premises VMs)| Custom (Bare Metal) | -| --------- | ------ | ------ | ------ | ------ | ------ | ----- | -| [Agile Stacks](https://www.agilestacks.com/products/kubernetes)| | ✔ | ✔ | | | -| [Alibaba Cloud](https://www.alibabacloud.com/product/kubernetes)| | ✔ | | | | -| [Amazon](https://aws.amazon.com) | [Amazon EKS](https://aws.amazon.com/eks/) |[Amazon EC2](https://aws.amazon.com/ec2/) | | | | -| [AppsCode](https://appscode.com/products/pharmer/) | ✔ | | | | | -| [APPUiO](https://appuio.ch/)  | ✔ | ✔ | ✔ | | | | -| [Banzai Cloud Pipeline Kubernetes Engine (PKE)](https://banzaicloud.com/products/pke/) | | ✔ | | ✔ | ✔ | ✔ | -| [CenturyLink Cloud](https://www.ctl.io/) | | ✔ | | | | -| [Cisco Container Platform](https://cisco.com/go/containers) | | | ✔ | | | -| [Cloud Foundry Container Runtime (CFCR)](https://docs-cfcr.cfapps.io/) | | | | ✔ |✔ | -| [CloudStack](https://cloudstack.apache.org/) | | | | | ✔| -| [Canonical](https://ubuntu.com/kubernetes) | ✔ | ✔ | ✔ | ✔ |✔ | ✔ -| [Containership](https://containership.io) | ✔ |✔ | | | | -| [D2iQ](https://d2iq.com/) | | [Kommander](https://docs.d2iq.com/ksphere/kommander/) | [Konvoy](https://d2iq.com/solutions/ksphere/konvoy) | [Konvoy](https://d2iq.com/solutions/ksphere/konvoy) | [Konvoy](https://d2iq.com/solutions/ksphere/konvoy) | [Konvoy](https://d2iq.com/solutions/ksphere/konvoy) | -| [Digital Rebar](https://provision.readthedocs.io/en/tip/README.html) | | | | | | ✔ -| [DigitalOcean](https://www.digitalocean.com/products/kubernetes/) | ✔ | | | | | -| [Docker Enterprise](https://www.docker.com/products/docker-enterprise) | |✔ | ✔ | | | ✔ -| [Gardener](https://gardener.cloud/) | ✔ | ✔ | ✔ | ✔ | ✔ | [Custom Extensions](https://github.com/gardener/gardener/blob/master/docs/extensions/overview.md) | -| [Giant Swarm](https://www.giantswarm.io/) | ✔ | ✔ | ✔ | | -| [Google](https://cloud.google.com/) | [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine/) | [Google Compute Engine (GCE)](https://cloud.google.com/compute/)|[GKE On-Prem](https://cloud.google.com/gke-on-prem/) | | | | | | | | -| [Hidora](https://hidora.com/) | ✔ | ✔| ✔ | | | | | | | | -| [IBM](https://www.ibm.com/in-en/cloud) | [IBM Cloud Kubernetes Service](https://cloud.ibm.com/kubernetes/catalog/cluster)| |[IBM Cloud Private](https://www.ibm.com/in-en/cloud/private) | | -| [Ionos](https://www.ionos.com/enterprise-cloud) | [Ionos Managed Kubernetes](https://www.ionos.com/enterprise-cloud/managed-kubernetes) | [Ionos Enterprise Cloud](https://www.ionos.com/enterprise-cloud) | | -| [Kontena Pharos](https://www.kontena.io/pharos/) | |✔| ✔ | | | -| [KubeOne](https://kubeone.io/) | | ✔ | ✔ | ✔ | ✔ | ✔ | -| [Kubermatic](https://kubermatic.io/) | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | -| [KubeSail](https://kubesail.com/) | ✔ | | | | | -| [Kubespray](https://kubespray.io/#/) | | | |✔ | ✔ | ✔ | -| [Kublr](https://kublr.com/) |✔ | ✔ |✔ |✔ |✔ |✔ | -| [Microsoft Azure](https://azure.microsoft.com) | [Azure Kubernetes Service (AKS)](https://azure.microsoft.com/en-us/services/kubernetes-service/) | | | | | -| [Mirantis Cloud Platform](https://www.mirantis.com/software/kubernetes/) | | | ✔ | | | -| [NetApp Kubernetes Service (NKS)](https://cloud.netapp.com/kubernetes-service) | ✔ | ✔ | ✔ | | | -| [Nirmata](https://www.nirmata.com/) | | ✔ | ✔ | | | -| [Nutanix](https://www.nutanix.com/en) | [Nutanix Karbon](https://www.nutanix.com/products/karbon) | [Nutanix Karbon](https://www.nutanix.com/products/karbon) | | | [Nutanix AHV](https://www.nutanix.com/products/acropolis/virtualization) | -| [OpenNebula](https://www.opennebula.org) |[OpenNebula Kubernetes](https://marketplace.opennebula.systems/docs/service/kubernetes.html) | | | | | -| [OpenShift](https://www.openshift.com) |[OpenShift Dedicated](https://www.openshift.com/products/dedicated/) and [OpenShift Online](https://www.openshift.com/products/online/) | | [OpenShift Container Platform](https://www.openshift.com/products/container-platform/) | | [OpenShift Container Platform](https://www.openshift.com/products/container-platform/) |[OpenShift Container Platform](https://www.openshift.com/products/container-platform/) -| [Oracle Cloud Infrastructure Container Engine for Kubernetes (OKE)](https://docs.cloud.oracle.com/iaas/Content/ContEng/Concepts/contengoverview.htm) | ✔ | ✔ | | | | -| [oVirt](https://www.ovirt.org/) | | | | | ✔ | -| [Pivotal](https://pivotal.io/) | | [Enterprise Pivotal Container Service (PKS)](https://pivotal.io/platform/pivotal-container-service) | [Enterprise Pivotal Container Service (PKS)](https://pivotal.io/platform/pivotal-container-service) | | | -| [Platform9](https://platform9.com/) | [Platform9 Managed Kubernetes](https://platform9.com/managed-kubernetes/) | | [Platform9 Managed Kubernetes](https://platform9.com/managed-kubernetes/) | ✔ | ✔ | ✔ -| [Rancher](https://rancher.com/) | | [Rancher 2.x](https://rancher.com/docs/rancher/v2.x/en/) | | [Rancher Kubernetes Engine (RKE)](https://rancher.com/docs/rke/latest/en/) | | [k3s](https://k3s.io/) -| [Supergiant](https://supergiant.io/) | |✔ | | | | -| [SUSE](https://www.suse.com/) | | ✔ | | | | -| [SysEleven](https://www.syseleven.io/) | ✔ | | | | | -| [Tencent Cloud](https://intl.cloud.tencent.com/) | [Tencent Kubernetes Engine](https://intl.cloud.tencent.com/product/tke) | ✔ | ✔ | | | ✔ | -| [VEXXHOST](https://vexxhost.com/) | ✔ | ✔ | | | | -| [VMware](https://cloud.vmware.com/) | [VMware Cloud PKS](https://cloud.vmware.com/vmware-cloud-pks) |[VMware Enterprise PKS](https://cloud.vmware.com/vmware-enterprise-pks) | [VMware Enterprise PKS](https://cloud.vmware.com/vmware-enterprise-pks) | [VMware Essential PKS](https://cloud.vmware.com/vmware-essential-pks) | |[VMware Essential PKS](https://cloud.vmware.com/vmware-essential-pks) -| [Z.A.R.V.I.S.](https://zarvis.ai/) | ✔ | | | | | | +For a list of [Certified Kubernetes](https://github.com/cncf/k8s-conformance/#certified-kubernetes) providers, see "[Partners](https://kubernetes.io/partners/#conformance)". {{% /capture %}} From 7d60db8e8e3ef2d4b32738800000809930e249ee Mon Sep 17 00:00:00 2001 From: Tim Bannister Date: Wed, 18 Mar 2020 15:26:48 +0000 Subject: [PATCH 121/140] Fix blog article Kong Ingress Controller and Service Mesh: Setting up Ingress to Istio on Kubernetes (#19702) * Fix article formatting * Fix environment variable name $PROXY_URL contains a URL, so that's a better name than $PROXY_IP --- ...ong-Ingress-Controller-and-Service-Mesh.md | 21 ++++++++++--------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/content/en/blog/_posts/Kong-Ingress-Controller-and-Service-Mesh.md b/content/en/blog/_posts/Kong-Ingress-Controller-and-Service-Mesh.md index 9e9a4a5dd4..2023f2e1da 100644 --- a/content/en/blog/_posts/Kong-Ingress-Controller-and-Service-Mesh.md +++ b/content/en/blog/_posts/Kong-Ingress-Controller-and-Service-Mesh.md @@ -11,8 +11,8 @@ Kubernetes has become the de facto way to orchestrate containers and the service Ingress is a group of rules that will proxy inbound connections to endpoints defined by a backend. However, Kubernetes does not know what to do with Ingress resources without an Ingress controller, which is where an open source controller can come into play. In this post, we are going to use one option for this: the Kong Ingress Controller. The Kong Ingress Controller was open-sourced a year ago and recently reached one million downloads. In the recent 0.7 release, service mesh support was also added. Other features of this release include: -* **Built-In Kubernetes Admission Controller,** which validates Custom Resource Definitions (CRD) as they are created or updated and rejects any invalid configurations. -* **n-Memory Mode** - Each pod’s controller actively configures the Kong container in its pod, which limits the blast radius of failure of a single container of Kong or controller container to that pod only. +* **Built-In Kubernetes Admission Controller**, which validates Custom Resource Definitions (CRD) as they are created or updated and rejects any invalid configurations. +* **In-memory Mode** - Each pod’s controller actively configures the Kong container in its pod, which limits the blast radius of failure of a single container of Kong or controller container to that pod only. * **Native gRPC Routing** - gRPC traffic can now be routed via Kong Ingress Controller natively with support for method-based routing. ![K4K-gRPC](/images/blog/Kong-Ingress-Controller-and-Service-Mesh/KIC-gRPC.png) @@ -50,7 +50,7 @@ $ kubectl label namespace kong istio-injection=enabled namespace/kong labeled ``` -Having both namespaces labeled istio-injection=enabled is necessary. Or else the default configuration will not inject a sidecar into the pods of your namespaces. +Having both namespaces labeled `istio-injection=enabled` is necessary. Or else the default configuration will not inject a sidecar container into the pods of your namespaces. Now deploy your BookInfo application with the following command: @@ -72,7 +72,7 @@ serviceaccount/bookinfo-productpage created deployment.apps/productpage-v1 created ``` -Let’s double-check our services and pods to make sure that we have it all set up correctly: +Let’s double-check our Services and Pods to make sure that we have it all set up correctly: ``` $ kubectl get services @@ -87,6 +87,7 @@ reviews ClusterIP 10.104.207.136 9080/TCP 28s You should see four new services: details, productpage, ratings, and reviews. None of them have an external IP so we will use the [Kong gateway](https://github.com/Kong/kong) to expose the necessary services. And to check pods, run the following command: ``` +$ kubectl get pods NAME READY STATUS RESTARTS AGE details-v1-c5b5f496d-9wm29 2/2 Running 0 101s productpage-v1-7d6cfb7dfd-5mc96 2/2 Running 0 100s @@ -96,7 +97,7 @@ reviews-v2-ccffdd984-9jnsj 2/2 Running 0 101s reviews-v3-98dc67b68-nzw97 2/2 Running 0 101s ``` -is command outputs useful data, so let’s take a second to understand it. If you examine the READY column, each pod has two containers running: the service and an Envoy sidecar injected alongside it. Another thing to highlight is that there are three review pods but only 1 review service. The Envoy sidecar will load balance the traffic to three different review pods that contain different versions, giving us the ability to A/B test our changes. With that said, you should now be able to access your product page! +This command outputs useful data, so let’s take a second to understand it. If you examine the READY column, each pod has two containers running: the service and an Envoy sidecar injected alongside it. Another thing to highlight is that there are three review pods but only 1 review service. The Envoy sidecar will load balance the traffic to three different review pods that contain different versions, giving us the ability to A/B test our changes. With that said, you should now be able to access your product page! ``` $ kubectl exec -it $(kubectl get pod -l app=ratings -o jsonpath='{.items[0].metadata.name}') -c ratings -- curl productpage:9080/productpage | grep -o ".*" @@ -131,11 +132,11 @@ NAME READY STATUS RESTARTS AGE pod/ingress-kong-8b44c9856-9s42v 3/3 Running 0 2m26s ``` -There will be three containers within this pod. The first container is the Kong Gateway that will be the Ingress point to your cluster. The second container is the Ingress controller. It uses Ingress resources and updates the proxy to follow rules defined in the resource. And lastly, the third container is the Envoy proxy injected by Istio. Kong will route traffic through the Envoy sidecar proxy to the appropriate service. To send requests into the cluster via our newly deployed Kong Gateway, setup an environment variable with the IP address at which Kong is accessible. +There will be three containers within this pod. The first container is the Kong Gateway that will be the Ingress point to your cluster. The second container is the Ingress controller. It uses Ingress resources and updates the proxy to follow rules defined in the resource. And lastly, the third container is the Envoy proxy injected by Istio. Kong will route traffic through the Envoy sidecar proxy to the appropriate service. To send requests into the cluster via our newly deployed Kong Gateway, setup an environment variable with the a URL based on the IP address at which Kong is accessible. ``` -$ export PROXY_IP=$(minikube service -n kong kong-proxy --url | head -1) -$ echo $PROXY_IP +$ export PROXY_URL="$(minikube service -n kong kong-proxy --url | head -1)" +$ echo $PROXY_URL http://192.168.99.100:32728 ``` @@ -182,10 +183,10 @@ spec: ingress.extensions/productpage created ``` -And just like that, the Kong Ingress Controller is able to understand the rules you defined in the Ingress resource and routes it to the productpage service! To view the product page service’s GUI, go to [http://](http://{Your)$PROXY_IP/productpage. Or to test it in your command line, try: +And just like that, the Kong Ingress Controller is able to understand the rules you defined in the Ingress resource and routes it to the productpage service! To view the product page service’s GUI, go to `$PROXY_URL/productpage` in your browser. Or to test it in your command line, try: ``` -$ curl $PROXY_IP/productpage +$ curl $PROXY_URL/productpage ``` That is all I have for this walk-through. If you enjoyed the technologies used in this post, please check out their repositories since they are all open source and would love to have more contributors! Here are their links for your convenience: From e881e91b9a1799237d682b02e9b6881a77a7c14b Mon Sep 17 00:00:00 2001 From: Tim Bannister Date: Wed, 18 Mar 2020 22:50:44 +0000 Subject: [PATCH 122/140] Improve RBAC guide (#17538) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Update to match website style guide * Use glossary shortcodes * Overall tidying * Drop mention of old (unsupported) releases * Reword RBAC notes & warnings * Write “role bindings” as two words * Tweak RBAC guide wording * Tweak RBAC out-of-the-box roles table * Mention other authorizers There are other authorizers than RBAC and ABAC; hint that these exist and that the API server might run with these configured. * Fix formatting of API discovery roles Also add table caption * Drop incorrect reference to cluster-status * Drop vestigial RBAC warning --- .../docs/reference/access-authn-authz/rbac.md | 707 +++++++++++------- 1 file changed, 425 insertions(+), 282 deletions(-) diff --git a/content/en/docs/reference/access-authn-authz/rbac.md b/content/en/docs/reference/access-authn-authz/rbac.md index 3e18ae283e..b4e2f5ed6e 100644 --- a/content/en/docs/reference/access-authn-authz/rbac.md +++ b/content/en/docs/reference/access-authn-authz/rbac.md @@ -10,35 +10,61 @@ weight: 70 --- {{% capture overview %}} -Role-based access control (RBAC) is a method of regulating access to computer or network resources based on the roles of individual users within an enterprise. +Role-based access control (RBAC) is a method of regulating access to computer or +network resources based on the roles of individual users within your organization. {{% /capture %}} {{% capture body %}} -`RBAC` uses the `rbac.authorization.k8s.io` {{< glossary_tooltip text="API Group" term_id="api-group" >}} -to drive authorization decisions, allowing admins to dynamically configure policies -through the Kubernetes API. +RBAC authorization uses the `rbac.authorization.k8s.io` +{{< glossary_tooltip text="API group" term_id="api-group" >}} to drive authorization +decisions, allowing you to dynamically configure policies through the Kubernetes API. -As of 1.8, RBAC mode is stable and backed by the rbac.authorization.k8s.io/v1 API. +To enable RBAC, start the {{< glossary_tooltip text="API server" term_id="kube-apiserver" >}} +with the `--authorization-mode` flag set to a comma-separated list that includes `RBAC`; +for example: +```shell +kube-apiserver --authorization-mode=Example,RBAC --other-options --more-options +``` -To enable RBAC, start the apiserver with `--authorization-mode=RBAC`. +## API objects {#api-overview} -## API Overview +The RBAC API declares four kinds of Kubernetes object: _Role_, _ClusterRole_, +_RoleBinding_ and _ClusterRoleBinding_. You can +[describe objects](/docs/concepts/overview/working-with-objects/kubernetes-objects/#understanding-kubernetes-objects), +or amend them, using tools such as `kubectl,` just like any other Kubernetes object. -The RBAC API declares four top-level types which will be covered in this -section. Users can interact with these resources as they would with any other -API resource (via `kubectl`, API calls, etc.). For instance, -`kubectl apply -f (resource).yml` can be used with any of these examples, -though readers who wish to follow along should review the section on -bootstrapping first. +{{< caution >}} +These objects, by design, impose access restrictions. If you are making changes +to a cluster as you learn, see +[privilege escalation prevention and bootstrapping](#privilege-escalation-prevention-and-bootstrapping) +to understand how those restrictions can prevent you making some changes. +{{< /caution >}} ### Role and ClusterRole -In the RBAC API, a role contains rules that represent a set of permissions. +An RBAC _Role_ or _ClusterRole_ contains rules that represent a set of permissions. Permissions are purely additive (there are no "deny" rules). -A role can be defined within a namespace with a `Role`, or cluster-wide with a `ClusterRole`. -A `Role` can only be used to grant access to resources within a single namespace. -Here's an example `Role` in the "default" namespace that can be used to grant read access to pods: +A Role always sets permissions within a particular {{< glossary_tooltip text="namespace" term_id="namespace" >}}; +when you create a Role, you have to specify the namespace it belongs in. + +ClusterRole, by contrast, is a non-namespaced resource. The resources have different names (Role +and ClusterRole) because a Kubernetes object always has to be either namespaced or not namespaced; +it can't be both. + +ClusterRoles have several uses. You can use a ClusterRole to: + +1. define permissions on namespaced resources and be granted within individual namespace(s) +1. define permissions on namespaced resources and be granted across all namespaces +1. define permissions on cluster-scoped resources + +If you want to define a role within a namespace, use a Role; if you want to define +a role cluster-wide, use a ClusterRole. + +#### Role example + +Here's an example Role in the "default" namespace that can be used to grant read access to +{{< glossary_tooltip text="pods" term_id="pod" >}}: ```yaml apiVersion: rbac.authorization.k8s.io/v1 @@ -52,14 +78,19 @@ rules: verbs: ["get", "watch", "list"] ``` -A `ClusterRole` can be used to grant the same permissions as a `Role`, -but because they are cluster-scoped, they can also be used to grant access to: +#### ClusterRole example -* cluster-scoped resources (like nodes) -* non-resource endpoints (like "/healthz") -* namespaced resources (like pods) across all namespaces (needed to run `kubectl get pods --all-namespaces`, for example) +A ClusterRole can be used to grant the same permissions as a Role. +Because ClusterRoles are cluster-scoped, you can also use them to grant access to: -The following `ClusterRole` can be used to grant read access to secrets in any particular namespace, +* cluster-scoped resources (like {{< glossary_tooltip text="nodes" term_id="node" >}}) +* non-resource endpoints (like `/healthz`) +* namespaced resources (like Pods), across all namespaces + For example: you can use a ClusterRole to allow a particular user to run + `kubectl get pods --all-namespaces`. + +Here is an example of a ClusterRole that can be used to grant read access to +{{< glossary_tooltip text="secrets" term_id="secret" >}} in any particular namespace, or across all namespaces (depending on how it is [bound](#rolebinding-and-clusterrolebinding)): ```yaml @@ -70,6 +101,9 @@ metadata: name: secret-reader rules: - apiGroups: [""] + # + # at the HTTP level, the name of the resource for accessing Secret + # objects is "secrets" resources: ["secrets"] verbs: ["get", "watch", "list"] ``` @@ -80,51 +114,65 @@ The name of a Role or a ClusterRole object must be a valid ### RoleBinding and ClusterRoleBinding A role binding grants the permissions defined in a role to a user or set of users. -It holds a list of subjects (users, groups, or service accounts), and a reference to the role being granted. -Permissions can be granted within a namespace with a `RoleBinding`, or cluster-wide with a `ClusterRoleBinding`. +It holds a list of *subjects* (users, groups, or service accounts), and a reference to the +role being granted. +A RoleBinding grants permissions within a specific namespace whereas a ClusterRoleBinding +grants that access cluster-wide. -A `RoleBinding` may reference a `Role` in the same namespace. -The name of a `RoleBinding` object must be a valid +A RoleBinding may reference any Role in the same namespace. Alternatively, a RoleBinding +can reference a ClusterRole and bind that ClusterRole to the namespace of the RoleBinding. +If you want to bind a ClusterRole to all the namespaces in your cluster, you use a +ClusterRoleBinding. + +The name of a RoleBinding or ClusterRoleBinding object must be a valid [path segment name](/docs/concepts/overview/working-with-objects/names#path-segment-names). -The following `RoleBinding` grants the "pod-reader" role to the user "jane" within the "default" namespace. -This allows "jane" to read pods in the "default" namespace. +#### RoleBinding examples {#rolebinding-example} -`roleRef` is how you will actually create the binding. The `kind` will be either `Role` or `ClusterRole`, and the `name` will reference the name of the specific `Role` or `ClusterRole` you want. In the example below, this RoleBinding is using `roleRef` to bind the user "jane" to the `Role` created above named `pod-reader`. +Here is an example of a RoleBinding that grants the "pod-reader" Role to the user "jane" +within the "default" namespace. +This allows "jane" to read pods in the "default" namespace. ```yaml apiVersion: rbac.authorization.k8s.io/v1 # This role binding allows "jane" to read pods in the "default" namespace. +# You need to already have a Role named "pod-reader" in that namespace. kind: RoleBinding metadata: name: read-pods namespace: default subjects: +# You can specify more than one "subject" - kind: User - name: jane # Name is case sensitive + name: jane # "name" is case sensitive apiGroup: rbac.authorization.k8s.io roleRef: + # "roleRef" specifies the binding to a Role / ClusterRole kind: Role #this must be Role or ClusterRole name: pod-reader # this must match the name of the Role or ClusterRole you wish to bind to apiGroup: rbac.authorization.k8s.io ``` -A `RoleBinding` may also reference a `ClusterRole` to grant the permissions to namespaced -resources defined in the `ClusterRole` within the `RoleBinding`'s namespace. -This allows administrators to define a set of common roles for the entire cluster, -then reuse them within multiple namespaces. +A RoleBinding can also reference a ClusterRole to grant the permissions defined in that +ClusterRole to resources inside the RoleBinding's namespace. This kind of reference +lets you define a set of common roles across your cluster, then reuse them within +multiple namespaces. -For instance, even though the following `RoleBinding` refers to a `ClusterRole`, -"dave" (the subject, case sensitive) will only be able to read secrets in the "development" -namespace (the namespace of the `RoleBinding`). +For instance, even though the following RoleBinding refers to a ClusterRole, +"dave" (the subject, case sensitive) will only be able to read Secrets in the "development" +namespace, because the RoleBinding's namespace (in its metadata) is "development". ```yaml apiVersion: rbac.authorization.k8s.io/v1 # This role binding allows "dave" to read secrets in the "development" namespace. +# You need to already have a ClusterRole named "secret-reader". kind: RoleBinding metadata: name: read-secrets - namespace: development # This only grants permissions within the "development" namespace. + # + # The namespace of the RoleBinding determines where the permissions are granted. + # This only grants permissions within the "development" namespace. + namespace: development subjects: - kind: User name: dave # Name is case sensitive @@ -135,10 +183,10 @@ roleRef: apiGroup: rbac.authorization.k8s.io ``` -Finally, a `ClusterRoleBinding` may be used to grant permission at the cluster level and in all namespaces. - The name of a `ClusterRoleBinding` object must be a valid -[path segment name](/docs/concepts/overview/working-with-objects/names#path-segment-names). -The following `ClusterRoleBinding` allows any user in the group "manager" to read +#### ClusterRoleBinding example + +To grant permissions across a whole cluster, you can use a ClusterRoleBinding. +The following ClusterRoleBinding allows any user in the group "manager" to read secrets in any namespace. ```yaml @@ -157,37 +205,43 @@ roleRef: apiGroup: rbac.authorization.k8s.io ``` -You cannot modify which `Role` or `ClusterRole` a binding object refers to. -Attempts to change the `roleRef` field of a binding object will result in a validation error. -To change the `roleRef` field on an existing binding object, the binding object must be deleted and recreated. -There are two primary reasons for this restriction: +After you create a binding, you cannot change the Role or ClusterRole that it refers to. +If you try to change a binding's `roleRef`, you get a validation error. If you do want +to change the `roleRef` for a binding, you need to remove the binding object and create +a replacement. -1. A binding to a different role is a fundamentally different binding. +There are two reasons for this restriction: + +1. Making `roleRef` immutable allows granting someone `update` permission on an existing binding +object, so that they can manage the list of subjects, without being able to change +the role that is granted to those subjects. +1. A binding to a different role is a fundamentally different binding. Requiring a binding to be deleted/recreated in order to change the `roleRef` ensures the full list of subjects in the binding is intended to be granted -the new role (as opposed to enabling accidentally modifying just the roleRef -without verifying all of the existing subjects should be given the new role's permissions). -2. Making `roleRef` immutable allows giving `update` permission on an existing binding object -to a user, which lets them manage the list of subjects, without being able to change the -role that is granted to those subjects. +the new role (as opposed to enabling accidentally modifying just the roleRef +without verifying all of the existing subjects should be given the new role's +permissions). The `kubectl auth reconcile` command-line utility creates or updates a manifest file containing RBAC objects, -and handles deleting and recreating binding objects if required to change the role they refer to. +and handles deleting and recreating binding objects if required to change the role they refer to. See [command usage and examples](#kubectl-auth-reconcile) for more information. -### Referring to Resources +### Referring to resources -Most resources are represented by a string representation of their name, such as "pods", just as it -appears in the URL for the relevant API endpoint. However, some Kubernetes APIs involve a -"subresource", such as the logs for a pod. The URL for the pods logs endpoint is: +In the Kubernetes API, most resources are represented and accessed using a string representation of +their object name, such as `pods` for a Pod. RBAC refers to resources using exactly the same +name that appears in the URL for the relevant API endpoint. +Some Kubernetes APIs involve a +_subresource_, such as the logs for a Pod. A request for a Pod's logs looks like: ```http GET /api/v1/namespaces/{namespace}/pods/{name}/log ``` -In this case, "pods" is the namespaced resource, and "log" is a subresource of pods. To represent -this in an RBAC role, use a slash to delimit the resource and subresource. To allow a subject -to read both pods and pod logs, you would write: +In this case, `pods` is the namespaced resource for Pod resources, and `log` is a +subresource of `pods`. To represent this in an RBAC role, use a slash (`/`) to +delimit the resource and subresource. To allow a subject to read `pods` and +also access the `log` subresource for each of those Pods, you write: ```yaml apiVersion: rbac.authorization.k8s.io/v1 @@ -201,9 +255,11 @@ rules: verbs: ["get", "list"] ``` -Resources can also be referred to by name for certain requests through the `resourceNames` list. -When specified, requests can be restricted to individual instances of a resource. To restrict a -subject to only "get" and "update" a single configmap, you would write: +You can also refer to resources by name for certain requests through the `resourceNames` list. +When specified, requests can be restricted to individual instances of a resource. +Here is an example that restricts its subject to only `get` or `update` a +{{< glossary_tooltip term_id="ConfigMap" >}} named `my-configmap`: + ```yaml apiVersion: rbac.authorization.k8s.io/v1 @@ -213,19 +269,30 @@ metadata: name: configmap-updater rules: - apiGroups: [""] + # + # at the HTTP level, the name of the resource for accessing ConfigMap + # objects is "configmaps" resources: ["configmaps"] resourceNames: ["my-configmap"] verbs: ["update", "get"] ``` -Note that `create` requests cannot be restricted by resourceName, as the object name is not known at -authorization time. The other exception is `deletecollection`. +{{< note >}} +You cannot restrict `create` or `deletecollection` requests by resourceName. For `create`, this +limitation is because the object name is not known at authorization time. +{{< /note >}} + ### Aggregated ClusterRoles -As of 1.9, ClusterRoles can be created by combining other ClusterRoles using an `aggregationRule`. The -permissions of aggregated ClusterRoles are controller-managed, and filled in by unioning the rules of any -ClusterRole that matches the provided label selector. An example aggregated ClusterRole: +You can _aggregate_ several ClusterRoles into one combined ClusterRole. +A controller, running as part of the cluster control plane, watches for ClusterRole +objects with an `aggregationRule` set. The `aggregationRule` defines a label +{{< glossary_tooltip text="selector" term_id="selector" >}} that the controller +uses to match other ClusterRole objects that should be combined into the `rules` +field of this one. + +Here is an example aggregated ClusterRole: ```yaml apiVersion: rbac.authorization.k8s.io/v1 @@ -236,12 +303,13 @@ aggregationRule: clusterRoleSelectors: - matchLabels: rbac.example.com/aggregate-to-monitoring: "true" -rules: [] # Rules are automatically filled in by the controller manager. +rules: [] # The control plane automatically fills in the rules ``` -Creating a ClusterRole that matches the label selector will add rules to the aggregated ClusterRole. In this case -rules can be added to the "monitoring" ClusterRole by creating another ClusterRole that has the label -`rbac.example.com/aggregate-to-monitoring: true`. +If you create a new ClusterRole that matches the label selector of an existing aggregated ClusterRole, +that change triggers adding the new rules into the aggregated ClusterRole. +Here is an example that adds rules to the "monitoring" ClusterRole, by creating another +ClusterRole labeled `rbac.example.com/aggregate-to-monitoring: true`. ```yaml apiVersion: rbac.authorization.k8s.io/v1 @@ -250,19 +318,22 @@ metadata: name: monitoring-endpoints labels: rbac.example.com/aggregate-to-monitoring: "true" -# These rules will be added to the "monitoring" role. +# When you create the "monitoring-endpoints" ClusterRole, +# the rules below will be added to the "monitoring" ClusterRole. rules: - apiGroups: [""] resources: ["services", "endpoints", "pods"] verbs: ["get", "list", "watch"] ``` -The default user-facing roles (described below) use ClusterRole aggregation. This lets admins include rules -for custom resources, such as those served by CustomResourceDefinitions or Aggregated API servers, on the -default roles. +The [default user-facing roles](#default-roles-and-rolebindings) use ClusterRole aggregation. This lets you, +as a cluster administrator, include rules for custom resources, such as those served by +{{< glossary_tooltip term_id="CustomResourceDefinition" text="CustomResourceDefinitions" >}} +or aggregated API servers, to extend the default roles. -For example, the following ClusterRoles let the "admin" and "edit" default roles manage the custom resource -"CronTabs" and the "view" role perform read-only actions on the resource. +For example: the following ClusterRoles let the "admin" and "edit" default roles manage the custom resource +named CronTab, whereas the "view" role can perform just read actions on CronTab resources. +You can assume that CronTab objects are named `"crontabs"` in URLs as seen by the API server. ```yaml apiVersion: rbac.authorization.k8s.io/v1 @@ -291,60 +362,87 @@ rules: verbs: ["get", "list", "watch"] ``` -#### Role Examples +#### Role examples -Only the `rules` section is shown in the following examples. +The following examples are excerpts from Role or ClusterRole objects, showing only +the `rules` section. -Allow reading the resource "pods" in the core {{< glossary_tooltip text="API Group" term_id="api-group" >}}: +Allow reading `"pods"` resources in the core +{{< glossary_tooltip text="API Group" term_id="api-group" >}}: ```yaml rules: - apiGroups: [""] + # + # at the HTTP level, the name of the resource for accessing Pod + # objects is "pods" resources: ["pods"] verbs: ["get", "list", "watch"] ``` -Allow reading/writing "deployments" in both the "extensions" and "apps" API groups: +Allow reading/writing Deployments (at the HTTP level: objects with `"deployments"` +in the resource part of their URL) in both the `"extensions"` and `"apps"` API groups: ```yaml rules: - apiGroups: ["extensions", "apps"] + # + # at the HTTP level, the name of the resource for accessing Deployment + # objects is "deployments" resources: ["deployments"] verbs: ["get", "list", "watch", "create", "update", "patch", "delete"] ``` -Allow reading "pods" and reading/writing "jobs": +Allow reading Pods in the core API group, as well as reading or writing Job +resources in the `"batch"` or `"extensions"` API groups: ```yaml rules: - apiGroups: [""] + # + # at the HTTP level, the name of the resource for accessing Pod + # objects is "pods" resources: ["pods"] verbs: ["get", "list", "watch"] - apiGroups: ["batch", "extensions"] + # + # at the HTTP level, the name of the resource for accessing Job + # objects is "jobs" resources: ["jobs"] verbs: ["get", "list", "watch", "create", "update", "patch", "delete"] ``` -Allow reading a `ConfigMap` named "my-config" (must be bound with a `RoleBinding` to limit to a single `ConfigMap` in a single namespace): +Allow reading a ConfigMap named "my-config" (must be bound with a +RoleBinding to limit to a single ConfigMap in a single namespace): ```yaml rules: - apiGroups: [""] + # + # at the HTTP level, the name of the resource for accessing ConfigMap + # objects is "configmaps" resources: ["configmaps"] resourceNames: ["my-config"] verbs: ["get"] ``` -Allow reading the resource "nodes" in the core group (because a `Node` is cluster-scoped, this must be in a `ClusterRole` bound with a `ClusterRoleBinding` to be effective): +Allow reading the resource `"nodes"` in the core group (because a +Node is cluster-scoped, this must be in a ClusterRole bound with a +ClusterRoleBinding to be effective): ```yaml rules: - apiGroups: [""] + # + # at the HTTP level, the name of the resource for accessing Node + # objects is "nodes" resources: ["nodes"] verbs: ["get", "list", "watch"] ``` -Allow "GET" and "POST" requests to the non-resource endpoint "/healthz" and all subpaths (must be in a `ClusterRole` bound with a `ClusterRoleBinding` to be effective): +Allow GET and POST requests to the non-resource endpoint `/healthz` and +all subpaths (must be in a ClusterRole bound with a ClusterRoleBinding +to be effective): ```yaml rules: @@ -352,32 +450,44 @@ rules: verbs: ["get", "post"] ``` -### Referring to Subjects +### Referring to subjects -A `RoleBinding` or `ClusterRoleBinding` binds a role to *subjects*. -Subjects can be groups, users or service accounts. +A RoleBinding or ClusterRoleBinding binds a role to subjects. +Subjects can be groups, users or +{{< glossary_tooltip text="ServiceAccounts" term_id="service-account" >}}. -Users are represented by strings. These can be plain usernames, like -"alice", email-style names, like "bob@example.com", or numeric IDs -represented as a string. It is up to the Kubernetes admin to configure -the [authentication modules](/docs/reference/access-authn-authz/authentication/) to produce -usernames in the desired format. The RBAC authorization system does -not require any particular format. However, the prefix `system:` is -reserved for Kubernetes system use, and so the admin should ensure -usernames do not contain this prefix by accident. +Kubernetes represents usernames as strings. +These can be: plain names, such as "alice"; email-style names, like "bob@example.com"; +or numeric user IDs represented as a string. It is up to you as a cluster administrator +to configure the [authentication modules](/docs/reference/access-authn-authz/authentication/) +so that authentication produces usernames in the format you want. -Group information in Kubernetes is currently provided by the Authenticator -modules. Groups, like users, are represented as strings, and that string -has no format requirements, other than that the prefix `system:` is reserved. +{{< caution >}} +The prefix `system:` is reserved for Kubernetes system use, so you should ensure +that you don't have users or groups with names that start with `system:` by +accident. +Other than this special prefix, the RBAC authorization system does not require any format +for usernames. +{{< /caution >}} -[Service Accounts](/docs/tasks/configure-pod-container/configure-service-account/) have usernames with the `system:serviceaccount:` prefix and belong -to groups with the `system:serviceaccounts:` prefix. +In Kubernetes, Authenticator modules provide group information. +Groups, like users, are represented as strings, and that string has no format requirements, +other than that the prefix `system:` is reserved. -#### Role Binding Examples +[ServiceAccounts](/docs/tasks/configure-pod-container/configure-service-account/) have names prefixed +with `system:serviceaccount:`, and belong to groups that have names prefixed with `system:serviceaccounts:`. -Only the `subjects` section of a `RoleBinding` is shown in the following examples. +{{< note >}} +- `system:serviceaccount:` (singular) is the prefix for service account usernames. +- `system:serviceaccounts:` (plural) is the prefix for service account groups. +{{< /note >}} -For a user named "alice@example.com": +#### RoleBinding examples {#role-binding-examples} + +The following examples are `RoleBinding` excerpts that only +show the `subjects` section. + +For a user named `alice@example.com`: ```yaml subjects: @@ -386,7 +496,7 @@ subjects: apiGroup: rbac.authorization.k8s.io ``` -For a group named "frontend-admins": +For a group named `frontend-admins`: ```yaml subjects: @@ -395,7 +505,7 @@ subjects: apiGroup: rbac.authorization.k8s.io ``` -For the default service account in the kube-system namespace: +For the default service account in the "kube-system" namespace: ```yaml subjects: @@ -413,7 +523,7 @@ subjects: apiGroup: rbac.authorization.k8s.io ``` -For all service accounts everywhere: +For all service accounts in any namespace: ```yaml subjects: @@ -422,7 +532,7 @@ subjects: apiGroup: rbac.authorization.k8s.io ``` -For all authenticated users (version 1.5+): +For all authenticated users: ```yaml subjects: @@ -431,7 +541,7 @@ subjects: apiGroup: rbac.authorization.k8s.io ``` -For all unauthenticated users (version 1.5+): +For all unauthenticated users: ```yaml subjects: @@ -440,7 +550,7 @@ subjects: apiGroup: rbac.authorization.k8s.io ``` -For all users (version 1.5+): +For all users: ```yaml subjects: @@ -452,42 +562,51 @@ subjects: apiGroup: rbac.authorization.k8s.io ``` -## Default Roles and Role Bindings +## Default roles and role bindings -API servers create a set of default `ClusterRole` and `ClusterRoleBinding` objects. -Many of these are `system:` prefixed, which indicates that the resource is "owned" by the infrastructure. -Modifications to these resources can result in non-functional clusters. One example is the `system:node` ClusterRole. -This role defines permissions for kubelets. If the role is modified, it can prevent kubelets from working. +API servers create a set of default ClusterRole and ClusterRoleBinding objects. +Many of these are `system:` prefixed, which indicates that the resource is directly +managed by the cluster control plane. +All of the default ClusterRoles and ClusterRoleBindings are labeled with `kubernetes.io/bootstrapping=rbac-defaults`. -All of the default cluster roles and rolebindings are labeled with `kubernetes.io/bootstrapping=rbac-defaults`. +{{< caution >}} +Take care when modifying ClusterRoles and ClusterRoleBindings with names +that have a `system:` prefix. +Modifications to these resources can result in non-functional clusters. +{{< /caution >}} ### Auto-reconciliation At each start-up, the API server updates default cluster roles with any missing permissions, and updates default cluster role bindings with any missing subjects. -This allows the cluster to repair accidental modifications, -and to keep roles and rolebindings up-to-date as permissions and subjects change in new releases. +This allows the cluster to repair accidental modifications, and helps to keep roles and role bindings +up-to-date as permissions and subjects change in new Kubernetes releases. -To opt out of this reconciliation, set the `rbac.authorization.kubernetes.io/autoupdate` +To opt out of this reconciliation, set the `rbac.authorization.kubernetes.io/autoupdate` annotation on a default cluster role or rolebinding to `false`. Be aware that missing default permissions and subjects can result in non-functional clusters. -Auto-reconciliation is enabled in Kubernetes version 1.6+ when the RBAC authorizer is active. +Auto-reconciliation is enabled by default if the RBAC authorizer is active. -### Discovery Roles +### API discovery roles {#discovery-roles} -Default role bindings authorize unauthenticated and authenticated users to read API information that is deemed safe to be publicly accessible (including CustomResourceDefinitions). To disable anonymous unauthenticated access add `--anonymous-auth=false` to the API server configuration. +Default role bindings authorize unauthenticated and authenticated users to read API information that is deemed safe to be publicly accessible (including CustomResourceDefinitions). To disable anonymous unauthenticated access, add `--anonymous-auth=false` to the API server configuration. To view the configuration of these roles via `kubectl` run: -``` +```shell kubectl get clusterroles system:discovery -o yaml ``` -NOTE: editing the role is not recommended as changes will be overwritten on API server restart via auto-reconciliation (see above). +{{< note >}} +If you edit that ClusterRole, your changes will be overwritten on API server restart +via [auto-reconciliation](#auto-reconciliation). To avoid that overwriting, +either do not manually edit the role, or disable auto-reconciliation. +{{< /note >}} - + + @@ -496,30 +615,30 @@ NOTE: editing the role is not recommended as changes will be overwritten on API - + - + - +
    Kubernetes RBAC API discovery roles
    Default ClusterRole Default ClusterRoleBinding
    system:basic-user system:authenticated groupAllows a user read-only access to basic information about themselves. Prior to 1.14, this role was also bound to `system:unauthenticated` by default.Allows a user read-only access to basic information about themselves. Prior to v1.14, this role was also bound to system:unauthenticated by default.
    system:discovery system:authenticated groupAllows read-only access to API discovery endpoints needed to discover and negotiate an API level. Prior to 1.14, this role was also bound to `system:unauthenticated` by default.Allows read-only access to API discovery endpoints needed to discover and negotiate an API level. Prior to v1.14, this role was also bound to system:unauthenticated by default.
    system:public-info-viewer system:authenticated and system:unauthenticated groupsAllows read-only access to non-sensitive information about the cluster. Introduced in 1.14.Allows read-only access to non-sensitive information about the cluster. Introduced in Kubernetes v1.14.
    -### User-facing Roles +### User-facing roles -Some of the default roles are not `system:` prefixed. These are intended to be user-facing roles. -They include super-user roles (`cluster-admin`), -roles intended to be granted cluster-wide using ClusterRoleBindings (`cluster-status`), -and roles intended to be granted within particular namespaces using RoleBindings (`admin`, `edit`, `view`). +Some of the default ClusterRoles are not `system:` prefixed. These are intended to be user-facing roles. +They include super-user roles (`cluster-admin`), roles intended to be granted cluster-wide +using ClusterRoleBindings, and roles intended to be granted within particular +namespaces using RoleBindings (`admin`, `edit`, `view`). -As of 1.9, user-facing roles use [ClusterRole Aggregation](#aggregated-clusterroles) to allow admins to include -rules for custom resources on these roles. To add rules to the "admin", "edit", or "view" role, create a -ClusterRole with one or more of the following labels: +User-facing ClusterRoles use [ClusterRole aggregation](#aggregated-clusterroles) to allow admins to include +rules for custom resources on these ClusterRoles. To add rules to the `admin`, `edit`, or `view` roles, create +a ClusterRole with one or more of the following labels: ```yaml metadata: @@ -541,32 +660,40 @@ metadata: system:masters group Allows super-user access to perform any action on any resource. When used in a ClusterRoleBinding, it gives full control over every resource in the cluster and in all namespaces. -When used in a RoleBinding, it gives full control over every resource in the rolebinding's namespace, including the namespace itself. +When used in a RoleBinding, it gives full control over every resource in the role binding's namespace, including the namespace itself. admin None Allows admin access, intended to be granted within a namespace using a RoleBinding. If used in a RoleBinding, allows read/write access to most resources in a namespace, -including the ability to create roles and rolebindings within the namespace. -It does not allow write access to resource quota or to the namespace itself. +including the ability to create roles and role bindings within the namespace. +This role does not allow write access to resource quota or to the namespace itself. edit None Allows read/write access to most objects in a namespace. -It does not allow viewing or modifying roles or rolebindings. + +This role does not allow viewing or modifying roles or role bindings. +However, this role allows accessing Secrets and running Pods as any ServiceAccount in +the namespace, so it can be used to gain the API access levels of any ServiceAccount in +the namespace. view None Allows read-only access to see most objects in a namespace. -It does not allow viewing roles or rolebindings. -It does not allow viewing secrets, since those are escalating. +It does not allow viewing roles or role bindings. + +This role does not allow viewing Secrets, since reading +the contents of Secrets enables access to ServiceAccount credentials +in the namespace, which would allow API access as any ServiceAccount +in the namespace (a form of privilege escalation). -### Core Component Roles +### Core component roles @@ -578,7 +705,7 @@ It does not allow viewing secrets, since those are escalating. - + @@ -588,28 +715,27 @@ It does not allow viewing secrets, since those are escalating. - + - - + - +
    system:kube-scheduler system:kube-scheduler userAllows access to the resources required by the kube-scheduler component.Allows access to the resources required by the {{< glossary_tooltip term_id="kube-scheduler" text="scheduler" >}} component.
    system:volume-scheduler
    system:kube-controller-manager system:kube-controller-manager userAllows access to the resources required by the kube-controller-manager component. -The permissions required by individual control loops are contained in the controller roles.Allows access to the resources required by the {{< glossary_tooltip term_id="kube-controller-manager" text="controller manager" >}} component. +The permissions required by individual controllers are detailed in the controller roles.
    system:nodeNone in 1.8+Allows access to resources required by the kubelet component, including read access to all secrets, and write access to all pod status objects. +NoneAllows access to resources required by the kubelet, including read access to all secrets, and write access to all pod status objects. -As of 1.7, use of the Node authorizer and NodeRestriction admission plugin is recommended instead of this role, and allow granting API access to kubelets based on the pods scheduled to run on them. -Prior to 1.7, this role was automatically bound to the `system:nodes` group. -In 1.7, this role was automatically bound to the `system:nodes` group if the `Node` authorization mode is not enabled. -In 1.8+, no binding is automatically created. +You should use the Node authorizer and NodeRestriction admission plugin instead of the system:node role, and allow granting API access to kubelets based on the Pods scheduled to run on them. + +The system:node role only exists for compatibility with Kubernetes clusters upgraded from versions prior to v1.8.
    system:node-proxier system:kube-proxy userAllows access to the resources required by the kube-proxy component.Allows access to the resources required by the {{< glossary_tooltip term_id="kube-proxy" text="kube-proxy" >}} component.
    -### Other Component Roles +### Other component roles @@ -627,7 +753,7 @@ This is commonly used by add-on API servers for unified authentication and autho - + @@ -648,7 +774,7 @@ This is commonly used by add-on API servers for unified authentication and autho +kubelet TLS bootstrapping. @@ -662,73 +788,80 @@ This is commonly used by add-on API servers for unified authentication and autho
    system:heapster NoneRole for the Heapster component.Role for the Heapster component (deprecated).
    system:kube-aggregatorsystem:node-bootstrapper None Allows access to the resources required to perform -Kubelet TLS bootstrapping.
    system:node-problem-detector
    -### Controller Roles +### Roles for built-in controllers {#controller-roles} -The [Kubernetes controller manager](/docs/admin/kube-controller-manager/) runs core control loops. -When invoked with `--use-service-account-credentials`, each control loop is started using a separate service account. -Corresponding roles exist for each control loop, prefixed with `system:controller:`. -If the controller manager is not started with `--use-service-account-credentials`, -it runs all control loops using its own credential, which must be granted all the relevant roles. +The Kubernetes {{< glossary_tooltip term_id="kube-controller-manager" text="controller manager" >}} runs +{{< glossary_tooltip term_id="controller" text="controllers" >}} that are built in to the Kubernetes +control plane. +When invoked with `--use-service-account-credentials`, kube-controller-manager starts each controller +using a separate service account. +Corresponding roles exist for each built-in controller, prefixed with `system:controller:`. +If the controller manager is not started with `--use-service-account-credentials`, it runs all control loops +using its own credential, which must be granted all the relevant roles. These roles include: -* system:controller:attachdetach-controller -* system:controller:certificate-controller -* system:controller:clusterrole-aggregation-controller -* system:controller:cronjob-controller -* system:controller:daemon-set-controller -* system:controller:deployment-controller -* system:controller:disruption-controller -* system:controller:endpoint-controller -* system:controller:expand-controller -* system:controller:generic-garbage-collector -* system:controller:horizontal-pod-autoscaler -* system:controller:job-controller -* system:controller:namespace-controller -* system:controller:node-controller -* system:controller:persistent-volume-binder -* system:controller:pod-garbage-collector -* system:controller:pv-protection-controller -* system:controller:pvc-protection-controller -* system:controller:replicaset-controller -* system:controller:replication-controller -* system:controller:resourcequota-controller -* system:controller:root-ca-cert-publisher -* system:controller:route-controller -* system:controller:service-account-controller -* system:controller:service-controller -* system:controller:statefulset-controller -* system:controller:ttl-controller +* `system:controller:attachdetach-controller` +* `system:controller:certificate-controller` +* `system:controller:clusterrole-aggregation-controller` +* `system:controller:cronjob-controller` +* `system:controller:daemon-set-controller` +* `system:controller:deployment-controller` +* `system:controller:disruption-controller` +* `system:controller:endpoint-controller` +* `system:controller:expand-controller` +* `system:controller:generic-garbage-collector` +* `system:controller:horizontal-pod-autoscaler` +* `system:controller:job-controller` +* `system:controller:namespace-controller` +* `system:controller:node-controller` +* `system:controller:persistent-volume-binder` +* `system:controller:pod-garbage-collector` +* `system:controller:pv-protection-controller` +* `system:controller:pvc-protection-controller` +* `system:controller:replicaset-controller` +* `system:controller:replication-controller` +* `system:controller:resourcequota-controller` +* `system:controller:root-ca-cert-publisher` +* `system:controller:route-controller` +* `system:controller:service-account-controller` +* `system:controller:service-controller` +* `system:controller:statefulset-controller` +* `system:controller:ttl-controller` -## Privilege Escalation Prevention and Bootstrapping +## Privilege escalation prevention and bootstrapping The RBAC API prevents users from escalating privileges by editing roles or role bindings. Because this is enforced at the API level, it applies even when the RBAC authorizer is not in use. -A user can only create/update a role if at least one of the following things is true: +### Restrictions on role creation or update -1. They already have all the permissions contained in the role, at the same scope as the object being modified -(cluster-wide for a `ClusterRole`, within the same namespace or cluster-wide for a `Role`) -2. They are given explicit permission to perform the `escalate` verb on the `roles` or `clusterroles` resource in the `rbac.authorization.k8s.io` API group (Kubernetes 1.12 and newer) +You can only create/update a role if at least one of the following things is true: -For example, if "user-1" does not have the ability to list secrets cluster-wide, they cannot create a `ClusterRole` +1. You already have all the permissions contained in the role, at the same scope as the object being modified +(cluster-wide for a ClusterRole, within the same namespace or cluster-wide for a Role). +2. You are granted explicit permission to perform the `escalate` verb on the `roles` or `clusterroles` resource in the `rbac.authorization.k8s.io` API group. + +For example, if `user-1` does not have the ability to list Secrets cluster-wide, they cannot create a ClusterRole containing that permission. To allow a user to create/update roles: -1. Grant them a role that allows them to create/update `Role` or `ClusterRole` objects, as desired. -2. Grant them permission to include specific permissions in the roles the create/update: - * implicitly, by giving them those permissions (if they attempt to create or modify a `Role` or `ClusterRole` with permissions they themselves have not been granted, the API request will be forbidden) - * or explicitly allow specifying any permission in a `Role` or `ClusterRole` by giving them permission to perform the `escalate` verb on `roles` or `clusterroles` resources in the `rbac.authorization.k8s.io` API group (Kubernetes 1.12 and newer) +1. Grant them a role that allows them to create/update Role or ClusterRole objects, as desired. +2. Grant them permission to include specific permissions in the roles they create/update: + * implicitly, by giving them those permissions (if they attempt to create or modify a Role or ClusterRole with permissions they themselves have not been granted, the API request will be forbidden) + * or explicitly allow specifying any permission in a `Role` or `ClusterRole` by giving them permission to perform the `escalate` verb on `roles` or `clusterroles` resources in the `rbac.authorization.k8s.io` API group -A user can only create/update a role binding if they already have all the permissions contained in the referenced role -(at the same scope as the role binding) *or* if they've been given explicit permission to perform the `bind` verb on the referenced role. -For example, if "user-1" does not have the ability to list secrets cluster-wide, they cannot create a `ClusterRoleBinding` +### Restrictions on role binding creation or update + +You can only create/update a role binding if you already have all the permissions contained in the referenced role +(at the same scope as the role binding) *or* if you have been authorized to perform the `bind` verb on the referenced role. +For example, if `user-1` does not have the ability to list Secrets cluster-wide, they cannot create a ClusterRoleBinding to a role that grants that permission. To allow a user to create/update role bindings: -1. Grant them a role that allows them to create/update `RoleBinding` or `ClusterRoleBinding` objects, as desired. +1. Grant them a role that allows them to create/update RoleBinding or ClusterRoleBinding objects, as desired. 2. Grant them permissions needed to bind a particular role: * implicitly, by giving them the permissions contained in the role. - * explicitly, by giving them permission to perform the `bind` verb on the particular role (or cluster role). + * explicitly, by giving them permission to perform the `bind` verb on the particular Role (or ClusterRole). -For example, this cluster role and role binding would allow "user-1" to grant other users the `admin`, `edit`, and `view` roles in the "user-1-namespace" namespace: +For example, this ClusterRole and RoleBinding would allow `user-1` to grant other users the `admin`, `edit`, and `view` roles in the namespace `user-1-namespace`: ```yaml apiVersion: rbac.authorization.k8s.io/v1 @@ -762,126 +895,126 @@ subjects: When bootstrapping the first roles and role bindings, it is necessary for the initial user to grant permissions they do not yet have. To bootstrap initial roles and role bindings: -* Use a credential with the `system:masters` group, which is bound to the `cluster-admin` super-user role by the default bindings. +* Use a credential with the "system:masters" group, which is bound to the "cluster-admin" super-user role by the default bindings. * If your API server runs with the insecure port enabled (`--insecure-port`), you can also make API calls via that port, which does not enforce authentication or authorization. -## Command-line Utilities +## Command-line utilities ### `kubectl create role` -Creates a `Role` object defining permissions within a single namespace. Examples: +Creates a Role object defining permissions within a single namespace. Examples: -* Create a `Role` named "pod-reader" that allows user to perform "get", "watch" and "list" on pods: +* Create a Role named "pod-reader" that allows users to perform `get`, `watch` and `list` on pods: - ``` + ```shell kubectl create role pod-reader --verb=get --verb=list --verb=watch --resource=pods ``` -* Create a `Role` named "pod-reader" with resourceNames specified: +* Create a Role named "pod-reader" with resourceNames specified: - ``` + ```shell kubectl create role pod-reader --verb=get --resource=pods --resource-name=readablepod --resource-name=anotherpod ``` -* Create a `Role` named "foo" with apiGroups specified: +* Create a Role named "foo" with apiGroups specified: - ``` + ```shell kubectl create role foo --verb=get,list,watch --resource=replicasets.apps ``` -* Create a `Role` named "foo" with subresource permissions: +* Create a Role named "foo" with subresource permissions: - ``` + ```shell kubectl create role foo --verb=get,list,watch --resource=pods,pods/status ``` -* Create a `Role` named "my-component-lease-holder" with permissions to get/update a resource with a specific name: +* Create a Role named "my-component-lease-holder" with permissions to get/update a resource with a specific name: - ``` + ```shell kubectl create role my-component-lease-holder --verb=get,list,watch,update --resource=lease --resource-name=my-component ``` ### `kubectl create clusterrole` -Creates a `ClusterRole` object. Examples: +Creates a ClusterRole. Examples: -* Create a `ClusterRole` named "pod-reader" that allows user to perform "get", "watch" and "list" on pods: +* Create a ClusterRole named "pod-reader" that allows user to perform `get`, `watch` and `list` on pods: - ``` + ```shell kubectl create clusterrole pod-reader --verb=get,list,watch --resource=pods ``` -* Create a `ClusterRole` named "pod-reader" with resourceNames specified: +* Create a ClusterRole named "pod-reader" with resourceNames specified: - ``` + ```shell kubectl create clusterrole pod-reader --verb=get --resource=pods --resource-name=readablepod --resource-name=anotherpod ``` -* Create a `ClusterRole` named "foo" with apiGroups specified: +* Create a ClusterRole named "foo" with apiGroups specified: - ``` + ```shell kubectl create clusterrole foo --verb=get,list,watch --resource=replicasets.apps ``` -* Create a `ClusterRole` named "foo" with subresource permissions: +* Create a ClusterRole named "foo" with subresource permissions: - ``` + ```shell kubectl create clusterrole foo --verb=get,list,watch --resource=pods,pods/status ``` -* Create a `ClusterRole` name "foo" with nonResourceURL specified: +* Create a ClusterRole named "foo" with nonResourceURL specified: - ``` + ```shell kubectl create clusterrole "foo" --verb=get --non-resource-url=/logs/* ``` -* Create a `ClusterRole` name "monitoring" with aggregationRule specified: +* Create a ClusterRole named "monitoring" with an aggregationRule specified: - ``` + ```shell kubectl create clusterrole monitoring --aggregation-rule="rbac.example.com/aggregate-to-monitoring=true" ``` ### `kubectl create rolebinding` -Grants a `Role` or `ClusterRole` within a specific namespace. Examples: +Grants a Role or ClusterRole within a specific namespace. Examples: -* Within the namespace "acme", grant the permissions in the `admin` `ClusterRole` to a user named "bob": +* Within the namespace "acme", grant the permissions in the "admin" ClusterRole to a user named "bob": - ``` + ```shell kubectl create rolebinding bob-admin-binding --clusterrole=admin --user=bob --namespace=acme ``` -* Within the namespace "acme", grant the permissions in the `view` `ClusterRole` to the service account in the namespace "acme" named "myapp" : +* Within the namespace "acme", grant the permissions in the "view" ClusterRole to the service account in the namespace "acme" named "myapp": - ``` + ```shell kubectl create rolebinding myapp-view-binding --clusterrole=view --serviceaccount=acme:myapp --namespace=acme ``` -* Within the namespace "acme", grant the permissions in the `view` `ClusterRole` to a service account in the namespace "myappnamespace" named "myapp": +* Within the namespace "acme", grant the permissions in the "view" ClusterRole to a service account in the namespace "myappnamespace" named "myapp": - ``` + ```shell kubectl create rolebinding myappnamespace-myapp-view-binding --clusterrole=view --serviceaccount=myappnamespace:myapp --namespace=acme ``` ### `kubectl create clusterrolebinding` -Grants a `ClusterRole` across the entire cluster, including all namespaces. Examples: +Grants a ClusterRole across the entire cluster (all namespaces). Examples: -* Across the entire cluster, grant the permissions in the `cluster-admin` `ClusterRole` to a user named "root": +* Across the entire cluster, grant the permissions in the "cluster-admin" ClusterRole to a user named "root": - ``` + ```shell kubectl create clusterrolebinding root-cluster-admin-binding --clusterrole=cluster-admin --user=root ``` -* Across the entire cluster, grant the permissions in the `system:node-proxier ` `ClusterRole` to a user named "system:kube-proxy": +* Across the entire cluster, grant the permissions in the "system:node-proxier" ClusterRole to a user named "system:kube-proxy": - ``` + ```shell kubectl create clusterrolebinding kube-proxy-binding --clusterrole=system:node-proxier --user=system:kube-proxy ``` -* Across the entire cluster, grant the permissions in the `view` `ClusterRole` to a service account named "myapp" in the namespace "acme": +* Across the entire cluster, grant the permissions in the "view" ClusterRole to a service account named "myapp" in the namespace "acme": - ``` + ```shell kubectl create clusterrolebinding myapp-view-binding --clusterrole=view --serviceaccount=acme:myapp ``` @@ -901,33 +1034,32 @@ Examples: * Test applying a manifest file of RBAC objects, displaying changes that would be made: - ``` + ```shell kubectl auth reconcile -f my-rbac-rules.yaml --dry-run ``` * Apply a manifest file of RBAC objects, preserving any extra permissions (in roles) and any extra subjects (in bindings): - ``` + ```shell kubectl auth reconcile -f my-rbac-rules.yaml ``` * Apply a manifest file of RBAC objects, removing any extra permissions (in roles) and any extra subjects (in bindings): - ``` + ```shell kubectl auth reconcile -f my-rbac-rules.yaml --remove-extra-subjects --remove-extra-permissions ``` -See the CLI help for detailed usage. - -## Service Account Permissions +## ServiceAccount permissions {#service-account-permissions} Default RBAC policies grant scoped permissions to control-plane components, nodes, and controllers, but grant *no permissions* to service accounts outside the `kube-system` namespace (beyond discovery permissions given to all authenticated users). -This allows you to grant particular roles to particular service accounts as needed. +This allows you to grant particular roles to particular ServiceAccounts as needed. Fine-grained role bindings provide greater security, but require more effort to administrate. -Broader grants can give unnecessary (and potentially escalating) API access to service accounts, but are easier to administrate. +Broader grants can give unnecessary (and potentially escalating) API access to +ServiceAccounts, but are easier to administrate. In order from most secure to least secure, the approaches are: @@ -949,9 +1081,10 @@ In order from most secure to least secure, the approaches are: If an application does not specify a `serviceAccountName`, it uses the "default" service account. - {{< note >}}Permissions given to the "default" service - account are available to any pod in the namespace that does not - specify a `serviceAccountName`.{{< /note >}} + {{< note >}} + Permissions given to the "default" service account are available to any pod + in the namespace that does not specify a `serviceAccountName`. + {{< /note >}} For example, grant read-only permission within "my-namespace" to the "default" service account: @@ -962,12 +1095,15 @@ In order from most secure to least secure, the approaches are: --namespace=my-namespace ``` - Many [add-ons](/docs/concepts/cluster-administration/addons/) currently run as the "default" service account in the `kube-system` namespace. - To allow those add-ons to run with super-user access, grant cluster-admin permissions to the "default" service account in the `kube-system` namespace. + Many [add-ons](/docs/concepts/cluster-administration/addons/) run as the + "default" service account in the `kube-system` namespace. + To allow those add-ons to run with super-user access, grant cluster-admin + permissions to the "default" service account in the `kube-system` namespace. - {{< note >}}Enabling this means the `kube-system` - namespace contains secrets that grant super-user access to the - API.{{< /note >}} + {{< caution >}} + Enabling this means the `kube-system` namespace contains Secrets + that grant super-user access to your cluster's API. + {{< /caution >}} ```shell kubectl create clusterrolebinding add-on-cluster-admin \ @@ -1006,9 +1142,9 @@ In order from most secure to least secure, the approaches are: If you don't care about partitioning permissions at all, you can grant super-user access to all service accounts. {{< warning >}} - This allows any user with read access - to secrets or the ability to create a pod to access super-user - credentials. + This allows any application full access to your cluster, and also grants + any user with read access to Secrets (or the ability to create any pod) + full access to your cluster. {{< /warning >}} ```shell @@ -1017,10 +1153,11 @@ In order from most secure to least secure, the approaches are: --group=system:serviceaccounts ``` -## Upgrading from 1.5 +## Upgrading from ABAC -Prior to Kubernetes 1.6, many deployments used very permissive ABAC policies, -including granting full API access to all service accounts. +Clusters that originally ran older Kubernetes versions often used +permissive ABAC policies, including granting full API access to all +service accounts. Default RBAC policies grant scoped permissions to control-plane components, nodes, and controllers, but grant *no permissions* to service accounts outside the `kube-system` namespace @@ -1029,28 +1166,31 @@ and controllers, but grant *no permissions* to service accounts outside the `kub While far more secure, this can be disruptive to existing workloads expecting to automatically receive API permissions. Here are two approaches for managing this transition: -### Parallel Authorizers +### Parallel authorizers Run both the RBAC and ABAC authorizers, and specify a policy file that contains -[the legacy ABAC policy](/docs/reference/access-authn-authz/abac/#policy-file-format): +the [legacy ABAC policy](/docs/reference/access-authn-authz/abac/#policy-file-format): ``` ---authorization-mode=RBAC,ABAC --authorization-policy-file=mypolicy.json +--authorization-mode=...,RBAC,ABAC --authorization-policy-file=mypolicy.json ``` -The RBAC authorizer will attempt to authorize requests first. If it denies an API request, -the ABAC authorizer is then run. This means that any request allowed by *either* the RBAC -or ABAC policies is allowed. +To explain that first command line option in detail: if earlier authorizers, such as Node, +deny a request, then the the RBAC authorizer attempts to authorize the API request. If RBAC +also denies that API request, the ABAC authorizer is then run. This means that any request +allowed by *either* the RBAC or ABAC policies is allowed. -When the apiserver is run with a log level of 5 or higher for the RBAC component (`--vmodule=rbac*=5` or `--v=5`), -you can see RBAC denials in the apiserver log (prefixed with `RBAC DENY:`). +When the kube-apiserver is run with a log level of 5 or higher for the RBAC component +(`--vmodule=rbac*=5` or `--v=5`), you can see RBAC denials in the API server log +(prefixed with `RBAC DENY:`). You can use that information to determine which roles need to be granted to which users, groups, or service accounts. -Once you have [granted roles to service accounts](#service-account-permissions) and workloads are running with no RBAC denial messages -in the server logs, you can remove the ABAC authorizer. -## Permissive RBAC Permissions +Once you have [granted roles to service accounts](#service-account-permissions) and workloads +are running with no RBAC denial messages in the server logs, you can remove the ABAC authorizer. -You can replicate a permissive policy using RBAC role bindings. +### Permissive RBAC permissions + +You can replicate a permissive ABAC policy using RBAC role bindings. {{< warning >}} The following policy allows **ALL** service accounts to act as cluster administrators. @@ -1058,7 +1198,7 @@ Any application running in a container receives service account credentials auto and could perform any action against the API, including viewing secrets and modifying permissions. This is not a recommended policy. -``` +```shell kubectl create clusterrolebinding permissive-binding \ --clusterrole=cluster-admin \ --user=admin \ @@ -1067,4 +1207,7 @@ kubectl create clusterrolebinding permissive-binding \ ``` {{< /warning >}} +After you have transitioned to use RBAC, you should adjust the access controls +for your cluster to ensure that these meet your information security needs. + {{% /capture %}} From 4ed2cf77485fa35bbfa8d8d32d13d13bc8afe968 Mon Sep 17 00:00:00 2001 From: "Yuk, Yongsu" Date: Thu, 19 Mar 2020 09:02:43 +0900 Subject: [PATCH 123/140] Edit kube-scheduler glossary. (#19708) * Edit kube-scheduler glossary. #19707 * Apply suggestions from code review Co-Authored-By: Tim Bannister Co-authored-by: Tim Bannister --- content/en/docs/reference/glossary/kube-scheduler.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/reference/glossary/kube-scheduler.md b/content/en/docs/reference/glossary/kube-scheduler.md index e3babcab3f..a1a91a1527 100755 --- a/content/en/docs/reference/glossary/kube-scheduler.md +++ b/content/en/docs/reference/glossary/kube-scheduler.md @@ -11,7 +11,7 @@ tags: - architecture --- Control plane component that watches for newly created -{{< glossary_tooltip term_id="node" >}} with no assigned +{{< glossary_tooltip term_id="pod" text="Pods" >}} with no assigned {{< glossary_tooltip term_id="node" text="node">}}, and selects a node for them to run on. From c944c3ab6d377912a36f639acb77f5c0bcdd50bf Mon Sep 17 00:00:00 2001 From: Karen Bradshaw Date: Thu, 19 Mar 2020 13:32:36 -0400 Subject: [PATCH 124/140] update redirects, netlify changes (#19695) fixes and updates --- static/_redirects | 80 +++++++++++++++++------------------------------ 1 file changed, 29 insertions(+), 51 deletions(-) diff --git a/static/_redirects b/static/_redirects index e032dcbdf9..1c470854bb 100644 --- a/static/_redirects +++ b/static/_redirects @@ -4,24 +4,21 @@ # test at https://play.netlify.com/redirects # ############################################### -/api-ref/ https://github.com/kubernetes/kubernetes/milestones/ 301 /concepts/containers/container-lifecycle-hooks/ /docs/concepts/containers/container-lifecycle-hooks/ 301 -/docs/ /docs/home/ 301 -/de/docs/ /de/docs/home/ 301 -/es/docs/ /es/docs/home/ 301 -/fr/docs/ /fr/docs/home/ 301 -/id/docs/ /id/docs/home/ 301 -/ja/docs/ /ja/docs/home/ 301 -/ko/docs/ /ko/docs/home/ 301 -/no/docs/ /no/docs/home/ 301 -/pl/docs/ /pl/docs/home/ 301 -/pt/docs/ /pt/docs/home/ 301 -/ru/docs/ /ru/docs/home/ 301 -/vi/docs/ /vi/docs/home/ 301 -/zh/docs/ /zh/docs/home/ 301 - -/blog/2018/03/kubernetes-1.10-stabilizing-storage-security-networking/ /blog/2018/03/27/kubernetes-1.10-stabilizing-storage-security-networking/ 301 - +/docs/ /docs/home/ 301! +/de/docs/ /de/docs/home/ 301! +/es/docs/ /es/docs/home/ 301! +/fr/docs/ /fr/docs/home/ 301! +/id/docs/ /id/docs/home/ 301! +/ja/docs/ /ja/docs/home/ 301! +/ko/docs/ /ko/docs/home/ 301! +/no/docs/ /no/docs/home/ 301! +/pl/docs/ /pl/docs/home/ 301! +/pt/docs/ /pt/docs/home/ 301! +/ru/docs/ /ru/docs/home/ 301! +/vi/docs/ /vi/docs/home/ 301! +/zh/docs/ /zh/docs/home/ 301! +/blog/2018/03/kubernetes-1.10-stabilizing-storage-security-networking/ /blog/2018/03/26/kubernetes-1.10-stabilizing-storage-security-networking/ 301! /docs/admin/ /docs/concepts/cluster-administration/cluster-administration-overview/ 301 /docs/admin/add-ons/ /docs/concepts/cluster-administration/addons/ 301 /docs/admin/addons/ /docs/concepts/cluster-administration/addons/ 301 @@ -36,12 +33,11 @@ /docs/admin/dns/ /docs/concepts/services-networking/dns-pod-service/ 301 /docs/admin/etcd/ /docs/tasks/administer-cluster/configure-upgrade-etcd/ 301 /docs/admin/etcd_upgrade/ /docs/tasks/administer-cluster/configure-upgrade-etcd/ 301 -/docs/admin/extensible-admission-controllers.md /docs/admin/extensible-admission-controllers/ 301 +/docs/admin/extensible-admission-controllers.md /docs/reference/access-authn-authz/extensible-admission-controllers/ 301 /docs/admin/garbage-collection/ /docs/concepts/cluster-administration/kubelet-garbage-collection/ 301 /docs/admin/ha-master-gce/ /docs/tasks/administer-cluster/highly-available-master/ 301 /docs/admin/ha-master-gce.md/ /docs/tasks/administer-cluster/highly-available-master/ 301 -/docs/admin/high-availability/ /docs/admin/high-availability/building/ 301 -/docs/admin/kubeadm-upgrade-1-7/ /docs/tasks/administer-cluster/upgrade-downgrade/kubeadm-upgrade-1-7/ 301 +/docs/admin/high-availability/ /docs/setup/production-environment/tools/kubeadm/high-availability/ 301 /docs/admin/kubelet-authentication-authorization/ /docs/reference/command-line-tools-reference/kubelet-authentication-authorization/ 301 /docs/admin/kubelet-tls-bootstrapping/ /docs/reference/command-line-tools-reference/kubelet-tls-bootstrapping/ 301 /docs/admin/limitrange/ /docs/tasks/administer-cluster/cpu-memory-limit/ 301 @@ -56,6 +52,7 @@ /docs/admin/node-allocatable/ /docs/tasks/administer-cluster/reserve-compute-resources/ 301 /docs/admin/node-allocatable.md /docs/tasks/administer-cluster/reserve-compute-resources/ 301 /docs/admin/node-conformance.md /docs/admin/node-conformance/ 301 +/docs/admin/node-conformance/ /docs/setup/best-practices/node-conformance/ 301 /docs/admin/node-problem/ /docs/tasks/debug-application-cluster/monitor-node-health/ 301 /docs/admin/out-of-resource/ /docs/tasks/administer-cluster/out-of-resource/ 301 /docs/admin/rescheduler/ /docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods/ 301 @@ -65,18 +62,10 @@ /docs/admin/static-pods/ /docs/tasks/administer-cluster/static-pod/ 301 /docs/admin/sysctls/ /docs/tasks/administer-cluster/sysctl-cluster/ 301 /docs/admin/resource-quota/ /docs/concepts/policy/resource-quotas/ 301 -/docs/admin/upgrade-1-6/ /docs/tasks/administer-cluster/upgrade-downgrade/upgrade-1-6/ 301 /docs/api/ /docs/concepts/overview/kubernetes-api/ 301 /docs/api-reference/labels-annotations-taints/ /docs/reference/labels-annotations-taints/ 301 -/docs/api-reference/v1.6/* https://v1-6.docs.kubernetes.io/docs/reference/ 301 -/docs/api-reference/v1.7/* https://v1-7.docs.kubernetes.io/docs/reference/ 301 -/docs/api-reference/v1.8/* https://v1-8.docs.kubernetes.io/docs/api-reference/v1.8/:splat 301 -/docs/api-reference/v1.9/ https://v1-9.docs.kubernetes.io/docs/reference/generated/kubernetes-api/v1.9/ 301 -/docs/api-reference/v1/definitions/ /docs/reference/generated/kubernetes-api/v1.10/ 301 -/docs/api-reference/v1/definitions.html /docs/reference/generated/kubernetes-api/v1.10/ 301 -/docs/api-reference/v1/operations/ /docs/reference/generated/kubernetes-api/v1.10/ 301 /docs/concepts/abstractions/controllers/garbage-collection/ /docs/concepts/workloads/controllers/garbage-collection/ 301 /docs/concepts/abstractions/controllers/statefulsets/ /docs/concepts/workloads/controllers/statefulset/ 301 @@ -132,7 +121,7 @@ /docs/concepts/workloads/controllers/deployment/docs/concepts/workloads/pods/pod/ /docs/concepts/workloads/pods/pod/ 301 /docs/concepts/workloads/controllers/job/ /docs/concepts/workloads/controllers/jobs-run-to-completion/ 301 /docs/concepts/workloads/controllers/statefulsets/ /docs/concepts/workloads/controllers/statefulset/ 301 -/docs/concepts/workloads/controllers/statefulset.md /docs/concepts/workloads/controllers/statefulset/ 301 +/docs/concepts/workloads/controllers/statefulset.md /docs/concepts/workloads/controllers/statefulset/ 301! /docs/concepts/workloads/pods/init-containers/Kubernetes/ /docs/concepts/workloads/pods/init-containers/ 301 /docs/consumer-guideline/pod-security-coverage/ /docs/concepts/policy/pod-security-policy/ 301 @@ -209,19 +198,14 @@ /docs/reference/glossary/maintainer/ /docs/reference/glossary/approver/ 301 /docs/reference/kubectl/kubectl/kubectl_*.md /docs/reference/generated/kubectl/kubectl-commands#:splat 301 -/docs/reference/workloads-18-19/ https://v1-9.docs.kubernetes.io/docs/reference/workloads-18-19/ 301 /docs/reporting-security-issues/ /security/ 301 -/docs/resources-reference/1_6/* /docs/resources-reference/v1.6/ 301 -/docs/resources-reference/1_7/* /docs/resources-reference/v1.7/ 301 -/docs/resources-reference/v1.8/* /docs/api-reference/v1.8/:splat 301 - /docs/roadmap/ https://github.com/kubernetes/kubernetes/milestones/ 301 /docs/samples/ /docs/tutorials/ 301 /docs/stable/user-guide/labels/ /docs/concepts/overview/working-with-objects/labels/ 301 -/docs/tasks/access-application-cluster/access-cluster.md /docs/tasks/access-application-cluster/access-cluster/ 301 +/docs/tasks/access-application-cluster/access-cluster.md /docs/tasks/access-application-cluster/access-cluster/ 301! /docs/tasks/access-application-cluster/authenticate-across-clusters-kubeconfig/ /docs/tasks/access-application-cluster/configure-access-multiple-clusters/ 301 /docs/tasks/access-kubernetes-api/access-kubernetes-api/http-proxy-access-api/ /docs/tasks/access-kubernetes-api/http-proxy-access-api/ 301 /docs/tasks/access-kubernetes-api/extend-api-custom-resource-definitions/ /docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions/ 301 @@ -244,9 +228,9 @@ /docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade-ha-1-13 /docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/ 301 /docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade-1-14 /docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/ 301 /docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade-1-15 /docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/ 301 -/docs/tasks/administer-cluster/kubeadm-upgrade-1-7/ /docs/tasks/administer-cluster/upgrade-downgrade/kubeadm-upgrade-1-7/ 301 -/docs/tasks/administer-cluster/kubeadm-upgrade-1-8/ /docs/tasks/administer-cluster/upgrade-downgrade/kubeadm-upgrade-1-8/ 301 -/docs/tasks/administer-cluster/kubeadm-upgrade-1-9/ /docs/tasks/administer-cluster/upgrade-downgrade/kubeadm-upgrade-1-9/ 301 +#/docs/tasks/administer-cluster/kubeadm-upgrade-1-7/ /docs/tasks/administer-cluster/upgrade-downgrade/kubeadm-upgrade-1-7/ 301 +#/docs/tasks/administer-cluster/kubeadm-upgrade-1-8/ /docs/tasks/administer-cluster/upgrade-downgrade/kubeadm-upgrade-1-8/ 301 +#/docs/tasks/administer-cluster/kubeadm-upgrade-1-9/ /docs/tasks/administer-cluster/upgrade-downgrade/kubeadm-upgrade-1-9/ 301 /docs/tasks/administer-cluster/kubeadm-upgrade-ha/ /docs/tasks/administer-cluster/upgrade-downgrade/kubeadm-upgrade-ha/ 301 /docs/tasks/administer-cluster/kube-router-network-policy/ /docs/tasks/administer-cluster/network-policy-provider/kube-router-network-policy/ 301 /docs/tasks/administer-cluster/memory-constraint-namespace/ /docs/tasks/administer-cluster/manage-resources/memory-constraint-namespace/ 301 @@ -260,7 +244,7 @@ /docs/tasks/administer-cluster/running-cloud-controller.md /docs/tasks/administer-cluster/running-cloud-controller/ 301 /docs/tasks/administer-cluster/share-configuration/ /docs/tasks/access-application-cluster/configure-access-multiple-clusters/ 301 /docs/tasks/administer-cluster/static-pod/ /docs/tasks/configure-pod-container/static-pod/ 301 -/docs/tasks/administer-cluster/upgrade-1-6/ /docs/tasks/administer-cluster/upgrade-downgrade/upgrade-1-6/ 301 +#/docs/tasks/administer-cluster/upgrade-1-6/ /docs/tasks/administer-cluster/upgrade-downgrade/upgrade-1-6/ 301 /docs/tasks/administer-cluster/weave-network-policy/ /docs/tasks/administer-cluster/network-policy-provider/weave-network-policy/ 301 /docs/tasks/configure-pod-container/apply-resource-quota-limit/ /docs/tasks/administer-cluster/apply-resource-quota-limit/ 301 /docs/tasks/configure-pod-container/assign-cpu-ram-container/ /docs/tasks/configure-pod-container/assign-memory-resource/ 301 @@ -378,13 +362,13 @@ /docs/user-guide/kubeconfig-file/ /docs/tasks/access-application-cluster/authenticate-across-clusters-kubeconfig/ 301 /docs/user-guide/kubectl-overview/ /docs/reference/kubectl/overview/ /docs/user-guide/kubectl/ /docs/reference/generated/kubectl/kubectl-options/ -/docs/user-guide/kubectl/v1.8/* https://v1-8.docs.kubernetes.io/docs/reference/generated/kubectl/kubectl-commands/:splat 301 -/docs/user-guide/kubectl/v1.9/* https://v1-9.docs.kubernetes.io/docs/reference/generated/kubectl/kubectl-commands/:splat 301 -/docs/user-guide/kubectl/v1.10/* /docs/reference/generated/kubectl/kubectl-commands/:splat 301 +#/docs/user-guide/kubectl/v1.8/* https://v1-8.docs.kubernetes.io/docs/reference/generated/kubectl/kubectl-commands/:splat 301 +#/docs/user-guide/kubectl/v1.9/* https://v1-9.docs.kubernetes.io/docs/reference/generated/kubectl/kubectl-commands/:splat 301 +#/docs/user-guide/kubectl/v1.10/* /docs/reference/generated/kubectl/kubectl-commands/:splat 301 /docs/user-guide/kubectl-conventions/ /docs/reference/kubectl/conventions/ /docs/user-guide/kubectl-cheatsheet/ /docs/reference/kubectl/cheatsheet/ /docs/user-guide/kubectl/kubectl_*/ /docs/reference/generated/kubectl/kubectl-commands#:splat 301 -/docs/user-guide/kubectl/v1.6/node_modules/* https://v1-6.docs.kubernetes.io/docs/user-guide/kubectl/v1.6/ 301 +#/docs/user-guide/kubectl/v1.6/node_modules/* https://v1-6.docs.kubernetes.io/docs/user-guide/kubectl/v1.6/ 301 /docs/user-guide/labels/ /docs/concepts/overview/working-with-objects/labels/ 301 /docs/user-guide/liveness/ /docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/ 301 /docs/user-guide/load-balancer/ /docs/tasks/access-application-cluster/create-external-load-balancer/ 301 @@ -445,25 +429,19 @@ /horizontal-pod-autoscaler/ /docs/tasks/run-application/horizontal-pod-autoscale/ 301 /kubernetes/ /docs/ 301 /kubernetes-bootcamp/* /docs/tutorials/kubernetes-basics/ 301 -/kubernetes/swagger-spec/ https://github.com/kubernetes/kubernetes/tree/master/api/swagger-spec/ 301 /kubernetes/third_party/swagger-ui/ /docs/reference/ 301 /latest/docs/ /docs/home/ 301 /media/ /docs/community 301 /news/ /docs/community 301 /resource-quota/ /docs/concepts/policy/resource-quotas/ 301 /serviceaccount/token/ /docs/tasks/configure-pod-container/configure-service-account/ 301 -/swagger-spec/* https://github.com/kubernetes/kubernetes/tree/master/api/swagger-spec/ 301 /third_party/swagger-ui/* /docs/reference/ 301 -/v1.1/docs/admin/networking.html /docs/concepts/cluster-administration/networking/ 301 - -https://kubernetes-io-v1-7.netlify.com/* https://v1-7.docs.kubernetes.io/:splat 301 /docs/admin/cloud-controller-manager/ /docs/reference/generated/cloud-controller-manager/ 301 /docs/admin/kube-apiserver/ /docs/reference/generated/kube-apiserver/ 301 /docs/admin/kube-controller-manager/ /docs/reference/generated/kube-controller-manager/ 301 /docs/admin/kube-proxy/ /docs/reference/generated/kube-proxy/ 301 /docs/admin/kube-scheduler/ /docs/reference/generated/kube-scheduler/ 301 -/docs/admin/kube-scheduler/ /docs/reference/generated/kube-scheduler/ 301 /docs/admin/kubeadm/ /docs/reference/generated/kubeadm/ 301 /docs/admin/kubelet/ /docs/reference/generated/kubelet/ 301 /docs/admin/federation-controller-manager/ /docs/reference/generated/federation-controller-manager/ 301 @@ -484,6 +462,7 @@ https://kubernetes-io-v1-7.netlify.com/* https://v1-7.docs.kubernetes.io/:spl /docs/admin/admission-controllers/ /docs/reference/access-authn-authz/admission-controllers/ 301 /docs/admin/authentication/ /docs/reference/access-authn-authz/authentication/ 301 /docs/admin/bootstrap-tokens/ /docs/reference/access-authn-authz/bootstrap-tokens/ 301 + /docs/admin/extensible-admission-controllers/ /docs/reference/access-authn-authz/extensible-admission-controllers/ 301 /docs/admin/service-accounts-admin/ /docs/reference/access-authn-authz/service-accounts-admin/ 301 /docs/admin/authorization/abac/ /docs/reference/access-authn-authz/abac/ 301 @@ -491,8 +470,7 @@ https://kubernetes-io-v1-7.netlify.com/* https://v1-7.docs.kubernetes.io/:spl /docs/admin/authorization/rbac/ /docs/reference/access-authn-authz/rbac/ 301 /docs/admin/authorization/webhook/ /docs/reference/access-authn-authz/webhook/ 301 /docs/admin/authorization/ /docs/reference/access-authn-authz/authorization/ 301 - -/docs/admin/high-availability/building/ /docs/setup/independent/high-availability/ 301 +/docs/admin/high-availability/building/ /docs/setup/production-environment/tools/kubeadm/high-availability/ 301 /code-of-conduct/ /community/code-of-conduct/ 301 /docs/setup/version-skew-policy/ /docs/setup/release/version-skew-policy/ 301 From 4ca2b2c3244694a899542eff7719875e36d24875 Mon Sep 17 00:00:00 2001 From: Karen Bradshaw Date: Thu, 19 Mar 2020 13:58:36 -0400 Subject: [PATCH 125/140] clean up css for tablet, nav (#19477) --- assets/sass/_base.sass | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/assets/sass/_base.sass b/assets/sass/_base.sass index 2315ae5e94..c5a576995c 100644 --- a/assets/sass/_base.sass +++ b/assets/sass/_base.sass @@ -109,6 +109,7 @@ header box-shadow: 0 0 0 transparent transition: 0.3s text-align: center + overflow: hidden .logo @@ -244,8 +245,7 @@ header background-color: white #mainNav - display: none - + h5 color: $blue font-weight: normal From 60a265e6bdb62d51429d59ff434da40422c146df Mon Sep 17 00:00:00 2001 From: Lorenzo Paris Date: Thu, 19 Mar 2020 11:16:36 -0700 Subject: [PATCH 126/140] Update authentication.md with link to anon reqs (#19706) Suggest adding a link in case the reader is not familiar with what "anonymous requests" means. This paragraph also includes a reference to "anonymous user," so I think the link is warranted. Thanks. --- content/en/docs/reference/access-authn-authz/authentication.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/reference/access-authn-authz/authentication.md b/content/en/docs/reference/access-authn-authz/authentication.md index 0089c08a91..cd941bcadb 100644 --- a/content/en/docs/reference/access-authn-authz/authentication.md +++ b/content/en/docs/reference/access-authn-authz/authentication.md @@ -33,7 +33,7 @@ stored as `Secrets`, which are mounted into pods allowing in-cluster processes to talk to the Kubernetes API. API requests are tied to either a normal user or a service account, or are treated -as anonymous requests. This means every process inside or outside the cluster, from +as [anonymous requests](#anonymous-requests). This means every process inside or outside the cluster, from a human user typing `kubectl` on a workstation, to `kubelets` on nodes, to members of the control plane, must authenticate when making requests to the API server, or be treated as an anonymous user. From e19f9121753447c1dbd435c7ffc60cc3b4a944e9 Mon Sep 17 00:00:00 2001 From: Alena Prokharchyk Date: Thu, 19 Mar 2020 12:08:36 -0700 Subject: [PATCH 127/140] Lease heartbeat: more description on error handling (#19673) --- content/en/docs/concepts/architecture/nodes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/concepts/architecture/nodes.md b/content/en/docs/concepts/architecture/nodes.md index 338e9a2408..97188ee9ad 100644 --- a/content/en/docs/concepts/architecture/nodes.md +++ b/content/en/docs/concepts/architecture/nodes.md @@ -184,7 +184,7 @@ a Lease object. timeout for unreachable nodes). - The kubelet creates and then updates its Lease object every 10 seconds (the default update interval). Lease updates occur independently from the - `NodeStatus` updates. + `NodeStatus` updates. If the Lease update fails, the kubelet retries with exponential backoff starting at 200 milliseconds and capped at 7 seconds. #### Reliability From 735259f7326e30191cf5826efafe2473b07b771c Mon Sep 17 00:00:00 2001 From: Tim Bannister Date: Thu, 19 Mar 2020 20:32:36 +0000 Subject: [PATCH 128/140] Fix ordering (#19729) Put entries in alphabetical order. Fixup for commit 55de25393429a94c92b98a883cbd30b0062c8eeb. --- i18n/en.toml | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/i18n/en.toml b/i18n/en.toml index 48c0c5f469..c3a5ab69ae 100644 --- a/i18n/en.toml +++ b/i18n/en.toml @@ -58,6 +58,9 @@ other = "Was this page helpful?" [feedback_yes] other = "Yes" +[input_placeholder_email_address] +other = "email address" + [latest_version] other = "latest version." @@ -188,7 +191,4 @@ other = "To check the version, enter " other = "Warning:" [whatsnext_heading] -other = "What's next" - -[input_placeholder_email_address] -other = "email address" \ No newline at end of file +other = "What's next" \ No newline at end of file From 3b241356e50f05003bee139e6eb7b3376c17eeb1 Mon Sep 17 00:00:00 2001 From: Maciej Filocha <12587791+mfilocha@users.noreply.github.com> Date: Fri, 20 Mar 2020 11:14:36 +0100 Subject: [PATCH 129/140] Synchronize Polish translation with master (#19681) Update translated content to uptream master version as for March 15th. Commits from 90ee7559a0fcb73da45e2e5ce3fa2ee620f2cd2a to d4167faa2ce59612b36e4edcc7357c1a63131bbc. --- content/pl/_index.html | 4 +- content/pl/docs/concepts/_index.md | 2 +- .../pl/docs/concepts/overview/components.md | 8 +- .../docs/concepts/overview/kubernetes-api.md | 2 +- content/pl/docs/contribute/_index.md | 75 ++++++------------- content/pl/docs/reference/_index.md | 24 +++--- content/pl/docs/reference/glossary/cluster.md | 6 +- .../reference/glossary/container-runtime.md | 8 +- .../glossary/kube-controller-manager.md | 2 +- .../docs/reference/glossary/kube-scheduler.md | 4 +- content/pl/docs/setup/_index.md | 9 +-- content/pl/docs/tasks/_index.md | 4 - content/pl/docs/tutorials/_index.md | 2 - content/pl/docs/tutorials/hello-minikube.md | 31 ++++---- .../create-cluster/cluster-intro.html | 2 +- .../deploy-app/deploy-interactive.html | 9 +++ .../expose/expose-intro.html | 5 -- content/pl/examples/minikube/Dockerfile | 2 +- 18 files changed, 80 insertions(+), 119 deletions(-) diff --git a/content/pl/_index.html b/content/pl/_index.html index 4d7de6d3b4..4e55c429bd 100644 --- a/content/pl/_index.html +++ b/content/pl/_index.html @@ -45,12 +45,12 @@ Kubernetes jako projekt open-source daje Ci wolność wyboru ⏤ skorzystaj z pr


    - Weź udział w KubeCon w Amsterdamie 30.03-2.04.2020 + Weź udział w KubeCon w Amsterdamie (lipiec/sierpień)



    - Weź udział w KubeCon w Szanghaju 28-30.07.2020 + Weź udział w KubeCon w Bostonie 17-20.11.2020

    diff --git a/content/pl/docs/concepts/_index.md b/content/pl/docs/concepts/_index.md index 8be3830b9c..3f147f5f43 100644 --- a/content/pl/docs/concepts/_index.md +++ b/content/pl/docs/concepts/_index.md @@ -26,7 +26,7 @@ Gdy tylko zdefiniujesz zamierzony stan, warstwa sterowania Kubernetes (*Kubernet ## Obiekty Kubernetes -Kubernetes składa się z różnych abstrakcyjnych obiektów, które reprezentują stan systemu: wdrożone aplikacje i zadania w kontenerach, powiązane zasoby sieciowe i dyskowe oraz inne informacje o tym, co się dzieje na klasterze. Te abstrakcyjne obiekty są reprezentowane przez API Kubernetes. [Opis Obiektów w Kubernetesie](/docs/concepts/overview/working-with-objects/kubernetes-objects/) zawiera więcej szczegółów na ten temat. +Kubernetes składa się z różnych abstrakcyjnych obiektów, które reprezentują stan systemu: wdrożone aplikacje i zadania w kontenerach, powiązane zasoby sieciowe i dyskowe oraz inne informacje o tym, co się dzieje na klasterze. Te abstrakcyjne obiekty są reprezentowane przez API Kubernetes. [Opis obiektów w Kubernetesie](/docs/concepts/overview/working-with-objects/kubernetes-objects/#kubernetes-objects) zawiera więcej szczegółów na ten temat. Do podstawowych obiektów Kubernetes należą: diff --git a/content/pl/docs/concepts/overview/components.md b/content/pl/docs/concepts/overview/components.md index eda8ed7519..966f67b004 100644 --- a/content/pl/docs/concepts/overview/components.md +++ b/content/pl/docs/concepts/overview/components.md @@ -10,7 +10,7 @@ card: {{% capture overview %}} W wyniku instalacji Kubernetes otrzymujesz klaster. -{{< glossary_definition term_id="cluster" length="all" prepend="Klaster to">}} +{{< glossary_definition term_id="cluster" length="all" prepend="Klaster Kubernetes to">}} W tym dokumencie opisujemy składniki niezbędne do zbudowania kompletnego, poprawnie działającego klastra Kubernetes. @@ -20,11 +20,11 @@ Poniższy rysunek przedstawia klaster Kubernetes i powiązania pomiędzy jego r {{% /capture %}} {{% capture body %}} -## Master — częsci składowe +## Częsci składowe warstwy sterowania -Komponenty *master* odpowiadają za warstwę sterowania klastra. Podejmują ogólne decyzje dotyczące klastra (np. zlecanie zadań), wykrywają i reagują na zdarzenia w klastrze (przykładowo, start nowego {{< glossary_tooltip text="poda" term_id="pod">}}, kiedy wartość `replicas` dla deploymentu nie zgadza się z faktyczną liczbą replik). +Komponenty warstwy sterowania podejmują ogólne decyzje dotyczące klastra (np. zlecanie zadań), a także wykrywają i reagują na zdarzenia w klastrze (przykładowo, start nowego {{< glossary_tooltip text="poda" term_id="pod">}}, kiedy wartość `replicas` dla deploymentu nie zgadza się z faktyczną liczbą replik). -Komponenty *master* mogą być uruchomione na dowolnej maszynie w klastrze. Dla uproszczenia skrypty instalacyjne zazwyczaj startują wszystkie składniki na tej samej maszynie i jednocześnie nie pozwalają na uruchamianie na niej kontenerów użytkowników. Na stronie [Tworzenie Wysoko Dostępnych Klastrów](/docs/admin/high-availability/) jest więcej informacji o konfiguracji typu *multi-master-VM*. +Komponenty warstwy sterowania mogą być uruchomione na dowolnej maszynie w klastrze. Dla uproszczenia jednak skrypty instalacyjne zazwyczaj startują wszystkie składniki na tej samej maszynie i jednocześnie nie pozwalają na uruchamianie na niej kontenerów użytkowników. Na stronie [Tworzenie Wysoko Dostępnych Klastrów](/docs/admin/high-availability/) jest więcej informacji o konfiguracji typu *multi-master-VM*. ### kube-apiserver diff --git a/content/pl/docs/concepts/overview/kubernetes-api.md b/content/pl/docs/concepts/overview/kubernetes-api.md index 5813ada50e..eb86295321 100644 --- a/content/pl/docs/concepts/overview/kubernetes-api.md +++ b/content/pl/docs/concepts/overview/kubernetes-api.md @@ -57,7 +57,7 @@ GET /swagger-2.0.0.pb-v1.gz | GET /openapi/v2 **Accept**: application/com.github W Kubernetes zaimplementowany jest alternatywny format serializacji na potrzeby API oparty o Protobuf, który jest przede wszystkim przeznaczony na potrzeby wewnętrznej komunikacji w klastrze i opisany w [design proposal](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/api-machinery/protobuf.md). Pliki IDL dla każdego ze schematów można znaleźć w pakietach Go, które definiują obiekty API. Przed wersją 1.14, apiserver Kubernetes udostępniał też specyfikację API [Swagger v1.2](http://swagger.io/) poprzez `/swaggerapi`. -Ten punkt końcowy jest fazie wycofywania i zostanie ostatecznie usunięty w wersji Kubernetes 1.14. +Ten punkt końcowy został skierowany do wycofania i ostatecznie usunięty w wersji Kubernetes 1.14. ## Obsługa wersji API diff --git a/content/pl/docs/contribute/_index.md b/content/pl/docs/contribute/_index.md index ad2a2adb70..bd40241596 100644 --- a/content/pl/docs/contribute/_index.md +++ b/content/pl/docs/contribute/_index.md @@ -13,65 +13,38 @@ lub strony www Kubernetesa! Nieważne, czy dopiero poznajesz projekt, czy jeste z nami już od dawna, czy uważasz się za programistę, użytkownika, czy po prostu nie możesz patrzeć na literówki. -Więcej informacji na temat zawartości dokumentacji Kubernetesa i jej stylu, -znajdziesz w - [Opisie stylu dokumentacji](/docs/contribute/style/). +{{% /capture %}} {{% capture body %}} -## Rodzaje uczestnictwa w procesie tworzenia dokumentacji +## Od czego zacząć? -- _Członek_ (_member_) organizacji Kubernetes, który [podpisał CLA](/docs/contribute/start#sign-the-cla) - i poświęcił swój czas oraz wysiłek na rzecz projektu. Dokument - [Członkostwo w organizacji](https://github.com/kubernetes/community/blob/master/community-membership.md) - zawiera szczegóły z tym związane. -- _Recenzent_ (_reviewer_) SIG Docs to członek organizacji Kubernetes, który zgłosił - swoją chęć weryfikacji propozycji zmian w dokumentacji (PR) i został dodany - do odpowiedniej grupy GitHub i pliku 'OWNERS' w repozytorium GitHub przez - osobę zatwierdzającą SIG Docs. -- _Osoba zatwierdzająca_ (_approver_) SIG Docs to członek organizacji o uznanej reputacji, - który wykazał się długotrwałym zaangażowaniem w prace projektu. - Osoba zatwierdzająca może włączać propozycje zmian do repozytoriów i publikować - treści w imieniu organizacji Kubernetes. - Osoby zatwierdzające mogą również reprezentować SIG Docs na szerszym forum - społeczności Kubernetes. - Niektóre wymagania związane z tą rolą, jak na przykład koordynacja kolejnego wydania, - wymagają poświęcenia znacznej ilości czasu. +Każdy może otworzyć zgłoszenie, które zawiera opis problemu czy oczekiwane usprawnienia dokumentacji lub samemu zaproponować zmianę poprzez *pull request* (PR). +Do realizacji niektórych zadań potrzeba wyższego poziomu zaufania i odpowiednich uprawnień w organizacji Kubernetes. +Zajrzyj do [Participating in SIG Docs](/docs/contribute/participating/) po więcej szczegółów +dotyczących ról i uprawnień. -## Sposoby współpracy przy tworzeniu dokumentacji +Dokumentacja Kubernetesa znajduje się w repozytorium GitHub. Zapraszamy wszystkich +do aktywnych działań na rzecz jej rozwoju, niemniej aby móc sprawnie funkcjonować w społeczności Kubernetes, +wymagana jest pewna biegłość w korzystaniu z git i GitHuba. -Poniższa lista podzielona jest na rzeczy, które może robić każdy, te, które może -robić członek organizacji Kubernetes oraz na takie, które wymagają wyższych uprawnień -i znajomości procesów SIG Docs. W miarę postępującej współpracy, będziesz mógł lepiej -zrozumieć niektóre narzędzia czy decyzje, które zostały wcześniej podjęte -na poziomie organizacyjnym. +Aby zaangażować się w prace nad dokumentacją należy: -Ta lista nie wyczerpuje wszystkich możliwości udziału, ale powinna być pomocna -na początku. +1. Podpisać [Contributor License Agreement](https://github.com/kubernetes/community/blob/master/CLA.md) CNCF. +2. Zapoznać się z [repozytorium dokumentacji](https://github.com/kubernetes/website) i z [generatorem statycznej strony](https://gohugo.io) www. +3. Zrozumieć podstawowe procesy [ulepszania zawartości](https://kubernetes.io/docs/contribute/start/#improve-existing-content) i [recenzowania propozycji zmian](https://kubernetes.io/docs/contribute/start/#review-docs-pull-requests). + +## Najlepsze praktyki zgłaszania zmian + +- Opis GIT commit powinien być jasny i zrozumiały. +- Należy używać _Github Special Keywords_, które odwołują się do zgłoszenia _(issue)_ i automatycznie je zamykają, kiedy PR zostaje zaakceptowany. +- Kiedy wprowadzasz drobne zmiany do PR, takie jak literówki czy poprawki stylu lub gramatyki, pamiętaj o ich zgrupowaniu _(squash)_, aby uniknąć sytuacji, kiedy mamy dużą liczbę commitów dla stosunkowo niewielkiej zmiany. +- Dołącz dobry opis PR, który tłumaczy zmiany w kodzie, powód dla tych zmian i wszystkie informacje wystarczające, aby recenzent zrozumiał Twój PR. +- Dodatkowa literatura: + - [chris.beams.io/posts/git-commit/](https://chris.beams.io/posts/git-commit/) + - [github.com/blog/1506-closing-issues-via-pull-requests ](https://github.com/blog/1506-closing-issues-via-pull-requests) + - [davidwalsh.name/squash-commits-git ](https://davidwalsh.name/squash-commits-git ) -- [Każdy](/docs/contribute/start/) - - Otwieranie wszelkiego rodzaju zgłoszeń, względem których mogą zostać podjęte jakieś działania -- [Członek](/docs/contribute/start/) - - Ulepszanie istniejącej dokumentacji - - Zgłaszanie pomysłów na ulepszenia poprzez komunikator [Slack](http://slack.k8s.io/) lub [listę dystrybucyjną SIG docs](https://groups.google.com/forum/#!forum/kubernetes-sig-docs) - - Zwiększanie dostępności dokumentacji - - Zgłaszanie niewiążących uwag do propozycji zmian (PR) - - Pisanie bloga lub studium przypadku -- [Recenzent](/docs/contribute/intermediate/) - - Opisywanie nowych funkcjonalności - - Przydzielanie kategorii i klasyfikowanie zgłoszeń - - Recenzowanie propozycji zmian - - Tworzenie schematów, grafik, osadzonych prezentacji (_screencasts_) i filmów - - Tłumaczenie - - Współtworzenie zawartości innych repozytoriów jako przedstawiciel zespołu dokumentacji - - Opracowywanie osadzonych w oprogramowaniu komunikatów dla użytkownika - - Ulepszanie komentarzy w oprogramowaniu, Godoc -- [Osoba zatwierdzająca](/docs/contribute/advanced/) - - Publikowanie dostarczonych treści poprzez zatwierdzanie propozycji zmian i włączanie ich do repozytorium - - Udział w pracach zespołu przygotowującego nowe wydanie Kubernetesa jako przedstawiciel zespołu dokumentacji - - Proponowanie ulepszeń wytycznych dotyczących stylu - - Proponowanie ulepszeń testowania dokumentacji - - Proponowanie ulepszeń strony Kubernetes lub innych narzędzi ## Inne metody współpracy diff --git a/content/pl/docs/reference/_index.md b/content/pl/docs/reference/_index.md index 92177da44b..41a213e5e3 100644 --- a/content/pl/docs/reference/_index.md +++ b/content/pl/docs/reference/_index.md @@ -17,12 +17,7 @@ Tutaj znajdziesz dokumentację źródłową Kubernetes. ## Dokumentacja API * [Kubernetes API Overview](/docs/reference/using-api/api-overview/) - Ogólne informacje na temat Kubernetes API. -* Wersje Kubernetes API - * [1.17](/docs/reference/generated/kubernetes-api/v1.17/) - * [1.16](/docs/reference/generated/kubernetes-api/v1.16/) - * [1.15](/docs/reference/generated/kubernetes-api/v1.15/) - * [1.14](/docs/reference/generated/kubernetes-api/v1.14/) - * [1.13](/docs/reference/generated/kubernetes-api/v1.13/) + * [Dokumentacja źródłowa Kubernetes API {{< latest-version >}}](/docs/reference/generated/kubernetes-api/{{< latest-version >}}/) ## Biblioteki klientów API @@ -37,18 +32,17 @@ biblioteki to: ## Dokumentacja poleceń tekstowych *(CLI)* -* [kubectl](/docs/user-guide/kubectl-overview) - Główne narzędzie tekstowe (linii poleceń) do zarządzania klastrem Kubernetes. - * [JSONPath](/docs/user-guide/jsonpath/) - Podręcznik składni [wyrażeń JSONPath](http://goessner.net/articles/JsonPath/) dla kubectl. -* [kubeadm](/docs/admin/kubeadm/) - Narzędzie tekstowe do łatwego budowania klastra Kubernetes spełniającego niezbędne wymogi bezpieczeństwa. -* [kubefed](/docs/admin/kubefed/) - Narzędzie tekstowe poleceń do zarządzania klastrami w federacji. +* [kubectl](/docs/reference/kubectl/overview/) - Główne narzędzie tekstowe (linii poleceń) do zarządzania klastrem Kubernetes. + * [JSONPath](/docs/reference/kubectl/jsonpath/) - Podręcznik składni [wyrażeń JSONPath](http://goessner.net/articles/JsonPath/) dla kubectl. +* [kubeadm](/docs/reference/setup-tools/kubeadm/kubeadm/) - Narzędzie tekstowe do łatwego budowania klastra Kubernetes spełniającego niezbędne wymogi bezpieczeństwa. ## Dokumentacja konfiguracji -* [kubelet](/docs/admin/kubelet/) - Główny agent działający na każdym węźle. Kubelet pobiera zestaw definicji PodSpecs i gwarantuje, że opisane przez nie kontenery poprawnie działają. -* [kube-apiserver](/docs/admin/kube-apiserver/) - REST API, które sprawdza poprawność i konfiguruje obiekty API, takie jak pody, serwisy czy kontrolery replikacji. -* [kube-controller-manager](/docs/admin/kube-controller-manager/) - Proces wykonujący główne pętle sterowania Kubernetes. -* [kube-proxy](/docs/admin/kube-proxy/) - Przekazuje bezpośrednio dane przepływające w transmisji TCP/UDP lub dystrybuuje ruch TCP/UDP zgodnie ze schematem *round-robin* pomiędzy usługi back-endu. -* [kube-scheduler](/docs/admin/kube-scheduler/) - Scheduler odpowiada za dostępność, wydajność i zasoby. +* [kubelet](/docs/reference/command-line-tools-reference/kubelet/) - Główny agent działający na każdym węźle. Kubelet pobiera zestaw definicji PodSpecs i gwarantuje, że opisane przez nie kontenery poprawnie działają. +* [kube-apiserver](/docs/reference/command-line-tools-reference/kube-apiserver/) - REST API, które sprawdza poprawność i konfiguruje obiekty API, takie jak pody, serwisy czy kontrolery replikacji. +* [kube-controller-manager](/docs/reference/command-line-tools-reference/kube-controller-manager/) - Proces wykonujący główne pętle sterowania Kubernetes. +* [kube-proxy](/docs/reference/command-line-tools-reference/kube-proxy/) - Przekazuje bezpośrednio dane przepływające w transmisji TCP/UDP lub dystrybuuje ruch TCP/UDP zgodnie ze schematem *round-robin* pomiędzy usługi back-endu. +* [kube-scheduler](/docs/reference/command-line-tools-reference/kube-scheduler/) - Scheduler odpowiada za dostępność, wydajność i zasoby. ## Dokumentacja projektowa diff --git a/content/pl/docs/reference/glossary/cluster.md b/content/pl/docs/reference/glossary/cluster.md index 0021caefe0..650db52c3d 100755 --- a/content/pl/docs/reference/glossary/cluster.md +++ b/content/pl/docs/reference/glossary/cluster.md @@ -4,14 +4,14 @@ id: cluster date: 2019-06-15 full_link: short_description: > - Zestaw maszyn, nazywanych węzłami, na których uruchamiane są aplikacje zarządzane przez Kubernetes. Klaster posiada przynajmniej jeden węzeł roboczy (*node*) i jeden węzeł typu master (*master node*). + Zestaw maszyn roboczych, nazywanych węzłami, na których uruchamiane są aplikacje w kontenerach. Każdy klaster musi posiadać przynajmniej jeden węzeł. aka: tags: - fundamental - operation --- -Zestaw maszyn, nazywanych węzłami, na których uruchamiane są aplikacje zarządzane przez Kubernetes. Klaster posiada przynajmniej jeden węzeł roboczy (*node*) i jeden węzeł typu master (*master node*). +Zestaw maszyn roboczych, nazywanych węzłami, na których uruchamiane są aplikacje w kontenerach. Każdy klaster musi posiadać przynajmniej jeden węzeł. -Na węźle (lub węzłach) roboczych rozmieszczane są pody, które są częściami składowymi aplikacji. Węzeł (lub węzły) typu master zarządzają węzłami roboczymi i podami należącymi do klastra. Zwielokrotnione węzły typu master zapewniają większą niezawodność i odporność klastra na awarie. +Na węźle (lub węzłach) roboczych rozmieszczane są pody, które są częściami składowymi aplikacji. Warstwa sterowania zarządza węzłami roboczymi i podami należącymi do klastra. W środowisku produkcyjnym warstwa sterowania rozłożona jest zazwyczaj na kilka maszyn, a klaster uruchomiony jest na wielu węzłach zapewniając większą niezawodność i odporność na awarie. diff --git a/content/pl/docs/reference/glossary/container-runtime.md b/content/pl/docs/reference/glossary/container-runtime.md index 804e367a70..170d500221 100644 --- a/content/pl/docs/reference/glossary/container-runtime.md +++ b/content/pl/docs/reference/glossary/container-runtime.md @@ -15,7 +15,7 @@ tags: -Kubernetes obsługuje różne *container runtimes*: [Docker](http://www.docker.com), -[containerd](https://containerd.io), [cri-o](https://cri-o.io/), -[rktlet](https://github.com/kubernetes-incubator/rktlet) oraz każdą implementację zgodną z -[Kubernetes CRI (Container Runtime Interface)](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-node/container-runtime-interface.md). +Kubernetes obsługuje różne *container runtimes*: {{< glossary_tooltip term_id="docker">}}, +{{< glossary_tooltip term_id="containerd" >}}, {{< glossary_tooltip term_id="cri-o" >}} +oraz każdą implementację zgodną z [Kubernetes CRI (Container Runtime +Interface)](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-node/container-runtime-interface.md). diff --git a/content/pl/docs/reference/glossary/kube-controller-manager.md b/content/pl/docs/reference/glossary/kube-controller-manager.md index 4a3a4e64b5..cd1ac2adfa 100755 --- a/content/pl/docs/reference/glossary/kube-controller-manager.md +++ b/content/pl/docs/reference/glossary/kube-controller-manager.md @@ -11,7 +11,7 @@ tags: - architecture - fundamental --- - Składnik *master* odpowiedzialny za uruchamianie {{< glossary_tooltip text="kontrolerów" term_id="controller" >}}. + Składnik warstwy sterowania odpowiedzialny za uruchamianie {{< glossary_tooltip text="kontrolerów" term_id="controller" >}}. diff --git a/content/pl/docs/reference/glossary/kube-scheduler.md b/content/pl/docs/reference/glossary/kube-scheduler.md index 074680ac02..6382cb0d6f 100755 --- a/content/pl/docs/reference/glossary/kube-scheduler.md +++ b/content/pl/docs/reference/glossary/kube-scheduler.md @@ -4,13 +4,13 @@ id: kube-scheduler date: 2018-04-12 full_link: /docs/reference/generated/kube-scheduler/ short_description: > - Składnik *master*, który monitoruje tworzenie nowych podów i przypisuje im węzły, na których powinny zostać uruchomione. + Składnik warstwy sterowania, który śledzi tworzenie nowych podów i przypisuje im węzły, na których powinny zostać uruchomione. aka: tags: - architecture --- -Składnik *master*, który monitoruje tworzenie nowych podów i przypisuje im węzły, na których powinny zostać uruchomione. +Składnik warstwy sterowania, który śledzi tworzenie nowych podów i przypisuje im węzły, na których powinny zostać uruchomione. diff --git a/content/pl/docs/setup/_index.md b/content/pl/docs/setup/_index.md index 0d0fde34b8..5c89d184d3 100644 --- a/content/pl/docs/setup/_index.md +++ b/content/pl/docs/setup/_index.md @@ -37,13 +37,12 @@ Aby uruchomić klaster Kubernetes do nauki na lokalnym komputerze, skorzystaj z |Społeczność |Ekosystem | | ------------ | -------- | | [Minikube](/docs/setup/learning-environment/minikube/) | [CDK on LXD](https://www.ubuntu.com/kubernetes/docs/install-local) | -| [kind (Kubernetes IN Docker)](https://github.com/kubernetes-sigs/kind) | [Docker Desktop](https://www.docker.com/products/docker-desktop)| +| [kind (Kubernetes IN Docker)](/docs/setup/learning-environment/kind/) | [Docker Desktop](https://www.docker.com/products/docker-desktop)| | | [Minishift](https://docs.okd.io/latest/minishift/)| | | [MicroK8s](https://microk8s.io/)| | | [IBM Cloud Private-CE (Community Edition)](https://github.com/IBM/deploy-ibm-cloud-private) | | | [IBM Cloud Private-CE (Community Edition) on Linux Containers](https://github.com/HSBawa/icp-ce-on-linux-containers)| | | [k3s](https://k3s.io)| -| | [Ubuntu on LXD](/docs/getting-started-guides/ubuntu/)| ## Środowisko produkcyjne {#srodowisko-produkcyjne} @@ -76,8 +75,6 @@ Poniższa tabela zawiera przegląd dostawców środowisk produkcyjnych i rozwią | [Digital Rebar](https://provision.readthedocs.io/en/tip/README.html) | | | | | | ✔ | [DigitalOcean](https://www.digitalocean.com/products/kubernetes/) | ✔ | | | | | | [Docker Enterprise](https://www.docker.com/products/docker-enterprise) | |✔ | ✔ | | | ✔ -| [Fedora (Multi Node)](https://kubernetes.io/docs/getting-started-guides/fedora/flannel_multi_node_cluster/)  | | | | | ✔ | ✔ -| [Fedora (Single Node)](https://kubernetes.io/docs/getting-started-guides/fedora/fedora_manual_config/)  | | | | | | ✔ | [Gardener](https://gardener.cloud/) | ✔ | ✔ | ✔ | ✔ | ✔ | [Custom Extensions](https://github.com/gardener/gardener/blob/master/docs/extensions/overview.md) | | [Giant Swarm](https://www.giantswarm.io/) | ✔ | ✔ | ✔ | | | [Google](https://cloud.google.com/) | [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine/) | [Google Compute Engine (GCE)](https://cloud.google.com/compute/)|[GKE On-Prem](https://cloud.google.com/gke-on-prem/) | | | | | | | | @@ -85,12 +82,13 @@ Poniższa tabela zawiera przegląd dostawców środowisk produkcyjnych i rozwią | [Ionos](https://www.ionos.com/enterprise-cloud) | [Ionos Managed Kubernetes](https://www.ionos.com/enterprise-cloud/managed-kubernetes) | [Ionos Enterprise Cloud](https://www.ionos.com/enterprise-cloud) | | | [Kontena Pharos](https://www.kontena.io/pharos/) | |✔| ✔ | | | | [KubeOne](https://kubeone.io/) | | ✔ | ✔ | ✔ | ✔ | ✔ | -| [Kubermatic](https://kubermatic.io/) | ✔ | ✔ | ✔ | ✔ | ✔ | | +| [Kubermatic](https://kubermatic.io/) | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | | [KubeSail](https://kubesail.com/) | ✔ | | | | | | [Kubespray](https://kubespray.io/#/) | | | |✔ | ✔ | ✔ | | [Kublr](https://kublr.com/) |✔ | ✔ |✔ |✔ |✔ |✔ | | [Microsoft Azure](https://azure.microsoft.com) | [Azure Kubernetes Service (AKS)](https://azure.microsoft.com/en-us/services/kubernetes-service/) | | | | | | [Mirantis Cloud Platform](https://www.mirantis.com/software/kubernetes/) | | | ✔ | | | +| [NetApp Kubernetes Service (NKS)](https://cloud.netapp.com/kubernetes-service) | ✔ | ✔ | ✔ | | | | [Nirmata](https://www.nirmata.com/) | | ✔ | ✔ | | | | [Nutanix](https://www.nutanix.com/en) | [Nutanix Karbon](https://www.nutanix.com/products/karbon) | [Nutanix Karbon](https://www.nutanix.com/products/karbon) | | | [Nutanix AHV](https://www.nutanix.com/products/acropolis/virtualization) | | [OpenNebula](https://www.opennebula.org) |[OpenNebula Kubernetes](https://marketplace.opennebula.systems/docs/service/kubernetes.html) | | | | | @@ -100,7 +98,6 @@ Poniższa tabela zawiera przegląd dostawców środowisk produkcyjnych i rozwią | [Pivotal](https://pivotal.io/) | | [Enterprise Pivotal Container Service (PKS)](https://pivotal.io/platform/pivotal-container-service) | [Enterprise Pivotal Container Service (PKS)](https://pivotal.io/platform/pivotal-container-service) | | | | [Platform9](https://platform9.com/) | [Platform9 Managed Kubernetes](https://platform9.com/managed-kubernetes/) | | [Platform9 Managed Kubernetes](https://platform9.com/managed-kubernetes/) | ✔ | ✔ | ✔ | [Rancher](https://rancher.com/) | | [Rancher 2.x](https://rancher.com/docs/rancher/v2.x/en/) | | [Rancher Kubernetes Engine (RKE)](https://rancher.com/docs/rke/latest/en/) | | [k3s](https://k3s.io/) -| [StackPoint](https://stackpoint.io/)  | ✔ | ✔ | | | | | [Supergiant](https://supergiant.io/) | |✔ | | | | | [SUSE](https://www.suse.com/) | | ✔ | | | | | [SysEleven](https://www.syseleven.io/) | ✔ | | | | | diff --git a/content/pl/docs/tasks/_index.md b/content/pl/docs/tasks/_index.md index 1d96d99a0b..253cd26cd2 100644 --- a/content/pl/docs/tasks/_index.md +++ b/content/pl/docs/tasks/_index.md @@ -57,10 +57,6 @@ Konfigurowanie aplikacji w taki sposób, aby korzystała i ufała łańcuchowi c Standardowe metody zarządzania klasterem. -## Administracja federacją - -Konfigurowanie federacji klastrów. - ## Zarządzanie aplikacjami ze stanem (_Stateful_) Popularne zadania związane z zarządzaniem aplikacjami stanowymi _(Stateful)_, w tym: skalowanie, usuwanie i rozwiązywanie problemów dotyczących _StatefulSets_. diff --git a/content/pl/docs/tutorials/_index.md b/content/pl/docs/tutorials/_index.md index 6005968936..0723c6f4a4 100644 --- a/content/pl/docs/tutorials/_index.md +++ b/content/pl/docs/tutorials/_index.md @@ -22,8 +22,6 @@ Przed zapoznaniem się z samouczkami warto stworzyć zakładkę do * [Podstawy Kubernetes](/docs/tutorials/kubernetes-basics/) to interaktywny samouczek, który pomoże zrozumieć system Kubernetes i wypróbować jego podstawowe możliwości. -* [Scalable Microservices with Kubernetes (Udacity)](https://www.udacity.com/course/scalable-microservices-with-kubernetes--ud615) - * [Introduction to Kubernetes (edX)](https://www.edx.org/course/introduction-kubernetes-linuxfoundationx-lfs158x#) * [Hello Minikube](/docs/tutorials/hello-minikube/) diff --git a/content/pl/docs/tutorials/hello-minikube.md b/content/pl/docs/tutorials/hello-minikube.md index f962199317..1a7da3cd3c 100644 --- a/content/pl/docs/tutorials/hello-minikube.md +++ b/content/pl/docs/tutorials/hello-minikube.md @@ -8,7 +8,7 @@ menu: weight: 10 post: >

    Jesteś gotowy ubrudzić ręce? Zbuduj własny klaster kubernetes z działającą na nim aplikacją "Hello World" w Node.js.

    -card: +card: name: tutorials weight: 10 --- @@ -49,7 +49,7 @@ Więcej informacji na temat polecenia `docker build` znajdziesz w [dokumentacji ## Stwórz klaster Minikube -1. Kliknij w **Launch Terminal** +1. Kliknij w **Launch Terminal** {{< kat-button >}} @@ -117,7 +117,7 @@ wykorzystując podany obraz Dockera. ```shell kubectl config view ``` - + {{< note >}}Więcej informacji na temat polecenia `kubectl` znajdziesz w [przeglądzie kubectl](/docs/user-guide/kubectl-overview/).{{< /note >}} ## Stwórz Serwis @@ -167,7 +167,7 @@ musisz najpierw wystawić Pod jako [*Serwis*](/docs/concepts/services-networking ## Włącz dodatki -Minikube ma zestaw wbudowanych dodatków, które mogą być włączane, wyłączane i otwierane w lokalnym środowisku Kubernetes. +Minikube ma zestaw wbudowanych {{< glossary_tooltip text="dodatków" term_id="addons" >}}, które mogą być włączane, wyłączane i otwierane w lokalnym środowisku Kubernetes. 1. Lista aktualnie obsługiwanych dodatków: @@ -184,7 +184,6 @@ Minikube ma zestaw wbudowanych dodatków, które mogą być włączane, wyłącz efk: disabled freshpod: disabled gvisor: disabled - heapster: disabled helm-tiller: disabled ingress: disabled ingress-dns: disabled @@ -198,16 +197,16 @@ Minikube ma zestaw wbudowanych dodatków, które mogą być włączane, wyłącz storage-provisioner-gluster: disabled ``` -2. Włącz dodatek, na przykład `heapster`: +2. Włącz dodatek, na przykład `metrics-server`: ```shell - minikube addons enable heapster + minikube addons enable metrics-server ``` - + Wynik powinien wyglądać podobnie do: ``` - heapster was successfully enabled + metrics-server was successfully enabled ``` 3. Sprawdź Pod i Serwis, który właśnie stworzyłeś: @@ -222,7 +221,7 @@ Minikube ma zestaw wbudowanych dodatków, które mogą być włączane, wyłącz NAME READY STATUS RESTARTS AGE pod/coredns-5644d7b6d9-mh9ll 1/1 Running 0 34m pod/coredns-5644d7b6d9-pqd2t 1/1 Running 0 34m - pod/heapster-9jttx 1/1 Running 0 26s + pod/metrics-server-67fb648c5 1/1 Running 0 26s pod/etcd-minikube 1/1 Running 0 34m pod/influxdb-grafana-b29w8 2/2 Running 0 26s pod/kube-addon-manager-minikube 1/1 Running 0 34m @@ -233,23 +232,23 @@ Minikube ma zestaw wbudowanych dodatków, które mogą być włączane, wyłącz pod/storage-provisioner 1/1 Running 0 34m NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE - service/heapster ClusterIP 10.96.241.45 80/TCP 26s + service/metrics-server ClusterIP 10.96.241.45 80/TCP 26s service/kube-dns ClusterIP 10.96.0.10 53/UDP,53/TCP 34m service/monitoring-grafana NodePort 10.99.24.54 80:30002/TCP 26s service/monitoring-influxdb ClusterIP 10.111.169.94 8083/TCP,8086/TCP 26s ``` -4. Wyłącz dodatek `heapster`: +4. Wyłącz dodatek `metrics-server`: ```shell - minikube addons disable heapster + minikube addons disable metrics-server ``` - + Wynik powinien wyglądać podobnie do: ``` - heapster was successfully disabled + heapster was successfully metrics-server ``` ## Porządkujemy po sobie @@ -278,7 +277,7 @@ minikube delete {{% capture whatsnext %}} * Dowiedz się więcej o [obiektach typu Deployment](/docs/concepts/workloads/controllers/deployment/). -* Dowiedz się więcej o [instalowaniu aplikacji](/docs/user-guide/deploying-applications/). +* Dowiedz się więcej o [instalowaniu aplikacji](/docs/tasks/run-application/run-stateless-application-deployment/). * Dowiedz się więcej o [obiektach typu Serwis](/docs/concepts/services-networking/service/). {{% /capture %}} diff --git a/content/pl/docs/tutorials/kubernetes-basics/create-cluster/cluster-intro.html b/content/pl/docs/tutorials/kubernetes-basics/create-cluster/cluster-intro.html index 64b9336dec..df3b93c61b 100644 --- a/content/pl/docs/tutorials/kubernetes-basics/create-cluster/cluster-intro.html +++ b/content/pl/docs/tutorials/kubernetes-basics/create-cluster/cluster-intro.html @@ -84,7 +84,7 @@ weight: 10
    -

    Kiedy instalujesz aplikację na Kubernetes, polecasz masterowi uruchomienie kontenera z aplikacją. Master zleca uruchomienie kontenera na węzłach klastra. Węzły komunikują się z masterem przy użyciu Kubernetes API, wystawianego przez mastera. Użytkownicy końcowi mogą korzystać bezpośrednio z Kubernetes API do komunikacji z klastrem.

    +

    Kiedy instalujesz aplikację na Kubernetes, polecasz masterowi uruchomienie kontenera z aplikacją. Master zleca uruchomienie kontenera na węzłach klastra. Węzły komunikują się z masterem przy użyciu Kubernetes API, wystawianego przez mastera. Użytkownicy końcowi mogą korzystać bezpośrednio z Kubernetes API do komunikacji z klastrem.

    Klaster Kubernetes może być zainstalowany zarówno na fizycznych, jak i na maszynach wirtualnych. Aby wypróbować Kubernetes, można też wykorzystać Minikube. Minikube to "lekka" implementacja Kubernetes, która tworzy VM na maszynie lokalnej i instaluje prosty klaster składający się tylko z jednego węzła. Minikube jest dostępne na systemy Linux, macOS i Windows. Narzędzie linii poleceń Minikube obsługuje podstawowe operacje na klastrze, takie jak start, stop, informacje o stanie i usunięcie klastra. Na potrzeby tego samouczka wykorzystamy jednak terminal online z zainstalowanym już wcześniej Minikube.

    diff --git a/content/pl/docs/tutorials/kubernetes-basics/deploy-app/deploy-interactive.html b/content/pl/docs/tutorials/kubernetes-basics/deploy-app/deploy-interactive.html index be179683c1..ea083b33e7 100644 --- a/content/pl/docs/tutorials/kubernetes-basics/deploy-app/deploy-interactive.html +++ b/content/pl/docs/tutorials/kubernetes-basics/deploy-app/deploy-interactive.html @@ -17,7 +17,16 @@ weight: 20
    +
    +
    +

    + Pod to podstawowy element odpowiedzialny za uruchomienie aplikacji na Kubernetesie. Każdy pod to część składowa całościowego obciążenia Twojego klastra. Dowiedz się więcej na temat Podów. +

    +
    +
    +
    +
    Do pracy z terminalem użyj wersji na desktop/tablet diff --git a/content/pl/docs/tutorials/kubernetes-basics/expose/expose-intro.html b/content/pl/docs/tutorials/kubernetes-basics/expose/expose-intro.html index 2dd9a35e99..d3fa9a8343 100644 --- a/content/pl/docs/tutorials/kubernetes-basics/expose/expose-intro.html +++ b/content/pl/docs/tutorials/kubernetes-basics/expose/expose-intro.html @@ -81,11 +81,6 @@ weight: 10
    -
    -
    -

    Możemy połączyć tworzenie Deploymentu i Serwisu stosując opcję
    --expose w kubectl.

    -
    -

    diff --git a/content/pl/examples/minikube/Dockerfile b/content/pl/examples/minikube/Dockerfile index 1fe745295a..dd58cb7e75 100644 --- a/content/pl/examples/minikube/Dockerfile +++ b/content/pl/examples/minikube/Dockerfile @@ -1,4 +1,4 @@ FROM node:6.14.2 EXPOSE 8080 COPY server.js . -CMD node server.js +CMD [ "node", "server.js" ] From 9df5b4ec4cf42853e94e5fc03d9b6b6fc1fc07d8 Mon Sep 17 00:00:00 2001 From: Danni Setiawan Date: Fri, 20 Mar 2020 18:50:36 +0700 Subject: [PATCH 130/140] Add ID translation for Labels (#19072) --- .../overview/working-with-objects/labels.md | 225 ++++++++++++++++++ 1 file changed, 225 insertions(+) create mode 100644 content/id/docs/concepts/overview/working-with-objects/labels.md diff --git a/content/id/docs/concepts/overview/working-with-objects/labels.md b/content/id/docs/concepts/overview/working-with-objects/labels.md new file mode 100644 index 0000000000..0b6060e2fd --- /dev/null +++ b/content/id/docs/concepts/overview/working-with-objects/labels.md @@ -0,0 +1,225 @@ +--- +title: Label dan Selektor +content_template: templates/concept +weight: 40 +--- + +{{% capture overview %}} + +_Label_ merupakan pasangan _key/value_ yang melekat pada objek-objek, misalnya pada Pod. +Label digunakan untuk menentukan atribut identitas dari objek agar memiliki arti dan relevan bagi para pengguna, namun tidak secara langsung memiliki makna terhadap sistem inti. +Label dapat digunakan untuk mengatur dan memilih sebagian dari banyak objek. Label-label dapat ditempelkan ke objek-objek pada saat dibuatnya objek-objek tersebut dan kemudian ditambahkan atau diubah kapan saja setelahnya. +Setiap objek dapat memiliki satu set label _key/value_. Setiap _Key_ harus unik untuk objek tersebut. + +```json +"metadata": { + "labels": { + "key1" : "value1", + "key2" : "value2" + } +} +``` + +Label memungkinkan untuk menjalankan kueri dan pengamatan dengan efisien, serta ideal untuk digunakan pada UI dan CLI. Informasi yang tidak digunakan untuk identifikasi sebaiknya menggunakan [anotasi](/id/docs/concepts/overview/working-with-objects/annotations/). + +{{% /capture %}} + + +{{% capture body %}} + +## Motivasi + +Label memungkinkan pengguna untuk memetakan struktur organisasi mereka ke dalam objek-objek sistem yang tidak terikat secara erat, tanpa harus mewajibkan klien untuk menyimpan pemetaan tersebut. + +_Service deployments_ dan _batch processing pipelines_ sering menjadi entitas yang berdimensi ganda (contohnya partisi berganda atau _deployment_, jalur rilis berganda, tingkatan berganda, _micro-services_ berganda per tingkatan). Manajemen seringkali membutuhkan operasi lintas tim, yang menyebabkan putusnya enkapsulasi dari representasi hierarki yang ketat, khususnya pada hierarki-hierarki kaku yang justru ditentukan oleh infrastruktur, bukan oleh pengguna. + +Contoh label: + + * `"release" : "stable"`, `"release" : "canary"` + * `"environment" : "dev"`, `"environment" : "qa"`, `"environment" : "production"` + * `"tier" : "frontend"`, `"tier" : "backend"`, `"tier" : "cache"` + * `"partition" : "customerA"`, `"partition" : "customerB"` + * `"track" : "daily"`, `"track" : "weekly"` + +Ini hanya contoh label yang biasa digunakan; kamu bebas mengembangkan caramu sendiri. Perlu diingat bahwa _Key_ dari label harus unik untuk objek tersebut. + +## Sintaksis dan set karakter + +_Label_ merupakan pasangan _key/value_. _Key-key_ dari Label yang valid memiliki dua segmen: sebuah prefiks dan nama yang opsional, yang dipisahkan oleh garis miring (`/`). Segmen nama wajib diisi dan tidak boleh lebih dari 63, dimulai dan diakhiri dengan karakter alfanumerik (`[a-z0-9A-Z]`) dengan tanda pisah (`-`), garis bawah (`_`), titik (`.`), dan alfanumerik di antaranya. Sedangkan prefiks bersifat opsional. Jika ditentukan, prefiks harus berupa subdomain DNS: rangkaian label DNS yang dipisahkan oleh titik (`.`), dengan total tidak lebih dari 253 karakter, yang diikuti oleh garis miring (`/`). + +Jika prefiks dihilangkan, _Key_ dari label diasumsikan privat bagi pengguna. Komponen sistem otomatis (contoh `kube-scheduler`, `kube-controller-manager`, `kube-apiserver`, `kubectl`, atau otomasi pihak ketiga lainnya) yang akan menambah label ke objek-objek milik pengguna akhir harus menentukan prefiks. + +Prefiks `kubernetes.io/` dan `k8s.io/` dikhususkan untuk komponen inti Kubernetes. + +Nilai label yang valid tidak boleh lebih dari 63 karakter dan harus kosong atau diawali dan diakhiri dengan karakter alfanumerik (`[a-z0-9A-Z]`) dengan tanda pisah (`-`), garis bawah (`_`), titik (`.`), dan alfanumerik di antaranya. + +Contoh di bawah ini merupakan berkas konfigurasi untuk Pod yang memiliki dua label `environment: production` dan `app: nginx` : + +```yaml + +apiVersion: v1 +kind: Pod +metadata: + name: label-demo + labels: + environment: production + app: nginx +spec: + containers: + - name: nginx + image: nginx:1.7.9 + ports: + - containerPort: 80 + +``` + +## Selektor label + +Tidak seperti [nama dan UID](/id/docs/concepts/overview/working-with-objects/names/), label tidak memberikan keunikan. Secara umum, kami memperkirakan bahwa banyak objek yang akan memiliki label yang sama. + +Menggunakan sebuah _label selector_, klien/pengguna dapat mengidentifikasi suatu kumpulan objek. Selektor label merupakan alat/cara pengelompokan utama pada Kubernetes. + +Saat ini API mendukung dua jenis selektor: _equality-based_ dan _set-based_. +Sebuah selektor label dapat dibuat dari kondisi berganda yang dipisahkan oleh koma. Pada kasus kondisi berganda, semua kondisi harus dipenuhi sehingga separator koma dapat bertindak sebagai operator logika _AND_ (`&&`). + +Makna dari selektor yang kosong atau tidak diisi tergantung dari konteks, dan tipe API yang menggunakan selektor harus mendokumentasikan keabsahan dan arti dari selektor yang kosong tersebut. + +{{< note >}} +Untuk beberapa tipe API, seperti ReplicaSet, selektor label untuk dua objek tidak boleh tumpang tindih dengan Namespace, jika tidak maka _controller_ akan melihatnya sebagai instruksi yang menyebabkan konflik dan akan gagal menentukan berapa banyak replika yang seharusnya tersedia. +{{< /note >}} + +{{< caution >}} +Untuk kedua kondisi _equality-based_ dan _set-based_ tidak ada logika operator _OR_ (`||`). Pastikan struktur pernyataan filter kamu ikut disesuaikan. +{{< /caution >}} + +### Kondisi _Equality-based_ + +Kondisi _Equality-based_ atau _inequality-based_ memungkinkan untuk melakukan filter dengan menggunakan _key_ dan _value_ dari label. Objek yang cocok harus memenuhi semua batasan label yang telah ditentukan, meskipun mereka dapat memiliki label tambahan lainnya. +Terdapat tiga jenis operator yang didukung yaitu `=`,`==`,`!=`. Dua operator pertama menyatakan kesamaan (keduanya hanyalah sinonim), sementara operator terakhir menyatakan ketidaksamaan. Contoh: + +``` +environment = production +tier != frontend +``` + +Kondisi pertama akan memilih semua sumber daya dengan _key_ `environment` dan nilai _key_ `production`. +Kondisi berikutnya akan memilih semua sumber daya dengan _key_ `tier` dan nilai _key_ selain `frontend`, dan semua sumber daya yang tidak memiliki label dengan _key_ `tier`. +Kamu juga dapat memfilter sumber daya dalam `production` selain `frontend` dengan menggunakan operator koma: `environment=production,tier!=frontend` + +Salah satu skenario penggunaan label dengan kondisi _equality-based_ yaitu untuk kriteria pemilihan Node untuk Pod-Pod. Sebagai contoh, Pod percontohan di bawah ini akan memilih Node dengan label "`accelerator=nvidia-tesla-p100`". + +```yaml +apiVersion: v1 +kind: Pod +metadata: + name: cuda-test +spec: + containers: + - name: cuda-test + image: "k8s.gcr.io/cuda-vector-add:v0.1" + resources: + limits: + nvidia.com/gpu: 1 + nodeSelector: + accelerator: nvidia-tesla-p100 +``` + +### Kondisi _Set-based_ + +Kondisi label _Set-based_ memungkinkan memfilter _key_ terhadap suatu kumpulan nilai. Terdapat tiga jenis operator yang didukung, yaitu: `in`,`notin`, dan `exists` (hanya _key_-nya saja). Contoh: + +``` +environment in (production, qa) +tier notin (frontend, backend) +partition +!partition +``` + +Contoh pertama akan memilih semua sumber daya dengan _key_ `environment` dan nilai `production` atau `qa`. +Contoh kedua akan memilih semua sumber daya dengan _key_ `tier` dan nilai selain `frontend` dan `backend`, serta semua sumber daya yang tidak memiliki label dengan _key_ `tier`. +Contoh ketiga akan memilih semua sumber daya yang memiliki _key_ dari label`partition`; nilainya tidak diperiksa. +Sedangkan contoh keempat akan memilih semua sumber daya yang tidak memiliki label dengan _key_ `partition`; nilainya tidak diperiksa. +Secara serupa, operator koma bertindak sebagai operator _AND_. Sehingga penyaringan sumber daya dengan _key_ `partition` (tidak peduli nilai dari _key_) dan `environment` yang tidak sama dengan `qa` dapat dicapai dengan `partition,environment notin (qa)`. +Selektor label _set-based_ merupakan bentuk umum persamaan karena `environment=production` sama dengan `environment in (production)`; demikian pula `!=` dan `notin`. + +Kondisi _Set-based_ dapat digabungkan dengan kondisi _equality-based_. Contoh: `partition in (customerA, customerB),environment!=qa`. + + +## API + +### Penyaringan LIST dan WATCH + +Operasi LIST dan WATCH dapat menentukan selektor label untuk memfilter suatu kumpulan objek yang didapat dengan menggunakan parameter kueri. Kedua jenis kondisi diperbolehkan (ditampilkan sebagai berikut, sama seperti saat tampil pada string kueri di URL): + + * Kondisi _equality-based_: `?labelSelector=environment%3Dproduction,tier%3Dfrontend` + * Kondisi _set-based_: `?labelSelector=environment+in+%28production%2Cqa%29%2Ctier+in+%28frontend%29` + +Kedua jenis selektor label dapat digunakan untuk menampilkan (_list_) dan mengamati (_watch_) sumber daya melalui klien REST. Contohnya, menargetkan `apiserver` dengan `kubectl` dan menggunakan _equality-based_ kamu dapat menuliskan: + +```shell +kubectl get pods -l environment=production,tier=frontend +``` + +atau menggunakan kondisi _set-based_: + +```shell +kubectl get pods -l 'environment in (production),tier in (frontend)' +``` + +Seperti yang telah disebutkan sebelumnya, kondisi _set-based_ lebih ekspresif. Sebagai contoh, mereka dapat digunakan untuk mengimplementasi operator _OR_ pada nilai: + +```shell +kubectl get pods -l 'environment in (production, qa)' +``` + +atau membatasi pencocokan negatif dengan operator _exists_: + +```shell +kubectl get pods -l 'environment,environment notin (frontend)' +``` + +### Mengatur referensi pada objek API + +Pada beberapa objek Kubernetes, seperti [`Service`](/docs/user-guide/services) dan [`ReplicationController`](/id/docs/concepts/workloads/controllers/replicationcontroller/), juga menggunakan selektor label untuk menentukan kumpulan dari sumber daya lain, seperti [Pod](/id/docs/concepts/workloads/pods/pod). + +#### Service dan ReplicationController + +Kumpulan Pod yang ditargetkan oleh sebuah `service` ditentukan dengan selektor label. Demikian pula kumpulan Pod yang harus ditangani oleh `replicationcontroller` juga ditentukan dengan selektor label. + +Selektor label untuk kedua objek tersebut ditentukan dalam berkas `json` atau `yaml` menggunakan _maps_, dan hanya mendukung kondisi _equality-based_: + +```json +"selector": { + "component" : "redis", +} +``` +atau + +```yaml +selector: + component: redis +``` + +selektor ini (baik dalam bentuk `json` atau `yaml`) sama dengan `component=redis` atau `component in (redis)`. + +#### Sumber daya yang mendukung kondisi set-based + +Sumber daya yang lebih baru, seperti [`Job`](/id/docs/concepts/workloads/controllers/jobs-run-to-completion/), [`Deployment`](/id/docs/concepts/workloads/controllers/deployment/), [`ReplicaSet`](/id/docs/concepts/workloads/controllers/replicaset/), dan [`DaemonSet`](/id/docs/concepts/workloads/controllers/daemonset/), juga mendukung kondisi _set-based_. + +```yaml +selector: + matchLabels: + component: redis + matchExpressions: + - {key: tier, operator: In, values: [cache]} + - {key: environment, operator: NotIn, values: [dev]} +``` + +`matchLabels` merupakan pemetaan dari pasangan `{key,value}`. Sebuah `{key,value}` pada pemetaan `matchLabels` adalah sama dengan elemen dari `matchExpressions`, yang nilai `key` nya adalah "key", dengan `operator` "In", dan _array_ `values` hanya berisi "value". `matchExpressions` merupakan daftar kondisi untuk selektor Pod. Operator yang valid termasuk In, NotIn, Exists, dan DoesNotExist. Kumpulan nilai ini tidak boleh kosong pada kasus In dan NotIn. Semua kondisi, baik dari `matchLabels` dan `matchExpressions` di-AND secara sekaligus -- mereka harus memenuhi semua kondisi agar cocok. + +#### Memilih kumpulan Node + +Salah satu contoh penggunaan pemilihan dengan menggunakan label yaitu untuk membatasi suatu kumpulan Node tertentu yang dapat digunakan oleh Pod. +Lihat dokumentasi pada [pemilihan Node](/id/docs/concepts/configuration/assign-pod-node/) untuk informasi lebih lanjut. + +{{% /capture %}} From a6d1aa81109a9c94bad7e798e3fc0222d0126481 Mon Sep 17 00:00:00 2001 From: chentanjun <2799194073@qq.com> Date: Sat, 21 Mar 2020 10:08:44 +0800 Subject: [PATCH 131/140] update zh-trans content/zh/docs/tasks/job/parallel-processing-expansion.md (#19636) --- .../job/parallel-processing-expansion.md | 643 ++++++++++-------- 1 file changed, 341 insertions(+), 302 deletions(-) diff --git a/content/zh/docs/tasks/job/parallel-processing-expansion.md b/content/zh/docs/tasks/job/parallel-processing-expansion.md index 5cbdc3e8da..e113804e73 100644 --- a/content/zh/docs/tasks/job/parallel-processing-expansion.md +++ b/content/zh/docs/tasks/job/parallel-processing-expansion.md @@ -1,302 +1,341 @@ ---- -title: 使用扩展进行并行处理 -content_template: templates/concept -weight: 20 ---- - - - -{{% capture overview %}} - - -在这个示例中，我们将运行从一个公共模板创建的多个 Kubernetes 作业。您可能希望熟悉 -[Jobs](/docs/concepts/workloads/controllers/jobs-run-to-completion/) 的基本、非并行使用。 - -{{% /capture %}} - - -{{% capture body %}} - - - -## 基本模板扩展 - - -首先，将以下作业模板下载到名为 `job-tmpl.yaml` 的文件中 - -{{< codenew file="application/job/job-tmpl.yaml" >}} - - -与 *pod 模板*不同，我们的 *job 模板*不是 Kubernetes API 类型。它只是作业对象的 yaml 表示， -YAML 文件有一些占位符，在使用它之前需要填充这些占位符。`$ITEM` 语法对 Kubernetes 没有意义。 - - -在这个例子中，容器所做的唯一处理是 `echo` 一个字符串并休眠一段时间。 -在真实的用例中，处理将是一些重要的计算，例如呈现电影的帧，或者处理数据库中的一系列行。例如，`$ITEM` 参数将指定帧号或行范围。 - - -这个作业及其 Pod 模板有一个标签: `jobgroup=jobexample`。这个标签在系统中没有什么特别之处。 -这个标签使得我们可以方便地同时操作组中的所有作业。 -我们还将相同的标签放在 pod 模板上，这样我们就可以用一个命令检查这些作业的所有 pod。 -创建作业之后，系统将添加更多的标签来区分一个作业的 pod 和另一个作业的 pod。 -注意，标签键 `jobgroup` 对 Kubernetes 并无特殊含义。您可以选择自己的标签方案。 - - -下一步，将模板展开到多个文件中，每个文件对应要处理的项。 - -```shell -# Expand files into a temporary directory -$ mkdir ./jobs -$ for i in apple banana cherry -do - cat job-tmpl.yaml | sed "s/\$ITEM/$i/" > ./jobs/job-$i.yaml -done -``` - - -检查是否工作正常： - -```shell -$ ls jobs/ -job-apple.yaml -job-banana.yaml -job-cherry.yaml -``` - - -在这里，我们使用 `sed` 将字符串 `$ITEM` 替换为循环变量。 -您可以使用任何类型的模板语言(jinja2, erb) 或编写程序来生成作业对象。 - - -接下来，使用 kubectl 命令创建所有作业： - -```shell -$ kubectl create -f ./jobs -job "process-item-apple" created -job "process-item-banana" created -job "process-item-cherry" created -``` - - -现在，检查这些作业： - -```shell -$ kubectl get jobs -l jobgroup=jobexample -NAME DESIRED SUCCESSFUL AGE -process-item-apple 1 1 31s -process-item-banana 1 1 31s -process-item-cherry 1 1 31s -``` - - -在这里，我们使用 `-l` 选项选择属于这组作业的所有作业。(系统中可能还有其他不相关的工作，我们不想看到。) - - -我们可以检查 pod 以及使用同样地标签选择器： - -```shell -$ kubectl get pods -l jobgroup=jobexample -NAME READY STATUS RESTARTS AGE -process-item-apple-kixwv 0/1 Completed 0 4m -process-item-banana-wrsf7 0/1 Completed 0 4m -process-item-cherry-dnfu9 0/1 Completed 0 4m -``` - - -没有一个命令可以一次检查所有作业的输出，但是循环遍历所有 pod 非常简单： - -```shell -$ for p in $(kubectl get pods -l jobgroup=jobexample -o name) -do - kubectl logs $p -done -Processing item apple -Processing item banana -Processing item cherry -``` - - - -## 多个模板参数 - - -在第一个示例中，模板的每个实例都有一个参数，该参数也用作标签。 -但是标签的键名在[可包含的字符](/docs/concepts/overview/working-with-objects/labels/#syntax-and-character-set)方面有一定的约束。 - - -这个稍微复杂一点的示例使用 jinja2 板语言来生成我们的对象。 -我们将使用一行 python 脚本将模板转换为文件。 - - -首先，粘贴作业对象的以下模板到一个名为 `job.yaml.jinja2` 的文件中： - -```liquid -{%- set params = [{ "name": "apple", "url": "http://www.orangepippin.com/apples", }, - { "name": "banana", "url": "https://en.wikipedia.org/wiki/Banana", }, - { "name": "raspberry", "url": "https://www.raspberrypi.org/" }] -%} -{%- for p in params %} -{%- set name = p["name"] %} -{%- set url = p["url"] %} -apiVersion: batch/v1 -kind: Job -metadata: - name: jobexample-{{ name }} - labels: - jobgroup: jobexample -spec: - template: - metadata: - name: jobexample - labels: - jobgroup: jobexample - spec: - containers: - - name: c - image: busybox - command: ["sh", "-c", "echo Processing URL {{ url }} && sleep 5"] - restartPolicy: Never ---- -{%- endfor %} - -``` - - -上面的模板使用 python dicts 列表（第1-4行）定义每个作业对象的参数。 -然后 for 循环为每组参数（剩余行）生成一个作业 yaml 对象。 -我们利用了多个 yaml 文档可以与 `---` 分隔符连接的事实（倒数第二行）。 -我们可以将输出直接传递给 kubectl 来创建对象。 - - -如果您还没有 jinja2 包则需要安装它: `pip install --user jinja2`。 -现在，使用这个一行 python 程序来展开模板: - -```shell -alias render_template='python -c "from jinja2 import Template; import sys; print(Template(sys.stdin.read()).render());"' -``` - - - -输出可以保存到一个文件，像这样： - -```shell -cat job.yaml.jinja2 | render_template > jobs.yaml -``` - - -或直接发送到 kubectl，如下所示： - -```shell -cat job.yaml.jinja2 | render_template | kubectl create -f - -``` - - -## 替代方案 - - -如果您有大量作业对象，您可能会发现： - - - -- 即使使用标签，管理这么多作业对象也很麻烦。 -- 在一次创建所有作业时，您超过了资源配额，可是您也不希望以递增方式创建作业并等待其完成。 -- 同时创建的大量作业会使 Kubernetes apiserver、控制器或调度程序过载。 - - - -在这种情况下，您可以考虑 -其他[工作模式](/docs/concepts/jobs/run-to-completion-finite-workloads/#job-patterns)。 - -{{% /capture %}} +--- +title: 使用扩展进行并行处理 +content_template: templates/concept +min-kubernetes-server-version: v1.8 +weight: 20 +--- + + + +{{% capture overview %}} + + +在这个示例中，我们将运行从一个公共模板创建的多个 Kubernetes Job。您可能需要先熟悉 [Jobs](/docs/concepts/workloads/controllers/jobs-run-to-completion/) 的基本概念、非并行以及如何使用它。 + +{{% /capture %}} + + +{{% capture body %}} + + + +## 基本模板扩展 + + +首先，将以下作业模板下载到名为 `job-tmpl.yaml` 的文件中。 + +{{< codenew file="application/job/job-tmpl.yaml" >}} + + +与 *pod 模板*不同，我们的 *job 模板*不是 Kubernetes API 类型。它只是 Job 对象的 yaml 表示， +YAML 文件有一些占位符，在使用它之前需要填充这些占位符。`$ITEM` 语法对 Kubernetes 没有意义。 + + +在这个例子中，容器所做的唯一处理是 `echo` 一个字符串并睡眠一段时间。 +在真实的用例中，处理将是一些重要的计算，例如渲染电影的一帧，或者处理数据库中的若干行。这时，`$ITEM` 参数将指定帧号或行范围。 + + +这个 Job 及其 Pod 模板有一个标签: `jobgroup=jobexample`。这个标签在系统中没有什么特别之处。 +这个标签使得我们可以方便地同时操作组中的所有作业。 +我们还将相同的标签放在 pod 模板上，这样我们就可以用一个命令检查这些 Job 的所有 pod。 +创建作业之后，系统将添加更多的标签来区分一个 Job 的 pod 和另一个 Job 的 pod。 +注意，标签键 `jobgroup` 对 Kubernetes 并无特殊含义。您可以选择自己的标签方案。 + + +下一步，将模板展开到多个文件中，每个文件对应要处理的项。 + +```shell +# 下载 job-templ.yaml +curl -L -s -O https://k8s.io/examples/application/job/job-tmpl.yaml + +# 创建临时目录，并且在目录中创建 job yaml 文件 +mkdir ./jobs +for i in apple banana cherry +do + cat job-tmpl.yaml | sed "s/\$ITEM/$i/" > ./jobs/job-$i.yaml +done +``` + + +检查是否工作正常： + +```shell +ls jobs/ +``` + + +输出类似以下内容： + +``` +job-apple.yaml +job-banana.yaml +job-cherry.yaml +``` + + +在这里，我们使用 `sed` 将字符串 `$ITEM` 替换为循环变量。 +您可以使用任何类型的模板语言(jinja2, erb) 或编写程序来生成 Job 对象。 + + +接下来，使用 kubectl 命令创建所有作业： + +```shell +kubectl create -f ./jobs +``` + + +输出类似以下内容： + +``` +job.batch/process-item-apple created +job.batch/process-item-banana created +job.batch/process-item-cherry created +``` + + +现在，检查这些作业： + +```shell +kubectl get jobs -l jobgroup=jobexample +``` + + +输出类似以下内容： + +``` +NAME COMPLETIONS DURATION AGE +process-item-apple 1/1 14s 20s +process-item-banana 1/1 12s 20s +process-item-cherry 1/1 12s 20s +``` + + +在这里，我们使用 `-l` 选项选择属于这组作业的所有作业。(系统中可能还有其他不相关的工作，我们不想看到。) + + +使用同样的标签选择器，我们还可以检查 pods： + +```shell +kubectl get pods -l jobgroup=jobexample +``` + + +输出类似以下内容： + +``` +NAME READY STATUS RESTARTS AGE +process-item-apple-kixwv 0/1 Completed 0 4m +process-item-banana-wrsf7 0/1 Completed 0 4m +process-item-cherry-dnfu9 0/1 Completed 0 4m +``` + + +我们可以使用以下操作命令一次性地检查所有作业的输出： + +```shell +kubectl logs -f -l jobgroup=jobexample +``` + + +输出内容为： + +``` +Processing item apple +Processing item banana +Processing item cherry +``` + + + +## 多个模板参数 + + +在第一个示例中，模板的每个实例都有一个参数，该参数也用作标签。 +但是标签的键名在[可包含的字符](/docs/concepts/overview/working-with-objects/labels/#syntax-and-character-set)方面有一定的约束。 + + +这个稍微复杂一点的示例使用 jinja2 模板语言来生成我们的对象。 +我们将使用一行 python 脚本将模板转换为文件。 + + +首先，粘贴 Job 对象的以下模板到一个名为 `job.yaml.jinja2` 的文件中： + +```liquid +{%- set params = [{ "name": "apple", "url": "https://www.orangepippin.com/varieties/apples", }, + { "name": "banana", "url": "https://en.wikipedia.org/wiki/Banana", }, + { "name": "raspberry", "url": "https://www.raspberrypi.org/" }] +%} +{%- for p in params %} +{%- set name = p["name"] %} +{%- set url = p["url"] %} +apiVersion: batch/v1 +kind: Job +metadata: + name: jobexample-{{ name }} + labels: + jobgroup: jobexample +spec: + template: + metadata: + name: jobexample + labels: + jobgroup: jobexample + spec: + containers: + - name: c + image: busybox + command: ["sh", "-c", "echo Processing URL {{ url }} && sleep 5"] + restartPolicy: Never +--- +{%- endfor %} + +``` + + +上面的模板使用 python 字典列表（第 1-4 行）定义每个作业对象的参数。 +然后使用 for 循环为每组参数（剩余行）生成一个作业 yaml 对象。 +我们利用了多个 yaml 文档可以与 `---` 分隔符连接的事实（倒数第二行）。 +我们可以将输出直接传递给 kubectl 来创建对象。 + + +如果您还没有 jinja2 包则需要安装它: `pip install --user jinja2`。 +现在，使用这个一行 python 程序来展开模板: + +```shell +alias render_template='python -c "from jinja2 import Template; import sys; print(Template(sys.stdin.read()).render());"' +``` + + + +输出可以保存到一个文件，像这样： + +```shell +cat job.yaml.jinja2 | render_template > jobs.yaml +``` + + +或直接发送到 kubectl，如下所示： + +```shell +cat job.yaml.jinja2 | render_template | kubectl apply -f - +``` + + +## 替代方案 + + +如果您有大量作业对象，您可能会发现： + + + +- 即使使用标签，管理这么多 Job 对象也很麻烦。 +- 在一次创建所有作业时，您超过了资源配额，可是您也不希望以递增方式创建 Job 并等待其完成。 +- 同时创建大量作业会使 Kubernetes apiserver、控制器或者调度器负压过大。 + + + +在这种情况下，您可以考虑其他的[作业模式](/docs/concepts/jobs/run-to-completion-finite-workloads/#job-patterns)。 + +{{% /capture %}} From f4ab262cba7cb47a504d5841c06e13a87c04bc88 Mon Sep 17 00:00:00 2001 From: xieyanker Date: Sat, 21 Mar 2020 10:38:44 +0800 Subject: [PATCH 132/140] fix service docs' jump and format error (#19741) * fix service docs' jump and format error * Add jump flag after title --- content/zh/docs/concepts/services-networking/service.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/content/zh/docs/concepts/services-networking/service.md b/content/zh/docs/concepts/services-networking/service.md index 1971035ebd..9c07ff3097 100644 --- a/content/zh/docs/concepts/services-networking/service.md +++ b/content/zh/docs/concepts/services-networking/service.md @@ -169,7 +169,7 @@ also named “my-service”. 上述配置创建一个名称为 "my-service" 的 `Service` 对象，它会将请求代理到使用 TCP 端口 9376，并且具有标签 `"app=MyApp"` 的 `Pod` 上。 Kubernetes 为该服务分配一个 IP 地址（有时称为 "集群IP" ），该 IP 地址由服务代理使用。 -(请参见下面的 [虚拟 IP 和服务代理](#virtual-ips-and-service-proxies)). +(请参见下面的 [VIP 和 Service 代理](#virtual-ips-and-service-proxies)). 服务选择器的控制器不断扫描与其选择器匹配的 Pod，然后将所有更新发布到也称为 “my-service” 的Endpoint对象。 {{< note >}} @@ -336,7 +336,7 @@ responsible for implementing a form of virtual IP for `Services` of type other than [`ExternalName`](#externalname). --> -## VIP 和 Service 代理 +## VIP 和 Service 代理 {#virtual-ips-and-service-proxies} 在 Kubernetes 集群中，每个 Node 运行一个 `kube-proxy` 进程。`kube-proxy` 负责为 `Service` 实现了一种 VIP（虚拟 IP）的形式，而不是 [`ExternalName`](#externalname) 的形式。 @@ -828,6 +828,7 @@ You can also use [Ingress](/docs/concepts/services-networking/ingress/) to expos Kubernetes `ServiceTypes` 允许指定一个需要的类型的 Service，默认是 `ClusterIP` 类型。 `Type` 的取值以及行为如下： + * `ClusterIP`：通过集群的内部 IP 暴露服务，选择该值，服务只能够在集群内部可以访问，这也是默认的 `ServiceType`。 * [`NodePort`](#nodeport)：通过每个 Node 上的 IP 和静态端口（`NodePort`）暴露服务。`NodePort` 服务会路由到 `ClusterIP` 服务，这个 `ClusterIP` 服务会自动创建。通过请求 `:`，可以从集群的外部访问一个 `NodePort` 服务。 * [`LoadBalancer`](#loadbalancer)：使用云提供商的负载局衡器，可以向外部暴露服务。外部的负载均衡器可以路由到 `NodePort` 服务和 `ClusterIP` 服务。 From ef4642ae22774865018d7f0b117f2380ec6b5c80 Mon Sep 17 00:00:00 2001 From: chentanjun <2799194073@qq.com> Date: Sat, 21 Mar 2020 10:40:44 +0800 Subject: [PATCH 133/140] add en content/zh/docs/tasks/inject-data-application/define-environment-variable-container.md (#19754) --- .../define-environment-variable-container.md | 68 +++++++++++++++++-- 1 file changed, 61 insertions(+), 7 deletions(-) diff --git a/content/zh/docs/tasks/inject-data-application/define-environment-variable-container.md b/content/zh/docs/tasks/inject-data-application/define-environment-variable-container.md index a7de28d98b..53795cf0d6 100644 --- a/content/zh/docs/tasks/inject-data-application/define-environment-variable-container.md +++ b/content/zh/docs/tasks/inject-data-application/define-environment-variable-container.md @@ -3,8 +3,21 @@ title: 为容器设置环境变量 content_template: templates/task --- + + {{% capture overview %}} + + 本页将展示如何为 kubernetes Pod 下的容器设置环境变量。 {{% /capture %}} @@ -19,46 +32,81 @@ content_template: templates/task {{% capture steps %}} + + ## 为容器设置一个环境变量 + + 创建 Pod 时，可以为其下的容器设置环境变量。通过配置文件的 `env` 或者 `envFrom` 字段来设置环境变量。 + + 本示例中，将创建一个只包含单个容器的 Pod。Pod 的配置文件中设置环境变量的名称为 `DEMO_GREETING`， 其值为 `"Hello from the environment"`。下面是 Pod 的配置文件内容： {{< codenew file="pods/inject/envars.yaml" >}} + 1. 基于 YAML 文件创建一个 Pod： ```shell kubectl apply -f https://k8s.io/examples/pods/inject/envars.yaml ``` - + + 1. 获取一下当前正在运行的 Pods 信息： ```shell kubectl get pods -l purpose=demonstrate-envars ``` - + + 查询结果应为： ```shell NAME READY STATUS RESTARTS AGE envar-demo 1/1 Running 0 9s ``` - + + 1. 进入该 Pod 下的容器并打开一个命令终端： ```shell kubectl exec -it envar-demo -- /bin/bash ``` + 1. 在命令终端中通过执行 `printenv` 打印出环境变量。 ```shell root@envar-demo:/# printenv ``` + 打印结果应为： ```shell @@ -69,7 +117,10 @@ content_template: templates/task DEMO_GREETING=Hello from the environment DEMO_FAREWELL=Such a sweet sorrow ``` - + + 1. 通过键入 `exit` 退出命令终端。 + * 有关环境变量的更多信息，请参阅[这里](/docs/tasks/inject-data-application/environment-variable-expose-pod-information/)。 * 有关如何通过环境变量来使用 Secret，请参阅[这里](/docs/user-guide/secrets/#using-secrets-as-environment-variables)。 * 关于 [EnvVarSource](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#envvarsource-v1-core) 资源的信息。 {{% /capture %}} - - - From 6cda04257bf3571cbf697494cf7c71b8328964c2 Mon Sep 17 00:00:00 2001 From: chentanjun <2799194073@qq.com> Date: Sat, 21 Mar 2020 14:00:44 +0800 Subject: [PATCH 134/140] sync en zh content/zh/examples/controllers/ (#19764) --- ...7.9.yaml => replication-nginx-1.14.2.yaml} | 32 +++++++------- ...9.2.yaml => replication-nginx-1.16.1.yaml} | 42 +++++++++---------- 2 files changed, 37 insertions(+), 37 deletions(-) rename content/zh/examples/controllers/{replication-nginx-1.7.9.yaml => replication-nginx-1.14.2.yaml} (84%) rename content/zh/examples/controllers/{replication-nginx-1.9.2.yaml => replication-nginx-1.16.1.yaml} (87%) diff --git a/content/zh/examples/controllers/replication-nginx-1.7.9.yaml b/content/zh/examples/controllers/replication-nginx-1.14.2.yaml similarity index 84% rename from content/zh/examples/controllers/replication-nginx-1.7.9.yaml rename to content/zh/examples/controllers/replication-nginx-1.14.2.yaml index 768ab92ca7..2da69a152f 100644 --- a/content/zh/examples/controllers/replication-nginx-1.7.9.yaml +++ b/content/zh/examples/controllers/replication-nginx-1.14.2.yaml @@ -1,16 +1,16 @@ -apiVersion: v1 -kind: ReplicationController -metadata: - name: my-nginx -spec: - replicas: 5 - template: - metadata: - labels: - app: nginx - spec: - containers: - - name: nginx - image: nginx:1.7.9 - ports: - - containerPort: 80 +apiVersion: v1 +kind: ReplicationController +metadata: + name: my-nginx +spec: + replicas: 5 + template: + metadata: + labels: + app: nginx + spec: + containers: + - name: nginx + image: nginx:1.14.2 + ports: + - containerPort: 80 diff --git a/content/zh/examples/controllers/replication-nginx-1.9.2.yaml b/content/zh/examples/controllers/replication-nginx-1.16.1.yaml similarity index 87% rename from content/zh/examples/controllers/replication-nginx-1.9.2.yaml rename to content/zh/examples/controllers/replication-nginx-1.16.1.yaml index f92f2657ed..64efa4d1c3 100644 --- a/content/zh/examples/controllers/replication-nginx-1.9.2.yaml +++ b/content/zh/examples/controllers/replication-nginx-1.16.1.yaml @@ -1,21 +1,21 @@ -apiVersion: v1 -kind: ReplicationController -metadata: - name: my-nginx-v4 -spec: - replicas: 5 - selector: - app: nginx - deployment: v4 - template: - metadata: - labels: - app: nginx - deployment: v4 - spec: - containers: - - name: nginx - image: nginx:1.9.2 - args: ["nginx", "-T"] - ports: - - containerPort: 80 +apiVersion: v1 +kind: ReplicationController +metadata: + name: my-nginx-v4 +spec: + replicas: 5 + selector: + app: nginx + deployment: v4 + template: + metadata: + labels: + app: nginx + deployment: v4 + spec: + containers: + - name: nginx + image: nginx:1.16.1 + args: ["nginx", "-T"] + ports: + - containerPort: 80 From 4605d2b1f9896a0e039bf411da7e93503c21f306 Mon Sep 17 00:00:00 2001 From: Joshua Bezaleel Abednego Date: Sat, 21 Mar 2020 16:04:45 +0700 Subject: [PATCH 135/140] Add translation for ReplicationController in ID localization (#19071) * Add translation for ReplicationController in ID localization * Fix typos like pod, node, web server, spesifikasi, etc. * Fix typos on several words --- .../controllers/replicationcontroller.md | 243 ++++++++++++++++++ .../id/examples/controllers/replication.yaml | 19 ++ 2 files changed, 262 insertions(+) create mode 100644 content/id/docs/concepts/workloads/controllers/replicationcontroller.md create mode 100644 content/id/examples/controllers/replication.yaml diff --git a/content/id/docs/concepts/workloads/controllers/replicationcontroller.md b/content/id/docs/concepts/workloads/controllers/replicationcontroller.md new file mode 100644 index 0000000000..3dad74fb07 --- /dev/null +++ b/content/id/docs/concepts/workloads/controllers/replicationcontroller.md @@ -0,0 +1,243 @@ +--- +title: ReplicationController +feature: + title: Reparasi otomatis + anchor: Bagaimana Sebuah ReplicationController Bekerja + description: > + Mengulang dan menjalankan kembali kontainer yang gagal, mengganti dan menjadwalkan ulang ketika ada Node yang mati, mematikan kontainer yang tidak memberikan respon terhadap health-check yang telah didefinisikan, dan tidak menunjukkannya ke klien sampai siap untuk digunakan. + +content_template: templates/concept +weight: 20 +--- + +{{% capture overview %}} + +{{< note >}} +[`Deployment`](/docs/concepts/workloads/controllers/deployment/) yang mengonfigurasi [`ReplicaSet`](/docs/concepts/workloads/controllers/replicaset/) sekarang menjadi cara yang direkomendasikan untuk melakukan replikasi. +{{< /note >}} + +Sebuah _ReplicationController_ memastikan bahwa terdapat sejumlah Pod yang sedang berjalan dalam suatu waktu tertentu. Dengan kata lain, ReplicationController memastikan bahwa sebuah Pod atau sebuah kumpulan Pod yang homogen selalu berjalan dan tersedia. + +{{% /capture %}} + + +{{% capture body %}} + +## Bagaimana ReplicationController Bekerja + +Jika terdapat terlalu banyak Pod, maka ReplicationController akan membatasi dan mematikan Pod-Pod yang berlebih. Jika terdapat terlalu sedikit, maka ReplicationController akan memulai dan menjalankan Pod-Pod baru lainnya. Tidak seperti Pod yang dibuat secara manual, Pod-Pod yang diatur oleh sebuah ReplicationController akan secara otomatis diganti jika mereka gagal, dihapus, ataupun dimatikan. +Sebagai contoh, Pod-Pod yang kamu miliki akan dibuat ulang dalam sebuah Node setelah terjadi proses pemeliharaan seperti pembaruan kernel. Untuk alasan ini, maka kamu sebaiknya memiliki sebuah ReplicationController bahkan ketika aplikasimu hanya membutuhkan satu buah Pod saja. Sebuah ReplicationController memiliki kemiripan dengan sebuah pengawas proses, tetapi alih-alih mengawasi sebuah proses individu pada sebuah Node, ReplicationController banyak Pod yang terdapat pada beberapa Node. + +ReplicationController seringkali disingkat sebagai "rc" dalam diskusi, dan sebagai _shortcut_ dalam perintah kubectl. + +Sebuah contoh sederhana adalah membuat sebuah objek ReplicationController untuk menjalankan sebuah _instance_ Pod secara berkelanjutan. Contoh pemakaian lainnya adalah untuk menjalankan beberapa replika identik dari sebuah servis yang direplikasi, seperti peladen web. + +## Menjalankan Sebuah Contoh ReplicationController + +Contoh ReplicationController ini mengonfigurasi tiga salinan dari peladen web nginx. + +{{< codenew file="controllers/replication.yaml" >}} + +Jalankan contoh di atas dengan mengunduh berkas contoh dan menjalankan perintah ini: + +```shell +kubectl apply -f https://k8s.io/examples/controllers/replication.yaml +``` +``` +replicationcontroller/nginx created +``` + +Periksa status dari ReplicationController menggunakan perintah ini: + +```shell +kubectl describe replicationcontrollers/nginx +``` +``` +Name: nginx +Namespace: default +Selector: app=nginx +Labels: app=nginx +Annotations: +Replicas: 3 current / 3 desired +Pods Status: 0 Running / 3 Waiting / 0 Succeeded / 0 Failed +Pod Template: + Labels: app=nginx + Containers: + nginx: + Image: nginx + Port: 80/TCP + Environment: + Mounts: + Volumes: +Events: + FirstSeen LastSeen Count From SubobjectPath Type Reason Message + --------- -------- ----- ---- ------------- ---- ------ ------- + 20s 20s 1 {replication-controller } Normal SuccessfulCreate Created pod: nginx-qrm3m + 20s 20s 1 {replication-controller } Normal SuccessfulCreate Created pod: nginx-3ntk0 + 20s 20s 1 {replication-controller } Normal SuccessfulCreate Created pod: nginx-4ok8v +``` + +Tiga Pod telah dibuat namun belum ada yang berjalan, kemungkinan karena _image_ yang sedang di-_pull_. +Beberapa waktu kemudian, perintah yang sama akan menunjukkan: + +```shell +Pods Status: 3 Running / 0 Waiting / 0 Succeeded / 0 Failed +``` + +Untuk melihat semua Pod yang dibuat oleh ReplicationController dalam bentuk yang lebih mudah dibaca mesin, kamu dapat menggunakan perintah seperti ini: + +```shell +pods=$(kubectl get pods --selector=app=nginx --output=jsonpath={.items..metadata.name}) +echo $pods +``` +``` +nginx-3ntk0 nginx-4ok8v nginx-qrm3m +``` + +Pada perintah di atas, selektor yang dimaksud adalah selektor yang sama dengan yang terdapat pada ReplicationController (yang dapat dilihat pada keluaran `kubectl describe`), dan dalam bentuk yang berbeda dengan yang terdapat pada `replication.yaml`. Opsi `--output=jsonpath` menentukan perintah untuh mendapatkan hanya nama dari setiap Pod yang ada pada daftar hasil. + + +## Menulis Spesifikasi ReplicationController + +Seperti semua konfigurasi Kubernetes lainnya, sebuah ReplicationController membutuhkan _field_ `apiVersion`, `kind`, dan `metadata`. + +Untuk informasi umum mengenai berkas konfigurasi, kamu dapat melihat [pengaturan objek](/docs/concepts/overview/working-with-objects/object-management/). + +Sebuah ReplicationController juga membutuhkan [bagian `.spec`](https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status). + +### Templat Pod + +`.spec.template` adalah satu-satunya _field_ yang diwajibkan pada `.spec`. + +`.spec.template` adalah sebuah [templat Pod](/docs/concepts/workloads/pods/pod-overview/#pod-templates). Ia memiliki skema yang sama persis dengan sebuah [Pod](/docs/concepts/workloads/pods/pod/), namun dapat berbentuk _nested_ dan tidak memiliki _field_ `apiVersion` ataupun `kind`. + +Selain _field-field_ yang diwajibkan untuk sebuah Pod, templat Pod pada ReplicationController harus menentukan label dan kebijakan pengulangan kembali yang tepat. Untuk label, pastikan untuk tidak tumpang tindih dengan kontroler lain. Lihat [selektor pod](#selektor-pod). + +Nilai yang diperbolehkan untuk [`.spec.template.spec.restartPolicy`](/docs/concepts/workloads/pods/pod-lifecycle/#restart-policy) hanyalah `Always`, yaitu nilai bawaan jika tidak ditentukan. + +Untuk pengulangan kembali dari sebuah kontainer lokal, ReplicationController mendelegasikannya ke agen pada Node, contohnya [Kubelet](/docs/admin/kubelet/) atau Docker. + +### Label pada ReplicationController + +ReplicationController itu sendiri dapat memiliki label (`.metadata.labels`). Biasanya, kamu akan mengaturnya untuk memiliki nilai yang sama dengan `.spec.template.metadata.labels`; jika `.metadata.labels` tidak ditentukan maka akan menggunakan nilai bawaan yaitu `.spec.template.metadata.labels`. Namun begitu, kedua label ini diperbolehkan untuk memiliki nilai yang berbeda, dan `.metadata.labels` tidak akan memengaruhi perilaku dari ReplicationController. + +### Selektor Pod + +_Field_ `.spec.selector` adalah sebuah [selektor label](/docs/concepts/overview/working-with-objects/labels/#label-selectors). Sebuah ReplicationController mengatur semua Pod dengan label yang sesuai dengan nilai selektor tersebut. Ia tidak membedakan antara Pod yang ia buat atau hapus atau Pod yang dibuat atau dihapus oleh orang atau proses lain. Hal ini memungkinkan ReplicationController untuk digantikan tanpa memengaruhi Pod-Pod yang sedang berjalan. + +Jika ditentukan, `.spec.template.metadata.labels` harus memiliki nilai yang sama dengan `.spec.selector`, atau akan ditolak oleh API. Jika `.spec.selector` tidak ditentukan, maka akan menggunakan nilai bawaan yaitu `.spec.template.metadata.labels`. + +Selain itu, kamu juga sebaiknya tidak membuat Pod dengan label yang cocok dengan selektor ini, baik secara langsung, dengan menggunakan ReplicationController lain, ataupun menggunakan kontroler lain seperti Job. Jika kamu melakukannya, ReplicationController akan menganggap bahwa ia telah membuat Pod-Pod lainnya. Kubernetes tidak akan menghentikan kamu untuk melakukan aksi ini. + +Jika kamu pada akhirnya memiliki beberapa kontroler dengan selektor-selektor yang tumpang tindih, kamu harus mengatur penghapusannya sendiri (lihat [di bawah](#bekerja-dengan-replicationcontroller)). + +### Beberapa Replika + +Kamu dapat menentukan jumlah Pod yang seharusnya berjalan secara bersamaan dengan mengatur nilai `.spec.replicas` dengan jumlah Pod yang kamu inginkan untuk berjalan secara bersamaan. Jumlah yang berjalan dalam satu satuan waktu dapat lebih tinggi ataupun lebih rendah, seperti jika replika-replika tersebut melewati proses penambahan atau pengurangan, atau jika sebuah Pod melalui proses _graceful shutdown_, dan penggantinya telah dijalankan terlebih dahulu. + +Jika kamu tidak menentukan nilai dari `.spec.replicas`, maka akan digunakan nilai bawaan 1. + +## Bekerja dengan ReplicationController + +### Menghapus Sebuah ReplicationController dan Pod-nya + +Untuk menghapus sebuah ReplicationController dan Pod-Pod yang berhubungan dengannya, gunakan perintah [`kubectl delete`](/docs/reference/generated/kubectl/kubectl-commands#delete). Kubectl akan mengatur ReplicationController ke nol dan menunggunya untuk menghapus setiap Pod sebelum menghapus ReplicationController itu sendiri. Jika perintah kubectl ini terhenti, maka dapat diulang kembali. + +Ketika menggunakan REST API atau _library_ klien go, maka kamu perlu melakukan langkah-langkahnya secara eksplisit (mengatur replika-replika ke 0, menunggu penghapusan Pod, dan barulah menghapus ReplicationController). + +### Menghapus Hanya ReplicationController + +Kamu dapat menghapus ReplicationController tanpa memengaruhi Pod-Pod yang berhubungan dengannya. + +Dengan menggunakan kubectl, tentukan opsi `--cascade=false` ke [`kubectl delete`](/docs/reference/generDeated/kubectl/kubectl-commands#delete). + +Ketika menggunakan REST API atau _library_ klien go, cukup hapus objek ReplicationController. + +Ketika ReplicationController yang asli telah dihapus, kamu dapat membuat ReplicationController yang baru sebagai penggantinya. Selama `.spec.selector` yang lama dan baru memiliki nilai yang sama, maka ReplicationController baru akan mengadopsi Pod-Pod yang lama. +Walaupun begitu, ia tidak akan melakukan usaha apapun untuk membuat Pod-Pod yang telah ada sebelumnya untuk sesuai dengan templat Pod yang baru dan berbeda. +Untuk memperbarui Pod-Pod ke spesifikasi yang baru dengan cara yang terkontrol, gunakan [pembaruan bergulir](#pembaruan-bergulir). + +### Mengisolasi Pod dari ReplicationController + +Pod-Pod dapat dihapus dari kumpulan target sebuah ReplicationController dengan mengganti nilai dari labelnya. Teknik ini dapat digunakan untuk mencopot Pod-Pod dari servis untuk keperluan pengawakutuan (_debugging_), pemulihan data, dan lainnya. Pod-Pod yang dicopot dengan cara ini dapat digantikan secara otomatis (dengan asumsi bahwa jumlah replika juga tidak berubah). + +## Pola penggunaan umum + +### Penjadwalan ulang + +Seperti yang telah disebutkan sebelumnya, baik kamu memiliki hanya 1 Pod untuk tetap dijalankan, ataupun 1000, ReplicationController akan memastikan tersedianya jumlah Pod yang telat ditentukan, bahkan ketika terjadi kegagalan Node atau terminasi Pod (sebagai contoh karena adanya tindakan dari agen kontrol lain). + +### Penskalaan + +ReplicationController memudahkan penskalaan jumlah replika, baik meningkatkan ataupun mengurangi, secara manual ataupun dengan agen kontrol penskalaan otomatis, dengan hanya mengubah nilai dari _field_ `replicas`. + +### Pembaruan bergulir + +ReplicationController didesain untuk memfasilitasi pembaruan bergulir untuk sebuah servis dengan mengganti Pod-Pod satu per satu. + +Seperti yang telah dijelaskan di [#1353](http://issue.k8s.io/1353), pendekatan yang direkomendasikan adalah dengan membuat ReplicationController baru dengan 1 replika, skala kontroler yang baru (+1) atau yang lama (-1) satu per satu, dan kemudian hapus kontroler lama setelah menyentuh angka 0 replika. Hal ini memungkinkan pembaruan dilakukan dengan dapat diprediksi terlepas dari adanya kegagalan yang tak terduga. + +Idealnya, kontroler pembaruan bergulir akan memperhitungkan kesiapan dari aplikasi, dan memastikan cukupnya jumlah Pod yang secara produktif meladen kapanpun. + +Dua ReplicationController diharuskan untuk memiliki setidaknya satu label yang berbeda, seperti _tag_ _image_ dari kontainer utama dari Pod, karena pembaruan bergulir biasanya dilakukan karena adanya pembaruan _image_. + +Pembaruan bergulir diimplementasikan pada perkakas klien [`kubectl rolling-update`](/docs/reference/generated/kubectl/kubectl-commands#rolling-update). Lihat [`kubectl rolling-update` task](/docs/tasks/run-application/rolling-update-replication-controller/) untuk contoh-contoh yang lebih konkrit. + +### Operasi rilis majemuk + +Selain menjalankan beberapa rilis dari sebuah aplikasi ketika proses pembaruan bergulir sedang berjalan, adalah hal yang awam untuk menjalankan beberapa rilis untuk suatu periode waktu tertentu, atau bahkan secara kontinu, menggunakan operasi rilis majemuk. Operasi-operasi ini akan dibedakan menggunakan label. + +Sebagai contoh, sebuah servis dapat menyasar semua Pod dengan `tier in (frontend), environment in (prod)`. Anggap kamu memiliki 10 Pod tiruan yang membangun _tier_ ini tetapi kamu ingin bisa menggunakan 'canary' terhadap versi baru dari komponen ini. Kamu dapat mengatur sebuah ReplicationController dengan nilai `replicas` 9 untuk replika-replikanya, dengan label `tier=frontend, environment=prod, track=stable`, dan ReplicationController lainnya dengan nilai `replicas` 1 untuk canary, dengan label `tier=frontend, environment=prod, track=canary`. Sekarang servis sudah mencakup baik canary maupun Pod-Pod yang bukan canary. Kamu juga dapat mencoba-coba ReplicationController secara terpisah untuk melakukan pengujian, mengamati hasilnya, dan lainnya. + +### Menggunakan ReplicationController dengan Service + +Beberapa ReplicationController dapat berada di belakang sebuah Service, sedemikian sehingga, sebagai contoh, sebagian _traffic_ dapat ditujukan ke versi lama, dan sebagian lainnya ke versi yang baru. + +Sebuah ReplicationController tidak akan berhenti dengan sendirinya, namun ia tidak diekspektasikan untuk berjalan selama Service-Service yang ada. Service dapat terdiri dari berbagai Pod yang dikontrol beberapa ReplicationController, dan terdapat kemungkinan bahwa beberapa ReplicationController untuk dibuat dan dimatikan dalam jangka waktu hidup Service (contohnya adalah untuk melakukan pembaruan Pod-Pod yang menjalankan Service). Baik Service itu sendiri dan kliennya harus tetap dalam keadaan tidak mempunyai pengetahuan terhadap ReplicationController yang memelihara Pod-Pod dari Service tersebut. + +## Menulis program untuk Replikasi + +Pod-Pod yang dibuat oleh ReplicationController ditujukan untuk dapat sepadan dan memiliki semantik yang identik, walaupun konfigurasi mereka dapat berbeda seiring keberjalanan waktunya. Ini adalah contoh yang cocok untuk peladen _stateless_, namun ReplicationController juga dapat digunakan untuk memelihara ketersediaan dari aplikasi-aplikasi yang _master-elected_, _sharded_, _worker-pool_. Aplikasi-aplikasi seperti itu sebaiknya menggunakan mekanisme penetapan kerja yang dinamis, seperti [antrian kerja RabbitMQ](https://www.rabbitmq.com/tutorials/tutorial-two-python.html), berlainan dengan pengubahan statis/satu kali dari konfigurasi setiap Pod, yang dipandang sebagai sebuah _anti-pattern_. Pengubahan apapun yang dilakukan terhadap Pod, seperti _auto-sizing_ vertikal dari sumber daya (misalnya cpu atau memori), sebaiknya dilakukan oleh proses kontroller luring lainnya, dan bukan oleh ReplicationController itu sendiri. + +## Tanggung Jawab ReplicationController + +ReplicationController hanya memastikan ketersediaan dari sejumlah Pod yang cocok dengan selektor label dan berjalan dengan baik. Saat ini, hanya Pod yang diterminasi yang dijadikan pengecualian dari penghitungan. Kedepannya, [kesiapan](http://issue.k8s.io/620) dan informasi yang ada lainnya dari sistem dapat menjadi pertimbangan, kami dapat meningkatkan kontrol terhadap kebijakan penggantian, dan kami berencana untuk menginformasikan kejadian (_event_) yang dapat digunakan klien eksternal untuk implementasi penggantian yang sesuai dan/atau kebijakan pengurangan. + +ReplicationController akan selalu dibatasi terhadap tanggung jawab spesifik ini. Ia tidak akan melakukan _probe_ kesiapan atau keaktifan. Daripada melakukan _auto-scaling_, ia ditujukan untuk dikontrol oleh _auto-scaler_ eksternal (seperti yang didiskusikan pada [#492](http://issue.k8s.io/492)), yang akan mengganti _field_ `replicas`. Kami tidak akan menambahkan kebijakan penjadwalan (contohnya [_spreading_](http://issue.k8s.io/367#issuecomment-48428019)) untuk ReplicationController. Ia juga tidak seharusnya melakukan verifikasi terhadap Pod-Pod yang sedang dikontrol yang cocok dengan spesifikasi templat saat ini, karena hal itu dapat menghambat _auto-sizing_ dan proses otomatis lainnya. Demikian pula batas waktu penyelesaian, pengurutan _dependencies_, ekspansi konfigurasi, dan fitur-fitur lain yang seharusnya berada di komponen lain. Kami juga bahkan berencana untuk mengeluarkan mekanisme pembuatan Pod secara serentak ([#170](http://issue.k8s.io/170)). + +ReplicationController ditujukan untuk menjadi primitif komponen yang dapat dibangun untuk berbagai kebutuhan. Kami menargetkan API dengan tingkatan yang lebih tinggi dan/atau perkakas-perkakas untuk dibangun di atasnya dan primitif tambahan lainnya untuk kenyamanan pengguna kedepannya. Operasi-operasi makro yang sudah didukung oleh kubectl (_run_, _scale_, _rolling-update_) adalah contoh _proof-of-concept_ dari konsep ini. Sebagai contohnya, kita dapat menganggap sesuatu seperti [Asgard](http://techblog.netflix.com/2012/06/asgard-web-based-cloud-management-and.html) yang mengatur beberapa ReplicationController, _auto-scaler_, servis, kebijakan penjadwalan, canary, dan yang lainnya. + + +## Objek API + +ReplicationController adalah sebuah sumber daya _top-level_ pada REST API Kubernetes. Detil dari objek API dapat ditemukan di: [objek API ReplicationController](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#replicationcontroller-v1-core). + +## Alternatif untuk ReplicationController + +### ReplicaSet + +[`ReplicaSet`](/docs/concepts/workloads/controllers/replicaset/) adalah kelanjutan dari ReplicationController yang mendukung selektor [selektor label _set-based_](/docs/concepts/overview/working-with-objects/labels/#set-based-requirement) yang baru. Umumnya digunakan oleh [`Deployment`](/docs/concepts/workloads/controllers/deployment/) sebagai mekanisme untuk mengorkestrasi pembuatan, penghapusan, dan pembaruan Pod. +Perhatikan bahwa kami merekomendasikan untuk menggunakan Deployment sebagai ganti dari menggunakan ReplicaSet secara langsung, kecuali jika kamu membutuhkan orkestrasi pembaruan khusus atau tidak membutuhkan pembaruan sama sekali. + + +### Deployment (Direkomendasikan) + +[`Deployment`](/docs/concepts/workloads/controllers/deployment/) adalah objek API tingkat tinggi yang memperbarui ReplicaSet dan Pod-Pod di bawahnya yang mirip dengan cara kerja `kubectl rolling-update`. Deployment direkomendasikan jika kamu menginginkan fungsionalitas dari pembaruan bergulir ini, karena tidak seperti `kubectl rolling-update`, Deployment memiliki sifat deklaratif, _server-side_, dan memiliki beberapa fitur tambahan lainnya. + +### Pod sederhana + +Tidak seperti pada kasus ketika pengguna secara langsung membuat Pod, ReplicationController menggantikan Pod-Pod yang dihapus atau dimatikan untuk alasan apapun, seperti pada kasus kegagalan Node atau pemeliharaan Node yang disruptif, seperti pembaruan kernel. Untuk alasan ini, kami merekomendasikan kamu untuk menggunakan ReplicationController bahkan ketika aplikasimu hanya membutuhkan satu Pod saja. Anggap hal ini mirip dengan pengawas proses, hanya pada kasus ini mengawasi banyak Pod yang terdapat pada berbagai Node dan bukan proses-proses tunggal pada satu Node. ReplicationController mendelegasikan pengulangan kontainer lokal ke agen yang terdapat dalam Node (contohnya Kubelet atau Docker). + +### Job + +Gunakan [`Job`](/docs/concepts/jobs/run-to-completion-finite-workloads/) sebagai ganti ReplicationController untuk Pod-Pod yang diharapkan diterminasi dengan sendirinya (seperti _batch jobs_). + +### DaemonSet + +Gunakan [`DaemonSet`](/docs/concepts/workloads/controllers/daemonset/) sebagai ganti ReplicationController untuk Pod-Pod yang menyediakan fungsi pada level mesin, seperti pengamatan mesin atau pencatatan mesin. Pod-Pod ini memiliki waktu hidup yang bergantung dengan waktu hidup mesin: Pod butuh untuk dijalankan di mesin sebelum Pod-Pod lainnya dimulai, dan aman untuk diterminasi ketika mesin sudah siap untuk dinyalakan ulang atau dimatikan. + +## Informasi lanjutan + +Baca [Menjalankan Kontroler Replikasi AP _Stateless_](/docs/tutorials/stateless-application/run-stateless-ap-replication-controller/). + +{{% /capture %}} diff --git a/content/id/examples/controllers/replication.yaml b/content/id/examples/controllers/replication.yaml new file mode 100644 index 0000000000..e43ccbc32d --- /dev/null +++ b/content/id/examples/controllers/replication.yaml @@ -0,0 +1,19 @@ +apiVersion: v1 +kind: ReplicationController +metadata: + name: nginx +spec: + replicas: 3 + selector: + app: nginx + template: + metadata: + name: nginx + labels: + app: nginx + spec: + containers: + - name: nginx + image: nginx + ports: + - containerPort: 80 \ No newline at end of file From 7c82b6e26bdaa02c567dfa599368117f261eaa6f Mon Sep 17 00:00:00 2001 From: xieyanker Date: Sat, 21 Mar 2020 20:08:44 +0800 Subject: [PATCH 136/140] fix some glossary translate (#19770) --- .../docs/reference/glossary/applications.md | 1 + .../docs/reference/glossary/control-plane.md | 3 +- .../docs/reference/glossary/pod-lifecycle.md | 40 ++++++++++--------- content/zh/docs/reference/glossary/service.md | 2 +- .../docs/reference/glossary/storage-class.md | 38 ++++++++++-------- .../zh/docs/reference/glossary/upstream.md | 2 +- 6 files changed, 47 insertions(+), 39 deletions(-) diff --git a/content/zh/docs/reference/glossary/applications.md b/content/zh/docs/reference/glossary/applications.md index cd02a4ee14..cbff609142 100644 --- a/content/zh/docs/reference/glossary/applications.md +++ b/content/zh/docs/reference/glossary/applications.md @@ -23,4 +23,5 @@ tags: - fundamental --- --> + 各种容器化应用运行所在的层。 diff --git a/content/zh/docs/reference/glossary/control-plane.md b/content/zh/docs/reference/glossary/control-plane.md index 1916feac7f..05f3f57978 100644 --- a/content/zh/docs/reference/glossary/control-plane.md +++ b/content/zh/docs/reference/glossary/control-plane.md @@ -25,7 +25,8 @@ tags: - fundamental --- --> + - 容器编排层，它暴露 API 和接口来定义、部署容器和管理容器的生命周期。 \ No newline at end of file + 容器编排层，它暴露 API 和接口来定义、部署容器和管理容器的生命周期。 diff --git a/content/zh/docs/reference/glossary/pod-lifecycle.md b/content/zh/docs/reference/glossary/pod-lifecycle.md index 67376a475c..bb3811ccf8 100644 --- a/content/zh/docs/reference/glossary/pod-lifecycle.md +++ b/content/zh/docs/reference/glossary/pod-lifecycle.md @@ -1,3 +1,17 @@ +--- +title: Pod 生命周期 +id: pod-lifecycle +date: 2019-02-17 +full-link: /docs/concepts/workloads/pods/pod-lifecycle/ +related: + - pod + - container +tags: + - fundamental +short_description: > + 关于 Pod 在其生命周期中处于哪个阶段的更高层次概述。 +--- + --> ---- -title: Pod 生命周期 -id: pod-lifecycle -date: 2019-02-17 -full-link: /docs/concepts/workloads/pods/pod-lifecycle/ -related: - - pod - - container -tags: - - fundamental -short_description: > - 关于 Pod 在其生命周期中处于哪个阶段的更高层次概述。 - ---- + + + 关于 Pod 在其生命周期中处于哪个阶段的更高层次概述。 - + + diff --git a/content/zh/docs/reference/glossary/service.md b/content/zh/docs/reference/glossary/service.md index 67a1ba8404..54a03b3b1a 100755 --- a/content/zh/docs/reference/glossary/service.md +++ b/content/zh/docs/reference/glossary/service.md @@ -4,7 +4,7 @@ id: service date: 2018-04-12 full_link: /docs/concepts/services-networking/service/ short_description: > - A way to expose an application running on a set of Pods as a network service. + 将运行在一组 {{< glossary_tooltip text="Pods" term_id="pod" >}} 上的应用程序公开为网络服务的抽象方法。 aka: tags: diff --git a/content/zh/docs/reference/glossary/storage-class.md b/content/zh/docs/reference/glossary/storage-class.md index 52886478eb..49cadefe44 100644 --- a/content/zh/docs/reference/glossary/storage-class.md +++ b/content/zh/docs/reference/glossary/storage-class.md @@ -1,20 +1,3 @@ - - --- title: 存储类别 id: storageclass @@ -29,9 +12,30 @@ tags: - storage --- + + + + StorageClass 是管理员用来描述不同的可用存储类型的一种方法。 + diff --git a/content/zh/docs/reference/glossary/upstream.md b/content/zh/docs/reference/glossary/upstream.md index 7f2c7ded3e..118b3838f7 100644 --- a/content/zh/docs/reference/glossary/upstream.md +++ b/content/zh/docs/reference/glossary/upstream.md @@ -4,7 +4,7 @@ id: upstream date: 2018-04-12 full_link: short_description: > - May refer to: core Kubernetes or the source repo from which a repo was forked. + 可以参考：核心 Kubernetes 仓库或作为当前仓库派生来源的来源仓库。 aka: tags: From 3dca72800ce4868416ea8ccb4a38714294e3d282 Mon Sep 17 00:00:00 2001 From: Gede Wahyu Adi Pramana Date: Sun, 22 Mar 2020 16:30:44 +0700 Subject: [PATCH 137/140] Translate networking page to ID localization (#19420) Co-Authored-By: yofriadi Co-Authored-By: Giri Kuncoro Co-authored-by: yofriadi Co-authored-by: Giri Kuncoro --- .../cluster-administration/networking.md | 228 ++++++++++++++++++ 1 file changed, 228 insertions(+) create mode 100644 content/id/docs/concepts/cluster-administration/networking.md diff --git a/content/id/docs/concepts/cluster-administration/networking.md b/content/id/docs/concepts/cluster-administration/networking.md new file mode 100644 index 0000000000..23fd828fa7 --- /dev/null +++ b/content/id/docs/concepts/cluster-administration/networking.md @@ -0,0 +1,228 @@ +--- +title: Jaringan Kluster +content_template: templates/concept +weight: 50 +--- + +{{% capture overview %}} +Jaringan adalah bagian utama dari Kubernetes, tetapi bisa menjadi sulit +untuk memahami persis bagaimana mengharapkannya bisa bekerja. +Ada 4 masalah yang berbeda untuk diatasi: + +1. Komunikasi antar kontainer yang sangat erat: hal ini diselesaikan oleh + [Pod](/docs/concepts/workloads/pods/pod/) dan komunikasi `localhost`. +2. Komunikasi antar Pod: ini adalah fokus utama dari dokumen ini. +3. Komunikasi Pod dengan Service: ini terdapat di [Service](/docs/concepts/services-networking/service/). +4. Komunikasi eksternal dengan Service: ini terdapat di [Service](/docs/concepts/services-networking/service/). + +{{% /capture %}} + + +{{% capture body %}} + +Kubernetes adalah tentang berbagi mesin antar aplikasi. Pada dasarnya, +saat berbagi mesin harus memastikan bahwa dua aplikasi tidak mencoba menggunakan +_port_ yang sama. Mengkoordinasikan _port_ di banyak pengembang sangat sulit +dilakukan pada skala yang berbeda dan memaparkan pengguna ke masalah +tingkat kluster yang di luar kendali mereka. + +Alokasi _port_ yang dinamis membawa banyak komplikasi ke sistem - setiap aplikasi +harus menganggap _port_ sebagai _flag_, _server_ API harus tahu cara memasukkan +nomor _port_ dinamis ke dalam blok konfigurasi, Service-Service harus tahu cara +menemukan satu sama lain, dll. Sebaliknya daripada berurusan dengan ini, +Kubernetes mengambil pendekatan yang berbeda. + +## Model jaringan Kubernetes + +Setiap Pod mendapatkan alamat IP sendiri. Ini berarti kamu tidak perlu secara langsung membuat tautan antara Pod dan kamu hampir tidak perlu berurusan dengan memetakan _port_ kontainer ke _port_ pada _host_. Ini menciptakan model yang bersih, kompatibel dengan yang sebelumnya dimana Pod dapat diperlakukan seperti halnya VM atau _host_ fisik dari perspektif alokasi _port_, penamaan, _service discovery_, _load balancing_, konfigurasi aplikasi, dan migrasi. + +Kubernetes memberlakukan persyaratan mendasar berikut pada setiap implementasi jaringan (kecuali kebijakan segmentasi jaringan yang disengaja): + + * Pod pada suatu Node dapat berkomunikasi dengan semua Pod pada semua Node tanpa NAT + * agen pada suatu simpul (mis. _daemon_ sistem, kubelet) dapat berkomunikasi dengan semua Pod pada Node itu + +Catatan: Untuk platform yang mendukung Pod yang berjalan di jaringan _host_ (mis. Linux): + + * Pod di jaringan _host_ dari sebuah Node dapat berkomunikasi dengan semua Pod pada semua Node tanpa NAT + +Model ini tidak hanya sedikit kompleks secara keseluruhan, tetapi pada prinsipnya kompatibel dengan keinginan Kubernetes untuk memungkinkan _low-friction porting_ dari aplikasi dari VM ke kontainer. Jika pekerjaan kamu sebelumnya dijalankan dalam VM, VM kamu memiliki IP dan dapat berbicara dengan VM lain di proyek yang sama. Ini adalah model dasar yang sama. + +Alamat IP Kubernetes ada di lingkup Pod - kontainer dalam Pod berbagi jaringan _namespace_ mereka - termasuk alamat IP mereka. Ini berarti bahwa kontainer dalam Pod semua dapat mencapai _port_ satu sama lain di `_localhost_`. Ini juga berarti bahwa kontainer dalam Pod harus mengoordinasikan penggunaan _port_, tetapi ini tidak berbeda dari proses di VM. Ini disebut model "IP-per-pod". + +## Bagaimana menerapkan model jaringan Kubernetes + +Ada beberapa cara agar model jaringan ini dapat diimplementasikan. Dokumen ini bukan studi lengkap tentang berbagai metode, tetapi semoga berfungsi sebagai pengantar ke berbagai teknologi dan berfungsi sebagai titik awal. + +Opsi jaringan berikut ini disortir berdasarkan abjad - urutan tidak menyiratkan status istimewa apa pun. + +### ACI + +[Infrastruktur Sentral Aplikasi Cisco](https://www.cisco.com/c/en/us/solutions/data-center-virtualization/application-centric-infrastructure/index.html) menawarkan solusi SDN overlay dan underlay terintegrasi yang mendukung kontainer, mesin virtual, dan _bare metal server_. [ACI](https://www.github.com/noironetworks/aci-containers) menyediakan integrasi jaringan kontainer untuk ACI. Tinjauan umum integrasi disediakan [di sini](https://www.cisco.com/c/dam/en/us/solutions/collateral/data-center-virtualization/application-centric-infrastructure/solution-overview-c22-739493.pdf). + +### AOS dari Apstra + +[AOS](http://www.apstra.com/products/aos/) adalah sistem Jaringan Berbasis Intent yang menciptakan dan mengelola lingkungan pusat data yang kompleks dari platform terintegrasi yang sederhana. AOS memanfaatkan desain terdistribusi sangat _scalable_ untuk menghilangkan pemadaman jaringan sambil meminimalkan biaya. + +Desain Referensi AOS saat ini mendukung _host_ yang terhubung dengan Lapis-3 yang menghilangkan masalah peralihan Lapis-2 yang lama. Host Lapis-3 ini bisa berupa _server_ Linux (Debian, Ubuntu, CentOS) yang membuat hubungan tetangga BGP secara langsung dengan _top of rack switches_ (TORs). AOS mengotomatisasi kedekatan perutean dan kemudian memberikan kontrol yang halus atas _route health injections_ (RHI) yang umum dalam _deployment_ Kubernetes. + +AOS memiliki banyak kumpulan endpoint REST API yang memungkinkan Kubernetes dengan cepat mengubah kebijakan jaringan berdasarkan persyaratan aplikasi. Peningkatan lebih lanjut akan mengintegrasikan model Grafik AOS yang digunakan untuk desain jaringan dengan penyediaan beban kerja, memungkinkan sistem manajemen ujung ke ujung untuk layanan cloud pribadi dan publik. + +AOS mendukung penggunaan peralatan vendor umum dari produsen termasuk Cisco, Arista, Dell, Mellanox, HPE, dan sejumlah besar sistem white-box dan sistem operasi jaringan terbuka seperti Microsoft SONiC, Dell OPX, dan Cumulus Linux. + +Detail tentang cara kerja sistem AOS dapat diakses di sini: http://www.apstra.com/products/how-it-works/ + +### AWS VPC CNI untuk Kubernetes + +[AWS VPC CNI](https://github.com/aws/amazon-vpc-cni-k8s) menawarkan jaringan AWS _Virtual Private Cloud_ (VPC) terintegrasi untuk kluster Kubernetes. Plugin CNI ini menawarkan _throughput_ dan ketersediaan tinggi, latensi rendah, dan _jitter_ jaringan minimal. Selain itu, pengguna dapat menerapkan jaringan AWS VPC dan praktik keamanan terbaik untuk membangun kluster Kubernetes. Ini termasuk kemampuan untuk menggunakan catatan aliran VPC, kebijakan perutean VPC, dan grup keamanan untuk isolasi lalu lintas jaringan. + +Menggunakan _plugin_ CNI ini memungkinkan Pod Kubernetes memiliki alamat IP yang sama di dalam Pod seperti yang mereka lakukan di jaringan VPC. CNI mengalokasikan AWS _Elastic Networking Interfaces_ (ENIs) ke setiap node Kubernetes dan menggunakan rentang IP sekunder dari setiap ENI untuk Pod pada Node. CNI mencakup kontrol untuk pra-alokasi ENI dan alamat IP untuk waktu mulai Pod yang cepat dan memungkinkan kluster besar hingga 2.000 Node. + +Selain itu, CNI dapat dijalankan bersama [Calico untuk penegakan kebijakan jaringan](https://docs.aws.amazon.com/eks/latest/userguide/calico.html). Proyek AWS VPC CNI adalah _open source_ dengan [dokumentasi di GitHub](https://github.com/aws/amazon-vpc-cni-k8s). + +### Big Cloud Fabric dari Big Switch Networks + +[Big Cloud Fabric](https://www.bigswitch.com/container-network-automation) adalah arsitektur jaringan asli layanan cloud, yang dirancang untuk menjalankan Kubernetes di lingkungan cloud pribadi / lokal. Dengan menggunakan SDN fisik & _virtual_ terpadu, Big Cloud Fabric menangani masalah yang sering melekat pada jaringan kontainer seperti penyeimbangan muatan, visibilitas, pemecahan masalah, kebijakan keamanan & pemantauan lalu lintas kontainer. + +Dengan bantuan arsitektur multi-penyewa Pod virtual pada Big Cloud Fabric, sistem orkestrasi kontainer seperti Kubernetes, RedHat OpenShift, Mesosphere DC/OS & Docker Swarm akan terintegrasi secara alami bersama dengan sistem orkestrasi VM seperti VMware, OpenStack & Nutanix. Pelanggan akan dapat terhubung dengan aman berapa pun jumlah klusternya dan memungkinkan komunikasi antar penyewa di antara mereka jika diperlukan. + +Terbaru ini BCF diakui oleh Gartner sebagai visioner dalam [_Magic Quadrant_](http://go.bigswitch.com/17GatedDocuments-MagicQuadrantforDataCenterNetworking_Reg.html). Salah satu penyebaran BCF Kubernetes di tempat (yang mencakup Kubernetes, DC/OS & VMware yang berjalan di beberapa DC di berbagai wilayah geografis) juga dirujuk [di sini](https://portworx.com/architects-corner-kubernetes-satya-komala-nio/). + +### Cilium + +[Cilium](https://github.com/cilium/cilium) adalah perangkat lunak _open source_ untuk menyediakan dan secara transparan mengamankan konektivitas jaringan antar kontainer aplikasi. Cilium mengetahui L7/HTTP dan dapat memberlakukan kebijakan jaringan pada L3-L7 menggunakan model keamanan berbasis identitas yang dipisahkan dari pengalamatan jaringan. + +### CNI-Genie dari Huawei + +[CNI-Genie](https://github.com/Huawei-PaaS/CNI-Genie) adalah _plugin_ CNI yang memungkinkan Kubernetes [secara bersamaan memiliki akses ke berbagai implementasi](https://github.com/Huawei-PaaS /CNI-Genie/blob/master/docs/multiple-cni-plugins/README.md#what-cni-genie-feature-1-multiple-cni-plugins-enables) dari [model jaringan Kubernetes] (https://git.k8s.io/website/docs/concepts/cluster-administration/networking.md#kubernetes-model) dalam _runtime_. Ini termasuk setiap implementasi yang berjalan sebagai [_plugin_ CNI](https://github.com/containernetworking/cni#3rd-party-plugins), seperti [Flannel](https://github.com/coreos/flannel#flannel), [Calico](http://docs.projectcalico.org/), [Romana](http://romana.io), [Weave-net](https://www.weave.works/products/weave-net/). + +CNI-Genie juga mendukung [menetapkan beberapa alamat IP ke sebuah Pod](https://github.com/Huawei-PaaS/CNI-Genie/blob/master/docs/multiple-ips/README.md#feature-2-extension-cni-genie-multiple-ip-address-per-pod), masing-masing dari _plugin_ CNI yang berbeda. + +### cni-ipvlan-vpc-k8s + +[cni-ipvlan-vpc-k8s](https://github.com/lyft/cni-ipvlan-vpc-k8s) berisi satu set _plugin_ CNI dan IPAM untuk menyediakan kemudahan, host-lokal, latensi rendah, _throughput_ tinggi , dan tumpukan jaringan yang sesuai untuk Kubernetes dalam lingkungan Amazon Virtual Private Cloud (VPC) dengan memanfaatkan Amazon Elastic Network Interfaces (ENI) dan mengikat IP yang dikelola AWS ke Pod-Pod menggunakan _driver_ IPvlan _kernel_ Linux dalam mode L2. + +Plugin ini dirancang untuk secara langsung mengkonfigurasi dan _deploy_ dalam VPC. Kubelet melakukan _booting_ dan kemudian mengkonfigurasi sendiri dan memperbanyak penggunaan IP mereka sesuai kebutuhan tanpa memerlukan kompleksitas yang sering direkomendasikan untuk mengelola jaringan _overlay_, BGP, menonaktifkan pemeriksaan sumber/tujuan, atau menyesuaikan tabel rute VPC untuk memberikan _subnet_ per _instance_ ke setiap _host_ (yang terbatas hingga 50-100 masukan per VPC). Singkatnya, cni-ipvlan-vpc-k8s secara signifikan mengurangi kompleksitas jaringan yang diperlukan untuk menggunakan Kubernetes yang berskala di dalam AWS. + +### Contiv + +[Contiv](https://github.com/contiv/netplugin) menyediakan jaringan yang dapat dikonfigurasi (_native_ l3 menggunakan BGP, _overlay_ menggunakan vxlan, classic l2, atau Cisco-SDN / ACI) untuk berbagai kasus penggunaan. [Contiv](http://contiv.io) semuanya open sourced. + +### Contrail / Tungsten Fabric + +[Contrail](http://www.juniper.net/us/en/products-services/sdn/contrail/contrail-networking/), berdasarkan [Tungsten Fabric](https://tungsten.io), adalah platform virtualisasi jaringan dan manajemen kebijakan _multi-cloud_ yang benar-benar terbuka. Contrail dan Tungsten Fabric terintegrasi dengan berbagai sistem orkestrasi seperti Kubernetes, OpenShift, OpenStack dan Mesos, dan menyediakan mode isolasi yang berbeda untuk mesin _virtual_, banyak kontainer / banyak Pod dan beban kerja _bare metal_. + +### DANM + +[DANM] (https://github.com/nokia/danm) adalah solusi jaringan untuk beban kerja telco yang berjalan di kluster Kubernetes. Dibangun dari komponen-komponen berikut: + + * Plugin CNI yang mampu menyediakan antarmuka IPVLAN dengan fitur-fitur canggih + * Modul IPAM built-in dengan kemampuan mengelola dengan jumlah banyak, _cluster-wide_, _discontinous_ jaringan L3 dan menyediakan skema dinamis, statis, atau tidak ada permintaan skema IP + * Metaplugin CNI yang mampu melampirkan beberapa antarmuka jaringan ke kontainer, baik melalui CNI sendiri, atau mendelegasikan pekerjaan ke salah satu solusi CNI populer seperti SRI-OV, atau Flannel secara paralel + * Pengontrol Kubernetes yang mampu mengatur secara terpusat antarmuka VxLAN dan VLAN dari semua _host_ Kubernetes + * Pengontrol Kubernetes lain yang memperluas konsep _service discovery_ berbasis servis untuk bekerja di semua antarmuka jaringan Pod + +Dengan _toolset_ ini, DANM dapat memberikan beberapa antarmuka jaringan yang terpisah, kemungkinan untuk menggunakan ujung belakang jaringan yang berbeda dan fitur IPAM canggih untuk Pod. + +### Flannel + +[Flannel] (https://github.com/coreos/flannel#flannel) adalah jaringan overlay yang sangat sederhana yang memenuhi persyaratan Kubernetes. Banyak orang telah melaporkan kesuksesan dengan Flannel dan Kubernetes. + +### Google Compute Engine (GCE) + +Untuk skrip konfigurasi kluster Google Compute Engine, [perutean lanjutan](https://cloud.google.com/vpc/docs/routes) digunakan untuk menetapkan setiap VM _subnet_ (standarnya adalah `/24` - 254 IP). Setiap lalu lintas yang terikat untuk _subnet_ itu akan dialihkan langsung ke VM oleh _fabric_ jaringan GCE. Ini adalah tambahan untuk alamat IP "utama" yang ditugaskan untuk VM, yang NAT'ed untuk akses internet keluar. Sebuah linux _bridge_ (disebut `cbr0`) dikonfigurasikan untuk ada pada subnet itu, dan diteruskan ke _flag_ `-bridge` milik docker. + +Docker dimulai dengan: + +```shell +DOCKER_OPTS="--bridge=cbr0 --iptables=false --ip-masq=false" +``` + +Jembatan ini dibuat oleh Kubelet (dikontrol oleh _flag_ `--network-plugin=kubenet`) sesuai dengan `.spec.podCIDR` yang dimiliki oleh Node. + +Docker sekarang akan mengalokasikan IP dari blok `cbr-cidr`. Kontainer dapat menjangkau satu sama lain dan Node di atas jembatan` cbr0`. IP-IP tersebut semuanya dapat dirutekan dalam jaringan proyek GCE. + +GCE sendiri tidak tahu apa-apa tentang IP ini, jadi tidak akan NAT untuk lalu lintas internet keluar. Untuk mencapai itu aturan iptables digunakan untuk menyamar (alias SNAT - untuk membuatnya seolah-olah paket berasal dari lalu lintas `Node` itu sendiri) yang terikat untuk IP di luar jaringan proyek GCE (10.0.0.0/8). + +```shell +iptables -t nat -A POSTROUTING ! -d 10.0.0.0/8 -o eth0 -j MASQUERADE +``` + +Terakhir IP forwarding diaktifkan di kernel (sehingga kernel akan memproses paket untuk kontainer yang dijembatani): + +```shell +sysctl net.ipv4.ip_forward=1 +``` + +Hasil dari semua ini adalah bahwa semua Pod dapat saling menjangkau dan dapat keluar lalu lintas ke internet. + +### Jaguar + +[Jaguar](https://gitlab.com/sdnlab/jaguar) adalah solusi open source untuk jaringan Kubernetes berdasarkan OpenDaylight. Jaguar menyediakan jaringan overlay menggunakan vxlan dan Jaguar CNIPlugin menyediakan satu alamat IP per Pod. + +### Knitter + +[Knitter](https://github.com/ZTE/Knitter/) adalah solusi jaringan yang mendukung banyak jaringan di Kubernetes. Solusi ini menyediakan kemampuan manajemen penyewa dan manajemen jaringan. Knitter mencakup satu set solusi jaringan kontainer NFV ujung ke ujung selain beberapa pesawat jaringan, seperti menjaga alamat IP untuk aplikasi, migrasi alamat IP, dll. + +### Kube-OVN + +[Kube-OVN](https://github.com/alauda/kube-ovn) adalah _fabric_ jaringan kubernetes berbasis OVN untuk _enterprises_. Dengan bantuan OVN/OVS, solusi ini menyediakan beberapa fitur jaringan _overlay_ canggih seperti _subnet_, QoS, alokasi IP statis, _mirroring traffic_, _gateway_, kebijakan jaringan berbasis _openflow_, dan proksi layanan. + +### Kube-router + +[Kube-router](https://github.com/cloudnativelabs/kube-router) adalah solusi jaringan yang dibuat khusus untuk Kubernetes yang bertujuan untuk memberikan kinerja tinggi dan kesederhanaan operasional. Kube-router menyediakan Linux [LVS/IPVS](http://www.linuxvirtualserver.org/software/ipvs.html) berbasis proksi layanan, solusi jaringan berbasis penerusan _pod-to-pod_ Linux _kernel_ tanpa _overlay_, dan penegak kebijakan jaringan berbasis _iptables/ipset_. + +### L2 networks and linux bridging + +Jika Anda memiliki jaringan L2 yang "bodoh", seperti saklar sederhana di _environment_ "bare-metal", kamu harus dapat melakukan sesuatu yang mirip dengan pengaturan GCE di atas. Perhatikan bahwa petunjuk ini hanya dicoba dengan sangat sederhana - sepertinya berhasil, tetapi belum diuji secara menyeluruh. Jika kamu menggunakan teknik ini dan telah menyempurnakan prosesnya, tolong beri tahu kami. + +Ikuti bagian "With Linux Bridge devices" dari [tutorial yang sangat bagus ini](http://blog.oddbit.com/2014/08/11/four-ways-to-connect-a-docker/) dari Lars Kellogg-Stedman. + +### Multus (plugin Multi-Jaringan) + +[Multus](https://github.com/Intel-Corp/multus-cni) adalah plugin Multi CNI untuk mendukung fitur Banyak Jaringan di Kubernetes menggunakan objek jaringan berbasis CRD di Kubernetes. + +Multus mendukung semua [plugin referensi](https://github.com/containernetworking/plugins) (mis. [Flannel](https://github.com/containernetworking/plugins/tree/master/plugins/meta/flannel), [DHCP](https://github.com/containernetworking/plugins/tree/master/plugins/ipam/dhcp), [Macvlan](https://github.com/containernetworking/plugins/tree/master/plugins/main / macvlan)) yang mengimplementasikan spesifikasi CNI dan plugin pihak ke-3 (mis. [Calico](https://github.com/projectcalico/cni-plugin), [Weave](https://github.com/weaveworks/weave ), [Cilium](https://github.com/cilium/cilium), [Contiv](https://github.com/contiv/netplugin)). Selain itu, Multus mendukung [SRIOV](https://github.com/hustcat/sriov-cni), [DPDK](https://github.com/Intel-Corp/sriov-cni), [OVS- DPDK & VPP](https://github.com/intel/vhost-user-net-plugin) beban kerja di Kubernetes dengan aplikasi cloud asli dan aplikasi berbasis NFV di Kubernetes. + +### NSX-T + +[VMware NSX-T](https://docs.vmware.com/en/VMware-NSX-T/index.html) adalah virtualisasi jaringan dan platform keamanan. NSX-T dapat menyediakan virtualisasi jaringan untuk lingkungan multi-cloud dan multi-hypervisor dan berfokus pada kerangka kerja dan arsitektur aplikasi yang muncul yang memiliki titik akhir dan tumpukan teknologi yang heterogen. Selain hypervisor vSphere, lingkungan ini termasuk hypervisor lainnya seperti KVM, wadah, dan bare metal. + +[NSX-T Container Plug-in (NCP)](https://docs.vmware.com/en/VMware-NSX-T/2.0/nsxt_20_ncp_kubernetes.pdf) menyediakan integrasi antara NSX-T dan pembuat wadah seperti Kubernetes, serta integrasi antara NSX-T dan platform CaaS / PaaS berbasis-kontainer seperti Pivotal Container Service (PKS) dan OpenShift. + +### Nuage Networks VCS (Layanan Cloud Virtual) + +[Nuage](http://www.nuagenetworks.net) menyediakan platform SDN (Software-Defined Networking) berbasis kebijakan yang sangat skalabel. Nuage menggunakan Open vSwitch _open source_ untuk data _plane_ bersama dengan SDN Controller yang kaya fitur yang dibangun pada standar terbuka. + +Platform Nuage menggunakan _overlay_ untuk menyediakan jaringan berbasis kebijakan yang mulus antara Kubernetes Pod-Pod dan lingkungan non-Kubernetes (VM dan server _bare metal_). Model abstraksi kebijakan Nuage dirancang dengan mempertimbangkan aplikasi dan membuatnya mudah untuk mendeklarasikan kebijakan berbutir halus untuk aplikasi. Mesin analisis _real-time_ platform memungkinkan pemantauan visibilitas dan keamanan untuk aplikasi Kubernetes. + +### OpenVSwitch + +[OpenVSwitch](https://www.openvswitch.org/) adalah cara yang agak lebih dewasa tetapi juga rumit untuk membangun jaringan _overlay_. Ini didukung oleh beberapa "Toko Besar" untuk jaringan. + +### OVN (Open Virtual Networking) + +OVN adalah solusi virtualisasi jaringan opensource yang dikembangkan oleh komunitas Open vSwitch. Ini memungkinkan seseorang membuat switch logis, router logis, ACL stateful, load-balancers dll untuk membangun berbagai topologi jaringan virtual. Proyek ini memiliki plugin dan dokumentasi Kubernetes spesifik di [ovn-kubernetes](https://github.com/openvswitch/ovn-kubernetes). + +### Project Calico + +[Project Calico](http://docs.projectcalico.org/) adalah penyedia jaringan wadah sumber terbuka dan mesin kebijakan jaringan. + +Calico menyediakan solusi jaringan dan kebijakan kebijakan jaringan yang sangat berskala untuk menghubungkan Pod Kubernetes berdasarkan prinsip jaringan IP yang sama dengan internet, untuk Linux (open source) dan Windows (milik - tersedia dari [Tigera](https://www.tigera.io/essentials/)). Calico dapat digunakan tanpa enkapsulasi atau _overlay_ untuk menyediakan jaringan pusat data skala tinggi yang berkinerja tinggi. Calico juga menyediakan kebijakan keamanan jaringan berbutir halus, berdasarkan niat untuk Pod Kubernetes melalui _firewall_ terdistribusi. + +Calico juga dapat dijalankan dalam mode penegakan kebijakan bersama dengan solusi jaringan lain seperti Flannel, alias [kanal](https://github.com/tigera/canal), atau jaringan GCE, AWS atau Azure asli. + +### Romana + +[Romana](http://romana.io) adalah jaringan sumber terbuka dan solusi otomasi keamanan yang memungkinkan kamu menggunakan Kubernetes tanpa jaringan hamparan. Romana mendukung Kubernetes [Kebijakan Jaringan](/docs/concepts/services-networking/network-policies/) untuk memberikan isolasi di seluruh ruang nama jaringan. + +### Weave Net dari Weaveworks + +[Weave Net](https://www.weave.works/products/weave-net/) adalah jaringan yang tangguh dan mudah digunakan untuk Kubernetes dan aplikasi yang dihostingnya. Weave Net berjalan sebagai [plug-in CNI](https://www.weave.works/docs/net/latest/cni-plugin/) atau berdiri sendiri. Di kedua versi, itu tidak memerlukan konfigurasi atau kode tambahan untuk dijalankan, dan dalam kedua kasus, jaringan menyediakan satu alamat IP per Pod - seperti standar untuk Kubernetes. + +{{% /capture %}} + +{{% capture whatsnext %}} + +Desain awal model jaringan dan alasannya, dan beberapa rencana masa depan dijelaskan secara lebih rinci dalam [dokumen desain jaringan](https://git.k8s.io/community/contributors/design-proposals/network/networking.md). + +{{% /capture %}} From d1233d35317d009eeda2af1edd3a8855b21b5749 Mon Sep 17 00:00:00 2001 From: chentanjun <2799194073@qq.com> Date: Sun, 22 Mar 2020 20:04:44 +0800 Subject: [PATCH 138/140] sync en zh yaml in content/zh/example/admin direcroty (#19766) --- .../zh/examples/admin/cloud/ccm-example.yaml | 138 +++++++++--------- .../admin/dns/dns-horizontal-autoscaler.yaml | 66 ++++----- content/zh/examples/admin/dns/dnsutils.yaml | 14 ++ .../admin/resource/cpu-constraints-pod-3.yaml | 4 +- .../admin/resource/quota-objects-pvc-2.yaml | 2 +- .../admin/resource/quota-objects-pvc.yaml | 2 +- .../zh/examples/admin/sched/my-scheduler.yaml | 134 ++++++++--------- 7 files changed, 187 insertions(+), 173 deletions(-) create mode 100644 content/zh/examples/admin/dns/dnsutils.yaml diff --git a/content/zh/examples/admin/cloud/ccm-example.yaml b/content/zh/examples/admin/cloud/ccm-example.yaml index 4c98162a70..27386be675 100644 --- a/content/zh/examples/admin/cloud/ccm-example.yaml +++ b/content/zh/examples/admin/cloud/ccm-example.yaml @@ -1,69 +1,69 @@ -# This is an example of how to setup cloud-controller-manger as a Daemonset in your cluster. -# It assumes that your masters can run pods and has the role node-role.kubernetes.io/master -# Note that this Daemonset will not work straight out of the box for your cloud, this is -# meant to be a guideline. - ---- -apiVersion: v1 -kind: ServiceAccount -metadata: - name: cloud-controller-manager - namespace: kube-system ---- -kind: ClusterRoleBinding -apiVersion: rbac.authorization.k8s.io/v1 -metadata: - name: system:cloud-controller-manager -roleRef: - apiGroup: rbac.authorization.k8s.io - kind: ClusterRole - name: cluster-admin -subjects: -- kind: ServiceAccount - name: cloud-controller-manager - namespace: kube-system ---- -apiVersion: apps/v1 -kind: DaemonSet -metadata: - labels: - k8s-app: cloud-controller-manager - name: cloud-controller-manager - namespace: kube-system -spec: - selector: - matchLabels: - k8s-app: cloud-controller-manager - template: - metadata: - labels: - k8s-app: cloud-controller-manager - spec: - serviceAccountName: cloud-controller-manager - containers: - - name: cloud-controller-manager - # for in-tree providers we use k8s.gcr.io/cloud-controller-manager - # this can be replaced with any other image for out-of-tree providers - image: k8s.gcr.io/cloud-controller-manager:v1.8.0 - command: - - /usr/local/bin/cloud-controller-manager - - --cloud-provider= # Add your own cloud provider here! - - --leader-elect=true - - --use-service-account-credentials - # these flags will vary for every cloud provider - - --allocate-node-cidrs=true - - --configure-cloud-routes=true - - --cluster-cidr=172.17.0.0/16 - tolerations: - # this is required so CCM can bootstrap itself - - key: node.cloudprovider.kubernetes.io/uninitialized - value: "true" - effect: NoSchedule - # this is to have the daemonset runnable on master nodes - # the taint may vary depending on your cluster setup - - key: node-role.kubernetes.io/master - effect: NoSchedule - # this is to restrict CCM to only run on master nodes - # the node selector may vary depending on your cluster setup - nodeSelector: - node-role.kubernetes.io/master: "" +# This is an example of how to setup cloud-controller-manger as a Daemonset in your cluster. +# It assumes that your masters can run pods and has the role node-role.kubernetes.io/master +# Note that this Daemonset will not work straight out of the box for your cloud, this is +# meant to be a guideline. + +--- +apiVersion: v1 +kind: ServiceAccount +metadata: + name: cloud-controller-manager + namespace: kube-system +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: system:cloud-controller-manager +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: cluster-admin +subjects: +- kind: ServiceAccount + name: cloud-controller-manager + namespace: kube-system +--- +apiVersion: apps/v1 +kind: DaemonSet +metadata: + labels: + k8s-app: cloud-controller-manager + name: cloud-controller-manager + namespace: kube-system +spec: + selector: + matchLabels: + k8s-app: cloud-controller-manager + template: + metadata: + labels: + k8s-app: cloud-controller-manager + spec: + serviceAccountName: cloud-controller-manager + containers: + - name: cloud-controller-manager + # for in-tree providers we use k8s.gcr.io/cloud-controller-manager + # this can be replaced with any other image for out-of-tree providers + image: k8s.gcr.io/cloud-controller-manager:v1.8.0 + command: + - /usr/local/bin/cloud-controller-manager + - --cloud-provider=[YOUR_CLOUD_PROVIDER] # Add your own cloud provider here! + - --leader-elect=true + - --use-service-account-credentials + # these flags will vary for every cloud provider + - --allocate-node-cidrs=true + - --configure-cloud-routes=true + - --cluster-cidr=172.17.0.0/16 + tolerations: + # this is required so CCM can bootstrap itself + - key: node.cloudprovider.kubernetes.io/uninitialized + value: "true" + effect: NoSchedule + # this is to have the daemonset runnable on master nodes + # the taint may vary depending on your cluster setup + - key: node-role.kubernetes.io/master + effect: NoSchedule + # this is to restrict CCM to only run on master nodes + # the node selector may vary depending on your cluster setup + nodeSelector: + node-role.kubernetes.io/master: "" diff --git a/content/zh/examples/admin/dns/dns-horizontal-autoscaler.yaml b/content/zh/examples/admin/dns/dns-horizontal-autoscaler.yaml index 5e6d55a6b2..3676a0acdb 100644 --- a/content/zh/examples/admin/dns/dns-horizontal-autoscaler.yaml +++ b/content/zh/examples/admin/dns/dns-horizontal-autoscaler.yaml @@ -1,33 +1,33 @@ -apiVersion: apps/v1 -kind: Deployment -metadata: - name: dns-autoscaler - namespace: kube-system - labels: - k8s-app: dns-autoscaler -spec: - selector: - matchLabels: - k8s-app: dns-autoscaler - template: - metadata: - labels: - k8s-app: dns-autoscaler - spec: - containers: - - name: autoscaler - image: k8s.gcr.io/cluster-proportional-autoscaler-amd64:1.1.1 - resources: - requests: - cpu: "20m" - memory: "10Mi" - command: - - /cluster-proportional-autoscaler - - --namespace=kube-system - - --configmap=dns-autoscaler - - --target= - # When cluster is using large nodes(with more cores), "coresPerReplica" should dominate. - # If using small nodes, "nodesPerReplica" should dominate. - - --default-params={"linear":{"coresPerReplica":256,"nodesPerReplica":16,"min":1}} - - --logtostderr=true - - --v=2 +apiVersion: apps/v1 +kind: Deployment +metadata: + name: dns-autoscaler + namespace: kube-system + labels: + k8s-app: dns-autoscaler +spec: + selector: + matchLabels: + k8s-app: dns-autoscaler + template: + metadata: + labels: + k8s-app: dns-autoscaler + spec: + containers: + - name: autoscaler + image: k8s.gcr.io/cluster-proportional-autoscaler-amd64:1.6.0 + resources: + requests: + cpu: 20m + memory: 10Mi + command: + - /cluster-proportional-autoscaler + - --namespace=kube-system + - --configmap=dns-autoscaler + - --target= + # When cluster is using large nodes(with more cores), "coresPerReplica" should dominate. + # If using small nodes, "nodesPerReplica" should dominate. + - --default-params={"linear":{"coresPerReplica":256,"nodesPerReplica":16,"min":1}} + - --logtostderr=true + - --v=2 diff --git a/content/zh/examples/admin/dns/dnsutils.yaml b/content/zh/examples/admin/dns/dnsutils.yaml new file mode 100644 index 0000000000..5eb9833f39 --- /dev/null +++ b/content/zh/examples/admin/dns/dnsutils.yaml @@ -0,0 +1,14 @@ +apiVersion: v1 +kind: Pod +metadata: + name: dnsutils + namespace: default +spec: + containers: + - name: dnsutils + image: gcr.io/kubernetes-e2e-test-images/dnsutils:1.3 + command: + - sleep + - "3600" + imagePullPolicy: IfNotPresent + restartPolicy: Always diff --git a/content/zh/examples/admin/resource/cpu-constraints-pod-3.yaml b/content/zh/examples/admin/resource/cpu-constraints-pod-3.yaml index 896d98ec2f..0a2083acd8 100644 --- a/content/zh/examples/admin/resource/cpu-constraints-pod-3.yaml +++ b/content/zh/examples/admin/resource/cpu-constraints-pod-3.yaml @@ -1,10 +1,10 @@ apiVersion: v1 kind: Pod metadata: - name: constraints-cpu-demo-4 + name: constraints-cpu-demo-3 spec: containers: - - name: constraints-cpu-demo-4-ctr + - name: constraints-cpu-demo-3-ctr image: nginx resources: limits: diff --git a/content/zh/examples/admin/resource/quota-objects-pvc-2.yaml b/content/zh/examples/admin/resource/quota-objects-pvc-2.yaml index 88c165d144..2539c2d309 100644 --- a/content/zh/examples/admin/resource/quota-objects-pvc-2.yaml +++ b/content/zh/examples/admin/resource/quota-objects-pvc-2.yaml @@ -1,5 +1,5 @@ -kind: PersistentVolumeClaim apiVersion: v1 +kind: PersistentVolumeClaim metadata: name: pvc-quota-demo-2 spec: diff --git a/content/zh/examples/admin/resource/quota-objects-pvc.yaml b/content/zh/examples/admin/resource/quota-objects-pvc.yaml index b38256b897..728bb4d708 100644 --- a/content/zh/examples/admin/resource/quota-objects-pvc.yaml +++ b/content/zh/examples/admin/resource/quota-objects-pvc.yaml @@ -1,5 +1,5 @@ -kind: PersistentVolumeClaim apiVersion: v1 +kind: PersistentVolumeClaim metadata: name: pvc-quota-demo spec: diff --git a/content/zh/examples/admin/sched/my-scheduler.yaml b/content/zh/examples/admin/sched/my-scheduler.yaml index ab0c385cd6..4903c6f54c 100644 --- a/content/zh/examples/admin/sched/my-scheduler.yaml +++ b/content/zh/examples/admin/sched/my-scheduler.yaml @@ -1,67 +1,67 @@ -apiVersion: v1 -kind: ServiceAccount -metadata: - name: my-scheduler - namespace: kube-system ---- -kind: ClusterRoleBinding -apiVersion: rbac.authorization.k8s.io/v1 -metadata: - name: my-scheduler-as-kube-scheduler -subjects: -- kind: ServiceAccount - name: my-scheduler - namespace: kube-system -roleRef: - kind: ClusterRole - name: kube-scheduler - apiGroup: rbac.authorization.k8s.io ---- -apiVersion: apps/v1 -kind: Deployment -metadata: - labels: - component: scheduler - tier: control-plane - name: my-scheduler - namespace: kube-system -spec: - selector: - matchLabels: - component: scheduler - tier: control-plane - replicas: 1 - template: - metadata: - labels: - component: scheduler - tier: control-plane - version: second - spec: - serviceAccountName: my-scheduler - containers: - - command: - - /usr/local/bin/kube-scheduler - - --address=0.0.0.0 - - --leader-elect=false - - --scheduler-name=my-scheduler - image: gcr.io/my-gcp-project/my-kube-scheduler:1.0 - livenessProbe: - httpGet: - path: /healthz - port: 10251 - initialDelaySeconds: 15 - name: kube-second-scheduler - readinessProbe: - httpGet: - path: /healthz - port: 10251 - resources: - requests: - cpu: '0.1' - securityContext: - privileged: false - volumeMounts: [] - hostNetwork: false - hostPID: false - volumes: [] +apiVersion: v1 +kind: ServiceAccount +metadata: + name: my-scheduler + namespace: kube-system +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: my-scheduler-as-kube-scheduler +subjects: +- kind: ServiceAccount + name: my-scheduler + namespace: kube-system +roleRef: + kind: ClusterRole + name: system:kube-scheduler + apiGroup: rbac.authorization.k8s.io +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + labels: + component: scheduler + tier: control-plane + name: my-scheduler + namespace: kube-system +spec: + selector: + matchLabels: + component: scheduler + tier: control-plane + replicas: 1 + template: + metadata: + labels: + component: scheduler + tier: control-plane + version: second + spec: + serviceAccountName: my-scheduler + containers: + - command: + - /usr/local/bin/kube-scheduler + - --address=0.0.0.0 + - --leader-elect=false + - --scheduler-name=my-scheduler + image: gcr.io/my-gcp-project/my-kube-scheduler:1.0 + livenessProbe: + httpGet: + path: /healthz + port: 10251 + initialDelaySeconds: 15 + name: kube-second-scheduler + readinessProbe: + httpGet: + path: /healthz + port: 10251 + resources: + requests: + cpu: '0.1' + securityContext: + privileged: false + volumeMounts: [] + hostNetwork: false + hostPID: false + volumes: [] From 870128c48f3e99f75daad33bd8556a63ebe5a79d Mon Sep 17 00:00:00 2001 From: 2BFL Date: Sun, 22 Mar 2020 20:06:44 +0800 Subject: [PATCH 139/140] zh-translation: upate hello-minikube.md (#19660) Fix missing items --- content/zh/docs/tutorials/hello-minikube.md | 394 ++++++++---------- .../tutorials/kubernetes-basics/_index.html | 5 + 2 files changed, 169 insertions(+), 230 deletions(-) diff --git a/content/zh/docs/tutorials/hello-minikube.md b/content/zh/docs/tutorials/hello-minikube.md index a8587d6967..c10be6b6f9 100644 --- a/content/zh/docs/tutorials/hello-minikube.md +++ b/content/zh/docs/tutorials/hello-minikube.md @@ -30,12 +30,13 @@ card: --> {{% capture overview %}} + -本教程向您展示如何使用 [Minikube](/docs/getting-started-guides/minikube) 和 Katacoda 在 Kubernetes 上运行一个简单的 “Hello World” Node.js 应用程序。Katacoda 提供免费的浏览器内 Kubernetes 环境。 +本教程向您展示如何使用 [Minikube](/docs/setup/learning-environment/minikube) 和 Katacoda 在 Kubernetes 上运行一个简单的 “Hello World” Node.js 应用程序。Katacoda 提供免费的浏览器内 Kubernetes 环境。 {{< note >}} @@ -68,6 +71,7 @@ This tutorial provides a container image built from the following files: {{< codenew language="js" file="minikube/server.js" >}} {{< codenew language="conf" file="minikube/Dockerfile" >}} + @@ -81,39 +85,36 @@ For more information on the `docker build` command, read the [Docker documentati ## Create a Minikube cluster 1. Click **Launch Terminal** +--> +## 创建 Minikube 集群 + +1. 点击 **启动终端** {{< kat-button >}} {{< note >}}If you installed Minikube locally, run `minikube start`.{{< /note >}} + -## 创建 Minikube 集群 - -1. 点击 **启动终端** - {{< kat-button >}} - - {{< note >}}如果您本地安装了 Minikube, 运行 `minikube start`.{{< /note >}} - 2. 在浏览器中打开 Kubernetes dashboard： ```shell minikube dashboard ``` + + 3. 仅限 Katacoda 环境：在终端窗口的顶部，单击加号，然后单击 **选择要在主机 1 上查看的端口**。 4. 仅限 Katacoda 环境：输入“30000”，然后单击 **显示端口**。 + ## 创建 Deployment Kubernetes [*Pod*](/docs/concepts/workloads/pods/pod/) 是由一个或多个为了管理和联网而绑定在一起的容器构成的组。本教程中的 Pod 只有一个容器。Kubernetes [*Deployment*](/docs/concepts/workloads/controllers/deployment/) 检查 Pod 的健康状况，并在 Pod 中的容器终止的情况下重新启动新的容器。Deployment 是管理 Pod 创建和扩展的推荐方法。 + + 1. 使用 `kubectl create` 命令创建管理 Pod 的 Deployment。该 Pod 根据提供的 Docker 镜像运行 Container。 ```shell kubectl create deployment hello-node --image=gcr.io/hello-minikube-zero-install/hello-node ``` + + 2. 查看 Deployment： ```shell kubectl get deployments ``` - 输出: + + + 输出结果类似于这样： - ```shell - NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE - hello-node 1 1 1 1 1m ``` + NAME READY UP-TO-DATE AVAILABLE AGE + hello-node 1/1 1 1 1m + ``` + + 3. 查看 Pod： ```shell kubectl get pods ``` - 输出: - ```shell + + + 输出结果类似于这样： + + ``` NAME READY STATUS RESTARTS AGE hello-node-5f76cf6ccf-br9b5 1/1 Running 0 1m ``` + + 4. 查看集群事件： ```shell kubectl get events ``` + + 5. 查看 `kubectl` 配置： ```shell kubectl config view ``` + + {{< note >}}有关 kubectl 命令的更多信息，请参阅 [kubectl 概述](/docs/user-guide/kubectl-overview/)。{{< /note >}} + ## 创建 Service 默认情况下，Pod 只能通过 Kubernetes 集群中的内部 IP 地址访问。要使得 `hello-node` 容器可以从 Kubernetes 虚拟网络的外部访问，您必须将 Pod 暴露为 Kubernetes [*Service*](/docs/concepts/services-networking/service/)。 + + 1. 使用 `kubectl expose` 命令将 Pod 暴露给公网： ```shell @@ -278,117 +233,69 @@ Kubernetes [*Service*](/docs/concepts/services-networking/service/). The `--type=LoadBalancer` flag indicates that you want to expose your Service outside of the cluster. + + 2. 查看您刚刚创建的服务: ```shell kubectl get services ``` - 输出: + - ```shell + 输出结果类似于这样: + + ``` NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE hello-node LoadBalancer 10.108.144.78 8080:30369/TCP 21s kubernetes ClusterIP 10.96.0.1 443/TCP 23m ``` + 在支持负载均衡器的云服务提供商上，将提供一个外部 IP 来访问该服务。在 Minikube 上，`LoadBalancer` 使得服务可以通过命令 `minikube service` 访问。 + 3. 运行下面的命令： ```shell minikube service hello-node ``` + 4. 仅限 Katacoda 环境：单击加号，然后单击 **选择要在主机 1 上查看的端口**。 -5. 仅限 Katacoda 环境：输入 `30369`（请参阅服务输出中与 `8080` 相对的端口），然后单击 + +5. 仅限 Katacoda 环境：请注意在 service 输出中与 `8080` 对应的长度为 5 位的端口号。此端口号是随机生成的，可能与您不同。在端口号文本框中输入您自己的端口号，然后单击显示端口。如果是上面那个例子，就需要输入 `30369`。 这将打开一个浏览器窗口，为您的应用程序提供服务并显示 “Hello World” 消息。 ## 启用插件 -Minikube 有一组内置的插件，可以在本地 Kubernetes 环境中启用、禁用和打开。 +Minikube 有一组内置的 {{< glossary_tooltip text="插件" term_id="addons" >}}，可以在本地 Kubernetes 环境中启用、禁用和打开。 1. 列出当前支持的插件： @@ -396,37 +303,55 @@ Minikube 有一组内置的插件，可以在本地 Kubernetes 环境中启用 minikube addons list ``` - 输出: + - ```shell + 输出结果类似于这样： + + ``` addon-manager: enabled - coredns: disabled dashboard: enabled default-storageclass: enabled efk: disabled freshpod: disabled - heapster: disabled + gvisor: disabled + helm-tiller: disabled ingress: disabled - kube-dns: enabled + ingress-dns: disabled + logviewer: disabled metrics-server: disabled nvidia-driver-installer: disabled nvidia-gpu-device-plugin: disabled registry: disabled registry-creds: disabled storage-provisioner: enabled + storage-provisioner-gluster: disabled ``` -2. 启用插件，例如 `heapster`： + + +2. 启用插件，例如 `metrics-server`： ```shell - minikube addons enable heapster + minikube addons enable metrics-server ``` - 输出: + + + 输出结果类似于这样： - ```shell - heapster was successfully enabled ``` + metrics-server was successfully enabled + ``` + + 3. 查看刚才创建的 Pod 和 Service： @@ -434,59 +359,60 @@ Minikube 有一组内置的插件，可以在本地 Kubernetes 环境中启用 kubectl get pod,svc -n kube-system ``` - 输出: + - ```shell + 输出结果类似于这样： + + ``` NAME READY STATUS RESTARTS AGE - pod/heapster-9jttx 1/1 Running 0 26s + pod/coredns-5644d7b6d9-mh9ll 1/1 Running 0 34m + pod/coredns-5644d7b6d9-pqd2t 1/1 Running 0 34m + pod/metrics-server-67fb648c5 1/1 Running 0 26s + pod/etcd-minikube 1/1 Running 0 34m pod/influxdb-grafana-b29w8 2/2 Running 0 26s pod/kube-addon-manager-minikube 1/1 Running 0 34m - pod/kube-dns-6dcb57bcc8-gv7mw 3/3 Running 0 34m - pod/kubernetes-dashboard-5498ccf677-cgspw 1/1 Running 0 34m + pod/kube-apiserver-minikube 1/1 Running 0 34m + pod/kube-controller-manager-minikube 1/1 Running 0 34m + pod/kube-proxy-rnlps 1/1 Running 0 34m + pod/kube-scheduler-minikube 1/1 Running 0 34m pod/storage-provisioner 1/1 Running 0 34m NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE - service/heapster ClusterIP 10.96.241.45 80/TCP 26s + service/metrics-server ClusterIP 10.96.241.45 80/TCP 26s service/kube-dns ClusterIP 10.96.0.10 53/UDP,53/TCP 34m - service/kubernetes-dashboard NodePort 10.109.29.1 80:30000/TCP 34m service/monitoring-grafana NodePort 10.99.24.54 80:30002/TCP 26s service/monitoring-influxdb ClusterIP 10.111.169.94 8083/TCP,8086/TCP 26s ``` -4. 禁用 `heapster`： + + +4. 禁用 `metrics-server`： + ```shell - minikube addons disable heapster + minikube addons disable metrics-server ``` - 输出: + - ```shell - heapster was successfully disabled + 输出结果类似于这样： + + ``` + metrics-server was successfully disabled ``` + ## 清理 现在可以清理您在集群中创建的资源： @@ -496,13 +422,21 @@ kubectl delete service hello-node kubectl delete deployment hello-node ``` -可以停止 Minikube VM： + + +可选的，停止 Minikube 虚拟机（VM）： ```shell minikube stop ``` -或者，删除 Minikube VM： + + +可选的，删除 Minikube 虚拟机（VM）： ```shell minikube delete @@ -514,11 +448,11 @@ minikube delete * 进一步了解 [Deployment 对象](/docs/concepts/workloads/controllers/deployment/)。 -* 学习更多关于 [部署应用](/docs/user-guide/deploying-applications/)。 +* 学习更多关于 [部署应用](/docs/tasks/run-application/run-stateless-application-deployment/)。 * 学习更多关于 [Service 对象](/docs/concepts/services-networking/service/)。 {{% /capture %}} diff --git a/content/zh/docs/tutorials/kubernetes-basics/_index.html b/content/zh/docs/tutorials/kubernetes-basics/_index.html index ecfbbd626a..39b52b2ce1 100644 --- a/content/zh/docs/tutorials/kubernetes-basics/_index.html +++ b/content/zh/docs/tutorials/kubernetes-basics/_index.html @@ -1,6 +1,11 @@ --- title: 学习 Kubernetes 基础知识 linkTitle: 学习 Kubernetes 基础知识 +weight: 10 +card: + name: tutorials + weight: 20 + title: 基础知识介绍 --- From 3cd0b293017cc2bfc1dac8ea5ccfe4ae65695f33 Mon Sep 17 00:00:00 2001 From: Rajesh Deshpande Date: Mon, 23 Mar 2020 00:38:45 +0530 Subject: [PATCH 140/140] Add new reviewer for sig-docs-en-reviews (#19661) * Adding reviewer for sig-docs-en-reviews Adding reviewer for sig-docs-en-reviews * Adding new entry following alphabetical sort order Adding new entry following alphabetical sort order --- OWNERS_ALIASES | 1 + 1 file changed, 1 insertion(+) diff --git a/OWNERS_ALIASES b/OWNERS_ALIASES index 43f27a88b2..9bc9a38f16 100644 --- a/OWNERS_ALIASES +++ b/OWNERS_ALIASES @@ -64,6 +64,7 @@ aliases: - makoscafee - onlydole - rajakavitha1 + - rajeshdeshpande02 - sftim - steveperry-53 - tengqm