Merge pull request #24761 from kasramp/dev-1.18-fa.1

Add hello minikube Persian translation
2020-10-28 09:43:57 -07:00 · 2020-10-27 08:42:27 +01:00 · 2020-09-16 22:36:34 +01:00 · 2020-09-09 11:59:08 -07:00 · 2020-09-09 10:23:08 -07:00 · 2020-09-09 10:58:21 -04:00
349 changed files with 23661 additions and 14177 deletions
@@ -35,3 +35,5 @@ Note that code issues should be filed against the main kubernetes repository, wh
 ### Submitting Documentation Pull Requests

 If you're fixing an issue in the existing documentation, you should submit a PR against the master branch.  Follow [these instructions to create a documentation pull request against the kubernetes.io repository](http://kubernetes.io/docs/home/contribute/create-pull-request/).
+
+For more information, see [contributing to Kubernetes docs](https://kubernetes.io/docs/contribute/).
@@ -27,7 +27,6 @@ aliases:
    - jimangel
    - kbarnard10
    - kbhawkey
-    - makoscafee
    - onlydole
    - savitharaghunathan
    - sftim
@@ -43,7 +42,6 @@ aliases:
    - jimangel
    - kbarnard10
    - kbhawkey
-    - makoscafee
    - onlydole
    - rajeshdeshpande02
    - sftim
@@ -60,6 +58,14 @@ aliases:
    - alexbrand
    # glo-pena
    - electrocucaracha
+  sig-docs-fa-owners: # Admins for Persian content
+    - FaezeAzimi
+    - kasramp
+    - sattarfeizollahibarough
+  sig-docs-fa-reviews: # PR reviews for Persian content
+    - FaezeAzimi
+    - kasramp
+    - sattarfeizollahibarough
  sig-docs-fr-owners: # Admins for French content
    - remyleone
    - perriea
@@ -40,13 +40,13 @@ Um die Kubernetes-Website lokal laufen zu lassen, empfiehlt es sich, ein speziel
 Wenn Sie Docker [installiert](https://www.docker.com/get-started) haben, erstellen Sie das Docker-Image `kubernetes-hugo` lokal:

 ```bash
-make docker-image
+make container-image
 ```

 Nachdem das Image erstellt wurde, können Sie die Site lokal ausführen:

 ```bash
-make docker-serve
+make container-serve
 ```

 Öffnen Sie Ihren Browser unter http://localhost:1313, um die Site anzuzeigen. Wenn Sie Änderungen an den Quelldateien vornehmen, aktualisiert Hugo die Site und erzwingt eine Browseraktualisierung.
@@ -33,13 +33,13 @@ El método recomendado para levantar una copia local del sitio web kubernetes.io
 Una vez tenga Docker [configurado en su máquina](https://www.docker.com/get-started), puede construir la imagen de Docker `kubernetes-hugo` localmente ejecutando el siguiente comando en la raíz del repositorio:

 ```bash
-make docker-image
+make container-image
 ```

 Una vez tenga la imagen construida, puede levantar el sitio web ejecutando:

 ```bash
-make docker-serve
+make container-serve
 ```

 Abra su navegador y visite http://localhost:1313 para acceder a su copia local del sitio. A medida que vaya haciendo cambios en el código fuente, Hugo irá actualizando la página y forzará la actualización en el navegador.
@@ -0,0 +1,92 @@
+<div dir="rtl">
+
+# مستندات کوبرنتیز
+
+[![Netlify Status](https://api.netlify.com/api/v1/badges/be93b718-a6df-402a-b4a4-855ba186c97d/deploy-status)](https://app.netlify.com/sites/kubernetes-io-master-staging/deploys) [![GitHub release](https://img.shields.io/github/release/kubernetes/website.svg)](https://github.com/kubernetes/website/releases/latest)
+
+این مخزن شامل مواردی است که برای ساخت [مستندات و وب سایت کوبرنتیز](https://kubernetes.io/) به آنها نیاز است. ما از این که شما قصد مشارکت دارید خوشحال هستیم.
+
+## اجرای وب سایت در سیستم خود با استفاده از هوگو
+
+برای دریافت مستندات نصب هوگو لطفاً به [سایت مستندات رسمی هوگو](https://gohugo.io/getting-started/installing/) وارد شوید. قبل از هر چیز مطمئن شوید که نسخه توسعه یافته هوگو را
+دریافت نموده‌اید. برای این کار باید متغییر محیطی `HUGO_VERSION` را در فایل [`netlify.toml`](netlify.toml#L10) بررسی نمایید.
+
+قبل از ساختن سایت، مخزن وب سایت کوبرنتیز را کلون نمایید:
+<div dir="ltr">
+
+```bash
+git clone https://github.com/kubernetes/website.git
+cd website
+git submodule update --init --recursive --depth 1
+```
+
+</div>
+
+**توجه:** وب سایت کوبرنتیز [تم هوگو داکی](https://github.com/google/docsy#readme) را برای استقرار استفاده می‌کند. 
+اگر مخزن وب سایت خود را بروزرسانی نکرده‌اید، مسیر `website/themes/docsy` خالی خواهد بود و وب سایت بدون یک کپی از آن تم ساخته نخواهد شد.
+
+برای بروزرسانی تم وب سایت از دستور زیر استفاده شود:
+<div dir="ltr">
+
+```bash
+git submodule update --init --recursive --depth 1
+```
+</div>
+
+برای ساخت و تست سایت در داخل سیستم خود می‌توانید از دستور زیر استفاده کنید:
+<div dir="ltr">
+
+```bash
+hugo server --buildFuture
+```
+</div>
+با اجرای دستور فوق سرور هوگو به صورت داخلی بر روی سیستم شما بر روی پورت 1313 شروع به کار خواهد کرد. برای دیدن وب سایت http://localhost:1313 در مرورگر خود باز کنید. با انجام هر تغییری در فایلهای سورس، هوگو مرورگر را وادار به بازخوانی صفحه می‌کند.
+
+## مشارکت در SIG مستندسازی
+
+شما می‌توانید با استفاده از لینک روبرو در مورد SIG مستندسازی کوبرنتیز و جلسات آنها اطلاعات بیشتری را بدست آورید. [صفحه جامعه کوبرنتیز](https://github.com/kubernetes/community/tree/master/sig-docs#meetings)
+
+همچنین شما می‌توانید با نگهدارنگان پروژه کوبرنتیز با استفاده از لینکهای زیر در تماس باشید:
+
+- [Slack](https://kubernetes.slack.com/messages/sig-docs)
+- [Mailing List](https://groups.google.com/forum/#!forum/kubernetes-sig-docs)
+
+## کمک به مستند سازی این پروژه
+
+با استفاده از دکمه **Fork** که در ناحیه بالا سمت راست قرار گرفته است می‌توانید یک کپی از این مخزن را در حساب Github خود کپی کنید. به این کپی اصطلاحاً *fork* گفته می شود. تغییراتی را که مد نظر دارید را
+در نسخه fork شده اعمال کنید و هنگامی که آماده بودید این تغییرات را برای ما ارسال کنید که این کار از طریق درخواست pull انجام می‌شود.
+
+زمانی که درخواست pull شما برای یکی از بازبینان کوبرنتیز ارسال شد، وی موظف به ایجاد یک بازخورد واضح و قابل اعمال است. به عنوان مالک درخواست کننده pull **وظیفه شماست که به اصلاح مواردی بپردازید که بازبین برای شما به عنوان بازخورد مشخص کرده است.**
+
+همچنین باید این نکته را توجه داشته باشید که ممکن است بیشتر از یک بازبین کوبرنتیز برای کار شما بازخورد ایجاد کند و یا اینکه حتی بازخوردی را دریافت کنید که با بازخورد ابتدایی متفاوت باشد.
+
+علاوه بر این، در برخی موارد ممکن است یکی از بازبینان در صورت لزوم از یک بازبین فنی بخواهد تا بازخوردی را برای شما انجام دهد. بازبینان تمام تلاش خود را می‌کند که بازخوردهای خود را در سریعترین زمان به شما پاسخ بدهند ولی زمانبندی این موضوع می‌تواند بنا به شرایط متفاوت باشد.
+
+برای اطلاعات بیشتر در خصوص مشارکت در مستندسازی کوبرنتیز به لینک‌های زیر مراجعه کنید:
+
+* [مشارکت در مستندسازی کوبرنتیز](https://kubernetes.io/docs/contribute/)
+* [انواع محتوای صفحات](https://kubernetes.io/docs/contribute/style/page-content-types/)
+* [راهنمای سبک مستندسازی](https://kubernetes.io/docs/contribute/style/style-guide/)
+* [مستند بومی سازی کوبنتیز](https://kubernetes.io/docs/contribute/localization/)
+
+## بومی‌سازی `README.md`'s
+
+| زبان  | زبان |
+|---|---|
+|[چینی](README-zh.md)|[کره‌ای](README-ko.md)|
+|[فرانسوی](README-fr.md)|[لهستانی](README-pl.md)|
+|[آلمانی](README-de.md)|[پرتغالی](README-pt.md)|
+|[هندی](README-hi.md)|[روسی](README-ru.md)|
+|[اندونزیایی](README-id.md)|[اسپانیایی](README-es.md)|
+|[ایتالیایی](README-it.md)|[اوکراینی](README-uk.md)|
+|[ژاپنی](README-ja.md)|[ویتنامی](README-vi.md)|
+|[فارسی](README-fa.md)|
+
+## مرام نامه
+
+مشارکت در جامعه کوبرنتیز توسط [CNCF مرام نامه](https://github.com/cncf/foundation/blob/master/code-of-conduct.md) انجام می شود.
+
+## سپاس
+
+کوبرنتیز با مشارکت جامعه شکوفا می شود و ما از کمک شما برای ارائه کمکهایتان به وب سایت و مستندسازی مستنداتمان سپاسگزار هستیم.
+</div>
@@ -38,13 +38,13 @@ La façon recommandée d'exécuter le site web Kubernetes localement est d'utili
 Si vous avez Docker [up and running](https://www.docker.com/get-started), construisez l'image Docker `kubernetes-hugo' localement:

 ```bash
-make docker-image
+make container-image
 ```

 Une fois l'image construite, vous pouvez exécuter le site localement :

 ```bash
-make docker-serve
+make container-serve
 ```

 Ouvrez votre navigateur à l'adresse: http://localhost:1313 pour voir le site.
@@ -41,13 +41,13 @@
 यदि आप [डॉकर](https://www.docker.com/get-started) चला रहे हैं, तो स्थानीय रूप से `कुबेरनेट्स-ह्यूगो` Docker image बनाएँ:

 ```bash
-make docker-image
+make container-image
 ```

 एक बार image बन जाने के बाद, आप साइट को स्थानीय रूप से चला सकते हैं:

 ```bash
-make docker-serve
+make container-serve
 ```

 साइट देखने के लिए अपने browser को `http://localhost:1313` पर खोलें। जैसा कि आप source फ़ाइलों में परिवर्तन करते हैं, Hugo साइट को अपडेट करता है और browser को refresh करने पर मजबूर करता है।
@@ -30,13 +30,13 @@ Petunjuk yang disarankan untuk menjalankan Dokumentasi Kubernetes pada mesin lok
 Jika kamu sudah memiliki **Docker** [yang sudah dapat digunakan](https://www.docker.com/get-started), kamu dapat melakukan **build** `kubernetes-hugo` **Docker image** secara lokal:

 ```bash
-make docker-image
+make container-image
 ```

 Setelah **image** berhasil di-**build**, kamu dapat menjalankan website tersebut pada mesin lokal-mu:

 ```bash
-make docker-serve
+make container-serve
 ```

 Buka **browser** kamu ke http://localhost:1313 untuk melihat laman dokumentasi. Selama kamu melakukan penambahan konten, **Hugo** akan secara otomatis melakukan perubahan terhadap laman dokumentasi apabila **browser** melakukan proses **refresh**.
@@ -30,13 +30,13 @@ Il modo consigliato per eseguire localmente il sito Web Kubernetes prevede l'uti
 Se hai Docker [attivo e funzionante](https://www.docker.com/get-started), crea l'immagine Docker `kubernetes-hugo` localmente:

 ```bash
-make docker-image
+make container-image
 ```

 Dopo aver creato l'immagine, è possibile eseguire il sito Web localmente:

 ```bash
-make docker-serve
+make container-serve
 ```

 Apri il tuo browser su http://localhost:1313 per visualizzare il sito Web. Mentre modifichi i file sorgenti, Hugo aggiorna automaticamente il sito Web e forza un aggiornamento della pagina visualizzata nel browser.
@@ -41,13 +41,13 @@
 도커 [동작 및 실행](https://www.docker.com/get-started) 환경이 있는 경우, 로컬에서 `kubernetes-hugo` 도커 이미지를 빌드 합니다:

 ```bash
-make docker-image
+make container-image
 ```

 해당 이미지가 빌드 된 이후, 사이트를 로컬에서 실행할 수 있습니다:

 ```bash
-make docker-serve
+make container-serve
 ```

 브라우저에서 http://localhost:1313 를 열어 사이트를 살펴봅니다. 소스 파일에 변경 사항이 있을 때, Hugo는 사이트를 업데이트하고 브라우저를 강제로 새로고침합니다.
@@ -49,13 +49,13 @@ choco install make
 Jeśli [zainstalowałeś i uruchomiłeś](https://www.docker.com/get-started) już Dockera, zbuduj obraz `kubernetes-hugo` lokalnie:

 ```bash
-make docker-image
+make container-image
 ```

 Po zbudowaniu obrazu, możesz uruchomić serwis lokalnie:

 ```bash
-make docker-serve
+make container-serve
 ```

 Aby obejrzeć zawartość serwisu otwórz w przeglądarce adres http://localhost:1313. Po każdej zmianie plików źródłowych, Hugo automatycznie aktualizuje stronę i odświeża jej widok w przeglądarce.
@@ -35,13 +35,13 @@ A maneira recomendada de executar o site do Kubernetes localmente é executar um
 Se você tiver o Docker [em funcionamento](https://www.docker.com/get-started), crie a imagem do Docker do `kubernetes-hugo` localmente:

 ```bash
-make docker-image
+make container-image
 ```

 Depois que a imagem foi criada, você pode executar o site localmente:

 ```bash
-make docker-serve
+make container-serve
 ```

 Abra seu navegador para http://localhost:1313 para visualizar o site. Conforme você faz alterações nos arquivos de origem, Hugo atualiza o site e força a atualização do navegador.
@@ -38,8 +38,8 @@ hugo server --buildFuture
 Узнать подробнее о том, как поучаствовать в документации Kubernetes, вы можете по ссылкам ниже:

 * [Начните вносить свой вклад](https://kubernetes.io/docs/contribute/)
-* [Использование шаблонов страниц](http://kubernetes.io/docs/contribute/style/page-templates/)
-* [Руководство по оформлению документации](http://kubernetes.io/docs/contribute/style/style-guide/)
+* [Использование шаблонов страниц](https://kubernetes.io/docs/contribute/style/page-content-types/)
+* [Руководство по оформлению документации](https://kubernetes.io/docs/contribute/style/style-guide/)
 * [Руководство по локализации Kubernetes](https://kubernetes.io/docs/contribute/localization/)

 ## Файл `README.md` на других языках
@@ -31,13 +31,13 @@ Cách được đề xuất để chạy trang web Kubernetes cục bộ là dù
 Nếu bạn có Docker đang [up và running](https://www.docker.com/get-started), build `kubernetes-hugo` Docker image cục bộ:

 ```bash
-make docker-image
+make container-image
 ```

 Khi image đã được built, bạn có thể chạy website cục bộ:

 ```bash
-make docker-serve
+make container-serve
 ```

 Mở trình duyệt và đến địa chỉ http://localhost:1313 để xem website. Khi bạn thay đổi các file nguồn, Hugo cập nhật website và buộc làm mới trình duyệt.
@@ -101,7 +101,7 @@ Learn more about SIG Docs Kubernetes community and meetings on the [community pa

 You can also reach the maintainers of this project at:

- [Slack](https://kubernetes.slack.com/messages/sig-docs)
+- [Slack](https://kubernetes.slack.com/messages/sig-docs) [Get an invite for this Slack](https://slack.k8s.io/)
 - [Mailing List](https://groups.google.com/forum/#!forum/kubernetes-sig-docs)

 # Contributing to the docs
@@ -17,14 +17,22 @@ limitations under the License.
  var Search = {
    init: function () {
      $(document).ready(function () {
+        // Fill the search input form with the current search keywords
+        const searchKeywords = new URLSearchParams(location.search).get('q');
+        if (searchKeywords !== null && searchKeywords !== '') {
+          const searchInput = document.querySelector('.td-search-input');
+          searchInput.focus();
+          searchInput.value = searchKeywords;
+        }
+
+        // Set a keydown event
        $(document).on("keypress", ".td-search-input", function (e) {
          if (e.keyCode !== 13) {
            return;
          }

          var query = $(this).val();
-          var searchPage = "{{ "docs/search/" | absURL }}?q=" + query;
-          document.location = searchPage;
+          document.location = "{{ "search/" | absURL }}?q=" + query;

          return false;
        });
@@ -42,6 +42,10 @@ $video-section-height: 200px;

 body {
  background-color: white;
+
+  a {
+    color: $blue;
+  }
 }

 section {
@@ -71,6 +75,7 @@ footer {
    background-color: $blue;
    text-decoration: none;
    font-size: 1rem;
+    border: 0px;
  }

  #cellophane {
@@ -336,7 +341,6 @@ dd {
    width: 100%;
    height: 45px;
    line-height: 45px;
-    font-family: "Roboto", sans-serif;
    font-size: 20px;
    color: $blue;
  }
@@ -612,7 +616,6 @@ section#cncf {
  padding-top: 30px;
  padding-bottom: 80px;
  background-size: auto;
-  // font-family: "Roboto Mono", monospace !important;
  font-size: 24px;
  // font-weight: bold;

@@ -20,6 +20,15 @@ $announcement-size-adjustment: 8px;
      padding-top: 2rem !important;
    }
  }
+
+  .ui-widget {
+    font-family: inherit;
+    font-size: inherit;
+  }
+
+  .ui-widget-content a {
+    color: $blue;
+  }
 }

 section {
@@ -268,22 +277,34 @@ main {

 // blockquotes and callouts

-blockquote {
-  padding: 0.4rem 0.4rem 0.4rem 1rem !important;
-}
+.td-content, body {
+  blockquote.callout {
+    padding: 0.4rem 0.4rem 0.4rem 1rem;
+    border: 1px solid #eee;
+    border-left-width: 0.5em;
+    background: #fff;
+    color: #000;
+    margin-top: 0.5em;
+    margin-bottom: 0.5em;
+  }
+  blockquote.callout {
+    border-radius: calc(1em/3);
+  }
+  .callout.caution {
+    border-left-color: #f0ad4e;
+  }

-// callouts are contained in static CSS as well. these require override.
+  .callout.note {
+    border-left-color: #428bca;
+  }

-.caution {
-  border-left-color: #f0ad4e !important;
-}
+  .callout.warning {
+    border-left-color: #d9534f;
+  }

-.note {
-  border-left-color: #428bca !important;
-}
-
-.warning {
-  border-left-color: #d9534f !important;
+  h1:first-of-type + blockquote.callout {
+    margin-top: 1.5em;
+  }
 }

 .deprecation-warning {
@@ -11,3 +11,5 @@ Add styles or override variables from the theme here. */
@import "base";
@import "tablet";
@import "desktop";
+
+$primary: #3371e3;
@@ -112,6 +112,8 @@ copyright_linux =  "Copyright © 2020 The Linux Foundation ®."
 version_menu = "Versions"

 time_format_blog = "Monday, January 02, 2006"
+time_format_default = "January 02, 2006 at 3:04 PM PST"
+
 description = "Production-Grade Container Orchestration"
 showedit = true

@@ -124,9 +126,13 @@ docsbranch = "master"
 deprecated = false
 currentUrl = "https://kubernetes.io/docs/home/"
 nextUrl = "https://kubernetes-io-vnext-staging.netlify.com/"
-githubWebsiteRepo = "github.com/kubernetes/website"
+
+# See codenew shortcode
 githubWebsiteRaw = "raw.githubusercontent.com/kubernetes/website"

+# GitHub repository link for editing a page and opening issues.
+github_repo = "https://github.com/kubernetes/website"
+
 # param for displaying an announcement block on every page.
 # See /i18n/en.toml for message text and title.
 announcement = true
@@ -313,11 +319,23 @@ languagedirection = "ltr"
 time_format_blog = "2006.01.02"
 language_alternatives = ["en"]

+[languages.fa]
+title = "Kubernetes"
+description = "ارکستراسیون کانتینرها برای محیطهای عملیاتی"
+languageName = "فارسی"
+contentDir = "content/fa"
+weight = 5
+languagedirection = "rtl"
+
+[languages.fa.params]
+time_format_blog = "2006.01.02"
+language_alternatives = ["en"]
+
 [languages.fr]
 title = "Kubernetes"
 description = "Solution professionnelle d’orchestration de conteneurs"
 languageName ="Français"
-weight = 5
+weight = 6
 contentDir = "content/fr"
 languagedirection = "ltr"

@@ -330,7 +348,7 @@ language_alternatives = ["en"]
 title = "Kubernetes"
 description = "Orchestrazione di Container in produzione"
 languageName = "Italiano"
-weight = 6
+weight = 7
 contentDir = "content/it"
 languagedirection = "ltr"

@@ -343,7 +361,7 @@ language_alternatives = ["en"]
 title = "Kubernetes"
 description = "Production-Grade Container Orchestration"
 languageName ="Norsk"
-weight = 7
+weight = 8
 contentDir = "content/no"
 languagedirection = "ltr"

@@ -356,7 +374,7 @@ language_alternatives = ["en"]
 title = "Kubernetes"
 description = "Produktionsreife Container-Orchestrierung"
 languageName ="Deutsch"
-weight = 8
+weight = 9
 contentDir = "content/de"
 languagedirection = "ltr"

@@ -369,7 +387,7 @@ language_alternatives = ["en"]
 title = "Kubernetes"
 description = "Orquestación de contenedores para producción"
 languageName ="Español"
-weight = 9
+weight = 10
 contentDir = "content/es"
 languagedirection = "ltr"

@@ -382,7 +400,7 @@ language_alternatives = ["en"]
 title = "Kubernetes"
 description = "Orquestração de contêineres em nível de produção"
 languageName ="Português"
-weight = 9
+weight = 10
 contentDir = "content/pt"
 languagedirection = "ltr"

@@ -395,7 +413,7 @@ language_alternatives = ["en"]
 title = "Kubernetes"
 description = "Orkestrasi Kontainer dengan Skala Produksi"
 languageName ="Bahasa Indonesia"
-weight = 10
+weight = 11
 contentDir = "content/id"
 languagedirection = "ltr"

@@ -408,7 +426,7 @@ language_alternatives = ["en"]
 title = "Kubernetes"
 description = "Production-Grade Container Orchestration"
 languageName = "Hindi"
-weight = 11
+weight = 12
 contentDir = "content/hi"
 languagedirection = "ltr"

@@ -421,14 +439,14 @@ title = "Kubernetes"
 description = "Giải pháp điều phối container trong môi trường production"
 languageName = "Tiếng Việt"
 contentDir = "content/vi"
-weight = 12
+weight = 13
 languagedirection = "ltr"

 [languages.ru]
 title = "Kubernetes"
 description = "Первоклассная оркестрация контейнеров"
 languageName = "Русский"
-weight = 12
+weight = 13
 contentDir = "content/ru"
 languagedirection = "ltr"

@@ -441,7 +459,7 @@ language_alternatives = ["en"]
 title = "Kubernetes"
 description = "Produkcyjny system zarządzania kontenerami"
 languageName = "Polski"
-weight = 13
+weight = 14
 contentDir = "content/pl"
 languagedirection = "ltr"

@@ -454,7 +472,7 @@ language_alternatives = ["en"]
 title = "Kubernetes"
 description = "Довершена система оркестрації контейнерів"
 languageName = "Українська"
-weight = 14
+weight = 15
 contentDir = "content/uk"
 languagedirection = "ltr"

@@ -54,7 +54,8 @@ KUBECONFIG=~/.kube/config:~/.kube/kubconfig2 kubectl config view
 # Zeigen Sie das Passwort für den e2e-Benutzer an
 kubectl config view -o jsonpath='{.users[?(@.name == "e2e")].user.password}'

-kubectl config view -o jsonpath='{.users[].name}'    # eine Liste der Benutzer erhalten
+kubectl config view -o jsonpath='{.users[].name}'    # den ersten Benutzer anzeigen
+kubectl config view -o jsonpath='{.users[*].name}'    # eine Liste der Benutzer erhalten
 kubectl config current-context                       # den aktuellen Kontext anzeigen
 kubectl config use-context my-cluster-name           # Setzen Sie den Standardkontext auf my-cluster-name

@@ -41,11 +41,6 @@ Kubernetes is open source giving you the freedom to take advantage of on-premise
        <button id="desktopShowVideoButton" onclick="kub.showVideo()">Watch Video</button>
        <br>
        <br>
-        <a href="https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/?utm_source=kubernetes.io&utm_medium=nav&utm_campaign=kccnceu20" button id="desktopKCButton">Attend KubeCon EU virtually on August 17-20, 2020</a>
-        <br>
-        <br>
-        <br>
-        <br>
        <a href="https://events.linuxfoundation.org/kubecon-cloudnativecon-north-america/?utm_source=kubernetes.io&utm_medium=nav&utm_campaign=kccncna20" button id="desktopKCButton">Attend KubeCon NA virtually on November 17-20, 2020</a>
 </div>
 <div id="videoPlayer">
@@ -45,7 +45,7 @@ Support for [dynamic maximum volume count](https://github.com/kubernetes/feature

 The StorageObjectInUseProtection feature is now stable and prevents the removal of both [Persistent Volumes](https://github.com/kubernetes/features/issues/499) that are bound to a Persistent Volume Claim, and [Persistent Volume Claims](https://github.com/kubernetes/features/issues/498) that are being used by a pod. This safeguard will help prevent issues from deleting a PV or a PVC that is currently tied to an active pod.

-Each Special Interest Group (SIG) within the community continues to deliver the most-requested enhancements, fixes, and functionality for their respective specialty areas. For a complete list of inclusions by SIG, please visit the [release notes](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.11.md#111-release-notes).
+Each Special Interest Group (SIG) within the community continues to deliver the most-requested enhancements, fixes, and functionality for their respective specialty areas. For a complete list of inclusions by SIG, please visit the [release notes](https://github.com/kubernetes/kubernetes/blob/release-1.11/CHANGELOG-1.11.md#111-release-notes).

 ## Availability

@@ -88,7 +88,7 @@ Is Kubernetes helping your team? Share your story with the community.
 * The CNCF recently expanded its certification offerings to include a Certified Kubernetes Application Developer exam. The CKAD exam certifies an individual's ability to design, build, configure, and expose cloud native applications for Kubernetes. More information can be found [here](https://www.cncf.io/blog/2018/03/16/cncf-announces-ckad-exam/).
 * The CNCF recently added a new partner category, Kubernetes Training Partners (KTP). KTPs are a tier of vetted training providers who have deep experience in cloud native technology training. View partners and learn more [here](https://www.cncf.io/certification/training/).
 * CNCF also offers [online training](https://www.cncf.io/certification/training/) that teaches the skills needed to create and configure a real-world Kubernetes cluster.
-* Kubernetes documentation now features [user journeys](https://k8s.io/docs/home/): specific pathways for learning based on who readers are and what readers want to do. Learning Kubernetes is easier than ever for beginners, and more experienced users can find task journeys specific to cluster admins and application developers.  
+* Kubernetes documentation now features [user journeys](https://k8s.io/docs/home/): specific pathways for learning based on who readers are and what readers want to do. Learning Kubernetes is easier than ever for beginners, and more experienced users can find task journeys specific to cluster admins and application developers.

 ## KubeCon

@@ -1,27 +1,27 @@
 ---
-layout: blog 
+layout: blog
 title: 'Kubernetes  1.19: Accentuate the Paw-sitive'
-date: 2020-08-25 
+date: 2020-08-26
 slug: kubernetes-release-1.19-accentuate-the-paw-sitive
 ---

 **Authors:** [Kubernetes 1.19 Release Team](https://github.com/kubernetes/sig-release/blob/master/releases/release-1.19/release_team.md)

-Finally, we have arrived with Kubernetes 1.19, the second release for 2020, and by far the longest release cycle lasting 20 weeks in total. It consists of 33 enhancements: 12 enhancements are moving to stable, 18 enhancements in beta, and 13 enhancements in alpha. 
+Finally, we have arrived with Kubernetes 1.19, the second release for 2020, and by far the longest release cycle lasting 20 weeks in total. It consists of 34 enhancements: 10 enhancements are moving to stable, 15 enhancements in beta, and 9 enhancements in alpha.

-The 1.19 release was quite different from a regular release due to COVID-19, the George Floyd protests, and several other global events that we experienced as a release team. Due to these events, we made the decision to adjust our timeline and allow the SIGs, Working Groups, and contributors more time to get things done. The extra time also allowed for people to take time to focus on their lives outside of the Kubernetes project, and ensure their mental wellbeing was in a good place. 
+The 1.19 release was quite different from a regular release due to COVID-19, the George Floyd protests, and several other global events that we experienced as a release team. Due to these events, we made the decision to adjust our timeline and allow the SIGs, Working Groups, and contributors more time to get things done. The extra time also allowed for people to take time to focus on their lives outside of the Kubernetes project, and ensure their mental wellbeing was in a good place.

-Contributors are the heart of Kubernetes, not the other way around. The Kubernetes code of conduct asks that people be excellent to one another and despite the unrest in our world, we saw nothing but greatness and humility from the community. 
+Contributors are the heart of Kubernetes, not the other way around. The Kubernetes code of conduct asks that people be excellent to one another and despite the unrest in our world, we saw nothing but greatness and humility from the community.

 ## Major Themes
 ### Increase Kubernetes support window to one year

-A survey conducted in early 2019 by the [Long Term Support (LTS) working group](https://github.com/kubernetes/community/tree/master/wg-lts#readme) showed that a significant subset of Kubernetes end-users fail to upgrade within the current 9-month support period. 
+A survey conducted in early 2019 by the [Long Term Support (LTS) working group](https://github.com/kubernetes/community/tree/master/wg-lts#readme) showed that a significant subset of Kubernetes end-users fail to upgrade within the current 9-month support period.
 This, and other responses from the survey, suggest that 30% of users would be able to keep their deployments on supported versions if the patch support period were extended to 12-14 months. This appears to be true regardless of whether the users are on self build or commercially vendored distributions. An extension would thus lead to more than 80% of users being on supported versions, instead of the 50-60% we have now.
 A yearly support period provides the cushion end-users appear to desire, and is more in harmony with familiar annual planning cycles.
 From Kubernetes version 1.19 on, the support window will be extended to one year.

-### Storage capacity tracking 
+### Storage capacity tracking

 Traditionally, the Kubernetes scheduler was based on the assumptions that additional persistent storage is available everywhere in the cluster and has infinite capacity. Topology constraints addressed the first point, but up to now pod scheduling was still done without considering that the remaining storage capacity may not be enough to start a new pod. [Storage capacity tracking](/docs/concepts/storage/storage-capacity/), a new alpha feature, addresses that by adding an API for a CSI driver to report storage capacity and uses that information in the Kubernetes scheduler when choosing a node for a pod. This feature serves as a stepping stone for supporting dynamic provisioning for local volumes and other volume types that are more capacity constrained.

@@ -35,7 +35,7 @@ All features supported with PersistentVolumeClaims are supported, such as storag
 The alpha version of CSI health monitoring is being released with Kubernetes 1.19. This feature enables CSI Drivers to share abnormal volume conditions from the underlying storage systems with Kubernetes so that they can be reported as events on PVCs or Pods. This feature serves as a stepping stone towards programmatic detection and resolution of individual volume health issues by Kubernetes.

 ### Ingress graduates to General Availability
-In terms of moving the Ingress API towards GA, the API itself has been available in beta for so long that it has attained de facto GA status through usage and adoption (both by users and by load balancer / ingress controller providers). Abandoning it without a full replacement is not a viable approach. It is clearly a useful API and captures a non-trivial set of use cases. At this point, it seems more prudent to declare the current API as something the community will support as a V1, codifying its status, while working on either a V2 Ingress API or an entirely different API with a superset of features. 
+In terms of moving the Ingress API towards GA, the API itself has been available in beta for so long that it has attained de facto GA status through usage and adoption (both by users and by load balancer / ingress controller providers). Abandoning it without a full replacement is not a viable approach. It is clearly a useful API and captures a non-trivial set of use cases. At this point, it seems more prudent to declare the current API as something the community will support as a V1, codifying its status, while working on either a V2 Ingress API or an entirely different API with a superset of features.

 ### Structured logging
 Before v1.19, logging in the Kubernetes control plane couldn't guarantee any uniform structure for log messages and references to Kubernetes objects in those logs. This makes parsing, processing, storing, querying and analyzing logs hard and forces administrators and developers to rely on ad-hoc solutions in most cases based on some regular expressions. Due to those problems any analytical solution based on those logs is hard to implement and maintain.
@@ -45,13 +45,13 @@ This Kubernetes release introduces new methods to the _klog_ library that provid

 ### Client TLS certificate rotation for kubelet

-A kubelet authenticates the kubelet to the kube-apiserver using a private key and certificate. The certificate is supplied to the kubelet when it is first booted, via an out-of-cluster mechanism. Since Kubernetes v1.8, clusters have included a (beta) process for obtaining the initial cert/key pair and rotating it as expiration of the certificate approaches. In Kubernetes v1.19 this graduates to stable. 
+A kubelet authenticates the kubelet to the kube-apiserver using a private key and certificate. The certificate is supplied to the kubelet when it is first booted, via an out-of-cluster mechanism. Since Kubernetes v1.8, clusters have included a (beta) process for obtaining the initial cert/key pair and rotating it as expiration of the certificate approaches. In Kubernetes v1.19 this graduates to stable.

 During the kubelet start-up sequence, the filesystem is scanned for an existing cert/key pair, which is managed by the certificate manager. In the case that a cert/key is available it will be loaded. If not, the kubelet checks its config file for an encoded certificate value or a file reference in the kubeconfig. If the certificate is a bootstrap certificate, this will be used to generate a key, create a certificate signing request and request a signed certificate from the API server.

 When an expiration approaches the cert manager takes care of providing the correct certificate, generating new private keys and requesting new certificates. With the kubelet requesting certificates be signed as part of its boot sequence, and on an ongoing basis, certificate signing requests from the kubelet need to be auto approved to make cluster administration manageable.

-## Other Updates 
+## Other Updates
 ### Graduated to Stable
 * [Seccomp](https://github.com/kubernetes/enhancements/issues/135)
 * [Kubelet client TLS certificate rotation](https://github.com/kubernetes/enhancements/issues/266)
@@ -76,20 +76,20 @@ Check out the full details of the Kubernetes 1.19 release in our [release notes]

 ## Availability

-Kubernetes 1.19 is available for download on [GitHub](https://github.com/kubernetes/kubernetes/releases/tag/v1.19.0). To get started with Kubernetes, check out these [interactive tutorials](https://kubernetes.io/docs/tutorials/) or run local Kubernetes clusters using Docker container “nodes” with [KinD](https://kind.sigs.k8s.io/) (Kubernetes in Docker). You can also easily install 1.19 using [kubeadm](https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/). 
+Kubernetes 1.19 is available for download on [GitHub](https://github.com/kubernetes/kubernetes/releases/tag/v1.19.0). To get started with Kubernetes, check out these [interactive tutorials](https://kubernetes.io/docs/tutorials/) or run local Kubernetes clusters using Docker container “nodes” with [KinD](https://kind.sigs.k8s.io/) (Kubernetes in Docker). You can also easily install 1.19 using [kubeadm](https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/).

 ## Release Team
-This release is made possible through the efforts of hundreds of individuals who contributed both technical and non-technical content. Special thanks to the [release team](https://github.com/kubernetes/sig-release/blob/master/releases/release-1.19/release_team.md) led by Taylor Dolezal, Senior Developer Advocate at HashiCorp. The 34 release team members coordinated many aspects of the release, from documentation to testing, validation, and feature completeness. 
+This release is made possible through the efforts of hundreds of individuals who contributed both technical and non-technical content. Special thanks to the [release team](https://github.com/kubernetes/sig-release/blob/master/releases/release-1.19/release_team.md) led by Taylor Dolezal, Senior Developer Advocate at HashiCorp. The 34 release team members coordinated many aspects of the release, from documentation to testing, validation, and feature completeness.

 As the Kubernetes community has grown, our release process represents an amazing demonstration of collaboration in open source software development. Kubernetes continues to gain new users at a rapid pace. This growth creates a positive feedback cycle where more contributors commit code creating a more vibrant ecosystem. Kubernetes has had over [49,000 individual contributors](https://k8s.devstats.cncf.io/d/24/overall-project-statistics?orgId=1) to date and an active community of more than 3,000 people.

 ## Release Logo
-All of you inspired this Kubernetes 1.19 release logo! This release was a bit more of a marathon and a testament to when the world is a wild place, we can come together and do unbelievable things. 
+All of you inspired this Kubernetes 1.19 release logo! This release was a bit more of a marathon and a testament to when the world is a wild place, we can come together and do unbelievable things.

 ![Kubernetes 1.19 Release Logo](/images/blog/2020-08-26-kubernetes-1.19-release-announcement/accentuate.png)


-"Accentuate the Paw-sitive" was chosen as the release theme because it captures the positive outlook that the release team had, despite the state of the world. The characters pictured in the 1.19 logo represent everyone's personalities on our release team, from emo to peppy, and beyond! 
+"Accentuate the Paw-sitive" was chosen as the release theme because it captures the positive outlook that the release team had, despite the state of the world. The characters pictured in the 1.19 logo represent everyone's personalities on our release team, from emo to peppy, and beyond!

 About the designer: Hannabeth Lagerlof is a Visual Designer based in Los Angeles, California, and she has an extensive background in Environments and Graphic Design. Hannabeth creates art and user experiences that inspire connection. You can find Hannabeth on Twitter as @emanate_design.

@@ -105,10 +105,10 @@ The milestone until which contributors implement the features was extended from
 * The CNCF just concluded its very first Virtual KubeCon. All talks are [on-demand]( https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/) for anyone registered, it's not too late!
 * The [Certified Kubernetes Security Specialist](https://www.cncf.io/blog/2020/07/15/certified-kubernetes-security-specialist-cks-coming-in-november/) (CKS) coming in November! CKS focuses on cluster & system hardening, minimizing microservice vulnerabilities and the security of the supply chain.
 * CNCF published the second [State of Cloud Native Development](https://www.cncf.io/blog/2020/08/14/state-of-cloud-native-development/), showing the massively growing number of cloud native developer using container and serverless technology.
-* [Kubernetes.dev](https://www.kubernetes.dev), a Kubernetes contributor focused website has been launched. It brings the  contributor documentation, resources and project event information into one central location.  
+* [Kubernetes.dev](https://www.kubernetes.dev), a Kubernetes contributor focused website has been launched. It brings the  contributor documentation, resources and project event information into one central location.

 ## Project Velocity
-The [Kubernetes DevStats dashboard](https://k8s.devstats.cncf.io/d/12/dashboards?orgId=1) illustrates the breakdown of contributions from major company contributors, as well as an impressive set of preconfigured reports on everything from individual contributors to pull request lifecycle times. If you want to gather numbers, facts and figures from Kubernetes and the CNCF community it is the best place to start. 
+The [Kubernetes DevStats dashboard](https://k8s.devstats.cncf.io/d/12/dashboards?orgId=1) illustrates the breakdown of contributions from major company contributors, as well as an impressive set of preconfigured reports on everything from individual contributors to pull request lifecycle times. If you want to gather numbers, facts and figures from Kubernetes and the CNCF community it is the best place to start.

 During this release cycle from April till August, 382 different companies and over 2,464 individuals contributed to Kubernetes. [Check out DevStats](https://k8s.devstats.cncf.io/d/11/companies-contributing-in-repository-groups?orgId=1&var-period=m&var-repogroup_name=All&from=1585692000000&to=1598392799000) to learn more about the overall velocity of the Kubernetes project and community.

@@ -0,0 +1,30 @@
+---
+layout: blog 
+title: 'Increasing the Kubernetes Support Window to One Year'
+date: 2020-08-31 
+slug: kubernetes-1-19-feature-one-year-support
+---
+
+**Authors:** Tim Pepper (VMware), Nick Young (VMware)
+
+Starting with Kubernetes 1.19, the support window for Kubernetes versions [will increase from 9 months to one year](https://github.com/kubernetes/enhancements/issues/1498). The longer support window is intended to allow organizations to perform major upgrades at a time of the year that works the best for them.
+
+This is a big change. For many years, the Kubernetes project has delivered a new minor release (e.g.: 1.13 or 1.14) every 3 months. The project provides bugfix support via patch releases (e.g.: 1.13.Y) for three parallel branches of the codebase. Combined, this led to each minor release (e.g.: 1.13) having a patch release stream of support for approximately 9 months. In the end, a cluster operator had to upgrade at least every 9 months to remain supported. 
+
+A survey conducted in early 2019 by the WG LTS showed that a significant subset of Kubernetes end-users fail to upgrade within the 9-month support period. 
+
+![Versions in Production](/images/blog/2020-08-31-increase-kubernetes-support-one-year/versions-in-production-text-2.png)
+
+This, and other responses from the survey, suggest that a considerable portion of our community would better be able to manage their deployments on supported versions if the patch support period were extended to 12-14 months. It appears to be true regardless of whether the users are on DIY builds or commercially vendored distributions. An extension in the patch support length of time would thus lead to a larger percentage of our user base running supported versions compared to what we have now.
+
+A yearly support period provides the cushion end-users appear to desire, and is more aligned with familiar annual planning cycles.
+There are many unknowns about changing the support windows for a project with as many moving parts as Kubernetes. Keeping the change relatively small (relatively being the important word), gives us the chance to find out what those unknowns are in detail and address them.
+From Kubernetes version 1.19 on, the support window will be extended to one year. For Kubernetes versions 1.16, 1.17, and 1.18, the story is more complicated.
+
+All of these versions still fall under the older “three releases support” model, and will drop out of support when 1.19, 1.20 and 1.21 are respectively released. However, because the 1.19 release has been delayed due to the events of 2020, they will end up with close to a year of support (depending on their exact release dates).
+
+For example, 1.19 was released on the 26th of August 2020, which is 11 months since the release of 1.16. Since 1.16 is still under the old release policy, this means that it is now out of support.
+
+![Support Timeline](/images/blog/2020-08-31-increase-kubernetes-support-one-year/support-timeline.png)
+
+If you’ve got thoughts or feedback, we’d love to hear them. Please contact us on [#wg-lts](https://kubernetes.slack.com/messages/wg-lts/) on the Kubernetes Slack, or to the [kubernetes-wg-lts mailing list](https://groups.google.com/g/kubernetes-wg-lts).
@@ -0,0 +1,394 @@
+---
+layout: blog
+title: 'Ephemeral volumes with storage capacity tracking: EmptyDir on steroids'
+date: 2020-09-01
+slug: ephemeral-volumes-with-storage-capacity-tracking
+---
+
+**Author:** Patrick Ohly (Intel)
+
+Some applications need additional storage but don't care whether that
+data is stored persistently across restarts. For example, caching
+services are often limited by memory size and can move infrequently
+used data into storage that is slower than memory with little impact
+on overall performance. Other applications expect some read-only input
+data to be present in files, like configuration data or secret keys.
+
+Kubernetes already supports several kinds of such [ephemeral
+volumes](/docs/concepts/storage/ephemeral-volumes), but the
+functionality of those is limited to what is implemented inside
+Kubernetes.
+
+[CSI ephemeral volumes](https://kubernetes.io/blog/2020/01/21/csi-ephemeral-inline-volumes/)
+made it possible to extend Kubernetes with CSI
+drivers that provide light-weight, local volumes. These [*inject
+arbitrary states, such as configuration, secrets, identity, variables
+or similar
+information*](https://github.com/kubernetes/enhancements/blob/master/keps/sig-storage/20190122-csi-inline-volumes.md#motivation).
+CSI drivers must be modified to support this Kubernetes feature,
+i.e. normal, standard-compliant CSI drivers will not work, and
+by design such volumes are supposed to be usable on whatever node
+is chosen for a pod.
+
+This is problematic for volumes which consume significant resources on
+a node or for special storage that is only available on some nodes.
+Therefore, Kubernetes 1.19 introduces two new alpha features for
+volumes that are conceptually more like the `EmptyDir` volumes:
+- [*generic* ephemeral volumes](/docs/concepts/storage/ephemeral-volumes#generic-ephemeral-volumes) and
+- [CSI storage capacity tracking](/docs/concepts/storage/storage-capacity).
+
+The advantages of the new approach are:
+- Storage can be local or network-attached.
+- Volumes can have a fixed size that applications are never able to exceed.
+- Works with any CSI driver that supports provisioning of persistent
+  volumes and (for capacity tracking) implements the CSI `GetCapacity` call.
+- Volumes may have some initial data, depending on the driver and
+  parameters.
+- All of the typical volume operations (snapshotting,
+  resizing, the future storage capacity tracking, etc.)
+  are supported.
+- The volumes are usable with any app controller that accepts
+  a Pod or volume specification.
+- The Kubernetes scheduler itself picks suitable nodes, i.e. there is
+  no need anymore to implement and configure scheduler extenders and
+  mutating webhooks.
+
+This makes generic ephemeral volumes a suitable solution for several
+use cases:
+
+# Use cases
+
+## Persistent Memory as DRAM replacement for memcached
+
+Recent releases of memcached added [support for using Persistent
+Memory](https://memcached.org/blog/persistent-memory/) (PMEM) instead
+of standard DRAM. When deploying memcached through one of the app
+controllers, generic ephemeral volumes make it possible to request a PMEM volume
+of a certain size from a CSI driver like
+[PMEM-CSI](https://intel.github.io/pmem-csi/).
+
+## Local LVM storage as scratch space
+
+Applications working with data sets that exceed the RAM size can
+request local storage with performance characteristics or size that is
+not met by the normal Kubernetes `EmptyDir` volumes. For example,
+[TopoLVM](https://github.com/cybozu-go/topolvm) was written for that
+purpose.
+
+## Read-only access to volumes with data
+
+Provisioning a volume might result in a non-empty volume:
+- [restore a snapshot](/docs/concepts/storage/persistent-volumes/#volume-snapshot-and-restore-volume-from-snapshot-support)
+- [cloning a volume](/docs/concepts/storage/volume-pvc-datasource)
+- [generic data populators](https://github.com/kubernetes/enhancements/blob/master/keps/sig-storage/20200120-generic-data-populators.md)
+
+Such volumes can be mounted read-only.
+
+# How it works
+
+## Generic ephemeral volumes
+
+The key idea behind generic ephemeral volumes is that a new volume
+source, the so-called
+[`EphemeralVolumeSource`](/docs/reference/generated/kubernetes-api/#ephemeralvolumesource-v1alpha1-core)
+contains all fields that are needed to created a volume claim
+(historically called persistent volume claim, PVC). A new controller
+in the `kube-controller-manager` waits for Pods which embed such a
+volume source and then creates a PVC for that pod. To a CSI driver
+deployment, that PVC looks like any other, so no special support is
+needed.
+
+As long as these PVCs exist, they can be used like any other volume claim. In
+particular, they can be referenced as data source in volume cloning or
+snapshotting. The PVC object also holds the current status of the
+volume.
+
+Naming of the automatically created PVCs is deterministic: the name is
+a combination of Pod name and volume name, with a hyphen (`-`) in the
+middle. This deterministic naming makes it easier to
+interact with the PVC because one does not have to search for it once
+the Pod name and volume name are known. The downside is that the name might
+be in use already. This is detected by Kubernetes and then blocks Pod
+startup.
+
+To ensure that the volume gets deleted together with the pod, the
+controller makes the Pod the owner of the volume claim. When the Pod
+gets deleted, the normal garbage-collection mechanism also removes the
+claim and thus the volume.
+
+Claims select the storage driver through the normal storage class
+mechanism. Although storage classes with both immediate and late
+binding (aka `WaitForFirstConsumer`) are supported, for ephemeral
+volumes it makes more sense to use `WaitForFirstConsumer`: then Pod
+scheduling can take into account both node utilization and
+availability of storage when choosing a node. This is where the other
+new feature comes in.
+
+## Storage capacity tracking
+
+Normally, the Kubernetes scheduler has no information about where a
+CSI driver might be able to create a volume. It also has no way of
+talking directly to a CSI driver to retrieve that information. It
+therefore tries different nodes until it finds one where all volumes
+can be made available (late binding) or leaves it entirely to the
+driver to choose a location (immediate binding).
+
+The new [`CSIStorageCapacity` alpha
+API](/docs/reference/generated/kubernetes-api/v1.19/#csistoragecapacity-v1alpha1-storage-k8s-io)
+allows storing the necessary information in etcd where it is available to the
+scheduler. In contrast to support for generic ephemeral volumes,
+storage capacity tracking must be [enabled when deploying a CSI
+driver](https://github.com/kubernetes-csi/external-provisioner/blob/master/README.md#capacity-support):
+the `external-provisioner` must be told to publish capacity
+information that it then retrieves from the CSI driver through the normal
+`GetCapacity` call.
+<!-- TODO: update the link with a revision once https://github.com/kubernetes-csi/external-provisioner/pull/450 is merged -->
+
+When the Kubernetes scheduler needs to choose a node for a Pod with an
+unbound volume that uses late binding and the CSI driver deployment
+has opted into the feature by setting the [`CSIDriver.storageCapacity`
+flag](/docs/reference/generated/kubernetes-api/v1.19/#csidriver-v1beta1-storage-k8s-io)
+flag, the scheduler automatically filters out nodes that do not have
+access to enough storage capacity. This works for generic ephemeral
+and persistent volumes but *not* for CSI ephemeral volumes because the
+parameters of those are opaque for Kubernetes.
+
+As usual, volumes with immediate binding get created before scheduling
+pods, with their location chosen by the storage driver. Therefore, the
+external-provisioner's default configuration skips storage
+classes with immediate binding as the information wouldn't be used anyway.
+
+Because the Kubernetes scheduler must act on potentially outdated
+information, it cannot be ensured that the capacity is still available
+when a volume is to be created. Still, the chances that it can be created
+without retries should be higher.
+
+# Security
+
+## CSIStorageCapacity
+
+CSIStorageCapacity objects are namespaced. When deploying each CSI
+drivers in its own namespace and, as recommended, limiting the RBAC
+permissions for CSIStorageCapacity to that namespace, it is
+always obvious where the data came from. However, Kubernetes does
+not check that and typically drivers get installed in the same
+namespace anyway, so ultimately drivers are *expected to behave* and
+not publish incorrect data.
+
+## Generic ephemeral volumes
+
+If users have permission to create a Pod (directly or indirectly),
+then they can also create generic ephemeral volumes even when they do
+not have permission to create a volume claim. That's because RBAC
+permission checks are applied to the controller which creates the
+PVC, not the original user. This is a fundamental change that must be
+[taken into
+account](/docs/concepts/storage/ephemeral-volumes#security) before
+enabling the feature in clusters where untrusted users are not
+supposed to have permission to create volumes.
+
+# Example
+
+A [special branch](https://github.com/intel/pmem-csi/commits/kubernetes-1-19-blog-post)
+in PMEM-CSI contains all the necessary changes to bring up a
+Kubernetes 1.19 cluster inside QEMU VMs with both alpha features
+enabled. The PMEM-CSI driver code is used unchanged, only the
+deployment was updated.
+
+On a suitable machine (Linux, non-root user can use Docker - see the
+[QEMU and
+Kubernetes](https://intel.github.io/pmem-csi/0.7/docs/autotest.html#qemu-and-kubernetes)
+section in the PMEM-CSI documentation), the following commands bring
+up a cluster and install the PMEM-CSI driver:
+
+```console
+git clone --branch=kubernetes-1-19-blog-post https://github.com/intel/pmem-csi.git
+cd pmem-csi
+export TEST_KUBERNETES_VERSION=1.19 TEST_FEATURE_GATES=CSIStorageCapacity=true,GenericEphemeralVolume=true TEST_PMEM_REGISTRY=intel
+make start && echo && test/setup-deployment.sh
+```
+
+If all goes well, the output contains the following usage
+instructions:
+
+```
+The test cluster is ready. Log in with [...]/pmem-csi/_work/pmem-govm/ssh.0, run
+kubectl once logged in.  Alternatively, use kubectl directly with the
+following env variable:
+   KUBECONFIG=[...]/pmem-csi/_work/pmem-govm/kube.config
+
+secret/pmem-csi-registry-secrets created
+secret/pmem-csi-node-secrets created
+serviceaccount/pmem-csi-controller created
+...
+To try out the pmem-csi driver ephemeral volumes:
+   cat deploy/kubernetes-1.19/pmem-app-ephemeral.yaml |
+   [...]/pmem-csi/_work/pmem-govm/ssh.0 kubectl create -f -
+```
+
+The CSIStorageCapacity objects are not meant to be human-readable, so
+some post-processing is needed. The following Golang template filters
+all objects by the storage class that the example uses and prints the
+name, topology and capacity:
+
+```console
+kubectl get \
+        -o go-template='{{range .items}}{{if eq .storageClassName "pmem-csi-sc-late-binding"}}{{.metadata.name}} {{.nodeTopology.matchLabels}} {{.capacity}}
+{{end}}{{end}}' \
+        csistoragecapacities
+```
+
+```
+csisc-2js6n map[pmem-csi.intel.com/node:pmem-csi-pmem-govm-worker2] 30716Mi
+csisc-sqdnt map[pmem-csi.intel.com/node:pmem-csi-pmem-govm-worker1] 30716Mi
+csisc-ws4bv map[pmem-csi.intel.com/node:pmem-csi-pmem-govm-worker3] 30716Mi
+```
+
+One individual object has the following content:
+
+```console
+kubectl describe csistoragecapacities/csisc-6cw8j
+```
+
+```
+Name:         csisc-sqdnt
+Namespace:    default
+Labels:       <none>
+Annotations:  <none>
+API Version:  storage.k8s.io/v1alpha1
+Capacity:     30716Mi
+Kind:         CSIStorageCapacity
+Metadata:
+  Creation Timestamp:  2020-08-11T15:41:03Z
+  Generate Name:       csisc-
+  Managed Fields:
+    ...
+  Owner References:
+    API Version:     apps/v1
+    Controller:      true
+    Kind:            StatefulSet
+    Name:            pmem-csi-controller
+    UID:             590237f9-1eb4-4208-b37b-5f7eab4597d1
+  Resource Version:  2994
+  Self Link:         /apis/storage.k8s.io/v1alpha1/namespaces/default/csistoragecapacities/csisc-sqdnt
+  UID:               da36215b-3b9d-404a-a4c7-3f1c3502ab13
+Node Topology:
+  Match Labels:
+    pmem-csi.intel.com/node:  pmem-csi-pmem-govm-worker1
+Storage Class Name:           pmem-csi-sc-late-binding
+Events:                       <none>
+```
+
+Now let's create the example app with one generic ephemeral
+volume. The `pmem-app-ephemeral.yaml` file contains:
+
+```yaml
+# This example Pod definition demonstrates
+# how to use generic ephemeral inline volumes
+# with a PMEM-CSI storage class.
+kind: Pod
+apiVersion: v1
+metadata:
+  name: my-csi-app-inline-volume
+spec:
+  containers:
+    - name: my-frontend
+      image: intel/pmem-csi-driver-test:v0.7.14
+      command: [ "sleep", "100000" ]
+      volumeMounts:
+      - mountPath: "/data"
+        name: my-csi-volume
+  volumes:
+  - name: my-csi-volume
+    ephemeral:
+      volumeClaimTemplate:
+        spec:
+          accessModes:
+          - ReadWriteOnce
+          resources:
+            requests:
+              storage: 4Gi
+          storageClassName: pmem-csi-sc-late-binding
+```
+
+After creating that as shown in the usage instructions above, we have one additional Pod and PVC:
+
+```console
+kubectl get pods/my-csi-app-inline-volume -o wide
+```
+
+```
+NAME                       READY   STATUS    RESTARTS   AGE     IP          NODE                         NOMINATED NODE   READINESS GATES
+my-csi-app-inline-volume   1/1     Running   0          6m58s   10.36.0.2   pmem-csi-pmem-govm-worker1   <none>           <none>
+```
+
+```console
+kubectl get pvc/my-csi-app-inline-volume-my-csi-volume
+```
+
+```
+NAME                                     STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS               AGE
+my-csi-app-inline-volume-my-csi-volume   Bound    pvc-c11eb7ab-a4fa-46fe-b515-b366be908823   4Gi        RWO            pmem-csi-sc-late-binding   9m21s
+```
+
+That PVC is owned by the Pod:
+
+```console
+kubectl get -o yaml pvc/my-csi-app-inline-volume-my-csi-volume
+```
+
+```
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  annotations:
+    pv.kubernetes.io/bind-completed: "yes"
+    pv.kubernetes.io/bound-by-controller: "yes"
+    volume.beta.kubernetes.io/storage-provisioner: pmem-csi.intel.com
+    volume.kubernetes.io/selected-node: pmem-csi-pmem-govm-worker1
+  creationTimestamp: "2020-08-11T15:44:57Z"
+  finalizers:
+  - kubernetes.io/pvc-protection
+  managedFields:
+    ...
+  name: my-csi-app-inline-volume-my-csi-volume
+  namespace: default
+  ownerReferences:
+  - apiVersion: v1
+    blockOwnerDeletion: true
+    controller: true
+    kind: Pod
+    name: my-csi-app-inline-volume
+    uid: 75c925bf-ca8e-441a-ac67-f190b7a2265f
+...
+```
+
+Eventually, the storage capacity information for `pmem-csi-pmem-govm-worker1` also gets updated:
+
+```
+csisc-2js6n map[pmem-csi.intel.com/node:pmem-csi-pmem-govm-worker2] 30716Mi
+csisc-sqdnt map[pmem-csi.intel.com/node:pmem-csi-pmem-govm-worker1] 26620Mi
+csisc-ws4bv map[pmem-csi.intel.com/node:pmem-csi-pmem-govm-worker3] 30716Mi
+```
+
+If another app needs more than 26620Mi, the Kubernetes
+scheduler will not pick `pmem-csi-pmem-govm-worker1` anymore.
+
+
+# Next steps
+
+Both features are under development. Several open questions were
+already raised during the alpha review process. The two enhancement
+proposals document the work that will be needed for migration to beta and what
+alternatives were already considered and rejected:
+
+* [KEP-1698: generic ephemeral inline
+volumes](https://github.com/kubernetes/enhancements/blob/9d7a75d/keps/sig-storage/1698-generic-ephemeral-volumes/README.md)
+* [KEP-1472: Storage Capacity
+Tracking](https://github.com/kubernetes/enhancements/tree/9d7a75d/keps/sig-storage/1472-storage-capacity-tracking)
+
+Your feedback is crucial for driving that development. SIG-Storage
+[meets
+regularly](https://github.com/kubernetes/community/tree/master/sig-storage#meetings)
+and can be reached via [Slack and a mailing
+list](https://github.com/kubernetes/community/tree/master/sig-storage#contact).
@@ -0,0 +1,46 @@
+---
+layout: blog
+title: 'Scaling Kubernetes Networking With EndpointSlices'
+date: 2020-09-02
+slug: scaling-kubernetes-networking-with-endpointslices
+---
+
+**Author:** Rob Scott (Google)
+
+EndpointSlices are an exciting new API that provides a scalable and extensible alternative to the Endpoints API. EndpointSlices track IP addresses, ports, readiness, and topology information for Pods backing a Service.
+
+In Kubernetes 1.19 this feature is enabled by default with kube-proxy reading from [EndpointSlices](/docs/concepts/services-networking/endpoint-slices/) instead of Endpoints. Although this will mostly be an invisible change, it should result in noticeable scalability improvements in large clusters. It also enables significant new features in future Kubernetes releases like [Topology Aware Routing](/docs/concepts/services-networking/service-topology/).
+
+## Scalability Limitations of the Endpoints API 
+With the Endpoints API, there was only one Endpoints resource for a Service. That meant that it needed to be able to store IP addresses and ports (network endpoints) for every Pod that was backing the corresponding Service. This resulted in huge API resources. To compound this problem, kube-proxy was running on every node and watching for any updates to Endpoints resources. If even a single network endpoint changed in an Endpoints resource, the whole object would have to be sent to each of those instances of kube-proxy.
+
+A further limitation of the Endpoints API is that it limits the number of network endpoints that can be tracked for a Service. The default size limit for an object stored in etcd is 1.5MB. In some cases that can limit an Endpoints resource to 5,000 Pod IPs. This is not an issue for most users, but it becomes a significant problem for users with Services approaching this size.
+
+To show just how significant these issues become at scale it helps to have a simple example. Think about a Service which has 5,000 Pods, it might end up with a 1.5MB Endpoints resource. If even a single network endpoint in that list changes, the full Endpoints resource will need to be distributed to each Node in the cluster. This becomes quite an issue in a large cluster with 3,000 Nodes. Each update would involve sending 4.5GB of data (1.5MB Endpoints * 3,000 Nodes) across the cluster. That's nearly enough to fill up a DVD, and it would happen for each Endpoints change. Imagine a rolling update that results in all 5,000 Pods being replaced - that's more than 22TB (or 5,000 DVDs) worth of data transferred.
+
+## Splitting endpoints up with the EndpointSlice API
+The EndpointSlice API was designed to address this issue with an approach similar to sharding. Instead of tracking all Pod IPs for a Service with a single Endpoints resource, we split them into multiple smaller EndpointSlices. 
+
+Consider an example where a Service is backed by 15 pods. We'd end up with a single Endpoints resource that tracked all of them. If EndpointSlices were configured to store 5 endpoints each, we'd end up with 3 different EndpointSlices:
+![EndpointSlices](/images/blog/2020-09-02-scaling-kubernetes-networking-endpointslices/endpoint-slices.png)
+
+By default, EndpointSlices  store as many as 100 endpoints each, though this can be configured with the `--max-endpoints-per-slice` flag on kube-controller-manager.
+
+## EndpointSlices provide 10x scalability improvements
+This API dramatically improves networking scalability. Now when a Pod is added or removed, only 1 small EndpointSlice needs to be updated. This difference becomes quite noticeable when hundreds or thousands of Pods are backing a single Service. 
+
+Potentially more significant, now that all Pod IPs for a Service don't need to be stored in a single resource, we don't have to worry about the size limit for objects stored in etcd. EndpointSlices have already been used to scale Services beyond 100,000 network endpoints.
+
+All of this is brought together with some significant performance improvements that have been made in kube-proxy. When using EndpointSlices at scale, significantly less data will be transferred for endpoints updates and kube-proxy should be faster to update iptables or ipvs rules. Beyond that, Services can now scale to at least 10 times beyond any previous limitations.
+
+## EndpointSlices enable new functionality
+Introduced as an alpha feature in Kubernetes v1.16, EndpointSlices were built to enable some exciting new functionality in future Kubernetes releases. This could include dual-stack Services, topology aware routing, and endpoint subsetting.
+
+Dual-Stack Services are an exciting new feature that has been in development alongside EndpointSlices. They will utilize both IPv4 and IPv6 addresses for Services and rely on the addressType field on EndpointSlices to track these addresses by IP family. 
+
+Topology aware routing will update kube-proxy to prefer routing requests within the same zone or region. This makes use of the topology fields stored for each endpoint in an EndpointSlice. As a further refinement of that, we're exploring the potential of endpoint subsetting. This would allow kube-proxy to only watch a subset of EndpointSlices. For example, this might be combined with topology aware routing so that kube-proxy would only need to watch EndpointSlices containing endpoints within the same zone. This would provide another very significant scalability improvement.
+
+## What does this mean for the Endpoints API?
+Although the EndpointSlice API is providing a newer and more scalable alternative to the Endpoints API, the Endpoints API will continue to be considered generally available and stable. The most significant change planned for the Endpoints API will involve beginning to truncate Endpoints that would otherwise run into scalability issues.
+
+The Endpoints API is not going away, but many new features will rely on the EndpointSlice API. To take advantage of the new scalability and functionality that EndpointSlices provide, applications that currently consume Endpoints will likely want to consider supporting EndpointSlices in the future.
@@ -0,0 +1,278 @@
+---
+layout: blog
+title: "Warning: Helpful Warnings Ahead"
+date: 2020-09-03
+slug: warnings
+---
+
+**Author**: Jordan Liggitt (Google)
+
+As Kubernetes maintainers, we're always looking for ways to improve usability while preserving compatibility.
+As we develop features, triage bugs, and answer support questions, we accumulate information that would be helpful for Kubernetes users to know.
+In the past, sharing that information was limited to out-of-band methods like release notes, announcement emails, documentation, and blog posts.
+Unless someone knew to seek out that information and managed to find it, they would not benefit from it.
+
+In Kubernetes v1.19, we added a feature that allows the Kubernetes API server to
+[send warnings to API clients](https://github.com/kubernetes/enhancements/tree/master/keps/sig-api-machinery/1693-warnings).
+The warning is sent using a [standard `Warning` response header](https://tools.ietf.org/html/rfc7234#section-5.5),
+so it does not change the status code or response body in any way.
+This allows the server to send warnings easily readable by any API client, while remaining compatible with previous client versions.
+
+Warnings are surfaced by `kubectl` v1.19+ in `stderr` output, and by the `k8s.io/client-go` client library v0.19.0+ in log output.
+The `k8s.io/client-go` behavior can be overridden [per-process](https://godoc.org/k8s.io/client-go/rest#SetDefaultWarningHandler)
+or [per-client](https://godoc.org/k8s.io/client-go/rest#Config).
+
+## Deprecation Warnings
+
+The first way we are using this new capability is to send warnings for use of deprecated APIs.
+
+Kubernetes is a [big, fast-moving project](https://www.cncf.io/cncf-kubernetes-project-journey/#development-velocity).
+Keeping up with the [changes](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.19.md#changelog-since-v1180)
+in each release can be daunting, even for people who work on the project full-time. One important type of change is API deprecations.
+As APIs in Kubernetes graduate to GA versions, pre-release API versions are deprecated and eventually removed.
+
+Even though there is an [extended deprecation period](/docs/reference/using-api/deprecation-policy/),
+and deprecations are [included in release notes](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.19.md#deprecation),
+they can still be hard to track. During the deprecation period, the pre-release API remains functional,
+allowing several releases to transition to the stable API version. However, we have found that users often don't even realize
+they are depending on a deprecated API version until they upgrade to the release that stops serving it.
+
+Starting in v1.19, whenever a request is made to a deprecated REST API, a warning is returned along with the API response.
+This warning includes details about the release in which the API will no longer be available, and the replacement API version.
+
+Because the warning originates at the server, and is intercepted at the client level, it works for all kubectl commands,
+including high-level commands like `kubectl apply`, and low-level commands like `kubectl get --raw`:
+
+<img alt="kubectl applying a manifest file, then displaying a warning message 'networking.k8s.io/v1beta1 Ingress is deprecated in v1.19+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress'."
+     src="kubectl-warnings.png"
+     style="width:637px;max-width:100%;">
+
+This helps people affected by the deprecation to know the request they are making is deprecated,
+how long they have to address the issue, and what API they should use instead.
+This is especially helpful when the user is applying a manifest they didn't create,
+so they have time to reach out to the authors to ask for an updated version.
+
+We also realized that the person *using* a deprecated API is often not the same person responsible for upgrading the cluster,
+so we added two administrator-facing tools to help track use of deprecated APIs and determine when upgrades are safe.
+
+### Metrics 
+
+Starting in Kubernetes v1.19, when a request is made to a deprecated REST API endpoint,
+an `apiserver_requested_deprecated_apis` gauge metric is set to `1` in the kube-apiserver process.
+This metric has labels for the API `group`, `version`, `resource`, and `subresource`,
+and a `removed_version` label that indicates the Kubernetes release in which the API will no longer be served.
+
+This is an example query using `kubectl`, [prom2json](https://github.com/prometheus/prom2json),
+and [jq](https://stedolan.github.io/jq/) to determine which deprecated APIs have been requested
+from the current instance of the API server:
+
+```sh
+kubectl get --raw /metrics | prom2json | jq '
+  .[] | select(.name=="apiserver_requested_deprecated_apis").metrics[].labels
+'
+```
+
+Output:
+
+```json
+{
+  "group": "extensions",
+  "removed_release": "1.22",
+  "resource": "ingresses",
+  "subresource": "",
+  "version": "v1beta1"
+}
+{
+  "group": "rbac.authorization.k8s.io",
+  "removed_release": "1.22",
+  "resource": "clusterroles",
+  "subresource": "",
+  "version": "v1beta1"
+}
+```
+
+This shows the deprecated `extensions/v1beta1` Ingress and `rbac.authorization.k8s.io/v1beta1` ClusterRole APIs
+have been requested on this server, and will be removed in v1.22.
+
+We can join that information with the `apiserver_request_total` metrics to get more details about the requests being made to these APIs:
+
+```sh
+kubectl get --raw /metrics | prom2json | jq '
+  # set $deprecated to a list of deprecated APIs
+  [
+    .[] | 
+    select(.name=="apiserver_requested_deprecated_apis").metrics[].labels |
+    {group,version,resource}
+  ] as $deprecated 
+  
+  |
+  
+  # select apiserver_request_total metrics which are deprecated
+  .[] | select(.name=="apiserver_request_total").metrics[] |
+  select(.labels | {group,version,resource} as $key | $deprecated | index($key))
+'
+```
+
+Output:
+
+```json
+{
+  "labels": {
+    "code": "0",
+    "component": "apiserver",
+    "contentType": "application/vnd.kubernetes.protobuf;stream=watch",
+    "dry_run": "",
+    "group": "extensions",
+    "resource": "ingresses",
+    "scope": "cluster",
+    "subresource": "",
+    "verb": "WATCH",
+    "version": "v1beta1"
+  },
+  "value": "21"
+}
+{
+  "labels": {
+    "code": "200",
+    "component": "apiserver",
+    "contentType": "application/vnd.kubernetes.protobuf",
+    "dry_run": "",
+    "group": "extensions",
+    "resource": "ingresses",
+    "scope": "cluster",
+    "subresource": "",
+    "verb": "LIST",
+    "version": "v1beta1"
+  },
+  "value": "1"
+}
+{
+  "labels": {
+    "code": "200",
+    "component": "apiserver",
+    "contentType": "application/json",
+    "dry_run": "",
+    "group": "rbac.authorization.k8s.io",
+    "resource": "clusterroles",
+    "scope": "cluster",
+    "subresource": "",
+    "verb": "LIST",
+    "version": "v1beta1"
+  },
+  "value": "1"
+}
+```
+
+The output shows that only read requests are being made to these APIs, and the most requests have been made to watch the deprecated Ingress API.
+
+You can also find that information through the following Prometheus query,
+which returns information about requests made to deprecated APIs which will be removed in v1.22:
+
+```promql
+apiserver_requested_deprecated_apis{removed_version="1.22"} * on(group,version,resource,subresource)
+group_right() apiserver_request_total
+```
+
+### Audit annotations
+
+Metrics are a fast way to check whether deprecated APIs are being used, and at what rate,
+but they don't include enough information to identify particular clients or API objects.
+Starting in Kubernetes v1.19, [audit events](/docs/tasks/debug-application-cluster/audit/)
+for requests to deprecated APIs include an audit annotation of `"k8s.io/deprecated":"true"`.
+Administrators can use those audit events to identify specific clients or objects that need to be updated.
+
+## Custom Resource Definitions
+
+Along with the API server ability to warn about deprecated API use, starting in v1.19, a CustomResourceDefinition can indicate a 
+[particular version of the resource it defines is deprecated](/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definition-versioning/#version-deprecation).
+When API requests to a deprecated version of a custom resource are made, a warning message is returned, matching the behavior of built-in APIs.
+
+The author of the CustomResourceDefinition can also customize the warning for each version if they want to.
+This allows them to give a pointer to a migration guide or other information if needed.
+
+```yaml
+apiVersion: apiextensions.k8s.io/v1
+kind: CustomResourceDefinition
+  name: crontabs.example.com
+spec:
+  versions:
+  - name: v1alpha1
+    # This indicates the v1alpha1 version of the custom resource is deprecated.
+    # API requests to this version receive a warning in the server response.
+    deprecated: true
+    # This overrides the default warning returned to clients making v1alpha1 API requests.
+    deprecationWarning: "example.com/v1alpha1 CronTab is deprecated; use example.com/v1 CronTab (see http://example.com/v1alpha1-v1)"
+    ...
+
+  - name: v1beta1
+    # This indicates the v1beta1 version of the custom resource is deprecated.
+    # API requests to this version receive a warning in the server response.
+    # A default warning message is returned for this version.
+    deprecated: true
+    ...
+
+  - name: v1
+    ...
+```
+
+## Admission Webhooks
+
+[Admission webhooks](/docs/reference/access-authn-authz/extensible-admission-controllers)
+are the primary way to integrate custom policies or validation with Kubernetes.
+Starting in v1.19, admission webhooks can [return warning messages](/docs/reference/access-authn-authz/extensible-admission-controllers/#response)
+that are passed along to the requesting API client. Warnings can be returned with allowed or rejected admission responses.
+
+As an example, to allow a request but warn about a configuration known not to work well, an admission webhook could send this response:
+
+```json
+{
+  "apiVersion": "admission.k8s.io/v1",
+  "kind": "AdmissionReview",
+  "response": {
+    "uid": "<value from request.uid>",
+    "allowed": true,
+    "warnings": [
+      ".spec.memory: requests >1GB do not work on Fridays"
+    ]
+  }
+}
+```
+
+If you are implementing a webhook that returns a warning message, here are some tips:
+
+* Don't include a "Warning:" prefix in the message (that is added by clients on output)
+* Use warning messages to describe problems the client making the API request should correct or be aware of
+* Be brief; limit warnings to 120 characters if possible
+
+There are many ways admission webhooks could use this new feature, and I'm looking forward to seeing what people come up with.
+Here are a couple ideas to get you started:
+
+* webhook implementations adding a "complain" mode, where they return warnings instead of rejections,
+  to allow trying out a policy to verify it is working as expected before starting to enforce it
+* "lint" or "vet"-style webhooks, inspecting objects and surfacing warnings when best practices are not followed
+
+## Kubectl strict mode
+
+If you want to be sure you notice deprecations as soon as possible and get a jump start on addressing them,
+`kubectl` added a `--warnings-as-errors` option in v1.19. When invoked with this option,
+`kubectl` treats any warnings it receives from the server as errors and exits with a non-zero exit code:
+
+<img alt="kubectl applying a manifest file with a --warnings-as-errors flag, displaying a warning message and exiting with a non-zero exit code."
+     src="kubectl-warnings-as-errors.png"
+     style="width:637px;max-width:100%;">
+
+This could be used in a CI job to apply manifests to a current server,
+and required to pass with a zero exit code in order for the CI job to succeed.
+
+## Future Possibilities
+
+Now that we have a way to communicate helpful information to users in context,
+we're already considering other ways we can use this to improve people's experience with Kubernetes.
+A couple areas we're looking at next are warning about [known problematic values](http://issue.k8s.io/64841#issuecomment-395141013)
+we cannot reject outright for compatibility reasons, and warning about use of deprecated fields or field values
+(like selectors using beta os/arch node labels, [deprecated in v1.14](/docs/reference/kubernetes-api/labels-annotations-taints/#beta-kubernetes-io-arch-deprecated)).
+I'm excited to see progress in this area, continuing to make it easier to use Kubernetes.
+
+---
+
+_[Jordan Liggitt](https://twitter.com/liggitt) is a software engineer at Google, and helps lead Kubernetes authentication, authorization, and API efforts._
@@ -0,0 +1,56 @@
+---
+layout: blog
+title: 'Introducing Structured Logs'
+date: 2020-09-04
+slug: kubernetes-1-19-Introducing-Structured-Logs
+---
+
+**Authors:** Marek Siarkowicz (Google), Nathan Beach (Google)
+
+Logs are an essential aspect of observability and a critical tool for debugging. But Kubernetes logs have traditionally been unstructured strings, making any automated parsing difficult and any downstream processing, analysis, or querying challenging to do reliably.
+
+In Kubernetes 1.19, we are adding support for structured logs, which natively support (key, value) pairs and object references. We have also updated many logging calls such that over 99% of logging volume in a typical deployment are now migrated to the structured format.
+
+To maintain backwards compatibility, structured logs will still be outputted as a string where the string contains representations of those "key"="value" pairs. Starting in alpha in 1.19, logs can also be outputted in JSON format using the `--logging-format=json` flag.
+
+## Using Structured Logs
+
+We've added two new methods to the klog library: InfoS and ErrorS. For example, this invocation of InfoS:
+
+```golang
+klog.InfoS("Pod status updated", "pod", klog.KObj(pod), "status", status)
+```
+
+will result in this log:
+
+```
+I1025 00:15:15.525108       1 controller_utils.go:116] "Pod status updated" pod="kube-system/kubedns" status="ready"
+```
+
+Or, if the --logging-format=json flag is set, it will result in this output:
+
+```json
+{
+  "ts": 1580306777.04728,
+  "msg": "Pod status updated",
+  "pod": {
+    "name": "coredns",
+    "namespace": "kube-system"
+  },
+  "status": "ready"
+}
+```
+
+This means downstream logging tools can easily ingest structured logging data and instead of using regular expressions to parse unstructured strings. This also makes processing logs easier, querying logs more robust, and analyzing logs much faster.
+
+With structured logs, all references to Kubernetes objects are structured the same way, so you can filter the output and only log entries referencing the particular pod. You can also find logs indicating how the scheduler was scheduling the pod, how the pod was created, the health probes of the pod, and all other changes in the lifecycle of the pod.
+
+Suppose you are debugging an issue with a pod. With structured logs, you can filter to only those log entries referencing the pod of interest, rather than needing to scan through potentially thousands of log lines to find the relevant ones.
+
+Not only are structured logs more useful when manual debugging of issues, they also enable richer features like automated pattern recognition within logs or tighter correlation of log and trace data.
+
+Finally, structured logs can help reduce storage costs for logs because most storage systems are more efficiently able to compress structured key=value data than unstructured strings.
+
+## Get Involved
+
+While we have updated over 99% of the log entries by log volume in a typical deployment, there are still thousands of logs to be updated. Pick a file or directory that you would like to improve and [migrate existing log calls to use structured logs](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-instrumentation/migration-to-structured-logging.md). It's a great and easy way to make your first contribution to Kubernetes!
@@ -23,7 +23,7 @@ mechanism that allows different cloud providers to integrate their platforms wit

 ## Design

-![Kubernetes components](/images/docs/components-of-kubernetes.png)
+![Kubernetes components](/images/docs/components-of-kubernetes.svg)

 The cloud controller manager runs in the control plane as a replicated set of processes
 (usually, these are containers in Pods). Each cloud-controller-manager implements
@@ -140,7 +140,7 @@ the {{< glossary_tooltip term_id="kube-controller-manager" >}}. These
 built-in controllers provide important core behaviors.

 The Deployment controller and Job controller are examples of controllers that
-come as part of Kubernetes itself (“built-in” controllers).
+come as part of Kubernetes itself ("built-in" controllers).
 Kubernetes lets you run a resilient control plane, so that if any of the built-in
 controllers were to fail, another part of the control plane will take over the work.

@@ -39,7 +39,7 @@ Before choosing a guide, here are some considerations:

 ## Managing a cluster

-* [Managing a cluster](/docs/tasks/administer-cluster/cluster-management/) describes several topics related to the lifecycle of a cluster: creating a new cluster, upgrading your cluster’s master and worker nodes, performing node maintenance (e.g. kernel upgrades), and upgrading the Kubernetes API version of a running cluster.
+* [Managing a cluster](/docs/tasks/administer-cluster/cluster-management/) describes several topics related to the lifecycle of a cluster: creating a new cluster, upgrading your cluster's master and worker nodes, performing node maintenance (e.g. kernel upgrades), and upgrading the Kubernetes API version of a running cluster.

 * Learn how to [manage nodes](/docs/concepts/architecture/nodes/).

@@ -5,12 +5,12 @@ content_type: concept

 <!-- overview -->

+{{% thirdparty-content %}}
+
 Add-ons extend the functionality of Kubernetes.

 This page lists some of the available add-ons and links to their respective installation instructions.

-Add-ons in each section are sorted alphabetically - the ordering does not imply any preferential status.
-
 <!-- body -->

 ## Networking and Network Policy
@@ -1,362 +0,0 @@
---
-title: Cloud Providers
-content_type: concept
-weight: 30
---
-
-<!-- overview -->
-This page explains how to manage Kubernetes running on a specific
-cloud provider. There are many other third-party cloud provider projects, but this list is specific to projects embedded within, or relied upon by Kubernetes itself.
-
-<!-- body -->
-### kubeadm
-[kubeadm](/docs/reference/setup-tools/kubeadm/kubeadm/) is a popular option for creating kubernetes clusters.
-kubeadm has configuration options to specify configuration information for cloud providers. For example a typical
-in-tree cloud provider can be configured using kubeadm as shown below:
-
-```yaml
-apiVersion: kubeadm.k8s.io/v1beta2
-kind: InitConfiguration
-nodeRegistration:
-  kubeletExtraArgs:
-    cloud-provider: "openstack"
-    cloud-config: "/etc/kubernetes/cloud.conf"
---
-apiVersion: kubeadm.k8s.io/v1beta2
-kind: ClusterConfiguration
-kubernetesVersion: v1.13.0
-apiServer:
-  extraArgs:
-    cloud-provider: "openstack"
-    cloud-config: "/etc/kubernetes/cloud.conf"
-  extraVolumes:
-  - name: cloud
-    hostPath: "/etc/kubernetes/cloud.conf"
-    mountPath: "/etc/kubernetes/cloud.conf"
-controllerManager:
-  extraArgs:
-    cloud-provider: "openstack"
-    cloud-config: "/etc/kubernetes/cloud.conf"
-  extraVolumes:
-  - name: cloud
-    hostPath: "/etc/kubernetes/cloud.conf"
-    mountPath: "/etc/kubernetes/cloud.conf"
-```
-
-The in-tree cloud providers typically need both `--cloud-provider` and `--cloud-config` specified in the command lines
-for the [kube-apiserver](/docs/reference/command-line-tools-reference/kube-apiserver/),
-[kube-controller-manager](/docs/reference/command-line-tools-reference/kube-controller-manager/) and the
-[kubelet](/docs/reference/command-line-tools-reference/kubelet/).
-The contents of the file specified in `--cloud-config` for each provider is documented below as well.
-
-For all external cloud providers, please follow the instructions on the individual repositories,
-which are listed under their headings below, or one may view [the list of all repositories](https://github.com/kubernetes?q=cloud-provider-&type=&language=)
-
-## AWS
-This section describes all the possible configurations which can
-be used when running Kubernetes on Amazon Web Services.
-
-If you wish to use the external cloud provider, its repository is [kubernetes/cloud-provider-aws](https://github.com/kubernetes/cloud-provider-aws#readme)
-
-### Node Name
-
-The AWS cloud provider uses the private DNS name of the AWS instance as the name of the Kubernetes Node object.
-
-### Load Balancers
-You can setup [external load balancers](/docs/tasks/access-application-cluster/create-external-load-balancer/)
-to use specific features in AWS by configuring the annotations as shown below.
-
-```yaml
-apiVersion: v1
-kind: Service
-metadata:
-  name: example
-  namespace: kube-system
-  labels:
-    run: example
-  annotations:
-     service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:xx-xxxx-x:xxxxxxxxx:xxxxxxx/xxxxx-xxxx-xxxx-xxxx-xxxxxxxxx #replace this value
-     service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http
-spec:
-  type: LoadBalancer
-  ports:
-  - port: 443
-    targetPort: 5556
-    protocol: TCP
-  selector:
-    app: example
-```
-Different settings can be applied to a load balancer service in AWS using _annotations_. The following describes the annotations supported on AWS ELBs:
-
-* `service.beta.kubernetes.io/aws-load-balancer-access-log-emit-interval`: Used to specify access log emit interval.
-* `service.beta.kubernetes.io/aws-load-balancer-access-log-enabled`: Used on the service to enable or disable access logs.
-* `service.beta.kubernetes.io/aws-load-balancer-access-log-s3-bucket-name`: Used to specify access log s3 bucket name.
-* `service.beta.kubernetes.io/aws-load-balancer-access-log-s3-bucket-prefix`: Used to specify access log s3 bucket prefix.
-* `service.beta.kubernetes.io/aws-load-balancer-additional-resource-tags`: Used on the service to specify a comma-separated list of key-value pairs which will be recorded as additional tags in the ELB. For example: `"Key1=Val1,Key2=Val2,KeyNoVal1=,KeyNoVal2"`.
-* `service.beta.kubernetes.io/aws-load-balancer-backend-protocol`: Used on the service to specify the protocol spoken by the backend (pod) behind a listener. If `http` (default) or `https`, an HTTPS listener that terminates the connection and parses headers is created. If set to `ssl` or `tcp`, a "raw" SSL listener is used. If set to `http` and `aws-load-balancer-ssl-cert` is not used then a HTTP listener is used.
-* `service.beta.kubernetes.io/aws-load-balancer-ssl-cert`: Used on the service to request a secure listener. Value is a valid certificate ARN. For more, see [ELB Listener Config](https://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/elb-listener-config.html) CertARN is an IAM or CM certificate ARN, for example `arn:aws:acm:us-east-1:123456789012:certificate/12345678-1234-1234-1234-123456789012`.
-* `service.beta.kubernetes.io/aws-load-balancer-connection-draining-enabled`: Used on the service to enable or disable connection draining.
-* `service.beta.kubernetes.io/aws-load-balancer-connection-draining-timeout`: Used on the service to specify a connection draining timeout.
-* `service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout`: Used on the service to specify the idle connection timeout.
-* `service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled`: Used on the service to enable or disable cross-zone load balancing.
-* `service.beta.kubernetes.io/aws-load-balancer-security-groups`: Used to specify the security groups to be added to ELB created. This replaces all other security groups previously assigned to the ELB. Security groups defined here should not be shared between services.
-* `service.beta.kubernetes.io/aws-load-balancer-extra-security-groups`: Used on the service to specify additional security groups to be added to ELB created
-* `service.beta.kubernetes.io/aws-load-balancer-internal`: Used on the service to indicate that we want an internal ELB.
-* `service.beta.kubernetes.io/aws-load-balancer-proxy-protocol`: Used on the service to enable the proxy protocol on an ELB. Right now we only accept the value `*` which means enabling the proxy protocol on all ELB backends. In the future we could adjust this to allow setting the proxy protocol only on certain backends.
-* `service.beta.kubernetes.io/aws-load-balancer-ssl-ports`: Used on the service to specify a comma-separated list of ports that will use SSL/HTTPS listeners. Defaults to `*` (all)
-
-The information for the annotations for AWS is taken from the comments on [aws.go](https://github.com/kubernetes/legacy-cloud-providers/blob/master/aws/aws.go)
-
-## Azure
-
-If you wish to use the external cloud provider, its repository is [kubernetes/cloud-provider-azure](https://github.com/kubernetes/cloud-provider-azure#readme)
-
-### Node Name
-
-The Azure cloud provider uses the hostname of the node (as determined by the kubelet or overridden with `--hostname-override`) as the name of the Kubernetes Node object.
-Note that the Kubernetes Node name must match the Azure VM name.
-
-## GCE
-
-If you wish to use the external cloud provider, its repository is [kubernetes/cloud-provider-gcp](https://github.com/kubernetes/cloud-provider-gcp#readme)
-
-### Node Name
-
-The GCE cloud provider uses the hostname of the node (as determined by the kubelet or overridden with `--hostname-override`) as the name of the Kubernetes Node object.
-Note that the first segment of the Kubernetes Node name must match the GCE instance name (e.g. a Node named `kubernetes-node-2.c.my-proj.internal` must correspond to an instance named `kubernetes-node-2`).
-
-## HUAWEI CLOUD
-
-If you wish to use the external cloud provider, its repository is [kubernetes-sigs/cloud-provider-huaweicloud](https://github.com/kubernetes-sigs/cloud-provider-huaweicloud).
-
-## OpenStack
-This section describes all the possible configurations which can
-be used when using OpenStack with Kubernetes.
-
-If you wish to use the external cloud provider, its repository is [kubernetes/cloud-provider-openstack](https://github.com/kubernetes/cloud-provider-openstack#readme)
-
-### Node Name
-
-The OpenStack cloud provider uses the instance name (as determined from OpenStack metadata) as the name of the Kubernetes Node object.
-Note that the instance name must be a valid Kubernetes Node name in order for the kubelet to successfully register its Node object.
-
-### Services
-
-The OpenStack cloud provider
-implementation for Kubernetes supports the use of these OpenStack services from
-the underlying cloud, where available:
-
-| Service                  | API Version(s) | Required |
-|--------------------------|----------------|----------|
-| Block Storage (Cinder)   | V1†, V2, V3    | No       |
-| Compute (Nova)           | V2             | No       |
-| Identity (Keystone)      | V2‡,  V3       | Yes      |
-| Load Balancing (Neutron) | V1§, V2        | No       |
-| Load Balancing (Octavia) | V2             | No       |
-
-† Block Storage V1 API support is deprecated, Block Storage V3 API support was
-added in Kubernetes 1.9.
-
-‡ Identity V2 API support is deprecated and will be removed from the provider in
-a future release. As of the "Queens" release, OpenStack will no longer expose the
-Identity V2 API.
-
-§ Load Balancing V1 API support was removed in Kubernetes 1.9.
-
-Service discovery is achieved by listing the service catalog managed by
-OpenStack Identity (Keystone) using the `auth-url` provided in the provider
-configuration. The provider will gracefully degrade in functionality when
-OpenStack services other than Keystone are not available and simply disclaim
-support for impacted features. Certain features are also enabled or disabled
-based on the list of extensions published by Neutron in the underlying cloud.
-
-### cloud.conf
-Kubernetes knows how to interact with OpenStack via the file cloud.conf. It is
-the file that will provide Kubernetes with credentials and location for the OpenStack auth endpoint.
-You can create a cloud.conf file by specifying the following details in it
-
-#### Typical configuration
-This is an example of a typical configuration that touches the values that most
-often need to be set. It points the provider at the OpenStack cloud's Keystone
-endpoint, provides details for how to authenticate with it, and configures the
-load balancer:
-
-```yaml
-[Global]
-username=user
-password=pass
-auth-url=https://<keystone_ip>/identity/v3
-tenant-id=c869168a828847f39f7f06edd7305637
-domain-id=2a73b8f597c04551a0fdc8e95544be8a
-
-[LoadBalancer]
-subnet-id=6937f8fa-858d-4bc9-a3a5-18d2c957166a
-```
-
-##### Global
-These configuration options for the OpenStack provider pertain to its global
-configuration and should appear in the `[Global]` section of the `cloud.conf`
-file:
-
-* `auth-url` (Required): The URL of the keystone API used to authenticate. On
-  OpenStack control panels, this can be found at Access and Security > API
-  Access > Credentials.
-* `username` (Required): Refers to the username of a valid user set in keystone.
-* `password` (Required): Refers to the password of a valid user set in keystone.
-* `tenant-id` (Required): Used to specify the id of the project where you want
-  to create your resources.
-* `tenant-name` (Optional): Used to specify the name of the project where you
-  want to create your resources.
-* `trust-id` (Optional): Used to specify the identifier of the trust to use for
-  authorization. A trust represents a user's (the trustor) authorization to
-  delegate roles to another user (the trustee), and optionally allow the trustee
-  to impersonate the trustor. Available trusts are found under the
-  `/v3/OS-TRUST/trusts` endpoint of the Keystone API.
-* `domain-id` (Optional): Used to specify the id of the domain your user belongs
-  to.
-* `domain-name` (Optional): Used to specify the name of the domain your user
-  belongs to.
-* `region` (Optional): Used to specify the identifier of the region to use when
-  running on a multi-region OpenStack cloud. A region is a general division of
-  an OpenStack deployment. Although a region does not have a strict geographical
-  connotation, a deployment can use a geographical name for a region identifier
-  such as `us-east`. Available regions are found under the `/v3/regions`
-  endpoint of the Keystone API.
-* `ca-file` (Optional): Used to specify the path to your custom CA file.
-
-
-When using Keystone V3 - which changes tenant to project - the `tenant-id` value
-is automatically mapped to the project construct in the API.
-
-#####  Load Balancer
-These configuration options for the OpenStack provider pertain to the load
-balancer and should appear in the `[LoadBalancer]` section of the `cloud.conf`
-file:
-
-* `lb-version` (Optional): Used to override automatic version detection. Valid
-  values are `v1` or `v2`. Where no value is provided automatic detection will
-  select the highest supported version exposed by the underlying OpenStack
-  cloud.
-* `use-octavia`(Optional): Whether or not to use Octavia for LoadBalancer type
-    of Service implementation instead of using Neutron-LBaaS. Default: true
-   Attention: Openstack CCM use Octavia as default load balancer implementation since v1.17.0
-* `subnet-id` (Optional): Used to specify the id of the subnet you want to
-  create your loadbalancer on. Can be found at Network > Networks. Click on the
-  respective network to get its subnets.
-* `floating-network-id` (Optional): If specified, will create a floating IP for
-  the load balancer.
-* `lb-method` (Optional): Used to specify an algorithm by which load will be
-  distributed amongst members of the load balancer pool. The value can be
-  `ROUND_ROBIN`, `LEAST_CONNECTIONS`, or `SOURCE_IP`. The default behavior if
-  none is specified is `ROUND_ROBIN`.
-* `lb-provider` (Optional): Used to specify the provider of the load balancer.
-  If not specified, the default provider service configured in neutron will be
-  used.
-* `create-monitor` (Optional): Indicates whether or not to create a health
-  monitor for the Neutron load balancer. Valid values are `true` and `false`.
-  The default is `false`. When `true` is specified then `monitor-delay`,
-  `monitor-timeout`, and `monitor-max-retries` must also be set.
-* `monitor-delay` (Optional): The time between sending probes to
-  members of the load balancer. Ensure that you specify a valid time unit. The valid time units are "ns", "us" (or "µs"), "ms", "s", "m", "h"
-* `monitor-timeout` (Optional): Maximum time for a monitor to wait
-  for a ping reply before it times out. The value must be less than the delay
-  value. Ensure that you specify a valid time unit. The valid time units are "ns", "us" (or "µs"), "ms", "s", "m", "h"
-* `monitor-max-retries` (Optional): Number of permissible ping failures before
-  changing the load balancer member's status to INACTIVE. Must be a number
-  between 1 and 10.
-* `manage-security-groups` (Optional): Determines whether or not the load
-  balancer should automatically manage the security group rules. Valid values
-  are `true` and `false`. The default is `false`. When `true` is specified
-  `node-security-group` must also be supplied.
-* `node-security-group` (Optional): ID of the security group to manage.
-
-##### Block Storage
-These configuration options for the OpenStack provider pertain to block storage
-and should appear in the `[BlockStorage]` section of the `cloud.conf` file:
-
-* `bs-version` (Optional): Used to override automatic version detection. Valid
-  values are `v1`, `v2`, `v3` and `auto`. When `auto` is specified automatic
-  detection will select the highest supported version exposed by the underlying
-  OpenStack cloud. The default value if none is provided is `auto`.
-* `trust-device-path` (Optional): In most scenarios the block device names
-  provided by Cinder (e.g. `/dev/vda`) can not be trusted. This boolean toggles
-  this behavior. Setting it to `true` results in trusting the block device names
-  provided by Cinder. The default value of `false` results in the discovery of
-  the device path based on its serial number and `/dev/disk/by-id` mapping and is
-  the recommended approach.
-* `ignore-volume-az` (Optional): Used to influence availability zone use when
-  attaching Cinder volumes. When Nova and Cinder have different availability
-  zones, this should be set to `true`. This is most commonly the case where
-  there are many Nova availability zones but only one Cinder availability zone.
-  The default value is `false` to preserve the behavior used in earlier
-  releases, but may change in the future.
-* `node-volume-attach-limit` (Optional): Maximum number of Volumes that can be
-  attached to the node, default is 256 for cinder.
-
-If deploying Kubernetes versions <= 1.8 on an OpenStack deployment that uses
-paths rather than ports to differentiate between endpoints it may be necessary
-to explicitly set the `bs-version` parameter. A path based endpoint is of the
-form `http://foo.bar/volume` while a port based endpoint is of the form
-`http://foo.bar:xxx`.
-
-In environments that use path based endpoints and Kubernetes is using the older
-auto-detection logic a `BS API version autodetection failed.` error will be
-returned on attempting volume detachment. To workaround this issue it is
-possible to force the use of Cinder API version 2 by adding this to the cloud
-provider configuration:
-
-```yaml
-[BlockStorage]
-bs-version=v2
-```
-
-##### Metadata
-These configuration options for the OpenStack provider pertain to metadata and
-should appear in the `[Metadata]` section of the `cloud.conf` file:
-
-* `search-order` (Optional): This configuration key influences the way that the
-  provider retrieves metadata relating to the instance(s) in which it runs. The
-  default value of `configDrive,metadataService` results in the provider
-  retrieving metadata relating to the instance from the config drive first if
-  available and then the metadata service. Alternative values are:
-  * `configDrive` - Only retrieve instance metadata from the configuration
-    drive.
-  * `metadataService` - Only retrieve instance metadata from the metadata
-    service.
-  * `metadataService,configDrive` - Retrieve instance metadata from the metadata
-    service first if available, then the configuration drive.
-
-  Influencing this behavior may be desirable as the metadata on the
-  configuration drive may grow stale over time, whereas the metadata service
-  always provides the most up to date view. Not all OpenStack clouds provide
-  both configuration drive and metadata service though and only one or the other
-  may be available which is why the default is to check both.
-
-##### Route
-
-These configuration options for the OpenStack provider pertain to the [kubenet]
-Kubernetes network plugin and should appear in the `[Route]` section of the
-`cloud.conf` file:
-
-* `router-id` (Optional): If the underlying cloud's Neutron deployment supports
-  the `extraroutes` extension then use `router-id` to specify a router to add
-  routes to.  The router chosen must span the private networks containing your
-  cluster nodes (typically there is only one node network, and this value should be
-  the default router for the node network).  This value is required to use
-  [kubenet](/docs/concepts/cluster-administration/network-plugins/#kubenet)
-  on OpenStack.
-
-[kubenet]: /docs/concepts/cluster-administration/network-plugins/#kubenet
-
-## vSphere
-
-{{< tabs name="vSphere cloud provider" >}}
-{{% tab name="vSphere >= 6.7U3" %}}
-For all vSphere deployments on vSphere >= 6.7U3, the [external vSphere cloud provider](https://github.com/kubernetes/cloud-provider-vsphere), along with the [vSphere CSI driver](https://github.com/kubernetes-sigs/vsphere-csi-driver) is recommended. See [Deploying a Kubernetes Cluster on vSphere with CSI and CPI](https://cloud-provider-vsphere.sigs.k8s.io/tutorials/kubernetes-on-vsphere-with-kubeadm.html) for a quick start guide.
-{{% /tab %}}
-{{% tab name="vSphere < 6.7U3" %}}
-If you are running vSphere < 6.7U3, the in-tree vSphere cloud provider is recommended. See [Running a Kubernetes Cluster on vSphere with kubeadm](https://cloud-provider-vsphere.sigs.k8s.io/tutorials/k8s-vcp-on-vsphere-with-kubeadm.html) for a quick start guide.
-{{% /tab %}}
-{{< /tabs >}}
-
-For in-depth documentation on the vSphere cloud provider, visit the [vSphere cloud provider docs site](https://cloud-provider-vsphere.sigs.k8s.io).
@@ -79,6 +79,8 @@ as an introduction to various technologies and serves as a jumping-off point.
 The following networking options are sorted alphabetically - the order does not
 imply any preferential status.

+{{% thirdparty-content %}}
+
 ### ACI

 [Cisco Application Centric Infrastructure](https://www.cisco.com/c/en/us/solutions/data-center-virtualization/application-centric-infrastructure/index.html) offers an integrated overlay and underlay SDN solution that supports containers, virtual machines, and bare metal servers. [ACI](https://www.github.com/noironetworks/aci-containers) provides container networking integration for ACI. An overview of the integration is provided [here](https://www.cisco.com/c/dam/en/us/solutions/collateral/data-center-virtualization/application-centric-infrastructure/solution-overview-c22-739493.pdf).
@@ -112,7 +114,7 @@ Additionally, the CNI can be run alongside [Calico for network policy enforcemen
 [Azure CNI](https://docs.microsoft.com/en-us/azure/virtual-network/container-networking-overview) is an [open source](https://github.com/Azure/azure-container-networking/blob/master/docs/cni.md) plugin that integrates Kubernetes Pods with an Azure Virtual Network (also known as VNet) providing network performance at par with VMs. Pods can connect to peered VNet and to on-premises over Express Route or site-to-site VPN and are also directly reachable from these networks. Pods can access Azure services, such as storage and SQL, that are protected by Service Endpoints or Private Link. You can use VNet security policies and routing to filter Pod traffic. The plugin assigns VNet IPs to Pods by utilizing a pool of secondary IPs pre-configured on the Network Interface of a Kubernetes node.

 Azure CNI is available natively in the [Azure Kubernetes Service (AKS)] (https://docs.microsoft.com/en-us/azure/aks/configure-azure-cni).
- 
+

 ### Big Cloud Fabric from Big Switch Networks

@@ -313,5 +315,4 @@ to run, and in both cases, the network provides one IP address per pod - as is s

 The early design of the networking model and its rationale, and some future
 plans are described in more detail in the
-[networking design document](https://git.k8s.io/community/contributors/design-proposals/network/networking.md).
-
+[networking design document](https://git.k8s.io/community/contributors/design-proposals/network/networking.md).
@@ -213,6 +213,8 @@ when new keys are projected to the Pod can be as long as the kubelet sync period
 propagation delay, where the cache propagation delay depends on the chosen cache type
 (it equals to watch propagation delay, ttl of cache, or zero correspondingly).

+## Immutable ConfigMaps {#configmap-immutable}
+
 {{< feature-state for_k8s_version="v1.19" state="beta" >}}

 The Kubernetes beta feature _Immutable Secrets and ConfigMaps_ provides an option to set
@@ -224,9 +226,10 @@ data has the following advantages:
 - improves performance of your cluster by significantly reducing load on kube-apiserver, by
 closing watches for config maps marked as immutable.

-To use this feature, enable the `ImmutableEphemeralVolumes`
-[feature gate](/docs/reference/command-line-tools-reference/feature-gates/) and set
-your Secret or ConfigMap `immutable` field to `true`. For example:
+This feature is controlled by the `ImmutableEphemeralVolumes` [feature
+gate](/docs/reference/command-line-tools-reference/feature-gates/),
+which is enabled by default since v1.19. You can create an immutable
+ConfigMap by setting the `immutable` field to `true`. For example,
 ```yaml
 apiVersion: v1
 kind: ConfigMap
@@ -359,7 +359,7 @@ The only component that considers both QoS and Pod priority is
 [kubelet out-of-resource eviction](/docs/tasks/administer-cluster/out-of-resource/).
 The kubelet ranks Pods for eviction first by whether or not their usage of the
 starved resource exceeds requests, then by Priority, and then by the consumption
-of the starved compute resource relative to the Pods’ scheduling requests.
+of the starved compute resource relative to the Pods' scheduling requests.
 See
 [evicting end-user pods](/docs/tasks/administer-cluster/out-of-resource/#evicting-end-user-pods)
 for more details.
@@ -717,37 +717,6 @@ A container using a Secret as a
 Secret updates.
 {{< /note >}}

-{{< feature-state for_k8s_version="v1.19" state="beta" >}}
-
-The Kubernetes beta feature _Immutable Secrets and ConfigMaps_ provides an option to set
-individual Secrets and ConfigMaps as immutable. For clusters that extensively use Secrets
-(at least tens of thousands of unique Secret to Pod mounts), preventing changes to their
-data has the following advantages:
-
- protects you from accidental (or unwanted) updates that could cause applications outages
- improves performance of your cluster by significantly reducing load on kube-apiserver, by
-closing watches for secrets marked as immutable.
-
-To use this feature, enable the `ImmutableEphemeralVolumes`
-[feature gate](/docs/reference/command-line-tools-reference/feature-gates/) and set
-your Secret or ConfigMap `immutable` field to `true`. For example:
-```yaml
-apiVersion: v1
-kind: Secret
-metadata:
-  ...
-data:
-  ...
-immutable: true
-```
-
-{{< note >}}
-Once a Secret or ConfigMap is marked as immutable, it is _not_ possible to revert this change
-nor to mutate the contents of the `data` field. You can only delete and recreate the Secret.
-Existing Pods maintain a mount point to the deleted Secret - it is recommended to recreate
-these pods.
-{{< /note >}}
-
 ### Using Secrets as environment variables

 To use a secret in an {{< glossary_tooltip text="environment variable" term_id="container-env-variables" >}}
@@ -808,6 +777,40 @@ The output is similar to:
 1f2d1e2e67df
 ```

+## Immutable Secrets {#secret-immutable}
+
+{{< feature-state for_k8s_version="v1.19" state="beta" >}}
+
+The Kubernetes beta feature _Immutable Secrets and ConfigMaps_ provides an option to set
+individual Secrets and ConfigMaps as immutable. For clusters that extensively use Secrets
+(at least tens of thousands of unique Secret to Pod mounts), preventing changes to their
+data has the following advantages:
+
+- protects you from accidental (or unwanted) updates that could cause applications outages
+- improves performance of your cluster by significantly reducing load on kube-apiserver, by
+closing watches for secrets marked as immutable.
+
+This feature is controlled by the `ImmutableEphemeralVolumes` [feature
+gate](/docs/reference/command-line-tools-reference/feature-gates/),
+which is enabled by default since v1.19. You can create an immutable
+Secret by setting the `immutable` field to `true`. For example,
+```yaml
+apiVersion: v1
+kind: Secret
+metadata:
+  ...
+data:
+  ...
+immutable: true
+```
+
+{{< note >}}
+Once a Secret or ConfigMap is marked as immutable, it is _not_ possible to revert this change
+nor to mutate the contents of the `data` field. You can only delete and recreate the Secret.
+Existing Pods maintain a mount point to the deleted Secret - it is recommended to recreate
+these pods.
+{{< /note >}}
+
 ### Using imagePullSecrets

 The `imagePullSecrets` field is a list of references to secrets in the same namespace.
@@ -914,7 +917,7 @@ Create the Secret:
 kubectl apply -f mysecret.yaml
 ```

-Use `envFrom` to define all of the Secret’s data as container environment variables. The key from the Secret becomes the environment variable name in the Pod.
+Use `envFrom` to define all of the Secret's data as container environment variables. The key from the Secret becomes the environment variable name in the Pod.

 ```yaml
 apiVersion: v1
@@ -46,7 +46,7 @@ list of devices it manages, and the kubelet is then in charge of advertising tho
 resources to the API server as part of the kubelet node status update.
 For example, after a device plugin registers `hardware-vendor.example/foo` with the kubelet
 and reports two healthy devices on a node, the node status is updated
-to advertise that the node has 2 “Foo” devices installed and available.
+to advertise that the node has 2 "Foo" devices installed and available.

 Then, users can request devices in a
 [Container](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#container-v1-core)
@@ -21,7 +21,7 @@ a complete and working Kubernetes cluster.

 Here's the diagram of a Kubernetes cluster with all the components tied together.

-![Components of Kubernetes](/images/docs/components-of-kubernetes.png)
+![Components of Kubernetes](/images/docs/components-of-kubernetes.svg)



@@ -59,17 +59,17 @@ That's how Kubernetes comes to the rescue! Kubernetes provides you with a framew

 Kubernetes provides you with:

-* **Service discovery and load balancing**  
+* **Service discovery and load balancing**
 Kubernetes can expose a container using the DNS name or using their own IP address. If traffic to a container is high, Kubernetes is able to load balance and distribute the network traffic so that the deployment is stable.
-* **Storage orchestration**  
+* **Storage orchestration**
 Kubernetes allows you to automatically mount a storage system of your choice, such as local storages, public cloud providers, and more.
-* **Automated rollouts and rollbacks**  
+* **Automated rollouts and rollbacks**
 You can describe the desired state for your deployed containers using Kubernetes, and it can change the actual state to the desired state at a controlled rate. For example, you can automate Kubernetes to create new containers for your deployment, remove existing containers and adopt all their resources to the new container.
-* **Automatic bin packing**  
+* **Automatic bin packing**
 You provide Kubernetes with a cluster of nodes that it can use to run containerized tasks. You tell Kubernetes how much CPU and memory (RAM) each container needs. Kubernetes can fit containers onto your nodes to make the best use of your resources.
-* **Self-healing**  
-Kubernetes restarts containers that fail, replaces containers, kills containers that don’t respond to your user-defined health check, and doesn’t advertise them to clients until they are ready to serve.
-* **Secret and configuration management**  
+* **Self-healing**
+Kubernetes restarts containers that fail, replaces containers, kills containers that don't respond to your user-defined health check, and doesn't advertise them to clients until they are ready to serve.
+* **Secret and configuration management**
 Kubernetes lets you store and manage sensitive information, such as passwords, OAuth tokens, and SSH keys. You can deploy and update secrets and application configuration without rebuilding your container images, and without exposing secrets in your stack configuration.

 ## What Kubernetes is not
@@ -84,7 +84,7 @@ Kubernetes:
 * Does not dictate logging, monitoring, or alerting solutions. It provides some integrations as proof of concept, and mechanisms to collect and export metrics.
 * Does not provide nor mandate a configuration language/system (for example, Jsonnet). It provides a declarative API that may be targeted by arbitrary forms of declarative specifications.
 * Does not provide nor adopt any comprehensive machine configuration, maintenance, management, or self-healing systems.
-* Additionally, Kubernetes is not a mere orchestration system. In fact, it eliminates the need for orchestration. The technical definition of orchestration is execution of a defined workflow: first do A, then B, then C. In contrast, Kubernetes comprises a set of independent, composable control processes that continuously drive the current state towards the provided desired state. It shouldn’t matter how you get from A to C. Centralized control is also not required. This results in a system that is easier to use and more powerful, robust, resilient, and extensible.
+* Additionally, Kubernetes is not a mere orchestration system. In fact, it eliminates the need for orchestration. The technical definition of orchestration is execution of a defined workflow: first do A, then B, then C. In contrast, Kubernetes comprises a set of independent, composable control processes that continuously drive the current state towards the provided desired state. It shouldn't matter how you get from A to C. Centralized control is also not required. This results in a system that is easier to use and more powerful, robust, resilient, and extensible.



@@ -69,7 +69,7 @@ If the prefix is omitted, the annotation Key is presumed to be private to the us

 The `kubernetes.io/` and `k8s.io/` prefixes are reserved for Kubernetes core components.

-For example, here’s the configuration file for a Pod that has the annotation `imageregistry: https://hub.docker.com/` :
+For example, here's the configuration file for a Pod that has the annotation `imageregistry: https://hub.docker.com/` :

 ```yaml

@@ -85,7 +85,7 @@ spec:
    image: nginx:1.14.2
    ports:
    - containerPort: 80
-          
+
 ```


@@ -54,7 +54,7 @@ The `kubernetes.io/` and `k8s.io/` prefixes are reserved for Kubernetes core com

 Valid label values must be 63 characters or less and must be empty or begin and end with an alphanumeric character (`[a-z0-9A-Z]`) with dashes (`-`), underscores (`_`), dots (`.`), and alphanumerics between.

-For example, here’s the configuration file for a Pod that has two labels `environment: production` and `app: nginx` :
+For example, here's the configuration file for a Pod that has two labels `environment: production` and `app: nginx` :

 ```yaml

@@ -54,7 +54,7 @@ Some resource types require their names to be able to be safely encoded as a
 path segment. In other words, the name may not be "." or ".." and the name may
 not contain "/" or "%".

-Here’s an example manifest for a Pod named `nginx-demo`.
+Here's an example manifest for a Pod named `nginx-demo`.

 ```yaml
 apiVersion: v1
@@ -0,0 +1,23 @@
+---
+title: Eviction Policy
+content_template: templates/concept
+weight: 60
+---
+
+<!-- overview -->
+
+This page is an overview of Kubernetes' policy for eviction.
+
+<!-- body -->
+
+## Eviction Policy
+
+The {{< glossary_tooltip text="Kubelet" term_id="kubelet" >}} can proactively monitor for and prevent total starvation of a
+compute resource. In those cases, the `kubelet` can reclaim the starved
+resource by proactively failing one or more Pods. When the `kubelet` fails
+a Pod, it terminates all of its containers and transitions its `PodPhase` to `Failed`.
+If the evicted Pod is managed by a Deployment, the Deployment will create another Pod
+to be scheduled by Kubernetes.
+
+## {{% heading "whatsnext" %}}
+- Read [Configure out of resource handling](/docs/tasks/administer-cluster/out-of-resource/) to learn more about eviction signals, thresholds, and handling.
@@ -33,7 +33,7 @@ kube-scheduler is designed so that, if you want and need to, you can
 write your own scheduling component and use that instead.

 For every newly created pod or other unscheduled pods, kube-scheduler
-selects an optimal node for them to run on.  However, every container in
+selects an optimal node for them to run on. However, every container in
 pods has different requirements for resources and every pod also has
 different requirements. Therefore, existing nodes need to be filtered
 according to the specific scheduling requirements.
@@ -77,12 +77,9 @@ one of these at random.
 There are two supported ways to configure the filtering and scoring behavior
 of the scheduler:

-1. [Scheduling Policies](/docs/reference/scheduling/policies) allow you to
-  configure _Predicates_ for filtering and _Priorities_ for scoring.
-1. [Scheduling Profiles](/docs/reference/scheduling/config/#profiles) allow you
-  to configure Plugins that implement different scheduling stages, including:
-  `QueueSort`, `Filter`, `Score`, `Bind`, `Reserve`, `Permit`, and others. You
-  can also configure the kube-scheduler to run different profiles.
+
+1. [Scheduling Policies](/docs/reference/scheduling/policies) allow you to configure _Predicates_ for filtering and _Priorities_ for scoring.
+1. [Scheduling Profiles](/docs/reference/scheduling/profiles) allow you to configure Plugins that implement different scheduling stages, including: `QueueSort`, `Filter`, `Score`, `Bind`, `Reserve`, `Permit`, and others. You can also configure the kube-scheduler to run different profiles.


 ## {{% heading "whatsnext" %}}
@@ -3,7 +3,7 @@ reviewers:
 - bsalamat
 title: Scheduler Performance Tuning
 content_type: concept
-weight: 70
+weight: 80
 ---

 <!-- overview -->
@@ -48,10 +48,13 @@ To change the value, edit the kube-scheduler configuration file (this is likely
 to be `/etc/kubernetes/config/kube-scheduler.yaml`), then restart the scheduler.

 After you have made this change, you can run
+
 ```bash
 kubectl get componentstatuses
 ```
+
 to verify that the kube-scheduler component is healthy. The output is similar to:
+
 ```
 NAME                 STATUS    MESSAGE             ERROR
 controller-manager   Healthy   ok
@@ -3,7 +3,7 @@ reviewers:
 - ahg-g
 title: Scheduling Framework
 content_type: concept
-weight: 60
+weight: 70
 ---

 <!-- overview -->
@@ -62,7 +62,7 @@ tolerations:
  effect: "NoSchedule"
 ```

-Here’s an example of a pod that uses tolerations:
+Here's an example of a pod that uses tolerations:

 {{< codenew file="pods/pod-with-toleration.yaml" >}}

@@ -317,6 +317,6 @@ restrict privileged permissions is lessened when the workload is isolated from t
 kernel. This allows for workloads requiring heightened permissions to still be isolated.

 Additionally, the protection of sandboxed workloads is highly dependent on the method of
-sandboxing. As such, no single ‘recommended’ policy is recommended for all sandboxed workloads.
+sandboxing. As such, no single recommended policy is recommended for all sandboxed workloads.


@@ -15,7 +15,7 @@ weight: 30

 Now that you have a continuously running, replicated application you can expose it on a network. Before discussing the Kubernetes approach to networking, it is worthwhile to contrast it with the "normal" way networking works with Docker.

-By default, Docker uses host-private networking, so containers can talk to other containers only if they are on the same machine. In order for Docker containers to communicate across nodes, there must be allocated ports on the machine’s own IP address, which are then forwarded or proxied to the containers. This obviously means that containers must either coordinate which ports they use very carefully or ports must be allocated dynamically.
+By default, Docker uses host-private networking, so containers can talk to other containers only if they are on the same machine. In order for Docker containers to communicate across nodes, there must be allocated ports on the machine's own IP address, which are then forwarded or proxied to the containers. This obviously means that containers must either coordinate which ports they use very carefully or ports must be allocated dynamically.

 Coordinating port allocations across multiple developers or teams that provide containers is very difficult to do at scale, and exposes users to cluster-level issues outside of their control. Kubernetes assumes that pods can communicate with other pods, regardless of which host they land on. Kubernetes gives every pod its own cluster-private IP address, so you do not need to explicitly create links between pods or map container ports to host ports. This means that containers within a Pod can all reach each other's ports on localhost, and all pods in a cluster can see each other without NAT. The rest of this document elaborates on how you can run reliable services on such a networking model.

@@ -72,12 +72,12 @@ In general a pod has the following DNS resolution:

 `pod-ip-address.my-namespace.pod.cluster-domain.example`.

-For example, if a pod in the `default` namespace has the IP address 172.17.0.3, 
+For example, if a pod in the `default` namespace has the IP address 172.17.0.3,
 and the domain name for your cluster is `cluster.local`, then the Pod has a DNS name:

 `172-17-0-3.default.pod.cluster.local`.

-Any pods created by a Deployment or DaemonSet exposed by a Service have the 
+Any pods created by a Deployment or DaemonSet exposed by a Service have the
 following DNS resolution available:

 `pod-ip-address.deployment-name.my-namespace.svc.cluster-domain.example`.
@@ -209,7 +209,7 @@ following pod-specific DNS policies. These policies are specified in the

 {{< note >}}
 "Default" is not the default DNS policy. If `dnsPolicy` is not
-explicitly specified, then “ClusterFirst” is used.
+explicitly specified, then "ClusterFirst" is used.
 {{< /note >}}


@@ -47,6 +47,7 @@ To enable IPv4/IPv6 dual-stack, enable the `IPv6DualStack` [feature gate](/docs/

   * kube-apiserver:
      * `--feature-gates="IPv6DualStack=true"`
+      * `--service-cluster-ip-range=<IPv4 CIDR>,<IPv6 CIDR>`
   * kube-controller-manager:
      * `--feature-gates="IPv6DualStack=true"`
      * `--cluster-cidr=<IPv4 CIDR>,<IPv6 CIDR>`
@@ -114,8 +114,8 @@ of the labels with the same names on the corresponding Node.
 Most often, the control plane (specifically, the endpoint slice
 {{< glossary_tooltip text="controller" term_id="controller" >}}) creates and
 manages EndpointSlice objects. There are a variety of other use cases for
-EndpointSlices, such as service mesh implementations, that could result in othe
-rentities or controllers managing additional sets of EndpointSlices.
+EndpointSlices, such as service mesh implementations, that could result in other
+entities or controllers managing additional sets of EndpointSlices.

 To ensure that multiple entities can manage EndpointSlices without interfering
 with each other, Kubernetes defines the
@@ -493,7 +493,6 @@ You can achieve the same outcome by invoking `kubectl replace -f` on a modified

 Techniques for spreading traffic across failure domains differs between cloud providers.
 Please check the documentation of the relevant [Ingress controller](/docs/concepts/services-networking/ingress-controllers) for details.
-for details on deploying Ingress in a federated cluster.

 ## Alternatives

@@ -9,11 +9,18 @@ weight: 50
 ---

 <!-- overview -->
-A network policy is a specification of how groups of {{< glossary_tooltip text="pods" term_id="pod">}} are allowed to communicate with each other and other network endpoints.

-NetworkPolicy resources use {{< glossary_tooltip text="labels" term_id="label">}} to select pods and define rules which specify what traffic is allowed to the selected pods.
+If you want to control traffic flow at the IP address or port level (OSI layer 3 or 4), then you might consider using Kubernetes NetworkPolicies for particular applications in your cluster.  NetworkPolicies are an application-centric construct which allow you to specify how a {{< glossary_tooltip text="pod" term_id="pod">}} is allowed to communicate with various network "entities" (we use the word "entity" here to avoid overloading the more common terms such as "endpoints" and "services", which have specific Kubernetes connotations) over the network. 

+The entities that a Pod can communicate with are identified through a combination of the following 3 identifiers:

+1. Other pods that are allowed (exception: a pod cannot block access to itself)
+2. Namespaces that are allowed 
+3. IP blocks (exception: traffic to and from the node where a Pod is running is always allowed, regardless of the IP address of the Pod or the node)
+
+When defining a pod- or namespace- based NetworkPolicy, you use a {{< glossary_tooltip text="selector" term_id="selector">}} to specify what traffic is allowed to and from the Pod(s) that match the selector.
+
+Meanwhile, when IP based NetworkPolicies are created, we define policies based on IP blocks (CIDR ranges).

 <!-- body -->
 ## Prerequisites
@@ -94,7 +101,7 @@ __egress__: Each NetworkPolicy may include a list of allowed `egress` rules.  Ea
 So, the example NetworkPolicy:

 1. isolates "role=db" pods in the "default" namespace for both ingress and egress traffic (if they weren't already isolated)
-2. (Ingress rules) allows connections to all pods in the “default” namespace with the label “role=db” on TCP port 6379 from:
+2. (Ingress rules) allows connections to all pods in the "default" namespace with the label "role=db" on TCP port 6379 from:

   * any pod in the "default" namespace with the label "role=frontend"
   * any pod in a namespace with the label "project=myproject"
@@ -212,8 +219,21 @@ When the feature gate is enabled, you can set the `protocol` field of a NetworkP
 You must be using a {{< glossary_tooltip text="CNI" term_id="cni" >}} plugin that supports SCTP protocol NetworkPolicies.
 {{< /note >}}

+# What you CAN'T do with network policy's (at least, not yet)

+As of Kubernetes 1.20, the following functionality does not exist in the NetworkPolicy API, but you might be able to implement workarounds using Operating System components (such as SELinux, OpenVSwitch, IPTables, and so on) or Layer 7 technologies (Ingress controllers, Service Mesh implementations) or admission controllers.  In case you are new to network security in Kubernetes, its worth noting that the following User Stories cannot (yet) be implemented using the NetworkPolicy API.  Some (but not all) of these user stories are actively being discussed for future releases of the NetworkPolicy API.

+- Forcing internal cluster traffic to go through a common gateway (this might be best served with a service mesh or other proxy).
+- Anything TLS related (use a service mesh or ingress controller for this).
+- Node specific policies (you can use CIDR notation for these, but you cannot target nodes by their Kubernetes identities specifically).
+- Targeting of namespaces or services by name (you can, however, target pods or namespaces by their{{< glossary_tooltip text="labels" term_id="label" >}}, which is often a viable workaround).
+- Creation or management of "Policy requests" that are fulfilled by a third party.
+- Default policies which are applied to all namespaces or pods (there are some third party Kubernetes distributions and projects which can do this).
+- Advanced policy querying and reachability tooling.
+- The ability to target ranges of Ports in a single policy declaration.
+- The ability to log network security events (for example connections that are blocked or accepted).
+- The ability to explicitly deny policies (currently the model for NetworkPolicies are deny by default, with only the ability to add allow rules).
+- The ability to prevent loopback or incoming host traffic (Pods cannot currently block localhost access, nor do they have the ability to block access from their resident node).

 ## {{% heading "whatsnext" %}}

@@ -221,5 +241,3 @@ You must be using a {{< glossary_tooltip text="CNI" term_id="cni" >}} plugin tha
 - See the [Declare Network Policy](/docs/tasks/administer-cluster/declare-network-policy/)
  walkthrough for further examples.
 - See more [recipes](https://github.com/ahmetb/kubernetes-network-policy-recipes) for common scenarios enabled by the NetworkPolicy resource.
-
-
@@ -33,8 +33,8 @@ Each Pod gets its own IP address, however in a Deployment, the set of Pods
 running in one moment in time could be different from
 the set of Pods running that application a moment later.

-This leads to a problem: if some set of Pods (call them “backends”) provides
-functionality to other Pods (call them “frontends”) inside your cluster,
+This leads to a problem: if some set of Pods (call them "backends") provides
+functionality to other Pods (call them "frontends") inside your cluster,
 how do the frontends find out and keep track of which IP address to connect
 to, so that the frontend can use the backend part of the workload?

@@ -91,7 +91,7 @@ spec:
      targetPort: 9376
 ```

-This specification creates a new Service object named “my-service”, which
+This specification creates a new Service object named "my-service", which
 targets TCP port 9376 on any Pod with the `app=MyApp` label.

 Kubernetes assigns this Service an IP address (sometimes called the "cluster IP"),
@@ -100,7 +100,7 @@ which is used by the Service proxies

 The controller for the Service selector continuously scans for Pods that
 match its selector, and then POSTs any updates to an Endpoint object
-also named “my-service”.
+also named "my-service".

 {{< note >}}
 A Service can map _any_ incoming `port` to a `targetPort`. By default and
@@ -316,7 +316,7 @@ falls back to running in iptables proxy mode.

 ![Services overview diagram for IPVS proxy](/images/docs/services-ipvs-overview.svg)

-In these proxy models, the traffic bound for the Service’s IP:Port is
+In these proxy models, the traffic bound for the Service's IP:Port is
 proxied to an appropriate backend without the clients knowing anything
 about Kubernetes or Services or Pods.

@@ -444,7 +444,7 @@ You can find more information about `ExternalName` resolution in
 ## Headless Services

 Sometimes you don't need load-balancing and a single Service IP.  In
-this case, you can create what are termed “headless” Services, by explicitly
+this case, you can create what are termed "headless" Services, by explicitly
 specifying `"None"` for the cluster IP (`.spec.clusterIP`).

 You can use a headless Service to interface with other service discovery mechanisms,
@@ -682,7 +682,7 @@ metadata:
 ```yaml
 [...]
 metadata:
-  annotations:  
+  annotations:
    service.kubernetes.io/qcloud-loadbalancer-internal-subnetid: subnet-xxxxx
 [...]
 ```
@@ -691,7 +691,7 @@ metadata:
 ```yaml
 [...]
 metadata:
-  annotations:  
+  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type: "intranet"
 [...]
 ```
@@ -950,25 +950,25 @@ There are other annotations for managing Cloud Load Balancers on TKE as shown be

        # ID of an existing load balancer
        service.kubernetes.io/tke-existed-lbid：lb-6swtxxxx
-        
+
        # Custom parameters for the load balancer (LB), does not support modification of LB type yet
        service.kubernetes.io/service.extensiveParameters: ""
-        
-        # Custom parameters for the LB listener 
+
+        # Custom parameters for the LB listener
        service.kubernetes.io/service.listenerParameters: ""
-        
+
        # Specifies the type of Load balancer;
        # valid values: classic (Classic Cloud Load Balancer) or application (Application Cloud Load Balancer)
        service.kubernetes.io/loadbalance-type: xxxxx

-        # Specifies the public network bandwidth billing method; 
+        # Specifies the public network bandwidth billing method;
        # valid values: TRAFFIC_POSTPAID_BY_HOUR(bill-by-traffic) and BANDWIDTH_POSTPAID_BY_HOUR (bill-by-bandwidth).
        service.kubernetes.io/qcloud-loadbalancer-internet-charge-type: xxxxxx

        # Specifies the bandwidth value (value range: [1,2000] Mbps).
        service.kubernetes.io/qcloud-loadbalancer-internet-max-bandwidth-out: "10"

-        # When this annotation is set，the loadbalancers will only register nodes 
+        # When this annotation is set，the loadbalancers will only register nodes
        # with pod running on it, otherwise all nodes will be registered.
        service.kubernetes.io/local-svc-only-bind-node-with-pod: true
 ```
@@ -1117,7 +1117,7 @@ connections on it.

 When a client connects to the Service's virtual IP address, the iptables
 rule kicks in, and redirects the packets to the proxy's own port.
-The “Service proxy” chooses a backend, and starts proxying traffic from the client to the backend.
+The "Service proxy" chooses a backend, and starts proxying traffic from the client to the backend.

 This means that Service owners can choose any port they want without risk of
 collision.  Clients can simply connect to an IP and port, without being aware
@@ -1178,7 +1178,7 @@ to expose HTTP / HTTPS Services.

 ### PROXY protocol

-If your cloud provider supports it (eg, [AWS](/docs/concepts/cluster-administration/cloud-providers/#aws)),
+If your cloud provider supports it,
 you can use a Service in LoadBalancer mode to configure a load balancer outside
 of Kubernetes itself, that will forward connections prefixed with
 [PROXY protocol](https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt).
@@ -33,7 +33,7 @@ from the API group `storage.k8s.io`. A cluster administrator can define as many
 that provisioner when provisioning.
 A cluster administrator can define and expose multiple flavors of storage (from
 the same or different storage systems) within a cluster, each with a custom set
-of parameters. This design also ensures that end users don’t have to worry
+of parameters. This design also ensures that end users don't have to worry
 about the complexity and nuances of how storage is provisioned, but still
 have the ability to select from multiple storage options.

@@ -85,8 +85,8 @@ is deprecated since v1.6. Users now can and should instead use the
 this field must match the name of a `StorageClass` configured by the
 administrator (see [below](#enabling-dynamic-provisioning)).

-To select the “fast” storage class, for example, a user would create the
-following `PersistentVolumeClaim`:
+To select the "fast" storage class, for example, a user would create the
+following PersistentVolumeClaim:

 ```yaml
 apiVersion: v1
@@ -215,8 +215,7 @@ See the [CephFS example](https://github.com/kubernetes/examples/tree/{{< param "
 ### cinder {#cinder}

 {{< note >}}
-Prerequisite: Kubernetes with OpenStack Cloud Provider configured. For cloudprovider
-configuration please refer [cloud provider openstack](/docs/concepts/cluster-administration/cloud-providers/#openstack).
+Prerequisite: Kubernetes with OpenStack Cloud Provider configured.
 {{< /note >}}

 `cinder` is used to mount OpenStack Cinder Volume into your Pod.
@@ -296,7 +296,7 @@ curl -X DELETE  'localhost:8080/apis/apps/v1/namespaces/default/replicasets/fron
 Once the original is deleted, you can create a new ReplicaSet to replace it.  As long
 as the old and new `.spec.selector` are the same, then the new one will adopt the old Pods.
 However, it will not make any effort to make existing Pods match a new, different pod template.
-To update Pods to a new spec in a controlled way, use a 
+To update Pods to a new spec in a controlled way, use a
 [Deployment](/docs/concepts/workloads/controllers/deployment/#creating-a-deployment), as ReplicaSets do not support a rolling update directly.

 ### Isolating Pods from a ReplicaSet
@@ -341,7 +341,7 @@ kubectl autoscale rs frontend --max=10 --min=3 --cpu-percent=50
 [`Deployment`](/docs/concepts/workloads/controllers/deployment/) is an object which can own ReplicaSets and update
 them and their Pods via declarative, server-side rolling updates.
 While ReplicaSets can be used independently, today they're  mainly used by Deployments as a mechanism to orchestrate Pod
-creation, deletion and updates. When you use Deployments you don’t have to worry about managing the ReplicaSets that
+creation, deletion and updates. When you use Deployments you don't have to worry about managing the ReplicaSets that
 they create. Deployments own and manage their ReplicaSets.
 As such, it is recommended to use Deployments when you want ReplicaSets.

@@ -254,8 +254,8 @@ API object can be found at:
 ### ReplicaSet

 [`ReplicaSet`](/docs/concepts/workloads/controllers/replicaset/) is the next-generation ReplicationController that supports the new [set-based label selector](/docs/concepts/overview/working-with-objects/labels/#set-based-requirement).
-It’s mainly used by [`Deployment`](/docs/concepts/workloads/controllers/deployment/) as a mechanism to orchestrate pod creation, deletion and updates.
-Note that we recommend using Deployments instead of directly using Replica Sets, unless you require custom update orchestration or don’t require updates at all.
+It's mainly used by [Deployment](/docs/concepts/workloads/controllers/deployment/) as a mechanism to orchestrate pod creation, deletion and updates.
+Note that we recommend using Deployments instead of directly using Replica Sets, unless you require custom update orchestration or don't require updates at all.


 ### Deployment (Recommended)
@@ -45,7 +45,7 @@ higher-level abstraction, called a
 managing the relatively disposable Pod instances.

 A given Pod (as defined by a UID) is never "rescheduled" to a different node; instead,
-that Pod can be replaced by a new, near-identical Pod, with even the same name i
+that Pod can be replaced by a new, near-identical Pod, with even the same name if
 desired, but with a different UID.

 When something is said to have the same lifetime as a Pod, such as a
@@ -107,7 +107,7 @@ Each state has a specific meaning:

 ### `Waiting` {#container-state-waiting}

-If a container is not in either the `Running` or `Terminated` state, it `Waiting`.
+If a container is not in either the `Running` or `Terminated` state, it is `Waiting`.
 A container in the `Waiting` state is still running the operations it requires in
 order to complete start up: for example, pulling the container image from a container
 image registry, or applying {{< glossary_tooltip text="Secret" term_id="secret" >}}
@@ -118,7 +118,7 @@ a Reason field to summarize why the container is in that state.
 ### `Running` {#container-state-running}

 The `Running` status indicates that a container is executing without issues. If there
-was a `postStart` hook configured, it has already executed and executed. When you use
+was a `postStart` hook configured, it has already executed and finished. When you use
 `kubectl` to query a Pod with a container that is `Running`, you also see information
 about when the container entered the `Running` state.

@@ -96,7 +96,7 @@ If we want an incoming Pod to be evenly spread with existing Pods across zones,

 {{< codenew file="pods/topology-spread-constraints/one-constraint.yaml" >}}

-`topologyKey: zone` implies the even distribution will only be applied to the nodes which have label pair "zone:&lt;any value&gt;" present. `whenUnsatisfiable: DoNotSchedule` tells the scheduler to let it stay pending if the incoming Pod can’t satisfy the constraint.
+`topologyKey: zone` implies the even distribution will only be applied to the nodes which have label pair "zone:&lt;any value&gt;" present. `whenUnsatisfiable: DoNotSchedule` tells the scheduler to let it stay pending if the incoming Pod can't satisfy the constraint.

 If the scheduler placed this incoming Pod into "zoneA", the Pods distribution would become [3, 1], hence the actual skew is 2 (3 - 1) - which violates `maxSkew: 1`. In this example, the incoming Pod can only be placed onto "zoneB":

@@ -114,7 +114,7 @@ You can tweak the Pod spec to meet various kinds of requirements:

 - Change `maxSkew` to a bigger value like "2" so that the incoming Pod can be placed onto "zoneA" as well.
 - Change `topologyKey` to "node" so as to distribute the Pods evenly across nodes instead of zones. In the above example, if `maxSkew` remains "1", the incoming Pod can only be placed onto "node4".
- Change `whenUnsatisfiable: DoNotSchedule` to `whenUnsatisfiable: ScheduleAnyway` to ensure the incoming Pod to be always schedulable (suppose other scheduling APIs are satisfied). However, it’s preferred to be placed onto the topology domain which has fewer matching Pods. (Be aware that this preferability is jointly normalized with other internal scheduling priorities like resource usage ratio, etc.)
+- Change `whenUnsatisfiable: DoNotSchedule` to `whenUnsatisfiable: ScheduleAnyway` to ensure the incoming Pod to be always schedulable (suppose other scheduling APIs are satisfied). However, it's preferred to be placed onto the topology domain which has fewer matching Pods. (Be aware that this preferability is jointly normalized with other internal scheduling priorities like resource usage ratio, etc.)

 ### Example: Multiple TopologySpreadConstraints

@@ -163,7 +163,7 @@ There are some implicit conventions worth noting here:
  1. the Pods located on those nodes do not impact `maxSkew` calculation - in the above example, suppose "node1" does not have label "zone", then the 2 Pods will be disregarded, hence the incoming Pod will be scheduled into "zoneA".
  2. the incoming Pod has no chances to be scheduled onto this kind of nodes - in the above example, suppose a "node5" carrying label `{zone-typo: zoneC}` joins the cluster, it will be bypassed due to the absence of label key "zone".

- Be aware of what will happen if the incoming Pod’s `topologySpreadConstraints[*].labelSelector` doesn’t match its own labels. In the above example, if we remove the incoming Pod’s labels, it can still be placed onto "zoneB" since the constraints are still satisfied. However, after the placement, the degree of imbalance of the cluster remains unchanged - it’s still zoneA having 2 Pods which hold label {foo:bar}, and zoneB having 1 Pod which holds label {foo:bar}. So if this is not what you expect, we recommend the workload’s `topologySpreadConstraints[*].labelSelector` to match its own labels.
+- Be aware of what will happen if the incomingPod's `topologySpreadConstraints[*].labelSelector` doesn't match its own labels. In the above example, if we remove the incoming Pod's labels, it can still be placed onto "zoneB" since the constraints are still satisfied. However, after the placement, the degree of imbalance of the cluster remains unchanged - it's still zoneA having 2 Pods which hold label {foo:bar}, and zoneB having 1 Pod which holds label {foo:bar}. So if this is not what you expect, we recommend the workload's `topologySpreadConstraints[*].labelSelector` to match its own labels.

 - If the incoming Pod has `spec.nodeSelector` or `spec.affinity.nodeAffinity` defined, nodes not matching them will be bypassed.

@@ -184,8 +184,8 @@ Begin and end meetings on time.

 ### Recording meetings on Zoom

-When you’re ready to start the recording, click Record to Cloud.
+When you're ready to start the recording, click Record to Cloud.

-When you’re ready to stop recording, click Stop.
+When you're ready to stop recording, click Stop.

 The video uploads automatically to YouTube.
@@ -127,7 +127,7 @@ Monitor your cherry-pick pull request until it is merged into the release branch

 {{< note >}}
 Proposing a cherry pick requires that you have permission to set a label and a
-milestone in your pull request. If you don’t have those permissions, you will
+milestone in your pull request. If you don't have those permissions, you will
 need to work with someone who can set the label and milestone for you.
 {{< /note >}}

@@ -20,7 +20,7 @@ Each day in a week-long shift as PR Wrangler:
 - Review [open pull requests](https://github.com/kubernetes/website/pulls) for quality and adherence to the [Style](/docs/contribute/style/style-guide/) and [Content](/docs/contribute/style/content-guide/) guides.
    - Start with the smallest PRs (`size/XS`) first, and end with the largest (`size/XXL`). Review as many PRs as you can.
 - Make sure PR contributors sign the [CLA](https://github.com/kubernetes/community/blob/master/CLA.md).
-    - Use [this](https://github.com/zparnold/k8s-docs-pr-botherer) script to remind contributors that haven’t signed the CLA to do so.
+    - Use [this](https://github.com/zparnold/k8s-docs-pr-botherer) script to remind contributors that haven't signed the CLA to do so.
 - Provide feedback on changes and ask for technical reviews from members of other SIGs.
    - Provide inline suggestions on the PR for the proposed content changes.
    - If you need to verify content, comment on the PR and request more details.
@@ -90,19 +90,39 @@ Renders to:

 ## Glossary

+There are two glossary tooltips.
+
 You can reference glossary terms with an inclusion that automatically updates and replaces content with the relevant links from [our glossary](/docs/reference/glossary/). When the term is moused-over by someone
 using the online documentation, the glossary entry displays a tooltip.

+As well as inclusions with tooltips, you can reuse the definitions from the glossary in
+page content.
+
 The raw data for glossary terms is stored at [https://github.com/kubernetes/website/tree/master/content/en/docs/reference/glossary](https://github.com/kubernetes/website/tree/master/content/en/docs/reference/glossary), with a content file for each glossary term.

-### Glossary Demo
+### Glossary demo

 For example, the following include within the markdown renders to {{< glossary_tooltip text="cluster" term_id="cluster" >}} with a tooltip:

-```liquid
+```
 {{</* glossary_tooltip text="cluster" term_id="cluster" */>}}
 ```

+Here's a short glossary definition:
+
+```
+{{</* glossary_definition prepend="A cluster is" term_id="cluster" length="short" */>}}
+```
+which renders as:
+{{< glossary_definition prepend="A cluster is" term_id="cluster" length="short" >}}
+
+You can also include a full definition:
+```
+{{</* glossary_definition term_id="cluster" length="all" */>}}
+```
+which renders as:
+{{< glossary_definition term_id="cluster" length="all" >}}
+
 ## Table captions

 You can make tables more accessible to screen readers by adding a table caption. To add a [caption](https://www.w3schools.com/tags/tag_caption.asp) to a table, enclose the table with a `table` shortcode and specify the caption with the `caption` parameter.
@@ -447,7 +447,7 @@ Use three hyphens (`---`) to create a horizontal rule. Use horizontal rules for
 {{< table caption = "Do and Don't - Links" >}}
 Do | Don't
 :--| :-----
-Write hyperlinks that give you context for the content they link to. For example: Certain ports are open on your machines. See <a href="#check-required-ports">Check required ports</a> for more details. | Use ambiguous terms such as “click here”. For example: Certain ports are open on your machines. See <a href="#check-required-ports">here</a> for more details.
+Write hyperlinks that give you context for the content they link to. For example: Certain ports are open on your machines. See <a href="#check-required-ports">Check required ports</a> for more details. | Use ambiguous terms such as "click here". For example: Certain ports are open on your machines. See <a href="#check-required-ports">here</a> for more details.
 Write Markdown-style links: `[link text](URL)`. For example: `[Hugo shortcodes](/docs/contribute/style/hugo-shortcodes/#table-captions)` and the output is [Hugo shortcodes](/docs/contribute/style/hugo-shortcodes/#table-captions). | Write HTML-style links: `<a href="/media/examples/link-element-example.css" target="_blank">Visit our tutorial!</a>`, or create links that open in new tabs or windows. For example: `[example website](https://example.com){target="_blank"}`
 {{< /table >}}

@@ -53,7 +53,7 @@ cards:
  button_path: /docs/reference
 - name: contribute
  title: Contribute to the docs
-  description: Anyone can contribute, whether you’re new to the project or you’ve been around a long time.
+  description: Anyone can contribute, whether you're new to the project or you've been around a long time.
  button: Contribute to the docs
  button_path: /docs/contribute
 - name: release-notes
@@ -64,4 +64,4 @@ cards:
 - name: about
  title: About the documentation
  description: This website contains documentation for the current and previous 4 versions of Kubernetes.
---
+---
@@ -1,30 +1,12 @@
 ---
-title: Supported Versions of the Kubernetes Documentation
-content_type: concept
+title: Available Documentation Versions
+content_type: custom
+layout: supported-versions
 card:
  name: about
  weight: 10
-  title: Supported Versions of the Documentation
+  title: Available Documentation Versions
 ---

-<!-- overview -->
-
 This website contains documentation for the current version of Kubernetes
 and the four previous versions of Kubernetes.
-
-
-
-<!-- body -->
-
-## Current version
-
-The current version is
-[{{< param "version" >}}](/).
-
-## Previous versions
-
-{{< versions-other >}}
-
-
-
-
@@ -202,8 +202,6 @@ is recommended instead.
 This admission controller mitigates the problem where the API server gets flooded by
 event requests. The cluster admin can specify event rate limits by:

- * Ensuring that `eventratelimit.admission.k8s.io/v1alpha1=true` is included in the
-   `--runtime-config` flag for the API server;
 * Enabling the `EventRateLimit` admission controller;
 * Referencing an `EventRateLimit` configuration file from the file provided to the API
   server's command line flag `--admission-control-config-file`:
@@ -30,10 +30,10 @@ In this regard, _Kubernetes does not have objects which represent normal user
 accounts._ Normal users cannot be added to a cluster through an API call.

 Even though normal user cannot be added via an API call, but any user that
-presents a valid certificate signed by the cluster’s certificate authority
+presents a valid certificate signed by the cluster's certificate authority
 (CA) is considered authenticated. In this configuration, Kubernetes determines
-the username from the common name field in the ‘subject’ of the cert (e.g.,
-“/CN=bob”). From there, the role based access control (RBAC) sub-system would
+the username from the common name field in the 'subject' of the cert (e.g.,
+"/CN=bob"). From there, the role based access control (RBAC) sub-system would
 determine whether the user is authorized to perform a specific operation on a
 resource. For more details, refer to the normal users topic in
 [certificate request](/docs/reference/access-authn-authz/certificate-signing-requests/#normal-user)
@@ -329,7 +329,7 @@ Kubernetes does not provide an OpenID Connect Identity Provider.
 You can use an existing public OpenID Connect Identity Provider (such as Google, or
 [others](https://connect2id.com/products/nimbus-oauth-openid-connect-sdk/openid-connect-providers)).
 Or, you can run your own Identity Provider, such as CoreOS [dex](https://github.com/coreos/dex),
-[Keycloak](https://github.com/keycloak/keycloak), 
+[Keycloak](https://github.com/keycloak/keycloak),
 CloudFoundry [UAA](https://github.com/cloudfoundry/uaa), or
 Tremolo Security's [OpenUnison](https://github.com/tremolosecurity/openunison).

@@ -46,7 +46,7 @@ This group and user name format match the identity created for each kubelet as p
 [kubelet TLS bootstrapping](/docs/reference/command-line-tools-reference/kubelet-tls-bootstrapping/).

 The value of `<nodeName>` **must** match precisely the name of the node as registered by the kubelet. By default, this is the host name as provided by `hostname`, or overridden via the [kubelet option](/docs/reference/command-line-tools-reference/kubelet/) `--hostname-override`. However, when using the `--cloud-provider` kubelet option, the specific hostname may be determined by the cloud provider, ignoring the local `hostname` and the `--hostname-override` option. 
-For specifics about how the kubelet determines the hostname, as well as cloud provider overrides, see the [kubelet options reference](/docs/reference/command-line-tools-reference/kubelet/) and the [cloud provider details](/docs/concepts/cluster-administration/cloud-providers/).
+For specifics about how the kubelet determines the hostname, see the [kubelet options reference](/docs/reference/command-line-tools-reference/kubelet/).

 To enable the Node authorizer, start the apiserver with `--authorization-mode=Node`.

@@ -423,7 +423,7 @@ Each feature gate is designed for enabling/disabling a specific feature:
 - `CustomResourceWebhookConversion`: Enable webhook-based conversion
  on resources created from [CustomResourceDefinition](/docs/concepts/extend-kubernetes/api-extension/custom-resources/).
  troubleshoot a running Pod.
- `DisableAcceleratorUsageMetrics`: [Disable accelerator metrics collected by the kubelet](/docs/concepts/cluster-administration/monitoring.md).
+- `DisableAcceleratorUsageMetrics`: [Disable accelerator metrics collected by the kubelet](/docs/concepts/cluster-administration/system-metrics/).
 - `DevicePlugins`: Enable the [device-plugins](/docs/concepts/cluster-administration/device-plugins/)
  based resource provisioning on nodes.
 - `DefaultPodTopologySpread`: Enables the use of `PodTopologySpread` scheduling plugin to do
@@ -475,9 +475,6 @@ Each feature gate is designed for enabling/disabling a specific feature:
 - `LegacyNodeRoleBehavior`: When disabled, legacy behavior in service load balancers and node disruption will ignore the `node-role.kubernetes.io/master` label in favor of the feature-specific labels provided by `NodeDisruptionExclusion` and `ServiceNodeExclusion`.
 - `LocalStorageCapacityIsolation`: Enable the consumption of [local ephemeral storage](/docs/concepts/configuration/manage-compute-resources-container/) and also the `sizeLimit` property of an [emptyDir volume](/docs/concepts/storage/volumes/#emptydir).
 - `LocalStorageCapacityIsolationFSQuotaMonitoring`: When `LocalStorageCapacityIsolation` is enabled for [local ephemeral storage](/docs/concepts/configuration/manage-compute-resources-container/) and the backing filesystem for [emptyDir volumes](/docs/concepts/storage/volumes/#emptydir) supports project quotas and they are enabled, use project quotas to monitor [emptyDir volume](/docs/concepts/storage/volumes/#emptydir) storage consumption rather than filesystem walk for better performance and accuracy.
-  [local ephemeral storage](/docs/concepts/configuration/manage-resources-containers/) and the backing filesystem for
-  [emptyDir volumes](/docs/concepts/storage/volumes/#emptydir) supports project quotas and they are enabled, use project quotas to monitor
-  [emptyDir volume](/docs/concepts/storage/volumes/#emptydir) storage consumption rather than filesystem walk for better performance and accuracy.
 - `MountContainers`: Enable using utility containers on host as the volume mounter.
 - `MountPropagation`: Enable sharing volume mounted by one container to other containers or pods.
  For more details, please see [mount propagation](/docs/concepts/storage/volumes/#mount-propagation).
@@ -15,7 +15,7 @@ A piece of code that intercepts requests to the Kubernetes API server prior to p

 <!--more-->

-Admission controllers are configurable for the Kubernetes API server and may be “validating”, “mutating”, or
+Admission controllers are configurable for the Kubernetes API server and may be "validating", "mutating", or
 both. Any admission controller may reject the request. Mutating controllers may modify the objects they admit;
 validating controllers may not.

@@ -2,7 +2,6 @@
 title: Cloud Provider
 id: cloud-provider
 date: 2018-04-12
-full_link: /docs/concepts/cluster-administration/cloud-providers
 short_description: >
  An organization that offers a cloud computing platform.

@@ -4,7 +4,7 @@ id: configmap
 date: 2018-04-12
 full_link: /docs/concepts/configuration/configmap/
 short_description: >
-  An API object used to store non-confidential data in key-value pairs. Can be consumed as environment variables, command-line arguments, or configuraton files in a volume.
+  An API object used to store non-confidential data in key-value pairs. Can be consumed as environment variables, command-line arguments, or configuration files in a volume.

 aka: 
 tags:
@@ -6,13 +6,13 @@ full_link: /docs/reference/generated/kubelet
 short_description: >
  An agent that runs on each node in the cluster. It makes sure that containers are running in a pod.

-aka: 
+aka:
 tags:
 - fundamental
 - core-object
 ---
 An agent that runs on each {{< glossary_tooltip text="node" term_id="node" >}} in the cluster. It makes sure that {{< glossary_tooltip text="containers" term_id="container" >}} are running in a {{< glossary_tooltip text="Pod" term_id="pod" >}}.

-<!--more--> 
+<!--more-->

-The kubelet takes a set of PodSpecs that are provided through various mechanisms and ensures that the containers described in those PodSpecs are running and healthy. The kubelet doesn’t manage containers which were not created by Kubernetes.
+The kubelet takes a set of PodSpecs that are provided through various mechanisms and ensures that the containers described in those PodSpecs are running and healthy. The kubelet doesn't manage containers which were not created by Kubernetes.
@@ -6,14 +6,14 @@ full_link: /docs/concepts/architecture/nodes/
 short_description: >
  A node is a worker machine in Kubernetes.

-aka: 
+aka:
 tags:
 - fundamental
 ---
 A node is a worker machine in Kubernetes.

-<!--more--> 
+<!--more-->

 A worker node may be a VM or physical machine, depending on the cluster. It has local daemons or services necessary to run {{< glossary_tooltip text="Pods" term_id="pod" >}} and is managed by the control plane. The daemons on a node include {{< glossary_tooltip text="kubelet" term_id="kubelet" >}}, {{< glossary_tooltip text="kube-proxy" term_id="kube-proxy" >}}, and a container runtime implementing the {{< glossary_tooltip text="CRI" term_id="cri" >}} such as {{< glossary_tooltip term_id="docker" >}}.

-In early Kubernetes versions, Nodes were called “Minions”.
+In early Kubernetes versions, Nodes were called "Minions".
@@ -23,7 +23,7 @@ You can also subscribe to an RSS feed of the above using [this link](https://gro

 ## Report a Vulnerability

-We’re extremely grateful for security researchers and users that report vulnerabilities to the Kubernetes Open Source Community. All reports are thoroughly investigated by a set of community volunteers.
+We're extremely grateful for security researchers and users that report vulnerabilities to the Kubernetes Open Source Community. All reports are thoroughly investigated by a set of community volunteers.

 To make a report, submit your vulnerability to the [Kubernetes bug bounty program](https://hackerone.com/kubernetes). This allows triage and handling of the vulnerability with standardized response times.

@@ -98,4 +98,16 @@ kubectl get pods -o=jsonpath="{range .items[*]}{.metadata.name}{\"\t\"}{.status.
 ```
 {{< /note >}}

+{{< note >}}

+JSONPath regular expressions are not supported. If you want to match using regular expressions, you can use a tool such as `jq`.
+
+```shell
+# kubectl does not support regular expressions for JSONpath output
+# The following command does not work
+kubectl get pods -o jsonpath='{.items[?(@.metadata.name=~/^test$/)].metadata.name}'
+
+# The following command achieves the desired result
+kubectl get pods -o json | jq -r '.items[] | select(.metadata.name | test("test-")).spec.containers[].image'
+```
+{{< /note >}}
@@ -8,7 +8,7 @@ card:
  weight: 40
 ---

-<img src="https://raw.githubusercontent.com/kubernetes/kubeadm/master/logos/stacked/color/kubeadm-stacked-color.png" align="right" width="150px">Kubeadm is a tool built to provide `kubeadm init` and `kubeadm join` as best-practice “fast paths” for creating Kubernetes clusters.
+<img src="https://raw.githubusercontent.com/kubernetes/kubeadm/master/logos/stacked/color/kubeadm-stacked-color.png" align="right" width="150px">Kubeadm is a tool built to provide `kubeadm init` and `kubeadm join` as best-practice "fast paths" for creating Kubernetes clusters.

 kubeadm performs the actions necessary to get a minimum viable cluster up and running. By design, it cares only about bootstrapping, not about provisioning machines. Likewise, installing various nice-to-have addons, like the Kubernetes Dashboard, monitoring solutions, and cloud-specific addons, is not in scope.

@@ -40,6 +40,8 @@ The following client libraries are officially maintained by

 ## Community-maintained client libraries

+{{% thirdparty-content %}}
+
 The following Kubernetes API client libraries are provided and maintained by
 their authors, not the Kubernetes team.

@@ -303,12 +303,12 @@ Starting in Kubernetes v1.19, making an API request to a deprecated REST API end
 2. Adds a `"k8s.io/deprecated":"true"` annotation to the [audit event](/docs/tasks/debug-application-cluster/audit/) recorded for the request.
 3. Sets an `apiserver_requested_deprecated_apis` gauge metric to `1` in the `kube-apiserver` 
   process. The metric has labels for `group`, `version`, `resource`, `subresource` that can be joined
-   to the `apiserver_request_total` metric, and a `removed_version` label that indicates the 
+   to the `apiserver_request_total` metric, and a `removed_release` label that indicates the
   Kubernetes release in which the API will no longer be served. The following Prometheus query
   returns information about requests made to deprecated APIs which will be removed in v1.22:
-   
+
   ```promql
-   apiserver_requested_deprecated_apis{removed_version="1.22"} * on(group,version,resource,subresource) group_right() apiserver_request_total
+   apiserver_requested_deprecated_apis{removed_release="1.22"} * on(group,version,resource,subresource) group_right() apiserver_request_total
   ```

 ### Fields of REST resources
@@ -124,8 +124,8 @@ Same considerations apply for the service account key pair:

 | private key path             | public key path             | command                 | argument                             |
 |------------------------------|-----------------------------|-------------------------|--------------------------------------|
-|  sa.key                      |                             | kube-controller-manager | service-account-private              |
-|                              | sa.pub                      | kube-apiserver          | service-account-key                  |
+|  sa.key                      |                             | kube-controller-manager | --service-account-private-key-file   |
+|                              | sa.pub                      | kube-apiserver          | --service-account-key-file           |

 ## Configure certificates for user accounts

@@ -55,7 +55,7 @@ This brief demo guides you on how to start, use, and delete Minikube locally. Fo

 2. Now, you can interact with your cluster using kubectl. For more information, see [Interacting with Your Cluster](#interacting-with-your-cluster).

-   Let’s create a Kubernetes Deployment using an existing image named `echoserver`, which is a simple HTTP server and expose it on port 8080 using `--port`.
+   Let's create a Kubernetes Deployment using an existing image named `echoserver`, which is a simple HTTP server and expose it on port 8080 using `--port`.

   ```shell
   kubectl create deployment hello-minikube --image=k8s.gcr.io/echoserver:1.10
@@ -452,11 +452,11 @@ Host folder sharing is not implemented in the KVM driver yet.

 | Driver | OS | HostFolder | VM |
 | --- | --- | --- | --- |
-| VirtualBox | Linux | /home | /hosthome |
-| VirtualBox | macOS | /Users | /Users |
-| VirtualBox | Windows | C://Users | /c/Users |
-| VMware Fusion | macOS | /Users | /mnt/hgfs/Users |
-| Xhyve | macOS | /Users | /Users |
+| VirtualBox | Linux | `/home` | `/hosthome` |
+| VirtualBox | macOS | `/Users` | `/Users` |
+| VirtualBox | Windows | `C://Users` | `/c/Users` |
+| VMware Fusion | macOS | `/Users` | `/mnt/hgfs/Users` |
+| Xhyve | macOS | `/Users` | `/Users` |

 ## Private Container Registries

@@ -81,7 +81,7 @@ apt-get update && apt-get install -y \
 ```

 ```shell
-# Add Docker’s official GPG key:
+# Add Docker's official GPG key:
 curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -
 ```

@@ -199,7 +199,7 @@ Use the following commands to install CRI-O on your system:

 {{< note >}}
 The CRI-O major and minor versions must match the Kubernetes major and minor versions.
-For more information, see the [CRI-O compatiblity matrix](https://github.com/cri-o/cri-o).
+For more information, see the [CRI-O compatibility matrix](https://github.com/cri-o/cri-o).
 {{< /note >}}

 ### Prerequisites
@@ -381,7 +381,7 @@ apt-get update && apt-get install -y apt-transport-https ca-certificates curl so
 ```

 ```shell
-## Add Docker’s official GPG key
+## Add Docker's official GPG key
 curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -
 ```

@@ -471,7 +471,12 @@ Start-Service containerd

 ### systemd

-To use the `systemd` cgroup driver, set `plugins.cri.systemd_cgroup = true` in `/etc/containerd/config.toml`.
+To use the `systemd` cgroup driver in `/etc/containerd/config.toml` set
+
+```
+[plugins.cri]
+systemd_cgroup = true
+```
 When using kubeadm, manually configure the
 [cgroup driver for kubelet](/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#configure-cgroup-driver-used-by-kubelet-on-control-plane-node)

@@ -1,4 +0,0 @@
---
-title: On-Premises VMs
-weight: 40
---
@@ -1,116 +0,0 @@
---
-reviewers:
- thockin
-title: Cloudstack
-content_type: concept
---
-
-<!-- overview -->
-
-[CloudStack](https://cloudstack.apache.org/) is a software to build public and private clouds based on hardware virtualization principles (traditional IaaS). To deploy Kubernetes on CloudStack there are several possibilities depending on the Cloud being used and what images are made available. CloudStack also has a vagrant plugin available, hence Vagrant could be used to deploy Kubernetes either using the existing shell provisioner or using new Salt based recipes.
-
-[CoreOS](https://coreos.com) templates for CloudStack are built [nightly](https://stable.release.core-os.net/amd64-usr/current/). CloudStack operators need to [register](https://docs.cloudstack.apache.org/projects/cloudstack-administration/en/latest/templates.html) this template in their cloud before proceeding with these Kubernetes deployment instructions.
-
-This guide uses a single [Ansible playbook](https://github.com/apachecloudstack/k8s), which is completely automated and can deploy Kubernetes on a CloudStack based Cloud using CoreOS images. The playbook, creates an ssh key pair, creates a security group and associated rules and finally starts coreOS instances configured via cloud-init.
-
-<!-- body -->
-
-## Prerequisites
-
-```shell
-sudo apt-get install -y python-pip libssl-dev
-sudo pip install cs
-sudo pip install sshpubkeys
-sudo apt-get install software-properties-common
-sudo apt-add-repository ppa:ansible/ansible
-sudo apt-get update
-sudo apt-get install ansible
-```
-
-On CloudStack server you also have to install libselinux-python :
-
-```shell
-yum install libselinux-python
-```
-
-[_cs_](https://github.com/exoscale/cs) is a python module for the CloudStack API.
-
-Set your CloudStack endpoint, API keys and HTTP method used.
-
-You can define them as environment variables: `CLOUDSTACK_ENDPOINT`, `CLOUDSTACK_KEY`, `CLOUDSTACK_SECRET` and `CLOUDSTACK_METHOD`.
-
-Or create a `~/.cloudstack.ini` file:
-
-```none
-[cloudstack]
-endpoint = <your cloudstack api endpoint>
-key = <your api access key>
-secret = <your api secret key>
-method = post
-```
-
-We need to use the http POST method to pass the _large_ userdata to the coreOS instances.
-
-### Clone the playbook
-
-```shell
-git clone https://github.com/apachecloudstack/k8s
-cd kubernetes-cloudstack
-```
-
-### Create a Kubernetes cluster
-
-You simply need to run the playbook.
-
-```shell
-ansible-playbook k8s.yml
-```
-
-Some variables can be edited in the `k8s.yml` file.
-
-```none
-vars:
-  ssh_key: k8s
-  k8s_num_nodes: 2
-  k8s_security_group_name: k8s
-  k8s_node_prefix: k8s2
-  k8s_template: <templatename>
-  k8s_instance_type: <serviceofferingname>
-```
-
-This will start a Kubernetes master node and a number of compute nodes (by default 2).
-The `instance_type` and `template` are specific, edit them to specify your CloudStack cloud specific template and instance type (i.e. service offering).
-
-Check the tasks and templates in `roles/k8s` if you want to modify anything.
-
-Once the playbook as finished, it will print out the IP of the Kubernetes master:
-
-```none
-TASK: [k8s | debug msg='k8s master IP is {{ k8s_master.default_ip }}'] ********
-```
-
-SSH to it using the key that was created and using the _core_ user.
-
-```shell
-ssh -i ~/.ssh/id_rsa_k8s core@<master IP>
-```
-
-And you can list the machines in your cluster:
-
-```shell
-fleetctl list-machines
-```
-
-```none
-MACHINE        IP             METADATA
-a017c422...    <node #1 IP>   role=node
-ad13bf84...    <master IP>    role=master
-e9af8293...    <node #2 IP>   role=node
-```
-
-## Support Level
-
-IaaS Provider        | Config. Mgmt | OS     | Networking  | Docs                                              | Conforms | Support Level
-------------------- | ------------ | ------ | ----------  | ---------------------------------------------     | ---------| ----------------------------
-CloudStack           | Ansible      | CoreOS | flannel     | [docs](/docs/setup/production-environment/on-premises-vm/cloudstack/)                             |          | Community ([@Guiques](https://github.com/ltupin/))
-
@@ -1,25 +0,0 @@
---
-reviewers:
- smugcloud
-title: Kubernetes on DC/OS
-content_type: concept
---
-
-<!-- overview -->
-
-Mesosphere provides an easy option to provision Kubernetes onto [DC/OS](https://mesosphere.com/product/), offering:
-
-* Pure upstream Kubernetes
-* Single-click cluster provisioning
-* Highly available and secure by default
-* Kubernetes running alongside fast-data platforms (e.g. Akka, Cassandra, Kafka, Spark)
-
-
-
-<!-- body -->
-
-## Official Mesosphere Guide
-
-The canonical source of getting started on DC/OS is located in the [quickstart repo](https://github.com/mesosphere/dcos-kubernetes-quickstart).
-
-
@@ -1,72 +0,0 @@
---
-reviewers:
- caesarxuchao
- erictune
-title: oVirt
-content_type: concept
---
-
-<!-- overview -->
-
-oVirt is a virtual datacenter manager that delivers powerful management of multiple virtual machines on multiple hosts. Using KVM and libvirt, oVirt can be installed on Fedora, CentOS, or Red Hat Enterprise Linux hosts to set up and manage your virtual data center.
-
-
-
-<!-- body -->
-
-## oVirt Cloud Provider Deployment
-
-The oVirt cloud provider allows to easily discover and automatically add new VM instances as nodes to your Kubernetes cluster.
-At the moment there are no community-supported or pre-loaded VM images including Kubernetes but it is possible to [import] or [install] Project Atomic (or Fedora) in a VM to [generate a template]. Any other distribution that includes Kubernetes may work as well.
-
-It is mandatory to [install the ovirt-guest-agent] in the guests for the VM ip address and hostname to be reported to ovirt-engine and ultimately to Kubernetes.
-
-Once the Kubernetes template is available it is possible to start instantiating VMs that can be discovered by the cloud provider.
-
-[import]: https://ovedou.blogspot.it/2014/03/importing-glance-images-as-ovirt.html
-[install]: https://www.ovirt.org/documentation/quickstart/quickstart-guide/#create-virtual-machines
-[generate a template]: https://www.ovirt.org/documentation/quickstart/quickstart-guide/#using-templates
-[install the ovirt-guest-agent]: https://www.ovirt.org/documentation/how-to/guest-agent/install-the-guest-agent-in-fedora/
-
-## Using the oVirt Cloud Provider
-
-The oVirt Cloud Provider requires access to the oVirt REST-API to gather the proper information, the required credential should be specified in the `ovirt-cloud.conf` file:
-
-```none
-[connection]
-uri = https://localhost:8443/ovirt-engine/api
-username = admin@internal
-password = admin
-```
-
-In the same file it is possible to specify (using the `filters` section) what search query to use to identify the VMs to be reported to Kubernetes:
-
-```none
-[filters]
-# Search query used to find nodes
-vms = tag=kubernetes
-```
-
-In the above example all the VMs tagged with the `kubernetes` label will be reported as nodes to Kubernetes.
-
-The `ovirt-cloud.conf` file then must be specified in kube-controller-manager:
-
-```shell
-kube-controller-manager ... --cloud-provider=ovirt --cloud-config=/path/to/ovirt-cloud.conf ...
-```
-
-## oVirt Cloud Provider Screencast
-
-This short screencast demonstrates how the oVirt Cloud Provider can be used to dynamically add VMs to your Kubernetes cluster.
-
-[![Screencast](https://img.youtube.com/vi/JyyST4ZKne8/0.jpg)](https://www.youtube.com/watch?v=JyyST4ZKne8)
-
-## Support Level
-
-
-IaaS Provider        | Config. Mgmt | OS     | Networking  | Docs                                              | Conforms | Support Level
-------------------- | ------------ | ------ | ----------  | ---------------------------------------------     | ---------| ----------------------------
-oVirt                |              |        |             | [docs](/docs/setup/production-environment/on-premises-vm/ovirt/)                                  |          | Community ([@simon3z](https://github.com/simon3z))
-
-
-
@@ -259,10 +259,10 @@ Cluster DNS (CoreDNS) will not start up before a network is installed.**

 - Take care that your Pod network must not overlap with any of the host
  networks: you are likely to see problems if there is any overlap.
-  (If you find a collision between your network plugin’s preferred Pod
+  (If you find a collision between your network plugin's preferred Pod
  network and some of your host networks, you should think of a suitable
  CIDR block to use instead, then use that during `kubeadm init` with
-  `--pod-network-cidr` and as a replacement in your network plugin’s YAML).
+  `--pod-network-cidr` and as a replacement in your network plugin's YAML).

 - By default, `kubeadm` sets up your cluster to use and enforce use of
  [RBAC](/docs/reference/access-authn-authz/rbac/) (role based access
@@ -25,7 +25,7 @@ For information how to create a cluster with kubeadm once you have performed thi
  - Red Hat Enterprise Linux (RHEL) 7
  - Fedora 25+
  - HypriotOS v1.0.1+
-  - Container Linux (tested with 1800.6.0)
+  - Flatcar Container Linux (tested with 2512.3.0)
 * 2 GB or more of RAM per machine (any less will leave little room for your apps)
 * 2 CPUs or more
 * Full network connectivity between all machines in the cluster (public or private network is fine)
@@ -220,7 +220,7 @@ sudo systemctl enable --now kubelet
  - You can leave SELinux enabled if you know how to configure it but it may require settings that are not supported by kubeadm.

 {{% /tab %}}
-{{% tab name="Fedora CoreOS" %}}
+{{% tab name="Fedora CoreOS or Flatcar Container Linux" %}}
 Install CNI plugins (required for most pod network):

 ```bash
@@ -231,6 +231,11 @@ curl -L "https://github.com/containernetworking/plugins/releases/download/${CNI_

 Define the directory to download command files  

+{{< note >}}
+The DOWNLOAD_DIR variable must be set to a writable directory.
+If you are running Flatcar Container Linux, set DOWNLOAD_DIR=/opt/bin.
+{{< /note >}}
+
 ```bash
 DOWNLOAD_DIR=/usr/local/bin
 sudo mkdir -p $DOWNLOAD_DIR
@@ -251,7 +256,7 @@ cd $DOWNLOAD_DIR
 sudo curl -L --remote-name-all https://storage.googleapis.com/kubernetes-release/release/${RELEASE}/bin/linux/amd64/{kubeadm,kubelet,kubectl}
 sudo chmod +x {kubeadm,kubelet,kubectl}

-RELEASE_VERSION="v0.2.7"
+RELEASE_VERSION="v0.4.0"
 curl -sSL "https://raw.githubusercontent.com/kubernetes/release/${RELEASE_VERSION}/cmd/kubepkg/templates/latest/deb/kubelet/lib/systemd/system/kubelet.service" | sed "s:/usr/bin:${DOWNLOAD_DIR}:g" | sudo tee /etc/systemd/system/kubelet.service
 sudo mkdir -p /etc/systemd/system/kubelet.service.d
 curl -sSL "https://raw.githubusercontent.com/kubernetes/release/${RELEASE_VERSION}/cmd/kubepkg/templates/latest/deb/kubeadm/10-kubeadm.conf" | sed "s:/usr/bin:${DOWNLOAD_DIR}:g" | sudo tee /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
@@ -262,6 +267,12 @@ Enable and start `kubelet`:
 ```bash
 systemctl enable --now kubelet
 ```
+
+{{< note >}}
+The Flatcar Container Linux distribution mounts the `/usr` directory as a read-only filesystem.
+Before bootstrapping your cluster, you need to take additional steps to configure a writable directory.
+See the [Kubeadm Troubleshooting guide](/docs/setup/production-environment/tools/kubeadm/troubleshooting-kubeadm/#usr-mounted-read-only/) to learn how to set up a writable directory.
+{{< /note >}}
 {{% /tab %}}
 {{< /tabs >}}

@@ -407,4 +407,12 @@ be advised that this is modifying a design principle of the Linux distribution.

 This error message is shown when upgrading a Kubernetes cluster with `kubeadm` in the case of running an external etcd. This is not a critical bug and happens because older versions of kubeadm perform a version check on the external etcd cluster. You can proceed with `kubeadm upgrade apply ...`. 

-This issue is fixed as of version 1.19. 
+This issue is fixed as of version 1.19.
+
+## `kubeadm reset` unmounts `/var/lib/kubelet`
+
+If `/var/lib/kubelet` is being mounted, performing a `kubeadm reset` will effectively unmount it.
+
+To workaround the issue, re-mount the `/var/lib/kubelet` directory after performing the `kubeadm reset` operation.
+
+This is a regression introduced in kubeadm 1.15. The issue is fixed in 1.20.
@@ -150,7 +150,7 @@ so that you can change the configuration more easily.

 ## Interact with the frontend Service

-Once you’ve created a Service of type LoadBalancer, you can use this
+Once you've created a Service of type LoadBalancer, you can use this
 command to find the external IP:

 ```shell
@@ -197,18 +197,6 @@ See [Node](/docs/concepts/architecture/nodes/) for more details.

 ## Advanced Topics

-### Upgrading to a different API version
-
-When a new API version is released, you may need to upgrade a cluster to support the new API version (e.g. switching from 'v1' to 'v2' when 'v2' is launched).
-
-This is an infrequent event, but it requires careful management. There is a sequence of steps to upgrade to a new API version.
-
-   1. Turn on the new API version.
-   1. Upgrade the cluster's storage to use the new version.
-   1. Upgrade all config files. Identify users of the old API version endpoints.
-   1. Update existing objects in the storage to new version by running `cluster/update-storage-objects.sh`.
-   1. Turn off the old API version.
-
 ### Turn on or off an API version for your cluster

 Specific API versions can be turned on or off by passing `--runtime-config=api/<version>` flag while bringing up the API server. For example: to turn off v1 API, pass `--runtime-config=api/v1=false`.
--- a/Show More
+++ b/Show More