Compare commits

..

2 Commits

Author SHA1 Message Date
Slava Semushin d48c8d87ca ru/docs/tasks/tools/install-kubectl.md: rephrase and fix typos. (#18622) 2020-01-12 03:55:38 -08:00
aisonaku 81d59b8236 Add russian translation for install kubectl page (#18111)
* first part of translation

* finish page

* Comments fixed

* fix typos
2020-01-09 22:06:34 -08:00
1096 changed files with 22821 additions and 49864 deletions
+17 -20
View File
@@ -1,20 +1,17 @@
<!-- 🛈
Hello!
Remember to ADD A DESCRIPTION and delete this note before submitting
your pull request. The description should explain what will change,
and why.
For overall help on editing and submitting pull requests, visit:
https://kubernetes.io/docs/contribute/start/#improve-existing-content
Use the default base branch, “master”, if you're documenting existing
features in the English localization.
If you're working on a different localization (not English), or you
are documenting a feature that will be part of a future release, see
https://kubernetes.io/docs/contribute/start#choose-which-git-branch-to-use
for advice.
-->
>^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> Remember to delete this note before submitting your pull request.
>
> For pull requests on 1.18 Features: set Milestone to 1.18 and Base Branch to dev-1.18
>
> For pull requests on Chinese localization, set Base Branch to release-1.16
> Feel free to ask questions in #kubernetes-docs-zh
>
> For pull requests on Korean Localization: set Base Branch to dev-1.16-ko.\<latest team milestone>
>
> If you need Help on editing and submitting pull requests, visit:
> https://kubernetes.io/docs/contribute/start/#improve-existing-content.
>
> If you need Help on choosing which branch to use, visit:
> https://kubernetes.io/docs/contribute/start#choose-which-git-branch-to-use.
>^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
+6 -6
View File
@@ -1,5 +1,5 @@
DOCKER = docker
HUGO_VERSION = $(shell grep ^HUGO_VERSION netlify.toml | tail -n 1 | cut -d '=' -f 2 | tr -d " \"\n")
HUGO_VERSION = 0.59.1
DOCKER_IMAGE = kubernetes-hugo
DOCKER_RUN = $(DOCKER) run --rm --interactive --tty --volume $(CURDIR):/src
NODE_BIN = node_modules/.bin
@@ -18,18 +18,15 @@ build: ## Build site with production settings and put deliverables in ./public
build-preview: ## Build site with drafts and future posts enabled
hugo --buildDrafts --buildFuture
deploy-preview: ## Deploy preview site via netlify
hugo --enableGitInfo --buildFuture -b $(DEPLOY_PRIME_URL)
functions-build:
$(NETLIFY_FUNC) build functions-src
check-headers-file:
scripts/check-headers-file.sh
production-build: build check-headers-file ## Build the production site and ensure that noindex headers aren't added
production-build: check-hugo-versions build check-headers-file ## Build the production site and ensure that noindex headers aren't added
non-production-build: ## Build the non-production site, which adds noindex headers to prevent indexing
non-production-build: check-hugo-versions ## Build the non-production site, which adds noindex headers to prevent indexing
hugo --enableGitInfo
serve: ## Boot the development server.
@@ -47,3 +44,6 @@ docker-serve:
test-examples:
scripts/test_examples.sh install
scripts/test_examples.sh run
check-hugo-versions:
scripts/hugo-version-check.sh $(HUGO_VERSION)
+10 -16
View File
@@ -30,6 +30,7 @@ aliases:
- onlydole
- parispittman
- vonguard
- onlydole
sig-docs-de-owners: # Admins for German content
- bene2k1
- mkorbi
@@ -39,33 +40,38 @@ aliases:
- mkorbi
- rlenferink
sig-docs-en-owners: # Admins for English content
- bradamant3
- bradtopol
- daminisatya
- gochist
- jaredbhatti
- jimangel
- kbarnard10
- kbhawkey
- makoscafee
- onlydole
- Rajakavitha1
- ryanmcginnis
- sftim
- simplytunde
- steveperry-53
- tengqm
- xiangpengzhao
- zacharysarah
- zparnold
sig-docs-en-reviews: # PR reviews for English content
- bradamant3
- bradtopol
- daminisatya
- gochist
- jaredbhatti
- jimangel
- kbarnard10
- kbhawkey
- makoscafee
- onlydole
- rajakavitha1
- rajeshdeshpande02
- ryanmcginnis
- sftim
- simplytunde
- steveperry-53
- tengqm
- xiangpengzhao
@@ -91,8 +97,6 @@ aliases:
- jygastaud
- awkif
- oussemos
- anthonydahanne
- feloy
sig-docs-fr-reviews: # PR reviews for French content
- remyleone
- perriea
@@ -105,8 +109,6 @@ aliases:
- jygastaud
- awkif
- oussemos
- anthonydahanne
- feloy
sig-docs-hi-owners: # Admins for Hindi content
- avidLearnerInProgress
- daminisatya
@@ -157,6 +159,7 @@ aliases:
- seokho-son
- ysyukr
sig-docs-maintainers: # Website maintainers
- bradamant3
- jimangel
- kbarnard10
- pwittrock
@@ -191,12 +194,10 @@ aliases:
- femrtnz
- jcjesus
- devlware
- jhonmike
sig-docs-pt-reviews: # PR reviews for Portugese content
- femrtnz
- jcjesus
- devlware
- jhonmike
sig-docs-vi-owners: # Admins for Vietnamese content
- huynguyennovem
- ngtuna
@@ -215,10 +216,3 @@ aliases:
- aisonaku
- potapy4
- dianaabv
sig-docs-pl-owners: # Admins for Polish content
- mfilocha
- nvtkaszpir
sig-docs-pl-reviews: # PR reviews for Polish content
- mfilocha
- nvtkaszpir
- kpucynski
+1 -1
View File
@@ -28,7 +28,7 @@ El método recomendado para levantar una copia local del sitio web kubernetes.io
> Para Windows, algunas otras herramientas como Make son necesarias. Puede instalarlas utilizando el gestor [Chocolatey](https://chocolatey.org). `choco install make` o siguiendo las instrucciones de [Make for Windows](http://gnuwin32.sourceforge.net/packages/make.htm).
> Si prefiere levantar el sitio web sin utilizar **Docker**, puede seguir las instrucciones disponibles en la sección [Levantando kubernetes.io en local con Hugo](#levantando-kubernetesio-en-local-con-hugo).
> Si prefiere levantar el sitio web sin utilizar **Docker**, puede seguir las instrucciones disponibles en la sección [Levantando kubernetes.io en local con Hugo](#levantando-kubernetes.io-en-local-con-hugo).
Una vez tenga Docker [configurado en su máquina](https://www.docker.com/get-started), puede construir la imagen de Docker `kubernetes-hugo` localmente ejecutando el siguiente comando en la raíz del repositorio:
+1 -1
View File
@@ -33,7 +33,7 @@ La façon recommandée d'exécuter le site web Kubernetes localement est d'utili
> Si vous êtes sous Windows, vous aurez besoin de quelques outils supplémentaires que vous pouvez installer avec [Chocolatey](https://chocolatey.org). `choco install install make`
> Si vous préférez exécuter le site Web localement sans Docker, voir [Exécuter le site localement avec Hugo](#exécuter-le-site-localement-en-utilisant-hugo) ci-dessous.
> Si vous préférez exécuter le site Web localement sans Docker, voir [Exécuter le site localement avec Hugo](#running-the-site-locally-using-hugo) ci-dessous.
Si vous avez Docker [up and running](https://www.docker.com/get-started), construisez l'image Docker `kubernetes-hugo' localement:
+4 -4
View File
@@ -9,7 +9,7 @@ Selamat datang! Repositori ini merupakan wadah bagi semua komponen yang dibutuhk
Pertama, kamu dapat menekan tombol **Fork** yang berada pada bagian atas layar, untuk menyalin repositori pada akun Github-mu. Salinan ini disebut sebagai **fork**. Kamu dapat menambahkan konten pada **fork** yang kamu miliki, setelah kamu merasa cukup untuk menambahkan konten yang kamu miliki dan ingin memberikan konten tersebut pada kami, kamu dapat melihat **fork** yang telah kamu buat dan membuat **pull request** untuk memberi tahu kami bahwa kamu ingin menambahkan konten yang telah kamu buat.
Setelah kamu membuat sebuah **pull request**, seorang **reviewer** akan memberikan masukan terhadap konten yang kamu sediakan serta beberapa hal yang dapat kamu lakukan apabila perbaikan diperlukan terhadap konten yang telah kamu sediakan. Sebagai seorang yang membuat **pull request**, **sudah menjadi kewajiban kamu untuk melakukan modifikasi terhadap konten yang kamu berikan sesuai dengan masukan yang diberikan oleh seorang reviewer Kubernetes**. Perlu kamu ketahui bahwa kamu dapat saja memiliki lebih dari satu orang **reviewer Kubernetes** atau dalam kasus kamu bisa saja mendapatkan **reviewer Kubernetes** yang berbeda dengan **reviewer Kubernetes** awal yang ditugaskan untuk memberikan masukan terhadap konten yang kamu sediakan. Selain itu, seorang **reviewer Kubernetes** bisa saja meminta masukan teknis dari [reviewer teknis Kubernetes](https://github.com/kubernetes/website/wiki/Tech-reviewers) jika diperlukan.
Setelah kamu membuat sebuah **pull request**, seorang **reviewer** akan memberikan masukan terhadap konten yang kamu sediakan serta beberapa hal yang dapat kamu lakukan apabila perbaikan diperlukan terhadap konten yang telah kamu sediakan. Sebagai seorang yang membuat **pull request**, **sudah menjadi kewajiban kamu untuk melakukan modifikasi terhadap konten yang kamu berikan sesuai dengan masukan yang diberikan oleh seorang reviewer Kubernetes**. Perlu kamu ketahui bahwa kamu dapat saja memiliki lebih dari satu orang **reviewer Kubernetes** atau dalam kasus kamu bisa saja mendapatkan **reviewer Kubernetes** yang berbeda dengan **reviewer Kubernetes** awal yang ditugaskan untuk memberikan masukan terhadap konten yang kamu sediakan. Selain itu, seorang **reviewer Kubernetes** bisa saja meminta masukan teknis dari [reviewer teknis Kubernetes](https://github.com/kubernetes/website/wiki/Tech-reviewers) jika diperlukan.
Untuk informasi lebih lanjut mengenai tata cara melakukan kontribusi, kamu dapat melihat tautan di bawah ini:
@@ -21,11 +21,11 @@ Untuk informasi lebih lanjut mengenai tata cara melakukan kontribusi, kamu dapat
## Menjalankan Dokumentasi Kubernetes pada Mesin Lokal Kamu
Petunjuk yang disarankan untuk menjalankan Dokumentasi Kubernetes pada mesin lokal kamus adalah dengan menggunakan [Docker](https://docker.com) **image** yang memiliki **package** [Hugo](https://gohugo.io), **Hugo** sendiri merupakan generator website statis.
Petunjuk yang disarankan untuk menjalankan Dokumentasi Kubernetes pada mesin lokal kamus adalah dengan menggunakan [Docker](https://docker.com) **image** yang memiliki **package** [Hugo](https://gohugo.io), **Hugo** sendiri merupakan generator website statis.
> Jika kamu menggunakan Windows, kamu mungkin membutuhkan beberapa langkah tambahan untuk melakukan instalasi perangkat lunak yang dibutuhkan. Instalasi ini dapat dilakukan dengan menggunakan [Chocolatey](https://chocolatey.org). `choco install make`
> Jika kamu ingin menjalankan **website** tanpa menggunakan **Docker**, kamu dapat melihat tautan berikut [Petunjuk untuk menjalankan website pada mesin lokal dengan menggunakan Hugo](#petunjuk-untuk-menjalankan-website-pada-mesin-lokal-dengan-menggunakan-hugo) di bagian bawah.
> Jika kamu ingin menjalankan **website** tanpa menggunakan **Docker**, kamu dapat melihat tautan berikut [Petunjuk untuk menjalankan website pada mesin lokal dengan menggunakan Hugo](#petunjuk-untuk-menjalankan-website-pada-mesin-lokal-denga-menggunakan-hugo) di bagian bawah.
Jika kamu sudah memiliki **Docker** [yang sudah dapat digunakan](https://www.docker.com/get-started), kamu dapat melakukan **build** `kubernetes-hugo` **Docker image** secara lokal:
@@ -44,7 +44,7 @@ Buka **browser** kamu ke http://localhost:1313 untuk melihat laman dokumentasi.
## Petunjuk untuk menjalankan website pada mesin lokal dengan menggunakan Hugo
Kamu dapat melihat [dokumentasi resmi Hugo](https://gohugo.io/getting-started/installing/) untuk mengetahui langkah yang diperlukan untuk melakukan instalasi **Hugo**. Pastikan kamu melakukan instalasi versi **Hugo** sesuai dengan versi yang tersedia pada **environment variable** `HUGO_VERSION` pada **file**[`netlify.toml`](netlify.toml#L9).
Kamu dapat melihat [dokumentasi resmi Hugo](https://gohugo.io/getting-started/installing/) untuk mengetahui langkah yang diperlukan untuk melakukan instalasi **Hugo**. Pastikan kamu melakukan instalasi versi **Hugo** sesuai dengan versi yang tersedia pada **environment variable** `HUGO_VERSION` pada **file**[`netlify.toml`](netlify.toml#L9).
Untuk menjalankan laman pada mesin lokal setelah instalasi **Hugo**, kamu dapat menjalankan perintah berikut:
-71
View File
@@ -1,71 +0,0 @@
# La documentazione di Kubernetes
[![Build Status](https://api.travis-ci.org/kubernetes/website.svg?branch=master)](https://travis-ci.org/kubernetes/website)
[![GitHub release](https://img.shields.io/github/release/kubernetes/website.svg)](https://github.com/kubernetes/website/releases/latest)
Benvenuto! Questo repository contiene tutte le informazioni necessarie per creare la [documentazione e il sito web di Kubernetes](https://kubernetes.io/). Siamo onorati che tu voglia contribuire!
## Come contribuire alla documentazione
Puoi premere il pulsante **Fork** in alto nella parte destra dello schermo per creare una copia di questo repository sotto la tua utenza di GitHub. Questa copia è chiamata un *fork*. Puoi fare tutte le modifiche che vuoi nel tuo fork, e quando sei pronto a mandarci i cambiamenti, vai sulla pagina del tuo fork e crea una nuova pull request per farcelo sapere.
Una volta che hai creato la pull request, un reviewer (revisore) Kubernetes la prenderà in carico per fornire un feedback chiaro e concretamente attuabile. Come owner della pull request, **è tua responsabilità modificare la tua pull request per rispondere al feedback che ti è stato fornito dal reviewer Kubernetes**. Inoltre, tieni presente che potresti ricevere feedback da più di un Reviewer di Kubernetes, incluse quindi persone diverse da quella che ti ha fornito il primo feedback. In alcuni casi, uno dei tuoi reviewer potrebbe richiedere una revisione tecnica da un [technical reviewer di Kubernetes](https://github.com/kubernetes/website/wiki/Tech-reviewers) quando necessario. I reviewer faranno del loro meglio per fornire feedback in modo tempestivo, ma i tempi di risposta possono variare in base alle circostanze.
Per maggiori informazioni su come contribuire alla documentazione Kubernetes, vedi:
* [Cominciare a contribuire](https://kubernetes.io/docs/contribute/start/)
* [Vedere le modifiche localmente](http://kubernetes.io/docs/contribute/intermediate#view-your-changes-locally)
* [Utilizzare i template delle pagine](http://kubernetes.io/docs/contribute/style/page-templates/)
* [Guida Stilistica per la documentazione di Kubernetes](http://kubernetes.io/docs/contribute/style/style-guide/)
* [Tradurre la documentazione di Kubernetes](https://kubernetes.io/docs/contribute/localization/)
## Eseguire il sito Web localmente usando Docker
Il modo consigliato per eseguire localmente il sito Web Kubernetes prevede l'utilizzo di un'immagine [Docker](https://docker.com) inclusa nel sito e configurata con tutti i software necessari, a partire dal generatore di siti web statici [Hugo](https://gohugo.io).
> Se stai utilizzando Windows, avrai bisogno di alcuni strumenti aggiuntivi che puoi installare con [Chocolatey](https://chocolatey.org). `choco install make`
> Se preferisci eseguire il sito Web localmente senza Docker, vedi [Eseguire il sito Web localmente utilizzando Hugo](#eseguire-il-sito-web-localmente-utilizzando-hugo) di seguito.
Se hai Docker [attivo e funzionante](https://www.docker.com/get-started), crea l'immagine Docker `kubernetes-hugo` localmente:
```bash
make docker-image
```
Dopo aver creato l'immagine, è possibile eseguire il sito Web localmente:
```bash
make docker-serve
```
Apri il tuo browser su http://localhost:1313 per visualizzare il sito Web. Mentre modifichi i file sorgenti, Hugo aggiorna automaticamente il sito Web e forza un aggiornamento della pagina visualizzata nel browser.
## Eseguire il sito Web localmente utilizzando Hugo
Vedi la [documentazione ufficiale di Hugo](https://gohugo.io/getting-started/installing/) per le istruzioni di installazione di Hugo. Assicurati di installare esattamente la versione di Hugo specificata dalla variabile d'ambiente `HUGO_VERSION` nel file [`netlify.toml`](netlify.toml#L9).
Per eseguire il sito Web localmente dopo aver installato Hugo:
```bash
make serve
```
Questo comando avvierà il server Hugo locale sulla porta 1313. Apri il tuo browser su http://localhost:1313 per visualizzare il sito Web. Mentre apporti modifiche ai file di origine, Hugo aggiorna il sito Web e forza un aggiornamento del browser.
## Community, discussioni, contribuire e supporto
Scopri come interagire con la community di Kubernetes nella [pagina della community](http://kubernetes.io/community/).
Puoi contattare i maintainers di questo progetto su:
- [Slack](https://kubernetes.slack.com/messages/sig-docs)
- [Mailing List](https://groups.google.com/forum/#!forum/kubernetes-sig-docs)
### Codice di condotta
La partecipazione alla comunità Kubernetes è regolata dal [Codice di condotta di Kubernetes](code-of-conduct.md).
## Grazie!
Kubernetes prospera grazie alla partecipazione della comunity e apprezziamo i tuoi contributi al nostro sito Web e alla nostra documentazione!
-87
View File
@@ -1,87 +0,0 @@
# Dokumentacja projektu Kubernetes
[![Build Status](https://api.travis-ci.org/kubernetes/website.svg?branch=master)](https://travis-ci.org/kubernetes/website)
[![GitHub release](https://img.shields.io/github/release/kubernetes/website.svg)](https://github.com/kubernetes/website/releases/latest)
Witamy!
W tym repozytorium znajdziesz wszystko, czego potrzebujesz do zbudowania [strony internetowej Kubernetesa wraz z dokumentacją](https://kubernetes.io/). Bardzo nam miło, że chcesz wziąć udział w jej współtworzeniu!
## Twój wkład w dokumentację
Możesz kliknąć w przycisk **Fork** w prawym górnym rogu ekranu, aby stworzyć kopię tego repozytorium na swoim koncie GitHub. Taki rodzaj kopii (odgałęzienia) nazywa się *fork*. Zmieniaj w nim, co chcesz, a kiedy będziesz już gotowy/a przesłać te zmiany do nas, przejdź do swojej kopii i stwórz nowy *pull request*, abyśmy zostali o tym poinformowani.
Po stworzeniu *pull request*, jeden z recenzentów projektu Kubernetes podejmie się przekazania jasnych wskazówek pozwalających podjąć następne działania. Na Tobie, jako właścicielu *pull requesta*, **spoczywa odpowiedzialność za wprowadzenie poprawek zgodnie z uwagami recenzenta.** Może też się zdarzyć, że swoje uwagi zgłosi więcej niż jeden recenzent, lub że recenzję będzie robił ktoś inny, niż ten, kto został przydzielony na początku. W niektórych przypadkach, jeśli zajdzie taka potrzeba, recenzent może poprosić dodatkowo o recenzję jednego z [recenzentów technicznych](https://github.com/kubernetes/website/wiki/Tech-reviewers). Recenzenci zrobią wszystko, aby odpowiedzieć sprawnie, ale konkretny czas odpowiedzi zależy od wielu czynników.
Więcej informacji na temat współpracy przy tworzeniu dokumentacji znajdziesz na stronach:
* [Jak rozpocząć współpracę](https://kubernetes.io/docs/contribute/start/)
* [Podgląd wprowadzanych zmian w dokumentacji](http://kubernetes.io/docs/contribute/intermediate#view-your-changes-locally)
* [Szablony stron](http://kubernetes.io/docs/contribute/style/page-templates/)
* [Styl pisania dokumentacji](http://kubernetes.io/docs/contribute/style/style-guide/)
* [Lokalizacja dokumentacji Kubernetes](https://kubernetes.io/docs/contribute/localization/)
## Różne wersje językowe `README.md`
| | |
|---|---|
|[README po francusku](README-fr.md)|[README po koreańsku](README-ko.md)|
|[README po niemiecku](README-de.md)|[README po portugalsku](README-pt.md)|
|[README w hindi](README-hi.md)|[README po hiszpańsku](README-es.md)|
|[README po indonezyjsku](README-id.md)|[README po chińsku](README-zh.md)|
|[README po japońsku](README-ja.md)|[README po polsku](README-pl.md)|
|||
## Jak uruchomić lokalną kopię strony przy pomocy Dockera?
Zalecaną metodą uruchomienia serwisu internetowego Kubernetesa lokalnie jest użycie specjalnego obrazu [Dockera](https://docker.com), który zawiera generator stron statycznych [Hugo](https://gohugo.io).
> Użytkownicy Windows będą potrzebowali dodatkowych narzędzi, które mogą zainstalować przy pomocy [Chocolatey](https://chocolatey.org).
```bash
choco install make
```
> Jeśli wolisz uruchomić serwis lokalnie bez Dockera, przeczytaj [jak uruchomić serwis lokalnie przy pomocy Hugo](#jak-uruchomić-serwis-lokalnie-przy-pomocy-hugo) poniżej.
Jeśli [zainstalowałeś i uruchomiłeś](https://www.docker.com/get-started) już Dockera, zbuduj obraz `kubernetes-hugo` lokalnie:
```bash
make docker-image
```
Po zbudowaniu obrazu, możesz uruchomić serwis lokalnie:
```bash
make docker-serve
```
Aby obejrzeć zawartość serwisu otwórz w przeglądarce adres http://localhost:1313. Po każdej zmianie plików źródłowych, Hugo automatycznie aktualizuje stronę i odświeża jej widok w przeglądarce.
## Jak uruchomić lokalną kopię strony przy pomocy Hugo?
Zajrzyj do [oficjalnej dokumentacji Hugo](https://gohugo.io/getting-started/installing/) po instrukcję instalacji. Upewnij się, że instalujesz rozszerzoną wersję Hugo, określoną przez zmienną środowiskową `HUGO_VERSION` w pliku [`netlify.toml`](netlify.toml#L9).
Aby uruchomić serwis lokalnie po instalacji Hugo, napisz:
```bash
make serve
```
Zostanie uruchomiony lokalny serwer Hugo na porcie 1313. Otwórz w przeglądarce adres http://localhost:1313, aby obejrzeć zawartość serwisu. Po każdej zmianie plików źródłowych, Hugo automatycznie aktualizuje stronę i odświeża jej widok w przeglądarce.
## Społeczność, listy dyskusyjne, uczestnictwo i wsparcie
Zajrzyj na stronę [społeczności](http://kubernetes.io/community/), aby dowiedzieć się, jak możesz zaangażować się w jej działania.
Możesz kontaktować się z gospodarzami projektu za pomocą:
* [Komunikatora Slack](https://kubernetes.slack.com/messages/sig-docs)
* [List dyskusyjnych](https://groups.google.com/forum/#!forum/kubernetes-sig-docs)
### Zasady postępowania
Udział w działaniach społeczności Kubernetes jest regulowany przez [Kodeks postępowania](code-of-conduct.md).
## Dziękujemy!
Kubernetes rozkwita dzięki zaangażowaniu społeczności — doceniamy twój wkład w tworzenie naszego serwisu i dokumentacji!
+7 -7
View File
@@ -7,7 +7,7 @@
## Вклад в документацию
Нажмите на кнопку **Fork** в правом верхнем углу, чтобы создать копию этого репозитория в ваш GitHub-аккаунт. Эта копия называется *форк-репозиторием*. Делайте любые изменения в вашем форк-репозитории, и когда вы будете готовы опубликовать изменения, откройте форк-репозиторий и создайте новый пулреквест, чтобы уведомить нас.
Нажмите на кнопку **Fork** в правом верхнем углу, чтобы создать копию этого репозитория в ваш GitHub-аккаунт. Это копия называется *форк-репозиторием*. Делайте любые изменения в вашем форк-репозитории, и когда вы будете готовы опубликовать изменения, откройте форк-репозиторий и создайте новый пулреквест, чтобы уведомить нас.
После того, как вы отправите пулреквест, ревьювер Kubernetes даст по нему обратную связь. Вы, как автор пулреквеста, **должны обновить свой пулреквест после его рассмотрения ревьювером Kubernetes.** Вполне возможно, что более одного ревьювера Kubernetes оставят свои комментарии или даже может быть так, что новый комментарий ревьювера Kubernetes будет отличаться от первоначального назначенного ревьювера. Кроме того, в некоторых случаях один из ревьюверов может запросить технический обзор у [технического ревьювера Kubernetes](https://github.com/kubernetes/website/wiki/Tech-reviewers), если это будет необходимо. Ревьюверы сделают все возможное, чтобы как можно оперативно оставить свои предложения и пожелания, но время ответа может варьироваться в зависимости от обстоятельств.
@@ -34,7 +34,7 @@
> Если вы используете Windows, вам необходимо установить дополнительные инструменты через [Chocolatey](https://chocolatey.org). `choco install make`
> Если вы хотите запустить сайт локально без Docker, обратитесь к разделу [Запуск сайта с помощью Hugo](#запуск-сайта-с-помощью-hugo) ниже на этой странице.
> Если вы хотите запустить сайт локально без Docker, обратитесь к разделу [Запуск сайта с помощью Hugo](#running-the-site-locally-using-hugo) ниже на этой странице.
Когда Docker [установлен и запущен](https://www.docker.com/get-started), соберите локально Docker-образ `kubernetes-hugo`, выполнив команду в консоли:
@@ -50,9 +50,9 @@ make docker-serve
Откройте браузер и перейдите по ссылке http://localhost:1313, чтобы открыть сайт. Если вы редактируете исходные файлы сайта, Hugo автоматически применит изменения и обновит страницу в браузере.
## Запуск сайта с помощью Hugo
## Запуск сайта с помощью Hugo {#running-the-site-locally-using-hugo}
Обратитесь к [официальной документации Hugo](https://gohugo.io/getting-started/installing/), чтобы установить Hugo. Убедитесь, что вы установили правильную версию Hugo, которая устанавливается в переменной окружения `HUGO_VERSION` в файле [`netlify.toml`](netlify.toml#L10).
Обратитесь к [официальной документации Hugo](https://gohugo.io/getting-started/installing/), чтобы установить Hugo. Убедитесь, что вы установили правильную версию Hugo, которая устанавливается в переменной окружения `HUGO_VERSION` в файле [`netlify.toml`](netlify.toml#L9).
После установки Hugo, чтобы запустить сайт, выполните в консоли:
@@ -60,7 +60,7 @@ make docker-serve
make serve
```
Эта команда запустит сервер Hugo на порту 1313. Откройте браузер и перейдите по ссылке http://localhost:1313, чтобы открыть сайт. Если вы отредактируете исходные файлы сайта, Hugo автоматически применит изменения и обновит страницу в браузере.
Эта команда запустит сервер Hugo на порту 1313. Откройте браузер и перейдите по ссылке http://localhost:1313, чтобы открыть сайта. Если вы отредактируете исходные файлы сайта, Hugo автоматически применит изменения и обновит страницу в браузере.
## Сообщество, обсуждение, вклад и поддержка
@@ -71,9 +71,9 @@ make serve
- [Канал в Slack](https://kubernetes.slack.com/messages/sig-docs)
- [Рассылка](https://groups.google.com/forum/#!forum/kubernetes-sig-docs)
### Кодекс поведения
### Нормы поведения
Участие в сообществе Kubernetes регулируется [кодексом поведения Kubernetes](code-of-conduct.md).
Участие в сообществе Kubernetes регулируется [нормами поведения Kubernetes](code-of-conduct.md).
## Спасибо!
+1 -1
View File
@@ -26,7 +26,7 @@ Cách được đề xuất để chạy trang web Kubernetes cục bộ là dù
> Nếu bạn làm việc trên môi trường Windows, bạn sẽ cần thêm môt vài công cụ mà bạn có thể cài đặt với [Chocolatey](https://chocolatey.org). `choco install make`
> Nếu bạn không muốn dùng Docker để chạy trang web cục bộ, hãy xem [Chạy website cục bộ dùng Hugo](#chạy-website-cục-bộ-dùng-hugo) dưới đây.
> Nếu bạn không muốn dùng Docker để chạy trang web cục bộ, hãy xem [Chạy website cục bộ dùng Hugo](#Chạy website cục bộ dùng Hugo) dưới đây.
Nếu bạn có Docker đang [up và running](https://www.docker.com/get-started), build `kubernetes-hugo` Docker image cục bộ:
+2 -3
View File
@@ -27,8 +27,7 @@ For more information about contributing to the Kubernetes documentation, see:
|[Hindi README](README-hi.md)|[Spanish README](README-es.md)|
|[Indonesian README](README-id.md)|[Chinese README](README-zh.md)|
|[Japanese README](README-ja.md)|[Vietnamese README](README-vi.md)|
|[Russian README](README-ru.md)|[Italian README](README-it.md)|
|[Polish README](README-pl.md)||
|[Russian README](README-ru.md)|
|||
## Running the website locally using Docker
@@ -37,7 +36,7 @@ The recommended way to run the Kubernetes website locally is to run a specialize
> If you are running on Windows, you'll need a few more tools which you can install with [Chocolatey](https://chocolatey.org). `choco install make`
> If you'd prefer to run the website locally without Docker, see [Running the website locally using Hugo](#running-the-website-locally-using-hugo) below.
> If you'd prefer to run the website locally without Docker, see [Running the website locally using Hugo](#running-the-site-locally-using-hugo) below.
If you have Docker [up and running](https://www.docker.com/get-started), build the `kubernetes-hugo` Docker image locally:
+1 -1
View File
@@ -10,6 +10,6 @@
# DO NOT REPORT SECURITY VULNERABILITIES DIRECTLY TO THESE NAMES, FOLLOW THE
# INSTRUCTIONS AT https://kubernetes.io/security/
bradamant3
jimangel
kbarnard10
zacharysarah
+2 -9
View File
@@ -107,9 +107,9 @@ header
z-index: 8888
background-color: transparent
box-shadow: 0 0 0 transparent
overflow: hidden
transition: 0.3s
text-align: center
overflow: hidden
.logo
@@ -245,7 +245,6 @@ header
background-color: white
#mainNav
h5
color: $blue
font-weight: normal
@@ -578,9 +577,6 @@ section
li
display: inline-block
height: 100%
margin-right: 10px
&:last-child
margin-right: 0
a
display: block
@@ -601,11 +597,11 @@ section
#vendorStrip
line-height: 44px
max-width: 100%
overflow-x: auto
-webkit-overflow-scrolling: touch
ul
float: none
overflow-x: auto
#searchBox
float: none
@@ -1055,9 +1051,6 @@ dd
a.issue
margin-left: 0px
.gridPageHome .flyout-button
display: none
.feedback--no
margin-left: 1em
+3 -5
View File
@@ -107,7 +107,7 @@ $video-section-height: 550px
padding-right: 10px
#home
section, header
section, header, footer
.main-section
max-width: 1000px
@@ -178,18 +178,16 @@ $video-section-height: 550px
nav
overflow: hidden
margin-bottom: 20px
display: flex
justify-content: space-between
a
width: auto
width: 16.65%
float: left
font-size: 24px
font-weight: 300
white-space: nowrap
.social
padding: 0
padding: 0 30px
max-width: 1200px
div
+2 -4
View File
@@ -133,21 +133,18 @@ $feature-box-div-width: 45%
max-width: 25%
max-height: 100%
transform: translateY(-50%)
width: 100%
&:nth-child(odd)
padding-right: 210px
.image-wrapper
right: 0
text-align: right
&:nth-child(even)
padding-left: 210px
.image-wrapper
left: 0
text-align: left
&:nth-child(1)
padding-right: 0
@@ -222,8 +219,9 @@ $feature-box-div-width: 45%
footer
nav
text-align: center
a
width: auto
width: 30%
padding: 0 20px
.social
+35 -47
View File
@@ -66,13 +66,13 @@ time_format_blog = "Monday, January 02, 2006"
description = "Production-Grade Container Orchestration"
showedit = true
latest = "v1.21"
latest = "v1.17"
fullversion = "v1.17.17"
fullversion = "v1.17.0"
version = "v1.17"
githubbranch = "v1.17.17"
docsbranch = "release-1.17"
deprecated = true
githubbranch = "master"
docsbranch = "master"
deprecated = false
currentUrl = "https://kubernetes.io/docs/home/"
nextUrl = "https://kubernetes-io-vnext-staging.netlify.com/"
githubWebsiteRepo = "github.com/kubernetes/website"
@@ -83,6 +83,13 @@ announcement = false
# announcement_message is only displayed when announcement = true; update with your specific message
announcement_message = "The Kubernetes Documentation team would like your feedback! Please take a <a href='https://www.surveymonkey.com/r/8R237FN' target='_blank'>short survey</a> so we can improve the Kubernetes online documentation."
[[params.versions]]
fullversion = "v1.17.0"
version = "v1.17"
githubbranch = "v1.17.0"
docsbranch = "release-1.17"
url = "https://kubernetes.io"
[params.pushAssets]
css = [
"callouts",
@@ -95,40 +102,32 @@ js = [
]
[[params.versions]]
fullversion = "v1.21.0"
version = "v1.21"
githubbranch = "v1.21.0"
docsbranch = "master"
url = "https://kubernetes.io"
fullversion = "v1.16.3"
version = "v1.16"
githubbranch = "v1.16.3"
docsbranch = "release-1.16"
url = "https://v1-16.docs.kubernetes.io"
[[params.versions]]
fullversion = "v1.20.5"
version = "v1.20"
githubbranch = "v1.20.5"
docsbranch = "release-1.20"
url = "https://v1-20.docs.kubernetes.io"
fullversion = "v1.15.6"
version = "v1.15"
githubbranch = "v1.15.6"
docsbranch = "release-1.15"
url = "https://v1-15.docs.kubernetes.io"
[[params.versions]]
fullversion = "v1.19.9"
version = "v1.19"
githubbranch = "v1.19.9"
docsbranch = "release-1.19"
url = "https://v1-19.docs.kubernetes.io"
fullversion = "v1.14.9"
version = "v1.14"
githubbranch = "v1.14.9"
docsbranch = "release-1.14"
url = "https://v1-14.docs.kubernetes.io"
[[params.versions]]
fullversion = "v1.18.17"
version = "v1.18"
githubbranch = "v1.18.17"
docsbranch = "release-1.18"
url = "https://v1-18.docs.kubernetes.io"
[[params.versions]]
fullversion = "v1.17.17"
version = "v1.17"
githubbranch = "v1.17.17"
docsbranch = "release-1.17"
url = "https://v1-17.docs.kubernetes.io"
fullversion = "v1.13.12"
version = "v1.13"
githubbranch = "v1.13.12"
docsbranch = "release-1.13"
url = "https://v1-13.docs.kubernetes.io"
# Language definitions.
@@ -175,7 +174,7 @@ language_alternatives = ["en"]
[languages.fr]
title = "Kubernetes"
description = "Solution professionnelle dorchestration de conteneurs"
description = "Production-Grade Container Orchestration"
languageName ="Français"
weight = 5
contentDir = "content/fr"
@@ -223,7 +222,7 @@ language_alternatives = ["en"]
[languages.es]
title = "Kubernetes"
description = "Orquestación de contenedores para producción"
description = "Production-Grade Container Orchestration"
languageName ="Español"
weight = 9
contentDir = "content/es"
@@ -247,7 +246,7 @@ language_alternatives = ["en"]
[languages.id]
title = "Kubernetes"
description = "Orkestrasi Kontainer dengan Skala Produksi"
description = "Production-Grade Container Orchestration"
languageName ="Bahasa Indonesia"
weight = 10
contentDir = "content/id"
@@ -287,14 +286,3 @@ time_format_blog = "02.01.2006"
# A list of language codes to look for untranslated content, ordered from left to right.
language_alternatives = ["en"]
[languages.pl]
title = "Kubernetes"
description = "Produkcyjny system zarządzania kontenerami"
languageName = "Polski"
weight = 13
contentDir = "content/pl"
[languages.pl.params]
time_format_blog = "01.02.2006"
# A list of language codes to look for untranslated content, ordered from left to right.
language_alternatives = ["en"]
+2 -2
View File
@@ -44,12 +44,12 @@ Kubernetes ist Open Source und bietet Ihnen die Freiheit, die Infrastruktur vor
<br>
<br>
<br>
<a href="https://events.linuxfoundation.org/events/kubecon-cloudnativecon-europe-2020/" button id="desktopKCButton">Besuchen Sie die KubeCon vom 30 März-2 Apr. 2020 in Amsterdam</a>
<a href="https://events.linuxfoundation.org/events/kubecon-cloudnativecon-north-america-2019" button id="desktopKCButton">KubeCon vom 18-21. November 2019 in San Diego</a>
<br>
<br>
<br>
<br>
<a href="https://www.lfasiallc.cn/kubecon-cloudnativecon-open-source-summit-china/" button id="desktopKCButton">Besuchen Sie die KubeCon vom 28-30 Juli 2020 in Shanghai</a>
<a href="https://events.linuxfoundation.org/events/kubecon-cloudnativecon-europe-2020/" button id="desktopKCButton">KubeCon vom 30 März-2 Apr. 2020 in Amsterdam</a>
</div>
<div id="videoPlayer">
<iframe data-url="https://www.youtube.com/embed/H06qrNmGqyE?autoplay=1" frameborder="0" allowfullscreen></iframe>
-26
View File
@@ -1,26 +0,0 @@
---
title: Community
layout: basic
cid: community
css: /css/community.css
---
<div class="community_main">
<h1>Kubernetes Community Code of Conduct</h1>
Kubernetes folgt dem
<a href="https://github.com/cncf/foundation/blob/master/code-of-conduct-languages/de.md">CNCF Verhaltenskodex</a>.
Der Kodex befindet sich weiter unten auf der Seite, wie er auch in
<a href="https://github.com/cncf/foundation/blob/214585e24aab747fb85c2ea44fbf4a2442e30de6/code-of-conduct-languages/de.md">Commit 214585e</a> gefunden werden kann.
Wenn dir auffällt, dass die hier gezeigte Version nicht mehr aktuell ist,
<a href="https://github.com/kubernetes/website/issues/new">eröffne bitte ein Issue</a>.
Wenn dir bei einem Event, einem Meeting, in Slack oder einem anderen
Kommunikationskanal ein Verstoß gegen den Verhaltenskodex auffällt, wende dich an das <a href="https://git.k8s.io/community/committee-code-of-conduct">Kubernetes Code of Conduct Committee</a>.
Du kannst das Komitee über E-Mail erreichen: <a href="mailto:conduct@kubernetes.io">conduct@kubernetes.io</a>.
Deine Anonymität wird geschützt.
<div class="cncf_coc_container">
{{< include "/static/cncf-code-of-conduct.md" >}}
</div>
</div>
-2
View File
@@ -1,2 +0,0 @@
The files in this directory have been imported from other sources. Do not
edit them directly, except by replacing them with new versions.
@@ -1,30 +0,0 @@
<!-- Do not edit this file directly. Get the latest from
https://github.com/cncf/foundation/blob/master/code-of-conduct.md -->
## CNCF Gemeinschafts-Verhaltenskodex v1.0
### Verhaltenskodex für Mitwirkende
Als Mitwirkende und Betreuer dieses Projekts und im Interesse der Förderung einer offenen und einladenden Gemeinschaft verpflichten wir uns dazu, alle Menschen zu respektieren, die durch Berichterstattung, Veröffentlichung von Eigenschaftsanfragen, Aktualisierung der Dokumentation, Einreichung von Pull-Anfragen oder Patches und anderen Aktivitäten einen Beitrag leisten.
Wir sind bestrebt, die Teilnahme an diesem Projekt für alle zu einer belästigungsfreien Erfahrung zu machen, unabhängig von Erfahrungsstand, Geschlecht, geschlechtsspezifischer Identität und Ausdruck, sexueller Orientierung, Behinderung, persönlichem Aussehen, Körpergröße, Rasse, ethnischer Herkunft, Alter, Religion oder Nationalität.
Beispiele für unzumutbares Verhalten der Teilnehmer sind:
- Der Gebrauch von sexualisierter Sprache oder Bildern
- Persönliche Angriffe
- Trolling oder beleidigende/herabwürdigende Kommentare
- Öffentliche oder private Belästigungen
- Veröffentlichung privater Informationen anderer, wie z.B. physischer oder elektronischer Adressen, ohne ausdrückliche Genehmigung
- Anderes unethisches oder unprofessionelles Verhalten.
Projektbetreuer haben das Recht und die Verantwortung, Kommentare, Commits, Code, Wiki-Bearbeitungen, Probleme und andere Beiträge zu entfernen, zu bearbeiten oder abzulehnen, die nicht mit diesem Verhaltenskodex übereinstimmen. Mit der Annahme dieses Verhaltenskodex verpflichten sich die Projektbetreuer, diese Grundsätze fair und konsequent auf jeden Aspekt der Projektleitung anzuwenden. Projektbetreuer, die den Verhaltenskodex nicht befolgen oder durchsetzen, können dauerhaft vom Projektteam ausgeschlossen werden.
Dieser Verhaltenskodex gilt sowohl innerhalb von Projekträumen als auch in öffentlichen Räumen, wenn eine Person das Projekt oder seine Gemeinschaft vertritt.
Fälle von missbräuchlichem, belästigendem oder anderweitig unzumutbarem Verhalten in Kubernetes können gemeldet werden, indem Sie sich an das [Kubernetes Komitee für Verhaltenskodex](https://git.k8s.io/community/committee-code-of-conduct) wenden unter <conduct@kubernetes.io>. Für andere Projekte wenden Sie sich bitte an einen CNCF-Projektbetreuer oder an unseren Mediator, Mishi Choudhary <mishi@linux.com>.
Dieser Verhaltenskodex wurde aus dem Contributor Covenant übernommen (http://contributor-covenant.org), Version 1.2.0, verfügbar unter http://contributor-covenant.org/version/1/2/0/
### CNCF Verhaltenskodex für Veranstaltungen
Für CNCF Veranstaltungen gilt der Verhaltenskodex der Linux Foundation, der auf der Veranstaltungsseite verfügbar ist. Diese ist so konzipiert, dass sie mit der oben genannten Richtlinie kompatibel ist und enthält auch weitere Details zur Reaktion auf Vorfälle.
+1 -1
View File
@@ -52,7 +52,7 @@ Wenn Sie beispielsweise mit der Kubernetes-API ein Deployment-Objekt erstellen,
### Kubernetes Master
Der Kubernetes-Master ist für Erhalt des gewünschten Status Ihres Clusters verantwortlich. Wenn Sie mit Kubernetes interagieren, beispielsweise mit dem Kommandozeilen-Tool `kubectl`, kommunizieren Sie mit dem Kubernetes-Master Ihres Clusters.
Der Kubernetes-Master ist für Erhalt des gewünschten Status Ihres Clusters verantwortlich. Wenn Sie mit Kubernetes interagieren, beispielsweise mit dem Kommanduzeilen-Tool `kubectl`, kommunizieren Sie mit dem Kubernetes-Master Ihres Clusters.
> Der Begriff "Master" bezeichnet dabei eine Reihe von Prozessen, die den Clusterstatus verwalten. Normalerweise werden diese Prozesse alle auf einem einzigen Node im Cluster ausgeführt. Dieser Node wird auch als Master bezeichnet. Der Master kann repliziert werden, um die Verfügbarkeit und Redundanz zu erhöhen.
@@ -1,56 +0,0 @@
---
title: Addons Installieren
content_template: templates/concept
---
{{% capture overview %}}
Add-Ons erweitern die Funktionalität von Kubernetes.
Diese Seite gibt eine Übersicht über einige verfügbare Add-Ons und verweist auf die entsprechenden Installationsanleitungen.
Die Add-Ons in den einzelnen Kategorien sind alphabetisch sortiert - Die Reihenfolge impliziert keine bevorzugung einzelner Projekte.
{{% /capture %}}
{{% capture body %}}
## Networking und Network Policy
* [ACI](https://www.github.com/noironetworks/aci-containers) bietet Container-Networking und Network-Security mit Cisco ACI.
* [Calico](https://docs.projectcalico.org/latest/introduction/) ist ein Networking- und Network-Policy-Provider. Calico unterstützt eine Reihe von Networking-Optionen, damit Du die richtige für deinen Use-Case wählen kannst. Dies beinhaltet Non-Overlaying and Overlaying-Networks mit oder ohne BGP. Calico nutzt die gleiche Engine um Network-Policies für Hosts, Pods und (falls Du Istio & Envoy benutzt) Anwendungen auf Service-Mesh-Ebene durchzusetzen.
* [Canal](https://github.com/tigera/canal/tree/master/k8s-install) vereint Flannel und Calico um Networking- und Network-Policies bereitzustellen.
* [Cilium](https://github.com/cilium/cilium) ist ein L3 Network- and Network-Policy-Plugin welches das transparent HTTP/API/L7-Policies durchsetzen kann. Sowohl Routing- als auch Overlay/Encapsulation-Modes werden uterstützt. Außerdem kann Cilium auf andere CNI-Plugins aufsetzen.
* [CNI-Genie](https://github.com/Huawei-PaaS/CNI-Genie) ermöglicht das nahtlose Verbinden von Kubernetes mit einer Reihe an CNI-Plugins wie z.B. Calico, Canal, Flannel, Romana, oder Weave.
* [Contiv](http://contiv.github.io) bietet konfigurierbares Networking (Native L3 auf BGP, Overlay mit vxlan, Klassisches L2, Cisco-SDN/ACI) für verschiedene Anwendungszwecke und auch umfangreiches Policy-Framework. Das Contiv-Projekt ist vollständig [Open Source](http://github.com/contiv). Der [installer](http://github.com/contiv/install) bietet sowohl kubeadm als auch nicht-kubeadm basierte Installationen.
* [Contrail](http://www.juniper.net/us/en/products-services/sdn/contrail/contrail-networking/), basierend auf [Tungsten Fabric](https://tungsten.io), ist eine Open Source, multi-Cloud Netzwerkvirtualisierungs- und Policy-Management Plattform. Contrail und Tungsten Fabric sind mit Orechstratoren wie z.B. Kubernetes, OpenShift, OpenStack und Mesos integriert und bieten Isolationsmodi für Virtuelle Maschinen, Container (bzw. Pods) und Bare Metal workloads.
* [Flannel](https://github.com/coreos/flannel/blob/master/Documentation/kubernetes.md) ist ein Overlay-Network-Provider der mit Kubernetes genutzt werden kann.
* [Knitter](https://github.com/ZTE/Knitter/) ist eine Network-Lösung die Mehrfach-Network in Kubernetes ermöglicht.
* [Multus](https://github.com/Intel-Corp/multus-cni) ist ein Multi-Plugin für Mehrfachnetzwerk-Unterstützung um alle CNI-Plugins (z.B. Calico, Cilium, Contiv, Flannel), zusätzlich zu SRIOV-, DPDK-, OVS-DPDK- und VPP-Basierten Workloads in Kubernetes zu unterstützen.
* [NSX-T](https://docs.vmware.com/en/VMware-NSX-T/2.0/nsxt_20_ncp_kubernetes.pdf) Container Plug-in (NCP) bietet eine Integration zwischen VMware NSX-T und einem Orchestator wie z.B. Kubernetes. Außerdem bietet es eine Integration zwischen NSX-T und Containerbasierten CaaS/PaaS-Plattformen wie z.B. Pivotal Container Service (PKS) und OpenShift.
* [Nuage](https://github.com/nuagenetworks/nuage-kubernetes/blob/v5.1.1-1/docs/kubernetes-1-installation.rst) ist eine SDN-Plattform die Policy-Basiertes Networking zwischen Kubernetes Pods und nicht-Kubernetes Umgebungen inklusive Sichtbarkeit und Security-Monitoring bereitstellt.
* [Romana](http://romana.io) ist eine Layer 3 Network-Lösung für Pod-Netzwerke welche auch die [NetworkPolicy API](/docs/concepts/services-networking/network-policies/) unterstützt. Details zur Installation als kubeadm Add-On sind [hier](https://github.com/romana/romana/tree/master/containerize) verfügbar.
* [Weave Net](https://www.weave.works/docs/net/latest/kube-addon/) bietet Networking and Network-Policies und arbeitet auf beiden Seiten der Network-Partition ohne auf eine externe Datenbank angwiesen zu sein.
## Service-Discovery
* [CoreDNS](https://coredns.io) ist ein flexibler, erweiterbarer DNS-Server der in einem Cluster [installiert](https://github.com/coredns/deployment/tree/master/kubernetes) werden kann und das Cluster-interne DNS für Pods bereitzustellen.
## Visualisierung &amp; Überwachung
* [Dashboard](https://github.com/kubernetes/dashboard#kubernetes-dashboard) ist ein Dashboard Web Interface für Kubernetes.
* [Weave Scope](https://www.weave.works/documentation/scope-latest-installing/#k8s) ist ein Tool um Container, Pods, Services usw. Grafisch zu visualieren. Kann in Verbindung mit einem [Weave Cloud Account](https://cloud.weave.works/) genutzt oder selbst gehosted werden.
## Infrastruktur
* [KubeVirt](https://kubevirt.io/user-guide/docs/latest/administration/intro.html#cluster-side-add-on-deployment) ist ein Add-On um Virtuelle Maschinen in Kubernetes auszuführen. Wird typischer auf Bare-Metal Clustern eingesetzt.
## Legacy Add-Ons
Es gibt einige weitere Add-Ons die in dem abgekündigten [cluster/addons](https://git.k8s.io/kubernetes/cluster/addons)-Verzeichnis dokumentiert sind.
Add-Ons die ordentlich gewartet werden dürfen gerne hier aufgezählt werden. Wir freuen uns auf PRs!
{{% /capture %}}
@@ -96,7 +96,7 @@ Das Google service Konto der Instanz hat einen `https://www.googleapis.com/auth/
Kubernetes eine native Unterstützung für die [Amazon Elastic Container Registry](https://aws.amazon.com/ecr/) wenn Knoten AWS EC2 Instanzen sind.
Es muss einfach nur der komplette Image Name (z.B. `ACCOUNT.dkr.ecr.REGION.amazonaws.com/imagename:tag`) in der Pod - Definition genutzt werden.
Es muss einfah nur der komplette Image Name (z.B. `ACCOUNT.dkr.ecr.REGION.amazonaws.com/imagename:tag`) in der Pod - Definition genutzt werden.
Alle Benutzer eines Clusters die Pods erstellen dürfen können dann jedes der Images in der ECR Registry zum Ausführen von Pods nutzen.
+1 -1
View File
@@ -3,7 +3,7 @@ title: Kubernetes Dokumentation
noedit: true
cid: docsHome
layout: docsportal_home
class: gridPage gridPageHome
class: gridPage
linkTitle: "Home"
main_menu: true
weight: 10
+3 -3
View File
@@ -20,11 +20,11 @@ Dieser Abschnitt der Kubernetes-Dokumentation enthält Referenzinformationen.
* [Kubernetes API Überblick](/docs/reference/using-api/api-overview/) - Übersicht über die API für Kubernetes.
* Kubernetes API Versionen
* [1.17](/docs/reference/generated/kubernetes-api/v1.17/)
* [1.16](/docs/reference/generated/kubernetes-api/v1.16/)
* [1.15](/docs/reference/generated/kubernetes-api/v1.15/)
* [1.14](/docs/reference/generated/kubernetes-api/v1.14/)
* [1.13](/docs/reference/generated/kubernetes-api/v1.13/)
* [1.12](/docs/reference/generated/kubernetes-api/v1.12/)
* [1.11](/docs/reference/generated/kubernetes-api/v1.11/)
* [1.10](/docs/reference/generated/kubernetes-api/v1.10/)
## API-Clientbibliotheken
+1 -1
View File
@@ -15,5 +15,5 @@ tags:
<!--more-->
Halten Sie immer einen Sicherungsplan für etcds Daten für Ihren Kubernetes-Cluster bereit. Ausführliche Informationen zu etcd finden Sie in der [etcd Dokumentation](https://etcd.io/docs).
Halten Sie immer einen Sicherungsplan für etcds Daten für Ihren Kubernetes-Cluster bereit. Ausführliche Informationen zu etcd finden Sie in der [etcd Dokumentation](https://github.com/coreos/etcd/blob/master/Documentation/docs.md).
@@ -27,7 +27,7 @@ source <(kubectl completion bash) # Wenn Sie autocomplete in bash in der aktuell
echo "source <(kubectl completion bash)" >> ~/.bashrc # Fügen Sie der Bash-Shell dauerhaft Autocomplete hinzu.
```
Sie können auch ein Abkürzungsalias für `kubectl` verwenden, welches auch mit Vervollständigung funktioniert:
Sie können auch ein Abkürzungsalias für `kubectl` verwenden, weleches auch mit Vervollständigung funktioniert:
```bash
alias k=kubectl
@@ -180,7 +180,7 @@ kubectl get events --sort-by=.metadata.creationTimestamp
## Ressourcen aktualisieren
Ab Version 1.11 ist das `rolling-update` veraltet (Lesen Sie [CHANGELOG-1.11.md](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.11.md) für weitere Informationen), verwenden Sie stattdessen `rollout`.
Ab Version 1.11 ist das `rolling-update` veraltet (Lesen Sie [CHANGELOG-1.11.md](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.11.md) für weitere Informationen), verwenden Sie stattdessen `rollout`.
```bash
kubectl set image deployment/frontend www=image:v2 # Fortlaufende Aktualisierung der "www" Container der "Frontend"-Bereitstellung, Aktualisierung des Images
+53 -73
View File
@@ -7,96 +7,76 @@ content_template: templates/concept
{{% capture overview %}}
Diese Sektion umfasst verschiedene Optionen zum Einrichten und Betrieb von Kubernetes.
Auf dieser Seite finden Sie die Lösung, die Ihren Anforderungen am besten entspricht.
Verschiedene Kubernetes Lösungen haben verschiedene Anforderungen: Einfache Wartung, Sicherheit, Kontrolle, verfügbare Resourcen und erforderliches Fachwissen zum Betrieb und zur Verwaltung dess folgende Diagramm zeigt die möglichen Abstraktionen eines Kubernetes-Clusters und ob eine Abstraktion selbst verwaltet oder von einem Anbieter verwaltet wird.
Sie können einen Kubernetes-Cluster auf einer lokalen Maschine, Cloud, On-Prem Datacenter bereitstellen; oder wählen Sie einen verwalteten Kubernetes-Cluster. Sie können auch eine individuelle Lösung über eine grosse Auswahl an Cloud Anbietern oder Bare-Metal-Umgebungen nutzen.
Noch einfacher können Sie einen Kubernetes-Cluster in einer Lern- und Produktionsumgebung erstellen.
Die Entscheidung, wo Kubernetes ausgeführt wird, hängt davon ab, welche Ressourcen Sie zur Verfügung haben und wie viel Flexibilität Sie benötigen.
Sie können Kubernetes fast überall ausführen, von Ihrem Laptop über VMs bei einem Cloud-Anbieter bis hin zu Bare-Metal-Servern.
Sie können einen vollständig verwalteten Cluster einrichten, indem Sie einen einzelnen Befehl ausführen oder einen eigenen benutzerdefinierten Cluster auf Ihren Bare-Metal-Servern erstellen.
{{% /capture %}}
{{% capture body %}}
## Lernumgebung
## Lösungen für lokale Maschinen
Benutzen Sie eine Docker-basierende Lösung, wenn Sie Kubernetes erlernen wollen: Von der Kubernetes-Community unterstützte Werkzeuge oder Werkzeuge in einem Ökosystem zum Einrichten eines Kubernetes-Clusters auf einer lokalen Maschine.
Eine lokale Lösung ist eine einfache Möglichkeit, mit Kubernetes anzufangen.
Sie können Kubernetes-Cluster erstellen und testen, ohne sich Gedanken über den Verbrauch von Cloud-Ressourcen und -Kontingenten machen zu müssen.
{{< table caption="Tabelle mit Lösungen für lokale Maschinen, in der die Tools aufgeführt sind, die von der Community und dem Ökosystem für die Bereitstellung von Kubernetes unterstützt werden." >}}
Sie sollten eine lokale Lösung auswählen, wenn Sie:
|Community |Ökosystem |
| ------------ | -------- |
| [Minikube](/docs/setup/learning-environment/minikube/) | [CDK on LXD](https://www.ubuntu.com/kubernetes/docs/install-local) |
| [kind (Kubernetes IN Docker)](https://github.com/kubernetes-sigs/kind) | [Docker Desktop](https://www.docker.com/products/docker-desktop)|
| | [Minishift](https://docs.okd.io/latest/minishift/)|
| | [MicroK8s](https://microk8s.io/)|
| | [IBM Cloud Private-CE (Community Edition)](https://github.com/IBM/deploy-ibm-cloud-private) |
| | [IBM Cloud Private-CE (Community Edition) on Linux Containers](https://github.com/HSBawa/icp-ce-on-linux-containers)|
| | [k3s](https://k3s.io)|
* Zum ausprobieren oder wenn Sie etwas über Kubernetes lernen möchten
* Cluster lokal entwickeln und testen
Wählen Sie eine [lokale Lösung](/de/docs/setup/) aus.
## Gehostete Lösungen
Gehostete Lösungen sind eine bequeme Möglichkeit, Kubernetes-Cluster zu erstellen und zu warten.
Sie verwalten und betreiben Ihre Cluster, so dass Sie dies nicht tun müssen.
Sie sollten eine gehostete Lösung auswählen, wenn Sie:
* Eine vollständig verwaltete Lösung wünschen
* Sie möchten sich auf die Entwicklung Ihrer Apps oder Dienste konzentrieren
* Sie haben kein dediziertes Site-Reliability-Engineering-Team (SRE), möchten jedoch eine hohe Verfügbarkeit
* Sie haben keine Ressourcen zum Hosten und Überwachen Ihrer Cluster
Wählen Sie eine [gehostete Lösung](/de/docs/setup/) aus.
## Schlüsselfertige Cloud-Lösungen
## Produktionsumgebung
Mit diesen Lösungen können Sie Kubernetes-Cluster mit nur wenigen Befehlen erstellen. Sie werden aktiv entwickelt und verfügen über eine Community-Unterstützung.
Sie können auch auf einer Reihe von Cloud-IaaS-Anbietern gehostet werden, und bieten mehr Freiheit und Flexibilität für eine Gegenleistung.
Überlegen Sie sich bei der Bewertung einer Lösung für eine Produktionsumgebung, welche Aspekte des Betriebs eines Kubernetes-Clusters (oder von _abstractions_) Sie selbst verwalten oder an einen Anbieter auslagern möchten.
Sie sollten eine schlüsselfertige Cloud-Lösung auswählen, wenn Sie:
Einige mögliche Abstraktionen von Kubernetes-Clustern sind {{< glossary_tooltip text="applications" term_id="applications" >}}, {{< glossary_tooltip text="data plane" term_id="data-plane" >}}, {{< glossary_tooltip text="control plane" term_id="control-plane" >}}, {{< glossary_tooltip text="cluster infrastructure" term_id="cluster-infrastructure" >}} und {{< glossary_tooltip text="cluster operations" term_id="cluster-operations" >}}.
* Mehr Kontrolle über Ihre Cluster haben möchten, als die gehosteten Lösungen zulassen
* Mehr operative Verantwortung übernehmen Möchten
Das folgende Diagramm zeigt die möglichen Abstraktionen eines Kubernetes-Clusters und ob eine Abstraktion selbst verwaltet oder von einem Anbieter verwaltet wird.
Wählen Sie eine [schlüsselfertige Cloud-Lösung](/de/docs/setup/) aus.
Lösungen für Produktionsumgebungen![Lösungen für Produktionsumgebungen](/images/docs/KubernetesSolutions.svg)
## Schlüsselfertige On-Premises-Lösungen
{{< table caption="Tabelle für Produktionsumgebungs-Lösungen listet Anbieter und deren Lösungen auf." >}}
Mit diesen Lösungen können Sie Kubernetes-Cluster mit nur wenigen Befehlen in Ihrem internen, sicheren Cloud-Netzwerk erstellen.
Die folgende Tabelle für Produktionsumgebungs-Lösungen listet Anbieter und deren Lösungen auf.
Sie sollten eine schlüsselfertige lokal betriebene Cloud-Lösung auswählen, wenn Sie:
|Providers | Managed | Turnkey cloud | On-Prem Datacenter | Custom (cloud) | Custom (On-premises VMs)| Custom (Bare Metal) |
| --------- | ------ | ------ | ------ | ------ | ------ | ----- |
| [Agile Stacks](https://www.agilestacks.com/products/kubernetes)| | &#x2714; | &#x2714; | | |
| [Alibaba Cloud](https://www.alibabacloud.com/product/kubernetes)| | &#x2714; | | | |
| [Amazon](https://aws.amazon.com) | [Amazon EKS](https://aws.amazon.com/eks/) |[Amazon EC2](https://aws.amazon.com/ec2/) | | | |
| [AppsCode](https://appscode.com/products/pharmer/) | &#x2714; | | | | |
| [APPUiO](https://appuio.ch/)  | &#x2714; | &#x2714; | &#x2714; | | | |
| [Banzai Cloud Pipeline Kubernetes Engine (PKE)](https://banzaicloud.com/products/pke/) | | &#x2714; | | &#x2714; | &#x2714; | &#x2714; |
| [CenturyLink Cloud](https://www.ctl.io/) | | &#x2714; | | | |
| [Cisco Container Platform](https://cisco.com/go/containers) | | | &#x2714; | | |
| [Cloud Foundry Container Runtime (CFCR)](https://docs-cfcr.cfapps.io/) | | | | &#x2714; |&#x2714; |
| [CloudStack](https://cloudstack.apache.org/) | | | | | &#x2714;|
| [Canonical](https://ubuntu.com/kubernetes) | &#x2714; | &#x2714; | &#x2714; | &#x2714; |&#x2714; | &#x2714;
| [Containership](https://containership.io) | &#x2714; |&#x2714; | | | |
| [D2iQ](https://d2iq.com/) | | [Kommander](https://d2iq.com/solutions/ksphere) | [Konvoy](https://d2iq.com/solutions/ksphere/konvoy) | [Konvoy](https://d2iq.com/solutions/ksphere/konvoy) | [Konvoy](https://d2iq.com/solutions/ksphere/konvoy) | [Konvoy](https://d2iq.com/solutions/ksphere/konvoy) |
| [Digital Rebar](https://provision.readthedocs.io/en/tip/README.html) | | | | | | &#x2714;
| [DigitalOcean](https://www.digitalocean.com/products/kubernetes/) | &#x2714; | | | | |
| [Docker Enterprise](https://www.docker.com/products/docker-enterprise) | |&#x2714; | &#x2714; | | | &#x2714;
| [Gardener](https://gardener.cloud/) | &#x2714; | &#x2714; | &#x2714; | &#x2714; | &#x2714; | [Custom Extensions](https://github.com/gardener/gardener/blob/master/docs/extensions/overview.md) |
| [Giant Swarm](https://www.giantswarm.io/) | &#x2714; | &#x2714; | &#x2714; | |
| [Google](https://cloud.google.com/) | [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine/) | [Google Compute Engine (GCE)](https://cloud.google.com/compute/)|[GKE On-Prem](https://cloud.google.com/gke-on-prem/) | | | | | | | |
| [IBM](https://www.ibm.com/in-en/cloud) | [IBM Cloud Kubernetes Service](https://cloud.ibm.com/kubernetes/catalog/cluster)| |[IBM Cloud Private](https://www.ibm.com/in-en/cloud/private) | |
| [Ionos](https://www.ionos.com/enterprise-cloud) | [Ionos Managed Kubernetes](https://www.ionos.com/enterprise-cloud/managed-kubernetes) | [Ionos Enterprise Cloud](https://www.ionos.com/enterprise-cloud) | |
| [Kontena Pharos](https://www.kontena.io/pharos/) | |&#x2714;| &#x2714; | | |
| [KubeOne](https://kubeone.io/) | | &#x2714; | &#x2714; | &#x2714; | &#x2714; | &#x2714; |
| [Kubermatic](https://kubermatic.io/) | &#x2714; | &#x2714; | &#x2714; | &#x2714; | &#x2714; | |
| [KubeSail](https://kubesail.com/) | &#x2714; | | | | |
| [Kubespray](https://kubespray.io/#/) | | | |&#x2714; | &#x2714; | &#x2714; |
| [Kublr](https://kublr.com/) |&#x2714; | &#x2714; |&#x2714; |&#x2714; |&#x2714; |&#x2714; |
| [Microsoft Azure](https://azure.microsoft.com) | [Azure Kubernetes Service (AKS)](https://azure.microsoft.com/en-us/services/kubernetes-service/) | | | | |
| [Mirantis Cloud Platform](https://www.mirantis.com/software/kubernetes/) | | | &#x2714; | | |
| [Nirmata](https://www.nirmata.com/) | | &#x2714; | &#x2714; | | |
| [Nutanix](https://www.nutanix.com/en) | [Nutanix Karbon](https://www.nutanix.com/products/karbon) | [Nutanix Karbon](https://www.nutanix.com/products/karbon) | | | [Nutanix AHV](https://www.nutanix.com/products/acropolis/virtualization) |
| [OpenNebula](https://www.opennebula.org) |[OpenNebula Kubernetes](https://marketplace.opennebula.systems/docs/service/kubernetes.html) | | | | |
| [OpenShift](https://www.openshift.com) |[OpenShift Dedicated](https://www.openshift.com/products/dedicated/) and [OpenShift Online](https://www.openshift.com/products/online/) | | [OpenShift Container Platform](https://www.openshift.com/products/container-platform/) | | [OpenShift Container Platform](https://www.openshift.com/products/container-platform/) |[OpenShift Container Platform](https://www.openshift.com/products/container-platform/)
| [Oracle Cloud Infrastructure Container Engine for Kubernetes (OKE)](https://docs.cloud.oracle.com/iaas/Content/ContEng/Concepts/contengoverview.htm) | &#x2714; | &#x2714; | | | |
| [oVirt](https://www.ovirt.org/) | | | | | &#x2714; |
| [Pivotal](https://pivotal.io/) | | [Enterprise Pivotal Container Service (PKS)](https://pivotal.io/platform/pivotal-container-service) | [Enterprise Pivotal Container Service (PKS)](https://pivotal.io/platform/pivotal-container-service) | | |
| [Platform9](https://platform9.com/) | [Platform9 Managed Kubernetes](https://platform9.com/managed-kubernetes/) | | [Platform9 Managed Kubernetes](https://platform9.com/managed-kubernetes/) | &#x2714; | &#x2714; | &#x2714;
| [Rancher](https://rancher.com/) | | [Rancher 2.x](https://rancher.com/docs/rancher/v2.x/en/) | | [Rancher Kubernetes Engine (RKE)](https://rancher.com/docs/rke/latest/en/) | | [k3s](https://k3s.io/)
| [StackPoint](https://stackpoint.io/)  | &#x2714; | &#x2714; | | | |
| [Supergiant](https://supergiant.io/) | |&#x2714; | | | |
| [SUSE](https://www.suse.com/) | | &#x2714; | | | |
| [SysEleven](https://www.syseleven.io/) | &#x2714; | | | | |
| [Tencent Cloud](https://intl.cloud.tencent.com/) | [Tencent Kubernetes Engine](https://intl.cloud.tencent.com/product/tke) | &#x2714; | &#x2714; | | | &#x2714; |
| [VEXXHOST](https://vexxhost.com/) | &#x2714; | &#x2714; | | | |
| [VMware](https://cloud.vmware.com/) | [VMware Cloud PKS](https://cloud.vmware.com/vmware-cloud-pks) |[VMware Enterprise PKS](https://cloud.vmware.com/vmware-enterprise-pks) | [VMware Enterprise PKS](https://cloud.vmware.com/vmware-enterprise-pks) | [VMware Essential PKS](https://cloud.vmware.com/vmware-essential-pks) | |[VMware Essential PKS](https://cloud.vmware.com/vmware-essential-pks)
| [Z.A.R.V.I.S.](https://zarvis.ai/) | &#x2714; | | | | | |
* Cluster in Ihrem privaten Cloud-Netzwerk bereitstellen möchten
* Ein engagiertes SRE-Team haben
* Über die Ressourcen zum Hosten und Überwachen Ihrer Cluster verfügen
Wählen Sie eine [schlüsselfertige On-Premises-Lösung](/de/docs/setup/) aus.
## Individuelle Lösungen
Individuelle Lösungen geben Ihnen die größte Freiheit in Ihren Clustern, erfordern jedoch das meiste Know-how.
Diese Lösungen reichen von Bare-Metal-Anbietern bis hin zu Cloud-Anbietern mit unterschiedlichen Betriebssystemen.
Wählen Sie eine [individuelle Lösung](/de/docs/setup/) aus.
{{% /capture %}}
{{% capture whatsnext %}}
Gehen Sie zu [Auswählen der richtigen Lösung](/de/docs/setup/) für eine vollständige Liste der möglichen Lösungen.
{{% /capture %}}
@@ -0,0 +1,4 @@
---
title: Indivduelle Cloud-Lösungen
weight: 50
---
+5
View File
@@ -0,0 +1,5 @@
---
title: "Cluster mit kubeadm bootstrappen"
weight: 30
---
+1 -1
View File
@@ -205,7 +205,7 @@ Weitere Informationen zu unterstützten Treibern und zur Installation von Plugin
### Lokale Images durch erneute Verwendung des Docker-Daemon ausführen
Wenn Sie eine einzige Kubernetes VM verwenden, ist es sehr praktisch, den integrierten Docker-Daemon von Minikube wiederzuverwenden; Dies bedeutet, dass Sie auf Ihrem lokalen Computer keine Docker-Registy erstellen und das Image in die Registry importieren müssen - Sie können einfach innerhalb desselben Docker-Daemons wie Minikube arbeiten, was lokale Experimente beschleunigt. Stellen Sie einfach sicher, dass Sie Ihr Docker-Image mit einem anderen Element als 'latest' versehen, und verwenden Sie dieses Tag, wenn Sie das Image laden. Andernfalls, wenn Sie keine Version Ihres Images angeben, wird es als `:latest` angenommen, mit der Pull-Image-Richtlinie von `Always` entsprechend, was schließlich zu `ErrImagePull` führen kann, da Sie möglicherweise noch keine Versionen Ihres Docker-Images in der Standard-Docker-Registry (normalerweise DockerHub) haben.
Wenn Sie eine einzige Kubernetes VM verwenden, ist es sehr praktisch, den integrierten Docker-Daemon von Minikube wiederzuverwenden; Dies bedeutet, dass Sie auf Ihrem lokalen Computer keine Docker-Registy erstellen und das Image in die Registry importortieren müssen - Sie können einfach innerhalb desselben Docker-Daemons wie Minikube arbeiten, was lokale Experimente beschleunigt. Stellen Sie einfach sicher, dass Sie Ihr Docker-Image mit einem anderen Element als 'latest' versehen, und verwenden Sie dieses Tag, wenn Sie das Image laden. Andernfalls, wenn Sie keine Version Ihres Images angeben, wird es als `:latest` angenommen, mit der Pull-Image-Richtlinie von `Always` entsprechend, was schließlich zu `ErrImagePull` führen kann, da Sie möglicherweise noch keine Versionen Ihres Docker-Images in der Standard-Docker-Registry (normalerweise DockerHub) haben.
Um mit dem Docker-Daemon auf Ihrem Mac/Linux-Computer arbeiten zu können, verwenden Sie den `docker-env`-Befehl in Ihrer Shell:
@@ -0,0 +1,4 @@
---
title: Lokale VMs
weight: 60
---
+4
View File
@@ -0,0 +1,4 @@
---
title: Schlüsselfertige Cloud-Lösungen
weight: 40
---
+4
View File
@@ -0,0 +1,4 @@
---
title: "Windows in Kubernetes"
weight: 65
---
@@ -49,7 +49,7 @@ Minikube unterstützt auch die Option `--vm-driver=none`, mit der die Kubernetes
Die einfachste Möglichkeit, Minikube unter macOS zu installieren, ist die Verwendung von [Homebrew](https://brew.sh):
```shell
brew install minikube
brew cask install minikube
```
Sie können es auch auf macOS installieren, indem Sie eine statische Binärdatei herunterladen:
+1 -1
View File
@@ -145,7 +145,7 @@ Um den "Hallo-Welt"-Container außerhalb des virtuellen Netzwerks von Kubernetes
```
Bei Cloud-Anbietern, die Load-Balancer unterstützen, wird eine externe IP-Adresse für den Zugriff auf den Dienst bereitgestellt.
Bei Minikube ermöglicht der Typ `LoadBalancer` den Dienst über den Befehl `minikube service` verfügbar zu machen.
Bei Minikube ermöglicht der Typ `LoadBalancer` den Dienst über den Befehl `minikube service` verfuügbar zu machen.
3. Führen Sie den folgenden Befehl aus:
+2 -2
View File
@@ -45,12 +45,12 @@ Kubernetes is open source giving you the freedom to take advantage of on-premise
<br>
<br>
<br>
<a href="https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/?utm_source=kubernetes.io&utm_medium=nav&utm_campaign=kccnceu20" button id="desktopKCButton">Attend KubeCon in Amsterdam in July/August TBD</a>
<a href="https://events.linuxfoundation.org/events/kubecon-cloudnativecon-europe-2020/" button id="desktopKCButton">Attend KubeCon in Amsterdam on Mar. 30-Apr. 2, 2020</a>
<br>
<br>
<br>
<br>
<a href="https://events.linuxfoundation.org/kubecon-cloudnativecon-north-america/?utm_source=kubernetes.io&utm_medium=nav&utm_campaign=kccncna20" button id="desktopKCButton">Attend KubeCon in Boston on November 17-20, 2020</a>
<a href="https://events.linuxfoundation.cn/kubecon-cloudnativecon-open-source-summit-china/" button id="desktopKCButton">Attend KubeCon in Shanghai on July 28-30, 2020</a>
</div>
<div id="videoPlayer">
<iframe data-url="https://www.youtube.com/embed/H06qrNmGqyE?autoplay=1" frameborder="0" allowfullscreen></iframe>
@@ -48,7 +48,7 @@ The following applications are now available:
1. [Create a chart](https://github.com/kubernetes/helm/blob/master/docs/charts.md)
2. Developer provides parameters via the [values.yaml](https://github.com/kubernetes/helm/blob/master/docs/charts.md#values-files) file allowing users to customize their deployment. This can be seen as the API between chart devs and chart users.
3. A [README](https://github.com/kubernetes/charts/tree/master/stable/mariadb) is written to help describe the application and its parameterized values.
4. Once the application installs properly and the values customize the deployment appropriately, the developer adds a [NOTES.txt](https://github.com/helm/helm/blob/dev-v2/docs/charts.md) file that is shown as soon as the user installs. This file generally points out the next steps for the user to connect to or use the application.
4. Once the application installs properly and the values customize the deployment appropriately, the developer adds a [NOTES.txt](https://github.com/kubernetes/helm/blob/master/docs/charts.md#chart-license-readme-and-notes) file that is shown as soon as the user installs. This file generally points out the next steps for the user to connect to or use the application.
5. If the application requires persistent storage, the developer adds a mechanism to store the data such that pod restarts do not lose data. Most charts requiring this today are using [dynamic volume provisioning](https://kubernetes.io/blog/2016/10/dynamic-provisioning-and-storage-in-kubernetes) to abstract away underlying storage details from the user which allows a single configuration to work against Kubernetes installations.
6. Submit a [Pull Request to the Kubernetes Charts repo](https://github.com/kubernetes/charts/pulls). Once tested and reviewed, the PR will be merged.
7. Once merged to the master branch, the chart will be packaged and released to Helms default repository and available for users to install.
@@ -5,7 +5,7 @@ tags:
date: 2018-03-26
modified_time: '2018-03-27T11:01:39.569-07:00'
blogger_id: tag:blogger.com,1999:blog-112706738355446097.post-6519705795358457586
blogger_orig_url: https://kubernetes.io/blog/2018/03/26/kubernetes-1.10-stabilizing-storage-security-networking/
blogger_orig_url: https://kubernetes.io/blog/2018/03/kubernetes-1.10-stabilizing-storage-security-networking
slug: kubernetes-1.10-stabilizing-storage-security-networking
date: 2018-03-26
---
@@ -85,7 +85,7 @@ That way, your clients can always use the short form on the left, and always be
As further reading, a more elaborate example for users is available in the [Multi-Cluster Service DNS with ExternalDNS guide](https://github.com/kubernetes-sigs/federation-v2/blob/master/docs/servicedns-with-externaldns.md).
# Try it yourself
To get started with Federation v2, please refer to the [user guide](https://github.com/kubernetes-sigs/federation-v2/blob/master/docs/userguide.md). Deployment can be accomplished with a [Helm chart](https://github.com/kubernetes-sigs/kubefed/blob/master/charts/kubefed/README.md), and once the control plane is available, the [user guides example](https://github.com/kubernetes-sigs/federation-v2/blob/master/docs/userguide.md#example) can be used to get some hands-on experience with using Federation V2.
To get started with Federation v2, please refer to the [user guide](https://github.com/kubernetes-sigs/federation-v2/blob/master/docs/userguide.md). Deployment can be accomplished with a [Helm chart](https://github.com/kubernetes-sigs/federation-v2/blob/master/charts/federation-v2/README.md), and once the control plane is available, the [user guides example](https://github.com/kubernetes-sigs/federation-v2/blob/master/docs/userguide.md#example) can be used to get some hands-on experience with using Federation V2.
Federation v2 can be deployed in both _cluster-scoped_ and _namespace-scoped_ configurations. A cluster-scoped deployment will require cluster-admin privileges to both host and member clusters, and may be a good fit for evaluating federation on clusters that are not running critical workloads. Namespace-scoped deployment requires access to only a single namespace on host and member clusters, and is a better fit for evaluating federation on clusters running workloads. Most of the user guide refers to cluster-scoped deployment, with the [namespaced federation](https://github.com/kubernetes-sigs/federation-v2/blob/master/docs/userguide.md#namespaced-federation) section documenting how use of a namespaced deployment differs. The same cluster can host multiple federations, and clusters can be part of multiple federations when using namespaced federation.
@@ -12,45 +12,21 @@ When APIs evolve, the old API is deprecated and eventually removed.
The **v1.16** release will stop serving the following deprecated API versions in favor of newer and more stable API versions:
* NetworkPolicy in the **extensions/v1beta1** API version is no longer served
* Migrate to use the **networking.k8s.io/v1** API version, available since v1.8.
Existing persisted data can be retrieved/updated via the new version.
* PodSecurityPolicy in the **extensions/v1beta1** API version
* NetworkPolicy (in the **extensions/v1beta1** API group)
* Migrate to use the **networking.k8s.io/v1** API, available since v1.8.
Existing persisted data can be retrieved/updated via the **networking.k8s.io/v1** API.
* PodSecurityPolicy (in the **extensions/v1beta1** API group)
* Migrate to use the **policy/v1beta1** API, available since v1.10.
Existing persisted data can be retrieved/updated via the new version.
* DaemonSet in the **extensions/v1beta1** and **apps/v1beta2** API versions is no longer served
* Migrate to use the **apps/v1** API version, available since v1.9.
Existing persisted data can be retrieved/updated via the new version.
* Notable changes:
* `spec.templateGeneration` is removed
* `spec.selector` is now required and immutable after creation; use the existing template labels as the selector for seamless upgrades
* `spec.updateStrategy.type` now defaults to `RollingUpdate` (the default in `extensions/v1beta1` was `OnDelete`)
* Deployment in the **extensions/v1beta1**, **apps/v1beta1**, and **apps/v1beta2** API versions is no longer served
* Migrate to use the **apps/v1** API version, available since v1.9.
Existing persisted data can be retrieved/updated via the new version.
* Notable changes:
* `spec.rollbackTo` is removed
* `spec.selector` is now required and immutable after creation; use the existing template labels as the selector for seamless upgrades
* `spec.progressDeadlineSeconds` now defaults to `600` seconds (the default in `extensions/v1beta1` was no deadline)
* `spec.revisionHistoryLimit` now defaults to `10` (the default in `apps/v1beta1` was `2`, the default in `extensions/v1beta1` was to retain all)
* `maxSurge` and `maxUnavailable` now default to `25%` (the default in `extensions/v1beta1` was `1`)
* StatefulSet in the **apps/v1beta1** and **apps/v1beta2** API versions is no longer served
* Migrate to use the **apps/v1** API version, available since v1.9.
Existing persisted data can be retrieved/updated via the new version.
* Notable changes:
* `spec.selector` is now required and immutable after creation; use the existing template labels as the selector for seamless upgrades
* `spec.updateStrategy.type` now defaults to `RollingUpdate` (the default in `apps/v1beta1` was `OnDelete`)
* ReplicaSet in the **extensions/v1beta1**, **apps/v1beta1**, and **apps/v1beta2** API versions is no longer served
* Migrate to use the **apps/v1** API version, available since v1.9.
Existing persisted data can be retrieved/updated via the new version.
* Notable changes:
* `spec.selector` is now required and immutable after creation; use the existing template labels as the selector for seamless upgrades
Existing persisted data can be retrieved/updated via the **policy/v1beta1** API.
* DaemonSet, Deployment, StatefulSet, and ReplicaSet (in the **extensions/v1beta1** and **apps/v1beta2** API groups)
* Migrate to use the **apps/v1** API, available since v1.9.
Existing persisted data can be retrieved/updated via the **apps/v1** API.
The **v1.22** release will stop serving the following deprecated API versions in favor of newer and more stable API versions:
The **v1.20** release will stop serving the following deprecated API versions in favor of newer and more stable API versions:
* Ingress in the **extensions/v1beta1** API version will no longer be served
* Migrate to use the **networking.k8s.io/v1beta1** API version, available since v1.14.
Existing persisted data can be retrieved/updated via the new version.
* Ingress (in the **extensions/v1beta1** API group)
* Migrate to use the **networking.k8s.io/v1beta1** API, serving Ingress since v1.14.
Existing persisted data can be retrieved/updated via the **networking.k8s.io/v1beta1** API.
# What To Do
@@ -84,8 +60,8 @@ apiserver startup arguments:
Deprecations are announced in the Kubernetes release notes. You can see these
announcements in
[1.14](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.14.md#deprecations)
and [1.15](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.15.md#deprecations-and-removals).
[1.14](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.14.md#deprecations)
and [1.15](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.15.md#deprecations-and-removals).
You can read more [in our deprecation policy document](https://kubernetes.io/docs/reference/using-api/deprecation-policy/#deprecating-parts-of-the-api)
about the deprecation policies for Kubernetes APIs, and other Kubernetes components.
@@ -92,7 +92,7 @@ Verify that the pod is RUNNING after some time
kubectl get pods web-server
```
```
```shell
NAME READY STATUS RESTARTS AGE
web-server 1/1 Running 0 39s
```
@@ -103,7 +103,7 @@ To confirm that the CSI driver is actually serving your requests it may be prude
kubectl logs {CSIdriverPodName} --container={CSIdriverContainerName}
```
```
```shell
/csi.v1.Controller/ControllerPublishVolume called with request: ...
Attaching disk ... to ...
ControllerPublishVolume succeeded for disk ... to instance ...
@@ -7,7 +7,7 @@ slug: Remembering-Brad-Childs
**Authors:** Paul Morie, Red Hat
Last year, the Kubernetes family lost one of its own. Brad Childs was a
Earlier this year, the Kubernetes family lost one of its own. Brad Childs was a
SIG Storage chair and long time contributor to the project. Brad worked on a
number of features in storage and was known as much for his friendliness and
sense of humor as for his technical contributions and leadership.
@@ -1,56 +0,0 @@
---
layout: blog
title: "Announcing the Kubernetes bug bounty program"
date: 2020-01-14T09:00:00-08:00
slug: kubernetes-bug-bounty-announcement
---
**Authors:** Maya Kaczorowski and Tim Allclair, Google, on behalf of the [Kubernetes Product Security Committee](https://github.com/kubernetes/community/tree/master/committee-product-security)
Today, the [Kubernetes Product Security Committee](https://github.com/kubernetes/community/tree/master/committee-product-security) is launching a [new bug bounty program](https://hackerone.com/kubernetes), funded by the [CNCF](https://www.cncf.io/), to reward researchers finding security vulnerabilities in Kubernetes.
## Setting up a new bug bounty program
We aimed to set up this bug bounty program as transparently as possible, with [an initial proposal](https://docs.google.com/document/d/1dvlQsOGODhY3blKpjTg6UXzRdPzv5y8V55RD_Pbo7ag/edit#heading=h.7t1efwpev42p), [evaluation of vendors](https://github.com/kubernetes/kubernetes/issues/73079), and [working draft of the components in scope](https://github.com/kubernetes/community/blob/master/contributors/guide/bug-bounty.md). Once we onboarded the selected bug bounty program vendor, [HackerOne](https://www.hackerone.com/), these documents were further refined based on the feedback from HackerOne, as well as what was learned in the recent [Kubernetes security audit](https://github.com/kubernetes/community/blob/master/wg-security-audit/findings/Kubernetes%20Final%20Report.pdf). The bug bounty program has been in a private release for several months now, with invited researchers able to submit bugs and help us test the triage process. After almost two years since the initial proposal, the program is now ready for all security researchers to contribute!
Whats exciting is that this is rare: a bug bounty for an open-source infrastructure tool. Some open-source bug bounty programs exist, such as the [Internet Bug Bounty](https://internetbugbounty.org/), this mostly covers core components that are consistently deployed across environments; but most bug bounties are still for hosted web apps. In fact, with more than[ 100 certified distributions of Kubernetes](https://www.cncf.io/certification/kcsp/), the bug bounty program needs to apply to the Kubernetes code that powers all of them. By far, the most time-consuming challenge here has been ensuring that the program provider (HackerOne) and their researchers who do the first line triage have the awareness of Kubernetes and the ability to easily test the validity of a reported bug. As part of the bootstrapping process, HackerOne had their team pass the [Certified Kubernetes Administrator](https://www.cncf.io/certification/cka/) (CKA) exam.
## Whats in scope
The bug bounty scope covers code from the main Kubernetes organizations on GitHub, as well as continuous integration, release, and documentation artifacts. Basically, most content youd think of as core Kubernetes, included at [https://github.com/kubernetes](https://github.com/kubernetes), is in scope. Were particularly interested in cluster attacks, such as privilege escalations, authentication bugs, and remote code execution in the kubelet or API server. Any information leak about a workload, or unexpected permission changes is also of interest. Stepping back from the cluster admins view of the world, youre also encouraged to look at the Kubernetes supply chain, including the build and release processes, which would allow any unauthorized access to commits, or the ability to publish unauthorized artifacts.
Notably out of scope is the community management tooling, e.g., the Kubernetes mailing lists or Slack channel. Container escapes, attacks on the Linux kernel, or other dependencies, such as etcd, are also out of scope and should be reported to the appropriate party. We would still appreciate that any Kubernetes vulnerability, even if not in scope for the bug bounty, be [disclosed privately](https://kubernetes.io/docs/reference/issues-security/security/#report-a-vulnerability) to the Kubernetes Product Security Committee. See the full scope on the [program reporting page](https://hackerone.com/kubernetes).
## How Kubernetes handles vulnerabilities and disclosures
Kubernetes [Product Security Committee](https://github.com/kubernetes/community/tree/master/committee-product-security) is a group of security-focused maintainers who are responsible for receiving and responding to reports of security issues in Kubernetes. This follows the documented [security vulnerability response process](https://kubernetes.io/docs/reference/issues-security/security/), which includes initial triage, assessing impact, generating and rolling out a fix.
With our bug bounty program, initial triage and initial assessment are handled by the bug bounty provider, in this case, HackerOne, enabling us better scale our limited Kubernetes security experts to handle only valid reports. Nothing else in this process is changing - the Product Security Committee will continue to develop fixes, build private patches, and coordinate special security releases. New releases with security patches will be announced at [kubernetes-security-announce@googlegroups.com](https://groups.google.com/forum/#!forum/kubernetes-security-announce).
If you want to report a bug, you dont need to use the bug bounty - you can still follow the [existing process](https://kubernetes.io/docs/reference/issues-security/security/#report-a-vulnerability) and report what youve found at [security@kubernetes.io](mailto:security@kubernetes.io).
## Get started
Just as many organizations support open source by hiring developers, paying bug bounties directly supports security researchers. This bug bounty is a critical step for Kubernetes to build up its community of security researchers and reward their hard work.
If youre a security researcher, and new to Kubernetes, check out these resources to learn more and get started bug hunting:
* **Hardening guides**
* Kubernetes.io: [https://kubernetes.io/docs/tasks/administer-cluster/securing-a-cluster/](https://kubernetes.io/docs/tasks/administer-cluster/securing-a-cluster/)
* **Frameworks**
* CIS benchmarks: [https://www.cisecurity.org/benchmark/kubernetes/](https://www.cisecurity.org/benchmark/kubernetes/)
* NIST 800-190: [https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-190.pdf](https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-190.pdf)
* **Talks**
* The Devil in the Details: Kubernetes First Security Assessment (KubeCon NA 2019): [https://www.youtube.com/watch?v=vknE5XEa_Do](https://www.youtube.com/watch?v=vknE5XEa_Do)
* Crafty Requests: Deep Dive into Kubernetes CVE-2018-1002105 (KubeCon EU 2019): [https://www.youtube.com/watch?v=VjSJqc13PNk](https://www.youtube.com/watch?v=VjSJqc13PNk)
* A Hackers Guide to Kubernetes and the Cloud (KubeCon EU 2018): [https://www.youtube.com/watch?v=dxKpCO2dAy8](https://www.youtube.com/watch?v=dxKpCO2dAy8)
* Shipping in pirate-infested waters (KubeCon NA 2017): [https://www.youtube.com/watch?v=ohTq0no0ZVU](https://www.youtube.com/watch?v=ohTq0no0ZVU)
* Hacking and Hardening Kubernetes clusters by example (KubeCon NA 2017): [https://www.youtube.com/watch?v=vTgQLzeBfRU](https://www.youtube.com/watch?v=vTgQLzeBfRU)
If you find something, please report a security bug to the Kubernetes bug bounty at [https://hackerone.com/kubernetes](https://hackerone.com/kubernetes).
<!-- Docs to Markdown version 1.0β17 -->
@@ -1,143 +0,0 @@
---
layout: blog
title: "Kubernetes on MIPS"
date: 2020-01-15
slug: Kubernetes-on-MIPS
---
**Authors:** TimYin Shi, Dominic Yin, Wang Zhan, Jessica Jiang, Will Cai, Jeffrey Gao, Simon Sun (Inspur)
## Background
[MIPS](https://en.wikipedia.org/wiki/MIPS_architecture) (Microprocessor without Interlocked Pipelined Stages) is a reduced instruction set computer (RISC) instruction set architecture (ISA), appeared in 1981 and developed by MIPS Technologies. Now MIPS architecture is widely used in many electronic products.
[Kubernetes](https://kubernetes.io) has officially supported a variety of CPU architectures such as x86, arm/arm64, ppc64le, s390x. However, it's a pity that Kubernetes doesn't support MIPS. With the widespread use of cloud native technology, users under MIPS architecture also have an urgent demand for Kubernetes on MIPS.
## Achievements
For many years, to enrich the ecology of the open-source community, we have been working on adjusting MIPS architecture for Kubernetes use cases. With the continuous iterative optimization and the performance improvement of the MIPS CPU, we have made some breakthrough progresses on the mips64el platform.
Over the years, we have been actively participating in the Kubernetes community and have rich experience in the using and optimization of Kubernetes technology. Recently, we tried to adapt the MIPS architecture platform for Kubernetes and achieved a new a stage on that journey. The team has completed migration and adaptation of Kubernetes and related components, built not only a stable and highly available MIPS cluster but also completed the conformance test for Kubernetes v1.16.2.
![Kubernetes on MIPS](/images/blog/2020-01-15-Kubernetes-on-MIPS/kubernetes-on-mips.png)
_Figure 1 Kubernetes on MIPS_
## K8S-MIPS component build
Almost all native cloud components related to Kubernetes do not provide a MIPS version installation package or image. The prerequisite of deploying Kubernetes on the MIPS platform is to compile and build all required components on the mips64el platform. These components include:
- golang
- docker-ce
- hyperkube
- pause
- etcd
- calico
- coredns
- metrics-server
Thanks to the excellent design of Golang and its good support for the MIPS platform, the compilation processes of the above cloud native components are greatly simplified. First of all, we compiled Golang on the latest stable version for the mips64el platform, and then we compiled most of the above components with source code.
During the compilation processes, we inevitably encountered many platform compatibility problems, such as a Golang system call compatibility problem (syscall), typecasting of syscall. Stat_t from uint32 to uint64, patching for EpollEvent, and so on.
To build K8S-MIPS components, we used cross-compilation technology. Our process involved integrating a QEMU tool to translate MIPS CPU instructions and modifying the build script of Kubernetes and E2E image script of Kubernetes, Hyperkube, and E2E test images on MIPS architecture.
After successfully building the above components, we use tools such as kubespray and kubeadm to complete kubernetes cluster construction.
| Name | Version | MIPS Repository |
| ------------------------------ | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| golang on MIPS | 1.12.5 | - |
| docker-ce on MIPS | 18.09.8 | - |
| metrics-server for CKE on MIPS | 0.3.2 | `registry.inspurcloud.cn/library/cke/kubernetes/metrics-server-mips64el:v0.3.2` |
| etcd for CKE on MIPS | 3.2.26 | `registry.inspurcloud.cn/library/cke/etcd/etcd-mips64el:v3.2.26` |
| pause for CKE on MIPS | 3.1 | `registry.inspurcloud.cn/library/cke/kubernetes/pause-mips64el:3.1` |
| hyperkube for CKE on MIPS | 1.14.3 | `registry.inspurcloud.cn/library/cke/kubernetes/hyperkube-mips64el:v1.14.3` |
| coredns for CKE on MIPS | 1.6.5 | `registry.inspurcloud.cn/library/cke/kubernetes/coredns-mips64el:v1.6.5` |
| calico for CKE on MIPS | 3.8.0 | `registry.inspurcloud.cn/library/cke/calico/cni-mips64el:v3.8.0` `registry.inspurcloud.cn/library/cke/calico/ctl-mips64el:v3.8.0` `registry.inspurcloud.cn/library/cke/calico/node-mips64el:v3.8.0` `registry.inspurcloud.cn/library/cke/calico/kube-controllers-mips64el:v3.8.0` |
**Note**: CKE is a Kubernetes-based cloud container engine launched by Inspur
![K8S-MIPS Cluster Components](/images/blog/2020-01-15-Kubernetes-on-MIPS/k8s-mips-cluster-components.png)
_Figure 2 K8S-MIPS Cluster Components_
![CPU Architecture](/images/blog/2020-01-15-Kubernetes-on-MIPS/cpu-architecture.png)
_Figure 3 CPU Architecture_
![Cluster Node Information](/images/blog/2020-01-15-Kubernetes-on-MIPS/cluster-node-information.png)
_Figure 4 Cluster Node Information_
## Run K8S Conformance Test
The most straightforward way to verify the stability and availability of the K8S-MIPS cluster is to run a Kubernetes [conformance test](https://github.com/kubernetes/kubernetes/blob/v1.16.2/cluster/images/conformance/README.md).
Conformance is a standalone container to launch Kubernetes end-to-end tests for conformance testing.
Once the test has started, it launches several pods for various end-to-end tests. The source code of those images used by these pods is mostly from `kubernetes/test/images`, and the built images are at `gcr.io/kubernetes-e2e-test-images`. Since there are no MIPS images in the repository, we must first build all needed images to run the test.
### Build needed images for test
The first step is to find all needed images for the test. We can run `sonobuoy images-p e2e` command to list all images, or we can find those images in [/test/utils/image/manifest.go](https://github.com/kubernetes/kubernetes/blob/master/test/utils/image/manifest.go). Although Kubernetes officially has a complete Makefile and shell-script that provides commands for building test images, there are still a number of architecture-related issues that have not been resolved, such as the incompatibilities of base images and dependencies. So we cannot directly build mips64el architecture images by executing these commands.
Most test images are in golang, then compiled into binaries and built as Docker image based on the corresponding Dockerfile. These images are easy to build. But note that most images are using alpine as their base image, which does not officially support mips64el architecture for now. For this moment, we are unable to make mips64el version of [alpine](https://www.alpinelinux.org/), so we have to replace the alpine to existing MIPS images, such as Debian-stretch, fedora, ubuntu. Replacing the base image also requires replacing the command to install the dependencies, even the version of these dependencies.
Some images are not in `kubernetes/test/images`, such as `gcr.io/google-samples/gb-frontend:v6`. There is no clear documentation explaining where these images are locaated, though we found the source code in repository [github.com/GoogleCloudPlatform/kubernetes-engine-samples](https://github.com/GoogleCloudPlatform/kubernetes-engine-samples). We soon ran into new problems: to build these google sample images, we have to build the base image it uses, even the base image of the base images, such as `php:5-apache`, `redis`, and `perl`.
After a long process of building an image, we finished with about four dozen images, including the images used by the test pod, and the base images. The last step before we run the tests is to place all those images into every node in the cluster and make sure the Pod image pull policy is `imagePullPolicy: ifNotPresent`.
Here are some of the images we built
- `docker.io/library/busybox:1.29`
- `docker.io/library/nginx:1.14-alpine`
- `docker.io/library/nginx:1.15-alpine`
- `docker.io/library/perl:5.26`
- `docker.io/library/httpd:2.4.38-alpine`
- `docker.io/library/redis:5.0.5-alpine`
- `gcr.io/google-containers/conformance:v1.16.2`
- `gcr.io/google-containers/hyperkube:v1.16.2`
- `gcr.io/google-samples/gb-frontend:v6`
- `gcr.io/kubernetes-e2e-test-images/agnhost:2.6`
- `gcr.io/kubernetes-e2e-test-images/apparmor-loader:1.0`
- `gcr.io/kubernetes-e2e-test-images/dnsutils:1.1`
- `gcr.io/kubernetes-e2e-test-images/echoserver:2.2`
- `gcr.io/kubernetes-e2e-test-images/ipc-utils:1.0`
- `gcr.io/kubernetes-e2e-test-images/jessie-dnsutils:1.0`
- `gcr.io/kubernetes-e2e-test-images/kitten:1.0`
- `gcr.io/kubernetes-e2e-test-images/metadata-concealment:1.2`
- `gcr.io/kubernetes-e2e-test-images/mounttest-user:1.0`
- `gcr.io/kubernetes-e2e-test-images/mounttest:1.0`
- `gcr.io/kubernetes-e2e-test-images/nautilus:1.0`
- `gcr.io/kubernetes-e2e-test-images/nonewprivs:1.0`
- `gcr.io/kubernetes-e2e-test-images/nonroot:1.0`
- `gcr.io/kubernetes-e2e-test-images/resource-consumer-controller:1.0`
- `gcr.io/kubernetes-e2e-test-images/resource-consumer:1.5`
- `gcr.io/kubernetes-e2e-test-images/sample-apiserver:1.10`
- `gcr.io/kubernetes-e2e-test-images/test-webserver:1.0`
- `gcr.io/kubernetes-e2e-test-images/volume/gluster:1.0`
- `gcr.io/kubernetes-e2e-test-images/volume/iscsi:2.0`
- `gcr.io/kubernetes-e2e-test-images/volume/nfs:1.0`
- `gcr.io/kubernetes-e2e-test-images/volume/rbd:1.0.1`
- `k8s.gcr.io/etcd:3.3.15`
- `k8s.gcr.io/pause:3.1`
Finally, we ran the tests and got the test result, include `e2e.log`, which showed that all test cases passed. Additionally, we submitted our test result to [k8s-conformance](https://github.com/cncf/k8s-conformance) as a [pull request](https://github.com/cncf/k8s-conformance/pull/779).
![Pull request for conformance test results](/images/blog/2020-01-15-Kubernetes-on-MIPS/pull-request-for-conformance-test-results.png)
_Figure 5 Pull request for conformance test results_
## What's next
We built the kubernetes-MIPS component manually and finished the conformance test, which verified the feasibility of Kubernetes On the MIPS platform and greatly enhanced our confidence in promoting the support of the MIPS architecture by Kubernetes.
In the future, we plan to actively contribute our experience and achievements to the community, submit PR, and patch for MIPS. We hope that more developers and companies in the community join us and promote Kubernetes on MIPS.
Contribution plan
- contribute the source of e2e test images for MIPS
- contribute the source of hyperkube for MIPS
- contribute the source of deploy tools like kubeadm for MIPS
---
@@ -1,99 +0,0 @@
---
layout: blog
title: "Reviewing 2019 in Docs"
date: 2020-01-21
slug: reviewing-2019-in-docs
---
**Author:** Zach Corleissen (Cloud Native Computing Foundation)
Hi, folks! I'm one of the co-chairs for the Kubernetes documentation special interest group (SIG Docs). This blog post is a review of SIG Docs in 2019. Our contributors did amazing work last year, and I want to highlight their successes.
Although I review 2019 in this post, my goal is to point forward to 2020. I observe some trends in SIG Docssome good, others troubling. I want to raise visibility before those challenges increase in severity.
## The good
There was much to celebrate in SIG Docs in 2019.
Kubernetes docs started the year with three localizations in progress. By the end of the year, we ended with ten localizations available, four of which (Chinese, French, Japanese, Korean) are reasonably complete. The Korean and French teams deserve special mentions for their contributions to git best practices across all localizations (Korean team) and help bootstrapping other localizations (French team).
Despite significant transition over the year, SIG Docs [improved its review velocity](https://k8s.devstats.cncf.io/d/44/pr-time-to-approve-and-merge?orgId=1&var-period=w&var-repogroup_name=SIG%20Docs&var-apichange=All&var-size_name=All&var-kind_name=All), with a median review time from PR open to merge of just over 24 hours.
Issue triage improved significantly in both volume and speed, largely due to the efforts of GitHub users @sftim, @tengqm, and @kbhawkey.
Doc sprints remain valuable at KubeCon contributor days, introducing new contributors to Kubernetes documentation.
The docs component of Kubernetes quarterly releases improved over 2019, thanks to iterative playbook improvements from release leads and their teams.
Site traffic increased over the year. The website ended the year with ~6 million page views per month in December, up from ~5M page views in January. The kubernetes.io website had 851k site visitors in October, a new all-time high. Reader satisfaction [remains general](https://kubernetes.io/blog/2019/10/29/kubernetes-documentation-end-user-survey/).
We onboarded a new SIG chair: @jimangel, a Cloud Architect at General Motors. Jim was a docs contributor for a year, during which he led the 1.14 docs release, before stepping up as chair.
## The not so good
While reader satisfaction is decent, **most respondents indicated dissatisfaction with stale content** in every area: concepts, tasks, tutorials, and reference. Additionally, readers requested more diagrams, advanced conceptual content, and code samples&mdash;things that technical writers excel at providing.
SIG Docs continues to solve how best to handle [third-party content](https://github.com/kubernetes/enhancements/pull/1327). **There's too much vendor content on kubernetes.io**, and guidelines for adding or rejecting third-party content remain unclear. The discussion so far has been powerful, including pushback demanding greater collaborative input&mdash;a powerful reminder that Kubernetes is in all ways a communal effort.
We're in the middle of our third chair transition in 18 months. Each chair transition has been healthy and collegial, but it's still a lot of turnover in a short time. Chairing any open source project is difficult, but especially so with SIG Docs. Chairship of SIG Docs requires a steep learning curve across multiple domains: docs (both written and generated from spec), information architecture, specialized contribution paths (for example, localization), how to run a release cycle, website development, CI/CD, community management, on and on. It's a role that requires multiple people to function successfully without burning people out. Training replacements is time-intensive.
Perhaps most pressing in the Not So Good category is that SIG Docs currently has only one technical writer dedicated full-time to Kubernetes docs. This has impacts on Kubernetes docs: some obvious, some less so.
## Impacts of understaffing on Kubernetes docs
<blockquote class="twitter-tweet"><p lang="en" dir="ltr">Me today: <a href="https://t.co/cDpHOWEsjf">pic.twitter.com/cDpHOWEsjf</a></p>&mdash; Benjamin Elder (@BenTheElder) <a href="https://twitter.com/BenTheElder/status/1215453579651104768?ref_src=twsrc%5Etfw">January 10, 2020</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
If Kubernetes continues through 2020 without more technical writers dedicated to the docs, here's what I see as the most likely possibilities.
### But first, a disclaimer
{{< caution >}}
It is very hard to predict, especially the future.
-Niels Bohr
{{< /caution >}}
Some of my predictions are almost certainly wrong. Any errors are mine alone.
That said...
### Effects in 2020
Current levels of function aren't self-sustaining. Even with a strong playbook, the release cycle still requires expert support from at least one (and usually two) chairs during every cycle. Without fail, each release breaks in new and unexpected ways, and it requires familiarity and expertise to diagnose and resolve. As chairs continue to cycle&mdash;and to be clear, regular transitions are part of a healthy project&mdash;we accrue the risks associated with a pool lacking sufficient professional depth and employer support.
Oddly enough, one of the challenges to staffing is that the docs appear good enough. Based on site analytics and survey responses, readers are pleased with the quality of the docs. When folks visit the site, they generally find what they need and behave like satisfied visitors.
The danger is that this will change over time: slowly with occasional losses of function, annoying at first, then increasingly critical. The more time passes without adequate staffing, the more difficult and costly fixes will become.
I suspect this is true because the challenges we face now at decent levels of reader satisfaction are already difficult to fix. API reference generation is complex and brittle; the site's UI is outdated; and our most consistent requests are for more tutorials, advanced concepts, diagrams, and code samples, all of which require ongoing, dedicated time to create.
**Release support remains strong.**
The release team continues a solid habit of leaving each successive team with better support than the previous release. This mostly takes the form of iterative improvements to the [docs release playbook](https://github.com/kubernetes/community/tree/master/sig-release#docs-lead), producing better documentation and reducing siloed knowledge.
**Staleness accelerates.**
Conceptual content becomes less accurate or relevant as features change or deprecate. Tutorial content degrades for the same reason.
The content structure will also degrade: the categories of concepts, tasks, and tutorials are legacy categories that may not best fit the needs of current readers, let alone future ones.
Cruft accumulates for both readers and contributors. Reference docs become increasingly brittle without intervention.
**Critical knowledge vanishes.**
As I mentioned previously, SIG Docs has a wide range of functions, some with a steep learning curve. As contributors change roles or jobs, their expertise and availability will diminish or reduce to zero. Contributors with specific knowledge may not be available for consultation, exposing critical vulnerabilities in docs function. Specific examples include reference generation and chair leadership.
### That's a lot to take in
It's difficult to strike a balance between the importance of SIG Docs' work to the community and our users, the joy it brings me personally, and the fact that things can't remain as they are without significant negative impacts (eventually). SIG Docs is by no means dying; it's a vibrant community with active contributors doing cool things. It's also a community with some critical knowledge and capacity shortages that can only be remedied with trained, paid staff dedicated to documentation.
## What the community can do for healthy docs
Hire technical writers dedicated to Kubernetes docs. Support advanced content creation, not just release docs and incremental feature updates.
Thanks, and Happy 2020.
@@ -1,251 +0,0 @@
---
title: CSI Ephemeral Inline Volumes
date: 2020-01-21
---
**Author:** Patrick Ohly (Intel)
Typically, volumes provided by an external storage driver in
Kubernetes are *persistent*, with a lifecycle that is completely
independent of pods or (as a special case) loosely coupled to the
first pod which uses a volume ([late binding
mode](https://kubernetes.io/docs/concepts/storage/storage-classes/#volume-binding-mode)).
The mechanism for requesting and defining such volumes in Kubernetes
are [Persistent Volume Claim (PVC) and Persistent Volume
(PV)](https://kubernetes.io/docs/concepts/storage/persistent-volumes/)
objects. Originally, volumes that are backed by a Container Storage Interface
(CSI) driver could only be used via this PVC/PV mechanism.
But there are also use cases for data volumes whose content and
lifecycle is tied to a pod. For example, a driver might populate a
volume with dynamically created secrets that are specific to the
application running in the pod. Such volumes need to be created
together with a pod and can be deleted as part of pod termination
(*ephemeral*). They get defined as part of the pod spec (*inline*).
Since Kubernetes 1.15, CSI drivers can also be used for such
*ephemeral inline* volumes. The [CSIInlineVolume feature
gate](https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/)
had to be set to enable it in 1.15 because support was still in alpha
state. In 1.16, the feature reached beta state, which typically means
that it is enabled in clusters by default.
CSI drivers have to be adapted to support this because although two
existing CSI gRPC calls are used (`NodePublishVolume` and `NodeUnpublishVolume`),
the way how they are
used is different and not covered by the CSI spec: for ephemeral
volumes, only `NodePublishVolume` is invoked by `kubelet` when asking
the CSI driver for a volume. All other calls
(like `CreateVolume`, `NodeStageVolume`, etc.) are skipped. The volume
parameters are provided in the pod spec and from there copied into the
`NodePublishVolumeRequest.volume_context` field. There are currently
no standardized parameters; even common ones like size must be
provided in a format that is defined by the CSI driver. Likewise, only
`NodeUnpublishVolume` gets called after the pod has terminated and the
volume needs to be removed.
Initially, the assumption was that CSI drivers would be specifically
written to provide either persistent or ephemeral volumes. But there
are also drivers which provide storage that is useful in both modes:
for example, [PMEM-CSI](https://github.com/intel/pmem-csi) manages
persistent memory (PMEM), a new kind of local storage that is provided
by [Intel® Optane™ DC Persistent
Memory](https://www.intel.com/content/www/us/en/architecture-and-technology/optane-dc-persistent-memory.html). Such
memory is useful both as persistent data storage (faster than normal SSDs)
and as ephemeral scratch space (higher capacity than DRAM).
Therefore the support in Kubernetes 1.16 was extended:
* Kubernetes and users can determine which kind of volumes a driver
supports via the `volumeLifecycleModes` field in the [`CSIDriver`
object](https://kubernetes-csi.github.io/docs/csi-driver-object.html#what-fields-does-the-csidriver-object-have).
* Drivers can get information about the volume mode by enabling the
["pod info on
mount"](https://kubernetes-csi.github.io/docs/pod-info.html) feature
which then will add the new `csi.storage.k8s.io/ephemeral` entry to
the `NodePublishRequest.volume_context`.
For more information about implementing support of ephemeral inline
volumes in a CSI driver, see the [Kubernetes-CSI
documentation](https://kubernetes-csi.github.io/docs/ephemeral-local-volumes.html)
and the [original design
document](https://github.com/kubernetes/enhancements/blob/master/keps/sig-storage/20190122-csi-inline-volumes.md).
What follows in this blog post are usage examples based on real drivers
and a summary at the end.
# Examples
## [PMEM-CSI](https://github.com/intel/pmem-csi)
Support for ephemeral inline volumes was added in [release
v0.6.0](https://github.com/intel/pmem-csi/releases/tag/v0.6.0). The
driver can be used on hosts with real Intel® Optane™ DC Persistent
Memory, on [special machines in
GCE](https://github.com/intel/pmem-csi/blob/v0.6.0/examples/gce.md) or
with hardware emulated by QEMU. The latter is fully [integrated into
the
makefile](https://github.com/intel/pmem-csi/tree/v0.6.0#qemu-and-kubernetes)
and only needs Go, Docker and KVM, so that approach was used for this
example:
```sh
git clone --branch release-0.6 https://github.com/intel/pmem-csi
cd pmem-csi
TEST_DISTRO=clear TEST_DISTRO_VERSION=32080 TEST_PMEM_REGISTRY=intel make start
```
Bringing up the four-node cluster can take a while but eventually should end with:
```
The test cluster is ready. Log in with /work/pmem-csi/_work/pmem-govm/ssh-pmem-govm, run kubectl once logged in.
Alternatively, KUBECONFIG=/work/pmem-csi/_work/pmem-govm/kube.config can also be used directly.
To try out the pmem-csi driver persistent volumes:
...
To try out the pmem-csi driver ephemeral volumes:
cat deploy/kubernetes-1.17/pmem-app-ephemeral.yaml | /work/pmem-csi/_work/pmem-govm/ssh-pmem-govm kubectl create -f -
```
`deploy/kubernetes-1.17/pmem-app-ephemeral.yaml` specifies one volume:
```
kind: Pod
apiVersion: v1
metadata:
name: my-csi-app-inline-volume
spec:
containers:
- name: my-frontend
image: busybox
command: [ "sleep", "100000" ]
volumeMounts:
- mountPath: "/data"
name: my-csi-volume
volumes:
- name: my-csi-volume
csi:
driver: pmem-csi.intel.com
fsType: "xfs"
volumeAttributes:
size: "2Gi"
nsmode: "fsdax"
```
Once we have created that pod, we can inspect the result:
```sh
kubectl describe pods/my-csi-app-inline-volume
```
```
Name: my-csi-app-inline-volume
...
Volumes:
my-csi-volume:
Type: CSI (a Container Storage Interface (CSI) volume source)
Driver: pmem-csi.intel.com
FSType: xfs
ReadOnly: false
VolumeAttributes: nsmode=fsdax
size=2Gi
```
```sh
kubectl exec my-csi-app-inline-volume -- df -h /data
```
```
Filesystem Size Used Available Use% Mounted on
/dev/ndbus0region0fsdax/d7eb073f2ab1937b88531fce28e19aa385e93696
1.9G 34.2M 1.8G 2% /data
```
## [Image Populator](https://github.com/kubernetes-csi/csi-driver-image-populator)
The image populator automatically unpacks a container image and makes
its content available as an ephemeral volume. It's still in
development, but canary images are already available which can be
installed with:
```sh
kubectl create -f https://github.com/kubernetes-csi/csi-driver-image-populator/raw/master/deploy/kubernetes-1.16/csi-image-csidriverinfo.yaml
kubectl create -f https://github.com/kubernetes-csi/csi-driver-image-populator/raw/master/deploy/kubernetes-1.16/csi-image-daemonset.yaml
```
This example pod will run nginx and have it serve data that
comes from the `kfox1111/misc:test` image:
```sh
kubectl create -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
name: nginx
spec:
containers:
- name: nginx
image: nginx:1.16-alpine
ports:
- containerPort: 80
volumeMounts:
- name: data
mountPath: /usr/share/nginx/html
volumes:
- name: data
csi:
driver: image.csi.k8s.io
volumeAttributes:
image: kfox1111/misc:test
EOF
```
```sh
kubectl exec nginx -- cat /usr/share/nginx/html/test
```
That `test` file just contains a single word:
```
testing
```
Such data containers can be built with Dockerfiles such as:
```
FROM scratch
COPY index.html /index.html
```
## [cert-manager-csi](https://github.com/jetstack/cert-manager-csi)
cert-manager-csi works together with
[cert-manager](https://github.com/jetstack/cert-manager). The goal for
this driver is to facilitate requesting and mounting certificate key
pairs to pods seamlessly. This is useful for facilitating mTLS, or
otherwise securing connections of pods with guaranteed present
certificates whilst having all of the features that cert-manager
provides. This project is experimental.
# Next steps
One of the issues with ephemeral inline volumes is that pods get
scheduled by Kubernetes onto nodes without knowing anything about the
currently available storage on that node. Once the pod has been
scheduled, the CSI driver must make the volume available one that
node. If that is currently not possible, the pod cannot start. This
will be retried until eventually the volume becomes ready. The
[storage capacity tracking
KEP](https://github.com/kubernetes/enhancements/pull/1353) is an
attempt to address this problem.
A related KEP introduces a [standardized size
parameter](https://github.com/kubernetes/enhancements/pull/1409).
Currently, CSI ephemeral inline volumes stay in beta while issues like
these are getting discussed. Your feedback is needed to decide how to
proceed with this feature. For the KEPs, the two PRs linked to above
is a good place to comment. The SIG Storage also [meets
regularly](https://github.com/kubernetes/community/tree/master/sig-storage#meetings)
and can be reached via [Slack and a mailing
list](https://github.com/kubernetes/community/tree/master/sig-storage#contact).
@@ -1,110 +0,0 @@
---
layout: blog
title: "KubeInvaders - Gamified Chaos Engineering Tool for Kubernetes"
date: 2020-01-22
slug: kubeinvaders-gamified-chaos-engineering-tool-for-kubernetes
---
**Authors** Eugenio Marzo, Sourcesense
Some months ago, I released my latest project called KubeInvaders. The
first time I shared it with the community was during an Openshift
Commons Briefing session. Kubenvaders is a Gamified Chaos Engineering
tool for Kubernetes and Openshift and helps test how resilient your
Kubernetes cluster is, in a fun way.
It is like Space Invaders, but the aliens are pods.
![](https://github.com/lucky-sideburn/KubeInvaders-kubernetes-post/raw/master/img1.png)
During my presentation at Codemotion Milan 2019, I started saying "of
course you can do it with few lines of Bash, but it is boring."
![](https://github.com/lucky-sideburn/KubeInvaders-kubernetes-post/raw/master/img2.png)
Using the code above you can kill random pods across a Kubernetes cluster, but I
think it is much more fun with the spaceship of KubeInvaders.
I published the code at
[https://github.com/lucky-sideburn/KubeInvaders](https://github.com/lucky-sideburn/KubeInvaders)
and there is a little community that is growing gradually. Some people
love to use it for demo sessions killing pods on a big screen.
![](https://github.com/lucky-sideburn/KubeInvaders-kubernetes-post/raw/master/img3.png)
## How to install KubeInvaders
I defined multiples modes to install it:
1. Helm Chart
[https://github.com/lucky-sideburn/KubeInvaders/tree/master/helm-charts/kubeinvaders](https://github.com/lucky-sideburn/KubeInvaders/tree/master/helm-charts/kubeinvaders)
2. Manual Installation for Openshift using a template
[https://github.com/lucky-sideburn/KubeInvaders\#install-kubeinvaders-on-openshift](https://github.com/lucky-sideburn/KubeInvaders#install-kubeinvaders-on-openshift)
3. Manual Installation for Kubernetes
[https://github.com/lucky-sideburn/KubeInvaders\#install-kubeinvaders-on-kubernetes](https://github.com/lucky-sideburn/KubeInvaders#install-kubeinvaders-on-kubernetes)
The preferred way, of course, is with a Helm chart:
```
# Please set target_namespace to set your target namespace!
helm install --set-string target_namespace="namespace1,namespace2" \
--name kubeinvaders --namespace kubeinvaders ./helm-charts/kubeinvaders
```
## How to use KubeInvaders
Once it is installed on your cluster you can use the following
functionalities:
* Key 'a' — Switch to automatic pilot
* Key 'm' — Switch to manual pilot
* Key 'i' — Show pod's name. Move the ship towards an alien
* Key 'h' — Print help
* Key 'n' — Jump between different namespaces (my favorite feature!)
## Tuning KubeInvaders
At Codemotion Milan 2019, my colleagues and I organized a desk with a
game station for playing KubeInvaders. People had to fight with Kubernetes to
win a t-shirt.
If you have pods that require a few seconds to start, you may lose. It
is possible to set the complexity of the game with these parameters as
environmment variables in the Kubernetes deployment:
* ALIENPROXIMITY — Reduce this value to increase the distance between aliens;
* HITSLIMIT — Seconds of CPU time to wait before shooting;
* UPDATETIME — Seconds to wait before updating pod status (you can set also 0.x Es: 0.5);
The result is a harder game experience against the machine.
## Use cases
Adopting chaos engineering strategies for your production environment is
really useful, because it is the only way to test if a system supports
unexpected destructive events.
KubeInvaders is a game — so please do not take it too seriously! — but it demonstrates
some important use cases:
* Test how resilient Kubernetes clusters are on unexpected pod deletion
* Collect metrics like pod restart time
* Tune readiness probes
## Next steps
I want to continue to add some cool features and integrate it into a
Kubernetes dashboard because I am planning to transform it into a
"Gamified Chaos Engineering and Development Tool for Kubernetes", to help
developer to interact with deployments in a Kubernetes environment. For
example:
* Point to the aliens to get pod logs
* Deploy Helm charts by shooting some particular objects
* Read messages stored in a specific label present in a deployment
Please feel free to contribute to
[https://github.com/lucky-sideburn/KubeInvaders](https://github.com/lucky-sideburn/KubeInvaders)
and stay updated following \#kubeinvaders news [on Twitter](https://twitter.com/luckysideburn).
@@ -1,760 +0,0 @@
---
layout: blog
title: "Deploying External OpenStack Cloud Provider with Kubeadm"
date: 2020-02-07
slug: Deploying-External-OpenStack-Cloud-Provider-with-Kubeadm
---
This document describes how to install a single control-plane Kubernetes cluster v1.15 with kubeadm on CentOS, and then deploy an external OpenStack cloud provider and Cinder CSI plugin to use Cinder volumes as persistent volumes in Kubernetes.
### Preparation in OpenStack
This cluster runs on OpenStack VMs, so let's create a few things in OpenStack first.
* A project/tenant for this Kubernetes cluster
* A user in this project for Kubernetes, to query node information and attach volumes etc
* A private network and subnet
* A router for this private network and connect it to a public network for floating IPs
* A security group for all Kubernetes VMs
* A VM as a control-plane node and a few VMs as worker nodes
The security group will have the following rules to open ports for Kubernetes.
**Control-Plane Node**
|Protocol | Port Number | Description|
|----------|-------------|------------|
|TCP |6443|Kubernetes API Server|
|TCP|2379-2380|etcd server client API|
|TCP|10250|Kubelet API|
|TCP|10251|kube-scheduler|
|TCP|10252|kube-controller-manager|
|TCP|10255|Read-only Kubelet API|
**Worker Nodes**
|Protocol | Port Number | Description|
|----------|-------------|------------|
|TCP|10250|Kubelet API|
|TCP|10255|Read-only Kubelet API|
|TCP|30000-32767|NodePort Services|
**CNI ports on both control-plane and worker nodes**
|Protocol | Port Number | Description|
|----------|-------------|------------|
|TCP|179|Calico BGP network|
|TCP|9099|Calico felix (health check)|
|UDP|8285|Flannel|
|UDP|8472|Flannel|
|TCP|6781-6784|Weave Net|
|UDP|6783-6784|Weave Net|
CNI specific ports are only required to be opened when that particular CNI plugin is used. In this guide, we will use Weave Net. Only the Weave Net ports (TCP 6781-6784 and UDP 6783-6784), will need to be opened in the security group.
The control-plane node needs at least 2 cores and 4GB RAM. After the VM is launched, verify its hostname and make sure it is the same as the node name in Nova.
If the hostname is not resolvable, add it to `/etc/hosts`.
For example, if the VM is called master1, and it has an internal IP 192.168.1.4. Add that to `/etc/hosts` and set hostname to master1.
```shell
echo "192.168.1.4 master1" >> /etc/hosts
hostnamectl set-hostname master1
```
### Install Docker and Kubernetes
Next, we'll follow the official documents to install docker and Kubernetes using kubeadm.
Install Docker following the steps from the [container runtime](/docs/setup/production-environment/container-runtimes/) documentation.
Note that it is a [best practice to use systemd as the cgroup driver](/docs/setup/production-environment/container-runtimes/#cgroup-drivers) for Kubernetes.
If you use an internal container registry, add them to the docker config.
```shell
# Install Docker CE
## Set up the repository
### Install required packages.
yum install yum-utils device-mapper-persistent-data lvm2
### Add Docker repository.
yum-config-manager \
--add-repo \
https://download.docker.com/linux/centos/docker-ce.repo
## Install Docker CE.
yum update && yum install docker-ce-18.06.2.ce
## Create /etc/docker directory.
mkdir /etc/docker
# Configure the Docker daemon
cat > /etc/docker/daemon.json <<EOF
{
"exec-opts": ["native.cgroupdriver=systemd"],
"log-driver": "json-file",
"log-opts": {
"max-size": "100m"
},
"storage-driver": "overlay2",
"storage-opts": [
"overlay2.override_kernel_check=true"
]
}
EOF
mkdir -p /etc/systemd/system/docker.service.d
# Restart Docker
systemctl daemon-reload
systemctl restart docker
systemctl enable docker
```
Install kubeadm following the steps from the [Installing Kubeadm](/docs/setup/production-environment/tools/kubeadm/install-kubeadm/) documentation.
```shell
cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
EOF
# Set SELinux in permissive mode (effectively disabling it)
# Caveat: In a production environment you may not want to disable SELinux, please refer to Kubernetes documents about SELinux
setenforce 0
sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config
yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes
systemctl enable --now kubelet
cat <<EOF > /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sysctl --system
# check if br_netfilter module is loaded
lsmod | grep br_netfilter
# if not, load it explicitly with
modprobe br_netfilter
```
The official document about how to create a single control-plane cluster can be found from the [Creating a single control-plane cluster with kubeadm](/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/) documentation.
We'll largely follow that document but also add additional things for the cloud provider.
To make things more clear, we'll use a `kubeadm-config.yml` for the control-plane node.
In this config we specify to use an external OpenStack cloud provider, and where to find its config.
We also enable storage API in API server's runtime config so we can use OpenStack volumes as persistent volumes in Kubernetes.
```yaml
apiVersion: kubeadm.k8s.io/v1beta1
kind: InitConfiguration
nodeRegistration:
kubeletExtraArgs:
cloud-provider: "external"
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
kubernetesVersion: "v1.15.1"
apiServer:
extraArgs:
enable-admission-plugins: NodeRestriction
runtime-config: "storage.k8s.io/v1=true"
controllerManager:
extraArgs:
external-cloud-volume-plugin: openstack
extraVolumes:
- name: "cloud-config"
hostPath: "/etc/kubernetes/cloud-config"
mountPath: "/etc/kubernetes/cloud-config"
readOnly: true
pathType: File
networking:
serviceSubnet: "10.96.0.0/12"
podSubnet: "10.224.0.0/16"
dnsDomain: "cluster.local"
```
Now we'll create the cloud config, `/etc/kubernetes/cloud-config`, for OpenStack.
Note that the tenant here is the one we created for all Kubernetes VMs in the beginning.
All VMs should be launched in this project/tenant.
In addition you need to create a user in this tenant for Kubernetes to do queries.
The ca-file is the CA root certificate for OpenStack's API endpoint, for example `https://openstack.cloud:5000/v3`
At the time of writing the cloud provider doesn't allow insecure connections (skip CA check).
```ini
[Global]
region=RegionOne
username=username
password=password
auth-url=https://openstack.cloud:5000/v3
tenant-id=14ba698c0aec4fd6b7dc8c310f664009
domain-id=default
ca-file=/etc/kubernetes/ca.pem
[LoadBalancer]
subnet-id=b4a9a292-ea48-4125-9fb2-8be2628cb7a1
floating-network-id=bc8a590a-5d65-4525-98f3-f7ef29c727d5
[BlockStorage]
bs-version=v2
[Networking]
public-network-name=public
ipv6-support-disabled=false
```
Next run kubeadm to initiate the control-plane node
```shell
kubeadm init --config=kubeadm-config.yml
```
With the initialization completed, copy admin config to .kube
```shell
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
```
At this stage, the control-plane node is created but not ready. All the nodes have the taint `node.cloudprovider.kubernetes.io/uninitialized=true:NoSchedule` and are waiting to be initialized by the cloud-controller-manager.
```console
# kubectl describe no master1
Name: master1
Roles: master
......
Taints: node-role.kubernetes.io/master:NoSchedule
node.cloudprovider.kubernetes.io/uninitialized=true:NoSchedule
node.kubernetes.io/not-ready:NoSchedule
......
```
Now deploy the OpenStack cloud controller manager into the cluster, following [using controller manager with kubeadm](https://github.com/kubernetes/cloud-provider-openstack/blob/master/docs/using-controller-manager-with-kubeadm.md).
Create a secret with the cloud-config for the openstack cloud provider.
```shell
kubectl create secret -n kube-system generic cloud-config --from-literal=cloud.conf="$(cat /etc/kubernetes/cloud-config)" --dry-run -o yaml > cloud-config-secret.yaml
kubectl apply -f cloud-config-secret.yaml
```
Get the CA certificate for OpenStack API endpoints and put that into `/etc/kubernetes/ca.pem`.
Create RBAC resources.
```shell
kubectl apply -f https://github.com/kubernetes/cloud-provider-openstack/raw/release-1.15/cluster/addons/rbac/cloud-controller-manager-roles.yaml
kubectl apply -f https://github.com/kubernetes/cloud-provider-openstack/raw/release-1.15/cluster/addons/rbac/cloud-controller-manager-role-bindings.yaml
```
We'll run the OpenStack cloud controller manager as a DaemonSet rather than a pod.
The manager will only run on the control-plane node, so if there are multiple control-plane nodes, multiple pods will be run for high availability.
Create `openstack-cloud-controller-manager-ds.yaml` containing the following manifests, then apply it.
```yaml
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: cloud-controller-manager
namespace: kube-system
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: openstack-cloud-controller-manager
namespace: kube-system
labels:
k8s-app: openstack-cloud-controller-manager
spec:
selector:
matchLabels:
k8s-app: openstack-cloud-controller-manager
updateStrategy:
type: RollingUpdate
template:
metadata:
labels:
k8s-app: openstack-cloud-controller-manager
spec:
nodeSelector:
node-role.kubernetes.io/master: ""
securityContext:
runAsUser: 1001
tolerations:
- key: node.cloudprovider.kubernetes.io/uninitialized
value: "true"
effect: NoSchedule
- key: node-role.kubernetes.io/master
effect: NoSchedule
- effect: NoSchedule
key: node.kubernetes.io/not-ready
serviceAccountName: cloud-controller-manager
containers:
- name: openstack-cloud-controller-manager
image: docker.io/k8scloudprovider/openstack-cloud-controller-manager:v1.15.0
args:
- /bin/openstack-cloud-controller-manager
- --v=1
- --cloud-config=$(CLOUD_CONFIG)
- --cloud-provider=openstack
- --use-service-account-credentials=true
- --address=127.0.0.1
volumeMounts:
- mountPath: /etc/kubernetes/pki
name: k8s-certs
readOnly: true
- mountPath: /etc/ssl/certs
name: ca-certs
readOnly: true
- mountPath: /etc/config
name: cloud-config-volume
readOnly: true
- mountPath: /usr/libexec/kubernetes/kubelet-plugins/volume/exec
name: flexvolume-dir
- mountPath: /etc/kubernetes
name: ca-cert
readOnly: true
resources:
requests:
cpu: 200m
env:
- name: CLOUD_CONFIG
value: /etc/config/cloud.conf
hostNetwork: true
volumes:
- hostPath:
path: /usr/libexec/kubernetes/kubelet-plugins/volume/exec
type: DirectoryOrCreate
name: flexvolume-dir
- hostPath:
path: /etc/kubernetes/pki
type: DirectoryOrCreate
name: k8s-certs
- hostPath:
path: /etc/ssl/certs
type: DirectoryOrCreate
name: ca-certs
- name: cloud-config-volume
secret:
secretName: cloud-config
- name: ca-cert
secret:
secretName: openstack-ca-cert
```
When the controller manager is running, it will query OpenStack to get information about the nodes and remove the taint. In the node info you'll see the VM's UUID in OpenStack.
```console
# kubectl describe no master1
Name: master1
Roles: master
......
Taints: node-role.kubernetes.io/master:NoSchedule
node.kubernetes.io/not-ready:NoSchedule
......
sage:docker: network plugin is not ready: cni config uninitialized
......
PodCIDR: 10.224.0.0/24
ProviderID: openstack:///548e3c46-2477-4ce2-968b-3de1314560a5
```
Now install your favourite CNI and the control-plane node will become ready.
For example, to install Weave Net, run this command:
```shell
kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"
```
Next we'll set up worker nodes.
Firstly, install docker and kubeadm in the same way as how they were installed in the control-plane node.
To join them to the cluster we need a token and ca cert hash from the output of control-plane node installation.
If it is expired or lost we can recreate it using these commands.
```shell
# check if token is expired
kubeadm token list
# re-create token and show join command
kubeadm token create --print-join-command
```
Create `kubeadm-config.yml` for worker nodes with the above token and ca cert hash.
```yaml
apiVersion: kubeadm.k8s.io/v1beta2
discovery:
bootstrapToken:
apiServerEndpoint: 192.168.1.7:6443
token: 0c0z4p.dnafh6vnmouus569
caCertHashes: ["sha256:fcb3e956a6880c05fc9d09714424b827f57a6fdc8afc44497180905946527adf"]
kind: JoinConfiguration
nodeRegistration:
kubeletExtraArgs:
cloud-provider: "external"
```
apiServerEndpoint is the control-plane node, token and caCertHashes can be taken from the join command printed in the output of 'kubeadm token create' command.
Run kubeadm and the worker nodes will be joined to the cluster.
```shell
kubeadm join --config kubeadm-config.yml
```
At this stage we'll have a working Kubernetes cluster with an external OpenStack cloud provider.
The provider tells Kubernetes about the mapping between Kubernetes nodes and OpenStack VMs.
If Kubernetes wants to attach a persistent volume to a pod, it can find out which OpenStack VM the pod is running on from the mapping, and attach the underlying OpenStack volume to the VM accordingly.
### Deploy Cinder CSI
The integration with Cinder is provided by an external Cinder CSI plugin, as described in the [Cinder CSI](https://github.com/kubernetes/cloud-provider-openstack/blob/master/docs/using-cinder-csi-plugin.md) documentation.
We'll perform the following steps to install the Cinder CSI plugin.
Firstly, create a secret with CA certs for OpenStack's API endpoints. It is the same cert file as what we use in cloud provider above.
```shell
kubectl create secret -n kube-system generic openstack-ca-cert --from-literal=ca.pem="$(cat /etc/kubernetes/ca.pem)" --dry-run -o yaml > openstack-ca-cert.yaml
kubectl apply -f openstack-ca-cert.yaml
```
Then create RBAC resources.
```shell
kubectl apply -f https://raw.githubusercontent.com/kubernetes/cloud-provider-openstack/release-1.15/manifests/cinder-csi-plugin/cinder-csi-controllerplugin-rbac.yaml
kubectl apply -f https://github.com/kubernetes/cloud-provider-openstack/raw/release-1.15/manifests/cinder-csi-plugin/cinder-csi-nodeplugin-rbac.yaml
```
The Cinder CSI plugin includes a controller plugin and a node plugin.
The controller communicates with Kubernetes APIs and Cinder APIs to create/attach/detach/delete Cinder volumes. The node plugin in-turn runs on each worker node to bind a storage device (attached volume) to a pod, and unbind it during deletion.
Create `cinder-csi-controllerplugin.yaml` and apply it to create csi controller.
```yaml
kind: Service
apiVersion: v1
metadata:
name: csi-cinder-controller-service
namespace: kube-system
labels:
app: csi-cinder-controllerplugin
spec:
selector:
app: csi-cinder-controllerplugin
ports:
- name: dummy
port: 12345
---
kind: StatefulSet
apiVersion: apps/v1
metadata:
name: csi-cinder-controllerplugin
namespace: kube-system
spec:
serviceName: "csi-cinder-controller-service"
replicas: 1
selector:
matchLabels:
app: csi-cinder-controllerplugin
template:
metadata:
labels:
app: csi-cinder-controllerplugin
spec:
serviceAccount: csi-cinder-controller-sa
containers:
- name: csi-attacher
image: quay.io/k8scsi/csi-attacher:v1.0.1
args:
- "--v=5"
- "--csi-address=$(ADDRESS)"
env:
- name: ADDRESS
value: /var/lib/csi/sockets/pluginproxy/csi.sock
imagePullPolicy: "IfNotPresent"
volumeMounts:
- name: socket-dir
mountPath: /var/lib/csi/sockets/pluginproxy/
- name: csi-provisioner
image: quay.io/k8scsi/csi-provisioner:v1.0.1
args:
- "--provisioner=csi-cinderplugin"
- "--csi-address=$(ADDRESS)"
env:
- name: ADDRESS
value: /var/lib/csi/sockets/pluginproxy/csi.sock
imagePullPolicy: "IfNotPresent"
volumeMounts:
- name: socket-dir
mountPath: /var/lib/csi/sockets/pluginproxy/
- name: csi-snapshotter
image: quay.io/k8scsi/csi-snapshotter:v1.0.1
args:
- "--connection-timeout=15s"
- "--csi-address=$(ADDRESS)"
env:
- name: ADDRESS
value: /var/lib/csi/sockets/pluginproxy/csi.sock
imagePullPolicy: Always
volumeMounts:
- mountPath: /var/lib/csi/sockets/pluginproxy/
name: socket-dir
- name: cinder-csi-plugin
image: docker.io/k8scloudprovider/cinder-csi-plugin:v1.15.0
args :
- /bin/cinder-csi-plugin
- "--v=5"
- "--nodeid=$(NODE_ID)"
- "--endpoint=$(CSI_ENDPOINT)"
- "--cloud-config=$(CLOUD_CONFIG)"
- "--cluster=$(CLUSTER_NAME)"
env:
- name: NODE_ID
valueFrom:
fieldRef:
fieldPath: spec.nodeName
- name: CSI_ENDPOINT
value: unix://csi/csi.sock
- name: CLOUD_CONFIG
value: /etc/config/cloud.conf
- name: CLUSTER_NAME
value: kubernetes
imagePullPolicy: "IfNotPresent"
volumeMounts:
- name: socket-dir
mountPath: /csi
- name: secret-cinderplugin
mountPath: /etc/config
readOnly: true
- mountPath: /etc/kubernetes
name: ca-cert
readOnly: true
volumes:
- name: socket-dir
hostPath:
path: /var/lib/csi/sockets/pluginproxy/
type: DirectoryOrCreate
- name: secret-cinderplugin
secret:
secretName: cloud-config
- name: ca-cert
secret:
secretName: openstack-ca-cert
```
Create `cinder-csi-nodeplugin.yaml` and apply it to create csi node.
```yaml
kind: DaemonSet
apiVersion: apps/v1
metadata:
name: csi-cinder-nodeplugin
namespace: kube-system
spec:
selector:
matchLabels:
app: csi-cinder-nodeplugin
template:
metadata:
labels:
app: csi-cinder-nodeplugin
spec:
serviceAccount: csi-cinder-node-sa
hostNetwork: true
containers:
- name: node-driver-registrar
image: quay.io/k8scsi/csi-node-driver-registrar:v1.1.0
args:
- "--v=5"
- "--csi-address=$(ADDRESS)"
- "--kubelet-registration-path=$(DRIVER_REG_SOCK_PATH)"
lifecycle:
preStop:
exec:
command: ["/bin/sh", "-c", "rm -rf /registration/cinder.csi.openstack.org /registration/cinder.csi.openstack.org-reg.sock"]
env:
- name: ADDRESS
value: /csi/csi.sock
- name: DRIVER_REG_SOCK_PATH
value: /var/lib/kubelet/plugins/cinder.csi.openstack.org/csi.sock
- name: KUBE_NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
imagePullPolicy: "IfNotPresent"
volumeMounts:
- name: socket-dir
mountPath: /csi
- name: registration-dir
mountPath: /registration
- name: cinder-csi-plugin
securityContext:
privileged: true
capabilities:
add: ["SYS_ADMIN"]
allowPrivilegeEscalation: true
image: docker.io/k8scloudprovider/cinder-csi-plugin:v1.15.0
args :
- /bin/cinder-csi-plugin
- "--nodeid=$(NODE_ID)"
- "--endpoint=$(CSI_ENDPOINT)"
- "--cloud-config=$(CLOUD_CONFIG)"
env:
- name: NODE_ID
valueFrom:
fieldRef:
fieldPath: spec.nodeName
- name: CSI_ENDPOINT
value: unix://csi/csi.sock
- name: CLOUD_CONFIG
value: /etc/config/cloud.conf
imagePullPolicy: "IfNotPresent"
volumeMounts:
- name: socket-dir
mountPath: /csi
- name: pods-mount-dir
mountPath: /var/lib/kubelet/pods
mountPropagation: "Bidirectional"
- name: kubelet-dir
mountPath: /var/lib/kubelet
mountPropagation: "Bidirectional"
- name: pods-cloud-data
mountPath: /var/lib/cloud/data
readOnly: true
- name: pods-probe-dir
mountPath: /dev
mountPropagation: "HostToContainer"
- name: secret-cinderplugin
mountPath: /etc/config
readOnly: true
- mountPath: /etc/kubernetes
name: ca-cert
readOnly: true
volumes:
- name: socket-dir
hostPath:
path: /var/lib/kubelet/plugins/cinder.csi.openstack.org
type: DirectoryOrCreate
- name: registration-dir
hostPath:
path: /var/lib/kubelet/plugins_registry/
type: Directory
- name: kubelet-dir
hostPath:
path: /var/lib/kubelet
type: Directory
- name: pods-mount-dir
hostPath:
path: /var/lib/kubelet/pods
type: Directory
- name: pods-cloud-data
hostPath:
path: /var/lib/cloud/data
type: Directory
- name: pods-probe-dir
hostPath:
path: /dev
type: Directory
- name: secret-cinderplugin
secret:
secretName: cloud-config
- name: ca-cert
secret:
secretName: openstack-ca-cert
```
When they are both running, create a storage class for Cinder.
```yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: csi-sc-cinderplugin
provisioner: csi-cinderplugin
```
Then we can create a PVC with this class.
```yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: myvol
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
storageClassName: csi-sc-cinderplugin
```
When the PVC is created, a Cinder volume is created correspondingly.
```console
# kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
myvol Bound pvc-14b8bc68-6c4c-4dc6-ad79-4cb29a81faad 1Gi RWO csi-sc-cinderplugin 3s
```
In OpenStack the volume name will match the Kubernetes persistent volume generated name. In this example it would be: _pvc-14b8bc68-6c4c-4dc6-ad79-4cb29a81faad_
Now we can create a pod with the PVC.
```yaml
apiVersion: v1
kind: Pod
metadata:
name: web
spec:
containers:
- name: web
image: nginx
ports:
- name: web
containerPort: 80
hostPort: 8081
protocol: TCP
volumeMounts:
- mountPath: "/usr/share/nginx/html"
name: mypd
volumes:
- name: mypd
persistentVolumeClaim:
claimName: myvol
```
When the pod is running, the volume will be attached to the pod.
If we go back to OpenStack, we can see the Cinder volume is mounted to the worker node where the pod is running on.
```console
# openstack volume show 6b5f3296-b0eb-40cd-bd4f-2067a0d6287f
+--------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+--------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| attachments | [{u'server_id': u'1c5e1439-edfa-40ed-91fe-2a0e12bc7eb4', u'attachment_id': u'11a15b30-5c24-41d4-86d9-d92823983a32', u'attached_at': u'2019-07-24T05:02:34.000000', u'host_name': u'compute-6', u'volume_id': u'6b5f3296-b0eb-40cd-bd4f-2067a0d6287f', u'device': u'/dev/vdb', u'id': u'6b5f3296-b0eb-40cd-bd4f-2067a0d6287f'}] |
| availability_zone | nova |
| bootable | false |
| consistencygroup_id | None |
| created_at | 2019-07-24T05:02:18.000000 |
| description | Created by OpenStack Cinder CSI driver |
| encrypted | False |
| id | 6b5f3296-b0eb-40cd-bd4f-2067a0d6287f |
| migration_status | None |
| multiattach | False |
| name | pvc-14b8bc68-6c4c-4dc6-ad79-4cb29a81faad |
| os-vol-host-attr:host | rbd:volumes@rbd#rbd |
| os-vol-mig-status-attr:migstat | None |
| os-vol-mig-status-attr:name_id | None |
| os-vol-tenant-attr:tenant_id | 14ba698c0aec4fd6b7dc8c310f664009 |
| properties | attached_mode='rw', cinder.csi.openstack.org/cluster='kubernetes' |
| replication_status | None |
| size | 1 |
| snapshot_id | None |
| source_volid | None |
| status | in-use |
| type | rbd |
| updated_at | 2019-07-24T05:02:35.000000 |
| user_id | 5f6a7a06f4e3456c890130d56babf591 |
+--------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
```
### Summary
In this walk-through, we deployed a Kubernetes cluster on OpenStack VMs and integrated it with OpenStack using an external OpenStack cloud provider. Then on this Kubernetes cluster we deployed Cinder CSI plugin which can create Cinder volumes and expose them in Kubernetes as persistent volumes.
@@ -1,45 +0,0 @@
---
layout: blog
title: "Contributor Summit Amsterdam Schedule Announced"
date: 2020-02-18
slug: Contributor-Summit-Amsterdam-Schedule-Announced
---
**Authors:** Jeffrey Sica (Red Hat), Amanda Katona (VMware)
tl;dr [Registration is open](https://events.linuxfoundation.org/kubernetes-contributor-summit-europe/) and the [schedule is live](https://kcseu2020.sched.com/) so register now and well see you in Amsterdam!
## Kubernetes Contributor Summit
**Sunday, March 29, 2020**
- Evening Contributor Celebration:
[ZuidPool](https://www.zuid-pool.nl/en/)
- Address: [Europaplein 22, 1078 GZ Amsterdam, Netherlands](https://www.google.com/search?q=KubeCon+Amsterdam+2020&ie=UTF-8&ibp=htl;events&rciv=evn&sa=X&ved=2ahUKEwiZoLvQ0dvnAhVST6wKHScBBZ8Q5bwDMAB6BAgSEAE#)
- Time: 18:00 - 21:00
**Monday, March 30, 2020**
- All Day Contributor Summit:
- [Amsterdam RAI](https://www.rai.nl/en/)
- Address: [Europaplein 24, 1078 GZ Amsterdam, Netherlands](https://www.google.com/search?q=kubecon+amsterdam+2020&oq=kubecon+amste&aqs=chrome.0.35i39j69i57j0l4j69i61l2.3957j1j4&sourceid=chrome&ie=UTF-8&ibp=htl;events&rciv=evn&sa=X&ved=2ahUKEwiZoLvQ0dvnAhVST6wKHScBBZ8Q5bwDMAB6BAgSEAE#)
- Time: 09:00 - 17:00 (Breakfast at 08:00)
![Contributor Summit](/images/blog/2020-02-18-Contributor-Summit-Amsterdam-Schedule-Announced/contribsummit.jpg)
Hello everyone and Happy 2020! Its hard to believe that KubeCon EU 2020 is less than six weeks away, and with that another contributor summit! This year we have the pleasure of being in Amsterdam in early spring, so be sure to pack some warmer clothing. This summit looks to be exciting with a lot of fantastic community-driven content. We received **26** submissions from the CFP. From that, the events team selected **12** sessions. Each of the sessions falls into one of four categories:
* Community
* Contributor Improvement
* Sustainability
* In-depth Technical
On top of the presentations, there will be a dedicated Docs Sprint as well as the New Contributor Workshop 101 and 201 Sessions. All told, we will have five separate rooms of content throughout the day on Monday. Please **[see the full schedule](https://kcseu2020.sched.com/)** to see what sessions youd be interested in. We hope between the content provided and the inevitable hallway track, everyone has a fun and enriching experience.
Speaking of fun, the social Sunday night should be a blast! Were hosting this summits social close to the conference center, at [ZuidPool](https://www.zuid-pool.nl/en/). There will be games, bingo, and unconference sign-up throughout the evening. It should be a relaxed way to kick off the week.
[Registration is open](https://events.linuxfoundation.org/kubernetes-contributor-summit-europe/)! Space is limited so its always a good idea to register early.
If you have any questions, reach out to the [Amsterdam Team](https://github.com/kubernetes/community/tree/master/events/2020/03-contributor-summit#team) on Slack in the [#contributor-summit](https://kubernetes.slack.com/archives/C7J893413) channel.
Hope to see you there!
@@ -1,82 +0,0 @@
---
title: Bring your ideas to the world with kubectl plugins
date: 2020-02-28
---
**Author:** Cornelius Weig (TNG Technology Consulting GmbH)
`kubectl` is the most critical tool to interact with Kubernetes and has to address multiple user personas, each with their own needs and opinions.
One way to make `kubectl` do what you need is to build new functionality into `kubectl`.
## Challenges with building commands into `kubectl`
However, that's easier said than done. Being such an important cornerstone of
Kubernetes, any meaningful change to `kubectl` needs to undergo a Kubernetes
Enhancement Proposal (KEP) where the intended change is discussed beforehand.
When it comes to implementation, you'll find that `kubectl` is an ingenious and
complex piece of engineering. It might take a long time to get used to
the processes and style of the codebase to get done what you want to achieve. Next
comes the review process which may go through several rounds until it meets all
the requirements of the Kubernetes maintainers -- after all, they need to take
over ownership of this feature and maintain it from the day it's merged.
When everything goes well, you can finally rejoice. Your code will be shipped
with the next Kubernetes release. Well, that could mean you need to wait
another 3 months to ship your idea in `kubectl` if you are unlucky.
So this was the happy path where everything goes well. But there are good
reasons why your new functionality may never make it into `kubectl`. For one,
`kubectl` has a particular look and feel and violating that style will not be
acceptable by the maintainers. For example, an interactive command that
produces output with colors would be inconsistent with the rest of `kubectl`.
Also, when it comes to tools or commands useful only to a minuscule proportion
of users, the maintainers may simply reject your proposal as `kubectl` needs to
address common needs.
But this doesnt mean you cant ship your ideas to `kubectl` users.
## What if you didnt have to change `kubectl` to add functionality?
This is where `kubectl` [plugins](https://kubernetes.io/docs/tasks/extend-kubectl/kubectl-plugins/) shine.
Since `kubectl` v1.12, you can simply
drop executables into your `PATH`, which follows the naming pattern
`kubectl-myplugin`. Then you can execute this plugin as `kubectl myplugin`, and
it will just feel like a normal sub-command of `kubectl`.
Plugins give you the opportunity to try out new experiences like terminal UIs,
colorful output, specialized functionality, or other innovative ideas. You can
go creative, as youre the owner of your own plugin.
Further, plugins offer safe experimentation space for commands youd like to
propose to `kubectl`. By pre-releasing as a plugin, you can push your
functionality faster to the end-users and quickly gather feedback. For example,
the [kubectl-debug](https://github.com/verb/kubectl-debug) plugin is proposed
to become a built-in command in `kubectl` in a
[KEP](https://github.com/kubernetes/enhancements/blob/master/keps/sig-cli/20190805-kubectl-debug.md)).
In the meanwhile, the plugin author can ship the functionality and collect
feedback using the plugin mechanism.
## How to get started with developing plugins
If you already have an idea for a plugin, how do you best make it happen?
First you have to ask yourself if you can implement it as a wrapper around
existing `kubectl` functionality. If so, writing the plugin as a shell script
is often the best way forward, because the resulting plugin will be small,
works cross-platform, and has a high level of trust because it is not
compiled.
On the other hand, if the plugin logic is complex, a general-purpose language
is usually better. The canonical choice here is Go, because you can use the
excellent `client-go` library to interact with the Kubernetes API. The Kubernetes
maintained [sample-cli-plugin](https://github.com/kubernetes/sample-cli-plugin)
demonstrates some best practices and can be used as a template for new plugin
projects.
When the development is done, you just need to ship your plugin to the
Kubernetes users. For the best plugin installation experience and discoverability,
you should consider doing so via the
[krew](https://github.com/kubernetes-sigs/krew) plugin manager. For an in-depth
discussion about the technical details around `kubectl` plugins, refer to the
documentation on [kubernetes.io](https://kubernetes.io/docs/tasks/extend-kubectl/kubectl-plugins/).
@@ -1,15 +0,0 @@
---
layout: blog
title: Contributor Summit Amsterdam Postponed
date: 2020-03-04
slug: Contributor-Summit-Delayed
---
**Authors:** Dawn Foster (VMware), Jorge Castro (VMware)
The CNCF has announced that [KubeCon + CloudNativeCon EU has been delayed](https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/attend/novel-coronavirus-update/) until July/August of 2020. As a result the Contributor Summit planning team is weighing options for how to proceed. Heres the current plan:
- There will be an in-person Contributor Summit as planned when KubeCon + CloudNativeCon is rescheduled.
- We are looking at options for having additional virtual contributor activities in the meantime.
We will communicate via this blog and the usual communications channels on the final plan. Please bear with us as we adapt when we get more information. Thank you for being patient as the team pivots to bring you a great Contributor Summit!
@@ -1,42 +0,0 @@
---
layout: blog
title: Join SIG Scalability and Learn Kubernetes the Hard Way
date: 2020-03-19
slug: join-sig-scalability
---
**Authors:** Alex Handy
Contributing to SIG Scalability is a great way to learn Kubernetes in all its depth and breadth, and the team would love to have you [join as a contributor](https://github.com/kubernetes/community/tree/master/sig-scalability#scalability-special-interest-group). I took a look at the value of learning the hard way and interviewed the current SIG chairs to give you an idea of what contribution feels like.
## The value of Learning The Hard Way
There is a belief in the software development community that pushes for the most challenging and rigorous possible method of learning a new language or system. These tend to go by the moniker of "Learn \_\_ the Hard Way." Examples abound: Learn Code the Hard Way, Learn Python the Hard Way, and many others originating with Zed Shaw's courses in the topic.
While there are folks out there who offer you a "Learn Kubernetes the Hard Way" type experience (most notably [Kelsey Hightower's](https://github.com/kelseyhightower/kubernetes-the-hard-way)), any "Hard Way" project should attempt to cover every aspect of the core topic's principles.
Therefore, the real way to "Learn Kubernetes the Hard Way," is to join the CNCF and get involved in the project itself. And there is only one SIG that could genuinely offer a full-stack learning experience for Kubernetes: SIG Scalability.
The team behind SIG Scalability is responsible for detecting and dealing with issues that arise when Kubernetes clusters are working with upwards of a thousand nodes. Said [Wojiciech Tyczynski](https://github.com/wojtek-t), a staff software engineer at Google and a member of SIG Scalability, the standard size for a test cluster for this SIG is over 5,000 nodes.
And yet, this SIG is not composed of Ph.D.'s in highly scalable systems designs. Many of the folks working with Tyczynski, for example, joined the SIG knowing very little about these types of issues, and often, very little about Kubernetes.
Working on SIG Scalability is like jumping into the deep end of the pool to learn to swim, and the SIG is inherently concerned with the entire Kubernetes project. SIG Scalability focuses on how Kubernetes functions as a whole and at scale. The SIG Scalability team members have an impetus to learn about every system and to understand how all systems interact with one another.
## A complex and rewarding contributor experience
While that may sound complicated (and it is!), that doesn't mean it's outside the reach of an average developer, tester, or administrator. Google software developer Matt Matejczyk has only been on the team since the beginning of 2019, and he's been a valued member of the team since then, ferreting out bugs.
"I am new here," said Matejczyk. "I joined the team in January [2019]. Before that, I worked on AdWords at Google in New York. Why did I join? I knew some people there, so that was one of the decisions for me to move. I thought at that time that Kubernetes is a unique, cutting edge technology. I thought it'd be cool to work on that."
Matejczyk was correct about the coolness. "It's cool," he said. "So actually, ramping up on scalability is not easy. There are many things you need to understand. You need to understand Kubernetes very well. It can use every part of Kubernetes. I am still ramping up after these 8 months. I think it took me maybe 3 months to get up to decent speed."
When Matejczyk spoke to what he had worked on during those 8 months, he answered, "An interesting example is a regression I have been working on recently. We noticed the overall slowness of Kubernetes control plane in specific scenarios, and we couldn't attribute it to any particular component. In the end, we realized that everything boiled down to the memory allocation on the golang level. It was very counterintuitive to have two completely separate pieces of code (running as a part of the same binary) affecting the performance of each other only because one of them was allocating memory too fast. But connecting all the dots and getting to the bottom of regression like this gives great satisfaction."
Tyczynski said that "It's not only debugging regressions, but it's also debugging and finding bottlenecks. In general, those can be regressions, but those can be things we can improve. The other significant area is extending what we want to guarantee to users. Extending SLA and SLO coverage of the system so users can rely on what they can expect from the system in terms of performance and scalability. Matt is doing much work in extending our tests to be more representative and cover more Kubernetes concepts."
## Give SIG Scalability a try
The SIG Scalability team is always in need of new members, and if you're the sort of developer or tester who loves taking on new complex challenges, and perhaps loves learning things the hard way, consider joining this SIG. As the team points out, adding Kubernetes expertise to your resume is never a bad idea, and this is the one SIG where you can learn it all from top to bottom.
See [the SIG's documentation](https://github.com/kubernetes/community/tree/master/sig-scalability#scalability-special-interest-group) to learn about upcoming meetings, its charter, and more. You can also join the [#sig-scalability Slack channel](https://kubernetes.slack.com/archives/C09QZTRH7) to see what it's like. We hope to see you join in to take advantage of this great opportunity to learn Kubernetes and contribute back at the same time.
@@ -1,199 +0,0 @@
---
layout: blog
title: 'Kong Ingress Controller and Service Mesh: Setting up Ingress to Istio on Kubernetes'
date: 2020-03-18
slug: kong-ingress-controller-and-istio-service-mesh
---
**Author:** Kevin Chen, Kong
Kubernetes has become the de facto way to orchestrate containers and the services within services. But how do we give services outside our cluster access to what is within? Kubernetes comes with the Ingress API object that manages external access to services within a cluster.
Ingress is a group of rules that will proxy inbound connections to endpoints defined by a backend. However, Kubernetes does not know what to do with Ingress resources without an Ingress controller, which is where an open source controller can come into play. In this post, we are going to use one option for this: the Kong Ingress Controller. The Kong Ingress Controller was open-sourced a year ago and recently reached one million downloads. In the recent 0.7 release, service mesh support was also added. Other features of this release include:
* **Built-In Kubernetes Admission Controller**, which validates Custom Resource Definitions (CRD) as they are created or updated and rejects any invalid configurations.
* **In-memory Mode** - Each pods controller actively configures the Kong container in its pod, which limits the blast radius of failure of a single container of Kong or controller container to that pod only.
* **Native gRPC Routing** - gRPC traffic can now be routed via Kong Ingress Controller natively with support for method-based routing.
![K4K-gRPC](/images/blog/Kong-Ingress-Controller-and-Service-Mesh/KIC-gRPC.png)
If you would like a deeper dive into Kong Ingress Controller 0.7, please check out the [GitHub repository](https://github.com/Kong/kubernetes-ingress-controller).
But lets get back to the service mesh support since that will be the main focal point of this blog post. Service mesh allows organizations to address microservices challenges related to security, reliability, and observability by abstracting inter-service communication into a mesh layer. But what if our mesh layer sits within Kubernetes and we still need to expose certain services beyond our cluster? Then you need an Ingress controller such as the Kong Ingress Controller. In this blog post, well cover how to deploy Kong Ingress Controller as your Ingress layer to an Istio mesh. Lets dive right in:
![Kong Kubernetes Ingress Controller](/images/blog/Kong-Ingress-Controller-and-Service-Mesh/k4k8s.png)
### Part 0: Set up Istio on Kubernetes
This blog will assume you have Istio set up on Kubernetes. If you need to catch up to this point, please check out the [Istio documentation](https://istio.io/docs/setup/). It will walk you through setting up Istio on Kubernetes.
### 1. Install the Bookinfo Application
First, we need to label the namespaces that will host our application and Kong proxy. To label our default namespace where the bookinfo app sits, run this command:
```
$ kubectl label namespace default istio-injection=enabled
namespace/default labeled
```
Then create a new namespace that will be hosting our Kong gateway and the Ingress controller:
```
$ kubectl create namespace kong
namespace/kong created
```
Because Kong will be sitting outside the default namespace, be sure you also label the Kong namespace with istio-injection enabled as well:
```
$ kubectl label namespace kong istio-injection=enabled
namespace/kong labeled
```
Having both namespaces labeled `istio-injection=enabled` is necessary. Or else the default configuration will not inject a sidecar container into the pods of your namespaces.
Now deploy your BookInfo application with the following command:
```
$ kubectl apply -f http://bit.ly/bookinfoapp
service/details created
serviceaccount/bookinfo-details created
deployment.apps/details-v1 created
service/ratings created
serviceaccount/bookinfo-ratings created
deployment.apps/ratings-v1 created
service/reviews created
serviceaccount/bookinfo-reviews created
deployment.apps/reviews-v1 created
deployment.apps/reviews-v2 created
deployment.apps/reviews-v3 created
service/productpage created
serviceaccount/bookinfo-productpage created
deployment.apps/productpage-v1 created
```
Lets double-check our Services and Pods to make sure that we have it all set up correctly:
```
$ kubectl get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
details ClusterIP 10.97.125.254 <none> 9080/TCP 29s
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 29h
productpage ClusterIP 10.97.62.68 <none> 9080/TCP 28s
ratings ClusterIP 10.96.15.180 <none> 9080/TCP 28s
reviews ClusterIP 10.104.207.136 <none> 9080/TCP 28s
```
You should see four new services: details, productpage, ratings, and reviews. None of them have an external IP so we will use the [Kong gateway](https://github.com/Kong/kong) to expose the necessary services. And to check pods, run the following command:
```
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
details-v1-c5b5f496d-9wm29 2/2 Running 0 101s
productpage-v1-7d6cfb7dfd-5mc96 2/2 Running 0 100s
ratings-v1-f745cf57b-hmkwf 2/2 Running 0 101s
reviews-v1-85c474d9b8-kqcpt 2/2 Running 0 101s
reviews-v2-ccffdd984-9jnsj 2/2 Running 0 101s
reviews-v3-98dc67b68-nzw97 2/2 Running 0 101s
```
This command outputs useful data, so lets take a second to understand it. If you examine the READY column, each pod has two containers running: the service and an Envoy sidecar injected alongside it. Another thing to highlight is that there are three review pods but only 1 review service. The Envoy sidecar will load balance the traffic to three different review pods that contain different versions, giving us the ability to A/B test our changes. With that said, you should now be able to access your product page!
```
$ kubectl exec -it $(kubectl get pod -l app=ratings -o jsonpath='{.items[0].metadata.name}') -c ratings -- curl productpage:9080/productpage | grep -o "<title>.*</title>"
<title>Simple Bookstore App</title>
```
### 2. Kong Kubernetes Ingress Controller Without Database
To expose your services to the world, we will deploy Kong as the north-south traffic gateway. [Kong 1.1](https://github.com/Kong/kong/releases/tag/1.1.2) released with declarative configuration and DB-less mode. Declarative configuration allows you to specify the desired system state through a YAML or JSON file instead of a sequence of API calls. Using declarative config provides several key benefits to reduce complexity, increase automation and enhance system performance. And with the Kong Ingress Controller, any Ingress rules you apply to the cluster will automatically be configured on the Kong proxy. Lets set up the Kong Ingress Controller and the actual Kong proxy first like this:
```
$ kubectl apply -f https://bit.ly/k4k8s
namespace/kong configured
customresourcedefinition.apiextensions.k8s.io/kongconsumers.configuration.konghq.com created
customresourcedefinition.apiextensions.k8s.io/kongcredentials.configuration.konghq.com created
customresourcedefinition.apiextensions.k8s.io/kongingresses.configuration.konghq.com created
customresourcedefinition.apiextensions.k8s.io/kongplugins.configuration.konghq.com created
serviceaccount/kong-serviceaccount created
clusterrole.rbac.authorization.k8s.io/kong-ingress-clusterrole created
clusterrolebinding.rbac.authorization.k8s.io/kong-ingress-clusterrole-nisa-binding created
configmap/kong-server-blocks created
service/kong-proxy created
service/kong-validation-webhook created
deployment.apps/ingress-kong created
```
To check if the Kong pod is up and running, run:
```
$ kubectl get pods -n kong
NAME READY STATUS RESTARTS AGE
pod/ingress-kong-8b44c9856-9s42v 3/3 Running 0 2m26s
```
There will be three containers within this pod. The first container is the Kong Gateway that will be the Ingress point to your cluster. The second container is the Ingress controller. It uses Ingress resources and updates the proxy to follow rules defined in the resource. And lastly, the third container is the Envoy proxy injected by Istio. Kong will route traffic through the Envoy sidecar proxy to the appropriate service. To send requests into the cluster via our newly deployed Kong Gateway, setup an environment variable with the a URL based on the IP address at which Kong is accessible.
```
$ export PROXY_URL="$(minikube service -n kong kong-proxy --url | head -1)"
$ echo $PROXY_URL
http://192.168.99.100:32728
```
Next, we need to change some configuration so that the side-car Envoy process can route the request correctly based on the host/authority header of the request. Run the following to stop the route from preserving host:
```
$ echo "
apiVersion: configuration.konghq.com/v1
kind: KongIngress
metadata:
name: do-not-preserve-host
route:
preserve_host: false
" | kubectl apply -f -
kongingress.configuration.konghq.com/do-not-preserve-host created
```
And annotate the existing productpage service to set service-upstream as true:
```
$ kubectl annotate svc productpage Ingress.kubernetes.io/service-upstream="true"
service/productpage annotated
```
Now that we have everything set up, we can look at how to use the Ingress resource to help route external traffic to the services within your Istio mesh. Well create an Ingress rule that routes all traffic with the path of `/` to our productpage service:
```
$ echo "
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: productpage
annotations:
configuration.konghq.com: do-not-preserve-host
spec:
rules:
- http:
paths:
- path: /
backend:
serviceName: productpage
servicePort: 9080
" | kubectl apply -f -
ingress.extensions/productpage created
```
And just like that, the Kong Ingress Controller is able to understand the rules you defined in the Ingress resource and routes it to the productpage service! To view the product page services GUI, go to `$PROXY_URL/productpage` in your browser. Or to test it in your command line, try:
```
$ curl $PROXY_URL/productpage
```
That is all I have for this walk-through. If you enjoyed the technologies used in this post, please check out their repositories since they are all open source and would love to have more contributors! Here are their links for your convenience:
* Kong: [[GitHub](https://github.com/Kong/kubernetes-ingress-controller)] [[Twitter](https://twitter.com/thekonginc)]
* Kubernetes: [[GitHub](https://github.com/kubernetes/kubernetes)] [[Twitter](https://twitter.com/kubernetesio)]
* Istio: [[GitHub](https://github.com/istio/istio)] [[Twitter](https://twitter.com/IstioMesh)]
* Envoy: [[GitHub](https://github.com/envoyproxy/envoy)] [[Twitter](https://twitter.com/EnvoyProxy)]
Thank you for following along!
+3 -3
View File
@@ -24,9 +24,9 @@ Once you've set your desired state, the *Kubernetes Control Plane* makes the clu
* **[kubelet](/docs/admin/kubelet/)**, which communicates with the Kubernetes Master.
* **[kube-proxy](/docs/admin/kube-proxy/)**, a network proxy which reflects Kubernetes networking services on each node.
## Kubernetes objects
## Kubernetes Objects
Kubernetes contains a number of abstractions that represent the state of your system: deployed containerized applications and workloads, their associated network and disk resources, and other information about what your cluster is doing. These abstractions are represented by objects in the Kubernetes API. See [Understanding Kubernetes objects](/docs/concepts/overview/working-with-objects/kubernetes-objects/#kubernetes-objects) for more details.
Kubernetes contains a number of abstractions that represent the state of your system: deployed containerized applications and workloads, their associated network and disk resources, and other information about what your cluster is doing. These abstractions are represented by objects in the Kubernetes API. See [Understanding Kubernetes Objects](/docs/concepts/overview/working-with-objects/kubernetes-objects/) for more details.
The basic Kubernetes objects include:
@@ -35,7 +35,7 @@ The basic Kubernetes objects include:
* [Volume](/docs/concepts/storage/volumes/)
* [Namespace](/docs/concepts/overview/working-with-objects/namespaces/)
Kubernetes also contains higher-level abstractions that rely on [controllers](/docs/concepts/architecture/controller/) to build upon the basic objects, and provide additional functionality and convenience features. These include:
Kubernetes also contains higher-level abstractions that rely on [Controllers](/docs/concepts/architecture/controller/) to build upon the basic objects, and provide additional functionality and convenience features. These include:
* [Deployment](/docs/concepts/workloads/controllers/deployment/)
* [DaemonSet](/docs/concepts/workloads/controllers/daemonset/)
@@ -224,13 +224,11 @@ rules:
The following cloud providers have implemented CCMs:
* [Alibaba Cloud](https://github.com/kubernetes/cloud-provider-alibaba-cloud)
* [AWS](https://github.com/kubernetes/cloud-provider-aws)
* [Azure](https://github.com/kubernetes/cloud-provider-azure)
* [BaiduCloud](https://github.com/baidu/cloud-provider-baiducloud)
* [DigitalOcean](https://github.com/digitalocean/digitalocean-cloud-controller-manager)
* [GCP](https://github.com/kubernetes/cloud-provider-gcp)
* [Hetzner](https://github.com/hetznercloud/hcloud-cloud-controller-manager)
* [Linode](https://github.com/linode/linode-cloud-controller-manager)
* [OpenStack](https://github.com/kubernetes/cloud-provider-openstack)
* [Oracle](https://github.com/oracle/oci-cloud-controller-manager)
@@ -26,7 +26,7 @@ closer to the desired state, by turning equipment on or off.
## Controller pattern
A controller tracks at least one Kubernetes resource type.
These [objects](/docs/concepts/overview/working-with-objects/kubernetes-objects/#kubernetes-objects)
These [objects](/docs/concepts/overview/working-with-objects/kubernetes-objects/)
have a spec field that represents the desired state. The
controller(s) for that resource are responsible for making the current
state come closer to that desired state.
+8 -16
View File
@@ -30,7 +30,7 @@ A node's status contains the following information:
* [Capacity and Allocatable](#capacity)
* [Info](#info)
Node status and other details about a node can be displayed using the following command:
Node status and other details about a node can be displayed using below command:
```shell
kubectl describe node <insert-node-name-here>
```
@@ -131,8 +131,6 @@ Kubernetes creates a node object internally (the representation), and
validates the node by health checking based on the `metadata.name` field. If the node is valid -- that is, if all necessary
services are running -- it is eligible to run a pod. Otherwise, it is
ignored for any cluster activity until it becomes valid.
The name of a Node object must be a valid
[DNS subdomain name](/docs/concepts/overview/working-with-objects/names#dns-subdomain-names).
{{< note >}}
Kubernetes keeps the object for the invalid node and keeps checking to see whether it becomes valid.
@@ -159,7 +157,7 @@ controller deletes the node from its list of nodes.
The third is monitoring the nodes' health. The node controller is
responsible for updating the NodeReady condition of NodeStatus to
ConditionUnknown when a node becomes unreachable (i.e. the node controller stops
receiving heartbeats for some reason, for example due to the node being down), and then later evicting
receiving heartbeats for some reason, e.g. due to the node being down), and then later evicting
all the pods from the node (using graceful termination) if the node continues
to be unreachable. (The default timeouts are 40s to start reporting
ConditionUnknown and 5m after that to start evicting pods.) The node controller
@@ -169,7 +167,7 @@ checks the state of each node every `--node-monitor-period` seconds.
Heartbeats, sent by Kubernetes nodes, help determine the availability of a node.
There are two forms of heartbeats: updates of `NodeStatus` and the
[Lease object](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#lease-v1-coordination-k8s-io).
[Lease object](/docs/reference/generated/kubernetes-api/{{< latest-version >}}/#lease-v1-coordination-k8s-io).
Each Node has an associated Lease object in the `kube-node-lease`
{{< glossary_tooltip term_id="namespace" text="namespace">}}.
Lease is a lightweight resource, which improves the performance
@@ -184,13 +182,13 @@ a Lease object.
timeout for unreachable nodes).
- The kubelet creates and then updates its Lease object every 10 seconds
(the default update interval). Lease updates occur independently from the
`NodeStatus` updates. If the Lease update fails, the kubelet retries with exponential backoff starting at 200 milliseconds and capped at 7 seconds.
`NodeStatus` updates.
#### Reliability
In Kubernetes 1.4, we updated the logic of the node controller to better handle
cases when a large number of nodes have problems with reaching the master
(e.g. because the master has networking problems). Starting with 1.4, the node
(e.g. because the master has networking problem). Starting with 1.4, the node
controller looks at the state of all nodes in the cluster when making a
decision about pod eviction.
@@ -214,9 +212,9 @@ there is only one availability zone (the whole cluster).
A key reason for spreading your nodes across availability zones is so that the
workload can be shifted to healthy zones when one entire zone goes down.
Therefore, if all nodes in a zone are unhealthy then the node controller evicts at
the normal rate of `--node-eviction-rate`. The corner case is when all zones are
completely unhealthy (i.e. there are no healthy nodes in the cluster). In such a
Therefore, if all nodes in a zone are unhealthy then node controller evicts at
the normal rate `--node-eviction-rate`. The corner case is when all zones are
completely unhealthy (i.e. there are no healthy nodes in the cluster). In such
case, the node controller assumes that there's some problem with master
connectivity and stops all evictions until some connectivity is restored.
@@ -277,12 +275,6 @@ and do not respect the unschedulable attribute on a node. This assumes that daem
the machine even if it is being drained of applications while it prepares for a reboot.
{{< /note >}}
{{< caution >}}
`kubectl cordon` marks a node as 'unschedulable', which has the side effect of the service
controller removing the node from any LoadBalancer node target lists it was previously
eligible for, effectively removing incoming load balancer traffic from the cordoned node(s).
{{< /caution >}}
### Node capacity
The capacity of the node (number of cpus and amount of memory) is part of the node object.
@@ -21,14 +21,14 @@ Add-ons in each section are sorted alphabetically - the ordering does not imply
* [ACI](https://www.github.com/noironetworks/aci-containers) provides integrated container networking and network security with Cisco ACI.
* [Calico](https://docs.projectcalico.org/latest/introduction/) is a networking and network policy provider. Calico supports a flexible set of networking options so you can choose the most efficient option for your situation, including non-overlay and overlay networks, with or without BGP. Calico uses the same engine to enforce network policy for hosts, pods, and (if using Istio & Envoy) applications at the service mesh layer.
* [Calico](https://docs.projectcalico.org/latest/getting-started/kubernetes/) is a secure L3 networking and network policy provider.
* [Canal](https://github.com/tigera/canal/tree/master/k8s-install) unites Flannel and Calico, providing networking and network policy.
* [Cilium](https://github.com/cilium/cilium) is a L3 network and network policy plugin that can enforce HTTP/API/L7 policies transparently. Both routing and overlay/encapsulation mode are supported, and it can work on top of other CNI plugins.
* [Cilium](https://github.com/cilium/cilium) is a L3 network and network policy plugin that can enforce HTTP/API/L7 policies transparently. Both routing and overlay/encapsulation mode are supported.
* [CNI-Genie](https://github.com/Huawei-PaaS/CNI-Genie) enables Kubernetes to seamlessly connect to a choice of CNI plugins, such as Calico, Canal, Flannel, Romana, or Weave.
* [Contiv](http://contiv.github.io) provides configurable networking (native L3 using BGP, overlay using vxlan, classic L2, and Cisco-SDN/ACI) for various use cases and a rich policy framework. Contiv project is fully [open sourced](http://github.com/contiv). The [installer](http://github.com/contiv/install) provides both kubeadm and non-kubeadm based installation options.
* [Contrail](http://www.juniper.net/us/en/products-services/sdn/contrail/contrail-networking/), based on [Tungsten Fabric](https://tungsten.io), is an open source, multi-cloud network virtualization and policy management platform. Contrail and Tungsten Fabric are integrated with orchestration systems such as Kubernetes, OpenShift, OpenStack and Mesos, and provide isolation modes for virtual machines, containers/pods and bare metal workloads.
* [Flannel](https://github.com/coreos/flannel/blob/master/Documentation/kubernetes.md) is an overlay network provider that can be used with Kubernetes.
* [Knitter](https://github.com/ZTE/Knitter/) is a plugin to support multiple network interfaces in a Kubernetes pod.
* [Knitter](https://github.com/ZTE/Knitter/) is a network solution supporting multiple networking in Kubernetes.
* [Multus](https://github.com/Intel-Corp/multus-cni) is a Multi plugin for multiple network support in Kubernetes to support all CNI plugins (e.g. Calico, Cilium, Contiv, Flannel), in addition to SRIOV, DPDK, OVS-DPDK and VPP based workloads in Kubernetes.
* [NSX-T](https://docs.vmware.com/en/VMware-NSX-T/2.0/nsxt_20_ncp_kubernetes.pdf) Container Plug-in (NCP) provides integration between VMware NSX-T and container orchestrators such as Kubernetes, as well as integration between NSX-T and container-based CaaS/PaaS platforms such as Pivotal Container Service (PKS) and OpenShift.
* [Nuage](https://github.com/nuagenetworks/nuage-kubernetes/blob/v5.1.1-1/docs/kubernetes-1-installation.rst) is an SDN platform that provides policy-based networking between Kubernetes Pods and non-Kubernetes environments with visibility and security monitoring.
@@ -94,7 +94,7 @@ Different settings can be applied to a load balancer service in AWS using _annot
* `service.beta.kubernetes.io/aws-load-balancer-access-log-s3-bucket-prefix`: Used to specify access log s3 bucket prefix.
* `service.beta.kubernetes.io/aws-load-balancer-additional-resource-tags`: Used on the service to specify a comma-separated list of key-value pairs which will be recorded as additional tags in the ELB. For example: `"Key1=Val1,Key2=Val2,KeyNoVal1=,KeyNoVal2"`.
* `service.beta.kubernetes.io/aws-load-balancer-backend-protocol`: Used on the service to specify the protocol spoken by the backend (pod) behind a listener. If `http` (default) or `https`, an HTTPS listener that terminates the connection and parses headers is created. If set to `ssl` or `tcp`, a "raw" SSL listener is used. If set to `http` and `aws-load-balancer-ssl-cert` is not used then a HTTP listener is used.
* `service.beta.kubernetes.io/aws-load-balancer-ssl-cert`: Used on the service to request a secure listener. Value is a valid certificate ARN. For more, see [ELB Listener Config](http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/elb-listener-config.html) CertARN is an IAM or CM certificate ARN, for example `arn:aws:acm:us-east-1:123456789012:certificate/12345678-1234-1234-1234-123456789012`.
* `service.beta.kubernetes.io/aws-load-balancer-ssl-cert`: Used on the service to request a secure listener. Value is a valid certificate ARN. For more, see [ELB Listener Config](http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/elb-listener-config.html) CertARN is an IAM or CM certificate ARN, e.g. `arn:aws:acm:us-east-1:123456789012:certificate/12345678-1234-1234-1234-123456789012`.
* `service.beta.kubernetes.io/aws-load-balancer-connection-draining-enabled`: Used on the service to enable or disable connection draining.
* `service.beta.kubernetes.io/aws-load-balancer-connection-draining-timeout`: Used on the service to specify a connection draining timeout.
* `service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout`: Used on the service to specify the idle connection timeout.
@@ -44,7 +44,7 @@ Note: Not all distros are actively maintained. Choose distros which have been te
* [Certificates](/docs/concepts/cluster-administration/certificates/) describes the steps to generate certificates using different tool chains.
* [Kubernetes Container Environment](/docs/concepts/containers/container-environment/) describes the environment for Kubelet managed containers on a Kubernetes node.
* [Kubernetes Container Environment](/docs/concepts/containers/container-environment-variables/) describes the environment for Kubelet managed containers on a Kubernetes node.
* [Controlling Access to the Kubernetes API](/docs/reference/access-authn-authz/controlling-access/) describes how to set up permissions for users and service accounts.
@@ -0,0 +1,50 @@
---
title: Controller manager metrics
content_template: templates/concept
weight: 100
---
{{% capture overview %}}
Controller manager metrics provide important insight into the performance and health of
the controller manager.
{{% /capture %}}
{{% capture body %}}
## What are controller manager metrics
Controller manager metrics provide important insight into the performance and health of the controller manager.
These metrics include common Go language runtime metrics such as go_routine count and controller specific metrics such as
etcd request latencies or Cloudprovider (AWS, GCE, OpenStack) API latencies that can be used
to gauge the health of a cluster.
Starting from Kubernetes 1.7, detailed Cloudprovider metrics are available for storage operations for GCE, AWS, Vsphere and OpenStack.
These metrics can be used to monitor health of persistent volume operations.
For example, for GCE these metrics are called:
```
cloudprovider_gce_api_request_duration_seconds { request = "instance_list"}
cloudprovider_gce_api_request_duration_seconds { request = "disk_insert"}
cloudprovider_gce_api_request_duration_seconds { request = "disk_delete"}
cloudprovider_gce_api_request_duration_seconds { request = "attach_disk"}
cloudprovider_gce_api_request_duration_seconds { request = "detach_disk"}
cloudprovider_gce_api_request_duration_seconds { request = "list_disk"}
```
## Configuration
In a cluster, controller-manager metrics are available from `http://localhost:10252/metrics`
from the host where the controller-manager is running.
The metrics are emitted in [prometheus format](https://prometheus.io/docs/instrumenting/exposition_formats/) and are human readable.
In a production environment you may want to configure prometheus or some other metrics scraper
to periodically gather these metrics and make them available in some kind of time series database.
{{% /capture %}}
@@ -0,0 +1,186 @@
---
title: Federation
content_template: templates/concept
weight: 80
---
{{% capture overview %}}
{{< deprecationfilewarning >}}
{{< include "federation-deprecation-warning-note.md" >}}
{{< /deprecationfilewarning >}}
This page explains why and how to manage multiple Kubernetes clusters using
federation.
{{% /capture %}}
{{% capture body %}}
## Why federation
Federation makes it easy to manage multiple clusters. It does so by providing 2
major building blocks:
* Sync resources across clusters: Federation provides the ability to keep
resources in multiple clusters in sync. For example, you can ensure that the same deployment exists in multiple clusters.
* Cross cluster discovery: Federation provides the ability to auto-configure DNS servers and load balancers with backends from all clusters. For example, you can ensure that a global VIP or DNS record can be used to access backends from multiple clusters.
Some other use cases that federation enables are:
* High Availability: By spreading load across clusters and auto configuring DNS
servers and load balancers, federation minimises the impact of cluster
failure.
* Avoiding provider lock-in: By making it easier to migrate applications across
clusters, federation prevents cluster provider lock-in.
Federation is not helpful unless you have multiple clusters. Some of the reasons
why you might want multiple clusters are:
* Low latency: Having clusters in multiple regions minimises latency by serving
users from the cluster that is closest to them.
* Fault isolation: It might be better to have multiple small clusters rather
than a single large cluster for fault isolation (for example: multiple
clusters in different availability zones of a cloud provider).
* Scalability: There are scalability limits to a single kubernetes cluster (this
should not be the case for most users. For more details:
[Kubernetes Scaling and Performance Goals](https://git.k8s.io/community/sig-scalability/goals.md)).
* [Hybrid cloud](#hybrid-cloud-capabilities): You can have multiple clusters on different cloud providers or
on-premises data centers.
### Caveats
While there are a lot of attractive use cases for federation, there are also
some caveats:
* Increased network bandwidth and cost: The federation control plane watches all
clusters to ensure that the current state is as expected. This can lead to
significant network cost if the clusters are running in different regions on
a cloud provider or on different cloud providers.
* Reduced cross cluster isolation: A bug in the federation control plane can
impact all clusters. This is mitigated by keeping the logic in federation
control plane to a minimum. It mostly delegates to the control plane in
kubernetes clusters whenever it can. The design and implementation also errs
on the side of safety and avoiding multi-cluster outage.
* Maturity: The federation project is relatively new and is not very mature.
Not all resources are available and many are still alpha. [Issue
88](https://github.com/kubernetes/federation/issues/88) enumerates
known issues with the system that the team is busy solving.
### Hybrid cloud capabilities
Federations of Kubernetes Clusters can include clusters running in
different cloud providers (e.g. Google Cloud, AWS), and on-premises
(e.g. on OpenStack). [Kubefed](/docs/tasks/federation/set-up-cluster-federation-kubefed/) is the recommended way to deploy federated clusters.
Thereafter, your [API resources](#api-resources) can span different clusters
and cloud providers.
## Setting up federation
To be able to federate multiple clusters, you first need to set up a federation
control plane.
Follow the [setup guide](/docs/tutorials/federation/set-up-cluster-federation-kubefed/) to set up the
federation control plane.
## API resources
Once you have the control plane set up, you can start creating federation API
resources.
The following guides explain some of the resources in detail:
* [Cluster](/docs/tasks/federation/administer-federation/cluster/)
* [ConfigMap](/docs/tasks/federation/administer-federation/configmap/)
* [DaemonSets](/docs/tasks/federation/administer-federation/daemonset/)
* [Deployment](/docs/tasks/federation/administer-federation/deployment/)
* [Events](/docs/tasks/federation/administer-federation/events/)
* [Hpa](/docs/tasks/federation/administer-federation/hpa/)
* [Ingress](/docs/tasks/federation/administer-federation/ingress/)
* [Jobs](/docs/tasks/federation/administer-federation/job/)
* [Namespaces](/docs/tasks/federation/administer-federation/namespaces/)
* [ReplicaSets](/docs/tasks/federation/administer-federation/replicaset/)
* [Secrets](/docs/tasks/federation/administer-federation/secret/)
* [Services](/docs/concepts/cluster-administration/federation-service-discovery/)
The [API reference docs](/docs/reference/federation/) list all the
resources supported by federation apiserver.
## Cascading deletion
Kubernetes version 1.6 includes support for cascading deletion of federated
resources. With cascading deletion, when you delete a resource from the
federation control plane, you also delete the corresponding resources in all underlying clusters.
Cascading deletion is not enabled by default when using the REST API. To enable
it, set the option `DeleteOptions.orphanDependents=false` when you delete a
resource from the federation control plane using the REST API. Using `kubectl
delete`
enables cascading deletion by default. You can disable it by running `kubectl
delete --cascade=false`
Note: Kubernetes version 1.5 included cascading deletion support for a subset of
federation resources.
## Scope of a single cluster
On IaaS providers such as Google Compute Engine or Amazon Web Services, a VM exists in a
[zone](https://cloud.google.com/compute/docs/zones) or [availability
zone](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html).
We suggest that all the VMs in a Kubernetes cluster should be in the same availability zone, because:
- compared to having a single global Kubernetes cluster, there are fewer single-points of failure.
- compared to a cluster that spans availability zones, it is easier to reason about the availability properties of a
single-zone cluster.
- when the Kubernetes developers are designing the system (e.g. making assumptions about latency, bandwidth, or
correlated failures) they are assuming all the machines are in a single data center, or otherwise closely connected.
It is recommended to run fewer clusters with more VMs per availability zone; but it is possible to run multiple clusters per availability zones.
Reasons to prefer fewer clusters per availability zone are:
- improved bin packing of Pods in some cases with more nodes in one cluster (less resource fragmentation).
- reduced operational overhead (though the advantage is diminished as ops tooling and processes mature).
- reduced costs for per-cluster fixed resource costs, e.g. apiserver VMs (but small as a percentage
of overall cluster cost for medium to large clusters).
Reasons to have multiple clusters include:
- strict security policies requiring isolation of one class of work from another (but, see Partitioning Clusters
below).
- test clusters to canary new Kubernetes releases or other cluster software.
## Selecting the right number of clusters
The selection of the number of Kubernetes clusters may be a relatively static choice, only revisited occasionally.
By contrast, the number of nodes in a cluster and the number of pods in a service may change frequently according to
load and growth.
To pick the number of clusters, first, decide which regions you need to be in to have adequate latency to all your end users, for services that will run
on Kubernetes (if you use a Content Distribution Network, the latency requirements for the CDN-hosted content need not
be considered). Legal issues might influence this as well. For example, a company with a global customer base might decide to have clusters in US, EU, AP, and SA regions.
Call the number of regions to be in `R`.
Second, decide how many clusters should be able to be unavailable at the same time, while still being available. Call
the number that can be unavailable `U`. If you are not sure, then 1 is a fine choice.
If it is allowable for load-balancing to direct traffic to any region in the event of a cluster failure, then
you need at least the larger of `R` or `U + 1` clusters. If it is not (e.g. you want to ensure low latency for all
users in the event of a cluster failure), then you need to have `R * (U + 1)` clusters
(`U + 1` in each of `R` regions). In any case, try to put each cluster in a different zone.
Finally, if any of your clusters would need more than the maximum recommended number of nodes for a Kubernetes cluster, then
you may need even more clusters. Kubernetes v1.3 supports clusters up to 1000 nodes in size. Kubernetes v1.8 supports
clusters up to 5000 nodes. See [Building Large Clusters](/docs/setup/best-practices/cluster-large/) for more guidance.
{{% /capture %}}
{{% capture whatsnext %}}
* Learn more about the [Federation
proposal](https://github.com/kubernetes/community/blob/{{< param "githubbranch" >}}/contributors/design-proposals/multicluster/federation.md).
* See this [setup guide](/docs/tutorials/federation/set-up-cluster-federation-kubefed/) for cluster federation.
* See this [Kubecon2016 talk on federation](https://www.youtube.com/watch?v=pq9lbkmxpS8)
* See this [Kubecon2017 Europe update on federation](https://www.youtube.com/watch?v=kwOvOLnFYck)
* See this [Kubecon2018 Europe update on sig-multicluster](https://www.youtube.com/watch?v=vGZo5DaThQU)
* See this [Kubecon2018 Europe Federation-v2 prototype presentation](https://youtu.be/q27rbaX5Jis?t=7m20s)
* See this [Federation-v2 Userguide](https://github.com/kubernetes-sigs/federation-v2/blob/master/docs/userguide.md)
{{% /capture %}}
@@ -76,7 +76,7 @@ should set up a solution to address that.
For example, in Kubernetes clusters, deployed by the `kube-up.sh` script,
there is a [`logrotate`](https://linux.die.net/man/8/logrotate)
tool configured to run each hour. You can also set up a container runtime to
rotate application's logs automatically, for example by using Docker's `log-opt`.
rotate application's logs automatically, e.g. by using Docker's `log-opt`.
In the `kube-up.sh` script, the latter approach is used for COS image on GCP,
and the former approach is used in any other environment. In both cases, by
default rotation is configured to take place when log file exceeds 10MB.
@@ -424,16 +424,16 @@ At some point, you'll eventually need to update your deployed application, typic
We'll guide you through how to create and update applications with Deployments.
Let's say you were running version 1.14.2 of nginx:
Let's say you were running version 1.7.9 of nginx:
```shell
kubectl run my-nginx --image=nginx:1.14.2 --replicas=3
kubectl run my-nginx --image=nginx:1.7.9 --replicas=3
```
```shell
deployment.apps/my-nginx created
```
To update to version 1.16.1, simply change `.spec.template.spec.containers[0].image` from `nginx:1.14.2` to `nginx:1.16.1`, with the kubectl commands we learned above.
To update to version 1.9.1, simply change `.spec.template.spec.containers[0].image` from `nginx:1.7.9` to `nginx:1.9.1`, with the kubectl commands we learned above.
```shell
kubectl edit deployment/my-nginx
@@ -1,132 +0,0 @@
---
title: Metrics For The Kubernetes Control Plane
reviewers:
- brancz
- logicalhan
- RainbowMango
content_template: templates/concept
weight: 60
aliases:
- controller-metrics.md
---
{{% capture overview %}}
System component metrics can give a better look into what is happening inside them. Metrics are particularly useful for building dashboards and alerts.
Metrics in Kubernetes control plane are emitted in [prometheus format](https://prometheus.io/docs/instrumenting/exposition_formats/) and are human readable.
{{% /capture %}}
{{% capture body %}}
## Metrics in Kubernetes
In most cases metrics are available on `/metrics` endpoint of the HTTP server. For components that doesn't expose endpoint by default it can be enabled using `--bind-address` flag.
Examples of those components:
* {{< glossary_tooltip term_id="kube-controller-manager" text="kube-controller-manager" >}}
* {{< glossary_tooltip term_id="kube-proxy" text="kube-proxy" >}}
* {{< glossary_tooltip term_id="kube-apiserver" text="kube-apiserver" >}}
* {{< glossary_tooltip term_id="kube-scheduler" text="kube-scheduler" >}}
* {{< glossary_tooltip term_id="kubelet" text="kubelet" >}}
In a production environment you may want to configure [Prometheus Server](https://prometheus.io/) or some other metrics scraper
to periodically gather these metrics and make them available in some kind of time series database.
Note that {{< glossary_tooltip term_id="kubelet" text="kubelet" >}} also exposes metrics in `/metrics/cadvisor`, `/metrics/resource` and `/metrics/probes` endpoints. Those metrics do not have same lifecycle.
If your cluster uses {{< glossary_tooltip term_id="rbac" text="RBAC" >}}, reading metrics requires authorization via a user, group or ServiceAccount with a ClusterRole that allows accessing `/metrics`.
For example:
```
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: prometheus
rules:
- nonResourceURLs:
- "/metrics"
verbs:
- get
```
## Metric lifecycle
Alpha metric → Stable metric → Deprecated metric → Hidden metric → Deletion
Alpha metrics have no stability guarantees; as such they can be modified or deleted at any time.
Stable metrics can be guaranteed to not change; Specifically, stability means:
* the metric itself will not be deleted (or renamed)
* the type of metric will not be modified
Deprecated metric signal that the metric will eventually be deleted; to find which version, you need to check annotation, which includes from which kubernetes version that metric will be considered deprecated.
Before deprecation:
```
# HELP some_counter this counts things
# TYPE some_counter counter
some_counter 0
```
After deprecation:
```
# HELP some_counter (Deprecated since 1.15.0) this counts things
# TYPE some_counter counter
some_counter 0
```
Once a metric is hidden then by default the metrics is not published for scraping. To use a hidden metric, you need to override the configuration for the relevant cluster component.
Once a metric is deleted, the metric is not published. You cannot change this using an override.
## Show Hidden Metrics
As described above, admins can enable hidden metrics through a command-line flag on a specific binary. This intends to be used as an escape hatch for admins if they missed the migration of the metrics deprecated in the last release.
The flag `show-hidden-metrics-for-version` takes a version for which you want to show metrics deprecated in that release. The version is expressed as x.y, where x is the major version, y is the minor version. The patch version is not needed even though a metrics can be deprecated in a patch release, the reason for that is the metrics deprecation policy runs against the minor release.
The flag can only take the previous minor version as it's value. All metrics hidden in previous will be emitted if admins set the previous version to `show-hidden-metrics-for-version`. The too old version is not allowed because this violates the metrics deprecated policy.
Take metric `A` as an example, here assumed that `A` is deprecated in 1.n. According to metrics deprecated policy, we can reach the following conclusion:
* In release `1.n`, the metric is deprecated, and it can be emitted by default.
* In release `1.n+1`, the metric is hidden by default and it can be emitted by command line `show-hidden-metrics-for-version=1.n`.
* In release `1.n+2`, the metric should be removed from the codebase. No escape hatch anymore.
If you're upgrading from release `1.12` to `1.13`, but still depend on a metric `A` deprecated in `1.12`, you should set hidden metrics via command line: `--show-hidden-metrics=1.12` and remember to remove this metric dependency before upgrading to `1.14`
## Component metrics
### kube-controller-manager metrics
Controller manager metrics provide important insight into the performance and health of the controller manager.
These metrics include common Go language runtime metrics such as go_routine count and controller specific metrics such as
etcd request latencies or Cloudprovider (AWS, GCE, OpenStack) API latencies that can be used
to gauge the health of a cluster.
Starting from Kubernetes 1.7, detailed Cloudprovider metrics are available for storage operations for GCE, AWS, Vsphere and OpenStack.
These metrics can be used to monitor health of persistent volume operations.
For example, for GCE these metrics are called:
```
cloudprovider_gce_api_request_duration_seconds { request = "instance_list"}
cloudprovider_gce_api_request_duration_seconds { request = "disk_insert"}
cloudprovider_gce_api_request_duration_seconds { request = "disk_delete"}
cloudprovider_gce_api_request_duration_seconds { request = "attach_disk"}
cloudprovider_gce_api_request_duration_seconds { request = "detach_disk"}
cloudprovider_gce_api_request_duration_seconds { request = "list_disk"}
```
{{% /capture %}}
{{% capture whatsnext %}}
* Read about the [Prometheus text format](https://github.com/prometheus/docs/blob/master/content/docs/instrumenting/exposition_formats.md#text-based-format) for metrics
* See the list of [stable Kubernetes metrics](https://github.com/kubernetes/kubernetes/blob/master/test/instrumentation/testdata/stable-metrics-list.yaml)
* Read about the [Kubernetes deprecation policy](https://kubernetes.io/docs/reference/using-api/deprecation-policy/#deprecating-a-feature-or-behavior )
{{% /capture %}}
@@ -111,12 +111,6 @@ Using this CNI plugin allows Kubernetes pods to have the same IP address inside
Additionally, the CNI can be run alongside [Calico for network policy enforcement](https://docs.aws.amazon.com/eks/latest/userguide/calico.html). The AWS VPC CNI project is open source with [documentation on GitHub](https://github.com/aws/amazon-vpc-cni-k8s).
### Azure CNI for Kubernetes
[Azure CNI](https://docs.microsoft.com/en-us/azure/virtual-network/container-networking-overview) is an [open source](https://github.com/Azure/azure-container-networking/blob/master/docs/cni.md) plugin that integrates Kubernetes Pods with an Azure Virtual Network (also known as VNet) providing network performance at par with VMs. Pods can connect to peered VNet and to on-premises over Express Route or site-to-site VPN and are also directly reachable from these networks. Pods can access Azure services, such as storage and SQL, that are protected by Service Endpoints or Private Link. You can use VNet security policies and routing to filter Pod traffic. The plugin assigns VNet IPs to Pods by utilizing a pool of secondary IPs pre-configured on the Network Interface of a Kubernetes node.
Azure CNI is available natively in the [Azure Kubernetes Service (AKS)] (https://docs.microsoft.com/en-us/azure/aks/configure-azure-cni).
### Big Cloud Fabric from Big Switch Networks
[Big Cloud Fabric](https://www.bigswitch.com/container-network-automation) is a cloud native networking architecture, designed to run Kubernetes in private cloud/on-premises environments. Using unified physical & virtual SDN, Big Cloud Fabric tackles inherent container networking problems such as load balancing, visibility, troubleshooting, security policies & container traffic monitoring.
@@ -131,7 +125,7 @@ BCF was recognized by Gartner as a visionary in the latest [Magic Quadrant](http
providing and transparently securing network connectivity between application
containers. Cilium is L7/HTTP aware and can enforce network policies on L3-L7
using an identity based security model that is decoupled from network
addressing, and it can be used in combination with other CNI plugins.
addressing.
### CNI-Genie from Huawei
@@ -17,7 +17,7 @@ There are several ways to do this, and the recommended approaches all use
[label selectors](/docs/concepts/overview/working-with-objects/labels/) to make the selection.
Generally such constraints are unnecessary, as the scheduler will automatically do a reasonable placement
(e.g. spread your pods across nodes, not place the pod on a node with insufficient free resources, etc.)
but there are some circumstances where you may want more control on a node where a pod lands, for example to ensure
but there are some circumstances where you may want more control on a node where a pod lands, e.g. to ensure
that a pod ends up on a machine with an SSD attached to it, or to co-locate pods from two different
services that communicate a lot into the same availability zone.
@@ -111,10 +111,9 @@ For example, `example.com.node-restriction.kubernetes.io/fips=true` or `example.
`nodeSelector` provides a very simple way to constrain pods to nodes with particular labels. The affinity/anti-affinity
feature, greatly expands the types of constraints you can express. The key enhancements are
1. The affinity/anti-affinity language is more expressive. The language offers more matching rules
besides exact matches created with a logical AND operation;
1. the language is more expressive (not just "AND of exact match")
2. you can indicate that the rule is "soft"/"preference" rather than a hard requirement, so if the scheduler
can't satisfy it, the pod will still be scheduled;
can't satisfy it, the pod will still be scheduled
3. you can constrain against labels on other pods running on the node (or other topological domain),
rather than against labels on the node itself, which allows rules about which pods can and cannot be co-located
@@ -160,9 +159,9 @@ You can use `NotIn` and `DoesNotExist` to achieve node anti-affinity behavior, o
If you specify both `nodeSelector` and `nodeAffinity`, *both* must be satisfied for the pod
to be scheduled onto a candidate node.
If you specify multiple `nodeSelectorTerms` associated with `nodeAffinity` types, then the pod can be scheduled onto a node **only if all** `nodeSelectorTerms` can be satisfied.
If you specify multiple `nodeSelectorTerms` associated with `nodeAffinity` types, then the pod can be scheduled onto a node **if one of** the `nodeSelectorTerms` is satisfied.
If you specify multiple `matchExpressions` associated with `nodeSelectorTerms`, then the pod can be scheduled onto a node **if one of** the `matchExpressions` is satisfied.
If you specify multiple `matchExpressions` associated with `nodeSelectorTerms`, then the pod can be scheduled onto a node **only if all** `matchExpressions` can be satisfied.
If you remove or change the label of the node where the pod is scheduled, the pod won't be removed. In other words, the affinity selection works only at the time of scheduling the pod.
@@ -177,7 +176,7 @@ Y is expressed as a LabelSelector with an optional associated list of namespaces
(and therefore the labels on pods are implicitly namespaced),
a label selector over pod labels must specify which namespaces the selector should apply to. Conceptually X is a topology domain
like node, rack, cloud provider zone, cloud provider region, etc. You express it using a `topologyKey` which is the
key for the node label that the system uses to denote such a topology domain; for example, see the label keys listed above
key for the node label that the system uses to denote such a topology domain, e.g. see the label keys listed above
in the section [Interlude: built-in node labels](#built-in-node-labels).
{{< note >}}
@@ -187,7 +186,7 @@ not recommend using them in clusters larger than several hundred nodes.
{{< /note >}}
{{< note >}}
Pod anti-affinity requires nodes to be consistently labelled, in other words every node in the cluster must have an appropriate label matching `topologyKey`. If some or all nodes are missing the specified `topologyKey` label, it can lead to unintended behavior.
Pod anti-affinity requires nodes to be consistently labelled, i.e. every node in the cluster must have an appropriate label matching `topologyKey`. If some or all nodes are missing the specified `topologyKey` label, it can lead to unintended behavior.
{{< /note >}}
As with node affinity, there are currently two types of pod affinity and anti-affinity, called `requiredDuringSchedulingIgnoredDuringExecution` and
@@ -229,7 +228,7 @@ for performance and security reasons, there are some constraints on topologyKey:
1. For affinity and for `requiredDuringSchedulingIgnoredDuringExecution` pod anti-affinity,
empty `topologyKey` is not allowed.
2. For `requiredDuringSchedulingIgnoredDuringExecution` pod anti-affinity, the admission controller `LimitPodHardAntiAffinityTopology` was introduced to limit `topologyKey` to `kubernetes.io/hostname`. If you want to make it available for custom topologies, you may modify the admission controller, or simply disable it.
3. For `preferredDuringSchedulingIgnoredDuringExecution` pod anti-affinity, empty `topologyKey` is not allowed.
3. For `preferredDuringSchedulingIgnoredDuringExecution` pod anti-affinity, empty `topologyKey` is interpreted as "all topologies" ("all topologies" here is now limited to the combination of `kubernetes.io/hostname`, `failure-domain.beta.kubernetes.io/zone` and `failure-domain.beta.kubernetes.io/region`).
4. Except for the above cases, the `topologyKey` can be any legal label-key.
In addition to `labelSelector` and `topologyKey`, you can optionally specify a list `namespaces`
@@ -319,7 +318,7 @@ spec:
topologyKey: "kubernetes.io/hostname"
containers:
- name: web-app
image: nginx:1.16-alpine
image: nginx:1.12-alpine
```
If we create the above two deployments, our three node cluster should look like below.
@@ -367,7 +366,7 @@ Some of the limitations of using `nodeName` to select nodes are:
some cases may be automatically deleted.
- If the named node does not have the resources to accommodate the
pod, the pod will fail and its reason will indicate why,
for example OutOfmemory or OutOfcpu.
e.g. OutOfmemory or OutOfcpu.
- Node names in cloud environments are not always predictable or
stable.
@@ -68,7 +68,13 @@ resource requests/limits of that type for each Container in the Pod.
## Meaning of CPU
Limits and requests for CPU resources are measured in *cpu* units.
One cpu, in Kubernetes, is equivalent to **1 vCPU/Core** for cloud providers and **1 hyperthread** on bare-metal Intel processors.
One cpu, in Kubernetes, is equivalent to:
- 1 AWS vCPU
- 1 GCP Core
- 1 Azure vCore
- 1 IBM vCPU
- 1 *Hyperthread* on a bare-metal Intel processor with Hyperthreading
Fractional requests are allowed. A Container with
`spec.containers[].resources.requests.cpu` of `0.5` is guaranteed half as much
@@ -185,7 +191,7 @@ resource limits, see the
The resource usage of a Pod is reported as part of the Pod status.
If [optional monitoring](/docs/tasks/debug-application-cluster/resource-metrics-pipeline/)
If [optional monitoring](http://releases.k8s.io/{{< param "githubbranch" >}}/cluster/addons/cluster-monitoring/README.md)
is configured for your cluster, then Pod resource usage can be retrieved from
the monitoring system.
@@ -30,7 +30,7 @@ This is a living document. If you think of something that is not on this list bu
- Put object descriptions in annotations, to allow better introspection.
## "Naked" Pods versus ReplicaSets, Deployments, and Jobs {#naked-pods-vs-replicasets-deployments-and-jobs}
## "Naked" Pods vs ReplicaSets, Deployments, and Jobs
- Don't use naked Pods (that is, Pods not bound to a [ReplicaSet](/docs/concepts/workloads/controllers/replicaset/) or [Deployment](/docs/concepts/workloads/controllers/deployment/)) if you can avoid it. Naked Pods will not be rescheduled in the event of a node failure.
@@ -87,7 +87,7 @@ The [imagePullPolicy](/docs/concepts/containers/images/#updating-images) and the
- `imagePullPolicy: Never`: the image is assumed to exist locally. No attempt is made to pull the image.
{{< note >}}
To make sure the container always uses the same version of the image, you can specify its [digest](https://docs.docker.com/engine/reference/commandline/pull/#pull-an-image-by-digest-immutable-identifier); replace `<image-name>:<tag>` with `<image-name>@<digest>` (for example, `image@sha256:45b23dee08af5e43a7fea6c4cf9c25ccf269ee113168c19722f87876677c5cb2`). The digest uniquely identifies a specific version of the image, so it is never updated by Kubernetes unless you change the digest value.
To make sure the container always uses the same version of the image, you can specify its [digest](https://docs.docker.com/engine/reference/commandline/pull/#pull-an-image-by-digest-immutable-identifier), for example `sha256:45b23dee08af5e43a7fea6c4cf9c25ccf269ee113168c19722f87876677c5cb2`. The digest uniquely identifies a specific version of the image, so it is never updated by Kubernetes unless you change the digest value.
{{< /note >}}
{{< note >}}
@@ -108,3 +108,4 @@ The caching semantics of the underlying image provider make even `imagePullPolic
{{% /capture %}}
@@ -16,25 +16,42 @@ importance of a Pod relative to other Pods. If a Pod cannot be scheduled, the
scheduler tries to preempt (evict) lower priority Pods to make scheduling of the
pending Pod possible.
In Kubernetes 1.9 and later, Priority also affects scheduling order of Pods and
out-of-resource eviction ordering on the Node.
Pod priority and preemption graduated to beta in Kubernetes 1.11 and to GA in
Kubernetes 1.14. They have been enabled by default since 1.11.
In Kubernetes versions where Pod priority and preemption is still an alpha-level
feature, you need to explicitly enable it. To use these features in the older
versions of Kubernetes, follow the instructions in the documentation for your
Kubernetes version, by going to the documentation archive version for your
Kubernetes version.
Kubernetes Version | Priority and Preemption State | Enabled by default
------------------ | :---------------------------: | :----------------:
1.8 | alpha | no
1.9 | alpha | no
1.10 | alpha | no
1.11 | beta | yes
1.14 | stable | yes
{{< warning >}}In a cluster where not all users are trusted, a
malicious user could create pods at the highest possible priorities, causing
other pods to be evicted/not get scheduled. To resolve this issue,
[ResourceQuota](/docs/concepts/policy/resource-quotas/) is
augmented to support Pod priority. An admin can create ResourceQuota for users
at specific priority levels, preventing them from creating pods at high
priorities. This feature is in beta since Kubernetes 1.12.
{{< /warning >}}
{{% /capture %}}
{{% capture body %}}
{{< warning >}}
In a cluster where not all users are trusted, a malicious user could create Pods
at the highest possible priorities, causing other Pods to be evicted/not get
scheduled.
An administrator can use ResourceQuota to prevent users from creating pods at
high priorities.
See [limit Priority Class consumption by default](/docs/concepts/policy/resource-quotas/#limit-priority-class-consumption-by-default)
for details.
{{< /warning >}}
## How to use priority and preemption
To use priority and preemption:
To use priority and preemption in Kubernetes 1.11 and later, follow these steps:
1. Add one or more [PriorityClasses](#priorityclass).
@@ -45,12 +62,6 @@ To use priority and preemption:
Keep reading for more information about these steps.
{{< note >}}
Kubernetes already ships with two PriorityClasses:
`system-cluster-critical` and `system-node-critical`.
These are common classes and are used to [ensure that critical components are always scheduled first](/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods/).
{{< /note >}}
If you try the feature and then decide to disable it, you must remove the
PodPriority command-line flag or set it to `false`, and then restart the API
server and scheduler. After the feature is disabled, the existing Pods keep
@@ -60,20 +71,21 @@ Pods.
## How to disable preemption
{{< caution >}}
Critical pods rely on scheduler preemption to be scheduled when a cluster
is under resource pressure. For this reason, it is not recommended to
disable preemption.
{{< /caution >}}
{{< note >}}
In Kubernetes 1.12+, critical pods rely on scheduler preemption to be scheduled
when a cluster is under resource pressure. For this reason, it is not
recommended to disable preemption.
{{< /note >}}
{{< note >}}
In Kubernetes 1.15 and later, if the feature `NonPreemptingPriority` is enabled,
In Kubernetes 1.15 and later,
if the feature `NonPreemptingPriority` is enabled,
PriorityClasses have the option to set `preemptionPolicy: Never`.
This will prevent pods of that PriorityClass from preempting other pods.
{{< /note >}}
Preemption is controlled by a kube-scheduler flag `disablePreemption`, which is
set to `false` by default.
In Kubernetes 1.11 and later, preemption is controlled by a kube-scheduler flag
`disablePreemption`, which is set to `false` by default.
If you want to disable preemption despite the above note, you can set
`disablePreemption` to `true`.
@@ -99,9 +111,6 @@ priority class name to the integer value of the priority. The name is specified
in the `name` field of the PriorityClass object's metadata. The value is
specified in the required `value` field. The higher the value, the higher the
priority.
The name of a PriorityClass object must be a valid
[DNS subdomain name](/docs/concepts/overview/working-with-objects/names#dns-subdomain-names),
and it cannot be prefixed with `system-`.
A PriorityClass object can have any 32-bit integer value smaller than or equal
to 1 billion. Larger numbers are reserved for critical system Pods that should
@@ -143,9 +152,12 @@ globalDefault: false
description: "This priority class should be used for XYZ service pods only."
```
## Non-preempting PriorityClass {#non-preempting-priority-class}
### Non-preempting PriorityClasses (alpha) {#non-preempting-priority-class}
{{< feature-state for_k8s_version="1.15" state="alpha" >}}
1.15 adds the `PreemptionPolicy` field as an alpha feature.
It is disabled by default in 1.15,
and requires the `NonPreemptingPriority`[feature gate](/docs/reference/command-line-tools-reference/feature-gates/
) to be enabled.
Pods with `PreemptionPolicy: Never` will be placed in the scheduling queue
ahead of lower-priority pods,
@@ -169,10 +181,6 @@ which will allow pods of that PriorityClass to preempt lower-priority pods
If `PreemptionPolicy` is set to `Never`,
pods in that PriorityClass will be non-preempting.
The use of the `PreemptionPolicy` field requires the `NonPreemptingPriority`
[feature gate](/docs/reference/command-line-tools-reference/feature-gates/)
to be enabled.
An example use case is for data science workloads.
A user may submit a job that they want to be prioritized above other workloads,
but do not wish to discard existing work by preempting running pods.
@@ -180,7 +188,7 @@ The high priority job with `PreemptionPolicy: Never` will be scheduled
ahead of other queued pods,
as soon as sufficient cluster resources "naturally" become free.
### Example Non-preempting PriorityClass
#### Example Non-preempting PriorityClass
```yaml
apiVersion: scheduling.k8s.io/v1
@@ -222,12 +230,12 @@ spec:
### Effect of Pod priority on scheduling order
When Pod priority is enabled, the scheduler orders pending Pods by
their priority and a pending Pod is placed ahead of other pending Pods
with lower priority in the scheduling queue. As a result, the higher
priority Pod may be scheduled sooner than Pods with lower priority if
its scheduling requirements are met. If such Pod cannot be scheduled,
scheduler will continue and tries to schedule other lower priority Pods.
In Kubernetes 1.9 and later, when Pod priority is enabled, scheduler orders
pending Pods by their priority and a pending Pod is placed ahead of other
pending Pods with lower priority in the scheduling queue. As a result, the
higher priority Pod may be scheduled sooner than Pods with lower priority if its
scheduling requirements are met. If such Pod cannot be scheduled, scheduler will
continue and tries to schedule other lower priority Pods.
## Preemption
@@ -273,12 +281,12 @@ point that scheduler preempts victims and the time that Pod P is scheduled. In
order to minimize this gap, one can set graceful termination period of lower
priority Pods to zero or a small number.
#### PodDisruptionBudget is supported, but not guaranteed
#### PodDisruptionBudget is supported, but not guaranteed!
A [Pod Disruption Budget (PDB)](/docs/concepts/workloads/pods/disruptions/)
allows application owners to limit the number of Pods of a replicated application
that are down simultaneously from voluntary disruptions. Kubernetes supports
PDB when preempting Pods, but respecting PDB is best effort. The scheduler tries
that are down simultaneously from voluntary disruptions. Kubernetes 1.9 supports
PDB when preempting Pods, but respecting PDB is best effort. The Scheduler tries
to find victims whose PDB are not violated by preemption, but if no such victims
are found, preemption will still happen, and lower priority Pods will be removed
despite their PDBs being violated.
@@ -329,23 +337,28 @@ gone, and Pod P could possibly be scheduled on Node N.
We may consider adding cross Node preemption in future versions if there is
enough demand and if we find an algorithm with reasonable performance.
## Troubleshooting
## Debugging Pod Priority and Preemption
Pod priority and pre-emption can have unwanted side effects. Here are some
examples of potential problems and ways to deal with them.
Pod Priority and Preemption is a major feature that could potentially disrupt
Pod scheduling if it has bugs.
### Pods are preempted unnecessarily
### Potential problems caused by Priority and Preemption
The followings are some of the potential problems that could be caused by bugs
in the implementation of the feature. This list is not exhaustive.
#### Pods are preempted unnecessarily
Preemption removes existing Pods from a cluster under resource pressure to make
room for higher priority pending Pods. If you give high priorities to
certain Pods by mistake, these unintentionally high priority Pods may cause
preemption in your cluster. Pod priority is specified by setting the
`priorityClassName` field in the Pod's specification. The integer value for
room for higher priority pending Pods. If a user gives high priorities to
certain Pods by mistake, these unintentional high priority Pods may cause
preemption in the cluster. As mentioned above, Pod priority is specified by
setting the `priorityClassName` field of `podSpec`. The integer value of
priority is then resolved and populated to the `priority` field of `podSpec`.
To address the problem, you can change the `priorityClassName` for those Pods
to use lower priority classes, or leave that field empty. An empty
`priorityClassName` is resolved to zero by default.
To resolve the problem, `priorityClassName` of the Pods must be changed to use
lower priority classes or should be left empty. Empty `priorityClassName` is
resolved to zero by default.
When a Pod is preempted, there will be events recorded for the preempted Pod.
Preemption should happen only when a cluster does not have enough resources for
@@ -354,31 +367,29 @@ Pod (preemptor) is higher than the victim Pods. Preemption must not happen when
there is no pending Pod, or when the pending Pods have equal or lower priority
than the victims. If preemption happens in such scenarios, please file an issue.
### Pods are preempted, but the preemptor is not scheduled
#### Pods are preempted, but the preemptor is not scheduled
When pods are preempted, they receive their requested graceful termination
period, which is by default 30 seconds. If the victim Pods do not terminate within
this period, they are forcibly terminated. Once all the victims go away, the
preemptor Pod can be scheduled.
period, which is by default 30 seconds, but it can be any different value as
specified in the PodSpec. If the victim Pods do not terminate within this period,
they are force-terminated. Once all the victims go away, the preemptor Pod can
be scheduled.
While the preemptor Pod is waiting for the victims to go away, a higher priority
Pod may be created that fits on the same Node. In this case, the scheduler will
Pod may be created that fits on the same node. In this case, the scheduler will
schedule the higher priority Pod instead of the preemptor.
This is expected behavior: the Pod with the higher priority should take the place
of a Pod with a lower priority. Other controller actions, such as
[cluster autoscaling](/docs/tasks/administer-cluster/cluster-management/#cluster-autoscaling),
may eventually provide capacity to schedule the pending Pods.
In the absence of such a higher priority Pod, we expect the preemptor Pod to be
scheduled after the graceful termination period of the victims is over.
### Higher priority Pods are preempted before lower priority pods
#### Higher priority Pods are preempted before lower priority pods
The scheduler tries to find nodes that can run a pending Pod. If no node is
found, the scheduler tries to remove Pods with lower priority from an arbitrary
node in order to make room for the pending pod.
If a node with low priority Pods is not feasible to run the pending Pod, the scheduler
may choose another node with higher priority Pods (compared to the Pods on the
other node) for preemption. The victims must still have lower priority than the
preemptor Pod.
The scheduler tries to find nodes that can run a pending Pod and if no node is
found, it tries to remove Pods with lower priority from one node to make room
for the pending pod. If a node with low priority Pods is not feasible to run the
pending Pod, the scheduler may choose another node with higher priority Pods
(compared to the Pods on the other node) for preemption. The victims must still
have lower priority than the preemptor Pod.
When there are multiple nodes available for preemption, the scheduler tries to
choose the node with a set of Pods with lowest priority. However, if such Pods
@@ -386,11 +397,13 @@ have PodDisruptionBudget that would be violated if they are preempted then the
scheduler may choose another node with higher priority Pods.
When multiple nodes exist for preemption and none of the above scenarios apply,
the scheduler chooses a node with the lowest priority.
we expect the scheduler to choose a node with the lowest priority. If that is
not the case, it may indicate a bug in the scheduler.
## Interactions between Pod priority and quality of service {#interactions-of-pod-priority-and-qos}
## Interactions of Pod priority and QoS
Pod priority and {{< glossary_tooltip text="QoS class" term_id="qos-class" >}}
Pod priority and
[QoS](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/node/resource-qos.md)
are two orthogonal features with few interactions and no default restrictions on
setting the priority of a Pod based on its QoS classes. The scheduler's
preemption logic does not consider QoS when choosing preemption targets.
@@ -401,20 +414,15 @@ to schedule the preemptor Pod, or if the lowest priority Pods are protected by
`PodDisruptionBudget`.
The only component that considers both QoS and Pod priority is
[kubelet out-of-resource eviction](/docs/tasks/administer-cluster/out-of-resource/).
[Kubelet out-of-resource eviction](/docs/tasks/administer-cluster/out-of-resource/).
The kubelet ranks Pods for eviction first by whether or not their usage of the
starved resource exceeds requests, then by Priority, and then by the consumption
of the starved compute resource relative to the Pods scheduling requests.
See
[evicting end-user pods](/docs/tasks/administer-cluster/out-of-resource/#evicting-end-user-pods)
for more details.
kubelet out-of-resource eviction does not evict Pods wheir their
[Evicting end-user pods](/docs/tasks/administer-cluster/out-of-resource/#evicting-end-user-pods)
for more details. Kubelet out-of-resource eviction does not evict Pods whose
usage does not exceed their requests. If a Pod with lower priority is not
exceeding its requests, it won't be evicted. Another Pod with higher priority
that exceeds its requests may be evicted.
{{% /capture %}}
{{% capture whatsnext %}}
* Read about using ResourceQuotas in connection with PriorityClasses: [limit Priority Class consumption by default](/docs/concepts/policy/resource-quotas/#limit-priority-class-consumption-by-default)
{{% /capture %}}
File diff suppressed because it is too large Load Diff
@@ -73,7 +73,6 @@ A toleration "matches" a taint if the keys are the same and the effects are the
`Operator` defaults to `Equal` if not specified.
{{< note >}}
There are two special cases:
* An empty `key` with operator `Exists` matches all keys, values and effects which means this
@@ -89,9 +88,8 @@ tolerations:
```yaml
tolerations:
- key: "key"
operator: "Exists"
operator: "Exists"
```
{{< /note >}}
The above example used `effect` of `NoSchedule`. Alternatively, you can use `effect` of `PreferNoSchedule`.
@@ -2,7 +2,7 @@
reviewers:
- mikedanese
- thockin
title: Container Environment
title: Container Environment Variables
content_template: templates/concept
weight: 20
---
@@ -116,7 +116,7 @@ Events:
{{% capture whatsnext %}}
* Learn more about the [Container environment](/docs/concepts/containers/container-environment/).
* Learn more about the [Container environment](/docs/concepts/containers/container-environment-variables/).
* Get hands-on experience
[attaching handlers to Container lifecycle events](/docs/tasks/configure-pod-container/attach-handler-lifecycle-event/).
+12 -21
View File
@@ -67,7 +67,6 @@ Credentials can be provided in several ways:
- use IAM roles and policies to control access to OCIR repositories
- Using Azure Container Registry (ACR)
- Using IBM Cloud Container Registry
- use IAM roles and policies to grant access to IBM Cloud Container Registry
- Configuring Nodes to Authenticate to a Private Registry
- all pods can read any configured private registries
- requires node configuration by cluster administrator
@@ -125,9 +124,9 @@ Troubleshooting:
- Verify all requirements above.
- Get $REGION (e.g. `us-west-2`) credentials on your workstation. SSH into the host and run Docker manually with those creds. Does it work?
- Verify kubelet is running with `--cloud-provider=aws`.
- Increase kubelet log level verbosity to at least 3 and check kubelet logs (e.g. `journalctl -u kubelet`) for log lines like:
- `aws_credentials.go:109] unable to get ECR credentials from cache, checking ECR API`
- `aws_credentials.go:116] Got ECR credentials from ECR API for <AWS account ID for ECR>.dkr.ecr.<AWS region>.amazonaws.com`
- Check kubelet logs (e.g. `journalctl -u kubelet`) for log lines like:
- `plugins.go:56] Registering credential provider: aws-ecr-key`
- `provider.go:91] Refreshing cache for provider: *aws_credentials.ecrProvider`
### Using Azure Container Registry (ACR)
When using [Azure Container Registry](https://azure.microsoft.com/en-us/services/container-registry/)
@@ -149,11 +148,11 @@ Once you have those variables filled in you can
[configure a Kubernetes Secret and use it to deploy a Pod](/docs/concepts/containers/images/#specifying-imagepullsecrets-on-a-pod).
### Using IBM Cloud Container Registry
IBM Cloud Container Registry provides a multi-tenant private image registry that you can use to safely store and share your images. By default, images in your private registry are scanned by the integrated Vulnerability Advisor to detect security issues and potential vulnerabilities. Users in your IBM Cloud account can access your images, or you can use IAM roles and policies to grant access to IBM Cloud Container Registry namespaces.
IBM Cloud Container Registry provides a multi-tenant private image registry that you can use to safely store and share your Docker images. By default, images in your private registry are scanned by the integrated Vulnerability Advisor to detect security issues and potential vulnerabilities. Users in your IBM Cloud account can access your images, or you can create a token to grant access to registry namespaces.
To install the IBM Cloud Container Registry CLI plug-in and create a namespace for your images, see [Getting started with IBM Cloud Container Registry](https://cloud.ibm.com/docs/Registry?topic=registry-getting-started).
To install the IBM Cloud Container Registry CLI plug-in and create a namespace for your images, see [Getting started with IBM Cloud Container Registry](https://cloud.ibm.com/docs/services/Registry?topic=registry-getting-started).
If you are using the same account and region, you can deploy images that are stored in IBM Cloud Container Registry into the default namespace of your IBM Cloud Kubernetes Service cluster without any additional configuration, see [Building containers from images](https://cloud.ibm.com/docs/containers?topic=containers-images). For other configuration options, see [Understanding how to authorize your cluster to pull images from a registry](https://cloud.ibm.com/docs/containers?topic=containers-registry#cluster_registry_auth).
You can use the IBM Cloud Container Registry to deploy containers from [IBM Cloud public images](https://cloud.ibm.com/docs/services/Registry?topic=registry-public_images) and your private images into the `default` namespace of your IBM Cloud Kubernetes Service cluster. To deploy a container into other namespaces, or to use an image from a different IBM Cloud Container Registry region or IBM Cloud account, create a Kubernetes `imagePullSecret`. For more information, see [Building containers from images](https://cloud.ibm.com/docs/containers?topic=containers-images).
### Configuring Nodes to Authenticate to a Private Registry
@@ -206,7 +205,7 @@ example, run these on your desktop/laptop:
Verify by creating a pod that uses a private image, e.g.:
```shell
```yaml
kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
@@ -219,27 +218,20 @@ spec:
imagePullPolicy: Always
command: [ "echo", "SUCCESS" ]
EOF
```
```
pod/private-image-test-1 created
```
If everything is working, then, after a few moments, you can run:
If everything is working, then, after a few moments, you should see:
```shell
kubectl logs private-image-test-1
```
and see that the command outputs:
```
SUCCESS
```
If you suspect that the command failed, you can run:
If it failed, then you will see:
```shell
kubectl describe pods/private-image-test-1 | grep 'Failed'
```
In case of failure, the output is similar to:
```
kubectl describe pods/private-image-test-1 | grep "Failed"
Fri, 26 Jun 2015 15:36:13 -0700 Fri, 26 Jun 2015 15:39:13 -0700 19 {kubelet node-i2hq} spec.containers{uses-private-image} failed Failed to pull image "user/privaterepo:v1": Error: image user/privaterepo:v1 not found
```
@@ -366,8 +358,7 @@ common use cases and suggested solutions.
- Generate registry credential for each tenant, put into secret, and populate secret to each tenant namespace.
- The tenant adds that secret to imagePullSecrets of each namespace.
{{% /capture %}}
If you need access to multiple registries, you can create one secret for each registry.
Kubelet will merge any `imagePullSecrets` into a single virtual `.docker/config.json`
{{% /capture %}}
@@ -1,45 +0,0 @@
---
reviewers:
- erictune
- thockin
title: Containers overview
content_template: templates/concept
weight: 1
---
{{% capture overview %}}
Containers are a technnology for packaging the (compiled) code for an
application along with the dependencies it needs at run time. Each
container that you run is repeatable; the standardisation from having
dependencies included means that you get the same behavior wherever you
run it.
Containers decouple applications from underlying host infrastructure.
This makes deployment easier in different cloud or OS environments.
{{% /capture %}}
{{% capture body %}}
## Container images
A [container image](/docs/concepts/containers/images/) is a ready-to-run
software package, containing everything needed to run an application:
the code and any runtime it requires, application and system libraries,
and default values for any essential settings.
By design, a container is immutable: you cannot change the code of a
container that is already running. If you have a containerized application
and want to make changes, you need to build a new container that includes
the change, then recreate the container to start from the updated image.
## Container runtimes
{{< glossary_definition term_id="container-runtime" length="all" >}}
{{% /capture %}}
{{% capture whatsnext %}}
* Read about [container images](/docs/concepts/containers/images/)
* Read about [Pods](/docs/concepts/workloads/pods/)
{{% /capture %}}
@@ -82,9 +82,6 @@ metadata:
handler: myconfiguration # The name of the corresponding CRI configuration
```
The name of a RuntimeClass object must be a valid
[DNS subdomain name](/docs/concepts/overview/working-with-objects/names#dns-subdomain-names).
{{< note >}}
It is recommended that RuntimeClass write operations (create/update/patch/delete) be
restricted to the cluster administrator. This is typically the default. See [Authorization
@@ -123,7 +120,7 @@ For more details on setting up CRI runtimes, see [CRI installation](/docs/setup/
Kubernetes built-in dockershim CRI does not support runtime handlers.
#### {{< glossary_tooltip term_id="containerd" >}}
#### [containerd](https://containerd.io/)
Runtime handlers are configured through containerd's configuration at
`/etc/containerd/config.toml`. Valid handlers are configured under the runtimes section:
@@ -135,20 +132,19 @@ Runtime handlers are configured through containerd's configuration at
See containerd's config documentation for more details:
https://github.com/containerd/cri/blob/master/docs/config.md
#### {{< glossary_tooltip term_id="cri-o" >}}
#### [cri-o](https://cri-o.io/)
Runtime handlers are configured through CRI-O's configuration at `/etc/crio/crio.conf`. Valid
Runtime handlers are configured through cri-o's configuration at `/etc/crio/crio.conf`. Valid
handlers are configured under the [crio.runtime
table](https://github.com/cri-o/cri-o/blob/master/docs/crio.conf.5.md#crioruntime-table):
table](https://github.com/kubernetes-sigs/cri-o/blob/master/docs/crio.conf.5.md#crioruntime-table):
```
[crio.runtime.runtimes.${HANDLER_NAME}]
runtime_path = "${PATH_TO_BINARY}"
```
See CRI-O's [config documentation][100] for more details.
[100]: https://raw.githubusercontent.com/cri-o/cri-o/9f11d1d/docs/crio.conf.5.md
See cri-o's config documentation for more details:
https://github.com/kubernetes-sigs/cri-o/blob/master/cmd/crio/config.go
### Scheduling
@@ -5,34 +5,30 @@ reviewers:
- cheftako
- chenopis
content_template: templates/concept
weight: 20
weight: 10
---
{{% capture overview %}}
The aggregation layer allows Kubernetes to be extended with additional APIs, beyond what is offered by the core Kubernetes APIs.
The additional APIs can either be ready-made solutions such as [service-catalog](/docs/concepts/extend-kubernetes/service-catalog/), or APIs that you develop yourself.
The aggregation layer is different from [Custom Resources](/docs/concepts/extend-kubernetes/api-extension/custom-resources/), which are a way to make the {{< glossary_tooltip term_id="kube-apiserver" text="kube-apiserver" >}} recognise new kinds of object.
The aggregation layer allows Kubernetes to be extended with additional APIs, beyond what is offered by the core Kubernetes APIs.
{{% /capture %}}
{{% capture body %}}
## Aggregation layer
## Overview
The aggregation layer runs in-process with the kube-apiserver. Until an extension resource is registered, the aggregation layer will do nothing. To register an API, you add an _APIService_ object, which "claims" the URL path in the Kubernetes API. At that point, the aggregation layer will proxy anything sent to that API path (e.g. `/apis/myextension.mycompany.io/v1/…`) to the registered APIService.
The aggregation layer enables installing additional Kubernetes-style APIs in your cluster. These can either be pre-built, existing 3rd party solutions, such as [service-catalog](https://github.com/kubernetes-incubator/service-catalog/blob/master/README.md), or user-created APIs like [apiserver-builder](https://github.com/kubernetes-incubator/apiserver-builder/blob/master/README.md), which can get you started.
The most common way to implement the APIService is to run an *extension API server* in Pod(s) that run in your cluster. If you're using the extension API server to manage resources in your cluster, the extension API server (also written as "extension-apiserver") is typically paired with one or more {{< glossary_tooltip text="controllers" term_id="controller" >}}. The apiserver-builder library provides a skeleton for both extension API servers and the associated controller(s).
The aggregation layer runs in-process with the kube-apiserver. Until an extension resource is registered, the aggregation layer will do nothing. To register an API, users must add an APIService object, which "claims" the URL path in the Kubernetes API. At that point, the aggregation layer will proxy anything sent to that API path (e.g. /apis/myextension.mycompany.io/v1/…) to the registered APIService.
### Response latency
Ordinarily, the APIService will be implemented by an *extension-apiserver* in a pod running in the cluster. This extension-apiserver will normally need to be paired with one or more controllers if active management of the added resources is needed. As a result, the apiserver-builder will actually provide a skeleton for both. As another example, when the service-catalog is installed, it provides both the extension-apiserver and controller for the services it provides.
Extension API servers should have low latency networking to and from the kube-apiserver.
Discovery requests are required to round-trip from the kube-apiserver in five seconds or less.
If your extension API server cannot achieve that latency requirement, consider making changes that let you meet it. You can also set the
`EnableAggregatedDiscoveryTimeout=false` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/) on the kube-apiserver
to disable the timeout restriction. This deprecated feature gate will be removed in a future release.
Extension-apiservers should have low latency connections to and from the kube-apiserver.
In particular, discovery requests are required to round-trip from the kube-apiserver in five seconds or less.
If your deployment cannot achieve this, you should consider how to change it. For now, setting the
`EnableAggregatedDiscoveryTimeout=false` feature gate on the kube-apiserver
will disable the timeout restriction. It will be removed in a future release.
{{% /capture %}}
@@ -41,6 +37,7 @@ to disable the timeout restriction. This deprecated feature gate will be removed
* To get the aggregator working in your environment, [configure the aggregation layer](/docs/tasks/access-kubernetes-api/configure-aggregation-layer/).
* Then, [setup an extension api-server](/docs/tasks/access-kubernetes-api/setup-extension-api-server/) to work with the aggregation layer.
* Also, learn how to [extend the Kubernetes API using Custom Resource Definitions](/docs/tasks/access-kubernetes-api/extend-api-custom-resource-definitions/).
* Read the specification for [APIService](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#apiservice-v1-apiregistration-k8s-io)
{{% /capture %}}
@@ -4,7 +4,7 @@ reviewers:
- enisoc
- deads2k
content_template: templates/concept
weight: 10
weight: 20
---
{{% capture overview %}}
@@ -37,14 +37,14 @@ On their own, custom resources simply let you store and retrieve structured data
When you combine a custom resource with a *custom controller*, custom resources
provide a true _declarative API_.
A [declarative API](/docs/concepts/overview/kubernetes-api/)
A [declarative API](/docs/concepts/overview/working-with-objects/kubernetes-objects/#understanding-kubernetes-objects)
allows you to _declare_ or specify the desired state of your resource and tries to
keep the current state of Kubernetes objects in sync with the desired state.
The controller interprets the structured data as a record of the user's
desired state, and continually maintains this state.
You can deploy and update a custom controller on a running cluster, independently
of the cluster's lifecycle. Custom controllers can work with any kind of resource,
of the cluster's own lifecycle. Custom controllers can work with any kind of resource,
but they are especially effective when combined with custom resources. The
[Operator pattern](https://coreos.com/blog/introducing-operators.html) combines custom
resources and custom controllers. You can use custom controllers to encode domain knowledge
@@ -61,7 +61,7 @@ When creating a new API, consider whether to [aggregate your API with the Kubern
| You want to view your new types in a Kubernetes UI, such as dashboard, alongside built-in types. | Kubernetes UI support is not required. |
| You are developing a new API. | You already have a program that serves your API and works well. |
| You are willing to accept the format restriction that Kubernetes puts on REST resource paths, such as API Groups and Namespaces. (See the [API Overview](/docs/concepts/overview/kubernetes-api/).) | You need to have specific REST paths to be compatible with an already defined REST API. |
| Your resources are naturally scoped to a cluster or namespaces of a cluster. | Cluster or namespace scoped resources are a poor fit; you need control over the specifics of resource paths. |
| Your resources are naturally scoped to a cluster or to namespaces of a cluster. | Cluster or namespace scoped resources are a poor fit; you need control over the specifics of resource paths. |
| You want to reuse [Kubernetes API support features](#common-features). | You don't need those features. |
### Declarative APIs
@@ -83,7 +83,7 @@ Signs that your API might not be declarative include:
- You talk about Remote Procedure Calls (RPCs).
- Directly storing large amounts of data (e.g. > a few kB per object, or >1000s of objects).
- High bandwidth access (10s of requests per second sustained) needed.
- Store end-user data (such as images, PII, etc.) or other large-scale data processed by applications.
- Store end-user data (such as images, PII, etc) or other large-scale data processed by applications.
- The natural operations on the objects are not CRUD-y.
- The API is not easily modeled as objects.
- You chose to represent pending operations with an operation ID or an operation object.
@@ -96,7 +96,7 @@ Use a ConfigMap if any of the following apply:
* You want to put the entire config file into one key of a configMap.
* The main use of the config file is for a program running in a Pod on your cluster to consume the file to configure itself.
* Consumers of the file prefer to consume via file in a Pod or environment variable in a pod, rather than the Kubernetes API.
* You want to perform rolling updates via Deployment, etc., when the file is updated.
* You want to perform rolling updates via Deployment, etc, when the file is updated.
{{< note >}}
Use a [secret](/docs/concepts/configuration/secret/) for sensitive data, which is similar to a configMap but more secure.
@@ -128,12 +128,7 @@ Regardless of how they are installed, the new resources are referred to as Custo
## CustomResourceDefinitions
The [CustomResourceDefinition](/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions/)
API resource allows you to define custom resources.
Defining a CRD object creates a new custom resource with a name and schema that you specify.
The Kubernetes API serves and handles the storage of your custom resource.
The name of a CRD object must be a valid
[DNS subdomain name](/docs/concepts/overview/working-with-objects/names#dns-subdomain-names).
The [CustomResourceDefinition](/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions/) API resource allows you to define custom resources. Defining a CRD object creates a new custom resource with a name and schema that you specify. The Kubernetes API serves and handles the storage of your custom resource.
This frees you from writing your own API server to handle the custom resource,
but the generic nature of the implementation means you have less flexibility than with
@@ -145,7 +140,7 @@ and use a controller to handle events.
## API server aggregation
Usually, each resource in the Kubernetes API requires code that handles REST requests and manages persistent storage of objects. The main Kubernetes API server handles built-in resources like *pods* and *services*, and can also generically handle custom resources through [CRDs](#customresourcedefinitions).
Usually, each resource in the Kubernetes API requires code that handles REST requests and manages persistent storage of objects. The main Kubernetes API server handles built-in resources like *pods* and *services*, and can also handle custom resources in a generic way through [CRDs](#customresourcedefinitions).
The [aggregation layer](/docs/concepts/extend-kubernetes/api-extension/apiserver-aggregation/) allows you to provide specialized
implementations for your custom resources by writing and deploying your own standalone API server.
@@ -167,7 +162,7 @@ CRDs are easier to create than Aggregated APIs.
| CRDs | Aggregated API |
| --------------------------- | -------------- |
| Do not require programming. Users can choose any language for a CRD controller. | Requires programming in Go and building binary and image. |
| Do not require programming. Users can choose any language for a CRD controller. | Requires programming in Go and building binary and image. Users can choose any language for a CRD controller. |
| No additional service to run; CRs are handled by API Server. | An additional service to create and that could fail. |
| No ongoing support once the CRD is created. Any bug fixes are picked up as part of normal Kubernetes Master upgrades. | May need to periodically pickup bug fixes from upstream and rebuild and update the Aggregated APIserver. |
| No need to handle multiple versions of your API. For example: when you control the client for this resource, you can upgrade it in sync with the API. | You need to handle multiple versions of your API, for example: when developing an extension to share with the world. |
@@ -184,7 +179,7 @@ Aggregated APIs offer more advanced API features and customization of other feat
| Custom Storage | If you need storage with a different performance mode (for example, time-series database instead of key-value store) or isolation for security (for example, encryption secrets or different | No | Yes |
| Custom Business Logic | Perform arbitrary checks or actions when creating, reading, updating or deleting an object | Yes, using [Webhooks](/docs/reference/access-authn-authz/extensible-admission-controllers/#admission-webhooks). | Yes |
| Scale Subresource | Allows systems like HorizontalPodAutoscaler and PodDisruptionBudget interact with your new resource | [Yes](/docs/tasks/access-kubernetes-api/extend-api-custom-resource-definitions/#scale-subresource) | Yes |
| Status Subresource | Allows fine-grained access control where user writes the spec section and the controller writes the status section. Allows incrementing object Generation on custom resource data mutation (requires separate spec and status sections in the resource) | [Yes](/docs/tasks/access-kubernetes-api/extend-api-custom-resource-definitions/#status-subresource) | Yes |
| Status Subresource | <ul><li>Finer-grained access control: user writes spec section, controller writes status section.</li><li>Allows incrementing object Generation on custom resource data mutation (requires separate spec and status sections in the resource)</li></ul> | [Yes](/docs/tasks/access-kubernetes-api/extend-api-custom-resource-definitions/#status-subresource) | Yes |
| Other Subresources | Add operations other than CRUD, such as "logs" or "exec". | No | Yes |
| strategic-merge-patch | The new endpoints support PATCH with `Content-Type: application/strategic-merge-patch+json`. Useful for updating objects that may be modified both locally, and by the server. For more information, see ["Update API Objects in Place Using kubectl patch"](/docs/tasks/run-application/update-api-object-kubectl-patch/) | No | Yes |
| Protocol Buffers | The new resource supports clients that want to use Protocol Buffers | No | Yes |
@@ -207,7 +202,7 @@ When you create a custom resource, either via a CRDs or an AA, you get many feat
| Finalizers | Block deletion of extension resources until external cleanup happens. |
| Admission Webhooks | Set default values and validate extension resources during any create/update/delete operation. |
| UI/CLI Display | Kubectl, dashboard can display extension resources. |
| Unset versus Empty | Clients can distinguish unset fields from zero-valued fields. |
| Unset vs Empty | Clients can distinguish unset fields from zero-valued fields. |
| Client Libraries Generation | Kubernetes provides generic client libraries, as well as tools to generate type-specific client libraries. |
| Labels and annotations | Common metadata across objects that tools know how to edit for core and custom resources. |
@@ -12,7 +12,7 @@ weight: 10
{{% capture overview %}}
{{< feature-state state="alpha" >}}
{{< caution >}}Alpha features can change rapidly. {{< /caution >}}
{{< warning >}}Alpha features change rapidly. {{< /warning >}}
Network plugins in Kubernetes come in a few flavors:
@@ -154,7 +154,7 @@ most network plugins.
Where needed, you can specify the MTU explicitly with the `network-plugin-mtu` kubelet option. For example,
on AWS the `eth0` MTU is typically 9001, so you might specify `--network-plugin-mtu=9001`. If you're using IPSEC you
might reduce it to allow for encapsulation overhead; for example: `--network-plugin-mtu=8873`.
might reduce it to allow for encapsulation overhead e.g. `--network-plugin-mtu=8873`.
This option is provided to the network-plugin; currently **only kubenet supports `network-plugin-mtu`**.
@@ -1,111 +1,117 @@
---
title: Poseidon-Firmament Scheduler
title: Poseidon-Firmament - An alternate scheduler
content_template: templates/concept
weight: 80
---
{{% capture overview %}}
{{< feature-state for_k8s_version="v1.6" state="alpha" >}}
**Current release of Poseidon-Firmament scheduler is an <code> alpha </code> release.**
The Poseidon-Firmament scheduler is an alternate scheduler that can be deployed alongside the default Kubernetes scheduler.
Poseidon-Firmament scheduler is an alternate scheduler that can be deployed alongside the default Kubernetes scheduler.
{{% /capture %}}
{{% capture body %}}
## Introduction
## Introduction
Poseidon is a service that acts as the integration glue between the [Firmament scheduler](https://github.com/Huawei-PaaS/firmament) and Kubernetes. Poseidon-Firmament augments the current Kubernetes scheduling capabilities. It incorporates novel flow network graph based scheduling capabilities alongside the default Kubernetes scheduler. The Firmament scheduler models workloads and clusters as flow networks and runs min-cost flow optimizations over these networks to make scheduling decisions.
Poseidon is a service that acts as the integration glue for the [Firmament scheduler](https://github.com/Huawei-PaaS/firmament) with Kubernetes. Poseidon-Firmament scheduler augments the current Kubernetes scheduling capabilities. It incorporates novel flow network graph based scheduling capabilities alongside the default Kubernetes Scheduler. Firmament scheduler models workloads and clusters as flow networks and runs min-cost flow optimizations over these networks to make scheduling decisions.
Firmament models the scheduling problem as a constraint-based optimization over a flow network graph. This is achieved by reducing scheduling to a min-cost max-flow optimization problem. The Poseidon-Firmament scheduler dynamically refines the workload placements.
It models the scheduling problem as a constraint-based optimization over a flow network graph. This is achieved by reducing scheduling to a min-cost max-flow optimization problem. The Poseidon-Firmament scheduler dynamically refines the workload placements.
Poseidon-Firmament scheduler runs alongside the default Kubernetes scheduler as an alternate scheduler. You can simultaneously run multiple, different schedulers.
Poseidon-Firmament scheduler runs alongside the default Kubernetes Scheduler as an alternate scheduler, so multiple schedulers run simultaneously.
Flow graph scheduling with the Poseidon-Firmament scheduler provides the following advantages:
## Key Advantages
- Workloads (Pods) are bulk scheduled to enable scheduling at massive scale.
The Poseidon-Firmament scheduler outperforms the Kubernetes default scheduler by a wide margin when it comes to throughput performance for scenarios where compute resource requirements are somewhat uniform across your workload (Deployments, ReplicaSets, Jobs).
- The Poseidon-Firmament's scheduler's end-to-end throughput performance and bind time improves as the number of nodes in a cluster increases. As you scale out, Poseidon-Firmament scheduler is able to amortize more and more work across workloads.
- Scheduling in Poseidon-Firmament is dynamic; it keeps cluster resources in a global optimal state during every scheduling run.
- The Poseidon-Firmament scheduler supports scheduling complex rule constraints.
### Flow graph scheduling based Poseidon-Firmament scheduler provides the following key advantages:
- Workloads (pods) are bulk scheduled to enable scheduling at massive scale..
- Based on the extensive performance test results, Poseidon-Firmament scales much better than the Kubernetes default scheduler as the number of nodes increase in a cluster. This is due to the fact that Poseidon-Firmament is able to amortize more and more work across workloads.
- Poseidon-Firmament Scheduler outperforms the Kubernetes default scheduler by a wide margin when it comes to throughput performance numbers for scenarios where compute resource requirements are somewhat uniform across jobs (Replicasets/Deployments/Jobs). Poseidon-Firmament scheduler end-to-end throughput performance numbers, including bind time, consistently get better as the number of nodes in a cluster increase. For example, for a 2,700 node cluster (shown in the graphs [here](https://github.com/kubernetes-sigs/poseidon/blob/master/docs/benchmark/README.md)), Poseidon-Firmament scheduler achieves a 7X or greater end-to-end throughput than the Kubernetes default scheduler, which includes bind time.
## How the Poseidon-Firmament scheduler works
- Availability of complex rule constraints.
- Scheduling in Poseidon-Firmament is dynamic; it keeps cluster resources in a global optimal state during every scheduling run.
- Highly efficient resource utilizations.
Kubernetes supports [using multiple schedulers](/docs/tasks/administer-cluster/configure-multiple-schedulers/). You can specify, for a particular Pod, that it is scheduled by a custom scheduler (“poseidon” for this case), by setting the `schedulerName` field in the PodSpec at the time of pod creation. The default scheduler will ignore that Pod and allow Poseidon-Firmament scheduler to schedule the Pod on a relevant node.
## Poseidon-Firmament Scheduler - How it works
For example:
As part of the Kubernetes multiple schedulers support, each new pod is typically scheduled by the default scheduler. Kubernetes can be instructed to use another scheduler by specifying the name of another custom scheduler (“poseidon” in our case) in the **schedulerName** field of the PodSpec at the time of pod creation. In this case, the default scheduler will ignore that Pod and allow Poseidon scheduler to schedule the Pod on a relevant node.
```yaml
apiVersion: v1
kind: Pod
...
spec:
schedulerName: poseidon
...
```
schedulerName: poseidon
```
## Batch scheduling
{{< note >}}
For details about the design of this project see the [design document](https://github.com/kubernetes-sigs/poseidon/blob/master/docs/design/README.md).
{{< /note >}}
## Possible Use Case Scenarios - When to use it
As mentioned earlier, Poseidon-Firmament scheduler enables an extremely high throughput scheduling environment at scale due to its bulk scheduling approach versus Kubernetes pod-at-a-time approach. In our extensive tests, we have observed substantial throughput benefits as long as resource requirements (CPU/Memory) for incoming Pods are uniform across jobs (Replicasets/Deployments/Jobs), mainly due to efficient amortization of work across jobs.
Although, Poseidon-Firmament scheduler is capable of scheduling various types of workloads, such as service, batch, etc., the following are a few use cases where it excels the most:
1. For “Big Data/AI” jobs consisting of large number of tasks, throughput benefits are tremendous.
2. Service or batch jobs where workload resource requirements are uniform across jobs (Replicasets/Deployments/Jobs).
1. For “Big Data/AI” jobs consisting of large number of tasks, throughput benefits are tremendous.
2. Service or batch jobs where workload resource requirements are uniform across jobs (Replicasets/Deployments/Jobs).
## Feature state
## Current Project Stage
Poseidon-Firmament is designed to work with Kubernetes release 1.6 and all subsequent releases.
- **Alpha Release - Incubation repo.** at https://github.com/kubernetes-sigs/poseidon.
- Currently, Poseidon-Firmament scheduler **does not provide support for high availability**, our implementation assumes that the scheduler cannot fail. The [design document](https://github.com/kubernetes-sigs/poseidon/blob/master/docs/design/README.md) describes possible ways to enable high availability, but we leave this to future work.
- We are **not aware of any production deployment** of Poseidon-Firmament scheduler at this time.
- Poseidon-Firmament is supported from Kubernetes release 1.6 and works with all subsequent releases.
- Release process for Poseidon and Firmament repos are in lock step. The current Poseidon release can be found [here](https://github.com/kubernetes-sigs/poseidon/releases) and the corresponding Firmament release can be found [here](https://github.com/Huawei-PaaS/firmament/releases).
{{< caution >}}
Poseidon-Firmament scheduler does not provide support for high availability; its implementation assumes that the scheduler cannot fail.
{{< /caution >}}
## Features Comparison Matrix
## Feature comparison {#feature-comparison-matrix}
{{< table caption="Feature comparison of Kubernetes and Poseidon-Firmament schedulers." >}}
|Feature|Kubernetes Default Scheduler|Poseidon-Firmament Scheduler|Notes|
|--- |--- |--- |--- |
|Node Affinity/Anti-Affinity|Y|Y||
|Pod Affinity/Anti-Affinity - including support for pod anti-affinity symmetry|Y|Y|The default scheduler outperforms the Poseidon-Firmament scheduler pod affinity/anti-affinity functionality.|
|Pod Affinity/Anti-Affinity - including support for pod anti-affinity symmetry|Y|Y|Currently, the default scheduler outperforms the Poseidon-Firmament scheduler pod affinity/anti-affinity functionality. We are working towards resolving this.|
|Taints & Tolerations|Y|Y||
|Baseline Scheduling capability in accordance to available compute resources (CPU & Memory) on a node|Y|Y†|**†** Not all Predicates & Priorities are supported with Poseidon-Firmament.|
|Extreme Throughput at scale|Y|Y|**†** Bulk scheduling approach scales or increases workload placement. Firmament scheduler offers high throughput when resource requirements (CPU/Memory) for incoming Pods are uniform across ReplicaSets/Deployments/Jobs.|
|Colocation Interference Avoidance|N|N||
|Priority Preemption|Y|N†|**†** Partially exists in Poseidon-Firmament versus extensive support in Kubernetes default scheduler.|
|Inherent Rescheduling|N|Y†|**†** Poseidon-Firmament scheduler supports workload re-scheduling. In each scheduling run, Poseidon-Firmament considers all Pods, including running Pods, and as a result can migrate or evict Pods a globally optimal scheduling environment.|
|Baseline Scheduling capability in accordance to available compute resources (CPU & Memory) on a node|Y|Y**|Not all Predicates & Priorities are supported at this time.|
|Extreme Throughput at scale|Y**|Y|Bulk scheduling approach scales or increases workload placement. Substantial throughput benefits using Firmament scheduler as long as resource requirements (CPU/Memory) for incoming Pods is uniform across Replicasets/Deployments/Jobs. This is mainly due to efficient amortization of work across Replicasets/Deployments/Jobs . 1) For “Big Data/AI” jobs consisting of large no. of tasks, throughput benefits are tremendous. 2) Substantial throughput benefits also for service or batch job scenarios where workload resource requirements are uniform across Replicasets/Deployments/Jobs.|
|Optimal Scheduling|Pod-by-Pod scheduler, processes one pod at a time (may result into sub-optimal scheduling)|Bulk Scheduling (Optimal scheduling)|Pod-by-Pod Kubernetes default scheduler may assign tasks to a sub-optimal machine. By contrast, Firmament considers all unscheduled tasks at the same time together with their soft and hard constraints.|
|Colocation Interference Avoidance|N|N**|Planned in Poseidon-Firmament.|
|Priority Pre-emption|Y|N**|Partially exists in Poseidon-Firmament versus extensive support in Kubernetes default scheduler.|
|Inherent Re-Scheduling|N|Y**|Poseidon-Firmament scheduler supports workload re-scheduling. In each scheduling run it considers all the pods, including running pods, and as a result can migrate or evict pods a globally optimal scheduling environment.|
|Gang Scheduling|N|Y||
|Support for Pre-bound Persistence Volume Scheduling|Y|Y||
|Support for Local Volume & Dynamic Persistence Volume Binding Scheduling|Y|N||
|High Availability|Y|N||
|Real-time metrics based scheduling|N|Y†|**†** Partially supported in Poseidon-Firmament using Heapster (now deprecated) for placing Pods using actual cluster utilization statistics rather than reservations.|
|Support for Local Volume & Dynamic Persistence Volume Binding Scheduling|Y|N**|Planned.|
|High Availability|Y|N**|Planned.|
|Real-time metrics based scheduling|N|Y**|Initially supported using Heapster (now deprecated) for placing pods using actual cluster utilization statistics rather than reservations. Plans to switch over to "metric server".|
|Support for Max-Pod per node|Y|Y|Poseidon-Firmament scheduler seamlessly co-exists with Kubernetes default scheduler.|
|Support for Ephemeral Storage, in addition to CPU/Memory|Y|Y||
{{< /table >}}
## Installation
The [Poseidon-Firmament installation guide](https://github.com/kubernetes-sigs/poseidon/blob/master/docs/install/README.md#Installation) explains how to deploy Poseidon-Firmament to your cluster.
## Installation
## Performance comparison
For in-cluster installation of Poseidon, please start at the [Installation instructions](https://github.com/kubernetes-sigs/poseidon/blob/master/docs/install/README.md).
## Development
For developers, please refer to the [Developer Setup instructions](https://github.com/kubernetes-sigs/poseidon/blob/master/docs/devel/README.md).
## Latest Throughput Performance Testing Results
Pod-by-pod schedulers, such as the Kubernetes default scheduler, typically process one pod at a time. These schedulers have the following crucial drawbacks:
1. The scheduler commits to a pod placement early and restricts the choices for other pods that wait to be placed.
2. There is limited opportunities for amortizing work across pods because they are considered for placement individually.
These downsides of pod-by-pod schedulers are addressed by batching or bulk scheduling in Poseidon-Firmament scheduler. Processing several pods in a batch allows the scheduler to jointly consider their placement, and thus to find the best trade-off for the whole batch instead of one pod. At the same time it amortizes work across pods resulting in much higher throughput.
{{< note >}}
Please refer to the [latest benchmark results](https://github.com/kubernetes-sigs/poseidon/blob/master/docs/benchmark/README.md) for detailed throughput performance comparison test results between Poseidon-Firmament scheduler and the Kubernetes default scheduler.
{{< /note >}}
Pod-by-pod schedulers, such as the Kubernetes default scheduler, process Pods in small batches (typically one at a time). These schedulers have the following crucial drawbacks:
1. The scheduler commits to a pod placement early and restricts the choices for other pods that wait to be placed.
2. There is limited opportunities for amortizing work across pods because they are considered for placement individually.
These downsides of pod-by-pod schedulers are addressed by batching or bulk scheduling in Poseidon-Firmament scheduler. Processing several pods in a batch allows the scheduler to jointly consider their placement, and thus to find the best trade-off for the whole batch instead of one pod. At the same time it amortizes work across pods resulting in much higher throughput.
{{% /capture %}}
{{% capture whatsnext %}}
* See [Poseidon-Firmament](https://github.com/kubernetes-sigs/poseidon#readme) on GitHub for more information.
* See the [design document](https://github.com/kubernetes-sigs/poseidon/blob/master/docs/design/README.md) for Poseidon.
* Read [Firmament: Fast, Centralized Cluster Scheduling at Scale](https://www.usenix.org/system/files/conference/osdi16/osdi16-gog.pdf), the academic paper on the Firmament scheduling design.
* If you'd like to contribute to Poseidon-Firmament, refer to the [developer setup instructions](https://github.com/kubernetes-sigs/poseidon/blob/master/docs/devel/README.md).
{{% /capture %}}
@@ -11,7 +11,7 @@ card:
{{% capture overview %}}
When you deploy Kubernetes, you get a cluster.
{{< glossary_definition term_id="cluster" length="all" prepend="A Kubernetes cluster consists of">}}
{{< glossary_definition term_id="cluster" length="all" prepend="A cluster is">}}
This document outlines the various components you need to have
a complete and working Kubernetes cluster.
@@ -23,12 +23,13 @@ Here's the diagram of a Kubernetes cluster with all the components tied together
{{% /capture %}}
{{% capture body %}}
## Control Plane Components
## Master Components
The Control Plane's components make global decisions about the cluster (for example, scheduling), as well as detecting and responding to cluster events (for example, starting up a new {{< glossary_tooltip text="pod" term_id="pod">}} when a deployment's `replicas` field is unsatisfied).
Master components provide the cluster's control plane. Master components make global decisions about the
cluster (for example, scheduling), and they detect and respond to cluster events (for example, starting up a new {{< glossary_tooltip text="pod" term_id="pod">}} when a deployment's `replicas` field is unsatisfied).
Control Plane components can be run on any machine in the cluster. However,
for simplicity, set up scripts typically start all Control Plane components on
Master components can be run on any machine in the cluster. However,
for simplicity, set up scripts typically start all master components on
the same machine, and do not run user containers on this machine. See
[Building High-Availability Clusters](/docs/admin/high-availability/) for an example multi-master-VM setup.
@@ -61,7 +61,7 @@ Kubernetes implements an alternative Protobuf based serialization format for the
Prior to 1.14, the Kubernetes apiserver also exposes an API that can be used to retrieve
the [Swagger v1.2](http://swagger.io/) Kubernetes API spec at `/swaggerapi`.
This endpoint is deprecated, and was removed in Kubernetes 1.14.
This endpoint is deprecated, and will be removed in Kubernetes 1.14.
## API versioning
@@ -121,22 +121,21 @@ There are two supported paths to extending the API with [custom resources](/docs
to make it seamless for clients.
## Enabling or disabling API groups
## Enabling API groups
Certain resources and API groups are enabled by default. They can be enabled or disabled by setting `--runtime-config`
on apiserver. `--runtime-config` accepts comma separated values. For example: to disable batch/v1, set
on apiserver. `--runtime-config` accepts comma separated values. For ex: to disable batch/v1, set
`--runtime-config=batch/v1=false`, to enable batch/v2alpha1, set `--runtime-config=batch/v2alpha1`.
The flag accepts comma separated set of key=value pairs describing runtime configuration of the apiserver.
{{< note >}}Enabling or disabling groups or resources requires restarting apiserver and controller-manager
to pick up the `--runtime-config` changes.{{< /note >}}
IMPORTANT: Enabling or disabling groups or resources requires restarting apiserver and controller-manager
to pick up the `--runtime-config` changes.
## Enabling specific resources in the extensions/v1beta1 group
## Enabling resources in the groups
DaemonSets, Deployments, StatefulSet, NetworkPolicies, PodSecurityPolicies and ReplicaSets in the `extensions/v1beta1` API group are disabled by default.
For example: to enable deployments and daemonsets, set
`--runtime-config=extensions/v1beta1/deployments=true,extensions/v1beta1/daemonsets=true`.
{{< note >}}Individual resource enablement/disablement is only supported in the `extensions/v1beta1` API group for legacy reasons.{{< /note >}}
DaemonSets, Deployments, HorizontalPodAutoscalers, Ingresses, Jobs and ReplicaSets are enabled by default.
Other extensions resources can be enabled by setting `--runtime-config` on
apiserver. `--runtime-config` accepts comma separated values. For example: to disable deployments and ingress, set
`--runtime-config=extensions/v1beta1/deployments=false,extensions/v1beta1/ingresses=false`
{{% /capture %}}
@@ -3,8 +3,6 @@ reviewers:
- bgrant0607
- mikedanese
title: What is Kubernetes
description: >
Kubernetes is a portable, extensible, open-source platform for managing containerized workloads and services, that facilitates both declarative configuration and automation. It has a large, rapidly growing ecosystem. Kubernetes services, support, and tools are widely available.
content_template: templates/concept
weight: 10
card:
@@ -19,10 +17,9 @@ This page is an overview of Kubernetes.
{{% capture body %}}
Kubernetes is a portable, extensible, open-source platform for managing containerized workloads and services, that facilitates both declarative configuration and automation. It has a large, rapidly growing ecosystem. Kubernetes services, support, and tools are widely available.
The name Kubernetes originates from Greek, meaning helmsman or pilot. Google open-sourced the Kubernetes project in 2014. Kubernetes combines [over 15 years of Google's experience](/blog/2015/04/borg-predecessor-to-kubernetes/) running production workloads at scale with best-of-breed ideas and practices from the community.
The name Kubernetes originates from Greek, meaning helmsman or pilot. Google open-sourced the Kubernetes project in 2014. Kubernetes builds upon a [decade and a half of experience that Google has with running production workloads at scale](https://ai.google/research/pubs/pub43438), combined with best-of-breed ideas and practices from the community.
## Going back in time
Let's take a look at why Kubernetes is so useful by going back in time.
![Deployment evolution](/images/docs/Container_Evolution.svg)
@@ -45,13 +42,13 @@ Containers have become popular because they provide extra benefits, such as:
* Dev and Ops separation of concerns: create application container images at build/release time rather than deployment time, thereby decoupling applications from infrastructure.
* Observability not only surfaces OS-level information and metrics, but also application health and other signals.
* Environmental consistency across development, testing, and production: Runs the same on a laptop as it does in the cloud.
* Cloud and OS distribution portability: Runs on Ubuntu, RHEL, CoreOS, on-premises, on major public clouds, and anywhere else.
* Cloud and OS distribution portability: Runs on Ubuntu, RHEL, CoreOS, on-prem, Google Kubernetes Engine, and anywhere else.
* Application-centric management: Raises the level of abstraction from running an OS on virtual hardware to running an application on an OS using logical resources.
* Loosely coupled, distributed, elastic, liberated micro-services: applications are broken into smaller, independent pieces and can be deployed and managed dynamically not a monolithic stack running on one big single-purpose machine.
* Resource isolation: predictable application performance.
* Resource utilization: high efficiency and density.
## Why you need Kubernetes and what it can do {#why-you-need-kubernetes-and-what-can-it-do}
## Why you need Kubernetes and what can it do
Containers are a good way to bundle and run your applications. In a production environment, you need to manage the containers that run the applications and ensure that there is no downtime. For example, if a container goes down, another container needs to start. Wouldn't it be easier if this behavior was handled by a system?
@@ -82,7 +82,7 @@ metadata:
spec:
containers:
- name: nginx
image: nginx:1.14.2
image: nginx:1.7.9
ports:
- containerPort: 80
@@ -3,7 +3,7 @@ title: Field Selectors
weight: 60
---
_Field selectors_ let you [select Kubernetes resources](/docs/concepts/overview/working-with-objects/kubernetes-objects) based on the value of one or more resource fields. Here are some examples of field selector queries:
_Field selectors_ let you [select Kubernetes resources](/docs/concepts/overview/working-with-objects/kubernetes-objects) based on the value of one or more resource fields. Here are some example field selector queries:
* `metadata.name=my-service`
* `metadata.namespace!=default`
@@ -12,9 +12,9 @@ This page explains how Kubernetes objects are represented in the Kubernetes API,
{{% /capture %}}
{{% capture body %}}
## Understanding Kubernetes objects {#kubernetes-objects}
## Understanding Kubernetes Objects
*Kubernetes objects* are persistent entities in the Kubernetes system. Kubernetes uses these entities to represent the state of your cluster. Specifically, they can describe:
*Kubernetes Objects* are persistent entities in the Kubernetes system. Kubernetes uses these entities to represent the state of your cluster. Specifically, they can describe:
* What containerized applications are running (and on which nodes)
* The resources available to those applications
@@ -26,31 +26,14 @@ To work with Kubernetes objects--whether to create, modify, or delete them--you'
### Object Spec and Status
Almost every Kubernetes object includes two nested object fields that govern
the object's configuration: the object *`spec`* and the object *`status`*.
For objects that have a `spec`, you have to set this when you create the object,
providing a description of the characteristics you want the resource to have:
its _desired state_.
Every Kubernetes object includes two nested object fields that govern the object's configuration: the object *spec* and the object *status*. The *spec*, which you must provide, describes your desired state for the object--the characteristics that you want the object to have. The *status* describes the *actual state* of the object, and is supplied and updated by the Kubernetes system. At any given time, the Kubernetes Control Plane actively manages an object's actual state to match the desired state you supplied.
The `status` describes the _current state_ of the object, supplied and updated
by the Kubernetes and its components. The Kubernetes
{{< glossary_tooltip text="control plane" term_id="control-plane" >}} continually
and actively manages every object's actual state to match the desired state you
supplied.
For example: in Kubernetes, a Deployment is an object that can represent an
application running on your cluster. When you create the Deployment, you
might set the Deployment `spec` to specify that you want three replicas of
the application to be running. The Kubernetes system reads the Deployment
spec and starts three instances of your desired application--updating
the status to match your spec. If any of those instances should fail
(a status change), the Kubernetes system responds to the difference
between spec and status by making a correction--in this case, starting
a replacement instance.
For example, a Kubernetes Deployment is an object that can represent an application running on your cluster. When you create the Deployment, you might set the Deployment spec to specify that you want three replicas of the application to be running. The Kubernetes system reads the Deployment spec and starts three instances of your desired application--updating the status to match your spec. If any of those instances should fail (a status change), the Kubernetes system responds to the difference between spec and status by making a correction--in this case, starting a replacement instance.
For more information on the object spec, status, and metadata, see the [Kubernetes API Conventions](https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md).
### Describing a Kubernetes object
### Describing a Kubernetes Object
When you create an object in Kubernetes, you must provide the object spec that describes its desired state, as well as some basic information about the object (such as a name). When you use the Kubernetes API to create the object (either directly or via `kubectl`), that API request must include that information as JSON in the request body. **Most often, you provide the information to `kubectl` in a .yaml file.** `kubectl` converts the information to JSON when making the API request.
@@ -68,7 +51,7 @@ kubectl apply -f https://k8s.io/examples/application/deployment.yaml --record
The output is similar to this:
```
```shell
deployment.apps/nginx-deployment created
```
@@ -82,15 +65,14 @@ In the `.yaml` file for the Kubernetes object you want to create, you'll need to
* `spec` - What state you desire for the object
The precise format of the object `spec` is different for every Kubernetes object, and contains nested fields specific to that object. The [Kubernetes API Reference](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/) can help you find the spec format for all of the objects you can create using Kubernetes.
For example, the `spec` format for a Pod can be found in
[PodSpec v1 core](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#podspec-v1-core),
and the `spec` format for a Deployment can be found in
[DeploymentSpec v1 apps](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#deploymentspec-v1-apps).
For example, the `spec` format for a `Pod` can be found
[here](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#podspec-v1-core),
and the `spec` format for a `Deployment` can be found
[here](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#deploymentspec-v1-apps).
{{% /capture %}}
{{% capture whatsnext %}}
* [Kubernetes API overview](/docs/reference/using-api/api-overview/) explains some more API concepts
* Learn about the most important basic Kubernetes objects, such as [Pod](/docs/concepts/workloads/pods/pod-overview/).
* Learn about [controllers](/docs/concepts/architecture/controller/) in Kubernetes
{{% /capture %}}
@@ -69,10 +69,10 @@ metadata:
spec:
containers:
- name: nginx
image: nginx:1.14.2
image: nginx:1.7.9
ports:
- containerPort: 80
```
## Label selectors
@@ -92,7 +92,7 @@ them.
For some API types, such as ReplicaSets, the label selectors of two instances must not overlap within a namespace, or the controller can see that as conflicting instructions and fail to determine how many replicas should be present.
{{< /note >}}
{{< caution >}}
{{< caution >}}
For both equality-based and set-based conditions there is no logical _OR_ (`||`) operator. Ensure your filter statements are structured accordingly.
{{< /caution >}}
@@ -210,7 +210,7 @@ this selector (respectively in `json` or `yaml` format) is equivalent to `compon
#### Resources that support set-based requirements
Newer resources, such as [`Job`](/docs/concepts/workloads/controllers/jobs-run-to-completion/), [`Deployment`](/docs/concepts/workloads/controllers/deployment/), [`ReplicaSet`](/docs/concepts/workloads/controllers/replicaset/), and [`DaemonSet`](/docs/concepts/workloads/controllers/daemonset/), support _set-based_ requirements as well.
Newer resources, such as [`Job`](/docs/concepts/jobs/run-to-completion-finite-workloads/), [`Deployment`](/docs/concepts/workloads/controllers/deployment/), [`Replica Set`](/docs/concepts/workloads/controllers/replicaset/), and [`Daemon Set`](/docs/concepts/workloads/controllers/daemonset/), support _set-based_ requirements as well.
```yaml
selector:
@@ -2,7 +2,7 @@
reviewers:
- mikedanese
- thockin
title: Object Names and IDs
title: Names
content_template: templates/concept
weight: 20
---
@@ -18,41 +18,14 @@ For non-unique user-provided attributes, Kubernetes provides [labels](/docs/conc
{{% /capture %}}
{{% capture body %}}
## Names
{{< glossary_definition term_id="name" length="all" >}}
Below are three types of commonly used name constraints for resources.
### DNS Subdomain Names
Most resource types require a name that can be used as a DNS subdomain name
as defined in [RFC 1123](https://tools.ietf.org/html/rfc1123).
This means the name must:
- contain no more than 253 characters
- contain only lowercase alphanumeric characters, '-' or '.'
- start with an alphanumeric character
- end with an alphanumeric character
### DNS Label Names
Some resource types require their names to follow the DNS
label standard as defined in [RFC 1123](https://tools.ietf.org/html/rfc1123).
This means the name must:
- contain at most 63 characters
- contain only lowercase alphanumeric characters or '-'
- start with an alphanumeric character
- end with an alphanumeric character
### Path Segment Names
Some resource types require their names to be able to be safely encoded as a
path segment. In other words, the name may not be "." or ".." and the name may
not contain "/" or "%".
Kubernetes resources can have names up to 253 characters long. The characters allowed in names are: digits (0-9), lower case letters (a-z), `-`, and `.`.
Heres an example manifest for a Pod named `nginx-demo`.
@@ -64,12 +37,11 @@ metadata:
spec:
containers:
- name: nginx
image: nginx:1.14.2
image: nginx:1.7.9
ports:
- containerPort: 80
```
{{< note >}}
Some resource types have additional restrictions on their names.
{{< /note >}}
+59 -62
View File
@@ -9,58 +9,56 @@ weight: 10
{{% capture overview %}}
By default, containers run with unbounded [compute resources](/docs/user-guide/compute-resources) on a Kubernetes cluster.
With resource quotas, cluster administrators can restrict resource consumption and creation on a namespace basis.
Within a namespace, a Pod or Container can consume as much CPU and memory as defined by the namespace's resource quota. There is a concern that one Pod or Container could monopolize all available resources. A LimitRange is a policy to constrain resource allocations (to Pods or Containers) in a namespace.
With Resource quotas, cluster administrators can restrict the resource consumption and creation on a namespace basis.
Within a namespace, a Pod or Container can consume as much CPU and memory as defined by the namespace's resource quota. There is a concern that one Pod or Container could monopolize all of the resources. Limit Range is a policy to constrain resource by Pod or Container in a namespace.
{{% /capture %}}
{{% capture body %}}
A _LimitRange_ provides constraints that can:
A limit range, defined by a `LimitRange` object, provides constraints that can:
- Enforce minimum and maximum compute resources usage per Pod or Container in a namespace.
- Enforce minimum and maximum storage request per PersistentVolumeClaim in a namespace.
- Enforce a ratio between request and limit for a resource in a namespace.
- Set default request/limit for compute resources in a namespace and automatically inject them to Containers at runtime.
## Enabling LimitRange
## Enabling Limit Range
LimitRange support is enabled by default for many Kubernetes distributions. It is
Limit Range support is enabled by default for many Kubernetes distributions. It is
enabled when the apiserver `--enable-admission-plugins=` flag has `LimitRanger` admission controller as
one of its arguments.
A LimitRange is enforced in a particular namespace when there is a
LimitRange object in that namespace.
A limit range is enforced in a particular namespace when there is a
`LimitRange` object in that namespace.
The name of a LimitRange object must be a valid
[DNS subdomain name](/docs/concepts/overview/working-with-objects/names#dns-subdomain-names).
### Overview of Limit Range
### Overview of Limit Range:
- The administrator creates one `LimitRange` in one namespace.
- Users create resources like Pods, Containers, and PersistentVolumeClaims in the namespace.
- The `LimitRanger` admission controller enforces defaults and limits for all Pods and Containers that do not set compute resource requirements and tracks usage to ensure it does not exceed resource minimum, maximum and ratio defined in any LimitRange present in the namespace.
- If creating or updating a resource (Pod, Container, PersistentVolumeClaim) that violates a LimitRange constraint, the request to the API server will fail with an HTTP status code `403 FORBIDDEN` and a message explaining the constraint that have been violated.
- If a LimitRange is activated in a namespace for compute resources like `cpu` and `memory`, users must specify
requests or limits for those values. Otherwise, the system may reject Pod creation.
- LimitRange validations occurs only at Pod Admission stage, not on Running Pods.
- The `LimitRanger` admission controller enforces defaults limits for all Pods and Container that do not set compute resource requirements and tracks usage to ensure it does not exceed resource minimum , maximum and ratio defined in any `LimitRange` present in the namespace.
- If creating or updating a resource (Pod, Container, PersistentVolumeClaim) violates a limit range constraint, the request to the API server will fail with HTTP status code `403 FORBIDDEN` and a message explaining the constraint that would have been violated.
- If limit range is activated in a namespace for compute resources like `cpu` and `memory`, users must specify
requests or limits for those values; otherwise, the system may reject pod creation.
- LimitRange validations occurs only at Pod Admission stage, not on Running pods.
Examples of policies that could be created using limit range are:
- In a 2 node cluster with a capacity of 8 GiB RAM and 16 cores, constrain Pods in a namespace to request 100m of CPU with a max limit of 500m for CPU and request 200Mi for Memory with a max limit of 600Mi for Memory.
- Define default CPU limit and request to 150m and memory default request to 300Mi for Containers started with no cpu and memory requests in their specs.
- In a 2 node cluster with a capacity of 8 GiB RAM, and 16 cores, constrain Pods in a namespace to request 100m and not exceeds 500m for CPU , request 200Mi and not exceed 600Mi
- Define default CPU limits and request to 150m and Memory default request to 300Mi for containers started with no cpu and memory requests in their spec.
In the case where the total limits of the namespace is less than the sum of the limits of the Pods/Containers,
there may be contention for resources. In this case, the Containers or Pods will not be created.
there may be contention for resources; The Containers or Pods will not be created.
Neither contention nor changes to a LimitRange will affect already created resources.
Neither contention nor changes to limitrange will affect already created resources.
## Limiting Container compute resources
The following section discusses the creation of a LimitRange acting at Container Level.
A Pod with 04 Containers is first created. Each Container within the Pod has a specific `spec.resource` configuration.
Each Container within the Pod is handled differently by the `LimitRanger` admission controller.
A Pod with 04 containers is first created; each container within the Pod has a specific `spec.resource` configuration
each container within the pod is handled differently by the LimitRanger admission controller.
Create a namespace `limitrange-demo` using the following kubectl command:
@@ -77,7 +75,7 @@ kubectl config set-context --current --namespace=limitrange-demo
Here is the configuration file for a LimitRange object:
{{< codenew file="admin/resource/limit-mem-cpu-container.yaml" >}}
This object defines minimum and maximum CPU/Memory limits, default CPU/Memory requests, and default limits for CPU/Memory resources to be apply to containers.
This object defines minimum and maximum Memory/CPU limits, default cpu/Memory requests and default limits for CPU/Memory resources to be apply to containers.
Create the `limit-mem-cpu-per-container` LimitRange in the `limitrange-demo` namespace with the following kubectl command:
@@ -96,7 +94,7 @@ Container cpu 100m 800m 110m 700m -
Container memory 99Mi 1Gi 111Mi 900Mi -
```
Here is the configuration file for a Pod with 04 Containers to demonstrate LimitRange features:
Here is the configuration file for a Pod with 04 containers to demonstrate LimitRange features :
{{< codenew file="admin/resource/limit-range-pod-1.yaml" >}}
Create the `busybox1` Pod:
@@ -110,7 +108,7 @@ kubectl apply -f https://k8s.io/examples/admin/resource/limit-range-pod-1.yaml -
View the `busybox-cnt01` resource configuration:
```shell
kubectl get po/busybox1 -n limitrange-demo -o json | jq ".spec.containers[0].resources"
kubectl get po/busybox1 -n limitrange-demo -o json | jq ".spec.containers[0].resources"
```
```json
@@ -127,9 +125,9 @@ kubectl get po/busybox1 -n limitrange-demo -o json | jq ".spec.containers[0].res
```
- The `busybox-cnt01` Container inside `busybox` Pod defined `requests.cpu=100m` and `requests.memory=100Mi`.
- `100m <= 500m <= 800m` , The Container cpu limit (500m) falls inside the authorized CPU LimitRange.
- `99Mi <= 200Mi <= 1Gi` , The Container memory limit (200Mi) falls inside the authorized Memory LimitRange.
- No request/limits ratio validation for CPU/Memory, so the Container is valid and created.
- `100m <= 500m <= 800m` , The container cpu limit (500m) falls inside the authorized CPU limit range.
- `99Mi <= 200Mi <= 1Gi` , The container memory limit (200Mi) falls inside the authorized Memory limit range.
- No request/limits ratio validation for CPU/Memory , thus the container is valid and created.
### Container spec with a valid CPU/Memory requests but no limits
@@ -137,7 +135,7 @@ kubectl get po/busybox1 -n limitrange-demo -o json | jq ".spec.containers[0].res
View the `busybox-cnt02` resource configuration
```shell
kubectl get po/busybox1 -n limitrange-demo -o json | jq ".spec.containers[1].resources"
kubectl get po/busybox1 -n limitrange-demo -o json | jq ".spec.containers[1].resources"
```
```json
@@ -153,18 +151,17 @@ kubectl get po/busybox1 -n limitrange-demo -o json | jq ".spec.containers[1].res
}
```
- The `busybox-cnt02` Container inside `busybox1` Pod defined `requests.cpu=100m` and `requests.memory=100Mi` but not limits for cpu and memory.
- The Container does not have a limits section. The default limits defined in the `limit-mem-cpu-per-container` LimitRange object are injected in to this Container: `limits.cpu=700mi` and `limits.memory=900Mi`.
- `100m <= 700m <= 800m` , The Container cpu limit (700m) falls inside the authorized CPU limit range.
- `99Mi <= 900Mi <= 1Gi` , The Container memory limit (900Mi) falls inside the authorized Memory limit range.
- No request/limits ratio set, so the Container is valid and created.
- The container do not have a limits section, the default limits defined in the limit-mem-cpu-per-container LimitRange object are injected to this container `limits.cpu=700mi` and `limits.memory=900Mi`.
- `100m <= 700m <= 800m` , The container cpu limit (700m) falls inside the authorized CPU limit range.
- `99Mi <= 900Mi <= 1Gi` , The container memory limit (900Mi) falls inside the authorized Memory limit range.
- No request/limits ratio set , thus the container is valid and created.
### Container spec with a valid CPU/Memory limits but no requests
View the `busybox-cnt03` resource configuration:
### Container spec with a valid CPU/Memory limits but no requests
View the `busybox-cnt03` resource configuration
```shell
kubectl get po/busybox1 -n limitrange-demo -o json | jq ".spec.containers[2].resources"
kubectl get po/busybox1 -n limitrange-demo -o json | jq ".spec.containers[2].resources"
```
```json
{
@@ -180,17 +177,17 @@ kubectl get po/busybox1 -n limitrange-demo -o json | jq ".spec.containers[2].res
```
- The `busybox-cnt03` Container inside `busybox1` Pod defined `limits.cpu=500m` and `limits.memory=200Mi` but no `requests` for cpu and memory.
- The Container does not define a request section. The default request defined in the limit-mem-cpu-per-container LimitRange is not used to fill its limits section, but the limits defined by the Container are set as requests `limits.cpu=500m` and `limits.memory=200Mi`.
- `100m <= 500m <= 800m` , The Container cpu limit (500m) falls inside the authorized CPU limit range.
- `99Mi <= 200Mi <= 1Gi` , The Container memory limit (200Mi) falls inside the authorized Memory limit range.
- No request/limits ratio set, so the Container is valid and created.
- The container do not define a request section, the defaultRequest defined in the limit-mem-cpu-per-container LimitRange is not used to fill its limits section but the limits defined by the container are set as requests `limits.cpu=500m` and `limits.memory=200Mi`.
- `100m <= 500m <= 800m` , The container cpu limit (500m) falls inside the authorized CPU limit range.
- `99Mi <= 200Mi <= 1Gi` , The container memory limit (200Mi) falls inside the authorized Memory limit range.
- No request/limits ratio set , thus the container is valid and created.
### Container spec with no CPU/Memory requests/limits
View the `busybox-cnt04` resource configuration:
```shell
kubectl get po/busybox1 -n limitrange-demo -o json | jq ".spec.containers[3].resources"
kubectl get po/busybox1 -n limitrange-demo -o json | jq ".spec.containers[3].resources"
```
```json
@@ -207,27 +204,27 @@ kubectl get po/busybox1 -n limitrange-demo -o json | jq ".spec.containers[3].res
```
- The `busybox-cnt04` Container inside `busybox1` define neither `limits` nor `requests`.
- The Container do not define a limit section, the default limit defined in the limit-mem-cpu-per-container LimitRange is used to fill its request
- The container do not define a limit section, the default limit defined in the limit-mem-cpu-per-container LimitRange is used to fill its request
`limits.cpu=700m and` `limits.memory=900Mi` .
- The Container do not define a request section, the defaultRequest defined in the `limit-mem-cpu-per-container` LimitRange is used to fill its request section requests.cpu=110m and requests.memory=111Mi
- `100m <= 700m <= 800m` , The Container cpu limit (700m) falls inside the authorized CPU limit range.
- `99Mi <= 900Mi <= 1Gi` , The Container memory limit (900Mi) falls inside the authorized Memory limit range .
- No request/limits ratio set, so the Container is valid and created.
- The container do not define a request section, the defaultRequest defined in the limit-mem-cpu-per-container LimitRange is used to fill its request section requests.cpu=110m and requests.memory=111Mi
- `100m <= 700m <= 800m` , The container cpu limit (700m) falls inside the authorized CPU limit range.
- `99Mi <= 900Mi <= 1Gi` , The container memory limit (900Mi) falls inside the authorized Memory limitrange .
- No request/limits ratio set , thus the container is valid and created.
All Containers defined in the `busybox` Pod passed LimitRange validations, so this the Pod is valid and created in the namespace.
All containers defined in the `busybox` Pod passed LimitRange validations, this the Pod is valid and create in the namespace.
## Limiting Pod compute resources
The following section discusses how to constrain resources at the Pod level.
The following section discusses how to constrain resources at Pod level.
{{< codenew file="admin/resource/limit-mem-cpu-pod.yaml" >}}
Without having to delete the `busybox1` Pod, create the `limit-mem-cpu-pod` LimitRange in the `limitrange-demo` namespace:
Without having to delete `busybox1` Pod, create the `limit-mem-cpu-pod` LimitRange in the `limitrange-demo` namespace:
```shell
kubectl apply -f https://k8s.io/examples/admin/resource/limit-mem-cpu-pod.yaml -n limitrange-demo
```
The LimitRange is created and limits CPU to 2 Core and Memory to 2Gi per Pod:
The limitrange is created and limits CPU to 2 Core and Memory to 2Gi per Pod:
```shell
limitrange/limit-mem-cpu-per-pod created
@@ -256,26 +253,26 @@ Now create the `busybox2` Pod:
kubectl apply -f https://k8s.io/examples/admin/resource/limit-range-pod-2.yaml -n limitrange-demo
```
The `busybox2` Pod definition is identical to `busybox1`, but an error is reported since the Pod's resources are now limited:
The `busybox2` Pod definition is identical to `busybox1` but an error is reported since Pod's resources are now limited:
```shell
Error from server (Forbidden): error when creating "limit-range-pod-2.yaml": pods "busybox2" is forbidden: [maximum cpu usage per Pod is 2, but limit is 2400m., maximum memory usage per Pod is 2Gi, but limit is 2306867200.]
```
```shell
kubectl get po/busybox1 -n limitrange-demo -o json | jq ".spec.containers[].resources.limits.memory"
kubectl get po/busybox1 -n limitrange-demo -o json | jq ".spec.containers[].resources.limits.memory"
"200Mi"
"900Mi"
"200Mi"
"900Mi"
```
`busybox2` Pod will not be admitted on the cluster since the total memory limit of its Container is greater than the limit defined in the LimitRange.
`busybox2` Pod will not be admitted on the cluster since the total memory limit of its container is greater than the limit defined in the LimitRange.
`busybox1` will not be evicted since it was created and admitted on the cluster before the LimitRange creation.
## Limiting Storage resources
You can enforce minimum and maximum size of [storage resources](/docs/concepts/storage/persistent-volumes/) that can be requested by each PersistentVolumeClaim in a namespace using a LimitRange:
You can enforce minimum and maximum size of [storage resources](/docs/concepts/storage/persistent-volumes/) that can be requested by each PersistentVolumeClaim in a namespace using a LimitRange:
{{< codenew file="admin/resource/storagelimits.yaml" >}}
@@ -308,7 +305,7 @@ PersistentVolumeClaim storage 1Gi 2Gi - - -
{{< codenew file="admin/resource/pvc-limit-lower.yaml" >}}
```shell
kubectl create -f https://k8s.io/examples/admin/resource/pvc-limit-lower.yaml -n limitrange-demo
kubectl create -f https://k8s.io/examples/admin/resource//pvc-limit-lower.yaml -n limitrange-demo
```
While creating a PVC with `requests.storage` lower than the Min value in the LimitRange, an Error thrown by the server:
@@ -331,9 +328,9 @@ Error from server (Forbidden): error when creating "pvc-limit-greater.yaml": per
## Limits/Requests Ratio
If `LimitRangeItem.maxLimitRequestRatio` is specified in the `LimitRangeSpec`, the named resource must have a request and limit that are both non-zero where limit divided by request is less than or equal to the enumerated value.
If `LimitRangeItem.maxLimitRequestRatio` is specified in the `LimitRangeSpec`, the named resource must have a request and limit that are both non-zero where limit divided by request is less than or equal to the enumerated value
The following LimitRange enforces memory limit to be at most twice the amount of the memory request for any Pod in the namespace:
The following `LimitRange` enforces memory limit to be at most twice the amount of the memory request for any pod in the namespace.
{{< codenew file="admin/resource/limit-memory-ratio-pod.yaml" >}}
@@ -344,7 +341,7 @@ kubectl apply -f https://k8s.io/examples/admin/resource/limit-memory-ratio-pod.y
Describe the <limit-memory-ratio-pod> LimitRange with the following kubectl command:
```shell
kubectl describe limitrange/limit-memory-ratio-pod
$ kubectl describe limitrange/limit-memory-ratio-pod
```
```shell
@@ -355,7 +352,7 @@ Type Resource Min Max Default Request Default Limit Max Limit/Reques
Pod memory - - - - 2
```
Create a pod with `requests.memory=100Mi` and `limits.memory=300Mi`:
Let's create a pod with `requests.memory=100Mi` and `limits.memory=300Mi`:
{{< codenew file="admin/resource/limit-range-pod-3.yaml" >}}
@@ -363,9 +360,9 @@ Create a pod with `requests.memory=100Mi` and `limits.memory=300Mi`:
kubectl apply -f https://k8s.io/examples/admin/resource/limit-range-pod-3.yaml
```
The pod creation failed as the ratio here (`3`) is greater than the enforced limit (`2`) in `limit-memory-ratio-pod` LimitRange:
The pod creation failed as the ratio here (`3`) is greater than the enforced limit (`2`) in `limit-memory-ratio-pod` LimitRange
```
```shell
Error from server (Forbidden): error when creating "limit-range-pod-3.yaml": pods "busybox3" is forbidden: memory max limit to request ratio per Pod is 2, but provided ratio is 3.000000.
```
@@ -22,7 +22,7 @@ updates.
## What is a Pod Security Policy?
A _Pod Security Policy_ is a cluster-level resource that controls security
sensitive aspects of the pod specification. The [PodSecurityPolicy](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#podsecuritypolicy-v1beta1-policy) objects
sensitive aspects of the pod specification. The `PodSecurityPolicy` objects
define a set of conditions that a pod must run with in order to be accepted into
the system, as well as defaults for the related fields. They allow an
administrator to control the following:
@@ -197,8 +197,6 @@ alias kubectl-user='kubectl --as=system:serviceaccount:psp-example:fake-user -n
Define the example PodSecurityPolicy object in a file. This is a policy that
simply prevents the creation of privileged pods.
The name of a PodSecurityPolicy object must be a valid
[DNS subdomain name](/docs/concepts/overview/working-with-objects/names#dns-subdomain-names).
{{< codenew file="policy/example-psp.yaml" >}}
@@ -363,7 +361,7 @@ podsecuritypolicy "example" deleted
### Example Policies
This is the least restrictive policy you can create, equivalent to not using the
This is the least restricted policy you can create, equivalent to not using the
pod security policy admission controller:
{{< codenew file="policy/privileged-psp.yaml" >}}
@@ -403,6 +401,8 @@ pods on the same node.
network namespace. Defined as a list of `HostPortRange`, with `min`(inclusive)
and `max`(inclusive). Defaults to no allowed host ports.
**AllowedHostPaths** - See [Volumes and file systems](#volumes-and-file-systems).
### Volumes and file systems
**Volumes** - Provides a whitelist of allowed volume types. The allowable values
@@ -421,10 +421,8 @@ The **recommended minimum set** of allowed volumes for new PSPs are:
- projected
{{< warning >}}
PodSecurityPolicy does not limit the types of `PersistentVolume` objects that
may be referenced by a `PersistentVolumeClaim`, and hostPath type
`PersistentVolumes` do not support read-only access mode. Only trusted users
should be granted permission to create `PersistentVolume` objects.
PodSecurityPolicy does not limit the types of `PersistentVolume` objects that may be referenced by a `PersistentVolumeClaim`.
Only trusted users should be granted permission to create `PersistentVolume` objects.
{{< /warning >}}
**FSGroup** - Controls the supplemental group applied to some volumes.
@@ -630,9 +628,3 @@ Refer to the [Sysctl documentation](
/docs/concepts/cluster-administration/sysctl-cluster/#podsecuritypolicy).
{{% /capture %}}
{{% capture whatsnext %}}
Refer to [Pod Security Policy Reference](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#podsecuritypolicy-v1beta1-policy) for the api details.
{{% /capture %}}
@@ -37,9 +37,6 @@ Resource quotas work like this:
the `LimitRanger` admission controller to force defaults for pods that make no compute resource requirements.
See the [walkthrough](/docs/tasks/administer-cluster/quota-memory-cpu-namespace/) for an example of how to avoid this problem.
The name of a `ResourceQuota` object must be a valid
[DNS subdomain name](/docs/concepts/overview/working-with-objects/names#dns-subdomain-names).
Examples of policies that could be created using namespaces and quotas are:
- In a cluster with a capacity of 32 GiB RAM, and 16 cores, let team A use 20 GiB and 10 cores,
@@ -379,7 +376,7 @@ pods 0 10
* `Exist`
* `DoesNotExist`
## Requests compared to Limits {#requests-vs-limits}
## Requests vs Limits
When allocating compute resources, each container may specify a request and a limit value for either CPU or memory.
The quota can be configured to quota either value.
@@ -35,7 +35,7 @@ kube-scheduler is designed so that, if you want and need to, you can
write your own scheduling component and use that instead.
For every newly created pod or other unscheduled pods, kube-scheduler
selects an optimal node for them to run on. However, every container in
selects a optimal node for them to run on. However, every container in
pods has different requirements for resources and every pod also has
different requirements. Therefore, existing nodes need to be filtered
according to the specific scheduling requirements.
@@ -142,7 +142,7 @@ Area of Concern for Code | Recommendation |
--------------------------------------------- | ------------ |
Access over TLS only | If your code needs to communicate via TCP, ideally it would be performing a TLS handshake with the client ahead of time. With the exception of a few cases, the default behavior should be to encrypt everything in transit. Going one step further, even "behind the firewall" in our VPC's it's still a good idea to encrypt network traffic between services. This can be done through a process known as mutual or [mTLS](https://en.wikipedia.org/wiki/Mutual_authentication) which performs a two sided verification of communication between two certificate holding services. There are numerous tools that can be used to accomplish this in Kubernetes such as [Linkerd](https://linkerd.io/) and [Istio](https://istio.io/). |
Limiting port ranges of communication | This recommendation may be a bit self-explanatory, but wherever possible you should only expose the ports on your service that are absolutely essential for communication or metric gathering. |
3rd Party Dependency Security | Since our applications tend to have dependencies outside of our own codebases, it is a good practice to regularly scan the code's dependencies to ensure that they are still secure with no vulnerabilities currently filed against them. Each language has a tool for performing this check automatically. |
3rd Party Dependency Security | Since our applications tend to have dependencies outside of our own codebases, it is a good practice to ensure that a regular scan of the code's dependencies are still secure with no CVE's currently filed against them. Each language has a tool for performing this check automatically. |
Static Code Analysis | Most languages provide a way for a snippet of code to be analyzed for any potentially unsafe coding practices. Whenever possible you should perform checks using automated tooling that can scan codebases for common security errors. Some of the tools can be found here: https://www.owasp.org/index.php/Source_Code_Analysis_Tools |
Dynamic probing attacks | There are a few automated tools that are able to be run against your service to try some of the well known attacks that commonly befall services. These include SQL injection, CSRF, and XSS. One of the most popular dynamic analysis tools is the OWASP Zed Attack proxy https://www.owasp.org/index.php/OWASP_Zed_Attack_Proxy_Project |
@@ -17,7 +17,7 @@ Now that you have a continuously running, replicated application you can expose
By default, Docker uses host-private networking, so containers can talk to other containers only if they are on the same machine. In order for Docker containers to communicate across nodes, there must be allocated ports on the machines own IP address, which are then forwarded or proxied to the containers. This obviously means that containers must either coordinate which ports they use very carefully or ports must be allocated dynamically.
Coordinating port allocations across multiple developers or teams that provide containers is very difficult to do at scale, and exposes users to cluster-level issues outside of their control. Kubernetes assumes that pods can communicate with other pods, regardless of which host they land on. Kubernetes gives every pod its own cluster-private IP address, so you do not need to explicitly create links between pods or map container ports to host ports. This means that containers within a Pod can all reach each other's ports on localhost, and all pods in a cluster can see each other without NAT. The rest of this document elaborates on how you can run reliable services on such a networking model.
Coordinating ports across multiple developers is very difficult to do at scale and exposes users to cluster-level issues outside of their control. Kubernetes assumes that pods can communicate with other pods, regardless of which host they land on. We give every pod its own cluster-private-IP address so you do not need to explicitly create links between pods or map container ports to host ports. This means that containers within a Pod can all reach each other's ports on localhost, and all pods in a cluster can see each other without NAT. The rest of this document will elaborate on how you can run reliable services on such a networking model.
This guide uses a simple nginx server to demonstrate proof of concept. The same principles are embodied in a more complete [Jenkins CI application](https://kubernetes.io/blog/2015/07/strong-simple-ssl-for-kubernetes).
@@ -422,8 +422,10 @@ LoadBalancer Ingress: a320587ffd19711e5a37606cf4a74574-1142138393.us-east-1.el
{{% capture whatsnext %}}
* Learn more about [Using a Service to Access an Application in a Cluster](/docs/tasks/access-application-cluster/service-access-application-cluster/)
* Learn more about [Connecting a Front End to a Back End Using a Service](/docs/tasks/access-application-cluster/connecting-frontend-backend/)
* Learn more about [Creating an External Load Balancer](/docs/tasks/access-application-cluster/create-external-load-balancer/)
Kubernetes also supports Federated Services, which can span multiple
clusters and cloud providers, to provide increased availability,
better fault tolerance and greater scalability for your services. See
the [Federated Services User Guide](/docs/concepts/cluster-administration/federation-service-discovery/)
for further information.
{{% /capture %}}
@@ -38,16 +38,14 @@ For more up-to-date specification, see
## Services
### A/AAAA records
### A records
"Normal" (not headless) Services are assigned a DNS A or AAAA record,
depending on the IP family of the service, for a name of the form
`my-svc.my-namespace.svc.cluster-domain.example`. This resolves to the cluster IP
"Normal" (not headless) Services are assigned a DNS A record for a name of the
form `my-svc.my-namespace.svc.cluster-domain.example`. This resolves to the cluster IP
of the Service.
"Headless" (without a cluster IP) Services are also assigned a DNS A or AAAA record,
depending on the IP family of the service, for a name of the form
`my-svc.my-namespace.svc.cluster-domain.example`. Unlike normal
"Headless" (without a cluster IP) Services are also assigned a DNS A record for
a name of the form `my-svc.my-namespace.svc.cluster-domain.example`. Unlike normal
Services, this resolves to the set of IPs of the pods selected by the Service.
Clients are expected to consume the set or else use standard round-robin
selection from the set.
@@ -130,22 +128,22 @@ spec:
```
If there exists a headless service in the same namespace as the pod and with
the same name as the subdomain, the cluster's DNS Server also returns an A or AAAA
the same name as the subdomain, the cluster's KubeDNS Server also returns an A
record for the Pod's fully qualified hostname.
For example, given a Pod with the hostname set to "`busybox-1`" and the subdomain set to
"`default-subdomain`", and a headless Service named "`default-subdomain`" in
the same namespace, the pod will see its own FQDN as
"`busybox-1.default-subdomain.my-namespace.svc.cluster-domain.example`". DNS serves an
A or AAAA record at that name, pointing to the Pod's IP. Both pods "`busybox1`" and
"`busybox2`" can have their distinct A or AAAA records.
A record at that name, pointing to the Pod's IP. Both pods "`busybox1`" and
"`busybox2`" can have their distinct A records.
The Endpoints object can specify the `hostname` for any endpoint addresses,
along with its IP.
{{< note >}}
Because A or AAAA records are not created for Pod names, `hostname` is required for the Pod's A or AAAA
Because A records are not created for Pod names, `hostname` is required for the Pod's A
record to be created. A Pod with no `hostname` but with `subdomain` will only create the
A or AAAA record for the headless service (`default-subdomain.my-namespace.svc.cluster-domain.example`),
A record for the headless service (`default-subdomain.my-namespace.svc.cluster-domain.example`),
pointing to the Pod's IP address. Also, Pod needs to become ready in order to have a
record unless `publishNotReadyAddresses=True` is set on the Service.
{{< /note >}}
@@ -31,6 +31,7 @@ Enabling IPv4/IPv6 dual-stack on your Kubernetes cluster provides the following
* Dual-stack Pod networking (a single IPv4 and IPv6 address assignment per Pod)
* IPv4 and IPv6 enabled Services (each Service must be for a single address family)
* Kubenet multi address family support (IPv4 and IPv6)
* Pod off-cluster egress routing (eg. the Internet) via both IPv4 and IPv6 interfaces
## Prerequisites
@@ -39,15 +40,13 @@ The following prerequisites are needed in order to utilize IPv4/IPv6 dual-stack
* Kubernetes 1.16 or later
* Provider support for dual-stack networking (Cloud provider or otherwise must be able to provide Kubernetes nodes with routable IPv4/IPv6 network interfaces)
* A network plugin that supports dual-stack (such as Kubenet or Calico)
* Kubenet network plugin
* Kube-proxy running in mode IPVS
## Enable IPv4/IPv6 dual-stack
To enable IPv4/IPv6 dual-stack, enable the `IPv6DualStack` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/) for the relevant components of your cluster, and set dual-stack cluster network assignments:
* kube-apiserver:
* `--feature-gates="IPv6DualStack=true"`
* kube-controller-manager:
* `--feature-gates="IPv6DualStack=true"`
* `--cluster-cidr=<IPv4 CIDR>,<IPv6 CIDR>` eg. `--cluster-cidr=10.244.0.0/16,fc00::/24`
@@ -57,7 +56,7 @@ To enable IPv4/IPv6 dual-stack, enable the `IPv6DualStack` [feature gate](/docs/
* `--feature-gates="IPv6DualStack=true"`
* kube-proxy:
* `--proxy-mode=ipvs`
* `--cluster-cidr=<IPv4 CIDR>,<IPv6 CIDR>`
* `--cluster-cidrs=<IPv4 CIDR>,<IPv6 CIDR>`
* `--feature-gates="IPv6DualStack=true"`
{{< caution >}}
@@ -1,9 +1,9 @@
---
reviewers:
- freehan
title: EndpointSlices
title: Endpoint Slices
feature:
title: EndpointSlices
title: Endpoint Slices
description: >
Scalable tracking of network endpoints in a Kubernetes cluster.
@@ -16,7 +16,7 @@ weight: 10
{{< feature-state for_k8s_version="v1.17" state="beta" >}}
_EndpointSlices_ provide a simple way to track network endpoints within a
_Endpoint Slices_ provide a simple way to track network endpoints within a
Kubernetes cluster. They offer a more scalable and extensible alternative to
Endpoints.
@@ -24,16 +24,13 @@ Endpoints.
{{% capture body %}}
## EndpointSlice resources {#endpointslice-resource}
## Endpoint Slice resources {#endpointslice-resource}
In Kubernetes, an EndpointSlice contains references to a set of network
endpoints. The EndpointSlice controller automatically creates EndpointSlices
for a Kubernetes Service when a {{< glossary_tooltip text="selector"
term_id="selector" >}} is specified. These EndpointSlices will include
references to any Pods that match the Service selector. EndpointSlices group
network endpoints together by unique Service and Port combinations.
The name of a EndpointSlice object must be a valid
[DNS subdomain name](/docs/concepts/overview/working-with-objects/names#dns-subdomain-names).
endpoints. The EndpointSlice controller automatically creates Endpoint Slices
for a Kubernetes Service when a selector is specified. These Endpoint Slices
will include references to any Pods that match the Service selector. Endpoint
Slices group network endpoints together by unique Service and Port combinations.
As an example, here's a sample EndpointSlice resource for the `example`
Kubernetes Service.
@@ -52,7 +49,7 @@ ports:
port: 80
endpoints:
- addresses:
- "10.1.2.3"
- "10.1.2.3"
conditions:
ready: true
hostname: pod-1
@@ -61,15 +58,15 @@ endpoints:
topology.kubernetes.io/zone: us-west2-a
```
By default, EndpointSlices managed by the EndpointSlice controller will have no
more than 100 endpoints each. Below this scale, EndpointSlices should map 1:1
By default, Endpoint Slices managed by the EndpointSlice controller will have no
more than 100 endpoints each. Below this scale, Endpoint Slices should map 1:1
with Endpoints and Services and have similar performance.
EndpointSlices can act as the source of truth for kube-proxy when it comes to
Endpoint Slices can act as the source of truth for kube-proxy when it comes to
how to route internal traffic. When enabled, they should provide a performance
improvement for services with large numbers of endpoints.
### Address Types
## Address Types
EndpointSlices support three address types:
@@ -77,94 +74,6 @@ EndpointSlices support three address types:
* IPv6
* FQDN (Fully Qualified Domain Name)
### Topology
Each endpoint within an EndpointSlice can contain relevant topology information.
This is used to indicate where an endpoint is, containing information about the
corresponding Node, zone, and region. When the values are available, the
following Topology labels will be set by the EndpointSlice controller:
* `kubernetes.io/hostname` - The name of the Node this endpoint is on.
* `topology.kubernetes.io/zone` - The zone this endpoint is in.
* `topology.kubernetes.io/region` - The region this endpoint is in.
The values of these labels are derived from resources associated with each
endpoint in a slice. The hostname label represents the value of the NodeName
field on the corresponding Pod. The zone and region labels represent the value
of the labels with the same names on the corresponding Node.
### Management
By default, EndpointSlices are created and managed by the EndpointSlice
controller. There are a variety of other use cases for EndpointSlices, such as
service mesh implementations, that could result in other entities or controllers
managing additional sets of EndpointSlices. To ensure that multiple entities can
manage EndpointSlices without interfering with each other, a
`endpointslice.kubernetes.io/managed-by` label is used to indicate the entity
managing an EndpointSlice. The EndpointSlice controller sets
`endpointslice-controller.k8s.io` as the value for this label on all
EndpointSlices it manages. Other entities managing EndpointSlices should also
set a unique value for this label.
### Ownership
In most use cases, EndpointSlices will be owned by the Service that it tracks
endpoints for. This is indicated by an owner reference on each EndpointSlice as
well as a `kubernetes.io/service-name` label that enables simple lookups of all
EndpointSlices belonging to a Service.
## EndpointSlice Controller
The EndpointSlice controller watches Services and Pods to ensure corresponding
EndpointSlices are up to date. The controller will manage EndpointSlices for
every Service with a selector specified. These will represent the IPs of Pods
matching the Service selector.
### Size of EndpointSlices
By default, EndpointSlices are limited to a size of 100 endpoints each. You can
configure this with the `--max-endpoints-per-slice` {{< glossary_tooltip
text="kube-controller-manager" term_id="kube-controller-manager" >}} flag up to
a maximum of 1000.
### Distribution of EndpointSlices
Each EndpointSlice has a set of ports that applies to all endpoints within the
resource. When named ports are used for a Service, Pods may end up with
different target port numbers for the same named port, requiring different
EndpointSlices. This is similar to the logic behind how subsets are grouped
with Endpoints.
The controller tries to fill EndpointSlices as full as possible, but does not
actively rebalance them. The logic of the controller is fairly straightforward:
1. Iterate through existing EndpointSlices, remove endpoints that are no longer
desired and update matching endpoints that have changed.
2. Iterate through EndpointSlices that have been modified in the first step and
fill them up with any new endpoints needed.
3. If there's still new endpoints left to add, try to fit them into a previously
unchanged slice and/or create new ones.
Importantly, the third step prioritizes limiting EndpointSlice updates over a
perfectly full distribution of EndpointSlices. As an example, if there are 10
new endpoints to add and 2 EndpointSlices with room for 5 more endpoints each,
this approach will create a new EndpointSlice instead of filling up the 2
existing EndpointSlices. In other words, a single EndpointSlice creation is
preferrable to multiple EndpointSlice updates.
With kube-proxy running on each Node and watching EndpointSlices, every change
to an EndpointSlice becomes relatively expensive since it will be transmitted to
every Node in the cluster. This approach is intended to limit the number of
changes that need to be sent to every Node, even if it may result with multiple
EndpointSlices that are not full.
In practice, this less than ideal distribution should be rare. Most changes
processed by the EndpointSlice controller will be small enough to fit in an
existing EndpointSlice, and if not, a new EndpointSlice is likely going to be
necessary soon anyway. Rolling updates of Deployments also provide a natural
repacking of EndpointSlices with all pods and their corresponding endpoints
getting replaced.
## Motivation
The Endpoints API has provided a simple and straightforward way of
@@ -177,14 +86,14 @@ Since all network endpoints for a Service were stored in a single Endpoints
resource, those resources could get quite large. That affected the performance
of Kubernetes components (notably the master control plane) and resulted in
significant amounts of network traffic and processing when Endpoints changed.
EndpointSlices help you mitigate those issues as well as provide an extensible
Endpoint Slices help you mitigate those issues as well as provide an extensible
platform for additional features such as topological routing.
{{% /capture %}}
{{% capture whatsnext %}}
* [Enabling EndpointSlices](/docs/tasks/administer-cluster/enabling-endpointslices)
* [Enabling Endpoint Slices](/docs/tasks/administer-cluster/enabling-endpointslices)
* Read [Connecting Applications with Services](/docs/concepts/services-networking/connect-applications-service/)
{{% /capture %}}
@@ -22,7 +22,6 @@ Kubernetes as a project currently supports and maintains [GCE](https://git.k8s.i
## Additional controllers
* [AKS Application Gateway Ingress Controller](https://github.com/Azure/application-gateway-kubernetes-ingress) is an ingress controller that enables ingress to [AKS clusters](https://docs.microsoft.com/azure/aks/kubernetes-walkthrough-portal) using the [Azure Application Gateway](https://docs.microsoft.com/azure/application-gateway/overview).
* [Ambassador](https://www.getambassador.io/) API Gateway is an [Envoy](https://www.envoyproxy.io) based ingress
controller with [community](https://www.getambassador.io/docs) or
[commercial](https://www.getambassador.io/pro/) support from [Datawire](https://www.datawire.io/).

Some files were not shown because too many files have changed in this diff Show More