/docs/reference/setup-tools/kubeadm/kubeadm/ is redirected to
/docs/reference/setup-tools/kubeadm/
This replaces the redirect links of kubeadm with the direct links.
NOTE: The pull request for `en` language has been already merged as https://github.com/kubernetes/website/pull/26919
* manispulador -> manipulador
* dentro do cgroup e namespace do container -> dentro dos cgroups e namespaces
* tcpSocker -> tcpSocket
* conêiner-> contêiner
* Os logs para um manipulador de _hook_ não expostos em eventos de Pod -> Os logs para um manipulador de _hook_ não são expostos em eventos de Pod.
* failedPreStopHook -> FailedPreStopHook
* remove usage of the "certificates" API for cert renewal
"--use-api" option is removed from kubeadm alpha certs renew command since k8s 1.19
* Update kubeadm-certs.md
* Update content/en/docs/tasks/administer-cluster/kubeadm/kubeadm-certs.md
Co-authored-by: Lubomir I. Ivanov <neolit123@gmail.com>
Co-authored-by: Lubomir I. Ivanov <neolit123@gmail.com>
* docs(configure-redis-using-configmap): update for clarity
Signed-off-by: Jai Govindani <jai@honestbank.com>
* fix(configure-redis-using-configmap): incorrect volumeMount index
Signed-off-by: Jai Govindani <jai@honestbank.com>
* fix(configure-redis-using-configmap): show Pod status as Running, separate commands from output
Signed-off-by: Jai Govindani <jai@honestbank.com>
* fix(configure-redis-using-configmap): typo
Signed-off-by: Jai Govindani <jai@honestbank.com>
* fix(configure-redis-using-configmap): configmap name
Signed-off-by: Jai Govindani <jai@honestbank.com>
* Fixed a misconception that I had and I forgot to fix before the merge
* Update content/es/docs/reference/glossary/limitrange.md
Added suggested glossary tooltip : line23.
Thanks
Co-authored-by: Rael Garcia <rael@rael.io>
Co-authored-by: Rael Garcia <rael@rael.io>
* Sync with english version in 'Update Ubuntu/Debian installation instructions to use Signed-By option (#26952)'
Signed-off-by: ydFu <ader.ydfu@gmail.com>
/docs/tasks/tools/install-kubectl/ is redirected to
/docs/tasks/tools/
This commit replace the redirect links for installing kubectl
with direct links.
* Fix: title is put in quotes. Update: Added description, probably deleted by mistake reviewing change suggestions
* Apply suggestions from code review
Realmente no estoy entendiendo por qué el "Commit suggestions" no está realizando el commit al fork. Lo voy a intentar nuevamente, pero esta vez haciendo un review ...
Éste mensaje se corresponde con el Commit suggestion al utilizar el "Add suggestion to batch" no creo que funcione tampoco en cuyo caso voy a intentar hacer (como mencioné en el párrafo anterior) un review para que los cambios persistan.
Co-authored-by: Rael Garcia <rael@rael.io>
Co-authored-by: Rael Garcia <rael@rael.io>
/docs/reference/setup-tools/kubeadm/kubeadm/ is redirected to /docs/reference/setup-tools/kubeadm/
This replaces the redirect links of kubeadm with the direct links.
NOTE: The pull request for `en` language has been already merged as https://github.com/kubernetes/website/pull/26919
* Move pod-overhead.md to schedulling-eviction
* Add pod-overhead.md translation
* Update content/pt/docs/concepts/scheduling-eviction/pod-overhead.md
* Fix name: test-Pod to name: test-pod
* Small fixes
* remove reviewers
* change 'fonte' to 'código fonte'
* Add the suggestion by code review
* Update the references with the original doc
* Update the _index.md
/docs/reference/setup-tools/kubeadm/kubeadm/ is redirected to
/docs/reference/setup-tools/kubeadm/
This replaces the redirect links of kubeadm with the direct links.
* README.md Added description/suggestion to install git submodules.
Created _index.md for es/docs/concept/policy.
Created limit-range.md, content-type:concept for es/docs/concepts/policy
* Changes to the README-es.md file discarded to apply a separate commit for it
* Fast fix, missed word
* Update content/es/docs/concepts/policy/limit-range.md
Suggestion, Remove "en" (fix). Personal note: I'm not sure if the suggested change makes clear that the system administrator can force the users to follow specifications with new politics.. Not always the user have control on the cluster
Co-authored-by: Rael Garcia <rael@rael.io>
* Update content/es/docs/concepts/policy/limit-range.md
Co-authored-by: Rael Garcia <rael@rael.io>
* Update content/es/docs/concepts/policy/limit-range.md
lgtm
Co-authored-by: Rael Garcia <rael@rael.io>
* Update content/es/docs/concepts/policy/limit-range.md
Nice
Co-authored-by: Rael Garcia <rael@rael.io>
* Update content/es/docs/concepts/policy/limit-range.md
Co-authored-by: Rael Garcia <rael@rael.io>
* Update content/es/docs/concepts/policy/limit-range.md
look at that trained eye! Good job thnaks
Co-authored-by: Rael Garcia <rael@rael.io>
* Update content/es/docs/concepts/policy/limit-range.md
lgtm
Co-authored-by: Rael Garcia <rael@rael.io>
* Update content/es/docs/concepts/policy/limit-range.md
Not sure about the translation, it sound better but it should be clear with terminology. A pod or container request resources it's not a requirement. But the same time we have [this definition](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/))
"Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value."
Co-authored-by: Rael Garcia <rael@rael.io>
* Update content/es/docs/concepts/policy/limit-range.md
Co-authored-by: Rael Garcia <rael@rael.io>
* Update content/es/docs/concepts/policy/limit-range.md
Lgtm
Co-authored-by: Rael Garcia <rael@rael.io>
* Suggested change discarded, reason: Contenedores is not defined
* Sorry my bad, Contenedores definition was there but as a singular noun not plural. Fiexed (line 51).. as well as pods (line 53
* Apply suggestions from code review
Wrong button last time. Added suggestions (as single commit)
Co-authored-by: Rael Garcia <rael@rael.io>
Co-authored-by: Victor Morales <chipahuac@hotmail.com>
* · Added: description/overview, file: content/es/docs/concepts/policy/_index.md
· Added: description, file: content/es/docs/concepts/policy/limit-range.md
· Midified: structure and dreafting, file: content/es/docs/concepts/policy/limit-range.md
· Added: term to glossary (es), file: content/es/docs/reference/glossary/limitrange.md
* Added Container word I deleted earlier today, lines 25,26,27.
Unnecessary repeated tooltip were removed allowing only once possible new concept per paragraph. Too many tooltip
* Fixed weight property commited by mistake previously.
* Added Suggested change manually
* Fixed Last suggested change
* All right I found the suggested change. Done
* Update content/es/docs/concepts/policy/limit-range.md
Co-authored-by: Rael Garcia <rael@rael.io>
* Update content/es/docs/concepts/policy/limit-range.md
Co-authored-by: Rael Garcia <rael@rael.io>
* Replaced occurrences of "Peticiones" by "Solicitudes" as it was suggested previously for line 29
Co-authored-by: Rael Garcia <rael@rael.io>
Co-authored-by: Victor Morales <chipahuac@hotmail.com>
This commit fixes the packages not found error during
Docker installation. The packages containerd.io 1.2.13-2,
docker-ce 19.03.11, and docker-cs-cli 19.03.11 are currently not in
Ubuntu 20.10. This commit instead points users to validated
versions of Docker.
fixes https://github.com/kubernetes/kubernetes/issues/99831
Signed-off-by: Enyinna Ochulor <eochulor@vmware.com>
Port 6379 was errantly identified as the mongo service port however mongo-service.yaml defines port 27017. I've updated the service command output to reflect that.
There is an extra space before the field 'type', so to create a secret failed. The message is "error: error parsing nginxsecret.yaml: error converting YAML to JSON: yaml: line 5: did not find expected key".
This simplifies the containerd installation instructions. All Linux distros will
now download from Docker repos, including Ubuntu 18.04 which previously
installed from Ubuntu repos.
The Docker runtime instructions have also been simplified. The RHEL/CentOS
specific option overlay2.override_kernel_check=true option has been removed.
It seems Docker will now autodetect overlay2 support on older Linux kernels
<4.0.0,>=3.10.0-514, which have back-ported overlay2 support on RHEL/CentOS 7.4+
See moby/moby#34368
Instead of putting "stringSlice" or "mapStringString" as data type for
flag data type, we can show more user friendly data type description
that is not GoLang specific.
* Add Concepts/Overview for Kubernetes and Components, and glossary itens in pt
Modify Glossary/Control-Plane definition in pt
Changes to be committed:
new file: content/pt/docs/concepts/overview/_index.md
new file: content/pt/docs/concepts/overview/components.md
new file: content/pt/docs/concepts/overview/what-is-kubernetes.md
new file: content/pt/docs/reference/glossary/cloud-controller-manager.md
new file: content/pt/docs/reference/glossary/cncf.md
new file: content/pt/docs/reference/glossary/container-runtime.md
modified: content/pt/docs/reference/glossary/control-plane.md
new file: content/pt/docs/reference/glossary/etcd.md
new file: content/pt/docs/reference/glossary/kube-apiserver.md
new file: content/pt/docs/reference/glossary/kube-controller-manager.md
new file: content/pt/docs/reference/glossary/kube-proxy.md
new file: content/pt/docs/reference/glossary/kube-scheduler.md
modified: package-lock.json
* Changes to be committed:
modified: package-lock.json
Untracked files:
content/pt/docs/concepts/overview/kubernetes-api.md
* Changes to be committed:
modified: package-lock.json
Untracked files:
content/pt/docs/concepts/overview/kubernetes-api.md
* ---
reviewers:
- lavalamp
- rikatz
title: Componentes do Kubernetes
content_type: concept
description: >
Um cluster Kubernetes consiste de componentes que representam a camada de gerenciamento, e um conjunto de máquinas chamadas nós.
weight: 20
card:
name: concepts
weight: 20
---
<!-- overview -->
Ao implantar o Kubernetes, você obtém um cluster.
{{< glossary_definition term_id="cluster" length="all" prepend="Um cluster Kubernetes consiste em">}}
Este documento descreve os vários componentes que você precisa ter para implantar um cluster Kubernetes completo e funcional.
Esse é o diagrama de um cluster Kubernetes com todos os componentes interligados.

<!-- body -->
## Componentes da camada de gerenciamento
Os componentes da camada de gerenciamento tomam decisões globais sobre o cluster (por exemplo, agendamento de _pods_), bem como detectam e respondem aos eventos do cluster (por exemplo, iniciando um novo _{{< glossary_tooltip text="pod" term_id="pod" >}}_ quando o campo `replicas` de um _Deployment_ não está atendido).
Os componentes da camada de gerenciamento podem ser executados em qualquer máquina do cluster. Contudo, para simplificar, os _scripts_ de configuração normalmente iniciam todos os componentes da camada de gerenciamento na mesma máquina, e não executa contêineres de usuário nesta máquina. Veja [Construindo clusters de alta disponibilidade](/docs/admin/high-availability/) para um exemplo de configuração de múltiplas VMs para camada de gerenciamento (_multi-main-VM_).
### kube-apiserver
{{< glossary_definition term_id="kube-apiserver" length="all" >}}
### etcd
{{< glossary_definition term_id="etcd" length="all" >}}
### kube-scheduler
{{< glossary_definition term_id="kube-scheduler" length="all" >}}
### kube-controller-manager
{{< glossary_definition term_id="kube-controller-manager" length="all" >}}
Alguns tipos desses controladores são:
* Controlador de nó: responsável por perceber e responder quando os nós caem.
* Controlador de _Job_: Observa os objetos _Job_ que representam tarefas únicas e, em seguida, cria _pods_ para executar essas tarefas até a conclusão.
* Controlador de _endpoints_: preenche o objeto _Endpoints_ (ou seja, junta os Serviços e os _pods_).
* Controladores de conta de serviço e de _token_: crie contas padrão e _tokens_ de acesso de API para novos _namespaces_.
### cloud-controller-manager
{{< glossary_definition term_id="cloud-controller-manager" length="short" >}}
O cloud-controller-manager executa apenas controladores que são específicos para seu provedor de nuvem.
Se você estiver executando o Kubernetes em suas próprias instalações ou em um ambiente de aprendizagem dentro de seu
próprio PC, o cluster não possui um gerenciador de controlador de nuvem.
Tal como acontece com o kube-controller-manager, o cloud-controller-manager combina vários ciclos de controle logicamente independentes em um binário único que você executa como um processo único. Você pode escalar horizontalmente (exectuar mais de uma cópia) para melhorar o desempenho ou para auxiliar na tolerância a falhas.
Os seguintes controladores podem ter dependências de provedor de nuvem:
* Controlador de nó: para verificar junto ao provedor de nuvem para determinar se um nó foi excluído da nuvem após parar de responder.
* Controlador de rota: para configurar rotas na infraestrutura de nuvem subjacente.
* Controlador de serviço: Para criar, atualizar e excluir balanceadores de carga do provedor de nuvem.
## Node Components
Os componentes de nó são executados em todos os nós, mantendo os _pods_ em execução e fornecendo o ambiente de execução do Kubernetes.
### kubelet
{{< glossary_definition term_id="kubelet" length="all" >}}
### kube-proxy
{{< glossary_definition term_id="kube-proxy" length="all" >}}
### Container runtime
{{< glossary_definition term_id="container-runtime" length="all" >}}
## Addons
Complementos (_addons_) usam recursos do Kubernetes ({{< glossary_tooltip term_id="daemonset" >}}, {{< glossary_tooltip term_id="deployment" >}}, etc) para implementar funcionalidades do cluster. Como fornecem funcionalidades em nível do cluster, recursos de _addons_ que necessitem ser criados dentro de um _namespace_ pertencem ao _namespace_ `kube-system`.
Alguns _addons_ selecionados são descritos abaixo; para uma lista estendida dos _addons_ disponíveis, por favor consulte [Addons](/docs/concepts/cluster-administration/addons/).
### DNS
Embora os outros complementos não sejam estritamente necessários, todos os clusters do Kubernetes devem ter um [DNS do cluster](/docs/concepts/services-networking/dns-pod-service/), já que muitos exemplos dependem disso.
O DNS do cluster é um servidor DNS, além de outros servidores DNS em seu ambiente, que fornece registros DNS para serviços do Kubernetes.
Os contêineres iniciados pelo Kubernetes incluem automaticamente esse servidor DNS em suas pesquisas DNS.
### Web UI (Dashboard)
[Dashboard](/docs/tasks/access-application-cluster/web-ui-dashboard/) é uma interface de usuário Web, de uso geral, para clusters do Kubernetes. Ele permite que os usuários gerenciem e solucionem problemas de aplicações em execução no cluster, bem como o próprio cluster.
### Monitoramento de recursos do contêiner
[Monitoramento de recursos do contêiner](/docs/tasks/debug-application-cluster/resource-usage-monitoring/) registra métricas de série temporal genéricas sobre os contêineres em um banco de dados central e fornece uma interface de usuário para navegar por esses dados.
### Logging a nivel do cluster
Um mecanismo de [_logging_ a nível do cluster](/docs/concepts/cluster-administration/logging/) é responsável por guardar os _logs_ dos contêineres em um armazenamento central de _logs_ com um interface para navegação/pesquisa.
## {{% heading "whatsnext" %}}
* Aprenda sobre [Nós](/docs/concepts/architecture/nodes/).
* Aprenda sobre [Controladores](/docs/concepts/architecture/controller/).
* Aprenda sobre [kube-scheduler](/docs/concepts/scheduling-eviction/kube-scheduler/).
* Leia a [documentação](https://etcd.io/docs/) oficial do **etcd**.
* ---
reviewers:
title: Componentes do Kubernetes
content_type: concept
description: >
Um cluster Kubernetes consiste de componentes que representam a camada de gerenciamento, e um conjunto de máquinas chamadas nós.
weight: 20
card:
name: concepts
weight: 20
---
<!-- overview -->
Ao implantar o Kubernetes, você obtém um cluster.
{{< glossary_definition term_id="cluster" length="all" prepend="Um cluster Kubernetes consiste em">}}
Este documento descreve os vários componentes que você precisa ter para implantar um cluster Kubernetes completo e funcional.
Esse é o diagrama de um cluster Kubernetes com todos os componentes interligados.

<!-- body -->
## Componentes da camada de gerenciamento
Os componentes da camada de gerenciamento tomam decisões globais sobre o cluster (por exemplo, agendamento de _pods_), bem como detectam e respondem aos eventos do cluster (por exemplo, iniciando um novo _{{< glossary_tooltip text="pod" term_id="pod" >}}_ quando o campo `replicas` de um _Deployment_ não está atendido).
Os componentes da camada de gerenciamento podem ser executados em qualquer máquina do cluster. Contudo, para simplificar, os _scripts_ de configuração normalmente iniciam todos os componentes da camada de gerenciamento na mesma máquina, e não executa contêineres de usuário nesta máquina. Veja [Construindo clusters de alta disponibilidade](/docs/admin/high-availability/) para um exemplo de configuração de múltiplas VMs para camada de gerenciamento (_multi-main-VM_).
### kube-apiserver
{{< glossary_definition term_id="kube-apiserver" length="all" >}}
### etcd
{{< glossary_definition term_id="etcd" length="all" >}}
### kube-scheduler
{{< glossary_definition term_id="kube-scheduler" length="all" >}}
### kube-controller-manager
{{< glossary_definition term_id="kube-controller-manager" length="all" >}}
Alguns tipos desses controladores são:
* Controlador de nó: responsável por perceber e responder quando os nós caem.
* Controlador de _Job_: Observa os objetos _Job_ que representam tarefas únicas e, em seguida, cria _pods_ para executar essas tarefas até a conclusão.
* Controlador de _endpoints_: preenche o objeto _Endpoints_ (ou seja, junta os Serviços e os _pods_).
* Controladores de conta de serviço e de _token_: crie contas padrão e _tokens_ de acesso de API para novos _namespaces_.
### cloud-controller-manager
{{< glossary_definition term_id="cloud-controller-manager" length="short" >}}
O cloud-controller-manager executa apenas controladores que são específicos para seu provedor de nuvem.
Se você estiver executando o Kubernetes em suas próprias instalações ou em um ambiente de aprendizagem dentro de seu
próprio PC, o cluster não possui um gerenciador de controlador de nuvem.
Tal como acontece com o kube-controller-manager, o cloud-controller-manager combina vários ciclos de controle logicamente independentes em um binário único que você executa como um processo único. Você pode escalar horizontalmente (exectuar mais de uma cópia) para melhorar o desempenho ou para auxiliar na tolerância a falhas.
Os seguintes controladores podem ter dependências de provedor de nuvem:
* Controlador de nó: para verificar junto ao provedor de nuvem para determinar se um nó foi excluído da nuvem após parar de responder.
* Controlador de rota: para configurar rotas na infraestrutura de nuvem subjacente.
* Controlador de serviço: Para criar, atualizar e excluir balanceadores de carga do provedor de nuvem.
## Node Components
Os componentes de nó são executados em todos os nós, mantendo os _pods_ em execução e fornecendo o ambiente de execução do Kubernetes.
### kubelet
{{< glossary_definition term_id="kubelet" length="all" >}}
### kube-proxy
{{< glossary_definition term_id="kube-proxy" length="all" >}}
### Container runtime
{{< glossary_definition term_id="container-runtime" length="all" >}}
## Addons
Complementos (_addons_) usam recursos do Kubernetes ({{< glossary_tooltip term_id="daemonset" >}}, {{< glossary_tooltip term_id="deployment" >}}, etc) para implementar funcionalidades do cluster. Como fornecem funcionalidades em nível do cluster, recursos de _addons_ que necessitem ser criados dentro de um _namespace_ pertencem ao _namespace_ `kube-system`.
Alguns _addons_ selecionados são descritos abaixo; para uma lista estendida dos _addons_ disponíveis, por favor consulte [Addons](/docs/concepts/cluster-administration/addons/).
### DNS
Embora os outros complementos não sejam estritamente necessários, todos os clusters do Kubernetes devem ter um [DNS do cluster](/docs/concepts/services-networking/dns-pod-service/), já que muitos exemplos dependem disso.
O DNS do cluster é um servidor DNS, além de outros servidores DNS em seu ambiente, que fornece registros DNS para serviços do Kubernetes.
Os contêineres iniciados pelo Kubernetes incluem automaticamente esse servidor DNS em suas pesquisas DNS.
### Web UI (Dashboard)
[Dashboard](/docs/tasks/access-application-cluster/web-ui-dashboard/) é uma interface de usuário Web, de uso geral, para clusters do Kubernetes. Ele permite que os usuários gerenciem e solucionem problemas de aplicações em execução no cluster, bem como o próprio cluster.
### Monitoramento de recursos do contêiner
[Monitoramento de recursos do contêiner](/docs/tasks/debug-application-cluster/resource-usage-monitoring/) registra métricas de série temporal genéricas sobre os contêineres em um banco de dados central e fornece uma interface de usuário para navegar por esses dados.
### Logging a nivel do cluster
Um mecanismo de [_logging_ a nível do cluster](/docs/concepts/cluster-administration/logging/) é responsável por guardar os _logs_ dos contêineres em um armazenamento central de _logs_ com um interface para navegação/pesquisa.
## {{% heading "whatsnext" %}}
* Aprenda sobre [Nós](/docs/concepts/architecture/nodes/).
* Aprenda sobre [Controladores](/docs/concepts/architecture/controller/).
* Aprenda sobre [kube-scheduler](/docs/concepts/scheduling-eviction/kube-scheduler/).
* Leia a [documentação](https://etcd.io/docs/) oficial do **etcd**.
* ---
reviewers:
title: O que é Kubernetes?
description: >
Kubernetes é um plataforma de código aberto, portável e extensiva para o gerenciamento de cargas de trabalho e serviços distribuídos em contêineres, que facilita tanto a configuração declarativa quanto a automação. Ele possui um ecossistema grande, e de rápido crescimento. Serviços, suporte, e ferramentas para Kubernetes estão amplamente disponíveis.
content_type: concept
weight: 10
card:
name: concepts
weight: 10
sitemap:
priority: 0.9
---
<!-- overview -->
Essa página é uma visão geral do Kubernetes.
<!-- body -->
Kubernetes é um plataforma de código aberto, portável e extensiva para o gerenciamento de cargas de trabalho e serviços distribuídos em contêineres, que facilita tanto a configuração declarativa quanto a automação. Ele possui um ecossistema grande, e de rápido crescimento. Serviços, suporte, e ferramentas para Kubernetes estão amplamente disponíveis.
O Google tornou Kubernetes um projeto de código-aberto em 2014. O Kubernetes combina [mais de 15 anos de experiência do Google](/blog/2015/04/borg-predecessor-to-kubernetes/) executando cargas de trabalho produtivas em escala, com as melhores idéias e práticas da comunidade.
O nome **Kubernetes** tem origem no Grego, significando _timoneiro_ ou _piloto_. **K8s** é a abreviação derivada pela troca das oito letras "ubernete" por "8", se tornado _K"8"s_.
## Voltando no tempo
Vamos dar uma olhada no porque o Kubernetes é tão útil, voltando no tempo.

**Era da implantação tradicional:** No início, as organizações executavam aplicações em servidores físicos. Não havia como definir limites de recursos para aplicações em um mesmo servidor físico, e isso causava problemas de alocação de recursos. Por exemplo, se várias aplicações fossem executadas em um mesmo servidor físico, poderia haver situações em que uma aplicação ocupasse a maior parte dos recursos e, como resultado, o desempenho das outras aplicações seria inferior. Uma solução para isso seria executar cada aplicação em um servidor físico diferente. Mas isso não escalava, pois os recursos eram subutilizados, e se tornava custoso para as organizações manter muitos servidores físicos.
**Era da implantação virtualizada:** Como solução, a virtualização foi introduzida. Esse modelo permite que você execute várias máquinas virtuais (VMs) em uma única CPU de um servidor físico. A virtualização permite que as aplicações sejam isoladas entre as VMs, e ainda fornece um nível de segurança, pois as informações de uma aplicação não podem ser acessadas livremente por outras aplicações.
A virtualização permite melhor utilização de recursos em um servidor físico, e permite melhor escalabilidade porque uma aplicação pode ser adicionada ou atualizada facilmente, reduz os custos de hardware e muito mais. Com a virtualização, você pode apresentar um conjunto de recursos físicos como um cluster de máquinas virtuais descartáveis.
Cada VM é uma máquina completa que executa todos os componentes, incluindo seu próprio sistema operacional, além do hardware virtualizado.
**Era da implantação em contêineres:** Contêineres são semelhantes às VMs, mas têm propriedades de isolamento flexibilizados para compartilhar o sistema operacional (SO) entre as aplicações. Portanto, os contêineres são considerados leves. Semelhante a uma VM, um contêiner tem seu próprio sistema de arquivos, compartilhamento de CPU, memória, espaço de processo e muito mais. Como eles estão separados da infraestrutura subjacente, eles são portáveis entre nuvens e distribuições de sistema operacional.
Contêineres se tornaram populares porque eles fornecem benefícios extra, tais como:
* Criação e implantação ágil de aplicações: aumento da facilidade e eficiência na criação de imagem de contêiner comparado ao uso de imagem de VM.
* Desenvolvimento, integração e implantação contínuos: fornece capacidade de criação e de implantação de imagens de contêiner de forma confiável e frequente, com a funcionalidade de efetuar reversões rápidas e eficientes (devido à imutabilidade da imagem).
* Separação de interesses entre Desenvolvimento e Operações: crie imagens de contêineres de aplicações no momento de construção/liberação em vez de no momento de implantação, desacoplando as aplicações da infraestrutura.
* A capacidade de observação (Observabilidade) não apenas apresenta informações e métricas no nível do sistema operacional, mas também a integridade da aplicação e outros sinais.
* Consistência ambiental entre desenvolvimento, teste e produção: funciona da mesma forma em um laptop e na nuvem.
* Portabilidade de distribuição de nuvem e sistema operacional: executa no Ubuntu, RHEL, CoreOS, localmente, nas principais nuvens públicas e em qualquer outro lugar.
* Gerenciamento centrado em aplicações: eleva o nível de abstração da execução em um sistema operacional em hardware virtualizado à execução de uma aplicação em um sistema operacional usando recursos lógicos.
* Microserviços fracamente acoplados, distribuídos, elásticos e livres: as aplicações são divididas em partes menores e independentes e podem ser implantados e gerenciados dinamicamente - não uma pilha monolítica em execução em uma grande máquina de propósito único.
* Isolamento de recursos: desempenho previsível de aplicações.
* Utilização de recursos: alta eficiência e densidade.
## Por que você precisa do Kubernetes e o que ele pode fazer{#why-you-need-kubernetes-and-what-can-it-do}
Os contêineres são uma boa maneira de agrupar e executar suas aplicações. Em um ambiente de produção, você precisa gerenciar os contêineres que executam as aplicações e garantir que não haja tempo de inatividade. Por exemplo, se um contêiner cair, outro contêiner precisa ser iniciado. Não seria mais fácil se esse comportamento fosse controlado por um sistema?
É assim que o Kubernetes vem ao resgate! O Kubernetes oferece uma estrutura para executar sistemas distribuídos de forma resiliente. Ele cuida do escalonamento e do recuperação à falha de sua aplicação, fornece padrões de implantação e muito mais. Por exemplo, o Kubernetes pode gerenciar facilmente uma implantação no método canário para seu sistema.
O Kubernetes oferece a você:
* **Descoberta de serviço e balanceamento de carga**
O Kubernetes pode expor um contêiner usando o nome DNS ou seu próprio endereço IP. Se o tráfego para um contêiner for alto, o Kubernetes pode balancear a carga e distribuir o tráfego de rede para que a implantação seja estável.
* **Orquestração de armazenamento**
O Kubernetes permite que você monte automaticamente um sistema de armazenamento de sua escolha, como armazenamentos locais, provedores de nuvem pública e muito mais.
* **Lançamentos e reversões automatizadas**
Você pode descrever o estado desejado para seus contêineres implantados usando o Kubernetes, e ele pode alterar o estado real para o estado desejado em um ritmo controlada. Por exemplo, você pode automatizar o Kubernetes para criar novos contêineres para sua implantação, remover os contêineres existentes e adotar todos os seus recursos para o novo contêiner.
* **Empacotamento binário automático**
Você fornece ao Kubernetes um cluster de nós que pode ser usado para executar tarefas nos contêineres. Você informa ao Kubernetes de quanta CPU e memória (RAM) cada contêiner precisa. O Kubernetes pode encaixar contêineres em seus nós para fazer o melhor uso de seus recursos.
* **Autocorreção**
O Kubernetes reinicia os contêineres que falham, substitui os contêineres, elimina os contêineres que não respondem à verificação de integridade definida pelo usuário e não os anuncia aos clientes até que estejam prontos para servir.
* **Gerenciamento de configuração e de segredos**
O Kubernetes permite armazenar e gerenciar informações confidenciais, como senhas, tokens OAuth e chaves SSH. Você pode implantar e atualizar segredos e configuração de aplicações sem reconstruir suas imagens de contêiner e sem expor segredos em sua pilha de configuração.
## O que o Kubernetes não é
O Kubernetes não é um sistema PaaS (plataforma como serviço) tradicional e completo. Como o Kubernetes opera no nível do contêiner, e não no nível do hardware, ele fornece alguns recursos geralmente aplicáveis comuns às ofertas de PaaS, como implantação, escalonamento, balanceamento de carga, e permite que os usuários integrem suas soluções de _logging_, monitoramento e alerta. No entanto, o Kubernetes não é monolítico, e essas soluções padrão são opcionais e conectáveis. O Kubernetes fornece os blocos de construção para a construção de plataformas de desenvolvimento, mas preserva a escolha e flexibilidade do usuário onde é importante.
Kubernetes:
* Não limita os tipos de aplicações suportadas. O Kubernetes visa oferecer suporte a uma variedade extremamente diversa de cargas de trabalho, incluindo cargas de trabalho sem estado, com estado e de processamento de dados. Se uma aplicação puder ser executada em um contêiner, ele deve ser executado perfeitamente no Kubernetes.
* Não implanta código-fonte e não constrói sua aplicação. Os fluxos de trabalho de integração contínua, entrega e implantação (CI/CD) são determinados pelas culturas e preferências da organização, bem como pelos requisitos técnicos.
* Não fornece serviços em nível de aplicação, tais como middleware (por exemplo, barramentos de mensagem), estruturas de processamento de dados (por exemplo, Spark), bancos de dados (por exemplo, MySQL), caches, nem sistemas de armazenamento em cluster (por exemplo, Ceph), como serviços integrados. Esses componentes podem ser executados no Kubernetes e/ou podem ser acessados por aplicações executadas no Kubernetes por meio de mecanismos portáteis, como o [Open Service Broker](https://openservicebrokerapi.org/).
* Não dita soluções de _logging_, monitoramento ou alerta. Ele fornece algumas integrações como prova de conceito e mecanismos para coletar e exportar métricas.
* Não fornece nem exige um sistema/idioma de configuração (por exemplo, Jsonnet). Ele fornece uma API declarativa que pode ser direcionada por formas arbitrárias de especificações declarativas.
* Não fornece nem adota sistemas abrangentes de configuração de máquinas, manutenção, gerenciamento ou autocorreção.
* Adicionalmente, o Kubernetes não é um mero sistema de orquestração. Na verdade, ele elimina a necessidade de orquestração. A definição técnica de orquestração é a execução de um fluxo de trabalho definido: primeiro faça A, depois B e depois C. Em contraste, o Kubernetes compreende um conjunto de processos de controle independentes e combináveis que conduzem continuamente o estado atual em direção ao estado desejado fornecido. Não importa como você vai de A para C. O controle centralizado também não é necessário. Isso resulta em um sistema que é mais fácil de usar e mais poderoso, robusto, resiliente e extensível.
## {{% heading "whatsnext" %}}
* Dê uma olhada em [Componentes do Kubernetes](/docs/concepts/overview/components/).
* Pronto para [Iniciar](/docs/setup/)?
The use of `apt-key` to install has also been removed as it is now deprecated
and will be last available in Debian 11 and Ubuntu 22.04.
Also updates the Docker repository setup instructions in container-runtimes.md,
to now refer to the respective instructions at https://docs.docker.com/engine/install/
which has already made the move to use the signed-by option.
* Add Dont Panic blog post in pt language
Add Dont Panic blog post in pt language.
Partial fix for kubernetes #13939
* Update content/pt/blog/_posts/2020-12-02-dont-panic-kubernetes-and-docker.md
Co-authored-by: Tim Bannister <tim@scalefactory.com>
* Update content/pt/blog/_posts/2020-12-02-dont-panic-kubernetes-and-docker.md
Co-authored-by: Tim Bannister <tim@scalefactory.com>
* Update 2020-12-02-dont-panic-kubernetes-and-docker.md
typo `:%s/\<kubernetes\>/Kubernetes/gc` and `:%s/é/e`
* Update 2020-12-02-dont-panic-kubernetes-and-docker.md
removendo depreciação e arrumando typos
* remove enterprise name
* Update content/pt/blog/_posts/2020-12-02-dont-panic-kubernetes-and-docker.md
Co-authored-by: Tim Bannister <tim@scalefactory.com>
Update content/pt/blog/_posts/2020-12-02-dont-panic-kubernetes-and-docker.md
Co-authored-by: Tim Bannister <tim@scalefactory.com>
Update 2020-12-02-dont-panic-kubernetes-and-docker.md
typo `:%s/\<kubernetes\>/Kubernetes/gc` and `:%s/é/e`
Update 2020-12-02-dont-panic-kubernetes-and-docker.md
removendo depreciação e arrumando typos
remove enterprise name
Co-authored-by: Tim Bannister <tim@scalefactory.com>
/docs/reference/setup-tools/kubeadm/kubeadm/ is redirected to
/docs/reference/setup-tools/kubeadm/
This replaces the redirect links of kubeadm with the direct links.
* Move accessing API from within pod to tasks
* Remove reviewers, version check; Add whatsnext
* Move to run applications
* Fix what's next section link
The `imagePullPolicy` field is set automatically based on the image tag
if it's initially omitted, but it is not updated if the image tag later
changes. This can lead to [confusing
behaviour](https://itnext.io/defaults-are-hard-kubernetes-deployment-edition-3b11095792f2).
This change attempts to warn users of this potential pitfall.
Fix `Source files` section in localization document, and remove {{< release-branch >}} variable
due to `release-1.20` branch does not exist.
- improve sentence
- Fix latest branch for latest version
- Add description if latest branch does not exist
- Describe about master branch
- Apply suggestions from code review
- Unify `development branch` to` localization branch`
- Remove description for switching upstream branch
- Add description for switching upstream
- Also, add description for merging to master and new release branch.
Co-authored-by: Qiming Teng <tengqim@cn.ibm.com>
Co-authored-by: Tim Bannister <tim@scalefactory.com>
Co-authored-by: Seokho Son <shsongist@gmail.com>
* Add EndpointSlice blog post in pt language
* Apply corrections into endpointslice blog post translation
Co-authored-by: Jhon Mike <jhon.msdev@gmail.com>
Co-authored-by: Jhon Mike <jhon.msdev@gmail.com>
This page listed `/var/run/docker.sock` as the UNIX domain socket path for the Docker container runtime, but it's actually `/var/run/dockershim.sock`, as the kubelet documentation indicates as the default value for the `--container-runtime-endpoint` argument:
https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/
The socket file specified (`/var/run/docker.sock`) is where the Docker daemon listens for requests for the Docker API, not the CRI interface.
* Add (stub) localization guide for Spanish
Co-authored-by: Victor Morales <chipahuac@hotmail.com>
* docs: Add brief page introduction
* docs: use archivo as discussed with the team
Co-authored-by: Victor Morales <chipahuac@hotmail.com>
Co-authored-by: Rael Garcia <rael@rael.io>
There is a warning deprecation message for kubernetes v1.19:
```
Warning: networking.k8s.io/v1beta1 Ingress is deprecated in v1.19+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
```
Switched apiversion in the example to `networking.k8s.io/v1`
* Translate Explore You App in Portuguese
* Update Explore Your App link in Learn Kubernetes Basics _index.html
* Update link to Explore Interactive page translation in Portuguese
* Fix typo and change the translation of the word *worker*.
Signed-off-by: Jailton Lopes <jailton@gmail.com>
* Fix typo
Signed-off-by: Jailton Lopes <jailton@gmail.com>
* Resync with english version in 'Dual-stack docs correction #26386'
* Update in 'administer-cluster\safely-drain-node.md' #25996
* Fix git rebase in services-networking\dual-stack.md
Signed-off-by: ydFu <ader.ydfu@gmail.com>
- Ko: add l10n contributor name in a blog (#26285)
- Translate reference/glossary/secret.md in Korean (#26391)
- Update outdated files in the dev-1.20-ko.4 branch (2) (#26342)
- Translate reference/glossary/quantity.md in Korean (#26390)
- Update outdated files in the dev-1.20-ko.4 branch (1) (#26427)
- Translate reference/glossary/storage-class.md in Korean (#26419)
- Update outdated files in dev-1.20-ko.4 branch (3) (#26599)
Co-authored-by: seokho-son <shsongist@gmail.com>
Co-authored-by: Jerry Park <jaehwa@gmail.com>
Co-authored-by: santachopa <santachopa@naver.com>
Co-authored-by: jmyung <jesang.myung@gmail.com>
Operator Pattern link was pointing to https://coreos.com/operator. This link was redirecting to the openshift.com homepage.
Provided new link on openshift.com pointing to Operator Pattern documentation.
The docs don't mention when the kubelet will attempt to renew a cert,
which causes concern when one notices that certain certificates are being
renewed and others are not. Adding the time frame adds certainty, so that
if an user notices a kubelet cert expiring in less than 30d, they know
something is misconfigured and should be looked at.
Without first mentioning where the number 40 is coming from, it was a bit unclear why 40 would be the threshold value for policy selection in this case.
I hope this patch clarifies it.
update based on English version of statefulset.md
1. added a block inside #stable-network-id
2. sync tables and blocks
3. added a sentence in #parallel-pod-management
the `!` in `!=` should use halfwidth form instead of fullwidth, otherwise it will cause trouble in K8s.
Signed-off-by: Hollow Man <hollowman@hollowman.ml>
The "Automatic mounting of manually created Secrets" section of the
Secrets documentation previously suggesting using PodPresets. PodPresets
have been removed, there is no alternate facility described, and it's
unclear if auto-mounting secrets based on associations with
ServiceAccounts was ever supported. Accordingly, the section should be
removed.
This is part of work in umbrella issue:
[zh] Umbrella issue: pages out of sync in tasks section #26178
Topology Manageer (L):
content/zh/docs/tasks/administer-cluster/topology-manager.md
This is part of the work in the following umbrella issue:
[zh] Umbrella issue: pages out of sync in tasks section
CRICTL (L)
content/zh/docs/tasks/debug-application-cluster/crictl.md
This is part of work in umbrella issue:
Umbrella issue: pages out of sync in tasks section #26178
HA (L):
content/zh/docs/tasks/administer-cluster/highly-available-master.md
The current layout partial is using scripts from a site not accessible
from behind the dam great firewall. This is a fix to cache the scripts
as we do before so that tabs, top menu works for everyone.
The original link has moved due to docs restructuring in the
cloud-provider-openstack repo. Also correcting the capitalization.
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
This commit adds a cosmetic change by adding backticks to show the env
var in monospace font when rendered.
Mention Flatcar Linux in kubeadm troubleshooting doc, alongwith Fedora
CoreOS.
Signed-off-by: Suraj Deshmukh <surajd.service@gmail.com>
The old link points to the code base of 1.11. Since then there have been
several changes to the configuration, either new flags added or removed.
Signed-off-by: Suraj Deshmukh <surajd.service@gmail.com>
1. change keyword replicationController to ReplicaSet
2. change word 连续 to 持续
3. other minor changes (e.g: add connection word for readability, add 'shell' for command line input)
This is for part of work in the umbrella issue:
[zh] Umbrella issue: pages out of sync in tasks section #26178
Portforward (L)
content/zh/docs/tasks/access-application-cluster/port-forward-access-application-cluster.md
- Update outdated files in the dev-1.20-ko.3 (1) (#26131)
- Update outdated files in the dev-1.20-ko.3 branch (2) (#26122)
Co-authored-by: seokho-son <shsongist@gmail.com>
Co-authored-by: Jerry Park <jaehwa@gmail.com>
Unless that first part of the paragraph is meant to be more like a subheading (in which case it could use some reformatting), I would like to suggest a minor rephrase so that it becomes a proper sentence and has a predicate.
Thanks for considering this!
This is for part of the items in the following umbrella issue:
[zh] Umbrella issue: pages out of sync in tasks section #26178
Service Account (L)
content/zh/docs/tasks/configure-pod-container/configure-service-account.md
Sync web page for part of the items in umbrella issue:
[zh] Umbrella issue: pages out of sync in tasks section #26178
content/zh/docs/tasks/administer-cluster/nodelocaldns.md
Sync web page for part of the items in umbrella issue:
Misc Batch 2 (S)
content/zh/docs/tasks/debug-application-cluster/debug-pod-replication-controller.md
content/zh/docs/tasks/manage-kubernetes-objects/update-api-object-kubectl-patch.md
content/zh/docs/tasks/access-application-cluster/configure-access-multiple-clusters.md
Just a small clarification on the description of the responsibilities of a Job. Kubernetes doesn't ensure that the Pods are successful, but the scheduler's responsibility is to continue retrying until success. Updating the docs with a slight wording change to more accurately reflect that a Job won't ensure success, but it will continue retrying until success.
Sync web page for part of the items in umbrella issue:
[zh] Umbrella issue: pages out of sync in concepts section #26177
Garbage Collection
content/zh/docs/concepts/workloads/controllers/garbage-collection.md
* [zh] Umbrella issue: pages out of sync in concepts section(Misc Batch 5)
```
[x] content/zh/docs/concepts/policy/resource-quotas.md
[x] content/zh/docs/concepts/cluster-administration/system-metrics.md
[x] content/zh/docs/concepts/cluster-administration/flow-control.md
```
* sync with english version in policy/pod-security-policy.md
Signed-off-by: ydFu ader.ydfu@gmail.com
* [zh] Umbrella issue: pages out of sync in concepts section #26177(Misc Batch 4)
```
[x] content/zh/docs/concepts/architecture/controller.md
[x] content/zh/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins.md
[x] content/zh/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins.md
[x] content/zh/docs/concepts/extend-kubernetes/operator.md
```
* sync with english version in home/_index.md
Signed-off-by: ydFu <ader.ydfu@gmail.com>
A few links to the flexVolume documentation do not resolve correctly due
to case sensitivity in the page anchor. This updates those links to
resolve to the correct section of the volumes doc.
Signed-off-by: hasheddan <georgedanielmangum@gmail.com>
* # This is a combination of 14 commits.
# The first commit's message is:
Update patch release manager to patch release team in version skew po… (#17008)
* Update patch release manager to patch release team in skew support policy.
* Adding link for additional patch-release information to version skew policy.
# This is the 2nd commit message:
translate the first part of secret.md document
# This is the 3rd commit message:
translate secrets montados se actualizan automáticamente
# This is the 4th commit message:
translation until Montaje Automatico de Secrets Creados Manualmente
# This is the 5th commit message:
translate restrictions
# This is the 6th commit message:
translate Interacción del Secret y Pod de por Vida
# This is the 7th commit message:
finish first translation version secret.md
# This is the 8th commit message:
fix spaces
# This is the 9th commit message:
checking first part
# This is the 10th commit message:
review translation ultil line 232
# This is the 11th commit message:
fix some issues line 509
# This is the 12th commit message:
finish first revision secret documentation
# This is the 13th commit message:
remove pod-overview
# This is the 14th commit message:
added secret content in spanish content/es/docs/concepts/configuration/secret.md
* “Secret” es un objet Kubernetes
* capital letter in Secret Kubernetes Object
* Remove ` from k8s native objects
Co-Authored-By: Tim Bannister <tim@scalefactory.com>
* Remove ` from k8s native objects
Co-Authored-By: Tim Bannister <tim@scalefactory.com>
* Capitalize Secret
Co-Authored-By: Tim Bannister <tim@scalefactory.com>
* Update reviewer
* Apply suggestions from code review
Co-authored-by: Mitesh Jain <47820816+miteshskj@users.noreply.github.com>
Co-authored-by: Rael Garcia <rael@rael.io>
Co-authored-by: Tim Bannister <tim@scalefactory.com>
`service2` is mapped to `second.bar.com` in the given example
```
- host: second.bar.com
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: service2
port:
number: 80
```
The stateless application tutorial is littered with unsuitable terminology. This update switches the database technology to a database that has already updated their own terminology as well as removes the terminology from the tutorial itself.
The advanced logging tutorial followup is set to Draft as it will take a larger effort to convert that, and I'm not convinced its even in a working state right now.
A followup PR to the examples repo to be referenced here will be made.
Addresses #22918
Signed-off-by: Paul Czarkowski <username.taken@gmail.com>
address comments in PR
Signed-off-by: Paul Czarkowski <username.taken@gmail.com>
remove confusing comment about dns, we can assume k8s has kube-dns
Signed-off-by: Paul Czarkowski <username.taken@gmail.com>
This PR updates the go.mod file so that tests of the example manifests
are run against the 1.20 branch. The missing test cases for newly added
examples are also added. To perform tests on your local machine, run the
following command on the root of your local clone:
```
go test k8s.io/website/content/en/examples
```
This patch updates the CRI-O documentation to the latest available
version. It also adds a dedicated cgroup driver section to mention that
the configuration has to be in sync.
Signed-off-by: Sascha Grunert <sgrunert@suse.com>
- Update outdated files in the dev-1.20-ko.2(p1) (#25915)
- Update outdated files in the dev-1.20-ko.2(p2) (#25916)
- Fix issue with links to already translated ko documents (#25991)
- Translate reference/glossary/dynamic-volume-provisioning.md in Korean (#26047)
Co-authored-by: seokho-son <shsongist@gmail.com>
Co-authored-by: Jerry Park <jaehwa@gmail.com>
Co-authored-by: santachopa <santachopa@naver.com>
* Enhancements in clarity to docs/tasks/access-application-cluster/connecting-frontend-backend.md
* Code review comments
* Reverting name of service due to backed value in Dockerimage
As suggested, removed the language related to common vernacular. I think the documentation is well written in the common labels section, and can possibly be enhanced as more and more of these labels are implemented. So, just a link in the best practice section is sufficient as suggested by you.
as the CKA requires taking these code snippets and using them quickly, spaces can be an issue
add two spaces to begin of line to make pod spec copying even faster (if this is the case)
* Add the code blocks in the Markdown spec to make it easy to read.
* Add description that distinguish between **command** and **output** make it easy to read.
* Adjust description in Kubernetes components for smoother reading.
Signed-off-by: ydFu <ader.ydfu@gmail.com>
I added a <br> after the end of the third bullet and backed out all of the other changes I suggested in the original pull request. I think this better matches the author's original intent. The only difference now between what's currently published and this edit is the line break coded after the third bullet.
The current explanation is misleading (for example: for consistently decreasing resource usage, the HPA stabilization does not impact the frequency of its actuation). This has confused users recently ( https://github.com/kubernetes/kubernetes/issues/96671 )
This change allows announcements to have an expiry date and / or a
“do not show until” date.
Separating this out also leaves room for future changes to enforce
a set of approvers.
Co-Authored-By: Karen Bradshaw <kbhawkey@gmail.com>
* Add Code blocks in the Markdown spec to make it easy to read.
* Uniform case and adjustment description for smoother writing.
Signed-off-by: ydFu <ader.ydfu@gmail.com>
The `/var/lib/kubelet/pod-resources/kubelet.sock` is required by device monitoring agent but not device plugin.
This word `plugin` is ambiguous.
plugin -> device monitoring agent
- Translate reference/glossary/persistent-volume.md in Korean (#25474)
- Update outdated files in the dev-1.20-ko.1 branch (1) (#25576)
- Translate blog/dont-panic-kubernetes-and-docker into Korean (#25596)
- Update outdated files in the dev-1.20-ko.1 branch(3) (#25728)
- Ko: enhance tutorials//cluster-intro translation (#25754)
- Fix Outdated files in the dev-1.20-ko.1 branch (2) (#25765)
Co-authored-by: seokho-son <shsongist@gmail.com>
Co-authored-by: jmyung <jesang.myung@gmail.com>
Co-authored-by: Jerry Park <jaehwa@gmail.com>
Co-authored-by: santachopa <santachopa@naver.com>
Co-authored-by: SEUNGHYUN KO <kosehy@gmail.com>
Co-authored-by: June Yi <gochist@gmail.com>
Add containerd installation on Debian
write docker keyring in a seprate file
Co-authored-by: Tim Bannister <tim@scalefactory.com>
edit comment
Co-authored-by: Tim Bannister <tim@scalefactory.com>
remove sudo
Co-authored-by: Tim Bannister <tim@scalefactory.com>
The language "For all service accounts in the "qa" namespace" in the example is confusing namespaces and groups. Language fixed to disambiguate between group and namespace. An additional example provided which uses both the group ("dev") AND the namespace ("development") to further illustrate this point
There is no need to restart the kubelet as part of configuring kubeadm to set the cgroup driver. At this point in the setup instructions, the kubernetes cluster isn't even up and running yet, as kubeadm init hasn't been run. The previous step even says the kubelet is in a crashloop waiting for kubeadm.
The improvements include:
- timezone-aware date formatting
- if getJSON() errors during a build, we can now ignore it
- You can set Hugo configuration via environment variables even if
snake_case.
sync with english version
Update debug-running-pod.md
apply suggestions
Update debug-running-pod.md
sync the left part
Update debug-running-pod.md
apply suggestions
Adding instructions for how to validate kubectl binaries against checksum files (Linux, MacOS, Windows)
Updating links to download from https://dl.k8s.io/
Updating Linux-specific install instructions to use install command, and
macOS-specific instructions to chown root the install to provide a trusted
kubectl.
Adding note annotation around optional download instructions
Markdown updates
* Updating numbered lists to use markdown syntax ("1." for each entry), should make it easier to add and remove list items in future
* Adding some syntax highlighting to the command snippets
Correcting "PowerShell" spelling
fixes: https://github.com/kubernetes/website/issues/25040
Signed-off-by: Nate W <4453979+nate-double-u@users.noreply.github.com>
resond to PR comments
change publish date and respond to some comments
respond to some more comments
fix formatting
fix image links and respond to comment
respond to formatting comments
remove gks reference
fix metrics names
remove gcp
pr feedback
* All base64 commands need `-w0` argument or else the base64_encoded_ca bash variable will contain space chars (" ") where newlines were
* All sed command are missing final "/" at the end of the expression. Command fails with the following error
```bash
/bin/sed: -e expression #1, char 95: unterminated `s' command
```
The code example uses `preferredDuringScheduling...`, as opposed to
`requiredDuringScheduling...`. So the anti-affinity rule is a soft one. It
says that the pod _should_ not be scheduled on [...] rather than that the pod
_cannot_ be scheduled on [...].
command: ["start", "--host", "\$(MY_SERVICE_NAME)"]
The back slash causes an error. So removed it.
The error detail below
```
Error: rawResources failed to read Resources: YAML file [deployment.yaml] encounters a format error.
error converting YAML to JSON: yaml: line 18: found unknown escape character
```
```
kubectl version
Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.3", GitCommit:"1e11e4a2108024935ecfcb2912226cedeafd99df", GitTreeState:"clean", BuildDate:"2020-10-14T12:50:19Z", GoVersion:"go1.15.2", Compiler:"gc", Platform:"windows/amd64"}
Server Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.3", GitCommit:"1e11e4a2108024935ecfcb2912226cedeafd99df", GitTreeState:"clean", BuildDate:"2020-10-14T12:41:49Z", GoVersion:"go1.15.2", Compiler:"gc", Platform:"linux/amd64"}
```
sync with english version
Apply suggestions from code review
Co-authored-by: Qiming Teng <tengqim@cn.ibm.com>
Co-authored-by: Qiming Teng <tengqim@cn.ibm.com>
adjust the format to rearrange the numbered list which is not continue
Update expose-external-ip-address.md
Update expose-external-ip-address.md
Update expose-external-ip-address.md
indent by 3 spaces from line 51 to 57
The "Set up the Docker daemon" step fails because the initial files have not been created. I copied the step of creating of the /etc/docker/ folder from the CentOS/RHEL instructions to the Ubuntu Instructions
A missing newline after the last list item caused an important note about plugins being hidden to be append to an unrelated note about debugging nodes.
The buildkit-cli-for-kubectl project aims to provide a drop-in replacement for
`docker build` with compatible UX. Some users looking to migrate off
dockershim may find this CLI plugin useful.
Signed-off-by: Daniel Hiltgen <hiltgend@vmware.com>
The description talks about the server one way and the overview text talks about it a bit differently.
This change aligns them and make them easier to understand in my opinion.
Adds a `caution` note that SSH key pairs do not establish trust between
clients and servers. A secondary method is required to establish trust
between an SSH client and host server, such as fixed `known_hosts` file.
Clients which do not establish adequate trust are vulnerable to "man in
the middle" impersonation attacks.
Signed-off-by: Adam Kaplan <adam.kaplan@redhat.com>
For someone following the guide to setup a cluster, sudo is being used in the initial commands, I think it would great to carry this on till the last commands.
If the EBS volume is partitioned, you have to specify which partition to
mount or the mounting process will fail.
```
mount: /var/lib/kubelet/plugins/kubernetes.io/aws-ebs/mounts/vol-<volumeid>: wrong fs type, bad option, bad superblock on /dev/nvme3n1, missing codepage or helper program, or other error.
```
This PR adds a paragraph explaining the insecure by default nature of k8s secrets, and points users at the documentation to turn on encryption at rest and RBAC.
I think a second page needs to be created showing the correct combination of RBAC rules for various cases, which should eventually replace the link to the RBAC documentation.
Default for periodSeconds field of startupProbe resource is 10 seconds.
$ kubectl explain pod.spec.containers.startupProbe.periodSeconds
KIND: Pod
VERSION: v1
FIELD: periodSeconds <integer>
DESCRIPTION:
How often (in seconds) to perform the probe. Default to 10 seconds. Minimum
value is 1.
@@ -17,9 +17,9 @@ Los revisores harán todo lo posible para proporcionar toda la información nece
Para obtener más información sobre cómo contribuir a la documentación de Kubernetes, puede consultar:
* [Empezando a contribuir](https://kubernetes.io/docs/contribute/start/)
* [Visualizando sus cambios en su entorno local](http://kubernetes.io/docs/contribute/intermediate#view-your-changes-locally)
* [Utilizando las plantillas de las páginas](http://kubernetes.io/docs/contribute/style/page-content-types/)
* [Guía de estilo de la documentación](http://kubernetes.io/docs/contribute/style/style-guide/)
* [Visualizando sus cambios en su entorno local](https://kubernetes.io/docs/contribute/intermediate#view-your-changes-locally)
* [Utilizando las plantillas de las páginas](https://kubernetes.io/docs/contribute/style/page-content-types/)
* [Guía de estilo de la documentación](https://kubernetes.io/docs/contribute/style/style-guide/)
* [Traduciendo la documentación de Kubernetes](https://kubernetes.io/docs/contribute/localization/)
## Levantando el sitio web kubernetes.io en su entorno local con Docker
@@ -30,6 +30,17 @@ El método recomendado para levantar una copia local del sitio web kubernetes.io
> Si prefiere levantar el sitio web sin utilizar **Docker**, puede seguir las instrucciones disponibles en la sección [Levantando kubernetes.io en local con Hugo](#levantando-kubernetesio-en-local-con-hugo).
**`Nota`: Para el procedimiento de construir una imagen de Docker e iniciar el servidor.**
El sitio web de Kubernetes utiliza Docsy Hugo theme. Se sugiere que se instale si aún no se ha hecho, los **submódulos** y otras dependencias de herramientas de desarrollo ejecutando el siguiente comando de `git`:
```bash
# pull de los submódulos del repositorio
git submodule update --init --recursive --depth 1
```
Si identifica que `git` reconoce una cantidad innumerable de cambios nuevos en el proyecto, la forma más simple de solucionarlo es cerrando y volviendo a abrir el proyecto en el editor. Los submódulos son automáticamente detectados por `git`, pero los plugins usados por los editores pueden tener dificultades para ser cargados.
Una vez tenga Docker [configurado en su máquina](https://www.docker.com/get-started), puede construir la imagen de Docker `kubernetes-hugo` localmente ejecutando el siguiente comando en la raíz del repositorio:
```bash
@@ -73,4 +84,4 @@ La participación en la comunidad de Kubernetes está regulada por el [Código d
Kubernetes es posible gracias a la participación de la comunidad y la documentación es vital para facilitar el acceso al proyecto.
Agradecemos muchísimo sus contribuciones a nuestro sitio web y nuestra documentación.
Agradecemos muchísimo sus contribuciones a nuestro sitio web y nuestra documentación.
Bemvindos! Este repositório abriga todos os recursos necessários para criar o [site e documentação do Kubernetes](https://kubernetes.io/). Estamos muito satisfeitos por você querer contribuir!
Bem-vindos! Este repositório contém todos os recursos necessários para criar o [website e documentação do Kubernetes](https://kubernetes.io/). Estamos muito satisfeitos por você querer contribuir!
## Contribuindo com os documentos
# Utilizando este repositório
Você pode clicar no botão **Fork** na área superior direita da tela para criar uma cópia desse repositório na sua conta do GitHub. Esta cópia é chamada de *fork*. Faça as alterações desejadas no seu fork e, quando estiver pronto para enviar as alterações para nós, vá até o fork e crie uma nova solicitação de pull para nos informar sobre isso.
Você pode executar o website localmente utilizando o Hugo (versão Extended), ou você pode executa-ló em um container runtime. É altamente recomendável utilizar um container runtime, pois garante a consistência na implantação do website real.
Depois que seu **pull request** for criado, um revisor do Kubernetes assumirá a responsabilidade de fornecer um feedback claro e objetivo. Como proprietário do pull request, **é sua responsabilidade modificar seu pull request para abordar o feedback que foi fornecido a você pelo revisor do Kubernetes.** Observe também que você pode acabar tendo mais de um revisor do Kubernetes para fornecer seu feedback ou você pode acabar obtendo feedback de um revisor do Kubernetes que é diferente daquele originalmente designado para lhe fornecer feedback. Além disso, em alguns casos, um de seus revisores pode solicitar uma revisão técnica de um [revisor de tecnologia Kubernetes](https://github.com/kubernetes/website/wiki/Tech-reviewers) quando necessário. Os revisores farão o melhor para fornecer feedback em tempo hábil, mas o tempo de resposta pode variar de acordo com as circunstâncias.
## Pré-requisitos
Para usar este repositório, você precisa instalar:
- [npm](https://www.npmjs.com/)
- [Go](https://golang.org/)
- [Hugo (versão Extended)](https://gohugo.io/)
- Um container runtime, por exemplo [Docker](https://www.docker.com/).
Antes de você iniciar, instale as dependências, clone o repositório e navegue até o diretório:
O website do Kubernetes utiliza o [tema Docsy Hugo](https://github.com/google/docsy#readme). Mesmo se você planeje executar o website em um container, é altamente recomendado baixar os submódulos e outras dependências executando o seguinte comando:
```
# Baixar o submódulo Docsy
git submodule update --init --recursive --depth 1
```
## Executando o website usando um container
Para executar o build do website em um container, execute o comando abaixo para criar a imagem do container e executa-lá:
```
make container-image
make container-serve
```
Abra seu navegador em http://localhost:1313 para visualizar o website. Conforme você faz alterações nos arquivos fontes, o Hugo atualiza o website e força a atualização do navegador.
## Executando o website localmente utilizando o Hugo
Consulte a [documentação oficial do Hugo](https://gohugo.io/getting-started/installing/) para instruções de instalação do Hugo. Certifique-se de instalar a versão do Hugo especificada pela variável de ambiente `HUGO_VERSION` no arquivo [`netlify.toml`](netlify.toml#L9).
Para executar o build e testar o website localmente, execute:
```bash
# instalar dependências
npm ci
make serve
```
Isso iniciará localmente o Hugo na porta 1313. Abra o seu navegador em http://localhost:1313 para visualizar o website. Conforme você faz alterações nos arquivos fontes, o Hugo atualiza o website e força uma atualização no navegador.
## Construindo a página de referência da API
A página de referência da API localizada em `content/en/docs/reference/kubernetes-api` é construída a partir da especificação do Swagger utilizando https://github.com/kubernetes-sigs/reference-docs/tree/master/gen-resourcesdocs.
Siga os passos abaixo para atualizar a página de referência para uma nova versão do Kubernetes:
OBS: modifique o "v1.20" no exemplo a seguir pela versão a ser atualizada
1. Obter o submódulo `kubernetes-resources-reference`:
```
git submodule update --init --recursive --depth 1
```
2. Criar a nova versão da API no submódulo e adicionar à especificação do Swagger:
4. Ajustar os arquivos `toc.yaml` e `fields.yaml` para refletir as mudanças entre as duas versões.
5. Em seguida, gerar as páginas:
```
make api-reference
```
Você pode validar o resultado localmente gerando e disponibilizando o site a partir da imagem do container:
```
make container-image
make container-serve
```
Abra o seu navegador em http://localhost:1313/docs/reference/kubernetes-api/ para visualizar a página de referência da API.
6. Quando todas as mudanças forem refletidas nos arquivos de configuração `toc.yaml` e `fields.yaml`, crie um pull request com a nova página de referência de API.
## Troubleshooting
### error: failed to transform resource: TOCSS: failed to transform "scss/main.scss" (text/x-scss): this feature is not available in your current Hugo version
Por motivos técnicos, o Hugo é disponibilizado em dois conjuntos de binários. O website atual funciona apenas na versão **Hugo Extended**. Na [página de releases](https://github.com/gohugoio/hugo/releases) procure por arquivos com `extended` no nome. Para confirmar, execute `hugo version` e procure pela palavra `extended`.
### Troubleshooting macOS for too many open files
Se você executar o comando `make serve` no macOS e retornar o seguinte erro:
```
ERROR 2020/08/01 19:09:18 Error: listen tcp 127.0.0.1:1313: socket: too many open files
make: *** [serve] Error 1
```
Verifique o limite atual para arquivos abertos:
`launchctl limit maxfiles`
Em seguida, execute os seguintes comandos (adaptado de https://gist.github.com/tombigel/d503800a282fcadbee14b537735d202c):
```shell
#!/bin/sh
# Esse são os links do gist original, vinculados ao meu gists agora.
Esta solução funciona tanto para o MacOS Catalina quanto para o MacOS Mojave.
### Erro de "Out of Memory"
Se você executar o comando `make container-serve` e retornar o seguinte erro:
```
make: *** [container-serve] Error 137
```
Verifique a quantidade de memória disponível para o agente de execução de contêiner. No caso do Docker Desktop para macOS, abra o menu "Preferences..." -> "Resources..." e tente disponibilizar mais memória.
# Comunidade, discussão, contribuição e apoio
Saiba mais sobre a comunidade Kubernetes SIG Docs e reuniões na [página da comunidade](http://kubernetes.io/community/).
Você também pode entrar em contato com os mantenedores deste projeto em:
- [Slack](https://kubernetes.slack.com/messages/sig-docs) ([Obter o convide para o este slack](https://slack.k8s.io/))
Você pode clicar no botão **Fork** na área superior direita da tela para criar uma cópia desse repositório na sua conta do GitHub. Esta cópia é chamada de *fork*. Faça as alterações desejadas no seu fork e, quando estiver pronto para enviar as alterações para nós, vá até o fork e crie um novo **pull request** para nos informar sobre isso.
Depois que seu **pull request** for criado, um revisor do Kubernetes assumirá a responsabilidade de fornecer um feedback claro e objetivo. Como proprietário do pull request, **é sua responsabilidade modificar seu pull request para atender ao feedback que foi fornecido a você pelo revisor do Kubernetes.**
Observe também que você pode acabar tendo mais de um revisor do Kubernetes para fornecer seu feedback ou você pode acabar obtendo feedback de um outro revisor do Kubernetes diferente daquele originalmente designado para lhe fornecer o feedback.
Além disso, em alguns casos, um de seus revisores pode solicitar uma revisão técnica de um [revisor técnico do Kubernetes](https://github.com/kubernetes/website/wiki/Tech-reviewers) quando necessário. Os revisores farão o melhor para fornecer feedbacks em tempo hábil, mas o tempo de resposta pode variar de acordo com as circunstâncias.
Para mais informações sobre como contribuir com a documentação do Kubernetes, consulte:
* [Comece a contribuir](https://kubernetes.io/docs/contribute/start/)
* [Preparando suas alterações na documentação](http://kubernetes.io/docs/contribute/intermediate#view-your-changes-locally)
* [Usando Modelos de Página](http://kubernetes.io/docs/contribute/style/page-templates/)
* [Contribua com a documentação do Kubernetes](https://kubernetes.io/docs/contribute/)
* [Tipos de conteúdo de página](https://kubernetes.io/docs/contribute/style/page-content-types/)
* [Guia de Estilo da Documentação](http://kubernetes.io/docs/contribute/style/style-guide/)
* [Localizando documentação do Kubernetes](https://kubernetes.io/docs/contribute/localization/)
Você pode contactar os mantenedores da localização em Português em:
Você pode contatar os mantenedores da localização em Português em:
* Felipe ([GitHub - @femrtnz](https://github.com/femrtnz))
A maneira recomendada de executar o site do Kubernetes localmente é executar uma imagem especializada do [Docker](https://docker.com) que inclui o gerador de site estático [Hugo](https://gohugo.io).
> Se você está rodando no Windows, você precisará de mais algumas ferramentas que você pode instalar com o [Chocolatey](https://chocolatey.org). `choco install make`
> Se você preferir executar o site localmente sem o Docker, consulte [Executando o site localmente usando o Hugo](#executando-o-site-localmente-usando-o-hugo) abaixo.
Se você tiver o Docker [em funcionamento](https://www.docker.com/get-started), crie a imagem do Docker do `kubernetes-hugo` localmente:
```bash
make container-image
```
Depois que a imagem foi criada, você pode executar o site localmente:
```bash
make container-serve
```
Abra seu navegador para http://localhost:1313 para visualizar o site. Conforme você faz alterações nos arquivos de origem, Hugo atualiza o site e força a atualização do navegador.
## Executando o site localmente usando o Hugo
Veja a [documentação oficial do Hugo](https://gohugo.io/getting-started/installing/) para instruções de instalação do Hugo. Certifique-se de instalar a versão do Hugo especificada pela variável de ambiente `HUGO_VERSION` no arquivo [`netlify.toml`](netlify.toml#L9).
Para executar o site localmente quando você tiver o Hugo instalado:
```bash
make serve
```
Isso iniciará o servidor Hugo local na porta 1313. Abra o navegador para http://localhost:1313 para visualizar o site. Conforme você faz alterações nos arquivos de origem, Hugo atualiza o site e força a atualização do navegador.
## Comunidade, discussão, contribuição e apoio
Aprenda a se envolver com a comunidade do Kubernetes na [página da comunidade](http://kubernetes.io/community/).
Você pode falar com os mantenedores deste projeto:
Добро пожаловать! Данный репозиторий содержит все необходимые файлы для сборки [сайта Kubernetes и документации](https://kubernetes.io/). Мы благодарим вас за старания!
Данный репозиторий содержит все необходимые файлы для сборки [сайта Kubernetes и документации](https://kubernetes.io/). Мы благодарим вас за желание внести свой вклад!
## Запуск сайта с помощью Hugo
# Использование этого репозитория
Обратитесь к [официальной документации Hugo](https://gohugo.io/getting-started/installing/), чтобы установить Hugo. Убедитесь, что вы установили правильную версию Hugo, которая устанавливается в переменной окружения `HUGO_VERSION` в файле [`netlify.toml`](netlify.toml#L10).
Запустить сайт локально можно с помощью Hugo (Extended version) или же в исполняемой среде для контейнеров. Мы настоятельно рекомендуем воспользоваться контейнерной средой, поскольку она обеспечивает консистивность развёртывания с оригинальным сайтом.
После установки Hugo, чтобы запустить сайт, выполните в консоли:
## Предварительные требования
```bash
Чтобы работать с этим репозиторием, понадобятся следующие компоненты, установленные локально:
- [npm](https://www.npmjs.com/)
- [Go](https://golang.org/)
- [Hugo (Extended version)](https://gohugo.io/)
- Исполняемая среда для контейнеров вроде [Docker](https://www.docker.com/)
Перед тем, как начать, установите зависимости. Склонируйте репозиторий и перейдите в его директорию:
Эта команда запустит сервер Hugo на порту 1313. Откройте браузер и перейдите по ссылке http://localhost:1313, чтобы открыть сайт. Если вы отредактируете исходные файлы сайта, Hugo автоматически применит изменения и обновит страницу в браузере.
Сайт Kubernetes использует [тему для Hugo под названием Docsy](https://github.com/google/docsy). Даже если вы планируете запускать сайт в контейнере, мы настоятельно рекомендуем загрузить соответствующий подмодуль и другие зависимости для разработки, выполнив следующую команду:
## Сообщество, обсуждение, вклад и поддержка
```
# загружаем Git-подмодуль Docsy
git submodule update --init --recursive --depth 1
```
Узнайте, как поучаствовать в жизни сообщества Kubernetes на [странице сообщества](http://kubernetes.io/community/).
## Запуск сайта в контейнере
Вы можете связаться с сопровождающими этого проекта по следующим ссылкам:
Чтобы собрать сайт в контейнере, выполните следующие команды — они собирают образ контейнера и запускают его:
- [Канал в Slack](https://kubernetes.slack.com/messages/sig-docs)
Откройте браузер и перейдите по ссылке http://localhost:1313, чтобы увидеть сайт. Если вы отредактируете исходные файлы сайта, Hugo автоматически обновит сам сайт и выполнит обновление страницы в браузере.
## Запуск сайта с помощью Hugo
Нажмите на кнопку **Fork** в правом верхнем углу, чтобы создать копию этого репозитория в ваш GitHub-аккаунт. Эта копия называется *форк-репозиторием*. Делайте любые изменения в вашем форк-репозитории, и когда вы будете готовы опубликовать изменения, откройте форк-репозиторий и создайте новый пулреквест, чтобы уведомить нас.
Убедитесь, что вы установили расширенную версию Hugo (extended version): она определена в переменной окружения `HUGO_VERSION` в файле [`netlify.toml`](netlify.toml#L10).
После того, как вы отправите пулреквест, ревьювер Kubernetes даст по нему обратную связь. Вы, как автор пулреквеста, **должны обновить свой пулреквест после его рассмотрения ревьювером Kubernetes.**
Чтобы собрать и протестировать сайт локально, выполните:
Вполне возможно, что более одного ревьювера Kubernetes оставят свои комментарии или даже может быть так, что новый комментарий ревьювера Kubernetes будет отличаться от первоначального назначенного ревьювера. Кроме того, в некоторых случаях один из ревьюверов может запросить технический обзор у [технического ревьювера Kubernetes](https://github.com/kubernetes/website/wiki/Tech-reviewers), если это будет необходимо. Ревьюверы сделают все возможное, чтобы как можно оперативно оставить свои предложения и пожелания, но время ответа может варьироваться в зависимости от обстоятельств.
```bash
# install dependencies
npm ci
make serve
```
Эти команды запустят локальный сервер Hugo на порту 1313. Откройте браузер и перейдите по ссылке http://localhost:1313, чтобы увидеть сайт. Если вы отредактируете исходные файлы сайта, Hugo автоматически обновит сам сайт и выполнит обновление страницы в браузере.
## Решение проблем
### error: failed to transform resource: TOCSS: failed to transform "scss/main.scss" (text/x-scss): this feature is not available in your current Hugo version
По техническим причинам Hugo поставляется с двумя наборами бинарников. Текущий сайт Kubernetes работает только в версии **Hugo Extended**. На [странице релизов](https://github.com/gohugoio/hugo/releases) ищите архивы со словом `extended` в названии. Чтобы убедиться в корректности, запустите команду `hugo version` и найдите в выводе слово `extended`.
### Решение проблемы на macOS с "too many open files"
Если вы запускаете `make serve` на macOS и получаете следующую ошибку:
```
ERROR 2020/08/01 19:09:18 Error: listen tcp 127.0.0.1:1313: socket: too many open files
make: *** [serve] Error 1
```
Попробуйте проверить текущий лимит для открытых файлов:
`launchctl limit maxfiles`
Затем выполните следующие команды (они взяты и адаптированы из https://gist.github.com/tombigel/d503800a282fcadbee14b537735d202c):
```shell
#!/bin/sh
# Ссылки на оригинальные gist-файлы закомментированы в пользу моих адаптированных.
Данное решение работает для macOS Catalina и Mojave.
# Участие в SIG Docs
Узнайте о Kubernetes-сообществе SIG Docs и его встречах на [странице сообщества](https://github.com/kubernetes/community/tree/master/sig-docs#meetings).
Вы можете связаться с сопровождающими этот проект по следующим ссылкам:
- [Канал в Slack](https://kubernetes.slack.com/messages/sig-docs) ([получите приглашение в этот Slack](https://slack.k8s.io/))
Нажмите на кнопку **Fork** в правом верхнем углу, чтобы создать копию этого репозитория для вашего GitHub-аккаунта. Эта копия называется *форк-репозиторием*. Делайте любые изменения в своем форк-репозитории и, когда будете готовы опубликовать изменения, зайдите в свой форк-репозиторий и создайте новый pull-запрос (PR), чтобы уведомить нас.
После того, как вы отправите pull-запрос, ревьювер из проекта Kubernetes даст по нему обратную связь. Вы, как автор pull-запроса, **должны обновить свой PR после его рассмотрения ревьювером Kubernetes.**
Вполне возможно, что более одного ревьювера Kubernetes оставят свои комментарии. Может быть даже так, что вы будете получать обратную связь уже не от того ревьювера, что был первоначально вам назначен. Кроме того, в некоторых случаях один из ревьюверов может запросить техническую рецензию от [технического ревьювера Kubernetes](https://github.com/kubernetes/website/wiki/Tech-reviewers), если это потребуется. Ревьюверы сделают все возможное, чтобы как можно оперативнее оставить свои предложения и пожелания, но время ответа может варьироваться в зависимости от обстоятельств.
Узнать подробнее о том, как поучаствовать в документации Kubernetes, вы можете по ссылкам ниже:
@@ -42,21 +121,22 @@ hugo server --buildFuture
* [Руководство по оформлению документации](https://kubernetes.io/docs/contribute/style/style-guide/)
* [Руководство по локализации Kubernetes](https://kubernetes.io/docs/contribute/localization/)
Участие в сообществе Kubernetes регулируется [кодексом поведения CNCF](https://github.com/cncf/foundation/blob/master/code-of-conduct.md).
Участие в сообществе Kubernetes регулируется [кодексом поведения CNCF](https://github.com/cncf/foundation/blob/master/code-of-conduct-languages/ru.md).
## Спасибо!
# Спасибо!
Kubernetes процветает благодаря сообществу и мы ценим ваш вклад в сайт и документацию!
<!-- This will start the local Hugo server on port 1313. Open up your browser to http://localhost:1313 to view the website. As you make changes to the source files, Hugo updates the website and forces a browser refresh. -->
@@ -82,4 +83,4 @@ hugo server --buildFuture
## Дякуємо!
<!-- Kubernetes thrives on community participation, and we appreciate your contributions to our website and our documentation! -->
Долучення до спільноти - запорука успішного розвитку Kubernetes. Ми цінуємо ваш внесок у наш сайт і документацію!
Долучення до спільноти - запорука успішного розвитку Kubernetes. Ми цінуємо ваш внесок у наш сайт і документацію!
This repository contains the assets required to build the [Kubernetes website and documentation](https://kubernetes.io/). We're glad that you want to contribute!
+ [Contributing to the docs](#contributing-to-the-docs)
+ [Localization ReadMes](#localization-readmemds)
# Using this repository
You can run the website locally using Hugo (Extended version), or you can run it in a container runtime. We strongly recommend using the container runtime, as it gives deployment consistency with the live website.
@@ -40,6 +43,8 @@ make container-image
make container-serve
```
If you see errors, it probably means that the hugo container did not have enough computing resources available. To solve it, increase the amount of allowed CPU and memory usage for Docker on your machine ([MacOSX](https://docs.docker.com/docker-for-mac/#resources) and [Windows](https://docs.docker.com/docker-for-windows/#resources)).
Open up your browser to http://localhost:1313 to view the website. As you make changes to the source files, Hugo updates the website and forces a browser refresh.
## Running the website locally using Hugo
@@ -56,6 +61,51 @@ make serve
This will start the local Hugo server on port 1313. Open up your browser to http://localhost:1313 to view the website. As you make changes to the source files, Hugo updates the website and forces a browser refresh.
## Building the API reference pages
The API reference pages located in `content/en/docs/reference/kubernetes-api` are built from the Swagger specification, using https://github.com/kubernetes-sigs/reference-docs/tree/master/gen-resourcesdocs.
To update the reference pages for a new Kubernetes release (replace v1.20 in the following examples with the release to update to):
1. Pull the `kubernetes-resources-reference` submodule:
```
git submodule update --init --recursive --depth 1
```
2. Create a new API revision into the submodule, and add the Swagger specification:
4. Adapt the files `toc.yaml` and `fields.yaml` to reflect the changes between the two releases
5. Next, build the pages:
```
make api-reference
```
You can test the results locally by making and serving the site from a container image:
```
make container-image
make container-serve
```
In a web browser, go to http://localhost:1313/docs/reference/kubernetes-api/ to view the API reference.
6. When all changes of the new contract are reflected into the configuration files `toc.yaml` and `fields.yaml`, create a Pull Request with the newly generated API reference pages.
## Troubleshooting
### error: failed to transform resource: TOCSS: failed to transform "scss/main.scss" (text/x-scss): this feature is not available in your current Hugo version
@@ -76,7 +126,7 @@ Try checking the current limit for open files:
Then run the following commands (adapted from https://gist.github.com/tombigel/d503800a282fcadbee14b537735d202c):
```
```shell
#!/bin/sh
# These are the original gist links, linking to my gists now.
<h3>Die Gewissheit, dass Kubernetes überall und für alle gut funktioniert.</h3>
<p>Verbinden Sie sich mit der Kubernetes-Community in unserem <ahref="http://slack.k8s.io/">Slack Kanal</a>, <ahref="https://discuss.kubernetes.io/">Diskussionsforum</a>, oder beteiligen Sie sich an der <ahref="https://groups.google.com/forum/#!forum/kubernetes-dev"> Kubernetes-dev-Google-Gruppe</a>. Eine wöchentlichesCommunity-Meeting findet per Videokonferenz statt, um den Stand der Dinge zu diskutieren, folgen Sie
<p>Verbinden Sie sich mit der Kubernetes-Community in unserem <ahref="http://slack.k8s.io/">Slack Kanal</a>, <ahref="https://discuss.kubernetes.io/">Diskussionsforum</a>, oder beteiligen Sie sich an der <ahref="https://groups.google.com/g/kubernetes-dev"> Kubernetes-dev-Google-Gruppe</a>. Eine wöchentlichesCommunity-Meeting findet per Videokonferenz statt, um den Stand der Dinge zu diskutieren, folgen Sie
<ahref="https://github.com/kubernetes/community/blob/master/events/community-meeting.md">diesen Anweisungen</a> für Informationen wie Sie teilnehmen können.</p>
<p>Sie können Kubernetes auch auf der ganzen Welt über unsere
<ahref="https://www.meetup.com/topics/kubernetes/">Kubernetes Meetup Community</a> und der
@@ -23,7 +23,7 @@ Dieser Verhaltenskodex gilt sowohl innerhalb von Projekträumen als auch in öff
Fälle von missbräuchlichem, belästigendem oder anderweitig unzumutbarem Verhalten in Kubernetes können gemeldet werden, indem Sie sich an das [Kubernetes Komitee für Verhaltenskodex](https://git.k8s.io/community/committee-code-of-conduct) wenden unter <conduct@kubernetes.io>. Für andere Projekte wenden Sie sich bitte an einen CNCF-Projektbetreuer oder an unseren Mediator, Mishi Choudhary <mishi@linux.com>.
Dieser Verhaltenskodex wurde aus dem Contributor Covenant übernommen (http://contributor-covenant.org), Version 1.2.0, verfügbar unter http://contributor-covenant.org/version/1/2/0/
Dieser Verhaltenskodex wurde aus dem Contributor Covenant übernommen (https://contributor-covenant.org), Version 1.2.0, verfügbar unter https://contributor-covenant.org/version/1/2/0/
@@ -26,7 +26,7 @@ On the other hand, CNI is more philosophically aligned with Kubernetes. It's far
Additionally, it's trivial to wrap a CNI plugin and produce a more customized CNI plugin — it can be done with a simple shell script. CNM is much more complex in this regard. This makes CNI an attractive option for rapid development and iteration. Early prototypes have proven that it's possible to eject almost 100% of the currently hard-coded network logic in kubelet into a plugin.
We investigated [writing a "bridge" CNM driver](https://groups.google.com/forum/#!topic/kubernetes-sig-network/5MWRPxsURUw) for Docker that ran CNI drivers. This turned out to be very complicated. First, the CNM and CNI models are very different, so none of the "methods" lined up. We still have the global vs. local and key-value issues discussed above. Assuming this driver would declare itself local, we have to get info about logical networks from Kubernetes.
We investigated [writing a "bridge" CNM driver](https://groups.google.com/g/kubernetes-sig-network/c/5MWRPxsURUw) for Docker that ran CNI drivers. This turned out to be very complicated. First, the CNM and CNI models are very different, so none of the "methods" lined up. We still have the global vs. local and key-value issues discussed above. Assuming this driver would declare itself local, we have to get info about logical networks from Kubernetes.
Unfortunately, Docker drivers are hard to map to other control planes like Kubernetes. Specifically, drivers are not told the name of the network to which a container is being attached — just an ID that Docker allocates internally. This makes it hard for a driver to map back to any concept of network that exists in another system.
@@ -34,6 +34,6 @@ This and other issues have been brought up to Docker developers by network vendo
For all of these reasons we have chosen to invest in CNI as the Kubernetes plugin model. There will be some unfortunate side-effects of this. Most of them are relatively minor (for example, `docker inspect` will not show an IP address), but some are significant. In particular, containers started by `docker run` might not be able to communicate with containers started by Kubernetes, and network integrators will have to provide CNI drivers if they want to fully integrate with Kubernetes. On the other hand, Kubernetes will get simpler and more flexible, and a lot of the ugliness of early bootstrapping (such as configuring Docker to use our bridge) will go away.
As we proceed down this path, we’ll certainly keep our eyes and ears open for better ways to integrate and simplify. If you have thoughts on how we can do that, we really would like to hear them — find us on [slack](http://slack.k8s.io/) or on our [network SIG mailing-list](https://groups.google.com/forum/#!forum/kubernetes-sig-network).
As we proceed down this path, we’ll certainly keep our eyes and ears open for better ways to integrate and simplify. If you have thoughts on how we can do that, we really would like to hear them — find us on [slack](http://slack.k8s.io/) or on our [network SIG mailing-list](https://groups.google.com/g/kubernetes-sig-network).
@@ -20,21 +20,14 @@ For example, if we want to require scheduling on a node that is in the us-centra
```
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: "failure-domain.beta.kubernetes.io/zone"
operator: In
values: ["us-central1-a"]
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: "failure-domain.beta.kubernetes.io/zone"
operator: In
values: ["us-central1-a"]
```
@@ -44,21 +37,14 @@ Preferred rules mean that if nodes match the rules, they will be chosen first, a
```
affinity:
nodeAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: "failure-domain.beta.kubernetes.io/zone"
operator: In
values: ["us-central1-a"]
affinity:
nodeAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: "failure-domain.beta.kubernetes.io/zone"
operator: In
values: ["us-central1-a"]
```
@@ -67,21 +53,14 @@ Node anti-affinity can be achieved by using negative operators. So for instance
```
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: "failure-domain.beta.kubernetes.io/zone"
operator: NotIn
values: ["us-central1-a"]
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: "failure-domain.beta.kubernetes.io/zone"
operator: NotIn
values: ["us-central1-a"]
```
@@ -99,7 +78,7 @@ The kubectl command allows you to set taints on nodes, for example:
```
kubectl taint nodes node1 key=value:NoSchedule
```
```
creates a taint that marks the node as unschedulable by any pods that do not have a toleration for taint with key key, value value, and effect NoSchedule. (The other taint effects are PreferNoSchedule, which is the preferred version of NoSchedule, and NoExecute, which means any pods that are running on the node when the taint is applied will be evicted unless they tolerate the taint.) The toleration you would add to a PodSpec to have the corresponding pod tolerate this taint would look like this
@@ -107,15 +86,11 @@ creates a taint that marks the node as unschedulable by any pods that do not hav
```
tolerations:
- key: "key"
operator: "Equal"
value: "value"
effect: "NoSchedule"
tolerations:
- key: "key"
operator: "Equal"
value: "value"
effect: "NoSchedule"
```
@@ -138,21 +113,13 @@ Let’s look at an example. Say you have front-ends in service S1, and they comm
@@ -56,13 +56,13 @@ Cri-containerd uses containerd to manage the full container lifecycle and all co
Let’s use an example to demonstrate how cri-containerd works for the case when Kubelet creates a single-container pod:
1.1.Kubelet calls cri-containerd, via the CRI runtime service API, to create a pod;
2.2.cri-containerd uses containerd to create and start a special [pause container](https://www.ianlewis.org/en/almighty-pause-container) (the _sandbox container_) and put that container inside the pod’s cgroups and namespace (steps omitted for brevity);
3.3.cri-containerd configures the pod’s network namespace using CNI;
4.4.Kubelet subsequently calls cri-containerd, via the CRI image service API, to pull the application container image;
5.5.cri-containerd further uses containerd to pull the image if the image is not present on the node;
6.6.Kubelet then calls cri-containerd, via the CRI runtime service API, to create and start the application container inside the pod using the pulled container image;
7.7.cri-containerd finally calls containerd to create the application container, put it inside the pod’s cgroups and namespace, then to start the pod’s new application container.
1. Kubelet calls cri-containerd, via the CRI runtime service API, to create a pod;
2. cri-containerd uses containerd to create and start a special [pause container](https://www.ianlewis.org/en/almighty-pause-container) (the _sandbox container_) and put that container inside the pod’s cgroups and namespace (steps omitted for brevity);
3. cri-containerd configures the pod’s network namespace using CNI;
4. Kubelet subsequently calls cri-containerd, via the CRI image service API, to pull the application container image;
5. cri-containerd further uses containerd to pull the image if the image is not present on the node;
6. Kubelet then calls cri-containerd, via the CRI runtime service API, to create and start the application container inside the pod using the pulled container image;
7. cri-containerd finally calls containerd to create the application container, put it inside the pod’s cgroups and namespace, then to start the pod’s new application container.
After these steps, a pod and its corresponding application container is created and running.
@@ -176,7 +176,7 @@ Cluster-distributed stateful services (e.g., Cassandra) can benefit from splitti
[Logs](/docs/concepts/cluster-administration/logging/) and [metrics](/docs/tasks/debug-application-cluster/resource-usage-monitoring/) (if collected and persistently retained) are valuable to diagnose outages, but given the variety of technologies available it will not be addressed in this blog. If Internet connectivity is available, it may be desirable to retain logs and metrics externally at a central location.
Your production deployment should utilize an automated installation, configuration and update tool (e.g., [Ansible](https://github.com/kubernetes-incubator/kubespray), [BOSH](https://github.com/cloudfoundry-incubator/kubo-deployment), [Chef](https://github.com/chef-cookbooks/kubernetes), [Juju](/docs/getting-started-guides/ubuntu/installation/), [kubeadm](/docs/reference/setup-tools/kubeadm/kubeadm/), [Puppet](https://forge.puppet.com/puppetlabs/kubernetes), etc.). A manual process will have repeatability issues, be labor intensive, error prone, and difficult to scale. [Certified distributions](https://www.cncf.io/certification/software-conformance/#logos) are likely to include a facility for retaining configuration settings across updates, but if you implement your own install and config toolchain, then retention, backup and recovery of the configuration artifacts is essential. Consider keeping your deployment components and settings under a version control system such as Git.
Your production deployment should utilize an automated installation, configuration and update tool (e.g., [Ansible](https://github.com/kubernetes-incubator/kubespray), [BOSH](https://github.com/cloudfoundry-incubator/kubo-deployment), [Chef](https://github.com/chef-cookbooks/kubernetes), [Juju](/docs/getting-started-guides/ubuntu/installation/), [kubeadm](/docs/reference/setup-tools/kubeadm/), [Puppet](https://forge.puppet.com/puppetlabs/kubernetes), etc.). A manual process will have repeatability issues, be labor intensive, error prone, and difficult to scale. [Certified distributions](https://www.cncf.io/certification/software-conformance/#logos) are likely to include a facility for retaining configuration settings across updates, but if you implement your own install and config toolchain, then retention, backup and recovery of the configuration artifacts is essential. Consider keeping your deployment components and settings under a version control system such as Git.
@@ -17,7 +17,7 @@ Let’s dive into the key features of this release:
## Simplified Kubernetes Cluster Management with kubeadm in GA
Most people who have gotten hands-on with Kubernetes have at some point been hands-on with kubeadm. It's an essential tool for managing the cluster lifecycle, from creation to configuration to upgrade; and now kubeadm is officially GA. [kubeadm](/docs/reference/setup-tools/kubeadm/kubeadm/) handles the bootstrapping of production clusters on existing hardware and configuring the core Kubernetes components in a best-practice-manner to providing a secure yet easy joining flow for new nodes and supporting easy upgrades. What’s notable about this GA release are the now graduated advanced features, specifically around pluggability and configurability. The scope of kubeadm is to be a toolbox for both admins and automated, higher-level system and this release is a significant step in that direction.
Most people who have gotten hands-on with Kubernetes have at some point been hands-on with kubeadm. It's an essential tool for managing the cluster lifecycle, from creation to configuration to upgrade; and now kubeadm is officially GA. [kubeadm](/docs/reference/setup-tools/kubeadm/) handles the bootstrapping of production clusters on existing hardware and configuring the core Kubernetes components in a best-practice-manner to providing a secure yet easy joining flow for new nodes and supporting easy upgrades. What’s notable about this GA release are the now graduated advanced features, specifically around pluggability and configurability. The scope of kubeadm is to be a toolbox for both admins and automated, higher-level system and this release is a significant step in that direction.
title: "A Custom Kubernetes Scheduler to Orchestrate Highly Available Applications"
date: 2020-12-21
slug: writing-crl-scheduler
---
**Author**: Chris Seto (Cockroach Labs)
As long as you're willing to follow the rules, deploying on Kubernetes and air travel can be quite pleasant. More often than not, things will "just work". However, if one is interested in travelling with an alligator that must remain alive or scaling a database that must remain available, the situation is likely to become a bit more complicated. It may even be easier to build one's own plane or database for that matter. Travelling with reptiles aside, scaling a highly available stateful system is no trivial task.
Scaling any system has two main components:
1. Adding or removing infrastructure that the system will run on, and
2. Ensuring that the system knows how to handle additional instances of itself being added and removed.
Most stateless systems, web servers for example, are created without the need to be aware of peers. Stateful systems, which includes databases like CockroachDB, have to coordinate with their peer instances and shuffle around data. As luck would have it, CockroachDB handles data redistribution and replication. The tricky part is being able to tolerate failures during these operations by ensuring that data and instances are distributed across many failure domains (availability zones).
One of Kubernetes' responsibilities is to place "resources" (e.g, a disk or container) into the cluster and satisfy the constraints they request. For example: "I must be in availability zone _A_" (see [Running in multiple zones](/docs/setup/best-practices/multiple-zones/#nodes-are-labeled)), or "I can't be placed onto the same node as this other Pod" (see [Affinity and anti-affinity](/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity)).
As an addition to those constraints, Kubernetes offers [Statefulsets](/docs/concepts/workloads/controllers/statefulset/) that provide identity to Pods as well as persistent storage that "follows" these identified pods. Identity in a StatefulSet is handled by an increasing integer at the end of a pod's name. It's important to note that this integer must always be contiguous: in a StatefulSet, if pods 1 and 3 exist then pod 2 must also exist.
Under the hood, CockroachCloud deploys each region of CockroachDB as a StatefulSet in its own Kubernetes cluster - see [Orchestrate CockroachDB in a Single Kubernetes Cluster](https://www.cockroachlabs.com/docs/stable/orchestrate-cockroachdb-with-kubernetes.html).
In this article, I'll be looking at an individual region, one StatefulSet and one Kubernetes cluster which is distributed across at least three availability zones.
A three-node CockroachCloud cluster would look something like this:
When adding additional resources to the cluster we also distribute them across zones. For the speediest user experience, we add all Kubernetes nodes at the same time and then scale up the StatefulSet.

Note that anti-affinities are satisfied no matter the order in which pods are assigned to Kubernetes nodes. In the example, pods 0, 1 and 2 were assigned to zones A, B, and C respectively, but pods 3 and 4 were assigned in a different order, to zones B and A respectively. The anti-affinity is still satisfied because the pods are still placed in different zones.
To remove resources from a cluster, we perform these operations in reverse order.
We first scale down the StatefulSet and then remove from the cluster any nodes lacking a CockroachDB pod.

Now, remember that pods in a StatefulSet of size _n_ must have ids in the range `[0,n)`. When scaling down a StatefulSet by _m_, Kubernetes removes _m_ pods, starting from the highest ordinals and moving towards the lowest, [the reverse in which they were added](/docs/concepts/workloads/controllers/statefulset/#deployment-and-scaling-guarantees).
Consider the cluster topology below:

As ordinals 5 through 3 are removed from this cluster, the statefulset continues to have a presence across all 3 availability zones.

However, Kubernetes' scheduler doesn't _guarantee_ the placement above as we expected at first.
Our combined knowledge of the following is what lead to this misconception.
* Kubernetes' ability to [automatically spread Pods across zone](/docs/setup/best-practices/multiple-zones/#pods-are-spread-across-zones)
* The behavior that a StatefulSet with _n_ replicas, when Pods are being deployed, they are created sequentially, in order from `{0..n-1}`. See [StatefulSet](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#deployment-and-scaling-guarantees) for more details.
Consider the following topology:

These pods were created in order and they are spread across all availability zones in the cluster. When ordinals 5 through 3 are terminated, this cluster will lose its presence in zone C!

Worse yet, our automation, at the time, would remove Nodes A-2, B-2, and C-2. Leaving CRDB-1 in an unscheduled state as persistent volumes are only available in the zone they are initially created in.
To correct the latter issue, we now employ a "hunt and peck" approach to removing machines from a cluster. Rather than blindly removing Kubernetes nodes from the cluster, only nodes without a CockroachDB pod would be removed. The much more daunting task was to wrangle the Kubernetes scheduler.
## A session of brainstorming left us with 3 options:
### 1. Upgrade to kubernetes 1.18 and make use of Pod Topology Spread Constraints
While this seems like it could have been the perfect solution, at the time of writing Kubernetes 1.18 was unavailable on the two most common managed Kubernetes services in public cloud, EKS and GKE.
Furthermore, [pod topology spread constraints](/docs/concepts/workloads/pods/pod-topology-spread-constraints/) were still a [beta feature in 1.18](https://v1-18.docs.kubernetes.io/docs/concepts/workloads/pods/pod-topology-spread-constraints/) which meant that it [wasn't guaranteed to be available in managed clusters](https://cloud.google.com/kubernetes-engine/docs/concepts/types-of-clusters#kubernetes_feature_choices) even when v1.18 became available.
The entire endeavour was concerningly reminiscent of checking [caniuse.com](https://caniuse.com/) when Internet Explorer 8 was still around.
### 2. Deploy a statefulset _per zone_.
Rather than having one StatefulSet distributed across all availability zones, a single StatefulSet with node affinities per zone would allow manual control over our zonal topology.
Our team had considered this as an option in the past which made it particularly appealing.
Ultimately, we decided to forego this option as it would have required a massive overhaul to our codebase and performing the migration on existing customer clusters would have been an equally large undertaking.
### 3. Write a custom Kubernetes scheduler.
Thanks to an example from [Kelsey Hightower](https://github.com/kelseyhightower/scheduler) and a blog post from [Banzai Cloud](https://banzaicloud.com/blog/k8s-custom-scheduler/), we decided to dive in head first and write our own [custom Kubernetes scheduler](/docs/tasks/extend-kubernetes/configure-multiple-schedulers/).
Once our proof-of-concept was deployed and running, we quickly discovered that the Kubernetes' scheduler is also responsible for mapping persistent volumes to the Pods that it schedules.
The output of [`kubectl get events`](/docs/tasks/extend-kubernetes/configure-multiple-schedulers/#verifying-that-the-pods-were-scheduled-using-the-desired-schedulers) had led us to believe there was another system at play.
In our journey to find the component responsible for storage claim mapping, we discovered the [kube-scheduler plugin system](/docs/concepts/scheduling-eviction/scheduling-framework/). Our next POC was a `Filter` plugin that determined the appropriate availability zone by pod ordinal, and it worked flawlessly!
Our [custom scheduler plugin](https://github.com/cockroachlabs/crl-scheduler) is open source and runs in all of our CockroachCloud clusters.
Having control over how our StatefulSet pods are being scheduled has let us scale out with confidence.
We may look into retiring our plugin once pod topology spread constraints are available in GKE and EKS, but the maintenance overhead has been surprisingly low.
Better still: the plugin's implementation is orthogonal to our business logic. Deploying it, or retiring it for that matter, is as simple as changing the `schedulerName` field in our StatefulSet definitions.
---
_[Chris Seto](https://twitter.com/_ostriches) is a software engineer at Cockroach Labs and works on their Kubernetes automation for [CockroachCloud](https://cockroachlabs.cloud), CockroachDB._
### What should I look out for when changing CRI implementations?
@@ -129,7 +129,7 @@ common things to consider when migrating are:
- Kubectl plugins that require docker CLI or the control socket
- Kubernetes tools that require direct access to Docker (e.g. kube-imagepuller)
- Configuration of functionality like `registry-mirrors` and insecure registries
- Other support scripts or daemons that expect docker to be available and are run
- Other support scripts or daemons that expect Docker to be available and are run
outside of Kubernetes (e.g. monitoring or security agents)
- GPUs or special hardware and how they integrate with your runtime and Kubernetes
@@ -140,14 +140,15 @@ runtime where possible.
Another thing to look out for is anything expecting to run for system maintenance
or nested inside a container when building images will no longer work. For the
former, you can use the [`crictl`][cr] tool as a drop-in replacement and for the
latter you can use newer container build options like [img], [buildah], or
[kaniko] that don’t require Docker.
former, you can use the [`crictl`][cr] tool as a drop-in replacement (see [mapping from docker cli to crictl](https://kubernetes.io/docs/tasks/debug-application-cluster/crictl/#mapping-from-docker-cli-to-crictl)) and for the
latter you can use newer container build options like [img], [buildah],
[kaniko], or [buildkit-cli-for-kubectl] that don’t require Docker.
We’re pleased to announce the release of Kubernetes 1.20, our third and final release of 2020! This release consists of 42 enhancements: 11 enhancements have graduated to stable, 15 enhancements are moving to beta, and 16 enhancements are entering alpha.
The 1.20 release cycle returned to its normal cadence of 11 weeks following the previous extended release cycle. This is one of the most feature dense releases in a while: the Kubernetes innovation cycle is still trending upward. This release has more alpha than stable enhancements, showing that there is still much to explore in the cloud native ecosystem.
## Major Themes
### Volume Snapshot Operations Goes Stable
This feature provides a standard way to trigger volume snapshot operations and allows users to incorporate snapshot operations in a portable manner on any Kubernetes environment and supported storage providers.
Additionally, these Kubernetes snapshot primitives act as basic building blocks that unlock the ability to develop advanced, enterprise-grade, storage administration features for Kubernetes, including application or cluster level backup solutions.
Note that snapshot support requires Kubernetes distributors to bundle the Snapshot controller, Snapshot CRDs, and validation webhook. A CSI driver supporting the snapshot functionality must also be deployed on the cluster.
### Kubectl Debug Graduates to Beta
The `kubectl alpha debug` features graduates to beta in 1.20, becoming `kubectl debug`. The feature provides support for common debugging workflows directly from kubectl. Troubleshooting scenarios supported in this release of kubectl include:
* Troubleshoot workloads that crash on startup by creating a copy of the pod that uses a different container image or command.
* Troubleshoot distroless containers by adding a new container with debugging tools, either in a new copy of the pod or using an ephemeral container. (Ephemeral containers are an alpha feature that are not enabled by default.)
* Troubleshoot on a node by creating a container running in the host namespaces and with access to the host’s filesystem.
Note that as a new built-in command, `kubectl debug` takes priority over any kubectl plugin named “debug”. You must rename the affected plugin.
Invocations using `kubectl alpha debug` are now deprecated and will be removed in a subsequent release. Update your scripts to use `kubectl debug`. For more information about `kubectl debug`, see [Debugging Running Pods](https://kubernetes.io/docs/tasks/debug-application-cluster/debug-running-pod/).
### Beta: API Priority and Fairness
Introduced in 1.18, Kubernetes 1.20 now enables API Priority and Fairness (APF) by default. This allows `kube-apiserver` to categorize incoming requests by priority levels.
### Alpha with updates: IPV4/IPV6
The IPv4/IPv6 dual stack has been reimplemented to support dual stack services based on user and community feedback. This allows both IPv4 and IPv6 service cluster IP addresses to be assigned to a single service, and also enables a service to be transitioned from single to dual IP stack and vice versa.
### GA: Process PID Limiting for Stability
Process IDs (pids) are a fundamental resource on Linux hosts. It is trivial to hit the task limit without hitting any other resource limits and cause instability to a host machine.
Administrators require mechanisms to ensure that user pods cannot induce pid exhaustion that prevents host daemons (runtime, kubelet, etc) from running. In addition, it is important to ensure that pids are limited among pods in order to ensure they have limited impact to other workloads on the node.
After being enabled-by-default for a year, SIG Node graduates PID Limits to GA on both `SupportNodePidsLimit` (node-to-pod PID isolation) and `SupportPodPidsLimit` (ability to limit PIDs per pod).
### Alpha: Graceful node shutdown
Users and cluster administrators expect that pods will adhere to expected pod lifecycle including pod termination. Currently, when a node shuts down, pods do not follow the expected pod termination lifecycle and are not terminated gracefully which can cause issues for some workloads.
The `GracefulNodeShutdown` feature is now in Alpha. `GracefulNodeShutdown` makes the kubelet aware of node system shutdowns, enabling graceful termination of pods during a system shutdown.
## Major Changes
### Dockershim Deprecation
Dockershim, the container runtime interface (CRI) shim for Docker is being deprecated. Support for Docker is deprecated and will be removed in a future release. Docker-produced images will continue to work in your cluster with all CRI compliant runtimes as Docker images follow the Open Container Initiative (OCI) image specification.
The Kubernetes community has written a [detailed blog post about deprecation](https://blog.k8s.io/2020/12/02/dont-panic-kubernetes-and-docker/) with [a dedicated FAQ page for it](https://blog.k8s.io/2020/12/02/dockershim-faq/).
### Exec Probe Timeout Handling
A longstanding bug regarding exec probe timeouts that may impact existing pod definitions has been fixed. Prior to this fix, the field `timeoutSeconds` was not respected for exec probes. Instead, probes would run indefinitely, even past their configured deadline, until a result was returned. With this change, the default value of `1 second` will be applied if a value is not specified and existing pod definitions may no longer be sufficient if a probe takes longer than one second. A feature gate, called `ExecProbeTimeout`, has been added with this fix that enables cluster operators to revert to the previous behavior, but this will be locked and removed in subsequent releases. In order to revert to the previous behavior, cluster operators should set this feature gate to `false`.
Please review the updated documentation regarding [configuring probes](docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/#configure-probes) for more details.
You can check out the full details of the 1.20 release in the [release notes](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.20.md).
# Availability of release
Kubernetes 1.20 is available for [download on GitHub](https://github.com/kubernetes/kubernetes/releases/tag/v1.20.0). There are some great resources out there for getting started with Kubernetes. You can check out some [interactive tutorials](https://kubernetes.io/docs/tutorials/) on the main Kubernetes site, or run a local cluster on your machine using Docker containers with [kind](https://kind.sigs.k8s.io). If you’d like to try building a cluster from scratch, check out the [Kubernetes the Hard Way](https://github.com/kelseyhightower/kubernetes-the-hard-way) tutorial by Kelsey Hightower.
# Release Team
This release was made possible by a very dedicated group of individuals, who came together as a team in the midst of a lot of things happening out in the world. A huge thank you to the release lead Jeremy Rickard, and to everyone else on the release team for supporting each other, and working so hard to deliver the 1.20 release for the community.
> The Kubernetes 1.20 Release has been the raddest release yet.
2020 has been a challenging year for many of us, but Kubernetes contributors have delivered a record-breaking number of enhancements in this release. That is a great accomplishment, so the release lead wanted to end the year with a little bit of levity and pay homage to [Kubernetes 1.14 - Caturnetes](https://github.com/kubernetes/sig-release/tree/master/releases/release-1.14) with a "rad" cat named Humphrey.
Humphrey is the release lead's cat and has a permanent [`blep`](https://www.inverse.com/article/42316-why-do-cats-blep-science-explains). *Rad* was pretty common slang in the 1990s in the United States, and so were laser backgrounds. Humphrey in a 1990s style school picture felt like a fun way to end the year. Hopefully, Humphrey and his *blep* bring you a little joy at the end of 2020!
The release logo was created by [Henry Hsu - @robotdancebattle](https://www.instagram.com/robotdancebattle/).
# User Highlights
- Apple is operating multi-thousand node Kubernetes clusters in data centers all over the world. Watch [Alena Prokharchyk's KubeCon NA Keynote](https://youtu.be/Tx8qXC-U3KM) to learn more about their cloud native journey.
# Project Velocity
The [CNCF K8s DevStats project](https://k8s.devstats.cncf.io/) aggregates a number of interesting data points related to the velocity of Kubernetes and various sub-projects. This includes everything from individual contributions to the number of companies that are contributing, and is a neat illustration of the depth and breadth of effort that goes into evolving this ecosystem.
In the v1.20 release cycle, which ran for 11 weeks (September 25 to December 9), we saw contributions from [967 companies](https://k8s.devstats.cncf.io/d/9/companies-table?orgId=1&var-period_name=v1.19.0%20-%20now&var-metric=contributions) and [1335 individuals](https://k8s.devstats.cncf.io/d/66/developer-activity-counts-by-companies?orgId=1&var-period_name=v1.19.0%20-%20now&var-metric=contributions&var-repogroup_name=Kubernetes&var-country_name=All&var-companies=All) ([44 of whom](https://k8s.devstats.cncf.io/d/52/new-contributors?orgId=1&from=1601006400000&to=1607576399000&var-repogroup_name=Kubernetes) made their first Kubernetes contribution) from [26 countries](https://k8s.devstats.cncf.io/d/50/countries-stats?orgId=1&from=1601006400000&to=1607576399000&var-period_name=Quarter&var-countries=All&var-repogroup_name=Kubernetes&var-metric=rcommitters&var-cum=countries).
# Ecosystem Updates
- KubeCon North America just wrapped up three weeks ago, the second such event to be virtual! All talks are [now available to all on-demand](https://www.youtube.com/playlist?list=PLj6h78yzYM2Pn8RxfLh2qrXBDftr6Qjut) for anyone still needing to catch up!
- In June, the Kubernetes community formed a new working group as a direct response to the Black Lives Matter protests occurring across America. WG Naming's goal is to remove harmful and unclear language in the Kubernetes project as completely as possible and to do so in a way that is portable to other CNCF projects. A great introductory talk on this important work and how it is conducted was given [at KubeCon 2020 North America](https://sched.co/eukp), and the initial impact of this labor [can actually be seen in the v1.20 release](https://github.com/kubernetes/enhancements/issues/2067).
- Previously announced this summer, [The Certified Kubernetes Security Specialist (CKS) Certification](https://www.cncf.io/announcements/2020/11/17/kubernetes-security-specialist-certification-now-available/) was released during Kubecon NA for immediate scheduling! Following the model of CKA and CKAD, the CKS is a performance-based exam, focused on security-themed competencies and domains. This exam is targeted at current CKA holders, particularly those who want to round out their baseline knowledge in securing cloud workloads (which is all of us, right?).
# Event Updates
KubeCon + CloudNativeCon Europe 2021 will take place May 4 - 7, 2021! Registration will open on January 11. You can find more information about the conference [here](https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/). Remember that [the CFP](https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/program/cfp/) closes on Sunday, December 13, 11:59pm PST!
# Upcoming release webinar
Stay tuned for the upcoming release webinar happening this January.
# Get Involved
If you’re interested in contributing to the Kubernetes community, Special Interest Groups (SIGs) are a great starting point. Many of them may align with your interests! If there are things you’d like to share with the community, you can join the weekly community meeting, or use any of the following channels:
* Find out more about contributing to Kubernetes at the new [Kubernetes Contributor website](https://www.kubernetes.dev/)
* Follow us on Twitter [@Kubernetesio](https://twitter.com/kubernetesio) for latest updates
* Join the community discussion on [Discuss](https://discuss.kubernetes.io/)
* Join the community on [Slack](http://slack.k8s.io/)
* Share your Kubernetes [story](https://docs.google.com/a/linuxfoundation.org/forms/d/e/1FAIpQLScuI7Ye3VQHQTwBASrgkjQDSS5TP0g3AXfFhwSM9YpHgxRKFA/viewform)
* Read more about what’s happening with Kubernetes on the [blog](https://kubernetes.io/blog/)
* Learn more about the [Kubernetes Release Team](https://github.com/kubernetes/sig-release/tree/master/release-team)
title: 'Kubernetes 1.20: Kubernetes Volume Snapshot Moves to GA'
date: 2020-12-10
slug: kubernetes-1.20-volume-snapshot-moves-to-ga
---
**Authors**: Xing Yang, VMware & Xiangqian Yu, Google
The Kubernetes Volume Snapshot feature is now GA in Kubernetes v1.20. It was introduced as [alpha](https://kubernetes.io/blog/2018/10/09/introducing-volume-snapshot-alpha-for-kubernetes/) in Kubernetes v1.12, followed by a [second alpha](https://kubernetes.io/blog/2019/01/17/update-on-volume-snapshot-alpha-for-kubernetes/) with breaking changes in Kubernetes v1.13, and promotion to [beta](https://kubernetes.io/blog/2019/12/09/kubernetes-1-17-feature-cis-volume-snapshot-beta/) in Kubernetes 1.17. This blog post summarizes the changes releasing the feature from beta to GA.
## What is a volume snapshot?
Many storage systems (like Google Cloud Persistent Disks, Amazon Elastic Block Storage, and many on-premise storage systems) provide the ability to create a “snapshot” of a persistent volume. A snapshot represents a point-in-time copy of a volume. A snapshot can be used either to rehydrate a new volume (pre-populated with the snapshot data) or to restore an existing volume to a previous state (represented by the snapshot).
## Why add volume snapshots to Kubernetes?
Kubernetes aims to create an abstraction layer between distributed applications and underlying clusters so that applications can be agnostic to the specifics of the cluster they run on and application deployment requires no “cluster-specific” knowledge.
The Kubernetes Storage SIG identified snapshot operations as critical functionality for many stateful workloads. For example, a database administrator may want to snapshot a database’s volumes before starting a database operation.
By providing a standard way to trigger volume snapshot operations in Kubernetes, this feature allows Kubernetes users to incorporate snapshot operations in a portable manner on any Kubernetes environment regardless of the underlying storage.
Additionally, these Kubernetes snapshot primitives act as basic building blocks that unlock the ability to develop advanced enterprise-grade storage administration features for Kubernetes, including application or cluster level backup solutions.
## What’s new since beta?
With the promotion of Volume Snapshot to GA, the feature is enabled by default on standard Kubernetes deployments and cannot be turned off.
Many enhancements have been made to improve the quality of this feature and to make it production-grade.
- The Volume Snapshot APIs and client library were moved to a separate Go module.
- A snapshot validation webhook has been added to perform necessary validation on volume snapshot objects. More details can be found in the [Volume Snapshot Validation Webhook Kubernetes Enhancement Proposal](https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/1900-volume-snapshot-validation-webhook).
- Along with the validation webhook, the volume snapshot controller will start labeling invalid snapshot objects that already existed. This allows users to identify, remove any invalid objects, and correct their workflows. Once the API is switched to the v1 type, those invalid objects will not be deletable from the system.
- To provide better insights into how the snapshot feature is performing, an initial set of operation metrics has been added to the volume snapshot controller.
- There are more end-to-end tests, running on GCP, that validate the feature in a real Kubernetes cluster. Stress tests (based on Google Persistent Disk and `hostPath` CSI Drivers) have been introduced to test the robustness of the system.
Other than introducing tightening validation, there is no difference between the v1beta1 and v1 Kubernetes volume snapshot API. In this release (with Kubernetes 1.20), both v1 and v1beta1 are served while the stored API version is still v1beta1. Future releases will switch the stored version to v1 and gradually remove v1beta1 support.
## Which CSI drivers support volume snapshots?
Snapshots are only supported for CSI drivers, not for in-tree or FlexVolume drivers. Ensure the deployed CSI driver on your cluster has implemented the snapshot interfaces. For more information, see [Container Storage Interface (CSI) for Kubernetes GA](https://kubernetes.io/blog/2019/01/15/container-storage-interface-ga/).
Currently more than [50 CSI drivers](https://kubernetes-csi.github.io/docs/drivers.html) support the Volume Snapshot feature. The [GCE Persistent Disk CSI Driver](https://github.com/kubernetes-sigs/gcp-compute-persistent-disk-csi-driver) has gone through the tests for upgrading from volume snapshots beta to GA. GA level support for other CSI drivers should be available soon.
## Who builds products using volume snapshots?
As of the publishing of this blog, the following participants from the [Kubernetes Data Protection Working Group](https://github.com/kubernetes/community/tree/master/wg-data-protection) are building products or have already built products using Kubernetes volume snapshots.
- CSI Driver along with [CSI Snapshotter sidecar](https://github.com/kubernetes-csi/external-snapshotter/tree/master/pkg/sidecar-controller)
It is strongly recommended that Kubernetes distributors bundle and deploy the volume snapshot controller, CRDs, and validation webhook as part of their Kubernetes cluster management process (independent of any CSI Driver).
{{< warning >}}
The snapshot validation webhook serves as a critical component to transition smoothly from using v1beta1 to v1 API. Not installing the snapshot validation webhook makes prevention of invalid volume snapshot objects from creation/updating impossible, which in turn will block deletion of invalid volume snapshot objects in coming upgrades.
{{< /warning >}}
If your cluster does not come pre-installed with the correct components, you may manually install them. See the [CSI Snapshotter](https://github.com/kubernetes-csi/external-snapshotter#readme) README for details.
## How to use volume snapshots?
Assuming all the required components (including CSI driver) have been already deployed and running on your cluster, you can create volume snapshots using the `VolumeSnapshot` API object, or use an existing `VolumeSnapshot` to restore a PVC by specifying the VolumeSnapshot data source on it. For more details, see the [volume snapshot documentation](/docs/concepts/storage/volume-snapshots/).
{{< note >}} The Kubernetes Snapshot API does not provide any application consistency guarantees. You have to prepare your application (pause application, freeze filesystem etc.) before taking the snapshot for data consistency either manually or using higher level APIs/controllers. {{< /note >}}
### Dynamically provision a volume snapshot
To dynamically provision a volume snapshot, create a `VolumeSnapshotClass` API object first.
Then create a `VolumeSnapshot` API object from a PVC by specifying the volume snapshot class.
```yaml
apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshot
metadata:
name: test-snapshot
namespace: ns1
spec:
volumeSnapshotClassName: test-snapclass
source:
persistentVolumeClaimName: test-pvc
```
### Importing an existing volume snapshot with Kubernetes
To import a pre-existing volume snapshot into Kubernetes, manually create a `VolumeSnapshotContent` object first.
```yaml
apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshotContent
metadata:
name: test-content
spec:
deletionPolicy: Delete
driver: testdriver.csi.k8s.io
source:
snapshotHandle: 7bdd0de3-xxx
volumeSnapshotRef:
name: test-snapshot
namespace: default
```
Then create a `VolumeSnapshot` object pointing to the `VolumeSnapshotContent` object.
```yaml
apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshot
metadata:
name: test-snapshot
spec:
source:
volumeSnapshotContentName: test-content
```
### Rehydrate volume from snapshot
A bound and ready `VolumeSnapshot` object can be used to rehydrate a new volume with data pre-populated from snapshotted data as shown here:
```yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: pvc-restore
namespace: demo-namespace
spec:
storageClassName: test-storageclass
dataSource:
name: test-snapshot
kind: VolumeSnapshot
apiGroup: snapshot.storage.k8s.io
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
```
## How to add support for snapshots in a CSI driver?
See the [CSI spec](https://github.com/container-storage-interface/spec/blob/master/spec.md) and the [Kubernetes-CSI Driver Developer Guide](https://kubernetes-csi.github.io/docs/snapshot-restore-feature.html) for more details on how to implement the snapshot feature in a CSI driver.
## What are the limitations?
The GA implementation of volume snapshots for Kubernetes has the following limitations:
- Does not support reverting an existing PVC to an earlier state represented by a snapshot (only supports provisioning a new volume from a snapshot).
### How to learn more?
The code repository for snapshot APIs and controller is here: https://github.com/kubernetes-csi/external-snapshotter
Check out additional documentation on the snapshot feature here: http://k8s.io/docs/concepts/storage/volume-snapshots and https://kubernetes-csi.github.io/docs/
## How to get involved?
This project, like all of Kubernetes, is the result of hard work by many contributors from diverse backgrounds working together.
We offer a huge thank you to the contributors who stepped up these last few quarters to help the project reach GA. We want to thank Saad Ali, Michelle Au, Tim Hockin, and Jordan Liggitt for their insightful reviews and thorough consideration with the design, thank Andi Li for his work on adding the support of the snapshot validation webhook, thank Grant Griffiths on implementing metrics support in the snapshot controller and handling password rotation in the validation webhook, thank Chris Henzie, Raunak Shah, and Manohar Reddy for writing critical e2e tests to meet the scalability and stability requirements for graduation, thank Kartik Sharma for moving snapshot APIs and client lib to a separate go module, and thank Raunak Shah and Prafull Ladha for their help with upgrade testing from beta to GA.
There are many more people who have helped to move the snapshot feature from beta to GA. We want to thank everyone who has contributed to this effort:
For those interested in getting involved with the design and development of CSI or any part of the Kubernetes Storage system, join the [Kubernetes Storage Special Interest Group](https://github.com/kubernetes/community/tree/master/sig-storage) (SIG). We’re rapidly growing and always welcome new contributors.
We also hold regular [Data Protection Working Group meetings](https://docs.google.com/document/d/15tLCV3csvjHbKb16DVk-mfUmFry_Rlwo-2uG6KNGsfw/edit#). New attendees are welcome to join in discussions.
Typically when a [CSI](https://github.com/container-storage-interface/spec/blob/baa71a34651e5ee6cb983b39c03097d7aa384278/spec.md) driver mounts credentials such as secrets and certificates, it has to authenticate against storage providers to access the credentials. However, the access to those credentials are controlled on the basis of the pods' identities rather than the CSI driver's identity. CSI drivers, therefore, need some way to retrieve pod's service account token.
Currently there are two suboptimal approaches to achieve this, either by granting CSI drivers the permission to use TokenRequest API or by reading tokens directly from the host filesystem.
Both of them exhibit the following drawbacks:
- Violating the principle of least privilege
- Every CSI driver needs to re-implement the logic of getting the pod’s service account token
The second approach is more problematic due to:
- The audience of the token defaults to the kube-apiserver
- The token is not guaranteed to be available (e.g. `AutomountServiceAccountToken=false`)
- The approach does not work for CSI drivers that run as a different (non-root) user from the pods. See [file permission section for service account token](https://github.com/kubernetes/enhancements/blob/f40c24a5da09390bd521be535b38a4dbab09380c/keps/sig-storage/20180515-svcacct-token-volumes.md#file-permission)
- The token might be legacy Kubernetes service account token which doesn’t expire if `BoundServiceAccountTokenVolume=false`
Kubernetes 1.20 introduces an alpha feature, `CSIServiceAccountToken`, to improve the security posture. The new feature allows CSI drivers to receive pods' [bound service account tokens](https://github.com/kubernetes/enhancements/blob/master/keps/sig-auth/1205-bound-service-account-tokens/README.md).
This feature also provides a knob to re-publish volumes so that short-lived volumes can be refreshed.
## Pod Impersonation
### Using GCP APIs
Using [Workload Identity](https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity), a Kubernetes service account can authenticate as a Google service account when accessing Google Cloud APIs. If a CSI driver needs to access GCP APIs on behalf of the pods that it is mounting volumes for, it can use the pod's service account token to [exchange for GCP tokens](https://cloud.google.com/iam/docs/reference/sts/rest). The pod's service account token is plumbed through the volume context in `NodePublishVolume` RPC calls when the feature `CSIServiceAccountToken` is enabled. For example: accessing [Google Secret Manager](https://cloud.google.com/secret-manager/) via a [secret store CSI driver](https://github.com/GoogleCloudPlatform/secrets-store-csi-driver-provider-gcp).
### Using Vault
If users configure [Kubernetes as an auth method](https://www.vaultproject.io/docs/auth/kubernetes), Vault uses the `TokenReview` API to validate the Kubernetes service account token. For CSI drivers using Vault as resources provider, they need to present the pod's service account to Vault. For example, [secrets store CSI driver](https://github.com/hashicorp/secrets-store-csi-driver-provider-vault) and [cert manager CSI driver](https://github.com/jetstack/cert-manager-csi).
## Short-lived Volumes
To keep short-lived volumes such as certificates effective, CSI drivers can specify `RequiresRepublish=true` in their`CSIDriver` object to have the kubelet periodically call `NodePublishVolume` on mounted volumes. These republishes allow CSI drivers to ensure that the volume content is up-to-date.
## Next steps
This feature is alpha and projected to move to beta in 1.21. See more in the following KEP and CSI documentation:
- [KEP-1855: Service Account Token for CSI Driver](https://github.com/kubernetes/enhancements/blob/master/keps/sig-storage/1855-csi-driver-service-account-token/README.md)
- SIG-Auth [meets regularly](https://github.com/kubernetes/community/tree/master/sig-auth#meetings) and can be reached via [Slack and the mailing list](https://github.com/kubernetes/community/tree/master/sig-auth#contact)
- SIG-Storage [meets regularly](https://github.com/kubernetes/community/tree/master/sig-storage#meetings) and can be reached via [Slack and the mailing list](https://github.com/kubernetes/community/tree/master/sig-storage#contact).
**Authors**: Hemant Kumar, Red Hat & Christian Huffman, Red Hat
Kubernetes 1.20 brings two important beta features, allowing Kubernetes admins and users alike to have more adequate control over how volume permissions are applied when a volume is mounted inside a Pod.
### Allow users to skip recursive permission changes on mount
Traditionally if your pod is running as a non-root user ([which you should](https://twitter.com/thockin/status/1333892204490735617)), you must specify a `fsGroup` inside the pod’s security context so that the volume can be readable and writable by the Pod. This requirement is covered in more detail in [here](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/).
But one side-effect of setting `fsGroup` is that, each time a volume is mounted, Kubernetes must recursively `chown()` and `chmod()` all the files and directories inside the volume - with a few exceptions noted below. This happens even if group ownership of the volume already matches the requested `fsGroup`, and can be pretty expensive for larger volumes with lots of small files, which causes pod startup to take a long time. This scenario has been a [known problem](https://github.com/kubernetes/kubernetes/issues/69699) for a while, and in Kubernetes 1.20 we are providing knobs to opt-out of recursive permission changes if the volume already has the correct permissions.
When configuring a pod’s security context, set `fsGroupChangePolicy` to "OnRootMismatch" so if the root of the volume already has the correct permissions, the recursive permission change can be skipped. Kubernetes ensures that permissions of the top-level directory are changed last the first time it applies permissions.
```yaml
securityContext:
runAsUser: 1000
runAsGroup: 3000
fsGroup: 2000
fsGroupChangePolicy: "OnRootMismatch"
```
You can learn more about this in [Configure volume permission and ownership change policy for Pods](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#configure-volume-permission-and-ownership-change-policy-for-pods).
### Allow CSI Drivers to declare support for fsGroup based permissions
Although the previous section implied that Kubernetes _always_ recursively changes permissions of a volume if a Pod has a `fsGroup`, this is not strictly true. For certain multi-writer volume types, such as NFS or Gluster, the cluster doesn’t perform recursive permission changes even if the pod has a `fsGroup`. Other volume types may not even support `chown()`/`chmod()`, which rely on Unix-style permission control primitives.
So how do we know when to apply recursive permission changes and when we shouldn't? For in-tree storage drivers, this was relatively simple. For [CSI](https://kubernetes-csi.github.io/docs/introduction.html#introduction) drivers that could span a multitude of platforms and storage types, this problem can be a bigger challenge.
Previously, whenever a CSI volume was mounted to a Pod, Kubernetes would attempt to automatically determine if the permissions and ownership should be modified. These methods were imprecise and could cause issues as we already mentioned, depending on the storage type.
The CSIDriver custom resource now has a `.spec.fsGroupPolicy` field, allowing storage drivers to explicitly opt in or out of these recursive modifications. By having the CSI driver specify a policy for the backing volumes, Kubernetes can avoid needless modification attempts. This optimization helps to reduce volume mount time and also cuts own reporting errors about modifications that would never succeed.
#### CSIDriver FSGroupPolicy API
Three FSGroupPolicy values are available as of Kubernetes 1.20, with more planned for future releases.
- **ReadWriteOnceWithFSType** - This is the default policy, applied if no `fsGroupPolicy` is defined; this preserves the behavior from previous Kubernetes releases. Each volume is examined at mount time to determine if permissions should be recursively applied.
- **File** - Always attempt to apply permission modifications, regardless of the filesystem type or PersistentVolumeClaim’s access mode.
- **None** - Never apply permission modifications.
#### How do I use it?
The only configuration needed is defining `fsGroupPolicy` inside of the `.spec` for a CSIDriver. Once that element is defined, any subsequently mounted volumes will automatically use the defined policy. There’s no additional deployment required!
#### What’s next?
Depending on feedback and adoption, the Kubernetes team plans to push these implementations to GA in either 1.21 or 1.22.
### How can I learn more?
This feature is explained in more detail in Kubernetes project documentation: [CSI Driver fsGroup Support](https://kubernetes-csi.github.io/docs/support-fsgroup.html) and [Configure volume permission and ownership change policy for Pods ](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#configure-volume-permission-and-ownership-change-policy-for-pods).
### How do I get involved?
The [Kubernetes Slack channel #csi](https://kubernetes.slack.com/messages/csi) and any of the [standard SIG Storage communication channels](https://github.com/kubernetes/community/blob/master/sig-storage/README.md#contact) are great mediums to reach out to the SIG Storage and the CSI team.
Those interested in getting involved with the design and development of CSI or any part of the Kubernetes Storage system, join the [Kubernetes Storage Special Interest Group (SIG)](https://github.com/kubernetes/community/tree/master/sig-storage). We’re rapidly growing and always welcome new contributors.
**Authors:** Renaud Gaubert (NVIDIA), David Ashpole (Google), and Pramod Ramarao (NVIDIA)
With Kubernetes 1.20, infrastructure teams who manage large scale Kubernetes clusters, are seeing the graduation of two exciting and long awaited features:
* The Pod Resources API (introduced in 1.13) is finally graduating to GA. This allows Kubernetes plugins to obtain information about the node’s resource usage and assignment; for example: which pod/container consumes which device.
* The `DisableAcceleratorMetrics` feature (introduced in 1.19) is graduating to beta and will be enabled by default. This removes device metrics reported by the kubelet in favor of the new plugin architecture.
Many of the features related to fundamental device support (device discovery, plugin, and monitoring) are reaching a strong level of stability.
Kubernetes users should see these features as stepping stones to enable more complex use cases (networking, scheduling, storage, etc.)!
One such example is Non Uniform Memory Access (NUMA) placement where, when selecting a device, an application typically wants to ensure that data transfer between CPU Memory and Device Memory is as fast as possible. In some cases, incorrect NUMA placement can nullify the benefit of offloading compute to an external device.
If these are topics of interest to you, consider joining the [Kubernetes Node Special Insterest Group](https://github.com/kubernetes/community/tree/master/sig-node) (SIG) for all topics related to the Kubernetes node, the COD (container orchestrated device) workgroup for topics related to runtimes, or the resource management forum for topics related to resource management!
## The Pod Resources API - Why does it need to exist?
Kubernetes is a vendor neutral platform. If we want it to support device monitoring, adding vendor-specific code in the Kubernetes code base is not an ideal solution. Ultimately, devices are a domain where deep expertise is needed and the best people to add and maintain code in that area are the device vendors themselves.
The Pod Resources API was built as a solution to this issue. Each vendor can build and maintain their own out-of-tree monitoring plugin. This monitoring plugin, often deployed as a separate pod within a cluster, can then associate the metrics a device emits with the associated pod that's using it.
For example, use the NVIDIA GPU dcgm-exporter to scrape metrics in Prometheus format:
```
$ curl -sL http://127.0.01:8080/metrics
# HELP DCGM_FI_DEV_SM_CLOCK SM clock frequency (in MHz).
# TYPE DCGM_FI_DEV_SM_CLOCK gauge
# HELP DCGM_FI_DEV_MEM_CLOCK Memory clock frequency (in MHz).
# TYPE DCGM_FI_DEV_MEM_CLOCK gauge
# HELP DCGM_FI_DEV_MEMORY_TEMP Memory temperature (in C).
Each agent is expected to adhere to the node monitoring guidelines. In other words, plugins are expected to generate metrics in Prometheus format, and new metrics should not have any dependency on the Kubernetes base directly.
This allows consumers of the metrics to use a compatible monitoring pipeline to collect and analyze metrics from a variety of agents, even if they are maintained by different vendors.
## Disabling the NVIDIA GPU metrics - Warning {#nvidia-gpu-metrics-deprecated}
With the graduation of the plugin monitoring system, Kubernetes is deprecating the NVIDIA GPU metrics that are being reported by the kubelet.
With the [DisableAcceleratorMetrics](/docs/concepts/cluster-administration/system-metrics/#disable-accelerator-metrics) feature being enabled by default in Kubernetes 1.20, NVIDIA GPUs are no longer special citizens in Kubernetes. This is a good thing in the spirit of being vendor-neutral, and enables the most suited people to maintain their plugin on their own release schedule!
Users will now need to either install the [NVIDIA GDGM exporter](https://github.com/NVIDIA/gpu-monitoring-tools) or use [bindings](https://github.com/nvidia/go-nvml) to gather more accurate and complete metrics about NVIDIA GPUs. This deprecation means that you can no longer rely on metrics that were reported by kubelet, such as `container_accelerator_duty_cycle` or `container_accelerator_memory_used_bytes` which were used to gather NVIDIA GPU memory utilization.
This means that users who used to rely on the NVIDIA GPU metrics reported by the kubelet, will need to update their reference and deploy the NVIDIA plugin. Namely the different metrics reported by Kubernetes map to the following metrics:
You might also be interested in other metrics such as `DCGM_FI_DEV_GPU_TEMP` (the GPU temperature) or DCGM_FI_DEV_POWER_USAGE (the power usage). The [default set](https://github.com/NVIDIA/gpu-monitoring-tools/blob/d5c9bb55b4d1529ca07068b7f81e690921ce2b59/etc/dcgm-exporter/default-counters.csv) is available in Nvidia's [Data Center GPU Manager documentation](https://docs.nvidia.com/datacenter/dcgm/latest/dcgm-api/group__dcgmFieldIdentifiers.html).
Note that for this release you can still set the `DisableAcceleratorMetrics` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/) to _false_, effectively re-enabling the ability for the kubelet to report NVIDIA GPU metrics.
Paired with the graduation of the Pod Resources API, these tools can be used to generate GPU telemetry [that can be used in visualization dashboards](https://grafana.com/grafana/dashboards/12239), below is an example:

## The Pod Resources API - What can I go on to do with this?
As soon as this interface was introduced, many vendors started using it for widely different use cases! To list a few examples:
The [kuryr-kubernetes](https://github.com/openstack/kuryr-kubernetes) CNI plugin in tandem with [intel-sriov-device-plugin](https://github.com/intel/sriov-network-device-plugin). This allowed the CNI plugin to know which allocation of SR-IOV Virtual Functions (VFs) the kubelet made and use that information to correctly setup the container network namespace and use a device with the appropriate NUMA node. We also expect this interface to be used to track the allocated and available resources with information about the NUMA topology of the worker node.
Another use-case is GPU telemetry, where GPU metrics can be associated with the containers and pods that the GPU is assigned to. One such example is the NVIDIA `dcgm-exporter`, but others can be easily built in the same paradigm.
The Pod Resources API is a simple gRPC service which informs clients of the pods the kubelet knows. The information concerns the devices assignment the kubelet made and the assignment of CPUs. This information is obtained from the internal state of the kubelet's Device Manager and CPU Manager respectively.
You can see below a sample example of the API and how a go client could use that information in a few lines:
Finally, note that you can watch the number of requests made to the Pod Resources endpoint by watching the new kubelet metric called `pod_resources_endpoint_requests_total` on the kubelet's `/metrics` endpoint.
## Is device monitoring suitable for production? Can I extend it? Can I contribute?
Yes! This feature released in 1.13, almost 2 years ago, has seen broad adoption, is already used by different cloud managed services, and with its graduation to G.A in Kubernetes 1.20 is production ready!
If you are a device vendor, you can start using it today! If you just want to monitor the devices in your cluster, go get the latest version of your monitoring plugin!
If you feel passionate about that area, join the kubernetes community, help improve the API or contribute the device monitoring plugins!
## Acknowledgements
We thank the members of the community who have contributed to this feature or given feedback including members of WG-Resource-Management, SIG-Node and the Resource management forum!
Authors: Marcin Maciaszczyk, Kubermatic & Sebastian Florek, Kubermatic
In October 2020, the Kubernetes Dashboard officially turned five. As main project maintainers, we can barely believe that so much time has passed since our very first commits to the project. However, looking back with a bit of nostalgia, we realize that quite a lot has happened since then. Now it’s due time to celebrate “our baby” with a short recap.
## How It All Began
The initial idea behind the Kubernetes Dashboard project was to provide a web interface for Kubernetes. We wanted to reflect the kubectl functionality through an intuitive web UI. The main benefit from using the UI is to be able to quickly see things that do not work as expected (monitoring and troubleshooting). Also, the Kubernetes Dashboard is a great starting point for users that are new to the Kubernetes ecosystem.
The very [first commit](https://github.com/kubernetes/dashboard/commit/5861187fa807ac1cc2d9b2ac786afeced065076c) to the Kubernetes Dashboard was made by Filip Grządkowski from Google on 16th October 2015 – just a few months from the initial commit to the Kubernetes repository. Our initial commits go back to November 2015 ([Sebastian committed on 16 November 2015](https://github.com/kubernetes/dashboard/commit/09e65b6bb08c49b926253de3621a73da05e400fd); [Marcin committed on 23 November 2015](https://github.com/kubernetes/dashboard/commit/1da4b1c25ef040818072c734f71333f9b4733f55)). Since that time, we’ve become regular contributors to the project. For the next two years, we worked closely with the Googlers, eventually becoming main project maintainers ourselves.
{{< figure src="first-ui.png" caption="The First Version of the User Interface" >}}
{{< figure src="along-the-way-ui.png" caption="Prototype of the New User Interface" >}}
{{< figure src="current-ui.png" caption="The Current User Interface" >}}
As you can see, the initial look and feel of the project were completely different from the current one. We have changed the design multiple times. The same has happened with the code itself.
## Growing Up - The Big Migration
At [the beginning of 2018](https://github.com/kubernetes/dashboard/pull/2727), we reached a point where AngularJS was getting closer to the end of its life, while the new Angular versions were published quite often. A lot of the libraries and the modules that we were using were following the trend. That forced us to spend a lot of the time rewriting the frontend part of the project to make it work with newer technologies.
The migration came with many benefits like being able to refactor a lot of the code, introduce design patterns, reduce code complexity, and benefit from the new modules. However, you can imagine that the scale of the migration was huge. Luckily, there were a number of contributions from the community helping us with the resource support, new Kubernetes version support, i18n, and much more. After many long days and nights, we finally released the [first beta version](https://github.com/kubernetes/dashboard/releases/tag/v2.0.0-beta1) in July 2019, followed by the [2.0 release](https://github.com/kubernetes/dashboard/releases/tag/v2.0.0) in April 2020 — our baby had grown up.
## Where Are We Standing in 2021?
Due to limited resources, unfortunately, we were not able to offer extensive support for many different Kubernetes versions. So, we’ve decided to always try and support the latest Kubernetes version available at the time of the Kubernetes Dashboard release. The latest release, [Dashboard v2.2.0](https://github.com/kubernetes/dashboard/releases/tag/v2.2.0) provides support for Kubernetes v1.20.
On top of that, we put in a great deal of effort into [improving resource support](https://github.com/kubernetes/dashboard/issues/5232). Meanwhile, we do offer support for most of the Kubernetes resources. Also, the Kubernetes Dashboard supports multiple languages: English, German, French, Japanese, Korean, Chinese (Traditional, Simplified, Traditional Hong Kong). Persian and Russian localizations are currently in progress. Moreover, we are working on the support for 3rd party themes and the design of the app in general. As you can see, quite a lot of things are going on.
Luckily, we do have regular contributors with domain knowledge who are taking care of the project, updating the Helm charts, translations, Go modules, and more. But as always, there could be many more hands on deck. So if you are thinking about contributing to Kubernetes, keep us in mind ;)
## What’s Next
The Kubernetes Dashboard has been growing and prospering for more than 5 years now. It provides the community with an intuitive Web UI, thereby decreasing the complexity of Kubernetes and increasing its accessibility to new community members. We are proud of what the project has achieved so far, but this is by far not the end. These are our priorities for the future:
* Keep providing support for the new Kubernetes versions
* Keep improving the support for the existing resources
* Keep working on auth system improvements
* [Rewrite the API to use gRPC and shared informers](https://github.com/kubernetes/dashboard/pull/5449): This will allow us to improve the performance of the application but, most importantly, to support live updates coming from the Kubernetes project. It is one of the most requested features from the community.
* Split the application into two containers, one with the UI and the second with the API running inside.
## The Kubernetes Dashboard in Numbers
* Initial commit made on October 16, 2015
* Over 100 million pulls from Dockerhub since the v2 release
* 8 supported languages and the next 2 in progress
* Over 3360 closed PRs
* Over 2260 closed issues
* 100% coverage of the supported core Kubernetes resources
* Over 9000 stars on GitHub
* Over 237 000 lines of code
## Join Us
As mentioned earlier, we are currently looking for more people to help us further develop and grow the project. We are open to contributions in multiple areas, i.e., [issues with help wanted label](https://github.com/kubernetes/dashboard/issues?q=is%3Aissue+is%3Aopen+label%3A%22help+wanted%22). Please feel free to reach out via GitHub or the #sig-ui channel in the [Kubernetes Slack](https://slack.k8s.io/).
@@ -37,8 +37,8 @@ when an individual is representing the project or its community.
Instances of abusive, harassing, or otherwise unacceptable behavior in Kubernetes may be reported by contacting the [Kubernetes Code of Conduct Committee](https://git.k8s.io/community/committee-code-of-conduct) via <conduct@kubernetes.io>. For other projects, please contact a CNCF project maintainer or our mediator, Mishi Choudhary <mishi@linux.com>.
This Code of Conduct is adapted from the Contributor Covenant
(http://contributor-covenant.org), version 1.2.0, available at
http://contributor-covenant.org/version/1/2/0/
(https://contributor-covenant.org), version 1.2.0, available at
<!-- Do not edit this file directly. Get the latest from
https://git.k8s.io/community/values.md -->
# Kubernetes Community Values
Kubernetes Community culture is frequently cited as a substantial contributor to the meteoric rise of this Open Source project. Below are the distilled values which have evolved over the last many years in our community pushing our project and peers toward constant improvement.
## Distribution is better than centralization
The scale of the Kubernetes project is only viable through high-trust and high-visibility distribution of work, which includes delegation of authority, decision making, technical design, code ownership, and documentation. Distributed asynchronous ownership, collaboration, communication and decision making are the cornerstone of our world-wide community.
## Community over product or company
We are here as a community first, our allegiance is to the intentional stewardship of the Kubernetes project for the benefit of all its members and users everywhere. We support working together publicly for the common goal of a vibrant interoperable ecosystem providing an excellent experience for our users. Individuals gain status through work, companies gain status through their commitments to support this community and fund the resources necessary for the project to operate.
## Automation over process
Large projects have a lot of less exciting, yet, hard work. We value time spent automating repetitive work more highly than toil. Where that work cannot be automated, it is our culture to recognize and reward all types of contributions. However, heroism is not sustainable.
## Inclusive is better than exclusive
Broadly successful and useful technology requires different perspectives and skill sets which can only be heard in a welcoming and respectful environment. Community membership is a privilege, not a right. Community Leadership is earned through effort, scope, quality, quantity, and duration of contributions. Our community shows respect for the time and effort put into a discussion regardless of where a contributor is on their growth path.
## Evolution is better than stagnation
Openness to new ideas and studied technological evolution make Kubernetes a stronger project. Continual improvement, servant leadership, mentorship and respect are the foundations of the Kubernetes project culture. It is the duty for leaders in the Kubernetes community to find, sponsor, and promote new community members. Leaders should expect to step aside. Community members should expect to step up.
**"Culture eats strategy for breakfast." --Peter Drucker**
This document catalogs the communication paths between the control plane (really the apiserver) and the Kubernetes cluster. The intent is to allow users to customize their installation to harden the network configuration such that the cluster can be run on an untrusted network (or on fully public IPs on a cloud provider).
This document catalogs the communication paths between the control plane (apiserver) and the Kubernetes cluster. The intent is to allow users to customize their installation to harden the network configuration such that the cluster can be run on an untrusted network (or on fully public IPs on a cloud provider).
<!-- body -->
## Node to Control Plane
Kubernetes has a "hub-and-spoke" API pattern. All API usage from nodes (or the pods they run) terminate at the apiserver (none of the other control plane components are designed to expose remote services). The apiserver is configured to listen for remote connections on a secure HTTPS port (typically 443) with one or more forms of client [authentication](/docs/reference/access-authn-authz/authentication/) enabled.
Kubernetes has a "hub-and-spoke" API pattern. All API usage from nodes (or the pods they run) terminates at the apiserver. None of the other control plane components are designed to expose remote services. The apiserver is configured to listen for remote connections on a secure HTTPS port (typically 443) with one or more forms of client [authentication](/docs/reference/access-authn-authz/authentication/) enabled.
One or more forms of [authorization](/docs/reference/access-authn-authz/authorization/) should be enabled, especially if [anonymous requests](/docs/reference/access-authn-authz/authentication/#anonymous-requests) or [service account tokens](/docs/reference/access-authn-authz/authentication/#service-account-tokens) are allowed.
Nodes should be provisioned with the public root certificate for the cluster such that they can connect securely to the apiserver along with valid client credentials. A good approach is that the client credentials provided to the kubelet are in the form of a client certificate. See [kubelet TLS bootstrapping](/docs/reference/command-line-tools-reference/kubelet-tls-bootstrapping/) for automated provisioning of kubelet client certificates.
Pods that wish to connect to the apiserver can do so securely by leveraging a service account so that Kubernetes will automatically inject the public root certificate and a valid bearer token into the pod when it is instantiated.
The `kubernetes` service (in all namespaces) is configured with a virtual IP address that is redirected (via kube-proxy) to the HTTPS endpoint on the apiserver.
The `kubernetes` service (in `default` namespace) is configured with a virtual IP address that is redirected (via kube-proxy) to the HTTPS endpoint on the apiserver.
The control plane components also communicate with the cluster apiserver over the secure port.
@@ -42,7 +42,7 @@ The connections from the apiserver to the kubelet are used for:
* Attaching (through kubectl) to running pods.
* Providing the kubelet's port-forwarding functionality.
These connections terminate at the kubelet's HTTPS endpoint. By default, the apiserver does not verify the kubelet's serving certificate, which makes the connection subject to man-in-the-middle attacks, and **unsafe** to run over untrusted and/or public networks.
These connections terminate at the kubelet's HTTPS endpoint. By default, the apiserver does not verify the kubelet's serving certificate, which makes the connection subject to man-in-the-middle attacks and **unsafe** to run over untrusted and/or public networks.
To verify this connection, use the `--kubelet-certificate-authority` flag to provide the apiserver with a root certificate bundle to use to verify the kubelet's serving certificate.
The connections from the apiserver to a node, pod, or service default to plain HTTP connections and are therefore neither authenticated nor encrypted. They can be run over a secure HTTPS connection by prefixing `https:` to the node, pod, or service name in the API URL, but they will not validate the certificate provided by the HTTPS endpoint nor provide client credentials so while the connection will be encrypted, it will not provide any guarantees of integrity. These connections **are not currently safe** to run over untrusted and/or public networks.
The connections from the apiserver to a node, pod, or service default to plain HTTP connections and are therefore neither authenticated nor encrypted. They can be run over a secure HTTPS connection by prefixing `https:` to the node, pod, or service name in the API URL, but they will not validate the certificate provided by the HTTPS endpoint nor provide client credentials. So while the connection will be encrypted, it will not provide any guarantees of integrity. These connections **are not currently safe** to run over untrusted or public networks.
### SSH tunnels
Kubernetes supports SSH tunnels to protect the control plane to nodes communication paths. In this configuration, the apiserver initiates an SSH tunnel to each node in the cluster (connecting to the ssh server listening on port 22) and passes all traffic destined for a kubelet, node, pod, or service through the tunnel.
This tunnel ensures that the traffic is not exposed outside of the network in which the nodes are running.
SSH tunnels are currently deprecated so you shouldn't opt to use them unless you know what you are doing. The Konnectivity service is a replacement for this communication channel.
SSH tunnels are currently deprecated, so you shouldn't opt to use them unless you know what you are doing. The Konnectivity service is a replacement for this communication channel.
As a replacement to the SSH tunnels, the Konnectivity service provides TCP level proxy for the control plane to cluster communication. The Konnectivity service consists of two parts: the Konnectivity server and the Konnectivity agents, running in the control plane network and the nodes network respectively. The Konnectivity agents initiate connections to the Konnectivity server and maintain the network connections.
As a replacement to the SSH tunnels, the Konnectivity service provides TCP level proxy for the control plane to cluster communication. The Konnectivity service consists of two parts: the Konnectivity server in the control plane network and the Konnectivity agents in the nodes network. The Konnectivity agents initiate connections to the Konnectivity server and maintain the network connections.
After enabling the Konnectivity service, all control plane to nodes traffic goes through these connections.
Follow the [Konnectivity service task](/docs/tasks/extend-kubernetes/setup-konnectivity/) to set up the Konnectivity service in your cluster.
Typically you have several nodes in a cluster; in a learning or resource-limited
environment, you might have just one.
environment, you might have only one node.
The [components](/docs/concepts/overview/components/#node-components) on a node include the
{{< glossary_tooltip text="kubelet" term_id="kubelet" >}}, a
@@ -30,7 +31,7 @@ The [components](/docs/concepts/overview/components/#node-components) on a node
There are two main ways to have Nodes added to the {{< glossary_tooltip text="API server" term_id="kube-apiserver" >}}:
1. The kubelet on a node self-registers to the control plane
2. You, or another human user, manually add a Node object
2. You (or another human user) manually add a Node object
After you create a Node object, or the kubelet on a node self-registers, the
control plane checks whether the new Node object is valid. For example, if you
@@ -51,8 +52,8 @@ try to create a Node from the following JSON manifest:
Kubernetes creates a Node object internally (the representation). Kubernetes checks
that a kubelet has registered to the API server that matches the `metadata.name`
field of the Node. If the node is healthy (if all necessary services are running),
it is eligible to run a Pod. Otherwise, that node is ignored for any cluster activity
field of the Node. If the node is healthy (i.e. all necessary services are running),
then it is eligible to run a Pod. Otherwise, that node is ignored for any cluster activity
until it becomes healthy.
{{< note >}}
@@ -95,14 +96,14 @@ You can create and modify Node objects using
When you want to create Node objects manually, set the kubelet flag `--register-node=false`.
You can modify Node objects regardless of the setting of `--register-node`.
For example, you can set labels on an existing Node, or mark it unschedulable.
For example, you can set labels on an existing Node or mark it unschedulable.
You can use labels on Nodes in conjunction with node selectors on Pods to control
scheduling. For example, you can constrain a Pod to only be eligible to run on
a subset of the available nodes.
Marking a node as unschedulable prevents the scheduler from placing new pods onto
that Node, but does not affect existing Pods on the Node. This is useful as a
that Node but does not affect existing Pods on the Node. This is useful as a
preparatory step before a node reboot or other maintenance.
To mark a Node unschedulable, run:
@@ -178,14 +179,14 @@ The node condition is represented as a JSON object. For example, the following s
]
```
If the Status of the Ready condition remains `Unknown` or `False` for longer than the `pod-eviction-timeout` (an argument passed to the {{< glossary_tooltip text="kube-controller-manager" term_id="kube-controller-manager" >}}), all the Pods on the node are scheduled for deletion by the node controller. The default eviction timeout duration is **five minutes**. In some cases when the node is unreachable, the API server is unable to communicate with the kubelet on the node. The decision to delete the pods cannot be communicated to the kubelet until communication with the API server is re-established. In the meantime, the pods that are scheduled for deletion may continue to run on the partitioned node.
If the Status of the Ready condition remains `Unknown` or `False` for longer than the `pod-eviction-timeout` (an argument passed to the {{< glossary_tooltip text="kube-controller-manager" term_id="kube-controller-manager" >}}), then all the Pods on the node are scheduled for deletion by the node controller. The default eviction timeout duration is **five minutes**. In some cases when the node is unreachable, the API server is unable to communicate with the kubelet on the node. The decision to delete the pods cannot be communicated to the kubelet until communication with the API server is re-established. In the meantime, the pods that are scheduled for deletion may continue to run on the partitioned node.
The node controller does not force delete pods until it is confirmed that they have stopped
running in the cluster. You can see the pods that might be running on an unreachable node as
being in the `Terminating` or `Unknown` state. In cases where Kubernetes cannot deduce from the
underlying infrastructure if a node has permanently left a cluster, the cluster administrator
may need to delete the node object by hand. Deleting the node object from Kubernetes causes
all the Pod objects running on the node to be deleted from the API server, and frees up their
may need to delete the node object by hand. Deleting the node object from Kubernetes causes
all the Pod objects running on the node to be deleted from the API server and frees up their
names.
The node lifecycle controller automatically creates
@@ -198,7 +199,7 @@ for more details.
### Capacity and Allocatable {#capacity}
Describes the resources available on the node: CPU, memory and the maximum
Describes the resources available on the node: CPU, memory, and the maximum
number of pods that can be scheduled onto the node.
The fields in the capacity block indicate the total amount of resources that a
@@ -224,25 +225,27 @@ CIDR block to the node when it is registered (if CIDR assignment is turned on).
The second is keeping the node controller's internal list of nodes up to date with
the cloud provider's list of available machines. When running in a cloud
environment, whenever a node is unhealthy, the node controller asks the cloud
environment and whenever a node is unhealthy, the node controller asks the cloud
provider if the VM for that node is still available. If not, the node
controller deletes the node from its list of nodes.
The third is monitoring the nodes' health. The node controller is
responsible for updating the NodeReady condition of NodeStatus to
ConditionUnknown when a node becomes unreachable (i.e. the node controller stops
receiving heartbeats for some reason, for example due to the node being down), and then later evicting
all the pods from the node (using graceful termination) if the node continues
to be unreachable. (The default timeouts are 40s to start reporting
ConditionUnknown and 5m after that to start evicting pods.) The node controller
checks the state of each node every `--node-monitor-period` seconds.
responsible for:
- Updating the NodeReady condition of NodeStatus to ConditionUnknown when a node
becomes unreachable, as the node controller stops receiving heartbeats for some
reason such as the node being down.
- Evicting all the pods from the node using graceful termination if
the node continues to be unreachable. The default timeouts are 40s to start
reporting ConditionUnknown and 5m after that to start evicting pods.
The node controller checks the state of each node every `--node-monitor-period` seconds.
#### Heartbeats
Heartbeats, sent by Kubernetes nodes, help determine the availability of a node.
There are two forms of heartbeats: updates of `NodeStatus` and the
Lease is a lightweight resource, which improves the performance
@@ -251,13 +254,14 @@ of the node heartbeats as the cluster scales.
The kubelet is responsible for creating and updating the `NodeStatus` and
a Lease object.
- The kubelet updates the `NodeStatus` either when there is change in status,
- The kubelet updates the `NodeStatus` either when there is change in status
or if there has been no update for a configured interval. The default interval
for `NodeStatus` updates is 5 minutes (much longer than the 40 second default
timeout for unreachable nodes).
for `NodeStatus` updates is 5 minutes, which is much longer than the 40 second default
timeout for unreachable nodes.
- The kubelet creates and then updates its Lease object every 10 seconds
(the default update interval). Lease updates occur independently from the
`NodeStatus` updates. If the Lease update fails, the kubelet retries with exponential backoff starting at 200 milliseconds and capped at 7 seconds.
`NodeStatus` updates. If the Lease update fails, the kubelet retries with
exponential backoff starting at 200 milliseconds and capped at 7 seconds.
#### Reliability
@@ -268,23 +272,25 @@ from more than 1 node per 10 seconds.
The node eviction behavior changes when a node in a given availability zone
becomes unhealthy. The node controller checks what percentage of nodes in the zone
are unhealthy (NodeReady condition is ConditionUnknown or ConditionFalse) at
the same time. If the fraction of unhealthy nodes is at least
`--unhealthy-zone-threshold` (default 0.55) then the eviction rate is reduced:
if the cluster is small (i.e. has less than or equal to
`--large-cluster-size-threshold` nodes - default 50) then evictions are
stopped, otherwise the eviction rate is reduced to
`--secondary-node-eviction-rate` (default 0.01) per second. The reason these
policies are implemented per availability zone is because one availability zone
might become partitioned from the master while the others remain connected. If
your cluster does not span multiple cloud provider availability zones, then
there is only one availability zone (the whole cluster).
the same time:
- If the fraction of unhealthy nodes is at least `--unhealthy-zone-threshold`
(default 0.55), then the eviction rate is reduced.
- If the cluster is small (i.e. has less than or equal to
`--large-cluster-size-threshold` nodes - default 50), then evictions are stopped.
- Otherwise, the eviction rate is reduced to `--secondary-node-eviction-rate`
(default 0.01) per second.
The reason these policies are implemented per availability zone is because one
availability zone might become partitioned from the master while the others remain
connected. If your cluster does not span multiple cloud provider availability zones,
then there is only one availability zone (i.e. the whole cluster).
A key reason for spreading your nodes across availability zones is so that the
workload can be shifted to healthy zones when one entire zone goes down.
Therefore, if all nodes in a zone are unhealthy then the node controller evicts at
Therefore, if all nodes in a zone are unhealthy, then the node controller evicts at
the normal rate of `--node-eviction-rate`. The corner case is when all zones are
completely unhealthy (i.e. there are no healthy nodes in the cluster). In such a
case, the node controller assumes that there's some problem with master
case, the node controller assumes that there is some problem with master
connectivity and stops all evictions until some connectivity is restored.
The node controller is also responsible for evicting pods running on nodes with
@@ -302,8 +308,8 @@ eligible for, effectively removing incoming load balancer traffic from the cordo
### Node capacity
Node objects track information about the Node's resource capacity (for example: the amount
of memory available, and the number of CPUs).
Node objects track information about the Node's resource capacity: for example, the amount
of memory available and the number of CPUs.
Nodes that [self register](#self-registration-of-nodes) report their capacity during
registration. If you [manually](#manual-node-administration) add a Node, then
you need to set the node's capacity information when you add it.
@@ -337,7 +343,7 @@ for more information.
If you have enabled the `GracefulNodeShutdown` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/), then the kubelet attempts to detect the node system shutdown and terminates pods running on the node.
Kubelet ensures that pods follow the normal [pod termination process](/docs/concepts/workloads/pods/pod-lifecycle/#pod-termination) during the node shutdown.
When the `GracefulNodeShutdown` feature gate is enabled, kubelet uses [systemd inhibitor locks](https://www.freedesktop.org/wiki/Software/systemd/inhibit/) to delay the node shutdown with a given duration. During a shutdown kubelet terminates pods in two phases:
When the `GracefulNodeShutdown` feature gate is enabled, kubelet uses [systemd inhibitor locks](https://www.freedesktop.org/wiki/Software/systemd/inhibit/) to delay the node shutdown with a given duration. During a shutdown, kubelet terminates pods in two phases:
1. Terminate regular pods running on the node.
2. Terminate [critical pods](/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods/#marking-pod-as-critical) running on the node.
@@ -26,12 +26,12 @@ See the guides in [Setup](/docs/setup/) for examples of how to plan, set up, and
Before choosing a guide, here are some considerations:
- Do you just want to try out Kubernetes on your computer, or do you want to build a high-availability, multi-node cluster? Choose distros best suited for your needs.
- Do you want to try out Kubernetes on your computer, or do you want to build a high-availability, multi-node cluster? Choose distros best suited for your needs.
- Will you be using **a hosted Kubernetes cluster**, such as [Google Kubernetes Engine](https://cloud.google.com/kubernetes-engine/), or **hosting your own cluster**?
- Will your cluster be **on-premises**, or **in the cloud (IaaS)**? Kubernetes does not directly support hybrid clusters. Instead, you can set up multiple clusters.
- **If you are configuring Kubernetes on-premises**, consider which [networking model](/docs/concepts/cluster-administration/networking/) fits best.
- Will you be running Kubernetes on **"bare metal" hardware** or on **virtual machines (VMs)**?
- Do you **just want to run a cluster**, or do you expect to do **active development of Kubernetes project code**? If the
- Do you **want to run a cluster**, or do you expect to do **active development of Kubernetes project code**? If the
latter, choose an actively-developed distro. Some distros only use binary releases, but
offer a greater variety of choices.
- Familiarize yourself with the [components](/docs/concepts/overview/components/) needed to run a cluster.
@@ -45,7 +45,7 @@ Before choosing a guide, here are some considerations:
## Securing a cluster
* [Certificates](/docs/concepts/cluster-administration/certificates/) describes the steps to generate certificates using different tool chains.
* [Generate Certificates](/docs/tasks/administer-cluster/certificates/) describes the steps to generate certificates using different tool chains.
* [Kubernetes Container Environment](/docs/concepts/containers/container-environment/) describes the environment for Kubelet managed containers on a Kubernetes node.
Garbage collection is a helpful function of kubelet that will clean up unused images and unused containers. Kubelet will perform garbage collection for containers every minute and garbage collection for images every five minutes.
Garbage collection is a helpful function of kubelet that will clean up unused [images](/docs/concepts/containers/#container-images) and unused [containers](/docs/concepts/containers/). Kubelet will perform garbage collection for containers every minute and garbage collection for images every five minutes.
External garbage collection tools are not recommended as these tools can potentially break the behavior of kubelet by removing containers expected to exist.
@@ -37,14 +37,14 @@ Containers that are not managed by kubelet are not subject to container garbage
## User Configuration
Users can adjust the following thresholds to tune image garbage collection with the following kubelet flags :
You can adjust the following thresholds to tune image garbage collection with the following kubelet flags :
1. `image-gc-high-threshold`, the percent of disk usage which triggers image garbage collection.
Default is 85%.
2. `image-gc-low-threshold`, the percent of disk usage to which image garbage collection attempts
to free. Default is 80%.
We also allow users to customize garbage collection policy through the following kubelet flags:
You can customize the garbage collection policy through the following kubelet flags:
1. `minimum-container-ttl-duration`, minimum age for a finished container before it is
garbage collected. Default is 0 minute, which means every finished container will be garbage collected.
@@ -84,4 +84,3 @@ Including:
See [Configuring Out Of Resource Handling](/docs/tasks/administer-cluster/out-of-resource/) for more details.
Application logs can help you understand what is happening inside your application. The logs are particularly useful for debugging problems and monitoring cluster activity. Most modern applications have some kind of logging mechanism; as such, most container engines are likewise designed to support some kind of logging. The easiest and most embraced logging method for containerized applications is to write to the standard output and standard error streams.
Application logs can help you understand what is happening inside your application. The logs are particularly useful for debugging problems and monitoring cluster activity. Most modern applications have some kind of logging mechanism. Likewise, container engines are designed to support logging. The easiest and most adopted logging method for containerized applications is writing to standard output and standard error streams.
However, the native functionality provided by a container engine or runtime is usually not enough for a complete logging solution. For example, if a container crashes, a pod is evicted, or a node dies, you'll usually still want to access your application's logs. As such, logs should have a separate storage and lifecycle independent of nodes, pods, or containers. This concept is called _cluster-level-logging_. Cluster-level logging requires a separate backend to store, analyze, and query logs. Kubernetes provides no native storage solution for log data, but you can integrate many existing logging solutions into your Kubernetes cluster.
However, the native functionality provided by a container engine or runtime is usually not enough for a complete logging solution.
For example, you may want access your application's logs if a container crashes; a pod gets evicted; or a node dies.
In a cluster, logs should have a separate storage and lifecycle independent of nodes, pods, or containers. This concept is called _cluster-level logging_.
<!-- body -->
Cluster-level logging architectures are described in assumption that
a logging backend is present inside or outside of your cluster. If you're
not interested in having cluster-level logging, you might still find
the description of how logs are stored and handled on the node to be useful.
Cluster-level logging architectures require a separate backend to store, analyze, and query logs. Kubernetes
does not provide a native storage solution for log data. Instead, there are many logging solutions that
integrate with Kubernetes. The following sections describe how to handle and store logs on nodes.
## Basic logging in Kubernetes
In this section, you can see an example of basic logging in Kubernetes that
outputs data to the standard output stream. This demonstration uses
a pod specification with a container that writes some text to standard output
once per second.
This example uses a`Pod` specification with a container
to write text to the standard output stream once per second.
{{< codenew file="debug/counter-pod.yaml" >}}
@@ -34,8 +33,10 @@ To run this pod, use the following command:
@@ -44,73 +45,73 @@ To fetch the logs, use the `kubectl logs` command, as follows:
```shell
kubectl logs counter
```
The output is:
```
```console
0: Mon Jan 1 00:00:00 UTC 2001
1: Mon Jan 1 00:00:01 UTC 2001
2: Mon Jan 1 00:00:02 UTC 2001
...
```
You can use `kubectl logs` to retrieve logs from a previous instantiation of a container with `--previous` flag, in case the container has crashed. If your pod has multiple containers, you should specify which container's logs you want to access by appending a container name to the command. See the [`kubectl logs` documentation](/docs/reference/generated/kubectl/kubectl-commands#logs) for more details.
You can use `kubectl logs --previous` to retrieve logs from a previous instantiation of a container. If your pod has multiple containers, specify which container's logs you want to access by appending a container name to the command. See the [`kubectl logs` documentation](/docs/reference/generated/kubectl/kubectl-commands#logs) for more details.
Everything a containerized application writes to `stdout` and `stderr` is handled and redirected somewhere by a container engine. For example, the Docker container engine redirects those two streams to [a logging driver](https://docs.docker.com/engine/admin/logging/overview), which is configured in Kubernetes to write to a file in json format.
A container engine handles and redirects any output generated to a containerized application's `stdout` and `stderr` streams.
For example, the Docker container engine redirects those two streams to [a logging driver](https://docs.docker.com/engine/admin/logging/overview), which is configured in Kubernetes to write to a file in JSON format.
{{< note >}}
The Docker json logging driver treats each line as a separate message. When using the Docker logging driver, there is no direct support for multi-line messages. You need to handle multi-line messages at the logging agent level or higher.
The Docker JSON logging driver treats each line as a separate message. When using the Docker logging driver, there is no direct support for multi-line messages. You need to handle multi-line messages at the logging agent level or higher.
{{< /note >}}
By default, if a container restarts, the kubelet keeps one terminated container with its logs. If a pod is evicted from the node, all corresponding containers are also evicted, along with their logs.
An important consideration in node-level logging is implementing log rotation,
so that logs don't consume all available storage on the node. Kubernetes
currently is not responsible for rotating logs, but rather a deployment tool
is not responsible for rotating logs, but rather a deployment tool
should set up a solution to address that.
For example, in Kubernetes clusters, deployed by the `kube-up.sh` script,
there is a [`logrotate`](https://linux.die.net/man/8/logrotate)
tool configured to run each hour. You can also set up a container runtime to
rotate application's logs automatically, for example by using Docker's `log-opt`.
In the `kube-up.sh` script, the latter approach is used for COS image on GCP,
and the former approach is used in any other environment. In both cases, by
default rotation is configured to take place when log file exceeds 10MB.
rotate an application's logs automatically.
As an example, you can find detailed information about how `kube-up.sh` sets
up logging for COS image on GCP in the corresponding
There are two types of system components: those that run in a container and those
that do not run in a container. For example:
* The Kubernetes scheduler and kube-proxy run in a container.
* The kubelet and container runtime, for example Docker, do not run in containers.
* The kubelet and container runtime do not run in containers.
On machines with systemd, the kubelet and container runtime write to journald. If
systemd is not present, they write to `.log` files in the `/var/log` directory.
System components inside containers always write to the `/var/log` directory,
bypassing the default logging mechanism. They use the [klog](https://github.com/kubernetes/klog)
systemd is not present, the kubelet and container runtime write to `.log` files
in the `/var/log` directory. System components inside containers always write
to the `/var/log` directory, bypassing the default logging mechanism.
They use the [`klog`](https://github.com/kubernetes/klog)
logging library. You can find the conventions for logging severity for those
components in the [development docs on logging](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-instrumentation/logging.md).
Similarly to the container logs, system component logs in the `/var/log`
Similar to the container logs, system component logs in the `/var/log`
directory should be rotated. In Kubernetes clusters brought up by
the `kube-up.sh` script, those logs are configured to be rotated by
the `logrotate` tool daily or once the size exceeds 100MB.
@@ -129,13 +130,14 @@ While Kubernetes does not provide a native solution for cluster-level logging, t
You can implement cluster-level logging by including a _node-level logging agent_ on each node. The logging agent is a dedicated tool that exposes logs or pushes logs to a backend. Commonly, the logging agent is a container that has access to a directory with log files from all of the application containers on that node.
Because the logging agent must run on every node, it's common to implement it as either a DaemonSet replica, a manifest pod, or a dedicated native process on the node. However the latter two approaches are deprecated and highly discouraged.
Because the logging agent must run on every node, it is recommended to run the agent
as a `DaemonSet`.
Using a node-level logging agent is the most common and encouraged approach for a Kubernetes cluster, because it creates only one agent per node, and it doesn't require any changes to the applications running on the node. However, node-level logging _only works for applications' standard output and standard error_.
Node-level logging creates only one agent per node and doesn't require any changes to the applications running on the node.
Kubernetes doesn't specify a logging agent, but two optional logging agents are packaged with the Kubernetes release: [Stackdriver Logging](/docs/tasks/debug-application-cluster/logging-stackdriver/) for use with Google Cloud Platform, and [Elasticsearch](/docs/tasks/debug-application-cluster/logging-elasticsearch-kibana/). You can find more information and instructions in the dedicated documents. Both use [fluentd](https://www.fluentd.org/) with custom configuration as an agent on the node.
Containers write stdout and stderr, but with no agreed format. A node-level agent collects these logs and forwards them for aggregation.
### Using a sidecar container with the logging agent
### Using a sidecar container with the logging agent {#sidecar-container-with-logging-agent}
You can use a sidecar container in one of the following ways:
@@ -146,28 +148,27 @@ You can use a sidecar container in one of the following ways:

By having your sidecar containers stream to their own `stdout` and `stderr`
By having your sidecar containers write to their own `stdout` and `stderr`
streams, you can take advantage of the kubelet and the logging agent that
already run on each node. The sidecar containers read logs from a file, a socket,
or the journald. Each individual sidecar container prints log to its own `stdout`
or `stderr` stream.
or journald. Each sidecar container prints a log to its own `stdout` or `stderr` stream.
This approach allows you to separate several log streams from different
parts of your application, some of which can lack support
for writing to `stdout` or `stderr`. The logic behind redirecting logs
is minimal, so it's hardly a significant overhead. Additionally, because
is minimal, so it's not a significant overhead. Additionally, because
`stdout` and `stderr` are handled by the kubelet, you can use built-in tools
like `kubectl logs`.
Consider the following example. A pod runs a single container, and the container
writes to two different log files, using two different formats. Here's a
For example, a pod runs a single container, and the container
writes to two different log files using two different formats. Here's a
`kubectl` will read any files with suffixes `.yaml`, `.yml`, or `.json`.
It is a recommended practice to put resources related to the same microservice or application tier into the same file, and to group all of the files associated with your application in the same directory. If the tiers of your application bind to each other using DNS, then you can then simply deploy all of the components of your stack en masse.
It is a recommended practice to put resources related to the same microservice or application tier into the same file, and to group all of the files associated with your application in the same directory. If the tiers of your application bind to each other using DNS, you can deploy all of the components of your stack together.
A URL can also be specified as a configuration source, which is handy for deploying directly from configuration files checked into github:
A URL can also be specified as a configuration source, which is handy for deploying directly from configuration files checked into GitHub:
@@ -314,11 +316,12 @@ For more information, please see [annotations](/docs/concepts/overview/working-w
## Scaling your application
When load on your application grows or shrinks, it's easy to scale with `kubectl`. For instance, to decrease the number of nginx replicas from 3 to 1, do:
When load on your application grows or shrinks, use `kubectl` to scale your application. For instance, to decrease the number of nginx replicas from 3 to 1, do:
```shell
kubectl scale deployment/my-nginx --replicas=1
```
```shell
deployment.apps/my-nginx scaled
```
@@ -328,6 +331,7 @@ Now you only have one pod managed by the deployment.
```shell
kubectl get pods -l app=nginx
```
```shell
NAME READY STATUS RESTARTS AGE
my-nginx-2035384211-j5fhi 1/1 Running 0 30m
@@ -338,6 +342,7 @@ To have the system automatically choose the number of nginx replicas as needed,
In some cases, you may need to update resource fields that cannot be updated once initialized, or you may just want to make a recursive change immediately, such as to fix broken pods created by a Deployment. To change such fields, use `replace --force`, which deletes and re-creates the resource. In this case, you can simply modify your original configuration file:
In some cases, you may need to update resource fields that cannot be updated once initialized, or you may want to make a recursive change immediately, such as to fix broken pods created by a Deployment. To change such fields, use `replace --force`, which deletes and re-creates the resource. In this case, you can modify your original configuration file:
To update to version 1.16.1, simply change `.spec.template.spec.containers[0].image` from `nginx:1.14.2` to `nginx:1.16.1`, with the kubectl commands we learned above.
To update to version 1.16.1, change `.spec.template.spec.containers[0].image` from `nginx:1.14.2` to `nginx:1.16.1` using the previous kubectl commands.
@@ -114,7 +114,7 @@ Additionally, the CNI can be run alongside [Calico for network policy enforcemen
### Azure CNI for Kubernetes
[Azure CNI](https://docs.microsoft.com/en-us/azure/virtual-network/container-networking-overview) is an [open source](https://github.com/Azure/azure-container-networking/blob/master/docs/cni.md) plugin that integrates Kubernetes Pods with an Azure Virtual Network (also known as VNet) providing network performance at par with VMs. Pods can connect to peered VNet and to on-premises over Express Route or site-to-site VPN and are also directly reachable from these networks. Pods can access Azure services, such as storage and SQL, that are protected by Service Endpoints or Private Link. You can use VNet security policies and routing to filter Pod traffic. The plugin assigns VNet IPs to Pods by utilizing a pool of secondary IPs pre-configured on the Network Interface of a Kubernetes node.
Azure CNI is available natively in the [Azure Kubernetes Service (AKS)] (https://docs.microsoft.com/en-us/azure/aks/configure-azure-cni).
Azure CNI is available natively in the [Azure Kubernetes Service (AKS)](https://docs.microsoft.com/en-us/azure/aks/configure-azure-cni).
Migration to structured log messages is an ongoing process. Not all log messages are structured in this version. When parsing log files, you must also handle unstructured log messages.
Log formatting and value serialization are subject to change.
{{< /warning>}}
Structured logging is a effort to introduce a uniform structure in log messages allowing for easy extraction of information, making logs easier and cheaper to store and process.
Structured logging introduces a uniform structure in log messages allowing for programmatic extraction of information. You can store and process structured logs with less effort and cost.
New message format is backward compatible and enabled by default.
Alpha metrics have no stability guarantees; as such they can be modified or deleted at any time.
Alpha metrics have no stability guarantees. These metrics can be modified or deleted at any time.
Stable metrics can be guaranteed to not change; Specifically, stability means:
Stable metrics are guaranteed to not change. This means:
* A stable metric without a deprecated signature will not be deleted or renamed
* A stable metric's type will not be modified
* the metric itself will not be deleted (or renamed)
* the type of metric will not be modified
Deprecated metrics are slated for deletion, but are still available for use.
These metrics include an annotation about the version in which they became deprecated.
Deprecated metric signal that the metric will eventually be deleted; to find which version, you need to check annotation, which includes from which kubernetes version that metric will be considered deprecated.
For example:
Before deprecation:
* Before deprecation
```
# HELP some_counter this counts things
# TYPE some_counter counter
some_counter 0
```
```
# HELP some_counter this counts things
# TYPE some_counter counter
some_counter 0
```
After deprecation:
* After deprecation
```
# HELP some_counter (Deprecated since 1.15.0) this counts things
# TYPE some_counter counter
some_counter 0
```
```
# HELP some_counter (Deprecated since 1.15.0) this counts things
# TYPE some_counter counter
some_counter 0
```
Once a metric is hidden then by default the metrics is not published for scraping. To use a hidden metric, you need to override the configuration for the relevant cluster component.
Hidden metrics are no longer published for scraping, but are still available for use. To use a hidden metric, please refer to the [Show hidden metrics](#show-hidden-metrics) section.
Once a metric is deleted, the metric is not published. You cannot change this using an override.
Deleted metrics are no longer published and cannot be used.
## Show Hidden Metrics
## Show hidden metrics
As described above, admins can enable hidden metrics through a command-line flag on a specific binary. This intends to be used as an escape hatch for admins if they missed the migration of the metrics deprecated in the last release.
@@ -154,5 +156,4 @@ endpoint on the scheduler. You must use the `--show-hidden-metrics-for-version=1
## {{% heading "whatsnext" %}}
* Read about the [Prometheus text format](https://github.com/prometheus/docs/blob/master/content/docs/instrumenting/exposition_formats.md#text-based-format) for metrics
* See the list of [stable Kubernetes metrics](https://github.com/kubernetes/kubernetes/blob/master/test/instrumentation/testdata/stable-metrics-list.yaml)
* Read about the [Kubernetes deprecation policy](/docs/reference/using-api/deprecation-policy/#deprecating-a-feature-or-behavior)
@@ -59,16 +59,18 @@ DNS server watches the Kubernetes API for new `Services` and creates a set of DN
- Avoid using `hostNetwork`, for the same reasons as `hostPort`.
- Use [headless Services](/docs/concepts/services-networking/service/#headless-services) (which have a `ClusterIP` of `None`) for easy service discovery when you don't need `kube-proxy` load balancing.
- Use [headless Services](/docs/concepts/services-networking/service/#headless-services) (which have a `ClusterIP` of `None`) for service discovery when you don't need `kube-proxy` load balancing.
## Using Labels
- Define and use [labels](/docs/concepts/overview/working-with-objects/labels/) that identify __semantic attributes__ of your application or Deployment, such as `{ app: myapp, tier: frontend, phase: test, deployment: v3 }`. You can use these labels to select the appropriate Pods for other resources; for example, a Service that selects all `tier: frontend` Pods, or all `phase: test` components of `app: myapp`. See the [guestbook](https://github.com/kubernetes/examples/tree/{{< param "githubbranch" >}}/guestbook/) app for examples of this approach.
A Service can be made to span multiple Deployments by omitting release-specific labels from its selector. [Deployments](/docs/concepts/workloads/controllers/deployment/) make it easy to update a running service without downtime.
A Service can be made to span multiple Deployments by omitting release-specific labels from its selector. When you need to update a running service without downtime, use a [Deployment](/docs/concepts/workloads/controllers/deployment/).
A desired state of an object is described by a Deployment, and if changes to that spec are _applied_, the deployment controller changes the actual state to the desired state at a controlled rate.
- Use the [Kubernetes common labels](/docs/concepts/overview/working-with-objects/common-labels/) for common use cases. These standardized labels enrich the metadata in a way that allows tools, including `kubectl` and [dashboard](/docs/tasks/access-application-cluster/web-ui-dashboard), to work in an interoperable way.
- You can manipulate labels for debugging. Because Kubernetes controllers (such as ReplicaSet) and Services match to Pods using selector labels, removing the relevant labels from a Pod will stop it from being considered by a controller or from being served traffic by a Service. If you remove the labels of an existing Pod, its controller will create a new Pod to take its place. This is a useful way to debug a previously "live" Pod in a "quarantine" environment. To interactively remove or add labels, use [`kubectl label`](/docs/reference/generated/kubectl/kubectl-commands#label).
## Container Images
@@ -79,9 +81,9 @@ The [imagePullPolicy](/docs/concepts/containers/images/#updating-images) and the
- `imagePullPolicy: Always`: every time the kubelet launches a container, the kubelet queries the container image registry to resolve the name to an image digest. If the kubelet has a container image with that exact digest cached locally, the kubelet uses its cached image; otherwise, the kubelet downloads (pulls) the image with the resolved digest, and uses that image to launch the container.
- `imagePullPolicy` is omitted and either the image tag is `:latest` or it is omitted: `Always` is applied.
- `imagePullPolicy` is omitted and either the image tag is `:latest` or it is omitted: `imagePullPolicy` is automatically set to `Always`. Note that this will _not_ be updated to `IfNotPresent` if the tag changes value.
- `imagePullPolicy` is omitted and the image tag is present but not `:latest`: `IfNotPresent` is applied.
- `imagePullPolicy` is omitted and the image tag is present but not `:latest`: `imagePullPolicy` is automatically set to `IfNotPresent`. Note that this will _not_ be updated to `Always` if the tag is later removed or changed to `:latest`.
- `imagePullPolicy: Never`: the image is assumed to exist locally. No attempt is made to pull the image.
@@ -94,7 +96,7 @@ You should avoid using the `:latest` tag when deploying containers in production
{{< /note >}}
{{< note >}}
The caching semantics of the underlying image provider make even `imagePullPolicy: Always` efficient. With Docker, for example, if the image already exists, the pull attempt is fast because all image layers are cached and no image download is needed.
The caching semantics of the underlying image provider make even `imagePullPolicy: Always` efficient, as long as the registry is reliably accessible. With Docker, for example, if the image already exists, the pull attempt is fast because all image layers are cached and no image download is needed.
@@ -24,6 +24,16 @@ a password, a token, or a key. Such information might otherwise be put in a
Pod specification or in an image. Users can create Secrets and the system
also creates some Secrets.
{{< caution >}}
Kubernetes Secrets are, by default, stored as unencrypted base64-encoded
strings. By default they can be retrieved - as plain text - by anyone with API
access, or anyone with access to Kubernetes' underlying data store, etcd. In
order to safely use Secrets, it is recommended you (at a minimum):
1. [Enable Encryption at Rest](/docs/tasks/administer-cluster/encrypt-data/) for Secrets.
2. [Enable or configure RBAC rules](/docs/reference/access-authn-authz/authorization/) that restrict reading and writing the Secret. Be aware that secrets can be obtained implicitly by anyone with the permission to create a Pod.
{{< /caution >}}
<!-- body -->
## Overview of Secrets
@@ -99,14 +109,14 @@ empty-secret Opaque 0 2m6s
```
The `DATA` column shows the number of data items stored in the Secret.
In this case, `0` means we have just created an empty Secret.
In this case, `0` means we have created an empty Secret.
### Service account token Secrets
A `kubernetes.io/service-account-token` type of Secret is used to store a
token that identifies a service account. When using this Secret type, you need
to ensure that the `kubernetes.io/service-account.name` annotation is set to an
existing service account name. An Kubernetes controller fills in some other
existing service account name. A Kubernetes controller fills in some other
fields such as the `kubernetes.io/service-account.uid` annotation and the
`token` key in the `data` field set to actual token content.
@@ -271,9 +281,16 @@ However, using the builtin Secret type helps unify the formats of your credentia
and the API server does verify if the required keys are provided in a Secret
configuration.
{{< caution >}}
SSH private keys do not establish trusted communication between an SSH client and
host server on their own. A secondary means of establishing trust is needed to
mitigate "man in the middle" attacks, such as a `known_hosts` file added to a
ConfigMap.
{{< /caution >}}
### TLS secrets
Kubernetes provides a builtin Secret type `kubernetes.io/tls` for to storing
Kubernetes provides a builtin Secret type `kubernetes.io/tls` for storing
a certificate and its associated key that are typically used for TLS . This
data is primarily used with TLS termination of the Ingress resource, but may
be used with other resources or directly by a workload.
@@ -351,7 +368,7 @@ data:
A bootstrap type Secret has the following keys specified under `data`:
- `token_id`: A random 6 character string as the token identifier. Required.
- `token-id`: A random 6 character string as the token identifier. Required.
- `token-secret`: A random 16 character string as the actual token secret. Required.
- `description`: A human-readable string that describes what the token is
used for. Optional.
@@ -652,7 +669,7 @@ The kubelet checks whether the mounted secret is fresh on every periodic sync.
However, the kubelet uses its local cache for getting the current value of the Secret.
The type of the cache is configurable using the `ConfigMapAndSecretChangeDetectionStrategy` field in
the [KubeletConfiguration struct](https://github.com/kubernetes/kubernetes/blob/{{< param "docsbranch" >}}/staging/src/k8s.io/kubelet/config/v1beta1/types.go).
A Secret can be either propagated by watch (default), ttl-based, or simply redirecting
A Secret can be either propagated by watch (default), ttl-based, or by redirecting
all requests directly to the API server.
As a result, the total delay from the moment when the Secret is updated to the moment
when new keys are projected to the Pod can be as long as the kubelet sync period + cache
@@ -701,7 +718,7 @@ spec:
#### Consuming Secret Values from environment variables
Inside a container that consumes a secret in an environment variables, the secret keys appear as
Inside a container that consumes a secret in the environment variables, the secret keys appear as
normal environment variables containing the base64 decoded values of the secret data.
This is the result of commands executed inside the container from the example above:
@@ -725,6 +742,11 @@ The output is similar to:
1f2d1e2e67df
```
#### Environment variables are not updated after a secret update
If a container already consumes a Secret in an environment variable, a Secret update will not be seen by the container unless it is restarted.
There are third party solutions for triggering restarts when secrets change.
The `imagePullSecrets` field is a list of references to secrets in the same namespace.
You can use an `imagePullSecrets` to pass a secret that contains a Docker (or other) image registry
password to the kubelet. The kubelet uses this information to pull a private image on behalf of your Pod.
See the [PodSpec API](/docs/reference/generated/kubernetes-api/{{< latest-version >}}/#podspec-v1-core) for more information about the `imagePullSecrets` field.
See the [PodSpec API](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#podspec-v1-core) for more information about the `imagePullSecrets` field.
#### Manually specifying an imagePullSecret
@@ -779,12 +801,6 @@ field set to that of the service account.
See [Add ImagePullSecrets to a service account](/docs/tasks/configure-pod-container/configure-service-account/#add-imagepullsecrets-to-a-service-account)
for a detailed explanation of that process.
### Automatic mounting of manually created Secrets
Manually created secrets (for example, one containing a token for accessing a GitHub account)
can be automatically attached to pods based on their service account.
See [Injecting Information into Pods Using a PodPreset](/docs/tasks/inject-data-application/podpreset/) for a detailed explanation of that process.
@@ -36,10 +36,13 @@ No parameters are passed to the handler.
`PreStop`
This hook is called immediately before a container is terminated due to an API request or management event such as liveness probe failure, preemption, resource contention and others. A call to the preStop hook fails if the container is already in terminated or completed state.
It is blocking, meaning it is synchronous,
so it must complete before the signal to stop the container can be sent.
No parameters are passed to the handler.
This hook is called immediately before a container is terminated due to an API request or management
event such as a liveness/startup probe failure, preemption, resource contention and others. A call
to the `PreStop` hook fails if the container is already in a terminated or completed state and the
hook must complete before the TERM signal to stop the container can be sent. The Pod's termination
grace period countdown begins before the `PreStop` hook is executed, so regardless of the outcome of
the handler, the container will eventually terminate within the Pod's termination grace period. No
parameters are passed to the handler.
A more detailed description of the termination behavior can be found in
[Termination of Pods](/docs/concepts/workloads/pods/pod-lifecycle/#pod-termination).
@@ -57,7 +60,7 @@ Resources consumed by the command are counted against the Container.
When a Container lifecycle management hook is called,
the Kubernetes management system execute the handler according to the hook action,
`exec` and `tcpSocket` are executed in the container, and `httpGet` is executed by the kubelet process.
`httpGet` and `tcpSocket` are executed by the kubelet process, and `exec` is executed in the container.
Hook handler calls are synchronous within the context of the Pod containing the Container.
This means that for a `PostStart` hook,
@@ -65,19 +68,15 @@ the Container ENTRYPOINT and hook fire asynchronously.
However, if the hook takes too long to run or hangs,
the Container cannot reach a `running` state.
`PreStop` hooks are not executed asynchronously from the signal
to stop the Container; the hook must complete its execution before
the signal can be sent.
If a `PreStop` hook hangs during execution,
the Pod's phase will be `Terminating` and remain there until the Pod is
killed after its `terminationGracePeriodSeconds`expires.
This grace period applies to the total time it takes for both
the `PreStop` hook to execute and for the Container to stop normally.
If, for example, `terminationGracePeriodSeconds` is 60, and the hook
takes 55 seconds to complete, and the Container takes 10 seconds to stop
normally after receiving the signal, then the Container will be killed
before it can stop normally, since `terminationGracePeriodSeconds` is
less than the total time (55+10) it takes for these two things to happen.
`PreStop` hooks are not executed asynchronously from the signal to stop the Container; the hook must
complete its execution before the TERM signal can be sent. If a `PreStop` hook hangs during
execution, the Pod's phase will be `Terminating` and remain there until the Pod is killed after its
`terminationGracePeriodSeconds` expires. This grace period applies to the total time it takes for
both the `PreStop` hook to execute and for the Container to stop normally. If, for example,
`terminationGracePeriodSeconds`is 60, and the hook takes 55 seconds to complete, and the Container
takes 10 seconds to stop normally after receiving the signal, then the Container will be killed
before it can stop normally, since `terminationGracePeriodSeconds` is less than the total time
@@ -49,16 +49,32 @@ Instead, specify a meaningful tag such as `v1.42.0`.
## Updating images
The default pull policy is `IfNotPresent` which causes the
{{< glossary_tooltip text="kubelet" term_id="kubelet" >}} to skip
pulling an image if it already exists. If you would like to always force a pull,
you can do one of the following:
When you first create a {{< glossary_tooltip text="Deployment" term_id="deployment" >}},
{{< glossary_tooltip text="StatefulSet" term_id="statefulset" >}}, Pod, or other
object that includes a Pod template, then by default the pull policy of all
containers in that pod will be set to `IfNotPresent` if it is not explicitly
specified. This policy causes the
{{< glossary_tooltip text="kubelet" term_id="kubelet" >}} to skip pulling an
image if it already exists.
If you would like to always force a pull, you can do one of the following:
- set the `imagePullPolicy` of the container to `Always`.
- omit the `imagePullPolicy` and use `:latest` as the tag for the image to use.
- omit the `imagePullPolicy` and use `:latest` as the tag for the image to use;
Kubernetes will set the policy to `Always`.
- omit the `imagePullPolicy` and the tag for the image to use.
- enable the [AlwaysPullImages](/docs/reference/access-authn-authz/admission-controllers/#alwayspullimages) admission controller.
{{< note >}}
The value of `imagePullPolicy` of the container is always set when the object is
first _created_, and is not updated if the image's tag later changes.
For example, if you create a Deployment with an image whose tag is _not_
`:latest`, and later update that Deployment's image to a `:latest` tag, the
`imagePullPolicy` field will _not_ change to `Always`. You must manually change
the pull policy of any object after its initial creation.
{{< /note >}}
When `imagePullPolicy` is defined without a specific value, it is also set to `Always`.
## Multi-architecture images with image indexes
@@ -119,7 +135,7 @@ Here are the recommended steps to configuring your nodes to use a private regist
example, run these on your desktop/laptop:
1. Run `docker login [server]` for each set of credentials you want to use. This updates `$HOME/.docker/config.json` on your PC.
1. View `$HOME/.docker/config.json` in an editor to ensure it contains just the credentials you want to use.
1. View `$HOME/.docker/config.json` in an editor to ensure it contains only the credentials you want to use.
1. Get a list of your nodes; for example:
- if you want the names: `nodes=$( kubectl get nodes -o jsonpath='{range.items[*].metadata}{.name} {end}' )`
- if you want to get the IP addresses: `nodes=$( kubectl get nodes -o jsonpath='{range .items[*].status.addresses[?(@.type=="ExternalIP")]}{.address} {end}' )`
@@ -145,7 +145,7 @@ Kubernetes provides several built-in authentication methods, and an [Authenticat
### Authorization
[Authorization](/docs/reference/access-authn-authz/webhook/) determines whether specific users can read, write, and do other operations on API resources. It just works at the level of whole resources -- it doesn't discriminate based on arbitrary object fields. If the built-in authorization options don't meet your needs, and [Authorization webhook](/docs/reference/access-authn-authz/webhook/) allows calling out to user-provided code to make an authorization decision.
[Authorization](/docs/reference/access-authn-authz/webhook/) determines whether specific users can read, write, and do other operations on API resources. It works at the level of whole resources -- it doesn't discriminate based on arbitrary object fields. If the built-in authorization options don't meet your needs, and [Authorization webhook](/docs/reference/access-authn-authz/webhook/) allows calling out to user-provided code to make an authorization decision.
@@ -31,7 +31,7 @@ Once a custom resource is installed, users can create and access its objects usi
## Custom controllers
On their own, custom resources simply let you store and retrieve structured data.
On their own, custom resources let you store and retrieve structured data.
When you combine a custom resource with a *custom controller*, custom resources
provide a true _declarative API_.
@@ -120,7 +120,7 @@ Kubernetes provides two ways to add custom resources to your cluster:
Kubernetes provides these two options to meet the needs of different users, so that neither ease of use nor flexibility is compromised.
Aggregated APIs are subordinate API servers that sit behind the primary API server, which acts as a proxy. This arrangement is called [API Aggregation](/docs/concepts/extend-kubernetes/api-extension/apiserver-aggregation/) (AA). To users, it simply appears that the Kubernetes API is extended.
Aggregated APIs are subordinate API servers that sit behind the primary API server, which acts as a proxy. This arrangement is called [API Aggregation](/docs/concepts/extend-kubernetes/api-extension/apiserver-aggregation/) (AA). To users, the Kubernetes API appears extended.
CRDs allow users to create new types of resources without adding another API server. You do not need to understand API Aggregation to use CRDs.
Support for the "PodResources service" requires `KubeletPodResources` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/) to be enabled.
@@ -24,7 +24,7 @@ Network plugins in Kubernetes come in a few flavors:
The kubelet has a single default network plugin, and a default network common to the entire cluster. It probes for plugins when it starts up, remembers what it finds, and executes the selected plugin at appropriate times in the pod lifecycle (this is only true for Docker, as CRI manages its own CNI plugins). There are two Kubelet command line parameters to keep in mind when using plugins:
* `cni-bin-dir`: Kubelet probes this directory for plugins on startup
* `network-plugin`: The network plugin to use from `cni-bin-dir`. It must match the name reported by a plugin probed from the plugin directory. For CNI plugins, this is simply "cni".
* `network-plugin`: The network plugin to use from `cni-bin-dir`. It must match the name reported by a plugin probed from the plugin directory. For CNI plugins, this is `cni`.
## Network Plugin Requirements
@@ -159,7 +159,7 @@ This option is provided to the network-plugin; currently **only kubenet supports
## Usage Summary
* `--network-plugin=cni` specifies that we use the `cni` network plugin with actual CNI plugin binaries located in `--cni-bin-dir` (default `/opt/cni/bin`) and CNI plugin configuration located in `--cni-conf-dir` (default `/etc/cni/net.d`).
* `--network-plugin=kubenet` specifies that we use the `kubenet` network plugin with CNI `bridge` and `host-local` plugins placed in `/opt/cni/bin` or `cni-bin-dir`.
* `--network-plugin=kubenet` specifies that we use the `kubenet` network plugin with CNI `bridge`, `lo` and `host-local` plugins placed in `/opt/cni/bin` or `cni-bin-dir`.
* `--network-plugin-mtu=9001` specifies the MTU to use, currently only used by the `kubenet` network plugin.
and [Network Plugins](/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins/))
and by kubectl.
@@ -146,7 +146,7 @@ Kubernetes provides several built-in authentication methods, and an [Authenticat
### Authorization
[Authorization](/docs/reference/access-authn-authz/webhook/) determines whether specific users can read, write, and do other operations on API resources. It just works at the level of whole resources -- it doesn't discriminate based on arbitrary object fields. If the built-in authorization options don't meet your needs, and [Authorization webhook](/docs/reference/access-authn-authz/webhook/) allows calling out to user-provided code to make an authorization decision.
[Authorization](/docs/reference/access-authn-authz/webhook/) determines whether specific users can read, write, and do other operations on API resources. It works at the level of whole resources -- it doesn't discriminate based on arbitrary object fields. If the built-in authorization options don't meet your needs, and [Authorization webhook](/docs/reference/access-authn-authz/webhook/) allows calling out to user-provided code to make an authorization decision.
### Dynamic Admission Control
@@ -161,7 +161,7 @@ After a request is authorized, if it is a write operation, it also goes through
* Learn more about [Custom Resources](/docs/concepts/extend-kubernetes/api-extension/custom-resources/)
* Find ready-made operators on [OperatorHub.io](https://operatorhub.io/) to suit your use case
* Use existing tools to write your own operator, eg:
* using [KUDO](https://kudo.dev/) (Kubernetes Universal Declarative Operator)
* using [kubebuilder](https://book.kubebuilder.io/)
* using [Metacontroller](https://metacontroller.app/) along with WebHooks that
you implement yourself
* using the [Operator Framework](https://operatorframework.io)
* [Publish](https://operatorhub.io/) your operator for other people to use
* Read [CoreOS' original article](https://coreos.com/blog/introducing-operators.html) that introduced the Operator pattern
* Read [CoreOS' original article](https://web.archive.org/web/20170129131616/https://coreos.com/blog/introducing-operators.html) that introduced the Operator pattern (this is an archived version of the original article).
* Read an [article](https://cloud.google.com/blog/products/containers-kubernetes/best-practices-for-building-kubernetes-operators-and-stateful-apps) from Google Cloud about best practices for building Operators
@@ -26,7 +26,7 @@ Fortunately, there is a cloud provider that offers message queuing as a managed
A cluster operator can setup Service Catalog and use it to communicate with the cloud provider's service broker to provision an instance of the message queuing service and make it available to the application within the Kubernetes cluster.
The application developer therefore does not need to be concerned with the implementation details or management of the message queue.
The application can simply use it as a service.
The application can access the message queue as a service.
@@ -43,7 +43,7 @@ Each VM is a full machine running all the components, including its own operatin
Containers have become popular because they provide extra benefits, such as:
* Agile application creation and deployment: increased ease and efficiency of container image creation compared to VM image use.
* Continuous development, integration, and deployment: provides for reliable and frequent container image build and deployment with quick and easy rollbacks (due to image immutability).
* Continuous development, integration, and deployment: provides for reliable and frequent container image build and deployment with quick and efficient rollbacks (due to image immutability).
* Dev and Ops separation of concerns: create application container images at build/release time rather than deployment time, thereby decoupling applications from infrastructure.
* Observability not only surfaces OS-level information and metrics, but also application health and other signals.
* Environmental consistency across development, testing, and production: Runs the same on a laptop as it does in the cloud.
These are just examples of commonly used labels; you are free to develop your own conventions. Keep in mind that label Key must be unique for a given object.
These are examples of commonly used labels; you are free to develop your own conventions. Keep in mind that label Key must be unique for a given object.
## Syntax and character set
@@ -52,7 +52,10 @@ If the prefix is omitted, the label Key is presumed to be private to the user. A
The `kubernetes.io/` and `k8s.io/` prefixes are reserved for Kubernetes core components.
Valid label values must be 63 characters or less and must be empty or begin and end with an alphanumeric character (`[a-z0-9A-Z]`) with dashes (`-`), underscores (`_`), dots (`.`), and alphanumerics between.
Valid label value:
* must be 63 characters or less (cannot be empty),
* must begin and end with an alphanumeric character (`[a-z0-9A-Z]`),
* could contain dashes (`-`), underscores (`_`), dots (`.`), and alphanumerics between.
For example, here's the configuration file for a Pod that has two labels `environment: production` and `app: nginx` :
@@ -98,7 +101,7 @@ For both equality-based and set-based conditions there is no logical _OR_ (`||`)
### _Equality-based_ requirement
_Equality-_ or _inequality-based_ requirements allow filtering by label keys and values. Matching objects must satisfy all of the specified label constraints, though they may have additional labels as well.
Three kinds of operators are admitted `=`,`==`,`!=`. The first two represent _equality_ (and are simply synonyms), while the latter represents _inequality_. For example:
Three kinds of operators are admitted `=`,`==`,`!=`. The first two represent _equality_ (and are synonyms), while the latter represents _inequality_. For example:
@@ -33,16 +33,16 @@ On certain Linux installations, the operating system sets the PIDs limit to a lo
such as `32768`. Consider raising the value of `/proc/sys/kernel/pid_max`.
{{< /note >}}
You can configure a kubelet to limit the number of PIDs a given pod can consume.
You can configure a kubelet to limit the number of PIDs a given Pod can consume.
For example, if your node's host OS is set to use a maximum of `262144` PIDs and
expect to host less than `250`pods, one can give each pod a budget of `1000`
expect to host less than `250`Pods, one can give each Pod a budget of `1000`
PIDs to prevent using up that node's overall number of available PIDs. If the
admin wants to overcommit PIDs similar to CPU or memory, they may do so as well
with some additional risks. Either way, a single pod will not be able to bring
with some additional risks. Either way, a single Pod will not be able to bring
the whole machine down. This kind of resource limiting helps to prevent simple
fork bombs from affecting operation of an entire cluster.
Per-pod PID limiting allows administrators to protect one pod from another, but
Per-Pod PID limiting allows administrators to protect one Pod from another, but
does not ensure that all Pods scheduled onto that host are unable to impact the node overall.
Per-Pod limiting also does not protect the node agents themselves from PID exhaustion.
@@ -92,8 +92,26 @@ the [feature gate](/docs/reference/command-line-tools-reference/feature-gates/)
`SupportPodPidsLimit` to work.
{{< /note >}}
## PID based eviction
You can configure kubelet to start terminating a Pod when it is misbehaving and consuming abnormal amount of resources.
This feature is called eviction. You can [Configure Out of Resource Handling](/docs/tasks/administer-cluster/out-of-resource) for various eviction signals.
Use `pid.available` eviction signal to configure the threshold for number of PIDs used by Pod.
You can set soft and hard eviction policies. However, even with the hard eviction policy, if the number of PIDs growing very fast,
node can still get into unstable state by hitting the node PIDs limit.
Eviction signal value is calculated periodically and does NOT enforce the limit.
PID limiting - per Pod and per Node sets the hard limit.
Once the limit is hit, workload will start experiencing failures when trying to get a new PID.
It may or may not lead to rescheduling of a Pod,
depending on how workload reacts on these failures and how liveleness and readiness
probes are configured for the Pod. However, if limits were set correctly,
you can guarantee that other Pods workload and system processes will not run out of PIDs
when one Pod is misbehaving.
## {{% heading "whatsnext" %}}
- Refer to the [PID Limiting enhancement document](https://github.com/kubernetes/enhancements/blob/097b4d8276bc9564e56adf72505d43ce9bc5e9e8/keps/sig-node/20190129-pid-limiting.md) for more information.
- For historical context, read [Process ID Limiting for Stability Improvements in Kubernetes 1.14](/blog/2019/04/15/process-id-limiting-for-stability-improvements-in-kubernetes-1.14/).
- Read [Managing Resources for Containers](/docs/concepts/configuration/manage-resources-containers/).
- Learn how to [Configure Out of Resource Handling](/docs/tasks/administer-cluster/out-of-resource).
Error from server (Forbidden): error when creating "STDIN": pods "privileged" is forbidden: unable to validate against any pod security policy: [spec.containers[0].securityContext.privileged: Invalid value: true: Privileged containers are not allowed]
1. the Pod's `priorityClassName` is not specified.
1. the Pod's `priorityClassName` is specified to a value other than `cluster-services`.
1. the Pod's `priorityClassName` is set to `cluster-services`, it is to be created
in the `kube-system` namespace, and it has passed the resource quota check.
A Pod creation request is rejected if its `priorityClassName` is set to `cluster-services`
and it is to be created in a namespace other than `kube-system`.
## {{% heading "whatsnext" %}}
- See [ResourceQuota design doc](https://git.k8s.io/community/contributors/design-proposals/resource-management/admission_control_resource_quota.md) for more information.
for many more examples of pod affinity and anti-affinity, both the `requiredDuringSchedulingIgnoredDuringExecution`
@@ -261,7 +261,7 @@ for performance and security reasons, there are some constraints on topologyKey:
and `preferredDuringSchedulingIgnoredDuringExecution`.
2. For pod anti-affinity, empty `topologyKey` is also not allowed in both `requiredDuringSchedulingIgnoredDuringExecution`
and `preferredDuringSchedulingIgnoredDuringExecution`.
3. For `requiredDuringSchedulingIgnoredDuringExecution` pod anti-affinity, the admission controller `LimitPodHardAntiAffinityTopology` was introduced to limit `topologyKey` to `kubernetes.io/hostname`. If you want to make it available for custom topologies, you may modify the admission controller, or simply disable it.
3. For `requiredDuringSchedulingIgnoredDuringExecution` pod anti-affinity, the admission controller `LimitPodHardAntiAffinityTopology` was introduced to limit `topologyKey` to `kubernetes.io/hostname`. If you want to make it available for custom topologies, you may modify the admission controller, or disable it.
4. Except for the above cases, the `topologyKey` can be any legal label-key.
In addition to `labelSelector` and `topologyKey`, you can optionally specify a list `namespaces`
The scheduling framework is a pluggable architecture for Kubernetes Scheduler
that makes scheduler customizations easy. It adds a new set of "plugin" APIs to
the existing scheduler. Plugins are compiled into the scheduler. The APIs
allow most scheduling features to be implemented as plugins, while keeping the
scheduling "core" simple and maintainable. Refer to the [design proposal of the
The scheduling framework is a pluggable architecture for the Kubernetes scheduler.
It adds a new set of "plugin" APIs to the existing scheduler. Plugins are compiled into the scheduler. The APIs allow most scheduling features to be implemented as plugins, while keeping the
scheduling "core" lightweight and maintainable. Refer to the [design proposal of the
scheduling framework][kep] for more technical information on the design of the
framework.
@@ -185,7 +183,7 @@ the three things:
{{< note >}}
While any plugin can access the list of "waiting" Pods and approve them
(see [`FrameworkHandle`](https://github.com/kubernetes/enhancements/blob/master/keps/sig-scheduling/20180409-scheduling-framework.md#frameworkhandle)), we expect only the permit
(see [`FrameworkHandle`](https://git.k8s.io/enhancements/keps/sig-scheduling/624-scheduling-framework#frameworkhandle)), we expect only the permit
plugins to approve binding of reserved Pods that are in "waiting" state. Once a Pod
is approved, it is sent to the [PreBind](#pre-bind) phase.
{{< /note >}}
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.