Merge remote-tracking branch 'upstream/master' into dev-1.19

This commit is contained in:
Savitha Raghunathan
2020-07-27 19:10:42 -04:00
556 changed files with 9174 additions and 6944 deletions
@@ -64,7 +64,7 @@ Before choosing a guide, here are some considerations:
* [Auditing](/docs/tasks/debug-application-cluster/audit/) describes how to interact with Kubernetes' audit logs.
### Securing the kubelet
* [Master-Node communication](/docs/concepts/architecture/master-node-communication/)
* [Control Plane-Node communication](/docs/concepts/architecture/control-plane-node-communication/)
* [TLS bootstrapping](/docs/reference/command-line-tools-reference/kubelet-tls-bootstrapping/)
* [Kubelet authentication/authorization](/docs/admin/kubelet-authentication-authorization/)
@@ -303,6 +303,9 @@ to get a mapping of UIDs to names for both FlowSchemas and
PriorityLevelConfigurations.
## Observability
### Metrics
When you enable the API Priority and Fairness feature, the kube-apiserver
exports additional metrics. Monitoring these can help you determine whether your
configuration is inappropriately throttling important traffic, or find
@@ -365,9 +368,65 @@ poorly-behaved workloads that may be harming system health.
long requests took to actually execute, grouped by the FlowSchema that matched the
request and the PriorityLevel to which it was assigned.
### Debug endpoints
When you enable the API Priority and Fairness feature, the kube-apiserver serves the following additional paths at its HTTP[S] ports.
- `/debug/api_priority_and_fairness/dump_priority_levels` - a listing of all the priority levels and the current state of each. You can fetch like this:
```shell
kubectl get --raw /debug/api_priority_and_fairness/dump_priority_levels
```
The output is similar to this:
```
PriorityLevelName, ActiveQueues, IsIdle, IsQuiescing, WaitingRequests, ExecutingRequests,
workload-low, 0, true, false, 0, 0,
global-default, 0, true, false, 0, 0,
exempt, <none>, <none>, <none>, <none>, <none>,
catch-all, 0, true, false, 0, 0,
system, 0, true, false, 0, 0,
leader-election, 0, true, false, 0, 0,
workload-high, 0, true, false, 0, 0,
```
- `/debug/api_priority_and_fairness/dump_queues` - a listing of all the queues and their current state. You can fetch like this:
```shell
kubectl get --raw /debug/api_priority_and_fairness/dump_queues
```
The output is similar to this:
```
PriorityLevelName, Index, PendingRequests, ExecutingRequests, VirtualStart,
workload-high, 0, 0, 0, 0.0000,
workload-high, 1, 0, 0, 0.0000,
workload-high, 2, 0, 0, 0.0000,
...
leader-election, 14, 0, 0, 0.0000,
leader-election, 15, 0, 0, 0.0000,
```
- `/debug/api_priority_and_fairness/dump_requests` - a listing of all the requests that are currently waiting in a queue. You can fetch like this:
```shell
kubectl get --raw /debug/api_priority_and_fairness/dump_requests
```
The output is similar to this:
```
PriorityLevelName, FlowSchemaName, QueueIndex, RequestIndexInQueue, FlowDistingsher, ArriveTime,
exempt, <none>, <none>, <none>, <none>, <none>,
system, system-nodes, 12, 0, system:node:127.0.0.1, 2020-07-23T15:26:57.179170694Z,
```
In addition to the queued requests, the output includeas one phantom line for each priority level that is exempt from limitation.
You can get a more detailed listing with a command like this:
```shell
kubectl get --raw '/debug/api_priority_and_fairness/dump_requests?includeRequestDetails=1'
```
The output is similar to this:
```
PriorityLevelName, FlowSchemaName, QueueIndex, RequestIndexInQueue, FlowDistingsher, ArriveTime, UserName, Verb, APIPath, Namespace, Name, APIVersion, Resource, SubResource,
system, system-nodes, 12, 0, system:node:127.0.0.1, 2020-07-23T15:31:03.583823404Z, system:node:127.0.0.1, create, /api/v1/namespaces/scaletest/configmaps,
system, system-nodes, 12, 1, system:node:127.0.0.1, 2020-07-23T15:31:03.594555947Z, system:node:127.0.0.1, create, /api/v1/namespaces/scaletest/configmaps,
```
## {{% heading "whatsnext" %}}
@@ -323,7 +323,7 @@ When load on your application grows or shrinks, it's easy to scale with `kubectl
kubectl scale deployment/my-nginx --replicas=1
```
```shell
deployment.extensions/my-nginx scaled
deployment.apps/my-nginx scaled
```
Now you only have one pod managed by the deployment.
@@ -12,7 +12,7 @@ understand exactly how it is expected to work. There are 4 distinct networking
problems to address:
1. Highly-coupled container-to-container communications: this is solved by
[pods](/docs/concepts/workloads/pods/pod/) and `localhost` communications.
{{< glossary_tooltip text="Pods" term_id="pod" >}} and `localhost` communications.
2. Pod-to-Pod communications: this is the primary focus of this document.
3. Pod-to-Service communications: this is covered by [services](/docs/concepts/services-networking/service/).
4. External-to-Service communications: this is covered by [services](/docs/concepts/services-networking/service/).
@@ -224,7 +224,7 @@ data has the following advantages:
- improves performance of your cluster by significantly reducing load on kube-apiserver, by
closing watches for config maps marked as immutable.
To use this feature, enable the `ImmutableEmphemeralVolumes`
To use this feature, enable the `ImmutableEphemeralVolumes`
[feature gate](/docs/reference/command-line-tools-reference/feature-gates/) and set
your Secret or ConfigMap `immutable` field to `true`. For example:
```yaml
@@ -132,11 +132,9 @@ metadata:
name: frontend
spec:
containers:
- name: db
image: mysql
- name: app
image: images.my-company.example/app:v4
env:
- name: MYSQL_ROOT_PASSWORD
value: "password"
resources:
requests:
memory: "64Mi"
@@ -144,8 +142,8 @@ spec:
limits:
memory: "128Mi"
cpu: "500m"
- name: wp
image: wordpress
- name: log-aggregator
image: images.my-company.example/log-aggregator:v6
resources:
requests:
memory: "64Mi"
@@ -330,18 +328,15 @@ metadata:
name: frontend
spec:
containers:
- name: db
image: mysql
env:
- name: MYSQL_ROOT_PASSWORD
value: "password"
- name: app
image: images.my-company.example/app:v4
resources:
requests:
ephemeral-storage: "2Gi"
limits:
ephemeral-storage: "4Gi"
- name: wp
image: wordpress
- name: log-aggregator
image: images.my-company.example/log-aggregator:v6
resources:
requests:
ephemeral-storage: "2Gi"
@@ -217,7 +217,7 @@ makes Pod P eligible to preempt Pods on another Node.
#### Graceful termination of preemption victims
When Pods are preempted, the victims get their
[graceful termination period](/docs/concepts/workloads/pods/pod/#termination-of-pods).
[graceful termination period](/docs/concepts/workloads/pods/pod-lifecycle/#pod-termination).
They have that much time to finish their work and exit. If they don't, they are
killed. This graceful termination period creates a time gap between the point
that the scheduler preempts Pods and the time when the pending Pod (P) can be
@@ -230,7 +230,7 @@ priority Pods to zero or a small number.
#### PodDisruptionBudget is supported, but not guaranteed
A [Pod Disruption Budget (PDB)](/docs/concepts/workloads/pods/disruptions/)
A [PodDisruptionBudget](/docs/concepts/workloads/pods/disruptions/) (PDB)
allows application owners to limit the number of Pods of a replicated application
that are down simultaneously from voluntary disruptions. Kubernetes supports
PDB when preempting Pods, but respecting PDB is best effort. The scheduler tries
@@ -42,7 +42,7 @@ so it must complete before the call to delete the container can be sent.
No parameters are passed to the handler.
A more detailed description of the termination behavior can be found in
[Termination of Pods](/docs/concepts/workloads/pods/pod/#termination-of-pods).
[Termination of Pods](/docs/concepts/workloads/pods/pod-lifecycle/#pod-termination).
### Hook handler implementations
@@ -129,7 +129,7 @@ example, run these on your desktop/laptop:
- for example, to test this out: `for n in $nodes; do scp ~/.docker/config.json root@"$n":/var/lib/kubelet/config.json; done`
{{< note >}}
For production clusers, use a configuration management tool so that you can apply this
For production clusters, use a configuration management tool so that you can apply this
setting to all the nodes where you need it.
{{< /note >}}
@@ -1,112 +0,0 @@
---
title: Poseidon-Firmament Scheduler
content_type: concept
weight: 80
---
<!-- overview -->
{{< feature-state for_k8s_version="v1.6" state="alpha" >}}
The Poseidon-Firmament scheduler is an alternate scheduler that can be deployed alongside the default Kubernetes scheduler.
<!-- body -->
## Introduction
Poseidon is a service that acts as the integration glue between the [Firmament scheduler](https://github.com/Huawei-PaaS/firmament) and Kubernetes. Poseidon-Firmament augments the current Kubernetes scheduling capabilities. It incorporates novel flow network graph based scheduling capabilities alongside the default Kubernetes scheduler. The Firmament scheduler models workloads and clusters as flow networks and runs min-cost flow optimizations over these networks to make scheduling decisions.
Firmament models the scheduling problem as a constraint-based optimization over a flow network graph. This is achieved by reducing scheduling to a min-cost max-flow optimization problem. The Poseidon-Firmament scheduler dynamically refines the workload placements.
Poseidon-Firmament scheduler runs alongside the default Kubernetes scheduler as an alternate scheduler. You can simultaneously run multiple, different schedulers.
Flow graph scheduling with the Poseidon-Firmament scheduler provides the following advantages:
- Workloads (Pods) are bulk scheduled to enable scheduling at massive scale.
The Poseidon-Firmament scheduler outperforms the Kubernetes default scheduler by a wide margin when it comes to throughput performance for scenarios where compute resource requirements are somewhat uniform across your workload (Deployments, ReplicaSets, Jobs).
- The Poseidon-Firmament's scheduler's end-to-end throughput performance and bind time improves as the number of nodes in a cluster increases. As you scale out, Poseidon-Firmament scheduler is able to amortize more and more work across workloads.
- Scheduling in Poseidon-Firmament is dynamic; it keeps cluster resources in a global optimal state during every scheduling run.
- The Poseidon-Firmament scheduler supports scheduling complex rule constraints.
## How the Poseidon-Firmament scheduler works
Kubernetes supports [using multiple schedulers](/docs/tasks/administer-cluster/configure-multiple-schedulers/). You can specify, for a particular Pod, that it is scheduled by a custom scheduler (“poseidon” for this case), by setting the `schedulerName` field in the PodSpec at the time of pod creation. The default scheduler will ignore that Pod and allow Poseidon-Firmament scheduler to schedule the Pod on a relevant node.
For example:
```yaml
apiVersion: v1
kind: Pod
...
spec:
schedulerName: poseidon
...
```
## Batch scheduling
As mentioned earlier, Poseidon-Firmament scheduler enables an extremely high throughput scheduling environment at scale due to its bulk scheduling approach versus Kubernetes pod-at-a-time approach. In our extensive tests, we have observed substantial throughput benefits as long as resource requirements (CPU/Memory) for incoming Pods are uniform across jobs (Replicasets/Deployments/Jobs), mainly due to efficient amortization of work across jobs.
Although, Poseidon-Firmament scheduler is capable of scheduling various types of workloads, such as service, batch, etc., the following are a few use cases where it excels the most:
1. For “Big Data/AI” jobs consisting of large number of tasks, throughput benefits are tremendous.
2. Service or batch jobs where workload resource requirements are uniform across jobs (Replicasets/Deployments/Jobs).
## Feature state
Poseidon-Firmament is designed to work with Kubernetes release 1.6 and all subsequent releases.
{{< caution >}}
Poseidon-Firmament scheduler does not provide support for high availability; its implementation assumes that the scheduler cannot fail.
{{< /caution >}}
## Feature comparison {#feature-comparison-matrix}
{{< table caption="Feature comparison of Kubernetes and Poseidon-Firmament schedulers." >}}
|Feature|Kubernetes Default Scheduler|Poseidon-Firmament Scheduler|Notes|
|--- |--- |--- |--- |
|Node Affinity/Anti-Affinity|Y|Y||
|Pod Affinity/Anti-Affinity - including support for pod anti-affinity symmetry|Y|Y|The default scheduler outperforms the Poseidon-Firmament scheduler pod affinity/anti-affinity functionality.|
|Taints & Tolerations|Y|Y||
|Baseline Scheduling capability in accordance to available compute resources (CPU & Memory) on a node|Y|Y†|**†** Not all Predicates & Priorities are supported with Poseidon-Firmament.|
|Extreme Throughput at scale|Y†|Y|**†** Bulk scheduling approach scales or increases workload placement. Firmament scheduler offers high throughput when resource requirements (CPU/Memory) for incoming Pods are uniform across ReplicaSets/Deployments/Jobs.|
|Colocation Interference Avoidance|N|N||
|Priority Preemption|Y|N†|**†** Partially exists in Poseidon-Firmament versus extensive support in Kubernetes default scheduler.|
|Inherent Rescheduling|N|Y†|**†** Poseidon-Firmament scheduler supports workload re-scheduling. In each scheduling run, Poseidon-Firmament considers all Pods, including running Pods, and as a result can migrate or evict Pods a globally optimal scheduling environment.|
|Gang Scheduling|N|Y||
|Support for Pre-bound Persistence Volume Scheduling|Y|Y||
|Support for Local Volume & Dynamic Persistence Volume Binding Scheduling|Y|N||
|High Availability|Y|N||
|Real-time metrics based scheduling|N|Y†|**†** Partially supported in Poseidon-Firmament using Heapster (now deprecated) for placing Pods using actual cluster utilization statistics rather than reservations.|
|Support for Max-Pod per node|Y|Y|Poseidon-Firmament scheduler seamlessly co-exists with Kubernetes default scheduler.|
|Support for Ephemeral Storage, in addition to CPU/Memory|Y|Y||
{{< /table >}}
## Installation
The [Poseidon-Firmament installation guide](https://github.com/kubernetes-sigs/poseidon/blob/master/docs/install/README.md#Installation) explains how to deploy Poseidon-Firmament to your cluster.
## Performance comparison
{{< note >}}
Please refer to the [latest benchmark results](https://github.com/kubernetes-sigs/poseidon/blob/master/docs/benchmark/README.md) for detailed throughput performance comparison test results between Poseidon-Firmament scheduler and the Kubernetes default scheduler.
{{< /note >}}
Pod-by-pod schedulers, such as the Kubernetes default scheduler, process Pods in small batches (typically one at a time). These schedulers have the following crucial drawbacks:
1. The scheduler commits to a pod placement early and restricts the choices for other pods that wait to be placed.
2. There is limited opportunities for amortizing work across pods because they are considered for placement individually.
These downsides of pod-by-pod schedulers are addressed by batching or bulk scheduling in Poseidon-Firmament scheduler. Processing several pods in a batch allows the scheduler to jointly consider their placement, and thus to find the best trade-off for the whole batch instead of one pod. At the same time it amortizes work across pods resulting in much higher throughput.
## {{% heading "whatsnext" %}}
* See [Poseidon-Firmament](https://github.com/kubernetes-sigs/poseidon#readme) on GitHub for more information.
* See the [design document](https://github.com/kubernetes-sigs/poseidon/blob/master/docs/design/README.md) for Poseidon.
* Read [Firmament: Fast, Centralized Cluster Scheduling at Scale](https://www.usenix.org/system/files/conference/osdi16/osdi16-gog.pdf), the academic paper on the Firmament scheduling design.
* If you'd like to contribute to Poseidon-Firmament, refer to the [developer setup instructions](https://github.com/kubernetes-sigs/poseidon/blob/master/docs/devel/README.md).
@@ -25,8 +25,6 @@ The Kubernetes API lets you query and manipulate the state of objects in the Kub
API endpoints, resource types and samples are described in the [API Reference](/docs/reference/kubernetes-api/).
<!-- body -->
## API changes
@@ -87,7 +85,7 @@ Kubernetes implements an alternative Protobuf based serialization format for the
To make it easier to eliminate fields or restructure resource representations, Kubernetes supports
multiple API versions, each at a different API path, such as `/api/v1` or
`/apis/extensions/v1beta1`.
`/apis/rbac.authorization.k8s.io/v1alpha1`.
Versioning is done at the API level rather than at the resource or field level to ensure that the
API presents a clear, consistent view of system resources and behavior, and to enable controlling
@@ -157,14 +155,6 @@ The flag accepts comma separated set of key=value pairs describing runtime confi
{{< note >}}Enabling or disabling groups or resources requires restarting the kube-apiserver and the
kube-controller-manager to pick up the `--runtime-config` changes.{{< /note >}}
## Enabling specific resources in the extensions/v1beta1 group
DaemonSets, Deployments, StatefulSet, NetworkPolicies, PodSecurityPolicies and ReplicaSets in the `extensions/v1beta1` API group are disabled by default.
For example: to enable deployments and daemonsets, set
`--runtime-config=extensions/v1beta1/deployments=true,extensions/v1beta1/daemonsets=true`.
{{< note >}}Individual resource enablement/disablement is only supported in the `extensions/v1beta1` API group for legacy reasons.{{< /note >}}
## Persistence
Kubernetes stores its serialized state in terms of the API resources by writing them into
@@ -92,7 +92,7 @@ and the `spec` format for a Deployment can be found in
## {{% heading "whatsnext" %}}
* [Kubernetes API overview](/docs/reference/using-api/api-overview/) explains some more API concepts
* Learn about the most important basic Kubernetes objects, such as [Pod](/docs/concepts/workloads/pods/pod-overview/).
* Learn about the most important basic Kubernetes objects, such as [Pod](/docs/concepts/workloads/pods/).
* Learn about [controllers](/docs/concepts/architecture/controller/) in Kubernetes
@@ -236,11 +236,7 @@ well as lower-trust users.The following listed controls should be enforced/disal
spec.securityContext.supplementalGroups[*]<br>
spec.securityContext.fsGroup<br>
spec.containers[*].securityContext.runAsGroup<br>
spec.containers[*].securityContext.supplementalGroups[*]<br>
spec.containers[*].securityContext.fsGroup<br>
spec.initContainers[*].securityContext.runAsGroup<br>
spec.initContainers[*].securityContext.supplementalGroups[*]<br>
spec.initContainers[*].securityContext.fsGroup<br>
<br><b>Allowed Values:</b><br>
non-zero<br>
undefined / nil (except for `*.runAsGroup`)<br>
@@ -235,7 +235,7 @@ IngressClass resource will ensure that new Ingresses without an
If you have more than one IngressClass marked as the default for your cluster,
the admission controller prevents creating new Ingress objects that don't have
an `ingressClassName` specified. You can resolve this by ensuring that at most 1
IngressClasess are marked as default in your cluster.
IngressClasses are marked as default in your cluster.
{{< /caution >}}
## Types of Ingress
+1 -1
View File
@@ -1356,7 +1356,7 @@ persistent volume:
of a volume. This map must correspond to the map returned in the
`volume.attributes` field of the `CreateVolumeResponse` by the CSI driver as
defined in the [CSI spec](https://github.com/container-storage-interface/spec/blob/master/spec.md#createvolume).
The map is passed to the CSI driver via the `volume_attributes` field in the
The map is passed to the CSI driver via the `volume_context` field in the
`ControllerPublishVolumeRequest`, `NodeStageVolumeRequest`, and
`NodePublishVolumeRequest`.
- `controllerPublishSecretRef`: A reference to the secret object containing
@@ -60,7 +60,7 @@ A DaemonSet also needs a [`.spec`](https://git.k8s.io/community/contributors/dev
The `.spec.template` is one of the required fields in `.spec`.
The `.spec.template` is a [pod template](/docs/concepts/workloads/pods/pod-overview/#pod-templates). It has exactly the same schema as a [Pod](/docs/concepts/workloads/pods/pod/), except it is nested and does not have an `apiVersion` or `kind`.
The `.spec.template` is a [pod template](/docs/concepts/workloads/pods/#pod-templates). It has exactly the same schema as a {{< glossary_tooltip text="Pod" term_id="pod" >}}, except it is nested and does not have an `apiVersion` or `kind`.
In addition to required fields for a Pod, a Pod template in a DaemonSet has to specify appropriate
labels (see [pod selector](#pod-selector)).
@@ -13,8 +13,8 @@ weight: 30
<!-- overview -->
A _Deployment_ provides declarative updates for [Pods](/docs/concepts/workloads/pods/pod/) and
[ReplicaSets](/docs/concepts/workloads/controllers/replicaset/).
A _Deployment_ provides declarative updates for {{< glossary_tooltip text="Pods" term_id="pod" >}}
{{< glossary_tooltip term_id="replica-set" text="ReplicaSets" >}}.
You describe a _desired state_ in a Deployment, and the Deployment {{< glossary_tooltip term_id="controller" >}} changes the actual state to the desired state at a controlled rate. You can define Deployments to create new ReplicaSets, or to remove existing Deployments and adopt all their resources with new Deployments.
@@ -23,8 +23,6 @@ Do not manage ReplicaSets owned by a Deployment. Consider opening an issue in th
{{< /note >}}
<!-- body -->
## Use Case
@@ -1053,8 +1051,7 @@ A Deployment also needs a [`.spec` section](https://git.k8s.io/community/contrib
The `.spec.template` and `.spec.selector` are the only required field of the `.spec`.
The `.spec.template` is a [Pod template](/docs/concepts/workloads/pods/pod-overview/#pod-templates). It has exactly the same schema as a [Pod](/docs/concepts/workloads/pods/pod/), except it is nested and does not have an
`apiVersion` or `kind`.
The `.spec.template` is a [Pod template](/docs/concepts/workloads/pods/#pod-templates). It has exactly the same schema as a {{< glossary_tooltip text="Pod" term_id="pod" >}}, except it is nested and does not have an `apiVersion` or `kind`.
In addition to required fields for a Pod, a Pod template in a Deployment must specify appropriate
labels and an appropriate restart policy. For labels, make sure not to overlap with other controllers. See [selector](#selector)).
@@ -1155,10 +1152,6 @@ created Pod should be ready without any of its containers crashing, for it to be
This defaults to 0 (the Pod will be considered available as soon as it is ready). To learn more about when
a Pod is considered ready, see [Container Probes](/docs/concepts/workloads/pods/pod-lifecycle/#container-probes).
### Rollback To
Field `.spec.rollbackTo` has been deprecated in API versions `extensions/v1beta1` and `apps/v1beta1`, and is no longer supported in API versions starting `apps/v1beta2`. Instead, `kubectl rollout undo` as introduced in [Rolling Back to a Previous Revision](#rolling-back-to-a-previous-revision) should be used.
### Revision History Limit
A Deployment's revision history is stored in the ReplicaSets it controls.
@@ -111,12 +111,6 @@ To control the cascading deletion policy, set the `propagationPolicy`
field on the `deleteOptions` argument when deleting an Object. Possible values include "Orphan",
"Foreground", or "Background".
Prior to Kubernetes 1.9, the default garbage collection policy for many controller resources was `orphan`.
This included ReplicationController, ReplicaSet, StatefulSet, DaemonSet, and
Deployment. For kinds in the `extensions/v1beta1`, `apps/v1beta1`, and `apps/v1beta2` group versions, unless you
specify otherwise, dependent objects are orphaned by default. In Kubernetes 1.9, for all kinds in the `apps/v1`
group version, dependent objects are deleted by default.
Here's an example that deletes dependents in background:
```shell
@@ -122,7 +122,7 @@ A Job also needs a [`.spec` section](https://git.k8s.io/community/contributors/d
The `.spec.template` is the only required field of the `.spec`.
The `.spec.template` is a [pod template](/docs/concepts/workloads/pods/pod-overview/#pod-templates). It has exactly the same schema as a [pod](/docs/user-guide/pods), except it is nested and does not have an `apiVersion` or `kind`.
The `.spec.template` is a [pod template](/docs/concepts/workloads/pods/#pod-templates). It has exactly the same schema as a {{< glossary_tooltip text="Pod" term_id="pod" >}}, except it is nested and does not have an `apiVersion` or `kind`.
In addition to required fields for a Pod, a pod template in a Job must specify appropriate
labels (see [pod selector](#pod-selector)) and an appropriate restart policy.
@@ -126,7 +126,7 @@ A ReplicationController also needs a [`.spec` section](https://git.k8s.io/commun
The `.spec.template` is the only required field of the `.spec`.
The `.spec.template` is a [pod template](/docs/concepts/workloads/pods/pod-overview/#pod-templates). It has exactly the same schema as a [pod](/docs/concepts/workloads/pods/pod/), except it is nested and does not have an `apiVersion` or `kind`.
The `.spec.template` is a [pod template](/docs/concepts/workloads/pods/#pod-templates). It has exactly the same schema as a {{< glossary_tooltip text="Pod" term_id="pod" >}}, except it is nested and does not have an `apiVersion` or `kind`.
In addition to required fields for a Pod, a pod template in a ReplicationController must specify appropriate
labels and an appropriate restart policy. For labels, make sure not to overlap with other controllers. See [pod selector](#pod-selector).
+267 -1
View File
@@ -1,5 +1,271 @@
---
title: "Pods"
reviewers:
- erictune
title: Pods
content_type: concept
weight: 10
no_list: true
card:
name: concepts
weight: 60
---
<!-- overview -->
_Pods_ are the smallest deployable units of computing that you can create and manage in Kubernetes.
A _Pod_ (as in a pod of whales or pea pod) is a group of one or more
{{< glossary_tooltip text="containers" term_id="container" >}}, with shared storage/network resources, and a specification
for how to run the containers. A Pod's contents are always co-located and
co-scheduled, and run in a shared context. A Pod models an
application-specific "logical host": it contains one or more application
containers which are relatively tightly coupled.
In non-cloud contexts, applications executed on the same physical or virtual machine are analogous to cloud applications executed on the same logical host.
As well as application containers, a Pod can contain
[init containers](/docs/concepts/workloads/pods/init-containers/) that run
during Pod startup. You can also inject
[ephemeral containers](/docs/concepts/workloads/pods/ephemeral-containers/)
for debugging if your cluster offers this.
<!-- body -->
## What is a Pod?
{{< note >}}
While Kubernetes supports more
{{< glossary_tooltip text="container runtimes" term_id="container-runtime" >}}
than just Docker, [Docker](https://www.docker.com/) is the most commonly known
runtime, and it helps to describe Pods using some terminology from Docker.
{{< /note >}}
The shared context of a Pod is a set of Linux namespaces, cgroups, and
potentially other facets of isolation - the same things that isolate a Docker
container. Within a Pod's context, the individual applications may have
further sub-isolations applied.
In terms of Docker concepts, a Pod is similar to a group of Docker containers
with shared namespaces and shared filesystem volumes.
## Using Pods
Usually you don't need to create Pods directly, even singleton Pods. Instead, create them using workload resources such as {{< glossary_tooltip text="Deployment"
term_id="deployment" >}} or {{< glossary_tooltip text="Job" term_id="job" >}}.
If your Pods need to track state, consider the
{{< glossary_tooltip text="StatefulSet" term_id="statefulset" >}} resource.
Pods in a Kubernetes cluster are used in two main ways:
* **Pods that run a single container**. The "one-container-per-Pod" model is the
most common Kubernetes use case; in this case, you can think of a Pod as a
wrapper around a single container; Kubernetes manages Pods rather than managing
the containers directly.
* **Pods that run multiple containers that need to work together**. A Pod can
encapsulate an application composed of multiple co-located containers that are
tightly coupled and need to share resources. These co-located containers
form a single cohesive unit of service—for example, one container serving data
stored in a shared volume to the public, while a separate _sidecar_ container
refreshes or updates those files.
The Pod wraps these containers, storage resources, and an ephemeral network
identity together as a single unit.
{{< note >}}
Grouping multiple co-located and co-managed containers in a single Pod is a
relatively advanced use case. You should use this pattern only in specific
instances in which your containers are tightly coupled.
{{< /note >}}
Each Pod is meant to run a single instance of a given application. If you want to
scale your application horizontally (to provide more overall resources by running
more instances), you should use multiple Pods, one for each instance. In
Kubernetes, this is typically referred to as _replication_.
Replicated Pods are usually created and managed as a group by a workload resource
and its {{< glossary_tooltip text="controller" term_id="controller" >}}.
See [Pods and controllers](#pods-and-controllers) for more information on how
Kubernetes uses workload resources, and their controllers, to implement application
scaling and auto-healing.
### How Pods manage multiple containers
Pods are designed to support multiple cooperating processes (as containers) that form
a cohesive unit of service. The containers in a Pod are automatically co-located and
co-scheduled on the same physical or virtual machine in the cluster. The containers
can share resources and dependencies, communicate with one another, and coordinate
when and how they are terminated.
For example, you might have a container that
acts as a web server for files in a shared volume, and a separate "sidecar" container
that updates those files from a remote source, as in the following diagram:
{{< figure src="/images/docs/pod.svg" alt="example pod diagram" width="50%" >}}
Some Pods have {{< glossary_tooltip text="init containers" term_id="init-container" >}} as well as {{< glossary_tooltip text="app containers" term_id="app-container" >}}. Init containers run and complete before the app containers are started.
Pods natively provide two kinds of shared resources for their constituent containers:
[networking](#pod-networking) and [storage](#pod-storage).
## Working with Pods
You'll rarely create individual Pods directly in Kubernetes—even singleton Pods. This
is because Pods are designed as relatively ephemeral, disposable entities. When
a Pod gets created (directly by you, or indirectly by a
{{< glossary_tooltip text="controller" term_id="controller" >}}), the new Pod is
scheduled to run on a {{< glossary_tooltip term_id="node" >}} in your cluster.
The Pod remains on that node until the Pod finishes execution, the Pod object is deleted,
the Pod is *evicted* for lack of resources, or the node fails.
{{< note >}}
Restarting a container in a Pod should not be confused with restarting a Pod. A Pod
is not a process, but an environment for running container(s). A Pod persists until
it is deleted.
{{< /note >}}
When you create the manifest for a Pod object, make sure the name specified is a valid
[DNS subdomain name](/docs/concepts/overview/working-with-objects/names#dns-subdomain-names).
### Pods and controllers
You can use workload resources to create and manage multiple Pods for you. A controller
for the resource handles replication and rollout and automatic healing in case of
Pod failure. For example, if a Node fails, a controller notices that Pods on that
Node have stopped working and creates a replacement Pod. The scheduler places the
replacement Pod onto a healthy Node.
Here are some examples of workload resources that manage one or more Pods:
* {{< glossary_tooltip text="Deployment" term_id="deployment" >}}
* {{< glossary_tooltip text="StatefulSet" term_id="statefulset" >}}
* {{< glossary_tooltip text="DaemonSet" term_id="daemonset" >}}
### Pod templates
Controllers for {{< glossary_tooltip text="workload" term_id="workload" >}} resources create Pods
from a _pod template_ and manage those Pods on your behalf.
PodTemplates are specifications for creating Pods, and are included in workload resources such as
[Deployments](/docs/concepts/workloads/controllers/deployment/),
[Jobs](/docs/concepts/jobs/run-to-completion-finite-workloads/), and
[DaemonSets](/docs/concepts/workloads/controllers/daemonset/).
Each controller for a workload resource uses the `PodTemplate` inside the workload
object to make actual Pods. The `PodTemplate` is part of the desired state of whatever
workload resource you used to run your app.
The sample below is a manifest for a simple Job with a `template` that starts one
container. The container in that Pod prints a message then pauses.
```yaml
apiVersion: batch/v1
kind: Job
metadata:
name: hello
spec:
template:
# This is the pod template
spec:
containers:
- name: hello
image: busybox
command: ['sh', '-c', 'echo "Hello, Kubernetes!" && sleep 3600']
restartPolicy: OnFailure
# The pod template ends here
```
Modifying the pod template or switching to a new pod template has no effect on the
Pods that already exist. Pods do not receive template updates directly. Instead,
a new Pod is created to match the revised pod template.
For example, the deployment controller ensures that the running Pods match the current
pod template for each Deployment object. If the template is updated, the Deployment has
to remove the existing Pods and create new Pods based on the updated template. Each workload
resource implements its own rules for handling changes to the Pod template.
On Nodes, the {{< glossary_tooltip term_id="kubelet" text="kubelet" >}} does not
directly observe or manage any of the details around pod templates and updates; those
details are abstracted away. That abstraction and separation of concerns simplifies
system semantics, and makes it feasible to extend the cluster's behavior without
changing existing code.
## Resource sharing and communication
Pods enable data sharing and communication among their constituent
containters.
### Storage in Pods {#pod-storage}
A Pod can specify a set of shared storage
{{< glossary_tooltip text="volumes" term_id="volume" >}}. All containers
in the Pod can access the shared volumes, allowing those containers to
share data. Volumes also allow persistent data in a Pod to survive
in case one of the containers within needs to be restarted. See
[Storage](/docs/concepts/storage/) for more information on how
Kubernetes implements shared storage and makes it available to Pods.
### Pod networking
Each Pod is assigned a unique IP address for each address family. Every
container in a Pod shares the network namespace, including the IP address and
network ports. Inside a Pod (and **only** then), the containers that belong to the Pod
can communicate with one another using `localhost`. When containers in a Pod communicate
with entities *outside the Pod*,
they must coordinate how they use the shared network resources (such as ports).
Within a Pod, containers share an IP address and port space, and
can find each other via `localhost`. The containers in a Pod can also communicate
with each other using standard inter-process communications like SystemV semaphores
or POSIX shared memory. Containers in different Pods have distinct IP addresses
and can not communicate by IPC without
[special configuration](/docs/concepts/policy/pod-security-policy/).
Containers that want to interact with a container running in a different Pod can
use IP networking to comunicate.
Containers within the Pod see the system hostname as being the same as the configured
`name` for the Pod. There's more about this in the [networking](/docs/concepts/cluster-administration/networking/)
section.
## Privileged mode for containers
Any container in a Pod can enable privileged mode, using the `privileged` flag on the [security context](/docs/tasks/configure-pod-container/security-context/) of the container spec. This is useful for containers that want to use operating system administrative capabilities such as manipulating the network stack or accessing hardware devices.
Processes within a privileged container get almost the same privileges that are available to processes outside a container.
{{< note >}}
Your {{< glossary_tooltip text="container runtime" term_id="container-runtime" >}} must support the concept of a privileged container for this setting to be relevant.
{{< /note >}}
## Static Pods
_Static Pods_ are managed directly by the kubelet daemon on a specific node,
without the {{< glossary_tooltip text="API server" term_id="kube-apiserver" >}}
observing them.
Whereas most Pods are managed by the control plane (for example, a
{{< glossary_tooltip text="Deployment" term_id="deployment" >}}), for static
Pods, the kubelet directly supervises each static Pod (and restarts it if it fails).
Static Pods are always bound to one {{< glossary_tooltip term_id="kubelet" >}} on a specific node.
The main use for static Pods is to run a self-hosted control plane: in other words,
using the kubelet to supervise the individual [control plane components](/docs/concepts/overview/components/#control-plane-components).
The kubelet automatically tries to create a {{< glossary_tooltip text="mirror Pod" term_id="mirror-pod" >}}
on the Kubernetes API server for each static Pod.
This means that the Pods running on a node are visible on the API server,
but cannot be controlled from there.
## {{% heading "whatsnext" %}}
* Learn about the [lifecycle of a Pod](/docs/concepts/workloads/pods/pod-lifecycle/).
* Learn about [PodPresets](/docs/concepts/workloads/pods/podpreset/).
* Lean about [RuntimeClass](/docs/concepts/containers/runtime-class/) and how you can use it to
configure different Pods with different container runtime configurations.
* Read about [Pod topology spread constraints](/docs/concepts/workloads/pods/pod-topology-spread-constraints/).
* Read about [PodDisruptionBudget](https://kubernetes.io/docs/concepts/workloads/pods/disruptions/) and how you can use it to manage application availability during disruptions.
* Pod is a top-level resource in the Kubernetes REST API.
The [Pod](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#pod-v1-core)
object definition describes the object in detail.
* [The Distributed System Toolkit: Patterns for Composite Containers](https://kubernetes.io/blog/2015/06/the-distributed-system-toolkit-patterns) explains common layouts for Pods with more than one container.
To understand the context for why Kubernetes wraps a common Pod API in other resources (such as {{< glossary_tooltip text="StatefulSets" term_id="statefulset" >}} or {{< glossary_tooltip text="Deployments" term_id="deployment" >}}, you can read about the prior art, including:
* [Aurora](http://aurora.apache.org/documentation/latest/reference/configuration/#job-schema)
* [Borg](https://research.google.com/pubs/pub43438.html)
* [Marathon](https://mesosphere.github.io/marathon/docs/rest-api.html)
* [Omega](https://research.google/pubs/pub41684/)
* [Tupperware](https://engineering.fb.com/data-center-engineering/tupperware/).
@@ -11,17 +11,15 @@ weight: 60
<!-- overview -->
This guide is for application owners who want to build
highly available applications, and thus need to understand
what types of Disruptions can happen to Pods.
what types of disruptions can happen to Pods.
It is also for Cluster Administrators who want to perform automated
It is also for cluster administrators who want to perform automated
cluster actions, like upgrading and autoscaling clusters.
<!-- body -->
## Voluntary and Involuntary Disruptions
## Voluntary and involuntary disruptions
Pods do not disappear until someone (a person or a controller) destroys them, or
there is an unavoidable hardware or system software error.
@@ -48,7 +46,7 @@ Administrator. Typical application owner actions include:
- updating a deployment's pod template causing a restart
- directly deleting a pod (e.g. by accident)
Cluster Administrator actions include:
Cluster administrator actions include:
- [Draining a node](/docs/tasks/administer-cluster/safely-drain-node/) for repair or upgrade.
- Draining a node from a cluster to scale the cluster down (learn about
@@ -68,7 +66,7 @@ Not all voluntary disruptions are constrained by Pod Disruption Budgets. For exa
deleting deployments or pods bypasses Pod Disruption Budgets.
{{< /caution >}}
## Dealing with Disruptions
## Dealing with disruptions
Here are some ways to mitigate involuntary disruptions:
@@ -90,58 +88,58 @@ of cluster (node) autoscaling may cause voluntary disruptions to defragment and
Your cluster administrator or hosting provider should have documented what level of voluntary
disruptions, if any, to expect.
Kubernetes offers features to help run highly available applications at the same
time as frequent voluntary disruptions. We call this set of features
*Disruption Budgets*.
## How Disruption Budgets Work
## Pod disruption budgets
{{< feature-state for_k8s_version="v1.5" state="beta" >}}
An Application Owner can create a `PodDisruptionBudget` object (PDB) for each application.
A PDB limits the number of pods of a replicated application that are down simultaneously from
voluntary disruptions. For example, a quorum-based application would
Kubernetes offers features to help you run highly available applications even when you
introduce frequent voluntary disruptions.
As an application owner, you can create a PodDisruptionBudget (PDB) for each application.
A PDB limits the number of Pods of a replicated application that are down simultaneously from
voluntary disruptions. For example, a quorum-based application would
like to ensure that the number of replicas running is never brought below the
number needed for a quorum. A web front end might want to
ensure that the number of replicas serving load never falls below a certain
percentage of the total.
Cluster managers and hosting providers should use tools which
respect Pod Disruption Budgets by calling the [Eviction API](/docs/tasks/administer-cluster/safely-drain-node/#the-eviction-api)
instead of directly deleting pods or deployments. Examples are the `kubectl drain` command
and the Kubernetes-on-GCE cluster upgrade script (`cluster/gce/upgrade.sh`).
respect PodDisruptionBudgets by calling the [Eviction API](/docs/tasks/administer-cluster/safely-drain-node/#the-eviction-api)
instead of directly deleting pods or deployments.
When a cluster administrator wants to drain a node
they use the `kubectl drain` command. That tool tries to evict all
the pods on the machine. The eviction request may be temporarily rejected,
and the tool periodically retries all failed requests until all pods
are terminated, or until a configurable timeout is reached.
For example, the `kubectl drain` subcommand lets you mark a node as going out of
service. When you run `kubectl drain`, the tool tries to evict all of the Pods on
the Node you're taking out of service. The eviction request that `kubectl` submits on
your behalf may be temporarily rejected, so the tool periodically retries all failed
requests until all Pods on the target node are terminated, or until a configurable timeout
is reached.
A PDB specifies the number of replicas that an application can tolerate having, relative to how
many it is intended to have. For example, a Deployment which has a `.spec.replicas: 5` is
supposed to have 5 pods at any given time. If its PDB allows for there to be 4 at a time,
then the Eviction API will allow voluntary disruption of one, but not two pods, at a time.
then the Eviction API will allow voluntary disruption of one (but not two) pods at a time.
The group of pods that comprise the application is specified using a label selector, the same
as the one used by the application's controller (deployment, stateful-set, etc).
The "intended" number of pods is computed from the `.spec.replicas` of the pods controller.
The controller is discovered from the pods using the `.metadata.ownerReferences` of the object.
The "intended" number of pods is computed from the `.spec.replicas` of the workload resource
that is managing those pods. The control plane discovers the owning workload resource by
examining the `.metadata.ownerReferences` of the Pod.
PDBs cannot prevent [involuntary disruptions](#voluntary-and-involuntary-disruptions) from
occurring, but they do count against the budget.
Pods which are deleted or unavailable due to a rolling upgrade to an application do count
against the disruption budget, but controllers (like deployment and stateful-set)
are not limited by PDBs when doing rolling upgrades -- the handling of failures
during application updates is configured in the controller spec.
(Learn about [updating a deployment](/docs/concepts/workloads/controllers/deployment/#updating-a-deployment).)
against the disruption budget, but workload resources (such as Deployment and StatefulSet)
are not limited by PDBs when doing rolling upgrades. Instead, the handling of failures
during application updates is configured in the spec for the specific workload resource.
When a pod is evicted using the eviction API, it is gracefully terminated (see
`terminationGracePeriodSeconds` in [PodSpec](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#podspec-v1-core).)
When a pod is evicted using the eviction API, it is gracefully
[terminated](/docs/concepts/workloads/pods/pod-lifecycle/#pod-termination), honoring the
`terminationGracePeriodSeconds` setting in its [PodSpec](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#podspec-v1-core).)
## PDB Example
## PodDisruptionBudget example {#pdb-example}
Consider a cluster with 3 nodes, `node-1` through `node-3`.
The cluster is running several applications. One of them has 3 replicas initially called
@@ -272,4 +270,6 @@ the nodes in your cluster, such as a node or system software upgrade, here are s
* Learn more about [draining nodes](/docs/tasks/administer-cluster/safely-drain-node/)
* Learn about [updating a deployment](/docs/concepts/workloads/controllers/deployment/#updating-a-deployment)
including steps to maintain its availability during the rollout.
@@ -6,16 +6,60 @@ weight: 30
<!-- overview -->
{{< comment >}}Updated: 4/14/2015{{< /comment >}}
{{< comment >}}Edited and moved to Concepts section: 2/2/17{{< /comment >}}
This page describes the lifecycle of a Pod.
This page describes the lifecycle of a Pod. Pods follow a defined lifecycle, starting
in the `Pending` [phase](#pod-phase), moving through `Running` if at least one
of its primary containers starts OK, and then through either the `Succeeded` or
`Failed` phases depending on whether any container in the Pod terminated in failure.
Whilst a Pod is running, the kubelet is able to restart containers to handle some
kind of faults. Within a Pod, Kubernetes tracks different container
[states](#container-states) and handles
In the Kubernetes API, Pods have both a specification and an actual status. The
status for a Pod object consists of a set of [Pod conditions](#pod-conditions).
You can also inject [custom readiness information](#pod-readiness-gate) into the
condition data for a Pod, if that is useful to your application.
Pods are only [scheduled](/docs/concepts/scheduling-eviction/) once in their lifetime.
Once a Pod is scheduled (assigned) to a Node, the Pod runs on that Node until it stops
or is [terminated](#pod-termination).
<!-- body -->
## Pod lifetime
Like individual application containers, Pods are considered to be relatively
ephemeral (rather than durable) entities. Pods are created, assigned a unique
ID ([UID](/docs/concepts/overview/working-with-objects/names/#uids)), and scheduled
to nodes where they remain until termination (according to restart policy) or
deletion.
If a {{< glossary_tooltip term_id="node" >}} dies, the Pods scheduled to that node
are [scheduled for deletion](#pod-garbage-collection) after a timeout period.
Pods do not, by themselves, self-heal. If a Pod is scheduled to a
{{< glossary_tooltip text="node" term_id="node" >}} that then fails,
or if the scheduling operation itself fails, the Pod is deleted; likewise, a Pod won't
survive an eviction due to a lack of resources or Node maintenance. Kubernetes uses a
higher-level abstraction, called a
{{< glossary_tooltip term_id="controller" text="controller" >}}, that handles the work of
managing the relatively disposable Pod instances.
A given Pod (as defined by a UID) is never "rescheduled" to a different node; instead,
that Pod can be replaced by a new, near-identical Pod, with even the same name i
desired, but with a different UID.
When something is said to have the same lifetime as a Pod, such as a
{{< glossary_tooltip term_id="volume" text="volume" >}},
that means that the thing exists as long as that specific Pod (with that exact UID)
exists. If that Pod is deleted for any reason, and even if an identical replacement
is created, the related thing (a volume, in this example) is also destroyed and
created anew.
{{< figure src="/images/docs/pod.svg" title="Pod diagram" width="50%" >}}
*A multi-container Pod that contains a file puller and a
web server that uses a persistent volume for shared storage between the containers.*
## Pod phase
A Pod's `status` field is a
@@ -24,7 +68,7 @@ object, which has a `phase` field.
The phase of a Pod is a simple, high-level summary of where the Pod is in its
lifecycle. The phase is not intended to be a comprehensive rollup of observations
of Container or Pod state, nor is it intended to be a comprehensive state machine.
of container or Pod state, nor is it intended to be a comprehensive state machine.
The number and meanings of Pod phase values are tightly guarded.
Other than what is documented here, nothing should be assumed about Pods that
@@ -34,188 +78,106 @@ Here are the possible values for `phase`:
Value | Description
:-----|:-----------
`Pending` | The Pod has been accepted by the Kubernetes system, but one or more of the Container images has not been created. This includes time before being scheduled as well as time spent downloading images over the network, which could take a while.
`Running` | The Pod has been bound to a node, and all of the Containers have been created. At least one Container is still running, or is in the process of starting or restarting.
`Succeeded` | All Containers in the Pod have terminated in success, and will not be restarted.
`Failed` | All Containers in the Pod have terminated, and at least one Container has terminated in failure. That is, the Container either exited with non-zero status or was terminated by the system.
`Unknown` | For some reason the state of the Pod could not be obtained, typically due to an error in communicating with the host of the Pod.
`Pending` | The Pod has been accepted by the Kubernetes cluster, but one or more of the containers has not been set up and made ready to run. This includes time a Pod spends waiting to bescheduled as well as the time spent downloading container images over the network.
`Running` | The Pod has been bound to a node, and all of the containers have been created. At least one container is still running, or is in the process of starting or restarting.
`Succeeded` | All containers in the Pod have terminated in success, and will not be restarted.
`Failed` | All containers in the Pod have terminated, and at least one container has terminated in failure. That is, the container either exited with non-zero status or was terminated by the system.
`Unknown` | For some reason the state of the Pod could not be obtained. This phase typically occurs due to an error in communicating with the node where the Pod should be running.
If a node dies or is disconnected from the rest of the cluster, Kubernetes
applies a policy for setting the `phase` of all Pods on the lost node to Failed.
## Container states
As well as the [phase](#pod-phase) of the Pod overall, Kubernetes tracks the state of
each container inside a Pod. You can use
[container lifecycle hooks](/docs/concepts/containers/container-lifecycle-hooks/) to
trigger events to run at certain points in a container's lifecycle.
Once the {{< glossary_tooltip text="scheduler" term_id="kube-scheduler" >}}
assigns a Pod to a Node, the kubelet starts creating containers for that Pod
using a {{< glossary_tooltip text="container runtime" term_id="container-runtime" >}}.
There are three possible container states: `Waiting`, `Running`, and `Terminated`.
To the check state of a Pod's containers, you can use
`kubectl describe pod <name-of-pod>`. The output shows the state for each container
within that Pod.
Each state has a specific meaning:
### `Waiting` {#container-state-waiting}
If a container is not in either the `Running` or `Terminated` state, it `Waiting`.
A container in the `Waiting` state is still running the operations it requires in
order to complete start up: for example, pulling the container image from a container
image registry, or applying {{< glossary_tooltip text="Secret" term_id="secret" >}}
data.
When you use `kubectl` to query a Pod with a container that is `Waiting`, you also see
a Reason field to summarize why the container is in that state.
### `Running` {#container-state-running}
The `Running` status indicates that a container is executing without issues. If there
was a `postStart` hook configured, it has already executed and executed. When you use
`kubectl` to query a Pod with a container that is `Running`, you also see information
about when the container entered the `Running` state.
### `Terminated` {#container-state-terminated}
A container in the `Terminated` state has begin execution and has then either run to
completion or has failed for some reason. When you use `kubectl` to query a Pod with
a container that is `Terminated`, you see a reason, and exit code, and the start and
finish time for that container's period of execution.
If a container has a `preStop` hook configured, that runs before the container enters
the `Terminated` state.
## Container restart policy {#restart-policy}
The `spec` of a Pod has a `restartPolicy` field with possible values Always, OnFailure,
and Never. The default value is Always.
The `restartPolicy` applies to all containers in the Pod. `restartPolicy` only
refers to restarts of the containers by the kubelet on the same node. After containers
in a Pod exit, the kubelet restarts them with an exponential back-off delay (10s, 20s,
40s, …), that is capped at five minutes. Once a container has executed with no problems
for 10 minutes without any problems, the kubelet resets the restart backoff timer for
that container.
## Pod conditions
A Pod has a PodStatus, which has an array of
[PodConditions](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#podcondition-v1-core)
through which the Pod has or has not passed. Each element of the PodCondition
array has six possible fields:
through which the Pod has or has not passed:
* The `lastProbeTime` field provides a timestamp for when the Pod condition
was last probed.
* `PodScheduled`: the Pod has been scheduled to a node.
* `ContainersReady`: all containers in the Pod are ready.
* `Initialized`: all [init containers](/docs/concepts/workloads/pods/init-containers/)
have started successfully.
* `Ready`: the Pod is able to serve requests and should be added to the load
balancing pools of all matching Services.
* The `lastTransitionTime` field provides a timestamp for when the Pod
last transitioned from one status to another.
* The `message` field is a human-readable message indicating details
about the transition.
* The `reason` field is a unique, one-word, CamelCase reason for the condition's last transition.
* The `status` field is a string, with possible values "`True`", "`False`", and "`Unknown`".
* The `type` field is a string with the following possible values:
* `PodScheduled`: the Pod has been scheduled to a node;
* `Ready`: the Pod is able to serve requests and should be added to the load
balancing pools of all matching Services;
* `Initialized`: all [init containers](/docs/concepts/workloads/pods/init-containers)
have started successfully;
* `ContainersReady`: all containers in the Pod are ready.
Field name | Description
:--------------------|:-----------
`type` | Name of this Pod condition.
`status` | Indicates whether that condition is applicable, with possible values "`True`", "`False`", or "`Unknown`".
`lastProbeTime` | Timestamp of when the Pod condition was last probed.
`lastTransitionTime` | Timestamp for when the Pod last transitioned from one status to another.
`reason` | Machine-readable, UpperCamelCase text indicating the reason for the condition's last transition.
`message` | Human-readable message indicating details about the last status transition.
## Container probes
A [Probe](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#probe-v1-core) is a diagnostic
performed periodically by the [kubelet](/docs/admin/kubelet/)
on a Container. To perform a diagnostic,
the kubelet calls a
[Handler](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#handler-v1-core) implemented by
the Container. There are three types of handlers:
* [ExecAction](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#execaction-v1-core):
Executes a specified command inside the Container. The diagnostic
is considered successful if the command exits with a status code of 0.
* [TCPSocketAction](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#tcpsocketaction-v1-core):
Performs a TCP check against the Container's IP address on
a specified port. The diagnostic is considered successful if the port is open.
* [HTTPGetAction](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#httpgetaction-v1-core):
Performs an HTTP Get request against the Container's IP
address on a specified port and path. The diagnostic is considered successful
if the response has a status code greater than or equal to 200 and less than 400.
Each probe has one of three results:
* Success: The Container passed the diagnostic.
* Failure: The Container failed the diagnostic.
* Unknown: The diagnostic failed, so no action should be taken.
The kubelet can optionally perform and react to three kinds of probes on running
Containers:
* `livenessProbe`: Indicates whether the Container is running. If
the liveness probe fails, the kubelet kills the Container, and the Container
is subjected to its [restart policy](#restart-policy). If a Container does not
provide a liveness probe, the default state is `Success`.
* `readinessProbe`: Indicates whether the Container is ready to service requests.
If the readiness probe fails, the endpoints controller removes the Pod's IP
address from the endpoints of all Services that match the Pod. The default
state of readiness before the initial delay is `Failure`. If a Container does
not provide a readiness probe, the default state is `Success`.
* `startupProbe`: Indicates whether the application within the Container is started.
All other probes are disabled if a startup probe is provided, until it succeeds.
If the startup probe fails, the kubelet kills the Container, and the Container
is subjected to its [restart policy](#restart-policy). If a Container does not
provide a startup probe, the default state is `Success`.
### When should you use a liveness probe?
{{< feature-state for_k8s_version="v1.0" state="stable" >}}
If the process in your Container is able to crash on its own whenever it
encounters an issue or becomes unhealthy, you do not necessarily need a liveness
probe; the kubelet will automatically perform the correct action in accordance
with the Pod's `restartPolicy`.
If you'd like your Container to be killed and restarted if a probe fails, then
specify a liveness probe, and specify a `restartPolicy` of Always or OnFailure.
### When should you use a readiness probe?
{{< feature-state for_k8s_version="v1.0" state="stable" >}}
If you'd like to start sending traffic to a Pod only when a probe succeeds,
specify a readiness probe. In this case, the readiness probe might be the same
as the liveness probe, but the existence of the readiness probe in the spec means
that the Pod will start without receiving any traffic and only start receiving
traffic after the probe starts succeeding.
If your Container needs to work on loading large data, configuration files, or migrations during startup, specify a readiness probe.
If you want your Container to be able to take itself down for maintenance, you
can specify a readiness probe that checks an endpoint specific to readiness that
is different from the liveness probe.
Note that if you just want to be able to drain requests when the Pod is deleted,
you do not necessarily need a readiness probe; on deletion, the Pod automatically
puts itself into an unready state regardless of whether the readiness probe exists.
The Pod remains in the unready state while it waits for the Containers in the Pod
to stop.
### When should you use a startup probe?
{{< feature-state for_k8s_version="v1.16" state="alpha" >}}
If your Container usually starts in more than `initialDelaySeconds + failureThreshold × periodSeconds`, you should specify a startup probe that checks the same endpoint as the liveness probe. The default for `periodSeconds` is 30s.
You should then set its `failureThreshold` high enough to allow the Container to start, without changing the default values of the liveness probe. This helps to protect against deadlocks.
For more information about how to set up a liveness, readiness, startup probe, see
[Configure Liveness, Readiness and Startup Probes](/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/).
## Pod and Container status
For detailed information about Pod Container status, see
[PodStatus](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#podstatus-v1-core)
and
[ContainerStatus](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#containerstatus-v1-core).
Note that the information reported as Pod status depends on the current
[ContainerState](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#containerstatus-v1-core).
## Container States
Once Pod is assigned to a node by scheduler, kubelet starts creating containers using container runtime.There are three possible states of containers: Waiting, Running and Terminated. To check state of container, you can use `kubectl describe pod [POD_NAME]`. State is displayed for each container within that Pod.
* `Waiting`: Default state of container. If container is not in either Running or Terminated state, it is in Waiting state. A container in Waiting state still runs its required operations, like pulling images, applying Secrets, etc. Along with this state, a message and reason about the state are displayed to provide more information.
```yaml
...
State: Waiting
Reason: ErrImagePull
...
```
* `Running`: Indicates that the container is executing without issues. The `postStart` hook (if any) is executed prior to the container entering a Running state. This state also displays the time when the container entered Running state.
```yaml
...
State: Running
Started: Wed, 30 Jan 2019 16:46:38 +0530
...
```
* `Terminated`: Indicates that the container completed its execution and has stopped running. A container enters into this when it has successfully completed execution or when it has failed for some reason. Regardless, a reason and exit code is displayed, as well as the container's start and finish time. Before a container enters into Terminated, `preStop` hook (if any) is executed.
```yaml
...
State: Terminated
Reason: Completed
Exit Code: 0
Started: Wed, 30 Jan 2019 11:45:26 +0530
Finished: Wed, 30 Jan 2019 11:45:26 +0530
...
```
## Pod readiness {#pod-readiness-gate}
### Pod readiness {#pod-readiness-gate}
{{< feature-state for_k8s_version="v1.14" state="stable" >}}
Your application can inject extra feedback or signals into PodStatus:
_Pod readiness_. To use this, set `readinessGates` in the PodSpec to specify
a list of additional conditions that the kubelet evaluates for Pod readiness.
_Pod readiness_. To use this, set `readinessGates` in the Pod's `spec` to
specify a list of additional conditions that the kubelet evaluates for Pod readiness.
Readiness gates are determined by the current state of `status.condition`
fields for the Pod. If Kubernetes cannot find such a
condition in the `status.conditions` field of a Pod, the status of the condition
fields for the Pod. If Kubernetes cannot find such a condition in the
`status.conditions` field of a Pod, the status of the condition
is defaulted to "`False`".
Here is an example:
@@ -258,152 +220,226 @@ For a Pod that uses custom conditions, that Pod is evaluated to be ready **only*
when both the following statements apply:
* All containers in the Pod are ready.
* All conditions specified in `ReadinessGates` are `True`.
* All conditions specified in `readinessGates` are `True`.
When a Pod's containers are Ready but at least one custom condition is missing or
`False`, the kubelet sets the Pod's condition to `ContainersReady`.
`False`, the kubelet sets the Pod's [condition](#pod-condition) to `ContainersReady`.
## Restart policy
## Container probes
A PodSpec has a `restartPolicy` field with possible values Always, OnFailure,
and Never. The default value is Always.
`restartPolicy` applies to all Containers in the Pod. `restartPolicy` only
refers to restarts of the Containers by the kubelet on the same node. Exited
Containers that are restarted by the kubelet are restarted with an exponential
back-off delay (10s, 20s, 40s ...) capped at five minutes, and is reset after ten
minutes of successful execution. As discussed in the
[Pods document](/docs/user-guide/pods/#durability-of-pods-or-lack-thereof),
once bound to a node, a Pod will never be rebound to another node.
A [Probe](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#probe-v1-core) is a diagnostic
performed periodically by the [kubelet](/docs/admin/kubelet/)
on a Container. To perform a diagnostic,
the kubelet calls a
[Handler](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#handler-v1-core) implemented by
the container. There are three types of handlers:
* [ExecAction](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#execaction-v1-core):
Executes a specified command inside the container. The diagnostic
is considered successful if the command exits with a status code of 0.
## Pod lifetime
* [TCPSocketAction](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#tcpsocketaction-v1-core):
Performs a TCP check against the Pod's IP address on
a specified port. The diagnostic is considered successful if the port is open.
In general, Pods remain until a human or
* [HTTPGetAction](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#httpgetaction-v1-core):
Performs an HTTP `GET` request against the Pod's IP
address on a specified port and path. The diagnostic is considered successful
if the response has a status code greater than or equal to 200 and less than 400.
Each probe has one of three results:
* `Success`: The container passed the diagnostic.
* `Failure`: The container failed the diagnostic.
* `Unknown`: The diagnostic failed, so no action should be taken.
The kubelet can optionally perform and react to three kinds of probes on running
containers:
* `livenessProbe`: Indicates whether the container is running. If
the liveness probe fails, the kubelet kills the container, and the container
is subjected to its [restart policy](#restart-policy). If a Container does not
provide a liveness probe, the default state is `Success`.
* `readinessProbe`: Indicates whether the container is ready to respond to requests.
If the readiness probe fails, the endpoints controller removes the Pod's IP
address from the endpoints of all Services that match the Pod. The default
state of readiness before the initial delay is `Failure`. If a Container does
not provide a readiness probe, the default state is `Success`.
* `startupProbe`: Indicates whether the application within the container is started.
All other probes are disabled if a startup probe is provided, until it succeeds.
If the startup probe fails, the kubelet kills the container, and the container
is subjected to its [restart policy](#restart-policy). If a Container does not
provide a startup probe, the default state is `Success`.
For more information about how to set up a liveness, readiness, or startup probe,
see [Configure Liveness, Readiness and Startup Probes](/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/).
### When should you use a liveness probe?
{{< feature-state for_k8s_version="v1.0" state="stable" >}}
If the process in your container is able to crash on its own whenever it
encounters an issue or becomes unhealthy, you do not necessarily need a liveness
probe; the kubelet will automatically perform the correct action in accordance
with the Pod's `restartPolicy`.
If you'd like your container to be killed and restarted if a probe fails, then
specify a liveness probe, and specify a `restartPolicy` of Always or OnFailure.
### When should you use a readiness probe?
{{< feature-state for_k8s_version="v1.0" state="stable" >}}
If you'd like to start sending traffic to a Pod only when a probe succeeds,
specify a readiness probe. In this case, the readiness probe might be the same
as the liveness probe, but the existence of the readiness probe in the spec means
that the Pod will start without receiving any traffic and only start receiving
traffic after the probe starts succeeding.
If your container needs to work on loading large data, configuration files, or
migrations during startup, specify a readiness probe.
If you want your container to be able to take itself down for maintenance, you
can specify a readiness probe that checks an endpoint specific to readiness that
is different from the liveness probe.
{{< note >}}
If you just want to be able to drain requests when the Pod is deleted, you do not
necessarily need a readiness probe; on deletion, the Pod automatically puts itself
into an unready state regardless of whether the readiness probe exists.
The Pod remains in the unready state while it waits for the containers in the Pod
to stop.
{{< /note >}}
### When should you use a startup probe?
{{< feature-state for_k8s_version="v1.16" state="alpha" >}}
Startup probes are useful for Pods that have containers that take a long time to
come into service. Rather than set a long liveness interval, you can configure
a separate configuration for probing the container as it starts up, allowing
a time longer than the liveness interval would allow.
If your container usually starts in more than
`initialDelaySeconds + failureThreshold × periodSeconds`, you should specify a
startup probe that checks the same endpoint as the liveness probe. The default for
`periodSeconds` is 30s. You should then set its `failureThreshold` high enough to
allow the container to start, without changing the default values of the liveness
probe. This helps to protect against deadlocks.
## Termination of Pods {#pod-termination}
Because Pods represent processes running on nodes in the cluster, it is important to
allow those processes to gracefully terminate when they are no longer needed (rather
than being abruptly stopped with a `KILL` signal and having no chance to clean up).
The design aim is for you to be able to request deletion and know when processes
terminate, but also be able to ensure that deletes eventually complete.
When you request deletion of a Pod, the cluster records and tracks the intended grace period
before the Pod is allowed to be forcefully killed. With that forceful shutdown tracking in
place, the {{< glossary_tooltip text="kubelet" term_id="kubelet" >}} attempts graceful
shutdown.
Typically, the container runtime sends a a TERM signal is sent to the main process in each
container. Once the grace period has expired, the KILL signal is sent to any remainig
processes, and the Pod is then deleted from the
{{< glossary_tooltip text="API Server" term_id="kube-apiserver" >}}. If the kubelet or the
container runtime's management service is restarted while waiting for processes to terminate, the
cluster retries from the start including the full original grace period.
An example flow:
1. You use the `kubectl` tool to manually delete a specific Pod, with the default grace period
(30 seconds).
1. The Pod in the API server is updated with the time beyond which the Pod is considered "dead"
along with the grace period.
If you use `kubectl describe` to check on the Pod you're deleting, that Pod shows up as
"Terminating".
On the node where the Pod is running: as soon as the kubelet sees that a Pod has been marked
as terminating (a graceful shutdown duration has been set), the kubelet begins the local Pod
shutdown process.
1. If one of the Pod's containers has defined a `preStop`
[hook](/docs/concepts/containers/container-lifecycle-hooks/#hook-details), the kubelet
runs that hook inside of the container. If the `preStop` hook is still running after the
grace period expires, the kubelet requests a small, one-off grace period extension of 2
seconds.
{{< note >}}
If the `preStop` hook needs longer to complete than the default grace period allows,
you must modify `terminationGracePeriodSeconds` to suit this.
{{< /note >}}
1. The kubelet triggers the container runtime to send a TERM signal to process 1 inside each
container.
{{< note >}}
The containers in the Pod receive the TERM signal at different times and in an arbitrary
order. If the order of shutdowns matters, consider using a `preStop` hook to synchronize.
{{< /note >}}
1. At the same time as the kubelet is starting graceful shutdown, the control plane removes that
shutting-down Pod from Endpoints (and, if enabled, EndpointSlice) objects where these represent
a {{< glossary_tooltip term_id="service" text="Service" >}} with a configured
{{< glossary_tooltip text="selector" term_id="selector" >}}.
{{< glossary_tooltip text="ReplicaSets" term_id="replica-set" >}} and other workload resources
no longer treat the shutting-down Pod as a valid, in-service replica. Pods that shut down slowly
cannot continue to serve traffic as load balancers (like the service proxy) remove the Pod from
the list of endpoints as soon as the termination grace period _begins_.
1. When the grace period expires, the kubelet triggers forcible shutdown. The container runtime sends
`SIGKILL` to any processes still running in any container in the Pod.
The kubelet also cleans up a hidden `pause` container if that container runtime uses one.
1. The kubelet triggers forcible removal of Pod object from the API server, by setting grace period
to 0 (immediate deletion).
1. The API server deletes the Pod's API object, which is then no longer visible from any client.
### Forced Pod termination {#pod-termination-forced}
{{< caution >}}
Forced deletions can be potentially disruptiove for some workloads and their Pods.
{{< /caution >}}
By default, all deletes are graceful within 30 seconds. The `kubectl delete` command supports
the `--grace-period=<seconds>` option which allows you to override the default and specify your
own value.
Setting the grace period to `0` forcibly and immediately deletes the Pod from the API
server. If the pod was still running on a node, that forcible deletion triggers the kubelet to
begin immediate cleanup.
{{< note >}}
You must specify an additional flag `--force` along with `--grace-period=0` in order to perform force deletions.
{{< /note >}}
When a force deletion is performed, the API server does not wait for confirmation
from the kubelet that the Pod has been terminated on the node it was running on. It
removes the Pod in the API immediately so a new Pod can be created with the same
name. On the node, Pods that are set to terminate immediately will still be given
a small grace period before being force killed.
If you need to force-delete Pods that are part of a StatefulSet, refer to the task
documentation for
[deleting Pods from a StatefulSet](/docs/tasks/run-application/force-delete-stateful-set-pod/).
### Garbage collection of failed Pods {#pod-garbage-collection}
For failed Pods, the API objects remain in the cluster's API until a human or
{{< glossary_tooltip term_id="controller" text="controller" >}} process
explicitly removes them.
The control plane cleans up terminated Pods (with a phase of `Succeeded` or
`Failed`), when the number of Pods exceeds the configured threshold
(determined by `terminated-pod-gc-threshold` in the kube-controller-manager).
This avoids a resource leak as Pods are created and terminated over time.
There are different kinds of resources for creating Pods:
- Use a {{< glossary_tooltip term_id="deployment" >}},
{{< glossary_tooltip term_id="replica-set" >}} or {{< glossary_tooltip term_id="statefulset" >}}
for Pods that are not expected to terminate, for example, web servers.
- Use a {{< glossary_tooltip term_id="job" >}}
for Pods that are expected to terminate once their work is complete;
for example, batch computations. Jobs are appropriate only for Pods with
`restartPolicy` equal to OnFailure or Never.
- Use a {{< glossary_tooltip term_id="daemonset" >}}
for Pods that need to run one per eligible node.
All workload resources contain a PodSpec. It is recommended to create the
appropriate workload resource and let the resource's controller create Pods
for you, rather than directly create Pods yourself.
If a node dies or is disconnected from the rest of the cluster, Kubernetes
applies a policy for setting the `phase` of all Pods on the lost node to Failed.
## Examples
### Advanced liveness probe example
Liveness probes are executed by the kubelet, so all requests are made in the
kubelet network namespace.
```yaml
apiVersion: v1
kind: Pod
metadata:
labels:
test: liveness
name: liveness-http
spec:
containers:
- args:
- /server
image: k8s.gcr.io/liveness
livenessProbe:
httpGet:
# when "host" is not defined, "PodIP" will be used
# host: my-host
# when "scheme" is not defined, "HTTP" scheme will be used. Only "HTTP" and "HTTPS" are allowed
# scheme: HTTPS
path: /healthz
port: 8080
httpHeaders:
- name: X-Custom-Header
value: Awesome
initialDelaySeconds: 15
timeoutSeconds: 1
name: liveness
```
### Example states
* Pod is running and has one Container. Container exits with success.
* Log completion event.
* If `restartPolicy` is:
* Always: Restart Container; Pod `phase` stays Running.
* OnFailure: Pod `phase` becomes Succeeded.
* Never: Pod `phase` becomes Succeeded.
* Pod is running and has one Container. Container exits with failure.
* Log failure event.
* If `restartPolicy` is:
* Always: Restart Container; Pod `phase` stays Running.
* OnFailure: Restart Container; Pod `phase` stays Running.
* Never: Pod `phase` becomes Failed.
* Pod is running and has two Containers. Container 1 exits with failure.
* Log failure event.
* If `restartPolicy` is:
* Always: Restart Container; Pod `phase` stays Running.
* OnFailure: Restart Container; Pod `phase` stays Running.
* Never: Do not restart Container; Pod `phase` stays Running.
* If Container 1 is not running, and Container 2 exits:
* Log failure event.
* If `restartPolicy` is:
* Always: Restart Container; Pod `phase` stays Running.
* OnFailure: Restart Container; Pod `phase` stays Running.
* Never: Pod `phase` becomes Failed.
* Pod is running and has one Container. Container runs out of memory.
* Container terminates in failure.
* Log OOM event.
* If `restartPolicy` is:
* Always: Restart Container; Pod `phase` stays Running.
* OnFailure: Restart Container; Pod `phase` stays Running.
* Never: Log failure event; Pod `phase` becomes Failed.
* Pod is running, and a disk dies.
* Kill all Containers.
* Log appropriate event.
* Pod `phase` becomes Failed.
* If running under a controller, Pod is recreated elsewhere.
* Pod is running, and its node is segmented out.
* Node controller waits for timeout.
* Node controller sets Pod `phase` to Failed.
* If running under a controller, Pod is recreated elsewhere.
## {{% heading "whatsnext" %}}
* Get hands-on experience
[attaching handlers to Container lifecycle events](/docs/tasks/configure-pod-container/attach-handler-lifecycle-event/).
* Get hands-on experience
[Configure Liveness, Readiness and Startup Probes](/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/).
* Learn more about [Container lifecycle hooks](/docs/concepts/containers/container-lifecycle-hooks/).
[configuring Liveness, Readiness and Startup Probes](/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/).
* Learn more about [container lifecycle hooks](/docs/concepts/containers/container-lifecycle-hooks/).
* For detailed information about Pod / Container status in the API, see [PodStatus](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#podstatus-v1-core)
and
[ContainerStatus](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#containerstatus-v1-core).
@@ -1,123 +0,0 @@
---
reviewers:
- erictune
title: Pod Overview
content_type: concept
weight: 10
card:
name: concepts
weight: 60
---
<!-- overview -->
This page provides an overview of `Pod`, the smallest deployable object in the Kubernetes object model.
<!-- body -->
## Understanding Pods
A *Pod* is the basic execution unit of a Kubernetes application--the smallest and simplest unit in the Kubernetes object model that you create or deploy. A Pod represents processes running on your {{< glossary_tooltip term_id="cluster" text="cluster" >}}.
A Pod encapsulates an application's container (or, in some cases, multiple containers), storage resources, a unique network identity (IP address), as well as options that govern how the container(s) should run. A Pod represents a unit of deployment: *a single instance of an application in Kubernetes*, which might consist of either a single {{< glossary_tooltip text="container" term_id="container" >}} or a small number of containers that are tightly coupled and that share resources.
[Docker](https://www.docker.com) is the most common container runtime used in a Kubernetes Pod, but Pods support other [container runtimes](/docs/setup/production-environment/container-runtimes/) as well.
Pods in a Kubernetes cluster can be used in two main ways:
* **Pods that run a single container**. The "one-container-per-Pod" model is the most common Kubernetes use case; in this case, you can think of a Pod as a wrapper around a single container, and Kubernetes manages the Pods rather than the containers directly.
* **Pods that run multiple containers that need to work together**. A Pod might encapsulate an application composed of multiple co-located containers that are tightly coupled and need to share resources. These co-located containers might form a single cohesive unit of service--one container serving files from a shared volume to the public, while a separate "sidecar" container refreshes or updates those files. The Pod wraps these containers and storage resources together as a single manageable entity.
Each Pod is meant to run a single instance of a given application. If you want to scale your application horizontally (to provide more overall resources by running more instances), you should use multiple Pods, one for each instance. In Kubernetes, this is typically referred to as _replication_.
Replicated Pods are usually created and managed as a group by a workload resource and its {{< glossary_tooltip text="_controller_" term_id="controller" >}}.
See [Pods and controllers](#pods-and-controllers) for more information on how Kubernetes uses controllers to implement workload scaling and healing.
### How Pods manage multiple containers
Pods are designed to support multiple cooperating processes (as containers) that form a cohesive unit of service. The containers in a Pod are automatically co-located and co-scheduled on the same physical or virtual machine in the cluster. The containers can share resources and dependencies, communicate with one another, and coordinate when and how they are terminated.
Note that grouping multiple co-located and co-managed containers in a single Pod is a relatively advanced use case. You should use this pattern only in specific instances in which your containers are tightly coupled. For example, you might have a container that acts as a web server for files in a shared volume, and a separate "sidecar" container that updates those files from a remote source, as in the following diagram:
{{< figure src="/images/docs/pod.svg" alt="example pod diagram" width="50%" >}}
Some Pods have {{< glossary_tooltip text="init containers" term_id="init-container" >}} as well as {{< glossary_tooltip text="app containers" term_id="app-container" >}}. Init containers run and complete before the app containers are started.
Pods provide two kinds of shared resources for their constituent containers: *networking* and *storage*.
#### Networking
Each Pod is assigned a unique IP address for each address family. Every container in a Pod shares the network namespace, including the IP address and network ports. Containers *inside a Pod* can communicate with one another using `localhost`. When containers in a Pod communicate with entities *outside the Pod*, they must coordinate how they use the shared network resources (such as ports).
#### Storage
A Pod can specify a set of shared storage {{< glossary_tooltip text="volumes" term_id="volume" >}}. All containers in the Pod can access the shared volumes, allowing those containers to share data. Volumes also allow persistent data in a Pod to survive in case one of the containers within needs to be restarted. See [Volumes](/docs/concepts/storage/volumes/) for more information on how Kubernetes implements shared storage in a Pod.
## Working with Pods
You'll rarely create individual Pods directly in Kubernetes--even singleton Pods. This is because Pods are designed as relatively ephemeral, disposable entities. When a Pod gets created (directly by you, or indirectly by a {{< glossary_tooltip text="_controller_" term_id="controller" >}}), it is scheduled to run on a {{< glossary_tooltip term_id="node" >}} in your cluster. The Pod remains on that node until the process is terminated, the pod object is deleted, the Pod is *evicted* for lack of resources, or the node fails.
{{< note >}}
Restarting a container in a Pod should not be confused with restarting a Pod. A Pod is not a process, but an environment for running a container. A Pod persists until it is deleted.
{{< /note >}}
Pods do not, by themselves, self-heal. If a Pod is scheduled to a Node that fails, or if the scheduling operation itself fails, the Pod is deleted; likewise, a Pod won't survive an eviction due to a lack of resources or Node maintenance. Kubernetes uses a higher-level abstraction, called a controller, that handles the work of managing the relatively disposable Pod instances. Thus, while it is possible to use Pod directly, it's far more common in Kubernetes to manage your pods using a controller.
### Pods and controllers
You can use workload resources to create and manage multiple Pods for you. A controller for the resource handles replication and rollout and automatic healing in case of Pod failure. For example, if a Node fails, a controller notices that Pods on that Node have stopped working and creates a replacement Pod. The scheduler places the replacement Pod onto a healthy Node.
Here are some examples of workload resources that manage one or more Pods:
* {{< glossary_tooltip text="Deployment" term_id="deployment" >}}
* {{< glossary_tooltip text="StatefulSet" term_id="statefulset" >}}
* {{< glossary_tooltip text="DaemonSet" term_id="daemonset" >}}
## Pod templates
Controllers for {{< glossary_tooltip text="workload" term_id="workload" >}} resources create Pods
from a pod template and manage those Pods on your behalf.
PodTemplates are specifications for creating Pods, and are included in workload resources such as
[Deployments](/docs/concepts/workloads/controllers/deployment/),
[Jobs](/docs/concepts/jobs/run-to-completion-finite-workloads/), and
[DaemonSets](/docs/concepts/workloads/controllers/daemonset/).
Each controller for a workload resource uses the PodTemplate inside the workload object to make actual Pods. The PodTemplate is part of the desired state of whatever workload resource you used to run your app.
The sample below is a manifest for a simple Job with a `template` that starts one container. The container in that Pod prints a message then pauses.
```yaml
apiVersion: batch/v1
kind: Job
metadata:
name: hello
spec:
template:
# This is the pod template
spec:
containers:
- name: hello
image: busybox
command: ['sh', '-c', 'echo "Hello, Kubernetes!" && sleep 3600']
restartPolicy: OnFailure
# The pod template ends here
```
Modifying the pod template or switching to a new pod template has no effect on the Pods that already exist. Pods do not receive template updates directly; instead, a new Pod is created to match the revised pod template.
For example, a Deployment controller ensures that the running Pods match the current pod template. If the template is updated, the controller has to remove the existing Pods and create new Pods based on the updated template. Each workload controller implements its own rules for handling changes to the Pod template.
On Nodes, the {{< glossary_tooltip term_id="kubelet" text="kubelet" >}} does not directly observe or manage any of the details around pod templates and updates; those details are abstracted away. That abstraction and separation of concerns simplifies system semantics, and makes it feasible to extend the cluster's behavior without changing existing code.
## {{% heading "whatsnext" %}}
* Learn more about [Pods](/docs/concepts/workloads/pods/pod/)
* [The Distributed System Toolkit: Patterns for Composite Containers](https://kubernetes.io/blog/2015/06/the-distributed-system-toolkit-patterns) explains common layouts for Pods with more than one container
* Learn more about Pod behavior:
* [Pod Termination](/docs/concepts/workloads/pods/pod/#termination-of-pods)
* [Pod Lifecycle](/docs/concepts/workloads/pods/pod-lifecycle/)
@@ -1,7 +1,7 @@
---
title: Pod Topology Spread Constraints
content_type: concept
weight: 50
weight: 40
---
<!-- overview -->
@@ -1,209 +0,0 @@
---
reviewers:
title: Pods
content_type: concept
weight: 20
---
<!-- overview -->
_Pods_ are the smallest deployable units of computing that can be created and
managed in Kubernetes.
<!-- body -->
## What is a Pod?
A _Pod_ (as in a pod of whales or pea pod) is a group of one or more
{{< glossary_tooltip text="containers" term_id="container" >}} (such as
Docker containers), with shared storage/network, and a specification
for how to run the containers. A Pod's contents are always co-located and
co-scheduled, and run in a shared context. A Pod models an
application-specific "logical host" - it contains one or more application
containers which are relatively tightly coupled &mdash; in a pre-container
world, being executed on the same physical or virtual machine would mean being
executed on the same logical host.
While Kubernetes supports more container runtimes than just Docker, Docker is
the most commonly known runtime, and it helps to describe Pods in Docker terms.
The shared context of a Pod is a set of Linux namespaces, cgroups, and
potentially other facets of isolation - the same things that isolate a Docker
container. Within a Pod's context, the individual applications may have
further sub-isolations applied.
Containers within a Pod share an IP address and port space, and
can find each other via `localhost`. They can also communicate with each
other using standard inter-process communications like SystemV semaphores or
POSIX shared memory. Containers in different Pods have distinct IP addresses
and can not communicate by IPC without
[special configuration](/docs/concepts/policy/pod-security-policy/).
These containers usually communicate with each other via Pod IP addresses.
Applications within a Pod also have access to shared {{< glossary_tooltip text="volumes" term_id="volume" >}}, which are defined
as part of a Pod and are made available to be mounted into each application's
filesystem.
In terms of [Docker](https://www.docker.com/) constructs, a Pod is modelled as
a group of Docker containers with shared namespaces and shared filesystem
volumes.
Like individual application containers, Pods are considered to be relatively
ephemeral (rather than durable) entities. As discussed in
[pod lifecycle](/docs/concepts/workloads/pods/pod-lifecycle/), Pods are created, assigned a unique ID (UID), and
scheduled to nodes where they remain until termination (according to restart
policy) or deletion. If a {{< glossary_tooltip term_id="node" >}} dies, the Pods scheduled to that node are
scheduled for deletion, after a timeout period. A given Pod (as defined by a UID) is not
"rescheduled" to a new node; instead, it can be replaced by an identical Pod,
with even the same name if desired, but with a new UID (see [replication
controller](/docs/concepts/workloads/controllers/replicationcontroller/) for more details).
When something is said to have the same lifetime as a Pod, such as a volume,
that means that it exists as long as that Pod (with that UID) exists. If that
Pod is deleted for any reason, even if an identical replacement is created, the
related thing (e.g. volume) is also destroyed and created anew.
{{< figure src="/images/docs/pod.svg" title="Pod diagram" width="50%" >}}
*A multi-container Pod that contains a file puller and a
web server that uses a persistent volume for shared storage between the containers.*
## Motivation for Pods
### Management
Pods are a model of the pattern of multiple cooperating processes which form a
cohesive unit of service. They simplify application deployment and management
by providing a higher-level abstraction than the set of their constituent
applications. Pods serve as unit of deployment, horizontal scaling, and
replication. Colocation (co-scheduling), shared fate (e.g. termination),
coordinated replication, resource sharing, and dependency management are
handled automatically for containers in a Pod.
### Resource sharing and communication
Pods enable data sharing and communication among their constituents.
The applications in a Pod all use the same network namespace (same IP and port
space), and can thus "find" each other and communicate using `localhost`.
Because of this, applications in a Pod must coordinate their usage of ports.
Each Pod has an IP address in a flat shared networking space that has full
communication with other physical computers and Pods across the network.
Containers within the Pod see the system hostname as being the same as the configured
`name` for the Pod. There's more about this in the [networking](/docs/concepts/cluster-administration/networking/)
section.
In addition to defining the application containers that run in the Pod, the Pod
specifies a set of shared storage volumes. Volumes enable data to survive
container restarts and to be shared among the applications within the Pod.
## Uses of pods
Pods can be used to host vertically integrated application stacks (e.g. LAMP),
but their primary motivation is to support co-located, co-managed helper
programs, such as:
* content management systems, file and data loaders, local cache managers, etc.
* log and checkpoint backup, compression, rotation, snapshotting, etc.
* data change watchers, log tailers, logging and monitoring adapters, event publishers, etc.
* proxies, bridges, and adapters
* controllers, managers, configurators, and updaters
Individual Pods are not intended to run multiple instances of the same
application, in general.
For a longer explanation, see [The Distributed System ToolKit: Patterns for
Composite
Containers](https://kubernetes.io/blog/2015/06/the-distributed-system-toolkit-patterns).
## Alternatives considered
_Why not just run multiple programs in a single (Docker) container?_
1. Transparency. Making the containers within the Pod visible to the
infrastructure enables the infrastructure to provide services to those
containers, such as process management and resource monitoring. This
facilitates a number of conveniences for users.
1. Decoupling software dependencies. The individual containers may be
versioned, rebuilt and redeployed independently. Kubernetes may even support
live updates of individual containers someday.
1. Ease of use. Users don't need to run their own process managers, worry about
signal and exit-code propagation, etc.
1. Efficiency. Because the infrastructure takes on more responsibility,
containers can be lighter weight.
_Why not support affinity-based co-scheduling of containers?_
That approach would provide co-location, but would not provide most of the
benefits of Pods, such as resource sharing, IPC, guaranteed fate sharing, and
simplified management.
## Durability of pods (or lack thereof)
Pods aren't intended to be treated as durable entities. They won't survive scheduling failures, node failures, or other evictions, such as due to lack of resources, or in the case of node maintenance.
In general, users shouldn't need to create Pods directly. They should almost
always use controllers even for singletons, for example,
[Deployments](/docs/concepts/workloads/controllers/deployment/).
Controllers provide self-healing with a cluster scope, as well as replication
and rollout management.
Controllers like [StatefulSet](/docs/concepts/workloads/controllers/statefulset.md)
can also provide support to stateful Pods.
The use of collective APIs as the primary user-facing primitive is relatively common among cluster scheduling systems, including [Borg](https://research.google.com/pubs/pub43438.html), [Marathon](https://mesosphere.github.io/marathon/docs/rest-api.html), [Aurora](http://aurora.apache.org/documentation/latest/reference/configuration/#job-schema), and [Tupperware](https://www.slideshare.net/Docker/aravindnarayanan-facebook140613153626phpapp02-37588997).
Pod is exposed as a primitive in order to facilitate:
* scheduler and controller pluggability
* support for pod-level operations without the need to "proxy" them via controller APIs
* decoupling of Pod lifetime from controller lifetime, such as for bootstrapping
* decoupling of controllers and services &mdash; the endpoint controller just watches Pods
* clean composition of Kubelet-level functionality with cluster-level functionality &mdash; Kubelet is effectively the "pod controller"
* high-availability applications, which will expect Pods to be replaced in advance of their termination and certainly in advance of deletion, such as in the case of planned evictions or image prefetching.
## Termination of Pods
Because Pods represent running processes on nodes in the cluster, it is important to allow those processes to gracefully terminate when they are no longer needed (vs being violently killed with a KILL signal and having no chance to clean up). Users should be able to request deletion and know when processes terminate, but also be able to ensure that deletes eventually complete. When a user requests deletion of a Pod, the system records the intended grace period before the Pod is allowed to be forcefully killed, and a TERM signal is sent to the main process in each container. Once the grace period has expired, the KILL signal is sent to those processes, and the Pod is then deleted from the API server. If the Kubelet or the container manager is restarted while waiting for processes to terminate, the termination will be retried with the full grace period.
An example flow:
1. User sends command to delete Pod, with default grace period (30s)
1. The Pod in the API server is updated with the time beyond which the Pod is considered "dead" along with the grace period.
1. Pod shows up as "Terminating" when listed in client commands
1. (simultaneous with 3) When the Kubelet sees that a Pod has been marked as terminating because the time in 2 has been set, it begins the Pod shutdown process.
1. If one of the Pod's containers has defined a [preStop hook](/docs/concepts/containers/container-lifecycle-hooks/#hook-details), it is invoked inside of the container. If the `preStop` hook is still running after the grace period expires, step 2 is then invoked with a small (2 second) one-time extended grace period. You must modify `terminationGracePeriodSeconds` if the `preStop` hook needs longer to complete.
1. The container is sent the TERM signal. Note that not all containers in the Pod will receive the TERM signal at the same time and may each require a `preStop` hook if the order in which they shut down matters.
1. (simultaneous with 3) Pod is removed from endpoints list for service, and are no longer considered part of the set of running Pods for replication controllers. Pods that shutdown slowly cannot continue to serve traffic as load balancers (like the service proxy) remove them from their rotations.
1. When the grace period expires, any processes still running in the Pod are killed with SIGKILL.
1. The Kubelet will finish deleting the Pod on the API server by setting grace period 0 (immediate deletion). The Pod disappears from the API and is no longer visible from the client.
By default, all deletes are graceful within 30 seconds. The `kubectl delete` command supports the `--grace-period=<seconds>` option which allows a user to override the default and specify their own value. The value `0` [force deletes](/docs/concepts/workloads/pods/pod/#force-deletion-of-pods) the Pod.
You must specify an additional flag `--force` along with `--grace-period=0` in order to perform force deletions.
### Force deletion of pods
Force deletion of a Pod is defined as deletion of a Pod from the cluster state and etcd immediately. When a force deletion is performed, the API server does not wait for confirmation from the kubelet that the Pod has been terminated on the node it was running on. It removes the Pod in the API immediately so a new Pod can be created with the same name. On the node, Pods that are set to terminate immediately will still be given a small grace period before being force killed.
Force deletions can be potentially dangerous for some Pods and should be performed with caution. In case of StatefulSet Pods, please refer to the task documentation for [deleting Pods from a StatefulSet](/docs/tasks/run-application/force-delete-stateful-set-pod/).
## Privileged mode for pod containers
Any container in a Pod can enable privileged mode, using the `privileged` flag on the [security context](/docs/tasks/configure-pod-container/security-context/) of the container spec. This is useful for containers that want to use Linux capabilities like manipulating the network stack and accessing devices. Processes within the container get almost the same privileges that are available to processes outside a container. With privileged mode, it should be easier to write network and volume plugins as separate Pods that don't need to be compiled into the kubelet.
{{< note >}}
Your container runtime must support the concept of a privileged container for this setting to be relevant.
{{< /note >}}
## API Object
Pod is a top-level resource in the Kubernetes REST API.
The [Pod API object](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#pod-v1-core) definition
describes the object in detail.
When creating the manifest for a Pod object, make sure the name specified is a valid
[DNS subdomain name](/docs/concepts/overview/working-with-objects/names#dns-subdomain-names).
@@ -1,7 +1,7 @@
---
reviewers:
- jessfraz
title: Pod Preset
title: Pod Presets
content_type: concept
weight: 50
---
@@ -32,20 +32,20 @@ specific service do not need to know all the details about that service.
In order to use Pod presets in your cluster you must ensure the following:
1. You have enabled the API type `settings.k8s.io/v1alpha1/podpreset`. For
example, this can be done by including `settings.k8s.io/v1alpha1=true` in
the `--runtime-config` option for the API server. In minikube add this flag
`--extra-config=apiserver.runtime-config=settings.k8s.io/v1alpha1=true` while
starting the cluster.
1. You have enabled the admission controller `PodPreset`. One way to doing this
is to include `PodPreset` in the `--enable-admission-plugins` option value specified
for the API server. In minikube, add this flag
```shell
--extra-config=apiserver.enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota,PodPreset
```
while starting the cluster.
1. You have enabled the API type `settings.k8s.io/v1alpha1/podpreset`. For
example, this can be done by including `settings.k8s.io/v1alpha1=true` in
the `--runtime-config` option for the API server. In minikube add this flag
`--extra-config=apiserver.runtime-config=settings.k8s.io/v1alpha1=true` while
starting the cluster.
1. You have enabled the admission controller named `PodPreset`. One way to doing this
is to include `PodPreset` in the `--enable-admission-plugins` option value specified
for the API server. For example, if you use Minikube, add this flag:
```shell
--extra-config=apiserver.enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota,PodPreset
```
while starting your cluster.
## How it works
@@ -64,31 +64,28 @@ When a pod creation request occurs, the system does the following:
modified by a `PodPreset`. The annotation is of the form
`podpreset.admission.kubernetes.io/podpreset-<pod-preset name>: "<resource version>"`.
Each Pod can be matched by zero or more Pod Presets; and each `PodPreset` can be
applied to zero or more pods. When a `PodPreset` is applied to one or more
Pods, Kubernetes modifies the Pod Spec. For changes to `Env`, `EnvFrom`, and
`VolumeMounts`, Kubernetes modifies the container spec for all containers in
the Pod; for changes to `Volume`, Kubernetes modifies the Pod Spec.
Each Pod can be matched by zero or more PodPresets; and each PodPreset can be
applied to zero or more Pods. When a PodPreset is applied to one or more
Pods, Kubernetes modifies the Pod Spec. For changes to `env`, `envFrom`, and
`volumeMounts`, Kubernetes modifies the container spec for all containers in
the Pod; for changes to `volumes`, Kubernetes modifies the Pod Spec.
{{< note >}}
A Pod Preset is capable of modifying the following fields in a Pod spec when appropriate:
- The `.spec.containers` field.
- The `initContainers` field (requires Kubernetes version 1.14.0 or later).
- The `.spec.containers` field
- The `.spec.initContainers` field
{{< /note >}}
### Disable Pod Preset for a Specific Pod
### Disable Pod Preset for a specific pod
There may be instances where you wish for a Pod to not be altered by any Pod
Preset mutations. In these cases, you can add an annotation in the Pod Spec
preset mutations. In these cases, you can add an annotation in the Pod's `.spec`
of the form: `podpreset.admission.kubernetes.io/exclude: "true"`.
## {{% heading "whatsnext" %}}
See [Injecting data into a Pod using PodPreset](/docs/tasks/inject-data-application/podpreset/)
For more information about the background, see the [design proposal for PodPreset](https://git.k8s.io/community/contributors/design-proposals/service-catalog/pod-preset.md).
+35 -14
View File
@@ -28,41 +28,62 @@ Kubernetes documentation welcomes improvements from all contributors, new and ex
## Getting started
Anyone can open an issue about documentation, or contribute a change with a pull request (PR) to the [`kubernetes/website` GitHub repository](https://github.com/kubernetes/website). You need to be comfortable with [git](https://git-scm.com/) and [GitHub](https://lab.github.com/) to operate effectively in the Kubernetes community.
Anyone can open an issue about documentation, or contribute a change with a
pull request (PR) to the
[`kubernetes/website` GitHub repository](https://github.com/kubernetes/website).
You need to be comfortable with
[git](https://git-scm.com/) and
[GitHub](https://lab.github.com/)
to work effectively in the Kubernetes community.
To get involved with documentation:
1. Sign the CNCF [Contributor License Agreement](https://github.com/kubernetes/community/blob/master/CLA.md).
2. Familiarize yourself with the [documentation repository](https://github.com/kubernetes/website) and the website's [static site generator](https://gohugo.io).
3. Make sure you understand the basic processes for [opening a pull request](/docs/contribute/new-content/new-content/) and [reviewing changes](/docs/contribute/review/reviewing-prs/).
1. Familiarize yourself with the [documentation repository](https://github.com/kubernetes/website)
and the website's [static site generator](https://gohugo.io).
1. Make sure you understand the basic processes for
[opening a pull request](/docs/contribute/new-content/open-a-pr/) and
[reviewing changes](/docs/contribute/review/reviewing-prs/).
Some tasks require more trust and more access in the Kubernetes organization.
See [Participating in SIG Docs](/docs/contribute/participating/) for more details about
See [Participating in SIG Docs](/docs/contribute/participate/) for more details about
roles and permissions.
## Your first contribution
- Read the [Contribution overview](/docs/contribute/new-content/overview/) to learn about the different ways you can contribute.
- See [Contribute to kubernetes/website](https://github.com/kubernetes/website/contribute) to find issues that make good entry points.
- [Open a pull request using GitHub](/docs/contribute/new-content/new-content/#changes-using-github) to existing documentation and learn more about filing issues in GitHub.
- [Review pull requests](/docs/contribute/review/reviewing-prs/) from other Kubernetes community members for accuracy and language.
- Read the Kubernetes [content](/docs/contribute/style/content-guide/) and [style guides](/docs/contribute/style/style-guide/) so you can leave informed comments.
- Learn about [page content types](/docs/contribute/style/page-content-types/) and [Hugo shortcodes](/docs/contribute/style/hugo-shortcodes/).
- Read the [Contribution overview](/docs/contribute/new-content/overview/) to
learn about the different ways you can contribute.
- Check [kubernetes/website issues list](/https://github.com/kubernetes/website/issues/)
for issues that make good entry points.
- [Open a pull request using GitHub](/docs/contribute/new-content/open-a-pr/#changes-using-github)
to existing documentation and learn more about filing issues in GitHub.
- [Review pull requests](/docs/contribute/review/reviewing-prs/) from other
Kubernetes community members for accuracy and language.
- Read the Kubernetes [content](/docs/contribute/style/content-guide/) and
[style guides](/docs/contribute/style/style-guide/) so you can leave informed comments.
- Learn about [page content types](/docs/contribute/style/page-content-types/)
and [Hugo shortcodes](/docs/contribute/style/hugo-shortcodes/).
## Next steps
- Learn to [work from a local clone](/docs/contribute/new-content/new-content/#fork-the-repo) of the repository.
- Learn to [work from a local clone](/docs/contribute/new-content/open-a-pr/#fork-the-repo)
of the repository.
- Document [features in a release](/docs/contribute/new-content/new-features/).
- Participate in [SIG Docs](/docs/contribute/participating/), and become a [member or reviewer](/docs/contribute/participating/#roles-and-responsibilities).
- Participate in [SIG Docs](/docs/contribute/participate/), and become a
[member or reviewer](/docs/contribute/participate/roles-and-responsibilities/).
- Start or help with a [localization](/docs/contribute/localization/).
## Get involved with SIG Docs
[SIG Docs](/docs/contribute/participating/) is the group of contributors who publish and maintain Kubernetes documentation and the website. Getting involved with SIG Docs is a great way for Kubernetes contributors (feature development or otherwise) to have a large impact on the Kubernetes project.
[SIG Docs](/docs/contribute/participate/) is the group of contributors who
publish and maintain Kubernetes documentation and the website. Getting
involved with SIG Docs is a great way for Kubernetes contributors (feature
development or otherwise) to have a large impact on the Kubernetes project.
SIG Docs communicates with different methods:
- [Join `#sig-docs` on the Kubernetes Slack instance](http://slack.k8s.io/). Make sure to
- [Join `#sig-docs` on the Kubernetes Slack instance](https://slack.k8s.io/). Make sure to
introduce yourself!
- [Join the `kubernetes-sig-docs` mailing list](https://groups.google.com/forum/#!forum/kubernetes-sig-docs),
where broader discussions take place and official decisions are recorded.
+19 -14
View File
@@ -13,13 +13,12 @@ This page assumes that you understand how to
to learn about more ways to contribute. You need to use the Git command line
client and other tools for some of these tasks.
<!-- body -->
## Propose improvements
SIG Docs [members](/docs/contribute/participating/#members) can propose improvements.
SIG Docs [members](/docs/contribute/participate/roles-and-responsibilities/#members)
can propose improvements.
After you've been contributing to the Kubernetes documentation for a while, you
may have ideas for improving the [Style Guide](/docs/contribute/style/style-guide/)
@@ -42,8 +41,8 @@ documentation testing might involve working with sig-testing.
## Coordinate docs for a Kubernetes release
SIG Docs [approvers](/docs/contribute/participating/#approvers) can coordinate
docs for a Kubernetes release.
SIG Docs [approvers](/docs/contribute/participate/roles-and-responsibilities/#approvers)
can coordinate docs for a Kubernetes release.
Each Kubernetes release is coordinated by a team of people participating in the
sig-release Special Interest Group (SIG). Others on the release team for a given
@@ -73,8 +72,8 @@ rotated among SIG Docs approvers.
## Serve as a New Contributor Ambassador
SIG Docs [approvers](/docs/contribute/participating/#approvers) can serve as
New Contributor Ambassadors.
SIG Docs [approvers](/docs/contribute/participate/roles-and-responsibilities/#approvers)
can serve as New Contributor Ambassadors.
New Contributor Ambassadors welcome new contributors to SIG-Docs,
suggest PRs to new contributors, and mentor new contributors through their first
@@ -92,14 +91,14 @@ Current New Contributor Ambassadors are announced at each SIG-Docs meeting, and
## Sponsor a new contributor
SIG Docs [reviewers](/docs/contribute/participating/#reviewers) can sponsor
new contributors.
SIG Docs [reviewers](/docs/contribute/participate/roles-and-responsibilities/#reviewers)
can sponsor new contributors.
After a new contributor has successfully submitted 5 substantive pull requests
to one or more Kubernetes repositories, they are eligible to apply for
[membership](/docs/contribute/participating#members) in the Kubernetes
organization. The contributor's membership needs to be backed by two sponsors
who are already reviewers.
[membership](/docs/contribute/participate/roles-and-responsibilities/#members)
in the Kubernetes organization. The contributor's membership needs to be
backed by two sponsors who are already reviewers.
New docs contributors can request sponsors by asking in the #sig-docs channel
on the [Kubernetes Slack instance](https://kubernetes.slack.com) or on the
@@ -111,7 +110,8 @@ membership in the Kubernetes organization.
## Serve as a SIG Co-chair
SIG Docs [approvers](/docs/contribute/participating/#approvers) can serve a term as a co-chair of SIG Docs.
SIG Docs [approvers](/docs/contribute/participate/roles-and-responsibilities/#approvers)
can serve a term as a co-chair of SIG Docs.
### Prerequisites
@@ -120,7 +120,12 @@ Approvers must meet the following requirements to be a co-chair:
- Have been a SIG Docs approver for at least 6 months
- Have [led a Kubernetes docs release](/docs/contribute/advanced/#coordinate-docs-for-a-kubernetes-release) or shadowed two releases
- Understand SIG Docs workflows and tooling: git, Hugo, localization, blog subproject
- Understand how other Kubernetes SIGs and repositories affect the SIG Docs workflow, including: [teams in k/org](https://github.com/kubernetes/org/blob/master/config/kubernetes/sig-docs/teams.yaml), [process in k/community](https://github.com/kubernetes/community/tree/master/sig-docs), plugins in [k/test-infra](https://github.com/kubernetes/test-infra/), and the role of [SIG Architecture](https://github.com/kubernetes/community/tree/master/sig-architecture).
- Understand how other Kubernetes SIGs and repositories affect the SIG Docs
workflow, including:
[teams in k/org](https://github.com/kubernetes/org/blob/master/config/kubernetes/sig-docs/teams.yaml),
[process in k/community](https://github.com/kubernetes/community/tree/master/sig-docs),
plugins in [k/test-infra](https://github.com/kubernetes/test-infra/), and the role of
[SIG Architecture](https://github.com/kubernetes/community/tree/master/sig-architecture).
- Commit at least 5 hours per week (and often more) to the role for a minimum of 6 months
### Responsibilities
@@ -15,21 +15,16 @@ like
[kubectl apply](/docs/reference/generated/kubectl/kubectl-commands#apply) and
[kubectl taint](/docs/reference/generated/kubectl/kubectl-commands#taint).
This topic does not show how to generate the
[kubectl](/docs/reference/generated/kubectl/kubectl/)
[kubectl](/docs/reference/generated/kubectl/kubectl-commands/)
options reference page. For instructions on how to generate the kubectl options
reference page, see
[Generating Reference Pages for Kubernetes Components and Tools](/docs/home/contribute/generated-reference/kubernetes-components/).
[Generating Reference Pages for Kubernetes Components and Tools](/docs/contribute/generate-ref-docs/kubernetes-components/).
{{< /note >}}
## {{% heading "prerequisites" %}}
{{< include "prerequisites-ref-docs.md" >}}
<!-- steps -->
## Setting up the local repositories
@@ -194,16 +194,14 @@ The use of `make docker-serve` is deprecated. Please use `make container-serve`
In `<web-base>` run `git add` and `git commit` to commit the change.
Submit your changes as a
[pull request](/docs/contribute/start/) to the
[pull request](/docs/contribute/new-content/open-a-pr/) to the
[kubernetes/website](https://github.com/kubernetes/website) repository.
Monitor your pull request, and respond to reviewer comments as needed. Continue
to monitor your pull request until it has been merged.
## {{% heading "whatsnext" %}}
* [Generating Reference Documentation Quickstart](/docs/contribute/generate-ref-docs/quickstart/)
* [Generating Reference Docs for Kubernetes Components and Tools](/docs/contribute/generate-ref-docs/kubernetes-components/)
* [Generating Reference Documentation for kubectl Commands](/docs/contribute/generate-ref-docs/kubectl/)
@@ -18,4 +18,5 @@
- You need to know how to create a pull request to a GitHub repository.
This involves creating your own fork of the repository. For more
information, see [Work from a local clone](/docs/contribute/intermediate/#work_from_a_local_clone).
information, see [Work from a local clone](/docs/contribute/new-content/open-a-pr/#fork-the-repo).
@@ -10,15 +10,10 @@ This page shows how to use the `update-imported-docs` script to generate
the Kubernetes reference documentation. The script automates
the build setup and generates the reference documentation for a release.
## {{% heading "prerequisites" %}}
{{< include "prerequisites-ref-docs.md" >}}
<!-- steps -->
## Getting the docs repository
@@ -87,7 +82,7 @@ The `update-imported-docs` script performs the following steps:
the sections in the `kubectl` command reference.
When the generated files are in your local clone of the `<web-base>`
repository, you can submit them in a [pull request](/docs/contribute/start/)
repository, you can submit them in a [pull request](/docs/contribute/new-content/open-a-pr/)
to `<web-base>`.
## Configuration file format
+1 -1
View File
@@ -183,7 +183,7 @@ Description | URLs
-----|-----
Home | [All heading and subheading URLs](/docs/home/)
Setup | [All heading and subheading URLs](/docs/setup/)
Tutorials | [Kubernetes Basics](/docs/tutorials/kubernetes-basics/), [Hello Minikube](/docs/tutorials/stateless-application/hello-minikube/)
Tutorials | [Kubernetes Basics](/docs/tutorials/kubernetes-basics/), [Hello Minikube](/docs/tutorials/hello-minikube/)
Site strings | [All site strings in a new localized TOML file](https://github.com/kubernetes/website/tree/master/i18n)
Translated documents must reside in their own `content/**/` subdirectory, but otherwise follow the same URL path as the English source. For example, to prepare the [Kubernetes Basics](/docs/tutorials/kubernetes-basics/) tutorial for translation into German, create a subfolder under the `content/de/` folder and copy the English source:
@@ -1,6 +1,5 @@
---
title: Opening a pull request
slug: new-content
content_type: concept
weight: 10
card:
@@ -20,8 +20,12 @@ This section contains information you should know before contributing new conten
- Write Kubernetes documentation in Markdown and build the Kubernetes site using [Hugo](https://gohugo.io/).
- The source is in [GitHub](https://github.com/kubernetes/website). You can find Kubernetes documentation at `/content/en/docs/`. Some of the reference documentation is automatically generated from scripts in the `update-imported-docs/` directory.
- [Page content types](/docs/contribute/style/page-content-types/) describe the presentation of documentation content in Hugo.
- In addition to the standard Hugo shortcodes, we use a number of [custom Hugo shortcodes](/docs/contribute/style/hugo-shortcodes/) in our documentation to control the presentation of content.
- Documentation source is available in multiple languages in `/content/`. Each language has its own folder with a two-letter code determined by the [ISO 639-1 standard](https://www.loc.gov/standards/iso639-2/php/code_list.php). For example, English documentation source is stored in `/content/en/docs/`.
- In addition to the standard Hugo shortcodes, we use a number of
[custom Hugo shortcodes](/docs/contribute/style/hugo-shortcodes/) in our documentation to control the presentation of content.
- Documentation source is available in multiple languages in `/content/`. Each
language has its own folder with a two-letter code determined by the
[ISO 639-1 standard](https://www.loc.gov/standards/iso639-2/php/code_list.php). For
example, English documentation source is stored in `/content/en/docs/`.
- For more information about contributing to documentation in multiple languages or starting a new translation, see [localization](/docs/contribute/localization).
## Before you begin {#before-you-begin}
@@ -20,18 +20,18 @@ SIG Docs welcomes content and reviews from all contributors. Anyone can open a
pull request (PR), and anyone is welcome to file issues about content or comment
on pull requests in progress.
You can also become a [member](/docs/contribute/participating/roles-and-responsibilities/#members),
[reviewer](/docs/contribute/participating/roles-and-responsibilities/#reviewers), or [approver](/docs/contribute/participating/roles-and-responsibilities/#approvers). These roles require greater
access and entail certain responsibilities for approving and committing changes.
See [community-membership](https://github.com/kubernetes/community/blob/master/community-membership.md)
You can also become a [member](/docs/contribute/participate/roles-and-responsibilities/#members),
[reviewer](/docs/contribute/participate/roles-and-responsibilities/#reviewers), or
[approver](/docs/contribute/participate/roles-and-responsibilities/#approvers).
These roles require greater access and entail certain responsibilities for
approving and committing changes. See
[community-membership](https://github.com/kubernetes/community/blob/master/community-membership.md)
for more information on how membership works within the Kubernetes community.
The rest of this document outlines some unique ways these roles function within
SIG Docs, which is responsible for maintaining one of the most public-facing
aspects of Kubernetes -- the Kubernetes website and documentation.
<!-- body -->
## SIG Docs chairperson
@@ -58,8 +58,9 @@ There are two categories of SIG Docs [teams](https://github.com/orgs/kubernetes/
Each can be referenced with their `@name` in GitHub comments to communicate with
everyone in that group.
Sometimes Prow and GitHub teams overlap without matching exactly. For assignment of issues, pull requests, and to support PR approvals,
the automation uses information from `OWNERS` files.
Sometimes Prow and GitHub teams overlap without matching exactly. For
assignment of issues, pull requests, and to support PR approvals, the
automation uses information from `OWNERS` files.
### OWNERS files and front-matter
@@ -114,6 +115,6 @@ SIG Docs approvers. Here's how it works.
For more information about contributing to the Kubernetes documentation, see:
- [Contributing new content](/docs/contribute/overview/)
- [Contributing new content](/docs/contribute/new-content/overview/)
- [Reviewing content](/docs/contribute/review/reviewing-prs)
- [Documentation style guide](/docs/contribute/style/)
@@ -36,16 +36,16 @@ The following queries are helpful when wrangling.
After working through these queries, the remaining list of PRs to review is usually small.
These queries exclude localization PRs. All queries are against the main branch except the last one.
- [No CLA, not eligible to merge](https://github.com/kubernetes/website/pulls?q=is%3Aopen+is%3Apr+label%3A%22cncf-cla%3A+no%22+-label%3Ado-not-merge+label%3Alanguage%2Fen):
- [No CLA, not eligible to merge](https://github.com/kubernetes/website/pulls?q=is%3Aopen+is%3Apr+label%3A%22cncf-cla%3A+no%22+-label%3A%22do-not-merge%2Fwork-in-progress%22+-label%3A%22do-not-merge%2Fhold%22+label%3Alanguage%2Fen):
Remind the contributor to sign the CLA. If both the bot and a human have reminded them, close
the PR and remind them that they can open it after signing the CLA.
**Do not review PRs whose authors have not signed the CLA!**
- [Needs LGTM](https://github.com/kubernetes/website/pulls?utf8=%E2%9C%93&q=is%3Aopen+is%3Apr+-label%3Ado-not-merge+label%3Alanguage%2Fen+-label%3Algtm+):
- [Needs LGTM](https://github.com/kubernetes/website/pulls?q=is%3Aopen+is%3Apr+-label%3A%22cncf-cla%3A+no%22+-label%3Ado-not-merge%2Fwork-in-progress+-label%3Ado-not-merge%2Fhold+label%3Alanguage%2Fen+-label%3Algtm):
Lists PRs that need an LGTM from a member. If the PR needs technical review, loop in one of the reviewers suggested by the bot. If the content needs work, add suggestions and feedback in-line.
- [Has LGTM, needs docs approval](https://github.com/kubernetes/website/pulls?q=is%3Aopen+is%3Apr+-label%3Ado-not-merge+label%3Alanguage%2Fen+label%3Algtm):
- [Has LGTM, needs docs approval](https://github.com/kubernetes/website/pulls?q=is%3Aopen+is%3Apr+-label%3Ado-not-merge%2Fwork-in-progress+-label%3Ado-not-merge%2Fhold+label%3Alanguage%2Fen+label%3Algtm+):
Lists PRs that need an `/approve` comment to merge.
- [Quick Wins](https://github.com/kubernetes/website/pulls?utf8=%E2%9C%93&q=is%3Apr+is%3Aopen+base%3Amaster+-label%3A%22do-not-merge%2Fwork-in-progress%22+-label%3A%22do-not-merge%2Fhold%22+label%3A%22cncf-cla%3A+yes%22+label%3A%22size%2FXS%22+label%3A%22language%2Fen%22+): Lists PRs against the main branch with no clear blockers. (change "XS" in the size label as you work through the PRs [XS, S, M, L, XL, XXL]).
- [Not against the main branch](https://github.com/kubernetes/website/pulls?utf8=%E2%9C%93&q=is%3Aopen+is%3Apr+-label%3Ado-not-merge+label%3Alanguage%2Fen+-base%3Amaster): If the PR is against a `dev-` branch, it's for an upcoming release. Assign the [docs release manager](https://github.com/kubernetes/sig-release/tree/master/release-team#kubernetes-release-team-roles) using: `/assign @<manager's_github-username>`. If the PR is against an old branch, help the author figure out whether it's targeted against the best branch.
- [Quick Wins](https://github.com/kubernetes/website/pulls?utf8=%E2%9C%93&q=is%3Apr+is%3Aopen+base%3Amaster+-label%3A%22do-not-merge%2Fwork-in-progress%22+-label%3A%22do-not-merge%2Fhold%22+label%3A%22cncf-cla%3A+yes%22+label%3A%22size%2FXS%22+label%3A%22language%2Fen%22): Lists PRs against the main branch with no clear blockers. (change "XS" in the size label as you work through the PRs [XS, S, M, L, XL, XXL]).
- [Not against the main branch](https://github.com/kubernetes/website/pulls?q=is%3Aopen+is%3Apr+label%3Alanguage%2Fen+-base%3Amaster): If the PR is against a `dev-` branch, it's for an upcoming release. Assign the [docs release manager](https://github.com/kubernetes/sig-release/tree/master/release-team#kubernetes-release-team-roles) using: `/assign @<manager's_github-username>`. If the PR is against an old branch, help the author figure out whether it's targeted against the best branch.
### When to close Pull Requests
@@ -0,0 +1,237 @@
---
title: Roles and responsibilities
content_type: concept
weight: 10
---
<!-- overview -->
Anyone can contribute to Kubernetes. As your contributions to SIG Docs grow,
you can apply for different levels of membership in the community.
These roles allow you to take on more responsibility within the community.
Each role requires more time and commitment. The roles are:
- Anyone: regular contributors to the Kubernetes documentation
- Members: can assign and triage issues and provide non-binding review on pull requests
- Reviewers: can lead reviews on documentation pull requests and can vouch for a change's quality
- Approvers: can lead reviews on documentation and merge changes
<!-- body -->
## Anyone
Anyone with a GitHub account can contribute to Kubernetes. SIG Docs welcomes all new contributors!
Anyone can:
- Open an issue in any [Kubernetes](https://github.com/kubernetes/)
repository, including
[`kubernetes/website`](https://github.com/kubernetes/website)
- Give non-binding feedback on a pull request
- Contribute to a localization
- Suggest improvements on [Slack](https://slack.k8s.io/) or the
[SIG docs mailing list](https://groups.google.com/forum/#!forum/kubernetes-sig-docs).
After [signing the CLA](/docs/contribute/new-content/overview/#sign-the-cla), anyone can also:
- Open a pull request to improve existing content, add new content, or write a blog post or case study
- Create diagrams, graphics assets, and embeddable screencasts and videos
For more information, see [contributing new content](/docs/contribute/new-content/).
## Members
A member is someone who has submitted multiple pull requests to
`kubernetes/website`. Members are a part of the
[Kubernetes GitHub organization](https://github.com/kubernetes).
Members can:
- Do everything listed under [Anyone](#anyone)
- Use the `/lgtm` comment to add the LGTM (looks good to me) label to a pull request
{{< note >}}
Using `/lgtm` triggers automation. If you want to provide non-binding
approval, simply commenting "LGTM" works too!
{{< /note >}}
- Use the `/hold` comment to block merging for a pull request
- Use the `/assign` comment to assign a reviewer to a pull request
- Provide non-binding review on pull requests
- Use automation to triage and categorize issues
- Document new features
### Becoming a member
After submitting at least 5 substantial pull requests and meeting the other
[requirements](https://github.com/kubernetes/community/blob/master/community-membership.md#member):
1. Find two [reviewers](#reviewers) or [approvers](#approvers) to
[sponsor](/docs/contribute/advanced#sponsor-a-new-contributor) your
membership.
Ask for sponsorship in the [#sig-docs channel on Slack](https://kubernetes.slack.com) or on the
[SIG Docs mailing list](https://groups.google.com/forum/#!forum/kubernetes-sig-docs).
{{< note >}}
Don't send a direct email or Slack direct message to an individual
SIG Docs member. You must request sponsorship before submitting your application.
{{< /note >}}
1. Open a GitHub issue in the
[`kubernetes/org`](https://github.com/kubernetes/org/) repository. Use the
**Organization Membership Request** issue template.
1. Let your sponsors know about the GitHub issue. You can either:
- Mention their GitHub username in an issue (`@<GitHub-username>`)
- Send them the issue link using Slack or email.
Sponsors will approve your request with a `+1` vote. Once your sponsors
approve the request, a Kubernetes GitHub admin adds you as a member.
Congratulations!
If your membership request is not accepted you will receive feedback.
After addressing the feedback, apply again.
1. Accept the invitation to the Kubernetes GitHub organization in your email account.
{{< note >}}
GitHub sends the invitation to the default email address in your account.
{{< /note >}}
## Reviewers
Reviewers are responsible for reviewing open pull requests. Unlike member
feedback, you must address reviewer feedback. Reviewers are members of the
[@kubernetes/sig-docs-{language}-reviews](https://github.com/orgs/kubernetes/teams?query=sig-docs)
GitHub team.
Reviewers can:
- Do everything listed under [Anyone](#anyone) and [Members](#members)
- Review pull requests and provide binding feedback
{{< note >}}
To provide non-binding feedback, prefix your comments with a phrase like "Optionally: ".
{{< /note >}}
- Edit user-facing strings in code
- Improve code comments
You can be a SIG Docs reviewer, or a reviewer for docs in a specific subject area.
### Assigning reviewers to pull requests
Automation assigns reviewers to all pull requests. You can request a
review from a specific person by commenting: `/assign
[@_github_handle]`.
If the assigned reviewer has not commented on the PR, another reviewer can
step in. You can also assign technical reviewers as needed.
### Using `/lgtm`
LGTM stands for "Looks good to me" and indicates that a pull request is
technically accurate and ready to merge. All PRs need a `/lgtm` comment from a
reviewer and a `/approve` comment from an approver to merge.
A `/lgtm` comment from reviewer is binding and triggers automation that adds the `lgtm` label.
### Becoming a reviewer
When you meet the
[requirements](https://github.com/kubernetes/community/blob/master/community-membership.md#reviewer),
you can become a SIG Docs reviewer. Reviewers in other SIGs must apply
separately for reviewer status in SIG Docs.
To apply:
1. Open a pull request that adds your GitHub user name to a section of the
[OWNERS_ALIASES](https://github.com/kubernetes/website/blob/master/OWNERS) file
in the `kubernetes/website` repository.
{{< note >}}
If you aren't sure where to add yourself, add yourself to `sig-docs-en-reviews`.
{{< /note >}}
1. Assign the PR to one or more SIG-Docs approvers (user names listed under
`sig-docs-{language}-owners`).
If approved, a SIG Docs lead adds you to the appropriate GitHub team. Once added,
[K8s-ci-robot](https://github.com/kubernetes/test-infra/tree/master/prow#bots-home)
assigns and suggests you as a reviewer on new pull requests.
## Approvers
Approvers review and approve pull requests for merging. Approvers are members of the
[@kubernetes/sig-docs-{language}-owners](https://github.com/orgs/kubernetes/teams/?query=sig-docs)
GitHub teams.
Approvers can do the following:
- Everything listed under [Anyone](#anyone), [Members](#members) and [Reviewers](#reviewers)
- Publish contributor content by approving and merging pull requests using the `/approve` comment
- Propose improvements to the style guide
- Propose improvements to docs tests
- Propose improvements to the Kubernetes website or other tooling
If the PR already has a `/lgtm`, or if the approver also comments with
`/lgtm`, the PR merges automatically. A SIG Docs approver should only leave a
`/lgtm` on a change that doesn't need additional technical review.
### Approving pull requests
Approvers and SIG Docs leads are the only ones who can merge pull requests
into the website repository. This comes with certain responsibilities.
- Approvers can use the `/approve` command, which merges PRs into the repo.
{{< warning >}}
A careless merge can break the site, so be sure that when you merge something, you mean it.
{{< /warning >}}
- Make sure that proposed changes meet the
[contribution guidelines](/docs/contribute/style/content-guide/#contributing-content).
If you ever have a question, or you're not sure about something, feel free
to call for additional review.
- Verify that Netlify tests pass before you `/approve` a PR.
<img src="/images/docs/contribute/netlify-pass.png" width="75%" alt="Netlify tests must pass before approving" />
- Visit the Netlify page preview for a PR to make sure things look good before approving.
- Participate in the
[PR Wrangler rotation schedule](https://github.com/kubernetes/website/wiki/PR-Wranglers)
for weekly rotations. SIG Docs expects all approvers to participate in this
rotation. See [PR wranglers](/docs/contribute/participate/pr-wranglers/).
for more details.
### Becoming an approver
When you meet the
[requirements](https://github.com/kubernetes/community/blob/master/community-membership.md#approver),
you can become a SIG Docs approver. Approvers in other SIGs must apply
separately for approver status in SIG Docs.
To apply:
1. Open a pull request adding yourself to a section of the
[OWNERS_ALIASES](https://github.com/kubernetes/website/blob/master/OWNERS)
file in the `kubernetes/website` repository.
{{< note >}}
If you aren't sure where to add yourself, add yourself to `sig-docs-en-owners`.
{{< /note >}}
2. Assign the PR to one or more current SIG Docs approvers.
If approved, a SIG Docs lead adds you to the appropriate GitHub team. Once added,
[@k8s-ci-robot](https://github.com/kubernetes/test-infra/tree/master/prow#bots-home)
assigns and suggests you as a reviewer on new pull requests.
## {{% heading "whatsnext" %}}
- Read about [PR wrangling](/docs/contribute/participate/pr-wranglers/), a role all approvers take on rotation.
@@ -1,195 +0,0 @@
---
title: Roles and responsibilities
content_type: concept
weight: 10
---
<!-- overview -->
Anyone can contribute to Kubernetes. As your contributions to SIG Docs grow, you can apply for different levels of membership in the community.
These roles allow you to take on more responsibility within the community.
Each role requires more time and commitment. The roles are:
- Anyone: regular contributors to the Kubernetes documentation
- Members: can assign and triage issues and provide non-binding review on pull requests
- Reviewers: can lead reviews on documentation pull requests and can vouch for a change's quality
- Approvers: can lead reviews on documentation and merge changes
<!-- body -->
## Anyone
Anyone with a GitHub account can contribute to Kubernetes. SIG Docs welcomes all new contributors!
Anyone can:
- Open an issue in any [Kubernetes](https://github.com/kubernetes/) repository, including [`kubernetes/website`](https://github.com/kubernetes/website)
- Give non-binding feedback on a pull request
- Contribute to a localization
- Suggest improvements on [Slack](http://slack.k8s.io/) or the [SIG docs mailing list](https://groups.google.com/forum/#!forum/kubernetes-sig-docs).
After [signing the CLA](/docs/contribute/new-content/overview/#sign-the-cla), anyone can also:
- Open a pull request to improve existing content, add new content, or write a blog post or case study
- Create diagrams, graphics assets, and embeddable screencasts and videos
For more information, see [contributing new content](/docs/contribute/new-content/).
## Members
A member is someone who has submitted multiple pull requests to `kubernetes/website`. Members are a part of the [Kubernetes GitHub organization](https://github.com/kubernetes).
Members can:
- Do everything listed under [Anyone](#anyone)
- Use the `/lgtm` comment to add the LGTM (looks good to me) label to a pull request
{{< note >}}
Using `/lgtm` triggers automation. If you want to provide non-binding approval, simply commenting "LGTM" works too!
{{< /note >}}
- Use the `/hold` comment to block merging for a pull request
- Use the `/assign` comment to assign a reviewer to a pull request
- Provide non-binding review on pull requests
- Use automation to triage and categorize issues
- Document new features
### Becoming a member
After submitting at least 5 substantial pull requests and meeting the other [requirements](https://github.com/kubernetes/community/blob/master/community-membership.md#member):
1. Find two [reviewers](#reviewers) or [approvers](#approvers) to [sponsor](/docs/contribute/advanced#sponsor-a-new-contributor) your membership.
Ask for sponsorship in the [#sig-docs channel on Slack](https://kubernetes.slack.com) or on the
[SIG Docs mailing list](https://groups.google.com/forum/#!forum/kubernetes-sig-docs).
{{< note >}}
Don't send a direct email or Slack direct message to an individual
SIG Docs member. You must request sponsorship before submitting your application.
{{< /note >}}
2. Open a GitHub issue in the [`kubernetes/org`](https://github.com/kubernetes/org/) repository. Use the **Organization Membership Request** issue template.
3. Let your sponsors know about the GitHub issue. You can either:
- Mention their GitHub username in an issue (`@<GitHub-username>`)
- Send them the issue link using Slack or email.
Sponsors will approve your request with a `+1` vote. Once your sponsors approve the request, a Kubernetes GitHub admin adds you as a member. Congratulations!
If your membership request is not accepted you will receive feedback. After addressing the feedback, apply again.
4. Accept the invitation to the Kubernetes GitHub organization in your email account.
{{< note >}}
GitHub sends the invitation to the default email address in your account.
{{< /note >}}
## Reviewers
Reviewers are responsible for reviewing open pull requests. Unlike member feedback, you must address reviewer feedback. Reviewers are members of the [@kubernetes/sig-docs-{language}-reviews](https://github.com/orgs/kubernetes/teams?query=sig-docs) GitHub team.
Reviewers can:
- Do everything listed under [Anyone](#anyone) and [Members](#members)
- Review pull requests and provide binding feedback
{{< note >}}
To provide non-binding feedback, prefix your comments with a phrase like "Optionally: ".
{{< /note >}}
- Edit user-facing strings in code
- Improve code comments
You can be a SIG Docs reviewer, or a reviewer for docs in a specific subject area.
### Assigning reviewers to pull requests
Automation assigns reviewers to all pull requests. You can request a
review from a specific person by commenting: `/assign
[@_github_handle]`.
If the assigned reviewer has not commented on the PR, another reviewer can step in. You can also assign technical reviewers as needed.
### Using `/lgtm`
LGTM stands for "Looks good to me" and indicates that a pull request is technically accurate and ready to merge. All PRs need a `/lgtm` comment from a reviewer and a `/approve` comment from an approver to merge.
A `/lgtm` comment from reviewer is binding and triggers automation that adds the `lgtm` label.
### Becoming a reviewer
When you meet the
[requirements](https://github.com/kubernetes/community/blob/master/community-membership.md#reviewer), you can become a SIG Docs reviewer. Reviewers in other SIGs must apply separately for reviewer status in SIG Docs.
To apply:
1. Open a pull request that adds your GitHub user name to a section of the
[OWNERS_ALIASES](https://github.com/kubernetes/website/blob/master/OWNERS) file
in the `kubernetes/website` repository.
{{< note >}}
If you aren't sure where to add yourself, add yourself to `sig-docs-en-reviews`.
{{< /note >}}
2. Assign the PR to one or more SIG-Docs approvers (user names listed under `sig-docs-{language}-owners`).
If approved, a SIG Docs lead adds you to the appropriate GitHub team. Once added, [K8s-ci-robot](https://github.com/kubernetes/test-infra/tree/master/prow#bots-home) assigns and suggests you as a reviewer on new pull requests.
## Approvers
Approvers review and approve pull requests for merging. Approvers are members of the
[@kubernetes/sig-docs-{language}-owners](https://github.com/orgs/kubernetes/teams/?query=sig-docs) GitHub teams.
Approvers can do the following:
- Everything listed under [Anyone](#anyone), [Members](#members) and [Reviewers](#reviewers)
- Publish contributor content by approving and merging pull requests using the `/approve` comment
- Propose improvements to the style guide
- Propose improvements to docs tests
- Propose improvements to the Kubernetes website or other tooling
If the PR already has a `/lgtm`, or if the approver also comments with `/lgtm`, the PR merges automatically. A SIG Docs approver should only leave a `/lgtm` on a change that doesn't need additional technical review.
### Approving pull requests
Approvers and SIG Docs leads are the only ones who can merge pull requests into the website repository. This comes with certain responsibilities.
- Approvers can use the `/approve` command, which merges PRs into the repo.
{{< warning >}}
A careless merge can break the site, so be sure that when you merge something, you mean it.
{{< /warning >}}
- Make sure that proposed changes meet the [contribution guidelines](/docs/contribute/style/content-guide/#contributing-content).
If you ever have a question, or you're not sure about something, feel free to call for additional review.
- Verify that Netlify tests pass before you `/approve` a PR.
<img src="/images/docs/contribute/netlify-pass.png" width="75%" alt="Netlify tests must pass before approving" />
- Visit the Netlify page preview for a PR to make sure things look good before approving.
- Participate in the [PR Wrangler rotation schedule](https://github.com/kubernetes/website/wiki/PR-Wranglers) for weekly rotations. SIG Docs expects all approvers to participate in this
rotation. See [PR wranglers](/docs/contribute/participating/pr-wranglers/).
for more details.
### Becoming an approver
When you meet the [requirements](https://github.com/kubernetes/community/blob/master/community-membership.md#approver), you can become a SIG Docs approver. Approvers in other SIGs must apply separately for approver status in SIG Docs.
To apply:
1. Open a pull request adding yourself to a section of the [OWNERS_ALIASES](https://github.com/kubernetes/website/blob/master/OWNERS) file in the `kubernetes/website` repository.
{{< note >}}
If you aren't sure where to add yourself, add yourself to `sig-docs-en-owners`.
{{< /note >}}
2. Assign the PR to one or more current SIG Docs approvers.
If approved, a SIG Docs lead adds you to the appropriate GitHub team. Once added, [@k8s-ci-robot](https://github.com/kubernetes/test-infra/tree/master/prow#bots-home) assigns and suggests you as a reviewer on new pull requests.
## {{% heading "whatsnext" %}}
- Read about [PR wrangling](/docs/contribute/participating/pr-wranglers), a role all approvers take on rotation.
@@ -8,7 +8,9 @@ weight: 20
<!-- overview -->
SIG Docs [Reviewers](/docs/contribute/participating/#reviewers) and [Approvers](/docs/contribute/participating/#approvers) do a few extra things when reviewing a change.
SIG Docs [Reviewers](/docs/contribute/participate/#reviewers) and
[Approvers](/docs/contribute/participate/#approvers) do a few extra things
when reviewing a change.
Every week a specific docs approver volunteers to triage
and review pull requests. This
@@ -19,9 +21,6 @@ requests (PRs) that are not already under active review.
In addition to the rotation, a bot assigns reviewers and approvers
for the PR based on the owners for the affected files.
<!-- body -->
## Reviewing a PR
@@ -202,9 +201,9 @@ Sample response to a request for support:
This issue sounds more like a request for support and less
like an issue specifically for docs. I encourage you to bring
your question to the `#kubernetes-users` channel in
[Kubernetes slack](http://slack.k8s.io/). You can also search
[Kubernetes slack](https://slack.k8s.io/). You can also search
resources like
[Stack Overflow](http://stackoverflow.com/questions/tagged/kubernetes)
[Stack Overflow](https://stackoverflow.com/questions/tagged/kubernetes)
for answers to similar questions.
You can also open issues for Kubernetes functionality in
@@ -16,10 +16,10 @@ It helps you learn the code base and build trust with other contributors.
Before reviewing, it's a good idea to:
- Read the [content guide](/docs/contribute/style/content-guide/) and
[style guide](/docs/contribute/style/style-guide/) so you can leave informed comments.
- Understand the different [roles and responsibilities](/docs/contribute/participating/#roles-and-responsibilities) in the Kubernetes documentation community.
[style guide](/docs/contribute/style/style-guide/) so you can leave informed comments.
- Understand the different
[roles and responsibilities](/docs/contribute/participate/roles-and-responsibilities/)
in the Kubernetes documentation community.
<!-- body -->
@@ -10,9 +10,9 @@ weight: 10
This page contains guidelines for Kubernetes documentation.
If you have questions about what's allowed, join the #sig-docs channel in
[Kubernetes Slack](http://slack.k8s.io/) and ask!
[Kubernetes Slack](https://slack.k8s.io/) and ask!
You can register for Kubernetes Slack at http://slack.k8s.io/.
You can register for Kubernetes Slack at https://slack.k8s.io/.
For information on creating new content for the Kubernetes
docs, follow the [style guide](/docs/contribute/style/style-guide).
@@ -67,7 +67,7 @@ ask for help in [#sig-docs on Kubernetes Slack](https://kubernetes.slack.com/mes
### More information
If you have questions about allowed content, join the [Kubernetes Slack](http://slack.k8s.io/) #sig-docs channel and ask!
If you have questions about allowed content, join the [Kubernetes Slack](https://slack.k8s.io/) #sig-docs channel and ask!
@@ -232,7 +232,7 @@ Renders to:
{{< tabs name="tab_with_file_include" >}}
{{< tab name="Content File #1" include="example1" />}}
{{< tab name="Content File #2" include="example2" />}}
{{< tab name="JSON File" include="podtemplate" />}}
{{< tab name="JSON File" include="podtemplate.json" />}}
{{< /tabs >}}
@@ -242,6 +242,6 @@ Renders to:
* Learn about [Hugo](https://gohugo.io/).
* Learn about [writing a new topic](/docs/contribute/style/write-new-topic/).
* Learn about [page content types](/docs/contribute/style/page-content-types/).
* Learn about [creating a pull request](/docs/contribute/new-content/new-content/).
* Learn about [opening a pull request](/docs/contribute/new-content/open-a-pr/).
* Learn about [advanced contributing](/docs/contribute/advanced/).
@@ -191,7 +191,7 @@ Within each section, write your content. Use the following guidelines:
interested in reading next.
An example of a published tutorial topic is
[Running a Stateless Application Using a Deployment](/docs/tutorials/stateless-application/run-stateless-application-deployment/).
[Running a Stateless Application Using a Deployment](/docs/tasks/run-application/run-stateless-application-deployment/).
### Reference
@@ -22,8 +22,11 @@ discussion.
<!-- body -->
{{< note >}}
Kubernetes documentation uses [Blackfriday Markdown Renderer](https://github.com/russross/blackfriday) along with a few [Hugo Shortcodes](/docs/home/contribute/includes/) to support glossary entries, tabs,
and representing feature state.
Kubernetes documentation uses
[Goldmark Markdown Renderer](https://github.com/yuin/goldmark)
with some adjustments along with a few
[Hugo Shortcodes](/docs/contribute/style/hugo-shortcodes/) to support
glossary entries, tabs, and representing feature state.
{{< /note >}}
## Language
@@ -584,12 +587,8 @@ The Federation feature provides ... | The new Federation feature provides ...
{{< /table >}}
## {{% heading "whatsnext" %}}
* Learn about [writing a new topic](/docs/contribute/style/write-new-topic/).
* Learn about [using page templates](/docs/contribute/style/page-content-types/).
* Learn about [staging your changes](/docs/contribute/stage-documentation-changes/)
* Learn about [creating a pull request](/docs/contribute/new-content/open-a-pr/).
@@ -11,7 +11,7 @@ This page shows how to create a new topic for the Kubernetes docs.
## {{% heading "prerequisites" %}}
Create a fork of the Kubernetes documentation repository as described in
[Open a PR](/docs/new-content/open-a-pr/).
[Open a PR](/docs/contribute/new-content/open-a-pr/).
<!-- steps -->
@@ -160,7 +160,7 @@ submitted to ensure all examples pass the tests.
{{< /note >}}
For an example of a topic that uses this technique, see
[Running a Single-Instance Stateful Application](/docs/tutorials/stateful-application/run-stateful-application/).
[Running a Single-Instance Stateful Application](/docs/tasks/run-application/run-single-instance-stateful-application/).
## Adding images to a topic
@@ -677,9 +677,6 @@ for more information.
This admission controller acts on creation and modification of the pod and determines if it should be admitted
based on the requested security context and the available Pod Security Policies.
For Kubernetes < 1.6.0, the API Server must enable the extensions/v1beta1/podsecuritypolicy API
extensions group (`--runtime-config=extensions/v1beta1/podsecuritypolicy=true`).
See also [Pod Security Policy documentation](/docs/concepts/policy/pod-security-policy/)
for more information.
@@ -26,7 +26,7 @@ even a file with a list of usernames and passwords. In this regard, _Kubernetes
does not have objects which represent normal user accounts._ Normal users
cannot be added to a cluster through an API call.
Even though normal user cannot be added via an API call, but any user that presents a valid certificate signed by the clusters certificate authority (CA) is considered authenticated. In this configuration, Kubernetes determines the username from the common name field in the subject of the cert (e.g., “/CN=bob”). From there, the role based access control (RBAC) sub-system would determine whether the user is authorized to perform a specific operation a resource. You can refer to [creating user certificate request](/docs/reference/access-authn-authz/certificate-signing-requests/#user-csr) for more details about this.
Even though normal user cannot be added via an API call, but any user that presents a valid certificate signed by the clusters certificate authority (CA) is considered authenticated. In this configuration, Kubernetes determines the username from the common name field in the subject of the cert (e.g., “/CN=bob”). From there, the role based access control (RBAC) sub-system would determine whether the user is authorized to perform a specific operation on a resource. You can refer to [creating user certificate request](/docs/reference/access-authn-authz/certificate-signing-requests/#user-csr) for more details about this.
In contrast, service accounts are users managed by the Kubernetes API. They are
bound to specific namespaces, and created automatically by the API server or
@@ -997,7 +997,7 @@ See https://kubernetes.io/docs/concepts/overview/working-with-objects/labels for
### Matching requests: matchPolicy
API servers can make objects available via multiple API groups or versions.
For example, the Kubernetes API server allows creating and modifying `Deployment` objects
For example, the Kubernetes API server may allow creating and modifying `Deployment` objects
via `extensions/v1beta1`, `apps/v1beta1`, `apps/v1beta2`, and `apps/v1` APIs.
For example, if a webhook only specified a rule for some API groups/versions (like `apiGroups:["apps"], apiVersions:["v1","v1beta1"]`),
@@ -606,12 +606,15 @@ either do not manually edit the role, or disable auto-reconciliation.
<table>
<caption>Kubernetes RBAC API discovery roles</caption>
<colgroup><col width="25%" /><col width="25%" /><col /></colgroup>
<colgroup><col style="width: 25%;" /><col style="width: 25%;" /><col /></colgroup>
<thead>
<tr>
<th>Default ClusterRole</th>
<th>Default ClusterRoleBinding</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><b>system:basic-user</b></td>
<td><b>system:authenticated</b> group</td>
@@ -627,6 +630,7 @@ either do not manually edit the role, or disable auto-reconciliation.
<td><b>system:authenticated</b> and <b>system:unauthenticated</b> groups</td>
<td>Allows read-only access to non-sensitive information about the cluster. Introduced in Kubernetes v1.14.</td>
</tr>
</tbody>
</table>
### User-facing roles
@@ -649,12 +653,15 @@ metadata:
```
<table>
<colgroup><col width="25%"><col width="25%"><col></colgroup>
<colgroup><col style="width: 25%;" /><col style="width: 25%;" /><col /></colgroup>
<thead>
<tr>
<th>Default ClusterRole</th>
<th>Default ClusterRoleBinding</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><b>cluster-admin</b></td>
<td><b>system:masters</b> group</td>
@@ -691,17 +698,21 @@ the contents of Secrets enables access to ServiceAccount credentials
in the namespace, which would allow API access as any ServiceAccount
in the namespace (a form of privilege escalation).</td>
</tr>
</tbody>
</table>
### Core component roles
<table>
<colgroup><col width="25%"><col width="25%"><col></colgroup>
<colgroup><col style="width: 25%;" /><col style="width: 25%;" /><col /></colgroup>
<thead>
<tr>
<th>Default ClusterRole</th>
<th>Default ClusterRoleBinding</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><b>system:kube-scheduler</b></td>
<td><b>system:kube-scheduler</b> user</td>
@@ -733,17 +744,21 @@ The <tt>system:node</tt> role only exists for compatibility with Kubernetes clus
<td><b>system:kube-proxy</b> user</td>
<td>Allows access to the resources required by the {{< glossary_tooltip term_id="kube-proxy" text="kube-proxy" >}} component.</td>
</tr>
</tbody>
</table>
### Other component roles
<table>
<colgroup><col width="25%"><col width="25%"><col></colgroup>
<colgroup><col style="width: 25%;" /><col style="width: 25%;" /><col /></colgroup>
<thead>
<tr>
<th>Default ClusterRole</th>
<th>Default ClusterRoleBinding</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><b>system:auth-delegator</b></td>
<td>None</td>
@@ -786,6 +801,7 @@ This is commonly used by add-on API servers for unified authentication and autho
<td>None</td>
<td>Allows access to the resources required by most <a href="/docs/concepts/storage/persistent-volumes/#provisioner">dynamic volume provisioners</a>.</td>
</tr>
<tbody>
</table>
### Roles for built-in controllers {#controller-roles}
+1 -1
View File
@@ -2,7 +2,7 @@
title: Pod
id: pod
date: 2018-04-12
full_link: /docs/concepts/workloads/pods/pod-overview/
full_link: /docs/concepts/workloads/pods/
short_description: >
A Pod represents a set of running containers in your cluster.
@@ -33,7 +33,7 @@ if you are writing an application using the Kubernetes API.
To eliminate fields or restructure resource representations, Kubernetes supports
multiple API versions, each at a different API path. For example: `/api/v1` or
`/apis/extensions/v1beta1`.
`/apis/rbac.authorization.k8s.io/v1alpha1`.
The version is set at the API level rather than at the resource or field level to:
@@ -106,10 +106,3 @@ When you enable or disable groups or resources, you need to restart the apiserve
to pick up the `--runtime-config` changes.
{{< /note >}}
## Enabling specific resources in the extensions/v1beta1 group
DaemonSets, Deployments, StatefulSet, NetworkPolicies, PodSecurityPolicies and ReplicaSets in the `extensions/v1beta1` API group are disabled by default.
For example: to enable deployments and daemonsets, set
`--runtime-config=extensions/v1beta1/deployments=true,extensions/v1beta1/daemonsets=true`.
{{< note >}}Individual resource enablement/disablement is only supported in the `extensions/v1beta1` API group for legacy reasons.{{< /note >}}
@@ -0,0 +1,103 @@
---
title: Kubernetes API health endpoints
reviewers:
- logicalhan
content_type: concept
weight: 50
---
<!-- overview -->
The Kubernetes {{< glossary_tooltip term_id="kube-apiserver" text="API server" >}} provides API endpoints to indicate the current status of the API server.
This page describes these API endpoints and explains how you can use them.
<!-- body -->
## API endpoints for health
The Kubernetes API server provides 3 API endpoints (`healthz`, `livez` and `readyz`) to indicate the current status of the API server.
The `healthz` endpoint is deprecated (since Kubernetes v1.16), and you should use the more specific `livez` and `readyz` endpoints instead.
The `livez` endpoint can be used with the `--livez-grace-period` [flag](/docs/reference/command-line-tools-reference/kube-apiserver) to specify the startup duration.
For a graceful shutdown you can specify the `--shutdown-delay-duration` [flag](/docs/reference/command-line-tools-reference/kube-apiserver) with the `/readyz` endpoint.
Machines that check the `health`/`livez`/`readyz` of the API server should rely on the HTTP status code.
A status code `200` indicates the the API server is `healthy`/`live`/`ready`, depending of the called endpoint.
The more verbose options shown below are intended to be used by human operators to debug their cluster or specially the state of the API server.
The following examples will show how you can interact with the health API endpoints.
For all endpoints you can use the `verbose` parameter to print out the checks and their status.
This can be useful for a human operator to debug the current status of the Api server, it is not intended to be consumed by a machine:
```shell
curl -k https://localhost:6443/livez?verbose
```
or from a remote host with authentication:
```shell
kubectl get --raw='/readyz?verbose'
```
The output will look like this:
[+]ping ok
[+]log ok
[+]etcd ok
[+]poststarthook/start-kube-apiserver-admission-initializer ok
[+]poststarthook/generic-apiserver-start-informers ok
[+]poststarthook/start-apiextensions-informers ok
[+]poststarthook/start-apiextensions-controllers ok
[+]poststarthook/crd-informer-synced ok
[+]poststarthook/bootstrap-controller ok
[+]poststarthook/rbac/bootstrap-roles ok
[+]poststarthook/scheduling/bootstrap-system-priority-classes ok
[+]poststarthook/start-cluster-authentication-info-controller ok
[+]poststarthook/start-kube-aggregator-informers ok
[+]poststarthook/apiservice-registration-controller ok
[+]poststarthook/apiservice-status-available-controller ok
[+]poststarthook/kube-apiserver-autoregistration ok
[+]autoregister-completion ok
[+]poststarthook/apiservice-openapi-controller ok
healthz check passed
The Kubernetes API server also supports to exclude specific checks.
The query parameters can also be combined like in this example:
```shell
curl -k 'https://localhost:6443/readyz?verbose&exclude=etcd'
```
The output show that the `etcd` check is excluded:
[+]ping ok
[+]log ok
[+]etcd excluded: ok
[+]poststarthook/start-kube-apiserver-admission-initializer ok
[+]poststarthook/generic-apiserver-start-informers ok
[+]poststarthook/start-apiextensions-informers ok
[+]poststarthook/start-apiextensions-controllers ok
[+]poststarthook/crd-informer-synced ok
[+]poststarthook/bootstrap-controller ok
[+]poststarthook/rbac/bootstrap-roles ok
[+]poststarthook/scheduling/bootstrap-system-priority-classes ok
[+]poststarthook/start-cluster-authentication-info-controller ok
[+]poststarthook/start-kube-aggregator-informers ok
[+]poststarthook/apiservice-registration-controller ok
[+]poststarthook/apiservice-status-available-controller ok
[+]poststarthook/kube-apiserver-autoregistration ok
[+]autoregister-completion ok
[+]poststarthook/apiservice-openapi-controller ok
[+]shutdown ok
healthz check passed
## Individual health checks
{{< feature-state state="alpha" >}}
Each individual health check exposes an http endpoint and could can be checked individually.
The schema for the individual health checks is `/livez/<healthcheck-name>` where `livez` and `readyz` and be used to indicate if you want to check thee liveness or the readiness of the API server.
The `<healthcheck-name>` path can be discovered using the `verbose` flag from above and take the path between `[+]` and `ok`.
These individual health checks should not be consumed by machines but can be helpful for a human operator to debug a system:
```shell
curl -k https://localhost:6443/livez/etcd
```
@@ -20,7 +20,7 @@ At {{< param "version" >}}, Kubernetes supports clusters with up to 5000 nodes.
A cluster is a set of nodes (physical or virtual machines) running Kubernetes agents, managed by a "master" (the cluster-level control plane).
Normally the number of nodes in a cluster is controlled by the value `NUM_NODES` in the platform-specific `config-default.sh` file (for example, see [GCE's `config-default.sh`](http://releases.k8s.io/{{< param "githubbranch" >}}/cluster/gce/config-default.sh)).
Normally the number of nodes in a cluster is controlled by the value `NUM_NODES` in the platform-specific `config-default.sh` file (for example, see [GCE's `config-default.sh`](https://releases.k8s.io/{{< param "githubbranch" >}}/cluster/gce/config-default.sh)).
Simply changing that value to something very large, however, may cause the setup script to fail for many cloud providers. A GCE deployment, for example, will run in to quota issues and fail to bring the cluster up.
@@ -80,7 +80,7 @@ On AWS, master node sizes are currently set at cluster startup time and do not c
### Addon Resources
To prevent memory leaks or other resource issues in [cluster addons](https://releases.k8s.io/{{< param "githubbranch" >}}/cluster/addons) from consuming all the resources available on a node, Kubernetes sets resource limits on addon containers to limit the CPU and Memory resources they can consume (See PR [#10653](http://pr.k8s.io/10653/files) and [#10778](http://pr.k8s.io/10778/files)).
To prevent memory leaks or other resource issues in [cluster addons](https://releases.k8s.io/{{< param "githubbranch" >}}/cluster/addons) from consuming all the resources available on a node, Kubernetes sets resource limits on addon containers to limit the CPU and Memory resources they can consume (See PR [#10653](https://pr.k8s.io/10653/files) and [#10778](https://pr.k8s.io/10778/files)).
For example:
@@ -94,28 +94,26 @@ For example:
memory: 200Mi
```
Except for Heapster, these limits are static and are based on data we collected from addons running on 4-node clusters (see [#10335](http://issue.k8s.io/10335#issuecomment-117861225)). The addons consume a lot more resources when running on large deployment clusters (see [#5880](http://issue.k8s.io/5880#issuecomment-113984085)). So, if a large cluster is deployed without adjusting these values, the addons may continuously get killed because they keep hitting the limits.
Except for Heapster, these limits are static and are based on data we collected from addons running on 4-node clusters (see [#10335](https://issue.k8s.io/10335#issuecomment-117861225)). The addons consume a lot more resources when running on large deployment clusters (see [#5880](http://issue.k8s.io/5880#issuecomment-113984085)). So, if a large cluster is deployed without adjusting these values, the addons may continuously get killed because they keep hitting the limits.
To avoid running into cluster addon resource issues, when creating a cluster with many nodes, consider the following:
* Scale memory and CPU limits for each of the following addons, if used, as you scale up the size of cluster (there is one replica of each handling the entire cluster so memory and CPU usage tends to grow proportionally with size/load on cluster):
* [InfluxDB and Grafana](http://releases.k8s.io/{{< param "githubbranch" >}}/cluster/addons/cluster-monitoring/influxdb/influxdb-grafana-controller.yaml)
* [kubedns, dnsmasq, and sidecar](http://releases.k8s.io/{{< param "githubbranch" >}}/cluster/addons/dns/kube-dns/kube-dns.yaml.in)
* [Kibana](http://releases.k8s.io/{{< param "githubbranch" >}}/cluster/addons/fluentd-elasticsearch/kibana-deployment.yaml)
* [InfluxDB and Grafana](https://releases.k8s.io/{{< param "githubbranch" >}}/cluster/addons/cluster-monitoring/influxdb/influxdb-grafana-controller.yaml)
* [kubedns, dnsmasq, and sidecar](https://releases.k8s.io/{{< param "githubbranch" >}}/cluster/addons/dns/kube-dns/kube-dns.yaml.in)
* [Kibana](https://releases.k8s.io/{{< param "githubbranch" >}}/cluster/addons/fluentd-elasticsearch/kibana-deployment.yaml)
* Scale number of replicas for the following addons, if used, along with the size of cluster (there are multiple replicas of each so increasing replicas should help handle increased load, but, since load per replica also increases slightly, also consider increasing CPU/memory limits):
* [elasticsearch](http://releases.k8s.io/{{< param "githubbranch" >}}/cluster/addons/fluentd-elasticsearch/es-statefulset.yaml)
* [elasticsearch](https://releases.k8s.io/{{< param "githubbranch" >}}/cluster/addons/fluentd-elasticsearch/es-statefulset.yaml)
* Increase memory and CPU limits slightly for each of the following addons, if used, along with the size of cluster (there is one replica per node but CPU/memory usage increases slightly along with cluster load/size as well):
* [FluentD with ElasticSearch Plugin](http://releases.k8s.io/{{< param "githubbranch" >}}/cluster/addons/fluentd-elasticsearch/fluentd-es-ds.yaml)
* [FluentD with GCP Plugin](http://releases.k8s.io/{{< param "githubbranch" >}}/cluster/addons/fluentd-gcp/fluentd-gcp-ds.yaml)
* [FluentD with ElasticSearch Plugin](https://releases.k8s.io/{{< param "githubbranch" >}}/cluster/addons/fluentd-elasticsearch/fluentd-es-ds.yaml)
* [FluentD with GCP Plugin](https://releases.k8s.io/{{< param "githubbranch" >}}/cluster/addons/fluentd-gcp/fluentd-gcp-ds.yaml)
Heapster's resource limits are set dynamically based on the initial size of your cluster (see [#16185](http://issue.k8s.io/16185)
and [#22940](http://issue.k8s.io/22940)). If you find that Heapster is running
out of resources, you should adjust the formulas that compute heapster memory request (see those PRs for details).
For directions on how to detect if addon containers are hitting resource limits, see the [Troubleshooting section of Compute Resources](/docs/concepts/configuration/manage-compute-resources-container/#troubleshooting).
In the [future](http://issue.k8s.io/13048), we anticipate to set all cluster addon resource limits based on cluster size, and to dynamically adjust them if you grow or shrink your cluster.
We welcome PRs that implement those features.
For directions on how to detect if addon containers are hitting resource limits, see the
[Troubleshooting section of Compute Resources](/docs/concepts/configuration/manage-resources-containers/#troubleshooting).
### Allowing minor node failure at startup
@@ -126,3 +124,4 @@ running `kube-up.sh` set the environment variable `ALLOWED_NOTREADY_NODES` to wh
with. This will allow `kube-up.sh` to succeed with fewer than `NUM_NODES` coming up. Depending on the
reason for the failure, those additional nodes may join later or the cluster may remain at a size of
`NUM_NODES - ALLOWED_NOTREADY_NODES`.
@@ -78,7 +78,7 @@ federation support).
a single master node by default. While services are highly
available and can tolerate the loss of a zone, the control plane is
located in a single zone. Users that want a highly available control
plane should follow the [high availability](/docs/admin/high-availability) instructions.
plane should follow the [high availability](/docs/setup/production-environment/tools/kubeadm/high-availability/) instructions.
### Volume limitations
The following limitations are addressed with [topology-aware volume binding](/docs/concepts/storage/storage-classes/#volume-binding-mode).
@@ -198,7 +198,7 @@ This brief demo guides you on how to start, use, and delete Minikube locally. Fo
The `minikube start` command can be used to start your cluster.
This command creates and configures a Virtual Machine that runs a single-node Kubernetes cluster.
This command also configures your [kubectl](/docs/user-guide/kubectl-overview/) installation to communicate with this cluster.
This command also configures your [kubectl](/docs/reference/kubectl/overview/) installation to communicate with this cluster.
{{< note >}}
If you are behind a web proxy, you need to pass this information to the `minikube start` command:
@@ -514,6 +514,6 @@ For more information about Minikube, see the [proposal](https://git.k8s.io/commu
## Community
Contributions, questions, and comments are all welcomed and encouraged! Minikube developers hang out on [Slack](https://kubernetes.slack.com) in the #minikube channel (get an invitation [here](http://slack.kubernetes.io/)). We also have the [kubernetes-dev Google Groups mailing list](https://groups.google.com/forum/#!forum/kubernetes-dev). If you are posting to the list please prefix your subject with "minikube: ".
Contributions, questions, and comments are all welcomed and encouraged! Minikube developers hang out on [Slack](https://kubernetes.slack.com) in the `#minikube` channel (get an invitation [here](https://slack.kubernetes.io/)). We also have the [kubernetes-dev Google Groups mailing list](https://groups.google.com/forum/#!forum/kubernetes-dev). If you are posting to the list please prefix your subject with "minikube: ".
@@ -9,12 +9,10 @@ content_type: concept
[CloudStack](https://cloudstack.apache.org/) is a software to build public and private clouds based on hardware virtualization principles (traditional IaaS). To deploy Kubernetes on CloudStack there are several possibilities depending on the Cloud being used and what images are made available. CloudStack also has a vagrant plugin available, hence Vagrant could be used to deploy Kubernetes either using the existing shell provisioner or using new Salt based recipes.
[CoreOS](http://coreos.com) templates for CloudStack are built [nightly](http://stable.release.core-os.net/amd64-usr/current/). CloudStack operators need to [register](http://docs.cloudstack.apache.org/projects/cloudstack-administration/en/latest/templates.html) this template in their cloud before proceeding with these Kubernetes deployment instructions.
[CoreOS](https://coreos.com) templates for CloudStack are built [nightly](https://stable.release.core-os.net/amd64-usr/current/). CloudStack operators need to [register](https://docs.cloudstack.apache.org/projects/cloudstack-administration/en/latest/templates.html) this template in their cloud before proceeding with these Kubernetes deployment instructions.
This guide uses a single [Ansible playbook](https://github.com/apachecloudstack/k8s), which is completely automated and can deploy Kubernetes on a CloudStack based Cloud using CoreOS images. The playbook, creates an ssh key pair, creates a security group and associated rules and finally starts coreOS instances configured via cloud-init.
<!-- body -->
## Prerequisites
@@ -112,10 +110,7 @@ e9af8293... <node #2 IP> role=node
## Support Level
IaaS Provider | Config. Mgmt | OS | Networking | Docs | Conforms | Support Level
-------------------- | ------------ | ------ | ---------- | --------------------------------------------- | ---------| ----------------------------
CloudStack | Ansible | CoreOS | flannel | [docs](/docs/setup/production-environment/on-premises-vm/cloudstack/) | | Community ([@Guiques](https://github.com/ltupin/))
@@ -140,7 +140,7 @@ you choose for organization reasons (e.g. you are allowed to create records unde
but not under `example.com`).
Let's assume you're using `dev.example.com` as your hosted zone. You create that hosted zone using
the [normal process](http://docs.aws.amazon.com/Route53/latest/DeveloperGuide/CreatingNewSubdomain.html), or
the [normal process](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/CreatingNewSubdomain.html), or
with a command such as `aws route53 create-hosted-zone --name dev.example.com --caller-reference 1`.
You must then set up your NS records in the parent domain, so that records in the domain will resolve. Here,
@@ -231,9 +231,8 @@ See the [list of add-ons](/docs/concepts/cluster-administration/addons/) to expl
## {{% heading "whatsnext" %}}
* Learn more about Kubernetes [concepts](/docs/concepts/) and [`kubectl`](/docs/user-guide/kubectl-overview/).
* Learn more about Kubernetes [concepts](/docs/concepts/) and [`kubectl`](/docs/reference/kubectl/overview/).
* Learn more about `kops` [advanced usage](https://kops.sigs.k8s.io/) for tutorials, best practices and advanced configuration options.
* Follow `kops` community discussions on Slack: [community discussions](https://github.com/kubernetes/kops#other-ways-to-communicate-with-the-contributors)
* Contribute to `kops` by addressing or raising an issue [GitHub Issues](https://github.com/kubernetes/kops/issues)
@@ -8,7 +8,7 @@ weight: 30
<!-- overview -->
<img src="https://raw.githubusercontent.com/kubernetes/kubeadm/master/logos/stacked/color/kubeadm-stacked-color.png" align="right" width="150px">The `kubeadm` tool helps you bootstrap a minimum viable Kubernetes cluster that conforms to best practices. In fact, you can use `kubeadm` to set up a cluster that will pass the [Kubernetes Conformance tests](https://kubernetes.io/blog/2017/10/software-conformance-certification).
<img src="https://raw.githubusercontent.com/kubernetes/kubeadm/master/logos/stacked/color/kubeadm-stacked-color.png" align="right" width="150px">The `kubeadm` tool helps you bootstrap a minimum viable Kubernetes cluster that conforms to best practices. In fact, you can use `kubeadm` to set up a cluster that will pass the [Kubernetes Conformance tests](https://kubernetes.io/blog/2017/10/software-conformance-certification).
`kubeadm` also supports other cluster
lifecycle functions, such as [bootstrap tokens](/docs/reference/access-authn-authz/bootstrap-tokens/) and cluster upgrades.
@@ -254,11 +254,11 @@ Read all of this advice carefully before proceeding.
**You must deploy a
{{< glossary_tooltip text="Container Network Interface" term_id="cni" >}}
(CNI) based Pod network add-on so that your Pods can communicate with each other.
(CNI) based Pod network add-on so that your Pods can communicate with each other.
Cluster DNS (CoreDNS) will not start up before a network is installed.**
- Take care that your Pod network must not overlap with any of the host
networks: you are likely to see problems if there is any overlap.
networks: you are likely to see problems if there is any overlap.
(If you find a collision between your network plugins preferred Pod
network and some of your host networks, you should think of a suitable
CIDR block to use instead, then use that during `kubeadm init` with
@@ -266,13 +266,13 @@ Cluster DNS (CoreDNS) will not start up before a network is installed.**
- By default, `kubeadm` sets up your cluster to use and enforce use of
[RBAC](/docs/reference/access-authn-authz/rbac/) (role based access
control).
control).
Make sure that your Pod network plugin supports RBAC, and so do any manifests
that you use to deploy it.
- If you want to use IPv6--either dual-stack, or single-stack IPv6 only
networking--for your cluster, make sure that your Pod network plugin
supports IPv6.
supports IPv6.
IPv6 support was added to CNI in [v0.6.0](https://github.com/containernetworking/cni/releases/tag/v0.6.0).
{{< /caution >}}
@@ -284,10 +284,10 @@ tracker instead of the kubeadm or kubernetes issue trackers.
{{< /note >}}
Several external projects provide Kubernetes Pod networks using CNI, some of which also
support [Network Policy](/docs/concepts/services-networking/networkpolicies/).
support [Network Policy](/docs/concepts/services-networking/network-policies/).
See the list of available
[networking and network policy add-ons](/docs/concepts/cluster-administration/addons/#networking-and-network-policy).
See a list of add-ons that implement the
[Kubernetes networking model](/docs/concepts/cluster-administration/networking/#how-to-implement-the-kubernetes-networking-model).
You can install a Pod network add-on with the following command on the
control-plane node or a node that has the kubeconfig credentials:
@@ -297,79 +297,6 @@ kubectl apply -f <add-on.yaml>
```
You can install only one Pod network per cluster.
Below you can find installation instructions for some popular Pod network plugins:
{{< tabs name="tabs-pod-install" >}}
{{% tab name="Calico" %}}
[Calico](https://docs.projectcalico.org/latest/introduction/) is a networking and network policy provider. Calico supports a flexible set of networking options so you can choose the most efficient option for your situation, including non-overlay and overlay networks, with or without BGP. Calico uses the same engine to enforce network policy for hosts, pods, and (if using Istio & Envoy) applications at the service mesh layer. Calico works on several architectures, including `amd64`, `arm64`, and `ppc64le`.
Calico will automatically detect which IP address range to use for pod IPs based on the value provided via the `--pod-network-cidr` flag or via kubeadm's configuration.
```shell
kubectl apply -f https://docs.projectcalico.org/v3.14/manifests/calico.yaml
```
{{% /tab %}}
{{% tab name="Cilium" %}}
To deploy Cilium you just need to run:
```shell
kubectl create -f https://raw.githubusercontent.com/cilium/cilium/v1.8/install/kubernetes/quick-install.yaml
```
Once all Cilium Pods are marked as `READY`, you start using your cluster.
```shell
kubectl get pods -n kube-system --selector=k8s-app=cilium
```
The output is similar to this:
```
NAME READY STATUS RESTARTS AGE
cilium-drxkl 1/1 Running 0 18m
```
Cilium can be used as a replacement for kube-proxy, see [Kubernetes without kube-proxy](https://docs.cilium.io/en/stable/gettingstarted/kubeproxy-free).
For more information about using Cilium with Kubernetes, see [Kubernetes Install guide for Cilium](https://docs.cilium.io/en/stable/kubernetes/).
{{% /tab %}}
{{% tab name="Contiv-VPP" %}}
[Contiv-VPP](https://contivpp.io/) employs a programmable CNF vSwitch based on [FD.io VPP](https://fd.io/),
offering feature-rich & high-performance cloud-native networking and services.
It implements k8s services and network policies in the user space (on VPP).
Please refer to this installation guide: [Contiv-VPP Manual Installation](https://github.com/contiv/vpp/blob/master/docs/setup/MANUAL_INSTALL.md)
{{% /tab %}}
{{% tab name="Kube-router" %}}
Kube-router relies on kube-controller-manager to allocate Pod CIDR for the nodes. Therefore, use `kubeadm init` with the `--pod-network-cidr` flag.
Kube-router provides Pod networking, network policy, and high-performing IP Virtual Server(IPVS)/Linux Virtual Server(LVS) based service proxy.
For information on using the `kubeadm` tool to set up a Kubernetes cluster with Kube-router, please see the official [setup guide](https://github.com/cloudnativelabs/kube-router/blob/master/docs/kubeadm.md).
{{% /tab %}}
{{% tab name="Weave Net" %}}
For more information on setting up your Kubernetes cluster with Weave Net, please see [Integrating Kubernetes via the Addon](https://www.weave.works/docs/net/latest/kube-addon/).
Weave Net works on `amd64`, `arm`, `arm64` and `ppc64le` platforms without any extra action required.
Weave Net sets hairpin mode by default. This allows Pods to access themselves via their Service IP address
if they don't know their PodIP.
```shell
kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"
```
{{% /tab %}}
{{< /tabs >}}
Once a Pod network has been installed, you can confirm that it is working by
checking that the CoreDNS Pod is `Running` in the output of `kubectl get pods --all-namespaces`.
@@ -578,9 +505,9 @@ options.
* <a id="lifecycle" />See [Upgrading kubeadm clusters](/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/)
for details about upgrading your cluster using `kubeadm`.
* Learn about advanced `kubeadm` usage in the [kubeadm reference documentation](/docs/reference/setup-tools/kubeadm/kubeadm)
* Learn more about Kubernetes [concepts](/docs/concepts/) and [`kubectl`](/docs/user-guide/kubectl-overview/).
* Learn more about Kubernetes [concepts](/docs/concepts/) and [`kubectl`](/docs/reference/kubectl/overview/).
* See the [Cluster Networking](/docs/concepts/cluster-administration/networking/) page for a bigger list
of Pod network add-ons.
of Pod network add-ons.
* <a id="other-addons" />See the [list of add-ons](/docs/concepts/cluster-administration/addons/) to
explore other add-ons, including tools for logging, monitoring, network policy, visualization &amp;
control of your Kubernetes cluster.
@@ -22,7 +22,7 @@ and environment. [This comparison topic](/docs/setup/production-environment/tool
If you encounter issues with setting up the HA cluster, please provide us with feedback
in the kubeadm [issue tracker](https://github.com/kubernetes/kubeadm/issues/new).
See also [The upgrade documentation](/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade-1-15).
See also [The upgrade documentation](/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/).
{{< caution >}}
This page does not address running your cluster on a cloud provider. In a cloud
@@ -30,8 +30,6 @@ environment, neither approach documented here works with Service objects of type
LoadBalancer, or with dynamic PersistentVolumes.
{{< /caution >}}
## {{% heading "prerequisites" %}}
@@ -51,8 +49,6 @@ For the external etcd cluster only, you also need:
- Three additional machines for etcd members
<!-- steps -->
## First steps for both methods
@@ -13,14 +13,12 @@ weight: 100
kubeadm allows you to experimentally create a _self-hosted_ Kubernetes control
plane. This means that key components such as the API server, controller
manager, and scheduler run as [DaemonSet pods](/docs/concepts/workloads/controllers/daemonset/)
configured via the Kubernetes API instead of [static pods](/docs/tasks/administer-cluster/static-pod/)
configured via the Kubernetes API instead of [static pods](/docs/tasks/configure-pod-container/static-pod/)
configured in the kubelet via static files.
To create a self-hosted cluster see the
[kubeadm alpha selfhosting pivot](/docs/reference/setup-tools/kubeadm/kubeadm-alpha/#cmd-selfhosting) command.
<!-- body -->
#### Caveats
@@ -15,11 +15,10 @@ If your problem is not listed below, please follow the following steps:
- Go to [github.com/kubernetes/kubeadm](https://github.com/kubernetes/kubeadm/issues) and search for existing issues.
- If no issue exists, please [open one](https://github.com/kubernetes/kubeadm/issues/new) and follow the issue template.
- If you are unsure about how kubeadm works, you can ask on [Slack](http://slack.k8s.io/) in #kubeadm, or open a question on [StackOverflow](https://stackoverflow.com/questions/tagged/kubernetes). Please include
- If you are unsure about how kubeadm works, you can ask on [Slack](https://slack.k8s.io/) in `#kubeadm`,
or open a question on [StackOverflow](https://stackoverflow.com/questions/tagged/kubernetes). Please include
relevant tags like `#kubernetes` and `#kubeadm` so folks can help you.
<!-- body -->
## Not possible to join a v1.18 Node to a v1.17 cluster due to missing RBAC
@@ -8,7 +8,7 @@ weight: 30
This quickstart helps to install a Kubernetes cluster hosted on GCE, Azure, OpenStack, AWS, vSphere, Packet (bare metal), Oracle Cloud Infrastructure (Experimental) or Baremetal with [Kubespray](https://github.com/kubernetes-sigs/kubespray).
Kubespray is a composition of [Ansible](http://docs.ansible.com/) playbooks, [inventory](https://github.com/kubernetes-sigs/kubespray/blob/master/docs/ansible.md), provisioning tools, and domain knowledge for generic OS/Kubernetes clusters configuration management tasks. Kubespray provides:
Kubespray is a composition of [Ansible](https://docs.ansible.com/) playbooks, [inventory](https://github.com/kubernetes-sigs/kubespray/blob/master/docs/ansible.md), provisioning tools, and domain knowledge for generic OS/Kubernetes clusters configuration management tasks. Kubespray provides:
* a highly available cluster
* composable attributes
@@ -21,9 +21,8 @@ Kubespray is a composition of [Ansible](http://docs.ansible.com/) playbooks, [in
* openSUSE Leap 15
* continuous integration tests
To choose a tool which best fits your use case, read [this comparison](https://github.com/kubernetes-sigs/kubespray/blob/master/docs/comparisons.md) to [kubeadm](/docs/admin/kubeadm/) and [kops](/docs/setup/production-environment/tools/kops/).
To choose a tool which best fits your use case, read [this comparison](https://github.com/kubernetes-sigs/kubespray/blob/master/docs/comparisons.md) to
[kubeadm](/docs/reference/setup-tools/kubeadm/kubeadm/) and [kops](/docs/setup/production-environment/tools/kops/).
<!-- body -->
@@ -50,7 +49,7 @@ Kubespray provides the following utilities to help provision your environment:
### (2/5) Compose an inventory file
After you provision your servers, create an [inventory file for Ansible](http://docs.ansible.com/ansible/intro_inventory.html). You can do this manually or via a dynamic inventory script. For more information, see "[Building your own inventory](https://github.com/kubernetes-sigs/kubespray/blob/master/docs/getting-started.md#building-your-own-inventory)".
After you provision your servers, create an [inventory file for Ansible](https://docs.ansible.com/ansible/intro_inventory.html). You can do this manually or via a dynamic inventory script. For more information, see "[Building your own inventory](https://github.com/kubernetes-sigs/kubespray/blob/master/docs/getting-started.md#building-your-own-inventory)".
### (3/5) Plan your cluster deployment
@@ -68,7 +67,7 @@ Kubespray provides the ability to customize many aspects of the deployment:
* {{< glossary_tooltip term_id="cri-o" >}}
* Certificate generation methods
Kubespray customizations can be made to a [variable file](http://docs.ansible.com/ansible/playbooks_variables.html). If you are just getting started with Kubespray, consider using the Kubespray defaults to deploy your cluster and explore Kubernetes.
Kubespray customizations can be made to a [variable file](https://docs.ansible.com/ansible/playbooks_variables.html). If you are just getting started with Kubespray, consider using the Kubespray defaults to deploy your cluster and explore Kubernetes.
### (4/5) Deploy a Cluster
@@ -110,11 +109,9 @@ When running the reset playbook, be sure not to accidentally target your product
## Feedback
* Slack Channel: [#kubespray](https://kubernetes.slack.com/messages/kubespray/) (You can get your invite [here](http://slack.k8s.io/))
* Slack Channel: [#kubespray](https://kubernetes.slack.com/messages/kubespray/) (You can get your invite [here](https://slack.k8s.io/))
* [GitHub Issues](https://github.com/kubernetes-sigs/kubespray/issues)
## {{% heading "whatsnext" %}}
@@ -48,7 +48,7 @@ export PATH=<path/to/kubernetes-directory>/platforms/darwin/amd64:$PATH
export PATH=<path/to/kubernetes-directory>/platforms/linux/amd64:$PATH
```
An up-to-date documentation page for this tool is available here: [kubectl manual](/docs/user-guide/kubectl/)
An up-to-date documentation page for this tool is available here: [kubectl manual](/docs/reference/kubectl/kubectl/)
By default, `kubectl` will use the `kubeconfig` file generated during the cluster startup for authenticating against the API.
For more information, please read [kubeconfig files](/docs/tasks/access-application-cluster/configure-access-multiple-clusters/)
@@ -63,7 +63,8 @@ For more complete applications, please look in the [examples directory](https://
## Scaling the cluster
Adding and removing nodes through `kubectl` is not supported. You can still scale the amount of nodes manually through adjustments of the 'Desired' and 'Max' properties within the [Auto Scaling Group](http://docs.aws.amazon.com/autoscaling/latest/userguide/as-manual-scaling.html), which was created during the installation.
Adding and removing nodes through `kubectl` is not supported. You can still scale the amount of nodes manually through adjustments of the 'Desired' and 'Max' properties within the
[Auto Scaling Group](https://docs.aws.amazon.com/autoscaling/latest/userguide/as-manual-scaling.html), which was created during the installation.
## Tearing down the cluster
@@ -80,13 +81,8 @@ cluster/kube-down.sh
IaaS Provider | Config. Mgmt | OS | Networking | Docs | Conforms | Support Level
-------------------- | ------------ | ------------- | ---------- | --------------------------------------------- | ---------| ----------------------------
AWS | kops | Debian | k8s (VPC) | [docs](https://github.com/kubernetes/kops) | | Community ([@justinsb](https://github.com/justinsb))
AWS | CoreOS | CoreOS | flannel | [docs](/docs/getting-started-guides/aws) | | Community
AWS | Juju | Ubuntu | flannel, calico, canal | [docs](/docs/getting-started-guides/ubuntu) | 100% | Commercial, Community
AWS | CoreOS | CoreOS | flannel | - | | Community
AWS | Juju | Ubuntu | flannel, calico, canal | - | 100% | Commercial, Community
AWS | KubeOne | Ubuntu, CoreOS, CentOS | canal, weavenet | [docs](https://github.com/kubermatic/kubeone) | 100% | Commercial, Community
## Further reading
Please see the [Kubernetes docs](/docs/) for more details on administering
and using a Kubernetes cluster.
@@ -72,7 +72,7 @@ cluster/kube-up.sh
If you want more than one cluster running in your project, want to use a different name, or want a different number of worker nodes, see the `<kubernetes>/cluster/gce/config-default.sh` file for more fine-grained configuration before you start up your cluster.
If you run into trouble, please see the section on [troubleshooting](/docs/setup/production-environment/turnkey/gce/#troubleshooting), post to the
[Kubernetes Forum](https://discuss.kubernetes.io), or come ask questions on [Slack](/docs/troubleshooting/#slack).
[Kubernetes Forum](https://discuss.kubernetes.io), or come ask questions on `#gke` Slack channel.
The next few steps will show you:
@@ -85,7 +85,7 @@ The next few steps will show you:
The cluster startup script will leave you with a running cluster and a `kubernetes` directory on your workstation.
The [kubectl](/docs/user-guide/kubectl/) tool controls the Kubernetes cluster
The [kubectl](/docs/reference/kubectl/kubectl/) tool controls the Kubernetes cluster
manager. It lets you inspect your cluster resources, create, delete, and update
components, and much more. You will use it to look at your new cluster and bring
up example apps.
@@ -98,7 +98,7 @@ gcloud components install kubectl
{{< note >}}
The kubectl version bundled with `gcloud` may be older than the one
downloaded by the get.k8s.io install script. See [Installing kubectl](/docs/tasks/kubectl/install/)
downloaded by the get.k8s.io install script. See [Installing kubectl](/docs/tasks/tools/install-kubectl/)
document to see how you can set up the latest `kubectl` on your workstation.
{{< /note >}}
@@ -112,7 +112,7 @@ Once `kubectl` is in your path, you can use it to look at your cluster. E.g., ru
kubectl get --all-namespaces services
```
should show a set of [services](/docs/user-guide/services) that look something like this:
should show a set of [services](/docs/concepts/services-networking/service/) that look something like this:
```shell
NAMESPACE NAME TYPE CLUSTER_IP EXTERNAL_IP PORT(S) AGE
@@ -122,7 +122,7 @@ kube-system kube-ui ClusterIP 10.0.0.3 <none>
...
```
Similarly, you can take a look at the set of [pods](/docs/user-guide/pods) that were created during cluster startup.
Similarly, you can take a look at the set of [pods](/docs/concepts/workloads/pods/pod/) that were created during cluster startup.
You can do this via the
```shell
@@ -149,7 +149,7 @@ Some of the pods may take a few seconds to start up (during this time they'll sh
### Run some examples
Then, see [a simple nginx example](/docs/user-guide/simple-nginx) to try out your new cluster.
Then, see [a simple nginx example](/docs/tasks/run-application/run-stateless-application-deployment/) to try out your new cluster.
For more complete applications, please look in the [examples directory](https://github.com/kubernetes/examples/tree/{{< param "githubbranch" >}}/). The [guestbook example](https://github.com/kubernetes/examples/tree/{{< param "githubbranch" >}}/guestbook/) is a good "getting started" walkthrough.
@@ -221,9 +221,3 @@ IaaS Provider | Config. Mgmt | OS | Networking | Docs
GCE | Saltstack | Debian | GCE | [docs](/docs/setup/production-environment/turnkey/gce/) | | Project
## Further reading
Please see the [Kubernetes docs](/docs/) for more details on administering
and using a Kubernetes cluster.
@@ -17,7 +17,7 @@ Windows applications constitute a large portion of the services and applications
## Windows containers in Kubernetes
To enable the orchestration of Windows containers in Kubernetes, simply include Windows nodes in your existing Linux cluster. Scheduling Windows containers in [Pods](/docs/concepts/workloads/pods/pod-overview/) on Kubernetes is as simple and easy as scheduling Linux-based containers.
To enable the orchestration of Windows containers in Kubernetes, simply include Windows nodes in your existing Linux cluster. Scheduling Windows containers in {{< glossary_tooltip text="Pods" term_id="pod" >}} on Kubernetes is as simple and easy as scheduling Linux-based containers.
In order to run Windows containers, your Kubernetes cluster must include multiple operating systems, with control plane nodes running Linux and workers running either Windows or Linux depending on your workload needs. Windows Server 2019 is the only Windows operating system supported, enabling [Kubernetes Node](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/architecture/architecture.md#the-kubernetes-node) on Windows (including kubelet, [container runtime](https://docs.microsoft.com/en-us/virtualization/windowscontainers/deploy-containers/containerd), and kube-proxy). For a detailed explanation of Windows distribution channels see the [Microsoft documentation](https://docs.microsoft.com/en-us/windows-server/get-started-19/servicing-channels-19).
@@ -65,7 +65,7 @@ From an API and kubectl perspective, Windows containers behave in much the same
Key Kubernetes elements work the same way in Windows as they do in Linux. In this section, we talk about some of the key workload enablers and how they map to Windows.
* [Pods](/docs/concepts/workloads/pods/pod-overview/)
* [Pods](/docs/concepts/workloads/pods/)
A Pod is the basic building block of Kubernetesthe smallest and simplest unit in the Kubernetes object model that you create or deploy. You may not deploy Windows and Linux containers in the same Pod. All containers in a Pod are scheduled onto a single Node where each Node represents a specific platform and architecture. The following Pod capabilities, properties and events are supported with Windows containers:
@@ -21,7 +21,7 @@ Specific cluster deployment tools may place additional restrictions on version s
## Supported versions
Kubernetes versions are expressed as **x.y.z**,
where **x** is the major version, **y** is the minor version, and **z** is the patch version, following [Semantic Versioning](http://semver.org/) terminology.
where **x** is the major version, **y** is the minor version, and **z** is the patch version, following [Semantic Versioning](https://semver.org/) terminology.
For more information, see [Kubernetes Release Versioning](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/release/versioning.md#kubernetes-release-versioning).
The Kubernetes project maintains release branches for the most recent three minor releases ({{< skew latestVersion >}}, {{< skew prevMinorVersion >}}, {{< skew oldestMinorVersion >}}). Kubernetes 1.19 and newer receive approximately 1 year of patch support. Kubernetes 1.18 and older received approximately 9 months of patch support.
@@ -132,7 +132,7 @@ The following file is an Ingress resource that sends traffic to your Service via
1. Create `example-ingress.yaml` from the following file:
apiVersion: networking.k8s.io/v1beta1 # for versions before 1.14 use extensions/v1beta1
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: example-ingress
@@ -243,7 +243,7 @@ The following file is an Ingress resource that sends traffic to your Service via
Output:
```shell
ingress.extensions/example-ingress configured
ingress.networking/example-ingress configured
```
## Test Your Ingress
@@ -45,13 +45,14 @@ Here is the configuration file for the application Deployment:
kubectl apply -f https://k8s.io/examples/service/access/hello-application.yaml
```
The preceding command creates a
[Deployment](/docs/concepts/workloads/controllers/deployment/)
object and an associated
[ReplicaSet](/docs/concepts/workloads/controllers/replicaset/)
object. The ReplicaSet has two
[Pods](/docs/concepts/workloads/pods/pod/),
{{< glossary_tooltip text="Deployment" term_id="deployment" >}}
and an associated
{{< glossary_tooltip term_id="replica-set" text="ReplicaSet" >}}.
The ReplicaSet has two
{{< glossary_tooltip text="Pods" term_id="pod" >}}
each of which runs the Hello World application.
1. Display information about the Deployment:
```shell
kubectl get deployments hello-world
@@ -19,15 +19,15 @@ PersistentVolume.
## Why change reclaim policy of a PersistentVolume
`PersistentVolumes` can have various reclaim policies, including "Retain",
"Recycle", and "Delete". For dynamically provisioned `PersistentVolumes`,
PersistentVolumes can have various reclaim policies, including "Retain",
"Recycle", and "Delete". For dynamically provisioned PersistentVolumes,
the default reclaim policy is "Delete". This means that a dynamically provisioned
volume is automatically deleted when a user deletes the corresponding
`PersistentVolumeClaim`. This automatic behavior might be inappropriate if the volume
PersistentVolumeClaim. This automatic behavior might be inappropriate if the volume
contains precious data. In that case, it is more appropriate to use the "Retain"
policy. With the "Retain" policy, if a user deletes a `PersistentVolumeClaim`,
the corresponding `PersistentVolume` is not be deleted. Instead, it is moved to the
`Released` phase, where all of its data can be manually recovered.
policy. With the "Retain" policy, if a user deletes a PersistentVolumeClaim,
the corresponding PersistentVolume is not be deleted. Instead, it is moved to the
Released phase, where all of its data can be manually recovered.
## Changing the reclaim policy of a PersistentVolume
@@ -160,7 +160,7 @@ kubectl scale deployment --replicas=0 dns-autoscaler --namespace=kube-system
The output is:
deployment.extensions/dns-autoscaler scaled
deployment.apps/dns-autoscaler scaled
Verify that the replica count is zero:
@@ -140,7 +140,7 @@ curl -L https://github.com/kubernetes-sigs/sig-windows-tools/releases/latest/dow
### Joining a Windows worker node
{{< note >}}
You must install the `Containers` feature and install Docker. Instructions
to do so are available at [Install Docker Engine - Enterprise on Windows Servers](https://docs.docker.com/ee/docker-ee/windows/docker-ee/#install-docker-engine---enterprise).
to do so are available at [Install Docker Engine - Enterprise on Windows Servers](https://docs.mirantis.com/docker-enterprise/v3.1/dockeree-products/docker-engine-enterprise/dee-windows.html).
{{< /note >}}
{{< note >}}
@@ -36,7 +36,7 @@ This example demonstrates how to use Kubernetes namespaces to subdivide your clu
This example assumes the following:
1. You have an [existing Kubernetes cluster](/docs/setup/).
2. You have a basic understanding of Kubernetes _[Pods](/docs/concepts/workloads/pods/pod/)_, _[Services](/docs/concepts/services-networking/service/)_, and _[Deployments](/docs/concepts/workloads/controllers/deployment/)_.
2. You have a basic understanding of Kubernetes {{< glossary_tooltip text="Pods" term_id="pod" >}}, {{< glossary_tooltip term_id="service" text="Services" >}}, and {{< glossary_tooltip text="Deployments" term_id="deployment" >}}.
## Understand the default namespace
@@ -13,7 +13,7 @@ This page shows how to view, work in, and delete {{< glossary_tooltip text="name
## {{% heading "prerequisites" %}}
* Have an [existing Kubernetes cluster](/docs/setup/).
* Have a basic understanding of Kubernetes _[Pods](/docs/concepts/workloads/pods/pod/)_, _[Services](/docs/concepts/services-networking/service/)_, and _[Deployments](/docs/concepts/workloads/controllers/deployment/)_.
2. You have a basic understanding of Kubernetes {{< glossary_tooltip text="Pods" term_id="pod" >}}, {{< glossary_tooltip term_id="service" text="Services" >}}, and {{< glossary_tooltip text="Deployments" term_id="deployment" >}}.
<!-- steps -->
@@ -34,7 +34,7 @@ This task assumes that you have met the following prerequisites:
You can use `kubectl drain` to safely evict all of your pods from a
node before you perform maintenance on the node (e.g. kernel upgrade,
hardware maintenance, etc.). Safe evictions allow the pod's containers
to [gracefully terminate](/docs/concepts/workloads/pods/pod/#termination-of-pods)
to [gracefully terminate](/docs/concepts/workloads/pods/pod-lifecycle/#pod-termination)
and will respect the `PodDisruptionBudgets` you have specified.
{{< note >}}
@@ -75,7 +75,7 @@ set to RUNNING until the postStart handler completes.
Kubernetes sends the preStop event immediately before the Container is terminated.
Kubernetes' management of the Container blocks until the preStop handler completes,
unless the Pod's grace period expires. For more details, see
[Termination of Pods](/docs/concepts/workloads/pods/pod/#termination-of-pods).
[Pod Lifecycle](/docs/concepts/workloads/pods/pod-lifecycle/).
{{< note >}}
Kubernetes only sends the preStop event when a Pod is *terminated*.
@@ -14,7 +14,7 @@ without the {{< glossary_tooltip text="API server" term_id="kube-apiserver" >}}
observing them.
Unlike Pods that are managed by the control plane (for example, a
{{< glossary_tooltip text="Deployment" term_id="deployment" >}});
instead, the kubelet watches each static Pod (and restarts it if it crashes).
instead, the kubelet watches each static Pod (and restarts it if it fails).
Static Pods are always bound to one {{< glossary_tooltip term_id="kubelet" >}} on a specific node.
@@ -133,6 +133,40 @@ log audit backend using the following `kube-apiserver` flags:
- `--audit-log-maxbackup` defines the maximum number of audit log files to retain
- `--audit-log-maxsize` defines the maximum size in megabytes of the audit log file before it gets rotated
In case kube-apiserver is configured as a Pod,remember to mount the hostPath to the location of the policy file and log file. For example,
`
--audit-policy-file=/etc/kubernetes/audit-policy.yaml
--audit-log-path=/var/log/audit.log
`
then mount the volumes:
```
volumeMounts:
- mountPath: /etc/kubernetes/audit-policy.yaml
name: audit
readOnly: true
- mountPath: /var/log/audit.log
name: audit-log
readOnly: false
```
finally the hostPath:
```
- name: audit
hostPath:
path: /etc/kubernetes/audit-policy.yaml
type: File
- name: audit-log
hostPath:
path: /var/log/audit.log
type: FileOrCreate
```
### Webhook backend
Webhook backend sends audit events to a remote API, which is assumed to be the
@@ -206,7 +240,7 @@ By default truncate is disabled in both `webhook` and `log`, a cluster administr
If you're extending the Kubernetes API with the [aggregation
layer](/docs/concepts/extend-kubernetes/api-extension/apiserver-aggregation/),
y ou can also set up audit logging for the aggregated apiserver. To do this,
you can also set up audit logging for the aggregated apiserver. To do this,
pass the configuration options in the same format as described above to the
aggregated apiserver and set up the log ingesting pipeline to pick up audit
logs. Different apiservers can have different audit configurations and
@@ -118,7 +118,7 @@ You can view this resource with:
kubectl get endpoints ${SERVICE_NAME}
```
Make sure that the endpoints match up with the number of containers that you expect to be a member of your service.
Make sure that the endpoints match up with the number of pods that you expect to be members of your service.
For example, if your Service is for an nginx container with 3 replicas, you would expect to see three different
IP addresses in the Service's endpoints.
@@ -17,7 +17,8 @@ This page shows how to debug Pods and ReplicationControllers.
{{< include "task-tutorial-prereqs.md" >}} {{< version-check >}}
* You should be familiar with the basics of
[Pods](/docs/concepts/workloads/pods/pod/) and [Pod Lifecycle](/docs/concepts/workloads/pods/pod-lifecycle/).
{{< glossary_tooltip text="Pods" term_id="pod" >}} and with
Pods' [lifecycles](/docs/concepts/workloads/pods/pod-lifecycle/).
@@ -1,103 +0,0 @@
---
reviewers:
- soltysh
- sttts
- ericchiang
content_type: concept
title: Auditing with Falco
---
<!-- overview -->
### Use Falco to collect audit events
[Falco](https://falco.org/) is an open source project for intrusion and abnormality detection for Cloud Native platforms.
This section describes how to set up Falco, how to send audit events to the Kubernetes Audit endpoint exposed by Falco, and how Falco applies a set of rules to automatically detect suspicious behavior.
<!-- body -->
#### Install Falco
Install Falco by using one of the following methods:
- [Standalone Falco](https://falco.org/docs/installation)
- [Kubernetes DaemonSet](https://falco.org/docs/installation)
- [Falco Helm Chart](https://github.com/falcosecurity/charts/tree/master/falco)
Once Falco is installed make sure it is configured to expose the Audit webhook. To do so, use the following configuration:
```yaml
webserver:
enabled: true
listen_port: 8765
k8s_audit_endpoint: /k8s_audit
ssl_enabled: false
ssl_certificate: /etc/falco/falco.pem
```
This configuration is typically found in the `/etc/falco/falco.yaml` file. If Falco is installed as a Kubernetes DaemonSet, edit the `falco-config` ConfigMap and add this configuration.
#### Configure Kubernetes Audit
1. Create a [kubeconfig file](/docs/concepts/configuration/organize-cluster-access-kubeconfig/)
for the [kube-apiserver](/docs/reference/generated/kube-apiserver/) webhook audit backend.
cat <<EOF > /etc/kubernetes/audit-webhook-kubeconfig
apiVersion: v1
kind: Config
clusters:
- cluster:
server: http://<ip_of_falco>:8765/k8s_audit
name: falco
contexts:
- context:
cluster: falco
user: ""
name: default-context
current-context: default-context
preferences: {}
users: []
EOF
1. Start `kube-apiserver` with the following options:
```shell
--audit-policy-file=/etc/kubernetes/audit-policy.yaml --audit-webhook-config-file=/etc/kubernetes/audit-webhook-kubeconfig
```
#### Audit Rules
Rules devoted to Kubernetes Audit Events can be found in [k8s_audit_rules.yaml](https://github.com/falcosecurity/falco/blob/master/rules/k8s_audit_rules.yaml).
If Audit Rules is installed as a native package or using the official Docker images, Falco copies the rules file to `/etc/falco/`, so they are available for use.
There are three classes of rules.
The first class of rules looks for suspicious or exceptional activities, such as:
- Any activity by an unauthorized or anonymous user.
- Creating a pod with an unknown or disallowed image.
- Creating a privileged pod, a pod mounting a sensitive filesystem from the host, or a pod using host networking.
- Creating a NodePort service.
- Creating a ConfigMap containing private credentials, such as passwords and cloud provider secrets.
- Attaching to or executing a command on a running pod.
- Creating a namespace external to a set of allowed namespaces.
- Creating a pod or service account in the kube-system or kube-public namespaces.
- Trying to modify or delete a system ClusterRole.
- Creating a ClusterRoleBinding to the cluster-admin role.
- Creating a ClusterRole with wildcarded verbs or resources. For example, overly permissive.
- Creating a ClusterRole with write permissions or a ClusterRole that can execute commands on pods.
A second class of rules tracks resources being created or destroyed, including:
- Deployments
- Services
- ConfigMaps
- Namespaces
- Service accounts
- Role/ClusterRoles
- Role/ClusterRoleBindings
The final class of rules simply displays any Audit Event received by Falco. This rule is disabled by default, as it can be quite noisy.
For further details, see [Kubernetes Audit Events](https://falco.org/docs/event-sources/kubernetes-audit) in the Falco documentation.
@@ -129,45 +129,8 @@ If RBAC is enabled on your cluster, you must update the `system:kube-scheduler`
```
kubectl edit clusterrole system:kube-scheduler
```
```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
annotations:
rbac.authorization.kubernetes.io/autoupdate: "true"
labels:
kubernetes.io/bootstrapping: rbac-defaults
name: system:kube-scheduler
rules:
- apiGroups:
- coordination.k8s.io
resources:
- leases
verbs:
- create
- apiGroups:
- coordination.k8s.io
resourceNames:
- kube-scheduler
- my-scheduler
resources:
- leases
verbs:
- get
- update
- apiGroups:
- ""
resourceNames:
- kube-scheduler
- my-scheduler
resources:
- endpoints
verbs:
- delete
- get
- patch
- update
```
{{< codenew file="admin/sched/clusterrole.yaml" >}}
## Specify schedulers for pods
@@ -58,7 +58,7 @@ kubectl delete pods -l app=myapp
### Persistent Volumes
Deleting the Pods in a StatefulSet will not delete the associated volumes. This is to ensure that you have the chance to copy data off the volume before deleting it. Deleting the PVC after the pods have left the [terminating state](/docs/concepts/workloads/pods/pod/#termination-of-pods) might trigger deletion of the backing Persistent Volumes depending on the storage class and reclaim policy. You should never assume ability to access a volume after claim deletion.
Deleting the Pods in a StatefulSet will not delete the associated volumes. This is to ensure that you have the chance to copy data off the volume before deleting it. Deleting the PVC after the pods have terminated might trigger deletion of the backing Persistent Volumes depending on the storage class and reclaim policy. You should never assume ability to access a volume after claim deletion.
{{< note >}}
Use caution when deleting a PVC, as it may lead to data loss.
@@ -37,7 +37,7 @@ You can perform a graceful pod deletion with the following command:
kubectl delete pods <pod>
```
For the above to lead to graceful termination, the Pod **must not** specify a `pod.Spec.TerminationGracePeriodSeconds` of 0. The practice of setting a `pod.Spec.TerminationGracePeriodSeconds` of 0 seconds is unsafe and strongly discouraged for StatefulSet Pods. Graceful deletion is safe and will ensure that the [Pod shuts down gracefully](/docs/concepts/workloads/pods/pod/#termination-of-pods) before the kubelet deletes the name from the apiserver.
For the above to lead to graceful termination, the Pod **must not** specify a `pod.Spec.TerminationGracePeriodSeconds` of 0. The practice of setting a `pod.Spec.TerminationGracePeriodSeconds` of 0 seconds is unsafe and strongly discouraged for StatefulSet Pods. Graceful deletion is safe and will ensure that the Pod [shuts down gracefully](/docs/concepts/workloads/pods/pod-lifecycle/#pod-termination) before the kubelet deletes the name from the apiserver.
Kubernetes (versions 1.5 or newer) will not delete Pods just because a Node is unreachable. The Pods running on an unreachable Node enter the 'Terminating' or 'Unknown' state after a [timeout](/docs/admin/node/#node-condition). Pods may also enter these states when the user attempts graceful deletion of a Pod on an unreachable Node. The only ways in which a Pod in such a state can be removed from the apiserver are as follows:
@@ -47,7 +47,7 @@ The Dockerfile has the following content:
```
FROM php:5-apache
ADD index.php /var/www/html/index.php
COPY index.php /var/www/html/index.php
RUN chmod a+rx index.php
```
+35
View File
@@ -2,5 +2,40 @@
title: "Install Tools"
description: Set up Kubernetes tools on your computer.
weight: 10
no_list: true
---
## kubectl
The Kubernetes command-line tool, `kubectl`, allows you to run commands against
Kubernetes clusters. You can use kubectl to deploy applications, inspect and manage
cluster resources, and view logs.
See [Install and Set Up kubectl](/docs/tasks/tools/install-kubectl/) for information about how to
download and install `kubectl` and set it up for accessing your cluster.
You can also read the [`kubectl` reference documentation](/docs/reference/kubectl/).
## Minikube
[Minikube](https://minikube.sigs.k8s.io/) is a tool that lets you run
Kubernetes locally. Minikube runs a single-node Kubernetes cluster on your personal
computer (including Windows, macOS and Linux PCs) so that you can try out Kubernetes,
or for daily development work.
You can follow the official [Get Started!](https://minikube.sigs.k8s.io/docs/start/)
guide, or read [Install Minikube](/docs/tasks/tools/install-minikube/) if your focus
is on getting the tool installed.
Once you have Minikube working, you can use it to
[run a sample application](/docs/tutorials/hello-minikube/).
## kind
Like Minikube, [kind](https://kind.sigs.k8s.io/docs/) lets you run Kubernetes on
your local compute. Unlike Minikuke, kind only works with a single container runtime:
it requires that you have [Docker](https://docs.docker.com/get-docker/) installed
and configured.
[Quick Start](https://kind.sigs.k8s.io/docs/user/quick-start/) shows you what you
need to do to get up and running with kind.
@@ -28,7 +28,7 @@ You must use a kubectl version that is within one minor version difference of yo
1. Download the latest release with the command:
```
curl -LO https://storage.googleapis.com/kubernetes-release/release/`curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt`/bin/linux/amd64/kubectl
curl -LO "https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl"
```
To download a specific version, replace the `$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)` portion of the command with the specific version.
+1 -1
View File
@@ -65,7 +65,7 @@ This tutorial provides a container image that uses NGINX to echo back all the re
## Create a Deployment
A Kubernetes [*Pod*](/docs/concepts/workloads/pods/pod/) is a group of one or more Containers,
A Kubernetes [*Pod*](/docs/concepts/workloads/pods/) is a group of one or more Containers,
tied together for the purposes of administration and networking. The Pod in this
tutorial has only one Container. A Kubernetes
[*Deployment*](/docs/concepts/workloads/controllers/deployment/) checks on the health of your
@@ -20,7 +20,7 @@ weight: 20
<div class="row">
<div class="col-md-12">
<p>
A Pod is the basic execution unit of a Kubernetes application. Each Pod represents a part of a workload that is running on your cluster. <a href="/docs/concepts/workloads/pods/pod-overview/#understanding-pods">Learn more about Pods</a>.
A Pod is the basic execution unit of a Kubernetes application. Each Pod represents a part of a workload that is running on your cluster. <a href="/docs/concepts/workloads/pods/">Learn more about Pods</a>.
</p>
</div>
</div>
@@ -28,7 +28,7 @@ weight: 10
<div class="col-md-8">
<h3>Overview of Kubernetes Services</h3>
<p>Kubernetes <a href="/docs/concepts/workloads/pods/pod-overview/">Pods</a> are mortal. Pods in fact have a <a href="/docs/concepts/workloads/pods/pod-lifecycle/">lifecycle</a>. When a worker node dies, the Pods running on the Node are also lost. A <a href="/docs/concepts/workloads/controllers/replicaset/">ReplicaSet</a> might then dynamically drive the cluster back to desired state via creation of new Pods to keep your application running. As another example, consider an image-processing backend with 3 replicas. Those replicas are exchangeable; the front-end system should not care about backend replicas or even if a Pod is lost and recreated. That said, each Pod in a Kubernetes cluster has a unique IP address, even Pods on the same Node, so there needs to be a way of automatically reconciling changes among Pods so that your applications continue to function.</p>
<p>Kubernetes <a href="/docs/concepts/workloads/pods/">Pods</a> are mortal. Pods in fact have a <a href="/docs/concepts/workloads/pods/pod-lifecycle/">lifecycle</a>. When a worker node dies, the Pods running on the Node are also lost. A <a href="/docs/concepts/workloads/controllers/replicaset/">ReplicaSet</a> might then dynamically drive the cluster back to desired state via creation of new Pods to keep your application running. As another example, consider an image-processing backend with 3 replicas. Those replicas are exchangeable; the front-end system should not care about backend replicas or even if a Pod is lost and recreated. That said, each Pod in a Kubernetes cluster has a unique IP address, even Pods on the same Node, so there needs to be a way of automatically reconciling changes among Pods so that your applications continue to function.</p>
<p>A Service in Kubernetes is an abstraction which defines a logical set of Pods and a policy by which to access them. Services enable a loose coupling between dependent Pods. A Service is defined using YAML <a href="/docs/concepts/configuration/overview/#general-configuration-tips">(preferred)</a> or JSON, like all Kubernetes objects. The set of Pods targeted by a Service is usually determined by a <i>LabelSelector</i> (see below for why you might want a Service without including <code>selector</code> in the spec).</p>
+27 -34
View File
@@ -1,6 +1,7 @@
---
title: Using Source IP
content_type: tutorial
mermaid: true
min-kubernetes-server-version: v1.5
---
@@ -177,7 +178,7 @@ service/nodeport exposed
```shell
NODEPORT=$(kubectl get -o jsonpath="{.spec.ports[0].nodePort}" services nodeport)
NODES=$(kubectl get nodes -o jsonpath='{ $.items[*].status.addresses[?(@.type=="ExternalIP")].address }')
NODES=$(kubectl get nodes -o jsonpath='{ $.items[*].status.addresses[?(@.type=="InternalIP")].address }')
```
If you're running on a cloud provider, you may need to open up a firewall-rule
@@ -206,18 +207,19 @@ Note that these are not the correct client IPs, they're cluster internal IPs. Th
Visually:
```
client
\ ^
\ \
v \
node 1 <--- node 2
| ^ SNAT
| | --->
v |
endpoint
```
{{< mermaid >}}
graph LR;
client(client)-->node2[Node 2];
node2-->client;
node2-. SNAT .->node1[Node 1];
node1-. SNAT .->node2;
node1-->endpoint(Endpoint);
classDef plain fill:#ddd,stroke:#fff,stroke-width:4px,color:#000;
classDef k8s fill:#326ce5,stroke:#fff,stroke-width:4px,color:#fff;
class node1,node2,endpoint k8s;
class client plain;
{{</ mermaid >}}
To avoid this, Kubernetes has a feature to
[preserve the client source IP](/docs/tasks/access-application-cluster/create-external-load-balancer/#preserving-the-client-source-ip).
@@ -261,17 +263,18 @@ This is what happens:
Visually:
```
client
^ / \
/ / \
/ v X
node 1 node 2
^ |
| |
| v
endpoint
```
{{< mermaid >}}
graph TD;
client --> node1[Node 1];
client(client) --x node2[Node 2];
node1 --> endpoint(endpoint);
endpoint --> node1;
classDef plain fill:#ddd,stroke:#fff,stroke-width:4px,color:#000;
classDef k8s fill:#326ce5,stroke:#fff,stroke-width:4px,color:#fff;
class node1,node2,endpoint k8s;
class client plain;
{{</ mermaid >}}
@@ -324,17 +327,7 @@ deliberately failing health checks.
Visually:
```
client
|
lb VIP
/ ^
v /
health check ---> node 1 node 2 <--- health check
200 <--- ^ | ---> 500
| V
endpoint
```
![Source IP with externalTrafficPolicy](/images/docs/sourceip-externaltrafficpolicy.svg)
You can test this by setting the annotation:
@@ -52,11 +52,11 @@ kubectl apply -f https://k8s.io/examples/service/load-balancer-example.yaml
The preceding command creates a
[Deployment](/docs/concepts/workloads/controllers/deployment/)
object and an associated
[ReplicaSet](/docs/concepts/workloads/controllers/replicaset/)
object. The ReplicaSet has five
[Pods](/docs/concepts/workloads/pods/pod/),
{{< glossary_tooltip text="Deployment" term_id="deployment" >}}
and an associated
{{< glossary_tooltip term_id="replica-set" text="ReplicaSet" >}}.
The ReplicaSet has five
{{< glossary_tooltip text="Pods" term_id="pod" >}}
each of which runs the Hello World application.
1. Display information about the Deployment: