Merge remote-tracking branch 'upstream/master' into HEAD

This commit is contained in:
simplytunde
2019-08-05 17:30:25 -05:00
385 changed files with 11537 additions and 4863 deletions
+1 -1
View File
@@ -7,7 +7,7 @@ weight: 40
{{% capture overview %}}
The Concepts section helps you learn about the parts of the Kubernetes system and the abstractions Kubernetes uses to represent your cluster, and helps you obtain a deeper understanding of how Kubernetes works.
The Concepts section helps you learn about the parts of the Kubernetes system and the abstractions Kubernetes uses to represent your {{< glossary_tooltip text="cluster" term_id="cluster" length="all" >}}, and helps you obtain a deeper understanding of how Kubernetes works.
{{% /capture %}}
@@ -30,6 +30,10 @@ A node's status contains the following information:
* [Capacity and Allocatable](#capacity)
* [Info](#info)
Node status and other details about a node can be displayed using below command:
```shell
kubectl describe node <insert-node-name-here>
```
Each section is described in detail below.
### Addresses
@@ -162,7 +162,7 @@ Finally, add the same parameters into the API server start parameters.
}
}
1. Create a JSON config file for CA certificate signing request (CSR), for example,
`ca-csr.json`. Be sure the replace the values marked with angle brackets with
`ca-csr.json`. Be sure to replace the values marked with angle brackets with
real values you want to use.
{
@@ -336,10 +336,10 @@ should appear in the `[Metadata]` section of the `cloud.conf` file:
both configuration drive and metadata service though and only one or the other
may be available which is why the default is to check both.
##### Router
##### Route
These configuration options for the OpenStack provider pertain to the [kubenet]
Kubernetes network plugin and should appear in the `[Router]` section of the
Kubernetes network plugin and should appear in the `[Route]` section of the
`cloud.conf` file:
* `router-id` (Optional): If the underlying cloud's Neutron deployment supports
@@ -36,6 +36,9 @@ To run this pod, use the following command:
```shell
kubectl apply -f https://k8s.io/examples/debug/counter-pod.yaml
```
The output is:
```
pod/counter created
```
@@ -43,6 +46,9 @@ To fetch the logs, use the `kubectl logs` command, as follows:
```shell
kubectl logs counter
```
The output is:
```
0: Mon Jan 1 00:00:00 UTC 2001
1: Mon Jan 1 00:00:01 UTC 2001
2: Mon Jan 1 00:00:02 UTC 2001
@@ -105,7 +111,7 @@ systemd is not present, they write to `.log` files in the `/var/log` directory.
System components inside containers always write to the `/var/log` directory,
bypassing the default logging mechanism. They use the [klog][klog]
logging library. You can find the conventions for logging severity for those
components in the [development docs on logging](https://git.k8s.io/community/contributors/devel/logging.md).
components in the [development docs on logging](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-instrumentation/logging.md).
Similarly to the container logs, system component logs in the `/var/log`
directory should be rotated. In Kubernetes clusters brought up by
@@ -1,6 +1,5 @@
---
reviewers:
- bgrant0607
- janetkuo
- mikedanese
title: Managing Resources
@@ -98,6 +98,14 @@ AOS supports the use of common vendor equipment from manufacturers including Cis
Details on how the AOS system works can be accessed here: http://www.apstra.com/products/how-it-works/
### AWS VPC CNI for Kubernetes
The [AWS VPC CNI](https://github.com/aws/amazon-vpc-cni-k8s) offers integrated AWS Virtual Private Cloud (VPC) networking for Kubernetes clusters. This CNI plugin offers high throughput and availability, low latency, and minimal network jitter. Additionally, users can apply existing AWS VPC networking and security best practices for building Kubernetes clusters. This includes the ability to use VPC flow logs, VPC routing policies, and security groups for network traffic isolation.
Using this CNI plugin allows Kubernetes pods to have the same IP address inside the pod as they do on the VPC network. The CNI allocates AWS Elastic Networking Interfaces (ENIs) to each Kubernetes node and using the secondary IP range from each ENI for pods on the node. The CNI includes controls for pre-allocation of ENIs and IP addresses for fast pod startup times and enables large clusters of up to 2,000 nodes.
Additionally, the CNI can be run alongside [Calico for network policy enforcement](https://docs.aws.amazon.com/eks/latest/userguide/calico.html). The AWS VPC CNI project is open source with [documentation on GitHub](https://github.com/aws/amazon-vpc-cni-k8s).
### Big Cloud Fabric from Big Switch Networks
[Big Cloud Fabric](https://www.bigswitch.com/container-network-automation) is a cloud native networking architecture, designed to run Kubernetes in private cloud/on-premises environments. Using unified physical & virtual SDN, Big Cloud Fabric tackles inherent container networking problems such as load balancing, visibility, troubleshooting, security policies & container traffic monitoring.
@@ -213,6 +221,10 @@ traffic to the internet.
[Knitter](https://github.com/ZTE/Knitter/) is a network solution which supports multiple networking in Kubernetes. It provides the ability of tenant management and network management. Knitter includes a set of end-to-end NFV container networking solutions besides multiple network planes, such as keeping IP address for applications, IP address migration, etc.
### Kube-OVN
[Kube-OVN](https://github.com/alauda/kube-ovn) is an OVN-based kubernetes network fabric for enterprises. With the help of OVN/OVS, it provides some advanced overlay network features like subnet, QoS, static IP allocation, traffic mirroring, gateway, openflow-based network policy and service proxy.
### Kube-router
[Kube-router](https://github.com/cloudnativelabs/kube-router) is a purpose-built networking solution for Kubernetes that aims to provide high performance and operational simplicity. Kube-router provides a Linux [LVS/IPVS](http://www.linuxvirtualserver.org/software/ipvs.html)-based service proxy, a Linux kernel forwarding-based pod-to-pod networking solution with no overlays, and iptables/ipset-based network policy enforcer.
@@ -11,8 +11,9 @@ weight: 30
{{% capture overview %}}
You can constrain a [pod](/docs/concepts/workloads/pods/pod/) to only be able to run on particular [nodes](/docs/concepts/architecture/nodes/) or to prefer to
run on particular nodes. There are several ways to do this, and the recommended approaches all use
You can constrain a {{< glossary_tooltip text="Pod" term_id="pod" >}} to only be able to run on particular
{{< glossary_tooltip text="Node(s)" term_id="node" >}}, or to prefer to run on particular nodes.
There are several ways to do this, and the recommended approaches all use
[label selectors](/docs/concepts/overview/working-with-objects/labels/) to make the selection.
Generally such constraints are unnecessary, as the scheduler will automatically do a reasonable placement
(e.g. spread your pods across nodes, not place the pod on a node with insufficient free resources, etc.)
@@ -20,9 +21,6 @@ but there are some circumstances where you may want more control on a node where
that a pod ends up on a machine with an SSD attached to it, or to co-locate pods from two different
services that communicate a lot into the same availability zone.
You can find all the files for these examples [in our docs
repo here](https://github.com/kubernetes/website/tree/{{< param "docsbranch" >}}/content/en/docs/concepts/configuration/).
{{% /capture %}}
{{% capture body %}}
@@ -38,14 +36,12 @@ Let's walk through an example of how to use `nodeSelector`.
### Step Zero: Prerequisites
This example assumes that you have a basic understanding of Kubernetes pods and that you have [turned up a Kubernetes cluster](https://github.com/kubernetes/kubernetes#documentation).
This example assumes that you have a basic understanding of Kubernetes pods and that you have [set up a Kubernetes cluster](/docs/setup/).
### Step One: Attach label to the node
Run `kubectl get nodes` to get the names of your cluster's nodes. Pick out the one that you want to add a label to, and then run `kubectl label nodes <node-name> <label-key>=<label-value>` to add a label to the node you've chosen. For example, if my node name is 'kubernetes-foo-node-1.c.a-robinson.internal' and my desired label is 'disktype=ssd', then I can run `kubectl label nodes kubernetes-foo-node-1.c.a-robinson.internal disktype=ssd`.
If this fails with an "invalid command" error, you're likely using an older version of kubectl that doesn't have the `label` command. In that case, see the [previous version](https://github.com/kubernetes/kubernetes/blob/a053dbc313572ed60d89dae9821ecab8bfd676dc/examples/node-selection/README.md) of this guide for instructions on how to manually set labels on a node.
You can verify that it worked by re-running `kubectl get nodes --show-labels` and checking that the node now has a label. You can also use `kubectl describe node "nodename"` to see the full list of labels of the given node.
### Step Two: Add a nodeSelector field to your pod configuration
@@ -74,17 +70,17 @@ the Pod will get scheduled on the node that you attached the label to. You can
verify that it worked by running `kubectl get pods -o wide` and looking at the
"NODE" that the Pod was assigned to.
## Interlude: built-in node labels
## Interlude: built-in node labels {#built-in-node-labels}
In addition to labels you [attach](#step-one-attach-label-to-the-node), nodes come pre-populated
with a standard set of labels. As of Kubernetes v1.4 these labels are
with a standard set of labels. These labels are
* `kubernetes.io/hostname`
* `failure-domain.beta.kubernetes.io/zone`
* `failure-domain.beta.kubernetes.io/region`
* `beta.kubernetes.io/instance-type`
* `kubernetes.io/os`
* `kubernetes.io/arch`
* [`kubernetes.io/hostname`](/docs/reference/kubernetes-api/labels-annotations-taints/#kubernetes-io-hostname)
* [`failure-domain.beta.kubernetes.io/zone`](/docs/reference/kubernetes-api/labels-annotations-taints/#failure-domain-beta-kubernetes-io-zone)
* [`failure-domain.beta.kubernetes.io/region`](/docs/reference/kubernetes-api/labels-annotations-taints/#failure-domain-beta-kubernetes-io-region)
* [`beta.kubernetes.io/instance-type`](/docs/reference/kubernetes-api/labels-annotations-taints/#beta-kubernetes-io-instance-type)
* [`kubernetes.io/os`](/docs/reference/kubernetes-api/labels-annotations-taints/#kubernetes-io-os)
* [`kubernetes.io/arch`](/docs/reference/kubernetes-api/labels-annotations-taints/#kubernetes-io-arch)
{{< note >}}
The value of these labels is cloud provider specific and is not guaranteed to be reliable.
@@ -103,14 +99,15 @@ and influencing the scheduler to schedule workloads to the compromised node.
The `NodeRestriction` admission plugin prevents kubelets from setting or modifying labels with a `node-restriction.kubernetes.io/` prefix.
To make use of that label prefix for node isolation:
1. Ensure you are using the [Node authorizer](/docs/reference/access-authn-authz/node/) and have enabled the [NodeRestriction admission plugin](/docs/reference/access-authn-authz/admission-controllers/#noderestriction).
2. Add labels under the `node-restriction.kubernetes.io/` prefix to your Node objects, and use those labels in your node selectors.
1. Check that you're using Kubernetes v1.11+ so that NodeRestriction is available.
2. Ensure you are using the [Node authorizer](/docs/reference/access-authn-authz/node/) and have _enabled_ the [NodeRestriction admission plugin](/docs/reference/access-authn-authz/admission-controllers/#noderestriction).
3. Add labels under the `node-restriction.kubernetes.io/` prefix to your Node objects, and use those labels in your node selectors.
For example, `example.com.node-restriction.kubernetes.io/fips=true` or `example.com.node-restriction.kubernetes.io/pci-dss=true`.
## Affinity and anti-affinity
`nodeSelector` provides a very simple way to constrain pods to nodes with particular labels. The affinity/anti-affinity
feature, currently in beta, greatly expands the types of constraints you can express. The key enhancements are
feature, greatly expands the types of constraints you can express. The key enhancements are
1. the language is more expressive (not just "AND of exact match")
2. you can indicate that the rule is "soft"/"preference" rather than a hard requirement, so if the scheduler
@@ -126,9 +123,8 @@ described in the third item listed above, in addition to having the first and se
`nodeSelector` continues to work as usual, but will eventually be deprecated, as node affinity can express
everything that `nodeSelector` can express.
### Node affinity (beta feature)
### Node affinity
Node affinity was introduced as alpha in Kubernetes 1.2.
Node affinity is conceptually similar to `nodeSelector` -- it allows you to constrain which nodes your
pod is eligible to be scheduled on, based on labels on the node.
@@ -143,7 +139,7 @@ met, the pod will still continue to run on the node. In the future we plan to of
except that it will evict pods from nodes that cease to satisfy the pods' node affinity requirements.
Thus an example of `requiredDuringSchedulingIgnoredDuringExecution` would be "only run the pod on nodes with Intel CPUs"
and an example `preferredDuringSchedulingIgnoredDuringExecution` would be "try to run this set of pods in availability
and an example `preferredDuringSchedulingIgnoredDuringExecution` would be "try to run this set of pods in failure
zone XYZ, but if it's not possible, then allow some to run elsewhere".
Node affinity is specified as field `nodeAffinity` of field `affinity` in the PodSpec.
@@ -172,21 +168,17 @@ If you remove or change the label of the node where the pod is scheduled, the po
The `weight` field in `preferredDuringSchedulingIgnoredDuringExecution` is in the range 1-100. For each node that meets all of the scheduling requirements (resource request, RequiredDuringScheduling affinity expressions, etc.), the scheduler will compute a sum by iterating through the elements of this field and adding "weight" to the sum if the node matches the corresponding MatchExpressions. This score is then combined with the scores of other priority functions for the node. The node(s) with the highest total score are the most preferred.
For more information on node affinity, see the
[design doc](https://git.k8s.io/community/contributors/design-proposals/scheduling/nodeaffinity.md).
### Inter-pod affinity and anti-affinity
### Inter-pod affinity and anti-affinity (beta feature)
Inter-pod affinity and anti-affinity were introduced in Kubernetes 1.4.
Inter-pod affinity and anti-affinity allow you to constrain which nodes your pod is eligible to be scheduled *based on
labels on pods that are already running on the node* rather than based on labels on nodes. The rules are of the form "this pod should (or, in the case of
anti-affinity, should not) run in an X if that X is already running one or more pods that meet rule Y". Y is expressed
as a LabelSelector with an associated list of namespaces; unlike nodes, because pods are namespaced
labels on pods that are already running on the node* rather than based on labels on nodes. The rules are of the form
"this pod should (or, in the case of anti-affinity, should not) run in an X if that X is already running one or more pods that meet rule Y".
Y is expressed as a LabelSelector with an optional associated list of namespaces; unlike nodes, because pods are namespaced
(and therefore the labels on pods are implicitly namespaced),
a label selector over pod labels must specify which namespaces the selector should apply to. Conceptually X is a topology domain
like node, rack, cloud provider zone, cloud provider region, etc. You express it using a `topologyKey` which is the
key for the node label that the system uses to denote such a topology domain, e.g. see the label keys listed above
in the section [Interlude: built-in node labels](#interlude-built-in-node-labels).
in the section [Interlude: built-in node labels](#built-in-node-labels).
{{< note >}}
Inter-pod affinity and anti-affinity require substantial amount of
@@ -360,12 +352,6 @@ no two instances are located on the same host.
See [ZooKeeper tutorial](/docs/tutorials/stateful-application/zookeeper/#tolerating-node-failure)
for an example of a StatefulSet configured with anti-affinity for high availability, using the same technique.
For more information on inter-pod affinity/anti-affinity, see the
[design doc](https://git.k8s.io/community/contributors/design-proposals/scheduling/podaffinity.md).
You may want to check [Taints](/docs/concepts/configuration/taint-and-toleration/)
as well, which allow a *node* to *repel* a set of pods.
## nodeName
`nodeName` is the simplest form of node selection constraint, but due
@@ -405,4 +391,10 @@ The above pod will run on the node kube-01.
{{% capture whatsnext %}}
[Taints](/docs/concepts/configuration/taint-and-toleration/) allow a Node to *repel* a set of Pods.
The design documents for
[node affinity](https://git.k8s.io/community/contributors/design-proposals/scheduling/nodeaffinity.md)
and for [inter-pod affinity/anti-affinity](https://git.k8s.io/community/contributors/design-proposals/scheduling/podaffinity.md) contain extra background information about these features.
{{% /capture %}}
@@ -94,7 +94,7 @@ old-style command line options. Below is a sample component config to disable
preemption:
```yaml
apiVersion: componentconfig/v1alpha1
apiVersion: kubescheduler.config.k8s.io/v1alpha1
kind: KubeSchedulerConfiguration
algorithmSource:
provider: DefaultProvider
@@ -18,7 +18,7 @@ scheduling "core" simple and maintainable. Refer to the [design proposal of the
scheduling framework][kep] for more technical information on the design of the
framework.
[kep]: (https://github.com/kubernetes/enhancements/blob/master/keps/sig-scheduling/20180409-scheduling-framework.md)
[kep]: https://github.com/kubernetes/enhancements/blob/master/keps/sig-scheduling/20180409-scheduling-framework.md
{{% /capture %}}
@@ -54,7 +54,7 @@ equivalent to "Predicate" and "Scoring" is equivalent to "Priority function".
One plugin may register at multiple extension points to perform more complex or
stateful tasks.
{{< figure src="/images/docs/scheduling-framework.png" title="scheduling framework extension points" >}}
{{< figure src="/images/docs/scheduling-framework-extensions.png" title="scheduling framework extension points" >}}
### Queue sort
@@ -238,7 +238,7 @@ decode and process the configuration.
The following example shows a scheduler configuration that enables some
plugins at `reserve` and `preBind` extension points and disables a plugin. It
also provides a configuration to plugin `foo`.
also provides a configuration to plugin `foo`.
```yaml
apiVersion: kubescheduler.config.k8s.io/v1alpha1
@@ -274,7 +274,7 @@ calling default plugins is desired, default plugins must be `disabled` and
`enabled` in the desired order.
Assuming there is a default plugin called `foo` at `reserve` and we are adding
pluing `bar` that we want to be invoked before `foo`, we should disable `foo`
plugin `bar` that we want to be invoked before `foo`, we should disable `foo`
and enable `bar` and `foo` in order. The following example shows the
configuration that achieves this:
@@ -359,6 +359,38 @@ echo 'MWYyZDFlMmU2N2Rm' | base64 --decode
1f2d1e2e67df
```
#### Editing a Secret
An existing secret may be edited with the following command:
```shell
kubectl edit secrets mysecret
```
This will open the default configured editor and allow for updating the base64 encoded secret values in the `data` field:
```
# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: v1
data:
username: YWRtaW4=
password: MWYyZDFlMmU2N2Rm
kind: Secret
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: { ... }
creationTimestamp: 2016-01-22T18:41:56Z
name: mysecret
namespace: default
resourceVersion: "164619"
selfLink: /api/v1/namespaces/default/secrets/mysecret
uid: cfee02d6-c137-11e5-8d73-42010af00002
type: Opaque
```
## Using Secrets
Secrets can be mounted as data volumes or be exposed as
@@ -833,7 +865,7 @@ EOF
Apply all those objects on the Apiserver by
```shell
kubectl apply --k .
kubectl apply -k .
```
Both containers will have the following files present on their filesystems with the values for each container's environment:
@@ -60,7 +60,7 @@ Credentials can be provided in several ways:
- Per-cluster
- automatically configured on Google Compute Engine or Google Kubernetes Engine
- all pods can read the project's private registry
- Using AWS EC2 Container Registry (ECR)
- Using Amazon Elastic Container Registry (ECR)
- use IAM roles and policies to control access to ECR repositories
- automatically refreshes ECR login credentials
- Using Oracle Cloud Infrastructure Registry (OCIR)
@@ -93,10 +93,9 @@ Google service account. The service account on the instance
will have a `https://www.googleapis.com/auth/devstorage.read_only`,
so it can pull from the project's GCR, but not push.
### Using AWS EC2 Container Registry
### Using Amazon Elastic Container Registry
Kubernetes has native support for the [AWS EC2 Container
Registry](https://aws.amazon.com/ecr/), when nodes are AWS EC2 instances.
Kubernetes has native support for the [Amazon Elastic Container Registry](https://aws.amazon.com/ecr/), when nodes are AWS EC2 instances.
Simply use the full image name (e.g. `ACCOUNT.dkr.ecr.REGION.amazonaws.com/imagename:tag`)
in the Pod definition.
@@ -142,7 +142,7 @@ In order to monitor resources provided by device plugins, monitoring agents need
discover the set of devices that are in-use on the node and obtain metadata to describe which
container the metric should be associated with. Prometheus metrics exposed by device monitoring
agents should follow the
[Kubernetes Instrumentation Guidelines](https://github.com/kubernetes/community/blob/master/contributors/devel/instrumentation.md),
[Kubernetes Instrumentation Guidelines](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-instrumentation/instrumentation.md),
which requires identifying containers using `pod`, `namespace`, and `container` prometheus labels.
The kubelet provides a gRPC service to enable discovery of in-use devices, and to provide metadata
for these devices:
@@ -16,7 +16,7 @@ there is rarely a need to fork or submit patches to the Kubernetes
project code.
This guide describes the options for customizing a Kubernetes
cluster. It is aimed at {{< glossary_tooltip text="Cluster Operators" term_id="cluster-operator" >}} who want to
cluster. It is aimed at {{< glossary_tooltip text="cluster operators" term_id="cluster-operator" >}} who want to
understand how to adapt their Kubernetes cluster to the needs of
their work environment. Developers who are prospective {{< glossary_tooltip text="Platform Developers" term_id="platform-developer" >}} or Kubernetes Project {{< glossary_tooltip text="Contributors" term_id="contributor" >}} will also find it
useful as an introduction to what extension points and patterns
@@ -122,7 +122,7 @@ For more about Custom Resources, see the [Custom Resources concept guide](/docs/
### Combining New APIs with Automation
Often, when you add a new API, you also add a control loop that reads and/or writes the new APIs. When the combination of a Custom API and a control loop is used to manage a specific, usually stateful, application, this is called the *Operator* pattern. Custom APIs and control loops can also be used to control other resources, such as storage, policies, and so on.
The combination of a custom resource API and a control loop is called the [Operator pattern](/docs/concepts/extend-kubernetes/operator/). The Operator pattern is used to manage specific, usually stateful, applications. These custom APIs and control loops can also be used to control other resources, such as storage or policies.
### Changing Built-in Resources
@@ -205,7 +205,6 @@ the nodes chosen for a pod.
* [Network Plugins](/docs/concepts/cluster-administration/network-plugins/)
* [Device Plugins](/docs/concepts/cluster-administration/device-plugins/)
* Learn about [kubectl plugins](/docs/tasks/extend-kubectl/kubectl-plugins/)
* See examples of Automation
* [List of Operators](https://github.com/operator-framework/awesome-operators)
* Learn about the [Operator pattern](/docs/concepts/extend-kubernetes/operator/)
{{% /capture %}}
@@ -0,0 +1,132 @@
---
title: Operator pattern
content_template: templates/concept
weight: 30
---
{{% capture overview %}}
Operators are software extensions to Kubernetes that make use of [custom
resources](/docs/concepts/extend-kubernetes/api-extension/custom-resources/)
to manage applications and their components. Operators follow
Kubernetes principles, notably the [control loop](/docs/concepts/#kubernetes-control-plane).
{{% /capture %}}
{{% capture body %}}
## Motivation
The Operator pattern aims to capture the key aim of a human operator who
is managing a service or set of services. Human operators who look after
specific applications and services have deep knowledge of how the system
ought to behave, how to deploy it, and how to react if there are problems.
People who run workloads on Kubernetes often like to use automation to take
care of repeatable tasks. The Operator pattern captures how you can write
code to automate a task beyond what Kubernetes itself provides.
## Operators in Kubernetes
Kubernetes is designed for automation. Out of the box, you get lots of
built-in automation from the core of Kubernetes. You can use Kubernetes
to automate deploying and running workloads, *and* you can automate how
Kubernetes does that.
Kubernetes' {{< glossary_tooltip text="controllers" term_id="controller" >}}
concept lets you extend the cluster's behaviour without modifying the code
of Kubernetes itself.
Operators are clients of the Kubernetes API that act as controllers for
a [Custom Resource](/docs/concepts/api-extension/custom-resources/).
## An example Operator {#example}
Some of the things that you can use an operator to automate include:
* deploying an application on demand
* taking and restoring backups of that application's state
* handling upgrades of the application code alongside related changes such
as database schemas or extra configuration settings
* publishing a Service to applications that don't support Kubernetes APIs to
discover them
* simulating failure in all or part of your cluster to test its resilience
* choosing a leader for a distributed application without an internal
member election process
What might an Operator look like in more detail? Here's an example in more
detail:
1. A custom resource named SampleDB, that you can configure into the cluster.
2. A Deployment that makes sure a Pod is running that contains the
controller part of the operator.
3. A container image of the operator code.
4. Controller code that queries the control plane to find out what SampleDB
resources are configured.
5. The core of the Operator is code to tell the API server how to make
reality match the configured resources.
* If you add a new SampleDB, the operator sets up PersistentVolumeClaims
to provide durable database storage, a StatefulSet to run SampleDB and
a Job to handle initial configuration.
* If you delete it, the Operator takes a snapshot, then makes sure that
the the StatefulSet and Volumes are also removed.
6. The operator also manages regular database backups. For each SampleDB
resource, the operator determines when to create a Pod that can connect
to the database and take backups. These Pods would rely on a ConfigMap
and / or a Secret that has database connection details and credentials.
7. Because the Operator aims to provide robust automation for the resource
it manages, there would be additional supporting code. For this example,
code checks to see if the database is running an old version and, if so,
creates Job objects that upgrade it for you.
## Deploying Operators
The most common way to deploy an Operator is to add the
Custom Resource Definition and its associated Controller to your cluster.
The Controller will normally run outside of the
{{< glossary_tooltip text="control plane" term_id="control-plane" >}},
much as you would run any containerized application.
For example, you can run the controller in your cluster as a Deployment.
## Using an Operator {#using-operators}
Once you have an Operator deployed, you'd use it by adding, modifying or
deleting the kind of resource that the Operator uses. Following the above
example, you would set up a Deployment for the Operator itself, and then:
```shell
kubectl get SampleDB # find configured databases
kubectl edit SampleDB/example-database # manually change some settings
```
&hellip;and that's it! The Operator will take care of applying the changes
as well as keeping the existing service in good shape.
## Writing your own Operator {#writing-operator}
If there isn't an Operator in the ecosystem that implements the behavior you
want, you can code your own. In [What's next](#what-s-next) you'll find a few
links to libraries and tools you can use to write your own cloud native
Operator.
You also implement an Operator (that is, a Controller) using any language / runtime
that can act as a [client for the Kubernetes API](/docs/reference/using-api/client-libraries/).
{{% /capture %}}
{{% capture whatsnext %}}
* Learn more about [Custom Resources](/docs/concepts/extend-kubernetes/api-extension/custom-resources/)
* Find ready-made operators on [OperatorHub.io](https://operatorhub.io/) to suit your use case
* Use existing tools to write your own operator, eg:
* using [KUDO](https://kudo.dev/) (Kubernetes Universal Declarative Operator)
* using [kubebuilder](https://book.kubebuilder.io/)
* using [Metacontroller](https://metacontroller.app/) along with WebHooks that
you implement yourself
* using the [Operator Framework](https://github.com/operator-framework/getting-started)
* [Publish](https://operatorhub.io/) your operator for other people to use
* Read [CoreOS' original article](https://coreos.com/blog/introducing-operators.html) that introduced the Operator pattern
* Read an [article](https://cloud.google.com/blog/products/containers-kubernetes/best-practices-for-building-kubernetes-operators-and-stateful-apps) from Google Cloud about best practices for building Operators
{{% /capture %}}
@@ -18,7 +18,7 @@ deliver a functioning Kubernetes cluster.
## Master Components
Master components provide the cluster's control plane. Master components make global decisions about the
cluster (for example, scheduling), and they detect and respond to cluster events (for example, starting up a new pod when a replication controller's `replicas` field is unsatisfied).
cluster (for example, scheduling), and they detect and respond to cluster events (for example, starting up a new {{< glossary_tooltip text="pod" term_id="pod">}} when a deployment's `replicas` field is unsatisfied).
Master components can be run on any machine in the cluster. However,
for simplicity, set up scripts typically start all master components on
@@ -82,11 +82,13 @@ Node components run on every node, maintaining running pods and providing the Ku
## Addons
Addons are pods and services that implement cluster features. The pods may be managed
by Deployments, ReplicationControllers, and so on. Namespaced addon objects are created in
the `kube-system` namespace.
Addons use Kubernetes resources ({{< glossary_tooltip term_id="daemonset" >}},
{{< glossary_tooltip term_id="deployment" >}}, etc)
to implement cluster features. Because these are providing cluster-level features, namespaced resources
for addons belong within the `kube-system` namespace.
Selected addons are described below, for an extended list of available addons please see [Addons](/docs/concepts/cluster-administration/addons/).
Selected addons are described below; for an extended list of available addons, please
see [Addons](/docs/concepts/cluster-administration/addons/).
### DNS
@@ -111,5 +113,8 @@ A [Cluster-level logging](/docs/concepts/cluster-administration/logging/) mechan
saving container logs to a central log store with search/browsing interface.
{{% /capture %}}
{{% capture whatsnext %}}
* Learn about [Nodes](/docs/concepts/architecture/nodes/)
* Learn about [kube-scheduler](/docs/concepts/scheduling/kube-scheduler/)
* Read etcd's official [documentation](https://etcd.io/docs/)
{{% /capture %}}
@@ -48,9 +48,9 @@ Containers are becoming popular because they have many benefits. Some of the con
* Resource isolation: predictable application performance.
* Resource utilization: high efficiency and density.
## Why do I need Kubernetes and what can it do
## Why you need Kubernetes and what can it do
Containers are a good way to bundle and run your applications. In a production environment, you need to manage the containers that run the applications and ensure that there is no downtime. For example, if a container goes down, another container needs to restart. Wouldn't it be easier if this behavior was handled by a system?
Containers are a good way to bundle and run your applications. In a production environment, you need to manage the containers that run the applications and ensure that there is no downtime. For example, if a container goes down, another container needs to start. Wouldn't it be easier if this behavior was handled by a system?
That's how Kubernetes comes to the rescue! Kubernetes provides you with a framework to run distributed systems resiliently. It takes care of your scaling requirements, failover, deployment patterns, and more. For example, Kubernetes can easily manage a canary deployment for your system.
@@ -145,10 +145,10 @@ metadata:
labels:
app.kubernetes.io/name: mysql
app.kubernetes.io/instance: mysql-abcxzy
app.kubernetes.io/version: "5.7.21"
app.kubernetes.io/managed-by: helm
app.kubernetes.io/component: database
app.kubernetes.io/part-of: wordpress
app.kubernetes.io/version: "5.7.21"
...
```
@@ -161,10 +161,10 @@ metadata:
labels:
app.kubernetes.io/name: mysql
app.kubernetes.io/instance: mysql-abcxzy
app.kubernetes.io/version: "5.7.21"
app.kubernetes.io/managed-by: helm
app.kubernetes.io/component: database
app.kubernetes.io/part-of: wordpress
app.kubernetes.io/version: "5.7.21"
...
```
@@ -56,5 +56,5 @@ kubectl get pods --field-selector=status.phase!=Running,spec.restartPolicy=Alway
You use field selectors across multiple resource types. This `kubectl` command selects all Statefulsets and Services that are not in the `default` namespace:
```shell
kubectl get statefulsets,services --field-selector metadata.namespace!=default
kubectl get statefulsets,services --all-namespaces --field-selector metadata.namespace!=default
```
@@ -65,13 +65,13 @@ Kubernetes starts with three initial namespaces:
### Setting the namespace for a request
To temporarily set the namespace for a request, use the `--namespace` flag.
To set the namespace for a current request, use the `--namespace` flag.
For example:
```shell
kubectl --namespace=<insert-namespace-name-here> run nginx --image=nginx
kubectl --namespace=<insert-namespace-name-here> get pods
kubectl run nginx --image=nginx --namespace=<insert-namespace-name-here>
kubectl get pods --namespace=<insert-namespace-name-here>
```
### Setting the namespace preference
@@ -80,7 +80,7 @@ You can permanently save the namespace for all subsequent kubectl commands in th
context.
```shell
kubectl config set-context $(kubectl config current-context) --namespace=<insert-namespace-name-here>
kubectl config set-context --current --namespace=<insert-namespace-name-here>
# Validate it
kubectl config view | grep namespace:
```
@@ -69,7 +69,7 @@ kubectl create namespace limitrange-demo
To avoid passing the target limitrange-demo in your kubectl commands, change your context with the following command
```shell
kubectl config set-context $(kubectl config current-context) --namespace=limitrange-demo
kubectl config set-context --current --namespace=limitrange-demo
```
Here is the configuration file for a LimitRange object:
@@ -158,12 +158,16 @@ also be used to provide default values for many of the fields that it
controls. When multiple policies are available, the pod security policy
controller selects policies according to the following criteria:
1. If any policies successfully validate the pod without altering it, they are
used.
2. If it is a pod creation request, then the first valid policy in alphabetical
order is used.
3. Otherwise, if it is a pod update request, an error is returned, because pod mutations
are disallowed during update operations.
1. PodSecurityPolicies which allow the pod as-is, without changing defaults or
mutating the pod, are preferred. The order of these non-mutating
PodSecurityPolicies doesn't matter.
2. If the pod must be defaulted or mutated, the first PodSecurityPolicy
(ordered by name) to allow the pod is selected.
{{< note >}}
During update operations (during which mutations to pod specs are disallowed)
only non-mutating PodSecurityPolicies are used to validate the pod.
{{< /note >}}
## Example
@@ -0,0 +1,5 @@
---
title: "Scheduling"
weight: 90
---
@@ -0,0 +1,186 @@
---
title: Kubernetes Scheduler
content_template: templates/concept
weight: 60
---
{{% capture overview %}}
In Kubernetes, _scheduling_ refers to making sure that {{< glossary_tooltip text="Pods" term_id="pod" >}}
are matched to {{< glossary_tooltip text="Nodes" term_id="node" >}} so that
{{< glossary_tooltip term_id="kubelet" >}} can run them.
{{% /capture %}}
{{% capture body %}}
## Scheduling overview {#scheduling}
A scheduler watches for newly created Pods that have no Node assigned. For
every Pod that the scheduler discovers, the scheduler becomes responsible
for finding the best Node for that Pod to run on. The scheduler reaches
this placement decision taking into account the scheduling principles
described below.
If you want to understand why Pods are placed onto a particular Node,
or if you're planning to implement a custom scheduler yourself, this
page will help you learn about scheduling.
## kube-scheduler
[kube-scheduler](https://kubernetes.io/docs/reference/command-line-tools-reference/kube-scheduler/)
is the default scheduler for Kubernetes and runs as part of the
{{< glossary_tooltip text="control plane" term_id="control-plane" >}}.
kube-scheduler is designed so that, if you want and need to, you can
write your own scheduling component and use that instead.
For every newly created pods or other unscheduled pods, kube-scheduler
selects a optimal node for them to run on. However, every container in
pods has different requirements for resources and every pod also has
different requirements. Therefore, existing nodes need to be filtered
according to the specific scheduling requirements.
In a cluster, Nodes that meet the scheduling requirements for a Pod
are called _feasible_ nodes. If none of the nodes are suitable, the pod
remains unscheduled until the scheduler is able to place it.
The scheduler finds feasible Nodes for a Pod and then runs a set of
functions to score the feasible Nodes and picks a Node with the highest
score among the feasible ones to run the Pod. The scheduler then notifies
the API server about this decision in a process called _binding_.
Factors that need taken into account for scheduling decisions include
individual and collective resource requirements, hardware / software /
policy constraints, affinity and anti-affinity specifications, data
locality, inter-workload interference, and so on.
## Scheduling with kube-scheduler {#kube-scheduler-implementation}
kube-scheduler selects a node for the pod in a 2-step operation:
1. Filtering
2. Scoring
The _filtering_ step finds the set of Nodes where it's feasible to
schedule the Pod. For example, the PodFitsResources filter checks whether a
candidate Node has enough available resource to meet a Pod's specific
resource requests. After this step, the node list contains any suitable
Nodes; often, there will be more than one. If the list is empty, that
Pod isn't (yet) schedulable.
In the _scoring_ step, the scheduler ranks the remaining nodes to choose
the most suitable Pod placement. The scheduler assigns a score to each Node
that survived filtering, basing this score on the active scoring rules.
Finally, kube-scheduler assigns the Pod to the Node with the highest ranking.
If there is more than one node with equal scores, kube-scheduler selects
one of these at random.
### Default policies
kube-scheduler has a default set of scheduling policies.
### Filtering
- `PodFitsHostPorts`: Checks if a Node has free ports (the network protocol kind)
for the Pod ports the the Pod is requesting.
- `PodFitsHost`: Checks if a Pod specifies a specific Node by it hostname.
- `PodFitsResources`: Checks if the Node has free resources (eg, CPU and Memory)
to meet the requirement of the Pod.
- `PodMatchNodeSelector`: Checks if a Pod's Node {{< glossary_tooltip term_id="selector" >}}
matches the Node's {{< glossary_tooltip text="label(s)" term_id="label" >}}.
- `NoVolumeZoneConflict`: Evaluate if the {{< glossary_tooltip text="Volumes" term_id="volume" >}}
that a Pod requests are available on the Node, given the failure zone restrictions for
that storage.
- `NoDiskConflict`: Evaluates if a Pod can fit on a Node due to the volumes it requests,
and those that are already mounted.
- `MaxCSIVolumeCount`: Decides how many {{< glossary_tooltip text="CSI" term_id="csi" >}}
volumes should be attached, and whether that's over a configured limit.
- `CheckNodeMemoryPressure`: If a Node is reporting memory pressure, and there's no
configured exception, the Pod won't be scheduled there.
- `CheckNodePIDPressure`: If a Node is reporting that process IDs are scarce, and
there's no configured exception, the Pod won't be scheduled there.
- `CheckNodeDiskPressure`: If a Node is reporting storage pressure (a filesystem that
is full or nearly full), and there's no configured exception, the Pod won't be
scheduled there.
- `CheckNodeCondition`: Nodes can report that they have a completely full filesystem,
that networking isn't available or that kubelet is otherwise not ready to run Pods.
If such a condition is set for a Node, and there's no configured exception, the Pod
won't be scheduled there.
- `PodToleratesNodeTaints`: checks if a Pod's {{< glossary_tooltip text="tolerations" term_id="toleration" >}}
can tolerate the Node's {{< glossary_tooltip text="taints" term_id="taint" >}}.
- `CheckVolumeBinding`: Evaluates if a Pod can fit due to the volumes it requests.
This applies for both bound and unbound
{{< glossary_tooltip text="PVCs" term_id="persistent-volume-claim" >}}
### Scoring
- `SelectorSpreadPriority`: Spreads Pods across hosts, considering Pods that
belonging to the same {{< glossary_tooltip text="Service" term_id="service" >}},
{{< glossary_tooltip term_id="statefulset" >}} or
{{< glossary_tooltip term_id="replica-set" >}}.
- `InterPodAffinityPriority`: Computes a sum by iterating through the elements
of weightedPodAffinityTerm and adding “weight” to the sum if the corresponding
PodAffinityTerm is satisfied for that node; the node(s) with the highest sum
are the most preferred.
- `LeastRequestedPriority`: Favors nodes with fewer requested resources. In other
words, the more Pods that are placed on a Node, and the more resources those
Pods use, the lower the ranking this policy will give.
- `MostRequestedPriority`: Favors nodes with most requested resources. This policy
will fit the scheduled Pods onto the smallest number of Nodes needed to run your
overall set of workloads.
- `RequestedToCapacityRatioPriority`: Creates a requestedToCapacity based ResourceAllocationPriority using default resource scoring function shape.
- `BalancedResourceAllocation`: Favors nodes with balanced resource usage.
- `NodePreferAvoidPodsPriority`: Priorities nodes according to the node annotation
`scheduler.alpha.kubernetes.io/preferAvoidPods`. You can use this to hint that
two different Pods shouldn't run on the same Node.
- `NodeAffinityPriority`: Prioritizes nodes according to node affinity scheduling
preferences indicated in PreferredDuringSchedulingIgnoredDuringExecution.
You can read more about this in [Assigning Pods to Nodes](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/)
- `TaintTolerationPriority`: Prepares the priority list for all the nodes, based on
the number of intolerable taints on the node. This policy adjusts a node's rank
taking that list into account.
- `ImageLocalityPriority`: Favors nodes that already have the
{{< glossary_tooltip text="container images" term_id="image" >}} for that
Pod cached locally.
- `ServiceSpreadingPriority`: For a given Service, this policy aims to make sure that
the Pods for the Service run on different nodes. It favouring scheduling onto nodes
that don't have Pods for the service already assigned there. The overall outcome is
that the Service becomes more resilient to a single Node failure.
- `CalculateAntiAffinityPriorityMap`: This policy helps implement
[pod anti-affinity](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity).
- `EqualPriorityMap`: Gives an equal weight of one to all nodes.
{{% /capture %}}
{{% capture whatsnext %}}
* Read about [scheduler performance tuning](/docs/concepts/scheduling/scheduler-perf-tuning/)
* Read the [reference documentation](/docs/reference/command-line-tools-reference/kube-scheduler/) for kube-scheduler
* Learn about [configuring multiple schedulers](https://kubernetes.io/docs/tasks/administer-cluster/configure-multiple-schedulers/)
{{% /capture %}}
@@ -10,13 +10,19 @@ weight: 70
{{< feature-state for_k8s_version="1.14" state="beta" >}}
Kube-scheduler is the Kubernetes default scheduler. It is responsible for
placement of Pods on Nodes in a cluster. Nodes in a cluster that meet the
scheduling requirements of a Pod are called "feasible" Nodes for the Pod. The
scheduler finds feasible Nodes for a Pod and then runs a set of functions to
score the feasible Nodes and picks a Node with the highest score among the
feasible ones to run the Pod. The scheduler then notifies the API server about
this decision in a process called "Binding".
[kube-scheduler](/docs/concepts/scheduling/kube-scheduler/#kube-scheduler)
is the Kubernetes default scheduler. It is responsible for placement of Pods
on Nodes in a cluster.
Nodes in a cluster that meet the scheduling requirements of a Pod are
called _feasible_ Nodes for the Pod. The scheduler finds feasible Nodes
for a Pod and then runs a set of functions to score the feasible Nodes,
picking a Node with the highest score among the feasible ones to run
the Pod. The scheduler then notifies the API server about this decision
in a process called _Binding_.
This page explains performance tuning optimizations that are relevant for
large Kubernetes clusters.
{{% /capture %}}
@@ -37,7 +43,7 @@ size of the cluster if it is not specified in the configuration. It uses a
linear formula which yields 50% for a 100-node cluster. The formula yields 10%
for a 5000-node cluster. The lower bound for the automatic value is 5%. In other
words, the scheduler always scores at least 5% of the cluster no matter how
large the cluster is, unless the user provides the config option with a value
large the cluster is, unless the user provides the config option with a value
smaller than 5.
Below is an example configuration that sets `percentageOfNodesToScore` to 50%.
@@ -13,7 +13,7 @@ Kubernetes Security (and security in general) is an immense topic that has many
highly interrelated parts. In today's era where open source software is
integrated into many of the systems that help web applications run,
there are some overarching concepts that can help guide your intuition about how you can
think about security holistically. This guide will define a mental model for
think about security holistically. This guide will define a mental model
for some general concepts surrounding Cloud Native Security. The mental model is completely arbitrary
and you should only use it if it helps you think about where to secure your software
stack.
@@ -32,11 +32,7 @@ Kubernetes as a project currently supports and maintains [GCE](https://git.k8s.i
* F5 Networks provides [support and maintenance](https://support.f5.com/csp/article/K86859508)
for the [F5 BIG-IP Controller for Kubernetes](http://clouddocs.f5.com/products/connectors/k8s-bigip-ctlr/latest).
* [Gloo](https://gloo.solo.io) is an open-source ingress controller based on [Envoy](https://www.envoyproxy.io) which offers API Gateway functionality with enterprise support from [solo.io](https://www.solo.io).
* [HAProxy](http://www.haproxy.org/) based ingress controller
[jcmoraisjr/haproxy-ingress](https://github.com/jcmoraisjr/haproxy-ingress) which is mentioned on the blog post
[HAProxy Ingress Controller for Kubernetes](https://www.haproxy.com/blog/haproxy_ingress_controller_for_kubernetes/).
[HAProxy Technologies](https://www.haproxy.com/) offers support and maintenance for HAProxy Enterprise and
the ingress controller [jcmoraisjr/haproxy-ingress](https://github.com/jcmoraisjr/haproxy-ingress).
* [HAProxy Technologies](https://www.haproxy.com/) offers support and maintenance for the [HAProxy Ingress Controller for Kubernetes](https://github.com/haproxytech/kubernetes-ingress). See the [official documentation](https://www.haproxy.com/documentation/hapee/1-9r1/traffic-management/kubernetes-ingress-controller/).
* [Istio](https://istio.io/) based ingress controller
[Control Ingress Traffic](https://istio.io/docs/tasks/traffic-management/ingress/).
* [Kong](https://konghq.com/) offers [community](https://discuss.konghq.com/c/kubernetes) or
@@ -44,6 +40,7 @@ Kubernetes as a project currently supports and maintains [GCE](https://git.k8s.i
[Kong Ingress Controller for Kubernetes](https://github.com/Kong/kubernetes-ingress-controller).
* [NGINX, Inc.](https://www.nginx.com/) offers support and maintenance for the
[NGINX Ingress Controller for Kubernetes](https://www.nginx.com/products/nginx/kubernetes-ingress-controller).
* [Skipper](https://opensource.zalando.com/skipper/kubernetes/ingress-controller/) HTTP router and reverse proxy for service composition, including use cases like Kubernetes Ingress, designed as a library to build your custom proxy
* [Traefik](https://github.com/containous/traefik) is a fully featured ingress controller
([Let's Encrypt](https://letsencrypt.org), secrets, http2, websocket), and it also comes with commercial
support by [Containous](https://containo.us/services).
@@ -46,7 +46,7 @@ Traffic routing is controlled by rules defined on the Ingress resource.
[ Services ]
```
An Ingress can be configured to give services externally-reachable URLs, load balance traffic, terminate SSL / TLS, and offer name based virtual hosting. An [Ingress controller](/docs/concepts/services-networking/ingress-controllers) is responsible for fulfilling the Ingress, usually with a load balancer, though it may also configure your edge router or additional frontends to help handle the traffic.
An Ingress can be configured to give Services externally-reachable URLs, load balance traffic, terminate SSL / TLS, and offer name based virtual hosting. An [Ingress controller](/docs/concepts/services-networking/ingress-controllers) is responsible for fulfilling the Ingress, usually with a load balancer, though it may also configure your edge router or additional frontends to help handle the traffic.
An Ingress does not expose arbitrary ports or protocols. Exposing services other than HTTP and HTTPS to the internet typically
uses a service of type [Service.Type=NodePort](/docs/concepts/services-networking/service/#nodeport) or
@@ -56,14 +56,14 @@ uses a service of type [Service.Type=NodePort](/docs/concepts/services-networkin
You must have an [ingress controller](/docs/concepts/services-networking/ingress-controllers) to satisfy an Ingress. Only creating an Ingress resource has no effect.
You may need to deploy an ingress controller such as [ingress-nginx](https://kubernetes.github.io/ingress-nginx/deploy/). There are a number of
[ingress controllers](/docs/concepts/services-networking/ingress-controllers) you may choose from.
You may need to deploy an Ingress controller such as [ingress-nginx](https://kubernetes.github.io/ingress-nginx/deploy/). You can choose from a number of
[Ingress controllers](/docs/concepts/services-networking/ingress-controllers).
Ideally, all ingress controllers should fit the reference specification. In reality, the various ingress
Ideally, all Ingress controllers should fit the reference specification. In reality, the various Ingress
controllers operate slightly differently.
{{< note >}}
Make sure you review your ingress controller's documentation to understand the caveats of choosing it.
Make sure you review your Ingress controller's documentation to understand the caveats of choosing it.
{{< /note >}}
## The Ingress Resource
@@ -101,19 +101,19 @@ for directing HTTP traffic.
### Ingress rules
Each http rule contains the following information:
Each HTTP rule contains the following information:
* An optional host. In this example, no host is specified, so the rule applies to all inbound
HTTP traffic through the IP address specified. If a host is provided (for example,
foo.bar.com), the rules apply to that host.
* a list of paths (for example, `/testpath`), each of which has an associated backend defined with a `serviceName`
* A list of paths (for example, `/testpath`), each of which has an associated backend defined with a `serviceName`
and `servicePort`. Both the host and path must match the content of an incoming request before the
load balancer will direct traffic to the referenced service.
* A backend is a combination of service and port names as described in the
[services doc](/docs/concepts/services-networking/service/). HTTP (and HTTPS) requests to the
Ingress matching the host and path of the rule will be sent to the listed backend.
load balancer directs traffic to the referenced Service.
* A backend is a combination of Service and port names as described in the
[Service doc](/docs/concepts/services-networking/service/). HTTP (and HTTPS) requests to the
Ingress that matches the host and path of the rule are sent to the listed backend.
A default backend is often configured in an Ingress controller that will service any requests that do not
A default backend is often configured in an Ingress controller to service any requests that do not
match a path in the spec.
### Default Backend
@@ -151,12 +151,12 @@ this Ingress.
{{< note >}}
Ingress controllers and load balancers may take a minute or two to allocate an IP address.
Until that time you will often see the address listed as `<pending>`.
Until that time, you often see the address listed as `<pending>`.
{{< /note >}}
### Simple fanout
A fanout configuration routes traffic from a single IP address to more than one service,
A fanout configuration routes traffic from a single IP address to more than one Service,
based on the HTTP URI being requested. An Ingress allows you to keep the number of load balancers
down to a minimum. For example, a setup like:
@@ -189,7 +189,7 @@ spec:
servicePort: 8080
```
When you create the ingress with `kubectl apply -f`:
When you create the Ingress with `kubectl apply -f`:
```shell
kubectl describe ingress simple-fanout-example
@@ -215,7 +215,7 @@ Events:
```
The Ingress controller provisions an implementation-specific load balancer
that satisfies the Ingress, as long as the services (`s1`, `s2`) exist.
that satisfies the Ingress, as long as the Services (`s1`, `s2`) exist.
When it has done so, you can see the address of the load balancer at the
Address field.
@@ -299,11 +299,11 @@ spec:
You can secure an Ingress by specifying a {{< glossary_tooltip term_id="secret" >}}
that contains a TLS private key and certificate. Currently the Ingress only
supports a single TLS port, 443, and assumes TLS termination. If the TLS
configuration section in an Ingress specifies different hosts, they will be
configuration section in an Ingress specifies different hosts, they are
multiplexed on the same port according to the hostname specified through the
SNI TLS extension (provided the Ingress controller supports SNI). The TLS secret
must contain keys named `tls.crt` and `tls.key` that contain the certificate
and private key to use for TLS, e.g.:
and private key to use for TLS. For example:
```yaml
apiVersion: v1
@@ -317,7 +317,7 @@ data:
type: kubernetes.io/tls
```
Referencing this secret in an Ingress will tell the Ingress controller to
Referencing this secret in an Ingress tells the Ingress controller to
secure the channel from the client to the load balancer using TLS. You need to make
sure the TLS secret you created came from a certificate that contains a CN
for `sslexample.foo.com`.
@@ -397,7 +397,7 @@ Events:
kubectl edit ingress test
```
This should pop up an editor with the existing configuration in yaml format.
This pops up an editor with the existing configuration in YAML format.
Modify it to include the new Host:
```yaml
@@ -420,7 +420,7 @@ spec:
..
```
After you save your changes, kubectl will update the resource in the API server, which should tell the
After you save your changes, kubectl updates the resource in the API server, which tells the
Ingress controller to reconfigure the load balancer.
Verify this:
@@ -449,7 +449,7 @@ Events:
Normal ADD 45s loadbalancer-controller default/test
```
You can achieve the same outcome by invoking `kubectl replace -f` on a modified Ingress yaml file.
You can achieve the same outcome by invoking `kubectl replace -f` on a modified Ingress YAML file.
## Failing across availability zones
@@ -5,7 +5,7 @@ title: Service
feature:
title: Service discovery and load balancing
description: >
No need to modify your application to use an unfamiliar service discovery mechanism. Kubernetes gives pods their own IP addresses and a single DNS name for a set of pods, and can load-balance across them.
No need to modify your application to use an unfamiliar service discovery mechanism. Kubernetes gives Pods their own IP addresses and a single DNS name for a set of Pods, and can load-balance across them.
content_template: templates/concept
weight: 10
@@ -16,8 +16,8 @@ weight: 10
{{< glossary_definition term_id="service" length="short" >}}
No need to modify your application to use an unfamiliar service discovery mechanism.
Kubernetes gives pods their own IP addresses and a single DNS name for a set of pods,
With Kubernetes you don't need to modify your application to use an unfamiliar service discovery mechanism.
Kubernetes gives Pods their own IP addresses and a single DNS name for a set of Pods,
and can load-balance across them.
{{% /capture %}}
@@ -26,18 +26,18 @@ and can load-balance across them.
## Motivation
Kubernetes {{< glossary_tooltip term_id="pod" text="Pods" >}} are mortal.
Kubernetes {{< glossary_tooltip term_id="pod" text="Pods" >}} are mortal.
They are born and when they die, they are not resurrected.
If you use a {{< glossary_tooltip term_id="deployment" >}} to run your app,
it can create and destroy Pods dynamically (e.g. when scaling out or in).
it can create and destroy Pods dynamically.
Each Pod gets its own IP address, however the set of Pods
for a Deployment running in one moment in time could be different from
Each Pod gets its own IP address, however in a Deployment, the set of Pods
running in one moment in time could be different from
the set of Pods running that application a moment later.
This leads to a problem: if some set of Pods (call them “backends”) provides
functionality to other Pods (call them “frontends”) inside your cluster,
how do those frontends find out and keep track of which IP address to connect
how do the frontends find out and keep track of which IP address to connect
to, so that the frontend can use the backend part of the workload?
Enter _Services_.
@@ -45,13 +45,13 @@ Enter _Services_.
## Service resources {#service-resource}
In Kubernetes, a Service is an abstraction which defines a logical set of Pods
and a policy by which to access them (you'll sometimes see this pattern called
and a policy by which to access them (sometimes this pattern is called
a micro-service). The set of Pods targeted by a Service is usually determined
by a {{< glossary_tooltip text="selector" term_id="selector" >}}
(see [below](#services-without-selectors) for why you might want a Service
_without_ a selector).
For example: consider a stateless image-processing backend which is running with
For example, consider a stateless image-processing backend which is running with
3 replicas. Those replicas are fungible&mdash;frontends do not care which backend
they use. While the actual Pods that compose the backend set may change, the
frontend clients should not need to be aware of that, nor should they need to keep
@@ -63,19 +63,19 @@ The Service abstraction enables this decoupling.
If you're able to use Kubernetes APIs for service discovery in your application,
you can query the {{< glossary_tooltip text="API server" term_id="kube-apiserver" >}}
for Endpoints, that will be updated whenever the set of Pods in a Service changes.
for Endpoints, that get updated whenever the set of Pods in a Service changes.
For non-native applications, Kubernetes offers ways to place a network port or load
balancer in between your application and the backend Pods.
## Defining a service
## Defining a Service
A Service in Kubernetes is a REST object, similar to a Pod. Like all of the
REST objects, you can `POST` a Service definition to the API server to create
a new instance.
For example, suppose you have a set of Pods that each listen on TCP port 9376
and carry a label `"app=MyApp"`:
and carry a label `app=MyApp`:
```yaml
apiVersion: v1
@@ -86,37 +86,37 @@ spec:
selector:
app: MyApp
ports:
- protocol: TCP
port: 80
targetPort: 9376
- protocol: TCP
port: 80
targetPort: 9376
```
This specification will create a new Service object named “my-service” which
targets TCP port 9376 on any Pod with the `"app=MyApp"` label.
This specification creates a new Service object named “my-service”, which
targets TCP port 9376 on any Pod with the `app=MyApp` label.
This Service will also be assigned an IP address (sometimes called the "cluster IP"),
which is used by the service proxies
Kubernetes assigns this Service an IP address (sometimes called the "cluster IP"),
which is used by the Service proxies
(see [Virtual IPs and service proxies](#virtual-ips-and-service-proxies) below).
The controller for the Service selector will continuously scan for Pods that
match its selector, and will then POST any updates to an Endpoint object
The controller for the Service selector continuously scans for Pods that
match its selector, and then POSTs any updates to an Endpoint object
also named “my-service”.
{{< note >}}
A Service can map _any_ incoming `port` to a `targetPort`. By default, and
for convenience, the `targetPort` will be set to the same value as the `port`
A Service can map _any_ incoming `port` to a `targetPort`. By default and
for convenience, the `targetPort` is set to the same value as the `port`
field.
{{< /note >}}
Port definitions in Pods have names, and you can reference these names in the
targetPort attribute of a Service. This will work even if there are a mixture
of Pods in the Service, with the same network protocol available via different
port numbers but a single configured name.
`targetPort` attribute of a Service. This works even if there is a mixture
of Pods in the Service using a single configured name, with the same network
protocol available via different port numbers.
This offers a lot of flexibility for deploying and evolving your Services.
For example, you can change the port number that pods expose in the next
For example, you can change the port numbers that Pods expose in the next
version of your backend software, without breaking clients.
The default protocol for services is TCP; you can also use any other
The default protocol for Services is TCP; you can also use any other
[supported protocol](#protocol-support).
As many Services need to expose more than one port, Kubernetes supports multiple
@@ -126,16 +126,17 @@ Each port definition can have the same `protocol`, or a different one.
### Services without selectors
Services most commonly abstract access to Kubernetes Pods, but they can also
abstract other kinds of backends. For example:
abstract other kinds of backends.
For example:
* You want to have an external database cluster in production, but in your
test environment you use your own databases.
* You want to point your service to a service in a different
* You want to point your Service to a Service in a different
{{< glossary_tooltip term_id="namespace" >}} or on another cluster.
* You are migrating a workload to Kubernetes. Whilst evaluating the approach,
you run only a proportion of your backends in Kubernetes.
In any of these scenarios you can define a service _without_ a Pod selector.
In any of these scenarios you can define a Service _without_ a Pod selector.
For example:
```yaml
@@ -145,13 +146,13 @@ metadata:
name: my-service
spec:
ports:
- protocol: TCP
port: 80
targetPort: 9376
- protocol: TCP
port: 80
targetPort: 9376
```
Because this service has no selector, the corresponding Endpoint object will *not* be
created automatically. You can manually map the service to the network address and port
Because this Service has no selector, the corresponding Endpoint object is *not*
created automatically. You can manually map the Service to the network address and port
where it's running, by adding an Endpoint object manually:
```yaml
@@ -170,16 +171,16 @@ subsets:
The endpoint IPs _must not_ be: loopback (127.0.0.0/8 for IPv4, ::1/128 for IPv6), or
link-local (169.254.0.0/16 and 224.0.0.0/24 for IPv4, fe80::/64 for IPv6).
Endpoint IP addresses also cannot be the cluster IPs of other Kubernetes services,
Endpoint IP addresses cannot be the cluster IPs of other Kubernetes Services,
because {{< glossary_tooltip term_id="kube-proxy" >}} doesn't support virtual IPs
as a destination.
{{< /note >}}
Accessing a Service without a selector works the same as if it had a selector.
In the example above, traffic will be routed to the single endpoint defined in
In the example above, traffic is routed to the single endpoint defined in
the YAML: `192.0.2.42:9376` (TCP).
An ExternalName Service is a special case of service that does not have
An ExternalName Service is a special case of Service that does not have
selectors and uses DNS names instead. For more information, see the
[ExternalName](#externalname) section later in this document.
@@ -219,7 +220,7 @@ Kubernetes v1.8 added ipvs proxy mode.
In this mode, kube-proxy watches the Kubernetes master for the addition and
removal of Service and Endpoint objects. For each Service it opens a
port (randomly chosen) on the local node. Any connections to this "proxy port"
will be proxied to one of the Service's backend Pods (as reported via
is proxied to one of the Service's backend Pods (as reported via
Endpoints). kube-proxy takes the `SessionAffinity` setting of the Service into
account when deciding which backend Pod to use.
@@ -235,19 +236,19 @@ By default, kube-proxy in userspace mode chooses a backend via a round-robin alg
In this mode, kube-proxy watches the Kubernetes control plane for the addition and
removal of Service and Endpoint objects. For each Service, it installs
iptables rules which capture traffic to the Service's `clusterIP` (which is
virtual) and `port` and redirects that traffic to one of the Service's
iptables rules, which capture traffic to the Service's `clusterIP` and `port`,
and redirect that traffic to one of the Service's
backend sets. For each Endpoint object, it installs iptables rules which
select a backend Pod.
By default, kube-proxy in iptables mode chooses a backend at random.
Using iptables to handle traffic has a lower system overhead, because traffic
is handled by Linux netfilter without the need switch between userspace and the
is handled by Linux netfilter without the need to switch between userspace and the
kernel space. This approach is also likely to be more reliable.
If kube-proxy is running in iptables mode and the first Pod that's selected
does not respond, the connection will fail. This is different from userspace
does not respond, the connection fails. This is different from userspace
mode: in that scenario, kube-proxy would detect that the connection to the first
Pod had failed and would automatically retry with a different backend Pod.
@@ -267,7 +268,7 @@ calls `netlink` interface to create IPVS rules accordingly and synchronizes
IPVS rules with Kubernetes Services and Endpoints periodically.
This control loop ensures that IPVS status matches the desired
state.
When accessing a Service, IPVS will direct traffic to one of the backend Pods.
When accessing a Service, IPVS directs traffic to one of the backend Pods.
The IPVS proxy mode is based on netfilter hook function that is similar to
iptables mode, but uses hash table as the underlying data structure and works
@@ -291,22 +292,22 @@ these are:
To run kube-proxy in IPVS mode, you must make the IPVS Linux available on
the node before you starting kube-proxy.
When kube-proxy starts in IPVS proxy mode, it will verify whether IPVS
kernel modules are available, and if those are not detected then kube-proxy
fall back to running in iptables proxy mode.
When kube-proxy starts in IPVS proxy mode, it verifies whether IPVS
kernel modules are available. If the IPVS kernel modules are not detected, then kube-proxy
falls back to running in iptables proxy mode.
{{< /note >}}
![Services overview diagram for IPVS proxy](/images/docs/services-ipvs-overview.svg)
In any of these proxy models, any traffic bound for the Services IP:Port is
In these proxy models, the traffic bound for the Services IP:Port is
proxied to an appropriate backend without the clients knowing anything
about Kubernetes or Services or Pods.
If you want to make sure that connections from a particular client
are passed to the same Pod each time, you can select session affinity based
are passed to the same Pod each time, you can select the session affinity based
the on client's IP addresses by setting `service.spec.sessionAffinity` to "ClientIP"
(the default is "None").
You can then also set the maximum session sticky time by setting
You can also set the maximum session sticky time by setting
`service.spec.sessionAffinityConfig.clientIP.timeoutSeconds` appropriately.
(the default value is 10800, which works out to be 3 hours).
@@ -315,7 +316,8 @@ You can then also set the maximum session sticky time by setting
For some Services, you need to expose more than one port.
Kubernetes lets you configure multiple port definitions on a Service object.
When using multiple ports for a Service, you must give all of your ports names
so that these are unambiguous. For example:
so that these are unambiguous.
For example:
```yaml
apiVersion: v1
@@ -326,14 +328,14 @@ spec:
selector:
app: MyApp
ports:
- name: http
protocol: TCP
port: 80
targetPort: 9376
- name: https
protocol: TCP
port: 443
targetPort: 9377
- name: http
protocol: TCP
port: 80
targetPort: 9376
- name: https
protocol: TCP
port: 443
targetPort: 9377
```
{{< note >}}
@@ -371,7 +373,7 @@ and simpler `{SVCNAME}_SERVICE_HOST` and `{SVCNAME}_SERVICE_PORT` variables,
where the Service name is upper-cased and dashes are converted to underscores.
For example, the Service `"redis-master"` which exposes TCP port 6379 and has been
allocated cluster IP address 10.0.0.11 produces the following environment
allocated cluster IP address 10.0.0.11, produces the following environment
variables:
```shell
@@ -385,7 +387,7 @@ REDIS_MASTER_PORT_6379_TCP_ADDR=10.0.0.11
```
{{< note >}}
When you have a Pod that might need to acccess a Service, and you are using
When you have a Pod that needs to access a Service, and you are using
the environment variable method to publish the port and cluster IP to the client
Pods, you must create the Service *before* the client Pods come into existence.
Otherwise, those client Pods won't have their environment variables populated.
@@ -405,7 +407,7 @@ throughout your cluster then all Pods should automatically be able to resolve
Services by their DNS name.
For example, if you have a Service called `"my-service"` in a Kubernetes
Namespace `"my-ns"`, the control plane and the DNS service acting together will
Namespace `"my-ns"`, the control plane and the DNS Service acting together
create a DNS record for `"my-service.my-ns"`. Pods in the `"my-ns"` Namespace
should be able to find it by simply doing a name lookup for `my-service`
(`"my-service.my-ns"` would also work).
@@ -413,7 +415,7 @@ should be able to find it by simply doing a name lookup for `my-service`
Pods in other Namespaces must qualify the name as `my-service.my-ns`. These names
will resolve to the cluster IP assigned for the Service.
Kubernetes also supports DNS SRV (service) records for named ports. If the
Kubernetes also supports DNS SRV (Service) records for named ports. If the
`"my-service.my-ns"` Service has a port named `"http"` with protocol set to
`TCP`, you can do a DNS SRV query for `_http._tcp.my-service.my-ns` to discover
the port number for `"http"`, as well as the IP address.
@@ -422,86 +424,84 @@ The Kubernetes DNS server is the only way to access `ExternalName` Services.
You can find more information about `ExternalName` resolution in
[DNS Pods and Services](/docs/concepts/services-networking/dns-pod-service/).
## Headless services
## Headless Services
Sometimes you don't need or want load-balancing and a single service IP. In
Sometimes you don't need load-balancing and a single Service IP. In
this case, you can create what are termed “headless” Services, by explicitly
specifying `"None"` for the cluster IP (`.spec.clusterIP`).
You can use a headless Service to interface with other service discovery mechanisms,
without being tied to Kubernetes' implementation. For example, you could implement
a custom [Operator](
be built upon this API.
without being tied to Kubernetes' implementation.
For such `Services`, a cluster IP is not allocated, kube-proxy does not handle
these services, and there is no load balancing or proxying done by the platform
for them. How DNS is automatically configured depends on whether the service has
selectors defined.
For headless `Services`, a cluster IP is not allocated, kube-proxy does not handle
these Services, and there is no load balancing or proxying done by the platform
for them. How DNS is automatically configured depends on whether the Service has
selectors defined:
### With selectors
For headless services that define selectors, the endpoints controller creates
`Endpoints` records in the API, and modifies the DNS configuration to return A
For headless Services that define selectors, the endpoints controller creates
`Endpoints` records in the API, and modifies the DNS configuration to return
records (addresses) that point directly to the `Pods` backing the `Service`.
### Without selectors
For headless services that do not define selectors, the endpoints controller does
For headless Services that do not define selectors, the endpoints controller does
not create `Endpoints` records. However, the DNS system looks for and configures
either:
* CNAME records for [`ExternalName`](#externalname)-type services.
* A records for any `Endpoints` that share a name with the service, for all
* CNAME records for [`ExternalName`](#externalname)-type Services.
* A records for any `Endpoints` that share a name with the Service, for all
other types.
## Publishing services (ServiceTypes) {#publishing-services-service-types}
## Publishing Services (ServiceTypes) {#publishing-services-service-types}
For some parts of your application (e.g. frontends) you may want to expose a
Service onto an external IP address, one that's outside of your cluster.
For some parts of your application (for example, frontends) you may want to expose a
Service onto an external IP address, that's outside of your cluster.
Kubernetes `ServiceTypes` allow you to specify what kind of service you want.
Kubernetes `ServiceTypes` allow you to specify what kind of Service you want.
The default is `ClusterIP`.
`Type` values and their behaviors are:
* `ClusterIP`: Exposes the service on a cluster-internal IP. Choosing this value
makes the service only reachable from within the cluster. This is the
* `ClusterIP`: Exposes the Service on a cluster-internal IP. Choosing this value
makes the Service only reachable from within the cluster. This is the
default `ServiceType`.
* [`NodePort`](#nodeport): Exposes the service on each Node's IP at a static port
(the `NodePort`). A `ClusterIP` service, to which the `NodePort` service will
route, is automatically created. You'll be able to contact the `NodePort` service,
* [`NodePort`](#nodeport): Exposes the Service on each Node's IP at a static port
(the `NodePort`). A `ClusterIP` Service, to which the `NodePort` Service
routes, is automatically created. You'll be able to contact the `NodePort` Service,
from outside the cluster,
by requesting `<NodeIP>:<NodePort>`.
* [`LoadBalancer`](#loadbalancer): Exposes the service externally using a cloud
provider's load balancer. `NodePort` and `ClusterIP` services, to which the external
load balancer will route, are automatically created.
* [`ExternalName`](#externalname): Maps the service to the contents of the
* [`LoadBalancer`](#loadbalancer): Exposes the Service externally using a cloud
provider's load balancer. `NodePort` and `ClusterIP` Services, to which the external
load balancer routes, are automatically created.
* [`ExternalName`](#externalname): Maps the Service to the contents of the
`externalName` field (e.g. `foo.bar.example.com`), by returning a `CNAME` record
with its value. No proxying of any kind is set up.
{{< note >}}
You need CoreDNS version 1.7 or higher to use the `ExternalName` type.
{{< /note >}}
{{< note >}}
You need CoreDNS version 1.7 or higher to use the `ExternalName` type.
{{< /note >}}
You can also use [Ingress](/docs/concepts/services-networking/ingress/) to expose your Service. Ingress is not a Service type, but it acts as the entry point for your cluster. It lets you consolidate your routing rules into a single resource as it can expose multiple services under the same IP address.
### Type NodePort {#nodeport}
If you set the `type` field to `NodePort`, the Kubernetes control plane will
allocate a port from a range specified by `--service-node-port-range` flag (default: 30000-32767).
Each node will proxy that port each (the same port number on every Node) into your Service.
Your service will report that allocated port in its `.spec.ports[*].nodePort` field.
If you set the `type` field to `NodePort`, the Kubernetes control plane
allocates a port from a range specified by `--service-node-port-range` flag (default: 30000-32767).
Each node proxies that port (the same port number on every Node) into your Service.
Your Service reports the allocated port in its `.spec.ports[*].nodePort` field.
If you want to specify particular IP(s) to proxy the port, you can set the `--nodeport-addresses` flag in kube-proxy to particular IP block(s); this is supported since Kubernetes v1.10.
This flag takes a comma-delimited list of IP blocks (e.g. 10.0.0.0/8, 192.0.2.0/25) to specify IP address ranges that kube-proxy should consider as local to this node.
For example, if you start kube-proxy with flag `--nodeport-addresses=127.0.0.0/8`, kube-proxy will select only the loopback interface for NodePort Services. The default for `--nodeport-addresses` is an empty list, and means that kube-proxy should consider all available network interfaces for NodePort. (That's also compatible with earlier Kubernetes releases).
For example, if you start kube-proxy with the `--nodeport-addresses=127.0.0.0/8` flag, kube-proxy only selects the loopback interface for NodePort Services. The default for `--nodeport-addresses` is an empty list. This means that kube-proxy should consider all available network interfaces for NodePort. (That's also compatible with earlier Kubernetes releases).
If you want a specific port number, you can specify a value in the `nodePort`
field. The control plane will either allocate you that port or report that
the API transaction failed.
This means that you need to take care about possible port collisions yourself).
This means that you need to take care about possible port collisions yourself.
You also have to use a valid port number, one that's inside the range configured
for NodePort use.
@@ -509,16 +509,17 @@ Using a NodePort gives you the freedom to set up your own load balancing solutio
to configure environments that are not fully supported by Kubernetes, or even
to just expose one or more nodes' IPs directly.
Note that this Service will be visible as both `<NodeIP>:spec.ports[*].nodePort`
Note that this Service is visible as `<NodeIP>:spec.ports[*].nodePort`
and `.spec.clusterIP:spec.ports[*].port`. (If the `--nodeport-addresses` flag in kube-proxy is set, <NodeIP> would be filtered NodeIP(s).)
### Type LoadBalancer {#loadbalancer}
On cloud providers which support external load balancers, setting the `type`
field to `LoadBalancer` will provision a load balancer for your Service.
field to `LoadBalancer` provisions a load balancer for your Service.
The actual creation of the load balancer happens asynchronously, and
information about the provisioned balancer will be published in the Service's
`.status.loadBalancer` field. For example:
information about the provisioned balancer is published in the Service's
`.status.loadBalancer` field.
For example:
```yaml
apiVersion: v1
@@ -529,26 +530,26 @@ spec:
selector:
app: MyApp
ports:
- protocol: TCP
port: 80
targetPort: 9376
- protocol: TCP
port: 80
targetPort: 9376
clusterIP: 10.0.171.239
loadBalancerIP: 78.11.24.19
type: LoadBalancer
status:
loadBalancer:
ingress:
- ip: 146.148.47.155
- ip: 146.148.47.155
```
Traffic from the external load balancer will be directed at the backend Pods,
though exactly how that works depends on the cloud provider.
Traffic from the external load balancer is directed at the backend Pods. The cloud provider decides how it is load balanced.
Some cloud providers allow you to specify the `loadBalancerIP`. In those cases, the load-balancer will be created
Some cloud providers allow you to specify the `loadBalancerIP`. In those cases, the load-balancer is created
with the user-specified `loadBalancerIP`. If the `loadBalancerIP` field is not specified,
the loadBalancer will be set up with an ephemeral IP address. If you specify a `loadBalancerIP`
the loadBalancer is set up with an ephemeral IP address. If you specify a `loadBalancerIP`
but your cloud provider does not support the feature, the `loadbalancerIP` field that you
set will be ignored.
set is ignored.
{{< note >}}
If you're using SCTP, see the [caveat](#caveat-sctp-loadbalancer-service-type) below about the
@@ -567,13 +568,13 @@ Specify the assigned IP address as loadBalancerIP. Ensure that you have updated
{{< /note >}}
#### Internal load balancer
In a mixed environment it is sometimes necessary to route traffic from services inside the same
In a mixed environment it is sometimes necessary to route traffic from Services inside the same
(virtual) network address block.
In a split-horizon DNS environment you would need two services to be able to route both external and internal traffic to your endpoints.
In a split-horizon DNS environment you would need two Services to be able to route both external and internal traffic to your endpoints.
You can achieve this by adding one the following annotations to a Service.
The annotation to add depends on the cloud service provider you're using.
The annotation to add depends on the cloud Service provider you're using.
{{< tabs name="service_tabs" >}}
{{% tab name="Default" %}}
@@ -658,15 +659,15 @@ metadata:
```
The second annotation specifies which protocol a Pod speaks. For HTTPS and
SSL, the ELB will expect the Pod to authenticate itself over the encrypted
SSL, the ELB expects the Pod to authenticate itself over the encrypted
connection, using a certificate.
HTTP and HTTPS will select layer 7 proxying: the ELB will terminate
HTTP and HTTPS selects layer 7 proxying: the ELB terminates
the connection with the user, parse headers and inject the `X-Forwarded-For`
header with the user's IP address (Pods will only see the IP address of the
header with the user's IP address (Pods only see the IP address of the
ELB at the other end of its connection) when forwarding requests.
TCP and SSL will select layer 4 proxying: the ELB will forward traffic without
TCP and SSL selects layer 4 proxying: the ELB forwards traffic without
modifying the headers.
In a mixed-use environment where some ports are secured and others are left unencrypted,
@@ -680,12 +681,12 @@ you can use the following annotations:
service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443,8443"
```
In the above example, if the service contained three ports, `80`, `443`, and
In the above example, if the Service contained three ports, `80`, `443`, and
`8443`, then `443` and `8443` would use the SSL certificate, but `80` would just
be proxied HTTP.
From Kubernetes v1.9 onwrds you can use [predefined AWS SSL policies](http://docs.aws.amazon.com/elasticloadbalancing/latest/classic/elb-security-policy-table.html) with HTTPS or SSL listeners for your Services.
To see which policies are available for use, you can the `aws` command line tool:
To see which policies are available for use, you can use the `aws` command line tool:
```bash
aws elb describe-load-balancer-policies --query 'PolicyDescriptions[].PolicyName'
@@ -720,7 +721,7 @@ and cannot be configured otherwise.
#### ELB Access Logs on AWS
There are several annotations to manage access logs for ELB services on AWS.
There are several annotations to manage access logs for ELB Services on AWS.
The annotation `service.beta.kubernetes.io/aws-load-balancer-access-log-enabled`
controls whether access logs are enabled.
@@ -805,11 +806,15 @@ There are other annotations to manage Classic Elastic Load Balancers that are de
# A list of additional security groups to be added to the ELB
```
#### Network Load Balancer support on AWS
#### Network Load Balancer support on AWS [alpha] {#aws-nlb-support}
{{< feature-state for_k8s_version="v1.15" state="beta" >}}
{{< warning >}}
This is an alpha feature and is not yet recommended for production clusters.
{{< /warning >}}
To use a Network Load Balancer on AWS, use the annotation `service.beta.kubernetes.io/aws-load-balancer-type` with the value set to `nlb`.
Starting from Kubernetes v1.9.0, you can use AWS Network Load Balancer (NLB) with Services. To
use a Network Load Balancer on AWS, use the annotation `service.beta.kubernetes.io/aws-load-balancer-type`
with the value set to `nlb`.
```yaml
metadata:
@@ -824,13 +829,13 @@ on Elastic Load Balancing for a list of supported instance types.
{{< /note >}}
Unlike Classic Elastic Load Balancers, Network Load Balancers (NLBs) forward the
client's IP address through to the node. If a service's `.spec.externalTrafficPolicy`
is set to `Cluster`, the client's IP address will not be propagated to the end
pods.
client's IP address through to the node. If a Service's `.spec.externalTrafficPolicy`
is set to `Cluster`, the client's IP address is not propagated to the end
Pods.
By setting `.spec.externalTrafficPolicy` to `Local`, client IP addresses will be
propagated to the end pods, but this could result in uneven distribution of
traffic. Nodes without any pods for a particular LoadBalancer service will fail
By setting `.spec.externalTrafficPolicy` to `Local`, the client IP addresses is
propagated to the end Pods, but this could result in uneven distribution of
traffic. Nodes without any Pods for a particular LoadBalancer Service will fail
the NLB Target Group's health check on the auto-assigned
`.spec.healthCheckNodePort` and not receive any traffic.
@@ -856,12 +861,12 @@ specify `loadBalancerSourceRanges`.
```yaml
spec:
loadBalancerSourceRanges:
- "143.231.0.0/16"
- "143.231.0.0/16"
```
{{< note >}}
If `.spec.loadBalancerSourceRanges` is not set, Kubernetes will
allow traffic from `0.0.0.0/0` to the Node Security Group(s). If nodes have
If `.spec.loadBalancerSourceRanges` is not set, Kubernetes
allows traffic from `0.0.0.0/0` to the Node Security Group(s). If nodes have
public IP addresses, be aware that non-NLB traffic can also reach all instances
in those modified security groups.
@@ -869,8 +874,8 @@ in those modified security groups.
### Type ExternalName {#externalname}
Services of type ExternalName map a service to a DNS name, not to a typical selector such as
`my-service` or `cassandra`. You specify these services with the `spec.externalName` parameter.
Services of type ExternalName map a Service to a DNS name, not to a typical selector such as
`my-service` or `cassandra`. You specify these Services with the `spec.externalName` parameter.
This Service definition, for example, maps
the `my-service` Service in the `prod` namespace to `my.database.example.com`:
@@ -888,15 +893,15 @@ spec:
{{< note >}}
ExternalName accepts an IPv4 address string, but as a DNS names comprised of digits, not as an IP address. ExternalNames that resemble IPv4 addresses are not resolved by CoreDNS or ingress-nginx because ExternalName
is intended to specify a canonical DNS name. To hardcode an IP address, consider using
[headless services](#headless-services).
[headless Services](#headless-services).
{{< /note >}}
When looking up the host `my-service.prod.svc.cluster.local`, the cluster DNS service
will return a `CNAME` record with the value `my.database.example.com`. Accessing
When looking up the host `my-service.prod.svc.cluster.local`, the cluster DNS Service
returns a `CNAME` record with the value `my.database.example.com`. Accessing
`my-service` works in the same way as other Services but with the crucial
difference that redirection happens at the DNS level rather than via proxying or
forwarding. Should you later decide to move your database into your cluster, you
can start its pods, add appropriate selectors or endpoints, and change the
can start its Pods, add appropriate selectors or endpoints, and change the
Service's `type`.
@@ -907,9 +912,9 @@ This section is indebted to the [Kubernetes Tips - Part
### External IPs
If there are external IPs that route to one or more cluster nodes, Kubernetes services can be exposed on those
`externalIPs`. Traffic that ingresses into the cluster with the external IP (as destination IP), on the service port,
will be routed to one of the service endpoints. `externalIPs` are not managed by Kubernetes and are the responsibility
If there are external IPs that route to one or more cluster nodes, Kubernetes Services can be exposed on those
`externalIPs`. Traffic that ingresses into the cluster with the external IP (as destination IP), on the Service port,
will be routed to one of the Service endpoints. `externalIPs` are not managed by Kubernetes and are the responsibility
of the cluster administrator.
In the Service spec, `externalIPs` can be specified along with any of the `ServiceTypes`.
@@ -924,17 +929,17 @@ spec:
selector:
app: MyApp
ports:
- name: http
protocol: TCP
port: 80
targetPort: 9376
- name: http
protocol: TCP
port: 80
targetPort: 9376
externalIPs:
- 80.11.12.10
- 80.11.12.10
```
## Shortcomings
Using the userspace proxy for VIPs will work at small to medium scale, but will
Using the userspace proxy for VIPs, work at small to medium scale, but will
not scale to very large clusters with thousands of Services. The [original
design proposal for portals](http://issue.k8s.io/1107) has more details on
this.
@@ -969,7 +974,7 @@ In order to allow you to choose a port number for your Services, we must
ensure that no two Services can collide. Kubernetes does that by allocating each
Service its own IP address.
To ensure each service receives a unique IP, an internal allocator atomically
To ensure each Service receives a unique IP, an internal allocator atomically
updates a global allocation map in {{< glossary_tooltip term_id="etcd" >}}
prior to creating each Service. The map object must exist in the registry for
Services to get IP address assignments, otherwise creations will
@@ -1036,7 +1041,7 @@ through a load-balancer, though in those cases the client IP does get altered.
#### IPVS
iptables operations slow down dramatically in large scale cluster e.g 10,000 Services.
IPVS is designed for load balancing and based on in-kernel hash tables. So you can achieve performance consistency in large number of services from IPVS-based kube-proxy. Meanwhile, IPVS-based kube-proxy has more sophisticated load balancing algorithms (least conns, locality, weighted, persistence).
IPVS is designed for load balancing and based on in-kernel hash tables. So you can achieve performance consistency in large number of Services from IPVS-based kube-proxy. Meanwhile, IPVS-based kube-proxy has more sophisticated load balancing algorithms (least conns, locality, weighted, persistence).
## API Object
@@ -1049,13 +1054,13 @@ about the API object at: [Service API object](/docs/reference/generated/kubernet
{{< feature-state for_k8s_version="v1.0" state="stable" >}}
You can use TCP for any kind of service, and it's the default network protocol.
You can use TCP for any kind of Service, and it's the default network protocol.
### UDP
{{< feature-state for_k8s_version="v1.0" state="stable" >}}
You can use UDP for most services. For type=LoadBalancer services, UDP support
You can use UDP for most Services. For type=LoadBalancer Services, UDP support
depends on the cloud provider offering this facility.
### HTTP
@@ -1068,7 +1073,7 @@ of the Service.
{{< note >}}
You can also use {{< glossary_tooltip term_id="ingress" >}} in place of Service
to expose HTTP / HTTPS services.
to expose HTTP / HTTPS Services.
{{< /note >}}
### PROXY protocol
@@ -233,11 +233,14 @@ or is done when Pod is running and underlying file system supports online expans
FlexVolumes allow resize if the driver is set with the `RequiresFSResize` capability to true.
The FlexVolume can be resized on pod restart.
{{< feature-state for_k8s_version="v1.11" state="alpha" >}}
#### Resizing an in-use PersistentVolumeClaim
Expanding in-use PVCs is a beta feature and is enabled by default via `ExpandInUsePersistentVolumes` feature gate.
{{< feature-state for_k8s_version="v1.15" state="beta" >}}
{{< note >}}
Expanding in-use PVCs is available as beta since 1.15, and as alpha since Kubernetes 1.11. The `ExpandInUsePersistentVolumes` feature must be enabled, which is the case automatically for many clusters for beta features. Please refer to the [feature gate](/docs/reference/command-line-tools-reference/feature-gates/) documentation for more information.
{{< /note >}}
In this case, you don't need to delete and recreate a Pod or deployment that is using an existing PVC.
Any in-use PVC automatically becomes available to its Pod as soon as its file system has been expanded.
This feature has no effect on PVCs that are not in use by a Pod or deployment. You must create a Pod which
@@ -346,7 +349,7 @@ In the CLI, the access modes are abbreviated to:
| AzureDisk | &#x2713; | - | - |
| CephFS | &#x2713; | &#x2713; | &#x2713; |
| Cinder | &#x2713; | - | - |
| CSI | &depends on the driver | depends on the driver | depends on the driver |
| CSI | depends on the driver | depends on the driver | depends on the driver |
| FC | &#x2713; | &#x2713; | - |
| Flexvolume | &#x2713; | &#x2713; | depends on the driver |
| Flocker | &#x2713; | - | - |
@@ -52,8 +52,11 @@ metadata:
name: clone-of-pvc-1
namespace: myns
spec:
capacity:
storage: 10Gi
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi
dataSource:
kind: PersistentVolumeClaim
name: pvc-1
+43 -6
View File
@@ -208,7 +208,7 @@ writers simultaneously.
You must have your own Ceph server running with the share exported before you can use it.
{{< /caution >}}
See the [CephFS example](https://github.com/kubernetes/examples/tree/{{< param "githubbranch" >}}/staging/volumes/cephfs/) for more details.
See the [CephFS example](https://github.com/kubernetes/examples/tree/{{< param "githubbranch" >}}/volumes/cephfs/) for more details.
### cinder {#cinder}
@@ -534,7 +534,7 @@ simultaneously.
You must have your own GlusterFS installation running before you can use it.
{{< /caution >}}
See the [GlusterFS example](https://github.com/kubernetes/examples/tree/{{< param "githubbranch" >}}/staging/volumes/glusterfs) for more details.
See the [GlusterFS example](https://github.com/kubernetes/examples/tree/{{< param "githubbranch" >}}/volumes/glusterfs) for more details.
### hostPath {#hostpath}
@@ -618,7 +618,7 @@ and then serve it in parallel from as many Pods as you need. Unfortunately,
iSCSI volumes can only be mounted by a single consumer in read-write mode - no
simultaneous writers allowed.
See the [iSCSI example](https://github.com/kubernetes/examples/tree/{{< param "githubbranch" >}}/staging/volumes/iscsi) for more details.
See the [iSCSI example](https://github.com/kubernetes/examples/tree/{{< param "githubbranch" >}}/volumes/iscsi) for more details.
### local {#local}
@@ -938,7 +938,7 @@ and then serve it in parallel from as many Pods as you need. Unfortunately,
RBD volumes can only be mounted by a single consumer in read-write mode - no
simultaneous writers allowed.
See the [RBD example](https://github.com/kubernetes/examples/tree/{{< param "githubbranch" >}}/staging/volumes/rbd) for more details.
See the [RBD example](https://github.com/kubernetes/examples/tree/{{< param "githubbranch" >}}/volumes/rbd) for more details.
### scaleIO {#scaleio}
@@ -1052,7 +1052,7 @@ spec:
```
For more information including Dynamic Provisioning and Persistent Volume Claims, please see the
[StorageOS examples](https://github.com/kubernetes/examples/blob/master/staging/volumes/storageos).
[StorageOS examples](https://github.com/kubernetes/examples/blob/master/volumes/storageos).
### vsphereVolume {#vspherevolume}
@@ -1313,6 +1313,43 @@ feature gates which must be enabled for this feature are `BlockVolume` and
Learn how to
[setup your PV/PVC with raw block volume support](/docs/concepts/storage/persistent-volumes/#raw-block-volume-support).
#### CSI ephemeral volumes
{{< feature-state for_k8s_version="v1.15" state="alpha" >}}
This feature allows CSI volumes to be directly embedded in the Pod specification instead of a PersistentVolume. Volumes specified in this way are ephemeral and do not persist across Pod restarts.
Example:
```yaml
kind: Pod
apiVersion: v1
metadata:
name: my-csi-app
spec:
containers:
- name: my-frontend
image: busybox
volumeMounts:
- mountPath: "/data"
name: my-csi-inline-vol
command: [ "sleep", "1000000" ]
volumes:
- name: my-csi-inline-vol
csi:
driver: inline.storage.kubernetes.io
volumeAttributes:
foo: bar
```
This feature requires CSIInlineVolume feature gate to be enabled:
```
--feature-gates=CSIInlineVolume=true
```
CSI ephemeral volumes are only supported by a subset of CSI drivers. Please see the list of CSI drivers [here](https://kubernetes-csi.github.io/docs/drivers.html).
# Developer resources
For more information on how to develop a CSI driver, refer to the [kubernetes-csi
documentation](https://kubernetes-csi.github.io/docs/)
@@ -1342,7 +1379,7 @@ drivers. Flexvolume driver binaries must be installed in a pre-defined volume
plugin path on each node (and in some cases master).
Pods interact with Flexvolume drivers through the `flexvolume` in-tree plugin.
More details can be found [here](https://github.com/kubernetes/community/blob/master/contributors/devel/flexvolume.md).
More details can be found [here](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-storage/flexvolume.md).
## Mount propagation
File diff suppressed because it is too large Load Diff
@@ -93,7 +93,7 @@ collector deletes the object's dependents. Once the garbage collector has delete
the owner object.
Note that in the "foregroundDeletion", only dependents with
`ownerReference.blockOwnerDeletion` block the deletion of the owner object.
`ownerReference.blockOwnerDeletion=true` block the deletion of the owner object.
Kubernetes version 1.7 added an [admission controller](/docs/reference/access-authn-authz/admission-controllers/#ownerreferencespermissionenforcement) that controls user access to set
`blockOwnerDeletion` to true based on delete permissions on the owner object, so that
unauthorized dependents cannot delay deletion of an owner object.
@@ -109,7 +109,7 @@ nginx-3ntk0 nginx-4ok8v nginx-qrm3m
```
Here, the selector is the same as the selector for the ReplicationController (seen in the
`kubectl describe` output, and in a different form in `replication.yaml`. The `--output=jsonpath` option
`kubectl describe` output), and in a different form in `replication.yaml`. The `--output=jsonpath` option
specifies an expression that just gets the name from each pod in the returned list.
@@ -84,6 +84,6 @@ very small. Please be aware of this risk when setting a non-zero TTL.
[Clean up Jobs automatically](/docs/concepts/workloads/controllers/jobs-run-to-completion/#clean-up-finished-jobs-automatically)
[Design doc](https://github.com/kubernetes/community/blob/master/keps/sig-apps/0026-ttl-after-finish.md)
[Design doc](https://github.com/kubernetes/enhancements/blob/master/keps/sig-apps/0026-ttl-after-finish.md)
{{% /capture %}}
@@ -7,128 +7,103 @@ weight: 40
---
{{% capture overview %}}
This page provides an overview of Init Containers, which are specialized
Containers that run before app Containers and can contain utilities or setup
scripts not present in an app image.
This page provides an overview of init containers: specialized containers that run
before app containers in a {{< glossary_tooltip text="Pod" term_id="pod" >}}.
Init containers can contain utilities or setup scripts not present in an app image.
You can specify init containers in the Pod specification alongside the `containers`
array (which describes app containers).
{{% /capture %}}
This feature has exited beta in 1.6. Init Containers can be specified in the PodSpec
alongside the app `containers` array. The beta annotation value will still be respected
and overrides the PodSpec field value, however, they are deprecated in 1.6 and 1.7.
In 1.8, the annotations are no longer supported and must be converted to the PodSpec field.
{{% capture body %}}
## Understanding Init Containers
A [Pod](/docs/concepts/workloads/pods/pod-overview/) can have multiple Containers running
apps within it, but it can also have one or more Init Containers, which are run
before the app Containers are started.
## Understanding init containers
Init Containers are exactly like regular Containers, except:
A {{< glossary_tooltip text="Pod" term_id="pod" >}} can have multiple containers
running apps within it, but it can also have one or more init containers, which are run
before the app containers are started.
* They always run to completion.
* Each one must complete successfully before the next one is started.
Init containers are exactly like regular containers, except:
If an Init Container fails for a Pod, Kubernetes restarts the Pod repeatedly until the Init
Container succeeds. However, if the Pod has a `restartPolicy` of Never, it is not restarted.
* Init containers always run to completion.
* Each init container must complete successfully before the next one starts.
To specify a Container as an Init Container, add the `initContainers` field on the PodSpec as
a JSON array of objects of type
[Container](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#container-v1-core)
If a Pod's init container fails, Kubernetes repeatedly restarts the Pod until the init container
succeeds. However, if the Pod has a `restartPolicy` of Never, Kubernetes does not restart the Pod.
To specify an init container for a Pod, add the `initContainers` field into
the Pod specification, as an array of objects of type
[Container](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#container-v1-core),
alongside the app `containers` array.
The status of the init containers is returned in `.status.initContainerStatuses`
field as an array of the container statuses (similar to the `.status.containerStatuses`
field).
### Differences from regular Containers
### Differences from regular containers
Init Containers support all the fields and features of app Containers,
Init containers support all the fields and features of app containers,
including resource limits, volumes, and security settings. However, the
resource requests and limits for an Init Container are handled slightly
differently, which are documented in [Resources](#resources) below. Also, Init Containers do not
support readiness probes because they must run to completion before the Pod can
be ready.
resource requests and limits for an init container are handled differently,
as documented in [Resources](#resources).
If multiple Init Containers are specified for a Pod, those Containers are run
one at a time in sequential order. Each must succeed before the next can run.
When all of the Init Containers have run to completion, Kubernetes initializes
the Pod and runs the application Containers as usual.
Also, init containers do not support readiness probes because they must run to
completion before the Pod can be ready.
## What can Init Containers be used for?
If you specify multiple init containers for a Pod, Kubelet runs each init
container sequentially. Each init container must succeed before the next can run.
When all of the init containers have run to completion, Kubelet initializes
the application containers for the Pod and runs them as usual.
Because Init Containers have separate images from app Containers, they
## Using init containers
Because init containers have separate images from app containers, they
have some advantages for start-up related code:
* They can contain and run utilities that are not desirable to include in the
app Container image for security reasons.
* They can contain utilities or custom code for setup that is not present in an app
* Init containers can contain utilities or custom code for setup that are not present in an app
image. For example, there is no need to make an image `FROM` another image just to use a tool like
`sed`, `awk`, `python`, or `dig` during setup.
* Init containers can securely run utilities that would make an app container image less secure.
* The application image builder and deployer roles can work independently without
the need to jointly build a single app image.
* They use Linux namespaces so that they have different filesystem views from app Containers.
Consequently, they can be given access to Secrets that app Containers are not able to
access.
* They run to completion before any app Containers start, whereas app
Containers run in parallel, so Init Containers provide an easy way to block or
delay the startup of app Containers until some set of preconditions are met.
* Init containers can run with a different view of the filesystem than app containers in the
same Pod. Consequently, they can be given access to
{{< glossary_tooltip text="Secrets" term_id="secret" >}} that app containers cannot access.
* Because init containers run to completion before any app containers start, init containers offer
a mechanism to block or delay app container startup until a set of preconditions are met. Once
preconditions are met, all of the app containers in a Pod can start in parallel.
### Examples
Here are some ideas for how to use Init Containers:
Here are some ideas for how to use init containers:
* Wait for a service to be created with a shell command like:
for i in {1..100}; do sleep 1; if dig myservice; then exit 0; fi; done; exit 1
* Wait for a {{< glossary_tooltip text="Service" term_id="service">}} to
be created, using a shell one-line command like:
```shell
for i in {1..100}; do sleep 1; if dig myservice; then exit 0; fi; done; exit 1
```
* Register this Pod with a remote server from the downward API with a command like:
```shell
curl -X POST http://$MANAGEMENT_SERVICE_HOST:$MANAGEMENT_SERVICE_PORT/register -d 'instance=$(<POD_NAME>)&ip=$(<POD_IP>)'
```
`curl -X POST http://$MANAGEMENT_SERVICE_HOST:$MANAGEMENT_SERVICE_PORT/register -d 'instance=$(<POD_NAME>)&ip=$(<POD_IP>)'`
* Wait for some time before starting the app container with a command like
```shell
sleep 60
```
* Clone a Git repository into a {{< glossary_tooltip text="Volume" term_id="volume" >}}
* Wait for some time before starting the app Container with a command like `sleep 60`.
* Clone a git repository into a volume.
* Place values into a configuration file and run a template tool to dynamically
generate a configuration file for the main app Container. For example,
place the POD_IP value in a configuration and generate the main app
generate a configuration file for the main app container. For example,
place the `POD_IP` value in a configuration and generate the main app
configuration file using Jinja.
More detailed usage examples can be found in the [StatefulSets documentation](/docs/concepts/workloads/controllers/statefulset/)
and the [Production Pods guide](/docs/tasks/configure-pod-container/configure-pod-initialization/).
#### Init containers in use
### Init Containers in use
The following yaml file for Kubernetes 1.5 outlines a simple Pod which has two Init Containers.
The first waits for `myservice` and the second waits for `mydb`. Once both
containers complete, the Pod will begin.
```yaml
apiVersion: v1
kind: Pod
metadata:
name: myapp-pod
labels:
app: myapp
annotations:
pod.beta.kubernetes.io/init-containers: '[
{
"name": "init-myservice",
"image": "busybox:1.28",
"command": ["sh", "-c", "until nslookup myservice; do echo waiting for myservice; sleep 2; done;"]
},
{
"name": "init-mydb",
"image": "busybox:1.28",
"command": ["sh", "-c", "until nslookup mydb; do echo waiting for mydb; sleep 2; done;"]
}
]'
spec:
containers:
- name: myapp-container
image: busybox:1.28
command: ['sh', '-c', 'echo The app is running! && sleep 3600']
```
There is a new syntax in Kubernetes 1.6, although the old annotation syntax still works for 1.6 and 1.7. The new syntax must be used for 1.8 or greater. We have moved the declaration of Init Containers to `spec`:
This example defines a simple Pod that has two init containers.
The first waits for `myservice`, and the second waits for `mydb`. Once both
init containers complete, the Pod runs the app container from its `spec` section.
```yaml
apiVersion: v1
@@ -151,9 +126,8 @@ spec:
command: ['sh', '-c', 'until nslookup mydb; do echo waiting for mydb; sleep 2; done;']
```
1.5 syntax still works on 1.6, but we recommend using 1.6 syntax. In Kubernetes 1.6, Init Containers were made a field in the API. The beta annotation is still respected in 1.6 and 1.7, but is not supported in 1.8 or greater.
Yaml file below outlines the `mydb` and `myservice` services:
The following YAML file outlines the `mydb` and `myservice` services:
```yaml
apiVersion: v1
@@ -177,7 +151,7 @@ spec:
targetPort: 9377
```
This Pod can be started and debugged with the following commands:
You can start this Pod by running:
```shell
kubectl apply -f myapp.yaml
@@ -186,6 +160,7 @@ kubectl apply -f myapp.yaml
pod/myapp-pod created
```
And check on its status with:
```shell
kubectl get -f myapp.yaml
```
@@ -194,6 +169,7 @@ NAME READY STATUS RESTARTS AGE
myapp-pod 0/1 Init:0/2 0 6m
```
or for more details:
```shell
kubectl describe -f myapp.yaml
```
@@ -231,13 +207,42 @@ Events:
13s 13s 1 {kubelet 172.17.4.201} spec.initContainers{init-myservice} Normal Created Created container with docker id 5ced34a04634; Security:[seccomp=unconfined]
13s 13s 1 {kubelet 172.17.4.201} spec.initContainers{init-myservice} Normal Started Started container with docker id 5ced34a04634
```
To see logs for the init containers in this Pod, run:
```shell
kubectl logs myapp-pod -c init-myservice # Inspect the first init container
kubectl logs myapp-pod -c init-mydb # Inspect the second init container
```
Once we start the `mydb` and `myservice` services, we can see the Init Containers
complete and the `myapp-pod` is created:
At this point, those init containers will be waiting to discover Services named
`mydb` and `myservice`.
Here's a configuration you can use to make those Services appear:
```yaml
---
apiVersion: v1
kind: Service
metadata:
name: myservice
spec:
ports:
- protocol: TCP
port: 80
targetPort: 9376
---
apiVersion: v1
kind: Service
metadata:
name: mydb
spec:
ports:
- protocol: TCP
port: 80
targetPort: 9377
```
To create the `mydb` and `myservice` services:
```shell
kubectl apply -f services.yaml
@@ -247,104 +252,97 @@ service/myservice created
service/mydb created
```
You'll then see that those init containers complete, and that the `myapp-pod`
Pod moves into the Running state:
```shell
kubectl get -f myapp.yaml
```
```
NAME READY STATUS RESTARTS AGE
myapp-pod 1/1 Running 0 9m
```
This example is very simple but should provide some inspiration for you to
create your own Init Containers.
This simple example should provide some inspiration for you to create your own
init containers. [What's next](#what-s-next) contains a link to a more detailed example.
## Detailed behavior
During the startup of a Pod, the Init Containers are started in order, after the
network and volumes are initialized. Each Container must exit successfully before
the next is started. If a Container fails to start due to the runtime or
During the startup of a Pod, each init container starts in order, after the
network and volumes are initialized. Each container must exit successfully before
the next container starts. If a container fails to start due to the runtime or
exits with failure, it is retried according to the Pod `restartPolicy`. However,
if the Pod `restartPolicy` is set to Always, the Init Containers use
`RestartPolicy` OnFailure.
if the Pod `restartPolicy` is set to Always, the init containers use
`restartPolicy` OnFailure.
A Pod cannot be `Ready` until all Init Containers have succeeded. The ports on an
Init Container are not aggregated under a service. A Pod that is initializing
A Pod cannot be `Ready` until all init containers have succeeded. The ports on an
init container are not aggregated under a Service. A Pod that is initializing
is in the `Pending` state but should have a condition `Initializing` set to true.
If the Pod is [restarted](#pod-restart-reasons), all Init Containers must
execute again.
If the Pod [restarts](#pod-restart-reasons), or is restarted, all init containers
must execute again.
Changes to the Init Container spec are limited to the container image field.
Altering an Init Container image field is equivalent to restarting the Pod.
Changes to the init container spec are limited to the container image field.
Altering an init container image field is equivalent to restarting the Pod.
Because Init Containers can be restarted, retried, or re-executed, Init Container
Because init containers can be restarted, retried, or re-executed, init container
code should be idempotent. In particular, code that writes to files on `EmptyDirs`
should be prepared for the possibility that an output file already exists.
Init Containers have all of the fields of an app Container. However, Kubernetes
prohibits `readinessProbe` from being used because Init Containers cannot
Init containers have all of the fields of an app container. However, Kubernetes
prohibits `readinessProbe` from being used because init containers cannot
define readiness distinct from completion. This is enforced during validation.
Use `activeDeadlineSeconds` on the Pod and `livenessProbe` on the Container to
prevent Init Containers from failing forever. The active deadline includes Init
Containers.
Use `activeDeadlineSeconds` on the Pod and `livenessProbe` on the container to
prevent init containers from failing forever. The active deadline includes init
containers.
The name of each app and Init Container in a Pod must be unique; a
validation error is thrown for any Container sharing a name with another.
The name of each app and init container in a Pod must be unique; a
validation error is thrown for any container sharing a name with another.
### Resources
Given the ordering and execution for Init Containers, the following rules
Given the ordering and execution for init containers, the following rules
for resource usage apply:
* The highest of any particular resource request or limit defined on all Init
Containers is the *effective init request/limit*
* The highest of any particular resource request or limit defined on all init
containers is the *effective init request/limit*
* The Pod's *effective request/limit* for a resource is the higher of:
* the sum of all app Containers request/limit for a resource
* the sum of all app containers request/limit for a resource
* the effective init request/limit for a resource
* Scheduling is done based on effective requests/limits, which means
Init Containers can reserve resources for initialization that are not used
init containers can reserve resources for initialization that are not used
during the life of the Pod.
* QoS tier of the Pod's *effective QoS tier* is the QoS tier for Init Containers
and app containers alike.
* The QoS (quality of service) tier of the Pod's *effective QoS tier* is the
QoS tier for init containers and app containers alike.
Quota and limits are applied based on the effective Pod request and
limit.
Pod level cgroups are based on the effective Pod request and limit, the
same as the scheduler.
Pod level control groups (cgroups) are based on the effective Pod request and
limit, the same as the scheduler.
### Pod restart reasons
A Pod can restart, causing re-execution of Init Containers, for the following
A Pod can restart, causing re-execution of init containers, for the following
reasons:
* A user updates the PodSpec causing the Init Container image to change. Any
changes to the Init Container image restarts the Pod. App Container image
changes only restart the app Container.
* A user updates the Pod specification, causing the init container image to change.
Any changes to the init container image restarts the Pod. App container image
changes only restart the app container.
* The Pod infrastructure container is restarted. This is uncommon and would
have to be done by someone with root access to nodes.
* All containers in a Pod are terminated while `restartPolicy` is set to Always,
forcing a restart, and the Init Container completion record has been lost due
forcing a restart, and the init container completion record has been lost due
to garbage collection.
## Support and compatibility
A cluster with Apiserver version 1.6.0 or greater supports Init Containers
using the `.spec.initContainers` field. Previous versions support Init Containers
using the alpha or beta annotations. The `.spec.initContainers` field is also mirrored
into alpha and beta annotations so that Kubelets version 1.3.0 or greater can execute
Init Containers, and so that a version 1.6 apiserver can safely be rolled back to version
1.5.x without losing Init Container functionality for existing created pods.
In Apiserver and Kubelet versions 1.8.0 or greater, support for the alpha and beta annotations
is removed, requiring a conversion from the deprecated annotations to the
`.spec.initContainers` field.
{{% /capture %}}
{{% capture whatsnext %}}
* [Creating a Pod that has an Init Container](/docs/tasks/configure-pod-container/configure-pod-initialization/#creating-a-pod-that-has-an-init-container)
* Read about [creating a Pod that has an init container](/docs/tasks/configure-pod-container/configure-pod-initialization/#creating-a-pod-that-has-an-init-container)
* Learn how to [debug init containers](/docs/tasks/debug-application-cluster/debug-init-containers/)
{{% /capture %}}
@@ -21,7 +21,7 @@ A *Pod* is the basic execution unit of a Kubernetes application--the smallest an
A Pod encapsulates an application's container (or, in some cases, multiple containers), storage resources, a unique network IP, and options that govern how the container(s) should run. A Pod represents a unit of deployment: *a single instance of an application in Kubernetes*, which might consist of either a single {{< glossary_tooltip text="container" term_id="container" >}} or a small number of containers that are tightly coupled and that share resources.
[Docker](https://www.docker.com) is the most common container runtime used in a Kubernetes Pod, but Pods support other container runtimes as well.
[Docker](https://www.docker.com) is the most common container runtime used in a Kubernetes Pod, but Pods support other [container runtimes](https://kubernetes.io/docs/setup/production-environment/container-runtimes/) as well.
Pods in a Kubernetes cluster can be used in two main ways:
@@ -186,7 +186,7 @@ You must specify an additional flag `--force` along with `--grace-period=0` in o
### Force deletion of pods
Force deletion of a Pod is defined as deletion of a Pod from the cluster state and etcd immediately. When a force deletion is performed, the apiserver does not wait for confirmation from the kubelet that the Pod has been terminated on the node it was running on. It removes the Pod in the API immediately so a new Pod can be created with the same name. On the node, Pods that are set to terminate immediately will still be given a small grace period before being force killed.
Force deletion of a Pod is defined as deletion of a Pod from the cluster state and etcd immediately. When a force deletion is performed, the API server does not wait for confirmation from the kubelet that the Pod has been terminated on the node it was running on. It removes the Pod in the API immediately so a new Pod can be created with the same name. On the node, Pods that are set to terminate immediately will still be given a small grace period before being force killed.
Force deletions can be potentially dangerous for some Pods and should be performed with caution. In case of StatefulSet Pods, please refer to the task documentation for [deleting Pods from a StatefulSet](/docs/tasks/run-application/force-delete-stateful-set-pod/).
@@ -74,7 +74,11 @@ In order to use Pod Presets in your cluster you must ensure the following:
1. You have enabled the admission controller `PodPreset`. One way to doing this
is to include `PodPreset` in the `--enable-admission-plugins` option value specified
for the API server. In minikube add this flag
`--extra-config=apiserver.enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota,PodPreset`
```shell
--extra-config=apiserver.enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota,PodPreset
```
while starting the cluster.
1. You have defined your Pod Presets by creating `PodPreset` objects in the
namespace you will use.
+21 -4
View File
@@ -29,8 +29,6 @@ The PR wranglers duties include:
- Ensure that the CLA is signed by each contributor.
- Help new contributors sign the [CLA](https://github.com/kubernetes/community/blob/master/CLA.md).
- Use [this](https://github.com/zparnold/k8s-docs-pr-botherer) script to automatically remind contributors that havent signed the CLA to sign the CLA.
- Close any PR where the CLA hasnt been signed for two weeks.
PR authors can reopen the PR after signing the CLA, so this is a low-risk way to make sure nothing gets merged without a signed CLA.
- Provide feedback on proposed changes and help facilitate technical reviews from members of other SIGs.
- Provide inline suggestions on the PR for the proposed content changes.
- If you need to verify content, comment on the PR and request more details.
@@ -59,6 +57,25 @@ reviewed is usually small. These queries specifically exclude localization PRs,
Determine whether any additional changes or updates need to be made for the PR to be merged. If you think the PR is ready to be merged, comment `/approve`.
- [Not against master](https://github.com/kubernetes/website/pulls?utf8=%E2%9C%93&q=is%3Aopen+is%3Apr+-label%3Ado-not-merge+label%3Alanguage%2Fen+-base%3Amaster): If it's against a `dev-` branch, it's for an upcoming release. Make sure the [release meister](https://github.com/kubernetes/sig-release/tree/master/release-team) knows about it by adding a comment with `/assign @<meister's_github-username>`. If it's against an old branch, help the PR author figure out whether it's targeted against the best branch.
### When to close Pull Requests
Reviews and approvals are one tool to keep our PR queue short and current. Another tool is closure.
- Close any PR where the CLA hasnt been signed for two weeks.
PR authors can reopen the PR after signing the CLA, so this is a low-risk way to make sure nothing gets merged without a signed CLA.
- Close any PR where the author has not responded to comments or feedback in 2 or more weeks.
Don't be afraid to close pull requests. Contributors can easily reopen and resume works in progress. Oftentimes a closure notice is what spurs an author to resume and finish their contribution.
To close a pull request, leave a `/close` comment on the PR.
{{< note >}}
An automated service, [`fejta-bot`](https://github.com/fejta-bot) automatically marks issues as stale after 90 days of inactivity, then closes them after an additional 30 days of inactivity when they become rotten. PR wranglers should close issues after 14-30 days of inactivity.
{{< /note >}}
## Propose improvements
SIG Docs
@@ -176,7 +193,7 @@ Responsibilities include:
- Learn and set best practices for the SIG by updating contribution guidelines
- Schedule and run SIG meetings: weekly status updates, quarterly retro/planning sessions, and others as needed
- Schedule and run doc sprints at KubeCon events and other conferences
- - Recruit for and advocate on behalf of SIG Docs with the CNCF and its platinum partners, including Google, Oracle, Azure, IBM, and Huawei
- Recruit for and advocate on behalf of SIG Docs with the {{< glossary_tooltip text="CNCF" term_id="cncf" >}} and its platinum partners, including Google, Oracle, Azure, IBM, and Huawei
- Keep the SIG running smoothly
### Running effective meetings
@@ -190,7 +207,7 @@ To schedule and run effective meetings, these guidelines show what to do, how to
**Set a clear agenda**:
- Set a clear agenda of topics
- Publish the agenda in advance
- Publish the agenda in advance
For weekly meetings, copypaste the previous week's notes into the "Past meetings" section of the notes
@@ -48,64 +48,80 @@ information, see
{{% capture steps %}}
## Getting three repositories
## Setting up the local repositories
If you don't already have the kubernetes/kubernetes repository, get it now:
Create a local workspace and set your `GOPATH`.
```shell
mkdir $GOPATH/src
cd $GOPATH/src
go get github.com/kubernetes/kubernetes
mkdir -p $HOME/<workspace>
export GOPATH=$HOME/<workspace>
```
Determine the base directory of your clone of the
[kubernetes/kubernetes](https://github.com/kubernetes/kubernetes) repository.
For example, if you followed the preceding step to get the repository, your
base directory is `$GOPATH/src/github.com/kubernetes/kubernetes.`
The remaining steps refer to your base directory as `<k8s-base>`.
Get a local clone of the following repositories:
```shell
go get -u github.com/spf13/pflag
go get -u github.com/spf13/cobra
go get -u gopkg.in/yaml.v2
go get -u kubernetes-incubator/reference-docs
```
If you don't already have the kubernetes/website repository, get it now:
```shell
mkdir $GOPATH/src
cd $GOPATH/src
go get github.com/kubernetes/website
git clone https://github.com/<your-username>/website $GOPATH/src/github.com/<your-username>/website
```
Determine the base directory of your clone of the
[kubernetes/website](https://github.com/kubernetes/website) repository.
For example, if you followed the preceding step to get the repository, your
base directory is `$GOPATH/src/github.com/kubernetes/website.`
The remaining steps refer to your base directory as `<web-base>`.
If you don't already have the kubernetes-incubator/reference-docs repository, get it now:
Get a clone of the kubernetes/kubernetes repository as k8s.io/kubernetes:
```shell
mkdir $GOPATH/src
cd $GOPATH/src
go get github.com/kubernetes-incubator/reference-docs
git clone https://github.com/kubernetes/kubernetes $GOPATH/src/k8s.io/kubernetes
```
Determine the base directory of your clone of the
Remove the spf13 package from `$GOPATH/src/k8s.io/kubernetes/vendor/github.com`.
```shell
rm -rf $GOPATH/src/k8s.io/kubernetes/vendor/github.com/spf13
```
The kubernetes/kubernetes repository provides access to the kubectl and kustomize source code.
* Determine the base directory of your clone of the
[kubernetes/kubernetes](https://github.com/kubernetes/kubernetes) repository.
For example, if you followed the preceding step to get the repository, your
base directory is `$GOPATH/src/k8s.io/kubernetes.`
The remaining steps refer to your base directory as `<k8s-base>`.
* Determine the base directory of your clone of the
[kubernetes/website](https://github.com/kubernetes/website) repository.
For example, if you followed the preceding step to get the repository, your
base directory is `$GOPATH/src/github.com/<your-username>/website.`
The remaining steps refer to your base directory as `<web-base>`.
* Determine the base directory of your clone of the
[kubernetes-incubator/reference-docs](https://github.com/kubernetes-incubator/reference-docs) repository.
For example, if you followed the preceding step to get the repository, your
base directory is `$GOPATH/src/github.com/kubernetes-incubator/reference-docs.`
The remaining steps refer to your base directory as `<rdocs-base>`.
In your local kubernetes/kubernetes repository, check out the branch of interest,
In your local k8s.io/kubernetes repository, check out the branch of interest,
and make sure it is up to date. For example, if you want to generate docs for
Kubernetes 1.9, you could use these commands:
Kubernetes 1.15, you could use these commands:
```shell
cd <k8s-base>
git checkout release-1.9
git pull https://github.com/kubernetes/kubernetes release-1.9
git checkout release-1.15
git pull https://github.com/kubernetes/kubernetes release-1.15
```
If you do not need to edit the kubectl source code, follow the instructions to [Edit the Makefile](#editing-makefile).
## Editing the kubectl source code
The reference documentation for the kubectl commands is automatically generated from
kubectl source code. If you want to change the reference documentation, the first step
The kubectl command reference documentation is automatically generated from
the kubectl source code. If you want to change the reference documentation, the first step
is to change one or more comments in the kubectl source code. Make the change in your
local kubernetes/kubernetes repository, and then submit a pull request to the master branch of
[github.com/kubernetes/kubernetes](https://github.com/kubernetes/kubernetes).
@@ -124,7 +140,7 @@ version that has already been released, you need to propose that your change be
cherry picked into the release branch.
For example, suppose the master branch is being used to develop Kubernetes 1.10,
and you want to backport your change to the release-1.9 branch. For instructions
and you want to backport your change to the release-1.15 branch. For instructions
on how to do this, see
[Propose a Cherry Pick](https://git.k8s.io/community/contributors/devel/sig-release/cherry-picks.md).
@@ -138,104 +154,115 @@ need to work with someone who can set the label and milestone for you.
## Editing Makefile
Go to `<rdocs-base>`, and open `Makefile` for editing:
Go to `<rdocs-base>`, and open the `Makefile` for editing:
Set `K8SROOT` to the base directory of your local kubernetes/kubernetes
repository. Set `WEBROOT` to the base directory of your local kubernetes/website repository.
Set `MINOR_VERSION` to the minor version of the docs you want to build. For example,
if you want to build docs for Kubernetes 1.9, set `MINOR_VERSION` to 9. Save and close `Makefile`.
* Set `K8SROOT` to `<k8s-base>`.
* Set `WEBROOT` to `<web-base>`.
* Set `MINOR_VERSION` to the minor version of the docs you want to build. For example,
if you want to build docs for Kubernetes 1.15, set `MINOR_VERSION` to 15. Save and close the `Makefile`.
## Building the brodocs image
For example, update the following variables:
The doc generation code requires the `pwittrock/brodocs` Docker image.
This command creates the `pwittrock/brodocs` Docker image. It also tries to push the image to
DockerHub, but it's OK if that step fails. As long as you have the image locally, the code generation
can succeed.
```shell
make brodocs
```
Verify that you have the brodocs image:
```shell
docker images
```
The output shows `pwittrock/brodocs` as one of the available images:
```shell
REPOSITORY TAG IMAGE ID CREATED SIZE
pwittrock/brodocs latest 999d34a50d56 5 weeks ago 714MB
WEBROOT=$(GOPATH)/src/github.com/<your-username>/website
K8SROOT=$(GOPATH)/src/k8s.io/kubernetes
MINOR_VERSION=15
```
## Creating a version directory
In the `gen-kubectldocs/generators` directory, if you do not already
have a directory named `v1_MINOR_VERSION`, create one now by copying the directory
The version directory is a staging area for the kubectl command reference build.
The YAML files in this directory are used to create the structure and navigation
of the kubectl command reference.
In the `<rdocs-base>/gen-kubectldocs/generators` directory, if you do not already
have a directory named `v1_<MINOR_VERSION>`, create one now by copying the directory
for the previous version. For example, suppose you want to generate docs for
Kubernetes 1.9, but you don't already have a `v1_9` directory. Then you could
create and populate a `v1_9` directory by running these commands:
Kubernetes 1.15, but you don't already have a `v1_15` directory. Then you could
create and populate a `v1_15` directory by running these commands:
```shell
mkdir gen-kubectldocs/generators/v1_9
cp -r gen-kubectldocs/generators/v1_8/* gen-kubectldocs/generators/v1_9
mkdir gen-kubectldocs/generators/v1_15
cp -r gen-kubectldocs/generators/v1_14/* gen-kubectldocs/generators/v1_15
```
## Checking out a branch in kubernetes/kubernetes
## Checking out a branch in k8s.io/kubernetes
In you local kubernetes/kubernetes repository, checkout the branch that has
In your local <k8s-base> repository, checkout the branch that has
the version of Kubernetes that you want to document. For example, if you want
to generate docs for Kubernetes 1.9, checkout the release-1.9 branch. Make sure
to generate docs for Kubernetes 1.15, checkout the release-1.15 branch. Make sure
you local branch is up to date.
```shell
cd <k8s-base>
git checkout release-1.15
git pull https://github.com/kubernetes/kubernetes release-1.15
```
## Running the doc generation code
In you local kubernetes-incubator/reference-docs repository, build and run the
doc generation code. You might need to run the command as root:
```shell
cd <rdocs-base>
make cli
```
## Locate the generated files
These two files are the primary output of a successful build. Verify that they exist:
* `<rdocs-base>/gen-kubectldocs/generators/build/index.html`
* `<rdocs-base>/gen-kubectldocs/generators/build/navData.js`
## Copying files to the kubernetes/website repository
Copy the generated files from your local kubernetes-incubator/reference-docs
repository to your local kubernetes/website repository.
In your local kubernetes-incubator/reference-docs repository, build and run the
kubectl command reference generation code. You might need to run the command as root:
```shell
cd <rdocs-base>
make copycli
```
## Adding and committing changes in kubernetes/website
The `copycli` command will clean the staging directories, generate the kubectl command files,
and copy the collated kubectl reference HTML page and assets to `<web-base>`.
List the files that were generated and copied to the `kubernetes/website`
repository:
## Locate the generated files
Verify that these two files have been generated:
```shell
[ -e "<rdocs-base>/gen-kubectldocs/generators/build/index.html" ] && echo "index.html built" || echo "no index.html"
[ -e "<rdocs-base>/gen-kubectldocs/generators/build/navData.js" ] && echo "navData.js built" || echo "no navData.js"
```
## Locate the copied files
Verify that all generated files have been copied to your `<web-base>`:
```shell
cd <web-base>
git status
```
The output shows the new and modified files. For example, the output
might look like this:
The output should include the modified files:
```
static/docs/reference/generated/kubectl/kubectl-commands.html
static/docs/reference/generated/kubectl/navData.js
```
Additionally, the output might show the modified files:
```
static/docs/reference/generated/kubectl/scroll.js
static/docs/reference/generated/kubectl/stylesheet.css
static/docs/reference/generated/kubectl/tabvisibility.js
static/docs/reference/generated/kubectl/node_modules/bootstrap/dist/css/bootstrap.min.css
static/docs/reference/generated/kubectl/node_modules/highlight.js/styles/default.css
static/docs/reference/generated/kubectl/node_modules/jquery.scrollto/jquery.scrollTo.min.js
static/docs/reference/generated/kubectl/node_modules/jquery/dist/jquery.min.js
static/docs/reference/generated/kubectl/node_modules/font-awesome/css/font-awesome.min.css
```
## Locally test the documentation
Build the Kubernetes documentation in your local `<web-base>`.
```shell
modified: docs/reference/generated/kubectl/kubectl-commands.html
modified: docs/reference/generated/kubectl/navData.js
cd <web-base>
make docker-serve
```
View the [local preview](https://localhost:1313/docs/reference/generated/kubectl/kubectl-commands/).
## Adding and committing changes in kubernetes/website
Run `git add` and `git commit` to commit the files.
## Creating a pull request
@@ -257,7 +284,4 @@ topics will be visible in the
* [Generating Reference Documentation for the Kubernetes API](/docs/home/contribute/generated-reference/kubernetes-api/)
* [Generating Reference Documentation for the Kubernetes Federation API](/docs/home/contribute/generated-reference/federation-api/)
{{% /capture %}}
{{% /capture %}}
@@ -5,8 +5,16 @@ content_template: templates/task
{{% capture overview %}}
This page shows how to update the generated reference docs for the
Kubernetes API.
This page shows how to update the generated reference docs for the Kubernetes API.
The Kubernetes API reference documentation is built from the
[Kubernetes OpenAPI spec](https://github.com/kubernetes/kubernetes/blob/master/api/openapi-spec/swagger.json)
and tools from [kubernetes-incubator/reference-docs](https://github.com/kubernetes-incubator/reference-docs).
If you find bugs in the generated documentation, you need to
[fix them upstream](/docs/contribute/generate-ref-docs/contribute-upstream/).
If you need only to regenerate the reference documentation from the [OpenAPI](https://github.com/OAI/OpenAPI-Specification)
spec, continue reading this page.
{{% /capture %}}
@@ -18,8 +26,6 @@ You need to have these tools installed:
* [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)
* [Golang](https://golang.org/doc/install) version 1.9.1 or later
Your $GOPATH environment variable must be set.
You need to know how to create a pull request (PR) to a GitHub repository.
Typically, this involves creating a fork of the repository. For more
information, see
@@ -31,96 +37,78 @@ information, see
{{% capture steps %}}
## The big picture
## Setting up the local repositories
The reference documentation for the Kubernetes API is generated in two separate stages:
1. Generate an OpenAPI spec from the Kubernetes source code. The tools for
this stage are at [kubernetes/kubernetes/hack](https://github.com/kubernetes/kubernetes/tree/master/hack).
1. Generate an HTML file from the OpenAPI spec. The tools for this stage are at
[kubernetes-incubator/reference-docs](https://github.com/kubernetes-incubator/reference-docs).
If you find bugs in the generated documentation generated, you need to
[fix them upstream](/docs/contribute/generate-ref-docs/contribute-upstream/).
If you need only to regenerate the reference documentation from the OpenAPI
spec, continue reading this page.
## Getting three repositories
If you don't already have the kubernetes/kubernetes repository, get it now:
Create a local workspace and set your `GOPATH`.
```shell
mkdir $GOPATH/src
cd $GOPATH/src
go get github.com/kubernetes/kubernetes
mkdir -p $HOME/<workspace>
export GOPATH=$HOME/<workspace>
```
Determine the base directory of your clone of the
[kubernetes/kubernetes](https://github.com/kubernetes/kubernetes) repository.
For example, if you followed the preceding step to get the repository, your
base directory is `$GOPATH/src/github.com/kubernetes/kubernetes.`
The remaining steps refer to your base directory as `<k8s-base>`.
Get a local clone of the following repositories:
```shell
go get -u github.com/kubernetes-incubator/reference-docs
go get -u github.com/go-openapi/loads
go get -u github.com/go-openapi/spec
```
If you don't already have the kubernetes/website repository, get it now:
```shell
mkdir $GOPATH/src
cd $GOPATH/src
go get github.com/kubernetes/website
git clone https://github.com/<your-username>/website $GOPATH/src/github.com/<your-username>/website
```
Determine the base directory of your clone of the
[kubernetes/website](https://github.com/kubernetes/website) repository.
For example, if you followed the preceding step to get the repository, your
base directory is `$GOPATH/src/github.com/kubernetes/website.`
The remaining steps refer to your base directory as `<web-base>`.
If you don't already have the kubernetes-incubator/reference-docs repository, get it now:
Get a clone of the kubernetes/kubernetes repository as k8s.io/kubernetes:
```shell
mkdir $GOPATH/src
cd $GOPATH/src
go get github.com/kubernetes-incubator/reference-docs
git clone https://github.com/kubernetes/kubernetes $GOPATH/src/k8s.io/kubernetes
```
Determine the base directory of your clone of the
[kubernetes-incubator/reference-docs](https://github.com/kubernetes-incubator/reference-docs) repository.
For example, if you followed the preceding step to get the repository, your
base directory is `$GOPATH/src/github.com/kubernetes-incubator/reference-docs.`
* The base directory of your clone of the
[kubernetes/kubernetes](https://github.com/kubernetes/kubernetes) repository is
`$GOPATH/src/k8s.io/kubernetes.`
The remaining steps refer to your base directory as `<k8s-base>`.
* The base directory of your clone of the
[kubernetes/website](https://github.com/kubernetes/website) repository is
`$GOPATH/src/github.com/<your username>/website.`
The remaining steps refer to your base directory as `<web-base>`.
* The base directory of your clone of the
[kubernetes-incubator/reference-docs](https://github.com/kubernetes-incubator/reference-docs)
repository is `$GOPATH/src/github.com/kubernetes-incubator/reference-docs.`
The remaining steps refer to your base directory as `<rdocs-base>`.
## Generating the API reference docs for publishing
The preceding section showed how to edit a source file and then generate
several files, including `api/openapi-spec/swagger.json` in the
`kubernetes/kubernetes` repository.
## Generating the API reference docs
This section shows how to generate the
[published Kubernetes API reference documentation](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/),
which is generated by the tools at
[kubernetes-incubator/reference-docs](https://github.com/kubernetes-incubator/reference-docs).
Those tools take the `api/openapi-spec/swagger.json` file as input.
[published Kubernetes API reference documentation](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/).
### Editing Makefile in kubernetes-incubator/reference-docs
### Modifying the Makefile
Go to `<rdocs-base>`, and open `Makefile` for editing:
Go to `<rdocs-base>`, and open the `Makefile` for editing:
Set `K8SROOT` to the base directory of your local kubernetes/kubernetes
repository. Set `WEBROOT` to the base directory of your local kubernetes/website repository.
Set `MINOR_VERSION` to the minor version of the docs you want to build. For example,
if you want to build docs for Kubernetes 1.9, set `MINOR_VERSION` to 9. Save and close `Makefile`.
* Set `K8SROOT` to `<k8s-base>`.
* Set `WEBROOT` to `<web-base>`.
* Set `MINOR_VERSION` to the minor version of the docs you want to build. For example,
if you want to build docs for Kubernetes 1.15, set `MINOR_VERSION` to 15. Save and close the `Makefile`.
For example, update the following variables:
```
WEBROOT=$(GOPATH)/src/github.com/<your-username>/website
K8SROOT=$(GOPATH)/src/k8s.io/kubernetes
MINOR_VERSION=15
```
### Copying the OpenAPI spec
The doc generation code needs a local copy of the OpenAPI spec for the Kubernetes API.
Go to `<k8s-base>` and check out the branch that has the OpenAPI spec you want to use.
For example, if you want to generate docs for Kubernetes 1.9, checkout the release-1.9
branch.
Go back to `<rdocs-base>`. Enter the following command to copy the OpenAPI spec from the
`kubernetes/kubernetes` repository to a local directory:
Run the following command in `<rdocs-base>`:
```shell
make updateapispec
@@ -129,43 +117,44 @@ make updateapispec
The output shows that the file was copied:
```shell
cp ~/src/github.com/kubernetes/kubernetes/api/openapi-spec/swagger.json gen-apidocs/generators/openapi-spec/swagger.json
cp ~/src/k8s.io/kubernetes/api/openapi-spec/swagger.json gen-apidocs/generators/openapi-spec/swagger.json
```
### Building the API reference docs
Run the following command to generate the API reference docs:
Run the following command in `<rdocs-base>`:
```shell
cd <rdocs-base>
make api
```
### Locate the generated files
Verify that these two files have been generated:
These two files are the output of a successful build. Verify that they exist:
```shell
[ -e "<rdocs-base>/gen-apidocs/generators/build/index.html" ] && echo "index.html built" || echo "no index.html"
[ -e "<rdocs-base>/gen-apidocs/generators/build/navData.js" ] && echo "navData.js built" || echo "no navData.js"
```
* `<rdocs-base>/gen-apidocs/generators/build/index.html`
* `<rdocs-base>/gen-apidocs/generators/build/navData.js`
### Creating directories for published docs
Create the directories in `<web-base>` for the generated API reference files:
```shell
mkdir -p <web-base>/static/docs/reference/generated/kubernetes-api/v1.<minor-version>
mkdir -p <web-base>/static/docs/reference/generated/kubernetes-api/v1.<minor-version>/css
mkdir -p <web-base>/static/docs/reference/generated/kubernetes-api/v1.<minor-version>/fonts
```
## Copying the generated docs to the kubernetes/website repository
The preceding sections showed how to generate reference documentation for publication.
This section shows how to copy the generated reference to the
[kubernetes/website](https://github.com/kubernetes/website) repository. The files
in the `kubernetes/website` repository are published in the
[kubernetes.io](https://kubernetes.io) website. In particular, the generated
`index.html` file is published [here](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/).
Enter the following command to copy the generated files to
Run the following command in `<rdocs-base>` to copy the generated files to
your local kubernetes/website repository:
```shell
make copyapi
```
Go to the base of your local kubernetes/website repository, and
Go to the base of your local kubernetes/website repository, and
see which files have been modified:
```shell
@@ -175,15 +164,52 @@ git status
The output shows the modified files:
```shell
On branch master
...
modified: docs/reference/generated/kubernetes-api/v1.9/index.html
```
static/docs/reference/generated/kubernetes-api/v1.15/css/bootstrap.min.css
static/docs/reference/generated/kubernetes-api/v1.15/css/font-awesome.min.css
static/docs/reference/generated/kubernetes-api/v1.15/css/stylesheet.css
static/docs/reference/generated/kubernetes-api/v1.15/fonts/FontAwesome.otf
static/docs/reference/generated/kubernetes-api/v1.15/fonts/fontawesome-webfont.eot
static/docs/reference/generated/kubernetes-api/v1.15/fonts/fontawesome-webfont.svg
static/docs/reference/generated/kubernetes-api/v1.15/fonts/fontawesome-webfont.ttf
static/docs/reference/generated/kubernetes-api/v1.15/fonts/fontawesome-webfont.woff
static/docs/reference/generated/kubernetes-api/v1.15/fonts/fontawesome-webfont.woff2
static/docs/reference/generated/kubernetes-api/v1.15/index.html
static/docs/reference/generated/kubernetes-api/v1.15/jquery.scrollTo.min.js
static/docs/reference/generated/kubernetes-api/v1.15/navData.js
static/docs/reference/generated/kubernetes-api/v1.15/scroll.js
```
In this example, only one file has been modified. Recall that you generated both
`index.html` and `navData.js`. But apparently the generated `navata.js` is not different
from the `navData.js` that was already in the kubernetes/website` repository.
## Updating the API reference index pages
* Open `<web-base>/content/en/docs/reference/kubernetes-api/index.md` for editing, and update the API reference
version number. For example:
```
---
title: v1.15
---
[Kubernetes API v1.15](/docs/reference/generated/kubernetes-api/v1.15/)
```
* Open `<web-base>/content/en/docs/reference/_index.md` for editing, and add a
new link for the latest API reference. Remove the oldest API reference version.
There should be five links to the most recent API references.
## Locally test the API reference
Publish a local version of the API reference.
Verify the [local preview](http://localhost:1313/docs/reference/generated/kubernetes-api/v1.15/).
```shell
cd <web-base>
make docker-serve
```
## Commit the changes
In `<web-base>` run `git add` and `git commit` to commit the change.
@@ -193,9 +219,6 @@ Submit your changes as a
Monitor your pull request, and respond to reviewer comments as needed. Continue
to monitor your pull request until it has been merged.
A few minutes after your pull request is merged, your changes will be visible
in the [published reference documentation](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/).
{{% /capture %}}
{{% capture whatsnext %}}
@@ -204,5 +227,4 @@ in the [published reference documentation](/docs/reference/generated/kubernetes-
* [Generating Reference Documentation for kubectl Commands](/docs/home/contribute/generated-reference/kubectl/)
* [Generating Reference Documentation for the Kubernetes Federation API](/docs/home/contribute/generated-reference/federation-api/)
{{% /capture %}}
{{% /capture %}}
+18 -42
View File
@@ -63,7 +63,7 @@ active participants.
Before you start reviewing PRs, make sure you are familiar with the
[Documentation Style Guide](/docs/contribute/style/style-guide/)
and the [code of conduct](/community/code-of-conduct/)
and the [code of conduct](/community/code-of-conduct/).
### Find a PR to review
@@ -101,7 +101,7 @@ more information about the responsibilities of reviewers and approvers, see
indicates that a PR is technically accurate by leaving a `/lgtm` comment on
the PR.
{{< note >}}Don't add an `/lgtm` unless you are confident in the technical
{{< note >}}Don't add a `/lgtm` unless you are confident in the technical
accuracy of the documentation modified or introduced in the PR.{{< /note >}}
- An approver reviews pull request content for docs quality and adherence to
@@ -109,13 +109,19 @@ more information about the responsibilities of reviewers and approvers, see
[style guide](/docs/contribute/style/style-guide). Only people listed as
approvers in the
[`OWNERS`](https://github.com/kubernetes/website/blob/master/OWNERS) file can
approve a PR. To approve a PR, leave an `/approved` comment on the PR.
approve a PR. To approve a PR, leave an `/approve` comment on the PR.
A PR is merged when it has both a `/lgtm` comment from anyone in the Kubernetes
organization and an `/approved` comment from an approver in the
organization and an `/approve` comment from an approver in the
`sig-docs-maintainers` group, as long as it is not on hold and the PR author
has signed the CLA.
{{< note >}}
The ["Participating"](/docs/contribute/participating/#approvers) section contains more information for reviewers and approvers, including specific responsibilities for approvers.
{{< /note >}}
### Review a PR
1. Read the PR description and read any attached issues or links, if
@@ -172,9 +178,9 @@ has signed the CLA.
provide technical review.
- To prevent the PR from being merged, add `/hold`. This sets the
label `do-not-merge/hold`.
- If a PR has no conflicts and has the `lgtm` and `approved` label but
- If a PR has no conflicts and has the `lgtm` and `approve` labels but
no `hold` label, it is merged automatically.
- If a PR has the `lgtm` and/or `approved` labels and new changes are
- If a PR has the `lgtm` and/or `approve` labels and new changes are
detected, these labels are removed automatically.
See
@@ -575,45 +581,15 @@ These guidelines are not set in stone and are subject to change.
not be assigned automatically. A bug is a problem with existing content or
functionality, and a feature is a request for new content or functionality.
The `kind/documentation` label is not currently in use.
- Priority labels: define the relative severity of the issue. These do not
conform to those outlined in the
[Kubernetes contributor guide](https://github.com/kubernetes/community/blob/master/contributors/guide/issue-triage.md#define-priority), and can be one of `P1`, `P2`, or `P3`, if set.
- To add a label, you can use GitHub's **Labels** widget if you are a Sig Docs
approver. Anyone who is a member of the Kubernetes organization can add a
label by leaving a comment like `/label <label-to-add>`. The label must
- Priority labels: define the relative severity of the issue, as outlined in the
[Kubernetes contributor guide](https://github.com/kubernetes/community/blob/master/contributors/guide/issue-triage.md#define-priority).
- To add a label, leave a comment like `/label <label-to-add>`. The label must
already exist. If you try to add a label that does not exist, the command is
silently ignored.
### Priorities
An issue's priority influences how quickly it is addressed. For documentation,
here are the guidelines for setting a priority on an issue:
#### P1
- Major content errors affecting more than 1 page
- Broken code sample on a heavily trafficked page
- Errors on a “getting started” page
- Well known or highly publicized customer pain points
- Automation issues
#### P2
This is the default for new issues and pull requests.
- Broken code for sample that is not heavily used
- Minor content issues in a heavily trafficked page
- Major content issues on a lower-trafficked page
#### P3
- Typos and broken anchor links
- Documentation feature requests
- "Nice to have" items
### Handling special issue types
We've encountered the following types of issues often enough to document how
We encounter the following types of issues often enough to document how
to handle them.
#### Duplicate issues
@@ -629,8 +605,8 @@ same problem.
Depending on where the dead link is reported, different actions are required to
resolve the issue. Dead links in the API and Kubectl docs are automation issues
and should be assigned a P1 until the problem can be fully understood. All other
dead links are issues that need to be manually fixed and can be assigned a P3.
and should be assigned `/priority critical-urgent` until the problem can be fully understood. All other
dead links are issues that need to be manually fixed and can be assigned `/priority important-longterm`.
#### Blog issues
+4 -2
View File
@@ -195,9 +195,11 @@ cp content/en/docs/tutorials/kubernetes-basics.md content/de/docs/tutorials/kube
Translation tools can speed up the translation process. For example, some editors offers plugins to quickly translate text.
{{< caution >}}
Use translation tools with care. Their output is not always accurate.
Machine-generated translation alone does not meet the minimum standard of quality and requires extensive human review to meet that standard.
{{< /caution >}}
To ensure accuracy in grammar and meaning, members of your localization team should carefully review all machine-generated translations before publishing.
### Source files
Localizations must be based on the English files from the most recent release, {{< latest-version >}}.
@@ -283,6 +285,6 @@ You can also help add or improve content to an existing localization. Join the [
Once a localization meets requirements for workflow and minimum output, SIG docs will:
- Enable language selection on the website
- Publicize the localization's availability through [Cloud Native Computing Foundation](https://www.cncf.io/) (CNCF) channels, including the [Kubernetes blog](https://kubernetes.io/blog/).
- Publicize the localization's availability through [Cloud Native Computing Foundation](https://www.cncf.io/about/) (CNCF) channels, including the [Kubernetes blog](https://kubernetes.io/blog/).
{{% /capture %}}
+17 -9
View File
@@ -201,21 +201,29 @@ If you are approved, request that a current SIG Docs approver add you to the
GitHub group. Only members of the `kubernetes-website-admins` GitHub group can
add new members to a GitHub group.
#### Becoming a website admin
#### Approver responsibilities
Members of the `kubernetes-website-admins` GitHub group can manage GitHub group
membership and have full administrative rights to the settings of the repository,
including the ability to add, remove, and troubleshoot webhooks. Not all SIG
Docs approvers need this level of access.
Approvers improve the documentation by reviewing and merging pull requests into the website repository. Because this role carries additional privileges, approvers have additional responsibilities:
If you think you need this level of access, talk to an existing website admin or
ask in the #sig-docs channel on [Kubernetes Slack](https://kubernetes.slack.com).
- Approvers can use the `/approve` command, which merges PRs into the repo.
A careless merge can break the site, so be sure that when you merge something, you mean it.
- Make sure that proposed changes meet the contribution guidelines.
If you ever have a question, or you're not sure about something, feel free to call for additional review.
- Verify that netlify tests pass before you `/approve` a PR.
<img src="/images/docs/contribute/netlify-pass.png" width="75%" alt="Netlify tests must pass before approving" />
- Visit the netlify page preview for a PR to make sure things look good before approving.
#### PR Wrangler
SIG Docs approvers are added to the
SIG Docs approvers participate in the
[PR Wrangler rotation scheduler](https://github.com/kubernetes/website/wiki/PR-Wranglers)
for weekly rotations. All SIG Docs approvers are expected to take part in this
for weekly rotations. SIG Docs expects all approvers to participate in this
rotation. See
[Be the PR Wrangler for a week](/docs/contribute/advanced#be-the-pr-wrangler-for-a-week)
for more details.
+8 -7
View File
@@ -32,13 +32,14 @@ The Kubernetes documentation is written in Markdown and processed and deployed u
You can file issues, edit content, and review changes from others, all from the
GitHub website. You can also use GitHub's embedded history and search tools.
Not all tasks can be done in the GitHub UI, but these are discussed in the
Not all tasks can be done in the GitHub UI, but these are discussed in the
[intermediate](/docs/contribute/intermediate/) and
[advanced](/docs/contribute/advanced/) docs contribution guides.
### Participating in SIG Docs
The Kubernetes documentation is maintained by a Special Interest Group (SIG)
The Kubernetes documentation is maintained by a
{{< glossary_tooltip text="Special Interest Group" term_id="sig" >}} (SIG)
called SIG Docs. We communicate using a Slack channel, a mailing list, and
weekly video meetings. New participants are welcome. For more information, see
[Participating in SIG Docs](/docs/contribute/participating/).
@@ -68,9 +69,9 @@ We make use of the standard Hugo shortcodes, as well as a few that are custom to
the Kubernetes documentation. See [Custom Hugo shortcodes](/docs/contribute/style/hugo-shortcodes/) for
information about how to use them.
### Multiple languages
### Multiple languages
Documentation source is available in multiple languages in `/content/`. Each language has its own folder with a two-letter code determined by the [ISO 639-1 standard](https://www.loc.gov/standards/iso639-2/php/code_list.php). For example, English documentation source is stored in `/content/en/docs/`.
Documentation source is available in multiple languages in `/content/`. Each language has its own folder with a two-letter code determined by the [ISO 639-1 standard](https://www.loc.gov/standards/iso639-2/php/code_list.php). For example, English documentation source is stored in `/content/en/docs/`.
For more information about contributing to documentation in multiple languages, see ["Localize content"](/docs/contribute/intermediate#localize-content) in the intermediate contributing guide.
@@ -158,7 +159,7 @@ upcoming Kubernetes release, your process is a bit different. See
process guidelines and information about deadlines.
{{< /note >}}
### Sign the CLA
### Sign the CNCF CLA {#sign-the-cla}
Before you can contribute code or documentation to Kubernetes, you **must** read
the [Contributor guide](https://github.com/kubernetes/community/blob/master/contributors/guide/README.md) and
@@ -299,7 +300,7 @@ conventions and etiquette and get used to the workflow.
PR, and if applicable, also look at any linked issues. If you see a problem
or room for improvement, hover over the line and click the `+` symbol that
appears.
You can type a comment, and either choose **Add single comment** or **Start
a review**. Typically, starting a review is better because it allows you to
leave multiple comments and notifies the PR owner only when you have
@@ -342,7 +343,7 @@ or follow the steps below.
Case studies highlight how organizations are using Kubernetes to solve
real-world problems. They are written in collaboration with the Kubernetes
marketing team, which is handled by the CNCF.
marketing team, which is handled by the {{< glossary_tooltip text="CNCF" term_id="cncf" >}}.
Have a look at the source for the
[existing case studies](https://github.com/kubernetes/website/tree/master/content/en/case-studies).
@@ -29,7 +29,14 @@ and representing feature state.
## Language
Kubernetes documentation uses US English.
Kubernetes documentation has been translated into multiple languages
(see [Localization READMEs](https://github.com/kubernetes/website/blob/master/README.md#localization-readmemds)).
The way of localizing the docs for a different language is described in [Localizing Kubernetes Documentation](https://kubernetes.io/docs/contribute/localization/).
The English-language documentation uses U.S. English spelling and grammar.
{{< comment >}}[If you're localizing this page, you can omit the point about US English.]{{< /comment >}}
## Documentation formatting standards
@@ -209,7 +216,7 @@ Hugo [Shortcodes](https://gohugo.io/content-management/shortcodes) help create d
```
{{</* note */>}}
No need to include a prefix; the shortcode automatically provides on (Note:, Caution:, etc.).
No need to include a prefix; the shortcode automatically provides one. (Note:, Caution:, etc.)
{{</* /note */>}}
```
+1 -1
View File
@@ -20,11 +20,11 @@ This section of the Kubernetes documentation contains references.
* [Kubernetes API Overview](/docs/reference/using-api/api-overview/) - Overview of the API for Kubernetes.
* Kubernetes API Versions
* [1.15](/docs/reference/generated/kubernetes-api/v1.15/)
* [1.14](/docs/reference/generated/kubernetes-api/v1.14/)
* [1.13](/docs/reference/generated/kubernetes-api/v1.13/)
* [1.12](/docs/reference/generated/kubernetes-api/v1.12/)
* [1.11](/docs/reference/generated/kubernetes-api/v1.11/)
* [1.10](/docs/reference/generated/kubernetes-api/v1.10/)
## API Client Libraries
@@ -249,6 +249,22 @@ plugins:
...
```
Alternatively, you can embed the configuration directly in the file:
```yaml
apiVersion: apiserver.k8s.io/v1alpha1
kind: AdmissionConfiguration
plugins:
- name: ImagePolicyWebhook
configuration:
imagePolicy:
kubeConfigFile: /path/to/file
allowTTL: 50
denyTTL: 50
retryBackoff: 500
defaultAllow: true
```
The ImagePolicyWebhook config file must reference a [kubeconfig](/docs/concepts/cluster-administration/authenticate-across-clusters-kubeconfig/) formatted file which sets up the connection to the backend. It is required that the backend communicate over TLS.
The kubeconfig file's cluster field must point to the remote service, and the user field must contain the returned authorizer.
@@ -292,15 +308,15 @@ An example request body:
"image":"myrepo/myimage@sha256:beb6bd6a68f114c1dc2ea4b28db81bdf91de202a9014972bec5e4d9171d90ed"
}
],
"annotations":[
"annotations":{
"mycluster.image-policy.k8s.io/ticket-1234": "break-glass"
],
},
"namespace":"mynamespace"
}
}
```
The remote service is expected to fill the ImageReviewStatus field of the request and respond to either allow or disallow access. The response body's "spec" field is ignored and may be omitted. A permissive response would return:
The remote service is expected to fill the `ImageReviewStatus` field of the request and respond to either allow or disallow access. The response body's "spec" field is ignored and may be omitted. A permissive response would return:
```json
{
@@ -337,7 +353,7 @@ Examples of information you might put here are:
* a ticket number from a ticket system that documents the break-glass request
* provide a hint to the policy server as to the imageID of the image being provided, to save it a lookup
In any case, the annotations are provided by the user and are not validated by Kubernetes in any way. In the future, if an annotation is determined to be widely useful, it may be promoted to a named field of ImageReviewSpec.
In any case, the annotations are provided by the user and are not validated by Kubernetes in any way. In the future, if an annotation is determined to be widely useful, it may be promoted to a named field of `ImageReviewSpec`.
### LimitPodHardAntiAffinityTopology {#limitpodhardantiaffinitytopology}
@@ -47,14 +47,21 @@ Kubernetes reviews only the following API request attributes:
* **extra** - A map of arbitrary string keys to string values, provided by the authentication layer.
* **API** - Indicates whether the request is for an API resource.
* **Request path** - Path to miscellaneous non-resource endpoints like `/api` or `/healthz`.
* **API request verb** - API verbs `get`, `list`, `create`, `update`, `patch`, `watch`, `proxy`, `redirect`, `delete`, and `deletecollection` are used for resource requests. To determine the request verb for a resource API endpoint, see [Determine the request verb](/docs/reference/access-authn-authz/authorization/#determine-the-request-verb).
* **HTTP request verb** - HTTP verbs `get`, `post`, `put`, and `delete` are used for non-resource requests.
* **API request verb** - API verbs like `get`, `list`, `create`, `update`, `patch`, `watch`, `delete`, and `deletecollection` are used for resource requests. To determine the request verb for a resource API endpoint, see [Determine the request verb](/docs/reference/access-authn-authz/authorization/#determine-the-request-verb).
* **HTTP request verb** - Lowercased HTTP methods like `get`, `post`, `put`, and `delete` are used for non-resource requests.
* **Resource** - The ID or name of the resource that is being accessed (for resource requests only) -- For resource requests using `get`, `update`, `patch`, and `delete` verbs, you must provide the resource name.
* **Subresource** - The subresource that is being accessed (for resource requests only).
* **Namespace** - The namespace of the object that is being accessed (for namespaced resource requests only).
* **API group** - The API group being accessed (for resource requests only). An empty string designates the [core API group](/docs/concepts/overview/kubernetes-api/).
## Determine the Request Verb
**Non-resource requests**
Requests to endpoints other than `/api/v1/...` or `/apis/<group>/<version>/...`
are considered "non-resource requests", and use the lower-cased HTTP method of the request as the verb.
For example, a `GET` request to endpoints like `/api` or `/healthz` would use `get` as the verb.
**Resource requests**
To determine the request verb for a resource API endpoint, review the HTTP verb
used and whether or not the request acts on an individual resource or a
collection of resources:
@@ -62,20 +69,25 @@ collection of resources:
HTTP verb | request verb
----------|---------------
POST | create
GET, HEAD | get (for individual resources), list (for collections)
GET, HEAD | get (for individual resources), list (for collections, including full object content), watch (for watching an individual resource or collection of resources)
PUT | update
PATCH | patch
DELETE | delete (for individual resources), deletecollection (for collections)
Kubernetes sometimes checks authorization for additional permissions using specialized verbs. For example:
* [PodSecurityPolicy](/docs/concepts/policy/pod-security-policy/) checks for authorization of the `use` verb on `podsecuritypolicies` resources in the `policy` API group.
* [RBAC](/docs/reference/access-authn-authz/rbac/#privilege-escalation-prevention-and-bootstrapping) checks for authorization
of the `bind` verb on `roles` and `clusterroles` resources in the `rbac.authorization.k8s.io` API group.
* [Authentication](/docs/reference/access-authn-authz/authentication/) layer checks for authorization of the `impersonate` verb on `users`, `groups`, and `serviceaccounts` in the core API group, and the `userextras` in the `authentication.k8s.io` API group.
* [PodSecurityPolicy](/docs/concepts/policy/pod-security-policy/)
* `use` verb on `podsecuritypolicies` resources in the `policy` API group.
* [RBAC](/docs/reference/access-authn-authz/rbac/#privilege-escalation-prevention-and-bootstrapping)
* `bind` and `escalate` verbs on `roles` and `clusterroles` resources in the `rbac.authorization.k8s.io` API group.
* [Authentication](/docs/reference/access-authn-authz/authentication/)
* `impersonate` verb on `users`, `groups`, and `serviceaccounts` in the core API group, and the `userextras` in the `authentication.k8s.io` API group.
## Authorization Modules
* **Node** - A special-purpose authorizer that grants permissions to kubelets based on the pods they are scheduled to run. To learn more about using the Node authorization mode, see [Node Authorization](/docs/reference/access-authn-authz/node/).
## Authorization Modes {#authorization-modules}
The Kubernetes API server may authorize a request using one of several authorization modes:
* **Node** - A special-purpose authorization mode that grants permissions to kubelets based on the pods they are scheduled to run. To learn more about using the Node authorization mode, see [Node Authorization](/docs/reference/access-authn-authz/node/).
* **ABAC** - Attribute-based access control (ABAC) defines an access control paradigm whereby access rights are granted to users through the use of policies which combine attributes together. The policies can use any type of attributes (user attributes, resource attributes, object, environment attributes, etc). To learn more about using the ABAC mode, see [ABAC Mode](/docs/reference/access-authn-authz/abac/).
* **RBAC** - Role-based access control (RBAC) is a method of regulating access to computer or network resources based on the roles of individual users within an enterprise. In this context, access is the ability of an individual user to perform a specific task, such as view, create, or modify a file. To learn more about using the RBAC mode, see [RBAC Mode](/docs/reference/access-authn-authz/rbac/)
* When specified RBAC (Role-Based Access Control) uses the `rbac.authorization.k8s.io` API group to drive authorization decisions, allowing admins to dynamically configure permission policies through the Kubernetes API.
@@ -1,6 +1,5 @@
---
reviewers:
- bgrant0607
- erictune
- lavalamp
title: Controlling Access to the Kubernetes API
@@ -690,7 +690,7 @@ or the server could power off before persisting the object.
Additionally, webhooks with side effects should skip those side-effects when `dryRun: true` admission requests are handled.
A webhook must explicitly indicate that it will not have side-effects when run with `dryRun`,
or the dry-run request will not be sent to the webhook and the API request fill fail instead.
or the dry-run request will not be sent to the webhook and the API request will fail instead.
Webhooks indicate whether they have side effects using the `sideEffects` field in the webhook configuration.
`sideEffects` may be set to `Unknown`, `None`, `Some`, `NoneOnDryRun`. The default is `Unknown`.
@@ -752,12 +752,13 @@ and mutating webhooks can specify a `reinvocationPolicy` to control whether they
* `Never`: the webhook must not be called more than once in a single admission evaluation
* `IfNeeded`: the webhook may be called again as part of the admission evaluation if the object
being admitted is modified by other admission plugins after the initial webhook call.
{{< note >}}
* the number of additional invocations is not guaranteed to be exactly one.
* if additional invocations result in further modifications to the object, webhooks are not guaranteed to be invoked again.
* webhooks that use this option may be reordered to minimize the number of additional invocations.
* to validate an object after all mutations are guaranteed complete, use a validating admission webhook instead (recommended for webhooks with side-effects).
{{< /note >}}
The important elements to note are:
* The number of additional invocations is not guaranteed to be exactly one.
* If additional invocations result in further modifications to the object, webhooks are not guaranteed to be invoked again.
* Webhooks that use this option may be reordered to minimize the number of additional invocations.
* To validate an object after all mutations are guaranteed complete, use a validating admission webhook instead (recommended for webhooks with side-effects).
Here is an example of a mutating webhook opting into being re-invoked if later admission plugins modify the object:
@@ -778,7 +779,7 @@ in an object could already exist in the user-provided object, but it is essentia
### Failure policy
`failurePolicy` defines how unrecognized errors and timeout errors from the admission webhook
are handled. Allowed values are `Ignore` or `Fail`. Defaults to `Ignore` in v1beta1.
are handled. Allowed values are `Ignore` or `Fail`. Defaults to `Ignore` in v1beta1, and v1 has changed the default to `Fail`.
* `Ignore` means that an error calling the webhook is ignored and the API request is allowed to continue.
* `Fail` means that an error calling the webhook causes the admission to fail and the API request to be rejected.
@@ -72,12 +72,16 @@ different Kubernetes components.
| `CSIPersistentVolume` | `true` | GA | 1.13 | - |
| `CustomCPUCFSQuotaPeriod` | `false` | Alpha | 1.12 | |
| `CustomPodDNS` | `false` | Alpha | 1.9 | 1.9 |
| `CustomPodDNS` | `true` | Beta| 1.10 | |
| `CustomPodDNS` | `true` | Beta| 1.10 | 1.13 |
| `CustomPodDNS` | `true` | GA | 1.14 | - |
| `CustomResourcePublishOpenAPI` | `false` | Alpha| 1.14 | 1.14 |
| `CustomResourcePublishOpenAPI` | `true` | Beta| 1.15 | |
| `CustomResourceSubresources` | `false` | Alpha | 1.10 | 1.11 |
| `CustomResourceSubresources` | `true` | Beta | 1.11 | - |
| `CustomResourceValidation` | `false` | Alpha | 1.8 | 1.8 |
| `CustomResourceValidation` | `true` | Beta | 1.9 | |
| `CustomResourceWebhookConversion` | `false` | Alpha | 1.13 | |
| `CustomResourceWebhookConversion` | `false` | Alpha | 1.13 | 1.14 |
| `CustomResourceWebhookConversion` | `true` | Beta | 1.15 | |
| `DebugContainers` | `false` | Alpha | 1.10 | |
| `DevicePlugins` | `false` | Alpha | 1.8 | 1.9 |
| `DevicePlugins` | `true` | Beta | 1.10 | |
@@ -89,8 +93,9 @@ different Kubernetes components.
| `DynamicVolumeProvisioning` | `true` | Alpha | 1.3 | 1.7 |
| `DynamicVolumeProvisioning` | `true` | GA | 1.8 | |
| `EnableEquivalenceClassCache` | `false` | Alpha | 1.8 | |
| `ExpandCSIVolumes` | `false` | Alpha | 1.14 | | |
| `ExpandInUsePersistentVolumes` | `false` | Alpha | 1.11 | 1.13 | |
| `ExpandCSIVolumes` | `false` | Alpha | 1.14 | |
| `ExpandInUsePersistentVolumes` | `false` | Alpha | 1.11 | 1.14 |
| `ExpandInUsePersistentVolumes` | `true` | Beta | 1.15 | |
| `ExpandPersistentVolumes` | `false` | Alpha | 1.8 | 1.10 |
| `ExpandPersistentVolumes` | `true` | Beta | 1.11 | |
| `ExperimentalCriticalPodAnnotation` | `false` | Alpha | 1.5 | |
@@ -107,7 +112,8 @@ different Kubernetes components.
| `KubeletPluginsWatcher` | `false` | Alpha | 1.11 | 1.11 |
| `KubeletPluginsWatcher` | `true` | Beta | 1.12 | 1.12 |
| `KubeletPluginsWatcher` | `true` | GA | 1.13 | - |
| `KubeletPodResources` | `false` | Alpha | 1.13 | |
| `KubeletPodResources` | `false` | Alpha | 1.13 | 1.14 |
| `KubeletPodResources` | `true` | Beta | 1.15 | |
| `LocalStorageCapacityIsolation` | `false` | Alpha | 1.7 | 1.9 |
| `LocalStorageCapacityIsolation` | `true` | Beta| 1.10 | |
| `LocalStorageCapacityIsolationFSQuotaMonitoring` | `false` | Alpha| 1.15 | |
@@ -117,19 +123,23 @@ different Kubernetes components.
| `MountPropagation` | `true` | GA | 1.12 | |
| `NodeLease` | `false` | Alpha | 1.12 | 1.13 |
| `NodeLease` | `true` | Beta | 1.14 | |
| `NonPreemptingPriority` | `false` | Alpha | 1.15 | |
| `PersistentLocalVolumes` | `false` | Alpha | 1.7 | 1.9 |
| `PersistentLocalVolumes` | `true` | Beta | 1.10 | 1.13 |
| `PersistentLocalVolumes` | `true` | GA | 1.14 | |
| `PodPriority` | `false` | Alpha | 1.8 | 1.10 |
| `PodPriority` | `true` | Beta | 1.11 | 1.13 |
| `PodPriority` | `true` | GA | 1.14 | |
| `PodReadinessGates` | `false` | Alpha | 1.11 | |
| `PodReadinessGates` | `true` | Beta | 1.12 | |
| `PodReadinessGates` | `false` | Alpha | 1.11 | 1.11 |
| `PodReadinessGates` | `true` | Beta | 1.12 | 1.13 |
| `PodReadinessGates` | `true` | GA | 1.14 | - |
| `PodShareProcessNamespace` | `false` | Alpha | 1.10 | |
| `PodShareProcessNamespace` | `true` | Beta | 1.12 | |
| `ProcMountType` | `false` | Alpha | 1.12 | |
| `PVCProtection` | `false` | Alpha | 1.9 | 1.9 |
| `RemainingItemCount` | `false` | Alpha | 1.15 | |
| `ResourceLimitsPriorityFunction` | `false` | Alpha | 1.9 | |
| `RequestManagement` | `false` | Alpha | 1.15 | |
| `ResourceQuotaScopeSelectors` | `false` | Alpha | 1.11 | 1.11 |
| `ResourceQuotaScopeSelectors` | `true` | Beta | 1.12 | |
| `RotateKubeletClientCertificate` | `true` | Beta | 1.8 | |
@@ -139,14 +149,19 @@ different Kubernetes components.
| `RuntimeClass` | `true` | Beta | 1.14 | |
| `SCTPSupport` | `false` | Alpha | 1.12 | |
| `ServerSideApply` | `false` | Alpha | 1.14 | |
| `ServiceLoadBalancerFinalizer` | `false` | Alpha | 1.15 | |
| `ServiceNodeExclusion` | `false` | Alpha | 1.8 | |
| `StorageObjectInUseProtection` | `true` | Beta | 1.10 | 1.10 |
| `StorageObjectInUseProtection` | `true` | GA | 1.11 | |
| `StorageVersionHash` | `false` | Alpha | 1.14 | 1.14 |
| `StorageVersionHash` | `true` | Beta | 1.15 | |
| `StreamingProxyRedirects` | `true` | Beta | 1.5 | |
| `SupportIPVSProxyMode` | `false` | Alpha | 1.8 | 1.8 |
| `SupportIPVSProxyMode` | `false` | Beta | 1.9 | 1.9 |
| `SupportIPVSProxyMode` | `true` | Beta | 1.10 | 1.10 |
| `SupportIPVSProxyMode` | `true` | GA | 1.11 | |
| `SupportNodePidsLimit` | `false` | Alpha | 1.14 | 1.14 |
| `SupportNodePidsLimit` | `true` | Beta | 1.15 | |
| `SupportPodPidsLimit` | `false` | Alpha | 1.10 | 1.13 |
| `SupportPodPidsLimit` | `true` | Beta | 1.14 | |
| `Sysctls` | `true` | Beta | 1.11 | |
@@ -159,6 +174,7 @@ different Kubernetes components.
| `TokenRequestProjection` | `false` | Alpha | 1.11 | 1.11 |
| `TokenRequestProjection` | `true` | Beta | 1.12 | |
| `TTLAfterFinished` | `false` | Alpha | 1.12 | |
| `VolumePVCDataSource` | `false` | Alpha | 1.15 | |
| `VolumeScheduling` | `false` | Alpha | 1.9 | 1.9 |
| `VolumeScheduling` | `true` | Beta | 1.10 | 1.12 |
| `VolumeScheduling` | `true` | GA | 1.13 | |
@@ -167,6 +183,7 @@ different Kubernetes components.
| `VolumeSnapshotDataSource` | `false` | Alpha | 1.12 | - |
| `ScheduleDaemonSetPods` | `false` | Alpha | 1.11 | 1.11 |
| `ScheduleDaemonSetPods` | `true` | Beta | 1.12 | |
| `WatchBookmark` | `false` | Alpha | 1.15 | |
| `WindowsGMSA` | `false` | Alpha | 1.14 | |
## Using a Feature
@@ -233,6 +250,7 @@ Each feature gate is designed for enabling/disabling a specific feature:
- `CRIContainerLogRotation`: Enable container log rotation for cri container runtime.
- `CSIBlockVolume`: Enable external CSI volume drivers to support block storage. See the [`csi` raw block volume support](/docs/concepts/storage/volumes/#csi-raw-block-volume-support) documentation for more details.
- `CSIDriverRegistry`: Enable all logic related to the CSIDriver API object in csi.storage.k8s.io.
- `CSIInlineVolume`: Enable CSI Inline volumes support for pods.
- `CSIMigration`: Enables shims and translation logic to route volume operations from in-tree plugins to corresponding pre-installed CSI plugins
- `CSIMigrationAWS`: Enables shims and translation logic to route volume operations from the AWS-EBS in-tree plugin to EBS CSI plugin
- `CSIMigrationAzureDisk`: Enables shims and translation logic to route volume operations from the Azure-Disk in-tree plugin to Azure Disk CSI plugin
@@ -248,6 +266,7 @@ Each feature gate is designed for enabling/disabling a specific feature:
- `CustomPodDNS`: Enable customizing the DNS settings for a Pod using its `dnsConfig` property.
Check [Pod's DNS Config](/docs/concepts/services-networking/dns-pod-service/#pods-dns-config)
for more details.
- `CustomResourcePublishOpenAPI`: Enables publishing of CRD OpenAPI specs.
- `CustomResourceSubresources`: Enable `/status` and `/scale` subresources
on resources created from [CustomResourceDefinition](/docs/concepts/api-extension/custom-resources/).
- `CustomResourceValidation`: Enable schema based validation on resources created from
@@ -288,6 +307,7 @@ Each feature gate is designed for enabling/disabling a specific feature:
- `MountPropagation`: Enable sharing volume mounted by one container to other containers or pods.
For more details, please see [mount propagation](/docs/concepts/storage/volumes/#mount-propagation).
- `NodeLease`: Enable the new Lease API to report node heartbeats, which could be used as a node health signal.
- `NonPreemptingPriority`: Enable NonPreempting option for PriorityClass and Pod.
- `PersistentLocalVolumes`: Enable the usage of `local` volume type in Pods.
Pod affinity has to be specified if requesting a `local` volume.
- `PodPriority`: Enable the descheduling and preemption of Pods based on their [priorities](/docs/concepts/configuration/pod-priority-preemption/).
@@ -302,6 +322,7 @@ Each feature gate is designed for enabling/disabling a specific feature:
assigns a lowest possible score of 1 to a node that satisfies at least one of
the input Pod's cpu and memory limits. The intent is to break ties between
nodes with same scores.
- `RequestManagement`: Enable managing request concurrency with prioritization and fairness at each server.
- `ResourceQuotaScopeSelectors`: Enable resource quota scope selectors.
- `RotateKubeletClientCertificate`: Enable the rotation of the client TLS certificate on the kubelet.
See [kubelet configuration](/docs/reference/command-line-tools-reference/kubelet-tls-bootstrapping/#kubelet-configuration) for more details.
@@ -312,10 +333,12 @@ Each feature gate is designed for enabling/disabling a specific feature:
- `ScheduleDaemonSetPods`: Enable DaemonSet Pods to be scheduled by the default scheduler instead of the DaemonSet controller.
- `SCTPSupport`: Enables the usage of SCTP as `protocol` value in `Service`, `Endpoint`, `NetworkPolicy` and `Pod` definitions
- `ServerSideApply`: Enables the [Sever Side Apply (SSA)](/docs/reference/using-api/api-concepts/#server-side-apply) path at the API Server.
- `ServiceLoadBalancerFinalizer`: Enable finalizer protection for Service load balancers.
- `ServiceNodeExclusion`: Enable the exclusion of nodes from load balancers created by a cloud provider.
A node is eligible for exclusion if annotated with "`alpha.service-controller.kubernetes.io/exclude-balancer`" key.
- `StorageObjectInUseProtection`: Postpone the deletion of PersistentVolume or
PersistentVolumeClaim objects if they are still being used.
- `StorageVersionHash`: Allow apiservers to expose the storage version hash in the discovery.
- `StreamingProxyRedirects`: Instructs the API server to intercept (and follow)
redirects from the backend (kubelet) for streaming requests.
Examples of streaming requests include the `exec`, `attach` and `port-forward` requests.
@@ -331,12 +354,14 @@ Each feature gate is designed for enabling/disabling a specific feature:
- `TokenRequestProjection`: Enable the injection of service account tokens into
a Pod through the [`projected` volume](/docs/concepts/storage/volumes/#projected).
- `TTLAfterFinished`: Allow a [TTL controller](/docs/concepts/workloads/controllers/ttlafterfinished/) to clean up resources after they finish execution.
- `VolumePVCDataSource`: Enable support for specifying an existing PVC as a DataSource.
- `VolumeScheduling`: Enable volume topology aware scheduling and make the
PersistentVolumeClaim (PVC) binding aware of scheduling decisions. It also
enables the usage of [`local`](/docs/concepts/storage/volumes/#local) volume
type when used together with the `PersistentLocalVolumes` feature gate.
- `VolumeSnapshotDataSource`: Enable volume snapshot data source support.
- `VolumeSubpathEnvExpansion`: Enable `subPathExpr` field for expanding environment variables into a `subPath`.
- `WatchBookmark`: Enable support for watch bookmark events.
- `WindowsGMSA`: Enables passing of GMSA credential specs from pods to container runtimes.
{{% /capture %}}
@@ -948,7 +948,7 @@ kube-apiserver [flags]
<td colspan="2">--tls-cipher-suites stringSlice</td>
</tr>
<tr>
<td></td><td style="line-height: 130%; word-wrap: break-word;">Comma-separated list of cipher suites for the server. If omitted, the default Go cipher suites will be use. Possible values: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_RC4_128_SHA,TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_RC4_128_SHA,TLS_RSA_WITH_3DES_EDE_CBC_SHA,TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_128_CBC_SHA256,TLS_RSA_WITH_AES_128_GCM_SHA256,TLS_RSA_WITH_AES_256_CBC_SHA,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_RC4_128_SHA</td>
<td></td><td style="line-height: 130%; word-wrap: break-word;">Comma-separated list of cipher suites for the server. If omitted, the default Go cipher suites will be used. Possible values: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_RC4_128_SHA,TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_RC4_128_SHA,TLS_RSA_WITH_3DES_EDE_CBC_SHA,TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_128_CBC_SHA256,TLS_RSA_WITH_AES_128_GCM_SHA256,TLS_RSA_WITH_AES_256_CBC_SHA,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_RC4_128_SHA</td>
</tr>
<tr>
@@ -59,13 +59,6 @@ kubelet [flags]
<td></td><td style="line-height: 130%; word-wrap: break-word;">The IP address for the Kubelet to serve on (set to 0.0.0.0 for all IPv4 interfaces and `::` for all IPv6 interfaces) (default 0.0.0.0)</td>
</tr>
<tr>
<td colspan="2">--allow-privileged</td>
</tr>
<tr>
<td></td><td style="line-height: 130%; word-wrap: break-word;">If true, allow containers to request privileged mode.</td>
</tr>
<tr>
<td colspan="2">--alsologtostderr</td>
</tr>
@@ -0,0 +1,17 @@
---
title: cgroup (control group)
id: cgroup
date: 2019-06-25
full_link:
short_description: >
A group of Linux processes with optional resource isolation, accounting and limits.
aka:
tags:
- fundamental
---
A group of Linux processes with optional resource isolation, accounting and limits.
<!--more-->
cgroup is a Linux kernel feature that limits, accounts for, and isolates the resource usage (CPU, memory, disk I/O, network) for a collection of processes.
@@ -1,19 +1,17 @@
---
title: Cluster
id: cluster
date: 2018-04-12
date: 2019-06-15
full_link:
short_description: >
A set of machines, called nodes, that run containerized applications managed by Kubernetes.
A set of machines, called nodes, that run containerized applications managed by Kubernetes. A cluster has at least one worker node and at least one master node.
aka:
tags:
- fundamental
- operation
---
A set of machines, called nodes, that run containerized applications managed by Kubernetes.
<!--more-->
A cluster has several worker nodes and at least one master node.
A set of machines, called nodes, that run containerized applications managed by Kubernetes. A cluster has at least one worker node and at least one master node.
<!--more-->
The worker node(s) host the pods that are the components of the application. The master node(s) manages the worker nodes and the pods in the cluster. Multiple master nodes are used to provide a cluster with failover and high availability.
+23
View File
@@ -0,0 +1,23 @@
---
title: Cloud Native Computing Foundation (CNCF)
id: cncf
date: 2019-05-26
full_link: https://cncf.io/
short_description: >
Cloud Native Computing Foundation
aka:
tags:
- community
---
The Cloud Native Computing Foundation (CNCF) builds sustainable ecosystems and
fosters a community around [projects](https://www.cncf.io/projects/) that
orchestrate containers as part of a microservices architecture.
Kubernetes is a CNCF project.
<!--more-->
The CNCF is a sub-foundation of [the Linux Foundation](https://www.linuxfoundation.org/).
Its mission is to make cloud native computing ubiquitous.
+1 -1
View File
@@ -10,7 +10,7 @@ aka:
tags:
- tool
---
A tool that lets you use OCI container runtimes witk Kubernetes CRI.
A tool that lets you use OCI container runtimes with Kubernetes CRI.
<!--more-->
@@ -2,21 +2,24 @@
title: kube-proxy
id: kube-proxy
date: 2018-04-12
full_link: /docs/reference/generated/kube-proxy
full_link: /docs/reference/command-line-tools-reference/kube-proxy/
short_description: >
`kube-proxy` is a network proxy that runs on each node in the cluster.
aka:
aka:
tags:
- fundamental
- core-object
- networking
---
`kube-proxy` is a network proxy that runs on each node in the cluster.
[kube-proxy](/docs/reference/command-line-tools-reference/kube-proxy/) is a
network proxy that runs on each node in your cluster, implementing part of
the Kubernetes {{< glossary_tooltip term_id="service">}} concept.
It enables the Kubernetes service abstraction by maintaining network rules on
the host and performing connection forwarding.
<!--more-->
<!--more-->
`kube-proxy` is responsible for request forwarding. `kube-proxy` allows TCP and UDP stream forwarding or round robin TCP and UDP forwarding across a set of backend functions.
kube-proxy maintains network rules on nodes. These network rules allow
network communication to your Pods from network sessions inside or outside
of your cluster.
kube-proxy uses the operating system packet filtering layer if there is one
and it's available. Otherwise, kube-proxy forwards the traffic itself.
+24
View File
@@ -0,0 +1,24 @@
---
title: Operator pattern
id: operator-pattern
date: 2019-05-21
full_link: /docs/concepts/extend-kubernetes/operator/
short_description: >
A specialized controller used to manage a custom resource
aka:
tags:
- architecture
---
The [operator pattern](/docs/concepts/extend-kubernetes/operator/) is a system
design that links a {{< glossary_tooltip term_id="controller" >}} to one or more custom
resources.
<!--more-->
You can extend Kubernetes by adding controllers to your cluster, beyond the built-in
controllers that come as part of Kubernetes itself.
If a running application acts as a controller and has API access to carry out tasks
against a custom resource that's defined in the control plane, that's an example of
the Operator pattern.
@@ -1,7 +1,6 @@
---
title: kubectl Cheat Sheet
reviewers:
- bgrant0607
- erictune
- krousey
- clove
@@ -160,7 +159,7 @@ kubectl get services --sort-by=.metadata.name # List Services Sorted by Name
kubectl get pods --sort-by='.status.containerStatuses[0].restartCount'
# Get the version label of all pods with label app=cassandra
kubectl get pods --selector=app=cassandra rc -o \
kubectl get pods --selector=app=cassandra -o \
jsonpath='{.items[*].metadata.labels.version}'
# Get all worker nodes (use a selector to exclude results that have a label
@@ -349,7 +348,7 @@ Kubectl verbosity is controlled with the `-v` or `--v` flags followed by an inte
Verbosity | Description
--------------| -----------
`--v=0` | Generally useful for this to ALWAYS be visible to an operator.
`--v=0` | Generally useful for this to *always* be visible to a cluster operator.
`--v=1` | A reasonable default log level if you don't want verbosity.
`--v=2` | Useful steady state information about the service and important log messages that may correlate to significant changes in the system. This is the recommended default log level for most systems.
`--v=3` | Extended information about changes.
@@ -1,7 +1,6 @@
---
title: kubectl Usage Conventions
reviewers:
- bgrant0607
- janetkuo
content_template: templates/concept
---
@@ -37,15 +36,19 @@ For `kubectl run` to satisfy infrastructure as code:
You can create the following resources using `kubectl run` with the `--generator` flag:
| Resource | api group | kubectl command |
|---------------------------------|--------------------|---------------------------------------------------|
| Pod | v1 | `kubectl run --generator=run-pod/v1` |
| Replication controller | v1 | `kubectl run --generator=run/v1` |
| Deployment (deprecated) | extensions/v1beta1 | `kubectl run --generator=deployment/v1beta1` |
| Deployment (deprecated) | apps/v1beta1 | `kubectl run --generator=deployment/apps.v1beta1` |
| Job (deprecated) | batch/v1 | `kubectl run --generator=job/v1` |
| CronJob (default) | batch/v1beta1 | `kubectl run --generator=cronjob/v1beta1` |
| CronJob (deprecated) | batch/v2alpha1 | `kubectl run --generator=cronjob/v2alpha1` |
| Resource | api group | kubectl command |
|-------------------------------------|--------------------|---------------------------------------------------|
| Pod | v1 | `kubectl run --generator=run-pod/v1` |
| Replication controller (deprecated) | v1 | `kubectl run --generator=run/v1` |
| Deployment (deprecated) | extensions/v1beta1 | `kubectl run --generator=deployment/v1beta1` |
| Deployment (deprecated) | apps/v1beta1 | `kubectl run --generator=deployment/apps.v1beta1` |
| Job (deprecated) | batch/v1 | `kubectl run --generator=job/v1` |
| CronJob (deprecated) | batch/v1beta1 | `kubectl run --generator=cronjob/v1beta1` |
| CronJob (deprecated) | batch/v2alpha1 | `kubectl run --generator=cronjob/v2alpha1` |
{{< note >}}
`kubectl run --generator` except for `run-pod/v1` is deprecated in v1.12.
{{< /note >}}
If you do not specify a generator flag, other flags prompt you to use a specific generator. The following table lists the flags that force you to use specific generators, depending on the version of the cluster:
@@ -75,4 +78,4 @@ flag, which provides the object to be submitted to the cluster.
* You can use `kubectl apply` to create or update resources. For more information about using kubectl apply to update resources, see [Kubectl Book](https://kubectl.docs.kubernetes.io).
{{% /capture %}}
{{% /capture %}}
@@ -2,7 +2,6 @@
title: kubectl for Docker Users
content_template: templates/concept
reviewers:
- bgrant0607
- brendandburns
- thockin
---
@@ -1,6 +1,5 @@
---
reviewers:
- bgrant0607
- hw-qiaolei
title: Overview of kubectl
content_template: templates/concept
@@ -1,5 +1,5 @@
---
title: v1.14
title: v1.15
---
[Kubernetes API v1.14](/docs/reference/generated/kubernetes-api/v1.14/)
[Kubernetes API v1.15](/docs/reference/generated/kubernetes-api/v1.15/)
@@ -206,55 +206,9 @@ point to the CA certificate and key.
### Managing the kubeadm drop-in file for the kubelet {#kubelet-drop-in}
The kubeadm package ships with configuration for how the kubelet should
be run. Note that the `kubeadm` CLI command never touches this drop-in file.
This drop-in file belongs to the kubeadm deb/rpm package.
The `kubeadm` package ships with a configuration file for running the `kubelet` by `systemd`. Note that the kubeadm CLI never touches this drop-in file. This drop-in file is part of the kubeadm DEB/RPM package.
This is what it looks like:
```
[Service]
Environment="KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf
--kubeconfig=/etc/kubernetes/kubelet.conf"
Environment="KUBELET_CONFIG_ARGS=--config=/var/lib/kubelet/config.yaml"
# This is a file that "kubeadm init" and "kubeadm join" generates at runtime, populating
the KUBELET_KUBEADM_ARGS variable dynamically
EnvironmentFile=-/var/lib/kubelet/kubeadm-flags.env
# This is a file that the user can use for overrides of the kubelet args as a last resort. Preferably,
#the user should use the .NodeRegistration.KubeletExtraArgs object in the configuration files instead.
# KUBELET_EXTRA_ARGS should be sourced from this file.
EnvironmentFile=-/etc/default/kubelet
ExecStart=
ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS
```
Here's a breakdown of what/why:
* `--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf` path to a kubeconfig
file that is used to get client certificates for kubelet during node join.
On success, a kubeconfig file is written to the path specified by `--kubeconfig`.
* `--kubeconfig=/etc/kubernetes/kubelet.conf` points to the kubeconfig file that
tells the kubelet where the API server is. This file also has the kubelet's
credentials.
* `--pod-manifest-path=/etc/kubernetes/manifests` specifies from where to read
static Pod manifests used for starting the control plane.
* `--allow-privileged=true` allows this kubelet to run privileged Pods.
* `--network-plugin=cni` uses CNI networking.
* `--cni-conf-dir=/etc/cni/net.d` specifies where to look for the
[CNI spec file(s)](https://github.com/containernetworking/cni/blob/master/SPEC.md).
* `--cni-bin-dir=/opt/cni/bin` specifies where to look for the actual CNI binaries.
* `--cluster-dns=10.96.0.10` use this cluster-internal DNS server for `nameserver`
entries in Pods' `/etc/resolv.conf`.
* `--cluster-domain=cluster.local` uses this cluster-internal DNS domain for
`search` entries in Pods' `/etc/resolv.conf`.
* `--client-ca-file=/etc/kubernetes/pki/ca.crt` authenticates requests to the Kubelet
API using this CA certificate.
* `--authorization-mode=Webhook` authorizes requests to the Kubelet API by `POST`-ing
a `SubjectAccessReview` to the API server.
* `--rotate-certificates` auto rotate the kubelet client certificates by requesting new
certificates from the `kube-apiserver` when the certificate expiration approaches.
* `--cert-dir`the directory where the TLS certs are located.
For further information, see [Managing the kubeadm drop-in file for systemd](/docs/setup/production-environment/tools/kubeadm/kubelet-integration/#the-kubelet-drop-in-file-for-systemd).
### Use kubeadm with CRI runtimes
@@ -131,7 +131,7 @@ using one of the other modes if possible.
**Example `kubeadm join` command:**
```shell
kubeadm join --token abcdef.1234567890abcdef --discovery-token-unsafe-skip-ca-verification 1.2.3.4:6443`
kubeadm join --token abcdef.1234567890abcdef --discovery-token-unsafe-skip-ca-verification 1.2.3.4:6443
```
**Advantages:**
@@ -9,12 +9,16 @@ card:
name: reference
weight: 40
---
<img src="https://raw.githubusercontent.com/cncf/artwork/master/projects/kubernetes/certified-kubernetes/versionless/color/certified-kubernetes-color.png" align="right" width="150px">Kubeadm is a tool built to provide `kubeadm init` and `kubeadm join` as best-practice “fast paths” for creating Kubernetes clusters.
<img src="https://raw.githubusercontent.com/kubernetes/kubeadm/master/logos/stacked/color/kubeadm-stacked-color.png" align="right" width="150px">Kubeadm is a tool built to provide `kubeadm init` and `kubeadm join` as best-practice “fast paths” for creating Kubernetes clusters.
kubeadm performs the actions necessary to get a minimum viable cluster up and running. By design, it cares only about bootstrapping, not about provisioning machines. Likewise, installing various nice-to-have addons, like the Kubernetes Dashboard, monitoring solutions, and cloud-specific addons, is not in scope.
Instead, we expect higher-level and more tailored tooling to be built on top of kubeadm, and ideally, using kubeadm as the basis of all deployments will make it easier to create conformant clusters.
## How to install
To install kubeadm, see the [installation guide](/docs/setup/production-environment/tools/kubeadm/install-kubeadm).
## What's next
* [kubeadm init](/docs/reference/setup-tools/kubeadm/kubeadm-init) to bootstrap a Kubernetes control-plane node
@@ -1,7 +1,6 @@
---
title: Kubernetes API Concepts
reviewers:
- bgrant0607
- smarterclayton
- lavalamp
- liggitt
@@ -1,7 +1,6 @@
---
title: Kubernetes API Overview
reviewers:
- bgrant0607
- erictune
- lavalamp
- jbeda
@@ -48,7 +47,7 @@ The API versioning and software versioning are indirectly related. The [API and
versioning proposal](https://git.k8s.io/community/contributors/design-proposals/release/versioning.md) describes the relationship between API versioning and software versioning.
{{< /note >}}
Different API versions indicate different levels of stability and support. You can find more information about the criteria for each level in the [API Changes documentation](https://git.k8s.io/community/contributors/devel/api_changes.md#alpha-beta-and-stable-versions).
Different API versions indicate different levels of stability and support. You can find more information about the criteria for each level in the [API Changes documentation](https://git.k8s.io/community/contributors/devel/sig-architecture/api_changes.md#alpha-beta-and-stable-versions).
Here's a summary of each level:
@@ -35,6 +35,7 @@ Machinery](https://github.com/kubernetes/community/tree/master/sig-api-machinery
| Java | [github.com/kubernetes-client/java](https://github.com/kubernetes-client/java/) | [browse](https://github.com/kubernetes-client/java#installation)
| dotnet | [github.com/kubernetes-client/csharp](https://github.com/kubernetes-client/csharp) | [browse](https://github.com/kubernetes-client/csharp/tree/master/examples/simple)
| JavaScript | [github.com/kubernetes-client/javascript](https://github.com/kubernetes-client/javascript) | [browse](https://github.com/kubernetes-client/javascript/tree/master/examples)
| Haskell | [github.com/kubernetes-client/haskell](https://github.com/kubernetes-client/haskell) | [browse](https://github.com/kubernetes-client/haskell/tree/master/kubernetes-client/example)
## Community-maintained client libraries
+6 -3
View File
@@ -41,8 +41,8 @@ If you're learning Kubernetes, use the Docker-based solutions: tools supported b
|Community |Ecosystem |
| ------------ | -------- |
| [Minikube](/docs/setup/learning-environment/minikube/) | [CDK on LXD](https://www.ubuntu.com/kubernetes/docs/install-local) |
| [Kubeadm-dind](https://github.com/kubernetes-sigs/kubeadm-dind-cluster) | [Docker Desktop](https://www.docker.com/products/docker-desktop)|
| [Kubernetes IN Docker](https://github.com/kubernetes-sigs/kind) | [Minishift](https://docs.okd.io/latest/minishift/)|
| [kind (Kubernetes IN Docker)](https://github.com/kubernetes-sigs/kind) | [Docker Desktop](https://www.docker.com/products/docker-desktop)|
| | [Minishift](https://docs.okd.io/latest/minishift/)|
| | [MicroK8s](https://microk8s.io/)|
| | [IBM Cloud Private-CE (Community Edition)](https://github.com/IBM/deploy-ibm-cloud-private) |
| | [IBM Cloud Private-CE (Community Edition) on Linux Containers](https://github.com/HSBawa/icp-ce-on-linux-containers)|
@@ -70,6 +70,7 @@ The following production environment solutions table lists the providers and the
| [Amazon](https://aws.amazon.com) | [Amazon EKS](https://aws.amazon.com/eks/) |[Amazon EC2](https://aws.amazon.com/ec2/) | | | |
| [AppsCode](https://appscode.com/products/pharmer/) | &#x2714; | | | | |
| [APPUiO](https://appuio.ch/)  | &#x2714; | &#x2714; | &#x2714; | | | |
| [Banzai Cloud Pipeline Kubernetes Engine (PKE)](https://banzaicloud.com/products/pke/) | | &#x2714; | | &#x2714; | &#x2714; | &#x2714; |
| [CenturyLink Cloud](https://www.ctl.io/) | | &#x2714; | | | |
| [Cisco Container Platform](https://cisco.com/go/containers) | | | &#x2714; | | |
| [Cloud Foundry Container Runtime (CFCR)](https://docs-cfcr.cfapps.io/) | | | | &#x2714; |&#x2714; |
@@ -81,10 +82,11 @@ The following production environment solutions table lists the providers and the
| [Docker Enterprise](https://www.docker.com/products/docker-enterprise) | |&#x2714; | &#x2714; | | | &#x2714;
| [Fedora (Multi Node)](https://kubernetes.io/docs/getting-started-guides/fedora/flannel_multi_node_cluster/)  | | | | | &#x2714; | &#x2714;
| [Fedora (Single Node)](https://kubernetes.io/docs/getting-started-guides/fedora/fedora_manual_config/)  | | | | | | &#x2714;
| [Gardner](https://gardener.cloud/) | |&#x2714; | | &#x2714; | |
| [Gardener](https://gardener.cloud/) | |&#x2714; | | &#x2714; | |
| [Giant Swarm](https://giantswarm.io/) | &#x2714; | &#x2714; | &#x2714; | |
| [Google](https://cloud.google.com/) | [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine/) | [Google Compute Engine (GCE)](https://cloud.google.com/compute/)|[GKE On-Prem](https://cloud.google.com/gke-on-prem/) | | | | | | | |
| [IBM](https://www.ibm.com/in-en/cloud) | [IBM Cloud Kubernetes Service](https://cloud.ibm.com/kubernetes/catalog/cluster)| |[IBM Cloud Private](https://www.ibm.com/in-en/cloud/private) | |
| [Ionos](https://www.ionos.com/enterprise-cloud) | [Ionos Managed Kubernetes](https://www.ionos.com/enterprise-cloud/managed-kubernetes) | [Ionos Enterprise Cloud](https://www.ionos.com/enterprise-cloud) | |
| [Kontena Pharos](https://www.kontena.io/pharos/) | |&#x2714;| &#x2714; | | |
| [Kubermatic](https://www.loodse.com/) | &#x2714; | &#x2714; | &#x2714; | | |
| [KubeSail](https://kubesail.com/) | &#x2714; | | | | |
@@ -104,6 +106,7 @@ The following production environment solutions table lists the providers and the
| [Supergiant](https://supergiant.io/) | |&#x2714; | | | |
| [SUSE](https://www.suse.com/) | | &#x2714; | | | |
| [SysEleven](https://www.syseleven.io/) | &#x2714; | | | | |
| [Tencent Cloud](https://intl.cloud.tencent.com/) | [Tencent Kubernetes Engine](https://intl.cloud.tencent.com/product/tke) | &#x2714; | &#x2714; | | | &#x2714; |
| [VEXXHOST](https://vexxhost.com/) | &#x2714; | &#x2714; | | | |
| [VMware](https://cloud.vmware.com/) | [VMware Cloud PKS](https://cloud.vmware.com/vmware-cloud-pks) |[VMware Enterprise PKS](https://cloud.vmware.com/vmware-enterprise-pks) | [VMware Enterprise PKS](https://cloud.vmware.com/vmware-enterprise-pks) | [VMware Essential PKS](https://cloud.vmware.com/vmware-essential-pks) | |[VMware Essential PKS](https://cloud.vmware.com/vmware-essential-pks)
@@ -455,10 +455,8 @@ export no_proxy=$no_proxy,$(minikube ip)
```
## Known Issues
* Features that require a Cloud Provider will not work in Minikube. These include:
* LoadBalancers
* Features that require multiple nodes. These include:
* Advanced scheduling policies
Features that require multiple nodes will not work in Minikube.
## Design
@@ -29,7 +29,7 @@ KRIB features:
## Creating a cluster
Review [Digital Rebar documentation](https://https://provision.readthedocs.io/en/tip/README.html) for details about installing the platform.
Review [Digital Rebar documentation](https://provision.readthedocs.io/en/tip/README.html) for details about installing the platform.
The Digital Rebar Provision Golang binary should be installed on a Linux-like system with 16 GB of RAM or larger (Packet.net Tiny and Rasberry Pi are also acceptable).
@@ -21,10 +21,15 @@ The `extraArgs` field consist of `key: value` pairs. To override a flag for a co
1. Add the appropriate fields to your configuration.
2. Add the flags to override to the field.
3. Run `kubeadm init` with `--config <YOUR CONFIG YAML>`.
For more details on each field in the configuration you can navigate to our
[API reference pages](https://godoc.org/k8s.io/kubernetes/cmd/kubeadm/app/apis/kubeadm/v1beta2#ClusterConfiguration).
{{< note >}}
You can generate a `ClusterConfiguration` object with default values by running `kubeadm config print init-defaults` and saving the output to a file of your choice.
{{< /note >}}
{{% /capture %}}
{{% capture body %}}
@@ -38,12 +43,10 @@ Example usage:
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
kubernetesVersion: v1.13.0
metadata:
name: 1.13-sample
apiServer:
extraArgs:
advertise-address: 192.168.0.103
anonymous-auth: false
anonymous-auth: "false"
enable-admission-plugins: AlwaysPullImages,DefaultStorageClass
audit-log-path: /home/johndoe/audit.log
```
@@ -57,13 +60,11 @@ Example usage:
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
kubernetesVersion: v1.13.0
metadata:
name: 1.13-sample
controllerManager:
extraArgs:
cluster-signing-key-file: /home/johndoe/keys/ca.key
bind-address: 0.0.0.0
deployment-controller-sync-period: 50
deployment-controller-sync-period: "50"
```
## Scheduler flags
@@ -75,8 +76,6 @@ Example usage:
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
kubernetesVersion: v1.13.0
metadata:
name: 1.13-sample
scheduler:
extraArgs:
address: 0.0.0.0
@@ -8,7 +8,7 @@ weight: 30
{{% capture overview %}}
<img src="https://raw.githubusercontent.com/cncf/artwork/master/projects/kubernetes/certified-kubernetes/versionless/color/certified-kubernetes-color.png" align="right" width="150px">**kubeadm** helps you bootstrap a minimum viable Kubernetes cluster that conforms to best practices. With kubeadm, your cluster should pass [Kubernetes Conformance tests](https://kubernetes.io/blog/2017/10/software-conformance-certification). Kubeadm also supports other cluster
<img src="https://raw.githubusercontent.com/kubernetes/kubeadm/master/logos/stacked/color/kubeadm-stacked-color.png" align="right" width="150px">**kubeadm** helps you bootstrap a minimum viable Kubernetes cluster that conforms to best practices. With kubeadm, your cluster should pass [Kubernetes Conformance tests](https://kubernetes.io/blog/2017/10/software-conformance-certification). Kubeadm also supports other cluster
lifecycle functions, such as upgrades, downgrade, and managing [bootstrap tokens](/docs/reference/access-authn-authz/bootstrap-tokens/).
Because you can install kubeadm on various types of machine (e.g. laptop, server,
@@ -295,7 +295,7 @@ Make sure that your network manifest supports RBAC.
Also, beware, that your Pod network must not overlap with any of the host networks as this can cause issues.
If you find a collision between your network plugins preferred Pod network and some of your host networks, you should think of a suitable CIDR replacement and use that during `kubeadm init` with `--pod-network-cidr` and as a replacement in your network plugins YAML.
You can install a pod network add-on with the following command:
You can install a pod network add-on with the following command on the control-plane node or a node that has the kubeconfig credentials:
```bash
kubectl apply -f <add-on.yaml>
@@ -308,13 +308,19 @@ You can install only one pod network per cluster.
Please select one of the tabs to see installation instructions for the respective third-party Pod Network Provider.
{{% /tab %}}
{{% tab name="AWS VPC" %}}
AWS VPC CNI provides native AWS VPC networking to Kubernetes clusters.
For installation, please refer to the [AWS VPC CNI setup guide](https://github.com/aws/amazon-vpc-cni-k8s#setup).
{{% /tab %}}
{{% tab name="Calico" %}}
For more information about using Calico, see [Quickstart for Calico on Kubernetes](https://docs.projectcalico.org/latest/getting-started/kubernetes/), [Installing Calico for policy and networking](https://docs.projectcalico.org/latest/getting-started/kubernetes/installation/calico), and other related resources.
For Calico to work correctly, you need to pass `--pod-network-cidr=192.168.0.0/16` to `kubeadm init` or update the `calico.yml` file to match your Pod network. Note that Calico works on `amd64`, `arm64`, and `ppc64le` only.
```shell
kubectl apply -f https://docs.projectcalico.org/v3.7/manifests/calico.yaml
kubectl apply -f https://docs.projectcalico.org/v3.8/manifests/calico.yaml
```
{{% /tab %}}
@@ -324,7 +330,7 @@ Canal uses Calico for policy and Flannel for networking. Refer to the Calico doc
For Canal to work correctly, `--pod-network-cidr=10.244.0.0/16` has to be passed to `kubeadm init`. Note that Canal works on `amd64` only.
```shell
kubectl apply -f https://docs.projectcalico.org/v3.7/manifests/canal.yaml
kubectl apply -f https://docs.projectcalico.org/v3.8/manifests/canal.yaml
```
{{% /tab %}}
@@ -361,6 +367,16 @@ cilium-drxkl 1/1 Running 0 18m
```
{{% /tab %}}
{{% tab name="Contiv-VPP" %}}
[Contiv-VPP](https://contivpp.io/) employs a programmable CNF vSwitch based on [FD.io VPP](https://fd.io/),
offering feature-rich & high-performance cloud-native networking and services.
It implements k8s services and network policies in the user space (on VPP).
Please refer to this installation guide: [Contiv-VPP Manual Installation](https://github.com/contiv/vpp/blob/master/docs/setup/MANUAL_INSTALL.md)
{{% /tab %}}
{{% tab name="Flannel" %}}
For `flannel` to work correctly, you must pass `--pod-network-cidr=10.244.0.0/16` to `kubeadm init`.
@@ -384,6 +400,16 @@ For more information about `flannel`, see [the CoreOS flannel repository on GitH
](https://github.com/coreos/flannel).
{{% /tab %}}
{{% tab name="JuniperContrail/TungstenFabric" %}}
Provides overlay SDN solution, delivering multicloud networking, hybrid cloud networking,
simultaneous overlay-underlay support, network policy enforcement, network isolation,
service chaining and flexible load balancing.
There are multiple, flexible ways to install JuniperContrail/TungstenFabric CNI.
Kindly refer to this quickstart: [TungstenFabric](https://tungstenfabric.github.io/website/)
{{% /tab %}}
{{% tab name="Kube-router" %}}
Set `/proc/sys/net/bridge/bridge-nf-call-iptables` to `1` by running `sysctl net.bridge.bridge-nf-call-iptables=1`
to pass bridged IPv4 traffic to iptables' chains. This is a requirement for some CNI plugins to work, for more information
@@ -426,25 +452,6 @@ kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl versio
```
{{% /tab %}}
{{% tab name="JuniperContrail/TungstenFabric" %}}
Provides overlay SDN solution, delivering multicloud networking, hybrid cloud networking,
simultaneous overlay-underlay support, network policy enforcement, network isolation,
service chaining and flexible load balancing.
There are multiple, flexible ways to install JuniperContrail/TungstenFabric CNI.
Kindly refer to this quickstart: [TungstenFabric](https://tungstenfabric.github.io/website/)
{{% /tab %}}
{{% tab name="Contiv-VPP" %}}
[Contiv-VPP](https://contivpp.io/) employs a programmable CNF vSwitch based on [FD.io VPP](https://fd.io/),
offering feature-rich & high-performance cloud-native networking and services.
It implements k8s services and network policies in the user space (on VPP).
Please refer to this installation guide: [Contiv-VPP Manual Installation](https://github.com/contiv/vpp/blob/master/docs/setup/MANUAL_INSTALL.md)
{{% /tab %}}
{{< /tabs >}}
@@ -30,7 +30,7 @@ kubeadm that run control plane components.
Each control plane node runs an instance of the `kube-apiserver`, `kube-scheduler`, and `kube-controller-manager`.
The `kube-apiserver` is exposed to worker nodes using a load balancer.
Each control plane node creates a local etcd member and this etcd member communicate only with
Each control plane node creates a local etcd member and this etcd member communicates only with
the `kube-apiserver` of this node. The same applies to the local `kube-controller-manager`
and `kube-scheduler` instances.
@@ -105,10 +105,13 @@ option. Your cluster requirements may need a different configuration.
```sh
sudo kubeadm init --control-plane-endpoint "LOAD_BALANCER_DNS:LOAD_BALANCER_PORT" --upload-certs
```
<<<<<<< HEAD
- You can use the `--kubernetes-version` flag to set the Kubernetes version to use.
It is recommended that the versions of kubeadm, kubelet, kubectl and Kubernetes match.
- The `--control-plane-endpoint` flag should be set to the address or DNS and port of the load balancer.
=======
>>>>>>> upstream/master
- The `--upload-certs` flag is used to upload the certificates that should be shared
across all the control-plane instances to the cluster. If instead, you prefer to copy certs across
control-plane nodes manually or using automation tools, please remove this flag and refer to [Manual
@@ -10,7 +10,7 @@ card:
{{% capture overview %}}
<img src="https://raw.githubusercontent.com/cncf/artwork/master/projects/kubernetes/certified-kubernetes/versionless/color/certified-kubernetes-color.png" align="right" width="150px">This page shows how to install the `kubeadm` toolbox.
<img src="https://raw.githubusercontent.com/kubernetes/kubeadm/master/logos/stacked/color/kubeadm-stacked-color.png" align="right" width="150px">This page shows how to install the `kubeadm` toolbox.
For information how to create a cluster with kubeadm once you have performed this installation process, see the [Using kubeadm to Create a Cluster](/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/) page.
{{% /capture %}}
@@ -169,7 +169,6 @@ enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
exclude=kube*
EOF
# Set SELinux in permissive mode (effectively disabling it)
@@ -138,7 +138,7 @@ If the reload and restart are successful, the normal `kubeadm init` workflow con
### Workflow when using `kubeadm join`
When you run `kubeadm join`, kubeadm uses the Bootstrap Token credential perform
When you run `kubeadm join`, kubeadm uses the Bootstrap Token credential to perform
a TLS bootstrap, which fetches the credential needed to download the
`kubelet-config-1.X` ConfigMap and writes it to `/var/lib/kubelet/config.yaml`. The dynamic
environment file is generated in exactly the same way as `kubeadm init`.
@@ -16,13 +16,18 @@ manager, and scheduler run as [DaemonSet pods](/docs/concepts/workloads/controll
configured via the Kubernetes API instead of [static pods](/docs/tasks/administer-cluster/static-pod/)
configured in the kubelet via static files.
To create a self-hosted cluster see the `kubeadm alpha selfhosting pivot` command.
To create a self-hosted cluster see the
[kubeadm alpha selfhosting pivot](/docs/reference/setup-tools/kubeadm/kubeadm-alpha/#cmd-selfhosting) command.
{{% /capture %}}
{{% capture body %}}
#### Caveats
{{< caution >}}
This feature pivots your cluster into an unsupported state, rendering kubeadm unable
to manage you cluster any longer. This includes `kubeadm upagrade`.
to manage you cluster any longer. This includes `kubeadm upgrade`.
{{< /caution >}}
1. Self-hosting in 1.8 and later has some important limitations. In particular, a
@@ -30,7 +35,7 @@ to manage you cluster any longer. This includes `kubeadm upagrade`.
without manual intervention.
1. By default, self-hosted control plane Pods rely on credentials loaded from
[`hostPath`](https://kubernetes.io/docs/concepts/storage/volumes/#hostpath)
[`hostPath`](/docs/concepts/storage/volumes/#hostpath)
volumes. Except for initial creation, these credentials are not managed by
kubeadm.
@@ -61,3 +66,5 @@ In summary, `kubeadm alpha selfhosting` works as follows:
1. When the original static control plane stops, the new self-hosted control
plane is able to bind to listening ports and become active.
{{% /capture %}}
@@ -53,7 +53,8 @@ this example.
cat << EOF > /etc/systemd/system/kubelet.service.d/20-etcd-service-manager.conf
[Service]
ExecStart=
ExecStart=/usr/bin/kubelet --address=127.0.0.1 --pod-manifest-path=/etc/kubernetes/manifests --allow-privileged=true
# Replace "systemd" with the cgroup driver of your container runtime. The default value in the kubelet is "cgroupfs".
ExecStart=/usr/bin/kubelet --address=127.0.0.1 --pod-manifest-path=/etc/kubernetes/manifests --cgroup-driver=systemd
Restart=always
EOF
@@ -0,0 +1,19 @@
---
title: Running Kubernetes on Tencent Kubernetes Engine
---
## Tencent Kubernetes Engine
[Tencent Cloud Tencent Kubernetes Engine (TKE)](https://intl.cloud.tencent.com/product/tke) provides native Kubernetes container management services. You can deploy and manage a Kubernetes cluster with TKE in just a few steps. For detailed directions, see [Deploy Tencent Kubernetes Engine](https://intl.cloud.tencent.com/document/product/457/11741).
TKE is a [Certified Kubernetes product](https://www.cncf.io/certification/software-conformance/).It is fully compatible with the native Kubernetes API.
## Custom Deployment
The core of Tencent Kubernetes Engine is open source and available [on GitHub](https://github.com/TencentCloud/tencentcloud-cloud-controller-manager/).
When using TKE to create a Kubernetes cluster, you can choose managed mode or independent deployment mode. In addition, you can customize the deployment as needed; for example, you can choose an existing Cloud Virtual Machine instance for cluster creation or enable Kube-proxy in IPVS mode.
## What's Next
To learn more, see the [TKE documentation](https://intl.cloud.tencent.com/document/product/457).
@@ -235,7 +235,6 @@ The following networking functionality is not supported on Windows nodes
* Local NodePort access from the node itself fails (works for other nodes or external clients)
* Accessing service VIPs from nodes will be available with a future release of Windows Server
* Overlay networking support in kube-proxy is an alpha release. In addition, it requires [KB4482887](https://support.microsoft.com/en-us/help/4482887/windows-10-update-kb4482887) to be installed on Windows Server 2019
* `kubectl port-forward`
* Local Traffic Policy and DSR mode
* Windows containers connected to l2bridge, l2tunnel, or overlay networks do not support communicating over the IPv6 stack. There is outstanding Windows platform work required to enable these network drivers to consume IPv6 addresses and subsequent Kubernetes work in kubelet, kube-proxy, and CNI plugins.
* Outbound communication using the ICMP protocol via the win-overlay, win-bridge, and Azure-CNI plugin. Specifically, the Windows data plane ([VFP](https://www.microsoft.com/en-us/research/project/azure-virtual-filtering-platform/)) doesn't support ICMP packet transpositions. This means:
@@ -244,6 +243,10 @@ The following networking functionality is not supported on Windows nodes
* ICMP packets directed to pass through a remote network (e.g. pod to external internet communication via ping) cannot be transposed and thus will not be routed back to their source
* Since TCP/UDP packets can still be transposed, one can substitute `ping <destination>` with `curl <destination>` to be able to debug connectivity to the outside world.
These features were added in Kubernetes v1.15:
* `kubectl port-forward`
##### CNI Plugins
* Windows reference network plugins win-bridge and win-overlay do not currently implement [CNI spec](https://github.com/containernetworking/cni/blob/master/SPEC.md) v0.4.0 due to missing "CHECK" implementation.
@@ -386,10 +389,10 @@ Your main source of help for troubleshooting your Kubernetes cluster should star
nssm start flanneld
# Register kubelet.exe
# Microsoft releases the pause infrastructure container at mcr.microsoft.com/k8s/core/pause:1.0.0
# Microsoft releases the pause infrastructure container at mcr.microsoft.com/k8s/core/pause:1.2.0
# For more info search for "pause" in the "Guide for adding Windows Nodes in Kubernetes"
nssm install kubelet C:\k\kubelet.exe
nssm set kubelet AppParameters --hostname-override=<hostname> --v=6 --pod-infra-container-image=mcr.microsoft.com/k8s/core/pause:1.0.0 --resolv-conf="" --allow-privileged=true --enable-debugging-handlers --cluster-dns=<DNS-service-IP> --cluster-domain=cluster.local --kubeconfig=c:\k\config --hairpin-mode=promiscuous-bridge --image-pull-progress-deadline=20m --cgroups-per-qos=false --log-dir=<log directory> --logtostderr=false --enforce-node-allocatable="" --network-plugin=cni --cni-bin-dir=c:\k\cni --cni-conf-dir=c:\k\cni\config
nssm set kubelet AppParameters --hostname-override=<hostname> --v=6 --pod-infra-container-image=mcr.microsoft.com/k8s/core/pause:1.2.0 --resolv-conf="" --allow-privileged=true --enable-debugging-handlers --cluster-dns=<DNS-service-IP> --cluster-domain=cluster.local --kubeconfig=c:\k\config --hairpin-mode=promiscuous-bridge --image-pull-progress-deadline=20m --cgroups-per-qos=false --log-dir=<log directory> --logtostderr=false --enforce-node-allocatable="" --network-plugin=cni --cni-bin-dir=c:\k\cni --cni-conf-dir=c:\k\cni\config
nssm set kubelet AppDirectory C:\k
nssm start kubelet
@@ -499,12 +502,17 @@ Your main source of help for troubleshooting your Kubernetes cluster should star
Check that your pause image is compatible with your OS version. The [instructions](https://docs.microsoft.com/en-us/virtualization/windowscontainers/kubernetes/deploying-resources) assume that both the OS and the containers are version 1803. If you have a later version of Windows, such as an Insider build, you need to adjust the images accordingly. Please refer to the Microsoft's [Docker repository](https://hub.docker.com/u/microsoft/) for images. Regardless, both the pause image Dockerfile and the sample service expect the image to be tagged as :latest.
Starting with Kubernetes v1.14, Microsoft releases the pause infrastructure container at `mcr.microsoft.com/k8s/core/pause:1.0.0`. For more information search for "pause" in the [Guide for adding Windows Nodes in Kubernetes](../user-guide-windows-nodes).
Starting with Kubernetes v1.14, Microsoft releases the pause infrastructure container at `mcr.microsoft.com/k8s/core/pause:1.2.0`. For more information search for "pause" in the [Guide for adding Windows Nodes in Kubernetes](../user-guide-windows-nodes).
1. DNS resolution is not properly working
Check the DNS limitations for Windows in this [section](#dns-limitations).
1. `kubectl port-forward` fails with "unable to do port forwarding: wincat not found"
This was implemented in Kubernetes 1.15, and the pause infrastructure container `mcr.microsoft.com/k8s/core/pause:1.2.0`. Be sure to use these versions or newer ones.
If you would like to build your own pause infrastructure container, be sure to include [wincat](https://github.com/kubernetes-sigs/sig-windows-tools/tree/master/cmd/wincat)
### Further investigation
If these steps don't resolve your problem, you can get help running Windows containers on Windows nodes in Kubernetes through:
@@ -533,7 +541,7 @@ We have a lot of features in our roadmap. An abbreviated high level list is incl
### CRI-ContainerD
ContainerD is another OCI-compliant runtime that recently graduated as a CNCF project. It's currently tested on Linux, but 1.3 will bring support for Windows and Hyper-V. [[reference](https://blog.docker.com/2019/02/containerd-graduates-within-the-cncf/)]
{{< glossary_tooltip term_id="containerd" >}} is another OCI-compliant runtime that recently graduated as a {{< glossary_tooltip text="CNCF" term_id="cncf" >}} project. It's currently tested on Linux, but 1.3 will bring support for Windows and Hyper-V. [[reference](https://blog.docker.com/2019/02/containerd-graduates-within-the-cncf/)]
The CRI-ContainerD interface will be able to manage sandboxes based on Hyper-V. This provides a foundation where RuntimeClass could be implemented for new use cases including:
@@ -22,7 +22,7 @@ The Kubernetes platform can now be used to run both Linux and Windows containers
* Obtain a [Windows Server license](https://www.microsoft.com/en-us/cloud-platform/windows-server-pricing) in order to configure the Windows node that hosts Windows containers. You can use your organization's licenses for the cluster, or acquire one from Microsoft, a reseller, or via the major cloud providers such as GCP, AWS, and Azure by provisioning a virtual machine running Windows Server through their marketplaces. A [time-limited trial](https://www.microsoft.com/en-us/cloud-platform/windows-server-trial) is also available.
* Build a Linux-based Kubernetes cluster in which you have access to the control plane (some examples include [Getting Started from Scratch](https://github.com/kubernetes/kubernetes/tree/master/build/), [kubeadm/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/), [AKS Engine](/docs/setup/production-environment/turnkey/azure/), [GCE](/docs/setup/production-environment/turnkey/gce/), [AWS](/docs/setup/production-environment/turnkey/aws/).
* Build a Linux-based Kubernetes cluster in which you have access to the control plane (some examples include [Creating a single control-plane cluster with kubeadm](/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/), [AKS Engine](/docs/setup/production-environment/turnkey/azure/), [GCE](/docs/setup/production-environment/turnkey/gce/), [AWS](/docs/setup/production-environment/turnkey/aws/).
## Getting Started: Adding a Windows Node to Your Cluster
@@ -189,7 +189,7 @@ All code snippets in Windows sections are to be run in a PowerShell environment
```
{{< note >}}
The "pause" (infrastructure) image is hosted on Microsoft Container Registry (MCR). You can access it using "docker pull mcr.microsoft.com/k8s/core/pause:1.0.0". The DOCKERFILE is available at https://github.com/Microsoft/SDN/blob/master/Kubernetes/windows/Dockerfile.
The "pause" (infrastructure) image is hosted on Microsoft Container Registry (MCR). You can access it using "docker pull mcr.microsoft.com/k8s/core/pause:1.2.0". The DOCKERFILE is available at https://github.com/kubernetes-sigs/sig-windows-tools/tree/master/cmd/wincat.
{{< /note >}}
1. Prepare a Windows directory for Kubernetes
File diff suppressed because it is too large Load Diff
@@ -135,7 +135,7 @@ The following file is an Ingress resource that sends traffic to your Service via
1. Create `example-ingress.yaml` from the following file:
apiVersion: extensions/v1beta1
apiVersion: networking.k8s.io/v1beta1 # for versions before 1.14 use extensions/v1beta1
kind: Ingress
metadata:
name: example-ingress
@@ -160,7 +160,7 @@ The following file is an Ingress resource that sends traffic to your Service via
Output:
```shell
ingress.extensions/example-ingress created
ingress.networking.k8s.io/example-ingress created
```
1. Verify the IP address is set:
@@ -33,7 +33,7 @@ for database debugging.
The output of a successful command verifies that the deployment was created:
deployment.apps/redis-master created
View the pod status to check that it is ready:
kubectl get pods
@@ -92,19 +92,19 @@ for database debugging.
1. `kubectl port-forward` allows using resource name, such as a pod name, to select a matching pod to port forward to since Kubernetes v1.10.
kubectl port-forward redis-master-765d459796-258hz 7000:6379
kubectl port-forward redis-master-765d459796-258hz 7000:6379
which is the same as
kubectl port-forward pods/redis-master-765d459796-258hz 7000:6379
or
or
kubectl port-forward deployment/redis-master 7000:6379
kubectl port-forward deployment/redis-master 7000:6379
or
kubectl port-forward rs/redis-master 7000:6379
kubectl port-forward rs/redis-master 7000:6379
or
@@ -117,7 +117,7 @@ for database debugging.
2. Start the Redis command line interface:
redis-cli
redis-cli -p 7000
3. At the Redis command line prompt, enter the `ping` command:
@@ -33,17 +33,22 @@ provides load balancing for an application that has two running instances.
## Creating a service for an application running in two pods
Here is the configuration file for the application Deployment:
{{< codenew file="service/access/hello-application.yaml" >}}
1. Run a Hello World application in your cluster:
Create the application Deployment using the file above:
```shell
kubectl run hello-world --replicas=2 --labels="run=load-balancer-example" --image=gcr.io/google-samples/node-hello:1.0 --port=8080
```
The preceding command creates a
[Deployment](/docs/concepts/workloads/controllers/deployment/)
object and an associated
[ReplicaSet](/docs/concepts/workloads/controllers/replicaset/)
object. The ReplicaSet has two
[Pods](/docs/concepts/workloads/pods/pod/),
each of which runs the Hello World application.
kubectl apply -f https://k8s.io/examples/service/access/hello-application.yaml
```
The preceding command creates a
[Deployment](/docs/concepts/workloads/controllers/deployment/)
object and an associated
[ReplicaSet](/docs/concepts/workloads/controllers/replicaset/)
object. The ReplicaSet has two
[Pods](/docs/concepts/workloads/pods/pod/),
each of which runs the Hello World application.
1. Display information about the Deployment:
```shell
@@ -30,7 +30,7 @@ Dashboard also provides information on the state of Kubernetes resources in your
The Dashboard UI is not deployed by default. To deploy it, run the following command:
```
kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/master/aio/deploy/recommended/kubernetes-dashboard.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.0-beta1/aio/deploy/recommended.yaml
```
## Accessing the Dashboard UI
@@ -48,7 +48,7 @@ You can access Dashboard using the kubectl command-line tool by running the foll
kubectl proxy
```
Kubectl will make Dashboard available at http://localhost:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/.
Kubectl will make Dashboard available at http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/.
The UI can _only_ be accessed from the machine where the command is executed. See `kubectl proxy --help` for more options.
@@ -34,8 +34,8 @@ level of your CustomResourceDefinitions or advance your API to a new version wit
The CustomResourceDefinition API supports a `versions` field that you can use to
support multiple versions of custom resources that you have developed. Versions
can have different schemas with a conversion webhook to convert custom resources between versions.
Webhook conversions should follow the [Kubernetes API conventions](https://github.com/kubernetes/community/blob/master/contributors/devel/api-conventions.md) wherever applicable.
Specifically, See the [API change documentation](https://github.com/kubernetes/community/blob/master/contributors/devel/api_changes.md) for a set of useful gotchas and suggestions.
Webhook conversions should follow the [Kubernetes API conventions](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md) wherever applicable.
Specifically, See the [API change documentation](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api_changes.md) for a set of useful gotchas and suggestions.
{{< note >}}
Earlier iterations included a `version` field instead of `versions`. The
@@ -205,7 +205,7 @@ A structural schema is an [OpenAPI v3.0 validation schema](/docs/tasks/access-ku
1. specifies a non-empty type (via `type` in OpenAPI) for the root, for each specified field of an object node (via `properties` or `additionalProperties` in OpenAPI) and for each item in an array node (via `items` in OpenAPI), with the exception of:
* a node with `x-kubernetes-int-or-string: true`
* a node with `x-kubernetes-preserve-unknown-fields: true`
2. for each each field in an object and each item in an array which is specified within any of `allOf`, `anyOf`, `oneOf` or `not`, the schema also specifies the field/item outside of those logical junctors (compare example 1 and 2).
2. for each field in an object and each item in an array which is specified within any of `allOf`, `anyOf`, `oneOf` or `not`, the schema also specifies the field/item outside of those logical junctors (compare example 1 and 2).
3. does not set `description`, `type`, `default`, `additionProperties`, `nullable` within an `allOf`, `anyOf`, `oneOf` or `not`, with the exception of the two pattern for `x-kubernetes-int-or-string: true` (see below).
4. if `metadata` is specified, then only restrictions on `metadata.name` and `metadata.generateName` are allowed.
@@ -541,7 +541,7 @@ object. When all finalizers have been executed, the resource is deleted.
The value of `metadata.deletionGracePeriodSeconds` controls the interval between
polling updates.
It is the responsibility of each controller to removes its finalizer from the list.
It is the responsibility of each controller to remove its finalizer from the list.
Kubernetes only finally deletes the object if the list of finalizers is empty,
meaning all finalizers have been executed.
@@ -760,7 +760,7 @@ metadata:
spec:
cronSpec: "5 0 * * *"
image: my-awesome-cron-image
replaces: 1
replicas: 1
```
Note that defaulting happens on the object
@@ -39,9 +39,9 @@ content_template: templates/task
Operating etcd with limited resources is suitable only for testing purposes. For deploying in production, advanced hardware configuration is required. Before deploying etcd in production, see [resource requirement reference documentation](https://github.com/coreos/etcd/blob/master/Documentation/op-guide/hardware.md#example-hardware-configurations).
## Starting Kubernetes API server
## Starting etcd clusters
This section covers starting a Kubernetes API server with an etcd cluster in the deployment.
This section covers starting a single-node and multi-node etcd cluster.
### Single-node etcd cluster
@@ -19,7 +19,7 @@ This page describes the CoreDNS upgrade process and how to install CoreDNS inste
## About CoreDNS
[CoreDNS](https://coredns.io) is a flexible, extensible DNS server that can serve as the Kubernetes cluster DNS.
Like Kubernetes, the CoreDNS project is hosted by the [CNCF.](http://www.cncf.io)
Like Kubernetes, the CoreDNS project is hosted by the {{< glossary_tooltip text="CNCF" term_id="cncf" >}}.
You can use CoreDNS instead of kube-dns in your cluster by replacing kube-dns in an existing
deployment, or by using tools like kubeadm that will deploy and upgrade the cluster for you.
@@ -95,7 +95,7 @@ Use the `kubectl get pods` command to verify that the DNS pod is running.
For CoreDNS:
```shell
kubectl get pods --namespace=kube-system -l k8s-app=coredns
kubectl get pods --namespace=kube-system -l k8s-app=kube-dns
NAME READY STATUS RESTARTS AGE
...
coredns-7b96bf9f76-5hsxb 1/1 Running 0 1h
@@ -122,7 +122,7 @@ Use `kubectl logs` command to see logs for the DNS containers.
For CoreDNS:
```shell
for p in $(kubectl get pods --namespace=kube-system -l k8s-app=coredns -o name); do kubectl logs --namespace=kube-system $p; done
for p in $(kubectl get pods --namespace=kube-system -l k8s-app=kube-dns -o name); do kubectl logs --namespace=kube-system $p; done
```
Here is an example of a healthy CoreDNS log:
@@ -68,7 +68,7 @@ are merged into a single document.
{{< tabs name="k8s_install_kubeadm_first_cp" >}}
{{% tab name="Ubuntu, Debian or HypriotOS" %}}
# replace x in 1.14.x-00 with the latest patch version
apt-mark unhold kubeadm && \
apt-mark unhold kubeadm kubelet && \
apt-get update && apt-get install -y kubeadm=1.14.x-00 && \
apt-mark hold kubeadm
{{% /tab %}}
@@ -272,7 +272,7 @@ without compromising the minimum required capacity for running your workloads.
{{< tabs name="k8s_install_kubeadm_worker_nodes" >}}
{{% tab name="Ubuntu, Debian or HypriotOS" %}}
# replace x in 1.14.x-00 with the latest patch version
apt-mark unhold kubeadm && \
apt-mark unhold kubeadm kubelet && \
apt-get update && apt-get install -y kubeadm=1.14.x-00 && \
apt-mark hold kubeadm
{{% /tab %}}
@@ -56,7 +56,9 @@ The upgrade workflow at high level is the following:
{{% /tab %}}
{{< /tabs >}}
## Upgrade the first control plane node
## Upgrading control plane nodes
### Upgrade the first control plane node
1. On your first control plane node, upgrade kubeadm:
@@ -220,28 +222,9 @@ The upgrade workflow at high level is the following:
Check the [addons](/docs/concepts/cluster-administration/addons/) page to
find your CNI provider and see whether additional upgrade steps are required.
1. Upgrade the kubelet and kubectl on the control plane node:
This step is not required on additional control plane nodes if the CNI provider runs as a DaemonSet.
{{< tabs name="k8s_install_kubelet" >}}
{{% tab name="Ubuntu, Debian or HypriotOS" %}}
# replace x in 1.15.x-00 with the latest patch version
apt-mark unhold kubelet && \
apt-get update && apt-get install -y kubelet=1.15.x-00 kubectl=1.15.x-00 && \
apt-mark hold kubelet
{{% /tab %}}
{{% tab name="CentOS, RHEL or Fedora" %}}
# replace x in 1.15.x-0 with the latest patch version
yum install -y kubelet-1.15.x-0 kubectl-1.15.x-0 --disableexcludes=kubernetes
{{% /tab %}}
{{< /tabs >}}
1. Restart the kubelet
```shell
sudo systemctl restart kubelet
```
## Upgrade additional control plane nodes
### Upgrade additional control plane nodes
1. Same as the first control plane node but use:
@@ -257,6 +240,29 @@ sudo kubeadm upgrade apply
Also `sudo kubeadm upgrade plan` is not needed.
### Upgrade kubelet and kubectl
1. Upgrade the kubelet and kubectl on all control plane nodes:
{{< tabs name="k8s_install_kubelet" >}}
{{% tab name="Ubuntu, Debian or HypriotOS" %}}
# replace x in 1.15.x-00 with the latest patch version
apt-mark unhold kubelet kubectl && \
apt-get update && apt-get install -y kubelet=1.15.x-00 kubectl=1.15.x-00 && \
apt-mark hold kubelet kubectl
{{% /tab %}}
{{% tab name="CentOS, RHEL or Fedora" %}}
# replace x in 1.15.x-0 with the latest patch version
yum install -y kubelet-1.15.x-0 kubectl-1.15.x-0 --disableexcludes=kubernetes
{{% /tab %}}
{{< /tabs >}}
1. Restart the kubelet
```shell
sudo systemctl restart kubelet
```
## Upgrade worker nodes
The upgrade procedure on worker nodes should be executed one node at a time or few nodes at a time,
@@ -279,7 +285,7 @@ without compromising the minimum required capacity for running your workloads.
{{% /tab %}}
{{< /tabs >}}
### Cordon the node
### Drain the node
1. Prepare the node for maintenance by marking it unschedulable and evicting the workloads. Run:
@@ -287,16 +293,12 @@ without compromising the minimum required capacity for running your workloads.
kubectl drain $NODE --ignore-daemonsets
```
You should see output similar to this:
You should see output similar to this:
```shell
kubectl drain ip-172-31-85-18
node "ip-172-31-85-18" cordoned
error: unable to drain node "ip-172-31-85-18", aborting command...
There are pending nodes to be drained:
ip-172-31-85-18
error: DaemonSet-managed pods (use --ignore-daemonsets to ignore): calico-node-5798d, kube-proxy-thjp9
node/ip-172-31-85-18 cordoned
WARNING: ignoring DaemonSet-managed Pods: kube-system/kube-proxy-dj7d7, kube-system/weave-net-z65qx
node/ip-172-31-85-18 drained
```
### Upgrade the kubelet configuration
@@ -309,7 +311,7 @@ without compromising the minimum required capacity for running your workloads.
### Upgrade kubelet and kubectl
1. Upgrade the Kubernetes package version by running the Linux package manager for your distribution:
1. Upgrade the kubelet and kubectl on all worker nodes:
{{< tabs name="k8s_kubelet_and_kubectl" >}}
{{% tab name="Ubuntu, Debian or HypriotOS" %}}
@@ -225,7 +225,7 @@ At this point, all requests we make to the Kubernetes cluster from the command l
Let's create some contents.
```shell
kubectl run snowflake --image=kubernetes/serve_hostname --replicas=2
kubectl run snowflake --image=k8s.gcr.io/serve_hostname --replicas=2
```
We have just created a deployment whose replica size is 2 that is running the pod called `snowflake` with a basic container that just serves the hostname.
Note that `kubectl run` creates deployments only on Kubernetes cluster >= v1.2. If you are running older versions, it creates replication controllers instead.
@@ -266,7 +266,7 @@ kubectl get pods
Production likes to run cattle, so let's create some cattle pods.
```shell
kubectl run cattle --image=kubernetes/serve_hostname --replicas=5
kubectl run cattle --image=k8s.gcr.io/serve_hostname --replicas=5
kubectl get deployment
```

Some files were not shown because too many files have changed in this diff Show More