whentimeslows/website
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Resolving merge conflicts with master

Browse Source
This commit is contained in:
zacharysarah
2017-11-30 18:53:33 -06:00
parent 53f460ad9e a49d85186a
commit 538dd33ff2
5068 changed files with 25074 additions and 855452 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/concepts/abstractions/controllers/my-repset.yaml
-17
View File
@@ -1,17 +0,0 @@
apiVersion: extensions/v1beta1
kind: ReplicaSet
metadata:
name: my-repset
spec:
replicas: 3
selector:
matchLabels:
pod-is-for: garbage-collection-example
template:
metadata:
labels:
pod-is-for: garbage-collection-example
spec:
containers:
- name: nginx
image: nginx
docs/concepts/abstractions/nginx-deployment.yaml
-16
View File
@@ -1,16 +0,0 @@
apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: nginx-deployment
spec:
replicas: 3
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.7.9
ports:
- containerPort: 80
docs/concepts/abstractions/pod-termination.md
-16
View File
@@ -1,16 +0,0 @@
---
---
#### <a name="pod-termination"></a> Pod Termination
Since Pods represent processes running on your cluster, Kubernetes provides for *graceful termination* when Pods are no longer needed. Kubernetes implements graceful termination by applying a default *grace period* of 30 seconds from the time that you issue a termination request. A typical Pod termination in Kubernetes involves the following steps:
1. You send a command or API call to terminate the Pod.
1. Kubernetes updates the Pod status to reflect the time after which the Pod is to be considered "dead" (the time of the termination request plus the grace period).
1. Kubernetes marks the Pod state as "Terminating" and stops sending traffic to the Pod.
1. Kubernetes sends a `TERM` signal to the Pod, indicating that the Pod should shut down.
1. When the grace period expires, Kubernetes issues a `SIGKILL` to any processes still running in the Pod.
1. Kubernetes removes the Pod from the API server on the Kubernetes Master.
> **Note:** The grace period is configurable; you can set your own grace period when interacting with the cluster to request termination, such as using the `kubectl delete` command. See the [Terminating a Pod]() tutorial for more information.
{: .note}
docs/concepts/api-extension/custom-resources.md
+173 -26
View File
@@ -6,16 +6,14 @@ approvers:
---
{% capture overview %}
This page explains the concept of *custom resources*, which are extensions of the Kubernetes API.
This page explains [*custom resources*](/docs/concepts/api-extension/custom-resources/), which are extensions of the Kubernetes API. This page explains when to add a custom resource to your Kubernetes cluster and when to use a standalone service. It describes the two methods for adding custom resources and how to choose between them.
{% endcapture %}
{% capture body %}
## Custom resources
A *resource* is an endpoint in the [Kubernetes API](/docs/reference/api-overview/) that stores a
collection of [API objects](/docs/concepts/overview/working-with-objects/kubernetes-objects/) of a
certain kind.
For example, the built-in *pods* resource contains a collection of Pod objects.
A *resource* is an endpoint in the [Kubernetes API](/docs/reference/api-overview/) that stores a collection of [API objects](/docs/concepts/overview/working-with-objects/kubernetes-objects/) of a certain kind. For example, the built-in *pods* resource contains a collection of Pod objects.
A *custom resource* is an extension of the Kubernetes API that is not necessarily available on every
Kubernetes cluster.
@@ -26,34 +24,95 @@ and cluster admins can update custom resources independently of the cluster itse
Once a custom resource is installed, users can create and access its objects with
[kubectl](/docs/user-guide/kubectl-overview/), just as they do for built-in resources like *pods*.
## Custom controllers
### Custom controllers
On their own, custom resources simply let you store and retrieve structured data.
It is only when combined with a *controller* that they become a true
[declarative API](/docs/concepts/overview/working-with-objects/kubernetes-objects/#understanding-kubernetes-objects).
The controller interprets the structured data as a record of the user's desired state,
and continually takes action to achieve and maintain that state.
It is only when combined with a *controller* that they become a true [declarative API](/docs/concepts/overview/working-with-objects/kubernetes-objects/#understanding-kubernetes-objects). The controller interprets the structured data as a record of the user's desired state, and continually takes action to achieve and maintain that state.
A *custom controller* is a controller that users can deploy and update on a running cluster,
independently of the cluster's own lifecycle.
Custom controllers can work with any kind of resource, but they are especially effective when
combined with custom resources.
The [Operator](https://coreos.com/blog/introducing-operators.html) pattern is one example of such a
combination. It allows developers to encode domain knowledge for specific applications into an
extension of the Kubernetes API.
A *custom controller* is a controller that users can deploy and update on a running cluster, independently of the cluster's own lifecycle. Custom controllers can work with any kind of resource, but they are especially effective when combined with custom resources. The [Operator](https://coreos.com/blog/introducing-operators.html) pattern is one example of such a combination. It allows developers to encode domain knowledge for specific applications into an extension of the Kubernetes API.
### Should I add a custom resource to my Kubernetes Cluster?
When creating a new API, consider whether to [aggregate your API with the Kubernetes cluster APIs](/docs/concepts/api-extension/apiserver-aggregation/) or let your API stand alone.
| Consider API aggregation if: | Prefer a stand-alone API if: |
|-|-|
| Your API is [Declarative](#declarative-apis). | Your API does not fit the [Declarative](#declarative-apis) model. |
| You want your new types to be readable and writable using `kubectl`.| `kubectl` support is not required |
| You want to view your new types in a Kubernetes UI, such as dashboard, alongside built-in types. | Kuberetes UI support is not required. |
| You are developing a new API. | You already have program that serves your API and works well. |
| You are willing to accept the format restriction that Kubernetes puts on REST resource paths, such as API Groups and Namespaces. (See the [API Overview](/docs/concepts/overview/kubernetes-api/).) | You need to have specific REST paths to be compatible with an already defined REST API. |
| Your resources are naturally scoped to a cluster or to namespaces of a cluster. | Cluster or namespace scoped resources are a poor fit; you need control over the specifics of resource paths. |
| You want to reuse [Kubernetes API support features](#common-features). | You don't need those features. |
#### Declarative APIs
In a Declarative API, typically:
- your API consists of a relatively small number of relatively small objects (resources).
- the objects define configuration of applications or infrastructure
- the objects are updated relatively infrequently
- humans often need to read and write the objects
- the main operations on the objects are CRUD-y (creating, reading, updating and deleting)
- transactions across objects are not required: the API represents a desired state, not an exact state.
Imperative APIs are not declarative.
Signs that your API might not be declarative include:
- the client says "do this", and then gets a synchornous response back when it is done.
- the client says "do this", and then gets an operation ID back, and has to check a separate Operation objects to determine completion of the request.
- you talk about Remote Procedure Calls (RPCs)
- directly stoing large amounts of data (e.g. > a few kB per object, or >1000s of objects)
- high bandwidth access (10s of requests per second sustained) needed
- store end-user data (such as images, PII, etc) or other large-scale data processed by applications
- the natural operations on the objects are not CRUD-y.
- the API is not easily modeled as objects.
- you chose to represent pending operations with an operation ID or operation object.
### Should I use a configMap or a custom resource?
Use a ConfigMap if any of the following apply:
* There is an existing, well-documented config file format, such as a `mysql.cnf` or `pom.xml`.
* You want to put the entire config file into one key of a configMap.
* The main use of the config file is for a program running in a Pod on your cluster to consume the file to configure itself.
* Consumers of the file prefer to consume via file in a Pod or environment variable in a pod, rather than the Kubernetes API.
* You want to perform rolling updates via Deployment, etc, when the file is updated.
**Note:** Use a [secret](/docs/concepts/configuration/secret/) for sensitive data, which is similar to a configMap but more secure.
{: .note}
Use a custom resource (CRD or Aggregated API) if most of the following apply:
* You want to use Kubernetes client libraries and CLIs to create and update the new resource.
* You want top-level support from kubectl (for example: `kubectl get my-object object-name`).
* You want to build new automation that watches for updates on the new object, and then CRUD other objects, or vice versa.
* You want to write automation that handles updates to the object.
* You want to use Kubernetes API conventions like `.spec`, `.status`, and `.metadata`.
* You want the object to be an abstraction over a collection of controlled resources, or a summarization of other resources.
## Adding custom resources
Kubernetes provides two ways to add custom resources to your cluster:
- [Custom Resource Definitions](/docs/concepts/api-extension/custom-resources/) (CRDs) are easier to use: they do not require any programming in some cases.
- [API Aggregation](/docs/concepts/api-extension/apiserver-aggregation/) requires programming, but allows more control over API behaviors like how data is stored and conversion between API versions.
Kubernetes provides these two options to meet the needs of different users, so that neither ease of use nor flexibility are compromised.
Aggregated APIs are subordinate APIServers that sit behind the primary API server, which acts as a proxy. This arrangement is called [API Aggregation](docs/concepts/api-extension/apiserver-aggregation.md) (AA). To users, it simply appears that the Kubernetes API is extended.
Custom Resource Definitions (CRDS) allow users to create new types of resources without adding another APIserver. You do not need to understand API Aggregation to use CRDs.
Regardless of whether they are installed via CRDs or AA, the new resources are called Custom Resources to distinguish them from built-in Kubernetes resources (like pods)
## CustomResourceDefinitions
[CustomResourceDefinition](/docs/tasks/access-kubernetes-api/extend-api-custom-resource-definitions/)
(CRD) is a built-in API that offers a simple way to create custom resources.
Deploying a CRD into the cluster causes the Kubernetes API server to begin serving the specified
custom resource on your behalf.
The [CustomResourceDefinition](/docs/tasks/access-kubernetes-api/extend-api-custom-resource-definitions/) (CRD) API resource allows you to define custom resources. Defining a CRD object creates an new custom resource with a name and schema that you specify. The Kubernetes API serves and handles the storage of your custom resource.
This frees you from writing your own API server to handle the custom resource,
but the generic nature of the implementation means you have less flexibility than with
[API server aggregation](#api-server-aggregation).
Refer to the [Custom Resource Example](https://github.com/kubernetes/kubernetes/tree/master/staging/src/k8s.io/apiextensions-apiserver/examples/client-go)
Refer to the [Custom Controler example, which uses Custom Resources](https://github.com/kubernetes/sample-controller)
for a demonstration of how to register a new custom resource, work with instances of your new resource type,
and setup a controller to handle events.
@@ -62,16 +121,104 @@ and setup a controller to handle events.
## API server aggregation
Usually, each resource in the Kubernetes API requires code that handles REST requests and manages
persistent storage of objects.
The main Kubernetes API server handles built-in resources like *pods* and *services*,
and can also handle custom resources in a generic way through [CustomResourceDefinitions](#customresourcedefinitions).
Usually, each resource in the Kubernetes API requires code that handles REST requests and manages persistent storage of objects. The main Kubernetes API server handles built-in resources like *pods* and *services*, and can also handle custom resources in a generic way through [CustomResourceDefinitions](#customresourcedefinitions).
The [aggregation layer](/docs/concepts/api-extension/apiserver-aggregation/) allows you to provide specialized
implementations for your custom resources by writing and deploying your own standalone API server.
The main API server delegates requests to you for the custom resources that you handle,
making them available to all of its clients.
### Choosing a method for adding custom resources
CRDs are easier to use. Aggregated APIs are more flexible. Choose the method that best meets your needs.
Typically, CRDs are a good fit if:
* You are have a handful of fields
* You are using the resource within your company, or as part of a small open-source project (as opposed to a commercial product)
#### Comparing ease of use
CRDs are easier to create than Aggregated APIs.
| Custom Resource Definitions | Aggregated API |
|-|-|
| Do not require programming. Users can choose any language for a CRD controller. | Requires programming in Go and building binary and image. Users can choose any language for a CRD controller. |
| No additional service to run; CRs are handled by API Server. | An additional service to create and that could fail. |
| No ongoing support once the CRD is created. Any bug fixes are picked up as part of normal Kubernetes Master upgrades. | May need to periodically pickup bug fixes from upstream and rebuild and update the Aggregated APIserver. |
| No need to handle multiple versions of your API. For example: when you control the client for this resource, you can upgrade it in sync with the API. | You need to handle multiple versions of your API, for example: when developing an extension to share with the world. |
### Advanced features and flexibility
Aggregated APIs offer more advanced API features and customization of other features, for example: the storage layer.
| Feature | Description | CRDs | Aggregated API |
|-|-|-|-|
| Validation | Help users prevent errors and allow you to evolve your API independently of your clients. These features are most useful when there are many clients who can't all update at the same time. | Alpha feature of CRDs in v1.8. Checks limited to what is supported by OpenAPI v3.0. | Yes, arbitrary validation checks |
| Defaulting | See above | No, but can achieve the same effect with an Initializer (requires programming) | Yes |
| Multi-versioning | Allows serving the same object through two API versions. Can help ease API changes like renaming fields. Less important if you control your client versions. | No | Yes |
| Custom Storage | If you need storage with a different performance mode (for example, time-series database instead of key-value store) or isolation for security (for example, encryption secrets or different | No | Yes |
| Custom Business Logic | Perform arbitrary checks or actions when creating, reading, updating or deleting an object | No, but can get some of the same effects with Initializers or Finalizers (requires programming) | Yes |
| Subresources | <ul><li>Add extra operations other than CRUD, such as "scale" or "exec"</li><li>Allows systems like like HorizontalPodAutoscaler and PodDisruptionBudget interact with your new resource</li><li>Finer-grained access control: user writes spec section, controller writes status section.</li><li>Allows incrementing object Generation on custom resource data mutation (requires separate spec and status sections in the resource)</li></ul> | No but planned | Yes, any Subresource |
| strategic-merge-patch | The new endpoints support PATCH with `Content-Type: application/strategic-merge-patch+json`. Useful for updating objects that may be modified both locally, and by the server. For more information, see ["Update API Objects in Place Using kubectl patch"](/docs/tasks/run-application/update-api-object-kubectl-patch/) | No | Yes |
| Protocol Buffers | The new resource supports clients that want to use Protocol Buffers | No | Yes |
| OpenAPI Schema | Is there an OpenAPI (swagger) schema for the types that can be dynamically fetched from the server? Is the user protected from misspelling field names by ensuring only allowed fields are set? Are types enforced (in other words, don't put an `int` in a `string` field?) | No but planned | Yes |
#### Common Features
When you create a custom resource, either via a CRDs or an AA, you get many features for your API, compared to implementing it outside the Kubernetes platform:
| Feature | What it does |
|-|-|
| CRUD | The new endpoints support CRUD basic operations via HTTP and `kubectl` |
| Watch | The new endpoints support Kubernetes Watch operations via HTTP |
| Discovery | Clients like kubectl and dashboard automatically offer list, display, and field edit operations on your resources |
| json-patch | The new endpoints support PATCH with `Content-Type: application/json-patch+json` |
| merge-patch | The new endpoints support PATCH with `Content-Type: application/merge-patch+json` |
| HTTPS | The new endpoints uses HTTPS |
| Built-in Authentication | Access to the extension uses the core apiserver (aggregation layer) for authentication |
| Built-in Authorization | Access the the extension can reuse the authorization used by the core apiserver (e.g. RBAC) |
| Finalizers | Block deletion of extension resources until external cleanup happens. |
| Admission Webhooks | Set default values and validate extension resources during any create/update/delete operation. |
| UI/CLI Display | Kubectl, dashboard can display extension resources. |
| Unset vs Empty | Clients can distinguish unset fields from zero-valued fields. |
| Client Libraries Generation | Kubernetes provides generic clent libraries, as well as tools to generate type-specific client libraries. |
| Labels and annotations | Common metadata across objects that tools know how to edit for core and custom resources. |
## Preparing to install a custom resource
There are several points to be aware of before adding a custom resource to your cluster.
### Third party code and new points of failure
While creating a CRD does not automatically add any new points of failure (for example, by causing third party code to run on your API server), packages (for example, Charts) or other installation bundles often include CRDs as well as a Deployment of third-party code that implements the business logic for a new custom resource.
Installing an Aggregated APIserver always involves running a new Deployment.
### Storage
Custom resources consume storage space in the same way that ConfigMaps do. Creating too many custom resources may overload your API server's storage space.
Aggregated API servers may use the same storage as the main API server, in which case the same warning applies.
### Authentication, authorization, and auditing
CRDs always use the same authentication, authorization, and audit logging as the built-in resources of your API Server.
If you use RBAC for authorization, most RBAC roles will not grant access to the new resources (except the cluster-admin role or any role created with wildcard rules). You'll need to explicitly grant access to the new resources. CRDs and Aggregated APIs often come bundled with new role definitions for the types they add.
Aggregated API servers may or may not use the same authentication, authorization, and auditing as the primary API server.
## Accessing a custom resource
Kubernetes [client libraries](/docs/reference/client-libraries/) can be used to access custom resources. Not all client libraries support custom resources. The go and python client libraries do.
When you add a custom resource, you can access it using:
- kubectl
- the kubernetes dynamic client
- a REST client that you write
- a client generated using Kubernetes client generation tools (generating one is an advanced undertaking, but some projects may provide a client along with the CRD or AA).
{% endcapture %}
{% capture whatsnext %}
docs/concepts/architecture/cloud-controller.md
+251
View File
@@ -0,0 +1,251 @@
---
title: Concepts Underlying the Cloud Controller Manager
---
## Cloud Controller Manager
The cloud controller manager (CCM) concept (not to be confused with the binary) was originally created to allow cloud specific vendor code and the Kubernetes core to evolve independent of one another. The cloud controller manager runs alongside other master components such as the Kubernetes controller manager, the API server, and scheduler. It can also be started as a Kubernetes addon, in which case, it runs on top of Kubernetes.
The cloud controller manager's design is based on a plugin mechanism that allows new cloud providers to integrate with Kubernetes easily by using plugins. There are plans in place for on-boarding new cloud providers on Kubernetes, and for migrating cloud provider from the old model to the new CCM model.
This document discusses the concepts behind the the cloud controller manager, and gives details about its associated functions.
Here's the architecture of a Kubernetes cluster without the cloud controller manager:
![Pre CCM Kube Arch](/images/docs/pre-ccm-arch.png)
## Design
In the preceding diagram, Kubernetes and the cloud provider are integrated through several different components:
* Kubelet
* Kubernetes controller manager
* Kubernetes API server
The CCM consolidates all of the cloud-dependent logic from the preceding three components to create a single point of integration with the cloud. The new architecture with the CCM looks like this:
![CCM Kube Arch](/images/docs/post-ccm-arch.png)
## Components of the CCM
The CCM breaks away some of the functionality of Kubernetes controller manager (KCM) and runs it as a separate process. Specifically, it breaks away those controllers in the KCM that are cloud dependent. The KCM has the following cloud dependent controller loops:
* Node controller
* Volume controller
* Route controller
* Service controller
In version 1.8, the CCM currently runs the following controllers from the preceding list:
* Node controller
* Route controller
* Service controller
Additionally, it runs another controller called the PersistentVolumeLabels controller. This controller is responsible for setting the zone and region labels on PersistentVolumes created in GCP and AWS clouds.
**Note:** Volume controller was deliberately chosen to not be a part of CCM. Due to the complexity involved and due to the existing efforts to abstract away vendor specific volume logic, it was decided that volume controller will not be moved to CCM.
{: .note}
The original plan to support volumes using CCM was to use Flex volumes to support pluggable volumes. However, a competing effort known as CSI is being planned to replace Flex.
Considering these dynamics, we decided to have an intermediate stop gap measure until CSI becomes ready.
Work is in progress by the cloud provider working group (wg-cloud-provider) to enable PersistentVolume support using CCM. See [kubernetes/kubernetes#52371](https://github.com/kubernetes/kubernetes/pull/52371).
## Functions of the CCM
The CCM inherits its functions from components of Kubernetes that are dependent on a cloud provider. This section is structured based on the components from which CCM inherits its functions.
### 1. Kubernetes controller manager
The majority of the CCM's functions are derived from the KCM. As mentioned in the previous section, the CCM runs the following control loops:
* Node controller
* Route controller
* Service controller
* PersistentVolumeLabels controller
#### Node controller
The Node controller is responsible for initializing a node by obtaining information about the nodes running in the cluster from the cloud provider. The node controller performs the following functions:
1. Initialize a node with cloud specific zone/region labels.
2. Initialize a node with cloud specific instance details, for example, type and size.
3. Obtain the node's network addresses and hostname.
4. In case a node becomes unresponsive, check the cloud to see if the node has been deleted from the cloud.
If the node has been deleted from the cloud, delete the Kubernetes Node object.
#### Route controller
The Route controller is responsible for configuring routes in the cloud appropriately so that containers on different nodes in the Kubernetes cluster can communicate with each other. The route controller is only applicable for Google Compute Engine clusters.
#### Service Controller
The Service controller is responsible for listening to service create, update, and delete events. Based on the current state of the services in Kubernetes, it configures cloud load balancers (such as ELB, or Google LB) to reflect the state of the services in Kubernetes. Additionally, it ensures that service backends for cloud load balancers are up to date.
#### PersistentVolumeLabels controller
The PersistentVolumeLabels controller applies labels on AWS EBS, GCE PD volumes when they are created. This removes the need for users to manually set the labels on these volumes.
These labels are essential for the scheduling of pods, as these volumes are constrained to work only within the region/zone that they are in, and therefore any Pod using these volumes needs to be scheduled in the same region/zone.
The PersistentVolumeLabels controller was created specifically for the CCM; that is, it did not exist before the CCM was created. This was done to move the PV labelling logic in the Kubernetes API server (it was an admission controller) to the CCM. It does not run on the KCM.
### 2. Kubelet
The Node controller contains the cloud-dependent functionality of the kubelet. Prior to the introduction of the CCM, the kubelet was responsible for initializing a node with cloud-specific details such as IP addresses, region/zone labels and instance type information. The introduction of the CCM has moved this initialization operation from the kubelet into the CCM.
In this new model, the kubelet initializes a node without cloud-specific information. However, it adds a taint to the newly created node that makes the node unschedulable until the CCM initializes the node with cloud-specific information, and then removes this taint.
### 3. Kubernets API server
The PersistentVolumeLabels controller moves the cloud-dependent functionality of the Kubernetes API server to the CCM as described in the preceding sections.
## Plugin mechanism
The cloud controller manager uses Go interfaces to allow implementations from any cloud to be plugged in. Specifically, it uses the CloudProvider Interface defined [here](https://github.com/kubernetes/kubernetes/blob/master/pkg/cloudprovider/cloud.go)
The implementation of the four shared controllers highlighted above, and some scaffolding along with the shared cloudprovider interface, will stay in the Kubernetes core, but implementations specific to cloud providers will
be built outside of the core, and implement interfaces defined in the core.
For more information about developing plugins, see
[Developing Cloud Controller Manager](/docs/tasks/administer-cluster/developing-cloud-controller-manager/).
## Authorization
This section breaks down the access required on various API objects by the CCM to perform its operations.
### Node Controller
The Node controller only works with Node objects. It requires full access to get, list, create, update, patch, watch, and delete Node objects.
v1/Node:
- Get
- List
- Create
- Update
- Patch
- Watch
### Route controller
The route controller listens to Node object creation and configures routes appropriately. It requires get access to Node objects.
v1/Node:
- Get
### Service controller
The service controller listens to Service object create, update and delete events and then configures endpoints for those Services appropriately.
To access Services, it requires list, and watch access. To update Services, it requires patch and update access.
To set up endpoints for the Services, it requires access to create, list, get, watch, and update.
v1/Service:
- List
- Get
- Watch
- Patch
- Update
### PersistentVolumeLabels controller
The PersistentVolumeLabels controller listens on PersistentVolume (PV) create events and then updates them. This controller requires access to list, watch, get and update PVs.
v1/PersistentVolume:
- Get
- List
- Watch
- Update
### Others
The implementation of the core of CCM requires access to create events, and to ensure secure operation, it requires access to create ServiceAccounts.
v1/Event:
- Create
- Patch
- Update
v1/ServiceAccount:
- Create
The RBAC ClusterRole for the CCM looks like this:
```yaml
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
name: cloud-controller-manager
rules:
- apiGroups:
- ""
resources:
- events
verbs:
- create
- patch
- update
- apiGroups:
- ""
resources:
- nodes
verbs:
- '*'
- apiGroups:
- ""
resources:
- nodes/status
verbs:
- patch
- apiGroups:
- ""
resources:
- services
verbs:
- list
- patch
- update
- watch
- apiGroups:
- ""
resources:
- serviceaccounts
verbs:
- create
- apiGroups:
- ""
resources:
- persistentvolumes
verbs:
- get
- list
- update
- watch
- apiGroups:
- ""
resources:
- endpoints
verbs:
- create
- get
- list
- watch
- update
```
## Vendor Implementations
The following cloud providers have implemented CCMs for their own clouds.
* [Digital Ocean]()
* [Oracle]()
* [Azure]()
* [GCE]()
* [AWS]()
## Cluster Administration
Complete instructions for configuring and running the CCM are provided
[here](/docs/tasks/administer-cluster/running-cloud-controller/#cloud-controller-manager).
docs/concepts/architecture/master-node-communication.md
+1 -1
View File
@@ -99,7 +99,7 @@ public networks.
### SSH Tunnels
[Google Container Engine](https://cloud.google.com/container-engine/docs/) uses
[Google Kubernetes Engine](https://cloud.google.com/kubernetes-engine/) uses
SSH tunnels to protect the Master -> Cluster communication paths. In this
configuration, the apiserver initiates an SSH tunnel to each node in the
cluster (connecting to the ssh server listening on port 22) and passes all
docs/concepts/architecture/nodes.md
+12 -12
View File
@@ -12,7 +12,7 @@ title: Nodes
A `node` is a worker machine in Kubernetes, previously known as a `minion`. A node
may be a VM or physical machine, depending on the cluster. Each node has
the services necessary to run [pods](/docs/user-guide/pods) and is managed by the master
the services necessary to run [pods](/docs/concepts/workloads/pods/pod/) and is managed by the master
components. The services on a node include Docker, kubelet and kube-proxy. See
[The Kubernetes Node](https://git.k8s.io/community/contributors/design-proposals/architecture/architecture.md#the-kubernetes-node) section in the
architecture design doc for more details.
@@ -21,11 +21,11 @@ architecture design doc for more details.
A node's status contains the following information:
* [Addresses](#Addresses)
* ~~[Phase](#Phase)~~ **deprecated**
* [Condition](#Condition)
* [Capacity](#Capacity)
* [Info](#Info)
* [Addresses](#addresses)
* ~~[Phase](#phase)~~ **deprecated**
* [Condition](#condition)
* [Capacity](#capacity)
* [Info](#info)
Each section is described in detail below.
@@ -64,7 +64,7 @@ The node condition is represented as a JSON object. For example, the following r
]
```
If the Status of the Ready condition is "Unknown" or "False" for longer than the `pod-eviction-timeout`, an argument is passed to the [kube-controller-manager](/docs/admin/kube-controller-manager) and all of the Pods on the node are scheduled for deletion by the Node Controller. The default eviction timeout duration is **five minutes**. In some cases when the node is unreachable, the apiserver is unable to communicate with the kubelet on it. The decision to delete the pods cannot be communicated to the kubelet until it re-establishes communication with the apiserver. In the meantime, the pods which are scheduled for deletion may continue to run on the partitioned node.
If the Status of the Ready condition is "Unknown" or "False" for longer than the `pod-eviction-timeout`, an argument is passed to the [kube-controller-manager](/docs/admin/kube-controller-manager/) and all of the Pods on the node are scheduled for deletion by the Node Controller. The default eviction timeout duration is **five minutes**. In some cases when the node is unreachable, the apiserver is unable to communicate with the kubelet on it. The decision to delete the pods cannot be communicated to the kubelet until it re-establishes communication with the apiserver. In the meantime, the pods which are scheduled for deletion may continue to run on the partitioned node.
In versions of Kubernetes prior to 1.5, the node controller would [force delete](/docs/concepts/workloads/pods/pod/#force-deletion-of-pods)
these unreachable pods from the apiserver. However, in 1.5 and higher, the node controller does not force delete pods until it is
@@ -74,14 +74,14 @@ permanently left a cluster, the cluster administrator may need to delete the nod
Kubernetes causes all the Pod objects running on it to be deleted from the apiserver, freeing up their names.
Version 1.8 introduces an alpha feature that automatically creates
[taints](/docs/concepts/configuration/taint-and-toleration) that represent conditions.
[taints](/docs/concepts/configuration/taint-and-toleration/) that represent conditions.
To enable this behavior, pass an additional feature gate flag `--feature-gates=...,TaintNodesByCondition=true`
to the API server, controller manager, and scheduler.
When `TaintNodesByCondition` is enabled, the scheduler ignores conditions when considering a Node; instead
it looks at the Node's taints and a Pod's tolerations.
Now users can choose between the old scheduling model and a new, more flexible scheduling model.
A Pod that does not have any tolerations gets scheduled according to the old model. But a Pod that
A Pod that does not have any tolerations gets scheduled according to the old model. But a Pod that
tolerates the taints of a particular Node can be scheduled on that Node.
Note that because of small delay, usually less than one second, between time when condition is observed and a taint
@@ -101,7 +101,7 @@ The information is gathered by Kubelet from the node.
## Management
Unlike [pods](/docs/user-guide/pods) and [services](/docs/user-guide/services),
Unlike [pods](/docs/concepts/workloads/pods/pod/) and [services](/docs/concepts/workloads/pods/pod/),
a node is not inherently created by Kubernetes: it is created externally by cloud
providers like Google Compute Engine, or exists in your pool of physical or virtual
machines. What this means is that when Kubernetes creates a node, it is really
@@ -192,7 +192,7 @@ Starting in Kubernetes 1.6, the NodeController is also responsible for evicting
pods that are running on nodes with `NoExecute` taints, when the pods do not tolerate
the taints. Additionally, as an alpha feature that is disabled by default, the
NodeController is responsible for adding taints corresponding to node problems like
node unreachable or not ready. See [this documentation](/docs/concepts/configuration/taint-and-toleration)
node unreachable or not ready. See [this documentation](/docs/concepts/configuration/taint-and-toleration/)
for details about `NoExecute` taints and the alpha feature.
Starting in version 1.8, the node controller can be made responsible for creating taints that represent
@@ -209,7 +209,7 @@ For self-registration, the kubelet is started with the following options:
- `--cloud-provider` - How to talk to a cloud provider to read metadata about itself.
- `--register-node` - Automatically register with the API server.
- `--register-with-taints` - Register the node with the given list of taints (comma separated `<key>=<value>:<effect>`). No-op if `register-node` is false.
- `--node-ip` IP address of the node.
- `--node-ip` - IP address of the node.
- `--node-labels` - Labels to add when registering the node in the cluster.
- `--node-status-update-frequency` - Specifies how often kubelet posts node status to master.
docs/concepts/cluster-administration/addons.md
+4 -1
View File
@@ -15,11 +15,14 @@ Add-ons in each section are sorted alphabetically - the ordering does not imply
* [Calico](http://docs.projectcalico.org/latest/getting-started/kubernetes/installation/hosted/) is a secure L3 networking and network policy provider.
* [Canal](https://github.com/tigera/canal/tree/master/k8s-install) unites Flannel and Calico, providing networking and network policy.
* [Cilium](https://github.com/cilium/cilium) is a L3 network and network policy plugin that can enforce HTTP/API/L7 policies transparently. Both routing and overlay/encapsulation mode are supported.
* [CNI-Genie](https://github.com/Huawei-PaaS/CNI-Genie) enables Kubernetes to seamlessly connect to a choice of CNI plugins, such as Calico, Canal, Flannel, Romana, or Weave.
* [Contiv](http://contiv.github.io) provides configurable networking (native L3 using BGP, overlay using vxlan, classic L2, and Cisco-SDN/ACI) for various use cases and a rich policy framework. Contiv project is fully [open sourced](http://github.com/contiv). The [installer](http://github.com/contiv/install) provides both kubeadm and non-kubeadm based installation options.
* [Flannel](https://github.com/coreos/flannel/blob/master/Documentation/kube-flannel.yml) is an overlay network provider that can be used with Kubernetes.
* [Multus](https://github.com/Intel-Corp/multus-cni) is a Multi plugin for multiple network support in Kubernetes to support all CNI plugins (e.g. Calico, Cilium, Contiv, Flannel), in addition to SRIOV, DPDK, OVS-DPDK and VPP based workloads in Kubernetes.
* [NSX-T](https://docs.vmware.com/en/VMware-NSX-T/2.0/nsxt_20_ncp_kubernetes.pdf) Container Plug-in (NCP) provides integration between VMware NSX-T and container orchestrators such as Kubernetes, as well as integration between NSX-T and container-based CaaS/PaaS platforms such as Pivotal Container Service (PKS) and Openshift.
* [Nuage](https://github.com/nuagenetworks/nuage-kubernetes/blob/v5.1.1-1/docs/kubernetes-1-installation.rst) is an SDN platform that provides policy-based networking between Kubernetes Pods and non-Kubernetes environments with visibility and security monitoring.
* [Romana](http://romana.io) is a Layer 3 networking solution for pod networks that also supports the [NetworkPolicy API](/docs/concepts/services-networking/network-policies/). Kubeadm add-on installation details available [here](https://github.com/romana/romana/tree/master/containerize).
* [Weave Net](https://www.weave.works/docs/net/latest/kube-addon/) provides networking and network policy, will carry on working on both sides of a network partition, and does not require an external database.
* [CNI-Genie](https://github.com/Huawei-PaaS/CNI-Genie) enables Kubernetes to seamlessly connect to a choice of CNI plugins, such as Flannel, Calico, Canal, Romana, or Weave.
## Service Discovery
docs/concepts/cluster-administration/certificates.md
+260
View File
@@ -0,0 +1,260 @@
---
title: Certificates
---
* TOC
{:toc}
## Creating Certificates
When using client certificate authentication, you can generate certificates
using an existing deployment script or manually through `easyrsa`, `openssl`
or `cfssl`.
### Using an Existing Deployment Script
**Using an existing deployment script** is implemented at
`cluster/saltbase/salt/generate-cert/make-ca-cert.sh`.
Execute this script with two parameters. The first is the IP address
of API server. The second is a list of subject alternate names in the form `IP:<ip-address> or DNS:<dns-name>`.
The script generates three files: `ca.crt`, `server.crt`, and `server.key`.
Finally, add the following parameters into API server start parameters:
```
--client-ca-file=/srv/kubernetes/ca.crt
--tls-cert-file=/srv/kubernetes/server.crt
--tls-private-key-file=/srv/kubernetes/server.key
```
### easyrsa
**easyrsa** can manually generate certificates for your cluster.
1. Download, unpack, and initialize the patched version of easyrsa3.
curl -L -O https://storage.googleapis.com/kubernetes-release/easy-rsa/easy-rsa.tar.gz
tar xzf easy-rsa.tar.gz
cd easy-rsa-master/easyrsa3
./easyrsa init-pki
1. Generate a CA. (`--batch` set automatic mode. `--req-cn` default CN to use.)
./easyrsa --batch "--req-cn=${MASTER_IP}@`date +%s`" build-ca nopass
1. Generate server certificate and key.
The argument `--subject-alt-name` sets the possible IPs and DNS names the API server will
be accessed with. The `MASTER_CLUSTER_IP` is usually the first IP from the service CIDR
that is specified as the `--service-cluster-ip-range` argument for both the API server and
the controller manager component. The argument `--days` is used to set the number of days
after which the certificate expires.
The sample below also assume that you are using `cluster.local` as the default
DNS domain name.
./easyrsa --subject-alt-name="IP:${MASTER_IP}"\
"IP:${MASTER_CLUSTER_IP},"\
"DNS:kubernetes,"\
"DNS:kubernetes.default,"\
"DNS:kubernetes.default.svc,"\
"DNS:kubernetes.default.svc.cluster,"\
"DNS:kubernetes.default.svc.cluster.local" \
--days=10000 \
build-server-full server nopass
1. Copy `pki/ca.crt`, `pki/issued/server.crt`, and `pki/private/server.key` to your directory.
1. Fill in and add the following parameters into the API server start parameters:
--client-ca-file=/yourdirectory/ca.crt
--tls-cert-file=/yourdirectory/server.crt
--tls-private-key-file=/yourdirectory/server.key
### openssl
**openssl** can manually generate certificates for your cluster.
1. Generate a ca.key with 2048bit:
openssl genrsa -out ca.key 2048
1. According to the ca.key generate a ca.crt (use -days to set the certificate effective time):
openssl req -x509 -new -nodes -key ca.key -subj "/CN=${MASTER_IP}" -days 10000 -out ca.crt
1. Generate a server.key with 2048bit:
openssl genrsa -out server.key 2048
1. Create a config file for generating a Certificate Signing Request (CSR).
Be sure to substitute the values marked with angle brackets (e.g. `<MASTER_IP>`)
with real values before saving this to a file (e.g. `csr.conf`).
Note that the value for `MASTER_CLUSTER_IP` is the service cluster IP for the
API server as described in previous subsection.
The sample below also assume that you are using `cluster.local` as the default
DNS domain name.
[ req ]
default_bits = 2048
prompt = no
default_md = sha256
req_extensions = req_ext
distinguished_name = dn
[ dn ]
C = <country>
ST = <state>
L = <city>
O = <organization>
OU = <organization unit>
CN = <MASTER_IP>
[ req_ext ]
subjectAltName = @alt_names
[ alt_names ]
DNS.1 = kubernetes
DNS.2 = kubernetes.default
DNS.3 = kubernetes.default.svc
DNS.4 = kubernetes.default.svc.cluster
DNS.5 = kubernetes.default.svc.cluster.local
IP.1 = <MASTER_IP>
IP.2 = <MASTER_CLUSTER_IP>
[ v3_ext ]
authorityKeyIdentifier=keyid,issuer:always
basicConstraints=CA:FALSE
keyUsage=keyEncipherment,dataEncipherment
extendedKeyUsage=serverAuth,clientAuth
subjectAltName=@alt_names
1. Generate the certificate signing request based on the config file:
openssl req -new -key server.key -out server.csr -config csr.conf
1. Generate the server certificate using the ca.key, ca.crt and server.csr:
openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key \
-CAcreateserial -out server.crt -days 10000 \
-extensions v3_ext -extfile csr.conf
1. View the certificate:
openssl x509 -noout -text -in ./server.crt
Finally, add the same parameters into the API server start parameters.
### cfssl
**cfssl** is another tool for certificate generation.
1. Download, unpack and prepare the command line tools as shown below.
Note that you may need to adapt the sample commands based on the hardware
architecture and cfssl version you are using.
curl -LO https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 -o cfssl
chmod +x cfssl
curl -LO https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 -o cfssljson
chmod +x cfssljson
curl -LO https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64 -o cfssl-certinfo
chmod +x cfssl-certinfo
1. Create a directory to hold the artifacts and initialize cfssl:
mkdir cert
cd cert
../cfssl print-defaults config > config.json
../cfssl print-defaults csr > csr.json
1. Create a JSON config file for generating the CA file, for example, `ca-config.json`:
{
"signing": {
"default": {
"expiry": "8760h"
},
"profiles": {
"kubernetes": {
"usages": [
"signing",
"key encipherment",
"server auth",
"client auth",
],
"expiry": "8760h"
}
}
}
}
1. Create a JSON config file for CA certificate signing request (CSR), for example,
`ca-csr.json`. Be sure the replace the values marked with angle brackets with
real values you want to use.
{
"CN": "kubernetes",
"key": {
"algo": "rsa",
"size": 2048
},
"names":[{
"C": "<country>",
"ST": "<state>",
"L": "<city>",
"O": "<organization>",
"OU": "<organization unit>",
}]
}
1. Generate CA key (`ca-key.pem`) and certificate (`ca.pem`):
../cfssl gencert -initca ca-csr.json | ../cfssljson -bare ca
1. Create a JSON config file for generating keys and certificates for the API
server as shown below. Be sure to replace the values in angle brackets with
real values you want to use. The `MASTER_CLUSTER_IP` is the service cluster
IP for the API server as described in previous subsection.
The sample below also assume that you are using `cluster.local` as the default
DNS domain name.
{
"CN": "kubernetes",
"hosts": [
"127.0.0.1",
"<MASTER_IP>",
"<MASTER_CLUSTER_IP>",
"kubernetes",
"kubernetes.default",
"kubernetes.default.svc",
"kubernetes.default.svc.cluster",
"kubernetes.default.svc.cluster.local"
],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [{
"C": "<country>",
"ST": "<state>",
"L": "<city>",
"O": "<organization>",
"OU": "<organization unit>"
}]
}
1. Generate the key and certificate for the API server, which are by default
saved into file `server-key.pem` and `server.pem` respectively:
../cfssl gencert -ca=ca.pem -ca-key=ca-key.pem \
--config=ca-config.json -profile=kubernetes \
server-csr.json | ../cfssljson -bare server
## Distributing Self-Signed CA Certificate
A client node may refuse to recognize a self-signed CA certificate as valid.
For a non-production deployment, or for a deployment that runs behind a company
firewall, you can distribute a self-signed CA certificate to all clients and
refresh the local list for valid certificates.
On each client, perform the following operations:
```bash
$ sudo cp ca.crt /usr/local/share/ca-certificates/kubernetes.crt
$ sudo update-ca-certificates
Updating certificates in /etc/ssl/certs...
1 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d....
done.
```
## Certificates API
You can use the `certificates.k8s.io` API to provision
x509 certificates to use for authentication as documented
[here](/docs/tasks/tls/managing-tls-in-a-cluster).
docs/concepts/cluster-administration/cloud-providers.md
+196 -1
View File
@@ -49,13 +49,208 @@ Different settings can be applied to a load balancer service in AWS using _annot
* `service.beta.kubernetes.io/aws-load-balancer-connection-draining-timeout`: Used on the service to specify a connection draining timeout.
* `service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout`: Used on the service to specify the idle connection timeout.
* `service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled`: Used on the service to enable or disable cross-zone load balancing.
* `service.beta.kubernetes.io/aws-load-balancer-extra-security-groups`: Used one the service to specify additional security groups to be added to ELB created
* `service.beta.kubernetes.io/aws-load-balancer-extra-security-groups`: Used on the service to specify additional security groups to be added to ELB created
* `service.beta.kubernetes.io/aws-load-balancer-internal`: Used on the service to indicate that we want an internal ELB.
* `service.beta.kubernetes.io/aws-load-balancer-proxy-protocol`: Used on the service to enable the proxy protocol on an ELB. Right now we only accept the value `*` which means enable the proxy protocol on all ELB backends. In the future we could adjust this to allow setting the proxy protocol only on certain backends.
* `service.beta.kubernetes.io/aws-load-balancer-ssl-ports`: Used on the service to specify a comma-separated list of ports that will use SSL/HTTPS listeners. Defaults to `*` (all)
The information for the annotations for AWS is taken from the comments on [aws.go](https://github.com/kubernetes/kubernetes/blob/master/pkg/cloudprovider/providers/aws/aws.go)
# OpenStack
This section describes all the possible configurations which can
be used when using OpenStack with Kubernetes. The OpenStack cloud provider
implementation for Kubernetes supports the use of these OpenStack services from
the underlying cloud, where available:
| Service | API Version(s) | Required |
|--------------------------|----------------|----------|
| Block Storage (Cinder) | V1†, V2 | No |
| Compute (Nova) | V2 | No |
| Identity (Keystone) | V2‡, V3 | Yes |
| Load Balancing (Neutron) | V1§, V2 | No |
† Block Storage V1 API support is deprecated, support for Block Storage V3 will
be added in the future.
‡ Identity V2 API support is deprecated and will be removed from the provider in
a future release. As of the "Queens" release, OpenStack will no longer expose the
Identity V2 API.
§ Load Balancing V1 API support is deprecated and will be removed from the
provider in a future release.
Service discovery is achieved by listing the service catalog managed by
OpenStack Identity (Keystone) using the `auth-url` provided in the provider
configuration. The provider will gracefully degrade in functionality when
OpenStack services other than Keystone are not available and simply disclaim
support for impacted features. Certain features are also enabled or disabled
based on the list of extensions published by Neutron in the underlying cloud.
## cloud.conf
Kubernetes knows how to interact with OpenStack via the file cloud.conf. It is
the file that will provide Kubernetes with credentials and location for the OpenStack auth endpoint.
You can create a cloud.conf file by specifying the following details in it
### Typical configuration
This is an example of a typical configuration that touches the values that most
often need to be set. It points the provider at the OpenStack cloud's Keystone
endpoint, provides details for how to authenticate with it, and configures the
load balancer:
```yaml
[Global]
username=user
password=pass
auth-url=https://<keystone_ip>/identity/v3
tenant-id=c869168a828847f39f7f06edd7305637
domain-id=2a73b8f597c04551a0fdc8e95544be8a
[LoadBalancer]
subnet-id=6937f8fa-858d-4bc9-a3a5-18d2c957166a
```
#### Global
These configuration options for the OpenStack provider pertain to its global
configuration and should appear in the `[Global]` section of the `cloud.conf`
file:
* `auth-url` (Required): The URL of the keystone API used to authenticate. On
OpenStack control panels, this can be found at Access and Security > API
Access > Credentials.
* `username` (Required): Refers to the username of a valid user set in keystone.
* `password` (Required): Refers to the password of a valid user set in keystone.
* `tenant-id` (Required): Used to specify the id of the project where you want
to create your resources.
* `tenant-name` (Optional): Used to specify the name of the project where you
want to create your resources.
* `trust-id` (Optional): Used to specify the identifier of the trust to use for
authorization. A trust represents a user's (the trustor) authorization to
delegate roles to another user (the trustee), and optionally allow the trustee
to impersonate the trustor. Available trusts are found under the
`/v3/OS-TRUST/trusts` endpoint of the Keystone API.
* `domain-id` (Optional): Used to specify the id of the domain your user belongs
to.
* `domain-name` (Optional): Used to specify the name of the domain your user
belongs to.
* `region` (Optional): Used to specify the identifier of the region to use when
running on a multi-region OpenStack cloud. A region is a general division of
an OpenStack deployment. Although a region does not have a strict geographical
connotation, a deployment can use a geographical name for a region identifier
such as `us-east`. Available regions are found under the `/v3/regions`
endpoint of the Keystone API.
* `ca-file` (Optional): TODO
When using Keystone V3 - which changes tenant to project - the `tenant-id` value
is automatically mapped to the project construct in the API.
#### Load Balancer
These configuration options for the OpenStack provider pertain to the load
balancer and should appear in the `[LoadBalancer]` section of the `cloud.conf`
file:
* `lb-version` (Optional): Used to override automatic version detection. Valid
values are `v1` or `v2`. Where no value is provided automatic detection will
select the highest supported version exposed by the underlying OpenStack
cloud.
* `subnet-id` (Optional): Used to specify the id of the subnet you want to
create your loadbalancer on. Can be found at Network > Networks. Click on the
respective network to get its subnets.
* `floating-network-id` (Optional): If specified, will create a floating IP for
the load balancer.
* `lb-method` (Optional): Used to specify algorithm by which load will be
distributed amongst members of the load balancer pool. The value can be
`ROUND_ROBIN`, `LEAST_CONNECTIONS`, or `SOURCE_IP`. The default behavior if
none is specified is `ROUND_ROBIN`.
* `lb-provider` (Optional): Used to specify the provider of the load balancer.
If not specified, the default provider service configured in neutron will be
used.
* `create-monitor` (Optional): Indicates whether or not to create a health
monitor for the Neutron load balancer. Valid values are `true` and `false`.
The default is `false`. When `true` is specified then `monitor-delay`,
`monitor-timeout`, and `monitor-max-retries` must also be set.
* `monitor-delay` (Optional): The time, in seconds, between sending probes to
members of the load balancer.
* `monitor-timeout` (Optional): Maximum number of seconds for a monitor to wait
for a ping reply before it times out. The value must be less than the delay
value.
* `monitor-max-retries` (Optional): Number of permissible ping failures before
changing the load balancer member's status to INACTIVE. Must be a number
between 1 and 10.
* `manage-security-groups` (Optional): Determines whether or not the load
balancer should automatically manage the security group rules. Valid values
are `true` and `false`. The default is `false`. When `true` is specified
`node-security-group` must also be supplied.
* `node-security-group` (Optional): ID of the security group to manage.
#### Block Storage
These configuration options for the OpenStack provider pertain to block storage
and should appear in the `[BlockStorage]` section of the `cloud.conf` file:
* `bs-version` (Optional): Used to override automatic version detection. Valid
values are `v1`, `v2`, `v3` and `auto`. When `auto` is specified automatic
detection will select the highest supported version exposed by the underlying
OpenStack cloud. The default value if none is provided is `auto`.
* `trust-device-path` (Optional): In most scenarios the block device names
provided by Cinder (e.g. `/dev/vda`) can not be trusted. This boolean toggles
this behavior. Setting it to `true` results in trusting the block device names
provided by Cinder. The default value of `false` results in the discovery of
the device path based on its serial number and `/dev/disk/by-id` mapping and is
the recommended approach.
If deploying Kubernetes versions <= 1.8 on an OpenStack deployment that uses
paths rather than ports to differentiate between endpoints it may be necessary
to explicitly set the `bs-version` parameter. A path based endpoint is of the
form `http://foo.bar/volume` while a port based endpoint is of the form
`http://foo.bar:xxx`.
In environments that use path based endpoints and Kubernetes is using the older
auto-detection logic a `BS API version autodetection failed.` error will be
returned on attempting volume detachment. To workaround this issue it is
possible to force the use of Cinder API version 2 by adding this to the cloud
provider configuration:
```yaml
[BlockStorage]
bs-version=v2
```
#### Metadata
These configuration options for the OpenStack provider pertain to metadata and
should appear in the `[Metadata]` section of the `cloud.conf` file:
* `search-order` (Optional): This configuration key influences the way that the
provider retrieves metadata relating to the instance(s) in which it runs. The
default value of `configDrive,metadataService` results in the provider
retrieving metadata relating to the instance from the config drive first if
available and then the metadata service. Alternative values are:
* `configDrive` - Only retrieve instance metadata from the configuration
drive.
* `metadataService` - Only retrieve instance metadata from the metadata
service.
* `metadataService,configDrive` - Retrieve instance metadata from the metadata
service first if available, then the configuration drive.
Influencing this behavior may be desirable as the metadata on the
configuration drive may grow stale over time, whereas the metadata service
always provides the most up to date view. Not all OpenStack clouds provide
both configuration drive and metadata service though and only one or the other
may be available which is why the default is to check both.
#### Router
These configuration options for the OpenStack provider pertain to the [kubenet]
Kubernetes network plugin and should appear in the `[Router]` section of the
`cloud.conf` file:
* `router-id` (Optional): If the underlying cloud's Neutron deployment supports
the `extraroutes` extension then use `router-id` to specify a router to add
routes to. The router chosen must span the private networks containing your
cluster nodes (typically there is only one node network, and this value should be
the default router for the node network). This value is required to use [kubenet]
on OpenStack.
[kubenet]: https://kubernetes.io/docs/concepts/cluster-administration/network-plugins/#kubenet
{% endcapture %}
{% include templates/concept.md %}
docs/concepts/cluster-administration/cluster-administration-overview.md
+7 -5
View File
@@ -7,7 +7,7 @@ title: Cluster Administration Overview
{% capture overview %}
The cluster administration overview is for anyone creating or administering a Kubernetes cluster.
It assumes some familiarity with concepts in the [User Guide](/docs/user-guide/).
It assumes some familiarity with core Kubernetes [concepts](/docs/concepts/).
{% endcapture %}
{% capture body %}
@@ -18,10 +18,10 @@ See the guides in [Picking the Right Solution](/docs/setup/pick-right-solution/)
Before choosing a guide, here are some considerations:
- Do you just want to try out Kubernetes on your computer, or do you want to build a high-availability, multi-node cluster? Choose distros best suited for your needs.
- **If you are designing for high-availability**, learn about configuring [clusters in multiple zones](/docs/admin/multi-cluster/).
- Will you be using **a hosted Kubernetes cluster**, such as [Google Container Engine (GKE)](https://cloud.google.com/container-engine/), or **hosting your own cluster**?
- **If you are designing for high-availability**, learn about configuring [clusters in multiple zones](/docs/concepts/cluster-administration/federation/).
- Will you be using **a hosted Kubernetes cluster**, such as [Google Kubernetes Engine](https://cloud.google.com/kubernetes-engine/), or **hosting your own cluster**?
- Will your cluster be **on-premises**, or **in the cloud (IaaS)**? Kubernetes does not directly support hybrid clusters. Instead, you can set up multiple clusters.
- **If you are configuring Kubernetes on-premises**, consider which [networking model](/docs/admin/networking/) fits best. One option for custom networking is [*OpenVSwitch GRE/VxLAN networking*](/docs/admin/ovs-networking/), which uses OpenVSwitch to set up networking between pods across Kubernetes nodes.
- **If you are configuring Kubernetes on-premises**, consider which [networking model](/docs/concepts/cluster-administration/networking/) fits best. One option for custom networking is [*OpenVSwitch GRE/VxLAN networking*](/docs/admin/ovs-networking/), which uses OpenVSwitch to set up networking between pods across Kubernetes nodes.
- Will you be running Kubernetes on **"bare metal" hardware** or on **virtual machines (VMs)**?
- Do you **just want to run a cluster**, or do you expect to do **active development of Kubernetes project code**? If the
latter, choose a actively-developed distro. Some distros only use binary releases, but
@@ -34,7 +34,7 @@ If you are using a guide involving Salt, see [Configuring Kubernetes with Salt](
## Managing a cluster
* [Managing a cluster](/docs/concepts/cluster-administration/cluster-management/) describes several topics related to the lifecycle of a cluster: creating a new cluster, upgrading your cluster’s master and worker nodes, performing node maintenance (e.g. kernel upgrades), and upgrading the Kubernetes API version of a running cluster.
* [Managing a cluster](/docs/tasks/administer-cluster/cluster-management/) describes several topics related to the lifecycle of a cluster: creating a new cluster, upgrading your cluster’s master and worker nodes, performing node maintenance (e.g. kernel upgrades), and upgrading the Kubernetes API version of a running cluster.
* Learn how to [manage nodes](/docs/concepts/nodes/node/).
@@ -42,6 +42,8 @@ If you are using a guide involving Salt, see [Configuring Kubernetes with Salt](
## Securing a cluster
* [Certificates](/docs/concepts/cluster-administration/certificates/) describes the steps to generate certificates using different tool chains.
* [Kubernetes Container Environment](/docs/concepts/containers/container-environment-variables/) describes the environment for Kubelet managed containers on a Kubernetes node.
* [Controlling Access to the Kubernetes API](/docs/admin/accessing-the-api/) describes how to set up permissions for users and service accounts.
docs/concepts/cluster-administration/device-plugins.md
+7 -6
View File
@@ -7,7 +7,8 @@ description: Use the Kubernetes device plugin framework to implement plugins for
{% include feature-state-alpha.md %}
{% capture overview %}
Starting in version 1.8, Kubernetes provides a [device plugin framework](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/device-plugin.md)
Starting in version 1.8, Kubernetes provides a
[device plugin framework](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/resource-management/device-plugin.md)
for vendors to advertise their resources to the kubelet without changing Kubernetes core code.
Instead of writing custom Kubernetes code, vendors can implement a device plugin that can
be deployed manually or as a DaemonSet. The targeted devices include GPUs,
@@ -48,7 +49,7 @@ Then, developers can request devices in a
[Container](/docs/api-reference/{{page.version}}/#container-v1-core)
specification by using the same process that is used for
[opaque integer resources](/docs/tasks/configure-pod-container/opaque-integer-resource/).
In version 1.8, extended resources are spported only as integer resources and must have
In version 1.8, extended resources are supported only as integer resources and must have
`limit` equal to `request` in the Container specification.
## Device plugin implementation
@@ -57,7 +58,7 @@ The general workflow of a device plugin includes the following steps:
* Initialization. During this phase, the device plugin performs vendor specific
initialization and setup to make sure the devices are in a ready state.
* The plugin starts a gRPC service, with a Unix socket under host path
`/var/lib/kubelet/device-plugins/`, that implements the following interfaces:
@@ -97,17 +98,17 @@ A device plugin can be deployed manually or as a DaemonSet. Being deployed as a
the benefit that Kubernetes can restart the device plugin if it fails.
Otherwise, an extra mechanism is needed to recover from device plugin failures.
The canonical directory `/var/lib/kubelet/device-plugins` requires privileged access,
so a device plugin must run in a privileged security context.
so a device plugin must run in a privileged security context.
If a device plugin is running as a DaemonSet, `/var/lib/kubelet/device-plugins`
must be mounted as a
[Volume](/docs/api-reference/{{page.version}}/#volume-v1-core)
in the plugin's
[PodSpec](/docs/api-reference/{{paage.version}}/#podspec-v1-core).
[PodSpec](/docs/api-reference/{{page.version}}/#podspec-v1-core).
## Examples
For an example device plugin implementation, see
[nvidia GPU device plugin for COS base OS](https://github.com/GoogleCloudPlatform/container-engine-accelerators/tree/master/nvidia_gpu).
[nvidia GPU device plugin for COS base OS](https://github.com/GoogleCloudPlatform/container-engine-accelerators/tree/master/cmd/nvidia_gpu).
{% endcapture %}
docs/concepts/cluster-administration/federation.md
+4 -3
View File
@@ -34,7 +34,7 @@ why you might want multiple clusters are:
* Fault isolation: It might be better to have multiple small clusters rather
than a single large cluster for fault isolation (for example: multiple
clusters in different availability zones of a cloud provider).
See [Multi cluster guide](/docs/admin/multi-cluster) for details.
See [Multi cluster guide](/docs/concepts/cluster-administration/federation/) for details.
* Scalability: There are scalability limits to a single kubernetes cluster (this
should not be the case for most users. For more details:
[Kubernetes Scaling and Performance Goals](https://git.k8s.io/community/sig-scalability/goals.md)).
@@ -165,13 +165,14 @@ users in the event of a cluster failure), then you need to have `R * (U + 1)` cl
(`U + 1` in each of `R` regions). In any case, try to put each cluster in a different zone.
Finally, if any of your clusters would need more than the maximum recommended number of nodes for a Kubernetes cluster, then
you may need even more clusters. Kubernetes v1.3 supports clusters up to 1000 nodes in size.
you may need even more clusters. Kubernetes v1.3 supports clusters up to 1000 nodes in size. Kubernetes v1.8 supports
clusters up to 5000 nodes. See [Building Large Clusters](/docs/admin/cluster-large/) for more guidance.
{% endcapture %}
{% capture whatsnext %}
* Learn more about the [Federation
proposal](https://github.com/kubernetes/community/blob/{{page.githubbranch}}/contributors/design-proposals/federation/federation.md).
proposal](https://github.com/kubernetes/community/blob/{{page.githubbranch}}/contributors/design-proposals/multicluster/federation.md).
* See this [setup guide](/docs/tutorials/federation/set-up-cluster-federation-kubefed/) for cluster federation.
* See this [Kubecon2016 talk on federation](https://www.youtube.com/watch?v=pq9lbkmxpS8)
{% endcapture %}
docs/concepts/cluster-administration/manage-deployment.md
+2 -2
View File
@@ -9,7 +9,7 @@ title: Managing Resources
You've deployed your application and exposed it via a service. Now what? Kubernetes provides a number of tools to help you manage your application deployment, including scaling and updating. Among the features that we will discuss in more depth are [configuration files](/docs/concepts/configuration/overview/) and [labels](/docs/concepts/overview/working-with-objects/labels/).
You can find all the files for this example [in our docs
repo here](https://github.com/kubernetes/kubernetes.github.io/tree/{{page.docsbranch}}/docs/user-guide/).
repo here](https://github.com/kubernetes/website/tree/{{page.docsbranch}}/docs/user-guide/).
* TOC
{:toc}
@@ -301,7 +301,7 @@ deployment "my-nginx" autoscaled
Now your nginx replicas will be scaled up and down as needed, automatically.
For more information, please see [kubectl scale](/docs/user-guide/kubectl/{{page.version}}/#scale), [kubectl autoscale](/docs/user-guide/kubectl/v1.6/#autoscale) and [horizontal pod autoscaler](/docs/tasks/run-application/horizontal-pod-autoscale/) document.
For more information, please see [kubectl scale](/docs/user-guide/kubectl/{{page.version}}/#scale), [kubectl autoscale](/docs/user-guide/kubectl/{{page.version}}/#autoscale) and [horizontal pod autoscaler](/docs/tasks/run-application/horizontal-pod-autoscale/) document.
## In-place updates of resources
docs/concepts/cluster-administration/networking.md
+13 -1
View File
@@ -184,6 +184,18 @@ Follow the "With Linux Bridge devices" section of [this very nice
tutorial](http://blog.oddbit.com/2014/08/11/four-ways-to-connect-a-docker/) from
Lars Kellogg-Stedman.
### Multus (a Multi Network plugin)
[Multus](https://github.com/Intel-Corp/multus-cni) is a Multi CNI plugin to support the Multi Networking feature in Kubernetes using CRD based network objects in Kubernetes.
Multus supports all [reference plugins](https://github.com/containernetworking/plugins) (eg. [Flannel](https://github.com/containernetworking/plugins/tree/master/plugins/meta/flannel), [DHCP](https://github.com/containernetworking/plugins/tree/master/plugins/ipam/dhcp), [Macvlan](https://github.com/containernetworking/plugins/tree/master/plugins/main/macvlan)) that implement the CNI specification and 3rd party plugins (eg. [Calico](https://github.com/projectcalico/cni-plugin), [Weave](https://github.com/weaveworks/weave), [Cilium](https://github.com/cilium/cilium), [Contiv](https://github.com/contiv/netplugin)). In addition to it, Multus supports [SRIOV](https://github.com/hustcat/sriov-cni), [DPDK](https://github.com/Intel-Corp/sriov-cni), [OVS-DPDK & VPP](https://github.com/intel/vhost-user-net-plugin) workloads in Kubernetes with both cloud native and NFV based applications in Kubernetes.
### NSX-T
[VMware NSX-T](https://docs.vmware.com/en/VMware-NSX-T/index.html) is a network virtualization and security platform. NSX-T can provide network virtualization for a multi-cloud and multi-hypervisor environment and is focused on emerging application frameworks and architectures that have heterogeneous endpoints and technology stacks. In addition to vSphere hypervisors, these environments include other hypervisors such as KVM, containers, and bare metal.
[NSX-T Container Plug-in (NCP)](https://docs.vmware.com/en/VMware-NSX-T/2.0/nsxt_20_ncp_kubernetes.pdf) provides integration between NSX-T and container orchestrators such as Kubernetes, as well as integration between NSX-T and container-based CaaS/PaaS platforms such as Pivotal Container Service (PKS) and Openshift.
### Nuage Networks VCS (Virtualized Cloud Services)
[Nuage](http://www.nuagenetworks.net) provides a highly scalable policy-based Software-Defined Networking (SDN) platform. Nuage uses the open source Open vSwitch for the data plane along with a feature rich SDN Controller built on open standards.
@@ -226,7 +238,7 @@ to run, and in both cases, the network provides one IP address per pod - as is s
### CNI-Genie from Huawei
[CNI-Genie](https://github.com/Huawei-PaaS/CNI-Genie) is a CNI plugin that enables Kubernetes to [simultanously have access to different implementations](https://github.com/Huawei-PaaS/CNI-Genie/blob/master/docs/multiple-cni-plugins/README.md#what-cni-genie-feature-1-multiple-cni-plugins-enables) of the [Kubernetes network model](https://git.k8s.io/kubernetes.github.io/docs/concepts/cluster-administration/networking.md#kubernetes-model) in runtime. This includes any implementation that runs as a [CNI plugin](https://github.com/containernetworking/cni#3rd-party-plugins), such as [Flannel](https://github.com/coreos/flannel#flannel), [Calico](http://docs.projectcalico.org/), [Romana](http://romana.io), [Weave-net](https://www.weave.works/products/weave-net/).
[CNI-Genie](https://github.com/Huawei-PaaS/CNI-Genie) is a CNI plugin that enables Kubernetes to [simultanously have access to different implementations](https://github.com/Huawei-PaaS/CNI-Genie/blob/master/docs/multiple-cni-plugins/README.md#what-cni-genie-feature-1-multiple-cni-plugins-enables) of the [Kubernetes network model](https://git.k8s.io/website/docs/concepts/cluster-administration/networking.md#kubernetes-model) in runtime. This includes any implementation that runs as a [CNI plugin](https://github.com/containernetworking/cni#3rd-party-plugins), such as [Flannel](https://github.com/coreos/flannel#flannel), [Calico](http://docs.projectcalico.org/), [Romana](http://romana.io), [Weave-net](https://www.weave.works/products/weave-net/).
CNI-Genie also supports [assigning multiple IP addresses to a pod](https://github.com/Huawei-PaaS/CNI-Genie/blob/master/docs/multiple-ips/README.md#feature-2-extension-cni-genie-multiple-ip-addresses-per-pod), each from a different CNI plugin.
docs/concepts/cluster-administration/proxies.md
+14 -5
View File
@@ -12,14 +12,17 @@ This page explains proxies used with Kubernetes.
There are several different proxies you may encounter when using Kubernetes:
1. The [kubectl proxy](#directly-accessing-the-rest-api):
1. The [kubectl proxy](/docs/tasks/access-application-cluster/access-cluster/#directly-accessing-the-rest-api):
- runs on a user's desktop or in a pod
- proxies from a localhost address to the Kubernetes apiserver
- client to proxy uses HTTP
- proxy to apiserver uses HTTPS
- locates apiserver
- adds authentication headers
1. The [apiserver proxy](#discovering-builtin-services):
1. The [apiserver proxy](/docs/tasks/access-application-cluster/access-cluster/#discovering-builtin-services):
- is a bastion built into the apiserver
- connects a user outside of the cluster to cluster IPs which otherwise might not be reachable
- runs in the apiserver processes
@@ -27,17 +30,23 @@ There are several different proxies you may encounter when using Kubernetes:
- proxy to target may use HTTP or HTTPS as chosen by proxy using available information
- can be used to reach a Node, Pod, or Service
- does load balancing when used to reach a Service
1. The [kube proxy](/docs/concepts/services-networking/service/#ips-and-vips):
1. The [kube proxy](/docs/concepts/services-networking/service/#ips-and-vips):
- runs on each node
- proxies UDP and TCP
- does not understand HTTP
- provides load balancing
- is just used to reach services
1. A Proxy/Load-balancer in front of apiserver(s):
1. A Proxy/Load-balancer in front of apiserver(s):
- existence and implementation varies from cluster to cluster (e.g. nginx)
- sits between all clients and one or more apiservers
- acts as load balancer if there are several apiservers.
1. Cloud Load Balancers on external services:
1. Cloud Load Balancers on external services:
- are provided by some cloud providers (e.g. AWS ELB, Google Cloud Load Balancer)
- are created automatically when the Kubernetes service has type `LoadBalancer`
- use UDP/TCP only
docs/concepts/cluster-administration/sysctl-cluster.md
+5 -1
View File
@@ -93,7 +93,11 @@ flag of the kubelet, e.g.:
```shell
$ kubelet --experimental-allowed-unsafe-sysctls 'kernel.msg*,net.ipv4.route.min_pmtu' ...
```
For minikube, this can be done via the `extra-config` flag:
```shell
$ minikube start --extra-config="kubelet.AllowedUnsafeSysctls=kernel.msg*,net.ipv4.route.min_pmtu"...
```
Only _namespaced_ sysctls can be enabled this way.
## Setting Sysctls for a Pod
@@ -118,5 +122,5 @@ spec:
**Note**: a pod with the _unsafe_ sysctls specified above will fail to launch on
any node which has not enabled those two _unsafe_ sysctls explicitly. As with
_node-level_ sysctls it is recommended to use [_taints and toleration_
feature](/docs/user-guide/kubectl/v1.6/#taint) or [taints on nodes](/docs/concepts/configuration/taint-and-toleration/)
feature](/docs/user-guide/kubectl/{{page.version}}/#taint) or [taints on nodes](/docs/concepts/configuration/taint-and-toleration/)
to schedule those pods onto the right nodes.
docs/concepts/configuration/assign-pod-node.md
+29 -6
View File
@@ -16,7 +16,7 @@ that a pod ends up on a machine with an SSD attached to it, or to co-locate pods
services that communicate a lot into the same availability zone.
You can find all the files for these examples [in our docs
repo here](https://github.com/kubernetes/kubernetes.github.io/tree/{{page.docsbranch}}/docs/user-guide/node-selection).
repo here](https://github.com/kubernetes/website/tree/{{page.docsbranch}}/docs/user-guide/node-selection).
* TOC
{:toc}
@@ -157,6 +157,10 @@ like node, rack, cloud provider zone, cloud provider region, etc. You express it
key for the node label that the system uses to denote such a topology domain, e.g. see the label keys listed above
in the section [Interlude: built-in node labels](#interlude-built-in-node-labels).
**Note:** Inter-pod affinity and anti-affinity require substantial amount of
processing which can slow down scheduling in large clusters significantly. We do
not recommend using them in clusters larger than several hundred nodes.
As with node affinity, there are currently two types of pod affinity and anti-affinity, called `requiredDuringSchedulingIgnoredDuringExecution` and
`preferredDuringSchedulingIgnoredDuringExecution` which denote "hard" vs. "soft" requirements.
See the description in the node affinity section earlier.
@@ -215,7 +219,7 @@ be co-located in the same defined topology, eg., the same node.
##### Always co-located in the same node
In a three node cluster, a web application has in-memory cache such as redis. We want the web-servers to be co-located with the cache as much as possible.
Here is the yaml snippet of a simple redis deployment with three replicas and selector label `app=store`
Here is the yaml snippet of a simple redis deployment with three replicas and selector label `app=store`. The deployment has `PodAntiAffinity` configured to ensure the scheduler does not co-locate replicas on a single node.
```yaml
apiVersion: apps/v1beta1 # for versions before 1.6.0 use extensions/v1beta1
@@ -229,13 +233,22 @@ spec:
labels:
app: store
spec:
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app
operator: In
values:
- store
topologyKey: "kubernetes.io/hostname"
containers:
- name: redis-server
image: redis:3.2-alpine
```
Below yaml snippet of the webserver deployment has `podAffinity` configured, this informs the scheduler that all its replicas are to be
co-located with pods that has selector label `app=store`
The below yaml snippet of the webserver deployment has `podAntiAffinity` and `podAffinity` configured. This informs the scheduler that all its replicas are to be co-located with pods that have selector label `app=store`. This will also ensure that each web-server replica does not co-locate on a single node.
```yaml
apiVersion: apps/v1beta1 # for versions before 1.6.0 use extensions/v1beta1
@@ -250,6 +263,15 @@ spec:
app: web-store
spec:
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app
operator: In
values:
- web-store
topologyKey: "kubernetes.io/hostname"
podAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
@@ -261,9 +283,10 @@ spec:
topologyKey: "kubernetes.io/hostname"
containers:
- name: web-app
image: nginx:1.12-alpine
```
if we create the above two deployments, our three node cluster could look like below.
If we create the above two deployments, our three node cluster should look like below.
| node-1 | node-2 | node-3 |
|:--------------------:|:-------------------:|:------------------:|
@@ -283,7 +306,7 @@ web-server-1287567482-6f7v5 1/1 Running 0 7m 10.192.4
web-server-1287567482-s330j 1/1 Running 0 7m 10.192.3.2 kube-node-2
```
Best practice is to configure these highly available stateful workloads such as redis with AntiAffinity rules for more guaranteed spreading, which we will see in the next section.
Best practice is to configure these highly available stateful workloads such as redis with AntiAffinity rules for more guaranteed spreading.
##### Never co-located in the same node
docs/concepts/configuration/commands.yaml
+1
View File
@@ -10,3 +10,4 @@ spec:
image: debian
command: ["printenv"]
args: ["HOSTNAME", "KUBERNETES_PORT"]
restartPolicy: OnFailure
docs/concepts/configuration/manage-compute-resources-container.md
+15 -8
View File
@@ -4,7 +4,7 @@ title: Managing Compute Resources for Containers
{% capture overview %}
When you specify a [Pod](/docs/user-guide/pods), you can optionally specify how
When you specify a [Pod](/docs/concepts/workloads/pods/pod/), you can optionally specify how
much CPU and memory (RAM) each Container needs. When Containers have resource
requests specified, the scheduler can make better decisions about which nodes to
place Pods on. And when Containers have their limits specified, contention for
@@ -27,7 +27,7 @@ CPU and memory are collectively referred to as *compute resources*, or just
resources are measurable quantities that can be requested, allocated, and
consumed. They are distinct from
[API resources](/docs/concepts/overview/kubernetes-api/). API resources, such as Pods and
[Services](/docs/user-guide/services) are objects that can be read and modified
[Services](/docs/concepts/services-networking/service/) are objects that can be read and modified
through the Kubernetes API server.
## Resource requests and limits of Pod and Container
@@ -70,7 +70,7 @@ CPU is always requested as an absolute quantity, never as a relative quantity;
Limits and requests for `memory` are measured in bytes. You can express memory as
a plain integer or as a fixed-point integer using one of these suffixes:
E, P, T, G, M, K. You can also use the power-of-two equivalents: Ei, Pi, Ti, Gi,
E, P, T, G, M, k. You can also use the power-of-two equivalents: Ei, Pi, Ti, Gi,
Mi, Ki. For example, the following represent roughly the same value:
```shell
@@ -92,6 +92,9 @@ spec:
containers:
- name: db
image: mysql
env:
- name: MYSQL_ROOT_PASSWORD
value: "password"
resources:
requests:
memory: "64Mi"
@@ -295,7 +298,7 @@ You can call `kubectl get pod` with the `-o go-template=...` option to fetch the
of previously terminated Containers:
```shell{% raw %}
[13:59:01] $ kubectl get pod -o go-template='{{range.status.containerStatuses}}{{"Container Name: "}}{{.name}}{{"\r\nLastState: "}}{{.lastState}}{{end}}' simmemleak-60xbc
[13:59:01] $ kubectl get pod -o go-template='{{range.status.containerStatuses}}{{"Container Name: "}}{{.name}}{{"\r\nLastState: "}}{{.lastState}}{{end}}' simmemleak-hra99
Container Name: simmemleak
LastState: map[terminated:map[exitCode:137 reason:OOM Killed startedAt:2015-07-07T20:58:43Z finishedAt:2015-07-07T20:58:43Z containerID:docker://0e4095bba1feccdfe7ef9fb6ebffe972b4b14285d5acdec6f0d3ae8a22fad8b2]]{% endraw %}
```
@@ -309,7 +312,7 @@ Kubernetes version 1.8 introduces a new resource, _ephemeral-storage_ for managi
This partition is “ephemeral” and applications cannot expect any performance SLAs (Disk IOPS for example) from this partition. Local ephemeral storage management only applies for the root partition; the optional partition for image layer and writable layer is out of scope.
**Note:** If an optional runntime partition is used, root parition will not hold any image layer or writable layers.
**Note:** If an optional runntime partition is used, root partition will not hold any image layer or writable layers.
{: .note}
### Requests and limits setting for local ephemeral storage
@@ -338,6 +341,9 @@ spec:
containers:
- name: db
image: mysql
env:
- name: MYSQL_ROOT_PASSWORD
value: "password"
resources:
requests:
ephemeral-storage: "2Gi"
@@ -354,7 +360,7 @@ spec:
### How Pods with ephemeral-storage requests are scheduled
When you create a Pod, the Kubernetes scheduler selects a node for the Pod to
When you create a Pod, the Kubernetes scheduler selects a node for the Pod to
run on. Each node has a maximum amount of local ephemeral storage it can provide for Pods. (For more information, see ["Node Allocatable"](/docs/tasks/administer-cluster/reserve-compute-resources/#node-allocatable) The scheduler ensures that the sum of the resource requests of the scheduled Containers is less than the capacity of the node.
### How Pods with ephemeral-storage limits run
@@ -558,8 +564,9 @@ consistency across providers and platforms.
{% capture whatsnext %}
* Get hands-on experience
[assigning CPU and RAM resources to a container](/docs/tasks/configure-pod-container/assign-cpu-ram-container/).
* Get hands-on experience [assigning Memory resources to containers and pods](/docs/tasks/configure-pod-container/assign-memory-resource/).
* Get hands-on experience [assigning CPU resources to containers and pods](/docs/tasks/configure-pod-container/assign-cpu-resource/).
* [Container](/docs/api-reference/{{page.version}}/#container-v1-core)
docs/concepts/configuration/overview.md
+16 -12
View File
@@ -5,7 +5,7 @@ title: Configuration Best Practices
---
{% capture overview %}
This document highlights and consolidates configuration best practices that are introduced throughout the user-guide, getting-started documentation, and examples.
This document highlights and consolidates configuration best practices that are introduced throughout the user-guide, getting-started documentation and examples.
This is a living document. If you think of something that is not on this list but might be useful to others, please don't hesitate to file an issue or submit a PR.
{% endcapture %}
@@ -23,7 +23,7 @@ This is a living document. If you think of something that is not on this list bu
Note also that many `kubectl` commands can be called on a directory, so you can also call `kubectl create` on a directory of config files. See below for more details.
- Don't specify default values unnecessarily, in order to simplify and minimize configs, and to reduce error. For example, omit the selector and labels in a `ReplicationController` if you want them to be the same as the labels in its `podTemplate`, since those fields are populated from the `podTemplate` labels by default. See the [guestbook app's](https://github.com/kubernetes/examples/tree/{{page.githubbranch}}/guestbook/) .yaml files for some [examples](https://github.com/kubernetes/examples/tree/{{page.githubbranch}}/guestbook/frontend-deployment.yaml) of this.
- Don't specify default values unnecessarily -- simple and minimal configs will reduce errors.
- Put an object description in an annotation to allow better introspection.
@@ -37,15 +37,19 @@ This is a living document. If you think of something that is not on this list bu
## Services
- It's typically best to create a [service](/docs/concepts/services-networking/service/) before corresponding [replication controllers](/docs/concepts/workloads/controllers/replicationcontroller/). This lets the scheduler spread the pods that comprise the service.
- It's typically best to create a [service](/docs/concepts/services-networking/service/) before the corresponding [replication controllers](/docs/concepts/workloads/controllers/replicationcontroller/). This lets the scheduler spread the pods that comprise the service.
You can also use this process to ensure that at least one replica works before creating lots of them:
1. Create a replication controller without specifying replicas (this will set replicas=1);
2. Create a service;
3. Then scale up the replication controller.
- Don't use `hostPort` unless it is absolutely necessary (for example: for a node daemon). It specifies the port number to expose on the host. When you bind a Pod to a `hostPort`, there are a limited number of places to schedule a pod due to port conflicts— you can only schedule as many such Pods as there are nodes in your Kubernetes cluster.
- Don't use `hostPort` unless it is absolutely necessary (for example: for a node daemon).
It specifies the port number to expose on the host.
When you bind a Pod to a `hostPort`, there are a limited number of places to schedule a pod due to port conflicts.
The conflict comes from the requirement of an unique <hostIP,hostPort,protocol> combination.
Different <hostIP,hostPort,protocol> combinations mean different requirements.
For example, a pod that binds to host port 80 on 127.0.0.1 with TCP protocol has no conflict with another Pod that binds to host port 80 on 127.0.0.2 with TCP protocol.
*Special notes on hostIP and protocol*: If you don't specify the hostIP and protocol explicitly,
kubernetes will use 0.0.0.0 and TCP as the default hostIP and protocol,
where "0.0.0.0" is a wildcard IP that will match all <*,hostPort,protocol> on the node the pod is scheduled on.
Specifically, it will match all <IP,hostPort,protocol> tuples for all IPs on the host.
If you only need access to the port for debugging purposes, you can use the [kubectl proxy and apiserver proxy](/docs/tasks/access-kubernetes-api/http-proxy-access-api/) or [kubectl port-forward](/docs/tasks/access-application-cluster/port-forward-access-application-cluster/).
You can use a [Service](/docs/concepts/services-networking/service/) object for external service access.
@@ -62,9 +66,9 @@ This is a living document. If you think of something that is not on this list bu
A service can be made to span multiple deployments, such as is done across [rolling updates](/docs/tasks/run-application/rolling-update-replication-controller/), by simply omitting release-specific labels from its selector, rather than updating a service's selector to match the replication controller's selector fully.
- To facilitate rolling updates, include version info in replication controller names, for example as a suffix to the name. It is useful to set a 'version' label as well. The rolling update creates a new controller as opposed to modifying the existing controller. So, there will be issues with version-agnostic controller names. See the [documentation](/docs/tasks/run-application/rolling-update-replication-controller/) on the rolling-update command for more detail.
- To facilitate rolling updates, include version info in replication controller names, for example as a suffix to the name. It is useful to set a `version` label as well. The rolling update creates a new controller as opposed to modifying the existing controller. So, there will be issues with version-agnostic controller names. See the [documentation](/docs/tasks/run-application/rolling-update-replication-controller/) on the rolling-update command for more detail.
Note that the [Deployment](/docs/concepts/workloads/controllers/deployment/) object obviates the need to manage replication controller 'version names'. A desired state of an object is described by a Deployment, and if changes to that spec are _applied_, the deployment controller changes the actual state to the desired state at a controlled rate. (Deployment objects are currently part of the [`extensions` API Group](/docs/concepts/overview/kubernetes-api/#api-groups).)
Note that the [Deployment](/docs/concepts/workloads/controllers/deployment/) object obviates the need to manage replication controller `version names`. A desired state of an object is described by a Deployment, and if changes to that spec are _applied_, the deployment controller changes the actual state to the desired state at a controlled rate. (Deployment objects are currently part of the [`extensions` API Group](/docs/concepts/overview/kubernetes-api/#api-groups).)
- You can manipulate labels for debugging. Because Kubernetes replication controllers and services match to pods using labels, this allows you to remove a pod from being considered by a controller, or served traffic by a service, by removing the relevant selector labels. If you remove the labels of an existing pod, its controller will create a new pod to take its place. This is a useful way to debug a previously "live" pod in a quarantine environment. See the [`kubectl label`](/docs/concepts/overview/working-with-objects/labels/) command.
docs/concepts/configuration/pod-priority-preemption.md
+14 -15
View File
@@ -15,7 +15,7 @@ the scheduler tries to preempt (evict) lower priority Pods to make scheduling of
pending Pod possible. In a future Kubernetes release, priority will also affect
out-of-resource eviction ordering on the Node.
**Note:** Preemption does not respect PodDisruptionBudget; see
**Note:** Preemption does not respect PodDisruptionBudget; see
[the limitations section](#poddisruptionbudget-is-not-supported) for more details.
{: .note}
@@ -31,7 +31,7 @@ To use priority and preemption in Kubernetes 1.8, follow these steps:
1. Add one or more PriorityClasses.
1. Create Pods with `PriorityClassName` set to one of the added PriorityClasses.
Of course you do not need to create the Pods directly; normally you would add
Of course you do not need to create the Pods directly; normally you would add
`PriorityClassName` to the Pod template of a collection object like a Deployment.
The following sections provide more information about these steps.
@@ -39,18 +39,17 @@ The following sections provide more information about these steps.
## Enabling priority and preemption
Pod priority and preemption is disabled by default in Kubernetes 1.8.
To enable the feature, set this command-line flag for the API server
and the scheduler:
To enable the feature, set this command-line flag for the API server, scheduler and kubelet:
```
--feature-gates=PodPriority=true
```
Also set this flag for API server:
Also enable scheduling.k8s.io/v1alpha1 API and Priority [admission controller](/docs/admin/admission-controllers/) in API server:
```
--runtime-config=scheduling.k8s.io/v1alpha1=true
--runtime-config=scheduling.k8s.io/v1alpha1=true --admission-control=Controller-Foo,Controller-Bar,...,Priority
```
After the feature is enabled, you can create [PriorityClasses](#priorityclass)
@@ -67,7 +66,7 @@ cannot set PriorityClassName in new Pods.
A PriorityClass is a non-namespaced object that defines a mapping from a priority
class name to the integer value of the priority. The name is specified in the `name`
field of the PriorityClass object's metadata. The value is specified in the required
`value` field. The higher the value, the higher the priority.
`value` field. The higher the value, the higher the priority.
A PriorityClass object can have any 32-bit integer value smaller than or equal to
1 billion. Larger numbers are reserved for critical system Pods that should not
@@ -100,7 +99,7 @@ that use the name of the deleted PriorityClass.
### Example PriorityClass
```yaml
apiVersion: v1
apiVersion: scheduling.k8s.io/v1alpha1
kind: PriorityClass
metadata:
name: high-priority
@@ -140,11 +139,11 @@ spec:
When Pods are created, they go to a queue and wait to be scheduled. The scheduler
picks a Pod from the queue and tries to schedule it on a Node. If no Node is found
that satisfies all the specified requirements of the Pod, preemption logic is triggered
that satisfies all the specified requirements of the Pod, preemption logic is triggered
for the pending Pod. Let's call the pending pod P. Preemption logic tries to find a Node
where removal of one or more Pods with lower priority than P would enable P to be scheduled
on that Node. If such a Node is found, one or more lower priority Pods get
deleted from the Node. After the Pods are gone, P can be scheduled on the Node.
deleted from the Node. After the Pods are gone, P can be scheduled on the Node.
### Limitations of preemption (alpha version)
@@ -168,7 +167,7 @@ problematic in clusters with a high Pod creation rate.
We will address this problem in the beta version of Pod preemption. The solution
we plan to implement is
[provided here](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/pod-preemption.md#preemption-mechanics).
[provided here](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/scheduling/pod-preemption.md#preemption-mechanics).
#### PodDisruptionBudget is not supported
@@ -188,7 +187,7 @@ the answer to this question is yes: "If all the Pods with lower priority than
the pending Pod are removed from the Node, can the pending pod be scheduled on
the Node?"
**Note:** Preemption does not necessarily remove all lower-priority Pods. If the
**Note:** Preemption does not necessarily remove all lower-priority Pods. If the
pending pod can be scheduled by removing fewer than all lower-priority Pods, then
only a portion of the lower-priority Pods are removed. Even so, the answer to the
preceding question must be yes. If the answer is no, the Node is not considered
@@ -204,10 +203,10 @@ or it might not. There is no guarantee that the pending Pod can be scheduled.
We might address this issue in future versions, but we don't have a clear plan yet.
We will not consider it a blocker for Beta or GA. Part
of the reason is that finding the set of lower-priority Pods that satisfy all
inter-Pod affinity rules is computationally expensive, and adds substantial
inter-Pod affinity rules is computationally expensive, and adds substantial
complexity to the preemption logic. Besides, even if preemption keeps the lower-priority
Pods to satisfy inter-Pod affinity, the lower priority Pods might be preempted
later by other Pods, which removes the benefits of having the complex logic of
later by other Pods, which removes the benefits of having the complex logic of
respecting inter-Pod affinity.
Our recommended solution for this problem is to create inter-Pod affinity only towards
@@ -231,7 +230,7 @@ If Pod Q were removed from its Node, the anti-affinity violation would be gone,
and Pod P could possibly be scheduled on Node N.
We may consider adding cross Node preemption in future versions if we find an
algorithm with reasonable performance. We cannot promise anything at this point,
algorithm with reasonable performance. We cannot promise anything at this point,
and cross Node preemption will not be considered a blocker for Beta or GA.
{% endcapture %}
docs/concepts/configuration/secret.md
+7 -4
View File
@@ -37,7 +37,7 @@ The automatic creation and use of API credentials can be disabled or overridden
if desired. However, if all you need to do is securely access the apiserver,
this is the recommended workflow.
See the [Service Account](/docs/user-guide/service-accounts) documentation for more
See the [Service Account](/docs/tasks/configure-pod-container/configure-service-account/) documentation for more
information on how Service Accounts work.
### Creating your own Secrets
@@ -121,7 +121,7 @@ The data field is a map. Its keys must match
[`DNS_SUBDOMAIN`](https://git.k8s.io/community/contributors/design-proposals/architecture/identifiers.md), except that leading dots are also
allowed. The values are arbitrary data, encoded using base64.
Create the secret using [`kubectl create`](/docs/user-guide/kubectl/v1.7/#create):
Create the secret using [`kubectl create`](/docs/user-guide/kubectl/{{page.version}}/#create):
```shell
$ kubectl create -f ./secret.yaml
@@ -343,7 +343,7 @@ projected to the pod can be as long as kubelet sync period + ttl of secrets cach
To use a secret in an environment variable in a pod:
1. Create a secret or use an existing one. Multiple pods can reference the same secret.
1. Modify your Pod definition in each container that you wish to consume the value of a secret key to add an environment variable for each secret key you wish to consume. The environment variable that consumes the secret key should populate the secret's name and key in `env[x].valueFrom.secretKeyRef`.
1. Modify your Pod definition in each container that you wish to consume the value of a secret key to add an environment variable for each secret key you wish to consume. The environment variable that consumes the secret key should populate the secret's name and key in `env[].valueFrom.secretKeyRef`.
1. Modify your image and/or command line so that the program looks for values in the specified environment variables
This is an example of a pod that uses secrets from environment variables:
@@ -406,7 +406,7 @@ See [Adding ImagePullSecrets to a service account](/docs/tasks/configure-pod-con
Manually created secrets (e.g. one containing a token for accessing a github account)
can be automatically attached to pods based on their service account.
See [Injecting Information into Pods Using a PodPreset](/docs/tasks/run-application/podpreset/) for a detailed explanation of that process.
See [Injecting Information into Pods Using a PodPreset](/docs/tasks/inject-data-application/podpreset/) for a detailed explanation of that process.
## Details
@@ -743,3 +743,6 @@ Pod level](#use-case-secret-visible-to-one-container-in-a-pod).
by impersonating the kubelet. It is a planned feature to only send secrets to
nodes that actually require them, to restrict the impact of a root exploit on a
single node.
**Note:** As of 1.7 [encryption of secret data at rest is supported](https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/).
{: .note}
docs/concepts/configuration/taint-and-toleration.md
+16 -12
View File
@@ -17,9 +17,12 @@ marks that the node should not accept any pods that do not tolerate the taints.
Tolerations are applied to pods, and allow (but do not require) the pods to schedule
onto nodes with matching taints.
* TOC
{:toc}
## Concepts
You add a taint to a node using [kubectl taint](/docs/user-guide/kubectl/v1.7/#taint).
You add a taint to a node using [kubectl taint](/docs/user-guide/kubectl/{{page.version}}/#taint).
For example,
```shell
@@ -191,22 +194,22 @@ support for representing node problems. In other words, the node controller
automatically taints a node when certain condition is true. The built-in taints
currently include:
* `node.alpha.kubernetes.io/notReady`: Node is not ready. This corresponds to
* `node.kubernetes.io/not-ready`: Node is not ready. This corresponds to
the NodeCondition `Ready` being "`False`".
* `node.alpha.kubernetes.io/unreachable`: Node is unreachable from the node
controller. This corresponds to the NodeCondition `Ready` being "`Unknown`".
* `node.kubernetes.io/outOfDisk`: Node becomes out of disk.
* `node.kubernetes.io/memoryPressure`: Node has memory pressure.
* `node.kubernetes.io/diskPressure`: Node has disk pressure.
* `node.kubernetes.io/networkUnavailable`: Node's network is unavailable.
* `node.kubernetes.io/out-of-disk`: Node becomes out of disk.
* `node.kubernetes.io/memory-pressure`: Node has memory pressure.
* `node.kubernetes.io/disk-pressure`: Node has disk pressure.
* `node.kubernetes.io/network-unavailable`: Node's network is unavailable.
* `node.cloudprovider.kubernetes.io/uninitialized`: When kubelet is started
with "external" cloud provider, it sets this taint on a node to mark it
as unusable. When a controller from the cloud-controller-manager initializes
this node, kubelet removes this taint.
When the `TaintBasedEvictions` alpha feature is enabled (you can do this by
including `TaintBasedEvictions=true` in `--feature-gates`, such as
`--feature-gates=FooBar=true,TaintBasedEvictions=true`), the taints are automatically
including `TaintBasedEvictions=true` in `--feature-gates` for Kubernetes controller manager,
such as `--feature-gates=FooBar=true,TaintBasedEvictions=true`), the taints are automatically
added by the NodeController (or kubelet) and the normal logic for evicting pods from nodes
based on the Ready NodeCondition is disabled.
(Note: To maintain the existing [rate limiting](/docs/concepts/architecture/nodes/)
@@ -230,9 +233,9 @@ tolerations:
```
Note that Kubernetes automatically adds a toleration for
`node.alpha.kubernetes.io/notReady` with `tolerationSeconds=300`
`node.kubernetes.io/not-ready` with `tolerationSeconds=300`
unless the pod configuration provided
by the user already has a toleration for `node.alpha.kubernetes.io/notReady`.
by the user already has a toleration for `node.kubernetes.io/not-ready`.
Likewise it adds a toleration for
`node.alpha.kubernetes.io/unreachable` with `tolerationSeconds=300`
unless the pod configuration provided
@@ -248,7 +251,7 @@ admission controller](https://git.k8s.io/kubernetes/plugin/pkg/admission/default
`NoExecute` tolerations for the following taints with no `tolerationSeconds`:
* `node.alpha.kubernetes.io/unreachable`
* `node.alpha.kubernetes.io/notReady`
* `node.kubernetes.io/not-ready`
This ensures that DaemonSet pods are never evicted due to these problems,
which matches the behavior when this feature is disabled.
@@ -256,7 +259,8 @@ which matches the behavior when this feature is disabled.
## Taint Nodes by Condition
Version 1.8 introduces an alpha feature that causes the node controller to create taints corresponding to
Node conditions. When this feature is enabled, the scheduler does not check conditions; instead the scheduler checks taints. This assures that conditions don't affect what's scheduled onto the Node. The user can choose to ignore some of the Node's problems (represented as conditions) by adding appropriate Pod tolerations.
Node conditions. When this feature is enabled (you can do this by including `TaintNodesByCondition=true` in the `--feature-gates` command line flag to the scheduler, such as
`--feature-gates=FooBar=true,TaintNodesByCondition=true`), the scheduler does not check Node conditions; instead the scheduler checks taints. This assures that Node conditions don't affect what's scheduled onto the Node. The user can choose to ignore some of the Node's problems (represented as Node conditions) by adding appropriate Pod tolerations.
To make sure that turning on this feature doesn't break DaemonSets, starting in version 1.8, the DaemonSet controller automatically adds the following `NoSchedule` tolerations to all daemons:
docs/concepts/containers/container-lifecycle-hooks.md
+1 -1
View File
@@ -65,7 +65,7 @@ the Container cannot reach a `running` state.
The behavior is similar for a `PreStop` hook.
If the hook hangs during execution,
the Pod phase stays in a `running` state and never reaches `failed`.
the Pod phase stays in a `Terminating` state and is killed after `terminationGracePeriodSeconds` of pod ends.
If a `PostStart` or `PreStop` hook fails,
it kills the Container.
docs/concepts/containers/images.md
+8 -10
View File
@@ -39,7 +39,7 @@ Credentials can be provided in several ways:
- Using Google Container Registry
- Per-cluster
- automatically configured on Google Compute Engine or Google Container Engine
- automatically configured on Google Compute Engine or Google Kubernetes Engine
- all pods can read the project's private registry
- Using AWS EC2 Container Registry (ECR)
- use IAM roles and policies to control access to ECR repositories
@@ -60,7 +60,7 @@ Each option is described in more detail below.
Kubernetes has native support for the [Google Container
Registry (GCR)](https://cloud.google.com/tools/container-registry/), when running on Google Compute
Engine (GCE). If you are running your cluster on GCE or Google Container Engine (GKE), simply
Engine (GCE). If you are running your cluster on GCE or Google Kubernetes Engine, simply
use the full image name (e.g. gcr.io/my_project/image:tag).
All pods in a cluster will have read access to images in this registry.
@@ -128,8 +128,7 @@ Once you have those variables filled in you can
### Configuring Nodes to Authenticate to a Private Repository
**Note:** if you are running on Google Container Engine (GKE), there will already be a `.dockercfg` on each node
with credentials for Google Container Registry. You cannot use this approach.
**Note:** if you are running on Google Kubernetes Engine, there will already be a `.dockercfg` on each node with credentials for Google Container Registry. You cannot use this approach.
**Note:** if you are running on AWS EC2 and are using the EC2 Container Registry (ECR), the kubelet on each node will
manage and update the ECR login credentials. You cannot use this approach.
@@ -199,8 +198,7 @@ It should also work for a private registry such as quay.io, but that has not bee
### Pre-pulling Images
**Note:** if you are running on Google Container Engine (GKE), there will already be a `.dockercfg` on each node
with credentials for Google Container Registry. You cannot use this approach.
**Note:** if you are running on Google Kubernetes Engine, there will already be a `.dockercfg` on each node with credentials for Google Container Registry. You cannot use this approach.
**Note:** this approach is suitable if you can control node configuration. It
will not work reliably on GCE, and any other cloud provider that does automatic
@@ -219,7 +217,7 @@ All pods will have read access to any pre-pulled images.
### Specifying ImagePullSecrets on a Pod
**Note:** This approach is currently the recommended approach for GKE, GCE, and any cloud-providers
**Note:** This approach is currently the recommended approach for Google Kubernetes Engine, GCE, and any cloud-providers
where node creation is automated.
Kubernetes supports specifying registry keys on a pod.
@@ -295,7 +293,7 @@ However, setting of this field can be automated by setting the imagePullSecrets
in a [serviceAccount](/docs/user-guide/service-accounts) resource.
You can use this in conjunction with a per-node `.docker/config.json`. The credentials
will be merged. This approach will work on Google Container Engine (GKE).
will be merged. This approach will work on Google Kubernetes Engine.
### Use Cases
@@ -305,7 +303,7 @@ common use cases and suggested solutions.
1. Cluster running only non-proprietary (e.g. open-source) images. No need to hide images.
- Use public images on the Docker hub.
- No configuration required.
- On GCE/GKE, a local mirror is automatically used for improved speed and availability.
- On GCE/Google Kubernetes Engine, a local mirror is automatically used for improved speed and availability.
1. Cluster running some proprietary images which should be hidden to those outside the company, but
visible to all cluster users.
- Use a hosted private [Docker registry](https://docs.docker.com/registry/).
@@ -313,7 +311,7 @@ common use cases and suggested solutions.
- Manually configure .docker/config.json on each node as described above.
- Or, run an internal private registry behind your firewall with open read access.
- No Kubernetes configuration is required.
- Or, when on GCE/GKE, use the project's Google Container Registry.
- Or, when on GCE/Google Kubernetes Engine, use the project's Google Container Registry.
- It will work better with cluster autoscaling than manual node configuration.
- Or, on a cluster where changing the node configuration is inconvenient, use `imagePullSecrets`.
1. Cluster with a proprietary images, a few of which require stricter access control.
docs/concepts/jobs/job.yaml
-15
View File
@@ -1,15 +0,0 @@
apiVersion: batch/v1
kind: Job
metadata:
name: pi
spec:
template:
metadata:
name: pi
spec:
containers:
- name: pi
image: perl
command: ["perl", "-Mbignum=bpi", "-wle", "print bpi(2000)"]
restartPolicy: Never
docs/concepts/overview/extending.md
+215
View File
@@ -0,0 +1,215 @@
---
title: Extending your Kubernetes Cluster
approvers:
- erictune
- lavalamp
- cheftako
- chenopis
---
{% capture overview %}
Kubernetes is highly configurable and extensible. As a result,
there is rarely a need to fork or submit patches to the Kubernetes
project code.
This guide describes the options for customizing a Kubernetes
cluster. It is aimed at {% glossary_tooltip text="Cluster Operators" term_id="cluster-operator" %} who want to
understand how to adapt their Kubernetes cluster to the needs of
their work environment. Developers who are prospective {% glossary_tooltip text="Platform
Developers" term_id="platform-developer" %} or Kubernetes Project {% glossary_tooltip text="Contributors" term_id="contributor" %} will also find it
useful as an introduction to what extension points and patterns
exist, and their trade-offs and limitations.
{% endcapture %}
{% capture body %}
## Overview
Customization approaches can be broadly divided into *configuration*, which only involves changing flags, local configuration files, or API resources; and *extensions*, which involve running additional programs or services. This document is primarily about extensions.
## Configuration
*Configuration files* and *flags* are documented in the Reference section of the online documentation, under each binary:
* [kubelet](/docs/admin/kubelet/)
* [kube-apiserver](/docs/admin/kube-apiserver/)
* [kube-controller-manager](/docs/admin/kube-controller-manager/)
* [kube-scheduler](/docs/admin/kube-scheduler/).
Flags and configuration files may not always be changeable in a hosted Kubernetes service or a distribution with managed installation. When they are changeable, they are usually only changeable by the cluster administrator. Also, they are subject to change in future Kubernetes versions, and setting them may require restarting processes. For those reasons, they should be used only when there are no other options.
*Built-in Policy APIs*, such as [ResourceQuota](/docs/concepts/policy/resource-quotas/), [PodSecurityPolicies](/docs/concepts/policy/pod-security-policy/), [NetworkPolicy](/docs/concepts/services-networking/network-policies/) and Role-based Access Control ([RBAC](/docs/admin/authorization/rbac/)), are built-in Kubernetes APIs. APIs are typically used with hosted Kubernetes services and with managed Kubernetes installations. They are declarative and use the same conventions as other Kubernetes resources like pods, so new cluster configuration can be repeatable and be managed the same way as applications. And, where they are stable, they enjoy a [defined support policy](/docs/reference/deprecation-policy/) like other Kubernetes APIs. For these reasons, they are preferred over *configuration files* and *flags* where suitable.
## Extensions
Extensions are software components that extend and deeply integrate with Kubernetes.
They adapt it to support new types and new kinds of hardware.
Most cluster administrators will use a hosted or distribution
instance of Kubernetes. As a result, most Kubernetes users will need to
install extensions and fewer will need to author new ones.
## Extension Patterns
Kubernetes is designed to be automated by writing client programs. Any
program that reads and/or writes to the Kubernetes API can provide useful
automation. *Automation* can run on the cluster or off it. By following
the guidance in this doc you can write highly available and robust automation.
Automation generally works with any Kubernetes cluster, including hosted
clusters and managed installations.
There is a specific pattern for writing client programs that work well with
Kubernetes called the *Controller* pattern. Controllers typically read an
object's `.spec`, possibly do things, and then update the object's `.status`.
A controller is a client of Kubernetes. When Kubernetes is the client and
calls out to a remote service, it is called a *Webhook*. The remote service
is called a *Webhook Backend*. Like Controllers, Webhooks do add a point of
failure.
In the webhook model, Kubernetes makes a network request to a remote service.
In the *Binary Plugin* model, Kubernetes executes a binary (program).
Binary plugins are used by the kubelet (e.g. [Flex Volume
Plugins](https://github.com/kubernetes/community/blob/master/contributors/devel/flexvolume.md)
and [Network
Plugins](/docs/concepts/cluster-administration/network-plugins/))
and by kubectl.
Below is a diagram showing how the extensions points interact with the
Kubernetes control plane.
<img src="https://docs.google.com/drawings/d/e/2PACX-1vQBRWyXLVUlQPlp7BvxvV9S1mxyXSM6rAc_cbLANvKlu6kCCf-kGTporTMIeG5GZtUdxXz1xowN7RmL/pub?w=960&h=720">
<!-- image source drawing https://docs.google.com/drawings/d/1muJ7Oxuj_7Gtv7HV9-2zJbOnkQJnjxq-v1ym_kZfB-4/edit?ts=5a01e054 -->
## Extension Points
This diagram shows the extension points in a Kubernetes system.
<img src="https://docs.google.com/drawings/d/e/2PACX-1vSH5ZWUO2jH9f34YHenhnCd14baEb4vT-pzfxeFC7NzdNqRDgdz4DDAVqArtH4onOGqh0bhwMX0zGBb/pub?w=425&h=809">
<!-- image source diagrams: https://docs.google.com/drawings/d/1k2YdJgNTtNfW7_A8moIIkij-DmVgEhNrn3y2OODwqQQ/view -->
1. Users often interact with the Kubernetes API using `kubectl`. [Kubectl plugins](docs/tasks/extend-kubectl/kubectl-plugins) extend the kubectl binary. They only affect the individual user's local environment, and so cannot enforce site-wide policies.
2. The apiserver handles all requests. Several types of extension points in the apiserver allow authenticating requests, or blocking them based on their content, editing content, and handling deletion. These are described in the [API Access Extensions](docs/concepts/overview/extending#api-access-extensions) section.
3. The apiserver serves various kinds of *resources*. *Built-in resource kinds*, like `pods`, are defined by the Kubernetes project and can't be changed. You can also add resources that you define, or that other projects have defined, called *Custom Resources*, as explained in the [Custom Resources](docs/concepts/overview/extending#custom-resources) section. Custom Resources are often used with API Access Extensions.
4. The Kubernetes scheduler decides which nodes to place pods on. There are several ways to extend scheduling. These are described in the [Scheduler Extensions](docs/concepts/overview/extending#shceduler-extensions) section.
5. Much of the behavior of Kubernetes is implemented by programs called Controllers which are clients of the API-Server. Controllers are often used in conjunction with Custom Resources.
6. The kubelet runs on servers, and helps pods appear like virtual servers with their own IPs on the cluster network. [Network Plugins](docs/concepts/overview/extending#network-plugins) allow for different implementations of pod networking.
7. The kubelet also mounts and unmounts volumes for containers. New types of storage can be supported via [Storage Plugins](docs/concepts/overview/extending#storage-plugins).
If you are unsure where to start, this flowchart can help. Note that some solutions may involve several types of extensions.
<img src="https://docs.google.com/drawings/d/e/2PACX-1vRWXNNIVWFDqzDY0CsKZJY3AR8sDeFDXItdc5awYxVH8s0OLherMlEPVUpxPIB1CSUu7GPk7B2fEnzM/pub?w=1440&h=1080">
<!-- image source drawing: https://docs.google.com/drawings/d/1sdviU6lDz4BpnzJNHfNpQrqI9F19QZ07KnhnxVrp2yg/edit -->
## API Extensions
### User-Defined Types
Consider adding a Custom Resource to Kubernetes if you want to define new controllers, application configuration objects or other declarative APIs, and to manage them using Kubernetes tools, such as `kubectl`.
Do not use a Custom Resource as data storage for application, user, or monitoring data.
For more about Custom Resources, see the [Custom Resources concept guide](/docs/concepts/api-extension/custom-resources.md).
### Combining New APIs with Automation
Often, when you add a new API, you also add a control loop that reads and/or writes the new APIs. When the combination of a Custom API and a control loop is used to manage a specific, usually stateful, application, this is called the *Operator* pattern. Custom APIs and control loops can also be used to control other resources, such as storage, policies, and so on.
### Changing Built-in Resources
When you extend the Kubernetes API by adding custom resources, the added resources always fall into a new API Groups. You cannot replace or change existing API groups.
Adding an API does not directly let you affect the behavior of existing APIs (e.g. Pods), but API Access Extensions do.
### API Access Extensions
When a request reaches the Kubernetes API Server, it is first Authenticated, then Authorized, then subject to various types of Admission Control. See [[Accessing the API](/docs/admin/accessing-the-api/)] for more on this flow.
Each of these steps offers extension points.
Kubernetes has several built-in authentication methods that it supports. It can also sit behind an authenticating proxy, and it can send a token from an Authorization header to a remote service for verification (a webhook). All of these methods are covered in the [Authentication documentation](/docs/admin/authentication/).
### Authentication
[Authentication](/docs/admin/authentication) maps headers or certificates in all requests to a username for the client making the request.
Kubernetes provides several built-in authentication methods, and an [Authentication webhook](/docs/admin/authentication/#webhook-token-authentication) method if those don't meet your needs.
### Authorization
[Authorization](/docs/admin/authorization/webhook/) determines whether specific users can read, write, and do other operations on API resources. It just works at the level of whole resources -- it doesn't discriminate based on arbitrary object fields. If the built-in authorization options don't meet your needs, and [Authorization webhook](/docs/admin/authorization/webhook/) allows calling out to user-provided code to make an authorization decision.
### Dynamic Admission Control
After a request is authorized, if it is a write operation, it also t goes through [Admission Control](/docs/admin/admission-controllers/) steps. In addition to the built-in steps, there are several extensions:
* The [Image Policy webhook](/docs/admin/admission-controllers/#imagepolicywebhook) restricts what images can be run in containers.
* To make arbitrary admission control decisions, a general [Admission webhook](/docs/admin/extensible-admission-controllers/#external-admission-webhooks) can be used. Admission Webhooks can reject creations or updates.
* [Initializers](/docs/admin/extensible-admission-controllers/#initializers) are controllers that can modify objects before they are created. Initializers can modify initial object creations but cannot affect updates to objects. Initializers can also reject objects.
## Infrastructure Extensions
### Storage Plugins
[Flex Volumes](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/storage/flexvolume-deployment.md
) allow users to mount volume types without built-in support by having the
Kubelet call a Binary Plugin to mount the volume.
### Device Plugins
Device plugins allow a node to discover new Node resources (in addition to the
builtin ones like cpu and memory) via a [Device
Plugin](/docs/concepts/cluster-administration/device-plugins/).
### Network Plugins
Different networking fabrics can be supported via node-level [Network Plugins](/docs/admin/network-plugins/).
### Scheduler Extensions
The scheduler is a special type of controller that watches pods, and assigns
pods to nodes. The default scheduler can be be replaced entirely, while
continuing to use other Kubernetes components, or [multiple
schedulers](/docs/tasks/administer-cluster/configure-multiple-schedulers/)
can run at the same time.
This is a significant undertaking, and almost all Kubernetes users find they
do not need to modify the scheduler.
The scheduler also supports a
[webhook](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/scheduling/scheduler_extender.md)
that permits a webhook backend (scheduler extension) to filter and prioritize
the nodes chosen for a pod.
{% endcapture %}
{% capture whatsnext %}
* Learn more about [Custom Resources](/docs/concepts/api-extension/custom-resources/)
* Learn about [Dynamic admission control](/docs/admin/extensible-admission-controller)
* Learn more about Infrastructure extensions
* [Network Plugins](/docs/concepts/cluster-administration/network-plugin)
* [Device Plugins](/docs/concepts/cluster-administration/device-plugins.md)
* Learn about [kubectl plugins](/docs/tasks/extend-kubectl/kubectl-plugin)
* See examples of Automation
* [List of Operators](https://github.com/coreos/awesome-kubernetes-extensions)
{% endcapture %}
{% include templates/concept.md %}
docs/concepts/overview/kubernetes-api.md
+3 -3
View File
@@ -10,7 +10,7 @@ API endpoints, resource types and samples are described in [API Reference](/docs
Remote access to the API is discussed in the [access doc](/docs/admin/accessing-the-api).
The Kubernetes API also serves as the foundation for the declarative configuration schema for the system. The [Kubectl](/docs/user-guide/kubectl) command-line tool can be used to create, update, delete, and get API objects.
The Kubernetes API also serves as the foundation for the declarative configuration schema for the system. The [kubectl](/docs/user-guide/kubectl/) command-line tool can be used to create, update, delete, and get API objects.
Kubernetes also stores its serialized state (currently in [etcd](https://coreos.com/docs/distributed-configuration/getting-started-with-etcd/)) in terms of the API resources.
@@ -73,8 +73,8 @@ The API group is specified in a REST path and in the `apiVersion` field of a ser
Currently there are several API groups in use:
1. The "core" (oftentimes called "legacy", due to not having explicit group name) group, which is at
REST path `/api/v1` and is not specified as part of the `apiVersion` field, e.g. `apiVersion: v1`.
1. The *core* group, often referred to as the *legacy group*, is at the REST path `/api/v1` and uses `apiVersion: v1`.
1. The named groups are at REST path `/apis/$GROUP_NAME/$VERSION`, and use `apiVersion: $GROUP_NAME/$VERSION`
(e.g. `apiVersion: batch/v1`). Full list of supported API groups can be seen in [Kubernetes API reference](/docs/reference/).
docs/concepts/overview/what-is-kubernetes.md
+2 -2
View File
@@ -53,7 +53,7 @@ Summary of container benefits:
* **Environmental consistency across development, testing, and production**:
Runs the same on a laptop as it does in the cloud.
* **Cloud and OS distribution portability**:
Runs on Ubuntu, RHEL, CoreOS, on-prem, Google Container Engine, and anywhere else.
Runs on Ubuntu, RHEL, CoreOS, on-prem, Google Kubernetes Engine, and anywhere else.
* **Application-centric management**:
Raises the level of abstraction from running an OS on virtual hardware to run an application on an OS using logical resources.
* **Loosely coupled, distributed, elastic, liberated [micro-services](https://martinfowler.com/articles/microservices.html)**:
@@ -121,7 +121,7 @@ The name **Kubernetes** originates from Greek, meaning *helmsman* or *pilot*, an
{% endcapture %}
{% capture whatsnext %}
* Ready to [Get Started](/docs/getting-started-guides/)?
* Ready to [Get Started](/docs/setup/)?
* For more details, see the [Kubernetes Documentation](/docs/home/).
{% endcapture %}
{% include templates/concept.md %}
docs/concepts/overview/working-with-objects/kubernetes-objects.md
+3 -3
View File
@@ -36,10 +36,10 @@ Here's an example `.yaml` file that shows the required fields and object spec fo
{% include code.html language="yaml" file="nginx-deployment.yaml" ghlink="/docs/concepts/overview/working-with-objects/nginx-deployment.yaml" %}
One way to create a Deployment using a `.yaml` file like the one above is to use the [`kubectl create`](/docs/user-guide/kubectl/v1.7/#create) command in the `kubectl` command-line interface, passing the `.yaml` file as an argument. Here's an example:
One way to create a Deployment using a `.yaml` file like the one above is to use the [`kubectl create`](/docs/user-guide/kubectl/{{page.version}}/#create) command in the `kubectl` command-line interface, passing the `.yaml` file as an argument. Here's an example:
```shell
$ kubectl create -f docs/user-guide/nginx-deployment.yaml --record
$ kubectl create -f https://k8s.io/docs/user-guide/nginx-deployment.yaml --record
```
The output is similar to this:
@@ -61,7 +61,7 @@ You'll also need to provide the object `spec` field. The precise format of the o
{% endcapture %}
{% capture whatsnext %}
* Learn about the most important basic Kubernetes objects, such as [Pod](/docs/concepts/abstractions/pod/).
* Learn about the most important basic Kubernetes objects, such as [Pod](/docs/concepts/workloads/pods/pod-overview/).
{% endcapture %}
{% include templates/concept.md %}
docs/concepts/overview/working-with-objects/labels.md
+3 -3
View File
@@ -16,7 +16,7 @@ Each object can have a set of key/value labels defined. Each Key must be unique
}
```
We'll eventually index and reverse-index labels for efficient queries and watches, use them to sort and group in UIs and CLIs, etc. We don't want to pollute labels with non-identifying, especially large and/or structured, data. Non-identifying information should be recorded using [annotations](/docs/user-guide/annotations).
We'll eventually index and reverse-index labels for efficient queries and watches, use them to sort and group in UIs and CLIs, etc. We don't want to pollute labels with non-identifying, especially large and/or structured, data. Non-identifying information should be recorded using [annotations](/docs/concepts/overview/working-with-objects/annotations/).
* TOC
{:toc}
@@ -51,7 +51,7 @@ Unlike [names and UIDs](/docs/user-guide/identifiers), labels do not provide uni
Via a _label selector_, the client/user can identify a set of objects. The label selector is the core grouping primitive in Kubernetes.
The API currently supports two types of selectors: _equality-based_ and _set-based_.
A label selector can be made of multiple _requirements_ which are comma-separated. In the case of multiple requirements, all must be satisfied so the comma separator acts as an _AND_ logical operator.
A label selector can be made of multiple _requirements_ which are comma-separated. In the case of multiple requirements, all must be satisfied so the comma separator acts as a logical _AND_ (`&&`) operator.
An empty label selector (that is, one with zero requirements) selects every object in the collection.
@@ -170,4 +170,4 @@ selector:
#### Selecting sets of nodes
One use case for selecting over labels is to constrain the set of nodes onto which a pod can schedule.
See the documentation on [node selection](/docs/user-guide/node-selection) for more information.
See the documentation on [node selection](/docs/concepts/configuration/assign-pod-node/) for more information.
docs/concepts/overview/working-with-objects/names.md
+1 -1
View File
@@ -7,7 +7,7 @@ title: Names
All objects in the Kubernetes REST API are unambiguously identified by a Name and a UID.
For non-unique user-provided attributes, Kubernetes provides [labels](/docs/user-guide/labels) and [annotations](/docs/user-guide/annotations).
For non-unique user-provided attributes, Kubernetes provides [labels](/docs/user-guide/labels) and [annotations](/docs/concepts/overview/working-with-objects/annotations/).
## Names
docs/concepts/overview/working-with-objects/namespaces.md
+5 -3
View File
@@ -18,7 +18,7 @@ need the features they provide.
Namespaces provide a scope for names. Names of resources need to be unique within a namespace, but not across namespaces.
Namespaces are a way to divide cluster resources between multiple uses (via [resource quota](/docs/concepts/policy/resource-quotas/)).
Namespaces are a way to divide cluster resources between multiple users (via [resource quota](/docs/concepts/policy/resource-quotas/)).
In future versions of Kubernetes, objects in the same namespace will have the same
access control policies by default.
@@ -41,12 +41,14 @@ $ kubectl get namespaces
NAME STATUS AGE
default Active 1d
kube-system Active 1d
kube-public Active 1d
```
Kubernetes starts with two initial namespaces:
Kubernetes starts with three initial namespaces:
* `default` The default namespace for objects with no other namespace
* `kube-system` The namespace for objects created by the Kubernetes system
* `kube-public` The namespace is created automatically and readable by all users (including those not authenticated). This namespace is mostly reserved for cluster usage, in case that some resources should be visible and readable publicly throughout the whole cluster. The public aspect of this namespace is only a convention, not a requirement.
### Setting the namespace for a request
@@ -72,7 +74,7 @@ $ kubectl config view | grep namespace:
## Namespaces and DNS
When you create a [Service](/docs/user-guide/services), it creates a corresponding [DNS entry](/docs/admin/dns).
When you create a [Service](/docs/user-guide/services), it creates a corresponding [DNS entry](/docs/concepts/services-networking/dns-pod-service/).
This entry is of the form `<service-name>.<namespace-name>.svc.cluster.local`, which means
that if a container just uses `<service-name>`, it will resolve to the service which
is local to a namespace. This is useful for using the same configuration across
docs/concepts/overview/working-with-objects/nginx-deployment.yaml
+4 -1
View File
@@ -1,9 +1,12 @@
apiVersion: apps/v1beta1
apiVersion: apps/v1beta2 # for versions before 1.8.0 use apps/v1beta1
kind: Deployment
metadata:
name: nginx-deployment
spec:
replicas: 3
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
docs/concepts/policy/pod-security-policy.md
+27 -5
View File
@@ -37,8 +37,9 @@ administrator to control the following:
| Configuring allowable supplemental groups | [`supplementalGroups`](#supplementalgroups) |
| Allocating an FSGroup that owns the pod's volumes | [`fsGroup`](#fsgroup) |
| Requiring the use of a read only root file system | `readOnlyRootFilesystem` |
| Running of a container that allow privilege escalation from its parent | [`allowPrivilegeEscalation`](#allowPrivilegeEscalation) |
| Control whether a process can gain more privileges than its parent process | [`defaultAllowPrivilegeEscalation`](#defaultAllowPrivilegeEscalation) |
| Running of a container that allow privilege escalation from its parent | [`allowPrivilegeEscalation`](#allowprivilegeescalation) |
| Control whether a process can gain more privileges than its parent process | [`defaultAllowPrivilegeEscalation`](#defaultallowprivilegeescalation) |
| Whitelist of allowed host paths | [`allowedHostPaths`](#allowedhostpaths) |
_Pod Security Policies_ are comprised of settings and strategies that
control the security features a pod has access to. These settings fall
@@ -141,10 +142,31 @@ allows privilege escalation so as to not break setuid binaries. Setting it to `f
ensures that no child process of a container can gain more privileges than
its parent.
### AllowedHostPaths
This specifies a whitelist of host paths that are allowed to be used by Pods.
An empty list means there is no restriction on host paths used.
Each item in the list must specify a string value named `pathPrefix` that
defines a host path to match. The value cannot be "`*`" though.
An example is shown below:
```yaml
apiVersion: extensions/v1beta1
kind: PodSecurityPolicy
metadata:
name: custom-paths
spec:
allowedHostPaths:
# This allows "/foo", "/foo/", "/foo/bar" etc., but
# disallows "/fool", "/etc/foo" etc.
- pathPrefix: "/foo"
```
## Admission
_Admission control_ with `PodSecurityPolicy` allows for control over the
creation and modification of resources based on the capabilities allowed in the cluster.
[_Admission control_ with `PodSecurityPolicy`](/docs/admin/admission-controllers/#podsecuritypolicy)
allows for control over the creation and modification of resources based on the
capabilities allowed in the cluster.
Admission uses the following approach to create the final security context for
the pod:
@@ -210,7 +232,7 @@ In order to use Pod Security Policies in your cluster you must ensure the
following
1. You have enabled the API type `extensions/v1beta1/podsecuritypolicy` (only for versions prior 1.6)
1. You have enabled the admission controller `PodSecurityPolicy`
1. [You have enabled the admission control plug-in `PodSecurityPolicy`](/docs/admin/admission-controllers/#how-do-i-turn-on-an-admission-control-plug-in)
1. You have defined your policies
## Working With RBAC
docs/concepts/service-catalog/index.md
+234
View File
@@ -0,0 +1,234 @@
---
title: Service Catalog
approvers:
- chenopis
---
{% capture overview %}
{% glossary_definition term_id="service-catalog" length="all" prepend="Service Catalog is " %}
A *Service Broker*, as defined by the [Open Service Broker API spec](https://github.com/openservicebrokerapi/servicebroker/blob/v2.13/spec.md), is an endpoint for a set of Managed Services offered and maintained by a third-party, which could be a cloud provider such as AWS, GCP, or Azure.
Some examples of *Managed Services* are Microsoft Azure Cloud Queue, Amazon Simple Queue Service, and Google Cloud Pub/Sub, but they can be any software offering that can be used by an application.
Using Service Catalog, a {% glossary_tooltip text="Cluster Operator" term_id="cluster-operator" %} can browse the list of {% glossary_tooltip text="Managed Services" term_id="managed-service" %} offered by a {% glossary_tooltip text="Service Brokers" term_id="service-broker" %}, provision an instance of a Managed Service, and bind with it to make it available to an application within the Kubernetes cluster.
{% endcapture %}
{% capture body %}
## Example use case
An {% glossary_tooltip text="Application Developer" term_id="application-developer" %} wants to use message queuing as part of their application running in a Kubernetes cluster.
However, they do not want to deal with the overhead of setting such a service up and administering it themselves.
Fortunately, there is a cloud provider that offers message queuing as a *Managed Service* through their *Service Broker*.
A {% glossary_tooltip text="Cluster Operator" term_id="cluster-operator" %} can setup Service Catalog and use it to communicate with the cloud provider's {% glossary_tooltip text="Service Broker" term_id="service-broker" %} to provision an instance of the message queuing service and make it available to the application within the Kubernetes cluster.
The {% glossary_tooltip text="Application Developer" term_id="application-developer" %} therefore does not need to concern themselves with the implementation details or management of the message queue.
Their application can simply use it as a service.
## Architecture
Service Catalog uses the [Open Service Broker API](https://github.com/openservicebrokerapi/servicebroker) to communicate with Service Brokers, acting as an intermediary for the Kubernetes API Server in order to negotiate the initial provisioning and retrieve the credentials necessary for the application to use a Managed Service.
It is implemented as an extension API server and a controller manager, using Etcd for storage. It also uses the [aggregation layer](/docs/concepts/api-extension/apiserver-aggregation/) available in Kubernetes 1.7+ to present its API.
<br>
![Service Catalog Architecture](/images/docs/service-catalog-architecture.svg)
### API Resources
Service Catalog installs the `servicecatalog.k8s.io` API and provides the following Kubernetes resources:
* `ClusterServiceBroker`: An in-cluster representation of a Service Broker, encapsulating its server connection details.
These are created and managed by Cluster Operators who wish to use that broker server to make new types of Managed Services available within their cluster.
* `ClusterServiceClass`: A Managed Service offered by a particular Service Broker.
When a new `ClusterServiceBroker` resource is added to the cluster, the Service Catalog controller connects to the Service Broker to obtain a list of available Managed Services. It then creates a new `ClusterServiceClass` resource corresponding to each Managed Service.
* `ClusterServicePlan`: A specific offering of a Managed Service. For example, a Managed Service may have different plans available, such as a free tier or paid tier, or it may have different configuration options, such as using SSD storage or having more resources. Similar to `ClusterServiceClass`, when a new `ClusterServiceBroker` is added to the cluster, the Service Catalog creates a new `ClusterServicePlan` resource corresponding to each Service Plan available for each Managed Service.
* `ServiceInstance`: A provisioned instance of a `ClusterServiceClass`.
These are created by Cluster Operators to make a specific instance of a Managed Service available for use by one or more in-cluster applications.
When a new `ServiceInstance` resource is created, the Service Catalog controller will connect to the appropriate Service Broker and instruct it to provision the service instance.
* `ServiceBinding`: Access credentials to a `ServiceInstance`.
These are created by Cluster Operators who want their applications to make use of a Service `ServiceInstance`.
Upon creation, the Service Catalog controller will create a Kubernetes `Secret` containing connection details and credentials for the Service Instance, which can be mounted into Pods.
### Authentication
Service Catalog supports these methods of authentication:
* Basic (username/password)
* [OAuth 2.0 Bearer Token](https://tools.ietf.org/html/rfc6750)
## Usage
A {% glossary_tooltip text="Cluster Operator" term_id="cluster-operator" %} can use the Service Catalog API Resources to provision Managed Services and make them available within a Kubernetes cluster. The steps involved are:
1. Listing the Managed Services and Service Plans available from a Service Broker.
1. Provisioning a new instance of the Managed Service.
1. Binding to the Managed Service, which returns the connection credentials.
1. Mapping the connection credentials into the application.
### Listing Managed Services and Service Plans
First, a {% glossary_tooltip text="Cluster Operator" term_id="cluster-operator" %} must create a `ClusterServiceBroker` resource within the `servicecatalog.k8s.io` group. This resource contains the URL and connection details necessary to access a Service Broker endpoint.
This is an example of a `ClusterServiceBroker` resource:
```yaml
apiVersion: servicecatalog.k8s.io/v1beta1
kind: ClusterServiceBroker
metadata:
name: cloud-broker
spec:
# Points to the endpoint of a Service Broker. (This example is not a working URL.)
url: https://servicebroker.somecloudprovider.com/v1alpha1/projects/service-catalog/brokers/default
#####
# Additional values can be added here, which may be used to communicate
# with the Service Broker, such as bearer token info or a caBundle for TLS.
#####
```
The following is a sequence diagram illustrating the steps involved in listing Managed Services and Plans available from a Service Broker:
![List Services](/images/docs/service-catalog-list.svg){:height="80%" width="80%"}
1. Once the `ClusterServiceBroker` resource is added to Service Catalog, it triggers a *List Services* call to the external Service Broker.
1. The Service Broker returns a list of available Managed Services and Service Plans, which are cached locally in `ClusterServiceClass` and `ClusterServicePlan` resources.
1. A {% glossary_tooltip text="Cluster Operator" term_id="cluster-operator" %} can then get the list of available Managed Services using the following command:
kubectl get clusterserviceclasses -o=custom-columns=SERVICE\ NAME:.metadata.name,EXTERNAL\ NAME:.spec.externalName
It should output a list of service names with a format similar to:
SERVICE NAME EXTERNAL NAME
4f6e6cf6-ffdd-425f-a2c7-3c9258ad2468 cloud-provider-service
... ...
They can also view the Service Plans available using the following command:
kubectl get clusterserviceplans -o=custom-columns=PLAN\ NAME:.metadata.name,EXTERNAL\ NAME:.spec.externalName
It should output a list of plan names with a format similar to:
PLAN NAME EXTERNAL NAME
86064792-7ea2-467b-af93-ac9694d96d52 service-plan-name
... ...
### Provisioning a new instance
A {% glossary_tooltip text="Cluster Operator" term_id="cluster-operator" %} can initiate the provisioning of a new instance by creating a `ServiceInstance` resource.
This is an example of a `ServiceInstance` resource:
```yaml
apiVersion: servicecatalog.k8s.io/v1beta1
kind: ServiceInstance
metadata:
name: cloud-queue-instance
namespace: cloud-apps
spec:
# References one of the previously returned services
clusterServiceClassExternalName: cloud-provider-service
clusterServicePlanExternalName: service-plan-name
#####
# Additional parameters can be added here,
# which may be used by the Service Broker.
#####
```
The following sequence diagram illustrates the steps involved in provisioning a new instance of a Managed Service:
![Provision a Service](/images/docs/service-catalog-provision.svg){:height="80%" width="80%"}
1. When the `ServiceInstance` resource is created, Service Catalog initiates a *Provision Instance* call to the external Service Broker.
1. The Service Broker creates a new instance of the Managed Service and returns an HTTP response.
1. A {% glossary_tooltip text="Cluster Operator" term_id="cluster-operator" %} can then check the status of the instance to see if it is ready.
### Binding to a Managed Service
After a new instance has been provisioned, a {% glossary_tooltip text="Cluster Operator" term_id="cluster-operator" %} must bind to the Managed Service to get the connection credentials and service account details necessary for the application to use the service. This is done by creating a `ServiceBinding` resource.
The following is an example of a `ServiceBinding` resource:
```yaml
apiVersion: servicecatalog.k8s.io/v1beta1
kind: ServiceBinding
metadata:
name: cloud-queue-binding
namespace: cloud-apps
spec:
instanceRef:
name: cloud-queue-instance
#####
# Additional information can be added here, such as a secretName or
# service account parameters, which may be used by the Service Broker.
#####
```
The following sequence diagram illustrates the steps involved in binding to a Managed Service instance:
![Bind to a Managed Service](/images/docs/service-catalog-bind.svg){:height="80%" width="80%"}
1. After the `ServiceBinding` is created, Service Catalog makes a *Bind Instance* call to the external Service Broker.
1. The Service Broker enables the application permissions/roles for the appropriate service account.
1. The Service Broker returns the information necessary to connect and access the Managed Service instance. This is provider and service-specific so the information returned may differ between Service Providers and their Managed Services.
### Mapping the connection credentials
After binding, the final step involves mapping the connection credentials and service-specific information into the application.
These pieces of information are stored in secrets that the application in the cluster can access and use to connect directly with the Managed Service.
<br>
![Map connection credentials](/images/docs/service-catalog-map.svg)
#### Pod Configuration File
One method to perform this mapping is to use a declarative Pod configuration.
The following example describes how to map service account credentials into the application. A key called `sa-key` is stored in a volume named `provider-cloud-key`, and the application mounts this volume at `/var/secrets/provider/key.json`. The environment variable `GOOGLE_APPLICATION_CREDENTIALS` is mapped from the value of the mounted file.
```yaml
...
spec:
volumes:
- name: provider-cloud-key
secret:
secretName: sa-key
containers:
...
volumeMounts:
- name: provider-cloud-key
mountPath: /var/secrets/provider
env:
- name: PROVIDER_APPLICATION_CREDENTIALS
value: "/var/secrets/provider/key.json"
```
The following example describes how to map secret values into application environment variables. In this example, the messaging queue topic name is mapped from a secret named `provider-queue-credentials` with a key named `topic` to the environment variable `TOPIC`.
```yaml
...
env:
- name: "TOPIC"
valueFrom:
secretKeyRef:
name: provider-queue-credentials
key: topic
```
{% endcapture %}
{% capture whatsnext %}
* If you are familiar with {% glossary_tooltip text="Helm Charts" term_id="helm-chart" %}, [install Service Catalog using Helm](/docs/tasks/service-catalog/install-service-catalog-using-helm/) into your Kubernetes cluster. Alternatively, you can [install Service Catalog using the SC tool](/docs/tasks/service-catalog/install-service-catalog-using-sc/).
* View [sample service brokers](https://github.com/openservicebrokerapi/servicebroker/blob/master/gettingStarted.md#sample-service-brokers).
* Explore the [kubernetes-incubator/service-catalog](https://github.com/kubernetes-incubator/service-catalog) project.
{% endcapture %}
{% include templates/concept.md %}
docs/concepts/services-networking/connect-applications-service.md
+32 -6
View File
@@ -168,9 +168,9 @@ Till now we have only accessed the nginx server from within the cluster. Before
* Self signed certificates for https (unless you already have an identity certificate)
* An nginx server configured to use the certificates
* A [secret](/docs/user-guide/secrets) that makes the certificates accessible to pods
* A [secret](/docs/concepts/configuration/secret/) that makes the certificates accessible to pods
You can acquire all these from the [nginx https example](https://github.com/kubernetes/examples/tree/{{page.githubbranch}}/staging/https-nginx/), in short:
You can acquire all these from the [nginx https example](https://github.com/kubernetes/examples/tree/{{page.githubbranch}}/staging/https-nginx/). This requires having go and make tools installed. If you don't want to install those, then follow the manual steps later. In short:
```shell
$ make keys secret KEY=/tmp/nginx.key CERT=/tmp/nginx.crt SECRET=/tmp/secret.json
@@ -181,6 +181,36 @@ NAME TYPE DATA AGE
default-token-il9rc kubernetes.io/service-account-token 1 1d
nginxsecret Opaque 2 1m
```
Following are the manual steps to follow in case you run into problems running make (on windows for example):
```shell
#create a public private key pair
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /d/tmp/nginx.key -out /d/tmp/nginx.crt -subj "/CN=nginxsvc/O=nginxsvc"
#convert the keys to base64 encoding
cat /d/tmp/nginx.crt | base 64
cat /d/tmp/nginx.key | base 64
```
Use the output from the previous commands to create a yaml file as follows. The base64 encoded value should all be on a single line.
```yaml
apiVersion: "v1"
kind: "Secret"
metadata:
name: "nginxsecret"
namespace: "default"
data:
nginx.crt: "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURIekNDQWdlZ0F3SUJBZ0lKQUp5M3lQK0pzMlpJTUEwR0NTcUdTSWIzRFFFQkJRVUFNQ1l4RVRBUEJnTlYKQkFNVENHNW5hVzU0YzNaak1SRXdEd1lEVlFRS0V3aHVaMmx1ZUhOMll6QWVGdzB4TnpFd01qWXdOekEzTVRKYQpGdzB4T0RFd01qWXdOekEzTVRKYU1DWXhFVEFQQmdOVkJBTVRDRzVuYVc1NGMzWmpNUkV3RHdZRFZRUUtFd2h1CloybHVlSE4yWXpDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBSjFxSU1SOVdWM0IKMlZIQlRMRmtobDRONXljMEJxYUhIQktMSnJMcy8vdzZhU3hRS29GbHlJSU94NGUrMlN5ajBFcndCLzlYTnBwbQppeW1CL3JkRldkOXg5UWhBQUxCZkVaTmNiV3NsTVFVcnhBZW50VWt1dk1vLzgvMHRpbGhjc3paenJEYVJ4NEo5Ci82UVRtVVI3a0ZTWUpOWTVQZkR3cGc3dlVvaDZmZ1Voam92VG42eHNVR0M2QURVODBpNXFlZWhNeVI1N2lmU2YKNHZpaXdIY3hnL3lZR1JBRS9mRTRqakxCdmdONjc2SU90S01rZXV3R0ljNDFhd05tNnNTSzRqYUNGeGpYSnZaZQp2by9kTlEybHhHWCtKT2l3SEhXbXNhdGp4WTRaNVk3R1ZoK0QrWnYvcW1mMFgvbVY0Rmo1NzV3ajFMWVBocWtsCmdhSXZYRyt4U1FVQ0F3RUFBYU5RTUU0d0hRWURWUjBPQkJZRUZPNG9OWkI3YXc1OUlsYkROMzhIYkduYnhFVjcKTUI4R0ExVWRJd1FZTUJhQUZPNG9OWkI3YXc1OUlsYkROMzhIYkduYnhFVjdNQXdHQTFVZEV3UUZNQU1CQWY4dwpEUVlKS29aSWh2Y05BUUVGQlFBRGdnRUJBRVhTMW9FU0lFaXdyMDhWcVA0K2NwTHI3TW5FMTducDBvMm14alFvCjRGb0RvRjdRZnZqeE04Tzd2TjB0clcxb2pGSW0vWDE4ZnZaL3k4ZzVaWG40Vm8zc3hKVmRBcStNZC9jTStzUGEKNmJjTkNUekZqeFpUV0UrKzE5NS9zb2dmOUZ3VDVDK3U2Q3B5N0M3MTZvUXRUakViV05VdEt4cXI0Nk1OZWNCMApwRFhWZmdWQTRadkR4NFo3S2RiZDY5eXM3OVFHYmg5ZW1PZ05NZFlsSUswSGt0ejF5WU4vbVpmK3FqTkJqbWZjCkNnMnlwbGQ0Wi8rUUNQZjl3SkoybFIrY2FnT0R4elBWcGxNSEcybzgvTHFDdnh6elZPUDUxeXdLZEtxaUMwSVEKQ0I5T2wwWW5scE9UNEh1b2hSUzBPOStlMm9KdFZsNUIyczRpbDlhZ3RTVXFxUlU9Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K"
nginx.key: "LS0tLS1CRUdJTiBQUklWQVRFIEtFWS0tLS0tCk1JSUV2UUlCQURBTkJna3Foa2lHOXcwQkFRRUZBQVNDQktjd2dnU2pBZ0VBQW9JQkFRQ2RhaURFZlZsZHdkbFIKd1V5eFpJWmVEZWNuTkFhbWh4d1NpeWF5N1AvOE9ta3NVQ3FCWmNpQ0RzZUh2dGtzbzlCSzhBZi9WemFhWm9zcApnZjYzUlZuZmNmVUlRQUN3WHhHVFhHMXJKVEVGSzhRSHA3VkpMcnpLUC9QOUxZcFlYTE0yYzZ3MmtjZUNmZitrCkU1bEVlNUJVbUNUV09UM3c4S1lPNzFLSWVuNEZJWTZMMDUrc2JGQmd1Z0ExUE5JdWFubm9UTWtlZTRuMG4rTDQKb3NCM01ZUDhtQmtRQlAzeE9JNHl3YjREZXUraURyU2pKSHJzQmlIT05Xc0RadXJFaXVJMmdoY1kxeWIyWHI2UAozVFVOcGNSbC9pVG9zQngxcHJHclk4V09HZVdPeGxZZmcvbWIvNnBuOUYvNWxlQlkrZStjSTlTMkQ0YXBKWUdpCkwxeHZzVWtGQWdNQkFBRUNnZ0VBZFhCK0xkbk8ySElOTGo5bWRsb25IUGlHWWVzZ294RGQwci9hQ1Zkank4dlEKTjIwL3FQWkUxek1yall6Ry9kVGhTMmMwc0QxaTBXSjdwR1lGb0xtdXlWTjltY0FXUTM5SjM0VHZaU2FFSWZWNgo5TE1jUHhNTmFsNjRLMFRVbUFQZytGam9QSFlhUUxLOERLOUtnNXNrSE5pOWNzMlY5ckd6VWlVZWtBL0RBUlBTClI3L2ZjUFBacDRuRWVBZmI3WTk1R1llb1p5V21SU3VKdlNyblBESGtUdW1vVlVWdkxMRHRzaG9reUxiTWVtN3oKMmJzVmpwSW1GTHJqbGtmQXlpNHg0WjJrV3YyMFRrdWtsZU1jaVlMbjk4QWxiRi9DSmRLM3QraTRoMTVlR2ZQegpoTnh3bk9QdlVTaDR2Q0o3c2Q5TmtEUGJvS2JneVVHOXBYamZhRGR2UVFLQmdRRFFLM01nUkhkQ1pKNVFqZWFKClFGdXF4cHdnNzhZTjQyL1NwenlUYmtGcVFoQWtyczJxWGx1MDZBRzhrZzIzQkswaHkzaE9zSGgxcXRVK3NHZVAKOWRERHBsUWV0ODZsY2FlR3hoc0V0L1R6cEdtNGFKSm5oNzVVaTVGZk9QTDhPTm1FZ3MxMVRhUldhNzZxelRyMgphRlpjQ2pWV1g0YnRSTHVwSkgrMjZnY0FhUUtCZ1FEQmxVSUUzTnNVOFBBZEYvL25sQVB5VWs1T3lDdWc3dmVyClUycXlrdXFzYnBkSi9hODViT1JhM05IVmpVM25uRGpHVHBWaE9JeXg5TEFrc2RwZEFjVmxvcG9HODhXYk9lMTAKMUdqbnkySmdDK3JVWUZiRGtpUGx1K09IYnRnOXFYcGJMSHBzUVpsMGhucDBYSFNYVm9CMUliQndnMGEyOFVadApCbFBtWmc2d1BRS0JnRHVIUVV2SDZHYTNDVUsxNFdmOFhIcFFnMU16M2VvWTBPQm5iSDRvZUZKZmcraEppSXlnCm9RN3hqWldVR3BIc3AyblRtcHErQWlSNzdyRVhsdlhtOElVU2FsbkNiRGlKY01Pc29RdFBZNS9NczJMRm5LQTQKaENmL0pWb2FtZm1nZEN0ZGtFMXNINE9MR2lJVHdEbTRpb0dWZGIwMllnbzFyb2htNUpLMUI3MkpBb0dBUW01UQpHNDhXOTVhL0w1eSt5dCsyZ3YvUHM2VnBvMjZlTzRNQ3lJazJVem9ZWE9IYnNkODJkaC8xT2sybGdHZlI2K3VuCnc1YytZUXRSTHlhQmd3MUtpbGhFZDBKTWU3cGpUSVpnQWJ0LzVPbnlDak9OVXN2aDJjS2lrQ1Z2dTZsZlBjNkQKckliT2ZIaHhxV0RZK2Q1TGN1YSt2NzJ0RkxhenJsSlBsRzlOZHhrQ2dZRUF5elIzT3UyMDNRVVV6bUlCRkwzZAp4Wm5XZ0JLSEo3TnNxcGFWb2RjL0d5aGVycjFDZzE2MmJaSjJDV2RsZkI0VEdtUjZZdmxTZEFOOFRwUWhFbUtKCnFBLzVzdHdxNWd0WGVLOVJmMWxXK29xNThRNTBxMmk1NVdUTThoSDZhTjlaMTltZ0FGdE5VdGNqQUx2dFYxdEYKWSs4WFJkSHJaRnBIWll2NWkwVW1VbGc9Ci0tLS0tRU5EIFBSSVZBVEUgS0VZLS0tLS0K"
```
Now create the secrets using the file:
```shell
$ kubectl create -f nginxsecrets.yaml
$ kubectl get secrets
NAME TYPE DATA AGE
default-token-il9rc kubernetes.io/service-account-token 1 1d
nginxsecret Opaque 2 1m
```
Now modify your nginx replicas to start an https server using the certificate in the secret, and the Service, to expose both ports (80 and 443):
@@ -296,7 +326,3 @@ clusters and cloud providers, to provide increased availability,
better fault tolerance and greater scalability for your services. See
the [Federated Services User Guide](/docs/concepts/cluster-administration/federation-service-discovery/)
for further information.
## What's next?
[Learn about more Kubernetes features that will help you run containers reliably in production.](/docs/user-guide/production-pods)
docs/concepts/services-networking/dns-pod-service.md
+7 -1
View File
@@ -5,6 +5,9 @@ approvers:
title: DNS Pods and Services
---
* TOC
{:toc}
## Introduction
As of Kubernetes 1.3, DNS is a built-in service launched automatically using the addon manager [cluster add-on](http://releases.k8s.io/{{page.githubbranch}}/cluster/addons/README.md).
@@ -30,6 +33,8 @@ DNS query for `foo.bar`.
The following sections detail the supported record types and layout that is
supported. Any other layout or names or queries that happen to work are
considered implementation details and are subject to change without warning.
For more up-to-date specification, see
[Kubernetes DNS-Based Service Discovery](https://github.com/kubernetes/dns/blob/master/docs/specification.md).
### Services
@@ -220,7 +225,7 @@ If you see that, DNS is working correctly.
If the nslookup command fails, check the following:
#### Check the local DNS configuration first
Take a look inside the resolv.conf file. (See "Inheriting DNS from the node" and "Known issues" below for more information)
Take a look inside the resolv.conf file. (See [Inheriting DNS from the node](#inheriting-dns-from-the-node) and [Known issues](#known-issues) below for more information)
```
kubectl exec busybox cat /etc/resolv.conf
@@ -338,6 +343,7 @@ kubectl get ep kube-dns --namespace=kube-system
```
You should see something like:
```
NAME ENDPOINTS AGE
kube-dns 10.180.3.17:53,10.180.3.17:53 1h
docs/concepts/services-networking/ingress.md
+19 -14
View File
@@ -1,12 +1,14 @@
---
approvers:
- bprashanth
title: Ingress Resources
title: Ingress
---
* TOC
{:toc}
{% capture overview %}
{% glossary_definition term_id="ingress" length="all" %}
{% endcapture %}
{% capture body %}
__Terminology__
Throughout this doc you will see a few terms that are sometimes used interchangeably elsewhere, that might cause confusion. This section attempts to clarify them.
@@ -38,15 +40,15 @@ An Ingress is a collection of rules that allow inbound connections to reach the
[ Services ]
```
It can be configured to give services externally-reachable URLs, load balance traffic, terminate SSL, offer name based virtual hosting etc. Users request ingress by POSTing the Ingress resource to the API server. An [Ingress controller](#ingress-controllers) is responsible for fulfilling the Ingress, usually with a loadbalancer, though it may also configure your edge router or additional frontends to help handle the traffic in an HA manner.
It can be configured to give services externally-reachable URLs, load balance traffic, terminate SSL, offer name based virtual hosting, and more. Users request ingress by POSTing the Ingress resource to the API server. An [Ingress controller](#ingress-controllers) is responsible for fulfilling the Ingress, usually with a loadbalancer, though it may also configure your edge router or additional frontends to help handle the traffic in an HA manner.
## Prerequisites
Before you start using the Ingress resource, there are a few things you should understand. The Ingress is a beta resource, not available in any Kubernetes release prior to 1.1. You need an Ingress controller to satisfy an Ingress, simply creating the resource will have no effect.
GCE/GKE deploys an ingress controller on the master. You can deploy any number of custom ingress controllers in a pod. You must annotate each ingress with the appropriate class, as indicated [here](https://git.k8s.io/ingress/controllers/nginx#running-multiple-ingress-controllers) and [here](https://git.k8s.io/ingress/controllers/gce/BETA_LIMITATIONS.md#disabling-glbc).
GCE/Google Kubernetes Engine deploys an ingress controller on the master. You can deploy any number of custom ingress controllers in a pod. You must annotate each ingress with the appropriate class, as indicated [here](https://git.k8s.io/ingress#running-multiple-ingress-controllers) and [here](https://git.k8s.io/ingress-gce/BETA_LIMITATIONS.md#disabling-glbc).
Make sure you review the [beta limitations](https://git.k8s.io/ingress/controllers/gce/BETA_LIMITATIONS.md) of this controller. In environments other than GCE/GKE, you need to [deploy a controller](https://git.k8s.io/ingress/controllers) as a pod.
Make sure you review the [beta limitations](https://github.com/kubernetes/ingress-gce/blob/master/BETA_LIMITATIONS.md#glbc-beta-limitations) of this controller. In environments other than GCE/Google Kubernetes Engine, you need to [deploy a controller](https://git.k8s.io/ingress-nginx/README.md) as a pod.
## The Ingress Resource
@@ -71,7 +73,7 @@ spec:
*POSTing this to the API server will have no effect if you have not configured an [Ingress controller](#ingress-controllers).*
__Lines 1-6__: As with all other Kubernetes config, an Ingress needs `apiVersion`, `kind`, and `metadata` fields. For general information about working with config files, see [deploying applications](/docs/tasks/run-application/run-stateless-application-deployment/), [configuring containers](/docs/tasks/configure-pod-container/configmap/), [managing resources](/docs/concepts/cluster-administration/manage-deployment/) and [ingress configuration rewrite](https://github.com/kubernetes/ingress/blob/master/controllers/nginx/configuration.md#rewrite).
__Lines 1-6__: As with all other Kubernetes config, an Ingress needs `apiVersion`, `kind`, and `metadata` fields. For general information about working with config files, see [deploying applications](/docs/tasks/run-application/run-stateless-application-deployment/), [configuring containers](/docs/tasks/configure-pod-container/configmap/), [managing resources](/docs/concepts/cluster-administration/manage-deployment/) and [ingress configuration rewrite](https://github.com/kubernetes/ingress-nginx/blob/master/docs/examples/rewrite/README.md).
__Lines 7-9__: Ingress [spec](https://git.k8s.io/community/contributors/devel/api-conventions.md#spec-and-status) has all the information needed to configure a loadbalancer or proxy server. Most importantly, it contains a list of rules matched against all incoming requests. Currently the Ingress resource only supports http rules.
@@ -83,11 +85,11 @@ __Global Parameters__: For the sake of simplicity the example Ingress has no glo
## Ingress controllers
In order for the Ingress resource to work, the cluster must have an Ingress controller running. This is unlike other types of controllers, which typically run as part of the `kube-controller-manager` binary, and which are typically started automatically as part of cluster creation. You need to choose the ingress controller implementation that is the best fit for your cluster, or implement one. Examples and instructions can be found [here](https://git.k8s.io/ingress/controllers).
In order for the Ingress resource to work, the cluster must have an Ingress controller running. This is unlike other types of controllers, which typically run as part of the `kube-controller-manager` binary, and which are typically started automatically as part of cluster creation. You need to choose the ingress controller implementation that is the best fit for your cluster, or implement one. We currently support and maintain [GCE](https://git.k8s.io/ingress-gce/README.md) and [nginx](https://git.k8s.io/ingress-nginx/README.md) controllers.
## Before you begin
The following document describes a set of cross platform features exposed through the Ingress resource. Ideally, all Ingress controllers should fulfill this specification, but we're not there yet. The docs for the GCE and nginx controllers are [here](https://git.k8s.io/ingress/controllers/gce/README.md) and [here](https://git.k8s.io/ingress/controllers/nginx/README.md) respectively. **Make sure you review controller specific docs so you understand the caveats of each one**.
The following document describes a set of cross platform features exposed through the Ingress resource. Ideally, all Ingress controllers should fulfill this specification, but we're not there yet. We currently support and maintain [GCE](https://git.k8s.io/ingress-gce/README.md) and [nginx](https://git.k8s.io/ingress-nginx/README.md) controllers. **Make sure you review controller specific docs so you understand the caveats of each one**.
## Types of Ingress
@@ -212,19 +214,19 @@ metadata:
name: no-rules-map
spec:
tls:
- secretName: testsecret
- secretName: testsecret
backend:
serviceName: s1
servicePort: 80
```
Note that there is a gap between TLS features supported by various Ingress controllers. Please refer to documentation on [nginx](https://git.k8s.io/ingress/controllers/nginx/README.md#https), [GCE](https://git.k8s.io/ingress/controllers/gce/README.md#tls), or any other platform specific Ingress controller to understand how TLS works in your environment.
Note that there is a gap between TLS features supported by various Ingress controllers. Please refer to documentation on [nginx](https://git.k8s.io/ingress-nginx/README.md#https), [GCE](https://git.k8s.io/ingress-gce/README.md#frontend-https), or any other platform specific Ingress controller to understand how TLS works in your environment.
### Loadbalancing
An Ingress controller is bootstrapped with some loadbalancing policy settings that it applies to all Ingress, such as the loadbalancing algorithm, backend weight scheme etc. More advanced loadbalancing concepts (e.g.: persistent sessions, dynamic weights) are not yet exposed through the Ingress. You can still get these features through the [service loadbalancer](https://git.k8s.io/contrib/service-loadbalancer). With time, we plan to distill loadbalancing patterns that are applicable cross platform into the Ingress resource.
An Ingress controller is bootstrapped with some load balancing policy settings that it applies to all Ingress, such as the load balancing algorithm, backend weight scheme, and others. More advanced load balancing concepts (e.g.: persistent sessions, dynamic weights) are not yet exposed through the Ingress. You can still get these features through the [service loadbalancer](https://git.k8s.io/contrib/service-loadbalancer). With time, we plan to distill load balancing patterns that are applicable cross platform into the Ingress resource.
It's also worth noting that even though health checks are not exposed directly through the Ingress, there exist parallel concepts in Kubernetes such as [readiness probes](/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/) which allow you to achieve the same end result. Please review the controller specific docs to see how they handle health checks ([nginx](https://git.k8s.io/ingress/controllers/nginx/README.md), [GCE](https://git.k8s.io/ingress/controllers/gce/README.md#health-checks)).
It's also worth noting that even though health checks are not exposed directly through the Ingress, there exist parallel concepts in Kubernetes such as [readiness probes](/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/) which allow you to achieve the same end result. Please review the controller specific docs to see how they handle health checks ([nginx](https://git.k8s.io/ingress-nginx/README.md), [GCE](https://git.k8s.io/ingress-gce/README.md#health-checks)).
## Updating an Ingress
@@ -261,7 +263,7 @@ spec:
..
```
Saving it will update the resource in the API server, which should tell the Ingress controller to reconfigure the loadbalancer.
Saving the yaml will update the resource in the API server, which should tell the Ingress controller to reconfigure the loadbalancer.
```shell
$ kubectl get ing
@@ -296,3 +298,6 @@ You can expose a Service in multiple ways that don't directly involve the Ingres
* Use [Service.Type=NodePort](/docs/concepts/services-networking/service/#type-nodeport)
* Use a [Port Proxy](https://git.k8s.io/contrib/for-demos/proxy-to-service)
* Deploy the [Service loadbalancer](https://git.k8s.io/contrib/service-loadbalancer). This allows you to share a single IP among multiple Services and achieve more advanced loadbalancing through Service Annotations.
{% endcapture %}
{% include templates/concept.md %}
docs/concepts/services-networking/network-policies.md
+12 -16
View File
@@ -68,31 +68,27 @@ spec:
*POSTing this to the API server will have no effect unless your chosen networking solution supports network policy.*
__Mandatory Fields__: As with all other Kubernetes config, a `NetworkPolicy` needs `apiVersion`, `kind`, and `metadata` fields. For general information about working with config files, see [here](/docs/user-guide/simple-yaml), [here](/docs/user-guide/configuring-containers), and [here](/docs/user-guide/working-with-resources).
__Mandatory Fields__: As with all other Kubernetes config, a `NetworkPolicy` needs `apiVersion`, `kind`, and `metadata` fields. For general information about working with config files, see [Configure Containers Using a ConfigMap](/docs/tasks/configure-pod-container/configmap/), and [Object Management](https://kubernetes.io/docs/tutorials/object-management-kubectl/object-management/).
__spec__: `NetworkPolicy` [spec](https://git.k8s.io/community/contributors/devel/api-conventions.md#spec-and-status) has all the information needed to define a particular network policy in the given namespace.
__podSelector__: Each `NetworkPolicy` includes a `podSelector` which selects the grouping of pods to which the policy applies. Since `NetworkPolicy` currently only supports defining `ingress` rules, this `podSelector` essentially defines the "destination pods" for the policy. The example policy selects pods with the label "role=db". An empty `podSelector` selects all pods in the namespace.
__podSelector__: Each `NetworkPolicy` includes a `podSelector` which selects the grouping of pods to which the policy applies. The example policy selects pods with the label "role=db". An empty `podSelector` selects all pods in the namespace.
__policyTypes__: Each `NetworkPolicy` includes a `policyTypes` list which may include either `Ingress`, `Egress`, or both. The `policyTypes` field indicates whether or not the given policy applies to ingress traffic to selected pod, egress traffic from selected pods, or both. If no `policyTypes` are specified on a NetworkPolicy then by default `Ingress` will always be set and `Egress` will be set if the NetworkPolicy has any egress rules.
__ingress__: Each `NetworkPolicy` may include a list of whitelist `ingress` rules. Each rule allows traffic which matches both the `from` and `ports` sections. The example policy contains a single rule, which matches traffic on a single port, from either of two sources, the first specified via a `namespaceSelector` and the second specified via a `podSelector`.
__ingress__: Each `NetworkPolicy` may include a list of whitelist `ingress` rules. Each rule allows traffic which matches both the `from` and `ports` sections. The example policy contains a single rule, which matches traffic on a single port, from one of three sources, the first specified via an `ipBlock`, the second via a `namespaceSelector` and the third via a `podSelector`.
__egress__: Each `NetworkPolicy` may include a list of whitelist `egress` rules. Each rule allows traffic which matches both the `to` and `ports` sections. The example policy contains a single rule, which matches traffic on a single port to any destination in `10.0.0.0/24`.
__ipBlock__: `ipBlock` describes a particular CIDR that is allowed to
the pods matched by a NetworkPolicySpec's podSelector. The `except` entry
is a slice of CIDRs that should not be included within an IP Block. Except
values will be rejected if they are outside the CIDR range.
So, the example NetworkPolicy:
1. isolates "role=db" pods in the "default" namespace for both ingress and egress traffic (if they weren't already isolated)
2. allows connections to TCP port 6379 of "role=db" pods in the "default" namespace from any pod in the "default" namespace with the label "role=frontend"
3. allows connections to TCP port 6379 of "role=db" pods in the "default" namespace from any pod in a namespace with the label "project=myproject"
3. allows connections from any pod in the "default" namespace with the label "role=db" to CIDR 10.0.0.0/24 on TCP port 5978
4. allows connections to TCP port 6379 of "role=db" pods in the "default" namespace from IP addresses that are in CIDR 172.17.0.0/16 and not in 172.17.1.0/24
5. allows connections from any pod in the "default" namespace with the label "role=db" to CIDR 10.0.0.0/24 on TCP port 5978
See the [NetworkPolicy getting started guide](/docs/getting-started-guides/network-policy/walkthrough) for further examples.
See the [Declare Network Policy](/docs/tasks/administer-cluster/declare-network-policy/) walkthrough for further examples.
## Default policies
@@ -109,7 +105,7 @@ kind: NetworkPolicy
metadata:
name: default-deny
spec:
podSelector:
podSelector: {}
policyTypes:
- Ingress
```
@@ -126,12 +122,12 @@ kind: NetworkPolicy
metadata:
name: allow-all
spec:
podSelector:
podSelector: {}
ingress:
- {}
```
### Default deny all egress traffic.
### Default deny all egress traffic
You can create a "default" egress isolation policy for a namespace by creating a NetworkPolicy that selects all pods but does not allow any egress traffic from those pods.
@@ -141,7 +137,7 @@ kind: NetworkPolicy
metadata:
name: default-deny
spec:
podSelector:
podSelector: {}
policyTypes:
- Egress
```
@@ -159,7 +155,7 @@ kind: NetworkPolicy
metadata:
name: allow-all
spec:
podSelector:
podSelector: {}
egress:
- {}
```
@@ -174,7 +170,7 @@ kind: NetworkPolicy
metadata:
name: default-deny
spec:
podSelector:
podSelector: {}
policyTypes:
- Ingress
- Egress
docs/concepts/services-networking/service.md
+63 -40
View File
@@ -4,10 +4,10 @@ approvers:
title: Services
---
Kubernetes [`Pods`](/docs/user-guide/pods) are mortal. They are born and when they die, they
are not resurrected. [`ReplicationControllers`](/docs/user-guide/replication-controller) in
Kubernetes [`Pods`](/docs/concepts/workloads/pods/pod/) are mortal. They are born and when they die, they
are not resurrected. [`ReplicationControllers`](/docs/concepts/workloads/controllers/replicationcontroller/) in
particular create and destroy `Pods` dynamically (e.g. when scaling up or down
or when doing [rolling updates](/docs/user-guide/kubectl/v1.7/#rolling-update)). While each `Pod` gets its own IP address, even
or when doing [rolling updates](/docs/user-guide/kubectl/{{page.version}}/#rolling-update)). While each `Pod` gets its own IP address, even
those IP addresses cannot be relied upon to be stable over time. This leads to
a problem: if some set of `Pods` (let's call them backends) provides
functionality to other `Pods` (let's call them frontends) inside the Kubernetes
@@ -84,7 +84,7 @@ abstract other kinds of backends. For example:
* You want to have an external database cluster in production, but in test
you use your own databases.
* You want to point your service to a service in another
[`Namespace`](/docs/user-guide/namespaces) or on another cluster.
[`Namespace`](/docs/concepts/overview/working-with-objects/namespaces/) or on another cluster.
* You are migrating your workload to Kubernetes and some of your backends run
outside of Kubernetes.
@@ -117,7 +117,7 @@ subsets:
- port: 9376
```
NOTE: Endpoint IPs may not be loopback (127.0.0.0/8), link-local
**NOTE:** Endpoint IPs may not be loopback (127.0.0.0/8), link-local
(169.254.0.0/16), or link-local multicast (224.0.0.0/24).
Accessing a `Service` without a selector works the same as if it had a selector.
@@ -151,13 +151,11 @@ its pods, add appropriate selectors or endpoints and change the service `type`.
Every node in a Kubernetes cluster runs a `kube-proxy`. `kube-proxy` is
responsible for implementing a form of virtual IP for `Services` of type other
than `ExternalName`.
In Kubernetes v1.0 the proxy was purely in userspace. In Kubernetes v1.1 an
iptables proxy was added, but was not the default operating mode. Since
Kubernetes v1.2, the iptables proxy is the default.
As of Kubernetes v1.0, `Services` are a "layer 4" (TCP/UDP over IP) construct.
In Kubernetes v1.1 the `Ingress` API was added (beta) to represent "layer 7"
(HTTP) services.
In Kubernetes v1.0, `Services` are a "layer 4" (TCP/UDP over IP) construct, the
proxy was purely in userspace. In Kubernetes v1.1, the `Ingress` API was added
(beta) to represent "layer 7"(HTTP) services, iptables proxy was added too,
and become the default operating mode since Kubernetes v1.2. In Kubernetes v1.9-alpha,
ipvs proxy was added.
### Proxy-mode: userspace
@@ -169,37 +167,20 @@ will be proxied to one of the `Service`'s backend `Pods` (as reported in
`SessionAffinity` of the `Service`. Lastly, it installs iptables rules which
capture traffic to the `Service`'s `clusterIP` (which is virtual) and `Port`
and redirects that traffic to the proxy port which proxies the backend `Pod`.
The net result is that any traffic bound for the `Service`'s IP:Port is proxied
to an appropriate backend without the clients knowing anything about Kubernetes
or `Services` or `Pods`.
By default, the choice of backend is round robin. Client-IP based session affinity
can be selected by setting `service.spec.sessionAffinity` to `"ClientIP"` (the
default is `"None"`), and you can set the max session sticky time by setting the field
`service.spec.sessionAffinityConfig.clientIP.timeoutSeconds` if you have already set
`service.spec.sessionAffinity` to `"ClientIP"` (the default is "10800").
By default, the choice of backend is round robin.
![Services overview diagram for userspace proxy](/images/docs/services-userspace-overview.svg)
### Proxy-mode: iptables
In this mode, kube-proxy watches the Kubernetes master for the addition and
removal of `Service` and `Endpoints` objects. For each `Service` it installs
removal of `Service` and `Endpoints` objects. For each `Service`, it installs
iptables rules which capture traffic to the `Service`'s `clusterIP` (which is
virtual) and `Port` and redirects that traffic to one of the `Service`'s
backend sets. For each `Endpoints` object it installs iptables rules which
select a backend `Pod`.
backend sets. For each `Endpoints` object, it installs iptables rules which
select a backend `Pod`.By default, the choice of backend is random.
By default, the choice of backend is random. Client-IP based session affinity
can be selected by setting `service.spec.sessionAffinity` to `"ClientIP"` (the
default is `"None"`), and you can set the max session sticky time by setting the field
`service.spec.sessionAffinityConfig.clientIP.timeoutSeconds` if you have already set
`service.spec.sessionAffinity` to `"ClientIP"` (the default is "10800").
As with the userspace proxy, the net result is that any traffic bound for the
`Service`'s IP:Port is proxied to an appropriate backend without the clients
knowing anything about Kubernetes or `Services` or `Pods`. This should be
Obviously, iptables need not switch back between userspace and kernelspace, it should be
faster and more reliable than the userspace proxy. However, unlike the
userspace proxier, the iptables proxier cannot automatically retry another
`Pod` if the one it initially selects does not respond, so it depends on
@@ -207,6 +188,45 @@ having working [readiness probes](/docs/tasks/configure-pod-container/configure-
![Services overview diagram for iptables proxy](/images/docs/services-iptables-overview.svg)
### Proxy-mode: ipvs[alpha]
**Warning:** This is an alpha feature and not recommended for production clusters yet.
In this mode, kube-proxy watches Kubernetes `services` and `endpoints`,
call `netlink` interface create ipvs rules accordingly and sync ipvs rules with Kubernetes
`services` and `endpoints` periodically, to make sure ipvs status is
consistent with the expectation. When access the `service`, traffic will
be redirect to one of the backend `pod`.
Similar to iptables, Ipvs is based on netfilter hook function, but use hash
table as the underlying data structure and work in the kernal state.
That means ipvs redirects traffic can be much faster, and have much
better performance when sync proxy rules. Furthermore, ipvs provides more
options for load balancing algorithm, such as:
- rr: round-robin
- lc: least connection
- dh: destination hashing
- sh: source hashing
- sed: shortest expected delay
- nq: never queue
**Note:** ipvs mode assumed IPVS kernel modules are installed on the node
before running kube-proxy. When kube-proxy starts with ipvs proxy mode,
kube-proxy would validate if IPVS modules are installed on the node, if
it's not installed kube-proxy will fall back to iptables proxy mode.
![Services overview diagram for ipvs proxy](/images/docs/services-ipvs-overview.svg)
In any of proxy model, any traffic bound for the Service’s IP:Port is
proxied to an appropriate backend without the clients knowing anything
about Kubernetes or Services or Pods. Client-IP based session affinity
can be selected by setting service.spec.sessionAffinity to "ClientIP"
(the default is "None"), and you can set the max session sticky time by
setting the field service.spec.sessionAffinityConfig.clientIP.timeoutSeconds
if you have already set service.spec.sessionAffinity to "ClientIP"
(the default is “10800”).
## Multi-Port Services
Many `Services` need to expose more than one port. For this case, Kubernetes
@@ -389,7 +409,7 @@ configure environments that are not fully supported by Kubernetes, or
even to just expose one or more nodes' IPs directly.
Note that this Service will be visible as both `<NodeIP>:spec.ports[*].nodePort`
and `spec.clusterIp:spec.ports[*].port`.
and `spec.clusterIP:spec.ports[*].port`.
### Type LoadBalancer
@@ -411,7 +431,6 @@ spec:
- protocol: TCP
port: 80
targetPort: 9376
nodePort: 30061
clusterIP: 10.0.171.239
loadBalancerIP: 78.11.24.19
type: LoadBalancer
@@ -449,11 +468,11 @@ Select one of the tabs.
metadata:
name: my-service
annotations:
cloud.google.com/load-balancer-type: "internal"
cloud.google.com/load-balancer-type: "Internal"
[...]
```
For more information, see the [docs](https://cloud.google.com/container-engine/docs/internal-load-balancing).
Use `cloud.google.com/load-balancer-type: "internal"` for masters with version 1.7.0 to 1.7.3.
For more information, see the [docs](https://cloud.google.com/kubernetes-engine/docs/internal-load-balancing).
{% endcapture %}
{% capture aws %}
@@ -679,6 +698,10 @@ work, and the client IP is not altered.
This same basic flow executes when traffic comes in through a node-port or
through a load-balancer, though in those cases the client IP does get altered.
#### Ipvs
Iptables operations slow down dramatically in large scale cluster e.g 10,000 Services. IPVS is designed for load balancing and based on in-kernel hash tables. So we can achieve performance consistency in large number of services from IPVS-based kube-proxy. Meanwhile, IPVS-based kube-proxy has more sophisticated load balancing algorithms (least conns, locality, weighted, persistence).
## API Object
Service is a top-level resource in the Kubernetes REST API. More details about the
@@ -687,4 +710,4 @@ object](/docs/api-reference/{{page.version}}/#service-v1-core).
## For More Information
Read [Connecting a Front End to a Back End Using a Service](/docs/tutorials/connecting-apps/connecting-frontend-backend/).
Read [Connecting a Front End to a Back End Using a Service](/docs/tasks/access-application-cluster/connecting-frontend-backend/).
docs/concepts/storage/dynamic-provisioning.md
+124
View File
@@ -0,0 +1,124 @@
---
approvers:
- saad-ali
title: Dynamic Volume Provisioning
---
{% capture overview %}
Dynamic volume provisioning allows storage volumes to be created on-demand.
Without dynamic provisioning, cluster administrators have to manually make
calls to their cloud or storage provider to create new storage volumes, and
then create [`PersistentVolume` objects](/docs/concepts/storage/persistent-volumes/)
to represent them in Kubernetes. The dynamic provisioning feature eliminates
the need for cluster administrators to pre-provision storage. Instead, it
automatically provisions storage when it is requested by users.
{% endcapture %}
{:toc}
{% capture body %}
## Background
The implementation of dynamic volume provisioning is based on the API object `StorageClass`
from the API group `storage.k8s.io`. A cluster administrator can define as many
`StorageClass` objects as needed, each specifying a *volume plugin* (aka
*provisioner*) that provisions a volume and the set of parameters to pass to
that provisioner when provisioning.
A cluster administrator can define and expose multiple flavors of storage (from
the same or different storage systems) within a cluster, each with a custom set
of parameters. This design also ensures that end users don’t have to worry
about the the complexity and nuances of how storage is provisioned, but still
have the ability to select from multiple storage options.
More information on storage classes can be found
[here](/docs/concepts/storage/persistent-volumes/#storageclasses).
## Enabling Dynamic Provisioning
To enable dynamic provisioning, a cluster administrator needs to pre-create
one or more StorageClass objects for users.
StorageClass objects define which provisioner should be used and what parameters
should be passed to that provisioner when dynamic provisioning is invoked.
The following manifest creates a storage class "slow" which provisions standard
disk-like persistent disks.
```yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: slow
provisioner: kubernetes.io/gce-pd
parameters:
type: pd-standard
```
The following manifest creates a storage class "fast" which provisions
SSD-like persistent disks.
```yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: fast
provisioner: kubernetes.io/gce-pd
parameters:
type: pd-ssd
```
## Using Dynamic Provisioning
Users request dynamically provisioned storage by including a storage class in
their `PersistentVolumeClaim`. Before Kubernetes v1.6, this was done via the
`volume.beta.kubernetes.io/storage-class` annotation. However, this annotation
is deprecated since v1.6. Users now can and should instead use the
`storageClassName` field of the `PersistentVolumeClaim` object. The value of
this field must match the name of a `StorageClass` configured by the
administrator (see [below](#enabling-dynamic-provisioning)).
To select the “fast” storage class, for example, a user would create the
following `PersistentVolumeClaim`:
```yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: claim1
spec:
accessModes:
- ReadWriteOnce
storageClassName: fast
resources:
requests:
storage: 30Gi
```
This claim results in an SSD-like Persistent Disk being automatically
provisioned. When the claim is deleted, the volume is destroyed.
## Defaulting Behavior
Dynamic provisioning can be enabled on a cluster such that all claims are
dynamically provisioned if no storage class is specified. A cluster administrator
can enable this behavior by:
- Marking one `StorageClass` object as *default*;
- Making sure that the [`DefaultStorageClass` admission controller](/docs/admin/admission-controllers/#defaultstorageclass)
is enabled on the API server.
An administrator can mark a specific `StorageClass` as default by adding the
`storageclass.kubernetes.io/is-default-class` annotation to it.
When a default `StorageClass` exists in a cluster and a user creates a
`PersistentVolumeClaim` with `storageClassName` unspecified, the
`DefaultStorageClass` admission controller automatically adds the
`storageClassName` field pointing to the default storage class.
Note that there can be at most one *default* storage class on a cluster, or
a `PersistentVolumeClaim` with `storageClassName` explicitly specified cannot
be created.
{% endcapture %}
{% include templates/concept.md %}
docs/concepts/storage/persistent-volumes.md
+52 -517
View File
@@ -28,13 +28,6 @@ ways than just size and access modes, without exposing users to the details of
how those volumes are implemented. For these needs there is the `StorageClass`
resource.
A `StorageClass` provides a way for administrators to describe the "classes" of
storage they offer. Different classes might map to quality-of-service levels,
or to backup policies, or to arbitrary policies determined by the cluster
administrators. Kubernetes itself is unopinionated about what classes
represent. This concept is sometimes called "profiles" in other storage
systems.
Please see the [detailed walkthrough with working examples](/docs/tasks/configure-pod-container/configure-persistent-volume-storage/).
@@ -50,7 +43,20 @@ There are two ways PVs may be provisioned: statically or dynamically.
A cluster administrator creates a number of PVs. They carry the details of the real storage which is available for use by cluster users. They exist in the Kubernetes API and are available for consumption.
#### Dynamic
When none of the static PVs the administrator created matches a user's `PersistentVolumeClaim`, the cluster may try to dynamically provision a volume specially for the PVC. This provisioning is based on `StorageClasses`: the PVC must request a class and the administrator must have created and configured that class in order for dynamic provisioning to occur. Claims that request the class `""` effectively disable dynamic provisioning for themselves.
When none of the static PVs the administrator created matches a user's `PersistentVolumeClaim`,
the cluster may try to dynamically provision a volume specially for the PVC.
This provisioning is based on `StorageClasses`: the PVC must request a
[storage class](/docs/concepts/storage/storage-classes/) and
the administrator must have created and configured that class in order for dynamic
provisioning to occur. Claims that request the class `""` effectively disable
dynamic provisioning for themselves.
To enable dynamic storage provisioning based on storage class, the cluster administrator
needs to enable the `DefaultStorageClass` [admission controller](/docs/admin/admission-controllers/#defaultstorageclass)
on the API server. This can be done, for example, by ensuring that `DefaultStorageClass` is
among the comma-delimited, ordered list of values for the `--admission-control` flag of
the API server component. For more information on API server command line flags,
please check [kube-apiserver](/docs/admin/kube-apiserver/) documentation.
### Binding
@@ -107,7 +113,7 @@ However, the particular path specified in the custom recycler pod template in th
#### Deleting
For volume plugins that support the Delete reclaim policy, deletion removes both the `PersistentVolume` object from Kubernetes, as well as deleting the associated storage asset in the external infrastructure, such as an AWS EBS, GCE PD, Azure Disk, or Cinder volume. Volumes that were dynamically provisioned inherit the [reclaim policy of their `StorageClass`](#reclaim-policy-1), which defaults to Delete. The administrator should configure the `StorageClass` according to users' expectations, otherwise the PV must be edited or patched after it is created. See [Change the Reclaim Policy of a PersistentVolume](https://kubernetes.io/docs/tasks/administer-cluster/change-pv-reclaim-policy/).
For volume plugins that support the Delete reclaim policy, deletion removes both the `PersistentVolume` object from Kubernetes, as well as deleting the associated storage asset in the external infrastructure, such as an AWS EBS, GCE PD, Azure Disk, or Cinder volume. Volumes that were dynamically provisioned inherit the [reclaim policy of their `StorageClass`](#reclaim-policy), which defaults to Delete. The administrator should configure the `StorageClass` according to users' expectations, otherwise the PV must be edited or patched after it is created. See [Change the Reclaim Policy of a PersistentVolume](https://kubernetes.io/docs/tasks/administer-cluster/change-pv-reclaim-policy/).
### Expanding Persistent Volumes Claims
@@ -139,7 +145,7 @@ allowVolumeExpansion: true
Once both feature gate and aforementioned admission plug-in are turned on, an user can request larger volume for their `PersistentVolumeClaim`
by simply editing the claim and requesting bigger size. This in turn will trigger expansion of volume that is backing underlying `PersistentVolume`.
Under no circustances a new `PersistentVolume` gets created to satisfy the claim. Kubernetes will attempt to resize existing volume to satisfy the claim.
Under no circumstances a new `PersistentVolume` gets created to satisfy the claim. Kubernetes will attempt to resize existing volume to satisfy the claim.
## Types of Persistent Volumes
@@ -234,7 +240,7 @@ In the CLI, the access modes are abbreviated to:
| Quobyte | &#x2713; | &#x2713; | &#x2713; |
| NFS | &#x2713; | &#x2713; | &#x2713; |
| RBD | &#x2713; | &#x2713; | - |
| VsphereVolume | &#x2713; | - | - |
| VsphereVolume | &#x2713; | - | - (works when pods are collocated) |
| PortworxVolume | &#x2713; | - | &#x2713; |
| ScaleIO | &#x2713; | &#x2713; | - |
| StorageOS | &#x2713; | - | - |
@@ -243,7 +249,8 @@ In the CLI, the access modes are abbreviated to:
A PV can have a class, which is specified by setting the
`storageClassName` attribute to the name of a
`StorageClass`. A PV of a particular class can only be bound to PVCs requesting
[StorageClass](/docs/concepts/storage/storage-classes/).
A PV of a particular class can only be bound to PVCs requesting
that class. A PV with no `storageClassName` has no class and can only be bound
to PVCs that request no particular class.
@@ -344,7 +351,8 @@ All of the requirements, from both `matchLabels` and `matchExpressions` are ANDe
### Class
A claim can request a particular class by specifying the name of a
`StorageClass` using the attribute `storageClassName`.
[StorageClass](/docs/concepts/storage/storage-classes/)
using the attribute `storageClassName`.
Only PVs of the requested class, ones with the same `storageClassName` as the PVC, can
be bound to the PVC.
@@ -357,17 +365,17 @@ by the cluster depending on whether the
is turned on.
* If the admission plugin is turned on, the administrator may specify a
default `StorageClass`. All PVCs that have no `storageClassName` can be bound only to
PVs of that default. Specifying a default `StorageClass` is done by setting the
annotation `storageclass.kubernetes.io/is-default-class` equal to "true" in
a `StorageClass` object. If the administrator does not specify a default, the
cluster responds to PVC creation as if the admission plugin were turned off. If
more than one default is specified, the admission plugin forbids the creation of
all PVCs.
default `StorageClass`. All PVCs that have no `storageClassName` can be bound only to
PVs of that default. Specifying a default `StorageClass` is done by setting the
annotation `storageclass.kubernetes.io/is-default-class` equal to "true" in
a `StorageClass` object. If the administrator does not specify a default, the
cluster responds to PVC creation as if the admission plugin were turned off. If
more than one default is specified, the admission plugin forbids the creation of
all PVCs.
* If the admission plugin is turned off, there is no notion of a default
`StorageClass`. All PVCs that have no `storageClassName` can be bound only to PVs that
have no class. In this case, the PVCs that have no `storageClassName` are treated the
same way as PVCs that have their `storageClassName` set to `""`.
`StorageClass`. All PVCs that have no `storageClassName` can be bound only to PVs that
have no class. In this case, the PVCs that have no `storageClassName` are treated the
same way as PVCs that have their `storageClassName` set to `""`.
Depending on installation method, a default StorageClass may be deployed
to Kubernetes cluster by addon manager during installation.
@@ -409,502 +417,29 @@ spec:
`PersistentVolumes` binds are exclusive, and since `PersistentVolumeClaims` are namespaced objects, mounting claims with "Many" modes (`ROX`, `RWX`) is only possible within one namespace.
## StorageClasses
Each `StorageClass` contains the fields `provisioner`, `parameters`, and
`reclaimPolicy`, which are used when a `PersistentVolume` belonging to the
class needs to be dynamically provisioned.
The name of a `StorageClass` object is significant, and is how users can
request a particular class. Administrators set the name and other parameters
of a class when first creating `StorageClass` objects, and the objects cannot
be updated once they are created.
Administrators can specify a default `StorageClass` just for PVCs that don't
request any particular class to bind to: see the
[`PersistentVolumeClaim` section](#persistentvolumeclaims)
for details.
```yaml
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: standard
provisioner: kubernetes.io/aws-ebs
parameters:
type: gp2
reclaimPolicy: Retain
mountOptions:
- debug
```
### Provisioner
Storage classes have a provisioner that determines what volume plugin is used
for provisioning PVs. This field must be specified.
| Volume Plugin | Internal Provisioner| Config Example |
| :--- | :---: | :---: |
| AWSElasticBlockStore | &#x2713; | [AWS](#aws) |
| AzureFile | &#x2713; | [Azure File](#azure-file) |
| AzureDisk | &#x2713; | [Azure Disk](#azure-disk) |
| CephFS | - | - |
| Cinder | &#x2713; | [OpenStack Cinder](#openstack-cinder)|
| FC | - | - |
| FlexVolume | - | - |
| Flocker | &#x2713; | - |
| GCEPersistentDisk | &#x2713; | [GCE](#gce) |
| Glusterfs | &#x2713; | [Glusterfs](#glusterfs) |
| iSCSI | - | - |
| PhotonPersistentDisk | &#x2713; | - |
| Quobyte | &#x2713; | [Quobyte](#quobyte) |
| NFS | - | - |
| RBD | &#x2713; | [Ceph RBD](#ceph-rbd) |
| VsphereVolume | &#x2713; | [vSphere](#vsphere) |
| PortworxVolume | &#x2713; | [Portworx Volume](#portworx-volume) |
| ScaleIO | &#x2713; | [ScaleIO](#scaleio) |
You are not restricted to specifying the "internal" provisioners
listed here (whose names are prefixed with "kubernetes.io" and shipped
alongside Kubernetes). You can also run and specify external provisioners,
which are independent programs that follow a [specification](https://git.k8s.io/community/contributors/design-proposals/storage/volume-provisioning.md)
defined by Kubernetes. Authors of external provisioners have full discretion
over where their code lives, how the provisioner is shipped, how it needs to be
run, what volume plugin it uses (including Flex), etc. The repository [kubernetes-incubator/external-storage](https://github.com/kubernetes-incubator/external-storage)
houses a library for writing external provisioners that implements the bulk of
the specification plus various community-maintained external provisioners.
For example, NFS doesn't provide an internal provisioner, but an external provisioner
can be used. Some external provisioners are listed under the repository [kubernetes-incubator/external-storage](https://github.com/kubernetes-incubator/external-storage).
There are also cases when 3rd party storage vendors provide their own external
provisioner.
### Reclaim Policy
Persistent Volumes that are dynamically created by a storage class will have the
reclaim policy specified in the `reclaimPolicy` field of the class, which can be
either `Delete` or `Retain`. If no `reclaimPolicy` is specified when a
`StorageClass` object is created, it will default to `Delete`.
Persistent Volumes that are created manually and managed via a storage class will have
whatever reclaim policy they were assigned at creation.
### Mount Options
Persistent Volumes that are dynamically created by a storage class will have the
mount options specified in the `mountOptions` field of the class.
If the volume plugin does not support mount options but mount options are
specified, provisioning will fail. Mount options are not validated on neither
the class nor PV, so mount of the PV will simply fail if one is invalid.
### Parameters
Storage classes have parameters that describe volumes belonging to the storage
class. Different parameters may be accepted depending on the `provisioner`. For
example, the value `io1`, for the parameter `type`, and the parameter
`iopsPerGB` are specific to EBS. When a parameter is omitted, some default is
used.
#### AWS
```yaml
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: slow
provisioner: kubernetes.io/aws-ebs
parameters:
type: io1
zones: us-east-1d, us-east-1c
iopsPerGB: "10"
```
* `type`: `io1`, `gp2`, `sc1`, `st1`. See [AWS docs](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSVolumeTypes.html) for details. Default: `gp2`.
* `zone`: AWS zone. If neither `zone` nor `zones` is specified, volumes are generally round-robin-ed across all active zones where Kubernetes cluster has a node. `zone` and `zones` parameters must not be used at the same time.
* `zones`: A comma separated list of AWS zone(s). If neither `zone` nor `zones` is specified, volumes are generally round-robin-ed across all active zones where Kubernetes cluster has a node. `zone` and `zones` parameters must not be used at the same time.
* `iopsPerGB`: only for `io1` volumes. I/O operations per second per GiB. AWS volume plugin multiplies this with size of requested volume to compute IOPS of the volume and caps it at 20 000 IOPS (maximum supported by AWS, see [AWS docs](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSVolumeTypes.html). A string is expected here, i.e. `"10"`, not `10`.
* `encrypted`: denotes whether the EBS volume should be encrypted or not. Valid values are `"true"` or `"false"`. A string is expected here, i.e. `"true"`, not `true`.
* `kmsKeyId`: optional. The full Amazon Resource Name of the key to use when encrypting the volume. If none is supplied but `encrypted` is true, a key is generated by AWS. See AWS docs for valid ARN value.
#### GCE
```yaml
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: slow
provisioner: kubernetes.io/gce-pd
parameters:
type: pd-standard
zones: us-central1-a, us-central1-b
```
* `type`: `pd-standard` or `pd-ssd`. Default: `pd-standard`
* `zone`: GCE zone. If neither `zone` nor `zones` is specified, volumes are generally round-robin-ed across all active zones where Kubernetes cluster has a node. `zone` and `zones` parameters must not be used at the same time.
* `zones`: A comma separated list of GCE zone(s). If neither `zone` nor `zones` is specified, volumes are generally round-robin-ed across all active zones where Kubernetes cluster has a node. `zone` and `zones` parameters must not be used at the same time.
#### Glusterfs
```yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: slow
provisioner: kubernetes.io/glusterfs
parameters:
resturl: "http://127.0.0.1:8081"
clusterid: "630372ccdc720a92c681fb928f27b53f"
restauthenabled: "true"
restuser: "admin"
secretNamespace: "default"
secretName: "heketi-secret"
gidMin: "40000"
gidMax: "50000"
volumetype: "replicate:3"
```
* `resturl`: Gluster REST service/Heketi service url which provision gluster volumes on demand. The general format should be `IPaddress:Port` and this is a mandatory parameter for GlusterFS dynamic provisioner. If Heketi service is exposed as a routable service in openshift/kubernetes setup, this can have a format similar to
`http://heketi-storage-project.cloudapps.mystorage.com` where the fqdn is a resolvable heketi service url.
* `restauthenabled` : Gluster REST service authentication boolean that enables authentication to the REST server. If this value is 'true', `restuser` and `restuserkey` or `secretNamespace` + `secretName` have to be filled. This option is deprecated, authentication is enabled when any of `restuser`, `restuserkey`, `secretName` or `secretNamespace` is specified.
* `restuser` : Gluster REST service/Heketi user who has access to create volumes in the Gluster Trusted Pool.
* `restuserkey` : Gluster REST service/Heketi user's password which will be used for authentication to the REST server. This parameter is deprecated in favor of `secretNamespace` + `secretName`.
* `secretNamespace`, `secretName` : Identification of Secret instance that contains user password to use when talking to Gluster REST service. These parameters are optional, empty password will be used when both `secretNamespace` and `secretName` are omitted. The provided secret must have type "kubernetes.io/glusterfs", e.g. created in this way:
```
$ kubectl create secret generic heketi-secret --type="kubernetes.io/glusterfs" --from-literal=key='opensesame' --namespace=default
```
Example of a secret can be found in [glusterfs-provisioning-secret.yaml](https://github.com/kubernetes/examples/tree/master/staging/persistent-volume-provisioning/glusterfs/glusterfs-secret.yaml).
* `clusterid`: `630372ccdc720a92c681fb928f27b53f` is the ID of the cluster which will be used by Heketi when provisioning the volume. It can also be a list of clusterids, for ex:
"8452344e2becec931ece4e33c4674e4e,42982310de6c63381718ccfa6d8cf397". This is an optional parameter.
* `gidMin`, `gidMax` : The minimum and maximum value of GID range for the storage class. A unique value (GID) in this range ( gidMin-gidMax ) will be used for dynamically provisioned volumes. These are optional values. If not specified, the volume will be provisioned with a value between 2000-2147483647 which are defaults for gidMin and gidMax respectively.
* `volumetype` : The volume type and its parameters can be configured with this optional value. If the volume type is not mentioned, it's up to the provisioner to decide the volume type.
For example:
'Replica volume':
`volumetype: replicate:3` where '3' is replica count.
'Disperse/EC volume':
`volumetype: disperse:4:2` where '4' is data and '2' is the redundancy count.
'Distribute volume':
`volumetype: none`
For available volume types and administration options, refer to the [Administration Guide](https://access.redhat.com/documentation/en-US/Red_Hat_Storage/3.1/html/Administration_Guide/part-Overview.html).
For further reference information, see [How to configure Heketi](https://github.com/heketi/heketi/wiki/Setting-up-the-topology).
When persistent volumes are dynamically provisioned, the Gluster plugin automatically creates an endpoint and a headless service in the name `gluster-dynamic-<claimname>`. The dynamic endpoint and service are automatically deleted when the persistent volume claim is deleted.
#### OpenStack Cinder
```yaml
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: gold
provisioner: kubernetes.io/cinder
parameters:
type: fast
availability: nova
```
* `type`: [VolumeType](https://docs.openstack.org/user-guide/dashboard-manage-volumes.html) created in Cinder. Default is empty.
* `availability`: Availability Zone. If not specified, volumes are generally round-robin-ed across all active zones where Kubernetes cluster has a node.
#### vSphere
1. Create a persistent volume with a user specified disk format.
```yaml
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: fast
provisioner: kubernetes.io/vsphere-volume
parameters:
diskformat: zeroedthick
```
- `diskformat`: `thin`, `zeroedthick` and `eagerzeroedthick`. Default: `"thin"`.
2. Create a persistent volume with a disk format on a user specified datastore.
```yaml
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: fast
provisioner: kubernetes.io/vsphere-volume
parameters:
diskformat: zeroedthick
datastore: VSANDatastore
```
- `diskformat`: `thin`, `zeroedthick` and `eagerzeroedthick`. Default: `"thin"`.
- `datastore`: The user can also specify the datastore in the Storageclass. The volume will be created on the datastore specified in the storage class which in this case is `VSANDatastore`. This field is optional. If not specified as in previous YAML description, the volume will be created on the datastore specified in the vsphere config file used to initialize the vSphere Cloud Provider.
3. Create a persistent volume with user specified VSAN storage capabilities.
```yaml
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: vsan-policy-fast
provisioner: kubernetes.io/vsphere-volume
parameters:
diskformat: thin
hostFailuresToTolerate: "1"
diskStripes: "2"
cacheReservation: "20"
datastore: VSANDatastore
```
- Here, the user can specify VSAN storage capabilities for dynamic volume provisioning inside Kubernetes.
- Storage Policies capture storage requirements, such as performance and availability, for persistent volumes. These policies determine how the container volume storage objects are provisioned and allocated within the datastore to guarantee the requested Quality of Service. Storage policies are composed of storage capabilities, typically represented by a key-value pair. The key is a specific property that the datastore can offer and the value is a metric, or a range, that the datastore guarantees for a provisioned object, such as a container volume backed by a virtual disk.
- As described in [official documentation](https://pubs.vmware.com/vsphere-65/index.jsp?topic=%2Fcom.vmware.vsphere.virtualsan.doc%2FGUID-08911FD3-2462-4C1C-AE81-0D4DBC8F7990.html), VSAN exposes multiple storage capabilities. The below table lists VSAN storage capabilities that are currently supported by vSphere Cloud Provider.
Storage Capability Name | Description
-------------------- | ------------
cacheReservation | Flash read cache reservation
diskStripes | Number of disk stripes per object
forceProvisioning | Force provisioning
hostFailuresToTolerate | Number of failures to tolerate
iopsLimit | IOPS limit for object
objectSpaceReservation | Object space reservation
vSphere Infrastructure(VI) administrator can specify storage requirements for applications in terms of storage capabilities while creating a storage class inside Kubernetes. Please note that while creating a StorageClass, administrator should specify storage capability names used in the table above as these names might differ from the ones used by VSAN. For example - Number of disk stripes per object is referred to as stripeWidth in VSAN documentation however vSphere Cloud Provider uses a friendly name diskStripes.
You can see [vSphere example](https://github.com/kubernetes/examples/tree/master/staging/volumes/vsphere) for more details.
#### Ceph RBD
```yaml
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: fast
provisioner: kubernetes.io/rbd
parameters:
monitors: 10.16.153.105:6789
adminId: kube
adminSecretName: ceph-secret
adminSecretNamespace: kube-system
pool: kube
userId: kube
userSecretName: ceph-secret-user
fsType: ext4
imageFormat: "2"
imageFeatures: "layering"
```
* `monitors`: Ceph monitors, comma delimited. This parameter is required.
* `adminId`: Ceph client ID that is capable of creating images in the pool. Default is "admin".
* `adminSecretNamespace`: The namespace for `adminSecret`. Default is "default".
* `adminSecret`: Secret Name for `adminId`. This parameter is required. The provided secret must have type "kubernetes.io/rbd".
* `pool`: Ceph RBD pool. Default is "rbd".
* `userId`: Ceph client ID that is used to map the RBD image. Default is the same as `adminId`.
* `userSecretName`: The name of Ceph Secret for `userId` to map RBD image. It must exist in the same namespace as PVCs. This parameter is required. The provided secret must have type "kubernetes.io/rbd", e.g. created in this way:
```
$ kubectl create secret generic ceph-secret --type="kubernetes.io/rbd" --from-literal=key='QVFEQ1pMdFhPUnQrSmhBQUFYaERWNHJsZ3BsMmNjcDR6RFZST0E9PQ==' --namespace=kube-system
```
* `fsType`: fsType that is supported by kubernetes. Default: `"ext4"`.
* `imageFormat`: Ceph RBD image format, "1" or "2". Default is "1".
* `imageFeatures`: This parameter is optional and should only be used if you set `imageFormat` to "2". Currently supported features are `layering` only. Default is "", and no features are turned on.
#### Quobyte
```yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: slow
provisioner: kubernetes.io/quobyte
parameters:
quobyteAPIServer: "http://138.68.74.142:7860"
registry: "138.68.74.142:7861"
adminSecretName: "quobyte-admin-secret"
adminSecretNamespace: "kube-system"
user: "root"
group: "root"
quobyteConfig: "BASE"
quobyteTenant: "DEFAULT"
```
* `quobyteAPIServer`: API Server of Quobyte in the format `http(s)://api-server:7860`
* `registry`: Quobyte registry to use to mount the volume. You can specify the registry as ``<host>:<port>`` pair or if you want to specify multiple registries you just have to put a comma between them e.q. ``<host1>:<port>,<host2>:<port>,<host3>:<port>``. The host can be an IP address or if you have a working DNS you can also provide the DNS names.
* `adminSecretNamespace`: The namespace for `adminSecretName`. Default is "default".
* `adminSecretName`: secret that holds information about the Quobyte user and the password to authenticate against the API server. The provided secret must have type "kubernetes.io/quobyte", e.g. created in this way:
```
$ kubectl create secret generic quobyte-admin-secret --type="kubernetes.io/quobyte" --from-literal=key='opensesame' --namespace=kube-system
```
* `user`: maps all access to this user. Default is "root".
* `group`: maps all access to this group. Default is "nfsnobody".
* `quobyteConfig`: use the specified configuration to create the volume. You can create a new configuration or modify an existing one with the Web console or the quobyte CLI. Default is "BASE".
* `quobyteTenant`: use the specified tenant ID to create/delete the volume. This Quobyte tenant has to be already present in Quobyte. Default is "DEFAULT".
#### Azure Disk
##### Azure Unmanaged Disk Storage Class
```yaml
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: slow
provisioner: kubernetes.io/azure-disk
parameters:
skuName: Standard_LRS
location: eastus
storageAccount: azure_storage_account_name
```
* `skuName`: Azure storage account Sku tier. Default is empty.
* `location`: Azure storage account location. Default is empty.
* `storageAccount`: Azure storage account name. If a storage account is provided, it must reside in the same resource group as the cluster, and `location` is ignored. If a storage account is not provided, a new storage account will be created in the same resource group as the cluster.
##### New Azure Disk Storage Class (starting from v1.7.2)
```yaml
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: slow
provisioner: kubernetes.io/azure-disk
parameters:
storageaccounttype: Standard_LRS
kind: Shared
```
* `storageaccounttype`: Azure storage account Sku tier. Default is empty.
* `kind`: Possible values are `shared` (default), `dedicated`, and `managed`. When `kind` is `shared`, all unmanaged disks are created in a few shared storage accounts in the same resource group as the cluster. When `kind` is `dedicated`, a new dedicated storage account will be created for the new unmanaged disk in the same resource group as the cluster.
- Premium VM can attach both Standard_LRS and Premium_LRS disks, while Standard VM can only attach Standard_LRS disks.
- Managed VM can only attach managed disks and unmanaged VM can only attach unmanaged disks.
#### Azure File
```yaml
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: azurefile
provisioner: kubernetes.io/azure-file
parameters:
skuName: Standard_LRS
location: eastus
storageAccount: azure_storage_account_name
```
* `skuName`: Azure storage account Sku tier. Default is empty.
* `location`: Azure storage account location. Default is empty.
* `storageAccount`: Azure storage account name. Default is empty. If a storage account is not provided, all storage accounts associated with the resource group are searched to find one that matches `skuName` and `location`. If a storage account is provided, it must reside in the same resource group as the cluster, and `skuName` and `location` are ignored.
During provision, a secret is created for mounting credentials. If the cluster has enabled both [RBAC](/docs/admin/authorization/rbac/) and [Controller Roles](/docs/admin/authorization/rbac/#controller-roles), add the `create` permission of resource `secret` for clusterrole `system:controller:persistent-volume-binder`.
#### Portworx Volume
```yaml
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: portworx-io-priority-high
provisioner: kubernetes.io/portworx-volume
parameters:
repl: "1"
snap_interval: "70"
io_priority: "high"
```
* `fs`: filesystem to be laid out: [none/xfs/ext4] (default: `ext4`).
* `block_size`: block size in Kbytes (default: `32`).
* `repl`: number of synchronous replicas to be provided in the form of replication factor [1..3] (default: `1`) A string is expected here i.e.`"1"` and not `1`.
* `io_priority`: determines whether the volume will be created from higher performance or a lower priority storage [high/medium/low] (default: `low`).
* `snap_interval`: clock/time interval in minutes for when to trigger snapshots. Snapshots are incremental based on difference with the prior snapshot, 0 disables snaps (default: `0`). A string is expected here i.e. `"70"` and not `70`.
* `aggregation_level`: specifies the number of chunks the volume would be distributed into, 0 indicates a non-aggregated volume (default: `0`). A string is expected here i.e. `"0"` and not `0`
* `ephemeral`: specifies whether the volume should be cleaned-up after unmount or should be persistent. `emptyDir` use case can set this value to true and `persistent volumes` use case such as for databases like Cassandra should set to false, [true/false] (default `false`). A string is expected here i.e. `"true"` and not `true`.
#### ScaleIO
```yaml
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: slow
provisioner: kubernetes.io/scaleio
parameters:
gateway: https://192.168.99.200:443/api
system: scaleio
protectionDomain: pd0
storagePool: sp1
storageMode: ThinProvisionned
secretRef: sio-secret
readOnly: false
fsType: xfs
```
* `provisioner`: attribute is set to `kubernetes.io/scaleio`
* `gateway`: address to a ScaleIO API gateway (required)
* `system`: the name of the ScaleIO system (required)
* `protectionDomain`: the name of the ScaleIO protection domain (required)
* `storagePool`: the name of the volume storage pool (required)
* `storageMode`: the storage provision mode: `ThinProvisionned` (default) or `ThickProvisionned`
* `secretRef`: reference to a configured Secret object (required)
* `readOnly`: specifies the access mode to the mounted volume (default false)
* `fsType`: the file system to use for the volume (default ext4)
The ScaleIO Kubernetes volume plugin requires a configured Secret object.
The secret must be created with type `kubernetes.io/scaleio` and use the same namespace value as that of the PVC where it is referenced
as shown in the following command:
```
$> kubectl create secret generic sio-secret --type="kubernetes.io/scaleio" --from-literal=username=sioadmin --from-literal=password=d2NABDNjMA== --namespace=default
```
#### StorageOS
```yaml
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: fast
provisioner: kubernetes.io/storageos
parameters:
pool: default
description: Kubernetes volume
fsType: ext4
adminSecretNamespace: default
adminSecretName: storageos-secret
```
* `pool`: The name of the StorageOS distributed capacity pool to provision the volume from. Uses the `default` pool which is normally present if not specified.
* `description`: The description to assign to volumes that were created dynamically. All volume descriptions will be the same for the storage class, but different storage classes can be used to allow descriptions for different use cases. Defaults to `Kubernetes volume`.
* `fsType`: The default filesystem type to request. Note that user-defined rules within StorageOS may override this value. Defaults to `ext4`.
* `adminSecretNamespace`: The namespace where the API configuration secret is located. Required if adminSecretName set.
* `adminSecretName`: The name of the secret to use for obtaining the StorageOS API credentials. If not specified, default values will be attempted.
The StorageOS Kubernetes volume plugin can use a Secret object to specify an endpoint and credentials to access the StorageOS API. This is only required when the defaults have been changed.
The secret must be created with type `kubernetes.io/storageos` as shown in the following command:
```
$ kubectl create secret generic storageos-secret --type="kubernetes.io/storageos" --from-literal=apiAddress=tcp://localhost:5705 --from-literal=apiUsername=storageos --from-literal=apiPassword=storageos --namespace=default
```
Secrets used for dynamically provisioned volumes may be created in any namespace and referenced with the `adminSecretNamespace` parameter. Secrets used by pre-provisioned volumes must be created in the same namespace as the PVC that references it.
## Writing Portable Configuration
If you're writing configuration templates or examples that run on a wide range of clusters
and need persistent storage, we recommend that you use the following pattern:
- Do include PersistentVolumeClaim objects in your bundle of config (alongside Deployments, ConfigMaps, etc).
- Do not include PersistentVolume objects in the config, since the user instantiating the config may not have
permission to create PersistentVolumes.
- Give the user the option of providing a storage class name when instantiating the template.
- If the user provides a storage class name, and the cluster is version 1.4 or newer, put that value into the `volume.beta.kubernetes.io/storage-class` annotation of the PVC.
This will cause the PVC to match the right storage class if the cluster has StorageClasses enabled by the admin.
- If the user does not provide a storage class name or the cluster is version 1.3, then instead put a `volume.alpha.kubernetes.io/storage-class: default` annotation on the PVC.
- This will cause a PV to be automatically provisioned for the user with sane default characteristics on some clusters.
- Despite the word `alpha` in the name, the code behind this annotation has `beta` level support.
- Do not use `volume.beta.kubernetes.io/storage-class:` with any value including the empty string since it will prevent DefaultStorageClass admission controller
from running if enabled.
- In your tooling, do watch for PVCs that are not getting bound after some time and surface this to the user, as this may indicate that the cluster has no dynamic
storage support (in which case the user should create a matching PV) or the cluster has no storage system (in which case the user cannot deploy config requiring
PVCs).
- In the future, we expect most clusters to have `DefaultStorageClass` enabled, and to have some form of storage available. However, there may not be any
storage class names which work on all clusters, so continue to not set one by default.
At some point, the alpha annotation will cease to have meaning, but the unset `storageClass` field on the PVC
will have the desired effect.
- Do include PersistentVolumeClaim objects in your bundle of config (alongside
Deployments, ConfigMaps, etc).
- Do not include PersistentVolume objects in the config, since the user instantiating
the config may not have permission to create PersistentVolumes.
- Give the user the option of providing a storage class name when instantiating
the template.
- If the user provides a storage class name, put that value into the
`persistentVolumeClaim.storageClassName` field.
This will cause the PVC to match the right storage
class if the cluster has StorageClasses enabled by the admin.
- If the user does not provide a storage class name, leave the
`persistentVolumeClaim.storageClassName` field as nil.
- This will cause a PV to be automatically provisioned for the user with
the default StorageClass in the cluster. Many cluster environments have
a default StorageClass installed, or administrators can create their own
default StorageClass.
- In your tooling, do watch for PVCs that are not getting bound after some time
and surface this to the user, as this may indicate that the cluster has no
dynamic storage support (in which case the user should create a matching PV)
or the cluster has no storage system (in which case the user cannot deploy
config requiring PVCs).
docs/concepts/storage/storage-classes.md
+636
View File
@@ -0,0 +1,636 @@
---
approvers:
- jsafrane
- mikedanese
- saad-ali
- thockin
title: Storage Classes
---
This document describes the concept of `StorageClass` in Kubernetes. Familiarity
with [volumes](/docs/concepts/storage/volumes/) and
[persistent volumes](/docs/concepts/storage/persistent-volumes) is suggested.
* TOC
{:toc}
## Introduction
A `StorageClass` provides a way for administrators to describe the "classes" of
storage they offer. Different classes might map to quality-of-service levels,
or to backup policies, or to arbitrary policies determined by the cluster
administrators. Kubernetes itself is unopinionated about what classes
represent. This concept is sometimes called "profiles" in other storage
systems.
## The StorageClass Resource
Each `StorageClass` contains the fields `provisioner`, `parameters`, and
`reclaimPolicy`, which are used when a `PersistentVolume` belonging to the
class needs to be dynamically provisioned.
The name of a `StorageClass` object is significant, and is how users can
request a particular class. Administrators set the name and other parameters
of a class when first creating `StorageClass` objects, and the objects cannot
be updated once they are created.
Administrators can specify a default `StorageClass` just for PVCs that don't
request any particular class to bind to: see the
[`PersistentVolumeClaim` section](#persistentvolumeclaims)
for details.
```yaml
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: standard
provisioner: kubernetes.io/aws-ebs
parameters:
type: gp2
reclaimPolicy: Retain
mountOptions:
- debug
```
### Provisioner
Storage classes have a provisioner that determines what volume plugin is used
for provisioning PVs. This field must be specified.
| Volume Plugin | Internal Provisioner| Config Example |
| :--- | :---: | :---: |
| AWSElasticBlockStore | &#x2713; | [AWS](#aws) |
| AzureFile | &#x2713; | [Azure File](#azure-file) |
| AzureDisk | &#x2713; | [Azure Disk](#azure-disk) |
| CephFS | - | - |
| Cinder | &#x2713; | [OpenStack Cinder](#openstack-cinder)|
| FC | - | - |
| FlexVolume | - | - |
| Flocker | &#x2713; | - |
| GCEPersistentDisk | &#x2713; | [GCE](#gce) |
| Glusterfs | &#x2713; | [Glusterfs](#glusterfs) |
| iSCSI | - | - |
| PhotonPersistentDisk | &#x2713; | - |
| Quobyte | &#x2713; | [Quobyte](#quobyte) |
| NFS | - | - |
| RBD | &#x2713; | [Ceph RBD](#ceph-rbd) |
| VsphereVolume | &#x2713; | [vSphere](#vsphere) |
| PortworxVolume | &#x2713; | [Portworx Volume](#portworx-volume) |
| ScaleIO | &#x2713; | [ScaleIO](#scaleio) |
| StorageOS | &#x2713; | [StorageOS](#storageos) |
You are not restricted to specifying the "internal" provisioners
listed here (whose names are prefixed with "kubernetes.io" and shipped
alongside Kubernetes). You can also run and specify external provisioners,
which are independent programs that follow a [specification](https://git.k8s.io/community/contributors/design-proposals/storage/volume-provisioning.md)
defined by Kubernetes. Authors of external provisioners have full discretion
over where their code lives, how the provisioner is shipped, how it needs to be
run, what volume plugin it uses (including Flex), etc. The repository [kubernetes-incubator/external-storage](https://github.com/kubernetes-incubator/external-storage)
houses a library for writing external provisioners that implements the bulk of
the specification plus various community-maintained external provisioners.
For example, NFS doesn't provide an internal provisioner, but an external provisioner
can be used. Some external provisioners are listed under the repository [kubernetes-incubator/external-storage](https://github.com/kubernetes-incubator/external-storage).
There are also cases when 3rd party storage vendors provide their own external
provisioner.
### Reclaim Policy
Persistent Volumes that are dynamically created by a storage class will have the
reclaim policy specified in the `reclaimPolicy` field of the class, which can be
either `Delete` or `Retain`. If no `reclaimPolicy` is specified when a
`StorageClass` object is created, it will default to `Delete`.
Persistent Volumes that are created manually and managed via a storage class will have
whatever reclaim policy they were assigned at creation.
### Mount Options
Persistent Volumes that are dynamically created by a storage class will have the
mount options specified in the `mountOptions` field of the class.
If the volume plugin does not support mount options but mount options are
specified, provisioning will fail. Mount options are not validated on neither
the class nor PV, so mount of the PV will simply fail if one is invalid.
## Parameters
Storage classes have parameters that describe volumes belonging to the storage
class. Different parameters may be accepted depending on the `provisioner`. For
example, the value `io1`, for the parameter `type`, and the parameter
`iopsPerGB` are specific to EBS. When a parameter is omitted, some default is
used.
### AWS
```yaml
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: slow
provisioner: kubernetes.io/aws-ebs
parameters:
type: io1
zones: us-east-1d, us-east-1c
iopsPerGB: "10"
```
* `type`: `io1`, `gp2`, `sc1`, `st1`. See
[AWS docs](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSVolumeTypes.html)
for details. Default: `gp2`.
* `zone`: AWS zone. If neither `zone` nor `zones` is specified, volumes are
generally round-robin-ed across all active zones where Kubernetes cluster
has a node. `zone` and `zones` parameters must not be used at the same time.
* `zones`: A comma separated list of AWS zone(s). If neither `zone` nor `zones`
is specified, volumes are generally round-robin-ed across all active zones
where Kubernetes cluster has a node. `zone` and `zones` parameters must not
be used at the same time.
* `iopsPerGB`: only for `io1` volumes. I/O operations per second per GiB. AWS
volume plugin multiplies this with size of requested volume to compute IOPS
of the volume and caps it at 20 000 IOPS (maximum supported by AWS, see
[AWS docs](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSVolumeTypes.html).
A string is expected here, i.e. `"10"`, not `10`.
* `encrypted`: denotes whether the EBS volume should be encrypted or not.
Valid values are `"true"` or `"false"`. A string is expected here,
i.e. `"true"`, not `true`.
* `kmsKeyId`: optional. The full Amazon Resource Name of the key to use when
encrypting the volume. If none is supplied but `encrypted` is true, a key is
generated by AWS. See AWS docs for valid ARN value.
### GCE
```yaml
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: slow
provisioner: kubernetes.io/gce-pd
parameters:
type: pd-standard
zones: us-central1-a, us-central1-b
```
* `type`: `pd-standard` or `pd-ssd`. Default: `pd-standard`
* `zone`: GCE zone. If neither `zone` nor `zones` is specified, volumes are
generally round-robin-ed across all active zones where Kubernetes cluster has
a node. `zone` and `zones` parameters must not be used at the same time.
* `zones`: A comma separated list of GCE zone(s). If neither `zone` nor `zones`
is specified, volumes are generally round-robin-ed across all active zones
where Kubernetes cluster has a node. `zone` and `zones` parameters must not
be used at the same time.
### Glusterfs
```yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: slow
provisioner: kubernetes.io/glusterfs
parameters:
resturl: "http://127.0.0.1:8081"
clusterid: "630372ccdc720a92c681fb928f27b53f"
restauthenabled: "true"
restuser: "admin"
secretNamespace: "default"
secretName: "heketi-secret"
gidMin: "40000"
gidMax: "50000"
volumetype: "replicate:3"
```
* `resturl`: Gluster REST service/Heketi service url which provision gluster
volumes on demand. The general format should be `IPaddress:Port` and this is
a mandatory parameter for GlusterFS dynamic provisioner. If Heketi service is
exposed as a routable service in openshift/kubernetes setup, this can have a
format similar to `http://heketi-storage-project.cloudapps.mystorage.com`
where the fqdn is a resolvable heketi service url.
* `restauthenabled` : Gluster REST service authentication boolean that enables
authentication to the REST server. If this value is 'true', `restuser` and
`restuserkey` or `secretNamespace` + `secretName` have to be filled. This
option is deprecated, authentication is enabled when any of `restuser`,
`restuserkey`, `secretName` or `secretNamespace` is specified.
* `restuser` : Gluster REST service/Heketi user who has access to create volumes
in the Gluster Trusted Pool.
* `restuserkey` : Gluster REST service/Heketi user's password which will be used
for authentication to the REST server. This parameter is deprecated in favor
of `secretNamespace` + `secretName`.
* `secretNamespace`, `secretName` : Identification of Secret instance that
contains user password to use when talking to Gluster REST service. These
parameters are optional, empty password will be used when both
`secretNamespace` and `secretName` are omitted. The provided secret must have
type "kubernetes.io/glusterfs", e.g. created in this way:
```
kubectl create secret generic heketi-secret \
--type="kubernetes.io/glusterfs" --from-literal=key='opensesame' \
--namespace=default
```
Example of a secret can be found in
[glusterfs-provisioning-secret.yaml](https://github.com/kubernetes/examples/tree/master/staging/persistent-volume-provisioning/glusterfs/glusterfs-secret.yaml).
* `clusterid`: `630372ccdc720a92c681fb928f27b53f` is the ID of the cluster
which will be used by Heketi when provisioning the volume. It can also be a
list of clusterids, for example:
`"8452344e2becec931ece4e33c4674e4e,42982310de6c63381718ccfa6d8cf397"`. This
is an optional parameter.
* `gidMin`, `gidMax` : The minimum and maximum value of GID range for the
storage class. A unique value (GID) in this range ( gidMin-gidMax ) will be
used for dynamically provisioned volumes. These are optional values. If not
specified, the volume will be provisioned with a value between 2000-2147483647
which are defaults for gidMin and gidMax respectively.
* `volumetype` : The volume type and its parameters can be configured with this
optional value. If the volume type is not mentioned, it's up to the provisioner
to decide the volume type.
For example:
'Replica volume':
`volumetype: replicate:3` where '3' is replica count.
'Disperse/EC volume':
`volumetype: disperse:4:2` where '4' is data and '2' is the redundancy count.
'Distribute volume':
`volumetype: none`
For available volume types and administration options, refer to the
[Administration Guide](https://access.redhat.com/documentation/en-US/Red_Hat_Storage/3.1/html/Administration_Guide/part-Overview.html).
For further reference information, see
[How to configure Heketi](https://github.com/heketi/heketi/wiki/Setting-up-the-topology).
When persistent volumes are dynamically provisioned, the Gluster plugin
automatically creates an endpoint and a headless service in the name
`gluster-dynamic-<claimname>`. The dynamic endpoint and service are automatically
deleted when the persistent volume claim is deleted.
### OpenStack Cinder
```yaml
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: gold
provisioner: kubernetes.io/cinder
parameters:
type: fast
availability: nova
```
* `type`: [VolumeType](https://docs.openstack.org/user-guide/dashboard-manage-volumes.html)
created in Cinder. Default is empty.
* `availability`: Availability Zone. If not specified, volumes are generally
round-robin-ed across all active zones where Kubernetes cluster has a node.
### vSphere
1. Create a StorageClass with a user specified disk format.
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: fast
provisioner: kubernetes.io/vsphere-volume
parameters:
diskformat: zeroedthick
`diskformat`: `thin`, `zeroedthick` and `eagerzeroedthick`. Default: `"thin"`.
2. Create a StorageClass with a disk format on a user specified datastore.
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: fast
provisioner: kubernetes.io/vsphere-volume
parameters:
diskformat: zeroedthick
datastore: VSANDatastore
`datastore`: The user can also specify the datastore in the StorageClass.
The volume will be created on the datastore specified in the storage class,
which in this case is `VSANDatastore`. This field is optional. If the
datastore is not specified, then the volume will be created on the datastore
specified in the vSphere config file used to initialize the vSphere Cloud
Provider.
3. Storage Policy Management inside kubernetes
* Using existing vCenter SPBM policy
One of the most important features of vSphere for Storage Management is
policy based Management. Storage Policy Based Management (SPBM) is a
storage policy framework that provides a single unified control plane
across a broad range of data services and storage solutions. SPBM enables
vSphere administrators to overcome upfront storage provisioning challenges,
such as capacity planning, differentiated service levels and managing
capacity headroom.
The SPBM policies can be specified in the StorageClass using the
`storagePolicyName` parameter.
* Virtual SAN policy support inside Kubernetes
Vsphere Infrastructure (VI) Admins will have the ability to specify custom
Virtual SAN Storage Capabilities during dynamic volume provisioning. You
can now define storage requirements, such as performance and availability,
in the form of storage capabilities during dynamic volume provisioning.
The storage capability requirements are converted into a Virtual SAN
policy which are then pushed down to the Virtual SAN layer when a
persistent volume (virtual disk) is being created. The virtual disk is
distributed across the Virtual SAN datastore to meet the requirements.
You can see [Storage Policy Based Management for dynamic provisioning of volumes](https://vmware.github.io/vsphere-storage-for-kubernetes/documentation/policy-based-mgmt.html)
for more details on how to use storage policies for persistent volumes
management.
There are few
[vSphere examples](https://github.com/kubernetes/examples/tree/master/staging/volumes/vsphere)
which you try out for persistent volume management inside Kubernetes for vSphere.
### Ceph RBD
```yaml
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: fast
provisioner: kubernetes.io/rbd
parameters:
monitors: 10.16.153.105:6789
adminId: kube
adminSecretName: ceph-secret
adminSecretNamespace: kube-system
pool: kube
userId: kube
userSecretName: ceph-secret-user
fsType: ext4
imageFormat: "2"
imageFeatures: "layering"
```
* `monitors`: Ceph monitors, comma delimited. This parameter is required.
* `adminId`: Ceph client ID that is capable of creating images in the pool.
Default is "admin".
* `adminSecretNamespace`: The namespace for `adminSecret`. Default is "default".
* `adminSecret`: Secret Name for `adminId`. This parameter is required.
The provided secret must have type "kubernetes.io/rbd".
* `pool`: Ceph RBD pool. Default is "rbd".
* `userId`: Ceph client ID that is used to map the RBD image. Default is the
same as `adminId`.
* `userSecretName`: The name of Ceph Secret for `userId` to map RBD image. It
must exist in the same namespace as PVCs. This parameter is required.
The provided secret must have type "kubernetes.io/rbd", e.g. created in this
way:
```
kubectl create secret generic ceph-secret --type="kubernetes.io/rbd" \
--from-literal=key='QVFEQ1pMdFhPUnQrSmhBQUFYaERWNHJsZ3BsMmNjcDR6RFZST0E9PQ==' \
--namespace=kube-system
```
* `fsType`: fsType that is supported by kubernetes. Default: `"ext4"`.
* `imageFormat`: Ceph RBD image format, "1" or "2". Default is "1".
* `imageFeatures`: This parameter is optional and should only be used if you
set `imageFormat` to "2". Currently supported features are `layering` only.
Default is "", and no features are turned on.
#### Quobyte
```yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: slow
provisioner: kubernetes.io/quobyte
parameters:
quobyteAPIServer: "http://138.68.74.142:7860"
registry: "138.68.74.142:7861"
adminSecretName: "quobyte-admin-secret"
adminSecretNamespace: "kube-system"
user: "root"
group: "root"
quobyteConfig: "BASE"
quobyteTenant: "DEFAULT"
```
* `quobyteAPIServer`: API Server of Quobyte in the format
`"http(s)://api-server:7860"`
* `registry`: Quobyte registry to use to mount the volume. You can specify the
registry as ``<host>:<port>`` pair or if you want to specify multiple
registries you just have to put a comma between them e.q.
``<host1>:<port>,<host2>:<port>,<host3>:<port>``.
The host can be an IP address or if you have a working DNS you can also
provide the DNS names.
* `adminSecretNamespace`: The namespace for `adminSecretName`.
Default is "default".
* `adminSecretName`: secret that holds information about the Quobyte user and
the password to authenticate against the API server. The provided secret
must have type "kubernetes.io/quobyte", e.g. created in this way:
```
kubectl create secret generic quobyte-admin-secret \
--type="kubernetes.io/quobyte" --from-literal=key='opensesame' \
--namespace=kube-system
```
* `user`: maps all access to this user. Default is "root".
* `group`: maps all access to this group. Default is "nfsnobody".
* `quobyteConfig`: use the specified configuration to create the volume. You
can create a new configuration or modify an existing one with the Web
console or the quobyte CLI. Default is "BASE".
* `quobyteTenant`: use the specified tenant ID to create/delete the volume.
This Quobyte tenant has to be already present in Quobyte.
Default is "DEFAULT".
### Azure Disk
#### Azure Unmanaged Disk Storage Class
```yaml
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: slow
provisioner: kubernetes.io/azure-disk
parameters:
skuName: Standard_LRS
location: eastus
storageAccount: azure_storage_account_name
```
* `skuName`: Azure storage account Sku tier. Default is empty.
* `location`: Azure storage account location. Default is empty.
* `storageAccount`: Azure storage account name. If a storage account is provided,
it must reside in the same resource group as the cluster, and `location` is
ignored. If a storage account is not provided, a new storage account will be
created in the same resource group as the cluster.
#### New Azure Disk Storage Class (starting from v1.7.2)
```yaml
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: slow
provisioner: kubernetes.io/azure-disk
parameters:
storageaccounttype: Standard_LRS
kind: Shared
```
* `storageaccounttype`: Azure storage account Sku tier. Default is empty.
* `kind`: Possible values are `shared` (default), `dedicated`, and `managed`.
When `kind` is `shared`, all unmanaged disks are created in a few shared
storage accounts in the same resource group as the cluster. When `kind` is
`dedicated`, a new dedicated storage account will be created for the new
unmanaged disk in the same resource group as the cluster.
- Premium VM can attach both Standard_LRS and Premium_LRS disks, while Standard
VM can only attach Standard_LRS disks.
- Managed VM can only attach managed disks and unmanaged VM can only attach
unmanaged disks.
### Azure File
```yaml
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: azurefile
provisioner: kubernetes.io/azure-file
parameters:
skuName: Standard_LRS
location: eastus
storageAccount: azure_storage_account_name
```
* `skuName`: Azure storage account Sku tier. Default is empty.
* `location`: Azure storage account location. Default is empty.
* `storageAccount`: Azure storage account name. Default is empty. If a storage
account is not provided, all storage accounts associated with the resource
group are searched to find one that matches `skuName` and `location`. If a
storage account is provided, it must reside in the same resource group as the
cluster, and `skuName` and `location` are ignored.
During provision, a secret is created for mounting credentials. If the cluster
has enabled both [RBAC](/docs/admin/authorization/rbac/) and
[Controller Roles](/docs/admin/authorization/rbac/#controller-roles), add the
`create` permission of resource `secret` for clusterrole
`system:controller:persistent-volume-binder`.
### Portworx Volume
```yaml
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: portworx-io-priority-high
provisioner: kubernetes.io/portworx-volume
parameters:
repl: "1"
snap_interval: "70"
io_priority: "high"
```
* `fs`: filesystem to be laid out: [none/xfs/ext4] (default: `ext4`).
* `block_size`: block size in Kbytes (default: `32`).
* `repl`: number of synchronous replicas to be provided in the form of
replication factor [1..3] (default: `1`) A string is expected here i.e.
`"1"` and not `1`.
* `io_priority`: determines whether the volume will be created from higher
performance or a lower priority storage [high/medium/low] (default: `low`).
* `snap_interval`: clock/time interval in minutes for when to trigger snapshots.
Snapshots are incremental based on difference with the prior snapshot, 0
disables snaps (default: `0`). A string is expected here i.e.
`"70"` and not `70`.
* `aggregation_level`: specifies the number of chunks the volume would be
distributed into, 0 indicates a non-aggregated volume (default: `0`). A string
is expected here i.e. `"0"` and not `0`
* `ephemeral`: specifies whether the volume should be cleaned-up after unmount
or should be persistent. `emptyDir` use case can set this value to true and
`persistent volumes` use case such as for databases like Cassandra should set
to false, [true/false] (default `false`). A string is expected here i.e.
`"true"` and not `true`.
### ScaleIO
```yaml
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: slow
provisioner: kubernetes.io/scaleio
parameters:
gateway: https://192.168.99.200:443/api
system: scaleio
protectionDomain: pd0
storagePool: sp1
storageMode: ThinProvisioned
secretRef: sio-secret
readOnly: false
fsType: xfs
```
* `provisioner`: attribute is set to `kubernetes.io/scaleio`
* `gateway`: address to a ScaleIO API gateway (required)
* `system`: the name of the ScaleIO system (required)
* `protectionDomain`: the name of the ScaleIO protection domain (required)
* `storagePool`: the name of the volume storage pool (required)
* `storageMode`: the storage provision mode: `ThinProvisioned` (default) or
`ThickProvisioned`
* `secretRef`: reference to a configured Secret object (required)
* `readOnly`: specifies the access mode to the mounted volume (default false)
* `fsType`: the file system to use for the volume (default ext4)
The ScaleIO Kubernetes volume plugin requires a configured Secret object.
The secret must be created with type `kubernetes.io/scaleio` and use the same
namespace value as that of the PVC where it is referenced
as shown in the following command:
```shell
kubectl create secret generic sio-secret --type="kubernetes.io/scaleio" \
--from-literal=username=sioadmin --from-literal=password=d2NABDNjMA== \
--namespace=default
```
### StorageOS
```yaml
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: fast
provisioner: kubernetes.io/storageos
parameters:
pool: default
description: Kubernetes volume
fsType: ext4
adminSecretNamespace: default
adminSecretName: storageos-secret
```
* `pool`: The name of the StorageOS distributed capacity pool to provision the
volume from. Uses the `default` pool which is normally present if not specified.
* `description`: The description to assign to volumes that were created dynamically.
All volume descriptions will be the same for the storage class, but different
storage classes can be used to allow descriptions for different use cases.
Defaults to `Kubernetes volume`.
* `fsType`: The default filesystem type to request. Note that user-defined rules
within StorageOS may override this value. Defaults to `ext4`.
* `adminSecretNamespace`: The namespace where the API configuration secret is
located. Required if adminSecretName set.
* `adminSecretName`: The name of the secret to use for obtaining the StorageOS
API credentials. If not specified, default values will be attempted.
The StorageOS Kubernetes volume plugin can use a Secret object to specify an
endpoint and credentials to access the StorageOS API. This is only required when
the defaults have been changed.
The secret must be created with type `kubernetes.io/storageos` as shown in the
following command:
```shell
kubectl create secret generic storageos-secret \
--type="kubernetes.io/storageos" \
--from-literal=apiAddress=tcp://localhost:5705 \
--from-literal=apiUsername=storageos \
--from-literal=apiPassword=storageos \
--namespace=default
```
Secrets used for dynamically provisioned volumes may be created in any namespace
and referenced with the `adminSecretNamespace` parameter. Secrets used by
pre-provisioned volumes must be created in the same namespace as the PVC that
references it.
docs/concepts/storage/volumes.md
+461 -437
View File
@@ -65,33 +65,118 @@ mount each volume.
Kubernetes supports several types of Volumes:
* `emptyDir`
* `hostPath`
* `gcePersistentDisk`
* `awsElasticBlockStore`
* `nfs`
* `iscsi`
* `fc (fibre channel)`
* `flocker`
* `glusterfs`
* `rbd`
* `cephfs`
* `gitRepo`
* `secret`
* `persistentVolumeClaim`
* `downwardAPI`
* `projected`
* `azureFileVolume`
* `azureDisk`
* `vsphereVolume`
* `Quobyte`
* `PortworxVolume`
* `ScaleIO`
* `StorageOS`
* `azureFile`
* `cephfs`
* `downwardAPI`
* `emptyDir`
* `fc` (fibre channel)
* `flocker`
* `gcePersistentDisk`
* `gitRepo`
* `glusterfs`
* `hostPath`
* `iscsi`
* `local`
* `nfs`
* `persistentVolumeClaim`
* `projected`
* `portworxVolume`
* `quobyte`
* `rbd`
* `scaleIO`
* `secret`
* `storageos`
* `vsphereVolume`
We welcome additional contributions.
### awsElasticBlockStore
An `awsElasticBlockStore` volume mounts an Amazon Web Services (AWS) [EBS
Volume](http://aws.amazon.com/ebs/) into your pod. Unlike
`emptyDir`, which is erased when a Pod is removed, the contents of an EBS
volume are preserved and the volume is merely unmounted. This means that an
EBS volume can be pre-populated with data, and that data can be "handed off"
between pods.
**Important:** You must create an EBS volume using `aws ec2 create-volume` or the AWS API before you can use it.
{: .caution}
There are some restrictions when using an awsElasticBlockStore volume:
* the nodes on which pods are running must be AWS EC2 instances
* those instances need to be in the same region and availability-zone as the EBS volume
* EBS only supports a single EC2 instance mounting a volume
#### Creating an EBS volume
Before you can use an EBS volume with a pod, you need to create it.
```shell
aws ec2 create-volume --availability-zone=eu-west-1a --size=10 --volume-type=gp2
```
Make sure the zone matches the zone you brought up your cluster in. (And also check that the size and EBS volume
type are suitable for your use!)
#### AWS EBS Example configuration
```yaml
apiVersion: v1
kind: Pod
metadata:
name: test-ebs
spec:
containers:
- image: gcr.io/google_containers/test-webserver
name: test-container
volumeMounts:
- mountPath: /test-ebs
name: test-volume
volumes:
- name: test-volume
# This AWS EBS volume must already exist.
awsElasticBlockStore:
volumeID: <volume-id>
fsType: ext4
```
### azureDisk
A `azureDisk` is used to mount a Microsoft Azure [Data Disk](https://azure.microsoft.com/en-us/documentation/articles/virtual-machines-linux-about-disks-vhds/) into a Pod.
More details can be found [here](https://github.com/kubernetes/examples/tree/{{page.githubbranch}}/staging/volumes/azure_disk/README.md).
### azureFile
A `azureFile` is used to mount a Microsoft Azure File Volume (SMB 2.1 and 3.0)
into a Pod.
More details can be found [here](https://github.com/kubernetes/examples/tree/{{page.githubbranch}}/staging/volumes/azure_file/README.md).
### cephfs
A `cephfs` volume allows an existing CephFS volume to be
mounted into your pod. Unlike `emptyDir`, which is erased when a Pod is
removed, the contents of a `cephfs` volume are preserved and the volume is merely
unmounted. This means that a CephFS volume can be pre-populated with data, and
that data can be "handed off" between pods. CephFS can be mounted by multiple
writers simultaneously.
**Important:** You must have your own Ceph server running with the share exported before you can use it.
{: .caution}
See the [CephFS example](https://github.com/kubernetes/examples/tree/{{page.githubbranch}}/staging/volumes/cephfs/) for more details.
### downwardAPI
A `downwardAPI` volume is used to make downward API data available to applications.
It mounts a directory and writes the requested data in plain text files.
See the [`downwardAPI` volume example](/docs/tasks/inject-data-application/downward-api-volume-expose-pod-information/) for more details.
### emptyDir
An `emptyDir` volume is first created when a Pod is assigned to a Node, and
@@ -99,7 +184,7 @@ exists as long as that Pod is running on that node. As the name says, it is
initially empty. Containers in the pod can all read and write the same
files in the `emptyDir` volume, though that volume can be mounted at the same
or different paths in each container. When a Pod is removed from a node for
any reason, the data in the `emptyDir` is deleted forever.
any reason, the data in the `emptyDir` is deleted forever.
**Note:** a container crashing does *NOT* remove a pod from a node, so the data in an `emptyDir` volume is safe across container crashes.
{: .note}
@@ -138,6 +223,132 @@ spec:
emptyDir: {}
```
### fc (fibre channel)
An `fc` volume allows an existing fibre channel volume to be mounted in a pod.
You can specify single or multiple target World Wide Names using the parameter
`targetWWNs` in your volume configuration. If multiple WWNs are specified,
targetWWNs expect that those WWNs are from multi-path connections.
**Important:** You must configure FC SAN Zoning to allocate and mask those LUNs (volumes) to the target WWNs beforehand so that Kubernetes hosts can access them.
{: .caution}
See the [FC example](https://github.com/kubernetes/examples/tree/{{page.githubbranch}}/staging/volumes/fibre_channel) for more details.
### flocker
[Flocker](https://clusterhq.com/flocker) is an open-source clustered container data volume manager. It provides management
and orchestration of data volumes backed by a variety of storage backends.
A `flocker` volume allows a Flocker dataset to be mounted into a pod. If the
dataset does not already exist in Flocker, it needs to be first created with the Flocker
CLI or by using the Flocker API. If the dataset already exists it will be
reattached by Flocker to the node that the pod is scheduled. This means data
can be "handed off" between pods as required.
**Important:** You must have your own Flocker installation running before you can use it.
{: .caution}
See the [Flocker example](https://github.com/kubernetes/examples/tree/{{page.githubbranch}}/staging/volumes/flocker) for more details.
### gcePersistentDisk
A `gcePersistentDisk` volume mounts a Google Compute Engine (GCE) [Persistent
Disk](http://cloud.google.com/compute/docs/disks) into your pod. Unlike
`emptyDir`, which is erased when a Pod is removed, the contents of a PD are
preserved and the volume is merely unmounted. This means that a PD can be
pre-populated with data, and that data can be "handed off" between pods.
**Important:** You must create a PD using `gcloud` or the GCE API or UI before you can use it.
{: .caution}
There are some restrictions when using a `gcePersistentDisk`:
* the nodes on which pods are running must be GCE VMs
* those VMs need to be in the same GCE project and zone as the PD
A feature of PD is that they can be mounted as read-only by multiple consumers
simultaneously. This means that you can pre-populate a PD with your dataset
and then serve it in parallel from as many pods as you need. Unfortunately,
PDs can only be mounted by a single consumer in read-write mode - no
simultaneous writers allowed.
Using a PD on a pod controlled by a ReplicationController will fail unless
the PD is read-only or the replica count is 0 or 1.
#### Creating a PD
Before you can use a GCE PD with a pod, you need to create it.
```shell
gcloud compute disks create --size=500GB --zone=us-central1-a my-data-disk
```
#### Example pod
```yaml
apiVersion: v1
kind: Pod
metadata:
name: test-pd
spec:
containers:
- image: gcr.io/google_containers/test-webserver
name: test-container
volumeMounts:
- mountPath: /test-pd
name: test-volume
volumes:
- name: test-volume
# This GCE PD must already exist.
gcePersistentDisk:
pdName: my-data-disk
fsType: ext4
```
### gitRepo
A `gitRepo` volume is an example of what can be done as a volume plugin. It
mounts an empty directory and clones a git repository into it for your pod to
use. In the future, such volumes may be moved to an even more decoupled model,
rather than extending the Kubernetes API for every such use case.
Here is an example for gitRepo volume:
```yaml
apiVersion: v1
kind: Pod
metadata:
name: server
spec:
containers:
- image: nginx
name: nginx
volumeMounts:
- mountPath: /mypath
name: git-volume
volumes:
- name: git-volume
gitRepo:
repository: "git@somewhere:me/my-git-repository.git"
revision: "22f1d8406d464b0c0874075539c1f2e96c253775"
```
### glusterfs
A `glusterfs` volume allows a [Glusterfs](http://www.gluster.org) (an open
source networked filesystem) volume to be mounted into your pod. Unlike
`emptyDir`, which is erased when a Pod is removed, the contents of a
`glusterfs` volume are preserved and the volume is merely unmounted. This
means that a glusterfs volume can be pre-populated with data, and that data can
be "handed off" between pods. GlusterFS can be mounted by multiple writers
simultaneously.
**Important:** You must have your own GlusterFS installation running before you can use it.
{: .caution}
See the [GlusterFS example](https://github.com/kubernetes/examples/tree/{{page.githubbranch}}/staging/volumes/glusterfs) for more details.
### hostPath
A `hostPath` volume mounts a file or directory from the host node's filesystem
@@ -202,126 +413,6 @@ spec:
type: Directory
```
### gcePersistentDisk
A `gcePersistentDisk` volume mounts a Google Compute Engine (GCE) [Persistent
Disk](http://cloud.google.com/compute/docs/disks) into your pod. Unlike
`emptyDir`, which is erased when a Pod is removed, the contents of a PD are
preserved and the volume is merely unmounted. This means that a PD can be
pre-populated with data, and that data can be "handed off" between pods.
**Important:** You must create a PD using `gcloud` or the GCE API or UI before you can use it.
{: .caution}
There are some restrictions when using a `gcePersistentDisk`:
* the nodes on which pods are running must be GCE VMs
* those VMs need to be in the same GCE project and zone as the PD
A feature of PD is that they can be mounted as read-only by multiple consumers
simultaneously. This means that you can pre-populate a PD with your dataset
and then serve it in parallel from as many pods as you need. Unfortunately,
PDs can only be mounted by a single consumer in read-write mode - no
simultaneous writers allowed.
Using a PD on a pod controlled by a ReplicationController will fail unless
the PD is read-only or the replica count is 0 or 1.
#### Creating a PD
Before you can use a GCE PD with a pod, you need to create it.
```shell
gcloud compute disks create --size=500GB --zone=us-central1-a my-data-disk
```
#### Example pod
```yaml
apiVersion: v1
kind: Pod
metadata:
name: test-pd
spec:
containers:
- image: gcr.io/google_containers/test-webserver
name: test-container
volumeMounts:
- mountPath: /test-pd
name: test-volume
volumes:
- name: test-volume
# This GCE PD must already exist.
gcePersistentDisk:
pdName: my-data-disk
fsType: ext4
```
### awsElasticBlockStore
An `awsElasticBlockStore` volume mounts an Amazon Web Services (AWS) [EBS
Volume](http://aws.amazon.com/ebs/) into your pod. Unlike
`emptyDir`, which is erased when a Pod is removed, the contents of an EBS
volume are preserved and the volume is merely unmounted. This means that an
EBS volume can be pre-populated with data, and that data can be "handed off"
between pods.
**Important:** You must create an EBS volume using `aws ec2 create-volume` or the AWS API before you can use it.
{: .caution}
There are some restrictions when using an awsElasticBlockStore volume:
* the nodes on which pods are running must be AWS EC2 instances
* those instances need to be in the same region and availability-zone as the EBS volume
* EBS only supports a single EC2 instance mounting a volume
#### Creating an EBS volume
Before you can use an EBS volume with a pod, you need to create it.
```shell
aws ec2 create-volume --availability-zone=eu-west-1a --size=10 --volume-type=gp2
```
Make sure the zone matches the zone you brought up your cluster in. (And also check that the size and EBS volume
type are suitable for your use!)
#### AWS EBS Example configuration
```yaml
apiVersion: v1
kind: Pod
metadata:
name: test-ebs
spec:
containers:
- image: gcr.io/google_containers/test-webserver
name: test-container
volumeMounts:
- mountPath: /test-ebs
name: test-volume
volumes:
- name: test-volume
# This AWS EBS volume must already exist.
awsElasticBlockStore:
volumeID: <volume-id>
fsType: ext4
```
### nfs
An `nfs` volume allows an existing NFS (Network File System) share to be
mounted into your pod. Unlike `emptyDir`, which is erased when a Pod is
removed, the contents of an `nfs` volume are preserved and the volume is merely
unmounted. This means that an NFS volume can be pre-populated with data, and
that data can be "handed off" between pods. NFS can be mounted by multiple
writers simultaneously.
**Important:** You must have your own NFS server running with the share exported before you can use it.
{: .caution}
See the [NFS example](https://github.com/kubernetes/examples/tree/{{page.githubbranch}}/staging/volumes/nfs) for more details.
### iscsi
An `iscsi` volume allows an existing iSCSI (SCSI over IP) volume to be mounted
@@ -341,123 +432,71 @@ simultaneous writers allowed.
See the [iSCSI example](https://github.com/kubernetes/examples/tree/{{page.githubbranch}}/staging/volumes/iscsi) for more details.
### fc (fibre channel)
### local
An `fc` volume allows an existing fibre channel volume to be mounted in a pod.
You can specify single or multiple target World Wide Names using the parameter
`targetWWNs` in your volume configuration. If multiple WWNs are specified,
targetWWNs expect that those WWNs are from multi-path connections.
This volume type is alpha in 1.7.
**Important:** You must configure FC SAN Zoning to allocate and mask those LUNs (volumes) to the target WWNs beforehand so that Kubernetes hosts can access them.
{: .caution}
A `local` volume represents a mounted local storage device such as a disk,
partition or directory.
See the [FC example](https://github.com/kubernetes/examples/tree/{{page.githubbranch}}/staging/volumes/fibre_channel) for more details.
Local volumes can only be used as a statically created PersistentVolume.
### flocker
Compared to HostPath volumes, local volumes can be used in a durable manner
without manually scheduling pods to nodes, as the system is aware of the volume's
node constraints.
[Flocker](https://clusterhq.com/flocker) is an open-source clustered container data volume manager. It provides management
and orchestration of data volumes backed by a variety of storage backends.
However, local volumes are still subject to the availability of the underlying
node and are not suitable for all applications.
A `flocker` volume allows a Flocker dataset to be mounted into a pod. If the
dataset does not already exist in Flocker, it needs to be first created with the Flocker
CLI or by using the Flocker API. If the dataset already exists it will be
reattached by Flocker to the node that the pod is scheduled. This means data
can be "handed off" between pods as required.
The following is an example PersistentVolume spec using a `local` volume:
**Important:** You must have your own Flocker installation running before you can use it.
{: .caution}
See the [Flocker example](https://github.com/kubernetes/examples/tree/{{page.githubbranch}}/staging/volumes/flocker) for more details.
### glusterfs
A `glusterfs` volume allows a [Glusterfs](http://www.gluster.org) (an open
source networked filesystem) volume to be mounted into your pod. Unlike
`emptyDir`, which is erased when a Pod is removed, the contents of a
`glusterfs` volume are preserved and the volume is merely unmounted. This
means that a glusterfs volume can be pre-populated with data, and that data can
be "handed off" between pods. GlusterFS can be mounted by multiple writers
simultaneously.
**Important:** You must have your own GlusterFS installation running before you can use it.
{: .caution}
See the [GlusterFS example](https://github.com/kubernetes/examples/tree/{{page.githubbranch}}/staging/volumes/glusterfs) for more details.
### rbd
An `rbd` volume allows a [Rados Block
Device](http://ceph.com/docs/master/rbd/rbd/) volume to be mounted into your
pod. Unlike `emptyDir`, which is erased when a Pod is removed, the contents of
a `rbd` volume are preserved and the volume is merely unmounted. This
means that a RBD volume can be pre-populated with data, and that data can
be "handed off" between pods.
**Important:** You must have your own Ceph installation running before you can use RBD.
{: .caution}
A feature of RBD is that it can be mounted as read-only by multiple consumers
simultaneously. This means that you can pre-populate a volume with your dataset
and then serve it in parallel from as many pods as you need. Unfortunately,
RBD volumes can only be mounted by a single consumer in read-write mode - no
simultaneous writers allowed.
See the [RBD example](https://github.com/kubernetes/examples/tree/{{page.githubbranch}}/staging/volumes/rbd) for more details.
### cephfs
A `cephfs` volume allows an existing CephFS volume to be
mounted into your pod. Unlike `emptyDir`, which is erased when a Pod is
removed, the contents of a `cephfs` volume are preserved and the volume is merely
unmounted. This means that a CephFS volume can be pre-populated with data, and
that data can be "handed off" between pods. CephFS can be mounted by multiple
writers simultaneously.
**Important:** You must have your own Ceph server running with the share exported before you can use it.
{: .caution}
See the [CephFS example](https://github.com/kubernetes/examples/tree/{{page.githubbranch}}/staging/volumes/cephfs/) for more details.
### gitRepo
A `gitRepo` volume is an example of what can be done as a volume plugin. It
mounts an empty directory and clones a git repository into it for your pod to
use. In the future, such volumes may be moved to an even more decoupled model,
rather than extending the Kubernetes API for every such use case.
Here is an example for gitRepo volume:
```yaml
``` yaml
apiVersion: v1
kind: Pod
kind: PersistentVolume
metadata:
name: server
name: example-pv
annotations:
"volume.alpha.kubernetes.io/node-affinity": '{
"requiredDuringSchedulingIgnoredDuringExecution": {
"nodeSelectorTerms": [
{ "matchExpressions": [
{ "key": "kubernetes.io/hostname",
"operator": "In",
"values": ["example-node"]
}
]}
]}
}'
spec:
containers:
- image: nginx
name: nginx
volumeMounts:
- mountPath: /mypath
name: git-volume
volumes:
- name: git-volume
gitRepo:
repository: "git@somewhere:me/my-git-repository.git"
revision: "22f1d8406d464b0c0874075539c1f2e96c253775"
capacity:
storage: 100Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Delete
storageClassName: local-storage
local:
path: /mnt/disks/ssd1
```
### secret
**Note:** The local PersistentVolume cleanup and deletion requires manual intervention without the external provisioner.
{: .note}
A `secret` volume is used to pass sensitive information, such as passwords, to
pods. You can store secrets in the Kubernetes API and mount them as files for
use by pods without coupling to Kubernetes directly. `secret` volumes are
backed by tmpfs (a RAM-backed filesystem) so they are never written to
non-volatile storage.
For details on the `local` volume type, see the [Local Persistent Storage
user guide](https://github.com/kubernetes-incubator/external-storage/tree/master/local-volume).
**Important:** You must create a secret in the Kubernetes API before you can use it.
### nfs
An `nfs` volume allows an existing NFS (Network File System) share to be
mounted into your pod. Unlike `emptyDir`, which is erased when a Pod is
removed, the contents of an `nfs` volume are preserved and the volume is merely
unmounted. This means that an NFS volume can be pre-populated with data, and
that data can be "handed off" between pods. NFS can be mounted by multiple
writers simultaneously.
**Important:** You must have your own NFS server running with the share exported before you can use it.
{: .caution}
Secrets are described in more detail [here](/docs/user-guide/secrets).
See the [NFS example](https://github.com/kubernetes/examples/tree/{{page.githubbranch}}/staging/volumes/nfs) for more details.
### persistentVolumeClaim
@@ -469,13 +508,6 @@ iSCSI volume) without knowing the details of the particular cloud environment.
See the [PersistentVolumes example](/docs/concepts/storage/persistent-volumes/) for more
details.
### downwardAPI
A `downwardAPI` volume is used to make downward API data available to applications.
It mounts a directory and writes the requested data in plain text files.
See the [`downwardAPI` volume example](/docs/tasks/inject-data-application/downward-api-volume-expose-pod-information/) for more details.
### projected
A `projected` volume maps several existing volume sources into the same directory.
@@ -483,7 +515,7 @@ A `projected` volume maps several existing volume sources into the same director
Currently, the following types of volume sources can be projected:
- [`secret`](#secret)
- [`downwardAPI`](#downardapi)
- [`downwardAPI`](#downwardapi)
- `configMap`
All sources are required to be in the same namespace as the pod. For more details, see the [all-in-one volume design document](https://github.com/kubernetes/community/blob/{{page.githubbranch}}/contributors/design-proposals/node/all-in-one-volume.md).
@@ -564,27 +596,189 @@ Each projected volume source is listed in the spec under `sources`. The
parameters are nearly the same with two exceptions:
* For secrets, the `secretName` field has been changed to `name` to be consistent
with ConfigMap naming.
with ConfigMap naming.
* The `defaultMode` can only be specified at the projected level and not for each
volume source. However, as illustrated above, you can explicitly set the `mode`
for each individual projection.
volume source. However, as illustrated above, you can explicitly set the `mode`
for each individual projection.
### AzureFileVolume
### portworxVolume
A `AzureFileVolume` is used to mount a Microsoft Azure File Volume (SMB 2.1 and 3.0)
into a Pod.
A `portworxVolume` is an elastic block storage layer that runs hyperconverged with
Kubernetes. Portworx fingerprints storage in a server, tiers based on capabilities,
and aggregates capacity across multiple servers. Portworx runs in-guest in virtual
machines or on bare metal Linux nodes.
More details can be found [here](https://github.com/kubernetes/examples/tree/{{page.githubbranch}}/staging/volumes/azure_file/README.md).
A `portworxVolume` can be dynamically created through Kubernetes or it can also
be pre-provisioned and referenced inside a Kubernetes pod.
Here is an example pod referencing a pre-provisioned PortworxVolume:
### AzureDiskVolume
```yaml
apiVersion: v1
kind: Pod
metadata:
name: test-portworx-volume-pod
spec:
containers:
- image: gcr.io/google_containers/test-webserver
name: test-container
volumeMounts:
- mountPath: /mnt
name: pxvol
volumes:
- name: pxvol
# This Portworx volume must already exist.
portworxVolume:
volumeID: "pxvol"
fsType: "<fs-type>"
```
A `AzureDiskVolume` is used to mount a Microsoft Azure [Data Disk](https://azure.microsoft.com/en-us/documentation/articles/virtual-machines-linux-about-disks-vhds/) into a Pod.
**Important:** Make sure you have an existing PortworxVolume with name `pxvol`
before using it in the pod.
{: .caution}
More details can be found [here](https://github.com/kubernetes/examples/tree/{{page.githubbranch}}/staging/volumes/azure_disk/README.md).
More details and examples can be found [here](https://github.com/kubernetes/examples/tree/{{page.githubbranch}}/staging/volumes/portworx/README.md).
### quobyte
A `quobyte` volume allows an existing [Quobyte](http://www.quobyte.com) volume to
be mounted into your pod.
**Important:** You must have your own Quobyte setup running with the volumes
created before you can use it.
{: .caution}
See the [Quobyte example](https://github.com/kubernetes/examples/tree/{{page.githubbranch}}/staging/volumes/quobyte) for more details.
### rbd
An `rbd` volume allows a [Rados Block
Device](http://ceph.com/docs/master/rbd/rbd/) volume to be mounted into your
pod. Unlike `emptyDir`, which is erased when a Pod is removed, the contents of
a `rbd` volume are preserved and the volume is merely unmounted. This
means that a RBD volume can be pre-populated with data, and that data can
be "handed off" between pods.
**Important:** You must have your own Ceph installation running before you can use RBD.
{: .caution}
A feature of RBD is that it can be mounted as read-only by multiple consumers
simultaneously. This means that you can pre-populate a volume with your dataset
and then serve it in parallel from as many pods as you need. Unfortunately,
RBD volumes can only be mounted by a single consumer in read-write mode - no
simultaneous writers allowed.
See the [RBD example](https://github.com/kubernetes/examples/tree/{{page.githubbranch}}/staging/volumes/rbd) for more details.
### scaleIO
ScaleIO is a software-based storage platform that can use existing hardware to
create clusters of scalable shared block networked storage. The `scaleIO` volume
plugin allows deployed pods to access existing ScaleIO
volumes (or it can dynamically provision new volumes for persistent volume claims, see
[ScaleIO Persistent Volumes](/docs/concepts/storage/persistent-volumes/#scaleio)).
**Important:** You must have an existing ScaleIO cluster already setup and
running with the volumes created before you can use them.
{: .caution}
The following is an example pod configuration with ScaleIO:
```yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-0
spec:
containers:
- image: gcr.io/google_containers/test-webserver
name: pod-0
volumeMounts:
- mountPath: /test-pd
name: vol-0
volumes:
- name: vol-0
scaleIO:
gateway: https://localhost:443/api
system: scaleio
protectionDomain: sd0
storagePool: sp1
volumeName: vol-0
secretRef:
name: sio-secret
fsType: xfs
```
For further detail, please the see the [ScaleIO examples](https://github.com/kubernetes/examples/tree/{{page.githubbranch}}/staging/volumes/scaleio).
### secret
A `secret` volume is used to pass sensitive information, such as passwords, to
pods. You can store secrets in the Kubernetes API and mount them as files for
use by pods without coupling to Kubernetes directly. `secret` volumes are
backed by tmpfs (a RAM-backed filesystem) so they are never written to
non-volatile storage.
**Important:** You must create a secret in the Kubernetes API before you can use it.
{: .caution}
Secrets are described in more detail [here](/docs/user-guide/secrets).
### storageOS
A `storageos` volume allows an existing [StorageOS](https://www.storageos.com)
volume to be mounted into your pod.
StorageOS runs as a container within your Kubernetes environment, making local
or attached storage accessible from any node within the Kubernetes cluster.
Data can be replicated to protect against node failure. Thin provisioning and
compression can improve utilization and reduce cost.
At its core, StorageOS provides block storage to containers, accessible via a file system.
The StorageOS container requires 64-bit Linux and has no additional dependencies.
A free developer license is available.
**Important:** You must run the StorageOS container on each node that wants to
access StorageOS volumes or that will contribute storage capacity to the pool.
For installation instructions, consult the
[StorageOS documentation](https://docs.storageos.com).
{: .caution}
```yaml
apiVersion: v1
kind: Pod
metadata:
labels:
name: redis
role: master
name: test-storageos-redis
spec:
containers:
- name: master
image: kubernetes/redis:v1
env:
- name: MASTER
value: "true"
ports:
- containerPort: 6379
volumeMounts:
- mountPath: /redis-master-data
name: redis-data
volumes:
- name: redis-data
storageos:
# The `redis-vol01` volume must already exist within StorageOS in the `default` namespace.
volumeName: redis-vol01
fsType: ext4
```
For more information including Dynamic Provisioning and Persistent Volume Claims, please see the
[StorageOS examples](https://github.com/kubernetes/kubernetes/tree/master/examples/volumes/storageos).
### vsphereVolume
**Prerequisite:** Kubernetes with vSphere Cloud Provider configured. For cloudprovider configuration please refer [vSphere getting started guide](/docs/getting-started-guides/vsphere/).
**Prerequisite:** Kubernetes with vSphere Cloud Provider configured. For cloudprovider
configuration please refer [vSphere getting started guide](/docs/getting-started-guides/vsphere/).
{: .note}
A `vsphereVolume` is used to mount a vSphere VMDK Volume into your Pod. The contents
@@ -638,183 +832,10 @@ spec:
volumePath: "[DatastoreName] volumes/myDisk"
fsType: ext4
```
More examples can be found [here](https://github.com/kubernetes/examples/tree/master/staging/volumes/vsphere).
### Quobyte
A `Quobyte` volume allows an existing [Quobyte](http://www.quobyte.com) volume to be mounted into your pod.
**Important:** You must have your own Quobyte setup running with the volumes created before you can use it.
{: .caution}
See the [Quobyte example](https://github.com/kubernetes/examples/tree/{{page.githubbranch}}/staging/volumes/quobyte) for more details.
### PortworxVolume
A `PortworxVolume` is an elastic block storage layer that runs hyperconverged with Kubernetes. Portworx fingerprints storage in a
server, tiers based on capabilities, and aggregates capacity across multiple servers. Portworx runs in-guest in virtual machines or on bare metal
Linux nodes.
A `PortworxVolume` can be dynamically created through Kubernetes or it can also be pre-provisioned and referenced inside a Kubernetes pod.
Here is an example pod referencing a pre-provisioned PortworxVolume:
```yaml
apiVersion: v1
kind: Pod
metadata:
name: test-portworx-volume-pod
spec:
containers:
- image: gcr.io/google_containers/test-webserver
name: test-container
volumeMounts:
- mountPath: /mnt
name: pxvol
volumes:
- name: pxvol
# This Portworx volume must already exist.
portworxVolume:
volumeID: "pxvol"
fsType: "<fs-type>"
```
**Important:** Make sure you have an existing PortworxVolume with name `pxvol` before using it in the pod.
{: .caution}
More details and examples can be found [here](https://github.com/kubernetes/examples/tree/{{page.githubbranch}}/staging/volumes/portworx/README.md).
### ScaleIO
ScaleIO is a software-based storage platform that can use existing hardware to create clusters of scalable
shared block networked storage. The ScaleIO volume plugin allows deployed pods to access existing ScaleIO
volumes (or it can dynamically provision new volumes for persistent volume claims, see
[ScaleIO Persistent Volumes](/docs/concepts/storage/persistent-volumes/#scaleio)).
**Important:** You must have an existing ScaleIO cluster already setup and running with the volumes created before you can use them.
{: .caution}
The following is an example pod configuration with ScaleIO:
```yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-0
spec:
containers:
- image: gcr.io/google_containers/test-webserver
name: pod-0
volumeMounts:
- mountPath: /test-pd
name: vol-0
volumes:
- name: vol-0
scaleIO:
gateway: https://localhost:443/api
system: scaleio
protectionDomain: sd0
storagePool: sp1
volumeName: vol-0
secretRef:
name: sio-secret
fsType: xfs
```
For further detail, please the see the [ScaleIO examples](https://github.com/kubernetes/examples/tree/{{page.githubbranch}}/staging/volumes/scaleio).
### StorageOS
A `storageos` volume allows an existing [StorageOS](https://www.storageos.com) volume to be mounted into your pod.
StorageOS runs as a container within your Kubernetes environment, making local or attached storage accessible from any node within the Kubernetes cluster. Data can be replicated to protect against node failure. Thin provisioning and compression can improve utilization and reduce cost.
At its core, StorageOS provides block storage to containers, accessible via a file system.
The StorageOS container requires 64-bit Linux and has no additional dependencies. A free developer licence is available.
**Important:** You must run the StorageOS container on each node that wants to access StorageOS volumes or that will contribute storage capacity to the pool. For installation instructions, consult the [StorageOS documentation](https://docs.storageos.com).
{: .caution}
```yaml
apiVersion: v1
kind: Pod
metadata:
labels:
name: redis
role: master
name: test-storageos-redis
spec:
containers:
- name: master
image: kubernetes/redis:v1
env:
- name: MASTER
value: "true"
ports:
- containerPort: 6379
volumeMounts:
- mountPath: /redis-master-data
name: redis-data
volumes:
- name: redis-data
storageos:
# The `redis-vol01` volume must already exist within StorageOS in the `default` namespace.
volumeName: redis-vol01
fsType: ext4
```
For more information including Dynamic Provisioning and Persistent Volume Claims, please see the [StorageOS examples](https://github.com/kubernetes/examples/tree/{{page.githubbranch}}/staging/volumes/storageos).
### local
This volume type is alpha in 1.7.
A `local` volume represents a mounted local storage device such as a disk,
partition or directory.
Local volumes can only be used as a statically created PersistentVolume.
Compared to HostPath volumes, local volumes can be used in a durable manner
without manually scheduling pods to nodes, as the system is aware of the volume's
node constraints.
However, local volumes are still subject to the availability of the underlying
node and are not suitable for all applications.
The following is an example PersistentVolume spec using a `local` volume:
``` yaml
apiVersion: v1
kind: PersistentVolume
metadata:
name: example-pv
annotations:
"volume.alpha.kubernetes.io/node-affinity": '{
"requiredDuringSchedulingIgnoredDuringExecution": {
"nodeSelectorTerms": [
{ "matchExpressions": [
{ "key": "kubernetes.io/hostname",
"operator": "In",
"values": ["example-node"]
}
]}
]}
}'
spec:
capacity:
storage: 100Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Delete
storageClassName: local-storage
local:
path: /mnt/disks/ssd1
```
**Note:** The local PersistentVolume cleanup and deletion requires manual intervention without the external provisioner.
{: .note}
For details on the `local` volume type, see the [Local Persistent Storage
user guide](https://github.com/kubernetes-incubator/external-storage/tree/master/local-volume).
## Using subPath
Sometimes, it is useful to share one volume for multiple uses in a single pod. The `volumeMounts.subPath`
@@ -832,12 +853,15 @@ spec:
containers:
- name: mysql
image: mysql
env:
- name: MYSQL_ROOT_PASSWORD
value: "rootpasswd"
volumeMounts:
- mountPath: /var/lib/mysql
name: site-data
subPath: mysql
- name: php
image: php
image: php:7.0-apache
volumeMounts:
- mountPath: /var/www/html
name: site-data
docs/concepts/workloads/controllers/cron-jobs.md
+2 -3
View File
@@ -94,7 +94,7 @@ your job name and pod name would be different.
```shell
# Replace "hello-4111706356" with the job name in your system
$ pods=$(kubectl get pods --selector=job-name=hello-4111706356 --output=jsonpath={.items..metadata.name})
$ pods=$(kubectl get pods -a --selector=job-name=hello-4111706356 --output=jsonpath={.items..metadata.name})
$ echo $pods
hello-4111706356-o9qcm
@@ -129,8 +129,7 @@ of the set of pods. A cron job does not examine pods at all.
As with all other Kubernetes configs, a cron job needs `apiVersion`, `kind`, and `metadata` fields. For general
information about working with config files, see [deploying applications](/docs/user-guide/deploying-applications),
[configuring containers](/docs/user-guide/configuring-containers), and
[using kubectl to manage resources](/docs/user-guide/working-with-resources) documents.
and [using kubectl to manage resources](/docs/user-guide/working-with-resources) documents.
A cron job also needs a [`.spec` section](https://git.k8s.io/community/contributors/devel/api-conventions.md#spec-and-status).
docs/concepts/workloads/controllers/daemonset.md
+13 -16
View File
@@ -32,7 +32,7 @@ different flags and/or different memory and cpu requests for different hardware
### Create a DaemonSet
You can describe a DaemonSet in a YAML file. For example, the daemonset.yaml file below describes a DaemonSet that runs the fluentd-elasticsearch Docker image:
You can describe a DaemonSet in a YAML file. For example, the `daemonset.yaml` file below describes a DaemonSet that runs the fluentd-elasticsearch Docker image:
{% include code.html language="yaml" file="daemonset.yaml" ghlink="/docs/concepts/workloads/controllers/daemonset.yaml" %}
@@ -45,7 +45,7 @@ kubectl create -f daemonset.yaml
As with all other Kubernetes config, a DaemonSet needs `apiVersion`, `kind`, and `metadata` fields. For
general information about working with config files, see [deploying applications](/docs/user-guide/deploying-applications/),
[configuring containers](/docs/user-guide/configuring-containers/), and [working with resources](/docs/concepts/tools/kubectl/object-management-overview/) documents.
[configuring containers](/docs/tasks/), and [working with resources](/docs/concepts/tools/kubectl/object-management-overview/) documents.
A DaemonSet also needs a [`.spec`](https://git.k8s.io/community/contributors/devel/api-conventions.md#spec-and-status) section.
@@ -66,10 +66,10 @@ A Pod Template in a DaemonSet must have a [`RestartPolicy`](/docs/user-guide/pod
The `.spec.selector` field is a pod selector. It works the same as the `.spec.selector` of
a [Job](/docs/concepts/jobs/run-to-completion-finite-workloads/).
As of Kubernetes 1.8, you must specify a pod selector that matches the labels of the
`.spec.template`. The pod selector will no longer be defaulted when left empty. Selector
defaulting was not compatible with `kubectl apply`. Also, once a DaemonSet is created,
its `spec.selector` can not be mutated. Mutating the pod selector can lead to the
As of Kubernetes 1.8, you must specify a pod selector that matches the labels of the
`.spec.template`. The pod selector will no longer be defaulted when left empty. Selector
defaulting was not compatible with `kubectl apply`. Also, once a DaemonSet is created,
its `spec.selector` can not be mutated. Mutating the pod selector can lead to the
unintentional orphaning of Pods, and it was found to be confusing to users.
The `spec.selector` is an object consisting of two fields:
@@ -80,8 +80,7 @@ The `spec.selector` is an object consisting of two fields:
When the two are specified the result is ANDed.
If the `.spec.selector` is specified, it must match the `.spec.template.metadata.labels`. If not
specified, they are defaulted to be equal. Config with these not matching will be rejected by the API.
If the `.spec.selector` is specified, it must match the `.spec.template.metadata.labels`. Config with these not matching will be rejected by the API.
Also you should not normally create any Pods whose labels match this selector, either directly, via
another DaemonSet, or via other controller such as ReplicaSet. Otherwise, the DaemonSet
@@ -89,7 +88,7 @@ controller will think that those Pods were created by it. Kubernetes will not s
this. One case where you might want to do this is manually create a Pod with a different value on
a node for testing.
If you attempt to create a DaemonSet such that
If you attempt to create a DaemonSet such that
### Running Pods on Only Some Nodes
@@ -110,10 +109,10 @@ when the Pod is created, so it is ignored by the scheduler). Therefore:
- The DaemonSet controller can make Pods even when the scheduler has not been started, which can help cluster
bootstrap.
Daemon Pods do respect [taints and tolerations](/docs/concepts/configuration/assign-pod-node/#taints-and-tolerations-beta-feature),
Daemon Pods do respect [taints and tolerations](/docs/concepts/configuration/taint-and-toleration),
but they are created with `NoExecute` tolerations for the following taints with no `tolerationSeconds`:
- `node.alpha.kubernetes.io/notReady`
- `node.kubernetes.io/not-ready`
- `node.alpha.kubernetes.io/unreachable`
This ensures that when the `TaintBasedEvictions` alpha feature is enabled,
@@ -122,7 +121,7 @@ they will not be evicted when there are node problems such as a network partitio
due to hard-coded behavior of the NodeController rather than due to tolerations).
They also tolerate following `NoSchedule` taints:
- `node.kubernetes.io/memory-pressure`
- `node.kubernetes.io/disk-pressure`
@@ -132,6 +131,8 @@ labelled as critical, the Daemon pods are created with an additional
Note that all above `NoSchedule` taints above are created only in version 1.8 or later if the alpha feature `TaintNodesByCondition` is enabled.
Also note that the `node-role.kubernetes.io/master` `NoSchedule` toleration specified in the above example is needed on 1.6 or later to schedule on *master* nodes as this is not a default toleration.
## Communicating with Daemon Pods
Some possible patterns for communicating with Pods in a DaemonSet are:
@@ -163,8 +164,6 @@ You will need to force new Pod creation by deleting the Pod or deleting the node
In Kubernetes version 1.6 and later, you can [perform a rolling update](/docs/tasks/manage-daemon/update-daemon-set/) on a DaemonSet.
Future releases of Kubernetes will support controlled updating of nodes.
## Alternatives to DaemonSet
### Init Scripts
@@ -175,8 +174,6 @@ running such processes via a DaemonSet:
- Ability to monitor and manage logs for daemons in the same way as applications.
- Same config language and tools (e.g. Pod templates, `kubectl`) for daemons and applications.
- Future versions of Kubernetes will likely support integration between DaemonSet-created
Pods and node upgrade workflows.
- Running daemons in containers with resource limits increases isolation between daemons from app
containers. However, this can also be accomplished by running the daemons in a container but not in a Pod
(e.g. start directly via Docker).
docs/concepts/workloads/controllers/daemonset.yaml
+4 -1
View File
@@ -1,4 +1,4 @@
apiVersion: apps/v1beta2
apiVersion: apps/v1beta2 # for versions before 1.8.0 use extensions/v1beta1
kind: DaemonSet
metadata:
name: fluentd-elasticsearch
@@ -14,6 +14,9 @@ spec:
labels:
name: fluentd-elasticsearch
spec:
tolerations:
- key: node-role.kubernetes.io/master
effect: NoSchedule
containers:
- name: fluentd-elasticsearch
image: gcr.io/google-containers/fluentd-elasticsearch:1.20
docs/concepts/workloads/controllers/deployment.md
+12 -4
View File
@@ -43,11 +43,19 @@ In this example:
* A Deployment named `nginx-deployment` is created, indicated by the `metadata: name` field.
* The Deployment creates three replicated Pods, indicated by the `replicas` field.
* The `selector` field defines how the Deployment finds which Pods to manage.
In this case, we simply select on one label defined in the Pod template (`app: nginx`).
However, more sophisticated selection rules are possible,
as long as the Pod template itself satisfies the rule.
* The Pod template's specification, or `template: spec` field, indicates that
the Pods run one container, `nginx`, which runs the `nginx`
[Docker Hub](https://hub.docker.com/) image at version 1.7.9.
* The Deployment opens port 80 for use by the Pods.
Note: `matchLabels` is a map of {key,value} pairs. A single {key,value} in the matchLabels map
is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In",
and the values array contains only "value". The requirements are ANDed.
The `template` field contains the following instructions:
* The Pods are labeled `app: nginx`
@@ -58,7 +66,7 @@ The `template` field contains the following instructions:
To create this Deployment, run the following command:
```shell
kubectl create -f https://raw.githubusercontent.com/kubernetes/kubernetes.github.io/master/docs/concepts/workloads/controllers/nginx-deployment.yaml
kubectl create -f https://raw.githubusercontent.com/kubernetes/website/master/docs/concepts/workloads/controllers/nginx-deployment.yaml
```
Note: You can append `--record` to this command to record the current command in the annotations of
@@ -88,7 +96,7 @@ Notice how the values in each field correspond to the values in the Deployment s
* The number of desired replicas is 3 according to `spec: replicas` field.
* The number of current replicas is 0 according to the `.status.replicas` field.
* The number of up-to-date replicas is 0 accoridng to the `.status.updatedReplicas` field.
* The number of up-to-date replicas is 0 according to the `.status.updatedReplicas` field.
* The number of available replicas is 0 according to the `.status.availableReplicas` field.
To see the Deployment rollout status, run `kubectl rollout status deployment/nginx-deployment`. This command returns the following output:
@@ -691,7 +699,7 @@ lack of progress for a Deployment after 10 minutes:
```shell
$ kubectl patch deployment/nginx-deployment -p '{"spec":{"progressDeadlineSeconds":600}}'
"nginx-deployment" patched
deployment "nginx-deployment" patched
```
Once the deadline has been exceeded, the Deployment controller adds a DeploymentCondition with the following
attributes to the Deployment's `status.conditions`:
@@ -728,7 +736,7 @@ Conditions:
<...>
```
If you run `kubectl get deployment nginx-deployment -o yaml`, the Deployement status might look like this:
If you run `kubectl get deployment nginx-deployment -o yaml`, the Deployment status might look like this:
```
status:
docs/concepts/workloads/controllers/frontend.yaml
+1 -1
View File
@@ -1,4 +1,4 @@
apiVersion: apps/v1beta2 # for versions before 1.6.0 use extensions/v1beta1
apiVersion: apps/v1beta2 # for versions before 1.8.0 use apps/v1beta1
kind: ReplicaSet
metadata:
name: frontend
docs/concepts/workloads/controllers/garbage-collection.md
+11 -10
View File
@@ -24,9 +24,10 @@ points to the owning object.
Sometimes, Kubernetes sets the value of `ownerReference` automatically. For
example, when you create a ReplicaSet, Kubernetes automatically sets the
`ownerReference` field of each Pod in the ReplicaSet. In 1.6, Kubernetes
`ownerReference` field of each Pod in the ReplicaSet. In 1.8, Kubernetes
automatically sets the value of `ownerReference` for objects created or adopted
by ReplicationController, ReplicaSet, StatefulSet, DaemonSet, and Deployment.
by ReplicationController, ReplicaSet, StatefulSet, DaemonSet, Deployment, Job
and CronJob.
You can also specify relationships between owners and dependents by manually
setting the `ownerReference` field.
@@ -39,7 +40,7 @@ If you create the ReplicaSet and then view the Pod metadata, you can see
OwnerReferences field:
```shell
kubectl create -f https://k8s.io/docs/concepts/abstractions/controllers/my-repset.yaml
kubectl create -f https://k8s.io/docs/concepts/controllers/my-repset.yaml
kubectl get pods --output=yaml
```
@@ -69,12 +70,6 @@ deletion*. There are two modes of *cascading deletion*: *background* and *foreg
If you delete an object without deleting its dependents
automatically, the dependents are said to be *orphaned*.
### Background cascading deletion
In *background cascading deletion*, Kubernetes deletes the owner object
immediately and the garbage collector then deletes the dependents in
the background.
### Foreground cascading deletion
In *foreground cascading deletion*, the root object first
@@ -92,13 +87,19 @@ the owner object.
Note that in the "foregroundDeletion", only dependents with
`ownerReference.blockOwnerDeletion` block the deletion of the owner object.
Kubernetes version 1.7 will add an admission controller that controls user access to set
Kubernetes version 1.7 added an [admission controller](/docs/admin/admission-controllers/#ownerreferencespermissionenforcement) that controls user access to set
`blockOwnerDeletion` to true based on delete permissions on the owner object, so that
unauthorized dependents cannot delay deletion of an owner object.
If an object's `ownerReferences` field is set by a controller (such as Deployment or ReplicaSet),
blockOwnerDeletion is set automatically and you do not need to manually modify this field.
### Background cascading deletion
In *background cascading deletion*, Kubernetes deletes the owner object
immediately and the garbage collector then deletes the dependents in
the background.
### Setting the cascading deletion policy
To control the cascading deletion policy, set the `deleteOptions.propagationPolicy`
docs/concepts/workloads/controllers/job.yaml
+1 -2
View File
@@ -12,5 +12,4 @@ spec:
image: perl
command: ["perl", "-Mbignum=bpi", "-wle", "print bpi(2000)"]
restartPolicy: Never
backoffLimit: 4
backoffLimit: 4
docs/concepts/workloads/controllers/jobs-run-to-completion.md
+20 -8
View File
@@ -76,7 +76,7 @@ To list all the pods that belong to a job in a machine readable form, you can us
```shell
$ pods=$(kubectl get pods --show-all --selector=job-name=pi --output=jsonpath={.items..metadata.name})
echo $pods
$ echo $pods
pi-aiw0a
```
@@ -125,7 +125,7 @@ There are three main types of jobs:
- the job is complete when there is one successful pod for each value in the range 1 to `.spec.completions`.
- **not implemented yet:** each pod passed a different index in the range 1 to `.spec.completions`.
1. Parallel Jobs with a *work queue*:
 - do not specify `.spec.completions`, default to `.spec.Parallelism`.
 - do not specify `.spec.completions`, default to `.spec.parallelism`.
 - the pods must coordinate with themselves or an external service to determine what each should work on.
- each pod is independently capable of determining whether or not all its peers are done, thus the entire Job is done.
- when _any_ pod terminates with success, no new pods are created.
@@ -155,14 +155,14 @@ A job can be scaled up using the `kubectl scale` command. For example, the foll
command sets `.spec.parallelism` of a job called `myjob` to 10:
```shell
$ kubectl scale --replicas=$N jobs/myjob
$ kubectl scale --replicas=10 jobs/myjob
job "myjob" scaled
```
You can also use the `scale` subresource of the Job resource.
Actual parallelism (number of pods running at any instant) may be more or less than requested
parallelism, for a variety or reasons:
parallelism, for a variety of reasons:
- For Fixed Completion Count jobs, the actual number of pods running in parallel will not exceed the number of
remaining completions. Higher values of `.spec.parallelism` are effectively ignored.
@@ -180,7 +180,7 @@ a non-zero exit code, or the Container was killed for exceeding a memory limit,
happens, and the `.spec.template.spec.restartPolicy = "OnFailure"`, then the Pod stays
on the node, but the Container is re-run. Therefore, your program needs to handle the case when it is
restarted locally, or else specify `.spec.template.spec.restartPolicy = "Never"`.
See [pods-states](/docs/user-guide/pod-states) for more information on `restartPolicy`.
See [pods-states](/docs/concepts/workloads/pods/pod-lifecycle/#example-states) for more information on `restartPolicy`.
An entire Pod can also fail, for a number of reasons, such as when the pod is kicked off the node
(node is upgraded, rebooted, deleted, etc.), or if a container of the Pod fails and the
@@ -198,9 +198,20 @@ multiple pods running at once. Therefore, your pods must also be tolerant of co
### Pod Backoff failure policy
There are situations where you want to fail a Job after some amount of retries due to a logical error in configuration etc.
To do so set `.spec.template.spec.backoffLimit` to specify the number of retries before considering a Job as failed.
The back-off limit is set by default to 6. Failed Pods associated with the Job are recreated by the Job controller with an exponential back-off delay (10s, 20s, 40s ...) capped at six minutes, The back-off limit is reset if no new failed Pods appear before the Job's next status check.
There are situations where you want to fail a Job after some amount of retries
due to a logical error in configuration etc.
To do so, set `.spec.backoffLimit` to specify the number of retries before
considering a Job as failed. The back-off limit is set by default to 6. Failed
Pods associated with the Job are recreated by the Job controller with an
exponential back-off delay (10s, 20s, 40s ...) capped at six minutes, The
back-off limit is reset if no new failed Pods appear before the Job's next
status check.
**Note:** Due to a known issue [#54870](https://github.com/kubernetes/kubernetes/issues/54870),
when the `spec.template.spec.restartPolicy` field is set to "`OnFailure`", the
back-off limit may be ineffective. As a short-term workaround, set the restart
policy for the embedded template to "`Never`".
{: .note}
## Job Termination and Cleanup
@@ -226,6 +237,7 @@ kind: Job
metadata:
name: pi-with-timeout
spec:
backoffLimit: 5
activeDeadlineSeconds: 100
template:
metadata:
docs/concepts/workloads/controllers/nginx-deployment.yaml
+1 -1
View File
@@ -1,4 +1,4 @@
apiVersion: apps/v1beta2 # for versions before 1.7.0 use apps/v1beta1
apiVersion: apps/v1beta2 # for versions before 1.8.0 use apps/v1beta1
kind: Deployment
metadata:
name: nginx-deployment
docs/concepts/workloads/controllers/replicaset.md
+7 -8
View File
@@ -96,15 +96,14 @@ frontend-qhloh 1/1 Running 0 1m
## Writing a ReplicaSet Spec
As with all other Kubernetes API objects, a ReplicaSet needs the `apiVersion`, `kind`, and `metadata` fields. For
general information about working with manifests, see [here](/docs/user-guide/simple-yaml/),
[here](/docs/user-guide/configuring-containers/), and [here](/docs/concepts/tools/kubectl/object-management-overview/).
general information about working with manifests, see [Object Management](/docs/concepts/tools/kubectl/object-management-overview/).
A ReplicaSet also needs a [`.spec` section](https://git.k8s.io/community/contributors/devel/api-conventions.md#spec-and-status).
### Pod Template
The `.spec.template` is the only required field of the `.spec`. The `.spec.template` is a
[pod template](/docs/concepts/workloads/pods/pod-overview/#pod-templates). It has exactly the same schema as a
The `.spec.template` is the only required field of the `.spec`. The `.spec.template` is a
[pod template](/docs/concepts/workloads/pods/pod-overview/#pod-templates). It has exactly the same schema as a
[pod](/docs/concepts/workloads/pods/pod/), except that it is nested and does not have an `apiVersion` or `kind`.
In addition to required fields of a pod, a pod template in a ReplicaSet must specify appropriate
@@ -119,7 +118,7 @@ for example the [Kubelet](/docs/admin/kubelet/) or Docker.
### Pod Selector
The `.spec.selector` field is a [label selector](/docs/user-guide/labels/#label-selectors). A ReplicaSet
The `.spec.selector` field is a [label selector](/docs/concepts/overview/working-with-objects/labels/). A ReplicaSet
manages all the pods with labels that match the selector. It does not distinguish
between pods that it created or deleted and pods that another person or process created or
deleted. This allows the ReplicaSet to be replaced without affecting the running pods.
@@ -129,8 +128,8 @@ be rejected by the API.
In Kubernetes 1.8 the API version `apps/v1beta2` on the ReplicaSet kind is the current version and is enabled by default. The API version `extensions/v1beta1` is deprecated. In API version `apps/v1beta2`, `.spec.selector` and `.metadata.labels` no longer default to `.spec.template.metadata.labels` if not set. So they must be set explicitly. Also note that `.spec.selector` is immutable after creation starting in API version `apps/v1beta2`.
Also you should not normally create any pods whose labels match this selector, either directly, with
another ReplicaSet, or with another controller such as a Deployment. If you do so, the ReplicaSet thinks that it
Also you should not normally create any pods whose labels match this selector, either directly, with
another ReplicaSet, or with another controller such as a Deployment. If you do so, the ReplicaSet thinks that it
created the other pods. Kubernetes does not stop you from doing this.
If you do end up with multiple controllers that have overlapping selectors, you
@@ -175,7 +174,7 @@ To update pods to a new spec in a controlled way, use a [rolling update](#rollin
### Isolating pods from a ReplicaSet
Pods may be removed from a ReplicaSet's target set by changing their labels. This technique may be used to remove pods
Pods may be removed from a ReplicaSet's target set by changing their labels. This technique may be used to remove pods
from service for debugging, data recovery, etc. Pods that are removed in this way will be replaced automatically (
assuming that the number of replicas is not also changed).
docs/concepts/workloads/controllers/replicationcontroller.md
+3 -4
View File
@@ -99,9 +99,8 @@ specifies an expression that just gets the name from each pod in the returned li
## Writing a ReplicationController Spec
As with all other Kubernetes config, a ReplicationController needs `apiVersion`, `kind`, and `metadata` fields. For
general information about working with config files, see [here](/docs/user-guide/simple-yaml/),
[here](/docs/user-guide/configuring-containers/), and [here](/docs/concepts/tools/kubectl/object-management-overview/).
As with all other Kubernetes config, a ReplicationController needs `apiVersion`, `kind`, and `metadata` fields.
For general information about working with config files, see [Object Management](/docs/concepts/tools/kubectl/object-management-overview/).
A ReplicationController also needs a [`.spec` section](https://git.k8s.io/community/contributors/devel/api-conventions.md#spec-and-status).
@@ -230,7 +229,7 @@ The ReplicationController simply ensures that the desired number of pods matches
The ReplicationController is forever constrained to this narrow responsibility. It itself will not perform readiness nor liveness probes. Rather than performing auto-scaling, it is intended to be controlled by an external auto-scaler (as discussed in [#492](http://issue.k8s.io/492)), which would change its `replicas` field. We will not add scheduling policies (for example, [spreading](http://issue.k8s.io/367#issuecomment-48428019)) to the ReplicationController. Nor should it verify that the pods controlled match the currently specified template, as that would obstruct auto-sizing and other automated processes. Similarly, completion deadlines, ordering dependencies, configuration expansion, and other features belong elsewhere. We even plan to factor out the mechanism for bulk pod creation ([#170](http://issue.k8s.io/170)).
The ReplicationController is intended to be a composable building-block primitive. We expect higher-level APIs and/or tools to be built on top of it and other complementary primitives for user convenience in the future. The "macro" operations currently supported by kubectl (run, stop, scale, rolling-update) are proof-of-concept examples of this. For instance, we could imagine something like [Asgard](http://techblog.netflix.com/2012/06/asgard-web-based-cloud-management-and.html) managing ReplicationControllers, auto-scalers, services, scheduling policies, canaries, etc.
The ReplicationController is intended to be a composable building-block primitive. We expect higher-level APIs and/or tools to be built on top of it and other complementary primitives for user convenience in the future. The "macro" operations currently supported by kubectl (run, scale, rolling-update) are proof-of-concept examples of this. For instance, we could imagine something like [Asgard](http://techblog.netflix.com/2012/06/asgard-web-based-cloud-management-and.html) managing ReplicationControllers, auto-scalers, services, scheduling policies, canaries, etc.
## API Object
docs/concepts/workloads/controllers/statefulset.md
+7 -7
View File
@@ -10,10 +10,10 @@ title: StatefulSets
---
{% capture overview %}
**StatefulSet is the workload API object used to manage stateful applications.
**StatefulSet is the workload API object used to manage stateful applications.
StatefulSets are beta in 1.8.**
{% include templates/glossary/snippet.md term="statefulset" length="long" %}
{% glossary_definition term_id="statefulset" length="all" %}
{% endcapture %}
{% capture body %}
@@ -49,7 +49,7 @@ The example below demonstrates the components of a StatefulSet.
* A Headless Service, named nginx, is used to control the network domain.
* The StatefulSet, named web, has a Spec that indicates that 3 replicas of the nginx container will be launched in unique Pods.
* The volumeClaimTemplates will provide stable storage using [PersistentVolumes](/docs/concepts/storage/volumes/) provisioned by a PersistentVolume Provisioner.
* The volumeClaimTemplates will provide stable storage using [PersistentVolumes](/docs/concepts/storage/persistent-volumes/) provisioned by a PersistentVolume Provisioner.
```yaml
apiVersion: v1
@@ -123,8 +123,8 @@ is `$(statefulset name)-$(ordinal)`. The example above will create three Pods
named `web-0,web-1,web-2`.
A StatefulSet can use a [Headless Service](/docs/concepts/services-networking/service/#headless-services)
to control the domain of its Pods. The domain managed by this Service takes the form:
`$(service name).$(namespace).svc.cluster.local`, where "cluster.local"
is the [cluster domain](http://releases.k8s.io/{{page.githubbranch}}/cluster/addons/dns/README.md).
`$(service name).$(namespace).svc.cluster.local`, where "cluster.local" is the
cluster domain.
As each Pod is created, it gets a matching DNS subdomain, taking the form:
`$(podname).$(governing service domain)`, where the governing service is defined
by the `serviceName` field on the StatefulSet.
@@ -139,11 +139,11 @@ Cluster Domain | Service (ns/name) | StatefulSet (ns/name) | StatefulSet Domain
kube.local | foo/nginx | foo/web | nginx.foo.svc.kube.local | web-{0..N-1}.nginx.foo.svc.kube.local | web-{0..N-1} |
Note that Cluster Domain will be set to `cluster.local` unless
[otherwise configured](http://releases.k8s.io/{{page.githubbranch}}/cluster/addons/dns/README.md).
[otherwise configured](/docs/concepts/services-networking/dns-pod-service/#how-it-works).
### Stable Storage
Kubernetes creates one [PersistentVolume](/docs/concepts/storage/volumes/) for each
Kubernetes creates one [PersistentVolume](/docs/concepts/storage/persistent-volumes/) for each
VolumeClaimTemplate. In the nginx example above, each Pod will receive a single PersistentVolume
with a StorageClass of `my-storage-class` and 1 Gib of provisioned storage. If no StorageClass
is specified, then the default StorageClass will be used. When a Pod is (re)scheduled
docs/concepts/workloads/pods/disruptions.md
+3 -3
View File
@@ -78,10 +78,10 @@ or across zones (if using a
The frequency of voluntary disruptions varies. On a basic Kubernetes cluster, there are
no voluntary disruptions at all. However, your cluster administrator or hosting provider
may run some additional services which cause voluntary disruptions. For example,
rolling out node software updates can cause voluntary updates. Also, some implementations
may run some additional services which cause voluntary disruptions. For example,
rolling out node software updates can cause voluntary disruptions. Also, some implementations
of cluster (node) autoscaling may cause voluntary disruptions to defragment and compact nodes.
You cluster administrator or hosting provider should have documented what level of voluntary
Your cluster administrator or hosting provider should have documented what level of voluntary
disruptions, if any, to expect.
Kubernetes offers features to help run highly available applications at the same
docs/concepts/workloads/pods/init-containers.md
+5 -5
View File
@@ -20,7 +20,7 @@ In 1.8, the annotations are no longer supported and must be converted to the Pod
{% capture body %}
## Understanding Init Containers
A [Pod](/docs/concepts/abstractions/pod/) can have multiple Containers running
A [Pod](/docs/concepts/workloads/pods/pod-overview/) can have multiple Containers running
apps within it, but it can also have one or more Init Containers, which are run
before the app Containers are started.
@@ -75,11 +75,11 @@ Here are some ideas for how to use Init Containers:
* Wait for a service to be created with a shell command like:
for i in {1..100}; do sleep 1; if dig myservice; then exit 0; fi; exit 1
for i in {1..100}; do sleep 1; if dig myservice; then exit 0; fi; done; exit 1
* Register this Pod with a remote server from the downward API with a command like:
curl -X POST http://$MANAGEMENT_SERVICE_HOST:$MANAGEMENT_SERVICE_PORT/register -d 'instance=$(<POD_NAME>)&ip=$(<POD_IP>)'
curl -X POST http://$MANAGEMENT_SERVICE_HOST:$MANAGEMENT_SERVICE_PORT/register -d 'instance=$(<POD_NAME>)&ip=$(<POD_IP>)'
* Wait for some time before starting the app Container with a command like `sleep 60`.
* Clone a git repository into a volume.
@@ -88,8 +88,8 @@ Here are some ideas for how to use Init Containers:
place the POD_IP value in a configuration and generate the main app
configuration file using Jinja.
More detailed usage examples can be found in the [StatefulSets documentation](/docs/concepts/abstractions/controllers/statefulsets/)
and the [Production Pods guide](/docs/tasks/#handling-initialization).
More detailed usage examples can be found in the [StatefulSets documentation](/docs/concepts/workloads/controllers/statefulset/)
and the [Production Pods guide](/docs/tasks/configure-pod-container/configure-pod-initialization/).
### Init Containers in use
docs/concepts/workloads/pods/pod-lifecycle.md
+2 -2
View File
@@ -208,8 +208,8 @@ spec:
path: /healthz
port: 8080
httpHeaders:
- name: X-Custom-Header
value: Awesome
- name: X-Custom-Header
value: Awesome
initialDelaySeconds: 15
timeoutSeconds: 1
name: liveness
docs/concepts/workloads/pods/pod-overview.md
+18 -2
View File
@@ -51,7 +51,7 @@ A Pod can specify a set of shared storage *volumes*. All containers in the Pod c
## Working with Pods
You'll rarely create individual Pods directly in Kubernetes--even singleton Pods. This is because Pods are designed as relatively ephemeral, disposable entities. When a Pod gets created (directly by you, or indirectly by a Controller), it is scheduled to run on a Node in your cluster. The Pod remains on that Node until the process is terminated, the pod object is deleted, or the pod is *evicted* for lack of resources, or the Node fails.
You'll rarely create individual Pods directly in Kubernetes--even singleton Pods. This is because Pods are designed as relatively ephemeral, disposable entities. When a Pod gets created (directly by you, or indirectly by a Controller), it is scheduled to run on a Node in your cluster. The Pod remains on that Node until the process is terminated, the pod object is deleted, the pod is *evicted* for lack of resources, or the Node fails.
> Note: Restarting a container in a Pod should not be confused with restarting the Pod. The Pod itself does not run, but is an environment the containers run in and persists until it is deleted.
@@ -64,7 +64,7 @@ A Controller can create and manage multiple Pods for you, handling replication a
Some examples of Controllers that contain one or more pods include:
* [Deployment](/docs/concepts/workloads/controllers/deployment/)
* [StatefulSet](/docs/concepts/abstractions/controllers/statefulsets/)
* [StatefulSet](/docs/concepts/workloads/controllers/statefulset/)
* [DaemonSet](/docs/concepts/workloads/controllers/daemonset/)
In general, Controllers use a Pod Template that you provide to create the Pods for which it is responsible.
@@ -74,6 +74,22 @@ In general, Controllers use a Pod Template that you provide to create the Pods f
Pod templates are pod specifications which are included in other objects, such as
[Replication Controllers](/docs/concepts/workloads/controllers/replicationcontroller/), [Jobs](/docs/concepts/jobs/run-to-completion-finite-workloads/), and
[DaemonSets](/docs/concepts/workloads/controllers/daemonset/). Controllers use Pod Templates to make actual pods.
The sample below is a simple manifest for a Pod which contains a container that prints
a message.
```yaml
apiVersion: v1
kind: Pod
metadata:
name: myapp-pod
labels:
app: myapp
spec:
containers:
- name: myapp-container
image: busybox
command: ['sh', '-c', 'echo Hello Kubernetes! && sleep 3600']
```
Rather than specifying the current desired state of all replicas, pod templates are like cookie cutters. Once a cookie has been cut, the cookie has no relationship to the cutter. There is no quantum entanglement. Subsequent changes to the template or even switching to a new template has no direct effect on the pods already created. Similarly, pods created by a replication controller may subsequently be updated directly. This is in deliberate contrast to pods, which do specify the current desired state of all containers belonging to the pod. This approach radically simplifies system semantics and increases the flexibility of the primitive.
docs/concepts/workloads/pods/podpreset.md
+84
View File
@@ -0,0 +1,84 @@
---
approvers:
- jessfraz
title: Pod Preset
---
{% capture overview %}
This page provides an overview of PodPresets, which are objects for injecting
certain information into pods at creation time. The information can include
secrets, volumes, volume mounts, and environment variables.
{% endcapture %}
{:toc}
{% capture body %}
## Understanding Pod Presets
A `Pod Preset` is an API resource for injecting additional runtime requirements
into a Pod at creation time.
You use [label selectors](/docs/concepts/overview/working-with-objects/labels/#label-selectors)
to specify the Pods to which a given Pod Preset applies.
Using a Pod Preset allows pod template authors to not have to explicitly provide
all information for every pod. This way, authors of pod templates consuming a
specific service do not need to know all the details about that service.
For more information about the background, see the [design proposal for PodPreset](https://git.k8s.io/community/contributors/design-proposals/service-catalog/pod-preset.md).
## How It Works
Kubernetes provides an admission controller (`PodPreset`) which, when enabled,
applies Pod Presets to incoming pod creation requests.
When a pod creation request occurs, the system does the following:
1. Retrieve all `PodPresets` available for use.
1. Check if the label selectors of any `PodPreset` matches the labels on the
pod being created.
1. Attempt to merge the various resources defined by the `PodPreset` into the
Pod being created.
1. On error, throw an event documenting the merge error on the pod, and create
the pod _without_ any injected resources from the `PodPreset`.
1. Annotate the resulting modified Pod spec to indicate that it has been
modified by a `PodPreset`. The annotation is of the form
`podpreset.admission.kubernetes.io/podpreset-<pod-preset name>": "<resource version>"`.
Each Pod can be matched zero or more Pod Presets; and each `PodPreset` can be
applied to zero or more pods. When a `PodPreset` is applied to one or more
Pods, Kubernetes modifies the Pod Spec. For changes to `Env`, `EnvFrom`, and
`VolumeMounts`, Kubernetes modifies the container spec for all containers in
the Pod; for changes to `Volume`, Kubernetes modifies the Pod Spec.
**Note:** A Pod Preset is capable of modifying the `spec.containers` field in a
Pod spec when appropriate. *No* resource definition from the Pod Preset will be
applied to the `initContainers` field.
{: .note}
### Disable Pod Preset for a Specific Pod
There may be instances where you wish for a Pod to not be altered by any Pod
Preset mutations. In these cases, you can add an annotation in the Pod Spec
of the form: `podpreset.admission.kubernetes.io/exclude: "true"`.
## Enable Pod Preset
In order to use Pod Presets in your cluster you must ensure the following:
1. You have enabled the API type `settings.k8s.io/v1alpha1/podpreset`. For
example, this can be done by including `settings.k8s.io/v1alpha1=true` in
the `--runtime-config` option for the API server.
1. You have enabled the admission controller `PodPreset`. One way to doing this
is to include `PodPreset` in the `--admission-control` option value specified
for the API server.
1. You have defined your Pod Presets by creating `PodPreset` objects in the
namespace you will use.
{% endcapture %}
{% capture whatsnext %}
* [Injecting data into a Pod using PodPreset](/docs/tasks/inject-data-application/podpreset/)
{% endcapture %}
{% include templates/concept.md %}