Revert v1.17 release changes on v1.16 branch (#18123)
This reverts commits:1131f8603e.cf5b8b200c.75de70a431.dff2f7b960.cf925bb688.ffd1363c0e.87a2cafb20.ffe259d6f2.
This commit is contained in:
Tim Bannister
committed by
Kubernetes Prow Robot
Kubernetes Prow Robot
parent
5733771d62
commit
065ccd94be
@@ -81,10 +81,19 @@ the `Terminating` or `Unknown` state. In cases where Kubernetes cannot deduce fr
|
||||
permanently left a cluster, the cluster administrator may need to delete the node object by hand. Deleting the node object from
|
||||
Kubernetes causes all the Pod objects running on the node to be deleted from the apiserver, and frees up their names.
|
||||
|
||||
The node lifecycle controller automatically creates
|
||||
In version 1.12, `TaintNodesByCondition` feature is promoted to beta, so node lifecycle controller automatically creates
|
||||
[taints](/docs/concepts/configuration/taint-and-toleration/) that represent conditions.
|
||||
When the scheduler is assigning a Pod to a Node, the scheduler takes the Node's taints
|
||||
into account, except for any taints that the Pod tolerates.
|
||||
Similarly the scheduler ignores conditions when considering a Node; instead
|
||||
it looks at the Node's taints and a Pod's tolerations.
|
||||
|
||||
Now users can choose between the old scheduling model and a new, more flexible scheduling model.
|
||||
A Pod that does not have any tolerations gets scheduled according to the old model. But a Pod that
|
||||
tolerates the taints of a particular Node can be scheduled on that Node.
|
||||
|
||||
{{< caution >}}
|
||||
Enabling this feature creates a small delay between the
|
||||
time when a condition is observed and when a taint is created. This delay is usually less than one second, but it can increase the number of Pods that are successfully scheduled but rejected by the kubelet.
|
||||
{{< /caution >}}
|
||||
|
||||
### Capacity and Allocatable {#capacity}
|
||||
|
||||
@@ -163,28 +172,18 @@ to be unreachable. (The default timeouts are 40s to start reporting
|
||||
ConditionUnknown and 5m after that to start evicting pods.) The node controller
|
||||
checks the state of each node every `--node-monitor-period` seconds.
|
||||
|
||||
#### Heartbeats
|
||||
|
||||
Heartbeats, sent by Kubernetes nodes, help determine the availability of a node.
|
||||
There are two forms of heartbeats: updates of `NodeStatus` and the
|
||||
[Lease object](/docs/reference/generated/kubernetes-api/{{< latest-version >}}/#lease-v1-coordination-k8s-io).
|
||||
Each Node has an associated Lease object in the `kube-node-lease`
|
||||
{{< glossary_tooltip term_id="namespace" text="namespace">}}.
|
||||
Lease is a lightweight resource, which improves the performance
|
||||
of the node heartbeats as the cluster scales.
|
||||
|
||||
The kubelet is responsible for creating and updating the `NodeStatus` and
|
||||
a Lease object.
|
||||
|
||||
- The kubelet updates the `NodeStatus` either when there is change in status,
|
||||
or if there has been no update for a configured interval. The default interval
|
||||
for `NodeStatus` updates is 5 minutes (much longer than the 40 second default
|
||||
timeout for unreachable nodes).
|
||||
- The kubelet creates and then updates its Lease object every 10 seconds
|
||||
(the default update interval). Lease updates occur independently from the
|
||||
`NodeStatus` updates.
|
||||
|
||||
#### Reliability
|
||||
In versions of Kubernetes prior to 1.13, NodeStatus is the heartbeat from the
|
||||
node. Node lease feature is enabled by default since 1.14 as a beta feature
|
||||
(feature gate `NodeLease`, [KEP-0009](https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/0009-node-heartbeat.md)).
|
||||
When node lease feature is enabled, each node has an associated `Lease` object in
|
||||
`kube-node-lease` namespace that is renewed by the node periodically, and both
|
||||
NodeStatus and node lease are treated as heartbeats from the node. Node leases
|
||||
are renewed frequently while NodeStatus is reported from node to master only
|
||||
when there is some change or enough time has passed (default is 1 minute, which
|
||||
is longer than the default timeout of 40 seconds for unreachable nodes). Since
|
||||
node lease is much more lightweight than NodeStatus, this feature makes node
|
||||
heartbeat significantly cheaper from both scalability and performance
|
||||
perspectives.
|
||||
|
||||
In Kubernetes 1.4, we updated the logic of the node controller to better handle
|
||||
cases when a large number of nodes have problems with reaching the master
|
||||
|
||||
@@ -76,12 +76,9 @@ In addition to labels you [attach](#step-one-attach-label-to-the-node), nodes co
|
||||
with a standard set of labels. These labels are
|
||||
|
||||
* [`kubernetes.io/hostname`](/docs/reference/kubernetes-api/labels-annotations-taints/#kubernetes-io-hostname)
|
||||
* [`failure-domain.beta.kubernetes.io/zone`](/docs/reference/kubernetes-api/labels-annotations-taints/#failure-domainbetakubernetesiozone)
|
||||
* [`failure-domain.beta.kubernetes.io/region`](/docs/reference/kubernetes-api/labels-annotations-taints/#failure-domainbetakubernetesioregion)
|
||||
* [`topology.kubernetes.io/zone`](/docs/reference/kubernetes-api/labels-annotations-taints/#topologykubernetesiozone)
|
||||
* [`topology.kubernetes.io/region`](/docs/reference/kubernetes-api/labels-annotations-taints/#topologykubernetesiozone)
|
||||
* [`failure-domain.beta.kubernetes.io/zone`](/docs/reference/kubernetes-api/labels-annotations-taints/#failure-domain-beta-kubernetes-io-zone)
|
||||
* [`failure-domain.beta.kubernetes.io/region`](/docs/reference/kubernetes-api/labels-annotations-taints/#failure-domain-beta-kubernetes-io-region)
|
||||
* [`beta.kubernetes.io/instance-type`](/docs/reference/kubernetes-api/labels-annotations-taints/#beta-kubernetes-io-instance-type)
|
||||
* [`node.kubernetes.io/instance-type`](/docs/reference/kubernetes-api/labels-annotations-taints/#nodekubernetesioinstance-type)
|
||||
* [`kubernetes.io/os`](/docs/reference/kubernetes-api/labels-annotations-taints/#kubernetes-io-os)
|
||||
* [`kubernetes.io/arch`](/docs/reference/kubernetes-api/labels-annotations-taints/#kubernetes-io-arch)
|
||||
|
||||
|
||||
@@ -280,7 +280,7 @@ which matches the behavior when this feature is disabled.
|
||||
|
||||
## Taint Nodes by Condition
|
||||
|
||||
The node lifecycle controller automatically creates taints corresponding to
|
||||
In version 1.12, `TaintNodesByCondition` feature is promoted to beta, so node lifecycle controller automatically creates taints corresponding to
|
||||
Node conditions.
|
||||
Similarly the scheduler does not check Node conditions; instead the scheduler checks taints. This assures that Node conditions don't affect what's scheduled onto the Node. The user can choose to ignore some of the Node's problems (represented as Node conditions) by adding appropriate Pod tolerations.
|
||||
Note that `TaintNodesByCondition` only taints nodes with `NoSchedule` effect. `NoExecute` effect is controlled by `TaintBasedEviction` which is a beta feature and enabled by default since version 1.13.
|
||||
|
||||
@@ -174,7 +174,7 @@ Aggregated APIs offer more advanced API features and customization of other feat
|
||||
| Feature | Description | CRDs | Aggregated API |
|
||||
| ------- | ----------- | ---- | -------------- |
|
||||
| Validation | Help users prevent errors and allow you to evolve your API independently of your clients. These features are most useful when there are many clients who can't all update at the same time. | Yes. Most validation can be specified in the CRD using [OpenAPI v3.0 validation](/docs/tasks/access-kubernetes-api/extend-api-custom-resource-definitions/#validation). Any other validations supported by addition of a [Validating Webhook](/docs/reference/access-authn-authz/admission-controllers/#validatingadmissionwebhook-alpha-in-1-8-beta-in-1-9). | Yes, arbitrary validation checks |
|
||||
| Defaulting | See above | Yes, either via [OpenAPI v3.0 validation](/docs/tasks/access-kubernetes-api/extend-api-custom-resource-definitions/#defaulting) `default` keyword (GA in 1.17), or via a [Mutating Webhook](/docs/reference/access-authn-authz/admission-controllers/#mutatingadmissionwebhook) (though this will not be run when reading from etcd for old objects) | Yes |
|
||||
| Defaulting | See above | Yes, either via [OpenAPI v3.0 validation](/docs/tasks/access-kubernetes-api/extend-api-custom-resource-definitions/#defaulting) `default` keyword (beta in 1.16), or via a [Mutating Webhook](/docs/reference/access-authn-authz/admission-controllers/#mutatingadmissionwebhook-beta-in-1-9) | Yes |
|
||||
| Multi-versioning | Allows serving the same object through two API versions. Can help ease API changes like renaming fields. Less important if you control your client versions. | [Yes](/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definition-versioning) | Yes |
|
||||
| Custom Storage | If you need storage with a different performance mode (for example, time-series database instead of key-value store) or isolation for security (for example, encryption secrets or different | No | Yes |
|
||||
| Custom Business Logic | Perform arbitrary checks or actions when creating, reading, updating or deleting an object | Yes, using [Webhooks](/docs/reference/access-authn-authz/extensible-admission-controllers/#admission-webhooks). | Yes |
|
||||
|
||||
@@ -184,32 +184,6 @@ DaemonSet, `/var/lib/kubelet/pod-resources` must be mounted as a
|
||||
|
||||
Support for the "PodResources service" requires `KubeletPodResources` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/) to be enabled. It is enabled by default starting with Kubernetes 1.15.
|
||||
|
||||
## Device Plugin integration with the Topology Manager
|
||||
|
||||
{{< feature-state for_k8s_version="v1.17" state="alpha" >}}
|
||||
|
||||
The Topology Manager is a Kubelet component that allows resources to be co-ordintated in a Topology aligned manner. In order to do this, the Device Plugin API was extended to include a `TopologyInfo` struct.
|
||||
|
||||
|
||||
```gRPC
|
||||
message TopologyInfo {
|
||||
repeated NUMANode nodes = 1;
|
||||
}
|
||||
|
||||
message NUMANode {
|
||||
int64 ID = 1;
|
||||
}
|
||||
```
|
||||
Device Plugins that wish to leverage the Topology Manager can send back a populated TopologyInfo struct as part of the device registration, along with the device IDs and the health of the device. The device manager will then use this information to consult with the Topology Manager and make resource assingment decisions.
|
||||
|
||||
`TopologyInfo` supports a `nodes` field that is either `nil` (the default) or a list of NUMA nodes. This lets the Device Plugin publish that can span NUMA nodes.
|
||||
|
||||
An example `TopologyInfo` struct populated for a device by a Device Plugin:
|
||||
|
||||
```
|
||||
pluginapi.Device{ID: "25102017", Health: pluginapi.Healthy, Topology:&pluginapi.TopologyInfo{Nodes: []*pluginapi.NUMANode{&pluginapi.NUMANode{ID: 0,},}}}
|
||||
```
|
||||
|
||||
## Device plugin examples {#examples}
|
||||
|
||||
Here are some examples of device plugin implementations:
|
||||
@@ -231,6 +205,5 @@ Here are some examples of device plugin implementations:
|
||||
* Learn about [scheduling GPU resources](/docs/tasks/manage-gpus/scheduling-gpus/) using device plugins
|
||||
* Learn about [advertising extended resources](/docs/tasks/administer-cluster/extended-resource-node/) on a node
|
||||
* Read about using [hardware acceleration for TLS ingress](https://kubernetes.io/blog/2019/04/24/hardware-accelerated-ssl/tls-termination-in-ingress-controllers-using-kubernetes-device-plugins-and-runtimeclass/) with Kubernetes
|
||||
* Learn about the [Topology Manager] (/docs/tasks/adminster-cluster/topology-manager/)
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
@@ -537,33 +537,12 @@ With this mechanism, operators will be able to restrict usage of certain high pr
|
||||
|
||||
To enforce this, kube-apiserver flag `--admission-control-config-file` should be used to pass path to the following configuration file:
|
||||
|
||||
{{< tabs name="example1" >}}
|
||||
{{% tab name="apiserver.config.k8s.io/v1" %}}
|
||||
```yaml
|
||||
apiVersion: apiserver.config.k8s.io/v1
|
||||
kind: AdmissionConfiguration
|
||||
plugins:
|
||||
- name: "ResourceQuota"
|
||||
configuration:
|
||||
apiVersion: apiserver.config.k8s.io/v1
|
||||
kind: ResourceQuotaConfiguration
|
||||
limitedResources:
|
||||
- resource: pods
|
||||
matchScopes:
|
||||
- scopeName: PriorityClass
|
||||
operator: In
|
||||
values: ["cluster-services"]
|
||||
```
|
||||
{{% /tab %}}
|
||||
{{% tab name="apiserver.k8s.io/v1alpha1" %}}
|
||||
```yaml
|
||||
# Deprecated in v1.17 in favor of apiserver.config.k8s.io/v1
|
||||
apiVersion: apiserver.k8s.io/v1alpha1
|
||||
kind: AdmissionConfiguration
|
||||
plugins:
|
||||
- name: "ResourceQuota"
|
||||
configuration:
|
||||
# Deprecated in v1.17 in favor of apiserver.config.k8s.io/v1, ResourceQuotaConfiguration
|
||||
apiVersion: resourcequota.admission.k8s.io/v1beta1
|
||||
kind: Configuration
|
||||
limitedResources:
|
||||
@@ -573,8 +552,6 @@ plugins:
|
||||
operator: In
|
||||
values: ["cluster-services"]
|
||||
```
|
||||
{{% /tab %}}
|
||||
{{< /tabs >}}
|
||||
|
||||
Now, "cluster-services" pods will be allowed in only those namespaces where a quota object with a matching `scopeSelector` is present.
|
||||
For example:
|
||||
|
||||
@@ -51,7 +51,6 @@ To enable IPv4/IPv6 dual-stack, enable the `IPv6DualStack` [feature gate](/docs/
|
||||
* `--feature-gates="IPv6DualStack=true"`
|
||||
* `--cluster-cidr=<IPv4 CIDR>,<IPv6 CIDR>` eg. `--cluster-cidr=10.244.0.0/16,fc00::/24`
|
||||
* `--service-cluster-ip-range=<IPv4 CIDR>,<IPv6 CIDR>`
|
||||
* `--node-cidr-mask-size-ipv4|--node-cidr-mask-size-ipv6` defaults to /24 for IPv4 and /64 for IPv6
|
||||
* kubelet:
|
||||
* `--feature-gates="IPv6DualStack=true"`
|
||||
* kube-proxy:
|
||||
@@ -99,7 +98,9 @@ The use of publicly routable and non-publicly routable IPv6 address blocks is ac
|
||||
|
||||
## Known Issues
|
||||
|
||||
* IPv6 network block assignment uses the default IPv4 CIDR block size (/24)
|
||||
* Kubenet forces IPv4,IPv6 positional reporting of IPs (--cluster-cidr)
|
||||
* Dual-stack networking does not function if the `EndpointSlice` feature gate is enabled.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
|
||||
@@ -14,7 +14,7 @@ weight: 10
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
{{< feature-state for_k8s_version="v1.17" state="beta" >}}
|
||||
{{< feature-state for_k8s_version="v1.16" state="alpha" >}}
|
||||
|
||||
_Endpoint Slices_ provide a simple way to track network endpoints within a
|
||||
Kubernetes cluster. They offer a more scalable and extensible alternative to
|
||||
@@ -26,7 +26,7 @@ Endpoints.
|
||||
|
||||
## Endpoint Slice resources {#endpointslice-resource}
|
||||
|
||||
In Kubernetes, an EndpointSlice contains references to a set of network
|
||||
In Kubernetes, an Endpoint Slice contains references to a set of network
|
||||
endpoints. The EndpointSlice controller automatically creates Endpoint Slices
|
||||
for a Kubernetes Service when a selector is specified. These Endpoint Slices
|
||||
will include references to any Pods that match the Service selector. Endpoint
|
||||
@@ -36,13 +36,13 @@ As an example, here's a sample EndpointSlice resource for the `example`
|
||||
Kubernetes Service.
|
||||
|
||||
```yaml
|
||||
apiVersion: discovery.k8s.io/v1beta1
|
||||
apiVersion: discovery.k8s.io/v1alpha1
|
||||
kind: EndpointSlice
|
||||
metadata:
|
||||
name: example-abc
|
||||
labels:
|
||||
kubernetes.io/service-name: example
|
||||
addressType: IPv4
|
||||
addressType: IP
|
||||
ports:
|
||||
- name: http
|
||||
protocol: TCP
|
||||
@@ -50,6 +50,7 @@ ports:
|
||||
endpoints:
|
||||
- addresses:
|
||||
- "10.1.2.3"
|
||||
- "2001:db8::1234:5678"
|
||||
conditions:
|
||||
ready: true
|
||||
hostname: pod-1
|
||||
@@ -66,14 +67,6 @@ Endpoint Slices can act as the source of truth for kube-proxy when it comes to
|
||||
how to route internal traffic. When enabled, they should provide a performance
|
||||
improvement for services with large numbers of endpoints.
|
||||
|
||||
## Address Types
|
||||
|
||||
EndpointSlices support three address types:
|
||||
|
||||
* IPv4
|
||||
* IPv6
|
||||
* FQDN (Fully Qualified Domain Name)
|
||||
|
||||
## Motivation
|
||||
|
||||
The Endpoints API has provided a simple and straightforward way of
|
||||
|
||||
@@ -1,127 +0,0 @@
|
||||
---
|
||||
reviewers:
|
||||
- johnbelamaric
|
||||
- imroc
|
||||
title: Service Topology
|
||||
feature:
|
||||
title: Service Topology
|
||||
description: >
|
||||
Routing of service traffice based upon cluster topology.
|
||||
|
||||
content_template: templates/concept
|
||||
weight: 10
|
||||
---
|
||||
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
{{< feature-state for_k8s_version="v1.17" state="alpha" >}}
|
||||
|
||||
_Service Topology_ enables a service to route traffic based upon the Node
|
||||
topology of the cluster. For example, a service can specify that traffic be
|
||||
preferentially routed to endpoints that are on the same Node as the client, or
|
||||
in the same availability zone.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture body %}}
|
||||
|
||||
## Introduction
|
||||
|
||||
By default, traffic sent to a `ClusterIP` or `NodePort` Service may be routed to
|
||||
any backend address for the Service. Since Kubernetes 1.7 it has been possible
|
||||
to route "external" traffic to the Pods running on the Node that received the
|
||||
traffic, but this is not supported for `ClusterIP` Services, and more complex
|
||||
topologies — such as routing zonally — have not been possible. The
|
||||
_Service Topology_ feature resolves this by allowing the Service creator to
|
||||
define a policy for routing traffic based upon the Node labels for the
|
||||
originating and destination Nodes.
|
||||
|
||||
By using Node label matching between the source and destination, the operator
|
||||
may designate groups of Nodes that are "closer" and "farther" from one another,
|
||||
using whatever metric makes sense for that operator's requirements. For many
|
||||
operators in public clouds, for example, there is a preference to keep service
|
||||
traffic withing the same zone, because interzonal traffic has a cost associated
|
||||
with it, while intrazonal traffic does not. Other common needs include being able
|
||||
to route traffic to a local Pod managed by a DaemonSet, or keeping traffic to
|
||||
Nodes connected to the same top-of-rack switch for the lowest latency.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
The following prerequisites are needed in order to enable topology aware service
|
||||
routing:
|
||||
|
||||
* Kubernetes 1.17 or later
|
||||
* Kube-proxy running in iptables mode or IPVS mode
|
||||
* Enable [Endpoint Slices](/docs/concepts/services-networking/endpoint-slices/)
|
||||
|
||||
## Enable Service Topology
|
||||
|
||||
To enable service topology, enable the `ServiceTopology` feature gate for
|
||||
kube-apiserver and kube-proxy:
|
||||
|
||||
```
|
||||
--feature-gates="ServiceTopology=true"
|
||||
```
|
||||
|
||||
## Using Service Topology
|
||||
|
||||
If your cluster has Service Topology enabled, you can control Service traffic
|
||||
routing by specifying the `topologyKeys` field on the Service spec. This field
|
||||
is a preference-order list of Node labels which will be used to sort endpoints
|
||||
when accessing this Service. Traffic will be directed to a Node whose value for
|
||||
the first label matches the originating Node's value for that label. If there is
|
||||
no backend for the Service on a matching Node, then the second label will be
|
||||
considered, and so forth, until no labels remain.
|
||||
|
||||
If no match is found, the traffic will be rejected, just as if there were no
|
||||
backends for the Service at all. That is, endpoints are chosen based on the first
|
||||
topology key with available backends. If this field is specified and all entries
|
||||
have no backends that match the topology of the client, the service has no
|
||||
backends for that client and connections should fail. The special value `"*"` may
|
||||
be used to mean "any topology". This catch-all value, if used, only makes sense
|
||||
as the last value in the list.
|
||||
|
||||
If `topologyKeys` is not specified or empty, no topology constraints will be applied.
|
||||
|
||||
Consider a cluster with Nodes that are labeled with their hostname, zone name,
|
||||
and region name. Then you can set the `topologyKeys` values of a service to direct
|
||||
traffic as follows.
|
||||
|
||||
* Only to endpoints on the same node, failing if no endpoint exists on the node:
|
||||
`["kubernetes.io/hostname"]`.
|
||||
* Preferentially to endpoints on the same node, falling back to endpoints in the
|
||||
same zone, followed by the same region, and failing otherwise: `["kubernetes.io/hostname",
|
||||
"topology.kubernetes.io/zone", "topology.kubernetes.io/region"]`.
|
||||
This may be useful, for example, in cases where data locality is critical.
|
||||
* Preferentially to the same zone, but fallback on any available endpoint if
|
||||
none are available within this zone:
|
||||
`["topology.kubernetes.io/zone", "*"]`.
|
||||
|
||||
|
||||
|
||||
## Constraints
|
||||
|
||||
* Service topology is not compatible with `externalTrafficPolicy=Local`, and
|
||||
therefore a Service cannot use both of these features. It is possible to use
|
||||
both features in the same cluster on different Services, just not on the same
|
||||
Service.
|
||||
|
||||
* Valid topology keys are currently limited to `kubernetes.io/hostname`,
|
||||
`topology.kubernetes.io/zone`, and `topology.kubernetes.io/region`, but will
|
||||
be generalized to other node labels in the future.
|
||||
|
||||
* Topology keys must be valid label keys and at most 16 keys may be specified.
|
||||
|
||||
* The catch-all value, `"*"`, must be the last value in the topology keys, if
|
||||
it is used.
|
||||
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture whatsnext %}}
|
||||
|
||||
* Read about [enabling Service Topology](/docs/tasks/administer-cluster/enabling-service-topology)
|
||||
* Read [Connecting Applications with Services](/docs/concepts/services-networking/connect-applications-service/)
|
||||
|
||||
{{% /capture %}}
|
||||
@@ -185,7 +185,7 @@ selectors and uses DNS names instead. For more information, see the
|
||||
[ExternalName](#externalname) section later in this document.
|
||||
|
||||
### Endpoint Slices
|
||||
{{< feature-state for_k8s_version="v1.17" state="beta" >}}
|
||||
{{< feature-state for_k8s_version="v1.16" state="alpha" >}}
|
||||
|
||||
Endpoint Slices are an API resource that can provide a more scalable alternative
|
||||
to Endpoints. Although conceptually quite similar to Endpoints, Endpoint Slices
|
||||
|
||||
@@ -185,10 +185,10 @@ The following plugins support `WaitForFirstConsumer` with pre-created Persistent
|
||||
* All of the above
|
||||
* [Local](#local)
|
||||
|
||||
{{< feature-state state="stable" for_k8s_version="1.17" >}}
|
||||
{{< feature-state state="beta" for_k8s_version="1.14" >}}
|
||||
[CSI volumes](/docs/concepts/storage/volumes/#csi) are also supported with dynamic provisioning
|
||||
and pre-created PVs, but you'll need to look at the documentation for a specific CSI driver
|
||||
to see its supported topology keys and examples.
|
||||
to see its supported topology keys and examples. The `CSINodeInfo` feature gate must be enabled.
|
||||
|
||||
### Allowed Topologies
|
||||
|
||||
|
||||
@@ -38,8 +38,6 @@ that can be attached to a Node:
|
||||
|
||||
You can change these limits by setting the value of the
|
||||
`KUBE_MAX_PD_VOLS` environment variable, and then starting the scheduler.
|
||||
CSI drivers might have a different procedure, see their documentation
|
||||
on how to customize their limits.
|
||||
|
||||
Use caution if you set a limit that is higher than the default limit. Consult
|
||||
the cloud provider's documentation to make sure that Nodes can actually support
|
||||
@@ -49,7 +47,10 @@ The limit applies to the entire cluster, so it affects all Nodes.
|
||||
|
||||
## Dynamic volume limits
|
||||
|
||||
{{< feature-state state="stable" for_k8s_version="v1.17" >}}
|
||||
{{< feature-state state="beta" for_k8s_version="v1.12" >}}
|
||||
|
||||
Kubernetes 1.11 introduced support for dynamic volume limits based on Node type as an Alpha feature.
|
||||
In Kubernetes 1.12 this feature is graduating to Beta and will be enabled by default.
|
||||
|
||||
Dynamic volume limits are supported for following volume types.
|
||||
|
||||
@@ -58,12 +59,14 @@ Dynamic volume limits are supported for following volume types.
|
||||
- Azure Disk
|
||||
- CSI
|
||||
|
||||
For volumes managed by in-tree volume plugins, Kubernetes automatically determines the Node
|
||||
type and enforces the appropriate maximum number of volumes for the node. For example:
|
||||
|
||||
When the dynamic volume limits feature is enabled, Kubernetes automatically
|
||||
determines the Node type and enforces the appropriate number of attachable
|
||||
volumes for the node. For example:
|
||||
|
||||
* On
|
||||
<a href="https://cloud.google.com/compute/">Google Compute Engine</a>,
|
||||
up to 127 volumes can be attached to a node, [depending on the node
|
||||
up to 128 volumes can be attached to a node, [depending on the node
|
||||
type](https://cloud.google.com/compute/docs/disks/#pdnumberlimits).
|
||||
|
||||
* For Amazon EBS disks on M5,C5,R5,T3 and Z1D instance types, Kubernetes allows only 25
|
||||
@@ -73,9 +76,7 @@ Kubernetes allows 39 volumes to be attached to a Node.
|
||||
|
||||
* On Azure, up to 64 disks can be attached to a node, depending on the node type. For more details, refer to [Sizes for virtual machines in Azure](https://docs.microsoft.com/en-us/azure/virtual-machines/windows/sizes).
|
||||
|
||||
* If a CSI storage driver advertises a maximum number of volumes for a Node (using `NodeGetInfo`), the {{< glossary_tooltip text="kube-scheduler" term_id="kube-scheduler" >}} honors that limit.
|
||||
Refer to the [CSI specifications](https://github.com/container-storage-interface/spec/blob/master/spec.md#nodegetinfo) for details.
|
||||
|
||||
* For volumes managed by in-tree plugins that have been migrated to a CSI driver, the maximum number of volumes will be the one reported by the CSI driver.
|
||||
* For CSI, any driver that advertises volume attach limits via CSI specs will have those limits available as the Node's allocatable property
|
||||
and the Scheduler will not schedule Pods with volumes on any Node that is already at its capacity. Refer to the [CSI specs](https://github.com/container-storage-interface/spec/blob/master/spec.md#nodegetinfo) for more details.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
@@ -1,11 +1,9 @@
|
||||
---
|
||||
reviewers:
|
||||
- jsafrane
|
||||
- saad-ali
|
||||
- thockin
|
||||
- msau42
|
||||
- jingxu97
|
||||
- xing-yang
|
||||
- yuxiangqian
|
||||
title: Volume Snapshot Classes
|
||||
content_template: templates/concept
|
||||
weight: 30
|
||||
@@ -30,7 +28,7 @@ way to describe the "classes" of storage when provisioning a volume snapshot.
|
||||
|
||||
## The VolumeSnapshotClass Resource
|
||||
|
||||
Each `VolumeSnapshotClass` contains the fields `driver`, `deletionPolicy`, and `parameters`,
|
||||
Each `VolumeSnapshotClass` contains the fields `snapshotter` and `parameters`,
|
||||
which are used when a `VolumeSnapshot` belonging to the class needs to be
|
||||
dynamically provisioned.
|
||||
|
||||
@@ -43,30 +41,23 @@ Administrators can specify a default `VolumeSnapshotClass` just for VolumeSnapsh
|
||||
that don't request any particular class to bind to.
|
||||
|
||||
```yaml
|
||||
apiVersion: snapshot.storage.k8s.io/v1beta1
|
||||
apiVersion: snapshot.storage.k8s.io/v1alpha1
|
||||
kind: VolumeSnapshotClass
|
||||
metadata:
|
||||
name: csi-hostpath-snapclass
|
||||
driver: hostpath.csi.k8s.io
|
||||
deletionPolicy: Delete
|
||||
snapshotter: csi-hostpath
|
||||
parameters:
|
||||
```
|
||||
|
||||
### Driver
|
||||
### Snapshotter
|
||||
|
||||
Volume snapshot classes have a driver that determines what CSI volume plugin is
|
||||
Volume snapshot classes have a snapshotter that determines what CSI volume plugin is
|
||||
used for provisioning VolumeSnapshots. This field must be specified.
|
||||
|
||||
### DeletionPolicy
|
||||
|
||||
Volume snapshot classes have a deletionPolicy. It enables you to configure what happens to a `VolumeSnapshotContent` when the `VolumeSnapshot` object it is bound to is to be deleted. The deletionPolicy of a volume snapshot can either be `Retain` or `Delete`. This field must be specified.
|
||||
|
||||
If the deletionPolicy is `Delete`, then the underlying storage snapshot will be deleted along with the `VolumeSnapshotContent` object. If the deletionPolicy is `Retain`, then both the underlying snapshot and `VolumeSnapshotContent` remain.
|
||||
|
||||
## Parameters
|
||||
|
||||
Volume snapshot classes have parameters that describe volume snapshots belonging to
|
||||
the volume snapshot class. Different parameters may be accepted depending on the
|
||||
`driver`.
|
||||
`snapshotter`.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
@@ -1,11 +1,9 @@
|
||||
---
|
||||
reviewers:
|
||||
- jsafrane
|
||||
- saad-ali
|
||||
- thockin
|
||||
- msau42
|
||||
- jingxu97
|
||||
- xing-yang
|
||||
- yuxiangqian
|
||||
title: Volume Snapshots
|
||||
content_template: templates/concept
|
||||
weight: 20
|
||||
@@ -13,8 +11,8 @@ weight: 20
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
{{< feature-state for_k8s_version="1.17" state="beta" >}}
|
||||
In Kubernetes, a _VolumeSnapshot_ represents a snapshot of a volume on a storage system. This document assumes that you are already familiar with Kubernetes [persistent volumes](/docs/concepts/storage/persistent-volumes/).
|
||||
{{< feature-state for_k8s_version="v1.12" state="alpha" >}}
|
||||
This document describes the current state of `VolumeSnapshots` in Kubernetes. Familiarity with [persistent volumes](/docs/concepts/storage/persistent-volumes/) is suggested.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
@@ -29,15 +27,18 @@ A `VolumeSnapshotContent` is a snapshot taken from a volume in the cluster that
|
||||
|
||||
A `VolumeSnapshot` is a request for snapshot of a volume by a user. It is similar to a PersistentVolumeClaim.
|
||||
|
||||
`VolumeSnapshotClass` allows you to specify different attributes belonging to a `VolumeSnapshot`. These attibutes may differ among snapshots taken from the same volume on the storage system and therefore cannot be expressed by using the same `StorageClass` of a `PersistentVolumeClaim`.
|
||||
While `VolumeSnapshots` allow a user to consume abstract storage resources, cluster administrators
|
||||
need to be able to offer a variety of `VolumeSnapshotContents` without exposing
|
||||
users to the details of how those volume snapshots should be provisioned. For these needs
|
||||
there is the `VolumeSnapshotClass` resource.
|
||||
|
||||
Users need to be aware of the following when using this feature:
|
||||
|
||||
* API Objects `VolumeSnapshot`, `VolumeSnapshotContent`, and `VolumeSnapshotClass` are {{< glossary_tooltip term_id="CustomResourceDefinition" text="CRDs" >}}, not part of the core API.
|
||||
* API Objects `VolumeSnapshot`, `VolumeSnapshotContent`, and `VolumeSnapshotClass` are CRDs, not part of the core API.
|
||||
* `VolumeSnapshot` support is only available for CSI drivers.
|
||||
* As part of the deployment process in the beta version of `VolumeSnapshot`, the Kubernetes team provides a snapshot controller to be deployed into the control plane, and a sidecar helper container called csi-snapshotter to be deployed together with the CSI driver. The snapshot controller watches `VolumeSnapshot` and `VolumeSnapshotContent` objects and is responsible for the creation and deletion of `VolumeSnapshotContent` object in dynamic provisioning. The sidecar csi-snapshotter watches `VolumeSnapshotContent` objects and triggers `CreateSnapshot` and `DeleteSnapshot` operations against a CSI endpoint.
|
||||
* CSI drivers may or may not have implemented the volume snapshot functionality. The CSI drivers that have provided support for volume snapshot will likely use the csi-snapshotter. See [CSI Driver documentation](https://kubernetes-csi.github.io/docs/) for details.
|
||||
* The CRDs and snapshot controller installations are the responsibility of the Kubernetes distribution.
|
||||
* As part of the deployment process, the Kubernetes team provides a sidecar helper container for the snapshot controller called `external-snapshotter`. It watches `VolumeSnapshot` objects and triggers `CreateSnapshot` and `DeleteSnapshot` operations against a CSI endpoint.
|
||||
* CSI drivers may or may not have implemented the volume snapshot functionality. The CSI drivers that have provided support for volume snapshot will likely use `external-snapshotter`.
|
||||
* The CSI drivers that support volume snapshot will automatically install CRDs defined for the volume snapshots.
|
||||
|
||||
## Lifecycle of a volume snapshot and volume snapshot content
|
||||
|
||||
@@ -45,104 +46,89 @@ Users need to be aware of the following when using this feature:
|
||||
|
||||
### Provisioning Volume Snapshot
|
||||
|
||||
There are two ways snapshots may be provisioned: pre-provisioned or dynamically provisioned.
|
||||
There are two ways snapshots may be provisioned: statically or dynamically.
|
||||
|
||||
#### Pre-provisioned {#static}
|
||||
A cluster administrator creates a number of `VolumeSnapshotContents`. They carry the details of the real volume snapshot on the storage system which is available for use by cluster users. They exist in the Kubernetes API and are available for consumption.
|
||||
#### Static
|
||||
A cluster administrator creates a number of `VolumeSnapshotContents`. They carry the details of the real storage which is available for use by cluster users. They exist in the Kubernetes API and are available for consumption.
|
||||
|
||||
#### Dynamic
|
||||
Instead of using a pre-existing snapshot, you can request that a snapshot to be dynamically taken from a PersistentVolumeClaim. The [VolumeSnapshotClass](/docs/concepts/storage/volume-snapshot-classes/) specifies storage provider-specific parameters to use when taking a snapshot.
|
||||
When none of the static `VolumeSnapshotContents` the administrator created matches a user's `VolumeSnapshot`,
|
||||
the cluster may try to dynamically provision a volume snapshot specially for the `VolumeSnapshot` object.
|
||||
This provisioning is based on `VolumeSnapshotClasses`: the `VolumeSnapshot` must request a
|
||||
[volume snapshot class](/docs/concepts/storage/volume-snapshot-classes/) and
|
||||
the administrator must have created and configured that class in order for dynamic
|
||||
provisioning to occur.
|
||||
|
||||
### Binding
|
||||
|
||||
The snapshot controller handles the binding of a `VolumeSnapshot` object with an appropriate `VolumeSnapshotContent` object, in both pre-provisioned and dynamically provisioned scenarios. The binding is a one-to-one mapping.
|
||||
A user creates, or has already created in the case of dynamic provisioning, a `VolumeSnapshot` with a specific amount of storage requested and with certain access modes. A control loop watches for new VolumeSnapshots, finds a matching VolumeSnapshotContent (if possible), and binds them together. If a VolumeSnapshotContent was dynamically provisioned for a new VolumeSnapshot, the loop will always bind that VolumeSnapshotContent to the VolumeSnapshot. Once bound, `VolumeSnapshot` binds are exclusive, regardless of how they were bound. A VolumeSnapshot to VolumeSnapshotContent binding is a one-to-one mapping.
|
||||
|
||||
In the case of pre-provisioned binding, the VolumeSnapshot will remain unbound until the requested VolumeSnapshotContent object is created.
|
||||
VolumeSnapshots will remain unbound indefinitely if a matching VolumeSnapshotContent does not exist. VolumeSnapshots will be bound as matching VolumeSnapshotContents become available.
|
||||
|
||||
### Persistent Volume Claim as Snapshot Source Protection
|
||||
### Persistent Volume Claim in Use Protection
|
||||
|
||||
The purpose of this protection is to ensure that in-use PersistentVolumeClaim API objects are not removed from the system while a snapshot is being taken from it (as this may result in data loss).
|
||||
The purpose of the Persistent Volume Claim Object in Use Protection feature is to ensure that in-use PVC API objects are not removed from the system (as this may result in data loss).
|
||||
|
||||
While a snapshot is being taken of a PersistentVolumeClaim, that PersistentVolumeClaim is in-use. If you delete a PersistentVolumeClaim API object in active use as a snapshot source, the PersistentVolumeClaim object is not removed immediately. Instead, removal of the PersistentVolumeClaim object is postponed until the snapshot is readyToUse or aborted.
|
||||
If a PVC is in active use by a snapshot as a source to create the snapshot, the PVC is in-use. If a user deletes a PVC API object in active use as a snapshot source, the PVC object is not removed immediately. Instead, removal of the PVC object is postponed until the PVC is no longer actively used by any snapshots. A PVC is no longer used as a snapshot source when `ReadyToUse` of the snapshot `Status` becomes `true`.
|
||||
|
||||
### Delete
|
||||
|
||||
Deletion is triggered by deleting the `VolumeSnapshot` object, and the `DeletionPolicy` will be followed. If the `DeletionPolicy` is `Delete`, then the underlying storage snapshot will be deleted along with the `VolumeSnapshotContent` object. If the `DeletionPolicy` is `Retain`, then both the underlying snapshot and `VolumeSnapshotContent` remain.
|
||||
|
||||
## VolumeSnapshots
|
||||
|
||||
Each VolumeSnapshot contains a spec and a status.
|
||||
|
||||
```yaml
|
||||
apiVersion: snapshot.storage.k8s.io/v1beta1
|
||||
kind: VolumeSnapshot
|
||||
metadata:
|
||||
name: new-snapshot-test
|
||||
spec:
|
||||
volumeSnapshotClassName: csi-hostpath-snapclass
|
||||
source:
|
||||
persistentVolumeClaimName: pvc-test
|
||||
```
|
||||
|
||||
`persistentVolumeClaimName` is the name of the PersistentVolumeClaim data source for the snapshot. This field is required for dynamically provisioning a snapshot.
|
||||
|
||||
A volume snapshot can request a particular class by specifying the name of a
|
||||
[VolumeSnapshotClass](/docs/concepts/storage/volume-snapshot-classes/)
|
||||
using the attribute `volumeSnapshotClassName`. If nothing is set, then the default class is used if available.
|
||||
|
||||
For pre-provisioned snapshots, you need to specify a `volumeSnapshotContentName` as the source for the snapshot as shown in the following example. The `volumeSnapshotContentName` source field is required for pre-provisioned snapshots.
|
||||
|
||||
```
|
||||
apiVersion: snapshot.storage.k8s.io/v1beta1
|
||||
kind: VolumeSnapshot
|
||||
metadata:
|
||||
name: test-snapshot
|
||||
spec:
|
||||
source:
|
||||
volumeSnapshotContentName: test-content
|
||||
```
|
||||
Deletion removes both the `VolumeSnapshotContent` object from the Kubernetes API, as well as the associated storage asset in the external infrastructure.
|
||||
|
||||
## Volume Snapshot Contents
|
||||
|
||||
Each VolumeSnapshotContent contains a spec and status. In dynamic provisioning, the snapshot common controller creates `VolumeSnapshotContent` objects. Here is an example:
|
||||
Each VolumeSnapshotContent contains a spec, which is the specification of the volume snapshot.
|
||||
|
||||
```yaml
|
||||
apiVersion: snapshot.storage.k8s.io/v1beta1
|
||||
kind: VolumeSnapshotContent
|
||||
metadata:
|
||||
name: snapcontent-72d9a349-aacd-42d2-a240-d775650d2455
|
||||
spec:
|
||||
deletionPolicy: Delete
|
||||
driver: hostpath.csi.k8s.io
|
||||
source:
|
||||
volumeHandle: ee0cfb94-f8d4-11e9-b2d8-0242ac110002
|
||||
volumeSnapshotClassName: csi-hostpath-snapclass
|
||||
volumeSnapshotRef:
|
||||
name: new-snapshot-test
|
||||
namespace: default
|
||||
uid: 72d9a349-aacd-42d2-a240-d775650d2455
|
||||
```
|
||||
|
||||
`volumeHandle` is the unique identifier of the volume created on the storage backend and returned by the CSI driver during the volume creation. This field is required for dynamically provisioning a snapshot. It specifies the volume source of the snapshot.
|
||||
|
||||
For pre-provisioned snapshots, you (as cluster administrator) are responsible for creating the `VolumeSnapshotContent` object as follows.
|
||||
|
||||
```yaml
|
||||
apiVersion: snapshot.storage.k8s.io/v1beta1
|
||||
apiVersion: snapshot.storage.k8s.io/v1alpha1
|
||||
kind: VolumeSnapshotContent
|
||||
metadata:
|
||||
name: new-snapshot-content-test
|
||||
spec:
|
||||
deletionPolicy: Delete
|
||||
driver: hostpath.csi.k8s.io
|
||||
snapshotClassName: csi-hostpath-snapclass
|
||||
source:
|
||||
snapshotHandle: 7bdd0de3-aaeb-11e8-9aae-0242ac110002
|
||||
volumeSnapshotRef:
|
||||
name: new-snapshot-test
|
||||
namespace: default
|
||||
name: pvc-test
|
||||
kind: PersistentVolumeClaim
|
||||
volumeSnapshotSource:
|
||||
csiVolumeSnapshotSource:
|
||||
creationTime: 1535478900692119403
|
||||
driver: csi-hostpath
|
||||
restoreSize: 10Gi
|
||||
snapshotHandle: 7bdd0de3-aaeb-11e8-9aae-0242ac110002
|
||||
```
|
||||
|
||||
`snapshotHandle` is the unique identifier of the volume snapshot created on the storage backend. This field is required for the pre-provisioned snapshots. It specifies the CSI snapshot id on the storage system that this `VolumeSnapshotContent` represents.
|
||||
### Class
|
||||
|
||||
A VolumeSnapshotContent can have a class, which is specified by setting the
|
||||
`snapshotClassName` attribute to the name of a
|
||||
[VolumeSnapshotClass](/docs/concepts/storage/volume-snapshot-classes/).
|
||||
A VolumeSnapshotContent of a particular class can only be bound to VolumeSnapshots requesting
|
||||
that class. A VolumeSnapshotContent with no `snapshotClassName` has no class and can only be bound
|
||||
to VolumeSnapshots that request no particular class.
|
||||
|
||||
## VolumeSnapshots
|
||||
|
||||
Each VolumeSnapshot contains a spec and a status, which is the specification and status of the volume snapshot.
|
||||
|
||||
```yaml
|
||||
apiVersion: snapshot.storage.k8s.io/v1alpha1
|
||||
kind: VolumeSnapshot
|
||||
metadata:
|
||||
name: new-snapshot-test
|
||||
spec:
|
||||
snapshotClassName: csi-hostpath-snapclass
|
||||
source:
|
||||
name: pvc-test
|
||||
kind: PersistentVolumeClaim
|
||||
```
|
||||
|
||||
### Class
|
||||
|
||||
A volume snapshot can request a particular class by specifying the name of a
|
||||
[VolumeSnapshotClass](/docs/concepts/storage/volume-snapshot-classes/)
|
||||
using the attribute `snapshotClassName`.
|
||||
Only VolumeSnapshotContents of the requested class, ones with the same `snapshotClassName`
|
||||
as the VolumeSnapshot, can be bound to the VolumeSnapshot.
|
||||
|
||||
## Provisioning Volumes from Snapshots
|
||||
|
||||
|
||||
@@ -151,19 +151,14 @@ spec:
|
||||
|
||||
#### CSI Migration
|
||||
|
||||
{{< feature-state for_k8s_version="v1.17" state="beta" >}}
|
||||
{{< feature-state for_k8s_version="v1.14" state="alpha" >}}
|
||||
|
||||
The CSI Migration feature for awsElasticBlockStore, when enabled, shims all plugin operations
|
||||
from the existing in-tree plugin to the `ebs.csi.aws.com` Container
|
||||
Storage Interface (CSI) Driver. In order to use this feature, the [AWS EBS CSI
|
||||
Driver](https://github.com/kubernetes-sigs/aws-ebs-csi-driver)
|
||||
must be installed on the cluster and the `CSIMigration` and `CSIMigrationAWS`
|
||||
Beta features must be enabled.
|
||||
|
||||
#### CSI Migration Complete
|
||||
{{< feature-state for_k8s_version="v1.17" state="alpha" >}}
|
||||
|
||||
To turn off the awsElasticBlockStore storage plugin from being loaded by controller manager and kubelet, you need to set this feature flag to true. This requires `ebs.csi.aws.com` Container Storage Interface (CSI) driver being installed on all worker nodes.
|
||||
Alpha features must be enabled.
|
||||
|
||||
### azureDisk {#azuredisk}
|
||||
|
||||
@@ -484,14 +479,14 @@ spec:
|
||||
|
||||
#### CSI Migration
|
||||
|
||||
{{< feature-state for_k8s_version="v1.17" state="beta" >}}
|
||||
{{< feature-state for_k8s_version="v1.14" state="alpha" >}}
|
||||
|
||||
The CSI Migration feature for GCE PD, when enabled, shims all plugin operations
|
||||
from the existing in-tree plugin to the `pd.csi.storage.gke.io` Container
|
||||
Storage Interface (CSI) Driver. In order to use this feature, the [GCE PD CSI
|
||||
Driver](https://github.com/kubernetes-sigs/gcp-compute-persistent-disk-csi-driver)
|
||||
must be installed on the cluster and the `CSIMigration` and `CSIMigrationGCE`
|
||||
Beta features must be enabled.
|
||||
Alpha features must be enabled.
|
||||
|
||||
### gitRepo (deprecated) {#gitrepo}
|
||||
|
||||
@@ -1159,7 +1154,7 @@ spec:
|
||||
|
||||
### Using subPath with expanded environment variables
|
||||
|
||||
{{< feature-state for_k8s_version="v1.17" state="stable" >}}
|
||||
{{< feature-state for_k8s_version="v1.15" state="beta" >}}
|
||||
|
||||
|
||||
Use the `subPathExpr` field to construct `subPath` directory names from Downward API environment variables.
|
||||
|
||||
@@ -101,9 +101,21 @@ If you do not specify either, then the DaemonSet controller will create Pods on
|
||||
|
||||
## How Daemon Pods are Scheduled
|
||||
|
||||
### Scheduled by default scheduler
|
||||
### Scheduled by DaemonSet controller (disabled by default since 1.12)
|
||||
|
||||
{{< feature-state state="stable" for-kubernetes-version="1.17" >}}
|
||||
Normally, the machine that a Pod runs on is selected by the Kubernetes scheduler. However, Pods
|
||||
created by the DaemonSet controller have the machine already selected (`.spec.nodeName` is specified
|
||||
when the Pod is created, so it is ignored by the scheduler). Therefore:
|
||||
|
||||
- The [`unschedulable`](/docs/admin/node/#manual-node-administration) field of a node is not respected
|
||||
by the DaemonSet controller.
|
||||
- The DaemonSet controller can make Pods even when the scheduler has not been started, which can help cluster
|
||||
bootstrap.
|
||||
|
||||
|
||||
### Scheduled by default scheduler (enabled by default since 1.12)
|
||||
|
||||
{{< feature-state state="beta" for-kubernetes-version="1.12" >}}
|
||||
|
||||
A DaemonSet ensures that all eligible nodes run a copy of a Pod. Normally, the
|
||||
node that a Pod runs on is selected by the Kubernetes scheduler. However,
|
||||
|
||||
Reference in New Issue
Block a user