Merge remote-tracking branch 'upstream/master' into HEAD
This commit is contained in:
vineeth
@@ -184,7 +184,7 @@ a Lease object.
|
||||
timeout for unreachable nodes).
|
||||
- The kubelet creates and then updates its Lease object every 10 seconds
|
||||
(the default update interval). Lease updates occur independently from the
|
||||
`NodeStatus` updates.
|
||||
`NodeStatus` updates. If the Lease update fails, the kubelet retries with exponential backoff starting at 200 milliseconds and capped at 7 seconds.
|
||||
|
||||
#### Reliability
|
||||
|
||||
|
||||
@@ -44,7 +44,7 @@ Note: Not all distros are actively maintained. Choose distros which have been te
|
||||
|
||||
* [Certificates](/docs/concepts/cluster-administration/certificates/) describes the steps to generate certificates using different tool chains.
|
||||
|
||||
* [Kubernetes Container Environment](/docs/concepts/containers/container-environment-variables/) describes the environment for Kubelet managed containers on a Kubernetes node.
|
||||
* [Kubernetes Container Environment](/docs/concepts/containers/container-environment/) describes the environment for Kubelet managed containers on a Kubernetes node.
|
||||
|
||||
* [Controlling Access to the Kubernetes API](/docs/reference/access-authn-authz/controlling-access/) describes how to set up permissions for users and service accounts.
|
||||
|
||||
|
||||
@@ -1,186 +0,0 @@
|
||||
---
|
||||
title: Federation
|
||||
content_template: templates/concept
|
||||
weight: 80
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
{{< deprecationfilewarning >}}
|
||||
{{< include "federation-deprecation-warning-note.md" >}}
|
||||
{{< /deprecationfilewarning >}}
|
||||
|
||||
This page explains why and how to manage multiple Kubernetes clusters using
|
||||
federation.
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture body %}}
|
||||
## Why federation
|
||||
|
||||
Federation makes it easy to manage multiple clusters. It does so by providing 2
|
||||
major building blocks:
|
||||
|
||||
* Sync resources across clusters: Federation provides the ability to keep
|
||||
resources in multiple clusters in sync. For example, you can ensure that the same deployment exists in multiple clusters.
|
||||
* Cross cluster discovery: Federation provides the ability to auto-configure DNS servers and load balancers with backends from all clusters. For example, you can ensure that a global VIP or DNS record can be used to access backends from multiple clusters.
|
||||
|
||||
Some other use cases that federation enables are:
|
||||
|
||||
* High Availability: By spreading load across clusters and auto configuring DNS
|
||||
servers and load balancers, federation minimises the impact of cluster
|
||||
failure.
|
||||
* Avoiding provider lock-in: By making it easier to migrate applications across
|
||||
clusters, federation prevents cluster provider lock-in.
|
||||
|
||||
|
||||
Federation is not helpful unless you have multiple clusters. Some of the reasons
|
||||
why you might want multiple clusters are:
|
||||
|
||||
* Low latency: Having clusters in multiple regions minimises latency by serving
|
||||
users from the cluster that is closest to them.
|
||||
* Fault isolation: It might be better to have multiple small clusters rather
|
||||
than a single large cluster for fault isolation (for example: multiple
|
||||
clusters in different availability zones of a cloud provider).
|
||||
* Scalability: There are scalability limits to a single kubernetes cluster (this
|
||||
should not be the case for most users. For more details:
|
||||
[Kubernetes Scaling and Performance Goals](https://git.k8s.io/community/sig-scalability/goals.md)).
|
||||
* [Hybrid cloud](#hybrid-cloud-capabilities): You can have multiple clusters on different cloud providers or
|
||||
on-premises data centers.
|
||||
|
||||
### Caveats
|
||||
|
||||
While there are a lot of attractive use cases for federation, there are also
|
||||
some caveats:
|
||||
|
||||
* Increased network bandwidth and cost: The federation control plane watches all
|
||||
clusters to ensure that the current state is as expected. This can lead to
|
||||
significant network cost if the clusters are running in different regions on
|
||||
a cloud provider or on different cloud providers.
|
||||
* Reduced cross cluster isolation: A bug in the federation control plane can
|
||||
impact all clusters. This is mitigated by keeping the logic in federation
|
||||
control plane to a minimum. It mostly delegates to the control plane in
|
||||
kubernetes clusters whenever it can. The design and implementation also errs
|
||||
on the side of safety and avoiding multi-cluster outage.
|
||||
* Maturity: The federation project is relatively new and is not very mature.
|
||||
Not all resources are available and many are still alpha. [Issue
|
||||
88](https://github.com/kubernetes/federation/issues/88) enumerates
|
||||
known issues with the system that the team is busy solving.
|
||||
|
||||
### Hybrid cloud capabilities
|
||||
|
||||
Federations of Kubernetes Clusters can include clusters running in
|
||||
different cloud providers (e.g. Google Cloud, AWS), and on-premises
|
||||
(e.g. on OpenStack). [Kubefed](/docs/tasks/federation/set-up-cluster-federation-kubefed/) is the recommended way to deploy federated clusters.
|
||||
|
||||
Thereafter, your [API resources](#api-resources) can span different clusters
|
||||
and cloud providers.
|
||||
|
||||
## Setting up federation
|
||||
|
||||
To be able to federate multiple clusters, you first need to set up a federation
|
||||
control plane.
|
||||
Follow the [setup guide](/docs/tutorials/federation/set-up-cluster-federation-kubefed/) to set up the
|
||||
federation control plane.
|
||||
|
||||
## API resources
|
||||
|
||||
Once you have the control plane set up, you can start creating federation API
|
||||
resources.
|
||||
The following guides explain some of the resources in detail:
|
||||
|
||||
* [Cluster](/docs/tasks/federation/administer-federation/cluster/)
|
||||
* [ConfigMap](/docs/tasks/federation/administer-federation/configmap/)
|
||||
* [DaemonSets](/docs/tasks/federation/administer-federation/daemonset/)
|
||||
* [Deployment](/docs/tasks/federation/administer-federation/deployment/)
|
||||
* [Events](/docs/tasks/federation/administer-federation/events/)
|
||||
* [Hpa](/docs/tasks/federation/administer-federation/hpa/)
|
||||
* [Ingress](/docs/tasks/federation/administer-federation/ingress/)
|
||||
* [Jobs](/docs/tasks/federation/administer-federation/job/)
|
||||
* [Namespaces](/docs/tasks/federation/administer-federation/namespaces/)
|
||||
* [ReplicaSets](/docs/tasks/federation/administer-federation/replicaset/)
|
||||
* [Secrets](/docs/tasks/federation/administer-federation/secret/)
|
||||
* [Services](/docs/concepts/cluster-administration/federation-service-discovery/)
|
||||
|
||||
|
||||
The [API reference docs](/docs/reference/federation/) list all the
|
||||
resources supported by federation apiserver.
|
||||
|
||||
## Cascading deletion
|
||||
|
||||
Kubernetes version 1.6 includes support for cascading deletion of federated
|
||||
resources. With cascading deletion, when you delete a resource from the
|
||||
federation control plane, you also delete the corresponding resources in all underlying clusters.
|
||||
|
||||
Cascading deletion is not enabled by default when using the REST API. To enable
|
||||
it, set the option `DeleteOptions.orphanDependents=false` when you delete a
|
||||
resource from the federation control plane using the REST API. Using `kubectl
|
||||
delete`
|
||||
enables cascading deletion by default. You can disable it by running `kubectl
|
||||
delete --cascade=false`
|
||||
|
||||
Note: Kubernetes version 1.5 included cascading deletion support for a subset of
|
||||
federation resources.
|
||||
|
||||
## Scope of a single cluster
|
||||
|
||||
On IaaS providers such as Google Compute Engine or Amazon Web Services, a VM exists in a
|
||||
[zone](https://cloud.google.com/compute/docs/zones) or [availability
|
||||
zone](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html).
|
||||
We suggest that all the VMs in a Kubernetes cluster should be in the same availability zone, because:
|
||||
|
||||
- compared to having a single global Kubernetes cluster, there are fewer single-points of failure.
|
||||
- compared to a cluster that spans availability zones, it is easier to reason about the availability properties of a
|
||||
single-zone cluster.
|
||||
- when the Kubernetes developers are designing the system (e.g. making assumptions about latency, bandwidth, or
|
||||
correlated failures) they are assuming all the machines are in a single data center, or otherwise closely connected.
|
||||
|
||||
It is recommended to run fewer clusters with more VMs per availability zone; but it is possible to run multiple clusters per availability zones.
|
||||
|
||||
Reasons to prefer fewer clusters per availability zone are:
|
||||
|
||||
- improved bin packing of Pods in some cases with more nodes in one cluster (less resource fragmentation).
|
||||
- reduced operational overhead (though the advantage is diminished as ops tooling and processes mature).
|
||||
- reduced costs for per-cluster fixed resource costs, for example apiserver VMs (but small as a percentage
|
||||
of overall cluster cost for medium to large clusters).
|
||||
|
||||
Reasons to have multiple clusters include:
|
||||
|
||||
- strict security policies requiring isolation of one class of work from another (but, see Partitioning Clusters
|
||||
below).
|
||||
- test clusters to canary new Kubernetes releases or other cluster software.
|
||||
|
||||
## Selecting the right number of clusters
|
||||
|
||||
The selection of the number of Kubernetes clusters may be a relatively static choice, only revisited occasionally.
|
||||
By contrast, the number of nodes in a cluster and the number of pods in a service may change frequently according to
|
||||
load and growth.
|
||||
|
||||
To pick the number of clusters, first, decide which regions you need to be in to have adequate latency to all your end users, for services that will run
|
||||
on Kubernetes (if you use a Content Distribution Network, the latency requirements for the CDN-hosted content need not
|
||||
be considered). Legal issues might influence this as well. For example, a company with a global customer base might decide to have clusters in US, EU, AP, and SA regions.
|
||||
Call the number of regions to be in `R`.
|
||||
|
||||
Second, decide how many clusters should be able to be unavailable at the same time, while still being available. Call
|
||||
the number that can be unavailable `U`. If you are not sure, then 1 is a fine choice.
|
||||
|
||||
If it is allowable for load-balancing to direct traffic to any region in the event of a cluster failure, then
|
||||
you need at least the larger of `R` or `U + 1` clusters. If it is not (e.g. you want to ensure low latency for all
|
||||
users in the event of a cluster failure), then you need to have `R * (U + 1)` clusters
|
||||
(`U + 1` in each of `R` regions). In any case, try to put each cluster in a different zone.
|
||||
|
||||
Finally, if any of your clusters would need more than the maximum recommended number of nodes for a Kubernetes cluster, then
|
||||
you may need even more clusters. Kubernetes v1.3 supports clusters up to 1000 nodes in size. Kubernetes v1.8 supports
|
||||
clusters up to 5000 nodes. See [Building Large Clusters](/docs/setup/best-practices/cluster-large/) for more guidance.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture whatsnext %}}
|
||||
* Learn more about the [Federation
|
||||
proposal](https://github.com/kubernetes/community/blob/{{< param "githubbranch" >}}/contributors/design-proposals/multicluster/federation.md).
|
||||
* See this [setup guide](/docs/tutorials/federation/set-up-cluster-federation-kubefed/) for cluster federation.
|
||||
* See this [Kubecon2016 talk on federation](https://www.youtube.com/watch?v=pq9lbkmxpS8)
|
||||
* See this [Kubecon2017 Europe update on federation](https://www.youtube.com/watch?v=kwOvOLnFYck)
|
||||
* See this [Kubecon2018 Europe update on sig-multicluster](https://www.youtube.com/watch?v=vGZo5DaThQU)
|
||||
* See this [Kubecon2018 Europe Federation-v2 prototype presentation](https://youtu.be/q27rbaX5Jis?t=7m20s)
|
||||
* See this [Federation-v2 Userguide](https://github.com/kubernetes-sigs/federation-v2/blob/master/docs/userguide.md)
|
||||
{{% /capture %}}
|
||||
@@ -424,16 +424,16 @@ At some point, you'll eventually need to update your deployed application, typic
|
||||
|
||||
We'll guide you through how to create and update applications with Deployments.
|
||||
|
||||
Let's say you were running version 1.7.9 of nginx:
|
||||
Let's say you were running version 1.14.2 of nginx:
|
||||
|
||||
```shell
|
||||
kubectl run my-nginx --image=nginx:1.7.9 --replicas=3
|
||||
kubectl run my-nginx --image=nginx:1.14.2 --replicas=3
|
||||
```
|
||||
```shell
|
||||
deployment.apps/my-nginx created
|
||||
```
|
||||
|
||||
To update to version 1.9.1, simply change `.spec.template.spec.containers[0].image` from `nginx:1.7.9` to `nginx:1.9.1`, with the kubectl commands we learned above.
|
||||
To update to version 1.16.1, simply change `.spec.template.spec.containers[0].image` from `nginx:1.14.2` to `nginx:1.16.1`, with the kubectl commands we learned above.
|
||||
|
||||
```shell
|
||||
kubectl edit deployment/my-nginx
|
||||
|
||||
@@ -111,9 +111,10 @@ For example, `example.com.node-restriction.kubernetes.io/fips=true` or `example.
|
||||
`nodeSelector` provides a very simple way to constrain pods to nodes with particular labels. The affinity/anti-affinity
|
||||
feature, greatly expands the types of constraints you can express. The key enhancements are
|
||||
|
||||
1. the language is more expressive (not just "AND or exact match")
|
||||
1. The affinity/anti-affinity language is more expressive. The language offers more matching rules
|
||||
besides exact matches created with a logical AND operation;
|
||||
2. you can indicate that the rule is "soft"/"preference" rather than a hard requirement, so if the scheduler
|
||||
can't satisfy it, the pod will still be scheduled
|
||||
can't satisfy it, the pod will still be scheduled;
|
||||
3. you can constrain against labels on other pods running on the node (or other topological domain),
|
||||
rather than against labels on the node itself, which allows rules about which pods can and cannot be co-located
|
||||
|
||||
@@ -159,9 +160,9 @@ You can use `NotIn` and `DoesNotExist` to achieve node anti-affinity behavior, o
|
||||
If you specify both `nodeSelector` and `nodeAffinity`, *both* must be satisfied for the pod
|
||||
to be scheduled onto a candidate node.
|
||||
|
||||
If you specify multiple `nodeSelectorTerms` associated with `nodeAffinity` types, then the pod can be scheduled onto a node **if one of** the `nodeSelectorTerms` is satisfied.
|
||||
If you specify multiple `nodeSelectorTerms` associated with `nodeAffinity` types, then the pod can be scheduled onto a node **only if all** `nodeSelectorTerms` can be satisfied.
|
||||
|
||||
If you specify multiple `matchExpressions` associated with `nodeSelectorTerms`, then the pod can be scheduled onto a node **only if all** `matchExpressions` can be satisfied.
|
||||
If you specify multiple `matchExpressions` associated with `nodeSelectorTerms`, then the pod can be scheduled onto a node **if one of** the `matchExpressions` is satisfied.
|
||||
|
||||
If you remove or change the label of the node where the pod is scheduled, the pod won't be removed. In other words, the affinity selection works only at the time of scheduling the pod.
|
||||
|
||||
@@ -228,7 +229,7 @@ for performance and security reasons, there are some constraints on topologyKey:
|
||||
1. For affinity and for `requiredDuringSchedulingIgnoredDuringExecution` pod anti-affinity,
|
||||
empty `topologyKey` is not allowed.
|
||||
2. For `requiredDuringSchedulingIgnoredDuringExecution` pod anti-affinity, the admission controller `LimitPodHardAntiAffinityTopology` was introduced to limit `topologyKey` to `kubernetes.io/hostname`. If you want to make it available for custom topologies, you may modify the admission controller, or simply disable it.
|
||||
3. For `preferredDuringSchedulingIgnoredDuringExecution` pod anti-affinity, empty `topologyKey` is interpreted as "all topologies" ("all topologies" here is now limited to the combination of `kubernetes.io/hostname`, `failure-domain.beta.kubernetes.io/zone` and `failure-domain.beta.kubernetes.io/region`).
|
||||
3. For `preferredDuringSchedulingIgnoredDuringExecution` pod anti-affinity, empty `topologyKey` is not allowed.
|
||||
4. Except for the above cases, the `topologyKey` can be any legal label-key.
|
||||
|
||||
In addition to `labelSelector` and `topologyKey`, you can optionally specify a list `namespaces`
|
||||
@@ -318,7 +319,7 @@ spec:
|
||||
topologyKey: "kubernetes.io/hostname"
|
||||
containers:
|
||||
- name: web-app
|
||||
image: nginx:1.12-alpine
|
||||
image: nginx:1.16-alpine
|
||||
```
|
||||
|
||||
If we create the above two deployments, our three node cluster should look like below.
|
||||
|
||||
@@ -392,7 +392,7 @@ spec:
|
||||
### How Pods with ephemeral-storage requests are scheduled
|
||||
|
||||
When you create a Pod, the Kubernetes scheduler selects a node for the Pod to
|
||||
run on. Each node has a maximum amount of local ephemeral storage it can provide for Pods. For more information, see ["Node Allocatable"](/docs/tasks/administer-cluster/reserve-compute-resources/#node-allocatable).
|
||||
run on. Each node has a maximum amount of local ephemeral storage it can provide for Pods. For more information, see ["Node Allocatable"](/docs/tasks/administer-cluster/reserve-compute-resources/#node-allocatable).
|
||||
|
||||
The scheduler ensures that the sum of the resource requests of the scheduled Containers is less than the capacity of the node.
|
||||
|
||||
|
||||
@@ -16,42 +16,25 @@ importance of a Pod relative to other Pods. If a Pod cannot be scheduled, the
|
||||
scheduler tries to preempt (evict) lower priority Pods to make scheduling of the
|
||||
pending Pod possible.
|
||||
|
||||
In Kubernetes 1.9 and later, Priority also affects scheduling order of Pods and
|
||||
out-of-resource eviction ordering on the Node.
|
||||
|
||||
Pod priority and preemption graduated to beta in Kubernetes 1.11 and to GA in
|
||||
Kubernetes 1.14. They have been enabled by default since 1.11.
|
||||
|
||||
In Kubernetes versions where Pod priority and preemption is still an alpha-level
|
||||
feature, you need to explicitly enable it. To use these features in the older
|
||||
versions of Kubernetes, follow the instructions in the documentation for your
|
||||
Kubernetes version, by going to the documentation archive version for your
|
||||
Kubernetes version.
|
||||
|
||||
Kubernetes Version | Priority and Preemption State | Enabled by default
|
||||
------------------ | :---------------------------: | :----------------:
|
||||
1.8 | alpha | no
|
||||
1.9 | alpha | no
|
||||
1.10 | alpha | no
|
||||
1.11 | beta | yes
|
||||
1.14 | stable | yes
|
||||
|
||||
{{< warning >}}In a cluster where not all users are trusted, a
|
||||
malicious user could create pods at the highest possible priorities, causing
|
||||
other pods to be evicted/not get scheduled. To resolve this issue,
|
||||
[ResourceQuota](/docs/concepts/policy/resource-quotas/) is
|
||||
augmented to support Pod priority. An admin can create ResourceQuota for users
|
||||
at specific priority levels, preventing them from creating pods at high
|
||||
priorities. This feature is in beta since Kubernetes 1.12.
|
||||
{{< /warning >}}
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture body %}}
|
||||
|
||||
|
||||
{{< warning >}}
|
||||
In a cluster where not all users are trusted, a malicious user could create Pods
|
||||
at the highest possible priorities, causing other Pods to be evicted/not get
|
||||
scheduled.
|
||||
An administrator can use ResourceQuota to prevent users from creating pods at
|
||||
high priorities.
|
||||
|
||||
See [limit Priority Class consumption by default](/docs/concepts/policy/resource-quotas/#limit-priority-class-consumption-by-default)
|
||||
for details.
|
||||
{{< /warning >}}
|
||||
|
||||
## How to use priority and preemption
|
||||
|
||||
To use priority and preemption in Kubernetes 1.11 and later, follow these steps:
|
||||
To use priority and preemption:
|
||||
|
||||
1. Add one or more [PriorityClasses](#priorityclass).
|
||||
|
||||
@@ -77,21 +60,20 @@ Pods.
|
||||
|
||||
## How to disable preemption
|
||||
|
||||
{{< note >}}
|
||||
In Kubernetes 1.12+, critical pods rely on scheduler preemption to be scheduled
|
||||
when a cluster is under resource pressure. For this reason, it is not
|
||||
recommended to disable preemption.
|
||||
{{< /note >}}
|
||||
{{< caution >}}
|
||||
Critical pods rely on scheduler preemption to be scheduled when a cluster
|
||||
is under resource pressure. For this reason, it is not recommended to
|
||||
disable preemption.
|
||||
{{< /caution >}}
|
||||
|
||||
{{< note >}}
|
||||
In Kubernetes 1.15 and later,
|
||||
if the feature `NonPreemptingPriority` is enabled,
|
||||
In Kubernetes 1.15 and later, if the feature `NonPreemptingPriority` is enabled,
|
||||
PriorityClasses have the option to set `preemptionPolicy: Never`.
|
||||
This will prevent pods of that PriorityClass from preempting other pods.
|
||||
{{< /note >}}
|
||||
|
||||
In Kubernetes 1.11 and later, preemption is controlled by a kube-scheduler flag
|
||||
`disablePreemption`, which is set to `false` by default.
|
||||
Preemption is controlled by a kube-scheduler flag `disablePreemption`, which is
|
||||
set to `false` by default.
|
||||
If you want to disable preemption despite the above note, you can set
|
||||
`disablePreemption` to `true`.
|
||||
|
||||
@@ -117,6 +99,9 @@ priority class name to the integer value of the priority. The name is specified
|
||||
in the `name` field of the PriorityClass object's metadata. The value is
|
||||
specified in the required `value` field. The higher the value, the higher the
|
||||
priority.
|
||||
The name of a PriorityClass object must be a valid
|
||||
[DNS subdomain name](/docs/concepts/overview/working-with-objects/names#dns-subdomain-names),
|
||||
and it cannot be prefixed with `system-`.
|
||||
|
||||
A PriorityClass object can have any 32-bit integer value smaller than or equal
|
||||
to 1 billion. Larger numbers are reserved for critical system Pods that should
|
||||
@@ -237,12 +222,12 @@ spec:
|
||||
|
||||
### Effect of Pod priority on scheduling order
|
||||
|
||||
In Kubernetes 1.9 and later, when Pod priority is enabled, scheduler orders
|
||||
pending Pods by their priority and a pending Pod is placed ahead of other
|
||||
pending Pods with lower priority in the scheduling queue. As a result, the
|
||||
higher priority Pod may be scheduled sooner than Pods with lower priority if its
|
||||
scheduling requirements are met. If such Pod cannot be scheduled, scheduler will
|
||||
continue and tries to schedule other lower priority Pods.
|
||||
When Pod priority is enabled, the scheduler orders pending Pods by
|
||||
their priority and a pending Pod is placed ahead of other pending Pods
|
||||
with lower priority in the scheduling queue. As a result, the higher
|
||||
priority Pod may be scheduled sooner than Pods with lower priority if
|
||||
its scheduling requirements are met. If such Pod cannot be scheduled,
|
||||
scheduler will continue and tries to schedule other lower priority Pods.
|
||||
|
||||
## Preemption
|
||||
|
||||
@@ -288,12 +273,12 @@ point that scheduler preempts victims and the time that Pod P is scheduled. In
|
||||
order to minimize this gap, one can set graceful termination period of lower
|
||||
priority Pods to zero or a small number.
|
||||
|
||||
#### PodDisruptionBudget is supported, but not guaranteed!
|
||||
#### PodDisruptionBudget is supported, but not guaranteed
|
||||
|
||||
A [Pod Disruption Budget (PDB)](/docs/concepts/workloads/pods/disruptions/)
|
||||
allows application owners to limit the number of Pods of a replicated application
|
||||
that are down simultaneously from voluntary disruptions. Kubernetes 1.9 supports
|
||||
PDB when preempting Pods, but respecting PDB is best effort. The Scheduler tries
|
||||
that are down simultaneously from voluntary disruptions. Kubernetes supports
|
||||
PDB when preempting Pods, but respecting PDB is best effort. The scheduler tries
|
||||
to find victims whose PDB are not violated by preemption, but if no such victims
|
||||
are found, preemption will still happen, and lower priority Pods will be removed
|
||||
despite their PDBs being violated.
|
||||
@@ -344,28 +329,23 @@ gone, and Pod P could possibly be scheduled on Node N.
|
||||
We may consider adding cross Node preemption in future versions if there is
|
||||
enough demand and if we find an algorithm with reasonable performance.
|
||||
|
||||
## Debugging Pod Priority and Preemption
|
||||
## Troubleshooting
|
||||
|
||||
Pod Priority and Preemption is a major feature that could potentially disrupt
|
||||
Pod scheduling if it has bugs.
|
||||
Pod priority and pre-emption can have unwanted side effects. Here are some
|
||||
examples of potential problems and ways to deal with them.
|
||||
|
||||
### Potential problems caused by Priority and Preemption
|
||||
|
||||
The followings are some of the potential problems that could be caused by bugs
|
||||
in the implementation of the feature. This list is not exhaustive.
|
||||
|
||||
#### Pods are preempted unnecessarily
|
||||
### Pods are preempted unnecessarily
|
||||
|
||||
Preemption removes existing Pods from a cluster under resource pressure to make
|
||||
room for higher priority pending Pods. If a user gives high priorities to
|
||||
certain Pods by mistake, these unintentional high priority Pods may cause
|
||||
preemption in the cluster. As mentioned above, Pod priority is specified by
|
||||
setting the `priorityClassName` field of `podSpec`. The integer value of
|
||||
room for higher priority pending Pods. If you give high priorities to
|
||||
certain Pods by mistake, these unintentionally high priority Pods may cause
|
||||
preemption in your cluster. Pod priority is specified by setting the
|
||||
`priorityClassName` field in the Pod's specification. The integer value for
|
||||
priority is then resolved and populated to the `priority` field of `podSpec`.
|
||||
|
||||
To resolve the problem, `priorityClassName` of the Pods must be changed to use
|
||||
lower priority classes or should be left empty. Empty `priorityClassName` is
|
||||
resolved to zero by default.
|
||||
To address the problem, you can change the `priorityClassName` for those Pods
|
||||
to use lower priority classes, or leave that field empty. An empty
|
||||
`priorityClassName` is resolved to zero by default.
|
||||
|
||||
When a Pod is preempted, there will be events recorded for the preempted Pod.
|
||||
Preemption should happen only when a cluster does not have enough resources for
|
||||
@@ -374,29 +354,31 @@ Pod (preemptor) is higher than the victim Pods. Preemption must not happen when
|
||||
there is no pending Pod, or when the pending Pods have equal or lower priority
|
||||
than the victims. If preemption happens in such scenarios, please file an issue.
|
||||
|
||||
#### Pods are preempted, but the preemptor is not scheduled
|
||||
### Pods are preempted, but the preemptor is not scheduled
|
||||
|
||||
When pods are preempted, they receive their requested graceful termination
|
||||
period, which is by default 30 seconds, but it can be any different value as
|
||||
specified in the PodSpec. If the victim Pods do not terminate within this period,
|
||||
they are force-terminated. Once all the victims go away, the preemptor Pod can
|
||||
be scheduled.
|
||||
period, which is by default 30 seconds. If the victim Pods do not terminate within
|
||||
this period, they are forcibly terminated. Once all the victims go away, the
|
||||
preemptor Pod can be scheduled.
|
||||
|
||||
While the preemptor Pod is waiting for the victims to go away, a higher priority
|
||||
Pod may be created that fits on the same node. In this case, the scheduler will
|
||||
Pod may be created that fits on the same Node. In this case, the scheduler will
|
||||
schedule the higher priority Pod instead of the preemptor.
|
||||
|
||||
In the absence of such a higher priority Pod, we expect the preemptor Pod to be
|
||||
scheduled after the graceful termination period of the victims is over.
|
||||
This is expected behavior: the Pod with the higher priority should take the place
|
||||
of a Pod with a lower priority. Other controller actions, such as
|
||||
[cluster autoscaling](/docs/tasks/administer-cluster/cluster-management/#cluster-autoscaling),
|
||||
may eventually provide capacity to schedule the pending Pods.
|
||||
|
||||
#### Higher priority Pods are preempted before lower priority pods
|
||||
### Higher priority Pods are preempted before lower priority pods
|
||||
|
||||
The scheduler tries to find nodes that can run a pending Pod and if no node is
|
||||
found, it tries to remove Pods with lower priority from one node to make room
|
||||
for the pending pod. If a node with low priority Pods is not feasible to run the
|
||||
pending Pod, the scheduler may choose another node with higher priority Pods
|
||||
(compared to the Pods on the other node) for preemption. The victims must still
|
||||
have lower priority than the preemptor Pod.
|
||||
The scheduler tries to find nodes that can run a pending Pod. If no node is
|
||||
found, the scheduler tries to remove Pods with lower priority from an arbitrary
|
||||
node in order to make room for the pending pod.
|
||||
If a node with low priority Pods is not feasible to run the pending Pod, the scheduler
|
||||
may choose another node with higher priority Pods (compared to the Pods on the
|
||||
other node) for preemption. The victims must still have lower priority than the
|
||||
preemptor Pod.
|
||||
|
||||
When there are multiple nodes available for preemption, the scheduler tries to
|
||||
choose the node with a set of Pods with lowest priority. However, if such Pods
|
||||
@@ -404,13 +386,11 @@ have PodDisruptionBudget that would be violated if they are preempted then the
|
||||
scheduler may choose another node with higher priority Pods.
|
||||
|
||||
When multiple nodes exist for preemption and none of the above scenarios apply,
|
||||
we expect the scheduler to choose a node with the lowest priority. If that is
|
||||
not the case, it may indicate a bug in the scheduler.
|
||||
the scheduler chooses a node with the lowest priority.
|
||||
|
||||
## Interactions of Pod priority and QoS
|
||||
## Interactions between Pod priority and quality of service {#interactions-of-pod-priority-and-qos}
|
||||
|
||||
Pod priority and
|
||||
[QoS](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/node/resource-qos.md)
|
||||
Pod priority and {{< glossary_tooltip text="QoS class" term_id="qos-class" >}}
|
||||
are two orthogonal features with few interactions and no default restrictions on
|
||||
setting the priority of a Pod based on its QoS classes. The scheduler's
|
||||
preemption logic does not consider QoS when choosing preemption targets.
|
||||
@@ -421,15 +401,20 @@ to schedule the preemptor Pod, or if the lowest priority Pods are protected by
|
||||
`PodDisruptionBudget`.
|
||||
|
||||
The only component that considers both QoS and Pod priority is
|
||||
[Kubelet out-of-resource eviction](/docs/tasks/administer-cluster/out-of-resource/).
|
||||
[kubelet out-of-resource eviction](/docs/tasks/administer-cluster/out-of-resource/).
|
||||
The kubelet ranks Pods for eviction first by whether or not their usage of the
|
||||
starved resource exceeds requests, then by Priority, and then by the consumption
|
||||
of the starved compute resource relative to the Pods’ scheduling requests.
|
||||
See
|
||||
[Evicting end-user pods](/docs/tasks/administer-cluster/out-of-resource/#evicting-end-user-pods)
|
||||
for more details. Kubelet out-of-resource eviction does not evict Pods whose
|
||||
[evicting end-user pods](/docs/tasks/administer-cluster/out-of-resource/#evicting-end-user-pods)
|
||||
for more details.
|
||||
|
||||
kubelet out-of-resource eviction does not evict Pods wheir their
|
||||
usage does not exceed their requests. If a Pod with lower priority is not
|
||||
exceeding its requests, it won't be evicted. Another Pod with higher priority
|
||||
that exceeds its requests may be evicted.
|
||||
|
||||
{{% /capture %}}
|
||||
{{% capture whatsnext %}}
|
||||
* Read about using ResourceQuotas in connection with PriorityClasses: [limit Priority Class consumption by default](/docs/concepts/policy/resource-quotas/#limit-priority-class-consumption-by-default)
|
||||
{{% /capture %}}
|
||||
|
||||
+1
-1
@@ -2,7 +2,7 @@
|
||||
reviewers:
|
||||
- mikedanese
|
||||
- thockin
|
||||
title: Container Environment Variables
|
||||
title: Container Environment
|
||||
content_template: templates/concept
|
||||
weight: 20
|
||||
---
|
||||
@@ -116,7 +116,7 @@ Events:
|
||||
|
||||
{{% capture whatsnext %}}
|
||||
|
||||
* Learn more about the [Container environment](/docs/concepts/containers/container-environment-variables/).
|
||||
* Learn more about the [Container environment](/docs/concepts/containers/container-environment/).
|
||||
* Get hands-on experience
|
||||
[attaching handlers to Container lifecycle events](/docs/tasks/configure-pod-container/attach-handler-lifecycle-event/).
|
||||
|
||||
|
||||
@@ -0,0 +1,45 @@
|
||||
---
|
||||
reviewers:
|
||||
- erictune
|
||||
- thockin
|
||||
title: Containers overview
|
||||
content_template: templates/concept
|
||||
weight: 1
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
Containers are a technnology for packaging the (compiled) code for an
|
||||
application along with the dependencies it needs at run time. Each
|
||||
container that you run is repeatable; the standardisation from having
|
||||
dependencies included means that you get the same behavior wherever you
|
||||
run it.
|
||||
|
||||
Containers decouple applications from underlying host infrastructure.
|
||||
This makes deployment easier in different cloud or OS environments.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
|
||||
{{% capture body %}}
|
||||
|
||||
## Container images
|
||||
A [container image](/docs/concepts/containers/images/) is a ready-to-run
|
||||
software package, containing everything needed to run an application:
|
||||
the code and any runtime it requires, application and system libraries,
|
||||
and default values for any essential settings.
|
||||
|
||||
By design, a container is immutable: you cannot change the code of a
|
||||
container that is already running. If you have a containerized application
|
||||
and want to make changes, you need to build a new container that includes
|
||||
the change, then recreate the container to start from the updated image.
|
||||
|
||||
## Container runtimes
|
||||
|
||||
{{< glossary_definition term_id="container-runtime" length="all" >}}
|
||||
|
||||
{{% /capture %}}
|
||||
{{% capture whatsnext %}}
|
||||
* Read about [container images](/docs/concepts/containers/images/)
|
||||
* Read about [Pods](/docs/concepts/workloads/pods/)
|
||||
{{% /capture %}}
|
||||
@@ -82,6 +82,9 @@ metadata:
|
||||
handler: myconfiguration # The name of the corresponding CRI configuration
|
||||
```
|
||||
|
||||
The name of a RuntimeClass object must be a valid
|
||||
[DNS subdomain name](/docs/concepts/overview/working-with-objects/names#dns-subdomain-names).
|
||||
|
||||
{{< note >}}
|
||||
It is recommended that RuntimeClass write operations (create/update/patch/delete) be
|
||||
restricted to the cluster administrator. This is typically the default. See [Authorization
|
||||
|
||||
@@ -5,30 +5,34 @@ reviewers:
|
||||
- cheftako
|
||||
- chenopis
|
||||
content_template: templates/concept
|
||||
weight: 10
|
||||
weight: 20
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
The aggregation layer allows Kubernetes to be extended with additional APIs, beyond what is offered by the core Kubernetes APIs.
|
||||
The aggregation layer allows Kubernetes to be extended with additional APIs, beyond what is offered by the core Kubernetes APIs.
|
||||
The additional APIs can either be ready-made solutions such as [service-catalog](/docs/concepts/extend-kubernetes/service-catalog/), or APIs that you develop yourself.
|
||||
|
||||
The aggregation layer is different from [Custom Resources](/docs/concepts/extend-kubernetes/api-extension/custom-resources/), which are a way to make the {{< glossary_tooltip term_id="kube-apiserver" text="kube-apiserver" >}} recognise new kinds of object.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture body %}}
|
||||
|
||||
## Overview
|
||||
## Aggregation layer
|
||||
|
||||
The aggregation layer enables installing additional Kubernetes-style APIs in your cluster. These can either be pre-built, existing 3rd party solutions, such as [service-catalog](https://github.com/kubernetes-incubator/service-catalog/blob/master/README.md), or user-created APIs like [apiserver-builder](https://github.com/kubernetes-incubator/apiserver-builder/blob/master/README.md), which can get you started.
|
||||
The aggregation layer runs in-process with the kube-apiserver. Until an extension resource is registered, the aggregation layer will do nothing. To register an API, you add an _APIService_ object, which "claims" the URL path in the Kubernetes API. At that point, the aggregation layer will proxy anything sent to that API path (e.g. `/apis/myextension.mycompany.io/v1/…`) to the registered APIService.
|
||||
|
||||
The aggregation layer runs in-process with the kube-apiserver. Until an extension resource is registered, the aggregation layer will do nothing. To register an API, users must add an APIService object, which "claims" the URL path in the Kubernetes API. At that point, the aggregation layer will proxy anything sent to that API path (e.g. /apis/myextension.mycompany.io/v1/…) to the registered APIService.
|
||||
The most common way to implement the APIService is to run an *extension API server* in Pod(s) that run in your cluster. If you're using the extension API server to manage resources in your cluster, the extension API server (also written as "extension-apiserver") is typically paired with one or more {{< glossary_tooltip text="controllers" term_id="controller" >}}. The apiserver-builder library provides a skeleton for both extension API servers and the associated controller(s).
|
||||
|
||||
Ordinarily, the APIService will be implemented by an *extension-apiserver* in a pod running in the cluster. This extension-apiserver will normally need to be paired with one or more controllers if active management of the added resources is needed. As a result, the apiserver-builder will actually provide a skeleton for both. As another example, when the service-catalog is installed, it provides both the extension-apiserver and controller for the services it provides.
|
||||
### Response latency
|
||||
|
||||
Extension-apiservers should have low latency connections to and from the kube-apiserver.
|
||||
In particular, discovery requests are required to round-trip from the kube-apiserver in five seconds or less.
|
||||
If your deployment cannot achieve this, you should consider how to change it. For now, setting the
|
||||
`EnableAggregatedDiscoveryTimeout=false` feature gate on the kube-apiserver
|
||||
will disable the timeout restriction. It will be removed in a future release.
|
||||
Extension API servers should have low latency networking to and from the kube-apiserver.
|
||||
Discovery requests are required to round-trip from the kube-apiserver in five seconds or less.
|
||||
|
||||
If your extension API server cannot achieve that latency requirement, consider making changes that let you meet it. You can also set the
|
||||
`EnableAggregatedDiscoveryTimeout=false` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/) on the kube-apiserver
|
||||
to disable the timeout restriction. This deprecated feature gate will be removed in a future release.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
@@ -37,7 +41,6 @@ will disable the timeout restriction. It will be removed in a future release.
|
||||
* To get the aggregator working in your environment, [configure the aggregation layer](/docs/tasks/access-kubernetes-api/configure-aggregation-layer/).
|
||||
* Then, [setup an extension api-server](/docs/tasks/access-kubernetes-api/setup-extension-api-server/) to work with the aggregation layer.
|
||||
* Also, learn how to [extend the Kubernetes API using Custom Resource Definitions](/docs/tasks/access-kubernetes-api/extend-api-custom-resource-definitions/).
|
||||
* Read the specification for [APIService](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#apiservice-v1-apiregistration-k8s-io)
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@ reviewers:
|
||||
- enisoc
|
||||
- deads2k
|
||||
content_template: templates/concept
|
||||
weight: 20
|
||||
weight: 10
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
@@ -167,7 +167,7 @@ CRDs are easier to create than Aggregated APIs.
|
||||
|
||||
| CRDs | Aggregated API |
|
||||
| --------------------------- | -------------- |
|
||||
| Do not require programming. Users can choose any language for a CRD controller. | Requires programming in Go and building binary and image. Users can choose any language for a CRD controller. |
|
||||
| Do not require programming. Users can choose any language for a CRD controller. | Requires programming in Go and building binary and image. |
|
||||
| No additional service to run; CRs are handled by API Server. | An additional service to create and that could fail. |
|
||||
| No ongoing support once the CRD is created. Any bug fixes are picked up as part of normal Kubernetes Master upgrades. | May need to periodically pickup bug fixes from upstream and rebuild and update the Aggregated APIserver. |
|
||||
| No need to handle multiple versions of your API. For example: when you control the client for this resource, you can upgrade it in sync with the API. | You need to handle multiple versions of your API, for example: when developing an extension to share with the world. |
|
||||
|
||||
+55
-61
@@ -1,117 +1,111 @@
|
||||
---
|
||||
title: Poseidon-Firmament - An alternate scheduler
|
||||
title: Poseidon-Firmament Scheduler
|
||||
content_template: templates/concept
|
||||
weight: 80
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
**Current release of Poseidon-Firmament scheduler is an <code> alpha </code> release.**
|
||||
{{< feature-state for_k8s_version="v1.6" state="alpha" >}}
|
||||
|
||||
Poseidon-Firmament scheduler is an alternate scheduler that can be deployed alongside the default Kubernetes scheduler.
|
||||
The Poseidon-Firmament scheduler is an alternate scheduler that can be deployed alongside the default Kubernetes scheduler.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture body %}}
|
||||
|
||||
|
||||
## Introduction
|
||||
## Introduction
|
||||
|
||||
Poseidon is a service that acts as the integration glue for the [Firmament scheduler](https://github.com/Huawei-PaaS/firmament) with Kubernetes. Poseidon-Firmament scheduler augments the current Kubernetes scheduling capabilities. It incorporates novel flow network graph based scheduling capabilities alongside the default Kubernetes Scheduler. Firmament scheduler models workloads and clusters as flow networks and runs min-cost flow optimizations over these networks to make scheduling decisions.
|
||||
Poseidon is a service that acts as the integration glue between the [Firmament scheduler](https://github.com/Huawei-PaaS/firmament) and Kubernetes. Poseidon-Firmament augments the current Kubernetes scheduling capabilities. It incorporates novel flow network graph based scheduling capabilities alongside the default Kubernetes scheduler. The Firmament scheduler models workloads and clusters as flow networks and runs min-cost flow optimizations over these networks to make scheduling decisions.
|
||||
|
||||
It models the scheduling problem as a constraint-based optimization over a flow network graph. This is achieved by reducing scheduling to a min-cost max-flow optimization problem. The Poseidon-Firmament scheduler dynamically refines the workload placements.
|
||||
Firmament models the scheduling problem as a constraint-based optimization over a flow network graph. This is achieved by reducing scheduling to a min-cost max-flow optimization problem. The Poseidon-Firmament scheduler dynamically refines the workload placements.
|
||||
|
||||
Poseidon-Firmament scheduler runs alongside the default Kubernetes Scheduler as an alternate scheduler, so multiple schedulers run simultaneously.
|
||||
Poseidon-Firmament scheduler runs alongside the default Kubernetes scheduler as an alternate scheduler. You can simultaneously run multiple, different schedulers.
|
||||
|
||||
## Key Advantages
|
||||
Flow graph scheduling with the Poseidon-Firmament scheduler provides the following advantages:
|
||||
|
||||
### Flow graph scheduling based Poseidon-Firmament scheduler provides the following key advantages:
|
||||
- Workloads (pods) are bulk scheduled to enable scheduling at massive scale..
|
||||
- Based on the extensive performance test results, Poseidon-Firmament scales much better than the Kubernetes default scheduler as the number of nodes increase in a cluster. This is due to the fact that Poseidon-Firmament is able to amortize more and more work across workloads.
|
||||
- Poseidon-Firmament Scheduler outperforms the Kubernetes default scheduler by a wide margin when it comes to throughput performance numbers for scenarios where compute resource requirements are somewhat uniform across jobs (Replicasets/Deployments/Jobs). Poseidon-Firmament scheduler end-to-end throughput performance numbers, including bind time, consistently get better as the number of nodes in a cluster increase. For example, for a 2,700 node cluster (shown in the graphs [here](https://github.com/kubernetes-sigs/poseidon/blob/master/docs/benchmark/README.md)), Poseidon-Firmament scheduler achieves a 7X or greater end-to-end throughput than the Kubernetes default scheduler, which includes bind time.
|
||||
- Workloads (Pods) are bulk scheduled to enable scheduling at massive scale.
|
||||
The Poseidon-Firmament scheduler outperforms the Kubernetes default scheduler by a wide margin when it comes to throughput performance for scenarios where compute resource requirements are somewhat uniform across your workload (Deployments, ReplicaSets, Jobs).
|
||||
- The Poseidon-Firmament's scheduler's end-to-end throughput performance and bind time improves as the number of nodes in a cluster increases. As you scale out, Poseidon-Firmament scheduler is able to amortize more and more work across workloads.
|
||||
- Scheduling in Poseidon-Firmament is dynamic; it keeps cluster resources in a global optimal state during every scheduling run.
|
||||
- The Poseidon-Firmament scheduler supports scheduling complex rule constraints.
|
||||
|
||||
- Availability of complex rule constraints.
|
||||
- Scheduling in Poseidon-Firmament is dynamic; it keeps cluster resources in a global optimal state during every scheduling run.
|
||||
- Highly efficient resource utilizations.
|
||||
## How the Poseidon-Firmament scheduler works
|
||||
|
||||
## Poseidon-Firmament Scheduler - How it works
|
||||
Kubernetes supports [using multiple schedulers](/docs/tasks/administer-cluster/configure-multiple-schedulers/). You can specify, for a particular Pod, that it is scheduled by a custom scheduler (“poseidon” for this case), by setting the `schedulerName` field in the PodSpec at the time of pod creation. The default scheduler will ignore that Pod and allow Poseidon-Firmament scheduler to schedule the Pod on a relevant node.
|
||||
|
||||
As part of the Kubernetes multiple schedulers support, each new pod is typically scheduled by the default scheduler. Kubernetes can be instructed to use another scheduler by specifying the name of another custom scheduler (“poseidon” in our case) in the **schedulerName** field of the PodSpec at the time of pod creation. In this case, the default scheduler will ignore that Pod and allow Poseidon scheduler to schedule the Pod on a relevant node.
|
||||
For example:
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
|
||||
...
|
||||
spec:
|
||||
schedulerName: poseidon
|
||||
```
|
||||
schedulerName: poseidon
|
||||
...
|
||||
```
|
||||
|
||||
|
||||
{{< note >}}
|
||||
For details about the design of this project see the [design document](https://github.com/kubernetes-sigs/poseidon/blob/master/docs/design/README.md).
|
||||
{{< /note >}}
|
||||
|
||||
## Possible Use Case Scenarios - When to use it
|
||||
## Batch scheduling
|
||||
|
||||
As mentioned earlier, Poseidon-Firmament scheduler enables an extremely high throughput scheduling environment at scale due to its bulk scheduling approach versus Kubernetes pod-at-a-time approach. In our extensive tests, we have observed substantial throughput benefits as long as resource requirements (CPU/Memory) for incoming Pods are uniform across jobs (Replicasets/Deployments/Jobs), mainly due to efficient amortization of work across jobs.
|
||||
|
||||
Although, Poseidon-Firmament scheduler is capable of scheduling various types of workloads, such as service, batch, etc., the following are a few use cases where it excels the most:
|
||||
|
||||
1. For “Big Data/AI” jobs consisting of large number of tasks, throughput benefits are tremendous.
|
||||
2. Service or batch jobs where workload resource requirements are uniform across jobs (Replicasets/Deployments/Jobs).
|
||||
1. For “Big Data/AI” jobs consisting of large number of tasks, throughput benefits are tremendous.
|
||||
2. Service or batch jobs where workload resource requirements are uniform across jobs (Replicasets/Deployments/Jobs).
|
||||
|
||||
## Current Project Stage
|
||||
## Feature state
|
||||
|
||||
- **Alpha Release - Incubation repo.** at https://github.com/kubernetes-sigs/poseidon.
|
||||
- Currently, Poseidon-Firmament scheduler **does not provide support for high availability**, our implementation assumes that the scheduler cannot fail. The [design document](https://github.com/kubernetes-sigs/poseidon/blob/master/docs/design/README.md) describes possible ways to enable high availability, but we leave this to future work.
|
||||
- We are **not aware of any production deployment** of Poseidon-Firmament scheduler at this time.
|
||||
- Poseidon-Firmament is supported from Kubernetes release 1.6 and works with all subsequent releases.
|
||||
- Release process for Poseidon and Firmament repos are in lock step. The current Poseidon release can be found [here](https://github.com/kubernetes-sigs/poseidon/releases) and the corresponding Firmament release can be found [here](https://github.com/Huawei-PaaS/firmament/releases).
|
||||
Poseidon-Firmament is designed to work with Kubernetes release 1.6 and all subsequent releases.
|
||||
|
||||
## Features Comparison Matrix
|
||||
{{< caution >}}
|
||||
Poseidon-Firmament scheduler does not provide support for high availability; its implementation assumes that the scheduler cannot fail.
|
||||
{{< /caution >}}
|
||||
|
||||
## Feature comparison {#feature-comparison-matrix}
|
||||
|
||||
{{< table caption="Feature comparison of Kubernetes and Poseidon-Firmament schedulers." >}}
|
||||
|Feature|Kubernetes Default Scheduler|Poseidon-Firmament Scheduler|Notes|
|
||||
|--- |--- |--- |--- |
|
||||
|Node Affinity/Anti-Affinity|Y|Y||
|
||||
|Pod Affinity/Anti-Affinity - including support for pod anti-affinity symmetry|Y|Y|Currently, the default scheduler outperforms the Poseidon-Firmament scheduler pod affinity/anti-affinity functionality. We are working towards resolving this.|
|
||||
|Pod Affinity/Anti-Affinity - including support for pod anti-affinity symmetry|Y|Y|The default scheduler outperforms the Poseidon-Firmament scheduler pod affinity/anti-affinity functionality.|
|
||||
|Taints & Tolerations|Y|Y||
|
||||
|Baseline Scheduling capability in accordance to available compute resources (CPU & Memory) on a node|Y|Y**|Not all Predicates & Priorities are supported at this time.|
|
||||
|Extreme Throughput at scale|Y**|Y|Bulk scheduling approach scales or increases workload placement. Substantial throughput benefits using Firmament scheduler as long as resource requirements (CPU/Memory) for incoming Pods is uniform across Replicasets/Deployments/Jobs. This is mainly due to efficient amortization of work across Replicasets/Deployments/Jobs . 1) For “Big Data/AI” jobs consisting of large no. of tasks, throughput benefits are tremendous. 2) Substantial throughput benefits also for service or batch job scenarios where workload resource requirements are uniform across Replicasets/Deployments/Jobs.|
|
||||
|Optimal Scheduling|Pod-by-Pod scheduler, processes one pod at a time (may result into sub-optimal scheduling)|Bulk Scheduling (Optimal scheduling)|Pod-by-Pod Kubernetes default scheduler may assign tasks to a sub-optimal machine. By contrast, Firmament considers all unscheduled tasks at the same time together with their soft and hard constraints.|
|
||||
|Colocation Interference Avoidance|N|N**|Planned in Poseidon-Firmament.|
|
||||
|Priority Pre-emption|Y|N**|Partially exists in Poseidon-Firmament versus extensive support in Kubernetes default scheduler.|
|
||||
|Inherent Re-Scheduling|N|Y**|Poseidon-Firmament scheduler supports workload re-scheduling. In each scheduling run it considers all the pods, including running pods, and as a result can migrate or evict pods – a globally optimal scheduling environment.|
|
||||
|Baseline Scheduling capability in accordance to available compute resources (CPU & Memory) on a node|Y|Y†|**†** Not all Predicates & Priorities are supported with Poseidon-Firmament.|
|
||||
|Extreme Throughput at scale|Y†|Y|**†** Bulk scheduling approach scales or increases workload placement. Firmament scheduler offers high throughput when resource requirements (CPU/Memory) for incoming Pods are uniform across ReplicaSets/Deployments/Jobs.|
|
||||
|Colocation Interference Avoidance|N|N||
|
||||
|Priority Preemption|Y|N†|**†** Partially exists in Poseidon-Firmament versus extensive support in Kubernetes default scheduler.|
|
||||
|Inherent Rescheduling|N|Y†|**†** Poseidon-Firmament scheduler supports workload re-scheduling. In each scheduling run, Poseidon-Firmament considers all Pods, including running Pods, and as a result can migrate or evict Pods – a globally optimal scheduling environment.|
|
||||
|Gang Scheduling|N|Y||
|
||||
|Support for Pre-bound Persistence Volume Scheduling|Y|Y||
|
||||
|Support for Local Volume & Dynamic Persistence Volume Binding Scheduling|Y|N**|Planned.|
|
||||
|High Availability|Y|N**|Planned.|
|
||||
|Real-time metrics based scheduling|N|Y**|Initially supported using Heapster (now deprecated) for placing pods using actual cluster utilization statistics rather than reservations. Plans to switch over to "metric server".|
|
||||
|Support for Local Volume & Dynamic Persistence Volume Binding Scheduling|Y|N||
|
||||
|High Availability|Y|N||
|
||||
|Real-time metrics based scheduling|N|Y†|**†** Partially supported in Poseidon-Firmament using Heapster (now deprecated) for placing Pods using actual cluster utilization statistics rather than reservations.|
|
||||
|Support for Max-Pod per node|Y|Y|Poseidon-Firmament scheduler seamlessly co-exists with Kubernetes default scheduler.|
|
||||
|Support for Ephemeral Storage, in addition to CPU/Memory|Y|Y||
|
||||
{{< /table >}}
|
||||
|
||||
## Installation
|
||||
|
||||
## Installation
|
||||
The [Poseidon-Firmament installation guide](https://github.com/kubernetes-sigs/poseidon/blob/master/docs/install/README.md#Installation) explains how to deploy Poseidon-Firmament to your cluster.
|
||||
|
||||
For in-cluster installation of Poseidon, please start at the [Installation instructions](https://github.com/kubernetes-sigs/poseidon/blob/master/docs/install/README.md).
|
||||
|
||||
|
||||
## Development
|
||||
|
||||
For developers, please refer to the [Developer Setup instructions](https://github.com/kubernetes-sigs/poseidon/blob/master/docs/devel/README.md).
|
||||
|
||||
## Latest Throughput Performance Testing Results
|
||||
|
||||
Pod-by-pod schedulers, such as the Kubernetes default scheduler, typically process one pod at a time. These schedulers have the following crucial drawbacks:
|
||||
|
||||
1. The scheduler commits to a pod placement early and restricts the choices for other pods that wait to be placed.
|
||||
2. There is limited opportunities for amortizing work across pods because they are considered for placement individually.
|
||||
|
||||
These downsides of pod-by-pod schedulers are addressed by batching or bulk scheduling in Poseidon-Firmament scheduler. Processing several pods in a batch allows the scheduler to jointly consider their placement, and thus to find the best trade-off for the whole batch instead of one pod. At the same time it amortizes work across pods resulting in much higher throughput.
|
||||
## Performance comparison
|
||||
|
||||
{{< note >}}
|
||||
Please refer to the [latest benchmark results](https://github.com/kubernetes-sigs/poseidon/blob/master/docs/benchmark/README.md) for detailed throughput performance comparison test results between Poseidon-Firmament scheduler and the Kubernetes default scheduler.
|
||||
{{< /note >}}
|
||||
|
||||
Pod-by-pod schedulers, such as the Kubernetes default scheduler, process Pods in small batches (typically one at a time). These schedulers have the following crucial drawbacks:
|
||||
|
||||
1. The scheduler commits to a pod placement early and restricts the choices for other pods that wait to be placed.
|
||||
2. There is limited opportunities for amortizing work across pods because they are considered for placement individually.
|
||||
|
||||
These downsides of pod-by-pod schedulers are addressed by batching or bulk scheduling in Poseidon-Firmament scheduler. Processing several pods in a batch allows the scheduler to jointly consider their placement, and thus to find the best trade-off for the whole batch instead of one pod. At the same time it amortizes work across pods resulting in much higher throughput.
|
||||
|
||||
{{% /capture %}}
|
||||
{{% capture whatsnext %}}
|
||||
* See [Poseidon-Firmament](https://github.com/kubernetes-sigs/poseidon#readme) on GitHub for more information.
|
||||
* See the [design document](https://github.com/kubernetes-sigs/poseidon/blob/master/docs/design/README.md) for Poseidon.
|
||||
* Read [Firmament: Fast, Centralized Cluster Scheduling at Scale](https://www.usenix.org/system/files/conference/osdi16/osdi16-gog.pdf), the academic paper on the Firmament scheduling design.
|
||||
* If you'd like to contribute to Poseidon-Firmament, refer to the [developer setup instructions](https://github.com/kubernetes-sigs/poseidon/blob/master/docs/devel/README.md).
|
||||
{{% /capture %}}
|
||||
|
||||
@@ -121,21 +121,22 @@ There are two supported paths to extending the API with [custom resources](/docs
|
||||
to make it seamless for clients.
|
||||
|
||||
|
||||
## Enabling API groups
|
||||
## Enabling or disabling API groups
|
||||
|
||||
Certain resources and API groups are enabled by default. They can be enabled or disabled by setting `--runtime-config`
|
||||
on apiserver. `--runtime-config` accepts comma separated values. For ex: to disable batch/v1, set
|
||||
on apiserver. `--runtime-config` accepts comma separated values. For example: to disable batch/v1, set
|
||||
`--runtime-config=batch/v1=false`, to enable batch/v2alpha1, set `--runtime-config=batch/v2alpha1`.
|
||||
The flag accepts comma separated set of key=value pairs describing runtime configuration of the apiserver.
|
||||
|
||||
IMPORTANT: Enabling or disabling groups or resources requires restarting apiserver and controller-manager
|
||||
to pick up the `--runtime-config` changes.
|
||||
{{< note >}}Enabling or disabling groups or resources requires restarting apiserver and controller-manager
|
||||
to pick up the `--runtime-config` changes.{{< /note >}}
|
||||
|
||||
## Enabling resources in the groups
|
||||
## Enabling specific resources in the extensions/v1beta1 group
|
||||
|
||||
DaemonSets, Deployments, HorizontalPodAutoscalers, Ingresses, Jobs and ReplicaSets are enabled by default.
|
||||
Other extensions resources can be enabled by setting `--runtime-config` on
|
||||
apiserver. `--runtime-config` accepts comma separated values. For example: to disable deployments and ingress, set
|
||||
`--runtime-config=extensions/v1beta1/deployments=false,extensions/v1beta1/ingresses=false`
|
||||
DaemonSets, Deployments, StatefulSet, NetworkPolicies, PodSecurityPolicies and ReplicaSets in the `extensions/v1beta1` API group are disabled by default.
|
||||
For example: to enable deployments and daemonsets, set
|
||||
`--runtime-config=extensions/v1beta1/deployments=true,extensions/v1beta1/daemonsets=true`.
|
||||
|
||||
{{< note >}}Individual resource enablement/disablement is only supported in the `extensions/v1beta1` API group for legacy reasons.{{< /note >}}
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
@@ -3,6 +3,8 @@ reviewers:
|
||||
- bgrant0607
|
||||
- mikedanese
|
||||
title: What is Kubernetes
|
||||
description: >
|
||||
Kubernetes is a portable, extensible, open-source platform for managing containerized workloads and services, that facilitates both declarative configuration and automation. It has a large, rapidly growing ecosystem. Kubernetes services, support, and tools are widely available.
|
||||
content_template: templates/concept
|
||||
weight: 10
|
||||
card:
|
||||
@@ -17,9 +19,10 @@ This page is an overview of Kubernetes.
|
||||
{{% capture body %}}
|
||||
Kubernetes is a portable, extensible, open-source platform for managing containerized workloads and services, that facilitates both declarative configuration and automation. It has a large, rapidly growing ecosystem. Kubernetes services, support, and tools are widely available.
|
||||
|
||||
The name Kubernetes originates from Greek, meaning helmsman or pilot. Google open-sourced the Kubernetes project in 2014. Kubernetes builds upon a [decade and a half of experience that Google has with running production workloads at scale](https://ai.google/research/pubs/pub43438), combined with best-of-breed ideas and practices from the community.
|
||||
The name Kubernetes originates from Greek, meaning helmsman or pilot. Google open-sourced the Kubernetes project in 2014. Kubernetes combines [over 15 years of Google's experience](/blog/2015/04/borg-predecessor-to-kubernetes/) running production workloads at scale with best-of-breed ideas and practices from the community.
|
||||
|
||||
## Going back in time
|
||||
|
||||
Let's take a look at why Kubernetes is so useful by going back in time.
|
||||
|
||||
|
||||
@@ -42,13 +45,13 @@ Containers have become popular because they provide extra benefits, such as:
|
||||
* Dev and Ops separation of concerns: create application container images at build/release time rather than deployment time, thereby decoupling applications from infrastructure.
|
||||
* Observability not only surfaces OS-level information and metrics, but also application health and other signals.
|
||||
* Environmental consistency across development, testing, and production: Runs the same on a laptop as it does in the cloud.
|
||||
* Cloud and OS distribution portability: Runs on Ubuntu, RHEL, CoreOS, on-prem, Google Kubernetes Engine, and anywhere else.
|
||||
* Cloud and OS distribution portability: Runs on Ubuntu, RHEL, CoreOS, on-premises, on major public clouds, and anywhere else.
|
||||
* Application-centric management: Raises the level of abstraction from running an OS on virtual hardware to running an application on an OS using logical resources.
|
||||
* Loosely coupled, distributed, elastic, liberated micro-services: applications are broken into smaller, independent pieces and can be deployed and managed dynamically – not a monolithic stack running on one big single-purpose machine.
|
||||
* Resource isolation: predictable application performance.
|
||||
* Resource utilization: high efficiency and density.
|
||||
|
||||
## Why you need Kubernetes and what can it do
|
||||
## Why you need Kubernetes and what it can do {#why-you-need-kubernetes-and-what-can-it-do}
|
||||
|
||||
Containers are a good way to bundle and run your applications. In a production environment, you need to manage the containers that run the applications and ensure that there is no downtime. For example, if a container goes down, another container needs to start. Wouldn't it be easier if this behavior was handled by a system?
|
||||
|
||||
|
||||
@@ -82,7 +82,7 @@ metadata:
|
||||
spec:
|
||||
containers:
|
||||
- name: nginx
|
||||
image: nginx:1.7.9
|
||||
image: nginx:1.14.2
|
||||
ports:
|
||||
- containerPort: 80
|
||||
|
||||
|
||||
@@ -26,10 +26,27 @@ To work with Kubernetes objects--whether to create, modify, or delete them--you'
|
||||
|
||||
### Object Spec and Status
|
||||
|
||||
Every Kubernetes object includes two nested object fields that govern the object's configuration: the object *spec* and the object *status*. The *spec*, which you must provide, describes your desired state for the object--the characteristics that you want the object to have. The *status* describes the *actual state* of the object, and is supplied and updated by the Kubernetes system. At any given time, the Kubernetes Control Plane actively manages an object's actual state to match the desired state you supplied.
|
||||
Almost every Kubernetes object includes two nested object fields that govern
|
||||
the object's configuration: the object *`spec`* and the object *`status`*.
|
||||
For objects that have a `spec`, you have to set this when you create the object,
|
||||
providing a description of the characteristics you want the resource to have:
|
||||
its _desired state_.
|
||||
|
||||
The `status` describes the _current state_ of the object, supplied and updated
|
||||
by the Kubernetes and its components. The Kubernetes
|
||||
{{< glossary_tooltip text="control plane" term_id="control-plane" >}} continually
|
||||
and actively manages every object's actual state to match the desired state you
|
||||
supplied.
|
||||
|
||||
For example, a Kubernetes Deployment is an object that can represent an application running on your cluster. When you create the Deployment, you might set the Deployment spec to specify that you want three replicas of the application to be running. The Kubernetes system reads the Deployment spec and starts three instances of your desired application--updating the status to match your spec. If any of those instances should fail (a status change), the Kubernetes system responds to the difference between spec and status by making a correction--in this case, starting a replacement instance.
|
||||
For example: in Kubernetes, a Deployment is an object that can represent an
|
||||
application running on your cluster. When you create the Deployment, you
|
||||
might set the Deployment `spec` to specify that you want three replicas of
|
||||
the application to be running. The Kubernetes system reads the Deployment
|
||||
spec and starts three instances of your desired application--updating
|
||||
the status to match your spec. If any of those instances should fail
|
||||
(a status change), the Kubernetes system responds to the difference
|
||||
between spec and status by making a correction--in this case, starting
|
||||
a replacement instance.
|
||||
|
||||
For more information on the object spec, status, and metadata, see the [Kubernetes API Conventions](https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md).
|
||||
|
||||
|
||||
@@ -69,7 +69,7 @@ metadata:
|
||||
spec:
|
||||
containers:
|
||||
- name: nginx
|
||||
image: nginx:1.7.9
|
||||
image: nginx:1.14.2
|
||||
ports:
|
||||
- containerPort: 80
|
||||
|
||||
|
||||
@@ -64,7 +64,7 @@ metadata:
|
||||
spec:
|
||||
containers:
|
||||
- name: nginx
|
||||
image: nginx:1.7.9
|
||||
image: nginx:1.14.2
|
||||
ports:
|
||||
- containerPort: 80
|
||||
```
|
||||
|
||||
@@ -9,55 +9,58 @@ weight: 10
|
||||
{{% capture overview %}}
|
||||
|
||||
By default, containers run with unbounded [compute resources](/docs/user-guide/compute-resources) on a Kubernetes cluster.
|
||||
With Resource quotas, cluster administrators can restrict the resource consumption and creation on a namespace basis.
|
||||
Within a namespace, a Pod or Container can consume as much CPU and memory as defined by the namespace's resource quota. There is a concern that one Pod or Container could monopolize all of the resources. Limit Range is a policy to constrain resource by Pod or Container in a namespace.
|
||||
With resource quotas, cluster administrators can restrict resource consumption and creation on a namespace basis.
|
||||
Within a namespace, a Pod or Container can consume as much CPU and memory as defined by the namespace's resource quota. There is a concern that one Pod or Container could monopolize all available resources. A LimitRange is a policy to constrain resource allocations (to Pods or Containers) in a namespace.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
|
||||
{{% capture body %}}
|
||||
|
||||
A limit range, defined by a `LimitRange` object, provides constraints that can:
|
||||
A _LimitRange_ provides constraints that can:
|
||||
|
||||
- Enforce minimum and maximum compute resources usage per Pod or Container in a namespace.
|
||||
- Enforce minimum and maximum storage request per PersistentVolumeClaim in a namespace.
|
||||
- Enforce a ratio between request and limit for a resource in a namespace.
|
||||
- Set default request/limit for compute resources in a namespace and automatically inject them to Containers at runtime.
|
||||
|
||||
## Enabling Limit Range
|
||||
## Enabling LimitRange
|
||||
|
||||
Limit Range support is enabled by default for many Kubernetes distributions. It is
|
||||
LimitRange support is enabled by default for many Kubernetes distributions. It is
|
||||
enabled when the apiserver `--enable-admission-plugins=` flag has `LimitRanger` admission controller as
|
||||
one of its arguments.
|
||||
|
||||
A limit range is enforced in a particular namespace when there is a
|
||||
`LimitRange` object in that namespace.
|
||||
A LimitRange is enforced in a particular namespace when there is a
|
||||
LimitRange object in that namespace.
|
||||
|
||||
The name of a LimitRange object must be a valid
|
||||
[DNS subdomain name](/docs/concepts/overview/working-with-objects/names#dns-subdomain-names).
|
||||
|
||||
### Overview of Limit Range
|
||||
|
||||
- The administrator creates one `LimitRange` in one namespace.
|
||||
- Users create resources like Pods, Containers, and PersistentVolumeClaims in the namespace.
|
||||
- The `LimitRanger` admission controller enforces defaults limits for all Pods and Container that do not set compute resource requirements and tracks usage to ensure it does not exceed resource minimum , maximum and ratio defined in any `LimitRange` present in the namespace.
|
||||
- If creating or updating a resource (Pod, Container, PersistentVolumeClaim) violates a limit range constraint, the request to the API server will fail with HTTP status code `403 FORBIDDEN` and a message explaining the constraint that would have been violated.
|
||||
- If limit range is activated in a namespace for compute resources like `cpu` and `memory`, users must specify
|
||||
requests or limits for those values; otherwise, the system may reject pod creation.
|
||||
- LimitRange validations occurs only at Pod Admission stage, not on Running pods.
|
||||
- The `LimitRanger` admission controller enforces defaults and limits for all Pods and Containers that do not set compute resource requirements and tracks usage to ensure it does not exceed resource minimum, maximum and ratio defined in any LimitRange present in the namespace.
|
||||
- If creating or updating a resource (Pod, Container, PersistentVolumeClaim) that violates a LimitRange constraint, the request to the API server will fail with an HTTP status code `403 FORBIDDEN` and a message explaining the constraint that have been violated.
|
||||
- If a LimitRange is activated in a namespace for compute resources like `cpu` and `memory`, users must specify
|
||||
requests or limits for those values. Otherwise, the system may reject Pod creation.
|
||||
- LimitRange validations occurs only at Pod Admission stage, not on Running Pods.
|
||||
|
||||
Examples of policies that could be created using limit range are:
|
||||
|
||||
- In a 2 node cluster with a capacity of 8 GiB RAM, and 16 cores, constrain Pods in a namespace to request 100m and not exceeds 500m for CPU , request 200Mi and not exceed 600Mi
|
||||
- Define default CPU limits and request to 150m and Memory default request to 300Mi for containers started with no cpu and memory requests in their spec.
|
||||
- In a 2 node cluster with a capacity of 8 GiB RAM and 16 cores, constrain Pods in a namespace to request 100m of CPU with a max limit of 500m for CPU and request 200Mi for Memory with a max limit of 600Mi for Memory.
|
||||
- Define default CPU limit and request to 150m and memory default request to 300Mi for Containers started with no cpu and memory requests in their specs.
|
||||
|
||||
In the case where the total limits of the namespace is less than the sum of the limits of the Pods/Containers,
|
||||
there may be contention for resources; The Containers or Pods will not be created.
|
||||
there may be contention for resources. In this case, the Containers or Pods will not be created.
|
||||
|
||||
Neither contention nor changes to limitrange will affect already created resources.
|
||||
Neither contention nor changes to a LimitRange will affect already created resources.
|
||||
|
||||
## Limiting Container compute resources
|
||||
|
||||
The following section discusses the creation of a LimitRange acting at Container Level.
|
||||
A Pod with 04 containers is first created; each container within the Pod has a specific `spec.resource` configuration
|
||||
each container within the pod is handled differently by the LimitRanger admission controller.
|
||||
A Pod with 04 Containers is first created. Each Container within the Pod has a specific `spec.resource` configuration.
|
||||
Each Container within the Pod is handled differently by the `LimitRanger` admission controller.
|
||||
|
||||
Create a namespace `limitrange-demo` using the following kubectl command:
|
||||
|
||||
@@ -74,9 +77,7 @@ kubectl config set-context --current --namespace=limitrange-demo
|
||||
Here is the configuration file for a LimitRange object:
|
||||
{{< codenew file="admin/resource/limit-mem-cpu-container.yaml" >}}
|
||||
|
||||
This object defines minimum and maximum Memory/CPU limits, default cpu/Memory requests and default limits for CPU/Memory resources to be apply to containers.
|
||||
The name of a LimitRange object must be a valid
|
||||
[DNS subdomain name](/docs/concepts/overview/working-with-objects/names#dns-subdomain-names).
|
||||
This object defines minimum and maximum CPU/Memory limits, default CPU/Memory requests, and default limits for CPU/Memory resources to be apply to containers.
|
||||
|
||||
Create the `limit-mem-cpu-per-container` LimitRange in the `limitrange-demo` namespace with the following kubectl command:
|
||||
|
||||
@@ -95,7 +96,7 @@ Container cpu 100m 800m 110m 700m -
|
||||
Container memory 99Mi 1Gi 111Mi 900Mi -
|
||||
```
|
||||
|
||||
Here is the configuration file for a Pod with 04 containers to demonstrate LimitRange features :
|
||||
Here is the configuration file for a Pod with 04 Containers to demonstrate LimitRange features:
|
||||
{{< codenew file="admin/resource/limit-range-pod-1.yaml" >}}
|
||||
|
||||
Create the `busybox1` Pod:
|
||||
@@ -126,9 +127,9 @@ kubectl get po/busybox1 -n limitrange-demo -o json | jq ".spec.containers[0].res
|
||||
```
|
||||
|
||||
- The `busybox-cnt01` Container inside `busybox` Pod defined `requests.cpu=100m` and `requests.memory=100Mi`.
|
||||
- `100m <= 500m <= 800m` , The container cpu limit (500m) falls inside the authorized CPU limit range.
|
||||
- `99Mi <= 200Mi <= 1Gi` , The container memory limit (200Mi) falls inside the authorized Memory limit range.
|
||||
- No request/limits ratio validation for CPU/Memory , thus the container is valid and created.
|
||||
- `100m <= 500m <= 800m` , The Container cpu limit (500m) falls inside the authorized CPU LimitRange.
|
||||
- `99Mi <= 200Mi <= 1Gi` , The Container memory limit (200Mi) falls inside the authorized Memory LimitRange.
|
||||
- No request/limits ratio validation for CPU/Memory, so the Container is valid and created.
|
||||
|
||||
|
||||
### Container spec with a valid CPU/Memory requests but no limits
|
||||
@@ -152,14 +153,15 @@ kubectl get po/busybox1 -n limitrange-demo -o json | jq ".spec.containers[1].res
|
||||
}
|
||||
```
|
||||
- The `busybox-cnt02` Container inside `busybox1` Pod defined `requests.cpu=100m` and `requests.memory=100Mi` but not limits for cpu and memory.
|
||||
- The container do not have a limits section, the default limits defined in the limit-mem-cpu-per-container LimitRange object are injected to this container `limits.cpu=700mi` and `limits.memory=900Mi`.
|
||||
- `100m <= 700m <= 800m` , The container cpu limit (700m) falls inside the authorized CPU limit range.
|
||||
- `99Mi <= 900Mi <= 1Gi` , The container memory limit (900Mi) falls inside the authorized Memory limit range.
|
||||
- No request/limits ratio set , thus the container is valid and created.
|
||||
- The Container does not have a limits section. The default limits defined in the `limit-mem-cpu-per-container` LimitRange object are injected in to this Container: `limits.cpu=700mi` and `limits.memory=900Mi`.
|
||||
- `100m <= 700m <= 800m` , The Container cpu limit (700m) falls inside the authorized CPU limit range.
|
||||
- `99Mi <= 900Mi <= 1Gi` , The Container memory limit (900Mi) falls inside the authorized Memory limit range.
|
||||
- No request/limits ratio set, so the Container is valid and created.
|
||||
|
||||
|
||||
### Container spec with a valid CPU/Memory limits but no requests
|
||||
View the `busybox-cnt03` resource configuration
|
||||
### Container spec with a valid CPU/Memory limits but no requests
|
||||
|
||||
View the `busybox-cnt03` resource configuration:
|
||||
|
||||
```shell
|
||||
kubectl get po/busybox1 -n limitrange-demo -o json | jq ".spec.containers[2].resources"
|
||||
@@ -178,10 +180,10 @@ kubectl get po/busybox1 -n limitrange-demo -o json | jq ".spec.containers[2].res
|
||||
```
|
||||
|
||||
- The `busybox-cnt03` Container inside `busybox1` Pod defined `limits.cpu=500m` and `limits.memory=200Mi` but no `requests` for cpu and memory.
|
||||
- The container do not define a request section, the defaultRequest defined in the limit-mem-cpu-per-container LimitRange is not used to fill its limits section but the limits defined by the container are set as requests `limits.cpu=500m` and `limits.memory=200Mi`.
|
||||
- `100m <= 500m <= 800m` , The container cpu limit (500m) falls inside the authorized CPU limit range.
|
||||
- `99Mi <= 200Mi <= 1Gi` , The container memory limit (200Mi) falls inside the authorized Memory limit range.
|
||||
- No request/limits ratio set , thus the container is valid and created.
|
||||
- The Container does not define a request section. The default request defined in the limit-mem-cpu-per-container LimitRange is not used to fill its limits section, but the limits defined by the Container are set as requests `limits.cpu=500m` and `limits.memory=200Mi`.
|
||||
- `100m <= 500m <= 800m` , The Container cpu limit (500m) falls inside the authorized CPU limit range.
|
||||
- `99Mi <= 200Mi <= 1Gi` , The Container memory limit (200Mi) falls inside the authorized Memory limit range.
|
||||
- No request/limits ratio set, so the Container is valid and created.
|
||||
|
||||
### Container spec with no CPU/Memory requests/limits
|
||||
|
||||
@@ -205,27 +207,27 @@ kubectl get po/busybox1 -n limitrange-demo -o json | jq ".spec.containers[3].res
|
||||
```
|
||||
|
||||
- The `busybox-cnt04` Container inside `busybox1` define neither `limits` nor `requests`.
|
||||
- The container do not define a limit section, the default limit defined in the limit-mem-cpu-per-container LimitRange is used to fill its request
|
||||
- The Container do not define a limit section, the default limit defined in the limit-mem-cpu-per-container LimitRange is used to fill its request
|
||||
`limits.cpu=700m and` `limits.memory=900Mi` .
|
||||
- The container do not define a request section, the defaultRequest defined in the limit-mem-cpu-per-container LimitRange is used to fill its request section requests.cpu=110m and requests.memory=111Mi
|
||||
- `100m <= 700m <= 800m` , The container cpu limit (700m) falls inside the authorized CPU limit range.
|
||||
- `99Mi <= 900Mi <= 1Gi` , The container memory limit (900Mi) falls inside the authorized Memory limitrange .
|
||||
- No request/limits ratio set , thus the container is valid and created.
|
||||
- The Container do not define a request section, the defaultRequest defined in the `limit-mem-cpu-per-container` LimitRange is used to fill its request section requests.cpu=110m and requests.memory=111Mi
|
||||
- `100m <= 700m <= 800m` , The Container cpu limit (700m) falls inside the authorized CPU limit range.
|
||||
- `99Mi <= 900Mi <= 1Gi` , The Container memory limit (900Mi) falls inside the authorized Memory limit range .
|
||||
- No request/limits ratio set, so the Container is valid and created.
|
||||
|
||||
All containers defined in the `busybox` Pod passed LimitRange validations, this the Pod is valid and create in the namespace.
|
||||
All Containers defined in the `busybox` Pod passed LimitRange validations, so this the Pod is valid and created in the namespace.
|
||||
|
||||
## Limiting Pod compute resources
|
||||
|
||||
The following section discusses how to constrain resources at Pod level.
|
||||
The following section discusses how to constrain resources at the Pod level.
|
||||
|
||||
{{< codenew file="admin/resource/limit-mem-cpu-pod.yaml" >}}
|
||||
|
||||
Without having to delete `busybox1` Pod, create the `limit-mem-cpu-pod` LimitRange in the `limitrange-demo` namespace:
|
||||
Without having to delete the `busybox1` Pod, create the `limit-mem-cpu-pod` LimitRange in the `limitrange-demo` namespace:
|
||||
|
||||
```shell
|
||||
kubectl apply -f https://k8s.io/examples/admin/resource/limit-mem-cpu-pod.yaml -n limitrange-demo
|
||||
```
|
||||
The limitrange is created and limits CPU to 2 Core and Memory to 2Gi per Pod:
|
||||
The LimitRange is created and limits CPU to 2 Core and Memory to 2Gi per Pod:
|
||||
|
||||
```shell
|
||||
limitrange/limit-mem-cpu-per-pod created
|
||||
@@ -254,7 +256,7 @@ Now create the `busybox2` Pod:
|
||||
kubectl apply -f https://k8s.io/examples/admin/resource/limit-range-pod-2.yaml -n limitrange-demo
|
||||
```
|
||||
|
||||
The `busybox2` Pod definition is identical to `busybox1` but an error is reported since Pod's resources are now limited:
|
||||
The `busybox2` Pod definition is identical to `busybox1`, but an error is reported since the Pod's resources are now limited:
|
||||
|
||||
```shell
|
||||
Error from server (Forbidden): error when creating "limit-range-pod-2.yaml": pods "busybox2" is forbidden: [maximum cpu usage per Pod is 2, but limit is 2400m., maximum memory usage per Pod is 2Gi, but limit is 2306867200.]
|
||||
@@ -268,12 +270,12 @@ kubectl get po/busybox1 -n limitrange-demo -o json | jq ".spec.containers[].reso
|
||||
"900Mi"
|
||||
```
|
||||
|
||||
`busybox2` Pod will not be admitted on the cluster since the total memory limit of its container is greater than the limit defined in the LimitRange.
|
||||
`busybox2` Pod will not be admitted on the cluster since the total memory limit of its Container is greater than the limit defined in the LimitRange.
|
||||
`busybox1` will not be evicted since it was created and admitted on the cluster before the LimitRange creation.
|
||||
|
||||
## Limiting Storage resources
|
||||
|
||||
You can enforce minimum and maximum size of [storage resources](/docs/concepts/storage/persistent-volumes/) that can be requested by each PersistentVolumeClaim in a namespace using a LimitRange:
|
||||
You can enforce minimum and maximum size of [storage resources](/docs/concepts/storage/persistent-volumes/) that can be requested by each PersistentVolumeClaim in a namespace using a LimitRange:
|
||||
|
||||
{{< codenew file="admin/resource/storagelimits.yaml" >}}
|
||||
|
||||
@@ -329,9 +331,9 @@ Error from server (Forbidden): error when creating "pvc-limit-greater.yaml": per
|
||||
|
||||
## Limits/Requests Ratio
|
||||
|
||||
If `LimitRangeItem.maxLimitRequestRatio` is specified in the `LimitRangeSpec`, the named resource must have a request and limit that are both non-zero where limit divided by request is less than or equal to the enumerated value
|
||||
If `LimitRangeItem.maxLimitRequestRatio` is specified in the `LimitRangeSpec`, the named resource must have a request and limit that are both non-zero where limit divided by request is less than or equal to the enumerated value.
|
||||
|
||||
The following `LimitRange` enforces memory limit to be at most twice the amount of the memory request for any pod in the namespace.
|
||||
The following LimitRange enforces memory limit to be at most twice the amount of the memory request for any Pod in the namespace:
|
||||
|
||||
{{< codenew file="admin/resource/limit-memory-ratio-pod.yaml" >}}
|
||||
|
||||
@@ -353,7 +355,7 @@ Type Resource Min Max Default Request Default Limit Max Limit/Reques
|
||||
Pod memory - - - - 2
|
||||
```
|
||||
|
||||
Let's create a pod with `requests.memory=100Mi` and `limits.memory=300Mi`:
|
||||
Create a pod with `requests.memory=100Mi` and `limits.memory=300Mi`:
|
||||
|
||||
{{< codenew file="admin/resource/limit-range-pod-3.yaml" >}}
|
||||
|
||||
@@ -361,9 +363,9 @@ Let's create a pod with `requests.memory=100Mi` and `limits.memory=300Mi`:
|
||||
kubectl apply -f https://k8s.io/examples/admin/resource/limit-range-pod-3.yaml
|
||||
```
|
||||
|
||||
The pod creation failed as the ratio here (`3`) is greater than the enforced limit (`2`) in `limit-memory-ratio-pod` LimitRange
|
||||
The pod creation failed as the ratio here (`3`) is greater than the enforced limit (`2`) in `limit-memory-ratio-pod` LimitRange:
|
||||
|
||||
```shell
|
||||
```
|
||||
Error from server (Forbidden): error when creating "limit-range-pod-3.yaml": pods "busybox3" is forbidden: memory max limit to request ratio per Pod is 2, but provided ratio is 3.000000.
|
||||
```
|
||||
|
||||
|
||||
@@ -197,6 +197,8 @@ alias kubectl-user='kubectl --as=system:serviceaccount:psp-example:fake-user -n
|
||||
|
||||
Define the example PodSecurityPolicy object in a file. This is a policy that
|
||||
simply prevents the creation of privileged pods.
|
||||
The name of a PodSecurityPolicy object must be a valid
|
||||
[DNS subdomain name](/docs/concepts/overview/working-with-objects/names#dns-subdomain-names).
|
||||
|
||||
{{< codenew file="policy/example-psp.yaml" >}}
|
||||
|
||||
@@ -419,8 +421,10 @@ The **recommended minimum set** of allowed volumes for new PSPs are:
|
||||
- projected
|
||||
|
||||
{{< warning >}}
|
||||
PodSecurityPolicy does not limit the types of `PersistentVolume` objects that may be referenced by a `PersistentVolumeClaim`.
|
||||
Only trusted users should be granted permission to create `PersistentVolume` objects.
|
||||
PodSecurityPolicy does not limit the types of `PersistentVolume` objects that
|
||||
may be referenced by a `PersistentVolumeClaim`, and hostPath type
|
||||
`PersistentVolumes` do not support read-only access mode. Only trusted users
|
||||
should be granted permission to create `PersistentVolume` objects.
|
||||
{{< /warning >}}
|
||||
|
||||
**FSGroup** - Controls the supplemental group applied to some volumes.
|
||||
|
||||
@@ -422,10 +422,8 @@ LoadBalancer Ingress: a320587ffd19711e5a37606cf4a74574-1142138393.us-east-1.el
|
||||
|
||||
{{% capture whatsnext %}}
|
||||
|
||||
Kubernetes also supports Federated Services, which can span multiple
|
||||
clusters and cloud providers, to provide increased availability,
|
||||
better fault tolerance and greater scalability for your services. See
|
||||
the [Federated Services User Guide](/docs/concepts/cluster-administration/federation-service-discovery/)
|
||||
for further information.
|
||||
* Learn more about [Using a Service to Access an Application in a Cluster](/docs/tasks/access-application-cluster/service-access-application-cluster/)
|
||||
* Learn more about [Connecting a Front End to a Back End Using a Service](/docs/tasks/access-application-cluster/connecting-frontend-backend/)
|
||||
* Learn more about [Creating an External Load Balancer](/docs/tasks/access-application-cluster/create-external-load-balancer/)
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
@@ -37,7 +37,7 @@ Traffic routing is controlled by rules defined on the Ingress resource.
|
||||
[ Services ]
|
||||
```
|
||||
|
||||
An Ingress can be configured to give Services externally-reachable URLs, load balance traffic, terminate SSL / TLS, and offer name based virtual hosting. An [Ingress controller](/docs/concepts/services-networking/ingress-controllers) is responsible for fulfilling the Ingress, usually with a load balancer, though it may also configure your edge router or additional frontends to help handle the traffic.
|
||||
An Ingress may be configured to give Services externally-reachable URLs, load balance traffic, terminate SSL / TLS, and offer name based virtual hosting. An [Ingress controller](/docs/concepts/services-networking/ingress-controllers) is responsible for fulfilling the Ingress, usually with a load balancer, though it may also configure your edge router or additional frontends to help handle the traffic.
|
||||
|
||||
An Ingress does not expose arbitrary ports or protocols. Exposing services other than HTTP and HTTPS to the internet typically
|
||||
uses a service of type [Service.Type=NodePort](/docs/concepts/services-networking/service/#nodeport) or
|
||||
|
||||
@@ -1189,19 +1189,6 @@ SCTP is not supported on Windows based nodes.
|
||||
The kube-proxy does not support the management of SCTP associations when it is in userspace mode.
|
||||
{{< /warning >}}
|
||||
|
||||
## Future work
|
||||
|
||||
In the future, the proxy policy for Services can become more nuanced than
|
||||
simple round-robin balancing, for example master-elected or sharded. We also
|
||||
envision that some Services will have "real" load balancers, in which case the
|
||||
virtual IP address will simply transport the packets there.
|
||||
|
||||
The Kubernetes project intends to improve support for L7 (HTTP) Services.
|
||||
|
||||
The Kubernetes project intends to have more flexible ingress modes for Services
|
||||
that encompass the current ClusterIP, NodePort, and LoadBalancer modes and more.
|
||||
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture whatsnext %}}
|
||||
|
||||
@@ -46,6 +46,9 @@ To enable dynamic provisioning, a cluster administrator needs to pre-create
|
||||
one or more StorageClass objects for users.
|
||||
StorageClass objects define which provisioner should be used and what parameters
|
||||
should be passed to that provisioner when dynamic provisioning is invoked.
|
||||
The name of a StorageClass object must be a valid
|
||||
[DNS subdomain name](/docs/concepts/overview/working-with-objects/names#dns-subdomain-names).
|
||||
|
||||
The following manifest creates a storage class "slow" which provisions standard
|
||||
disk-like persistent disks.
|
||||
|
||||
|
||||
@@ -4,6 +4,7 @@ reviewers:
|
||||
- saad-ali
|
||||
- thockin
|
||||
- msau42
|
||||
- xing-yang
|
||||
title: Persistent Volumes
|
||||
feature:
|
||||
title: Storage orchestration
|
||||
@@ -672,7 +673,7 @@ Only statically provisioned volumes are supported for alpha release. Administrat
|
||||
|
||||
## Volume Snapshot and Restore Volume from Snapshot Support
|
||||
|
||||
{{< feature-state for_k8s_version="v1.12" state="alpha" >}}
|
||||
{{< feature-state for_k8s_version="v1.17" state="beta" >}}
|
||||
|
||||
Volume snapshot feature was added to support CSI Volume Plugins only. For details, see [volume snapshots](/docs/concepts/storage/volume-snapshots/).
|
||||
|
||||
@@ -747,5 +748,17 @@ and need persistent storage, it is recommended that you use the following patter
|
||||
dynamic storage support (in which case the user should create a matching PV)
|
||||
or the cluster has no storage system (in which case the user cannot deploy
|
||||
config requiring PVCs).
|
||||
|
||||
{{% /capture %}}
|
||||
{{% capture whatsnext %}}
|
||||
|
||||
* Learn more about [Creating a Persistent Volume](/docs/tasks/configure-pod-container/configure-persistent-volume-storage/#create-a-persistentvolume).
|
||||
* Learn more about [Creating a Persistent Volume Claim](/docs/tasks/configure-pod-container/configure-persistent-volume-storage/#create-a-persistentvolumeclaim).
|
||||
* Read the [Persistent Storage design document](https://git.k8s.io/community/contributors/design-proposals/storage/persistent-storage.md).
|
||||
|
||||
### Reference
|
||||
|
||||
* [PersistentVolume](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#persistentvolume-v1-core)
|
||||
* [PersistentVolumeSpec](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#persistentvolumespec-v1-core)
|
||||
* [PersistentVolumeClaim](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#persistentvolumeclaim-v1-core)
|
||||
* [PersistentVolumeClaimSpec](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#persistentvolumeclaimspec-v1-core)
|
||||
{{% /capture %}}
|
||||
|
||||
@@ -605,6 +605,38 @@ spec:
|
||||
type: Directory
|
||||
```
|
||||
|
||||
{{< caution >}}
|
||||
It should be noted that the `FileOrCreate` mode does not create the parent directory of the file. If the parent directory of the mounted file does not exist, the pod fails to start. To ensure that this mode works, you can try to mount directories and files separately, as shown below.
|
||||
{{< /caution >}}
|
||||
|
||||
#### Example Pod FileOrCreate
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: test-webserver
|
||||
spec:
|
||||
containers:
|
||||
- name: test-webserver
|
||||
image: k8s.gcr.io/test-webserver:latest
|
||||
volumeMounts:
|
||||
- mountPath: /var/local/aaa
|
||||
name: mydir
|
||||
- mountPath: /var/local/aaa/1.txt
|
||||
name: myfile
|
||||
volumes:
|
||||
- name: mydir
|
||||
hostPath:
|
||||
# Ensure the file directory is created.
|
||||
path: /var/local/aaa
|
||||
type: DirectoryOrCreate
|
||||
- name: myfile
|
||||
hostPath:
|
||||
path: /var/local/aaa/1.txt
|
||||
type: FileOrCreate
|
||||
```
|
||||
|
||||
### iscsi {#iscsi}
|
||||
|
||||
An `iscsi` volume allows an existing iSCSI (SCSI over IP) volume to be mounted
|
||||
|
||||
@@ -64,7 +64,7 @@ In this example:
|
||||
* The Pods are labeled `app: nginx`using the `labels` field.
|
||||
* The Pod template's specification, or `.template.spec` field, indicates that
|
||||
the Pods run one container, `nginx`, which runs the `nginx`
|
||||
[Docker Hub](https://hub.docker.com/) image at version 1.7.9.
|
||||
[Docker Hub](https://hub.docker.com/) image at version 1.14.2.
|
||||
* Create one container and name it `nginx` using the `name` field.
|
||||
|
||||
Follow the steps given below to create the above Deployment:
|
||||
@@ -153,15 +153,15 @@ is changed, for example if the labels or container images of the template are up
|
||||
|
||||
Follow the steps given below to update your Deployment:
|
||||
|
||||
1. Let's update the nginx Pods to use the `nginx:1.9.1` image instead of the `nginx:1.7.9` image.
|
||||
1. Let's update the nginx Pods to use the `nginx:1.16.1` image instead of the `nginx:1.14.2` image.
|
||||
|
||||
```shell
|
||||
kubectl --record deployment.apps/nginx-deployment set image deployment.v1.apps/nginx-deployment nginx=nginx:1.9.1
|
||||
kubectl --record deployment.apps/nginx-deployment set image deployment.v1.apps/nginx-deployment nginx=nginx:1.16.1
|
||||
```
|
||||
or simply use the following command:
|
||||
|
||||
```shell
|
||||
kubectl set image deployment/nginx-deployment nginx=nginx:1.9.1 --record
|
||||
kubectl set image deployment/nginx-deployment nginx=nginx:1.16.1 --record
|
||||
```
|
||||
|
||||
The output is similar to this:
|
||||
@@ -169,7 +169,7 @@ Follow the steps given below to update your Deployment:
|
||||
deployment.apps/nginx-deployment image updated
|
||||
```
|
||||
|
||||
Alternatively, you can `edit` the Deployment and change `.spec.template.spec.containers[0].image` from `nginx:1.7.9` to `nginx:1.9.1`:
|
||||
Alternatively, you can `edit` the Deployment and change `.spec.template.spec.containers[0].image` from `nginx:1.14.2` to `nginx:1.16.1`:
|
||||
|
||||
```shell
|
||||
kubectl edit deployment.v1.apps/nginx-deployment
|
||||
@@ -265,7 +265,7 @@ up to 3 replicas, as well as scaling down the old ReplicaSet to 0 replicas.
|
||||
Labels: app=nginx
|
||||
Containers:
|
||||
nginx:
|
||||
Image: nginx:1.9.1
|
||||
Image: nginx:1.16.1
|
||||
Port: 80/TCP
|
||||
Environment: <none>
|
||||
Mounts: <none>
|
||||
@@ -306,11 +306,11 @@ If you update a Deployment while an existing rollout is in progress, the Deploym
|
||||
as per the update and start scaling that up, and rolls over the ReplicaSet that it was scaling up previously
|
||||
-- it will add it to its list of old ReplicaSets and start scaling it down.
|
||||
|
||||
For example, suppose you create a Deployment to create 5 replicas of `nginx:1.7.9`,
|
||||
but then update the Deployment to create 5 replicas of `nginx:1.9.1`, when only 3
|
||||
replicas of `nginx:1.7.9` had been created. In that case, the Deployment immediately starts
|
||||
killing the 3 `nginx:1.7.9` Pods that it had created, and starts creating
|
||||
`nginx:1.9.1` Pods. It does not wait for the 5 replicas of `nginx:1.7.9` to be created
|
||||
For example, suppose you create a Deployment to create 5 replicas of `nginx:1.14.2`,
|
||||
but then update the Deployment to create 5 replicas of `nginx:1.16.1`, when only 3
|
||||
replicas of `nginx:1.14.2` had been created. In that case, the Deployment immediately starts
|
||||
killing the 3 `nginx:1.14.2` Pods that it had created, and starts creating
|
||||
`nginx:1.16.1` Pods. It does not wait for the 5 replicas of `nginx:1.14.2` to be created
|
||||
before changing course.
|
||||
|
||||
### Label selector updates
|
||||
@@ -347,10 +347,10 @@ This means that when you roll back to an earlier revision, only the Deployment's
|
||||
rolled back.
|
||||
{{< /note >}}
|
||||
|
||||
* Suppose that you made a typo while updating the Deployment, by putting the image name as `nginx:1.91` instead of `nginx:1.9.1`:
|
||||
* Suppose that you made a typo while updating the Deployment, by putting the image name as `nginx:1.161` instead of `nginx:1.16.1`:
|
||||
|
||||
```shell
|
||||
kubectl set image deployment.v1.apps/nginx-deployment nginx=nginx:1.91 --record=true
|
||||
kubectl set image deployment.v1.apps/nginx-deployment nginx=nginx:1.161 --record=true
|
||||
```
|
||||
|
||||
The output is similar to this:
|
||||
@@ -427,7 +427,7 @@ rolled back.
|
||||
Labels: app=nginx
|
||||
Containers:
|
||||
nginx:
|
||||
Image: nginx:1.91
|
||||
Image: nginx:1.161
|
||||
Port: 80/TCP
|
||||
Host Port: 0/TCP
|
||||
Environment: <none>
|
||||
@@ -468,13 +468,13 @@ Follow the steps given below to check the rollout history:
|
||||
deployments "nginx-deployment"
|
||||
REVISION CHANGE-CAUSE
|
||||
1 kubectl apply --filename=https://k8s.io/examples/controllers/nginx-deployment.yaml --record=true
|
||||
2 kubectl set image deployment.v1.apps/nginx-deployment nginx=nginx:1.9.1 --record=true
|
||||
3 kubectl set image deployment.v1.apps/nginx-deployment nginx=nginx:1.91 --record=true
|
||||
2 kubectl set image deployment.v1.apps/nginx-deployment nginx=nginx:1.16.1 --record=true
|
||||
3 kubectl set image deployment.v1.apps/nginx-deployment nginx=nginx:1.161 --record=true
|
||||
```
|
||||
|
||||
`CHANGE-CAUSE` is copied from the Deployment annotation `kubernetes.io/change-cause` to its revisions upon creation. You can specify the`CHANGE-CAUSE` message by:
|
||||
|
||||
* Annotating the Deployment with `kubectl annotate deployment.v1.apps/nginx-deployment kubernetes.io/change-cause="image updated to 1.9.1"`
|
||||
* Annotating the Deployment with `kubectl annotate deployment.v1.apps/nginx-deployment kubernetes.io/change-cause="image updated to 1.16.1"`
|
||||
* Append the `--record` flag to save the `kubectl` command that is making changes to the resource.
|
||||
* Manually editing the manifest of the resource.
|
||||
|
||||
@@ -488,10 +488,10 @@ Follow the steps given below to check the rollout history:
|
||||
deployments "nginx-deployment" revision 2
|
||||
Labels: app=nginx
|
||||
pod-template-hash=1159050644
|
||||
Annotations: kubernetes.io/change-cause=kubectl set image deployment.v1.apps/nginx-deployment nginx=nginx:1.9.1 --record=true
|
||||
Annotations: kubernetes.io/change-cause=kubectl set image deployment.v1.apps/nginx-deployment nginx=nginx:1.16.1 --record=true
|
||||
Containers:
|
||||
nginx:
|
||||
Image: nginx:1.9.1
|
||||
Image: nginx:1.16.1
|
||||
Port: 80/TCP
|
||||
QoS Tier:
|
||||
cpu: BestEffort
|
||||
@@ -549,7 +549,7 @@ Follow the steps given below to rollback the Deployment from the current version
|
||||
CreationTimestamp: Sun, 02 Sep 2018 18:17:55 -0500
|
||||
Labels: app=nginx
|
||||
Annotations: deployment.kubernetes.io/revision=4
|
||||
kubernetes.io/change-cause=kubectl set image deployment.v1.apps/nginx-deployment nginx=nginx:1.9.1 --record=true
|
||||
kubernetes.io/change-cause=kubectl set image deployment.v1.apps/nginx-deployment nginx=nginx:1.16.1 --record=true
|
||||
Selector: app=nginx
|
||||
Replicas: 3 desired | 3 updated | 3 total | 3 available | 0 unavailable
|
||||
StrategyType: RollingUpdate
|
||||
@@ -559,7 +559,7 @@ Follow the steps given below to rollback the Deployment from the current version
|
||||
Labels: app=nginx
|
||||
Containers:
|
||||
nginx:
|
||||
Image: nginx:1.9.1
|
||||
Image: nginx:1.16.1
|
||||
Port: 80/TCP
|
||||
Host Port: 0/TCP
|
||||
Environment: <none>
|
||||
@@ -722,7 +722,7 @@ apply multiple fixes in between pausing and resuming without triggering unnecess
|
||||
|
||||
* Then update the image of the Deployment:
|
||||
```shell
|
||||
kubectl set image deployment.v1.apps/nginx-deployment nginx=nginx:1.9.1
|
||||
kubectl set image deployment.v1.apps/nginx-deployment nginx=nginx:1.16.1
|
||||
```
|
||||
|
||||
The output is similar to this:
|
||||
|
||||
@@ -39,7 +39,7 @@ It takes around 10s to complete.
|
||||
You can run the example with this command:
|
||||
|
||||
```shell
|
||||
kubectl apply -f https://k8s.io/examples/controllers/job.yaml
|
||||
kubectl apply -f https://kubernetes.io/examples/controllers/job.yaml
|
||||
```
|
||||
```
|
||||
job.batch/pi created
|
||||
|
||||
@@ -122,10 +122,10 @@ spec:
|
||||
initContainers:
|
||||
- name: init-myservice
|
||||
image: busybox:1.28
|
||||
command: ['sh', '-c', 'until nslookup myservice; do echo waiting for myservice; sleep 2; done;']
|
||||
command: ['sh', '-c', "until nslookup myservice.$(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace).svc.cluster.local; do echo waiting for myservice; sleep 2; done"]
|
||||
- name: init-mydb
|
||||
image: busybox:1.28
|
||||
command: ['sh', '-c', 'until nslookup mydb; do echo waiting for mydb; sleep 2; done;']
|
||||
command: ['sh', '-c', "until nslookup mydb.$(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace).svc.cluster.local; do echo waiting for mydb; sleep 2; done"]
|
||||
```
|
||||
|
||||
You can start this Pod by running:
|
||||
|
||||
@@ -175,7 +175,7 @@ An example flow:
|
||||
1. The Pod in the API server is updated with the time beyond which the Pod is considered "dead" along with the grace period.
|
||||
1. Pod shows up as "Terminating" when listed in client commands
|
||||
1. (simultaneous with 3) When the Kubelet sees that a Pod has been marked as terminating because the time in 2 has been set, it begins the Pod shutdown process.
|
||||
1. If one of the Pod's containers has defined a [preStop hook](/docs/concepts/containers/container-lifecycle-hooks/#hook-details), it is invoked inside of the container. If the `preStop` hook is still running after the grace period expires, step 2 is then invoked with a small (2 second) extended grace period.
|
||||
1. If one of the Pod's containers has defined a [preStop hook](/docs/concepts/containers/container-lifecycle-hooks/#hook-details), it is invoked inside of the container. If the `preStop` hook is still running after the grace period expires, step 2 is then invoked with a small (2 second) one-time extended grace period. You must modify `terminationGracePeriodSeconds` if the `preStop` hook needs longer to complete.
|
||||
1. The container is sent the TERM signal. Note that not all containers in the Pod will receive the TERM signal at the same time and may each require a `preStop` hook if the order in which they shut down matters.
|
||||
1. (simultaneous with 3) Pod is removed from endpoints list for service, and are no longer considered part of the set of running Pods for replication controllers. Pods that shutdown slowly cannot continue to serve traffic as load balancers (like the service proxy) remove them from their rotations.
|
||||
1. When the grace period expires, any processes still running in the Pod are killed with SIGKILL.
|
||||
|
||||
Reference in New Issue
Block a user