Release docs for Kubernetes 1.11 (#9171)
* Seperate priority and preemption (#8144) * Doc about PID pressure condition. (#8211) * Doc about PID pressure condition. Signed-off-by: Da K. Ma <klaus1982.cn@gmail.com> * "so" -> "too" * Update version selector for 1.11 * StorageObjectInUseProtection is GA (#8291) * Feature gate: StorageObjectInUseProtection is GA Update feature gate reference for 1.11 * Trivial commit to re-trigger Netlify * CRIContainerLogRotation is Beta in 1.11 (#8665) * Seperate priority and preemption (#8144) * CRIContainerLogRotation is Beta in 1.11 xref: kubernetes/kubernetes#64046 * Bring StorageObjectInUseProtection feature to GA (#8159) * StorageObjectInUseProtection is GA (#8291) * Feature gate: StorageObjectInUseProtection is GA Update feature gate reference for 1.11 * Trivial commit to re-trigger Netlify * Bring StorageObjectInUseProtection feature to GA StorageObjectInUseProtection is Beta in K8s 1.10. It's brought to GA in K8s 1.11. * Fixed typo and added feature state tags. * Remove KUBE_API_VERSIONS doc (#8292) The support to the KUBER_API_VERSIONS environment variable is completely dropped (no deprecation). This PR removes the related doc in release-1.11. xref: kubernetes/kubernetes#63165 * Remove InitialResources from admission controllers (#8293) The feature (was experimental) is dropped in 1.11. xref: kubernetes/kubernetes#58784 * Remove docs related to in-tree support to GPU (#8294) * Remove docs related to in-tree support to GPU The in-tree support to GPU is completely removed in release 1.11. This PR removes the related docs in release-1.11 branch. xref: kubernetes/kubernetes#61498 * Update content updated by PR to Hugo syntax Signed-off-by: Misty Stanley-Jones <mistyhacks@google.com> * Update the doc about extra volume in kubeadm config (#8453) Signed-off-by: Xianglin Gao <xianglin.gxl@alibaba-inc.com> * Update CRD Subresources for 1.11 (#8519) * coredns: update notes in administer-cluster/coredns.md (#8697) CoreDNS is installed by default in 1.11. Add notes on how to install kube-dns instead. Update notes about CoreDNS->CoreDNS upgrades as in 1.11 the Corefile is retained. Add example on upgrading from kube-dns to CoreDNS. * kubeadm-alpha: CoreDNS related changes (#8727) Update note about CoreDNS feature gate. This change also updates a tab as a kubeadm sub-command will change. It looks for a new generated file: generated/kubeadm_alpha_phase_addon_coredns.md instead of: generated/kubeadm_alpha_phase_addon_kube-dns.md * Update cloud controller manager docs to beta 1.11 (#8756) * Update cloud controller manager docs to beta 1.11 * Use Hugo shortcode for feature state * kubeadm-upgrade: include new command `kubeadm upgrade diff` (#8617) Also: - Include note that this was added in 1.11. - Modify the note about upgrade guidance. * independent: update CoreDNS mentions for kubeadm (#8753) Give CoreDNS instead of kube-dns examples in: - docs/setup/independent/create-cluster-kubeadm.md - docs/setup/independent/troubleshooting-kubeadm.md * update 1.11 --server-print info (#8870) * update 1.11 --server-print info * Copyedit * Mark ExpandPersistentVolumes feature to beta (#8778) * Update version selector for 1.11 * Mark ExpandPersistentVolumes Beta xref: kubernetes/kubernetes#64288 * fix shortcode, add placeholder files to fix deploy failures (#8874) * declare ipvs ga (#8850) * kubeadm: update info about CoreDNS in kubeadm-init.md (#8728) Add info to install kube-dns instead of CoreDNS, as CoreDNS is the default DNS server in 1.11. Add notes that kubeadm config images can be used to list and pull the required images in 1.11. * kubeadm: update implementation-details.md about CoreDNS (#8829) - Replace examples from kube-dns to CoreDNS - Add notes about the CoreDNS feature gate status in 1.11 - Add note that the service name for CoreDNS is also called `kube-dns` * Update block device support for 1.11 (#8895) * Update block device support for 1.11 * Copyedits * Fix typo 'fiber channel' (#8957) Signed-off-by: Misty Stanley-Jones <mistyhacks@google.com> * kubeadm-upgrade: add the 'node [config]' sub-command (#8960) - Add includes for the generated pages - Include placeholder generated pages * kubeadm-init: update the example for the MasterConfiguration (#8958) - include godocs link for MasterConfiguration - include example MasterConfiguration - add note that `kubeadm config print-default` can be used * kubeadm-config: include new commands (#8862) Add notes and includes for these new commands in 1.11: - kubeadm config print-default - kubeadm config migrate - kubeadm config images list - kubeadm config images pull Include placeholder generated files for the above. * administer-cluster/coredns: include more changes (#8985) It was requested that for this page a couple of methods should be outlined: - manual installation for CoreDNS explained at the Kubernetes section of the GitHub project for CoreDNS - installation and upgrade via kubeadm Make the above changes and also add a section "About CoreDNS". This commit also lowercases a section title. * Update CRD subresources doc for 1.11 (#8918) * Add docs for volume expansion and online resizing (#8896) * Add docs for volume expansion going beta * Copyedit * Address feedback * Update exec plugin docs with TLS credentials (#8826) * Update exec plugin docs with TLS credentials kubernetes/kubernetes#61803 implements TLS client credential support for 1.11. * Copyedit * More copyedits for clarification * Additional copyedit * Change token->credential * NodeRestriction admission prevents kubelet taint removal (#8911) * dns-custom-namerserver: break down the page into mutliple sections (#8900) * dns-custom-namerserver: break down the page into mutliple sections This page is currently about kube-dns and is a bit outdated. Introduce the heading `# Customizing kube-dns`. Introduce a separate section about CoreDNS. * Copyedits, fix headings for customizing DNS Hey Lubomir, I coypedited pretty heavily because this workflow is so much easier for docs and because I'm trying to help improve everything touching kubeadm as much as possible. But there's one outstanding issue wrt headings and intro content: you can't add a heading 1 to a topic to do what you wanted to do. The page title in the front matter is rendered as a heading 1 and everything else has to start at heading 2. (We still need to doc this better in the docs contributing content, I know.) Instead, I think we need to rewrite the top-of-page intro content to explain better the relationship between kube-dns and CoreDNS. I'm happy to write something, but I thought I'd push this commit first so you can see what I'm doing. Hope it's all clear -- ping here or on Slack with any questions ~ Jennifer * Interim fix for talking about CoreDNS * Fix CoreDNS details * PSP readOnly hostPath (#8898) * Add documentation for crictl (#8880) * Add documentation for crictl * Copyedit Signed-off-by: Misty Stanley-Jones <mistyhacks@google.com> * Final copyedit * VolumeSubpathEnvExpansion alpha feature (#8835) * Note that Heapster is deprecated (#8827) * Note that Heapster is deprecated This notes that Heapster is deprecated, and migrates the relevant docs to talk about metrics-server or other solutions by default. * Copyedits and improvements Signed-off-by: Misty Stanley-Jones <mistyhacks@google.com> * Address feedback * fix shortcode to troubleshoot deploy (#9057) * update dynamic kubelet config docs for v1.11 (#8766) * update dynamic kubelet config docs for v1.11 * Substantial copyedit * Address feedback * Reference doc for kubeadm (release-1.11) (#9044) * Reference doc for kubeadm (release-1.11) * fix shortcode to troubleshoot deploy (#9057) * Reference doc for kube-components (release-1.11) (#9045) * Reference doc for kube-components (release-1.11) * Update cloud-controller-manager.md * fix shortcode to troubleshoot deploy (#9057) * Documentation on lowercasing kubeadm init apiserver SANs (#9059) * Documentation on lowercasing kubeadm init apiserver SANs * fix shortcode to troubleshoot deploy (#9057) * Clarification in dynamic Kubelet config doc (#9061) * Promote sysctls to Beta (#8804) * Promote sysctls to Beta * Copyedits Signed-off-by: Misty Stanley-Jones <mistyhacks@google.com> * Review comments * Address feedback * More feedback * kubectl reference docs for 1.11 (#9080) * Update Kubernetes API 1.11 ref docs (#8977) * Update v1alpha1 to v1beta1. * Adjust left nav for 1.11 ref docs. * Trim list of old ref docs. * Update Federation API ref docs for 1.11. (#9064) * Update Federation API ref docs for 1.11. * Add titles. * Update definitions.html * CRD versioning Public Documentation (#8834) * CRD versioning Public Documentation * Copyedit Signed-off-by: Misty Stanley-Jones <mistyhacks@google.com> * Address feedback * More rewrites * Address feedback * Update main CRD page in light of versioning * Reorg CRD docs * Further reorg * Tweak title * CSI documentation update for raw block volume support (#8927) * CSI documetation update for raw block volume support * minor edits for "CSI raw block volume support" Some small grammar and style nits. * minor CSIBlockVolume edits * Update kubectl component ref page for 1.11. (#9094) * Update kubectl component ref page for 1.11. * Add title. Replace stevepe with username. * crd versioning doc: fix nits (#9142) * Update `DynamicKubeletConfig` feature to beta (#9110) xref: kubernetes/kubernetes#64275 * Documentation for dynamic volume limits based on node type (#8871) * add cos for storage limits * Update docs specific for aws and gce * fix some minor things * Update storage-limits.md * Add k8s version to feature-state shortcode * The Doc update for ScheduleDaemonSetPods (#8842) Signed-off-by: Da K. Ma <klaus1982.cn@gmail.com> * Update docs related to PersistentVolumeLabel admission control (#9109) The said admission controller is disabled by default in 1.11 (kubernetes/kubernetes#64326) and scheduled to be removed in future release. * client exec auth: updates for 1.11 (#9154) * Updates HA kubeadm docs (#9066) * Updates HA kubeadm docs Signed-off-by: Chuck Ha <ha.chuck@gmail.com> * kubeadm HA - Add stacked control plane steps * ssh instructions and some typos in the bash scripts Signed-off-by: Chuck Ha <ha.chuck@gmail.com> * Fix typos and copypasta errors * Fix rebase issues * Integrate more changes Signed-off-by: Chuck Ha <ha.chuck@gmail.com> * copyedits, layout and formatting fixes * final copyedits * Adds a sanity check for load balancer connection Signed-off-by: Chuck Ha <ha.chuck@gmail.com> * formatting fixes, copyedits * fix typos, formatting * Document the Pod Ready++ feature (#9180) Closes: #9107 Xref: kubernetes/kubernetes#64057 * Mention 'KubeletPluginsWatcher' feature (#9177) * Mention 'KubeletPluginsWatcher' feature This feature is more developers oriented than users oriented, so simply mention it in the feature gate should be fine. In future, when the design doc is migrated from Google doc to the kubernetes/community repo, we can add links to it for users who want to dig deeper. Closes: #9108 Xref: kubernetes/kubernetes#63328, kubernetes/kubernetes#64605 * Copyedit * Amend dynamic volume list docs (#9181) The dynamic volume list feature has been documented but the feature gate related was not there yet. Closes: #9105 * Document for service account projection (#9182) This adds docs for the service account projection feature. Xref: kubernetes/kubernetes#63819, kubernetes/community#1973 Closes: #9102 * Update pod priority and preemption user docs (#9172) * Update pod priority and preemption user docs * Copyedit * Documentation on setting node name with Kubeadm (#8925) * Documentation on setting node name with Kubeadm * copyedit * Add kubeadm upgrade docs for 1.11 (#9089) * Add kubeadm upgrade docs for 1.11 * Initial docs review feedback * Add 1-11 to outline * Fix formatting on tab blocks * Move file to correct location * Add `kubeadm upgrade node config` step * Overzealous ediffing * copyedit, fix lists and headings * clarify --force flag for fixing bad state * Get TOML ready for 1.11 release * Blog post for 1.11 release (#9254) * Blog post for 1.11 release * Update 2018-06-26-kubernetes-1.11-release-announcement.md * Update 2018-06-26-kubernetes-1.11-release-announcement.md * Update 2018-06-26-kubernetes-1.11-release-announcement.md
This commit is contained in:
Misty Linville
committed by
k8s-ci-robot
k8s-ci-robot
parent
e091e29a5d
commit
5f30e7d4c3
@@ -51,6 +51,7 @@ The `conditions` field describes the status of all `Running` nodes.
|
||||
| `OutOfDisk` | `True` if there is insufficient free space on the node for adding new pods, otherwise `False` |
|
||||
| `Ready` | `True` if the node is healthy and ready to accept pods, `False` if the node is not healthy and is not accepting pods, and `Unknown` if the node controller has not heard from the node in the last `node-monitor-grace-period` (default is 40 seconds) |
|
||||
| `MemoryPressure` | `True` if pressure exists on the node memory -- that is, if the node memory is low; otherwise `False` |
|
||||
| `PIDPressure` | `True` if pressure exists on the processes -- that is, if there are too many processes on the node; otherwise `False` |
|
||||
| `DiskPressure` | `True` if pressure exists on the disk size -- that is, if the disk capacity is low; otherwise `False` |
|
||||
| `NetworkUnavailable` | `True` if the network for the node is not correctly configured, otherwise `False` |
|
||||
| `ConfigOK` | `True` if the kubelet is correctly configured, otherwise `False` |
|
||||
|
||||
@@ -144,9 +144,7 @@ When using Docker:
|
||||
multiplied by 100. The resulting value is the total amount of CPU time that a container can use
|
||||
every 100ms. A container cannot use more than its share of CPU time during this interval.
|
||||
|
||||
{{< note >}}
|
||||
**Note**: The default quota period is 100ms. The minimum resolution of CPU quota is 1ms.
|
||||
{{< /note >}}
|
||||
{{< note >}}**Note**: The default quota period is 100ms. The minimum resolution of CPU quota is 1ms.{{ {{</ note >}}}
|
||||
|
||||
- The `spec.containers[].resources.limits.memory` is converted to an integer, and
|
||||
used as the value of the
|
||||
@@ -209,12 +207,10 @@ $ kubectl describe nodes e2e-test-minion-group-4lw4
|
||||
Name: e2e-test-minion-group-4lw4
|
||||
[ ... lines removed for clarity ...]
|
||||
Capacity:
|
||||
alpha.kubernetes.io/nvidia-gpu: 0
|
||||
cpu: 2
|
||||
memory: 7679792Ki
|
||||
pods: 110
|
||||
Allocatable:
|
||||
alpha.kubernetes.io/nvidia-gpu: 0
|
||||
cpu: 1800m
|
||||
memory: 7474992Ki
|
||||
pods: 110
|
||||
@@ -300,10 +296,10 @@ Container in the Pod was terminated and restarted five times.
|
||||
You can call `kubectl get pod` with the `-o go-template=...` option to fetch the status
|
||||
of previously terminated Containers:
|
||||
|
||||
```shell
|
||||
```shell{% raw %}
|
||||
[13:59:01] $ kubectl get pod -o go-template='{{range.status.containerStatuses}}{{"Container Name: "}}{{.name}}{{"\r\nLastState: "}}{{.lastState}}{{end}}' simmemleak-hra99
|
||||
Container Name: simmemleak
|
||||
LastState: map[terminated:map[exitCode:137 reason:OOM Killed startedAt:2015-07-07T20:58:43Z finishedAt:2015-07-07T20:58:43Z containerID:docker://0e4095bba1feccdfe7ef9fb6ebffe972b4b14285d5acdec6f0d3ae8a22fad8b2]]
|
||||
LastState: map[terminated:map[exitCode:137 reason:OOM Killed startedAt:2015-07-07T20:58:43Z finishedAt:2015-07-07T20:58:43Z containerID:docker://0e4095bba1feccdfe7ef9fb6ebffe972b4b14285d5acdec6f0d3ae8a22fad8b2]]{% endraw %}
|
||||
```
|
||||
|
||||
You can see that the Container was terminated because of `reason:OOM Killed`,
|
||||
@@ -545,6 +541,4 @@ consistency across providers and platforms.
|
||||
|
||||
* [ResourceRequirements](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#resourcerequirements-v1-core)
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
|
||||
{{% /capture %}}
|
||||
@@ -9,55 +9,92 @@ weight: 70
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
{{< feature-state state="alpha" >}}
|
||||
{{< feature-state for_k8s_version="1.8" state="alpha" >}}
|
||||
{{< feature-state for_k8s_version="1.11" state="beta" >}}
|
||||
|
||||
[Pods](/docs/user-guide/pods) in Kubernetes 1.8 and later can have priority. Priority
|
||||
indicates the importance of a Pod relative to other Pods. When a Pod cannot be scheduled,
|
||||
[Pods](/docs/user-guide/pods) can have _priority_. Priority
|
||||
indicates the importance of a Pod relative to other Pods. If a Pod cannot be scheduled,
|
||||
the scheduler tries to preempt (evict) lower priority Pods to make scheduling of the
|
||||
pending Pod possible. In Kubernetes 1.9 and later, Priority also affects scheduling
|
||||
pending Pod possible.
|
||||
|
||||
In Kubernetes 1.9 and later, Priority also affects scheduling
|
||||
order of Pods and out-of-resource eviction ordering on the Node.
|
||||
|
||||
Pod priority and preemption are moved to beta since Kubernetes 1.11 and are enabled by default in
|
||||
this release and later.
|
||||
|
||||
In Kubernetes versions where Pod priority and preemption is still an alpha-level
|
||||
feature, you need to explicitly enable it. To use these features in the older versions of
|
||||
Kubernetes, follow the instructions in the documentation for your Kubernetes version, by
|
||||
going to the documentation archive version for your Kubernetes version.
|
||||
|
||||
| Kubernetes Version | Priority and Preemption State | Enabled by default |
|
||||
| -------- |:-----:|:----:|
|
||||
| 1.8 | alpha | no |
|
||||
| 1.9 | alpha | no |
|
||||
| 1.10 | alpha | no |
|
||||
| 1.11 | beta | yes |
|
||||
|
||||
{{< warning >}}
|
||||
**Warning**: In a cluster where not all users are trusted, a malicious
|
||||
user could create pods at the highest possible priorities, causing
|
||||
other pods to be evicted/not get scheduled. To resolve this issue,
|
||||
[ResourceQuota](https://kubernetes.io/docs/concepts/policy/resource-quotas/) is augmented to support
|
||||
Pod priority. An admin can create ResourceQuota for users at specific priority levels, preventing
|
||||
them from creating pods at high priorities. However, this feature is in alpha as of Kubernetes 1.11.
|
||||
{{< /warning >}}
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture body %}}
|
||||
|
||||
## How to use priority and preemption
|
||||
To use priority and preemption in Kubernetes 1.8 and later, follow these steps:
|
||||
To use priority and preemption in Kubernetes 1.11 and later, follow these steps:
|
||||
|
||||
1. Enable the feature.
|
||||
1. Add one or more [PriorityClasses](#priorityclass).
|
||||
|
||||
1. Add one or more PriorityClasses.
|
||||
|
||||
1. Create Pods with `priorityClassName` set to one of the added PriorityClasses.
|
||||
1. Create Pods with[`priorityClassName`](#pod-priority) set to one of the added PriorityClasses.
|
||||
Of course you do not need to create the Pods directly; normally you would add
|
||||
`priorityClassName` to the Pod template of a collection object like a Deployment.
|
||||
|
||||
The following sections provide more information about these steps.
|
||||
|
||||
## Enabling priority and preemption
|
||||
|
||||
Pod priority and preemption is disabled by default in Kubernetes 1.8.
|
||||
To enable the feature, set this command-line flag for the API server, scheduler and kubelet:
|
||||
|
||||
```
|
||||
--feature-gates=PodPriority=true
|
||||
```
|
||||
|
||||
Also enable scheduling.k8s.io/v1alpha1 API and Priority [admission controller](/docs/admin/admission-controllers/) in API server:
|
||||
|
||||
|
||||
```
|
||||
--runtime-config=scheduling.k8s.io/v1alpha1=true --enable-admission-plugins=Controller-Foo,Controller-Bar,...,Priority
|
||||
```
|
||||
|
||||
After the feature is enabled, you can create [PriorityClasses](#priorityclass)
|
||||
and create Pods with [`priorityClassName`](#pod-priority) set.
|
||||
Keep reading for more information about these steps.
|
||||
|
||||
If you try the feature and then decide to disable it, you must remove the PodPriority
|
||||
command-line flag or set it to false, and then restart the API server and
|
||||
command-line flag or set it to `false`, and then restart the API server and
|
||||
scheduler. After the feature is disabled, the existing Pods keep their priority
|
||||
fields, but preemption is disabled, and priority fields are ignored, and you
|
||||
cannot set `priorityClassName` in new Pods.
|
||||
fields, but preemption is disabled, and priority fields are ignored. If the feature
|
||||
is disabled, you cannot set `priorityClassName` in new Pods.
|
||||
|
||||
## How to disable preemption
|
||||
|
||||
{{< note >}}
|
||||
**Note**: In Kubernetes 1.11, critical pods (except DaemonSet pods, which are
|
||||
still scheduled by the DaemonSet controller) rely on scheduler preemption to be
|
||||
scheduled when a cluster is under resource pressure. For this reason, we do not
|
||||
recommend disabling this feature. If you still have to disable this feature,
|
||||
follow the instructions below.
|
||||
{{< /note >}}
|
||||
|
||||
In Kubernetes 1.11 and later, preemption is controlled by a kube-scheduler flag
|
||||
`disablePreemption`, which is set to `false` by default.
|
||||
|
||||
To disable preemption, set `disablePreemption` to true. This keeps pod priority
|
||||
enabled but disables preemption. Here is a sample configuration:
|
||||
|
||||
```yaml
|
||||
apiVersion: componentconfig/v1alpha1
|
||||
kind: KubeSchedulerConfiguration
|
||||
algorithmSource:
|
||||
provider: DefaultProvider
|
||||
|
||||
...
|
||||
|
||||
disablePreemption: true
|
||||
|
||||
```
|
||||
|
||||
Although preemption of the scheduler is enabled by default, it is disabled if `PodPriority`
|
||||
feature is disabled.
|
||||
|
||||
## PriorityClass
|
||||
|
||||
@@ -80,22 +117,17 @@ with `globalDefault` set, the priority of Pods with no `priorityClassName` is ze
|
||||
The `description` field is an arbitrary string. It is meant to tell users of
|
||||
the cluster when they should use this PriorityClass.
|
||||
|
||||
{{< note >}}
|
||||
**Note 1**: If you upgrade your existing cluster and enable this feature, the priority
|
||||
of your existing Pods will be considered to be zero.
|
||||
{{< /note >}}
|
||||
### Notes about PodPriority and existing clusters
|
||||
- If you upgrade your existing cluster and enable this feature, the priority
|
||||
of your existing Pods is effectively zero.
|
||||
|
||||
{{< note >}}
|
||||
**Note 2**: Addition of a PriorityClass with `globalDefault` set to true does not
|
||||
- Addition of a PriorityClass with `globalDefault` set to `true` does not
|
||||
change the priorities of existing Pods. The value of such a PriorityClass is used only
|
||||
for Pods created after the PriorityClass is added.
|
||||
{{< /note >}}
|
||||
|
||||
{{< note >}}
|
||||
**Note 3**: If you delete a PriorityClass, existing Pods that use the name of the
|
||||
deleted priority class remain unchanged, but you are not able to create more Pods
|
||||
- If you delete a PriorityClass, existing Pods that use the name of the
|
||||
deleted PriorityClass remain unchanged, but you cannot create more Pods
|
||||
that use the name of the deleted PriorityClass.
|
||||
{{< /note >}}
|
||||
|
||||
### Example PriorityClass
|
||||
|
||||
|
||||
@@ -204,23 +204,23 @@ running on the node as follows
|
||||
* pods that tolerate the taint with a specified `tolerationSeconds` remain
|
||||
bound for the specified amount of time
|
||||
|
||||
In addition, Kubernetes 1.6 has alpha
|
||||
support for representing node problems. In other words, the node controller
|
||||
automatically taints a node when certain condition is true. The built-in taints
|
||||
currently include:
|
||||
In addition, Kubernetes 1.6 introduced alpha support for representing node
|
||||
problems. In other words, the node controller automatically taints a node when
|
||||
certain condition is true. The following taints are built in:
|
||||
|
||||
* `node.kubernetes.io/not-ready`: Node is not ready. This corresponds to
|
||||
the NodeCondition `Ready` being "`False`".
|
||||
* `node.alpha.kubernetes.io/unreachable`: Node is unreachable from the node
|
||||
* `node.kubernetes.io/unreachable`: Node is unreachable from the node
|
||||
controller. This corresponds to the NodeCondition `Ready` being "`Unknown`".
|
||||
* `node.kubernetes.io/out-of-disk`: Node becomes out of disk.
|
||||
* `node.kubernetes.io/memory-pressure`: Node has memory pressure.
|
||||
* `node.kubernetes.io/disk-pressure`: Node has disk pressure.
|
||||
* `node.kubernetes.io/network-unavailable`: Node's network is unavailable.
|
||||
* `node.cloudprovider.kubernetes.io/uninitialized`: When kubelet is started
|
||||
with "external" cloud provider, it sets this taint on a node to mark it
|
||||
as unusable. When a controller from the cloud-controller-manager initializes
|
||||
this node, kubelet removes this taint.
|
||||
* `node.kubernetes.io/unschedulable`: Node is unschedulable.
|
||||
* `node.cloudprovider.kubernetes.io/uninitialized`: When the kubelet is started
|
||||
with "external" cloud provider, this taint is set on a node to mark it
|
||||
as unusable. After a controller from the cloud-controller-manager initializes
|
||||
this node, the kubelet removes this taint.
|
||||
|
||||
When the `TaintBasedEvictions` alpha feature is enabled (you can do this by
|
||||
including `TaintBasedEvictions=true` in `--feature-gates` for Kubernetes controller manager,
|
||||
@@ -277,17 +277,15 @@ Version 1.8 introduces an alpha feature that causes the node controller to creat
|
||||
Node conditions. When this feature is enabled (you can do this by including `TaintNodesByCondition=true` in the `--feature-gates` command line flag to the scheduler, such as
|
||||
`--feature-gates=FooBar=true,TaintNodesByCondition=true`), the scheduler does not check Node conditions; instead the scheduler checks taints. This assures that Node conditions don't affect what's scheduled onto the Node. The user can choose to ignore some of the Node's problems (represented as Node conditions) by adding appropriate Pod tolerations.
|
||||
|
||||
To make sure that turning on this feature doesn't break DaemonSets, starting in version 1.8, the DaemonSet controller automatically adds the following `NoSchedule` tolerations to all daemons:
|
||||
Starting in Kubernetes 1.8, the DaemonSet controller automatically adds the
|
||||
following `NoSchedule` tolerations to all daemons, to prevent DaemonSets from
|
||||
breaking.
|
||||
|
||||
* `node.kubernetes.io/memory-pressure`
|
||||
* `node.kubernetes.io/disk-pressure`
|
||||
* `node.kubernetes.io/out-of-disk` (*only for critical pods*)
|
||||
* `node.kubernetes.io/unschedulable` (1.10 or later)
|
||||
* `node.kubernetes.io/network-unavailable` (*host network only*)
|
||||
|
||||
The above settings ensure backward compatibility, but we understand they may not fit all user's needs, which is why
|
||||
cluster admin may choose to add arbitrary tolerations to DaemonSets.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture whatsnext %}}
|
||||
|
||||
{{% /capture %}}
|
||||
Adding these tolerations ensures backward compatibility. You can also add
|
||||
arbitrary tolerations to DaemonSets.
|
||||
|
||||
@@ -8,7 +8,11 @@ weight: 20
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
This page explains [*custom resources*](/docs/concepts/api-extension/custom-resources/), which are extensions of the Kubernetes API. This page explains when to add a custom resource to your Kubernetes cluster and when to use a standalone service. It describes the two methods for adding custom resources and how to choose between them.
|
||||
|
||||
This page explains *custom resources*, which are extensions of the Kubernetes
|
||||
API, including when to add a custom resource to your Kubernetes cluster and when
|
||||
to use a standalone service. It describes the two methods for adding custom
|
||||
resources and how to choose between them.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
@@ -103,20 +107,20 @@ Use a custom resource (CRD or Aggregated API) if most of the following apply:
|
||||
|
||||
Kubernetes provides two ways to add custom resources to your cluster:
|
||||
|
||||
- [Custom Resource Definitions](/docs/concepts/api-extension/custom-resources/) (CRDs) are easier to use: they do not require any programming in some cases.
|
||||
- CRDs are simple and can be created without any programming.
|
||||
- [API Aggregation](/docs/concepts/api-extension/apiserver-aggregation/) requires programming, but allows more control over API behaviors like how data is stored and conversion between API versions.
|
||||
|
||||
Kubernetes provides these two options to meet the needs of different users, so that neither ease of use nor flexibility are compromised.
|
||||
|
||||
Aggregated APIs are subordinate APIServers that sit behind the primary API server, which acts as a proxy. This arrangement is called [API Aggregation](/docs/concepts/api-extension/apiserver-aggregation/) (AA). To users, it simply appears that the Kubernetes API is extended.
|
||||
|
||||
Custom Resource Definitions (CRDS) allow users to create new types of resources without adding another APIserver. You do not need to understand API Aggregation to use CRDs.
|
||||
CRDs allow users to create new types of resources without adding another APIserver. You do not need to understand API Aggregation to use CRDs.
|
||||
|
||||
Regardless of whether they are installed via CRDs or AA, the new resources are called Custom Resources to distinguish them from built-in Kubernetes resources (like pods).
|
||||
Regardless of how they are installed, the new resources are referred to as Custom Resources to distinguish them from built-in Kubernetes resources (like pods).
|
||||
|
||||
## CustomResourceDefinitions
|
||||
|
||||
The [CustomResourceDefinition](/docs/tasks/access-kubernetes-api/extend-api-custom-resource-definitions/) (CRD) API resource allows you to define custom resources. Defining a CRD object creates a new custom resource with a name and schema that you specify. The Kubernetes API serves and handles the storage of your custom resource.
|
||||
The [CustomResourceDefinition](/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions/) API resource allows you to define custom resources. Defining a CRD object creates a new custom resource with a name and schema that you specify. The Kubernetes API serves and handles the storage of your custom resource.
|
||||
|
||||
This frees you from writing your own API server to handle the custom resource,
|
||||
but the generic nature of the implementation means you have less flexibility than with
|
||||
@@ -132,7 +136,7 @@ and setup a controller to handle events.
|
||||
|
||||
## API server aggregation
|
||||
|
||||
Usually, each resource in the Kubernetes API requires code that handles REST requests and manages persistent storage of objects. The main Kubernetes API server handles built-in resources like *pods* and *services*, and can also handle custom resources in a generic way through [CustomResourceDefinitions](#customresourcedefinitions).
|
||||
Usually, each resource in the Kubernetes API requires code that handles REST requests and manages persistent storage of objects. The main Kubernetes API server handles built-in resources like *pods* and *services*, and can also handle custom resources in a generic way through [CRDs](#customresourcedefinitions).
|
||||
|
||||
The [aggregation layer](/docs/concepts/api-extension/apiserver-aggregation/) allows you to provide specialized
|
||||
implementations for your custom resources by writing and deploying your own standalone API server.
|
||||
@@ -152,7 +156,7 @@ Typically, CRDs are a good fit if:
|
||||
|
||||
CRDs are easier to create than Aggregated APIs.
|
||||
|
||||
| Custom Resource Definitions | Aggregated API |
|
||||
| CRDs | Aggregated API |
|
||||
| --------------------------- | -------------- |
|
||||
| Do not require programming. Users can choose any language for a CRD controller. | Requires programming in Go and building binary and image. Users can choose any language for a CRD controller. |
|
||||
| No additional service to run; CRs are handled by API Server. | An additional service to create and that could fail. |
|
||||
|
||||
@@ -421,7 +421,8 @@ minimum value of the first range as the default. Validates against all ranges.
|
||||
to be used by hostPath volumes. An empty list means there is no restriction on
|
||||
host paths used. This is defined as a list of objects with a single `pathPrefix`
|
||||
field, which allows hostPath volumes to mount a path that begins with an
|
||||
allowed prefix. For example:
|
||||
allowed prefix, and a `readOnly` field indicating it must be mounted read-only.
|
||||
For example:
|
||||
|
||||
```yaml
|
||||
allowedHostPaths:
|
||||
@@ -429,11 +430,18 @@ allowedHostPaths:
|
||||
# disallows "/fool", "/etc/foo" etc.
|
||||
# "/foo/../" is never valid.
|
||||
- pathPrefix: "/foo"
|
||||
readOnly: true # only allow read-only mounts
|
||||
```
|
||||
|
||||
_Note: There are many ways a container with unrestricted access to the host
|
||||
{{< warning >}}**Warning:** There are many ways a container with unrestricted access to the host
|
||||
filesystem can escalate privileges, including reading data from other
|
||||
containers, and abusing the credentials of system services, such as Kubelet._
|
||||
containers, and abusing the credentials of system services, such as Kubelet.
|
||||
|
||||
Writeable hostPath directory volumes allow containers to write
|
||||
to the filesystem in ways that let them traverse the host filesystem outside the `pathPrefix`.
|
||||
`readOnly: true`, available in Kubernetes 1.11+, must be used on **all** `allowedHostPaths`
|
||||
to effectively limit access to the specified `pathPrefix`.
|
||||
{{< /warning >}}
|
||||
|
||||
**ReadOnlyRootFilesystem** - Requires that containers must run with a read-only
|
||||
root filesystem (i.e. no writable layer).
|
||||
|
||||
@@ -78,14 +78,13 @@ Pods use claims as volumes. The cluster inspects the claim to find the bound vol
|
||||
Once a user has a claim and that claim is bound, the bound PV belongs to the user for as long as they need it. Users schedule Pods and access their claimed PVs by including a `persistentVolumeClaim` in their Pod's volumes block. [See below for syntax details](#claims-as-volumes).
|
||||
|
||||
### Storage Object in Use Protection
|
||||
{{< feature-state for_k8s_version="v1.10" state="beta" >}}
|
||||
The purpose of the Storage Object in Use Protection feature is to ensure that Persistent Volume Claims (PVCs) in active use by a pod and Persistent Volume (PVs) that are bound to PVCs are not removed from the system as this may result in data loss.
|
||||
|
||||
{{< note >}}
|
||||
**Note:** PVC is in active use by a pod when the pod status is `Pending` and the pod is assigned to a node or the pod status is `Running`.
|
||||
{{< /note >}}
|
||||
|
||||
When the [Storage Object in Use Protection beta feature](/docs/tasks/administer-cluster/storage-object-in-use-protection/) is enabled, if a user deletes a PVC in active use by a pod, the PVC is not removed immediately. PVC removal is postponed until the PVC is no longer actively used by any pods, and also if admin deletes a PV that is bound to a PVC, the PV is not removed immediately. PV removal is postponed until the PV is not bound to a PVC any more.
|
||||
When the [Storage Object in Use Protection feature](/docs/tasks/administer-cluster/storage-object-in-use-protection/) is enabled, if a user deletes a PVC in active use by a pod, the PVC is not removed immediately. PVC removal is postponed until the PVC is no longer actively used by any pods, and also if admin deletes a PV that is bound to a PVC, the PV is not removed immediately. PV removal is postponed until the PV is not bound to a PVC any more.
|
||||
|
||||
You can see that a PVC is protected when the PVC's status is `Terminating` and the `Finalizers` list includes `kubernetes.io/pvc-protection`:
|
||||
|
||||
@@ -176,20 +175,21 @@ However, the particular path specified in the custom recycler pod template in th
|
||||
|
||||
### Expanding Persistent Volumes Claims
|
||||
|
||||
Kubernetes 1.8 added Alpha support for expanding persistent volumes. In v1.9, the following volume types support expanding Persistent volume claims:
|
||||
{{< feature-state for_k8s_version="v1.8" state="alpha" >}}
|
||||
{{< feature-state for_k8s_version="v1.11" state="beta" >}}
|
||||
Support for expanding PersistentVolumeClaims (PVCs) is now enabled by default. You can expand
|
||||
the following types of volumes:
|
||||
|
||||
* gcePersistentDisk
|
||||
* awsElasticBlockStore
|
||||
* Cinder
|
||||
* glusterfs
|
||||
* rbd
|
||||
* Azure File
|
||||
* Azure Disk
|
||||
* Portworx
|
||||
|
||||
Administrator can allow expanding persistent volume claims by setting `ExpandPersistentVolumes` feature gate to true. Administrator
|
||||
should also enable [`PersistentVolumeClaimResize` admission plugin](/docs/admin/admission-controllers/#persistentvolumeclaimresize)
|
||||
to perform additional validations of volumes that can be resized.
|
||||
|
||||
Once `PersistentVolumeClaimResize` admission plug-in has been turned on, resizing will only be allowed for storage classes
|
||||
whose `allowVolumeExpansion` field is set to true.
|
||||
You can only expand a PVC if its storage class's `allowVolumeExpansion` field is set to true.
|
||||
|
||||
``` yaml
|
||||
kind: StorageClass
|
||||
@@ -205,17 +205,33 @@ parameters:
|
||||
allowVolumeExpansion: true
|
||||
```
|
||||
|
||||
Once both feature gate and the aforementioned admission plug-in are turned on, a user can request larger volume for their `PersistentVolumeClaim`
|
||||
by simply editing the claim and requesting a larger size. This in turn will trigger expansion of the volume that is backing the underlying `PersistentVolume`.
|
||||
To request a larger volume for a PVC, edit the PVC object and specify a larger
|
||||
size. This triggers expansion of the volume that backs the underlying `PersistentVolume`. A
|
||||
new `PersistentVolume` is never created to satisfy the claim. Instead, an existing volume is resized.
|
||||
|
||||
Under no circumstances will a new `PersistentVolume` be created to satisfy the claim. Kubernetes will instead attempt to resize the existing volume.
|
||||
#### Resizing a volume containing a file system
|
||||
|
||||
For expanding volumes containing a file system, file system resizing is only performed when a new Pod is started using the `PersistentVolumeClaim` in
|
||||
ReadWrite mode. In other words, if a volume being expanded is used in a pod or deployment, you will need to delete and recreate the pod for file system
|
||||
resizing to take place. Also, file system resizing is only supported for following file system types:
|
||||
You can only resize volumes containing a file system if the file system is XFS, Ext3, or Ext4.
|
||||
|
||||
* XFS
|
||||
* Ext3, Ext4
|
||||
When a volume contains a file system, the file system is only resized when a new Pod is started using
|
||||
the `PersistentVolumeClaim` in ReadWrite mode. Therefore, if a pod or deployment is using a volume and
|
||||
you want to expand it, you need to delete or recreate the pod after the volume has been exxpanded by the cloud provider in the controller-manager. You can check the status of resize operation by running the `kubectl describe pvc` command:
|
||||
|
||||
```
|
||||
kubectl describe pvc <pvc_name>
|
||||
```
|
||||
|
||||
If the `PersistentVolumeClaim` has the status `FileSystemResizePending`, it is safe to recreate the pod using the PersistentVolumeClaim.
|
||||
|
||||
#### Resizing an in-use PersistentVolumeClaim
|
||||
|
||||
{{< feature-state for_k8s_version="v1.11" state="alpha" >}}
|
||||
|
||||
Expanding in-use PVCs is an alpha feature. To use it, enable the `ExpandInUsePersistentVolumes` feature gate.
|
||||
In this case, you don't need to delete and recreate a Pod or deployment that is using an existing PVC.
|
||||
Any in-use PVC automatically becomes available to its Pod as soon as its file system has been expanded.
|
||||
This feature has no effect on PVCs that are not in use by a Pod or deployment. You must create a Pod which
|
||||
uses the PVC before the expansion can complete.
|
||||
|
||||
{{< note >}}
|
||||
**Note:** Expanding EBS volumes is a time consuming operation. Also, there is a per-volume quota of one modification every 6 hours.
|
||||
@@ -230,7 +246,7 @@ resizing to take place. Also, file system resizing is only supported for followi
|
||||
* AWSElasticBlockStore
|
||||
* AzureFile
|
||||
* AzureDisk
|
||||
* FC (Fibre Channel)**
|
||||
* FC (Fibre Channel)
|
||||
* FlexVolume
|
||||
* Flocker
|
||||
* NFS
|
||||
@@ -246,8 +262,6 @@ resizing to take place. Also, file system resizing is only supported for followi
|
||||
* ScaleIO Volumes
|
||||
* StorageOS
|
||||
|
||||
Raw Block Support exists for these plugins only.
|
||||
|
||||
## Persistent Volumes
|
||||
|
||||
Each PV contains a spec and status, which is the specification and status of the volume.
|
||||
@@ -281,11 +295,14 @@ Currently, storage size is the only resource that can be set or requested. Futu
|
||||
|
||||
### Volume Mode
|
||||
|
||||
Prior to v1.9, the default behavior for all volume plugins was to create a filesystem on the persistent volume. With v1.9, the user can specify a `volumeMode` which will now support raw block devices in addition to file systems. Valid values for `volumeMode` are "Filesystem" or "Block". If left unspecified, `volumeMode` defaults to "Filesystem" internally. This is an optional API parameter.
|
||||
{{< feature-state for_k8s_version="v1.9" state="alpha" >}}
|
||||
|
||||
{{< note >}}
|
||||
**Note:** This feature is alpha in v1.9 and may change in the future.
|
||||
{{< /note >}}
|
||||
To enable this feature, enable the `BlockVolume` feature gate on the apiserver, controller-manager and the kubelet.
|
||||
|
||||
Prior to Kubernetes 1.9, all volume plugins created a filesystem on the persistent volume.
|
||||
Now, you can set the value of `volumeMode` to `raw` to use a raw block device, or `filesystem`
|
||||
to use a filesystem. `filesystem` is the default if the value is omitted. This is an optional API
|
||||
parameter.
|
||||
|
||||
### Access Modes
|
||||
|
||||
@@ -508,7 +525,26 @@ spec:
|
||||
|
||||
## Raw Block Volume Support
|
||||
|
||||
Static provisioning support for Raw Block Volumes is included as an alpha feature for v1.9. With this change are some new API fields that need to be used to facilitate this functionality. Kubernetes v1.10 supports only Fibre Channel and Local Volume plugins for this feature.
|
||||
{{< feature-state for_k8s_version="v1.9" state="alpha" >}}
|
||||
|
||||
To enable support for raw block volumes, enable the `BlockVolume` feature gate on the
|
||||
apiserver, controller-manager and the kubelet.
|
||||
|
||||
The following volume plugins support raw block volumes, including dynamic provisioning where
|
||||
applicable.
|
||||
|
||||
* AWSElasticBlockStore
|
||||
* AzureDisk
|
||||
* FC (Fibre Channel)
|
||||
* GCEPersistentDisk
|
||||
* iSCSI
|
||||
* Local volume
|
||||
* RBD (Ceph Block Device)
|
||||
|
||||
{{< note >}}
|
||||
**Note**: Only FC and iSCSI volumes supported raw block volumes in Kubernetes 1.9.
|
||||
Support for the additional plugins was added in 1.10.
|
||||
{{< /note >}}
|
||||
|
||||
### Persistent Volumes using a Raw Block Volume
|
||||
```yaml
|
||||
|
||||
@@ -0,0 +1,75 @@
|
||||
---
|
||||
reviewers:
|
||||
- jsafrane
|
||||
- saad-ali
|
||||
- thockin
|
||||
- msau42
|
||||
title: Node-specific Volume Limits
|
||||
content_template: templates/concept
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
This page describes the maximum number of volumes that can be attached
|
||||
to a node for various cloud providers.
|
||||
|
||||
Cloud providers like Google, Amazon, and Microsoft typically have a limit on
|
||||
how many volumes can be attached to a node. It is important for Kubernetes to
|
||||
respect those limits. Otherwise, Pods scheduled on a node could get stuck
|
||||
waiting for volumes to attach.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture body %}}
|
||||
|
||||
## Kubernetes default limits
|
||||
|
||||
The Kubernetes scheduler has default limits on the number of volumes
|
||||
that can be attached to a node:
|
||||
|
||||
<table>
|
||||
<tr><th>Cloud service</th><th>Maximum volumes per node</th></tr>
|
||||
<tr><td><a href="https://aws.amazon.com/ebs/">Amazon Elastic Block Store (EBS)</a></td><td>39</td></tr>
|
||||
<tr><td><a href="https://cloud.google.com/persistent-disk/">Google Persistent Disk</a></td><td>16</td></tr>
|
||||
<tr><td><a href="https://azure.microsoft.com/en-us/services/storage/main-disks/">Microsoft Azure Disk Storage</a></td><td>16</td></tr>
|
||||
</table>
|
||||
|
||||
## Custom limits
|
||||
|
||||
You can change these limits by setting the value of the
|
||||
`KUBE_MAX_PD_VOLS` environment variable, and then starting the scheduler.
|
||||
|
||||
Use caution if you set a limit that is higher than the default limit. Consult
|
||||
the cloud provider's documentation to make sure that nodes can actually support
|
||||
the limit you set.
|
||||
|
||||
The limit applies to the entire cluster, so it affects all nodes.
|
||||
|
||||
## Dynamic volume limits
|
||||
|
||||
{{< feature-state state="alpha" for_k8s_version="v1.11" >}}
|
||||
|
||||
Kubernetes 1.11 introduces dynamic volume limits based on node type. This
|
||||
is an alpha feature that supports these services:
|
||||
|
||||
- Amazon EBS
|
||||
- Google Persistent Disk
|
||||
|
||||
To enable dynamic volume limits, set the `AttachVolumeLimit`
|
||||
[feature gate](/docs/reference/command-line-tools-reference/feature-gates/)
|
||||
to True.
|
||||
|
||||
When the dynamic volume limits feature is enabled, Kubernetes automatically
|
||||
determines the node type and supports the appropriate number of attachable
|
||||
volumes for the node. For example:
|
||||
|
||||
* On
|
||||
<a href="https://cloud.google.com/compute/">Google Compute Engine</a>,
|
||||
up to 128 volumes could be attached to a node, depending on the node type.
|
||||
|
||||
* For Amazon EBS disks on M5/C5 instance types, Kubernetes would permit only 25
|
||||
volumes to be attached to a node. For other instance types on
|
||||
<a href="https://aws.amazon.com/ec2/">Amazon Elastic Compute Cloud (EC2)</a>,
|
||||
Kubernetes would permit 39 volumes to be attached.
|
||||
|
||||
{{% /capture %}}
|
||||
@@ -646,8 +646,15 @@ Currently, the following types of volume sources can be projected:
|
||||
- [`secret`](#secret)
|
||||
- [`downwardAPI`](#downwardapi)
|
||||
- [`configMap`](#configmap)
|
||||
- `serviceAccountToken`
|
||||
|
||||
All sources are required to be in the same namespace as the Pod. For more details, see the [all-in-one volume design document](https://github.com/kubernetes/community/blob/{{< param "githubbranch" >}}/contributors/design-proposals/node/all-in-one-volume.md).
|
||||
All sources are required to be in the same namespace as the Pod. For more details,
|
||||
see the [all-in-one volume design document](https://github.com/kubernetes/community/blob/{{< param "githubbranch" >}}/contributors/design-proposals/node/all-in-one-volume.md).
|
||||
|
||||
The projection of service account tokens is a feature introduced in Kubernetes
|
||||
1.11. To enable this feature, you need to explicitly set the `TokenRequestProjection`
|
||||
[feature gate](/docs/reference/command-line-tools-reference/feature-gates/) to
|
||||
True.
|
||||
|
||||
#### Example Pod with a secret, a downward API, and a configmap.
|
||||
|
||||
@@ -730,6 +737,45 @@ parameters are nearly the same with two exceptions:
|
||||
volume source. However, as illustrated above, you can explicitly set the `mode`
|
||||
for each individual projection.
|
||||
|
||||
When the `TokenRequestProjection` feature is enabled, you can inject the token
|
||||
for the current [service account](/docs/reference/access-authn-authz/authentication/#service-account-tokens)
|
||||
into a Pod at a specified path. Below is an example:
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: sa-token-test
|
||||
spec:
|
||||
containers:
|
||||
- name: container-test
|
||||
image: busybox
|
||||
volumeMounts:
|
||||
- name: token-vol
|
||||
mountPath: "/sevice-account"
|
||||
readOnly: true
|
||||
volumes:
|
||||
- name: token-vol
|
||||
projected:
|
||||
sources:
|
||||
- serviceAccountToken:
|
||||
audience: api
|
||||
expirationSeconds: 3600
|
||||
path: token
|
||||
```
|
||||
|
||||
The example Pod has a projected volume containing the injected service account
|
||||
token. This token can be used by Pod containers to access the Kubernetes API
|
||||
server, for example. The `audience` field contains the intended audience of the
|
||||
token. A recipient of the token must identify itself with an identifier specified
|
||||
in the audience of the token, and otherwise should reject the token. This field
|
||||
is optional and it defaults to the identifier of the API server.
|
||||
|
||||
The `expirationSeconds` is the expected duration of validity of the service account
|
||||
token. It defaults to 1 hour and must be at least 10 minutes (600 seconds).
|
||||
The `path` field specifies a relative path to the mount point of the projected
|
||||
volume.
|
||||
|
||||
{{< note >}}
|
||||
**Note:** A Container using a projected volume source as a [subPath](#using-subpath) volume mount will not
|
||||
receive updates for those volume sources.
|
||||
@@ -1018,6 +1064,43 @@ spec:
|
||||
claimName: my-lamp-site-data
|
||||
```
|
||||
|
||||
### Using subPath with expanded environment variables
|
||||
|
||||
{{< feature-state for_k8s_version="v1.11" state="alpha" >}}
|
||||
|
||||
|
||||
`subPath` directory names can also be constructed from Downward API environment variables.
|
||||
Before you use this feature, you must enable the `VolumeSubpathEnvExpansion`feature gate.
|
||||
|
||||
In this example, a Pod uses `subPath` to create a directory `pod1` within the hostPath volume `/var/log/pods`, using the pod name from the Downward API. The host directory `/var/log/pods/pod1` is mounted at `/logs` in the container.
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: pod1
|
||||
spec:
|
||||
containers:
|
||||
- name: container1
|
||||
env:
|
||||
- name: POD_NAME
|
||||
valueFrom:
|
||||
fieldRef:
|
||||
apiVersion: v1
|
||||
fieldPath: metadata.name
|
||||
image: busybox
|
||||
command: [ "sh", "-c", "while [ true ]; do echo 'Hello'; sleep 10; done | tee -a /logs/hello.txt" ]
|
||||
volumeMounts:
|
||||
- name: workdir1
|
||||
mountPath: /logs
|
||||
subPath: $(POD_NAME)
|
||||
restartPolicy: Never
|
||||
volumes:
|
||||
- name: workdir1
|
||||
hostPath:
|
||||
path: /var/log/pods
|
||||
```
|
||||
|
||||
## Resources
|
||||
|
||||
The storage media (Disk, SSD, etc.) of an `emptyDir` volume is determined by the
|
||||
@@ -1116,6 +1199,26 @@ persistent volume:
|
||||
secret is required. If the secret object contains more than one secret, all
|
||||
secrets are passed.
|
||||
|
||||
#### CSI raw block volume support
|
||||
|
||||
{{< feature-state for_k8s_version="v1.11" state="alpha" >}}
|
||||
|
||||
Starting with version 1.11, CSI introduced support for raw block volumes, which
|
||||
relies on the raw block volume feature that was introduced in a previous version of
|
||||
Kubernetes. This feature will make it possible for vendors with external CSI drivers to
|
||||
implement raw block volumes support in Kubernetes workloads.
|
||||
|
||||
CSI block volume support is feature-gated and turned off by default. To run CSI with
|
||||
block volume support enabled, a cluster administrator must enable the feature for each
|
||||
Kubernetes component using the following feature gate flags:
|
||||
|
||||
```
|
||||
--feature-gates=BlockVolume=true,CSIBlockVolume=true
|
||||
```
|
||||
|
||||
Learn how to
|
||||
[setup your PV/PVC with raw block volume support](/docs/concepts/storage/persistent-volumes/#raw-block-volume-support).
|
||||
|
||||
### FlexVolume
|
||||
|
||||
`FlexVolume` is an out-of-tree plugin interface that has existed in Kubernetes
|
||||
|
||||
@@ -103,7 +103,9 @@ If you do not specify either, then the DaemonSet controller will create Pods on
|
||||
|
||||
## How Daemon Pods are Scheduled
|
||||
|
||||
Normally, the machine that a Pod runs on is selected by the Kubernetes scheduler. However, Pods
|
||||
### Scheduled by DaemonSet controller (default)
|
||||
|
||||
Normally, the machine that a Pod runs on is selected by the Kubernetes scheduler. However, Pods
|
||||
created by the DaemonSet controller have the machine already selected (`.spec.nodeName` is specified
|
||||
when the Pod is created, so it is ignored by the scheduler). Therefore:
|
||||
|
||||
@@ -112,29 +114,72 @@ when the Pod is created, so it is ignored by the scheduler). Therefore:
|
||||
- The DaemonSet controller can make Pods even when the scheduler has not been started, which can help cluster
|
||||
bootstrap.
|
||||
|
||||
Daemon Pods do respect [taints and tolerations](/docs/concepts/configuration/taint-and-toleration),
|
||||
but they are created with `NoExecute` tolerations for the following taints with no `tolerationSeconds`:
|
||||
|
||||
- `node.kubernetes.io/not-ready`
|
||||
- `node.alpha.kubernetes.io/unreachable`
|
||||
### Scheduled by default scheduler
|
||||
|
||||
This ensures that when the `TaintBasedEvictions` alpha feature is enabled,
|
||||
they will not be evicted when there are node problems such as a network partition. (When the
|
||||
`TaintBasedEvictions` feature is not enabled, they are also not evicted in these scenarios, but
|
||||
due to hard-coded behavior of the NodeController rather than due to tolerations).
|
||||
{{< feature-state state="alpha" for-kubernetes-version="1.11" >}}
|
||||
|
||||
They also tolerate following `NoSchedule` taints:
|
||||
A DaemonSet ensures that all eligible nodes run a copy of a Pod. Normally, the
|
||||
node that a Pod runs on is selected by the Kubernetes scheduler. However,
|
||||
DaemonSet pods are created and scheduled by the DaemonSet controller instead.
|
||||
That introduces the following issues:
|
||||
|
||||
- `node.kubernetes.io/memory-pressure`
|
||||
- `node.kubernetes.io/disk-pressure`
|
||||
* Inconsistent Pod behavior: Normal Pods waiting to be scheduled are created
|
||||
and in `Pending` state, but DaemonSet pods are not created in `Pending`
|
||||
state. This is confusing to the user.
|
||||
* [Pod preemption](/docs/concepts/configuration/pod-priority-preemption/)
|
||||
is handled by default scheduler. When preemption is enabled, the DaemonSet controller
|
||||
will make scheduling decisions without considering pod priority and preemption.
|
||||
|
||||
`ScheduleDaemonSetPods` allows you to schedule DaemonSets using the default
|
||||
scheduler instead of the DaemonSet controller, by adding the `NodeAffinity` term
|
||||
to the DaemonSet pods, instead of the `.spec.nodeName` term. The default
|
||||
scheduler is then used to bind the pod to the target host. If node affinity of
|
||||
the DaemonSet pod already exists, it is replaced. The DaemonSet controller only
|
||||
performs these operations when creating or modifying DaemonSet pods, and no
|
||||
changes are made to the `spec.template` of the DaemonSet.
|
||||
|
||||
```yaml
|
||||
nodeAffinity:
|
||||
requiredDuringSchedulingIgnoredDuringExecution:
|
||||
nodeSelectorTerms:
|
||||
- matchFields:
|
||||
- key: metadata.name
|
||||
operator: In
|
||||
values:
|
||||
- target-host-name
|
||||
```
|
||||
|
||||
In addition, `node.kubernetes.io/unschedulable:NoSchedule` toleration is added
|
||||
automatically to DaemonSet Pods. The DaemonSet controller ignores
|
||||
`unschedulable` Nodes when scheduling DaemonSet Pods. You must enable
|
||||
`TaintModesByCondition` to ensure that the default scheduler behaves the same
|
||||
way and schedules DaemonSet pods on `unschedulable` nodes.
|
||||
|
||||
When this feature and `TaintNodesByCondition` are enabled together, if DaemonSet
|
||||
uses the host network, you must also add the
|
||||
`node.kubernetes.io/network-unavailable:NoSchedule toleration`.
|
||||
|
||||
|
||||
### Taints and Tolerations
|
||||
|
||||
Although Daemon Pods respect
|
||||
[taints and tolerations](/docs/concepts/configuration/taint-and-toleration),
|
||||
the following tolerations are added to DamonSet Pods automatically according to
|
||||
the related features.
|
||||
|
||||
| Toleration Key | Effect | Alpha Features | Version | Description |
|
||||
| ---------------------------------------- | ---------- | ------------------------------------------------------------ | ------- | ------------------------------------------------------------ |
|
||||
| `node.kubernetes.io/not-ready` | NoExecute | `TaintBasedEvictions` | 1.8+ | when `TaintBasedEvictions` is enabled,they will not be evicted when there are node problems such as a network partition. |
|
||||
| `node.kubernetes.io/unreachable` | NoExecute | `TaintBasedEvictions` | 1.8+ | when `TaintBasedEvictions` is enabled,they will not be evicted when there are node problems such as a network partition. |
|
||||
| `node.kubernetes.io/disk-pressure` | NoSchedule | `TaintNodesByCondition` | 1.8+ | |
|
||||
| `node.kubernetes.io/memory-pressure` | NoSchedule | `TaintNodesByCondition` | 1.8+ | |
|
||||
| `node.kubernetes.io/unschedulable` | NoSchedule | `ScheduleDaemonSetPods`, `TaintNodesByCondition` | 1.11+ | When ` ScheduleDaemonSetPods` is enabled, ` TaintNodesByCondition` is necessary to make sure DaemonSet pods tolerate unschedulable attributes by default scheduler. |
|
||||
| `node.kubernetes.io/network-unavailable` | NoSchedule | `ScheduleDaemonSetPods`, `TaintNodesByCondition`, hostnework | 1.11+ | When ` ScheduleDaemonSetPods` is enabled, ` TaintNodesByCondition` is necessary to make sure DaemonSet pods, who uses host network, tolerate network-unavailable attributes by default scheduler. |
|
||||
| `node.kubernetes.io/out-of-disk` | NoSchedule | `ExperimentalCriticalPodAnnotation` (critical pod only), `TaintNodesByCondition` | 1.8+ | |
|
||||
|
||||
When the support to critical pods is enabled and the pods in a DaemonSet are
|
||||
labeled as critical, the Daemon pods are created with an additional
|
||||
`NoSchedule` toleration for the `node.kubernetes.io/out-of-disk` taint.
|
||||
|
||||
Note that all above `NoSchedule` taints above are created only in version 1.8 or later if the alpha feature `TaintNodesByCondition` is enabled.
|
||||
|
||||
Also note that the `node-role.kubernetes.io/master` `NoSchedule` toleration specified in the above example is needed on 1.6 or later to schedule on *master* nodes as this is not a default toleration.
|
||||
|
||||
## Communicating with Daemon Pods
|
||||
|
||||
|
||||
@@ -56,65 +56,22 @@ array has six possible fields:
|
||||
* The `message` field is a human-readable message indicating details
|
||||
about the transition.
|
||||
|
||||
* The `reason` field is a unique, one-word, CamelCase reason for the
|
||||
transition.
|
||||
A Pod has a PodStatus, which has an array of
|
||||
[PodConditions](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#podcondition-v1-core). Each element
|
||||
of the PodCondition array has a `type` field and a `status` field. The `type`
|
||||
field is a string with the following possible values:
|
||||
|
||||
* The `type` field is a string enum with the following possible values:
|
||||
* `PodScheduled`: the Pod has been scheduled to a node;
|
||||
* `Ready`: the Pod is able to serve requests and should be added to the load
|
||||
balancing pools of all matching Services;
|
||||
* `Initialized`: all [init containers](/docs/concepts/workloads/pods/init-containers)
|
||||
have started successfully;
|
||||
* `Unschedulable`: the scheduler cannot schedule the Pod right now, for example
|
||||
due to lacking of resources or other constraints;
|
||||
* `ContainersReady`: all containers in the Pod are ready.
|
||||
|
||||
Condition | Description
|
||||
:---------|:-----------
|
||||
`Initialized` | The init containers in the Pod have started successfully
|
||||
`PodScheduled` | The Pod has been scheduled
|
||||
`Ready` | The Pod is able to service requests and should be added to the load balancing pools of any matching services
|
||||
`ContainersReady` | All Containers in the Pod are currently ready
|
||||
`Unschedulable` | The scheduler can't currently schedule the Pod, perhaps because not enough resources are available
|
||||
|
||||
* The `status` field is a string with three possible values, `True`, `False`,
|
||||
and `Unknown`, and indicates whether the corresponding condition indicated
|
||||
by the `type` field has been reached. For example, if the `type` field is
|
||||
`Initialized` and the corresponding `status` field is `True`, then the Pod
|
||||
has indeed been initialized; if, however, the `status` field is `False` then
|
||||
the Pod has *not* been initialized.
|
||||
|
||||
### Pod conditions example
|
||||
|
||||
Let's say that you have a Pod called `my-application` running in a Kubernetes
|
||||
cluster. This `kubectl` query would fetch the PodCondition information for
|
||||
the Pod:
|
||||
|
||||
```shell
|
||||
kubectl get pod my-application -o json | jq .status.conditions
|
||||
```
|
||||
|
||||
If that query returned this example JSON:
|
||||
|
||||
```json
|
||||
[
|
||||
{
|
||||
"lastProbeTime": null,
|
||||
"lastTransitionTime": "2018-06-22T18:28:32Z",
|
||||
"status": "True",
|
||||
"type": "Initialized"
|
||||
},
|
||||
{
|
||||
"lastProbeTime": null,
|
||||
"lastTransitionTime": "2018-06-22T18:29:04Z",
|
||||
"status": "True",
|
||||
"type": "Ready"
|
||||
},
|
||||
{
|
||||
"lastProbeTime": null,
|
||||
"lastTransitionTime": "2018-06-22T18:28:32Z",
|
||||
"status": "True",
|
||||
"type": "PodScheduled"
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
We can see here that the `my-application` Pod has passed through three different
|
||||
Pod conditions: `Initialized`, `Ready`, and `PodScheduled`. If the `status` of
|
||||
`PodScheduled` were `False`, for example, then the Pod would have passed through
|
||||
only two conditions.
|
||||
The `status` field is a string, with possible values "`True`", "`False`", and
|
||||
"`Unknown`".
|
||||
|
||||
## Container probes
|
||||
|
||||
@@ -186,7 +143,8 @@ puts itself into an unready state regardless of whether the readiness probe exis
|
||||
The Pod remains in the unready state while it waits for the Containers in the Pod
|
||||
to stop.
|
||||
|
||||
For more information about how to set up a liveness or readiness probe, see [Configure Liveness and Readiness Probes](/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/).
|
||||
For more information about how to set up a liveness or readiness probe, see
|
||||
[Configure Liveness and Readiness Probes](/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/).
|
||||
|
||||
## Pod and Container status
|
||||
|
||||
@@ -197,6 +155,58 @@ and
|
||||
Note that the information reported as Pod status depends on the current
|
||||
[ContainerState](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#containerstatus-v1-core).
|
||||
|
||||
## Pod readiness gate
|
||||
|
||||
{{< feature-state for_k8s_version="v1.11" state="alpha" >}}
|
||||
|
||||
In order to add extensibility to Pod readiness by enabling the injection of
|
||||
extra feedbacks or signals into `PodStatus`, Kubernetes 1.11 introduced a
|
||||
feature named [Pod ready++](https://github.com/kubernetes/community/blob/master/keps/sig-network/0007-pod-ready%2B%2B.md).
|
||||
You can use the new field `ReadinessGate` in the `PodSpec` to specify additional
|
||||
conditions to be evaluated for Pod readiness. If Kubernetes cannot find such a
|
||||
condition in the `status.conditions` field of a Pod, the status of the condition
|
||||
is default to "`False`". Below is an example:
|
||||
|
||||
```yaml
|
||||
Kind: Pod
|
||||
...
|
||||
spec:
|
||||
readinessGates:
|
||||
- conditionType: "www.example.com/feature-1"
|
||||
status:
|
||||
conditions:
|
||||
- type: Ready # this is a builtin PodCondition
|
||||
status: "True"
|
||||
lastProbeTime: null
|
||||
lastTransitionTime: 2018-01-01T00:00:00Z
|
||||
- type: "www.example.com/feature-1" # an extra PodCondition
|
||||
status: "False"
|
||||
lastProbeTIme: null
|
||||
lastTransitionTime: 2018-01-01T00:00:00Z
|
||||
containerStatuses:
|
||||
- containerID: docker://abcd...
|
||||
ready: true
|
||||
...
|
||||
```
|
||||
|
||||
The new Pod conditions must comply with Kubernetes [label key format](/docs/concepts/overview/working-with-objects/labels/#syntax-and-character-set).
|
||||
Since the `kubectl patch` command still doesn't support patching object status,
|
||||
the new Pod conditions have to be injected through the `PATCH` action using
|
||||
one of the [KubeClient libraries](/docs/reference/using-api/client-librarie/).
|
||||
|
||||
With the introduction of new Pod conditions, a Pod is evaluated to be ready **only**
|
||||
when both the following statements are true:
|
||||
|
||||
* All containers in the Pod are ready.
|
||||
* All conditions specified in `ReadinessGates` are "`True`".
|
||||
|
||||
To facilitate this change to Pod readiness evaluation, a new Pod condition
|
||||
`ContainersReady` is introduced to capture the old Pod `Ready` condition.
|
||||
|
||||
As an alpha feature, the "Pod Ready++" feature has to be explicitly enabled by
|
||||
setting the `PodReadinessGates` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/)
|
||||
to True.
|
||||
|
||||
## Restart policy
|
||||
|
||||
A PodSpec has a `restartPolicy` field with possible values Always, OnFailure,
|
||||
@@ -210,7 +220,6 @@ minutes of successful execution. As discussed in the
|
||||
once bound to a node, a Pod will never be rebound to another node.
|
||||
|
||||
|
||||
|
||||
## Pod lifetime
|
||||
|
||||
In general, Pods do not disappear until someone destroys them. This might be a
|
||||
|
||||
Reference in New Issue
Block a user