diff --git a/_data/tasks.yml b/_data/tasks.yml index 470a5a07e5..d6a0376601 100644 --- a/_data/tasks.yml +++ b/_data/tasks.yml @@ -15,7 +15,7 @@ toc: - docs/tasks/configure-pod-container/assign-cpu-resource.md - docs/tasks/configure-pod-container/quality-service-pod.md - docs/tasks/configure-pod-container/assign-cpu-ram-container.md - - docs/tasks/configure-pod-container/opaque-integer-resource.md + - docs/tasks/configure-pod-container/extended-resource.md - docs/tasks/configure-pod-container/configure-volume-storage.md - docs/tasks/configure-pod-container/configure-persistent-volume-storage.md - docs/tasks/configure-pod-container/configure-projected-volume-storage.md @@ -128,7 +128,7 @@ toc: - docs/tasks/administer-cluster/quota-memory-cpu-namespace.md - docs/tasks/administer-cluster/quota-pod-namespace.md - docs/tasks/administer-cluster/quota-api-object.md - - docs/tasks/administer-cluster/opaque-integer-resource-node.md + - docs/tasks/administer-cluster/extended-resource-node.md - docs/tasks/administer-cluster/cpu-management-policies.md - docs/tasks/administer-cluster/access-cluster-api.md - docs/tasks/administer-cluster/access-cluster-services.md diff --git a/docs/concepts/cluster-administration/cloud-providers.md b/docs/concepts/cluster-administration/cloud-providers.md index 2d4e676280..17b8ecadcc 100644 --- a/docs/concepts/cluster-administration/cloud-providers.md +++ b/docs/concepts/cluster-administration/cloud-providers.md @@ -152,6 +152,11 @@ file: values are `v1` or `v2`. Where no value is provided automatic detection will select the highest supported version exposed by the underlying OpenStack cloud. +* `use-octavia` (Optional): Used to determine whether to look for and use an + Octavia LBaaS V2 service catalog endpoint. Valid values are `true` or `false`. + Where `true` is specified and an Octaiva LBaaS V2 entry can not be found, the + provider will fall back and attempt to find a Neutron LBaaS V2 endpoint + instead. The default value is `false`. * `subnet-id` (Optional): Used to specify the id of the subnet you want to create your loadbalancer on. Can be found at Network > Networks. Click on the respective network to get its subnets. @@ -196,6 +201,12 @@ and should appear in the `[BlockStorage]` section of the `cloud.conf` file: provided by Cinder. The default value of `false` results in the discovery of the device path based on its serial number and `/dev/disk/by-id` mapping and is the recommended approach. +* `ignore-volume-az` (Optional): Used to influence availability zone use when + attaching Cinder volumes. When Nova and Cinder have different availability + zones, this should be set to `true`. This is most commonly the case where + there are many Nova availability zones but only one Cinder availability zone. + The default value is `false` to preserve the behavior used in earlier + releases, but may change in the future. If deploying Kubernetes versions <= 1.8 on an OpenStack deployment that uses paths rather than ports to differentiate between endpoints it may be necessary diff --git a/docs/concepts/workloads/pods/podpreset.md b/docs/concepts/workloads/pods/podpreset.md index bf9ca2eb67..316fe8b6da 100644 --- a/docs/concepts/workloads/pods/podpreset.md +++ b/docs/concepts/workloads/pods/podpreset.md @@ -41,7 +41,7 @@ When a pod creation request occurs, the system does the following: the pod _without_ any injected resources from the `PodPreset`. 1. Annotate the resulting modified Pod spec to indicate that it has been modified by a `PodPreset`. The annotation is of the form - `podpreset.admission.kubernetes.io/podpreset-": ""`. + `podpreset.admission.kubernetes.io/podpreset-: ""`. Each Pod can be matched zero or more Pod Presets; and each `PodPreset` can be applied to zero or more pods. When a `PodPreset` is applied to one or more diff --git a/docs/tasks/administer-cluster/opaque-integer-resource-node.md b/docs/tasks/administer-cluster/extended-resource-node.md similarity index 68% rename from docs/tasks/administer-cluster/opaque-integer-resource-node.md rename to docs/tasks/administer-cluster/extended-resource-node.md index c71f45aa6b..a26daf08e6 100644 --- a/docs/tasks/administer-cluster/opaque-integer-resource-node.md +++ b/docs/tasks/administer-cluster/extended-resource-node.md @@ -1,15 +1,15 @@ --- -title: Advertise Opaque Integer Resources for a Node +title: Advertise Extended Resources for a Node --- {% capture overview %} -This page shows how to specify opaque integer resources for a Node. -Opaque integer resources allow cluster administrators to advertise node-level +This page shows how to specify extended resources for a Node. +Extended resources allow cluster administrators to advertise node-level resources that would otherwise be unknown to Kubernetes. -{% include feature-state-deprecated.md %} +{% include feature-state-stable.md %} {% endcapture %} @@ -31,9 +31,9 @@ kubectl get nodes Choose one of your Nodes to use for this exercise. -## Advertise a new opaque integer resource on one of your Nodes +## Advertise a new extended resource on one of your Nodes -To advertise a new opaque integer resource on a Node, send an HTTP PATCH request to +To advertise a new extended resource on a Node, send an HTTP PATCH request to the Kubernetes API server. For example, suppose one of your Nodes has four dongles attached. Here's an example of a PATCH request that advertises four dongle resources for your Node. @@ -47,7 +47,7 @@ Host: k8s-master:8080 [ { "op": "add", - "path": "/status/capacity/pod.alpha.kubernetes.io~1opaque-int-resource-dongle", + "path": "/status/capacity/example.com~1dongle", "value": "4" } ] @@ -69,7 +69,7 @@ Replace `` with the name of your Node: ```shell curl --header "Content-Type: application/json-patch+json" \ --request PATCH \ ---data '[{"op": "add", "path": "/status/capacity/pod.alpha.kubernetes.io~1opaque-int-resource-dongle", "value": "4"}]' \ +--data '[{"op": "add", "path": "/status/capacity/example.com~1dongle", "value": "4"}]' \ http://localhost:8001/api/v1/nodes//status ``` @@ -85,7 +85,7 @@ The output shows that the Node has a capacity of 4 dongles: "alpha.kubernetes.io/nvidia-gpu": "0", "cpu": "2", "memory": "2049008Ki", - "pod.alpha.kubernetes.io/opaque-int-resource-dongle": "4", + "example.com/dongle": "4", ``` Describe your Node: @@ -98,53 +98,52 @@ Once again, the output shows the dongle resource: ```yaml Capacity: - alpha.kubernetes.io/nvidia-gpu: 0 - cpu: 2 - memory: 2049008Ki - pod.alpha.kubernetes.io/opaque-int-resource-dongle: 4 + alpha.kubernetes.io/nvidia-gpu: 0 + cpu: 2 + memory: 2049008Ki + example.com/dongle: 4 ``` Now, application developers can create Pods that request a certain number of dongles. See -[Assign Opaque Integer Resources to a Container](/docs/tasks/configure-pod-container/opaque-integer-resource/). +[Assign Extended Resources to a Container](/docs/tasks/configure-pod-container/extended-resource/). ## Discussion -Opaque integer resources are similar to memory and CPU resources. For example, +Extended resources are similar to memory and CPU resources. For example, just as a Node has a certain amount of memory and CPU to be shared by all components running on the Node, it can have a certain number of dongles to be shared by all components running on the Node. And just as application developers can create Pods that request a certain amount of memory and CPU, they can create Pods that request a certain number of dongles. -Opaque integer resources are called opaque because Kubernetes does not +Extended resources are opaque to Kubernetes; Kubernetes does not know anything about what they are. Kubernetes knows only that a Node -has a certain number of them. They are called integer resources because -they must be advertised in integer amounts. For example, a Node can advertise -four dongles, but not 4.5 dongles. +has a certain number of them. Extended resources must be advertised in integer +amounts. For example, a Node can advertise four dongles, but not 4.5 dongles. ### Storage example Suppose a Node has 800 GiB of a special kind of disk storage. You could -create a name for the special storage, say opaque-int-resource-special-storage. +create a name for the special storage, say example.com/special-storage. Then you could advertise it in chunks of a certain size, say 100 GiB. In that case, your Node would advertise that it has eight resources of type -opaque-int-resource-special-storage. +example.com/special-storage. ```yaml Capacity: ... - pod.alpha.kubernetes.io/opaque-int-resource-special-storage: 8 + example.com/special-storage: 8 ``` If you want to allow arbitrary requests for special storage, you could advertise special storage in chunks of size 1 byte. In that case, you would advertise -800Gi resources of type opaque-int-resource-special-storage. +800Gi resources of type example.com/special-storage. ```yaml Capacity: ... - pod.alpha.kubernetes.io/opaque-int-resource-special-storage: 800Gi + example.com/special-storage: 800Gi ``` Then a Container could request any number of bytes of special storage, up to 800Gi. @@ -162,7 +161,7 @@ Host: k8s-master:8080 [ { "op": "remove", - "path": "/status/capacity/pod.alpha.kubernetes.io~1opaque-int-resource-dongle", + "path": "/status/capacity/example.com~1dongle", } ] ``` @@ -179,7 +178,7 @@ Replace `` with the name of your Node: ```shell curl --header "Content-Type: application/json-patch+json" \ --request PATCH \ ---data '[{"op": "remove", "path": "/status/capacity/pod.alpha.kubernetes.io~1opaque-int-resource-dongle"}]' \ +--data '[{"op": "remove", "path": "/status/capacity/example.com~1dongle"}]' \ http://localhost:8001/api/v1/nodes//status ``` @@ -196,7 +195,7 @@ kubectl describe node | grep dongle ### For application developers -* [Assign Opaque Integer Resources to a Container](/docs/tasks/configure-pod-container/opaque-integer-resource/) +* [Assign Extended Resources to a Container](/docs/tasks/configure-pod-container/extended-resource/) ### For cluster administrators diff --git a/docs/tasks/administer-cluster/out-of-resource.md b/docs/tasks/administer-cluster/out-of-resource.md index 87e88709ed..7461c7329a 100644 --- a/docs/tasks/administer-cluster/out-of-resource.md +++ b/docs/tasks/administer-cluster/out-of-resource.md @@ -19,8 +19,8 @@ nodes become unstable. ## Eviction Policy The `kubelet` can proactively monitor for and prevent total starvation of a -compute resource. In those cases, the `kubelet` can reclaim the starved -resource by proactively failing one or more Pods. When the `kubelet` fails +compute resource. In those cases, the `kubelet` can reclaim the starved +resource by proactively failing one or more Pods. When the `kubelet` fails a Pod, it terminates all of its containers and transitions its `PodPhase` to `Failed`. ### Eviction Signals @@ -42,11 +42,11 @@ The percentage based value is calculated relative to the total capacity associated with each signal. The value for `memory.available` is derived from the cgroupfs instead of tools -like `free -m`. This is important because `free -m` does not work in a +like `free -m`. This is important because `free -m` does not work in a container, and if users use the [node allocatable](/docs/tasks/administer-cluster/reserve-compute-resources/#node-allocatable) feature, out of resource decisions are made local to the end user Pod part of the cgroup hierarchy as well as the -root node. This +root node. This [script](/docs/tasks/administer-cluster/out-of-resource/memory-available.sh) reproduces the same set of steps that the `kubelet` performs to calculate `memory.available`. The `kubelet` excludes inactive_file (i.e. # of bytes of @@ -60,7 +60,7 @@ memory is reclaimable under pressure. container writable layers. `imagefs` is optional. `kubelet` auto-discovers these filesystems using -cAdvisor. `kubelet` does not care about any other filesystems. Any other types +cAdvisor. `kubelet` does not care about any other filesystems. Any other types of configurations are not currently supported by the kubelet. For example, it is *not OK* to store volumes and logs in a dedicated `filesystem`. @@ -78,7 +78,7 @@ Each threshold has the following form: where: -* `eviction-signal` is a eviction signal token as defined in the previous table. +* `eviction-signal` is an eviction signal token as defined in the previous table. * `operator` is the desired relational operator, such as `<` (less than). * `quantity` is the eviction threshhold quantity, such as `1Gi`. These tokens must match the quantity representation used by Kubernetes. An eviction threshold can also @@ -208,7 +208,7 @@ relative to their request for that resource are killed first. If no Pod has exceeded its request, the strategy targets the largest consumer of the starved resource. * `Guaranteed` Pods are guaranteed only when requests and limits are specified -for all the containers and they are equal A `Guaranteed` Pod is guaranteed to +for all the containers and they are equal. A `Guaranteed` Pod is guaranteed to never be evicted because of another Pod's resource consumption. If a system daemon (such as `kubelet`, `docker`, and `journald`) is consuming more resources than were reserved via `system-reserved` or `kube-reserved` allocations, and the @@ -322,7 +322,7 @@ and trigger eviction assuming those Pods use less than their configured request. ### DaemonSet -It is never desired for a `kubelet` to evict a `DaemonSet` Pod, since the Pod is +It is never desired for `kubelet` to evict a `DaemonSet` Pod, since the Pod is immediately recreated and rescheduled back to the same node. At the moment, the `kubelet` has no ability to distinguish a Pod created diff --git a/docs/tasks/administer-cluster/reserve-compute-resources.md b/docs/tasks/administer-cluster/reserve-compute-resources.md index 6bc254fbb5..ee83d76817 100644 --- a/docs/tasks/administer-cluster/reserve-compute-resources.md +++ b/docs/tasks/administer-cluster/reserve-compute-resources.md @@ -52,8 +52,8 @@ Resources can be reserved for two categories of system daemons in the `kubelet`. ### Enabling QoS and Pod level cgroups To properly enforce node allocatable constraints on the node, you must -enable the new cgroup hierarchy via the `--cgroups-per-qos` flag. This flag is -enabled by default. When enabled, the `kubelet` will parent all end-user pods +enable the new cgroup hierarchy via the `--cgroups-per-qos` flag. This flag is +enabled by default. When enabled, the `kubelet` will parent all end-user pods under a cgroup hierarchy managed by the `kubelet`. ### Configuring a cgroup driver @@ -71,7 +71,7 @@ transient slices for resources that are supported by that init system. Depending on the configuration of the associated container runtime, operators may have to choose a particular cgroup driver to ensure -proper system behavior. For example, if operators use the `systemd` +proper system behavior. For example, if operators use the `systemd` cgroup driver provided by the `docker` runtime, the `kubelet` must be configured to use the `systemd` cgroup driver. @@ -199,7 +199,7 @@ Under this scenario, `Allocatable` will be `14.5 CPUs`, `28.5Gi` of memory and `98Gi` of local storage. Scheduler ensures that the total memory `requests` across all pods on this node does not exceed `28.5Gi` and storage doesn't exceed `88Gi`. -Kubelet evicts pods whenever the overall memory usage exceeds across pods exceed `28.5Gi`, +Kubelet evicts pods whenever the overall memory usage across pods exceeds `28.5Gi`, or if overall disk usage exceeds `88Gi` If all processes on the node consume as much CPU as they can, pods together cannot consume more than `14.5 CPUs`. @@ -224,7 +224,7 @@ kubelet flag. Note that unless `--kube-reserved`, or `--system-reserved` or not affect existing deployments. As of Kubernetes version 1.6, `kubelet` launches pods in their own cgroup -sandbox in a dedicated part of the cgroup hierarchy it manages. Operators are +sandbox in a dedicated part of the cgroup hierarchy it manages. Operators are required to drain their nodes prior to upgrade of the `kubelet` from prior versions in order to ensure pods and their associated containers are launched in the proper part of the cgroup hierarchy. diff --git a/docs/tasks/configure-pod-container/oir-pod.yaml b/docs/tasks/configure-pod-container/extended-resource-pod-2.yaml similarity index 50% rename from docs/tasks/configure-pod-container/oir-pod.yaml rename to docs/tasks/configure-pod-container/extended-resource-pod-2.yaml index bd85a5d4f6..33dec523fb 100644 --- a/docs/tasks/configure-pod-container/oir-pod.yaml +++ b/docs/tasks/configure-pod-container/extended-resource-pod-2.yaml @@ -1,11 +1,11 @@ apiVersion: v1 kind: Pod metadata: - name: oir-demo + name: extended-resource-demo-2 spec: containers: - - name: oir-demo-ctr + - name: extended-resource-demo-2-ctr image: nginx resources: requests: - pod.alpha.kubernetes.io/opaque-int-resource-dongle: 3 + example.com/dongle: 2 diff --git a/docs/tasks/configure-pod-container/extended-resource-pod.yaml b/docs/tasks/configure-pod-container/extended-resource-pod.yaml new file mode 100644 index 0000000000..e0df8c95b0 --- /dev/null +++ b/docs/tasks/configure-pod-container/extended-resource-pod.yaml @@ -0,0 +1,11 @@ +apiVersion: v1 +kind: Pod +metadata: + name: extended-resource-demo +spec: + containers: + - name: extended-resource-demo-ctr + image: nginx + resources: + requests: + example.com/dongle: 3 diff --git a/docs/tasks/configure-pod-container/opaque-integer-resource.md b/docs/tasks/configure-pod-container/extended-resource.md similarity index 53% rename from docs/tasks/configure-pod-container/opaque-integer-resource.md rename to docs/tasks/configure-pod-container/extended-resource.md index 7092cfd5e7..137144a1cd 100644 --- a/docs/tasks/configure-pod-container/opaque-integer-resource.md +++ b/docs/tasks/configure-pod-container/extended-resource.md @@ -1,12 +1,12 @@ --- -title: Assign Opaque Integer Resources to a Container +title: Assign Extended Resources to a Container --- {% capture overview %} -This page shows how to assign opaque integer resources to a Container. +This page shows how to assign extended resources to a Container. -{% include feature-state-deprecated.md %} +{% include feature-state-stable.md %} {% endcapture %} @@ -16,7 +16,7 @@ This page shows how to assign opaque integer resources to a Container. {% include task-tutorial-prereqs.md %} Before you do this exercise, do the exercise in -[Advertise Opaque Integer Resources for a Node](/docs/tasks/administer-cluster/opaque-integer-resource-node/). +[Advertise Extended Resources for a Node](/docs/tasks/administer-cluster/extended-resource-node/). That will configure one of your Nodes to advertise a dongle resource. {% endcapture %} @@ -24,40 +24,43 @@ That will configure one of your Nodes to advertise a dongle resource. {% capture steps %} -## Assign an opaque integer resource to a Pod +## Assign an extended resource to a Pod -To request an opaque integer resource, include the `resources:requests` field in your -Container manifest. Opaque integer resources have the prefix `pod.alpha.kubernetes.io/opaque-int-resource-`. +To request an extended resource, include the `resources:requests` field in your +Container manifest. Extended resources are fully qualified with any domain outside of +`*.kubernetes.io/`. Valid extended resource names have the form `example.com/foo` where +`example.com` is replaced with your organization's domain and `foo` is a +descriptive resource name. Here is the configuration file for a Pod that has one Container: -{% include code.html language="yaml" file="oir-pod.yaml" ghlink="/docs/tasks/configure-pod-container/oir-pod.yaml" %} +{% include code.html language="yaml" file="extended-resource-pod.yaml" ghlink="/docs/tasks/configure-pod-container/extended-resource-pod.yaml" %} In the configuration file, you can see that the Container requests 3 dongles. Create a Pod: ```shell -kubectl create -f https://k8s.io/docs/tasks/configure-pod-container/oir-pod.yaml +kubectl create -f https://k8s.io/docs/tasks/configure-pod-container/extended-resource-pod.yaml ``` Verify that the Pod is running: ```shell -kubectl get pod oir-demo +kubectl get pod extended-resource-demo ``` Describe the Pod: ```shell -kubectl describe pod oir-demo +kubectl describe pod extended-resource-demo ``` The output shows dongle requests: ```yaml Requests: - pod.alpha.kubernetes.io/opaque-int-resource-dongle: 3 + example.com/dongle: 3 ``` ## Attempt to create a second Pod @@ -65,7 +68,7 @@ Requests: Here is the configuration file for a Pod that has one Container. The Container requests two dongles. -{% include code.html language="yaml" file="oir-pod-2.yaml" ghlink="/docs/tasks/configure-pod-container/oir-pod-2.yaml" %} +{% include code.html language="yaml" file="extended-resource-pod-2.yaml" ghlink="/docs/tasks/configure-pod-container/extended-resource-pod-2.yaml" %} Kubernetes will not be able to satisfy the request for two dongles, because the first Pod used three of the four available dongles. @@ -73,13 +76,13 @@ used three of the four available dongles. Attempt to create a Pod: ```shell -kubectl create -f https://k8s.io/docs/tasks/configure-pod-container/oir-pod-2.yaml +kubectl create -f https://k8s.io/docs/tasks/configure-pod-container/extended-resource-pod-2.yaml ``` Describe the Pod ```shell -kubectl describe pod oir-demo-2 +kubectl describe pod extended-resource-demo-2 ``` The output shows that the Pod cannot be scheduled, because there is no Node that has @@ -93,22 +96,22 @@ Conditions: ... Events: ... - ... Warning FailedScheduling pod (oir-demo-2) failed to fit in any node -fit failure summary on nodes : Insufficient pod.alpha.kubernetes.io/opaque-int-resource-dongle (1) + ... Warning FailedScheduling pod (extended-resource-demo-2) failed to fit in any node +fit failure summary on nodes : Insufficient example.com/dongle (1) ``` View the Pod status: ```shell -kubectl get pod oir-demo-2 +kubectl get pod extended-resource-demo-2 ``` The output shows that the Pod was created, but not scheduled to run on a Node. It has a status of Pending: ```yaml -NAME READY STATUS RESTARTS AGE -oir-demo-2 0/1 Pending 0 6m +NAME READY STATUS RESTARTS AGE +extended-resource-demo-2 0/1 Pending 0 6m ``` ## Clean up @@ -116,7 +119,7 @@ oir-demo-2 0/1 Pending 0 6m Delete the Pod that you created for this exercise: ```shell -kubectl delete pod oir-demo-2 +kubectl delete pod extended-resource-demo-2 ``` {% endcapture %} @@ -130,7 +133,7 @@ kubectl delete pod oir-demo-2 ### For cluster administrators -* [Advertise Opaque Integer Resources for a Node](/docs/tasks/administer-cluster/opaque-integer-resource-node/) +* [Advertise Extended Resources for a Node](/docs/tasks/administer-cluster/extended-resource-node/) {% endcapture %} diff --git a/docs/tasks/configure-pod-container/oir-pod-2.yaml b/docs/tasks/configure-pod-container/oir-pod-2.yaml deleted file mode 100644 index cfe4b96076..0000000000 --- a/docs/tasks/configure-pod-container/oir-pod-2.yaml +++ /dev/null @@ -1,11 +0,0 @@ -apiVersion: v1 -kind: Pod -metadata: - name: oir-demo-2 -spec: - containers: - - name: oir-demo-2-ctr - image: nginx - resources: - requests: - pod.alpha.kubernetes.io/opaque-int-resource-dongle: 2