Update device plugin documentation + related pages (#14331)

* Update device plugin docs for Kubernetes 1.14

& related pages

This change is mainly about feature-state shortcodes & similar.

(if there's a way to get the short version string, eg "1.14" rather
than "v1.14", then that could go in place of the hard coded value
in this commit).

* Fix code block formatting

eg code blocks incorrectly marked as:
```shell

* Use glossary shortcodes where appropriate

* Hyperlink to Prometheus

* Tidy Markdown formatting

* Change example vendor domain name

Use a name inside ".example" to highlight that this is a DNS
domain name.

* Reword device plugins documentation

* Tweak headings for device plugins

* Add "what's next" to device plugins docs

* Tweak wording for device plugins docs

* Add KubeVirt device plugins

content/en/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins.md

@@ -6,31 +6,33 @@ content_template: templates/concept
 weight: 20
 ---
 
-{{< feature-state state="beta" >}}
-
 {{% capture overview %}}
-Starting in version 1.8, Kubernetes provides a
+{{< feature-state for_k8s_version="v1.10" state="beta" >}}
-[device plugin framework](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/resource-management/device-plugin.md)
+
-for vendors to advertise their resources to the kubelet without changing Kubernetes core code.
+Kubernetes provides a [device plugin framework](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/resource-management/device-plugin.md)
-Instead of writing custom Kubernetes code, vendors can implement a device plugin that can
+that you can use to advertise system hardware resources to the
-be deployed manually or as a DaemonSet. The targeted devices include GPUs,
+{{< glossary_tooltip term_id="kubelet" >}}.
-High-performance NICs, FPGAs, InfiniBand, and other similar computing resources
+
-that may require vendor specific initialization and setup.
+Instead of customising the code for Kubernetes itself, vendors can implement a
+device plugin that you deploy either manually or as a {{< glossary_tooltip term_id="daemonset" >}}.
+The targeted devices include GPUs, high-performance NICs, FPGAs, InfiniBand adapters,
+and other similar computing resources that may require vendor specific initialization
+and setup.
+
 {{% /capture %}}
 
 {{% capture body %}}
 
 ## Device plugin registration
 
-The device plugins feature is gated by the `DevicePlugins` feature gate which
+The kubelet exports a `Registration` gRPC service:
-is disabled by default before 1.10.  When the device plugins feature is enabled,
-the kubelet exports a `Registration` gRPC service:
 
 ```gRPC
 service Registration {
 	rpc Register(RegisterRequest) returns (Empty) {}
 }
 ```
+
 A device plugin can register itself with the kubelet through this gRPC service.
 During the registration, the device plugin needs to send:
 
@@ -38,15 +40,15 @@ During the registration, the device plugin needs to send:
   * The Device Plugin API version against which it was built.
   * The `ResourceName` it wants to advertise. Here `ResourceName` needs to follow the
     [extended resource naming scheme](/docs/concepts/configuration/manage-compute-resources-container/#extended-resources)
-    as `vendor-domain/resource`.
+    as `vendor-domain/resourcetype`.
-    For example, an Nvidia GPU is advertised as `nvidia.com/gpu`.
+    (For example, an NVIDIA GPU is advertised as `nvidia.com/gpu`.)
 
 Following a successful registration, the device plugin sends the kubelet the
 list of devices it manages, and the kubelet is then in charge of advertising those
 resources to the API server as part of the kubelet node status update.
-For example, after a device plugin registers `vendor-domain/foo` with the kubelet
+For example, after a device plugin registers `hardware-vendor.example/foo` with the kubelet
 and reports two healthy devices on a node, the node status is updated
-to advertise 2 `vendor-domain/foo`.
+to advertise that the node has 2 “Foo” devices installed and available.
 
 Then, users can request devices in a
 [Container](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#container-v1-core)
@@ -55,10 +57,11 @@ specification as they request other types of resources, with the following limit
 * Extended resources are only supported as integer resources and cannot be overcommitted.
 * Devices cannot be shared among Containers.
 
-Suppose a Kubernetes cluster is running a device plugin that advertises resource `vendor-domain/resource`
+Suppose a Kubernetes cluster is running a device plugin that advertises resource `hardware-vendor.example/foo`
-on certain nodes, here is an example user pod requesting this resource:
+on certain nodes. Here is an example of a pod requesting this resource to run a demo workload:
 
 ```yaml
+---
 apiVersion: v1
 kind: Pod
 metadata:
@@ -69,7 +72,14 @@ spec:
       image: k8s.gcr.io/pause:2.0
       resources:
         limits:
-          vendor-domain/resource: 2 # requesting 2 vendor-domain/resource
+          hardware-vendor.example/foo: 2
+#
+# This Pod needs 2 of the hardware-vendor.example/foo devices
+# and can only schedule onto a Node that's able to satisfy
+# that need.
+#
+# If the Node has more than 2 of those devices available, the
+# remainder would be available for other Pods to use.
 ```
 
 ## Device plugin implementation
@@ -107,6 +117,8 @@ If the operations succeed, the device plugin returns an `AllocateResponse` that
 runtime configurations for accessing the allocated devices. The kubelet passes this information
 to the container runtime.
 
+### Handling kubelet restarts
+
 A device plugin is expected to detect kubelet restarts and re-register itself with the new
 kubelet instance. In the current implementation, a new kubelet instance deletes all the existing Unix sockets
 under `/var/lib/kubelet/device-plugins` when it starts. A device plugin can monitor the deletion
@@ -114,37 +126,44 @@ of its Unix socket and re-register itself upon such an event.
 
 ## Device plugin deployment
 
-A device plugin can be deployed manually or as a DaemonSet. Being deployed as a DaemonSet has
+You can deploy a device plugin as a DaemonSet, as a package for your node's operating system,
-the benefit that Kubernetes can restart the device plugin if it fails.
+or manually.
-Otherwise, an extra mechanism is needed to recover from device plugin failures.
+
 The canonical directory `/var/lib/kubelet/device-plugins` requires privileged access,
 so a device plugin must run in a privileged security context.
-If a device plugin is running as a DaemonSet, `/var/lib/kubelet/device-plugins`
+If you're deploying a device plugin as a DaemonSet, `/var/lib/kubelet/device-plugins`
-must be mounted as a
+must be mounted as a {{< glossary_tooltip term_id="volume" >}}
-[Volume](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#volume-v1-core)
 in the plugin's
 [PodSpec](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#podspec-v1-core).
 
-Kubernetes device plugin support is in beta. As development continues, its API version can
+If you choose the DaemonSet approach you can rely on Kubernetes to: place the device plugin's
-change. We recommend that device plugin developers do the following:
+Pod onto Nodes, to restart the daemon Pod after failure, and to help automate upgrades.
+
+## API compatibility
+
+Kubernetes device plugin support is in beta. The API may change before stabilization,
+in incompatible ways. As a project, Kubernetes recommends that device plugin developers:
 
 * Watch for changes in future releases.
 * Support multiple versions of the device plugin API for backward/forward compatibility.
 
 If you enable the DevicePlugins feature and run device plugins on nodes that need to be upgraded to
 a Kubernetes release with a newer device plugin API version, upgrade your device plugins
-to support both versions before upgrading these nodes to
+to support both versions before upgrading these nodes. Taking that approach will
 ensure the continuous functioning of the device allocations during the upgrade.
 
 ## Monitoring Device Plugin Resources
 
-In order to monitor resources provided by device plugins, monitoring agents need to be able to 
+{{< feature-state for_k8s_version="v1.13" state="alpha" >}}
-discover the set of devices that are in-use on the node and obtain metadata to describe which 
+
-container the metric should be associated with.  Prometheus metrics exposed by device monitoring 
+In order to monitor resources provided by device plugins, monitoring agents need to be able to
-agents should follow the 
+discover the set of devices that are in-use on the node and obtain metadata to describe which
-[Kubernetes Instrumentation Guidelines](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-instrumentation/instrumentation.md), 
+container the metric should be associated with. [Prometheus](https://prometheus.io/) metrics
-which requires identifying containers using `pod`, `namespace`, and `container` prometheus labels.  
+exposed by device monitoring agents should follow the
-The kubelet provides a gRPC service to enable discovery of in-use devices, and to provide metadata 
+[Kubernetes Instrumentation Guidelines](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-instrumentation/instrumentation.md),
+identifying containers using `pod`, `namespace`, and `container` prometheus labels.
+
+The kubelet provides a gRPC service to enable discovery of in-use devices, and to provide metadata
 for these devices:
 
 ```gRPC
@@ -155,31 +174,36 @@ service PodResourcesLister {
 }
 ```
 
-The gRPC service is served over a unix socket at `/var/lib/kubelet/pod-resources/kubelet.sock`. 
+The gRPC service is served over a unix socket at `/var/lib/kubelet/pod-resources/kubelet.sock`.
-Monitoring agents for device plugin resources can be deployed as a daemon, or as a DaemonSet. 
+Monitoring agents for device plugin resources can be deployed as a daemon, or as a DaemonSet.
-The canonical directory `/var/lib/kubelet/pod-resources` requires privileged access, so monitoring 
+The canonical directory `/var/lib/kubelet/pod-resources` requires privileged access, so monitoring
-agents must run in a privileged security context.  If a device monitoring agent is running as a 
+agents must run in a privileged security context.  If a device monitoring agent is running as a
-DaemonSet, `/var/lib/kubelet/pod-resources` must be mounted as a 
+DaemonSet, `/var/lib/kubelet/pod-resources` must be mounted as a
-[Volume](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#volume-v1-core)
+{{< glossary_tooltip term_id="volume" >}} in the plugin's
-in the plugin's
 [PodSpec](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#podspec-v1-core).
 
 Support for the "PodResources service" is in beta, and is enabled by default.
 
-## Examples
+## Device plugin examples {#examples}
 
-For examples of device plugin implementations, see:
+Here are some examples of device plugin implementations:
 
-* The official [NVIDIA GPU device plugin](https://github.com/NVIDIA/k8s-device-plugin)
+* The [AMD GPU device plugin](https://github.com/RadeonOpenCompute/k8s-device-plugin)
-    * Requires [nvidia-docker 2.0](https://github.com/NVIDIA/nvidia-docker) which allows you to run GPU enabled docker containers.
+* The [Intel device plugins](https://github.com/intel/intel-device-plugins-for-kubernetes) for Intel GPU, FPGA and QuickAssist devices
-    * A detailed guide on how to [schedule NVIDIA GPUs](/docs/tasks/manage-gpus/scheduling-gpus) on k8s.
+* The [KubeVirt device plugins](https://github.com/kubevirt/kubernetes-device-plugins) for hardware-assisted virtualization
-* The [NVIDIA GPU device plugin for COS base OS](https://github.com/GoogleCloudPlatform/container-engine-accelerators/tree/master/cmd/nvidia_gpu)
+* The [NVIDIA GPU device plugin](https://github.com/NVIDIA/k8s-device-plugin)
+    * Requires [nvidia-docker](https://github.com/NVIDIA/nvidia-docker) 2.0, which allows you to run GPU-enabled Docker containers.
+* The [NVIDIA GPU device plugin for Container-Optimized OS](https://github.com/GoogleCloudPlatform/container-engine-accelerators/tree/master/cmd/nvidia_gpu)
 * The [RDMA device plugin](https://github.com/hustcat/k8s-rdma-device-plugin)
 * The [Solarflare device plugin](https://github.com/vikaschoudhary16/sfc-device-plugin)
-* The [AMD GPU device plugin](https://github.com/RadeonOpenCompute/k8s-device-plugin)
+* The [SR-IOV Network device plugin](https://github.com/intel/sriov-network-device-plugin)
-* The [SRIOV Network device plugin](https://github.com/intel/sriov-network-device-plugin)
-* The [Intel device plugins](https://github.com/intel/intel-device-plugins-for-kubernetes) for GPU, FPGA and QuickAssist devices
 * The [Xilinx FPGA device plugins](https://github.com/Xilinx/FPGA_as_a_Service/tree/master/k8s-fpga-device-plugin/trunk) for Xilinx FPGA devices
 
 {{% /capture %}}
+{{% capture whatsnext %}}
 
+* Learn about [scheduling GPU resources](/docs/tasks/manage-gpus/scheduling-gpus/) using device plugins
+* Learn about [advertising extended resources](/docs/tasks/administer-cluster/extended-resource-node/) on a node
+* Read about using [hardware acceleration for TLS ingress](https://kubernetes.io/blog/2019/04/24/hardware-accelerated-ssl-tls-termination-in-ingress-controllers-using-kubernetes-device-plugins-and-runtimeclass/) with Kubernetes
+
+{{% /capture %}}

content/en/docs/reference/glossary/device-plugin.md

@@ -4,7 +4,7 @@ id: device-plugin
 date: 2019-02-02
 full_link: /docs/concepts/extend-kubernetes/compute-storage-net/device-plugins/
 short_description: >
-  Device Plugins are containers running in Kubernetes that provide access to a vendor specific resource.
+  Containers running in Kubernetes that provide access to a vendor specific resource.
 aka:
 tags:
 - fundamental
@@ -14,4 +14,4 @@ tags:
 
 <!--more-->
 
-[Device Plugin](/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins/) are containers running in Kubernetes that provide access to a vendor specific resource. Device Plugins advertise these resources to kubelet and can be deployed manually or as a DeamonSet, rather than writing custom Kubernetes code.
+[Device Plugins](/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins/) are containers running in Kubernetes that provide access to a vendor-specific resource. Device Plugins advertise these resources to {{< glossary_tooltip term_id="kubelet" >}}. They can be deployed manually or as a {{< glossary_tooltip term_id="daemonset" >}}, rather than writing custom Kubernetes code.

content/en/docs/tasks/administer-cluster/extended-resource-node.md

@@ -10,8 +10,6 @@ This page shows how to specify extended resources for a Node.
 Extended resources allow cluster administrators to advertise node-level
 resources that would otherwise be unknown to Kubernetes.
 
-{{< feature-state state="stable" >}}
-
 {{% /capture %}}
 
 
@@ -60,7 +58,7 @@ you call dongles.
 
 Start a proxy, so that you can easily send requests to the Kubernetes API server:
 
-```
+```shell
 kubectl proxy
 ```
 
@@ -153,7 +151,7 @@ Then a Container could request any number of bytes of special storage, up to 800
 
 Here is a PATCH request that removes the dongle advertisement from a Node.
 
-```shell
+```
 PATCH /api/v1/nodes/<your-node-name>/status HTTP/1.1
 Accept: application/json
 Content-Type: application/json-patch+json
@@ -169,7 +167,7 @@ Host: k8s-master:8080
 
 Start a proxy, so that you can easily send requests to the Kubernetes API server:
 
-```
+```shell
 kubectl proxy
 ```
 
@@ -189,6 +187,8 @@ Verify that the dongle advertisement has been removed:
 kubectl describe node <your-node-name> | grep dongle
 ```
 
+(you should not see any output)
+
 {{% /capture %}}
 
 

content/en/docs/tasks/configure-pod-container/extended-resource.md

@@ -6,10 +6,10 @@ weight: 40
 
 {{% capture overview %}}
 
-This page shows how to assign extended resources to a Container.
-
 {{< feature-state state="stable" >}}
 
+This page shows how to assign extended resources to a Container.
+
 {{% /capture %}}
 
 
@@ -141,9 +141,3 @@ kubectl delete pod extended-resource-demo-2
 * [Advertise Extended Resources for a Node](/docs/tasks/administer-cluster/extended-resource-node/)
 
 {{% /capture %}}
-
-
-
-
-
-

content/en/docs/tasks/manage-gpus/scheduling-gpus.md

@@ -7,10 +7,10 @@ title: Schedule GPUs
 
 {{% capture overview %}}
 
-Kubernetes includes **experimental** support for managing AMD and NVIDIA GPUs spread
+{{< feature-state state="beta" for_k8s_version="1.10" >}}
-across nodes. The support for NVIDIA GPUs was added in v1.6 and has gone through
+
-multiple backwards incompatible iterations.  The support for AMD GPUs was added in
+Kubernetes includes **experimental** support for managing AMD and NVIDIA GPUs
-v1.9 via [device plugin](#deploying-amd-gpu-device-plugin).
+(graphical processing units) across several nodes.
 
 This page describes how users can consume GPUs across different Kubernetes versions
 and the current limitations.
@@ -20,22 +20,20 @@ and the current limitations.
 
 {{% capture body %}}
 
-## v1.8 onwards
+## Using device plugins
 
-**From 1.8 onwards, the recommended way to consume GPUs is to use [device
+Kubernetes implements {{< glossary_tooltip text="Device Plugins" term_id="device-plugin" >}}
-plugins](/docs/concepts/cluster-administration/device-plugins).**
+to let Pods access specialized hardware features such as GPUs.
 
-To enable GPU support through device plugins before 1.10, the `DevicePlugins`
+As an administrator, you have to install GPU drivers from the corresponding
-feature gate has to be explicitly set to true across the system:
+hardware vendor on the nodes and run the corresponding device plugin from the
-`--feature-gates="DevicePlugins=true"`. This is no longer required starting
+GPU vendor:
-from 1.10.
 
-Then you have to install GPU drivers from the corresponding vendor on the nodes
+* [AMD](#deploying-amd-gpu-device-plugin)
-and run the corresponding device plugin from the GPU vendor
+* [NVIDIA](#deploying-nvidia-gpu-device-plugin)
-([AMD](#deploying-amd-gpu-device-plugin), [NVIDIA](#deploying-nvidia-gpu-device-plugin)).
 
-When the above conditions are true, Kubernetes will expose `nvidia.com/gpu` or
+When the above conditions are true, Kubernetes will expose `amd.com/gpu` or
-`amd.com/gpu` as a schedulable resource.
+`nvidia.com/gpu` as a schedulable resource.
 
 You can consume these GPUs from your containers by requesting
 `<vendor>.com/gpu` just like you request `cpu` or `memory`.
@@ -48,7 +46,7 @@ when using GPUs:
   * You can specify GPU in both `limits` and `requests` but these two values
     must be equal.
   * You cannot specify GPU `requests` without specifying `limits`.
-- Containers (and pods) do not share GPUs. There's no overcommitting of GPUs.
+- Containers (and Pods) do not share GPUs. There's no overcommitting of GPUs.
 - Each container can request one or more GPUs. It is not possible to request a
   fraction of a GPU.
 
@@ -79,14 +77,12 @@ has the following requirements:
 
 To deploy the AMD device plugin once your cluster is running and the above
 requirements are satisfied:
+```shell
+kubectl create -f https://raw.githubusercontent.com/RadeonOpenCompute/k8s-device-plugin/v1.10/k8s-ds-amdgpu-dp.yaml
 ```
-# For Kubernetes v1.9
-kubectl create -f https://raw.githubusercontent.com/RadeonOpenCompute/k8s-device-plugin/r1.9/k8s-ds-amdgpu-dp.yaml
 
-# For Kubernetes v1.10
+You can report issues with this third-party device plugin by logging an issue in
-kubectl create -f https://raw.githubusercontent.com/RadeonOpenCompute/k8s-device-plugin/r1.10/k8s-ds-amdgpu-dp.yaml
+[RadeonOpenCompute/k8s-device-plugin](https://github.com/RadeonOpenCompute/k8s-device-plugin).
-```
-Report issues with this device plugin to [RadeonOpenCompute/k8s-device-plugin](https://github.com/RadeonOpenCompute/k8s-device-plugin).
 
 ### Deploying NVIDIA GPU device plugin
 
@@ -99,22 +95,20 @@ has the following requirements:
 
 - Kubernetes nodes have to be pre-installed with NVIDIA drivers.
 - Kubernetes nodes have to be pre-installed with [nvidia-docker 2.0](https://github.com/NVIDIA/nvidia-docker)
-- nvidia-container-runtime must be configured as the [default runtime](https://github.com/NVIDIA/k8s-device-plugin#preparing-your-gpu-nodes)
+- Kubelet must use Docker as its container runtime
-  for docker instead of runc.
+- `nvidia-container-runtime` must be configured as the [default runtime](https://github.com/NVIDIA/k8s-device-plugin#preparing-your-gpu-nodes)
-- NVIDIA drivers ~= 361.93
+  for Docker, instead of runc.
+- The version of the NVIDIA drivers must match the constraint ~= 361.93
 
 To deploy the NVIDIA device plugin once your cluster is running and the above
 requirements are satisfied:
 
-```
+```shell
-# For Kubernetes v1.8
+kubectl create -f https://raw.githubusercontent.com/NVIDIA/k8s-device-plugin/1.0.0-beta/nvidia-device-plugin.yml
-kubectl create -f https://raw.githubusercontent.com/NVIDIA/k8s-device-plugin/v1.8/nvidia-device-plugin.yml
-
-# For Kubernetes v1.9
-kubectl create -f https://raw.githubusercontent.com/NVIDIA/k8s-device-plugin/v1.9/nvidia-device-plugin.yml
 ```
 
-Report issues with this device plugin to [NVIDIA/k8s-device-plugin](https://github.com/NVIDIA/k8s-device-plugin).
+You can report issues with this third-party device plugin by logging an issue in
+[NVIDIA/k8s-device-plugin](https://github.com/NVIDIA/k8s-device-plugin).
 
 #### NVIDIA GPU device plugin used by GCE
 
@@ -124,9 +118,9 @@ that is compatible with the Kubernetes Container Runtime Interface (CRI). It's t
 on [Container-Optimized OS](https://cloud.google.com/container-optimized-os/)
 and has experimental code for Ubuntu from 1.9 onwards.
 
-On your 1.12 cluster, you can use the following commands to install the NVIDIA drivers and device plugin:
+You can use the following commands to install the NVIDIA drivers and device plugin:
 
-```
+```shell
 # Install NVIDIA drivers on Container-Optimized OS:
 kubectl create -f https://raw.githubusercontent.com/GoogleCloudPlatform/container-engine-accelerators/stable/daemonset.yaml
 
@@ -134,13 +128,13 @@ kubectl create -f https://raw.githubusercontent.com/GoogleCloudPlatform/containe
 kubectl create -f https://raw.githubusercontent.com/GoogleCloudPlatform/container-engine-accelerators/stable/nvidia-driver-installer/ubuntu/daemonset.yaml
 
 # Install the device plugin:
-kubectl create -f https://raw.githubusercontent.com/kubernetes/kubernetes/release-1.12/cluster/addons/device-plugins/nvidia-gpu/daemonset.yaml
+kubectl create -f https://raw.githubusercontent.com/kubernetes/kubernetes/release-1.14/cluster/addons/device-plugins/nvidia-gpu/daemonset.yaml
 ```
 
-Report issues with this device plugin and installation method to [GoogleCloudPlatform/container-engine-accelerators](https://github.com/GoogleCloudPlatform/container-engine-accelerators).
+You can report issues with using or deploying this third-party device plugin by logging an issue in
+[GoogleCloudPlatform/container-engine-accelerators](https://github.com/GoogleCloudPlatform/container-engine-accelerators).
 
-Instructions for using NVIDIA GPUs on GKE are
+Google publishes its own [instructions](https://cloud.google.com/kubernetes-engine/docs/how-to/gpus) for using NVIDIA GPUs on GKE .
-[here](https://cloud.google.com/kubernetes-engine/docs/how-to/gpus)
 
 ## Clusters containing different types of GPUs
 
@@ -156,7 +150,14 @@ kubectl label nodes <node-with-k80> accelerator=nvidia-tesla-k80
 kubectl label nodes <node-with-p100> accelerator=nvidia-tesla-p100
 ```
 
-For AMD GPUs, you can deploy [Node Labeller](https://github.com/RadeonOpenCompute/k8s-device-plugin/tree/master/cmd/k8s-node-labeller), which automatically labels your nodes with GPU properties. Currently supported properties:
+## Automatic node labelling {#node-labeller}
+
+If you're using AMD GPU devices, you can deploy
+[Node Labeller](https://github.com/RadeonOpenCompute/k8s-device-plugin/tree/master/cmd/k8s-node-labeller).
+Node Labeller is a {{< glossary_tooltip text="controller" term_id="controller" >}} that automatically
+labels your nodes with GPU device properties.
+
+At the moment, that controller can add labels for:
 
 * Device ID (-device-id)
 * VRAM Size (-vram)
@@ -172,13 +173,11 @@ For AMD GPUs, you can deploy [Node Labeller](https://github.com/RadeonOpenComput
   * AI - Arctic Islands
   * RV - Raven
 
-Example result:
+```shell
-
-```console
 kubectl describe node cluster-node-23
 ```
-The output is similar to:
 
+```
     Name:               cluster-node-23
     Roles:              <none>
     Labels:             beta.amd.com/gpu.cu-count.64=1
@@ -191,9 +190,10 @@ The output is similar to:
                         kubernetes.io/hostname=cluster-node-23
     Annotations:        kubeadm.alpha.kubernetes.io/cri-socket: /var/run/dockershim.sock
                         node.alpha.kubernetes.io/ttl: 0
-    ......
+    …
+```
 
-Specify the GPU type in the pod spec:
+With the Node Labeller in use, you can specify the GPU type in the Pod spec:
 
 ```yaml
 apiVersion: v1
@@ -213,5 +213,7 @@ spec:
     accelerator: nvidia-tesla-p100 # or nvidia-tesla-k80 etc.
 ```
 
-This will ensure that the pod will be scheduled to a node that has the GPU type
+This will ensure that the Pod will be scheduled to a node that has the GPU type
 you specified.
+
+{{% /capture %}}