Merge pull request #26958 from alculquicondor/indexed-job

Documentation for Indexed completion mode

content/en/docs/concepts/workloads/controllers/job.md

@@ -145,8 +145,8 @@ There are three main types of task suitable to run as a Job:
    - the Job is complete as soon as its Pod terminates successfully.
 1. Parallel Jobs with a *fixed completion count*:
    - specify a non-zero positive value for `.spec.completions`.
-   - the Job represents the overall task, and is complete when there is one successful Pod for each value in the range 1 to `.spec.completions`.
+   - the Job represents the overall task, and is complete when there are `.spec.completions` successful Pods.
-   - **not implemented yet:** Each Pod is passed a different index in the range 1 to `.spec.completions`.
+   - when using `.spec.completionMode="Indexed"`, each Pod gets a different index in the range 0 to `.spec.completions-1`.
 1. Parallel Jobs with a *work queue*:
    - do not specify `.spec.completions`, default to `.spec.parallelism`.
    - the Pods must coordinate amongst themselves or an external service to determine what each should work on. For example, a Pod might fetch a batch of up to N items from the work queue.
@@ -166,7 +166,6 @@ a non-negative integer.
 
 For more information about how to make use of the different types of job, see the [job patterns](#job-patterns) section.
 
-
 #### Controlling parallelism
 
 The requested parallelism (`.spec.parallelism`) can be set to any non-negative value.
@@ -185,6 +184,33 @@ parallelism, for a variety of reasons:
 - The Job controller may throttle new Pod creation due to excessive previous pod failures in the same Job.
 - When a Pod is gracefully shut down, it takes time to stop.
 
+### Completion mode
+
+{{< feature-state for_k8s_version="v1.21" state="alpha" >}}
+
+{{< note >}}
+To be able to create Indexed Jobs, make sure to enable the `IndexedJob`
+[feature gate](/docs/reference/command-line-tools-reference/feature-gates/)
+on the [API server](docs/reference/command-line-tools-reference/kube-apiserver/)
+and the [controller manager](/docs/reference/command-line-tools-reference/kube-controller-manager/).
+{{< /note >}}
+
+Jobs with _fixed completion count_ - that is, jobs that have non null
+`.spec.completions` - can have a completion mode that is specified in `.spec.completionMode`:
+
+- `NonIndexed` (default): the Job is considered complete when there have been
+  `.spec.completions` successfully completed Pods. In other words, each Pod
+  completion is homologous to each other. Note that Jobs that have null
+  `.spec.completions` are implicitly `NonIndexed`.
+- `Indexed`: the Pods of a Job get an associated completion index from 0 to
+  `.spec.completions-1`, available in the annotation `batch.kubernetes.io/job-completion-index`.
+  The Job is considered complete when there is one successfully completed Pod
+  for each index. For more information about how to use this mode, see
+  [Indexed Job for Parallel Processing with Static Work Assignment](/docs/tasks/job/indexed-parallel-processing-static/).
+  Note that, although rare, more than one Pod could be started for the same
+  index, but only one of them will count towards the completion count.
+
+
 ## Handling Pod and container failures
 
 A container in a Pod may fail for a number of reasons, such as because the process in it exited with
@@ -348,12 +374,12 @@ The tradeoffs are:
 The tradeoffs are summarized here, with columns 2 to 4 corresponding to the above tradeoffs.
 The pattern names are also links to examples and more detailed description.
 
-|                            Pattern                                   | Single Job object | Fewer pods than work items? | Use app unmodified? |  Works in Kube 1.1? |
+|                  Pattern                  | Single Job object | Fewer pods than work items? | Use app unmodified? |
-| -------------------------------------------------------------------- |:-----------------:|:---------------------------:|:-------------------:|:-------------------:|
+| ----------------------------------------- |:-----------------:|:---------------------------:|:-------------------:|
-| [Job Template Expansion](/docs/tasks/job/parallel-processing-expansion/)            |                   |                             |          ✓          |          ✓          |
+| [Queue with Pod Per Work Item]            |         ✓         |                             |      sometimes      |
-| [Queue with Pod Per Work Item](/docs/tasks/job/coarse-parallel-processing-work-queue/)   |         ✓         |                             |      sometimes      |          ✓          |
+| [Queue with Variable Pod Count]           |         ✓         |             ✓               |                     |
-| [Queue with Variable Pod Count](/docs/tasks/job/fine-parallel-processing-work-queue/)  |         ✓         |             ✓               |                     |          ✓          |
+| [Indexed Job with Static Work Assignment] |         ✓         |                             |          ✓          | 
-| Single Job with Static Work Assignment                               |         ✓         |                             |          ✓          |                     |
+| [Job Template Expansion]                  |                   |                             |          ✓          |
 
 When you specify completions with `.spec.completions`, each Pod created by the Job controller
 has an identical [`spec`](https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status).  This means that
@@ -364,13 +390,17 @@ are different ways to arrange for pods to work on different things.
 This table shows the required settings for `.spec.parallelism` and `.spec.completions` for each of the patterns.
 Here, `W` is the number of work items.
 
-|                             Pattern                                  | `.spec.completions` |  `.spec.parallelism` |
+|             Pattern                       | `.spec.completions` |  `.spec.parallelism` |
-| -------------------------------------------------------------------- |:-------------------:|:--------------------:|
+| ----------------------------------------- |:-------------------:|:--------------------:|
-| [Job Template Expansion](/docs/tasks/job/parallel-processing-expansion/)           |          1          |     should be 1      |
+| [Queue with Pod Per Work Item]            |          W          |        any           |
-| [Queue with Pod Per Work Item](/docs/tasks/job/coarse-parallel-processing-work-queue/)   |          W          |        any           |
+| [Queue with Variable Pod Count]           |         null        |        any           |
-| [Queue with Variable Pod Count](/docs/tasks/job/fine-parallel-processing-work-queue/)  |          1          |        any           |
+| [Indexed Job with Static Work Assignment] |          W          |        any           |
-| Single Job with Static Work Assignment                               |          W          |        any           |
+| [Job Template Expansion]                  |          1          |     should be 1      |
 
+[Queue with Pod Per Work Item]: /docs/tasks/job/coarse-parallel-processing-work-queue/
+[Queue with Variable Pod Count]: /docs/tasks/job/fine-parallel-processing-work-queue/
+[Indexed Job with Static Work Assignment]: /docs/tasks/job/indexed-parallel-processing-static/
+[Job Template Expansion]: /docs/tasks/job/parallel-processing-expansion/
 
 ## Advanced usage
 

content/en/docs/reference/command-line-tools-reference/feature-gates.md

@@ -261,6 +261,7 @@ different Kubernetes components.
 | `ImmutableEphemeralVolumes` | `false` | Alpha | 1.18 | 1.18 |
 | `ImmutableEphemeralVolumes` | `true` | Beta | 1.19 | 1.20 |
 | `ImmutableEphemeralVolumes` | `true` | GA | 1.21 | |
+| `IndexedJob` | `false` | Alpha | 1.21 | |
 | `Initializers` | `false` | Alpha | 1.7 | 1.13 |
 | `Initializers` | - | Deprecated | 1.14 | - |
 | `KubeletConfigFile` | `false` | Alpha | 1.8 | 1.9 |
@@ -630,10 +631,12 @@ Each feature gate is designed for enabling/disabling a specific feature:
 - `HyperVContainer`: Enable
   [Hyper-V isolation](https://docs.microsoft.com/en-us/virtualization/windowscontainers/manage-containers/hyperv-container)
   for Windows containers.
-- `IPv6DualStack`: Enable [dual stack](/docs/concepts/services-networking/dual-stack/)
-  support for IPv6.
 - `ImmutableEphemeralVolumes`: Allows for marking individual Secrets and ConfigMaps as
   immutable for better safety and performance.
+- `IndexedJob`: Allows the [Job](/docs/concepts/workloads/controllers/job/)
+  controller to manage Pod completions per completion index.
+- `IPv6DualStack`: Enable [dual stack](/docs/concepts/services-networking/dual-stack/)
+  support for IPv6.
 - `KubeletConfigFile` (*deprecated*): Enable loading kubelet configuration from
   a file specified using a config file.
   See [setting kubelet parameters via a config file](/docs/tasks/administer-cluster/kubelet-config-file/)

content/en/docs/tasks/job/coarse-parallel-processing-work-queue.md

@@ -2,7 +2,7 @@
 title: Coarse Parallel Processing Using a Work Queue
 min-kubernetes-server-version: v1.8
 content_type: task
-weight: 30
+weight: 20
 ---
 
 

content/en/docs/tasks/job/fine-parallel-processing-work-queue.md

@@ -2,7 +2,7 @@
 title: Fine Parallel Processing Using a Work Queue
 content_type: task
 min-kubernetes-server-version: v1.8
-weight: 40
+weight: 30
 ---
 
 <!-- overview -->

content/en/docs/tasks/job/indexed-parallel-processing-static.md

@@ -0,0 +1,176 @@
+---
+title: Indexed Job for Parallel Processing with Static Work Assignment
+content_type: task
+min-kubernetes-server-version: v1.21
+weight: 30
+---
+
+{{< feature-state for_k8s_version="v1.21" state="alpha" >}}
+
+<!-- overview -->
+
+
+In this example, you will run a Kubernetes Job that uses multiple parallel
+worker processes.
+Each worker is a different container running in its own Pod. The Pods have an
+_index number_ that the control plane sets automatically, which allows each Pod
+to identify which part of the overall task to work on.
+
+The pod index is available in the {{< glossary_tooltip text="annotation" term_id="annotation" >}}
+`batch.kubernetes.io/job-completion-index` as string representing its
+decimal value. In order for the containerized task process to obtain this index,
+you can publish the value of the annotation using the [downward API](/docs/tasks/inject-data-application/downward-api-volume-expose-pod-information/#the-downward-api)
+mechanism.
+For convenience, the control plane automatically sets the downward API to
+expose the index in the `JOB_COMPLETION_INDEX` environment variable.
+
+Here is an overview of the steps in this example:
+
+1. **Create an image that can read the pod index**. You might modify the worker
+   program or add a script wrapper.
+2. **Start an Indexed Job**. The downward API allows you to pass the annotation
+   as an environment variable or file to the container.
+
+## {{% heading "prerequisites" %}}
+
+Be familiar with the basic,
+non-parallel, use of [Job](/docs/concepts/workloads/controllers/job/).
+
+{{< include "task-tutorial-prereqs.md" >}} {{< version-check >}}
+
+To be able to create Indexed Jobs, make sure to enable the `IndexedJob`
+[feature gate](/docs/reference/command-line-tools-reference/feature-gates/)
+on the [API server](docs/reference/command-line-tools-reference/kube-apiserver/)
+and the [controller manager](/docs/reference/command-line-tools-reference/kube-controller-manager/).
+
+<!-- steps -->
+
+## Choose an approach
+
+To access the work item from the worker program, you have a few options:
+
+1. Read the `JOB_COMPLETION_INDEX` environment variable. The Job
+   {{< glossary_tooltip text="controller" term_id="controller" >}}
+   automatically links this variable to the annotation containing the completion
+   index.
+1. Read a file that contains the completion index.
+1. Assuming that you can't modify the program, you can wrap it with a script
+   that reads the index using any of the methods above and converts it into
+   something that the program can use as input.
+  
+For this example, imagine that you chose option 3 and you want to run the
+[rev](https://man7.org/linux/man-pages/man1/rev.1.html) utility. This
+program accepts a file as an argument and prints its content reversed.
+
+```shell
+rev data.txt
+```
+
+For this example, you'll use the `rev` tool from the
+[`busybox`](https://hub.docker.com/_/busybox) container image.
+
+## Define an Indexed Job
+
+Here is a job definition. You'll need to edit the container image to match your
+preferred registry.
+
+{{< codenew language="yaml" file="application/job/indexed-job.yaml" >}}
+
+In the example above, you use the builtin `JOB_COMPLETION_INDEX` environment
+variable set by the Job controller for all containers. An [init container](/docs/concepts/workloads/pods/init-containers/)
+maps the index to a static value and writes it to a file that is shared with the
+container running the worker through an [emptyDir volume](/docs/concepts/storage/volumes/#emptydir).
+Optionally, you can [define your own environment variable through the downward
+API](/docs/tasks/inject-data-application/environment-variable-expose-pod-information/)
+to publish the index to containers. You can also choose to load a list of values
+from a [ConfigMap as an environment variable or file](/docs/tasks/configure-pod-container/configure-pod-configmap/).
+
+Alternatively, you can directly [use the downward API to pass the annotation
+value as a volume file](/docs/tasks/inject-data-application/downward-api-volume-expose-pod-information/#store-pod-fields),
+like shown in the following example:
+
+{{< codenew language="yaml" file="application/job/indexed-job-vol.yaml" >}}
+
+## Running the Job
+
+Now run the Job:
+
+```shell
+kubectl apply -f ./indexed-job.yaml
+```
+
+Wait a bit, then check on the job:
+
+```shell
+kubectl describe jobs/indexed-job
+```
+
+The output is similar to:
+
+```
+Name:              indexed-job
+Namespace:         default
+Selector:          controller-uid=bf865e04-0b67-483b-9a90-74cfc4c3e756
+Labels:            controller-uid=bf865e04-0b67-483b-9a90-74cfc4c3e756
+                   job-name=indexed-job
+Annotations:       <none>
+Parallelism:       3
+Completions:       5
+Start Time:        Thu, 11 Mar 2021 15:47:34 +0000
+Pods Statuses:     2 Running / 3 Succeeded / 0 Failed
+Completed Indexes: 0-2
+Pod Template:
+  Labels:  controller-uid=bf865e04-0b67-483b-9a90-74cfc4c3e756
+           job-name=indexed-job
+  Init Containers:
+   input:
+    Image:      docker.io/library/bash
+    Port:       <none>
+    Host Port:  <none>
+    Command:
+      bash
+      -c
+      items=(foo bar baz qux xyz)
+      echo ${items[$JOB_COMPLETION_INDEX]} > /input/data.txt
+
+    Environment:  <none>
+    Mounts:
+      /input from input (rw)
+  Containers:
+   worker:
+    Image:      docker.io/library/busybox
+    Port:       <none>
+    Host Port:  <none>
+    Command:
+      rev
+      /input/data.txt
+    Environment:  <none>
+    Mounts:
+      /input from input (rw)
+  Volumes:
+   input:
+    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
+    Medium:
+    SizeLimit:  <unset>
+Events:
+  Type    Reason            Age   From            Message
+  ----    ------            ----  ----            -------
+  Normal  SuccessfulCreate  4s    job-controller  Created pod: indexed-job-njkjj
+  Normal  SuccessfulCreate  4s    job-controller  Created pod: indexed-job-9kd4h
+  Normal  SuccessfulCreate  4s    job-controller  Created pod: indexed-job-qjwsz
+  Normal  SuccessfulCreate  1s    job-controller  Created pod: indexed-job-fdhq5
+  Normal  SuccessfulCreate  1s    job-controller  Created pod: indexed-job-ncslj
+```
+
+In this example, we run the job with custom values for each index. You can
+inspect the output of the pods:
+
+```shell
+kubectl logs indexed-job-fdhq5 # Change this to match the name of a Pod in your cluster.
+```
+
+The output is similar to:
+
+```
+xuq
+```

content/en/docs/tasks/job/parallel-processing-expansion.md

@@ -2,7 +2,7 @@
 title: Parallel Processing using Expansions
 content_type: task
 min-kubernetes-server-version: v1.8
-weight: 20
+weight: 50
 ---
 
 <!-- overview -->

content/en/examples/application/job/indexed-job-vol.yaml

@@ -0,0 +1,27 @@
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: 'indexed-job'
+spec:
+  completions: 5
+  parallelism: 3
+  completionMode: Indexed
+  template:
+    spec:
+      restartPolicy: Never
+      containers:
+      - name: 'worker'
+        image: 'docker.io/library/busybox'
+        command:
+        - "rev"
+        - "/input/data.txt"
+        volumeMounts:
+        - mountPath: /input
+          name: input
+      volumes:
+      - name: input
+        downwardAPI:
+          items:
+          - path: "data.txt"
+            fieldRef:
+              fieldPath: metadata.annotations['batch.kubernetes.io/job-completion-index']

content/en/examples/application/job/indexed-job.yaml

@@ -0,0 +1,35 @@
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: 'indexed-job'
+spec:
+  completions: 5
+  parallelism: 3
+  completionMode: Indexed
+  template:
+    spec:
+      restartPolicy: Never
+      initContainers:
+      - name: 'input'
+        image: 'docker.io/library/bash'
+        command:
+        - "bash"
+        - "-c"
+        - |
+          items=(foo bar baz qux xyz)
+          echo ${items[$JOB_COMPLETION_INDEX]} > /input/data.txt
+        volumeMounts:
+        - mountPath: /input
+          name: input
+      containers:
+      - name: 'worker'
+        image: 'docker.io/library/busybox'
+        command:
+        - "rev"
+        - "/input/data.txt"
+        volumeMounts:
+        - mountPath: /input
+          name: input
+      volumes:
+      - name: input
+        emptyDir: {}