whentimeslows/website
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Switch language name 'zh' to 'zh-cn'

Browse Source 
This is the first step to rename 'zh' to 'zh-cn'. There are several reasons why we rename the language name.

- The upstream docsy theme changed the language name, leading to many warnings during site build;
  The side-effect is that the i18n strings are no longer working.
- We believe renaming the language is the right thing to do, because this move can make room for other variants of Chinese language, such as 'zh-tw', 'zh-sg' etc.

There would be several follow-ups to this PR, such as fixing the intra-site links, adding redirects etc.

We will lock up changes to zh/zh-cn pages for the moment, until this one gets in.

This PR is based on commit cdad0a7342.
This commit is contained in:
Qiming Teng
2022-06-10 20:25:27 +08:00
parent 3d345b1816
commit c52818c03d
1347 changed files with 8 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
content/zh-cn/docs/tasks/job/_index.md
+5
View File
@@ -0,0 +1,5 @@
---
title: "运行 Jobs"
weight: 50
description: 使用并行处理运行 Jobs。
---
content/zh-cn/docs/tasks/job/automated-tasks-with-cron-jobs.md
+368
View File
@@ -0,0 +1,368 @@
---
title: 使用 CronJob 运行自动化任务
content_type: task
weight: 10
min-kubernetes-server-version: v1.21
---
<!--
title: Running Automated Tasks with a CronJob
reviewers:
- chenopis
content_type: task
weight: 10
min-kubernetes-server-version: v1.21
-->
<!-- overview -->
<!--
CronJobs was promoted to general availability in Kubernetes v1.21. If you are using an older version of
Kubernetes, please refer to the documentation for the version of Kubernetes that you are using,
so that you see accurate information. Older Kubernetes versions do not support the `batch/v1` CronJob API.
You can use [CronJobs](/docs/concepts/workloads/controllers/cron-jobs) to run jobs on a time-based schedule.
These automated jobs run like [Cron](https://en.wikipedia.org/wiki/Cron) tasks on a Linux or UNIX system.
Cron jobs are useful for creating periodic and recurring tasks, like running backups or sending emails.
Cron jobs can also schedule individual tasks for a specific time, such as if you want to schedule a job for a low activity period.
-->
在Kubernetes v1.21 版本中，CronJob 被提升为通用版本。如果你使用的是旧版本的 Kubernetes，请参考你正在使用的 Kubernetes 版本的文档，这样你就能看到准确的信息。旧的 Kubernetes 版本不支持`batch/v1` CronJob API。
你可以利用 [CronJobs](/zh/docs/concepts/workloads/controllers/cron-jobs) 执行基于时间调度的任务。这些自动化任务和 Linux 或者 Unix 系统的 [Cron](https://en.wikipedia.org/wiki/Cron) 任务类似。
CronJobs 在创建周期性以及重复性的任务时很有帮助，例如执行备份操作或者发送邮件。CronJobs 也可以在特定时间调度单个任务，例如你想调度低活跃周期的任务。
<!--
Cron jobs have limitations and idiosyncrasies.
For example, in certain circumstances, a single cron job can create multiple jobs.
Therefore, jobs should be idempotent.
For more limitations, see [CronJobs](/docs/concepts/workloads/controllers/cron-jobs).
-->
CronJobs 有一些限制和特点。
例如，在特定状况下，同一个 CronJob 可以创建多个任务。
因此，任务应该是幂等的。
查看更多限制，请参考 [CronJobs](/zh/docs/concepts/workloads/controllers/cron-jobs)。
## {{% heading "prerequisites" %}}
* {{< include "task-tutorial-prereqs.md" >}} {{< version-check >}}
<!-- steps -->
<!--
## Creating a Cron Job
Cron jobs require a config file.
This example cron job config `.spec` file prints the current time and a hello message every minute:
-->
## 创建 CronJob
CronJob 需要一个配置文件。
本例中 CronJob 的`.spec` 配置文件每分钟打印出当前时间和一个问好信息：
{{< codenew file="application/job/cronjob.yaml" >}}
<!--
Run the example cron job by downloading the example file and then running this command:
-->
想要运行示例的 CronJob，可以下载示例文件并执行命令：
```shell
kubectl create -f https://k8s.io/examples/application/job/cronjob.yaml
```
```
cronjob.batch/hello created
```
<!--
After creating the cron job, get its status using this command:
-->
创建好 CronJob 后，使用下面的命令来获取其状态：
```shell
kubectl get cronjob hello
```
<!--
The output is similar to this:
-->
输出类似于：
```
NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE
hello */1 * * * * False 0 50s 75s
```
<!--
As you can see from the results of the command, the cron job has not scheduled or run any jobs yet.
Watch for the job to be created in around one minute:
-->
就像你从命令返回结果看到的那样，CronJob 还没有调度或执行任何任务。大约需要一分钟任务才能创建好。
```shell
kubectl get jobs --watch
```
```
NAME COMPLETIONS DURATION AGE
hello-4111706356 0/1 0s
hello-4111706356 0/1 0s 0s
hello-4111706356 1/1 5s 5s
```
<!--
Now you've seen one running job scheduled by the "hello" cron job.
You can stop watching the job and view the cron job again to see that it scheduled the job:
-->
现在你已经看到了一个运行中的任务被 “hello” CronJob 调度。
你可以停止监视这个任务，然后再次查看 CronJob 就能看到它调度任务：
```shell
kubectl get cronjob hello
```
<!--
The output is similar to this:
-->
输出类似于：
```
NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE
hello */1 * * * * False 0 50s 75s
```
<!--
You should see that the cron job `hello` successfully scheduled a job at the time specified in `LAST SCHEDULE`. There are currently 0 active jobs, meaning that the job has completed or failed.
Now, find the pods that the last scheduled job created and view the standard output of one of the pods.
-->
你应该能看到 `hello` CronJob 在 `LAST SCHEDULE` 声明的时间点成功的调度了一次任务。
有 0 个活跃的任务意味着任务执行完毕或者执行失败。
现在，找到最后一次调度任务创建的 Pod 并查看一个 Pod 的标准输出。
<!--
The job name and pod name are different.
-->
{{< note >}}
Job 名称和 Pod 名称不同。
{{< /note >}}
```shell
# 在你的系统上将 "hello-4111706356" 替换为 Job 名称
pods=$(kubectl get pods --selector=job-name=hello-4111706356 --output=jsonpath={.items..metadata.name})
```
<!--
Show pod log:
-->
查看 Pod 日志：
```shell
kubectl logs $pods
```
<!--
The output is similar to this:
-->
输出与此类似：
```
Fri Feb 22 11:02:09 UTC 2019
Hello from the Kubernetes cluster
```
<!--
## Deleting a Cron Job
When you don't need a cron job any more, delete it with `kubectl delete cronjob <cronjob name>`：
-->
## 删除 CronJob
当你不再需要 CronJob 时，可以用 `kubectl delete cronjob <cronjob name>` 删掉它：
```shell
kubectl delete cronjob hello
```
<!--
Deleting the cron job removes all the jobs and pods it created and stops it from creating additional jobs.
You can read more about removing jobs in [garbage collection](/docs/concepts/workloads/controllers/garbage-collection/).
-->
删除 CronJob 会清除它创建的所有任务和 Pod，并阻止它创建额外的任务。你可以查阅
[垃圾收集](/zh/docs/concepts/workloads/controllers/garbage-collection/)。
<!--
## Writing a Cron Job Spec
As with all other Kubernetes configs, a cron job needs `apiVersion`, `kind`, and `metadata` fields. For general
information about working with config files, see [deploying applications](/docs/tasks/run-application/run-stateless-application-deployment/),
and [using kubectl to manage resources](/docs/concepts/overview/working-with-objects/object-management/) documents.
A cron job config also needs a [`.spec` section](https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status).
-->
## 编写 CronJob 声明信息
像 Kubernetes 的其他配置一样，CronJob 需要 `apiVersion`、`kind`、和 `metadata` 域。
配置文件的一般信息，请参考
[部署应用](/zh/docs/tasks/run-application/run-stateless-application-deployment/) 和
[使用 kubectl 管理资源](/zh/docs/concepts/overview/working-with-objects/object-management/).
CronJob 配置也需要包括
[`.spec`](https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status).
<!--
All modifications to a cron job, especially its `.spec`, are applied only to the following runs.
-->
{{< note >}}
对 CronJob 的所有改动，特别是它的 `.spec`，只会影响将来的运行实例。
{{< /note >}}
<!--
### Schedule
The `.spec.schedule` is a required field of the `.spec`.
It takes a [Cron](https://en.wikipedia.org/wiki/Cron) format string, such as `0 * * * *` or `@hourly`, as schedule time of its jobs to be created and executed.
-->
### 时间安排
`.spec.schedule` 是 `.spec` 需要的域。它使用了 [Cron](https://en.wikipedia.org/wiki/Cron)
格式串，例如 `0 * * * *` or `@hourly` ，作为它的任务被创建和执行的调度时间。
<!--
The format also includes extended "Vixie cron" step values. As explained in the [FreeBSD manual](https://www.freebsd.org/cgi/man.cgi?crontab%285%29):
-->
该格式也包含了扩展的 "Vixie cron" 步长值。
[FreeBSD 手册](https://www.freebsd.org/cgi/man.cgi?crontab%285%29)中解释如下:
<!--
> Step values can be used in conjunction with ranges. Following a range
> with `/<number>` specifies skips of the number's value through the
> range. For example, `0-23/2` can be used in the hours field to specify
> command execution every other hour (the alternative in the V7 standard is
> `0,2,4,6,8,10,12,14,16,18,20,22`). Steps are also permitted after an
> asterisk, so if you want to say "every two hours", just use `*/2`.
-->
> 步长可被用于范围组合。范围后面带有 `/<数字>` 可以声明范围内的步幅数值。
> 例如，`0-23/2` 可被用在小时域来声明命令在其他数值的小时数执行
> （ V7 标准中对应的方法是`0,2,4,6,8,10,12,14,16,18,20,22`）。
> 步长也可以放在通配符后面，因此如果你想表达 "每两小时"，就用 `*/2` 。
<!--
A question mark (`?`) in the schedule has the same meaning as an asterisk `*`, that is, it stands for any of available value for a given field.
-->
{{< note >}}
调度中的问号 (`?`) 和星号 `*` 含义相同，表示给定域的任何可用值。
{{< /note >}}
<!--
### Job Template
The `.spec.jobTemplate` is the template for the job, and it is required.
It has exactly the same schema as a [Job](/docs/concepts/workloads/controllers/job/),
except that it is nested and does not have an `apiVersion` or `kind`.
For information about writing a job `.spec`, see
[Writing a Job Spec](/docs/concepts/workloads/controllers/job/#writing-a-job-spec).
-->
### 任务模版
`.spec.jobTemplate`是任务的模版，它是必须的。它和
[Job](/zh/docs/concepts/workloads/controllers/job/)的语法完全一样，
除了它是嵌套的没有 `apiVersion` 和 `kind`。
编写任务的 `.spec` ，请参考
[编写 Job 的Spec](/zh/docs/concepts/workloads/controllers/job/#writing-a-job-spec)。
<!--
### Starting Deadline
The `.spec.startingDeadlineSeconds` field is optional.
It stands for the deadline in seconds for starting the job if it misses its scheduled time for any reason.
After the deadline, the cron job does not start the job.
Jobs that do not meet their deadline in this way count as failed jobs.
If this field is not specified, the jobs have no deadline.
-->
### 开始的最后期限 {#starting-deadline}
`.spec.startingDeadlineSeconds` 域是可选的。
它表示任务如果由于某种原因错过了调度时间，开始该任务的截止时间的秒数。过了截止时间，CronJob 就不会开始任务。
不满足这种最后期限的任务会被统计为失败任务。如果该域没有声明，那任务就没有最后期限。
<!--
If the `.spec.startingDeadlineSeconds` field is set (not null), the CronJob
controller measures the time between when a job is expected to be created and
now. If the difference is higher than that limit, it will skip this execution.
For example, if it is set to `200`, it allows a job to be created for up to 200
seconds after the actual schedule.
-->
如果`.spec.startingDeadlineSeconds`字段被设置(非空)，CronJob 控制器会计算从预期创建 Job 到当前时间的时间差。
如果时间差大于该限制，则跳过此次执行。
例如，如果将其设置为 `200`，则 Job 控制器允许在实际调度之后最多 200 秒内创建 Job。
<!--
### Concurrency Policy
The `.spec.concurrencyPolicy` field is also optional.
It specifies how to treat concurrent executions of a job that is created by this cron job.
the spec may specify only one of the following concurrency policies:
* `Allow` (default): The cron job allows concurrently running jobs
* `Forbid`: The cron job does not allow concurrent runs; if it is time for a new job run and the previous job run hasn't finished yet, the cron job skips the new job run
* `Replace`: If it is time for a new job run and the previous job run hasn't finished yet, the cron job replaces the currently running job run with a new job run
Note that concurrency policy only applies to the jobs created by the same cron job.
If there are multiple cron jobs, their respective jobs are always allowed to run concurrently.
-->
### 并发性规则
`.spec.concurrencyPolicy` 也是可选的。它声明了 CronJob 创建的任务执行时发生重叠如何处理。
spec 仅能声明下列规则中的一种：
* `Allow` (默认)：CronJob 允许并发任务执行。
* `Forbid`： CronJob 不允许并发任务执行；如果新任务的执行时间到了而老任务没有执行完，CronJob 会忽略新任务的执行。
* `Replace`：如果新任务的执行时间到了而老任务没有执行完，CronJob 会用新任务替换当前正在运行的任务。
请注意，并发性规则仅适用于相同 CronJob 创建的任务。如果有多个 CronJob，它们相应的任务总是允许并发执行的。
<!--
### Suspend
The `.spec.suspend` field is also optional.
If it is set to `true`, all subsequent executions are suspended.
This setting does not apply to already started executions.
Defaults to false.
-->
### 挂起
`.spec.suspend`域也是可选的。如果设置为 `true` ，后续发生的执行都会挂起。
这个设置对已经开始的执行不起作用。默认是关闭的。
<!--
Executions that are suspended during their scheduled time count as missed jobs.
When `.spec.suspend` changes from `true` to `false` on an existing cron job without a [starting deadline](#starting-deadline), the missed jobs are scheduled immediately.
-->
{{< caution >}}
在调度时间内挂起的执行都会被统计为错过的任务。当 `.spec.suspend` 从 `true` 改为 `false` 时，
且没有 [开始的最后期限](#starting-deadline)，错过的任务会被立即调度。
{{< /caution >}}
<!--
### Jobs History Limits
The `.spec.successfulJobsHistoryLimit` and `.spec.failedJobsHistoryLimit` fields are optional.
These fields specify how many completed and failed jobs should be kept.
By default, they are set to 3 and 1 respectively. Setting a limit to `0` corresponds to keeping none of the corresponding kind of jobs after they finish.
-->
### 任务历史限制
`.spec.successfulJobsHistoryLimit` 和 `.spec.failedJobsHistoryLimit`是可选的。
这两个字段指定应保留多少已完成和失败的任务。
默认设置为3和1。限制设置为 `0` 代表相应类型的任务完成后不会保留。
content/zh-cn/docs/tasks/job/coarse-parallel-processing-work-queue.md
+502
View File
@@ -0,0 +1,502 @@
---
title: 使用工作队列进行粗粒度并行处理
min-kubernetes-server-version: v1.8
content_type: task
weight: 20
---
<!--
---
title: Coarse Parallel Processing Using a Work Queue
min-kubernetes-server-version: v1.8
content_type: task
weight: 20
---
-->
<!-- overview -->
<!--
In this example, we will run a Kubernetes Job with multiple parallel
worker processes.
In this example, as each pod is created, it picks up one unit of work
from a task queue, completes it, deletes it from the queue, and exits.
Here is an overview of the steps in this example:
1. **Start a message queue service.** In this example, we use RabbitMQ, but you could use another
one. In practice you would set up a message queue service once and reuse it for many jobs.
1. **Create a queue, and fill it with messages.** Each message represents one task to be done. In
this example, a message is an integer that we will do a lengthy computation on.
1. **Start a Job that works on tasks from the queue**. The Job starts several pods. Each pod takes
one task from the message queue, processes it, and repeats until the end of the queue is reached.
-->
本例中，我们会运行包含多个并行工作进程的 Kubernetes Job。
本例中，每个 Pod 一旦被创建，会立即从任务队列中取走一个工作单元并完成它，然后将工作单元从队列中删除后再退出。
下面是本次示例的主要步骤：
1. **启动一个消息队列服务** 本例中，我们使用 RabbitMQ，你也可以用其他的消息队列服务。在实际工作环境中，你可以创建一次消息队列服务然后在多个任务中重复使用。
1. **创建一个队列，放上消息数据** 每个消息表示一个要执行的任务。本例中，每个消息是一个整数值。我们将基于这个整数值执行很长的计算操作。
1. **启动一个在队列中执行这些任务的 Job**。该 Job 启动多个 Pod。每个 Pod 从消息队列中取走一个任务，处理它，然后重复执行，直到队列的队尾。
## {{% heading "prerequisites" %}}
<!--
Be familiar with the basic,
non-parallel, use of [Job](/docs/concepts/jobs/run-to-completion-finite-workloads/).
-->
要熟悉 Job 基本用法（非并行的），请参考
[Job](/zh/docs/concepts/workloads/controllers/job/)。
{{< include "task-tutorial-prereqs.md" >}} {{< version-check >}}
<!-- steps -->
<!--
## Starting a message queue service
This example uses RabbitMQ, however, you can adapt the example to use another AMQP-type message service.
In practice you could set up a message queue service once in a
cluster and reuse it for many jobs, as well as for long-running services.
Start RabbitMQ as follows:
-->
## 启动消息队列服务
本例使用了 RabbitMQ，但你可以更改该示例，使用其他 AMQP 类型的消息服务。
在实际工作中，在集群中一次性部署某个消息队列服务，之后在很多 Job 中复用，包括需要长期运行的服务。
按下面的方法启动 RabbitMQ：
```shell
kubectl create -f https://raw.githubusercontent.com/kubernetes/kubernetes/release-1.3/examples/celery-rabbitmq/rabbitmq-service.yaml
```
```
service "rabbitmq-service" created
```
```shell
kubectl create -f https://raw.githubusercontent.com/kubernetes/kubernetes/release-1.3/examples/celery-rabbitmq/rabbitmq-controller.yaml
```
```
replicationcontroller "rabbitmq-controller" created
```
<!--
We will only use the rabbitmq part from the [celery-rabbitmq example](https://github.com/kubernetes/kubernetes/tree/release-1.3/examples/celery-rabbitmq).
-->
我们仅用到 [celery-rabbitmq 示例](https://github.com/kubernetes/kubernetes/tree/release-1.3/examples/celery-rabbitmq) 中描述的部分功能。
<!--
## Testing the message queue service
Now, we can experiment with accessing the message queue. We will
create a temporary interactive pod, install some tools on it,
and experiment with queues.
First create a temporary interactive Pod.
-->
## 测试消息队列服务
现在，我们可以试着访问消息队列。我们将会创建一个临时的可交互的 Pod，在它上面安装一些工具，然后用队列做实验。
首先创建一个临时的可交互的 Pod：
```shell
# 创建一个临时的可交互的 Pod
kubectl run -i --tty temp --image ubuntu:14.04
```
```
Waiting for pod default/temp-loe07 to be running, status is Pending, pod ready: false
... [ previous line repeats several times .. hit return when it stops ] ...
```
<!--
Note that your pod name and command prompt will be different.
Next install the `amqp-tools` so we can work with message queues.
-->
请注意你的 Pod 名称和命令提示符将会不同。
接下来安装 `amqp-tools` ，这样我们就能用消息队列了。
```shell
# 安装一些工具
root@temp-loe07:/# apt-get update
.... [ lots of output ] ....
root@temp-loe07:/# apt-get install -y curl ca-certificates amqp-tools python dnsutils
.... [ lots of output ] ....
```
<!--
Later, we will make a docker image that includes these packages.
Next, we will check that we can discover the rabbitmq service:
-->
后续，我们将制作一个包含这些包的 Docker 镜像。
接着，我们将要验证我们发现 RabbitMQ 服务：
<!--
# Note the rabbitmq-service has a DNS name, provided by Kubernetes:
-->
```
# 请注意 rabbitmq-service 有Kubernetes 提供的 DNS 名称，
root@temp-loe07:/# nslookup rabbitmq-service
Server: 10.0.0.10
Address: 10.0.0.10#53
Name: rabbitmq-service.default.svc.cluster.local
Address: 10.0.147.152
# 你的 IP 地址会不同
```
<!--
If Kube-DNS is not setup correctly, the previous step may not work for you.
You can also find the service IP in an env var:
-->
如果 Kube-DNS 没有正确安装，上一步可能会出错。
你也可以在环境变量中找到服务 IP。
<!--
# Your address will vary.
-->
```
# env | grep RABBIT | grep HOST
RABBITMQ_SERVICE_SERVICE_HOST=10.0.147.152
# 你的 IP 地址会有所不同
```
<!--
Next we will verify we can create a queue, and publish and consume messages.
-->
接着我们将要确认可以创建队列，并能发布消息和消费消息。
<!--
# In the next line, rabbitmq-service is the hostname where the rabbitmq-service
# can be reached. 5672 is the standard port for rabbitmq.
# If you could not resolve "rabbitmq-service" in the previous step,
# then use this command instead:
# root@temp-loe07:/# BROKER_URL=amqp://guest:guest@$RABBITMQ_SERVICE_SERVICE_HOST:5672
# Now create a queue:
# and publish a message to it:
# and get it back.
-->
```shell
# 下一行，rabbitmq-service 是访问 rabbitmq-service 的主机名。5672是 rabbitmq 的标准端口。
root@temp-loe07:/# export BROKER_URL=amqp://guest:guest@rabbitmq-service:5672
# 如果上一步中你不能解析 "rabbitmq-service"，可以用下面的命令替换：
# root@temp-loe07:/# BROKER_URL=amqp://guest:guest@$RABBITMQ_SERVICE_SERVICE_HOST:5672
# 现在创建队列：
root@temp-loe07:/# /usr/bin/amqp-declare-queue --url=$BROKER_URL -q foo -d foo
# 向它推送一条消息:
root@temp-loe07:/# /usr/bin/amqp-publish --url=$BROKER_URL -r foo -p -b Hello
# 然后取回它.
root@temp-loe07:/# /usr/bin/amqp-consume --url=$BROKER_URL -q foo -c 1 cat && echo
Hello
root@temp-loe07:/#
```
<!--
In the last command, the `amqp-consume` tool takes one message (`-c 1`)
from the queue, and passes that message to the standard input of an arbitrary command. In this case, the program `cat` prints out the characters read from standard input, and the echo adds a carriage
return so the example is readable.
-->
最后一个命令中， `amqp-consume` 工具从队列中取走了一个消息，并把该消息传递给了随机命令的标准输出。
在这种情况下，`cat` 会打印它从标准输入中读取的字符，echo 会添加回车符以便示例可读。
<!--
## Filling the Queue with tasks
Now let's fill the queue with some "tasks". In our example, our tasks are strings to be
printed.
In a practice, the content of the messages might be:
- names of files to that need to be processed
- extra flags to the program
- ranges of keys in a database table
- configuration parameters to a simulation
- frame numbers of a scene to be rendered
-->
## 为队列增加任务
现在让我们给队列增加一些任务。在我们的示例中，任务是多个待打印的字符串。
实践中，消息的内容可以是：
- 待处理的文件名
- 程序额外的参数
- 数据库表的关键字范围
- 模拟任务的配置参数
- 待渲染的场景的帧序列号
<!--
In practice, if there is large data that is needed in a read-only mode by all pods
of the Job, you will typically put that in a shared file system like NFS and mount
that readonly on all the pods, or the program in the pod will natively read data from
a cluster file system like HDFS.
For our example, we will create the queue and fill it using the amqp command line tools.
In practice, you might write a program to fill the queue using an amqp client library.
-->
本例中，如果有大量的数据需要被 Job 的所有 Pod 读取，典型的做法是把它们放在一个共享文件系统中，如NFS，并以只读的方式挂载到所有 Pod，或者 Pod 中的程序从类似 HDFS 的集群文件系统中读取。
例如，我们创建队列并使用 amqp 命令行工具向队列中填充消息。实践中，你可以写个程序来利用 amqp 客户端库来填充这些队列。
```shell
/usr/bin/amqp-declare-queue --url=$BROKER_URL -q job1 -d job1
for f in apple banana cherry date fig grape lemon melon
do
/usr/bin/amqp-publish --url=$BROKER_URL -r job1 -p -b $f
done
```
<!--
So, we filled the queue with 8 messages.
## Create an Image
Now we are ready to create an image that we will run as a job.
We will use the `amqp-consume` utility to read the message
from the queue and run our actual program. Here is a very simple
example program:
-->
这样，我们给队列中填充了8个消息。
## 创建镜像
现在我们可以创建一个做为 Job 来运行的镜像。
我们将用 `amqp-consume` 来从队列中读取消息并实际运行我们的程序。这里给出一个非常简单的示例程序：
{{< codenew language="python" file="application/job/rabbitmq/worker.py" >}}
<!--
Now, build an image. If you are working in the source
tree, then change directory to `examples/job/work-queue-1`.
Otherwise, make a temporary directory, change to it,
download the [Dockerfile](/examples/application/job/rabbitmq/Dockerfile),
and [worker.py](/examples/application/job/rabbitmq/worker.py). In either case,
build the image with this command:
-->
现在，编译镜像。如果你在用源代码树，那么切换到目录 `examples/job/work-queue-1`。
否则的话，创建一个临时目录，切换到这个目录。下载
[Dockerfile](/examples/application/job/rabbitmq/Dockerfile)，和
[worker.py](/examples/application/job/rabbitmq/worker.py)。
无论哪种情况，都可以用下面的命令编译镜像
```shell
docker build -t job-wq-1 .
```
<!--
For the [Docker Hub](https://hub.docker.com/), tag your app image with
your username and push to the Hub with the below commands. Replace
`<username>` with your Hub username.
-->
对于 [Docker Hub](https://hub.docker.com/), 给你的应用镜像打上标签，
标签为你的用户名，然后用下面的命令推送到 Hub。用你的 Hub 用户名替换 `<username>`。
```shell
docker tag job-wq-1 <username>/job-wq-1
docker push <username>/job-wq-1
```
<!--
If you are using [Google Container
Registry](https://cloud.google.com/tools/container-registry/), tag
your app image with your project ID, and push to GCR. Replace
`<project>` with your project ID.
-->
如果你在用[谷歌容器仓库](https://cloud.google.com/tools/container-registry/)，
用你的项目 ID 作为标签打到你的应用镜像上，然后推送到 GCR。
用你的项目 ID 替换 `<project>`。
```shell
docker tag job-wq-1 gcr.io/<project>/job-wq-1
gcloud docker -- push gcr.io/<project>/job-wq-1
```
<!--
## Defining a Job
Here is a job definition. You'll need to make a copy of the Job and edit the
image to match the name you used, and call it `./job.yaml`.
-->
## 定义 Job
这里给出一个 Job 定义 yaml文件。你需要拷贝一份并编辑镜像以匹配你使用的名称，保存为 `./job.yaml`。
{{< codenew file="application/job/rabbitmq/job.yaml" >}}
<!--
In this example, each pod works on one item from the queue and then exits.
So, the completion count of the Job corresponds to the number of work items
done. So we set, `.spec.completions: 8` for the example, since we put 8 items in the queue.
## Running the Job
So, now run the Job:
-->
本例中，每个 Pod 使用队列中的一个消息然后退出。这样，Job 的完成计数就代表了完成的工作项的数量。本例中我们设置 `.spec.completions: 8`，因为我们放了8项内容在队列中。
## 运行 Job
现在我们运行 Job：
```shell
kubectl create -f ./job.yaml
```
<!--
Now wait a bit, then check on the job.
-->
稍等片刻，然后检查 Job。
```shell
kubectl describe jobs/job-wq-1
```
```
Name: job-wq-1
Namespace: default
Selector: controller-uid=41d75705-92df-11e7-b85e-fa163ee3c11f
Labels: controller-uid=41d75705-92df-11e7-b85e-fa163ee3c11f
job-name=job-wq-1
Annotations: <none>
Parallelism: 2
Completions: 8
Start Time: Wed, 06 Sep 2017 16:42:02 +0800
Pods Statuses: 0 Running / 8 Succeeded / 0 Failed
Pod Template:
Labels: controller-uid=41d75705-92df-11e7-b85e-fa163ee3c11f
job-name=job-wq-1
Containers:
c:
Image: gcr.io/causal-jigsaw-637/job-wq-1
Port:
Environment:
BROKER_URL: amqp://guest:guest@rabbitmq-service:5672
QUEUE: job1
Mounts: <none>
Volumes: <none>
Events:
FirstSeen LastSeen Count From SubobjectPath Type Reason Message
───────── ──────── ───── ──── ───────────── ────── ────── ───────
27s 27s 1 {job } Normal SuccessfulCreate Created pod: job-wq-1-hcobb
27s 27s 1 {job } Normal SuccessfulCreate Created pod: job-wq-1-weytj
27s 27s 1 {job } Normal SuccessfulCreate Created pod: job-wq-1-qaam5
27s 27s 1 {job } Normal SuccessfulCreate Created pod: job-wq-1-b67sr
26s 26s 1 {job } Normal SuccessfulCreate Created pod: job-wq-1-xe5hj
15s 15s 1 {job } Normal SuccessfulCreate Created pod: job-wq-1-w2zqe
14s 14s 1 {job } Normal SuccessfulCreate Created pod: job-wq-1-d6ppa
14s 14s 1 {job } Normal SuccessfulCreate Created pod: job-wq-1-p17e0
```
<!--
All our pods succeeded. Yay.
-->
我们所有的 Pod 都成功了。耶！
<!-- discussion -->
<!--
## Alternatives
This approach has the advantage that you
do not need to modify your "worker" program to be aware that there is a work queue.
It does require that you run a message queue service.
If running a queue service is inconvenient, you may
want to consider one of the other [job patterns](/docs/concepts/jobs/run-to-completion-finite-workloads/#job-patterns).
-->
## 替代方案
本文所讲述的处理方法的好处是你不需要修改你的 "worker" 程序使其知道工作队列的存在。
本文所描述的方法需要你运行一个消息队列服务。如果不方便运行消息队列服务，你也许会考虑另外一种
[任务模式](/zh/docs/concepts/workloads/controllers/job/#job-patterns)。
<!--
This approach creates a pod for every work item. If your work items only take a few seconds,
though, creating a Pod for every work item may add a lot of overhead. Consider another
[example](/docs/tasks/job/fine-parallel-processing-work-queue/), that executes multiple work items per Pod.
In this example, we used use the `amqp-consume` utility to read the message
from the queue and run our actual program. This has the advantage that you
do not need to modify your program to be aware of the queue.
A [different example](/docs/tasks/job/fine-parallel-processing-work-queue/), shows how to
communicate with the work queue using a client library.
-->
本文所述的方法为每个工作项创建了一个 Pod。
如果你的工作项仅需数秒钟，为每个工作项创建 Pod会增加很多的常规消耗。
可以考虑另外的方案请参考[示例](/zh/docs/tasks/job/fine-parallel-processing-work-queue/)，
这种方案可以实现每个 Pod 执行多个工作项。
示例中，我们使用 `amqp-consume` 从消息队列读取消息并执行我们真正的程序。
这样的好处是你不需要修改你的程序使其知道队列的存在。
要了解怎样使用客户端库和工作队列通信，请参考
[不同的示例](/zh/docs/tasks/job/fine-parallel-processing-work-queue/)。
<!--
## Caveats
If the number of completions is set to less than the number of items in the queue, then
not all items will be processed.
If the number of completions is set to more than the number of items in the queue,
then the Job will not appear to be completed, even though all items in the queue
have been processed. It will start additional pods which will block waiting
for a message.
There is an unlikely race with this pattern. If the container is killed in between the time
that the message is acknowledged by the amqp-consume command and the time that the container
exits with success, or if the node crashes before the kubelet is able to post the success of the pod
back to the api-server, then the Job will not appear to be complete, even though all items
in the queue have been processed.
-->
## 友情提醒
如果设置的完成数量小于队列中的消息数量，会导致一部分消息项不会被执行。
如果设置的完成数量大于队列中的消息数量，当队列中所有的消息都处理完成后，
Job 也会显示为未完成。Job 将创建 Pod 并阻塞等待消息输入。
当发生下面两种情况时，即使队列中所有的消息都处理完了，Job 也不会显示为完成状态：
* 在 amqp-consume 命令拿到消息和容器成功退出之间的时间段内，执行杀死容器操作；
* 在 kubelet 向 api-server 传回 Pod 成功运行之前，发生节点崩溃。
content/zh-cn/docs/tasks/job/fine-parallel-processing-work-queue.md
+369
View File
@@ -0,0 +1,369 @@
---
title: 使用工作队列进行精细的并行处理
content_type: task
min-kubernetes-server-version: v1.8
weight: 30
---
<!--
title: Fine Parallel Processing Using a Work Queue
content_type: task
weight: 30
min-kubernetes-server-version: v1.8
-->
<!-- overview -->
<!--
In this example, we will run a Kubernetes Job with multiple parallel
worker processes in a given pod.
-->
在这个例子中，我们会运行一个Kubernetes Job，其中的 Pod 会运行多个并行工作进程。
<!--
In this example, as each pod is created, it picks up one unit of work
from a task queue, processes it, and repeats until the end of the queue is reached.
Here is an overview of the steps in this example:
-->
在这个例子中，当每个pod被创建时，它会从一个任务队列中获取一个工作单元，处理它，然后重复，直到到达队列的尾部。
下面是这个示例的步骤概述：
<!--
1. **Start a storage service to hold the work queue.** In this example, we use Redis to store
our work items. In the previous example, we used RabbitMQ. In this example, we use Redis and
a custom work-queue client library because AMQP does not provide a good way for clients to
detect when a finite-length work queue is empty. In practice you would set up a store such
as Redis once and reuse it for the work queues of many jobs, and other things.
-->
1. **启动存储服务用于保存工作队列。** 在这个例子中，我们使用 Redis 来存储工作项。
在上一个例子中，我们使用了 RabbitMQ。
在这个例子中，由于 AMQP 不能为客户端提供一个良好的方法来检测一个有限长度的工作队列是否为空，
我们使用了 Redis 和一个自定义的工作队列客户端库。
在实践中，你可能会设置一个类似于 Redis 的存储库，并将其同时用于多项任务或其他事务的工作队列。
<!--
1. **Create a queue, and fill it with messages.** Each message represents one task to be done. In
this example, a message is an integer that we will do a lengthy computation on.
-->
2. **创建一个队列，然后向其中填充消息。** 每个消息表示一个将要被处理的工作任务。
在这个例子中，消息是一个我们将用于进行长度计算的整数。
<!--
1. **Start a Job that works on tasks from the queue**. The Job starts several pods. Each pod takes
one task from the message queue, processes it, and repeats until the end of the queue is reached.
-->
3. **启动一个 Job 对队列中的任务进行处理**。这个 Job 启动了若干个 Pod 。
每个 Pod 从消息队列中取出一个工作任务，处理它，然后重复，直到到达队列的尾部。
## {{% heading "prerequisites" %}}
{{< include "task-tutorial-prereqs.md" >}} {{< version-check >}}
<!--
Be familiar with the basic,
non-parallel, use of [Job](/docs/concepts/workloads/controllers/job/).
-->
熟悉基本的、非并行的 [Job](/zh/docs/concepts/workloads/controllers/job/)。
<!-- steps -->
<!--
## Starting Redis
For this example, for simplicity, we will start a single instance of Redis.
See the [Redis Example](https://github.com/kubernetes/examples/tree/master/guestbook) for an example
of deploying Redis scalably and redundantly.
-->
## 启动 Redis
对于这个例子，为了简单起见，我们将启动一个单实例的 Redis。
了解如何部署一个可伸缩、高可用的 Redis 例子，请查看
[Redis 示例](https://github.com/kubernetes/examples/tree/master/guestbook)
<!--
You could also download the following files directly:
-->
你也可以直接下载如下文件：
- [`redis-pod.yaml`](/examples/application/job/redis/redis-pod.yaml)
- [`redis-service.yaml`](/examples/application/job/redis/redis-service.yaml)
- [`Dockerfile`](/examples/application/job/redis/Dockerfile)
- [`job.yaml`](/examples/application/job/redis/job.yaml)
- [`rediswq.py`](/examples/application/job/redis/rediswq.py)
- [`worker.py`](/examples/application/job/redis/worker.py)
<!--
## Filling the Queue with tasks
Now let's fill the queue with some "tasks". In our example, our tasks are strings to be
printed.
Start a temporary interactive pod for running the Redis CLI.
-->
## 使用任务填充队列
现在，让我们往队列里添加一些“任务”。在这个例子中，我们的任务是一些将被打印出来的字符串。
启动一个临时的可交互的 pod 用于运行 Redis 命令行界面。
```shell
kubectl run -i --tty temp --image redis --command "/bin/sh"
```
```
Waiting for pod default/redis2-c7h78 to be running, status is Pending, pod ready: false
Hit enter for command prompt
```
<!--
Now hit enter, start the redis CLI, and create a list with some work items in it.
-->
现在按回车键，启动 redis 命令行界面，然后创建一个存在若干个工作项的列表。
```shell
# redis-cli -h redis
redis:6379> rpush job2 "apple"
(integer) 1
redis:6379> rpush job2 "banana"
(integer) 2
redis:6379> rpush job2 "cherry"
(integer) 3
redis:6379> rpush job2 "date"
(integer) 4
redis:6379> rpush job2 "fig"
(integer) 5
redis:6379> rpush job2 "grape"
(integer) 6
redis:6379> rpush job2 "lemon"
(integer) 7
redis:6379> rpush job2 "melon"
(integer) 8
redis:6379> rpush job2 "orange"
(integer) 9
redis:6379> lrange job2 0 -1
1) "apple"
2) "banana"
3) "cherry"
4) "date"
5) "fig"
6) "grape"
7) "lemon"
8) "melon"
9) "orange"
```
<!--
So, the list with key `job2` will be our work queue.
-->
因此，这个键为 `job2` 的列表就是我们的工作队列。
<!--
Note: if you do not have Kube DNS setup correctly, you may need to change
the first step of the above block to `redis-cli -h $REDIS_SERVICE_HOST`.
-->
注意：如果你还没有正确地配置 Kube DNS，你可能需要将上面的第一步改为
`redis-cli -h $REDIS_SERVICE_HOST`。
<!--
## Create an Image
Now we are ready to create an image that we will run.
We will use a python worker program with a redis client to read
the messages from the message queue.
A simple Redis work queue client library is provided,
called rediswq.py ([Download](/examples/application/job/redis/rediswq.py)).
-->
## 创建镜像
现在我们已经准备好创建一个我们要运行的镜像
我们会使用一个带有 redis 客户端的 python 工作程序从消息队列中读出消息。
这里提供了一个简单的 Redis 工作队列客户端库，叫 rediswq.py ([下载](/examples/application/job/redis/rediswq.py))。
<!--
The "worker" program in each Pod of the Job uses the work queue
client library to get work. Here it is:
-->
Job 中每个 Pod 内的 “工作程序” 使用工作队列客户端库获取工作。如下：
{{< codenew language="python" file="application/job/redis/worker.py" >}}
<!--
You could download [`worker.py`](/examples/application/job/redis/worker.py), [`rediswq.py`](/examples/application/job/redis/rediswq.py), and [`Dockerfile`](/examples/application/job/redis/Dockerfile)
using above links. Then build the image:
-->
你也可以下载 [`worker.py`](/examples/application/job/redis/worker.py)、
[`rediswq.py`](/examples/application/job/redis/rediswq.py) 和
[`Dockerfile`](/examples/application/job/redis/Dockerfile)。然后构建镜像：
```shell
docker build -t job-wq-2 .
```
<!--
### Push the image
For the [Docker Hub](https://hub.docker.com/), tag your app image with
your username and push to the Hub with the below commands. Replace
`<username>` with your Hub username.
-->
### Push 镜像
对于 [Docker Hub](https://hub.docker.com/)，请先用你的用户名给镜像打上标签，
然后使用下面的命令 push 你的镜像到仓库。请将 `<username>` 替换为你自己的 Hub 用户名。
```shell
docker tag job-wq-2 <username>/job-wq-2
docker push <username>/job-wq-2
```
<!--
You need to push to a public repository or [configure your cluster to be able to access
your private repository](/docs/concepts/containers/images/).
-->
你需要将镜像 push 到一个公共仓库或者
[配置集群访问你的私有仓库](/zh/docs/concepts/containers/images/)。
<!--
If you are using [Google Container
Registry](https://cloud.google.com/tools/container-registry/), tag
your app image with your project ID, and push to GCR. Replace
`<project>` with your project ID.
-->
如果你使用的是 [Google Container Registry](https://cloud.google.com/tools/container-registry/)，
请先用你的 project ID 给你的镜像打上标签，然后 push 到 GCR 。请将 `<project>` 替换为你自己的 project ID
```shell
docker tag job-wq-2 gcr.io/<project>/job-wq-2
gcloud docker -- push gcr.io/<project>/job-wq-2
```
<!--
## Defining a Job
Here is the job definition:
-->
## 定义一个 Job
这是 job 定义：
{{< codenew file="application/job/redis/job.yaml" >}}
<!--
Be sure to edit the job template to
change `gcr.io/myproject` to your own path.
-->
请确保将 job 模板中的 `gcr.io/myproject` 更改为你自己的路径。
<!--
In this example, each pod works on several items from the queue and then exits when there are no more items.
Since the workers themselves detect when the workqueue is empty, and the Job controller does not
know about the workqueue, it relies on the workers to signal when they are done working.
The workers signal that the queue is empty by exiting with success. So, as soon as any worker
exits with success, the controller knows the work is done, and the Pods will exit soon.
So, we set the completion count of the Job to 1. The job controller will wait for the other pods to complete
too.
-->
在这个例子中，每个 pod 处理了队列中的多个项目，直到队列中没有项目时便退出。
因为是由工作程序自行检测工作队列是否为空，并且 Job 控制器不知道工作队列的存在，
这依赖于工作程序在完成工作时发出信号。
工作程序以成功退出的形式发出信号表示工作队列已经为空。
所以，只要有任意一个工作程序成功退出，控制器就知道工作已经完成了，所有的 Pod 将很快会退出。
因此，我们将 Job 的完成计数（Completion Count）设置为 1 。
尽管如此，Job 控制器还是会等待其它 Pod 完成。
<!--
## Running the Job
So, now run the Job:
-->
## 运行 Job
现在运行这个 Job ：
```shell
kubectl apply -f ./job.yaml
```
<!--
Now wait a bit, then check on the job.
-->
稍等片刻，然后检查这个 Job。
```shell
kubectl describe jobs/job-wq-2
```
```
Name: job-wq-2
Namespace: default
Selector: controller-uid=b1c7e4e3-92e1-11e7-b85e-fa163ee3c11f
Labels: controller-uid=b1c7e4e3-92e1-11e7-b85e-fa163ee3c11f
job-name=job-wq-2
Annotations: <none>
Parallelism: 2
Completions: <unset>
Start Time: Mon, 11 Jan 2016 17:07:59 -0800
Pods Statuses: 1 Running / 0 Succeeded / 0 Failed
Pod Template:
Labels: controller-uid=b1c7e4e3-92e1-11e7-b85e-fa163ee3c11f
job-name=job-wq-2
Containers:
c:
Image: gcr.io/exampleproject/job-wq-2
Port:
Environment: <none>
Mounts: <none>
Volumes: <none>
Events:
FirstSeen LastSeen Count From SubobjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
33s 33s 1 {job-controller } Normal SuccessfulCreate Created pod: job-wq-2-lglf8
```
查看日志：
```shell
kubectl logs pods/job-wq-2-7r7b2
```
```
Worker with sessionID: bbd72d0a-9e5c-4dd6-abf6-416cc267991f
Initial queue state: empty=False
Working on banana
Working on date
Working on lemon
```
<!--
As you can see, one of our pods worked on several work units.
-->
你可以看到，其中的一个 pod 处理了若干个工作单元。
<!-- discussion -->
<!--
## Alternatives
-->
## 替代方案
<!--
If running a queue service or modifying your containers to use a work queue is inconvenient, you may
want to consider one of the other [job patterns](/docs/concepts/jobs/run-to-completion-finite-workloads/#job-patterns).
-->
如果你不方便运行一个队列服务或者修改你的容器用于运行一个工作队列，你可以考虑其它的
[Job 模式](/zh/docs/concepts/workloads/controllers/job/#job-patterns)。
<!--
If you have a continuous stream of background processing work to run, then
consider running your background workers with a `ReplicaSet` instead,
and consider running a background processing library such as
[https://github.com/resque/resque](https://github.com/resque/resque).
-->
如果你有持续的后台处理业务，那么可以考虑使用 `ReplicaSet` 来运行你的后台业务，
和运行一个类似 [https://github.com/resque/resque](https://github.com/resque/resque)
的后台处理库。
content/zh-cn/docs/tasks/job/indexed-parallel-processing-static.md
+277
View File
@@ -0,0 +1,277 @@
---
title: 使用索引作业完成静态工作分配下的并行处理
content_type: task
min-kubernetes-server-version: v1.21
weight: 30
---
<!--
title: Indexed Job for Parallel Processing with Static Work Assignment
content_type: task
min-kubernetes-server-version: v1.21
weight: 30
-->
{{< feature-state for_k8s_version="v1.24" state="stable" >}}
<!-- overview -->
<!--
In this example, you will run a Kubernetes Job that uses multiple parallel
worker processes.
Each worker is a different container running in its own Pod. The Pods have an
_index number_ that the control plane sets automatically, which allows each Pod
to identify which part of the overall task to work on.
-->
在此示例中，你将运行一个使用多个并行工作进程的 Kubernetes Job。
每个 worker 都是在自己的 Pod 中运行的不同容器。
Pod 具有控制平面自动设置的 _索引编号（index number）_，
这些编号使得每个 Pod 能识别出要处理整个任务的哪个部分。
<!--
The pod index is available in the {{< glossary_tooltip text="annotation" term_id="annotation" >}}
`batch.kubernetes.io/job-completion-index` as a string representing its
decimal value. In order for the containerized task process to obtain this index,
you can publish the value of the annotation using the [downward API](/docs/tasks/inject-data-application/downward-api-volume-expose-pod-information/#the-downward-api)
mechanism.
For convenience, the control plane automatically sets the downward API to
expose the index in the `JOB_COMPLETION_INDEX` environment variable.
-->
Pod 索引在{{<glossary_tooltip text="注解" term_id="annotation" >}}
`batch.kubernetes.io/job-completion-index` 中呈现，具体表示为一个十进制值字符串。
为了让容器化的任务进程获得此索引，你可以使用
[downward API](/zh/docs/tasks/inject-data-application/downward-api-volume-expose-pod-information/#the-downward-api)
机制发布注解的值。为方便起见，
控制平面自动设置 downward API 以在 `JOB_COMPLETION_INDEX` 环境变量中公开索引。
<!--
Here is an overview of the steps in this example:
1. **Define a Job manifest using indexed completion**.
The downward API allows you to pass the pod index annotation as an
environment variable or file to the container.
2. **Start an `Indexed` Job based on that manifest**.
-->
以下是此示例中步骤的概述：
1. **定义使用带索引完成信息的 Job 清单**。
Downward API 使你可以将 Pod 索引注释作为环境变量或文件传递给容器。
2. **根据该清单启动一个带索引（`Indexed`）的 Job**。
## {{% heading "prerequisites" %}}
<!--
You should already be familiar with the basic,
non-parallel, use of [Job](/docs/concepts/workloads/controllers/job/).
-->
你应该已经熟悉 [Job](/zh/docs/concepts/workloads/controllers/job/) 的基本的、非并行的用法。
{{< include "task-tutorial-prereqs.md" >}} {{< version-check >}}
<!-- steps -->
<!-- ## Choose an approach -->
## 选择一种方法
<!--
To access the work item from the worker program, you have a few options:
1. Read the `JOB_COMPLETION_INDEX` environment variable. The Job
{{< glossary_tooltip text="controller" term_id="controller" >}}
automatically links this variable to the annotation containing the completion
index.
1. Read a file that contains the completion index.
1. Assuming that you can't modify the program, you can wrap it with a script
that reads the index using any of the methods above and converts it into
something that the program can use as input.
-->
要从工作程序访问工作项，你有几个选择：
1. 读取 `JOB_COMPLETION_INDEX` 环境变量。Job
{{< glossary_tooltip text="控制器" term_id="controller" >}}
自动将此变量链接到包含完成索引的注解。
1. 读取包含完整索引的文件。
1. 假设你无法修改程序，你可以使用脚本包装它，
该脚本使用上述任意方法读取索引并将其转换为程序可以用作输入的内容。
<!--
For this example, imagine that you chose option 3 and you want to run the
[rev](https://man7.org/linux/man-pages/man1/rev.1.html) utility. This
program accepts a file as an argument and prints its content reversed.
-->
对于此示例，假设你选择了方法 3 并且想要运行
[rev](https://man7.org/linux/man-pages/man1/rev.1.html) 实用程序。
这个程序接受一个文件作为参数并按逆序打印其内容。
```shell
rev data.txt
```
<!--
You'll use the `rev` tool from the
[`busybox`](https://hub.docker.com/_/busybox) container image.
-->
你将使用 [`busybox`](https://hub.docker.com/_/busybox) 容器映像中的 `rev` 工具。
<!--
As this is only an example, each Pod only does a tiny piece of work (reversing a short
string). In a real workload you might, for example, create a Job that represents
the
task of producing 60 seconds of video based on scene data.
Each work item in the video rendering Job would be to render a particular
frame of that video clip. Indexed completion would mean that each Pod in
the Job knows which frame to render and publish, by counting frames from
the start of the clip.
-->
由于这只是一个例子，每个 Pod 只做一小部分工作（反转一个短字符串）。
例如，在实际工作负载中，你可能会创建一个表示基于场景数据制作 60 秒视频的任务的 Job 。
视频渲染 Job 中的每个工作项都将渲染该视频剪辑的特定帧。
索引完成意味着 Job 中的每个 Pod 都知道通过从剪辑开始计算帧数，来确定渲染和发布哪一帧，。
<!-- ## Define an Indexed Job -->
## 定义索引作业
<!--
Here is a sample Job manifest that uses `Indexed` completion mode:
-->
这是一个使用 `Indexed` 完成模式的示例 Job 清单：
{{< codenew language="yaml" file="application/job/indexed-job.yaml" >}}
<!--
In the example above, you use the builtin `JOB_COMPLETION_INDEX` environment
variable set by the Job controller for all containers. An [init container](/docs/concepts/workloads/pods/init-containers/)
maps the index to a static value and writes it to a file that is shared with the
container running the worker through an [emptyDir volume](/docs/concepts/storage/volumes/#emptydir).
Optionally, you can [define your own environment variable through the downward
API](/docs/tasks/inject-data-application/environment-variable-expose-pod-information/)
to publish the index to containers. You can also choose to load a list of values
from a [ConfigMap as an environment variable or file](/docs/tasks/configure-pod-container/configure-pod-configmap/).
-->
在上面的示例中，你使用 Job 控制器为所有容器设置的内置 `JOB_COMPLETION_INDEX` 环境变量。
[Init 容器](/zh/docs/concepts/workloads/pods/init-containers/)
将索引映射到一个静态值，并将其写入一个文件，该文件通过
[emptyDir 卷](/zh/docs/concepts/storage/volumes/#emptydir)
与运行 worker 的容器共享。或者，你可以
[通过 Downward API 定义自己的环境变量](/zh/docs/tasks/inject-data-application/environment-variable-expose-pod-information/)
将索引发布到容器。你还可以选择从
[包含 ConfigMap 的环境变量或文件](/zh/docs/tasks/configure-pod-container/configure-pod-configmap/)
加载值列表。
<!--
Alternatively, you can directly [use the downward API to pass the annotation
value as a volume file](/docs/tasks/inject-data-application/downward-api-volume-expose-pod-information/#store-pod-fields),
like shown in the following example:
-->
或者也可以直接
[使用 Downward API 将注解值作为卷文件传递](/zh/docs/tasks/inject-data-application/downward-api-volume-expose-pod-information/#store-pod-fields)，
如下例所示：
{{< codenew language="yaml" file="application/job/indexed-job-vol.yaml" >}}
<!-- ## Running the Job -->
## 执行 Job
<!-- Now run the Job: -->
现在执行 Job：
```shell
# 使用第一种方法（依赖于 $JOB_COMPLETION_INDEX）
kubectl apply -f https://kubernetes.io/examples/application/job/indexed-job.yaml
```
<!--
When you create this Job, the control plane creates a series of Pods, one for each index you specified. The value of `.spec.parallelism` determines how many can run at once whereas `.spec.completions` determines how many Pods the Job creates in total.
Because `.spec.parallelism` is less than `.spec.completions`, the control plane waits for some of the first Pods to complete before starting more of them.
Once you have created the Job, wait a moment then check on progress:
-->
当你创建此 Job 时，控制平面会创建一系列 Pod，每个索引都由你指定。
`.spec.parallelism` 的值决定了一次可以运行多少个，
而 `.spec.completions` 决定了 Job 总共创建了多少个 Pod。
因为 `.spec.parallelism` 小于 `.spec.completions`，
控制平面在启动更多 Pod 之前，等待部分第一批 Pod 完成。
创建 Job 后，稍等片刻，然后检查进度：
```shell
kubectl describe jobs/indexed-job
```
<!-- The output is similar to: -->
输出类似于：
```
Name: indexed-job
Namespace: default
Selector: controller-uid=bf865e04-0b67-483b-9a90-74cfc4c3e756
Labels: controller-uid=bf865e04-0b67-483b-9a90-74cfc4c3e756
job-name=indexed-job
Annotations: <none>
Parallelism: 3
Completions: 5
Start Time: Thu, 11 Mar 2021 15:47:34 +0000
Pods Statuses: 2 Running / 3 Succeeded / 0 Failed
Completed Indexes: 0-2
Pod Template:
Labels: controller-uid=bf865e04-0b67-483b-9a90-74cfc4c3e756
job-name=indexed-job
Init Containers:
input:
Image: docker.io/library/bash
Port: <none>
Host Port: <none>
Command:
bash
-c
items=(foo bar baz qux xyz)
echo ${items[$JOB_COMPLETION_INDEX]} > /input/data.txt
Environment: <none>
Mounts:
/input from input (rw)
Containers:
worker:
Image: docker.io/library/busybox
Port: <none>
Host Port: <none>
Command:
rev
/input/data.txt
Environment: <none>
Mounts:
/input from input (rw)
Volumes:
input:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal SuccessfulCreate 4s job-controller Created pod: indexed-job-njkjj
Normal SuccessfulCreate 4s job-controller Created pod: indexed-job-9kd4h
Normal SuccessfulCreate 4s job-controller Created pod: indexed-job-qjwsz
Normal SuccessfulCreate 1s job-controller Created pod: indexed-job-fdhq5
Normal SuccessfulCreate 1s job-controller Created pod: indexed-job-ncslj
```
<!--
In this example, you run the Job with custom values for each index. You can
inspect the output of one of the pods:
-->
在此示例中，你使用每个索引的自定义值运行 Job。
你可以检查其中一个 Pod 的输出：
```shell
kubectl logs indexed-job-fdhq5 # 更改它以匹配来自该 Job 的 Pod 的名称
```
<!-- The output is similar to: -->
输出类似于：
```
xuq
```
content/zh-cn/docs/tasks/job/parallel-processing-expansion.md
+497
View File
@@ -0,0 +1,497 @@
---
title: 使用展开的方式进行并行处理
content_type: task
min-kubernetes-server-version: v1.8
weight: 50
---
<!--
title: Parallel Processing using Expansions
content_type: task
min-kubernetes-server-version: v1.8
weight: 50
-->
<!-- overview -->
<!--
This task demonstrates running multiple {{< glossary_tooltip text="Jobs" term_id="job" >}}
based on a common template. You can use this approach to process batches of work in
parallel.
For this example there are only three items: _apple_, _banana_, and _cherry_.
The sample Jobs process each item by printing a string then pausing.
See [using Jobs in real workloads](#using-jobs-in-real-workloads) to learn about how
this pattern fits more realistic use cases.
-->
本任务展示基于一个公共的模板运行多个{{< glossary_tooltip text="Jobs" term_id="job" >}}。
你可以用这种方法来并行执行批处理任务。
在本任务示例中，只有三个工作条目：_apple_、_banana_ 和 _cherry_。
示例任务处理每个条目时打印一个字符串之后结束。
参考[在真实负载中使用 Job](#using-jobs-in-real-workloads)了解更适用于真实使用场景的模式。
## {{% heading "prerequisites" %}}
<!--
You should be familiar with the basic,
non-parallel, use of [Job](/docs/concepts/workloads/controllers/job/).
-->
你应先熟悉基本的、非并行的 [Job](/zh/docs/concepts/workloads/controllers/job/)
的用法。
{{< include "task-tutorial-prereqs.md" >}}
<!--
For basic templating you need the command-line utility `sed`.
To follow the advanced templating example, you need a working installation of
[Python](https://www.python.org/), and the Jinja2 template
library for Python.
Once you have Python set up, you can install Jinja2 by running:
-->
任务中的基本模板示例要求安装命令行工具 `sed`。
要使用较高级的模板示例，你需要安装 [Python](https://www.python.org/)，
并且要安装 Jinja2 模板库。
一旦 Python 已经安装好，你可以运行下面的命令安装 Jinja2：
```shell
pip install --user jinja2
```
<!-- steps -->
<!--
## Create Jobs based on a template
-->
## 基于模板创建 Job {#create-jobs-based-on-a-template}
<!--
First, download the following template of a job to a file called `job-tmpl.yaml`
-->
首先，将以下作业模板下载到名为 `job-tmpl.yaml` 的文件中。
{{< codenew file="application/job/job-tmpl.yaml" >}}
```shell
# 使用 curl 下载 job-tmpl.yaml
curl -L -s -O https://k8s.io/examples/application/job/job-tmpl.yaml
```
<!--
The file you downloaded is not yet a valid Kubernetes
{{< glossary_tooltip text="manifest" term_id="manifest" >}}.
Instead that template is a YAML representation of a Job object with some placeholders
that need to be filled in before it can be used. The `$ITEM` syntax is not meaningful to Kubernetes.
-->
你所下载的文件不是一个合法的 Kubernetes {{< glossary_tooltip text="清单" term_id="manifest" >}}。
这里的模板只是 Job 对象的 yaml 表示，其中包含一些占位符，在使用它之前需要被填充。
`$ITEM` 语法对 Kubernetes 没有意义。
<!--
### Create manifests from the template
The following shell snippet uses `sed` to replace the string `$ITEM` with the loop
variable, writing into a temporary directory named `jobs`. Run this now:
-->
### 基于模板创建清单
下面的 Shell 代码片段使用 `sed` 将字符串 `$ITEM` 替换为循环变量，并将结果
写入到一个名为 `jobs` 的临时目录。
```shell
# 展开模板文件到多个文件中，每个文件对应一个要处理的条目
mkdir ./jobs
for i in apple banana cherry
do
cat job-tmpl.yaml | sed "s/\$ITEM/$i/" > ./jobs/job-$i.yaml
done
```
<!--
Check if it worked:
-->
检查上述脚本的输出：
```shell
ls jobs/
```
<!--
The output is similar to this:
-->
输出类似于：
```
job-apple.yaml
job-banana.yaml
job-cherry.yaml
```
<!--
You could use any type of template language (for example: Jinja2; ERB), or
write a program to generate the Job manifests.
-->
你可以使用任何一种模板语言（例如：Jinja2、ERB），或者编写一个程序来
生成 Job 清单。
<!--
### Create Jobs from the manifests
Next, create all the Jobs with one kubectl command:
-->
### 基于清单创建 Job
接下来用一个 kubectl 命令创建所有的 Job：
```shell
kubectl create -f ./jobs
```
<!--
The output is similar to this:
-->
输出类似于：
```
job.batch/process-item-apple created
job.batch/process-item-banana created
job.batch/process-item-cherry created
```
<!--
Now, check on the jobs:
-->
现在检查 Job：
```shell
kubectl get jobs -l jobgroup=jobexample
```
<!--
The output is similar to this:
-->
输出类似于：
```
NAME COMPLETIONS DURATION AGE
process-item-apple 1/1 14s 22s
process-item-banana 1/1 12s 21s
process-item-cherry 1/1 12s 20s
```
<!--
Using the `-l` option to kubectl selects only the Jobs that are part
of this group of jobs (there might be other unrelated jobs in the system).
You can check on the Pods as well using the same
{{< glossary_tooltip text="label selector" term_id="selector" >}}:
-->
使用 kubectl 的 `-l` 选项可以仅选择属于当前 Job 组的对象
（系统中可能存在其他不相关的 Job）。
你可以使用相同的 {{< glossary_tooltip text="标签选择算符" term_id="selector" >}}
来过滤 Pods：
```shell
kubectl get pods -l jobgroup=jobexample
```
<!--
The output is similar to:
-->
输出类似于：
```
NAME READY STATUS RESTARTS AGE
process-item-apple-kixwv 0/1 Completed 0 4m
process-item-banana-wrsf7 0/1 Completed 0 4m
process-item-cherry-dnfu9 0/1 Completed 0 4m
```
<!--
We can use this single command to check on the output of all jobs at once:
-->
我们可以用下面的命令查看所有 Job 的输出：
```shell
kubectl logs -f -l jobgroup=jobexample
```
<!--
The output should be:
-->
输出类似于：
```
Processing item apple
Processing item banana
Processing item cherry
```
<!--
### Clean up {#cleanup-1}
-->
### 清理 {#cleanup-1}
```shell
# 删除所创建的 Job
# 集群会自动清理 Job 对应的 Pod
kubectl delete job -l jobgroup=jobexample
```
<!--
## Use advanced template parameters
In the [first example](#create-jobs-based-on-a-template), each instance of the template had one
parameter, and that parameter was also used in the Job's name. However,
[names](/docs/concepts/overview/working-with-objects/names/#names) are restricted
to contain only certain characters.
-->
## 使用高级模板参数
在[第一个例子](#create-jobs-based-on-a-template)中，模板的每个示例都有一个参数
而该参数也用在 Job 名称中。不过，对象
[名称](/zh/docs/concepts/overview/working-with-objects/names/#names)
被限制只能使用某些字符。
<!--
This slightly more complex example uses the
[Jinja template language](https://palletsprojects.com/p/jinja/) to generate manifests
and then objects from those manifests, with a multiple parameters for each Job.
For this part of the task, you are going to use a one-line Python script to
convert the template to a set of manifests.
First, copy and paste the following template of a Job object, into a file called `job.yaml.jinja2`:
-->
这里的略微复杂的例子使用 [Jinja 模板语言](https://palletsprojects.com/p/jinja/)
来生成清单，并基于清单来生成对象，每个 Job 都有多个参数。
在本任务中，你将会使用一个一行的 Python 脚本，将模板转换为一组清单文件。
首先，复制下面的 Job 对象模板到一个名为 `job.yaml.jinja2` 的文件。
```liquid
{% set params = [{ "name": "apple", "url": "http://dbpedia.org/resource/Apple", },
{ "name": "banana", "url": "http://dbpedia.org/resource/Banana", },
{ "name": "cherry", "url": "http://dbpedia.org/resource/Cherry" }]
%}
{% for p in params %}
{% set name = p["name"] %}
{% set url = p["url"] %}
---
apiVersion: batch/v1
kind: Job
metadata:
name: jobexample-{{ name }}
labels:
jobgroup: jobexample
spec:
template:
metadata:
name: jobexample
labels:
jobgroup: jobexample
spec:
containers:
- name: c
image: busybox:1.28
command: ["sh", "-c", "echo Processing URL {{ url }} && sleep 5"]
restartPolicy: Never
{% endfor %}
```
<!--
The above template defines two parameters for each Job object using a list of
python dicts (lines 1-4). A `for` loop emits one Job manifest for each
set of parameters (remaining lines).
This example relies on a feature of YAML. One YAML file can contain multiple
documents (Kubernetes manifests, in this case), separated by `---` on a line
by itself.
You can pipe the output directly to `kubectl` to create the Jobs.
Next, use this one-line Python program to expand the template:
-->
上面的模板使用 python 字典列表（第 1-4 行）定义每个作业对象的参数。
然后使用 for 循环为每组参数（剩余行）生成一个作业 yaml 对象。
我们利用了多个 YAML 文档（这里的 Kubernetes 清单）可以用 `---` 分隔符连接的事实。
我们可以将输出直接传递给 kubectl 来创建对象。
接下来我们用单行的 Python 程序将模板展开。
```shell
alias render_template='python -c "from jinja2 import Template; import sys; print(Template(sys.stdin.read()).render());"'
```
<!--
Use `render_template` to convert the parameters and template into a single
YAML file containing Kubernetes manifests:
-->
使用 `render_template` 将参数和模板转换成一个 YAML 文件，其中包含 Kubernetes
资源清单：
<!--
```shell
# This requires the alias you defined earlier
cat job.yaml.jinja2 | render_template > jobs.yaml
```
-->
```shell
# 此命令需要之前定义的别名
cat job.yaml.jinja2 | render_template > jobs.yaml
```
<!--
You can view `jobs.yaml` to verify that the `render_template` script worked
correctly.
Once you are happy that `render_template` is working how you intend,
you can pipe its output into `kubectl`:
-->
你可以查看 `jobs.yaml` 以验证 `render_template` 脚本是否正常工作。
当你对输出结果比较满意时，可以用管道将其输出发送给 kubectl，如下所示：
```shell
cat job.yaml.jinja2 | render_template | kubectl apply -f -
```
<!--
Kubernetes accepts and runs the Jobs you created.
-->
Kubernetes 接收清单文件并执行你所创建的 Job。
<!--
### Clean up {#cleanup-2}
```shell
# Remove the Jobs you created
# Your cluster automatically cleans up their Pods
kubectl delete job -l jobgroup=jobexample
```
-->
### 清理 {#cleanup-2}
```shell
# 删除所创建的 Job
# 集群会自动清理 Job 对应的 Pod
kubectl delete job -l jobgroup=jobexample
```
<!-- discussion -->
<!--
## Using Jobs in real workloads
In a real use case, each Job performs some substantial computation, such as rendering a frame
of a movie, or processing a range of rows in a database. If you were rendering a movie
you would set `$ITEM` to the frame number. If you were processing rows from a database
table, you would set `$ITEM` to represent the range of database rows to process.
In the task, you ran a command to collect the output from Pods by fetching
their logs. In a real use case, each Pod for a Job writes its output to
durable storage before completing. You can use a PersistentVolume for each Job,
or an external storage service. For example, if you are rendering frames for a movie,
use HTTP to `PUT` the rendered frame data to a URL, using a different URL for each
frame.
-->
## 在真实负载中使用 Job {#using-jobs-in-real-workloads}
在真实的负载中，每个 Job 都会执行一些重要的计算，例如渲染电影的一帧，
或者处理数据库中的若干行。这时，`$ITEM` 参数将指定帧号或行范围。
在此任务中，你运行一个命令通过取回 Pod 的日志来收集其输出。
在真实应用场景中，Job 的每个 Pod 都会在结束之前将其输出写入到某持久性存储中。
你可以为每个 Job 指定 PersistentVolume 卷，或者使用其他外部存储服务。
例如，如果你在渲染视频帧，你可能会使用 HTTP 协议将渲染完的帧数据
用 'PUT' 请求发送到某 URL，每个帧使用不同的 URl。
<!--
## Labels on Jobs and Pods
After you create a Job, Kubernetes automatically adds additional
{{< glossary_tooltip text="labels" term_id="label" >}} that
distinguish one Job's pods from another Job's pods.
In this example, each Job and its Pod template have a label:
`jobgroup=jobexample`.
Kubernetes itself pays no attention to labels named `jobgroup`. Setting a label
for all the Jobs you create from a template makes it convenient to operate on all
those Jobs at once.
In the [first example](#create-jobs-based-on-a-template) you used a template to
create several Jobs. The template ensures that each Pod also gets the same label, so
you can check on all Pods for these templated Jobs with a single command.
-->
## Job 和 Pod 上的标签
你创建了 Job 之后，Kubernetes 自动为 Job 的 Pod 添加
{{< glossary_tooltip text="标签" term_id="label" >}}，以便能够将一个 Job
的 Pod 与另一个 Job 的 Pod 区分开来。
在本例中，每个 Job 及其 Pod 模板有一个标签: `jobgroup=jobexample`。
Kubernetes 自身对标签名 `jobgroup` 没有什么要求。
为创建自同一模板的所有 Job 使用同一标签使得我们可以方便地同时操作组中的所有作业。
在[第一个例子](#create-jobs-based-on-a-template)中，你使用模板来创建了若干 Job。
模板确保每个 Pod 都能够获得相同的标签，这样你可以用一条命令检查这些模板化
Job 所生成的全部 Pod。
<!--
The label key `jobgroup` is not special or reserved.
You can pick your own labelling scheme.
There are [recommended labels](/docs/concepts/overview/working-with-objects/common-labels/#labels)
that you can use if you wish.
-->
{{< note >}}
标签键 `jobgroup` 没什么特殊的，也不是保留字。 你可以选择你自己的标签方案。
如果愿意，有一些[建议的标签](/zh/docs/concepts/overview/working-with-objects/common-labels/#labels)
可供使用。
{{< /note >}}
<!--
## Alternatives
If you plan to create a large number of Job objects, you may find that:
-->
## 替代方案
如果你有计划创建大量 Job 对象，你可能会发现：
<!--
- Even using labels, managing so many Job objects is cumbersome.
- If you create many Jobs in a batch, you might place high load
on the Kubernetes control plane. Alternatively, the Kubernetes API
server could rate limit you, temporarily rejecting your requests with a 429 status.
- You are limited by a {{< glossary_tooltip text="resource quota" term_id="resource-quota" >}}
on Jobs: the API server permanently rejects some of your requests
when you create a great deal of work in one batch.
-->
- 即使使用标签，管理这么多 Job 对象也很麻烦。
- 如果你一次性创建很多 Job，很可能会给 Kubernetes 控制面带来很大压力。
一种替代方案是，Kubernetes API 可能对请求施加速率限制，通过 429 返回
状态值临时拒绝你的请求。
- 你可能会受到 Job 相关的{{< glossary_tooltip text="资源配额" term_id="resource-quota" >}}
限制：如果你在一个批量请求中触发了太多的任务，API 服务器会永久性地拒绝你的某些请求。
<!--
There are other [job patterns](/docs/concepts/workloads/controllers/job/#job-patterns)
that you can use to process large amounts of work without creating very many Job
objects.
You could also consider writing your own [controller](/docs/concepts/architecture/controller/)
to manage Job objects automatically.
-->
还有一些其他[作业模式](/zh/docs/concepts/workloads/controllers/job/#job-patterns)
可供选择，这些模式都能用来处理大量任务而又不会创建过多的 Job 对象。
你也可以考虑编写自己的[控制器](/zh/docs/concepts/architecture/controller/)
来自动管理 Job 对象。