First Japanese l10n work for release-1.13 (#12998)
* [ja] add basic files for 1.13 (#11571) * [ja] add basic files for 1.13 * [ja] add some base files * Translate setup/independent/_index.md (#11573) * Translate content/ja/docs/home/_index.md in Japanese (#11569) * Translate content/ja/docs/setup/custom-cloud/_index.md in Japanese (#11572) * Translate content/en/docs/setup/on-premises-vm/_index.md in Japanese (#11574) * Translate content/ja/docs/setup/release/_index.md in Japanese (#11576) * ja-trans: Translate content/ja/docs/tutorials/kubernetes-basics/explore/_index.md (#11580) * Translate content/ja/docs/setup/turnkey/_index.md (#11582) * Translate content/ja/docs/tutorials/kubernetes-basics/update/_index.m… (#11579) * Translate content/ja/docs/tutorials/kubernetes-basics/update/_index.md in Japanese * Fix title * Translated Tutorials/Learn Kubenetes Basics/Deploy an App in Japanese. (#11583) * translate tutorials/kubernetes-basics/expose/_index.md (#11584) * Dev 1.13 ja.1 tutorials kubernetes basics scale (#11577) * Translate content/ja/docs/tutorials/kubernetes-basics/scale/_index.md in Japanese * Fix title * translate deprecated state description (#11578) * Fix the build doesn't pass at dev-1.13-ja.1 (#11609) * delete files not at minimum requirements to pass the build. * copy necessary file for pass build from content/en * translate content/ja/_index.html (#11585) * ja-trans: add docs/_index.md (#11721) * Remove copied docs/index.md by mistake. (#11735) * Translate stable state description (#11642) * translate stable state description * Update content/ja/docs/templates/feature-state-stable.txt Co-Authored-By: auifzysr <38824461+auifzysr@users.noreply.github.com> * apply the suggestion directly * Translate alpha state description (#11753) * [ja] add ja section (#11581) * [ja] translate case-studies (#12060) * [ja] translate case-studies * remove comment * fix /ja/docs/ content (#12062) * Translate content/ja/docs/tutorials/kubernetes-basics/create-cluster/_index.md in Japanese (#12059) * [ja] translate supported doc versions (#12068) * [ja] add ja.toml (#11595) * Remove reviewers block from front matter. (#12092) * Translate beta state description (#12023) * [ja] translate setup (#12070) * translate setup * add translation * Update _index.md * Update _index.md * 表記ゆれ * 表記ゆれ * [ja] translate what-is-kubernetes (#12065) * translate what-is-kubernetes * add more translation * finish basic translation * Update content/ja/docs/concepts/overview/what-is-kubernetes.md Co-Authored-By: d-kuro <34958495+d-kuro@users.noreply.github.com> * Update what-is-kubernetes.md * Update content/ja/docs/concepts/overview/what-is-kubernetes.md Co-Authored-By: inductor <kohei.ota@zozo.com> * Update content/ja/docs/concepts/overview/what-is-kubernetes.md Co-Authored-By: inductor <kohei.ota@zozo.com> * Update content/ja/docs/concepts/overview/what-is-kubernetes.md Co-Authored-By: inductor <kohei.ota@zozo.com> * fix new lines * fix review * Update content/ja/docs/concepts/overview/what-is-kubernetes.md Co-Authored-By: inductor <kohei.ota@zozo.com> * Update what-is-kubernetes.md * Update what-is-kubernetes.md * rephrase プラクティス to 知見 * Update content/ja/docs/concepts/overview/what-is-kubernetes.md Co-Authored-By: inductor <kohei.ota@zozo.com> * Update content/ja/docs/concepts/overview/what-is-kubernetes.md Co-Authored-By: inductor <kohei.ota@zozo.com> * italic * オーケストレーション * [ja] tutorials/index (#12071) * translate tutorial index * fix page link * add ja to path for kubernetes-basic because it's already in progress of translation * Update _index.md * review * remove typo * Update content/ja/docs/tutorials/_index.md Co-Authored-By: inductor <kohei.ota@zozo.com> * Update content/ja/docs/tutorials/_index.md Co-Authored-By: inductor <kohei.ota@zozo.com> * [ja] translate cri installation (#12095) * [ja] translate cri installation * Update content/ja/docs/setup/cri.md Co-Authored-By: auifzysr <38824461+auifzysr@users.noreply.github.com> * apply comments * apply comments * [ja]translate tutorials/kubernetes-basics (#12074) * start translation * translate index * wording * wording * cluster-interactive * cluster-intro * update interactive * update some data * fix link * deploy-intro * japanize * fix path for public data * wording * start translation of expose * expose intro * けーしょん * scale-intro * update-intro * fix wrong word * fix wording * translate missing string * Update content/ja/docs/tutorials/kubernetes-basics/_index.html Co-Authored-By: inductor <kohei.ota@zozo.com> * Update content/ja/docs/tutorials/kubernetes-basics/_index.html Co-Authored-By: inductor <kohei.ota@zozo.com> * Update content/ja/docs/tutorials/kubernetes-basics/scale/scale-intro.html Co-Authored-By: inductor <kohei.ota@zozo.com> * Update content/ja/docs/tutorials/kubernetes-basics/expose/expose-intro.html Co-Authored-By: inductor <kohei.ota@zozo.com> * Update content/ja/docs/tutorials/kubernetes-basics/expose/expose-interactive.html Co-Authored-By: inductor <kohei.ota@zozo.com> * Update content/ja/docs/tutorials/kubernetes-basics/_index.html Co-Authored-By: inductor <kohei.ota@zozo.com> * fix wording * Update content/ja/docs/tutorials/kubernetes-basics/create-cluster/cluster-intro.html Co-Authored-By: inductor <kohei.ota@zozo.com> * Update content/ja/docs/tutorials/kubernetes-basics/create-cluster/cluster-intro.html Co-Authored-By: inductor <kohei.ota@zozo.com> * Update content/ja/docs/tutorials/kubernetes-basics/create-cluster/cluster-intro.html Co-Authored-By: inductor <kohei.ota@zozo.com> * Update content/ja/docs/tutorials/kubernetes-basics/scale/scale-interactive.html Co-Authored-By: inductor <kohei.ota@zozo.com> * Update content/ja/docs/tutorials/kubernetes-basics/expose/expose-interactive.html Co-Authored-By: inductor <kohei.ota@zozo.com> * Update content/ja/docs/tutorials/kubernetes-basics/explore/explore-interactive.html Co-Authored-By: inductor <kohei.ota@zozo.com> * Update content/ja/docs/tutorials/kubernetes-basics/deploy-app/deploy-interactive.html Co-Authored-By: inductor <kohei.ota@zozo.com> * Update content/ja/docs/tutorials/kubernetes-basics/create-cluster/cluster-interactive.html Co-Authored-By: inductor <kohei.ota@zozo.com> * Update content/ja/docs/tutorials/kubernetes-basics/deploy-app/deploy-intro.html Co-Authored-By: inductor <kohei.ota@zozo.com> * Update content/ja/docs/tutorials/kubernetes-basics/deploy-app/deploy-intro.html Co-Authored-By: inductor <kohei.ota@zozo.com> * Update content/ja/docs/tutorials/kubernetes-basics/explore/explore-intro.html Co-Authored-By: inductor <kohei.ota@zozo.com> * Update content/ja/docs/tutorials/kubernetes-basics/scale/scale-intro.html Co-Authored-By: inductor <kohei.ota@zozo.com> * lowercase for kubectl * ja-trans: tutorials/hello-minikube.md (#11648) * trns-ja: tutorials/hello-minikube.md * Update content/ja/docs/tutorials/hello-minikube.md Co-Authored-By: lkougi <45655192+lkougi@users.noreply.github.com> * Update content/ja/docs/tutorials/hello-minikube.md Co-Authored-By: lkougi <45655192+lkougi@users.noreply.github.com> * Update content/ja/docs/tutorials/hello-minikube.md Co-Authored-By: lkougi <45655192+lkougi@users.noreply.github.com> * Update content/ja/docs/tutorials/hello-minikube.md Co-Authored-By: lkougi <45655192+lkougi@users.noreply.github.com> * Update content/ja/docs/tutorials/hello-minikube.md Co-Authored-By: lkougi <45655192+lkougi@users.noreply.github.com> * Update content/ja/docs/tutorials/hello-minikube.md Co-Authored-By: lkougi <45655192+lkougi@users.noreply.github.com> * Update content/ja/docs/tutorials/hello-minikube.md Co-Authored-By: lkougi <45655192+lkougi@users.noreply.github.com> * Update content/ja/docs/tutorials/hello-minikube.md Co-Authored-By: lkougi <45655192+lkougi@users.noreply.github.com> * Update content/ja/docs/tutorials/hello-minikube.md Co-Authored-By: lkougi <45655192+lkougi@users.noreply.github.com> * Update content/ja/docs/tutorials/hello-minikube.md Co-Authored-By: lkougi <45655192+lkougi@users.noreply.github.com> * Update content/ja/docs/tutorials/hello-minikube.md Co-Authored-By: lkougi <45655192+lkougi@users.noreply.github.com> * Update content/ja/docs/tutorials/hello-minikube.md Co-Authored-By: lkougi <45655192+lkougi@users.noreply.github.com> * Update content/ja/docs/tutorials/hello-minikube.md Co-Authored-By: lkougi <45655192+lkougi@users.noreply.github.com> * Update content/ja/docs/tutorials/hello-minikube.md Co-Authored-By: lkougi <45655192+lkougi@users.noreply.github.com> * Update hello-minikube.md 大変、大変遅くなりました。丁寧に見ていただいて感謝です。いただいたコメントを反映しました。 * Update content/ja/docs/tutorials/hello-minikube.md Co-Authored-By: lkougi <45655192+lkougi@users.noreply.github.com> * Update content/ja/docs/tutorials/hello-minikube.md Co-Authored-By: lkougi <45655192+lkougi@users.noreply.github.com> * Update content/ja/docs/tutorials/hello-minikube.md Co-Authored-By: lkougi <45655192+lkougi@users.noreply.github.com> * Update content/ja/docs/tutorials/hello-minikube.md Co-Authored-By: lkougi <45655192+lkougi@users.noreply.github.com> * Update content/ja/docs/tutorials/hello-minikube.md Co-Authored-By: lkougi <45655192+lkougi@users.noreply.github.com> * Update content/ja/docs/tutorials/hello-minikube.md Co-Authored-By: lkougi <45655192+lkougi@users.noreply.github.com> * Update hello-minikube.md <修正点> ・10行目の「本チュートリアルでは」を削除 ・クラスターをクラスタに統一 * Update hello-minikube.md 10行目の実践を手を動かすに修正 * Update hello-minikube.md 10行目を「手を動かす準備はできていますか?本チュートリアルでは、Node.jsを使った簡単な"Hello World"を実行するKubernetesクラスタをビルドします。」に差し替え。 * Update content/ja/docs/tutorials/hello-minikube.md Co-Authored-By: lkougi <45655192+lkougi@users.noreply.github.com> * Update content/ja/docs/tutorials/hello-minikube.md Co-Authored-By: lkougi <45655192+lkougi@users.noreply.github.com> * ja-trans: setup/custom-cloud/coreos/ (#12731) * ja-trans: setup/release/building-from-source/ (#12721) * translate building-from-source * improve translation * ja-trans: translate setup/certificates/ (#12722) * translate certificates.md * change translation about Paths * ja-trans: setup/custom-cloud/kubespray/ (#12733) * ja-trans: setup/node-conformance/ (#12728) * ja-trans: setup/node-conformance/ * Update content/ja/docs/setup/node-conformance.md LGTM Co-Authored-By: makocchi-git <makocchi@gmail.com> * Update content/ja/docs/setup/node-conformance.md LGTM Co-Authored-By: makocchi-git <makocchi@gmail.com> * Update content/ja/docs/setup/node-conformance.md LGTM Co-Authored-By: makocchi-git <makocchi@gmail.com> * ja-trans: setup/cluster-large/ (#12723) * ja-trans: setup/cluster-large/ * translate quota and addon * ja-trans: setup/pick-right-solution/ (#12729) * ja-trans: setup/pick-right-solution/ * revise translating solutions * ending with a noun * ja-trans: setup/custom-cloud/kops/ (#12732) * ja-trans: setup/custom-cloud/kops/ * improve translation * translate build * translate explore and add-ons * ja-trans: setup/independent/control-plane-flags/ (#12745) * ja-trans: setup/minikube/ (#12724) * ja-trans: setup/minikube/ * Update content/ja/docs/setup/minikube.md LGTM Co-Authored-By: makocchi-git <makocchi@gmail.com> * translate features and add-ons * improve translation * improve translation * fix translation style * ja-trans: setup/multiple-zones/ (#12725) * ja-trans: setup/multiple-zones/ * ja-trans: setup/multiple-zones/ (2) * ending with a noun * fix translation style * ja-trans: setup/scratch/ (#12730) * ja-trans: setup/scratch/ * revise translating connectivity * improve translation * Update content/ja/docs/setup/scratch.md LGTM Co-Authored-By: makocchi-git <makocchi@gmail.com> * Update content/ja/docs/setup/scratch.md LGTM Co-Authored-By: makocchi-git <makocchi@gmail.com> * Update content/ja/docs/setup/scratch.md LGTM Co-Authored-By: makocchi-git <makocchi@gmail.com> * Update content/ja/docs/setup/scratch.md LGTM Co-Authored-By: makocchi-git <makocchi@gmail.com> * revise translation * revert some words to English * fix translation style * fix title * ja-trans: setup/independent/create-cluster-kubeadm/ (#12750) * ja-trans: setup/independent/create-cluster-kubeadm/ * translate Instructions * fix translation style * ja-trans: setup/independent/kubelet-integration/ (#12754) * ja-trans: setup/independent/kubelet-integration/ * fix translation style * ja-trans: setup/independent/setup-ha-etcd-with-kubeadm/ (#12755) * ja-trans: setup/independent/setup-ha-etcd-with-kubeadm/ * fix translation style * ja-trans: setup/independent/troubleshooting-kubeadm/ (#12757) * ja-trans: setup/independent/troubleshooting-kubeadm/ * pod -> Pod * ja-trans: setup/on-premises-vm/cloudstack/ (#12772) * ja-trans: setup/independent/high-availability/ (#12753) * ja-trans: setup/independent/high-availability/ * fix translation style * translate Stacked and worker node * ja-trans: setup/on-premises-metal/krib/ (#12770) * ja-trans: setup/on-premises-metal/krib/ * Update content/ja/docs/setup/on-premises-metal/krib.md Co-Authored-By: makocchi-git <makocchi@gmail.com> * ja-trans: setup/on-premises-vm/ovirt/ (#12781) * ja-trans: setup/on-premises-vm/dcos/ (#12780) * ja-trans: setup/on-premises-vm/dcos/ * fix translation * Update content/ja/docs/setup/on-premises-vm/dcos.md Co-Authored-By: makocchi-git <makocchi@gmail.com> * ja-trans: setup/turnkey/alibaba-cloud/ (#12786) * ja-trans: setup/turnkey/alibaba-cloud/ * tiny fix * Update content/ja/docs/setup/turnkey/alibaba-cloud.md Co-Authored-By: makocchi-git <makocchi@gmail.com> * fix translation * ja-trans: setup/turnkey/aws/ (#12788) * ja-trans: setup/turnkey/aws/ * translate production grade * fix translation * ja-trans: setup/release/notes/ (#12791) * ja-trans: setup/independent/install-kubeadm.md (#12812) * ja-trans: setup/independent/install-kubeadm.md * ja-trans: fix internal links in setup/independent/install-kubeadm.md * ja-trans: setup/turnkey/clc/ (#12824) * ja-trans: setup/turnkey/clc/ * Update content/ja/docs/setup/turnkey/clc.md Co-Authored-By: makocchi-git <makocchi@gmail.com> * Update content/ja/docs/setup/turnkey/clc.md Co-Authored-By: makocchi-git <makocchi@gmail.com> * ja-trans: setup/turnkey/stackpoint/ (#12853) * ja-trans: concepts/ (#12820) * ja-trans: concepts/ * fix translation * ja: fix formatting in what is kubernetes (#12694) * fix formatting in what is kubernetes * Update content/ja/docs/concepts/overview/what-is-kubernetes.md Co-Authored-By: inductor <kohei.ota@zozo.com> * ? * format (#12866) * ja-trans: setup/turnkey/gce.md (#12813) * ja-trans: setup/turnkey/gce.md * Update content/ja/docs/setup/turnkey/gce.md Co-Authored-By: auifzysr <38824461+auifzysr@users.noreply.github.com> * Update content/ja/docs/setup/turnkey/gce.md Co-Authored-By: auifzysr <38824461+auifzysr@users.noreply.github.com> * ja-trans: modify a word in setup/turnkey/gce.md * Translated docs/setup/turnkey/azure.md. (#12951) * Translated docs/setup/turnkey/azure.md. * Update content/ja/docs/setup/turnkey/azure.md Applied a suggestion. Co-Authored-By: dzeyelid <dzeyelid@gmail.com> * Update content/ja/docs/setup/turnkey/azure.md Applied a suggestion. Co-Authored-By: dzeyelid <dzeyelid@gmail.com> * Update content/ja/docs/setup/turnkey/azure.md Applied suggestion. Co-Authored-By: dzeyelid <dzeyelid@gmail.com> * Applied review suggestions. * Applied review suggestions. * fix language setting order.
This commit is contained in:
committed by
Kubernetes Prow Robot
parent
5c99d3431a
commit
edb233cac2
@@ -0,0 +1,18 @@
|
||||
apiVersion: batch/v1beta1
|
||||
kind: CronJob
|
||||
metadata:
|
||||
name: hello
|
||||
spec:
|
||||
schedule: "*/1 * * * *"
|
||||
jobTemplate:
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
containers:
|
||||
- name: hello
|
||||
image: busybox
|
||||
args:
|
||||
- /bin/sh
|
||||
- -c
|
||||
- date; echo Hello from the Kubernetes cluster
|
||||
restartPolicy: OnFailure
|
||||
@@ -0,0 +1,18 @@
|
||||
apiVersion: batch/v1
|
||||
kind: Job
|
||||
metadata:
|
||||
name: process-item-$ITEM
|
||||
labels:
|
||||
jobgroup: jobexample
|
||||
spec:
|
||||
template:
|
||||
metadata:
|
||||
name: jobexample
|
||||
labels:
|
||||
jobgroup: jobexample
|
||||
spec:
|
||||
containers:
|
||||
- name: c
|
||||
image: busybox
|
||||
command: ["sh", "-c", "echo Processing item $ITEM && sleep 5"]
|
||||
restartPolicy: Never
|
||||
@@ -0,0 +1,10 @@
|
||||
# Specify BROKER_URL and QUEUE when running
|
||||
FROM ubuntu:14.04
|
||||
|
||||
RUN apt-get update && \
|
||||
apt-get install -y curl ca-certificates amqp-tools python \
|
||||
--no-install-recommends \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
COPY ./worker.py /worker.py
|
||||
|
||||
CMD /usr/bin/amqp-consume --url=$BROKER_URL -q $QUEUE -c 1 /worker.py
|
||||
@@ -0,0 +1,20 @@
|
||||
apiVersion: batch/v1
|
||||
kind: Job
|
||||
metadata:
|
||||
name: job-wq-1
|
||||
spec:
|
||||
completions: 8
|
||||
parallelism: 2
|
||||
template:
|
||||
metadata:
|
||||
name: job-wq-1
|
||||
spec:
|
||||
containers:
|
||||
- name: c
|
||||
image: gcr.io/<project>/job-wq-1
|
||||
env:
|
||||
- name: BROKER_URL
|
||||
value: amqp://guest:guest@rabbitmq-service:5672
|
||||
- name: QUEUE
|
||||
value: job1
|
||||
restartPolicy: OnFailure
|
||||
@@ -0,0 +1,7 @@
|
||||
#!/usr/bin/env python
|
||||
|
||||
# Just prints standard out and sleeps for 10 seconds.
|
||||
import sys
|
||||
import time
|
||||
print("Processing " + sys.stdin.lines())
|
||||
time.sleep(10)
|
||||
@@ -0,0 +1,6 @@
|
||||
FROM python
|
||||
RUN pip install redis
|
||||
COPY ./worker.py /worker.py
|
||||
COPY ./rediswq.py /rediswq.py
|
||||
|
||||
CMD python worker.py
|
||||
@@ -0,0 +1,14 @@
|
||||
apiVersion: batch/v1
|
||||
kind: Job
|
||||
metadata:
|
||||
name: job-wq-2
|
||||
spec:
|
||||
parallelism: 2
|
||||
template:
|
||||
metadata:
|
||||
name: job-wq-2
|
||||
spec:
|
||||
containers:
|
||||
- name: c
|
||||
image: gcr.io/myproject/job-wq-2
|
||||
restartPolicy: OnFailure
|
||||
@@ -0,0 +1,15 @@
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: redis-master
|
||||
labels:
|
||||
app: redis
|
||||
spec:
|
||||
containers:
|
||||
- name: master
|
||||
image: redis
|
||||
env:
|
||||
- name: MASTER
|
||||
value: "true"
|
||||
ports:
|
||||
- containerPort: 6379
|
||||
@@ -0,0 +1,10 @@
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: redis
|
||||
spec:
|
||||
ports:
|
||||
- port: 6379
|
||||
targetPort: 6379
|
||||
selector:
|
||||
app: redis
|
||||
@@ -0,0 +1,130 @@
|
||||
#!/usr/bin/env python
|
||||
|
||||
# Based on http://peter-hoffmann.com/2012/python-simple-queue-redis-queue.html
|
||||
# and the suggestion in the redis documentation for RPOPLPUSH, at
|
||||
# http://redis.io/commands/rpoplpush, which suggests how to implement a work-queue.
|
||||
|
||||
|
||||
import redis
|
||||
import uuid
|
||||
import hashlib
|
||||
|
||||
class RedisWQ(object):
|
||||
"""Simple Finite Work Queue with Redis Backend
|
||||
|
||||
This work queue is finite: as long as no more work is added
|
||||
after workers start, the workers can detect when the queue
|
||||
is completely empty.
|
||||
|
||||
The items in the work queue are assumed to have unique values.
|
||||
|
||||
This object is not intended to be used by multiple threads
|
||||
concurrently.
|
||||
"""
|
||||
def __init__(self, name, **redis_kwargs):
|
||||
"""The default connection parameters are: host='localhost', port=6379, db=0
|
||||
|
||||
The work queue is identified by "name". The library may create other
|
||||
keys with "name" as a prefix.
|
||||
"""
|
||||
self._db = redis.StrictRedis(**redis_kwargs)
|
||||
# The session ID will uniquely identify this "worker".
|
||||
self._session = str(uuid.uuid4())
|
||||
# Work queue is implemented as two queues: main, and processing.
|
||||
# Work is initially in main, and moved to processing when a client picks it up.
|
||||
self._main_q_key = name
|
||||
self._processing_q_key = name + ":processing"
|
||||
self._lease_key_prefix = name + ":leased_by_session:"
|
||||
|
||||
def sessionID(self):
|
||||
"""Return the ID for this session."""
|
||||
return self._session
|
||||
|
||||
def _main_qsize(self):
|
||||
"""Return the size of the main queue."""
|
||||
return self._db.llen(self._main_q_key)
|
||||
|
||||
def _processing_qsize(self):
|
||||
"""Return the size of the main queue."""
|
||||
return self._db.llen(self._processing_q_key)
|
||||
|
||||
def empty(self):
|
||||
"""Return True if the queue is empty, including work being done, False otherwise.
|
||||
|
||||
False does not necessarily mean that there is work available to work on right now,
|
||||
"""
|
||||
return self._main_qsize() == 0 and self._processing_qsize() == 0
|
||||
|
||||
# TODO: implement this
|
||||
# def check_expired_leases(self):
|
||||
# """Return to the work queueReturn True if the queue is empty, False otherwise."""
|
||||
# # Processing list should not be _too_ long since it is approximately as long
|
||||
# # as the number of active and recently active workers.
|
||||
# processing = self._db.lrange(self._processing_q_key, 0, -1)
|
||||
# for item in processing:
|
||||
# # If the lease key is not present for an item (it expired or was
|
||||
# # never created because the client crashed before creating it)
|
||||
# # then move the item back to the main queue so others can work on it.
|
||||
# if not self._lease_exists(item):
|
||||
# TODO: transactionally move the key from processing queue to
|
||||
# to main queue, while detecting if a new lease is created
|
||||
# or if either queue is modified.
|
||||
|
||||
def _itemkey(self, item):
|
||||
"""Returns a string that uniquely identifies an item (bytes)."""
|
||||
return hashlib.sha224(item).hexdigest()
|
||||
|
||||
def _lease_exists(self, item):
|
||||
"""True if a lease on 'item' exists."""
|
||||
return self._db.exists(self._lease_key_prefix + self._itemkey(item))
|
||||
|
||||
def lease(self, lease_secs=60, block=True, timeout=None):
|
||||
"""Begin working on an item the work queue.
|
||||
|
||||
Lease the item for lease_secs. After that time, other
|
||||
workers may consider this client to have crashed or stalled
|
||||
and pick up the item instead.
|
||||
|
||||
If optional args block is true and timeout is None (the default), block
|
||||
if necessary until an item is available."""
|
||||
if block:
|
||||
item = self._db.brpoplpush(self._main_q_key, self._processing_q_key, timeout=timeout)
|
||||
else:
|
||||
item = self._db.rpoplpush(self._main_q_key, self._processing_q_key)
|
||||
if item:
|
||||
# Record that we (this session id) are working on a key. Expire that
|
||||
# note after the lease timeout.
|
||||
# Note: if we crash at this line of the program, then GC will see no lease
|
||||
# for this item a later return it to the main queue.
|
||||
itemkey = self._itemkey(item)
|
||||
self._db.setex(self._lease_key_prefix + itemkey, lease_secs, self._session)
|
||||
return item
|
||||
|
||||
def complete(self, value):
|
||||
"""Complete working on the item with 'value'.
|
||||
|
||||
If the lease expired, the item may not have completed, and some
|
||||
other worker may have picked it up. There is no indication
|
||||
of what happened.
|
||||
"""
|
||||
self._db.lrem(self._processing_q_key, 0, value)
|
||||
# If we crash here, then the GC code will try to move the value, but it will
|
||||
# not be here, which is fine. So this does not need to be a transaction.
|
||||
itemkey = self._itemkey(value)
|
||||
self._db.delete(self._lease_key_prefix + itemkey, self._session)
|
||||
|
||||
# TODO: add functions to clean up all keys associated with "name" when
|
||||
# processing is complete.
|
||||
|
||||
# TODO: add a function to add an item to the queue. Atomically
|
||||
# check if the queue is empty and if so fail to add the item
|
||||
# since other workers might think work is done and be in the process
|
||||
# of exiting.
|
||||
|
||||
# TODO(etune): move to my own github for hosting, e.g. github.com/erictune/rediswq-py and
|
||||
# make it so it can be pip installed by anyone (see
|
||||
# http://stackoverflow.com/questions/8247605/configuring-so-that-pip-install-can-work-from-github)
|
||||
|
||||
# TODO(etune): finish code to GC expired leases, and call periodically
|
||||
# e.g. each time lease times out.
|
||||
|
||||
@@ -0,0 +1,23 @@
|
||||
#!/usr/bin/env python
|
||||
|
||||
import time
|
||||
import rediswq
|
||||
|
||||
host="redis"
|
||||
# Uncomment next two lines if you do not have Kube-DNS working.
|
||||
# import os
|
||||
# host = os.getenv("REDIS_SERVICE_HOST")
|
||||
|
||||
q = rediswq.RedisWQ(name="job2", host="redis")
|
||||
print("Worker with sessionID: " + q.sessionID())
|
||||
print("Initial queue state: empty=" + str(q.empty()))
|
||||
while not q.empty():
|
||||
item = q.lease(lease_secs=10, block=True, timeout=2)
|
||||
if item is not None:
|
||||
itemstr = item.decode("utf=8")
|
||||
print("Working on " + itemstr)
|
||||
time.sleep(10) # Put your actual work here instead of sleep.
|
||||
q.complete(item)
|
||||
else:
|
||||
print("Waiting for work")
|
||||
print("Queue empty, exiting")
|
||||
Reference in New Issue
Block a user