Merge branch 'kubernetes:master' into ingress-nginx-stabilization
This commit is contained in:
James Strong
@@ -19,7 +19,11 @@ help: ## Show this help.
|
||||
@awk 'BEGIN {FS = ":.*?## "} /^[a-zA-Z_-]+:.*?## / {sub("\\\\n",sprintf("\n%22c"," "), $$2);printf "\033[36m%-20s\033[0m %s\n", $$1, $$2}' $(MAKEFILE_LIST)
|
||||
|
||||
module-check:
|
||||
@git submodule status --recursive | awk '/^[+-]/ {printf "\033[31mWARNING\033[0m Submodule not initialized: \033[34m%s\033[0m\n",$$2}' 1>&2
|
||||
@git submodule status --recursive | awk '/^[+-]/ {err = 1; printf "\033[31mWARNING\033[0m Submodule not initialized: \033[34m%s\033[0m\n",$$2} END { if (err != 0) print "You need to run \033[32mmake module-init\033[0m to initialize missing modules first"; exit err }' 1>&2
|
||||
|
||||
module-init:
|
||||
@echo "Initializing submodules..." 1>&2
|
||||
@git submodule update --init --recursive --depth 1
|
||||
|
||||
all: build ## Build site with production settings and put deliverables in ./public
|
||||
|
||||
|
||||
@@ -37,6 +37,13 @@ Um die Kubernetes-Website lokal laufen zu lassen, empfiehlt es sich, ein speziel
|
||||
|
||||
> Wenn Sie die Website lieber lokal ohne Docker ausführen möchten, finden Sie weitere Informationen unter [Website lokal mit Hugo ausführen](#Die-Site-lokal-mit-Hugo-ausführen).
|
||||
|
||||
Das benötigte [Docsy Hugo theme](https://github.com/google/docsy#readme) muss als git submodule installiert werden:
|
||||
|
||||
```
|
||||
#Füge das Docsy submodule hinzu
|
||||
git submodule update --init --recursive --depth 1
|
||||
```
|
||||
|
||||
Wenn Sie Docker [installiert](https://www.docker.com/get-started) haben, erstellen Sie das Docker-Image `kubernetes-hugo` lokal:
|
||||
|
||||
```bash
|
||||
@@ -55,9 +62,18 @@ make container-serve
|
||||
|
||||
Hugo-Installationsanweisungen finden Sie in der [offiziellen Hugo-Dokumentation](https://gohugo.io/getting-started/installing/). Stellen Sie sicher, dass Sie die Hugo-Version installieren, die in der Umgebungsvariablen `HUGO_VERSION` in der Datei [`netlify.toml`](netlify.toml#L9) angegeben ist.
|
||||
|
||||
Das benötigte [Docsy Hugo theme](https://github.com/google/docsy#readme) muss als git submodule installiert werden:
|
||||
|
||||
```
|
||||
#Füge das Docsy submodule hinzu
|
||||
git submodule update --init --recursive --depth 1
|
||||
```
|
||||
|
||||
So führen Sie die Site lokal aus, wenn Sie Hugo installiert haben:
|
||||
|
||||
```bash
|
||||
# Installieren der JavaScript Abhängigkeiten
|
||||
npm ci
|
||||
make serve
|
||||
```
|
||||
|
||||
|
||||
+2
-2
@@ -174,7 +174,7 @@ Learn more about SIG Docs Kubernetes community and meetings on the [community pa
|
||||
|
||||
You can also reach the maintainers of this project at:
|
||||
|
||||
- [Slack](https://kubernetes.slack.com/messages/sig-docs)
|
||||
- [Slack](https://kubernetes.slack.com/messages/sig-docs) [Get an invite for this Slack](https://slack.k8s.io/)
|
||||
- [Mailing List](https://groups.google.com/forum/#!forum/kubernetes-sig-docs)
|
||||
-->
|
||||
# 参与 SIG Docs 工作
|
||||
@@ -184,7 +184,7 @@ You can also reach the maintainers of this project at:
|
||||
|
||||
你也可以通过以下渠道联系本项目的维护人员：
|
||||
|
||||
- [Slack](https://kubernetes.slack.com/messages/sig-docs)
|
||||
- [Slack](https://kubernetes.slack.com/messages/sig-docs) [加入Slack](https://slack.k8s.io/)
|
||||
- [邮件列表](https://groups.google.com/forum/#!forum/kubernetes-sig-docs)
|
||||
|
||||
<!--
|
||||
|
||||
@@ -43,12 +43,12 @@ Kubernetes is open source giving you the freedom to take advantage of on-premise
|
||||
<button id="desktopShowVideoButton" onclick="kub.showVideo()">Watch Video</button>
|
||||
<br>
|
||||
<br>
|
||||
<a href="https://events.linuxfoundation.org/kubecon-cloudnativecon-north-america/?utm_source=kubernetes.io&utm_medium=nav&utm_campaign=kccncna20" button id="desktopKCButton">Attend KubeCon NA virtually on November 17-20, 2020</a>
|
||||
<a href="https://events.linuxfoundation.org/kubecon-cloudnativecon-north-america/?utm_source=kubernetes.io&utm_medium=nav&utm_campaign=kccncna21" button id="desktopKCButton">Attend KubeCon North America on October 11-15, 2021</a>
|
||||
<br>
|
||||
<br>
|
||||
<br>
|
||||
<br>
|
||||
<a href="https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/?utm_source=kubernetes.io&utm_medium=nav&utm_campaign=kccnceu21" button id="desktopKCButton">Attend KubeCon EU virtually on May 4 – 7, 2021</a>
|
||||
<a href="https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/?utm_source=kubernetes.io&utm_medium=nav&utm_campaign=kccnceu21" button id="desktopKCButton">Revisit KubeCon EU 2021</a>
|
||||
</div>
|
||||
<div id="videoPlayer">
|
||||
<iframe data-url="https://www.youtube.com/embed/H06qrNmGqyE?autoplay=1" frameborder="0" allowfullscreen></iframe>
|
||||
|
||||
@@ -95,7 +95,7 @@ The core workloads API surface is stable, but it’s still software, and softwar
|
||||
--Kenneth Owens, Software Engineer, Google
|
||||
|
||||
|
||||
- [Download](http://get.k8s.io/) Kubernetes
|
||||
- [Download](https://get.k8s.io/) Kubernetes
|
||||
- Get involved with the Kubernetes project on [GitHub](https://github.com/kubernetes/kubernetes)
|
||||
- Post questions (or answer questions) on [Stack Overflow](http://stackoverflow.com/questions/tagged/kubernetes)
|
||||
- Connect with the community on [Slack](http://slack.k8s.io/)
|
||||
|
||||
@@ -120,7 +120,7 @@ Run the following command to have Docker run the application in a container and
|
||||
```
|
||||
docker run -p 5001:5000 hello-python
|
||||
```
|
||||
Now navigate to http://localhost:5001, and you should see the “Hello form Python!” message.
|
||||
Now navigate to http://localhost:5001, and you should see the “Hello from Python!” message.
|
||||
|
||||
### More info
|
||||
* [Get started with Docker](https://docs.docker.com/get-started/)
|
||||
@@ -201,7 +201,7 @@ kubectl get pods
|
||||
```
|
||||
<img src="/images/blog/get-started-with-kubernetes-using-python/kubectl-get-pods.png" alt="Pod listing" />
|
||||
|
||||
Now navigate to http://localhost:6000, and you should see the “Hello form Python!” message.
|
||||
Now navigate to http://localhost:6000, and you should see the “Hello from Python!” message.
|
||||
|
||||
That’s it! The application is now running in Kubernetes!
|
||||
|
||||
|
||||
@@ -526,6 +526,6 @@ When you enable the API Priority and Fairness feature, the kube-apiserver serves
|
||||
|
||||
|
||||
For background information on design details for API priority and fairness, see
|
||||
the [enhancement proposal](https://github.com/kubernetes/enhancements/blob/master/keps/sig-api-machinery/20190228-priority-and-fairness.md).
|
||||
the [enhancement proposal](https://github.com/kubernetes/enhancements/tree/master/keps/sig-api-machinery/1040-priority-and-fairness).
|
||||
You can make suggestions and feature requests via [SIG API Machinery](https://github.com/kubernetes/community/tree/master/sig-api-machinery)
|
||||
or the feature's [slack channel](http://kubernetes.slack.com/messages/api-priority-and-fairness).
|
||||
|
||||
@@ -253,7 +253,7 @@ message AllocatableResourcesResponse {
|
||||
|
||||
`ContainerDevices` do expose the topology information declaring to which NUMA cells the device is affine.
|
||||
The NUMA cells are identified using a opaque integer ID, which value is consistent to what device
|
||||
plugins report [when they register themselves to the kubelet](https://kubernetes.io/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins/#device-plugin-integration-with-the-topology-manager).
|
||||
plugins report [when they register themselves to the kubelet](/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins/#device-plugin-integration-with-the-topology-manager).
|
||||
|
||||
|
||||
The gRPC service is served over a unix socket at `/var/lib/kubelet/pod-resources/kubelet.sock`.
|
||||
|
||||
@@ -48,7 +48,7 @@ kubectl get pods --field-selector=status.phase!=Running,spec.restartPolicy=Alway
|
||||
|
||||
## Multiple resource types
|
||||
|
||||
You use field selectors across multiple resource types. This `kubectl` command selects all Statefulsets and Services that are not in the `default` namespace:
|
||||
You can use field selectors across multiple resource types. This `kubectl` command selects all Statefulsets and Services that are not in the `default` namespace:
|
||||
|
||||
```shell
|
||||
kubectl get statefulsets,services --all-namespaces --field-selector metadata.namespace!=default
|
||||
|
||||
@@ -464,12 +464,12 @@ allowed prefix, and a `readOnly` field indicating it must be mounted read-only.
|
||||
For example:
|
||||
|
||||
```yaml
|
||||
allowedHostPaths:
|
||||
# This allows "/foo", "/foo/", "/foo/bar" etc., but
|
||||
# disallows "/fool", "/etc/foo" etc.
|
||||
# "/foo/../" is never valid.
|
||||
- pathPrefix: "/foo"
|
||||
readOnly: true # only allow read-only mounts
|
||||
allowedHostPaths:
|
||||
# This allows "/foo", "/foo/", "/foo/bar" etc., but
|
||||
# disallows "/fool", "/etc/foo" etc.
|
||||
# "/foo/../" is never valid.
|
||||
- pathPrefix: "/foo"
|
||||
readOnly: true # only allow read-only mounts
|
||||
```
|
||||
|
||||
{{< warning >}}There are many ways a container with unrestricted access to the host
|
||||
|
||||
@@ -26,40 +26,41 @@ each resource to score nodes based on the request to capacity ratio. This
|
||||
allows users to bin pack extended resources by using appropriate parameters
|
||||
and improves the utilization of scarce resources in large clusters. The
|
||||
behavior of the `RequestedToCapacityRatioResourceAllocation` priority function
|
||||
can be controlled by a configuration option called
|
||||
`requestedToCapacityRatioArguments`. This argument consists of two parameters
|
||||
`shape` and `resources`. The `shape` parameter allows the user to tune the
|
||||
function as least requested or most requested based on `utilization` and
|
||||
`score` values. The `resources` parameter consists of `name` of the resource
|
||||
to be considered during scoring and `weight` specify the weight of each
|
||||
resource.
|
||||
can be controlled by a configuration option called `RequestedToCapacityRatioArgs`.
|
||||
This argument consists of two parameters `shape` and `resources`. The `shape`
|
||||
parameter allows the user to tune the function as least requested or most
|
||||
requested based on `utilization` and `score` values. The `resources` parameter
|
||||
consists of `name` of the resource to be considered during scoring and `weight`
|
||||
specify the weight of each resource.
|
||||
|
||||
Below is an example configuration that sets
|
||||
`requestedToCapacityRatioArguments` to bin packing behavior for extended
|
||||
resources `intel.com/foo` and `intel.com/bar`.
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Policy
|
||||
apiVersion: kubescheduler.config.k8s.io/v1beta1
|
||||
kind: KubeSchedulerConfiguration
|
||||
profiles:
|
||||
# ...
|
||||
priorities:
|
||||
# ...
|
||||
- name: RequestedToCapacityRatioPriority
|
||||
weight: 2
|
||||
argument:
|
||||
requestedToCapacityRatioArguments:
|
||||
shape:
|
||||
- utilization: 0
|
||||
score: 0
|
||||
- utilization: 100
|
||||
score: 10
|
||||
resources:
|
||||
- name: intel.com/foo
|
||||
weight: 3
|
||||
- name: intel.com/bar
|
||||
weight: 5
|
||||
pluginConfig:
|
||||
- name: RequestedToCapacityRatio
|
||||
args:
|
||||
shape:
|
||||
- utilization: 0
|
||||
score: 10
|
||||
- utilization: 100
|
||||
score: 0
|
||||
resources:
|
||||
- name: intel.com/foo
|
||||
weight: 3
|
||||
- name: intel.com/bar
|
||||
weight: 5
|
||||
```
|
||||
|
||||
Referencing the `KubeSchedulerConfiguration` file with the kube-scheduler
|
||||
flag `--config=/path/to/config/file` will pass the configuration to the
|
||||
scheduler.
|
||||
|
||||
**This feature is disabled by default**
|
||||
|
||||
### Tuning the Priority Function
|
||||
|
||||
@@ -10,7 +10,7 @@ weight: 40
|
||||
|
||||
|
||||
<!-- overview -->
|
||||
[_Node affinity_](/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity),
|
||||
[_Node affinity_](/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity)
|
||||
is a property of {{< glossary_tooltip text="Pods" term_id="pod" >}} that *attracts* them to
|
||||
a set of {{< glossary_tooltip text="nodes" term_id="node" >}} (either as a preference or a
|
||||
hard requirement). _Taints_ are the opposite -- they allow a node to repel a set of pods.
|
||||
|
||||
@@ -7,6 +7,7 @@ content_type: concept
|
||||
weight: 20
|
||||
---
|
||||
<!-- overview -->
|
||||
|
||||
Kubernetes creates DNS records for services and pods. You can contact
|
||||
services with consistent DNS names instead of IP addresses.
|
||||
|
||||
@@ -261,6 +262,8 @@ spec:
|
||||
|
||||
### Pod's DNS Config {#pod-dns-config}
|
||||
|
||||
{{< feature-state for_k8s_version="v1.14" state="stable" >}}
|
||||
|
||||
Pod's DNS Config allows users more control on the DNS settings for a Pod.
|
||||
|
||||
The `dnsConfig` field is optional and it can work with any `dnsPolicy` settings.
|
||||
@@ -310,18 +313,6 @@ search default.svc.cluster-domain.example svc.cluster-domain.example cluster-dom
|
||||
options ndots:5
|
||||
```
|
||||
|
||||
### Feature availability
|
||||
|
||||
The availability of Pod DNS Config and DNS Policy "`None`" is shown as below.
|
||||
|
||||
| k8s version | Feature support |
|
||||
| :---------: |:-----------:|
|
||||
| 1.14 | Stable |
|
||||
| 1.10 | Beta (on by default)|
|
||||
| 1.9 | Alpha |
|
||||
|
||||
|
||||
|
||||
## {{% heading "whatsnext" %}}
|
||||
|
||||
|
||||
|
||||
@@ -212,9 +212,9 @@ This ensures that even pods that aren't selected by any other NetworkPolicy will
|
||||
|
||||
## SCTP support
|
||||
|
||||
{{< feature-state for_k8s_version="v1.19" state="beta" >}}
|
||||
{{< feature-state for_k8s_version="v1.20" state="stable" >}}
|
||||
|
||||
As a beta feature, this is enabled by default. To disable SCTP at a cluster level, you (or your cluster administrator) will need to disable the `SCTPSupport` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/) for the API server with `--feature-gates=SCTPSupport=false,…`.
|
||||
As a stable feature, this is enabled by default. To disable SCTP at a cluster level, you (or your cluster administrator) will need to disable the `SCTPSupport` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/) for the API server with `--feature-gates=SCTPSupport=false,…`.
|
||||
When the feature gate is enabled, you can set the `protocol` field of a NetworkPolicy to `SCTP`.
|
||||
|
||||
{{< note >}}
|
||||
|
||||
@@ -13,7 +13,7 @@ weight: 45
|
||||
|
||||
_Topology Aware Hints_ enable topology aware routing by including suggestions
|
||||
for how clients should consume endpoints. This approach adds metadata to enable
|
||||
consumers of EndpointSlice and / or and Endpoints objects, so that traffic to
|
||||
consumers of EndpointSlice and / or Endpoints objects, so that traffic to
|
||||
those network endpoints can be routed closer to where it originated.
|
||||
|
||||
For example, you can route traffic within a locality to reduce
|
||||
|
||||
@@ -86,7 +86,7 @@ rolling out node software updates can cause voluntary disruptions. Also, some im
|
||||
of cluster (node) autoscaling may cause voluntary disruptions to defragment and compact nodes.
|
||||
Your cluster administrator or hosting provider should have documented what level of voluntary
|
||||
disruptions, if any, to expect. Certain configuration options, such as
|
||||
[using PriorityClasses](https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/)
|
||||
[using PriorityClasses](/docs/concepts/configuration/pod-priority-preemption/)
|
||||
in your pod spec can also cause voluntary (and involuntary) disruptions.
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,25 @@
|
||||
---
|
||||
title: Viewing Site Analytics
|
||||
content_type: concept
|
||||
weight: 100
|
||||
card:
|
||||
name: contribute
|
||||
weight: 100
|
||||
---
|
||||
|
||||
<!-- overview -->
|
||||
|
||||
This page contains information about the kubernetes.io analytics dashboard.
|
||||
|
||||
|
||||
<!-- body -->
|
||||
|
||||
[View the dashboard](https://datastudio.google.com/reporting/fede2672-b2fd-402a-91d2-7473bdb10f04).
|
||||
|
||||
This dashboard is built using Google Data Studio and shows information collected on kubernetes.io using Google Analytics.
|
||||
|
||||
### Using the dashboard
|
||||
|
||||
By default, the dashboard shows all collected analytics for the past 30 days. Use the date selector to see data from a different date range. Other filtering options allow you to view data based on user location, the device used to access the site, the translation of the docs used, and more.
|
||||
|
||||
If you notice an issue with this dashboard, or would like to request any improvements, please [open an issue](https://github.com/kubernetes/website/issues/new/choose).
|
||||
@@ -134,7 +134,6 @@ Go to `<k8s-base>` and run these scripts:
|
||||
hack/update-generated-swagger-docs.sh
|
||||
hack/update-openapi-spec.sh
|
||||
hack/update-generated-protobuf.sh
|
||||
hack/update-api-reference-docs.sh
|
||||
```
|
||||
|
||||
Run `git status` to see what was generated.
|
||||
@@ -143,8 +142,6 @@ Run `git status` to see what was generated.
|
||||
On branch master
|
||||
...
|
||||
modified: api/openapi-spec/swagger.json
|
||||
modified: api/swagger-spec/apps_v1.json
|
||||
modified: docs/api-reference/apps/v1/definitions.html
|
||||
modified: staging/src/k8s.io/api/apps/v1/generated.proto
|
||||
modified: staging/src/k8s.io/api/apps/v1/types.go
|
||||
modified: staging/src/k8s.io/api/apps/v1/types_swagger_doc_generated.go
|
||||
|
||||
@@ -38,7 +38,7 @@ client libraries:
|
||||
- [Kubernetes Python client library](https://github.com/kubernetes-client/python)
|
||||
- [Kubernetes Java client library](https://github.com/kubernetes-client/java)
|
||||
- [Kubernetes JavaScript client library](https://github.com/kubernetes-client/javascript)
|
||||
- [Kubernetes Dotnet client library](https://github.com/kubernetes-client/csharp)
|
||||
- [Kubernetes C# client library](https://github.com/kubernetes-client/csharp)
|
||||
- [Kubernetes Haskell Client library](https://github.com/kubernetes-client/haskell)
|
||||
|
||||
## CLI
|
||||
|
||||
@@ -58,7 +58,7 @@ It acts synchronously to modify pods as they are created or updated. When this p
|
||||
1. It ensures that the `ServiceAccount` referenced by the pod exists, and otherwise rejects it.
|
||||
1. It adds a `volume` to the pod which contains a token for API access if neither the ServiceAccount `automountServiceAccountToken` nor the Pod's `automountServiceAccountToken` is set to `false`.
|
||||
1. It adds a `volumeSource` to each container of the pod mounted at `/var/run/secrets/kubernetes.io/serviceaccount`, if the previous step has created a volume for ServiceAccount token.
|
||||
1. If the pod does not contain any `ImagePullSecrets`, then `ImagePullSecrets` of the `ServiceAccount` are added to the pod.
|
||||
1. If the pod does not contain any `imagePullSecrets`, then `imagePullSecrets` of the `ServiceAccount` are added to the pod.
|
||||
|
||||
#### Bound Service Account Token Volume
|
||||
|
||||
@@ -91,14 +91,14 @@ add the following projected volume instead of a Secret-based volume for the non-
|
||||
This projected volume consists of three sources:
|
||||
|
||||
1. A ServiceAccountToken acquired from kube-apiserver via TokenRequest API. It will expire after 1 hour by default or when the pod is deleted. It is bound to the pod and has kube-apiserver as the audience.
|
||||
1. A ConfigMap containing a CA bundle used for verifying connections to the kube-apiserver. This feature depends on the `RootCAConfigMap` feature gate being enabled, which publishes a "kube-root-ca.crt" ConfigMap to every namespace. `RootCAConfigMap` is enabled by default in 1.20, and always enabled in 1.21+.
|
||||
1. A ConfigMap containing a CA bundle used for verifying connections to the kube-apiserver. This feature depends on the `RootCAConfigMap` feature gate, which publishes a "kube-root-ca.crt" ConfigMap to every namespace. `RootCAConfigMap` feature gate is graduated to GA in 1.21 and default to true. (This flag will be removed from --feature-gate arg in 1.22)
|
||||
1. A DownwardAPI that references the namespace of the pod.
|
||||
|
||||
See more details about [projected volumes](/docs/tasks/configure-pod-container/configure-projected-volume-storage/).
|
||||
|
||||
You can manually migrate a secret-based service account volume to a projected volume when
|
||||
You can manually migrate a Secret-based service account volume to a projected volume when
|
||||
the `BoundServiceAccountTokenVolume` feature gate is not enabled by adding the above
|
||||
projected volume to the pod spec. However, `RootCAConfigMap` needs to be enabled.
|
||||
projected volume to the pod spec.
|
||||
|
||||
### Token Controller
|
||||
|
||||
|
||||
@@ -844,7 +844,7 @@ Each feature gate is designed for enabling/disabling a specific feature:
|
||||
- `ValidateProxyRedirects`: This flag controls whether the API server should
|
||||
validate that redirects are only followed to the same host. Only used if the
|
||||
`StreamingProxyRedirects` flag is enabled.
|
||||
- 'VolumeCapacityPriority`: Enable support for prioritizing nodes in different
|
||||
- `VolumeCapacityPriority`: Enable support for prioritizing nodes in different
|
||||
topologies based on available PV capacity.
|
||||
- `VolumePVCDataSource`: Enable support for specifying an existing PVC as a DataSource.
|
||||
- `VolumeScheduling`: Enable volume topology aware scheduling and make the
|
||||
|
||||
@@ -27,7 +27,7 @@ each Pod in the scheduling queue according to constraints and available
|
||||
resources. The scheduler then ranks each valid Node and binds the Pod to a
|
||||
suitable Node. Multiple different schedulers may be used within a cluster;
|
||||
kube-scheduler is the reference implementation.
|
||||
See [scheduling](https://kubernetes.io/docs/concepts/scheduling-eviction/)
|
||||
See [scheduling](/docs/concepts/scheduling-eviction/)
|
||||
for more information about scheduling and the kube-scheduler component.
|
||||
|
||||
```
|
||||
|
||||
@@ -222,7 +222,9 @@ When a single IngressClass resource has this annotation set to `"true"`, new Ing
|
||||
|
||||
## kubernetes.io/ingress.class (deprecated)
|
||||
|
||||
{{< note >}} Starting in v1.18, this annotation is deprecated in favor of `spec.ingressClassName`. {{< /note >}}
|
||||
{{< note >}}
|
||||
Starting in v1.18, this annotation is deprecated in favor of `spec.ingressClassName`.
|
||||
{{< /note >}}
|
||||
|
||||
## storageclass.kubernetes.io/is-default-class
|
||||
|
||||
@@ -230,7 +232,8 @@ Example: `storageclass.kubernetes.io/is-default-class=true`
|
||||
|
||||
Used on: StorageClass
|
||||
|
||||
When a single StorageClass resource has this annotation set to `"true"`, new Physical Volume Claim resource without a class specified will be assigned this default class.
|
||||
When a single StorageClass resource has this annotation set to `"true"`, new PersistentVolumeClaim
|
||||
resource without a class specified will be assigned this default class.
|
||||
|
||||
## alpha.kubernetes.io/provided-node-ip
|
||||
|
||||
|
||||
@@ -250,7 +250,7 @@ only has one pending pods queue.
|
||||
|
||||
## {{% heading "whatsnext" %}}
|
||||
|
||||
* Read the [kube-scheduler reference](https://kubernetes.io/docs/reference/command-line-tools-reference/kube-scheduler/)
|
||||
* Read the [kube-scheduler reference](/docs/reference/command-line-tools-reference/kube-scheduler/)
|
||||
* Learn about [scheduling](/docs/concepts/scheduling-eviction/kube-scheduler/)
|
||||
* Read the [kube-scheduler configuration (v1beta1)](/docs/reference/config-api/kube-scheduler-config.v1beta1/) reference
|
||||
|
||||
|
||||
+1
-1
@@ -17,7 +17,7 @@ Generate keys and certificate signing requests
|
||||
|
||||
Generates keys and certificate signing requests (CSRs) for all the certificates required to run the control plane. This command also generates partial kubeconfig files with private key data in the "users > user > client-key-data" field, and for each kubeconfig file an accompanying ".csr" file is created.
|
||||
|
||||
This command is designed for use in [Kubeadm External CA Mode](https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-certs/#external-ca-mode). It generates CSRs which you can then submit to your external certificate authority for signing.
|
||||
This command is designed for use in [Kubeadm External CA Mode](/docs/tasks/administer-cluster/kubeadm/kubeadm-certs/#external-ca-mode). It generates CSRs which you can then submit to your external certificate authority for signing.
|
||||
|
||||
The PEM encoded signed certificates should then be saved alongside the key files, using ".crt" as the file extension, or in the case of kubeconfig files, the PEM encoded signed certificate should be base64 encoded and added to the kubeconfig file in the "users > user > client-certificate-data" field.
|
||||
|
||||
|
||||
@@ -49,7 +49,7 @@ Some resource types will have one or more sub-resources, represented as sub path
|
||||
* Cluster-scoped subresource: `GET /apis/GROUP/VERSION/RESOURCETYPE/NAME/SUBRESOURCE`
|
||||
* Namespace-scoped subresource: `GET /apis/GROUP/VERSION/namespaces/NAMESPACE/RESOURCETYPE/NAME/SUBRESOURCE`
|
||||
|
||||
The verbs supported for each subresource will differ depending on the object - see the API documentation more information. It is not possible to access sub-resources across multiple resources - generally a new virtual resource type would be used if that becomes necessary.
|
||||
The verbs supported for each subresource will differ depending on the object - see the API documentation for more information. It is not possible to access sub-resources across multiple resources - generally a new virtual resource type would be used if that becomes necessary.
|
||||
|
||||
|
||||
## Efficient detection of changes
|
||||
@@ -442,7 +442,7 @@ feature, see the section on
|
||||
|
||||
## Resource Versions
|
||||
|
||||
Resource versions are strings that identify the server's internal version of an object. Resource versions can be used by clients to determine when objects have changed, or to express data consistency requirements when getting, listing and watching resources. Resource versions must be treated as opaque by clients and passed unmodified back to the server. For example, clients must not assume resource versions are numeric, and may only compare two resource version for equality (i.e. must not compare resource versions for greater-than or less-than relationships).
|
||||
Resource versions are strings that identify the server's internal version of an object. Resource versions can be used by clients to determine when objects have changed, or to express data consistency requirements when getting, listing and watching resources. Resource versions must be treated as opaque by clients and passed unmodified back to the server. For example, clients must not assume resource versions are numeric, and may only compare two resource versions for equality (i.e. must not compare resource versions for greater-than or less-than relationships).
|
||||
|
||||
### ResourceVersion in metadata
|
||||
|
||||
@@ -454,7 +454,7 @@ Clients find resource versions in resources, including the resources in watch ev
|
||||
|
||||
### The ResourceVersion Parameter
|
||||
|
||||
The get, list and watch operations support the `resourceVersion` parameter.
|
||||
The get, list, and watch operations support the `resourceVersion` parameter.
|
||||
|
||||
The exact meaning of this parameter differs depending on the operation and the value of `resourceVersion`.
|
||||
|
||||
|
||||
@@ -66,7 +66,6 @@ their authors, not the Kubernetes team.
|
||||
| PHP | [github.com/maclof/kubernetes-client](https://github.com/maclof/kubernetes-client) |
|
||||
| PHP | [github.com/travisghansen/kubernetes-client-php](https://github.com/travisghansen/kubernetes-client-php) |
|
||||
| PHP | [github.com/renoki-co/php-k8s](https://github.com/renoki-co/php-k8s) |
|
||||
| Python | [github.com/eldarion-gondor/pykube](https://github.com/eldarion-gondor/pykube) |
|
||||
| Python | [github.com/fiaas/k8s](https://github.com/fiaas/k8s) |
|
||||
| Python | [github.com/mnubo/kubernetes-py](https://github.com/mnubo/kubernetes-py) |
|
||||
| Python | [github.com/tomplus/kubernetes_asyncio](https://github.com/tomplus/kubernetes_asyncio) |
|
||||
|
||||
@@ -74,7 +74,7 @@ The **policy/v1beta1** API version of PodDisruptionBudget will no longer be serv
|
||||
PodSecurityPolicy in the **policy/v1beta1** API version will no longer be served in v1.25, and the PodSecurityPolicy admission controller will be removed.
|
||||
|
||||
PodSecurityPolicy replacements are still under discussion, but current use can be migrated to
|
||||
[3rd-party admission webhooks](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/) now.
|
||||
[3rd-party admission webhooks](/docs/reference/access-authn-authz/extensible-admission-controllers/) now.
|
||||
|
||||
#### RuntimeClass {#runtimeclass-v125}
|
||||
|
||||
|
||||
@@ -245,7 +245,7 @@ field tags.
|
||||
|
||||
### Compatibility across topology changes
|
||||
|
||||
On rare occurences, a CRD or built-in type author may want to change the
|
||||
On rare occurrences, a CRD or built-in type author may want to change the
|
||||
specific topology of a field in their resource without incrementing its
|
||||
version. Changing the topology of types, by upgrading the cluster or
|
||||
updating the CRD, has different consequences when updating existing
|
||||
@@ -253,7 +253,7 @@ objects. There are two categories of changes: when a field goes from
|
||||
`map`/`set`/`granular` to `atomic` and the other way around.
|
||||
|
||||
When the `listType`, `mapType`, or `structType` changes from
|
||||
`map`/`set`/`granular` to `atomic`, the whole list, map or struct of
|
||||
`map`/`set`/`granular` to `atomic`, the whole list, map, or struct of
|
||||
existing objects will end-up being owned by actors who owned an element
|
||||
of these types. This means that any further change to these objects
|
||||
would cause a conflict.
|
||||
@@ -310,7 +310,7 @@ simplify the update logic of your controller. The main differences with a
|
||||
read-modify-write and/or patch are the following:
|
||||
|
||||
* the applied object must contain all the fields that the controller cares about.
|
||||
* there are no way to remove fields that haven't been applied by the controller
|
||||
* there is no way to remove fields that haven't been applied by the controller
|
||||
before (controller can still send a PATCH/UPDATE for these use-cases).
|
||||
* the object doesn't have to be read beforehand, `resourceVersion` doesn't have
|
||||
to be specified.
|
||||
@@ -473,7 +473,7 @@ have an opinion about.
|
||||
## Clearing ManagedFields
|
||||
|
||||
It is possible to strip all managedFields from an object by overwriting them
|
||||
using `MergePatch`, `StrategicMergePatch`, `JSONPatch` or `Update`, so every
|
||||
using `MergePatch`, `StrategicMergePatch`, `JSONPatch`, or `Update`, so every
|
||||
non-apply operation. This can be done by overwriting the managedFields field
|
||||
with an empty entry. Two examples are:
|
||||
|
||||
|
||||
@@ -3,11 +3,11 @@ reviewers:
|
||||
- brendandburns
|
||||
- erictune
|
||||
- mikedanese
|
||||
no_issue: true
|
||||
title: Getting started
|
||||
main_menu: true
|
||||
weight: 20
|
||||
content_type: concept
|
||||
no_list: true
|
||||
card:
|
||||
name: setup
|
||||
weight: 20
|
||||
@@ -24,16 +24,40 @@ This section lists the different ways to set up and run Kubernetes.
|
||||
When you install Kubernetes, choose an installation type based on: ease of maintenance, security,
|
||||
control, available resources, and expertise required to operate and manage a cluster.
|
||||
|
||||
You can deploy a Kubernetes cluster on a local machine, cloud, on-prem datacenter, or choose a managed Kubernetes cluster. There are also custom solutions across a wide range of cloud providers, or bare metal environments.
|
||||
You can [download Kubernetes](/releases/download/) to deploy a Kubernetes cluster
|
||||
on a local machine, into the cloud, or for your own datacenter.
|
||||
|
||||
If you don't want to manage a Kubernetes cluster yourself, you could pick a managed service, including
|
||||
[certified platforms](/docs/setup/production-environment/turnkey-solutions/).
|
||||
There are also other standardized and custom solutions across a wide range of cloud and
|
||||
bare metal environments.
|
||||
|
||||
<!-- body -->
|
||||
|
||||
## Learning environment
|
||||
|
||||
If you're learning Kubernetes, use the tools supported by the Kubernetes community, or tools in the ecosystem to set up a Kubernetes cluster on a local machine.
|
||||
If you're learning Kubernetes, use the tools supported by the Kubernetes community,
|
||||
or tools in the ecosystem to set up a Kubernetes cluster on a local machine.
|
||||
See [Install tools](/docs/tasks/tools/).
|
||||
|
||||
## Production environment
|
||||
|
||||
When evaluating a solution for a production environment, consider which aspects of operating a Kubernetes cluster (or _abstractions_) you want to manage yourself or offload to a provider.
|
||||
When evaluating a solution for a
|
||||
[production environment](/docs/setup/production-environment/), consider which aspects of
|
||||
operating a Kubernetes cluster (or _abstractions_) you want to manage yourself and which you
|
||||
prefer to hand off to a provider.
|
||||
|
||||
[Kubernetes Partners](https://kubernetes.io/partners/#conformance) includes a list of [Certified Kubernetes](https://github.com/cncf/k8s-conformance/#certified-kubernetes) providers.
|
||||
For a cluster you're managing yourself, the officially supported tool
|
||||
for deploying Kubernetes is [kubeadm](/docs/setup/production-environment/tools/kubeadm/).
|
||||
|
||||
## {{% heading "whatsnext" %}}
|
||||
|
||||
- [Download Kubernetes](/releases/download/)
|
||||
- Download and [install tools](/docs/tasks/tools/) including `kubectl`
|
||||
- Select a [container runtime](/docs/setup/production-environment/container-runtimes/) for your new cluster
|
||||
- Learn about [best practices](/docs/setup/best-practices/) for cluster setup
|
||||
|
||||
Kubernetes is designed for its {{< glossary_tooltip term_id="control-plane" text="control plane" >}} to
|
||||
run on Linux. Within your cluster you can run applications on Linux or other operating systems, including
|
||||
Windows.
|
||||
- Learn to [set up clusters with Windows nodes](/docs/setup/production-environment/windows/)
|
||||
|
||||
@@ -12,7 +12,7 @@ or virtual machines) running Kubernetes agents, managed by the
|
||||
Kubernetes {{< param "version" >}} supports clusters with up to 5000 nodes. More specifically,
|
||||
Kubernetes is designed to accommodate configurations that meet *all* of the following criteria:
|
||||
|
||||
* No more than 100 pods per node
|
||||
* No more than 110 pods per node
|
||||
* No more than 5000 nodes
|
||||
* No more than 150000 total pods
|
||||
* No more than 300000 total containers
|
||||
@@ -66,8 +66,8 @@ When creating a cluster, you can (using custom tooling):
|
||||
* start and configure additional etcd instance
|
||||
* configure the {{< glossary_tooltip term_id="kube-apiserver" text="API server" >}} to use it for storing events
|
||||
|
||||
See [Operating etcd clusters for Kubernetes](https://kubernetes.io/docs/tasks/administer-cluster/configure-upgrade-etcd/) and
|
||||
[Set up a High Availability etcd cluster with kubeadm](docs/setup/production-environment/tools/kubeadm/setup-ha-etcd-with-kubeadm/)
|
||||
See [Operating etcd clusters for Kubernetes](/docs/tasks/administer-cluster/configure-upgrade-etcd/) and
|
||||
[Set up a High Availability etcd cluster with kubeadm](/docs/setup/production-environment/tools/kubeadm/setup-ha-etcd-with-kubeadm/)
|
||||
for details on configuring and managing etcd for a large cluster.
|
||||
|
||||
## Addon resources
|
||||
|
||||
@@ -49,7 +49,7 @@ access cluster resources. You can use role-based access control
|
||||
security mechanisms to make sure that users and workloads can get access to the
|
||||
resources they need, while keeping workloads, and the cluster itself, secure.
|
||||
You can set limits on the resources that users and workloads can access
|
||||
by managing [policies](https://kubernetes.io/docs/concepts/policy/) and
|
||||
by managing [policies](/docs/concepts/policy/) and
|
||||
[container resources](/docs/concepts/configuration/manage-resources-containers/).
|
||||
|
||||
Before building a Kubernetes production environment on your own, consider
|
||||
@@ -286,8 +286,8 @@ and the
|
||||
deployment methods.
|
||||
- Configure user management by determining your
|
||||
[Authentication](/docs/reference/access-authn-authz/authentication/) and
|
||||
[Authorization](docs/reference/access-authn-authz/authorization/) methods.
|
||||
[Authorization](/docs/reference/access-authn-authz/authorization/) methods.
|
||||
- Prepare for application workloads by setting up
|
||||
[resource limits](docs/tasks/administer-cluster/manage-resources/),
|
||||
[resource limits](/docs/tasks/administer-cluster/manage-resources/),
|
||||
[DNS autoscaling](/docs/tasks/administer-cluster/dns-horizontal-autoscaling/)
|
||||
and [service accounts](/docs/reference/access-authn-authz/service-accounts-admin/).
|
||||
|
||||
+4
-4
@@ -23,7 +23,7 @@ of Containers for each.
|
||||
|
||||
- Fetch all Pods in all namespaces using `kubectl get pods --all-namespaces`
|
||||
- Format the output to include only the list of Container image names
|
||||
using `-o jsonpath={..image}`. This will recursively parse out the
|
||||
using `-o jsonpath={.items[*].spec.containers[*].image}`. This will recursively parse out the
|
||||
`image` field from the returned json.
|
||||
- See the [jsonpath reference](/docs/reference/kubectl/jsonpath/)
|
||||
for further information on how to use jsonpath.
|
||||
@@ -33,7 +33,7 @@ of Containers for each.
|
||||
- Use `uniq` to aggregate image counts
|
||||
|
||||
```shell
|
||||
kubectl get pods --all-namespaces -o jsonpath="{..image}" |\
|
||||
kubectl get pods --all-namespaces -o jsonpath="{.items[*].spec.containers[*].image}" |\
|
||||
tr -s '[[:space:]]' '\n' |\
|
||||
sort |\
|
||||
uniq -c
|
||||
@@ -80,7 +80,7 @@ To target only Pods matching a specific label, use the -l flag. The
|
||||
following matches only Pods with labels matching `app=nginx`.
|
||||
|
||||
```shell
|
||||
kubectl get pods --all-namespaces -o=jsonpath="{..image}" -l app=nginx
|
||||
kubectl get pods --all-namespaces -o=jsonpath="{.items[*].spec.containers[*].image}" -l app=nginx
|
||||
```
|
||||
|
||||
## List Container images filtering by Pod namespace
|
||||
@@ -89,7 +89,7 @@ To target only pods in a specific namespace, use the namespace flag. The
|
||||
following matches only Pods in the `kube-system` namespace.
|
||||
|
||||
```shell
|
||||
kubectl get pods --namespace kube-system -o jsonpath="{..image}"
|
||||
kubectl get pods --namespace kube-system -o jsonpath="{.items[*].spec.containers[*].image}"
|
||||
```
|
||||
|
||||
## List Container images using a go-template instead of jsonpath
|
||||
|
||||
@@ -26,7 +26,7 @@ volume is automatically deleted when a user deletes the corresponding
|
||||
PersistentVolumeClaim. This automatic behavior might be inappropriate if the volume
|
||||
contains precious data. In that case, it is more appropriate to use the "Retain"
|
||||
policy. With the "Retain" policy, if a user deletes a PersistentVolumeClaim,
|
||||
the corresponding PersistentVolume is not be deleted. Instead, it is moved to the
|
||||
the corresponding PersistentVolume will not be deleted. Instead, it is moved to the
|
||||
Released phase, where all of its data can be manually recovered.
|
||||
|
||||
## Changing the reclaim policy of a PersistentVolume
|
||||
|
||||
+54
-46
@@ -1,16 +1,17 @@
|
||||
---
|
||||
reviewers:
|
||||
- jszczepkowski
|
||||
title: Set up High-Availability Kubernetes Masters
|
||||
title: Set up a High-Availability Control Plane
|
||||
content_type: task
|
||||
aliases: [ '/docs/tasks/administer-cluster/highly-available-master/' ]
|
||||
---
|
||||
|
||||
<!-- overview -->
|
||||
|
||||
{{< feature-state for_k8s_version="v1.5" state="alpha" >}}
|
||||
|
||||
You can replicate Kubernetes masters in `kube-up` or `kube-down` scripts for Google Compute Engine.
|
||||
This document describes how to use kube-up/down scripts to manage highly available (HA) masters and how HA masters are implemented for use with GCE.
|
||||
You can replicate Kubernetes control plane nodes in `kube-up` or `kube-down` scripts for Google Compute Engine.
|
||||
This document describes how to use kube-up/down scripts to manage a highly available (HA) control plane and how HA control planes are implemented for use with GCE.
|
||||
|
||||
|
||||
|
||||
@@ -28,17 +29,17 @@ This document describes how to use kube-up/down scripts to manage highly availab
|
||||
|
||||
To create a new HA-compatible cluster, you must set the following flags in your `kube-up` script:
|
||||
|
||||
* `MULTIZONE=true` - to prevent removal of master replicas kubelets from zones different than server's default zone.
|
||||
Required if you want to run master replicas in different zones, which is recommended.
|
||||
* `MULTIZONE=true` - to prevent removal of control plane kubelets from zones different than server's default zone.
|
||||
Required if you want to run control plane nodes in different zones, which is recommended.
|
||||
|
||||
* `ENABLE_ETCD_QUORUM_READ=true` - to ensure that reads from all API servers will return most up-to-date data.
|
||||
If true, reads will be directed to leader etcd replica.
|
||||
Setting this value to true is optional: reads will be more reliable but will also be slower.
|
||||
|
||||
Optionally, you can specify a GCE zone where the first master replica is to be created.
|
||||
Optionally, you can specify a GCE zone where the first control plane node is to be created.
|
||||
Set the following flag:
|
||||
|
||||
* `KUBE_GCE_ZONE=zone` - zone where the first master replica will run.
|
||||
* `KUBE_GCE_ZONE=zone` - zone where the first control plane node will run.
|
||||
|
||||
The following sample command sets up a HA-compatible cluster in the GCE zone europe-west1-b:
|
||||
|
||||
@@ -46,50 +47,52 @@ The following sample command sets up a HA-compatible cluster in the GCE zone eur
|
||||
MULTIZONE=true KUBE_GCE_ZONE=europe-west1-b ENABLE_ETCD_QUORUM_READS=true ./cluster/kube-up.sh
|
||||
```
|
||||
|
||||
Note that the commands above create a cluster with one master;
|
||||
however, you can add new master replicas to the cluster with subsequent commands.
|
||||
Note that the commands above create a cluster with one control plane node;
|
||||
however, you can add new control plane nodes to the cluster with subsequent commands.
|
||||
|
||||
## Adding a new master replica
|
||||
## Adding a new control plane node
|
||||
|
||||
After you have created an HA-compatible cluster, you can add master replicas to it.
|
||||
You add master replicas by using a `kube-up` script with the following flags:
|
||||
After you have created an HA-compatible cluster, you can add control plane nodes to it.
|
||||
You add control plane nodes by using a `kube-up` script with the following flags:
|
||||
|
||||
* `KUBE_REPLICATE_EXISTING_MASTER=true` - to create a replica of an existing
|
||||
master.
|
||||
* `KUBE_REPLICATE_EXISTING_MASTER=true` - to create a replica of an existing control plane
|
||||
node.
|
||||
|
||||
* `KUBE_GCE_ZONE=zone` - zone where the master replica will run.
|
||||
Must be in the same region as other replicas' zones.
|
||||
* `KUBE_GCE_ZONE=zone` - zone where the control plane node will run.
|
||||
Must be in the same region as other control plane nodes' zones.
|
||||
|
||||
You don't need to set the `MULTIZONE` or `ENABLE_ETCD_QUORUM_READS` flags,
|
||||
as those are inherited from when you started your HA-compatible cluster.
|
||||
|
||||
The following sample command replicates the master on an existing HA-compatible cluster:
|
||||
The following sample command replicates the control plane node on an existing
|
||||
HA-compatible cluster:
|
||||
|
||||
```shell
|
||||
KUBE_GCE_ZONE=europe-west1-c KUBE_REPLICATE_EXISTING_MASTER=true ./cluster/kube-up.sh
|
||||
```
|
||||
|
||||
## Removing a master replica
|
||||
## Removing a control plane node
|
||||
|
||||
You can remove a master replica from an HA cluster by using a `kube-down` script with the following flags:
|
||||
You can remove a control plane node from an HA cluster by using a `kube-down` script with the following flags:
|
||||
|
||||
* `KUBE_DELETE_NODES=false` - to restrain deletion of kubelets.
|
||||
|
||||
* `KUBE_GCE_ZONE=zone` - the zone from where master replica will be removed.
|
||||
* `KUBE_GCE_ZONE=zone` - the zone from where the control plane node will be removed.
|
||||
|
||||
* `KUBE_REPLICA_NAME=replica_name` - (optional) the name of master replica to remove.
|
||||
If empty: any replica from the given zone will be removed.
|
||||
* `KUBE_REPLICA_NAME=replica_name` - (optional) the name of control plane node to
|
||||
remove. If empty: any replica from the given zone will be removed.
|
||||
|
||||
The following sample command removes a master replica from an existing HA cluster:
|
||||
The following sample command removes a control plane node from an existing HA cluster:
|
||||
|
||||
```shell
|
||||
KUBE_DELETE_NODES=false KUBE_GCE_ZONE=europe-west1-c ./cluster/kube-down.sh
|
||||
```
|
||||
|
||||
## Handling master replica failures
|
||||
## Handling control plane node failures
|
||||
|
||||
If one of the master replicas in your HA cluster fails,
|
||||
the best practice is to remove the replica from your cluster and add a new replica in the same zone.
|
||||
If one of the control plane nodes in your HA cluster fails,
|
||||
the best practice is to remove the node from your cluster and add a new control plane
|
||||
node in the same zone.
|
||||
The following sample commands demonstrate this process:
|
||||
|
||||
1. Remove the broken replica:
|
||||
@@ -98,26 +101,31 @@ The following sample commands demonstrate this process:
|
||||
KUBE_DELETE_NODES=false KUBE_GCE_ZONE=replica_zone KUBE_REPLICA_NAME=replica_name ./cluster/kube-down.sh
|
||||
```
|
||||
|
||||
<ol start="2"><li>Add a new replica in place of the old one:</li></ol>
|
||||
<ol start="2"><li>Add a new node in place of the old one:</li></ol>
|
||||
|
||||
```shell
|
||||
KUBE_GCE_ZONE=replica-zone KUBE_REPLICATE_EXISTING_MASTER=true ./cluster/kube-up.sh
|
||||
```
|
||||
|
||||
## Best practices for replicating masters for HA clusters
|
||||
## Best practices for replicating control plane nodes for HA clusters
|
||||
|
||||
* Try to place master replicas in different zones. During a zone failure, all masters placed inside the zone will fail.
|
||||
* Try to place control plane nodes in different zones. During a zone failure, all
|
||||
control plane nodes placed inside the zone will fail.
|
||||
To survive zone failure, also place nodes in multiple zones
|
||||
(see [multiple-zones](/docs/setup/best-practices/multiple-zones/) for details).
|
||||
|
||||
* Do not use a cluster with two master replicas. Consensus on a two-replica cluster requires both replicas running when changing persistent state.
|
||||
As a result, both replicas are needed and a failure of any replica turns cluster into majority failure state.
|
||||
A two-replica cluster is thus inferior, in terms of HA, to a single replica cluster.
|
||||
* Do not use a cluster with two control plane nodes. Consensus on a two-node
|
||||
control plane requires both nodes running when changing persistent state.
|
||||
As a result, both nodes are needed and a failure of any node turns the cluster
|
||||
into majority failure state.
|
||||
A two-node control plane is thus inferior, in terms of HA, to a cluster with
|
||||
one control plane node.
|
||||
|
||||
* When you add a master replica, cluster state (etcd) is copied to a new instance.
|
||||
* When you add a control plane node, cluster state (etcd) is copied to a new instance.
|
||||
If the cluster is large, it may take a long time to duplicate its state.
|
||||
This operation may be sped up by migrating etcd data directory, as described [here](https://coreos.com/etcd/docs/latest/admin_guide.html#member-migration)
|
||||
(we are considering adding support for etcd data dir migration in future).
|
||||
This operation may be sped up by migrating the etcd data directory, as described in
|
||||
the [etcd administration guide](https://etcd.io/docs/v2.3/admin_guide/#member-migration)
|
||||
(we are considering adding support for etcd data dir migration in the future).
|
||||
|
||||
|
||||
|
||||
@@ -129,7 +137,7 @@ This operation may be sped up by migrating etcd data directory, as described [he
|
||||
|
||||
### Overview
|
||||
|
||||
Each of master replicas will run the following components in the following mode:
|
||||
Each of the control plane nodes will run the following components in the following mode:
|
||||
|
||||
* etcd instance: all instances will be clustered together using consensus;
|
||||
|
||||
@@ -143,9 +151,9 @@ In addition, there will be a load balancer in front of API servers that will rou
|
||||
|
||||
### Load balancing
|
||||
|
||||
When starting the second master replica, a load balancer containing the two replicas will be created
|
||||
When starting the second control plane node, a load balancer containing the two replicas will be created
|
||||
and the IP address of the first replica will be promoted to IP address of load balancer.
|
||||
Similarly, after removal of the penultimate master replica, the load balancer will be removed and its IP address will be assigned to the last remaining replica.
|
||||
Similarly, after removal of the penultimate control plane node, the load balancer will be removed and its IP address will be assigned to the last remaining replica.
|
||||
Please note that creation and removal of load balancer are complex operations and it may take some time (~20 minutes) for them to propagate.
|
||||
|
||||
### Master service & kubelets
|
||||
@@ -153,17 +161,17 @@ Please note that creation and removal of load balancer are complex operations an
|
||||
Instead of trying to keep an up-to-date list of Kubernetes apiserver in the Kubernetes service,
|
||||
the system directs all traffic to the external IP:
|
||||
|
||||
* in one master cluster the IP points to the single master,
|
||||
* in case of a single node control plane, the IP points to the control plane node,
|
||||
|
||||
* in multi-master cluster the IP points to the load balancer in-front of the masters.
|
||||
* in case of an HA control plane, the IP points to the load balancer in-front of the masters.
|
||||
|
||||
Similarly, the external IP will be used by kubelets to communicate with master.
|
||||
Similarly, the external IP will be used by kubelets to communicate with the control plane.
|
||||
|
||||
### Master certificates
|
||||
### Control plane node certificates
|
||||
|
||||
Kubernetes generates Master TLS certificates for the external public IP and local IP for each replica.
|
||||
There are no certificates for the ephemeral public IP for replicas;
|
||||
to access a replica via its ephemeral public IP, you must skip TLS verification.
|
||||
Kubernetes generates TLS certificates for the external public IP and local IP for each control plane node.
|
||||
There are no certificates for the ephemeral public IP for control plane nodes;
|
||||
to access a control plane node via its ephemeral public IP, you must skip TLS verification.
|
||||
|
||||
### Clustering etcd
|
||||
|
||||
@@ -172,7 +180,7 @@ To make such deployment secure, communication between etcd instances is authoriz
|
||||
|
||||
### API server identity
|
||||
|
||||
{{< feature-state state="alpha" for_k8s_version="v1.20" >}}
|
||||
{{< feature-state state="alpha" for_k8s_version="v1.20" >}}
|
||||
|
||||
The API Server Identity feature is controlled by a
|
||||
[feature gate](/docs/reference/command-line-tools-reference/feature-gates/)
|
||||
@@ -188,7 +188,7 @@ To install a specific version of containerD specify the version with -ContainerD
|
||||
|
||||
```powershell
|
||||
# Example
|
||||
.\Install-Containerd.ps1 -ContainerDVersion v1.4.1
|
||||
.\Install-Containerd.ps1 -ContainerDVersion 1.4.1
|
||||
```
|
||||
|
||||
{{< /note >}}
|
||||
|
||||
@@ -239,7 +239,7 @@ The field `serverTLSBootstrap: true` will enable the bootstrap of kubelet servin
|
||||
certificates by requesting them from the `certificates.k8s.io` API. One known limitation
|
||||
is that the CSRs (Certificate Signing Requests) for these certificates cannot be automatically
|
||||
approved by the default signer in the kube-controller-manager -
|
||||
[`kubernetes.io/kubelet-serving`](https://kubernetes.io/docs/reference/access-authn-authz/certificate-signing-requests/#kubernetes-signers).
|
||||
[`kubernetes.io/kubelet-serving`](/docs/reference/access-authn-authz/certificate-signing-requests/#kubernetes-signers).
|
||||
This will require action from the user or a third party controller.
|
||||
|
||||
These CSRs can be viewed using:
|
||||
|
||||
@@ -23,7 +23,7 @@ You need to have a Kubernetes cluster and the kubectl command-line tool must be
|
||||
|
||||
## Set the Username for a Pod
|
||||
|
||||
To specify the username with which to execute the Pod's container processes, include the `securityContext` field ([PodSecurityContext](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#podsecuritycontext-v1-core) in the Pod specification, and within it, the `windowsOptions` ([WindowsSecurityContextOptions](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#windowssecuritycontextoptions-v1-core) field containing the `runAsUserName` field.
|
||||
To specify the username with which to execute the Pod's container processes, include the `securityContext` field ([PodSecurityContext](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#podsecuritycontext-v1-core)) in the Pod specification, and within it, the `windowsOptions` ([WindowsSecurityContextOptions](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#windowssecuritycontextoptions-v1-core)) field containing the `runAsUserName` field.
|
||||
|
||||
The Windows security context options that you specify for a Pod apply to all Containers and init Containers in the Pod.
|
||||
|
||||
@@ -63,7 +63,7 @@ ContainerUser
|
||||
|
||||
## Set the Username for a Container
|
||||
|
||||
To specify the username with which to execute a Container's processes, include the `securityContext` field ([SecurityContext](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#securitycontext-v1-core)) in the Container manifest, and within it, the `windowsOptions` ([WindowsSecurityContextOptions](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#windowssecuritycontextoptions-v1-core) field containing the `runAsUserName` field.
|
||||
To specify the username with which to execute a Container's processes, include the `securityContext` field ([SecurityContext](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#securitycontext-v1-core)) in the Container manifest, and within it, the `windowsOptions` ([WindowsSecurityContextOptions](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#windowssecuritycontextoptions-v1-core)) field containing the `runAsUserName` field.
|
||||
|
||||
The Windows security context options that you specify for a Container apply only to that individual Container, and they override the settings made at the Pod level.
|
||||
|
||||
|
||||
@@ -167,8 +167,8 @@ The output is similar to this:
|
||||
Name: build-robot-secret
|
||||
Namespace: default
|
||||
Labels: <none>
|
||||
Annotations: kubernetes.io/service-account.name=build-robot
|
||||
kubernetes.io/service-account.uid=da68f9c6-9d26-11e7-b84e-002dc52800da
|
||||
Annotations: kubernetes.io/service-account.name: build-robot
|
||||
kubernetes.io/service-account.uid: da68f9c6-9d26-11e7-b84e-002dc52800da
|
||||
|
||||
Type: kubernetes.io/service-account-token
|
||||
|
||||
|
||||
@@ -54,7 +54,7 @@ If you use a Docker credentials store, you won't see that `auth` entry but a `cr
|
||||
|
||||
## Create a Secret based on existing Docker credentials {#registry-secret-existing-credentials}
|
||||
|
||||
A Kubernetes cluster uses the Secret of `docker-registry` type to authenticate with
|
||||
A Kubernetes cluster uses the Secret of `kubernetes.io/dockerconfigjson` type to authenticate with
|
||||
a container registry to pull a private image.
|
||||
|
||||
If you already ran `docker login`, you can copy that credential into Kubernetes:
|
||||
|
||||
@@ -45,8 +45,12 @@ kubectl create namespace qos-example
|
||||
|
||||
For a Pod to be given a QoS class of Guaranteed:
|
||||
|
||||
* Every Container, including init containers, in the Pod must have a memory limit and a memory request, and they must be the same.
|
||||
* Every Container, including init containers, in the Pod must have a CPU limit and a CPU request, and they must be the same.
|
||||
* Every Container in the Pod must have a memory limit and a memory request.
|
||||
* For every Container in the Pod, the memory limit must equal the memory request.
|
||||
* Every Container in the Pod must have a CPU limit and a CPU request.
|
||||
* For every Container in the Pod, the CPU limit must equal the CPU request.
|
||||
|
||||
These restrictions apply to init containers and app containers equally.
|
||||
|
||||
Here is the configuration file for a Pod that has one Container. The Container has a memory limit and a
|
||||
memory request, both equal to 200 MiB. The Container has a CPU limit and a CPU request, both equal to 700 milliCPU:
|
||||
@@ -272,5 +276,3 @@ kubectl delete namespace qos-example
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -41,7 +41,7 @@ the container starts.
|
||||
|
||||
kubectl apply -f https://k8s.io/examples/debug/termination.yaml
|
||||
|
||||
In the YAML file, in the `cmd` and `args` fields, you can see that the
|
||||
In the YAML file, in the `command` and `args` fields, you can see that the
|
||||
container sleeps for 10 seconds and then writes "Sleep expired" to
|
||||
the `/dev/termination-log` file. After the container writes
|
||||
the "Sleep expired" message, it terminates.
|
||||
|
||||
@@ -1,371 +0,0 @@
|
||||
---
|
||||
reviewers:
|
||||
- piosz
|
||||
- x13n
|
||||
title: Logging Using Stackdriver
|
||||
content_type: concept
|
||||
---
|
||||
|
||||
<!-- overview -->
|
||||
|
||||
Before reading this page, it's highly recommended to familiarize yourself
|
||||
with the [overview of logging in Kubernetes](/docs/concepts/cluster-administration/logging).
|
||||
|
||||
{{< note >}}
|
||||
By default, Stackdriver logging collects only your container's standard output and
|
||||
standard error streams. To collect any logs your application writes to a file (for example),
|
||||
see the [sidecar approach](/docs/concepts/cluster-administration/logging#sidecar-container-with-a-logging-agent)
|
||||
in the Kubernetes logging overview.
|
||||
{{< /note >}}
|
||||
|
||||
|
||||
|
||||
|
||||
<!-- body -->
|
||||
|
||||
## Deploying
|
||||
|
||||
To ingest logs, you must deploy the Stackdriver Logging agent to each node in your cluster.
|
||||
The agent is a configured `fluentd` instance, where the configuration is stored in a `ConfigMap`
|
||||
and the instances are managed using a Kubernetes `DaemonSet`. The actual deployment of the
|
||||
`ConfigMap` and `DaemonSet` for your cluster depends on your individual cluster setup.
|
||||
|
||||
### Deploying to a new cluster
|
||||
|
||||
#### Google Kubernetes Engine
|
||||
|
||||
Stackdriver is the default logging solution for clusters deployed on Google Kubernetes Engine.
|
||||
Stackdriver Logging is deployed to a new cluster by default unless you explicitly opt-out.
|
||||
|
||||
#### Other platforms
|
||||
|
||||
To deploy Stackdriver Logging on a *new* cluster that you're
|
||||
creating using `kube-up.sh`, do the following:
|
||||
|
||||
1. Set the `KUBE_LOGGING_DESTINATION` environment variable to `gcp`.
|
||||
1. **If not running on GCE**, include the `beta.kubernetes.io/fluentd-ds-ready=true`
|
||||
in the `KUBE_NODE_LABELS` variable.
|
||||
|
||||
Once your cluster has started, each node should be running the Stackdriver Logging agent.
|
||||
The `DaemonSet` and `ConfigMap` are configured as addons. If you're not using `kube-up.sh`,
|
||||
consider starting a cluster without a pre-configured logging solution and then deploying
|
||||
Stackdriver Logging agents to the running cluster.
|
||||
|
||||
{{< warning >}}
|
||||
The Stackdriver logging daemon has known issues on platforms other
|
||||
than Google Kubernetes Engine. Proceed at your own risk.
|
||||
{{< /warning >}}
|
||||
|
||||
### Deploying to an existing cluster
|
||||
|
||||
1. Apply a label on each node, if not already present.
|
||||
|
||||
The Stackdriver Logging agent deployment uses node labels to determine to which nodes
|
||||
it should be allocated. These labels were introduced to distinguish nodes with the
|
||||
Kubernetes version 1.6 or higher. If the cluster was created with Stackdriver Logging
|
||||
configured and node has version 1.5.X or lower, it will have fluentd as static pod. Node
|
||||
cannot have more than one instance of fluentd, therefore only apply labels to the nodes
|
||||
that don't have fluentd pod allocated already. You can ensure that your node is labelled
|
||||
properly by running `kubectl describe` as follows:
|
||||
|
||||
```
|
||||
kubectl describe node $NODE_NAME
|
||||
```
|
||||
|
||||
The output should be similar to this:
|
||||
|
||||
```
|
||||
Name: NODE_NAME
|
||||
Role:
|
||||
Labels: beta.kubernetes.io/fluentd-ds-ready=true
|
||||
...
|
||||
```
|
||||
|
||||
Ensure that the output contains the label `beta.kubernetes.io/fluentd-ds-ready=true`. If it
|
||||
is not present, you can add it using the `kubectl label` command as follows:
|
||||
|
||||
```
|
||||
kubectl label node $NODE_NAME beta.kubernetes.io/fluentd-ds-ready=true
|
||||
```
|
||||
|
||||
{{< note >}}
|
||||
If a node fails and has to be recreated, you must re-apply the label to
|
||||
the recreated node. To make this easier, you can use Kubelet's command-line parameter
|
||||
for applying node labels in your node startup script.
|
||||
{{< /note >}}
|
||||
|
||||
1. Deploy a `ConfigMap` with the logging agent configuration by running the following command:
|
||||
|
||||
```
|
||||
kubectl apply -f https://k8s.io/examples/debug/fluentd-gcp-configmap.yaml
|
||||
```
|
||||
|
||||
The command creates the `ConfigMap` in the `default` namespace. You can download the file
|
||||
manually and change it before creating the `ConfigMap` object.
|
||||
|
||||
1. Deploy the logging agent `DaemonSet` by running the following command:
|
||||
|
||||
```
|
||||
kubectl apply -f https://k8s.io/examples/debug/fluentd-gcp-ds.yaml
|
||||
```
|
||||
|
||||
You can download and edit this file before using it as well.
|
||||
|
||||
## Verifying your Logging Agent Deployment
|
||||
|
||||
After Stackdriver `DaemonSet` is deployed, you can discover logging agent deployment status
|
||||
by running the following command:
|
||||
|
||||
```shell
|
||||
kubectl get ds --all-namespaces
|
||||
```
|
||||
|
||||
If you have 3 nodes in the cluster, the output should looks similar to this:
|
||||
|
||||
```
|
||||
NAMESPACE NAME DESIRED CURRENT READY NODE-SELECTOR AGE
|
||||
...
|
||||
default fluentd-gcp-v2.0 3 3 3 beta.kubernetes.io/fluentd-ds-ready=true 5m
|
||||
...
|
||||
```
|
||||
|
||||
To understand how logging with Stackdriver works, consider the following
|
||||
synthetic log generator pod specification [counter-pod.yaml](/examples/debug/counter-pod.yaml):
|
||||
|
||||
{{< codenew file="debug/counter-pod.yaml" >}}
|
||||
|
||||
This pod specification has one container that runs a bash script
|
||||
that writes out the value of a counter and the datetime once per
|
||||
second, and runs indefinitely. Let's create this pod in the default namespace.
|
||||
|
||||
```shell
|
||||
kubectl apply -f https://k8s.io/examples/debug/counter-pod.yaml
|
||||
```
|
||||
|
||||
You can observe the running pod:
|
||||
|
||||
```shell
|
||||
kubectl get pods
|
||||
```
|
||||
```
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
counter 1/1 Running 0 5m
|
||||
```
|
||||
|
||||
For a short period of time you can observe the 'Pending' pod status, because the kubelet
|
||||
has to download the container image first. When the pod status changes to `Running`
|
||||
you can use the `kubectl logs` command to view the output of this counter pod.
|
||||
|
||||
```shell
|
||||
kubectl logs counter
|
||||
```
|
||||
```
|
||||
0: Mon Jan 1 00:00:00 UTC 2001
|
||||
1: Mon Jan 1 00:00:01 UTC 2001
|
||||
2: Mon Jan 1 00:00:02 UTC 2001
|
||||
...
|
||||
```
|
||||
|
||||
As described in the logging overview, this command fetches log entries
|
||||
from the container log file. If the container is killed and then restarted by
|
||||
Kubernetes, you can still access logs from the previous container. However,
|
||||
if the pod is evicted from the node, log files are lost. Let's demonstrate this
|
||||
by deleting the currently running counter container:
|
||||
|
||||
```shell
|
||||
kubectl delete pod counter
|
||||
```
|
||||
```
|
||||
pod "counter" deleted
|
||||
```
|
||||
|
||||
and then recreating it:
|
||||
|
||||
```shell
|
||||
kubectl create -f https://k8s.io/examples/debug/counter-pod.yaml
|
||||
```
|
||||
```
|
||||
pod/counter created
|
||||
```
|
||||
|
||||
After some time, you can access logs from the counter pod again:
|
||||
|
||||
```shell
|
||||
kubectl logs counter
|
||||
```
|
||||
```
|
||||
0: Mon Jan 1 00:01:00 UTC 2001
|
||||
1: Mon Jan 1 00:01:01 UTC 2001
|
||||
2: Mon Jan 1 00:01:02 UTC 2001
|
||||
...
|
||||
```
|
||||
|
||||
As expected, only recent log lines are present. However, for a real-world
|
||||
application you will likely want to be able to access logs from all containers,
|
||||
especially for the debug purposes. This is exactly when the previously enabled
|
||||
Stackdriver Logging can help.
|
||||
|
||||
## Viewing logs
|
||||
|
||||
Stackdriver Logging agent attaches metadata to each log entry, for you to use later
|
||||
in queries to select only the messages you're interested in: for example,
|
||||
the messages from a particular pod.
|
||||
|
||||
The most important pieces of metadata are the resource type and log name.
|
||||
The resource type of a container log is `container`, which is named
|
||||
`GKE Containers` in the UI (even if the Kubernetes cluster is not on Google Kubernetes Engine).
|
||||
The log name is the name of the container, so that if you have a pod with
|
||||
two containers, named `container_1` and `container_2` in the spec, their logs
|
||||
will have log names `container_1` and `container_2` respectively.
|
||||
|
||||
System components have resource type `compute`, which is named
|
||||
`GCE VM Instance` in the interface. Log names for system components are fixed.
|
||||
For a Google Kubernetes Engine node, every log entry from a system component has one of the following
|
||||
log names:
|
||||
|
||||
* docker
|
||||
* kubelet
|
||||
* kube-proxy
|
||||
|
||||
You can learn more about viewing logs on [the dedicated Stackdriver page](https://cloud.google.com/logging/docs/view/logs_viewer).
|
||||
|
||||
One of the possible ways to view logs is using the
|
||||
[`gcloud logging`](https://cloud.google.com/logging/docs/api/gcloud-logging)
|
||||
command line interface from the [Google Cloud SDK](https://cloud.google.com/sdk/).
|
||||
It uses Stackdriver Logging [filtering syntax](https://cloud.google.com/logging/docs/view/advanced_filters)
|
||||
to query specific logs. For example, you can run the following command:
|
||||
|
||||
```none
|
||||
gcloud beta logging read 'logName="projects/$YOUR_PROJECT_ID/logs/count"' --format json | jq '.[].textPayload'
|
||||
```
|
||||
```
|
||||
...
|
||||
"2: Mon Jan 1 00:01:02 UTC 2001\n"
|
||||
"1: Mon Jan 1 00:01:01 UTC 2001\n"
|
||||
"0: Mon Jan 1 00:01:00 UTC 2001\n"
|
||||
...
|
||||
"2: Mon Jan 1 00:00:02 UTC 2001\n"
|
||||
"1: Mon Jan 1 00:00:01 UTC 2001\n"
|
||||
"0: Mon Jan 1 00:00:00 UTC 2001\n"
|
||||
```
|
||||
|
||||
As you can see, it outputs messages for the count container from both
|
||||
the first and second runs, despite the fact that the kubelet already deleted
|
||||
the logs for the first container.
|
||||
|
||||
### Exporting logs
|
||||
|
||||
You can export logs to [Google Cloud Storage](https://cloud.google.com/storage/)
|
||||
or to [BigQuery](https://cloud.google.com/bigquery/) to run further
|
||||
analysis. Stackdriver Logging offers the concept of sinks, where you can
|
||||
specify the destination of log entries. More information is available on
|
||||
the Stackdriver [Exporting Logs page](https://cloud.google.com/logging/docs/export/configure_export_v2).
|
||||
|
||||
## Configuring Stackdriver Logging Agents
|
||||
|
||||
Sometimes the default installation of Stackdriver Logging may not suit your needs, for example:
|
||||
|
||||
* You may want to add more resources because default performance doesn't suit your needs.
|
||||
* You may want to introduce additional parsing to extract more metadata from your log messages,
|
||||
like severity or source code reference.
|
||||
* You may want to send logs not only to Stackdriver or send it to Stackdriver only partially.
|
||||
|
||||
In this case you need to be able to change the parameters of `DaemonSet` and `ConfigMap`.
|
||||
|
||||
### Prerequisites
|
||||
|
||||
If you're using GKE and Stackdriver Logging is enabled in your cluster, you
|
||||
cannot change its configuration, because it's managed and supported by GKE.
|
||||
However, you can disable the default integration and deploy your own.
|
||||
|
||||
{{< note >}}
|
||||
You will have to support and maintain a newly deployed configuration
|
||||
yourself: update the image and configuration, adjust the resources and so on.
|
||||
{{< /note >}}
|
||||
|
||||
To disable the default logging integration, use the following command:
|
||||
|
||||
```
|
||||
gcloud beta container clusters update --logging-service=none CLUSTER
|
||||
```
|
||||
|
||||
You can find notes on how to then install Stackdriver Logging agents into
|
||||
a running cluster in the [Deploying section](#deploying).
|
||||
|
||||
### Changing `DaemonSet` parameters
|
||||
|
||||
When you have the Stackdriver Logging `DaemonSet` in your cluster, you can modify the
|
||||
`template` field in its spec. The DaemonSet controller manages the pods for you.
|
||||
For example, assume you've installed the Stackdriver Logging as described above. Now you want to
|
||||
change the memory limit to give fluentd more memory to safely process more logs.
|
||||
|
||||
Get the spec of `DaemonSet` running in your cluster:
|
||||
|
||||
```shell
|
||||
kubectl get ds fluentd-gcp-v2.0 --namespace kube-system -o yaml > fluentd-gcp-ds.yaml
|
||||
```
|
||||
|
||||
Then edit resource requirements in the spec file and update the `DaemonSet` object
|
||||
in the apiserver using the following command:
|
||||
|
||||
```shell
|
||||
kubectl replace -f fluentd-gcp-ds.yaml
|
||||
```
|
||||
|
||||
After some time, Stackdriver Logging agent pods will be restarted with the new configuration.
|
||||
|
||||
### Changing fluentd parameters
|
||||
|
||||
Fluentd configuration is stored in the `ConfigMap` object. It is effectively a set of configuration
|
||||
files that are merged together. You can learn about fluentd configuration on the
|
||||
[official site](https://docs.fluentd.org).
|
||||
|
||||
Imagine you want to add a new parsing logic to the configuration, so that fluentd can understand
|
||||
default Python logging format. An appropriate fluentd filter looks similar to this:
|
||||
|
||||
```
|
||||
<filter reform.**>
|
||||
type parser
|
||||
format /^(?<severity>\w):(?<logger_name>\w):(?<log>.*)/
|
||||
reserve_data true
|
||||
suppress_parse_error_log true
|
||||
key_name log
|
||||
</filter>
|
||||
```
|
||||
|
||||
Now you have to put it in the configuration and make Stackdriver Logging agents pick it up.
|
||||
Get the current version of the Stackdriver Logging `ConfigMap` in your cluster
|
||||
by running the following command:
|
||||
|
||||
```shell
|
||||
kubectl get cm fluentd-gcp-config --namespace kube-system -o yaml > fluentd-gcp-configmap.yaml
|
||||
```
|
||||
|
||||
Then in the value of the key `containers.input.conf` insert a new filter right after
|
||||
the `source` section.
|
||||
|
||||
{{< note >}}
|
||||
Order is important.
|
||||
{{< /note >}}
|
||||
|
||||
Updating `ConfigMap` in the apiserver is more complicated than updating `DaemonSet`. It's better
|
||||
to consider `ConfigMap` to be immutable. Then, in order to update the configuration, you should
|
||||
create `ConfigMap` with a new name and then change `DaemonSet` to point to it
|
||||
using [guide above](#changing-daemonset-parameters).
|
||||
|
||||
### Adding fluentd plugins
|
||||
|
||||
Fluentd is written in Ruby and allows to extend its capabilities using
|
||||
[plugins](https://www.fluentd.org/plugins). If you want to use a plugin, which is not included
|
||||
in the default Stackdriver Logging container image, you have to build a custom image. Imagine
|
||||
you want to add Kafka sink for messages from a particular container for additional processing.
|
||||
You can re-use the default [container image sources](https://git.k8s.io/contrib/fluentd/fluentd-gcp-image)
|
||||
with minor changes:
|
||||
|
||||
* Change Makefile to point to your container repository, for example `PREFIX=gcr.io/<your-project-id>`.
|
||||
* Add your dependency to the Gemfile, for example `gem 'fluent-plugin-kafka'`.
|
||||
|
||||
Then run `make build push` from this directory. After updating `DaemonSet` to pick up the
|
||||
new image, you can use the plugin you installed in the fluentd configuration.
|
||||
|
||||
|
||||
+1
-1
@@ -80,7 +80,7 @@ Removing an old version:
|
||||
If this occurs, switch back to using `served:true` on the old version, migrate the
|
||||
remaining clients to the new version and repeat this step.
|
||||
1. Ensure the [upgrade of existing objects to the new stored version](#upgrade-existing-objects-to-a-new-stored-version) step has been completed.
|
||||
1. Verify that the `stored` is set to `true` for the new version in the `spec.versions` list in the CustomResourceDefinition.
|
||||
1. Verify that the `storage` is set to `true` for the new version in the `spec.versions` list in the CustomResourceDefinition.
|
||||
1. Verify that the old version is no longer listed in the CustomResourceDefinition `status.storedVersions`.
|
||||
1. Remove the old version from the CustomResourceDefinition `spec.versions` list.
|
||||
1. Drop conversion support for the old version in conversion webhooks.
|
||||
|
||||
@@ -180,7 +180,7 @@ spec:
|
||||
containers:
|
||||
- name: app
|
||||
image: my-app
|
||||
volumeMount:
|
||||
volumeMounts:
|
||||
- name: config
|
||||
mountPath: /config
|
||||
volumes:
|
||||
@@ -234,7 +234,7 @@ spec:
|
||||
containers:
|
||||
- image: my-app
|
||||
name: app
|
||||
volumeMount:
|
||||
volumeMounts:
|
||||
- mountPath: /config
|
||||
name: config
|
||||
volumes:
|
||||
@@ -327,7 +327,7 @@ spec:
|
||||
containers:
|
||||
- name: app
|
||||
image: my-app
|
||||
volumeMount:
|
||||
volumeMounts:
|
||||
- name: password
|
||||
mountPath: /secrets
|
||||
volumes:
|
||||
|
||||
@@ -1,21 +0,0 @@
|
||||
---
|
||||
title: "gcloud kubectl install"
|
||||
description: "How to install kubectl with gcloud snippet for inclusion in each OS-specific tab."
|
||||
headless: true
|
||||
---
|
||||
|
||||
You can install kubectl as part of the Google Cloud SDK.
|
||||
|
||||
1. Install the [Google Cloud SDK](https://cloud.google.com/sdk/).
|
||||
|
||||
1. Run the `kubectl` installation command:
|
||||
|
||||
```shell
|
||||
gcloud components install kubectl
|
||||
```
|
||||
|
||||
1. Test to ensure the version you installed is up-to-date:
|
||||
|
||||
```shell
|
||||
kubectl version --client
|
||||
```
|
||||
@@ -22,7 +22,6 @@ The following methods exist for installing kubectl on Linux:
|
||||
- [Install kubectl binary with curl on Linux](#install-kubectl-binary-with-curl-on-linux)
|
||||
- [Install using native package management](#install-using-native-package-management)
|
||||
- [Install using other package management](#install-using-other-package-management)
|
||||
- [Install on Linux as part of the Google Cloud SDK](#install-on-linux-as-part-of-the-google-cloud-sdk)
|
||||
|
||||
### Install kubectl binary with curl on Linux
|
||||
|
||||
@@ -168,10 +167,6 @@ kubectl version --client
|
||||
|
||||
{{< /tabs >}}
|
||||
|
||||
### Install on Linux as part of the Google Cloud SDK
|
||||
|
||||
{{< include "included/install-kubectl-gcloud.md" >}}
|
||||
|
||||
## Verify kubectl configuration
|
||||
|
||||
{{< include "included/verify-kubectl.md" >}}
|
||||
|
||||
@@ -22,7 +22,6 @@ The following methods exist for installing kubectl on macOS:
|
||||
- [Install kubectl binary with curl on macOS](#install-kubectl-binary-with-curl-on-macos)
|
||||
- [Install with Homebrew on macOS](#install-with-homebrew-on-macos)
|
||||
- [Install with Macports on macOS](#install-with-macports-on-macos)
|
||||
- [Install on macOS as part of the Google Cloud SDK](#install-on-macos-as-part-of-the-google-cloud-sdk)
|
||||
|
||||
### Install kubectl binary with curl on macOS
|
||||
|
||||
@@ -31,6 +30,7 @@ The following methods exist for installing kubectl on macOS:
|
||||
{{< tabs name="download_binary_macos" >}}
|
||||
{{< tab name="Intel" codelang="bash" >}}
|
||||
curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/darwin/amd64/kubectl"
|
||||
chmod +x kubectl
|
||||
{{< /tab >}}
|
||||
{{< tab name="Apple Silicon" codelang="bash" >}}
|
||||
curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/darwin/arm64/kubectl"
|
||||
@@ -148,11 +148,6 @@ If you are on macOS and using [Macports](https://macports.org/) package manager,
|
||||
kubectl version --client
|
||||
```
|
||||
|
||||
|
||||
### Install on macOS as part of the Google Cloud SDK
|
||||
|
||||
{{< include "included/install-kubectl-gcloud.md" >}}
|
||||
|
||||
## Verify kubectl configuration
|
||||
|
||||
{{< include "included/verify-kubectl.md" >}}
|
||||
|
||||
@@ -21,7 +21,6 @@ The following methods exist for installing kubectl on Windows:
|
||||
|
||||
- [Install kubectl binary with curl on Windows](#install-kubectl-binary-with-curl-on-windows)
|
||||
- [Install on Windows using Chocolatey or Scoop](#install-on-windows-using-chocolatey-or-scoop)
|
||||
- [Install on Windows as part of the Google Cloud SDK](#install-on-windows-as-part-of-the-google-cloud-sdk)
|
||||
|
||||
|
||||
### Install kubectl binary with curl on Windows
|
||||
@@ -127,10 +126,6 @@ If you have installed Docker Desktop before, you may need to place your `PATH` e
|
||||
Edit the config file with a text editor of your choice, such as Notepad.
|
||||
{{< /note >}}
|
||||
|
||||
### Install on Windows as part of the Google Cloud SDK
|
||||
|
||||
{{< include "included/install-kubectl-gcloud.md" >}}
|
||||
|
||||
## Verify kubectl configuration
|
||||
|
||||
{{< include "included/verify-kubectl.md" >}}
|
||||
|
||||
@@ -224,7 +224,7 @@ The minikube tool includes a set of built-in {{< glossary_tooltip text="addons"
|
||||
The output is similar to:
|
||||
|
||||
```
|
||||
metrics-server was successfully enabled
|
||||
The 'metrics-server' addon is enabled
|
||||
```
|
||||
|
||||
3. View the Pod and Service you created:
|
||||
|
||||
@@ -10,10 +10,10 @@ For general information about Kubernetes release cycle, see the
|
||||
|
||||
## Cadence
|
||||
|
||||
Our typical patch release cadence is monthly. It is
|
||||
Our typical patch release cadence is monthly. It is
|
||||
commonly a bit faster (1 to 2 weeks) for the earliest patch releases
|
||||
after a 1.X minor release. Critical bug fixes may cause a more
|
||||
immediate release outside of the normal cadence. We also aim to not make
|
||||
after a 1.X minor release. Critical bug fixes may cause a more
|
||||
immediate release outside of the normal cadence. We also aim to not make
|
||||
releases during major holiday periods.
|
||||
|
||||
## Contact
|
||||
@@ -23,7 +23,7 @@ See the [Release Managers page][release-managers] for full contact details on th
|
||||
Please give us a business day to respond - we may be in a different timezone!
|
||||
|
||||
In between releases the team is looking at incoming cherry pick
|
||||
requests on a weekly basis. The team will get in touch with
|
||||
requests on a weekly basis. The team will get in touch with
|
||||
submitters via GitHub PR, SIG channels in Slack, and direct messages
|
||||
in Slack and [email](mailto:release-managers-private@kubernetes.io)
|
||||
if there are questions on the PR.
|
||||
@@ -34,8 +34,8 @@ Please follow the [cherry pick process][cherry-picks].
|
||||
|
||||
Cherry picks must be merge-ready in GitHub with proper labels (e.g.,
|
||||
`approved`, `lgtm`, `release-note`) and passing CI tests ahead of the
|
||||
cherry pick deadline. This is typically two days before the target
|
||||
release, but may be more. Earlier PR readiness is better, as we
|
||||
cherry pick deadline. This is typically two days before the target
|
||||
release, but may be more. Earlier PR readiness is better, as we
|
||||
need time to get CI signal after merging your cherry picks ahead
|
||||
of the actual release.
|
||||
|
||||
@@ -73,15 +73,15 @@ dates for simplicity (every month has it).
|
||||
## Upcoming Monthly Releases
|
||||
|
||||
Timelines may vary with the severity of bug fixes, but for easier planning we
|
||||
will target the following monthly release points. Unplanned, critical
|
||||
will target the following monthly release points. Unplanned, critical
|
||||
releases may also occur in between these.
|
||||
|
||||
| Monthly Patch Release | Target date |
|
||||
| --- | --- |
|
||||
| June 2021 | 2021-06-16 |
|
||||
| July 2021 | 2021-07-14 |
|
||||
| August 2021 | 2021-08-11 |
|
||||
| September 2021 | 2021-09-15 |
|
||||
| --------------------- | ----------- |
|
||||
| June 2021 | 2021-06-16 |
|
||||
| July 2021 | 2021-07-14 |
|
||||
| August 2021 | 2021-08-11 |
|
||||
| September 2021 | 2021-09-15 |
|
||||
|
||||
## Detailed Release History for Active Branches
|
||||
|
||||
@@ -92,7 +92,7 @@ releases may also occur in between these.
|
||||
End of Life for **1.21** is **2022-06-28**
|
||||
|
||||
| PATCH RELEASE | CHERRY PICK DEADLINE | TARGET DATE |
|
||||
|--- |--- |--- |
|
||||
| ------------- | -------------------- | ----------- |
|
||||
| 1.21.2 | 2021-06-12 | 2021-06-16 |
|
||||
| 1.21.1 | 2021-05-07 | 2021-05-12 |
|
||||
|
||||
@@ -102,16 +102,16 @@ End of Life for **1.21** is **2022-06-28**
|
||||
|
||||
End of Life for **1.20** is **2022-02-28**
|
||||
|
||||
| PATCH RELEASE | CHERRY PICK DEADLINE | TARGET DATE |
|
||||
|--- |--- |--- |
|
||||
| 1.20.8 | 2021-06-12 | 2021-06-16 |
|
||||
| 1.20.7 | 2021-05-07 | 2021-05-12 |
|
||||
| 1.20.6 | 2021-04-09 | 2021-04-14 |
|
||||
| 1.20.5 | 2021-03-12 | 2021-03-17 |
|
||||
| 1.20.4 | 2021-02-12 | 2021-02-18 |
|
||||
| PATCH RELEASE | CHERRY PICK DEADLINE | TARGET DATE |
|
||||
| ------------- | ----------------------------------------------------------------------------------- | ----------- |
|
||||
| 1.20.8 | 2021-06-12 | 2021-06-16 |
|
||||
| 1.20.7 | 2021-05-07 | 2021-05-12 |
|
||||
| 1.20.6 | 2021-04-09 | 2021-04-14 |
|
||||
| 1.20.5 | 2021-03-12 | 2021-03-17 |
|
||||
| 1.20.4 | 2021-02-12 | 2021-02-18 |
|
||||
| 1.20.3 | [Conformance Tests Issue](https://groups.google.com/g/kubernetes-dev/c/oUpY9vWgzJo) | 2021-02-17 |
|
||||
| 1.20.2 | 2021-01-08 | 2021-01-13 |
|
||||
| 1.20.1 | [Tagging Issue](https://groups.google.com/g/kubernetes-dev/c/dNH2yknlCBA) | 2020-12-18 |
|
||||
| 1.20.2 | 2021-01-08 | 2021-01-13 |
|
||||
| 1.20.1 | [Tagging Issue](https://groups.google.com/g/kubernetes-dev/c/dNH2yknlCBA) | 2020-12-18 |
|
||||
|
||||
### 1.19
|
||||
|
||||
@@ -119,46 +119,46 @@ End of Life for **1.20** is **2022-02-28**
|
||||
|
||||
End of Life for **1.19** is **2021-10-28**
|
||||
|
||||
| PATCH RELEASE | CHERRY PICK DEADLINE | TARGET DATE |
|
||||
|--- |--- |--- |
|
||||
| 1.19.12 | 2021-06-12 | 2021-06-16 |
|
||||
| 1.19.11 | 2021-05-07 | 2021-05-12 |
|
||||
| 1.19.10 | 2021-04-09 | 2021-04-14 |
|
||||
| 1.19.9 | 2021-03-12 | 2021-03-17 |
|
||||
| 1.19.8 | 2021-02-12 | 2021-02-17 |
|
||||
| 1.19.7 | 2021-01-08 | 2021-01-13 |
|
||||
| PATCH RELEASE | CHERRY PICK DEADLINE | TARGET DATE |
|
||||
| ------------- | ------------------------------------------------------------------------- | ----------- |
|
||||
| 1.19.12 | 2021-06-12 | 2021-06-16 |
|
||||
| 1.19.11 | 2021-05-07 | 2021-05-12 |
|
||||
| 1.19.10 | 2021-04-09 | 2021-04-14 |
|
||||
| 1.19.9 | 2021-03-12 | 2021-03-17 |
|
||||
| 1.19.8 | 2021-02-12 | 2021-02-17 |
|
||||
| 1.19.7 | 2021-01-08 | 2021-01-13 |
|
||||
| 1.19.6 | [Tagging Issue](https://groups.google.com/g/kubernetes-dev/c/dNH2yknlCBA) | 2020-12-18 |
|
||||
| 1.19.5 | 2020-12-04 | 2020-12-09 |
|
||||
| 1.19.4 | 2020-11-06 | 2020-11-11 |
|
||||
| 1.19.3 | 2020-10-09 | 2020-10-14 |
|
||||
| 1.19.2 | 2020-09-11 | 2020-09-16 |
|
||||
| 1.19.1 | 2020-09-04 | 2020-09-09 |
|
||||
| 1.19.5 | 2020-12-04 | 2020-12-09 |
|
||||
| 1.19.4 | 2020-11-06 | 2020-11-11 |
|
||||
| 1.19.3 | 2020-10-09 | 2020-10-14 |
|
||||
| 1.19.2 | 2020-09-11 | 2020-09-16 |
|
||||
| 1.19.1 | 2020-09-04 | 2020-09-09 |
|
||||
|
||||
## Non-Active Branch History
|
||||
|
||||
These releases are no longer supported.
|
||||
|
||||
| Minor Version | Final Patch Release | EOL date |
|
||||
| --- | --- | --- |
|
||||
| 1.18 | 1.18.19 | 2021-05-12 |
|
||||
| 1.17 | 1.17.17 | 2021-01-13 |
|
||||
| 1.16 | 1.16.15 | 2020-09-02 |
|
||||
| 1.15 | 1.15.12 | 2020-05-06 |
|
||||
| 1.14 | 1.14.10 | 2019-12-11 |
|
||||
| 1.13 | 1.13.12 | 2019-10-15 |
|
||||
| 1.12 | 1.12.10 | 2019-07-08 |
|
||||
| 1.11 | 1.11.10 | 2019-05-01 |
|
||||
| 1.10 | 1.10.13 | 2019-02-13 |
|
||||
| 1.9 | 1.9.11 | 2018-09-29 |
|
||||
| 1.8 | 1.8.15 | 2018-07-12 |
|
||||
| 1.7 | 1.7.16 | 2018-04-04 |
|
||||
| 1.6 | 1.6.13 | 2017-11-23 |
|
||||
| 1.5 | 1.5.8 | 2017-10-01 |
|
||||
| 1.4 | 1.4.12 | 2017-04-21 |
|
||||
| 1.3 | 1.3.10 | 2016-11-01 |
|
||||
| 1.2 | 1.2.7 | 2016-10-23 |
|
||||
| Minor Version | Final Patch Release | EOL date |
|
||||
| ------------- | ------------------- | ---------- |
|
||||
| 1.18 | 1.18.19 | 2021-05-12 |
|
||||
| 1.17 | 1.17.17 | 2021-01-13 |
|
||||
| 1.16 | 1.16.15 | 2020-09-02 |
|
||||
| 1.15 | 1.15.12 | 2020-05-06 |
|
||||
| 1.14 | 1.14.10 | 2019-12-11 |
|
||||
| 1.13 | 1.13.12 | 2019-10-15 |
|
||||
| 1.12 | 1.12.10 | 2019-07-08 |
|
||||
| 1.11 | 1.11.10 | 2019-05-01 |
|
||||
| 1.10 | 1.10.13 | 2019-02-13 |
|
||||
| 1.9 | 1.9.11 | 2018-09-29 |
|
||||
| 1.8 | 1.8.15 | 2018-07-12 |
|
||||
| 1.7 | 1.7.16 | 2018-04-04 |
|
||||
| 1.6 | 1.6.13 | 2017-11-23 |
|
||||
| 1.5 | 1.5.8 | 2017-10-01 |
|
||||
| 1.4 | 1.4.12 | 2017-04-21 |
|
||||
| 1.3 | 1.3.10 | 2016-11-01 |
|
||||
| 1.2 | 1.2.7 | 2016-10-23 |
|
||||
|
||||
[cherry-picks]: https://git.k8s.io/community/contributors/devel/sig-release/cherry-picks.md
|
||||
[cherry-picks]: https://github.com/kubernetes/community/blob/master/contributors/devel/sig-release/cherry-picks.md
|
||||
[release-managers]: /release-managers.md
|
||||
[release process description]: /release.md
|
||||
[yearly-support]: https://git.k8s.io/enhancements/keps/sig-release/1498-kubernetes-yearly-support-period/README.md
|
||||
|
||||
@@ -192,6 +192,8 @@ GitHub team: [@kubernetes/sig-release-leads](https://github.com/orgs/kubernetes/
|
||||
|
||||
### Technical Leads
|
||||
|
||||
- Adolfo García Veytia ([@puerco](https://github.com/puerco))
|
||||
- Carlos Panato ([@cpanato](https://github.com/cpanato))
|
||||
- Daniel Mangum ([@hasheddan](https://github.com/hasheddan))
|
||||
- Jeremy Rickard ([@jeremyrickard](https://github.com/jeremyrickard))
|
||||
|
||||
|
||||
@@ -3,6 +3,7 @@ title: Kubernetes Release Cycle
|
||||
type: docs
|
||||
auto_generated: true
|
||||
---
|
||||
|
||||
<!-- THIS CONTENT IS AUTO-GENERATED via ./scripts/releng/update-release-info.sh in k/website -->
|
||||
|
||||
{{< warning >}}
|
||||
@@ -89,43 +90,43 @@ The general labeling process should be consistent across artifact types.
|
||||
|
||||
## Definitions
|
||||
|
||||
- *issue owners*: Creator, assignees, and user who moved the issue into a
|
||||
- _issue owners_: Creator, assignees, and user who moved the issue into a
|
||||
release milestone
|
||||
|
||||
- *Release Team*: Each Kubernetes release has a team doing project management
|
||||
- _Release Team_: Each Kubernetes release has a team doing project management
|
||||
tasks described [here][release-team].
|
||||
|
||||
The contact info for the team associated with any given release can be found
|
||||
[here](https://git.k8s.io/sig-release/releases/).
|
||||
|
||||
- *Y days*: Refers to business days
|
||||
- _Y days_: Refers to business days
|
||||
|
||||
- *enhancement*: see "[Is My Thing an Enhancement?](https://git.k8s.io/enhancements/README.md#is-my-thing-an-enhancement)"
|
||||
- _enhancement_: see "[Is My Thing an Enhancement?](https://git.k8s.io/enhancements/README.md#is-my-thing-an-enhancement)"
|
||||
|
||||
- *[Enhancements Freeze][enhancements-freeze]*:
|
||||
- _[Enhancements Freeze][enhancements-freeze]_:
|
||||
the deadline by which [KEPs][keps] have to be completed in order for
|
||||
enhancements to be part of the current release
|
||||
|
||||
- *[Exception Request][exceptions]*:
|
||||
- _[Exception Request][exceptions]_:
|
||||
The process of requesting an extension on the deadline for a particular
|
||||
Enhancement
|
||||
|
||||
- *[Code Freeze][code-freeze]*:
|
||||
- _[Code Freeze][code-freeze]_:
|
||||
The period of ~4 weeks before the final release date, during which only
|
||||
critical bug fixes are merged into the release.
|
||||
|
||||
- *[Pruning](https://git.k8s.io/sig-release/releases/release_phases.md#pruning)*:
|
||||
- _[Pruning](https://git.k8s.io/sig-release/releases/release_phases.md#pruning)_:
|
||||
The process of removing an Enhancement from a release milestone if it is not
|
||||
fully implemented or is otherwise considered not stable.
|
||||
|
||||
- *release milestone*: semantic version string or
|
||||
- _release milestone_: semantic version string or
|
||||
[GitHub milestone](https://help.github.com/en/github/managing-your-work-on-github/associating-milestones-with-issues-and-pull-requests)
|
||||
referring to a release MAJOR.MINOR `vX.Y` version.
|
||||
|
||||
See also
|
||||
[release versioning](/contributors/design-proposals/release/versioning.md).
|
||||
|
||||
- *release branch*: Git branch `release-X.Y` created for the `vX.Y` milestone.
|
||||
- _release branch_: Git branch `release-X.Y` created for the `vX.Y` milestone.
|
||||
|
||||
Created at the time of the `vX.Y-rc.0` release and maintained after the
|
||||
release for approximately 12 months with `vX.Y.Z` patch releases.
|
||||
@@ -160,7 +161,7 @@ conjunction with the Release Team's [Enhancements Lead](https://git.k8s.io/sig-r
|
||||
|
||||
After Enhancements Freeze, tracking milestones on PRs and issues is important.
|
||||
Items within the milestone are used as a punchdown list to complete the
|
||||
release. *On issues*, milestones must be applied correctly, via triage by the
|
||||
release. _On issues_, milestones must be applied correctly, via triage by the
|
||||
SIG, so that [Release Team][release-team] can track bugs and enhancements (any
|
||||
enhancement-related issue needs a milestone).
|
||||
|
||||
@@ -354,7 +355,7 @@ issue kind labels must be set:
|
||||
- `kind/feature`: New functionality.
|
||||
- `kind/flake`: CI test case is showing intermittent failures.
|
||||
|
||||
[cherry-picks]: /contributors/devel/sig-release/cherry-picks.md
|
||||
[cherry-picks]: /community/blob/master/contributors/devel/sig-release/cherry-picks.md
|
||||
[code-freeze]: https://git.k8s.io/sig-release/releases/release_phases.md#code-freeze
|
||||
[enhancements-freeze]: https://git.k8s.io/sig-release/releases/release_phases.md#enhancements-freeze
|
||||
[exceptions]: https://git.k8s.io/sig-release/releases/release_phases.md#exceptions
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
title: Container Runtime
|
||||
id: container-runtime
|
||||
date: 2019-06-05
|
||||
full_link: /es/docs/reference/generated/container-runtime
|
||||
full_link: /docs/setup/production-environment/container-runtimes
|
||||
short_description: >
|
||||
El _Container Runtime_, entorno de ejecución de un contenedor, es el software responsable de ejecutar contenedores.
|
||||
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
title: Container Runtime
|
||||
id: container-runtime
|
||||
date: 2019-06-05
|
||||
full_link: /docs/reference/generated/container-runtime
|
||||
full_link: /docs/setup/production-environment/container-runtimes
|
||||
short_description: >
|
||||
L'environnement d'exécution de conteneurs est le logiciel responsable de l'exécution des conteneurs.
|
||||
|
||||
|
||||
+1
-1
@@ -225,7 +225,7 @@ Si la startup probe ne réussit jamais, le conteneur est tué après 300s puis s
|
||||
|
||||
Parfois, les applications sont temporairement incapables de servir le trafic.
|
||||
Par exemple, une application peut avoir besoin de charger des larges données ou des fichiers de configuration pendant le démarrage, ou elle peut dépendre de services externes après le démarrage.
|
||||
Dans ces cas, vous ne voulez pas tuer l'application, mais tu ne veux pas non plus lui envoyer de requêtes. Kubernetes fournit des readiness probes pour détecter et atténuer ces situations. Un pod avec des conteneurs qui signale qu'elle n'est pas prête ne reçoit pas de trafic par les services de Kubernetes.
|
||||
Dans ces cas, vous ne voulez pas tuer l'application, mais vous ne voulez pas non plus lui envoyer de requêtes. Kubernetes fournit des readiness probes pour détecter et atténuer ces situations. Un pod avec des conteneurs qui signale qu'elle n'est pas prête ne reçoit pas de trafic par les services de Kubernetes.
|
||||
|
||||
{{< note >}}
|
||||
Readiness probes fonctionnent sur le conteneur pendant tout son cycle de vie.
|
||||
|
||||
@@ -0,0 +1,53 @@
|
||||
---
|
||||
title: アドオンのインストール
|
||||
content_type: concept
|
||||
---
|
||||
|
||||
<!-- overview -->
|
||||
|
||||
{{% thirdparty-content %}}
|
||||
|
||||
アドオンはKubernetesの機能を拡張するものです。
|
||||
|
||||
このページでは、利用可能なアドオンの一部の一覧と、それぞれのアドオンのインストール方法へのリンクを提供します。
|
||||
|
||||
<!-- body -->
|
||||
|
||||
## ネットワークとネットワークポリシー
|
||||
|
||||
* [ACI](https://www.github.com/noironetworks/aci-containers)は、統合されたコンテナネットワークとネットワークセキュリティをCisco ACIを使用して提供します。
|
||||
* [Antrea](https://antrea.io/)は、L3またはL4で動作して、Open vSwitchをネットワークデータプレーンとして活用する、Kubernetes向けのネットワークとセキュリティサービスを提供します。
|
||||
* [Calico](https://docs.projectcalico.org/latest/introduction/)はネットワークとネットワークプリシーのプロバイダーです。Calicoは、BGPを使用または未使用の非オーバーレイおよびオーバーレイネットワークを含む、フレキシブルなさまざまなネットワークオプションをサポートします。Calicoはホスト、Pod、そして(IstioとEnvoyを使用している場合には)サービスメッシュ上のアプリケーションに対してネットワークポリシーを強制するために、同一のエンジンを使用します。
|
||||
* [Canal](https://github.com/tigera/canal/tree/master/k8s-install)はFlannelとCalicoをあわせたもので、ネットワークとネットワークポリシーを提供します。
|
||||
* [Cilium](https://github.com/cilium/cilium)は、L3のネットワークとネットワークポリシーのプラグインで、HTTP/API/L7のポリシーを透過的に強制できます。ルーティングとoverlay/encapsulationモードの両方をサポートしており、他のCNIプラグイン上で機能できます。
|
||||
* [CNI-Genie](https://github.com/Huawei-PaaS/CNI-Genie)は、KubernetesをCalico、Canal、Flannel、Romana、Weaveなど選択したCNIプラグインをシームレスに接続できるようにするプラグインです。
|
||||
* [Contiv](https://contiv.github.io)は、さまざまなユースケースと豊富なポリシーフレームワーク向けに設定可能なネットワーク(BGPを使用したネイティブのL3、vxlanを使用したオーバーレイ、古典的なL2、Cisco-SDN/ACI)を提供します。Contivプロジェクトは完全に[オープンソース](https://github.com/contiv)です。[インストーラ](https://github.com/contiv/install)はkubeadmとkubeadm以外の両方をベースとしたインストールオプションがあります。
|
||||
* [Contrail](https://www.juniper.net/us/en/products-services/sdn/contrail/contrail-networking/)は、[Tungsten Fabric](https://tungsten.io)をベースにしている、オープンソースでマルチクラウドに対応したネットワーク仮想化およびポリシー管理プラットフォームです。ContrailおよびTungsten Fabricは、Kubernetes、OpenShift、OpenStack、Mesosなどのオーケストレーションシステムと統合されており、仮想マシン、コンテナ/Pod、ベアメタルのワークロードに隔離モードを提供します。
|
||||
* [Flannel](https://github.com/coreos/flannel/blob/master/Documentation/kubernetes.md)は、Kubernetesで使用できるオーバーレイネットワークプロバイダーです。
|
||||
* [Knitter](https://github.com/ZTE/Knitter/)は、1つのKubernetes Podで複数のネットワークインターフェイスをサポートするためのプラグインです。
|
||||
* [Multus](https://github.com/Intel-Corp/multus-cni)は、すべてのCNIプラグイン(たとえば、Calico、Cilium、Contiv、Flannel)に加えて、SRIOV、DPDK、OVS-DPDK、VPPをベースとするKubernetes上のワークロードをサポートする、複数のネットワークサポートのためのマルチプラグインです。
|
||||
* [OVN-Kubernetes](https://github.com/ovn-org/ovn-kubernetes/)は、Open vSwitch(OVS)プロジェクトから生まれた仮想ネットワーク実装である[OVN(Open Virtual Network)](https://github.com/ovn-org/ovn/)をベースとする、Kubernetesのためのネットワークプロバイダです。OVN-Kubernetesは、OVSベースのロードバランサーおよびネットワークポリシーの実装を含む、Kubernetes向けのオーバーレイベースのネットワーク実装を提供します。
|
||||
* [OVN4NFV-K8S-Plugin](https://github.com/opnfv/ovn4nfv-k8s-plugin)は、クラウドネイティブベースのService function chaining(SFC)、Multiple OVNオーバーレイネットワーク、動的なサブネットの作成、動的な仮想ネットワークの作成、VLANプロバイダーネットワーク、Directプロバイダーネットワークを提供し、他のMulti-networkプラグインと付け替え可能なOVNベースのCNIコントローラープラグインです。
|
||||
* [NSX-T](https://docs.vmware.com/en/VMware-NSX-T/2.0/nsxt_20_ncp_kubernetes.pdf) Container Plug-in(NCP)は、VMware NSX-TとKubernetesなどのコンテナオーケストレーター間のインテグレーションを提供します。また、NSX-Tと、Pivotal Container Service(PKS)とOpenShiftなどのコンテナベースのCaaS/PaaSプラットフォームとのインテグレーションも提供します。
|
||||
* [Nuage](https://github.com/nuagenetworks/nuage-kubernetes/blob/v5.1.1-1/docs/kubernetes-1-installation.rst)は、Kubernetes Podと非Kubernetes環境間で可視化とセキュリティモニタリングを使用してポリシーベースのネットワークを提供するSDNプラットフォームです。
|
||||
* [Romana](https://romana.io)は、[NetworkPolicy API](/docs/concepts/services-networking/network-policies/)もサポートするPodネットワーク向けのL3のネットワークソリューションです。Kubeadmアドオンのインストールの詳細は[こちら](https://github.com/romana/romana/tree/master/containerize)で確認できます。
|
||||
* [Weave Net](https://www.weave.works/docs/net/latest/kubernetes/kube-addon/)は、ネットワークパーティションの両面で機能し、外部データベースを必要とせずに、ネットワークとネットワークポリシーを提供します。
|
||||
|
||||
## サービスディスカバリ
|
||||
|
||||
* [CoreDNS](https://coredns.io)は、フレキシブルで拡張可能なDNSサーバーです。Pod向けのクラスター内DNSとして[インストール](https://github.com/coredns/deployment/tree/master/kubernetes)できます。
|
||||
|
||||
## 可視化と制御
|
||||
|
||||
* [Dashboard](https://github.com/kubernetes/dashboard#kubernetes-dashboard)はKubernetes向けのダッシュボードを提供するウェブインターフェイスです。
|
||||
* [Weave Scope](https://www.weave.works/documentation/scope-latest-installing/#k8s)は、コンテナ、Pod、Serviceなどをグラフィカルに可視化するツールです。[Weave Cloud account](https://cloud.weave.works/)と組み合わせて使うか、UIを自分でホストして使います。
|
||||
|
||||
## インフラストラクチャ
|
||||
|
||||
* [KubeVirt](https://kubevirt.io/user-guide/#/installation/installation)は仮想マシンをKubernetes上で実行するためのアドオンです。通常、ベアメタルのクラスタで実行します。
|
||||
|
||||
## レガシーなアドオン
|
||||
|
||||
いくつかのアドオンは、廃止された[cluster/addons](https://git.k8s.io/kubernetes/cluster/addons)ディレクトリに掲載されています。
|
||||
|
||||
よくメンテナンスされたアドオンはここにリンクしてください。PRを歓迎しています。
|
||||
@@ -0,0 +1,67 @@
|
||||
---
|
||||
title: コンテナイメージのガベージコレクション
|
||||
content_type: concept
|
||||
weight: 70
|
||||
---
|
||||
|
||||
<!-- overview -->
|
||||
|
||||
ガベージコレクションは、未使用の[イメージ](/ja/docs/concepts/containers/#container-images)と未使用の[コンテナ](/ja/docs/concepts/containers/)をクリーンアップするkubeletの便利な機能です。kubeletコンテナのガベージコレクションを1分ごとに行い、イメージのガベージコレクションは5分ごとに行います。
|
||||
|
||||
存在することが期待されているコンテナを削除してkubeletの動作を壊す可能性があるため、外部のガベージコレクションのツールは推奨されません。
|
||||
|
||||
<!-- body -->
|
||||
|
||||
## イメージのガベージコレクション
|
||||
|
||||
Kubernetesでは、すべてのイメージのライフサイクルの管理はcadvisorと協調してimageManager経由で行います。
|
||||
|
||||
イメージのガベージコレクションのポリシーについて考えるときは、`HighThresholdPercent`および`LowThresholdPercent`という2つの要因について考慮する必要があります。ディスク使用量がhigh thresholdを超えると、ガベージコレクションがトリガされます。ガベージコレクションは、low
|
||||
thresholdが満たされるまで、最後に使われてから最も時間が経った(least recently used)イメージを削除します。
|
||||
|
||||
## コンテナのガベージコレクション
|
||||
|
||||
コンテナのガベージコレクションのポリシーは、3つのユーザー定義の変数を考慮に入れます。`MinAge`は、ガベージコレクションできるコンテナの最小の年齢です。`MaxPerPodContainer`は、すべての単一のPod(UID、コンテナ名)が保持することを許されているdead状態のコンテナの最大値です。`MaxContainers`はdead状態のコンテナの合計の最大値です。これらの変数は、`MinAge`は0に、`MaxPerPodContainer`と`MaxContainers`は0未満にそれぞれ設定することで個別に無効にできます。
|
||||
|
||||
kubeletは、未指定のコンテナ、削除されたコンテナ、前述のフラグにより設定された境界の外にあるコンテナに対して動作します。一般に、最も古いコンテナが最初に削除されます。`MaxPerPodContainer`と`MaxContainer`は、Podごとの保持するコンテナの最大値(`MaxPerPodContainer`)がグローバルのdead状態のコンテナの許容範囲(`MaxContainers`)外である場合には、互いに競合する可能性があります。このような状況では、`MaxPerPodContainer`が調整されます。最悪のケースのシナリオでは、`MaxPerPodContainer`が1にダウングレードされ、最も古いコンテナが強制退去されます。さらに、`MinAge`より古くなると、削除済みのPodが所有するコンテナが削除されます。
|
||||
|
||||
kubeletによって管理されないコンテナは、コンテナのガベージコレクションの対象にはなりません。
|
||||
|
||||
## ユーザー設定
|
||||
|
||||
イメージのガベージコレクションを調整するために、以下のkubeletのフラグを使用して次のようなしきい値を調整できます。
|
||||
|
||||
1. `image-gc-high-threshold`: イメージのガベージコレクションをトリガするディスク使用量の割合(%)。デフォルトは85%。
|
||||
2. `image-gc-low-threshold`: イメージのガベージコレクションが解放を試みるディスク使用量の割合(%)。デフォルトは80%。
|
||||
|
||||
ガベージコレクションのポリシーは、以下のkubeletのフラグを使用してカスタマイズできます。
|
||||
|
||||
1. `minimum-container-ttl-duration`: 完了したコンテナがガベージコレクションされる前に経過するべき最小期間。デフォルトは0分です。つまり、すべての完了したコンテナはガベージコレクションされます。
|
||||
2. `maximum-dead-containers-per-container`: コンテナごとに保持される古いインスタンスの最大値です。デフォルトは1です。
|
||||
3. `maximum-dead-containers`: グローバルに保持するべき古いコンテナのインスタンスの最大値です。デフォルトは-1です。つまり、グローバルなリミットは存在しません。
|
||||
|
||||
コンテナは役に立たなくなる前にガベージコレクションされる可能性があります。こうしたコンテナには、トラブルシューティングに役立つログや他のデータが含まれるかもしれません。そのため、期待されるコンテナごとに最低でも1つのdead状態のコンテナが許容されるようにするために、`maximum-dead-containers-per-container`には十分大きな値を設定することが強く推奨されます。同様の理由で、`maximum-dead-containers`にも、より大きな値を設定することが推奨されます。詳しくは、[こちらのissue](https://github.com/kubernetes/kubernetes/issues/13287)を読んでください。
|
||||
|
||||
## 廃止
|
||||
|
||||
このドキュメントにあるkubeletの一部のガベージコレクションの機能は、将来kubelet evictionで置換される予定です。
|
||||
|
||||
これには以下のものが含まれます。
|
||||
|
||||
| 既存のフラグ | 新しいフラグ | 理由 |
|
||||
| ------------- | -------- | --------- |
|
||||
| `--image-gc-high-threshold` | `--eviction-hard`または`--eviction-soft` | 既存のevictionのシグナルがイメージのガベージコレクションをトリガする可能性がある |
|
||||
| `--image-gc-low-threshold` | `--eviction-minimum-reclaim` | eviction reclaimが同等の動作を実現する |
|
||||
| `--maximum-dead-containers` | | 古いログがコンテナのコンテキストの外部に保存されるようになったら廃止 |
|
||||
| `--maximum-dead-containers-per-container` | | 古いログがコンテナのコンテキストの外部に保存されるようになったら廃止 |
|
||||
| `--minimum-container-ttl-duration` | | 古いログがコンテナのコンテキストの外部に保存されるようになったら廃止 |
|
||||
| `--low-diskspace-threshold-mb` | `--eviction-hard` or `eviction-soft` | evictionはディスクのしきい値を他のリソースに一般化している |
|
||||
| `--outofdisk-transition-frequency` | `--eviction-pressure-transition-period` | evictionはディスクのpressure transitionを他のリソースに一般化している |
|
||||
|
||||
|
||||
|
||||
## {{% heading "whatsnext" %}}
|
||||
|
||||
|
||||
詳細については、[リソース不足のハンドリング方法を設定する](/docs/tasks/administer-cluster/out-of-resource/)を参照してください。
|
||||
|
||||
@@ -0,0 +1,126 @@
|
||||
---
|
||||
title: システムログ
|
||||
content_type: concept
|
||||
weight: 60
|
||||
---
|
||||
|
||||
<!-- overview -->
|
||||
|
||||
システムコンポーネントのログは、クラスター内で起こったイベントを記録します。このログはデバッグのために非常に役立ちます。ログのverbosityを設定すると、ログをどの程度詳細に見るのかを変更できます。ログはコンポーネント内のエラーを表示する程度の荒い粒度にすることも、イベントのステップバイステップのトレース(HTTPのアクセスログ、Podの状態の変更、コントローラーの動作、スケジューラーの決定など)を表示するような細かい粒度に設定することもできます。
|
||||
|
||||
<!-- body -->
|
||||
|
||||
## klog
|
||||
|
||||
klogは、Kubernetesのログライブラリです。[klog](https://github.com/kubernetes/klog)は、Kubernetesのシステムコンポーネント向けのログメッセージを生成します。
|
||||
|
||||
klogの設定に関する詳しい情報については、[コマンドラインツールのリファレンス](/docs/reference/command-line-tools-reference/)を参照してください。
|
||||
|
||||
klogネイティブ形式の例:
|
||||
|
||||
```
|
||||
I1025 00:15:15.525108 1 httplog.go:79] GET /api/v1/namespaces/kube-system/pods/metrics-server-v0.3.1-57c75779f-9p8wg: (1.512ms) 200 [pod_nanny/v0.0.0 (linux/amd64) kubernetes/$Format 10.56.1.19:51756]
|
||||
```
|
||||
|
||||
### 構造化ログ
|
||||
|
||||
{{< feature-state for_k8s_version="v1.19" state="alpha" >}}
|
||||
|
||||
{{< warning >}}
|
||||
構造化ログへのマイグレーションは現在進行中の作業です。このバージョンでは、すべてのログメッセージが構造化されているわけではありません。ログファイルをパースする場合、JSONではないログの行にも対処しなければなりません。
|
||||
|
||||
ログの形式と値のシリアライズは変更される可能性があります。
|
||||
{{< /warning>}}
|
||||
|
||||
構造化ログは、ログメッセージに単一の構造を導入し、プログラムで情報の抽出ができるようにするものです。構造化ログは、僅かな労力とコストで保存・処理できます。新しいメッセージ形式は後方互換性があり、デフォルトで有効化されます。
|
||||
|
||||
構造化ログの形式:
|
||||
|
||||
```ini
|
||||
<klog header> "<message>" <key1>="<value1>" <key2>="<value2>" ...
|
||||
```
|
||||
|
||||
例:
|
||||
|
||||
```ini
|
||||
I1025 00:15:15.525108 1 controller_utils.go:116] "Pod status updated" pod="kube-system/kubedns" status="ready"
|
||||
```
|
||||
|
||||
|
||||
### JSONログ形式
|
||||
|
||||
{{< feature-state for_k8s_version="v1.19" state="alpha" >}}
|
||||
|
||||
{{<warning >}}
|
||||
JSONの出力は多数の標準のklogフラグをサポートしていません。非対応のklogフラグの一覧については、[コマンドラインツールリファレンス](/docs/reference/command-line-tools-reference/)を参照してください。
|
||||
|
||||
すべてのログがJSON形式で書き込むことに対応しているわけではありません(たとえば、プロセスの開始時など)。ログのパースを行おうとしている場合、JSONではないログの行に対処できるようにしてください。
|
||||
|
||||
フィールド名とJSONのシリアライズは変更される可能性があります。
|
||||
{{< /warning >}}
|
||||
|
||||
`--logging-format=json`フラグは、ログの形式をネイティブ形式klogからJSON形式に変更します。以下は、JSONログ形式の例(pretty printしたもの)です。
|
||||
|
||||
```json
|
||||
{
|
||||
"ts": 1580306777.04728,
|
||||
"v": 4,
|
||||
"msg": "Pod status updated",
|
||||
"pod":{
|
||||
"name": "nginx-1",
|
||||
"namespace": "default"
|
||||
},
|
||||
"status": "ready"
|
||||
}
|
||||
```
|
||||
|
||||
特別な意味を持つキー:
|
||||
* `ts` - Unix時間のタイムスタンプ(必須、float)
|
||||
* `v` - verbosity (必須、int、デフォルトは0)
|
||||
* `err` - エラー文字列 (オプション、string)
|
||||
* `msg` - メッセージ (必須、string)
|
||||
|
||||
現在サポートされているJSONフォーマットの一覧:
|
||||
* {{< glossary_tooltip term_id="kube-controller-manager" text="kube-controller-manager" >}}
|
||||
* {{< glossary_tooltip term_id="kube-apiserver" text="kube-apiserver" >}}
|
||||
* {{< glossary_tooltip term_id="kube-scheduler" text="kube-scheduler" >}}
|
||||
* {{< glossary_tooltip term_id="kubelet" text="kubelet" >}}
|
||||
|
||||
### ログのサニタイズ
|
||||
|
||||
{{< feature-state for_k8s_version="v1.20" state="alpha" >}}
|
||||
|
||||
{{<warning >}}
|
||||
ログのサニタイズ大きな計算のオーバーヘッドを引き起こす可能性があるため、本番環境では有効にするべきではありません。
|
||||
{{< /warning >}}
|
||||
|
||||
`--experimental-logging-sanitization`フラグはklogのサニタイズフィルタを有効にします。有効にすると、すべてのログの引数が機密データ(パスワード、キー、トークンなど)としてタグ付けされたフィールドについて検査され、これらのフィールドのログの記録は防止されます。
|
||||
|
||||
現在ログのサニタイズをサポートしているコンポーネント一覧:
|
||||
* kube-controller-manager
|
||||
* kube-apiserver
|
||||
* kube-scheduler
|
||||
* kubelet
|
||||
|
||||
{{< note >}}
|
||||
ログのサニタイズフィルターは、ユーザーのワークロードのログが機密データを漏洩するのを防げるわけではありません。
|
||||
{{< /note >}}
|
||||
|
||||
### ログのverbosityレベル
|
||||
|
||||
`-v`フラグはログのverbosityを制御します。値を増やすとログに記録されるイベントの数が増えます。値を減らすとログに記録されるイベントの数が減ります。verbosityの設定を増やすと、ますます多くの深刻度の低いイベントをログに記録するようになります。verbosityの設定を0にすると、クリティカルなイベントだけをログに記録します。
|
||||
|
||||
### ログの場所
|
||||
|
||||
システムコンポーネントには2種類あります。コンテナ内で実行されるコンポーネントと、コンテナ内で実行されないコンポーネントです。たとえば、次のようなコンポーネントがあります。
|
||||
|
||||
* Kubernetesのスケジューラーやkube-proxyはコンテナ内で実行されます。
|
||||
* kubeletやDockerのようなコンテナランタイムはコンテナ内で実行されません。
|
||||
|
||||
systemdを使用しているマシンでは、kubeletとコンテナランタイムはjournaldに書き込みを行います。それ以外のマシンでは、`/var/log`ディレクトリ内の`.log`ファイルに書き込みます。コンテナ内部のシステムコンポーネントは、デフォルトのログ機構をバイパスするため、常に`/var/log`ディレクトリ内の`.log`ファイルに書き込みます。コンテナのログと同様に、`/var/log`ディレクトリ内のシステムコンポーネントのログはローテートする必要があります。`kube-up.sh`スクリプトによって作成されたKubernetesクラスターでは、ログローテーションは`logrotate`ツールで設定されます。`logrotate`ツールはログを1日ごとまたはログのサイズが100MBを超えたときにローテートします。
|
||||
|
||||
## {{% heading "whatsnext" %}}
|
||||
|
||||
* [Kubernetesのログのアーキテクチャ](/docs/concepts/cluster-administration/logging/)について読む。
|
||||
* [構造化ログ](https://github.com/kubernetes/enhancements/tree/master/keps/sig-instrumentation/1602-structured-logging)について読む。
|
||||
* [ログの深刻度の慣習](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-instrumentation/logging.md)について読む。
|
||||
@@ -237,7 +237,7 @@ kubeletは、`tmpfs`のemptyDirボリュームをローカルのエフェメラ
|
||||
|
||||
### ローカルのエフェメラルストレージの要求と制限設定
|
||||
|
||||
ローカルのエフェメラルストレージを管理するためには_ephemeral-storage_パラメーターを利用することができます。
|
||||
ローカルのエフェメラルストレージを管理するためには _ephemeral-storage_ パラメーターを利用することができます。
|
||||
Podの各コンテナは、次の1つ以上を指定できます。
|
||||
* `spec.containers[].resources.limits.ephemeral-storage`
|
||||
* `spec.containers[].resources.requests.ephemeral-storage`
|
||||
|
||||
@@ -0,0 +1,114 @@
|
||||
---
|
||||
title: kubeconfigファイルを使用してクラスターアクセスを組織する
|
||||
content_type: concept
|
||||
weight: 60
|
||||
---
|
||||
|
||||
<!-- overview -->
|
||||
|
||||
kubeconfigを使用すると、クラスターに、ユーザー、名前空間、認証の仕組みに関する情報を組織できます。`kubectl`コマンドラインツールはkubeconfigファイルを使用してクラスターを選択するために必要な情報を見つけ、クラスターのAPIサーバーと通信します。
|
||||
|
||||
{{< note >}}
|
||||
クラスターへのアクセスを設定するために使われるファイルは*kubeconfigファイル*と呼ばれます。これは設定ファイルを指すために使われる一般的な方法です。`kubeconfig`という名前を持つファイルが存在するという意味ではありません。
|
||||
{{< /note >}}
|
||||
|
||||
デフォルトでは、`kubectl`は`$HOME/.kube`ディレクトリ内にある`config`という名前のファイルを探します。`KUBECONFIG`環境変数を設定するか、[`--kubeconfig`](/docs/reference/generated/kubectl/kubectl/)フラグで指定することで、別のkubeconfigファイルを指定することもできます。
|
||||
|
||||
kubeconfigファイルの作成と指定に関するステップバイステップの手順を知りたいときは、[複数のクラスターへのアクセスを設定する](/docs/tasks/access-application-cluster/configure-access-multiple-clusters)を参照してください。
|
||||
|
||||
<!-- body -->
|
||||
|
||||
## 複数のクラスター、ユーザ、認証の仕組みのサポート
|
||||
|
||||
複数のクラスターを持っていて、ユーザーやコンポーネントがさまざまな方法で認証を行う次のような状況を考えてみます。
|
||||
|
||||
- 実行中のkubeletが証明書を使用して認証を行う可能性がある。
|
||||
- ユーザーがトークンを使用して認証を行う可能性がある。
|
||||
- 管理者が個別のユーザに提供する複数の証明書を持っている可能性がある。
|
||||
|
||||
kubeconfigファイルを使用すると、クラスター、ユーザー、名前空間を組織化することができます。また、contextを定義することで、複数のクラスターや名前空間を素早く簡単に切り替えられます。
|
||||
|
||||
## Context
|
||||
|
||||
kubeconfigファイルの*context*要素は、アクセスパラメーターを使いやすい名前でグループ化するために使われます。各contextは3つのパラメータ、cluster、namespace、userを持ちます。デフォルトでは、`kubectl`コマンドラインツールはクラスターとの通信に*current context*のパラメーターを使用します。
|
||||
|
||||
current contextを選択するには、以下のコマンドを使用します。
|
||||
|
||||
```
|
||||
kubectl config use-context
|
||||
```
|
||||
|
||||
## KUBECONFIG環境変数
|
||||
|
||||
`KUBECONFIG`環境変数には、kubeconfigファイルのリストを指定できます。LinuxとMacでは、リストはコロン区切りです。Windowsでは、セミコロン区切りです。`KUBECONFIG`環境変数は必須ではありません。`KUBECONFIG`環境変数が存在しない場合は、`kubectl`はデフォルトのkubeconfigファイルである`$HOME/.kube/config`を使用します。
|
||||
|
||||
`KUBECONFIG`環境変数が存在する場合は、`kubectl`は`KUBECONFIG`環境変数にリストされているファイルをマージした結果を有効な設定として使用します。
|
||||
|
||||
## kubeconfigファイルのマージ
|
||||
|
||||
設定ファイルを確認するには、以下のコマンドを実行します。
|
||||
|
||||
```shell
|
||||
kubectl config view
|
||||
```
|
||||
|
||||
上で説明したように、出力は1つのkubeconfigファイルから作られる場合も、複数のkubeconfigファイルをマージした結果となる場合もあります。
|
||||
|
||||
`kubectl`がkubeconfigファイルをマージするときに使用するルールを以下に示します。
|
||||
|
||||
1. もし`--kubeconfig`フラグが設定されていた場合、指定したファイルだけが使用されます。マージは行いません。このフラグに指定できるのは1つのファイルだけです。
|
||||
|
||||
そうでない場合、`KUBECONFIG`環境変数が設定されていた場合には、それをマージするべきファイルのリストとして使用します。`KUBECONFIG`環境変数にリストされたファイルのマージは、次のようなルールに従って行われます。
|
||||
|
||||
* 空のファイルを無視する。
|
||||
* デシリアライズできない内容のファイルに対してエラーを出す。
|
||||
* 特定の値やmapのキーを設定する最初のファイルが勝つ。
|
||||
* 値やmapのキーは決して変更しない。
|
||||
例: 最初のファイルが指定した`current-context`を保持する。
|
||||
例: 2つのファイルが`red-user`を指定した場合、1つ目のファイルの`red-user`だけを使用する。もし2つ目のファイルの`red-user`以下に競合しないエントリーがあったとしても、それらは破棄する。
|
||||
|
||||
`KUBECONFIG`環境変数を設定する例については、[KUBECONFIG環境変数を設定する](/ja/docs/tasks/access-application-cluster/configure-access-multiple-clusters/#set-the-kubeconfig-environment-variable)を参照してください。
|
||||
|
||||
それ以外の場合は、デフォルトのkubeconfigファイル`$HOME/.kube/config`をマージせずに使用します。
|
||||
|
||||
1. 以下のチェーンで最初に見つかったものをもとにして、使用するcontextを決定する。
|
||||
|
||||
1. `--context`コマンドラインフラグが存在すれば、それを使用する。
|
||||
1. マージしたkubeconrfigファイルから`current-context`を使用する。
|
||||
|
||||
この時点では、空のcontextも許容されます。
|
||||
|
||||
1. クラスターとユーザーを決定する。この時点では、contextである場合もそうでない場合もあります。以下のチェーンで最初に見つかったものをもとにして、クラスターとユーザーを決定します。この手順はユーザーとクラスターについてそれぞれ1回ずつ、合わせて2回実行されます。
|
||||
|
||||
1. もし存在すれば、コマンドラインフラグ`--user`または`--cluster`を使用する。
|
||||
1. もしcontextが空でなければ、contextからユーザーまたはクラスターを取得する。
|
||||
|
||||
この時点では、ユーザーとクラスターは空である可能性があります。
|
||||
|
||||
1. 使用する実際のクラスター情報を決定する。この時点では、クラスター情報は存在しない可能性があります。以下のチェーンで最初に見つかったものをもとにして、クラスター情報の各パーツをそれぞれを構築します。
|
||||
|
||||
1. もし存在すれば、`--server`、`--certificate-authority`、`--insecure-skip-tls-verify`コマンドラインフラグを使用する。
|
||||
1. もしマージしたkubeconfigファイルにクラスター情報の属性が存在すれば、それを使用する。
|
||||
1. もしサーバーの場所が存在しなければ、マージは失敗する。
|
||||
|
||||
1. 使用する実際のユーザー情報を決定する。クラスター情報の場合と同じルールを使用して、ユーザー情報を構築します。ただし、ユーザーごとに許可される認証方法は1つだけです。
|
||||
|
||||
1. もし存在すれば、`--client-certificate`、`--client-key`、`--username`、`--password`、`--token`コマンドラインフラグを使用する。
|
||||
1. マージしたkubeconfigファイルの`user`フィールドを使用する。
|
||||
1. もし2つの競合する方法が存在する場合、マージは失敗する。
|
||||
|
||||
1. もし何らかの情報がまだ不足していれば、デフォルトの値を使用し、認証情報については場合によってはプロンプトを表示する。
|
||||
|
||||
## ファイルリファレンス
|
||||
|
||||
kubeconfigファイル内のファイルとパスのリファレンスは、kubeconfigファイルの位置からの相対パスで指定します。コマンドライン上のファイルのリファレンスは、現在のワーキングディレクトリからの相対パスです。`$HOME/.kube/config`内では、相対パスは相対のまま、絶対パスは絶対のまま保存されます。
|
||||
|
||||
## {{% heading "whatsnext" %}}
|
||||
|
||||
|
||||
* [複数のクラスターへのアクセスを設定する](/docs/tasks/access-application-cluster/configure-access-multiple-clusters/)
|
||||
* [`kubectl config`](/docs/reference/generated/kubectl/kubectl-commands#config)
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -58,11 +58,11 @@ weight: 10
|
||||
|
||||
## ラベルの使用
|
||||
|
||||
- `{ app: myapp, tier: frontend, phase: test, deployment: v3 }`のように、アプリケーションまたはデプロイメントの__セマンティック属性__を識別する[ラベル](/ja/docs/concepts/overview/working-with-objects/labels/)を定義して使いましょう。これらのラベルを使用して、他のリソースに適切なポッドを選択できます。例えば、すべての`tier：frontend`を持つPodを選択するServiceや、`app：myapp`に属するすべての`phase：test`コンポーネント、などです。このアプローチの例を知るには、[ゲストブック](https://github.com/kubernetes/examples/tree/{{< param "githubbranch" >}}/guestbook/)アプリも合わせてご覧ください。
|
||||
- `{ app: myapp, tier: frontend, phase: test, deployment: v3 }`のように、アプリケーションまたはデプロイメントの __セマンティック属性__ を識別する[ラベル](/ja/docs/concepts/overview/working-with-objects/labels/)を定義して使いましょう。これらのラベルを使用して、他のリソースに適切なポッドを選択できます。例えば、すべての`tier：frontend`を持つPodを選択するServiceや、`app：myapp`に属するすべての`phase：test`コンポーネント、などです。このアプローチの例を知るには、[ゲストブック](https://github.com/kubernetes/examples/tree/{{< param "githubbranch" >}}/guestbook/)アプリも合わせてご覧ください。
|
||||
|
||||
セレクターからリリース固有のラベルを省略することで、Serviceを複数のDeploymentにまたがるように作成できます。 [Deployment](/ja/docs/concepts/workloads/controllers/deployment/)により、ダウンタイムなしで実行中のサービスを簡単に更新できます。
|
||||
|
||||
オブジェクトの望ましい状態はDeploymentによって記述され、その仕様への変更が_適用_されると、Deploymentコントローラは制御された速度で実際の状態を望ましい状態に変更します。
|
||||
オブジェクトの望ましい状態はDeploymentによって記述され、その仕様への変更が _適用_ されると、Deploymentコントローラは制御された速度で実際の状態を望ましい状態に変更します。
|
||||
|
||||
- デバッグ用にラベルを操作できます。Kubernetesコントローラー（ReplicaSetなど）とServiceはセレクターラベルを使用してPodとマッチするため、Podから関連ラベルを削除すると、コントローラーによって考慮されたり、Serviceによってトラフィックを処理されたりすることがなくなります。既存のPodのラベルを削除すると、そのコントローラーはその代わりに新しいPodを作成します。これは、「隔離」環境で以前の「ライブ」Podをデバッグするのに便利な方法です。対話的にラベルを削除または追加するには、[`kubectl label`](/docs/reference/generated/kubectl/kubectl-commands#label)を使います。
|
||||
|
||||
|
||||
@@ -18,7 +18,7 @@ no_list: true
|
||||
|
||||
<!-- body -->
|
||||
|
||||
## コンテナイメージ
|
||||
## コンテナイメージ {#container-images}
|
||||
[コンテナイメージ](/docs/concepts/containers/images/)はすぐに実行可能なソフトウェアパッケージで、アプリケーションの実行に必要なものをすべて含んています。コードと必要なランタイム、アプリケーションとシステムのライブラリ、そして必須な設定項目のデフォルト値を含みます。
|
||||
|
||||
設計上、コンテナは不変で、既に実行中のコンテナのコードを変更することはできません。コンテナ化されたアプリケーションがあり変更したい場合は、変更を含んだ新しいイメージをビルドし、コンテナを再作成して、更新されたイメージから起動する必要があります。
|
||||
|
||||
@@ -140,7 +140,7 @@ RuntimeClassのnodeSelectorはアドミッション機能によりPodのnodeSele
|
||||
|
||||
{{< feature-state for_k8s_version="v1.18" state="beta" >}}
|
||||
|
||||
Podが稼働する時に関連する_オーバーヘッド_リソースを指定できます。オーバーヘッドを宣言すると、クラスター(スケジューラーを含む)がPodとリソースに関する決定を行うときにオーバーヘッドを考慮することができます。
|
||||
Podが稼働する時に関連する _オーバーヘッド_ リソースを指定できます。オーバーヘッドを宣言すると、クラスター(スケジューラーを含む)がPodとリソースに関する決定を行うときにオーバーヘッドを考慮することができます。
|
||||
Podオーバーヘッドを使うためには、PodOverhead[フィーチャーゲート](/ja/docs/reference/command-line-tools-reference/feature-gates/)を有効にしなければなりません。(デフォルトではonです)
|
||||
|
||||
PodのオーバーヘッドはRuntimeClass内の`overhead`フィールドによって定義されます。
|
||||
|
||||
@@ -83,7 +83,7 @@ Nodeにラベルを付与することで、Podは特定のNodeやNodeグルー
|
||||
`NodeRestriction`プラグインは、kubeletが`node-restriction.kubernetes.io/`プレフィックスを有するラベルの設定や上書きを防ぎます。
|
||||
Nodeの隔離にラベルのプレフィックスを使用するためには、以下のようにします。
|
||||
|
||||
1. [Node authorizer](/docs/reference/access-authn-authz/node/)を使用していることと、[NodeRestriction admission plugin](/docs/reference/access-authn-authz/admission-controllers/#noderestriction)が_有効_になっていること。
|
||||
1. [Node authorizer](/docs/reference/access-authn-authz/node/)を使用していることと、[NodeRestriction admission plugin](/docs/reference/access-authn-authz/admission-controllers/#noderestriction)が _有効_ になっていること。
|
||||
2. Nodeに`node-restriction.kubernetes.io/` プレフィックスのラベルを付与し、そのラベルがnode selectorに指定されていること。
|
||||
例えば、`example.com.node-restriction.kubernetes.io/fips=true` または `example.com.node-restriction.kubernetes.io/pci-dss=true`のようなラベルです。
|
||||
|
||||
|
||||
@@ -26,11 +26,11 @@ kube-schedulerは、もし希望するのであれば自分自身でスケジュ
|
||||
kube-schedulerは、新規に作成された各Podや他のスケジューリングされていないPodを稼働させるために最適なNodeを選択します。
|
||||
しかし、Pod内の各コンテナにはそれぞれ異なるリソースの要件があり、各Pod自体にもそれぞれ異なる要件があります。そのため、既存のNodeは特定のスケジューリング要求によってフィルターされる必要があります。
|
||||
|
||||
クラスター内でPodに対する割り当て要求を満たしたNodeは_割り当て可能_ なNodeと呼ばれます。
|
||||
クラスター内でPodに対する割り当て要求を満たしたNodeは _割り当て可能_ なNodeと呼ばれます。
|
||||
もし適切なNodeが一つもない場合、スケジューラーがNodeを割り当てることができるまで、そのPodはスケジュールされずに残ります。
|
||||
|
||||
スケジューラーはPodに対する割り当て可能なNodeをみつけ、それらの割り当て可能なNodeにスコアをつけます。その中から最も高いスコアのNodeを選択し、Podに割り当てるためのいくつかの関数を実行します。
|
||||
スケジューラーは_binding_ と呼ばれる処理中において、APIサーバーに対して割り当てが決まったNodeの情報を通知します。
|
||||
スケジューラーは _binding_ と呼ばれる処理中において、APIサーバーに対して割り当てが決まったNodeの情報を通知します。
|
||||
|
||||
スケジューリングを決定する上で考慮が必要な要素としては、個別または複数のリソース要求や、ハードウェア/ソフトウェアのポリシー制約、affinityやanti-affinityの設定、データの局所性や、ワークロード間での干渉などが挙げられます。
|
||||
|
||||
@@ -52,7 +52,7 @@ _スコアリング_ ステップでは、Podを割り当てるのに最も適
|
||||
|
||||
スケジューラーのフィルタリングとスコアリングの動作に関する設定には2つのサポートされた手法があります。
|
||||
|
||||
1. [スケジューリングポリシー](/docs/reference/scheduling/policies) は、フィルタリングのための_Predicates_とスコアリングのための_Priorities_の設定することができます。
|
||||
1. [スケジューリングポリシー](/docs/reference/scheduling/policies) は、フィルタリングのための _Predicates_ とスコアリングのための _Priorities_ の設定することができます。
|
||||
1. [スケジューリングプロファイル](/docs/reference/scheduling/config/#profiles)は、`QueueSort`、 `Filter`、 `Score`、 `Bind`、 `Reserve`、 `Permit`やその他を含む異なるスケジューリングの段階を実装するプラグインを設定することができます。kube-schedulerを異なるプロファイルを実行するように設定することもできます。
|
||||
|
||||
|
||||
|
||||
@@ -95,7 +95,7 @@ Serviceは多くの場合、KubernetesのPodに対するアクセスを抽象化
|
||||
* Serviceを、異なる{{< glossary_tooltip term_id="namespace" >}}のServiceや他のクラスターのServiceに向ける場合
|
||||
* ワークロードをKubernetesに移行するとき、アプリケーションに対する処理をしながら、バックエンドの一部をKubernetesで実行する場合
|
||||
|
||||
このような場合において、ユーザーはPodセレクター_なしで_ Serviceを定義できます。
|
||||
このような場合において、ユーザーはPodセレクター*なしで*Serviceを定義できます。
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
@@ -882,7 +882,7 @@ Kubernetesは各Serviceに、それ自身のIPアドレスを割り当てるこ
|
||||
### ServiceのIPアドレス {#ips-and-vips}
|
||||
|
||||
実際に固定された向き先であるPodのIPアドレスとは異なり、ServiceのIPは実際には単一のホストによって応答されません。
|
||||
その代わり、kube-proxyは必要な時に透過的にリダイレクトされる_仮想_ IPアドレスを定義するため、iptables(Linuxのパケット処理ロジック)を使用します。
|
||||
その代わり、kube-proxyは必要な時に透過的にリダイレクトされる*仮想*IPアドレスを定義するため、iptables(Linuxのパケット処理ロジック)を使用します。
|
||||
クライアントがVIPに接続する時、そのトラフィックは自動的に適切なEndpointsに転送されます。
|
||||
Service用の環境変数とDNSは、Serviceの仮想IPアドレス(とポート)の面において、自動的に生成されます。
|
||||
|
||||
|
||||
+1
-1
@@ -232,7 +232,7 @@ contexts:
|
||||
|
||||
上記の設定ファイルは、`dev-ramp-up`というコンテキストを表します。
|
||||
|
||||
## KUBECONFIG環境変数を設定する
|
||||
## KUBECONFIG環境変数を設定する {#set-the-kubeconfig-environment-variable}
|
||||
|
||||
`KUBECONFIG`という環境変数が存在するかを確認してください。もし存在する場合は、後で復元できるようにバックアップしてください。例えば:
|
||||
|
||||
|
||||
@@ -55,7 +55,7 @@ kubectl proxy
|
||||
|
||||
kubectlは、ダッシュボードを http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/ で利用できるようにします。
|
||||
|
||||
UIはコマンドを実行しているマシンから_のみ_ アクセスできます。オプションについては`kubectl proxy --help`を参照してください。
|
||||
UIはコマンドを実行しているマシンから _のみ_ アクセスできます。オプションについては`kubectl proxy --help`を参照してください。
|
||||
|
||||
{{< note >}}
|
||||
Kubeconfigの認証方法は、外部IDプロバイダーやx509証明書ベースの認証には対応していません。
|
||||
|
||||
@@ -0,0 +1,147 @@
|
||||
---
|
||||
title: 静的な処理の割り当てを使用した並列処理のためのインデックス付きJob
|
||||
content_type: task
|
||||
min-kubernetes-server-version: v1.21
|
||||
weight: 30
|
||||
---
|
||||
|
||||
{{< feature-state for_k8s_version="v1.21" state="alpha" >}}
|
||||
|
||||
<!-- overview -->
|
||||
|
||||
この例では、複数の並列ワーカープロセスを使用するKubernetesのJobを実行します。各ワーカーは、それぞれが自分のPod内で実行される異なるコンテナです。Podはコントロールプレーンが自動的に設定する*インデックス値*を持ち、この値を利用することで、各Podは処理するタスク全体のどの部分を処理するのかを特定できます。
|
||||
|
||||
Podのインデックスは、{{< glossary_tooltip text="アノテーション" term_id="annotation" >}}内の`batch.kubernetes.io/job-completion-index`を整数値の文字列表現として利用できます。コンテナ化されたタスクプロセスがこのインデックスを取得できるようにするために、このアノテーションの値は[downward API](/docs/tasks/inject-data-application/downward-api-volume-expose-pod-information/#the-downward-api)の仕組みを利用することで公開できます。利便性のために、コントロールプレーンは自動的にdownward APIを設定して、`JOB_COMPLETION_INDEX`環境変数にインデックスを公開します。
|
||||
|
||||
以下に、この例で実行するステップの概要を示します。
|
||||
|
||||
1. **completionのインデックスを使用してJobのマニフェストを定義する**。downward APIはPodのインデックスのアノテーションを環境変数またはファイルとしてコンテナに渡してくれます。
|
||||
2. **そのマニフェストに基づいてインデックス付き(Indexed)のJobを開始する**。
|
||||
|
||||
## {{% heading "prerequisites" %}}
|
||||
|
||||
あらかじめ基本的な非並列の[Job](/docs/concepts/workloads/controllers/job/)の使用に慣れている必要があります。
|
||||
|
||||
{{< include "task-tutorial-prereqs.md" >}} {{< version-check >}}
|
||||
|
||||
インデックス付きJobを作成できるようにするには、[APIサーバー](/docs/reference/command-line-tools-reference/kube-apiserver/)と[コントローラーマネージャー](/docs/reference/command-line-tools-reference/kube-controller-manager/)上で`IndexedJob`[フィーチャーゲート](/ja/docs/reference/command-line-tools-reference/feature-gates/)を有効にしていることを確認してください。
|
||||
|
||||
<!-- steps -->
|
||||
|
||||
## アプローチを選択する
|
||||
|
||||
ワーカープログラムから処理アイテムにアクセスするには、いくつかの選択肢があります。
|
||||
|
||||
1. `JOB_COMPLETION_INDEX`環境変数を読み込む。Job{{< glossary_tooltip text="コントローラー" term_id="controller" >}}は、この変数をcompletion indexを含むアノテーションに自動的にリンクします。
|
||||
1. completion indexを含むファイルを読み込む。
|
||||
1. プログラムを修正できない場合、プログラムをスクリプトでラップし、上のいずれかの方法でインデックスを読み取り、プログラムが入力として使用できるものに変換する。
|
||||
|
||||
この例では、3番目のオプションを選択肢して、[rev](https://man7.org/linux/man-pages/man1/rev.1.html)ユーティリティを実行したいと考えているとしましょう。このプログラムはファイルを引数として受け取り、内容を逆さまに表示します。
|
||||
|
||||
```shell
|
||||
rev data.txt
|
||||
```
|
||||
|
||||
`rev`ツールは[`busybox`](https://hub.docker.com/_/busybox)コンテナイメージから利用できます。
|
||||
|
||||
これは単なる例であるため、各Podはごく簡単な処理(短い文字列を逆にする)をするだけです。現実のワークロードでは、たとえば、シーンデータをもとに60秒の動画を生成するというようなタスクを記述したJobを作成するかもしれません。ビデオレンダリングJobの各処理アイテムは、ビデオクリップの特定のフレームのレンダリングを行うものになるでしょう。その場合、インデックス付きの完了が意味するのは、クリップの最初からフレームをカウントすることで、Job内の各Podがレンダリングと公開をするのがどのフレームであるかがわかるということです。
|
||||
|
||||
## インデックス付きJobを定義する
|
||||
|
||||
以下は、completion modeとして`Indexed`を使用するJobのマニフェストの例です。
|
||||
|
||||
{{< codenew language="yaml" file="application/job/indexed-job.yaml" >}}
|
||||
|
||||
上記の例では、Jobコントローラーがすべてのコンテナに設定する組み込みの`JOB_COMPLETION_INDEX`環境変数を使っています。[initコンテナ](/ja/docs/concepts/workloads/pods/init-containers/)がインデックスを静的な値にマッピングし、その値をファイルに書き込み、ファイルを[emptyDir volume](/docs/concepts/storage/volumes/#emptydir)を介してワーカーを実行しているコンテナと共有します。オプションとして、インデックスとコンテナに公開するために[downward APIを使用して独自の環境変数を定義する](/ja/docs/tasks/inject-data-application/environment-variable-expose-pod-information/)こともできます。[環境変数やファイルとして設定したConfigMap](/ja/docs/tasks/configure-pod-container/configure-pod-configmap/)から値のリストを読み込むという選択肢もあります。
|
||||
|
||||
他には、以下の例のように、直接[downward APIを使用してアノテーションの値をボリュームファイルとして渡す](/docs/tasks/inject-data-application/downward-api-volume-expose-pod-information/#store-pod-fields)こともできます。
|
||||
|
||||
{{< codenew language="yaml" file="application/job/indexed-job-vol.yaml" >}}
|
||||
|
||||
## Jobを実行する
|
||||
|
||||
次のコマンドでJobを実行します。
|
||||
|
||||
```shell
|
||||
# このコマンドでは1番目のアプローチを使っています ($JOB_COMPLETION_INDEX に依存しています)
|
||||
kubectl apply -f https://kubernetes.io/examples/application/job/indexed-job.yaml
|
||||
```
|
||||
|
||||
このJobを作成したら、コントロールプレーンは指定した各インデックスごとに一連のPodを作成します。`.spec.parallelism`の値が同時に実行できるPodの数を決定し、`.spec.completions`の値がJobが作成するPodの合計数を決定します。
|
||||
|
||||
`.spec.parallelism`は`.spec.completions`より小さいため、コントロールプレーンは別のPodを開始する前に最初のPodの一部が完了するまで待機します。
|
||||
|
||||
Jobを作成したら、少し待ってから進行状況を確認します。
|
||||
|
||||
```shell
|
||||
kubectl describe jobs/indexed-job
|
||||
```
|
||||
|
||||
出力は次のようになります。
|
||||
|
||||
```
|
||||
Name: indexed-job
|
||||
Namespace: default
|
||||
Selector: controller-uid=bf865e04-0b67-483b-9a90-74cfc4c3e756
|
||||
Labels: controller-uid=bf865e04-0b67-483b-9a90-74cfc4c3e756
|
||||
job-name=indexed-job
|
||||
Annotations: <none>
|
||||
Parallelism: 3
|
||||
Completions: 5
|
||||
Start Time: Thu, 11 Mar 2021 15:47:34 +0000
|
||||
Pods Statuses: 2 Running / 3 Succeeded / 0 Failed
|
||||
Completed Indexes: 0-2
|
||||
Pod Template:
|
||||
Labels: controller-uid=bf865e04-0b67-483b-9a90-74cfc4c3e756
|
||||
job-name=indexed-job
|
||||
Init Containers:
|
||||
input:
|
||||
Image: docker.io/library/bash
|
||||
Port: <none>
|
||||
Host Port: <none>
|
||||
Command:
|
||||
bash
|
||||
-c
|
||||
items=(foo bar baz qux xyz)
|
||||
echo ${items[$JOB_COMPLETION_INDEX]} > /input/data.txt
|
||||
|
||||
Environment: <none>
|
||||
Mounts:
|
||||
/input from input (rw)
|
||||
Containers:
|
||||
worker:
|
||||
Image: docker.io/library/busybox
|
||||
Port: <none>
|
||||
Host Port: <none>
|
||||
Command:
|
||||
rev
|
||||
/input/data.txt
|
||||
Environment: <none>
|
||||
Mounts:
|
||||
/input from input (rw)
|
||||
Volumes:
|
||||
input:
|
||||
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
|
||||
Medium:
|
||||
SizeLimit: <unset>
|
||||
Events:
|
||||
Type Reason Age From Message
|
||||
---- ------ ---- ---- -------
|
||||
Normal SuccessfulCreate 4s job-controller Created pod: indexed-job-njkjj
|
||||
Normal SuccessfulCreate 4s job-controller Created pod: indexed-job-9kd4h
|
||||
Normal SuccessfulCreate 4s job-controller Created pod: indexed-job-qjwsz
|
||||
Normal SuccessfulCreate 1s job-controller Created pod: indexed-job-fdhq5
|
||||
Normal SuccessfulCreate 1s job-controller Created pod: indexed-job-ncslj
|
||||
```
|
||||
|
||||
この例では、各インデックスごとにカスタムの値を使用してJobを実行します。次のコマンドでPodの1つの出力を確認できます。
|
||||
|
||||
```shell
|
||||
kubectl logs indexed-job-fdhq5 # これを対象のJobのPodの名前に一致するように変更してください。
|
||||
```
|
||||
|
||||
出力は次のようになります。
|
||||
|
||||
```
|
||||
xuq
|
||||
```
|
||||
@@ -0,0 +1,5 @@
|
||||
---
|
||||
title: "Kubernetesオブジェクトの管理"
|
||||
description: Kubernetes APIと対話するための宣言型および命令型のパラダイム。
|
||||
weight: 25
|
||||
---
|
||||
Executable
+6
@@ -0,0 +1,6 @@
|
||||
---
|
||||
title: "ネットワーク"
|
||||
description: クラスターのネットワークの設定方法を学びます。
|
||||
weight: 160
|
||||
---
|
||||
|
||||
@@ -0,0 +1,232 @@
|
||||
---
|
||||
min-kubernetes-server-version: v1.20
|
||||
title: IPv4/IPv6デュアルスタックの検証
|
||||
content_type: task
|
||||
---
|
||||
|
||||
<!-- overview -->
|
||||
このドキュメントでは、IPv4/IPv6デュアルスタックが有効化されたKubernetesクラスターを検証する方法について共有します。
|
||||
|
||||
|
||||
## {{% heading "prerequisites" %}}
|
||||
|
||||
|
||||
* プロバイダーがデュアルスタックのネットワークをサポートしていること (クラウドプロバイダーか、ルーティングできるIPv4/IPv6ネットワークインターフェイスを持つKubernetesノードが提供できること)
|
||||
* (KubenetやCalicoなど)デュアルスタックをサポートする[ネットワークプラグイン](/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins/)
|
||||
* [デュアルスタックを有効化](/ja/docs/concepts/services-networking/dual-stack/)したクラスター
|
||||
|
||||
{{< version-check >}}
|
||||
|
||||
|
||||
|
||||
<!-- steps -->
|
||||
|
||||
## アドレスの検証
|
||||
|
||||
### ノードアドレスの検証
|
||||
|
||||
各デュアルスタックのノードは、1つのIPv4ブロックと1つのIPv6ブロックを割り当てる必要があります。IPv4/IPv6のPodアドレスの範囲が設定されていることを検証するには、次のコマンドを実行します。例の中のノード名は、自分のクラスターの有効なデュアルスタックのノードの名前に置換してください。この例では、ノードの名前は`k8s-linuxpool1-34450317-0`になっています。
|
||||
|
||||
```shell
|
||||
kubectl get nodes k8s-linuxpool1-34450317-0 -o go-template --template='{{range .spec.podCIDRs}}{{printf "%s\n" .}}{{end}}'
|
||||
```
|
||||
```
|
||||
10.244.1.0/24
|
||||
a00:100::/24
|
||||
```
|
||||
|
||||
IPv4ブロックとIPv6ブロックがそれぞれ1つずつ割り当てられているはずです。
|
||||
|
||||
ノードが検出されたIPv4とIPv6のインターフェイスを持っていることを検証します。ノード名は自分のクラスター内の有効なノード名に置換してください。この例では、ノード名は`k8s-linuxpool1-34450317-0`になっています。
|
||||
|
||||
```shell
|
||||
kubectl get nodes k8s-linuxpool1-34450317-0 -o go-template --template='{{range .status.addresses}}{{printf "%s: %s\n" .type .address}}{{end}}'
|
||||
```
|
||||
```
|
||||
Hostname: k8s-linuxpool1-34450317-0
|
||||
InternalIP: 10.240.0.5
|
||||
InternalIP: 2001:1234:5678:9abc::5
|
||||
```
|
||||
|
||||
### Podアドレスの検証
|
||||
|
||||
PodにIPv4とIPv6のアドレスが割り当てられていることを検証します。Podの名前は自分のクラスター内の有効なPodの名前と置換してください。この例では、Podの名前は`pod01`になっています。
|
||||
|
||||
```shell
|
||||
kubectl get pods pod01 -o go-template --template='{{range .status.podIPs}}{{printf "%s\n" .ip}}{{end}}'
|
||||
```
|
||||
```
|
||||
10.244.1.4
|
||||
a00:100::4
|
||||
```
|
||||
|
||||
Downward APIを使用して、`status.podIPs`のfieldPath経由でPod IPを検証することもできます。次のスニペットは、Pod IPを`MY_POD_IPS`という名前の環境変数経由でコンテナ内に公開する方法を示しています。
|
||||
|
||||
```
|
||||
env:
|
||||
- name: MY_POD_IPS
|
||||
valueFrom:
|
||||
fieldRef:
|
||||
fieldPath: status.podIPs
|
||||
```
|
||||
|
||||
次のコマンドを実行すると、`MY_POD_IPS`環境変数の値をコンテナ内から表示できます。値はカンマ区切りのリストであり、PodのIPv4とIPv6のアドレスに対応しています。
|
||||
|
||||
```shell
|
||||
kubectl exec -it pod01 -- set | grep MY_POD_IPS
|
||||
```
|
||||
```
|
||||
MY_POD_IPS=10.244.1.4,a00:100::4
|
||||
```
|
||||
|
||||
PodのIPアドレスは、コンテナ内の`/etc/hosts`にも書き込まれます。次のコマンドは、デュアルスタックのPod上で`/etc/hosts`に対してcatコマンドを実行します。出力を見ると、Pod用のIPv4およびIPv6のIPアドレスの両方が確認できます。
|
||||
|
||||
```shell
|
||||
kubectl exec -it pod01 -- cat /etc/hosts
|
||||
```
|
||||
```
|
||||
# Kubernetes-managed hosts file.
|
||||
127.0.0.1 localhost
|
||||
::1 localhost ip6-localhost ip6-loopback
|
||||
fe00::0 ip6-localnet
|
||||
fe00::0 ip6-mcastprefix
|
||||
fe00::1 ip6-allnodes
|
||||
fe00::2 ip6-allrouters
|
||||
10.244.1.4 pod01
|
||||
a00:100::4 pod01
|
||||
```
|
||||
|
||||
## Serviceの検証
|
||||
|
||||
`.spec.isFamilyPolicy`を明示的に定義していない、以下のようなServiceを作成してみます。Kubernetesは最初に設定した`service-cluster-ip-range`の範囲からServiceにcluster IPを割り当てて、`.spec.ipFamilyPolicy`を`SingleStack`に設定します。
|
||||
|
||||
{{< codenew file="service/networking/dual-stack-default-svc.yaml" >}}
|
||||
|
||||
`kubectl`を使ってServiceのYAMLを表示します。
|
||||
|
||||
```shell
|
||||
kubectl get svc my-service -o yaml
|
||||
```
|
||||
|
||||
Serviceの`.spec.ipFamilyPolicy`は`SingleStack`に設定され、`.spec.clusterIP`にはkube-controller-manager上の`--service-cluster-ip-range`フラグで最初に設定した範囲から1つのIPv4アドレスが設定されているのがわかります。
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: my-service
|
||||
namespace: default
|
||||
spec:
|
||||
clusterIP: 10.0.217.164
|
||||
clusterIPs:
|
||||
- 10.0.217.164
|
||||
ipFamilies:
|
||||
- IPv4
|
||||
ipFamilyPolicy: SingleStack
|
||||
ports:
|
||||
- port: 80
|
||||
protocol: TCP
|
||||
targetPort: 9376
|
||||
selector:
|
||||
app: MyApp
|
||||
sessionAffinity: None
|
||||
type: ClusterIP
|
||||
status:
|
||||
loadBalancer: {}
|
||||
```
|
||||
|
||||
`.spec.ipFamilies`内の配列の1番目の要素に`IPv6`を明示的に指定した、次のようなServiceを作成してみます。Kubernetesは`service-cluster-ip-range`で設定したIPv6の範囲からcluster IPを割り当てて、`.spec.ipFamilyPolicy`を`SingleStack`に設定します。
|
||||
|
||||
{{< codenew file="service/networking/dual-stack-ipfamilies-ipv6.yaml" >}}
|
||||
|
||||
`kubectl`を使ってServiceのYAMLを表示します。
|
||||
|
||||
```shell
|
||||
kubectl get svc my-service -o yaml
|
||||
```
|
||||
|
||||
Serviceの`.spec.ipFamilyPolicy`は`SingleStack`に設定され、`.spec.clusterIP`には、kube-controller-manager上の`--service-cluster-ip-range`フラグで指定された最初の設定範囲から1つのIPv6アドレスが設定されているのがわかります。
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
labels:
|
||||
app: MyApp
|
||||
name: my-service
|
||||
spec:
|
||||
clusterIP: fd00::5118
|
||||
clusterIPs:
|
||||
- fd00::5118
|
||||
ipFamilies:
|
||||
- IPv6
|
||||
ipFamilyPolicy: SingleStack
|
||||
ports:
|
||||
- port: 80
|
||||
protocol: TCP
|
||||
targetPort: 80
|
||||
selector:
|
||||
app: MyApp
|
||||
sessionAffinity: None
|
||||
type: ClusterIP
|
||||
status:
|
||||
loadBalancer: {}
|
||||
```
|
||||
|
||||
`.spec.ipFamiliePolicy`に`PreferDualStack`を明示的に指定した、次のようなServiceを作成してみます。Kubernetesは(クラスターでデュアルスタックを有効化しているため)IPv4およびIPv6のアドレスの両方を割り当て、`.spec.ClusterIPs`のリストから、`.spec.ipFamilies`配列の最初の要素のアドレスファミリーに基づいた`.spec.ClusterIP`を設定します。
|
||||
|
||||
{{< codenew file="service/networking/dual-stack-preferred-svc.yaml" >}}
|
||||
|
||||
{{< note >}}
|
||||
`kubectl get svc`コマンドは、`CLUSTER-IP`フィールドにプライマリーのIPだけしか表示しません。
|
||||
|
||||
```shell
|
||||
kubectl get svc -l app=MyApp
|
||||
|
||||
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
|
||||
my-service ClusterIP 10.0.216.242 <none> 80/TCP 5s
|
||||
```
|
||||
{{< /note >}}
|
||||
|
||||
`kubectl describe`を使用して、ServiceがIPv4およびIPv6アドレスのブロックからcluster IPを割り当てられていることを検証します。その後、ServiceにIPアドレスとポートを使用してアクセスできることを検証することもできます。
|
||||
|
||||
```shell
|
||||
kubectl describe svc -l app=MyApp
|
||||
```
|
||||
|
||||
```
|
||||
Name: my-service
|
||||
Namespace: default
|
||||
Labels: app=MyApp
|
||||
Annotations: <none>
|
||||
Selector: app=MyApp
|
||||
Type: ClusterIP
|
||||
IP Family Policy: PreferDualStack
|
||||
IP Families: IPv4,IPv6
|
||||
IP: 10.0.216.242
|
||||
IPs: 10.0.216.242,fd00::af55
|
||||
Port: <unset> 80/TCP
|
||||
TargetPort: 9376/TCP
|
||||
Endpoints: <none>
|
||||
Session Affinity: None
|
||||
Events: <none>
|
||||
```
|
||||
|
||||
### デュアルスタックのLoadBalancer Serviceを作成する
|
||||
|
||||
クラウドプロバイダーがIPv6を有効化した外部ロードバランサーのプロビジョニングをサポートする場合、`.spec.ipFamilyPolicy`に`PreferDualStack`を指定し、`.spec.ipFamilies`の最初の要素を`IPv6`にして、`type`フィールドに`LoadBalancer`を指定したServiceを作成できます。
|
||||
|
||||
{{< codenew file="service/networking/dual-stack-prefer-ipv6-lb-svc.yaml" >}}
|
||||
|
||||
Serviceを確認します。
|
||||
|
||||
```shell
|
||||
kubectl get svc -l app=MyApp
|
||||
```
|
||||
|
||||
ServiceがIPv6アドレスブロックから`CLUSTER-IP`のアドレスと`EXTERNAL-IP`を割り当てられていることを検証します。その後、IPとポートを用いたServiceへのアクセスを検証することもできます。
|
||||
|
||||
```shell
|
||||
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
|
||||
my-service LoadBalancer fd00::7ebc 2603:1030:805::5 80:30790/TCP 35s
|
||||
```
|
||||
@@ -392,7 +392,7 @@ client_address=198.51.100.79
|
||||
|
||||
2. クライアントからロードバランサーのVIPに送信されたリクエストが、中間のプロキシーではなく、クライアントの送信元IPとともにノードまで到達するようなパケット転送が使用される。
|
||||
|
||||
1つめのカテゴリーのロードバランサーの場合、真のクライアントIPと通信するために、 HTTPの[Forwarded](https://tools.ietf.org/html/rfc7239#section-5.2)ヘッダーや[X-FORWARDED-FOR](https://ja.wikipedia.org/wiki/X-Forwarded-For)ヘッダー、[proxy protocol](https://www.haproxy.org/download/1.5/doc/proxy-protocol.txt)などの、ロードバランサーとバックエンドの間で合意されたプロトコルを使用する必要があります。2つ目のカテゴリーのロードバランサーの場合、Serviceの`service.spec.healthCheckNodePort`フィールドに保存されたポートを指すHTTPのヘルスチェックを作成することで、上記の機能を活用できます。
|
||||
1つめのカテゴリーのロードバランサーの場合、真のクライアントIPと通信するために、 HTTPの[Forwarded](https://tools.ietf.org/html/rfc7239#section-5.2)ヘッダーや[X-FORWARDED-FOR](https://ja.wikipedia.org/wiki/X-Forwarded-For)ヘッダー、[proxy protocol](https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt)などの、ロードバランサーとバックエンドの間で合意されたプロトコルを使用する必要があります。2つ目のカテゴリーのロードバランサーの場合、Serviceの`service.spec.healthCheckNodePort`フィールドに保存されたポートを指すHTTPのヘルスチェックを作成することで、上記の機能を活用できます。
|
||||
|
||||
## {{% heading "cleanup" %}}
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
apiVersion: batch/v1beta1
|
||||
apiVersion: batch/v1
|
||||
kind: CronJob
|
||||
metadata:
|
||||
name: hello
|
||||
|
||||
@@ -0,0 +1,27 @@
|
||||
apiVersion: batch/v1
|
||||
kind: Job
|
||||
metadata:
|
||||
name: 'indexed-job'
|
||||
spec:
|
||||
completions: 5
|
||||
parallelism: 3
|
||||
completionMode: Indexed
|
||||
template:
|
||||
spec:
|
||||
restartPolicy: Never
|
||||
containers:
|
||||
- name: 'worker'
|
||||
image: 'docker.io/library/busybox'
|
||||
command:
|
||||
- "rev"
|
||||
- "/input/data.txt"
|
||||
volumeMounts:
|
||||
- mountPath: /input
|
||||
name: input
|
||||
volumes:
|
||||
- name: input
|
||||
downwardAPI:
|
||||
items:
|
||||
- path: "data.txt"
|
||||
fieldRef:
|
||||
fieldPath: metadata.annotations['batch.kubernetes.io/job-completion-index']
|
||||
@@ -0,0 +1,35 @@
|
||||
apiVersion: batch/v1
|
||||
kind: Job
|
||||
metadata:
|
||||
name: 'indexed-job'
|
||||
spec:
|
||||
completions: 5
|
||||
parallelism: 3
|
||||
completionMode: Indexed
|
||||
template:
|
||||
spec:
|
||||
restartPolicy: Never
|
||||
initContainers:
|
||||
- name: 'input'
|
||||
image: 'docker.io/library/bash'
|
||||
command:
|
||||
- "bash"
|
||||
- "-c"
|
||||
- |
|
||||
items=(foo bar baz qux xyz)
|
||||
echo ${items[$JOB_COMPLETION_INDEX]} > /input/data.txt
|
||||
volumeMounts:
|
||||
- mountPath: /input
|
||||
name: input
|
||||
containers:
|
||||
- name: 'worker'
|
||||
image: 'docker.io/library/busybox'
|
||||
command:
|
||||
- "rev"
|
||||
- "/input/data.txt"
|
||||
volumeMounts:
|
||||
- mountPath: /input
|
||||
name: input
|
||||
volumes:
|
||||
- name: input
|
||||
emptyDir: {}
|
||||
@@ -2,10 +2,11 @@ apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: my-service
|
||||
labels:
|
||||
app: MyApp
|
||||
spec:
|
||||
selector:
|
||||
app: MyApp
|
||||
ports:
|
||||
- protocol: TCP
|
||||
port: 80
|
||||
targetPort: 9376
|
||||
@@ -0,0 +1,14 @@
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: my-service
|
||||
labels:
|
||||
app: MyApp
|
||||
spec:
|
||||
ipFamilies:
|
||||
- IPv6
|
||||
selector:
|
||||
app: MyApp
|
||||
ports:
|
||||
- protocol: TCP
|
||||
port: 80
|
||||
@@ -0,0 +1,16 @@
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: my-service
|
||||
labels:
|
||||
app: MyApp
|
||||
spec:
|
||||
ipFamilyPolicy: PreferDualStack
|
||||
ipFamilies:
|
||||
- IPv6
|
||||
type: LoadBalancer
|
||||
selector:
|
||||
app: MyApp
|
||||
ports:
|
||||
- protocol: TCP
|
||||
port: 80
|
||||
@@ -0,0 +1,16 @@
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: my-service
|
||||
labels:
|
||||
app: MyApp
|
||||
spec:
|
||||
ipFamilyPolicy: PreferDualStack
|
||||
ipFamilies:
|
||||
- IPv6
|
||||
- IPv4
|
||||
selector:
|
||||
app: MyApp
|
||||
ports:
|
||||
- protocol: TCP
|
||||
port: 80
|
||||
@@ -0,0 +1,13 @@
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: my-service
|
||||
labels:
|
||||
app: MyApp
|
||||
spec:
|
||||
ipFamilyPolicy: PreferDualStack
|
||||
selector:
|
||||
app: MyApp
|
||||
ports:
|
||||
- protocol: TCP
|
||||
port: 80
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 7.6 KiB |
@@ -1,86 +0,0 @@
|
||||
---
|
||||
title: Adform Case Study
|
||||
linkTitle: Adform
|
||||
case_study_styles: true
|
||||
cid: caseStudies
|
||||
logo: adform_featured_logo.png
|
||||
draft: false
|
||||
featured: true
|
||||
weight: 47
|
||||
quote: >
|
||||
Kubernetes enabled the self-healing and immutable infrastructure. We can do faster releases, so our developers are really happy. They can ship our features faster than before, and that makes our clients happier.
|
||||
|
||||
new_case_study_styles: true
|
||||
heading_background: /images/case-studies/adform/banner1.jpg
|
||||
heading_title_logo: /images/adform_logo.png
|
||||
subheading: >
|
||||
Improving Performance and Morale with Cloud Native
|
||||
case_study_details:
|
||||
- Company: AdForm
|
||||
- Location: Copenhagen, Denmark
|
||||
- Industry: Adtech
|
||||
---
|
||||
|
||||
<h2>Challenge</h2>
|
||||
|
||||
<p><a href="https://site.adform.com/">Adform's</a> mission is to provide a secure and transparent full stack of advertising technology to enable digital ads across devices. The company has a large infrastructure: <a href="https://www.openstack.org/">OpenStack</a>-based private clouds running on 1,100 physical servers in 7 data centers around the world, 3 of which were opened in the past year. With the company's growth, the infrastructure team felt that "our private cloud was not really flexible enough," says IT System Engineer Edgaras Apšega. "The biggest pain point is that our developers need to maintain their virtual machines, so rolling out technology and new software takes time. We were really struggling with our releases, and we didn't have self-healing infrastructure."</p>
|
||||
|
||||
<h2>Solution</h2>
|
||||
|
||||
<p>The team, which had already been using <a href="https://prometheus.io/">Prometheus</a> for monitoring, embraced <a href="https://kubernetes.io/">Kubernetes</a> and cloud native practices in 2017. "To start our Kubernetes journey, we had to adapt all our software, so we had to choose newer frameworks," says Apšega. "We also adopted the microservices way, so observability is much better because you can inspect the bug or the services separately."</p>
|
||||
|
||||
<h2>Impact</h2>
|
||||
|
||||
<p>"Kubernetes helps our business a lot because our features are coming to market faster," says Apšega. The release process went from several hours to several minutes. Autoscaling has been at least 6 times faster than the semi-manual VM bootstrapping and application deployment required before. The team estimates that the company has experienced cost savings of 4-5x due to less hardware and fewer man hours needed to set up the hardware and virtual machines, metrics, and logging. Utilization of the hardware resources has been reduced as well, with containers notching 2-3 times more efficiency over virtual machines. "The deployments are very easy because developers just push the code and it automatically appears on Kubernetes," says Apšega. Prometheus has also had a positive impact: "It provides high availability for metrics and alerting. We monitor everything starting from hardware to applications. Having all the metrics in <a href="https://grafana.com/">Grafana</a> dashboards provides great insight on your systems."</p>
|
||||
|
||||
{{< case-studies/quote author="Edgaras Apšega, IT Systems Engineer, Adform" >}}
|
||||
"Kubernetes enabled the self-healing and immutable infrastructure. We can do faster releases, so our developers are really happy. They can ship our features faster than before, and that makes our clients happier."
|
||||
{{< /case-studies/quote >}}
|
||||
|
||||
{{< case-studies/lead >}}
|
||||
Adform made <a href="https://www.wsj.com/articles/fake-ad-operation-used-to-steal-from-publishers-is-uncovered-1511290981">headlines</a> last year when it detected the HyphBot ad fraud network that was costing some businesses hundreds of thousands of dollars a day.
|
||||
{{< /case-studies/lead >}}
|
||||
|
||||
<p>With its mission to provide a secure and transparent full stack of advertising technology to enable an open internet, Adform published a <a href="https://site.adform.com/media/85132/hyphbot_whitepaper_.pdf">white paper</a> revealing what it did—and others could too—to limit customers' exposure to the scam.</p>
|
||||
|
||||
<p>In that same spirit, Adform is sharing its cloud native journey. "When you see that everyone shares their best practices, it inspires you to contribute back to the project," says IT Systems Engineer Edgaras Apšega.</p>
|
||||
|
||||
<p>The company has a large infrastructure: <a href="https://www.openstack.org/">OpenStack</a>-based private clouds running on 1,100 physical servers in their own seven data centers around the world, three of which were opened in the past year. With the company's growth, the infrastructure team felt that "our private cloud was not really flexible enough," says Apšega. "The biggest pain point is that our developers need to maintain their virtual machines, so rolling out technology and new software really takes time. We were really struggling with our releases, and we didn't have self-healing infrastructure."</p>
|
||||
|
||||
{{< case-studies/quote
|
||||
image="/images/case-studies/adform/banner3.jpg"
|
||||
author="Edgaras Apšega, IT Systems Engineer, Adform"
|
||||
>}}
|
||||
"The fact that Cloud Native Computing Foundation incubated Kubernetes was a really big point for us because it was vendor neutral. And we can see that a community really gathers around it. Everyone shares their experiences, their knowledge, and the fact that it's open source, you can contribute."
|
||||
{{< /case-studies/quote >}}
|
||||
|
||||
<p>The team, which had already been using Prometheus for monitoring, embraced Kubernetes, microservices, and cloud native practices. "The fact that Cloud Native Computing Foundation incubated Kubernetes was a really big point for us because it was vendor neutral," says Apšega. "And we can see that a community really gathers around it."</p>
|
||||
|
||||
<p>A proof of concept project was started, with a Kubernetes cluster running on bare metal in the data center. When developers saw how quickly containers could be spun up compared to the virtual machine process, "they wanted to ship their containers in production right away, and we were still doing proof of concept," says IT Systems Engineer Andrius Cibulskis.</p>
|
||||
|
||||
<p>Of course, a lot of work still had to be done. "First of all, we had to learn Kubernetes, see all of the moving parts, how they glue together," says Apšega. "Second of all, the whole CI/CD part had to be redone, and our DevOps team had to invest more man hours to implement it. And third is that developers had to rewrite the code, and they're still doing it."</p>
|
||||
|
||||
<p>The first production cluster was launched in the spring of 2018, and is now up to 20 physical machines dedicated for pods throughout three data centers, with plans for separate clusters in the other four data centers. The user-facing Adform application platform, data distribution platform, and back ends are now all running on Kubernetes. "Many APIs for critical applications are being developed for Kubernetes," says Apšega. "Teams are rewriting their applications to .NET core, because it supports containers, and preparing to move to Kubernetes. And new applications, by default, go in containers."</p>
|
||||
|
||||
{{< case-studies/quote
|
||||
image="/images/case-studies/adform/banner4.jpg"
|
||||
author="Andrius Cibulskis, IT Systems Engineer, Adform"
|
||||
>}}
|
||||
"Releases are really nice for them, because they just push their code to Git and that's it. They don't have to worry about their virtual machines anymore."
|
||||
{{< /case-studies/quote >}}
|
||||
|
||||
<p>This big push has been driven by the real impact that these new practices have had. "Kubernetes helps our business a lot because our features are coming to market faster," says Apšega. "The deployments are very easy because developers just push the code and it automatically appears on Kubernetes." The release process went from several hours to several minutes. Autoscaling is at least six times faster than the semi-manual VM bootstrapping and application deployment required before.</p>
|
||||
|
||||
<p>The team estimates that the company has experienced cost savings of 4-5x due to less hardware and fewer man hours needed to set up the hardware and virtual machines, metrics, and logging. Utilization of the hardware resources has been reduced as well, with containers notching two to three times more efficiency over virtual machines.</p>
|
||||
|
||||
<p>Prometheus has also had a positive impact: "It provides high availability for metrics and alerting," says Apšega. "We monitor everything starting from hardware to applications. Having all the metrics in Grafana dashboards provides great insight on our systems."</p>
|
||||
|
||||
{{< case-studies/quote author="Edgaras Apšega, IT Systems Engineer, Adform" >}}
|
||||
"I think that our company just started our cloud native journey. It seems like a huge road ahead, but we're really happy that we joined it."
|
||||
{{< /case-studies/quote >}}
|
||||
|
||||
<p>All of these benefits have trickled down to individual team members, whose working lives have been changed for the better. "They used to have to get up at night to re-start some services, and now Kubernetes handles all of that," says Apšega. Adds Cibulskis: "Releases are really nice for them, because they just push their code to Git and that's it. They don't have to worry about their virtual machines anymore." Even the security teams have been impacted. "Security teams are always not happy," says Apšega, "and now they're happy because they can easily inspect the containers."</p>
|
||||
|
||||
<p>The company plans to remain in the data centers for now, "mostly because we want to keep all the data, to not share it in any way," says Cibulskis, "and it's cheaper at our scale." But, Apšega says, the possibility of using a hybrid cloud for computing is intriguing: "One of the projects we're interested in is the <a href="https://github.com/virtual-kubelet/virtual-kubelet">Virtual Kubelet</a> that lets you spin up the working nodes on different clouds to do some computing."</p>
|
||||
|
||||
<p>Apšega, Cibulskis and their colleagues are keeping tabs on how the cloud native ecosystem develops, and are excited to contribute where they can. "I think that our company just started our cloud native journey," says Apšega. "It seems like a huge road ahead, but we're really happy that we joined it."</p>
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 6.8 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 19 KiB |
@@ -1,84 +0,0 @@
|
||||
---
|
||||
title: Amadeus Case Study
|
||||
case_study_styles: true
|
||||
cid: caseStudies
|
||||
|
||||
new_case_study_styles: true
|
||||
heading_background: /images/case-studies/amadeus/banner1.jpg
|
||||
heading_title_logo: /images/amadeus_logo.png
|
||||
subheading: >
|
||||
Another Technical Evolution for a 30-Year-Old Company
|
||||
case_study_details:
|
||||
- Company: Amadeus IT Group
|
||||
- Location: Madrid, Spain
|
||||
- Industry: Travel Technology
|
||||
---
|
||||
|
||||
<h2>Challenge</h2>
|
||||
|
||||
<p>In the past few years, Amadeus, which provides IT solutions to the travel industry around the world, found itself in need of a new platform for the 5,000 services supported by its service-oriented architecture. The 30-year-old company operates its own data center in Germany, and there were growing demands internally and externally for solutions that needed to be geographically dispersed. And more generally, "we had objectives of being even more highly available," says Eric Mountain, Senior Expert, Distributed Systems at Amadeus. Among the company's goals: to increase automation in managing its infrastructure, optimize the distribution of workloads, use data center resources more efficiently, and adopt new technologies more easily.</p>
|
||||
|
||||
<h2>Solution</h2>
|
||||
|
||||
<p>Mountain has been overseeing the company's migration to <a href="http://kubernetes.io/">Kubernetes</a>, using <a href="https://www.openshift.org/">OpenShift</a> Container Platform, <a href="https://www.redhat.com/en">Red Hat</a>'s enterprise container platform.</p>
|
||||
|
||||
<h2>Impact</h2>
|
||||
|
||||
<p>One of the first projects the team deployed in Kubernetes was the Amadeus Airline Cloud Availability solution, which helps manage ever-increasing flight-search volume. "It's now handling in production several thousand transactions per second, and it's deployed in multiple data centers throughout the world," says Mountain. "It's not a migration of an existing workload; it's a whole new workload that we couldn't have done otherwise. [This platform] gives us access to market opportunities that we didn't have before."</p>
|
||||
|
||||
{{< case-studies/quote author="Eric Mountain, Senior Expert, Distributed Systems at Amadeus IT Group" >}}
|
||||
"We want multi-data center capabilities, and we want them for our mainstream system as well. We didn't think that we could achieve them with our existing system. We need new automation, things that Kubernetes and OpenShift bring."
|
||||
{{< /case-studies/quote >}}
|
||||
|
||||
{{< case-studies/lead >}}
|
||||
In his two decades at Amadeus, Eric Mountain has been the migrations guy.
|
||||
{{< /case-studies/lead >}}
|
||||
|
||||
<p>Back in the day, he worked on the company's move from Unix to Linux, and now he's overseeing the journey to cloud native. "Technology just keeps changing, and we embrace it," he says. "We are celebrating our 30 years this year, and we continue evolving and innovating to stay cost-efficient and enhance everyone's travel experience, without interrupting workflows for the customers who depend on our technology."</p>
|
||||
|
||||
<p>That was the challenge that Amadeus—which provides IT solutions to the travel industry around the world, from flight searches to hotel bookings to customer feedback—faced in 2014. The technology team realized it was in need of a new platform for the 5,000 services supported by its service-oriented architecture.</p>
|
||||
|
||||
<p>The tipping point occurred when they began receiving many requests, internally and externally, for solutions that needed to be geographically outside the company's main data center in Germany. "Some requests were for running our applications on customer premises," Mountain says. "There were also new services we were looking to offer that required response time to the order of a few hundred milliseconds, which we couldn't achieve with transatlantic traffic. Or at least, not without eating into a considerable portion of the time available to our applications for them to process individual queries."</p>
|
||||
|
||||
<p>More generally, the company was interested in leveling up on high availability, increasing automation in managing infrastructure, optimizing the distribution of workloads and using data center resources more efficiently. "We have thousands and thousands of servers," says Mountain. "These servers are assigned roles, so even if the setup is highly automated, the machine still has a given role. It's wasteful on many levels. For instance, an application doesn't necessarily use the machine very optimally. Virtualization can help a bit, but it's not a silver bullet. If that machine breaks, you still want to repair it because it has that role and you can't simply say, 'Well, I'll bring in another machine and give it that role.' It's not fast. It's not efficient. So we wanted the next level of automation."</p>
|
||||
|
||||
{{< case-studies/quote image="/images/case-studies/amadeus/banner3.jpg" >}}
|
||||
"We hope that if we build on what others have built, what we do might actually be upstream-able. As Kubernetes and OpenShift progress, we see that we are indeed able to remove some of the additional layers we implemented to compensate for gaps we perceived earlier."
|
||||
{{< /case-studies/quote >}}
|
||||
|
||||
<p>While mainly a C++ and Java shop, Amadeus also wanted to be able to adopt new technologies more easily. Some of its developers had started using languages like <a href="https://www.python.org/">Python</a> and databases like <a href="https://www.couchbase.com/">Couchbase</a>, but Mountain wanted still more options, he says, "in order to better adapt our technical solutions to the products we offer, and open up entirely new possibilities to our developers." Working with recent technologies and cool new things would also make it easier to attract new talent.</p>
|
||||
|
||||
<p>All of those needs led Mountain and his team on a search for a new platform. "We did a set of studies and proofs of concept over a fairly short period, and we considered many technologies," he says. "In the end, we were left with three choices: build everything on premise, build on top of <a href="http://kubernetes.io/">Kubernetes</a> whatever happens to be missing from our point of view, or go with <a href="https://www.openshift.com/">OpenShift</a> and build whatever remains there."</p>
|
||||
|
||||
<p>The team decided against building everything themselves—though they'd done that sort of thing in the past—because "people were already inventing things that looked good," says Mountain.</p>
|
||||
|
||||
<p>Ultimately, they went with OpenShift Container Platform, <a href="https://www.redhat.com/en">Red Hat</a>'s Kubernetes-based enterprise offering, instead of building on top of Kubernetes because "there was a lot of synergy between what we wanted and the way Red Hat was anticipating going with OpenShift," says Mountain. "They were clearly developing Kubernetes, and developing certain things ahead of time in OpenShift, which were important to us, such as more security."</p>
|
||||
|
||||
<p>The hope was that those particular features would eventually be built into Kubernetes, and, in the case of security, Mountain feels that has happened. "We realize that there's always a certain amount of automation that we will probably have to develop ourselves to compensate for certain gaps," says Mountain. "The less we do that, the better for us. We hope that if we build on what others have built, what we do might actually be upstream-able. As Kubernetes and OpenShift progress, we see that we are indeed able to remove some of the additional layers we implemented to compensate for gaps we perceived earlier."</p>
|
||||
|
||||
{{< case-studies/quote image="/images/case-studies/amadeus/banner4.jpg" >}}
|
||||
"It's not a migration of an existing workload; it's a whole new workload that we couldn't have done otherwise. [This platform] gives us access to market opportunities that we didn't have before."
|
||||
{{< /case-studies/quote >}}
|
||||
|
||||
<p>The first project the team tackled was one that they knew had to run outside the data center in Germany. Because of the project's needs, "We couldn't rely only on the built-in Kubernetes service discovery; we had to layer on top of that an extra service discovery level that allows us to load balance at the operation level within our system," says Mountain. They also built a stream dedicated to monitoring, which at the time wasn't offered in the Kubernetes or OpenShift ecosystem. Now that <a href="https://www.prometheus.io/">Prometheus</a> and other products are available, Mountain says the company will likely re-evaluate their monitoring system: "We obviously always like to leverage what Kubernetes and OpenShift can offer."
|
||||
</p>
|
||||
|
||||
<p>The second project ended up going into production first: the Amadeus Airline Cloud Availability solution, which helps manage ever-increasing flight-search volume and was deployed in public cloud. Launched in early 2016, it is "now handling in production several thousand transactions per second, and it's deployed in multiple data centers throughout the world," says Mountain. "It's not a migration of an existing workload; it's a whole new workload that we couldn't have done otherwise. [This platform] gives us access to market opportunities that we didn't have before."</p>
|
||||
|
||||
<p>Having been through this kind of technical evolution more than once, Mountain has advice on how to handle the cultural changes. "That's one aspect that we can tackle progressively," he says. "We have to go on supplying our customers with new features on our pre-existing products, and we have to keep existing products working. So we can't simply do absolutely everything from one day to the next. And we mustn't sell it that way."</p>
|
||||
|
||||
<p>The first order of business, then, is to pick one or two applications to demonstrate that the technology works. Rather than choosing a high-impact, high-risk project, Mountain's team selected a smaller application that was representative of all the company's other applications in its complexity: "We just made sure we picked something that's complex enough, and we showed that it can be done."</p>
|
||||
|
||||
{{< case-studies/quote >}}
|
||||
"The bottom line is we want these multi-data center capabilities, and we want them as well for our mainstream system," he says. "And we don't think that we can implement them with our previous system. We need the new automation, homogeneity, and scale that Kubernetes and OpenShift bring."
|
||||
{{< /case-studies/quote >}}
|
||||
|
||||
<p>Next comes convincing people. "On the operations side and on the R&D side, there will be people who say quite rightly, 'There is a system, and it works, so why change?'" Mountain says. "The only thing that really convinces people is showing them the value." For Amadeus, people realized that the Airline Cloud Availability product could not have been made available on the public cloud with the company's existing system. The question then became, he says, "Do we go into a full-blown migration? Is that something that is justified?"</p>
|
||||
|
||||
<p>"The bottom line is we want these multi-data center capabilities, and we want them as well for our mainstream system," he says. "And we don't think that we can implement them with our previous system. We need the new automation, homogeneity, and scale that Kubernetes and OpenShift bring."</p>
|
||||
|
||||
<p>So how do you get everyone on board? "Make sure you have good links between your R&D and your operations," he says. "Also make sure you're going to talk early on to the investors and stakeholders. Figure out what it is that they will be expecting from you, that will convince them or not, that this is the right way for your company."</p>
|
||||
|
||||
<p>His other advice is simply to make the technology available for people to try it. "Kubernetes and OpenShift Origin are open source software, so there's no complicated license key for the evaluation period and you're not limited to 30 days," he points out. "Just go and get it running." Along with that, he adds, "You've got to be prepared to rethink how you do things. Of course making your applications as cloud native as possible is how you'll reap the most benefits: 12 factors, CI/CD, which is continuous integration, continuous delivery, but also continuous deployment."</p>
|
||||
|
||||
<p>And while they explore that aspect of the technology, Mountain and his team will likely be practicing what he preaches to others taking the cloud native journey. "See what happens when you break it, because it's important to understand the limits of the system," he says. Or rather, he notes, the advantages of it. "Breaking things on Kube is actually one of the nice things about it—it recovers. It's the only real way that you'll see that you might be able to do things."</p>
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 20 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 21 KiB |
@@ -1,92 +0,0 @@
|
||||
---
|
||||
title: Ancestry Case Study
|
||||
case_study_styles: true
|
||||
cid: caseStudies
|
||||
|
||||
new_case_study_styles: true
|
||||
heading_background: /images/case-studies/ancestry/banner1.jpg
|
||||
heading_title_logo: /images/ancestry_logo.png
|
||||
subheading: >
|
||||
Digging Into the Past With New Technology
|
||||
case_study_details:
|
||||
- Company: Ancestry
|
||||
- Location: Lehi, Utah
|
||||
- Industry: Internet Company, Online Services
|
||||
---
|
||||
|
||||
<h2>Challenge</h2>
|
||||
|
||||
<p>Ancestry, the global leader in family history and consumer genomics, uses sophisticated engineering and technology to help everyone, everywhere discover the story of what led to them. The company has spent more than 30 years innovating and building products and technologies that at their core, result in real and emotional human responses. <a href="https://www.ancestry.com">Ancestry</a> currently serves more than 2.6 million paying subscribers, holds 20 billion historical records, 90 million family trees and more than four million people are in its AncestryDNA network, making it the largest consumer genomics DNA network in the world. The company's popular website, <a href="https://www.ancestry.com">ancestry.com</a>, has been working with big data long before the term was popularized. The site was built on hundreds of services, technologies and a traditional deployment methodology. "It's worked well for us in the past," says Paul MacKay, software engineer and architect at Ancestry, "but had become quite cumbersome in its processing and is time-consuming. As a primarily online service, we are constantly looking for ways to accelerate to be more agile in delivering our solutions and our products."</p>
|
||||
|
||||
<h2>Solution</h2>
|
||||
|
||||
<p>The company is transitioning to cloud native infrastructure, using <a href="https://www.docker.com">Docker</a> containerization, <a href="https://kubernetes.io">Kubernetes</a> orchestration and <a href="https://prometheus.io">Prometheus</a> for cluster monitoring.</p>
|
||||
|
||||
<h2>Impact</h2>
|
||||
|
||||
<p>"Every single product, every decision we make at Ancestry, focuses on delighting our customers with intimate, sometimes life-changing discoveries about themselves and their families," says MacKay. "As the company continues to grow, the increased productivity gains from using Kubernetes has helped Ancestry make customer discoveries faster. With the move to Dockerization for example, instead of taking between 20 to 50 minutes to deploy a new piece of code, we can now deploy in under a minute for much of our code. We've truly experienced significant time savings in addition to the various features and benefits from cloud native and Kubernetes-type technologies."</p>
|
||||
|
||||
{{< case-studies/quote author="PAUL MACKAY, SOFTWARE ENGINEER AND ARCHITECT AT ANCESTRY" >}}
|
||||
"At a certain point, you have to step back if you're going to push a new technology and get key thought leaders with engineers within the organization to become your champions for new technology adoption. At training sessions, the development teams were always the ones that were saying, 'Kubernetes saved our time tremendously; it's an enabler. It really is incredible.'"
|
||||
{{< /case-studies/quote >}}
|
||||
|
||||
{{< case-studies/lead >}}
|
||||
It started with a Shaky Leaf.
|
||||
{{< /case-studies/lead >}}
|
||||
|
||||
<p>Since its introduction a decade ago, the Shaky Leaf icon has become one of Ancestry's signature features, which signals to users that there's a helpful hint you can use to find out more about your family tree.</p>
|
||||
|
||||
<p>So when the company decided to begin moving its infrastructure to cloud native technology, the first service that was launched on <a href="https://kubernetes.io">Kubernetes</a>, the open source platform for managing application containers across clusters of hosts, was this hint system. Think of it as Amazon's recommended products, but instead of recommending products the company recommends records, stories, or familial connections. "It was a very important part of the site," says Ancestry software engineer and architect Paul MacKay, "but also small enough for a pilot project that we knew we could handle in a very appropriate, secure way."</p>
|
||||
|
||||
<p>And when it went live smoothly in early 2016, "our deployment time for this service literally was cut down from 50 minutes to 2 or 5 minutes," MacKay adds. "The development team was just thrilled because we're focused on supplying a great experience for our customers. And that means features, it means stability, it means all those things that we need for a first-in-class type operation."</p>
|
||||
|
||||
<p>The stability of that Shaky Leaf was a signal for MacKay and his team that their decision to embrace cloud native technologies was the right one for the company. With a private data center, Ancestry built its website (which launched in 1996) on hundreds of services and technologies and a traditional deployment methodology. "It worked well for us in the past, but the sum of the legacy systems became quite cumbersome in its processing and was time-consuming," says MacKay. "We were looking for other ways to accelerate, to be more agile in delivering our solutions and our products."</p>
|
||||
|
||||
{{< case-studies/quote image="/images/case-studies/ancestry/banner3.jpg" >}}
|
||||
"And when it [Kubernetes] went live smoothly in early 2016, 'our deployment time for this service literally was cut down from 50 minutes to 2 or 5 minutes,' MacKay adds. 'The development team was just thrilled because we're focused on supplying a great experience for our customers. And that means features, it means stability, it means all those things that we need for a first-in-class type operation.'"
|
||||
{{< /case-studies/quote >}}
|
||||
|
||||
<p>That need led them in 2015 to explore containerization. Ancestry engineers had already been using technology like <a href="https://www.java.com/en/">Java</a> and <a href="https://www.python.org">Python</a> on Linux, so part of the decision was about making the infrastructure more Linux-friendly. They quickly decided that they wanted to go with Docker for containerization, "but it always comes down to the orchestration part of it to make it really work," says MacKay.</p>
|
||||
|
||||
<p>His team looked at orchestration platforms offered by <a href="https://docs.docker.com/compose/">Docker Compose</a>, <a href="http://mesos.apache.org">Mesos</a> and <a href="https://www.openstack.org/software/">OpenStack</a>, and even started to prototype some homegrown solutions. And then they started hearing rumblings of the imminent release of Kubernetes v1.0. "At the forefront, we were looking at the secret store, so we didn't have to manage that all ourselves, the config maps, the methodology of seamless deployment strategy," he says. "We found that how Kubernetes had done their resources, their types, their labels and just their interface was so much further advanced than the other things we had seen. It was a feature fit."</p>
|
||||
|
||||
{{< case-studies/lead >}}
|
||||
Plus, MacKay says, "I just believed in the confidence that comes with the history that Google has with containerization. So we started out right on the leading edge of it. And we haven't looked back since."
|
||||
{{< /case-studies/lead >}}
|
||||
|
||||
<p>Which is not to say that adopting a new technology hasn't come with some challenges. "Change is hard," says MacKay. "Not because the technology is hard or that the technology is not good. It's just that people like to do things like they had done [before]. You have the early adopters and you have those who are coming in later. It was a learning experience on both sides."</p>
|
||||
|
||||
<p>Figuring out the best deployment operations for Ancestry was a big part of the work it took to adopt cloud native infrastructure. "We want to make sure the process is easy and also controlled in the manner that allows us the highest degree of security that we demand and our customers demand," says MacKay. "With Kubernetes and other products, there are some good solutions, but a little bit of glue is needed to bring it into corporate processes and governances. It's like having a set of gloves that are generic, but when you really do want to grab something you have to make it so it's customized to you. That's what we had to do."</p>
|
||||
|
||||
<p>Their best practices include allowing their developers to deploy into development stage and production, but then controlling the aspects that need governance and auditing, such as secrets. They found that having one namespace per service is useful for achieving that containment of secrets and config maps. And for their needs, having one container per pod makes it easier to manage and to have a smaller unit of deployment.
|
||||
</p>
|
||||
|
||||
{{< case-studies/quote image="/images/case-studies/ancestry/banner4.jpg" >}}
|
||||
"The success of Ancestry's first deployment of the hint system on Kubernetes helped create momentum for greater adoption of the technology."
|
||||
{{< /case-studies/quote >}}
|
||||
|
||||
<p>With that process established, the time spent on deployment was cut down to under a minute for some services. "As programmers, we have what's called REPL: read, evaluate, print, and loop, but with Kubernetes, we have CDEL: compile, deploy, execute, and loop," says MacKay. "It's a very quick loop back and a great benefit to understand that when our services are deployed in production, they're the same as what we tested in the pre-production environments. The approach of cloud native for Ancestry provides us a better ability to scale and to accommodate the business needs as work loads occur."</p>
|
||||
|
||||
<p>The success of Ancestry's first deployment of the hint system on Kubernetes helped create momentum for greater adoption of the technology. "Engineers like to code, they like to do features, they don't like to sit around waiting for things to be deployed and worrying about scaling up and out and down," says MacKay. "After a while the engineers became our champions. At training sessions, the development teams were always the ones saying, 'Kubernetes saved our time tremendously; it's an enabler; it really is incredible.' Over time, we were able to convince our management that this was a transition that the industry is making and that we needed to be a part of it."</p>
|
||||
|
||||
<p>A year later, Ancestry has transitioned a good number of applications to Kubernetes. "We have many different services that make up the rich environment that [the website] has from both the DNA side and the family history side," says MacKay. "We have front-end stacks, back-end stacks and back-end processing type stacks that are in the cluster."</p>
|
||||
|
||||
<p>The company continues to weigh which services it will move forward to Kubernetes, which ones will be kept as is, and which will be replaced in the future and thus don't have to be moved over. MacKay estimates that the company is "approaching halfway on those features that are going forward. We don't have to do a lot of convincing anymore. It's more of an issue of timing with getting product management and engineering staff the knowledge and information that they need."</p>
|
||||
|
||||
{{< case-studies/quote >}}
|
||||
"... 'I believe in Kubernetes. I believe in containerization. I think if we can get there and establish ourselves in that world, we will be further along and far better off being agile and all the things we talk about, and it'll go forward.'"
|
||||
{{< /case-studies/quote >}}
|
||||
|
||||
<p>Looking ahead, MacKay sees Ancestry maximizing the benefits of Kubernetes in 2017. "We're very close to having everything that should be or could be in a Linux-friendly world in Kubernetes by the end of the year," he says, adding that he's looking forward to features such as federation and horizontal pod autoscaling that are currently in the works. "Kubernetes has been very wonderful for us and we continue to ride the wave."</p>
|
||||
|
||||
<p>That wave, he points out, has everything to do with the vibrant Kubernetes community, which has grown by leaps and bounds since Ancestry joined it as an early adopter. "This is just a very rough way of judging it, but on Slack in June 2015, there were maybe 500 on there," MacKay says. "The last time I looked there were maybe 8,500 just on the Slack channel. There are so many major companies and different kinds of companies involved now. It's the variety of contributors, the number of contributors, the incredibly competent and friendly community."</p>
|
||||
|
||||
<p>As much as he and his team at Ancestry have benefited from what he calls "the goodness and the technical abilities of many" in the community, they've also contributed information about best practices, logged bug issues and participated in the open source conversation. And they've been active in attending <a href="https://www.meetup.com/Utah-Kubernetes-Meetup/">meetups</a> to help educate and give back to the local tech community in Utah. Says MacKay: "We're trying to give back as far as our experience goes, rather than just code."</p>
|
||||
|
||||
<p>When he meets with companies considering adopting cloud native infrastructure, the best advice he has to give from Ancestry's Kubernetes journey is this: "Start small, but with hard problems," he says. And "you need a patron who understands the vision of containerization, to help you tackle the political as well as other technical roadblocks that can occur when change is needed."</p>
|
||||
|
||||
<p>With the changes that MacKay's team has led over the past year and a half, cloud native will be part of Ancestry's technological genealogy for years to come. MacKay has been such a champion of the technology that he says people have jokingly accused him of having a Kubernetes tattoo.</p>
|
||||
|
||||
<p>"I really don't," he says with a laugh. "But I'm passionate. I'm not exclusive to any technology; I use whatever I need that's out there that makes us great. If it's something else, I'll use it. But right now I believe in Kubernetes. I believe in containerization. I think if we can get there and establish ourselves in that world, we will be further along and far better off being agile and all the things we talk about, and it'll go forward."</p>
|
||||
|
||||
<p>He pauses. "So, yeah, I guess you can say I'm an evangelist for Kubernetes," he says. "But I'm not getting a tattoo!"</p>
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 7.7 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 7.7 KiB |
@@ -1,85 +0,0 @@
|
||||
---
|
||||
title: BlaBlaCar Case Study
|
||||
case_study_styles: true
|
||||
cid: caseStudies
|
||||
|
||||
new_case_study_styles: true
|
||||
heading_background: /images/case-studies/blablacar/banner1.jpg
|
||||
heading_title_logo: /images/blablacar_logo.png
|
||||
subheading: >
|
||||
Turning to Containerization to Support Millions of Rideshares
|
||||
case_study_details:
|
||||
- Company: BlaBlaCar
|
||||
- Location: Paris, France
|
||||
- Industry: Ridesharing Company
|
||||
---
|
||||
|
||||
<h2>Challenge</h2>
|
||||
|
||||
<p>The world's largest long-distance carpooling community, <a href="https://www.blablacar.com/">BlaBlaCar</a>, connects 40 million members across 22 countries. The company has been experiencing exponential growth since 2012 and needed its infrastructure to keep up. "When you're thinking about doubling the number of servers, you start thinking, 'What should I do to be more efficient?'" says Simon Lallemand, Infrastructure Engineer at BlaBlaCar. "The answer is not to hire more and more people just to deal with the servers and installation." The team knew they had to scale the platform, but wanted to stay on their own bare metal servers.</p>
|
||||
|
||||
<h2>Solution</h2>
|
||||
|
||||
<p>Opting not to shift to cloud virtualization or use a private cloud on their own servers, the BlaBlaCar team became early adopters of containerization, using the CoreOs runtime <a href="https://coreos.com/rkt">rkt</a>, initially deployed using <a href="https://coreos.com/fleet/docs/latest/launching-containers-fleet.html">fleet</a> cluster manager. Last year, the company switched to <a href="http://kubernetes.io/">Kubernetes</a> orchestration, and now also uses <a href="https://prometheus.io/">Prometheus</a> for monitoring.</p>
|
||||
|
||||
<h2>Impact</h2>
|
||||
|
||||
<p>"Before using containers, it would take sometimes a day, sometimes two, just to create a new service," says Lallemand. "With all the tooling that we made around the containers, copying a new service now is a matter of minutes. It's really a huge gain. We are better at capacity planning in our data center because we have fewer constraints due to this abstraction between the services and the hardware we run on. For the developers, it also means they can focus only on the features that they're developing, and not on the infrastructure."</p>
|
||||
|
||||
{{< case-studies/quote author="Simon Lallemand, Infrastructure Engineer at BlaBlaCar" >}}
|
||||
"When you're switching to this cloud-native model and running everything in containers, you have to make sure that at any moment you can reboot without any downtime and without losing traffic. [With Kubernetes] our infrastructure is much more resilient and we have better availability than before."
|
||||
{{< /case-studies/quote >}}
|
||||
|
||||
{{< case-studies/lead >}}
|
||||
For the 40 million users of <a href="https://www.blablacar.com/">BlaBlaCar</a>, it's easy to find strangers headed in the same direction to share rides and costs. You can even choose how much "bla bla" chatter you want from a long-distance ride mate.
|
||||
{{< /case-studies/lead >}}
|
||||
|
||||
<p>Behind the scenes, though, the infrastructure was falling woefully behind the rider community's exponential growth. Founded in 2006, the company hit its current stride around 2012. "Our infrastructure was very traditional," says Infrastructure Engineer Simon Lallemand, who began working at the company in 2014. "In the beginning, it was a bit chaotic because we had to [grow] fast. But then comes the time when you have to design things to make it manageable."</p>
|
||||
|
||||
<p>By 2015, the company had about 50 bare metal servers. The team was using a <a href="https://www.mysql.com/">MySQL</a> database and <a href="http://php.net/">PHP</a>, but, Lallemand says, "it was a very static way." They also utilized the configuration management system, <a href="https://www.chef.io/chef/">Chef</a>, but had little automation in its process. "When you're thinking about doubling the number of servers, you start thinking, 'What should I do to be more efficient?'" says Lallemand. "The answer is not to hire more and more people just to deal with the servers and installation."</p>
|
||||
|
||||
<p>Instead, BlaBlaCar began its cloud-native journey but wasn't sure which route to take. "We could either decide to go into cloud virtualization or even use a private cloud on our own servers," says Lallemand. "But going into the cloud meant we had to make a lot of changes in our application work, and we were just not ready to make the switch from on premise to the cloud." They wanted to keep the great performance they got on bare metal, so they didn't want to go to virtualization on premise.</p>
|
||||
|
||||
<p>The solution: containerization. This was early 2015 and containers were still relatively new. "It was a bold move at the time," says Lallemand. "We decided that the next servers that we would buy in the new data center would all be the same model, so we could outsource the maintenance of the servers. And we decided to go with containers and with <a href="https://coreos.com/">CoreOS</a> Container Linux as an abstraction for this hardware. It seemed future-proof to go with containers because we could see what companies were already doing with containers."</p>
|
||||
|
||||
{{< case-studies/quote image="/images/case-studies/blablacar/banner3.jpg">}}
|
||||
"With all the tooling that we made around the containers, copying a new service is a matter of minutes. It's a huge gain. For the developers, it means they can focus only on the features that they're developing and not on the infrastructure or the hour they would test their code, or the hour that it would get deployed."
|
||||
{{< /case-studies/quote >}}
|
||||
|
||||
<p>Next, they needed to choose a runtime for the containers, but "there were very few deployments in production at that time," says Lallemand. They experimented with <a href="https://www.docker.com/">Docker</a> but decided to go with <a href="https://coreos.com/rkt">rkt</a>. Lallemand explains that for BlaBlaCar, it was "much simpler to integrate things that are on rkt." At the time, the project was still pre-v1.0, so "we could speak with the developers of rkt and give them feedback. It was an advantage." Plus, he notes, rkt was very stable, even at this early stage.</p>
|
||||
|
||||
<p>Once those decisions were made that summer, the company came up with a plan for implementation. First, they formed a task force to create a workflow that would be tested by three of the 10 members on Lallemand's team. But they took care to run regular workshops with all 10 members to make sure everyone was on board. "When you're focused on your product sometimes you forget if it's really user friendly, whether other people can manage to create containers too," Lallemand says. "So we did a lot of iterations to find a good workflow."</p>
|
||||
|
||||
<p>After establishing the workflow, Lallemand says with a smile that "we had this strange idea that we should try the most difficult thing first. Because if it works, it will work for everything." So the first project the team decided to containerize was the database. "Nobody did that at the time, and there were really no existing tools for what we wanted to do, including building container images," he says. So the team created their own tools, such as <a href="https://github.com/blablacar/dgr">dgr</a>, which builds container images so that the whole team has a common framework to build on the same images with the same standards. They also revamped the service-discovery tools <a href="https://github.com/airbnb/nerve">Nerve</a> and <a href="http://airbnb.io/projects/synapse/">Synapse</a>; their versions, <a href="https://github.com/blablacar/go-nerve">Go-Nerve</a> and <a href="https://github.com/blablacar/go-synapse">Go-Synapse</a>, were written in Go and built to be more efficient and include new features. All of these tools were open-sourced.</p>
|
||||
|
||||
<p>At the same time, the company was working to migrate its entire platform to containers with a deadline set for Christmas 2015. With all the work being done in parallel, BlaBlaCar was able to get about 80 percent of its production into containers by its deadline with live traffic running on containers during December. (It's now at 100 percent.) "It's a really busy time for traffic," says Lallemand. "We knew that by using those new servers with containers, it would help us handle the traffic."</p>
|
||||
|
||||
<p>In the middle of that peak season for carpooling, everything worked well. "The biggest impact that we had was for the deployment of new services," says Lallemand. "Before using containers, we had to first deploy a new server and create configurations with Chef. It would take sometimes a day, sometimes two, just to create a new service. And with all the tooling that we made around the containers, copying a new service is a matter of minutes. So it's really a huge gain. For the developers, it means they can focus only on the features that they're developing and not on the infrastructure or the hour they would test their code, or the hour that it would get deployed."</p>
|
||||
|
||||
{{< case-studies/quote image="/images/case-studies/blablacar/banner4.jpg" >}}
|
||||
"We realized that there was a really strong community around it [Kubernetes], which meant we would not have to maintain a lot of tools of our own," says Lallemand. "It was better if we could contribute to some bigger project like Kubernetes."
|
||||
{{< /case-studies/quote >}}
|
||||
|
||||
<p>In order to meet their self-imposed deadline, one of the decisions they made was to not do any "orchestration magic" for containers in the first production alignment. Instead, they used the basic <a href="https://coreos.com/fleet/docs/latest/launching-containers-fleet.html">fleet</a> tool from CoreOS to deploy their containers. (They did build a tool called <a href="https://github.com/blablacar/ggn">GGN</a>, which they've open-sourced, to make it more manageable for their system engineers to use.)</p>
|
||||
|
||||
<p>Still, the team knew that they'd want more orchestration. "Our tool was doing a pretty good job, but at some point you want to give more autonomy to the developer team," Lallemand says. "We also realized that we don't want to be the single point of contact for developers when they want to launch new services." By the summer of 2016, they found their answer in <a href="http://kubernetes.io/">Kubernetes</a>, which had just begun supporting rkt implementation.</p>
|
||||
|
||||
<p>After discussing their needs with their contacts at CoreOS and Google, they were convinced that Kubernetes would work for BlaBlaCar. "We realized that there was a really strong community around it, which meant we would not have to maintain a lot of tools of our own," says Lallemand. "It was better if we could contribute to some bigger project like Kubernetes." They also started using <a href="https://prometheus.io/">Prometheus</a>, as they were looking for "service-oriented monitoring that could be updated nightly." Production on Kubernetes began in December 2016. "We like to do crazy stuff around Christmas," he adds with a laugh.</p>
|
||||
|
||||
<p>BlaBlaCar now has about 3,000 pods, with 1200 of them running on Kubernetes. Lallemand leads a "foundations team" of 25 members who take care of the networks, databases and systems for about 100 developers. There have been some challenges getting to this point. "The rkt implementation is still not 100 percent finished," Lallemand points out. "It's really good, but there are some features still missing. We have questions about how we do things with stateful services, like databases. We know how we will be migrating some of the services; some of the others are a bit more complicated to deal with. But the Kubernetes community is making a lot of progress on that part."</p>
|
||||
|
||||
<p>The team is particularly happy that they're now able to plan capacity better in the company's data center. "We have fewer constraints since we have this abstraction between the services and the hardware we run on," says Lallemand. "If we lose a server because there's a hardware problem on it, we just move the containers onto another server. It's much more efficient. We do that by just changing a line in the configuration file. And with Kubernetes, it should be automatic, so we would have nothing to do."</p>
|
||||
|
||||
{{< case-studies/quote >}}
|
||||
"If we lose a server because there's a hardware problem on it, we just move the containers onto another server. It's much more efficient. We do that by just changing a line in the configuration file. With Kubernetes, it should be automatic, so we would have nothing to do."
|
||||
{{< /case-studies/quote >}}
|
||||
|
||||
<p>And these advances ultimately trickle down to BlaBlaCar's users. "We have improved availability overall on our website," says Lallemand. "When you're switching to this cloud-native model with running everything in containers, you have to make sure that you can at any moment reboot a server or a data container without any downtime, without losing traffic. So now our infrastructure is much more resilient and we have better availability than before."</p>
|
||||
|
||||
<p>Within BlaBlaCar's technology department, the cloud-native journey has created some profound changes. Lallemand thinks that the regular meetings during the conception stage and the training sessions during implementation helped. "After that everybody took part in the migration process," he says. "Then we split the organization into different 'tribes'—teams that gather developers, product managers, data analysts, all the different jobs, to work on a specific part of the product. Before, they were organized by function. The idea is to give all these tribes access to the infrastructure directly in a self-service way without having to ask. These people are really autonomous. They have responsibility of that part of the product, and they can make decisions faster."</p>
|
||||
|
||||
<p>This DevOps transformation turned out to be a positive one for the company's staffers. "The team was very excited about the DevOps transformation because it was new, and we were working to make things more reliable, more future-proof," says Lallemand. "We like doing things that very few people are doing, other than the internet giants."</p>
|
||||
|
||||
<p>With these changes already making an impact, BlaBlaCar is looking to split up more and more of its application into services. "I don't say microservices because they're not so micro," Lallemand says. "If we can split the responsibilities between the development teams, it would be easier to manage and more reliable, because we can easily add and remove services if one fails. You can handle it easily, instead of adding a big monolith that we still have."</p>
|
||||
|
||||
<p>When Lallemand speaks to other European companies curious about what BlaBlaCar has done with its infrastructure, he tells them to come along for the ride. "I tell them that it's such a pleasure to deal with the infrastructure that we have today compared to what we had before," he says. "They just need to keep in mind their real motive, whether it's flexibility in development or reliability or so on, and then go step by step towards reaching those objectives. That's what we've done. It's important not to do technology for the sake of technology. Do it for a purpose. Our focus was on helping the developers."</p>
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 6.2 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 6.3 KiB |
@@ -1,83 +0,0 @@
|
||||
---
|
||||
title: BlackRock Case Study
|
||||
case_study_styles: true
|
||||
cid: caseStudies
|
||||
|
||||
new_case_study_styles: true
|
||||
heading_background: /images/case-studies/blackrock/banner1.jpg
|
||||
heading_title_logo: /images/blackrock_logo.png
|
||||
subheading: >
|
||||
Rolling Out Kubernetes in Production in 100 Days
|
||||
case_study_details:
|
||||
- Company: BlackRock
|
||||
- Location: New York, NY
|
||||
- Industry: Financial Services
|
||||
---
|
||||
|
||||
<h2>Challenge</h2>
|
||||
|
||||
<p>The world's largest asset manager, <a href="https://www.blackrock.com/investing">BlackRock</a> operates a very controlled static deployment scheme, which has allowed for scalability over the years. But in their data science division, there was a need for more dynamic access to resources. "We want to be able to give every investor access to data science, meaning <a href="https://www.python.org">Python</a> notebooks, or even something much more advanced, like a MapReduce engine based on <a href="https://spark.apache.org">Spark</a>," says Michael Francis, a Managing Director in BlackRock's Product Group, which runs the company's investment management platform. "Managing complex Python installations on users' desktops is really hard because everyone ends up with slightly different environments. We have existing environments that do these things, but we needed to make it real, expansive and scalable. Being able to spin that up on demand, tear it down, make that much more dynamic, became a critical thought process for us. It's not so much that we had to solve our main core production problem, it's how do we extend that? How do we evolve?"</p>
|
||||
|
||||
<h2>Solution</h2>
|
||||
|
||||
<p>Drawing from what they learned during a pilot done last year using <a href="https://www.docker.com">Docker</a> environments, Francis put together a cross-sectional team of 20 to build an investor research web app using <a href="https://kubernetes.io">Kubernetes</a> with the goal of getting it into production within one quarter.</p>
|
||||
|
||||
<h2>Impact</h2>
|
||||
|
||||
<p>"Our goal was: How do you give people tools rapidly without having to install them on their desktop?" says Francis. And the team hit the goal within 100 days. Francis is pleased with the results and says, "We're going to use this infrastructure for lots of other application workloads as time goes on. It's not just data science; it's this style of application that needs the dynamism. But I think we're 6-12 months away from making a [large scale] decision. We need to gain experience of running the system in production, we need to understand failure modes and how best to manage operational issues. What's interesting is that just having this technology there is changing the way our developers are starting to think about their future development."</p>
|
||||
|
||||
{{< case-studies/quote author="Michael Francis, Managing Director, BlackRock">}}
|
||||
"My message to other enterprises like us is you can actually integrate Kubernetes into an existing, well-orchestrated machinery. You don't have to throw out everything you do. And using Kubernetes made a complex problem significantly easier."
|
||||
{{< /case-studies/quote >}}
|
||||
|
||||
<p>One of the management objectives for BlackRock's Product Group employees in 2017 was to "build cool stuff." Led by Managing Director Michael Francis, a cross-sectional group of 20 did just that: They rolled out a full production Kubernetes environment and released a new investor research web app on it. In 100 days.</p>
|
||||
|
||||
<p>For a company that's the world's largest asset manager, "just equipment procurement can take 100 days sometimes, let alone from inception to delivery," says Karl Wieman, a Senior System Administrator. "It was an aggressive schedule. But it moved the dial." In fact, the project achieved two goals: It solved a business problem (creating the needed web app) as well as provided real-world, in-production experience with Kubernetes, a cloud-native technology that the company was eager to explore. "It's not so much that we had to solve our main core production problem, it's how do we extend that? How do we evolve?" says Francis. The ultimate success of this project, beyond delivering the app, lies in the fact that "we've managed to integrate a radically new thought process into a controlled infrastructure that we didn't want to change."</p>
|
||||
|
||||
<p>After all, in its three decades of existence, BlackRock has "a very well-established environment for managing our compute resources," says Francis. "We manage large cluster processes on machines, so we do a lot of orchestration and management for our main production processes in a way that's very cloudish in concept. We're able to manage them in a very controlled, static deployment scheme, and that has given us a huge amount of scalability."</p>
|
||||
|
||||
<p>Though that works well for the core production, the company has found that some data science workloads require more dynamic access to resources. "It's a very bursty process," says Francis, who is head of data for the company's Aladdin investment management platform division.</p>
|
||||
|
||||
<p>Aladdin, which connects the people, information and technology needed for money management in real time, is used internally and is also sold as a platform to other asset managers and insurance companies. "We want to be able to give every investor access to data science, meaning <a href="https://www.python.org">Python</a> notebooks, or even something much more advanced, like a MapReduce engine based on <a href="https://spark.apache.org">Spark</a>," says Francis. But "managing complex Python installations on users' desktops is really hard because everyone ends up with slightly different environments. Docker allows us to flatten that environment."</p>
|
||||
|
||||
{{< case-studies/quote image="/images/case-studies/blackrock/banner3.jpg">}}
|
||||
"We manage large cluster processes on machines, so we do a lot of orchestration and management for our main production processes in a way that's very cloudish in concept. We're able to manage them in a very controlled, static deployment scheme, and that has given us a huge amount of scalability."
|
||||
{{< /case-studies/quote >}}
|
||||
|
||||
<p>Still, challenges remain. "If you have a shared cluster, you get this storming herd problem where everyone wants to do the same thing at the same time," says Francis. "You could put limits on it, but you'd have to build an infrastructure to define limits for our processes, and the Python notebooks weren't really designed for that. We have existing environments that do these things, but we needed to make it real, expansive, and scalable. Being able to spin that up on demand, tear it down, and make that much more dynamic, became a critical thought process for us."</p>
|
||||
|
||||
<p>Made up of managers from technology, infrastructure, production operations, development and information security, Francis's team was able to look at the problem holistically and come up with a solution that made sense for BlackRock. "Our initial straw man was that we were going to build everything using <a href="https://www.ansible.com">Ansible</a> and run it all using some completely different distributed environment," says Francis. "That would have been absolutely the wrong thing to do. Had we gone off on our own as the dev team and developed this solution, it would have been a very different product. And it would have been very expensive. We would not have gone down the route of running under our existing orchestration system. Because we don't understand it. These guys [in operations and infrastructure] understand it. Having the multidisciplinary team allowed us to get to the right solutions and that actually meant we didn't build anywhere near the amount we thought we were going to end up building."</p>
|
||||
|
||||
<p>In search of a solution in which they could manage usage on a user-by-user level, Francis's team gravitated to Red Hat's <a href="https://www.openshift.com">OpenShift</a> Kubernetes offering. The company had already experimented with other cloud-native environments, but the team liked that Kubernetes was open source, and "we felt the winds were blowing in the direction of Kubernetes long term," says Francis. "Typically we make technology choices that we believe are going to be here in 5-10 years' time, in some form. And right now, in this space, Kubernetes feels like the one that's going to be there." Adds Uri Morris, Vice President of Production Operations: "When you see that the non-Google committers to Kubernetes overtook the Google committers, that's an indicator of the momentum."</p>
|
||||
|
||||
<p>Once that decision was made, the major challenge was figuring out how to make Kubernetes work within BlackRock's existing framework. "It's about understanding how we can operate, manage and support a platform like this, in addition to tacking it onto our existing technology platform," says Project Manager Michael Maskallis. "All the controls we have in place, the change management process, the software development lifecycle, onboarding processes we go through—how can we do all these things?"</p>
|
||||
|
||||
<p>The first (anticipated) speed bump was working around issues behind BlackRock's corporate firewalls. "One of our challenges is there are no firewalls in most open source software," says Francis. "So almost all install scripts fail in some bizarre way, and pulling down packages doesn't necessarily work." The team ran into these types of problems using <a href="/docs/getting-started-guides/minikube/">Minikube</a> and did a few small pushes back to the open source project.</p>
|
||||
|
||||
{{< case-studies/quote image="/images/case-studies/blackrock/banner4.jpg">}}
|
||||
"Typically we make technology choices that we believe are going to be here in 5-10 years' time, in some form. And right now, in this space, Kubernetes feels like the one that's going to be there."
|
||||
{{< /case-studies/quote >}}
|
||||
|
||||
<p>There were also questions about service discovery. "You can think of Aladdin as a cloud of services with APIs between them that allows us to build applications rapidly," says Francis. "It's all on a proprietary message bus, which gives us all sorts of advantages but at the same time, how does that play in a third party [platform]?"</p>
|
||||
|
||||
<p>Another issue they had to navigate was that in BlackRock's existing system, the messaging protocol has different instances in the different development, test and production environments. While Kubernetes enables a more DevOps-style model, it didn't make sense for BlackRock. "I think what we are very proud of is that the ability for us to push into production is still incredibly rapid in this [new] infrastructure, but we have the control points in place, and we didn't have to disrupt everything," says Francis. "A lot of the cost of this development was thinking how best to leverage our internal tools. So it was less costly than we actually thought it was going to be."</p>
|
||||
|
||||
<p>The project leveraged tools associated with the messaging bus, for example. "The way that the Kubernetes cluster will talk to our internal messaging platform is through a gateway program, and this gateway program already has built-in checks and throttles," says Morris. "We can use them to control and potentially throttle the requests coming in from Kubernetes's very elastic infrastructure to the production infrastructure. We'll continue to go in that direction. It enables us to scale as we need to from the operational perspective."</p>
|
||||
|
||||
<p>The solution also had to be complementary with BlackRock's centralized operational support team structure. "The core infrastructure components of Kubernetes are hooked into our existing orchestration framework, which means that anyone in our support team has both control and visibility to the cluster using the existing operational tools," Morris explains. "That means that I don't need to hire more people."</p>
|
||||
|
||||
<p>With those points established, the team created a procedure for the project: "We rolled this out first to a development environment, then moved on to a testing environment and then eventually to two production environments, in that sequential order," says Maskallis. "That drove a lot of our learning curve. We have all these moving parts, the software components on the infrastructure side, the software components with Kubernetes directly, the interconnectivity with the rest of the environment that we operate here at BlackRock, and how we connect all these pieces. If we came across issues, we fixed them, and then moved on to the different environments to replicate that until we eventually ended up in our production environment where this particular cluster is supposed to live."</p>
|
||||
|
||||
<p>The team had weekly one-hour working sessions with all the members (who are located around the world) participating, and smaller breakout or deep-dive meetings focusing on specific technical details. Possible solutions would be reported back to the group and debated the following week. "I think what made it a successful experiment was people had to work to learn, and they shared their experiences with others," says Vice President and Software Developer Fouad Semaan. Then, Francis says, "We gave our engineers the space to do what they're good at. This hasn't been top-down."</p>
|
||||
|
||||
{{< case-studies/quote >}}
|
||||
"The core infrastructure components of Kubernetes are hooked into our existing orchestration framework, which means that anyone in our support team has both control and visibility to the cluster using the existing operational tools. That means that I don't need to hire more people."
|
||||
{{< /case-studies/quote >}}
|
||||
|
||||
<p>They were led by one key axiom: To stay focused and avoid scope creep. This meant that they wouldn't use features that weren't in the core of Kubernetes and Docker. But if there was a real need, they'd build the features themselves. Luckily, Francis says, "Because of the rapidity of the development, a lot of things we thought we would have to build ourselves have been rolled into the core product. [The package manager<a href="https://helm.sh"> Helm</a> is one example]. People have similar problems."</p>
|
||||
|
||||
<p>By the end of the 100 days, the app was up and running for internal BlackRock users. The initial capacity of 30 users was hit within hours, and quickly increased to 150. "People were immediately all over it," says Francis. In the next phase of this project, they are planning to scale up the cluster to have more capacity.</p>
|
||||
|
||||
<p>Even more importantly, they now have in-production experience with Kubernetes that they can continue to build on—and a complete framework for rolling out new applications. "We're going to use this infrastructure for lots of other application workloads as time goes on. It's not just data science; it's this style of application that needs the dynamism," says Francis. "Is it the right place to move our core production processes onto? It might be. We're not at a point where we can say yes or no, but we felt that having real production experience with something like Kubernetes at some form and scale would allow us to understand that. I think we're 6-12 months away from making a [large scale] decision. We need to gain experience of running the system in production, we need to understand failure modes and how best to manage operational issues."</p>
|
||||
|
||||
<p>For other big companies considering a project like this, Francis says commitment and dedication are key: "We got the signoff from [senior management] from day one, with the commitment that we were able to get the right people. If I had to isolate what makes something complex like this succeed, I would say senior hands-on people who can actually drive it make a huge difference." With that in place, he adds, "My message to other enterprises like us is you can actually integrate Kubernetes into an existing, well-orchestrated machinery. You don't have to throw out everything you do. And using Kubernetes made a complex problem significantly easier."</p>
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 5.1 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 4.7 KiB |
@@ -1,83 +0,0 @@
|
||||
---
|
||||
title: Buffer Case Study
|
||||
case_study_styles: true
|
||||
cid: caseStudies
|
||||
|
||||
new_case_study_styles: true
|
||||
heading_background: /images/case-studies/buffer/banner3.jpg
|
||||
heading_title_logo: /images/buffer.png
|
||||
subheading: >
|
||||
Making Deployments Easy for a Small, Distributed Team
|
||||
case_study_details:
|
||||
- Company: Buffer
|
||||
- Location: Around the World
|
||||
- Industry: Social Media Technology
|
||||
---
|
||||
|
||||
<h2>Challenge</h2>
|
||||
|
||||
<p>With a small but fully distributed team of 80 working across almost a dozen time zones, Buffer—which offers social media management to agencies and marketers—was looking to solve its "classic monolithic code base problem," says Architect Dan Farrelly. "We wanted to have the kind of liquid infrastructure where a developer could create an app and deploy it and scale it horizontally as necessary."</p>
|
||||
|
||||
<h2>Solution</h2>
|
||||
|
||||
<p>Embracing containerization, Buffer moved its infrastructure from Amazon Web Services' Elastic Beanstalk to Docker on AWS, orchestrated with Kubernetes.</p>
|
||||
|
||||
<h2>Impact</h2>
|
||||
|
||||
<p>The new system "leveled up our ability with deployment and rolling out new changes," says Farrelly. "Building something on your computer and knowing that it's going to work has shortened things up a lot. Our feedback cycles are a lot faster now too."</p>
|
||||
|
||||
{{< case-studies/quote author="DAN FARRELLY, BUFFER ARCHITECT" >}}
|
||||
"It's amazing that we can use the Kubernetes solution off the shelf with our team. And it just keeps getting better. Before we even know that we need something, it's there in the next release or it's coming in the next few months."
|
||||
{{< /case-studies/quote >}}
|
||||
|
||||
{{< case-studies/lead >}}
|
||||
Dan Farrelly uses a carpentry analogy to explain the problem his company, <a href="https://buffer.com">Buffer</a>, began having as its team of developers grew over the past few years.
|
||||
{{< /case-studies/lead >}}
|
||||
|
||||
<p>"If you're building a table by yourself, it's fine," the company's architect says. "If you bring in a second person to work on the table, maybe that person can start sanding the legs while you're sanding the top. But when you bring a third or fourth person in, someone should probably work on a different table." Needing to work on more and more different tables led Buffer on a path toward microservices and containerization made possible by Kubernetes.</p>
|
||||
|
||||
<p>Since around 2012, Buffer had already been using <a href="https://aws.amazon.com/elasticbeanstalk/">Elastic Beanstalk</a>, the orchestration service for deploying infrastructure offered by <a href="https://aws.amazon.com">Amazon Web Services</a>. "We were deploying a single monolithic <a href="http://php.net/manual/en/intro-whatis.php">PHP</a> application, and it was the same application across five or six environments," says Farrelly. "We were very much a product-driven company. It was all about shipping new features quickly and getting things out the door, and if something was not broken, we didn't spend too much time on it. If things were getting a little bit slow, we'd maybe use a faster server or just scale up one instance, and it would be good enough. We'd move on."</p>
|
||||
|
||||
<p>But things came to a head in 2016. With the growing number of committers on staff, Farrelly and Buffer's then-CTO, Sunil Sadasivan, decided it was time to re-architect and rethink their infrastructure. "It was a classic monolithic code base problem," says Farrelly.<br><br>Some of the company's team was already successfully using <a href="https://www.docker.com">Docker</a> in their development environment, but the only application running on Docker in production was a marketing website that didn't see real user traffic. They wanted to go further with Docker, and the next step was looking at options for orchestration.</p>
|
||||
|
||||
{{< case-studies/quote image="/images/case-studies/buffer/banner1.jpg" >}}
|
||||
And all the things Kubernetes did well suited Buffer's needs. "We wanted to have the kind of liquid infrastructure where a developer could create an app and deploy it and scale it horizontally as necessary," says Farrelly. "We quickly used some scripts to set up a couple of test clusters, we built some small proof-of-concept applications in containers, and we deployed things within an hour. We had very little experience in running containers in production. It was amazing how quickly we could get a handle on it [Kubernetes]."
|
||||
{{< /case-studies/quote >}}
|
||||
|
||||
<p>First they considered <a href="https://mesosphere.com">Mesosphere</a>, <a href="https://dcos.io">DC/OS</a> and <a href="https://aws.amazon.com/ecs/">Amazon Elastic Container Service</a> (which their data systems team was already using for some data pipeline jobs). While they were impressed by these offerings, they ultimately went with Kubernetes. "We run on AWS still, so spinning up, creating services and creating load balancers on demand for us without having to configure them manually was a great way for our team to get into this," says Farrelly. "We didn't need to figure out how to configure this or that, especially coming from a former Elastic Beanstalk environment that gave us an automatically-configured load balancer. I really liked Kubernetes' controls of the command line. It just took care of ports. It was a lot more flexible. Kubernetes was designed for doing what it does, so it does it very well."</p>
|
||||
|
||||
<p>And all the things Kubernetes did well suited Buffer's needs. "We wanted to have the kind of liquid infrastructure where a developer could create an app and deploy it and scale it horizontally as necessary," says Farrelly. "We quickly used some scripts to set up a couple of test clusters, we built some small proof-of-concept applications in containers, and we deployed things within an hour. We had very little experience in running containers in production. It was amazing how quickly we could get a handle on it [Kubernetes]."</p>
|
||||
|
||||
<p>Above all, it provided a powerful solution for one of the company's most distinguishing characteristics: their remote team that's spread across a dozen different time zones. "The people with deep knowledge of our infrastructure live in time zones different from our peak traffic time zones, and most of our product engineers live in other places," says Farrelly. "So we really wanted something where anybody could get a grasp of the system early on and utilize it, and not have to worry that the deploy engineer is asleep. Otherwise people would sit around for 12 to 24 hours for something. It's been really cool to see people moving much faster."</p>
|
||||
|
||||
<p>With a relatively small engineering team—just 25 people, and only a handful working on infrastructure, with the majority front-end developers—Buffer needed "something robust for them to deploy whatever they wanted," says Farrelly. Before, "it was only a couple of people who knew how to set up everything in the old way. With this system, it was easy to review documentation and get something out extremely quickly. It lowers the bar for us to get everything in production. We don't have the big team to build all these tools or manage the infrastructure like other larger companies might."</p>
|
||||
|
||||
{{< case-studies/quote image="/images/case-studies/buffer/banner4.jpg" >}}
|
||||
"In our old way of working, the feedback loop was a lot longer, and it was delicate because if you deployed something, the risk was high to potentially break something else," Farrelly says. "With the kind of deploys that we built around Kubernetes, we were able to detect bugs and fix them, and get them deployed super fast. The second someone is fixing [a bug], it's out the door."
|
||||
{{< /case-studies/quote >}}
|
||||
|
||||
<p>To help with this, Buffer developers wrote a deploy bot that wraps the Kubernetes deploy process and can be used by every team. "Before, our data analysts would update, say, a <a href="https://www.python.org">Python</a> analysis script and have to wait for the lead on that team to click the button and deploy it," Farrelly explains. "Now our data analysts can make a change, enter a <a href="https://slack.com">Slack</a> command, '/deploy,' and it goes out instantly. They don't need to wait on these slow turnaround times. They don't even know where it's running; it doesn't matter."</p>
|
||||
|
||||
<p>One of the first applications the team built from scratch using Kubernetes was a new image resizing service. As a social media management tool that allows marketing teams to collaborate on posts and send updates across multiple social media profiles and networks, Buffer has to be able to resize photographs as needed to meet the varying limitations of size and format posed by different social networks. "We always had these hacked together solutions," says Farrelly.</p>
|
||||
|
||||
<p>To create this new service, one of the senior product engineers was assigned to learn Docker and Kubernetes, then build the service, test it, deploy it and monitor it—which he was able to do relatively quickly. "In our old way of working, the feedback loop was a lot longer, and it was delicate because if you deployed something, the risk was high to potentially break something else," Farrelly says. "With the kind of deploys that we built around Kubernetes, we were able to detect bugs and fix them, and get them deployed super fast. The second someone is fixing [a bug], it's out the door."</p>
|
||||
|
||||
<p>Plus, unlike with their old system, they could scale things horizontally with one command. "As we rolled it out," Farrelly says, "we could anticipate and just click a button. This allowed us to deal with the demand that our users were placing on the system and easily scale it to handle it."</p>
|
||||
|
||||
<p>Another thing they weren't able to do before was a canary deploy. This new capability "made us so much more confident in deploying big changes," says Farrelly. "Before, it took a lot of testing, which is still good, but it was also a lot of 'fingers crossed.' And this is something that gets run 800,000 times a day, the core of our business. If it doesn't work, our business doesn't work. In a Kubernetes world, I can do a canary deploy to test it for 1 percent and I can shut it down very quickly if it isn't working. This has leveled up our ability to deploy and roll out new changes quickly while reducing risk."</p>
|
||||
|
||||
{{< case-studies/quote >}}
|
||||
"If you want to run containers in production, with nearly the power that Google uses internally, this [Kubernetes] is a great way to do that," Farrelly says. "We're a relatively small team that's actually running Kubernetes, and we've never run anything like it before. So it's more approachable than you might think. That's the one big thing that I tell people who are experimenting with it. Pick a couple of things, roll it out, kick the tires on this for a couple of months and see how much it can handle. You start learning a lot this way."
|
||||
{{< /case-studies/quote >}}
|
||||
|
||||
<p>By October 2016, 54 percent of Buffer's traffic was going through their Kubernetes cluster. "There's a lot of our legacy functionality that still runs alright, and those parts might move to Kubernetes or stay in our old setup forever," says Farrelly. But the company made the commitment at that time that going forward, "all new development, all new features, will be running on Kubernetes."</p>
|
||||
|
||||
<p>The plan for 2017 is to move all the legacy applications to a new Kubernetes cluster, and run everything they've pulled out of their old infrastructure, plus the new services they're developing in Kubernetes, on another cluster. "I want to bring all the benefits that we've seen on our early services to everyone on the team," says Farrelly.</p>
|
||||
|
||||
{{< case-studies/lead >}}
|
||||
For Buffer's engineers, it's an exciting process. "Every time we're deploying a new service, we need to figure out: OK, what's the architecture? How do these services communicate? What's the best way to build this service?" Farrelly says. "And then we use the different features that Kubernetes has to glue all the pieces together. It's enabling us to experiment as we're learning how to design a service-oriented architecture. Before, we just wouldn't have been able to do it. This is actually giving us a blank white board so we can do whatever we want on it."
|
||||
{{< /case-studies/lead >}}
|
||||
|
||||
<p>Part of that blank slate is the flexibility that Kubernetes offers should the time come when Buffer may want or need to change its cloud. "It's cloud agnostic so maybe one day we could switch to Google or somewhere else," Farrelly says. "We're very deep in Amazon but it's nice to know we could move away if we need to."</p>
|
||||
|
||||
<p>At this point, the team at Buffer can't imagine running their infrastructure any other way—and they're happy to spread the word. "If you want to run containers in production, with nearly the power that Google uses internally, this [Kubernetes] is a great way to do that," Farrelly says. "We're a relatively small team that's actually running Kubernetes, and we've never run anything like it before. So it's more approachable than you might think. That's the one big thing that I tell people who are experimenting with it. Pick a couple of things, roll it out, kick the tires on this for a couple of months and see how much it can handle. You start learning a lot this way."</p>
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 11 KiB |
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user