diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 9fe6cea5f3..e14c07a89c 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -18,7 +18,7 @@ The [Kubernetes SIG Docs Discussion Group](https://groups.google.com/forum/#!for ### Style Guides and Templates -Before submitting a pull request to create new content, please review the [Kubernetes.io style guide](http://kubernetes.io/docs/home/contribute/style-guide/) and follow the [instructions for using page templates](http://kubernetes.io/docs/home/contribute/page-templates/). +Before submitting a pull request to create new content, please review the [Kubernetes.io style guide](http://kubernetes.io/docs/home/contribute/style-guide/) and follow the [instructions for using page templates](https://kubernetes.io/docs/contribute/style/page-content-types/). ## Contributing to Documentation diff --git a/OWNERS b/OWNERS index 18bcac1558..f352793ec1 100644 --- a/OWNERS +++ b/OWNERS @@ -9,6 +9,7 @@ approvers: emeritus_approvers: # - chenopis, commented out to disable PR assignments # - jaredbhatti, commented out to disable PR assignments +# - steveperry-53, commented out to disable PR assignments - stewart-yu - zacharysarah diff --git a/OWNERS_ALIASES b/OWNERS_ALIASES index a8804048ae..5f4a86153b 100644 --- a/OWNERS_ALIASES +++ b/OWNERS_ALIASES @@ -34,7 +34,6 @@ aliases: - sftim - steveperry-53 - tengqm - - xiangpengzhao - zacharysarah - zparnold sig-docs-en-reviews: # PR reviews for English content @@ -49,7 +48,6 @@ aliases: - sftim - steveperry-53 - tengqm - - xiangpengzhao - zparnold sig-docs-es-owners: # Admins for Spanish content - raelga @@ -156,7 +154,6 @@ aliases: - SataQiu - tanjunchen - tengqm - - xiangpengzhao - xichengliudui # zhangxiaoyu-zidif sig-docs-zh-reviews: # PR reviews for Chinese content @@ -167,7 +164,6 @@ aliases: - SataQiu - tanjunchen - tengqm - - xiangpengzhao - xichengliudui # zhangxiaoyu-zidif sig-docs-pt-owners: # Admins for Portuguese content diff --git a/README-hi.md b/README-hi.md index 649ece0e55..01d119a22b 100644 --- a/README-hi.md +++ b/README-hi.md @@ -17,7 +17,7 @@ * [योगदान देना शुरू करें](https://kubernetes.io/docs/contribute/start/) * [परिवर्तनों को अंतिम चरण में लेजाएं](http://kubernetes.io/docs/contribute/intermediate#view-your-changes-locally) -* [पेज टेम्पलेट](http://kubernetes.io/docs/contribute/style/page-templates/) +* [पेज टेम्पलेट](https://kubernetes.io/docs/contribute/style/page-content-types/) * [प्रलेखन शैली गाइड](http://kubernetes.io/docs/contribute/style/style-guide/) * [स्थानीयकरण कुबरनेट्स प्रलेखन](https://kubernetes.io/docs/contribute/localization/) @@ -81,4 +81,4 @@ make serve कुबरनेट्स सामुदायिक भागीदारी पर पनपती है, और हम वास्तव में हमारी साइट और हमारे प्रलेखन में आपके योगदान की सराहना करते हैं! -कुबरनेट्स आपकी भागीदारी पर निर्भर है, और हम हमारी साइट और प्रलेखन में आपके योगदान का मान करते हैं! \ No newline at end of file +कुबरनेट्स आपकी भागीदारी पर निर्भर है, और हम हमारी साइट और प्रलेखन में आपके योगदान का मान करते हैं! diff --git a/README-ko.md b/README-ko.md index f5b549439a..e23027c32a 100644 --- a/README-ko.md +++ b/README-ko.md @@ -15,7 +15,7 @@ * [기여 시작하기](https://kubernetes.io/docs/contribute/start/) * [문서화 변경 사항 스테이징하기](http://kubernetes.io/docs/contribute/intermediate#view-your-changes-locally) -* [페이지 템플릿 사용하기](http://kubernetes.io/docs/contribute/style/page-templates/) +* [페이지 템플릿 사용하기](https://kubernetes.io/docs/contribute/style/page-content-types/) * [문서화 스타일 가이드](http://kubernetes.io/docs/contribute/style/style-guide/) * [쿠버네티스 문서화 로컬라이징](https://kubernetes.io/docs/contribute/localization/) diff --git a/README.md b/README.md index 52a1b20b87..5966a17b36 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,7 @@ This repository contains the assets required to build the [Kubernetes website an # Using this repository -You can run the website locally using Hugo, or you can run it in a container runtime. We strongly recommend using the container runtime, as it gives deployment consistency with the live website. +You can run the website locally using Hugo (Extended version), or you can run it in a container runtime. We strongly recommend using the container runtime, as it gives deployment consistency with the live website. ## Prerequisites @@ -15,7 +15,7 @@ To use this repository, you need the following installed locally: - [yarn](https://yarnpkg.com/) - [npm](https://www.npmjs.com/) - [Go](https://golang.org/) -- [Hugo](https://gohugo.io/) +- [Hugo (Extended version)](https://gohugo.io/) - A container runtime, like [Docker](https://www.docker.com/). Before you start, install the dependencies. Clone the repository and navigate to the directory: @@ -58,6 +58,11 @@ make serve This will start the local Hugo server on port 1313. Open up your browser to http://localhost:1313 to view the website. As you make changes to the source files, Hugo updates the website and forces a browser refresh. +## Troubleshooting +### error: failed to transform resource: TOCSS: failed to transform "scss/main.scss" (text/x-scss): this feature is not available in your current Hugo version + +Hugo is shipped in two set of binaries for technical reasons. The current website runs based on the **Hugo Extended** version only. In the [release page](https://github.com/gohugoio/hugo/releases) look for archives with `extended` in the name. To confirm, run `hugo version` and look for the word `extended`. + ### Troubleshooting macOS for too many open files If you run `make serve` on macOS and receive the following error: diff --git a/assets/scss/_case-studies.scss b/assets/scss/_case-studies.scss index d61d24189c..4f44864127 100644 --- a/assets/scss/_case-studies.scss +++ b/assets/scss/_case-studies.scss @@ -2,6 +2,7 @@ hr { background-color: #999999; + margin-top: 0; } h2 { @@ -10,7 +11,7 @@ h2 { .subhead { padding-bottom: 2% !important; - padding-top: 0% !important; + padding-top: 0.75em !important; } .details { diff --git a/assets/scss/_custom.scss b/assets/scss/_custom.scss index d236327b39..3c598673ad 100644 --- a/assets/scss/_custom.scss +++ b/assets/scss/_custom.scss @@ -11,11 +11,6 @@ $announcement-size-adjustment: 8px; max-width: 100%; } - table { - margin-bottom: 1rem; - display: initial; - } - @media only screen and (min-width: 768px) { padding-top: 2rem !important; } diff --git a/content/en/blog/_posts/2016-01-00-Kubernetes-Community-Meeting-Notes_28.md b/content/en/blog/_posts/2016-01-00-Kubernetes-Community-Meeting-Notes_28.md index b778d8eec1..5509e8666a 100644 --- a/content/en/blog/_posts/2016-01-00-Kubernetes-Community-Meeting-Notes_28.md +++ b/content/en/blog/_posts/2016-01-00-Kubernetes-Community-Meeting-Notes_28.md @@ -9,57 +9,37 @@ url: /blog/2016/01/Kubernetes-Community-Meeting-Notes_28 Note taker: Rob Hirshfeld - Use Case (10 min): [SFDC Paul Brown](https://docs.google.com/a/google.com/presentation/d/1MEI97efplr3f-GDX1GcWGfkEuGKKV-4niu27kHOeMLk/edit?usp=sharing_eid&ts=56a114f8) - - SIG Report - SIG-config and the story of [#18215](https://github.com/kubernetes/kubernetes/pull/18215) + + - SIG Report - SIG-config and the story of [#18215](https://github.com/kubernetes/kubernetes/pull/18215). + - Application config IN K8s not deployment of K8s + - Topic has been reuse of configuration,specifically parameterization(aka templates). Needs: + - include scoping(cluster namespace) + - slight customization (naming changes, but not major config) + - multiple positions on how todo this including allowing external or simple extensions + - PetSet creates instances w/stable namespace - - A[p](https://github.com/kubernetes/kubernetes/pull/18215)p[l](https://github.com/kubernetes/kubernetes/pull/18215)i[c](https://github.com/kubernetes/kubernetes/pull/18215)a[t](https://github.com/kubernetes/kubernetes/pull/18215)i[o](https://github.com/kubernetes/kubernetes/pull/18215)n[](https://github.com/kubernetes/kubernetes/pull/18215)c[o](https://github.com/kubernetes/kubernetes/pull/18215)n[f](https://github.com/kubernetes/kubernetes/pull/18215)i[g](https://github.com/kubernetes/kubernetes/pull/18215) [I](https://github.com/kubernetes/kubernetes/pull/18215)N[](https://github.com/kubernetes/kubernetes/pull/18215)K[8](https://github.com/kubernetes/kubernetes/pull/18215)s[,](https://github.com/kubernetes/kubernetes/pull/18215) [n](https://github.com/kubernetes/kubernetes/pull/18215)o[t](https://github.com/kubernetes/kubernetes/pull/18215) [d](https://github.com/kubernetes/kubernetes/pull/18215)e[p](https://github.com/kubernetes/kubernetes/pull/18215)l[o](https://github.com/kubernetes/kubernetes/pull/18215)y[m](https://github.com/kubernetes/kubernetes/pull/18215)e[n](https://github.com/kubernetes/kubernetes/pull/18215)t[](https://github.com/kubernetes/kubernetes/pull/18215)o[f](https://github.com/kubernetes/kubernetes/pull/18215) [K](https://github.com/kubernetes/kubernetes/pull/18215)8[s](https://github.com/kubernetes/kubernetes/pull/18215) - - [T](https://github.com/kubernetes/kubernetes/pull/18215)o[p](https://github.com/kubernetes/kubernetes/pull/18215)i[c](https://github.com/kubernetes/kubernetes/pull/18215) [h](https://github.com/kubernetes/kubernetes/pull/18215)a[s](https://github.com/kubernetes/kubernetes/pull/18215) [b](https://github.com/kubernetes/kubernetes/pull/18215)e[e](https://github.com/kubernetes/kubernetes/pull/18215)n[](https://github.com/kubernetes/kubernetes/pull/18215)r[e](https://github.com/kubernetes/kubernetes/pull/18215)u[s](https://github.com/kubernetes/kubernetes/pull/18215)e[](https://github.com/kubernetes/kubernetes/pull/18215)o[f](https://github.com/kubernetes/kubernetes/pull/18215) [c](https://github.com/kubernetes/kubernetes/pull/18215)o[n](https://github.com/kubernetes/kubernetes/pull/18215)f[i](https://github.com/kubernetes/kubernetes/pull/18215)g[u](https://github.com/kubernetes/kubernetes/pull/18215)r[a](https://github.com/kubernetes/kubernetes/pull/18215)t[i](https://github.com/kubernetes/kubernetes/pull/18215)o[n](https://github.com/kubernetes/kubernetes/pull/18215),[](https://github.com/kubernetes/kubernetes/pull/18215)s[p](https://github.com/kubernetes/kubernetes/pull/18215)e[c](https://github.com/kubernetes/kubernetes/pull/18215)i[f](https://github.com/kubernetes/kubernetes/pull/18215)i[c](https://github.com/kubernetes/kubernetes/pull/18215)a[l](https://github.com/kubernetes/kubernetes/pull/18215)l[y](https://github.com/kubernetes/kubernetes/pull/18215) [p](https://github.com/kubernetes/kubernetes/pull/18215)a[r](https://github.com/kubernetes/kubernetes/pull/18215)a[m](https://github.com/kubernetes/kubernetes/pull/18215)e[t](https://github.com/kubernetes/kubernetes/pull/18215)e[r](https://github.com/kubernetes/kubernetes/pull/18215)i[z](https://github.com/kubernetes/kubernetes/pull/18215)a[t](https://github.com/kubernetes/kubernetes/pull/18215)i[o](https://github.com/kubernetes/kubernetes/pull/18215)n[](https://github.com/kubernetes/kubernetes/pull/18215)([a](https://github.com/kubernetes/kubernetes/pull/18215)k[a](https://github.com/kubernetes/kubernetes/pull/18215) [t](https://github.com/kubernetes/kubernetes/pull/18215)e[m](https://github.com/kubernetes/kubernetes/pull/18215)p[l](https://github.com/kubernetes/kubernetes/pull/18215)a[t](https://github.com/kubernetes/kubernetes/pull/18215)e[s](https://github.com/kubernetes/kubernetes/pull/18215))[. Needs:](https://github.com/kubernetes/kubernetes/pull/18215) + - Workflow proposal + - Distributed Chron. Challenge is that configs need to create multiple objects in sequence + - Trying to figure out how balance the many config options out there (compose, terraform,ansible/etc) + - Goal is to “meet people where they are” to keep it simple + - Q: is there an opinion for the keystore sizing + - large size / data blob would not be appropriate + - you can pull data(config) from another store for larger objects - - n[e](https://github.com/kubernetes/kubernetes/pull/18215)e[d](https://github.com/kubernetes/kubernetes/pull/18215)s[](https://github.com/kubernetes/kubernetes/pull/18215)i[n](https://github.com/kubernetes/kubernetes/pull/18215)c[l](https://github.com/kubernetes/kubernetes/pull/18215)u[d](https://github.com/kubernetes/kubernetes/pull/18215)e[](https://github.com/kubernetes/kubernetes/pull/18215)s[c](https://github.com/kubernetes/kubernetes/pull/18215)o[p](https://github.com/kubernetes/kubernetes/pull/18215)i[n](https://github.com/kubernetes/kubernetes/pull/18215)g[](https://github.com/kubernetes/kubernetes/pull/18215)([c](https://github.com/kubernetes/kubernetes/pull/18215)l[u](https://github.com/kubernetes/kubernetes/pull/18215)s[t](https://github.com/kubernetes/kubernetes/pull/18215)e[r](https://github.com/kubernetes/kubernetes/pull/18215) [n](https://github.com/kubernetes/kubernetes/pull/18215)a[m](https://github.com/kubernetes/kubernetes/pull/18215)e[s](https://github.com/kubernetes/kubernetes/pull/18215)p[a](https://github.com/kubernetes/kubernetes/pull/18215)c[e](https://github.com/kubernetes/kubernetes/pull/18215)) - - s[l](https://github.com/kubernetes/kubernetes/pull/18215)i[g](https://github.com/kubernetes/kubernetes/pull/18215)h[t](https://github.com/kubernetes/kubernetes/pull/18215) [c](https://github.com/kubernetes/kubernetes/pull/18215)u[s](https://github.com/kubernetes/kubernetes/pull/18215)t[o](https://github.com/kubernetes/kubernetes/pull/18215)m[i](https://github.com/kubernetes/kubernetes/pull/18215)z[a](https://github.com/kubernetes/kubernetes/pull/18215)t[i](https://github.com/kubernetes/kubernetes/pull/18215)o[n (naming changes, but not major config)](https://github.com/kubernetes/kubernetes/pull/18215) - - [m](https://github.com/kubernetes/kubernetes/pull/18215)u[l](https://github.com/kubernetes/kubernetes/pull/18215)t[i](https://github.com/kubernetes/kubernetes/pull/18215)p[l](https://github.com/kubernetes/kubernetes/pull/18215)e[](https://github.com/kubernetes/kubernetes/pull/18215)p[o](https://github.com/kubernetes/kubernetes/pull/18215)s[i](https://github.com/kubernetes/kubernetes/pull/18215)t[i](https://github.com/kubernetes/kubernetes/pull/18215)o[n](https://github.com/kubernetes/kubernetes/pull/18215)s[](https://github.com/kubernetes/kubernetes/pull/18215)o[n](https://github.com/kubernetes/kubernetes/pull/18215) [h](https://github.com/kubernetes/kubernetes/pull/18215)o[w](https://github.com/kubernetes/kubernetes/pull/18215) [t](https://github.com/kubernetes/kubernetes/pull/18215)o[](https://github.com/kubernetes/kubernetes/pull/18215)d[o](https://github.com/kubernetes/kubernetes/pull/18215) [t](https://github.com/kubernetes/kubernetes/pull/18215)h[i](https://github.com/kubernetes/kubernetes/pull/18215)s[](https://github.com/kubernetes/kubernetes/pull/18215)i[n](https://github.com/kubernetes/kubernetes/pull/18215)c[l](https://github.com/kubernetes/kubernetes/pull/18215)u[d](https://github.com/kubernetes/kubernetes/pull/18215)i[n](https://github.com/kubernetes/kubernetes/pull/18215)g[](https://github.com/kubernetes/kubernetes/pull/18215) - - - -a[l](https://github.com/kubernetes/kubernetes/pull/18215)l[o](https://github.com/kubernetes/kubernetes/pull/18215)w[i](https://github.com/kubernetes/kubernetes/pull/18215)n[g](https://github.com/kubernetes/kubernetes/pull/18215) [e](https://github.com/kubernetes/kubernetes/pull/18215)x[t](https://github.com/kubernetes/kubernetes/pull/18215)e[r](https://github.com/kubernetes/kubernetes/pull/18215)n[a](https://github.com/kubernetes/kubernetes/pull/18215)l[](https://github.com/kubernetes/kubernetes/pull/18215)o[r](https://github.com/kubernetes/kubernetes/pull/18215) [s](https://github.com/kubernetes/kubernetes/pull/18215)i[m](https://github.com/kubernetes/kubernetes/pull/18215)p[l](https://github.com/kubernetes/kubernetes/pull/18215)e[](https://github.com/kubernetes/kubernetes/pull/18215)e[x](https://github.com/kubernetes/kubernetes/pull/18215)t[e](https://github.com/kubernetes/kubernetes/pull/18215)n[s](https://github.com/kubernetes/kubernetes/pull/18215)i[o](https://github.com/kubernetes/kubernetes/pull/18215)n[s](https://github.com/kubernetes/kubernetes/pull/18215). - - -P[e](https://github.com/kubernetes/kubernetes/pull/18215)t[S](https://github.com/kubernetes/kubernetes/pull/18215)e[t](https://github.com/kubernetes/kubernetes/pull/18215) [c](https://github.com/kubernetes/kubernetes/pull/18215)r[e](https://github.com/kubernetes/kubernetes/pull/18215)a[t](https://github.com/kubernetes/kubernetes/pull/18215)e[s](https://github.com/kubernetes/kubernetes/pull/18215) [i](https://github.com/kubernetes/kubernetes/pull/18215)n[s](https://github.com/kubernetes/kubernetes/pull/18215)t[a](https://github.com/kubernetes/kubernetes/pull/18215)n[c](https://github.com/kubernetes/kubernetes/pull/18215)e[s](https://github.com/kubernetes/kubernetes/pull/18215) [w](https://github.com/kubernetes/kubernetes/pull/18215)/[](https://github.com/kubernetes/kubernetes/pull/18215)s[t](https://github.com/kubernetes/kubernetes/pull/18215)a[b](https://github.com/kubernetes/kubernetes/pull/18215)l[e](https://github.com/kubernetes/kubernetes/pull/18215) [n](https://github.com/kubernetes/kubernetes/pull/18215)a[m](https://github.com/kubernetes/kubernetes/pull/18215)e[s](https://github.com/kubernetes/kubernetes/pull/18215)p[a](https://github.com/kubernetes/kubernetes/pull/18215)c[e](https://github.com/kubernetes/kubernetes/pull/18215) - - -[W](https://github.com/kubernetes/kubernetes/pull/18215)o[r](https://github.com/kubernetes/kubernetes/pull/18215)k[f](https://github.com/kubernetes/kubernetes/pull/18215)l[o](https://github.com/kubernetes/kubernetes/pull/18215)w[](https://github.com/kubernetes/kubernetes/pull/18215)p[r](https://github.com/kubernetes/kubernetes/pull/18215)o[p](https://github.com/kubernetes/kubernetes/pull/18215)o[s](https://github.com/kubernetes/kubernetes/pull/18215)a[l](https://github.com/kubernetes/kubernetes/pull/18215) - - -[D](https://github.com/kubernetes/kubernetes/pull/18215)i[s](https://github.com/kubernetes/kubernetes/pull/18215)t[r](https://github.com/kubernetes/kubernetes/pull/18215)i[b](https://github.com/kubernetes/kubernetes/pull/18215)u[t](https://github.com/kubernetes/kubernetes/pull/18215)e[d](https://github.com/kubernetes/kubernetes/pull/18215) [C](https://github.com/kubernetes/kubernetes/pull/18215)h[r](https://github.com/kubernetes/kubernetes/pull/18215)o[n](https://github.com/kubernetes/kubernetes/pull/18215). - - -C[h](https://github.com/kubernetes/kubernetes/pull/18215)a[l](https://github.com/kubernetes/kubernetes/pull/18215)l[e](https://github.com/kubernetes/kubernetes/pull/18215)n[g](https://github.com/kubernetes/kubernetes/pull/18215)e[](https://github.com/kubernetes/kubernetes/pull/18215)i[s](https://github.com/kubernetes/kubernetes/pull/18215) [t](https://github.com/kubernetes/kubernetes/pull/18215)h[a](https://github.com/kubernetes/kubernetes/pull/18215)t[](https://github.com/kubernetes/kubernetes/pull/18215)c[o](https://github.com/kubernetes/kubernetes/pull/18215)n[f](https://github.com/kubernetes/kubernetes/pull/18215)i[g](https://github.com/kubernetes/kubernetes/pull/18215)s[](https://github.com/kubernetes/kubernetes/pull/18215)n[e](https://github.com/kubernetes/kubernetes/pull/18215)e[d](https://github.com/kubernetes/kubernetes/pull/18215) [t](https://github.com/kubernetes/kubernetes/pull/18215)o[](https://github.com/kubernetes/kubernetes/pull/18215)c[r](https://github.com/kubernetes/kubernetes/pull/18215)e[a](https://github.com/kubernetes/kubernetes/pull/18215)t[e](https://github.com/kubernetes/kubernetes/pull/18215) [m](https://github.com/kubernetes/kubernetes/pull/18215)u[l](https://github.com/kubernetes/kubernetes/pull/18215)t[i](https://github.com/kubernetes/kubernetes/pull/18215)p[l](https://github.com/kubernetes/kubernetes/pull/18215)e[](https://github.com/kubernetes/kubernetes/pull/18215)o[b](https://github.com/kubernetes/kubernetes/pull/18215)j[e](https://github.com/kubernetes/kubernetes/pull/18215)c[t](https://github.com/kubernetes/kubernetes/pull/18215)s[](https://github.com/kubernetes/kubernetes/pull/18215)i[n](https://github.com/kubernetes/kubernetes/pull/18215) [s](https://github.com/kubernetes/kubernetes/pull/18215)e[q](https://github.com/kubernetes/kubernetes/pull/18215)u[e](https://github.com/kubernetes/kubernetes/pull/18215)n[c](https://github.com/kubernetes/kubernetes/pull/18215)e - - -T[r](https://github.com/kubernetes/kubernetes/pull/18215)y[i](https://github.com/kubernetes/kubernetes/pull/18215)n[g](https://github.com/kubernetes/kubernetes/pull/18215) [t](https://github.com/kubernetes/kubernetes/pull/18215)o[](https://github.com/kubernetes/kubernetes/pull/18215)f[i](https://github.com/kubernetes/kubernetes/pull/18215)g[u](https://github.com/kubernetes/kubernetes/pull/18215)r[e](https://github.com/kubernetes/kubernetes/pull/18215) [o](https://github.com/kubernetes/kubernetes/pull/18215)u[t](https://github.com/kubernetes/kubernetes/pull/18215) [h](https://github.com/kubernetes/kubernetes/pull/18215)o[w](https://github.com/kubernetes/kubernetes/pull/18215) [b](https://github.com/kubernetes/kubernetes/pull/18215)a[l](https://github.com/kubernetes/kubernetes/pull/18215)a[n](https://github.com/kubernetes/kubernetes/pull/18215)c[e](https://github.com/kubernetes/kubernetes/pull/18215) [t](https://github.com/kubernetes/kubernetes/pull/18215)h[e](https://github.com/kubernetes/kubernetes/pull/18215) [m](https://github.com/kubernetes/kubernetes/pull/18215)a[n](https://github.com/kubernetes/kubernetes/pull/18215)y[](https://github.com/kubernetes/kubernetes/pull/18215)c[o](https://github.com/kubernetes/kubernetes/pull/18215)n[f](https://github.com/kubernetes/kubernetes/pull/18215)i[g](https://github.com/kubernetes/kubernetes/pull/18215) [o](https://github.com/kubernetes/kubernetes/pull/18215)p[t](https://github.com/kubernetes/kubernetes/pull/18215)i[o](https://github.com/kubernetes/kubernetes/pull/18215)n[s](https://github.com/kubernetes/kubernetes/pull/18215) [o](https://github.com/kubernetes/kubernetes/pull/18215)u[t](https://github.com/kubernetes/kubernetes/pull/18215) [t](https://github.com/kubernetes/kubernetes/pull/18215)h[e](https://github.com/kubernetes/kubernetes/pull/18215)r[e](https://github.com/kubernetes/kubernetes/pull/18215) [(](https://github.com/kubernetes/kubernetes/pull/18215)c[o](https://github.com/kubernetes/kubernetes/pull/18215)m[p](https://github.com/kubernetes/kubernetes/pull/18215)o[s](https://github.com/kubernetes/kubernetes/pull/18215)e[,](https://github.com/kubernetes/kubernetes/pull/18215) [t](https://github.com/kubernetes/kubernetes/pull/18215)e[r](https://github.com/kubernetes/kubernetes/pull/18215)r[a](https://github.com/kubernetes/kubernetes/pull/18215)f[o](https://github.com/kubernetes/kubernetes/pull/18215)r[m](https://github.com/kubernetes/kubernetes/pull/18215),[](https://github.com/kubernetes/kubernetes/pull/18215)a[n](https://github.com/kubernetes/kubernetes/pull/18215)s[i](https://github.com/kubernetes/kubernetes/pull/18215)b[l](https://github.com/kubernetes/kubernetes/pull/18215)e[/](https://github.com/kubernetes/kubernetes/pull/18215)e[t](https://github.com/kubernetes/kubernetes/pull/18215)c[)](https://github.com/kubernetes/kubernetes/pull/18215) - - -[G](https://github.com/kubernetes/kubernetes/pull/18215)o[a](https://github.com/kubernetes/kubernetes/pull/18215)l[](https://github.com/kubernetes/kubernetes/pull/18215)i[s](https://github.com/kubernetes/kubernetes/pull/18215) [t](https://github.com/kubernetes/kubernetes/pull/18215)o[](https://github.com/kubernetes/kubernetes/pull/18215)“[m](https://github.com/kubernetes/kubernetes/pull/18215)e[e](https://github.com/kubernetes/kubernetes/pull/18215)t[](https://github.com/kubernetes/kubernetes/pull/18215)p[e](https://github.com/kubernetes/kubernetes/pull/18215)o[p](https://github.com/kubernetes/kubernetes/pull/18215)l[e](https://github.com/kubernetes/kubernetes/pull/18215) [w](https://github.com/kubernetes/kubernetes/pull/18215)h[e](https://github.com/kubernetes/kubernetes/pull/18215)r[e](https://github.com/kubernetes/kubernetes/pull/18215) [t](https://github.com/kubernetes/kubernetes/pull/18215)h[e](https://github.com/kubernetes/kubernetes/pull/18215)y[](https://github.com/kubernetes/kubernetes/pull/18215)a[r](https://github.com/kubernetes/kubernetes/pull/18215)e[”](https://github.com/kubernetes/kubernetes/pull/18215) [t](https://github.com/kubernetes/kubernetes/pull/18215)o[](https://github.com/kubernetes/kubernetes/pull/18215)k[e](https://github.com/kubernetes/kubernetes/pull/18215)e[p](https://github.com/kubernetes/kubernetes/pull/18215) [i](https://github.com/kubernetes/kubernetes/pull/18215)t[](https://github.com/kubernetes/kubernetes/pull/18215)s[i](https://github.com/kubernetes/kubernetes/pull/18215)m[p](https://github.com/kubernetes/kubernetes/pull/18215)l[e](https://github.com/kubernetes/kubernetes/pull/18215) - - -[Q](https://github.com/kubernetes/kubernetes/pull/18215): is there an opinion for the keystore sizing - - - -large size / data blob would not be appropriate - - -you can pull data(config) from another store for larger objects - - -[SIG Report - SIG-federation - progress on Ubernetes-Lite & Ubernetes design](https://github.com/kubernetes/kubernetes/pull/18215) - - - -[G](https://github.com/kubernetes/kubernetes/pull/18215)o[a](https://github.com/kubernetes/kubernetes/pull/18215)l[](https://github.com/kubernetes/kubernetes/pull/18215)i[t](https://github.com/kubernetes/kubernetes/pull/18215) [t](https://github.com/kubernetes/kubernetes/pull/18215)o[](https://github.com/kubernetes/kubernetes/pull/18215)b[e](https://github.com/kubernetes/kubernetes/pull/18215) [a](https://github.com/kubernetes/kubernetes/pull/18215)b[l](https://github.com/kubernetes/kubernetes/pull/18215)e[](https://github.com/kubernetes/kubernetes/pull/18215)t[o](https://github.com/kubernetes/kubernetes/pull/18215) [h](https://github.com/kubernetes/kubernetes/pull/18215)a[v](https://github.com/kubernetes/kubernetes/pull/18215)e[](https://github.com/kubernetes/kubernetes/pull/18215)a[](https://github.com/kubernetes/kubernetes/pull/18215)c[l](https://github.com/kubernetes/kubernetes/pull/18215)u[s](https://github.com/kubernetes/kubernetes/pull/18215)t[e](https://github.com/kubernetes/kubernetes/pull/18215)r[](https://github.com/kubernetes/kubernetes/pull/18215)m[a](https://github.com/kubernetes/kubernetes/pull/18215)n[a](https://github.com/kubernetes/kubernetes/pull/18215)g[e](https://github.com/kubernetes/kubernetes/pull/18215)r[,](https://github.com/kubernetes/kubernetes/pull/18215) [s](https://github.com/kubernetes/kubernetes/pull/18215)o[](https://github.com/kubernetes/kubernetes/pull/18215)y[o](https://github.com/kubernetes/kubernetes/pull/18215)u[](https://github.com/kubernetes/kubernetes/pull/18215)c[a](https://github.com/kubernetes/kubernetes/pull/18215)n[](https://github.com/kubernetes/kubernetes/pull/18215)f[e](https://github.com/kubernetes/kubernetes/pull/18215)d[e](https://github.com/kubernetes/kubernetes/pull/18215)r[a](https://github.com/kubernetes/kubernetes/pull/18215)t[e](https://github.com/kubernetes/kubernetes/pull/18215) [c](https://github.com/kubernetes/kubernetes/pull/18215)l[u](https://github.com/kubernetes/kubernetes/pull/18215)s[t](https://github.com/kubernetes/kubernetes/pull/18215)e[r](https://github.com/kubernetes/kubernetes/pull/18215)s[.  They will automatically distribute the pods.](https://github.com/kubernetes/kubernetes/pull/18215) - - -P[l](https://github.com/kubernetes/kubernetes/pull/18215)a[n](https://github.com/kubernetes/kubernetes/pull/18215) [i](https://github.com/kubernetes/kubernetes/pull/18215)s[](https://github.com/kubernetes/kubernetes/pull/18215)t[o](https://github.com/kubernetes/kubernetes/pull/18215) [u](https://github.com/kubernetes/kubernetes/pull/18215)s[e](https://github.com/kubernetes/kubernetes/pull/18215) [t](https://github.com/kubernetes/kubernetes/pull/18215)h[e](https://github.com/kubernetes/kubernetes/pull/18215) [s](https://github.com/kubernetes/kubernetes/pull/18215)a[m](https://github.com/kubernetes/kubernetes/pull/18215)e[](https://github.com/kubernetes/kubernetes/pull/18215)A[P](https://github.com/kubernetes/kubernetes/pull/18215)I[](https://github.com/kubernetes/kubernetes/pull/18215)f[o](https://github.com/kubernetes/kubernetes/pull/18215)r[](https://github.com/kubernetes/kubernetes/pull/18215)t[h](https://github.com/kubernetes/kubernetes/pull/18215)e[](https://github.com/kubernetes/kubernetes/pull/18215)m[a](https://github.com/kubernetes/kubernetes/pull/18215)s[t](https://github.com/kubernetes/kubernetes/pull/18215)e[r](https://github.com/kubernetes/kubernetes/pull/18215) [c](https://github.com/kubernetes/kubernetes/pull/18215)l[u](https://github.com/kubernetes/kubernetes/pull/18215)s[te](https://github.com/kubernetes/kubernetes/pull/18215)r - - Quinton's Ubernetes Talk:  https://youtu.be/L2ZK24JojB4 - - Design for Ubernetes: https://github.com/kubernetes/kubernetes/pull/19313 + - SIG Report - SIG-federation - progress on Ubernetes-Lite & Ubernetes design + - Goal is to be able to have a cluster manager, so you can federate clusters. They will automatically distribute the pods. + - Plan is to use the same API for the master cluster + - [Quinton's Kubernetes Talk](https://youtu.be/L2ZK24JojB4) + - [Design for Kubernetes:](https://github.com/kubernetes/kubernetes/pull/19313) - - Conformance testing Q+A [Isaac Hollander McCreery] - + - Conformance testing Q+A Isaac Hollander McCreery - status on conformance testing for release process - - - expect to be forward compatible but not backwards + - expect to be forward compatible but not backwards - is there interest for a sig-testing meeting - testing needs to a higher priority for the project - lots of focus on trying to make this a higher priority -To get involved in the Kubernetes community consider joining our [Slack channel](http://slack.k8s.io/), taking a look at the [Kubernetes project](https://github.com/kubernetes/) on GitHub, or join the [Kubernetes-dev Google group](https://groups.google.com/forum/#!forum/kubernetes-dev). If you’re really excited, you can do all of the above and join us for the next community conversation -- January 27th, 2016. Please add yourself or a topic you want to know about to the [agenda](https://docs.google.com/document/d/1VQDIAB0OqiSjIHI8AWMvSdceWhnz56jNpZrLs6o7NJY/edit) and get a calendar invitation by joining [this group](https://groups.google.com/forum/#!forum/kubernetes-community-video-chat). - +To get involved in the Kubernetes community consider joining our [Slack channel](http://slack.k8s.io/), taking a look at the [Kubernetes project](https://github.com/kubernetes/) on GitHub, or join the [Kubernetes-dev Google group](https://groups.google.com/forum/#!forum/kubernetes-dev). If you’re really excited, you can do all of the above and join us for the next community conversation -- January 27th, 2016. Please add yourself or a topic you want to know about to [the agenda](https://docs.google.com/document/d/1VQDIAB0OqiSjIHI8AWMvSdceWhnz56jNpZrLs6o7NJY/edit) and get a calendar invitation by joining [this group](https://groups.google.com/forum/#!forum/kubernetes-community-video-chat). Still want more Kubernetes? Check out the [recording](https://www.youtube.com/watch?v=izQLFx_6kwY&feature=youtu.be&list=PL69nYSiGNLP1pkHsbPjzAewvMgGUpkCnJ) of this meeting and the growing of the archive of [Kubernetes Community Meetings](https://www.youtube.com/playlist?list=PL69nYSiGNLP1pkHsbPjzAewvMgGUpkCnJ). diff --git a/content/en/blog/_posts/2020-09-03-warnings/index.md b/content/en/blog/_posts/2020-09-03-warnings/index.md index 2e93d5a573..d88dd8328d 100644 --- a/content/en/blog/_posts/2020-09-03-warnings/index.md +++ b/content/en/blog/_posts/2020-09-03-warnings/index.md @@ -19,8 +19,7 @@ so it does not change the status code or response body in any way. This allows the server to send warnings easily readable by any API client, while remaining compatible with previous client versions. Warnings are surfaced by `kubectl` v1.19+ in `stderr` output, and by the `k8s.io/client-go` client library v0.19.0+ in log output. -The `k8s.io/client-go` behavior can be overridden [per-process](https://godoc.org/k8s.io/client-go/rest#SetDefaultWarningHandler) -or [per-client](https://godoc.org/k8s.io/client-go/rest#Config). +The `k8s.io/client-go` behavior can be [overridden per-process or per-client](#customize-client-handling). ## Deprecation Warnings @@ -173,7 +172,7 @@ apiserver_requested_deprecated_apis{removed_version="1.22"} * on(group,version,r group_right() apiserver_request_total ``` -### Audit annotations +### Audit Annotations Metrics are a fast way to check whether deprecated APIs are being used, and at what rate, but they don't include enough information to identify particular clients or API objects. @@ -251,7 +250,63 @@ Here are a couple ideas to get you started: to allow trying out a policy to verify it is working as expected before starting to enforce it * "lint" or "vet"-style webhooks, inspecting objects and surfacing warnings when best practices are not followed -## Kubectl strict mode +## Customize Client Handling + +Applications that use the `k8s.io/client-go` library to make API requests can customize +how warnings returned from the server are handled. By default, warnings are logged to +stderr as they are received, but this behavior can be customized +[per-process](https://godoc.org/k8s.io/client-go/rest#SetDefaultWarningHandler) +or [per-client](https://godoc.org/k8s.io/client-go/rest#Config). + +This example shows how to make your application behave like `kubectl`, +overriding message handling process-wide to deduplicate warnings +and highlighting messages using colored output where supported: + +```go +import ( + "os" + "k8s.io/client-go/rest" + "k8s.io/kubectl/pkg/util/term" + ... +) + +func main() { + rest.SetDefaultWarningHandler( + rest.NewWarningWriter(os.Stderr, rest.WarningWriterOptions{ + // only print a given warning the first time we receive it + Deduplicate: true, + // highlight the output with color when the output supports it + Color: term.AllowsColorOutput(os.Stderr), + }, + ), + ) + + ... +``` + +The next example shows how to construct a client that ignores warnings. +This is useful for clients that operate on metadata for all resource types +(found dynamically at runtime using the discovery API) +and do not benefit from warnings about a particular resource being deprecated. +Suppressing deprecation warnings is not recommended for clients that require use of particular APIs. + +```go +import ( + "k8s.io/client-go/rest" + "k8s.io/client-go/kubernetes" +) + +func getClientWithoutWarnings(config *rest.Config) (kubernetes.Interface, error) { + // copy to avoid mutating the passed-in config + config = rest.CopyConfig(config) + // set the warning handler for this client to ignore warnings + config.WarningHandler = rest.NoWarnings{} + // construct and return the client + return kubernetes.NewForConfig(config) +} +``` + +## Kubectl Strict Mode If you want to be sure you notice deprecations as soon as possible and get a jump start on addressing them, `kubectl` added a `--warnings-as-errors` option in v1.19. When invoked with this option, diff --git a/content/en/case-studies/adform/index.html b/content/en/case-studies/adform/index.html index be35a2d837..1de43d0637 100644 --- a/content/en/case-studies/adform/index.html +++ b/content/en/case-studies/adform/index.html @@ -3,116 +3,84 @@ title: Adform Case Study linkTitle: Adform case_study_styles: true cid: caseStudies -css: /css/style_case_studies.css logo: adform_featured_logo.png draft: false featured: true weight: 47 quote: > Kubernetes enabled the self-healing and immutable infrastructure. We can do faster releases, so our developers are really happy. They can ship our features faster than before, and that makes our clients happier. + +new_case_study_styles: true +heading_background: /images/case-studies/adform/banner1.jpg +heading_title_logo: /images/adform_logo.png +subheading: > + Improving Performance and Morale with Cloud Native +case_study_details: + - Company: AdForm + - Location: Copenhagen, Denmark + - Industry: Adtech --- -
-

CASE STUDY:
Improving Performance and Morale with Cloud Native +

Challenge

-

+

Adform's mission is to provide a secure and transparent full stack of advertising technology to enable digital ads across devices. The company has a large infrastructure: OpenStack-based private clouds running on 1,100 physical servers in 7 data centers around the world, 3 of which were opened in the past year. With the company's growth, the infrastructure team felt that "our private cloud was not really flexible enough," says IT System Engineer Edgaras Apšega. "The biggest pain point is that our developers need to maintain their virtual machines, so rolling out technology and new software takes time. We were really struggling with our releases, and we didn't have self-healing infrastructure."

-
+

Solution

-
- Company  AdForm     Location  Copenhagen, Denmark     Industry  Adtech -
- -
-
-
-
-

Challenge

- Adform’s mission is to provide a secure and transparent full stack of advertising technology to enable digital ads across devices. The company has a large infrastructure: OpenStack-based private clouds running on 1,100 physical servers in 7 data centers around the world, 3 of which were opened in the past year. With the company’s growth, the infrastructure team felt that "our private cloud was not really flexible enough," says IT System Engineer Edgaras Apšega. "The biggest pain point is that our developers need to maintain their virtual machines, so rolling out technology and new software takes time. We were really struggling with our releases, and we didn’t have self-healing infrastructure." - - -
- -

Solution

- The team, which had already been using Prometheus for monitoring, embraced Kubernetes and cloud native practices in 2017. "To start our Kubernetes journey, we had to adapt all our software, so we had to choose newer frameworks," says Apšega. "We also adopted the microservices way, so observability is much better because you can inspect the bug or the services separately." - - -
- -
+

The team, which had already been using Prometheus for monitoring, embraced Kubernetes and cloud native practices in 2017. "To start our Kubernetes journey, we had to adapt all our software, so we had to choose newer frameworks," says Apšega. "We also adopted the microservices way, so observability is much better because you can inspect the bug or the services separately."

Impact

- "Kubernetes helps our business a lot because our features are coming to market faster," says Apšega. The release process went from several hours to several minutes. Autoscaling has been at least 6 times faster than the semi-manual VM bootstrapping and application deployment required before. The team estimates that the company has experienced cost savings of 4-5x due to less hardware and fewer man hours needed to set up the hardware and virtual machines, metrics, and logging. Utilization of the hardware resources has been reduced as well, with containers notching 2-3 times more efficiency over virtual machines. "The deployments are very easy because developers just push the code and it automatically appears on Kubernetes," says Apšega. Prometheus has also had a positive impact: "It provides high availability for metrics and alerting. We monitor everything starting from hardware to applications. Having all the metrics in Grafana dashboards provides great insight on your systems." +

"Kubernetes helps our business a lot because our features are coming to market faster," says Apšega. The release process went from several hours to several minutes. Autoscaling has been at least 6 times faster than the semi-manual VM bootstrapping and application deployment required before. The team estimates that the company has experienced cost savings of 4-5x due to less hardware and fewer man hours needed to set up the hardware and virtual machines, metrics, and logging. Utilization of the hardware resources has been reduced as well, with containers notching 2-3 times more efficiency over virtual machines. "The deployments are very easy because developers just push the code and it automatically appears on Kubernetes," says Apšega. Prometheus has also had a positive impact: "It provides high availability for metrics and alerting. We monitor everything starting from hardware to applications. Having all the metrics in Grafana dashboards provides great insight on your systems."

-
+{{< case-studies/quote author="Edgaras Apšega, IT Systems Engineer, Adform" >}} +"Kubernetes enabled the self-healing and immutable infrastructure. We can do faster releases, so our developers are really happy. They can ship our features faster than before, and that makes our clients happier." +{{< /case-studies/quote >}} -
-
-
-
-"Kubernetes enabled the self-healing and immutable infrastructure. We can do faster releases, so our developers are really happy. They can ship our features faster than before, and that makes our clients happier."

— Edgaras Apšega, IT Systems Engineer, Adform +{{< case-studies/lead >}} +Adform made headlines last year when it detected the HyphBot ad fraud network that was costing some businesses hundreds of thousands of dollars a day. +{{< /case-studies/lead >}} -
-
+

With its mission to provide a secure and transparent full stack of advertising technology to enable an open internet, Adform published a white paper revealing what it did—and others could too—to limit customers' exposure to the scam.

+

In that same spirit, Adform is sharing its cloud native journey. "When you see that everyone shares their best practices, it inspires you to contribute back to the project," says IT Systems Engineer Edgaras Apšega.

-
-
-

Adform made headlines last year when it detected the HyphBot ad fraud network that was costing some businesses hundreds of thousands of dollars a day.

With its mission to provide a secure and transparent full stack of advertising technology to enable an open internet, Adform published a white paper revealing what it did—and others could too—to limit customers’ exposure to the scam.

-In that same spirit, Adform is sharing its cloud native journey. "When you see that everyone shares their best practices, it inspires you to contribute back to the project," says IT Systems Engineer Edgaras Apšega.

-The company has a large infrastructure: OpenStack-based private clouds running on 1,100 physical servers in their own seven data centers around the world, three of which were opened in the past year. With the company’s growth, the infrastructure team felt that "our private cloud was not really flexible enough," says Apšega. "The biggest pain point is that our developers need to maintain their virtual machines, so rolling out technology and new software really takes time. We were really struggling with our releases, and we didn’t have self-healing infrastructure." +

The company has a large infrastructure: OpenStack-based private clouds running on 1,100 physical servers in their own seven data centers around the world, three of which were opened in the past year. With the company's growth, the infrastructure team felt that "our private cloud was not really flexible enough," says Apšega. "The biggest pain point is that our developers need to maintain their virtual machines, so rolling out technology and new software really takes time. We were really struggling with our releases, and we didn't have self-healing infrastructure."

+{{< case-studies/quote + image="/images/case-studies/adform/banner3.jpg" + author="Edgaras Apšega, IT Systems Engineer, Adform" +>}} +"The fact that Cloud Native Computing Foundation incubated Kubernetes was a really big point for us because it was vendor neutral. And we can see that a community really gathers around it. Everyone shares their experiences, their knowledge, and the fact that it's open source, you can contribute." +{{< /case-studies/quote >}} -
-
-
-
- "The fact that Cloud Native Computing Foundation incubated Kubernetes was a really big point for us because it was vendor neutral. And we can see that a community really gathers around it. Everyone shares their experiences, their knowledge, and the fact that it’s open source, you can contribute."

— Edgaras Apšega, IT Systems Engineer, Adform -
-
-
-
+

The team, which had already been using Prometheus for monitoring, embraced Kubernetes, microservices, and cloud native practices. "The fact that Cloud Native Computing Foundation incubated Kubernetes was a really big point for us because it was vendor neutral," says Apšega. "And we can see that a community really gathers around it."

-The team, which had already been using Prometheus for monitoring, embraced Kubernetes, microservices, and cloud native practices. "The fact that Cloud Native Computing Foundation incubated Kubernetes was a really big point for us because it was vendor neutral," says Apšega. "And we can see that a community really gathers around it."

-A proof of concept project was started, with a Kubernetes cluster running on bare metal in the data center. When developers saw how quickly containers could be spun up compared to the virtual machine process, "they wanted to ship their containers in production right away, and we were still doing proof of concept," says IT Systems Engineer Andrius Cibulskis. -Of course, a lot of work still had to be done. "First of all, we had to learn Kubernetes, see all of the moving parts, how they glue together," says Apšega. "Second of all, the whole CI/CD part had to be redone, and our DevOps team had to invest more man hours to implement it. And third is that developers had to rewrite the code, and they’re still doing it." -

-The first production cluster was launched in the spring of 2018, and is now up to 20 physical machines dedicated for pods throughout three data centers, with plans for separate clusters in the other four data centers. The user-facing Adform application platform, data distribution platform, and back ends are now all running on Kubernetes. "Many APIs for critical applications are being developed for Kubernetes," says Apšega. "Teams are rewriting their applications to .NET core, because it supports containers, and preparing to move to Kubernetes. And new applications, by default, go in containers." +

A proof of concept project was started, with a Kubernetes cluster running on bare metal in the data center. When developers saw how quickly containers could be spun up compared to the virtual machine process, "they wanted to ship their containers in production right away, and we were still doing proof of concept," says IT Systems Engineer Andrius Cibulskis.

+

Of course, a lot of work still had to be done. "First of all, we had to learn Kubernetes, see all of the moving parts, how they glue together," says Apšega. "Second of all, the whole CI/CD part had to be redone, and our DevOps team had to invest more man hours to implement it. And third is that developers had to rewrite the code, and they're still doing it."

-
-
-
-
-"Releases are really nice for them, because they just push their code to Git and that’s it. They don’t have to worry about their virtual machines anymore."

— Andrius Cibulskis, IT Systems Engineer, Adform -
-
+

The first production cluster was launched in the spring of 2018, and is now up to 20 physical machines dedicated for pods throughout three data centers, with plans for separate clusters in the other four data centers. The user-facing Adform application platform, data distribution platform, and back ends are now all running on Kubernetes. "Many APIs for critical applications are being developed for Kubernetes," says Apšega. "Teams are rewriting their applications to .NET core, because it supports containers, and preparing to move to Kubernetes. And new applications, by default, go in containers."

-
-
-This big push has been driven by the real impact that these new practices have had. "Kubernetes helps our business a lot because our features are coming to market faster," says Apšega. "The deployments are very easy because developers just push the code and it automatically appears on Kubernetes." The release process went from several hours to several minutes. Autoscaling is at least six times faster than the semi-manual VM bootstrapping and application deployment required before.

-The team estimates that the company has experienced cost savings of 4-5x due to less hardware and fewer man hours needed to set up the hardware and virtual machines, metrics, and logging. Utilization of the hardware resources has been reduced as well, with containers notching two to three times more efficiency over virtual machines.

-Prometheus has also had a positive impact: "It provides high availability for metrics and alerting," says Apšega. "We monitor everything starting from hardware to applications. Having all the metrics in Grafana dashboards provides great insight on our systems." +{{< case-studies/quote + image="/images/case-studies/adform/banner4.jpg" + author="Andrius Cibulskis, IT Systems Engineer, Adform" +>}} +"Releases are really nice for them, because they just push their code to Git and that's it. They don't have to worry about their virtual machines anymore." +{{< /case-studies/quote >}} +

This big push has been driven by the real impact that these new practices have had. "Kubernetes helps our business a lot because our features are coming to market faster," says Apšega. "The deployments are very easy because developers just push the code and it automatically appears on Kubernetes." The release process went from several hours to several minutes. Autoscaling is at least six times faster than the semi-manual VM bootstrapping and application deployment required before.

+

The team estimates that the company has experienced cost savings of 4-5x due to less hardware and fewer man hours needed to set up the hardware and virtual machines, metrics, and logging. Utilization of the hardware resources has been reduced as well, with containers notching two to three times more efficiency over virtual machines.

-
+

Prometheus has also had a positive impact: "It provides high availability for metrics and alerting," says Apšega. "We monitor everything starting from hardware to applications. Having all the metrics in Grafana dashboards provides great insight on our systems."

-
-
- "I think that our company just started our cloud native journey. It seems like a huge road ahead, but we’re really happy that we joined it."

— Edgaras Apšega, IT Systems Engineer, Adform -
-
+{{< case-studies/quote author="Edgaras Apšega, IT Systems Engineer, Adform" >}} +"I think that our company just started our cloud native journey. It seems like a huge road ahead, but we're really happy that we joined it." +{{< /case-studies/quote >}} -
-All of these benefits have trickled down to individual team members, whose working lives have been changed for the better. "They used to have to get up at night to re-start some services, and now Kubernetes handles all of that," says Apšega. Adds Cibulskis: "Releases are really nice for them, because they just push their code to Git and that’s it. They don’t have to worry about their virtual machines anymore." Even the security teams have been impacted. "Security teams are always not happy," says Apšega, "and now they’re happy because they can easily inspect the containers." -The company plans to remain in the data centers for now, "mostly because we want to keep all the data, to not share it in any way," says Cibulskis, "and it’s cheaper at our scale." But, Apšega says, the possibility of using a hybrid cloud for computing is intriguing: "One of the projects we’re interested in is the Virtual Kubelet that lets you spin up the working nodes on different clouds to do some computing." -

-Apšega, Cibulskis and their colleagues are keeping tabs on how the cloud native ecosystem develops, and are excited to contribute where they can. "I think that our company just started our cloud native journey," says Apšega. "It seems like a huge road ahead, but we’re really happy that we joined it." +

All of these benefits have trickled down to individual team members, whose working lives have been changed for the better. "They used to have to get up at night to re-start some services, and now Kubernetes handles all of that," says Apšega. Adds Cibulskis: "Releases are really nice for them, because they just push their code to Git and that's it. They don't have to worry about their virtual machines anymore." Even the security teams have been impacted. "Security teams are always not happy," says Apšega, "and now they're happy because they can easily inspect the containers."

+

The company plans to remain in the data centers for now, "mostly because we want to keep all the data, to not share it in any way," says Cibulskis, "and it's cheaper at our scale." But, Apšega says, the possibility of using a hybrid cloud for computing is intriguing: "One of the projects we're interested in is the Virtual Kubelet that lets you spin up the working nodes on different clouds to do some computing."

- -
- -
+

Apšega, Cibulskis and their colleagues are keeping tabs on how the cloud native ecosystem develops, and are excited to contribute where they can. "I think that our company just started our cloud native journey," says Apšega. "It seems like a huge road ahead, but we're really happy that we joined it."

diff --git a/content/en/case-studies/adidas/index.html b/content/en/case-studies/adidas/index.html index 5f9d0da24a..3a95a5aa1b 100644 --- a/content/en/case-studies/adidas/index.html +++ b/content/en/case-studies/adidas/index.html @@ -3,106 +3,76 @@ title: adidas Case Study linkTitle: adidas case_study_styles: true cid: caseStudies -css: /css/case-studies-gradient.css featured: false + +new_case_study_styles: true +heading_background: /images/case-studies/adidas/banner1.png +heading_title_text: adidas +use_gradient_overlay: true +subheading: > + Staying True to Its Culture, adidas Got 40% of Its Most Impactful Systems Running on Kubernetes in a Year +case_study_details: + - Company: adidas + - Location: Herzogenaurach, Germany + - Industry: Fashion --- -​ +

Challenge

+ +

In recent years, the adidas team was happy with its software choices from a technology perspective—but accessing all of the tools was a problem. For instance, "just to get a developer VM, you had to send a request form, give the purpose, give the title of the project, who's responsible, give the internal cost center a call so that they can do recharges," says Daniel Eichten, Senior Director of Platform Engineering. "The best case is you got your machine in half an hour. Worst case is half a week or sometimes even a week."

+ +

Solution

+ +

To improve the process, "we started from the developer point of view," and looked for ways to shorten the time it took to get a project up and running and into the adidas infrastructure, says Senior Director of Platform Engineering Fernando Cornago. They found the solution with containerization, agile development, continuous delivery, and a cloud native platform that includes Kubernetes and Prometheus.

-
-

CASE STUDY: adidas

-
Staying True to Its Culture, adidas Got 40% of Its Most Impactful Systems Running on Kubernetes in a Year
-
-​ -​ -
- Company  adidas     Location  Herzogenaurach, Germany     Industry  Fashion -
-​ -
-
-
-
-

Challenge

- In recent years, the adidas team was happy with its software choices from a technology perspective—but accessing all of the tools was a problem. For instance, “just to get a developer VM, you had to send a request form, give the purpose, give the title of the project, who’s responsible, give the internal cost center a call so that they can do recharges,” says Daniel Eichten, Senior Director of Platform Engineering. “The best case is you got your machine in half an hour. Worst case is half a week or sometimes even a week.” -

Solution

- To improve the process, “we started from the developer point of view,” and looked for ways to shorten the time it took to get a project up and running and into the adidas infrastructure, says Senior Director of Platform Engineering Fernando Cornago. They found the solution with containerization, agile development, continuous delivery, and a cloud native platform that includes Kubernetes and Prometheus. -​

Impact

- Just six months after the project began, 100% of the adidas e-commerce site was running on Kubernetes. Load time for the e-commerce site was reduced by half. Releases went from every 4-6 weeks to 3-4 times a day. With 4,000 pods, 200 nodes, and 80,000 builds per month, adidas is now running 40% of its most critical, impactful systems on its cloud native platform. -
-
-
-
-
- "For me, Kubernetes is a platform made by engineers for engineers. It’s relieving the development team from tasks that they don’t want to do, but at the same time giving the visibility of what is behind the curtain, so they can also control it." -

- FERNANDO CORNAGO, SENIOR DIRECTOR OF PLATFORM ENGINEERING AT ADIDAS

-
-
-​ -​ -
-
-

In recent years, the adidas team was happy with its software choices from a technology perspective—but accessing all of the tools was a problem.

-

For engineers at adidas, says Daniel Eichten, Senior Director of Platform Engineering, "it felt like being an artist with your hands tied behind your back, and you’re supposed to paint something."

-

- For instance, "just to get a developer VM, you had to send a request form, give the purpose, give the title of the project, who’s responsible, give the internal cost center a call so that they can do recharges," says Eichten. "Eventually, after a ton of approvals, then the provisioning of the machine happened within minutes, and then the best case is you got your machine in half an hour. Worst case is half a week or sometimes even a week." -

-

- To improve the process, "we started from the developer point of view," and looked for ways to shorten the time it took to get a project up and running and into the adidas infrastructure, says Senior Director of Platform Engineering Fernando Cornago.

-
-
-​ -
-
- "I call our cloud native platform the field of dreams. We built it, and we never anticipated that people would come and just love it."

- DANIEL EICHTEN, SENIOR DIRECTOR OF PLATFORM ENGINEERING AT ADIDAS

-
-
-​ -
-
-

- "We were engineers before," adds Eichten. "We know what a typical engineer needs, is craving for, what he or she doesn’t want to take care of. For us it was pretty clear. We filled the gaps that no one wants to take care of, and we make the stuff that is usually painful as painless as possible." The goals: to improve speed, operability, and observability. -

-

- Cornago and Eichten found the solution with containerization, agile development, continuous delivery, and a cloud native platform that includes Kubernetes and Prometheus. "Choosing Kubernetes was pretty clear," says Eichten. "Day zero, deciding, easy. Day one, installing, configuring, easy. Day two, keeping it up and running even with small workloads, if something goes wrong, you don’t know how these things work in detail, you’re lost. For day two problems, we needed a partner who’s helping us." -

-

- In early 2017, adidas chose Giant Swarm to consult, install, configure, and run all of its Kubernetes clusters in AWS and on premise. "There is no competitive edge over our competitors like Puma or Nike in running and operating a Kubernetes cluster," says Eichten. "Our competitive edge is that we teach our internal engineers how to build cool e-comm stores that are fast, that are resilient, that are running perfectly." -

-
-
-​ -​ -
-
- “There is no competitive edge over our competitors like Puma or Nike in running and operating a Kubernetes cluster. Our competitive edge is that we teach our internal engineers how to build cool e-comm stores that are fast, that are resilient, that are running perfectly.”

- DANIEL EICHTEN, SENIOR DIRECTOR OF PLATFORM ENGINEERING AT ADIDAS

-
-
-​ -
-
-

- Adds Cornago: “For me, our Kubernetes platform is made by engineers for engineers. It’s relieving the development team from tasks that they don’t want to do, but at the same time giving the visibility of what is behind the curtain, so they can also control it.” -

-

- Case in point: For Cyber Week, the team has to create a lot of custom metrics. In November 2017, “because we used the same Prometheus that we use for monitoring the cluster, we really filled the Prometheus database, and we were not able to reduce the retention period [enough],” says Cornago. So during the freeze period before the peak shopping week, five engineers from the platform team worked with five engineers from the e-comm team to figure out a federated solution that was implemented in two days. -

-

- In addition to being ready for Cyber Week—100% of the adidas e-commerce site was running on Kubernetes then, just six months after the project began—the cloud native stack has had other impressive results. Load time for the e-commerce site was reduced by half. Releases went from every 4-6 weeks to 3-4 times a day. With 4,000 pods, 200 nodes, and 80,000 builds per month, adidas is now running 40% of its most critical, impactful systems on its cloud native platform. -

-

- And adoption has spread quickly among adidas’s 300-strong engineering corps. “I call our cloud native platform the field of dreams,” says Eichten. “We built it, and we never anticipated that people would come and just love it.” -

-

- For one thing, “everybody who can touch a line of code” has spent one full week onboarding and learning the platform with members of the 35-person platform engineering team, says Cornago. “We try to spend 50% of our time sitting with the teams, because this is the only way to understand how our platform is being used. And this is how the teams will feel safe that there is someone on the other side of the wall, also feeling the pain.” -

-

- Additionally, Cornago and Eichten took advantage of the fact that as a fashion athletic wear brand, adidas has sports and competition in its DNA. “Top-down mandates don’t work at adidas, but gamification works,” says Cornago. “So this year we had a DevOps Cup competition. Every team created new technical capabilities and had a hypothesis of how this affected business value. We announced the winner at a big internal tech summit with more than 600 people. It’s been really, really useful for the teams.” -

-

- So if they had any advice for other companies looking to start a cloud native journey, it would be this: “There is no one-size-fits-all for all companies,” says Cornago. “Apply your company’s culture to everything that you do.” -

-
-
- + +

Just six months after the project began, 100% of the adidas e-commerce site was running on Kubernetes. Load time for the e-commerce site was reduced by half. Releases went from every 4-6 weeks to 3-4 times a day. With 4,000 pods, 200 nodes, and 80,000 builds per month, adidas is now running 40% of its most critical, impactful systems on its cloud native platform.

+ +{{< case-studies/quote + image="/images/case-studies/adidas/banner2.png" + author="FERNANDO CORNAGO, SENIOR DIRECTOR OF PLATFORM ENGINEERING AT ADIDAS" +>}} +"For me, Kubernetes is a platform made by engineers for engineers. It's relieving the development team from tasks that they don't want to do, but at the same time giving the visibility of what is behind the curtain, so they can also control it." +{{< /case-studies/quote >}} + +{{< case-studies/lead >}} +In recent years, the adidas team was happy with its software choices from a technology perspective—but accessing all of the tools was a problem. +{{< /case-studies/lead >}} + +

For engineers at adidas, says Daniel Eichten, Senior Director of Platform Engineering, "it felt like being an artist with your hands tied behind your back, and you're supposed to paint something."

+ +

For instance, "just to get a developer VM, you had to send a request form, give the purpose, give the title of the project, who's responsible, give the internal cost center a call so that they can do recharges," says Eichten. "Eventually, after a ton of approvals, then the provisioning of the machine happened within minutes, and then the best case is you got your machine in half an hour. Worst case is half a week or sometimes even a week."

+ +

To improve the process, "we started from the developer point of view," and looked for ways to shorten the time it took to get a project up and running and into the adidas infrastructure, says Senior Director of Platform Engineering Fernando Cornago.

+ +{{< case-studies/quote author="DANIEL EICHTEN, SENIOR DIRECTOR OF PLATFORM ENGINEERING AT ADIDAS" >}} +"I call our cloud native platform the field of dreams. We built it, and we never anticipated that people would come and just love it." +{{< /case-studies/quote >}} + +

"We were engineers before," adds Eichten. "We know what a typical engineer needs, is craving for, what he or she doesn't want to take care of. For us it was pretty clear. We filled the gaps that no one wants to take care of, and we make the stuff that is usually painful as painless as possible." The goals: to improve speed, operability, and observability.

+ +

Cornago and Eichten found the solution with containerization, agile development, continuous delivery, and a cloud native platform that includes Kubernetes and Prometheus. "Choosing Kubernetes was pretty clear," says Eichten. "Day zero, deciding, easy. Day one, installing, configuring, easy. Day two, keeping it up and running even with small workloads, if something goes wrong, you don't know how these things work in detail, you're lost. For day two problems, we needed a partner who's helping us."

+ +

In early 2017, adidas chose Giant Swarm to consult, install, configure, and run all of its Kubernetes clusters in AWS and on premise. "There is no competitive edge over our competitors like Puma or Nike in running and operating a Kubernetes cluster," says Eichten. "Our competitive edge is that we teach our internal engineers how to build cool e-comm stores that are fast, that are resilient, that are running perfectly."

+ +{{< case-studies/quote + image="/images/case-studies/adidas/banner3.png" + author="DANIEL EICHTEN, SENIOR DIRECTOR OF PLATFORM ENGINEERING AT ADIDAS" +>}} +"There is no competitive edge over our competitors like Puma or Nike in running and operating a Kubernetes cluster. Our competitive edge is that we teach our internal engineers how to build cool e-comm stores that are fast, that are resilient, that are running perfectly." +{{< /case-studies/quote >}} + +

Adds Cornago: "For me, our Kubernetes platform is made by engineers for engineers. It's relieving the development team from tasks that they don't want to do, but at the same time giving the visibility of what is behind the curtain, so they can also control it."

+ +

Case in point: For Cyber Week, the team has to create a lot of custom metrics. In November 2017, "because we used the same Prometheus that we use for monitoring the cluster, we really filled the Prometheus database, and we were not able to reduce the retention period [enough]," says Cornago. So during the freeze period before the peak shopping week, five engineers from the platform team worked with five engineers from the e-comm team to figure out a federated solution that was implemented in two days.

+ +

In addition to being ready for Cyber Week—100% of the adidas e-commerce site was running on Kubernetes then, just six months after the project began—the cloud native stack has had other impressive results. Load time for the e-commerce site was reduced by half. Releases went from every 4-6 weeks to 3-4 times a day. With 4,000 pods, 200 nodes, and 80,000 builds per month, adidas is now running 40% of its most critical, impactful systems on its cloud native platform.

+ +

And adoption has spread quickly among adidas's 300-strong engineering corps. "I call our cloud native platform the field of dreams," says Eichten. "We built it, and we never anticipated that people would come and just love it."

+ +

For one thing, "everybody who can touch a line of code" has spent one full week onboarding and learning the platform with members of the 35-person platform engineering team, says Cornago. "We try to spend 50% of our time sitting with the teams, because this is the only way to understand how our platform is being used. And this is how the teams will feel safe that there is someone on the other side of the wall, also feeling the pain."

+ +

Additionally, Cornago and Eichten took advantage of the fact that as a fashion athletic wear brand, adidas has sports and competition in its DNA. "Top-down mandates don't work at adidas, but gamification works," says Cornago. "So this year we had a DevOps Cup competition. Every team created new technical capabilities and had a hypothesis of how this affected business value. We announced the winner at a big internal tech summit with more than 600 people. It's been really, really useful for the teams."

+ +

So if they had any advice for other companies looking to start a cloud native journey, it would be this: "There is no one-size-fits-all for all companies," says Cornago. "Apply your company's culture to everything that you do."

diff --git a/content/en/case-studies/amadeus/index.html b/content/en/case-studies/amadeus/index.html index 8b6294d4f9..bd647003ec 100644 --- a/content/en/case-studies/amadeus/index.html +++ b/content/en/case-studies/amadeus/index.html @@ -1,105 +1,84 @@ --- title: Amadeus Case Study - case_study_styles: true cid: caseStudies -css: /css/style_amadeus.css + +new_case_study_styles: true +heading_background: /images/case-studies/amadeus/banner1.jpg +heading_title_logo: /images/amadeus_logo.png +subheading: > + Another Technical Evolution for a 30-Year-Old Company +case_study_details: + - Company: Amadeus IT Group + - Location: Madrid, Spain + - Industry: Travel Technology --- -
-

CASE STUDY:
Another Technical Evolution for a 30-Year-Old Company -

-
-
- Company  Amadeus IT Group     Location  Madrid, Spain     Industry  Travel Technology -
-
-
-
-
-

Challenge

- In the past few years, Amadeus, which provides IT solutions to the travel industry around the world, found itself in need of a new platform for the 5,000 services supported by its service-oriented architecture. The 30-year-old company operates its own data center in Germany, and there were growing demands internally and externally for solutions that needed to be geographically dispersed. And more generally, "we had objectives of being even more highly available," says Eric Mountain, Senior Expert, Distributed Systems at Amadeus. Among the company’s goals: to increase automation in managing its infrastructure, optimize the distribution of workloads, use data center resources more efficiently, and adopt new technologies more easily. -
-
-

Solution

- Mountain has been overseeing the company’s migration to Kubernetes, using OpenShift Container Platform, Red Hat’s enterprise container platform. -

+

Challenge

+ +

In the past few years, Amadeus, which provides IT solutions to the travel industry around the world, found itself in need of a new platform for the 5,000 services supported by its service-oriented architecture. The 30-year-old company operates its own data center in Germany, and there were growing demands internally and externally for solutions that needed to be geographically dispersed. And more generally, "we had objectives of being even more highly available," says Eric Mountain, Senior Expert, Distributed Systems at Amadeus. Among the company's goals: to increase automation in managing its infrastructure, optimize the distribution of workloads, use data center resources more efficiently, and adopt new technologies more easily.

+ +

Solution

+ +

Mountain has been overseeing the company's migration to Kubernetes, using OpenShift Container Platform, Red Hat's enterprise container platform.

+

Impact

- One of the first projects the team deployed in Kubernetes was the Amadeus Airline Cloud Availability solution, which helps manage ever-increasing flight-search volume. "It’s now handling in production several thousand transactions per second, and it’s deployed in multiple data centers throughout the world," says Mountain. "It’s not a migration of an existing workload; it’s a whole new workload that we couldn’t have done otherwise. [This platform] gives us access to market opportunities that we didn’t have before." -
-
-
-
-
- "We want multi-data center capabilities, and we want them for our mainstream system as well. We didn’t think that we could achieve them with our existing system. We need new automation, things that Kubernetes and OpenShift bring."

- Eric Mountain, Senior Expert, Distributed Systems at Amadeus IT Group -
-
-
+

One of the first projects the team deployed in Kubernetes was the Amadeus Airline Cloud Availability solution, which helps manage ever-increasing flight-search volume. "It's now handling in production several thousand transactions per second, and it's deployed in multiple data centers throughout the world," says Mountain. "It's not a migration of an existing workload; it's a whole new workload that we couldn't have done otherwise. [This platform] gives us access to market opportunities that we didn't have before."

-
-

In his two decades at Amadeus, Eric Mountain has been the migrations guy.

- Back in the day, he worked on the company’s move from Unix to Linux, and now he’s overseeing the journey to cloud native. "Technology just keeps changing, and we embrace it," he says. "We are celebrating our 30 years this year, and we continue evolving and innovating to stay cost-efficient and enhance everyone’s travel experience, without interrupting workflows for the customers who depend on our technology."

- That was the challenge that Amadeus—which provides IT solutions to the travel industry around the world, from flight searches to hotel bookings to customer feedback—faced in 2014. The technology team realized it was in need of a new platform for the 5,000 services supported by its service-oriented architecture.

- The tipping point occurred when they began receiving many requests, internally and externally, for solutions that needed to be geographically outside the company’s main data center in Germany. "Some requests were for running our applications on customer premises," Mountain says. "There were also new services we were looking to offer that required response time to the order of a few hundred milliseconds, which we couldn’t achieve with transatlantic traffic. Or at least, not without eating into a considerable portion of the time available to our applications for them to process individual queries."

- More generally, the company was interested in leveling up on high availability, increasing automation in managing infrastructure, optimizing the distribution of workloads and using data center resources more efficiently. "We have thousands and thousands of servers," says Mountain. "These servers are assigned roles, so even if the setup is highly automated, the machine still has a given role. It’s wasteful on many levels. For instance, an application doesn’t necessarily use the machine very optimally. Virtualization can help a bit, but it’s not a silver bullet. If that machine breaks, you still want to repair it because it has that role and you can’t simply say, ‘Well, I’ll bring in another machine and give it that role.’ It’s not fast. It’s not efficient. So we wanted the next level of automation."

-
-
+{{< case-studies/quote author="Eric Mountain, Senior Expert, Distributed Systems at Amadeus IT Group" >}} +"We want multi-data center capabilities, and we want them for our mainstream system as well. We didn't think that we could achieve them with our existing system. We need new automation, things that Kubernetes and OpenShift bring." +{{< /case-studies/quote >}} -
-
- "We hope that if we build on what others have built, what we do might actually be upstream-able. As Kubernetes and OpenShift progress, we see that we are indeed able to remove some of the additional layers we implemented to compensate for gaps we perceived earlier." -
-
+{{< case-studies/lead >}} +In his two decades at Amadeus, Eric Mountain has been the migrations guy. +{{< /case-studies/lead >}} -
-
- While mainly a C++ and Java shop, Amadeus also wanted to be able to adopt new technologies more easily. Some of its developers had started using languages like Python and databases like Couchbase, but Mountain wanted still more options, he says, "in order to better adapt our technical solutions to the products we offer, and open up entirely new possibilities to our developers." Working with recent technologies and cool new things would also make it easier to attract new talent. -

- All of those needs led Mountain and his team on a search for a new platform. "We did a set of studies and proofs of concept over a fairly short period, and we considered many technologies," he says. "In the end, we were left with three choices: build everything on premise, build on top of Kubernetes whatever happens to be missing from our point of view, or go with OpenShift and build whatever remains there." -

- The team decided against building everything themselves—though they’d done that sort of thing in the past—because "people were already inventing things that looked good," says Mountain. -

- Ultimately, they went with OpenShift Container Platform, Red Hat’s Kubernetes-based enterprise offering, instead of building on top of Kubernetes because "there was a lot of synergy between what we wanted and the way Red Hat was anticipating going with OpenShift," says Mountain. "They were clearly developing Kubernetes, and developing certain things ahead of time in OpenShift, which were important to us, such as more security." -

- The hope was that those particular features would eventually be built into Kubernetes, and, in the case of security, Mountain feels that has happened. "We realize that there’s always a certain amount of automation that we will probably have to develop ourselves to compensate for certain gaps," says Mountain. "The less we do that, the better for us. We hope that if we build on what others have built, what we do might actually be upstream-able. As Kubernetes and OpenShift progress, we see that we are indeed able to remove some of the additional layers we implemented to compensate for gaps we perceived earlier." -
-
+

Back in the day, he worked on the company's move from Unix to Linux, and now he's overseeing the journey to cloud native. "Technology just keeps changing, and we embrace it," he says. "We are celebrating our 30 years this year, and we continue evolving and innovating to stay cost-efficient and enhance everyone's travel experience, without interrupting workflows for the customers who depend on our technology."

-
-
- "It’s not a migration of an existing workload; it’s a whole new workload that we couldn’t have done otherwise. [This platform] gives us access to market opportunities that we didn’t have before." -
-
+

That was the challenge that Amadeus—which provides IT solutions to the travel industry around the world, from flight searches to hotel bookings to customer feedback—faced in 2014. The technology team realized it was in need of a new platform for the 5,000 services supported by its service-oriented architecture.

-
-
- The first project the team tackled was one that they knew had to run outside the data center in Germany. Because of the project’s needs, "We couldn’t rely only on the built-in Kubernetes service discovery; we had to layer on top of that an extra service discovery level that allows us to load balance at the operation level within our system," says Mountain. They also built a stream dedicated to monitoring, which at the time wasn’t offered in the Kubernetes or OpenShift ecosystem. Now that Prometheus and other products are available, Mountain says the company will likely re-evaluate their monitoring system: "We obviously always like to leverage what Kubernetes and OpenShift can offer." -

- The second project ended up going into production first: the Amadeus Airline Cloud Availability solution, which helps manage ever-increasing flight-search volume and was deployed in public cloud. Launched in early 2016, it is "now handling in production several thousand transactions per second, and it’s deployed in multiple data centers throughout the world," says Mountain. "It’s not a migration of an existing workload; it’s a whole new workload that we couldn’t have done otherwise. [This platform] gives us access to market opportunities that we didn’t have before." -

- Having been through this kind of technical evolution more than once, Mountain has advice on how to handle the cultural changes. "That’s one aspect that we can tackle progressively," he says. "We have to go on supplying our customers with new features on our pre-existing products, and we have to keep existing products working. So we can’t simply do absolutely everything from one day to the next. And we mustn’t sell it that way." -

- The first order of business, then, is to pick one or two applications to demonstrate that the technology works. Rather than choosing a high-impact, high-risk project, Mountain’s team selected a smaller application that was representative of all the company’s other applications in its complexity: "We just made sure we picked something that’s complex enough, and we showed that it can be done." -
-
+

The tipping point occurred when they began receiving many requests, internally and externally, for solutions that needed to be geographically outside the company's main data center in Germany. "Some requests were for running our applications on customer premises," Mountain says. "There were also new services we were looking to offer that required response time to the order of a few hundred milliseconds, which we couldn't achieve with transatlantic traffic. Or at least, not without eating into a considerable portion of the time available to our applications for them to process individual queries."

-
-
- "The bottom line is we want these multi-data center capabilities, and we want them as well for our mainstream system," he says. "And we don’t think that we can implement them with our previous system. We need the new automation, homogeneity, and scale that Kubernetes and OpenShift bring." -
-
+

More generally, the company was interested in leveling up on high availability, increasing automation in managing infrastructure, optimizing the distribution of workloads and using data center resources more efficiently. "We have thousands and thousands of servers," says Mountain. "These servers are assigned roles, so even if the setup is highly automated, the machine still has a given role. It's wasteful on many levels. For instance, an application doesn't necessarily use the machine very optimally. Virtualization can help a bit, but it's not a silver bullet. If that machine breaks, you still want to repair it because it has that role and you can't simply say, 'Well, I'll bring in another machine and give it that role.' It's not fast. It's not efficient. So we wanted the next level of automation."

-
-
- Next comes convincing people. "On the operations side and on the R&D side, there will be people who say quite rightly, ‘There is a system, and it works, so why change?’" Mountain says. "The only thing that really convinces people is showing them the value." For Amadeus, people realized that the Airline Cloud Availability product could not have been made available on the public cloud with the company’s existing system. The question then became, he says, "Do we go into a full-blown migration? Is that something that is justified?" -

- "The bottom line is we want these multi-data center capabilities, and we want them as well for our mainstream system," he says. "And we don’t think that we can implement them with our previous system. We need the new automation, homogeneity, and scale that Kubernetes and OpenShift bring." -

- So how do you get everyone on board? "Make sure you have good links between your R&D and your operations," he says. "Also make sure you’re going to talk early on to the investors and stakeholders. Figure out what it is that they will be expecting from you, that will convince them or not, that this is the right way for your company." -

- His other advice is simply to make the technology available for people to try it. "Kubernetes and OpenShift Origin are open source software, so there’s no complicated license key for the evaluation period and you’re not limited to 30 days," he points out. "Just go and get it running." Along with that, he adds, "You’ve got to be prepared to rethink how you do things. Of course making your applications as cloud native as possible is how you’ll reap the most benefits: 12 factors, CI/CD, which is continuous integration, continuous delivery, but also continuous deployment." -

- And while they explore that aspect of the technology, Mountain and his team will likely be practicing what he preaches to others taking the cloud native journey. "See what happens when you break it, because it’s important to understand the limits of the system," he says. Or rather, he notes, the advantages of it. "Breaking things on Kube is actually one of the nice things about it—it recovers. It’s the only real way that you’ll see that you might be able to do things." -
-
+{{< case-studies/quote image="/images/case-studies/amadeus/banner3.jpg" >}} +"We hope that if we build on what others have built, what we do might actually be upstream-able. As Kubernetes and OpenShift progress, we see that we are indeed able to remove some of the additional layers we implemented to compensate for gaps we perceived earlier." +{{< /case-studies/quote >}} + +

While mainly a C++ and Java shop, Amadeus also wanted to be able to adopt new technologies more easily. Some of its developers had started using languages like Python and databases like Couchbase, but Mountain wanted still more options, he says, "in order to better adapt our technical solutions to the products we offer, and open up entirely new possibilities to our developers." Working with recent technologies and cool new things would also make it easier to attract new talent.

+ +

All of those needs led Mountain and his team on a search for a new platform. "We did a set of studies and proofs of concept over a fairly short period, and we considered many technologies," he says. "In the end, we were left with three choices: build everything on premise, build on top of Kubernetes whatever happens to be missing from our point of view, or go with OpenShift and build whatever remains there."

+ +

The team decided against building everything themselves—though they'd done that sort of thing in the past—because "people were already inventing things that looked good," says Mountain.

+ +

Ultimately, they went with OpenShift Container Platform, Red Hat's Kubernetes-based enterprise offering, instead of building on top of Kubernetes because "there was a lot of synergy between what we wanted and the way Red Hat was anticipating going with OpenShift," says Mountain. "They were clearly developing Kubernetes, and developing certain things ahead of time in OpenShift, which were important to us, such as more security."

+ +

The hope was that those particular features would eventually be built into Kubernetes, and, in the case of security, Mountain feels that has happened. "We realize that there's always a certain amount of automation that we will probably have to develop ourselves to compensate for certain gaps," says Mountain. "The less we do that, the better for us. We hope that if we build on what others have built, what we do might actually be upstream-able. As Kubernetes and OpenShift progress, we see that we are indeed able to remove some of the additional layers we implemented to compensate for gaps we perceived earlier."

+ +{{< case-studies/quote image="/images/case-studies/amadeus/banner4.jpg" >}} +"It's not a migration of an existing workload; it's a whole new workload that we couldn't have done otherwise. [This platform] gives us access to market opportunities that we didn't have before." +{{< /case-studies/quote >}} + +

The first project the team tackled was one that they knew had to run outside the data center in Germany. Because of the project's needs, "We couldn't rely only on the built-in Kubernetes service discovery; we had to layer on top of that an extra service discovery level that allows us to load balance at the operation level within our system," says Mountain. They also built a stream dedicated to monitoring, which at the time wasn't offered in the Kubernetes or OpenShift ecosystem. Now that Prometheus and other products are available, Mountain says the company will likely re-evaluate their monitoring system: "We obviously always like to leverage what Kubernetes and OpenShift can offer." +

+ +

The second project ended up going into production first: the Amadeus Airline Cloud Availability solution, which helps manage ever-increasing flight-search volume and was deployed in public cloud. Launched in early 2016, it is "now handling in production several thousand transactions per second, and it's deployed in multiple data centers throughout the world," says Mountain. "It's not a migration of an existing workload; it's a whole new workload that we couldn't have done otherwise. [This platform] gives us access to market opportunities that we didn't have before."

+ +

Having been through this kind of technical evolution more than once, Mountain has advice on how to handle the cultural changes. "That's one aspect that we can tackle progressively," he says. "We have to go on supplying our customers with new features on our pre-existing products, and we have to keep existing products working. So we can't simply do absolutely everything from one day to the next. And we mustn't sell it that way."

+ +

The first order of business, then, is to pick one or two applications to demonstrate that the technology works. Rather than choosing a high-impact, high-risk project, Mountain's team selected a smaller application that was representative of all the company's other applications in its complexity: "We just made sure we picked something that's complex enough, and we showed that it can be done."

+ +{{< case-studies/quote >}} +"The bottom line is we want these multi-data center capabilities, and we want them as well for our mainstream system," he says. "And we don't think that we can implement them with our previous system. We need the new automation, homogeneity, and scale that Kubernetes and OpenShift bring." +{{< /case-studies/quote >}} + +

Next comes convincing people. "On the operations side and on the R&D side, there will be people who say quite rightly, 'There is a system, and it works, so why change?'" Mountain says. "The only thing that really convinces people is showing them the value." For Amadeus, people realized that the Airline Cloud Availability product could not have been made available on the public cloud with the company's existing system. The question then became, he says, "Do we go into a full-blown migration? Is that something that is justified?"

+ +

"The bottom line is we want these multi-data center capabilities, and we want them as well for our mainstream system," he says. "And we don't think that we can implement them with our previous system. We need the new automation, homogeneity, and scale that Kubernetes and OpenShift bring."

+ +

So how do you get everyone on board? "Make sure you have good links between your R&D and your operations," he says. "Also make sure you're going to talk early on to the investors and stakeholders. Figure out what it is that they will be expecting from you, that will convince them or not, that this is the right way for your company."

+ +

His other advice is simply to make the technology available for people to try it. "Kubernetes and OpenShift Origin are open source software, so there's no complicated license key for the evaluation period and you're not limited to 30 days," he points out. "Just go and get it running." Along with that, he adds, "You've got to be prepared to rethink how you do things. Of course making your applications as cloud native as possible is how you'll reap the most benefits: 12 factors, CI/CD, which is continuous integration, continuous delivery, but also continuous deployment."

+ +

And while they explore that aspect of the technology, Mountain and his team will likely be practicing what he preaches to others taking the cloud native journey. "See what happens when you break it, because it's important to understand the limits of the system," he says. Or rather, he notes, the advantages of it. "Breaking things on Kube is actually one of the nice things about it—it recovers. It's the only real way that you'll see that you might be able to do things."

diff --git a/content/en/case-studies/ancestry/index.html b/content/en/case-studies/ancestry/index.html index a992a284ac..8cab3c19c6 100644 --- a/content/en/case-studies/ancestry/index.html +++ b/content/en/case-studies/ancestry/index.html @@ -1,117 +1,92 @@ --- title: Ancestry Case Study - case_study_styles: true cid: caseStudies -css: /css/style_ancestry.css + +new_case_study_styles: true +heading_background: /images/case-studies/ancestry/banner1.jpg +heading_title_logo: /images/ancestry_logo.png +subheading: > + Digging Into the Past With New Technology +case_study_details: + - Company: Ancestry + - Location: Lehi, Utah + - Industry: Internet Company, Online Services --- -
-

CASE STUDY:
Digging Into the Past With New Technology

- -
- -
- Company  Ancestry     Location  Lehi, Utah     Industry  Internet Company, Online Services -
- -
- -
-
-
-

Challenge

-Ancestry, the global leader in family history and consumer genomics, uses sophisticated engineering and technology to help everyone, everywhere discover the story of what led to them. The company has spent more than 30 years innovating and building products and technologies that at their core, result in real and emotional human responses. Ancestry currently serves more than 2.6 million paying subscribers, holds 20 billion historical records, 90 million family trees and more than four million people are in its AncestryDNA network, making it the largest consumer genomics DNA network in the world. The company's popular website, ancestry.com, has been working with big data long before the term was popularized. The site was built on hundreds of services, technologies and a traditional deployment methodology. "It's worked well for us in the past," says Paul MacKay, software engineer and architect at Ancestry, "but had become quite cumbersome in its processing and is time-consuming. As a primarily online service, we are constantly looking for ways to accelerate to be more agile in delivering our solutions and our products." -
+

Ancestry, the global leader in family history and consumer genomics, uses sophisticated engineering and technology to help everyone, everywhere discover the story of what led to them. The company has spent more than 30 years innovating and building products and technologies that at their core, result in real and emotional human responses. Ancestry currently serves more than 2.6 million paying subscribers, holds 20 billion historical records, 90 million family trees and more than four million people are in its AncestryDNA network, making it the largest consumer genomics DNA network in the world. The company's popular website, ancestry.com, has been working with big data long before the term was popularized. The site was built on hundreds of services, technologies and a traditional deployment methodology. "It's worked well for us in the past," says Paul MacKay, software engineer and architect at Ancestry, "but had become quite cumbersome in its processing and is time-consuming. As a primarily online service, we are constantly looking for ways to accelerate to be more agile in delivering our solutions and our products."

-
+

Solution

-
-

Solution

+

The company is transitioning to cloud native infrastructure, using Docker containerization, Kubernetes orchestration and Prometheus for cluster monitoring.

- The company is transitioning to cloud native infrastructure, using Docker containerization, Kubernetes orchestration and Prometheus for cluster monitoring.
-

Impact

- "Every single product, every decision we make at Ancestry, focuses on delighting our customers with intimate, sometimes life-changing discoveries about themselves and their families," says MacKay. "As the company continues to grow, the increased productivity gains from using Kubernetes has helped Ancestry make customer discoveries faster. With the move to Dockerization for example, instead of taking between 20 to 50 minutes to deploy a new piece of code, we can now deploy in under a minute for much of our code. We’ve truly experienced significant time savings in addition to the various features and benefits from cloud native and Kubernetes-type technologies." -
-
-
-
-
- "At a certain point, you have to step back if you're going to push a new technology and get key thought leaders with engineers within the organization to become your champions for new technology adoption. At training sessions, the development teams were always the ones that were saying, 'Kubernetes saved our time tremendously; it's an enabler. It really is incredible.'"

- PAUL MACKAY, SOFTWARE ENGINEER AND ARCHITECT AT ANCESTRY -
-
+

"Every single product, every decision we make at Ancestry, focuses on delighting our customers with intimate, sometimes life-changing discoveries about themselves and their families," says MacKay. "As the company continues to grow, the increased productivity gains from using Kubernetes has helped Ancestry make customer discoveries faster. With the move to Dockerization for example, instead of taking between 20 to 50 minutes to deploy a new piece of code, we can now deploy in under a minute for much of our code. We've truly experienced significant time savings in addition to the various features and benefits from cloud native and Kubernetes-type technologies."

-
-
-

It started with a Shaky Leaf.

+{{< case-studies/quote author="PAUL MACKAY, SOFTWARE ENGINEER AND ARCHITECT AT ANCESTRY" >}} +"At a certain point, you have to step back if you're going to push a new technology and get key thought leaders with engineers within the organization to become your champions for new technology adoption. At training sessions, the development teams were always the ones that were saying, 'Kubernetes saved our time tremendously; it's an enabler. It really is incredible.'" +{{< /case-studies/quote >}} - Since its introduction a decade ago, the Shaky Leaf icon has become one of Ancestry's signature features, which signals to users that there's a helpful hint you can use to find out more about your family tree.

- So when the company decided to begin moving its infrastructure to cloud native technology, the first service that was launched on Kubernetes, the open source platform for managing application containers across clusters of hosts, was this hint system. Think of it as Amazon's recommended products, but instead of recommending products the company recommends records, stories, or familial connections. "It was a very important part of the site," says Ancestry software engineer and architect Paul MacKay, "but also small enough for a pilot project that we knew we could handle in a very appropriate, secure way."

- And when it went live smoothly in early 2016, "our deployment time for this service literally was cut down from 50 minutes to 2 or 5 minutes," MacKay adds. "The development team was just thrilled because we're focused on supplying a great experience for our customers. And that means features, it means stability, it means all those things that we need for a first-in-class type operation."

- The stability of that Shaky Leaf was a signal for MacKay and his team that their decision to embrace cloud native technologies was the right one for the company. With a private data center, Ancestry built its website (which launched in 1996) on hundreds of services and technologies and a traditional deployment methodology. "It worked well for us in the past, but the sum of the legacy systems became quite cumbersome in its processing and was time-consuming," says MacKay. "We were looking for other ways to accelerate, to be more agile in delivering our solutions and our products." -
-
+{{< case-studies/lead >}} +It started with a Shaky Leaf. +{{< /case-studies/lead >}} -
-
+

Since its introduction a decade ago, the Shaky Leaf icon has become one of Ancestry's signature features, which signals to users that there's a helpful hint you can use to find out more about your family tree.

+ +

So when the company decided to begin moving its infrastructure to cloud native technology, the first service that was launched on Kubernetes, the open source platform for managing application containers across clusters of hosts, was this hint system. Think of it as Amazon's recommended products, but instead of recommending products the company recommends records, stories, or familial connections. "It was a very important part of the site," says Ancestry software engineer and architect Paul MacKay, "but also small enough for a pilot project that we knew we could handle in a very appropriate, secure way."

+ +

And when it went live smoothly in early 2016, "our deployment time for this service literally was cut down from 50 minutes to 2 or 5 minutes," MacKay adds. "The development team was just thrilled because we're focused on supplying a great experience for our customers. And that means features, it means stability, it means all those things that we need for a first-in-class type operation."

+ +

The stability of that Shaky Leaf was a signal for MacKay and his team that their decision to embrace cloud native technologies was the right one for the company. With a private data center, Ancestry built its website (which launched in 1996) on hundreds of services and technologies and a traditional deployment methodology. "It worked well for us in the past, but the sum of the legacy systems became quite cumbersome in its processing and was time-consuming," says MacKay. "We were looking for other ways to accelerate, to be more agile in delivering our solutions and our products."

+ +{{< case-studies/quote image="/images/case-studies/ancestry/banner3.jpg" >}} "And when it [Kubernetes] went live smoothly in early 2016, 'our deployment time for this service literally was cut down from 50 minutes to 2 or 5 minutes,' MacKay adds. 'The development team was just thrilled because we're focused on supplying a great experience for our customers. And that means features, it means stability, it means all those things that we need for a first-in-class type operation.'" -
-
+{{< /case-studies/quote >}} -
-
- That need led them in 2015 to explore containerization. Ancestry engineers had already been using technology like Java and Python on Linux, so part of the decision was about making the infrastructure more Linux-friendly. They quickly decided that they wanted to go with Docker for containerization, "but it always comes down to the orchestration part of it to make it really work," says MacKay.

- His team looked at orchestration platforms offered by Docker Compose, Mesos and OpenStack, and even started to prototype some homegrown solutions. And then they started hearing rumblings of the imminent release of Kubernetes v1.0. "At the forefront, we were looking at the secret store, so we didn't have to manage that all ourselves, the config maps, the methodology of seamless deployment strategy," he says. "We found that how Kubernetes had done their resources, their types, their labels and just their interface was so much further advanced than the other things we had seen. It was a feature fit."

-
- Plus, MacKay says, "I just believed in the confidence that comes with the history that Google has with containerization. So we started out right on the leading edge of it. And we haven't looked back since."

- Which is not to say that adopting a new technology hasn't come with some challenges. "Change is hard," says MacKay. "Not because the technology is hard or that the technology is not good. It's just that people like to do things like they had done [before]. You have the early adopters and you have those who are coming in later. It was a learning experience on both sides."

- Figuring out the best deployment operations for Ancestry was a big part of the work it took to adopt cloud native infrastructure. "We want to make sure the process is easy and also controlled in the manner that allows us the highest degree of security that we demand and our customers demand," says MacKay. "With Kubernetes and other products, there are some good solutions, but a little bit of glue is needed to bring it into corporate processes and governances. It's like having a set of gloves that are generic, but when you really do want to grab something you have to make it so it's customized to you. That's what we had to do."

- Their best practices include allowing their developers to deploy into development stage and production, but then controlling the aspects that need governance and auditing, such as secrets. They found that having one namespace per service is useful for achieving that containment of secrets and config maps. And for their needs, having one container per pod makes it easier to manage and to have a smaller unit of deployment. -

-
-
+

That need led them in 2015 to explore containerization. Ancestry engineers had already been using technology like Java and Python on Linux, so part of the decision was about making the infrastructure more Linux-friendly. They quickly decided that they wanted to go with Docker for containerization, "but it always comes down to the orchestration part of it to make it really work," says MacKay.

-
-
+

His team looked at orchestration platforms offered by Docker Compose, Mesos and OpenStack, and even started to prototype some homegrown solutions. And then they started hearing rumblings of the imminent release of Kubernetes v1.0. "At the forefront, we were looking at the secret store, so we didn't have to manage that all ourselves, the config maps, the methodology of seamless deployment strategy," he says. "We found that how Kubernetes had done their resources, their types, their labels and just their interface was so much further advanced than the other things we had seen. It was a feature fit."

+{{< case-studies/lead >}} +Plus, MacKay says, "I just believed in the confidence that comes with the history that Google has with containerization. So we started out right on the leading edge of it. And we haven't looked back since." +{{< /case-studies/lead >}} + +

Which is not to say that adopting a new technology hasn't come with some challenges. "Change is hard," says MacKay. "Not because the technology is hard or that the technology is not good. It's just that people like to do things like they had done [before]. You have the early adopters and you have those who are coming in later. It was a learning experience on both sides."

+ +

Figuring out the best deployment operations for Ancestry was a big part of the work it took to adopt cloud native infrastructure. "We want to make sure the process is easy and also controlled in the manner that allows us the highest degree of security that we demand and our customers demand," says MacKay. "With Kubernetes and other products, there are some good solutions, but a little bit of glue is needed to bring it into corporate processes and governances. It's like having a set of gloves that are generic, but when you really do want to grab something you have to make it so it's customized to you. That's what we had to do."

+ +

Their best practices include allowing their developers to deploy into development stage and production, but then controlling the aspects that need governance and auditing, such as secrets. They found that having one namespace per service is useful for achieving that containment of secrets and config maps. And for their needs, having one container per pod makes it easier to manage and to have a smaller unit of deployment. +

+ +{{< case-studies/quote image="/images/case-studies/ancestry/banner4.jpg" >}} "The success of Ancestry's first deployment of the hint system on Kubernetes helped create momentum for greater adoption of the technology." +{{< /case-studies/quote >}} -
-
+

With that process established, the time spent on deployment was cut down to under a minute for some services. "As programmers, we have what's called REPL: read, evaluate, print, and loop, but with Kubernetes, we have CDEL: compile, deploy, execute, and loop," says MacKay. "It's a very quick loop back and a great benefit to understand that when our services are deployed in production, they're the same as what we tested in the pre-production environments. The approach of cloud native for Ancestry provides us a better ability to scale and to accommodate the business needs as work loads occur."

-
-
- With that process established, the time spent on deployment was cut down to under a minute for some services. "As programmers, we have what's called REPL: read, evaluate, print, and loop, but with Kubernetes, we have CDEL: compile, deploy, execute, and loop," says MacKay. "It's a very quick loop back and a great benefit to understand that when our services are deployed in production, they're the same as what we tested in the pre-production environments. The approach of cloud native for Ancestry provides us a better ability to scale and to accommodate the business needs as work loads occur."

- The success of Ancestry's first deployment of the hint system on Kubernetes helped create momentum for greater adoption of the technology. "Engineers like to code, they like to do features, they don't like to sit around waiting for things to be deployed and worrying about scaling up and out and down," says MacKay. "After a while the engineers became our champions. At training sessions, the development teams were always the ones saying, 'Kubernetes saved our time tremendously; it's an enabler; it really is incredible.' Over time, we were able to convince our management that this was a transition that the industry is making and that we needed to be a part of it."

- A year later, Ancestry has transitioned a good number of applications to Kubernetes. "We have many different services that make up the rich environment that [the website] has from both the DNA side and the family history side," says MacKay. "We have front-end stacks, back-end stacks and back-end processing type stacks that are in the cluster."

- The company continues to weigh which services it will move forward to Kubernetes, which ones will be kept as is, and which will be replaced in the future and thus don't have to be moved over. MacKay estimates that the company is "approaching halfway on those features that are going forward. We don't have to do a lot of convincing anymore. It's more of an issue of timing with getting product management and engineering staff the knowledge and information that they need." -
-
+

The success of Ancestry's first deployment of the hint system on Kubernetes helped create momentum for greater adoption of the technology. "Engineers like to code, they like to do features, they don't like to sit around waiting for things to be deployed and worrying about scaling up and out and down," says MacKay. "After a while the engineers became our champions. At training sessions, the development teams were always the ones saying, 'Kubernetes saved our time tremendously; it's an enabler; it really is incredible.' Over time, we were able to convince our management that this was a transition that the industry is making and that we needed to be a part of it."

-
-
- "... 'I believe in Kubernetes. I believe in containerization. I think - if we can get there and establish ourselves in that world, we will be further along and far better off being agile and all the things we talk about, - and it'll go forward.'" -
-
+

A year later, Ancestry has transitioned a good number of applications to Kubernetes. "We have many different services that make up the rich environment that [the website] has from both the DNA side and the family history side," says MacKay. "We have front-end stacks, back-end stacks and back-end processing type stacks that are in the cluster."

-
-
+

The company continues to weigh which services it will move forward to Kubernetes, which ones will be kept as is, and which will be replaced in the future and thus don't have to be moved over. MacKay estimates that the company is "approaching halfway on those features that are going forward. We don't have to do a lot of convincing anymore. It's more of an issue of timing with getting product management and engineering staff the knowledge and information that they need."

+{{< case-studies/quote >}} +"... 'I believe in Kubernetes. I believe in containerization. I think if we can get there and establish ourselves in that world, we will be further along and far better off being agile and all the things we talk about, and it'll go forward.'" +{{< /case-studies/quote >}} -Looking ahead, MacKay sees Ancestry maximizing the benefits of Kubernetes in 2017. "We're very close to having everything that should be or could be in a Linux-friendly world in Kubernetes by the end of the year," he says, adding that he's looking forward to features such as federation and horizontal pod autoscaling that are currently in the works. "Kubernetes has been very wonderful for us and we continue to ride the wave."

-That wave, he points out, has everything to do with the vibrant Kubernetes community, which has grown by leaps and bounds since Ancestry joined it as an early adopter. "This is just a very rough way of judging it, but on Slack in June 2015, there were maybe 500 on there," MacKay says. "The last time I looked there were maybe 8,500 just on the Slack channel. There are so many major companies and different kinds of companies involved now. It's the variety of contributors, the number of contributors, the incredibly competent and friendly community."

-As much as he and his team at Ancestry have benefited from what he calls "the goodness and the technical abilities of many" in the community, they've also contributed information about best practices, logged bug issues and participated in the open source conversation. And they've been active in attending meetups to help educate and give back to the local tech community in Utah. Says MacKay: "We're trying to give back as far as our experience goes, rather than just code." -

When he meets with companies considering adopting cloud native infrastructure, the best advice he has to give from Ancestry's Kubernetes journey is this: "Start small, but with hard problems," he says. And "you need a patron who understands the vision of containerization, to help you tackle the political as well as other technical roadblocks that can occur when change is needed."

-With the changes that MacKay's team has led over the past year and a half, cloud native will be part of Ancestry's technological genealogy for years to come. MacKay has been such a champion of the technology that he says people have jokingly accused him of having a Kubernetes tattoo.

-"I really don't," he says with a laugh. "But I'm passionate. I'm not exclusive to any technology; I use whatever I need that's out there that makes us great. If it's something else, I'll use it. But right now I believe in Kubernetes. I believe in containerization. I think if we can get there and establish ourselves in that world, we will be further along and far better off being agile and all the things we talk about, and it'll go forward."

-He pauses. "So, yeah, I guess you can say I'm an evangelist for Kubernetes," he says. "But I'm not getting a tattoo!" +

Looking ahead, MacKay sees Ancestry maximizing the benefits of Kubernetes in 2017. "We're very close to having everything that should be or could be in a Linux-friendly world in Kubernetes by the end of the year," he says, adding that he's looking forward to features such as federation and horizontal pod autoscaling that are currently in the works. "Kubernetes has been very wonderful for us and we continue to ride the wave."

+

That wave, he points out, has everything to do with the vibrant Kubernetes community, which has grown by leaps and bounds since Ancestry joined it as an early adopter. "This is just a very rough way of judging it, but on Slack in June 2015, there were maybe 500 on there," MacKay says. "The last time I looked there were maybe 8,500 just on the Slack channel. There are so many major companies and different kinds of companies involved now. It's the variety of contributors, the number of contributors, the incredibly competent and friendly community."

-
-
+

As much as he and his team at Ancestry have benefited from what he calls "the goodness and the technical abilities of many" in the community, they've also contributed information about best practices, logged bug issues and participated in the open source conversation. And they've been active in attending meetups to help educate and give back to the local tech community in Utah. Says MacKay: "We're trying to give back as far as our experience goes, rather than just code."

+ +

When he meets with companies considering adopting cloud native infrastructure, the best advice he has to give from Ancestry's Kubernetes journey is this: "Start small, but with hard problems," he says. And "you need a patron who understands the vision of containerization, to help you tackle the political as well as other technical roadblocks that can occur when change is needed."

+ +

With the changes that MacKay's team has led over the past year and a half, cloud native will be part of Ancestry's technological genealogy for years to come. MacKay has been such a champion of the technology that he says people have jokingly accused him of having a Kubernetes tattoo.

+ +

"I really don't," he says with a laugh. "But I'm passionate. I'm not exclusive to any technology; I use whatever I need that's out there that makes us great. If it's something else, I'll use it. But right now I believe in Kubernetes. I believe in containerization. I think if we can get there and establish ourselves in that world, we will be further along and far better off being agile and all the things we talk about, and it'll go forward."

+ +

He pauses. "So, yeah, I guess you can say I'm an evangelist for Kubernetes," he says. "But I'm not getting a tattoo!"

diff --git a/content/en/case-studies/ant-financial/index.html b/content/en/case-studies/ant-financial/index.html index 1711ef97b8..542352902d 100644 --- a/content/en/case-studies/ant-financial/index.html +++ b/content/en/case-studies/ant-financial/index.html @@ -3,94 +3,80 @@ title: Ant Financial Case Study linkTitle: ant-financial case_study_styles: true cid: caseStudies -css: /css/style_case_studies.css featured: false + +new_case_study_styles: true +heading_background: /images/case-studies/antfinancial/banner1.jpg +heading_title_logo: /images/antfinancial_logo.png +subheading: > + Ant Financial's Hypergrowth Strategy Using Kubernetes +case_study_details: + - Company: Ant Financial + - Location: Hangzhou, China + - Industry: Financial Services --- -
-

CASE STUDY:
Ant Financial’s Hypergrowth Strategy Using Kubernetes +

Challenge

-

+

Officially founded in October 2014, Ant Financial originated from Alipay, the world's largest online payment platform that launched in 2004. The company also offers numerous other services leveraging technology innovation. With the volume of transactions Alipay handles for its 900+ million users worldwide (through its local and global partners)—256,000 transactions per second at the peak of Double 11 Singles Day 2017, and total gross merchandise value of $31 billion for Singles Day 2018—not to mention that of its other services, Ant Financial faces "data processing challenge in a whole new way," says Haojie Hang, who is responsible for Product Management for the Storage and Compute Group. "We see three major problems of operating at that scale: how to provide real-time compute, storage, and processing capability, for instance to make real-time recommendations for fraud detection; how to provide intelligence on top of this data, because there's too much data and then we're not getting enough insight; and how to apply security in the application level, in the middleware level, the system level, even the chip level." In order to provide reliable and consistent services to its customers, Ant Financial embraced containers in early 2014, and soon needed an orchestration solution for the tens-of-thousands-of-node clusters in its data centers.

-
+

Solution

-
- Company  Ant Financial     Location  Hangzhou, China     Industry  Financial Services -
+

After investigating several technologies, the team chose Kubernetes for orchestration, as well as a number of other CNCF projects, including Prometheus, OpenTracing, etcd and CoreDNS. "In late 2016, we decided that Kubernetes will be the de facto standard," says Hang. "Looking back, we made the right bet on the right technology. But then we needed to move the production workload from the legacy infrastructure to the latest Kubernetes-enabled platform, and that took some time, because we are very careful in terms of reliability and consistency." All core financial systems were containerized by November 2017, and the migration to Kubernetes is ongoing.

-
-
-
-
-

Challenge

- Officially founded in October 2014, Ant Financial originated from Alipay, the world’s largest online payment platform that launched in 2004. The company also offers numerous other services leveraging technology innovation. With the volume of transactions Alipay handles for its 900+ million users worldwide (through its local and global partners)—256,000 transactions per second at the peak of Double 11 Singles Day 2017, and total gross merchandise value of $31 billion for Singles Day 2018—not to mention that of its other services, Ant Financial faces “data processing challenge in a whole new way,” says Haojie Hang, who is responsible for Product Management for the Storage and Compute Group. “We see three major problems of operating at that scale: how to provide real-time compute, storage, and processing capability, for instance to make real-time recommendations for fraud detection; how to provide intelligence on top of this data, because there’s too much data and then we’re not getting enough insight; and how to apply security in the application level, in the middleware level, the system level, even the chip level.” In order to provide reliable and consistent services to its customers, Ant Financial embraced containers in early 2014, and soon needed an orchestration solution for the tens-of-thousands-of-node clusters in its data centers. - -

Solution

- After investigating several technologies, the team chose Kubernetes for orchestration, as well as a number of other CNCF projects, including Prometheus, OpenTracing, etcd and CoreDNS. “In late 2016, we decided that Kubernetes will be the de facto standard,” says Hang. “Looking back, we made the right bet on the right technology. But then we needed to move the production workload from the legacy infrastructure to the latest Kubernetes-enabled platform, and that took some time, because we are very careful in terms of reliability and consistency.” All core financial systems were containerized by November 2017, and the migration to Kubernetes is ongoing. -

Impact

- “We’ve seen at least tenfold in improvement in terms of the operations with cloud native technology, which means you can have tenfold increase in terms of output,” says Hang. Ant also provides its fully integrated financial cloud platform to business partners around the world, and hopes to power the next generation of digital banking with deep experience in service innovation and technology expertise. Hang says the team hasn’t begun to focus on optimizing the Kubernetes platform, either: “Because we’re still in the hyper growth stage, we’re not in a mode where we do cost saving yet.” -
-
-
-
-
- "In late 2016, we decided that Kubernetes will be the de facto standard. Looking back, we made the right bet on the right technology." -

- HAOJIE HANG, PRODUCT MANAGEMENT, ANT FINANCIAL -
-
-
-
-

A spinoff of the multinational conglomerate Alibaba, Ant Financial boasts a $150+ billion valuation and the scale to match. The fintech startup, launched in 2014, is comprised of Alipay, the world’s largest online payment platform, and numerous other services leveraging technology innovation.

- And the volume of transactions that Alipay handles for over 900 million users worldwide (through its local and global partners) is staggering: 256,000 per second at the peak of Double 11 Singles Day 2017, and total gross merchandise value of $31 billion for Singles Day 2018. With the mission of “bringing the world equal opportunities,” Ant Financial is dedicated to creating an open, shared credit system and financial services platform through technology innovations. -

- Combine that with the operations of its other properties—such as the Huabei online credit system, Jiebei lending service, and the 350-million-user Ant Forest green energy mobile app—and Ant Financial faces “data processing challenge in a whole new way,” says Haojie Hang, who is responsible for Product Management for the Storage and Compute Group. “We see three major problems of operating at that scale: how to provide real-time compute, storage, and processing capability, for instance to make real-time recommendations for fraud detection; how to provide intelligence on top of this data, because there’s too much data and we’re not getting enough insight; and how to apply security in the application level, in the middleware level, the system level, even the chip level.” -

- To address those challenges and provide reliable and consistent services to its customers, Ant Financial embraced Docker containerization in 2014. But they soon realized that they needed an orchestration solution for some tens-of-thousands-of-node clusters in the company’s data centers. -
-
-
-
- "On Double 11 this year, we had plenty of nodes on Kubernetes, but compared to the whole scale of our infrastructure, this is still in progress."

- RANGER YU, GLOBAL TECHNOLOGY PARTNERSHIP & DEVELOPMENT, ANT FINANCIAL +

"We've seen at least tenfold in improvement in terms of the operations with cloud native technology, which means you can have tenfold increase in terms of output," says Hang. Ant also provides its fully integrated financial cloud platform to business partners around the world, and hopes to power the next generation of digital banking with deep experience in service innovation and technology expertise. Hang says the team hasn't begun to focus on optimizing the Kubernetes platform, either: "Because we're still in the hyper growth stage, we're not in a mode where we do cost saving yet."

-
-
-
-
- The team investigated several technologies, including Docker Swarm and Mesos. “We did a lot of POCs, but we’re very careful in terms of production systems, because we want to make sure we don’t lose any data,” says Hang. “You cannot afford to have a service downtime for one minute; even one second has a very, very big impact. We operate every day under pressure to provide reliable and consistent services to consumers and businesses in China and globally.” -

- Ultimately, Hang says Ant chose Kubernetes because it checked all the boxes: a strong community, technology that “will be relevant in the next three to five years,” and a good match for the company’s engineering talent. “In late 2016, we decided that Kubernetes will be the de facto standard,” says Hang. “Looking back, we made the right bet on the right technology. But then we needed to move the production workload from the legacy infrastructure to the latest Kubernetes-enabled platform. We spent a lot of time learning and then training our people to build applications on Kubernetes well.” -

- All core financial systems were containerized by November 2017, and the migration to Kubernetes is ongoing. Ant’s platform also leverages a number of other CNCF projects, including Prometheus, OpenTracing, etcd and CoreDNS. “On Double 11 this year, we had plenty of nodes on Kubernetes, but compared to the whole scale of our infrastructure, this is still in progress,” says Ranger Yu, Global Technology Partnership & Development. -
-
-
-
- "We’re very grateful for CNCF and this amazing technology, which we need as we continue to scale globally. We’re definitely embracing the community and open source more in the future."

- HAOJIE HANG, PRODUCT MANAGEMENT, ANT FINANCIAL -
-
+{{< case-studies/quote author="HAOJIE HANG, PRODUCT MANAGEMENT, ANT FINANCIAL" >}} +"In late 2016, we decided that Kubernetes will be the de facto standard. Looking back, we made the right bet on the right technology." +{{< /case-studies/quote >}} -
-
- Still, there has already been an impact. “Cloud native technology has benefited us greatly in terms of efficiency,” says Hang. “In general, we want to make sure our infrastructure is nimble and flexible enough for the work that could happen tomorrow. That’s the goal. And with cloud native technology, we’ve seen at least tenfold improvement in operations, which means you can have tenfold increase in terms of output. Let’s say you are operating 10 nodes with one person. With cloud native, tomorrow you can have 100 nodes.” -

- Ant also provides its financial cloud platform to partners around the world, and hopes to power the next generation of digital banking with deep experience in service innovation and technology expertise. Hang says the team hasn’t begun to focus on optimizing the Kubernetes platform, either: “Because we’re still in the hyper growth stage, we’re not in a mode where we do cost-saving yet.” -

- The CNCF community has also been a valuable asset during Ant Financial’s move to cloud native. “If you are applying a new technology, it’s very good to have a community to discuss technical problems with other users,” says Hang. “We’re very grateful for CNCF and this amazing technology, which we need as we continue to scale globally. We’re definitely embracing the community and open sourcing more in the future.” -
+{{< case-studies/lead >}} +A spinoff of the multinational conglomerate Alibaba, Ant Financial boasts a $150+ billion valuation and the scale to match. The fintech startup, launched in 2014, is comprised of Alipay, the world's largest online payment platform, and numerous other services leveraging technology innovation. +{{< /case-studies/lead >}} -
-
-"In China, we are the North Star in terms of innovation in financial and other related services,” says Hang. “We definitely want to make sure we’re still leading in the next 5 to 10 years with our investment in technology."

- RANGER YU, GLOBAL TECHNOLOGY PARTNERSHIP & DEVELOPMENT, ANT FINANCIAL
-
+

And the volume of transactions that Alipay handles for over 900 million users worldwide (through its local and global partners) is staggering: 256,000 per second at the peak of Double 11 Singles Day 2017, and total gross merchandise value of $31 billion for Singles Day 2018. With the mission of "bringing the world equal opportunities," Ant Financial is dedicated to creating an open, shared credit system and financial services platform through technology innovations.

-
- In fact, the company has already started to open source some of its cloud native middleware. “We are going to be very proactive about that,” says Yu. “CNCF provided a platform so everyone can plug in or contribute components. This is very good open source governance.” -

- Looking ahead, the Ant team will continue to evaluate many other CNCF projects. Building a service mesh community in China, the team has brought together many China-based companies and developers to discuss the potential of that technology. “Service mesh is very attractive for Chinese developers and end users because we have a lot of legacy systems running now, and it’s an ideal mid-layer to glue everything together, both new and legacy,” says Hang. “For new technologies, we look very closely at whether they will last.” -

- At Ant, Kubernetes passed that test with flying colors, and the team hopes other companies will follow suit. “In China, we are the North Star in terms of innovation in financial and other related services,” says Hang. “We definitely want to make sure we’re still leading in the next 5 to 10 years with our investment in technology.” +

Combine that with the operations of its other properties—such as the Huabei online credit system, Jiebei lending service, and the 350-million-user Ant Forest green energy mobile app—and Ant Financial faces "data processing challenge in a whole new way," says Haojie Hang, who is responsible for Product Management for the Storage and Compute Group. "We see three major problems of operating at that scale: how to provide real-time compute, storage, and processing capability, for instance to make real-time recommendations for fraud detection; how to provide intelligence on top of this data, because there's too much data and we're not getting enough insight; and how to apply security in the application level, in the middleware level, the system level, even the chip level."

-
-
+

To address those challenges and provide reliable and consistent services to its customers, Ant Financial embraced Docker containerization in 2014. But they soon realized that they needed an orchestration solution for some tens-of-thousands-of-node clusters in the company's data centers.

+ +{{< case-studies/quote + image="/images/case-studies/antfinancial/banner3.jpg" + author="RANGER YU, GLOBAL TECHNOLOGY PARTNERSHIP & DEVELOPMENT, ANT FINANCIAL" +>}} +"On Double 11 this year, we had plenty of nodes on Kubernetes, but compared to the whole scale of our infrastructure, this is still in progress." +{{< /case-studies/quote >}} + +

The team investigated several technologies, including Docker Swarm and Mesos. "We did a lot of POCs, but we're very careful in terms of production systems, because we want to make sure we don't lose any data," says Hang. "You cannot afford to have a service downtime for one minute; even one second has a very, very big impact. We operate every day under pressure to provide reliable and consistent services to consumers and businesses in China and globally."

+ +

Ultimately, Hang says Ant chose Kubernetes because it checked all the boxes: a strong community, technology that "will be relevant in the next three to five years," and a good match for the company's engineering talent. "In late 2016, we decided that Kubernetes will be the de facto standard," says Hang. "Looking back, we made the right bet on the right technology. But then we needed to move the production workload from the legacy infrastructure to the latest Kubernetes-enabled platform. We spent a lot of time learning and then training our people to build applications on Kubernetes well."

+ +

All core financial systems were containerized by November 2017, and the migration to Kubernetes is ongoing. Ant's platform also leverages a number of other CNCF projects, including Prometheus, OpenTracing, etcd and CoreDNS. "On Double 11 this year, we had plenty of nodes on Kubernetes, but compared to the whole scale of our infrastructure, this is still in progress," says Ranger Yu, Global Technology Partnership & Development.

+ +{{< case-studies/quote + image="/images/case-studies/antfinancial/banner4.jpg" + author="HAOJIE HANG, PRODUCT MANAGEMENT, ANT FINANCIAL" +>}} +"We're very grateful for CNCF and this amazing technology, which we need as we continue to scale globally. We're definitely embracing the community and open source more in the future." +{{< /case-studies/quote >}} + +

Still, there has already been an impact. "Cloud native technology has benefited us greatly in terms of efficiency," says Hang. "In general, we want to make sure our infrastructure is nimble and flexible enough for the work that could happen tomorrow. That's the goal. And with cloud native technology, we've seen at least tenfold improvement in operations, which means you can have tenfold increase in terms of output. Let's say you are operating 10 nodes with one person. With cloud native, tomorrow you can have 100 nodes."

+ +

Ant also provides its financial cloud platform to partners around the world, and hopes to power the next generation of digital banking with deep experience in service innovation and technology expertise. Hang says the team hasn't begun to focus on optimizing the Kubernetes platform, either: "Because we're still in the hyper growth stage, we're not in a mode where we do cost-saving yet."

+ +

The CNCF community has also been a valuable asset during Ant Financial's move to cloud native. "If you are applying a new technology, it's very good to have a community to discuss technical problems with other users," says Hang. "We're very grateful for CNCF and this amazing technology, which we need as we continue to scale globally. We're definitely embracing the community and open sourcing more in the future."

+ +{{< case-studies/quote + image="/images/case-studies/antfinancial/banner4.jpg" + author="RANGER YU, GLOBAL TECHNOLOGY PARTNERSHIP & DEVELOPMENT, ANT FINANCIAL" +>}} +"In China, we are the North Star in terms of innovation in financial and other related services," says Hang. "We definitely want to make sure we're still leading in the next 5 to 10 years with our investment in technology." +{{< /case-studies/quote >}} + +

In fact, the company has already started to open source some of its cloud native middleware. "We are going to be very proactive about that," says Yu. "CNCF provided a platform so everyone can plug in or contribute components. This is very good open source governance."

+ +

Looking ahead, the Ant team will continue to evaluate many other CNCF projects. Building a service mesh community in China, the team has brought together many China-based companies and developers to discuss the potential of that technology. "Service mesh is very attractive for Chinese developers and end users because we have a lot of legacy systems running now, and it's an ideal mid-layer to glue everything together, both new and legacy," says Hang. "For new technologies, we look very closely at whether they will last."

+ +

At Ant, Kubernetes passed that test with flying colors, and the team hopes other companies will follow suit. "In China, we are the North Star in terms of innovation in financial and other related services," says Hang. "We definitely want to make sure we're still leading in the next 5 to 10 years with our investment in technology."

diff --git a/content/en/case-studies/appdirect/index.html b/content/en/case-studies/appdirect/index.html index ca6b0b8fe9..ffb06c67b9 100644 --- a/content/en/case-studies/appdirect/index.html +++ b/content/en/case-studies/appdirect/index.html @@ -1,99 +1,85 @@ --- title: AppDirect Case Study - linkTitle: AppDirect case_study_styles: true cid: caseStudies -css: /css/style_case_studies.css logo: appdirect_featured_logo.png featured: true weight: 4 quote: > We made the right decisions at the right time. Kubernetes and the cloud native technologies are now seen as the de facto ecosystem. + +new_case_study_styles: true +heading_background: /images/case-studies/appdirect/banner1.jpg +heading_title_logo: /images/appdirect_logo.png +subheading: > + AppDirect: How AppDirect Supported the 10x Growth of Its Engineering Staff with Kubernetess +case_study_details: + - Company: AppDirect + - Location: San Francisco, California + - Industry: Software --- -
-

CASE STUDY:
AppDirect: How AppDirect Supported the 10x Growth of Its Engineering Staff with Kubernetess -

+

Challenge

-
+

AppDirect provides an end-to-end commerce platform for cloud-based products and services. When Director of Software Development Pierre-Alexandre Lacerte began working there in 2014, the company had a monolith application deployed on a "tomcat infrastructure, and the whole release process was complex for what it should be," he says. "There were a lot of manual steps involved, with one engineer building a feature, then another team picking up the change. So you had bottlenecks in the pipeline to ship a feature to production." At the same time, the engineering team was growing, and the company realized it needed a better infrastructure to both support that growth and increase velocity.

-
- Company  AppDirect     Location  San Francisco, California -     Industry  Software -
+

Solution

+ +

"My idea was: Let's create an environment where teams can deploy their services faster, and they will say, 'Okay, I don't want to build in the monolith anymore. I want to build a service,'" says Lacerte. They considered and prototyped several different technologies before deciding to adopt Kubernetes in early 2016. Lacerte's team has also integrated Prometheus monitoring into the platform; tracing is next. Today, AppDirect has more than 50 microservices in production and 15 Kubernetes clusters deployed on AWS and on premise around the world.

-
-
-
-
-

Challenge

- AppDirect provides an end-to-end commerce platform for cloud-based products and services. When Director of Software Development Pierre-Alexandre Lacerte began working there in 2014, the company had a monolith application deployed on a "tomcat infrastructure, and the whole release process was complex for what it should be," he says. "There were a lot of manual steps involved, with one engineer building a feature, then another team picking up the change. So you had bottlenecks in the pipeline to ship a feature to production." At the same time, the engineering team was growing, and the company realized it needed a better infrastructure to both support that growth and increase velocity. -

-

Solution

- "My idea was: Let’s create an environment where teams can deploy their services faster, and they will say, ‘Okay, I don’t want to build in the monolith anymore. I want to build a service,’" says Lacerte. They considered and prototyped several different technologies before deciding to adopt Kubernetes in early 2016. Lacerte’s team has also integrated Prometheus monitoring into the platform; tracing is next. Today, AppDirect has more than 50 microservices in production and 15 Kubernetes clusters deployed on AWS and on premise around the world. -

Impact

- The Kubernetes platform has helped support the engineering team’s 10x growth over the past few years. Coupled with the fact that they were continually adding new features, Lacerte says, "I think our velocity would have slowed down a lot if we didn’t have this new infrastructure." Moving to Kubernetes and services has meant that deployments have become much faster due to less dependency on custom-made, brittle shell scripts with SCP commands. Time to deploy a new version has shrunk from 4 hours to a few minutes. Additionally, the company invested a lot of effort to make things self-service for developers. "Onboarding a new service doesn’t require Jira tickets or meeting with three different teams," says Lacerte. Today, the company sees 1,600 deployments per week, compared to 1-30 before. The company also achieved cost savings by moving its marketplace and billing monoliths to Kubernetes from legacy EC2 hosts as well as by leveraging autoscaling, as traffic is higher during business hours. -
-
-
-
-
- "It was an immense engineering culture shift, but the benefits are undeniable in terms of scale and speed." -

- Alexandre Gervais, Staff Software Developer, AppDirect -
-
-
-
-

With its end-to-end commerce platform for cloud-based products and services, AppDirect has been helping organizations such as Comcast and GoDaddy simplify the digital supply chain since 2009.

-
- When Director of Software Development Pierre-Alexandre Lacerte started working there in 2014, the company had a monolith application deployed on a "tomcat infrastructure, and the whole release process was complex for what it should be," he says. "There were a lot of manual steps involved, with one engineer building a feature then creating a pull request, and a QA or another engineer validating the feature. Then it gets merged and someone else will take care of the deployment. So we had bottlenecks in the pipeline to ship a feature to production."

- At the same time, the engineering team of 40 was growing, and the company wanted to add an increasing number of features to its products. As a member of the platform team, Lacerte began hearing from multiple teams that wanted to deploy applications using different frameworks and languages, from Node.js to Spring Boot Java. He soon realized that in order to both support growth and increase velocity, the company needed a better infrastructure, and a system in which teams are autonomous, can do their own deploys, and be responsible for their services in production. -
-
-
-
- "We made the right decisions at the right time. Kubernetes and the cloud native technologies are now seen as the de facto ecosystem. We know where to focus our efforts in order to tackle the new wave of challenges we face as we scale out. The community is so active and vibrant, which is a great complement to our awesome internal team."

- Alexandre Gervais, Staff Software Developer, AppDirect - +

The Kubernetes platform has helped support the engineering team's 10x growth over the past few years. Coupled with the fact that they were continually adding new features, Lacerte says, "I think our velocity would have slowed down a lot if we didn't have this new infrastructure." Moving to Kubernetes and services has meant that deployments have become much faster due to less dependency on custom-made, brittle shell scripts with SCP commands. Time to deploy a new version has shrunk from 4 hours to a few minutes. Additionally, the company invested a lot of effort to make things self-service for developers. "Onboarding a new service doesn't require Jira tickets or meeting with three different teams," says Lacerte. Today, the company sees 1,600 deployments per week, compared to 1-30 before. The company also achieved cost savings by moving its marketplace and billing monoliths to Kubernetes from legacy EC2 hosts as well as by leveraging autoscaling, as traffic is higher during business hours.

-
-
-
-
- From the beginning, Lacerte says, "My idea was: Let’s create an environment where teams can deploy their services faster, and they will say, ‘Okay, I don’t want to build in the monolith anymore. I want to build a service.’" (Lacerte left the company in 2019.)

- Working with the operations team, Lacerte’s group got more control and access to the company’s AWS infrastructure, and started prototyping several orchestration technologies. "Back then, Kubernetes was a little underground, unknown," he says. "But we looked at the community, the number of pull requests, the velocity on GitHub, and we saw it was getting traction. And we found that it was much easier for us to manage than the other technologies." - They spun up the first few services on Kubernetes using Chef and Terraform provisioning, and as more services were added, more automation was, too. "We have clusters around the world—in Korea, in Australia, in Germany, and in the U.S.," says Lacerte. "Automation is critical for us." They’re now largely using Kops, and are looking at managed Kubernetes offerings from several cloud providers.

- Today, though the monolith still exists, there are fewer and fewer commits and features. All teams are deploying on the new infrastructure, and services are the norm. AppDirect now has more than 50 microservices in production and 15 Kubernetes clusters deployed on AWS and on premise around the world.

- Lacerte’s strategy ultimately worked because of the very real impact the Kubernetes platform has had to deployment time. Due to less dependency on custom-made, brittle shell scripts with SCP commands, time to deploy a new version has shrunk from 4 hours to a few minutes. Additionally, the company invested a lot of effort to make things self-service for developers. "Onboarding a new service doesn’t require Jira tickets or meeting with three different teams," says Lacerte. Today, the company sees 1,600 deployments per week, compared to 1-30 before. -
-
-
-
- "I think our velocity would have slowed down a lot if we didn’t have this new infrastructure."

- Pierre-Alexandre Lacerte, Director of Software Development, AppDirect -
-
+{{< case-studies/quote author="Alexandre Gervais, Staff Software Developer, AppDirect" >}} +"It was an immense engineering culture shift, but the benefits are undeniable in terms of scale and speed." +{{< /case-studies/quote >}} -
+{{< case-studies/lead >}} +With its end-to-end commerce platform for cloud-based products and services, AppDirect has been helping organizations such as Comcast and GoDaddy simplify the digital supply chain since 2009. +{{< /case-studies/lead >}} -
- Additionally, the Kubernetes platform has helped support the engineering team’s 10x growth over the past few years. "Ownership, a core value of AppDirect, reflects in our ability to ship services independently of our monolith code base," says Staff Software Developer Alexandre Gervais, who worked with Lacerte on the initiative. "Small teams now own critical parts of our business domain model, and they operate in their decoupled domain of expertise, with limited knowledge of the entire codebase. This reduces and isolates some of the complexity." Coupled with the fact that they were continually adding new features, Lacerte says, "I think our velocity would have slowed down a lot if we didn’t have this new infrastructure." - The company also achieved cost savings by moving its marketplace and billing monoliths to Kubernetes from legacy EC2 hosts as well as by leveraging autoscaling, as traffic is higher during business hours.

- AppDirect’s cloud native stack also includes gRPC and Fluentd, and the team is currently working on setting up OpenCensus. The platform already has Prometheus integrated, so "when teams deploy their service, they have their notifications, alerts and configurations," says Lacerte. "For example, in the test environment, I want to get a message on Slack, and in production, I want a Slack message and I also want to get paged. We have integration with pager duty. Teams have more ownership on their services." +

When Director of Software Development Pierre-Alexandre Lacerte started working there in 2014, the company had a monolith application deployed on a "tomcat infrastructure, and the whole release process was complex for what it should be," he says. "There were a lot of manual steps involved, with one engineer building a feature then creating a pull request, and a QA or another engineer validating the feature. Then it gets merged and someone else will take care of the deployment. So we had bottlenecks in the pipeline to ship a feature to production."

-
+

At the same time, the engineering team of 40 was growing, and the company wanted to add an increasing number of features to its products. As a member of the platform team, Lacerte began hearing from multiple teams that wanted to deploy applications using different frameworks and languages, from Node.js to Spring Boot Java. He soon realized that in order to both support growth and increase velocity, the company needed a better infrastructure, and a system in which teams are autonomous, can do their own deploys, and be responsible for their services in production.

-
-
-"We moved from a culture limited to ‘pushing code in a branch’ to exciting new responsibilities outside of the code base: deployment of features and configurations; monitoring of application and business metrics; and on-call support in case of outages. It was an immense engineering culture shift, but the benefits are undeniable in terms of scale and speed."

- Pierre-Alexandre Lacerte, Director of Software Development, AppDirect
-
+{{< case-studies/quote + image="/images/case-studies/appdirect/banner3.jpg" + author="Alexandre Gervais, Staff Software Developer, AppDirect" +>}} +"We made the right decisions at the right time. Kubernetes and the cloud native technologies are now seen as the de facto ecosystem. We know where to focus our efforts in order to tackle the new wave of challenges we face as we scale out. The community is so active and vibrant, which is a great complement to our awesome internal team." +{{< /case-studies/quote >}} -
- That of course also means more responsibility. "We asked engineers to expand their horizons," says Gervais. "We moved from a culture limited to ‘pushing code in a branch’ to exciting new responsibilities outside of the code base: deployment of features and configurations; monitoring of application and business metrics; and on-call support in case of outages. It was an immense engineering culture shift, but the benefits are undeniable in terms of scale and speed."

- As the engineering ranks continue to grow, the platform team has a new challenge, of making sure that the Kubernetes platform is accessible and easily utilized by everyone. "How can we make sure that when we add more people to our team that they are efficient, productive, and know how to ramp up on the platform?" Lacerte says. So we have the evangelists, the documentation, some project examples. We do demos, we have AMA sessions. We’re trying different strategies to get everyone’s attention."

- Three and a half years into their Kubernetes journey, Gervais feels AppDirect "made the right decisions at the right time," he says. "Kubernetes and the cloud native technologies are now seen as the de facto ecosystem. We know where to focus our efforts in order to tackle the new wave of challenges we face as we scale out. The community is so active and vibrant, which is a great complement to our awesome internal team. Going forward, our focus will really be geared towards benefiting from the ecosystem by providing added business value in our day-to-day operations." +

From the beginning, Lacerte says, "My idea was: Let's create an environment where teams can deploy their services faster, and they will say, 'Okay, I don't want to build in the monolith anymore. I want to build a service.'" (Lacerte left the company in 2019.)

+

Working with the operations team, Lacerte's group got more control and access to the company's AWS infrastructure, and started prototyping several orchestration technologies. "Back then, Kubernetes was a little underground, unknown," he says. "But we looked at the community, the number of pull requests, the velocity on GitHub, and we saw it was getting traction. And we found that it was much easier for us to manage than the other technologies."

-
-
+

They spun up the first few services on Kubernetes using Chef and Terraform provisioning, and as more services were added, more automation was, too. "We have clusters around the world—in Korea, in Australia, in Germany, and in the U.S.," says Lacerte. "Automation is critical for us." They're now largely using Kops, and are looking at managed Kubernetes offerings from several cloud providers.

+ +

Today, though the monolith still exists, there are fewer and fewer commits and features. All teams are deploying on the new infrastructure, and services are the norm. AppDirect now has more than 50 microservices in production and 15 Kubernetes clusters deployed on AWS and on premise around the world.

+ +

Lacerte's strategy ultimately worked because of the very real impact the Kubernetes platform has had to deployment time. Due to less dependency on custom-made, brittle shell scripts with SCP commands, time to deploy a new version has shrunk from 4 hours to a few minutes. Additionally, the company invested a lot of effort to make things self-service for developers. "Onboarding a new service doesn't require Jira tickets or meeting with three different teams," says Lacerte. Today, the company sees 1,600 deployments per week, compared to 1-30 before.

+ +{{< case-studies/quote + image="/images/case-studies/appdirect/banner4.jpg" + author="Pierre-Alexandre Lacerte, Director of Software Development, AppDirect" +>}} +"I think our velocity would have slowed down a lot if we didn't have this new infrastructure." +{{< /case-studies/quote >}} + +

Additionally, the Kubernetes platform has helped support the engineering team's 10x growth over the past few years. "Ownership, a core value of AppDirect, reflects in our ability to ship services independently of our monolith code base," says Staff Software Developer Alexandre Gervais, who worked with Lacerte on the initiative. "Small teams now own critical parts of our business domain model, and they operate in their decoupled domain of expertise, with limited knowledge of the entire codebase. This reduces and isolates some of the complexity." Coupled with the fact that they were continually adding new features, Lacerte says, "I think our velocity would have slowed down a lot if we didn't have this new infrastructure."

+ +

The company also achieved cost savings by moving its marketplace and billing monoliths to Kubernetes from legacy EC2 hosts as well as by leveraging autoscaling, as traffic is higher during business hours.

+ +

AppDirect's cloud native stack also includes gRPC and Fluentd, and the team is currently working on setting up OpenCensus. The platform already has Prometheus integrated, so "when teams deploy their service, they have their notifications, alerts and configurations," says Lacerte. "For example, in the test environment, I want to get a message on Slack, and in production, I want a Slack message and I also want to get paged. We have integration with pager duty. Teams have more ownership on their services."

+ +{{< case-studies/quote author="Pierre-Alexandre Lacerte, Director of Software Development, AppDirect" >}} +"We moved from a culture limited to 'pushing code in a branch' to exciting new responsibilities outside of the code base: deployment of features and configurations; monitoring of application and business metrics; and on-call support in case of outages. It was an immense engineering culture shift, but the benefits are undeniable in terms of scale and speed." +{{< /case-studies/quote >}} + +

That of course also means more responsibility. "We asked engineers to expand their horizons," says Gervais. "We moved from a culture limited to 'pushing code in a branch' to exciting new responsibilities outside of the code base: deployment of features and configurations; monitoring of application and business metrics; and on-call support in case of outages. It was an immense engineering culture shift, but the benefits are undeniable in terms of scale and speed."

+ +

As the engineering ranks continue to grow, the platform team has a new challenge, of making sure that the Kubernetes platform is accessible and easily utilized by everyone. "How can we make sure that when we add more people to our team that they are efficient, productive, and know how to ramp up on the platform?" Lacerte says. So we have the evangelists, the documentation, some project examples. We do demos, we have AMA sessions. We're trying different strategies to get everyone's attention."

+ +

Three and a half years into their Kubernetes journey, Gervais feels AppDirect "made the right decisions at the right time," he says. "Kubernetes and the cloud native technologies are now seen as the de facto ecosystem. We know where to focus our efforts in order to tackle the new wave of challenges we face as we scale out. The community is so active and vibrant, which is a great complement to our awesome internal team. Going forward, our focus will really be geared towards benefiting from the ecosystem by providing added business value in our day-to-day operations."

diff --git a/content/en/case-studies/babylon/index.html b/content/en/case-studies/babylon/index.html index dce0612175..6b86096b02 100644 --- a/content/en/case-studies/babylon/index.html +++ b/content/en/case-studies/babylon/index.html @@ -3,107 +3,82 @@ title: Babylon Case Study linkTitle: Babylon case_study_styles: true cid: caseStudies -css: /css/case-studies-gradient.css logo: babylon_featured_logo.svg featured: true weight: 1 quote: > Kubernetes is a great platform for machine learning because it comes with all the scheduling and scalability that you need. + +new_case_study_styles: true +heading_background: /images/case-studies/babylon/banner4.jpg +heading_title_text: Babylon +use_gradient_overlay: true +subheading: > + AppDirect: How Cloud Native Is Enabling Babylon's Medical AI Innovations +case_study_details: + - Company: Babylon + - Location: United Kingdom + - Industry: AI, Healthcare --- +

Challenge

-
-

CASE STUDY: Babylon

-
How Cloud Native Is Enabling Babylon’s Medical AI Innovations
-
+

A large number of Babylon's products leverage machine learning and artificial intelligence, and in 2019, there wasn't enough computing power in-house to run a particular experiment. The company was also growing (from 100 to 1,600 in three years) and planning expansion into other countries.

-
- Company  Babylon     Location  United Kingdom     Industry  AI, Healthcare -
+

Solution

-
-
-
-
-

Challenge

- A large number of Babylon’s products leverage machine learning and artificial intelligence, and in 2019, there wasn’t enough computing power in-house to run a particular experiment. The company was also growing (from 100 to 1,600 in three years) and planning expansion into other countries. - - -

Solution

- Babylon had migrated its user-facing applications to a Kubernetes platform in 2018, so the infrastructure team turned to Kubeflow, a toolkit for machine learning on Kubernetes. “We tried to create a Kubernetes core server, we deployed Kubeflow, and we orchestrated the whole experiment, which ended up being a really good success,” says AI Infrastructure Lead Jérémie Vallée. The team began building a self-service AI training platform on top of Kubernetes. +

Babylon had migrated its user-facing applications to a Kubernetes platform in 2018, so the infrastructure team turned to Kubeflow, a toolkit for machine learning on Kubernetes. "We tried to create a Kubernetes core server, we deployed Kubeflow, and we orchestrated the whole experiment, which ended up being a really good success," says AI Infrastructure Lead Jérémie Vallée. The team began building a self-service AI training platform on top of Kubernetes.

Impact

- Instead of waiting hours or days to be able to compute, teams can get access instantaneously. Clinical validations used to take 10 hours; now they are done in under 20 minutes. The portability of the cloud native platform has also enabled Babylon to expand into other countries.
-
-
-
-
- “Kubernetes is a great platform for machine learning because it comes with all the scheduling and scalability that you need.” -

- JÉRÉMIE VALLÉE, AI INFRASTRUCTURE LEAD AT BABYLON

-
-
+

Instead of waiting hours or days to be able to compute, teams can get access instantaneously. Clinical validations used to take 10 hours; now they are done in under 20 minutes. The portability of the cloud native platform has also enabled Babylon to expand into other countries.

-
-
-

Babylon’s mission is to put accessible and affordable healthcare services in the hands of every person on earth.

+{{< case-studies/quote + image="/images/case-studies/babylon/banner1.jpg" + author="JÉRÉMIE VALLÉE, AI INFRASTRUCTURE LEAD AT BABYLON" +>}} +"Kubernetes is a great platform for machine learning because it comes with all the scheduling and scalability that you need." +{{< /case-studies/quote >}} -

Since its launch in the U.K. in 2013, the startup has facilitated millions of digital consultations around the world. In the U.K., patients were typically waiting a week or two for a doctor’s appointment. Through Babylon’s NHS service, GP at Hand—which has more than 75,000 registered patients—39% get an appointment through their phone within 30 minutes, and 89% within 6 hours.

+{{< case-studies/lead >}} +Babylon's mission is to put accessible and affordable healthcare services in the hands of every person on earth. +{{< /case-studies/lead >}} -

That’s just the start. “We try to combine different types of technology with the medical expertise that we have in-house to build products that will help patients manage and understand their health, and also help doctors be more efficient at what they do,” says Jérémie Vallée, AI Infrastructure Lead at Babylon.

+

Since its launch in the U.K. in 2013, the startup has facilitated millions of digital consultations around the world. In the U.K., patients were typically waiting a week or two for a doctor's appointment. Through Babylon's NHS service, GP at Hand—which has more than 75,000 registered patients—39% get an appointment through their phone within 30 minutes, and 89% within 6 hours.

-

A large number of these products leverage machine learning and artificial intelligence, and in 2019, researchers hit a pain point. “We have some servers in-house where our researchers were doing a lot of AI experiments and some training of models, and we came to a point where we didn’t have enough compute in-house to run a particular experiment,” says Vallée.

-

- Babylon had migrated its user-facing applications to a Kubernetes platform in 2018, “and we had a lot of Kubernetes knowledge thanks to the migration,” he adds. To optimize some of the models that had been created, the team turned to Kubeflow, a toolkit for machine learning on Kubernetes. “We tried to create a Kubernetes core server, we deployed Kubeflow, and we orchestrated the whole experiment, which ended up being a really good success,” he says.

-

- Based on that experience, Vallée’s team was tasked with building a self-service platform to help Babylon’s AI teams become more efficient, and by extension help get products to market faster. The main requirements: (1) the ability to give researchers and engineers access to the compute they needed, regardless of the size of the experiments they may need to run; (2) a way to provide teams with the best tools that they needed to do their work, on demand and in a centralized way; and (3) the training platform had to be close to the data that was being managed, because of the company’s expansion into different countries.

+

That's just the start. "We try to combine different types of technology with the medical expertise that we have in-house to build products that will help patients manage and understand their health, and also help doctors be more efficient at what they do," says Jérémie Vallée, AI Infrastructure Lead at Babylon.

+

A large number of these products leverage machine learning and artificial intelligence, and in 2019, researchers hit a pain point. "We have some servers in-house where our researchers were doing a lot of AI experiments and some training of models, and we came to a point where we didn't have enough compute in-house to run a particular experiment," says Vallée.

-
-
+

Babylon had migrated its user-facing applications to a Kubernetes platform in 2018, "and we had a lot of Kubernetes knowledge thanks to the migration," he adds. To optimize some of the models that had been created, the team turned to Kubeflow, a toolkit for machine learning on Kubernetes. "We tried to create a Kubernetes core server, we deployed Kubeflow, and we orchestrated the whole experiment, which ended up being a really good success," he says.

-
-
- “Delivering a self-service platform where users are empowered to run their own workload has enabled our data scientist community to do hyper parameter tuning and general algorithm development without any cloud skill and without the help of platform engineers, thus accelerating our innovation.”

- CAROLINE HARGROVE, CHIEF TECHNOLOGY OFFICER AT BABYLON

-
-
+

Based on that experience, Vallée's team was tasked with building a self-service platform to help Babylon's AI teams become more efficient, and by extension help get products to market faster. The main requirements: (1) the ability to give researchers and engineers access to the compute they needed, regardless of the size of the experiments they may need to run; (2) a way to provide teams with the best tools that they needed to do their work, on demand and in a centralized way; and (3) the training platform had to be close to the data that was being managed, because of the company's expansion into different countries.

-
-
-

- Kubernetes was an enabler on every count. “Kubernetes is a great platform for machine learning because it comes with all the scheduling and scalability that you need,” says Vallée. The need to keep data in every country in which Babylon operates requires a multi-region, multi-cloud strategy, and some countries might not even have a public cloud provider at all. “We wanted to make this platform portable so that we can run training jobs anywhere,” he says. “Kubernetes offered a base layer that allows you to deploy the platform outside of the cloud provider, and then deploy whatever tooling you need. That was a very good selling point for us.”

-

- Once the team decided to build the Babylon AI Research platform on top of Kubernetes, they referred to the Cloud Native Landscape to build out the stack: Prometheus and Grafana for monitoring; an Istio service mesh to control the network on the training platform and control what access all of the workflows would have; Helm to deploy the stack; and Flux to manage the GitOps part of the pipeline.

-

- The cloud native AI platform has had a huge impact at Babylon. The first research projects run on the platform mostly involved machine learning and natural language processing. These experiments required a huge amount of compute—1600 CPU, 3.2 TB RAM—which was much more than Babylon had in-house. Plus, access to compute used to take hours, or sometimes even days, depending on how busy the platform team was. “Now, with Kubernetes and the self-service platform that we provide, it’s pretty much instantaneous,” says Vallée.

-

- Another important type of work that’s done on the platform is clinical validation for new applications such as Babylon’s Symptom Checker, which calculates the probability of a disease given the evidence input by the user. “Being in healthcare, we want all of our models to be safe before they’re going to hit production,” says Vallée. Using Argo for GitOps “enabled us to scale the process massively.”

+{{< case-studies/quote author="CAROLINE HARGROVE, CHIEF TECHNOLOGY OFFICER AT BABYLON" >}} +"Delivering a self-service platform where users are empowered to run their own workload has enabled our data scientist community to do hyper parameter tuning and general algorithm development without any cloud skill and without the help of platform engineers, thus accelerating our innovation." +{{< /case-studies/quote >}} -

-
-
+

Kubernetes was an enabler on every count. "Kubernetes is a great platform for machine learning because it comes with all the scheduling and scalability that you need," says Vallée. The need to keep data in every country in which Babylon operates requires a multi-region, multi-cloud strategy, and some countries might not even have a public cloud provider at all. "We wanted to make this platform portable so that we can run training jobs anywhere," he says. "Kubernetes offered a base layer that allows you to deploy the platform outside of the cloud provider, and then deploy whatever tooling you need. That was a very good selling point for us."

+

Once the team decided to build the Babylon AI Research platform on top of Kubernetes, they referred to the Cloud Native Landscape to build out the stack: Prometheus and Grafana for monitoring; an Istio service mesh to control the network on the training platform and control what access all of the workflows would have; Helm to deploy the stack; and Flux to manage the GitOps part of the pipeline.

-
-
- “Giving a Kubernetes-based platform to our data scientists has meant increased security, increased innovation through empowerment, and a more affordable health service as our cloud engineers are building an experience that is used by hundreds on a daily basis, rather than supporting specific bespoke use cases.”

- JEAN MARIE FERDEGUE, DIRECTOR OF PLATFORM OPERATIONS AT BABYLON

-
-
+

The cloud native AI platform has had a huge impact at Babylon. The first research projects run on the platform mostly involved machine learning and natural language processing. These experiments required a huge amount of compute—1600 CPU, 3.2 TB RAM—which was much more than Babylon had in-house. Plus, access to compute used to take hours, or sometimes even days, depending on how busy the platform team was. "Now, with Kubernetes and the self-service platform that we provide, it's pretty much instantaneous," says Vallée.

-
-
-

- Researchers used to have to wait up to 10 hours to get results on new versions of their models. With Kubernetes, that time is now down to under 20 minutes. Plus, previously they could only run one clinical validation at a time, now they can run many parallel ones if they need to—a huge benefit considering that in the past three years, Babylon has grown from 100 to 1,600 employees.

-

- “Delivering a self-service platform where users are empowered to run their own workload has enabled our data scientist community to do hyper parameter tuning and general algorithm development without any cloud skill and without the help of platform engineers, thus accelerating our innovation,” says Chief Technology Officer Caroline Hargrove.

-

- Adds Director of Platform Operations Jean Marie Ferdegue: “Giving a Kubernetes-based platform to our data scientists has meant increased security, increased innovation through empowerment, and a more affordable health service as our cloud engineers are building an experience that is used by hundreds on a daily basis, rather than supporting specific bespoke use cases.”

-

- Plus, as Babylon continues to expand, “it will be very easy to onboard new countries,” says Vallée. “Fifteen months ago when we deployed this platform, we had one big environment in the U.K., but now we have one in Canada, we have one in Asia, and we have one coming in the U.S. This is one of the things that Kubernetes and the other cloud native projects have enabled for us.” -

-

- Babylon’s road map for cloud native involves onboarding all of the company’s AI efforts to the platform. Increasingly, that includes AI services of care. “I think this is going to be an interesting field where AI and healthcare meet,” Vallée says. “It’s kind of a complex problem and there’s a lot of issues around this. So with our platform, we want to say, ‘What can we do to make this less painful for our developers and machine learning engineers?’” -

-
-
- +

Another important type of work that's done on the platform is clinical validation for new applications such as Babylon's Symptom Checker, which calculates the probability of a disease given the evidence input by the user. "Being in healthcare, we want all of our models to be safe before they're going to hit production," says Vallée. Using Argo for GitOps "enabled us to scale the process massively."

+ +{{< case-studies/quote + image="/images/case-studies/babylon/banner2.jpg" + author="JEAN MARIE FERDEGUE, DIRECTOR OF PLATFORM OPERATIONS AT BABYLON" +>}} +"Giving a Kubernetes-based platform to our data scientists has meant increased security, increased innovation through empowerment, and a more affordable health service as our cloud engineers are building an experience that is used by hundreds on a daily basis, rather than supporting specific bespoke use cases." +{{< /case-studies/quote >}} + +

Researchers used to have to wait up to 10 hours to get results on new versions of their models. With Kubernetes, that time is now down to under 20 minutes. Plus, previously they could only run one clinical validation at a time, now they can run many parallel ones if they need to—a huge benefit considering that in the past three years, Babylon has grown from 100 to 1,600 employees.

+ +

"Delivering a self-service platform where users are empowered to run their own workload has enabled our data scientist community to do hyper parameter tuning and general algorithm development without any cloud skill and without the help of platform engineers, thus accelerating our innovation," says Chief Technology Officer Caroline Hargrove.

+ +

Adds Director of Platform Operations Jean Marie Ferdegue: "Giving a Kubernetes-based platform to our data scientists has meant increased security, increased innovation through empowerment, and a more affordable health service as our cloud engineers are building an experience that is used by hundreds on a daily basis, rather than supporting specific bespoke use cases."

+ +

Plus, as Babylon continues to expand, "it will be very easy to onboard new countries," says Vallée. "Fifteen months ago when we deployed this platform, we had one big environment in the U.K., but now we have one in Canada, we have one in Asia, and we have one coming in the U.S. This is one of the things that Kubernetes and the other cloud native projects have enabled for us."

+ +

Babylon's road map for cloud native involves onboarding all of the company's AI efforts to the platform. Increasingly, that includes AI services of care. "I think this is going to be an interesting field where AI and healthcare meet," Vallée says. "It's kind of a complex problem and there's a lot of issues around this. So with our platform, we want to say, 'What can we do to make this less painful for our developers and machine learning engineers?'"

diff --git a/content/en/case-studies/blablacar/index.html b/content/en/case-studies/blablacar/index.html index 2d55ffb8d0..e6537a672f 100644 --- a/content/en/case-studies/blablacar/index.html +++ b/content/en/case-studies/blablacar/index.html @@ -1,98 +1,85 @@ --- title: BlaBlaCar Case Study - case_study_styles: true cid: caseStudies -css: /css/style_blablacar.css + +new_case_study_styles: true +heading_background: /images/case-studies/blablacar/banner1.jpg +heading_title_logo: /images/blablacar_logo.png +subheading: > + Turning to Containerization to Support Millions of Rideshares +case_study_details: + - Company: BlaBlaCar + - Location: Paris, France + - Industry: Ridesharing Company --- -
-

CASE STUDY:
Turning to Containerization to Support Millions of Rideshares

+

Challenge

-
+

The world's largest long-distance carpooling community, BlaBlaCar, connects 40 million members across 22 countries. The company has been experiencing exponential growth since 2012 and needed its infrastructure to keep up. "When you're thinking about doubling the number of servers, you start thinking, 'What should I do to be more efficient?'" says Simon Lallemand, Infrastructure Engineer at BlaBlaCar. "The answer is not to hire more and more people just to deal with the servers and installation." The team knew they had to scale the platform, but wanted to stay on their own bare metal servers.

-
- Company  BlaBlaCar     Location  Paris, France     Industry  Ridesharing Company -
+

Solution

-
-
-
-
-

Challenge

- The world’s largest long-distance carpooling community, BlaBlaCar, connects 40 million members across 22 countries. The company has been experiencing exponential growth since 2012 and needed its infrastructure to keep up. "When you’re thinking about doubling the number of servers, you start thinking, ‘What should I do to be more efficient?’" says Simon Lallemand, Infrastructure Engineer at BlaBlaCar. "The answer is not to hire more and more people just to deal with the servers and installation." The team knew they had to scale the platform, but wanted to stay on their own bare metal servers. -
-
-

Solution

- Opting not to shift to cloud virtualization or use a private cloud on their own servers, the BlaBlaCar team became early adopters of containerization, using the CoreOs runtime rkt, initially deployed using fleet cluster manager. Last year, the company switched to Kubernetes orchestration, and now also uses Prometheus for monitoring. -
+

Opting not to shift to cloud virtualization or use a private cloud on their own servers, the BlaBlaCar team became early adopters of containerization, using the CoreOs runtime rkt, initially deployed using fleet cluster manager. Last year, the company switched to Kubernetes orchestration, and now also uses Prometheus for monitoring.

-

Impact

- "Before using containers, it would take sometimes a day, sometimes two, just to create a new service," says Lallemand. "With all the tooling that we made around the containers, copying a new service now is a matter of minutes. It’s really a huge gain. We are better at capacity planning in our data center because we have fewer constraints due to this abstraction between the services and the hardware we run on. For the developers, it also means they can focus only on the features that they’re developing, and not on the infrastructure." -
-
-
-
-
- "When you’re switching to this cloud-native model and running everything in containers, you have to make sure that at any moment you can reboot without any downtime and without losing traffic. [With Kubernetes] our infrastructure is much more resilient and we have better availability than before."

- Simon Lallemand, Infrastructure Engineer at BlaBlaCar -
-
-
-
-

For the 40 million users of BlaBlaCar, it’s easy to find strangers headed in the same direction to share rides and costs. You can even choose how much "bla bla" chatter you want from a long-distance ride mate.

- Behind the scenes, though, the infrastructure was falling woefully behind the rider community’s exponential growth. Founded in 2006, the company hit its current stride around 2012. "Our infrastructure was very traditional," says Infrastructure Engineer Simon Lallemand, who began working at the company in 2014. "In the beginning, it was a bit chaotic because we had to [grow] fast. But then comes the time when you have to design things to make it manageable."

- By 2015, the company had about 50 bare metal servers. The team was using a MySQL database and PHP, but, Lallemand says, "it was a very static way." They also utilized the configuration management system, Chef, but had little automation in its process. "When you’re thinking about doubling the number of servers, you start thinking, ‘What should I do to be more efficient?’" says Lallemand. "The answer is not to hire more and more people just to deal with the servers and installation."

- Instead, BlaBlaCar began its cloud-native journey but wasn’t sure which route to take. "We could either decide to go into cloud virtualization or even use a private cloud on our own servers," says Lallemand. "But going into the cloud meant we had to make a lot of changes in our application work, and we were just not ready to make the switch from on premise to the cloud." They wanted to keep the great performance they got on bare metal, so they didn’t want to go to virtualization on premise.

- The solution: containerization. This was early 2015 and containers were still relatively new. "It was a bold move at the time," says Lallemand. "We decided that the next servers that we would buy in the new data center would all be the same model, so we could outsource the maintenance of the servers. And we decided to go with containers and with CoreOS Container Linux as an abstraction for this hardware. It seemed future-proof to go with containers because we could see what companies were already doing with containers." -
-
+

"Before using containers, it would take sometimes a day, sometimes two, just to create a new service," says Lallemand. "With all the tooling that we made around the containers, copying a new service now is a matter of minutes. It's really a huge gain. We are better at capacity planning in our data center because we have fewer constraints due to this abstraction between the services and the hardware we run on. For the developers, it also means they can focus only on the features that they're developing, and not on the infrastructure."

-
-
- "With all the tooling that we made around the containers, copying a new service is a matter of minutes. It’s a huge gain. For the developers, it means they can focus only on the features that they’re developing and not on the infrastructure or the hour they would test their code, or the hour that it would get deployed." -
-
+{{< case-studies/quote author="Simon Lallemand, Infrastructure Engineer at BlaBlaCar" >}} +"When you're switching to this cloud-native model and running everything in containers, you have to make sure that at any moment you can reboot without any downtime and without losing traffic. [With Kubernetes] our infrastructure is much more resilient and we have better availability than before." +{{< /case-studies/quote >}} -
-
- Next, they needed to choose a runtime for the containers, but "there were very few deployments in production at that time," says Lallemand. They experimented with Docker but decided to go with rkt. Lallemand explains that for BlaBlaCar, it was "much simpler to integrate things that are on rkt." At the time, the project was still pre-v1.0, so "we could speak with the developers of rkt and give them feedback. It was an advantage." Plus, he notes, rkt was very stable, even at this early stage.

- Once those decisions were made that summer, the company came up with a plan for implementation. First, they formed a task force to create a workflow that would be tested by three of the 10 members on Lallemand’s team. But they took care to run regular workshops with all 10 members to make sure everyone was on board. "When you’re focused on your product sometimes you forget if it’s really user friendly, whether other people can manage to create containers too," Lallemand says. "So we did a lot of iterations to find a good workflow."

- After establishing the workflow, Lallemand says with a smile that "we had this strange idea that we should try the most difficult thing first. Because if it works, it will work for everything." So the first project the team decided to containerize was the database. "Nobody did that at the time, and there were really no existing tools for what we wanted to do, including building container images," he says. So the team created their own tools, such as dgr, which builds container images so that the whole team has a common framework to build on the same images with the same standards. They also revamped the service-discovery tools Nerve and Synapse; their versions, Go-Nerve and Go-Synapse, were written in Go and built to be more efficient and include new features. All of these tools were open-sourced.

- At the same time, the company was working to migrate its entire platform to containers with a deadline set for Christmas 2015. With all the work being done in parallel, BlaBlaCar was able to get about 80 percent of its production into containers by its deadline with live traffic running on containers during December. (It’s now at 100 percent.) "It’s a really busy time for traffic," says Lallemand. "We knew that by using those new servers with containers, it would help us handle the traffic."

- In the middle of that peak season for carpooling, everything worked well. "The biggest impact that we had was for the deployment of new services," says Lallemand. "Before using containers, we had to first deploy a new server and create configurations with Chef. It would take sometimes a day, sometimes two, just to create a new service. And with all the tooling that we made around the containers, copying a new service is a matter of minutes. So it’s really a huge gain. For the developers, it means they can focus only on the features that they’re developing and not on the infrastructure or the hour they would test their code, or the hour that it would get deployed." -
-
+{{< case-studies/lead >}} +For the 40 million users of BlaBlaCar, it's easy to find strangers headed in the same direction to share rides and costs. You can even choose how much "bla bla" chatter you want from a long-distance ride mate. +{{< /case-studies/lead >}} -
-
- "We realized that there was a really strong community around it [Kubernetes], which meant we would not have to maintain a lot of tools of our own," says Lallemand. "It was better if we could contribute to some bigger project like Kubernetes." -
-
+

Behind the scenes, though, the infrastructure was falling woefully behind the rider community's exponential growth. Founded in 2006, the company hit its current stride around 2012. "Our infrastructure was very traditional," says Infrastructure Engineer Simon Lallemand, who began working at the company in 2014. "In the beginning, it was a bit chaotic because we had to [grow] fast. But then comes the time when you have to design things to make it manageable."

-
-
- In order to meet their self-imposed deadline, one of the decisions they made was to not do any "orchestration magic" for containers in the first production alignment. Instead, they used the basic fleet tool from CoreOS to deploy their containers. (They did build a tool called GGN, which they’ve open-sourced, to make it more manageable for their system engineers to use.)

- Still, the team knew that they’d want more orchestration. "Our tool was doing a pretty good job, but at some point you want to give more autonomy to the developer team," Lallemand says. "We also realized that we don’t want to be the single point of contact for developers when they want to launch new services." By the summer of 2016, they found their answer in Kubernetes, which had just begun supporting rkt implementation.

- After discussing their needs with their contacts at CoreOS and Google, they were convinced that Kubernetes would work for BlaBlaCar. "We realized that there was a really strong community around it, which meant we would not have to maintain a lot of tools of our own," says Lallemand. "It was better if we could contribute to some bigger project like Kubernetes." They also started using Prometheus, as they were looking for "service-oriented monitoring that could be updated nightly." Production on Kubernetes began in December 2016. "We like to do crazy stuff around Christmas," he adds with a laugh.

- BlaBlaCar now has about 3,000 pods, with 1200 of them running on Kubernetes. Lallemand leads a "foundations team" of 25 members who take care of the networks, databases and systems for about 100 developers. There have been some challenges getting to this point. "The rkt implementation is still not 100 percent finished," Lallemand points out. "It’s really good, but there are some features still missing. We have questions about how we do things with stateful services, like databases. We know how we will be migrating some of the services; some of the others are a bit more complicated to deal with. But the Kubernetes community is making a lot of progress on that part."

- The team is particularly happy that they’re now able to plan capacity better in the company’s data center. "We have fewer constraints since we have this abstraction between the services and the hardware we run on," says Lallemand. "If we lose a server because there’s a hardware problem on it, we just move the containers onto another server. It’s much more efficient. We do that by just changing a line in the configuration file. And with Kubernetes, it should be automatic, so we would have nothing to do." -
-
+

By 2015, the company had about 50 bare metal servers. The team was using a MySQL database and PHP, but, Lallemand says, "it was a very static way." They also utilized the configuration management system, Chef, but had little automation in its process. "When you're thinking about doubling the number of servers, you start thinking, 'What should I do to be more efficient?'" says Lallemand. "The answer is not to hire more and more people just to deal with the servers and installation."

-
-
- "If we lose a server because there’s a hardware problem on it, we just move the containers onto another server. It’s much more efficient. We do that by just changing a line in the configuration file. With Kubernetes, it should be automatic, so we would have nothing to do." -
-
+

Instead, BlaBlaCar began its cloud-native journey but wasn't sure which route to take. "We could either decide to go into cloud virtualization or even use a private cloud on our own servers," says Lallemand. "But going into the cloud meant we had to make a lot of changes in our application work, and we were just not ready to make the switch from on premise to the cloud." They wanted to keep the great performance they got on bare metal, so they didn't want to go to virtualization on premise.

-
-
- And these advances ultimately trickle down to BlaBlaCar’s users. "We have improved availability overall on our website," says Lallemand. "When you’re switching to this cloud-native model with running everything in containers, you have to make sure that you can at any moment reboot a server or a data container without any downtime, without losing traffic. So now our infrastructure is much more resilient and we have better availability than before."

- Within BlaBlaCar’s technology department, the cloud-native journey has created some profound changes. Lallemand thinks that the regular meetings during the conception stage and the training sessions during implementation helped. "After that everybody took part in the migration process," he says. "Then we split the organization into different ‘tribes’—teams that gather developers, product managers, data analysts, all the different jobs, to work on a specific part of the product. Before, they were organized by function. The idea is to give all these tribes access to the infrastructure directly in a self-service way without having to ask. These people are really autonomous. They have responsibility of that part of the product, and they can make decisions faster."

- This DevOps transformation turned out to be a positive one for the company’s staffers. "The team was very excited about the DevOps transformation because it was new, and we were working to make things more reliable, more future-proof," says Lallemand. "We like doing things that very few people are doing, other than the internet giants."

- With these changes already making an impact, BlaBlaCar is looking to split up more and more of its application into services. "I don’t say microservices because they’re not so micro," Lallemand says. "If we can split the responsibilities between the development teams, it would be easier to manage and more reliable, because we can easily add and remove services if one fails. You can handle it easily, instead of adding a big monolith that we still have."

- When Lallemand speaks to other European companies curious about what BlaBlaCar has done with its infrastructure, he tells them to come along for the ride. "I tell them that it’s such a pleasure to deal with the infrastructure that we have today compared to what we had before," he says. "They just need to keep in mind their real motive, whether it’s flexibility in development or reliability or so on, and then go step by step towards reaching those objectives. That’s what we’ve done. It’s important not to do technology for the sake of technology. Do it for a purpose. Our focus was on helping the developers." -
-
+

The solution: containerization. This was early 2015 and containers were still relatively new. "It was a bold move at the time," says Lallemand. "We decided that the next servers that we would buy in the new data center would all be the same model, so we could outsource the maintenance of the servers. And we decided to go with containers and with CoreOS Container Linux as an abstraction for this hardware. It seemed future-proof to go with containers because we could see what companies were already doing with containers."

+ +{{< case-studies/quote image="/images/case-studies/blablacar/banner3.jpg">}} +"With all the tooling that we made around the containers, copying a new service is a matter of minutes. It's a huge gain. For the developers, it means they can focus only on the features that they're developing and not on the infrastructure or the hour they would test their code, or the hour that it would get deployed." +{{< /case-studies/quote >}} + +

Next, they needed to choose a runtime for the containers, but "there were very few deployments in production at that time," says Lallemand. They experimented with Docker but decided to go with rkt. Lallemand explains that for BlaBlaCar, it was "much simpler to integrate things that are on rkt." At the time, the project was still pre-v1.0, so "we could speak with the developers of rkt and give them feedback. It was an advantage." Plus, he notes, rkt was very stable, even at this early stage.

+ +

Once those decisions were made that summer, the company came up with a plan for implementation. First, they formed a task force to create a workflow that would be tested by three of the 10 members on Lallemand's team. But they took care to run regular workshops with all 10 members to make sure everyone was on board. "When you're focused on your product sometimes you forget if it's really user friendly, whether other people can manage to create containers too," Lallemand says. "So we did a lot of iterations to find a good workflow."

+ +

After establishing the workflow, Lallemand says with a smile that "we had this strange idea that we should try the most difficult thing first. Because if it works, it will work for everything." So the first project the team decided to containerize was the database. "Nobody did that at the time, and there were really no existing tools for what we wanted to do, including building container images," he says. So the team created their own tools, such as dgr, which builds container images so that the whole team has a common framework to build on the same images with the same standards. They also revamped the service-discovery tools Nerve and Synapse; their versions, Go-Nerve and Go-Synapse, were written in Go and built to be more efficient and include new features. All of these tools were open-sourced.

+ +

At the same time, the company was working to migrate its entire platform to containers with a deadline set for Christmas 2015. With all the work being done in parallel, BlaBlaCar was able to get about 80 percent of its production into containers by its deadline with live traffic running on containers during December. (It's now at 100 percent.) "It's a really busy time for traffic," says Lallemand. "We knew that by using those new servers with containers, it would help us handle the traffic."

+ +

In the middle of that peak season for carpooling, everything worked well. "The biggest impact that we had was for the deployment of new services," says Lallemand. "Before using containers, we had to first deploy a new server and create configurations with Chef. It would take sometimes a day, sometimes two, just to create a new service. And with all the tooling that we made around the containers, copying a new service is a matter of minutes. So it's really a huge gain. For the developers, it means they can focus only on the features that they're developing and not on the infrastructure or the hour they would test their code, or the hour that it would get deployed."

+ +{{< case-studies/quote image="/images/case-studies/blablacar/banner4.jpg" >}} +"We realized that there was a really strong community around it [Kubernetes], which meant we would not have to maintain a lot of tools of our own," says Lallemand. "It was better if we could contribute to some bigger project like Kubernetes." +{{< /case-studies/quote >}} + +

In order to meet their self-imposed deadline, one of the decisions they made was to not do any "orchestration magic" for containers in the first production alignment. Instead, they used the basic fleet tool from CoreOS to deploy their containers. (They did build a tool called GGN, which they've open-sourced, to make it more manageable for their system engineers to use.)

+ +

Still, the team knew that they'd want more orchestration. "Our tool was doing a pretty good job, but at some point you want to give more autonomy to the developer team," Lallemand says. "We also realized that we don't want to be the single point of contact for developers when they want to launch new services." By the summer of 2016, they found their answer in Kubernetes, which had just begun supporting rkt implementation.

+ +

After discussing their needs with their contacts at CoreOS and Google, they were convinced that Kubernetes would work for BlaBlaCar. "We realized that there was a really strong community around it, which meant we would not have to maintain a lot of tools of our own," says Lallemand. "It was better if we could contribute to some bigger project like Kubernetes." They also started using Prometheus, as they were looking for "service-oriented monitoring that could be updated nightly." Production on Kubernetes began in December 2016. "We like to do crazy stuff around Christmas," he adds with a laugh.

+ +

BlaBlaCar now has about 3,000 pods, with 1200 of them running on Kubernetes. Lallemand leads a "foundations team" of 25 members who take care of the networks, databases and systems for about 100 developers. There have been some challenges getting to this point. "The rkt implementation is still not 100 percent finished," Lallemand points out. "It's really good, but there are some features still missing. We have questions about how we do things with stateful services, like databases. We know how we will be migrating some of the services; some of the others are a bit more complicated to deal with. But the Kubernetes community is making a lot of progress on that part."

+ +

The team is particularly happy that they're now able to plan capacity better in the company's data center. "We have fewer constraints since we have this abstraction between the services and the hardware we run on," says Lallemand. "If we lose a server because there's a hardware problem on it, we just move the containers onto another server. It's much more efficient. We do that by just changing a line in the configuration file. And with Kubernetes, it should be automatic, so we would have nothing to do."

+ +{{< case-studies/quote >}} +"If we lose a server because there's a hardware problem on it, we just move the containers onto another server. It's much more efficient. We do that by just changing a line in the configuration file. With Kubernetes, it should be automatic, so we would have nothing to do." +{{< /case-studies/quote >}} + +

And these advances ultimately trickle down to BlaBlaCar's users. "We have improved availability overall on our website," says Lallemand. "When you're switching to this cloud-native model with running everything in containers, you have to make sure that you can at any moment reboot a server or a data container without any downtime, without losing traffic. So now our infrastructure is much more resilient and we have better availability than before."

+ +

Within BlaBlaCar's technology department, the cloud-native journey has created some profound changes. Lallemand thinks that the regular meetings during the conception stage and the training sessions during implementation helped. "After that everybody took part in the migration process," he says. "Then we split the organization into different 'tribes'—teams that gather developers, product managers, data analysts, all the different jobs, to work on a specific part of the product. Before, they were organized by function. The idea is to give all these tribes access to the infrastructure directly in a self-service way without having to ask. These people are really autonomous. They have responsibility of that part of the product, and they can make decisions faster."

+ +

This DevOps transformation turned out to be a positive one for the company's staffers. "The team was very excited about the DevOps transformation because it was new, and we were working to make things more reliable, more future-proof," says Lallemand. "We like doing things that very few people are doing, other than the internet giants."

+ +

With these changes already making an impact, BlaBlaCar is looking to split up more and more of its application into services. "I don't say microservices because they're not so micro," Lallemand says. "If we can split the responsibilities between the development teams, it would be easier to manage and more reliable, because we can easily add and remove services if one fails. You can handle it easily, instead of adding a big monolith that we still have."

+ +

When Lallemand speaks to other European companies curious about what BlaBlaCar has done with its infrastructure, he tells them to come along for the ride. "I tell them that it's such a pleasure to deal with the infrastructure that we have today compared to what we had before," he says. "They just need to keep in mind their real motive, whether it's flexibility in development or reliability or so on, and then go step by step towards reaching those objectives. That's what we've done. It's important not to do technology for the sake of technology. Do it for a purpose. Our focus was on helping the developers."

diff --git a/content/en/case-studies/blackrock/index.html b/content/en/case-studies/blackrock/index.html index 6bef0ec708..725d6bc057 100644 --- a/content/en/case-studies/blackrock/index.html +++ b/content/en/case-studies/blackrock/index.html @@ -1,112 +1,83 @@ --- title: BlackRock Case Study - case_study_styles: true cid: caseStudies -css: /css/style_blackrock.css + +new_case_study_styles: true +heading_background: /images/case-studies/blackrock/banner1.jpg +heading_title_logo: /images/blackrock_logo.png +subheading: > + Rolling Out Kubernetes in Production in 100 Days +case_study_details: + - Company: BlackRock + - Location: New York, NY + - Industry: Financial Services --- -
-

CASE STUDY:
-
Rolling Out Kubernetes in Production in 100 Days
-

+

Challenge

-
+

The world's largest asset manager, BlackRock operates a very controlled static deployment scheme, which has allowed for scalability over the years. But in their data science division, there was a need for more dynamic access to resources. "We want to be able to give every investor access to data science, meaning Python notebooks, or even something much more advanced, like a MapReduce engine based on Spark," says Michael Francis, a Managing Director in BlackRock's Product Group, which runs the company's investment management platform. "Managing complex Python installations on users' desktops is really hard because everyone ends up with slightly different environments. We have existing environments that do these things, but we needed to make it real, expansive and scalable. Being able to spin that up on demand, tear it down, make that much more dynamic, became a critical thought process for us. It's not so much that we had to solve our main core production problem, it's how do we extend that? How do we evolve?"

-
- Company  BlackRock     Location  New York, NY     Industry  Financial Services -
+

Solution

-
+

Drawing from what they learned during a pilot done last year using Docker environments, Francis put together a cross-sectional team of 20 to build an investor research web app using Kubernetes with the goal of getting it into production within one quarter.

-
- -
-
-

Challenge

- The world’s largest asset manager, BlackRock operates a very controlled static deployment scheme, which has allowed for scalability over the years. But in their data science division, there was a need for more dynamic access to resources. "We want to be able to give every investor access to data science, meaning Python notebooks, or even something much more advanced, like a MapReduce engine based on Spark," says Michael Francis, a Managing Director in BlackRock’s Product Group, which runs the company’s investment management platform. "Managing complex Python installations on users’ desktops is really hard because everyone ends up with slightly different environments. We have existing environments that do these things, but we needed to make it real, expansive and scalable. Being able to spin that up on demand, tear it down, make that much more dynamic, became a critical thought process for us. It’s not so much that we had to solve our main core production problem, it’s how do we extend that? How do we evolve?" -
- -
-

Solution

- Drawing from what they learned during a pilot done last year using Docker environments, Francis put together a cross-sectional team of 20 to build an investor research web app using Kubernetes with the goal of getting it into production within one quarter. -

Impact

- "Our goal was: How do you give people tools rapidly without having to install them on their desktop?" says Francis. And the team hit the goal within 100 days. Francis is pleased with the results and says, "We’re going to use this infrastructure for lots of other application workloads as time goes on. It’s not just data science; it’s this style of application that needs the dynamism. But I think we’re 6-12 months away from making a [large scale] decision. We need to gain experience of running the system in production, we need to understand failure modes and how best to manage operational issues. What’s interesting is that just having this technology there is changing the way our developers are starting to think about their future development." -
-
+

"Our goal was: How do you give people tools rapidly without having to install them on their desktop?" says Francis. And the team hit the goal within 100 days. Francis is pleased with the results and says, "We're going to use this infrastructure for lots of other application workloads as time goes on. It's not just data science; it's this style of application that needs the dynamism. But I think we're 6-12 months away from making a [large scale] decision. We need to gain experience of running the system in production, we need to understand failure modes and how best to manage operational issues. What's interesting is that just having this technology there is changing the way our developers are starting to think about their future development."

-
+{{< case-studies/quote author="Michael Francis, Managing Director, BlackRock">}} +"My message to other enterprises like us is you can actually integrate Kubernetes into an existing, well-orchestrated machinery. You don't have to throw out everything you do. And using Kubernetes made a complex problem significantly easier." +{{< /case-studies/quote >}} -
-
- "My message to other enterprises like us is you can actually integrate Kubernetes into an existing, well-orchestrated machinery. You don’t have to throw out everything you do. And using Kubernetes made a complex problem significantly easier."

- Michael Francis, Managing Director, BlackRock -
-
+

One of the management objectives for BlackRock's Product Group employees in 2017 was to "build cool stuff." Led by Managing Director Michael Francis, a cross-sectional group of 20 did just that: They rolled out a full production Kubernetes environment and released a new investor research web app on it. In 100 days.

-
+

For a company that's the world's largest asset manager, "just equipment procurement can take 100 days sometimes, let alone from inception to delivery," says Karl Wieman, a Senior System Administrator. "It was an aggressive schedule. But it moved the dial." In fact, the project achieved two goals: It solved a business problem (creating the needed web app) as well as provided real-world, in-production experience with Kubernetes, a cloud-native technology that the company was eager to explore. "It's not so much that we had to solve our main core production problem, it's how do we extend that? How do we evolve?" says Francis. The ultimate success of this project, beyond delivering the app, lies in the fact that "we've managed to integrate a radically new thought process into a controlled infrastructure that we didn't want to change."

-
- One of the management objectives for BlackRock’s Product Group employees in 2017 was to "build cool stuff." Led by Managing Director Michael Francis, a cross-sectional group of 20 did just that: They rolled out a full production Kubernetes environment and released a new investor research web app on it. In 100 days.

- For a company that’s the world’s largest asset manager, "just equipment procurement can take 100 days sometimes, let alone from inception to delivery," says Karl Wieman, a Senior System Administrator. "It was an aggressive schedule. But it moved the dial." - In fact, the project achieved two goals: It solved a business problem (creating the needed web app) as well as provided real-world, in-production experience with Kubernetes, a cloud-native technology that the company was eager to explore. "It’s not so much that we had to solve our main core production problem, it’s how do we extend that? How do we evolve?" says Francis. The ultimate success of this project, beyond delivering the app, lies in the fact that "we’ve managed to integrate a radically new thought process into a controlled infrastructure that we didn’t want to change."

- After all, in its three decades of existence, BlackRock has "a very well-established environment for managing our compute resources," says Francis. "We manage large cluster processes on machines, so we do a lot of orchestration and management for our main production processes in a way that’s very cloudish in concept. We’re able to manage them in a very controlled, static deployment scheme, and that has given us a huge amount of scalability."

- Though that works well for the core production, the company has found that some data science workloads require more dynamic access to resources. "It’s a very bursty process," says Francis, who is head of data for the company’s Aladdin investment management platform division.

- Aladdin, which connects the people, information and technology needed for money management in real time, is used internally and is also sold as a platform to other asset managers and insurance companies. "We want to be able to give every investor access to data science, meaning Python notebooks, or even something much more advanced, like a MapReduce engine based on Spark," says Francis. But "managing complex Python installations on users’ desktops is really hard because everyone ends up with slightly different environments. Docker allows us to flatten that environment." -
-
+

After all, in its three decades of existence, BlackRock has "a very well-established environment for managing our compute resources," says Francis. "We manage large cluster processes on machines, so we do a lot of orchestration and management for our main production processes in a way that's very cloudish in concept. We're able to manage them in a very controlled, static deployment scheme, and that has given us a huge amount of scalability."

-
-
- "We manage large cluster processes on machines, so we do a lot of orchestration and management for our main production processes in a way that’s very cloudish in concept. We’re able to manage them in a very controlled, static deployment scheme, and that has given us a huge amount of scalability." -
-
+

Though that works well for the core production, the company has found that some data science workloads require more dynamic access to resources. "It's a very bursty process," says Francis, who is head of data for the company's Aladdin investment management platform division.

-
-
- Still, challenges remain. "If you have a shared cluster, you get this storming herd problem where everyone wants to do the same thing at the same time," says Francis. "You could put limits on it, but you’d have to build an infrastructure to define limits for our processes, and the Python notebooks weren’t really designed for that. We have existing environments that do these things, but we needed to make it real, expansive, and scalable. Being able to spin that up on demand, tear it down, and make that much more dynamic, became a critical thought process for us."

- Made up of managers from technology, infrastructure, production operations, development and information security, Francis’s team was able to look at the problem holistically and come up with a solution that made sense for BlackRock. "Our initial straw man was that we were going to build everything using Ansible and run it all using some completely different distributed environment," says Francis. "That would have been absolutely the wrong thing to do. Had we gone off on our own as the dev team and developed this solution, it would have been a very different product. And it would have been very expensive. We would not have gone down the route of running under our existing orchestration system. Because we don’t understand it. These guys [in operations and infrastructure] understand it. Having the multidisciplinary team allowed us to get to the right solutions and that actually meant we didn’t build anywhere near the amount we thought we were going to end up building."

- In search of a solution in which they could manage usage on a user-by-user level, Francis’s team gravitated to Red Hat’s OpenShift Kubernetes offering. The company had already experimented with other cloud-native environments, but the team liked that Kubernetes was open source, and "we felt the winds were blowing in the direction of Kubernetes long term," says Francis. "Typically we make technology choices that we believe are going to be here in 5-10 years’ time, in some form. And right now, in this space, Kubernetes feels like the one that’s going to be there." Adds Uri Morris, Vice President of Production Operations: "When you see that the non-Google committers to Kubernetes overtook the Google committers, that’s an indicator of the momentum."

- Once that decision was made, the major challenge was figuring out how to make Kubernetes work within BlackRock’s existing framework. "It’s about understanding how we can operate, manage and support a platform like this, in addition to tacking it onto our existing technology platform," says Project Manager Michael Maskallis. "All the controls we have in place, the change management process, the software development lifecycle, onboarding processes we go through—how can we do all these things?"

- The first (anticipated) speed bump was working around issues behind BlackRock’s corporate firewalls. "One of our challenges is there are no firewalls in most open source software," says Francis. "So almost all install scripts fail in some bizarre way, and pulling down packages doesn’t necessarily work." The team ran into these types of problems using Minikube and did a few small pushes back to the open source project. +

Aladdin, which connects the people, information and technology needed for money management in real time, is used internally and is also sold as a platform to other asset managers and insurance companies. "We want to be able to give every investor access to data science, meaning Python notebooks, or even something much more advanced, like a MapReduce engine based on Spark," says Francis. But "managing complex Python installations on users' desktops is really hard because everyone ends up with slightly different environments. Docker allows us to flatten that environment."

+{{< case-studies/quote image="/images/case-studies/blackrock/banner3.jpg">}} +"We manage large cluster processes on machines, so we do a lot of orchestration and management for our main production processes in a way that's very cloudish in concept. We're able to manage them in a very controlled, static deployment scheme, and that has given us a huge amount of scalability." +{{< /case-studies/quote >}} -
-
+

Still, challenges remain. "If you have a shared cluster, you get this storming herd problem where everyone wants to do the same thing at the same time," says Francis. "You could put limits on it, but you'd have to build an infrastructure to define limits for our processes, and the Python notebooks weren't really designed for that. We have existing environments that do these things, but we needed to make it real, expansive, and scalable. Being able to spin that up on demand, tear it down, and make that much more dynamic, became a critical thought process for us."

-
-
- "Typically we make technology choices that we believe are going to be here in 5-10 years’ time, in some form. And right now, in this space, Kubernetes feels like the one that’s going to be there." -
-
+

Made up of managers from technology, infrastructure, production operations, development and information security, Francis's team was able to look at the problem holistically and come up with a solution that made sense for BlackRock. "Our initial straw man was that we were going to build everything using Ansible and run it all using some completely different distributed environment," says Francis. "That would have been absolutely the wrong thing to do. Had we gone off on our own as the dev team and developed this solution, it would have been a very different product. And it would have been very expensive. We would not have gone down the route of running under our existing orchestration system. Because we don't understand it. These guys [in operations and infrastructure] understand it. Having the multidisciplinary team allowed us to get to the right solutions and that actually meant we didn't build anywhere near the amount we thought we were going to end up building."

-
-
- There were also questions about service discovery. "You can think of Aladdin as a cloud of services with APIs between them that allows us to build applications rapidly," says Francis. "It’s all on a proprietary message bus, which gives us all sorts of advantages but at the same time, how does that play in a third party [platform]?"

- Another issue they had to navigate was that in BlackRock’s existing system, the messaging protocol has different instances in the different development, test and production environments. While Kubernetes enables a more DevOps-style model, it didn’t make sense for BlackRock. "I think what we are very proud of is that the ability for us to push into production is still incredibly rapid in this [new] infrastructure, but we have the control points in place, and we didn’t have to disrupt everything," says Francis. "A lot of the cost of this development was thinking how best to leverage our internal tools. So it was less costly than we actually thought it was going to be."

- The project leveraged tools associated with the messaging bus, for example. "The way that the Kubernetes cluster will talk to our internal messaging platform is through a gateway program, and this gateway program already has built-in checks and throttles," says Morris. "We can use them to control and potentially throttle the requests coming in from Kubernetes’s very elastic infrastructure to the production infrastructure. We’ll continue to go in that direction. It enables us to scale as we need to from the operational perspective."

- The solution also had to be complementary with BlackRock’s centralized operational support team structure. "The core infrastructure components of Kubernetes are hooked into our existing orchestration framework, which means that anyone in our support team has both control and visibility to the cluster using the existing operational tools," Morris explains. "That means that I don’t need to hire more people."

- With those points established, the team created a procedure for the project: "We rolled this out first to a development environment, then moved on to a testing environment and then eventually to two production environments, in that sequential order," says Maskallis. "That drove a lot of our learning curve. We have all these moving parts, the software components on the infrastructure side, the software components with Kubernetes directly, the interconnectivity with the rest of the environment that we operate here at BlackRock, and how we connect all these pieces. If we came across issues, we fixed them, and then moved on to the different environments to replicate that until we eventually ended up in our production environment where this particular cluster is supposed to live."

- The team had weekly one-hour working sessions with all the members (who are located around the world) participating, and smaller breakout or deep-dive meetings focusing on specific technical details. Possible solutions would be reported back to the group and debated the following week. "I think what made it a successful experiment was people had to work to learn, and they shared their experiences with others," says Vice President and Software Developer Fouad Semaan. Then, Francis says, "We gave our engineers the space to do what they’re good at. This hasn’t been top-down." +

In search of a solution in which they could manage usage on a user-by-user level, Francis's team gravitated to Red Hat's OpenShift Kubernetes offering. The company had already experimented with other cloud-native environments, but the team liked that Kubernetes was open source, and "we felt the winds were blowing in the direction of Kubernetes long term," says Francis. "Typically we make technology choices that we believe are going to be here in 5-10 years' time, in some form. And right now, in this space, Kubernetes feels like the one that's going to be there." Adds Uri Morris, Vice President of Production Operations: "When you see that the non-Google committers to Kubernetes overtook the Google committers, that's an indicator of the momentum."

+

Once that decision was made, the major challenge was figuring out how to make Kubernetes work within BlackRock's existing framework. "It's about understanding how we can operate, manage and support a platform like this, in addition to tacking it onto our existing technology platform," says Project Manager Michael Maskallis. "All the controls we have in place, the change management process, the software development lifecycle, onboarding processes we go through—how can we do all these things?"

-
-
+

The first (anticipated) speed bump was working around issues behind BlackRock's corporate firewalls. "One of our challenges is there are no firewalls in most open source software," says Francis. "So almost all install scripts fail in some bizarre way, and pulling down packages doesn't necessarily work." The team ran into these types of problems using Minikube and did a few small pushes back to the open source project.

-
-
- "The core infrastructure components of Kubernetes are hooked into our existing orchestration framework, which means that anyone in our support team has both control and visibility to the cluster using the existing operational tools. That means that I don’t need to hire more people." +{{< case-studies/quote image="/images/case-studies/blackrock/banner4.jpg">}} +"Typically we make technology choices that we believe are going to be here in 5-10 years' time, in some form. And right now, in this space, Kubernetes feels like the one that's going to be there." +{{< /case-studies/quote >}} -
-
+

There were also questions about service discovery. "You can think of Aladdin as a cloud of services with APIs between them that allows us to build applications rapidly," says Francis. "It's all on a proprietary message bus, which gives us all sorts of advantages but at the same time, how does that play in a third party [platform]?"

-
-
- They were led by one key axiom: To stay focused and avoid scope creep. This meant that they wouldn’t use features that weren’t in the core of Kubernetes and Docker. But if there was a real need, they’d build the features themselves. Luckily, Francis says, "Because of the rapidity of the development, a lot of things we thought we would have to build ourselves have been rolled into the core product. [The package manager Helm is one example]. People have similar problems."

- By the end of the 100 days, the app was up and running for internal BlackRock users. The initial capacity of 30 users was hit within hours, and quickly increased to 150. "People were immediately all over it," says Francis. In the next phase of this project, they are planning to scale up the cluster to have more capacity.

- Even more importantly, they now have in-production experience with Kubernetes that they can continue to build on—and a complete framework for rolling out new applications. "We’re going to use this infrastructure for lots of other application workloads as time goes on. It’s not just data science; it’s this style of application that needs the dynamism," says Francis. "Is it the right place to move our core production processes onto? It might be. We’re not at a point where we can say yes or no, but we felt that having real production experience with something like Kubernetes at some form and scale would allow us to understand that. I think we’re 6-12 months away from making a [large scale] decision. We need to gain experience of running the system in production, we need to understand failure modes and how best to manage operational issues."

- For other big companies considering a project like this, Francis says commitment and dedication are key: "We got the signoff from [senior management] from day one, with the commitment that we were able to get the right people. If I had to isolate what makes something complex like this succeed, I would say senior hands-on people who can actually drive it make a huge difference." With that in place, he adds, "My message to other enterprises like us is you can actually integrate Kubernetes into an existing, well-orchestrated machinery. You don’t have to throw out everything you do. And using Kubernetes made a complex problem significantly easier." +

Another issue they had to navigate was that in BlackRock's existing system, the messaging protocol has different instances in the different development, test and production environments. While Kubernetes enables a more DevOps-style model, it didn't make sense for BlackRock. "I think what we are very proud of is that the ability for us to push into production is still incredibly rapid in this [new] infrastructure, but we have the control points in place, and we didn't have to disrupt everything," says Francis. "A lot of the cost of this development was thinking how best to leverage our internal tools. So it was less costly than we actually thought it was going to be."

-
-
+

The project leveraged tools associated with the messaging bus, for example. "The way that the Kubernetes cluster will talk to our internal messaging platform is through a gateway program, and this gateway program already has built-in checks and throttles," says Morris. "We can use them to control and potentially throttle the requests coming in from Kubernetes's very elastic infrastructure to the production infrastructure. We'll continue to go in that direction. It enables us to scale as we need to from the operational perspective."

+ +

The solution also had to be complementary with BlackRock's centralized operational support team structure. "The core infrastructure components of Kubernetes are hooked into our existing orchestration framework, which means that anyone in our support team has both control and visibility to the cluster using the existing operational tools," Morris explains. "That means that I don't need to hire more people."

+ +

With those points established, the team created a procedure for the project: "We rolled this out first to a development environment, then moved on to a testing environment and then eventually to two production environments, in that sequential order," says Maskallis. "That drove a lot of our learning curve. We have all these moving parts, the software components on the infrastructure side, the software components with Kubernetes directly, the interconnectivity with the rest of the environment that we operate here at BlackRock, and how we connect all these pieces. If we came across issues, we fixed them, and then moved on to the different environments to replicate that until we eventually ended up in our production environment where this particular cluster is supposed to live."

+ +

The team had weekly one-hour working sessions with all the members (who are located around the world) participating, and smaller breakout or deep-dive meetings focusing on specific technical details. Possible solutions would be reported back to the group and debated the following week. "I think what made it a successful experiment was people had to work to learn, and they shared their experiences with others," says Vice President and Software Developer Fouad Semaan. Then, Francis says, "We gave our engineers the space to do what they're good at. This hasn't been top-down."

+ +{{< case-studies/quote >}} +"The core infrastructure components of Kubernetes are hooked into our existing orchestration framework, which means that anyone in our support team has both control and visibility to the cluster using the existing operational tools. That means that I don't need to hire more people." +{{< /case-studies/quote >}} + +

They were led by one key axiom: To stay focused and avoid scope creep. This meant that they wouldn't use features that weren't in the core of Kubernetes and Docker. But if there was a real need, they'd build the features themselves. Luckily, Francis says, "Because of the rapidity of the development, a lot of things we thought we would have to build ourselves have been rolled into the core product. [The package manager Helm is one example]. People have similar problems."

+ +

By the end of the 100 days, the app was up and running for internal BlackRock users. The initial capacity of 30 users was hit within hours, and quickly increased to 150. "People were immediately all over it," says Francis. In the next phase of this project, they are planning to scale up the cluster to have more capacity.

+ +

Even more importantly, they now have in-production experience with Kubernetes that they can continue to build on—and a complete framework for rolling out new applications. "We're going to use this infrastructure for lots of other application workloads as time goes on. It's not just data science; it's this style of application that needs the dynamism," says Francis. "Is it the right place to move our core production processes onto? It might be. We're not at a point where we can say yes or no, but we felt that having real production experience with something like Kubernetes at some form and scale would allow us to understand that. I think we're 6-12 months away from making a [large scale] decision. We need to gain experience of running the system in production, we need to understand failure modes and how best to manage operational issues."

+ +

For other big companies considering a project like this, Francis says commitment and dedication are key: "We got the signoff from [senior management] from day one, with the commitment that we were able to get the right people. If I had to isolate what makes something complex like this succeed, I would say senior hands-on people who can actually drive it make a huge difference." With that in place, he adds, "My message to other enterprises like us is you can actually integrate Kubernetes into an existing, well-orchestrated machinery. You don't have to throw out everything you do. And using Kubernetes made a complex problem significantly easier."

diff --git a/content/en/case-studies/booking-com/index.html b/content/en/case-studies/booking-com/index.html index 99369a2bf9..74d5b282b1 100644 --- a/content/en/case-studies/booking-com/index.html +++ b/content/en/case-studies/booking-com/index.html @@ -3,115 +3,84 @@ title: Booking.com Case Study linkTitle: Booking.com case_study_styles: true cid: caseStudies -css: /css/case-studies-gradient.css logo: booking.com_featured_logo.png featured: true weight: 3 quote: > We realized that we needed to learn Kubernetes better in order to fully use the potential of it. At that point, we made the shift to build our own Kubernetes platform. + +new_case_study_styles: true +heading_background: /images/case-studies/booking/banner1.jpg +heading_title_text: Booking.com +use_gradient_overlay: true +subheading: > + After Learning the Ropes with a Kubernetes Distribution, Booking.com Built a Platform of Its Own +case_study_details: + - Company: Booking.com + - Location: Netherlands + - Industry: Travel --- -​ +

Challenge

+

In 2016, Booking.com migrated to an OpenShift platform, which gave product developers faster access to infrastructure. But because Kubernetes was abstracted away from the developers, the infrastructure team became a "knowledge bottleneck" when challenges arose. Trying to scale that support wasn't sustainable.

-
-

CASE STUDY: Booking.com

-
After Learning the Ropes with a Kubernetes Distribution, Booking.com Built a Platform of Its Own
-
-​ -​ -
- Company  Booking.com     Location  Netherlands     Industry  Travel -
-​ -
-
-
-
-

Challenge

- In 2016, Booking.com migrated to an OpenShift platform, which gave product developers faster access to infrastructure. But because Kubernetes was abstracted away from the developers, the infrastructure team became a “knowledge bottleneck” when challenges arose. Trying to scale that support wasn’t sustainable. +

Solution

+

After a year operating OpenShift, the platform team decided to build its own vanilla Kubernetes platform—and ask developers to learn some Kubernetes in order to use it. "This is not a magical platform," says Ben Tyler, Principal Developer, B Platform Track. "We're not claiming that you can just use it with your eyes closed. Developers need to do some learning, and we're going to do everything we can to make sure they have access to that knowledge."

-

Solution

- After a year operating OpenShift, the platform team decided to build its own vanilla Kubernetes platform—and ask developers to learn some Kubernetes in order to use it. “This is not a magical platform,” says Ben Tyler, Principal Developer, B Platform Track. “We’re not claiming that you can just use it with your eyes closed. Developers need to do some learning, and we’re going to do everything we can to make sure they have access to that knowledge.” -​

Impact

- Despite the learning curve, there’s been a great uptick in adoption of the new Kubernetes platform. Before containers, creating a new service could take a couple of days if the developers understood Puppet, or weeks if they didn’t. On the new platform, it can take as few as 10 minutes. About 500 new services were built on the platform in the first 8 months. -
-
-
-
-
- “As our users learn Kubernetes and become more sophisticated Kubernetes users, they put pressure on us to provide a better, more native Kubernetes experience, which is great. It’s a super healthy dynamic.” -

- BEN TYLER, PRINCIPAL DEVELOPER, B PLATFORM TRACK AT BOOKING.COM

-
-
-​ -​ -
-
-

Booking.com has a long history with Kubernetes: In 2015, a team at the travel platform prototyped a container platform based on Mesos and Marathon. -

-

Impressed by what the technology offered, but in need of enterprise features at its scale—the site handles more than 1.5 million room-night reservations a day on average—the team decided to adopt an OpenShift platform.

+

Despite the learning curve, there's been a great uptick in adoption of the new Kubernetes platform. Before containers, creating a new service could take a couple of days if the developers understood Puppet, or weeks if they didn't. On the new platform, it can take as few as 10 minutes. About 500 new services were built on the platform in the first 8 months.

-

This platform, which was wrapped in a Heroku-style, high-level CLI interface, “was definitely popular with our product developers,” says Ben Tyler, Principal Developer, B Platform Track. “We gave them faster access to infrastructure.”

+{{< case-studies/quote + image="/images/case-studies/booking/banner2.jpg" + author="BEN TYLER, PRINCIPAL DEVELOPER, B PLATFORM TRACK AT BOOKING.COM" +>}} +"As our users learn Kubernetes and become more sophisticated Kubernetes users, they put pressure on us to provide a better, more native Kubernetes experience, which is great. It's a super healthy dynamic." +{{< /case-studies/quote >}} -

But, he adds, “anytime something went slightly off the rails, developers didn’t have any of the knowledge required to support themselves.”

+​{{< case-studies/lead >}} +Booking.com has a long history with Kubernetes: In 2015, a team at the travel platform prototyped a container platform based on Mesos and Marathon. +{{< /case-studies/lead >}} -

And after a year of operating this platform, the infrastructure team found that it had become “a knowledge bottleneck,” he says. “Most of the developers who used it did not know it was Kubernetes underneath. An application failure and a platform failure both looked like failures of that Heroku-style tool.”

-

- Scaling the necessary support did not seem feasible or sustainable, so the platform team needed a new solution. The understanding of Kubernetes that they had gained operating the OpenShift platform gave them confidence to build a vanilla Kubernetes platform of their own and customize it to suit the company’s needs.

+

Impressed by what the technology offered, but in need of enterprise features at its scale—the site handles more than 1.5 million room-night reservations a day on average—the team decided to adopt an OpenShift platform.

+

This platform, which was wrapped in a Heroku-style, high-level CLI interface, "was definitely popular with our product developers," says Ben Tyler, Principal Developer, B Platform Track. "We gave them faster access to infrastructure."

-
-
-​ -
-
- “For entering the landscape, OpenShift was definitely very helpful. It shows you what the technology can do, and it makes it easy for you to use it. After we spent some time on it, we realized that we needed to learn Kubernetes better in order to fully use the potential of it. At that point, we made the shift to build our own Kubernetes platform. We definitely benefit in the long term for taking that step and investing the time in gaining that knowledge.”

- EDUARD IACOBOAIA, SENIOR SYSTEM ADMINISTRATOR, B PLATFORM TRACK AT BOOKING.COM

-
-
-​ -
-
-

- “For entering the landscape, OpenShift was definitely very helpful,” says Eduard Iacoboaia, Senior System Administrator, B Platform Track. “It shows you what the technology can do, and it makes it easy for you to use it. After we spent some time on it, we realized that we needed to learn Kubernetes better in order to fully use the potential of it. At that point, we made the shift to build our own Kubernetes platform. We definitely benefit in the long term for taking that step and investing the time in gaining that knowledge.”

-

- Iacoboaia’s team had customized a lot of OpenShift tools to make them work at Booking.com, and “those integrations points were kind of fragile,” he says. “We spent much more time understanding all the components of Kubernetes, how they work, how they interact with each other.” That research led the team to switch from OpenShift’s built-in Ansible playbooks to Puppet deployments, which are used for the rest of Booking’s infrastructure. The control plane was also moved from inside the cluster onto bare metal, as the company runs tens of thousands of bare-metal servers and a large infrastructure for running applications on bare metal. (Booking runs Kubernetes in multiple clusters in multiple data centers across the various regions where it has compute.) “We decided to keep it as simple as possible and to also use the tools that we know best,” says Iacoboaia.

-

- The other big change was that product engineers would have to learn Kubernetes in order to onboard. “This is not a magical platform,” says Tyler. “We’re not claiming that you can just use it with your eyes closed. Developers need to do some learning, and we’re going to do everything we can to make sure they have access to that knowledge.” That includes trainings, blog posts, videos, and Udemy courses.

-

- Despite the learning curve, there’s been a great uptick in adoption of the new Kubernetes platform. “I think the reason we’ve been able to strike this bargain successfully is that we’re not asking them to learn a proprietary app system,” says Tyler. “We’re asking them to learn something that’s open source, where the knowledge is transferable. They’re investing in their own careers by learning Kubernetes.”

-

- One clear sign that this strategy has been a success is that in the support channel, when users have questions, other product engineers are jumping in to respond. “I haven’t seen that kind of community engagement around a particular platform product internally before,” says Tyler. “It helps a lot that it’s visibly an ecosystem standard outside of the company, so people feel value in investing in that knowledge and sharing it with others, which is really, really powerful.”

+

But, he adds, "anytime something went slightly off the rails, developers didn't have any of the knowledge required to support themselves."

+

And after a year of operating this platform, the infrastructure team found that it had become "a knowledge bottleneck," he says. "Most of the developers who used it did not know it was Kubernetes underneath. An application failure and a platform failure both looked like failures of that Heroku-style tool."

-
-
-​ -​ -
-
- “We have a tutorial. You follow the tutorial. Your code is running. Then, it’s business-logic time. The time to gain access to resources is decreased enormously.”

- BEN TYLER, PRINCIPAL DEVELOPER, B PLATFORM TRACK AT BOOKING.COM

-
-
-​ -
-
-

- There’s other quantifiable evidence too: Before containers, creating a new service could take a couple of days if the developers understood Puppet, or weeks if they didn’t. On the new platform, it takes 10 minutes. “We have a tutorial. You follow the tutorial. Your code is running. Then, it’s business-logic time,” says Tyler. “The time to gain access to resources is decreased enormously.” About 500 new services were built in the first 8 months on the platform, with hundreds of releases per day.

-

- The platform offers different “layers of contracts, so to speak,” says Tyler. “At the very base, it’s just Kubernetes. If you’re a pro Kubernetes user, here’s a Kubernetes API, just like you get from GKE or AKS. We’re trying to be a provider on that same level. But our whole job inside the company is to be a bigger value add than just vanilla infrastructure, so we provide a set of base images for our main stacks, Perl and Java.” -

-

- And “as our users learn Kubernetes and become more sophisticated Kubernetes users, they put pressure on us to provide a better more native Kubernetes experience, which is great,” says Tyler. “It’s a super healthy dynamic.”

-

- The platform also includes other CNCF technologies, such as Envoy, Helm, and Prometheus. Most of the critical service traffic for Booking.com is routed through Envoy, and Prometheus is used primarily to monitor infrastructure components. Helm is consumed as a packaging standard. The team also developed and open sourced Shipper, an extension for Kubernetes to add more complex rollout strategies and multi-cluster orchestration. -

-

- To be sure, there have been internal discussions about the wisdom of building a Kubernetes platform from the ground up. “This is not really our core competency—Kubernetes and travel, they’re kind of far apart, right?” says Tyler. “But we’ve made a couple of bets on CNCF components that have worked out really well for us. Envoy and Kubernetes, in particular, have been really beneficial to our organization. We were able to customize them, either because we could look at the source code or because they had extension points, and we were able to get value out of them very quickly without having to change any paradigms internally.” -

-
-
- +

Scaling the necessary support did not seem feasible or sustainable, so the platform team needed a new solution. The understanding of Kubernetes that they had gained operating the OpenShift platform gave them confidence to build a vanilla Kubernetes platform of their own and customize it to suit the company's needs.

+ +{{< case-studies/quote author="EDUARD IACOBOAIA, SENIOR SYSTEM ADMINISTRATOR, B PLATFORM TRACK AT BOOKING.COM" >}} +"For entering the landscape, OpenShift was definitely very helpful. It shows you what the technology can do, and it makes it easy for you to use it. After we spent some time on it, we realized that we needed to learn Kubernetes better in order to fully use the potential of it. At that point, we made the shift to build our own Kubernetes platform. We definitely benefit in the long term for taking that step and investing the time in gaining that knowledge." +{{< /case-studies/quote >}} + +

"For entering the landscape, OpenShift was definitely very helpful," says Eduard Iacoboaia, Senior System Administrator, B Platform Track. "It shows you what the technology can do, and it makes it easy for you to use it. After we spent some time on it, we realized that we needed to learn Kubernetes better in order to fully use the potential of it. At that point, we made the shift to build our own Kubernetes platform. We definitely benefit in the long term for taking that step and investing the time in gaining that knowledge."

+ +

Iacoboaia's team had customized a lot of OpenShift tools to make them work at Booking.com, and "those integrations points were kind of fragile," he says. "We spent much more time understanding all the components of Kubernetes, how they work, how they interact with each other." That research led the team to switch from OpenShift's built-in Ansible playbooks to Puppet deployments, which are used for the rest of Booking's infrastructure. The control plane was also moved from inside the cluster onto bare metal, as the company runs tens of thousands of bare-metal servers and a large infrastructure for running applications on bare metal. (Booking runs Kubernetes in multiple clusters in multiple data centers across the various regions where it has compute.) "We decided to keep it as simple as possible and to also use the tools that we know best," says Iacoboaia.

+ +

The other big change was that product engineers would have to learn Kubernetes in order to onboard. "This is not a magical platform," says Tyler. "We're not claiming that you can just use it with your eyes closed. Developers need to do some learning, and we're going to do everything we can to make sure they have access to that knowledge." That includes trainings, blog posts, videos, and Udemy courses.

+ +

Despite the learning curve, there's been a great uptick in adoption of the new Kubernetes platform. "I think the reason we've been able to strike this bargain successfully is that we're not asking them to learn a proprietary app system," says Tyler. "We're asking them to learn something that's open source, where the knowledge is transferable. They're investing in their own careers by learning Kubernetes."

+ +

One clear sign that this strategy has been a success is that in the support channel, when users have questions, other product engineers are jumping in to respond. "I haven't seen that kind of community engagement around a particular platform product internally before," says Tyler. "It helps a lot that it's visibly an ecosystem standard outside of the company, so people feel value in investing in that knowledge and sharing it with others, which is really, really powerful."

+ +{{< case-studies/quote + image="/images/case-studies/booking/banner3.jpg" + author="BEN TYLER, PRINCIPAL DEVELOPER, B PLATFORM TRACK AT BOOKING.COM" +>}} +"We have a tutorial. You follow the tutorial. Your code is running. Then, it's business-logic time. The time to gain access to resources is decreased enormously." +{{< /case-studies/quote >}} + +

There's other quantifiable evidence too: Before containers, creating a new service could take a couple of days if the developers understood Puppet, or weeks if they didn't. On the new platform, it takes 10 minutes. "We have a tutorial. You follow the tutorial. Your code is running. Then, it's business-logic time," says Tyler. "The time to gain access to resources is decreased enormously." About 500 new services were built in the first 8 months on the platform, with hundreds of releases per day.

+ +

The platform offers different "layers of contracts, so to speak," says Tyler. "At the very base, it's just Kubernetes. If you're a pro Kubernetes user, here's a Kubernetes API, just like you get from GKE or AKS. We're trying to be a provider on that same level. But our whole job inside the company is to be a bigger value add than just vanilla infrastructure, so we provide a set of base images for our main stacks, Perl and Java."

+ +

And "as our users learn Kubernetes and become more sophisticated Kubernetes users, they put pressure on us to provide a better more native Kubernetes experience, which is great," says Tyler. "It's a super healthy dynamic."

+ +

The platform also includes other CNCF technologies, such as Envoy, Helm, and Prometheus. Most of the critical service traffic for Booking.com is routed through Envoy, and Prometheus is used primarily to monitor infrastructure components. Helm is consumed as a packaging standard. The team also developed and open sourced Shipper, an extension for Kubernetes to add more complex rollout strategies and multi-cluster orchestration.

+ +

To be sure, there have been internal discussions about the wisdom of building a Kubernetes platform from the ground up. "This is not really our core competency—Kubernetes and travel, they're kind of far apart, right?" says Tyler. "But we've made a couple of bets on CNCF components that have worked out really well for us. Envoy and Kubernetes, in particular, have been really beneficial to our organization. We were able to customize them, either because we could look at the source code or because they had extension points, and we were able to get value out of them very quickly without having to change any paradigms internally."

diff --git a/content/en/case-studies/booz-allen/index.html b/content/en/case-studies/booz-allen/index.html index fdda5e976a..7b53dc01ae 100644 --- a/content/en/case-studies/booz-allen/index.html +++ b/content/en/case-studies/booz-allen/index.html @@ -3,101 +3,78 @@ title: Booz Allen Case Study linkTitle: Booz Allen Hamilton case_study_styles: true cid: caseStudies -css: /css/case-studies-gradient.css logo: booz-allen-featured-logo.svg featured: true weight: 2 quote: > - Kubernetes is a great solution for us. It allows us to rapidly iterate on our clients’ demands. + Kubernetes is a great solution for us. It allows us to rapidly iterate on our clients' demands. + +new_case_study_styles: true +heading_background: /images/case-studies/booz-allen/banner4.jpg +heading_title_text: Booz Allen Hamilton +use_gradient_overlay: true +subheading: > + How Booz Allen Hamilton Is Helping Modernize the Federal Government with Kubernetes +case_study_details: + - Company: Booz Allen Hamilton + - Location: United States + - Industry: Government --- -​ +

Challenge

-
-

CASE STUDY: Booz Allen Hamilton

-
How Booz Allen Hamilton Is Helping Modernize the Federal Government with Kubernetes
-
-​ -​ -
- Company  Booz Allen Hamilton     Location  United States     Industry  Government -
-​ -
-
-
-
-

Challenge

- In 2017, Booz Allen Hamilton’s Strategic Innovation Group worked with the federal government to relaunch the decade-old recreation.gov website, which provides information and real-time booking for more than 100,000 campsites and facilities on federal lands across the country. The infrastructure needed to be agile, reliable, and scalable—as well as repeatable for the other federal agencies that are among Booz Allen Hamilton’s customers. +

In 2017, Booz Allen Hamilton's Strategic Innovation Group worked with the federal government to relaunch the decade-old recreation.gov website, which provides information and real-time booking for more than 100,000 campsites and facilities on federal lands across the country. The infrastructure needed to be agile, reliable, and scalable—as well as repeatable for the other federal agencies that are among Booz Allen Hamilton's customers.

+ +

Solution

+ +

"The only way that we thought we could be successful with this problem across all the different agencies is to create a microservice architecture and containers, so that we could be very dynamic and very agile to any given agency for whatever requirements that they may have," says Booz Allen Hamilton Senior Lead Technologist Martin Folkoff. To meet those requirements, Folkoff's team looked to Kubernetes for orchestration.

-

Solution

- “The only way that we thought we could be successful with this problem across all the different agencies is to create a microservice architecture and containers, so that we could be very dynamic and very agile to any given agency for whatever requirements that they may have,” says Booz Allen Hamilton Senior Lead Technologist Martin Folkoff. To meet those requirements, Folkoff’s team looked to Kubernetes for orchestration. -​

Impact

- With the recreation.gov Kubernetes platform, changes can be implemented in about 30 minutes, compared to the multiple hours or even days legacy government applications require to review the code, get approval, and deploy the fix. Recreation.gov deploys to production on average 10 times a day. With monitoring, security, and logging built in, developers can create and publish new services to production within a week. Additionally, Folkoff says, “supporting the large, existing monoliths in the government is extremely expensive,” and migrating into a more modern platform has resulted in perhaps 50% cost savings. -
-
-
-
-
- "When there’s a regulatory change in an agency, or a legislative change in Congress, or an executive order that changes the way you do business, how do I deploy that and get that out to the people who need it rapidly? At the end of the day, that’s the problem we’re trying to help the government solve with tools like Kubernetes." -

- JOSH BOYD, CHIEF TECHNOLOGIST AT BOOZ ALLEN HAMILTON

-
-
-​ -​ -
-
-

The White House launched an IT modernization effort in 2017, and in addition to improving cybersecurity and shifting to the public cloud and a consolidated IT model, “the federal government is looking to provide a better experience to citizens in every way that we interact with the government through every channel,” says Booz Allen Hamilton Senior Lead Technologist Martin Folkoff.

-

To that end, Folkoff’s Strategic Innovation Group worked with the federal government last year to relaunch the decade-old recreation.gov website, which provides information and real-time booking for more than 100,000 campsites and facilities on federal lands across the country.

+

With the recreation.gov Kubernetes platform, changes can be implemented in about 30 minutes, compared to the multiple hours or even days legacy government applications require to review the code, get approval, and deploy the fix. Recreation.gov deploys to production on average 10 times a day. With monitoring, security, and logging built in, developers can create and publish new services to production within a week. Additionally, Folkoff says, "supporting the large, existing monoliths in the government is extremely expensive," and migrating into a more modern platform has resulted in perhaps 50% cost savings.

-

The infrastructure needed to be agile, reliable, and scalable—as well as repeatable for the other federal agencies that are among Booz Allen Hamilton’s customers. “The only way that we thought we could be successful with this problem across all the different agencies is to create a microservice architecture, so that we could be very dynamic and very agile to any given agency for whatever requirements that they may have,” says Folkoff.

-
-
+{{< case-studies/quote + image="/images/case-studies/booz-allen/banner2.jpg" + author="JOSH BOYD, CHIEF TECHNOLOGIST AT BOOZ ALLEN HAMILTON" +>}} +"When there's a regulatory change in an agency, or a legislative change in Congress, or an executive order that changes the way you do business, how do I deploy that and get that out to the people who need it rapidly? At the end of the day, that's the problem we're trying to help the government solve with tools like Kubernetes." +{{< /case-studies/quote >}} ​ -
-
- "With CNCF, there’s a lot of focus on scale, and so there’s a lot of comfort knowing that as the project grows, we’re going to be comfortable using that tool set."

- MARTIN FOLKOFF, SENIOR LEAD TECHNOLOGIST AT BOOZ ALLEN HAMILTON

-
-
+​{{< case-studies/lead >}} +The White House launched an IT modernization effort in 2017, and in addition to improving cybersecurity and shifting to the public cloud and a consolidated IT model, "the federal government is looking to provide a better experience to citizens in every way that we interact with the government through every channel," says Booz Allen Hamilton Senior Lead Technologist Martin Folkoff. +{{< /case-studies/lead >}} + +

To that end, Folkoff's Strategic Innovation Group worked with the federal government last year to relaunch the decade-old recreation.gov website, which provides information and real-time booking for more than 100,000 campsites and facilities on federal lands across the country.

+ +

The infrastructure needed to be agile, reliable, and scalable—as well as repeatable for the other federal agencies that are among Booz Allen Hamilton's customers. "The only way that we thought we could be successful with this problem across all the different agencies is to create a microservice architecture, so that we could be very dynamic and very agile to any given agency for whatever requirements that they may have," says Folkoff.

+ +{{< case-studies/quote author="MARTIN FOLKOFF, SENIOR LEAD TECHNOLOGIST AT BOOZ ALLEN HAMILTON" >}} +"With CNCF, there's a lot of focus on scale, and so there's a lot of comfort knowing that as the project grows, we're going to be comfortable using that tool set." +{{< /case-studies/quote >}} + +

Booz Allen Hamilton, which has provided consulting services to the federal government for more than a century, introduced microservices, Docker containers, and AWS to its federal agency clients about five years ago. The next logical step was Kubernetes for orchestration. "Knowing that we had to be really agile and really reliable and scalable, we felt that the only technology that we know that can enable those kinds of things are the ones the CNCF provides," Folkoff says. "One of the things that is always important for the government is to make sure that the things that we build really endure. Using technology that is supported across multiple different companies and has strong governance gives people a lot of confidence."

+ +

Kubernetes was also aligned with the government's open source and IT modernization initiatives, so there has been an uptick in its usage at federal agencies over the past two years. "Now that Kubernetes is becoming offered as a service by the cloud providers like AWS and Microsoft, we're starting to see even more interest," says Chief Technologist Josh Boyd. Adds Folkoff: "With CNCF, there's a lot of focus on scale, and so there's a lot of comfort knowing that as the project grows, we're going to be comfortable using that tool set."

+ +

The greenfield recreation.gov project allowed the team to build a new Kubernetes-enabled site running on AWS, and the migration lasted only a week, when the old site didn't take bookings. "For the actual transition, we just swapped a DNS server, and it only took about 35 seconds between the old site being down and our new site being up and available," Folkoff adds.

​ -
-
-

- Booz Allen Hamilton, which has provided consulting services to the federal government for more than a century, introduced microservices, Docker containers, and AWS to its federal agency clients about five years ago. The next logical step was Kubernetes for orchestration. “Knowing that we had to be really agile and really reliable and scalable, we felt that the only technology that we know that can enable those kinds of things are the ones the CNCF provides,” Folkoff says. “One of the things that is always important for the government is to make sure that the things that we build really endure. Using technology that is supported across multiple different companies and has strong governance gives people a lot of confidence.”

-

- Kubernetes was also aligned with the government’s open source and IT modernization initiatives, so there has been an uptick in its usage at federal agencies over the past two years. “Now that Kubernetes is becoming offered as a service by the cloud providers like AWS and Microsoft, we’re starting to see even more interest,” says Chief Technologist Josh Boyd. Adds Folkoff: “With CNCF, there’s a lot of focus on scale, and so there’s a lot of comfort knowing that as the project grows, we’re going to be comfortable using that tool set.”

-

- The greenfield recreation.gov project allowed the team to build a new Kubernetes-enabled site running on AWS, and the migration lasted only a week, when the old site didn’t take bookings. “For the actual transition, we just swapped a DNS server, and it only took about 35 seconds between the old site being down and our new site being up and available,” Folkoff adds.

-

-
-
-​ -​ -
-
- "Kubernetes alone enables a dramatic reduction in cost as resources are prioritized to the day’s event"

- MARTIN FOLKOFF, SENIOR LEAD TECHNOLOGIST AT BOOZ ALLEN HAMILTON

-
-
-​ -
-
-

- In addition to its work with the Department of Interior for recreation.gov, Booz Allen Hamilton has brought Kubernetes to various Defense, Intelligence, and civilian agencies. Says Boyd: “When there’s a regulatory change in an agency, or a legislative change in Congress, or an executive order that changes the way you do business, how do I deploy that and get that out to the people who need it rapidly? At the end of the day, that’s the problem we’re trying to help the government solve with tools like Kubernetes.”

-

- For recreation.gov, the impact was clear and immediate. With the Kubernetes platform, Folkoff says, “if a new requirement for a permit comes out, we have the ability to design and develop and implement that completely independently of reserving a campsite. It provides a much better experience to users.” Today, changes can be implemented in about 30 minutes, compared to the multiple hours or even days legacy government applications require to review the code, get approval, and deploy the fix. Recreation.gov deploys to production on average 10 times a day.

-

- Developer velocity has been improved. “When I want to do monitoring or security or logging, I don’t have to do anything to my services or my application to enable that anymore,” says Boyd. “I get all of this magic just by being on the Kubernetes platform.” With all of those things built in, developers can create and publish new services to production within one week.

-

- Additionally, Folkoff says, “supporting the large, existing monoliths in the government is extremely expensive,” and migrating into a more modern platform has resulted in perhaps 50% cost savings. “Kubernetes alone enables a dramatic reduction in cost as resources are prioritized to the day’s event,” he says. “For example, during a popular campsite release, camping-related services are scaled out while permit services are scaled down.”

-

- So far, “Kubernetes is a great solution for us,” says Folkoff. “It allows us to rapidly iterate on our clients’ demands.” Looking ahead, the team sees further adoption of the Kubernetes platform across federal agencies. Says Boyd: “You get the ability for the rapid delivery of business value for your customers. You now have observability into everything that you’re doing. You don’t have these onesies and twosies unicorn servers anymore. Now everything that you deploy is deployed in the same way, it’s all instrumented the same way, and it’s all built and deployed the same way through our CI/CD processes.”

-

- They also see a push toward re-platforming. “There’s still a lot of legacy workloads out there,” says Boyd. “We’ve got the new challenges of greenfield development and integration with legacy systems, but also that brown field of ‘Hey, how do I take this legacy monolith and get it onto a platform where now it’s instrumented with all the magic of the Kubernetes platform without having to do a whole lot to my application?’ I think re-platforming is a pretty big use case for the government right now.”

-

- And given the success that they’ve had with Kubernetes so far, Boyd says, “I think at this point that technology is becoming pretty easy to sell.” Adds Folkoff: “People are really excited about being able to deploy, scale, be reliable, and do cheaper maintenance of all of this.” -

-
-
- +{{< case-studies/quote + image="/images/case-studies/booz-allen/banner1.png" + author="MARTIN FOLKOFF, SENIOR LEAD TECHNOLOGIST AT BOOZ ALLEN HAMILTON" +>}} +"Kubernetes alone enables a dramatic reduction in cost as resources are prioritized to the day's event" +{{< /case-studies/quote >}} + +

In addition to its work with the Department of Interior for recreation.gov, Booz Allen Hamilton has brought Kubernetes to various Defense, Intelligence, and civilian agencies. Says Boyd: "When there's a regulatory change in an agency, or a legislative change in Congress, or an executive order that changes the way you do business, how do I deploy that and get that out to the people who need it rapidly? At the end of the day, that's the problem we're trying to help the government solve with tools like Kubernetes."

+ +

For recreation.gov, the impact was clear and immediate. With the Kubernetes platform, Folkoff says, "if a new requirement for a permit comes out, we have the ability to design and develop and implement that completely independently of reserving a campsite. It provides a much better experience to users." Today, changes can be implemented in about 30 minutes, compared to the multiple hours or even days legacy government applications require to review the code, get approval, and deploy the fix. Recreation.gov deploys to production on average 10 times a day.

+ +

Developer velocity has been improved. "When I want to do monitoring or security or logging, I don't have to do anything to my services or my application to enable that anymore," says Boyd. "I get all of this magic just by being on the Kubernetes platform." With all of those things built in, developers can create and publish new services to production within one week.

+ +

Additionally, Folkoff says, "supporting the large, existing monoliths in the government is extremely expensive," and migrating into a more modern platform has resulted in perhaps 50% cost savings. "Kubernetes alone enables a dramatic reduction in cost as resources are prioritized to the day's event," he says. "For example, during a popular campsite release, camping-related services are scaled out while permit services are scaled down."

+ +

So far, "Kubernetes is a great solution for us," says Folkoff. "It allows us to rapidly iterate on our clients' demands." Looking ahead, the team sees further adoption of the Kubernetes platform across federal agencies. Says Boyd: "You get the ability for the rapid delivery of business value for your customers. You now have observability into everything that you're doing. You don't have these onesies and twosies unicorn servers anymore. Now everything that you deploy is deployed in the same way, it's all instrumented the same way, and it's all built and deployed the same way through our CI/CD processes."

+ +

They also see a push toward re-platforming. "There's still a lot of legacy workloads out there," says Boyd. "We've got the new challenges of greenfield development and integration with legacy systems, but also that brown field of 'Hey, how do I take this legacy monolith and get it onto a platform where now it's instrumented with all the magic of the Kubernetes platform without having to do a whole lot to my application?' I think re-platforming is a pretty big use case for the government right now."

+ +

And given the success that they've had with Kubernetes so far, Boyd says, "I think at this point that technology is becoming pretty easy to sell." Adds Folkoff: "People are really excited about being able to deploy, scale, be reliable, and do cheaper maintenance of all of this."

diff --git a/content/en/case-studies/bose/index.html b/content/en/case-studies/bose/index.html index c77f416c13..70deb6ce68 100644 --- a/content/en/case-studies/bose/index.html +++ b/content/en/case-studies/bose/index.html @@ -3,101 +3,85 @@ title: Bose Case Study linkTitle: Bose case_study_styles: true cid: caseStudies -css: /css/style_case_studies.css logo: bose_featured_logo.png featured: false weight: 2 quote: > - The CNCF Landscape quickly explains what’s going on in all the different areas from storage to cloud providers to automation and so forth. This is our shopping cart to build a cloud infrastructure. We can go choose from the different aisles. + The CNCF Landscape quickly explains what's going on in all the different areas from storage to cloud providers to automation and so forth. This is our shopping cart to build a cloud infrastructure. We can go choose from the different aisles. + +new_case_study_styles: true +heading_background: /images/case-studies/bose/banner1.jpg +heading_title_logo: /images/bose_logo.png +subheading: > + Bose: Supporting Rapid Development for Millions of IoT Products With Kubernetes +case_study_details: + - Company: Bose Corporation + - Location: Framingham, Massachusetts + - Industry: Consumer Electronics --- -
-

CASE STUDY:
Bose: Supporting Rapid Development for Millions of IoT Products With Kubernetes +

Challenge

-

+

A household name in high-quality audio equipment, Bose has offered connected products for more than five years, and as that demand grew, the infrastructure had to change to support it. "We needed to provide a mechanism for developers to rapidly prototype and deploy services all the way to production pretty fast," says Lead Cloud Engineer Josh West. In 2016, the company decided to start building a platform from scratch. The primary goal: "To be one to two steps ahead of the different product groups so that we are never scrambling to catch up with their scale," says Cloud Architecture Manager Dylan O'Mahony.

-
+

Solution

-
- Company  Bose Corporation     Location  Framingham, Massachusetts -     Industry  Consumer Electronics -
+

From the beginning, the team knew it wanted a microservices architecture. After evaluating and prototyping a couple of orchestration solutions, the team decided to adopt Kubernetes for its scaled IoT Platform-as-a-Service running on AWS. The platform, which also incorporated Prometheus monitoring, launched in production in 2017, serving over 3 million connected products from the get-go. Bose has since adopted a number of other CNCF technologies, including Fluentd, CoreDNS, Jaeger, and OpenTracing.

-
-
-
-
-

Challenge

- A household name in high-quality audio equipment, Bose has offered connected products for more than five years, and as that demand grew, the infrastructure had to change to support it. "We needed to provide a mechanism for developers to rapidly prototype and deploy services all the way to production pretty fast,” says Lead Cloud Engineer Josh West. In 2016, the company decided to start building a platform from scratch. The primary goal: "To be one to two steps ahead of the different product groups so that we are never scrambling to catch up with their scale,” says Cloud Architecture Manager Dylan O’Mahony. -

-

Solution

- From the beginning, the team knew it wanted a microservices architecture. After evaluating and prototyping a couple of orchestration solutions, the team decided to adopt Kubernetes for its scaled IoT Platform-as-a-Service running on AWS. The platform, which also incorporated Prometheus monitoring, launched in production in 2017, serving over 3 million connected products from the get-go. Bose has since adopted a number of other CNCF technologies, including Fluentd, CoreDNS, Jaeger, and OpenTracing. -

Impact

- With about 100 engineers onboarded, the platform is now enabling 30,000 non-production deployments across dozens of microservices per year. In 2018, there were 1250+ production deployments. Just one production cluster holds 1,800 namespaces and 340 worker nodes. "We had a brand new service taken from concept through coding and deployment all the way to production, including hardening, security testing and so forth, in less than two and a half weeks,” says O’Mahony. -
-
-
-
-
- "At Bose we’re building an IoT platform that has enabled our physical products. If it weren’t for Kubernetes and the rest of the CNCF projects being free open source software with such a strong community, we would never have achieved scale, or even gotten to launch on schedule." -

- Josh West, Lead Cloud Engineer, Bose -
-
-
-
-

A household name in high-quality audio equipment, Bose has offered connected products for more than five years, and as that demand grew, the infrastructure had to change to support it.

- "We needed to provide a mechanism for developers to rapidly prototype and deploy services all the way to production pretty fast,” says Lead Cloud Engineer Josh West. "There were a lot of cloud capabilities we wanted to provide to support our audio equipment and experiences.”

-In 2016, the company decided to start building an IoT platform from scratch. The primary goal: "To be one to two steps ahead of the different product groups so that we are never scrambling to catch up with their scale,” says Cloud Architecture Manager Dylan O’Mahony. "If they release a new connected product, we want to be already well ahead of being able to handle whatever scale that they’re going to throw at us.”

-From the beginning, the team knew it wanted a microservices architecture and platform as a service. After evaluating and prototyping orchestration solutions, including Mesos and Docker Swarm, the team decided to adopt Kubernetes for its platform running on AWS. Kubernetes was still in 1.5, but already the technology could do much of what the team wanted and needed for the present and the future. For West, that meant having storage and network handled. O’Mahony points to Kubernetes’ portability in case Bose decides to go multi-cloud.

-"Bose is a company that looks out for the long term,” says West. "Going with a quick commercial off-the-shelf solution might’ve worked for that point in time, but it would not have carried us forward, which is what we needed from Kubernetes and the CNCF.” +

With about 100 engineers onboarded, the platform is now enabling 30,000 non-production deployments across dozens of microservices per year. In 2018, there were 1250+ production deployments. Just one production cluster holds 1,800 namespaces and 340 worker nodes. "We had a brand new service taken from concept through coding and deployment all the way to production, including hardening, security testing and so forth, in less than two and a half weeks," says O'Mahony.

+{{< case-studies/quote author="Josh West, Lead Cloud Engineer, Bose" >}} +"At Bose we're building an IoT platform that has enabled our physical products. If it weren't for Kubernetes and the rest of the CNCF projects being free open source software with such a strong community, we would never have achieved scale, or even gotten to launch on schedule." +{{< /case-studies/quote >}} -
-
-
-
- "Everybody on the team thinks in terms of automation, leaning out the processes, getting things done as quickly as possible. When you step back and look at what it means for a 50-plus-year-old speaker company to have that sort of culture, it really is quite incredible, and I think the tools that we use and the foundation that we’ve built with them is a huge piece of that."

- Dylan O’Mahony, Cloud Architecture Manager, Bose +{{< case-studies/lead >}} +A household name in high-quality audio equipment, Bose has offered connected products for more than five years, and as that demand grew, the infrastructure had to change to support it. +{{< /case-studies/lead >}} -
-
-
-
- The team spent time working on choosing tooling to make the experience easier for developers. "Our developers interact with tools provided by our Ops team, and the Ops team run all of their tooling on top of Kubernetes,” says O’Mahony. "We try not to make direct Kubernetes access the only way. In fact, ideally, our developers wouldn’t even need to know that they’re running on Kubernetes.”

- The platform, which also incorporated Prometheus monitoring from the beginning, backdoored its way into production in 2017, serving over 3 million connected products from the get-go. "Even though the speakers and the products that we were designing this platform for were still quite a ways away from being launched, we did have some connected speakers on the market,” says O’Mahony. "We basically started to point certain features of those speakers and the apps that go with those speakers to this platform.”

- Today, just one of Bose’s production clusters holds 1,800 namespaces/discrete services and 340 nodes. With about 100 engineers now onboarded, the platform infrastructure is now enabling 30,000 non-production deployments across dozens of microservices per year. In 2018, there were 1250+ production deployments.. It’s a staggering improvement over some of Bose’s previous deployment processes, which supported far fewer deployments and services. +

"We needed to provide a mechanism for developers to rapidly prototype and deploy services all the way to production pretty fast," says Lead Cloud Engineer Josh West. "There were a lot of cloud capabilities we wanted to provide to support our audio equipment and experiences."

-
-
-
-
- "The CNCF Landscape quickly explains what’s going on in all the different areas from storage to cloud providers to automation and so forth. This is our shopping cart to build a cloud infrastructure. We can go choose from the different aisles."

- Josh West, Lead Cloud Engineer, Bose -
-
+

In 2016, the company decided to start building an IoT platform from scratch. The primary goal: "To be one to two steps ahead of the different product groups so that we are never scrambling to catch up with their scale," says Cloud Architecture Manager Dylan O'Mahony. "If they release a new connected product, we want to be already well ahead of being able to handle whatever scale that they're going to throw at us."

-
-
- "We had a brand new service deployed from concept through coding and deployment all the way to production, including hardening, security testing and so forth, in less than two and a half weeks,” says O’Mahony. "Everybody thinks in terms of automation, leaning out the processes, getting things done as quickly as possible. When you step back and look at what it means for a 50-plus-year-old speaker company to have that sort of culture, it really is quite incredible, and I think the tools that we use and the foundation that we’ve built is a huge piece of that.”

- Many of those technologies—such as Fluentd, CoreDNS, Jaeger, and OpenTracing—come from the CNCF Landscape, which West and O’Mahony have relied upon throughout Bose’s cloud native journey. "The CNCF Landscape quickly explains what’s going on in all the different areas from storage to cloud providers to automation and so forth,” says West. "This is our shopping cart to build a cloud infrastructure. We can go choose from the different aisles.”

- And, he adds, "If it weren’t for Kubernetes and the rest of the CNCF projects being free open source software with such a strong community, we would never have achieved scale, or even gotten to launch on schedule.”

- Another benefit of going cloud native: "We are even attracting much more talent into Bose because we’re so involved with the CNCF Landscape,” says West. (Yes, they’re hiring.) "It’s just enabled so many people to do so many great things and really brought Bose into the future of cloud.” +

From the beginning, the team knew it wanted a microservices architecture and platform as a service. After evaluating and prototyping orchestration solutions, including Mesos and Docker Swarm, the team decided to adopt Kubernetes for its platform running on AWS. Kubernetes was still in 1.5, but already the technology could do much of what the team wanted and needed for the present and the future. For West, that meant having storage and network handled. O'Mahony points to Kubernetes' portability in case Bose decides to go multi-cloud.

+

"Bose is a company that looks out for the long term," says West. "Going with a quick commercial off-the-shelf solution might've worked for that point in time, but it would not have carried us forward, which is what we needed from Kubernetes and the CNCF."

-
+{{< case-studies/quote + image="/images/case-studies/bose/banner3.jpg" + author="Dylan O'Mahony, Cloud Architecture Manager, Bose" +>}} +"Everybody on the team thinks in terms of automation, leaning out the processes, getting things done as quickly as possible. When you step back and look at what it means for a 50-plus-year-old speaker company to have that sort of culture, it really is quite incredible, and I think the tools that we use and the foundation that we've built with them is a huge piece of that." +{{< /case-studies/quote >}} -
-
-"We have a lot going on to support many more of our business units at Bose in addition to the consumer electronics division, which we currently do. It’s only because of the cloud native landscape and the tools and the features that are available that we can provide such a fantastic cloud platform for all the developers and divisions that are trying to enable some pretty amazing experiences."

- Dylan O’Mahony, Cloud Architecture Manager, Bose
-
+

The team spent time working on choosing tooling to make the experience easier for developers. "Our developers interact with tools provided by our Ops team, and the Ops team run all of their tooling on top of Kubernetes," says O'Mahony. "We try not to make direct Kubernetes access the only way. In fact, ideally, our developers wouldn't even need to know that they're running on Kubernetes."

-
- In the coming year, the team wants to work on service mesh and serverless, as well as expansion around the world. "Getting our latency down by going multi-region is going to be a big focus for us,” says O’Mahony. "In order to make sure that our customers in Japan, Australia, and everywhere else are having a good experience, we want to have points of presence closer to them. It’s never been done at Bose before.”

- That won’t stop them, because the team is all about lofty goals. "We want to get to billions of connected products!” says West. "We have a lot going on to support many more of our business units at Bose in addition to the consumer electronics division, which we currently do. It’s only because of the cloud native landscape and the tools and the features that are available that we can provide such a fantastic cloud platform for all the developers and divisions that are trying to enable some pretty amazing experiences.”

- In fact, given the scale the platform is already supporting, says O’Mahony, "doing anything other than Kubernetes, I think, would be folly at this point.” +

The platform, which also incorporated Prometheus monitoring from the beginning, backdoored its way into production in 2017, serving over 3 million connected products from the get-go. "Even though the speakers and the products that we were designing this platform for were still quite a ways away from being launched, we did have some connected speakers on the market," says O'Mahony. "We basically started to point certain features of those speakers and the apps that go with those speakers to this platform."

+

Today, just one of Bose's production clusters holds 1,800 namespaces/discrete services and 340 nodes. With about 100 engineers now onboarded, the platform infrastructure is now enabling 30,000 non-production deployments across dozens of microservices per year. In 2018, there were 1250+ production deployments.. It's a staggering improvement over some of Bose's previous deployment processes, which supported far fewer deployments and services.

-
-
- - +{{< case-studies/quote + image="/images/case-studies/bose/banner4.jpg" + author="Josh West, Lead Cloud Engineer, Bose" +>}} +"The CNCF Landscape quickly explains what's going on in all the different areas from storage to cloud providers to automation and so forth. This is our shopping cart to build a cloud infrastructure. We can go choose from the different aisles." +{{< /case-studies/quote >}} + +

"We had a brand new service deployed from concept through coding and deployment all the way to production, including hardening, security testing and so forth, in less than two and a half weeks," says O'Mahony. "Everybody thinks in terms of automation, leaning out the processes, getting things done as quickly as possible. When you step back and look at what it means for a 50-plus-year-old speaker company to have that sort of culture, it really is quite incredible, and I think the tools that we use and the foundation that we've built is a huge piece of that."

+ +

Many of those technologies—such as Fluentd, CoreDNS, Jaeger, and OpenTracing—come from the CNCF Landscape, which West and O'Mahony have relied upon throughout Bose's cloud native journey. "The CNCF Landscape quickly explains what's going on in all the different areas from storage to cloud providers to automation and so forth," says West. "This is our shopping cart to build a cloud infrastructure. We can go choose from the different aisles."

+ +

And, he adds, "If it weren't for Kubernetes and the rest of the CNCF projects being free open source software with such a strong community, we would never have achieved scale, or even gotten to launch on schedule."

+ +

Another benefit of going cloud native: "We are even attracting much more talent into Bose because we're so involved with the CNCF Landscape," says West. (Yes, they're hiring.) "It's just enabled so many people to do so many great things and really brought Bose into the future of cloud."

+ +{{< case-studies/quote author="Dylan O'Mahony, Cloud Architecture Manager, Bose" >}} +"We have a lot going on to support many more of our business units at Bose in addition to the consumer electronics division, which we currently do. It's only because of the cloud native landscape and the tools and the features that are available that we can provide such a fantastic cloud platform for all the developers and divisions that are trying to enable some pretty amazing experiences." +{{< /case-studies/quote >}} + +

In the coming year, the team wants to work on service mesh and serverless, as well as expansion around the world. "Getting our latency down by going multi-region is going to be a big focus for us," says O'Mahony. "In order to make sure that our customers in Japan, Australia, and everywhere else are having a good experience, we want to have points of presence closer to them. It's never been done at Bose before."

+ +

That won't stop them, because the team is all about lofty goals. "We want to get to billions of connected products!" says West. "We have a lot going on to support many more of our business units at Bose in addition to the consumer electronics division, which we currently do. It's only because of the cloud native landscape and the tools and the features that are available that we can provide such a fantastic cloud platform for all the developers and divisions that are trying to enable some pretty amazing experiences."

+ +

In fact, given the scale the platform is already supporting, says O'Mahony, "doing anything other than Kubernetes, I think, would be folly at this point."

diff --git a/content/en/case-studies/box/index.html b/content/en/case-studies/box/index.html index bead8eb01a..058ff7f9a2 100644 --- a/content/en/case-studies/box/index.html +++ b/content/en/case-studies/box/index.html @@ -2,113 +2,99 @@ title: Box Case Study case_study_styles: true cid: caseStudies -css: /css/style_box.css video: https://www.youtube.com/embed/of45hYbkIZs?autoplay=1 quote: > Kubernetes has the opportunity to be the new cloud platform. The amount of innovation that's going to come from being able to standardize on Kubernetes as a platform is incredibly exciting - more exciting than anything I've seen in the last 10 years of working on the cloud. +new_case_study_styles: true +heading_background: /images/case-studies/box/banner1.jpg +heading_title_logo: /images/box_logo.png +subheading: > + An Early Adopter Envisions a New Cloud Platform +case_study_details: + - Company: Box + - Location: Redwood City, California + - Industry: Technology --- -
-

CASE STUDY:
-
An Early Adopter Envisions - a New Cloud Platform
-

-
+

Challenge

+

Founded in 2005, the enterprise content management company allows its more than 50 million users to manage content in the cloud. Box was built primarily with bare metal inside the company's own data centers, with a monolithic PHP code base. As the company was expanding globally, it needed to focus on "how we run our workload across many different cloud infrastructures from bare metal to public cloud," says Sam Ghods, Cofounder and Services Architect of Box. "It's been a huge challenge because of different clouds, especially bare metal, have very different interfaces."

-
- Company  Box     Location  Redwood City, California     Industry  Technology -
+

Solution

-
+

Over the past couple of years, Box has been decomposing its infrastructure into microservices, and became an early adopter of, as well as contributor to, Kubernetes container orchestration. Kubernetes, Ghods says, has allowed Box's developers to "target a universal set of concepts that are portable across all clouds."

-
+

Impact

-
-
+

"Before Kubernetes," Ghods says, "our infrastructure was so antiquated it was taking us more than six months to deploy a new microservice. Today, a new microservice takes less than five days to deploy. And we're working on getting it to an hour."

-

Challenge

- Founded in 2005, the enterprise content management company allows its more than 50 million users to manage content in the cloud. Box was built primarily with bare metal inside the company’s own data centers, with a monolithic PHP code base. As the company was expanding globally, it needed to focus on "how we run our workload across many different cloud infrastructures from bare metal to public cloud," says Sam Ghods, Cofounder and Services Architect of Box. "It’s been a huge challenge because of different clouds, especially bare metal, have very different interfaces." -
-
+{{< case-studies/quote author="SAM GHOUDS, CO-FOUNDER AND SERVICES ARCHITECT OF BOX" >}} +"We looked at a lot of different options, but Kubernetes really stood out....the fact that on day one it was designed to run on bare metal just as well as Google Cloud meant that we could actually migrate to it inside of our data centers, and then use those same tools and concepts to run across public cloud providers as well." +{{< /case-studies/quote >}} -
-

Solution

- Over the past couple of years, Box has been decomposing its infrastructure into microservices, and became an early adopter of, as well as contributor to, Kubernetes container orchestration. Kubernetes, Ghods says, has allowed Box’s developers to "target a universal set of concepts that are portable across all clouds."

+{{< case-studies/lead >}} +In the summer of 2014, Box was feeling the pain of a decade's worth of hardware and software infrastructure that wasn't keeping up with the company's needs. +{{< /case-studies/lead >}} -

Impact

- "Before Kubernetes," Ghods says, "our infrastructure was so antiquated it was taking us more than six months to deploy a new microservice. Today, a new microservice takes less than five days to deploy. And we’re working on getting it to an hour." -
-
+

A platform that allows its more than 50 million users (including governments and big businesses like General Electric) to manage and share content in the cloud, Box was originally a PHP monolith of millions of lines of code built exclusively with bare metal inside of its own data centers. It had already begun to slowly chip away at the monolith, decomposing it into microservices. And "as we've been expanding into regions around the globe, and as the public cloud wars have been heating up, we've been focusing a lot more on figuring out how we run our workload across many different environments and many different cloud infrastructure providers," says Box Cofounder and Services Architect Sam Ghods. "It's been a huge challenge thus far because of all these different providers, especially bare metal, have very different interfaces and ways in which you work with them."

-
+

Box's cloud native journey accelerated that June, when Ghods attended DockerCon. The company had come to the realization that it could no longer run its applications only off bare metal, and was researching containerizing with Docker, virtualizing with OpenStack, and supporting public cloud.

-
-
- "We looked at a lot of different options, but Kubernetes really stood out....the fact that on day one it was designed to run on bare metal just as well as Google Cloud meant that we could actually migrate to it inside of our data centers, and then use those same tools and concepts to run across public cloud providers as well."

- SAM GHOUDS, CO-FOUNDER AND SERVICES ARCHITECT OF BOX -
-
+

At that conference, Google announced the release of its Kubernetes container management system, and Ghods was won over. "We looked at a lot of different options, but Kubernetes really stood out, especially because of the incredibly strong team of Borg veterans and the vision of having a completely infrastructure-agnostic way of being able to run cloud software," he says, referencing Google's internal container orchestrator Borg. "The fact that on day one it was designed to run on bare metal just as well as Google Cloud meant that we could actually migrate to it inside of our data centers, and then use those same tools and concepts to run across public cloud providers as well."

-
+

Another plus: Ghods liked that Kubernetes has a universal set of API objects like pod, service, replica set and deployment object, which created a consistent surface to build tooling against. "Even PaaS layers like OpenShift or Deis that build on top of Kubernetes still treat those objects as first-class principles," he says. "We were excited about having these abstractions shared across the entire ecosystem, which would result in a lot more momentum than we saw in other potential solutions."

-
-

In the summer of 2014, Box was feeling the pain of a decade’s worth of hardware and software infrastructure that wasn’t keeping up with the company’s needs.

+

Box deployed Kubernetes in a cluster in a production data center just six months later. Kubernetes was then still pre-beta, on version 0.11. They started small: The very first thing Ghods's team ran on Kubernetes was a Box API checker that confirms Box is up. "That was just to write and deploy some software to get the whole pipeline functioning," he says. Next came some daemons that process jobs, which was "nice and safe because if they experienced any interruptions, we wouldn't fail synchronous incoming requests from customers."

- A platform that allows its more than 50 million users (including governments and big businesses like General Electric) to manage and share content in the cloud, Box was originally a PHP monolith of millions of lines of code built exclusively with bare metal inside of its own data centers. It had already begun to slowly chip away at the monolith, decomposing it into microservices. And "as we’ve been expanding into regions around the globe, and as the public cloud wars have been heating up, we’ve been focusing a lot more on figuring out how we run our workload across many different environments and many different cloud infrastructure providers," says Box Cofounder and Services Architect Sam Ghods. "It’s been a huge challenge thus far because of all these different providers, especially bare metal, have very different interfaces and ways in which you work with them."

- Box’s cloud native journey accelerated that June, when Ghods attended DockerCon. The company had come to the realization that it could no longer run its applications only off bare metal, and was researching containerizing with Docker, virtualizing with OpenStack, and supporting public cloud.

- At that conference, Google announced the release of its Kubernetes container management system, and Ghods was won over. "We looked at a lot of different options, but Kubernetes really stood out, especially because of the incredibly strong team of Borg veterans and the vision of having a completely infrastructure-agnostic way of being able to run cloud software," he says, referencing Google’s internal container orchestrator Borg. "The fact that on day one it was designed to run on bare metal just as well as Google Cloud meant that we could actually migrate to it inside of our data centers, and then use those same tools and concepts to run across public cloud providers as well."

- Another plus: Ghods liked that Kubernetes has a universal set of API objects like pod, service, replica set and deployment object, which created a consistent surface to build tooling against. "Even PaaS layers like OpenShift or Deis that build on top of Kubernetes still treat those objects as first-class principles," he says. "We were excited about having these abstractions shared across the entire ecosystem, which would result in a lot more momentum than we saw in other potential solutions."

- Box deployed Kubernetes in a cluster in a production data center just six months later. Kubernetes was then still pre-beta, on version 0.11. They started small: The very first thing Ghods’s team ran on Kubernetes was a Box API checker that confirms Box is up. "That was just to write and deploy some software to get the whole pipeline functioning," he says. Next came some daemons that process jobs, which was "nice and safe because if they experienced any interruptions, we wouldn’t fail synchronous incoming requests from customers." +{{< case-studies/quote image="/images/case-studies/box/banner3.jpg">}} +"As we've been expanding into regions around the globe, and as the public cloud wars have been heating up, we've been focusing a lot more on figuring out how we [can have Kubernetes help] run our workload across many different environments and many different cloud infrastructure providers." +{{< /case-studies/quote >}} -
-
+

The first live service, which the team could route to and ask for information, was launched a few months later. At that point, Ghods says, "We were comfortable with the stability of the Kubernetes cluster. We started to port some services over, then we would increase the cluster size and port a few more, and that's ended up to about 100 servers in each data center that are dedicated purely to Kubernetes. And that's going to be expanding a lot over the next 12 months, probably too many hundreds if not thousands."

-
-
- "As we’ve been expanding into regions around the globe, and as the public cloud wars have been heating up, we’ve been focusing a lot more on figuring out how we [can have Kubernetes help] run our workload across many different environments and many different cloud infrastructure providers." -
-
+

While observing teams who began to use Kubernetes for their microservices, "we immediately saw an uptick in the number of microservices being released," Ghods notes. "There was clearly a pent-up demand for a better way of building software through microservices, and the increase in agility helped our developers be more productive and make better architectural choices."

-
-
- The first live service, which the team could route to and ask for information, was launched a few months later. At that point, Ghods says, "We were comfortable with the stability of the Kubernetes cluster. We started to port some services over, then we would increase the cluster size and port a few more, and that’s ended up to about 100 servers in each data center that are dedicated purely to Kubernetes. And that’s going to be expanding a lot over the next 12 months, probably too many hundreds if not thousands."

- While observing teams who began to use Kubernetes for their microservices, "we immediately saw an uptick in the number of microservices being released," Ghods notes. "There was clearly a pent-up demand for a better way of building software through microservices, and the increase in agility helped our developers be more productive and make better architectural choices." -

"There was clearly a pent-up demand for a better way of building software through microservices, and the increase in agility helped our developers be more productive and make better architectural choices."

- Ghods reflects that as early adopters, Box had a different journey from what companies experience now. "We were definitely lock step with waiting for certain things to stabilize or features to get released," he says. "In the early days we were doing a lot of contributions [to components such as kubectl apply] and waiting for Kubernetes to release each of them, and then we’d upgrade, contribute more, and go back and forth several times. The entire project took about 18 months from our first real deployment on Kubernetes to having general availability. If we did that exact same thing today, it would probably be no more than six."

- In any case, Box didn’t have to make too many modifications to Kubernetes for it to work for the company. "The vast majority of the work our team has done to implement Kubernetes at Box has been making it work inside of our existing (and often legacy) infrastructure," says Ghods, "such as upgrading our base operating system from RHEL6 to RHEL7 or integrating it into Nagios, our monitoring infrastructure. But overall Kubernetes has been remarkably flexible with fitting into many of our constraints, and we’ve been running it very successfully on our bare metal infrastructure."

- Perhaps the bigger challenge for Box was a cultural one. "Kubernetes, and cloud native in general, represents a pretty big paradigm shift, and it’s not very incremental," Ghods says. "We’re essentially making this pitch that Kubernetes is going to solve everything because it does things the right way and everything is just suddenly better. But it’s important to keep in mind that it’s not nearly as proven as many other solutions out there. You can’t say how long this or that company took to do it because there just aren’t that many yet. Our team had to really fight for resources because our project was a bit of a moonshot." -
-
+{{< case-studies/lead >}} +"There was clearly a pent-up demand for a better way of building software through microservices, and the increase in agility helped our developers be more productive and make better architectural choices." +{{< /case-studies/lead >}} -
-
- "The vast majority of the work our team has done to implement Kubernetes at Box has been making it work inside of our existing [and often legacy] infrastructure....overall Kubernetes has been remarkably flexible with fitting into many of our constraints, and we’ve been running it very successfully on our bare metal infrastructure." -
-
+

Ghods reflects that as early adopters, Box had a different journey from what companies experience now. "We were definitely lock step with waiting for certain things to stabilize or features to get released," he says. "In the early days we were doing a lot of contributions [to components such as kubectl apply] and waiting for Kubernetes to release each of them, and then we'd upgrade, contribute more, and go back and forth several times. The entire project took about 18 months from our first real deployment on Kubernetes to having general availability. If we did that exact same thing today, it would probably be no more than six."

-
-
- Having learned from experience, Ghods offers these two pieces of advice for companies going through similar challenges: -

1. Deliver early and often.

Service discovery was a huge problem for Box, and the team had to decide whether to build an interim solution or wait for Kubernetes to natively satisfy Box’s unique requirements. After much debate, "we just started focusing on delivering something that works, and then dealing with potentially migrating to a more native solution later," Ghods says. "The above-all-else target for the team should always be to serve real production use cases on the infrastructure, no matter how trivial. This helps keep the momentum going both for the team itself and for the organizational perception of the project."

-

2. Keep an open mind about what your company has to abstract away from developers and what it doesn’t.

Early on, the team built an abstraction on top of Docker files to help ensure that images had the right security updates. - This turned out to be superfluous work, since container images are considered immutable and you can easily scan them post-build to ensure they do not contain vulnerabilities. Because managing infrastructure through containerization is such a discontinuous leap, it’s better to start by interacting directly with the native tools and learning their unique advantages and caveats. An abstraction should be built only after a practical need for it arises.

- In the end, the impact has been powerful. "Before Kubernetes," Ghods says, "our infrastructure was so antiquated it was taking us more than six months to deploy a new microservice. Now a new microservice takes less than five days to deploy. And we’re working on getting it to an hour. Granted, much of that six months was due to how broken our systems were, but bare metal is intrinsically a difficult platform to support unless you have a system like Kubernetes to help manage it."

- By Ghods’s estimate, Box is still several years away from his goal of being a 90-plus percent Kubernetes shop. "We’re very far along on having a mission-critical, stable Kubernetes deployment that provides a lot of value," he says. "Right now about five percent of all of our compute runs on Kubernetes, and I think in the next six months we’ll likely be between 20 to 50 percent. We’re working hard on enabling all stateless service use cases, and shift our focus to stateful services after that." -
-
+

In any case, Box didn't have to make too many modifications to Kubernetes for it to work for the company. "The vast majority of the work our team has done to implement Kubernetes at Box has been making it work inside of our existing (and often legacy) infrastructure," says Ghods, "such as upgrading our base operating system from RHEL6 to RHEL7 or integrating it into Nagios, our monitoring infrastructure. But overall Kubernetes has been remarkably flexible with fitting into many of our constraints, and we've been running it very successfully on our bare metal infrastructure."

-
-
- "Ghods predicts that Kubernetes has the opportunity to be the new cloud platform. '...because it’s a never-before-seen level of automation and intelligence surrounding infrastructure that is portable and agnostic to every way you can run your infrastructure.'" -
-
+

Perhaps the bigger challenge for Box was a cultural one. "Kubernetes, and cloud native in general, represents a pretty big paradigm shift, and it's not very incremental," Ghods says. "We're essentially making this pitch that Kubernetes is going to solve everything because it does things the right way and everything is just suddenly better. But it's important to keep in mind that it's not nearly as proven as many other solutions out there. You can't say how long this or that company took to do it because there just aren't that many yet. Our team had to really fight for resources because our project was a bit of a moonshot."

-
-
- In fact, that’s what he envisions across the industry: Ghods predicts that Kubernetes has the opportunity to be the new cloud platform. Kubernetes provides an API consistent across different cloud platforms including bare metal, and "I don’t think people have seen the full potential of what’s possible when you can program against one single interface," he says. "The same way AWS changed infrastructure so that you don’t have to think about servers or cabinets or networking equipment anymore, Kubernetes enables you to focus exclusively on the containers that you’re running, which is pretty exciting. That’s the vision."

- Ghods points to projects that are already in development or recently released for Kubernetes as a cloud platform: cluster federation, the Dashboard UI, and CoreOS’s etcd operator. "I honestly believe it’s the most exciting thing I’ve seen in cloud infrastructure," he says, "because it’s a never-before-seen level of automation and intelligence surrounding infrastructure that is portable and agnostic to every way you can run your infrastructure."

- Box, with its early decision to use bare metal, embarked on its Kubernetes journey out of necessity. But Ghods says that even if companies don’t have to be agnostic about cloud providers today, Kubernetes may soon become the industry standard, as more and more tooling and extensions are built around the API.

- "The same way it doesn’t make sense to deviate from Linux because it’s such a standard," Ghods says, "I think Kubernetes is going down the same path. It is still early days—the documentation still needs work and the user experience for writing and publishing specs to the Kubernetes clusters is still rough. When you’re on the cutting edge you can expect to bleed a little. But the bottom line is, this is where the industry is going. Three to five years from now it’s really going to be shocking if you run your infrastructure any other way." -
-
+{{< case-studies/quote image="/images/case-studies/box/banner4.jpg">}} +"The vast majority of the work our team has done to implement Kubernetes at Box has been making it work inside of our existing [and often legacy] infrastructure....overall Kubernetes has been remarkably flexible with fitting into many of our constraints, and we've been running it very successfully on our bare metal infrastructure." +{{< /case-studies/quote >}} + +

Having learned from experience, Ghods offers these two pieces of advice for companies going through similar challenges:

+ +{{< case-studies/lead >}} +1. Deliver early and often. +{{< /case-studies/lead >}} + +

Service discovery was a huge problem for Box, and the team had to decide whether to build an interim solution or wait for Kubernetes to natively satisfy Box's unique requirements. After much debate, "we just started focusing on delivering something that works, and then dealing with potentially migrating to a more native solution later," Ghods says. "The above-all-else target for the team should always be to serve real production use cases on the infrastructure, no matter how trivial. This helps keep the momentum going both for the team itself and for the organizational perception of the project."

+ +{{< case-studies/lead >}} +2. Keep an open mind about what your company has to abstract away from developers and what it doesn't. +{{< /case-studies/lead >}} + +

Early on, the team built an abstraction on top of Docker files to help ensure that images had the right security updates. This turned out to be superfluous work, since container images are considered immutable and you can easily scan them post-build to ensure they do not contain vulnerabilities. Because managing infrastructure through containerization is such a discontinuous leap, it's better to start by interacting directly with the native tools and learning their unique advantages and caveats. An abstraction should be built only after a practical need for it arises.

+ +

In the end, the impact has been powerful. "Before Kubernetes," Ghods says, "our infrastructure was so antiquated it was taking us more than six months to deploy a new microservice. Now a new microservice takes less than five days to deploy. And we're working on getting it to an hour. Granted, much of that six months was due to how broken our systems were, but bare metal is intrinsically a difficult platform to support unless you have a system like Kubernetes to help manage it."

+ +

By Ghods's estimate, Box is still several years away from his goal of being a 90-plus percent Kubernetes shop. "We're very far along on having a mission-critical, stable Kubernetes deployment that provides a lot of value," he says. "Right now about five percent of all of our compute runs on Kubernetes, and I think in the next six months we'll likely be between 20 to 50 percent. We're working hard on enabling all stateless service use cases, and shift our focus to stateful services after that."

+ +{{< case-studies/quote >}} +"Ghods predicts that Kubernetes has the opportunity to be the new cloud platform. '...because it's a never-before-seen level of automation and intelligence surrounding infrastructure that is portable and agnostic to every way you can run your infrastructure.'" +{{< /case-studies/quote >}} + +

In fact, that's what he envisions across the industry: Ghods predicts that Kubernetes has the opportunity to be the new cloud platform. Kubernetes provides an API consistent across different cloud platforms including bare metal, and "I don't think people have seen the full potential of what's possible when you can program against one single interface," he says. "The same way AWS changed infrastructure so that you don't have to think about servers or cabinets or networking equipment anymore, Kubernetes enables you to focus exclusively on the containers that you're running, which is pretty exciting. That's the vision."

+ +

Ghods points to projects that are already in development or recently released for Kubernetes as a cloud platform: cluster federation, the Dashboard UI, and CoreOS's etcd operator. "I honestly believe it's the most exciting thing I've seen in cloud infrastructure," he says, "because it's a never-before-seen level of automation and intelligence surrounding infrastructure that is portable and agnostic to every way you can run your infrastructure."

+ +

Box, with its early decision to use bare metal, embarked on its Kubernetes journey out of necessity. But Ghods says that even if companies don't have to be agnostic about cloud providers today, Kubernetes may soon become the industry standard, as more and more tooling and extensions are built around the API.

+ +

"The same way it doesn't make sense to deviate from Linux because it's such a standard," Ghods says, "I think Kubernetes is going down the same path. It is still early days—the documentation still needs work and the user experience for writing and publishing specs to the Kubernetes clusters is still rough. When you're on the cutting edge you can expect to bleed a little. But the bottom line is, this is where the industry is going. Three to five years from now it's really going to be shocking if you run your infrastructure any other way."

diff --git a/content/en/case-studies/buffer/index.html b/content/en/case-studies/buffer/index.html index 333db6a74e..bcf0896445 100644 --- a/content/en/case-studies/buffer/index.html +++ b/content/en/case-studies/buffer/index.html @@ -1,112 +1,83 @@ --- title: Buffer Case Study - case_study_styles: true cid: caseStudies -css: /css/style_buffer.css + +new_case_study_styles: true +heading_background: /images/case-studies/buffer/banner3.jpg +heading_title_logo: /images/buffer.png +subheading: > + Making Deployments Easy for a Small, Distributed Team +case_study_details: + - Company: Buffer + - Location: Around the World + - Industry: Social Media Technology --- -
-

CASE STUDY:
-
Making Deployments Easy for a Small, Distributed Team
-

-
+

Challenge

-
- Company Buffer     Location  Around the World     Industry  Social Media Technology -
+

With a small but fully distributed team of 80 working across almost a dozen time zones, Buffer—which offers social media management to agencies and marketers—was looking to solve its "classic monolithic code base problem," says Architect Dan Farrelly. "We wanted to have the kind of liquid infrastructure where a developer could create an app and deploy it and scale it horizontally as necessary."

-
+

Solution

-
-
-
+

Embracing containerization, Buffer moved its infrastructure from Amazon Web Services' Elastic Beanstalk to Docker on AWS, orchestrated with Kubernetes.

-

Challenge

- With a small but fully distributed team of 80 working across almost a dozen time zones, Buffer—which offers social media management to agencies and marketers—was looking to solve its "classic monolithic code base problem," says Architect Dan Farrelly. "We wanted to have the kind of liquid infrastructure where a developer could create an app and deploy it and scale it horizontally as necessary." -
- -
-

Solution

- Embracing containerization, Buffer moved its infrastructure from Amazon Web Services’ Elastic Beanstalk to Docker on AWS, orchestrated with Kubernetes. -
-

Impact

- The new system "leveled up our ability with deployment and rolling out new changes," says Farrelly. "Building something on your computer and knowing that it’s going to work has shortened things up a lot. Our feedback cycles are a lot faster now too." -
-
-
-
-
- "It’s amazing that we can use the Kubernetes solution off the shelf with our team. And it just keeps getting better. Before we even know that we need something, it’s there in the next release or it’s coming in the next few months."

- DAN FARRELLY, BUFFER ARCHITECT -
-
+

The new system "leveled up our ability with deployment and rolling out new changes," says Farrelly. "Building something on your computer and knowing that it's going to work has shortened things up a lot. Our feedback cycles are a lot faster now too."

-
-
-

Dan Farrelly uses a carpentry analogy to explain the problem his company, Buffer, began having as its team of developers grew over the past few years.

+{{< case-studies/quote author="DAN FARRELLY, BUFFER ARCHITECT" >}} +"It's amazing that we can use the Kubernetes solution off the shelf with our team. And it just keeps getting better. Before we even know that we need something, it's there in the next release or it's coming in the next few months." +{{< /case-studies/quote >}} - "If you’re building a table by yourself, it’s fine," the company’s architect says. "If you bring in a second person to work on the table, maybe that person can start sanding the legs while you’re sanding the top. But when you bring a third or fourth person in, someone should probably work on a different table." Needing to work on more and more different tables led Buffer on a path toward microservices and containerization made possible by Kubernetes.

- Since around 2012, Buffer had already been using Elastic Beanstalk, the orchestration service for deploying infrastructure offered by Amazon Web Services. "We were deploying a single monolithic PHP application, and it was the same application across five or six environments," says Farrelly. "We were very much a product-driven company. It was all about shipping new features quickly and getting things out the door, and if something was not broken, we didn’t spend too much time on it. If things were getting a little bit slow, we’d maybe use a faster server or just scale up one instance, and it would be good enough. We’d move on."

- But things came to a head in 2016. With the growing number of committers on staff, Farrelly and Buffer’s then-CTO, Sunil Sadasivan, decided it was time to re-architect and rethink their infrastructure. "It was a classic monolithic code base problem," says Farrelly.

Some of the company’s team was already successfully using Docker in their development environment, but the only application running on Docker in production was a marketing website that didn’t see real user traffic. They wanted to go further with Docker, and the next step was looking at options for orchestration. -
-
+{{< case-studies/lead >}} +Dan Farrelly uses a carpentry analogy to explain the problem his company, Buffer, began having as its team of developers grew over the past few years. +{{< /case-studies/lead >}} -
-
- And all the things Kubernetes did well suited Buffer’s needs. "We wanted to have the kind of liquid infrastructure where a developer could create an app and deploy it and scale it horizontally as necessary," says Farrelly. "We quickly used some scripts to set up a couple of test clusters, we built some small proof-of-concept applications in containers, and we deployed things within an hour. We had very little experience in running containers in production. It was amazing how quickly we could get a handle on it [Kubernetes]." -
-
+

"If you're building a table by yourself, it's fine," the company's architect says. "If you bring in a second person to work on the table, maybe that person can start sanding the legs while you're sanding the top. But when you bring a third or fourth person in, someone should probably work on a different table." Needing to work on more and more different tables led Buffer on a path toward microservices and containerization made possible by Kubernetes.

-
-
- First they considered Mesosphere, DC/OS and Amazon Elastic Container Service (which their data systems team was already using for some data pipeline jobs). While they were impressed by these offerings, they ultimately went with Kubernetes. "We run on AWS still, so spinning up, creating services and creating load balancers on demand for us without having to configure them manually was a great way for our team to get into this," says Farrelly. "We didn’t need to figure out how to configure this or that, especially coming from a former Elastic Beanstalk environment that gave us an automatically-configured load balancer. I really liked Kubernetes’ controls of the command line. It just took care of ports. It was a lot more flexible. Kubernetes was designed for doing what it does, so it does it very well."

- And all the things Kubernetes did well suited Buffer’s needs. "We wanted to have the kind of liquid infrastructure where a developer could create an app and deploy it and scale it horizontally as necessary," says Farrelly. "We quickly used some scripts to set up a couple of test clusters, we built some small proof-of-concept applications in containers, and we deployed things within an hour. We had very little experience in running containers in production. It was amazing how quickly we could get a handle on it [Kubernetes]."

- Above all, it provided a powerful solution for one of the company’s most distinguishing characteristics: their remote team that’s spread across a dozen different time zones. "The people with deep knowledge of our infrastructure live in time zones different from our peak traffic time zones, and most of our product engineers live in other places," says Farrelly. "So we really wanted something where anybody could get a grasp of the system early on and utilize it, and not have to worry that the deploy engineer is asleep. Otherwise people would sit around for 12 to 24 hours for something. It’s been really cool to see people moving much faster." -

- With a relatively small engineering team—just 25 people, and only a handful working on infrastructure, with the majority front-end developers—Buffer needed "something robust for them to deploy whatever they wanted," says Farrelly. Before, "it was only a couple of people who knew how to set up everything in the old way. With this system, it was easy to review documentation and get something out extremely quickly. It lowers the bar for us to get everything in production. We don't have the big team to build all these tools or manage the infrastructure like other larger companies might." -
-
+

Since around 2012, Buffer had already been using Elastic Beanstalk, the orchestration service for deploying infrastructure offered by Amazon Web Services. "We were deploying a single monolithic PHP application, and it was the same application across five or six environments," says Farrelly. "We were very much a product-driven company. It was all about shipping new features quickly and getting things out the door, and if something was not broken, we didn't spend too much time on it. If things were getting a little bit slow, we'd maybe use a faster server or just scale up one instance, and it would be good enough. We'd move on."

-
-
- "In our old way of working, the feedback loop was a lot longer, and it was delicate because if you deployed something, the risk was high to potentially break something else," Farrelly says. "With the kind of deploys that we built around Kubernetes, we were able to detect bugs and fix them, and get them deployed super fast. The second someone is fixing [a bug], it’s out the door." -
-
+

But things came to a head in 2016. With the growing number of committers on staff, Farrelly and Buffer's then-CTO, Sunil Sadasivan, decided it was time to re-architect and rethink their infrastructure. "It was a classic monolithic code base problem," says Farrelly.

Some of the company's team was already successfully using Docker in their development environment, but the only application running on Docker in production was a marketing website that didn't see real user traffic. They wanted to go further with Docker, and the next step was looking at options for orchestration.

-
-
- To help with this, Buffer developers wrote a deploy bot that wraps the Kubernetes deploy process and can be used by every team. "Before, our data analysts would update, say, a Python analysis script and have to wait for the lead on that team to click the button and deploy it," Farrelly explains. "Now our data analysts can make a change, enter a Slack command, ‘/deploy,’ and it goes out instantly. They don’t need to wait on these slow turnaround times. They don’t even know where it’s running; it doesn’t matter." -

- One of the first applications the team built from scratch using Kubernetes was a new image resizing service. As a social media management tool that allows marketing teams to collaborate on posts and send updates across multiple social media profiles and networks, Buffer has to be able to resize photographs as needed to meet the varying limitations of size and format posed by different social networks. "We always had these hacked together solutions," says Farrelly. -

- To create this new service, one of the senior product engineers was assigned to learn Docker and Kubernetes, then build the service, test it, deploy it and monitor it—which he was able to do relatively quickly. "In our old way of working, the feedback loop was a lot longer, and it was delicate because if you deployed something, the risk was high to potentially break something else," Farrelly says. "With the kind of deploys that we built around Kubernetes, we were able to detect bugs and fix them, and get them deployed super fast. The second someone is fixing [a bug], it’s out the door." -

- Plus, unlike with their old system, they could scale things horizontally with one command. "As we rolled it out," Farrelly says, "we could anticipate and just click a button. This allowed us to deal with the demand that our users were placing on the system and easily scale it to handle it." -

- Another thing they weren’t able to do before was a canary deploy. This new capability "made us so much more confident in deploying big changes," says Farrelly. "Before, it took a lot of testing, which is still good, but it was also a lot of ‘fingers crossed.’ And this is something that gets run 800,000 times a day, the core of our business. If it doesn’t work, our business doesn’t work. In a Kubernetes world, I can do a canary deploy to test it for 1 percent and I can shut it down very quickly if it isn’t working. This has leveled up our ability to deploy and roll out new changes quickly while reducing risk." +{{< case-studies/quote image="/images/case-studies/buffer/banner1.jpg" >}} +And all the things Kubernetes did well suited Buffer's needs. "We wanted to have the kind of liquid infrastructure where a developer could create an app and deploy it and scale it horizontally as necessary," says Farrelly. "We quickly used some scripts to set up a couple of test clusters, we built some small proof-of-concept applications in containers, and we deployed things within an hour. We had very little experience in running containers in production. It was amazing how quickly we could get a handle on it [Kubernetes]." +{{< /case-studies/quote >}} -
-
+

First they considered Mesosphere, DC/OS and Amazon Elastic Container Service (which their data systems team was already using for some data pipeline jobs). While they were impressed by these offerings, they ultimately went with Kubernetes. "We run on AWS still, so spinning up, creating services and creating load balancers on demand for us without having to configure them manually was a great way for our team to get into this," says Farrelly. "We didn't need to figure out how to configure this or that, especially coming from a former Elastic Beanstalk environment that gave us an automatically-configured load balancer. I really liked Kubernetes' controls of the command line. It just took care of ports. It was a lot more flexible. Kubernetes was designed for doing what it does, so it does it very well."

-
-
- "If you want to run containers in production, with nearly the power that Google uses internally, this [Kubernetes] is a great way to do that," Farrelly says. "We’re a relatively small team that’s actually running Kubernetes, and we’ve never run anything like it before. So it’s more approachable than you might think. That’s the one big thing that I tell people who are experimenting with it. Pick a couple of things, roll it out, kick the tires on this for a couple of months and see how much it can handle. You start learning a lot this way." -
-
+

And all the things Kubernetes did well suited Buffer's needs. "We wanted to have the kind of liquid infrastructure where a developer could create an app and deploy it and scale it horizontally as necessary," says Farrelly. "We quickly used some scripts to set up a couple of test clusters, we built some small proof-of-concept applications in containers, and we deployed things within an hour. We had very little experience in running containers in production. It was amazing how quickly we could get a handle on it [Kubernetes]."

-
-
- By October 2016, 54 percent of Buffer’s traffic was going through their Kubernetes cluster. "There’s a lot of our legacy functionality that still runs alright, and those parts might move to Kubernetes or stay in our old setup forever," says Farrelly. But the company made the commitment at that time that going forward, "all new development, all new features, will be running on Kubernetes." -

- The plan for 2017 is to move all the legacy applications to a new Kubernetes cluster, and run everything they’ve pulled out of their old infrastructure, plus the new services they’re developing in Kubernetes, on another cluster. "I want to bring all the benefits that we’ve seen on our early services to everyone on the team," says Farrelly. -

-

For Buffer’s engineers, it’s an exciting process. "Every time we’re deploying a new service, we need to figure out: OK, what’s the architecture? How do these services communicate? What’s the best way to build this service?" Farrelly says. "And then we use the different features that Kubernetes has to glue all the pieces together. It’s enabling us to experiment as we’re learning how to design a service-oriented architecture. Before, we just wouldn’t have been able to do it. This is actually giving us a blank white board so we can do whatever we want on it." -

- Part of that blank slate is the flexibility that Kubernetes offers should the time come when Buffer may want or need to change its cloud. "It’s cloud agnostic so maybe one day we could switch to Google or somewhere else," Farrelly says. "We’re very deep in Amazon but it’s nice to know we could move away if we need to." -

- At this point, the team at Buffer can’t imagine running their infrastructure any other way—and they’re happy to spread the word. "If you want to run containers in production, with nearly the power that Google uses internally, this [Kubernetes] is a great way to do that," Farrelly says. "We’re a relatively small team that’s actually running Kubernetes, and we’ve never run anything like it before. So it’s more approachable than you might think. That’s the one big thing that I tell people who are experimenting with it. Pick a couple of things, roll it out, kick the tires on this for a couple of months and see how much it can handle. You start learning a lot this way." -

-
-
+

Above all, it provided a powerful solution for one of the company's most distinguishing characteristics: their remote team that's spread across a dozen different time zones. "The people with deep knowledge of our infrastructure live in time zones different from our peak traffic time zones, and most of our product engineers live in other places," says Farrelly. "So we really wanted something where anybody could get a grasp of the system early on and utilize it, and not have to worry that the deploy engineer is asleep. Otherwise people would sit around for 12 to 24 hours for something. It's been really cool to see people moving much faster."

+ +

With a relatively small engineering team—just 25 people, and only a handful working on infrastructure, with the majority front-end developers—Buffer needed "something robust for them to deploy whatever they wanted," says Farrelly. Before, "it was only a couple of people who knew how to set up everything in the old way. With this system, it was easy to review documentation and get something out extremely quickly. It lowers the bar for us to get everything in production. We don't have the big team to build all these tools or manage the infrastructure like other larger companies might."

+ +{{< case-studies/quote image="/images/case-studies/buffer/banner4.jpg" >}} +"In our old way of working, the feedback loop was a lot longer, and it was delicate because if you deployed something, the risk was high to potentially break something else," Farrelly says. "With the kind of deploys that we built around Kubernetes, we were able to detect bugs and fix them, and get them deployed super fast. The second someone is fixing [a bug], it's out the door." +{{< /case-studies/quote >}} + +

To help with this, Buffer developers wrote a deploy bot that wraps the Kubernetes deploy process and can be used by every team. "Before, our data analysts would update, say, a Python analysis script and have to wait for the lead on that team to click the button and deploy it," Farrelly explains. "Now our data analysts can make a change, enter a Slack command, '/deploy,' and it goes out instantly. They don't need to wait on these slow turnaround times. They don't even know where it's running; it doesn't matter."

+ +

One of the first applications the team built from scratch using Kubernetes was a new image resizing service. As a social media management tool that allows marketing teams to collaborate on posts and send updates across multiple social media profiles and networks, Buffer has to be able to resize photographs as needed to meet the varying limitations of size and format posed by different social networks. "We always had these hacked together solutions," says Farrelly.

+ +

To create this new service, one of the senior product engineers was assigned to learn Docker and Kubernetes, then build the service, test it, deploy it and monitor it—which he was able to do relatively quickly. "In our old way of working, the feedback loop was a lot longer, and it was delicate because if you deployed something, the risk was high to potentially break something else," Farrelly says. "With the kind of deploys that we built around Kubernetes, we were able to detect bugs and fix them, and get them deployed super fast. The second someone is fixing [a bug], it's out the door."

+ +

Plus, unlike with their old system, they could scale things horizontally with one command. "As we rolled it out," Farrelly says, "we could anticipate and just click a button. This allowed us to deal with the demand that our users were placing on the system and easily scale it to handle it."

+ +

Another thing they weren't able to do before was a canary deploy. This new capability "made us so much more confident in deploying big changes," says Farrelly. "Before, it took a lot of testing, which is still good, but it was also a lot of 'fingers crossed.' And this is something that gets run 800,000 times a day, the core of our business. If it doesn't work, our business doesn't work. In a Kubernetes world, I can do a canary deploy to test it for 1 percent and I can shut it down very quickly if it isn't working. This has leveled up our ability to deploy and roll out new changes quickly while reducing risk."

+ +{{< case-studies/quote >}} +"If you want to run containers in production, with nearly the power that Google uses internally, this [Kubernetes] is a great way to do that," Farrelly says. "We're a relatively small team that's actually running Kubernetes, and we've never run anything like it before. So it's more approachable than you might think. That's the one big thing that I tell people who are experimenting with it. Pick a couple of things, roll it out, kick the tires on this for a couple of months and see how much it can handle. You start learning a lot this way." +{{< /case-studies/quote >}} + +

By October 2016, 54 percent of Buffer's traffic was going through their Kubernetes cluster. "There's a lot of our legacy functionality that still runs alright, and those parts might move to Kubernetes or stay in our old setup forever," says Farrelly. But the company made the commitment at that time that going forward, "all new development, all new features, will be running on Kubernetes."

+ +

The plan for 2017 is to move all the legacy applications to a new Kubernetes cluster, and run everything they've pulled out of their old infrastructure, plus the new services they're developing in Kubernetes, on another cluster. "I want to bring all the benefits that we've seen on our early services to everyone on the team," says Farrelly.

+ +{{< case-studies/lead >}} +For Buffer's engineers, it's an exciting process. "Every time we're deploying a new service, we need to figure out: OK, what's the architecture? How do these services communicate? What's the best way to build this service?" Farrelly says. "And then we use the different features that Kubernetes has to glue all the pieces together. It's enabling us to experiment as we're learning how to design a service-oriented architecture. Before, we just wouldn't have been able to do it. This is actually giving us a blank white board so we can do whatever we want on it." +{{< /case-studies/lead >}} + +

Part of that blank slate is the flexibility that Kubernetes offers should the time come when Buffer may want or need to change its cloud. "It's cloud agnostic so maybe one day we could switch to Google or somewhere else," Farrelly says. "We're very deep in Amazon but it's nice to know we could move away if we need to."

+ +

At this point, the team at Buffer can't imagine running their infrastructure any other way—and they're happy to spread the word. "If you want to run containers in production, with nearly the power that Google uses internally, this [Kubernetes] is a great way to do that," Farrelly says. "We're a relatively small team that's actually running Kubernetes, and we've never run anything like it before. So it's more approachable than you might think. That's the one big thing that I tell people who are experimenting with it. Pick a couple of things, roll it out, kick the tires on this for a couple of months and see how much it can handle. You start learning a lot this way."

diff --git a/content/en/case-studies/capital-one/index.html b/content/en/case-studies/capital-one/index.html index f95fb2acc7..22e7322ea0 100644 --- a/content/en/case-studies/capital-one/index.html +++ b/content/en/case-studies/capital-one/index.html @@ -2,95 +2,61 @@ title: Capital One Case Study case_study_styles: true cid: caseStudies -css: /css/style_case_studies.css + +new_case_study_styles: true +heading_background: /images/case-studies/capitalone/banner1.jpg +heading_title_logo: /images/capitalone-logo.png +subheading: > + Supporting Fast Decisioning Applications with Kubernetes +case_study_details: + - Company: Capital One + - Location: McLean, Virginia + - Industry: Retail banking --- -
-

CASE STUDY:
Supporting Fast Decisioning Applications with Kubernetes +

Challenge

-

+

The team set out to build a provisioning platform for Capital One applications deployed on AWS that use streaming, big-data decisioning, and machine learning. One of these applications handles millions of transactions a day; some deal with critical functions like fraud detection and credit decisioning. The key considerations: resilience and speed—as well as full rehydration of the cluster from base AMIs.

-
+

Solution

-
- Company  Capital One     Location  McLean, Virginia     Industry  Retail banking -
- -
-
-
-
-

Challenge

- The team set out to build a provisioning platform for Capital One applications deployed on AWS that use streaming, big-data decisioning, and machine learning. One of these applications handles millions of transactions a day; some deal with critical functions like fraud detection and credit decisioning. The key considerations: resilience and speed—as well as full rehydration of the cluster from base AMIs. - -
-

Solution

- The decision to run Kubernetes "is very strategic for us," says John Swift, Senior Director Software Engineering. "We use Kubernetes as a substrate or an operating system, if you will. There’s a degree of affinity in our product development." -
- -
+

The decision to run Kubernetes "is very strategic for us," says John Swift, Senior Director Software Engineering. "We use Kubernetes as a substrate or an operating system, if you will. There's a degree of affinity in our product development."

Impact

- "Kubernetes is a significant productivity multiplier," says Lead Software Engineer Keith Gasser, adding that to run the platform without Kubernetes would "easily see our costs triple, quadruple what they are now for the amount of pure AWS expense." Time to market has been improved as well: "Now, a team can come to us and we can have them up and running with a basic decisioning app in a fortnight, which before would have taken a whole quarter, if not longer." Deployments increased by several orders of magnitude. Plus, the rehydration/cluster-rebuild process, which took a significant part of a day to do manually, now takes a couple hours with Kubernetes automation and declarative configuration. + +

"Kubernetes is a significant productivity multiplier," says Lead Software Engineer Keith Gasser, adding that to run the platform without Kubernetes would "easily see our costs triple, quadruple what they are now for the amount of pure AWS expense." Time to market has been improved as well: "Now, a team can come to us and we can have them up and running with a basic decisioning app in a fortnight, which before would have taken a whole quarter, if not longer." Deployments increased by several orders of magnitude. Plus, the rehydration/cluster-rebuild process, which took a significant part of a day to do manually, now takes a couple hours with Kubernetes automation and declarative configuration.

+ +{{< case-studies/quote author="Jamil Jadallah, Scrum Master" >}} + +
+"With the scalability, the management, the coordination, Kubernetes really empowers us and gives us more time back than we had before." +{{< /case-studies/quote >}} + +

As a top 10 U.S. retail bank, Capital One has applications that handle millions of transactions a day. Big-data decisioning—for fraud detection, credit approvals and beyond—is core to the business. To support the teams that build applications with those functions for the bank, the cloud team led by Senior Director Software Engineering John Swift embraced Kubernetes for its provisioning platform. "Kubernetes and its entire ecosystem are very strategic for us," says Swift. "We use Kubernetes as a substrate or an operating system, if you will. There's a degree of affinity in our product development."

+ +

Almost two years ago, the team embarked on this journey by first working with Docker. Then came Kubernetes. "We wanted to put streaming services into Kubernetes as one feature of the workloads for fast decisioning, and to be able to do batch alongside it," says Lead Software Engineer Keith Gasser. "Once the data is streamed and batched, there are so many tool sets in Flink that we use for decisioning. We want to provide the tools in the same ecosystem, in a consistent way, rather than have a large custom snowflake ecosystem where every tool needs its own custom deployment. Kubernetes gives us the ability to bring all of these together, so the richness of the open source and even the license community dealing with big data can be corralled."

+ +{{< case-studies/quote image="/images/case-studies/capitalone/banner3.jpg" >}} +"We want to provide the tools in the same ecosystem, in a consistent way, rather than have a large custom snowflake ecosystem where every tool needs its own custom deployment. Kubernetes gives us the ability to bring all of these together, so the richness of the open source and even the license community dealing with big data can be corralled." +{{< /case-studies/quote >}} + +

In this first year, the impact has already been great. "Time to market is really huge for us," says Gasser. "Especially with fraud, you have to be very nimble in the way you respond to threats in the marketplace—being able to add and push new rules, detect new patterns of behavior, detect anomalies in account and transaction flows." With Kubernetes, "a team can come to us and we can have them up and running with a basic decisioning app in a fortnight, which before would have taken a whole quarter, if not longer. Kubernetes is a manifold productivity multiplier."

+ +

Teams now have the tools to be autonomous in their deployments, and as a result, deployments have increased by two orders of magnitude. "And that was with just seven dedicated resources, without needing a whole group sitting there watching everything," says Scrum Master Jamil Jadallah. "That's a huge cost savings. With the scalability, the management, the coordination, Kubernetes really empowers us and gives us more time back than we had before."

+ +{{< case-studies/quote image="/images/case-studies/capitalone/banner4.jpg" >}} +With Kubernetes, "a team can come to us and we can have them up and running with a basic decisioning app in a fortnight, which before would have taken a whole quarter, if not longer. Kubernetes is a manifold productivity multiplier." +{{< /case-studies/quote >}} + +

Kubernetes has also been a great time-saver for Capital One's required period "rehydration" of clusters from base AMIs. To minimize the attack vulnerability profile for applications in the cloud, "Our entire clusters get rebuilt from scratch periodically, with new fresh instances and virtual server images that are patched with the latest and greatest security patches," says Gasser. This process used to take the better part of a day, and personnel, to do manually. It's now a quick Kubernetes job.

+ +

Savings extend to both capital and operating expenses. "It takes very little to get into Kubernetes because it's all open source," Gasser points out. "We went the DIY route for building our cluster, and we definitely like the flexibility of being able to embrace the latest from the community immediately without waiting for a downstream company to do it. There's capex related to those licenses that we don't have to pay for. Moreover, there's capex savings for us from some of the proprietary software that we get to sunset in our particular domain. So that goes onto our ledger in a positive way as well." (Some of those open source technologies include Prometheus, Fluentd, gRPC, Istio, CNI, and Envoy.)

+ +{{< case-studies/quote >}} +"If we had to do all of this without Kubernetes, on underlying cloud services, I could easily see our costs triple, quadruple what they are now for the amount of pure AWS expense. That doesn't account for personnel to deploy and maintain all the additional infrastructure." +{{< /case-studies/quote >}} -
+

And on the opex side, Gasser says, the savings are high. "We run dozens of services, we have scores of pods, many daemon sets, and since we're data-driven, we take advantage of EBS-backed volume claims for all of our stateful services. If we had to do all of this without Kubernetes, on underlying cloud services, I could easily see our costs triple, quadruple what they are now for the amount of pure AWS expense. That doesn't account for personnel to deploy and maintain all the additional infrastructure."

-
-
-
-
-

-"With the scalability, the management, the coordination, Kubernetes really empowers us and gives us more time back than we had before." — Jamil Jadallah, Scrum Master -
-
- -
-
-

- As a top 10 U.S. retail bank, Capital One has applications that handle millions of transactions a day. Big-data decisioning—for fraud detection, credit approvals and beyond—is core to the business. To support the teams that build applications with those functions for the bank, the cloud team led by Senior Director Software Engineering John Swift embraced Kubernetes for its provisioning platform. "Kubernetes and its entire ecosystem are very strategic for us," says Swift. "We use Kubernetes as a substrate or an operating system, if you will. There’s a degree of affinity in our product development."

- Almost two years ago, the team embarked on this journey by first working with Docker. Then came Kubernetes. "We wanted to put streaming services into Kubernetes as one feature of the workloads for fast decisioning, and to be able to do batch alongside it," says Lead Software Engineer Keith Gasser. "Once the data is streamed and batched, there are so many tool sets in Flink that we use for decisioning. We want to provide the tools in the same ecosystem, in a consistent way, rather than have a large custom snowflake ecosystem where every tool needs its own custom deployment. Kubernetes gives us the ability to bring all of these together, so the richness of the open source and even the license community dealing with big data can be corralled." - - - -
-
-
-
- "We want to provide the tools in the same ecosystem, in a consistent way, rather than have a large custom snowflake ecosystem where every tool needs its own custom deployment. Kubernetes gives us the ability to bring all of these together, so the richness of the open source and even the license community dealing with big data can be corralled." - - -
-
-
-
- In this first year, the impact has already been great. "Time to market is really huge for us," says Gasser. "Especially with fraud, you have to be very nimble in the way you respond to threats in the marketplace—being able to add and push new rules, detect new patterns of behavior, detect anomalies in account and transaction flows." With Kubernetes, "a team can come to us and we can have them up and running with a basic decisioning app in a fortnight, which before would have taken a whole quarter, if not longer. Kubernetes is a manifold productivity multiplier."

- Teams now have the tools to be autonomous in their deployments, and as a result, deployments have increased by two orders of magnitude. "And that was with just seven dedicated resources, without needing a whole group sitting there watching everything," says Scrum Master Jamil Jadallah. "That’s a huge cost savings. With the scalability, the management, the coordination, Kubernetes really empowers us and gives us more time back than we had before." - -
-
-
-
- With Kubernetes, "a team can come to us and we can have them up and running with a basic decisioning app in a fortnight, which before would have taken a whole quarter, if not longer. Kubernetes is a manifold productivity multiplier." -
-
- -
-
- Kubernetes has also been a great time-saver for Capital One’s required period "rehydration" of clusters from base AMIs. To minimize the attack vulnerability profile for applications in the cloud, "Our entire clusters get rebuilt from scratch periodically, with new fresh instances and virtual server images that are patched with the latest and greatest security patches," says Gasser. This process used to take the better part of a day, and personnel, to do manually. It’s now a quick Kubernetes job.

- Savings extend to both capital and operating expenses. "It takes very little to get into Kubernetes because it’s all open source," Gasser points out. "We went the DIY route for building our cluster, and we definitely like the flexibility of being able to embrace the latest from the community immediately without waiting for a downstream company to do it. There’s capex related to those licenses that we don’t have to pay for. Moreover, there’s capex savings for us from some of the proprietary software that we get to sunset in our particular domain. So that goes onto our ledger in a positive way as well." (Some of those open source technologies include Prometheus, Fluentd, gRPC, Istio, CNI, and Envoy.) - -
- -
-
- "If we had to do all of this without Kubernetes, on underlying cloud services, I could easily see our costs triple, quadruple what they are now for the amount of pure AWS expense. That doesn’t account for personnel to deploy and maintain all the additional infrastructure." -
-
- -
- And on the opex side, Gasser says, the savings are high. "We run dozens of services, we have scores of pods, many daemon sets, and since we’re data-driven, we take advantage of EBS-backed volume claims for all of our stateful services. If we had to do all of this without Kubernetes, on underlying cloud services, I could easily see our costs triple, quadruple what they are now for the amount of pure AWS expense. That doesn’t account for personnel to deploy and maintain all the additional infrastructure."

- The team is confident that the benefits will continue to multiply—without a steep learning curve for the engineers being exposed to the new technology. "As we onboard additional tenants in this ecosystem, I think the need for folks to understand Kubernetes may not necessarily go up. In fact, I think it goes down, and that’s good," says Gasser. "Because that really demonstrates the scalability of the technology. You start to reap the benefits, and they can concentrate on all the features they need to build for great decisioning in the business— fraud decisions, credit decisions—and not have to worry about, ‘Is my AWS server broken? Is my pod not running?’" -
- -
+

The team is confident that the benefits will continue to multiply—without a steep learning curve for the engineers being exposed to the new technology. "As we onboard additional tenants in this ecosystem, I think the need for folks to understand Kubernetes may not necessarily go up. In fact, I think it goes down, and that's good," says Gasser. "Because that really demonstrates the scalability of the technology. You start to reap the benefits, and they can concentrate on all the features they need to build for great decisioning in the business— fraud decisions, credit decisions—and not have to worry about, 'Is my AWS server broken? Is my pod not running?'"

diff --git a/content/en/case-studies/cern/index.html b/content/en/case-studies/cern/index.html index 48e965d7fb..461aa8fb84 100644 --- a/content/en/case-studies/cern/index.html +++ b/content/en/case-studies/cern/index.html @@ -3,91 +3,79 @@ title: CERN Case Study linkTitle: cern case_study_styles: true cid: caseStudies -css: /css/style_case_studies.css logo: cern_featured_logo.png + +new_case_study_styles: true +heading_background: /images/case-studies/cern/banner1.jpg +heading_title_text: CERN +subheading: > + CERN: Processing Petabytes of Data More Efficiently with Kubernetes +case_study_details: + - Company: CERN + - Location: Geneva, Switzerland + - Industry: Particle physics research --- -
-

CASE STUDY: CERN
CERN: Processing Petabytes of Data More Efficiently with Kubernetes +

Challenge

-

+

At CERN, the European Organization for Nuclear Research, physicists conduct experiments to learn about fundamental science. In its particle accelerators, "we accelerate protons to very high energy, close to the speed of light, and we make the two beams of protons collide," says CERN Software Engineer Ricardo Rocha. "The end result is a lot of data that we have to process." CERN currently stores 330 petabytes of data in its data centers, and an upgrade of its accelerators expected in the next few years will drive that number up by 10x. Additionally, the organization experiences extreme peaks in its workloads during periods prior to big conferences, and needs its infrastructure to scale to those peaks. "We want to have a more hybrid infrastructure, where we have our on premise infrastructure but can make use of public clouds temporarily when these peaks come up," says Rocha. "We've been looking to new technologies that can help improve our efficiency in our infrastructure so that we can dedicate more of our resources to the actual processing of the data."

-
+

Solution

-
- Company  CERN     Location  Geneva, Switzerland -     Industry  Particle physics research -
+

CERN's technology team embraced containerization and cloud native practices, choosing Kubernetes for orchestration, Helm for deployment, Prometheus for monitoring, and CoreDNS for DNS resolution inside the clusters. Kubernetes federation has allowed the organization to run some production workloads both on premise and in public clouds.

-
-
-
-
-

Challenge

- At CERN, the European Organization for Nuclear Research, physicists conduct experiments to learn about fundamental science. In its particle accelerators, "we accelerate protons to very high energy, close to the speed of light, and we make the two beams of protons collide," says CERN Software Engineer Ricardo Rocha. "The end result is a lot of data that we have to process." CERN currently stores 330 petabytes of data in its data centers, and an upgrade of its accelerators expected in the next few years will drive that number up by 10x. Additionally, the organization experiences extreme peaks in its workloads during periods prior to big conferences, and needs its infrastructure to scale to those peaks. "We want to have a more hybrid infrastructure, where we have our on premise infrastructure but can make use of public clouds temporarily when these peaks come up," says Rocha. "We’ve been looking to new technologies that can help improve our efficiency in our infrastructure so that we can dedicate more of our resources to the actual processing of the data." -

-

Solution

- CERN’s technology team embraced containerization and cloud native practices, choosing Kubernetes for orchestration, Helm for deployment, Prometheus for monitoring, and CoreDNS for DNS resolution inside the clusters. Kubernetes federation has allowed the organization to run some production workloads both on premise and in public clouds. -

Impact

- "Kubernetes gives us the full automation of the application," says Rocha. "It comes with built-in monitoring and logging for all the applications and the workloads that deploy in Kubernetes. This is a massive simplification of our current deployments." The time to deploy a new cluster for a complex distributed storage system has gone from more than 3 hours to less than 15 minutes. Adding new nodes to a cluster used to take more than an hour; now it takes less than 2 minutes. The time it takes to autoscale replicas for system components has decreased from more than an hour to less than 2 minutes. Initially, virtualization gave 20% overhead, but with tuning this was reduced to ~5%. Moving to Kubernetes on bare metal would get this to 0%. Not having to host virtual machines is expected to also get 10% of memory capacity back. -
+

"Kubernetes gives us the full automation of the application," says Rocha. "It comes with built-in monitoring and logging for all the applications and the workloads that deploy in Kubernetes. This is a massive simplification of our current deployments." The time to deploy a new cluster for a complex distributed storage system has gone from more than 3 hours to less than 15 minutes. Adding new nodes to a cluster used to take more than an hour; now it takes less than 2 minutes. The time it takes to autoscale replicas for system components has decreased from more than an hour to less than 2 minutes. Initially, virtualization gave 20% overhead, but with tuning this was reduced to ~5%. Moving to Kubernetes on bare metal would get this to 0%. Not having to host virtual machines is expected to also get 10% of memory capacity back.

-
-
-
-
- "Kubernetes is something we can relate to very much because it’s naturally distributed. What it gives us is a uniform API across heterogeneous resources to define our workloads. This is something we struggled with a lot in the past when we want to expand our resources outside our infrastructure." -

- Ricardo Rocha, Software Engineer, CERN -
-
-
-
-

With a mission of researching fundamental science, and a stable of extremely large machines, the European Organization for Nuclear Research (CERN) operates at what can only be described as hyperscale.

- Experiments are conducted in particle accelerators, the biggest of which is 27 kilometers in circumference. "We accelerate protons to very high energy, to close to the speed of light, and we make the two beams of protons collide in well-defined places," says CERN Software Engineer Ricardo Rocha. "We build experiments around these places where we do the collisions. The end result is a lot of data that we have to process."

- And he does mean a lot: CERN currently stores and processes 330 petabytes of data—gathered from 4,300 projects and 3,300 users—using 10,000 hypervisors and 320,000 cores in its data centers.

- Over the years, the CERN technology department has built a large computing infrastructure, based on OpenStack private clouds, to help the organization’s physicists analyze and treat all this data. The organization experiences extreme peaks in its workloads. "Very often, just before conferences, physicists want to do an enormous amount of extra analysis to publish their papers, and we have to scale to these peaks, which means overcommitting resources in some cases," says Rocha. "We want to have a more hybrid infrastructure, where we have our on premise infrastructure but can make use of public clouds temporarily when these peaks come up."

- Additionally, few years ago, CERN announced that it would be doing a big upgrade of its accelerators, which will mean a ten-fold increase in the amount of data that can be collected. "So we’ve been looking to new technologies that can help improve our efficiency in our infrastructure, so that we can dedicate more of our resources to the actual processing of the data," says Rocha. +{{< case-studies/quote author="Ricardo Rocha, Software Engineer, CERN" >}} +"Kubernetes is something we can relate to very much because it's naturally distributed. What it gives us is a uniform API across heterogeneous resources to define our workloads. This is something we struggled with a lot in the past when we want to expand our resources outside our infrastructure." +{{< /case-studies/quote >}} -
-
-
-
- "Before, the tendency was always: ‘I need this, I get a couple of developers, and I implement it.’ Right now it’s ‘I need this, I’m sure other people also need this, so I’ll go and ask around.’ The CNCF is a good source because there’s a very large catalog of applications available. It’s very hard right now to justify developing a new product in-house. There is really no real reason to keep doing that. It’s much easier for us to try it out, and if we see it’s a good solution, we try to reach out to the community and start working with that community."

- Ricardo Rocha, Software Engineer, CERN +{{< case-studies/lead >}} +With a mission of researching fundamental science, and a stable of extremely large machines, the European Organization for Nuclear Research (CERN) operates at what can only be described as hyperscale. +{{< /case-studies/lead >}} -
-
-
-
- Rocha’s team started looking at Kubernetes and containerization in the second half of 2015. "We’ve been using distributed infrastructures for decades now," says Rocha. "Kubernetes is something we can relate to very much because it’s naturally distributed. What it gives us is a uniform API across heterogeneous resources to define our workloads. This is something we struggled with a lot in the past when we want to expand our resources outside our infrastructure."

- The team created a prototype system for users to deploy their own Kubernetes cluster in CERN’s infrastructure, and spent six months validating the use cases and making sure that Kubernetes integrated with CERN’s internal systems. The main use case is batch workloads, which represent more than 80% of resource usage at CERN. (One single project that does most of the physics data processing and analysis alone consumes 250,000 cores.) "This is something where the investment in simplification of the deployment, logging, and monitoring pays off very quickly," says Rocha. Other use cases include Spark-based data analysis and machine learning to improve physics analysis. "The fact that most of these technologies integrate very well with Kubernetes makes our lives easier," he adds.

- The system went into production in October 2016, also using Helm for deployment, Prometheus for monitoring, and CoreDNS for DNS resolution within the cluster. "One thing that Kubernetes gives us is the full automation of the application," says Rocha. "So it comes with built-in monitoring and logging for all the applications and the workloads that deploy in Kubernetes. This is a massive simplification of our current deployments." The time to deploy a new cluster for a complex distributed storage system has gone from more than 3 hours to less than 15 minutes.

Adding new nodes to a cluster used to take more than an hour; now it takes less than 2 minutes. The time it takes to autoscale replicas for system components has decreased from more than an hour to less than 2 minutes. +

Experiments are conducted in particle accelerators, the biggest of which is 27 kilometers in circumference. "We accelerate protons to very high energy, to close to the speed of light, and we make the two beams of protons collide in well-defined places," says CERN Software Engineer Ricardo Rocha. "We build experiments around these places where we do the collisions. The end result is a lot of data that we have to process."

-
-
-
-
- "With Kubernetes, there’s a well-established technology and a big community that we can contribute to. It allows us to do our physics analysis without having to focus so much on the lower level software. This is just exciting. We are looking forward to keep contributing to the community and collaborating with everyone."

- Ricardo Rocha, Software Engineer, CERN -
-
+

And he does mean a lot: CERN currently stores and processes 330 petabytes of data—gathered from 4,300 projects and 3,300 users—using 10,000 hypervisors and 320,000 cores in its data centers.

-
-
- Rocha points out that the metric used in the particle accelerators may be events per second, but in reality "it’s how fast and how much of the data we can process that actually counts." And efficiency has certainly been improved with Kubernetes. Initially, virtualization gave 20% overhead, but with tuning this was reduced to ~5%. Moving to Kubernetes on bare metal would get this to 0%. Not having to host virtual machines is expected to also get 10% of memory capacity back.

- Kubernetes federation, which CERN has been using for a portion of its production workloads since February 2018, has allowed the organization to adopt a hybrid cloud strategy. And it was remarkably simple to do. "We had a summer intern working on federation," says Rocha. "For many years, I’ve been developing distributed computing software, which took like a decade and a lot of effort from a lot of people to stabilize and make sure it works. And for our intern, in a couple of days he was able to demo to me and my team that we had a cluster at CERN and a few clusters outside in public clouds that were federated together and that we could submit workloads to. This was shocking for us. It really shows the power of using this kind of well-established technologies."

- With such results, adoption of Kubernetes has made rapid gains at CERN, and the team is eager to give back to the community. "If we look back into the ’90s and early 2000s, there were not a lot of companies focusing on systems that have to scale to this kind of size, storing petabytes of data, analyzing petabytes of data," says Rocha. "The fact that Kubernetes is supported by such a wide community and different backgrounds, it motivates us to contribute back." +

Over the years, the CERN technology department has built a large computing infrastructure, based on OpenStack private clouds, to help the organization's physicists analyze and treat all this data. The organization experiences extreme peaks in its workloads. "Very often, just before conferences, physicists want to do an enormous amount of extra analysis to publish their papers, and we have to scale to these peaks, which means overcommitting resources in some cases," says Rocha. "We want to have a more hybrid infrastructure, where we have our on premise infrastructure but can make use of public clouds temporarily when these peaks come up."

-
+

Additionally, few years ago, CERN announced that it would be doing a big upgrade of its accelerators, which will mean a ten-fold increase in the amount of data that can be collected. "So we've been looking to new technologies that can help improve our efficiency in our infrastructure, so that we can dedicate more of our resources to the actual processing of the data," says Rocha.

-
-
-This means that the physicist can build his or her analysis and publish it in a repository, share it with colleagues, and in 10 years redo the same analysis with new data. If we looked back even 10 years, this was just a dream."

- Ricardo Rocha, Software Engineer, CERN
-
+{{< case-studies/quote + image="/images/case-studies/cern/banner3.jpg" + author="Ricardo Rocha, Software Engineer, CERN" +>}} +"Before, the tendency was always: 'I need this, I get a couple of developers, and I implement it.' Right now it's 'I need this, I'm sure other people also need this, so I'll go and ask around.' The CNCF is a good source because there's a very large catalog of applications available. It's very hard right now to justify developing a new product in-house. There is really no real reason to keep doing that. It's much easier for us to try it out, and if we see it's a good solution, we try to reach out to the community and start working with that community." +{{< /case-studies/quote >}} -
- These new technologies aren’t just enabling infrastructure improvements. CERN also uses the Kubernetes-based Reana/Recast platform for reusable analysis, which is "the ability to define physics analysis as a set of workflows that are fully containerized in one single entry point," says Rocha. "This means that the physicist can build his or her analysis and publish it in a repository, share it with colleagues, and in 10 years redo the same analysis with new data. If we looked back even 10 years, this was just a dream."

- All of these things have changed the culture at CERN considerably. A decade ago, "The tendency was always: ‘I need this, I get a couple of developers, and I implement it,’" says Rocha. "Right now it’s ‘I need this, I’m sure other people also need this, so I’ll go and ask around.’ The CNCF is a good source because there’s a very large catalog of applications available. It’s very hard right now to justify developing a new product in-house. There is really no real reason to keep doing that. It’s much easier for us to try it out, and if we see it’s a good solution, we try to reach out to the community and start working with that community." +

Rocha's team started looking at Kubernetes and containerization in the second half of 2015. "We've been using distributed infrastructures for decades now," says Rocha. "Kubernetes is something we can relate to very much because it's naturally distributed. What it gives us is a uniform API across heterogeneous resources to define our workloads. This is something we struggled with a lot in the past when we want to expand our resources outside our infrastructure."

-
-
+

The team created a prototype system for users to deploy their own Kubernetes cluster in CERN's infrastructure, and spent six months validating the use cases and making sure that Kubernetes integrated with CERN's internal systems. The main use case is batch workloads, which represent more than 80% of resource usage at CERN. (One single project that does most of the physics data processing and analysis alone consumes 250,000 cores.) "This is something where the investment in simplification of the deployment, logging, and monitoring pays off very quickly," says Rocha. Other use cases include Spark-based data analysis and machine learning to improve physics analysis. "The fact that most of these technologies integrate very well with Kubernetes makes our lives easier," he adds.

+ +

The system went into production in October 2016, also using Helm for deployment, Prometheus for monitoring, and CoreDNS for DNS resolution within the cluster. "One thing that Kubernetes gives us is the full automation of the application," says Rocha. "So it comes with built-in monitoring and logging for all the applications and the workloads that deploy in Kubernetes. This is a massive simplification of our current deployments." The time to deploy a new cluster for a complex distributed storage system has gone from more than 3 hours to less than 15 minutes.

+ +

Adding new nodes to a cluster used to take more than an hour; now it takes less than 2 minutes. The time it takes to autoscale replicas for system components has decreased from more than an hour to less than 2 minutes.

+ +{{< case-studies/quote + image="/images/case-studies/cern/banner4.jpg" + author="Ricardo Rocha, Software Engineer, CERN" +>}} +"With Kubernetes, there's a well-established technology and a big community that we can contribute to. It allows us to do our physics analysis without having to focus so much on the lower level software. This is just exciting. We are looking forward to keep contributing to the community and collaborating with everyone." +{{< /case-studies/quote >}} + +

Rocha points out that the metric used in the particle accelerators may be events per second, but in reality "it's how fast and how much of the data we can process that actually counts." And efficiency has certainly been improved with Kubernetes. Initially, virtualization gave 20% overhead, but with tuning this was reduced to ~5%. Moving to Kubernetes on bare metal would get this to 0%. Not having to host virtual machines is expected to also get 10% of memory capacity back.

+ +

Kubernetes federation, which CERN has been using for a portion of its production workloads since February 2018, has allowed the organization to adopt a hybrid cloud strategy. And it was remarkably simple to do. "We had a summer intern working on federation," says Rocha. "For many years, I've been developing distributed computing software, which took like a decade and a lot of effort from a lot of people to stabilize and make sure it works. And for our intern, in a couple of days he was able to demo to me and my team that we had a cluster at CERN and a few clusters outside in public clouds that were federated together and that we could submit workloads to. This was shocking for us. It really shows the power of using this kind of well-established technologies."

+ +

With such results, adoption of Kubernetes has made rapid gains at CERN, and the team is eager to give back to the community. "If we look back into the '90s and early 2000s, there were not a lot of companies focusing on systems that have to scale to this kind of size, storing petabytes of data, analyzing petabytes of data," says Rocha. "The fact that Kubernetes is supported by such a wide community and different backgrounds, it motivates us to contribute back."

+ +{{< case-studies/quote author="Ricardo Rocha, Software Engineer, CERN" >}} +This means that the physicist can build his or her analysis and publish it in a repository, share it with colleagues, and in 10 years redo the same analysis with new data. If we looked back even 10 years, this was just a dream." +{{< /case-studies/quote >}} + +

These new technologies aren't just enabling infrastructure improvements. CERN also uses the Kubernetes-based Reana/Recast platform for reusable analysis, which is "the ability to define physics analysis as a set of workflows that are fully containerized in one single entry point," says Rocha. "This means that the physicist can build his or her analysis and publish it in a repository, share it with colleagues, and in 10 years redo the same analysis with new data. If we looked back even 10 years, this was just a dream."

+ +

All of these things have changed the culture at CERN considerably. A decade ago, "The tendency was always: 'I need this, I get a couple of developers, and I implement it,'" says Rocha. "Right now it's 'I need this, I'm sure other people also need this, so I'll go and ask around.' The CNCF is a good source because there's a very large catalog of applications available. It's very hard right now to justify developing a new product in-house. There is really no real reason to keep doing that. It's much easier for us to try it out, and if we see it's a good solution, we try to reach out to the community and start working with that community."

diff --git a/content/en/case-studies/chinaunicom/index.html b/content/en/case-studies/chinaunicom/index.html index 4479d60e67..4327e5c6a3 100644 --- a/content/en/case-studies/chinaunicom/index.html +++ b/content/en/case-studies/chinaunicom/index.html @@ -1,95 +1,77 @@ --- title: China Unicom Case Study - linkTitle: chinaunicom case_study_styles: true cid: caseStudies -css: /css/style_case_studies.css featured: false + +new_case_study_styles: true +heading_background: /images/case-studies/chinaunicom/banner1.jpg +heading_title_logo: /images/chinaunicom_logo.png +subheading: > + China Unicom: How China Unicom Leveraged Kubernetes to Boost Efficiency and Lower IT Costs +case_study_details: + - Company: China Unicom + - Location: Beijing, China + - Industry: Telecom --- -
-

CASE STUDY:
China Unicom: How China Unicom Leveraged Kubernetes to Boost Efficiency
and Lower IT Costs +

Challenge

-

+

China Unicom is one of the top three telecom operators in China, and to serve its 300 million users, the company runs several data centers with thousands of servers in each, using Docker containerization and VMWare and OpenStack infrastructure since 2016. Unfortunately, "the resource utilization rate was relatively low," says Chengyu Zhang, Group Leader of Platform Technology R&D, "and we didn't have a cloud platform to accommodate our hundreds of applications." Formerly an entirely state-owned company, China Unicom has in recent years taken private investment from BAT (Baidu, Alibaba, Tencent) and JD.com, and is now focusing on internal development using open source technology, rather than commercial products. As such, Zhang's China Unicom Lab team began looking for open source orchestration for its cloud infrastructure.

-
+

Solution

-
- Company  China Unicom     Location  Beijing, China     Industry  Telecom -
+

Because of its rapid growth and mature open source community, Kubernetes was a natural choice for China Unicom. The company's Kubernetes-enabled cloud platform now hosts 50 microservices and all new development going forward. "Kubernetes has improved our experience using cloud infrastructure," says Zhang. "There is currently no alternative technology that can replace it." China Unicom also uses Istio for its microservice framework, Envoy, CoreDNS, and Fluentd.

-
-
-
-
-

Challenge

- China Unicom is one of the top three telecom operators in China, and to serve its 300 million users, the company runs several data centers with thousands of servers in each, using Docker containerization and VMWare and OpenStack infrastructure since 2016. Unfortunately, "the resource utilization rate was relatively low," says Chengyu Zhang, Group Leader of Platform Technology R&D, "and we didn’t have a cloud platform to accommodate our hundreds of applications." Formerly an entirely state-owned company, China Unicom has in recent years taken private investment from BAT (Baidu, Alibaba, Tencent) and JD.com, and is now focusing on internal development using open source technology, rather than commercial products. As such, Zhang’s China Unicom Lab team began looking for open source orchestration for its cloud infrastructure. -

-

Solution

- Because of its rapid growth and mature open source community, Kubernetes was a natural choice for China Unicom. The company’s Kubernetes-enabled cloud platform now hosts 50 microservices and all new development going forward. "Kubernetes has improved our experience using cloud infrastructure," says Zhang. "There is currently no alternative technology that can replace it." China Unicom also uses Istio for its microservice framework, Envoy, CoreDNS, and Fluentd. -

Impact

- At China Unicom, Kubernetes has improved both operational and development efficiency. Resource utilization has increased by 20-50%, lowering IT infrastructure costs, and deployment time has gone from a couple of hours to 5-10 minutes. "This is mainly because of the self-healing and scalability, so we can increase our efficiency in operation and maintenance," Zhang says. "For example, we currently have only five people maintaining our multiple systems. We could never imagine we can achieve this scalability in such a short time." -
+

At China Unicom, Kubernetes has improved both operational and development efficiency. Resource utilization has increased by 20-50%, lowering IT infrastructure costs, and deployment time has gone from a couple of hours to 5-10 minutes. "This is mainly because of the self-healing and scalability, so we can increase our efficiency in operation and maintenance," Zhang says. "For example, we currently have only five people maintaining our multiple systems. We could never imagine we can achieve this scalability in such a short time."

-
-
-
-
- "Kubernetes has improved our experience using cloud infrastructure. There is currently no alternative technology that can replace it." -

- Chengyu Zhang, Group Leader of Platform Technology R&D, China Unicom -
-
-
-
-

With more than 300 million users, China Unicom is one of the country’s top three telecom operators.

+{{< case-studies/quote author="Chengyu Zhang, Group Leader of Platform Technology R&D, China Unicom" >}} +"Kubernetes has improved our experience using cloud infrastructure. There is currently no alternative technology that can replace it." +{{< /case-studies/quote >}} - Behind the scenes, the company runs multiple data centers with thousands of servers in each, using Docker containerization and VMWare and OpenStack infrastructure since 2016. Unfortunately, "the resource utilization rate was relatively low," says Chengyu Zhang, Group Leader of Platform Technology R&D, "and we didn’t have a cloud platform to accommodate our hundreds of applications."

- Zhang’s team, which is responsible for new technology, R&D and platforms, set out to find an IT management solution. Formerly an entirely state-owned company, China Unicom has in recent years taken private investment from BAT (Baidu, Alibaba, Tencent) and JD.com, and is now focusing on homegrown development using open source technology, rather than commercial products. For that reason, the team began looking for open source orchestration for its cloud infrastructure. +{{< case-studies/lead >}} +With more than 300 million users, China Unicom is one of the country's top three telecom operators. +{{< /case-studies/lead >}} -
-
-
-
- "We could never imagine we can achieve this scalability in such a short time."

- Chengyu Zhang, Group Leader of Platform Technology R&D, China Unicom +

Behind the scenes, the company runs multiple data centers with thousands of servers in each, using Docker containerization and VMWare and OpenStack infrastructure since 2016. Unfortunately, "the resource utilization rate was relatively low," says Chengyu Zhang, Group Leader of Platform Technology R&D, "and we didn't have a cloud platform to accommodate our hundreds of applications."

-
-
-
-
- Though China Unicom was already using Mesos for a core telecom operator system, the team felt that Kubernetes was a natural choice for the new cloud platform. "The main reason was that it has a mature community," says Zhang. "It grows very rapidly, and so we can learn a lot from others’ best practices." China Unicom also uses Istio for its microservice framework, Envoy, CoreDNS, and Fluentd.

- The company’s Kubernetes-enabled cloud platform now hosts 50 microservices and all new development going forward. China Unicom developers can easily leverage the technology through APIs, without doing the development work themselves. The cloud platform provides 20-30 services connected to the company’s data center PaaS platform, as well as supports things such as big data analysis for internal users in the branch offices across the 31 provinces in China.

- "Kubernetes has improved our experience using cloud infrastructure," says Zhang. "There is currently no alternative technology that can replace it." +

Zhang's team, which is responsible for new technology, R&D and platforms, set out to find an IT management solution. Formerly an entirely state-owned company, China Unicom has in recent years taken private investment from BAT (Baidu, Alibaba, Tencent) and JD.com, and is now focusing on homegrown development using open source technology, rather than commercial products. For that reason, the team began looking for open source orchestration for its cloud infrastructure.

-
-
-
-
- "This technology is relatively complicated, but as long as developers get used to it, they can enjoy all the benefits."

- Jie Jia, Member of Platform Technology R&D, China Unicom -
-
+{{< case-studies/quote + image="/images/case-studies/chinaunicom/banner3.jpg" + author="Chengyu Zhang, Group Leader of Platform Technology R&D, China Unicom" +>}} +"We could never imagine we can achieve this scalability in such a short time." +{{< /case-studies/quote >}} -
-
- In fact, Kubernetes has boosted both operational and development efficiency at China Unicom. Resource utilization has increased by 20-50%, lowering IT infrastructure costs, and deployment time has gone from a couple of hours to 5-10 minutes. "This is mainly because of the self-healing and scalability of Kubernetes, so we can increase our efficiency in operation and maintenance," Zhang says. "For example, we currently have only five people maintaining our multiple systems."

- With the wins China Unicom has experienced with Kubernetes, Zhang and his team are eager to give back to the community. That starts with participating in meetups and conferences, and offering advice to other companies that are considering a similar path. "Especially for those companies who have had traditional cloud computing system, I really recommend them to join the cloud native computing community," says Zhang. +

Though China Unicom was already using Mesos for a core telecom operator system, the team felt that Kubernetes was a natural choice for the new cloud platform. "The main reason was that it has a mature community," says Zhang. "It grows very rapidly, and so we can learn a lot from others' best practices." China Unicom also uses Istio for its microservice framework, Envoy, CoreDNS, and Fluentd.

-
+

The company's Kubernetes-enabled cloud platform now hosts 50 microservices and all new development going forward. China Unicom developers can easily leverage the technology through APIs, without doing the development work themselves. The cloud platform provides 20-30 services connected to the company's data center PaaS platform, as well as supports things such as big data analysis for internal users in the branch offices across the 31 provinces in China.

-
-
-"Companies can use the managed services offered by companies like Rancher, because they have already customized this technology, you can easily leverage this technology."

- Jie Jia, Member of Platform Technology R&D, China Unicom
-
+

"Kubernetes has improved our experience using cloud infrastructure," says Zhang. "There is currently no alternative technology that can replace it."

-
- Platform Technology R&D team member Jie Jia adds that though "this technology is relatively complicated, as long as developers get used to it, they can enjoy all the benefits." And Zhang points out that in his own experience with virtual machine cloud, "Kubernetes and these cloud native technologies are relatively simpler."

- Plus, "companies can use the managed services offered by companies like Rancher, because they have already customized this technology," says Jia. "You can easily leverage this technology."

- Looking ahead, China Unicom plans to develop more applications on Kubernetes, focusing on big data and machine learning. The team is continuing to optimize the cloud platform that it built, and hopes to pass the conformance test to join CNCF’s Certified Kubernetes Conformance Program. They’re also hoping to someday contribute code back to the community.

- If that sounds ambitious, it’s because the results they’ve gotten from adopting Kubernetes have been beyond even their greatest expectations. Says Zhang: "We could never imagine we can achieve this scalability in such a short time." +{{< case-studies/quote + image="/images/case-studies/chinaunicom/banner4.jpg" + author="Jie Jia, Member of Platform Technology R&D, China Unicom" +>}} +"This technology is relatively complicated, but as long as developers get used to it, they can enjoy all the benefits." +{{< /case-studies/quote >}} -
-
- - +

In fact, Kubernetes has boosted both operational and development efficiency at China Unicom. Resource utilization has increased by 20-50%, lowering IT infrastructure costs, and deployment time has gone from a couple of hours to 5-10 minutes. "This is mainly because of the self-healing and scalability of Kubernetes, so we can increase our efficiency in operation and maintenance," Zhang says. "For example, we currently have only five people maintaining our multiple systems."

+ +

With the wins China Unicom has experienced with Kubernetes, Zhang and his team are eager to give back to the community. That starts with participating in meetups and conferences, and offering advice to other companies that are considering a similar path. "Especially for those companies who have had traditional cloud computing system, I really recommend them to join the cloud native computing community," says Zhang.

+ +{{< case-studies/quote author="Jie Jia, Member of Platform Technology R&D, China Unicom" >}} +"Companies can use the managed services offered by companies like Rancher, because they have already customized this technology, you can easily leverage this technology." +{{< /case-studies/quote >}} + +

Platform Technology R&D team member Jie Jia adds that though "this technology is relatively complicated, as long as developers get used to it, they can enjoy all the benefits." And Zhang points out that in his own experience with virtual machine cloud, "Kubernetes and these cloud native technologies are relatively simpler."

+ +

Plus, "companies can use the managed services offered by companies like Rancher, because they have already customized this technology," says Jia. "You can easily leverage this technology."

+ +

Looking ahead, China Unicom plans to develop more applications on Kubernetes, focusing on big data and machine learning. The team is continuing to optimize the cloud platform that it built, and hopes to pass the conformance test to join CNCF's Certified Kubernetes Conformance Program. They're also hoping to someday contribute code back to the community.

+ +

If that sounds ambitious, it's because the results they've gotten from adopting Kubernetes have been beyond even their greatest expectations. Says Zhang: "We could never imagine we can achieve this scalability in such a short time."

diff --git a/content/en/case-studies/city-of-montreal/index.html b/content/en/case-studies/city-of-montreal/index.html index 55378c649e..f639a30bde 100644 --- a/content/en/case-studies/city-of-montreal/index.html +++ b/content/en/case-studies/city-of-montreal/index.html @@ -3,97 +3,79 @@ title: City of Montreal Case Study linkTitle: city-of-montreal case_study_styles: true cid: caseStudies -css: /css/style_case_studies.css featured: false + +new_case_study_styles: true +heading_background: /images/case-studies/montreal/banner1.jpg +heading_title_logo: /images/montreal_logo.png +subheading: > + City of Montréal - How the City of Montréal Is Modernizing Its 30-Year-Old, Siloed Architecture with Kubernetes +case_study_details: + - Company: City of Montréal + - Location: Montréal, Québec, Canada + - Industry: Government --- -
-

CASE STUDY:
City of Montréal - How the City of Montréal Is Modernizing Its 30-Year-Old, Siloed Architecture with Kubernetes +

Challenge

-

+

Like many governments, Montréal has a number of legacy systems, and "we have systems that are older than some developers working here," says the city's CTO, Jean-Martin Thibault. "We have mainframes, all flavors of Windows, various flavors of Linux, old and new Oracle systems, Sun servers, all kinds of databases. Like all big corporations, some of the most important systems, like Budget and Human Resources, were developed on mainframes in-house over the past 30 years." There are over 1,000 applications in all, and most of them were running on different ecosystems. In 2015, a new management team decided to break down those silos, and invest in IT in order to move toward a more integrated governance for the city. They needed to figure out how to modernize the architecture.

-
+

Solution

-
- Company  City of Montréal     Location  Montréal, Québec, Canada     Industry  Government -
+

The first step was containerization. The team started with a small Docker farm with four or five servers, with Rancher for providing access to the Docker containers and their logs and Jenkins to deploy. "We based our effort on the new trends; we understood the benefits of immutability and deployments without downtime and such things," says Solutions Architect Marc Khouzam. They soon realized they needed orchestration as well, and opted for Kubernetes. Says Enterprise Architect Morgan Martinet: "Kubernetes offered concepts on how you would describe an architecture for any kind of application, and based on those concepts, deploy what's required to run the infrastructure. It was becoming a de facto standard."

-
-
-
-
-

Challenge

- Like many governments, Montréal has a number of legacy systems, and “we have systems that are older than some developers working here,” says the city’s CTO, Jean-Martin Thibault. “We have mainframes, all flavors of Windows, various flavors of Linux, old and new Oracle systems, Sun servers, all kinds of databases. Like all big corporations, some of the most important systems, like Budget and Human Resources, were developed on mainframes in-house over the past 30 years.” There are over 1,000 applications in all, and most of them were running on different ecosystems. In 2015, a new management team decided to break down those silos, and invest in IT in order to move toward a more integrated governance for the city. They needed to figure out how to modernize the architecture. - -

Solution

- The first step was containerization. The team started with a small Docker farm with four or five servers, with Rancher for providing access to the Docker containers and their logs and Jenkins to deploy. “We based our effort on the new trends; we understood the benefits of immutability and deployments without downtime and such things,” says Solutions Architect Marc Khouzam. They soon realized they needed orchestration as well, and opted for Kubernetes. Says Enterprise Architect Morgan Martinet: “Kubernetes offered concepts on how you would describe an architecture for any kind of application, and based on those concepts, deploy what’s required to run the infrastructure. It was becoming a de facto standard.” -

Impact

- The time to market has improved drastically, from many months to a few weeks. Deployments went from months to hours. “In the past, you would have to ask for virtual machines, and that alone could take weeks, easily,” says Thibault. “Now you don’t even have to ask for anything. You just create your project and it gets deployed.” Kubernetes has also improved the efficiency of how the city uses its compute resources: “Before, the 200 application components we currently run on Kubernetes would have required hundreds of virtual machines, and now, if we’re talking about a single environment of production, we are able to run them on 8 machines, counting the masters of Kubernetes,” says Martinet. And it’s all done with a small team of just 5 people operating the Kubernetes clusters. -
-
-
-
-
- "We realized the limitations of having a non-orchestrated Docker environment. Kubernetes came to the rescue, bringing in all these features that make it a lot easier to manage and give a lot more benefits to the users." -

- JEAN-MARTIN THIBAULT, CTO, CITY OF MONTRÉAL -
-
-
-
-

The second biggest municipality in Canada, Montréal has a large number of legacy systems keeping the government running. And while they don’t quite date back to the city’s founding in 1642, “we have systems that are older than some developers working here,” jokes the city’s CTO, Jean-Martin Thibault.

- “We have mainframes, all flavors of Windows, various flavors of Linux, old and new Oracle systems, Sun servers, all kinds of databases. Some of the most important systems, like Budget and Human Resources, were developed on mainframes in-house over the past 30 years.” -

- In recent years, that fact became a big pain point. There are over 1,000 applications in all, running on almost as many different ecosystems. In 2015, a new city management team decided to break down those silos, and invest in IT in order to move toward a more integrated governance. “The organization was siloed, so as a result the architecture was siloed,” says Thibault. “Once we got integrated into one IT team, we decided to redo an overall enterprise architecture.” -

- The first step to modernize the architecture was containerization. “We based our effort on the new trends; we understood the benefits of immutability and deployments without downtime and such things,” says Solutions Architect Marc Khouzam. The team started with a small Docker farm with four or five servers, with Rancher for providing access to the Docker containers and their logs and Jenkins for deployment. -
-
-
-
- "Getting a project running in Kubernetes is entirely dependent on how long you need to program the actual software. It’s no longer dependent on deployment. Deployment is so fast that it’s negligible."

- MARC KHOUZAM, SOLUTIONS ARCHITECT, CITY OF MONTRÉAL +

The time to market has improved drastically, from many months to a few weeks. Deployments went from months to hours. "In the past, you would have to ask for virtual machines, and that alone could take weeks, easily," says Thibault. "Now you don't even have to ask for anything. You just create your project and it gets deployed." Kubernetes has also improved the efficiency of how the city uses its compute resources: "Before, the 200 application components we currently run on Kubernetes would have required hundreds of virtual machines, and now, if we're talking about a single environment of production, we are able to run them on 8 machines, counting the masters of Kubernetes," says Martinet. And it's all done with a small team of just 5 people operating the Kubernetes clusters.

-
-
-
-
- But this Docker farm setup had some limitations, including the lack of self-healing and dynamic scaling based on traffic, and the effort required to optimize server resources and scale to multiple instances of the same container. The team soon realized they needed orchestration as well. “Kubernetes came to the rescue,” says Thibault, “bringing in all these features that make it a lot easier to manage and give a lot more benefits to the users.” -

- The team had evaluated several orchestration solutions, but Kubernetes stood out because it addressed all of the pain points. (They were also inspired by Yahoo! Japan’s use case, which the team members felt came close to their vision.) “Kubernetes offered concepts on how you would describe an architecture for any kind of application, and based on those concepts, deploy what’s required to run the infrastructure,” says Enterprise Architect Morgan Martinet. “It was becoming a de facto standard. It also promised portability across cloud providers. The choice of Kubernetes now gives us many options such as running clusters in-house or in any IaaS provider, or even using Kubernetes-as-a-service in any of the major cloud providers.” -

- Another important factor in the decision was vendor neutrality. “As a government entity, it is essential for us to be neutral in our selection of products and providers,” says Thibault. “The independence of the Cloud Native Computing Foundation from any company provides this.” -
-
-
-
- "Kubernetes has been great. It’s been stable, and it provides us with elasticity, resilience, and robustness. While re-architecting for Kubernetes, we also benefited from the monitoring and logging aspects, with centralized logging, Prometheus logging, and Grafana dashboards. We have enhanced visibility of what’s being deployed."

- MORGAN MARTINET, ENTERPRISE ARCHITECT, CITY OF MONTRÉAL -
-
+{{< case-studies/quote author="JEAN-MARTIN THIBAULT, CTO, CITY OF MONTRÉAL" >}} +"We realized the limitations of having a non-orchestrated Docker environment. Kubernetes came to the rescue, bringing in all these features that make it a lot easier to manage and give a lot more benefits to the users." +{{< /case-studies/quote >}} -
-
- The Kubernetes implementation began with the deployment of a small cluster using an internal Ansible playbook, which was soon replaced by the Kismatic distribution. Given the complexity they saw in operating a Kubernetes platform, they decided to provide development groups with an automated CI/CD solution based on Helm. “An integrated CI/CD solution on Kubernetes standardized how the various development teams designed and deployed their solutions, but allowed them to remain independent,” says Khouzam. -

- During the re-architecting process, the team also added Prometheus for monitoring and alerting, Fluentd for logging, and Grafana for visualization. “We have enhanced visibility of what’s being deployed,” says Martinet. Adds Khouzam: “The big benefit is we can track anything, even things that don’t run inside the Kubernetes cluster. It’s our way to unify our monitoring effort.” -

- All together, the cloud native solution has had a positive impact on velocity as well as administrative overhead. With standardization, code generation, automatic deployments into Kubernetes, and standardized monitoring through Prometheus, the time to market has improved drastically, from many months to a few weeks. Deployments went from months and weeks of planning down to hours. “In the past, you would have to ask for virtual machines, and that alone could take weeks to properly provision,” says Thibault. Plus, for dedicated systems, experts often had to be brought in to install them with their own recipes, which could take weeks and months. -

- Now, says Khouzam, “we can deploy pretty much any application that’s been Dockerized without any help from anybody. Getting a project running in Kubernetes is entirely dependent on how long you need to program the actual software. It’s no longer dependent on deployment. Deployment is so fast that it’s negligible.” +{{< case-studies/lead >}} +The second biggest municipality in Canada, Montréal has a large number of legacy systems keeping the government running. And while they don't quite date back to the city's founding in 1642, "we have systems that are older than some developers working here," jokes the city's CTO, Jean-Martin Thibault. +{{< /case-studies/lead >}} -
+

"We have mainframes, all flavors of Windows, various flavors of Linux, old and new Oracle systems, Sun servers, all kinds of databases. Some of the most important systems, like Budget and Human Resources, were developed on mainframes in-house over the past 30 years."

-
-
-"We’re working with the market when possible, to put pressure on our vendors to support Kubernetes, because it’s a much easier solution to manage"

- MORGAN MARTINET, ENTERPRISE ARCHITECT, CITY OF MONTRÉAL
-
+

In recent years, that fact became a big pain point. There are over 1,000 applications in all, running on almost as many different ecosystems. In 2015, a new city management team decided to break down those silos, and invest in IT in order to move toward a more integrated governance. "The organization was siloed, so as a result the architecture was siloed," says Thibault. "Once we got integrated into one IT team, we decided to redo an overall enterprise architecture."

-
- Kubernetes has also improved the efficiency of how the city uses its compute resources: “Before, the 200 application components we currently run in Kubernetes would have required hundreds of virtual machines, and now, if we’re talking about a single environment of production, we are able to run them on 8 machines, counting the masters of Kubernetes,” says Martinet. And it’s all done with a small team of just five people operating the Kubernetes clusters. Adds Martinet: “It’s a dramatic improvement no matter what you measure.” -

- So it should come as no surprise that the team’s strategy going forward is to target Kubernetes as much as they can. “If something can’t run inside Kubernetes, we’ll wait for it,” says Thibault. That means they haven’t moved any of the city’s Windows systems onto Kubernetes, though it’s something they would like to do. “We’re working with the market when possible, to put pressure on our vendors to support Kubernetes, because it’s a much easier solution to manage,” says Martinet. -

- Thibault sees a near future where 60% of the city’s workloads are running on a Kubernetes platform—basically any and all of the use cases that they can get to work there. “It’s so much more efficient than the way we used to do things,” he says. “There’s no looking back.” +

The first step to modernize the architecture was containerization. "We based our effort on the new trends; we understood the benefits of immutability and deployments without downtime and such things," says Solutions Architect Marc Khouzam. The team started with a small Docker farm with four or five servers, with Rancher for providing access to the Docker containers and their logs and Jenkins for deployment.

-
-
+{{< case-studies/quote + image="/images/case-studies/montreal/banner3.jpg" + author="MARC KHOUZAM, SOLUTIONS ARCHITECT, CITY OF MONTRÉAL" +>}} +"Getting a project running in Kubernetes is entirely dependent on how long you need to program the actual software. It's no longer dependent on deployment. Deployment is so fast that it's negligible." +{{< /case-studies/quote >}} + +

But this Docker farm setup had some limitations, including the lack of self-healing and dynamic scaling based on traffic, and the effort required to optimize server resources and scale to multiple instances of the same container. The team soon realized they needed orchestration as well. "Kubernetes came to the rescue," says Thibault, "bringing in all these features that make it a lot easier to manage and give a lot more benefits to the users."

+ +

The team had evaluated several orchestration solutions, but Kubernetes stood out because it addressed all of the pain points. (They were also inspired by Yahoo! Japan's use case, which the team members felt came close to their vision.) "Kubernetes offered concepts on how you would describe an architecture for any kind of application, and based on those concepts, deploy what's required to run the infrastructure," says Enterprise Architect Morgan Martinet. "It was becoming a de facto standard. It also promised portability across cloud providers. The choice of Kubernetes now gives us many options such as running clusters in-house or in any IaaS provider, or even using Kubernetes-as-a-service in any of the major cloud providers."

+ +

Another important factor in the decision was vendor neutrality. "As a government entity, it is essential for us to be neutral in our selection of products and providers," says Thibault. "The independence of the Cloud Native Computing Foundation from any company provides this."

+ +{{< case-studies/quote + image="/images/case-studies/montreal/banner4.jpg" + author="MORGAN MARTINET, ENTERPRISE ARCHITECT, CITY OF MONTRÉAL" +>}} +"Kubernetes has been great. It's been stable, and it provides us with elasticity, resilience, and robustness. While re-architecting for Kubernetes, we also benefited from the monitoring and logging aspects, with centralized logging, Prometheus logging, and Grafana dashboards. We have enhanced visibility of what's being deployed." +{{< /case-studies/quote >}} + +

The Kubernetes implementation began with the deployment of a small cluster using an internal Ansible playbook, which was soon replaced by the Kismatic distribution. Given the complexity they saw in operating a Kubernetes platform, they decided to provide development groups with an automated CI/CD solution based on Helm. "An integrated CI/CD solution on Kubernetes standardized how the various development teams designed and deployed their solutions, but allowed them to remain independent," says Khouzam.

+ +

During the re-architecting process, the team also added Prometheus for monitoring and alerting, Fluentd for logging, and Grafana for visualization. "We have enhanced visibility of what's being deployed," says Martinet. Adds Khouzam: "The big benefit is we can track anything, even things that don't run inside the Kubernetes cluster. It's our way to unify our monitoring effort."

+ +

All together, the cloud native solution has had a positive impact on velocity as well as administrative overhead. With standardization, code generation, automatic deployments into Kubernetes, and standardized monitoring through Prometheus, the time to market has improved drastically, from many months to a few weeks. Deployments went from months and weeks of planning down to hours. "In the past, you would have to ask for virtual machines, and that alone could take weeks to properly provision," says Thibault. Plus, for dedicated systems, experts often had to be brought in to install them with their own recipes, which could take weeks and months.

+ +

Now, says Khouzam, "we can deploy pretty much any application that's been Dockerized without any help from anybody. Getting a project running in Kubernetes is entirely dependent on how long you need to program the actual software. It's no longer dependent on deployment. Deployment is so fast that it's negligible."

+ +{{< case-studies/quote author="MORGAN MARTINET, ENTERPRISE ARCHITECT, CITY OF MONTRÉAL">}} +"We're working with the market when possible, to put pressure on our vendors to support Kubernetes, because it's a much easier solution to manage" +{{< /case-studies/quote >}} + +

Kubernetes has also improved the efficiency of how the city uses its compute resources: "Before, the 200 application components we currently run in Kubernetes would have required hundreds of virtual machines, and now, if we're talking about a single environment of production, we are able to run them on 8 machines, counting the masters of Kubernetes," says Martinet. And it's all done with a small team of just five people operating the Kubernetes clusters. Adds Martinet: "It's a dramatic improvement no matter what you measure."

+ +

So it should come as no surprise that the team's strategy going forward is to target Kubernetes as much as they can. "If something can't run inside Kubernetes, we'll wait for it," says Thibault. That means they haven't moved any of the city's Windows systems onto Kubernetes, though it's something they would like to do. "We're working with the market when possible, to put pressure on our vendors to support Kubernetes, because it's a much easier solution to manage," says Martinet.

+ +

Thibault sees a near future where 60% of the city's workloads are running on a Kubernetes platform—basically any and all of the use cases that they can get to work there. "It's so much more efficient than the way we used to do things," he says. "There's no looking back."

diff --git a/content/en/case-studies/crowdfire/index.html b/content/en/case-studies/crowdfire/index.html index 227a5c0839..98caae2830 100644 --- a/content/en/case-studies/crowdfire/index.html +++ b/content/en/case-studies/crowdfire/index.html @@ -1,101 +1,85 @@ --- title: Crowdfire Case Study - case_study_styles: true cid: caseStudies -css: /css/style_crowdfire.css + +new_case_study_styles: true +heading_background: /images/case-studies/crowdfire/banner1.jpg +heading_title_logo: /images/crowdfire_logo.png +subheading: > + How to Keep Iterating a Fast-Growing App With a Cloud-Native Approach +case_study_details: + - Company: Crowdfire + - Location: Mumbai, India + - Industry: Social Media Software --- -
-

CASE STUDY:
How to Keep Iterating a Fast-Growing App With a Cloud-Native Approach

+

Challenge

-
+

Crowdfire helps content creators create their content anywhere on the Internet and publish it everywhere else in the right format. Since its launch in 2010, it has grown to 16 million users. The product began as a monolith app running on Google App Engine, and in 2015, the company began a transformation to microservices running on Amazon Web Services Elastic Beanstalk. "It was okay for our use cases initially, but as the number of services, development teams and scale increased, the deploy times, self-healing capabilities and resource utilization started to become problems for us," says Software Engineer Amanpreet Singh, who leads the infrastructure team for Crowdfire.

-
- Company  Crowdfire     Location  Mumbai, India     Industry  Social Media Software -
+

Solution

-
-
-
-
-

Challenge

- Crowdfire helps content creators create their content anywhere on the Internet and publish it everywhere else in the right format. Since its launch in 2010, it has grown to 16 million users. The product began as a monolith app running on Google App Engine, and in 2015, the company began a transformation to microservices running on Amazon Web Services Elastic Beanstalk. "It was okay for our use cases initially, but as the number of services, development teams and scale increased, the deploy times, self-healing capabilities and resource utilization started to become problems for us," says Software Engineer Amanpreet Singh, who leads the infrastructure team for Crowdfire.
-

Solution

- "We realized that we needed a more cloud-native approach to deal with these issues," says Singh. The team decided to implement a custom setup of Kubernetes based on Terraform and Ansible. -
-
- -
+

"We realized that we needed a more cloud-native approach to deal with these issues," says Singh. The team decided to implement a custom setup of Kubernetes based on Terraform and Ansible.

Impact

- "Kubernetes has helped us reduce the deployment time from 15 minutes to less than a minute," says Singh. "Due to Kubernetes’s self-healing nature, the operations team doesn’t need to do any manual intervention in case of a node or pod failure." Plus, he says, "Dev-Prod parity has improved since developers can experiment with options in dev/staging clusters, and when it’s finalized, they just commit the config changes in the respective code repositories. These changes automatically get replicated on the production cluster via CI/CD pipelines." -
-
-
-
-
- "In the 15 months that we’ve been using Kubernetes, it has been amazing for us. It enabled us to iterate quickly, increase development speed, and continuously deliver new features and bug fixes to our users, while keeping our operational costs and infrastructure management overhead under control."

- Amanpreet Singh, Software Engineer at Crowdfire -
-
-
-
-

"If you build it, they will come."

- For most content creators, only half of that movie quote may ring true. Sure, platforms like Wordpress, YouTube and Shopify have made it simple for almost anyone to start publishing new content online, but attracting an audience isn’t as easy. Crowdfire "helps users publish their content to all possible places where their audience exists," says Amanpreet Singh, a Software Engineer at the company based in Mumbai, India. Crowdfire has gained more than 16 million users—from bloggers and artists to makers and small businesses—since its launch in 2010.

- With that kind of growth—and a high demand from users for new features and continuous improvements—the Crowdfire team struggled to keep up behind the scenes. In 2015, they moved their monolith Java application to Amazon Web Services Elastic Beanstalk and started breaking it down into microservices.

- It was a good first step, but the team soon realized they needed to go further down the cloud-native path, which would lead them to Kubernetes. "It was okay for our use cases initially, but as the number of services and development teams increased and we scaled further, deploy times, self-healing capabilities and resource utilization started to become problematic," says Singh, who leads the infrastructure team at Crowdfire. "We realized that we needed a more cloud-native approach to deal with these issues."

- As he looked around for solutions, Singh had a checklist of what Crowdfire needed. "We wanted to keep some things separate so they could be shipped independent of other things; this would help remove blockers and let different teams work at their own pace," he says. "We also make a lot of data-driven decisions, so shipping a feature and its iterations quickly was a must."

- Kubernetes checked all the boxes and then some. "One of the best things was the built-in service discovery," he says. "When you have a bunch of microservices that need to call each other, having internal DNS readily available and service IPs and ports automatically set as environment variables help a lot." Plus, he adds, "Kubernetes’s opinionated approach made it easier to get started." +

"Kubernetes has helped us reduce the deployment time from 15 minutes to less than a minute," says Singh. "Due to Kubernetes's self-healing nature, the operations team doesn't need to do any manual intervention in case of a node or pod failure." Plus, he says, "Dev-Prod parity has improved since developers can experiment with options in dev/staging clusters, and when it's finalized, they just commit the config changes in the respective code repositories. These changes automatically get replicated on the production cluster via CI/CD pipelines."

-
-
-
-
- "We realized that we needed a more cloud-native approach to deal with these issues," says Singh. The team decided to implement a custom setup of Kubernetes based on Terraform and Ansible." -
-
-
-
- There was another compelling business reason for the cloud-native approach. "In today’s world of ever-changing business requirements, using cloud native technology provides a variety of options to choose from—even the ability to run services in a hybrid cloud environment," says Singh. "Businesses can keep services in a region closest to the users, and thus benefit from high-availability and resiliency."

- So in February 2016, Singh set up a test Kubernetes cluster using the kube-up scripts provided. "I explored the features and was able to deploy an application pretty easily," he says. "However, it seemed like a black box since I didn’t understand the components completely, and had no idea what the kube-up script did under the hood. So when it broke, it was hard to find the issue and fix it."

- To get a better understanding, Singh dove into the internals of Kubernetes, reading the docs and even some of the code. And he looked to the Kubernetes community for more insight. "I used to stay up a little late every night (a lot of users were active only when it’s night here in India) and would try to answer questions on the Kubernetes community Slack from users who were getting started," he says. "I would also follow other conversations closely. I must admit I was able to avoid a lot of issues in our setup because I knew others had faced the same issues."

- Based on the knowledge he gained, Singh decided to implement a custom setup of Kubernetes based on Terraform and Ansible. "I wrote Terraform to launch Kubernetes master and nodes (Auto Scaling Groups) and an Ansible playbook to install the required components," he says. (The company recently switched to using prebaked AMIs to make the node bringup faster, and is planning to change its networking layer.)

+{{< case-studies/quote author="Amanpreet Singh, Software Engineer at Crowdfire" >}} +"In the 15 months that we've been using Kubernetes, it has been amazing for us. It enabled us to iterate quickly, increase development speed, and continuously deliver new features and bug fixes to our users, while keeping our operational costs and infrastructure management overhead under control." +{{< /case-studies/quote >}} -
-
-
-
- "Kubernetes helped us reduce the deployment time from 15 minutes to less than a minute. Due to Kubernetes’s self-healing nature, the operations team doesn’t need to do any manual intervention in case of a node or pod failure." -
-
+{{< case-studies/lead >}} +"If you build it, they will come." +{{< /case-studies/lead >}} -
-
- First, the team migrated a few staging services from Elastic Beanstalk to the new Kubernetes staging cluster, and then set up a production cluster a month later to deploy some services. The results were convincing. "By the end of March 2016, we established that all the new services must be deployed on Kubernetes," says Singh. "Kubernetes helped us reduce the deployment time from 15 minutes to less than a minute. Due to Kubernetes’s self-healing nature, the operations team doesn’t need to do any manual intervention in case of a node or pod failure." On top of that, he says, "Dev-Prod parity has improved since developers can experiment with options in dev/staging clusters, and when it’s finalized, they just commit the config changes in the respective code repositories. These changes automatically get replicated on the production cluster via CI/CD pipelines. This brings more visibility into the changes being made, and keeping an audit trail."

- Over the next six months, the team worked on migrating all the services from Elastic Beanstalk to Kubernetes, except for the few that were deprecated and would soon be terminated anyway. The services were moved one at a time, and their performance was monitored for two to three days each. Today, "We’re completely migrated and we run all new services on Kubernetes," says Singh.

- The impact has been considerable: With Kubernetes, the company has experienced a 90% cost savings on Elastic Load Balancer, which is now only used for their public, user-facing services. Their EC2 operating expenses have been decreased by as much as 50%.

- All 30 engineers at Crowdfire were onboarded at once. "I gave an internal talk where I shared the basic components and demoed the usage of kubectl," says Singh. "Everyone was excited and happy about using Kubernetes. Developers have more control and visibility into their applications running in production now. Most of all, they’re happy with the low deploy times and self-healing services."

- And they’re much more productive, too. "Where we used to do about 5 deployments per day," says Singh, "now we’re doing 30+ production and 50+ staging deployments almost every day." +

For most content creators, only half of that movie quote may ring true. Sure, platforms like Wordpress, YouTube and Shopify have made it simple for almost anyone to start publishing new content online, but attracting an audience isn't as easy. Crowdfire "helps users publish their content to all possible places where their audience exists," says Amanpreet Singh, a Software Engineer at the company based in Mumbai, India. Crowdfire has gained more than 16 million users—from bloggers and artists to makers and small businesses—since its launch in 2010.

+

With that kind of growth—and a high demand from users for new features and continuous improvements—the Crowdfire team struggled to keep up behind the scenes. In 2015, they moved their monolith Java application to Amazon Web Services Elastic Beanstalk and started breaking it down into microservices.

-
-
-
-
- The impact has been considerable: With Kubernetes, the company has experienced a 90% cost savings on Elastic Load Balancer, which is now only used for their public, user-facing services. Their EC2 operating expenses have been decreased by as much as 50%. +

It was a good first step, but the team soon realized they needed to go further down the cloud-native path, which would lead them to Kubernetes. "It was okay for our use cases initially, but as the number of services and development teams increased and we scaled further, deploy times, self-healing capabilities and resource utilization started to become problematic," says Singh, who leads the infrastructure team at Crowdfire. "We realized that we needed a more cloud-native approach to deal with these issues."

-
-
-
-
+

As he looked around for solutions, Singh had a checklist of what Crowdfire needed. "We wanted to keep some things separate so they could be shipped independent of other things; this would help remove blockers and let different teams work at their own pace," he says. "We also make a lot of data-driven decisions, so shipping a feature and its iterations quickly was a must."

- Singh notes that almost all of the engineers interact with the staging cluster on a daily basis, and that has created a cultural change at Crowdfire. "Developers are more aware of the cloud infrastructure now," he says. "They’ve started following cloud best practices like better health checks, structured logs to stdout [standard output], and config via files or environment variables."

- With Crowdfire’s commitment to Kubernetes, Singh is looking to expand the company’s cloud-native stack. The team already uses Prometheus for monitoring, and he says he is evaluating Linkerd and Envoy Proxy as a way to "get more metrics about request latencies and failures, and handle them better." Other CNCF projects, including OpenTracing and gRPC are also on his radar.

- Singh has found that the cloud-native community is growing in India, too, particularly in Bangalore. "A lot of startups and new companies are starting to run their infrastructure on Kubernetes," he says.

- And when people ask him about Crowdfire’s experience, he has this advice to offer: "Kubernetes is a great piece of technology, but it might not be right for you, especially if you have just one or two services or your app isn’t easy to run in a containerized environment," he says. "Assess your situation and the value that Kubernetes provides before going all in. If you do decide to use Kubernetes, make sure you understand the components that run under the hood and what role they play in smoothly running the cluster. Another thing to consider is if your apps are ‘Kubernetes-ready,’ meaning if they have proper health checks and handle termination signals to shut down gracefully."

- And if your company fits that profile, go for it. Crowdfire clearly did—and is now reaping the benefits. "In the 15 months that we’ve been using Kubernetes, it has been amazing for us," says Singh. "It enabled us to iterate quickly, increase development speed and continuously deliver new features and bug fixes to our users, while keeping our operational costs and infrastructure management overhead under control." +

Kubernetes checked all the boxes and then some. "One of the best things was the built-in service discovery," he says. "When you have a bunch of microservices that need to call each other, having internal DNS readily available and service IPs and ports automatically set as environment variables help a lot." Plus, he adds, "Kubernetes's opinionated approach made it easier to get started."

+{{< case-studies/quote image="/images/case-studies/crowdfire/banner3.jpg" >}} +"We realized that we needed a more cloud-native approach to deal with these issues," says Singh. The team decided to implement a custom setup of Kubernetes based on Terraform and Ansible." +{{< /case-studies/quote >}} -
-
+

There was another compelling business reason for the cloud-native approach. "In today's world of ever-changing business requirements, using cloud native technology provides a variety of options to choose from—even the ability to run services in a hybrid cloud environment," says Singh. "Businesses can keep services in a region closest to the users, and thus benefit from high-availability and resiliency."

+ +

So in February 2016, Singh set up a test Kubernetes cluster using the kube-up scripts provided. "I explored the features and was able to deploy an application pretty easily," he says. "However, it seemed like a black box since I didn't understand the components completely, and had no idea what the kube-up script did under the hood. So when it broke, it was hard to find the issue and fix it."

+ +

To get a better understanding, Singh dove into the internals of Kubernetes, reading the docs and even some of the code. And he looked to the Kubernetes community for more insight. "I used to stay up a little late every night (a lot of users were active only when it's night here in India) and would try to answer questions on the Kubernetes community Slack from users who were getting started," he says. "I would also follow other conversations closely. I must admit I was able to avoid a lot of issues in our setup because I knew others had faced the same issues."

+ +

Based on the knowledge he gained, Singh decided to implement a custom setup of Kubernetes based on Terraform and Ansible. "I wrote Terraform to launch Kubernetes master and nodes (Auto Scaling Groups) and an Ansible playbook to install the required components," he says. (The company recently switched to using prebaked AMIs to make the node bringup faster, and is planning to change its networking layer.)

+ +{{< case-studies/quote image="/images/case-studies/crowdfire/banner4.jpg" >}} +"Kubernetes helped us reduce the deployment time from 15 minutes to less than a minute. Due to Kubernetes's self-healing nature, the operations team doesn't need to do any manual intervention in case of a node or pod failure." +{{< /case-studies/quote >}} + +

First, the team migrated a few staging services from Elastic Beanstalk to the new Kubernetes staging cluster, and then set up a production cluster a month later to deploy some services. The results were convincing. "By the end of March 2016, we established that all the new services must be deployed on Kubernetes," says Singh. "Kubernetes helped us reduce the deployment time from 15 minutes to less than a minute. Due to Kubernetes's self-healing nature, the operations team doesn't need to do any manual intervention in case of a node or pod failure." On top of that, he says, "Dev-Prod parity has improved since developers can experiment with options in dev/staging clusters, and when it's finalized, they just commit the config changes in the respective code repositories. These changes automatically get replicated on the production cluster via CI/CD pipelines. This brings more visibility into the changes being made, and keeping an audit trail."

+ +

Over the next six months, the team worked on migrating all the services from Elastic Beanstalk to Kubernetes, except for the few that were deprecated and would soon be terminated anyway. The services were moved one at a time, and their performance was monitored for two to three days each. Today, "We're completely migrated and we run all new services on Kubernetes," says Singh.

+ +

The impact has been considerable: With Kubernetes, the company has experienced a 90% cost savings on Elastic Load Balancer, which is now only used for their public, user-facing services. Their EC2 operating expenses have been decreased by as much as 50%.

+ +

All 30 engineers at Crowdfire were onboarded at once. "I gave an internal talk where I shared the basic components and demoed the usage of kubectl," says Singh. "Everyone was excited and happy about using Kubernetes. Developers have more control and visibility into their applications running in production now. Most of all, they're happy with the low deploy times and self-healing services."

+ +

And they're much more productive, too. "Where we used to do about 5 deployments per day," says Singh, "now we're doing 30+ production and 50+ staging deployments almost every day."

+ +{{< case-studies/quote >}} +The impact has been considerable: With Kubernetes, the company has experienced a 90% cost savings on Elastic Load Balancer, which is now only used for their public, user-facing services. Their EC2 operating expenses have been decreased by as much as 50%. +{{< /case-studies/quote >}} + +

Singh notes that almost all of the engineers interact with the staging cluster on a daily basis, and that has created a cultural change at Crowdfire. "Developers are more aware of the cloud infrastructure now," he says. "They've started following cloud best practices like better health checks, structured logs to stdout [standard output], and config via files or environment variables."

+ +

With Crowdfire's commitment to Kubernetes, Singh is looking to expand the company's cloud-native stack. The team already uses Prometheus for monitoring, and he says he is evaluating Linkerd and Envoy Proxy as a way to "get more metrics about request latencies and failures, and handle them better." Other CNCF projects, including OpenTracing and gRPC are also on his radar.

+ +

Singh has found that the cloud-native community is growing in India, too, particularly in Bangalore. "A lot of startups and new companies are starting to run their infrastructure on Kubernetes," he says.

+ +

And when people ask him about Crowdfire's experience, he has this advice to offer: "Kubernetes is a great piece of technology, but it might not be right for you, especially if you have just one or two services or your app isn't easy to run in a containerized environment," he says. "Assess your situation and the value that Kubernetes provides before going all in. If you do decide to use Kubernetes, make sure you understand the components that run under the hood and what role they play in smoothly running the cluster. Another thing to consider is if your apps are 'Kubernetes-ready,' meaning if they have proper health checks and handle termination signals to shut down gracefully."

+ +

And if your company fits that profile, go for it. Crowdfire clearly did—and is now reaping the benefits. "In the 15 months that we've been using Kubernetes, it has been amazing for us," says Singh. "It enabled us to iterate quickly, increase development speed and continuously deliver new features and bug fixes to our users, while keeping our operational costs and infrastructure management overhead under control."

diff --git a/content/en/case-studies/denso/index.html b/content/en/case-studies/denso/index.html index 27ef1c77ed..31fb1279b9 100644 --- a/content/en/case-studies/denso/index.html +++ b/content/en/case-studies/denso/index.html @@ -3,108 +3,80 @@ title: Denso Case Study linkTitle: Denso case_study_styles: true cid: caseStudies -css: /css/case-studies-gradient.css logo: denso_featured_logo.svg featured: true weight: 4 quote: > We got Kubernetes experts involved on our team, and it dramatically accelerated development speed. + +new_case_study_styles: true +heading_background: /images/case-studies/denso/banner2.jpg +heading_title_text: Denso +use_gradient_overlay: true +subheading: > + How DENSO Is Fueling Development on the Vehicle Edge with Kubernetes +case_study_details: + - Company: Denso + - Location: Japan + - Industry: Automotive, Edge --- +

Challenge

-
-

CASE STUDY: Denso

-
How DENSO Is Fueling Development on the Vehicle Edge with Kubernetes
-
+

DENSO Corporation is one of the biggest automotive components suppliers in the world. With the advent of connected cars, the company launched a Digital Innovation Department to expand into software, working on vehicle edge and vehicle cloud products. But there were several technical challenges to creating an integrated vehicle edge/cloud platform: "the amount of computing resources, the occasional lack of mobile signal, and an enormous number of distributed vehicles," says R&D Product Manager Seiichi Koizumi.

-
- Company  Denso     Location  Japan     Industry  Automotive, Edge -
+

Solution

-
-
-
-
-

Challenge

- DENSO Corporation is one of the biggest automotive components suppliers in the world. With the advent of connected cars, the company launched a Digital Innovation Department to expand into software, working on vehicle edge and vehicle cloud products. But there were several technical challenges to creating an integrated vehicle edge/cloud platform: "the amount of computing resources, the occasional lack of mobile signal, and an enormous number of distributed vehicles," says R&D Product Manager Seiichi Koizumi. - - -

Solution

- Koizumi’s team realized that because mobility services evolve every day, they needed the flexibility of the cloud native ecosystem for their platform. After considering other orchestrators, DENSO went with Kubernetes for orchestration and added Prometheus, Fluentd, Envoy, Istio, and Helm to the platform. Today, DENSO is using a vehicle edge computer, a private Kubernetes cloud, and managed Kubernetes (GKE, EKS, AKS). +

Koizumi's team realized that because mobility services evolve every day, they needed the flexibility of the cloud native ecosystem for their platform. After considering other orchestrators, DENSO went with Kubernetes for orchestration and added Prometheus, Fluentd, Envoy, Istio, and Helm to the platform. Today, DENSO is using a vehicle edge computer, a private Kubernetes cloud, and managed Kubernetes (GKE, EKS, AKS).

Impact

- Critical layer features can take 2-3 years to implement in the traditional, waterfall model of development at DENSO. With the Kubernetes platform and agile methods, there’s a 2-month development cycle for non-critical software. Now, ten new applications are released a year, and a new prototype is introduced every week. "By utilizing Kubernetes managed services, such as GKE/EKS/AKS, we can unify the environment and simplify our maintenance operation," says Koizumi. -
-
-
-
- "Another disruptive innovation is coming, so to survive in this situation, we need to change our culture." -

- SEIICHI KOIZUMI, R&D PRODUCT MANAGER, DIGITAL INNOVATION DEPARTMENT AT DENSO

-
-
+

Critical layer features can take 2-3 years to implement in the traditional, waterfall model of development at DENSO. With the Kubernetes platform and agile methods, there's a 2-month development cycle for non-critical software. Now, ten new applications are released a year, and a new prototype is introduced every week. "By utilizing Kubernetes managed services, such as GKE/EKS/AKS, we can unify the environment and simplify our maintenance operation," says Koizumi.

-
-
-

Spun off from Toyota in 1949, DENSO Corporation is one of the top automotive suppliers in the world today, with consolidated net revenue of $48.3 billion. -

+{{< case-studies/quote + image="/images/case-studies/denso/banner1.png" + author="SEIICHI KOIZUMI, R&D PRODUCT MANAGER, DIGITAL INNOVATION DEPARTMENT AT DENSO" +>}} +"Another disruptive innovation is coming, so to survive in this situation, we need to change our culture." +{{< /case-studies/quote >}} -

The company’s mission is "contributing to a better world by creating value together with a vision for the future"—and part of that vision in recent years has been development on the vehicle edge and vehicle cloud.

+{{< case-studies/lead >}} +Spun off from Toyota in 1949, DENSO Corporation is one of the top automotive suppliers in the world today, with consolidated net revenue of $48.3 billion. +{{< /case-studies/lead >}} -

With the advent of connected cars, DENSO established a Digital Innovation Department to expand its business beyond the critical layer of the engine, braking systems, and other automotive parts into the non-critical analytics and entertainment layer. Comparing connected cars to smartphones, R&D Product Manager Seiichi Koizumi says DENSO wants the ability to quickly and easily develop and install apps for the "blank slate" of the car, and iterate them based on the driver’s preferences. Thus "we need a flexible application platform," he says.

+

The company's mission is "contributing to a better world by creating value together with a vision for the future"—and part of that vision in recent years has been development on the vehicle edge and vehicle cloud.

+ +

With the advent of connected cars, DENSO established a Digital Innovation Department to expand its business beyond the critical layer of the engine, braking systems, and other automotive parts into the non-critical analytics and entertainment layer. Comparing connected cars to smartphones, R&D Product Manager Seiichi Koizumi says DENSO wants the ability to quickly and easily develop and install apps for the "blank slate" of the car, and iterate them based on the driver's preferences. Thus "we need a flexible application platform," he says.

But working on vehicle edge and vehicle cloud products meant there were several technical challenges: "the amount of computing resources, the occasional lack of mobile signal, and an enormous number of distributed vehicles," says Koizumi. "We are tackling these challenges to create an integrated vehicle edge/cloud platform."

+{{< case-studies/quote author="SEIICHI KOIZUMI, R&D PRODUCT MANAGER, DIGITAL INNOVATION DEPARTMENT AT DENSO" >}} +"We got Kubernetes experts involved on our team, and it dramatically accelerated development speed." +{{< /case-studies/quote >}} -
-
+

Koizumi's team realized that because mobility services evolve every day, they needed the flexibility of the cloud native ecosystem for their platform. As they evaluated technologies, they were led by these criteria: Because their service-enabler business needed to support multiple cloud and on-premise environments, the solution needed to be cloud agnostic, with no vendor lock-in and open governance. It also had to support an edge-cloud integrated environment.

-
-
- "We got Kubernetes experts involved on our team, and it dramatically accelerated development speed."

— SEIICHI KOIZUMI, R&D PRODUCT MANAGER, DIGITAL INNOVATION DEPARTMENT AT DENSO

-
-
+

After considering other orchestrators, DENSO went with Kubernetes for orchestration and added Prometheus, Fluentd, Envoy, Istio, and Helm to the platform. During implementation, the team used "design thinking to clarify use cases and their value proposition," says Koizumi. Next, an agile development team worked on a POC, then an MVP, in DevOps style. "Even in the development phase, we are keeping a channel to end users," he adds.

-
-
-

- Koizumi’s team realized that because mobility services evolve every day, they needed the flexibility of the cloud native ecosystem for their platform. As they evaluated technologies, they were led by these criteria: Because their service-enabler business needed to support multiple cloud and on-premise environments, the solution needed to be cloud agnostic, with no vendor lock-in and open governance. It also had to support an edge-cloud integrated environment.

-

- After considering other orchestrators, DENSO went with Kubernetes for orchestration and added Prometheus, Fluentd, Envoy, Istio, and Helm to the platform. During implementation, the team used "design thinking to clarify use cases and their value proposition," says Koizumi. Next, an agile development team worked on a POC, then an MVP, in DevOps style. "Even in the development phase, we are keeping a channel to end users," he adds.

-

- One lesson learned during this process was the value of bringing in experts. "We tried to learn Kubernetes and cloud native technologies from scratch, but it took more time than expected," says Koizumi. "We got Kubernetes experts involved on our team, and it dramatically accelerated development speed."

+

One lesson learned during this process was the value of bringing in experts. "We tried to learn Kubernetes and cloud native technologies from scratch, but it took more time than expected," says Koizumi. "We got Kubernetes experts involved on our team, and it dramatically accelerated development speed."

-

-
-
+{{< case-studies/quote + image="/images/case-studies/denso/banner4.jpg" + author="SEIICHI KOIZUMI, R&D PRODUCT MANAGER, DIGITAL INNOVATION DEPARTMENT AT DENSO" +>}} +"By utilizing Kubernetes managed services, such as GKE/EKS/AKS, we can unify the environment and simplify our maintenance operation." +{{< /case-studies/quote >}} +

Today, DENSO is using a vehicle edge computer, a private Kubernetes cloud, and managed Kubernetes on GKE, EKS, and AKS. "We are developing a vehicle edge/cloud integrated platform based on a microservice and service mesh architecture," says Koizumi. "We extend cloud into multiple vehicle edges and manage it as a unified platform."

-
-
- "By utilizing Kubernetes managed services, such as GKE/EKS/AKS, we can unify the environment and simplify our maintenance operation."

- SEIICHI KOIZUMI, R&D PRODUCT MANAGER, DIGITAL INNOVATION DEPARTMENT AT DENSO

-
-
+

Cloud native has enabled DENSO to deliver applications via its new dash cam, which has a secure connection that collects data to the cloud. "It's like a smartphone," he says. "We are installing new applications and getting the data through the cloud, and we can keep updating new applications all through the dash cam."

-
-
-

- Today, DENSO is using a vehicle edge computer, a private Kubernetes cloud, and managed Kubernetes on GKE, EKS, and AKS. "We are developing a vehicle edge/cloud integrated platform based on a microservice and service mesh architecture," says Koizumi. "We extend cloud into multiple vehicle edges and manage it as a unified platform."

-

- Cloud native has enabled DENSO to deliver applications via its new dash cam, which has a secure connection that collects data to the cloud. "It’s like a smartphone," he says. "We are installing new applications and getting the data through the cloud, and we can keep updating new applications all through the dash cam."

-

- The unified cloud native platform, combined with agile development, has had a positive impact on productivity. Critical layer features—those involving engines or braking systems, for example—can take 2-3 years to implement at DENSO, because of the time needed to test safety, but also because of the traditional, waterfall model of development. With the Kubernetes platform and agile methods, there’s a 2-month development cycle for non-critical software. Now, ten new applications are released a year, and with the department’s scrum-style development, a new prototype is introduced every week.

-

- Application portability has also led to greater developer efficiency. "There’s no need to care about differences in the multi-cloud platform anymore," says Koizumi. Now, "we are also trying to have the same portability between vehicle edge and cloud platform." -

-

- Another improvement: Automotive Tier-1 suppliers like DENSO always have multiple Tier-2 suppliers. "To provide automotive-grade high-availability services, we tried to do the same thing on a multi-cloud platform," says Koizumi. Before Kubernetes, maintaining two different systems simultaneously was difficult. "By utilizing Kubernetes managed services, such as GKE/EKS/AKS, we can unify the environment and simplify our maintenance operation," he says. -

-

-Cloud native has also profoundly changed the culture at DENSO. The Digital Innovation Department is known as "Noah’s Ark," and it has grown from 2 members to 70—with plans to more than double in the next year. The way they operate is completely different from the traditional Japanese automotive culture. But just as the company embraced change brought by hybrid cars in the past decade, Koizumi says, they’re doing it again now, as technology companies have moved into the connected car space. "Another disruptive innovation is coming," he says, "so to survive in this situation, we need to change our culture." -

-

-Looking ahead, Koizumi and his team are expecting serverless and zero-trust security architecture to be important enhancements of Kubernetes. They are glad DENSO has come along for the ride. "Mobility service businesses require agility and flexibility," he says. "DENSO is trying to bring cloud native flexibility into the vehicle infrastructure." -

-
-
- +

The unified cloud native platform, combined with agile development, has had a positive impact on productivity. Critical layer features—those involving engines or braking systems, for example—can take 2-3 years to implement at DENSO, because of the time needed to test safety, but also because of the traditional, waterfall model of development. With the Kubernetes platform and agile methods, there's a 2-month development cycle for non-critical software. Now, ten new applications are released a year, and with the department's scrum-style development, a new prototype is introduced every week.

+ +

Application portability has also led to greater developer efficiency. "There's no need to care about differences in the multi-cloud platform anymore," says Koizumi. Now, "we are also trying to have the same portability between vehicle edge and cloud platform."

+ +

Another improvement: Automotive Tier-1 suppliers like DENSO always have multiple Tier-2 suppliers. "To provide automotive-grade high-availability services, we tried to do the same thing on a multi-cloud platform," says Koizumi. Before Kubernetes, maintaining two different systems simultaneously was difficult. "By utilizing Kubernetes managed services, such as GKE/EKS/AKS, we can unify the environment and simplify our maintenance operation," he says.

+ +

Cloud native has also profoundly changed the culture at DENSO. The Digital Innovation Department is known as "Noah's Ark," and it has grown from 2 members to 70—with plans to more than double in the next year. The way they operate is completely different from the traditional Japanese automotive culture. But just as the company embraced change brought by hybrid cars in the past decade, Koizumi says, they're doing it again now, as technology companies have moved into the connected car space. "Another disruptive innovation is coming," he says, "so to survive in this situation, we need to change our culture."

+ +

Looking ahead, Koizumi and his team are expecting serverless and zero-trust security architecture to be important enhancements of Kubernetes. They are glad DENSO has come along for the ride. "Mobility service businesses require agility and flexibility," he says. "DENSO is trying to bring cloud native flexibility into the vehicle infrastructure."

diff --git a/content/en/case-studies/golfnow/index.html b/content/en/case-studies/golfnow/index.html index f4bf4d4f27..9e82d90cd1 100644 --- a/content/en/case-studies/golfnow/index.html +++ b/content/en/case-studies/golfnow/index.html @@ -1,125 +1,89 @@ --- title: GolfNow Case Study - case_study_styles: true cid: caseStudies -css: /css/style_golfnow.css + +new_case_study_styles: true +heading_background: /images/case-studies/golfnow/banner1.jpg +heading_title_logo: /images/golfnow_logo.png +subheading: > + Saving Time and Money with Cloud Native Infrastructure +case_study_details: + - Company: GolfNow + - Location: Orlando, Florida + - Industry: Golf Industry Technology and Services Provider --- -
-

CASE STUDY:
-
Saving Time and Money with Cloud Native Infrastructure
-

-
+

Challenge

-
- Company GolfNow     Location Orlando, Florida     Industry Golf Industry Technology and Services Provider -
+

A member of the NBC Sports Group, GolfNow is the golf industry's technology and services leader, managing 10 different products, as well as the largest e-commerce tee time marketplace in the world. As its business began expanding rapidly and globally, GolfNow's monolithic application became problematic. "We kept growing our infrastructure vertically rather than horizontally, and the cost of doing business became problematic," says Sheriff Mohamed, GolfNow's Director, Architecture. "We wanted the ability to more easily expand globally."

-
+

Solution

-
-
-
+

Turning to microservices and containerization, GolfNow began moving its applications and databases from third-party services to its own clusters running on Docker and Kubernetes.

-

Challenge

- A member of the NBC Sports Group, GolfNow is the golf industry’s technology and services leader, managing 10 different products, as well as the largest e-commerce tee time marketplace in the world. As its business began expanding rapidly and globally, GolfNow’s monolithic application became problematic. "We kept growing our infrastructure vertically rather than horizontally, and the cost of doing business became problematic," says Sheriff Mohamed, GolfNow’s Director, Architecture. "We wanted the ability to more easily expand globally." -
-
+

Impact

-
-

Solution

- Turning to microservices and containerization, GolfNow began moving its applications and databases from third-party services to its own clusters running on Docker and Kubernetes.

+

The results were immediate. While maintaining the same capacity—and beyond, during peak periods—GolfNow saw its infrastructure costs for the first application virtually cut in half.

-

Impact

- The results were immediate. While maintaining the same capacity—and beyond, during peak periods—GolfNow saw its infrastructure costs for the first application virtually cut in half. -
-
-
+{{< case-studies/quote author="SHERIFF MOHAMED, DIRECTOR, ARCHITECTURE AT GOLFNOW" >}} +"With our growth we obviously needed to expand our infrastructure, and we kept growing vertically rather than horizontally. We were basically wasting money and doubling the cost of our infrastructure." +{{< /case-studies/quote >}} +{{< case-studies/lead >}} +It's not every day that you can say you've slashed an operating expense by half. +{{< /case-studies/lead >}} -
-
- "With our growth we obviously needed to expand our infrastructure, and we kept growing vertically rather than horizontally. We were basically wasting money and doubling the cost of our infrastructure."

- SHERIFF MOHAMED, DIRECTOR, ARCHITECTURE AT GOLFNOW -
-
+

But Sheriff Mohamed and Josh Chandler did just that when they helped lead their company, GolfNow, on a journey from a monolithic to a containerized, cloud native infrastructure managed by Kubernetes.

-
-
-

It’s not every day that you can say you’ve slashed an operating expense by half.

+

A top-performing business within the NBC Sports Group, GolfNow is a technology and services company with the largest tee time marketplace in the world. GolfNow serves 5 million active golfers across 10 different products. In recent years, the business had grown so fast that the infrastructure supporting their giant monolithic application (written in C#.NET and backed by SQL Server database management system) could not keep up. "With our growth we obviously needed to expand our infrastructure, and we kept growing vertically rather than horizontally," says Sheriff, GolfNow's Director, Architecture. "Our costs were growing exponentially. And on top of that, we had to build a Disaster Recovery (DR) environment, which then meant we'd have to copy exactly what we had in our original data center to another data center that was just the standby. We were basically wasting money and doubling the cost of our infrastructure."

- But Sheriff Mohamed and Josh Chandler did just that when they helped lead their company, GolfNow, on a journey from a monolithic to a containerized, cloud native infrastructure managed by Kubernetes. -

- A top-performing business within the NBC Sports Group, GolfNow is a technology and services company with the largest tee time marketplace in the world. GolfNow serves 5 million active golfers across 10 different products. In recent years, the business had grown so fast that the infrastructure supporting their giant monolithic application (written in C#.NET and backed by SQL Server database management system) could not keep up. "With our growth we obviously needed to expand our infrastructure, and we kept growing vertically rather than horizontally," says Sheriff, GolfNow’s Director, Architecture. "Our costs were growing exponentially. And on top of that, we had to build a Disaster Recovery (DR) environment, which then meant we’d have to copy exactly what we had in our original data center to another data center that was just the standby. We were basically wasting money and doubling the cost of our infrastructure." -

- In moving just the first of GolfNow’s important applications—a booking engine for golf courses and B2B marketing platform—from third-party services to their own Kubernetes environment, "our bill went down drastically," says Sheriff. -

- The path to those stellar results began in late 2014. In order to support GolfNow’s global growth, the team decided that the company needed to have multiple data centers and the ability to quickly and easily re-route traffic as needed. "From there we knew that we needed to go in a direction of breaking things apart, microservices, and containerization," says Sheriff. "At the time we were trying to get away from C#.NET and SQL Server since it didn’t run very well on Linux, where everything container was running smoothly." -

- To that end, the team shifted to working with Node.js, the open-source, cross-platform JavaScript runtime environment for developing tools and applications, and MongoDB, the open-source database program. At the time, Docker, the platform for deploying applications in containers, was still new. But once the team began experimenting with it, Sheriff says, "we realized that was the way we wanted to go, especially since that’s the way the industry is heading." -
-
+

In moving just the first of GolfNow's important applications—a booking engine for golf courses and B2B marketing platform—from third-party services to their own Kubernetes environment, "our bill went down drastically," says Sheriff.

-
-
- "The team migrated the rest of the application into their Kubernetes cluster. And the impact was immediate: On top of cutting monthly costs by a large percentage, says Sheriff, 'Running at the same capacity and during our peak time, we were able to horizontally grow. Since we were using our VMs more efficiently with containers, we didn’t have to pay extra money at all.'" -
-
+

The path to those stellar results began in late 2014. In order to support GolfNow's global growth, the team decided that the company needed to have multiple data centers and the ability to quickly and easily re-route traffic as needed. "From there we knew that we needed to go in a direction of breaking things apart, microservices, and containerization," says Sheriff. "At the time we were trying to get away from C#.NET and SQL Server since it didn't run very well on Linux, where everything container was running smoothly."

-
-
- GolfNow’s dev team ran an "internal, low-key" proof of concept and were won over. "We really liked how easy it was to be able to pass containers around to each other and have them up and running in no time, exactly the way it was running on my machine," says Sheriff. "Because that is always the biggest gripe that Ops has with developers, right? ‘It worked on my machine!’ But then we started getting to the point of, ‘How do we make sure that these things stay up and running?’"

- That led the team on a quest to find the right orchestration system for the company’s needs. Sheriff says the first few options they tried were either too heavy or "didn’t feel quite right." In late summer 2015, they discovered the just-released Kubernetes, which Sheriff immediately liked for its ease of use. "We did another proof of concept," he says, "and Kubernetes won because of the fact that the community backing was there, built on top of what Google had already done." -

- But before they could go with Kubernetes, NBC, GolfNow’s parent company, also asked them to comparison shop with another company. Sheriff and his team liked the competing company’s platform user interface, but didn’t like that its platform would not allow containers to run natively on Docker. With no clear decision in sight, Sheriff’s VP at GolfNow, Steve McElwee, set up a three-month trial during which a GolfNow team (consisting of Sheriff and Josh, who’s now Lead Architect, Open Platforms) would build out a Kubernetes environment, and a large NBC team would build out one with the other company’s platform. -

- "We spun up the cluster and we tried to get everything to run the way we wanted it to run," Sheriff says. "The biggest thing that we took away from it is that not only did we want our applications to run within Kubernetes and Docker, we also wanted our databases to run there. We literally wanted our entire infrastructure to run within Kubernetes." -

- At the time there was nothing in the community to help them get Kafka and MongoDB clusters running within a Kubernetes and Docker environment, so Sheriff and Josh figured it out on their own, taking a full month to get it right. "Everything started rolling from there," Sheriff says. "We were able to get all our applications connected, and we finished our side of the proof of concept a month in advance. My VP was like, ‘Alright, it’s over. Kubernetes wins.’" -

- The next step, beginning in January 2016, was getting everything working in production. The team focused first on one application that was already written in Node.js and MongoDB. A booking engine for golf courses and B2B marketing platform, the application was already going in the microservice direction but wasn’t quite finished yet. At the time, it was running in Heroku Compose and other third-party services—resulting in a large monthly bill. +

To that end, the team shifted to working with Node.js, the open-source, cross-platform JavaScript runtime environment for developing tools and applications, and MongoDB, the open-source database program. At the time, Docker, the platform for deploying applications in containers, was still new. But once the team began experimenting with it, Sheriff says, "we realized that was the way we wanted to go, especially since that's the way the industry is heading."

-
-
+{{< case-studies/quote image="/images/case-studies/golfnow/banner3.jpg" >}} +"The team migrated the rest of the application into their Kubernetes cluster. And the impact was immediate: On top of cutting monthly costs by a large percentage, says Sheriff, 'Running at the same capacity and during our peak time, we were able to horizontally grow. Since we were using our VMs more efficiently with containers, we didn't have to pay extra money at all.'" +{{< /case-studies/quote >}} -
-
- "'The time I spent actually moving the applications was under 30 seconds! We can move data centers in just incredible amounts of time. If you haven’t come from the Kubernetes world you wouldn’t believe me.' Sheriff puts it in these terms: 'Before Kubernetes I wasn’t sleeping at night, literally. I was woken up all the time, because things were down. After Kubernetes, I’ve been sleeping at night.'" -
-
+

GolfNow's dev team ran an "internal, low-key" proof of concept and were won over. "We really liked how easy it was to be able to pass containers around to each other and have them up and running in no time, exactly the way it was running on my machine," says Sheriff. "Because that is always the biggest gripe that Ops has with developers, right? 'It worked on my machine!' But then we started getting to the point of, 'How do we make sure that these things stay up and running?'"

-
-
- "The goal was to take all of that out and put it within this new platform we’ve created with Kubernetes on Google Compute Engine (GCE)," says Sheriff. "So we ended up building piece by piece, in parallel, what was out in Heroku and Compose, in our Kubernetes cluster. Then, literally, just switched configs in the background. So in Heroku we had the app running hitting a Compose database. We’d take the config, change it and make it hit the database that was running in our cluster." -

- Using this procedure, they were able to migrate piecemeal, without any downtime. The first migration was done during off hours, but to test the limits, the team migrated the second database in the middle of the day, when lots of users were running the application. "We did it," Sheriff says, "and again it was successful. Nobody noticed." -

- After three weeks of monitoring to make sure everything was running stable, the team migrated the rest of the application into their Kubernetes cluster. And the impact was immediate: On top of cutting monthly costs by a large percentage, says Sheriff, "Running at the same capacity and during our peak time, we were able to horizontally grow. Since we were using our VMs more efficiently with containers, we didn’t have to pay extra money at all." -

- Not only were they saving money, but they were also saving time. "I had a meeting this morning about migrating some applications from one cluster to another," says Josh. "I spent about 2 hours explaining the process. The time I spent actually moving the applications was under 30 seconds! We can move data centers in just incredible amounts of time. If you haven’t come from the Kubernetes world you wouldn’t believe me." Sheriff puts it in these terms: "Before Kubernetes I wasn’t sleeping at night, literally. I was woken up all the time, because things were down. After Kubernetes, I’ve been sleeping at night." -

- A small percentage of the applications on GolfNow have been migrated over to the Kubernetes environment. "Our Core Team is rewriting a lot of the .NET applications into .NET Core [which is compatible with Linux and Docker] so that we can run them within containers," says Sheriff. -

- Looking ahead, Sheriff and his team want to spend 2017 continuing to build a whole platform around Kubernetes with Drone, an open-source continuous delivery platform, to make it more developer-centric. "Now they’re able to manage configuration, they’re able to manage their deployments and things like that, making all these subteams that are now creating all these microservices, be self sufficient," he says. "So it can pull us away from applications and allow us to just make sure the cluster is running and healthy, and then actually migrate that over to our Ops team." +

That led the team on a quest to find the right orchestration system for the company's needs. Sheriff says the first few options they tried were either too heavy or "didn't feel quite right." In late summer 2015, they discovered the just-released Kubernetes, which Sheriff immediately liked for its ease of use. "We did another proof of concept," he says, "and Kubernetes won because of the fact that the community backing was there, built on top of what Google had already done."

-
-
+

But before they could go with Kubernetes, NBC, GolfNow's parent company, also asked them to comparison shop with another company. Sheriff and his team liked the competing company's platform user interface, but didn't like that its platform would not allow containers to run natively on Docker. With no clear decision in sight, Sheriff's VP at GolfNow, Steve McElwee, set up a three-month trial during which a GolfNow team (consisting of Sheriff and Josh, who's now Lead Architect, Open Platforms) would build out a Kubernetes environment, and a large NBC team would build out one with the other company's platform.

-
-
- "Having gone from complete newbies to production-ready in three months, the GolfNow team is eager to encourage other companies to follow their lead. 'This is The Six Million Dollar Man of the cloud right now,' adds Josh. 'Just try it out, watch it happen. I feel like the proof is in the pudding when you look at these kinds of application stacks. They’re faster, they’re more resilient.'" -
-
+

"We spun up the cluster and we tried to get everything to run the way we wanted it to run," Sheriff says. "The biggest thing that we took away from it is that not only did we want our applications to run within Kubernetes and Docker, we also wanted our databases to run there. We literally wanted our entire infrastructure to run within Kubernetes."

-
-
- And long-term, Sheriff has an even bigger goal for getting more people into the Kubernetes fold. "We’re actually trying to make this platform generic enough so that any of our sister companies can use it if they wish," he says. "Most definitely I think it can be used as a model. I think the way we migrated into it, the way we built it out, are all ways that I think other companies can learn from, and should not be afraid of." -

- The GolfNow team is also giving back to the Kubernetes community by open-sourcing a bot framework that Josh built. "We noticed that the dashboard user interface is actually moving a lot faster than when we started," says Sheriff. "However we realized what we needed was something that’s more of a bot that really helps us administer Kubernetes as a whole through Slack." Josh explains: "With the Kubernetes-Slack integration, you can essentially hook into a cluster and the issue commands and edit configurations. We’ve tried to simplify the security configuration as much as possible. We hope this will be our major thank you to Kubernetes, for everything you’ve given us." -

- Having gone from complete newbies to production-ready in three months, the GolfNow team is eager to encourage other companies to follow their lead. The lessons they’ve learned: "You’ve got to have buy-in from your boss," says Sheriff. "Another big deal is having two to three people dedicated to this type of endeavor. You can’t have people who are half in, half out." And if you don’t have buy-in from the get go, proving it out will get you there. -

- "This is The Six Million Dollar Man of the cloud right now," adds Josh. "Just try it out, watch it happen. I feel like the proof is in the pudding when you look at these kinds of application stacks. They’re faster, they’re more resilient." +

At the time there was nothing in the community to help them get Kafka and MongoDB clusters running within a Kubernetes and Docker environment, so Sheriff and Josh figured it out on their own, taking a full month to get it right. "Everything started rolling from there," Sheriff says. "We were able to get all our applications connected, and we finished our side of the proof of concept a month in advance. My VP was like, 'Alright, it's over. Kubernetes wins.'"

-
-
+

The next step, beginning in January 2016, was getting everything working in production. The team focused first on one application that was already written in Node.js and MongoDB. A booking engine for golf courses and B2B marketing platform, the application was already going in the microservice direction but wasn't quite finished yet. At the time, it was running in Heroku Compose and other third-party services—resulting in a large monthly bill.

+ +{{< case-studies/quote image="/images/case-studies/golfnow/banner4.jpg" >}} +"'The time I spent actually moving the applications was under 30 seconds! We can move data centers in just incredible amounts of time. If you haven't come from the Kubernetes world you wouldn't believe me.' Sheriff puts it in these terms: 'Before Kubernetes I wasn't sleeping at night, literally. I was woken up all the time, because things were down. After Kubernetes, I've been sleeping at night.'" +{{< /case-studies/quote >}} + +

"The goal was to take all of that out and put it within this new platform we've created with Kubernetes on Google Compute Engine (GCE)," says Sheriff. "So we ended up building piece by piece, in parallel, what was out in Heroku and Compose, in our Kubernetes cluster. Then, literally, just switched configs in the background. So in Heroku we had the app running hitting a Compose database. We'd take the config, change it and make it hit the database that was running in our cluster."

+ +

Using this procedure, they were able to migrate piecemeal, without any downtime. The first migration was done during off hours, but to test the limits, the team migrated the second database in the middle of the day, when lots of users were running the application. "We did it," Sheriff says, "and again it was successful. Nobody noticed."

+ +

After three weeks of monitoring to make sure everything was running stable, the team migrated the rest of the application into their Kubernetes cluster. And the impact was immediate: On top of cutting monthly costs by a large percentage, says Sheriff, "Running at the same capacity and during our peak time, we were able to horizontally grow. Since we were using our VMs more efficiently with containers, we didn't have to pay extra money at all."

+ +

Not only were they saving money, but they were also saving time. "I had a meeting this morning about migrating some applications from one cluster to another," says Josh. "I spent about 2 hours explaining the process. The time I spent actually moving the applications was under 30 seconds! We can move data centers in just incredible amounts of time. If you haven't come from the Kubernetes world you wouldn't believe me." Sheriff puts it in these terms: "Before Kubernetes I wasn't sleeping at night, literally. I was woken up all the time, because things were down. After Kubernetes, I've been sleeping at night."

+ +

A small percentage of the applications on GolfNow have been migrated over to the Kubernetes environment. "Our Core Team is rewriting a lot of the .NET applications into .NET Core [which is compatible with Linux and Docker] so that we can run them within containers," says Sheriff.

+ +

Looking ahead, Sheriff and his team want to spend 2017 continuing to build a whole platform around Kubernetes with Drone, an open-source continuous delivery platform, to make it more developer-centric. "Now they're able to manage configuration, they're able to manage their deployments and things like that, making all these subteams that are now creating all these microservices, be self sufficient," he says. "So it can pull us away from applications and allow us to just make sure the cluster is running and healthy, and then actually migrate that over to our Ops team."

+ +{{< case-studies/quote >}} +"Having gone from complete newbies to production-ready in three months, the GolfNow team is eager to encourage other companies to follow their lead. 'This is The Six Million Dollar Man of the cloud right now,' adds Josh. 'Just try it out, watch it happen. I feel like the proof is in the pudding when you look at these kinds of application stacks. They're faster, they're more resilient.'" +{{< /case-studies/quote >}} + +

And long-term, Sheriff has an even bigger goal for getting more people into the Kubernetes fold. "We're actually trying to make this platform generic enough so that any of our sister companies can use it if they wish," he says. "Most definitely I think it can be used as a model. I think the way we migrated into it, the way we built it out, are all ways that I think other companies can learn from, and should not be afraid of."

+ +

The GolfNow team is also giving back to the Kubernetes community by open-sourcing a bot framework that Josh built. "We noticed that the dashboard user interface is actually moving a lot faster than when we started," says Sheriff. "However we realized what we needed was something that's more of a bot that really helps us administer Kubernetes as a whole through Slack." Josh explains: "With the Kubernetes-Slack integration, you can essentially hook into a cluster and the issue commands and edit configurations. We've tried to simplify the security configuration as much as possible. We hope this will be our major thank you to Kubernetes, for everything you've given us."

+ +

Having gone from complete newbies to production-ready in three months, the GolfNow team is eager to encourage other companies to follow their lead. The lessons they've learned: "You've got to have buy-in from your boss," says Sheriff. "Another big deal is having two to three people dedicated to this type of endeavor. You can't have people who are half in, half out." And if you don't have buy-in from the get go, proving it out will get you there.

+ +

"This is The Six Million Dollar Man of the cloud right now," adds Josh. "Just try it out, watch it happen. I feel like the proof is in the pudding when you look at these kinds of application stacks. They're faster, they're more resilient."

diff --git a/content/en/case-studies/haufegroup/index.html b/content/en/case-studies/haufegroup/index.html index f4256ff569..580a282683 100644 --- a/content/en/case-studies/haufegroup/index.html +++ b/content/en/case-studies/haufegroup/index.html @@ -1,112 +1,85 @@ --- title: Haufe Group Case Study - case_study_styles: true cid: caseStudies -css: /css/style_haufegroup.css + +new_case_study_styles: true +heading_background: /images/case-studies/haufegroup/banner1.jpg +heading_title_logo: /images/haufegroup_logo.png +subheading: > + Paving the Way for Cloud Native for Midsize Companies +case_study_details: + - Company: Haufe Group + - Location: Freiburg, Germany + - Industry: Media and Software --- +

Challenge

-
-

CASE STUDY:
Paving the Way for Cloud Native for Midsize Companies

+

Founded in 1930 as a traditional publisher, Haufe Group has grown into a media and software company with 95 percent of its sales from digital products. Over the years, the company has gone from having "hardware in the basement" to outsourcing its infrastructure operations and IT. More recently, the development of new products, from Internet portals for tax experts to personnel training software, has created demands for increased speed, reliability and scalability. "We need to be able to move faster," says Solution Architect Martin Danielsson. "Adapting workloads is something that we really want to be able to do."

-
+

Solution

+

Haufe Group began its cloud-native journey when Microsoft Azure became available in Europe; the company needed cloud deployments for its desktop apps with bandwidth-heavy download services. "After that, it has been different projects trying out different things," says Danielsson. Two years ago, Holger Reinhardt joined Haufe Group as CTO and rapidly re-oriented the traditional host provider-based approach toward a cloud and API-first strategy.

-
- Company  Haufe Group     Location  Freiburg, Germany     Industry  Media and Software -
+

A core part of this strategy was a strong mandate to embrace infrastructure-as-code across the entire software deployment lifecycle via Docker. The company is now getting ready to go live with two services in production using Kubernetes orchestration on Microsoft Azure and Amazon Web Services. The team is also working on breaking up one of their core Java Enterprise desktop products into microservices to allow for better evolvability and dynamic scaling in the cloud.

-
- -
- -
-
-

Challenge

- Founded in 1930 as a traditional publisher, Haufe Group has grown into a media and software company with 95 percent of its sales from digital products. Over the years, the company has gone from having "hardware in the basement" to outsourcing its infrastructure operations and IT. More recently, the development of new products, from Internet portals for tax experts to personnel training software, has created demands for increased speed, reliability and scalability. "We need to be able to move faster," says Solution Architect Martin Danielsson. "Adapting workloads is something that we really want to be able to do." -
-
-

Solution

- Haufe Group began its cloud-native journey when Microsoft Azure became available in Europe; the company needed cloud deployments for its desktop apps with bandwidth-heavy download services. "After that, it has been different projects trying out different things," says Danielsson. Two years ago, Holger Reinhardt joined Haufe Group as CTO and rapidly re-oriented the traditional host provider-based approach toward a cloud and API-first strategy. -
-
- A core part of this strategy was a strong mandate to embrace infrastructure-as-code across the entire software deployment lifecycle via Docker. The company is now getting ready to go live with two services in production using Kubernetes orchestration on Microsoft Azure and Amazon Web Services. The team is also working on breaking up one of their core Java Enterprise desktop products into microservices to allow for better evolvability and dynamic scaling in the cloud. -
-

Impact

- With the ability to adapt workloads, Danielsson says, teams "will be able to scale down to around half the capacity at night, saving 30 percent of the hardware cost." Plus, shorter release times have had a major impact. "Before, we had to announce at least a week in advance when we wanted to do a release because there was a huge checklist of things that you had to do," he says. "By going cloud native, we have the infrastructure in place to be able to automate all of these things. Now we can get a new release done in half an hour instead of days." +

With the ability to adapt workloads, Danielsson says, teams "will be able to scale down to around half the capacity at night, saving 30 percent of the hardware cost." Plus, shorter release times have had a major impact. "Before, we had to announce at least a week in advance when we wanted to do a release because there was a huge checklist of things that you had to do," he says. "By going cloud native, we have the infrastructure in place to be able to automate all of these things. Now we can get a new release done in half an hour instead of days."

-
-
+{{< case-studies/quote author="Martin Danielsson, Solution Architect, Haufe Group" >}} +"Over the next couple of years, people won't even think that much about it when they want to run containers. Kubernetes is going to be the go-to solution." +{{< /case-studies/quote >}} -
+{{< case-studies/lead >}} +More than 80 years ago, Haufe Group was founded as a traditional publishing company, printing books and commentary on paper. +{{< /case-studies/lead >}} -
-
- "Over the next couple of years, people won’t even think that much about it when they want to run containers. Kubernetes is going to be the go-to solution."

- Martin Danielsson, Solution Architect, Haufe Group -
-
+

By the 1990s, though, the company's leaders recognized that the future was digital, and to their credit, were able to transform Haufe Group into a media and software business that now gets 95 percent of its sales from digital products. "Among the German companies doing this, we were one of the early adopters," says Martin Danielsson, Solution Architect for Haufe Group.

-
+

And now they're leading the way for midsize companies embracing cloud-native technology like Kubernetes. "The really big companies like Ticketmaster and Google get it right, and the startups get it right because they're faster," says Danielsson. "We're in this big lump of companies in the middle with a lot of legacy, a lot of structure, a lot of culture that does not easily fit the cloud technologies. We're just 1,500 people, but we have hundreds of customer-facing applications. So we're doing things that will be relevant for many companies of our size or even smaller."

-
-

More than 80 years ago, Haufe Group was founded as a traditional publishing company, printing books and commentary on paper.

By the 1990s, though, the company’s leaders recognized that the future was digital, and to their credit, were able to transform Haufe Group into a media and software business that now gets 95 percent of its sales from digital products. "Among the German companies doing this, we were one of the early adopters," says Martin Danielsson, Solution Architect for Haufe Group.

- And now they’re leading the way for midsize companies embracing cloud-native technology like Kubernetes. "The really big companies like Ticketmaster and Google get it right, and the startups get it right because they’re faster," says Danielsson. "We’re in this big lump of companies in the middle with a lot of legacy, a lot of structure, a lot of culture that does not easily fit the cloud technologies. We’re just 1,500 people, but we have hundreds of customer-facing applications. So we’re doing things that will be relevant for many companies of our size or even smaller."

- Many of those legacy challenges stemmed from simply following the technology trends of the times. "We used to do full DevOps," he says. In the 1990s and 2000s, "that meant that you had your hardware in the basement. And then 10 years ago, the hype of the moment was to outsource application operations, outsource everything, and strip down your IT department to take away the distraction of all these hardware things. That’s not our area of expertise. We didn’t want to be an infrastructure provider. And now comes the backlash of that."

- Haufe Group began feeling the pain as they were developing more new products, from Internet portals for tax experts to personnel training software, that have created demands for increased speed, reliability and scalability. "Right now, we have this break in workflows, where we go from writing concepts to developing, handing it over to production and then handing that over to your host provider," he says. "And then when things go bad we have no clue what went wrong. We definitely want to take back control, and we want to move a lot faster. Adapting workloads is something that we really want to be able to do."

- Those needs led them to explore cloud-native technology. Their first foray into the cloud was doing deployments in Microsoft Azure, once it became available in Europe, for desktop products that had built-in download services. Hosting expenses for such bandwidth-heavy services were too high, so the company turned to the cloud. "After that, it has been different projects trying out different things," says Danielsson. -
-
+

Many of those legacy challenges stemmed from simply following the technology trends of the times. "We used to do full DevOps," he says. In the 1990s and 2000s, "that meant that you had your hardware in the basement. And then 10 years ago, the hype of the moment was to outsource application operations, outsource everything, and strip down your IT department to take away the distraction of all these hardware things. That's not our area of expertise. We didn't want to be an infrastructure provider. And now comes the backlash of that."

-
-
- "We have been doing containers for the last two years, and we really got the hang of how they work," says Danielsson. "But it was always for development and test, never in production, because we didn’t fully understand how that would work. And to me, Kubernetes was definitely the technology that solved that." -
-
+

Haufe Group began feeling the pain as they were developing more new products, from Internet portals for tax experts to personnel training software, that have created demands for increased speed, reliability and scalability. "Right now, we have this break in workflows, where we go from writing concepts to developing, handing it over to production and then handing that over to your host provider," he says. "And then when things go bad we have no clue what went wrong. We definitely want to take back control, and we want to move a lot faster. Adapting workloads is something that we really want to be able to do."

-
-
+

Those needs led them to explore cloud-native technology. Their first foray into the cloud was doing deployments in Microsoft Azure, once it became available in Europe, for desktop products that had built-in download services. Hosting expenses for such bandwidth-heavy services were too high, so the company turned to the cloud. "After that, it has been different projects trying out different things," says Danielsson.

- Two years ago, Holger Reinhardt joined Haufe Group as CTO and rapidly re-oriented the traditional host provider-based approach toward a cloud and API-first strategy. A core part of this strategy was a strong mandate to embrace infrastructure-as-code across the entire software deployment lifecycle via Docker. - Some experiments went further than others; German regulations about sensitive data proved to be a road block in moving some workloads to Azure and Amazon Web Services. "Due to our history, Germany is really strict with things like personally identifiable data," Danielsson says.

- These experiments took on new life with the arrival of the Azure Sovereign Cloud for Germany (an Azure clone run by the German T-Systems provider). With the availability of Azure.de—which conforms to Germany’s privacy regulations—teams started to seriously consider deploying production loads in Docker into the cloud. "We have been doing containers for the last two years, and we really got the hang of how they work," says Danielsson. "But it was always for development and test, never in production, because we didn’t fully understand how that would work. And to me, Kubernetes was definitely the technology that solved that."

- In parallel, Danielsson had built an API management system with the aim of supporting CI/CD scenarios, aspects of which were missing in off-the-shelf API management products. With a foundation based on Mashape’s Kong gateway, it is open-sourced as wicked.haufe.io. He put wicked.haufe.io to use with his product team.

Otherwise, Danielsson says his philosophy was "don’t try to reinvent the wheel all the time. Go for what’s there and 99 percent of the time it will be enough. And if you think you really need something custom or additional, think perhaps once or twice again. One of the things that I find so amazing with this cloud-native framework is that everything ties in."

- Currently, Haufe Group is working on two projects using Kubernetes in production. One is a new mobile application for researching legislation and tax laws. "We needed a way to take out functionality from a legacy core and put an application on top of that with an API gateway—a lot of moving parts that screams containers," says Danielsson. So the team moved the build pipeline away from "deploying to some old, huge machine that you could deploy anything to" and onto a Kubernetes cluster where there would be automatic CI/CD "with feature branches and all these things that were a bit tedious in the past." -
-
+{{< case-studies/quote image="/images/case-studies/haufegroup/banner3.jpg" >}} +"We have been doing containers for the last two years, and we really got the hang of how they work," says Danielsson. "But it was always for development and test, never in production, because we didn't fully understand how that would work. And to me, Kubernetes was definitely the technology that solved that." +{{< /case-studies/quote >}} -
-
- "Before, we had to announce at least a week in advance when we wanted to do a release because there was a huge checklist of things that you had to do," says Danielsson. "By going cloud native, we have the infrastructure in place to be able to automate all of these things. Now we can get a new release done in half an hour instead of days." -
-
+

Two years ago, Holger Reinhardt joined Haufe Group as CTO and rapidly re-oriented the traditional host provider-based approach toward a cloud and API-first strategy. A core part of this strategy was a strong mandate to embrace infrastructure-as-code across the entire software deployment lifecycle via Docker. Some experiments went further than others; German regulations about sensitive data proved to be a road block in moving some workloads to Azure and Amazon Web Services. "Due to our history, Germany is really strict with things like personally identifiable data," Danielsson says.

-
-
- It was a proof of concept effort, and the proof was in the pudding. "Everyone was really impressed at what we accomplished in a week," says Danielsson. "We did these kinds of integrations just to make sure that we got a handle on how Kubernetes works. If you can create optimism and buzz around something, it’s half won. And if the developers and project managers know this is working, you’re more or less done." Adds Reinhardt: "You need to create some very visible, quick wins in order to overcome the status quo."

- The impact on the speed of deployment was clear: "Before, we had to announce at least a week in advance when we wanted to do a release because there was a huge checklist of things that you had to do," says Danielsson. "By going cloud native, we have the infrastructure in place to be able to automate all of these things. Now we can get a new release done in half an hour instead of days."

- The potential impact on cost was another bonus. "Hosting applications is quite expensive, so moving to the cloud is something that we really want to be able to do," says Danielsson. With the ability to adapt workloads, teams "will be able to scale down to around half the capacity at night, saving 30 percent of the hardware cost."

- Just as importantly, Danielsson says, there’s added flexibility: "When we try to move or rework applications that are really crucial, it’s often tricky to validate whether the path we want to take is going to work out well. In order to validate that, we would need to reproduce the environment and really do testing, and that’s prohibitively expensive and simply not doable with traditional host providers. Cloud native gives us the ability to do risky changes and validate them in a cost-effective way."

- As word of the two successful test projects spread throughout the company, interest in Kubernetes has grown. "We want to be able to support our developers in running Kubernetes clusters but we’re not there yet, so we allow them to do it as long as they’re aware that they are on their own," says Danielsson. "So that’s why we are also looking at things like [the managed Kubernetes platform] CoreOS Tectonic, Azure Container Service, ECS, etc. These kinds of services will be a lot more relevant to midsize companies that want to leverage cloud native but don’t have the IT departments or the structure around that."

- In the next year and a half, Danielsson says the company will be working on moving one of their legacy desktop products, a web app for researching legislation and tax laws originally built in Java Enterprise, onto cloud-native technology. "We’re doing a microservice split out right now so that we can independently deploy the different parts," he says. The main website, which provides free content for customers, is also moving to cloud native. +

These experiments took on new life with the arrival of the Azure Sovereign Cloud for Germany (an Azure clone run by the German T-Systems provider). With the availability of Azure.de—which conforms to Germany's privacy regulations—teams started to seriously consider deploying production loads in Docker into the cloud. "We have been doing containers for the last two years, and we really got the hang of how they work," says Danielsson. "But it was always for development and test, never in production, because we didn't fully understand how that would work. And to me, Kubernetes was definitely the technology that solved that."

-
-
+

In parallel, Danielsson had built an API management system with the aim of supporting CI/CD scenarios, aspects of which were missing in off-the-shelf API management products. With a foundation based on Mashape's Kong gateway, it is open-sourced as wicked.haufe.io. He put wicked.haufe.io to use with his product team.

Otherwise, Danielsson says his philosophy was "don't try to reinvent the wheel all the time. Go for what's there and 99 percent of the time it will be enough. And if you think you really need something custom or additional, think perhaps once or twice again. One of the things that I find so amazing with this cloud-native framework is that everything ties in."

-
-
- "the execution of a strategy requires alignment of culture, structure and technology. Only if those three dimensions are aligned can you successfully execute a transformation into microservices and cloud-native architectures. And it is only then that the Cloud will pay the dividends in much faster speeds in product innovation and much lower operational costs." +

Currently, Haufe Group is working on two projects using Kubernetes in production. One is a new mobile application for researching legislation and tax laws. "We needed a way to take out functionality from a legacy core and put an application on top of that with an API gateway—a lot of moving parts that screams containers," says Danielsson. So the team moved the build pipeline away from "deploying to some old, huge machine that you could deploy anything to" and onto a Kubernetes cluster where there would be automatic CI/CD "with feature branches and all these things that were a bit tedious in the past."

-
-
+{{< case-studies/quote image="/images/case-studies/haufegroup/banner4.jpg" >}} +"Before, we had to announce at least a week in advance when we wanted to do a release because there was a huge checklist of things that you had to do," says Danielsson. "By going cloud native, we have the infrastructure in place to be able to automate all of these things. Now we can get a new release done in half an hour instead of days." +{{< /case-studies/quote >}} -
-
- But with these goals, Danielsson believes there are bigger cultural challenges that need to be constantly addressed. The move to new technology, not to mention a shift toward DevOps, means a lot of change for employees. "The roles were rather fixed in the past," he says. "You had developers, you had project leads, you had testers. And now you get into these really, really important things like test automation. Testers aren’t actually doing click testing anymore, and they have to write automated testing. And if you really want to go full-blown CI/CD, all these little pieces have to work together so that you get the confidence to do a check in, and know this check in is going to land in production, because if I messed up, some test is going to break. This is a really powerful thing because whatever you do, whenever you merge something into the trunk or to the master, this is going live. And that’s where you either get the people or they run away screaming." - Danielsson understands that it may take some people much longer to get used to the new ways.

- "Culture is nothing that you can force on people," he says. "You have to live it for yourself. You have to evangelize. You have to show the advantages time and time again: This is how you can do it, this is what you get from it." To that end, his team has scheduled daylong workshops for the staff, bringing in outside experts to talk about everything from API to Devops to cloud.

- For every person who runs away screaming, many others get drawn in. "Get that foot in the door and make them really interested in this stuff," says Danielsson. "Usually it catches on. We have people you never would have expected chanting, ‘Docker Docker Docker’ now. It’s cool to see them realize that there is a world outside of their Python libraries. It’s awesome to see them really work with Kubernetes."

- Ultimately, Reinhardt says, "the execution of a strategy requires alignment of culture, structure and technology. Only if those three dimensions are aligned can you successfully execute a transformation into microservices and cloud-native architectures. And it is only then that the Cloud will pay the dividends in much faster speeds in product innovation and much lower operational costs." +

It was a proof of concept effort, and the proof was in the pudding. "Everyone was really impressed at what we accomplished in a week," says Danielsson. "We did these kinds of integrations just to make sure that we got a handle on how Kubernetes works. If you can create optimism and buzz around something, it's half won. And if the developers and project managers know this is working, you're more or less done." Adds Reinhardt: "You need to create some very visible, quick wins in order to overcome the status quo."

-
-
+

The impact on the speed of deployment was clear: "Before, we had to announce at least a week in advance when we wanted to do a release because there was a huge checklist of things that you had to do," says Danielsson. "By going cloud native, we have the infrastructure in place to be able to automate all of these things. Now we can get a new release done in half an hour instead of days."

+ +

The potential impact on cost was another bonus. "Hosting applications is quite expensive, so moving to the cloud is something that we really want to be able to do," says Danielsson. With the ability to adapt workloads, teams "will be able to scale down to around half the capacity at night, saving 30 percent of the hardware cost."

+ +

Just as importantly, Danielsson says, there's added flexibility: "When we try to move or rework applications that are really crucial, it's often tricky to validate whether the path we want to take is going to work out well. In order to validate that, we would need to reproduce the environment and really do testing, and that's prohibitively expensive and simply not doable with traditional host providers. Cloud native gives us the ability to do risky changes and validate them in a cost-effective way."

+ +

As word of the two successful test projects spread throughout the company, interest in Kubernetes has grown. "We want to be able to support our developers in running Kubernetes clusters but we're not there yet, so we allow them to do it as long as they're aware that they are on their own," says Danielsson. "So that's why we are also looking at things like [the managed Kubernetes platform] CoreOS Tectonic, Azure Container Service, ECS, etc. These kinds of services will be a lot more relevant to midsize companies that want to leverage cloud native but don't have the IT departments or the structure around that."

+ +

In the next year and a half, Danielsson says the company will be working on moving one of their legacy desktop products, a web app for researching legislation and tax laws originally built in Java Enterprise, onto cloud-native technology. "We're doing a microservice split out right now so that we can independently deploy the different parts," he says. The main website, which provides free content for customers, is also moving to cloud native.

+ +{{< case-studies/quote >}} +"the execution of a strategy requires alignment of culture, structure and technology. Only if those three dimensions are aligned can you successfully execute a transformation into microservices and cloud-native architectures. And it is only then that the Cloud will pay the dividends in much faster speeds in product innovation and much lower operational costs." +{{< /case-studies/quote >}} + +

But with these goals, Danielsson believes there are bigger cultural challenges that need to be constantly addressed. The move to new technology, not to mention a shift toward DevOps, means a lot of change for employees. "The roles were rather fixed in the past," he says. "You had developers, you had project leads, you had testers. And now you get into these really, really important things like test automation. Testers aren't actually doing click testing anymore, and they have to write automated testing. And if you really want to go full-blown CI/CD, all these little pieces have to work together so that you get the confidence to do a check in, and know this check in is going to land in production, because if I messed up, some test is going to break. This is a really powerful thing because whatever you do, whenever you merge something into the trunk or to the master, this is going live. And that's where you either get the people or they run away screaming." Danielsson understands that it may take some people much longer to get used to the new ways.

+ +

"Culture is nothing that you can force on people," he says. "You have to live it for yourself. You have to evangelize. You have to show the advantages time and time again: This is how you can do it, this is what you get from it." To that end, his team has scheduled daylong workshops for the staff, bringing in outside experts to talk about everything from API to Devops to cloud.

+ +

For every person who runs away screaming, many others get drawn in. "Get that foot in the door and make them really interested in this stuff," says Danielsson. "Usually it catches on. We have people you never would have expected chanting, 'Docker Docker Docker' now. It's cool to see them realize that there is a world outside of their Python libraries. It's awesome to see them really work with Kubernetes."

+ +

Ultimately, Reinhardt says, "the execution of a strategy requires alignment of culture, structure and technology. Only if those three dimensions are aligned can you successfully execute a transformation into microservices and cloud-native architectures. And it is only then that the Cloud will pay the dividends in much faster speeds in product innovation and much lower operational costs."

diff --git a/content/en/case-studies/huawei/index.html b/content/en/case-studies/huawei/index.html index 29de86f5c4..ec1cd212f2 100644 --- a/content/en/case-studies/huawei/index.html +++ b/content/en/case-studies/huawei/index.html @@ -1,101 +1,73 @@ --- title: Huawei Case Study - case_study_styles: true cid: caseStudies -css: /css/style_huawei.css + +new_case_study_styles: true +heading_background: /images/case-studies/huawei/banner1.jpg +heading_title_logo: /images/huawei_logo.png +subheading: > + Embracing Cloud Native as a User – and a Vendor +case_study_details: + - Company: Huawei + - Location: Shenzhen, China + - Industry: Telecommunications Equipment --- -
-

CASE STUDY:
Embracing Cloud Native as a User – and a Vendor

-
+

Challenge

-
- Company  Huawei     Location  Shenzhen, China     Industry  Telecommunications Equipment -
+

A multinational company that's the largest telecommunications equipment manufacturer in the world, Huawei has more than 180,000 employees. In order to support its fast business development around the globe, Huawei has eight data centers for its internal I.T. department, which have been running 800+ applications in 100K+ VMs to serve these 180,000 users. With the rapid increase of new applications, the cost and efficiency of management and deployment of VM-based apps all became critical challenges for business agility. "It's very much a distributed system so we found that managing all of the tasks in a more consistent way is always a challenge," says Peixin Hou, the company's Chief Software Architect and Community Director for Open Source. "We wanted to move into a more agile and decent practice."

-
+

Solution

-
+

After deciding to use container technology, Huawei began moving the internal I.T. department's applications to run on Kubernetes. So far, about 30 percent of these applications have been transferred to cloud native.

-
-
-

Challenge

- A multinational company that’s the largest telecommunications equipment manufacturer in the world, Huawei has more than 180,000 employees. In order to support its fast business development around the globe, Huawei has eight data centers for its internal I.T. department, which have been running 800+ applications in 100K+ VMs to serve these 180,000 users. With the rapid increase of new applications, the cost and efficiency of management and deployment of VM-based apps all became critical challenges for business agility. "It’s very much a distributed system so we found that managing all of the tasks in a more consistent way is always a challenge," says Peixin Hou, the company’s Chief Software Architect and Community Director for Open Source. "We wanted to move into a more agile and decent practice." -
- -
-

Solution

- After deciding to use container technology, Huawei began moving the internal I.T. department’s applications to run on Kubernetes. So far, about 30 percent of these applications have been transferred to cloud native. -
-

Impact

- "By the end of 2016, Huawei’s internal I.T. department managed more than 4,000 nodes with tens of thousands containers using a Kubernetes-based Platform as a Service (PaaS) solution," says Hou. "The global deployment cycles decreased from a week to minutes, and the efficiency of application delivery has been improved 10 fold." For the bottom line, he says, "We also see significant operating expense spending cut, in some circumstances 20-30 percent, which we think is very helpful for our business." Given the results Huawei has had internally – and the demand it is seeing externally – the company has also built the technologies into FusionStage™, the PaaS solution it offers its customers. -
-
+

"By the end of 2016, Huawei's internal I.T. department managed more than 4,000 nodes with tens of thousands containers using a Kubernetes-based Platform as a Service (PaaS) solution," says Hou. "The global deployment cycles decreased from a week to minutes, and the efficiency of application delivery has been improved 10 fold." For the bottom line, he says, "We also see significant operating expense spending cut, in some circumstances 20-30 percent, which we think is very helpful for our business." Given the results Huawei has had internally – and the demand it is seeing externally – the company has also built the technologies into FusionStage™, the PaaS solution it offers its customers.

-
+{{< case-studies/quote author="Peixin Hou, chief software architect and community director for open source" >}} +"If you're a vendor, in order to convince your customer, you should use it yourself. Luckily because Huawei has a lot of employees, we can demonstrate the scale of cloud we can build using this technology." +{{< /case-studies/quote >}} -
-
- "If you’re a vendor, in order to convince your customer, you should use it yourself. Luckily because Huawei has a lot of employees, we can demonstrate the scale of cloud we can build using this technology."

- Peixin Hou, chief software architect and community director for open source -
-
+

Huawei's Kubernetes journey began with one developer. Over two years ago, one of the engineers employed by the networking and telecommunications giant became interested in Kubernetes, the technology for managing application containers across clusters of hosts, and started contributing to its open source community. As the technology developed and the community grew, he kept telling his managers about it.

-
+

And as fate would have it, at the same time, Huawei was looking for a better orchestration system for its internal enterprise I.T. department, which supports every business flow processing. "We have more than 180,000 employees worldwide, and a complicated internal procedure, so probably every week this department needs to develop some new applications," says Peixin Hou, Huawei's Chief Software Architect and Community Director for Open Source. "Very often our I.T. departments need to launch tens of thousands of containers, with tasks running across thousands of nodes across the world. It's very much a distributed system, so we found that managing all of the tasks in a more consistent way is always a challenge."

-
- Huawei’s Kubernetes journey began with one developer. - Over two years ago, one of the engineers employed by the networking and telecommunications giant became interested in Kubernetes, the technology for managing application containers across clusters of hosts, and started contributing to its open source community. As the technology developed and the community grew, he kept telling his managers about it.

- And as fate would have it, at the same time, Huawei was looking for a better orchestration system for its internal enterprise I.T. department, which supports every business flow processing. "We have more than 180,000 employees worldwide, and a complicated internal procedure, so probably every week this department needs to develop some new applications," says Peixin Hou, Huawei’s Chief Software Architect and Community Director for Open Source. "Very often our I.T. departments need to launch tens of thousands of containers, with tasks running across thousands of nodes across the world. It’s very much a distributed system, so we found that managing all of the tasks in a more consistent way is always a challenge."

- In the past, Huawei had used virtual machines to encapsulate applications, but "every time when we start a VM," Hou says, "whether because it’s a new service or because it was a service that was shut down because of some abnormal node functioning, it takes a lot of time." Huawei turned to containerization, so the timing was right to try Kubernetes. It took a year to adopt that engineer’s suggestion – the process "is not overnight," says Hou – but once in use, he says, "Kubernetes basically solved most of our problems. Before, the time of deployment took about a week, now it only takes minutes. The developers are happy. That department is also quite happy."

- Hou sees great benefits to the company that come with using this technology: "Kubernetes brings agility, scale-out capability, and DevOps practice to the cloud-based applications," he says. "It provides us with the ability to customize the scheduling architecture, which makes possible the affinity between container tasks that gives greater efficiency. It supports multiple container formats. It has extensive support for various container networking solutions and container storage." -
-
+

In the past, Huawei had used virtual machines to encapsulate applications, but "every time when we start a VM," Hou says, "whether because it's a new service or because it was a service that was shut down because of some abnormal node functioning, it takes a lot of time." Huawei turned to containerization, so the timing was right to try Kubernetes. It took a year to adopt that engineer's suggestion – the process "is not overnight," says Hou – but once in use, he says, "Kubernetes basically solved most of our problems. Before, the time of deployment took about a week, now it only takes minutes. The developers are happy. That department is also quite happy."

-
-
- "Kubernetes basically solved most of our problems. Before, the time of deployment took about a week, now it only takes minutes. The developers are happy. That department is also quite happy." -
-
+

Hou sees great benefits to the company that come with using this technology: "Kubernetes brings agility, scale-out capability, and DevOps practice to the cloud-based applications," he says. "It provides us with the ability to customize the scheduling architecture, which makes possible the affinity between container tasks that gives greater efficiency. It supports multiple container formats. It has extensive support for various container networking solutions and container storage."

-
-
- And not least of all, there’s an impact on the bottom line. Says Hou: "We also see significant operating expense spending cut in some circumstances 20-30 percent, which is very helpful for our business."

- Pleased with those initial results, and seeing a demand for cloud native technologies from its customers, Huawei doubled down on Kubernetes. In the spring of 2016, the company became not only a user but also a vendor.

- "We built the Kubernetes technologies into our solutions," says Hou, referring to Huawei’s FusionStage™ PaaS offering. "Our customers, from very big telecommunications operators to banks, love the idea of cloud native. They like Kubernetes technology. But they need to spend a lot of time to decompose their applications to turn them into microservice architecture, and as a solution provider, we help them. We’ve started to work with some Chinese banks, and we see a lot of interest from our customers like China Mobile and Deutsche Telekom."

- "If you’re just a user, you’re just a user," adds Hou. "But if you’re a vendor, in order to even convince your customers, you should use it yourself. Luckily because Huawei has a lot of employees, we can demonstrate the scale of cloud we can build using this technology. We provide customer wisdom." While Huawei has its own private cloud, many of its customers run cross-cloud applications using Huawei’s solutions. It’s a big selling point that most of the public cloud providers now support Kubernetes. "This makes the cross-cloud transition much easier than with other solutions," says Hou.

-
-
+{{< case-studies/quote image="/images/case-studies/huawei/banner3.jpg" >}} +"Kubernetes basically solved most of our problems. Before, the time of deployment took about a week, now it only takes minutes. The developers are happy. That department is also quite happy." +{{< /case-studies/quote >}} -
-
- "Our customers, from very big telecommunications operators to banks, love the idea of cloud native. They like Kubernetes technology. But they need to spend a lot of time to decompose their applications to turn them into microservice architecture, and as a solution provider, we help them." -
-
+

And not least of all, there's an impact on the bottom line. Says Hou: "We also see significant operating expense spending cut in some circumstances 20-30 percent, which is very helpful for our business."

-
-
- Within Huawei itself, once his team completes the transition of the internal business procedure department to Kubernetes, Hou is looking to convince more departments to move over to the cloud native development cycle and practice. "We have a lot of software developers, so we will provide them with our platform as a service solution, our own product," he says. "We would like to see significant cuts in their iteration cycle."

- Having overseen the initial move to Kubernetes at Huawei, Hou has advice for other companies considering the technology: "When you start to design the architecture of your application, think about cloud native, think about microservice architecture from the beginning," he says. "I think you will benefit from that."

- But if you already have legacy applications, "start from some microservice-friendly part of those applications first, parts that are relatively easy to be decomposed into simpler pieces and are relatively lightweight," Hou says. "Don’t think from day one that within how many days I want to move the whole architecture, or move everything into microservices. Don’t put that as a kind of target. You should do it in a gradual manner. And I would say for legacy applications, not every piece would be suitable for microservice architecture. No need to force it."

- After all, as enthusiastic as Hou is about Kubernetes at Huawei, he estimates that "in the next 10 years, maybe 80 percent of the workload can be distributed, can be run on the cloud native environments. There’s still 20 percent that’s not, but it’s fine. If we can make 80 percent of our workload really be cloud native, to have agility, it’s a much better world at the end of the day." +

Pleased with those initial results, and seeing a demand for cloud native technologies from its customers, Huawei doubled down on Kubernetes. In the spring of 2016, the company became not only a user but also a vendor.

-
-
+

"We built the Kubernetes technologies into our solutions," says Hou, referring to Huawei's FusionStage™ PaaS offering. "Our customers, from very big telecommunications operators to banks, love the idea of cloud native. They like Kubernetes technology. But they need to spend a lot of time to decompose their applications to turn them into microservice architecture, and as a solution provider, we help them. We've started to work with some Chinese banks, and we see a lot of interest from our customers like China Mobile and Deutsche Telekom."

-
-
- "In the next 10 years, maybe 80 percent of the workload can be distributed, can be run on the cloud native environments. There’s still 20 percent that’s not, but it’s fine. If we can make 80 percent of our workload really be cloud native, to have agility, it’s a much better world at the end of the day." +

"If you're just a user, you're just a user," adds Hou. "But if you're a vendor, in order to even convince your customers, you should use it yourself. Luckily because Huawei has a lot of employees, we can demonstrate the scale of cloud we can build using this technology. We provide customer wisdom." While Huawei has its own private cloud, many of its customers run cross-cloud applications using Huawei's solutions. It's a big selling point that most of the public cloud providers now support Kubernetes. "This makes the cross-cloud transition much easier than with other solutions," says Hou.

-
-
-
-
- In the nearer future, Hou is looking forward to new features that are being developed around Kubernetes, not least of all the ones that Huawei is contributing to. Huawei engineers have worked on the federation feature (which puts multiple Kubernetes clusters in a single framework to be managed seamlessly), scheduling, container networking and storage, and a just-announced technology called Container Ops, which is a DevOps pipeline engine. "This will put every DevOps job into a container," he explains. "And then this container mechanism is running using Kubernetes, but is also used to test Kubernetes. With that mechanism, we can make the containerized DevOps jobs be created, shared and managed much more easily than before."

- Still, Hou sees this technology as only halfway to its full potential. First and foremost, he’d like to expand the scale it can orchestrate, which is important for supersized companies like Huawei – as well as some of its customers.

- Hou proudly notes that two years after that first Huawei engineer became a contributor to and evangelist for Kubernetes, Huawei is now a top contributor to the community. "We’ve learned that the more you contribute to the community," he says, "the more you get back." +{{< case-studies/quote image="/images/case-studies/huawei/banner4.jpg" >}} +"Our customers, from very big telecommunications operators to banks, love the idea of cloud native. They like Kubernetes technology. But they need to spend a lot of time to decompose their applications to turn them into microservice architecture, and as a solution provider, we help them." +{{< /case-studies/quote >}} -
-
+

Within Huawei itself, once his team completes the transition of the internal business procedure department to Kubernetes, Hou is looking to convince more departments to move over to the cloud native development cycle and practice. "We have a lot of software developers, so we will provide them with our platform as a service solution, our own product," he says. "We would like to see significant cuts in their iteration cycle."

+ +

Having overseen the initial move to Kubernetes at Huawei, Hou has advice for other companies considering the technology: "When you start to design the architecture of your application, think about cloud native, think about microservice architecture from the beginning," he says. "I think you will benefit from that."

+ +

But if you already have legacy applications, "start from some microservice-friendly part of those applications first, parts that are relatively easy to be decomposed into simpler pieces and are relatively lightweight," Hou says. "Don't think from day one that within how many days I want to move the whole architecture, or move everything into microservices. Don't put that as a kind of target. You should do it in a gradual manner. And I would say for legacy applications, not every piece would be suitable for microservice architecture. No need to force it."

+ +

After all, as enthusiastic as Hou is about Kubernetes at Huawei, he estimates that "in the next 10 years, maybe 80 percent of the workload can be distributed, can be run on the cloud native environments. There's still 20 percent that's not, but it's fine. If we can make 80 percent of our workload really be cloud native, to have agility, it's a much better world at the end of the day."

+ +{{< case-studies/quote >}} +"In the next 10 years, maybe 80 percent of the workload can be distributed, can be run on the cloud native environments. There's still 20 percent that's not, but it's fine. If we can make 80 percent of our workload really be cloud native, to have agility, it's a much better world at the end of the day." +{{< /case-studies/quote >}} + +

In the nearer future, Hou is looking forward to new features that are being developed around Kubernetes, not least of all the ones that Huawei is contributing to. Huawei engineers have worked on the federation feature (which puts multiple Kubernetes clusters in a single framework to be managed seamlessly), scheduling, container networking and storage, and a just-announced technology called Container Ops, which is a DevOps pipeline engine. "This will put every DevOps job into a container," he explains. "And then this container mechanism is running using Kubernetes, but is also used to test Kubernetes. With that mechanism, we can make the containerized DevOps jobs be created, shared and managed much more easily than before."

+ +

Still, Hou sees this technology as only halfway to its full potential. First and foremost, he'd like to expand the scale it can orchestrate, which is important for supersized companies like Huawei – as well as some of its customers.

+ +

Hou proudly notes that two years after that first Huawei engineer became a contributor to and evangelist for Kubernetes, Huawei is now a top contributor to the community. "We've learned that the more you contribute to the community," he says, "the more you get back."

diff --git a/content/en/case-studies/ibm/index.html b/content/en/case-studies/ibm/index.html index e9a78a9443..aa3f108e1c 100644 --- a/content/en/case-studies/ibm/index.html +++ b/content/en/case-studies/ibm/index.html @@ -1,108 +1,80 @@ --- title: IBM Case Study - linkTitle: IBM case_study_styles: true cid: caseStudies -css: /css/style_case_studies.css logo: ibm_featured_logo.svg featured: false + +new_case_study_styles: true +heading_background: /images/case-studies/ibm/banner1.jpg +heading_title_logo: /images/ibm_logo.png +subheading: > + Building an Image Trust Service on Kubernetes with Notary and TUF +case_study_details: + - Company: IBM + - Location: Armonk, New York + - Industry: Cloud Computing --- -
-

CASE STUDY:
Building an Image Trust Service on Kubernetes with Notary and TUF

+

Challenge

-
+

IBM Cloud offers public, private, and hybrid cloud functionality across a diverse set of runtimes from its OpenWhisk-based function as a service (FaaS) offering, managed Kubernetes and containers, to Cloud Foundry platform as a service (PaaS). These runtimes are combined with the power of the company's enterprise technologies, such as MQ and DB2, its modern artificial intelligence (AI) Watson, and data analytics services. Users of IBM Cloud can exploit capabilities from more than 170 different cloud native services in its catalog, including capabilities such as IBM's Weather Company API and data services. In the later part of 2017, the IBM Cloud Container Registry team wanted to build out an image trust service.

-
- Company  IBM     Location  Armonk, New York     Industry  Cloud Computing -
- -
-
-
-
-

Challenge

- IBM Cloud offers public, private, and hybrid cloud functionality across a diverse set of runtimes from its OpenWhisk-based function as a service (FaaS) offering, managed Kubernetes and containers, to Cloud Foundry platform as a service (PaaS). These runtimes are combined with the power of the company’s enterprise technologies, such as MQ and DB2, its modern artificial intelligence (AI) Watson, and data analytics services. Users of IBM Cloud can exploit capabilities from more than 170 different cloud native services in its catalog, including capabilities such as IBM’s Weather Company API and data services. In the later part of 2017, the IBM Cloud Container Registry team wanted to build out an image trust service. -

-

Solution

- The work on this new service culminated with its public availability in the IBM Cloud in February 2018. The image trust service, called Portieris, is fully based on the Cloud Native Computing Foundation (CNCF) open source project Notary, according to Michael Hough, a software developer with the IBM Cloud Container Registry team. Portieris is a Kubernetes admission controller for enforcing content trust. Users can create image security policies for each Kubernetes namespace, or at the cluster level, and enforce different levels of trust for different images. Portieris is a key part of IBM’s trust story, since it makes it possible for users to consume the company’s Notary offering from within their IKS clusters. The offering is that Notary server runs in IBM’s cloud, and then Portieris runs inside the IKS cluster. This enables users to be able to have their IKS cluster verify that the image they're loading containers from contains exactly what they expect it to, and Portieris is what allows an IKS cluster to apply that verification. - -
- -
+

Solution

+

The work on this new service culminated with its public availability in the IBM Cloud in February 2018. The image trust service, called Portieris, is fully based on the Cloud Native Computing Foundation (CNCF) open source project Notary, according to Michael Hough, a software developer with the IBM Cloud Container Registry team. Portieris is a Kubernetes admission controller for enforcing content trust. Users can create image security policies for each Kubernetes namespace, or at the cluster level, and enforce different levels of trust for different images. Portieris is a key part of IBM's trust story, since it makes it possible for users to consume the company's Notary offering from within their IKS clusters. The offering is that Notary server runs in IBM's cloud, and then Portieris runs inside the IKS cluster. This enables users to be able to have their IKS cluster verify that the image they're loading containers from contains exactly what they expect it to, and Portieris is what allows an IKS cluster to apply that verification.

Impact

- IBM's intention in offering a managed Kubernetes container service and image registry is to provide a fully secure end-to-end platform for its enterprise customers. "Image signing is one key part of that offering, and our container registry team saw Notary as the de facto way to implement that capability in the current Docker and container ecosystem," Hough says. The company had not been offering image signing before, and Notary is the tool it used to implement that capability. "We had a multi-tenant Docker Registry with private image hosting," Hough says. "The Docker Registry uses hashes to ensure that image content is correct, and data is encrypted both in flight and at rest. But it does not provide any guarantees of who pushed an image. We used Notary to enable users to sign images in their private registry namespaces if they so choose." +

IBM's intention in offering a managed Kubernetes container service and image registry is to provide a fully secure end-to-end platform for its enterprise customers. "Image signing is one key part of that offering, and our container registry team saw Notary as the de facto way to implement that capability in the current Docker and container ecosystem," Hough says. The company had not been offering image signing before, and Notary is the tool it used to implement that capability. "We had a multi-tenant Docker Registry with private image hosting," Hough says. "The Docker Registry uses hashes to ensure that image content is correct, and data is encrypted both in flight and at rest. But it does not provide any guarantees of who pushed an image. We used Notary to enable users to sign images in their private registry namespaces if they so choose."

-
+{{< case-studies/quote author="Michael Hough, a software developer with the IBM Container Registry team" >}} +"We see CNCF as a safe haven for cloud native open source, providing stability, longevity, and expected maintenance for member projects—no matter the originating vendor or project." +{{< /case-studies/quote >}} -
-
-
-
- "We see CNCF as a safe haven for cloud native open source, providing stability, longevity, and expected maintenance for member projects—no matter the originating vendor or project."

- Michael Hough, a software developer with the IBM Container Registry team -
-
-
-
-

Docker had already created the Notary project as an implementation of The Update Framework (TUF), and this implementation of TUF provided the capabilities for Docker Content Trust.

"After contribution to CNCF of both TUF and Notary, we perceived that it was becoming the de facto standard for image signing in the container ecosystem", says Michael Hough, a software developer with the IBM Cloud Container Registry team. -

-The key reason for selecting Notary was that it was already compatible with the existing authentication stack IBM’s container registry was using. So was the design of TUF, which does not require the registry team to have to enter the business of key management. Both of these were "attractive design decisions that confirmed our choice of Notary," he says. -

-The introduction of Notary to implement image signing capability in IBM Cloud encourages increased security across IBM's cloud platform, "where we expect it will include both the signing of official IBM images as well as expected use by security-conscious enterprise customers," Hough says. "When combined with security policy implementations, we expect an increased use of deployment policies in CI/CD pipelines that allow for fine-grained control of service deployment based on image signers." -The availability of image signing "is a huge benefit to security-conscious customers who require this level of image provenance and security," Hough says. "With our IBM Cloud Kubernetes as-a-service offering and the admission controller we have made available, it allows both IBM services as well as customers of the IBM public cloud to use security policies to control service deployment." +{{< case-studies/lead >}} +Docker had already created the Notary project as an implementation of The Update Framework (TUF), and this implementation of TUF provided the capabilities for Docker Content Trust. +{{< /case-studies/lead >}} +

"After contribution to CNCF of both TUF and Notary, we perceived that it was becoming the de facto standard for image signing in the container ecosystem", says Michael Hough, a software developer with the IBM Cloud Container Registry team.

-
-
-
-
- "Image signing is one key part of our Kubernetes container service offering, and our container registry team saw Notary as the de facto way to implement that capability in the current Docker and container ecosystem"

- Michael Hough, a software developer with the IBM Cloud Container Registry team -
-
-
-
- Now that the Notary-implemented service is generally available in IBM’s public cloud as a component of its existing IBM Cloud Container Registry, it is deployed as a highly available service across five IBM Cloud regions. This high-availability deployment has three instances across two zones in each of the five regions, load balanced with failover support. "We have also deployed it with end-to-end TLS support through to our back-end IBM Cloudant persistence storage service," Hough says. -

- The IBM team has created and open sourced a Kubernetes admission controller called Portieris, which uses Notary signing information combined with customer-defined security policies to control image deployment into their cluster. "We are hoping to drive adoption of Portieris through its use of our Notary offering," Hough says. -

- IBM has been a key player in the creation and support of open source foundations, including CNCF. Todd Moore, IBM's vice president of Open Technology, is the current CNCF governing board chair and a number of IBMers are active across many of the CNCF member projects. +

The key reason for selecting Notary was that it was already compatible with the existing authentication stack IBM's container registry was using. So was the design of TUF, which does not require the registry team to have to enter the business of key management. Both of these were "attractive design decisions that confirmed our choice of Notary," he says.

+

The introduction of Notary to implement image signing capability in IBM Cloud encourages increased security across IBM's cloud platform, "where we expect it will include both the signing of official IBM images as well as expected use by security-conscious enterprise customers," Hough says. "When combined with security policy implementations, we expect an increased use of deployment policies in CI/CD pipelines that allow for fine-grained control of service deployment based on image signers."

+

The availability of image signing "is a huge benefit to security-conscious customers who require this level of image provenance and security," Hough says. "With our IBM Cloud Kubernetes as-a-service offering and the admission controller we have made available, it allows both IBM services as well as customers of the IBM public cloud to use security policies to control service deployment."

-
-
-
-
- "With our IBM Cloud Kubernetes as-a-service offering and the admission controller we have made available, it allows both IBM services as well as customers of the IBM public cloud to use security policies to control service deployment."

- Michael Hough, a software developer with the IBM Cloud Container Registry team -
-
- +{{< case-studies/quote + image="/images/case-studies/ibm/banner3.jpg" + author="Michael Hough, a software developer with the IBM Cloud Container Registry team" +>}} +"Image signing is one key part of our Kubernetes container service offering, and our container registry team saw Notary as the de facto way to implement that capability in the current Docker and container ecosystem" +{{< /case-studies/quote >}} -
-
- "Given that, we see CNCF as a safe haven for cloud native open source, providing stability, longevity, and expected maintenance for member projects—no matter the originating vendor or project," Hough says. Because the entire cloud native world is a fast-moving area with many competing vendors and solutions, "we see the CNCF model as an arbiter of openness and fair play across the ecosystem," he says. -

-With both TUF and Notary as part of CNCF, IBM expects there to be standardization around these capabilities beyond just de facto standards for signing and provenance. IBM has determined to not simply consume Notary, but also to contribute to the open source project where applicable. "IBMers have contributed a CouchDB backend to support our use of IBM Cloudant as the persistent store; and are working on generalization of the pkcs11 provider, allowing support of other security hardware devices beyond Yubikey," Hough says. +

Now that the Notary-implemented service is generally available in IBM's public cloud as a component of its existing IBM Cloud Container Registry, it is deployed as a highly available service across five IBM Cloud regions. This high-availability deployment has three instances across two zones in each of the five regions, load balanced with failover support. "We have also deployed it with end-to-end TLS support through to our back-end IBM Cloudant persistence storage service," Hough says.

-
-
-
-
- "There are new projects addressing these challenges, including within CNCF. We will definitely be following these advancements with interest. We found the Notary community to be an active and friendly community open to changes, such as our addition of a CouchDB backend for persistent storage."

- Michael Hough, a software developer with the IBM Cloud Container Registry team -
-
-
-
-The company has used other CNCF projects containerd, Envoy, Prometheus, gRPC, and CNI, and is looking into SPIFFE and SPIRE as well for potential future use. -

-What advice does Hough have for other companies that are looking to deploy Notary or a cloud native infrastructure? -

-"While this is true for many areas of cloud native infrastructure software, we found that a high-availability, multi-region deployment of Notary requires a solid implementation to handle certificate management and rotation," he says. "There are new projects addressing these challenges, including within CNCF. We will definitely be following these advancements with interest. We found the Notary community to be an active and friendly community open to changes, such as our addition of a CouchDB backend for persistent storage." +

The IBM team has created and open sourced a Kubernetes admission controller called Portieris, which uses Notary signing information combined with customer-defined security policies to control image deployment into their cluster. "We are hoping to drive adoption of Portieris through its use of our Notary offering," Hough says.

-
+

IBM has been a key player in the creation and support of open source foundations, including CNCF. Todd Moore, IBM's vice president of Open Technology, is the current CNCF governing board chair and a number of IBMers are active across many of the CNCF member projects.

-
+{{< case-studies/quote + image="/images/case-studies/ibm/banner4.jpg" + author="Michael Hough, a software developer with the IBM Cloud Container Registry team" +>}} +"With our IBM Cloud Kubernetes as-a-service offering and the admission controller we have made available, it allows both IBM services as well as customers of the IBM public cloud to use security policies to control service deployment." +{{< /case-studies/quote >}} + +

"Given that, we see CNCF as a safe haven for cloud native open source, providing stability, longevity, and expected maintenance for member projects—no matter the originating vendor or project," Hough says. Because the entire cloud native world is a fast-moving area with many competing vendors and solutions, "we see the CNCF model as an arbiter of openness and fair play across the ecosystem," he says.

+ +

With both TUF and Notary as part of CNCF, IBM expects there to be standardization around these capabilities beyond just de facto standards for signing and provenance. IBM has determined to not simply consume Notary, but also to contribute to the open source project where applicable. "IBMers have contributed a CouchDB backend to support our use of IBM Cloudant as the persistent store; and are working on generalization of the pkcs11 provider, allowing support of other security hardware devices beyond Yubikey," Hough says.

+ +{{< case-studies/quote author="Michael Hough, a software developer with the IBM Cloud Container Registry team" >}} +"There are new projects addressing these challenges, including within CNCF. We will definitely be following these advancements with interest. We found the Notary community to be an active and friendly community open to changes, such as our addition of a CouchDB backend for persistent storage." +{{< /case-studies/quote >}} + +

The company has used other CNCF projects containerd, Envoy, Prometheus, gRPC, and CNI, and is looking into SPIFFE and SPIRE as well for potential future use.

+ +

What advice does Hough have for other companies that are looking to deploy Notary or a cloud native infrastructure?

+ +

"While this is true for many areas of cloud native infrastructure software, we found that a high-availability, multi-region deployment of Notary requires a solid implementation to handle certificate management and rotation," he says. "There are new projects addressing these challenges, including within CNCF. We will definitely be following these advancements with interest. We found the Notary community to be an active and friendly community open to changes, such as our addition of a CouchDB backend for persistent storage."

diff --git a/content/en/case-studies/ing/index.html b/content/en/case-studies/ing/index.html index 943daec2de..037ba9775d 100644 --- a/content/en/case-studies/ing/index.html +++ b/content/en/case-studies/ing/index.html @@ -5,95 +5,74 @@ case_study_styles: true cid: caseStudies weight: 50 featured: true -css: /css/style_case_studies.css quote: > - The big cloud native promise to our business is the ability to go from idea to production within 48 hours. We are some years away from this, but that’s quite feasible to us. + The big cloud native promise to our business is the ability to go from idea to production within 48 hours. We are some years away from this, but that's quite feasible to us. + +new_case_study_styles: true +heading_background: /images/case-studies/ing/banner1.jpg +heading_title_logo: /images/ing_logo.png +subheading: > + Driving Banking Innovation with Cloud Native +case_study_details: + - Company: ING + - Location: Amsterdam, Netherlands + - Industry: Finance --- +

Challenge

-
-

CASE STUDY:
Driving Banking Innovation with Cloud Native -

+

After undergoing an agile transformation, ING realized it needed a standardized platform to support the work their developers were doing. "Our DevOps teams got empowered to be autonomous," says Infrastructure Architect Thijs Ebbers. "It has benefits; you get all kinds of ideas. But a lot of teams are going to devise the same wheel. Teams started tinkering with Docker, Docker Swarm, Kubernetes, Mesos. Well, it's not really useful for a company to have one hundred wheels, instead of one good wheel.

-
+

Solution

-
- Company  ING     Location  Amsterdam, Netherlands -     Industry  Finance -
+

Using Kubernetes for container orchestration and Docker for containerization, the ING team began building an internal public cloud for its CI/CD pipeline and green-field applications. The pipeline, which has been built on Mesos Marathon, will be migrated onto Kubernetes. The bank-account management app Yolt in the U.K. (and soon France and Italy) market already is live hosted on a Kubernetes framework. At least two greenfield projects currently on the Kubernetes framework will be going into production later this year. By the end of 2018, the company plans to have converted a number of APIs used in the banking customer experience to cloud native APIs and host these on the Kubernetes-based platform.

-
-
-
-
-

Challenge

- After undergoing an agile transformation, ING realized it needed a standardized platform to support the work their developers were doing. "Our DevOps teams got empowered to be autonomous," says Infrastructure Architect Thijs Ebbers. "It has benefits; you get all kinds of ideas. But a lot of teams are going to devise the same wheel. Teams started tinkering with Docker, Docker Swarm, Kubernetes, Mesos. Well, it’s not really useful for a company to have one hundred wheels, instead of one good wheel. -
-
-

Solution

- Using Kubernetes for container orchestration and Docker for containerization, the ING team began building an internal public cloud for its CI/CD pipeline and green-field applications. The pipeline, which has been built on Mesos Marathon, will be migrated onto Kubernetes. The bank-account management app Yolt in the U.K. (and soon France and Italy) market already is live hosted on a Kubernetes framework. At least two greenfield projects currently on the Kubernetes framework will be going into production later this year. By the end of 2018, the company plans to have converted a number of APIs used in the banking customer experience to cloud native APIs and host these on the Kubernetes-based platform. +

Impact

-
-
+

"Cloud native technologies are helping our speed, from getting an application to test to acceptance to production," says Infrastructure Architect Onno Van der Voort. "If you walk around ING now, you see all these DevOps teams, doing stand-ups, demoing. They try to get new functionality out there really fast. We held a hackathon for one of our existing components and basically converted it to cloud native within 2.5 days, though of course the tail takes more time before code is fully production ready."

-
-
-

Impact

- "Cloud native technologies are helping our speed, from getting an application to test to acceptance to production," says Infrastructure Architect Onno Van der Voort. "If you walk around ING now, you see all these DevOps teams, doing stand-ups, demoing. They try to get new functionality out there really fast. We held a hackathon for one of our existing components and basically converted it to cloud native within 2.5 days, though of course the tail takes more time before code is fully production ready." -
-
-
-
-
-
- "The big cloud native promise to our business is the ability to go from idea to production within 48 hours. We are some years away from this, but that’s quite feasible to us." -

— Thijs Ebbers, Infrastructure Architect, ING -
-
-
-
-

ING has long embraced innovation in banking, launching the internet-based ING Direct in 1997.

In that same spirit, the company underwent an agile transformation a few years ago. "Our DevOps teams got empowered to be autonomous," says Infrastructure Architect Thijs Ebbers. "It has benefits; you get all kinds of ideas. But a lot of teams are going to devise the same wheel. Teams started tinkering with Docker, Docker Swarm, Kubernetes, Mesos. Well, it’s not really useful for a company to have one hundred wheels, instead of one good wheel."

- Looking to standardize the deployment process within the company’s strict security guidelines, the team looked at several solutions and found that in the past year, "Kubernetes won the container management framework wars," says Ebbers. "We decided to standardize ING on a Kubernetes framework." Everything is run on premise due to banking regulations, he adds, but "we will be building an internal public cloud. We are trying to get on par with what public clouds are doing. That’s one of the reasons we got Kubernetes."

- They also embraced Docker to address a major pain point in ING’s CI/CD pipeline. Before containerization, "Every development team had to order a VM, and it was quite a heavy delivery model for them," says Infrastructure Architect Onno Van der Voort. "Another use case for containerization is when the application travels through the pipeline, they fire up Docker containers to do test work against the applications and after they’ve done the work, the containers get killed again." +{{< case-studies/quote author="Thijs Ebbers, Infrastructure Architect, ING">}} +"The big cloud native promise to our business is the ability to go from idea to production within 48 hours. We are some years away from this, but that's quite feasible to us." +{{< /case-studies/quote >}} -
-
-
-
- "We decided to standardize ING on a Kubernetes framework." Everything is run on premise due to banking regulations, he adds, but "we will be building an internal public cloud. We are trying to get on par with what public clouds are doing. That’s one of the reasons we got Kubernetes." -

— Thijs Ebbers, Infrastructure Architect, ING -
-
-
-
- Because of industry regulations, applications are only allowed to go through the pipeline, where compliance is enforced, rather than be deployed directly into a container. "We have to run the complete platform of services we need, many routing from different places," says Van der Voort. "We need this Kubernetes framework for deploying the containers, with all those components, monitoring, logging. It’s complex." For that reason, ING has chosen to start on the OpenShift Origin Kubernetes distribution.

- Already, "cloud native technologies are helping our speed, from getting an application to test to acceptance to production," says Van der Voort. "If you walk around ING now, you see all these DevOps teams, doing stand-ups, demoing. They try to get new functionality out there really fast. We held a hackathon for one of our existing components and basically converted it to cloud native within 2.5 days, though of course the tail takes more time before code is fully production ready."

- The pipeline, which has been built on Mesos Marathon, will be migrated onto Kubernetes. Some legacy applications are also being rewritten as cloud native in order to run on the framework. At least two smaller greenfield projects built on Kubernetes will go into production this year. By the end of 2018, the company plans to have converted a number of APIs used in the banking customer experience to cloud native APIs and host these on the Kubernetes-based platform. +{{< case-studies/lead >}} +ING has long embraced innovation in banking, launching the internet-based ING Direct in 1997. +{{< /case-studies/lead >}} -
-
-
-
- "We have to run the complete platform of services we need, many routing from different places. We need this Kubernetes framework for deploying the containers, with all those components, monitoring, logging. It’s complex."

— Onno Van der Voort, Infrastructure Architect, ING -
-
+

In that same spirit, the company underwent an agile transformation a few years ago. "Our DevOps teams got empowered to be autonomous," says Infrastructure Architect Thijs Ebbers. "It has benefits; you get all kinds of ideas. But a lot of teams are going to devise the same wheel. Teams started tinkering with Docker, Docker Swarm, Kubernetes, Mesos. Well, it's not really useful for a company to have one hundred wheels, instead of one good wheel."

+

Looking to standardize the deployment process within the company's strict security guidelines, the team looked at several solutions and found that in the past year, "Kubernetes won the container management framework wars," says Ebbers. "We decided to standardize ING on a Kubernetes framework." Everything is run on premise due to banking regulations, he adds, but "we will be building an internal public cloud. We are trying to get on par with what public clouds are doing. That's one of the reasons we got Kubernetes."

-
- The team, however, doesn’t see the bank’s back-end systems going onto the Kubernetes platform. "Our philosophy is it only makes sense to move things to cloud if they are cloud native," says Van der Voort. "If you have traditional architecture, build traditional patterns, it doesn’t hold any value to go to the cloud." Adds Cloud Platform Architect Alfonso Fernandez-Barandiaran: "ING has a strategy about where we will go, in order to improve our agility. So it’s not about how cool this technology is, it’s about finding the right technology and the right approach."

- The Kubernetes framework will be hosting some greenfield projects that are high priority for ING: applications the company is developing in response to PSD2, the European Commission directive requiring more innovative online and mobile payments that went into effect at the beginning of 2018. For example, a bank-account management app called Yolt, serving the U.K. market (and soon France and Italy), was built on a Kubernetes platform and has gone into production. ING is also developing blockchain-enabled applications that will live on the Kubernetes platform. "We’ve been contacted by a lot of development teams that have ideas with what they want to do with containers," says Ebbers. +

They also embraced Docker to address a major pain point in ING's CI/CD pipeline. Before containerization, "Every development team had to order a VM, and it was quite a heavy delivery model for them," says Infrastructure Architect Onno Van der Voort. "Another use case for containerization is when the application travels through the pipeline, they fire up Docker containers to do test work against the applications and after they've done the work, the containers get killed again."

-
-
-
-
-Even with the particular requirements that come in banking, ING has managed to take a lead in technology and innovation. "Every time we have constraints, we look for maybe a better way that we can use this technology."

— Alfonso Fernandez-Barandiaran, Cloud Platform Architect, ING
-
+{{< case-studies/quote + image="/images/case-studies/ing/banner3.jpg" + author="Thijs Ebbers, Infrastructure Architect, ING" +>}} +"We decided to standardize ING on a Kubernetes framework." Everything is run on premise due to banking regulations, he adds, but "we will be building an internal public cloud. We are trying to get on par with what public clouds are doing. That's one of the reasons we got Kubernetes." +{{< /case-studies/quote >}} -
- Even with the particular requirements that come in banking, ING has managed to take a lead in technology and innovation. "Every time we have constraints, we look for maybe a better way that we can use this technology," says Fernandez-Barandiaran.

- The results, after all, are worth the effort. "The big cloud native promise to our business is the ability to go from idea to production within 48 hours," says Ebbers. "That would require all these projects to be mature. We are some years away from this, but that’s quite feasible to us." +

Because of industry regulations, applications are only allowed to go through the pipeline, where compliance is enforced, rather than be deployed directly into a container. "We have to run the complete platform of services we need, many routing from different places," says Van der Voort. "We need this Kubernetes framework for deploying the containers, with all those components, monitoring, logging. It's complex." For that reason, ING has chosen to start on the OpenShift Origin Kubernetes distribution.

-
+

Already, "cloud native technologies are helping our speed, from getting an application to test to acceptance to production," says Van der Voort. "If you walk around ING now, you see all these DevOps teams, doing stand-ups, demoing. They try to get new functionality out there really fast. We held a hackathon for one of our existing components and basically converted it to cloud native within 2.5 days, though of course the tail takes more time before code is fully production ready."

-
+

The pipeline, which has been built on Mesos Marathon, will be migrated onto Kubernetes. Some legacy applications are also being rewritten as cloud native in order to run on the framework. At least two smaller greenfield projects built on Kubernetes will go into production this year. By the end of 2018, the company plans to have converted a number of APIs used in the banking customer experience to cloud native APIs and host these on the Kubernetes-based platform.

+ +{{< case-studies/quote + image="/images/case-studies/ing/banner4.jpg" + author="Onno Van der Voort, Infrastructure Architect, ING" +>}} +"We have to run the complete platform of services we need, many routing from different places. We need this Kubernetes framework for deploying the containers, with all those components, monitoring, logging. It's complex." +{{< /case-studies/quote >}} + +

The team, however, doesn't see the bank's back-end systems going onto the Kubernetes platform. "Our philosophy is it only makes sense to move things to cloud if they are cloud native," says Van der Voort. "If you have traditional architecture, build traditional patterns, it doesn't hold any value to go to the cloud." Adds Cloud Platform Architect Alfonso Fernandez-Barandiaran: "ING has a strategy about where we will go, in order to improve our agility. So it's not about how cool this technology is, it's about finding the right technology and the right approach."

+ +

The Kubernetes framework will be hosting some greenfield projects that are high priority for ING: applications the company is developing in response to PSD2, the European Commission directive requiring more innovative online and mobile payments that went into effect at the beginning of 2018. For example, a bank-account management app called Yolt, serving the U.K. market (and soon France and Italy), was built on a Kubernetes platform and has gone into production. ING is also developing blockchain-enabled applications that will live on the Kubernetes platform. "We've been contacted by a lot of development teams that have ideas with what they want to do with containers," says Ebbers.

+ +{{< case-studies/quote author="Alfonso Fernandez-Barandiaran, Cloud Platform Architect, ING" >}} +Even with the particular requirements that come in banking, ING has managed to take a lead in technology and innovation. "Every time we have constraints, we look for maybe a better way that we can use this technology." +{{< /case-studies/quote >}} + +

Even with the particular requirements that come in banking, ING has managed to take a lead in technology and innovation. "Every time we have constraints, we look for maybe a better way that we can use this technology," says Fernandez-Barandiaran.

+ +

The results, after all, are worth the effort. "The big cloud native promise to our business is the ability to go from idea to production within 48 hours," says Ebbers. "That would require all these projects to be mature. We are some years away from this, but that's quite feasible to us."

diff --git a/content/en/case-studies/jd-com/index.html b/content/en/case-studies/jd-com/index.html index aed12fc54b..ae3360216b 100644 --- a/content/en/case-studies/jd-com/index.html +++ b/content/en/case-studies/jd-com/index.html @@ -3,95 +3,77 @@ title: JD.com Case Study linkTitle: jd-com case_study_styles: true cid: caseStudies -css: /css/style_case_studies.css featured: false + +new_case_study_styles: true +heading_background: /images/case-studies/jdcom/banner1.jpg +heading_title_logo: /images/jdcom_logo.png +subheading: > + JD.com: How JD.com Pioneered Kubernetes for E-Commerce at Hyperscale +case_study_details: + - Company: JD.com + - Location: Beijing, China + - Industry: eCommerce --- -
-

CASE STUDY:
JD.com: How JD.com Pioneered Kubernetes for E-Commerce at Hyperscale +

Challenge

-

+

With more than 300 million active users and total 2017 revenue of more than $55 billion, JD.com is China's largest retailer, and its operations are the epitome of hyperscale. For example, there are more than a trillion images in JD.com's product databases—with 100 million being added daily—and this enormous amount of data needs to be instantly accessible. In 2014, JD.com moved its applications to containers running on bare metal machines using OpenStack and Docker to "speed up the delivery of our computing resources and make the operations much simpler," says Haifeng Liu, JD.com's Chief Architect. But by the end of 2015, with tens of thousands of nodes running in multiple data centers, "we encountered a lot of problems because our platform was not strong enough, and we suffered from bottlenecks and scalability issues," says Liu. "We needed infrastructure for the next five years of development, now."

-
+

Solution

-
- Company  JD.com     Location  Beijing, China     Industry  eCommerce -
+

JD.com turned to Kubernetes to accommodate its clusters. At the beginning of 2016, the company began to transition from OpenStack to Kubernetes, and today, JD.com runs the world's largest Kubernetes cluster. "Kubernetes has provided a strong foundation on top of which we have customized the solution to suit our needs as China's largest retailer."

-
-
-
-
-

Challenge

- With more than 300 million active users and total 2017 revenue of more than $55 billion, JD.com is China’s largest retailer, and its operations are the epitome of hyperscale. For example, there are more than a trillion images in JD.com’s product databases—with 100 million being added daily—and this enormous amount of data needs to be instantly accessible. In 2014, JD.com moved its applications to containers running on bare metal machines using OpenStack and Docker to "speed up the delivery of our computing resources and make the operations much simpler," says Haifeng Liu, JD.com’s Chief Architect. But by the end of 2015, with tens of thousands of nodes running in multiple data centers, "we encountered a lot of problems because our platform was not strong enough, and we suffered from bottlenecks and scalability issues," says Liu. "We needed infrastructure for the next five years of development, now." - -

Solution

- JD.com turned to Kubernetes to accommodate its clusters. At the beginning of 2016, the company began to transition from OpenStack to Kubernetes, and today, JD.com runs the world’s largest Kubernetes cluster. "Kubernetes has provided a strong foundation on top of which we have customized the solution to suit our needs as China’s largest retailer." -

Impact

- "We have greater data center efficiency, better managed resources, and smarter deployment with the Kubernetes platform," says Liu. Deployment time went from several hours to tens of seconds. Efficiency has improved by 20-30%, measured in IT costs. With the further optimizations the team is working on, Liu believes there is the potential to save hundreds of millions of dollars a year. But perhaps the best indication of success was the annual Singles Day shopping event, which ran on the Kubernetes platform for the first time in 2018. Over 11 days, transaction volume on JD.com was $23 billion, and "our e-commerce platforms did great," says Liu. "Infrastructure led the way to prep for 11.11. We took the approach of predicting volume, emulating the behavior of customers to prepare beforehand, and drilled for malfunctions. Because of Kubernetes’s scalability, we were able to handle an extremely high level of demand." -
-
-
-
-
- "Kubernetes helped us reduce the complexity of operations to make distributed systems stable and scalable. Most importantly, we can leverage Kubernetes for scheduling resources to reduce hardware costs. That’s the big win." -

- HAIFENG LIU, CHIEF ARCHITECT, JD.com -
-
-
-
-

With more than 300 million active users and $55.7 billion in annual revenues last year, JD.com is China’s largest retailer, and its operations are the epitome of hyperscale.

- For example, there are more than a trillion images in JD.com’s product databases for customers, with 100 million being added daily. And this enormous amount of data needs to be instantly accessible to enable a smooth online customer experience. -

- In 2014, JD.com moved its applications to containers running on bare metal machines using OpenStack and Docker to "speed up the delivery of our computing resources and make the operations much simpler," says Haifeng Liu, JD.com’s Chief Architect. But by the end of 2015, with hundreds of thousands of nodes in multiple data centers, "we encountered a lot of problems because our platform was not strong enough, and we suffered from bottlenecks and scalability issues," Liu adds. "We needed infrastructure for the next five years of development, now." -

- After considering a number of orchestration technologies, JD.com decided to adopt Kubernetes to accommodate its ever-growing clusters. "The main reason is because Kubernetes can give us more efficient, scalable and much simpler application deployments, plus we can leverage it to do flexible platform scheduling," says Liu. +

"We have greater data center efficiency, better managed resources, and smarter deployment with the Kubernetes platform," says Liu. Deployment time went from several hours to tens of seconds. Efficiency has improved by 20-30%, measured in IT costs. With the further optimizations the team is working on, Liu believes there is the potential to save hundreds of millions of dollars a year. But perhaps the best indication of success was the annual Singles Day shopping event, which ran on the Kubernetes platform for the first time in 2018. Over 11 days, transaction volume on JD.com was $23 billion, and "our e-commerce platforms did great," says Liu. "Infrastructure led the way to prep for 11.11. We took the approach of predicting volume, emulating the behavior of customers to prepare beforehand, and drilled for malfunctions. Because of Kubernetes's scalability, we were able to handle an extremely high level of demand."

-
-
-
-
- "We customized Kubernetes and built a modern system on top of it. This entire ecosystem of Kubernetes plus our own optimizations have helped us save costs and time."

- HAIFENG LIU, CHIEF ARCHITECT, JD.com +{{< case-studies/quote author="HAIFENG LIU, CHIEF ARCHITECT, JD.com" >}} +"Kubernetes helped us reduce the complexity of operations to make distributed systems stable and scalable. Most importantly, we can leverage Kubernetes for scheduling resources to reduce hardware costs. That's the big win." +{{< /case-studies/quote >}} -
-
-
-
- The fact that Kubernetes is based on Google’s Borg also gave the company confidence. The team liked that Kubernetes has a clear and simple architecture, and that it’s developed mostly in Go, which is a popular language within JD.com. Though he felt that at the time Kubernetes "was not mature enough," Liu says, "we adopted it anyway." -

- The team spent a year developing the new container engine platform based on Kubernetes, and at the end of 2016, began promoting it within the company. "We wanted the cluster to be the default way for creating services, so scalability is easier," says Liu. "We talked to developers, interest grew, and we solved problems together." Some of these problems included networking performance and etcd scalability. "But during the past two years, Kubernetes has become more mature and very stable," he adds. -

- Today, the company runs the world’s largest Kubernetes cluster. "We customized Kubernetes and built a modern system on top of it," says Liu. "This entire ecosystem of Kubernetes plus our own optimizations have helped us save costs and time. We have greater data center efficiency, better managed resources, and smarter deployment with the Kubernetes platform." +{{< case-studies/lead >}} +With more than 300 million active users and $55.7 billion in annual revenues last year, JD.com is China's largest retailer, and its operations are the epitome of hyperscale. +{{< /case-studies/lead >}} -
-
-
-
- "My advice is first you need to combine this technology with your own businesses, and the second is you need clear goals. You cannot just use the technology because others are using it. You need to consider your own objectives."

- HAIFENG LIU, CHIEF ARCHITECT, JD.com -
-
+

For example, there are more than a trillion images in JD.com's product databases for customers, with 100 million being added daily. And this enormous amount of data needs to be instantly accessible to enable a smooth online customer experience.

-
-
- The results are clear: Deployment time went from several hours to tens of seconds. Efficiency has improved by 20-30%, measured in IT costs. But perhaps the best indication of success was the annual Singles Day shopping event, which ran on the Kubernetes platform for the first time in 2018. Over 11 days, transaction volume on JD.com was $23 billion, and "our e-commerce platforms did great," says Liu. "Infrastructure led the way to prep for 11.11. We took the approach of predicting volume, emulating the behavior of customers to prepare beforehand, and drilled for malfunctions. Because of Kubernetes’s scalability, we were able to handle an extremely high level of demand." -

- JD.com is now in its second stage with Kubernetes: The platform is already stable, scalable, and flexible, so the focus is on how to run things much more efficiently to further reduce costs. With the optimizations the team is working on with resource management, Liu believes there is the potential to save hundreds of millions of dollars a year. -

- "We run Kubernetes and container clusters on roughly tens of thousands of physical bare metal nodes," he says. "Using Kubernetes and leveraging our own machine learning pipeline to predict how many resources we need for each application we use, and our own intelligent scaling algorithm, we can improve our resource usage. If we boost the resource usage, for example, by several percent, that means we can reduce huge hardware costs. Then we don’t need that many servers to get that same amount of workload. That can save us a lot of resources." -
+

In 2014, JD.com moved its applications to containers running on bare metal machines using OpenStack and Docker to "speed up the delivery of our computing resources and make the operations much simpler," says Haifeng Liu, JD.com's Chief Architect. But by the end of 2015, with hundreds of thousands of nodes in multiple data centers, "we encountered a lot of problems because our platform was not strong enough, and we suffered from bottlenecks and scalability issues," Liu adds. "We needed infrastructure for the next five years of development, now."

-
-
-"We can share our successful experience with the community, and we also receive good feedback from others. So it’s mutually beneficial."

- HAIFENG LIU, CHIEF ARCHITECT, JD.com
-
+

After considering a number of orchestration technologies, JD.com decided to adopt Kubernetes to accommodate its ever-growing clusters. "The main reason is because Kubernetes can give us more efficient, scalable and much simpler application deployments, plus we can leverage it to do flexible platform scheduling," says Liu.

-
- JD.com, which won CNCF’s 2018 End User Award, is also using Helm, CNI, Harbor, and Vitess on its platform. JD.com developers have made considerable contributions to Vitess, the CNCF project for scalable MySQL cluster management, and the company hopes to donate its own project to CNCF in the near future. Community participation is a priority for JD.com. "We have a good partnership with this community," says Liu. "We can share our successful experience with the community, and we also receive good feedback from others. So it’s mutually beneficial." -

- To that end, Liu offers this advice for other companies considering adopting cloud native technology. "First you need to combine this technology with your own businesses, and the second is you need clear goals," he says. "You cannot just use the technology because others are using it. You need to consider your own objectives." -

- For JD.com’s objectives, these cloud native technologies have been an ideal fit with the company’s own homegrown innovation. "Kubernetes helped us reduce the complexity of operations to make distributed systems stable and scalable," says Liu. "Most importantly, we can leverage Kubernetes for scheduling resources to reduce hardware costs. That’s the big win." -
-
+{{< case-studies/quote + image="/images/case-studies/jdcom/banner3.jpg" + author="HAIFENG LIU, CHIEF ARCHITECT, JD.com" +>}} +"We customized Kubernetes and built a modern system on top of it. This entire ecosystem of Kubernetes plus our own optimizations have helped us save costs and time." +{{< /case-studies/quote >}} + +

The fact that Kubernetes is based on Google's Borg also gave the company confidence. The team liked that Kubernetes has a clear and simple architecture, and that it's developed mostly in Go, which is a popular language within JD.com. Though he felt that at the time Kubernetes "was not mature enough," Liu says, "we adopted it anyway."

+ +

The team spent a year developing the new container engine platform based on Kubernetes, and at the end of 2016, began promoting it within the company. "We wanted the cluster to be the default way for creating services, so scalability is easier," says Liu. "We talked to developers, interest grew, and we solved problems together." Some of these problems included networking performance and etcd scalability. "But during the past two years, Kubernetes has become more mature and very stable," he adds.

+ +

Today, the company runs the world's largest Kubernetes cluster. "We customized Kubernetes and built a modern system on top of it," says Liu. "This entire ecosystem of Kubernetes plus our own optimizations have helped us save costs and time. We have greater data center efficiency, better managed resources, and smarter deployment with the Kubernetes platform."

+ +{{< case-studies/quote + image="/images/case-studies/jdcom/banner4.jpg" + author="HAIFENG LIU, CHIEF ARCHITECT, JD.com" +>}} +"My advice is first you need to combine this technology with your own businesses, and the second is you need clear goals. You cannot just use the technology because others are using it. You need to consider your own objectives." +{{< /case-studies/quote >}} + +

The results are clear: Deployment time went from several hours to tens of seconds. Efficiency has improved by 20-30%, measured in IT costs. But perhaps the best indication of success was the annual Singles Day shopping event, which ran on the Kubernetes platform for the first time in 2018. Over 11 days, transaction volume on JD.com was $23 billion, and "our e-commerce platforms did great," says Liu. "Infrastructure led the way to prep for 11.11. We took the approach of predicting volume, emulating the behavior of customers to prepare beforehand, and drilled for malfunctions. Because of Kubernetes's scalability, we were able to handle an extremely high level of demand."

+ +

JD.com is now in its second stage with Kubernetes: The platform is already stable, scalable, and flexible, so the focus is on how to run things much more efficiently to further reduce costs. With the optimizations the team is working on with resource management, Liu believes there is the potential to save hundreds of millions of dollars a year.

+ +

"We run Kubernetes and container clusters on roughly tens of thousands of physical bare metal nodes," he says. "Using Kubernetes and leveraging our own machine learning pipeline to predict how many resources we need for each application we use, and our own intelligent scaling algorithm, we can improve our resource usage. If we boost the resource usage, for example, by several percent, that means we can reduce huge hardware costs. Then we don't need that many servers to get that same amount of workload. That can save us a lot of resources."

+ +{{< case-studies/quote author="HAIFENG LIU, CHIEF ARCHITECT, JD.com" >}} +"We can share our successful experience with the community, and we also receive good feedback from others. So it's mutually beneficial." +{{< /case-studies/quote >}} + +

JD.com, which won CNCF's 2018 End User Award, is also using Helm, CNI, Harbor, and Vitess on its platform. JD.com developers have made considerable contributions to Vitess, the CNCF project for scalable MySQL cluster management, and the company hopes to donate its own project to CNCF in the near future. Community participation is a priority for JD.com. "We have a good partnership with this community," says Liu. "We can share our successful experience with the community, and we also receive good feedback from others. So it's mutually beneficial."

+ +

To that end, Liu offers this advice for other companies considering adopting cloud native technology. "First you need to combine this technology with your own businesses, and the second is you need clear goals," he says. "You cannot just use the technology because others are using it. You need to consider your own objectives."

+ +

For JD.com's objectives, these cloud native technologies have been an ideal fit with the company's own homegrown innovation. "Kubernetes helped us reduce the complexity of operations to make distributed systems stable and scalable," says Liu. "Most importantly, we can leverage Kubernetes for scheduling resources to reduce hardware costs. That's the big win."

diff --git a/content/en/case-studies/naic/index.html b/content/en/case-studies/naic/index.html index 3deb91e480..23955bdef8 100644 --- a/content/en/case-studies/naic/index.html +++ b/content/en/case-studies/naic/index.html @@ -1,113 +1,87 @@ --- title: NAIC Case Study - linkTitle: NAIC case_study_styles: true cid: caseStudies -css: /css/style_case_studies.css logo: naic_featured_logo.png featured: false + +new_case_study_styles: true +heading_background: /images/case-studies/naic/banner1.jpg +heading_title_logo: /images/naic_logo.png +subheading: > + A Culture and Technology Transition Enabled by Kubernetes +case_study_details: + - Company: NAIC + - Location: Washington, DC + - Industry: Regulatory --- -
-

CASE STUDY:
A Culture and Technology Transition Enabled by Kubernetes

+

Challenge

-
+

The National Association of Insurance Commissioners (NAIC), the U.S. standard-setting and regulatory support organization, was looking for a way to deliver new services faster to provide more value for members and staff. It also needed greater agility to improve productivity internally.

-
- Company  National Association of Insurance Commissioners (NAIC)     Location  Washington, DC     Industry  Regulatory -
- -
-
-
-
-

Challenge

- The National Association of Insurance Commissioners (NAIC), the U.S. standard-setting and regulatory support organization, was looking for a way to deliver new services faster to provide more value for members and staff. It also needed greater agility to improve productivity internally. -

-

Solution

- Beginning in 2016, they started using Cloud Native Computing Foundation (CNCF) tools such as Prometheus. NAIC began hosting internal systems and development systems on Kubernetes at the beginning of 2018, as part of a broad move toward the public cloud. "Our culture and technology transition is a strategy embraced by our top leaders," says Dan Barker, Chief Enterprise Architect. "It has already proven successful by allowing us to accelerate our value pipeline by more than double while decreasing our costs by more than half. We are also seeing customer satisfaction increase as we add more and more applications to these new technologies." - - - -
- -
+

Solution

+

Beginning in 2016, they started using Cloud Native Computing Foundation (CNCF) tools such as Prometheus. NAIC began hosting internal systems and development systems on Kubernetes at the beginning of 2018, as part of a broad move toward the public cloud. "Our culture and technology transition is a strategy embraced by our top leaders," says Dan Barker, Chief Enterprise Architect. "It has already proven successful by allowing us to accelerate our value pipeline by more than double while decreasing our costs by more than half. We are also seeing customer satisfaction increase as we add more and more applications to these new technologies."

Impact

- Leveraging Kubernetes, "our development teams can create rapid prototypes far faster than they used to," Barker said. Applications running on Kubernetes are more resilient than those running in other environments. The deployment of open source solutions is helping influence company culture, as NAIC becomes a more open and transparent organization. -

- "We completed a small prototype in two days that would have previously taken at least a month," Barker says. Resiliency is currently measured in how much downtime systems have. "They’ve basically had none, and the occasional issue is remedied in minutes," he says. +

Leveraging Kubernetes, "our development teams can create rapid prototypes far faster than they used to," Barker said. Applications running on Kubernetes are more resilient than those running in other environments. The deployment of open source solutions is helping influence company culture, as NAIC becomes a more open and transparent organization.

-
+

"We completed a small prototype in two days that would have previously taken at least a month," Barker says. Resiliency is currently measured in how much downtime systems have. "They've basically had none, and the occasional issue is remedied in minutes," he says.

-
-
-
-
- "Our culture and technology transition is a strategy embraced by our top leaders. It has already proven successful by allowing us to accelerate our value pipeline by more than double while decreasing our costs by more than half. We are also seeing customer satisfaction increase as we add more and more applications to these new technologies."

- Dan Barker, Chief Enterprise Architect, NAIC -
-
-
-
- NAIC—which was created and overseen by the chief insurance regulators from the 50 states, the District of Columbia and five U.S. territories—provides a means through which state insurance regulators establish standards and best practices, conduct peer reviews, and coordinate their regulatory oversight. Their staff supports these efforts and represents the collective views of regulators in the United States and internationally. NAIC members, together with the organization’s central resources, form the national system of state-based insurance regulation in the United States.

-The organization has been using the cloud for years, and wanted to find more ways to quickly deliver new services that provide more value for members and staff. They looked to Kubernetes for a solution. Within NAIC, several groups are leveraging Kubernetes, one being the Platform Engineering Team. "The team building out these tools are not only deploying and operating Kubernetes, but they’re also using them," Barker says. "In fact, we’re using GitLab to deploy Kubernetes with a pipeline using kops. This team was created from developers, operators, and quality engineers from across the company, so their jobs have changed quite a bit."

-In addition, NAIC is onboarding teams to the new platform, and those teams have seen a lot of change in how they work and what they can do. "They now have more power in creating their own infrastructure and deploying their own applications," Barker says. They also use pipelines to facilitate their currently manual processes. NAIC has consumers who are using GitLab heavily, and they’re starting to use Kubernetes to deploy simple applications that help their internal processes. +{{< case-studies/quote author="Dan Barker, Chief Enterprise Architect, NAIC" >}} +"Our culture and technology transition is a strategy embraced by our top leaders. It has already proven successful by allowing us to accelerate our value pipeline by more than double while decreasing our costs by more than half. We are also seeing customer satisfaction increase as we add more and more applications to these new technologies." +{{< /case-studies/quote >}} +

NAIC—which was created and overseen by the chief insurance regulators from the 50 states, the District of Columbia and five U.S. territories—provides a means through which state insurance regulators establish standards and best practices, conduct peer reviews, and coordinate their regulatory oversight. Their staff supports these efforts and represents the collective views of regulators in the United States and internationally. NAIC members, together with the organization's central resources, form the national system of state-based insurance regulation in the United States.

-
-
-
-
- "In our experience, vendor lock-in and tooling that is highly specific results in less resilient technology with fewer minds working to solve problems and grow the community."

- Dan Barker, Chief Enterprise Architect, NAIC -
-
-
-
- "We needed greater agility to enable our own productivity internally," he says. "We decided it was right for us to move everything to the public cloud [Amazon Web Services] to help with that process and be able to access many of the native tools that allows us to move faster by not needing to build everything." -The NAIC also wanted to be cloud-agnostic, "and Kubernetes helps with this for our compute layer," Barker says. "Compute is pretty standard across the clouds, and now we can take advantage of any of them while getting all of the other features Kubernetes offers."

-The NAIC currently hosts internal systems and development systems on Kubernetes, and has already seen how impactful it can be. "Our development teams can create rapid prototypes in minutes instead of weeks," Barker says. "This recently happened with an internal tool that had no measurable wait time on the infrastructure. It was solely development bound. There is now a central shared resource that lives in AWS, which means it can grow as needed." -The native integrations into Kubernetes at NAIC has made it easy to write code and have it running in minutes instead of weeks. Applications running on Kubernetes have also proven to be more resilient than those running in other environments. "We even have teams using this to create more internal tools to help with communication or automating some of their current tasks," Barker says. -

-"We knew that Kubernetes had become the de facto standard for container orchestration," he says. "Two major factors for selecting this were the three major cloud vendors hosting their own versions and having it hosted in a neutral party as fully open source." -

-As for other CNCF projects, NAIC is using Prometheus on a small scale and hopes to continue using it moving forward because of the seamless integration with Kubernetes. The Association also is considering gRPC as its internal communications standard, Envoy in conjunction with Istio for service mesh, OpenTracing and Jaeger for tracing aggregation, and Fluentd with its Elasticsearch cluster. +

The organization has been using the cloud for years, and wanted to find more ways to quickly deliver new services that provide more value for members and staff. They looked to Kubernetes for a solution. Within NAIC, several groups are leveraging Kubernetes, one being the Platform Engineering Team. "The team building out these tools are not only deploying and operating Kubernetes, but they're also using them," Barker says. "In fact, we're using GitLab to deploy Kubernetes with a pipeline using kops. This team was created from developers, operators, and quality engineers from across the company, so their jobs have changed quite a bit."

-
-
-
-
- "We knew that Kubernetes had become the de facto standard for container orchestration. Two major factors for selecting this were the three major cloud vendors hosting their own versions and having it hosted in a neutral party as fully open source."

- Dan Barker, Chief Enterprise Architect, NAIC -
-
- +

In addition, NAIC is onboarding teams to the new platform, and those teams have seen a lot of change in how they work and what they can do. "They now have more power in creating their own infrastructure and deploying their own applications," Barker says. They also use pipelines to facilitate their currently manual processes. NAIC has consumers who are using GitLab heavily, and they're starting to use Kubernetes to deploy simple applications that help their internal processes.

-
-
+{{< case-studies/quote + image="/images/case-studies/naic/banner3.jpg" + author="Dan Barker, Chief Enterprise Architect, NAIC" +>}} +"In our experience, vendor lock-in and tooling that is highly specific results in less resilient technology with fewer minds working to solve problems and grow the community." +{{< /case-studies/quote >}} -The open governance and broad industry participation in CNCF provided a comfort level with the technology, Barker says. "We also see it as helping to influence our own company culture," he says. "We’re moving to be a more open and transparent company, and we are encouraging our staff to get involved with the different working groups and codebases. We recently became CNCF members to help further our commitment to community contribution and transparency."

-Factors such as vendor-neutrality and cross-industry investment were important in the selection. "In our experience, vendor lock-in and tooling that is highly specific results in less resilient technology with fewer minds working to solve problems and grow the community," Barker says.

-NAIC is a largely Oracle shop, Barker says, and has been running mostly Java on JBoss. "However, we have years of history with other applications," he says. "Some of these have been migrated by completely rewriting the application, while others are just being modified slightly to fit into this new paradigm."

-Running on AWS cloud, the Association has not specifically taken a microservices approach. "We are moving to microservices where practical, but we haven’t found that it’s a necessity to operate them within Kubernetes," Barker says

-All of its databases are currently running within public cloud services, but they have explored eventually running those in Kubernetes, as it makes sense. "We’re doing this to get more reuse from common components and to limit our failure domains to something more manageable and observable," Barker says. +

"We needed greater agility to enable our own productivity internally," he says. "We decided it was right for us to move everything to the public cloud [Amazon Web Services] to help with that process and be able to access many of the native tools that allows us to move faster by not needing to build everything." +The NAIC also wanted to be cloud-agnostic, "and Kubernetes helps with this for our compute layer," Barker says. "Compute is pretty standard across the clouds, and now we can take advantage of any of them while getting all of the other features Kubernetes offers."

+

The NAIC currently hosts internal systems and development systems on Kubernetes, and has already seen how impactful it can be. "Our development teams can create rapid prototypes in minutes instead of weeks," Barker says. "This recently happened with an internal tool that had no measurable wait time on the infrastructure. It was solely development bound. There is now a central shared resource that lives in AWS, which means it can grow as needed."

-
+

The native integrations into Kubernetes at NAIC has made it easy to write code and have it running in minutes instead of weeks. Applications running on Kubernetes have also proven to be more resilient than those running in other environments. "We even have teams using this to create more internal tools to help with communication or automating some of their current tasks," Barker says.

-
-
- "We have been able to move much faster at lower cost than we were able to in the past," Barker says. "We were able to complete one of our projects in a year, when the previous version took over two years. And the new project cost $500,000 while the original required $3 million, and with fewer defects. We are also able to push out new features much faster."

- Dan Barker, Chief Enterprise Architect, NAIC +

"We knew that Kubernetes had become the de facto standard for container orchestration," he says. "Two major factors for selecting this were the three major cloud vendors hosting their own versions and having it hosted in a neutral party as fully open source."

-
-
-
+

As for other CNCF projects, NAIC is using Prometheus on a small scale and hopes to continue using it moving forward because of the seamless integration with Kubernetes. The Association also is considering gRPC as its internal communications standard, Envoy in conjunction with Istio for service mesh, OpenTracing and Jaeger for tracing aggregation, and Fluentd with its Elasticsearch cluster.

-NAIC has seen a significant business impact from its efforts. "We have been able to move much faster at lower cost than we were able to in the past," Barker says. "We were able to complete one of our projects in a year, when the previous version took over two years. And the new project cost $500,000 while the original required $3 million, and with fewer defects. We are also able to push out new features much faster." -He says the organization is moving toward continuous deployment "because the business case makes sense. The research is becoming very hard to argue with. We want to reduce our batch sizes and optimize on delivering value to customers and not feature count. This is requiring a larger cultural shift than just a technology shift." -NAIC is "becoming more open and transparent, as well as more resilient to failure," Barker says. "Even our customers are wanting more and more of this and trying to figure out how they can work with us to accomplish our mutual goals faster. Members of the insurance industry have reached out so that we can better learn together and grow as an industry." +{{< case-studies/quote + image="/images/case-studies/naic/banner4.jpg" + author="Dan Barker, Chief Enterprise Architect, NAIC" +>}} +"We knew that Kubernetes had become the de facto standard for container orchestration. Two major factors for selecting this were the three major cloud vendors hosting their own versions and having it hosted in a neutral party as fully open source." +{{< /case-studies/quote >}} -
+

The open governance and broad industry participation in CNCF provided a comfort level with the technology, Barker says. "We also see it as helping to influence our own company culture," he says. "We're moving to be a more open and transparent company, and we are encouraging our staff to get involved with the different working groups and codebases. We recently became CNCF members to help further our commitment to community contribution and transparency."

-
+

Factors such as vendor-neutrality and cross-industry investment were important in the selection. "In our experience, vendor lock-in and tooling that is highly specific results in less resilient technology with fewer minds working to solve problems and grow the community," Barker says.

+ +

NAIC is a largely Oracle shop, Barker says, and has been running mostly Java on JBoss. "However, we have years of history with other applications," he says. "Some of these have been migrated by completely rewriting the application, while others are just being modified slightly to fit into this new paradigm."

+ +

Running on AWS cloud, the Association has not specifically taken a microservices approach. "We are moving to microservices where practical, but we haven't found that it's a necessity to operate them within Kubernetes," Barker says.

+ +

All of its databases are currently running within public cloud services, but they have explored eventually running those in Kubernetes, as it makes sense. "We're doing this to get more reuse from common components and to limit our failure domains to something more manageable and observable," Barker says.

+ +{{< case-studies/quote author="Dan Barker, Chief Enterprise Architect, NAIC" >}} +"We have been able to move much faster at lower cost than we were able to in the past," Barker says. "We were able to complete one of our projects in a year, when the previous version took over two years. And the new project cost $500,000 while the original required $3 million, and with fewer defects. We are also able to push out new features much faster." +{{< /case-studies/quote >}} + +

NAIC has seen a significant business impact from its efforts. "We have been able to move much faster at lower cost than we were able to in the past," Barker says. "We were able to complete one of our projects in a year, when the previous version took over two years. And the new project cost $500,000 while the original required $3 million, and with fewer defects. We are also able to push out new features much faster."

+ +

He says the organization is moving toward continuous deployment "because the business case makes sense. The research is becoming very hard to argue with. We want to reduce our batch sizes and optimize on delivering value to customers and not feature count. This is requiring a larger cultural shift than just a technology shift."

+ +

NAIC is "becoming more open and transparent, as well as more resilient to failure," Barker says. "Even our customers are wanting more and more of this and trying to figure out how they can work with us to accomplish our mutual goals faster. Members of the insurance industry have reached out so that we can better learn together and grow as an industry."

\ No newline at end of file diff --git a/content/en/case-studies/nav/index.html b/content/en/case-studies/nav/index.html index bd606e7314..6be206b5e3 100644 --- a/content/en/case-studies/nav/index.html +++ b/content/en/case-studies/nav/index.html @@ -1,95 +1,81 @@ --- title: Nav Case Study - linkTitle: Nav case_study_styles: true cid: caseStudies -css: /css/style_case_studies.css featured: false + +new_case_study_styles: true +heading_background: /images/case-studies/nav/banner1.jpg +heading_title_logo: /images/nav_logo.png +subheading: > + How A Startup Reduced Its Infrastructure Costs by 50% With Kubernetes +case_study_details: + - Company: Nav + - Location: Salt Lake City, Utah, and San Mateo, California + - Industry: Financial services for businesses --- -
-

CASE STUDY:
How A Startup Reduced Its Infrastructure Costs by 50% With Kubernetes -

+

Challenge

-
+

Founded in 2012, Nav provides small business owners with access to their business credit scores from all three major commercial credit bureaus—Equifax, Experian and Dun & Bradstreet—and financing options that best fit their needs. Five years in, the startup was growing rapidly, and "our cloud environments were getting very large, and our usage of those environments was extremely low, like under 1%," says Director of Engineering Travis Jeppson. "We wanted our usage of cloud environments to be more tightly coupled with what we actually needed, so we started looking at containerization and orchestration to help us be able to run workloads that were distinct from one another but could share a similar resource pool."

-
- Company  Nav     Location  Salt Lake City, Utah, and San Mateo, California     Industry  Financial services for businesses -
+

Solution

+ +

After evaluating a number of orchestration solutions, the Nav team decided to adopt Kubernetes running on AWS. The strength of the community around Kubernetes was a strong draw, as well as its Google provenance. Plus, "the other solutions tended to be fairly heavy-handed, really complex, really large, and really hard to manage just off the bat," says Jeppson. "Kubernetes gave us a very simple way to be able to step into an orchestration solution that fit our needs at the time, but also the extensibility of it allowed us to be able to grow with it and be able to build in more features and functionality later on."

-
-
-
-
-

-

Challenge

- Founded in 2012, Nav provides small business owners with access to their business credit scores from all three major commercial credit bureaus—Equifax, Experian and Dun & Bradstreet—and financing options that best fit their needs. Five years in, the startup was growing rapidly, and “our cloud environments were getting very large, and our usage of those environments was extremely low, like under 1%,” says Director of Engineering Travis Jeppson. “We wanted our usage of cloud environments to be more tightly coupled with what we actually needed, so we started looking at containerization and orchestration to help us be able to run workloads that were distinct from one another but could share a similar resource pool.” -

-

Solution

- After evaluating a number of orchestration solutions, the Nav team decided to adopt Kubernetes running on AWS. The strength of the community around Kubernetes was a strong draw, as well as its Google provenance. Plus, “the other solutions tended to be fairly heavy-handed, really complex, really large, and really hard to manage just off the bat,” says Jeppson. “Kubernetes gave us a very simple way to be able to step into an orchestration solution that fit our needs at the time, but also the extensibility of it allowed us to be able to grow with it and be able to build in more features and functionality later on.” -

Impact

- The four-person team got Kubernetes up and running in six months, and the full migration of Nav’s 25 microservices was completed in another six months. The results have been impressive: Resource utilization, which led the company on this path in the first place, has increased from 1% to 40%. Launching a new service used to take two developers two weeks; now it takes only one developer less than 10 minutes. Deployments have increased 5x. And the company is saving 50% in infrastructure costs. -
-
-
-
-
- -

- "Kubernetes gave us a very simple way to be able to step into an orchestration solution that fit our needs at the time, but also the extensibility of it allowed us to be able to grow with it and be able to build in more features and functionality later on." -

- Travis Jeppson, Director of Engineering, Nav -
-
-
-
-

Founded in 2012, Nav provides small business owners with access to their business credit scores from all three major commercial credit bureaus—Equifax, Experian and Dun & Bradstreet—as well as details on their businesses’ financial health and financing options that best fit their needs. Its mission boils down to this, says Director of Engineering Travis Jeppson: “to increase the success rate of small businesses.”

- A couple of years ago, Nav recognized an obstacle in its own path to success. The business was growing rapidly, and “our cloud environments were getting very large, and our usage of those environments was extremely low, like under 1%,” says Jeppson. “Most of the problem was around the ability to scale. We were just throwing money at it. ‘Let’s just spin up more servers. Let’s just do more things in order to handle an increased load.’ And with us being a startup, that could lead to our demise. We don’t have the money to burn on that kind of stuff.”

- Plus, every new service had to go through 10 different people, taking an unacceptably long two weeks to launch. “All of the patch management and the server management was done very manually, and so we all had to watch it and maintain it really well,” adds Jeppson. “It was just a very troublesome system.” +

The four-person team got Kubernetes up and running in six months, and the full migration of Nav's 25 microservices was completed in another six months. The results have been impressive: Resource utilization, which led the company on this path in the first place, has increased from 1% to 40%. Launching a new service used to take two developers two weeks; now it takes only one developer less than 10 minutes. Deployments have increased 5x. And the company is saving 50% in infrastructure costs.

+{{< case-studies/quote author="Travis Jeppson, Director of Engineering, Nav" >}} + +
+"Kubernetes gave us a very simple way to be able to step into an orchestration solution that fit our needs at the time, but also the extensibility of it allowed us to be able to grow with it and be able to build in more features and functionality later on." +{{< /case-studies/quote >}} -
-
-
-
- "The community is absolutely vital: being able to pass ideas around, talk about a lot of the similar challenges that we’re all facing, and just get help. I like that we’re able to tackle the same problems for different reasons but help each other along the way."

- Travis Jeppson, Director of Engineering, Nav +{{< case-studies/lead >}} +Founded in 2012, Nav provides small business owners with access to their business credit scores from all three major commercial credit bureaus—Equifax, Experian and Dun & Bradstreet—as well as details on their businesses' financial health and financing options that best fit their needs. Its mission boils down to this, says Director of Engineering Travis Jeppson: "to increase the success rate of small businesses." +{{< /case-studies/lead >}} -
-
-
-
- Jeppson had worked with containers at his previous job, and pitched that technology to Nav’s management as a solution to these problems. He got the green light in early 2017. “We wanted our usage of cloud environments to be more tightly coupled with what we actually needed, so we started looking at containerization and orchestration to help us be able to run workloads that were distinct from one another but could share a similar resource pool,” he says.

- After evaluating a number of orchestration solutions, the company decided to adopt Kubernetes running on AWS. The strength of the community around Kubernetes was a strong draw, as was its Google origins. Additionally, “the other solutions tended to be fairly heavy-handed, really complex, really large, and really hard to manage just off the bat,” says Jeppson. “Kubernetes gave us a very simple way to be able to step into an orchestration solution that fit our needs at the time, but the extensibility of it would also allow us to grow with it and build in more features and functionality later on.”

- Jeppson’s four-person Engineering Services team got Kubernetes up and running in six months (they decided to use Kubespray to spin up clusters), and the full migration of Nav’s 25 microservices and one primary monolith was completed in another six months. “We couldn’t rewrite everything; we couldn’t stop,” he says. “We had to stay up, we had to stay available, and we had to have minimal amount of downtime. So we got really comfortable around our building pipeline, our metrics and logging, and then around Kubernetes itself: how to launch it, how to upgrade it, how to service it. And we moved little by little.” -
-
-
-
- “Kubernetes has brought so much value to Nav by allowing all of these new freedoms that we had just never had before.”

- Travis Jeppson, Director of Engineering, Nav -
-
+

A couple of years ago, Nav recognized an obstacle in its own path to success. The business was growing rapidly, and "our cloud environments were getting very large, and our usage of those environments was extremely low, like under 1%," says Jeppson. "Most of the problem was around the ability to scale. We were just throwing money at it. 'Let's just spin up more servers. Let's just do more things in order to handle an increased load.' And with us being a startup, that could lead to our demise. We don't have the money to burn on that kind of stuff."

-
+

Plus, every new service had to go through 10 different people, taking an unacceptably long two weeks to launch. "All of the patch management and the server management was done very manually, and so we all had to watch it and maintain it really well," adds Jeppson. "It was just a very troublesome system."

-
- A crucial part of the process involved educating Nav’s 50 engineers and being transparent regarding the new workflow as well as the roadmap for the migration. Jeppson did regular presentations along the way, and a week of four-hours-a-day labs for the entire staff of engineers. He then created a repository in GitLab to house all of the information. “We showed all the frontend and backend developers how to go in, create their own namespace using kubectl, all themselves,” he says. “Now, a lot of times, they just come to us and say, ‘This is ready.’ We click a little button in GitLab to allow it to release into production, and they’re off to the races.”

- Since the migration was completed in early 2018, the results have been impressive: Resource utilization, which led the company on this path in the first place, has increased from 1% to 40%. Launching a new service used to take two developers two weeks; now it takes only one developer less than 10 minutes. Deployments have increased 5x, from 10 a day to 50 a day. And the company is saving 50% in infrastructure costs on the computational side. “Next we want to go in to address the database side, and once we do that, then we’re going to continue to drop that cost quite a bit more,” says Jeppson.

- Kubernetes has also helped Nav with its compliance needs. Before, “we had to map one application to one server, mostly due to different compliance regulations around data,” Jeppson says. “With the Kubernetes API, we could add in network policies and segregate that data and restrict it if needed.” The company segregates its cluster into an unrestricted zone and a restricted zone, which has its own set of nodes where data protection happens. The company also uses the Twistlock tool to ensure security, “and that makes it a lot easier to sleep at night,” he adds. +{{< case-studies/quote + image="/images/case-studies/nav/banner3.jpg" + author="Travis Jeppson, Director of Engineering, Nav" +>}} +"The community is absolutely vital: being able to pass ideas around, talk about a lot of the similar challenges that we're all facing, and just get help. I like that we're able to tackle the same problems for different reasons but help each other along the way." +{{< /case-studies/quote >}} -
+

Jeppson had worked with containers at his previous job, and pitched that technology to Nav's management as a solution to these problems. He got the green light in early 2017. "We wanted our usage of cloud environments to be more tightly coupled with what we actually needed, so we started looking at containerization and orchestration to help us be able to run workloads that were distinct from one another but could share a similar resource pool," he says.

-
-
-"We’re talking four to 10 times the amount of traffic that we handle now, and it’s just like, 'Oh, yeah. We’re good. Kubernetes handles this for us.'"

- Travis Jeppson, Director of Engineering, Nav
-
+

After evaluating a number of orchestration solutions, the company decided to adopt Kubernetes running on AWS. The strength of the community around Kubernetes was a strong draw, as was its Google origins. Additionally, "the other solutions tended to be fairly heavy-handed, really complex, really large, and really hard to manage just off the bat," says Jeppson. "Kubernetes gave us a very simple way to be able to step into an orchestration solution that fit our needs at the time, but the extensibility of it would also allow us to grow with it and build in more features and functionality later on."

-
- With Kubernetes in place, the Nav team also started improving the system’s metrics and logging by adopting Prometheus. “Prometheus created a standard around metrics that was really easy for a developer to adopt,” says Jeppson. “They have the freedom to display what they want, to do what they need, and keep their codebase clean, and that to us was absolutely a must.”

- Next up for Nav in the coming year: looking at tracing, storage, and service mesh. They’re currently evaluating Envoy, OpenTracing, and Jaeger after spending much of KubeCon talking to other companies. “The community is absolutely vital: being able to pass ideas around, talk about a lot of the similar challenges that we’re all facing, and just get help. I like that we’re able to tackle the same problems for different reasons but help each other along the way,” says Jeppson. “There’s still so, so much to do around scalability, around being able to really fully adopt a cloud native solution.”

- Of course, it all starts with Kubernetes. With that technology, Jeppson’s team has built a platform that allows Nav to scale, and that “has brought so much value to Nav by allowing all of these new freedoms that we had just never had before,” he says.

- Conversations about new products used to be bogged down by the fact they’d have to wait six months to get an environment set up with isolation and then figure out how to handle spikes of traffic. “But now it’s just nothing to us,” says Jeppson. “We’re talking four to 10 times the amount of traffic that we handle now, and it’s just like, ‘Oh, yeah. We’re good. Kubernetes handles this for us.’” +

Jeppson's four-person Engineering Services team got Kubernetes up and running in six months (they decided to use Kubespray to spin up clusters), and the full migration of Nav's 25 microservices and one primary monolith was completed in another six months. "We couldn't rewrite everything; we couldn't stop," he says. "We had to stay up, we had to stay available, and we had to have minimal amount of downtime. So we got really comfortable around our building pipeline, our metrics and logging, and then around Kubernetes itself: how to launch it, how to upgrade it, how to service it. And we moved little by little."

-
-
+{{< case-studies/quote + image="/images/case-studies/nav/banner4.jpg" + author="Travis Jeppson, Director of Engineering, Nav" +>}} +"Kubernetes has brought so much value to Nav by allowing all of these new freedoms that we had just never had before." +{{< /case-studies/quote >}} + +

A crucial part of the process involved educating Nav's 50 engineers and being transparent regarding the new workflow as well as the roadmap for the migration. Jeppson did regular presentations along the way, and a week of four-hours-a-day labs for the entire staff of engineers. He then created a repository in GitLab to house all of the information. "We showed all the frontend and backend developers how to go in, create their own namespace using kubectl, all themselves," he says. "Now, a lot of times, they just come to us and say, 'This is ready.' We click a little button in GitLab to allow it to release into production, and they're off to the races."

+ +

Since the migration was completed in early 2018, the results have been impressive: Resource utilization, which led the company on this path in the first place, has increased from 1% to 40%. Launching a new service used to take two developers two weeks; now it takes only one developer less than 10 minutes. Deployments have increased 5x, from 10 a day to 50 a day. And the company is saving 50% in infrastructure costs on the computational side. "Next we want to go in to address the database side, and once we do that, then we're going to continue to drop that cost quite a bit more," says Jeppson.

+ +

Kubernetes has also helped Nav with its compliance needs. Before, "we had to map one application to one server, mostly due to different compliance regulations around data," Jeppson says. "With the Kubernetes API, we could add in network policies and segregate that data and restrict it if needed." The company segregates its cluster into an unrestricted zone and a restricted zone, which has its own set of nodes where data protection happens. The company also uses the Twistlock tool to ensure security, "and that makes it a lot easier to sleep at night," he adds.

+ +{{< case-studies/quote author="Travis Jeppson, Director of Engineering, Nav" >}} +"We're talking four to 10 times the amount of traffic that we handle now, and it's just like, 'Oh, yeah. We're good. Kubernetes handles this for us.'" +{{< /case-studies/quote >}} + +

With Kubernetes in place, the Nav team also started improving the system's metrics and logging by adopting Prometheus. "Prometheus created a standard around metrics that was really easy for a developer to adopt," says Jeppson. "They have the freedom to display what they want, to do what they need, and keep their codebase clean, and that to us was absolutely a must."

+ +

Next up for Nav in the coming year: looking at tracing, storage, and service mesh. They're currently evaluating Envoy, OpenTracing, and Jaeger after spending much of KubeCon talking to other companies. "The community is absolutely vital: being able to pass ideas around, talk about a lot of the similar challenges that we're all facing, and just get help. I like that we're able to tackle the same problems for different reasons but help each other along the way," says Jeppson. "There's still so, so much to do around scalability, around being able to really fully adopt a cloud native solution."

+ +

Of course, it all starts with Kubernetes. With that technology, Jeppson's team has built a platform that allows Nav to scale, and that "has brought so much value to Nav by allowing all of these new freedoms that we had just never had before," he says.

+ +

Conversations about new products used to be bogged down by the fact they'd have to wait six months to get an environment set up with isolation and then figure out how to handle spikes of traffic. "But now it's just nothing to us," says Jeppson. "We're talking four to 10 times the amount of traffic that we handle now, and it's just like, 'Oh, yeah. We're good. Kubernetes handles this for us.'"

diff --git a/content/en/case-studies/nerdalize/index.html b/content/en/case-studies/nerdalize/index.html index 2756ce431c..d961c0a289 100644 --- a/content/en/case-studies/nerdalize/index.html +++ b/content/en/case-studies/nerdalize/index.html @@ -3,94 +3,79 @@ title: Prowise Case Study linkTitle: prowise case_study_styles: true cid: caseStudies -css: /css/style_case_studies.css featured: false + +new_case_study_styles: true +heading_background: /images/case-studies/nerdalize/banner1.jpg +heading_title_logo: /images/nerdalize_logo.png +subheading: > + Nerdalize: Providing Affordable and Sustainable Cloud Hosting with Kubernetes +case_study_details: + - Company: Nerdalize + - Location: Delft, Netherlands + - Industry: Cloud Provider --- -
-

CASE STUDY:
Nerdalize: Providing Affordable and Sustainable Cloud Hosting with Kubernetes -

-
+

Challenge

-
- Company  Nerdalize     Location  Delft, Netherlands      Industry  Cloud Provider -
+

Nerdalize offers affordable cloud hosting for customers—and free heat and hot water for people who sign up to house the heating devices that contain the company's servers. The savings Nerdalize realizes by not running data centers are passed on to its customers. When the team began using Docker to make its software more portable, it realized it also needed a container orchestration solution. "As a cloud provider, we have internal services for hosting our backends and billing our customers, but we also need to offer our compute to our end users," says Digital Product Engineer Ad van der Veer. "Since we have these heating devices spread across the Netherlands, we need some way of tying that all together."

-
-
-
-
+

Solution

+ +

After briefly using a basic scheduling setup with another open source tool, Nerdalize switched to Kubernetes. "On top of our heating devices throughout the Netherlands, we have a virtual machine layer, and on top of that we run Kubernetes clusters for our customers," says van der Veer. "As a small company, we have to provide a rock solid story in terms of the technology. Kubernetes allows us to offer a hybrid solution: 'You can run this on our cloud, but you can run it on other clouds as well. It runs in your internal hardware if you like.' And together with the Docker image standard and our multi-cloud dashboard, that allows them peace of mind."

-

Challenge

- Nerdalize offers affordable cloud hosting for customers—and free heat and hot water for people who sign up to house the heating devices that contain the company’s servers. The savings Nerdalize realizes by not running data centers are passed on to its customers. When the team began using Docker to make its software more portable, it realized it also needed a container orchestration solution. “As a cloud provider, we have internal services for hosting our backends and billing our customers, but we also need to offer our compute to our end users,” says Digital Product Engineer Ad van der Veer. “Since we have these heating devices spread across the Netherlands, we need some way of tying that all together.” -

Solution

- After briefly using a basic scheduling setup with another open source tool, Nerdalize switched to Kubernetes. “On top of our heating devices throughout the Netherlands, we have a virtual machine layer, and on top of that we run Kubernetes clusters for our customers,” says van der Veer. “As a small company, we have to provide a rock solid story in terms of the technology. Kubernetes allows us to offer a hybrid solution: ‘You can run this on our cloud, but you can run it on other clouds as well. It runs in your internal hardware if you like.’ And together with the Docker image standard and our multi-cloud dashboard, that allows them peace of mind.”

Impact

- Nerdalize prides itself on being a Kubernetes-native cloud provider that charges its customers prices 40% below that of other cloud providers. “Every euro that we have to invest for licensing of software that’s not open source comes from that 40%,” says van der Veer. If they had used a non-open source orchestration platform instead of Kubernetes, “that would reduce this proposition that we have of 40% less cost to like 30%. Kubernetes directly allows us to have this business model and this strategic advantage.” Nerdalize customers also benefit from time savings: One went from spending a day to set up VMs, network, and software, to spinning up a Kubernetes cluster in minutes. And for households using the heating devices, they save an average of 200 euro a year on their heating bill. The environmental impact? The annual reduction in CO2 emissions comes out to be 2 tons per Nerdalize household, which is equivalent to a car driving 8,000 km. -
-
-
-
-
- “We can walk into a boardroom and put a Kubernetes logo up, and people accept it as an established technology. It becomes this centerpiece where other cloud native projects can tie in, so there’s a network effect that each project empowers each other. This is something that has a lot of value when we have to talk to customers and convince them that our cloud fits their needs.” -

— AD VAN DER VEER, PRODUCT ENGINEER, NERDALIZE -
-
-
-
-

Nerdalize is a cloud hosting provider that has no data centers. Instead, the four-year-old startup places its servers in homes across the Netherlands, inside heating devices it developed to turn the heat produced by the servers into heating and hot water for the residents. -

- “Households save on their gas bills, and cloud users have a much more sustainable cloud solution,” says Maaike Stoops, Customer Experience Queen at Nerdalize. “And we don’t have the overhead of building a data center, so our cloud is up to 40% more affordable.” -

- That business model has been enabled by the company’s adoption of containerization and Kubernetes. “When we just got started, Docker was just introduced,” says Digital Product Engineer Ad van der Veer. “We began with a very basic bare metal setup, but once we developed the business, we saw that containerization technology was super useful to help our customers. As a cloud provider, we have internal services for hosting our backends and billing our customers, but we also need to offer our compute to our end users. Since we have these heating devices spread across the Netherlands, we need some way of tying that all together.” -

- After trying to develop its own scheduling system using another open source tool, Nerdalize found Kubernetes. “Kubernetes provided us with more functionality out of the gate,” says van der Veer. -
-
-
-
- “We always try to get a working version online first, like minimal viable products, and then move to stabilize that,” says van der Veer. “And I think that these kinds of day-two problems are now immediately solved. The rapid prototyping we saw internally is a very valuable aspect of Kubernetes.”

— AD VAN DER VEER, PRODUCT ENGINEER, NERDALIZE -
-
-
-
- The team first experimented with a basic use case to run customers’ workloads on Kubernetes. “Getting the data working was kind of difficult, and at the time the installation wasn’t that simple,” says van der Veer. “Then CNCF started, we saw the community grow, these problems got solved, and from there it became a very easy decision.” -

- The first Nerdalize product that was launched in 2017 was “100% containerized and Kubernetes native,” says van der Veer. “On top of our heating devices throughout the Netherlands, we have a virtual machine layer, and on top of that we run Kubernetes clusters for our customers. As a small company, we have to provide a rock solid story in terms of the technology. Kubernetes allows us to offer a hybrid solution: ‘You can run this on our cloud, but you can run it on other clouds as well. It runs in your internal hardware if you like.’ And together with the Docker image standard and our multi-cloud dashboard, that gives them peace of mind.” -

- Not to mention the 40% cost savings. “Every euro that we have to invest for licensing of software that’s not open source comes from that 40%,” says van der Veer. If Nerdalize had used a non-open source orchestration platform instead of Kubernetes, “that would reduce our cost savings proposition to like 30%. Kubernetes directly allows us to have this business model and this strategic advantage.” -
-
-
-
- “One of our customers used to spend up to a day setting up the virtual machines, network and software every time they wanted to run a project in the cloud. On our platform, with Docker and Kubernetes, customers can have their projects running in a couple of minutes.” -

- MAAIKE STOOPS, CUSTOMER EXPERIENCE QUEEN, NERDALIZE -
-
-
-
- Nerdalize now has customers, from individual engineers to data-intensive startups and established companies, all around the world. (For the time being, though, the heating devices are exclusive to the Netherlands.) One of the most common use cases is batch workloads used by data scientists and researchers, and the time savings for these end users is profound. “One of our customers used to spend up to a day setting up the virtual machines, network and software every time they wanted to run a project in the cloud,” says Stoops. “On our platform, with Docker and Kubernetes, customers can have their projects running in a couple of minutes.” -

- As for households using the heating devices, they save an average of 200 euro a year on their heating bill. The environmental impact? The annual reduction in CO2 emissions comes out to 2 tons per Nerdalize household, which is equivalent to a car driving 8,000 km. -

- For the Nerdalize team, feature development—such as the accessible command line interface called Nerd, which recently went live—has also been sped up by Kubernetes. “We always try to get a working version online first, like minimal viable products, and then move to stabilize that,” says van der Veer. “And I think that these kinds of day-two problems are now immediately solved. The rapid prototyping we saw internally is a very valuable aspect of Kubernetes.” -

- Another unexpected benefit has been the growing influence and reputation of Kubernetes. “We can walk into a boardroom and put a Kubernetes logo up, and people accept it as an established technology,” says van der Veer. “It becomes this centerpiece where other cloud native projects can tie in, so there’s a network effect that each project empowers each other. This is something that has a lot of value when we have to convince customers that our cloud fits their needs.” -
+

Nerdalize prides itself on being a Kubernetes-native cloud provider that charges its customers prices 40% below that of other cloud providers. "Every euro that we have to invest for licensing of software that's not open source comes from that 40%," says van der Veer. If they had used a non-open source orchestration platform instead of Kubernetes, "that would reduce this proposition that we have of 40% less cost to like 30%. Kubernetes directly allows us to have this business model and this strategic advantage." Nerdalize customers also benefit from time savings: One went from spending a day to set up VMs, network, and software, to spinning up a Kubernetes cluster in minutes. And for households using the heating devices, they save an average of 200 euro a year on their heating bill. The environmental impact? The annual reduction in CO2 emissions comes out to be 2 tons per Nerdalize household, which is equivalent to a car driving 8,000 km.

-
-
-“It shouldn’t be too big of a hassle and too large of a commitment. It should be fun and easy for end users. So we really love Kubernetes in that way.”

- MAAIKE STOOPS, CUSTOMER EXPERIENCE QUEEN, NERDALIZE
-
+{{< case-studies/quote author="AD VAN DER VEER, PRODUCT ENGINEER, NERDALIZE" >}} +"We can walk into a boardroom and put a Kubernetes logo up, and people accept it as an established technology. It becomes this centerpiece where other cloud native projects can tie in, so there's a network effect that each project empowers each other. This is something that has a lot of value when we have to talk to customers and convince them that our cloud fits their needs." +{{< /case-studies/quote >}} -
+{{< case-studies/lead >}} +Nerdalize is a cloud hosting provider that has no data centers. Instead, the four-year-old startup places its servers in homes across the Netherlands, inside heating devices it developed to turn the heat produced by the servers into heating and hot water for the residents. +{{< /case-studies/lead >}} - In fact, Nerdalize is currently looking into implementing other CNCF projects, such as Prometheus for monitoring and Rook, “which should help us with some of the data problems that we want to solve for our customers,” says van der Veer. -

- In the coming year, Nerdalize will scale up the number of households running its hardware to 50, or the equivalent of a small scale data center. Geographic redundancy and greater server ability for customers are two main goals. Spreading the word about Kubernetes is also in the game plan. “We offer a free namespace on our sandbox, multi-tenant Kubernetes cluster for anyone to try,” says van der Veer. “What’s more cool than trying your first Kubernetes project on houses, to warm a shower?” -

- Ultimately, this ties into Nerdalize’s mission of supporting affordable and sustainable cloud hosting. “We want to be the disrupter of the cloud space, showing organizations that running in the cloud is easy and affordable,” says Stoops. “It shouldn’t be too big of a hassle and too large of a commitment. It should be fun and easy for end users. So we really love Kubernetes in that way.” -
+

"Households save on their gas bills, and cloud users have a much more sustainable cloud solution," says Maaike Stoops, Customer Experience Queen at Nerdalize. "And we don't have the overhead of building a data center, so our cloud is up to 40% more affordable."

-
+

That business model has been enabled by the company's adoption of containerization and Kubernetes. "When we just got started, Docker was just introduced," says Digital Product Engineer Ad van der Veer. "We began with a very basic bare metal setup, but once we developed the business, we saw that containerization technology was super useful to help our customers. As a cloud provider, we have internal services for hosting our backends and billing our customers, but we also need to offer our compute to our end users. Since we have these heating devices spread across the Netherlands, we need some way of tying that all together."

+ +

After trying to develop its own scheduling system using another open source tool, Nerdalize found Kubernetes. "Kubernetes provided us with more functionality out of the gate," says van der Veer.

+ +{{< case-studies/quote + image="/images/case-studies/nerdalize/banner3.jpg" + author="AD VAN DER VEER, PRODUCT ENGINEER, NERDALIZE" +>}} +"We always try to get a working version online first, like minimal viable products, and then move to stabilize that," says van der Veer. "And I think that these kinds of day-two problems are now immediately solved. The rapid prototyping we saw internally is a very valuable aspect of Kubernetes." +{{< /case-studies/quote >}} + +

The team first experimented with a basic use case to run customers' workloads on Kubernetes. "Getting the data working was kind of difficult, and at the time the installation wasn't that simple," says van der Veer. "Then CNCF started, we saw the community grow, these problems got solved, and from there it became a very easy decision."

+ +

The first Nerdalize product that was launched in 2017 was "100% containerized and Kubernetes native," says van der Veer. "On top of our heating devices throughout the Netherlands, we have a virtual machine layer, and on top of that we run Kubernetes clusters for our customers. As a small company, we have to provide a rock solid story in terms of the technology. Kubernetes allows us to offer a hybrid solution: 'You can run this on our cloud, but you can run it on other clouds as well. It runs in your internal hardware if you like.' And together with the Docker image standard and our multi-cloud dashboard, that gives them peace of mind."

+ +

Not to mention the 40% cost savings. "Every euro that we have to invest for licensing of software that's not open source comes from that 40%," says van der Veer. If Nerdalize had used a non-open source orchestration platform instead of Kubernetes, "that would reduce our cost savings proposition to like 30%. Kubernetes directly allows us to have this business model and this strategic advantage."

+ +{{< case-studies/quote + image="/images/case-studies/nerdalize/banner4.jpg" + author="MAAIKE STOOPS, CUSTOMER EXPERIENCE QUEEN, NERDALIZE" +>}} +"One of our customers used to spend up to a day setting up the virtual machines, network and software every time they wanted to run a project in the cloud. On our platform, with Docker and Kubernetes, customers can have their projects running in a couple of minutes." +{{< /case-studies/quote >}} + +

Nerdalize now has customers, from individual engineers to data-intensive startups and established companies, all around the world. (For the time being, though, the heating devices are exclusive to the Netherlands.) One of the most common use cases is batch workloads used by data scientists and researchers, and the time savings for these end users is profound. "One of our customers used to spend up to a day setting up the virtual machines, network and software every time they wanted to run a project in the cloud," says Stoops. "On our platform, with Docker and Kubernetes, customers can have their projects running in a couple of minutes."

+ +

As for households using the heating devices, they save an average of 200 euro a year on their heating bill. The environmental impact? The annual reduction in CO2 emissions comes out to 2 tons per Nerdalize household, which is equivalent to a car driving 8,000 km.

+ +

For the Nerdalize team, feature development—such as the accessible command line interface called Nerd, which recently went live—has also been sped up by Kubernetes. "We always try to get a working version online first, like minimal viable products, and then move to stabilize that," says van der Veer. "And I think that these kinds of day-two problems are now immediately solved. The rapid prototyping we saw internally is a very valuable aspect of Kubernetes."

+ +

Another unexpected benefit has been the growing influence and reputation of Kubernetes. "We can walk into a boardroom and put a Kubernetes logo up, and people accept it as an established technology," says van der Veer. "It becomes this centerpiece where other cloud native projects can tie in, so there's a network effect that each project empowers each other. This is something that has a lot of value when we have to convince customers that our cloud fits their needs."

+ +{{< case-studies/quote author="MAAIKE STOOPS, CUSTOMER EXPERIENCE QUEEN, NERDALIZE" >}} +"It shouldn't be too big of a hassle and too large of a commitment. It should be fun and easy for end users. So we really love Kubernetes in that way." +{{< /case-studies/quote >}} + +

In fact, Nerdalize is currently looking into implementing other CNCF projects, such as Prometheus for monitoring and Rook, "which should help us with some of the data problems that we want to solve for our customers," says van der Veer.

+ +

In the coming year, Nerdalize will scale up the number of households running its hardware to 50, or the equivalent of a small scale data center. Geographic redundancy and greater server ability for customers are two main goals. Spreading the word about Kubernetes is also in the game plan. "We offer a free namespace on our sandbox, multi-tenant Kubernetes cluster for anyone to try," says van der Veer. "What's more cool than trying your first Kubernetes project on houses, to warm a shower?"

+ +

Ultimately, this ties into Nerdalize's mission of supporting affordable and sustainable cloud hosting. "We want to be the disrupter of the cloud space, showing organizations that running in the cloud is easy and affordable," says Stoops. "It shouldn't be too big of a hassle and too large of a commitment. It should be fun and easy for end users. So we really love Kubernetes in that way."

diff --git a/content/en/case-studies/netease/index.html b/content/en/case-studies/netease/index.html index 6cba5579ab..3a3a8ab3a7 100644 --- a/content/en/case-studies/netease/index.html +++ b/content/en/case-studies/netease/index.html @@ -3,87 +3,74 @@ title: NetEase Case Study linkTitle: NetEase case_study_styles: true cid: caseStudies -css: /css/style_case_studies.css logo: netease_featured_logo.png featured: false + +new_case_study_styles: true +heading_background: /images/case-studies/netease/banner1.jpg +heading_title_logo: /images/netease_logo.png +subheading: > + How NetEase Leverages Kubernetes to Support Internet Business Worldwide +case_study_details: + - Company: NetEase + - Location: Hangzhou, China + - Industry: Internet technology --- +

Challenge

-
-

CASE STUDY:
How NetEase Leverages Kubernetes to Support Internet Business Worldwide

+

Its gaming business is one of the largest in the world, but that's not all that NetEase provides to Chinese consumers. The company also operates e-commerce, advertising, music streaming, online education, and email platforms; the last of which serves almost a billion users with free email services through sites like 163.com. In 2015, the NetEase Cloud team providing the infrastructure for all of these systems realized that their R&D process was slowing down developers. "Our users needed to prepare all of the infrastructure by themselves," says Feng Changjian, Architect for NetEase Cloud and Container Service. "We were eager to provide the infrastructure and tools for our users automatically via serverless container service."

-
+

Solution

-
- Company  NetEase     Location  Hangzhou, China     Industry  Internet technology -
+

After considering building its own orchestration solution, NetEase decided to base its private cloud platform on Kubernetes. The fact that the technology came out of Google gave the team confidence that it could keep up with NetEase's scale. "After our 2-to-3-month evaluation, we believed it could satisfy our needs," says Feng. The team started working with Kubernetes in 2015, before it was even 1.0. Today, the NetEase internal cloud platform—which also leverages the CNCF projects Prometheus, Envoy, Harbor, gRPC, and Helm—runs 10,000 nodes in a production cluster and can support up to 30,000 nodes in a cluster. Based on its learnings from its internal platform, the company introduced a Kubernetes-based cloud and microservices-oriented PaaS product, NetEase Qingzhou Microservice, to outside customers.

-
-
-
-
-

Challenge

- Its gaming business is one of the largest in the world, but that’s not all that NetEase provides to Chinese consumers. The company also operates e-commerce, advertising, music streaming, online education, and email platforms; the last of which serves almost a billion users with free email services through sites like 163.com. In 2015, the NetEase Cloud team providing the infrastructure for all of these systems realized that their R&D process was slowing down developers. “Our users needed to prepare all of the infrastructure by themselves,” says Feng Changjian, Architect for NetEase Cloud and Container Service. “We were eager to provide the infrastructure and tools for our users automatically via serverless container service.” -

-

Solution

- After considering building its own orchestration solution, NetEase decided to base its private cloud platform on Kubernetes. The fact that the technology came out of Google gave the team confidence that it could keep up with NetEase’s scale. “After our 2-to-3-month evaluation, we believed it could satisfy our needs,” says Feng. The team started working with Kubernetes in 2015, before it was even 1.0. Today, the NetEase internal cloud platform—which also leverages the CNCF projects Prometheus, Envoy, Harbor, gRPC, and Helm—runs 10,000 nodes in a production cluster and can support up to 30,000 nodes in a cluster. Based on its learnings from its internal platform, the company introduced a Kubernetes-based cloud and microservices-oriented PaaS product, NetEase Qingzhou Microservice, to outside customers. - - -

Impact

- The NetEase team reports that Kubernetes has increased R&D efficiency by more than 100%. Deployment efficiency has improved by 280%. “In the past, if we wanted to do upgrades, we needed to work with other teams, even in other departments,” says Feng. “We needed special staff to prepare everything, so it took about half an hour. Now we can do it in only 5 minutes.” The new platform also allows for mixed deployments using GPU and CPU resources. “Before, if we put all the resources toward the GPU, we won’t have spare resources for the CPU. But now we have improvements thanks to the mixed deployments,” he says. Those improvements have also brought an increase in resource utilization. -
-
-
-
-
- "The system can support 30,000 nodes in a single cluster. In production, we have gotten the data of 10,000 nodes in a single cluster. The whole internal system is using this system for development, test, and production."

— Zeng Yuxing, Architect, NetEase -
-
-
-
-

Its gaming business is the fifth-largest in the world, but that’s not all that NetEase provides consumers.

The company also operates e-commerce, advertising, music streaming, online education, and email platforms in China; the last of which serves almost a billion users with free email services through popular sites like 163.com and 126.com. With that kind of scale, the NetEase Cloud team providing the infrastructure for all of these systems realized in 2015 that their R&D process was making it hard for developers to keep up with demand. “Our users needed to prepare all of the infrastructure by themselves,” says Feng Changjian, Architect for NetEase Cloud and Container Service. “We were eager to provide the infrastructure and tools for our users automatically via serverless container service.”

- After considering building its own orchestration solution, NetEase decided to base its private cloud platform on Kubernetes. The fact that the technology came out of Google gave the team confidence that it could keep up with NetEase’s scale. “After our 2-to-3-month evaluation, we believed it could satisfy our needs,” says Feng. -
-
-
-
- "We leveraged the programmability of Kubernetes so that we can build a platform to satisfy the needs of our internal customers for upgrades and deployment." -

- Feng Changjian, Architect for NetEase Cloud and Container Service, NetEase -
-
-
-
- The team started adopting Kubernetes in 2015, before it was even 1.0, because it was relatively easy to use and enabled DevOps at the company. “We abandoned some of the concepts of Kubernetes; we only wanted to use the standardized framework,” says Feng. “We leveraged the programmability of Kubernetes so that we can build a platform to satisfy the needs of our internal customers for upgrades and deployment.”

- The team first focused on building the container platform to manage resources better, and then turned their attention to improving its support of microservices by adding internal systems such as monitoring. That has meant integrating the CNCF projects Prometheus, Envoy, Harbor, gRPC, and Helm. “We are trying to provide a simplified and standardized process, so our users and customers can leverage our best practices,” says Feng.

- And the team is continuing to make improvements. For example, the e-commerce part of the business needs to leverage mixed deployments, which in the past required using two separate platforms: the infrastructure-as-a-service platform and the Kubernetes platform. More recently, NetEase has created a cross-platform application that enables using both with one-command deployment. -
-
-
-
- "As long as a company has a mature team and enough developers, I think Kubernetes is a very good technology that can help them." -

- Li Lanqing, Kubernetes Developer, NetEase -
-
- +

The NetEase team reports that Kubernetes has increased R&D efficiency by more than 100%. Deployment efficiency has improved by 280%. "In the past, if we wanted to do upgrades, we needed to work with other teams, even in other departments," says Feng. "We needed special staff to prepare everything, so it took about half an hour. Now we can do it in only 5 minutes." The new platform also allows for mixed deployments using GPU and CPU resources. "Before, if we put all the resources toward the GPU, we won't have spare resources for the CPU. But now we have improvements thanks to the mixed deployments," he says. Those improvements have also brought an increase in resource utilization.

-
-
- Today, the NetEase internal cloud platform “can support 30,000 nodes in a single cluster,” says Architect Zeng Yuxing. “In production, we have gotten the data of 10,000 nodes in a single cluster. The whole internal system is using this system for development, test, and production.”

- The NetEase team reports that Kubernetes has increased R&D efficiency by more than 100%. Deployment efficiency has improved by 280%. “In the past, if we wanted to do upgrades, we needed to work with other teams, even in other departments,” says Feng. “We needed special staff to prepare everything, so it took about half an hour. Now we can do it in only 5 minutes.” The new platform also allows for mixed deployments using GPU and CPU resources. “Before, if we put all the resources toward the GPU, we won’t have spare resources for the CPU. But now we have improvements thanks to the mixed deployments.” Those improvements have also brought an increase in resource utilization. +{{< case-studies/quote author="Zeng Yuxing, Architect, NetEase" >}} +"The system can support 30,000 nodes in a single cluster. In production, we have gotten the data of 10,000 nodes in a single cluster. The whole internal system is using this system for development, test, and production." +{{< /case-studies/quote >}} -
+{{< case-studies/lead >}} +Its gaming business is the fifth-largest in the world, but that's not all that NetEase provides consumers. +{{< /case-studies/lead >}} -
-
- "By engaging with this community, we can gain some experience from it and we can also benefit from it. We can see what are the concerns and the challenges faced by the community, so we can get involved."

- Li Lanqing, Kubernetes Developer, NetEase +

The company also operates e-commerce, advertising, music streaming, online education, and email platforms in China; the last of which serves almost a billion users with free email services through popular sites like 163.com and 126.com. With that kind of scale, the NetEase Cloud team providing the infrastructure for all of these systems realized in 2015 that their R&D process was making it hard for developers to keep up with demand. "Our users needed to prepare all of the infrastructure by themselves," says Feng Changjian, Architect for NetEase Cloud and Container Service. "We were eager to provide the infrastructure and tools for our users automatically via serverless container service."

-
-
-
- Based on the results and learnings from using its internal platform, the company introduced a Kubernetes-based cloud and microservices-oriented PaaS product, NetEase Qingzhou Microservice, to outside customers. “The idea is that we can find the problems encountered by our game and e-commerce and cloud music providers, so we can integrate their experiences and provide a platform to satisfy the needs of our users,” says Zeng.

- With or without the use of the NetEase product, the team encourages other companies to try Kubernetes. “As long as a company has a mature team and enough developers, I think Kubernetes is a very good technology that can help them,” says Kubernetes developer Li Lanqing.

- As an end user as well as a vendor, NetEase has become more involved in the community, learning from other companies and sharing what they’ve done. The team has been contributing to the Harbor and Envoy projects, providing feedback as the technologies are being tested at NetEase scale. “We are a team focusing on addressing the challenges of microservices architecture,” says Feng. “By engaging with this community, we can gain some experience from it and we can also benefit from it. We can see what are the concerns and the challenges faced by the community, so we can get involved.” -
-
+

After considering building its own orchestration solution, NetEase decided to base its private cloud platform on Kubernetes. The fact that the technology came out of Google gave the team confidence that it could keep up with NetEase's scale. "After our 2-to-3-month evaluation, we believed it could satisfy our needs," says Feng.

+ +{{< case-studies/quote + image="/images/case-studies/netease/banner3.jpg" + author="Feng Changjian, Architect for NetEase Cloud and Container Service, NetEase" +>}} +"We leveraged the programmability of Kubernetes so that we can build a platform to satisfy the needs of our internal customers for upgrades and deployment." +{{< /case-studies/quote >}} + +

The team started adopting Kubernetes in 2015, before it was even 1.0, because it was relatively easy to use and enabled DevOps at the company. "We abandoned some of the concepts of Kubernetes; we only wanted to use the standardized framework," says Feng. "We leveraged the programmability of Kubernetes so that we can build a platform to satisfy the needs of our internal customers for upgrades and deployment."

+ +

The team first focused on building the container platform to manage resources better, and then turned their attention to improving its support of microservices by adding internal systems such as monitoring. That has meant integrating the CNCF projects Prometheus, Envoy, Harbor, gRPC, and Helm. "We are trying to provide a simplified and standardized process, so our users and customers can leverage our best practices," says Feng.

+ +

And the team is continuing to make improvements. For example, the e-commerce part of the business needs to leverage mixed deployments, which in the past required using two separate platforms: the infrastructure-as-a-service platform and the Kubernetes platform. More recently, NetEase has created a cross-platform application that enables using both with one-command deployment.

+ +{{< case-studies/quote + image="/images/case-studies/netease/banner4.jpg" + author="Li Lanqing, Kubernetes Developer, NetEase" +>}} +"As long as a company has a mature team and enough developers, I think Kubernetes is a very good technology that can help them." +{{< /case-studies/quote >}} + +

Today, the NetEase internal cloud platform "can support 30,000 nodes in a single cluster," says Architect Zeng Yuxing. "In production, we have gotten the data of 10,000 nodes in a single cluster. The whole internal system is using this system for development, test, and production."

+ +

The NetEase team reports that Kubernetes has increased R&D efficiency by more than 100%. Deployment efficiency has improved by 280%. "In the past, if we wanted to do upgrades, we needed to work with other teams, even in other departments," says Feng. "We needed special staff to prepare everything, so it took about half an hour. Now we can do it in only 5 minutes." The new platform also allows for mixed deployments using GPU and CPU resources. "Before, if we put all the resources toward the GPU, we won't have spare resources for the CPU. But now we have improvements thanks to the mixed deployments." Those improvements have also brought an increase in resource utilization.

+ +{{< case-studies/quote author="Li Lanqing, Kubernetes Developer, NetEase">}} +"By engaging with this community, we can gain some experience from it and we can also benefit from it. We can see what are the concerns and the challenges faced by the community, so we can get involved." +{{< /case-studies/quote >}} + +

Based on the results and learnings from using its internal platform, the company introduced a Kubernetes-based cloud and microservices-oriented PaaS product, NetEase Qingzhou Microservice, to outside customers. "The idea is that we can find the problems encountered by our game and e-commerce and cloud music providers, so we can integrate their experiences and provide a platform to satisfy the needs of our users," says Zeng.

+ +

With or without the use of the NetEase product, the team encourages other companies to try Kubernetes. "As long as a company has a mature team and enough developers, I think Kubernetes is a very good technology that can help them," says Kubernetes developer Li Lanqing.

+ +

As an end user as well as a vendor, NetEase has become more involved in the community, learning from other companies and sharing what they've done. The team has been contributing to the Harbor and Envoy projects, providing feedback as the technologies are being tested at NetEase scale. "We are a team focusing on addressing the challenges of microservices architecture," says Feng. "By engaging with this community, we can gain some experience from it and we can also benefit from it. We can see what are the concerns and the challenges faced by the community, so we can get involved."

diff --git a/content/en/case-studies/newyorktimes/index.html b/content/en/case-studies/newyorktimes/index.html index 53dbd06a55..ae57a9ec7d 100644 --- a/content/en/case-studies/newyorktimes/index.html +++ b/content/en/case-studies/newyorktimes/index.html @@ -2,107 +2,73 @@ title: New York Times Case Study case_study_styles: true cid: caseStudies -css: /css/style_case_studies.css + +new_case_study_styles: true +heading_background: /images/case-studies/newyorktimes/banner1.jpg +heading_title_logo: /images/newyorktimes_logo.png +subheading: > + The New York Times: From Print to the Web to Cloud Native +case_study_details: + - Company: New York Times + - Location: New York, N.Y. + - Industry: News Media --- -
-

CASE STUDY:
The New York Times: From Print to the Web to Cloud Native +

Challenge

-

+

When the company decided a few years ago to move out of its data centers, its first deployments on the public cloud were smaller, less critical applications managed on virtual machines. "We started building more and more tools, and at some point we realized that we were doing a disservice by treating Amazon as another data center," says Deep Kapadia, Executive Director, Engineering at The New York Times. Kapadia was tapped to lead a Delivery Engineering Team that would "design for the abstractions that cloud providers offer us."

-
+

Solution

-
- Company  New York Times     Location  New York, N.Y. -     Industry  News Media -
+

The team decided to use Google Cloud Platform and its Kubernetes-as-a-service offering, GKE.

-
-
-
-
-

Challenge

- When the company decided a few years ago to move out of its data centers, its first deployments on the public cloud were smaller, less critical applications managed on virtual machines. "We started building more and more tools, and at some point we realized that we were doing a disservice by treating Amazon as another data center," says Deep Kapadia, Executive Director, Engineering at The New York Times. Kapadia was tapped to lead a Delivery Engineering Team that would "design for the abstractions that cloud providers offer us." +

Impact

-

-

Solution

- The team decided to use Google Cloud Platform and its Kubernetes-as-a-service offering, GKE. -
-
+

Speed of delivery increased. Some of the legacy VM-based deployments took 45 minutes; with Kubernetes, that time was "just a few seconds to a couple of minutes," says Engineering Manager Brian Balser. Adds Li: "Teams that used to deploy on weekly schedules or had to coordinate schedules with the infrastructure team now deploy their updates independently, and can do it daily when necessary." Adopting Cloud Native Computing Foundation technologies allows for a more unified approach to deployment across the engineering staff, and portability for the company.

-
- -

Impact

- Speed of delivery increased. Some of the legacy VM-based deployments took 45 minutes; with Kubernetes, that time was "just a few seconds to a couple of minutes," says Engineering Manager Brian Balser. Adds Li: "Teams that used to deploy on weekly schedules or had to coordinate schedules with the infrastructure team now deploy their updates independently, and can do it daily when necessary." Adopting Cloud Native Computing Foundation technologies allows for a more unified approach to deployment across the engineering staff, and portability for the company. -
-
- -
-
-
-
-
- - -
-
-
-

-
-
+{{< case-studies/quote author="Deep Kapadia, Executive Director, Engineering at The New York Times">}} + +
- "I think once you get over the initial hump, things get a lot easier and actually a lot faster." — Deep Kapadia, Executive Director, Engineering at The New York Times
-
- -
-
- Founded in 1851 and known as the newspaper of record, The New York Times is a digital pioneer: Its first website launched in 1996, before Google even existed. After the company decided a few years ago to move out of its private data centers—including one located in the pricy real estate of Manhattan. It recently took another step into the future by going cloud native.

- At first, the infrastructure team "managed the virtual machines in the Amazon cloud, and they deployed more critical applications in our data centers and the less critical ones on AWS as an experiment," says Deep Kapadia, Executive Director, Engineering at The New York Times. "We started building more and more tools, and at some point we realized that we were doing a disservice by treating Amazon as another data center."

- To get the most out of the cloud, Kapadia was tapped to lead a new Delivery Engineering Team that would "design for the abstractions that cloud providers offer us." In mid-2016, they began looking at the Google Cloud Platform and its Kubernetes-as-a-service offering, GKE.

- At the time, says team member Tony Li, a Site Reliability Engineer, "We had some internal tooling that attempted to do what Kubernetes does for containers, but for VMs. We asked why are we building and maintaining these tools ourselves?"

- In early 2017, the first production application—the nytimes.com mobile homepage—began running on Kubernetes, serving just 1% of the traffic. Today, almost 100% of the nytimes.com site’s end-user facing applications run on GCP, with the majority on Kubernetes. +"I think once you get over the initial hump, things get a lot easier and actually a lot faster." +{{< /case-studies/quote >}} -
-
-
-
- "We had some internal tooling that attempted to do what Kubernetes does for containers, but for VMs. We asked why are we building and maintaining these tools ourselves?" -
-
-
-
+

Founded in 1851 and known as the newspaper of record, The New York Times is a digital pioneer: Its first website launched in 1996, before Google even existed. After the company decided a few years ago to move out of its private data centers—including one located in the pricy real estate of Manhattan. It recently took another step into the future by going cloud native.

- The team found that the speed of delivery was immediately impacted. "Deploying Docker images versus spinning up VMs was quite a lot faster," says Engineering Manager Brian Balser. Some of the legacy VM-based deployments took 45 minutes; with Kubernetes, that time was "just a few seconds to a couple of minutes."

- The plan is to get as much as possible, not just the website, running on Kubernetes, and beyond that, moving toward serverless deployments. For instance, The New York Times crossword app was built on Google App Engine, which has been the main platform for the company’s experimentation with serverless. "The hardest part was getting the engineers over the hurdle of how little they had to do," Chief Technology Officer Nick Rockwell recently told The CTO Advisor. "Our experience has been very, very good. We have invested a lot of work into deploying apps on container services, and I’m really excited about experimenting with deploying those on App Engine Flex and AWS Fargate and seeing how that feels, because that’s a great migration path."

- There are some exceptions to the move to cloud native, of course. "We have the print publishing business as well," says Kapadia. "A lot of that is definitely not going down the cloud-native path because they’re using vendor software and even special machinery that prints the physical paper. But even those teams are looking at things like App Engine and Kubernetes if they can."

- Kapadia acknowledges that there was a steep learning curve for some engineers, but "I think once you get over the initial hump, things get a lot easier and actually a lot faster." +

At first, the infrastructure team "managed the virtual machines in the Amazon cloud, and they deployed more critical applications in our data centers and the less critical ones on AWS as an experiment," says Deep Kapadia, Executive Director, Engineering at The New York Times. "We started building more and more tools, and at some point we realized that we were doing a disservice by treating Amazon as another data center."

-
-
-
-
- "Right now, every team is running a small Kubernetes cluster, but it would be nice if we could all live in a larger ecosystem," says Kapadia. "Then we can harness the power of things like service mesh proxies that can actually do a lot of instrumentation between microservices, or service-to-service orchestration. Those are the new things that we want to experiment with as we go forward." +

To get the most out of the cloud, Kapadia was tapped to lead a new Delivery Engineering Team that would "design for the abstractions that cloud providers offer us." In mid-2016, they began looking at the Google Cloud Platform and its Kubernetes-as-a-service offering, GKE.

-
-
-
-
- At The New York Times, they did. As teams started sharing their own best practices with each other, "We’re no longer the bottleneck for figuring out certain things," Kapadia says. "Most of the infrastructure and systems were managed by a centralized function. We’ve sort of blown that up, partly because Google and Amazon have tools that allow us to do that. We provide teams with complete ownership of their Google Cloud Platform projects, and give them a set of sensible defaults or standards. We let them know, ‘If this works for you as is, great! If not, come talk to us and we’ll figure out how to make it work for you.’"

- As a result, "It’s really allowed teams to move at a much more rapid pace than they were able to in the past," says Kapadia. Adds Li: "The use of GKE means each team can get their own compute cluster, reducing the number of individual instances they have to care about since developers can treat the cluster as a whole. Because the ticket-based workflow was removed from requesting resources and connections, developers can just call an API to get what they want. Teams that used to deploy on weekly schedules or had to coordinate schedules with the infrastructure team now deploy their updates independently, and can do it daily when necessary."

- Another benefit to adopting Kubernetes: allowing for a more unified approach to deployment across the engineering staff. "Before, many teams were building their own tools for deployment," says Balser. With Kubernetes—as well as the other CNCF projects The New York Times uses, including Fluentd to collect logs for all of its AWS servers, gRPC for its Publishing Pipeline, Prometheus, and Envoy—"we can benefit from the advances that each of these technologies make, instead of trying to catch up." +

At the time, says team member Tony Li, a Site Reliability Engineer, "We had some internal tooling that attempted to do what Kubernetes does for containers, but for VMs. We asked why are we building and maintaining these tools ourselves?"

-
+

In early 2017, the first production application—the nytimes.com mobile homepage—began running on Kubernetes, serving just 1% of the traffic. Today, almost 100% of the nytimes.com site's end-user facing applications run on GCP, with the majority on Kubernetes.

-
-
+{{< case-studies/quote image="/images/case-studies/newyorktimes/banner3.jpg" >}} +"We had some internal tooling that attempted to do what Kubernetes does for containers, but for VMs. We asked why are we building and maintaining these tools ourselves?" +{{< /case-studies/quote >}} -Li calls the Cloud Native Computing Foundation’s projects "a northern star that we can all look at and follow." -
-
+

The team found that the speed of delivery was immediately impacted. "Deploying Docker images versus spinning up VMs was quite a lot faster," says Engineering Manager Brian Balser. Some of the legacy VM-based deployments took 45 minutes; with Kubernetes, that time was "just a few seconds to a couple of minutes."

-
- These open-source technologies have given the company more portability. "CNCF has enabled us to follow an industry standard," says Kapadia. "It allows us to think about whether we want to move away from our current service providers. Most of our applications are connected to Fluentd. If we wish to switch our logging provider from provider A to provider B we can do that. We’re running Kubernetes in GCP today, but if we want to run it in Amazon or Azure, we could potentially look into that as well."

- Li calls the Cloud Native Computing Foundation’s projects "a northern star that we can all look at and follow." Led by that star, the team is looking ahead to a year of onboarding the remaining half of the 40 or so product engineering teams to extract even more value out of the technology. "Right now, every team is running a small Kubernetes cluster, but it would be nice if we could all live in a larger ecosystem," says Kapadia. "Then we can harness the power of things like service mesh proxies that can actually do a lot of instrumentation between microservices, or service-to-service orchestration. Those are the new things that we want to experiment with as we go forward." +

The plan is to get as much as possible, not just the website, running on Kubernetes, and beyond that, moving toward serverless deployments. For instance, The New York Times crossword app was built on Google App Engine, which has been the main platform for the company's experimentation with serverless. "The hardest part was getting the engineers over the hurdle of how little they had to do," Chief Technology Officer Nick Rockwell recently told The CTO Advisor. "Our experience has been very, very good. We have invested a lot of work into deploying apps on container services, and I'm really excited about experimenting with deploying those on App Engine Flex and AWS Fargate and seeing how that feels, because that's a great migration path."

-
-
+

There are some exceptions to the move to cloud native, of course. "We have the print publishing business as well," says Kapadia. "A lot of that is definitely not going down the cloud-native path because they're using vendor software and even special machinery that prints the physical paper. But even those teams are looking at things like App Engine and Kubernetes if they can."

+ +

Kapadia acknowledges that there was a steep learning curve for some engineers, but "I think once you get over the initial hump, things get a lot easier and actually a lot faster."

+ +{{< case-studies/quote image="/images/case-studies/newyorktimes/banner4.jpg" >}} +"Right now, every team is running a small Kubernetes cluster, but it would be nice if we could all live in a larger ecosystem," says Kapadia. "Then we can harness the power of things like service mesh proxies that can actually do a lot of instrumentation between microservices, or service-to-service orchestration. Those are the new things that we want to experiment with as we go forward." +{{< /case-studies/quote >}} + +

At The New York Times, they did. As teams started sharing their own best practices with each other, "We're no longer the bottleneck for figuring out certain things," Kapadia says. "Most of the infrastructure and systems were managed by a centralized function. We've sort of blown that up, partly because Google and Amazon have tools that allow us to do that. We provide teams with complete ownership of their Google Cloud Platform projects, and give them a set of sensible defaults or standards. We let them know, 'If this works for you as is, great! If not, come talk to us and we'll figure out how to make it work for you.'"

+ +

As a result, "It's really allowed teams to move at a much more rapid pace than they were able to in the past," says Kapadia. Adds Li: "The use of GKE means each team can get their own compute cluster, reducing the number of individual instances they have to care about since developers can treat the cluster as a whole. Because the ticket-based workflow was removed from requesting resources and connections, developers can just call an API to get what they want. Teams that used to deploy on weekly schedules or had to coordinate schedules with the infrastructure team now deploy their updates independently, and can do it daily when necessary."

+ +

Another benefit to adopting Kubernetes: allowing for a more unified approach to deployment across the engineering staff. "Before, many teams were building their own tools for deployment," says Balser. With Kubernetes—as well as the other CNCF projects The New York Times uses, including Fluentd to collect logs for all of its AWS servers, gRPC for its Publishing Pipeline, Prometheus, and Envoy—"we can benefit from the advances that each of these technologies make, instead of trying to catch up."

+ +{{< case-studies/quote >}} +Li calls the Cloud Native Computing Foundation's projects "a northern star that we can all look at and follow." +{{< /case-studies/quote >}} + +

These open-source technologies have given the company more portability. "CNCF has enabled us to follow an industry standard," says Kapadia. "It allows us to think about whether we want to move away from our current service providers. Most of our applications are connected to Fluentd. If we wish to switch our logging provider from provider A to provider B we can do that. We're running Kubernetes in GCP today, but if we want to run it in Amazon or Azure, we could potentially look into that as well."

+ +

Li calls the Cloud Native Computing Foundation's projects "a northern star that we can all look at and follow." Led by that star, the team is looking ahead to a year of onboarding the remaining half of the 40 or so product engineering teams to extract even more value out of the technology. "Right now, every team is running a small Kubernetes cluster, but it would be nice if we could all live in a larger ecosystem," says Kapadia. "Then we can harness the power of things like service mesh proxies that can actually do a lot of instrumentation between microservices, or service-to-service orchestration. Those are the new things that we want to experiment with as we go forward."

diff --git a/content/en/case-studies/nokia/index.html b/content/en/case-studies/nokia/index.html index f824685327..0de1426853 100644 --- a/content/en/case-studies/nokia/index.html +++ b/content/en/case-studies/nokia/index.html @@ -3,91 +3,75 @@ title: Nokia Case Study linkTitle: Nokia case_study_styles: true cid: caseStudies -css: /css/style_case_studies.css logo: nokia_featured_logo.png + +new_case_study_styles: true +heading_background: /images/case-studies/nokia/banner1.jpg +heading_title_logo: /images/nokia_logo.png +subheading: > + Nokia: Enabling 5G and DevOps at a Telecom Company with Kubernetes +case_study_details: + - Company: Nokia + - Location: Espoo, Finland + - Industry: Telecommunications --- +

Challenge

-
-

CASE STUDY:
Nokia: Enabling 5G and DevOps at a Telecom Company with Kubernetes +

Nokia's core business is building telecom networks end-to-end; its main products are related to the infrastructure, such as antennas, switching equipment, and routing equipment. "As telecom vendors, we have to deliver our software to several telecom operators and put the software into their infrastructure, and each of the operators have a bit different infrastructure," says Gergely Csatari, Senior Open Source Engineer. "There are operators who are running on bare metal. There are operators who are running on virtual machines. There are operators who are running on VMware Cloud and OpenStack Cloud. We want to run the same product on all of these different infrastructures without changing the product itself."

-

+

Solution

-
+

The company decided that moving to cloud native technologies would allow teams to have infrastructure-agnostic behavior in their products. Teams at Nokia began experimenting with Kubernetes in pre-1.0 versions. "The simplicity of the label-based scheduling of Kubernetes was a sign that showed us this architecture will scale, will be stable, and will be good for our purposes," says Csatari. The first Kubernetes-based product, the Nokia Telephony Application Server, went live in early 2018. "Now, all the products are doing some kind of re-architecture work, and they're moving to Kubernetes."

-
- Company  Nokia     Location  Espoo, Finland -     Industry  Telecommunications -
- -
-
-
-
-

Challenge

- Nokia’s core business is building telecom networks end-to-end; its main products are related to the infrastructure, such as antennas, switching equipment, and routing equipment. "As telecom vendors, we have to deliver our software to several telecom operators and put the software into their infrastructure, and each of the operators have a bit different infrastructure," says Gergely Csatari, Senior Open Source Engineer. "There are operators who are running on bare metal. There are operators who are running on virtual machines. There are operators who are running on VMware Cloud and OpenStack Cloud. We want to run the same product on all of these different infrastructures without changing the product itself." - -

Solution

- The company decided that moving to cloud native technologies would allow teams to have infrastructure-agnostic behavior in their products. Teams at Nokia began experimenting with Kubernetes in pre-1.0 versions. "The simplicity of the label-based scheduling of Kubernetes was a sign that showed us this architecture will scale, will be stable, and will be good for our purposes," says Csatari. The first Kubernetes-based product, the Nokia Telephony Application Server, went live in early 2018. "Now, all the products are doing some kind of re-architecture work, and they’re moving to Kubernetes." -

Impact

- Kubernetes has enabled Nokia’s foray into 5G. "When you develop something that is part of the operator’s infrastructure, you have to develop it for the future, and Kubernetes and containers are the forward-looking technologies," says Csatari. The teams using Kubernetes are already seeing clear benefits. "By separating the infrastructure and the application layer, we have less dependencies in the system, which means that it’s easier to implement features in the application layer," says Csatari. And because teams can test the exact same binary artifact independently of the target execution environment, "we find more errors in early phases of the testing, and we do not need to run the same tests on different target environments, like VMware, OpenStack, or bare metal," he adds. As a result, "we save several hundred hours in every release." -
+

Kubernetes has enabled Nokia's foray into 5G. "When you develop something that is part of the operator's infrastructure, you have to develop it for the future, and Kubernetes and containers are the forward-looking technologies," says Csatari. The teams using Kubernetes are already seeing clear benefits. "By separating the infrastructure and the application layer, we have less dependencies in the system, which means that it's easier to implement features in the application layer," says Csatari. And because teams can test the exact same binary artifact independently of the target execution environment, "we find more errors in early phases of the testing, and we do not need to run the same tests on different target environments, like VMware, OpenStack, or bare metal," he adds. As a result, "we save several hundred hours in every release."

-
-
-
-
- "When people are picking up their phones and making a call on Nokia networks, they are creating containers in the background with Kubernetes." -

- Gergely Csatari, Senior Open Source Engineer, Nokia -
-
-
-
-

Nokia was the first name in mobile phones when they were becoming ubiquitous in the late 1990s and early 2000s. But by 2014, the company had sold off its mobile device division and was focusing its core business not on the handhelds used for calls, but on the networks.

- Today, Nokia is building telecom networks end-to-end—from antennas to switching and routing equipment—serving operators in more than 120 countries. "As telecom vendors, we have to deliver our software to several telecom operators and put the software into their infrastructure, and each of the operators have a bit different infrastructure," says Gergely Csatari, Senior Open Source Engineer at Nokia. "There are operators who are running on bare metal. There are operators who are running on virtual machines. There are operators who are running on VMware Cloud and OpenStack Cloud. We want to run the same product on all of these different infrastructures without changing the product itself."

- Looking for a way to allow its teams to build products with infrastructure-agnostic behavior, the company decided to embrace containerization, Kubernetes, and other cloud native technologies, a move that is being made across the telecom industry. Since early 2018, "when people are picking up their phones and making a call on Nokia networks, they are creating containers in the background with Kubernetes," says Csatari. "Now, all the products are doing some kind of re-architecture work, and they’re moving to Kubernetes." +{{< case-studies/quote author="Gergely Csatari, Senior Open Source Engineer, Nokia" >}} +"When people are picking up their phones and making a call on Nokia networks, they are creating containers in the background with Kubernetes." +{{< /case-studies/quote >}} -
-
-
-
- "Having the community and CNCF around Kubernetes is not only important for having a connection to other companies who are using Kubernetes and a forum where you can ask or discuss features of Kubernetes. But as a company who would like to contribute to Kubernetes, it was very important to have a CLA (Contributors License Agreement) which is connected to the CNCF and not to a particular company. That was a critical step for us to start contributing to Kubernetes and Helm."

- Gergely Csatari, Senior Open Source Engineer, Nokia +{{< case-studies/lead >}} +Nokia was the first name in mobile phones when they were becoming ubiquitous in the late 1990s and early 2000s. But by 2014, the company had sold off its mobile device division and was focusing its core business not on the handhelds used for calls, but on the networks. +{{< /case-studies/lead >}} -
-
-
-
- Nokia’s cloud native journey began about two years ago, when Csatari’s team was building the company’s Telephony Application Server (TAS). "We wanted to have a service execution engine in the product, which was a totally separate function from all other parts," he says. "There, we had the possibility to think about new architectures and new tools that we could use. We created this particular product based on Kubernetes, and we liked the work, so we started to talk about cloud native and containers and all of these things. We did a very extensive research of different container orchestration tools. We knew that we have some, let’s say, strange or different requirements because of the special environment that our software is running on."

- For one thing, Nokia’s software serves millions of people, and is required to have the carrier-grade "five nines" availability: to be up 99.999% of the time. "If you turn it to minutes, this means we’re allowed to have only 10 minutes of downtime in a whole year," says Csatari. "Downtime here means that you are not able to serve the person to full capacity, which means that we cannot fail. This includes software upgrades, everything, because when you call 911, you’re using our software, and you expect that it will work."

- That meant that they needed to be able to set affinity and anti-affinity rules in their orchestration tools. "You cannot put all of the functions to the same physical host because physical hosts are failing," Csatari explains. "If you fail with one physical host, then you lose all of the core processing processes. Then there are no calls going through. So we have to divide them among the different physical hosts. At that time, only Kubernetes was able to provide these features. The simplicity of the label-based scheduling of Kubernetes was a sign that showed us this architecture will scale, will be stable, and will be good for our purposes." +

Today, Nokia is building telecom networks end-to-end—from antennas to switching and routing equipment—serving operators in more than 120 countries. "As telecom vendors, we have to deliver our software to several telecom operators and put the software into their infrastructure, and each of the operators have a bit different infrastructure," says Gergely Csatari, Senior Open Source Engineer at Nokia. "There are operators who are running on bare metal. There are operators who are running on virtual machines. There are operators who are running on VMware Cloud and OpenStack Cloud. We want to run the same product on all of these different infrastructures without changing the product itself."

-
-
-
-
- "Kubernetes opened the window to all of these open source projects instead of implementing everything in house. Our engineers can focus more on the application level, which is actually the thing what we are selling, and not on the infrastructure level. For us, the most important thing about Kubernetes is it allows us to focus on value creation of our business."

- Gergely Csatari, Senior Open Source Engineer, Nokia -
-
+

Looking for a way to allow its teams to build products with infrastructure-agnostic behavior, the company decided to embrace containerization, Kubernetes, and other cloud native technologies, a move that is being made across the telecom industry. Since early 2018, "when people are picking up their phones and making a call on Nokia networks, they are creating containers in the background with Kubernetes," says Csatari. "Now, all the products are doing some kind of re-architecture work, and they're moving to Kubernetes."

-
-
- The TAS went live in early 2018, and now Kubernetes is also enabling Nokia’s foray into 5G. The company is introducing microservices architecture and Kubernetes while adding 5G features to existing products. And all new 5G product development will be on top of Kubernetes. "When you develop something that is part of the operator’s infrastructure, you have to develop it for the future, and Kubernetes and containers are the forward-looking technologies," says Csatari.

- There have been real time savings thanks to Kubernetes. "By separating the infrastructure and the application layer, we have less dependencies in the system, which means that it’s easier to implement features in the application layer," says Csatari. Because teams can test the exact same binary artifact independently of the target execution environment, "we find more errors in early phases of the testing, and we do not need to run the same tests on different target environments, like VMware, OpenStack or bare metal," he adds. As a result, "we save several hundred hours in every release."

- Moving from Nokia’s legacy cluster management system, which had been built in-house more than thirty years ago, to a Kubernetes platform also meant that "we started using Linux as a base operating system, so we just opened the window to all of these open source projects instead of implementing everything in house," says Csatari. (From CNCF’s ecosystem, the team is already using Helm, gRPC, CNI, Prometheus, and Envoy, and plans to implement CoreDNS.) "Our engineers can focus more on the application level, which is actually the thing what we are selling, and not on the infrastructure level. For us, the most important thing about Kubernetes is it allows us to focus on value creation of our business." +{{< case-studies/quote + image="/images/case-studies/nokia/banner3.jpg" + author="Gergely Csatari, Senior Open Source Engineer, Nokia" +>}} +"Having the community and CNCF around Kubernetes is not only important for having a connection to other companies who are using Kubernetes and a forum where you can ask or discuss features of Kubernetes. But as a company who would like to contribute to Kubernetes, it was very important to have a CLA (Contributors License Agreement) which is connected to the CNCF and not to a particular company. That was a critical step for us to start contributing to Kubernetes and Helm." +{{< /case-studies/quote >}} -
+

Nokia's cloud native journey began about two years ago, when Csatari's team was building the company's Telephony Application Server (TAS). "We wanted to have a service execution engine in the product, which was a totally separate function from all other parts," he says. "There, we had the possibility to think about new architectures and new tools that we could use. We created this particular product based on Kubernetes, and we liked the work, so we started to talk about cloud native and containers and all of these things. We did a very extensive research of different container orchestration tools. We knew that we have some, let's say, strange or different requirements because of the special environment that our software is running on."

-
-
-"I had some discussions at KubeCon with people from the networking SIG and the resource management working group, to work together on our requirements, and that’s very exciting for me and my colleagues,"

- Gergely Csatari, Senior Open Source Engineer, Nokia
-
+

For one thing, Nokia's software serves millions of people, and is required to have the carrier-grade "five nines" availability: to be up 99.999% of the time. "If you turn it to minutes, this means we're allowed to have only 10 minutes of downtime in a whole year," says Csatari. "Downtime here means that you are not able to serve the person to full capacity, which means that we cannot fail. This includes software upgrades, everything, because when you call 911, you're using our software, and you expect that it will work."

-
- The company has a long-term goal of moving the entire product portfolio into the Kubernetes platform. To that end, Nokia teams are working together with other companies to add the features needed to use Kubernetes with the real-time, nanosecond-sensitive applications close to the edge of the radio network.

- And the CNCF community is proving to be a great forum for that collaboration. "I had some discussions at KubeCon with people from the networking SIG and the resource management working group, to work together on our requirements, and that’s very exciting for me and my colleagues," says Csatari. "Previously, everybody had the same problem, but everybody just did it in his own, and now we are trying to solve the same problem together."

- Perhaps the biggest impact that Kubernetes is having on Nokia, Csatari believes, is that people are starting to think about how a telecom company can do DevOps. "We are building a DevOps pipeline, which reaches from the actual developer to the customers, and thinking about new ways how can we digitally deliver our software to our customers and get feedback from the customers right to the engineers," he says. "This is something that will fundamentally change how telecom companies are delivering software, and how quickly can we develop new features. This is because of the usage of containers and, of course, the usage of Kubernetes." +

That meant that they needed to be able to set affinity and anti-affinity rules in their orchestration tools. "You cannot put all of the functions to the same physical host because physical hosts are failing," Csatari explains. "If you fail with one physical host, then you lose all of the core processing processes. Then there are no calls going through. So we have to divide them among the different physical hosts. At that time, only Kubernetes was able to provide these features. The simplicity of the label-based scheduling of Kubernetes was a sign that showed us this architecture will scale, will be stable, and will be good for our purposes."

-
-
+{{< case-studies/quote + image="/images/case-studies/nokia/banner4.jpg" + author="Gergely Csatari, Senior Open Source Engineer, Nokia" +>}} +"Kubernetes opened the window to all of these open source projects instead of implementing everything in house. Our engineers can focus more on the application level, which is actually the thing what we are selling, and not on the infrastructure level. For us, the most important thing about Kubernetes is it allows us to focus on value creation of our business." +{{< /case-studies/quote >}} + +

The TAS went live in early 2018, and now Kubernetes is also enabling Nokia's foray into 5G. The company is introducing microservices architecture and Kubernetes while adding 5G features to existing products. And all new 5G product development will be on top of Kubernetes. "When you develop something that is part of the operator's infrastructure, you have to develop it for the future, and Kubernetes and containers are the forward-looking technologies," says Csatari.

+ +

There have been real time savings thanks to Kubernetes. "By separating the infrastructure and the application layer, we have less dependencies in the system, which means that it's easier to implement features in the application layer," says Csatari. Because teams can test the exact same binary artifact independently of the target execution environment, "we find more errors in early phases of the testing, and we do not need to run the same tests on different target environments, like VMware, OpenStack or bare metal," he adds. As a result, "we save several hundred hours in every release."

+ +

Moving from Nokia's legacy cluster management system, which had been built in-house more than thirty years ago, to a Kubernetes platform also meant that "we started using Linux as a base operating system, so we just opened the window to all of these open source projects instead of implementing everything in house," says Csatari. (From CNCF's ecosystem, the team is already using Helm, gRPC, CNI, Prometheus, and Envoy, and plans to implement CoreDNS.) "Our engineers can focus more on the application level, which is actually the thing what we are selling, and not on the infrastructure level. For us, the most important thing about Kubernetes is it allows us to focus on value creation of our business."

+ +{{< case-studies/quote author="Gergely Csatari, Senior Open Source Engineer, Nokia" >}} +"I had some discussions at KubeCon with people from the networking SIG and the resource management working group, to work together on our requirements, and that's very exciting for me and my colleagues," +{{< /case-studies/quote >}} + +

The company has a long-term goal of moving the entire product portfolio into the Kubernetes platform. To that end, Nokia teams are working together with other companies to add the features needed to use Kubernetes with the real-time, nanosecond-sensitive applications close to the edge of the radio network.

+ +

And the CNCF community is proving to be a great forum for that collaboration. "I had some discussions at KubeCon with people from the networking SIG and the resource management working group, to work together on our requirements, and that's very exciting for me and my colleagues," says Csatari. "Previously, everybody had the same problem, but everybody just did it in his own, and now we are trying to solve the same problem together."

+ +

Perhaps the biggest impact that Kubernetes is having on Nokia, Csatari believes, is that people are starting to think about how a telecom company can do DevOps. "We are building a DevOps pipeline, which reaches from the actual developer to the customers, and thinking about new ways how can we digitally deliver our software to our customers and get feedback from the customers right to the engineers," he says. "This is something that will fundamentally change how telecom companies are delivering software, and how quickly can we develop new features. This is because of the usage of containers and, of course, the usage of Kubernetes."

diff --git a/content/en/case-studies/nordstrom/index.html b/content/en/case-studies/nordstrom/index.html index 788453de35..73bc4e147e 100644 --- a/content/en/case-studies/nordstrom/index.html +++ b/content/en/case-studies/nordstrom/index.html @@ -2,109 +2,74 @@ title: Nordstrom Case Study case_study_styles: true cid: caseStudies -css: /css/style_case_studies.css + +new_case_study_styles: true +heading_background: /images/case-studies/nordstrom/banner1.jpg +heading_title_logo: /images/nordstrom_logo.png +subheading: > + Finding Millions in Potential Savings in a Tough Retail Climate +case_study_details: + - Company: Nordstrom + - Location: Seattle, Washington + - Industry: Retail --- -
-

CASE STUDY:
Finding Millions in Potential Savings in a Tough Retail Climate +

Challenge

+

Nordstrom wanted to increase the efficiency and speed of its technology operations, which includes the Nordstrom.com e-commerce site. At the same time, Nordstrom Technology was looking for ways to tighten its technology operational costs.

-

+

Solution

-
+

After embracing a DevOps transformation and launching a continuous integration/continuous deployment (CI/CD) project four years ago, the company reduced its deployment time from three months to 30 minutes. But they wanted to go even faster across environments, so they began their cloud native journey, adopting Docker containers orchestrated with Kubernetes.

-
- Company  Nordstrom     Location  Seattle, Washington     Industry  Retail -
+

Impact

-
-
-
-
-

Challenge

- Nordstrom wanted to increase the efficiency and speed of its technology operations, which includes the Nordstrom.com e-commerce site. At the same time, Nordstrom Technology was looking for ways to tighten its technology operational costs. +

Nordstrom Technology developers using Kubernetes now deploy faster and can "just focus on writing applications," says Dhawal Patel, a senior engineer on the team building a Kubernetes enterprise platform for Nordstrom. Furthermore, the team has increased Ops efficiency, improving CPU utilization from 5x to 12x depending on the workload. "We run thousands of virtual machines (VMs), but aren't effectively using all those resources," says Patel. "With Kubernetes, without even trying to make our cluster efficient, we are currently at a 10x increase."

-
-

Solution

- After embracing a DevOps transformation and launching a continuous integration/continuous deployment (CI/CD) project four years ago, the company reduced its deployment time from three months to 30 minutes. But they wanted to go even faster across environments, so they began their cloud native journey, adopting Docker containers orchestrated with Kubernetes. +{{< case-studies/quote author="Dhawal Patel, senior engineer at Nordstrom" >}} +"We are always looking for ways to optimize and provide more value through technology. With Kubernetes we are showcasing two types of efficiency that we can bring: Dev efficiency and Ops efficiency. It's a win-win." +{{< /case-studies/quote >}} -
+

When Dhawal Patel joined Nordstrom five years ago as an application developer for the retailer's website, he realized there was an opportunity to help speed up development cycles.

+

In those early DevOps days, Nordstrom Technology still followed a traditional model of silo teams and functions. "As a developer, I was spending more time fixing environments than writing code and adding value to business," Patel says. "I was passionate about that—so I was given the opportunity to help fix it."

-
+

The company was eager to move faster, too, and in 2013 launched the first continuous integration/continuous deployment (CI/CD) project. That project was the first step in Nordstrom's cloud native journey.

-
+

Dev and Ops team members built a CI/CD pipeline, working with the company's servers on premise. The team chose Chef, and wrote cookbooks that automated virtual IP creation, servers, and load balancing. "After we completed the project, deployment went from three months to 30 minutes," says Patel. "We still had multiple environments—dev, test, staging, then production—so with each environment running the Chef cookbooks, it took 30 minutes. It was a huge achievement at that point."

-

Impact

- Nordstrom Technology developers using Kubernetes now deploy faster and can "just focus on writing applications," says Dhawal Patel, a senior engineer on the team building a Kubernetes enterprise platform for Nordstrom. Furthermore, the team has increased Ops efficiency, improving CPU utilization from 5x to 12x depending on the workload. "We run thousands of virtual machines (VMs), but aren’t effectively using all those resources," says Patel. "With Kubernetes, without even trying to make our cluster efficient, we are currently at a 10x increase." +

But new environments still took too long to turn up, so the next step was working in the cloud. Today, Nordstrom Technology has built an enterprise platform that allows the company's 1,500 developers to deploy applications running as Docker containers in the cloud, orchestrated with Kubernetes.

-
-
-
-
-
- "We are always looking for ways to optimize and provide more value through technology. With Kubernetes we are showcasing two types of efficiency that we can bring: Dev efficiency and Ops efficiency. It’s a win-win." -

-— Dhawal Patel, senior engineer at Nordstrom -
-
-
-
- When Dhawal Patel joined Nordstrom five years ago as an application developer for the retailer’s website, he realized there was an opportunity to help speed up development cycles. -

- In those early DevOps days, Nordstrom Technology still followed a traditional model of silo teams and functions. "As a developer, I was spending more time fixing environments than writing code and adding value to business," Patel says. "I was passionate about that—so I was given the opportunity to help fix it." -

- The company was eager to move faster, too, and in 2013 launched the first continuous integration/continuous deployment (CI/CD) project. That project was the first step in Nordstrom’s cloud native journey. -

- Dev and Ops team members built a CI/CD pipeline, working with the company’s servers on premise. The team chose Chef, and wrote cookbooks that automated virtual IP creation, servers, and load balancing. "After we completed the project, deployment went from three months to 30 minutes," says Patel. "We still had multiple environments—dev, test, staging, then production—so with each environment running the Chef cookbooks, it took 30 minutes. It was a huge achievement at that point." -

But new environments still took too long to turn up, so the next step was working in the cloud. Today, Nordstrom Technology has built an enterprise platform that allows the company’s 1,500 developers to deploy applications running as Docker containers in the cloud, orchestrated with Kubernetes. +{{< case-studies/quote image="/images/case-studies/nordstrom/banner3.jpg" >}} +"We made a bet that Kubernetes was going to take off, informed by early indicators of community support and project velocity, so we rebuilt our system with Kubernetes at the core," +{{< /case-studies/quote >}} -
-
-
-
- "We made a bet that Kubernetes was going to take off, informed by early indicators of community support and project velocity, so we rebuilt our system with Kubernetes at the core," -
-
-
-
+

"The cloud provided faster access to resources, because it took weeks for us to get a virtual machine (VM) on premises," says Patel. "But now we can do the same thing in only five minutes."

-"The cloud provided faster access to resources, because it took weeks for us to get a virtual machine (VM) on premises," says Patel. "But now we can do the same thing in only five minutes." -

-Nordstrom’s first foray into scheduling containers on a cluster was a homegrown system based on CoreOS fleet. They began doing a few proofs of concept projects with that system until Kubernetes 1.0 was released when they made the switch. "We made a bet that Kubernetes was going to take off, informed by early indicators of community support and project velocity, so we rebuilt our system with Kubernetes at the core," says Marius Grigoriu, Sr. Manager of the Kubernetes team at Nordstrom. -While Kubernetes is often thought as a platform for microservices, the first application to launch on Kubernetes in a critical production role at Nordstrom was Jira. "It was not the ideal microservice we were hoping to get as our first application," Patel admits, "but the team that was working on it was really passionate about Docker and Kubernetes, and they wanted to try it out. They had their application running on premises, and wanted to move it to Kubernetes." -

-The benefits were immediate for the teams that came on board. "Teams running on our Kubernetes cluster loved the fact that they had fewer issues to worry about. They didn’t need to manage infrastructure or operating systems," says Grigoriu. "Early adopters loved the declarative nature of Kubernetes. They loved the reduced surface area they had to deal with." +

Nordstrom's first foray into scheduling containers on a cluster was a homegrown system based on CoreOS fleet. They began doing a few proofs of concept projects with that system until Kubernetes 1.0 was released when they made the switch. "We made a bet that Kubernetes was going to take off, informed by early indicators of community support and project velocity, so we rebuilt our system with Kubernetes at the core," says Marius Grigoriu, Sr. Manager of the Kubernetes team at Nordstrom.

-
-
-
-
- "Teams running on our Kubernetes cluster loved the fact that they had fewer issues to worry about. They didn’t need to manage infrastructure or operating systems," says Grigoriu. "Early adopters loved the declarative nature of Kubernetes. They loved the reduced surface area they had to deal with." -
-
+

While Kubernetes is often thought as a platform for microservices, the first application to launch on Kubernetes in a critical production role at Nordstrom was Jira. "It was not the ideal microservice we were hoping to get as our first application," Patel admits, "but the team that was working on it was really passionate about Docker and Kubernetes, and they wanted to try it out. They had their application running on premises, and wanted to move it to Kubernetes."

-
-
- To support these early adopters, Patel’s team began growing the cluster and building production-grade services. "We integrated with Prometheus for monitoring, with a Grafana front end; we used Fluentd to push logs to Elasticsearch, so that gives us log aggregation," says Patel. The team also added dozens of open-source components, including CNCF projects and has made contributions to Kubernetes, Terraform, and kube2iam. -

-There are now more than 60 development teams running Kubernetes in Nordstrom Technology, and as success stories have popped up, more teams have gotten on board. "Our initial customer base, the ones who were willing to try this out, are now going and evangelizing to the next set of users," says Patel. "One early adopter had Docker containers and he was not sure how to run it in production. We sat with him and within 15 minutes we deployed it in production. He thought it was amazing, and more people in his org started coming in." -

-For Nordstrom Technology, going cloud-native has vastly improved development and operational efficiency. The developers using Kubernetes now deploy faster and can focus on building value in their applications. One such team started with a 25-minute merge to deploy by launching virtual machines in the cloud. Switching to Kubernetes was a 5x speedup in their process, improving their merge to deploy time to 5 minutes. -
+

The benefits were immediate for the teams that came on board. "Teams running on our Kubernetes cluster loved the fact that they had fewer issues to worry about. They didn't need to manage infrastructure or operating systems," says Grigoriu. "Early adopters loved the declarative nature of Kubernetes. They loved the reduced surface area they had to deal with."

-
-
- "With Kubernetes, without even trying to make our cluster efficient, we are currently at 40 percent CPU utilization—a 10x increase. we are running 2600+ customer pods that would have been 2600+ VMs if they had gone directly to the cloud. We are running them on 40 VMs now, so that’s a huge reduction in operational overhead." -
-
+{{< case-studies/quote image="/images/case-studies/nordstrom/banner4.jpg">}} +"Teams running on our Kubernetes cluster loved the fact that they had fewer issues to worry about. They didn't need to manage infrastructure or operating systems," says Grigoriu. "Early adopters loved the declarative nature of Kubernetes. They loved the reduced surface area they had to deal with." +{{< /case-studies/quote >}} -
- Speed is great, and easily demonstrated, but perhaps the bigger impact lies in the operational efficiency. "We run thousands of VMs on AWS, and their overall average CPU utilization is about four percent," says Patel. "With Kubernetes, without even trying to make our cluster efficient, we are currently at 40 percent CPU utilization—a 10x increase. We are running 2600+ customer pods that would have been 2600+ VMs if they had gone directly to the cloud. We are running them on 40 VMs now, so that’s a huge reduction in operational overhead." -

- Nordstrom Technology is also exploring running Kubernetes on bare metal on premises. "If we can build an on-premises Kubernetes cluster," says Patel, "we could bring the power of cloud to provision resources fast on-premises. Then for the developer, their interface is Kubernetes; they might not even realize or care that their services are now deployed on premises because they’re only working with Kubernetes." - For that reason, Patel is eagerly following Kubernetes’ development of multi-cluster capabilities. "With cluster federation, we can have our on-premise as the primary cluster and the cloud as a secondary burstable cluster," he says. "So, when there is an anniversary sale or Black Friday sale, and we need more containers - we can go to the cloud." -

- That kind of possibility—as well as the impact that Grigoriu and Patel’s team has already delivered using Kubernetes—is what led Nordstrom on its cloud native journey in the first place. "The way the retail environment is today, we are trying to build responsiveness and flexibility where we can," says Grigoriu. "Kubernetes makes it easy to: bring efficiency to both the Dev and Ops side of the equation. It’s a win-win." +

To support these early adopters, Patel's team began growing the cluster and building production-grade services. "We integrated with Prometheus for monitoring, with a Grafana front end; we used Fluentd to push logs to Elasticsearch, so that gives us log aggregation," says Patel. The team also added dozens of open-source components, including CNCF projects and has made contributions to Kubernetes, Terraform, and kube2iam.

-
-
+

There are now more than 60 development teams running Kubernetes in Nordstrom Technology, and as success stories have popped up, more teams have gotten on board. "Our initial customer base, the ones who were willing to try this out, are now going and evangelizing to the next set of users," says Patel. "One early adopter had Docker containers and he was not sure how to run it in production. We sat with him and within 15 minutes we deployed it in production. He thought it was amazing, and more people in his org started coming in."

+ +

For Nordstrom Technology, going cloud-native has vastly improved development and operational efficiency. The developers using Kubernetes now deploy faster and can focus on building value in their applications. One such team started with a 25-minute merge to deploy by launching virtual machines in the cloud. Switching to Kubernetes was a 5x speedup in their process, improving their merge to deploy time to 5 minutes.

+ +{{< case-studies/quote >}} +"With Kubernetes, without even trying to make our cluster efficient, we are currently at 40 percent CPU utilization—a 10x increase. we are running 2600+ customer pods that would have been 2600+ VMs if they had gone directly to the cloud. We are running them on 40 VMs now, so that's a huge reduction in operational overhead." +{{< /case-studies/quote >}} + +

Speed is great, and easily demonstrated, but perhaps the bigger impact lies in the operational efficiency. "We run thousands of VMs on AWS, and their overall average CPU utilization is about four percent," says Patel. "With Kubernetes, without even trying to make our cluster efficient, we are currently at 40 percent CPU utilization—a 10x increase. We are running 2600+ customer pods that would have been 2600+ VMs if they had gone directly to the cloud. We are running them on 40 VMs now, so that's a huge reduction in operational overhead."

+ +

Nordstrom Technology is also exploring running Kubernetes on bare metal on premises. "If we can build an on-premises Kubernetes cluster," says Patel, "we could bring the power of cloud to provision resources fast on-premises. Then for the developer, their interface is Kubernetes; they might not even realize or care that their services are now deployed on premises because they're only working with Kubernetes."

+ +

For that reason, Patel is eagerly following Kubernetes' development of multi-cluster capabilities. "With cluster federation, we can have our on-premise as the primary cluster and the cloud as a secondary burstable cluster," he says. "So, when there is an anniversary sale or Black Friday sale, and we need more containers - we can go to the cloud."

+ +

That kind of possibility—as well as the impact that Grigoriu and Patel's team has already delivered using Kubernetes—is what led Nordstrom on its cloud native journey in the first place. "The way the retail environment is today, we are trying to build responsiveness and flexibility where we can," says Grigoriu. "Kubernetes makes it easy to: bring efficiency to both the Dev and Ops side of the equation. It's a win-win."

diff --git a/content/en/case-studies/northwestern-mutual/index.html b/content/en/case-studies/northwestern-mutual/index.html index 47b4bbc7be..a25d25484c 100644 --- a/content/en/case-studies/northwestern-mutual/index.html +++ b/content/en/case-studies/northwestern-mutual/index.html @@ -2,98 +2,68 @@ title: Northwestern Mutual Case Study case_study_styles: true cid: caseStudies -css: /css/style_case_studies.css + +new_case_study_styles: true +heading_background: /images/case-studies/northwestern/banner1.jpg +heading_title_logo: /images/northwestern_logo.png +subheading: > + Cloud Native at Northwestern Mutual +case_study_details: + - Company: Northwestern Mutual + - Location: Milwaukee, WI + - Industry: Insurance and Financial Services --- -
-

CASE STUDY:
Cloud Native at Northwestern Mutual +

Challenge

+

In the spring of 2015, Northwestern Mutual acquired a fintech startup, LearnVest, and decided to take "Northwestern Mutual's leading products and services and meld it with LearnVest's digital experience and innovative financial planning platform," says Brad Williams, Director of Engineering for Client Experience, Northwestern Mutual. The company's existing infrastructure had been optimized for batch workflows hosted on on-prem networks; deployments were very traditional, focused on following a process instead of providing deployment agility. "We had to build a platform that was elastically scalable, but also much more responsive, so we could quickly get data to the client website so our end-customers have the experience they expect," says Williams.

-

+

Solution

-
- -
- Company  Northwestern Mutual     Location  Milwaukee, WI     Industry  Insurance and Financial Services -
- -
-
-
-
-

Challenge

- In the spring of 2015, Northwestern Mutual acquired a fintech startup, LearnVest, and decided to take "Northwestern Mutual’s leading products and services and meld it with LearnVest’s digital experience and innovative financial planning platform," says Brad Williams, Director of Engineering for Client Experience, Northwestern Mutual. The company’s existing infrastructure had been optimized for batch workflows hosted on on-prem networks; deployments were very traditional, focused on following a process instead of providing deployment agility. "We had to build a platform that was elastically scalable, but also much more responsive, so we could quickly get data to the client website so our end-customers have the experience they expect," says Williams. -
-

Solution

- The platform team came up with a plan for using the public cloud (AWS), Docker containers, and Kubernetes for orchestration. "Kubernetes gave us that base framework so teams can be very autonomous in what they’re building and deliver very quickly and frequently," says Northwestern Mutual Cloud Native Engineer Frank Greco Jr. The team also built and open-sourced Kanali, a Kubernetes-native API management tool that uses OpenTracing, Jaeger, and gRPC. - - -
- -
+

The platform team came up with a plan for using the public cloud (AWS), Docker containers, and Kubernetes for orchestration. "Kubernetes gave us that base framework so teams can be very autonomous in what they're building and deliver very quickly and frequently," says Northwestern Mutual Cloud Native Engineer Frank Greco Jr. The team also built and open-sourced Kanali, a Kubernetes-native API management tool that uses OpenTracing, Jaeger, and gRPC.

Impact

- Before, infrastructure deployments could take weeks; now, it is done in a matter of minutes. The number of deployments has increased dramatically, from about 24 a year to over 500 in just the first 10 months of 2017. Availability has also increased: There used to be a six-hour control window for commits every Sunday morning, as well as other periods of general maintenance, during which outages could happen. "Now we have eliminated the planned outage windows," says Bryan Pfremmer, App Platform Teams Manager, Northwestern Mutual. Kanali has had an impact on the bottom line. The vendor API management product that the company previously used required 23 servers, "dedicated, to only API management," says Pfremmer. "Now it’s all integrated in the existing stack and running as another deployment on Kubernetes. And that’s just one environment. Between the three that we had plus the test, that’s hard dollar savings." -
-
-
-
-
-
-"In a large enterprise, you’re going to have people using Kubernetes, but then you’re also going to have people using WAS and .NET. You may not be at a point where your whole stack can be cloud native. What if you can take your API management tool and make it cloud native, but still proxy to legacy systems? Using different pieces that are cloud native, open source and Kubernetes native, you can do pretty innovative stuff." — Frank Greco Jr., Cloud Native Engineer at Northwestern Mutual
-
-
-
-

For more than 160 years, Northwestern Mutual has maintained its industry leadership in part by keeping a strong focus on risk management.

- For many years, the company took a similar approach to managing its technology and has recently undergone a digital transformation to advance the company’s digital strategy - including making a lot of noise in the cloud-native world.

-In the spring of 2015, this insurance and financial services company acquired a fintech startup, LearnVest, and decided to take "Northwestern Mutual’s leading products and services and meld it with LearnVest’s digital experience and innovative financial planning platform," says Brad Williams, Director of Engineering for Client Experience, Northwestern Mutual. The company’s existing infrastructure had been optimized for batch workflows hosted on an on-premise datacenter; deployments were very traditional and had to many manual steps that were error prone.

-In order to give the company’s 4.5 million clients the digital experience they’d come to expect, says Williams, "We had to build a platform that was elastically scalable, but also much more responsive, so we could quickly get data to the client website. We essentially said, 'You build the system that you think is necessary to support a new, modern-facing one.’ That’s why we departed from anything legacy." +

Before, infrastructure deployments could take weeks; now, it is done in a matter of minutes. The number of deployments has increased dramatically, from about 24 a year to over 500 in just the first 10 months of 2017. Availability has also increased: There used to be a six-hour control window for commits every Sunday morning, as well as other periods of general maintenance, during which outages could happen. "Now we have eliminated the planned outage windows," says Bryan Pfremmer, App Platform Teams Manager, Northwestern Mutual. Kanali has had an impact on the bottom line. The vendor API management product that the company previously used required 23 servers, "dedicated, to only API management," says Pfremmer. "Now it's all integrated in the existing stack and running as another deployment on Kubernetes. And that's just one environment. Between the three that we had plus the test, that's hard dollar savings."

+{{< case-studies/quote author="Frank Greco Jr., Cloud Native Engineer at Northwestern Mutual">}} +"In a large enterprise, you're going to have people using Kubernetes, but then you're also going to have people using WAS and .NET. You may not be at a point where your whole stack can be cloud native. What if you can take your API management tool and make it cloud native, but still proxy to legacy systems? Using different pieces that are cloud native, open source and Kubernetes native, you can do pretty innovative stuff." +{{< /case-studies/quote >}} -
-
-
-
- "Kubernetes has definitely been the right choice for us. It gave us that base framework so teams can be autonomous in what they’re building and deliver very quickly and frequently." +{{< case-studies/lead >}} +For more than 160 years, Northwestern Mutual has maintained its industry leadership in part by keeping a strong focus on risk management. +{{< /case-studies/lead >}} -
-
-
-
- Williams and the rest of the platform team decided that the first step would be to start moving from private data centers to AWS. With a new microservice architecture in mind—and the freedom to implement what was best for the organization—they began using Docker containers. After looking into the various container orchestration options, they went with Kubernetes, even though it was still in beta at the time. "There was some debate whether we should build something ourselves, or just leverage that product and evolve with it," says Northwestern Mutual Cloud Native Engineer Frank Greco Jr. "Kubernetes has definitely been the right choice for us. It gave us that base framework so teams can be autonomous in what they’re building and deliver very quickly and frequently."

-As early adopters, the team had to do a lot of work with Ansible scripts to stand up the cluster. "We had a lot of hard security requirements given the nature of our business," explains Bryan Pfremmer, App Platform Teams Manager, Northwestern Mutual. "We found ourselves running a configuration that very few other people ever tried." The client experience group was the first to use the new platform; today, a few hundred of the company’s 1,500 engineers are using it and more are eager to get on board. -The results have been dramatic. Before, infrastructure deployments could take two weeks; now, it is done in a matter of minutes. Now with a focus on Infrastructure automation, and self-service, "You can take an app to production in that same day if you want to," says Pfremmer. +

For many years, the company took a similar approach to managing its technology and has recently undergone a digital transformation to advance the company's digital strategy - including making a lot of noise in the cloud-native world.

+

In the spring of 2015, this insurance and financial services company acquired a fintech startup, LearnVest, and decided to take "Northwestern Mutual's leading products and services and meld it with LearnVest's digital experience and innovative financial planning platform," says Brad Williams, Director of Engineering for Client Experience, Northwestern Mutual. The company's existing infrastructure had been optimized for batch workflows hosted on an on-premise datacenter; deployments were very traditional and had to many manual steps that were error prone.

-
-
-
-
+

In order to give the company's 4.5 million clients the digital experience they'd come to expect, says Williams, "We had to build a platform that was elastically scalable, but also much more responsive, so we could quickly get data to the client website. We essentially said, 'You build the system that you think is necessary to support a new, modern-facing one.' That's why we departed from anything legacy."

+ +{{< case-studies/quote image="/images/case-studies/northwestern/banner3.jpg" >}} +"Kubernetes has definitely been the right choice for us. It gave us that base framework so teams can be autonomous in what they're building and deliver very quickly and frequently." +{{< /case-studies/quote >}} + +

Williams and the rest of the platform team decided that the first step would be to start moving from private data centers to AWS. With a new microservice architecture in mind—and the freedom to implement what was best for the organization—they began using Docker containers. After looking into the various container orchestration options, they went with Kubernetes, even though it was still in beta at the time. "There was some debate whether we should build something ourselves, or just leverage that product and evolve with it," says Northwestern Mutual Cloud Native Engineer Frank Greco Jr. "Kubernetes has definitely been the right choice for us. It gave us that base framework so teams can be autonomous in what they're building and deliver very quickly and frequently."

+ +

As early adopters, the team had to do a lot of work with Ansible scripts to stand up the cluster. "We had a lot of hard security requirements given the nature of our business," explains Bryan Pfremmer, App Platform Teams Manager, Northwestern Mutual. "We found ourselves running a configuration that very few other people ever tried." The client experience group was the first to use the new platform; today, a few hundred of the company's 1,500 engineers are using it and more are eager to get on board.

+ +

The results have been dramatic. Before, infrastructure deployments could take two weeks; now, it is done in a matter of minutes. Now with a focus on Infrastructure automation, and self-service, "You can take an app to production in that same day if you want to," says Pfremmer.

+ +{{< case-studies/quote image="/images/case-studies/northwestern/banner4.jpg" >}} "Now, developers have autonomy, they can use this whenever they want, however they want. It becomes more valuable the more instrumentation downstream that happens, as we mature in it." -
-
+{{< /case-studies/quote >}} -
-
- The process used to be so cumbersome that minor bug releases would be bundled with feature releases. With the new streamlined system enabled by Kubernetes, the number of deployments has increased from about 24 a year to more than 500 in just the first 10 months of 2017. Availability has also been improved: There used to be a six-hour control window for commits every early Sunday morning, as well as other periods of general maintenance, during which outages could happen. "Now there’s no planned outage window," notes Pfremmer.

-Northwestern Mutual built that API management tool—called Kanali—and open sourced it in the summer of 2017. The team took on the project because it was a key capability for what they were building and prior the solution worked in an "anti-cloud native way that was different than everything else we were doing," says Greco. Now API management is just another container deployed to Kubernetes along with a separate Jaeger deployment.

-Now the engineers using the Kubernetes deployment platform have the added benefit of visibility in production—and autonomy. Before, a centralized team and would have to run a trace. "Now, developers have autonomy, they can use this whenever they want, however they want. It becomes more valuable the more instrumentation downstream that happens, as we mature in it." says Greco. +

The process used to be so cumbersome that minor bug releases would be bundled with feature releases. With the new streamlined system enabled by Kubernetes, the number of deployments has increased from about 24 a year to more than 500 in just the first 10 months of 2017. Availability has also been improved: There used to be a six-hour control window for commits every early Sunday morning, as well as other periods of general maintenance, during which outages could happen. "Now there's no planned outage window," notes Pfremmer.

+

Northwestern Mutual built that API management tool—called Kanali—and open sourced it in the summer of 2017. The team took on the project because it was a key capability for what they were building and prior the solution worked in an "anti-cloud native way that was different than everything else we were doing," says Greco. Now API management is just another container deployed to Kubernetes along with a separate Jaeger deployment.

-
+

Now the engineers using the Kubernetes deployment platform have the added benefit of visibility in production—and autonomy. Before, a centralized team and would have to run a trace. "Now, developers have autonomy, they can use this whenever they want, however they want. It becomes more valuable the more instrumentation downstream that happens, as we mature in it." says Greco.

-
-
- "We’re trying to make what we’re doing known so that we can find people who are like, 'Yeah, that’s interesting. I want to come do it!’" -
-
+{{< case-studies/quote >}} +"We're trying to make what we're doing known so that we can find people who are like, 'Yeah, that's interesting. I want to come do it!'" +{{< /case-studies/quote >}} -
- But the team didn’t stop there. "In a large enterprise, you’re going to have people using Kubernetes, but then you’re also going to have people using WAS and .NET," says Greco. "You may not be at a point where your whole stack can be cloud native. What if you can take your API management tool and make it cloud native, but still proxy to legacy systems? Using different pieces that are cloud native, open source and Kubernetes native, you can do pretty innovative stuff."

- As the team continues to improve its stack and share its Kubernetes best practices, it feels that Northwestern Mutual’s reputation as a technology-first company is evolving too. "No one would think a company that’s 160-plus years old is foraying this deep into the cloud and infrastructure stack," says Pfremmer. And they’re hoping that means they’ll be able to attract new talent. "We’re trying to make what we’re doing known so that we can find people who are like, 'Yeah, that’s interesting. I want to come do it!’" +

But the team didn't stop there. "In a large enterprise, you're going to have people using Kubernetes, but then you're also going to have people using WAS and .NET," says Greco. "You may not be at a point where your whole stack can be cloud native. What if you can take your API management tool and make it cloud native, but still proxy to legacy systems? Using different pieces that are cloud native, open source and Kubernetes native, you can do pretty innovative stuff."

- -
- -
+

As the team continues to improve its stack and share its Kubernetes best practices, it feels that Northwestern Mutual's reputation as a technology-first company is evolving too. "No one would think a company that's 160-plus years old is foraying this deep into the cloud and infrastructure stack," says Pfremmer. And they're hoping that means they'll be able to attract new talent. "We're trying to make what we're doing known so that we can find people who are like, 'Yeah, that's interesting. I want to come do it!'"

diff --git a/content/en/case-studies/ocado/index.html b/content/en/case-studies/ocado/index.html index 79ac9bf3a8..4c4018009a 100644 --- a/content/en/case-studies/ocado/index.html +++ b/content/en/case-studies/ocado/index.html @@ -1,99 +1,83 @@ --- title: Ocado Case Study - linkTitle: Ocado case_study_styles: true cid: caseStudies -css: /css/style_case_studies.css logo: ocado_featured_logo.png featured: true weight: 4 quote: > - People at Ocado Technology have been quite amazed. They ask, ‘Can we do this on a Dev cluster?’ and 10 minutes later we have rolled out something that is deployed across the cluster. The speed from idea to implementation to deployment is amazing. + People at Ocado Technology have been quite amazed. They ask, 'Can we do this on a Dev cluster?' and 10 minutes later we have rolled out something that is deployed across the cluster. The speed from idea to implementation to deployment is amazing. + +new_case_study_styles: true +heading_background: /images/case-studies/ocado/banner1.jpg +heading_title_logo: /images/ocado_logo.png +subheading: > + Ocado: Running Grocery Warehouses with a Cloud Native Platform +case_study_details: + - Company: Ocado Technology + - Location: Hatfield, England + - Industry: Grocery retail technology and platforms --- -
-

CASE STUDY:
Ocado: Running Grocery Warehouses with a Cloud Native Platform

-
+

Challenge

-
- Company  Ocado Technology     Location  Hatfield, England     Industry  Grocery retail technology and platforms -
+

The world's largest online-only grocery retailer, Ocado developed the Ocado Smart Platform to manage its own operations, from websites to warehouses, and is now licensing the technology to other retailers such as Kroger. To set up the first warehouses for the platform, Ocado shifted from virtual machines and Puppet infrastructure to Docker containers, using CoreOS's fleet scheduler to provision all the services on its OpenStack-based private cloud on bare metal. As the Smart Platform grew and "fleet was going end-of-life," says Platform Engineer Mike Bryant, "we started looking for a more complete platform, with all of these disparate infrastructure services being brought together in one unified API."

-
-
-
-
-

Challenge

- The world’s largest online-only grocery retailer, Ocado developed the Ocado Smart Platform to manage its own operations, from websites to warehouses, and is now licensing the technology to other retailers such as Kroger. To set up the first warehouses for the platform, Ocado shifted from virtual machines and Puppet infrastructure to Docker containers, using CoreOS’s fleet scheduler to provision all the services on its OpenStack-based private cloud on bare metal. As the Smart Platform grew and "fleet was going end-of-life," says Platform Engineer Mike Bryant, "we started looking for a more complete platform, with all of these disparate infrastructure services being brought together in one unified API."

-

Solution

- The team decided to migrate from fleet to Kubernetes on Ocado’s private cloud. The Kubernetes stack currently uses kubeadm for bootstrapping, CNI with Weave Net for networking, Prometheus Operator for monitoring, Fluentd for logging, and OpenTracing for distributed tracing. The first app on Kubernetes, a business-critical service in the warehouses, went into production in the summer of 2017, with a mass migration continuing into 2018. Hundreds of Ocado engineers working on the Smart Platform are now deploying on Kubernetes. - -
- -
+

Solution

+

The team decided to migrate from fleet to Kubernetes on Ocado's private cloud. The Kubernetes stack currently uses kubeadm for bootstrapping, CNI with Weave Net for networking, Prometheus Operator for monitoring, Fluentd for logging, and OpenTracing for distributed tracing. The first app on Kubernetes, a business-critical service in the warehouses, went into production in the summer of 2017, with a mass migration continuing into 2018. Hundreds of Ocado engineers working on the Smart Platform are now deploying on Kubernetes.

Impact

- With Kubernetes, "the speed from idea to implementation to deployment is amazing," says Bryant. "I’ve seen features go from development to production inside of a week now. In the old world, a new application deployment could easily take over a month." And because there are no longer restrictive deployment windows in the warehouses, the rate of deployments has gone from as few as two per week to dozens per week. Ocado has also achieved cost savings because Kubernetes gives the team the ability to have more fine-grained resource allocation. Says DevOps Team Leader Kevin McCormack: "We have more confidence in the resource allocation/separation features of Kubernetes, so we have been able to migrate from around 10 fleet clusters to one Kubernetes cluster." The team also uses Prometheus and Grafana to visualize resource allocation, and makes the data available to developers. "The increased visibility offered by Prometheus means developers are more aware of what they are using and how their use impacts others, especially since we now have one shared cluster," says McCormack. "I’d estimate that we use about 15-25% less hardware resources to host the same applications in Kubernetes in our test environments." -
+

With Kubernetes, "the speed from idea to implementation to deployment is amazing," says Bryant. "I've seen features go from development to production inside of a week now. In the old world, a new application deployment could easily take over a month." And because there are no longer restrictive deployment windows in the warehouses, the rate of deployments has gone from as few as two per week to dozens per week. Ocado has also achieved cost savings because Kubernetes gives the team the ability to have more fine-grained resource allocation. Says DevOps Team Leader Kevin McCormack: "We have more confidence in the resource allocation/separation features of Kubernetes, so we have been able to migrate from around 10 fleet clusters to one Kubernetes cluster." The team also uses Prometheus and Grafana to visualize resource allocation, and makes the data available to developers. "The increased visibility offered by Prometheus means developers are more aware of what they are using and how their use impacts others, especially since we now have one shared cluster," says McCormack. "I'd estimate that we use about 15-25% less hardware resources to host the same applications in Kubernetes in our test environments."

-
-
-
-
- "People at Ocado Technology have been quite amazed. They ask, ‘Can we do this on a Dev cluster?’ and 10 minutes later we have rolled out something that is deployed across the cluster. The speed from idea to implementation to deployment is amazing."

- Mike Bryant, Platform Engineer, Ocado -
-
-
-
-

When it was founded in 2000, Ocado was an online-only grocery retailer in the U.K. In the years since, it has expanded from delivering produce to families to providing technology to other grocery retailers.

-The company began developing its Ocado Smart Platform to manage its own operations, from websites to warehouses, and is now licensing the technology to other grocery chains around the world, such as Kroger. To set up the first warehouses on the platform, Ocado shifted from virtual machines and Puppet infrastructure to Docker containers, using CoreOS’s fleet scheduler to provision all the services on its OpenStack-based private cloud on bare metal. As the Smart Platform grew, and "fleet was going end-of-life," says Platform Engineer Mike Bryant, "we started looking for a more complete platform, with all of these disparate infrastructure services being brought together in one unified API."

-Bryant had already been using Kubernetes with Code for Life, a children’s education project that’s part of Ocado’s charity arm. "We really liked it, so we started looking at it seriously for our production workloads," says Bryant. The team that managed fleet had researched orchestration solutions and landed on Kubernetes as well. "We were looking for a platform with wide adoption, and that was where the momentum was," says DevOps Team Leader Kevin McCormack. The two paths converged, and "We didn’t even go through any proof-of-concept stage. The Code for Life work served that purpose," says Bryant. +{{< case-studies/quote author="Mike Bryant, Platform Engineer, Ocado" >}} +"People at Ocado Technology have been quite amazed. They ask, 'Can we do this on a Dev cluster?' and 10 minutes later we have rolled out something that is deployed across the cluster. The speed from idea to implementation to deployment is amazing." +{{< /case-studies/quote >}} -
-
-
-
- "We were looking for a platform with wide adoption, and that was where the momentum was, the two paths converged, and we didn’t even go through any proof-of-concept stage. The Code for Life work served that purpose,"

- Kevin McCormack, DevOps Team Leader, Ocado -
-
-
-
- In the summer of 2016, the team began migrating from fleet to Kubernetes on Ocado’s private cloud. The Kubernetes stack currently uses kubeadm for bootstrapping, CNI with Weave Net for networking, Prometheus Operator for monitoring, Fluentd for logging, and OpenTracing for distributed tracing.

- The first app on Kubernetes, a business-critical service in the warehouses, went into production a year later. Once that app was running smoothly, a mass migration continued into 2018. Hundreds of Ocado engineers working on the Smart Platform are now deploying on Kubernetes, and the platform is live in Ocado’s warehouses, managing tens of thousands of orders a week. At full capacity, Ocado’s latest warehouse in Erith, southeast London, will deliver more than 200,000 orders per week, making it the world’s largest facility for online grocery.

- There are about 150 microservices now running on Kubernetes, with multiple instances of many of them. "We’re not just deploying all these microservices at once. We’re deploying them all for one warehouse, and then they’re all being deployed again for the next warehouse, and again and again," says Bryant.

- The move to Kubernetes was eye-opening for many people at Ocado Technology. "In the early days of putting the platform into our test infrastructure, the technical architect asked what network performance was like on Weave Net with encryption turned on," recalls Bryant. "So we found a Docker container for iPerf, wrote a daemon set, deployed it. A few moments later, we’ve deployed the entire thing across this cluster. He was pretty blown away by that." +{{< case-studies/lead >}} +When it was founded in 2000, Ocado was an online-only grocery retailer in the U.K. In the years since, it has expanded from delivering produce to families to providing technology to other grocery retailers. +{{< /case-studies/lead >}} -
-
-
-
- "The unified API of Kubernetes means this is all in one place, and it’s one flow for approval and rollout. I’ve seen features go from development to production inside of a week now. In the old world, a new application deployment could easily take over a month."

- Mike Bryant, Platform Engineer, Ocado -
-
- +

The company began developing its Ocado Smart Platform to manage its own operations, from websites to warehouses, and is now licensing the technology to other grocery chains around the world, such as Kroger. To set up the first warehouses on the platform, Ocado shifted from virtual machines and Puppet infrastructure to Docker containers, using CoreOS's fleet scheduler to provision all the services on its OpenStack-based private cloud on bare metal. As the Smart Platform grew, and "fleet was going end-of-life," says Platform Engineer Mike Bryant, "we started looking for a more complete platform, with all of these disparate infrastructure services being brought together in one unified API."

-
-
- Indeed, the impact has been profound. "Prior to containerization, we had quite restrictive deployment windows in our warehouses," says Bryant. "Moving to microservices, we’ve been able to deploy much more frequently. We’ve been able to move towards continuous delivery in a number of areas. In our older warehouse, new application deployments involve talking to a bunch of different teams for different levels of the stack: from VM provisioning, to storage, to load balancers, and so on. The unified API of Kubernetes means this is all in one place, and it’s one flow for approval and rollout. I’ve seen features go from development to production inside of a week now. In the old world, a new application deployment could easily take over a month."

- The rate of deployment has gone from as few as two per week to dozens per week. "With Kubernetes, some of our development teams have been able to deploy their application to production on the new platform without us noticing," says Bryant, "which means they’re faster at doing what they need to do and we have less work."

- Ocado has also achieved cost savings because Kubernetes gives the team the ability to have more fine-grained resource allocation. "That lets us shrink quite a lot of our deployments from being per-core VM deployments to having fractions of the core," says Bryant. Adds McCormack: "We have more confidence in the resource allocation/separation features of Kubernetes, so we have been able to migrate from around 10 fleet clusters to one Kubernetes cluster. This means we use our hardware better since if we have to always have two nodes of excess capacity available in case of node failures then we only need two extra instead of 20." +

Bryant had already been using Kubernetes with Code for Life, a children's education project that's part of Ocado's charity arm. "We really liked it, so we started looking at it seriously for our production workloads," says Bryant. The team that managed fleet had researched orchestration solutions and landed on Kubernetes as well. "We were looking for a platform with wide adoption, and that was where the momentum was," says DevOps Team Leader Kevin McCormack. The two paths converged, and "We didn't even go through any proof-of-concept stage. The Code for Life work served that purpose," says Bryant.

-
+{{< case-studies/quote + image="/images/case-studies/ocado/banner3.jpg" + author="Kevin McCormack, DevOps Team Leader, Ocado" +>}} +"We were looking for a platform with wide adoption, and that was where the momentum was, the two paths converged, and we didn't even go through any proof-of-concept stage. The Code for Life work served that purpose," +{{< /case-studies/quote >}} -
-
- "CNCF have provided us with support of different technologies. We’ve been able to adopt those in a very easy fashion. We do like that CNCF is vendor agnostic. We’re not being asked to commit to this one way of doing things. The vast diversity of viewpoints in CNCF lead to better technology."

- Mike Bryant, Platform Engineer, Ocado +

In the summer of 2016, the team began migrating from fleet to Kubernetes on Ocado's private cloud. The Kubernetes stack currently uses kubeadm for bootstrapping, CNI with Weave Net for networking, Prometheus Operator for monitoring, Fluentd for logging, and OpenTracing for distributed tracing.

-
-
+

The first app on Kubernetes, a business-critical service in the warehouses, went into production a year later. Once that app was running smoothly, a mass migration continued into 2018. Hundreds of Ocado engineers working on the Smart Platform are now deploying on Kubernetes, and the platform is live in Ocado's warehouses, managing tens of thousands of orders a week. At full capacity, Ocado's latest warehouse in Erith, southeast London, will deliver more than 200,000 orders per week, making it the world's largest facility for online grocery.

-
- The team also uses Prometheus and Grafana to visualize resource allocation, and makes the data available to developers. "The increased visibility offered by Prometheus means developers are more aware of what they are using and how their use impacts others, especially since we now have one shared cluster," says McCormack. "I’d estimate that we use about 15-25% less hardware resource to host the same applications in Kubernetes in our test environments."

- One of the broader benefits of cloud native, says Bryant, is the unified API. "We have one method of doing our deployments that covers the wide range of things we need to do, and we can extend the API," he says. In addition to using Prometheus Operator, the Ocado team has started writing its own operators, some of which have been open sourced. Plus, "CNCF has provided us with support of these different technologies. We’ve been able to adopt those in a very easy fashion. We do like that CNCF is vendor agnostic. We’re not being asked to commit to this one way of doing things. The vast diversity of viewpoints in the CNCF leads to better technology."

- Ocado’s own technology, in the form of its Smart Platform, will soon be used around the world. And cloud native plays a crucial role in this global expansion. "I wouldn’t have wanted to try it without Kubernetes," says Bryant. "Kubernetes has made it so much nicer, especially to have that consistent way of deploying all of the applications, then taking the same thing and being able to replicate it. It’s very valuable." +

There are about 150 microservices now running on Kubernetes, with multiple instances of many of them. "We're not just deploying all these microservices at once. We're deploying them all for one warehouse, and then they're all being deployed again for the next warehouse, and again and again," says Bryant.

-
-
+

The move to Kubernetes was eye-opening for many people at Ocado Technology. "In the early days of putting the platform into our test infrastructure, the technical architect asked what network performance was like on Weave Net with encryption turned on," recalls Bryant. "So we found a Docker container for iPerf, wrote a daemon set, deployed it. A few moments later, we've deployed the entire thing across this cluster. He was pretty blown away by that."

+ +{{< case-studies/quote + image="/images/case-studies/ocado/banner4.jpg" + author="Mike Bryant, Platform Engineer, Ocado" +>}} +"The unified API of Kubernetes means this is all in one place, and it's one flow for approval and rollout. I've seen features go from development to production inside of a week now. In the old world, a new application deployment could easily take over a month." +{{< /case-studies/quote >}} + +

Indeed, the impact has been profound. "Prior to containerization, we had quite restrictive deployment windows in our warehouses," says Bryant. "Moving to microservices, we've been able to deploy much more frequently. We've been able to move towards continuous delivery in a number of areas. In our older warehouse, new application deployments involve talking to a bunch of different teams for different levels of the stack: from VM provisioning, to storage, to load balancers, and so on. The unified API of Kubernetes means this is all in one place, and it's one flow for approval and rollout. I've seen features go from development to production inside of a week now. In the old world, a new application deployment could easily take over a month."

+ +

The rate of deployment has gone from as few as two per week to dozens per week. "With Kubernetes, some of our development teams have been able to deploy their application to production on the new platform without us noticing," says Bryant, "which means they're faster at doing what they need to do and we have less work."

+ +

Ocado has also achieved cost savings because Kubernetes gives the team the ability to have more fine-grained resource allocation. "That lets us shrink quite a lot of our deployments from being per-core VM deployments to having fractions of the core," says Bryant. Adds McCormack: "We have more confidence in the resource allocation/separation features of Kubernetes, so we have been able to migrate from around 10 fleet clusters to one Kubernetes cluster. This means we use our hardware better since if we have to always have two nodes of excess capacity available in case of node failures then we only need two extra instead of 20."

+ +{{< case-studies/quote author="Mike Bryant, Platform Engineer, Ocado" >}} +"CNCF have provided us with support of different technologies. We've been able to adopt those in a very easy fashion. We do like that CNCF is vendor agnostic. We're not being asked to commit to this one way of doing things. The vast diversity of viewpoints in CNCF lead to better technology." +{{< /case-studies/quote >}} + +

The team also uses Prometheus and Grafana to visualize resource allocation, and makes the data available to developers. "The increased visibility offered by Prometheus means developers are more aware of what they are using and how their use impacts others, especially since we now have one shared cluster," says McCormack. "I'd estimate that we use about 15-25% less hardware resource to host the same applications in Kubernetes in our test environments."

+ +

One of the broader benefits of cloud native, says Bryant, is the unified API. "We have one method of doing our deployments that covers the wide range of things we need to do, and we can extend the API," he says. In addition to using Prometheus Operator, the Ocado team has started writing its own operators, some of which have been open sourced. Plus, "CNCF has provided us with support of these different technologies. We've been able to adopt those in a very easy fashion. We do like that CNCF is vendor agnostic. We're not being asked to commit to this one way of doing things. The vast diversity of viewpoints in the CNCF leads to better technology."

+ +

Ocado's own technology, in the form of its Smart Platform, will soon be used around the world. And cloud native plays a crucial role in this global expansion. "I wouldn't have wanted to try it without Kubernetes," says Bryant. "Kubernetes has made it so much nicer, especially to have that consistent way of deploying all of the applications, then taking the same thing and being able to replicate it. It's very valuable."

diff --git a/content/en/case-studies/openAI/index.html b/content/en/case-studies/openAI/index.html index 1b95ec5f35..543c1dee64 100644 --- a/content/en/case-studies/openAI/index.html +++ b/content/en/case-studies/openAI/index.html @@ -2,98 +2,68 @@ title: OpenAI Case Study case_study_styles: true cid: caseStudies -css: /css/style_case_studies.css + +new_case_study_styles: true +heading_background: /images/case-studies/openAI/banner1.jpg +heading_title_logo: /images/openAI_logo.png +subheading: > + Launching and Scaling Up Experiments, Made Simple +case_study_details: + - Company: OpenAI + - Location: San Francisco, California + - Industry: Artificial Intelligence Research --- -
-

CASE STUDY:
Launching and Scaling Up Experiments, Made Simple +

Challenge

-

+

An artificial intelligence research lab, OpenAI needed infrastructure for deep learning that would allow experiments to be run either in the cloud or in its own data center, and to easily scale. Portability, speed, and cost were the main drivers.

-
+

Solution

-
- Company  OpenAI     Location  San Francisco, California     Industry  Artificial Intelligence Research -
- -
-
-
-
-

Challenge

- An artificial intelligence research lab, OpenAI needed infrastructure for deep learning that would allow experiments to be run either in the cloud or in its own data center, and to easily scale. Portability, speed, and cost were the main drivers. -
-

Solution

- OpenAI began running Kubernetes on top of AWS in 2016, and in early 2017 migrated to Azure. OpenAI runs key experiments in fields including robotics and gaming both in Azure and in its own data centers, depending on which cluster has free capacity. "We use Kubernetes mainly as a batch scheduling system and rely on our autoscaler to dynamically scale up and down our cluster," says Christopher Berner, Head of Infrastructure. "This lets us significantly reduce costs for idle nodes, while still providing low latency and rapid iteration." -
- -
+

OpenAI began running Kubernetes on top of AWS in 2016, and in early 2017 migrated to Azure. OpenAI runs key experiments in fields including robotics and gaming both in Azure and in its own data centers, depending on which cluster has free capacity. "We use Kubernetes mainly as a batch scheduling system and rely on our autoscaler to dynamically scale up and down our cluster," says Christopher Berner, Head of Infrastructure. "This lets us significantly reduce costs for idle nodes, while still providing low latency and rapid iteration."

Impact

- The company has benefited from greater portability: "Because Kubernetes provides a consistent API, we can move our research experiments very easily between clusters," says Berner. Being able to use its own data centers when appropriate is "lowering costs and providing us access to hardware that we wouldn’t necessarily have access to in the cloud," he adds. "As long as the utilization is high, the costs are much lower there." Launching experiments also takes far less time: "One of our researchers who is working on a new distributed training system has been able to get his experiment running in two or three days. In a week or two he scaled it out to hundreds of GPUs. Previously, that would have easily been a couple of months of work." +

The company has benefited from greater portability: "Because Kubernetes provides a consistent API, we can move our research experiments very easily between clusters," says Berner. Being able to use its own data centers when appropriate is "lowering costs and providing us access to hardware that we wouldn't necessarily have access to in the cloud," he adds. "As long as the utilization is high, the costs are much lower there." Launching experiments also takes far less time: "One of our researchers who is working on a new distributed training system has been able to get his experiment running in two or three days. In a week or two he scaled it out to hundreds of GPUs. Previously, that would have easily been a couple of months of work."

-
+{{< case-studies/quote >}} + +
+Check out "Building the Infrastructure that Powers the Future of AI" presented by Vicki Cheung, Member of Technical Staff & Jonas Schneider, Member of Technical Staff at OpenAI from KubeCon/CloudNativeCon Europe 2017. +{{< /case-studies/quote >}} -
-
-
-
+{{< case-studies/lead >}} +From experiments in robotics to old-school video game play research, OpenAI's work in artificial intelligence technology is meant to be shared. +{{< /case-studies/lead >}} -
-
-Check out "Building the Infrastructure that Powers the Future of AI" presented by Vicki Cheung, Member of Technical Staff & Jonas Schneider, Member of Technical Staff at OpenAI from KubeCon/CloudNativeCon Europe 2017. -
-
-
-
-
-

From experiments in robotics to old-school video game play research, OpenAI’s work in artificial intelligence technology is meant to be shared.

- With a mission to ensure powerful AI systems are safe, OpenAI cares deeply about open source—both benefiting from it and contributing safety technology into it. "The research that we do, we want to spread it as widely as possible so everyone can benefit," says OpenAI’s Head of Infrastructure Christopher Berner. The lab’s philosophy—as well as its particular needs—lent itself to embracing an open source, cloud native strategy for its deep learning infrastructure.

- OpenAI started running Kubernetes on top of AWS in 2016, and a year later, migrated the Kubernetes clusters to Azure. "We probably use Kubernetes differently from a lot of people," says Berner. "We use it for batch scheduling and as a workload manager for the cluster. It’s a way of coordinating a large number of containers that are all connected together. We rely on our autoscaler to dynamically scale up and down our cluster. This lets us significantly reduce costs for idle nodes, while still providing low latency and rapid iteration."

- In the past year, Berner has overseen the launch of several Kubernetes clusters in OpenAI’s own data centers. "We run them in a hybrid model where the control planes—the Kubernetes API servers, etcd and everything—are all in Azure, and then all of the Kubernetes nodes are in our own data center," says Berner. "The cloud is really convenient for managing etcd and all of the masters, and having backups and spinning up new nodes if anything breaks. This model allows us to take advantage of lower costs and have the availability of more specialized hardware in our own data center." +

With a mission to ensure powerful AI systems are safe, OpenAI cares deeply about open source—both benefiting from it and contributing safety technology into it. "The research that we do, we want to spread it as widely as possible so everyone can benefit," says OpenAI's Head of Infrastructure Christopher Berner. The lab's philosophy—as well as its particular needs—lent itself to embracing an open source, cloud native strategy for its deep learning infrastructure.

+

OpenAI started running Kubernetes on top of AWS in 2016, and a year later, migrated the Kubernetes clusters to Azure. "We probably use Kubernetes differently from a lot of people," says Berner. "We use it for batch scheduling and as a workload manager for the cluster. It's a way of coordinating a large number of containers that are all connected together. We rely on our autoscaler to dynamically scale up and down our cluster. This lets us significantly reduce costs for idle nodes, while still providing low latency and rapid iteration."

-
-
-
-
- OpenAI’s experiments take advantage of Kubernetes’ benefits, including portability. "Because Kubernetes provides a consistent API, we can move our research experiments very easily between clusters..." +

In the past year, Berner has overseen the launch of several Kubernetes clusters in OpenAI's own data centers. "We run them in a hybrid model where the control planes—the Kubernetes API servers, etcd and everything—are all in Azure, and then all of the Kubernetes nodes are in our own data center," says Berner. "The cloud is really convenient for managing etcd and all of the masters, and having backups and spinning up new nodes if anything breaks. This model allows us to take advantage of lower costs and have the availability of more specialized hardware in our own data center."

-
-
-
-
- Different teams at OpenAI currently run a couple dozen projects. While the largest-scale workloads manage bare cloud VMs directly, most of OpenAI’s experiments take advantage of Kubernetes’ benefits, including portability. "Because Kubernetes provides a consistent API, we can move our research experiments very easily between clusters," says Berner. The on-prem clusters are generally "used for workloads where you need lots of GPUs, something like training an ImageNet model. Anything that’s CPU heavy, that’s run in the cloud. But we also have a number of teams that run their experiments both in Azure and in our own data centers, just depending on which cluster has free capacity, and that’s hugely valuable."

- Berner has made the Kubernetes clusters available to all OpenAI teams to use if it’s a good fit. "I’ve worked a lot with our games team, which at the moment is doing research on classic console games," he says. "They had been running a bunch of their experiments on our dev servers, and they had been trying out Google cloud, managing their own VMs. We got them to try out our first on-prem Kubernetes cluster, and that was really successful. They’ve now moved over completely to it, and it has allowed them to scale up their experiments by 10x, and do that without needing to invest significant engineering time to figure out how to manage more machines. A lot of people are now following the same path." +{{< case-studies/quote image="/images/case-studies/openAI/banner3.jpg" >}} +OpenAI's experiments take advantage of Kubernetes' benefits, including portability. "Because Kubernetes provides a consistent API, we can move our research experiments very easily between clusters..." +{{< /case-studies/quote >}} -
-
-
-
+

Different teams at OpenAI currently run a couple dozen projects. While the largest-scale workloads manage bare cloud VMs directly, most of OpenAI's experiments take advantage of Kubernetes' benefits, including portability. "Because Kubernetes provides a consistent API, we can move our research experiments very easily between clusters," says Berner. The on-prem clusters are generally "used for workloads where you need lots of GPUs, something like training an ImageNet model. Anything that's CPU heavy, that's run in the cloud. But we also have a number of teams that run their experiments both in Azure and in our own data centers, just depending on which cluster has free capacity, and that's hugely valuable."

+ +

Berner has made the Kubernetes clusters available to all OpenAI teams to use if it's a good fit. "I've worked a lot with our games team, which at the moment is doing research on classic console games," he says. "They had been running a bunch of their experiments on our dev servers, and they had been trying out Google cloud, managing their own VMs. We got them to try out our first on-prem Kubernetes cluster, and that was really successful. They've now moved over completely to it, and it has allowed them to scale up their experiments by 10x, and do that without needing to invest significant engineering time to figure out how to manage more machines. A lot of people are now following the same path."

+ +{{< case-studies/quote image="/images/case-studies/openAI/banner4.jpg" >}} "One of our researchers who is working on a new distributed training system has been able to get his experiment running in two or three days," says Berner. "In a week or two he scaled it out to hundreds of GPUs. Previously, that would have easily been a couple of months of work." -
-
+{{< /case-studies/quote >}} -
-
- That path has been simplified by frameworks and tools that two of OpenAI’s teams have developed to handle interaction with Kubernetes. "You can just write some Python code, fill out a bit of configuration with exactly how many machines you need and which types, and then it will prepare all of those specifications and send it to the Kube cluster so that it gets launched there," says Berner. "And it also provides a bit of extra monitoring and better tooling that’s designed specifically for these machine learning projects."

- The impact that Kubernetes has had at OpenAI is impressive. With Kubernetes, the frameworks and tooling, including the autoscaler, in place, launching experiments takes far less time. "One of our researchers who is working on a new distributed training system has been able to get his experiment running in two or three days," says Berner. "In a week or two he scaled it out to hundreds of GPUs. Previously, that would have easily been a couple of months of work."

- Plus, the flexibility they now have to use their on-prem Kubernetes cluster when appropriate is "lowering costs and providing us access to hardware that we wouldn’t necessarily have access to in the cloud," he says. "As long as the utilization is high, the costs are much lower in our data center. To an extent, you can also customize your hardware to exactly what you need." +

That path has been simplified by frameworks and tools that two of OpenAI's teams have developed to handle interaction with Kubernetes. "You can just write some Python code, fill out a bit of configuration with exactly how many machines you need and which types, and then it will prepare all of those specifications and send it to the Kube cluster so that it gets launched there," says Berner. "And it also provides a bit of extra monitoring and better tooling that's designed specifically for these machine learning projects."

+

The impact that Kubernetes has had at OpenAI is impressive. With Kubernetes, the frameworks and tooling, including the autoscaler, in place, launching experiments takes far less time. "One of our researchers who is working on a new distributed training system has been able to get his experiment running in two or three days," says Berner. "In a week or two he scaled it out to hundreds of GPUs. Previously, that would have easily been a couple of months of work."

-
+

Plus, the flexibility they now have to use their on-prem Kubernetes cluster when appropriate is "lowering costs and providing us access to hardware that we wouldn't necessarily have access to in the cloud," he says. "As long as the utilization is high, the costs are much lower in our data center. To an extent, you can also customize your hardware to exactly what you need."

-
-
- "Research teams can now take advantage of the frameworks we’ve built on top of Kubernetes, which make it easy to launch experiments, scale them by 10x or 50x, and take little effort to manage."
— CHRISTOPHER BERNER, HEAD OF INFRASTRUCTURE FOR OPENAI -
-
+{{< case-studies/quote author="CHRISTOPHER BERNER, HEAD OF INFRASTRUCTURE FOR OPENAI" >}} +"Research teams can now take advantage of the frameworks we've built on top of Kubernetes, which make it easy to launch experiments, scale them by 10x or 50x, and take little effort to manage." +{{< /case-studies/quote >}} -
+

OpenAI is also benefiting from other technologies in the CNCF cloud-native ecosystem. gRPC is used by many of its systems for communications between different services, and Prometheus is in place "as a debugging tool if things go wrong," says Berner. "We actually haven't had any real problems in our Kubernetes clusters recently, so I don't think anyone has looked at our Prometheus monitoring in a while. If something breaks, it will be there."

- OpenAI is also benefiting from other technologies in the CNCF cloud-native ecosystem. gRPC is used by many of its systems for communications between different services, and Prometheus is in place "as a debugging tool if things go wrong," says Berner. "We actually haven’t had any real problems in our Kubernetes clusters recently, so I don’t think anyone has looked at our Prometheus monitoring in a while. If something breaks, it will be there."

- One of the things Berner continues to focus on is Kubernetes’ ability to scale, which is essential to deep learning experiments. OpenAI has been able to push one of its Kubernetes clusters on Azure up to more than 2,500 nodes. "I think we’ll probably hit the 5,000-machine number that Kubernetes has been tested at before too long," says Berner, adding, "We’re definitely hiring if you’re excited about working on these things!" -
- -
+

One of the things Berner continues to focus on is Kubernetes' ability to scale, which is essential to deep learning experiments. OpenAI has been able to push one of its Kubernetes clusters on Azure up to more than 2,500 nodes. "I think we'll probably hit the 5,000-machine number that Kubernetes has been tested at before too long," says Berner, adding, "We're definitely hiring if you're excited about working on these things!"

diff --git a/content/en/case-studies/peardeck/index.html b/content/en/case-studies/peardeck/index.html index 688754a620..4398277677 100644 --- a/content/en/case-studies/peardeck/index.html +++ b/content/en/case-studies/peardeck/index.html @@ -1,111 +1,87 @@ --- title: Pear Deck Case Study - case_study_styles: true cid: caseStudies -css: /css/style_peardeck.css + +new_case_study_styles: true +heading_background: /images/case-studies/peardeck/banner3.jpg +heading_title_logo: /images/peardeck_logo.png +subheading: > + Infrastructure for a Growing EdTech Startup +case_study_details: + - Company: Pear Deck + - Location: Iowa City, Iowa + - Industry: Educational Software --- -
-

CASE STUDY:
Infrastructure for a Growing EdTech Startup

+

Challenge

-
+

The three-year-old startup provides a web app for teachers to interact with their students in the classroom. The JavaScript app was built on Google's web app development platform Firebase, using Heroku. As the user base steadily grew, so did the development team. "We outgrew Heroku when we started wanting to have multiple services, and the deploying story got pretty horrendous. We were frustrated that we couldn't have the developers quickly stage a version," says CEO Riley Eynon-Lynch. "Tracing and monitoring became basically impossible." On top of that, many of Pear Deck's customers are behind government firewalls and connect through Firebase, not Pear Deck's servers, making troubleshooting even more difficult.

-
- Company  Pear Deck     Location  Iowa City, Iowa     Industry  Educational Software -
+

Solution

-
+

In 2016, the company began moving their code from Heroku to Docker containers running on Google Kubernetes Engine, orchestrated by Kubernetes and monitored with Prometheus.

-
-
-
-

Challenge

- The three-year-old startup provides a web app for teachers to interact with their students in the classroom. The JavaScript app was built on Google’s web app development platform Firebase, using Heroku. As the user base steadily grew, so did the development team. "We outgrew Heroku when we started wanting to have multiple services, and the deploying story got pretty horrendous. We were frustrated that we couldn’t have the developers quickly stage a version," says CEO Riley Eynon-Lynch. "Tracing and monitoring became basically impossible." On top of that, many of Pear Deck’s customers are behind government firewalls and connect through Firebase, not Pear Deck’s servers, making troubleshooting even more difficult. -
- -
-

Solution

- In 2016, the company began moving their code from Heroku to Docker containers running on Google Kubernetes Engine, orchestrated by Kubernetes and monitored with Prometheus. -
-

Impact

- The new cloud native stack immediately improved the development workflow, speeding up deployments. Prometheus gave Pear Deck "a lot of confidence, knowing that people are still logging into the app and using it all the time," says Eynon-Lynch. "The biggest impact is being able to work as a team on the configuration in git in a pull request, and the biggest confidence comes from the solidity of the abstractions and the trust that we have in Kubernetes actually making our yaml files a reality." -
-
-
-
-
- "We didn’t even realize how stressed out we were about our lack of insight into what was happening with the app. I’m really excited and have more and more confidence in the actual state of our application for our actual users, and not just what the CPU graphs are saying, because of Prometheus and Kubernetes."

– RILEY EYNON-LYNCH, CEO OF PEAR DECK -
-
+

The new cloud native stack immediately improved the development workflow, speeding up deployments. Prometheus gave Pear Deck "a lot of confidence, knowing that people are still logging into the app and using it all the time," says Eynon-Lynch. "The biggest impact is being able to work as a team on the configuration in git in a pull request, and the biggest confidence comes from the solidity of the abstractions and the trust that we have in Kubernetes actually making our yaml files a reality."

-
-
-

With the speed befitting a startup, Pear Deck delivered its first prototype to customers within three months of incorporating.

-As a former high school math teacher, CEO Riley Eynon-Lynch felt an urgency to provide a tech solution to classes where instructors struggle to interact with every student in a short amount of time. "Pear Deck is an app that students can use to interact with the teacher all at once," he says. "When the teacher asks a question, instead of just the kid at the front of the room answering again, everybody can answer every single question. It’s a huge fundamental shift in the messaging to the students about how much we care about them and how much they are a part of the classroom."

-Eynon-Lynch and his partners quickly built a JavaScript web app on Google’s web app development platform Firebase, and launched the minimum viable product [MVP] on Heroku "because it was fast and easy," he says. "We made everything as easy as we could." -

-But once it launched, the user base began growing steadily at a rate of 30 percent a month. "Our Heroku bill was getting totally insane," Eynon-Lynch says. But even more crucially, as the company hired more developers to keep pace, "we outgrew Heroku. We wanted to have multiple services and the deploying story got pretty horrendous. We were frustrated that we couldn’t have the developers quickly stage a version. Tracing and monitoring became basically impossible." -

-On top of that, many of Pear Deck’s customers are behind government firewalls and connect through Firebase, not Pear Deck’s servers, making troubleshooting even more difficult. -

-The team began looking around for another solution, and finally decided in early 2016 to start moving the app from Heroku to Docker containers running on Google Kubernetes Engine, orchestrated by Kubernetes and monitored with Prometheus. -
-
+{{< case-studies/quote author="RILEY EYNON-LYNCH, CEO OF PEAR DECK" >}} +"We didn't even realize how stressed out we were about our lack of insight into what was happening with the app. I'm really excited and have more and more confidence in the actual state of our application for our actual users, and not just what the CPU graphs are saying, because of Prometheus and Kubernetes." +{{< /case-studies/quote >}} -
-
- "When it became clear that Google Kubernetes Engine was going to have a lot of support from Google and be a fully-managed Kubernetes platform, it seemed very obvious to us that was the way to go," says Eynon-Lynch. -
-
+{{< case-studies/lead >}} +With the speed befitting a startup, Pear Deck delivered its first prototype to customers within three months of incorporating. +{{< /case-studies/lead >}} -
-
- They had considered other options like Google’s App Engine (which they were already using for one service) and Amazon’s Elastic Compute Cloud (EC2), while experimenting with running one small service that wasn’t accessible to the Internet in Kubernetes. "When it became clear that Google Kubernetes Engine was going to have a lot of support from Google and be a fully-managed Kubernetes platform, it seemed very obvious to us that was the way to go," says Eynon-Lynch. "We didn’t really consider Terraform and the other competitors because the abstractions offered by Kubernetes just jumped off the page to us."

- Once the team started porting its Heroku apps into Kubernetes, which was "super easy," he says, the impact was immediate. "Before, to make a new version of the app meant going to Heroku and reconfiguring 10 new services, so basically no one was willing to do it, and we never staged things," he says. "Now we can deploy our exact same configuration in lots of different clusters in 30 seconds. We have a full set up that’s always running, and then any of our developers or designers can stage new versions with one command, including their recent changes. We stage all the time now, and everyone stopped talking about how cool it is because it’s become invisible how great it is." -

- Along with Kubernetes came Prometheus. "Until pretty recently we didn’t have any kind of visibility into aggregate server metrics or performance," says Eynon-Lynch. The team had tried to use Google Kubernetes Engine’s Stackdriver monitoring, but had problems making it work, and considered New Relic. When they started looking at Prometheus in the fall of 2016, "the fit between the abstractions in Prometheus and the way we think about how our system works, was so clear and obvious," he says.

- The integration with Kubernetes made set-up easy. Once Helm installed Prometheus, "We started getting a graph of the health of all our Kubernetes nodes and pods immediately. I think we were pretty hooked at that point," Eynon-Lynch says. "Then we got our own custom instrumentation working in 15 minutes, and had an actively updated count of requests that we could do, rate on and get a sense of how many users are connected at a given point. And then it was another hour before we had alarms automatically showing up in our Slack channel. All that was in one afternoon. And it was an afternoon of gasping with delight, basically!" -
-
+

As a former high school math teacher, CEO Riley Eynon-Lynch felt an urgency to provide a tech solution to classes where instructors struggle to interact with every student in a short amount of time. "Pear Deck is an app that students can use to interact with the teacher all at once," he says. "When the teacher asks a question, instead of just the kid at the front of the room answering again, everybody can answer every single question. It's a huge fundamental shift in the messaging to the students about how much we care about them and how much they are a part of the classroom."

-
-
- "We started getting a graph of the health of all our Kubernetes nodes and pods immediately. I think we were pretty hooked at that point," Eynon-Lynch says. "Then we got our own custom instrumentation working in 15 minutes, and had an actively updated count of requests that we could do, rate on and get a sense of how many users are connected at a given point. And then it was another hour before we had alarms automatically showing up in our Slack channel. All that was in one afternoon. And it was an afternoon of gasping with delight, basically!" -
-
+

Eynon-Lynch and his partners quickly built a JavaScript web app on Google's web app development platform Firebase, and launched the minimum viable product [MVP] on Heroku "because it was fast and easy," he says. "We made everything as easy as we could."

-
-
- With Pear Deck’s specific challenges—traffic through Firebase as well as government firewalls—Prometheus was a game-changer. "We didn’t even realize how stressed out we were about our lack of insight into what was happening with the app," Eynon-Lynch says. Before, when a customer would report that the app wasn’t working, the team had to manually investigate the problem without knowing whether customers were affected all over the world, or whether Firebase was down, and where.

- To help solve that problem, the team wrote a script that pings Firebase from several different geographical locations, and then reports the responses to Prometheus in a histogram. "A huge impact that Prometheus had on us was just an amazing sigh of relief, of feeling like we knew what was happening," he says. "It took 45 minutes to implement [the Firebase alarm] because we knew that we had this trustworthy metrics platform in Prometheus. We weren’t going to have to figure out, ‘Where do we send these metrics? How do we aggregate the metrics? How do we understand them?’"

- Plus, Prometheus has allowed Pear Deck to build alarms for business goals. One measures the rate of successful app loads and goes off if the day’s loads are less than 90 percent of the loads from seven days before. "We run a JavaScript app behind ridiculous firewalls and all kinds of crazy browser extensions messing with it—Chrome will push a feature that breaks some CSS that we’re using," Eynon-Lynch says. "So that gives us a lot of confidence, and we at least know that people are still logging into the app and using it all the time."

- Now, when a customer complains, and none of the alarms have gone off, the team can feel confident that it’s not a widespread problem. "Just to be sure, we can go and double check the graphs and say, ‘Yep, there’s currently 10,000 people connected to that Firebase node. It’s definitely working. Let’s investigate your network settings, customer,’" he says. "And we can pass that back off to our support reps instead of the whole development team freaking out that Firebase is down."

- Pear Deck is also giving back to the community, building and open-sourcing a metrics aggregator that enables end-user monitoring in Prometheus. "We can measure, for example, the time to interactive-dom on the web clients," he says. "The users all report that to our aggregator, then the aggregator reports to Prometheus. So we can set an alarm for some client side errors."

- Most of Pear Deck’s services have now been moved onto Kubernetes. And all of the team’s new code is going on Kubernetes. "Kubernetes lets us experiment with service configurations and stage them on a staging cluster all at once, and test different scenarios and talk about them as a development team looking at code, not just talking about the steps we would eventually take as humans," says Eynon-Lynch. -
-
+

But once it launched, the user base began growing steadily at a rate of 30 percent a month. "Our Heroku bill was getting totally insane," Eynon-Lynch says. But even more crucially, as the company hired more developers to keep pace, "we outgrew Heroku. We wanted to have multiple services and the deploying story got pretty horrendous. We were frustrated that we couldn't have the developers quickly stage a version. Tracing and monitoring became basically impossible."

-
-
- "A huge impact that Prometheus had on us was just an amazing sigh of relief, of feeling like we knew what was happening. It took 45 minutes to implement [the Firebase alarm] because we knew that we had this trustworthy metrics platform in Prometheus...in terms of the cloud, Kubernetes and Prometheus have so much to offer," he says. -
-
+

On top of that, many of Pear Deck's customers are behind government firewalls and connect through Firebase, not Pear Deck's servers, making troubleshooting even more difficult.

-
-
- Looking ahead, the team is planning to explore autoscaling on Kubernetes. With users all over the world but mostly in the United States, there are peaks and valleys in the traffic. One service that’s still on App Engine can get as many as 10,000 requests a second during the day but far less at night. "We pay for the same servers at night, so I understand there’s autoscaling that we can be taking advantage of," he says. "Implementing it is a big worry, exposing the rest of our Kubernetes cluster to us and maybe messing that up. But it’s definitely our intention to move everything over, because now none of the developers want to work on that app anymore because it’s such a pain to deploy it." -

-They’re also eager to explore the work that Kubernetes is doing with stateful sets. "Right now all of the services we run in Kubernetes are stateless, and Google basically runs our databases for us and manages backups," Eynon-Lynch says. "But we’re interested in building our own web-socket solution that doesn’t have to be super stateful but will have maybe an hour’s worth of state on it." -

-That project will also involve Prometheus, for a dark launch of web socket connections. "We don’t know how reliable web socket connections behind all these horrible firewalls will be to our servers," he says. "We don’t know what work Firebase has done to make them more reliable. So I’m really looking forward to trying to get persistent connections with web sockets to our clients and have optional tools to understand if it’s working. That’s our next new adventure, into stateful servers." -

-As for Prometheus, Eynon-Lynch thinks the company has only gotten started. "We haven’t instrumented all our important features, especially those that depend on third parties," he says. "We have to wait for those third parties to tell us they’re down, which sometimes they don’t do for a long time. So I’m really excited and have more and more confidence in the actual state of our application for our actual users, and not just what the CPU graphs are saying, because of Prometheus and Kubernetes." -

-For a spry startup that’s continuing to grow rapidly—and yes, they’re hiring!—Pear Deck is notably satisfied with how its infrastructure has evolved in the cloud native ecosystem. "Usually I have some angsty thing where I want to get to the new, better technology," says Eynon-Lynch, "but in terms of the cloud, Kubernetes and Prometheus have so much to offer." +

The team began looking around for another solution, and finally decided in early 2016 to start moving the app from Heroku to Docker containers running on Google Kubernetes Engine, orchestrated by Kubernetes and monitored with Prometheus.

-

-
-
+{{< case-studies/quote image="/images/case-studies/peardeck/banner1.jpg" >}} +"When it became clear that Google Kubernetes Engine was going to have a lot of support from Google and be a fully-managed Kubernetes platform, it seemed very obvious to us that was the way to go," says Eynon-Lynch. +{{< /case-studies/quote >}} + +

They had considered other options like Google's App Engine (which they were already using for one service) and Amazon's Elastic Compute Cloud (EC2), while experimenting with running one small service that wasn't accessible to the Internet in Kubernetes. "When it became clear that Google Kubernetes Engine was going to have a lot of support from Google and be a fully-managed Kubernetes platform, it seemed very obvious to us that was the way to go," says Eynon-Lynch. "We didn't really consider Terraform and the other competitors because the abstractions offered by Kubernetes just jumped off the page to us."

+ +

Once the team started porting its Heroku apps into Kubernetes, which was "super easy," he says, the impact was immediate. "Before, to make a new version of the app meant going to Heroku and reconfiguring 10 new services, so basically no one was willing to do it, and we never staged things," he says. "Now we can deploy our exact same configuration in lots of different clusters in 30 seconds. We have a full set up that's always running, and then any of our developers or designers can stage new versions with one command, including their recent changes. We stage all the time now, and everyone stopped talking about how cool it is because it's become invisible how great it is."

+ +

Along with Kubernetes came Prometheus. "Until pretty recently we didn't have any kind of visibility into aggregate server metrics or performance," says Eynon-Lynch. The team had tried to use Google Kubernetes Engine's Stackdriver monitoring, but had problems making it work, and considered New Relic. When they started looking at Prometheus in the fall of 2016, "the fit between the abstractions in Prometheus and the way we think about how our system works, was so clear and obvious," he says.

+ +

The integration with Kubernetes made set-up easy. Once Helm installed Prometheus, "We started getting a graph of the health of all our Kubernetes nodes and pods immediately. I think we were pretty hooked at that point," Eynon-Lynch says. "Then we got our own custom instrumentation working in 15 minutes, and had an actively updated count of requests that we could do, rate on and get a sense of how many users are connected at a given point. And then it was another hour before we had alarms automatically showing up in our Slack channel. All that was in one afternoon. And it was an afternoon of gasping with delight, basically!"

+ +{{< case-studies/quote image="/images/case-studies/peardeck/banner2.jpg" >}} +"We started getting a graph of the health of all our Kubernetes nodes and pods immediately. I think we were pretty hooked at that point," Eynon-Lynch says. "Then we got our own custom instrumentation working in 15 minutes, and had an actively updated count of requests that we could do, rate on and get a sense of how many users are connected at a given point. And then it was another hour before we had alarms automatically showing up in our Slack channel. All that was in one afternoon. And it was an afternoon of gasping with delight, basically!" +{{< /case-studies/quote >}} + +

With Pear Deck's specific challenges—traffic through Firebase as well as government firewalls—Prometheus was a game-changer. "We didn't even realize how stressed out we were about our lack of insight into what was happening with the app," Eynon-Lynch says. Before, when a customer would report that the app wasn't working, the team had to manually investigate the problem without knowing whether customers were affected all over the world, or whether Firebase was down, and where.

+ +

To help solve that problem, the team wrote a script that pings Firebase from several different geographical locations, and then reports the responses to Prometheus in a histogram. "A huge impact that Prometheus had on us was just an amazing sigh of relief, of feeling like we knew what was happening," he says. "It took 45 minutes to implement [the Firebase alarm] because we knew that we had this trustworthy metrics platform in Prometheus. We weren't going to have to figure out, 'Where do we send these metrics? How do we aggregate the metrics? How do we understand them?'"

+ +

Plus, Prometheus has allowed Pear Deck to build alarms for business goals. One measures the rate of successful app loads and goes off if the day's loads are less than 90 percent of the loads from seven days before. "We run a JavaScript app behind ridiculous firewalls and all kinds of crazy browser extensions messing with it—Chrome will push a feature that breaks some CSS that we're using," Eynon-Lynch says. "So that gives us a lot of confidence, and we at least know that people are still logging into the app and using it all the time."

+ +

Now, when a customer complains, and none of the alarms have gone off, the team can feel confident that it's not a widespread problem. "Just to be sure, we can go and double check the graphs and say, 'Yep, there's currently 10,000 people connected to that Firebase node. It's definitely working. Let's investigate your network settings, customer,'" he says. "And we can pass that back off to our support reps instead of the whole development team freaking out that Firebase is down."

+ +

Pear Deck is also giving back to the community, building and open-sourcing a metrics aggregator that enables end-user monitoring in Prometheus. "We can measure, for example, the time to interactive-dom on the web clients," he says. "The users all report that to our aggregator, then the aggregator reports to Prometheus. So we can set an alarm for some client side errors."

+ +

Most of Pear Deck's services have now been moved onto Kubernetes. And all of the team's new code is going on Kubernetes. "Kubernetes lets us experiment with service configurations and stage them on a staging cluster all at once, and test different scenarios and talk about them as a development team looking at code, not just talking about the steps we would eventually take as humans," says Eynon-Lynch.

+ +{{< case-studies/quote >}} +"A huge impact that Prometheus had on us was just an amazing sigh of relief, of feeling like we knew what was happening. It took 45 minutes to implement [the Firebase alarm] because we knew that we had this trustworthy metrics platform in Prometheus...in terms of the cloud, Kubernetes and Prometheus have so much to offer," he says. +{{< /case-studies/quote >}} + +

Looking ahead, the team is planning to explore autoscaling on Kubernetes. With users all over the world but mostly in the United States, there are peaks and valleys in the traffic. One service that's still on App Engine can get as many as 10,000 requests a second during the day but far less at night. "We pay for the same servers at night, so I understand there's autoscaling that we can be taking advantage of," he says. "Implementing it is a big worry, exposing the rest of our Kubernetes cluster to us and maybe messing that up. But it's definitely our intention to move everything over, because now none of the developers want to work on that app anymore because it's such a pain to deploy it."

+ +

They're also eager to explore the work that Kubernetes is doing with stateful sets. "Right now all of the services we run in Kubernetes are stateless, and Google basically runs our databases for us and manages backups," Eynon-Lynch says. "But we're interested in building our own web-socket solution that doesn't have to be super stateful but will have maybe an hour's worth of state on it."

+ +

That project will also involve Prometheus, for a dark launch of web socket connections. "We don't know how reliable web socket connections behind all these horrible firewalls will be to our servers," he says. "We don't know what work Firebase has done to make them more reliable. So I'm really looking forward to trying to get persistent connections with web sockets to our clients and have optional tools to understand if it's working. That's our next new adventure, into stateful servers."

+ +

As for Prometheus, Eynon-Lynch thinks the company has only gotten started. "We haven't instrumented all our important features, especially those that depend on third parties," he says. "We have to wait for those third parties to tell us they're down, which sometimes they don't do for a long time. So I'm really excited and have more and more confidence in the actual state of our application for our actual users, and not just what the CPU graphs are saying, because of Prometheus and Kubernetes."

+ +

For a spry startup that's continuing to grow rapidly—and yes, they're hiring!—Pear Deck is notably satisfied with how its infrastructure has evolved in the cloud native ecosystem. "Usually I have some angsty thing where I want to get to the new, better technology," says Eynon-Lynch, "but in terms of the cloud, Kubernetes and Prometheus have so much to offer."

diff --git a/content/en/case-studies/pearson/index.html b/content/en/case-studies/pearson/index.html index 78f70228e5..501bcea8e7 100644 --- a/content/en/case-studies/pearson/index.html +++ b/content/en/case-studies/pearson/index.html @@ -3,85 +3,81 @@ title: Pearson Case Study linkTitle: Pearson case_study_styles: true cid: caseStudies -css: /css/style_case_studies.css featured: false quote: > - We’re already seeing tremendous benefits with Kubernetes—improved engineering productivity, faster delivery of applications and a simplified infrastructure. But this is just the beginning. Kubernetes will help transform the way that educational content is delivered online. + We're already seeing tremendous benefits with Kubernetes—improved engineering productivity, faster delivery of applications and a simplified infrastructure. But this is just the beginning. Kubernetes will help transform the way that educational content is delivered online. + +new_case_study_styles: true +heading_background: /images/case-studies/pearson/banner1.jpg +heading_title_logo: /images/pearson_logo.png +subheading: > + Reinventing the World's Largest Education Company With Kubernetes +case_study_details: + - Company: Pearson + - Location: Global + - Industry: Education --- -
-

CASE STUDY:
Reinventing the World’s Largest Education Company With Kubernetes -

-
-
- Company  Pearson     Location  Global -      Industry  Education -
-
-
-
-
-

Challenge

- A global education company serving 75 million learners, Pearson set a goal to more than double that number, to 200 million, by 2025. A key part of this growth is in digital learning experiences, and Pearson was having difficulty in scaling and adapting to its growing online audience. They needed an infrastructure platform that would be able to scale quickly and deliver products to market faster. -
-

Solution

- "To transform our infrastructure, we had to think beyond simply enabling automated provisioning," says Chris Jackson, Director for Cloud Platforms & SRE at Pearson. "We realized we had to build a platform that would allow Pearson developers to build, manage and deploy applications in a completely different way." The team chose Docker container technology and Kubernetes orchestration "because of its flexibility, ease of management and the way it would improve our engineers’ productivity." -
-
-
-

Impact

- With the platform, there has been substantial improvements in productivity and speed of delivery. "In some cases, we’ve gone from nine months to provision physical assets in a data center to just a few minutes to provision and get a new idea in front of a customer," says John Shirley, Lead Site Reliability Engineer for the Cloud Platform Team. Jackson estimates they’ve achieved 15-20% developer productivity savings. Before, outages were an issue during their busiest time of year, the back-to-school period. Now, there’s high confidence in their ability to meet aggressive customer SLAs. -
-
-
-
-
-
- "We’re already seeing tremendous benefits with Kubernetes—improved engineering productivity, faster delivery of applications and a simplified infrastructure. But this is just the beginning. Kubernetes will help transform the way that educational content is delivered online."

— Chris Jackson, Director for Cloud Platforms & SRE at Pearson -
-
-
-
- In 2015, Pearson was already serving 75 million learners as the world’s largest education company, offering curriculum and assessment tools for Pre-K through college and beyond. Understanding that innovating the digital education experience was the key to the future of all forms of education, the company set out to increase its reach to 200 million people by 2025.

- That goal would require a transformation of its existing infrastructure, which was in data centers. In some cases, it took nine months to provision physical assets. In order to adapt to the demands of its growing online audience, Pearson needed an infrastructure platform that would be able to scale quickly and deliver business-critical products to market faster. "We had to think beyond simply enabling automated provisioning," says Chris Jackson, Director for Cloud Platforms & SRE at Pearson. "We realized we had to build a platform that would allow Pearson developers to build, manage and deploy applications in a completely different way."

- With 400 development groups and diverse brands with varying business and technical needs, Pearson embraced Docker container technology so that each brand could experiment with building new types of content using their preferred technologies, and then deliver it using containers. Jackson chose Kubernetes orchestration "because of its flexibility, ease of management and the way it would improve our engineers’ productivity," he says.

- The team adopted Kubernetes when it was still version 1.2 and are still going strong now on 1.7; they use Terraform and Ansible to deploy it on to basic AWS primitives. "We were trying to understand how we can create value for Pearson from this technology," says Ben Somogyi, Principal Architect for the Cloud Platforms. "It turned out that Kubernetes’ benefits are huge. We’re trying to help our applications development teams that use our platform go faster, so we filled that gap with a CI/CD pipeline that builds their images for them, standardizes them, patches everything up, allows them to deploy their different environments onto the cluster, and obfuscating the details of how difficult the work underneath the covers is." -
-
-
-
- "Your internal customers need to feel like they are choosing the very best option for them. We are experiencing this first hand in the growth of adoption. We are seeing triple-digit, year-on-year growth of the service."

— Chris Jackson, Director for Cloud Platforms & SRE at Pearson -
-
-
-
- That work resulted in two tools for building and deploying applications in the cluster that Pearson has open sourced. "We’re an education company, so we want to share what we can," says Somogyi.

- Now that development teams no longer have to worry about infrastructure, there have been substantial improvements in productivity and speed of delivery. "In some cases, we’ve gone from nine months to provision physical assets in a data center to just a few minutes to provision and to get a new idea in front of a customer," says John Shirley, Lead Site Reliability Engineer for the Cloud Platform Team.

- According to Jackson, the Cloud Platforms team can "provision a new proof-of-concept environment for a development team in minutes, and then they can take that to production as quickly as they are able to. This is the value proposition of all major technology services, and we had to compete like one to become our developers’ preferred choice. Just because you work for the same company, you do not have the right to force people into a mediocre service. Your internal customers need to feel like they are choosing the very best option for them. We are experiencing this first hand in the growth of adoption. We are seeing triple-digit, year-on-year growth of the service."

- Jackson estimates they’ve achieved a 15-20% boost in productivity for developer teams who adopt the platform. They also see a reduction in the number of customer-impacting incidents. Plus, says Jackson, "Teams who were previously limited to 1-2 releases per academic year can now ship code multiple times per day!" -
-
-
-
- "Teams who were previously limited to 1-2 releases per academic year can now ship code multiple times per day!"

— Chris Jackson, Director for Cloud Platforms & SRE at Pearson -
-
-
-
- Availability has also been positively impacted. The back-to-school period is the company’s busiest time of year, and "you have to keep applications up," says Somogyi. Before, this was a pain point for the legacy infrastructure. Now, for the applications that have been migrated to the Kubernetes platform, "We have 100% uptime. We’re not worried about 9s. There aren’t any. It’s 100%, which is pretty astonishing for us, compared to some of the existing platforms that have legacy challenges," says Shirley. -

- "You can’t even begin to put a price on how much that saves the company," Jackson explains. "A reduction in the number of support cases takes load out of our operations. The customer sentiment of having a reliable product drives customer retention and growth. It frees us to think about investing more into our digital transformation and taking a better quality of education to a global scale." -

- The platform itself is also being broken down, "so we can quickly release smaller pieces of the platform, like upgrading our Kubernetes or all the different modules that make up our platform," says Somogyi. "One of the big focuses in 2018 is this scheme of delivery to update the platform itself." -

- Guided by Pearson’s overarching goal of getting to 200 million users, the team has run internal tests of the platform’s scalability. "We had a challenge: 28 million requests within a 10 minute period," says Shirley. "And we demonstrated that we can hit that, with an acceptable latency. We saw that we could actually get that pretty readily, and we scaled up in just a few seconds, using open source tools entirely. Shout out to Locustfor that one. So that’s amazing." -
-
-
- "We have 100% uptime. We’re not worried about 9s. There aren’t any. It’s 100%, which is pretty astonishing for us, compared to some of the existing platforms that have legacy challenges. You can’t even begin to put a price on how much that saves the company."

— Benjamin Somogyi, Principal Systems Architect at Pearson
-
-
- In just two years, "We’re already seeing tremendous benefits with Kubernetes—improved engineering productivity, faster delivery of applications and a simplified infrastructure," says Jackson. "But this is just the beginning. Kubernetes will help transform the way that educational content is delivered online."

- So far, about 15 production products are running on the new platform, including Pearson’s new flagship digital education service, the Global Learning Platform. The Cloud Platform team continues to prepare, onboard and support customers that are a good fit for the platform. Some existing products will be refactored into 12-factor apps, while others are being developed so that they can live on the platform from the get-go. "There are challenges with bringing in new customers of course, because we have to help them to see a different way of developing, a different way of building," says Shirley.

- But, he adds, "It is our corporate motto: Always Learning. We encourage those teams that haven’t started a cloud native journey, to see the future of technology, to learn, to explore. It will pique your interest. Keep learning." -
-
+ +

Challenge

+ +

A global education company serving 75 million learners, Pearson set a goal to more than double that number, to 200 million, by 2025. A key part of this growth is in digital learning experiences, and Pearson was having difficulty in scaling and adapting to its growing online audience. They needed an infrastructure platform that would be able to scale quickly and deliver products to market faster.

+ +

Solution

+ +

"To transform our infrastructure, we had to think beyond simply enabling automated provisioning," says Chris Jackson, Director for Cloud Platforms & SRE at Pearson. "We realized we had to build a platform that would allow Pearson developers to build, manage and deploy applications in a completely different way." The team chose Docker container technology and Kubernetes orchestration "because of its flexibility, ease of management and the way it would improve our engineers' productivity."

+ +

Impact

+ +

With the platform, there has been substantial improvements in productivity and speed of delivery. "In some cases, we've gone from nine months to provision physical assets in a data center to just a few minutes to provision and get a new idea in front of a customer," says John Shirley, Lead Site Reliability Engineer for the Cloud Platform Team. Jackson estimates they've achieved 15-20% developer productivity savings. Before, outages were an issue during their busiest time of year, the back-to-school period. Now, there's high confidence in their ability to meet aggressive customer SLAs.

+ +{{< case-studies/quote author="Chris Jackson, Director for Cloud Platforms & SRE at Pearson" >}} +"We're already seeing tremendous benefits with Kubernetes—improved engineering productivity, faster delivery of applications and a simplified infrastructure. But this is just the beginning. Kubernetes will help transform the way that educational content is delivered online." +{{< /case-studies/quote >}} + +

In 2015, Pearson was already serving 75 million learners as the world's largest education company, offering curriculum and assessment tools for Pre-K through college and beyond. Understanding that innovating the digital education experience was the key to the future of all forms of education, the company set out to increase its reach to 200 million people by 2025.

+ +

That goal would require a transformation of its existing infrastructure, which was in data centers. In some cases, it took nine months to provision physical assets. In order to adapt to the demands of its growing online audience, Pearson needed an infrastructure platform that would be able to scale quickly and deliver business-critical products to market faster. "We had to think beyond simply enabling automated provisioning," says Chris Jackson, Director for Cloud Platforms & SRE at Pearson. "We realized we had to build a platform that would allow Pearson developers to build, manage and deploy applications in a completely different way."

+ +

With 400 development groups and diverse brands with varying business and technical needs, Pearson embraced Docker container technology so that each brand could experiment with building new types of content using their preferred technologies, and then deliver it using containers. Jackson chose Kubernetes orchestration "because of its flexibility, ease of management and the way it would improve our engineers' productivity," he says.

+ +

The team adopted Kubernetes when it was still version 1.2 and are still going strong now on 1.7; they use Terraform and Ansible to deploy it on to basic AWS primitives. "We were trying to understand how we can create value for Pearson from this technology," says Ben Somogyi, Principal Architect for the Cloud Platforms. "It turned out that Kubernetes' benefits are huge. We're trying to help our applications development teams that use our platform go faster, so we filled that gap with a CI/CD pipeline that builds their images for them, standardizes them, patches everything up, allows them to deploy their different environments onto the cluster, and obfuscating the details of how difficult the work underneath the covers is."

+ +{{< case-studies/quote + image="/images/case-studies/pearson/banner3.jpg" + author="Chris Jackson, Director for Cloud Platforms & SRE at Pearson" +>}} +"Your internal customers need to feel like they are choosing the very best option for them. We are experiencing this first hand in the growth of adoption. We are seeing triple-digit, year-on-year growth of the service." +{{< /case-studies/quote >}} + +

That work resulted in two tools for building and deploying applications in the cluster that Pearson has open sourced. "We're an education company, so we want to share what we can," says Somogyi.

+ +

Now that development teams no longer have to worry about infrastructure, there have been substantial improvements in productivity and speed of delivery. "In some cases, we've gone from nine months to provision physical assets in a data center to just a few minutes to provision and to get a new idea in front of a customer," says John Shirley, Lead Site Reliability Engineer for the Cloud Platform Team.

+ +

According to Jackson, the Cloud Platforms team can "provision a new proof-of-concept environment for a development team in minutes, and then they can take that to production as quickly as they are able to. This is the value proposition of all major technology services, and we had to compete like one to become our developers' preferred choice. Just because you work for the same company, you do not have the right to force people into a mediocre service. Your internal customers need to feel like they are choosing the very best option for them. We are experiencing this first hand in the growth of adoption. We are seeing triple-digit, year-on-year growth of the service."

+ +

Jackson estimates they've achieved a 15-20% boost in productivity for developer teams who adopt the platform. They also see a reduction in the number of customer-impacting incidents. Plus, says Jackson, "Teams who were previously limited to 1-2 releases per academic year can now ship code multiple times per day!"

+ +{{< case-studies/quote + image="/images/case-studies/pearson/banner4.jpg" + author="Chris Jackson, Director for Cloud Platforms & SRE at Pearson" +>}} +"Teams who were previously limited to 1-2 releases per academic year can now ship code multiple times per day!" +{{< /case-studies/quote >}} + +

Availability has also been positively impacted. The back-to-school period is the company's busiest time of year, and "you have to keep applications up," says Somogyi. Before, this was a pain point for the legacy infrastructure. Now, for the applications that have been migrated to the Kubernetes platform, "We have 100% uptime. We're not worried about 9s. There aren't any. It's 100%, which is pretty astonishing for us, compared to some of the existing platforms that have legacy challenges," says Shirley.

+ +

"You can't even begin to put a price on how much that saves the company," Jackson explains. "A reduction in the number of support cases takes load out of our operations. The customer sentiment of having a reliable product drives customer retention and growth. It frees us to think about investing more into our digital transformation and taking a better quality of education to a global scale."

+ +

The platform itself is also being broken down, "so we can quickly release smaller pieces of the platform, like upgrading our Kubernetes or all the different modules that make up our platform," says Somogyi. "One of the big focuses in 2018 is this scheme of delivery to update the platform itself."

+ +

Guided by Pearson's overarching goal of getting to 200 million users, the team has run internal tests of the platform's scalability. "We had a challenge: 28 million requests within a 10 minute period," says Shirley. "And we demonstrated that we can hit that, with an acceptable latency. We saw that we could actually get that pretty readily, and we scaled up in just a few seconds, using open source tools entirely. Shout out to Locustfor that one. So that's amazing."

+ +{{< case-studies/quote author="Benjamin Somogyi, Principal Systems Architect at Pearson" >}} +"We have 100% uptime. We're not worried about 9s. There aren't any. It's 100%, which is pretty astonishing for us, compared to some of the existing platforms that have legacy challenges. You can't even begin to put a price on how much that saves the company." +{{< /case-studies/quote >}} + +

In just two years, "We're already seeing tremendous benefits with Kubernetes—improved engineering productivity, faster delivery of applications and a simplified infrastructure," says Jackson. "But this is just the beginning. Kubernetes will help transform the way that educational content is delivered online."

+ +

So far, about 15 production products are running on the new platform, including Pearson's new flagship digital education service, the Global Learning Platform. The Cloud Platform team continues to prepare, onboard and support customers that are a good fit for the platform. Some existing products will be refactored into 12-factor apps, while others are being developed so that they can live on the platform from the get-go. "There are challenges with bringing in new customers of course, because we have to help them to see a different way of developing, a different way of building," says Shirley.

+ +

But, he adds, "It is our corporate motto: Always Learning. We encourage those teams that haven't started a cloud native journey, to see the future of technology, to learn, to explore. It will pique your interest. Keep learning."

diff --git a/content/en/case-studies/pingcap/index.html b/content/en/case-studies/pingcap/index.html index 8d032c7a8b..07c1e3e086 100644 --- a/content/en/case-studies/pingcap/index.html +++ b/content/en/case-studies/pingcap/index.html @@ -3,94 +3,77 @@ title: pingcap Case Study linkTitle: pingcap case_study_styles: true cid: caseStudies -css: /css/style_case_studies.css featured: false + +new_case_study_styles: true +heading_background: /images/case-studies/pingcap/banner1.jpg +heading_title_logo: /images/pingcap_logo.png +subheading: > + PingCAP Bets on Cloud Native for Its TiDB Database Platform +case_study_details: + - Company: PingCAP + - Location: Beijing, China, and San Mateo, CA + - Industry: Software --- -
-

CASE STUDY:
PingCAP Bets on Cloud Native for Its TiDB Database Platform +

Challenge

-

+

PingCAP is the company leading the development of the popular open source NewSQL database TiDB, which is MySQL-compatible, can handle hybrid transactional and analytical processing (HTAP) workloads, and has a cloud native architectural design. "Having a hybrid multi-cloud product is an important part of our global go-to-market strategy," says Kevin Xu, General Manager of Global Strategy and Operations. In order to achieve that, the team had to address two challenges: "how to deploy, run, and manage a distributed stateful application, such as a distributed database like TiDB, in a containerized world," Xu says, and "how to deliver an easy-to-use, consistent, and reliable experience for our customers when they use TiDB in the cloud, any cloud, whether that's one cloud provider or a combination of different cloud environments." Knowing that using a distributed system isn't easy, they began looking for the right orchestration layer to help reduce some of that complexity for end users.

-
+

Solution

-
- Company  PingCAP     Location  Beijing, China, and San Mateo, CA     Industry  Software -
+

The team started looking at Kubernetes for orchestration early on. "We knew Kubernetes had the promise of helping us solve our problems," says Xu. "We were just waiting for it to mature." In early 2018, PingCAP began integrating Kubernetes into its internal development as well as in its TiDB product. At that point, the team has already had experience using other cloud native technologies, having integrated both Prometheus and gRPC as parts of the TiDB platform earlier on.

-
-
-
-
-

Challenge

- PingCAP is the company leading the development of the popular open source NewSQL database TiDB, which is MySQL-compatible, can handle hybrid transactional and analytical processing (HTAP) workloads, and has a cloud native architectural design. "Having a hybrid multi-cloud product is an important part of our global go-to-market strategy," says Kevin Xu, General Manager of Global Strategy and Operations. In order to achieve that, the team had to address two challenges: "how to deploy, run, and manage a distributed stateful application, such as a distributed database like TiDB, in a containerized world," Xu says, and "how to deliver an easy-to-use, consistent, and reliable experience for our customers when they use TiDB in the cloud, any cloud, whether that’s one cloud provider or a combination of different cloud environments." Knowing that using a distributed system isn’t easy, they began looking for the right orchestration layer to help reduce some of that complexity for end users. -

Solution

- The team started looking at Kubernetes for orchestration early on. "We knew Kubernetes had the promise of helping us solve our problems," says Xu. "We were just waiting for it to mature." In early 2018, PingCAP began integrating Kubernetes into its internal development as well as in its TiDB product. At that point, the team has already had experience using other cloud native technologies, having integrated both Prometheus and gRPC as parts of the TiDB platform earlier on. -

Impact

- Xu says that PingCAP customers have had a "very positive" response so far to Kubernetes being the tool to deploy and manage TiDB. Prometheus, with Grafana as the dashboard, is installed by default when customers deploy TiDB, so that they can monitor performance and make any adjustments needed to reach their target before and while deploying TiDB in production. That monitoring layer "makes the evaluation process and communication much smoother," says Xu. -

- With the company’s Kubernetes-based Operator implementation, which is open sourced, customers are now able to deploy, run, manage, upgrade, and maintain their TiDB clusters in the cloud with no downtime, and reduced workload, burden and overhead. And internally, says Xu, "we’ve completely switched to Kubernetes for our own development and testing, including our data center infrastructure and Schrodinger, an automated testing platform for TiDB. With Kubernetes, our resource usage is greatly improved. Our developers can allocate and deploy clusters themselves, and the deploying process has gone from hours to minutes, so we can devote fewer people to manage IDC resources. The productivity improvement is about 15%, and as we gain more Kubernetes knowledge on the debugging and diagnosis front, the productivity should improve to more than 20%." -
+

Xu says that PingCAP customers have had a "very positive" response so far to Kubernetes being the tool to deploy and manage TiDB. Prometheus, with Grafana as the dashboard, is installed by default when customers deploy TiDB, so that they can monitor performance and make any adjustments needed to reach their target before and while deploying TiDB in production. That monitoring layer "makes the evaluation process and communication much smoother," says Xu.

-
-
-
-
- "We knew Kubernetes had the promise of helping us solve our problems. We were just waiting for it to mature, so we can fold it into our own development and product roadmap." -

- KEVIN XU, GENERAL MANAGER OF GLOBAL STRATEGY AND OPERATIONS, PINGCAP -
-
-
-
-

Since it was introduced in 2015, the open source NewSQL database TiDB has gained a following for its compatibility with MySQL, its ability to handle hybrid transactional and analytical processing (HTAP) workloads—and its cloud native architectural design.

- PingCAP, the company behind TiDB, designed the platform with cloud in mind from day one, says Kevin Xu, General Manager of Global Strategy and Operations, and "having a hybrid multi-cloud product is an important part of our global go-to-market strategy." -

- In order to achieve that, the team had to address two challenges: "how to deploy, run, and manage a distributed stateful application, such as a distributed database like TiDB, in a containerized world," Xu says, and "how to deliver an easy-to-use, consistent, and reliable experience for our customers when they use TiDB in the cloud, any cloud, whether that’s one cloud provider or a combination of different cloud environments." -

- Knowing that using a distributed system isn’t easy, the PingCAP team began looking for the right orchestration layer to help reduce some of that complexity for end users. Kubernetes had been on their radar for quite some time. "We knew Kubernetes had the promise of helping us solve our problems," says Xu. "We were just waiting for it to mature." -
-
-
-
- "With the governance process being so open, it’s not hard to find out what’s the latest development in the technology and community, or figure out who to reach out to if we have problems or issues."

- KEVIN XU, GENERAL MANAGER OF GLOBAL STRATEGY AND OPERATIONS, PINGCAP +

With the company's Kubernetes-based Operator implementation, which is open sourced, customers are now able to deploy, run, manage, upgrade, and maintain their TiDB clusters in the cloud with no downtime, and reduced workload, burden and overhead. And internally, says Xu, "we've completely switched to Kubernetes for our own development and testing, including our data center infrastructure and Schrodinger, an automated testing platform for TiDB. With Kubernetes, our resource usage is greatly improved. Our developers can allocate and deploy clusters themselves, and the deploying process has gone from hours to minutes, so we can devote fewer people to manage IDC resources. The productivity improvement is about 15%, and as we gain more Kubernetes knowledge on the debugging and diagnosis front, the productivity should improve to more than 20%."

-
-
-
-
- That time came in early 2018, when PingCAP began integrating Kubernetes into its internal development as well as in its TiDB product. "Having Kubernetes be part of the CNCF, as opposed to having only the backing of one individual company, was valuable in having confidence in the longevity of the technology," says Xu. Plus, "with the governance process being so open, it’s not hard to find out what’s the latest development in the technology and community, or figure out who to reach out to if we have problems or issues." -

- TiDB’s cloud native architecture consists of a stateless SQL layer (also called TiDB) and a persistent key-value storage layer that supports distributed transactions (TiKV, which is now in the CNCF Sandbox), which are loosely coupled. "You can scale both out or in depending on your computation and storage needs, and the two scaling processes can happen independent of each other," says Xu. The PingCAP team also built the TiDB Operator based on Kubernetes, which helps bootstrap a TiDB cluster on any cloud environment and simplifies and automates deployment, scaling, scheduling, upgrades, and maintenance. The company also recently previewed its fully-managed TiDB Cloud offering. -
-
-
-
- "A cloud native infrastructure will not only save you money and allow you to be more in control of the infrastructure resources you consume, but also empower new product innovation, new experience for your users, and new business possibilities. It’s both a cost reducer and a money maker."

- KEVIN XU, GENERAL MANAGER OF GLOBAL STRATEGY AND OPERATIONS, PINGCAP -
-
+{{< case-studies/quote author="KEVIN XU, GENERAL MANAGER OF GLOBAL STRATEGY AND OPERATIONS, PINGCAP" >}} +"We knew Kubernetes had the promise of helping us solve our problems. We were just waiting for it to mature, so we can fold it into our own development and product roadmap." +{{< /case-studies/quote >}} -
-
- The entire TiDB platform leverages Kubernetes and other cloud native technologies, including Prometheus for monitoring and gRPC for interservice communication. -

- So far, the customer response to the Kubernetes-enabled platform has been "very positive." Prometheus, with Grafana as the dashboard, is installed by default when customers deploy TiDB, so that they can monitor and make any adjustments needed to reach their performance requirements before deploying TiDB in production. That monitoring layer "makes the evaluation process and communication much smoother," says Xu. With the company’s Kubernetes-based Operator implementation, customers are now able to deploy, run, manage, upgrade, and maintain their TiDB clusters in the cloud with no downtime, and reduced workload, burden and overhead. -

- These technologies have also had an impact internally. "We’ve completely switched to Kubernetes for our own development and testing, including our data center infrastructure and Schrodinger, an automated testing platform for TiDB," says Xu. "With Kubernetes, our resource usage is greatly improved. Our developers can allocate and deploy clusters themselves, and the deploying process takes less time, so we can devote fewer people to manage IDC resources. -
+{{< case-studies/lead >}} +Since it was introduced in 2015, the open source NewSQL database TiDB has gained a following for its compatibility with MySQL, its ability to handle hybrid transactional and analytical processing (HTAP) workloads—and its cloud native architectural design. +{{< /case-studies/lead >}} -
-
-"The entire cloud native community, whether it’s Kubernetes, CNCF in general, or cloud native vendors like us, have all gained enough experience—and have the battle scars to prove it—and are ready to help you succeed."

- KEVIN XU, GENERAL MANAGER OF GLOBAL STRATEGY AND OPERATIONS, PINGCAP
-
+

PingCAP, the company behind TiDB, designed the platform with cloud in mind from day one, says Kevin Xu, General Manager of Global Strategy and Operations, and "having a hybrid multi-cloud product is an important part of our global go-to-market strategy."

-
- The productivity improvement is about 15%, and as we gain more Kubernetes knowledge on the debugging and diagnosis front, the productivity should improve to more than 20%." -

- Kubernetes is now a crucial part of PingCAP’s product roadmap. For anyone else considering going cloud native, Xu has this advice: "There’s no better time to get started," he says. "The entire cloud native community, whether it’s Kubernetes, CNCF in general, or cloud native vendors like us, have all gained enough experience—and have the battle scars to prove it—and are ready to help you succeed." -

- In fact, the PingCAP team has seen more and more customers moving toward a cloud native approach, and for good reason. "IT infrastructure is quickly evolving from a cost-center and afterthought, to the core competency and competitiveness of any company," says Xu. "A cloud native infrastructure will not only save you money and allow you to be more in control of the infrastructure resources you consume, but also empower new product innovation, new experience for your users, and new business possibilities. It’s both a cost reducer and a money maker." +

In order to achieve that, the team had to address two challenges: "how to deploy, run, and manage a distributed stateful application, such as a distributed database like TiDB, in a containerized world," Xu says, and "how to deliver an easy-to-use, consistent, and reliable experience for our customers when they use TiDB in the cloud, any cloud, whether that's one cloud provider or a combination of different cloud environments."

-
-
+

Knowing that using a distributed system isn't easy, the PingCAP team began looking for the right orchestration layer to help reduce some of that complexity for end users. Kubernetes had been on their radar for quite some time. "We knew Kubernetes had the promise of helping us solve our problems," says Xu. "We were just waiting for it to mature."

+ +{{< case-studies/quote + image="/images/case-studies/pingcap/banner3.jpg" + author="KEVIN XU, GENERAL MANAGER OF GLOBAL STRATEGY AND OPERATIONS, PINGCAP" +>}} +"With the governance process being so open, it's not hard to find out what's the latest development in the technology and community, or figure out who to reach out to if we have problems or issues." +{{< /case-studies/quote >}} + +

That time came in early 2018, when PingCAP began integrating Kubernetes into its internal development as well as in its TiDB product. "Having Kubernetes be part of the CNCF, as opposed to having only the backing of one individual company, was valuable in having confidence in the longevity of the technology," says Xu. Plus, "with the governance process being so open, it's not hard to find out what's the latest development in the technology and community, or figure out who to reach out to if we have problems or issues."

+ +

TiDB's cloud native architecture consists of a stateless SQL layer (also called TiDB) and a persistent key-value storage layer that supports distributed transactions (TiKV, which is now in the CNCF Sandbox), which are loosely coupled. "You can scale both out or in depending on your computation and storage needs, and the two scaling processes can happen independent of each other," says Xu. The PingCAP team also built the TiDB Operator based on Kubernetes, which helps bootstrap a TiDB cluster on any cloud environment and simplifies and automates deployment, scaling, scheduling, upgrades, and maintenance. The company also recently previewed its fully-managed TiDB Cloud offering.

+ +{{< case-studies/quote + image="/images/case-studies/pingcap/banner4.jpg" + author="KEVIN XU, GENERAL MANAGER OF GLOBAL STRATEGY AND OPERATIONS, PINGCAP" +>}} +"A cloud native infrastructure will not only save you money and allow you to be more in control of the infrastructure resources you consume, but also empower new product innovation, new experience for your users, and new business possibilities. It's both a cost reducer and a money maker." +{{< /case-studies/quote >}} + +

The entire TiDB platform leverages Kubernetes and other cloud native technologies, including Prometheus for monitoring and gRPC for interservice communication.

+ +

So far, the customer response to the Kubernetes-enabled platform has been "very positive." Prometheus, with Grafana as the dashboard, is installed by default when customers deploy TiDB, so that they can monitor and make any adjustments needed to reach their performance requirements before deploying TiDB in production. That monitoring layer "makes the evaluation process and communication much smoother," says Xu. With the company's Kubernetes-based Operator implementation, customers are now able to deploy, run, manage, upgrade, and maintain their TiDB clusters in the cloud with no downtime, and reduced workload, burden and overhead.

+ +

These technologies have also had an impact internally. "We've completely switched to Kubernetes for our own development and testing, including our data center infrastructure and Schrodinger, an automated testing platform for TiDB," says Xu. "With Kubernetes, our resource usage is greatly improved. Our developers can allocate and deploy clusters themselves, and the deploying process takes less time, so we can devote fewer people to manage IDC resources.

+ +{{< case-studies/quote author="KEVIN XU, GENERAL MANAGER OF GLOBAL STRATEGY AND OPERATIONS, PINGCAP" >}} +"The entire cloud native community, whether it's Kubernetes, CNCF in general, or cloud native vendors like us, have all gained enough experience—and have the battle scars to prove it—and are ready to help you succeed." +{{< /case-studies/quote >}} + +

The productivity improvement is about 15%, and as we gain more Kubernetes knowledge on the debugging and diagnosis front, the productivity should improve to more than 20%."

+ +

Kubernetes is now a crucial part of PingCAP's product roadmap. For anyone else considering going cloud native, Xu has this advice: "There's no better time to get started," he says. "The entire cloud native community, whether it's Kubernetes, CNCF in general, or cloud native vendors like us, have all gained enough experience—and have the battle scars to prove it—and are ready to help you succeed."

+ +

In fact, the PingCAP team has seen more and more customers moving toward a cloud native approach, and for good reason. "IT infrastructure is quickly evolving from a cost-center and afterthought, to the core competency and competitiveness of any company," says Xu. "A cloud native infrastructure will not only save you money and allow you to be more in control of the infrastructure resources you consume, but also empower new product innovation, new experience for your users, and new business possibilities. It's both a cost reducer and a money maker."

diff --git a/content/en/case-studies/pinterest/index.html b/content/en/case-studies/pinterest/index.html index e4be7031bb..b95fff054a 100644 --- a/content/en/case-studies/pinterest/index.html +++ b/content/en/case-studies/pinterest/index.html @@ -3,106 +3,82 @@ title: Pinterest Case Study linkTitle: Pinterest case_study_styles: true cid: caseStudies -css: /css/style_case_studies.css featured: false weight: 30 quote: > We are in the position to run things at scale, in a public cloud environment, and test things out in way that a lot of people might not be able to do. + +new_case_study_styles: true +heading_background: /images/case-studies/pinterest/banner1.jpg +heading_title_logo: /images/pinterest_logo.png +subheading: > + Pinning Its Past, Present, and Future on Cloud Native +case_study_details: + - Company: Pinterest + - Location: San Francisco, California + - Industry: Web and Mobile App --- +

Challenge

-
-

CASE STUDY:
Pinning Its Past, Present, and Future on Cloud Native +

After eight years in existence, Pinterest had grown into 1,000 microservices and multiple layers of infrastructure and diverse set-up tools and platforms. In 2016 the company launched a roadmap towards a new compute platform, led by the vision of creating the fastest path from an idea to production, without making engineers worry about the underlying infrastructure.

-

+

Solution

-
- -
- Company  Pinterest     Location  San Francisco, California     Industry  Web and Mobile App -
- -
-
-
-
-

-

Challenge

- After eight years in existence, Pinterest had grown into 1,000 microservices and multiple layers of infrastructure and diverse set-up tools and platforms. In 2016 the company launched a roadmap towards a new compute platform, led by the vision of creating the fastest path from an idea to production, without making engineers worry about the underlying infrastructure. - -

- -

Solution

- The first phase involved moving services to Docker containers. Once these services went into production in early 2017, the team began looking at orchestration to help create efficiencies and manage them in a decentralized way. After an evaluation of various solutions, Pinterest went with Kubernetes. - -

+

The first phase involved moving services to Docker containers. Once these services went into production in early 2017, the team began looking at orchestration to help create efficiencies and manage them in a decentralized way. After an evaluation of various solutions, Pinterest went with Kubernetes.

Impact

- "By moving to Kubernetes the team was able to build on-demand scaling and new failover policies, in addition to simplifying the overall deployment and management of a complicated piece of infrastructure such as Jenkins," says Micheal Benedict, Product Manager for the Cloud and the Data Infrastructure Group at Pinterest. "We not only saw reduced build times but also huge efficiency wins. For instance, the team reclaimed over 80 percent of capacity during non-peak hours. As a result, the Jenkins Kubernetes cluster now uses 30 percent less instance-hours per-day when compared to the previous static cluster." -
-
+

"By moving to Kubernetes the team was able to build on-demand scaling and new failover policies, in addition to simplifying the overall deployment and management of a complicated piece of infrastructure such as Jenkins," says Micheal Benedict, Product Manager for the Cloud and the Data Infrastructure Group at Pinterest. "We not only saw reduced build times but also huge efficiency wins. For instance, the team reclaimed over 80 percent of capacity during non-peak hours. As a result, the Jenkins Kubernetes cluster now uses 30 percent less instance-hours per-day when compared to the previous static cluster."

-
-
- -


-"So far it’s been good, especially the elasticity around how we can configure our Jenkins workloads on that Kubernetes shared cluster. That is the win we were pushing for."

— Micheal Benedict, Product Manager for the Cloud and the Data Infrastructure Group at Pinterest +{{< case-studies/quote author="Micheal Benedict, Product Manager for the Cloud and the Data Infrastructure Group at Pinterest" >}} + +
+"So far it's been good, especially the elasticity around how we can configure our Jenkins workloads on that Kubernetes shared cluster. That is the win we were pushing for." +{{< /case-studies/quote >}} -
-
-
-
-

Pinterest was born on the cloud—running on AWS since day one in 2010—but even cloud native companies can experience some growing pains. Since its launch, Pinterest has become a household name, with more than 200 million active monthly users and 100 billion objects saved. Underneath the hood, there are 1,000 microservices running and hundreds of thousands of data jobs.

-With such growth came layers of infrastructure and diverse set-up tools and platforms for the different workloads, resulting in an inconsistent and complex end-to-end developer experience, and ultimately less velocity to get to production. -So in 2016, the company launched a roadmap toward a new compute platform, led by the vision of having the fastest path from an idea to production, without making engineers worry about the underlying infrastructure.

-The first phase involved moving to Docker. "Pinterest has been heavily running on virtual machines, on EC2 instances directly, for the longest time," says Micheal Benedict, Product Manager for the Cloud and the Data Infrastructure Group. "To solve the problem around packaging software and not make engineers own portions of the fleet and those kinds of challenges, we standardized the packaging mechanism and then moved that to the container on top of the VM. Not many drastic changes. We didn’t want to boil the ocean at that point." +{{< case-studies/lead >}} +Pinterest was born on the cloud—running on AWS since day one in 2010—but even cloud native companies can experience some growing pains. +{{< /case-studies/lead >}} -
-
-
-
- "Though Kubernetes lacked certain things we wanted, we realized that by the time we get to productionizing many of those things, we’ll be able to leverage what the community is doing."

— MICHEAL BENEDICT, PRODUCT MANAGER FOR THE CLOUD AND THE DATA INFRASTRUCTURE GROUP AT PINTEREST -
-
-
-
+

Since its launch, Pinterest has become a household name, with more than 200 million active monthly users and 100 billion objects saved. Underneath the hood, there are 1,000 microservices running and hundreds of thousands of data jobs.

-The first service that was migrated was the monolith API fleet that powers most of Pinterest. At the same time, Benedict’s infrastructure governance team built chargeback and capacity planning systems to analyze how the company uses its virtual machines on AWS. "It became clear that running on VMs is just not sustainable with what we’re doing," says Benedict. "A lot of resources were underutilized. There were efficiency efforts, which worked fine at a certain scale, but now you have to move to a more decentralized way of managing that. So orchestration was something we thought could help solve that piece."

-That led to the second phase of the roadmap. In July 2017, after an eight-week evaluation period, the team chose Kubernetes over other orchestration platforms. "Kubernetes lacked certain things at the time—for example, we wanted Spark on Kubernetes," says Benedict. "But we realized that the dev cycles we would put in to even try building that is well worth the outcome, both for Pinterest as well as the community. We’ve been in those conversations in the Big Data SIG. We realized that by the time we get to productionizing many of those things, we’ll be able to leverage what the community is doing."

-At the beginning of 2018, the team began onboarding its first use case into the Kubernetes system: Jenkins workloads. "Although we have builds happening during a certain period of the day, we always need to allocate peak capacity," says Benedict. "They don’t have any auto-scaling capabilities, so that capacity stays constant. It is difficult to speed up builds because ramping up takes more time. So given those kind of concerns, we thought that would be a perfect use case for us to work on." +

With such growth came layers of infrastructure and diverse set-up tools and platforms for the different workloads, resulting in an inconsistent and complex end-to-end developer experience, and ultimately less velocity to get to production. So in 2016, the company launched a roadmap toward a new compute platform, led by the vision of having the fastest path from an idea to production, without making engineers worry about the underlying infrastructure.

+

The first phase involved moving to Docker. "Pinterest has been heavily running on virtual machines, on EC2 instances directly, for the longest time," says Micheal Benedict, Product Manager for the Cloud and the Data Infrastructure Group. "To solve the problem around packaging software and not make engineers own portions of the fleet and those kinds of challenges, we standardized the packaging mechanism and then moved that to the container on top of the VM. Not many drastic changes. We didn't want to boil the ocean at that point."

-
-
-
-
-"So far it’s been good, especially the elasticity around how we can configure our Jenkins workloads on Kubernetes shared cluster. That is the win we were pushing for."

— MICHEAL BENEDICT, PRODUCT MANAGER FOR THE CLOUD AND THE DATA INFRASTRUCTURE GROUP AT PINTEREST -
-
+{{< case-studies/quote + image="/images/case-studies/pinterest/banner3.jpg" + author="MICHEAL BENEDICT, PRODUCT MANAGER FOR THE CLOUD AND THE DATA INFRASTRUCTURE GROUP AT PINTEREST" +>}} +"Though Kubernetes lacked certain things we wanted, we realized that by the time we get to productionizing many of those things, we'll be able to leverage what the community is doing." +{{< /case-studies/quote >}} -
-
- They ramped up the cluster, and working with a team of four people, got the Jenkins Kubernetes cluster ready for production. "We still have our static Jenkins cluster," says Benedict, "but on Kubernetes, we are doing similar builds, testing the entire pipeline, getting the artifact ready and just doing the comparison to see, how much time did it take to build over here. Is the SLA okay, is the artifact generated correct, are there issues there?"

-"So far it’s been good," he adds, "especially the elasticity around how we can configure our Jenkins workloads on Kubernetes shared cluster. That is the win we were pushing for."

-By the end of Q1 2018, the team successfully migrated Jenkins Master to run natively on Kubernetes and also collaborated on the Jenkins Kubernetes Plugin to manage the lifecycle of workers. "We’re currently building the entire Pinterest JVM stack (one of the larger monorepos at Pinterest which was recently bazelized) on this new cluster," says Benedict. "At peak, we run thousands of pods on a few hundred nodes. Overall, by moving to Kubernetes the team was able to build on-demand scaling and new failover policies, in addition to simplifying the overall deployment and management of a complicated piece of infrastructure such as Jenkins. We not only saw reduced build times but also huge efficiency wins. For instance, the team reclaimed over 80 percent of capacity during non-peak hours. As a result, the Jenkins Kubernetes cluster now uses 30 percent less instance-hours per-day when compared to the previous static cluster." +

The first service that was migrated was the monolith API fleet that powers most of Pinterest. At the same time, Benedict's infrastructure governance team built chargeback and capacity planning systems to analyze how the company uses its virtual machines on AWS. "It became clear that running on VMs is just not sustainable with what we're doing," says Benedict. "A lot of resources were underutilized. There were efficiency efforts, which worked fine at a certain scale, but now you have to move to a more decentralized way of managing that. So orchestration was something we thought could help solve that piece."

+

That led to the second phase of the roadmap. In July 2017, after an eight-week evaluation period, the team chose Kubernetes over other orchestration platforms. "Kubernetes lacked certain things at the time—for example, we wanted Spark on Kubernetes," says Benedict. "But we realized that the dev cycles we would put in to even try building that is well worth the outcome, both for Pinterest as well as the community. We've been in those conversations in the Big Data SIG. We realized that by the time we get to productionizing many of those things, we'll be able to leverage what the community is doing."

-
+

At the beginning of 2018, the team began onboarding its first use case into the Kubernetes system: Jenkins workloads. "Although we have builds happening during a certain period of the day, we always need to allocate peak capacity," says Benedict. "They don't have any auto-scaling capabilities, so that capacity stays constant. It is difficult to speed up builds because ramping up takes more time. So given those kind of concerns, we thought that would be a perfect use case for us to work on."

-
-
- "We are in the position to run things at scale, in a public cloud environment, and test things out in way that a lot of people might not be able to do."

— MICHEAL BENEDICT, PRODUCT MANAGER FOR THE CLOUD AND THE DATA INFRASTRUCTURE GROUP AT PINTEREST -
-
+{{< case-studies/quote + image="/images/case-studies/pinterest/banner4.jpg" + author="MICHEAL BENEDICT, PRODUCT MANAGER FOR THE CLOUD AND THE DATA INFRASTRUCTURE GROUP AT PINTEREST" +>}} +"So far it's been good, especially the elasticity around how we can configure our Jenkins workloads on Kubernetes shared cluster. That is the win we were pushing for." +{{< /case-studies/quote >}} -
- Benedict points to a "pretty robust roadmap" going forward. In addition to the Pinterest big data team’s experiments with Spark on Kubernetes, the company collaborated with Amazon’s EKS team on an ENI/CNI plug in.

-Once the Jenkins cluster is up and running out of dark mode, Benedict hopes to establish best practices, including having governance primitives established—including integration with the chargeback system—before moving on to migrating the next service. "We have a healthy pipeline of use-cases to be on-boarded. After Jenkins, we want to enable support for Tensorflow and Apache Spark. At some point, we aim to move the company’s monolithic API service. If we move that and understand the complexity around that, it builds our confidence," says Benedict. "It sets us up for migration of all our other services."

-After years of being a cloud native pioneer, Pinterest is eager to share its ongoing journey. "We are in the position to run things at scale, in a public cloud environment, and test things out in way that a lot of people might not be able to do," says Benedict. "We’re in a great position to contribute back some of those learnings." +

They ramped up the cluster, and working with a team of four people, got the Jenkins Kubernetes cluster ready for production. "We still have our static Jenkins cluster," says Benedict, "but on Kubernetes, we are doing similar builds, testing the entire pipeline, getting the artifact ready and just doing the comparison to see, how much time did it take to build over here. Is the SLA okay, is the artifact generated correct, are there issues there?"

+

"So far it's been good," he adds, "especially the elasticity around how we can configure our Jenkins workloads on Kubernetes shared cluster. That is the win we were pushing for."

+

By the end of Q1 2018, the team successfully migrated Jenkins Master to run natively on Kubernetes and also collaborated on the Jenkins Kubernetes Plugin to manage the lifecycle of workers. "We're currently building the entire Pinterest JVM stack (one of the larger monorepos at Pinterest which was recently bazelized) on this new cluster," says Benedict. "At peak, we run thousands of pods on a few hundred nodes. Overall, by moving to Kubernetes the team was able to build on-demand scaling and new failover policies, in addition to simplifying the overall deployment and management of a complicated piece of infrastructure such as Jenkins. We not only saw reduced build times but also huge efficiency wins. For instance, the team reclaimed over 80 percent of capacity during non-peak hours. As a result, the Jenkins Kubernetes cluster now uses 30 percent less instance-hours per-day when compared to the previous static cluster."

-
+{{< case-studies/quote author="MICHEAL BENEDICT, PRODUCT MANAGER FOR THE CLOUD AND THE DATA INFRASTRUCTURE GROUP AT PINTEREST">}} +"We are in the position to run things at scale, in a public cloud environment, and test things out in way that a lot of people might not be able to do." +{{< /case-studies/quote >}} -
+

Benedict points to a "pretty robust roadmap" going forward. In addition to the Pinterest big data team's experiments with Spark on Kubernetes, the company collaborated with Amazon's EKS team on an ENI/CNI plug in.

+ +

Once the Jenkins cluster is up and running out of dark mode, Benedict hopes to establish best practices, including having governance primitives established—including integration with the chargeback system—before moving on to migrating the next service. "We have a healthy pipeline of use-cases to be on-boarded. After Jenkins, we want to enable support for Tensorflow and Apache Spark. At some point, we aim to move the company's monolithic API service. If we move that and understand the complexity around that, it builds our confidence," says Benedict. "It sets us up for migration of all our other services."

+ +

After years of being a cloud native pioneer, Pinterest is eager to share its ongoing journey. "We are in the position to run things at scale, in a public cloud environment, and test things out in way that a lot of people might not be able to do," says Benedict. "We're in a great position to contribute back some of those learnings."

diff --git a/content/en/case-studies/prowise/index.html b/content/en/case-studies/prowise/index.html index 2f0beda5ae..b8b584aa41 100644 --- a/content/en/case-studies/prowise/index.html +++ b/content/en/case-studies/prowise/index.html @@ -3,97 +3,81 @@ title: Prowise Case Study linkTitle: prowise case_study_styles: true cid: caseStudies -css: /css/style_case_studies.css featured: false + +new_case_study_styles: true +heading_background: /images/case-studies/prowise/banner1.jpg +heading_title_logo: /images/prowise_logo.png +subheading: > + Prowise: How Kubernetes is Enabling the Edtech Solution's Global Expansion +case_study_details: + - Company: Prowise + - Location: Budel, The Netherlands + - Industry: Edtech --- +

Challenge

-
-

CASE STUDY:
Prowise: How Kubernetes is Enabling the Edtech Solution’s Global Expansion -

+

A Dutch company that produces educational devices and software used around the world, Prowise had an infrastructure based on Linux services with multiple availability zones in Europe, Australia, and the U.S. "We've grown a lot in the past couple of years, and we started to encounter problems with versioning and flexible scaling," says Senior DevOps Engineer Victor van den Bosch, "not only scaling in demands, but also in being able to deploy multiple products which all have their own versions, their own development teams, and their own problems that they're trying to solve. To be able to put that all on the same platform without much resistance is what we were looking for. We wanted to future proof our infrastructure, and also solve some of the problems that are associated with just running a normal Linux service."

-
+

Solution

-
- Company  Prowise     Location  Budel, The Netherlands      Industry  Edtech -
+

The Prowise team adopted containerization, spent time improving its CI/CD pipelines, and chose Microsoft Azure's managed Kubernetes service, AKS, for orchestration. "Kubernetes solves things like networking really well, in a way that fits our business model," says van den Bosch. "We want to focus on our core products, and that's the software that runs on it and not necessarily the infrastructure itself."

-
-
-
-
- -

Challenge

- A Dutch company that produces educational devices and software used around the world, Prowise had an infrastructure based on Linux services with multiple availability zones in Europe, Australia, and the U.S. “We’ve grown a lot in the past couple of years, and we started to encounter problems with versioning and flexible scaling,” says Senior DevOps Engineer Victor van den Bosch, “not only scaling in demands, but also in being able to deploy multiple products which all have their own versions, their own development teams, and their own problems that they’re trying to solve. To be able to put that all on the same platform without much resistance is what we were looking for. We wanted to future proof our infrastructure, and also solve some of the problems that are associated with just running a normal Linux service.” -

Solution

- The Prowise team adopted containerization, spent time improving its CI/CD pipelines, and chose Microsoft Azure’s managed Kubernetes service, AKS, for orchestration. “Kubernetes solves things like networking really well, in a way that fits our business model,” says van den Bosch. “We want to focus on our core products, and that’s the software that runs on it and not necessarily the infrastructure itself.”

Impact

- With its first web-based applications now running in beta on Prowise’s Kubernetes platform, the team is seeing the benefits of rapid and smooth deployments. “The old way of deploying took half an hour of preparations and half an hour deploying it. With Kubernetes, it’s a couple of seconds,” says Senior Developer Bart Haalstra. As a result, adds van den Bosch, “We’ve gone from quarterly releases to a release every month in production. We’re pretty much deploying every hour or just when we find that a feature is ready for production; before, our releases were mostly done on off-hours, where it couldn’t impact our customers, as our confidence in the process was relatively low. Kubernetes has also enabled us to follow up quickly on bugs and implement tweaks to our users with zero downtime between versions. For some bugs we’ve pushed code fixes to production minutes after detection.” Recently, the team launched a new single sign-on solution for use in an internal application. “Due to the resource based architecture of the Kubernetes platform, we were able to bring that application into an entirely new production environment in less than a day, most of that time used for testing after applying the already well-known resource definitions from staging to the new environment,” says van den Bosch. “On a traditional VM this would have likely cost a day or two, and then probably a few weeks to iron out the kinks in our provisioning scripts as we apply updates.” -
-
-
-
-
- "Because of Kubernetes, things have been much easier, our individual applications are better, and we can spend more time on functional implementation. We do not want to go back." -

— VICTOR VAN DEN BOSCH, SENIOR DEVOPS ENGINEER, PROWISE -
-
-
-
-

If you haven’t set foot in a school in awhile, you might be surprised by what you’d see in a digitally connected classroom these days: touchscreen monitors, laptops, tablets, touch tables, and more.

- One of the leaders in the space, the Dutch company Prowise, offers an integrated solution of hardware and software to help educators create a more engaging learning environment. -

- As the company expanded its offerings beyond the Netherlands in recent years—creating multiple availability zones in Europe, Australia, and the U.S., with as many as nine servers per zone—its Linux service-based infrastructure struggled to keep up. “We’ve grown a lot in the past couple of years, and we started to encounter problems with versioning and flexible scaling,” says Senior DevOps Engineer Victor van den Bosch, who was hired by the company in late 2017 to build a new platform. -

- Prowise’s products support ten languages, so the problem wasn’t just scaling in demands, he adds, “but also in being able to deploy multiple products which all have their own versions, their own development teams, and their own problems that they’re trying to solve. To be able to put that all on the same platform without much resistance is what we were looking for. We wanted to future proof our infrastructure, and also solve some of the problems that are associated with just running a normal Linux service.” -

- The company’s existing infrastructure on Microsoft Azure Cloud was all on virtual machines, “a pretty traditional setup,” van den Bosch says. “We decided that we want some features in our software that requires being able to scale quickly, being able to deploy new applications and versions on different versions of different programming languages quickly. And we didn’t really want the hassle of trying to keep those servers in a particular state.” -
-
-
-
- "You don’t have to go all-in immediately. You can just take a few projects, a service, run it alongside your more traditional stack, and build it up from there. Kubernetes scales, so as you add applications and services to it, it will scale with you. You don’t have to do it all at once, and that’s really a secret to everything, but especially true to Kubernetes."

— VICTOR VAN DEN BOSCH, SENIOR DEVOPS ENGINEER, PROWISE -
-
-
-
- After researching possible solutions, he opted for containerization and Kubernetes orchestration. “Containerization is the future,” van den Bosch says. “Kubernetes solves things like networking really well, in a way that fits our business model. We want to focus on our core products, and that’s the software that runs on it and not necessarily the infrastructure itself.” Plus, the Prowise team liked that there was no vendor lock-in. “We don’t want to be limited to one platform,” he says. “We try not to touch products that are very proprietary and can’t be ported easily to another vendor.” -

- The time to market with Kubernetes was very short: The first web-based applications on the platform went into beta within a few months. That was largely made possible by van den Bosch’s decision to use Azure’s managed Kubernetes service, AKS. The team then had to figure out which components to keep and which to replace. Monitoring tools like New Relic were taken out “because they tend to become very expensive when you scale it to different availability zones, and it’s just not very maintainable,” he says. -

- A lot of work also went into improving Prowise’s CI/CD pipelines. “We wanted to make sure that the pipelines are automated and easy to use,” he says. “We have a lot of settings and configurations figured out for the pipelines, and it’s just applying those scripts and those configurations to new projects from here on out.” -

- With its first web-based applications now running in beta on Prowise’s Kubernetes platform, the team is seeing the benefits of rapid and smooth deployments. “The old way of deploying took half an hour of preparations and half an hour deploying it. With Kubernetes, it’s a couple of seconds,” says Senior Developer Bart Haalstra. As a result, adds van den Bosch, “We’ve gone from quarterly releases to a release every month in production. We’re pretty much deploying every hour or just when we find that a feature is ready for production. Before, our releases were mostly done on off-hours, where it couldn’t impact our customers, as our confidence the process itself was relatively low. With Kubernetes, we dare to deploy in the middle of a busy day with high confidence the deployment will succeed.” -
-
-
-
- "Kubernetes allows us to really consider the best tools for a problem. Want to have a full-fledged analytics application developed by a third party that is just right for your use case? Run it. Dabbling in machine learning and AI algorithms but getting tired of waiting days for training to complete? It takes only seconds to scale it. Got a stubborn developer that wants to use a programming language no one has heard of? Let him, if it runs in a container, of course. And all of that while your operations team/DevOps get to sleep at night."

- VICTOR VAN DEN BOSCH, SENIOR DEVOPS ENGINEER, PROWISE -
-
+

With its first web-based applications now running in beta on Prowise's Kubernetes platform, the team is seeing the benefits of rapid and smooth deployments. "The old way of deploying took half an hour of preparations and half an hour deploying it. With Kubernetes, it's a couple of seconds," says Senior Developer Bart Haalstra. As a result, adds van den Bosch, "We've gone from quarterly releases to a release every month in production. We're pretty much deploying every hour or just when we find that a feature is ready for production; before, our releases were mostly done on off-hours, where it couldn't impact our customers, as our confidence in the process was relatively low. Kubernetes has also enabled us to follow up quickly on bugs and implement tweaks to our users with zero downtime between versions. For some bugs we've pushed code fixes to production minutes after detection." Recently, the team launched a new single sign-on solution for use in an internal application. "Due to the resource based architecture of the Kubernetes platform, we were able to bring that application into an entirely new production environment in less than a day, most of that time used for testing after applying the already well-known resource definitions from staging to the new environment," says van den Bosch. "On a traditional VM this would have likely cost a day or two, and then probably a few weeks to iron out the kinks in our provisioning scripts as we apply updates."

-
+{{< case-studies/quote author="VICTOR VAN DEN BOSCH, SENIOR DEVOPS ENGINEER, PROWISE" >}} +"Because of Kubernetes, things have been much easier, our individual applications are better, and we can spend more time on functional implementation. We do not want to go back." +{{< /case-studies/quote >}} -
- Plus, van den Bosch says, “Kubernetes has enabled us to follow up quickly on bugs and implement tweaks to our users with zero downtime between versions. For some bugs we’ve pushed code fixes to production minutes after detection.” -

- Recently, the team launched a new single sign-on solution for use in an internal application. “Due to the resource based architecture of the Kubernetes platform, we were able to bring that application into an entirely new production environment in less than a day, most of that time used for testing after applying the already well-known resource definitions from staging to the new environment,” says van den Bosch. “On a traditional VM this would have likely cost a day or two, and then probably a few weeks to iron out the kinks in our provisioning scripts as we apply updates.” -

- Legacy applications are also being moved to Kubernetes. Not long ago, the team needed to set up a Java-based application for compiling and running a frontend. “On a traditional VM, it would have taken quite a bit of time to set it up and keep it up to date, not to mention maintenance for that setup down the line,” says van den Bosch. Instead, it took less than half a day to Dockerize it and get it running on Kubernetes. “It was much easier, and we were able to save costs too because we didn’t have to spin up new VMs specially for it.” -
+{{< case-studies/lead >}} +If you haven't set foot in a school in awhile, you might be surprised by what you'd see in a digitally connected classroom these days: touchscreen monitors, laptops, tablets, touch tables, and more. +{{< /case-studies/lead >}} -
-
-"We’re really trying to deliver integrated solutions with our hardware and software and making it as easy as possible for users to use and collaborate from different places,” says van den Bosch. And, says Haalstra, “We cannot do it without Kubernetes."

- VICTOR VAN DEN BOSCH, SENIOR DEVOPS ENGINEER, PROWISE
-
+

One of the leaders in the space, the Dutch company Prowise, offers an integrated solution of hardware and software to help educators create a more engaging learning environment.

-
- Perhaps most importantly, van den Bosch says, “Kubernetes allows us to really consider the best tools for a problem and take full advantage of microservices architecture. Got a library in Node.js that excels at solving a certain problem? Use it. Want to have a full-fledged analytics application developed by a third party that is just right for your use case? Run it. Dabbling in machine learning and AI algorithms but getting tired of waiting days for training to complete? It takes only seconds to scale it. Got a stubborn developer that wants to use a programming language no one has heard of? Let him, if it runs in a container, of course. And all of that while your operations team/DevOps get to sleep at night.” -

- Looking ahead, all new web development, platforms, and APIs at Prowise will be on Kubernetes. One of the big greenfield projects is a platform for teachers and students that is launching for back-to-school season in September. Users will be able to log in and access a wide variety of educational applications. With the recent acquisition of the software company Oefenweb, Prowise plans to provide adaptive software that allows teachers to get an accurate view of their students’ progress and weak points, and automatically adjusts the difficulty level of assignments to suit individual students. “We will be leveraging Kubernetes’ power to integrate, supplement, and support our combined application portfolio and bring our solutions to more classrooms,” says van den Bosch. -

- Collaborative software is also a priority. With the single sign-in software, users’ settings and credentials are saved in the cloud and can be used on any screen in the world. “We’re really trying to deliver integrated solutions with our hardware and software and making it as easy as possible for users to use and collaborate from different places,” says van den Bosch. And, says Haalstra, “We cannot do it without Kubernetes.” -
+

As the company expanded its offerings beyond the Netherlands in recent years—creating multiple availability zones in Europe, Australia, and the U.S., with as many as nine servers per zone—its Linux service-based infrastructure struggled to keep up. "We've grown a lot in the past couple of years, and we started to encounter problems with versioning and flexible scaling," says Senior DevOps Engineer Victor van den Bosch, who was hired by the company in late 2017 to build a new platform.

-
+

Prowise's products support ten languages, so the problem wasn't just scaling in demands, he adds, "but also in being able to deploy multiple products which all have their own versions, their own development teams, and their own problems that they're trying to solve. To be able to put that all on the same platform without much resistance is what we were looking for. We wanted to future proof our infrastructure, and also solve some of the problems that are associated with just running a normal Linux service."

+ +

The company's existing infrastructure on Microsoft Azure Cloud was all on virtual machines, "a pretty traditional setup," van den Bosch says. "We decided that we want some features in our software that requires being able to scale quickly, being able to deploy new applications and versions on different versions of different programming languages quickly. And we didn't really want the hassle of trying to keep those servers in a particular state."

+ +{{< case-studies/quote + image="/images/case-studies/prowise/banner3.jpg" + author="VICTOR VAN DEN BOSCH, SENIOR DEVOPS ENGINEER, PROWISE" +>}} +"You don't have to go all-in immediately. You can just take a few projects, a service, run it alongside your more traditional stack, and build it up from there. Kubernetes scales, so as you add applications and services to it, it will scale with you. You don't have to do it all at once, and that's really a secret to everything, but especially true to Kubernetes." +{{< /case-studies/quote >}} + +

After researching possible solutions, he opted for containerization and Kubernetes orchestration. "Containerization is the future," van den Bosch says. "Kubernetes solves things like networking really well, in a way that fits our business model. We want to focus on our core products, and that's the software that runs on it and not necessarily the infrastructure itself." Plus, the Prowise team liked that there was no vendor lock-in. "We don't want to be limited to one platform," he says. "We try not to touch products that are very proprietary and can't be ported easily to another vendor."

+ +

The time to market with Kubernetes was very short: The first web-based applications on the platform went into beta within a few months. That was largely made possible by van den Bosch's decision to use Azure's managed Kubernetes service, AKS. The team then had to figure out which components to keep and which to replace. Monitoring tools like New Relic were taken out "because they tend to become very expensive when you scale it to different availability zones, and it's just not very maintainable," he says.

+ +

A lot of work also went into improving Prowise's CI/CD pipelines. "We wanted to make sure that the pipelines are automated and easy to use," he says. "We have a lot of settings and configurations figured out for the pipelines, and it's just applying those scripts and those configurations to new projects from here on out."

+ +

With its first web-based applications now running in beta on Prowise's Kubernetes platform, the team is seeing the benefits of rapid and smooth deployments. "The old way of deploying took half an hour of preparations and half an hour deploying it. With Kubernetes, it's a couple of seconds," says Senior Developer Bart Haalstra. As a result, adds van den Bosch, "We've gone from quarterly releases to a release every month in production. We're pretty much deploying every hour or just when we find that a feature is ready for production. Before, our releases were mostly done on off-hours, where it couldn't impact our customers, as our confidence the process itself was relatively low. With Kubernetes, we dare to deploy in the middle of a busy day with high confidence the deployment will succeed."

+ +{{< case-studies/quote + image="/images/case-studies/prowise/banner4.jpg" + author="VICTOR VAN DEN BOSCH, SENIOR DEVOPS ENGINEER, PROWISE" +>}} +"Kubernetes allows us to really consider the best tools for a problem. Want to have a full-fledged analytics application developed by a third party that is just right for your use case? Run it. Dabbling in machine learning and AI algorithms but getting tired of waiting days for training to complete? It takes only seconds to scale it. Got a stubborn developer that wants to use a programming language no one has heard of? Let him, if it runs in a container, of course. And all of that while your operations team/DevOps get to sleep at night." +{{< /case-studies/quote >}} + +

Plus, van den Bosch says, "Kubernetes has enabled us to follow up quickly on bugs and implement tweaks to our users with zero downtime between versions. For some bugs we've pushed code fixes to production minutes after detection."

+ +

Recently, the team launched a new single sign-on solution for use in an internal application. "Due to the resource based architecture of the Kubernetes platform, we were able to bring that application into an entirely new production environment in less than a day, most of that time used for testing after applying the already well-known resource definitions from staging to the new environment," says van den Bosch. "On a traditional VM this would have likely cost a day or two, and then probably a few weeks to iron out the kinks in our provisioning scripts as we apply updates."

+ +

Legacy applications are also being moved to Kubernetes. Not long ago, the team needed to set up a Java-based application for compiling and running a frontend. "On a traditional VM, it would have taken quite a bit of time to set it up and keep it up to date, not to mention maintenance for that setup down the line," says van den Bosch. Instead, it took less than half a day to Dockerize it and get it running on Kubernetes. "It was much easier, and we were able to save costs too because we didn't have to spin up new VMs specially for it."

+ +{{< case-studies/quote author="VICTOR VAN DEN BOSCH, SENIOR DEVOPS ENGINEER, PROWISE" >}} +"We're really trying to deliver integrated solutions with our hardware and software and making it as easy as possible for users to use and collaborate from different places," says van den Bosch. And, says Haalstra, "We cannot do it without Kubernetes." +{{< /case-studies/quote >}} + +

Perhaps most importantly, van den Bosch says, "Kubernetes allows us to really consider the best tools for a problem and take full advantage of microservices architecture. Got a library in Node.js that excels at solving a certain problem? Use it. Want to have a full-fledged analytics application developed by a third party that is just right for your use case? Run it. Dabbling in machine learning and AI algorithms but getting tired of waiting days for training to complete? It takes only seconds to scale it. Got a stubborn developer that wants to use a programming language no one has heard of? Let him, if it runs in a container, of course. And all of that while your operations team/DevOps get to sleep at night."

+ +

Looking ahead, all new web development, platforms, and APIs at Prowise will be on Kubernetes. One of the big greenfield projects is a platform for teachers and students that is launching for back-to-school season in September. Users will be able to log in and access a wide variety of educational applications. With the recent acquisition of the software company Oefenweb, Prowise plans to provide adaptive software that allows teachers to get an accurate view of their students' progress and weak points, and automatically adjusts the difficulty level of assignments to suit individual students. "We will be leveraging Kubernetes' power to integrate, supplement, and support our combined application portfolio and bring our solutions to more classrooms," says van den Bosch.

+ +

Collaborative software is also a priority. With the single sign-in software, users' settings and credentials are saved in the cloud and can be used on any screen in the world. "We're really trying to deliver integrated solutions with our hardware and software and making it as easy as possible for users to use and collaborate from different places," says van den Bosch. And, says Haalstra, "We cannot do it without Kubernetes."

diff --git a/content/en/case-studies/ricardo-ch/index.html b/content/en/case-studies/ricardo-ch/index.html index 2863ceac75..26a1975ab4 100644 --- a/content/en/case-studies/ricardo-ch/index.html +++ b/content/en/case-studies/ricardo-ch/index.html @@ -3,96 +3,77 @@ title: ricardo.ch Case Study linkTitle: ricardo-ch case_study_styles: true cid: caseStudies -css: /css/style_case_studies.css featured: false + +new_case_study_styles: true +heading_background: /images/case-studies/ricardoch/banner1.png +heading_title_logo: /images/ricardoch_logo.png +subheading: > + ricardo.ch: How Kubernetes Improved Velocity and DevOps Harmony +case_study_details: + - Company: ricardo.ch + - Location: Zurich, Switzerland + - Industry: E-commerce --- -
-

CASE STUDY:
ricardo.ch: How Kubernetes Improved Velocity and DevOps Harmony -

+

Challenge

-
+

A Swiss online marketplace, ricardo.ch was experiencing problems with velocity, as well as a "classic gap" between Development and Operations, with the two sides unable to work well together. "They wanted to, but they didn't have common ground," says Cedric Meury, Head of Platform Engineering. "This was one of the root causes that slowed us down." The company began breaking down the legacy monolith into microservices, and needed orchestration to support the new architecture in its own data centers—as well as bring together Dev and Ops.

-
- Company  ricardo.ch     Location  Zurich, Switzerland      Industry  E-commerce -
+

Solution

-
-
-
-
+

The company adopted Kubernetes for cluster management, Prometheus for monitoring, and Fluentd for logging. The first cluster was deployed on premise in December 2016, with the first service in production three months later. The migration is about half done, and the company plans to move completely to Google Cloud Platform by the end of 2018.

-

Challenge

- A Swiss online marketplace, ricardo.ch was experiencing problems with velocity, as well as a "classic gap" between Development and Operations, with the two sides unable to work well together. "They wanted to, but they didn’t have common ground," says Cedric Meury, Head of Platform Engineering. "This was one of the root causes that slowed us down." The company began breaking down the legacy monolith into microservices, and needed orchestration to support the new architecture in its own data centers—as well as bring together Dev and Ops. -

Solution

- The company adopted Kubernetes for cluster management, Prometheus for monitoring, and Fluentd for logging. The first cluster was deployed on premise in December 2016, with the first service in production three months later. The migration is about half done, and the company plans to move completely to Google Cloud Platform by the end of 2018.

Impact

- Splitting up the monolith into microservices "allowed higher velocity, and Kubernetes was crucial to support that," says Meury. The number of deployments to production has gone from fewer than 10 a week to 30-60 per day. Before, "when there was a problem with something in production, tickets or complaints would be thrown over the wall to operations, the classical problem. Now, people have the chance to look into operations and troubleshoot for themselves first because everything is deployed in a standardized way," says Meury. He sees the impact in everyday interactions: "A couple of weeks ago, I saw a product manager doing a pull request for a JSON file that contains some variables, and someone else accepted it. And it was deployed after a couple of minutes or seconds even, which was unthinkable before. There used to be quite a chain of things that needed to happen, the whole monolith was difficult to understand, even for engineers. So, previously requests would go into large, inefficient Kanban boards and hopefully someone will have done the change after weeks and months." Before, infrastructure- and platform-related projects took months or years to complete; now developers and operators can work together to deploy infrastructure parts via Kubernetes in a matter of weeks and sometimes days. In the long run, the company also expects to notch 50% cost savings going from custom data center and virtual machines to containerized infrastructure and cloud services. -
-
-
-
-
- "Splitting up the monolith allowed higher velocity, and Kubernetes was crucial to support that. Containerization and orchestration by Kubernetes helped us to drastically reduce the conflict between Dev and Ops and also allowed us to speak the same language on both sides of the aisle." -

— CEDRIC MEURY, HEAD OF PLATFORM ENGINEERING, RICARDO.CH -
-
-
-
-

When Cedric Meury joined ricardo.ch in 2016, he saw a clear divide between Operations and Development. In fact, there was literal distance between them: The engineering team worked in France, while the rest of the org was based in Switzerland. -



- "It was a classic gap between those departments and even some anger and frustration here and there," says Meury. "They wanted to work together, but they didn’t have common ground. This was one of the root causes that slowed us down." -

- That gap was hurting velocity at ricardo.ch, a Swiss online marketplace. The website processes up to 2.6 million searches on a peak day from both web and mobile apps, serving 3.2 million members with its live auctions. The technology team’s main challenge was to make sure that "the bids for items come in the right order, and before the auction is finished, and that this works in a fair way," says Meury. "We have a real-time requirement. We also provide an automated system to bid, and it needs to be accurate and correct. With a distributed system, you have the challenge of making sure that the ordering is right. And that’s one of the things we’re currently dealing with." -

- To address the velocity issue, ricardo.ch CTO Jeremy Seitz established a new software factory called EPD, which consists of 65 engineers, 7 product managers and 2 designers. "We brought these three departments together so that they can kind of streamline this and talk to each other much more closely," says Meury. -
-
-
-
- "Being in the End User Community demonstrates that we stand behind these technologies. In Switzerland, if all the companies see that ricardo.ch’s using it, I think that will help adoption. I also like that we’re connected to the other end users, so if there is a really heavy problem, I could go to the Slack channel, and say, ‘Hey, you guys…’ Like Reddit, Github and New York Times or whoever can give a recommendation on what to use here or how to solve that. So that’s kind of a superpower."

— CEDRIC MEURY, HEAD OF PLATFORM ENGINEERING, RICARDO.CH -
-
-
-
+

Splitting up the monolith into microservices "allowed higher velocity, and Kubernetes was crucial to support that," says Meury. The number of deployments to production has gone from fewer than 10 a week to 30-60 per day. Before, "when there was a problem with something in production, tickets or complaints would be thrown over the wall to operations, the classical problem. Now, people have the chance to look into operations and troubleshoot for themselves first because everything is deployed in a standardized way," says Meury. He sees the impact in everyday interactions: "A couple of weeks ago, I saw a product manager doing a pull request for a JSON file that contains some variables, and someone else accepted it. And it was deployed after a couple of minutes or seconds even, which was unthinkable before. There used to be quite a chain of things that needed to happen, the whole monolith was difficult to understand, even for engineers. So, previously requests would go into large, inefficient Kanban boards and hopefully someone will have done the change after weeks and months." Before, infrastructure- and platform-related projects took months or years to complete; now developers and operators can work together to deploy infrastructure parts via Kubernetes in a matter of weeks and sometimes days. In the long run, the company also expects to notch 50% cost savings going from custom data center and virtual machines to containerized infrastructure and cloud services.

- The company also began breaking down the legacy monolith into more than 100 microservices, and needed orchestration to support the new architecture in its own data centers. "Splitting up the monolith allowed higher velocity, and Kubernetes was crucial to support that," says Meury. "Containerization and orchestration by Kubernetes helped us to drastically reduce the conflict between Dev and Ops and also allowed us to speak the same language on both sides of the aisle." -

- Meury put together a platform engineering team to choose the tools—including Fluentd for logging and Prometheus for monitoring, with Grafana visualization—and lay the groundwork for the first Kubernetes cluster, which was installed on premise in December 2016. Within a few weeks, the new platform was available to teams, who were given training sessions and documentation. The platform engineering team then embedded with engineers to help them deploy their applications on the new platform. The first service in production was the ricardo.ch jobs page. "It was an exercise in front-end development, so the developers could experiment with a new stack," says Meury. -

- Meury estimates that half of the application has been migrated to Kubernetes. And the plan is to move everything to the Google Cloud Platform by the end of 2018. "We are still running some servers in our own data centers, but all of the containerization efforts and describing our services as Kubernetes manifests will allow us to quite easily make that shift," says Meury. -
-
-
-
- "One of the core moments was when a front-end developer asked me how to do a port forward from his laptop to a front-end application to debug, and I told him the command. And he was like, ‘Wow, that’s all I need to do?’ He was super excited and happy about it. That showed me that this power in the right hands can just accelerate development." -

- CEDRIC MEURY, HEAD OF PLATFORM ENGINEERING, RICARDO.CH -
-
+{{< case-studies/quote author="CEDRIC MEURY, HEAD OF PLATFORM ENGINEERING, RICARDO.CH" >}} +"Splitting up the monolith allowed higher velocity, and Kubernetes was crucial to support that. Containerization and orchestration by Kubernetes helped us to drastically reduce the conflict between Dev and Ops and also allowed us to speak the same language on both sides of the aisle." +{{< /case-studies/quote >}} -
-
- The impact has been great. Moving from custom data center and virtual machines to containerized infrastructure and cloud services is expected to result in 50% cost savings for the company. The number of deployments to production has gone from fewer than 10 a week to 30-60 per day. Before, "when there was a problem with something in production, tickets or complaints would be thrown over the wall to operations, the classical problem," says Meury. "Now, people have the chance to look into operations and troubleshoot for themselves first because everything is deployed in a standardized way. That reduces time and uncertainty." -

- Meury also sees the impact in everyday interactions: "A couple of weeks ago, I saw a product manager doing a pull request for a JSON file that contains some variables, and someone else accepted it. And it was deployed after a couple of minutes or seconds even, which was unthinkable before. There used to be quite a chain of things that needed to happen, the whole monolith was difficult to understand, even for engineers. So, previously requests would go into large, inefficient Kanban boards and hopefully someone will have done the change after weeks and months." -

- The divide between Dev and Ops has also diminished. "After a couple of months, I got requests by people saying, ‘Hey, could you help me install the Kubernetes client? I want to actually look at what’s going on,’" says Meury. "People were directly looking at the state of the system, bringing them much, much closer to the operations." Before, infrastructure- and platform-related projects took months or years to complete; now developers and operators can work together to deploy infrastructure parts via Kubernetes in a matter of weeks and sometimes days. -
+{{< case-studies/lead >}} +When Cedric Meury joined ricardo.ch in 2016, he saw a clear divide between Operations and Development. In fact, there was literal distance between them: The engineering team worked in France, while the rest of the org was based in Switzerland. +{{< /case-studies/lead >}} -
-
-"One of my colleagues was listening to all the talks at KubeCon, and he was overwhelmed by all the tools, technologies, frameworks out there that are currently lacking on our platform, but at the same time, he’s very happy to know that in the future there is so much that we can still explore and we can improve and we can work on."

- CEDRIC MEURY, HEAD OF PLATFORM ENGINEERING, RICARDO.CH
-
+

"It was a classic gap between those departments and even some anger and frustration here and there," says Meury. "They wanted to work together, but they didn't have common ground. This was one of the root causes that slowed us down."

-
+

That gap was hurting velocity at ricardo.ch, a Swiss online marketplace. The website processes up to 2.6 million searches on a peak day from both web and mobile apps, serving 3.2 million members with its live auctions. The technology team's main challenge was to make sure that "the bids for items come in the right order, and before the auction is finished, and that this works in a fair way," says Meury. "We have a real-time requirement. We also provide an automated system to bid, and it needs to be accurate and correct. With a distributed system, you have the challenge of making sure that the ordering is right. And that's one of the things we're currently dealing with."

+

To address the velocity issue, ricardo.ch CTO Jeremy Seitz established a new software factory called EPD, which consists of 65 engineers, 7 product managers and 2 designers. "We brought these three departments together so that they can kind of streamline this and talk to each other much more closely," says Meury.

- The ability to have insight into the system has extended to other parts of the company, too. "I found out that one of our customer support representatives looks at Grafana metrics to find out whether the system is running fine, which is fantastic," says Meury. "Prometheus is directly hooked into customer care." -

- The ricardo.ch cloud native journey has perhaps had the most impact on the Ops team. "We have an operations team that comes from a hardware-based background, and right now they are relearning how to operate in a more virtualized and cloud native world, with great success so far," says Meury. "So besides still operating on-site data center firewalls, they learn to code in Go or do some Python scripting at the same time. Former network administrators are writing Go code. It’s just really cool. -

- For Meury, the journey boils down to this. "One of my colleagues was listening to all the talks at KubeCon, and he was overwhelmed by all the tools, technologies, frameworks out there that are currently lacking on our platform," says Meury. "But at the same time, he’s very happy to know that in the future there is so much that we can still explore and we can improve and we can work on. We’re transitioning from seeing problems everywhere—like, ‘This is broken’ or ‘This is down, and we have to fix it’—more to, ‘How can we actually improve and automate more, and make it nicer for developers and ultimately for the end users?’" -
+{{< case-studies/quote + image="/images/case-studies/ricardoch/banner3.png" + author="CEDRIC MEURY, HEAD OF PLATFORM ENGINEERING, RICARDO.CH" +>}} +"Being in the End User Community demonstrates that we stand behind these technologies. In Switzerland, if all the companies see that ricardo.ch's using it, I think that will help adoption. I also like that we're connected to the other end users, so if there is a really heavy problem, I could go to the Slack channel, and say, 'Hey, you guys…' Like Reddit, Github and New York Times or whoever can give a recommendation on what to use here or how to solve that. So that's kind of a superpower." +{{< /case-studies/quote >}} -
+

The company also began breaking down the legacy monolith into more than 100 microservices, and needed orchestration to support the new architecture in its own data centers. "Splitting up the monolith allowed higher velocity, and Kubernetes was crucial to support that," says Meury. "Containerization and orchestration by Kubernetes helped us to drastically reduce the conflict between Dev and Ops and also allowed us to speak the same language on both sides of the aisle."

+ +

Meury put together a platform engineering team to choose the tools—including Fluentd for logging and Prometheus for monitoring, with Grafana visualization—and lay the groundwork for the first Kubernetes cluster, which was installed on premise in December 2016. Within a few weeks, the new platform was available to teams, who were given training sessions and documentation. The platform engineering team then embedded with engineers to help them deploy their applications on the new platform. The first service in production was the ricardo.ch jobs page. "It was an exercise in front-end development, so the developers could experiment with a new stack," says Meury.

+ +

Meury estimates that half of the application has been migrated to Kubernetes. And the plan is to move everything to the Google Cloud Platform by the end of 2018. "We are still running some servers in our own data centers, but all of the containerization efforts and describing our services as Kubernetes manifests will allow us to quite easily make that shift," says Meury.

+ +{{< case-studies/quote + image="/images/case-studies/ricardoch/banner4.png" + author="CEDRIC MEURY, HEAD OF PLATFORM ENGINEERING, RICARDO.CH" +>}} +"One of the core moments was when a front-end developer asked me how to do a port forward from his laptop to a front-end application to debug, and I told him the command. And he was like, 'Wow, that's all I need to do?' He was super excited and happy about it. That showed me that this power in the right hands can just accelerate development." +{{< /case-studies/quote >}} + +

The impact has been great. Moving from custom data center and virtual machines to containerized infrastructure and cloud services is expected to result in 50% cost savings for the company. The number of deployments to production has gone from fewer than 10 a week to 30-60 per day. Before, "when there was a problem with something in production, tickets or complaints would be thrown over the wall to operations, the classical problem," says Meury. "Now, people have the chance to look into operations and troubleshoot for themselves first because everything is deployed in a standardized way. That reduces time and uncertainty."

+ +

Meury also sees the impact in everyday interactions: "A couple of weeks ago, I saw a product manager doing a pull request for a JSON file that contains some variables, and someone else accepted it. And it was deployed after a couple of minutes or seconds even, which was unthinkable before. There used to be quite a chain of things that needed to happen, the whole monolith was difficult to understand, even for engineers. So, previously requests would go into large, inefficient Kanban boards and hopefully someone will have done the change after weeks and months."

+ +

The divide between Dev and Ops has also diminished. "After a couple of months, I got requests by people saying, 'Hey, could you help me install the Kubernetes client? I want to actually look at what's going on,'" says Meury. "People were directly looking at the state of the system, bringing them much, much closer to the operations." Before, infrastructure- and platform-related projects took months or years to complete; now developers and operators can work together to deploy infrastructure parts via Kubernetes in a matter of weeks and sometimes days.

+ +{{< case-studies/quote author="CEDRIC MEURY, HEAD OF PLATFORM ENGINEERING, RICARDO.CH" >}} +"One of my colleagues was listening to all the talks at KubeCon, and he was overwhelmed by all the tools, technologies, frameworks out there that are currently lacking on our platform, but at the same time, he's very happy to know that in the future there is so much that we can still explore and we can improve and we can work on." +{{< /case-studies/quote >}} + +

The ability to have insight into the system has extended to other parts of the company, too. "I found out that one of our customer support representatives looks at Grafana metrics to find out whether the system is running fine, which is fantastic," says Meury. "Prometheus is directly hooked into customer care."

+ +

The ricardo.ch cloud native journey has perhaps had the most impact on the Ops team. "We have an operations team that comes from a hardware-based background, and right now they are relearning how to operate in a more virtualized and cloud native world, with great success so far," says Meury. "So besides still operating on-site data center firewalls, they learn to code in Go or do some Python scripting at the same time. Former network administrators are writing Go code. It's just really cool.

+ +

For Meury, the journey boils down to this. "One of my colleagues was listening to all the talks at KubeCon, and he was overwhelmed by all the tools, technologies, frameworks out there that are currently lacking on our platform," says Meury. "But at the same time, he's very happy to know that in the future there is so much that we can still explore and we can improve and we can work on. We're transitioning from seeing problems everywhere—like, 'This is broken' or 'This is down, and we have to fix it'—more to, 'How can we actually improve and automate more, and make it nicer for developers and ultimately for the end users?'"

diff --git a/content/en/case-studies/slamtec/index.html b/content/en/case-studies/slamtec/index.html index 86ebe15f91..e34bf2bd08 100644 --- a/content/en/case-studies/slamtec/index.html +++ b/content/en/case-studies/slamtec/index.html @@ -3,86 +3,69 @@ title: Slamtec Case Study linkTitle: slamtec case_study_styles: true cid: caseStudies -css: /css/style_case_studies.css featured: false + +new_case_study_styles: true +heading_background: /images/case-studies/slamtec/banner1.jpg +heading_title_logo: /images/slamtec_logo.png +case_study_details: + - Company: Slamtec + - Location: Shanghai, China + - Industry: Robotics --- -
-

CASE STUDY:

-

-
+

Challenge

-
- Company  Slamtec     Location  Shanghai, China     Industry  Robotics -
+

Founded in 2013, SLAMTEC provides service robot autonomous localization and navigation solutions. The company's strength lies in its R&D team's ability to quickly introduce, and continually iterate on, its core products. In the past few years, the company, which had a legacy infrastructure based on Alibaba Cloud and VMware vSphere, began looking to build its own stable and reliable container cloud platform to host its Internet of Things applications. "Our needs for the cloud platform included high availability, scalability and security; multi-granularity monitoring alarm capability; friendliness to containers and microservices; and perfect CI/CD support," says Benniu Ji, Director of Cloud Computing Business Division.

-
-
-
-
-

Challenge

- Founded in 2013, SLAMTEC provides service robot autonomous localization and navigation solutions. The company’s strength lies in its R&D team’s ability to quickly introduce, and continually iterate on, its core products. In the past few years, the company, which had a legacy infrastructure based on Alibaba Cloud and VMware vSphere, began looking to build its own stable and reliable container cloud platform to host its Internet of Things applications. "Our needs for the cloud platform included high availability, scalability and security; multi-granularity monitoring alarm capability; friendliness to containers and microservices; and perfect CI/CD support," says Benniu Ji, Director of Cloud Computing Business Division. +

Solution

+ +

Ji's team chose Kubernetes for orchestration. "CNCF brings quality assurance and a complete ecosystem for Kubernetes, which is very important for the wide application of Kubernetes," says Ji. Thus Slamtec decided to adopt other CNCF projects as well: Prometheus monitoring, Fluentd logging, Harbor registry, and Helm package manager.

-

Solution

- Ji’s team chose Kubernetes for orchestration. "CNCF brings quality assurance and a complete ecosystem for Kubernetes, which is very important for the wide application of Kubernetes," says Ji. Thus Slamtec decided to adopt other CNCF projects as well: Prometheus monitoring, Fluentd logging, Harbor registry, and Helm package manager. -

Impact

- With the new platform, Ji reports that Slamtec has experienced "18+ months of 100% stability!" For users, there is now zero service downtime and seamless upgrades. "Kubernetes with third-party service mesh integration (Istio, along with Jaeger and Envoy) significantly reduced the microservice configuration and maintenance efforts by 50%," he adds. With centralized metrics monitoring and log aggregation provided by Prometheus on Fluentd, teams are saving 50% of time spent on troubleshooting and debugging. Harbor replication has allowed production/staging/testing environments to cross public cloud and the private Kubernetes cluster to share the same container registry, resulting in 30% savings of CI/CD efforts. Plus, Ji says, "Helm has accelerated prototype development and environment setup with its rich sharing charts." -
-
-
-
-
- "Cloud native technology helps us ensure high availability of our business, while improving development and testing efficiency, shortening the research and development cycle and enabling rapid product delivery." -

- BENNIU JI, DIRECTOR OF CLOUD COMPUTING BUSINESS DIVISION -
-
-
-
-

Founded in 2013, Slamtec provides service robot autonomous localization and navigation solutions. In this fast-moving space, the company built its success on the ability of its R&D team to quickly introduce, and continually iterate on, its core products. -

- To sustain that development velocity, the company over the past few years began looking to build its own stable and reliable container cloud platform to host its Internet of Things applications. With a legacy infrastructure based on Alibaba Cloud and VMware vSphere, Slamtec teams had already adopted microservice architecture and continuous delivery, for "fine granularity on-demand scaling, fault isolation, ease of development, testing, and deployment, and for facilitating high-speed iteration," says Benniu Ji, Director of Cloud Computing Business Division. So "our needs for the cloud platform included high availability, scalability and security; multi-granularity monitoring alarm capability; friendliness to containers and microservices; and perfect CI/CD support." -

- After an evaluation of existing technologies, Ji’s team chose Kubernetes for orchestration. "CNCF brings quality assurance and a complete ecosystem for Kubernetes, which is very important for the wide application of Kubernetes," says Ji. Plus, "avoiding binding to an infrastructure technology or provider can help us ensure that our business is deployed and migrated in cross-regional environments, and can serve users all over the world." -
-
-
-
- "CNCF brings quality assurance and a complete ecosystem for Kubernetes, which is very important for the wide application of Kubernetes."

- BENNIU JI, DIRECTOR OF CLOUD COMPUTING BUSINESS DIVISION +

With the new platform, Ji reports that Slamtec has experienced "18+ months of 100% stability!" For users, there is now zero service downtime and seamless upgrades. "Kubernetes with third-party service mesh integration (Istio, along with Jaeger and Envoy) significantly reduced the microservice configuration and maintenance efforts by 50%," he adds. With centralized metrics monitoring and log aggregation provided by Prometheus on Fluentd, teams are saving 50% of time spent on troubleshooting and debugging. Harbor replication has allowed production/staging/testing environments to cross public cloud and the private Kubernetes cluster to share the same container registry, resulting in 30% savings of CI/CD efforts. Plus, Ji says, "Helm has accelerated prototype development and environment setup with its rich sharing charts."

-
-
-
-
- Thus Slamtec decided to adopt other CNCF projects as well. "We built a monitoring and logging system based on Prometheus and Fluentd," says Ji. "The integration between Prometheus/Fluentd and Kubernetes is convenient, with multiple dimensions of data monitoring and log collection capabilities." -

- The company uses Harbor as a container image repository. "Harbor’s replication function helps us implement CI/CD on both private and public clouds," says Ji. "In addition, multi-project support, certification and policy configuration, and integration with Kubernetes are also excellent functions." Helm is also being used as a package manager, and the team is evaluating the Istio framework. "We’re very pleased that Kubernetes and these frameworks can be seamlessly integrated," Ji adds. -
-
-
-
- "Cloud native is suitable for microservice architecture, it’s suitable for fast iteration and agile development, and it has a relatively perfect ecosystem and active community."

- BENNIU JI, DIRECTOR OF CLOUD COMPUTING BUSINESS DIVISION -
-
+{{< case-studies/quote author="BENNIU JI, DIRECTOR OF CLOUD COMPUTING BUSINESS DIVISION" >}} +"Cloud native technology helps us ensure high availability of our business, while improving development and testing efficiency, shortening the research and development cycle and enabling rapid product delivery." +{{< /case-studies/quote >}} -
-
- With the new platform, Ji reports that Slamtec has experienced "18+ months of 100% stability!" For users, there is now zero service downtime and seamless upgrades. "We benefit from the abstraction of Kubernetes from network and storage," says Ji. "The dependence on external services can be decoupled from the service and placed under unified management in the cluster." -

- Using Kubernetes and Istio "significantly reduced the microservice configuration and maintenance efforts by 50%," he adds. With centralized metrics monitoring and log aggregation provided by Prometheus on Fluentd, teams are saving 50% of time spent on troubleshooting and debugging. Harbor replication has allowed production/staging/testing environments to cross public cloud and the private Kubernetes cluster to share the same container registry, resulting in 30% savings of CI/CD efforts. Plus, Ji adds, "Helm has accelerated prototype development and environment setup with its rich sharing charts." -

-In short, Ji says, Slamtec’s new platform is helping it achieve one of its primary goals: the quick and easy release of products. With multiple release models and a centralized control interface, the platform is changing developers’ lives for the better. Slamtec also offers a unified API for the development of automated deployment tools according to users’ specific needs. -
+{{< case-studies/lead >}} +Founded in 2013, Slamtec provides service robot autonomous localization and navigation solutions. In this fast-moving space, the company built its success on the ability of its R&D team to quickly introduce, and continually iterate on, its core products. +{{< /case-studies/lead >}} -
-
-"We benefit from the abstraction of Kubernetes from network and storage, the dependence on external services can be decoupled from the service and placed under unified management in the cluster."

- BENNIU JI, DIRECTOR OF CLOUD COMPUTING BUSINESS DIVISION
-
+

To sustain that development velocity, the company over the past few years began looking to build its own stable and reliable container cloud platform to host its Internet of Things applications. With a legacy infrastructure based on Alibaba Cloud and VMware vSphere, Slamtec teams had already adopted microservice architecture and continuous delivery, for "fine granularity on-demand scaling, fault isolation, ease of development, testing, and deployment, and for facilitating high-speed iteration," says Benniu Ji, Director of Cloud Computing Business Division. So "our needs for the cloud platform included high availability, scalability and security; multi-granularity monitoring alarm capability; friendliness to containers and microservices; and perfect CI/CD support."

-
- Given its own success with cloud native, Slamtec has just one piece of advice for organizations considering making the leap. "For already containerized services, you should migrate them to the cloud native architecture as soon as possible and enjoy the advantages brought by the cloud native ecosystem," Ji says. "To migrate traditional, non-containerized services, in addition to the architecture changes of the service itself, you need to fully consider the operation and maintenance workload required to build the cloud native architecture." -

- That said, the cost-benefit analysis has been simple for Slamtec. "Cloud native technology is suitable for microservice architecture, it’s suitable for fast iteration and agile development, and it has a relatively perfect ecosystem and active community," says Ji. "It helps us ensure high availability of our business, while improving development and testing efficiency, shortening the research and development cycle and enabling rapid product delivery." -
-
+

After an evaluation of existing technologies, Ji's team chose Kubernetes for orchestration. "CNCF brings quality assurance and a complete ecosystem for Kubernetes, which is very important for the wide application of Kubernetes," says Ji. Plus, "avoiding binding to an infrastructure technology or provider can help us ensure that our business is deployed and migrated in cross-regional environments, and can serve users all over the world."

+ +{{< case-studies/quote + image="/images/case-studies/slamtec/banner3.jpg" + author="BENNIU JI, DIRECTOR OF CLOUD COMPUTING BUSINESS DIVISION" +>}} +"CNCF brings quality assurance and a complete ecosystem for Kubernetes, which is very important for the wide application of Kubernetes." +{{< /case-studies/quote >}} + +

Thus Slamtec decided to adopt other CNCF projects as well. "We built a monitoring and logging system based on Prometheus and Fluentd," says Ji. "The integration between Prometheus/Fluentd and Kubernetes is convenient, with multiple dimensions of data monitoring and log collection capabilities."

+ +

The company uses Harbor as a container image repository. "Harbor's replication function helps us implement CI/CD on both private and public clouds," says Ji. "In addition, multi-project support, certification and policy configuration, and integration with Kubernetes are also excellent functions." Helm is also being used as a package manager, and the team is evaluating the Istio framework. "We're very pleased that Kubernetes and these frameworks can be seamlessly integrated," Ji adds.

+ +{{< case-studies/quote + image="/images/case-studies/slamtec/banner4.jpg" + author="BENNIU JI, DIRECTOR OF CLOUD COMPUTING BUSINESS DIVISION" +>}} +"Cloud native is suitable for microservice architecture, it's suitable for fast iteration and agile development, and it has a relatively perfect ecosystem and active community." +{{< /case-studies/quote >}} + +

With the new platform, Ji reports that Slamtec has experienced "18+ months of 100% stability!" For users, there is now zero service downtime and seamless upgrades. "We benefit from the abstraction of Kubernetes from network and storage," says Ji. "The dependence on external services can be decoupled from the service and placed under unified management in the cluster."

+ +

Using Kubernetes and Istio "significantly reduced the microservice configuration and maintenance efforts by 50%," he adds. With centralized metrics monitoring and log aggregation provided by Prometheus on Fluentd, teams are saving 50% of time spent on troubleshooting and debugging. Harbor replication has allowed production/staging/testing environments to cross public cloud and the private Kubernetes cluster to share the same container registry, resulting in 30% savings of CI/CD efforts. Plus, Ji adds, "Helm has accelerated prototype development and environment setup with its rich sharing charts."

+ +

In short, Ji says, Slamtec's new platform is helping it achieve one of its primary goals: the quick and easy release of products. With multiple release models and a centralized control interface, the platform is changing developers' lives for the better. Slamtec also offers a unified API for the development of automated deployment tools according to users' specific needs.

+ +{{< case-studies/quote author="BENNIU JI, DIRECTOR OF CLOUD COMPUTING BUSINESS DIVISION" >}} +"We benefit from the abstraction of Kubernetes from network and storage, the dependence on external services can be decoupled from the service and placed under unified management in the cluster." +{{< /case-studies/quote >}} + +

Given its own success with cloud native, Slamtec has just one piece of advice for organizations considering making the leap. "For already containerized services, you should migrate them to the cloud native architecture as soon as possible and enjoy the advantages brought by the cloud native ecosystem," Ji says. "To migrate traditional, non-containerized services, in addition to the architecture changes of the service itself, you need to fully consider the operation and maintenance workload required to build the cloud native architecture."

+ +

That said, the cost-benefit analysis has been simple for Slamtec. "Cloud native technology is suitable for microservice architecture, it's suitable for fast iteration and agile development, and it has a relatively perfect ecosystem and active community," says Ji. "It helps us ensure high availability of our business, while improving development and testing efficiency, shortening the research and development cycle and enabling rapid product delivery."

diff --git a/content/en/case-studies/slingtv/index.html b/content/en/case-studies/slingtv/index.html index 349ed8c2de..6de46f35ff 100644 --- a/content/en/case-studies/slingtv/index.html +++ b/content/en/case-studies/slingtv/index.html @@ -3,108 +3,77 @@ title: SlingTV Case Study linkTitle: Sling TV case_study_styles: true cid: caseStudies -css: /css/style_case_studies.css featured: true weight: 49 quote: > I would almost be so bold as to say that most of these applications that we are building now would not have been possible without the cloud native patterns and the flexibility that Kubernetes enables. + +new_case_study_styles: true +heading_background: /images/case-studies/slingtv/banner1.jpg +heading_title_logo: /images/slingtv_logo.png +subheading: > + Sling TV: Marrying Kubernetes and AI to Enable Proper Web Scale +case_study_details: + - Company: Sling TV + - Location: Englewood, Colorado + - Industry: Streaming television --- +

Challenge

-
-

CASE STUDY:
Sling TV: Marrying Kubernetes and AI to Enable Proper Web Scale +

Launched by DISH Network in 2015, Sling TV experienced great customer growth from the beginning. After just a year, "we were going through some growing pains of some of the legacy systems and trying to find the right architecture to enable our future," says Brad Linder, Sling TV's Cloud Native & Big Data Evangelist. The company has particular challenges: "We take live TV and distribute it over the internet out to a user's device that we do not control," says Linder. "In a lot of ways, we are working in the Wild West: The internet is what it is going to be, and if a customer's service does not work for whatever reason, they do not care why. They just want things to work. Those are the variables of the equation that we have to try to solve. We really have to try to enable optionality and good customer experience at web scale."

-

+

Solution

-
- -
- Company  Sling TV     Location  Englewood, Colorado     Industry  Streaming television -
- -
-
-
-
-

Challenge

- Launched by DISH Network in 2015, Sling TV experienced great customer growth from the beginning. After just a year, “we were going through some growing pains of some of the legacy systems and trying to find the right architecture to enable our future,” says Brad Linder, Sling TV’s Cloud Native & Big Data Evangelist. The company has particular challenges: “We take live TV and distribute it over the internet out to a user’s device that we do not control,” says Linder. “In a lot of ways, we are working in the Wild West: The internet is what it is going to be, and if a customer’s service does not work for whatever reason, they do not care why. They just want things to work. Those are the variables of the equation that we have to try to solve. We really have to try to enable optionality and good customer experience at web scale.” - -

- -

Solution

- Led by the belief that “the cloud native architectures and patterns really give us a lot of flexibility in meeting the needs of that sort of customer base,” Linder partnered with Rancher Labs to build Sling TV’s next-generation platform around Kubernetes. “We are going to need to enable a hybrid cloud strategy including multiple public clouds and an on-premise VMWare multi data center environment to meet the needs of the business at some point, so getting that sort of abstraction was a real goal,” he says. “That is one of the biggest reasons why we picked Kubernetes.” The team launched its first applications on Kubernetes in Sling TV’s two internal data centers. The push to enable AWS as a data center option is underway and should be available by the end of 2018. The team has added Prometheus for monitoring and Jaeger for tracing, to work alongside the company’s existing tool sets: Zenoss, New Relic and ELK. - -
-

-
+

Led by the belief that "the cloud native architectures and patterns really give us a lot of flexibility in meeting the needs of that sort of customer base," Linder partnered with Rancher Labs to build Sling TV's next-generation platform around Kubernetes. "We are going to need to enable a hybrid cloud strategy including multiple public clouds and an on-premise VMWare multi data center environment to meet the needs of the business at some point, so getting that sort of abstraction was a real goal," he says. "That is one of the biggest reasons why we picked Kubernetes." The team launched its first applications on Kubernetes in Sling TV's two internal data centers. The push to enable AWS as a data center option is underway and should be available by the end of 2018. The team has added Prometheus for monitoring and Jaeger for tracing, to work alongside the company's existing tool sets: Zenoss, New Relic and ELK.

Impact

- “We are getting to the place where we can one-click deploy an entire data center – the compute, network, Kubernetes, logging, monitoring and all the apps,” says Linder. “We have really enabled a platform thinking based approach to allowing applications to consume common tools. A new application can be onboarded in about an hour using common tooling and CI/CD processes. The gains on that side have been huge. Before, it took at least a few days to get things sorted for a new application to deploy. That does not consider the training of our operations staff to manage this new application. It is two or three orders of magnitude of savings in time and cost, and operationally it has given us the opportunity to let a core team of talented operations engineers manage common infrastructure and tooling to make our applications available at web scale.” + +

"We are getting to the place where we can one-click deploy an entire data center – the compute, network, Kubernetes, logging, monitoring and all the apps," says Linder. "We have really enabled a platform thinking based approach to allowing applications to consume common tools. A new application can be onboarded in about an hour using common tooling and CI/CD processes. The gains on that side have been huge. Before, it took at least a few days to get things sorted for a new application to deploy. That does not consider the training of our operations staff to manage this new application. It is two or three orders of magnitude of savings in time and cost, and operationally it has given us the opportunity to let a core team of talented operations engineers manage common infrastructure and tooling to make our applications available at web scale."

+ +{{< case-studies/quote author="Brad Linder, Cloud Native & Big Data Evangelist for Sling TV" >}} +"I would almost be so bold as to say that most of these applications that we are building now would not have been possible without the cloud native patterns and the flexibility that Kubernetes enables." +{{< /case-studies/quote >}} + +{{< case-studies/lead >}} +The beauty of streaming television, like the service offered by Sling TV, is that you can watch it from any device you want, wherever you want. +{{< /case-studies/lead >}} -
+

Of course, from the provider side of things, that creates a particular set of challenges "We take live TV and distribute it over the internet out to a user's device that we do not control," says Brad Linder, Sling TV's Cloud Native & Big Data Evangelist. "In a lot of ways, we are working in the Wild West: The internet is what it is going to be, and if a customer's service does not work for whatever reason, they do not care why. They just want things to work. Those are the variables of the equation that we have to try to solve. We really have to try to enable optionality and we have to do it at web scale."

-
-
-
-
- “I would almost be so bold as to say that most of these applications that we are building now would not have been possible without the cloud native patterns and the flexibility that Kubernetes enables.”

— Brad Linder, Cloud Native & Big Data Evangelist for Sling TV +

Indeed, Sling TV experienced great customer growth from the beginning of its launch by DISH Network in 2015. After just a year, "we were going through some growing pains of some of the legacy systems and trying to find the right architecture to enable our future," says Linder. Tasked with building a next-generation web scale platform for the "personalized customer experience," Linder has spent the past year bringing Kubernetes to Sling TV.

-
-
-
-
-

The beauty of streaming television, like the service offered by Sling TV, is that you can watch it from any device you want, wherever you want.

Of course, from the provider side of things, that creates a particular set of challenges -“We take live TV and distribute it over the internet out to a user’s device that we do not control,” says Brad Linder, Sling TV’s Cloud Native & Big Data Evangelist. “In a lot of ways, we are working in the Wild West: The internet is what it is going to be, and if a customer’s service does not work for whatever reason, they do not care why. They just want things to work. Those are the variables of the equation that we have to try to solve. We really have to try to enable optionality and we have to do it at web scale.”

-Indeed, Sling TV experienced great customer growth from the beginning of its launch by DISH Network in 2015. After just a year, “we were going through some growing pains of some of the legacy systems and trying to find the right architecture to enable our future,” says Linder. Tasked with building a next-generation web scale platform for the “personalized customer experience,” Linder has spent the past year bringing Kubernetes to Sling TV.

-Led by the belief that “the cloud native architectures and patterns really give us a lot of flexibility in meeting the needs of our customers,” Linder partnered with Rancher Labs to build the platform around Kubernetes. “They have really helped us get our head around how to use Kubernetes,” he says. “We needed the flexibility to enable our use case versus just a simple orchestrater. Enabling our future in a way that did not give us vendor lock-in was also a key part of our strategy. I think that is part of the Rancher value proposition.” +

Led by the belief that "the cloud native architectures and patterns really give us a lot of flexibility in meeting the needs of our customers," Linder partnered with Rancher Labs to build the platform around Kubernetes. "They have really helped us get our head around how to use Kubernetes," he says. "We needed the flexibility to enable our use case versus just a simple orchestrater. Enabling our future in a way that did not give us vendor lock-in was also a key part of our strategy. I think that is part of the Rancher value proposition."

+{{< case-studies/quote + image="/images/case-studies/slingtv/banner3.jpg" + author="Brad Linder, Cloud Native & Big Data Evangelist for Sling TV" +>}} +"We needed the flexibility to enable our use case versus just a simple orchestrater. Enabling our future in a way that did not give us vendor lock-in was also a key part of our strategy. I think that is part of the Rancher value proposition." +{{< /case-studies/quote >}} -
-
-
-
- “We needed the flexibility to enable our use case versus just a simple orchestrater. Enabling our future in a way that did not give us vendor lock-in was also a key part of our strategy. I think that is part of the Rancher value proposition.”

— Brad Linder, Cloud Native & Big Data Evangelist for Sling TV -
-
-
-
+

One big reason he chose Kubernetes was getting a level of abstraction that would enable the company to "enable a hybrid cloud strategy including multiple public clouds and an on-premise VMWare multi data center environment to meet the needs of the business," he says. Another factor was how much the Kubernetes ecosystem has matured over the past couple of years. "We have spent a lot of time and energy around making logging, monitoring and alerting production ready to give us insights into applications' well-being," says Linder. The team has added Prometheus for monitoring and Jaeger for tracing, to work alongside the company's existing tool sets: Zenoss, New Relic and ELK.

-One big reason he chose Kubernetes was getting a level of abstraction that would enable the company to “enable a hybrid cloud strategy including multiple public clouds and an on-premise VMWare multi data center environment to meet the needs of the business,” he says. Another factor was how much the Kubernetes ecosystem has matured over the past couple of years. “We have spent a lot of time and energy around making logging, monitoring and alerting production ready to give us insights into applications’ well-being,” says Linder. The team has added Prometheus for monitoring and Jaeger for tracing, to work alongside the company’s existing tool sets: Zenoss, New Relic and ELK.

-With the emphasis on common tooling, “We are getting to the place where we can one-click deploy an entire data center – the compute, network, Kubernetes, logging, monitoring and all the apps,” says Linder. “We have really enabled a platform thinking based approach to allowing applications to consume common tools and services. A new application can be onboarded in about an hour using common tooling and CI/CD processes. The gains on that side have been huge. Before, it took at least a few days to get things sorted for a new application to deploy. That does not consider the training of our operations staff to manage this new application. It is two or three orders of magnitude of savings in time and cost, and operationally it has given us the opportunity to let a core team of talented operations engineers manage common infrastructure and tooling to make our applications available at web scale.”

+

With the emphasis on common tooling, "We are getting to the place where we can one-click deploy an entire data center – the compute, network, Kubernetes, logging, monitoring and all the apps," says Linder. "We have really enabled a platform thinking based approach to allowing applications to consume common tools and services. A new application can be onboarded in about an hour using common tooling and CI/CD processes. The gains on that side have been huge. Before, it took at least a few days to get things sorted for a new application to deploy. That does not consider the training of our operations staff to manage this new application. It is two or three orders of magnitude of savings in time and cost, and operationally it has given us the opportunity to let a core team of talented operations engineers manage common infrastructure and tooling to make our applications available at web scale."

-
-
-
-
-“We have to be able to react to changes and hiccups in the matrix. It is the foundation for our ability to deliver a high-quality service for our customers."

— Brad Linder, Cloud Native & Big Data Evangelist for Sling TV -
-
+{{< case-studies/quote + image="/images/case-studies/slingtv/banner4.jpg" + author="Brad Linder, Cloud Native & Big Data Evangelist for Sling TV" +>}} +"We have to be able to react to changes and hiccups in the matrix. It is the foundation for our ability to deliver a high-quality service for our customers." +{{< /case-studies/quote >}} -
+

The team launched its first applications on Kubernetes in Sling TV's two internal data centers in the early part of Q1 2018 and began to enable AWS as a data center option. The company plans to expand into other public clouds in the future.

-
- The team launched its first applications on Kubernetes in Sling TV’s two internal data centers in the early part of Q1 2018 and began to enable AWS as a data center option. The company plans to expand into other public clouds in the future. -The first application that went into production is a web socket-based back-end notification service. “It allows back-end changes to trigger messages to our clients in the field without the polling,” says Linder. “We are talking about very high volumes of messages with this application. Without something like Kubernetes to be able to scale up and down, as well as just support that overall workload, that is pretty hard to do. I would almost be so bold as to say that most of these applications that we are building now would not have been possible without the cloud native patterns and the flexibility that Kubernetes enables.”

- Linder oversees three teams working together on building the next-generation platform: a platform engineering team; an enterprise middleware services team; and a big data and analytics team. “We have really tried to bring everything together to be able to have a client application interact with a cloud native middleware layer. That middleware layer must run on a platform, consume platform services and then have logs and events monitored by an artificial agent to keep things running smoothly,” says Linder. +

The first application that went into production is a web socket-based back-end notification service. "It allows back-end changes to trigger messages to our clients in the field without the polling," says Linder. "We are talking about very high volumes of messages with this application. Without something like Kubernetes to be able to scale up and down, as well as just support that overall workload, that is pretty hard to do. I would almost be so bold as to say that most of these applications that we are building now would not have been possible without the cloud native patterns and the flexibility that Kubernetes enables."

+

Linder oversees three teams working together on building the next-generation platform: a platform engineering team; an enterprise middleware services team; and a big data and analytics team. "We have really tried to bring everything together to be able to have a client application interact with a cloud native middleware layer. That middleware layer must run on a platform, consume platform services and then have logs and events monitored by an artificial agent to keep things running smoothly," says Linder.

-
+{{< case-studies/quote author="BRAD LINDER, CLOUD NATIVE & BIG DATA EVANGELIST FOR SLING TV">}} +This undertaking is about "trying to marry Kubernetes with AI to enable web scale that just works". +{{< /case-studies/quote >}} -
-
- This undertaking is about “trying to marry Kubernetes with AI to enable web scale that just works".

— BRAD LINDER, CLOUD NATIVE & BIG DATA EVANGELIST FOR SLING TV -
-
+

Ultimately, this undertaking is about "trying to marry Kubernetes with AI to enable web scale that just works," he adds. "We want the artificial agents and the big data platform using the actual logs and events coming out of the applications, Kubernetes, the infrastructure, backing services and changes to the environment to make decisions like, 'Hey we need more capacity for this service so please add more nodes.' From a platform perspective, if you are truly doing web scale stuff and you are not using AI and big data, in my opinion, you are going to implode under your own weight. It is not a question of if, it is when. If you are in a 'millions of users' sort of environment, that implosion is going to be catastrophic. We are on our way to this goal and have learned a lot along the way."

-
- Ultimately, this undertaking is about “trying to marry Kubernetes with AI to enable web scale that just works,” he adds. “We want the artificial agents and the big data platform using the actual logs and events coming out of the applications, Kubernetes, the infrastructure, backing services and changes to the environment to make decisions like, ‘Hey we need more capacity for this service so please add more nodes.’ From a platform perspective, if you are truly doing web scale stuff and you are not using AI and big data, in my opinion, you are going to implode under your own weight. It is not a question of if, it is when. If you are in a ‘millions of users’ sort of environment, that implosion is going to be catastrophic. We are on our way to this goal and have learned a lot along the way.”

-For Sling TV, moving to cloud native has been exactly what they needed. “We have to be able to react to changes and hiccups in the matrix,” says Linder. “It is the foundation for our ability to deliver a high-quality service for our customers. Building intelligent platforms, tools and clients in the field consuming those services has got to be part of all of this. In my eyes that is a big part of what cloud native is all about. It is taking these distributed, potentially unreliable entities and enabling a robust customer experience they expect.” - - - - - -
- -
+

For Sling TV, moving to cloud native has been exactly what they needed. "We have to be able to react to changes and hiccups in the matrix," says Linder. "It is the foundation for our ability to deliver a high-quality service for our customers. Building intelligent platforms, tools and clients in the field consuming those services has got to be part of all of this. In my eyes that is a big part of what cloud native is all about. It is taking these distributed, potentially unreliable entities and enabling a robust customer experience they expect."

diff --git a/content/en/case-studies/sos/index.html b/content/en/case-studies/sos/index.html index becf486413..ff0e3f2322 100644 --- a/content/en/case-studies/sos/index.html +++ b/content/en/case-studies/sos/index.html @@ -3,97 +3,81 @@ title: SOS International Case Study linkTitle: SOS International case_study_styles: true cid: caseStudies -css: /css/style_case_studies.css logo: sos_featured_logo.png + +new_case_study_styles: true +heading_background: /images/case-studies/sos/banner1.jpg +heading_title_logo: /images/sos_logo.png +subheading: > + SOS International: Using Kubernetes to Provide Emergency Assistance in a Connected World +case_study_details: + - Company: SOS International + - Location: Frederiksberg, Denmark + - Industry: Medical and Travel Assistance --- +

Challenge

-
-

CASE STUDY:
SOS International: Using Kubernetes to Provide Emergency Assistance in a Connected World +

For the past six decades, SOS International has been providing reliable medical and travel assistance in the Nordic region. In recent years, the company's business strategy has required increasingly intense development in the digital space, but when it came to its IT systems, "SOS has a very fragmented legacy," with three traditional monoliths (Java, .NET, and IBM's AS/400) and a waterfall approach, says Martin Ahrentsen, Head of Enterprise Architecture. "We have been forced to institute both new technology and new ways of working, so we could be more efficient with a shorter time to market. It was a much more agile approach, and we needed to have a platform that can help us deliver that to the business."

-

+

Solution

-
+

After an unsuccessful search for a standard system, the company decided to take a platform approach and look for a solution that rolls up Kubernetes and the container technology. RedHat OpenShift proved to be a perfect fit for SOS's fragmented systems. "We have a lot of different technologies that we use, both code languages and others, and all of them could use the resources on the new platform," says Ahrentsen. Of the company's three monoliths, "we can provide this new bleeding edge technology to two of them (.NET and Java)." The platform went live in the spring of 2018; there are now six greenfield projects based on microservices architecture underway, plus all of the company's Java applications are currently going through a "lift and shift" migration.

-
- Company  SOS International     Location  Frederiksberg, Denmark -     Industry  Medical and Travel Assistance -
- -
-
-
-
-

Challenge

- For the past six decades, SOS International has been providing reliable medical and travel assistance in the Nordic region. In recent years, the company’s business strategy has required increasingly intense development in the digital space, but when it came to its IT systems, "SOS has a very fragmented legacy," with three traditional monoliths (Java, .NET, and IBM’s AS/400) and a waterfall approach, says Martin Ahrentsen, Head of Enterprise Architecture. "We have been forced to institute both new technology and new ways of working, so we could be more efficient with a shorter time to market. It was a much more agile approach, and we needed to have a platform that can help us deliver that to the business." - -

-

Solution

- After an unsuccessful search for a standard system, the company decided to take a platform approach and look for a solution that rolls up Kubernetes and the container technology. RedHat OpenShift proved to be a perfect fit for SOS’s fragmented systems. "We have a lot of different technologies that we use, both code languages and others, and all of them could use the resources on the new platform," says Ahrentsen. Of the company’s three monoliths, "we can provide this new bleeding edge technology to two of them (.NET and Java)." The platform went live in the spring of 2018; there are now six greenfield projects based on microservices architecture underway, plus all of the company’s Java applications are currently going through a "lift and shift" migration. - -

Impact

- Kubernetes has delivered "improved time to market, agility, and the ability to adapt to changes and new technologies," says Ahrentsen. "Just the time between when the software is ready for release and when it can be released has dramatically been improved." The way of thinking at SOS International has also changed for the better: "Since we have Kubernetes and easy access to scripts that can help us automate, creating CI/CD pipelines easily, that has spawned a lot of internal interest in how to do this fully automated, all the way. It creates a very good climate in order to start the journey," he says. Moreover, being part of the cloud native community has helped the company attract talent. "They want to work with the cool, new technologies," says Ahrentsen. "During our onboarding, we could see that we were chosen by IT professionals because we provided the new technologies." -
+

Kubernetes has delivered "improved time to market, agility, and the ability to adapt to changes and new technologies," says Ahrentsen. "Just the time between when the software is ready for release and when it can be released has dramatically been improved." The way of thinking at SOS International has also changed for the better: "Since we have Kubernetes and easy access to scripts that can help us automate, creating CI/CD pipelines easily, that has spawned a lot of internal interest in how to do this fully automated, all the way. It creates a very good climate in order to start the journey," he says. Moreover, being part of the cloud native community has helped the company attract talent. "They want to work with the cool, new technologies," says Ahrentsen. "During our onboarding, we could see that we were chosen by IT professionals because we provided the new technologies."

-
-
-
-
- "The speed of the changes that cloud native software and technologies drive right now is amazing, and following and adopting it is very crucial for us. The amazing technology provided by Kubernetes and cloud native has started the change for SOS towards a digital future." -

- Martin Ahrentsen, Head of Enterprise Architecture, SOS International -
-
-
-
-

For six decades, SOS International has provided reliable emergency medical and travel assistance for customers in the Nordic countries.

- SOS operators handle a million cases and over a million phone calls a year. But in the past four years, the company’s business strategy has required increasingly intense development in the digital space.

- When it comes to its IT systems, "SOS has a very fragmented legacy," with three traditional monoliths running in the company’s own data centers and a waterfall approach, says Martin Ahrentsen, Head of Enterprise Architecture. "We had to institute both new technology and new ways of working so we could be more efficient, with a shorter time to market. It was a much more agile approach, and we needed to have a platform that can help us deliver that to the business."

- For a long time, Ahrentsen and his team searched for a standard solution that could work at SOS. "There aren’t that many assistance companies like us, so you cannot get a standard system that fits for that; there is no perfect match," he says. "We would have to take a standard system and twist it too much so it is not standard anymore. Based on that, we decided to find a technology platform instead, with some common components that we could use to build the new digital systems and core systems." +{{< case-studies/quote author="Martin Ahrentsen, Head of Enterprise Architecture, SOS International" >}} +"The speed of the changes that cloud native software and technologies drive right now is amazing, and following and adopting it is very crucial for us. The amazing technology provided by Kubernetes and cloud native has started the change for SOS towards a digital future." +{{< /case-studies/quote >}} +{{< case-studies/lead >}} +For six decades, SOS International has provided reliable emergency medical and travel assistance for customers in the Nordic countries. +{{< /case-studies/lead >}} +

SOS operators handle a million cases and over a million phone calls a year. But in the past four years, the company's business strategy has required increasingly intense development in the digital space.

-
-
-
-
- "We have to deliver new digital services, but we also have to migrate the old stuff, and we have to transform our core systems into new systems built on top of this platform. One of the reasons why we chose this technology is that we could build new digital services while changing the old one."

- Martin Ahrentsen, Head of Enterprise Architecture, SOS International +

When it comes to its IT systems, "SOS has a very fragmented legacy," with three traditional monoliths running in the company's own data centers and a waterfall approach, says Martin Ahrentsen, Head of Enterprise Architecture. "We had to institute both new technology and new ways of working so we could be more efficient, with a shorter time to market. It was a much more agile approach, and we needed to have a platform that can help us deliver that to the business."

-
-
-
-
- Sold on what Kubernetes could do, Ahrentsen zeroed in on platforms that could meet the business’s needs right away. The company opted to use RedHat’s OpenShift container platform, which incorporates Docker containers and Kubernetes, as well as a whole stack of technologies, including RedHat Hyperconverged Infrastructure and some midware components, all from the open source community.

- Based on the company’s criteria—technology fit, agility fit, legal requirements, and competencies—the OpenShift solution seemed like a perfect fit for SOS’s fragmented systems. "We have a lot of different technologies that we use, both code languages and others, and all of them could use the resources on the new platform," says Ahrentsen. Of the company’s three monoliths, "we can provide this new bleeding edge technology to two of them (.NET and Java)."

- The platform went live in the spring of 2018; six greenfield projects based on microservices architecture were initially launched, plus all of the company’s Java applications are currently going through a "lift and shift" migration. One of the first Kubernetes-based projects to go live is Remote Medical Treatment, a solution in which customers can contact the SOS alarm center via voice, chat, or video. "We managed to develop it in quite a short timeframe with focus on full CI/CD pipelining and a modern microservice architecture all running in a dual OpenShift cluster setup," says Ahrentsen. Onsite, which is used for dispatching rescue trucks around the Nordic countries, and Follow Your Truck, which allows customers to track tow trucks, are also being rolled out. +

For a long time, Ahrentsen and his team searched for a standard solution that could work at SOS. "There aren't that many assistance companies like us, so you cannot get a standard system that fits for that; there is no perfect match," he says. "We would have to take a standard system and twist it too much so it is not standard anymore. Based on that, we decided to find a technology platform instead, with some common components that we could use to build the new digital systems and core systems."

-
-
-
-
- "During our onboarding, we could see that we were chosen by IT professionals because we provided the new technologies."

- Martin Ahrentsen, Head of Enterprise Architecture, SOS International -
-
+{{< case-studies/quote + image="/images/case-studies/sos/banner3.jpg" + author="Martin Ahrentsen, Head of Enterprise Architecture, SOS International" +>}} +"We have to deliver new digital services, but we also have to migrate the old stuff, and we have to transform our core systems into new systems built on top of this platform. One of the reasons why we chose this technology is that we could build new digital services while changing the old one." +{{< /case-studies/quote >}} -
-
- The platform is still running on premise, because some of SOS’s customers in the insurance industry, for whom the company handles data, don’t yet have a cloud strategy. Kubernetes is allowing SOS to start in the data center and move to the cloud when the business is ready. "Over the next three to five years, all of them will have a strategy, and we could probably take the data and go to the cloud," says Ahrentsen. There’s also the possibility of moving to a hybrid cloud setup for sensitive and non-sensitive data.

- SOS’s technology is certainly in a state of transition. "We have to deliver new digital services, but we also have to migrate the old stuff, and we have to transform our core systems into new systems built on top of this platform," says Ahrentsen. "One of the reasons why we chose this technology is that we could build new digital services while changing the old one."

- But already, Kubernetes has delivered improved time to market, as evidenced by how quickly the greenfield projects were developed and released. "Just the time between when the software is ready for release and when it can be released has dramatically been improved," says Ahrentsen.

- Moreover, being part of the cloud native community has helped the company attract talent as it pursues a goal of growing the ranks of engineers, operators, and architects from 60 to 100 this year. "They want to work with the cool, new technologies," says Ahrentsen. "During our onboarding, we could see that we were chosen by IT professionals because we provided the new technologies." +

Sold on what Kubernetes could do, Ahrentsen zeroed in on platforms that could meet the business's needs right away. The company opted to use RedHat's OpenShift container platform, which incorporates Docker containers and Kubernetes, as well as a whole stack of technologies, including RedHat Hyperconverged Infrastructure and some midware components, all from the open source community.

-
+

Based on the company's criteria—technology fit, agility fit, legal requirements, and competencies—the OpenShift solution seemed like a perfect fit for SOS's fragmented systems. "We have a lot of different technologies that we use, both code languages and others, and all of them could use the resources on the new platform," says Ahrentsen. Of the company's three monoliths, "we can provide this new bleeding edge technology to two of them (.NET and Java)."

-
-
-"The future world where everything is connected and sends data will create a big potential for us in terms of new market opportunities. But it will also set a big demand on the IT platform and what we need to deliver."

- Martin Ahrentsen, Head of Enterprise Architecture, SOS International
-
+

The platform went live in the spring of 2018; six greenfield projects based on microservices architecture were initially launched, plus all of the company's Java applications are currently going through a "lift and shift" migration. One of the first Kubernetes-based projects to go live is Remote Medical Treatment, a solution in which customers can contact the SOS alarm center via voice, chat, or video. "We managed to develop it in quite a short timeframe with focus on full CI/CD pipelining and a modern microservice architecture all running in a dual OpenShift cluster setup," says Ahrentsen. Onsite, which is used for dispatching rescue trucks around the Nordic countries, and Follow Your Truck, which allows customers to track tow trucks, are also being rolled out.

-
- The way of thinking at SOS International has also changed dramatically: "Since we have Kubernetes and easy access to scripts that can help us automate, creating CI/CD pipelines easily, that has spawned a lot of internal interest in how to do this fully automated, all the way. It creates a very good climate in order to start the journey."

- For this journey at SOS, digitalization and optimization are the key words. "For IT to deliver this, we need to improve, and that is not just on the way of using Kubernetes and the platform," says Ahrentsen. "It’s also a way of building the systems to be ready for automation, and afterwards, machine learning and other interesting technologies that are on the way."

- Case in point: the introduction of the internet of things into automobiles. The European Commission now mandates all new cars to be equipped with eCall, which transmits location and other data in case of a serious traffic accident. SOS provides this service as smart auto assistance. "We receive the call and find out if an emergency response team needs to be sent, or if it’s not heavy impact," says Ahrentsen. "The future world where everything is connected and sends data will create a big potential for us in terms of new market opportunities. But it will also set a big demand on the IT platform and what we need to deliver."

- Ahrentsen feels that SOS is well equipped for the challenge, given the technology choices the company has made. "The speed of the changes that cloud native software and technologies drive right now is amazing, and following it and adopting it is very crucial for us," he says. "The amazing technology provided by Kubernetes and cloud native has started the change for SOS towards a digital future." -
-
+{{< case-studies/quote + image="/images/case-studies/sos/banner4.jpg" + author="Martin Ahrentsen, Head of Enterprise Architecture, SOS International" +>}} +"During our onboarding, we could see that we were chosen by IT professionals because we provided the new technologies." +{{< /case-studies/quote >}} + +

The platform is still running on premise, because some of SOS's customers in the insurance industry, for whom the company handles data, don't yet have a cloud strategy. Kubernetes is allowing SOS to start in the data center and move to the cloud when the business is ready. "Over the next three to five years, all of them will have a strategy, and we could probably take the data and go to the cloud," says Ahrentsen. There's also the possibility of moving to a hybrid cloud setup for sensitive and non-sensitive data.

+ +

SOS's technology is certainly in a state of transition. "We have to deliver new digital services, but we also have to migrate the old stuff, and we have to transform our core systems into new systems built on top of this platform," says Ahrentsen. "One of the reasons why we chose this technology is that we could build new digital services while changing the old one."

+ +

But already, Kubernetes has delivered improved time to market, as evidenced by how quickly the greenfield projects were developed and released. "Just the time between when the software is ready for release and when it can be released has dramatically been improved," says Ahrentsen.

+ +

Moreover, being part of the cloud native community has helped the company attract talent as it pursues a goal of growing the ranks of engineers, operators, and architects from 60 to 100 this year. "They want to work with the cool, new technologies," says Ahrentsen. "During our onboarding, we could see that we were chosen by IT professionals because we provided the new technologies."

+ +{{< case-studies/quote author="Martin Ahrentsen, Head of Enterprise Architecture, SOS International" >}} +"The future world where everything is connected and sends data will create a big potential for us in terms of new market opportunities. But it will also set a big demand on the IT platform and what we need to deliver." +{{< /case-studies/quote >}} + +

The way of thinking at SOS International has also changed dramatically: "Since we have Kubernetes and easy access to scripts that can help us automate, creating CI/CD pipelines easily, that has spawned a lot of internal interest in how to do this fully automated, all the way. It creates a very good climate in order to start the journey."

+ +

For this journey at SOS, digitalization and optimization are the key words. "For IT to deliver this, we need to improve, and that is not just on the way of using Kubernetes and the platform," says Ahrentsen. "It's also a way of building the systems to be ready for automation, and afterwards, machine learning and other interesting technologies that are on the way."

+ +

Case in point: the introduction of the internet of things into automobiles. The European Commission now mandates all new cars to be equipped with eCall, which transmits location and other data in case of a serious traffic accident. SOS provides this service as smart auto assistance. "We receive the call and find out if an emergency response team needs to be sent, or if it's not heavy impact," says Ahrentsen. "The future world where everything is connected and sends data will create a big potential for us in terms of new market opportunities. But it will also set a big demand on the IT platform and what we need to deliver."

+ +

Ahrentsen feels that SOS is well equipped for the challenge, given the technology choices the company has made. "The speed of the changes that cloud native software and technologies drive right now is amazing, and following it and adopting it is very crucial for us," he says. "The amazing technology provided by Kubernetes and cloud native has started the change for SOS towards a digital future."

diff --git a/content/en/case-studies/spotify/index.html b/content/en/case-studies/spotify/index.html index 63243b08f6..7448b40939 100644 --- a/content/en/case-studies/spotify/index.html +++ b/content/en/case-studies/spotify/index.html @@ -3,96 +3,77 @@ title: Spotify Case Study linkTitle: Spotify case_study_styles: true cid: caseStudies -css: /css/style_case_studies.css featured: false + +new_case_study_styles: true +heading_background: /images/case-studies/spotify/banner1.jpg +heading_title_text: Spotify +subheading: > + Spotify: An Early Adopter of Containers, Spotify Is Migrating from Homegrown Orchestration to Kubernetes +case_study_details: + - Company: Spotify + - Location: Global + - Industry: Entertainment --- -
-

CASE STUDY: Spotify
Spotify: An Early Adopter of Containers, Spotify Is Migrating from Homegrown Orchestration to Kubernetes +

Challenge

-

+

Launched in 2008, the audio-streaming platform has grown to over 200 million monthly active users across the world. "Our goal is to empower creators and enable a really immersive listening experience for all of the consumers that we have today—and hopefully the consumers we'll have in the future," says Jai Chakrabarti, Director of Engineering, Infrastructure and Operations. An early adopter of microservices and Docker, Spotify had containerized microservices running across its fleet of VMs with a homegrown container orchestration system called Helios. By late 2017, it became clear that "having a small team working on the features was just not as efficient as adopting something that was supported by a much bigger community," he says.

-
+

Solution

-
- Company  Spotify     Location  Global -     Industry  Entertainment -
+

"We saw the amazing community that had grown up around Kubernetes, and we wanted to be part of that," says Chakrabarti. Kubernetes was more feature-rich than Helios. Plus, "we wanted to benefit from added velocity and reduced cost, and also align with the rest of the industry on best practices and tools." At the same time, the team wanted to contribute its expertise and influence in the flourishing Kubernetes community. The migration, which would happen in parallel with Helios running, could go smoothly because "Kubernetes fit very nicely as a complement and now as a replacement to Helios," says Chakrabarti.

-
-
-
-
-

Challenge

- Launched in 2008, the audio-streaming platform has grown to over 200 million monthly active users across the world. "Our goal is to empower creators and enable a really immersive listening experience for all of the consumers that we have today—and hopefully the consumers we’ll have in the future," says Jai Chakrabarti, Director of Engineering, Infrastructure and Operations. An early adopter of microservices and Docker, Spotify had containerized microservices running across its fleet of VMs with a homegrown container orchestration system called Helios. By late 2017, it became clear that "having a small team working on the features was just not as efficient as adopting something that was supported by a much bigger community," he says. - -

-

Solution

- "We saw the amazing community that had grown up around Kubernetes, and we wanted to be part of that," says Chakrabarti. Kubernetes was more feature-rich than Helios. Plus, "we wanted to benefit from added velocity and reduced cost, and also align with the rest of the industry on best practices and tools." At the same time, the team wanted to contribute its expertise and influence in the flourishing Kubernetes community. The migration, which would happen in parallel with Helios running, could go smoothly because "Kubernetes fit very nicely as a complement and now as a replacement to Helios," says Chakrabarti. -

Impact

- The team spent much of 2018 addressing the core technology issues required for a migration, which started late that year and is a big focus for 2019. "A small percentage of our fleet has been migrated to Kubernetes, and some of the things that we’ve heard from our internal teams are that they have less of a need to focus on manual capacity provisioning and more time to focus on delivering features for Spotify," says Chakrabarti. The biggest service currently running on Kubernetes takes about 10 million requests per second as an aggregate service and benefits greatly from autoscaling, says Site Reliability Engineer James Wen. Plus, he adds, "Before, teams would have to wait for an hour to create a new service and get an operational host to run it in production, but with Kubernetes, they can do that on the order of seconds and minutes." In addition, with Kubernetes’s bin-packing and multi-tenancy capabilities, CPU utilization has improved on average two- to threefold. -
-
-
-
-
- "We saw the amazing community that’s grown up around Kubernetes, and we wanted to be part of that. We wanted to benefit from added velocity and reduced cost, and also align with the rest of the industry on best practices and tools." -

- Jai Chakrabarti, Director of Engineering, Infrastructure and Operations, Spotify -
-
-
-
-

"Our goal is to empower creators and enable a really immersive listening experience for all of the consumers that we have today—and hopefully the consumers we’ll have in the future," says Jai Chakrabarti, Director of Engineering, Infrastructure and Operations at Spotify. Since the audio-streaming platform launched in 2008, it has already grown to over 200 million monthly active users around the world, and for Chakrabarti’s team, the goal is solidifying Spotify’s infrastructure to support all those future consumers too.

-

- An early adopter of microservices and Docker, Spotify had containerized microservices running across its fleet of VMs since 2014. The company used an open source, homegrown container orchestration system called Helios, and in 2016-17 completed a migration from on premise data centers to Google Cloud. Underpinning these decisions, "We have a culture around autonomous teams, over 200 autonomous engineering squads who are working on different pieces of the pie, and they need to be able to iterate quickly," Chakrabarti says. "So for us to have developer velocity tools that allow squads to move quickly is really important."

- But by late 2017, it became clear that "having a small team working on the Helios features was just not as efficient as adopting something that was supported by a much bigger community," says Chakrabarti. "We saw the amazing community that had grown up around Kubernetes, and we wanted to be part of that. We wanted to benefit from added velocity and reduced cost, and also align with the rest of the industry on best practices and tools." At the same time, the team wanted to contribute its expertise and influence in the flourishing Kubernetes community. +

The team spent much of 2018 addressing the core technology issues required for a migration, which started late that year and is a big focus for 2019. "A small percentage of our fleet has been migrated to Kubernetes, and some of the things that we've heard from our internal teams are that they have less of a need to focus on manual capacity provisioning and more time to focus on delivering features for Spotify," says Chakrabarti. The biggest service currently running on Kubernetes takes about 10 million requests per second as an aggregate service and benefits greatly from autoscaling, says Site Reliability Engineer James Wen. Plus, he adds, "Before, teams would have to wait for an hour to create a new service and get an operational host to run it in production, but with Kubernetes, they can do that on the order of seconds and minutes." In addition, with Kubernetes's bin-packing and multi-tenancy capabilities, CPU utilization has improved on average two- to threefold.

+{{< case-studies/quote author="Jai Chakrabarti, Director of Engineering, Infrastructure and Operations, Spotify" >}} +"We saw the amazing community that's grown up around Kubernetes, and we wanted to be part of that. We wanted to benefit from added velocity and reduced cost, and also align with the rest of the industry on best practices and tools." +{{< /case-studies/quote >}} -
-
-
-
- "The community has been extremely helpful in getting us to work through all the technology much faster and much easier. And it’s helped us validate all the things we’re doing."

- Dave Zolotusky, Software Engineer, Infrastructure and Operations, Spotify +{{< case-studies/lead >}} +"Our goal is to empower creators and enable a really immersive listening experience for all of the consumers that we have today—and hopefully the consumers we'll have in the future," says Jai Chakrabarti, Director of Engineering, Infrastructure and Operations at Spotify. Since the audio-streaming platform launched in 2008, it has already grown to over 200 million monthly active users around the world, and for Chakrabarti's team, the goal is solidifying Spotify's infrastructure to support all those future consumers too. +{{< /case-studies/lead >}} -
-
-
-
- Another plus: "Kubernetes fit very nicely as a complement and now as a replacement to Helios, so we could have it running alongside Helios to mitigate the risks," says Chakrabarti. "During the migration, the services run on both, so we’re not having to put all of our eggs in one basket until we can validate Kubernetes under a variety of load circumstances and stress circumstances."

- The team spent much of 2018 addressing the core technology issues required for the migration. "We were able to use a lot of the Kubernetes APIs and extensibility features of Kubernetes to support and interface with our legacy infrastructure, so the integration was straightforward and easy," says Site Reliability Engineer James Wen.

- Migration started late that year and has accelerated in 2019. "Our focus is really on stateless services, and once we address our last remaining technology blocker, that’s where we hope that the uptick will come from," says Chakrabarti. "For stateful services there’s more work that we need to do."

- A small percentage of Spotify’s fleet, containing over 150 services, has been migrated to Kubernetes so far. "We’ve heard from our customers that they have less of a need to focus on manual capacity provisioning and more time to focus on delivering features for Spotify," says Chakrabarti. The biggest service currently running on Kubernetes takes over 10 million requests per second as an aggregate service and benefits greatly from autoscaling, says Wen. Plus, Wen adds, "Before, teams would have to wait for an hour to create a new service and get an operational host to run it in production, but with Kubernetes, they can do that on the order of seconds and minutes." In addition, with Kubernetes’s bin-packing and multi-tenancy capabilities, CPU utilization has improved on average two- to threefold. +

An early adopter of microservices and Docker, Spotify had containerized microservices running across its fleet of VMs since 2014. The company used an open source, homegrown container orchestration system called Helios, and in 2016-17 completed a migration from on premise data centers to Google Cloud. Underpinning these decisions, "We have a culture around autonomous teams, over 200 autonomous engineering squads who are working on different pieces of the pie, and they need to be able to iterate quickly," Chakrabarti says. "So for us to have developer velocity tools that allow squads to move quickly is really important."

-
-
-
-
- "We were able to use a lot of the Kubernetes APIs and extensibility features to support and interface with our legacy infrastructure, so the integration was straightforward and easy."

- James Wen, Site Reliability Engineer, Spotify -
-
+

But by late 2017, it became clear that "having a small team working on the Helios features was just not as efficient as adopting something that was supported by a much bigger community," says Chakrabarti. "We saw the amazing community that had grown up around Kubernetes, and we wanted to be part of that. We wanted to benefit from added velocity and reduced cost, and also align with the rest of the industry on best practices and tools." At the same time, the team wanted to contribute its expertise and influence in the flourishing Kubernetes community.

-
-
- Chakrabarti points out that for all four of the top-level metrics that Spotify looks at—lead time, deployment frequency, time to resolution, and operational load—"there is impact that Kubernetes is having."

- One success story that’s come out of the early days of Kubernetes is a tool called Slingshot that a Spotify team built on Kubernetes. "With a pull request, it creates a temporary staging environment that self destructs after 24 hours," says Chakrabarti. "It’s all facilitated by Kubernetes, so that’s kind of an exciting example of how, once the technology is out there and ready to use, people start to build on top of it and craft their own solutions, even beyond what we might have envisioned as the initial purpose of it."

- Spotify has also started to use gRPC and Envoy, replacing existing homegrown solutions, just as it had with Kubernetes. "We created things because of the scale we were at, and there was no other solution existing," says Dave Zolotusky, Software Engineer, Infrastructure and Operations. "But then the community kind of caught up and surpassed us, even for tools that work at that scale." +{{< case-studies/quote + image="/images/case-studies/spotify/banner3.jpg" + author="Dave Zolotusky, Software Engineer, Infrastructure and Operations, Spotify" +>}} +"The community has been extremely helpful in getting us to work through all the technology much faster and much easier. And it's helped us validate all the things we're doing." +{{< /case-studies/quote >}} -
+

Another plus: "Kubernetes fit very nicely as a complement and now as a replacement to Helios, so we could have it running alongside Helios to mitigate the risks," says Chakrabarti. "During the migration, the services run on both, so we're not having to put all of our eggs in one basket until we can validate Kubernetes under a variety of load circumstances and stress circumstances."

-
-
- "It’s been surprisingly easy to get in touch with anybody we wanted to, to get expertise on any of the things we’re working with. And it’s helped us validate all the things we’re doing." -

- James Wen, Site Reliability Engineer, Spotify
-
+

The team spent much of 2018 addressing the core technology issues required for the migration. "We were able to use a lot of the Kubernetes APIs and extensibility features of Kubernetes to support and interface with our legacy infrastructure, so the integration was straightforward and easy," says Site Reliability Engineer James Wen.

-
- Both of those technologies are in early stages of adoption, but already "we have reason to believe that gRPC will have a more drastic impact during early development by helping with a lot of issues like schema management, API design, weird backward compatibility issues, things like that," says Zolotusky. "So we’re leaning heavily on gRPC to help us in that space."

- As the team continues to fill out Spotify’s cloud native stack—tracing is up next—it is using the CNCF landscape as a helpful guide. "We look at things we need to solve, and if there are a bunch of projects, we evaluate them equivalently, but there is definitely value to the project being a CNCF project," says Zolotusky.

- Spotify’s experiences so far with Kubernetes bears this out. "The community has been extremely helpful in getting us to work through all the technology much faster and much easier," Zolotusky says. "It’s been surprisingly easy to get in touch with anybody we wanted to, to get expertise on any of the things we’re working with. And it’s helped us validate all the things we’re doing." +

Migration started late that year and has accelerated in 2019. "Our focus is really on stateless services, and once we address our last remaining technology blocker, that's where we hope that the uptick will come from," says Chakrabarti. "For stateful services there's more work that we need to do."

-
-
- - +

A small percentage of Spotify's fleet, containing over 150 services, has been migrated to Kubernetes so far. "We've heard from our customers that they have less of a need to focus on manual capacity provisioning and more time to focus on delivering features for Spotify," says Chakrabarti. The biggest service currently running on Kubernetes takes over 10 million requests per second as an aggregate service and benefits greatly from autoscaling, says Wen. Plus, Wen adds, "Before, teams would have to wait for an hour to create a new service and get an operational host to run it in production, but with Kubernetes, they can do that on the order of seconds and minutes." In addition, with Kubernetes's bin-packing and multi-tenancy capabilities, CPU utilization has improved on average two- to threefold.

+ +{{< case-studies/quote + image="/images/case-studies/spotify/banner4.jpg" + author="James Wen, Site Reliability Engineer, Spotify" +>}} +"We were able to use a lot of the Kubernetes APIs and extensibility features to support and interface with our legacy infrastructure, so the integration was straightforward and easy." +{{< /case-studies/quote >}} + +

Chakrabarti points out that for all four of the top-level metrics that Spotify looks at—lead time, deployment frequency, time to resolution, and operational load—"there is impact that Kubernetes is having."

+ +

One success story that's come out of the early days of Kubernetes is a tool called Slingshot that a Spotify team built on Kubernetes. "With a pull request, it creates a temporary staging environment that self destructs after 24 hours," says Chakrabarti. "It's all facilitated by Kubernetes, so that's kind of an exciting example of how, once the technology is out there and ready to use, people start to build on top of it and craft their own solutions, even beyond what we might have envisioned as the initial purpose of it."

+ +

Spotify has also started to use gRPC and Envoy, replacing existing homegrown solutions, just as it had with Kubernetes. "We created things because of the scale we were at, and there was no other solution existing," says Dave Zolotusky, Software Engineer, Infrastructure and Operations. "But then the community kind of caught up and surpassed us, even for tools that work at that scale."

+ +{{< case-studies/quote author="James Wen, Site Reliability Engineer, Spotify" >}} +"It's been surprisingly easy to get in touch with anybody we wanted to, to get expertise on any of the things we're working with. And it's helped us validate all the things we're doing." +{{< /case-studies/quote >}} + +

Both of those technologies are in early stages of adoption, but already "we have reason to believe that gRPC will have a more drastic impact during early development by helping with a lot of issues like schema management, API design, weird backward compatibility issues, things like that," says Zolotusky. "So we're leaning heavily on gRPC to help us in that space."

+ +

As the team continues to fill out Spotify's cloud native stack—tracing is up next—it is using the CNCF landscape as a helpful guide. "We look at things we need to solve, and if there are a bunch of projects, we evaluate them equivalently, but there is definitely value to the project being a CNCF project," says Zolotusky.

+ +

Spotify's experiences so far with Kubernetes bears this out. "The community has been extremely helpful in getting us to work through all the technology much faster and much easier," Zolotusky says. "It's been surprisingly easy to get in touch with anybody we wanted to, to get expertise on any of the things we're working with. And it's helped us validate all the things we're doing."

\ No newline at end of file diff --git a/content/en/case-studies/squarespace/index.html b/content/en/case-studies/squarespace/index.html index 27340835f4..461e466d8c 100644 --- a/content/en/case-studies/squarespace/index.html +++ b/content/en/case-studies/squarespace/index.html @@ -2,100 +2,70 @@ title: Squarespace Case Study case_study_styles: true cid: caseStudies -css: /css/style_case_studies.css + +new_case_study_styles: true +heading_background: /images/case-studies/squarespace/banner1.jpg +heading_title_logo: /images/squarespace_logo.png +subheading: > + Squarespace: Gaining Productivity and Resilience with Kubernetes +case_study_details: + - Company: Squarespace + - Location: New York, N.Y. + - Industry: Software as a Service, Website-Building Platform --- -
-

CASE STUDY:
Squarespace: Gaining Productivity and Resilience with Kubernetes -

+

Challenge

-
+

Moving from a monolith to microservices in 2014 "solved a problem on the development side, but it pushed that problem to the infrastructure team," says Kevin Lynch, Staff Engineer on the Site Reliability team at Squarespace. "The infrastructure deployment process on our 5,000 VM hosts was slowing everyone down."

-
- Company  Squarespace     Location  New York, N.Y.     Industry  Software as a Service, Website-Building Platform -
- -
-
-
-
-

Challenge

- Moving from a monolith to microservices in 2014 "solved a problem on the development side, but it pushed that problem to the infrastructure team," says Kevin Lynch, Staff Engineer on the Site Reliability team at Squarespace. "The infrastructure deployment process on our 5,000 VM hosts was slowing everyone down." -

Solution

-The team experimented with container orchestration platforms, and found that Kubernetes "answered all the questions that we had," says Lynch. The company began running Kubernetes in its data centers in 2016. -
- -
+

The team experimented with container orchestration platforms, and found that Kubernetes "answered all the questions that we had," says Lynch. The company began running Kubernetes in its data centers in 2016.

Impact

-Since Squarespace moved to Kubernetes, in conjunction with modernizing its networking stack, deployment time has been reduced by almost 85%. Before, their VM deployment would take half an hour; now, says Lynch, "someone can generate a templated application, deploy it within five minutes, and have actual instances containerized, running in our staging environment at that point." Because of that, "productivity time is the big cost saver," he adds. "When we started the Kubernetes project, we had probably a dozen microservices. Today there are twice that in the pipeline being actively worked on." Resilience has also been improved with Kubernetes: "If a node goes down, it’s rescheduled immediately and there’s no performance impact." -
+

Since Squarespace moved to Kubernetes, in conjunction with modernizing its networking stack, deployment time has been reduced by almost 85%. Before, their VM deployment would take half an hour; now, says Lynch, "someone can generate a templated application, deploy it within five minutes, and have actual instances containerized, running in our staging environment at that point." Because of that, "productivity time is the big cost saver," he adds. "When we started the Kubernetes project, we had probably a dozen microservices. Today there are twice that in the pipeline being actively worked on." Resilience has also been improved with Kubernetes: "If a node goes down, it's rescheduled immediately and there's no performance impact."

-
-
-
-
- -

"Once you prove that Kubernetes solves one problem, everyone immediately starts solving other problems without you even having to evangelize it." -

— Kevin Lynch, Staff Engineer on the Site Reliability team at Squarespace -
-
-
-
-

Since it was started in a dorm room in 2003, Squarespace has made it simple for millions of people to create their own websites.

Behind the scenes, though, the company’s monolithic Java application was making things not so simple for its developers to keep improving the platform. So in 2014, the company decided to "go down the microservices path," says Kevin Lynch, staff engineer on Squarespace’s Site Reliability team. "But we were always deploying our applications in vCenter VMware VMs [in our own data centers]. Microservices solved a problem on the development side, but it pushed that problem to the Infrastructure team. The infrastructure deployment process on our 5,000 VM hosts was slowing everyone down."

- After experimenting with another container orchestration platform and "breaking it in very painful ways," Lynch says, the team began experimenting with Kubernetes in mid-2016 and found that it "answered all the questions that we had." Deploying it in the data center rather than the public cloud was their biggest challenge, and at the time, not a lot of other companies were doing that. "We had to figure out how to deploy this in our infrastructure for ourselves, and we had to integrate it with our other applications," says Lynch.

- At the same time, Squarespace’s Network Engineering team was modernizing its networking stack, switching from a traditional layer-two network to a layer-three spine-and-leaf network. "It mapped beautifully with what we wanted to do with Kubernetes," says Lynch. "It gives us the ability to have our servers communicate directly with the top-of-rack switches. We use Calico for CNI networking for Kubernetes, so we can announce all these individual Kubernetes pod IP addresses and have them integrate seamlessly with our other services that are still provisioned in the VMs." +{{< case-studies/quote author="Kevin Lynch, Staff Engineer on the Site Reliability team at Squarespace" >}} + +
+"Once you prove that Kubernetes solves one problem, everyone immediately starts solving other problems without you even having to evangelize it." +{{< /case-studies/quote >}} -
-
-
-
- After experimenting with another container orchestration platform and "breaking it in very painful ways," Lynch says, the team began experimenting with Kubernetes in mid-2016 and found that it "answered all the questions that we had." +{{< case-studies/lead >}} +Since it was started in a dorm room in 2003, Squarespace has made it simple for millions of people to create their own websites. +{{< /case-studies/lead >}} -
-
-
-
- Within a couple months, they had a stable cluster for their internal use, and began rolling out Kubernetes for production. They also added Zipkin and CNCF projects Prometheus and fluentd to their cloud native stack. "We switched to Kubernetes, a new world, and we revamped all our other tooling as well," says Lynch. "It allowed us to streamline our process, so we can now easily create an entire microservice project from templates, generate the code and deployment pipeline for that, generate the Docker file, and then immediately just ship a workable, deployable project to Kubernetes." Deployments across Dev/QA/Stage/Prod were also "simplified drastically," Lynch adds. "Now there is little configuration variation." -

- And the whole process takes only five minutes, an almost 85% reduction in time compared to their VM deployment. "From end to end that probably took half an hour, and that’s not accounting for the fact that an infrastructure engineer would be responsible for doing that, so there’s some business delay in there as well." -

- With faster deployments, "productivity time is the big cost saver," says Lynch. "We had a team that was implementing a new file storage service, and they just started integrating that with our storage back end without our involvement"—which wouldn’t have been possible before Kubernetes. He adds: "When we started the Kubernetes project, we had probably a dozen microservices. Today there are twice that in the pipeline being actively worked on." +

Behind the scenes, though, the company's monolithic Java application was making things not so simple for its developers to keep improving the platform. So in 2014, the company decided to "go down the microservices path," says Kevin Lynch, staff engineer on Squarespace's Site Reliability team. "But we were always deploying our applications in vCenter VMware VMs [in our own data centers]. Microservices solved a problem on the development side, but it pushed that problem to the Infrastructure team. The infrastructure deployment process on our 5,000 VM hosts was slowing everyone down."

+

After experimenting with another container orchestration platform and "breaking it in very painful ways," Lynch says, the team began experimenting with Kubernetes in mid-2016 and found that it "answered all the questions that we had." Deploying it in the data center rather than the public cloud was their biggest challenge, and at the time, not a lot of other companies were doing that. "We had to figure out how to deploy this in our infrastructure for ourselves, and we had to integrate it with our other applications," says Lynch.

-
-
-
-
- "We switched to Kubernetes, a new world....It allowed us to streamline our process, so we can now easily create an entire microservice project from templates," Lynch says. And the whole process takes only five minutes, an almost 85% reduction in time compared to their VM deployment. -
-
+

At the same time, Squarespace's Network Engineering team was modernizing its networking stack, switching from a traditional layer-two network to a layer-three spine-and-leaf network. "It mapped beautifully with what we wanted to do with Kubernetes," says Lynch. "It gives us the ability to have our servers communicate directly with the top-of-rack switches. We use Calico for CNI networking for Kubernetes, so we can announce all these individual Kubernetes pod IP addresses and have them integrate seamlessly with our other services that are still provisioned in the VMs."

-
-
- There’s also been a positive impact on the application’s resilience. "When we’re deploying VMs, we have to build tooling to ensure that a service is spread across racks appropriately and can withstand failure," he says. "Kubernetes just does it. If a node goes down, it’s rescheduled immediately and there’s no performance impact." -

- Another big benefit is autoscaling. "It wasn’t really possible with the way we’ve been using VMware," says Lynch, "but now we can just add the appropriate autoscaling features via Kubernetes directly, and boom, it’s scaling up as demand increases. And it worked out of the box." -

- For others starting out with Kubernetes, Lynch says his best advice is to "fail fast": "Once you’ve planned things out, just execute. Kubernetes has been really great for trying something out quickly and seeing if it works or not." +{{< case-studies/quote image="/images/case-studies/squarespace/banner3.jpg" >}} +After experimenting with another container orchestration platform and "breaking it in very painful ways," Lynch says, the team began experimenting with Kubernetes in mid-2016 and found that it "answered all the questions that we had." +{{< /case-studies/quote >}} +

Within a couple months, they had a stable cluster for their internal use, and began rolling out Kubernetes for production. They also added Zipkin and CNCF projects Prometheus and fluentd to their cloud native stack. "We switched to Kubernetes, a new world, and we revamped all our other tooling as well," says Lynch. "It allowed us to streamline our process, so we can now easily create an entire microservice project from templates, generate the code and deployment pipeline for that, generate the Docker file, and then immediately just ship a workable, deployable project to Kubernetes." Deployments across Dev/QA/Stage/Prod were also "simplified drastically," Lynch adds. "Now there is little configuration variation."

-
+

And the whole process takes only five minutes, an almost 85% reduction in time compared to their VM deployment. "From end to end that probably took half an hour, and that's not accounting for the fact that an infrastructure engineer would be responsible for doing that, so there's some business delay in there as well."

-
-
- "When we’re deploying VMs, we have to build tooling to ensure that a service is spread across racks appropriately and can withstand failure," he says. "Kubernetes just does it. If a node goes down, it’s rescheduled immediately and there’s no performance impact." -
-
+

With faster deployments, "productivity time is the big cost saver," says Lynch. "We had a team that was implementing a new file storage service, and they just started integrating that with our storage back end without our involvement"—which wouldn't have been possible before Kubernetes. He adds: "When we started the Kubernetes project, we had probably a dozen microservices. Today there are twice that in the pipeline being actively worked on."

-
- Lynch and his team are planning to open source some of the tools they’ve developed to extend Kubernetes and use it as an API itself. The first tool injects dependent applications as containers in a pod. "When you ship an application, usually it comes along with a whole bunch of dependent applications that need to be shipped with that, for example, fluentd for logging," he explains. With this tool, the developer doesn’t need to worry about the configurations. -

- Going forward, all new services at Squarespace are going into Kubernetes, and the end goal is to convert everything it can. About a quarter of existing services have been migrated. "Our monolithic application is going to be the last one, just because it’s so big and complex," says Lynch. "But now I’m seeing other services get moved over, like the file storage service. Someone just did it and it worked—painlessly. So I believe if we tackle it, it’s probably going to be a lot easier than we fear. Maybe I should just take my own advice and fail fast!" +{{< case-studies/quote image="/images/case-studies/squarespace/banner4.jpg" >}} +"We switched to Kubernetes, a new world....It allowed us to streamline our process, so we can now easily create an entire microservice project from templates," Lynch says. And the whole process takes only five minutes, an almost 85% reduction in time compared to their VM deployment. +{{< /case-studies/quote >}} -
+

There's also been a positive impact on the application's resilience. "When we're deploying VMs, we have to build tooling to ensure that a service is spread across racks appropriately and can withstand failure," he says. "Kubernetes just does it. If a node goes down, it's rescheduled immediately and there's no performance impact."

-
+

Another big benefit is autoscaling. "It wasn't really possible with the way we've been using VMware," says Lynch, "but now we can just add the appropriate autoscaling features via Kubernetes directly, and boom, it's scaling up as demand increases. And it worked out of the box."

+ +

For others starting out with Kubernetes, Lynch says his best advice is to "fail fast": "Once you've planned things out, just execute. Kubernetes has been really great for trying something out quickly and seeing if it works or not."

+ +{{< case-studies/quote >}} +"When we're deploying VMs, we have to build tooling to ensure that a service is spread across racks appropriately and can withstand failure," he says. "Kubernetes just does it. If a node goes down, it's rescheduled immediately and there's no performance impact." +{{< /case-studies/quote >}} + +

Lynch and his team are planning to open source some of the tools they've developed to extend Kubernetes and use it as an API itself. The first tool injects dependent applications as containers in a pod. "When you ship an application, usually it comes along with a whole bunch of dependent applications that need to be shipped with that, for example, fluentd for logging," he explains. With this tool, the developer doesn't need to worry about the configurations.

+ +

Going forward, all new services at Squarespace are going into Kubernetes, and the end goal is to convert everything it can. About a quarter of existing services have been migrated. "Our monolithic application is going to be the last one, just because it's so big and complex," says Lynch. "But now I'm seeing other services get moved over, like the file storage service. Someone just did it and it worked—painlessly. So I believe if we tackle it, it's probably going to be a lot easier than we fear. Maybe I should just take my own advice and fail fast!"

diff --git a/content/en/case-studies/thredup/index.html b/content/en/case-studies/thredup/index.html index ad990356ff..fb2cae26d5 100644 --- a/content/en/case-studies/thredup/index.html +++ b/content/en/case-studies/thredup/index.html @@ -3,92 +3,75 @@ title: ThredUp Case Study linkTitle: thredup case_study_styles: true cid: caseStudies -css: /css/style_case_studies.css featured: false + +new_case_study_styles: true +heading_background: /images/case-studies/thredup/banner1.jpg +heading_title_logo: /images/thredup_logo.png +case_study_details: + - Company: ThredUp + - Location: San Francisco, CA + - Industry: eCommerce --- -
-

CASE STUDY:

-

-
+

Challenge

-
- Company  ThredUp     Location  San Francisco, CA     Industry  eCommerce -
+

The largest online consignment store for women's and children's clothes, ThredUP launched in 2009 with a monolithic application running on Amazon Web Services. Though the company began breaking up the monolith into microservices a few years ago, the infrastructure team was still dealing with handcrafted servers, which hampered productivity. "We've configured them just to get them out as fast as we could, but there was no standardization, and as we kept growing, that became a bigger and bigger chore to manage," says Cofounder/CTO Chris Homer. The infrastructure, they realized, needed to be modernized to enable the velocity the company needed. "It's really important to a company like us who's disrupting the retail industry to make sure that as we're building software and getting it out in front of our users, we can do it on a fast cycle and learn a ton as we experiment," adds Homer. "We wanted to make sure that our engineers could embrace the DevOps mindset as they built software. It was really important to us that they could own the life cycle from end to end, from conception at design, through shipping it and running it in production, from marketing to ecommerce, the user experience and our internal distribution center operations."

-
-
-
-
-

Challenge

- The largest online consignment store for women’s and children’s clothes, ThredUP launched in 2009 with a monolithic application running on Amazon Web Services. Though the company began breaking up the monolith into microservices a few years ago, the infrastructure team was still dealing with handcrafted servers, which hampered productivity. "We’ve configured them just to get them out as fast as we could, but there was no standardization, and as we kept growing, that became a bigger and bigger chore to manage," says Cofounder/CTO Chris Homer. The infrastructure, they realized, needed to be modernized to enable the velocity the company needed. "It’s really important to a company like us who’s disrupting the retail industry to make sure that as we’re building software and getting it out in front of our users, we can do it on a fast cycle and learn a ton as we experiment," adds Homer. "We wanted to make sure that our engineers could embrace the DevOps mindset as they built software. It was really important to us that they could own the life cycle from end to end, from conception at design, through shipping it and running it in production, from marketing to ecommerce, the user experience and our internal distribution center operations." -

+

Solution

+ +

In early 2017, the company adopted Kubernetes for container orchestration, and in the course of a year, the entire infrastructure was moved to Kubernetes.

-

Solution

- In early 2017, the company adopted Kubernetes for container orchestration, and in the course of a year, the entire infrastructure was moved to Kubernetes. -

Impact

- Before, "even considering that we already have all the infrastructure in the cloud, databases and services, and all these good things," says Infrastructure Engineer Oleksandr Snagovskyi, setting up a new service meant waiting 2-4 weeks just to get the environment. With Kubernetes, new application roll-out time has decreased from several days or weeks to minutes or hours. Now, says Infrastructure Engineer Oleksii Asiutin, "our developers can experiment with existing applications and create new services, and do it all blazingly fast." In fact, deployment time has decreased about 50% on average for key services. "Lead time" for all applications is under 20 minutes, enabling engineers to deploy multiple times a day. Plus, 3200+ ansible scripts have been deprecated in favor of helm charts. And impressively, hardware cost has decreased 56% while the number of services ThredUP runs has doubled. -
-
-
-
-
-

- "Moving towards cloud native technologies like Kubernetes really unlocks our ability to experiment quickly and learn from customers along the way." -

- CHRIS HOMER, COFOUNDER/CTO, THREDUP -
-
-
-
-

The largest online consignment store for women’s and children’s clothes, ThredUP is focused on getting consumers to think second-hand first. "We’re disrupting the retail industry, and it’s really important to us to make sure that as we’re building software and getting it out in front of our users, we can do it on a fast cycle and learn a ton as we experiment," says Cofounder/CTO Chris Homer. -

- But over the past few years, ThredUP, which was launched in 2009 with a monolithic application running on Amazon Web Services, was feeling growing pains as its user base passed the 20- million mark. Though the company had begun breaking up the monolith into microservices, the infrastructure team was still dealing with handcrafted servers, which hampered productivity. "We’ve configured them just to get them out as fast as we could, but there was no standardization, and as we kept growing, that became a bigger and bigger chore to manage," says Homer. The infrastructure, Homer realized, needed to be modernized to enable the velocity—and the culture—the company wanted. -

- "We wanted to make sure that our engineers could embrace the DevOps mindset as they built software," Homer says. "It was really important to us that they could own the life cycle from end to end, from conception at design, through shipping it and running it in production, from marketing to ecommerce, the user experience and our internal distribution center operations." -
-
-
-
- "Kubernetes enabled auto scaling in a seamless and easily manageable way on days like Black Friday. We no longer have to sit there adding instances, monitoring the traffic, doing a lot of manual work."

- CHRIS HOMER, COFOUNDER/CTO, THREDUP +

Before, "even considering that we already have all the infrastructure in the cloud, databases and services, and all these good things," says Infrastructure Engineer Oleksandr Snagovskyi, setting up a new service meant waiting 2-4 weeks just to get the environment. With Kubernetes, new application roll-out time has decreased from several days or weeks to minutes or hours. Now, says Infrastructure Engineer Oleksii Asiutin, "our developers can experiment with existing applications and create new services, and do it all blazingly fast." In fact, deployment time has decreased about 50% on average for key services. "Lead time" for all applications is under 20 minutes, enabling engineers to deploy multiple times a day. Plus, 3200+ ansible scripts have been deprecated in favor of helm charts. And impressively, hardware cost has decreased 56% while the number of services ThredUP runs has doubled.

-
-
-
-
- In early 2017, Homer found the solution with Kubernetes container orchestration. In the course of a year, the company migrated its entire infrastructure to Kubernetes, starting with its website applications and concluding with its operations backend. Teams are now also using Fluentd and Helm. "Initially there were skeptics about the value that this move to cloud native technologies would bring, but as we went through the process, people very quickly started to realize the benefit of having seamless upgrades and easy rollbacks without having to worry about what was happening," says Homer. "It unlocks the developers’ confidence in being able to deploy quickly, learn, and if you make a mistake, you can roll it back without any issue." -

- According to the infrastructure team, the key improvement was the consistent experience Kubernetes enabled for developers. "It lets developers work in the same environment that their application will be running in production," says Infrastructure Engineer Oleksandr Snagovskyi. Plus, "It became easier to test, easier to refine, and easier to deploy, because everything’s done automatically," says Infrastructure Engineer Oleksii Asiutin. "One of the main goals of our team is to make developers’ lives more comfortable, and we are achieving this with Kubernetes. They can experiment with existing applications and create new services, and do it all blazingly fast." -
-
-
-
- "One of the main goals of our team is to make developers’ lives more comfortable, and we are achieving this with Kubernetes. They can experiment with existing applications and create new services, and do it all blazingly fast."

- OLEKSII ASIUTIN, INFRASTRUCTURE ENGINEER, THREDUP -
-
+{{< case-studies/quote author="CHRIS HOMER, COFOUNDER/CTO, THREDUP" >}} + +
+"Moving towards cloud native technologies like Kubernetes really unlocks our ability to experiment quickly and learn from customers along the way." +{{< /case-studies/quote >}} -
-
- Before, "even considering that we already have all the infrastructure in the cloud, databases and services, and all these good things," says Snagovskyi, setting up a new service meant waiting 2-4 weeks just to get the environment. With Kubernetes, because of simple configuration and minimal dependency on the infrastructure team, the roll-out time for new applications has decreased from several days or weeks to minutes or hours. -

- In fact, deployment time has decreased about 50% on average for key services. "Fast deployment and parallel test execution in Kubernetes keep a ‘lead time’ for all applications under 20 minutes," allowing engineers to do multiple releases a day, says Director of Infrastructure Roman Chepurnyi. The infrastructure team’s jobs, he adds, have become less burdensome, too: "We can execute seamless upgrades frequently and keep cluster performance and security up-to-date because OS-level hardening and upgrades of a Kubernetes cluster is a non-blocking activity for production operations and does not involve coordination with multiple engineering teams." -

- More than 3,200 ansible scripts have been deprecated in favor of Helm charts. And impressively, hardware cost has decreased 56% while the number of services ThredUP runs has doubled. -
+{{< case-studies/lead >}} +The largest online consignment store for women's and children's clothes, ThredUP is focused on getting consumers to think second-hand first. "We're disrupting the retail industry, and it's really important to us to make sure that as we're building software and getting it out in front of our users, we can do it on a fast cycle and learn a ton as we experiment," says Cofounder/CTO Chris Homer. +{{< /case-studies/lead >}} -
-
-"Our future’s all about automation, and behind that, cloud native technologies are going to unlock our ability to embrace that and go full force towards the future."

- CHRIS HOMER, COFOUNDER/CTO, THREDUP
-
+

But over the past few years, ThredUP, which was launched in 2009 with a monolithic application running on Amazon Web Services, was feeling growing pains as its user base passed the 20- million mark. Though the company had begun breaking up the monolith into microservices, the infrastructure team was still dealing with handcrafted servers, which hampered productivity. "We've configured them just to get them out as fast as we could, but there was no standardization, and as we kept growing, that became a bigger and bigger chore to manage," says Homer. The infrastructure, Homer realized, needed to be modernized to enable the velocity—and the culture—the company wanted.

-
- Perhaps the impact is most evident on the busiest days in retail. "Kubernetes enabled auto scaling in a seamless and easily manageable way on days like Black Friday," says Homer. "We no longer have to sit there adding instances, monitoring the traffic, doing a lot of manual work. That’s handled for us, and instead we can actually have some turkey, drink some wine and enjoy our families." -

- For ThredUP, Kubernetes fits perfectly with the company’s vision for how it’s changing retail. Some of what ThredUP does is still very manual: "As our customers send bags of items to our distribution centers, they’re photographed, inspected, tagged, and put online today," says Homer. -

- But in every other aspect, "we use different forms of technology to drive everything we do," Homer says. "We have machine learning algorithms to help predict the likelihood of sale for items, which drives our pricing algorithm. We have personalization algorithms that look at the images and try to determine style and match users’ preferences across our systems." -

- Count Kubernetes as one of those drivers. "Our future’s all about automation," says Homer, "and behind that, cloud native technologies are going to unlock our ability to embrace that and go full force towards the future." -
-
+

"We wanted to make sure that our engineers could embrace the DevOps mindset as they built software," Homer says. "It was really important to us that they could own the life cycle from end to end, from conception at design, through shipping it and running it in production, from marketing to ecommerce, the user experience and our internal distribution center operations."

+ +{{< case-studies/quote + image="/images/case-studies/thredup/banner3.jpg" + author="CHRIS HOMER, COFOUNDER/CTO, THREDUP" +>}} +"Kubernetes enabled auto scaling in a seamless and easily manageable way on days like Black Friday. We no longer have to sit there adding instances, monitoring the traffic, doing a lot of manual work." +{{< /case-studies/quote >}} + +

In early 2017, Homer found the solution with Kubernetes container orchestration. In the course of a year, the company migrated its entire infrastructure to Kubernetes, starting with its website applications and concluding with its operations backend. Teams are now also using Fluentd and Helm. "Initially there were skeptics about the value that this move to cloud native technologies would bring, but as we went through the process, people very quickly started to realize the benefit of having seamless upgrades and easy rollbacks without having to worry about what was happening," says Homer. "It unlocks the developers' confidence in being able to deploy quickly, learn, and if you make a mistake, you can roll it back without any issue."

+ +

According to the infrastructure team, the key improvement was the consistent experience Kubernetes enabled for developers. "It lets developers work in the same environment that their application will be running in production," says Infrastructure Engineer Oleksandr Snagovskyi. Plus, "It became easier to test, easier to refine, and easier to deploy, because everything's done automatically," says Infrastructure Engineer Oleksii Asiutin. "One of the main goals of our team is to make developers' lives more comfortable, and we are achieving this with Kubernetes. They can experiment with existing applications and create new services, and do it all blazingly fast."

+ +{{< case-studies/quote + image="/images/case-studies/thredup/banner4.jpg" + author="OLEKSII ASIUTIN, INFRASTRUCTURE ENGINEER, THREDUP" +>}} +"One of the main goals of our team is to make developers' lives more comfortable, and we are achieving this with Kubernetes. They can experiment with existing applications and create new services, and do it all blazingly fast." +{{< /case-studies/quote >}} + +

Before, "even considering that we already have all the infrastructure in the cloud, databases and services, and all these good things," says Snagovskyi, setting up a new service meant waiting 2-4 weeks just to get the environment. With Kubernetes, because of simple configuration and minimal dependency on the infrastructure team, the roll-out time for new applications has decreased from several days or weeks to minutes or hours.

+ +

In fact, deployment time has decreased about 50% on average for key services. "Fast deployment and parallel test execution in Kubernetes keep a 'lead time' for all applications under 20 minutes," allowing engineers to do multiple releases a day, says Director of Infrastructure Roman Chepurnyi. The infrastructure team's jobs, he adds, have become less burdensome, too: "We can execute seamless upgrades frequently and keep cluster performance and security up-to-date because OS-level hardening and upgrades of a Kubernetes cluster is a non-blocking activity for production operations and does not involve coordination with multiple engineering teams."

+ +

More than 3,200 ansible scripts have been deprecated in favor of Helm charts. And impressively, hardware cost has decreased 56% while the number of services ThredUP runs has doubled.

+ +{{< case-studies/quote author="CHRIS HOMER, COFOUNDER/CTO, THREDUP">}} +"Our future's all about automation, and behind that, cloud native technologies are going to unlock our ability to embrace that and go full force towards the future." +{{< /case-studies/quote >}} + +

Perhaps the impact is most evident on the busiest days in retail. "Kubernetes enabled auto scaling in a seamless and easily manageable way on days like Black Friday," says Homer. "We no longer have to sit there adding instances, monitoring the traffic, doing a lot of manual work. That's handled for us, and instead we can actually have some turkey, drink some wine and enjoy our families."

+ +

For ThredUP, Kubernetes fits perfectly with the company's vision for how it's changing retail. Some of what ThredUP does is still very manual: "As our customers send bags of items to our distribution centers, they're photographed, inspected, tagged, and put online today," says Homer.

+ +

But in every other aspect, "we use different forms of technology to drive everything we do," Homer says. "We have machine learning algorithms to help predict the likelihood of sale for items, which drives our pricing algorithm. We have personalization algorithms that look at the images and try to determine style and match users' preferences across our systems."

+ +

Count Kubernetes as one of those drivers. "Our future's all about automation," says Homer, "and behind that, cloud native technologies are going to unlock our ability to embrace that and go full force towards the future."

diff --git a/content/en/case-studies/vsco/index.html b/content/en/case-studies/vsco/index.html index c2ac2a2a72..95d8ae011c 100644 --- a/content/en/case-studies/vsco/index.html +++ b/content/en/case-studies/vsco/index.html @@ -3,95 +3,77 @@ title: vsco Case Study linkTitle: vsco case_study_styles: true cid: caseStudies -css: /css/style_case_studies.css featured: false + +new_case_study_styles: true +heading_background: /images/case-studies/vsco/banner1.jpg +heading_title_logo: /images/vsco_logo.png +subheading: > + VSCO: How a Mobile App Saved 70% on Its EC2 Bill with Cloud Native +case_study_details: + - Company: VSCO + - Location: Oakland, CA + - Industry: Photo Mobile App --- -
-

CASE STUDY:
VSCO: How a Mobile App Saved 70% on Its EC2 Bill with Cloud Native -

+

Challenge

-
+

After moving from Rackspace to AWS in 2015, VSCO began building Node.js and Go microservices in addition to running its PHP monolith. The team containerized the microservices using Docker, but "they were all in separate groups of EC2 instances that were dedicated per service," says Melinda Lu, Engineering Manager for the Machine Learning Team. Adds Naveen Gattu, Senior Software Engineer on the Community Team: "That yielded a lot of wasted resources. We started looking for a way to consolidate and be more efficient in the AWS EC2 instances."

-
- Company  VSCO     Location  Oakland, CA     Industry  Photo Mobile App -
+

Solution

+ +

The team began exploring the idea of a scheduling system, and looked at several solutions including Mesos and Swarm before deciding to go with Kubernetes. VSCO also uses gRPC and Envoy in their cloud native stack.

-
-
-
-
-

Challenge

- After moving from Rackspace to AWS in 2015, VSCO began building Node.js and Go microservices in addition to running its PHP monolith. The team containerized the microservices using Docker, but "they were all in separate groups of EC2 instances that were dedicated per service," says Melinda Lu, Engineering Manager for the Machine Learning Team. Adds Naveen Gattu, Senior Software Engineer on the Community Team: "That yielded a lot of wasted resources. We started looking for a way to consolidate and be more efficient in the AWS EC2 instances." -

Solution

- The team began exploring the idea of a scheduling system, and looked at several solutions including Mesos and Swarm before deciding to go with Kubernetes. VSCO also uses gRPC and Envoy in their cloud native stack. -

Impact

- Before, deployments required "a lot of manual tweaking, in-house scripting that we wrote, and because of our disparate EC2 instances, Operations had to babysit the whole thing from start to finish," says Senior Software Engineer Brendan Ryan. "We didn't really have a story around testing in a methodical way, and using reusable containers or builds in a standardized way." There's a faster onboarding process now. Before, the time to first deploy was two days' hands-on setup time; now it's two hours. By moving to continuous integration, containerization, and Kubernetes, velocity was increased dramatically. The time from code-complete to deployment in production on real infrastructure went from one to two weeks to two to four hours for a typical service. Adds Gattu: "In man hours, that's one person versus a developer and a DevOps individual at the same time." With an 80% decrease in time for a single deployment to happen in production, the number of deployments has increased as well, from 1200/year to 3200/year. There have been real dollar savings too: With Kubernetes, VSCO is running at 2x to 20x greater EC2 efficiency, depending on the service, adding up to about 70% overall savings on the company's EC2 bill. Ryan points to the company's ability to go from managing one large monolithic application to 50+ microservices with "the same size developer team, more or less. And we've only been able to do that because we have increased trust in our tooling and a lot more flexibility, so we don't need to employ a DevOps engineer to tune every service." With Kubernetes, gRPC, and Envoy in place, VSCO has seen an 88% reduction in total minutes of outage time, mainly due to the elimination of JSON-schema errors and service-specific infrastructure provisioning errors, and an increased speed in fixing outages. -
+

Before, deployments required "a lot of manual tweaking, in-house scripting that we wrote, and because of our disparate EC2 instances, Operations had to babysit the whole thing from start to finish," says Senior Software Engineer Brendan Ryan. "We didn't really have a story around testing in a methodical way, and using reusable containers or builds in a standardized way." There's a faster onboarding process now. Before, the time to first deploy was two days' hands-on setup time; now it's two hours. By moving to continuous integration, containerization, and Kubernetes, velocity was increased dramatically. The time from code-complete to deployment in production on real infrastructure went from one to two weeks to two to four hours for a typical service. Adds Gattu: "In man hours, that's one person versus a developer and a DevOps individual at the same time." With an 80% decrease in time for a single deployment to happen in production, the number of deployments has increased as well, from 1200/year to 3200/year. There have been real dollar savings too: With Kubernetes, VSCO is running at 2x to 20x greater EC2 efficiency, depending on the service, adding up to about 70% overall savings on the company's EC2 bill. Ryan points to the company's ability to go from managing one large monolithic application to 50+ microservices with "the same size developer team, more or less. And we've only been able to do that because we have increased trust in our tooling and a lot more flexibility, so we don't need to employ a DevOps engineer to tune every service." With Kubernetes, gRPC, and Envoy in place, VSCO has seen an 88% reduction in total minutes of outage time, mainly due to the elimination of JSON-schema errors and service-specific infrastructure provisioning errors, and an increased speed in fixing outages.

-
-
-
-
- "I've been really impressed seeing how our engineers have come up with creative solutions to things by just combining a lot of Kubernetes primitives. Exposing Kubernetes constructs as a service to our engineers as opposed to exposing higher order constructs has worked well for us. It lets you get familiar with the technology and do more interesting things with it." -

- MELINDA LU, ENGINEERING MANAGER FOR VSCO'S MACHINE LEARNING TEAM -
-
-
-
-

A photography app for mobile, VSCO was born in the cloud in 2011. In the beginning, "we were using Rackspace and had one PHP monolith application talking to MySQL database, with FTP deployments, no containerization, no orchestration," says Software Engineer Brendan Ryan, "which was sufficient at the time."

- After VSCO moved to AWS in 2015 and its user base passed the 30 million mark, the team quickly realized that set-up wouldn't work anymore. Developers had started building some Node and Go microservices, which the team tried containerizing with Docker. But "they were all in separate groups of EC2 instances that were dedicated per service," says Melinda Lu, Engineering Manager for the Machine Learning Team. Adds Naveen Gattu, Senior Software Engineer on the Community Team: "That yielded a lot of wasted resources. We started looking for a way to consolidate and be more efficient in the EC2 instances." -

- With a checklist that included ease of use and implementation, level of support, and whether it was open source, the team evaluated a few scheduling solutions, including Mesos and Swarm, before deciding to go with Kubernetes. "Kubernetes seemed to have the strongest open source community around it," says Lu. Plus, "We had started to standardize on a lot of the Google stack, with Go as a language, and gRPC for almost all communication between our own services inside the data center. So it seemed pretty natural for us to choose Kubernetes." +{{< case-studies/quote author="MELINDA LU, ENGINEERING MANAGER FOR VSCO'S MACHINE LEARNING TEAM" >}} +"I've been really impressed seeing how our engineers have come up with creative solutions to things by just combining a lot of Kubernetes primitives. Exposing Kubernetes constructs as a service to our engineers as opposed to exposing higher order constructs has worked well for us. It lets you get familiar with the technology and do more interesting things with it." +{{< /case-studies/quote >}} -
-
-
-
- "Kubernetes seemed to have the strongest open source community around it, plus, we had started to standardize on a lot of the Google stack, with Go as a language, and gRPC for almost all communication between our own services inside the data center. So it seemed pretty natural for us to choose Kubernetes."

- MELINDA LU, ENGINEERING MANAGER FOR VSCO'S MACHINE LEARNING TEAM +{{< case-studies/lead >}} +A photography app for mobile, VSCO was born in the cloud in 2011. In the beginning, "we were using Rackspace and had one PHP monolith application talking to MySQL database, with FTP deployments, no containerization, no orchestration," says Software Engineer Brendan Ryan, "which was sufficient at the time." +{{< /case-studies/lead >}} -
-
-
-
- At the time, there were few managed Kubernetes offerings and less tooling available in the ecosystem, so the team stood up its own cluster and built some custom components for its specific deployment needs, such as an automatic ingress controller and policy constructs for canary deploys. "We had already begun breaking up the monolith, so we moved things one by one, starting with pretty small, low-risk services," says Lu. "Every single new service was deployed there." The first service was migrated at the end of 2016, and after one year, 80% of the entire stack was on Kubernetes, including the rest of the monolith. -

- The impact has been great. Deployments used to require "a lot of manual tweaking, in-house scripting that we wrote, and because of our disparate EC2 instances, Operations had to babysit the whole thing from start to finish," says Ryan. "We didn't really have a story around testing in a methodical way, and using reusable containers or builds in a standardized way." There's a faster onboarding process now. Before, the time to first deploy was two days' hands-on setup time; now it's two hours. -

- By moving to continuous integration, containerization, and Kubernetes, velocity was increased dramatically. The time from code-complete to deployment in production on real infrastructure went from one to two weeks to two to four hours for a typical service. Plus, says Gattu, "In man hours, that's one person versus a developer and a DevOps individual at the same time." With an 80% decrease in time for a single deployment to happen in production, the number of deployments has increased as well, from 1200/year to 3200/year. +

After VSCO moved to AWS in 2015 and its user base passed the 30 million mark, the team quickly realized that set-up wouldn't work anymore. Developers had started building some Node and Go microservices, which the team tried containerizing with Docker. But "they were all in separate groups of EC2 instances that were dedicated per service," says Melinda Lu, Engineering Manager for the Machine Learning Team. Adds Naveen Gattu, Senior Software Engineer on the Community Team: "That yielded a lot of wasted resources. We started looking for a way to consolidate and be more efficient in the EC2 instances."

-
-
-
-
- "I've been really impressed seeing how our engineers have come up with really creative solutions to things by just combining a lot of Kubernetes primitives, exposing Kubernetes constructs as a service to our engineers as opposed to exposing higher order constructs has worked well for us. It lets you get familiar with the technology and do more interesting things with it."

- MELINDA LU, ENGINEERING MANAGER FOR VSCO’S MACHINE LEARNING TEAM -
-
+

With a checklist that included ease of use and implementation, level of support, and whether it was open source, the team evaluated a few scheduling solutions, including Mesos and Swarm, before deciding to go with Kubernetes. "Kubernetes seemed to have the strongest open source community around it," says Lu. Plus, "We had started to standardize on a lot of the Google stack, with Go as a language, and gRPC for almost all communication between our own services inside the data center. So it seemed pretty natural for us to choose Kubernetes."

-
-
- There have been real dollar savings too: With Kubernetes, VSCO is running at 2x to 20x greater EC2 efficiency, depending on the service, adding up to about 70% overall savings on the company’s EC2 bill. -

- Ryan points to the company’s ability to go from managing one large monolithic application to 50+ microservices with “the same size developer team, more or less. And we’ve only been able to do that because we have increased trust in our tooling and a lot more flexibility when there are stress points in our system. You can increase CPU memory requirements of a service without having to bring up and tear down instances, and read through AWS pages just to be familiar with a lot of jargon, which isn’t really tenable for a company at our scale.” -

- Envoy and gRPC have also had a positive impact at VSCO. “We get many benefits from gRPC out of the box: type safety across multiple languages, ease of defining services with the gRPC IDL, built-in architecture like interceptors, and performance improvements over HTTP/1.1 and JSON,” says Lu. -

- VSCO was one of the first users of Envoy, getting it in production five days after it was open sourced. “We wanted to serve gRPC and HTTP/2 directly to mobile clients through our edge load balancers, and Envoy was our only reasonable solution,” says Lu. “The ability to send consistent and detailed stats by default across all services has made observability and standardization of dashboards much easier.” The metrics that come built in with Envoy have also “greatly helped with debugging,” says DevOps Engineer Ryan Nguyen. -
+{{< case-studies/quote + image="/images/case-studies/vsco/banner2.jpg" + author="MELINDA LU, ENGINEERING MANAGER FOR VSCO'S MACHINE LEARNING TEAM" +>}} +"Kubernetes seemed to have the strongest open source community around it, plus, we had started to standardize on a lot of the Google stack, with Go as a language, and gRPC for almost all communication between our own services inside the data center. So it seemed pretty natural for us to choose Kubernetes." +{{< /case-studies/quote >}} -
-
-"Because there’s now an organization that supports Kubernetes, does that build confidence? The answer is a resounding yes."

- NAVEEN GATTU, SENIOR SOFTWARE ENGINEER ON VSCO’S COMMUNITY TEAM
-
+

At the time, there were few managed Kubernetes offerings and less tooling available in the ecosystem, so the team stood up its own cluster and built some custom components for its specific deployment needs, such as an automatic ingress controller and policy constructs for canary deploys. "We had already begun breaking up the monolith, so we moved things one by one, starting with pretty small, low-risk services," says Lu. "Every single new service was deployed there." The first service was migrated at the end of 2016, and after one year, 80% of the entire stack was on Kubernetes, including the rest of the monolith.

-
- With Kubernetes, gRPC, and Envoy in place, VSCO has seen an 88% reduction in total minutes of outage time, mainly due to the elimination of JSON-schema errors and service-specific infrastructure provisioning errors, and an increased speed in fixing outages. -

- Given its success using CNCF projects, VSCO is starting to experiment with others, including CNI and Prometheus. “To have a large organization backing these technologies, we have a lot more confidence trying this software and deploying to production,” says Nguyen. -

- The team has made contributions to gRPC and Envoy, and is hoping to be even more active in the CNCF community. “I’ve been really impressed seeing how our engineers have come up with really creative solutions to things by just combining a lot of Kubernetes primitives,” says Lu. “Exposing Kubernetes constructs as a service to our engineers as opposed to exposing higher order constructs has worked well for us. It lets you get familiar with the technology and do more interesting things with it.” +

The impact has been great. Deployments used to require "a lot of manual tweaking, in-house scripting that we wrote, and because of our disparate EC2 instances, Operations had to babysit the whole thing from start to finish," says Ryan. "We didn't really have a story around testing in a methodical way, and using reusable containers or builds in a standardized way." There's a faster onboarding process now. Before, the time to first deploy was two days' hands-on setup time; now it's two hours.

-
-
+

By moving to continuous integration, containerization, and Kubernetes, velocity was increased dramatically. The time from code-complete to deployment in production on real infrastructure went from one to two weeks to two to four hours for a typical service. Plus, says Gattu, "In man hours, that's one person versus a developer and a DevOps individual at the same time." With an 80% decrease in time for a single deployment to happen in production, the number of deployments has increased as well, from 1200/year to 3200/year.

+ +{{< case-studies/quote + image="/images/case-studies/vsco/banner4.jpg" + author="MELINDA LU, ENGINEERING MANAGER FOR VSCO'S MACHINE LEARNING TEAM" +>}} +"I've been really impressed seeing how our engineers have come up with really creative solutions to things by just combining a lot of Kubernetes primitives, exposing Kubernetes constructs as a service to our engineers as opposed to exposing higher order constructs has worked well for us. It lets you get familiar with the technology and do more interesting things with it." +{{< /case-studies/quote >}} + +

There have been real dollar savings too: With Kubernetes, VSCO is running at 2x to 20x greater EC2 efficiency, depending on the service, adding up to about 70% overall savings on the company's EC2 bill.

+ +

Ryan points to the company's ability to go from managing one large monolithic application to 50+ microservices with "the same size developer team, more or less. And we've only been able to do that because we have increased trust in our tooling and a lot more flexibility when there are stress points in our system. You can increase CPU memory requirements of a service without having to bring up and tear down instances, and read through AWS pages just to be familiar with a lot of jargon, which isn't really tenable for a company at our scale."

+ +

Envoy and gRPC have also had a positive impact at VSCO. "We get many benefits from gRPC out of the box: type safety across multiple languages, ease of defining services with the gRPC IDL, built-in architecture like interceptors, and performance improvements over HTTP/1.1 and JSON," says Lu.

+ +

VSCO was one of the first users of Envoy, getting it in production five days after it was open sourced. "We wanted to serve gRPC and HTTP/2 directly to mobile clients through our edge load balancers, and Envoy was our only reasonable solution," says Lu. "The ability to send consistent and detailed stats by default across all services has made observability and standardization of dashboards much easier." The metrics that come built in with Envoy have also "greatly helped with debugging," says DevOps Engineer Ryan Nguyen.

+ +{{< case-studies/quote author="NAVEEN GATTU, SENIOR SOFTWARE ENGINEER ON VSCO'S COMMUNITY TEAM" >}} +"Because there's now an organization that supports Kubernetes, does that build confidence? The answer is a resounding yes." +{{< /case-studies/quote >}} + +

With Kubernetes, gRPC, and Envoy in place, VSCO has seen an 88% reduction in total minutes of outage time, mainly due to the elimination of JSON-schema errors and service-specific infrastructure provisioning errors, and an increased speed in fixing outages.

+ +

Given its success using CNCF projects, VSCO is starting to experiment with others, including CNI and Prometheus. "To have a large organization backing these technologies, we have a lot more confidence trying this software and deploying to production," says Nguyen.

+ +

The team has made contributions to gRPC and Envoy, and is hoping to be even more active in the CNCF community. "I've been really impressed seeing how our engineers have come up with really creative solutions to things by just combining a lot of Kubernetes primitives," says Lu. "Exposing Kubernetes constructs as a service to our engineers as opposed to exposing higher order constructs has worked well for us. It lets you get familiar with the technology and do more interesting things with it."

diff --git a/content/en/case-studies/wikimedia/index.html b/content/en/case-studies/wikimedia/index.html index abc74e3ee3..b10002af83 100644 --- a/content/en/case-studies/wikimedia/index.html +++ b/content/en/case-studies/wikimedia/index.html @@ -1,96 +1,66 @@ --- title: Wikimedia Case Study - -class: gridPage +case_study_styles: true cid: caseStudies + +new_case_study_styles: true +heading_title_text: Wikimedia +use_gradient_overlay: true +subheading: > + Using Kubernetes to Build Tools to Improve the World's Wikis +case_study_details: + - Company: Wikimedia + - Location: San Francisco, CA --- -
-

Wikimedia Case Study

-
+

The non-profit Wikimedia Foundation operates some of the largest collaboratively edited reference projects in the world, including Wikipedia. To help users maintain and use wikis, it runs Wikimedia Tool Labs, a hosting environment for community developers working on tools and bots to help editors and other volunteers do their work, including reducing vandalism. The community around Wikimedia Tool Labs began forming nearly 10 years ago.

-
-
-
-

Using Kubernetes to Build Tools to Improve the World's Wikis

-

- The non-profit Wikimedia Foundation operates some of the largest collaboratively edited reference projects in the world, including Wikipedia. To help users maintain and use wikis, it runs Wikimedia Tool Labs, a hosting environment for community developers working on tools and bots to help editors and other volunteers do their work, including reducing vandalism. The community around Wikimedia Tool Labs began forming nearly 10 years ago. -

-
- Wikimedia -

- "Wikimedia Tool Labs is vital for making sure wikis all around the world work as well as they possibly can. Because it's grown organically for almost 10 years, it has become an extremely challenging environment and difficult to maintain. It's like a big ball of mud — you really can't see through it. With Kubernetes, we're simplifying the environment and making it easier for developers to build the tools that make wikis run better." -

-

— Yuvi Panda, operations engineer at Wikimedia Foundation and Wikimedia Tool Labs

-
-
-
-
+{{< case-studies/quote author="Yuvi Panda, operations engineer at Wikimedia Foundation and Wikimedia Tool Labs">}} +Wikimedia +
+
+"Wikimedia Tool Labs is vital for making sure wikis all around the world work as well as they possibly can. Because it's grown organically for almost 10 years, it has become an extremely challenging environment and difficult to maintain. It's like a big ball of mud — you really can't see through it. With Kubernetes, we're simplifying the environment and making it easier for developers to build the tools that make wikis run better." +{{< /case-studies/quote >}} -
-
-
-
-

Challenges:

-
    -
  • Simplify a complex, difficult-to-manage infrastructure
    • -
  • Allow developers to continue writing tools and bots using existing techniques
    • -
-
-
-

Why Kubernetes:

-
    -
  • Wikimedia Tool Labs chose Kubernetes because it can mimic existing workflows, while reducing complexity
    • -
-
-
-

Approach:

-
    -
  • Migrate old systems and a complex infrastructure to Kubernetes
    • -
-
-
-

Results:

-
    -
  • 20 percent of web tools that account for more than 40 percent of web traffic now run on Kubernetes
    • -
  • A 25-node cluster that keeps up with each new Kubernetes release
    • -
  • Thousands of lines of old code have been deleted, thanks to Kubernetes
    • -
-
-
-
-
+

Challenges

-
-
-
-

Using Kubernetes to provide tools for maintaining wikis

-

- Wikimedia Tool Labs is run by a staff of four-and-a-half paid employees and two volunteers. The infrastructure didn't make it easy or intuitive for developers to build bots and other tools to make wikis work more easily. Yuvi says, "It's incredibly chaotic. We have lots of Perl and Bash duct tape on top of it. Everything is super fragile." -

-

- To solve the problem, Wikimedia Tool Labs migrated parts of its infrastructure to Kubernetes, in preparation for eventually moving its entire system. Yuvi said Kubernetes greatly simplifies maintenance. The goal is to allow developers creating bots and other tools to use whatever development methods they want, but make it easier for the Wikimedia Tool Labs to maintain the required infrastructure for hosting and sharing them. -

-

- "With Kubernetes, I've been able to remove a lot of our custom-made code, which makes everything easier to maintain. Our users' code also runs in a more stable way than previously," says Yuvi. -

-
-
-
+ -
-
-
-

Simplifying infrastructure and keeping wikis running better

-

- Wikimedia Tool Labs has seen great success with the initial Kubernetes deployment. Old code is being simplified and eliminated, contributing developers don't have to change the way they write their tools and bots, and those tools and bots run in a more stable fashion than they have in the past. The paid staff and volunteers are able to better keep up with fixing issues. -

-

- In the future, with a more complete migration to Kubernetes, Wikimedia Tool Labs expects to make it even easier to host and maintain the bots and tools that help run wikis across the world. The tool labs already host approximately 1,300 tools and bots from 800 volunteers, with many more being submitted every day. Twenty percent of the tool labs' web tools that account for more than 60 percent of web traffic now run on Kubernetes. The tool labs has a 25-node cluster that keeps up with each new Kubernetes release. Many existing web tools are migrating to Kubernetes. -

-

- "Our goal is to make sure that people all over the world can share knowledge as easily as possible. Kubernetes helps with that, by making it easier for wikis everywhere to have the tools they need to thrive," says Yuvi. -

-
-
-
+

Why Kubernetes

+ + + +

Approach

+ + + +

Results

+ + + +

Using Kubernetes to provide tools for maintaining wikis

+ +

Wikimedia Tool Labs is run by a staff of four-and-a-half paid employees and two volunteers. The infrastructure didn't make it easy or intuitive for developers to build bots and other tools to make wikis work more easily. Yuvi says, "It's incredibly chaotic. We have lots of Perl and Bash duct tape on top of it. Everything is super fragile."

+ +

To solve the problem, Wikimedia Tool Labs migrated parts of its infrastructure to Kubernetes, in preparation for eventually moving its entire system. Yuvi said Kubernetes greatly simplifies maintenance. The goal is to allow developers creating bots and other tools to use whatever development methods they want, but make it easier for the Wikimedia Tool Labs to maintain the required infrastructure for hosting and sharing them.

+ +

"With Kubernetes, I've been able to remove a lot of our custom-made code, which makes everything easier to maintain. Our users' code also runs in a more stable way than previously," says Yuvi.

+ +

Simplifying infrastructure and keeping wikis running better

+ +

Wikimedia Tool Labs has seen great success with the initial Kubernetes deployment. Old code is being simplified and eliminated, contributing developers don't have to change the way they write their tools and bots, and those tools and bots run in a more stable fashion than they have in the past. The paid staff and volunteers are able to better keep up with fixing issues.

+ +

In the future, with a more complete migration to Kubernetes, Wikimedia Tool Labs expects to make it even easier to host and maintain the bots and tools that help run wikis across the world. The tool labs already host approximately 1,300 tools and bots from 800 volunteers, with many more being submitted every day. Twenty percent of the tool labs' web tools that account for more than 60 percent of web traffic now run on Kubernetes. The tool labs has a 25-node cluster that keeps up with each new Kubernetes release. Many existing web tools are migrating to Kubernetes.

+ +

"Our goal is to make sure that people all over the world can share knowledge as easily as possible. Kubernetes helps with that, by making it easier for wikis everywhere to have the tools they need to thrive," says Yuvi.

diff --git a/content/en/case-studies/wink/index.html b/content/en/case-studies/wink/index.html index 3f47f8c779..1f8ea4c89d 100644 --- a/content/en/case-studies/wink/index.html +++ b/content/en/case-studies/wink/index.html @@ -1,109 +1,87 @@ --- title: Wink Case Study - case_study_styles: true cid: caseStudies -css: /css/style_wink.css + +new_case_study_styles: true +heading_background: /images/case-studies/wink/banner1.jpg +heading_title_logo: /images/wink_logo.png +subheading: > + Cloud-Native Infrastructure Keeps Your Smart Home Connected +case_study_details: + - Company: Wink + - Location: New York, N.Y. + - Industry: Internet of Things Platform --- -
-

CASE STUDY:
-
Cloud-Native Infrastructure Keeps Your Smart Home Connected
-

-
+

Challenge

+

Building a low-latency, highly reliable infrastructure to serve communications between millions of connected smart-home devices and the company's consumer hubs and mobile app, with an emphasis on horizontal scalability, the ability to encrypt everything quickly and connections that could be easily brought back up if anything went wrong.

-
- Company  Wink     Location  New York, N.Y.     Industry  Internet of Things Platform -
+

Solution

-
+

Across-the-board use of a Kubernetes-Docker-CoreOS Container Linux stack.

-
-
-
+

Impact

-

Challenge

- Building a low-latency, highly reliable infrastructure to serve communications between millions of connected smart-home devices and the company’s consumer hubs and mobile app, with an emphasis on horizontal scalability, the ability to encrypt everything quickly and connections that could be easily brought back up if anything went wrong. -

-

Solution

- Across-the-board use of a Kubernetes-Docker-CoreOS Container Linux stack.

-
+

"Two of the biggest American retailers [Home Depot and Walmart] are carrying and promoting the brand and the hardware," Wink Head of Engineering Kit Klein says proudly – though he adds that "it really comes with a lot of pressure. It's not a retail situation where you have a lot of tech enthusiasts. These are everyday people who want something that works and have no tolerance for technical excuses." And that's further testament to how much faith Klein has in the infrastructure that the Wink team has built. With 80 percent of Wink's workload running on a unified stack of Kubernetes-Docker-CoreOS, the company has put itself in a position to continually innovate and improve its products and services. Committing to this technology, says Klein, "makes building on top of the infrastructure relatively easy."

-
-

Impact

- "Two of the biggest American retailers [Home Depot and Walmart] are carrying and promoting the brand and the hardware,” Wink Head of Engineering Kit Klein says proudly – though he adds that "it really comes with a lot of pressure. It’s not a retail situation where you have a lot of tech enthusiasts. These are everyday people who want something that works and have no tolerance for technical excuses.” And that’s further testament to how much faith Klein has in the infrastructure that the Wink team has built. With 80 percent of Wink’s workload running on a unified stack of Kubernetes-Docker-CoreOS, the company has put itself in a position to continually innovate and improve its products and services. Committing to this technology, says Klein, "makes building on top of the infrastructure relatively easy.” -
+{{< case-studies/quote author="KIT KLEIN, HEAD OF ENGINEERING, WINK" >}} +"It's not proprietary, it's totally open, it's really portable. You can run all the workloads across different cloud providers. You can easily run a hybrid AWS or even bring in your own data center. That's the benefit of having everything unified on one open source Kubernetes-Docker-CoreOS Container Linux stack. There are massive security benefits if you only have one Linux distro/machine image to validate. The benefits are enormous because you save money, and you save time." +{{< /case-studies/quote >}} -
-
+{{< case-studies/lead >}} +How many people does it take to turn on a light bulb? +{{< /case-studies/lead >}} +

Kit Klein whips out his phone to demonstrate. With a few swipes, the head of engineering at Wink pulls up the smart-home app created by the New York City-based company and taps the light button. "Honestly when you're holding the phone and you're hitting the light," he says, "by the time you feel the pressure of your finger on the screen, it's on. It takes as long as the signal to travel to your brain."

-
-
- "It’s not proprietary, it’s totally open, it’s really portable. You can run all the workloads across different cloud providers. You can easily run a hybrid AWS or even bring in your own data center. That’s the benefit of having everything unified on one open source Kubernetes-Docker-CoreOS Container Linux stack. There are massive security benefits if you only have one Linux distro/machine image to validate. The benefits are enormous because you save money, and you save time.”

- KIT KLEIN, HEAD OF ENGINEERING, WINK -
-
+

Sure, it takes just one finger and less than 200 milliseconds to turn on the light – or lock a door or change a thermostat. But what allows Wink to help consumers manage their connected smart-home products with such speed and ease is a sophisticated, cloud native infrastructure that Klein and his team built and continue to develop using a unified stack of CoreOS, the open-source operating system designed for clustered deployments, and Kubernetes, an open-source platform for automating deployment, scaling, and operations of application containers across clusters of hosts, providing container-centric infrastructure. "When you have a big, complex network of interdependent microservices that need to be able to discover each other, and need to be horizontally scalable and tolerant to failure, that's what this is really optimized for," says Klein. "A lot of people end up relying on proprietary services [offered by some big cloud providers] to do some of this stuff, but what you get by adopting CoreOS/Kubernetes is portability, to not be locked in to anyone. You can really make your own fate."

+

Indeed, Wink did. The company's mission statement is to make the connected home accessible – that is, user-friendly for non-technical owners, affordable and perhaps most importantly, reliable. "If you can't trust that when you hit the switch, you know a light is going to go on, or if you're remote and you're checking on your house and that information isn't accurate, then the convenience of the system is lost," says Klein. "So that's where the infrastructure comes in."

-
-
-

How many people does it take to turn on a light bulb?

+

Wink was incubated within Quirky, a company that developed crowd-sourced inventions. The Wink app was first introduced in 2013, and at the time, it controlled only a few consumer products such as the PivotPower Strip that Quirky produced in collaboration with GE. As smart-home products proliferated, Wink was launched in 2014 in Home Depot stores nationwide. Its first project: a hub that could integrate with smart products from about a dozen brands like Honeywell and Chamberlain. The biggest challenge would be to build the infrastructure to serve all those communications between the hub and the products, with a focus on maximizing reliability and minimizing latency.

- Kit Klein whips out his phone to demonstrate. With a few swipes, the head of engineering at Wink pulls up the smart-home app created by the New York City-based company and taps the light button. "Honestly when you’re holding the phone and you’re hitting the light,” he says, "by the time you feel the pressure of your finger on the screen, it’s on. It takes as long as the signal to travel to your brain.”

- Sure, it takes just one finger and less than 200 milliseconds to turn on the light – or lock a door or change a thermostat. But what allows Wink to help consumers manage their connected smart-home products with such speed and ease is a sophisticated, cloud native infrastructure that Klein and his team built and continue to develop using a unified stack of CoreOS, the open-source operating system designed for clustered deployments, and Kubernetes, an open-source platform for automating deployment, scaling, and operations of application containers across clusters of hosts, providing container-centric infrastructure. "When you have a big, complex network of interdependent microservices that need to be able to discover each other, and need to be horizontally scalable and tolerant to failure, that’s what this is really optimized for,” says Klein. "A lot of people end up relying on proprietary services [offered by some big cloud providers] to do some of this stuff, but what you get by adopting CoreOS/Kubernetes is portability, to not be locked in to anyone. You can really make your own fate.”

- Indeed, Wink did. The company’s mission statement is to make the connected home accessible – that is, user-friendly for non-technical owners, affordable and perhaps most importantly, reliable. "If you can’t trust that when you hit the switch, you know a light is going to go on, or if you’re remote and you’re checking on your house and that information isn’t accurate, then the convenience of the system is lost,” says Klein. "So that’s where the infrastructure comes in.”

- Wink was incubated within Quirky, a company that developed crowd-sourced inventions. The Wink app was first introduced in 2013, and at the time, it controlled only a few consumer products such as the PivotPower Strip that Quirky produced in collaboration with GE. As smart-home products proliferated, Wink was launched in 2014 in Home Depot stores nationwide. Its first project: a hub that could integrate with smart products from about a dozen brands like Honeywell and Chamberlain. The biggest challenge would be to build the infrastructure to serve all those communications between the hub and the products, with a focus on maximizing reliability and minimizing latency.

- "When we originally started out, we were moving very fast trying to get the first product to market, the minimum viable product,” says Klein. "Lots of times you go down a path and end up having to backtrack and try different things. But in this particular case, we did a lot of the work up front, which led to us making a really sound decision to deploy it on CoreOS Container Linux. And that was very early in the life of it.” +

"When we originally started out, we were moving very fast trying to get the first product to market, the minimum viable product," says Klein. "Lots of times you go down a path and end up having to backtrack and try different things. But in this particular case, we did a lot of the work up front, which led to us making a really sound decision to deploy it on CoreOS Container Linux. And that was very early in the life of it."

-
-
+{{< case-studies/quote image="/images/case-studies/wink/banner3.jpg">}} +"...what you get by adopting CoreOS/Kubernetes is portability, to not be locked in to anyone. You can really make your own fate." +{{< /case-studies/quote >}} -
-
- "...what you get by adopting CoreOS/Kubernetes is portability, to not be locked in to anyone. You can really make your own fate.” -
-
+

Concern number one: Wink's products need to connect to consumer devices in people's homes, behind a firewall. "You don't have an end point like a URL, and you don't even know what ports are open behind that firewall," Klein explains. "So you essentially need to have this thing wake up and talk to your system and then open real-time, bidirectional communication between the cloud and the device. And it's really, really important that it's persistent because you want to decrease as much as possible the overhead of sending a message – you never know when someone is going to turn on the lights."

-
-
- Concern number one: Wink’s products need to connect to consumer devices in people’s homes, behind a firewall. "You don’t have an end point like a URL, and you don’t even know what ports are open behind that firewall,” Klein explains. "So you essentially need to have this thing wake up and talk to your system and then open real-time, bidirectional communication between the cloud and the device. And it’s really, really important that it’s persistent because you want to decrease as much as possible the overhead of sending a message – you never know when someone is going to turn on the lights.”

- With the earliest version of the Wink Hub, when you decided to turn your lights on or off, the request would be sent to the cloud and then executed. Subsequent updates to Wink’s software enabled local control, cutting latency down to about 10 milliseconds for many devices. But with the need for cloud-enabled integrations of an ever-growing ecosystem of smart home products, low-latency internet connectivity is still a critical consideration. -

-

"You essentially need to have this thing wake up and talk to your system and then open real-time, bidirectional communication between the cloud and the device. And it’s really, really important that it’s persistent...you never know when someone is going to turn on the lights.”

- In addition, Wink had other requirements: horizontal scalability, the ability to encrypt everything quickly, connections that could be easily brought back up if something went wrong. "Looking at this whole structure we started, we decided to make a secure socket-based service,” says Klein. "We’ve always used, I would say, some sort of clustering technology to deploy our services and so the decision we came to was, this thing is going to be containerized, running on Docker.”

- At the time – just over two years ago – Docker wasn’t yet widely used, but as Klein points out, "it was certainly understood by the people who were on the frontier of technology. We started looking at potential technologies that existed. One of the limiting factors was that we needed to deploy multi-port non-http/https services. It wasn’t really appropriate for some of the early cluster technology. We liked the project a lot and we ended up using it on other stuff for a while, but initially it was too targeted toward http workloads.”

- Once Wink’s backend engineering team decided on a Dockerized workload, they had to make decisions about the OS and the container orchestration platform. "Obviously you can’t just start the containers and hope everything goes well,” Klein says with a laugh. "You need to have a system that is helpful [in order] to manage where the workloads are being distributed out to. And when the container inevitably dies or something like that, to restart it, you have a load balancer. All sorts of housekeeping work is needed to have a robust infrastructure.” +

With the earliest version of the Wink Hub, when you decided to turn your lights on or off, the request would be sent to the cloud and then executed. Subsequent updates to Wink's software enabled local control, cutting latency down to about 10 milliseconds for many devices. But with the need for cloud-enabled integrations of an ever-growing ecosystem of smart home products, low-latency internet connectivity is still a critical consideration.

-
-
+{{< case-studies/lead >}} +"You essentially need to have this thing wake up and talk to your system and then open real-time, bidirectional communication between the cloud and the device. And it's really, really important that it's persistent...you never know when someone is going to turn on the lights." +{{< /case-studies/lead >}} -
-
- "Obviously you can’t just start the containers and hope everything goes well,” Klein says with a laugh. "You need to have a system that is helpful [in order] to manage where the workloads are being distributed out to. And when the container inevitably dies or something like that, to restart it, you have a load balancer. All sorts of housekeeping work is needed to have a robust infrastructure.” -
-
+

In addition, Wink had other requirements: horizontal scalability, the ability to encrypt everything quickly, connections that could be easily brought back up if something went wrong. "Looking at this whole structure we started, we decided to make a secure socket-based service," says Klein. "We've always used, I would say, some sort of clustering technology to deploy our services and so the decision we came to was, this thing is going to be containerized, running on Docker."

-
-
- Wink considered building directly on a general purpose Linux distro like Ubuntu (which would have required installing tools to run a containerized workload) and cluster management systems like Mesos (which was targeted toward enterprises with larger teams/workloads), but ultimately set their sights on CoreOS Container Linux. "A container-optimized Linux distribution system was exactly what we needed,” he says. "We didn’t have to futz around with trying to take something like a Linux distro and install everything. It’s got a built-in container orchestration system, which is Fleet, and an easy-to-use API. It’s not as feature-rich as some of the heavier solutions, but we realized that, at that moment, it was exactly what we needed.”

- Wink’s hub (along with a revamped app) was introduced in July 2014 with a short-term deployment, and within the first month, they had moved the service to the Dockerized CoreOS deployment. Since then, they’ve moved almost every other piece of their infrastructure – from third-party cloud-to-cloud integrations to their customer service and payment portals – onto CoreOS Container Linux clusters.

- Using this setup did require some customization. "Fleet is really nice as a basic container orchestration system, but it doesn’t take care of routing, sharing configurations, secrets, et cetera, among instances of a service,” Klein says. "All of those layers of functionality can be implemented, of course, but if you don’t want to spend a lot of time writing unit files manually – which of course nobody does – you need to create a tool to automate some of that, which we did.”

- Wink quickly embraced the Kubernetes container cluster manager when it was launched in 2015 and integrated with CoreOS core technology, and as promised, it ended up providing the features Wink wanted and had planned to build. "If not for Kubernetes, we likely would have taken the logic and library we implemented for the automation tool that we created, and would have used it in a higher level abstraction and tool that could be used by non-DevOps engineers from the command line to create and manage clusters,” Klein says. "But Kubernetes made that totally unnecessary – and is written and maintained by people with a lot more experience in cluster management than us, so all the better.” Now, an estimated 80 percent of Wink’s workload is run on Kubernetes on top of CoreOS Container Linux. +

At the time – just over two years ago – Docker wasn't yet widely used, but as Klein points out, "it was certainly understood by the people who were on the frontier of technology. We started looking at potential technologies that existed. One of the limiting factors was that we needed to deploy multi-port non-http/https services. It wasn't really appropriate for some of the early cluster technology. We liked the project a lot and we ended up using it on other stuff for a while, but initially it was too targeted toward http workloads."

-
-
+

Once Wink's backend engineering team decided on a Dockerized workload, they had to make decisions about the OS and the container orchestration platform. "Obviously you can't just start the containers and hope everything goes well," Klein says with a laugh. "You need to have a system that is helpful [in order] to manage where the workloads are being distributed out to. And when the container inevitably dies or something like that, to restart it, you have a load balancer. All sorts of housekeeping work is needed to have a robust infrastructure."

-
-
- "Stay close to the development. Understand why decisions are being made. If you understand the intent behind the project, from the technological intent to a certain philosophical intent, then it helps you understand how to build your system in harmony with those systems as opposed to trying to work against it.” -
-
+{{< case-studies/quote image="/images/case-studies/wink/banner4.jpg" >}} +"Obviously you can't just start the containers and hope everything goes well," Klein says with a laugh. "You need to have a system that is helpful [in order] to manage where the workloads are being distributed out to. And when the container inevitably dies or something like that, to restart it, you have a load balancer. All sorts of housekeeping work is needed to have a robust infrastructure." +{{< /case-studies/quote >}} -
-
- Wink’s reasons for going all in are clear: "It’s not proprietary, it’s totally open, it’s really portable,” Klein says. "You can run all the workloads across different cloud providers. You can easily run a hybrid AWS or even bring in your own data center. That’s the benefit of having everything unified on one Kubernetes-Docker-CoreOS Container Linux stack. There are massive security benefits if you only have one Linux distro to try to validate. The benefits are enormous because you save money, you save time.”

- Klein concedes that there are tradeoffs in every technology decision. "Cutting-edge technology is going to be scary for some people,” he says. "In order to take advantage of this, you really have to keep up with the technology. You can’t treat it like it’s a black box. Stay close to the development. Understand why decisions are being made. If you understand the intent behind the project, from the technological intent to a certain philosophical intent, then it helps you understand how to build your system in harmony with those systems as opposed to trying to work against it.”

- Wink, which was acquired by Flex in 2015, now controls 2.3 million connected devices in households all over the country. What’s next for the company? A new version of the hub - Wink Hub 2 - hit shelves last November – and is being offered for the first time at Walmart stores in addition to Home Depot. "Two of the biggest American retailers are carrying and promoting the brand and the hardware,” Klein says proudly – though he adds that "it really comes with a lot of pressure. It’s not a retail situation where you have a lot of tech enthusiasts. These are everyday people who want something that works and have no tolerance for technical excuses.” And that’s further testament to how much faith Klein has in the infrastructure that the Wink team has have built.

- Wink’s engineering team has grown exponentially since its early days, and behind the scenes, Klein is most excited about the machine learning Wink is using. "We built [a system of] containerized small sections of the data pipeline that feed each other and can have multiple outputs,” he says. "It’s like data pipelines as microservices.” Again, Klein points to having a unified stack running on CoreOS Container Linux and Kubernetes as the primary driver for the innovations to come. "You’re not reinventing the wheel every time,” he says. "You can just get down to work.”
-
+

Wink considered building directly on a general purpose Linux distro like Ubuntu (which would have required installing tools to run a containerized workload) and cluster management systems like Mesos (which was targeted toward enterprises with larger teams/workloads), but ultimately set their sights on CoreOS Container Linux. "A container-optimized Linux distribution system was exactly what we needed," he says. "We didn't have to futz around with trying to take something like a Linux distro and install everything. It's got a built-in container orchestration system, which is Fleet, and an easy-to-use API. It's not as feature-rich as some of the heavier solutions, but we realized that, at that moment, it was exactly what we needed."

+ +

Wink's hub (along with a revamped app) was introduced in July 2014 with a short-term deployment, and within the first month, they had moved the service to the Dockerized CoreOS deployment. Since then, they've moved almost every other piece of their infrastructure – from third-party cloud-to-cloud integrations to their customer service and payment portals – onto CoreOS Container Linux clusters.

+ +

Using this setup did require some customization. "Fleet is really nice as a basic container orchestration system, but it doesn't take care of routing, sharing configurations, secrets, et cetera, among instances of a service," Klein says. "All of those layers of functionality can be implemented, of course, but if you don't want to spend a lot of time writing unit files manually – which of course nobody does – you need to create a tool to automate some of that, which we did."

+ +

Wink quickly embraced the Kubernetes container cluster manager when it was launched in 2015 and integrated with CoreOS core technology, and as promised, it ended up providing the features Wink wanted and had planned to build. "If not for Kubernetes, we likely would have taken the logic and library we implemented for the automation tool that we created, and would have used it in a higher level abstraction and tool that could be used by non-DevOps engineers from the command line to create and manage clusters," Klein says. "But Kubernetes made that totally unnecessary – and is written and maintained by people with a lot more experience in cluster management than us, so all the better." Now, an estimated 80 percent of Wink's workload is run on Kubernetes on top of CoreOS Container Linux.

+ +{{< case-studies/quote >}} +"Stay close to the development. Understand why decisions are being made. If you understand the intent behind the project, from the technological intent to a certain philosophical intent, then it helps you understand how to build your system in harmony with those systems as opposed to trying to work against it." +{{< /case-studies/quote >}} + +

Wink's reasons for going all in are clear: "It's not proprietary, it's totally open, it's really portable," Klein says. "You can run all the workloads across different cloud providers. You can easily run a hybrid AWS or even bring in your own data center. That's the benefit of having everything unified on one Kubernetes-Docker-CoreOS Container Linux stack. There are massive security benefits if you only have one Linux distro to try to validate. The benefits are enormous because you save money, you save time."

+ +

Klein concedes that there are tradeoffs in every technology decision. "Cutting-edge technology is going to be scary for some people," he says. "In order to take advantage of this, you really have to keep up with the technology. You can't treat it like it's a black box. Stay close to the development. Understand why decisions are being made. If you understand the intent behind the project, from the technological intent to a certain philosophical intent, then it helps you understand how to build your system in harmony with those systems as opposed to trying to work against it."

+ +

Wink, which was acquired by Flex in 2015, now controls 2.3 million connected devices in households all over the country. What's next for the company? A new version of the hub - Wink Hub 2 - hit shelves last November – and is being offered for the first time at Walmart stores in addition to Home Depot. "Two of the biggest American retailers are carrying and promoting the brand and the hardware," Klein says proudly – though he adds that "it really comes with a lot of pressure. It's not a retail situation where you have a lot of tech enthusiasts. These are everyday people who want something that works and have no tolerance for technical excuses." And that's further testament to how much faith Klein has in the infrastructure that the Wink team has have built.

+ +

Wink's engineering team has grown exponentially since its early days, and behind the scenes, Klein is most excited about the machine learning Wink is using. "We built [a system of] containerized small sections of the data pipeline that feed each other and can have multiple outputs," he says. "It's like data pipelines as microservices." Again, Klein points to having a unified stack running on CoreOS Container Linux and Kubernetes as the primary driver for the innovations to come. "You're not reinventing the wheel every time," he says. "You can just get down to work."

diff --git a/content/en/case-studies/woorank/index.html b/content/en/case-studies/woorank/index.html index fbb86bdd24..d83240d878 100644 --- a/content/en/case-studies/woorank/index.html +++ b/content/en/case-studies/woorank/index.html @@ -3,94 +3,77 @@ title: Woorank Case Study linkTitle: woorank case_study_styles: true cid: caseStudies -css: /css/style_case_studies.css featured: false + +new_case_study_styles: true +heading_background: /images/case-studies/woorank/banner1.jpg +heading_title_logo: /images/woorank_logo.png +subheading: > + Woorank: How Kubernetes Helped a Startup Manage 50 Microservices with 12 Engineers—At 30% Less Cost +case_study_details: + - Company: Woorank + - Location: Brussels, Belgium + - Industry: Digital marketing tool --- +

Challenge

-
-

CASE STUDY:
Woorank: How Kubernetes Helped a Startup Manage 50 Microservices with
12 Engineers—At 30% Less Cost -

+

Founded in 2011, Woorank embraced microservices and containerization early on, so its core product, a tool that helps digital marketers improve their websites' visibility on the internet, consists of 50 applications developed and maintained by a technical team of 12. For two years, the infrastructure ran smoothly on Mesos, but "there were still lots of our own libraries that we had to roll and applications that we had to bring in, so it was very cumbersome for us as a small team to keep those things alive and to update them," says CTO/Cofounder Nils De Moor. So he began looking for a new solution with more automation and self-healing built in, that would better suit the company's human resources.

-
+

Solution

-
- Company  Woorank     Location  Brussels, Belgium     Industry  Digital marketing tool -
+

De Moor decided to switch to Kubernetes running on AWS, which "allows us to just define applications, how they need to run, how scalable they need to be, and it takes pain away from the developers thinking about that," he says. "When things fail and errors pop up, the system tries to heal itself, and that's really, for us, the key reason to work with Kubernetes." The company now also uses Fluentd, Prometheus, and OpenTracing.

-
-
-
-
- -

Challenge

- Founded in 2011, Woorank embraced microservices and containerization early on, so its core product, a tool that helps digital marketers improve their websites’ visibility on the internet, consists of 50 applications developed and maintained by a technical team of 12. For two years, the infrastructure ran smoothly on Mesos, but “there were still lots of our own libraries that we had to roll and applications that we had to bring in, so it was very cumbersome for us as a small team to keep those things alive and to update them,” says CTO/Cofounder Nils De Moor. So he began looking for a new solution with more automation and self-healing built in, that would better suit the company’s human resources. -

Solution

- De Moor decided to switch to Kubernetes running on AWS, which “allows us to just define applications, how they need to run, how scalable they need to be, and it takes pain away from the developers thinking about that,” he says. “When things fail and errors pop up, the system tries to heal itself, and that’s really, for us, the key reason to work with Kubernetes.” The company now also uses Fluentd, Prometheus, and OpenTracing.

Impact

- The company’s number one concern was immediately erased: Maintaining Kubernetes takes just one person on staff, and it’s not a fulltime job. Infrastructure updates used to take two active working days; now it’s just a matter of “a few hours of passively following the process,” says De Moor. Implementing new tools—which once took weeks of planning, installing, and onboarding—now only takes a few days. “We were already pretty flexible in our costs and taking on traffic peaks and higher load in general,” adds De Moor, “but with Kubernetes and the other CNCF tools we use, we have achieved about 30% in cost savings.” Plus, the rate of deployments per day has nearly doubled. -
-
-
-
-
- “It was definitely important for us to have CNCF as an umbrella above everything. We’ve always been working with open source libraries and tools and technologies. It works very well for us, but sometimes things can drift, maintainers drop out, and projects go haywire. For us, it was indeed important to know that whatever project gets taken under this umbrella, it’s taken very seriously. Our way of contributing back is also by joining this community. It’s, for us, a way to show our appreciation for what’s going on in this framework.” -

— NILS DE MOOR, CTO/COFOUNDER, WOORANK -
-
-
-
-

Woorank’s core product is a tool that enables digital marketers to improve their websites’ visibility on the internet.

- “We help them acquire lots of data and then present it to them in meaningful ways so they can work with it,” says CTO/Cofounder Nils De Moor. In its seven years as a startup, the company followed a familiar technological path to build that product: starting with a monolithic application, breaking it down into microservices, and then embracing containerization. “That’s where our modern infrastructure started out,” says De Moor. -

- As new features have been added to the product, it has grown to consist of 50 applications under the hood. Though Docker had made things easier to deploy, and the team had been using Mesos as an orchestration framework on AWS since 2015, De Moor realized there was still too much overhead to managing the infrastructure, especially with a technical team of just 12. -

- “The pain point was that there were still lots of our own libraries that we had to roll and applications that we had to bring in, so it was very cumbersome for us as a small team to keep those things alive and to update them,” says De Moor. “When things went wrong during deployment, someone manually had to come in and figure it out. It wasn’t necessarily that the technology or anything was wrong with Mesos; it was just not really fitting our model of being a small company, not having the human resources to make sure it all works and can be updated.” +

The company's number one concern was immediately erased: Maintaining Kubernetes takes just one person on staff, and it's not a fulltime job. Infrastructure updates used to take two active working days; now it's just a matter of "a few hours of passively following the process," says De Moor. Implementing new tools—which once took weeks of planning, installing, and onboarding—now only takes a few days. "We were already pretty flexible in our costs and taking on traffic peaks and higher load in general," adds De Moor, "but with Kubernetes and the other CNCF tools we use, we have achieved about 30% in cost savings." Plus, the rate of deployments per day has nearly doubled.

-
-
-
-
- "Cloud native technologies have brought to us a transparency on everything going on in our system, from the code to the server. It has brought huge cost savings and a better way of dealing with those costs and keeping them under control. And performance-wise, it has helped our team understand how we can make our code work better on the cloud native infrastructure."

— NILS DE MOOR, CTO/COFOUNDER, WOORANK +{{< case-studies/quote author="NILS DE MOOR, CTO/COFOUNDER, WOORANK" >}} +"It was definitely important for us to have CNCF as an umbrella above everything. We've always been working with open source libraries and tools and technologies. It works very well for us, but sometimes things can drift, maintainers drop out, and projects go haywire. For us, it was indeed important to know that whatever project gets taken under this umbrella, it's taken very seriously. Our way of contributing back is also by joining this community. It's, for us, a way to show our appreciation for what's going on in this framework." +{{< /case-studies/quote >}} -
-
+{{< case-studies/lead >}} +Woorank's core product is a tool that enables digital marketers to improve their websites' visibility on the internet. +{{< /case-studies/lead >}} -
-
- Around the time Woorank was grappling with these issues, Kubernetes was emerging as a technology. De Moor knew that he wanted a platform that would be more automated and self-healing, and when he began experimenting with Kubernetes, he found that it checked all those boxes. “Kubernetes allows us to just define applications, how they need to run, how scalable they need to be, and it takes pain away from the developers thinking about that,” he says. “When things fail and errors pop up, the system tries to heal itself, and that’s really, for us, the key reason to work with Kubernetes. It allowed us to set up certain testing frameworks to just be alerted when things go wrong, instead of having to look at whether everything went right. It’s made people’s lives much easier. It’s quite a big mindset change.” -

- Once one small Kubernetes cluster was up and running, the team began moving over a few applications at a time, gradually increasing the load over the course of several months. By early 2017, Woorank was 100% deployed on Kubernetes. -

- The company’s number one concern was immediately erased: Maintaining Kubernetes is the responsibility of just one person on staff, and it’s not his fulltime job. Updating the old infrastructure “was always a pain,” says De Moor: It used to take two active working days, “and it was always a bit scary when we did that.” With Kubernetes, it’s just a matter of “a few hours of passively following the process.” -
-
-
-
- "When things fail and errors pop up, the system tries to heal itself, and that’s really, for us, the key reason to work with Kubernetes. It allowed us to set up certain testing frameworks to just be alerted when things go wrong, instead of having to look at whether everything went right. It’s made people’s lives much easier. It’s quite a big mindset change."

- NILS DE MOOR, CTO/COFOUNDER, WOORANK -
-
+

"We help them acquire lots of data and then present it to them in meaningful ways so they can work with it," says CTO/Cofounder Nils De Moor. In its seven years as a startup, the company followed a familiar technological path to build that product: starting with a monolithic application, breaking it down into microservices, and then embracing containerization. "That's where our modern infrastructure started out," says De Moor.

-
-
- Transparency on all levels, from the code to the servers, has also been a byproduct of the move to Kubernetes. “It’s easier for the entire team to get a better understanding of the infrastructure, how it’s working, how it looks like, what’s going on,” says De Moor. “It’s not that thing that’s running, and no one really knows how it works except this one person. Now it’s really a team effort of everyone knowing, ‘Okay, when something goes wrong, it’s probably in this area or we need to check this.’” -

- To that end, Woorank has begun implementing other cloud native tools that help with visibility, such as Fluentd for logging, Prometheus for monitoring, and OpenTracing for distributed tracing. Implementing these new tools—which once took weeks of planning, installing, and onboarding—now only takes a few days. “With all the tools and projects under the CNCF umbrella, it’s easier for us to test and play with technology than it used to be,” says De Moor. “With Prometheus, we used it fairly early and couldn’t get it fairly stable. A couple of months ago, the question reappeared, so we set it up in two days, and now everyone is using it.” -

- Deployments, too, have been impacted: The rate has more than doubled, which De Moor partly attributes to the transparency of the new process. “With Kubernetes, you see that these three containers didn’t start for this reason,” he says. Plus, “now we bring deployment messages into Slack. If you see deployments rolling by every day, it does somehow indirectly enforce you, okay, I need to be part of this train, so I also need to deploy.” -
+

As new features have been added to the product, it has grown to consist of 50 applications under the hood. Though Docker had made things easier to deploy, and the team had been using Mesos as an orchestration framework on AWS since 2015, De Moor realized there was still too much overhead to managing the infrastructure, especially with a technical team of just 12.

-
-
-"We can plan those things over a certain timeline, try to fit our resource usage to that, and then bring in spot instances, which will hopefully drive the costs down more."

- NILS DE MOOR, CTO/COFOUNDER, WOORANK
-
+

"The pain point was that there were still lots of our own libraries that we had to roll and applications that we had to bring in, so it was very cumbersome for us as a small team to keep those things alive and to update them," says De Moor. "When things went wrong during deployment, someone manually had to come in and figure it out. It wasn't necessarily that the technology or anything was wrong with Mesos; it was just not really fitting our model of being a small company, not having the human resources to make sure it all works and can be updated."

-
- Perhaps the biggest impact, though, has been on the bottom line. “We were already pretty flexible in our costs and taking on traffic peaks and higher load in general, but with Kubernetes and the other CNCF tools we use, we have achieved about 30% in cost savings,” says De Moor. -

- And there’s room for even greater savings. Currently, most of Woorank’s infrastructure is running on AWS on demand; the company pays a fixed price and makes some reservations for its planned amount of resources needed. De Moor is planning to experiment more with spot instances with certain resource-heavy workloads such as web crawls: “We can plan those things over a certain timeline, try to fit our resource usage to that, and then bring in spot instances, which will hopefully drive the costs down more.” -

- Moving to Kubernetes has been so beneficial to Woorank that the company is doubling down on both cloud native technologies and the community. “It was definitely important for us to have CNCF as an umbrella above everything,” says De Moor. “We’ve always been working with open source libraries and tools and technologies. It works very well for us, but sometimes things can drift, maintainers drop out, and projects go haywire. For us, it was indeed important to know that whatever project gets taken under this umbrella, it’s taken very seriously. Our way of contributing back is also by joining this community. It’s, for us, a way to show our appreciation for what’s going on in this framework.” -
-
+{{< case-studies/quote + image="/images/case-studies/woorank/banner3.jpg" + author="NILS DE MOOR, CTO/COFOUNDER, WOORANK" +>}} +"Cloud native technologies have brought to us a transparency on everything going on in our system, from the code to the server. It has brought huge cost savings and a better way of dealing with those costs and keeping them under control. And performance-wise, it has helped our team understand how we can make our code work better on the cloud native infrastructure." +{{< /case-studies/quote >}} + +

Around the time Woorank was grappling with these issues, Kubernetes was emerging as a technology. De Moor knew that he wanted a platform that would be more automated and self-healing, and when he began experimenting with Kubernetes, he found that it checked all those boxes. "Kubernetes allows us to just define applications, how they need to run, how scalable they need to be, and it takes pain away from the developers thinking about that," he says. "When things fail and errors pop up, the system tries to heal itself, and that's really, for us, the key reason to work with Kubernetes. It allowed us to set up certain testing frameworks to just be alerted when things go wrong, instead of having to look at whether everything went right. It's made people's lives much easier. It's quite a big mindset change."

+ +

Once one small Kubernetes cluster was up and running, the team began moving over a few applications at a time, gradually increasing the load over the course of several months. By early 2017, Woorank was 100% deployed on Kubernetes.

+ +

The company's number one concern was immediately erased: Maintaining Kubernetes is the responsibility of just one person on staff, and it's not his fulltime job. Updating the old infrastructure "was always a pain," says De Moor: It used to take two active working days, "and it was always a bit scary when we did that." With Kubernetes, it's just a matter of "a few hours of passively following the process."

+ +{{< case-studies/quote + image="/images/case-studies/woorank/banner4.jpg" + author="NILS DE MOOR, CTO/COFOUNDER, WOORANK" +>}} +"When things fail and errors pop up, the system tries to heal itself, and that's really, for us, the key reason to work with Kubernetes. It allowed us to set up certain testing frameworks to just be alerted when things go wrong, instead of having to look at whether everything went right. It's made people's lives much easier. It's quite a big mindset change." +{{< /case-studies/quote >}} + +

Transparency on all levels, from the code to the servers, has also been a byproduct of the move to Kubernetes. "It's easier for the entire team to get a better understanding of the infrastructure, how it's working, how it looks like, what's going on," says De Moor. "It's not that thing that's running, and no one really knows how it works except this one person. Now it's really a team effort of everyone knowing, 'Okay, when something goes wrong, it's probably in this area or we need to check this.'"

+ +

To that end, Woorank has begun implementing other cloud native tools that help with visibility, such as Fluentd for logging, Prometheus for monitoring, and OpenTracing for distributed tracing. Implementing these new tools—which once took weeks of planning, installing, and onboarding—now only takes a few days. "With all the tools and projects under the CNCF umbrella, it's easier for us to test and play with technology than it used to be," says De Moor. "With Prometheus, we used it fairly early and couldn't get it fairly stable. A couple of months ago, the question reappeared, so we set it up in two days, and now everyone is using it."

+ +

Deployments, too, have been impacted: The rate has more than doubled, which De Moor partly attributes to the transparency of the new process. "With Kubernetes, you see that these three containers didn't start for this reason," he says. Plus, "now we bring deployment messages into Slack. If you see deployments rolling by every day, it does somehow indirectly enforce you, okay, I need to be part of this train, so I also need to deploy."

+ +{{< case-studies/quote author="NILS DE MOOR, CTO/COFOUNDER, WOORANK" >}} +"We can plan those things over a certain timeline, try to fit our resource usage to that, and then bring in spot instances, which will hopefully drive the costs down more." +{{< /case-studies/quote >}} + +

Perhaps the biggest impact, though, has been on the bottom line. "We were already pretty flexible in our costs and taking on traffic peaks and higher load in general, but with Kubernetes and the other CNCF tools we use, we have achieved about 30% in cost savings," says De Moor.

+ +

And there's room for even greater savings. Currently, most of Woorank's infrastructure is running on AWS on demand; the company pays a fixed price and makes some reservations for its planned amount of resources needed. De Moor is planning to experiment more with spot instances with certain resource-heavy workloads such as web crawls: "We can plan those things over a certain timeline, try to fit our resource usage to that, and then bring in spot instances, which will hopefully drive the costs down more."

+ +

Moving to Kubernetes has been so beneficial to Woorank that the company is doubling down on both cloud native technologies and the community. "It was definitely important for us to have CNCF as an umbrella above everything," says De Moor. "We've always been working with open source libraries and tools and technologies. It works very well for us, but sometimes things can drift, maintainers drop out, and projects go haywire. For us, it was indeed important to know that whatever project gets taken under this umbrella, it's taken very seriously. Our way of contributing back is also by joining this community. It's, for us, a way to show our appreciation for what's going on in this framework."

diff --git a/content/en/case-studies/workiva/index.html b/content/en/case-studies/workiva/index.html index 1c09503bfb..d8945bfd34 100644 --- a/content/en/case-studies/workiva/index.html +++ b/content/en/case-studies/workiva/index.html @@ -3,111 +3,91 @@ title: Workiva Case Study linkTitle: Workiva case_study_styles: true cid: caseStudies -css: /css/style_case_studies.css draft: true featured: true weight: 20 quote: > With OpenTracing, my team was able to look at a trace and make optimization suggestions to another team without ever looking at their code. + +new_case_study_styles: true +heading_background: /images/case-studies/workiva/banner1.jpg +heading_title_logo: /images/workiva_logo.png +subheading: > + Using OpenTracing to Help Pinpoint the Bottlenecks +case_study_details: + - Company: Workiva + - Location: Ames, Iowa + - Industry: Enterprise Software --- -
-

CASE STUDY:
Using OpenTracing to Help Pinpoint the Bottlenecks +

Challenge

-

+

Workiva offers a cloud-based platform for managing and reporting business data. This SaaS product, Wdesk, is used by more than 70 percent of the Fortune 500 companies. As the company made the shift from a monolith to a more distributed, microservice-based system, "We had a number of people working on this, all on different teams, so we needed to identify what the issues were and where the bottlenecks were," says Senior Software Architect MacLeod Broad. With back-end code running on Google App Engine, Google Compute Engine, as well as Amazon Web Services, Workiva needed a tracing system that was agnostic of platform. While preparing one of the company's first products utilizing AWS, which involved a "sync and link" feature that linked data from spreadsheets built in the new application with documents created in the old application on Workiva's existing system, Broad's team found an ideal use case for tracing: There were circular dependencies, and optimizations often turned out to be micro-optimizations that didn't impact overall speed.

-
+

Solution

-
- Company  Workiva     Location  Ames, Iowa     Industry  Enterprise Software -
+

Broad's team introduced the platform-agnostic distributed tracing system OpenTracing to help them pinpoint the bottlenecks.

-
-
-
-
-

Challenge

- Workiva offers a cloud-based platform for managing and reporting business data. This SaaS product, Wdesk, is used by more than 70 percent of the Fortune 500 companies. As the company made the shift from a monolith to a more distributed, microservice-based system, "We had a number of people working on this, all on different teams, so we needed to identify what the issues were and where the bottlenecks were," says Senior Software Architect MacLeod Broad. With back-end code running on Google App Engine, Google Compute Engine, as well as Amazon Web Services, Workiva needed a tracing system that was agnostic of platform. While preparing one of the company’s first products utilizing AWS, which involved a "sync and link" feature that linked data from spreadsheets built in the new application with documents created in the old application on Workiva’s existing system, Broad’s team found an ideal use case for tracing: There were circular dependencies, and optimizations often turned out to be micro-optimizations that didn’t impact overall speed. - -
- -
- -
-

Solution

- Broad’s team introduced the platform-agnostic distributed tracing system OpenTracing to help them pinpoint the bottlenecks. -

Impact

- Now used throughout the company, OpenTracing produced immediate results. Software Engineer Michael Davis reports: "Tracing has given us immediate, actionable insight into how to improve our service. Through a combination of seeing where each call spends its time, as well as which calls are most often used, we were able to reduce our average response time by 95 percent (from 600ms to 30ms) in a single fix." -
+

Now used throughout the company, OpenTracing produced immediate results. Software Engineer Michael Davis reports: "Tracing has given us immediate, actionable insight into how to improve our service. Through a combination of seeing where each call spends its time, as well as which calls are most often used, we were able to reduce our average response time by 95 percent (from 600ms to 30ms) in a single fix."

-
-
-
-
-"With OpenTracing, my team was able to look at a trace and make optimization suggestions to another team without ever looking at their code."
— MacLeod Broad, Senior Software Architect at Workiva +{{< case-studies/quote author="MacLeod Broad, Senior Software Architect at Workiva" >}} +"With OpenTracing, my team was able to look at a trace and make optimization suggestions to another team without ever looking at their code." +{{< /case-studies/quote >}} -
-
-
-
-

Last fall, MacLeod Broad’s platform team at Workiva was prepping one of the company’s first products utilizing Amazon Web Services when they ran into a roadblock.

- Early on, Workiva’s backend had run mostly on Google App Engine. But things changed along the way as Workiva’s SaaS offering, Wdesk, a cloud-based platform for managing and reporting business data, grew its customer base to more than 70 percent of the Fortune 500 companies. "As customer needs grew and the product offering expanded, we started to leverage a wider offering of services such as Amazon Web Services as well as other Google Cloud Platform services, creating a multi-vendor environment."

-With this new product, there was a "sync and link" feature by which data "went through a whole host of services starting with the new spreadsheet system [Amazon Aurora] into what we called our linking system, and then pushed through http to our existing system, and then a number of calculations would go on, and the results would be transmitted back into the new system," says Broad. "We were trying to optimize that for speed. We thought we had made this great optimization and then it would turn out to be a micro optimization, which didn’t really affect the overall speed of things."

-The challenges faced by Broad’s team may sound familiar to other companies that have also made the shift from monoliths to more distributed, microservice-based systems. "We had a number of people working on this, all on different teams, so it was difficult to get our head around what the issues were and where the bottlenecks were," says Broad.

- "Each service team was going through different iterations of their architecture and it was very hard to follow what was actually going on in each teams’ system," he adds. "We had circular dependencies where we’d have three or four different service teams unsure of where the issues really were, requiring a lot of back and forth communication. So we wasted a lot of time saying, ‘What part of this is slow? Which part of this is sometimes slow depending on the use case? Which part is degrading over time? Which part of this process is asynchronous so it doesn’t really matter if it’s long-running or not? What are we doing that’s redundant, and which part of this is buggy?’" +{{< case-studies/lead >}} +Last fall, MacLeod Broad's platform team at Workiva was prepping one of the company's first products utilizing Amazon Web Services when they ran into a roadblock. +{{< /case-studies/lead >}} +

Early on, Workiva's backend had run mostly on Google App Engine. But things changed along the way as Workiva's SaaS offering, Wdesk, a cloud-based platform for managing and reporting business data, grew its customer base to more than 70 percent of the Fortune 500 companies. "As customer needs grew and the product offering expanded, we started to leverage a wider offering of services such as Amazon Web Services as well as other Google Cloud Platform services, creating a multi-vendor environment."

-
-
-
-
- "A tracing system can at a glance explain an architecture, narrow down a performance bottleneck and zero in on it, and generally just help direct an investigation at a high level. Being able to do that at a glance is much faster than at a meeting or with three days of debugging, and it’s a lot faster than never figuring out the problem and just moving on."
— MACLEOD BROAD, SENIOR SOFTWARE ARCHITECT AT WORKIVA -
-
-
-
+

With this new product, there was a "sync and link" feature by which data "went through a whole host of services starting with the new spreadsheet system [Amazon Aurora] into what we called our linking system, and then pushed through http to our existing system, and then a number of calculations would go on, and the results would be transmitted back into the new system," says Broad. "We were trying to optimize that for speed. We thought we had made this great optimization and then it would turn out to be a micro optimization, which didn't really affect the overall speed of things."

-Simply put, it was an ideal use case for tracing. "A tracing system can at a glance explain an architecture, narrow down a performance bottleneck and zero in on it, and generally just help direct an investigation at a high level," says Broad. "Being able to do that at a glance is much faster than at a meeting or with three days of debugging, and it’s a lot faster than never figuring out the problem and just moving on."

-With Workiva’s back-end code running on Google Compute Engine as well as App Engine and AWS, Broad knew that he needed a tracing system that was platform agnostic. "We were looking at different tracing solutions," he says, "and we decided that because it seemed to be a very evolving market, we didn’t want to get stuck with one vendor. So OpenTracing seemed like the cleanest way to avoid vendor lock-in on what backend we actually had to use."

-Once they introduced OpenTracing into this first use case, Broad says, "The trace made it super obvious where the bottlenecks were." Even though everyone had assumed it was Workiva’s existing code that was slowing things down, that wasn’t exactly the case. "It looked like the existing code was slow only because it was reaching out to our next-generation services, and they were taking a very long time to service all those requests," says Broad. "On the waterfall graph you can see the exact same work being done on every request when it was calling back in. So every service request would look the exact same for every response being paged out. And then it was just a no-brainer of, ‘Why is it doing all this work again?’"

-Using the insight OpenTracing gave them, "My team was able to look at a trace and make optimization suggestions to another team without ever looking at their code," says Broad. "The way we named our traces gave us insight whether it’s doing a SQL call or it’s making an RPC. And so it was really easy to say, ‘OK, we know that it’s going to page through all these requests. Do the work once and stuff it in cache.’ And we were done basically. All those calls became sub-second calls immediately."

+

The challenges faced by Broad's team may sound familiar to other companies that have also made the shift from monoliths to more distributed, microservice-based systems. "We had a number of people working on this, all on different teams, so it was difficult to get our head around what the issues were and where the bottlenecks were," says Broad.

+

"Each service team was going through different iterations of their architecture and it was very hard to follow what was actually going on in each teams' system," he adds. "We had circular dependencies where we'd have three or four different service teams unsure of where the issues really were, requiring a lot of back and forth communication. So we wasted a lot of time saying, 'What part of this is slow? Which part of this is sometimes slow depending on the use case? Which part is degrading over time? Which part of this process is asynchronous so it doesn't really matter if it's long-running or not? What are we doing that's redundant, and which part of this is buggy?'"

+{{< case-studies/quote + image="/images/case-studies/workiva/banner3.jpg" + author="MACLEOD BROAD, SENIOR SOFTWARE ARCHITECT AT WORKIVA" +>}} +"A tracing system can at a glance explain an architecture, narrow down a performance bottleneck and zero in on it, and generally just help direct an investigation at a high level. Being able to do that at a glance is much faster than at a meeting or with three days of debugging, and it's a lot faster than never figuring out the problem and just moving on." +{{< /case-studies/quote >}} -
-
-
-
-"We were looking at different tracing solutions and we decided that because it seemed to be a very evolving market, we didn’t want to get stuck with one vendor. So OpenTracing seemed like the cleanest way to avoid vendor lock-in on what backend we actually had to use."
— MACLEOD BROAD, SENIOR SOFTWARE ARCHITECT AT WORKIVA -
-
+

Simply put, it was an ideal use case for tracing. "A tracing system can at a glance explain an architecture, narrow down a performance bottleneck and zero in on it, and generally just help direct an investigation at a high level," says Broad. "Being able to do that at a glance is much faster than at a meeting or with three days of debugging, and it's a lot faster than never figuring out the problem and just moving on."

-
-
- After the success of the first use case, everyone involved in the trial went back and fully instrumented their products. Tracing was added to a few more use cases. "We wanted to get through the initial implementation pains early without bringing the whole department along for the ride," says Broad. "Now, a lot of teams add it when they’re starting up a new service. We’re really pushing adoption now more than we were before."

-Some teams were won over quickly. "Tracing has given us immediate, actionable insight into how to improve our [Workspaces] service," says Software Engineer Michael Davis. "Through a combination of seeing where each call spends its time, as well as which calls are most often used, we were able to reduce our average response time by 95 percent (from 600ms to 30ms) in a single fix."

-Most of Workiva’s major products are now traced using OpenTracing, with data pushed into Google StackDriver. Even the products that aren’t fully traced have some components and libraries that are.

-Broad points out that because some of the engineers were working on App Engine and already had experience with the platform’s Appstats library for profiling performance, it didn’t take much to get them used to using OpenTracing. But others were a little more reluctant. "The biggest hindrance to adoption I think has been the concern about how much latency is introducing tracing [and StackDriver] going to cost," he says. "People are also very concerned about adding middleware to whatever they’re working on. Questions about passing the context around and how that’s done were common. A lot of our Go developers were fine with it, because they were already doing that in one form or another. Our Java developers were not super keen on doing that because they’d used other systems that didn’t require that."

-But the benefits clearly outweighed the concerns, and today, Workiva’s official policy is to use tracing." -In fact, Broad believes that tracing naturally fits in with Workiva’s existing logging and metrics systems. "This was the way we presented it internally, and also the way we designed our use," he says. "Our traces are logged in the exact same mechanism as our app metric and logging data, and they get pushed the exact same way. So we treat all that data exactly the same when it’s being created and when it’s being recorded. We have one internal library that we use for logging, telemetry, analytics and tracing." +

With Workiva's back-end code running on Google Compute Engine as well as App Engine and AWS, Broad knew that he needed a tracing system that was platform agnostic. "We were looking at different tracing solutions," he says, "and we decided that because it seemed to be a very evolving market, we didn't want to get stuck with one vendor. So OpenTracing seemed like the cleanest way to avoid vendor lock-in on what backend we actually had to use."

+

Once they introduced OpenTracing into this first use case, Broad says, "The trace made it super obvious where the bottlenecks were." Even though everyone had assumed it was Workiva's existing code that was slowing things down, that wasn't exactly the case. "It looked like the existing code was slow only because it was reaching out to our next-generation services, and they were taking a very long time to service all those requests," says Broad. "On the waterfall graph you can see the exact same work being done on every request when it was calling back in. So every service request would look the exact same for every response being paged out. And then it was just a no-brainer of, 'Why is it doing all this work again?'"

-
+

Using the insight OpenTracing gave them, "My team was able to look at a trace and make optimization suggestions to another team without ever looking at their code," says Broad. "The way we named our traces gave us insight whether it's doing a SQL call or it's making an RPC. And so it was really easy to say, 'OK, we know that it's going to page through all these requests. Do the work once and stuff it in cache.' And we were done basically. All those calls became sub-second calls immediately."

-
-
- "Tracing has given us immediate, actionable insight into how to improve our [Workspaces] service. Through a combination of seeing where each call spends its time, as well as which calls are most often used, we were able to reduce our average response time by 95 percent (from 600ms to 30ms) in a single fix."
— Michael Davis, Software Engineer, Workiva -
-
+{{< case-studies/quote + image="/images/case-studies/workiva/banner4.jpg" + author="MACLEOD BROAD, SENIOR SOFTWARE ARCHITECT AT WORKIVA" +>}} +"We were looking at different tracing solutions and we decided that because it seemed to be a very evolving market, we didn't want to get stuck with one vendor. So OpenTracing seemed like the cleanest way to avoid vendor lock-in on what backend we actually had to use." +{{< /case-studies/quote >}} -
- For Workiva, OpenTracing has become an essential tool for zeroing in on optimizations and determining what’s actually a micro-optimization by observing usage patterns. "On some projects we often assume what the customer is doing, and we optimize for these crazy scale cases that we hit 1 percent of the time," says Broad. "It’s been really helpful to be able to say, ‘OK, we’re adding 100 milliseconds on every request that does X, and we only need to add that 100 milliseconds if it’s the worst of the worst case, which only happens one out of a thousand requests or one out of a million requests."

-Unlike many other companies, Workiva also traces the client side. "For us, the user experience is important—it doesn’t matter if the RPC takes 100 milliseconds if it still takes 5 seconds to do the rendering to show it in the browser," says Broad. "So for us, those client times are important. We trace it to see what parts of loading take a long time. We’re in the middle of working on a definition of what is ‘loaded.’ Is it when you have it, or when it’s rendered, or when you can interact with it? Those are things we’re planning to use tracing for to keep an eye on and to better understand."

-That also requires adjusting for differences in external and internal clocks. "Before time correcting, it was horrible; our traces were more misleading than anything," says Broad. "So we decided that we would return a timestamp on the response headers, and then have the client reorient its time based on that—not change its internal clock but just calculate the offset on the response time to when the client got it. And if you end up in an impossible situation where a client RPC spans 210 milliseconds but the time on the response time is outside of that window, then we have to reorient that."

-Broad is excited about the impact OpenTracing has already had on the company, and is also looking ahead to what else the technology can enable. One possibility is using tracing to update documentation in real time. "Keeping documentation up to date with reality is a big challenge," he says. "Say, we just ran a trace simulation or we just ran a smoke test on this new deploy, and the architecture doesn’t match the documentation. We can find whose responsibility it is and let them know and have them update it. That’s one of the places I’d like to get in the future with tracing." +

After the success of the first use case, everyone involved in the trial went back and fully instrumented their products. Tracing was added to a few more use cases. "We wanted to get through the initial implementation pains early without bringing the whole department along for the ride," says Broad. "Now, a lot of teams add it when they're starting up a new service. We're really pushing adoption now more than we were before."

-
+

Some teams were won over quickly. "Tracing has given us immediate, actionable insight into how to improve our [Workspaces] service," says Software Engineer Michael Davis. "Through a combination of seeing where each call spends its time, as well as which calls are most often used, we were able to reduce our average response time by 95 percent (from 600ms to 30ms) in a single fix."

-
+

Most of Workiva's major products are now traced using OpenTracing, with data pushed into Google StackDriver. Even the products that aren't fully traced have some components and libraries that are.

+ +

Broad points out that because some of the engineers were working on App Engine and already had experience with the platform's Appstats library for profiling performance, it didn't take much to get them used to using OpenTracing. But others were a little more reluctant. "The biggest hindrance to adoption I think has been the concern about how much latency is introducing tracing [and StackDriver] going to cost," he says. "People are also very concerned about adding middleware to whatever they're working on. Questions about passing the context around and how that's done were common. A lot of our Go developers were fine with it, because they were already doing that in one form or another. Our Java developers were not super keen on doing that because they'd used other systems that didn't require that."But the benefits clearly outweighed the concerns, and today, Workiva's official policy is to use tracing."

+ +

In fact, Broad believes that tracing naturally fits in with Workiva's existing logging and metrics systems. "This was the way we presented it internally, and also the way we designed our use," he says. "Our traces are logged in the exact same mechanism as our app metric and logging data, and they get pushed the exact same way. So we treat all that data exactly the same when it's being created and when it's being recorded. We have one internal library that we use for logging, telemetry, analytics and tracing."

+ +{{< case-studies/quote author="Michael Davis, Software Engineer, Workiva" >}} +"Tracing has given us immediate, actionable insight into how to improve our [Workspaces] service. Through a combination of seeing where each call spends its time, as well as which calls are most often used, we were able to reduce our average response time by 95 percent (from 600ms to 30ms) in a single fix." +{{< /case-studies/quote >}} + +

For Workiva, OpenTracing has become an essential tool for zeroing in on optimizations and determining what's actually a micro-optimization by observing usage patterns. "On some projects we often assume what the customer is doing, and we optimize for these crazy scale cases that we hit 1 percent of the time," says Broad. "It's been really helpful to be able to say, 'OK, we're adding 100 milliseconds on every request that does X, and we only need to add that 100 milliseconds if it's the worst of the worst case, which only happens one out of a thousand requests or one out of a million requests."

+ +

Unlike many other companies, Workiva also traces the client side. "For us, the user experience is important—it doesn't matter if the RPC takes 100 milliseconds if it still takes 5 seconds to do the rendering to show it in the browser," says Broad. "So for us, those client times are important. We trace it to see what parts of loading take a long time. We're in the middle of working on a definition of what is 'loaded.' Is it when you have it, or when it's rendered, or when you can interact with it? Those are things we're planning to use tracing for to keep an eye on and to better understand."

+ +

That also requires adjusting for differences in external and internal clocks. "Before time correcting, it was horrible; our traces were more misleading than anything," says Broad. "So we decided that we would return a timestamp on the response headers, and then have the client reorient its time based on that—not change its internal clock but just calculate the offset on the response time to when the client got it. And if you end up in an impossible situation where a client RPC spans 210 milliseconds but the time on the response time is outside of that window, then we have to reorient that."

+ +

Broad is excited about the impact OpenTracing has already had on the company, and is also looking ahead to what else the technology can enable. One possibility is using tracing to update documentation in real time. "Keeping documentation up to date with reality is a big challenge," he says. "Say, we just ran a trace simulation or we just ran a smoke test on this new deploy, and the architecture doesn't match the documentation. We can find whose responsibility it is and let them know and have them update it. That's one of the places I'd like to get in the future with tracing."

diff --git a/content/en/case-studies/ygrene/index.html b/content/en/case-studies/ygrene/index.html index c07443249a..9afc1ec45b 100644 --- a/content/en/case-studies/ygrene/index.html +++ b/content/en/case-studies/ygrene/index.html @@ -1,111 +1,82 @@ --- title: Ygrene Case Study - linkTitle: Ygrene case_study_styles: true cid: caseStudies -css: /css/style_case_studies.css logo: ygrene_featured_logo.png featured: true weight: 48 quote: > - We had to change some practices and code, and the way things were built, but we were able to get our main systems onto Kubernetes in a month or so, and then into production within two months. That’s very fast for a finance company. + We had to change some practices and code, and the way things were built, but we were able to get our main systems onto Kubernetes in a month or so, and then into production within two months. That's very fast for a finance company. + +new_case_study_styles: true +heading_background: /images/case-studies/ygrene/banner1.jpg +heading_title_logo: /images/ygrene_logo.png +subheading: > + Ygrene: Using Cloud Native to Bring Security and Scalability to the Finance Industry +case_study_details: + - Company: Ygrene + - Location: Petaluma, Calif. + - Industry: Clean energy financing --- -
-

CASE STUDY:
Ygrene: Using Cloud Native to Bring Security and Scalability to the Finance Industry +

Challenge

-

+

A PACE (Property Assessed Clean Energy) financing company, Ygrene has funded more than $1 billion in loans since 2010. In order to approve and process those loans, "We have lots of data sources that are being aggregated, and we also have lots of systems that need to churn on that data," says Ygrene Development Manager Austin Adams. The company was utilizing massive servers, and "we just reached the limit of being able to scale them vertically. We had a really unstable system that became overwhelmed with requests just for doing background data processing in real time. The performance the users saw was very poor. We needed a solution that wouldn't require us to make huge refactors to the code base." As a finance company, Ygrene also needed to ensure that they were shipping their applications securely.

-
+

Solution

-
- Company  Ygrene     Location  Petaluma, Calif.     Industry  Clean energy financing -
- -
-
-
-
-

Challenge

- A PACE (Property Assessed Clean Energy) financing company, Ygrene has funded more than $1 billion in loans since 2010. In order to approve and process those loans, "We have lots of data sources that are being aggregated, and we also have lots of systems that need to churn on that data," says Ygrene Development Manager Austin Adams. The company was utilizing massive servers, and "we just reached the limit of being able to scale them vertically. We had a really unstable system that became overwhelmed with requests just for doing background data processing in real time. The performance the users saw was very poor. We needed a solution that wouldn’t require us to make huge refactors to the code base." As a finance company, Ygrene also needed to ensure that they were shipping their applications securely. - -
- -

Solution

- Moving from an Engine Yard platform and Amazon Elastic Beanstalk, the Ygrene team embraced cloud native technologies and practices: Kubernetes to help scale out vertically and distribute workloads, Notary to put in build-time controls and get trust on the Docker images being used with third-party dependencies, and Fluentd for "observing every part of our stack," all running on Amazon EC2 Spot. - -
- -
+

Moving from an Engine Yard platform and Amazon Elastic Beanstalk, the Ygrene team embraced cloud native technologies and practices: Kubernetes to help scale out vertically and distribute workloads, Notary to put in build-time controls and get trust on the Docker images being used with third-party dependencies, and Fluentd for "observing every part of our stack," all running on Amazon EC2 Spot.

Impact

- Before, deployments typically took three to four hours, and two or three months’ worth of work would be deployed at low-traffic times every week or two weeks. Now, they take five minutes for Kubernetes, and an hour for the overall deploy with smoke testing. And "we’re able to deploy three or four times a week, with just one week’s or two days’ worth of work," Adams says. "We’re deploying during the work week, in the daytime and without any downtime. We had to ask for business approval to take the systems down, even in the middle of the night, because people could be doing loans. Now we can deploy, ship code, and migrate databases, all without taking the system down. The company gets new features without worrying that some business will be lost or delayed." Additionally, by using the kops project, Ygrene can now run its Kubernetes clusters with AWS EC2 Spot, at a tenth of the previous cost. These cloud native technologies have "changed the game for scalability, observability, and security—we’re adding new data sources that are very secure," says Adams. "Without Kubernetes, Notary, and Fluentd, we couldn’t tell our investors and team members that we knew what was going on." +

Before, deployments typically took three to four hours, and two or three months' worth of work would be deployed at low-traffic times every week or two weeks. Now, they take five minutes for Kubernetes, and an hour for the overall deploy with smoke testing. And "we're able to deploy three or four times a week, with just one week's or two days' worth of work," Adams says. "We're deploying during the work week, in the daytime and without any downtime. We had to ask for business approval to take the systems down, even in the middle of the night, because people could be doing loans. Now we can deploy, ship code, and migrate databases, all without taking the system down. The company gets new features without worrying that some business will be lost or delayed." Additionally, by using the kops project, Ygrene can now run its Kubernetes clusters with AWS EC2 Spot, at a tenth of the previous cost. These cloud native technologies have "changed the game for scalability, observability, and security—we're adding new data sources that are very secure," says Adams. "Without Kubernetes, Notary, and Fluentd, we couldn't tell our investors and team members that we knew what was going on."

-
+{{< case-studies/quote author="Austin Adams, Development Manager, Ygrene Energy Fund" >}} +"CNCF projects are helping Ygrene determine the security and observability standards for the entire PACE industry. We're an emerging finance industry, and without these projects, especially Kubernetes, we couldn't be the industry leader that we are today." +{{< /case-studies/quote >}} -
-
-
-
-"CNCF projects are helping Ygrene determine the security and observability standards for the entire PACE industry. We’re an emerging finance industry, and without these projects, especially Kubernetes, we couldn’t be the industry leader that we are today."

— Austin Adams, Development Manager, Ygrene Energy Fund +{{< case-studies/lead >}} +In less than a decade, Ygrene has funded more than $1 billion in loans for renewable energy projects. +{{< /case-studies/lead >}} -
-
+

A PACE (Property Assessed Clean Energy) financing company, "We take the equity in a home or a commercial building, and use it to finance property improvements for anything that saves electricity, produces electricity, saves water, or reduces carbon emissions," says Development Manager Austin Adams.

-
-

In less than a decade, Ygrene has funded more than $1 billion in loans for renewable energy projects.

A PACE (Property Assessed Clean Energy) financing company, "We take the equity in a home or a commercial building, and use it to finance property improvements for anything that saves electricity, produces electricity, saves water, or reduces carbon emissions," says Development Manager Austin Adams.

-In order to approve those loans, the company processes an enormous amount of underwriting data. "We have tons of different points that we have to validate about the property, about the company, or about the person," Adams says. "So we have lots of data sources that are being aggregated, and we also have lots of systems that need to churn on that data in real time."

-By 2017, deployments and scalability had become pain points. The company was utilizing massive servers, and "we just reached the limit of being able to scale them vertically," he says. Migrating to AWS Elastic Beanstalk didn’t solve the problem: "The Scala services needed a lot of data from the main Ruby on Rails services and from different vendors, so they were asking for information from our Ruby services at a rate that those services couldn’t handle. We had lots of configuration misses with Elastic Beanstalk as well. It just came to a head, and we realized we had a really unstable system." +

In order to approve those loans, the company processes an enormous amount of underwriting data. "We have tons of different points that we have to validate about the property, about the company, or about the person," Adams says. "So we have lots of data sources that are being aggregated, and we also have lots of systems that need to churn on that data in real time."

-
- -
-
- "CNCF has been an amazing incubator for so many projects. Now we look at its webpage regularly to find out if there are any new, awesome, high-quality projects we can implement into our stack. It’s actually become a hub for us for knowing what software we need to be looking at to make our systems more secure or more scalable."

— Austin Adams, Development Manager, Ygrene Energy Fund -
-
-
-
+

By 2017, deployments and scalability had become pain points. The company was utilizing massive servers, and "we just reached the limit of being able to scale them vertically," he says. Migrating to AWS Elastic Beanstalk didn't solve the problem: "The Scala services needed a lot of data from the main Ruby on Rails services and from different vendors, so they were asking for information from our Ruby services at a rate that those services couldn't handle. We had lots of configuration misses with Elastic Beanstalk as well. It just came to a head, and we realized we had a really unstable system."

-Adams along with the rest of the team set out to find a solution that would be transformational, but "wouldn’t require us to make huge refactors to the code base," he says. And as a finance company, Ygrene needed security as much as scalability. They found the answer by embracing cloud native technologies: Kubernetes to help scale out vertically and distribute workloads, Notary to achieve reliable security at every level, and Fluentd for observability. "Kubernetes was where the community was going, and we wanted to be future proof," says Adams.

-With Kubernetes, the team was able to quickly containerize the Ygrene application with Docker. "We had to change some practices and code, and the way things were built," Adams says, "but we were able to get our main systems onto Kubernetes in a month or so, and then into production within two months. That’s very fast for a finance company."

-How? Cloud native has "changed the game for scalability, observability, and security—we’re adding new data sources that are very secure," says Adams. "Without Kubernetes, Notary, and Fluentd, we couldn’t tell our investors and team members that we knew what was going on."

-Notary, in particular, "has been a godsend," says Adams. "We need to know that our attack surface on third-party dependencies is low, or at least managed. We use it as a trust system and we also use it as a separation, so production images are signed by Notary, but some development images we don’t sign. That is to ensure that they can’t get into the production cluster. We’ve been using it in the test cluster to feel more secure about our builds." +{{< case-studies/quote + image="/images/case-studies/ygrene/banner3.jpg" + author="Austin Adams, Development Manager, Ygrene Energy Fund" +>}} +"CNCF has been an amazing incubator for so many projects. Now we look at its webpage regularly to find out if there are any new, awesome, high-quality projects we can implement into our stack. It's actually become a hub for us for knowing what software we need to be looking at to make our systems more secure or more scalable." +{{< /case-studies/quote >}} +

Adams along with the rest of the team set out to find a solution that would be transformational, but "wouldn't require us to make huge refactors to the code base," he says. And as a finance company, Ygrene needed security as much as scalability. They found the answer by embracing cloud native technologies: Kubernetes to help scale out vertically and distribute workloads, Notary to achieve reliable security at every level, and Fluentd for observability. "Kubernetes was where the community was going, and we wanted to be future proof," says Adams.

+

With Kubernetes, the team was able to quickly containerize the Ygrene application with Docker. "We had to change some practices and code, and the way things were built," Adams says, "but we were able to get our main systems onto Kubernetes in a month or so, and then into production within two months. That's very fast for a finance company."

-
-
-
-
-"We had to change some practices and code, and the way things were built," Adams says, "but we were able to get our main systems onto Kubernetes in a month or so, and then into production within two months. That’s very fast for a finance company." -
-
+

How? Cloud native has "changed the game for scalability, observability, and security—we're adding new data sources that are very secure," says Adams. "Without Kubernetes, Notary, and Fluentd, we couldn't tell our investors and team members that we knew what was going on."

-
-
- By using the kops project, Ygrene was able to move from Elastic Beanstalk to running its Kubernetes clusters on AWS EC2 Spot, at a tenth of the previous cost. "In order to scale before, we would need to up our instance sizes, incurring high cost for low value," says Adams. "Now with Kubernetes and kops, we are able to scale horizontally on Spot with multiple instance groups."

-That also helped them mitigate the risk that comes with running in the public cloud. "We figured out, essentially, that if we’re able to select instance classes using EC2 Spot that had an extremely low likelihood of interruption and zero history of interruption, and we’re willing to pay a price high enough, that we could virtually get the same guarantee using Kubernetes because we have enough nodes," says Software Engineer Zach Arnold, who led the migration to Kubernetes. "Now that we’ve re-architected these pieces of the application to not live on the same server, we can push out to many different servers and have a more stable deployment."

-As a result, the team can now ship code any time of day. "That was risky because it could bring down your whole loan management software with it," says Arnold. "But we now can deploy safely and securely during the day." +

Notary, in particular, "has been a godsend," says Adams. "We need to know that our attack surface on third-party dependencies is low, or at least managed. We use it as a trust system and we also use it as a separation, so production images are signed by Notary, but some development images we don't sign. That is to ensure that they can't get into the production cluster. We've been using it in the test cluster to feel more secure about our builds."

+{{< case-studies/quote image="/images/case-studies/ygrene/banner4.jpg">}} +"We had to change some practices and code, and the way things were built," Adams says, "but we were able to get our main systems onto Kubernetes in a month or so, and then into production within two months. That's very fast for a finance company." +{{< /case-studies/quote >}} -
+

By using the kops project, Ygrene was able to move from Elastic Beanstalk to running its Kubernetes clusters on AWS EC2 Spot, at a tenth of the previous cost. "In order to scale before, we would need to up our instance sizes, incurring high cost for low value," says Adams. "Now with Kubernetes and kops, we are able to scale horizontally on Spot with multiple instance groups."

-
-
- "In order to scale before, we would need to up our instance sizes, incurring high cost for low value," says Adams. "Now with Kubernetes and kops, we are able to scale horizontally on Spot with multiple instance groups." -
-
+

That also helped them mitigate the risk that comes with running in the public cloud. "We figured out, essentially, that if we're able to select instance classes using EC2 Spot that had an extremely low likelihood of interruption and zero history of interruption, and we're willing to pay a price high enough, that we could virtually get the same guarantee using Kubernetes because we have enough nodes," says Software Engineer Zach Arnold, who led the migration to Kubernetes. "Now that we've re-architected these pieces of the application to not live on the same server, we can push out to many different servers and have a more stable deployment."

-
- Before, deployments typically took three to four hours, and two or three months’ worth of work would be deployed at low-traffic times every week or two weeks. Now, they take five minutes for Kubernetes, and an hour for an overall deploy with smoke testing. And "we’re able to deploy three or four times a week, with just one week’s or two days’ worth of work," Adams says. "We’re deploying during the work week, in the daytime and without any downtime. We had to ask for business approval to take the systems down for 30 minutes to an hour, even in the middle of the night, because people could be doing loans. Now we can deploy, ship code, and migrate databases, all without taking the system down. The company gets new features without worrying that some business will be lost or delayed."

-Cloud native also affected how Ygrene’s 50+ developers and contractors work. Adams and Arnold spent considerable time "teaching people to think distributed out of the box," says Arnold. "We ended up picking what we call the Four S’s of Shipping: safely, securely, stably, and speedily." (For more on the security piece of it, see their article on their "continuous hacking" strategy.) As for the engineers, says Adams, "they have been able to advance as their software has advanced. I think that at the end of the day, the developers feel better about what they’re doing, and they also feel more connected to the modern software development community."

-Looking ahead, Adams is excited to explore more CNCF projects, including SPIFFE and SPIRE. "CNCF has been an amazing incubator for so many projects," he says. "Now we look at its webpage regularly to find out if there are any new, awesome, high-quality projects we can implement into our stack. It’s actually become a hub for us for knowing what software we need to be looking at to make our systems more secure or more scalable." +

As a result, the team can now ship code any time of day. "That was risky because it could bring down your whole loan management software with it," says Arnold. "But we now can deploy safely and securely during the day."

+{{< case-studies/quote >}} +"In order to scale before, we would need to up our instance sizes, incurring high cost for low value," says Adams. "Now with Kubernetes and kops, we are able to scale horizontally on Spot with multiple instance groups." +{{< /case-studies/quote >}} +

Before, deployments typically took three to four hours, and two or three months' worth of work would be deployed at low-traffic times every week or two weeks. Now, they take five minutes for Kubernetes, and an hour for an overall deploy with smoke testing. And "we're able to deploy three or four times a week, with just one week's or two days' worth of work," Adams says. "We're deploying during the work week, in the daytime and without any downtime. We had to ask for business approval to take the systems down for 30 minutes to an hour, even in the middle of the night, because people could be doing loans. Now we can deploy, ship code, and migrate databases, all without taking the system down. The company gets new features without worrying that some business will be lost or delayed."

-
+

Cloud native also affected how Ygrene's 50+ developers and contractors work. Adams and Arnold spent considerable time "teaching people to think distributed out of the box," says Arnold. "We ended up picking what we call the Four S's of Shipping: safely, securely, stably, and speedily." (For more on the security piece of it, see their article on their "continuous hacking" strategy.) As for the engineers, says Adams, "they have been able to advance as their software has advanced. I think that at the end of the day, the developers feel better about what they're doing, and they also feel more connected to the modern software development community."

-
+

Looking ahead, Adams is excited to explore more CNCF projects, including SPIFFE and SPIRE. "CNCF has been an amazing incubator for so many projects," he says. "Now we look at its webpage regularly to find out if there are any new, awesome, high-quality projects we can implement into our stack. It's actually become a hub for us for knowing what software we need to be looking at to make our systems more secure or more scalable."

diff --git a/content/en/case-studies/zalando/index.html b/content/en/case-studies/zalando/index.html index 49bad6ff9e..23363da401 100644 --- a/content/en/case-studies/zalando/index.html +++ b/content/en/case-studies/zalando/index.html @@ -1,101 +1,83 @@ --- title: Zalando Case Study - case_study_styles: true cid: caseStudies -css: /css/style_zalando.css + +new_case_study_styles: true +heading_background: /images/case-studies/zalando/banner1.jpg +heading_title_logo: /images/zalando_logo.png +subheading: > + Europe's Leading Online Fashion Platform Gets Radical with Cloud Native +case_study_details: + - Company: Zalando + - Location: Berlin, Germany + - Industry: Online Fashion --- -
-

CASE STUDY:
Europe’s Leading Online Fashion Platform Gets Radical with Cloud Native -

+

Challenge

-
+

Zalando, Europe's leading online fashion platform, has experienced exponential growth since it was founded in 2008. In 2015, with plans to further expand its original e-commerce site to include new services and products, Zalando embarked on a radical transformation resulting in autonomous self-organizing teams. This change requires an infrastructure that could scale with the growth of the engineering organization. Zalando's technology department began rewriting its applications to be cloud-ready and started moving its infrastructure from on-premise data centers to the cloud. While orchestration wasn't immediately considered, as teams migrated to Amazon Web Services (AWS): "We saw the pain teams were having with infrastructure and Cloud Formation on AWS," says Henning Jacobs, Head of Developer Productivity. "There's still too much operational overhead for the teams and compliance. " To provide better support, cluster management was brought into play.

-
- Company  Zalando     Location  Berlin, Germany     Industry  Online Fashion -
+

Solution

-
-
-
-
-

Challenge

- Zalando, Europe’s leading online fashion platform, has experienced exponential growth since it was founded in 2008. In 2015, with plans to further expand its original e-commerce site to include new services and products, Zalando embarked on a radical transformation resulting in autonomous self-organizing teams. This change requires an infrastructure that could scale with the growth of the engineering organization. Zalando’s technology department began rewriting its applications to be cloud-ready and started moving its infrastructure from on-premise data centers to the cloud. While orchestration wasn’t immediately considered, as teams migrated to Amazon Web Services (AWS): "We saw the pain teams were having with infrastructure and Cloud Formation on AWS," says Henning Jacobs, Head of Developer Productivity. "There’s still too much operational overhead for the teams and compliance. " To provide better support, cluster management was brought into play. +

The company now runs its Docker containers on AWS using Kubernetes orchestration.

-
- -
-

Solution

- The company now runs its Docker containers on AWS using Kubernetes orchestration. -

Impact

- With the old infrastructure "it was difficult to properly embrace new technologies, and DevOps teams were considered to be a bottleneck," says Jacobs. "Now, with this cloud infrastructure, they have this packaging format, which can contain anything that runs on the Linux kernel. This makes a lot of people pretty happy. The engineers love autonomy." -
-
-
-
-
- "We envision all Zalando delivery teams running their containerized applications on a state-of-the-art, reliable and scalable cluster infrastructure provided by Kubernetes."

- Henning Jacobs, Head of Developer Productivity at Zalando -
-
-
-
-

When Henning Jacobs arrived at Zalando in 2010, the company was just two years old with 180 employees running an online store for European shoppers to buy fashion items.

- "It started as a PHP e-commerce site which was easy to get started with, but was not scaling with the business' needs" says Jacobs, Head of Developer Productivity at Zalando.

- At that time, the company began expanding beyond its German origins into other European markets. Fast-forward to today and Zalando now has more than 14,000 employees, 3.6 billion Euro in revenue for 2016 and operates across 15 countries. "With growth in all dimensions, and constant scaling, it has been a once-in-a-lifetime experience," he says.

- Not to mention a unique opportunity for an infrastructure specialist like Jacobs. Just after he joined, the company began rewriting all their applications in-house. "That was generally our strategy," he says. "For example, we started with our own logistics warehouses but at first you don’t know how to do logistics software, so you have some vendor software. And then we replaced it with our own because with off-the-shelf software you’re not competitive. You need to optimize these processes based on your specific business needs."

- In parallel to rewriting their applications, Zalando had set a goal of expanding beyond basic e-commerce to a platform offering multi-tenancy, a dramatic increase in assortments and styles, same-day delivery and even your own personal online stylist.

- The need to scale ultimately led the company on a cloud-native journey. As did its embrace of a microservices-based software architecture that gives engineering teams more autonomy and ownership of projects. "This move to the cloud was necessary because in the data center you couldn’t have autonomous teams. You have the same infrastructure and it was very homogeneous, so you could only run your Java or Python app," Jacobs says. -
-
-
-
- "This move to the cloud was necessary because in the data center you couldn’t have autonomous teams. You have the same infrastructure and it was very homogeneous, so you could only run your Java or Python app." +

With the old infrastructure "it was difficult to properly embrace new technologies, and DevOps teams were considered to be a bottleneck," says Jacobs. "Now, with this cloud infrastructure, they have this packaging format, which can contain anything that runs on the Linux kernel. This makes a lot of people pretty happy. The engineers love autonomy."

-
-
-
-
- Zalando began moving its infrastructure from two on-premise data centers to the cloud, requiring the migration of older applications for cloud-readiness. "We decided to have a clean break," says Jacobs. "Our Amazon Web Services infrastructure was set up like so: Every team has its own AWS account, which is completely isolated, meaning there’s no ‘lift and shift.’ You basically have to rewrite your application to make it cloud-ready even down to the persistence layer. We bravely went back to the drawing board and redid everything, first choosing Docker as a common containerization, then building the infrastructure from there."

- The company decided to hold off on orchestration at the beginning, but as teams were migrated to AWS, "we saw the pain teams were having with infrastructure and cloud formation on AWS," says Jacobs.

- Zalandos 200+ autonomous engineering teams decide what technologies to use and could operate their own applications using their own AWS accounts. This setup proved to be a compliance challenge. Even with strict rules-of-play and automated compliance checks in place, engineering teams and IT-compliance were overburdened addressing compliance issues. "Violations appear for non-compliant behavior, which we detect when scanning the cloud infrastructure," says Jacobs. "Everything is possible and nothing enforced, so you have to live with violations (and resolve them) instead of preventing the error in the first place. This means overhead for teams—and overhead for compliance and operations. It also takes time to spin up new EC2 instances on AWS, which affects our deployment velocity."

- The team realized they needed to "leverage the value you get from cluster management," says Jacobs. When they first looked at Platform as a Service (PaaS) options in 2015, the market was fragmented; but "now there seems to be a clear winner. It seemed like a good bet to go with Kubernetes."

- The transition to Kubernetes started in 2016 during Zalando’s Hack Week where participants deployed their projects to a Kubernetes cluster. From there 60 members of the tech infrastructure department were on-boarded - and then engineering teams were brought on one at a time. "We always start by talking with them and make sure everyone’s expectations are clear," says Jacobs. "Then we conduct some Kubernetes training, which is mostly training for our CI/CD setup, because the user interface for our users is primarily through the CI/CD system. But they have to know fundamental Kubernetes concepts and the API. This is followed by a weekly sync with each team to check their progress. Once they have something in production, we want to see if everything is fine on top of what we can improve." +{{< case-studies/quote author="Henning Jacobs, Head of Developer Productivity at Zalando" >}} +"We envision all Zalando delivery teams running their containerized applications on a state-of-the-art, reliable and scalable cluster infrastructure provided by Kubernetes." +{{< /case-studies/quote >}} -
-
-
-
- Once Zalando began migrating applications to Kubernetes, the results were immediate. "Kubernetes is a cornerstone for our seamless end-to-end developer experience. We are able to ship ideas to production using a single consistent and declarative API," says Jacobs. -
-
+{{< case-studies/lead >}} +When Henning Jacobs arrived at Zalando in 2010, the company was just two years old with 180 employees running an online store for European shoppers to buy fashion items. +{{< /case-studies/lead >}} -
-
- At the moment, Zalando is running an initial 40 Kubernetes clusters with plans to scale for the foreseeable future. - Once Zalando began migrating applications to Kubernetes, the results were immediate. "Kubernetes is a cornerstone for our seamless end-to-end developer experience. We are able to ship ideas to production using a single consistent and declarative API," says Jacobs. "The self-healing infrastructure provides a frictionless experience with higher-level abstractions built upon low-level best practices. We envision all Zalando delivery teams will run their containerized applications on a state-of-the-art reliable and scalable cluster infrastructure provided by Kubernetes."

- With the old on-premise infrastructure "it was difficult to properly embrace new technologies, and DevOps teams were considered to be a bottleneck," says Jacobs. "Now, with this cloud infrastructure, they have this packaging format, which can contain anything that runs in the Linux kernel. This makes a lot of people pretty happy. The engineers love the autonomy." - There were a few challenges in Zalando’s Kubernetes implementation. "We are a team of seven people providing clusters to different engineering teams, and our goal is to provide a rock-solid experience for all of them," says Jacobs. "We don’t want pet clusters. We don’t want to have to understand what workload they have; it should just work out of the box. With that in mind, cluster autoscaling is important. There are many different ways of doing cluster management, and this is not part of the core. So we created two components to provision clusters, have a registry for clusters, and to manage the whole cluster life cycle."

- Jacobs’s team also worked to improve the Kubernetes-AWS integration. "Thus you're very restricted. You need infrastructure to scale each autonomous team’s idea.""

- Plus, "there are still a lot of best practices missing," says Jacobs. The team, for example, recently solved a pod security policy issue. "There was already a concept in Kubernetes but it wasn’t documented, so it was kind of tricky," he says. The large Kubernetes community was a big help to resolve the issue. To help other companies start down the same path, Jacobs compiled his team’s learnings in a document called Running Kubernetes in Production. +

"It started as a PHP e-commerce site which was easy to get started with, but was not scaling with the business' needs" says Jacobs, Head of Developer Productivity at Zalando.

-
-
+

At that time, the company began expanding beyond its German origins into other European markets. Fast-forward to today and Zalando now has more than 14,000 employees, 3.6 billion Euro in revenue for 2016 and operates across 15 countries. "With growth in all dimensions, and constant scaling, it has been a once-in-a-lifetime experience," he says.

-
-
- "The Kubernetes API allows us to run applications in a cloud provider-agnostic way, which gives us the freedom to revisit IaaS providers in the coming years... We expect the Kubernetes API to be the global standard for PaaS infrastructure and are excited about the continued journey." -
-
-
-
- In the end, Kubernetes made it possible for Zalando to introduce and maintain the new products the company envisioned to grow its platform. "The fashion advice product used Scala, and there were struggles to make this possible with our former infrastructure," says Jacobs. "It was a workaround, and that team needed more and more support from the platform team, just because they used different technologies. Now with Kubernetes, it’s autonomous. Whatever the workload is, that team can just go their way, and Kubernetes prevents other bottlenecks."

- Looking ahead, Jacobs sees Zalando’s new infrastructure as a great enabler for other things the company has in the works, from its new logistics software, to a platform feature connecting brands, to products dreamed up by data scientists. "One vision is if you watch the next James Bond movie and see the suit he’s wearing, you should be able to automatically order it, and have it delivered to you within an hour," says Jacobs. "It’s about connecting the full fashion sphere. This is definitely not possible if you have a bottleneck with everyone running in the same data center and thus very restricted. You need infrastructure to scale each autonomous team’s idea."

- For other companies considering this technology, Jacobs says he wouldn’t necessarily advise doing it exactly the same way Zalando did. "It’s okay to do so if you’re ready to fail at some things," he says. "You need to set the right expectations. Not everything will work. Rewriting apps and this type of organizational change can be disruptive. The first product we moved was critical. There were a lot of dependencies, and it took longer than expected. Maybe we should have started with something less complicated, less business critical, just to get our toes wet."

- But once they got to the other side "it was clear for everyone that there’s no big alternative," Jacobs adds. "The Kubernetes API allows us to run applications in a cloud provider-agnostic way, which gives us the freedom to revisit IaaS providers in the coming years. Zalando Technology benefits from migrating to Kubernetes as we are able to leverage our existing knowledge to create an engineering platform offering flexibility and speed to our engineers while significantly reducing the operational overhead. We expect the Kubernetes API to be the global standard for PaaS infrastructure and are excited about the continued journey." +

Not to mention a unique opportunity for an infrastructure specialist like Jacobs. Just after he joined, the company began rewriting all their applications in-house. "That was generally our strategy," he says. "For example, we started with our own logistics warehouses but at first you don't know how to do logistics software, so you have some vendor software. And then we replaced it with our own because with off-the-shelf software you're not competitive. You need to optimize these processes based on your specific business needs."

-
+

In parallel to rewriting their applications, Zalando had set a goal of expanding beyond basic e-commerce to a platform offering multi-tenancy, a dramatic increase in assortments and styles, same-day delivery and even your own personal online stylist.

-
+

The need to scale ultimately led the company on a cloud-native journey. As did its embrace of a microservices-based software architecture that gives engineering teams more autonomy and ownership of projects. "This move to the cloud was necessary because in the data center you couldn't have autonomous teams. You have the same infrastructure and it was very homogeneous, so you could only run your Java or Python app," Jacobs says.

+ +{{< case-studies/quote image="/images/case-studies/zalando/banner3.jpg" >}} +"This move to the cloud was necessary because in the data center you couldn't have autonomous teams. You have the same infrastructure and it was very homogeneous, so you could only run your Java or Python app." +{{< /case-studies/quote >}} + +

Zalando began moving its infrastructure from two on-premise data centers to the cloud, requiring the migration of older applications for cloud-readiness. "We decided to have a clean break," says Jacobs. "Our Amazon Web Services infrastructure was set up like so: Every team has its own AWS account, which is completely isolated, meaning there's no 'lift and shift.' You basically have to rewrite your application to make it cloud-ready even down to the persistence layer. We bravely went back to the drawing board and redid everything, first choosing Docker as a common containerization, then building the infrastructure from there."

+ +

The company decided to hold off on orchestration at the beginning, but as teams were migrated to AWS, "we saw the pain teams were having with infrastructure and cloud formation on AWS," says Jacobs.

+ +

Zalandos 200+ autonomous engineering teams decide what technologies to use and could operate their own applications using their own AWS accounts. This setup proved to be a compliance challenge. Even with strict rules-of-play and automated compliance checks in place, engineering teams and IT-compliance were overburdened addressing compliance issues. "Violations appear for non-compliant behavior, which we detect when scanning the cloud infrastructure," says Jacobs. "Everything is possible and nothing enforced, so you have to live with violations (and resolve them) instead of preventing the error in the first place. This means overhead for teams—and overhead for compliance and operations. It also takes time to spin up new EC2 instances on AWS, which affects our deployment velocity."

+ +

The team realized they needed to "leverage the value you get from cluster management," says Jacobs. When they first looked at Platform as a Service (PaaS) options in 2015, the market was fragmented; but "now there seems to be a clear winner. It seemed like a good bet to go with Kubernetes."

+ +

The transition to Kubernetes started in 2016 during Zalando's Hack Week where participants deployed their projects to a Kubernetes cluster. From there 60 members of the tech infrastructure department were on-boarded - and then engineering teams were brought on one at a time. "We always start by talking with them and make sure everyone's expectations are clear," says Jacobs. "Then we conduct some Kubernetes training, which is mostly training for our CI/CD setup, because the user interface for our users is primarily through the CI/CD system. But they have to know fundamental Kubernetes concepts and the API. This is followed by a weekly sync with each team to check their progress. Once they have something in production, we want to see if everything is fine on top of what we can improve."

+ +{{< case-studies/quote image="/images/case-studies/zalando/banner4.jpg" >}} +Once Zalando began migrating applications to Kubernetes, the results were immediate. "Kubernetes is a cornerstone for our seamless end-to-end developer experience. We are able to ship ideas to production using a single consistent and declarative API," says Jacobs. +{{< /case-studies/quote >}} + +

At the moment, Zalando is running an initial 40 Kubernetes clusters with plans to scale for the foreseeable future. Once Zalando began migrating applications to Kubernetes, the results were immediate. "Kubernetes is a cornerstone for our seamless end-to-end developer experience. We are able to ship ideas to production using a single consistent and declarative API," says Jacobs. "The self-healing infrastructure provides a frictionless experience with higher-level abstractions built upon low-level best practices. We envision all Zalando delivery teams will run their containerized applications on a state-of-the-art reliable and scalable cluster infrastructure provided by Kubernetes."

+ +

With the old on-premise infrastructure "it was difficult to properly embrace new technologies, and DevOps teams were considered to be a bottleneck," says Jacobs. "Now, with this cloud infrastructure, they have this packaging format, which can contain anything that runs in the Linux kernel. This makes a lot of people pretty happy. The engineers love the autonomy."

+ +

There were a few challenges in Zalando's Kubernetes implementation. "We are a team of seven people providing clusters to different engineering teams, and our goal is to provide a rock-solid experience for all of them," says Jacobs. "We don't want pet clusters. We don't want to have to understand what workload they have; it should just work out of the box. With that in mind, cluster autoscaling is important. There are many different ways of doing cluster management, and this is not part of the core. So we created two components to provision clusters, have a registry for clusters, and to manage the whole cluster life cycle."

+ +

Jacobs's team also worked to improve the Kubernetes-AWS integration. "Thus you're very restricted. You need infrastructure to scale each autonomous team's idea." Plus, "there are still a lot of best practices missing," says Jacobs. The team, for example, recently solved a pod security policy issue. "There was already a concept in Kubernetes but it wasn't documented, so it was kind of tricky," he says. The large Kubernetes community was a big help to resolve the issue. To help other companies start down the same path, Jacobs compiled his team's learnings in a document called Running Kubernetes in Production.

+ +{{< case-studies/quote >}} +"The Kubernetes API allows us to run applications in a cloud provider-agnostic way, which gives us the freedom to revisit IaaS providers in the coming years... We expect the Kubernetes API to be the global standard for PaaS infrastructure and are excited about the continued journey." +{{< /case-studies/quote >}} + +

In the end, Kubernetes made it possible for Zalando to introduce and maintain the new products the company envisioned to grow its platform. "The fashion advice product used Scala, and there were struggles to make this possible with our former infrastructure," says Jacobs. "It was a workaround, and that team needed more and more support from the platform team, just because they used different technologies. Now with Kubernetes, it's autonomous. Whatever the workload is, that team can just go their way, and Kubernetes prevents other bottlenecks."

+ +

Looking ahead, Jacobs sees Zalando's new infrastructure as a great enabler for other things the company has in the works, from its new logistics software, to a platform feature connecting brands, to products dreamed up by data scientists. "One vision is if you watch the next James Bond movie and see the suit he's wearing, you should be able to automatically order it, and have it delivered to you within an hour," says Jacobs. "It's about connecting the full fashion sphere. This is definitely not possible if you have a bottleneck with everyone running in the same data center and thus very restricted. You need infrastructure to scale each autonomous team's idea."

+ +

For other companies considering this technology, Jacobs says he wouldn't necessarily advise doing it exactly the same way Zalando did. "It's okay to do so if you're ready to fail at some things," he says. "You need to set the right expectations. Not everything will work. Rewriting apps and this type of organizational change can be disruptive. The first product we moved was critical. There were a lot of dependencies, and it took longer than expected. Maybe we should have started with something less complicated, less business critical, just to get our toes wet."

+ +

But once they got to the other side "it was clear for everyone that there's no big alternative," Jacobs adds. "The Kubernetes API allows us to run applications in a cloud provider-agnostic way, which gives us the freedom to revisit IaaS providers in the coming years. Zalando Technology benefits from migrating to Kubernetes as we are able to leverage our existing knowledge to create an engineering platform offering flexibility and speed to our engineers while significantly reducing the operational overhead. We expect the Kubernetes API to be the global standard for PaaS infrastructure and are excited about the continued journey."

diff --git a/content/en/docs/concepts/architecture/control-plane-node-communication.md b/content/en/docs/concepts/architecture/control-plane-node-communication.md index f581b7dcb4..2e7235a89f 100644 --- a/content/en/docs/concepts/architecture/control-plane-node-communication.md +++ b/content/en/docs/concepts/architecture/control-plane-node-communication.md @@ -49,7 +49,7 @@ To verify this connection, use the `--kubelet-certificate-authority` flag to pro If that is not possible, use [SSH tunneling](#ssh-tunnels) between the apiserver and kubelet if required to avoid connecting over an untrusted or public network. -Finally, [Kubelet authentication and/or authorization](/docs/admin/kubelet-authentication-authorization/) should be enabled to secure the kubelet API. +Finally, [Kubelet authentication and/or authorization](/docs/reference/command-line-tools-reference/kubelet-authentication-authorization/) should be enabled to secure the kubelet API. ### apiserver to nodes, pods, and services diff --git a/content/en/docs/concepts/architecture/controller.md b/content/en/docs/concepts/architecture/controller.md index 2705cdfcb7..8f70bdc05c 100644 --- a/content/en/docs/concepts/architecture/controller.md +++ b/content/en/docs/concepts/architecture/controller.md @@ -96,6 +96,16 @@ the current state closer in line. nodes in your cluster. See [Cluster autoscaling](/docs/tasks/administer-cluster/cluster-management/#cluster-autoscaling)). +The important point here is that the controller makes some change to bring about +your desired state, and then reports current state back to your cluster's API server. +Other control loops can observe that reported data and take their own actions. + +In the thermostat example, if the room is very cold then a different controller +might also turn on a frost protection heater. With Kubernetes clusters, the control +plane indirectly works with IP address management tools, storage services, +cloud provider APIS, and other services by +[extending Kubernetes](/docs/concepts/extend-kubernetes/) to implement that. + ## Desired versus current state {#desired-vs-current} Kubernetes takes a cloud-native view of systems, and is able to handle diff --git a/content/en/docs/concepts/cluster-administration/_index.md b/content/en/docs/concepts/cluster-administration/_index.md index e35c8e0ebc..c3cbbe46ba 100644 --- a/content/en/docs/concepts/cluster-administration/_index.md +++ b/content/en/docs/concepts/cluster-administration/_index.md @@ -51,7 +51,7 @@ Before choosing a guide, here are some considerations: * [Kubernetes Container Environment](/docs/concepts/containers/container-environment/) describes the environment for Kubelet managed containers on a Kubernetes node. -* [Controlling Access to the Kubernetes API](/docs/reference/access-authn-authz/controlling-access/) describes how to set up permissions for users and service accounts. +* [Controlling Access to the Kubernetes API](/docs/concepts/security/controlling-access) describes how Kubernetes implements access control for its own API. * [Authenticating](/docs/reference/access-authn-authz/authentication/) explains authentication in Kubernetes, including the various authentication options. diff --git a/content/en/docs/concepts/cluster-administration/logging.md b/content/en/docs/concepts/cluster-administration/logging.md index 3e7f398e66..6915a11ed4 100644 --- a/content/en/docs/concepts/cluster-administration/logging.md +++ b/content/en/docs/concepts/cluster-administration/logging.md @@ -24,8 +24,8 @@ the description of how logs are stored and handled on the node to be useful. In this section, you can see an example of basic logging in Kubernetes that outputs data to the standard output stream. This demonstration uses -a [pod specification](/examples/debug/counter-pod.yaml) with -a container that writes some text to standard output once per second. +a pod specification with a container that writes some text to standard output +once per second. {{< codenew file="debug/counter-pod.yaml" >}} diff --git a/content/en/docs/concepts/cluster-administration/system-metrics.md b/content/en/docs/concepts/cluster-administration/system-metrics.md index e727aa17ce..3dba687f4e 100644 --- a/content/en/docs/concepts/cluster-administration/system-metrics.md +++ b/content/en/docs/concepts/cluster-administration/system-metrics.md @@ -104,7 +104,7 @@ The kubelet collects accelerator metrics through cAdvisor. To collect these metr The responsibility for collecting accelerator metrics now belongs to the vendor rather than the kubelet. Vendors must provide a container that collects metrics and exposes them to the metrics service (for example, Prometheus). -The [`DisableAcceleratorUsageMetrics` feature gate](/docs/references/command-line-tools-reference/feature-gate.md#feature-gates-for-alpha-or-beta-features:~:text= DisableAcceleratorUsageMetrics,-false) disables metrics collected by the kubelet, with a [timeline for enabling this feature by default](https://github.com/kubernetes/enhancements/tree/411e51027db842355bd489691af897afc1a41a5e/keps/sig-node/1867-disable-accelerator-usage-metrics#graduation-criteria). +The [`DisableAcceleratorUsageMetrics` feature gate](/docs/reference/command-line-tools-reference/feature-gates/#feature-gates-for-alpha-or-beta-features:~:text= DisableAcceleratorUsageMetrics,-false) disables metrics collected by the kubelet, with a [timeline for enabling this feature by default](https://github.com/kubernetes/enhancements/tree/411e51027db842355bd489691af897afc1a41a5e/keps/sig-node/1867-disable-accelerator-usage-metrics#graduation-criteria). ## Component metrics diff --git a/content/en/docs/concepts/configuration/pod-priority-preemption.md b/content/en/docs/concepts/configuration/pod-priority-preemption.md index f8e7c85311..40c38ae21e 100644 --- a/content/en/docs/concepts/configuration/pod-priority-preemption.md +++ b/content/en/docs/concepts/configuration/pod-priority-preemption.md @@ -11,7 +11,7 @@ weight: 70 {{< feature-state for_k8s_version="v1.14" state="stable" >}} -[Pods](/docs/concepts/workloads/pods/pod/) can have _priority_. Priority indicates the +[Pods](/docs/concepts/workloads/pods/) can have _priority_. Priority indicates the importance of a Pod relative to other Pods. If a Pod cannot be scheduled, the scheduler tries to preempt (evict) lower priority Pods to make scheduling of the pending Pod possible. diff --git a/content/en/docs/concepts/containers/runtime-class.md b/content/en/docs/concepts/containers/runtime-class.md index c96bd7aba5..6589590bc4 100644 --- a/content/en/docs/concepts/containers/runtime-class.md +++ b/content/en/docs/concepts/containers/runtime-class.md @@ -183,7 +183,7 @@ are accounted for in Kubernetes. - [RuntimeClass Design](https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/585-runtime-class/README.md) - [RuntimeClass Scheduling Design](https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/585-runtime-class/README.md#runtimeclass-scheduling) -- Read about the [Pod Overhead](/docs/concepts/configuration/pod-overhead/) concept +- Read about the [Pod Overhead](/docs/concepts/scheduling-eviction/pod-overhead/) concept - [PodOverhead Feature Design](https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/20190226-pod-overhead.md) diff --git a/content/en/docs/concepts/extend-kubernetes/_index.md b/content/en/docs/concepts/extend-kubernetes/_index.md index 4ffb0a831f..429912a9ed 100644 --- a/content/en/docs/concepts/extend-kubernetes/_index.md +++ b/content/en/docs/concepts/extend-kubernetes/_index.md @@ -130,7 +130,7 @@ Adding an API does not directly let you affect the behavior of existing APIs (e. ### API Access Extensions -When a request reaches the Kubernetes API Server, it is first Authenticated, then Authorized, then subject to various types of Admission Control. See [Controlling Access to the Kubernetes API](/docs/reference/access-authn-authz/controlling-access/) for more on this flow. +When a request reaches the Kubernetes API Server, it is first Authenticated, then Authorized, then subject to various types of Admission Control. See [Controlling Access to the Kubernetes API](/docs/concepts/security/controlling-access/) for more on this flow. Each of these steps offers extension points. diff --git a/content/en/docs/concepts/extend-kubernetes/api-extension/custom-resources.md b/content/en/docs/concepts/extend-kubernetes/api-extension/custom-resources.md index 9a84267445..dcfef3f6b6 100644 --- a/content/en/docs/concepts/extend-kubernetes/api-extension/custom-resources.md +++ b/content/en/docs/concepts/extend-kubernetes/api-extension/custom-resources.md @@ -16,7 +16,7 @@ methods for adding custom resources and how to choose between them. ## Custom resources -A *resource* is an endpoint in the [Kubernetes API](/docs/reference/using-api/api-overview/) that stores a collection of +A *resource* is an endpoint in the [Kubernetes API](/docs/concepts/overview/kubernetes-api/) that stores a collection of [API objects](/docs/concepts/overview/working-with-objects/kubernetes-objects/) of a certain kind; for example, the built-in *pods* resource contains a collection of Pod objects. A *custom resource* is an extension of the Kubernetes API that is not necessarily available in a default diff --git a/content/en/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins.md b/content/en/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins.md index 441d3bd871..314a01354a 100644 --- a/content/en/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins.md +++ b/content/en/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins.md @@ -37,7 +37,7 @@ During the registration, the device plugin needs to send: * The name of its Unix socket. * The Device Plugin API version against which it was built. * The `ResourceName` it wants to advertise. Here `ResourceName` needs to follow the - [extended resource naming scheme](/docs/concepts/configuration/manage-resources-container/#extended-resources) + [extended resource naming scheme](/docs/concepts/configuration/manage-resources-containers/#extended-resources) as `vendor-domain/resourcetype`. (For example, an NVIDIA GPU is advertised as `nvidia.com/gpu`.) @@ -210,7 +210,7 @@ Support for the "PodResources service" requires `KubeletPodResources` [feature g {{< feature-state for_k8s_version="v1.18" state="beta" >}} -The Topology Manager is a Kubelet component that allows resources to be co-ordintated in a Topology aligned manner. In order to do this, the Device Plugin API was extended to include a `TopologyInfo` struct. +The Topology Manager is a Kubelet component that allows resources to be co-ordinated in a Topology aligned manner. In order to do this, the Device Plugin API was extended to include a `TopologyInfo` struct. ```gRPC diff --git a/content/en/docs/concepts/extend-kubernetes/extend-cluster.md b/content/en/docs/concepts/extend-kubernetes/extend-cluster.md index 76c72e74e4..d70f5cc9ff 100644 --- a/content/en/docs/concepts/extend-kubernetes/extend-cluster.md +++ b/content/en/docs/concepts/extend-kubernetes/extend-cluster.md @@ -131,7 +131,7 @@ Adding an API does not directly let you affect the behavior of existing APIs (e. ### API Access Extensions -When a request reaches the Kubernetes API Server, it is first Authenticated, then Authorized, then subject to various types of Admission Control. See [Controlling Access to the Kubernetes API](/docs/reference/access-authn-authz/controlling-access/) for more on this flow. +When a request reaches the Kubernetes API Server, it is first Authenticated, then Authorized, then subject to various types of Admission Control. See [Controlling Access to the Kubernetes API](/docs/concepts/security/controlling-access/) for more on this flow. Each of these steps offers extension points. diff --git a/content/en/docs/concepts/extend-kubernetes/operator.md b/content/en/docs/concepts/extend-kubernetes/operator.md index 892529633f..140cf2557a 100644 --- a/content/en/docs/concepts/extend-kubernetes/operator.md +++ b/content/en/docs/concepts/extend-kubernetes/operator.md @@ -35,7 +35,7 @@ Kubernetes' {{< glossary_tooltip text="controllers" term_id="controller" >}} concept lets you extend the cluster's behaviour without modifying the code of Kubernetes itself. Operators are clients of the Kubernetes API that act as controllers for -a [Custom Resource](/docs/concepts/api-extension/custom-resources/). +a [Custom Resource](/docs/concepts/extend-kubernetes/api-extension/custom-resources/). ## An example Operator {#example} diff --git a/content/en/docs/concepts/extend-kubernetes/service-catalog.md b/content/en/docs/concepts/extend-kubernetes/service-catalog.md index b40ca7ee14..3aa9675788 100644 --- a/content/en/docs/concepts/extend-kubernetes/service-catalog.md +++ b/content/en/docs/concepts/extend-kubernetes/service-catalog.md @@ -229,7 +229,7 @@ The following example describes how to map secret values into application enviro * If you are familiar with {{< glossary_tooltip text="Helm Charts" term_id="helm-chart" >}}, [install Service Catalog using Helm](/docs/tasks/service-catalog/install-service-catalog-using-helm/) into your Kubernetes cluster. Alternatively, you can [install Service Catalog using the SC tool](/docs/tasks/service-catalog/install-service-catalog-using-sc/). * View [sample service brokers](https://github.com/openservicebrokerapi/servicebroker/blob/master/gettingStarted.md#sample-service-brokers). -* Explore the [kubernetes-incubator/service-catalog](https://github.com/kubernetes-incubator/service-catalog) project. +* Explore the [kubernetes-sigs/service-catalog](https://github.com/kubernetes-sigs/service-catalog) project. * View [svc-cat.io](https://svc-cat.io/docs/). diff --git a/content/en/docs/concepts/overview/kubernetes-api.md b/content/en/docs/concepts/overview/kubernetes-api.md index 7580287fb3..ed89d1f5d1 100644 --- a/content/en/docs/concepts/overview/kubernetes-api.md +++ b/content/en/docs/concepts/overview/kubernetes-api.md @@ -25,7 +25,7 @@ The Kubernetes API lets you query and manipulate the state of objects in the Kub Most operations can be performed through the [kubectl](/docs/reference/kubectl/overview/) command-line interface or other command-line tools, such as -[kubeadm](/docs/reference/setup-tools/kubeadm/kubeadm/), which in turn use the +[kubeadm](/docs/reference/setup-tools/kubeadm/), which in turn use the API. However, you can also access the API directly using REST calls. Consider using one of the [client libraries](/docs/reference/using-api/client-libraries/) @@ -103,8 +103,8 @@ and behavior, and to enable controlling access to end-of-life and/or experimental APIs. To make it easier to evolve and to extend its API, Kubernetes implements -[API groups](/docs/reference/using-api/api-overview/#api-groups) that can be -[enabled or disabled](/docs/reference/using-api/api-overview/#enabling-or-disabling). +[API groups](/docs/reference/using-api/#api-groups) that can be +[enabled or disabled](/docs/reference/using-api/#enabling-or-disabling). API resources are distinguished by their API group, resource type, namespace (for namespaced resources), and name. The API server may serve the same @@ -115,7 +115,7 @@ versions `v1` and `v1beta1` for the same resource. An object created by the `v1beta1` version can then be read, updated, and deleted by either the `v1beta1` or the `v1` versions. -Refer to [API versions reference](/docs/reference/using-api/api-overview/#api-versioning) +Refer to [API versions reference](/docs/reference/using-api/#api-versioning) for more details on the API version level definitions. ## API Extension @@ -131,7 +131,7 @@ The Kubernetes API can be extended in one of two ways: - Learn how to extend the Kubernetes API by adding your own [CustomResourceDefinition](/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/). -- [Controlling API Access](/docs/reference/access-authn-authz/controlling-access/) describes +- [Controlling Access To The Kubernetes API](/docs/concepts/security/controlling-access/) describes how the cluster manages authentication and authorization for API access. - Learn about API endpoints, resource types and samples by reading [API Reference](/docs/reference/kubernetes-api/). diff --git a/content/en/docs/concepts/overview/what-is-kubernetes.md b/content/en/docs/concepts/overview/what-is-kubernetes.md index 6df252ede2..f3c4a1da02 100644 --- a/content/en/docs/concepts/overview/what-is-kubernetes.md +++ b/content/en/docs/concepts/overview/what-is-kubernetes.md @@ -38,7 +38,7 @@ Virtualization allows better utilization of resources in a physical server and a Each VM is a full machine running all the components, including its own operating system, on top of the virtualized hardware. -**Container deployment era:** Containers are similar to VMs, but they have relaxed isolation properties to share the Operating System (OS) among the applications. Therefore, containers are considered lightweight. Similar to a VM, a container has its own filesystem, CPU, memory, process space, and more. As they are decoupled from the underlying infrastructure, they are portable across clouds and OS distributions. +**Container deployment era:** Containers are similar to VMs, but they have relaxed isolation properties to share the Operating System (OS) among the applications. Therefore, containers are considered lightweight. Similar to a VM, a container has its own filesystem, share of CPU, memory, process space, and more. As they are decoupled from the underlying infrastructure, they are portable across clouds and OS distributions. Containers have become popular because they provide extra benefits, such as: @@ -94,4 +94,3 @@ Kubernetes: * Take a look at the [Kubernetes Components](/docs/concepts/overview/components/) * Ready to [Get Started](/docs/setup/)? - diff --git a/content/en/docs/concepts/overview/working-with-objects/kubernetes-objects.md b/content/en/docs/concepts/overview/working-with-objects/kubernetes-objects.md index ab447cdcd6..716955ca06 100644 --- a/content/en/docs/concepts/overview/working-with-objects/kubernetes-objects.md +++ b/content/en/docs/concepts/overview/working-with-objects/kubernetes-objects.md @@ -91,9 +91,9 @@ and the `spec` format for a Deployment can be found in ## {{% heading "whatsnext" %}} -* [Kubernetes API overview](/docs/reference/using-api/api-overview/) explains some more API concepts * Learn about the most important basic Kubernetes objects, such as [Pod](/docs/concepts/workloads/pods/). -* Learn about [controllers](/docs/concepts/architecture/controller/) in Kubernetes +* Learn about [controllers](/docs/concepts/architecture/controller/) in Kubernetes. +* [Using the Kubernetes API](/docs/reference/using-api/) explains some more API concepts. diff --git a/content/en/docs/concepts/overview/working-with-objects/labels.md b/content/en/docs/concepts/overview/working-with-objects/labels.md index 822935978c..f8530ce546 100644 --- a/content/en/docs/concepts/overview/working-with-objects/labels.md +++ b/content/en/docs/concepts/overview/working-with-objects/labels.md @@ -188,7 +188,7 @@ kubectl get pods -l 'environment,environment notin (frontend)' Some Kubernetes objects, such as [`services`](/docs/concepts/services-networking/service/) and [`replicationcontrollers`](/docs/concepts/workloads/controllers/replicationcontroller/), also use label selectors to specify sets of other resources, such as -[pods](/docs/concepts/workloads/pods/pod/). +[pods](/docs/concepts/workloads/pods/). #### Service and ReplicationController diff --git a/content/en/docs/concepts/policy/pod-security-policy.md b/content/en/docs/concepts/policy/pod-security-policy.md index 6ef71ec733..063f1234fe 100644 --- a/content/en/docs/concepts/policy/pod-security-policy.md +++ b/content/en/docs/concepts/policy/pod-security-policy.md @@ -140,13 +140,17 @@ For a complete example of authorizing a PodSecurityPolicy, see ### Troubleshooting -- The [Controller Manager](/docs/reference/command-line-tools-reference/kube-controller-manager/) must be run -against [the secured API port](/docs/reference/access-authn-authz/controlling-access/), -and must not have superuser permissions. Otherwise requests would bypass -authentication and authorization modules, all PodSecurityPolicy objects would be -allowed, and users would be able to create privileged containers. For more details -on configuring Controller Manager authorization, see -[Controller Roles](/docs/reference/access-authn-authz/rbac/#controller-roles). +- The [controller manager](/docs/reference/command-line-tools-reference/kube-controller-manager/) + must be run against the secured API port and must not have superuser permissions. See + [Controlling Access to the Kubernetes API](/docs/concepts/security/controlling-access) + to learn about API server access controls. + If the controller manager connected through the trusted API port (also known as the + `localhost` listener), requests would bypass authentication and authorization modules; + all PodSecurityPolicy objects would be allowed, and users would be able to create grant + themselves the ability to create privileged containers. + + For more details on configuring controller manager authorization, see + [Controller Roles](/docs/reference/access-authn-authz/rbac/#controller-roles). ## Policy Order diff --git a/content/en/docs/concepts/policy/resource-quotas.md b/content/en/docs/concepts/policy/resource-quotas.md index 50befe550a..2de7ac8740 100644 --- a/content/en/docs/concepts/policy/resource-quotas.md +++ b/content/en/docs/concepts/policy/resource-quotas.md @@ -18,17 +18,17 @@ Resource quotas are a tool for administrators to address this concern. A resource quota, defined by a `ResourceQuota` object, provides constraints that limit aggregate resource consumption per namespace. It can limit the quantity of objects that can be created in a namespace by type, as well as the total amount of compute resources that may -be consumed by resources in that project. +be consumed by resources in that namespace. Resource quotas work like this: - Different teams work in different namespaces. Currently this is voluntary, but support for making this mandatory via ACLs is planned. -- The administrator creates one `ResourceQuota` for each namespace. +- The administrator creates one ResourceQuota for each namespace. - Users create resources (pods, services, etc.) in the namespace, and the quota system - tracks usage to ensure it does not exceed hard resource limits defined in a `ResourceQuota`. + tracks usage to ensure it does not exceed hard resource limits defined in a ResourceQuota. - If creating or updating a resource violates a quota constraint, the request will fail with HTTP status code `403 FORBIDDEN` with a message explaining the constraint that would have been violated. @@ -40,7 +40,7 @@ Resource quotas work like this: See the [walkthrough](/docs/tasks/administer-cluster/manage-resources/quota-memory-cpu-namespace/) for an example of how to avoid this problem. -The name of a `ResourceQuota` object must be a valid +The name of a ResourceQuota object must be a valid [DNS subdomain name](/docs/concepts/overview/working-with-objects/names#dns-subdomain-names). Examples of policies that could be created using namespaces and quotas are: @@ -58,11 +58,11 @@ Neither contention nor changes to quota will affect already created resources. ## Enabling Resource Quota Resource Quota support is enabled by default for many Kubernetes distributions. It is -enabled when the apiserver `--enable-admission-plugins=` flag has `ResourceQuota` as +enabled when the API server `--enable-admission-plugins=` flag has `ResourceQuota` as one of its arguments. A resource quota is enforced in a particular namespace when there is a -`ResourceQuota` in that namespace. +ResourceQuota in that namespace. ## Compute Resource Quota @@ -76,6 +76,10 @@ The following resource types are supported: | `limits.memory` | Across all pods in a non-terminal state, the sum of memory limits cannot exceed this value. | | `requests.cpu` | Across all pods in a non-terminal state, the sum of CPU requests cannot exceed this value. | | `requests.memory` | Across all pods in a non-terminal state, the sum of memory requests cannot exceed this value. | +| `hugepages-` | Across all pods in a non-terminal state, the number of +huge page requests of the specified size cannot exceed this value. | +| `cpu` | Same as `requests.cpu` | +| `memory` | Same as `requests.memory` | ### Resource Quota For Extended Resources @@ -103,8 +107,8 @@ In addition, you can limit consumption of storage resources based on associated | Resource Name | Description | | --------------------- | ----------------------------------------------------------- | | `requests.storage` | Across all persistent volume claims, the sum of storage requests cannot exceed this value. | -| `persistentvolumeclaims` | The total number of [persistent volume claims](/docs/concepts/storage/persistent-volumes/#persistentvolumeclaims) that can exist in the namespace. | -| `.storageclass.storage.k8s.io/requests.storage` | Across all persistent volume claims associated with the storage-class-name, the sum of storage requests cannot exceed this value. | +| `persistentvolumeclaims` | The total number of [PersistentVolumeClaims](/docs/concepts/storage/persistent-volumes/#persistentvolumeclaims) that can exist in the namespace. | +| `.storageclass.storage.k8s.io/requests.storage` | Across all persistent volume claims associated with the ``, the sum of storage requests cannot exceed this value. | | `.storageclass.storage.k8s.io/persistentvolumeclaims` | Across all persistent volume claims associated with the storage-class-name, the total number of [persistent volume claims](/docs/concepts/storage/persistent-volumes/#persistentvolumeclaims) that can exist in the namespace. | For example, if an operator wants to quota storage with `gold` storage class separate from `bronze` storage class, the operator can @@ -119,12 +123,15 @@ In release 1.8, quota support for local ephemeral storage is added as an alpha f | ------------------------------- |----------------------------------------------------------- | | `requests.ephemeral-storage` | Across all pods in the namespace, the sum of local ephemeral storage requests cannot exceed this value. | | `limits.ephemeral-storage` | Across all pods in the namespace, the sum of local ephemeral storage limits cannot exceed this value. | +| `ephemeral-storage` | Same as `requests.ephemeral-storage`. | ## Object Count Quota -The 1.9 release added support to quota all standard namespaced resource types using the following syntax: +You can set quota for the total number of certain resources of all standard, +namespaced resource types using the following syntax: -* `count/.` +* `count/.` for resources from non-core groups +* `count/` for resources from the core group Here is an example set of resources users may want to put under object count quota: @@ -138,33 +145,30 @@ Here is an example set of resources users may want to put under object count quo * `count/statefulsets.apps` * `count/jobs.batch` * `count/cronjobs.batch` -* `count/deployments.extensions` -The 1.15 release added support for custom resources using the same syntax. +The same syntax can be used for custom resources. For example, to create a quota on a `widgets` custom resource in the `example.com` API group, use `count/widgets.example.com`. When using `count/*` resource quota, an object is charged against the quota if it exists in server storage. These types of quotas are useful to protect against exhaustion of storage resources. For example, you may -want to quota the number of secrets in a server given their large size. Too many secrets in a cluster can -actually prevent servers and controllers from starting! You may choose to quota jobs to protect against -a poorly configured cronjob creating too many jobs in a namespace causing a denial of service. - -Prior to the 1.9 release, it was possible to do generic object count quota on a limited set of resources. -In addition, it is possible to further constrain quota for particular resources by their type. +want to limit the number of Secrets in a server given their large size. Too many Secrets in a cluster can +actually prevent servers and controllers from starting. You can set a quota for Jobs to protect against +a poorly configured CronJob. CronJobs that create too many Jobs in a namespace can lead to a denial of service. +It is also possible to do generic object count quota on a limited set of resources. The following types are supported: | Resource Name | Description | | ------------------------------- | ------------------------------------------------- | -| `configmaps` | The total number of config maps that can exist in the namespace. | -| `persistentvolumeclaims` | The total number of [persistent volume claims](/docs/concepts/storage/persistent-volumes/#persistentvolumeclaims) that can exist in the namespace. | -| `pods` | The total number of pods in a non-terminal state that can exist in the namespace. A pod is in a terminal state if `.status.phase in (Failed, Succeeded)` is true. | -| `replicationcontrollers` | The total number of replication controllers that can exist in the namespace. | -| `resourcequotas` | The total number of [resource quotas](/docs/reference/access-authn-authz/admission-controllers/#resourcequota) that can exist in the namespace. | -| `services` | The total number of services that can exist in the namespace. | -| `services.loadbalancers` | The total number of services of type load balancer that can exist in the namespace. | -| `services.nodeports` | The total number of services of type node port that can exist in the namespace. | -| `secrets` | The total number of secrets that can exist in the namespace. | +| `configmaps` | The total number of ConfigMaps that can exist in the namespace. | +| `persistentvolumeclaims` | The total number of [PersistentVolumeClaims](/docs/concepts/storage/persistent-volumes/#persistentvolumeclaims) that can exist in the namespace. | +| `pods` | The total number of Pods in a non-terminal state that can exist in the namespace. A pod is in a terminal state if `.status.phase in (Failed, Succeeded)` is true. | +| `replicationcontrollers` | The total number of ReplicationControllers that can exist in the namespace. | +| `resourcequotas` | The total number of ResourceQuotas that can exist in the namespace. | +| `services` | The total number of Services that can exist in the namespace. | +| `services.loadbalancers` | The total number of Services of type `LoadBalancer` that can exist in the namespace. | +| `services.nodeports` | The total number of Services of type `NodePort` that can exist in the namespace. | +| `secrets` | The total number of Secrets that can exist in the namespace. | For example, `pods` quota counts and enforces a maximum on the number of `pods` created in a single namespace that are not terminal. You might want to set a `pods` @@ -173,7 +177,7 @@ exhausts the cluster's supply of Pod IPs. ## Quota Scopes -Each quota can have an associated set of scopes. A quota will only measure usage for a resource if it matches +Each quota can have an associated set of `scopes`. A quota will only measure usage for a resource if it matches the intersection of enumerated scopes. When a scope is added to the quota, it limits the number of resources it supports to those that pertain to the scope. @@ -185,22 +189,60 @@ Resources specified on the quota outside of the allowed set results in a validat | `NotTerminating` | Match pods where `.spec.activeDeadlineSeconds is nil` | | `BestEffort` | Match pods that have best effort quality of service. | | `NotBestEffort` | Match pods that do not have best effort quality of service. | +| `PriorityClass` | Match pods that references the specified [priority class](/docs/concepts/configuration/pod-priority-preemption). | -The `BestEffort` scope restricts a quota to tracking the following resource: `pods` +The `BestEffort` scope restricts a quota to tracking the following resource: -The `Terminating`, `NotTerminating`, and `NotBestEffort` scopes restrict a quota to tracking the following resources: - -* `cpu` -* `limits.cpu` -* `limits.memory` -* `memory` * `pods` + +The `Terminating`, `NotTerminating`, `NotBestEffort` and `PriorityClass` +scopes restrict a quota to tracking the following resources: + +* `pods` +* `cpu` +* `memory` * `requests.cpu` * `requests.memory` +* `limits.cpu` +* `limits.memory` + +Note that you cannot specify both the `Terminating` and the `NotTerminating` +scopes in the same quota, and you cannot specify both the `BestEffort` and +`NotBestEffort` scopes in the same quota either. + +The `scopeSelector` supports the following values in the `operator` field: + +* `In` +* `NotIn` +* `Exists` +* `DoesNotExist` + +When using one of the following values as the `scopeName` when defining the +`scopeSelector`, the `operator` must be `Exists`. + +* `Terminating` +* `NotTerminating` +* `BestEffort` +* `NotBestEffort` + +If the `operator` is `In` or `NotIn`, the `values` field must have at least +one value. For example: + +```yaml + scopeSelector: + matchExpressions: + - scopeName: PriorityClass + operator: In + values: + - middle +``` + +If the `operator` is `Exists` or `DoesNotExist`, the `values field must *NOT* be +specified. ### Resource Quota Per PriorityClass -{{< feature-state for_k8s_version="v1.12" state="beta" >}} +{{< feature-state for_k8s_version="v1.17" state="stable" >}} Pods can be created at a specific [priority](/docs/concepts/configuration/pod-priority-preemption/#pod-priority). You can control a pod's consumption of system resources based on a pod's priority, by using the `scopeSelector` @@ -283,7 +325,7 @@ Apply the YAML using `kubectl create`. kubectl create -f ./quota.yml ``` -```shell +``` resourcequota/pods-high created resourcequota/pods-medium created resourcequota/pods-low created @@ -295,7 +337,7 @@ Verify that `Used` quota is `0` using `kubectl describe quota`. kubectl describe quota ``` -```shell +``` Name: pods-high Namespace: default Resource Used Hard @@ -360,7 +402,7 @@ the other two quotas are unchanged. kubectl describe quota ``` -```shell +``` Name: pods-high Namespace: default Resource Used Hard @@ -388,13 +430,6 @@ memory 0 20Gi pods 0 10 ``` -`scopeSelector` supports the following values in the `operator` field: - -* `In` -* `NotIn` -* `Exist` -* `DoesNotExist` - ## Requests compared to Limits {#requests-vs-limits} When allocating compute resources, each container may specify a request and a limit value for either CPU or memory. @@ -458,7 +493,7 @@ kubectl create -f ./object-counts.yaml --namespace=myspace kubectl get quota --namespace=myspace ``` -```shell +``` NAME AGE compute-resources 30s object-counts 32s @@ -468,7 +503,7 @@ object-counts 32s kubectl describe quota compute-resources --namespace=myspace ``` -```shell +``` Name: compute-resources Namespace: myspace Resource Used Hard @@ -484,7 +519,7 @@ requests.nvidia.com/gpu 0 4 kubectl describe quota object-counts --namespace=myspace ``` -```shell +``` Name: object-counts Namespace: myspace Resource Used Hard @@ -506,7 +541,7 @@ kubectl create namespace myspace ``` ```shell -kubectl create quota test --hard=count/deployments.extensions=2,count/replicasets.extensions=4,count/pods=3,count/secrets=4 --namespace=myspace +kubectl create quota test --hard=count/deployments.apps=2,count/replicasets.apps=4,count/pods=3,count/secrets=4 --namespace=myspace ``` ```shell @@ -517,29 +552,29 @@ kubectl create deployment nginx --image=nginx --namespace=myspace --replicas=2 kubectl describe quota --namespace=myspace ``` -```shell +``` Name: test Namespace: myspace Resource Used Hard -------- ---- ---- -count/deployments.extensions 1 2 +count/deployments.apps 1 2 count/pods 2 3 -count/replicasets.extensions 1 4 +count/replicasets.apps 1 4 count/secrets 1 4 ``` ## Quota and Cluster Capacity -`ResourceQuotas` are independent of the cluster capacity. They are +ResourceQuotas are independent of the cluster capacity. They are expressed in absolute units. So, if you add nodes to your cluster, this does *not* automatically give each namespace the ability to consume more resources. Sometimes more complex policies may be desired, such as: - - Proportionally divide total cluster resources among several teams. - - Allow each tenant to grow resource usage as needed, but have a generous - limit to prevent accidental resource exhaustion. - - Detect demand from one namespace, add nodes, and increase quota. +- Proportionally divide total cluster resources among several teams. +- Allow each tenant to grow resource usage as needed, but have a generous + limit to prevent accidental resource exhaustion. +- Detect demand from one namespace, add nodes, and increase quota. Such policies could be implemented using `ResourceQuotas` as building blocks, by writing a "controller" that watches the quota usage and adjusts the quota @@ -550,14 +585,16 @@ restrictions around nodes: pods from several namespaces may run on the same node ## Limit Priority Class consumption by default -It may be desired that pods at a particular priority, eg. "cluster-services", should be allowed in a namespace, if and only if, a matching quota object exists. +It may be desired that pods at a particular priority, eg. "cluster-services", +should be allowed in a namespace, if and only if, a matching quota object exists. -With this mechanism, operators will be able to restrict usage of certain high priority classes to a limited number of namespaces and not every namespace will be able to consume these priority classes by default. +With this mechanism, operators are able to restrict usage of certain high +priority classes to a limited number of namespaces and not every namespace +will be able to consume these priority classes by default. -To enforce this, kube-apiserver flag `--admission-control-config-file` should be used to pass path to the following configuration file: +To enforce this, `kube-apiserver` flag `--admission-control-config-file` should be +used to pass path to the following configuration file: -{{< tabs name="example1" >}} -{{% tab name="apiserver.config.k8s.io/v1" %}} ```yaml apiVersion: apiserver.config.k8s.io/v1 kind: AdmissionConfiguration @@ -573,30 +610,10 @@ plugins: operator: In values: ["cluster-services"] ``` -{{% /tab %}} -{{% tab name="apiserver.k8s.io/v1alpha1" %}} -```yaml -# Deprecated in v1.17 in favor of apiserver.config.k8s.io/v1 -apiVersion: apiserver.k8s.io/v1alpha1 -kind: AdmissionConfiguration -plugins: -- name: "ResourceQuota" - configuration: - # Deprecated in v1.17 in favor of apiserver.config.k8s.io/v1, ResourceQuotaConfiguration - apiVersion: resourcequota.admission.k8s.io/v1beta1 - kind: Configuration - limitedResources: - - resource: pods - matchScopes: - - scopeName: PriorityClass - operator: In - values: ["cluster-services"] -``` -{{% /tab %}} -{{< /tabs >}} Now, "cluster-services" pods will be allowed in only those namespaces where a quota object with a matching `scopeSelector` is present. For example: + ```yaml scopeSelector: matchExpressions: @@ -605,13 +622,9 @@ For example: values: ["cluster-services"] ``` -See [LimitedResources](https://github.com/kubernetes/kubernetes/pull/36765) and [Quota support for priority class design doc](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/scheduling/pod-priority-resourcequota.md) for more information. - -## Example - -See a [detailed example for how to use resource quota](/docs/tasks/administer-cluster/quota-api-object/). - ## {{% heading "whatsnext" %}} - See [ResourceQuota design doc](https://git.k8s.io/community/contributors/design-proposals/resource-management/admission_control_resource_quota.md) for more information. - +- See a [detailed example for how to use resource quota](/docs/tasks/administer-cluster/quota-api-object/). +- Read [Quota support for priority class design doc](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/scheduling/pod-priority-resourcequota.md). +- See [LimitedResources](https://github.com/kubernetes/kubernetes/pull/36765) diff --git a/content/en/docs/concepts/scheduling-eviction/kube-scheduler.md b/content/en/docs/concepts/scheduling-eviction/kube-scheduler.md index cf7e2713b9..a3e548d237 100644 --- a/content/en/docs/concepts/scheduling-eviction/kube-scheduler.md +++ b/content/en/docs/concepts/scheduling-eviction/kube-scheduler.md @@ -79,7 +79,7 @@ of the scheduler: 1. [Scheduling Policies](/docs/reference/scheduling/policies) allow you to configure _Predicates_ for filtering and _Priorities_ for scoring. -1. [Scheduling Profiles](/docs/reference/scheduling/profiles) allow you to configure Plugins that implement different scheduling stages, including: `QueueSort`, `Filter`, `Score`, `Bind`, `Reserve`, `Permit`, and others. You can also configure the kube-scheduler to run different profiles. +1. [Scheduling Profiles](/docs/reference/scheduling/config/#profiles) allow you to configure Plugins that implement different scheduling stages, including: `QueueSort`, `Filter`, `Score`, `Bind`, `Reserve`, `Permit`, and others. You can also configure the kube-scheduler to run different profiles. ## {{% heading "whatsnext" %}} @@ -89,7 +89,7 @@ of the scheduler: * Read the [reference documentation](/docs/reference/command-line-tools-reference/kube-scheduler/) for kube-scheduler * Learn about [configuring multiple schedulers](/docs/tasks/extend-kubernetes/configure-multiple-schedulers/) * Learn about [topology management policies](/docs/tasks/administer-cluster/topology-manager/) -* Learn about [Pod Overhead](/docs/concepts/configuration/pod-overhead/) +* Learn about [Pod Overhead](/docs/concepts/scheduling-eviction/pod-overhead/) * Learn about scheduling of Pods that use volumes in: * [Volume Topology Support](/docs/concepts/storage/storage-classes/#volume-binding-mode) * [Storage Capacity Tracking](/docs/concepts/storage/storage-capacity/) diff --git a/content/en/docs/concepts/configuration/pod-overhead.md b/content/en/docs/concepts/scheduling-eviction/pod-overhead.md similarity index 100% rename from content/en/docs/concepts/configuration/pod-overhead.md rename to content/en/docs/concepts/scheduling-eviction/pod-overhead.md diff --git a/content/en/docs/concepts/scheduling-eviction/scheduling-framework.md b/content/en/docs/concepts/scheduling-eviction/scheduling-framework.md index 5274c2ba24..61b6619d01 100644 --- a/content/en/docs/concepts/scheduling-eviction/scheduling-framework.md +++ b/content/en/docs/concepts/scheduling-eviction/scheduling-framework.md @@ -18,7 +18,7 @@ scheduling "core" simple and maintainable. Refer to the [design proposal of the scheduling framework][kep] for more technical information on the design of the framework. -[kep]: https://github.com/kubernetes/enhancements/blob/master/keps/sig-scheduling/20180409-scheduling-framework.md +[kep]: https://github.com/kubernetes/enhancements/blob/master/keps/sig-scheduling/624-scheduling-framework/README.md diff --git a/content/en/docs/reference/access-authn-authz/controlling-access.md b/content/en/docs/concepts/security/controlling-access.md similarity index 58% rename from content/en/docs/reference/access-authn-authz/controlling-access.md rename to content/en/docs/concepts/security/controlling-access.md index e0273177aa..62dc273cf7 100644 --- a/content/en/docs/reference/access-authn-authz/controlling-access.md +++ b/content/en/docs/concepts/security/controlling-access.md @@ -4,7 +4,6 @@ reviewers: - lavalamp title: Controlling Access to the Kubernetes API content_type: concept -weight: 5 --- @@ -12,7 +11,7 @@ This page provides an overview of controlling access to the Kubernetes API. -Users [access the API](/docs/tasks/access-application-cluster/access-cluster/) using `kubectl`, +Users access the [Kubernetes API](/docs/concepts/overview/kubernetes-api/) using `kubectl`, client libraries, or by making REST requests. Both human users and [Kubernetes service accounts](/docs/tasks/configure-pod-container/configure-service-account/) can be authorized for API access. @@ -21,45 +20,46 @@ following diagram: ![Diagram of request handling steps for Kubernetes API request](/images/docs/admin/access-control-overview.svg) -## Transport Security +## Transport security -In a typical Kubernetes cluster, the API serves on port 443. -The API server presents a certificate. This certificate is -often self-signed, so `$USER/.kube/config` on the user's machine typically -contains the root certificate for the API server's certificate, which when specified -is used in place of the system default root certificate. This certificate is typically -automatically written into your `$USER/.kube/config` when you create a cluster yourself -using `kube-up.sh`. If the cluster has multiple users, then the creator needs to share -the certificate with other users. +In a typical Kubernetes cluster, the API serves on port 443, protected by TLS. +The API server presents a certificate. This certificate may be signed using +a private certificate authority (CA), or based on a public key infrastructure linked +to a generally recognized CA. + +If your cluster uses a private certificate authority, you need a copy of that CA +certifcate configured into your `~/.kube/config` on the client, so that you can +trust the connection and be confident it was not intercepted. + +Your client can present a TLS client certificate at this stage. ## Authentication Once TLS is established, the HTTP request moves to the Authentication step. This is shown as step **1** in the diagram. The cluster creation script or cluster admin configures the API server to run -one or more Authenticator Modules. -Authenticators are described in more detail [here](/docs/reference/access-authn-authz/authentication/). +one or more Authenticator modules. +Authenticators are described in more detail in +[Authentication](/docs/reference/access-authn-authz/authentication/). -The input to the authentication step is the entire HTTP request, however, it typically +The input to the authentication step is the entire HTTP request; however, it typically just examines the headers and/or client certificate. -Authentication modules include Client Certificates, Password, and Plain Tokens, -Bootstrap Tokens, and JWT Tokens (used for service accounts). +Authentication modules include client certificates, password, and plain tokens, +bootstrap tokens, and JSON Web Tokens (used for service accounts). Multiple authentication modules can be specified, in which case each one is tried in sequence, until one of them succeeds. -On GCE, Client Certificates, Password, Plain Tokens, and JWT Tokens are all enabled. - If the request cannot be authenticated, it is rejected with HTTP status code 401. Otherwise, the user is authenticated as a specific `username`, and the user name is available to subsequent steps to use in their decisions. Some authenticators also provide the group memberships of the user, while other authenticators do not. -While Kubernetes uses `usernames` for access control decisions and in request logging, -it does not have a `user` object nor does it store usernames or other information about -users in its object store. +While Kubernetes uses usernames for access control decisions and in request logging, +it does not have a `User` object nor does it store usernames or other information about +users in its API. ## Authorization @@ -101,16 +101,16 @@ If Bob makes a request to write (`create` or `update`) to the objects in the `pr Kubernetes authorization requires that you use common REST attributes to interact with existing organization-wide or cloud-provider-wide access control systems. It is important to use REST formatting because these control systems might interact with other APIs besides the Kubernetes API. -Kubernetes supports multiple authorization modules, such as ABAC mode, RBAC Mode, and Webhook mode. When an administrator creates a cluster, they configured the authorization modules that should be used in the API server. If more than one authorization modules are configured, Kubernetes checks each module, and if any module authorizes the request, then the request can proceed. If all of the modules deny the request, then the request is denied (HTTP status code 403). +Kubernetes supports multiple authorization modules, such as ABAC mode, RBAC Mode, and Webhook mode. When an administrator creates a cluster, they configure the authorization modules that should be used in the API server. If more than one authorization modules are configured, Kubernetes checks each module, and if any module authorizes the request, then the request can proceed. If all of the modules deny the request, then the request is denied (HTTP status code 403). -To learn more about Kubernetes authorization, including details about creating policies using the supported authorization modules, see [Authorization Overview](/docs/reference/access-authn-authz/authorization/). +To learn more about Kubernetes authorization, including details about creating policies using the supported authorization modules, see [Authorization](/docs/reference/access-authn-authz/authorization/). -## Admission Control +## Admission control -Admission Control Modules are software modules that can modify or reject requests. -In addition to the attributes available to Authorization Modules, Admission -Control Modules can access the contents of the object that is being created or modified. +Admission Control modules are software modules that can modify or reject requests. +In addition to the attributes available to Authorization modules, Admission +Control modules can access the contents of the object that is being created or modified. Admission controllers act on requests that create, modify, delete, or connect to (proxy) an object. Admission controllers do not act on requests that merely read objects. @@ -118,26 +118,26 @@ When multiple admission controllers are configured, they are called in order. This is shown as step **3** in the diagram. -Unlike Authentication and Authorization Modules, if any admission controller module +Unlike Authentication and Authorization modules, if any admission controller module rejects, then the request is immediately rejected. In addition to rejecting objects, admission controllers can also set complex defaults for fields. -The available Admission Control Modules are described [here](/docs/reference/access-authn-authz/admission-controllers/). +The available Admission Control modules are described in [Admission Controllers](/docs/reference/access-authn-authz/admission-controllers/). Once a request passes all admission controllers, it is validated using the validation routines for the corresponding API object, and then written to the object store (shown as step **4**). -## API Server Ports and IPs +## API server ports and IPs The previous discussion applies to requests sent to the secure port of the API server (the typical case). The API server can actually serve on 2 ports: By default the Kubernetes API server serves HTTP on 2 ports: - 1. `Localhost Port`: + 1. `localhost` port: - is intended for testing and bootstrap, and for other components of the master node (scheduler, controller-manager) to talk to the API @@ -148,7 +148,7 @@ By default the Kubernetes API server serves HTTP on 2 ports: - request handled by admission control module(s). - protected by need to have host access - 2. `Secure Port`: + 2. “Secure port”: - use whenever possible - uses TLS. Set cert with `--tls-cert-file` and key with `--tls-private-key-file` flag. @@ -158,8 +158,27 @@ By default the Kubernetes API server serves HTTP on 2 ports: - request handled by admission control module(s). - authentication and authorization modules run. -When the cluster is created by `kube-up.sh`, on Google Compute Engine (GCE), -and on several other cloud providers, the API server serves on port 443. On -GCE, a firewall rule is configured on the project to allow external HTTPS -access to the API. Other cluster setup methods vary. +## {{% heading "whatsnext" %}} +Read more documentation on authentication, authorization and API access control: + +- [Authenticating](/docs/reference/access-authn-authz/authentication/) + - [Authenticating with Bootstrap Tokens](/docs/reference/access-authn-authz/bootstrap-tokens/) +- [Admission Controllers](/docs/reference/access-authn-authz/admission-controllers/) + - [Dynamic Admission Control](/docs/reference/access-authn-authz/extensible-admission-controllers/) +- [Authorization](/docs/reference/access-authn-authz/authorization/) + - [Role Based Access Control](/docs/reference/access-authn-authz/rbac/) + - [Attribute Based Access Control](/docs/reference/access-authn-authz/abac/) + - [Node Authorization](/docs/reference/access-authn-authz/node/) + - [Webhook Authorization](/docs/reference/access-authn-authz/webhook/) +- [Certificate Signing Requests](/docs/reference/access-authn-authz/certificate-signing-requests/) + - including [CSR approval](/docs/reference/access-authn-authz/certificate-signing-requests/#approval-rejection) + and [certificate signing](/docs/reference/access-authn-authz/certificate-signing-requests/#signing) +- Service accounts + - [Developer guide](/docs/tasks/configure-pod-container/configure-service-account/) + - [Administration](/docs/reference/access-authn-authz/service-accounts-admin/) + +You can learn about: +- how Pods can use + [Secrets](/docs/concepts/configuration/secret/#service-accounts-automatically-create-and-attach-secrets-with-api-credentials) + to obtain API credentials. diff --git a/content/en/docs/concepts/security/overview.md b/content/en/docs/concepts/security/overview.md index 98776c7199..fe9129c109 100644 --- a/content/en/docs/concepts/security/overview.md +++ b/content/en/docs/concepts/security/overview.md @@ -103,7 +103,7 @@ areas of security concerns and recommendations for securing workloads running in Area of Concern for Workload Security | Recommendation | ------------------------------ | --------------------- | RBAC Authorization (Access to the Kubernetes API) | https://kubernetes.io/docs/reference/access-authn-authz/rbac/ -Authentication | https://kubernetes.io/docs/reference/access-authn-authz/controlling-access/ +Authentication | https://kubernetes.io/docs/concepts/security/controlling-access/ Application secrets management (and encrypting them in etcd at rest) | https://kubernetes.io/docs/concepts/configuration/secret/
https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/ Pod Security Policies | https://kubernetes.io/docs/concepts/policy/pod-security-policy/ Quality of Service (and Cluster resource management) | https://kubernetes.io/docs/tasks/configure-pod-container/quality-service-pod/ @@ -147,8 +147,8 @@ Learn about related Kubernetes security topics: * [Pod security standards](/docs/concepts/security/pod-security-standards/) * [Network policies for Pods](/docs/concepts/services-networking/network-policies/) +* [Controlling Access to the Kubernetes API](/docs/concepts/security/controlling-access) * [Securing your cluster](/docs/tasks/administer-cluster/securing-a-cluster/) -* [API access control](/docs/reference/access-authn-authz/controlling-access/) * [Data encryption in transit](/docs/tasks/tls/managing-tls-in-a-cluster/) for the control plane * [Data encryption at rest](/docs/tasks/administer-cluster/encrypt-data/) * [Secrets in Kubernetes](/docs/concepts/configuration/secret/) diff --git a/content/en/docs/concepts/services-networking/dual-stack.md b/content/en/docs/concepts/services-networking/dual-stack.md index 427b8b0a53..234e16dc25 100644 --- a/content/en/docs/concepts/services-networking/dual-stack.md +++ b/content/en/docs/concepts/services-networking/dual-stack.md @@ -98,7 +98,7 @@ On cloud providers which support IPv6 enabled external load balancers, setting t ## Egress Traffic -The use of publicly routable and non-publicly routable IPv6 address blocks is acceptable provided the underlying {{< glossary_tooltip text="CNI" term_id="cni" >}} provider is able to implement the transport. If you have a Pod that uses non-publicly routable IPv6 and want that Pod to reach off-cluster destinations (eg. the public Internet), you must set up IP masquerading for the egress traffic and any replies. The [ip-masq-agent](https://github.com/kubernetes-incubator/ip-masq-agent) is dual-stack aware, so you can use ip-masq-agent for IP masquerading on dual-stack clusters. +The use of publicly routable and non-publicly routable IPv6 address blocks is acceptable provided the underlying {{< glossary_tooltip text="CNI" term_id="cni" >}} provider is able to implement the transport. If you have a Pod that uses non-publicly routable IPv6 and want that Pod to reach off-cluster destinations (eg. the public Internet), you must set up IP masquerading for the egress traffic and any replies. The [ip-masq-agent](https://github.com/kubernetes-sigs/ip-masq-agent) is dual-stack aware, so you can use ip-masq-agent for IP masquerading on dual-stack clusters. ## Known Issues diff --git a/content/en/docs/concepts/services-networking/ingress.md b/content/en/docs/concepts/services-networking/ingress.md index 7743ae656a..3977cbc233 100644 --- a/content/en/docs/concepts/services-networking/ingress.md +++ b/content/en/docs/concepts/services-networking/ingress.md @@ -413,6 +413,13 @@ secure the channel from the client to the load balancer using TLS. You need to m sure the TLS secret you created came from a certificate that contains a Common Name (CN), also known as a Fully Qualified Domain Name (FQDN) for `https-example.foo.com`. +{{< note >}} +Keep in mind that TLS will not work on the default rule because the +certificates would have to be issued for all the possible sub-domains. Therefore, +`hosts` in the `tls` section need to explicitly match the `host` in the `rules` +section. +{{< /note >}} + {{< codenew file="service/networking/tls-example-ingress.yaml" >}} {{< note >}} diff --git a/content/en/docs/concepts/storage/ephemeral-volumes.md b/content/en/docs/concepts/storage/ephemeral-volumes.md index e4831cfd16..30029eae12 100644 --- a/content/en/docs/concepts/storage/ephemeral-volumes.md +++ b/content/en/docs/concepts/storage/ephemeral-volumes.md @@ -46,10 +46,9 @@ different purposes: [downwardAPI](/docs/concepts/storage/volumes/#downwardapi), [secret](/docs/concepts/storage/volumes/#secret): inject different kinds of Kubernetes data into a Pod -- [CSI ephemeral - volumes](docs/concepts/storage/volumes/#csi-ephemeral-volumes): - similar to the previous volume kinds, but provided by special [CSI - drivers](https://github.com/container-storage-interface/spec/blob/master/spec.md) +- [CSI ephemeral volumes](#csi-ephemeral-volume): + similar to the previous volume kinds, but provided by special + [CSI drivers](https://github.com/container-storage-interface/spec/blob/master/spec.md) which specifically [support this feature](https://kubernetes-csi.github.io/docs/drivers.html) - [generic ephemeral volumes](#generic-ephemeral-volumes), which can be provided by all storage drivers that also support persistent volumes @@ -181,8 +180,8 @@ spec: ### Lifecycle and PersistentVolumeClaim -The key design idea is that the [parameters for a -volume claim](/docs/reference/generated/kubernetes-api/#ephemeralvolumesource-v1alpha1-core) +The key design idea is that the +[parameters for a volume claim](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#ephemeralvolumesource-v1alpha1-core) are allowed inside a volume source of the Pod. Labels, annotations and the whole set of fields for a PersistentVolumeClaim are supported. When such a Pod gets created, the ephemeral volume controller then creates an actual PersistentVolumeClaim diff --git a/content/en/docs/concepts/storage/persistent-volumes.md b/content/en/docs/concepts/storage/persistent-volumes.md index 9fb9fdae45..cf915bfa52 100644 --- a/content/en/docs/concepts/storage/persistent-volumes.md +++ b/content/en/docs/concepts/storage/persistent-volumes.md @@ -183,27 +183,29 @@ By specifying a PersistentVolume in a PersistentVolumeClaim, you declare a bindi If the PersistentVolume exists and has not reserved PersistentVolumeClaims through its `claimRef` field, then the PersistentVolume and PersistentVolumeClaim will be bound. The binding happens regardless of some volume matching criteria, including node affinity. -The control plane still checks that [storage class](https://kubernetes.io/docs/concepts/storage/storage-classes/), access modes, and requested storage size are valid. +The control plane still checks that [storage class](/docs/concepts/storage/storage-classes/), access modes, and requested storage size are valid. -``` +```yaml apiVersion: v1 kind: PersistentVolumeClaim metadata: name: foo-pvc namespace: foo spec: + storageClassName: "" # Empty string must be explicitly set otherwise default StorageClass will be set volumeName: foo-pv ... ``` This method does not guarantee any binding privileges to the PersistentVolume. If other PersistentVolumeClaims could use the PV that you specify, you first need to reserve that storage volume. Specify the relevant PersistentVolumeClaim in the `claimRef` field of the PV so that other PVCs can not bind to it. -``` +```yaml apiVersion: v1 kind: PersistentVolume metadata: name: foo-pv spec: + storageClassName: "" claimRef: name: foo-pvc namespace: foo diff --git a/content/en/docs/concepts/storage/storage-classes.md b/content/en/docs/concepts/storage/storage-classes.md index 59579b443c..587cc8a501 100644 --- a/content/en/docs/concepts/storage/storage-classes.md +++ b/content/en/docs/concepts/storage/storage-classes.md @@ -94,7 +94,7 @@ run, what volume plugin it uses (including Flex), etc. The repository [kubernetes-sigs/sig-storage-lib-external-provisioner](https://github.com/kubernetes-sigs/sig-storage-lib-external-provisioner) houses a library for writing external provisioners that implements the bulk of the specification. Some external provisioners are listed under the repository -[kubernetes-incubator/external-storage](https://github.com/kubernetes-incubator/external-storage). +[kubernetes-sigs/external-storage](https://github.com/kubernetes-sigs/external-dns). For example, NFS doesn't provide an internal provisioner, but an external provisioner can be used. There are also cases when 3rd party storage diff --git a/content/en/docs/concepts/storage/volumes.md b/content/en/docs/concepts/storage/volumes.md index 19d491b1b3..e4f38e7d2b 100644 --- a/content/en/docs/concepts/storage/volumes.md +++ b/content/en/docs/concepts/storage/volumes.md @@ -18,7 +18,7 @@ Container starts with a clean state. Second, when running Containers together in a `Pod` it is often necessary to share files between those Containers. The Kubernetes `Volume` abstraction solves both of these problems. -Familiarity with [Pods](/docs/concepts/workloads/pods/pod/) is suggested. +Familiarity with [Pods](/docs/concepts/workloads/pods/) is suggested. @@ -52,8 +52,7 @@ field) and where to mount those into Containers (the field). A process in a container sees a filesystem view composed from their Docker -image and volumes. The [Docker -image](https://docs.docker.com/userguide/dockerimages/) is at the root of the +image and volumes. The [Docker image](https://docs.docker.com/userguide/dockerimages/) is at the root of the filesystem hierarchy, and any volumes are mounted at the specified paths within the image. Volumes can not mount onto other volumes or have hard links to other volumes. Each Container in the Pod must independently specify where to diff --git a/content/en/docs/concepts/workloads/_index.md b/content/en/docs/concepts/workloads/_index.md index 1aac095cb5..21e57aed03 100644 --- a/content/en/docs/concepts/workloads/_index.md +++ b/content/en/docs/concepts/workloads/_index.md @@ -3,5 +3,55 @@ title: "Workloads" weight: 50 description: > Understand Pods, the smallest deployable compute object in Kubernetes, and the higher-level abstractions that help you to run them. +no_list: true --- +{{< glossary_definition term_id="workload" length="short" >}} +Whether your workload is a single component or several that work together, on Kubernetes you run +it inside a set of [Pods](/docs/concepts/workloads/pods). +In Kubernetes, a Pod represents a set of running {{< glossary_tooltip text="containers" term_id="container" >}} +on your cluster. + +A Pod has a defined lifecycle. For example, once a Pod is running in your cluster then +a critical failure on the {{< glossary_tooltip text="node" term_id="node" >}} where that +Pod is running means that all the Pods on that node fail. Kubernetes treats that level +of failure as final: you would need to create a new Pod even if the node later recovers. + +However, to make life considerably easier, you don't need to manage each Pod directly. +Instead, you can use _workload resources_ that manage a set of Pods on your behalf. +These resources configure {{< glossary_tooltip term_id="controller" text="controllers" >}} +that make sure the right number of the right kind of Pod are running, to match the state +you specified. + +Those workload resources include: + +* [Deployment](/docs/concepts/workloads/controllers/deployment/) and [ReplicaSet](/docs/concepts/workloads/controllers/replicaset/) + (replacing the legacy resource {{< glossary_tooltip text="ReplicationController" term_id="replication-controller" >}}); +* [StatefulSet](/docs/concepts/workloads/controllers/statefulset/); +* [DaemonSet](/docs/concepts/workloads/controllers/daemonset/) for running Pods that provide + node-local facilities, such as a storage driver or network plugin; +* [Job](/docs/concepts/workloads/controllers/job/) and + [CronJob](/docs/concepts/workloads/controllers/cron-jobs/) + for tasks that run to completion. + +There are also two supporting concepts that you might find relevant: +* [Garbage collection](/docs/concepts/workloads/controllers/garbage-collection/) tidies up objects + from your cluster after their _owning resource_ has been removed. +* The [_time-to-live after finished_ controller](/docs/concepts/workloads/controllers/ttlafterfinished/) + removes Jobs once a defined time has passed since they completed. + +## {{% heading "whatsnext" %}} + +As well as reading about each resource, you can learn about specific tasks that relate to them: + +* [Run a stateless application using a Deployment](/docs/tasks/run-application/run-stateless-application-deployment/) +* Run a stateful application either as a [single instance](/docs/tasks/run-application/run-single-instance-stateful-application/) + or as a [replicated set](/docs/tasks/run-application/run-replicated-stateful-application/) +* [Run Automated Tasks with a CronJob](/docs/tasks/job/automated-tasks-with-cron-jobs/) + +Once your application is running, you might want to make it available on the internet as +a [Service](/docs/concepts/services-networking/service/) or, for web application only, +using an [Ingress](/docs/concepts/services-networking/ingress). + +You can also visit [Configuration](/docs/concepts/configuration/) to learn about Kubernetes' +mechanisms for separating code from configuration. diff --git a/content/en/docs/concepts/workloads/controllers/_index.md b/content/en/docs/concepts/workloads/controllers/_index.md index 6aaa7405b5..a18ad08a38 100755 --- a/content/en/docs/concepts/workloads/controllers/_index.md +++ b/content/en/docs/concepts/workloads/controllers/_index.md @@ -1,5 +1,5 @@ --- -title: "Controllers" +title: "Workload Resources" weight: 20 --- diff --git a/content/en/docs/concepts/workloads/controllers/deployment.md b/content/en/docs/concepts/workloads/controllers/deployment.md index 3eedc64d57..54448de92f 100644 --- a/content/en/docs/concepts/workloads/controllers/deployment.md +++ b/content/en/docs/concepts/workloads/controllers/deployment.md @@ -752,7 +752,7 @@ apply multiple fixes in between pausing and resuming without triggering unnecess REVISION CHANGE-CAUSE 1 ``` -* Get the rollout status to ensure that the Deployment is updates successfully: +* Get the rollout status to ensure that the Deployment is updated successfully: ```shell kubectl get rs ``` diff --git a/content/en/docs/concepts/workloads/controllers/replicaset.md b/content/en/docs/concepts/workloads/controllers/replicaset.md index bdded6d27e..ba0270b573 100644 --- a/content/en/docs/concepts/workloads/controllers/replicaset.md +++ b/content/en/docs/concepts/workloads/controllers/replicaset.md @@ -233,7 +233,7 @@ A ReplicaSet also needs a [`.spec` section](https://git.k8s.io/community/contrib ### Pod Template -The `.spec.template` is a [pod template](/docs/concepts/workloads/Pods/pod-overview/#pod-templates) which is also +The `.spec.template` is a [pod template](/docs/concepts/workloads/pods/#pod-templates) which is also required to have labels in place. In our `frontend.yaml` example we had one label: `tier: frontend`. Be careful not to overlap with the selectors of other controllers, lest they try to adopt this Pod. @@ -351,7 +351,7 @@ Unlike the case where a user directly created Pods, a ReplicaSet replaces Pods t ### Job -Use a [`Job`](/docs/concepts/jobs/run-to-completion-finite-workloads/) instead of a ReplicaSet for Pods that are expected to terminate on their own +Use a [`Job`](/docs/concepts/workloads/controllers/job/) instead of a ReplicaSet for Pods that are expected to terminate on their own (that is, batch jobs). ### DaemonSet diff --git a/content/en/docs/concepts/workloads/pods/_index.md b/content/en/docs/concepts/workloads/pods/_index.md index 9247cd3a0d..1353d297a1 100644 --- a/content/en/docs/concepts/workloads/pods/_index.md +++ b/content/en/docs/concepts/workloads/pods/_index.md @@ -145,7 +145,7 @@ from a _pod template_ and manage those Pods on your behalf. PodTemplates are specifications for creating Pods, and are included in workload resources such as [Deployments](/docs/concepts/workloads/controllers/deployment/), -[Jobs](/docs/concepts/jobs/run-to-completion-finite-workloads/), and +[Jobs](/docs/concepts/workloads/controllers/job/), and [DaemonSets](/docs/concepts/workloads/controllers/daemonset/). Each controller for a workload resource uses the `PodTemplate` inside the workload @@ -264,7 +264,7 @@ but cannot be controlled from there. * [The Distributed System Toolkit: Patterns for Composite Containers](https://kubernetes.io/blog/2015/06/the-distributed-system-toolkit-patterns) explains common layouts for Pods with more than one container. To understand the context for why Kubernetes wraps a common Pod API in other resources (such as {{< glossary_tooltip text="StatefulSets" term_id="statefulset" >}} or {{< glossary_tooltip text="Deployments" term_id="deployment" >}}, you can read about the prior art, including: - * [Aurora](http://aurora.apache.org/documentation/latest/reference/configuration/#job-schema) + * [Aurora](https://aurora.apache.org/documentation/latest/reference/configuration/#job-schema) * [Borg](https://research.google.com/pubs/pub43438.html) * [Marathon](https://mesosphere.github.io/marathon/docs/rest-api.html) * [Omega](https://research.google/pubs/pub41684/) diff --git a/content/en/docs/concepts/workloads/pods/pod-lifecycle.md b/content/en/docs/concepts/workloads/pods/pod-lifecycle.md index f0ac6e654b..2269a15293 100644 --- a/content/en/docs/concepts/workloads/pods/pod-lifecycle.md +++ b/content/en/docs/concepts/workloads/pods/pod-lifecycle.md @@ -77,13 +77,13 @@ have a given `phase` value. Here are the possible values for `phase`: -Value | Description -:-----|:----------- -`Pending` | The Pod has been accepted by the Kubernetes cluster, but one or more of the containers has not been set up and made ready to run. This includes time a Pod spends waiting to be scheduled as well as the time spent downloading container images over the network. -`Running` | The Pod has been bound to a node, and all of the containers have been created. At least one container is still running, or is in the process of starting or restarting. +Value | Description +:-----------|:----------- +`Pending` | The Pod has been accepted by the Kubernetes cluster, but one or more of the containers has not been set up and made ready to run. This includes time a Pod spends waiting to be scheduled as well as the time spent downloading container images over the network. +`Running` | The Pod has been bound to a node, and all of the containers have been created. At least one container is still running, or is in the process of starting or restarting. `Succeeded` | All containers in the Pod have terminated in success, and will not be restarted. -`Failed` | All containers in the Pod have terminated, and at least one container has terminated in failure. That is, the container either exited with non-zero status or was terminated by the system. -`Unknown` | For some reason the state of the Pod could not be obtained. This phase typically occurs due to an error in communicating with the node where the Pod should be running. +`Failed` | All containers in the Pod have terminated, and at least one container has terminated in failure. That is, the container either exited with non-zero status or was terminated by the system. +`Unknown` | For some reason the state of the Pod could not be obtained. This phase typically occurs due to an error in communicating with the node where the Pod should be running. If a node dies or is disconnected from the rest of the cluster, Kubernetes applies a policy for setting the `phase` of all Pods on the lost node to Failed. @@ -228,7 +228,8 @@ When a Pod's containers are Ready but at least one custom condition is missing o ## Container probes A [Probe](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#probe-v1-core) is a diagnostic -performed periodically by the [kubelet](/docs/admin/kubelet/) +performed periodically by the +[kubelet](/docs/reference/command-line-tools-reference/kubelet/) on a Container. To perform a diagnostic, the kubelet calls a [Handler](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#handler-v1-core) implemented by diff --git a/content/en/docs/contribute/localization.md b/content/en/docs/contribute/localization.md index 5e91d86ee8..48c48b6ed3 100644 --- a/content/en/docs/contribute/localization.md +++ b/content/en/docs/contribute/localization.md @@ -32,7 +32,7 @@ First, consult the [ISO 639-1 standard](https://www.loc.gov/standards/iso639-2/p ### Fork and clone the repo -First, [create your own fork](/docs/contribute/new-content/new-content/#fork-the-repo) of the [kubernetes/website](https://github.com/kubernetes/website) repository. +First, [create your own fork](/docs/contribute/new-content/open-a-pr/#fork-the-repo) of the [kubernetes/website](https://github.com/kubernetes/website) repository. Then, clone your fork and `cd` into it: diff --git a/content/en/docs/contribute/new-content/blogs-case-studies.md b/content/en/docs/contribute/new-content/blogs-case-studies.md index 2ec9f35ac0..6628925694 100644 --- a/content/en/docs/contribute/new-content/blogs-case-studies.md +++ b/content/en/docs/contribute/new-content/blogs-case-studies.md @@ -25,7 +25,7 @@ Anyone can write a blog post and submit it for review. ### Guidelines and expectations - Blog posts should not be vendor pitches. - - Articles must contain content that applies broadly to the Kubernetes community. For example, a submission should focus on upstream Kubernetes as opposed to vendor-specific configurations. Check the [Documentation style guide](https://kubernetes.io/docs/contribute/style/content-guide/#what-s-allowed) for what is typically allowed on Kubernetes properties. + - Articles must contain content that applies broadly to the Kubernetes community. For example, a submission should focus on upstream Kubernetes as opposed to vendor-specific configurations. Check the [Documentation style guide](/docs/contribute/style/content-guide/#what-s-allowed) for what is typically allowed on Kubernetes properties. - Links should primarily be to the official Kubernetes documentation. When using external references, links should be diverse - For example a submission shouldn't contain only links back to a single company's blog. - Sometimes this is a delicate balance. The [blog team](https://kubernetes.slack.com/messages/sig-docs-blog/) is there to give guidance on whether a post is appropriate for the Kubernetes blog, so don't hesitate to reach out. - Blog posts are not published on specific dates. @@ -56,7 +56,7 @@ The SIG Docs [blog subproject](https://github.com/kubernetes/community/tree/mast To submit a blog post follow these directions: -- [Open a pull request](/docs/contribute/new-content/new-content/#fork-the-repo) with a new blog post. New blog posts go under the [`content/en/blog/_posts`](https://github.com/kubernetes/website/tree/master/content/en/blog/_posts) directory. +- [Open a pull request](/docs/contribute/new-content/open-a-pr/#fork-the-repo) with a new blog post. New blog posts go under the [`content/en/blog/_posts`](https://github.com/kubernetes/website/tree/master/content/en/blog/_posts) directory. - Ensure that your blog post follows the correct naming conventions and the following frontmatter (metadata) information: diff --git a/content/en/docs/contribute/new-content/new-features.md b/content/en/docs/contribute/new-content/new-features.md index 98823185ff..4e70f6c9ec 100644 --- a/content/en/docs/contribute/new-content/new-features.md +++ b/content/en/docs/contribute/new-content/new-features.md @@ -98,7 +98,8 @@ deadlines. 1. Open a pull request against the `dev-{{< skew nextMinorVersion >}}` branch in the `kubernetes/website` repository, with a small commit that you will amend later. -2. Edit the pull request description to include links to `k/k` PR(s) and `k/enhancement` issue(s). +2. Edit the pull request description to include links to [kubernetes/kubernetes](https://github.com/kubernetes/kubernetes) +PR(s) and [kubernetes/enhancements](https://github.com/kubernetes/enhancements) issue(s). 3. Use the Prow command `/milestone {{< skew nextMinorVersion >}}` to assign the PR to the relevant milestone. This alerts the docs person managing this release that the feature docs are coming. @@ -115,7 +116,9 @@ When ready, populate your placeholder PR with feature documentation. Do your best to describe your feature and how to use it. If you need help structuring your documentation, ask in the `#sig-docs` slack channel. -When you complete your content, the documentation person assigned to your feature reviews it. Use their suggestions to get the content to a release ready state. +When you complete your content, the documentation person assigned to your feature reviews it. +To ensure technical accuracy, the content may also require a technical review from corresponding SIG(s). +Use their suggestions to get the content to a release ready state. If your feature needs documentation and the first draft content is not received, the feature may be removed from the milestone. diff --git a/content/en/docs/contribute/participate/pr-wranglers.md b/content/en/docs/contribute/participate/pr-wranglers.md index 00c2ebcdab..328de3b73c 100644 --- a/content/en/docs/contribute/participate/pr-wranglers.md +++ b/content/en/docs/contribute/participate/pr-wranglers.md @@ -6,7 +6,7 @@ weight: 20 -SIG Docs [approvers](/docs/contribute/participating/roles-and-responsibilites/#approvers) take week-long shifts [managing pull requests](https://github.com/kubernetes/website/wiki/PR-Wranglers) for the repository. +SIG Docs [approvers](/docs/contribute/participate/roles-and-responsibilities/#approvers) take week-long shifts [managing pull requests](https://github.com/kubernetes/website/wiki/PR-Wranglers) for the repository. This section covers the duties of a PR wrangler. For more information on giving good reviews, see [Reviewing changes](/docs/contribute/review/). diff --git a/content/en/docs/reference/_index.md b/content/en/docs/reference/_index.md index 06eeb347d8..bc95e7efeb 100644 --- a/content/en/docs/reference/_index.md +++ b/content/en/docs/reference/_index.md @@ -18,8 +18,8 @@ This section of the Kubernetes documentation contains references. ## API Reference -* [Kubernetes API Overview](/docs/reference/using-api/api-overview/) - Overview of the API for Kubernetes. * [Kubernetes API Reference {{< latest-version >}}](/docs/reference/generated/kubernetes-api/{{< latest-version >}}/) +* [Using The Kubernetes API](/docs/reference/using-api/) - overview of the API for Kubernetes. ## API Client Libraries @@ -36,7 +36,7 @@ client libraries: * [kubectl](/docs/reference/kubectl/overview/) - Main CLI tool for running commands and managing Kubernetes clusters. * [JSONPath](/docs/reference/kubectl/jsonpath/) - Syntax guide for using [JSONPath expressions](https://goessner.net/articles/JsonPath/) with kubectl. -* [kubeadm](/docs/reference/setup-tools/kubeadm/kubeadm/) - CLI tool to easily provision a secure Kubernetes cluster. +* [kubeadm](/docs/reference/setup-tools/kubeadm/) - CLI tool to easily provision a secure Kubernetes cluster. ## Components Reference diff --git a/content/en/docs/reference/access-authn-authz/_index.md b/content/en/docs/reference/access-authn-authz/_index.md index 4e1bff0818..d999e52bf5 100644 --- a/content/en/docs/reference/access-authn-authz/_index.md +++ b/content/en/docs/reference/access-authn-authz/_index.md @@ -1,4 +1,26 @@ --- -title: Accessing the API +title: API Access Control weight: 20 ---- \ No newline at end of file +no_list: true +--- + +For an introduction to how Kubernetes implements and controls API access, +read [Controlling Access to the Kubernetes API](/docs/concepts/security/controlling-access/). + +Reference documentation: + +- [Authenticating](/docs/reference/access-authn-authz/authentication/) + - [Authenticating with Bootstrap Tokens](/docs/reference/access-authn-authz/bootstrap-tokens/) +- [Admission Controllers](/docs/reference/access-authn-authz/admission-controllers/) + - [Dynamic Admission Control](/docs/reference/access-authn-authz/extensible-admission-controllers/) +- [Authorization](/docs/reference/access-authn-authz/authorization/) + - [Role Based Access Control](/docs/reference/access-authn-authz/rbac/) + - [Attribute Based Access Control](/docs/reference/access-authn-authz/abac/) + - [Node Authorization](/docs/reference/access-authn-authz/node/) + - [Webhook Authorization](/docs/reference/access-authn-authz/webhook/) +- [Certificate Signing Requests](/docs/reference/access-authn-authz/certificate-signing-requests/) + - including [CSR approval](/docs/reference/access-authn-authz/certificate-signing-requests/#approval-rejection) + and [certificate signing](/docs/reference/access-authn-authz/certificate-signing-requests/#signing) +- Service accounts + - [Developer guide](/docs/tasks/configure-pod-container/configure-service-account/) + - [Administration](/docs/reference/access-authn-authz/service-accounts-admin/) diff --git a/content/en/docs/reference/access-authn-authz/admission-controllers.md b/content/en/docs/reference/access-authn-authz/admission-controllers.md index 29639ac161..b4823245d3 100644 --- a/content/en/docs/reference/access-authn-authz/admission-controllers.md +++ b/content/en/docs/reference/access-authn-authz/admission-controllers.md @@ -730,7 +730,7 @@ See the [resourceQuota design doc](https://git.k8s.io/community/contributors/des For [RuntimeClass](/docs/concepts/containers/runtime-class/) definitions which describe an overhead associated with running a pod, this admission controller will set the pod.Spec.Overhead field accordingly. -See also [Pod Overhead](/docs/concepts/configuration/pod-overhead/) +See also [Pod Overhead](/docs/concepts/scheduling-eviction/pod-overhead/) for more information. ### SecurityContextDeny {#securitycontextdeny} diff --git a/content/en/docs/reference/access-authn-authz/authentication.md b/content/en/docs/reference/access-authn-authz/authentication.md index a97dca823f..e2e2d5c437 100644 --- a/content/en/docs/reference/access-authn-authz/authentication.md +++ b/content/en/docs/reference/access-authn-authz/authentication.md @@ -29,7 +29,7 @@ It is assumed that a cluster-independent service manages normal users in the fol In this regard, _Kubernetes does not have objects which represent normal user accounts._ Normal users cannot be added to a cluster through an API call. -Even though normal user cannot be added via an API call, but any user that +Even though a normal user cannot be added via an API call, any user that presents a valid certificate signed by the cluster's certificate authority (CA) is considered authenticated. In this configuration, Kubernetes determines the username from the common name field in the 'subject' of the cert (e.g., diff --git a/content/en/docs/reference/access-authn-authz/authorization.md b/content/en/docs/reference/access-authn-authz/authorization.md index 7a251726fb..a084a7e54e 100644 --- a/content/en/docs/reference/access-authn-authz/authorization.md +++ b/content/en/docs/reference/access-authn-authz/authorization.md @@ -17,7 +17,7 @@ policies using the supported authorization modules. In Kubernetes, you must be authenticated (logged in) before your request can be authorized (granted permission to access). For information about authentication, -see [Controlling Access to the Kubernetes API](/docs/reference/access-authn-authz/controlling-access/). +see [Controlling Access to the Kubernetes API](/docs/concepts/security/controlling-access/). Kubernetes expects attributes that are common to REST API requests. This means that Kubernetes authorization works with existing organization-wide or @@ -52,7 +52,7 @@ Kubernetes reviews only the following API request attributes: * **Resource** - The ID or name of the resource that is being accessed (for resource requests only) -- For resource requests using `get`, `update`, `patch`, and `delete` verbs, you must provide the resource name. * **Subresource** - The subresource that is being accessed (for resource requests only). * **Namespace** - The namespace of the object that is being accessed (for namespaced resource requests only). - * **API group** - The {{< glossary_tooltip text="API Group" term_id="api-group" >}} being accessed (for resource requests only). An empty string designates the [core API group](/docs/reference/using-api/api-overview/#api-groups). + * **API group** - The {{< glossary_tooltip text="API Group" term_id="api-group" >}} being accessed (for resource requests only). An empty string designates the _core_ [API group](/docs/reference/using-api/#api-groups). ## Determine the Request Verb @@ -202,6 +202,6 @@ action the account could take. This applies regardless of authorization mode. ## {{% heading "whatsnext" %}} -* To learn more about Authentication, see **Authentication** in [Controlling Access to the Kubernetes API](/docs/reference/access-authn-authz/controlling-access/). +* To learn more about Authentication, see **Authentication** in [Controlling Access to the Kubernetes API](/docs/concepts/security/controlling-access/). * To learn more about Admission Control, see [Using Admission Controllers](/docs/reference/access-authn-authz/admission-controllers/). diff --git a/content/en/docs/reference/access-authn-authz/bootstrap-tokens.md b/content/en/docs/reference/access-authn-authz/bootstrap-tokens.md index 542b5267be..856669a5d8 100644 --- a/content/en/docs/reference/access-authn-authz/bootstrap-tokens.md +++ b/content/en/docs/reference/access-authn-authz/bootstrap-tokens.md @@ -12,10 +12,10 @@ weight: 20 Bootstrap tokens are a simple bearer token that is meant to be used when creating new clusters or joining new nodes to an existing cluster. It was built -to support [kubeadm](/docs/reference/setup-tools/kubeadm/kubeadm/), but can be used in other contexts +to support [kubeadm](/docs/reference/setup-tools/kubeadm/), but can be used in other contexts for users that wish to start clusters without `kubeadm`. It is also built to -work, via RBAC policy, with the [Kubelet TLS -Bootstrapping](/docs/reference/command-line-tools-reference/kubelet-tls-bootstrapping/) system. +work, via RBAC policy, with the +[Kubelet TLS Bootstrapping](/docs/reference/command-line-tools-reference/kubelet-tls-bootstrapping/) system. diff --git a/content/en/docs/reference/access-authn-authz/certificate-signing-requests.md b/content/en/docs/reference/access-authn-authz/certificate-signing-requests.md index fc574c685f..6d05d0436a 100644 --- a/content/en/docs/reference/access-authn-authz/certificate-signing-requests.md +++ b/content/en/docs/reference/access-authn-authz/certificate-signing-requests.md @@ -21,17 +21,16 @@ A CertificateSigningRequest (CSR) resource is used to request that a certificate by a denoted signer, after which the request may be approved or denied before finally being signed. - - + ## Request signing process -The _CertificateSigningRequest_ resource type allows a client to ask for an X.509 certificate +The CertificateSigningRequest resource type allows a client to ask for an X.509 certificate be issued, based on a signing request. The CertificateSigningRequest object includes a PEM-encoded PKCS#10 signing request in -the `spec.request` field. The CertificateSigningRequest denotes the _signer_ (the +the `spec.request` field. The CertificateSigningRequest denotes the signer (the recipient that the request is being made to) using the `spec.signerName` field. -Note that `spec.signerName` is a required key after api version `certificates.k8s.io/v1`. +Note that `spec.signerName` is a required key after API version `certificates.k8s.io/v1`. Once created, a CertificateSigningRequest must be approved before it can be signed. Depending on the signer selected, a CertificateSigningRequest may be automatically approved @@ -68,54 +67,80 @@ This includes: 1. **Permitted subjects**: any restrictions on and behavior when a disallowed subject is requested. 1. **Permitted x509 extensions**: including IP subjectAltNames, DNS subjectAltNames, Email subjectAltNames, URI subjectAltNames etc, and behavior when a disallowed extension is requested. 1. **Permitted key usages / extended key usages**: any restrictions on and behavior when usages different than the signer-determined usages are specified in the CSR. -1. **Expiration/certificate lifetime**: whether it is fixed by the signer, configurable by the admin, determined by the CSR object etc and the behavior when an expiration is different than the signer-determined expiration that is specified in the CSR. +1. **Expiration/certificate lifetime**: whether it is fixed by the signer, configurable by the admin, determined by the CSR object etc + and the behavior when an expiration is different than the signer-determined expiration that is specified in the CSR. 1. **CA bit allowed/disallowed**: and behavior if a CSR contains a request a for a CA certificate when the signer does not permit it. -Commonly, the `status.certificate` field contains a single PEM-encoded X.509 certificate once the CSR is approved and the certificate is issued. Some signers store multiple certificates into the `status.certificate` field. In that case, the documentation for the signer should specify the meaning of additional certificates; for example, this might be the certificate plus intermediates to be presented during TLS handshakes. +Commonly, the `status.certificate` field contains a single PEM-encoded X.509 +certificate once the CSR is approved and the certificate is issued. Some +signers store multiple certificates into the `status.certificate` field. In +that case, the documentation for the signer should specify the meaning of +additional certificates; for example, this might be the certificate plus +intermediates to be presented during TLS handshakes. + +The PKCS#10 signing request format doesn't allow to specify a certificate +expiration or lifetime. The expiration or lifetime therefore has to be set +through e.g. an annotation on the CSR object. While it's theoretically +possible for a signer to use that expiration date, there is currently no +known implementation that does. (The built-in signers all use the same +`ClusterSigningDuration` configuration option, which defaults to 1 year, +and can be changed with the `--cluster-signing-duration` command-line +flag of the kube-controller-manager.) + ### Kubernetes signers Kubernetes provides built-in signers that each have a well-known `signerName`: -1. `kubernetes.io/kube-apiserver-client`: signs certificates that will be honored as client-certs by the kube-apiserver. - Never auto-approved by {{< glossary_tooltip term_id="kube-controller-manager" >}}. - 1. Trust distribution: signed certificates must be honored as client-certificates by the kube-apiserver. The CA bundle is not distributed by any other means. - 1. Permitted subjects - no subject restrictions, but approvers and signers may choose not to approve or sign. Certain subjects like cluster-admin level users or groups vary between distributions and installations, but deserve additional scrutiny before approval and signing. The `CertificateSubjectRestriction` admission plugin is available and enabled by default to restrict `system:masters`, but it is often not the only cluster-admin subject in a cluster. +1. `kubernetes.io/kube-apiserver-client`: signs certificates that will be honored as client certificates by the API server. + Never auto-approved by {{< glossary_tooltip term_id="kube-controller-manager" >}}. + 1. Trust distribution: signed certificates must be honored as client certificates by the API server. The CA bundle is not distributed by any other means. + 1. Permitted subjects - no subject restrictions, but approvers and signers may choose not to approve or sign. + Certain subjects like cluster-admin level users or groups vary between distributions and installations, + but deserve additional scrutiny before approval and signing. + The `CertificateSubjectRestriction` admission plugin is enabled by default to restrict `system:masters`, + but it is often not the only cluster-admin subject in a cluster. 1. Permitted x509 extensions - honors subjectAltName and key usage extensions and discards other extensions. - 1. Permitted key usages - must include []string{"client auth"}. Must not include key usages beyond []string{"digital signature", "key encipherment", "client auth"} - 1. Expiration/certificate lifetime - minimum of CSR signer or request. The signer is responsible for checking that the certificate lifetime is valid and permissible. + 1. Permitted key usages - must include `["client auth"]`. Must not include key usages beyond `["digital signature", "key encipherment", "client auth"]`. + 1. Expiration/certificate lifetime - set by the `--cluster-signing-duration` option for the + kube-controller-manager implementation of this signer. 1. CA bit allowed/disallowed - not allowed. -1. `kubernetes.io/kube-apiserver-client-kubelet`: signs client certificates that will be honored as client-certs by the - kube-apiserver. - May be auto-approved by {{< glossary_tooltip term_id="kube-controller-manager" >}}. - 1. Trust distribution: signed certificates must be honored as client-certificates by the kube-apiserver. The CA bundle +1. `kubernetes.io/kube-apiserver-client-kubelet`: signs client certificates that will be honored as client certificates by the + API server. + May be auto-approved by {{< glossary_tooltip term_id="kube-controller-manager" >}}. + 1. Trust distribution: signed certificates must be honored as client certificates by the API server. The CA bundle is not distributed by any other means. - 1. Permitted subjects - organizations are exactly `[]string{"system:nodes"}`, common name starts with `"system:node:"` + 1. Permitted subjects - organizations are exactly `["system:nodes"]`, common name starts with "`system:node:`". 1. Permitted x509 extensions - honors key usage extensions, forbids subjectAltName extensions and drops other extensions. - 1. Permitted key usages - exactly `[]string{"key encipherment", "digital signature", "client auth"}` - 1. Expiration/certificate lifetime - minimum of CSR signer or request. The signer is responsible for checking that the certificate lifetime is valid and permissible. + 1. Permitted key usages - exactly `["key encipherment", "digital signature", "client auth"]`. + 1. Expiration/certificate lifetime - set by the `--cluster-signing-duration` option for the + kube-controller-manager implementation of this signer. 1. CA bit allowed/disallowed - not allowed. 1. `kubernetes.io/kubelet-serving`: signs serving certificates that are honored as a valid kubelet serving certificate - by the kube-apiserver, but has no other guarantees. - Never auto-approved by {{< glossary_tooltip term_id="kube-controller-manager" >}}. - 1. Trust distribution: signed certificates must be honored by the kube-apiserver as valid to terminate connections to a kubelet. The CA bundle is not distributed by any other means. - 1. Permitted subjects - organizations are exactly `[]string{"system:nodes"}`, common name starts with `"system:node:"` - 1. Permitted x509 extensions - honors key usage and DNSName/IPAddress subjectAltName extensions, forbids EmailAddress and URI subjectAltName extensions, drops other extensions. At least one DNS or IP subjectAltName must be present. - 1. Permitted key usages - exactly `[]string{"key encipherment", "digital signature", "server auth"}` - 1. Expiration/certificate lifetime - minimum of CSR signer or request. + by the API server, but has no other guarantees. + Never auto-approved by {{< glossary_tooltip term_id="kube-controller-manager" >}}. + 1. Trust distribution: signed certificates must be honored by the API server as valid to terminate connections to a kubelet. + The CA bundle is not distributed by any other means. + 1. Permitted subjects - organizations are exactly `["system:nodes"]`, common name starts with "`system:node:`". + 1. Permitted x509 extensions - honors key usage and DNSName/IPAddress subjectAltName extensions, forbids EmailAddress and + URI subjectAltName extensions, drops other extensions. At least one DNS or IP subjectAltName must be present. + 1. Permitted key usages - exactly `["key encipherment", "digital signature", "server auth"]`. + 1. Expiration/certificate lifetime - set by the `--cluster-signing-duration` option for the + kube-controller-manager implementation of this signer. 1. CA bit allowed/disallowed - not allowed. 1. `kubernetes.io/legacy-unknown`: has no guarantees for trust at all. Some third-party distributions of Kubernetes - may honor client certificates signed by it. The stable CertificateSigningRequest API (version `certificates.k8s.io/v1` and later) - does not allow to set the `signerName` as `kubernetes.io/legacy-unknown`. - Never auto-approved by {{< glossary_tooltip term_id="kube-controller-manager" >}}. - 1. Trust distribution: None. There is no standard trust or distribution for this signer in a Kubernetes cluster. + may honor client certificates signed by it. The stable CertificateSigningRequest API (version `certificates.k8s.io/v1` and later) + does not allow to set the `signerName` as `kubernetes.io/legacy-unknown`. + Never auto-approved by {{< glossary_tooltip term_id="kube-controller-manager" >}}. + 1. Trust distribution: None. There is no standard trust or distribution for this signer in a Kubernetes cluster. 1. Permitted subjects - any 1. Permitted x509 extensions - honors subjectAltName and key usage extensions and discards other extensions. 1. Permitted key usages - any - 1. Expiration/certificate lifetime - minimum of CSR signer or request. The signer is responsible for checking that the certificate lifetime is valid and permissible. + 1. Expiration/certificate lifetime - set by the `--cluster-signing-duration` option for the + kube-controller-manager implementation of this signer. 1. CA bit allowed/disallowed - not allowed. {{< note >}} @@ -126,7 +151,7 @@ Distribution of trust happens out of band for these signers. Any trust outside coincidental. For instance, some distributions may honor `kubernetes.io/legacy-unknown` as client certificates for the kube-apiserver, but this is not a standard. None of these usages are related to ServiceAccount token secrets `.data[ca.crt]` in any way. That CA bundle is only -guaranteed to verify a connection to the kube-apiserver using the default service (`kubernetes.default.svc`). +guaranteed to verify a connection to the API server using the default service (`kubernetes.default.svc`). ## Authorization @@ -136,22 +161,7 @@ To allow creating a CertificateSigningRequest and retrieving any CertificateSign For example: -```yaml -apiVersion: rbac.authorization.k8s.io/v1 -kind: ClusterRole -metadata: - name: csr-creator -rules: -- apiGroups: - - certificates.k8s.io - resources: - - certificatesigningrequests - verbs: - - create - - get - - list - - watch -``` +{{< codenew file="access/certificate-signing-request/clusterrole-create.yaml" >}} To allow approving a CertificateSigningRequest: @@ -161,35 +171,7 @@ To allow approving a CertificateSigningRequest: For example: -```yaml -apiVersion: rbac.authorization.k8s.io/v1 -kind: ClusterRole -metadata: - name: csr-approver -rules: -- apiGroups: - - certificates.k8s.io - resources: - - certificatesigningrequests - verbs: - - get - - list - - watch -- apiGroups: - - certificates.k8s.io - resources: - - certificatesigningrequests/approval - verbs: - - update -- apiGroups: - - certificates.k8s.io - resources: - - signers - resourceNames: - - example.com/my-signer-name # example.com/* can be used to authorize for all signers in the 'example.com' domain - verbs: - - approve -``` +{{< codenew file="access/certificate-signing-request/clusterrole-approve.yaml" >}} To allow signing a CertificateSigningRequest: @@ -197,54 +179,32 @@ To allow signing a CertificateSigningRequest: * Verbs: `update`, group: `certificates.k8s.io`, resource: `certificatesigningrequests/status` * Verbs: `sign`, group: `certificates.k8s.io`, resource: `signers`, resourceName: `/` or `/*` -```yaml -apiVersion: rbac.authorization.k8s.io/v1 -kind: ClusterRole -metadata: - name: csr-signer -rules: -- apiGroups: - - certificates.k8s.io - resources: - - certificatesigningrequests - verbs: - - get - - list - - watch -- apiGroups: - - certificates.k8s.io - resources: - - certificatesigningrequests/status - verbs: - - update -- apiGroups: - - certificates.k8s.io - resources: - - signers - resourceName: - - example.com/my-signer-name # example.com/* can be used to authorize for all signers in the 'example.com' domain - verbs: - - sign -``` +{{< codenew file="access/certificate-signing-request/clusterrole-sign.yaml" >}} -## Normal User +## Normal user -A few steps are required in order to get normal user to be able to authenticate and invoke an API. First, this user must have certificate issued by the Kubernetes Cluster, and then present that Certificate to the API call as the Certificate Header or through the kubectl. +A few steps are required in order to get a normal user to be able to +authenticate and invoke an API. First, this user must have certificate issued +by the Kubernetes cluster, and then present that Certificate to the API call +as the Certificate Header or through the kubectl. -### Create Private Key +### Create private key -The following scripts show how to generate PKI private key and CSR. It is important to set CN and O attribute of the CSR. CN is the name of the user and O is the group that this user will belong to. You can refer to [RBAC](/docs/reference/access-authn-authz/rbac/) for standard groups. +The following scripts show how to generate PKI private key and CSR. It is +important to set CN and O attribute of the CSR. CN is the name of the user and +O is the group that this user will belong to. You can refer to +[RBAC](/docs/reference/access-authn-authz/rbac/) for standard groups. -``` +```shell openssl genrsa -out john.key 2048 openssl req -new -key john.key -out john.csr ``` -### Create Certificate Request Kubernetes Object +### Create CertificateSigningRequest Create a CertificateSigningRequest and submit it to a Kubernetes Cluster via kubectl. Below is a script to generate the CertificateSigningRequest. -``` +```shell cat < ``` Likewise, to deny a CSR: -```bash +```shell kubectl certificate deny ``` -### Approval & rejection using the Kubernetes API {#approval-rejection-api-client} +### Approval or rejection using the Kubernetes API {#approval-rejection-api-client} Users of the REST API can approve CSRs by submitting an UPDATE request to the `approval` subresource of the CSR to be approved. For example, you could write an @@ -400,7 +370,8 @@ you like. If you want to add a note just for human consumption, use the ### Control plane signer {#signer-control-plane} -The Kubernetes control plane implements each of the [Kubernetes signers](/docs/reference/access-authn-authz/certificate-signing-requests/#kubernetes-signers), +The Kubernetes control plane implements each of the +[Kubernetes signers](/docs/reference/access-authn-authz/certificate-signing-requests/#kubernetes-signers), as part of the kube-controller-manager. {{< note >}} @@ -460,13 +431,11 @@ status: certificate: "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JS..." ``` - - ## {{% heading "whatsnext" %}} - * Read [Manage TLS Certificates in a Cluster](/docs/tasks/tls/managing-tls-in-a-cluster/) * View the source code for the kube-controller-manager built in [signer](https://github.com/kubernetes/kubernetes/blob/32ec6c212ec9415f604ffc1f4c1f29b782968ff1/pkg/controller/certificates/signer/cfssl_signer.go) * View the source code for the kube-controller-manager built in [approver](https://github.com/kubernetes/kubernetes/blob/32ec6c212ec9415f604ffc1f4c1f29b782968ff1/pkg/controller/certificates/approver/sarapprove.go) * For details of X.509 itself, refer to [RFC 5280](https://tools.ietf.org/html/rfc5280#section-3.1) section 3.1 * For information on the syntax of PKCS#10 certificate signing requests, refer to [RFC 2986](https://tools.ietf.org/html/rfc2986) + diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates.md b/content/en/docs/reference/command-line-tools-reference/feature-gates.md index 2a79f727b4..bdd1850694 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates.md @@ -433,7 +433,7 @@ Each feature gate is designed for enabling/disabling a specific feature: - `DevicePlugins`: Enable the [device-plugins](/docs/concepts/cluster-administration/device-plugins/) based resource provisioning on nodes. - `DefaultPodTopologySpread`: Enables the use of `PodTopologySpread` scheduling plugin to do - [default spreading](/docs/concepts/workloads/pods/pod/pod-topology-spread-constraints/#internal-default-constraints). + [default spreading](/docs/concepts/workloads/pods/pod-topology-spread-constraints/#internal-default-constraints). - `DryRun`: Enable server-side [dry run](/docs/reference/using-api/api-concepts/#dry-run) requests so that validation, merging, and mutation can be tested without committing. - `DynamicAuditing`(*deprecated*): Used to enable dynamic auditing before v1.19. @@ -479,8 +479,8 @@ Each feature gate is designed for enabling/disabling a specific feature: - `KubeletPodResources`: Enable the kubelet's pod resources grpc endpoint. See [Support Device Monitoring](https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/compute-device-assignment.md) for more details. - `LegacyNodeRoleBehavior`: When disabled, legacy behavior in service load balancers and node disruption will ignore the `node-role.kubernetes.io/master` label in favor of the feature-specific labels provided by `NodeDisruptionExclusion` and `ServiceNodeExclusion`. -- `LocalStorageCapacityIsolation`: Enable the consumption of [local ephemeral storage](/docs/concepts/configuration/manage-compute-resources-container/) and also the `sizeLimit` property of an [emptyDir volume](/docs/concepts/storage/volumes/#emptydir). -- `LocalStorageCapacityIsolationFSQuotaMonitoring`: When `LocalStorageCapacityIsolation` is enabled for [local ephemeral storage](/docs/concepts/configuration/manage-compute-resources-container/) and the backing filesystem for [emptyDir volumes](/docs/concepts/storage/volumes/#emptydir) supports project quotas and they are enabled, use project quotas to monitor [emptyDir volume](/docs/concepts/storage/volumes/#emptydir) storage consumption rather than filesystem walk for better performance and accuracy. +- `LocalStorageCapacityIsolation`: Enable the consumption of [local ephemeral storage](/docs/concepts/configuration/manage-resources-containers/) and also the `sizeLimit` property of an [emptyDir volume](/docs/concepts/storage/volumes/#emptydir). +- `LocalStorageCapacityIsolationFSQuotaMonitoring`: When `LocalStorageCapacityIsolation` is enabled for [local ephemeral storage](/docs/concepts/configuration/manage-resources-containers/) and the backing filesystem for [emptyDir volumes](/docs/concepts/storage/volumes/#emptydir) supports project quotas and they are enabled, use project quotas to monitor [emptyDir volume](/docs/concepts/storage/volumes/#emptydir) storage consumption rather than filesystem walk for better performance and accuracy. - `MountContainers`: Enable using utility containers on host as the volume mounter. - `MountPropagation`: Enable sharing volume mounted by one container to other containers or pods. For more details, please see [mount propagation](/docs/concepts/storage/volumes/#mount-propagation). @@ -490,7 +490,7 @@ Each feature gate is designed for enabling/disabling a specific feature: - `PersistentLocalVolumes`: Enable the usage of `local` volume type in Pods. Pod affinity has to be specified if requesting a `local` volume. - `PodDisruptionBudget`: Enable the [PodDisruptionBudget](/docs/tasks/run-application/configure-pdb/) feature. -- `PodOverhead`: Enable the [PodOverhead](/docs/concepts/configuration/pod-overhead/) feature to account for pod overheads. +- `PodOverhead`: Enable the [PodOverhead](/docs/concepts/scheduling-eviction/pod-overhead/) feature to account for pod overheads. - `PodPriority`: Enable the descheduling and preemption of Pods based on their [priorities](/docs/concepts/configuration/pod-priority-preemption/). - `PodReadinessGates`: Enable the setting of `PodReadinessGate` field for extending Pod readiness evaluation. See [Pod readiness gate](/docs/concepts/workloads/pods/pod-lifecycle/#pod-readiness-gate) @@ -516,7 +516,7 @@ Each feature gate is designed for enabling/disabling a specific feature: - `RuntimeClass`: Enable the [RuntimeClass](/docs/concepts/containers/runtime-class/) feature for selecting container runtime configurations. - `ScheduleDaemonSetPods`: Enable DaemonSet Pods to be scheduled by the default scheduler instead of the DaemonSet controller. - `SCTPSupport`: Enables the _SCTP_ `protocol` value in Pod, Service, Endpoints, EndpointSlice, and NetworkPolicy definitions. -- `ServerSideApply`: Enables the [Sever Side Apply (SSA)](/docs/reference/using-api/api-concepts/#server-side-apply) path at the API Server. +- `ServerSideApply`: Enables the [Sever Side Apply (SSA)](/docs/reference/using-api/server-side-apply/) path at the API Server. - `ServiceAccountIssuerDiscovery`: Enable OIDC discovery endpoints (issuer and JWKS URLs) for the service account issuer in the API server. See [Configure Service Accounts for Pods](/docs/tasks/configure-pod-container/configure-service-account/#service-account-issuer-discovery) for more details. - `ServiceAppProtocol`: Enables the `AppProtocol` field on Services and Endpoints. - `ServiceLoadBalancerFinalizer`: Enable finalizer protection for Service load balancers. diff --git a/content/en/docs/reference/glossary/container.md b/content/en/docs/reference/glossary/container.md index 762a4fdcd5..cbf1f80fba 100755 --- a/content/en/docs/reference/glossary/container.md +++ b/content/en/docs/reference/glossary/container.md @@ -2,7 +2,7 @@ title: Container id: container date: 2018-04-12 -full_link: /docs/concepts/overview/what-is-kubernetes/#why-containers +full_link: /docs/concepts/containers/ short_description: > A lightweight and portable executable image that contains software and all of its dependencies. diff --git a/content/en/docs/reference/tools.md b/content/en/docs/reference/tools.md index ef210f2b07..ceeadbb279 100644 --- a/content/en/docs/reference/tools.md +++ b/content/en/docs/reference/tools.md @@ -20,7 +20,7 @@ Kubernetes contains several built-in tools to help you work with the Kubernetes ## Minikube -[`minikube`](/docs/tasks/tools/install-minikube/) is a tool that makes it +[`minikube`](https://minikube.sigs.k8s.io/docs/) is a tool that makes it easy to run a single-node Kubernetes cluster locally on your workstation for development and testing purposes. @@ -44,7 +44,7 @@ Use Helm to: ## Kompose -[`Kompose`](https://github.com/kubernetes-incubator/kompose) is a tool to help Docker Compose users move to Kubernetes. +[`Kompose`](https://github.com/kubernetes/kompose) is a tool to help Docker Compose users move to Kubernetes. Use Kompose to: diff --git a/content/en/docs/reference/using-api/_index.md b/content/en/docs/reference/using-api/_index.md index 9d6b7c4e36..8dc81fe549 100644 --- a/content/en/docs/reference/using-api/_index.md +++ b/content/en/docs/reference/using-api/_index.md @@ -1,4 +1,132 @@ --- -title: Using the Kubernetes API +title: Kubernetes API Overview +reviewers: +- erictune +- lavalamp +- jbeda +content_type: concept weight: 10 ---- \ No newline at end of file +card: + name: reference + weight: 50 + title: Overview of API +--- + + + +This section provides reference information for the Kubernetes API. + +The REST API is the fundamental fabric of Kubernetes. All operations and +communications between components, and external user commands are REST API +calls that the API Server handles. Consequently, everything in the Kubernetes +platform is treated as an API object and has a corresponding entry in the +[API](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/). + +The [Kubernetes API reference](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/) +lists the API for Kubernetes version {{< param "version" >}}. + +For general background information, read +[The Kubernetes API](/docs/concepts/overview/kubernetes-api/). +[Controlling Access to the Kubernetes API](/docs/concepts/security/controlling-access/) +describes how clients can authenticate to the Kubernetes API server, and how their +requests are authorized. + + + +The REST API is the fundamental fabric of Kubernetes. All operations and +communications between components, and external user commands are REST API +calls that the API Server handles. Consequently, everything in the Kubernetes +platform is treated as an API object and has a corresponding entry in the +API. + +## API versioning + +The JSON and Protobuf serialization schemas follow the same guidelines for +schema changes. The following descriptions cover both formats. + +The API versioning and software versioning are indirectly related. +The [API and release versioning proposal](https://git.k8s.io/community/contributors/design-proposals/release/versioning.md) +describes the relationship between API versioning and software versioning. + +Different API versions indicate different levels of stability and support. You +can find more information about the criteria for each level in the +[API Changes documentation](https://git.k8s.io/community/contributors/devel/sig-architecture/api_changes.md#alpha-beta-and-stable-versions). + +Here's a summary of each level: + +- Alpha: + - The version names contain `alpha` (for example, `v1alpha1`). + - The software may contain bugs. Enabling a feature may expose bugs. A + feature may be disabled by default. + - The support for a feature may be dropped at any time without notice. + - The API may change in incompatible ways in a later software release without notice. + - The software is recommended for use only in short-lived testing clusters, + due to increased risk of bugs and lack of long-term support. + +- Beta: + - The version names contain `beta` (for example, `v2beta3`). + - The software is well tested. Enabling a feature is considered safe. + Features are enabled by default. + - The support for a feature will not be dropped, though the details may change. + + - The schema and/or semantics of objects may change in incompatible ways in + a subsequent beta or stable release. When this happens, migration + instructions are provided. Schema changes may require deleting, editing, and + re-creating API objects. The editing process may not be straightforward. + The migration may require downtime for applications that rely on the feature. + - The software is not recommended for production uses. Subsequent releases + may introduce incompatible changes. If you have multiple clusters which + can be upgraded independently, you may be able to relax this restriction. + + {{< note >}} + Please try beta features and provide feedback. After the features exit beta, it + may not be practical to make more changes. + {{< /note >}} + +- Stable: + - The version name is `vX` where `X` is an integer. + - The stable versions of features appear in released software for many subsequent versions. + +## API groups + +[API groups](https://git.k8s.io/community/contributors/design-proposals/api-machinery/api-group.md) +make it easier to extend the Kubernetes API. +The API group is specified in a REST path and in the `apiVersion` field of a +serialized object. + +There are several API groups in Kubernetes: + +* The *core* (also called *legacy*) group is found at REST path `/api/v1`. + The core group is not specified as part of the `apiVersion` field, for + example, `apiVersion: v1`. +* The named groups are at REST path `/apis/$GROUP_NAME/$VERSION` and use + `apiVersion: $GROUP_NAME/$VERSION` (for example, `apiVersion: batch/v1`). + You can find the full list of supported API groups in + [Kubernetes API reference](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#-strong-api-groups-strong-). + +## Enabling or disabling API groups {#enabling-or-disabling} + +Certain resources and API groups are enabled by default. You can enable or +disable them by setting `--runtime-config` on the API server. The +`--runtime-config` flag accepts comma separated `[=]` pairs +describing the runtime configuration of the API server. If the `=` +part is omitted, it is treated as if `=true` is specified. For example: + + - to disable `batch/v1`, set `--runtime-config=batch/v1=false` + - to enable `batch/v2alpha1`, set `--runtime-config=batch/v2alpha1` + +{{< note >}} +When you enable or disable groups or resources, you need to restart the API +server and controller manager to pick up the `--runtime-config` changes. +{{< /note >}} + +## Persistence + +Kubernetes stores its serialized state in terms of the API resources by writing them into +{{< glossary_tooltip term_id="etcd" >}}. + +## {{% heading "whatsnext" %}} + +- Learn more about [API conventions](https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#api-conventions) +- Read the design documentation for + [aggregator](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/api-machinery/aggregated-api-servers.md) diff --git a/content/en/docs/reference/using-api/api-concepts.md b/content/en/docs/reference/using-api/api-concepts.md index c251e228ee..5f8f8337d1 100644 --- a/content/en/docs/reference/using-api/api-concepts.md +++ b/content/en/docs/reference/using-api/api-concepts.md @@ -431,444 +431,14 @@ Some values of an object are typically generated before the object is persisted. ## Server Side Apply -{{< feature-state for_k8s_version="v1.16" state="beta" >}} - -Starting from Kubernetes v1.18, if you have Server Side Apply enabled then the control plane tracks managed fields for all newly created objects. - -### Introduction - -Server Side Apply helps users and controllers manage their resources via -declarative configurations. It allows them to create and/or modify their -objects declaratively, simply by sending their fully specified intent. - -A fully specified intent is a partial object that only includes the fields and -values for which the user has an opinion. That intent either creates a new -object or is [combined](#merge-strategy), by the server, with the existing object. - -The system supports multiple appliers collaborating on a single object. - -Changes to an object's fields are tracked through a "[field management](#field-management)" -mechanism. When a field's value changes, ownership moves from its current -manager to the manager making the change. When trying to apply an object, fields -that have a different value and are owned by another manager will result in a -[conflict](#conflicts). This is done in order to signal that the operation might undo another -collaborator's changes. Conflicts can be forced, in which case the value will be -overridden, and the ownership will be transferred. - -If you remove a field from a configuration and apply the configuration, server side apply checks -if there are any other field managers that also own the field. If the field is -not owned by any other field managers, it is either deleted from the live -object or reset to its default value, if it has one. The same rule applies to associative list or -map items. - -Server side apply is meant both as a replacement for the original `kubectl -apply` and as a simpler mechanism for controllers to enact their changes. - -### Field Management - -Compared to the `last-applied` annotation managed by `kubectl`, Server Side -Apply uses a more declarative approach, which tracks a user's field management, -rather than a user's last applied state. This means that as a side effect of -using Server Side Apply, information about which field manager manages each -field in an object also becomes available. - -For a user to manage a field, in the Server Side Apply sense, means that the -user relies on and expects the value of the field not to change. The user who -last made an assertion about the value of a field will be recorded as the -current field manager. This can be done either by changing the value with -`POST`, `PUT`, or non-apply `PATCH`, or by including the field in a config sent -to the Server Side Apply endpoint. When using Server-Side Apply, trying to -change a field which is managed by someone else will result in a rejected -request (if not forced, see [Conflicts](#conflicts)). - -When two or more appliers set a field to the same value, they share ownership of -that field. Any subsequent attempt to change the value of the shared field, by any of -the appliers, results in a conflict. Shared field owners may give up ownership -of a field by removing it from their configuration. - -Field management is stored in a`managedFields` field that is part of an object's -[`metadata`](/docs/reference/generated/kubernetes-api/{{< latest-version >}}/#objectmeta-v1-meta). - -A simple example of an object created by Server Side Apply could look like this: - -```yaml -apiVersion: v1 -kind: ConfigMap -metadata: - name: test-cm - namespace: default - labels: - test-label: test - managedFields: - - manager: kubectl - operation: Apply - apiVersion: v1 - time: "2010-10-10T0:00:00Z" - fieldsType: FieldsV1 - fieldsV1: - f:metadata: - f:labels: - f:test-label: {} - f:data: - f:key: {} -data: - key: some value -``` - -The above object contains a single manager in `metadata.managedFields`. The -manager consists of basic information about the managing entity itself, like -operation type, API version, and the fields managed by it. - -{{< note >}} -This field is managed by the API server and should not be changed by -the user. -{{< /note >}} - -Nevertheless it is possible to change `metadata.managedFields` through an -`Update` operation. Doing so is highly discouraged, but might be a reasonable -option to try if, for example, the `managedFields` get into an inconsistent -state (which clearly should not happen). - -The format of the `managedFields` is described in the [API](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#fieldsv1-v1-meta). - -### Conflicts - -A conflict is a special status error that occurs when an `Apply` operation tries -to change a field, which another user also claims to manage. This prevents an -applier from unintentionally overwriting the value set by another user. When -this occurs, the applier has 3 options to resolve the conflicts: - -* **Overwrite value, become sole manager:** If overwriting the value was - intentional (or if the applier is an automated process like a controller) the - applier should set the `force` query parameter to true and make the request - again. This forces the operation to succeed, changes the value of the field, - and removes the field from all other managers' entries in managedFields. -* **Don't overwrite value, give up management claim:** If the applier doesn't - care about the value of the field anymore, they can remove it from their - config and make the request again. This leaves the value unchanged, and causes - the field to be removed from the applier's entry in managedFields. -* **Don't overwrite value, become shared manager:** If the applier still cares - about the value of the field, but doesn't want to overwrite it, they can - change the value of the field in their config to match the value of the object - on the server, and make the request again. This leaves the value unchanged, - and causes the field's management to be shared by the applier and all other - field managers that already claimed to manage it. - - -### Managers - -Managers identify distinct workflows that are modifying the object (especially -useful on conflicts!), and can be specified through the `fieldManager` query -parameter as part of a modifying request. It is required for the apply endpoint, -though kubectl will default it to `kubectl`. For other updates, its default is -computed from the user-agent. - -### Apply and Update - -The two operation types considered by this feature are `Apply` (`PATCH` with -content type `application/apply-patch+yaml`) and `Update` (all other operations -which modify the object). Both operations update the `managedFields`, but behave -a little differently. - -{{< note >}} -Whether you are submitting JSON data or YAML data, use `application/apply-patch+yaml` as the -`Content-Type` header value. - -All JSON documents are valid YAML. -{{< /note >}} - -For instance, only the apply operation fails on conflicts while update does -not. Also, apply operations are required to identify themselves by providing a -`fieldManager` query parameter, while the query parameter is optional for update -operations. Finally, when using the apply operation you cannot have `managedFields` in the object that is being applied. - -An example object with multiple managers could look like this: - -```yaml -apiVersion: v1 -kind: ConfigMap -metadata: - name: test-cm - namespace: default - labels: - test-label: test - managedFields: - - manager: kubectl - operation: Apply - apiVersion: v1 - fields: - f:metadata: - f:labels: - f:test-label: {} - - manager: kube-controller-manager - operation: Update - apiVersion: v1 - time: '2019-03-30T16:00:00.000Z' - fields: - f:data: - f:key: {} -data: - key: new value -``` - -In this example, a second operation was run as an `Update` by the manager called -`kube-controller-manager`. The update changed a value in the data field which -caused the field's management to change to the `kube-controller-manager`. - -If this update would have been an `Apply` operation, the operation -would have failed due to conflicting ownership. - -### Merge strategy - -The merging strategy, implemented with Server Side Apply, provides a generally -more stable object lifecycle. Server Side Apply tries to merge fields based on -the fact who manages them instead of overruling just based on values. This way -it is intended to make it easier and more stable for multiple actors updating -the same object by causing less unexpected interference. - -When a user sends a "fully-specified intent" object to the Server Side Apply -endpoint, the server merges it with the live object favoring the value in the -applied config if it is specified in both places. If the set of items present in -the applied config is not a superset of the items applied by the same user last -time, each missing item not managed by any other appliers is removed. For -more information about how an object's schema is used to make decisions when -merging, see -[sigs.k8s.io/structured-merge-diff](https://sigs.k8s.io/structured-merge-diff). - -A number of markers were added in Kubernetes 1.16 and 1.17, to allow API -developers to describe the merge strategy supported by lists, maps, and -structs. These markers can be applied to objects of the respective type, -in Go files or in the -[OpenAPI schema definition of the CRD](/docs/reference/generated/kubernetes-api/{{< param "version" >}}#jsonschemaprops-v1-apiextensions-k8s-io): - -| Golang marker | OpenAPI extension | Accepted values | Description | Introduced in | -|---|---|---|---|---| -| `//+listType` | `x-kubernetes-list-type` | `atomic`/`set`/`map` | Applicable to lists. `atomic` and `set` apply to lists with scalar elements only. `map` applies to lists of nested types only. If configured as `atomic`, the entire list is replaced during merge; a single manager manages the list as a whole at any one time. If `granular`, different managers can manage entries separately. | 1.16 | -| `//+listMapKey` | `x-kubernetes-list-map-keys` | Slice of map keys that uniquely identify entries for example `["port", "protocol"]` | Only applicable when `+listType=map`. A slice of strings whose values in combination must uniquely identify list entries. While there can be multiple keys, `listMapKey` is singular because keys need to be specified individually in the Go type. | 1.16 | -| `//+mapType` | `x-kubernetes-map-type` | `atomic`/`granular` | Applicable to maps. `atomic` means that the map can only be entirely replaced by a single manager. `granular` means that the map supports separate managers updating individual fields. | 1.17 | -| `//+structType` | `x-kubernetes-map-type` | `atomic`/`granular` | Applicable to structs; otherwise same usage and OpenAPI annotation as `//+mapType`.| 1.17 | - -### Custom Resources - -By default, Server Side Apply treats custom resources as unstructured data. All -keys are treated the same as struct fields, and all lists are considered atomic. - -If the Custom Resource Definition defines a -[schema](/docs/reference/generated/kubernetes-api/{{< param "version" >}}#jsonschemaprops-v1-apiextensions-k8s-io) -that contains annotations as defined in the previous "Merge Strategy" -section, these annotations will be used when merging objects of this -type. - -### Using Server-Side Apply in a controller - -As a developer of a controller, you can use server-side apply as a way to -simplify the update logic of your controller. The main differences with a -read-modify-write and/or patch are the following: - -* the applied object must contain all the fields that the controller cares about. -* there are no way to remove fields that haven't been applied by the controller - before (controller can still send a PATCH/UPDATE for these use-cases). -* the object doesn't have to be read beforehand, `resourceVersion` doesn't have - to be specified. - -It is strongly recommended for controllers to always "force" conflicts, since they -might not be able to resolve or act on these conflicts. - -### Transferring Ownership - -In addition to the concurrency controls provided by [conflict resolution](#conflicts), -Server Side Apply provides ways to perform coordinated -field ownership transfers from users to controllers. - -This is best explained by example. Let's look at how to safely transfer -ownership of the `replicas` field from a user to a controller while enabling -automatic horizontal scaling for a Deployment, using the HorizontalPodAutoscaler -resource and its accompanying controller. - -Say a user has defined deployment with `replicas` set to the desired value: - -{{< codenew file="application/ssa/nginx-deployment.yaml" >}} - -And the user has created the deployment using server side apply like so: - -```shell -kubectl apply -f https://k8s.io/examples/application/ssa/nginx-deployment.yaml --server-side -``` - -Then later, HPA is enabled for the deployment, e.g.: - -```shell -kubectl autoscale deployment nginx-deployment --cpu-percent=50 --min=1 --max=10 -``` - -Now, the user would like to remove `replicas` from their configuration, so they -don't accidentally fight with the HPA controller. However, there is a race: it -might take some time before HPA feels the need to adjust `replicas`, and if -the user removes `replicas` before the HPA writes to the field and becomes -its owner, then apiserver will set `replicas` to 1, its default value. This -is not what the user wants to happen, even temporarily. - -There are two solutions: - -- (easy) Leave `replicas` in the configuration; when HPA eventually writes to that - field, the system gives the user a conflict over it. At that point, it is safe - to remove from the configuration. - -- (more advanced) If, however, the user doesn't want to wait, for example - because they want to keep the cluster legible to coworkers, then they can take - the following steps to make it safe to remove `replicas` from their - configuration: - -First, the user defines a new configuration containing only the `replicas` field: - -{{< codenew file="application/ssa/nginx-deployment-replicas-only.yaml" >}} - -The user applies that configuration using the field manager name `handover-to-hpa`: - -```shell -kubectl apply -f https://k8s.io/examples/application/ssa/nginx-deployment-replicas-only.yaml --server-side --field-manager=handover-to-hpa --validate=false -``` - -If the apply results in a conflict with the HPA controller, then do nothing. The -conflict just indicates the controller has claimed the field earlier in the -process than it sometimes does. - -At this point the user may remove the `replicas` field from their configuration. - -{{< codenew file="application/ssa/nginx-deployment-no-replicas.yaml" >}} - -Note that whenever the HPA controller sets the `replicas` field to a new value, -the temporary field manager will no longer own any fields and will be -automatically deleted. No clean up is required. - -### Transferring Ownership Between Users - -Users can transfer ownership of a field between each other by setting the field -to the same value in both of their applied configs, causing them to share -ownership of the field. Once the users share ownership of the field, one of them -can remove the field from their applied configuration to give up ownership and -complete the transfer to the other user. - -### Comparison with Client Side Apply - -A consequence of the conflict detection and resolution implemented by Server -Side Apply is that an applier always has up to date field values in their local -state. If they don't, they get a conflict the next time they apply. Any of the -three options to resolve conflicts results in the applied configuration being an -up to date subset of the object on the server's fields. - -This is different from Client Side Apply, where outdated values which have been -overwritten by other users are left in an applier's local config. These values -only become accurate when the user updates that specific field, if ever, and an -applier has no way of knowing whether their next apply will overwrite other -users' changes. - -Another difference is that an applier using Client Side Apply is unable to -change the API version they are using, but Server Side Apply supports this use -case. - -### Upgrading from client-side apply to server-side apply - -Client-side apply users who manage a resource with `kubectl apply` can start -using server-side apply with the following flag. - -```shell -kubectl apply --server-side [--dry-run=server] -``` - -By default, field management of the object transfers from client-side apply -to kubectl server-side apply without encountering conflicts. - -{{< caution >}} -Keep the `last-applied-configuration` annotation up to date. -The annotation infers client-side apply's managed fields. -Any fields not managed by client-side apply raise conflicts. - -For example, if you used `kubectl scale` to update the replicas field after client-side apply, -then this field is not owned by client-side apply and creates conflicts on `kubectl apply --server-side`. -{{< /caution >}} - -This behavior applies to server-side apply with the `kubectl` field manager. As -an exception, you can opt-out of this behavior by specifying a different, -non-default field manager, as seen in the following example. The default field manager for kubectl -server-side apply is `kubectl`. - -```shell -kubectl apply --server-side --field-manager=my-manager [--dry-run=server] -``` - -### Downgrading from server-side apply to client-side apply - -If you manage a resource with `kubectl apply --server-side`, -you can downgrade to client-side apply directly with `kubectl apply`. - -Downgrading works because kubectl server-side apply keeps the -`last-applied-configuration` annotation up-to-date if you use -`kubectl apply`. - -This behavior applies to server-side apply with the `kubectl` field manager. As -an exception, you can opt-out of this behavior by specifying a different, -non-default field manager, as seen in the following example. The default field manager for kubectl -server-side apply is `kubectl`. - -```shell -kubectl apply --server-side --field-manager=my-manager [--dry-run=server] -``` - -### API Endpoint - -With the Server Side Apply feature enabled, the `PATCH` endpoint accepts the -additional `application/apply-patch+yaml` content type. Users of Server Side -Apply can send partially specified objects as YAML to this endpoint. -When applying a configuration, one should always include all the fields -that they have an opinion about. - -### Clearing ManagedFields - -It is possible to strip all managedFields from an object by overwriting them -using `MergePatch`, `StrategicMergePatch`, `JSONPatch` or `Update`, so every -non-apply operation. This can be done by overwriting the managedFields field -with an empty entry. Two examples are: - -```console -PATCH /api/v1/namespaces/default/configmaps/example-cm -Content-Type: application/merge-patch+json -Accept: application/json -Data: {"metadata":{"managedFields": [{}]}} -``` - -```console -PATCH /api/v1/namespaces/default/configmaps/example-cm -Content-Type: application/json-patch+json -Accept: application/json -Data: [{"op": "replace", "path": "/metadata/managedFields", "value": [{}]}] -``` - -This will overwrite the managedFields with a list containing a single empty -entry that then results in the managedFields being stripped entirely from the -object. Note that just setting the managedFields to an empty list will not reset -the field. This is on purpose, so managedFields never get stripped by clients -not aware of the field. - -In cases where the reset operation is combined with changes to other fields than -the managedFields, this will result in the managedFields being reset first and -the other changes being processed afterwards. As a result the applier takes -ownership of any fields updated in the same request. - -{{< caution >}} -Server Side Apply does not correctly track ownership on -sub-resources that don't receive the resource object type. If you are -using Server Side Apply with such a sub-resource, the changed fields -won't be tracked. -{{< /caution >}} - -### Disabling the feature - -Server Side Apply is a beta feature, so it is enabled by default. To turn this -[feature gate](/docs/reference/command-line-tools-reference/feature-gates) off, -you need to include the `--feature-gates ServerSideApply=false` flag when -starting `kube-apiserver`. If you have multiple `kube-apiserver` replicas, all -should have the same flag setting. +Starting from Kubernetes v1.18, you can enable the +[Server Side Apply](/docs/reference/using-api/server-side-apply/) +feature so that the control plane tracks managed fields for all newly created objects. +Server Side Apply provides a clear pattern for managing field conflicts, +offers server-side `Apply` and `Update` operations, and replaces the +client-side functionality of `kubectl apply`. For more details about this +feature, see the section on +[Server Side Apply](/docs/reference/using-api/server-side-apply/). ## Resource Versions diff --git a/content/en/docs/reference/using-api/api-overview.md b/content/en/docs/reference/using-api/api-overview.md deleted file mode 100644 index 3b39d72e18..0000000000 --- a/content/en/docs/reference/using-api/api-overview.md +++ /dev/null @@ -1,117 +0,0 @@ ---- -title: Kubernetes API Overview -reviewers: -- erictune -- lavalamp -- jbeda -content_type: concept -weight: 10 -card: - name: reference - weight: 50 - title: Overview of API ---- - - - -This page provides an overview of the Kubernetes API. - - - -The REST API is the fundamental fabric of Kubernetes. All operations and -communications between components, and external user commands are REST API -calls that the API Server handles. Consequently, everything in the Kubernetes -platform is treated as an API object and has a corresponding entry in the -[API](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/). - -## API versioning - -The JSON and Protobuf serialization schemas follow the same guidelines for -schema changes. The following descriptions cover both formats. - -The API versioning and software versioning are indirectly related. -The [API and release versioning proposal](https://git.k8s.io/community/contributors/design-proposals/release/versioning.md) -describes the relationship between API versioning and software versioning. - -Different API versions indicate different levels of stability and support. You -can find more information about the criteria for each level in the -[API Changes documentation](https://git.k8s.io/community/contributors/devel/sig-architecture/api_changes.md#alpha-beta-and-stable-versions). - -Here's a summary of each level: - -- Alpha: - - The version names contain `alpha` (for example, `v1alpha1`). - - The software may contain bugs. Enabling a feature may expose bugs. A - feature may be disabled by default. - - The support for a feature may be dropped at any time without notice. - - The API may change in incompatible ways in a later software release without notice. - - The software is recommended for use only in short-lived testing clusters, - due to increased risk of bugs and lack of long-term support. - -- Beta: - - The version names contain `beta` (for example, `v2beta3`). - - The software is well tested. Enabling a feature is considered safe. - Features are enabled by default. - - The support for a feature will not be dropped, though the details may change. - - - The schema and/or semantics of objects may change in incompatible ways in - a subsequent beta or stable release. When this happens, migration - instructions are provided. Schema changes may require deleting, editing, and - re-creating API objects. The editing process may not be straightforward. - The migration may require downtime for applications that rely on the feature. - - The software is not recommended for production uses. Subsequent releases - may introduce incompatible changes. If you have multiple clusters which - can be upgraded independently, you may be able to relax this restriction. - - {{< note >}} - Try the beta features and provide feedback. After the features exit beta, it - may not be practical to make more changes. - {{< /note >}} - -- Stable: - - The version name is `vX` where `X` is an integer. - - The stable versions of features appear in released software for many subsequent versions. - -## API groups - -[API groups](https://git.k8s.io/community/contributors/design-proposals/api-machinery/api-group.md) -make it easier to extend the Kubernetes API. -The API group is specified in a REST path and in the `apiVersion` field of a -serialized object. - -Currently, there are several API groups in use: - -* The *core* (also called *legacy*) group is found at REST path `/api/v1`. - The core group is not specified as part of the `apiVersion` field, for - example, `apiVersion: v1`. -* The named groups are at REST path `/apis/$GROUP_NAME/$VERSION` and use - `apiVersion: $GROUP_NAME/$VERSION` (for example, `apiVersion: batch/v1`). - You can find the full list of supported API groups in - [Kubernetes API reference](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/). - -## Enabling or disabling API groups {#enabling-or-disabling} - -Certain resources and API groups are enabled by default. You can enable or -disable them by setting `--runtime-config` on the API server. The -`--runtime-config` flag accepts comma separated `=` pairs -describing the runtime configuration of the API server. For example: - - - to disable `batch/v1`, set `--runtime-config=batch/v1=false` - - to enable `batch/v2alpha1`, set `--runtime-config=batch/v2alpha1` - -{{< note >}} -When you enable or disable groups or resources, you need to restart the API -server and controller manager to pick up the `--runtime-config` changes. -{{< /note >}} - -## Persistence - -Kubernetes stores its serialized state in terms of the API resources by writing them into -{{< glossary_tooltip term_id="etcd" >}}. - -## {{% heading "whatsnext" %}} - -- Learn more about [API conventions](https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#api-conventions) -- Read the design documentation for - [aggregator](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/api-machinery/aggregated-api-servers.md) - diff --git a/content/en/docs/reference/using-api/client-libraries.md b/content/en/docs/reference/using-api/client-libraries.md index 3939a59a76..c14f0fa9c7 100644 --- a/content/en/docs/reference/using-api/client-libraries.md +++ b/content/en/docs/reference/using-api/client-libraries.md @@ -12,7 +12,7 @@ API from various programming languages. -To write applications using the [Kubernetes REST API](/docs/reference/using-api/api-overview/), +To write applications using the [Kubernetes REST API](/docs/reference/using-api/), you do not need to implement the API calls and request/response types yourself. You can use a client library for the programming language you are using. diff --git a/content/en/docs/reference/using-api/deprecation-policy.md b/content/en/docs/reference/using-api/deprecation-policy.md index d93bbc03c9..17840f195b 100644 --- a/content/en/docs/reference/using-api/deprecation-policy.md +++ b/content/en/docs/reference/using-api/deprecation-policy.md @@ -24,7 +24,7 @@ a deprecation policy for aspects of the system that are slated to be removed. Since Kubernetes is an API-driven system, the API has evolved over time to reflect the evolving understanding of the problem space. The Kubernetes API is actually a set of APIs, called "API groups", and each API group is -independently versioned. [API versions](/docs/reference/using-api/api-overview/#api-versioning) fall +independently versioned. [API versions](/docs/reference/using-api/#api-versioning) fall into 3 main tracks, each of which has different policies for deprecation: | Example | Track | diff --git a/content/en/docs/reference/using-api/server-side-apply.md b/content/en/docs/reference/using-api/server-side-apply.md new file mode 100644 index 0000000000..5f37b25682 --- /dev/null +++ b/content/en/docs/reference/using-api/server-side-apply.md @@ -0,0 +1,462 @@ +--- +title: Server-Side Apply +reviewers: +- smarterclayton +- apelisse +- lavalamp +- liggitt +content_type: concept +weight: 25 +min-kubernetes-server-version: 1.16 +--- + + + +{{< feature-state for_k8s_version="v1.16" state="beta" >}} + +## Introduction + +Server Side Apply helps users and controllers manage their resources via +declarative configurations. It allows them to create and/or modify their +[objects](/docs/concepts/overview/working-with-objects/kubernetes-objects/) +declaratively, simply by sending their fully specified intent. + +A fully specified intent is a partial object that only includes the fields and +values for which the user has an opinion. That intent either creates a new +object or is [combined](#merge-strategy), by the server, with the existing object. + +The system supports multiple appliers collaborating on a single object. + +Changes to an object's fields are tracked through a "[field management](#field-management)" +mechanism. When a field's value changes, ownership moves from its current +manager to the manager making the change. When trying to apply an object, +fields that have a different value and are owned by another manager will +result in a [conflict](#conflicts). This is done in order to signal that the +operation might undo another collaborator's changes. Conflicts can be forced, +in which case the value will be overridden, and the ownership will be +transferred. + +If you remove a field from a configuration and apply the configuration, server +side apply checks if there are any other field managers that also own the +field. If the field is not owned by any other field managers, it is either +deleted from the live object or reset to its default value, if it has one. The +same rule applies to associative list or map items. + +Server side apply is meant both as a replacement for the original `kubectl +apply` and as a simpler mechanism for controllers to enact their changes. + +If you have Server Side Apply enabled, the control plane tracks managed fields +for all newlly created objects. + +## Field Management + +Compared to the `last-applied` annotation managed by `kubectl`, Server Side +Apply uses a more declarative approach, which tracks a user's field management, +rather than a user's last applied state. This means that as a side effect of +using Server Side Apply, information about which field manager manages each +field in an object also becomes available. + +For a user to manage a field, in the Server Side Apply sense, means that the +user relies on and expects the value of the field not to change. The user who +last made an assertion about the value of a field will be recorded as the +current field manager. This can be done either by changing the value with +`POST`, `PUT`, or non-apply `PATCH`, or by including the field in a config sent +to the Server Side Apply endpoint. When using Server-Side Apply, trying to +change a field which is managed by someone else will result in a rejected +request (if not forced, see [Conflicts](#conflicts)). + +When two or more appliers set a field to the same value, they share ownership of +that field. Any subsequent attempt to change the value of the shared field, by any of +the appliers, results in a conflict. Shared field owners may give up ownership +of a field by removing it from their configuration. + +Field management is stored in a`managedFields` field that is part of an object's +[`metadata`](/docs/reference/generated/kubernetes-api/{{< latest-version >}}/#objectmeta-v1-meta). + +A simple example of an object created by Server Side Apply could look like this: + +```yaml +apiVersion: v1 +kind: ConfigMap +metadata: + name: test-cm + namespace: default + labels: + test-label: test + managedFields: + - manager: kubectl + operation: Apply + apiVersion: v1 + time: "2010-10-10T0:00:00Z" + fieldsType: FieldsV1 + fieldsV1: + f:metadata: + f:labels: + f:test-label: {} + f:data: + f:key: {} +data: + key: some value +``` + +The above object contains a single manager in `metadata.managedFields`. The +manager consists of basic information about the managing entity itself, like +operation type, API version, and the fields managed by it. + +{{< note >}} +This field is managed by the API server and should not be changed by +the user. +{{< /note >}} + +Nevertheless it is possible to change `metadata.managedFields` through an +`Update` operation. Doing so is highly discouraged, but might be a reasonable +option to try if, for example, the `managedFields` get into an inconsistent +state (which clearly should not happen). + +The format of the `managedFields` is described in the +[API](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#fieldsv1-v1-meta). + +## Conflicts + +A conflict is a special status error that occurs when an `Apply` operation tries +to change a field, which another user also claims to manage. This prevents an +applier from unintentionally overwriting the value set by another user. When +this occurs, the applier has 3 options to resolve the conflicts: + +* **Overwrite value, become sole manager:** If overwriting the value was + intentional (or if the applier is an automated process like a controller) the + applier should set the `force` query parameter to true and make the request + again. This forces the operation to succeed, changes the value of the field, + and removes the field from all other managers' entries in managedFields. + +* **Don't overwrite value, give up management claim:** If the applier doesn't + care about the value of the field anymore, they can remove it from their + config and make the request again. This leaves the value unchanged, and causes + the field to be removed from the applier's entry in managedFields. + +* **Don't overwrite value, become shared manager:** If the applier still cares + about the value of the field, but doesn't want to overwrite it, they can + change the value of the field in their config to match the value of the object + on the server, and make the request again. This leaves the value unchanged, + and causes the field's management to be shared by the applier and all other + field managers that already claimed to manage it. + +## Managers + +Managers identify distinct workflows that are modifying the object (especially +useful on conflicts!), and can be specified through the `fieldManager` query +parameter as part of a modifying request. It is required for the apply endpoint, +though kubectl will default it to `kubectl`. For other updates, its default is +computed from the user-agent. + +## Apply and Update + +The two operation types considered by this feature are `Apply` (`PATCH` with +content type `application/apply-patch+yaml`) and `Update` (all other operations +which modify the object). Both operations update the `managedFields`, but behave +a little differently. + +{{< note >}} +Whether you are submitting JSON data or YAML data, use +`application/apply-patch+yaml` as the `Content-Type` header value. + +All JSON documents are valid YAML. +{{< /note >}} + +For instance, only the apply operation fails on conflicts while update does +not. Also, apply operations are required to identify themselves by providing a +`fieldManager` query parameter, while the query parameter is optional for update +operations. Finally, when using the apply operation you cannot have +`managedFields` in the object that is being applied. + +An example object with multiple managers could look like this: + +```yaml +apiVersion: v1 +kind: ConfigMap +metadata: + name: test-cm + namespace: default + labels: + test-label: test + managedFields: + - manager: kubectl + operation: Apply + apiVersion: v1 + fields: + f:metadata: + f:labels: + f:test-label: {} + - manager: kube-controller-manager + operation: Update + apiVersion: v1 + time: '2019-03-30T16:00:00.000Z' + fields: + f:data: + f:key: {} +data: + key: new value +``` + +In this example, a second operation was run as an `Update` by the manager called +`kube-controller-manager`. The update changed a value in the data field which +caused the field's management to change to the `kube-controller-manager`. + +If this update would have been an `Apply` operation, the operation +would have failed due to conflicting ownership. + +## Merge strategy + +The merging strategy, implemented with Server Side Apply, provides a generally +more stable object lifecycle. Server Side Apply tries to merge fields based on +the fact who manages them instead of overruling just based on values. This way +it is intended to make it easier and more stable for multiple actors updating +the same object by causing less unexpected interference. + +When a user sends a "fully-specified intent" object to the Server Side Apply +endpoint, the server merges it with the live object favoring the value in the +applied config if it is specified in both places. If the set of items present in +the applied config is not a superset of the items applied by the same user last +time, each missing item not managed by any other appliers is removed. For +more information about how an object's schema is used to make decisions when +merging, see +[sigs.k8s.io/structured-merge-diff](https://sigs.k8s.io/structured-merge-diff). + +A number of markers were added in Kubernetes 1.16 and 1.17, to allow API +developers to describe the merge strategy supported by lists, maps, and +structs. These markers can be applied to objects of the respective type, +in Go files or in the OpenAPI schema definition of the +[CRD](/docs/reference/generated/kubernetes-api/{{< param "version" >}}#jsonschemaprops-v1-apiextensions-k8s-io): + +| Golang marker | OpenAPI extension | Accepted values | Description | Introduced in | +|---|---|---|---|---| +| `//+listType` | `x-kubernetes-list-type` | `atomic`/`set`/`map` | Applicable to lists. `atomic` and `set` apply to lists with scalar elements only. `map` applies to lists of nested types only. If configured as `atomic`, the entire list is replaced during merge; a single manager manages the list as a whole at any one time. If `granular`, different managers can manage entries separately. | 1.16 | +| `//+listMapKey` | `x-kubernetes-list-map-keys` | Slice of map keys that uniquely identify entries for example `["port", "protocol"]` | Only applicable when `+listType=map`. A slice of strings whose values in combination must uniquely identify list entries. While there can be multiple keys, `listMapKey` is singular because keys need to be specified individually in the Go type. | 1.16 | +| `//+mapType` | `x-kubernetes-map-type` | `atomic`/`granular` | Applicable to maps. `atomic` means that the map can only be entirely replaced by a single manager. `granular` means that the map supports separate managers updating individual fields. | 1.17 | +| `//+structType` | `x-kubernetes-map-type` | `atomic`/`granular` | Applicable to structs; otherwise same usage and OpenAPI annotation as `//+mapType`.| 1.17 | + +### Custom Resources + +By default, Server Side Apply treats custom resources as unstructured data. All +keys are treated the same as struct fields, and all lists are considered atomic. + +If the Custom Resource Definition defines a +[schema](/docs/reference/generated/kubernetes-api/{{< param "version" >}}#jsonschemaprops-v1-apiextensions-k8s-io) +that contains annotations as defined in the previous "Merge Strategy" +section, these annotations will be used when merging objects of this +type. + +### Using Server-Side Apply in a controller + +As a developer of a controller, you can use server-side apply as a way to +simplify the update logic of your controller. The main differences with a +read-modify-write and/or patch are the following: + +* the applied object must contain all the fields that the controller cares about. +* there are no way to remove fields that haven't been applied by the controller + before (controller can still send a PATCH/UPDATE for these use-cases). +* the object doesn't have to be read beforehand, `resourceVersion` doesn't have + to be specified. + +It is strongly recommended for controllers to always "force" conflicts, since they +might not be able to resolve or act on these conflicts. + +### Transferring Ownership + +In addition to the concurrency controls provided by [conflict resolution](#conflicts), +Server Side Apply provides ways to perform coordinated +field ownership transfers from users to controllers. + +This is best explained by example. Let's look at how to safely transfer +ownership of the `replicas` field from a user to a controller while enabling +automatic horizontal scaling for a Deployment, using the HorizontalPodAutoscaler +resource and its accompanying controller. + +Say a user has defined deployment with `replicas` set to the desired value: + +{{< codenew file="application/ssa/nginx-deployment.yaml" >}} + +And the user has created the deployment using server side apply like so: + +```shell +kubectl apply -f https://k8s.io/examples/application/ssa/nginx-deployment.yaml --server-side +``` + +Then later, HPA is enabled for the deployment, e.g.: + +```shell +kubectl autoscale deployment nginx-deployment --cpu-percent=50 --min=1 --max=10 +``` + +Now, the user would like to remove `replicas` from their configuration, so they +don't accidentally fight with the HPA controller. However, there is a race: it +might take some time before HPA feels the need to adjust `replicas`, and if +the user removes `replicas` before the HPA writes to the field and becomes +its owner, then apiserver will set `replicas` to 1, its default value. This +is not what the user wants to happen, even temporarily. + +There are two solutions: + +- (easy) Leave `replicas` in the configuration; when HPA eventually writes to that + field, the system gives the user a conflict over it. At that point, it is safe + to remove from the configuration. + +- (more advanced) If, however, the user doesn't want to wait, for example + because they want to keep the cluster legible to coworkers, then they can take + the following steps to make it safe to remove `replicas` from their + configuration: + +First, the user defines a new configuration containing only the `replicas` field: + +{{< codenew file="application/ssa/nginx-deployment-replicas-only.yaml" >}} + +The user applies that configuration using the field manager name `handover-to-hpa`: + +```shell +kubectl apply -f https://k8s.io/examples/application/ssa/nginx-deployment-replicas-only.yaml \ + --server-side --field-manager=handover-to-hpa \ + --validate=false +``` + +If the apply results in a conflict with the HPA controller, then do nothing. The +conflict just indicates the controller has claimed the field earlier in the +process than it sometimes does. + +At this point the user may remove the `replicas` field from their configuration. + +{{< codenew file="application/ssa/nginx-deployment-no-replicas.yaml" >}} + +Note that whenever the HPA controller sets the `replicas` field to a new value, +the temporary field manager will no longer own any fields and will be +automatically deleted. No clean up is required. + +## Transferring Ownership Between Users + +Users can transfer ownership of a field between each other by setting the field +to the same value in both of their applied configs, causing them to share +ownership of the field. Once the users share ownership of the field, one of them +can remove the field from their applied configuration to give up ownership and +complete the transfer to the other user. + +## Comparison with Client Side Apply + +A consequence of the conflict detection and resolution implemented by Server +Side Apply is that an applier always has up to date field values in their local +state. If they don't, they get a conflict the next time they apply. Any of the +three options to resolve conflicts results in the applied configuration being an +up to date subset of the object on the server's fields. + +This is different from Client Side Apply, where outdated values which have been +overwritten by other users are left in an applier's local config. These values +only become accurate when the user updates that specific field, if ever, and an +applier has no way of knowing whether their next apply will overwrite other +users' changes. + +Another difference is that an applier using Client Side Apply is unable to +change the API version they are using, but Server Side Apply supports this use +case. + +## Upgrading from client-side apply to server-side apply + +Client-side apply users who manage a resource with `kubectl apply` can start +using server-side apply with the following flag. + +```shell +kubectl apply --server-side [--dry-run=server] +``` + +By default, field management of the object transfers from client-side apply to +kubectl server-side apply without encountering conflicts. + +{{< caution >}} +Keep the `last-applied-configuration` annotation up to date. +The annotation infers client-side apply's managed fields. +Any fields not managed by client-side apply raise conflicts. + +For example, if you used `kubectl scale` to update the replicas field after +client-side apply, then this field is not owned by client-side apply and +creates conflicts on `kubectl apply --server-side`. +{{< /caution >}} + +This behavior applies to server-side apply with the `kubectl` field manager. +As an exception, you can opt-out of this behavior by specifying a different, +non-default field manager, as seen in the following example. The default field +manager for kubectl server-side apply is `kubectl`. + +```shell +kubectl apply --server-side --field-manager=my-manager [--dry-run=server] +``` + +## Downgrading from server-side apply to client-side apply + +If you manage a resource with `kubectl apply --server-side`, +you can downgrade to client-side apply directly with `kubectl apply`. + +Downgrading works because kubectl server-side apply keeps the +`last-applied-configuration` annotation up-to-date if you use +`kubectl apply`. + +This behavior applies to server-side apply with the `kubectl` field manager. +As an exception, you can opt-out of this behavior by specifying a different, +non-default field manager, as seen in the following example. The default field +manager for kubectl server-side apply is `kubectl`. + +```shell +kubectl apply --server-side --field-manager=my-manager [--dry-run=server] +``` + +## API Endpoint + +With the Server Side Apply feature enabled, the `PATCH` endpoint accepts the +additional `application/apply-patch+yaml` content type. Users of Server Side +Apply can send partially specified objects as YAML to this endpoint. When +applying a configuration, one should always include all the fields that they +have an opinion about. + +## Clearing ManagedFields + +It is possible to strip all managedFields from an object by overwriting them +using `MergePatch`, `StrategicMergePatch`, `JSONPatch` or `Update`, so every +non-apply operation. This can be done by overwriting the managedFields field +with an empty entry. Two examples are: + +```console +PATCH /api/v1/namespaces/default/configmaps/example-cm +Content-Type: application/merge-patch+json +Accept: application/json +Data: {"metadata":{"managedFields": [{}]}} +``` + +```console +PATCH /api/v1/namespaces/default/configmaps/example-cm +Content-Type: application/json-patch+json +Accept: application/json +Data: [{"op": "replace", "path": "/metadata/managedFields", "value": [{}]}] +``` + +This will overwrite the managedFields with a list containing a single empty +entry that then results in the managedFields being stripped entirely from the +object. Note that just setting the managedFields to an empty list will not +reset the field. This is on purpose, so managedFields never get stripped by +clients not aware of the field. + +In cases where the reset operation is combined with changes to other fields +than the managedFields, this will result in the managedFields being reset +first and the other changes being processed afterwards. As a result the +applier takes ownership of any fields updated in the same request. + +{{< caution >}} +Server Side Apply does not correctly track ownership on +sub-resources that don't receive the resource object type. If you are +using Server Side Apply with such a sub-resource, the changed fields +won't be tracked. +{{< /caution >}} + +## Disabling the feature + +Server Side Apply is a beta feature, so it is enabled by default. To turn this +[feature gate](/docs/reference/command-line-tools-reference/feature-gates) off, +you need to include the `--feature-gates ServerSideApply=false` flag when +starting `kube-apiserver`. If you have multiple `kube-apiserver` replicas, all +should have the same flag setting. + diff --git a/content/en/docs/setup/learning-environment/_index.md b/content/en/docs/setup/learning-environment/_index.md index f17c7e2f9d..672bbd69ed 100644 --- a/content/en/docs/setup/learning-environment/_index.md +++ b/content/en/docs/setup/learning-environment/_index.md @@ -2,3 +2,32 @@ title: Learning environment weight: 20 --- + + + +## kind + +[`kind`](https://kind.sigs.k8s.io/docs/) lets you run Kubernetes on +your local computer. This tool requires that you have +[Docker](https://docs.docker.com/get-docker/) installed and configured. + +The kind [Quick Start](https://kind.sigs.k8s.io/docs/user/quick-start/) page +shows you what you need to do to get up and running with kind. + +## minikube + +Like `kind`, [`minikube`](https://minikube.sigs.k8s.io/) is a tool that lets you run Kubernetes +locally. `minikube` runs a single-node Kubernetes cluster on your personal +computer (including Windows, macOS and Linux PCs) so that you can try out +Kubernetes, or for daily development work. + +You can follow the official +[Get Started!](https://minikube.sigs.k8s.io/docs/start/) guide if your focus is +on getting the tool installed. diff --git a/content/en/docs/setup/learning-environment/kind.md b/content/en/docs/setup/learning-environment/kind.md deleted file mode 100644 index ac355bd157..0000000000 --- a/content/en/docs/setup/learning-environment/kind.md +++ /dev/null @@ -1,23 +0,0 @@ ---- -title: Installing Kubernetes with Kind -weight: 40 -content_type: concept ---- - - - -Kind is a tool for running local Kubernetes clusters using Docker container "nodes". - - - - - -## Installation - -See [Installing Kind](https://kind.sigs.k8s.io/docs/user/quick-start/). - - - - - - diff --git a/content/en/docs/setup/learning-environment/minikube.md b/content/en/docs/setup/learning-environment/minikube.md deleted file mode 100644 index 0e3e26ea2a..0000000000 --- a/content/en/docs/setup/learning-environment/minikube.md +++ /dev/null @@ -1,519 +0,0 @@ ---- -reviewers: -- dlorenc -- balopat -- aaron-prindle -title: Installing Kubernetes with Minikube -weight: 30 -content_type: concept ---- - - - -Minikube is a tool that makes it easy to run Kubernetes locally. Minikube runs a single-node Kubernetes cluster inside a Virtual Machine (VM) on your laptop for users looking to try out Kubernetes or develop with it day-to-day. - - - - - -## Minikube Features - -Minikube supports the following Kubernetes features: - -* DNS -* NodePorts -* ConfigMaps and Secrets -* Dashboards -* Container Runtime: [Docker](https://www.docker.com/), [CRI-O](https://cri-o.io/), and [containerd](https://github.com/containerd/containerd) -* Enabling CNI (Container Network Interface) -* Ingress - -## Installation - -See [Installing Minikube](/docs/tasks/tools/install-minikube/). - -## Quickstart - -This brief demo guides you on how to start, use, and delete Minikube locally. Follow the steps given below to start and explore Minikube. - -1. Start Minikube and create a cluster: - - ```shell - minikube start - ``` - - The output is similar to this: - - ``` - Starting local Kubernetes cluster... - Running pre-create checks... - Creating machine... - Starting local Kubernetes cluster... - ``` - - For more information on starting your cluster on a specific Kubernetes version, VM, or container runtime, see [Starting a Cluster](#starting-a-cluster). - -2. Now, you can interact with your cluster using kubectl. For more information, see [Interacting with Your Cluster](#interacting-with-your-cluster). - - Let's create a Kubernetes Deployment using an existing image named `echoserver`, which is a simple HTTP server and expose it on port 8080 using `--port`. - - ```shell - kubectl create deployment hello-minikube --image=k8s.gcr.io/echoserver:1.10 - ``` - - The output is similar to this: - - ``` - deployment.apps/hello-minikube created - ``` -3. To access the `hello-minikube` Deployment, expose it as a Service: - - ```shell - kubectl expose deployment hello-minikube --type=NodePort --port=8080 - ``` - - The option `--type=NodePort` specifies the type of the Service. - - The output is similar to this: - - ``` - service/hello-minikube exposed - ``` - -4. The `hello-minikube` Pod is now launched but you have to wait until the Pod is up before accessing it via the exposed Service. - - Check if the Pod is up and running: - - ```shell - kubectl get pod - ``` - - If the output shows the `STATUS` as `ContainerCreating`, the Pod is still being created: - - ``` - NAME READY STATUS RESTARTS AGE - hello-minikube-3383150820-vctvh 0/1 ContainerCreating 0 3s - ``` - - If the output shows the `STATUS` as `Running`, the Pod is now up and running: - - ``` - NAME READY STATUS RESTARTS AGE - hello-minikube-3383150820-vctvh 1/1 Running 0 13s - ``` - -5. Get the URL of the exposed Service to view the Service details: - - ```shell - minikube service hello-minikube --url - ``` - -6. To view the details of your local cluster, copy and paste the URL you got as the output, on your browser. - - The output is similar to this: - - ``` - Hostname: hello-minikube-7c77b68cff-8wdzq - - Pod Information: - -no pod information available- - - Server values: - server_version=nginx: 1.13.3 - lua: 10008 - - Request Information: - client_address=172.17.0.1 - method=GET - real path=/ - query= - request_version=1.1 - request_scheme=http - request_uri=http://192.168.99.100:8080/ - - Request Headers: - accept=*/* - host=192.168.99.100:30674 - user-agent=curl/7.47.0 - - Request Body: - -no body in request- - ``` - - If you no longer want the Service and cluster to run, you can delete them. - -7. Delete the `hello-minikube` Service: - - ```shell - kubectl delete services hello-minikube - ``` - - The output is similar to this: - - ``` - service "hello-minikube" deleted - ``` - -8. Delete the `hello-minikube` Deployment: - - ```shell - kubectl delete deployment hello-minikube - ``` - - The output is similar to this: - - ``` - deployment.extensions "hello-minikube" deleted - ``` - -9. Stop the local Minikube cluster: - - ```shell - minikube stop - ``` - - The output is similar to this: - - ``` - Stopping "minikube"... - "minikube" stopped. - ``` - - For more information, see [Stopping a Cluster](#stopping-a-cluster). - -10. Delete the local Minikube cluster: - - ```shell - minikube delete - ``` - The output is similar to this: - ``` - Deleting "minikube" ... - The "minikube" cluster has been deleted. - ``` - For more information, see [Deleting a cluster](#deleting-a-cluster). - -## Managing your Cluster - -### Starting a Cluster - -The `minikube start` command can be used to start your cluster. -This command creates and configures a Virtual Machine that runs a single-node Kubernetes cluster. -This command also configures your [kubectl](/docs/reference/kubectl/overview/) installation to communicate with this cluster. - -{{< note >}} -If you are behind a web proxy, you need to pass this information to the `minikube start` command: - -```shell -https_proxy= minikube start --docker-env http_proxy= --docker-env https_proxy= --docker-env no_proxy=192.168.99.0/24 -``` -Unfortunately, setting the environment variables alone does not work. - -Minikube also creates a "minikube" context, and sets it to default in kubectl. -To switch back to this context, run this command: `kubectl config use-context minikube`. -{{< /note >}} - -#### Specifying the Kubernetes version - -You can specify the version of Kubernetes for Minikube to use by -adding the `--kubernetes-version` string to the `minikube start` command. For -example, to run version {{< param "fullversion" >}}, you would run the following: - -``` -minikube start --kubernetes-version {{< param "fullversion" >}} -``` -#### Specifying the VM driver -You can change the VM driver by adding the `--driver=` flag to `minikube start`. -For example the command would be. -```shell -minikube start --driver= -``` -Minikube supports the following drivers: -{{< note >}} -See [DRIVERS](https://minikube.sigs.k8s.io/docs/reference/drivers/) for details on supported drivers and how to install -plugins. -{{< /note >}} - -* docker ([driver installation](https://minikube.sigs.k8s.io/docs/drivers/docker/)) -* virtualbox ([driver installation](https://minikube.sigs.k8s.io/docs/drivers/virtualbox/)) -* podman ([driver installation](https://minikube.sigs.k8s.io/docs/drivers/podman/)) (EXPERIMENTAL) -* vmwarefusion -* kvm2 ([driver installation](https://minikube.sigs.k8s.io/docs/reference/drivers/kvm2/)) -* hyperkit ([driver installation](https://minikube.sigs.k8s.io/docs/reference/drivers/hyperkit/)) -* hyperv ([driver installation](https://minikube.sigs.k8s.io/docs/reference/drivers/hyperv/)) -Note that the IP below is dynamic and can change. It can be retrieved with `minikube ip`. -* vmware ([driver installation](https://minikube.sigs.k8s.io/docs/reference/drivers/vmware/)) (VMware unified driver) -* parallels ([driver installation](https://minikube.sigs.k8s.io/docs/reference/drivers/parallels/)) -* none (Runs the Kubernetes components on the host and not in a virtual machine. You need to be running Linux and to have {{< glossary_tooltip term_id="docker" >}} installed.) - -{{< caution >}} -If you use the `none` driver, some Kubernetes components run as privileged containers that have side effects outside of the Minikube environment. Those side effects mean that the `none` driver is not recommended for personal workstations. -{{< /caution >}} - -#### Starting a cluster on alternative container runtimes -You can start Minikube on the following container runtimes. -{{< tabs name="container_runtimes" >}} -{{% tab name="containerd" %}} -To use [containerd](https://github.com/containerd/containerd) as the container runtime, run: -```bash -minikube start \ - --network-plugin=cni \ - --enable-default-cni \ - --container-runtime=containerd \ - --bootstrapper=kubeadm -``` - -Or you can use the extended version: - -```bash -minikube start \ - --network-plugin=cni \ - --enable-default-cni \ - --extra-config=kubelet.container-runtime=remote \ - --extra-config=kubelet.container-runtime-endpoint=unix:///run/containerd/containerd.sock \ - --extra-config=kubelet.image-service-endpoint=unix:///run/containerd/containerd.sock \ - --bootstrapper=kubeadm -``` -{{% /tab %}} -{{% tab name="CRI-O" %}} -To use [CRI-O](https://cri-o.io/) as the container runtime, run: -```bash -minikube start \ - --network-plugin=cni \ - --enable-default-cni \ - --container-runtime=cri-o \ - --bootstrapper=kubeadm -``` -Or you can use the extended version: - -```bash -minikube start \ - --network-plugin=cni \ - --enable-default-cni \ - --extra-config=kubelet.container-runtime=remote \ - --extra-config=kubelet.container-runtime-endpoint=/var/run/crio.sock \ - --extra-config=kubelet.image-service-endpoint=/var/run/crio.sock \ - --bootstrapper=kubeadm -``` -{{% /tab %}} -{{< /tabs >}} - -#### Use local images by re-using the Docker daemon - -When using a single VM for Kubernetes, it's useful to reuse Minikube's built-in Docker daemon. Reusing the built-in daemon means you don't have to build a Docker registry on your host machine and push the image into it. Instead, you can build inside the same Docker daemon as Minikube, which speeds up local experiments. - -{{< note >}} -Be sure to tag your Docker image with something other than latest and use that tag to pull the image. Because `:latest` is the default value, with a corresponding default image pull policy of `Always`, an image pull error (`ErrImagePull`) eventually results if you do not have the Docker image in the default Docker registry (usually DockerHub). -{{< /note >}} - -To work with the Docker daemon on your Mac/Linux host, run the last line from `minikube docker-env`. - -You can now use Docker at the command line of your host Mac/Linux machine to communicate with the Docker daemon inside the Minikube VM: - -```shell -docker ps -``` - -{{< note >}} -On Centos 7, Docker may report the following error: - -``` -Could not read CA certificate "/etc/docker/ca.pem": open /etc/docker/ca.pem: no such file or directory -``` - -You can fix this by updating /etc/sysconfig/docker to ensure that Minikube's environment changes are respected: - -```shell -< DOCKER_CERT_PATH=/etc/docker ---- -> if [ -z "${DOCKER_CERT_PATH}" ]; then -> DOCKER_CERT_PATH=/etc/docker -> fi -``` -{{< /note >}} - -### Configuring Kubernetes - -Minikube has a "configurator" feature that allows users to configure the Kubernetes components with arbitrary values. -To use this feature, you can use the `--extra-config` flag on the `minikube start` command. - -This flag is repeated, so you can pass it several times with several different values to set multiple options. - -This flag takes a string of the form `component.key=value`, where `component` is one of the strings from the below list, `key` is a value on the -configuration struct and `value` is the value to set. - -Valid keys can be found by examining the documentation for the Kubernetes `componentconfigs` for each component. -Here is the documentation for each supported configuration: - -* [kubelet](https://godoc.org/k8s.io/kubernetes/pkg/kubelet/apis/config#KubeletConfiguration) -* [apiserver](https://godoc.org/k8s.io/kubernetes/cmd/kube-apiserver/app/options#ServerRunOptions) -* [proxy](https://godoc.org/k8s.io/kubernetes/pkg/proxy/apis/config#KubeProxyConfiguration) -* [controller-manager](https://godoc.org/k8s.io/kubernetes/pkg/controller/apis/config#KubeControllerManagerConfiguration) -* [etcd](https://godoc.org/github.com/coreos/etcd/etcdserver#ServerConfig) -* [scheduler](https://godoc.org/k8s.io/kubernetes/pkg/scheduler/apis/config#KubeSchedulerConfiguration) - -#### Examples - -To change the `MaxPods` setting to 5 on the Kubelet, pass this flag: `--extra-config=kubelet.MaxPods=5`. - -This feature also supports nested structs. To change the `LeaderElection.LeaderElect` setting to `true` on the scheduler, pass this flag: `--extra-config=scheduler.LeaderElection.LeaderElect=true`. - -To set the `AuthorizationMode` on the `apiserver` to `RBAC`, you can use: `--extra-config=apiserver.authorization-mode=RBAC`. - -### Stopping a Cluster -The `minikube stop` command can be used to stop your cluster. -This command shuts down the Minikube Virtual Machine, but preserves all cluster state and data. -Starting the cluster again will restore it to its previous state. - -### Deleting a Cluster -The `minikube delete` command can be used to delete your cluster. -This command shuts down and deletes the Minikube Virtual Machine. No data or state is preserved. - -### Upgrading Minikube -If you are using macOS and [Brew Package Manager](https://brew.sh/) is installed run: - -```shell -brew update -brew upgrade minikube -``` - -## Interacting with Your Cluster - -### Kubectl - -The `minikube start` command creates a [kubectl context](/docs/reference/generated/kubectl/kubectl-commands#-em-set-context-em-) called "minikube". -This context contains the configuration to communicate with your Minikube cluster. - -Minikube sets this context to default automatically, but if you need to switch back to it in the future, run: - -`kubectl config use-context minikube` - -Or pass the context on each command like this: - -`kubectl get pods --context=minikube` - -### Dashboard - -To access the [Kubernetes Dashboard](/docs/tasks/access-application-cluster/web-ui-dashboard/), run this command in a shell after starting Minikube to get the address: - -```shell -minikube dashboard -``` - -### Services - -To access a Service exposed via a node port, run this command in a shell after starting Minikube to get the address: - -```shell -minikube service [-n NAMESPACE] [--url] NAME -``` - -## Networking - -The Minikube VM is exposed to the host system via a host-only IP address, that can be obtained with the `minikube ip` command. -Any services of type `NodePort` can be accessed over that IP address, on the NodePort. - -To determine the NodePort for your service, you can use a `kubectl` command like this: - -`kubectl get service $SERVICE --output='jsonpath="{.spec.ports[0].nodePort}"'` - -## Persistent Volumes -Minikube supports [PersistentVolumes](/docs/concepts/storage/persistent-volumes/) of type `hostPath`. -These PersistentVolumes are mapped to a directory inside the Minikube VM. - -The Minikube VM boots into a tmpfs, so most directories will not be persisted across reboots (`minikube stop`). -However, Minikube is configured to persist files stored under the following host directories: - -* `/data` -* `/var/lib/minikube` -* `/var/lib/docker` - -Here is an example PersistentVolume config to persist data in the `/data` directory: - -```yaml -apiVersion: v1 -kind: PersistentVolume -metadata: - name: pv0001 -spec: - accessModes: - - ReadWriteOnce - capacity: - storage: 5Gi - hostPath: - path: /data/pv0001/ -``` - -## Mounted Host Folders -Some drivers will mount a host folder within the VM so that you can easily share files between the VM and host. These are not configurable at the moment and different for the driver and OS you are using. - -{{< note >}} -Host folder sharing is not implemented in the KVM driver yet. -{{< /note >}} - -| Driver | OS | HostFolder | VM | -| --- | --- | --- | --- | -| VirtualBox | Linux | `/home` | `/hosthome` | -| VirtualBox | macOS | `/Users` | `/Users` | -| VirtualBox | Windows | `C://Users` | `/c/Users` | -| VMware Fusion | macOS | `/Users` | `/mnt/hgfs/Users` | -| Xhyve | macOS | `/Users` | `/Users` | - -## Private Container Registries - -To access a private container registry, follow the steps on [this page](/docs/concepts/containers/images/). - -We recommend you use `ImagePullSecrets`, but if you would like to configure access on the Minikube VM you can place the `.dockercfg` in the `/home/docker` directory or the `config.json` in the `/home/docker/.docker` directory. - -## Add-ons - -In order to have Minikube properly start or restart custom addons, -place the addons you wish to be launched with Minikube in the `~/.minikube/addons` -directory. Addons in this folder will be moved to the Minikube VM and -launched each time Minikube is started or restarted. - -## Using Minikube with an HTTP Proxy - -Minikube creates a Virtual Machine that includes Kubernetes and a Docker daemon. -When Kubernetes attempts to schedule containers using Docker, the Docker daemon may require external network access to pull containers. - -If you are behind an HTTP proxy, you may need to supply Docker with the proxy settings. -To do this, pass the required environment variables as flags during `minikube start`. - -For example: - -```shell -minikube start --docker-env http_proxy=http://$YOURPROXY:PORT \ - --docker-env https_proxy=https://$YOURPROXY:PORT -``` - -If your Virtual Machine address is 192.168.99.100, then chances are your proxy settings will prevent `kubectl` from directly reaching it. -To by-pass proxy configuration for this IP address, you should modify your no_proxy settings. You can do so with: - -```shell -export no_proxy=$no_proxy,$(minikube ip) -``` - -## Known Issues - -Features that require multiple nodes will not work in Minikube. - -## Design - -Minikube uses [libmachine](https://github.com/docker/machine/tree/master/libmachine) for provisioning VMs, and [kubeadm](https://github.com/kubernetes/kubeadm) to provision a Kubernetes cluster. - -For more information about Minikube, see the [proposal](https://git.k8s.io/community/contributors/design-proposals/cluster-lifecycle/local-cluster-ux.md). - -## Additional Links - -* **Goals and Non-Goals**: For the goals and non-goals of the Minikube project, please see our [roadmap](https://minikube.sigs.k8s.io/docs/contrib/roadmap/). -* **Development Guide**: See [Contributing](https://minikube.sigs.k8s.io/docs/contrib/) for an overview of how to send pull requests. -* **Building Minikube**: For instructions on how to build/test Minikube from source, see the [build guide](https://minikube.sigs.k8s.io/docs/contrib/building/). -* **Adding a New Dependency**: For instructions on how to add a new dependency to Minikube, see the [adding dependencies guide](https://minikube.sigs.k8s.io/docs/contrib/drivers/). -* **Adding a New Addon**: For instructions on how to add a new addon for Minikube, see the [adding an addon guide](https://minikube.sigs.k8s.io/docs/contrib/addons/). -* **MicroK8s**: Linux users wishing to avoid running a virtual machine may consider [MicroK8s](https://microk8s.io/) as an alternative. - -## Community - -Contributions, questions, and comments are all welcomed and encouraged! Minikube developers hang out on [Slack](https://kubernetes.slack.com) in the `#minikube` channel (get an invitation [here](https://slack.kubernetes.io/)). We also have the [kubernetes-dev Google Groups mailing list](https://groups.google.com/forum/#!forum/kubernetes-dev). If you are posting to the list please prefix your subject with "minikube: ". - - diff --git a/content/en/docs/setup/production-environment/container-runtimes.md b/content/en/docs/setup/production-environment/container-runtimes.md index feb2a72673..c702aa5ca7 100644 --- a/content/en/docs/setup/production-environment/container-runtimes.md +++ b/content/en/docs/setup/production-environment/container-runtimes.md @@ -7,62 +7,368 @@ content_type: concept weight: 10 --- -{{< feature-state for_k8s_version="v1.6" state="stable" >}} -To run containers in Pods, Kubernetes uses a container runtime. Here are -the installation instructions for various runtimes. - +You need to install a +{{< glossary_tooltip text="container runtime" term_id="container-runtime" >}} +into each node in the cluster so that Pods can run there. This page outlines +what is involved and describes related tasks for setting up nodes. +This page lists details for using several common container runtimes with +Kubernetes, on Linux: -{{< caution >}} -A flaw was found in the way runc handled system file descriptors when running containers. -A malicious container could use this flaw to overwrite contents of the runc binary and -consequently run arbitrary commands on the container host system. - -Please refer to [CVE-2019-5736](https://access.redhat.com/security/cve/cve-2019-5736) for more -information about the issue. -{{< /caution >}} - -### Applicability +- [containerd](#containerd) +- [CRI-O](#cri-o) +- [Docker](#docker) {{< note >}} -This document is written for users installing CRI onto Linux. For other operating -systems, look for documentation specific to your platform. +For other operating systems, look for documentation specific to your platform. {{< /note >}} -### Cgroup drivers - -When systemd is chosen as the init system for a Linux distribution, the init process generates -and consumes a root control group (`cgroup`) and acts as a cgroup manager. Systemd has a tight -integration with cgroups and will allocate cgroups per process. It's possible to configure your -container runtime and the kubelet to use `cgroupfs`. Using `cgroupfs` alongside systemd means -that there will be two different cgroup managers. +## Cgroup drivers Control groups are used to constrain resources that are allocated to processes. -A single cgroup manager will simplify the view of what resources are being allocated -and will by default have a more consistent view of the available and in-use resources. When we have -two managers we end up with two views of those resources. We have seen cases in the field -where nodes that are configured to use `cgroupfs` for the kubelet and Docker, and `systemd` -for the rest of the processes running on the node becomes unstable under resource pressure. + +When [systemd](https://www.freedesktop.org/wiki/Software/systemd/) is chosen as the init +system for a Linux distribution, the init process generates and consumes a root control group +(`cgroup`) and acts as a cgroup manager. +Systemd has a tight integration with cgroups and allocates a cgroup per systemd unit. It's possible +to configure your container runtime and the kubelet to use `cgroupfs`. Using `cgroupfs` alongside +systemd means that there will be two different cgroup managers. + +A single cgroup manager simplifies the view of what resources are being allocated +and will by default have a more consistent view of the available and in-use resources. +When there are two cgroup managers on a system, you end up with two views of those resources. +In the field, people have reported cases where nodes that are configured to use `cgroupfs` +for the kubelet and Docker, but `systemd` for the rest of the processes, become unstable under +resource pressure. Changing the settings such that your container runtime and kubelet use `systemd` as the cgroup driver -stabilized the system. Please note the `native.cgroupdriver=systemd` option in the Docker setup below. +stabilized the system. To configure this for Docker, set `native.cgroupdriver=systemd`. {{< caution >}} -Changing the cgroup driver of a Node that has joined a cluster is highly unrecommended. +Changing the cgroup driver of a Node that has joined a cluster is strongly *not* recommended. If the kubelet has created Pods using the semantics of one cgroup driver, changing the container -runtime to another cgroup driver can cause errors when trying to re-create the PodSandbox -for such existing Pods. Restarting the kubelet may not solve such errors. The recommendation -is to drain the Node from its workloads, remove it from the cluster and re-join it. +runtime to another cgroup driver can cause errors when trying to re-create the Pod sandbox +for such existing Pods. Restarting the kubelet may not solve such errors. + +If you have automation that makes it feasible, replace the node with another using the updated +configuration, or reinstall it using automation. {{< /caution >}} -## Docker +## Container runtimes -On each of your machines, install Docker. -Version 19.03.11 is recommended, but 1.13.1, 17.03, 17.06, 17.09, 18.06 and 18.09 are known to work as well. -Keep track of the latest verified Docker version in the Kubernetes release notes. +{{% thirdparty-content %}} + +### containerd + +This section contains the necessary steps to use `containerd` as CRI runtime. + +Use the following commands to install Containerd on your system: + +Install and configure prerequisites: + +```shell +cat <}} +{{% tab name="Ubuntu 16.04" %}} + +```shell +# (Install containerd) +## Set up the repository +### Install packages to allow apt to use a repository over HTTPS +sudo apt-get update && sudo apt-get install -y apt-transport-https ca-certificates curl software-properties-common +``` + +```shell +## Add Docker's official GPG key +curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add --keyring /etc/apt/trusted.gpg.d/docker.gpg - +``` + +```shell +## Add Docker apt repository. +sudo add-apt-repository \ + "deb [arch=amd64] https://download.docker.com/linux/ubuntu \ + $(lsb_release -cs) \ + stable" +``` + +```shell +## Install containerd +sudo apt-get update && sudo apt-get install -y containerd.io +``` + +```shell +# Configure containerd +sudo mkdir -p /etc/containerd +sudo containerd config default > /etc/containerd/config.toml +``` + +```shell +# Restart containerd +sudo systemctl restart containerd +``` +{{% /tab %}} +{{% tab name="CentOS/RHEL 7.4+" %}} + +```shell +# (Install containerd) +## Set up the repository +### Install required packages +sudo yum install -y yum-utils device-mapper-persistent-data lvm2 +``` + +```shell +## Add docker repository +sudo yum-config-manager \ + --add-repo \ + https://download.docker.com/linux/centos/docker-ce.repo +``` + +```shell +## Install containerd +sudo yum update -y && sudo yum install -y containerd.io +``` + +```shell +## Configure containerd +sudo mkdir -p /etc/containerd +sudo containerd config default > /etc/containerd/config.toml +``` + +```shell +# Restart containerd +sudo systemctl restart containerd +``` +{{% /tab %}} +{{% tab name="Windows (PowerShell)" %}} +```powershell +# (Install containerd) +# download containerd +cmd /c curl -OL https://github.com/containerd/containerd/releases/download/v1.4.0-beta.2/containerd-1.4.0-beta.2-windows-amd64.tar.gz +cmd /c tar xvf .\containerd-1.4.0-beta.2-windows-amd64.tar.gz +``` + +```powershell +# extract and configure +Copy-Item -Path ".\bin\" -Destination "$Env:ProgramFiles\containerd" -Recurse -Force +cd $Env:ProgramFiles\containerd\ +.\containerd.exe config default | Out-File config.toml -Encoding ascii + +# review the configuration. depending on setup you may want to adjust: +# - the sandbox_image (kubernetes pause image) +# - cni bin_dir and conf_dir locations +Get-Content config.toml +``` + +```powershell +# start containerd +.\containerd.exe --register-service +Start-Service containerd +``` +{{% /tab %}} +{{< /tabs >}} + +#### systemd + +To use the `systemd` cgroup driver in `/etc/containerd/config.toml` with `runc`, set + +``` +[plugins.cri] +systemd_cgroup = true +``` + +When using kubeadm, manually configure the +[cgroup driver for kubelet](/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#configure-cgroup-driver-used-by-kubelet-on-control-plane-node). + +### CRI-O + +This section contains the necessary steps to install CRI-O as a container runtime. + +Use the following commands to install CRI-O on your system: + +{{< note >}} +The CRI-O major and minor versions must match the Kubernetes major and minor versions. +For more information, see the [CRI-O compatibility matrix](https://github.com/cri-o/cri-o). +{{< /note >}} + +Install and configure prerequisites: + +```shell +sudo modprobe overlay +sudo modprobe br_netfilter + +# Set up required sysctl params, these persist across reboots. +cat <}} +{{% tab name="Debian" %}} + +To install CRI-O on the following operating systems, set the environment variable `OS` +to the appropriate value from the following table: + +| Operating system | `$OS` | +| ---------------- | ----------------- | +| Debian Unstable | `Debian_Unstable` | +| Debian Testing | `Debian_Testing` | + +
+Then, set `$VERSION` to the CRI-O version that matches your Kubernetes version. +For instance, if you want to install CRI-O 1.18, set `VERSION=1.18`. +You can pin your installation to a specific release. +To install version 1.18.3, set `VERSION=1.18:1.18.3`. +
+ +Then run +```shell +cat < +Then, set `$VERSION` to the CRI-O version that matches your Kubernetes version. +For instance, if you want to install CRI-O 1.18, set `VERSION=1.18`. +You can pin your installation to a specific release. +To install version 1.18.3, set `VERSION=1.18:1.18.3`. +
+ +Then run +```shell +cat < +Then, set `$VERSION` to the CRI-O version that matches your Kubernetes version. +For instance, if you want to install CRI-O 1.18, set `VERSION=1.18`. +You can pin your installation to a specific release. +To install version 1.18.3, set `VERSION=1.18:1.18.3`. +
+ +Then run +```shell +sudo curl -L -o /etc/yum.repos.d/devel:kubic:libcontainers:stable.repo https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/$OS/devel:kubic:libcontainers:stable.repo +sudo curl -L -o /etc/yum.repos.d/devel:kubic:libcontainers:stable:cri-o:$VERSION.repo https://download.opensuse.org/repositories/devel:kubic:libcontainers:stable:cri-o:$VERSION/$OS/devel:kubic:libcontainers:stable:cri-o:$VERSION.repo +sudo yum install cri-o +``` + +{{% /tab %}} + +{{% tab name="openSUSE Tumbleweed" %}} + +```shell +sudo zypper install cri-o +``` +{{% /tab %}} +{{% tab name="Fedora" %}} + +Set `$VERSION` to the CRI-O version that matches your Kubernetes version. +For instance, if you want to install CRI-O 1.18, `VERSION=1.18`. + +You can find available versions with: +```shell +sudo dnf module list cri-o +``` +CRI-O does not support pinning to specific releases on Fedora. + +Then run +```shell +sudo dnf module enable cri-o:$VERSION +sudo dnf install cri-o +``` + +{{% /tab %}} +{{< /tabs >}} + +Start CRI-O: + +```shell +sudo systemctl daemon-reload +sudo systemctl start crio +``` + +Refer to the [CRI-O installation guide](https://github.com/kubernetes-sigs/cri-o#getting-started) +for more information. + + + +### Docker + +On each of your nodes, install Docker CE. + +The Kubernetes release notes list which versions of Docker are compatible +with that version of Kubernetes. Use the following commands to install Docker on your system: @@ -79,7 +385,7 @@ sudo apt-get update && sudo apt-get install -y \ ```shell # Add Docker's official GPG key: -curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - +curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add --keyring /etc/apt/trusted.gpg.d/docker.gpg - ``` ```shell @@ -179,7 +485,7 @@ sudo systemctl restart docker {{% /tab %}} {{< /tabs >}} -If you want the docker service to start on boot, run the following command: +If you want the `docker` service to start on boot, run the following command: ```shell sudo systemctl enable docker @@ -188,301 +494,3 @@ sudo systemctl enable docker Refer to the [official Docker installation guides](https://docs.docker.com/engine/installation/) for more information. -## CRI-O - -This section contains the necessary steps to install `CRI-O` as CRI runtime. - -Use the following commands to install CRI-O on your system: - -{{< note >}} -The CRI-O major and minor versions must match the Kubernetes major and minor versions. -For more information, see the [CRI-O compatibility matrix](https://github.com/cri-o/cri-o). -{{< /note >}} - -### Prerequisites - -```shell -sudo modprobe overlay -sudo modprobe br_netfilter - -# Set up required sysctl params, these persist across reboots. -cat <}} -{{% tab name="Debian" %}} - -To install CRI-O on the following operating systems, set the environment variable $OS to the appropriate field in the following table: - -| Operating system | $OS | -| ---------------- | ----------------- | -| Debian Unstable | `Debian_Unstable` | -| Debian Testing | `Debian_Testing` | - -
-Then, set `$VERSION` to the CRI-O version that matches your Kubernetes version. -For instance, if you want to install CRI-O 1.18, set `VERSION=1.18`. -You can pin your installation to a specific release. -To install version 1.18.3, set `VERSION=1.18:1.18.3`. -
- -Then run -```shell -cat < -Then, set `$VERSION` to the CRI-O version that matches your Kubernetes version. -For instance, if you want to install CRI-O 1.18, set `VERSION=1.18`. -You can pin your installation to a specific release. -To install version 1.18.3, set `VERSION=1.18:1.18.3`. -
- -Then run -```shell -sudo echo "deb https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/$OS/ /" > /etc/apt/sources.list.d/devel:kubic:libcontainers:stable.list -sudo echo "deb http://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable:/cri-o:/$VERSION/$OS/ /" > /etc/apt/sources.list.d/devel:kubic:libcontainers:stable:cri-o:$VERSION.list - -curl -L https://download.opensuse.org/repositories/devel:kubic:libcontainers:stable:cri-o:$VERSION/$OS/Release.key | sudo apt-key add - -curl -L https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/$OS/Release.key | sudo apt-key add - - -sudo apt-get update -sudo apt-get install cri-o cri-o-runc -``` -{{% /tab %}} - -{{% tab name="CentOS" %}} - -To install on the following operating systems, set the environment variable $OS to the appropriate field in the following table: - -| Operating system | $OS | -| ---------------- | ----------------- | -| Centos 8 | `CentOS_8` | -| Centos 8 Stream | `CentOS_8_Stream` | -| Centos 7 | `CentOS_7` | - -
-Then, set `$VERSION` to the CRI-O version that matches your Kubernetes version. -For instance, if you want to install CRI-O 1.18, set `VERSION=1.18`. -You can pin your installation to a specific release. -To install version 1.18.3, set `VERSION=1.18:1.18.3`. -
- -Then run -```shell -sudo curl -L -o /etc/yum.repos.d/devel:kubic:libcontainers:stable.repo https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/$OS/devel:kubic:libcontainers:stable.repo -sudo curl -L -o /etc/yum.repos.d/devel:kubic:libcontainers:stable:cri-o:$VERSION.repo https://download.opensuse.org/repositories/devel:kubic:libcontainers:stable:cri-o:$VERSION/$OS/devel:kubic:libcontainers:stable:cri-o:$VERSION.repo -sudo yum install cri-o -``` - -{{% /tab %}} - -{{% tab name="openSUSE Tumbleweed" %}} - -```shell -sudo zypper install cri-o -``` -{{% /tab %}} -{{% tab name="Fedora" %}} - -Set `$VERSION` to the CRI-O version that matches your Kubernetes version. -For instance, if you want to install CRI-O 1.18, `VERSION=1.18` -You can find available versions with: -```shell -sudo dnf module list cri-o -``` -CRI-O does not support pinning to specific releases on Fedora. - -Then run -```shell -sudo dnf module enable cri-o:$VERSION -sudo dnf install cri-o -``` - -{{% /tab %}} -{{< /tabs >}} - -### Start CRI-O - -```shell -sudo systemctl daemon-reload -sudo systemctl start crio -``` - -Refer to the [CRI-O installation guide](https://github.com/kubernetes-sigs/cri-o#getting-started) -for more information. - -## Containerd - -This section contains the necessary steps to use `containerd` as CRI runtime. - -Use the following commands to install Containerd on your system: - -### Prerequisites - -```shell -cat <}} -{{% tab name="Ubuntu 16.04" %}} - -```shell -# (Install containerd) -## Set up the repository -### Install packages to allow apt to use a repository over HTTPS -sudo apt-get update && sudo apt-get install -y apt-transport-https ca-certificates curl software-properties-common -``` - -```shell -## Add Docker's official GPG key -curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - -``` - -```shell -## Add Docker apt repository. -sudo add-apt-repository \ - "deb [arch=amd64] https://download.docker.com/linux/ubuntu \ - $(lsb_release -cs) \ - stable" -``` - -```shell -## Install containerd -sudo apt-get update && sudo apt-get install -y containerd.io -``` - -```shell -# Configure containerd -sudo mkdir -p /etc/containerd -sudo containerd config default > /etc/containerd/config.toml -``` - -```shell -# Restart containerd -sudo systemctl restart containerd -``` -{{% /tab %}} -{{% tab name="CentOS/RHEL 7.4+" %}} - -```shell -# (Install containerd) -## Set up the repository -### Install required packages -sudo yum install -y yum-utils device-mapper-persistent-data lvm2 -``` - -```shell -## Add docker repository -sudo yum-config-manager \ - --add-repo \ - https://download.docker.com/linux/centos/docker-ce.repo -``` - -```shell -## Install containerd -sudo yum update -y && sudo yum install -y containerd.io -``` - -```shell -## Configure containerd -sudo mkdir -p /etc/containerd -sudo containerd config default > /etc/containerd/config.toml -``` - -```shell -# Restart containerd -sudo systemctl restart containerd -``` -{{% /tab %}} -{{% tab name="Windows (PowerShell)" %}} -```powershell -# (Install containerd) -# download containerd -cmd /c curl -OL https://github.com/containerd/containerd/releases/download/v1.4.0-beta.2/containerd-1.4.0-beta.2-windows-amd64.tar.gz -cmd /c tar xvf .\containerd-1.4.0-beta.2-windows-amd64.tar.gz -``` - -```powershell -# extract and configure -Copy-Item -Path ".\bin\" -Destination "$Env:ProgramFiles\containerd" -Recurse -Force -cd $Env:ProgramFiles\containerd\ -.\containerd.exe config default | Out-File config.toml -Encoding ascii - -# review the configuration. depending on setup you may want to adjust: -# - the sandbox_image (kubernetes pause image) -# - cni bin_dir and conf_dir locations -Get-Content config.toml -``` - -```powershell -# start containerd -.\containerd.exe --register-service -Start-Service containerd -``` -{{% /tab %}} -{{< /tabs >}} - -### systemd - -To use the `systemd` cgroup driver in `/etc/containerd/config.toml` set - -``` -[plugins.cri] -systemd_cgroup = true -``` -When using kubeadm, manually configure the -[cgroup driver for kubelet](/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#configure-cgroup-driver-used-by-kubelet-on-control-plane-node) - -## Other CRI runtimes: frakti - -Refer to the [Frakti QuickStart guide](https://github.com/kubernetes/frakti#quickstart) for more information. - - diff --git a/content/en/docs/setup/production-environment/tools/kubeadm/kubelet-integration.md b/content/en/docs/setup/production-environment/tools/kubeadm/kubelet-integration.md index af4eb4a101..f68b026ccb 100644 --- a/content/en/docs/setup/production-environment/tools/kubeadm/kubelet-integration.md +++ b/content/en/docs/setup/production-environment/tools/kubeadm/kubelet-integration.md @@ -200,6 +200,6 @@ The DEB and RPM packages shipped with the Kubernetes releases are: | `kubeadm` | Installs the `/usr/bin/kubeadm` CLI tool and the [kubelet drop-in file](#the-kubelet-drop-in-file-for-systemd) for the kubelet. | | `kubelet` | Installs the kubelet binary in `/usr/bin` and CNI binaries in `/opt/cni/bin`. | | `kubectl` | Installs the `/usr/bin/kubectl` binary. | -| `cri-tools` | Installs the `/usr/bin/crictl` binary from the [cri-tools git repository](https://github.com/kubernetes-incubator/cri-tools). | +| `cri-tools` | Installs the `/usr/bin/crictl` binary from the [cri-tools git repository](https://github.com/kubernetes-sigs/cri-tools). | diff --git a/content/en/docs/setup/production-environment/turnkey/aws.md b/content/en/docs/setup/production-environment/turnkey/aws.md index be75623158..7dd901aa0f 100644 --- a/content/en/docs/setup/production-environment/turnkey/aws.md +++ b/content/en/docs/setup/production-environment/turnkey/aws.md @@ -23,7 +23,7 @@ To create a Kubernetes cluster on AWS, you will need an Access Key ID and a Secr * [Kubernetes Operations](https://github.com/kubernetes/kops) - Production Grade K8s Installation, Upgrades, and Management. Supports running Debian, Ubuntu, CentOS, and RHEL in AWS. -* [kube-aws](https://github.com/kubernetes-incubator/kube-aws), creates and manages Kubernetes clusters with [Flatcar Linux](https://www.flatcar-linux.org/) nodes, using AWS tools: EC2, CloudFormation and Autoscaling. +* [kube-aws](https://github.com/kubernetes-retired/kube-aws), creates and manages Kubernetes clusters with [Flatcar Linux](https://www.flatcar-linux.org/) nodes, using AWS tools: EC2, CloudFormation and Autoscaling. * [KubeOne](https://github.com/kubermatic/kubeone) is an open source cluster lifecycle management tool that creates, upgrades and manages Kubernetes Highly-Available clusters. diff --git a/content/en/docs/tasks/access-application-cluster/access-cluster.md b/content/en/docs/tasks/access-application-cluster/access-cluster.md index d05de37f34..7f74320118 100644 --- a/content/en/docs/tasks/access-application-cluster/access-cluster.md +++ b/content/en/docs/tasks/access-application-cluster/access-cluster.md @@ -149,9 +149,8 @@ certificate. On some clusters, the apiserver does not require authentication; it may serve on localhost, or be protected by a firewall. There is not a standard -for this. [Configuring Access to the API](/docs/reference/access-authn-authz/controlling-access/) -describes how a cluster admin can configure this. Such approaches may conflict -with future high-availability support. +for this. [Controlling Access to the API](/docs/concepts/security/controlling-access) +describes how a cluster admin can configure this. ## Programmatic access to the API diff --git a/content/en/docs/tasks/access-application-cluster/configure-access-multiple-clusters.md b/content/en/docs/tasks/access-application-cluster/configure-access-multiple-clusters.md index 3f3100395c..7d1db15451 100644 --- a/content/en/docs/tasks/access-application-cluster/configure-access-multiple-clusters.md +++ b/content/en/docs/tasks/access-application-cluster/configure-access-multiple-clusters.md @@ -156,7 +156,7 @@ users: ``` The `fake-ca-file`, `fake-cert-file` and `fake-key-file` above are the placeholders -for the pathnames of the certificate files. You need change these to the actual pathnames +for the pathnames of the certificate files. You need to change these to the actual pathnames of certificate files in your environment. Sometimes you may want to use Base64-encoded data embedded here instead of separate diff --git a/content/en/docs/tasks/administer-cluster/access-cluster-api.md b/content/en/docs/tasks/administer-cluster/access-cluster-api.md index d09a29c671..ffe200b118 100644 --- a/content/en/docs/tasks/administer-cluster/access-cluster-api.md +++ b/content/en/docs/tasks/administer-cluster/access-cluster-api.md @@ -148,9 +148,8 @@ certificate. On some clusters, the API server does not require authentication; it may serve on localhost, or be protected by a firewall. There is not a standard -for this. [Configuring Access to the API](/docs/reference/access-authn-authz/controlling-access/) -describes how a cluster admin can configure this. Such approaches may conflict -with future high-availability support. +for this. [Controlling Access to the Kubernetes API](/docs/concepts/security/controlling-access) +describes how you can configure this as a cluster administrator. ### Programmatic access to the API diff --git a/content/en/docs/tasks/administer-cluster/cluster-management.md b/content/en/docs/tasks/administer-cluster/cluster-management.md index 3120f4c640..8141b228ab 100644 --- a/content/en/docs/tasks/administer-cluster/cluster-management.md +++ b/content/en/docs/tasks/administer-cluster/cluster-management.md @@ -73,7 +73,7 @@ Oracle creates and manages a set of master nodes in the Oracle control plane on Different providers, and tools, will manage upgrades differently. It is recommended that you consult their main documentation regarding upgrades. * [kops](https://github.com/kubernetes/kops) -* [kubespray](https://github.com/kubernetes-incubator/kubespray) +* [kubespray](https://github.com/kubernetes-sigs/kubespray) * [CoreOS Tectonic](https://coreos.com/tectonic/docs/latest/admin/upgrade.html) * [Digital Rebar](https://provision.readthedocs.io/en/tip/doc/content-packages/krib.html) * ... diff --git a/content/en/docs/tasks/administer-cluster/dns-horizontal-autoscaling.md b/content/en/docs/tasks/administer-cluster/dns-horizontal-autoscaling.md index f333b215a2..bdcb35ada2 100644 --- a/content/en/docs/tasks/administer-cluster/dns-horizontal-autoscaling.md +++ b/content/en/docs/tasks/administer-cluster/dns-horizontal-autoscaling.md @@ -143,7 +143,7 @@ The idea is that when a cluster is using nodes that have many cores, cores, `nodesPerReplica` dominates. There are other supported scaling patterns. For details, see -[cluster-proportional-autoscaler](https://github.com/kubernetes-incubator/cluster-proportional-autoscaler). +[cluster-proportional-autoscaler](https://github.com/kubernetes-sigs/cluster-proportional-autoscaler). ## Disable DNS horizontal autoscaling @@ -233,5 +233,5 @@ patterns: *linear* and *ladder*. * Read about [Guaranteed Scheduling For Critical Add-On Pods](/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods/). * Learn more about the -[implementation of cluster-proportional-autoscaler](https://github.com/kubernetes-incubator/cluster-proportional-autoscaler). +[implementation of cluster-proportional-autoscaler](https://github.com/kubernetes-sigs/cluster-proportional-autoscaler). diff --git a/content/en/docs/tasks/administer-cluster/ip-masq-agent.md b/content/en/docs/tasks/administer-cluster/ip-masq-agent.md index 9c2e1d3d5d..2997508a15 100644 --- a/content/en/docs/tasks/administer-cluster/ip-masq-agent.md +++ b/content/en/docs/tasks/administer-cluster/ip-masq-agent.md @@ -62,7 +62,7 @@ By default, in GCE/Google Kubernetes Engine starting with Kubernetes version 1.7 To create an ip-masq-agent, run the following kubectl command: ` -kubectl apply -f https://raw.githubusercontent.com/kubernetes-incubator/ip-masq-agent/master/ip-masq-agent.yaml +kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/ip-masq-agent/master/ip-masq-agent.yaml ` You must also apply the appropriate node label to any nodes in your cluster that you want the agent to run on. @@ -71,7 +71,7 @@ You must also apply the appropriate node label to any nodes in your cluster that kubectl label nodes my-node beta.kubernetes.io/masq-agent-ds-ready=true ` -More information can be found in the ip-masq-agent documentation [here](https://github.com/kubernetes-incubator/ip-masq-agent) +More information can be found in the ip-masq-agent documentation [here](https://github.com/kubernetes-sigs/ip-masq-agent) In most cases, the default set of rules should be sufficient; however, if this is not the case for your cluster, you can create and apply a [ConfigMap](/docs/tasks/configure-pod-container/configure-pod-configmap/) to customize the IP ranges that are affected. For example, to allow only 10.0.0.0/8 to be considered by the ip-masq-agent, you can create the following [ConfigMap](/docs/tasks/configure-pod-container/configure-pod-configmap/) in a file called "config". diff --git a/content/en/docs/tasks/administer-cluster/kubeadm/kubeadm-certs.md b/content/en/docs/tasks/administer-cluster/kubeadm/kubeadm-certs.md index 02687a85f2..abd072b5a3 100644 --- a/content/en/docs/tasks/administer-cluster/kubeadm/kubeadm-certs.md +++ b/content/en/docs/tasks/administer-cluster/kubeadm/kubeadm-certs.md @@ -10,7 +10,7 @@ weight: 10 {{< feature-state for_k8s_version="v1.15" state="stable" >}} -Client certificates generated by [kubeadm](/docs/reference/setup-tools/kubeadm/kubeadm/) expire after 1 year. This page explains how to manage certificate renewals with kubeadm. +Client certificates generated by [kubeadm](/docs/reference/setup-tools/kubeadm/) expire after 1 year. This page explains how to manage certificate renewals with kubeadm. ## {{% heading "prerequisites" %}} diff --git a/content/en/docs/tasks/administer-cluster/network-policy-provider/romana-network-policy.md b/content/en/docs/tasks/administer-cluster/network-policy-provider/romana-network-policy.md index 93312e199e..6a57d8cc0b 100644 --- a/content/en/docs/tasks/administer-cluster/network-policy-provider/romana-network-policy.md +++ b/content/en/docs/tasks/administer-cluster/network-policy-provider/romana-network-policy.md @@ -12,7 +12,7 @@ This page shows how to use Romana for NetworkPolicy. ## {{% heading "prerequisites" %}} -Complete steps 1, 2, and 3 of the [kubeadm getting started guide](/docs/reference/setup-tools/kubeadm/kubeadm/). +Complete steps 1, 2, and 3 of the [kubeadm getting started guide](/docs/reference/setup-tools/kubeadm/). diff --git a/content/en/docs/tasks/administer-cluster/network-policy-provider/weave-network-policy.md b/content/en/docs/tasks/administer-cluster/network-policy-provider/weave-network-policy.md index 988eafda70..fcbc9c4045 100644 --- a/content/en/docs/tasks/administer-cluster/network-policy-provider/weave-network-policy.md +++ b/content/en/docs/tasks/administer-cluster/network-policy-provider/weave-network-policy.md @@ -13,7 +13,7 @@ This page shows how to use Weave Net for NetworkPolicy. ## {{% heading "prerequisites" %}} You need to have a Kubernetes cluster. Follow the -[kubeadm getting started guide](/docs/reference/setup-tools/kubeadm/kubeadm/) to bootstrap one. +[kubeadm getting started guide](/docs/reference/setup-tools/kubeadm/) to bootstrap one. diff --git a/content/en/docs/tasks/configure-pod-container/assign-cpu-resource.md b/content/en/docs/tasks/configure-pod-container/assign-cpu-resource.md index 76c555aace..243072eff2 100644 --- a/content/en/docs/tasks/configure-pod-container/assign-cpu-resource.md +++ b/content/en/docs/tasks/configure-pod-container/assign-cpu-resource.md @@ -22,7 +22,7 @@ allocated as much CPU as it requests. Your cluster must have at least 1 CPU available for use to run the task examples. A few of the steps on this page require you to run the -[metrics-server](https://github.com/kubernetes-incubator/metrics-server) +[metrics-server](https://github.com/kubernetes-sigs/metrics-server) service in your cluster. If you have the metrics-server running, you can skip those steps. diff --git a/content/en/docs/tasks/configure-pod-container/assign-memory-resource.md b/content/en/docs/tasks/configure-pod-container/assign-memory-resource.md index 79bc2b86b6..7afb3cb249 100644 --- a/content/en/docs/tasks/configure-pod-container/assign-memory-resource.md +++ b/content/en/docs/tasks/configure-pod-container/assign-memory-resource.md @@ -21,7 +21,7 @@ but is not allowed to use more memory than its limit. Each node in your cluster must have at least 300 MiB of memory. A few of the steps on this page require you to run the -[metrics-server](https://github.com/kubernetes-incubator/metrics-server) +[metrics-server](https://github.com/kubernetes-sigs/metrics-server) service in your cluster. If you have the metrics-server running, you can skip those steps. diff --git a/content/en/docs/tasks/configure-pod-container/configure-persistent-volume-storage.md b/content/en/docs/tasks/configure-pod-container/configure-persistent-volume-storage.md index 2644dfd3cb..a53d278223 100644 --- a/content/en/docs/tasks/configure-pod-container/configure-persistent-volume-storage.md +++ b/content/en/docs/tasks/configure-pod-container/configure-persistent-volume-storage.md @@ -29,7 +29,7 @@ PersistentVolume. {{< glossary_tooltip text="kubectl" term_id="kubectl" >}} command-line tool must be configured to communicate with your cluster. If you do not already have a single-node cluster, you can create one by using -[Minikube](/docs/setup/learning-environment/minikube/). +[Minikube](https://minikube.sigs.k8s.io/docs/). * Familiarize yourself with the material in [Persistent Volumes](/docs/concepts/storage/persistent-volumes/). diff --git a/content/en/docs/tasks/configure-pod-container/configure-pod-initialization.md b/content/en/docs/tasks/configure-pod-container/configure-pod-initialization.md index 9a8a33f655..cca96f2b79 100644 --- a/content/en/docs/tasks/configure-pod-container/configure-pod-initialization.md +++ b/content/en/docs/tasks/configure-pod-container/configure-pod-initialization.md @@ -37,7 +37,7 @@ shared Volume at `/work-dir`, and the application container mounts the shared Volume at `/usr/share/nginx/html`. The init container runs the following command and then terminates: - wget -O /work-dir/index.html http://kubernetes.io + wget -O /work-dir/index.html http://info.cern.ch Notice that the init container writes the `index.html` file in the root directory of the nginx server. @@ -67,16 +67,13 @@ In your shell, send a GET request to the nginx server: The output shows that nginx is serving the web page that was written by the init container: - - +
+ http://info.cern.ch +
- - ... - "url": "http://kubernetes.io/"} - - +

http://info.cern.ch - home of the first website

... -

Kubernetes is open source giving you the freedom to take advantage ...

+
  • Browse the first website
    • ... diff --git a/content/en/docs/tasks/debug-application-cluster/crictl.md b/content/en/docs/tasks/debug-application-cluster/crictl.md index a047f194e9..ef80817082 100644 --- a/content/en/docs/tasks/debug-application-cluster/crictl.md +++ b/content/en/docs/tasks/debug-application-cluster/crictl.md @@ -15,7 +15,7 @@ content_type: task `crictl` is a command-line interface for CRI-compatible container runtimes. You can use it to inspect and debug container runtimes and applications on a Kubernetes node. `crictl` and its source are hosted in the -[cri-tools](https://github.com/kubernetes-incubator/cri-tools) repository. +[cri-tools](https://github.com/kubernetes-sigs/cri-tools) repository. @@ -31,7 +31,7 @@ Kubernetes node. `crictl` and its source are hosted in the ## Installing crictl You can download a compressed archive `crictl` from the cri-tools [release -page](https://github.com/kubernetes-incubator/cri-tools/releases), for several +page](https://github.com/kubernetes-sigs/cri-tools/releases), for several different architectures. Download the version that corresponds to your version of Kubernetes. Extract it and move it to a location on your system path, such as `/usr/local/bin/`. @@ -353,7 +353,7 @@ CONTAINER ID IMAGE CREATED STATE -See [kubernetes-incubator/cri-tools](https://github.com/kubernetes-incubator/cri-tools) +See [kubernetes-sigs/cri-tools](https://github.com/kubernetes-sigs/cri-tools) for more information. diff --git a/content/en/docs/tasks/debug-application-cluster/resource-metrics-pipeline.md b/content/en/docs/tasks/debug-application-cluster/resource-metrics-pipeline.md index f5f2e78ba2..a8c24693dc 100644 --- a/content/en/docs/tasks/debug-application-cluster/resource-metrics-pipeline.md +++ b/content/en/docs/tasks/debug-application-cluster/resource-metrics-pipeline.md @@ -54,7 +54,7 @@ The metric typically also includes some cached (file-backed) memory, because the ## Metrics Server -[Metrics Server](https://github.com/kubernetes-incubator/metrics-server) is a cluster-wide aggregator of resource usage data. +[Metrics Server](https://github.com/kubernetes-sigs/metrics-server) is a cluster-wide aggregator of resource usage data. By default, it is deployed in clusters created by `kube-up.sh` script as a Deployment object. If you use a different Kubernetes setup mechanism, you can deploy it using the provided [deployment components.yaml](https://github.com/kubernetes-sigs/metrics-server/releases) file. diff --git a/content/en/docs/tasks/debug-application-cluster/resource-usage-monitoring.md b/content/en/docs/tasks/debug-application-cluster/resource-usage-monitoring.md index 8255d06afe..957ee62192 100644 --- a/content/en/docs/tasks/debug-application-cluster/resource-usage-monitoring.md +++ b/content/en/docs/tasks/debug-application-cluster/resource-usage-monitoring.md @@ -30,7 +30,7 @@ cluster components such as the [Horizontal Pod Autoscaler](/docs/tasks/run-application/horizontal-pod-autoscale/) controller, as well as the `kubectl top` utility. These metrics are collected by the lightweight, short-term, in-memory -[metrics-server](https://github.com/kubernetes-incubator/metrics-server) and +[metrics-server](https://github.com/kubernetes-sigs/metrics-server) and are exposed via the `metrics.k8s.io` API. metrics-server discovers all nodes on the cluster and diff --git a/content/en/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions.md b/content/en/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions.md index 3b51469e56..110613c200 100644 --- a/content/en/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions.md +++ b/content/en/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions.md @@ -954,15 +954,15 @@ the `/scale` subresource will return an error on GET. - It is a required value. - Only JSONPaths under `.status` and with the dot notation are allowed. - - If there is no value under the `StatusReplicasPath` in the custom resource, + - If there is no value under the `statusReplicasPath` in the custom resource, the status replica value in the `/scale` subresource will default to 0. -- `LabelSelectorPath` defines the JSONPath inside of a custom resource that corresponds to `Scale.Status.Selector`. +- `labelSelectorPath` defines the JSONPath inside of a custom resource that corresponds to `Scale.Status.Selector`. - It is an optional value. - It must be set to work with HPA. - Only JSONPaths under `.status` or `.spec` and with the dot notation are allowed. - - If there is no value under the `LabelSelectorPath` in the custom resource, + - If there is no value under the `labelSelectorPath` in the custom resource, the status selector value in the `/scale` subresource will default to the empty string. - The field pointed by this JSON path must be a string field (not a complex selector struct) which contains a serialized label selector in string form. diff --git a/content/en/docs/tasks/inject-data-application/podpreset.md b/content/en/docs/tasks/inject-data-application/podpreset.md index 9eea082321..91c3c7beb6 100644 --- a/content/en/docs/tasks/inject-data-application/podpreset.md +++ b/content/en/docs/tasks/inject-data-application/podpreset.md @@ -18,7 +18,7 @@ This page shows how to use PodPreset objects to inject information like {{< glos ## {{% heading "prerequisites" %}} -You need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one using [Minikube](/docs/setup/learning-environment/minikube/). +You need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one using [Minikube](https://minikube.sigs.k8s.io/docs/). Make sure that you have [enabled PodPreset](/docs/concepts/workloads/pods/podpreset/#enable-pod-preset) in your cluster. diff --git a/content/en/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough.md b/content/en/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough.md index a89fb778a9..c840034f02 100644 --- a/content/en/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough.md +++ b/content/en/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough.md @@ -23,8 +23,8 @@ For more information on how Horizontal Pod Autoscaler behaves, see the This example requires a running Kubernetes cluster and kubectl, version 1.2 or later. -[metrics-server](https://github.com/kubernetes-incubator/metrics-server/) monitoring needs to be deployed in the cluster -to provide metrics via the resource metrics API, as Horizontal Pod Autoscaler uses this API to collect metrics. The instructions for deploying this are on the GitHub repository of [metrics-server](https://github.com/kubernetes-incubator/metrics-server/), if you followed [getting started on GCE guide](/docs/setup/production-environment/turnkey/gce/), +[metrics-server](https://github.com/kubernetes-sigs/metrics-server) monitoring needs to be deployed in the cluster +to provide metrics via the resource metrics API, as Horizontal Pod Autoscaler uses this API to collect metrics. The instructions for deploying this are on the GitHub repository of [metrics-server](https://github.com/kubernetes-sigs/metrics-server), if you followed [getting started on GCE guide](/docs/setup/production-environment/turnkey/gce/), metrics-server monitoring will be turned-on by default. To specify multiple resource metrics for a Horizontal Pod Autoscaler, you must have a Kubernetes cluster diff --git a/content/en/docs/tasks/run-application/horizontal-pod-autoscale.md b/content/en/docs/tasks/run-application/horizontal-pod-autoscale.md index 1da20c5219..927c146cf5 100644 --- a/content/en/docs/tasks/run-application/horizontal-pod-autoscale.md +++ b/content/en/docs/tasks/run-application/horizontal-pod-autoscale.md @@ -265,12 +265,12 @@ APIs, cluster administrators must ensure that: * The corresponding APIs are registered: - * For resource metrics, this is the `metrics.k8s.io` API, generally provided by [metrics-server](https://github.com/kubernetes-incubator/metrics-server). + * For resource metrics, this is the `metrics.k8s.io` API, generally provided by [metrics-server](https://github.com/kubernetes-sigs/metrics-server). It can be launched as a cluster addon. * For custom metrics, this is the `custom.metrics.k8s.io` API. It's provided by "adapter" API servers provided by metrics solution vendors. Check with your metrics pipeline, or the [list of known solutions](https://github.com/kubernetes/metrics/blob/master/IMPLEMENTATIONS.md#custom-metrics-api). - If you would like to write your own, check out the [boilerplate](https://github.com/kubernetes-incubator/custom-metrics-apiserver) to get started. + If you would like to write your own, check out the [boilerplate](https://github.com/kubernetes-sigs/custom-metrics-apiserver) to get started. * For external metrics, this is the `external.metrics.k8s.io` API. It may be provided by the custom metrics adapters provided above. diff --git a/content/en/docs/tasks/service-catalog/install-service-catalog-using-sc.md b/content/en/docs/tasks/service-catalog/install-service-catalog-using-sc.md index 937f1170a7..52a55457a2 100644 --- a/content/en/docs/tasks/service-catalog/install-service-catalog-using-sc.md +++ b/content/en/docs/tasks/service-catalog/install-service-catalog-using-sc.md @@ -78,6 +78,6 @@ sc uninstall ## {{% heading "whatsnext" %}} * View [sample service brokers](https://github.com/openservicebrokerapi/servicebroker/blob/master/gettingStarted.md#sample-service-brokers). -* Explore the [kubernetes-incubator/service-catalog](https://github.com/kubernetes-incubator/service-catalog) project. +* Explore the [kubernetes-sigs/service-catalog](https://github.com/kubernetes-sigs/service-catalog) project. diff --git a/content/en/docs/tasks/tools/_index.md b/content/en/docs/tasks/tools/_index.md index dfdbab3fb1..a705f9f52e 100755 --- a/content/en/docs/tasks/tools/_index.md +++ b/content/en/docs/tasks/tools/_index.md @@ -20,9 +20,20 @@ accessing your cluster. You can also read the [`kubectl` reference documentation](/docs/reference/kubectl/). +## kind + +[`kind`](https://kind.sigs.k8s.io/docs/) lets you run Kubernetes on +your local computer. This tool it requires that you have +[Docker](https://docs.docker.com/get-docker/) installed and configured. + +The kind [Quick Start](https://kind.sigs.k8s.io/docs/user/quick-start/) page +shows you what you need to do to get up and running with kind. + +View kind Quick Start Guide + ## minikube -[`minikube`](https://minikube.sigs.k8s.io/) is a tool that lets you run Kubernetes +Like `kind`, [`minikube`](https://minikube.sigs.k8s.io/) is a tool that lets you run Kubernetes locally. `minikube` runs a single-node Kubernetes cluster on your personal computer (including Windows, macOS and Linux PCs) so that you can try out Kubernetes, or for daily development work. @@ -36,14 +47,12 @@ on getting the tool installed. Once you have `minikube` working, you can use it to [run a sample application](/docs/tutorials/hello-minikube/). -## kind +## kubeadm -Like `minikube`, [`kind`](https://kind.sigs.k8s.io/docs/) lets you run Kubernetes on -your local computer. Unlike `minikube`, `kind` only works with a single container -runtime: it requires that you have [Docker](https://docs.docker.com/get-docker/) -installed and configured. +You can use the {{< glossary_tooltip term_id="kubeadm" text="kubeadm" >}} tool to create and manage Kubernetes clusters. +It performs the actions necessary to get a minimum viable, secure cluster up and running in a user friendly way. -[Quick Start](https://kind.sigs.k8s.io/docs/user/quick-start/) shows you what -you need to do to get up and running with `kind`. +[Installing kubeadm](/docs/setup/production-environment/tools/kubeadm/install-kubeadm/) shows you how to install kubeadm. +Once installed, you can use it to [create a cluster](/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/). -View kind Quick Start Guide +View kubeadm Install Guide diff --git a/content/en/docs/tasks/tools/install-kubectl.md b/content/en/docs/tasks/tools/install-kubectl.md index b6b3ad35e9..5bfebd226b 100644 --- a/content/en/docs/tasks/tools/install-kubectl.md +++ b/content/en/docs/tasks/tools/install-kubectl.md @@ -525,7 +525,7 @@ compinit ## {{% heading "whatsnext" %}} -* [Install Minikube](/docs/tasks/tools/install-minikube/) +* [Install Minikube](https://minikube.sigs.k8s.io/docs/start/) * See the [getting started guides](/docs/setup/) for more about creating clusters. * [Learn how to launch and expose your application.](/docs/tasks/access-application-cluster/service-access-application-cluster/) * If you need access to a cluster you didn't create, see the diff --git a/content/en/docs/tutorials/clusters/seccomp.md b/content/en/docs/tutorials/clusters/seccomp.md index 41c0937420..adb3d9c500 100644 --- a/content/en/docs/tutorials/clusters/seccomp.md +++ b/content/en/docs/tutorials/clusters/seccomp.md @@ -37,7 +37,7 @@ profiles that give only the necessary privileges to your container processes. In order to complete all steps in this tutorial, you must install [kind](https://kind.sigs.k8s.io/docs/user/quick-start/) and -[kubectl](/doc/tasks/tools/install-kubectl/). This tutorial will show examples +[kubectl](/docs/tasks/tools/install-kubectl/). This tutorial will show examples with both alpha (pre-v1.19) and generally available seccomp functionality, so make sure that your cluster is [configured correctly](https://kind.sigs.k8s.io/docs/user/quick-start/#setting-kubernetes-version) @@ -365,4 +365,4 @@ The default seccomp profile should provide adequate access for most workloads. Additional resources: * [A Seccomp Overview](https://lwn.net/Articles/656307/) -* [Seccomp Security Profiles for Docker](https://docs.docker.com/engine/security/seccomp/) \ No newline at end of file +* [Seccomp Security Profiles for Docker](https://docs.docker.com/engine/security/seccomp/) diff --git a/content/en/docs/tutorials/configuration/configure-java-microservice/configure-java-microservice.md b/content/en/docs/tutorials/configuration/configure-java-microservice/configure-java-microservice.md index 712bc64d55..dccf214c47 100644 --- a/content/en/docs/tutorials/configuration/configure-java-microservice/configure-java-microservice.md +++ b/content/en/docs/tutorials/configuration/configure-java-microservice/configure-java-microservice.md @@ -14,9 +14,9 @@ In this tutorial you will learn how and why to externalize your microservice’s ### Creating Kubernetes ConfigMaps & Secrets There are several ways to set environment variables for a Docker container in Kubernetes, including: Dockerfile, kubernetes.yml, Kubernetes ConfigMaps, and Kubernetes Secrets. In the tutorial, you will learn how to use the latter two for setting your environment variables whose values will be injected into your microservices. One of the benefits for using ConfigMaps and Secrets is that they can be re-used across multiple containers, including being assigned to different environment variables for the different containers. -ConfigMaps are API Objects that store non-confidential key-value pairs. In the Interactive Tutorial you will learn how to use a ConfigMap to store the application's name. For more information regarding ConfigMaps, you can find the documentation [here](https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/). +ConfigMaps are API Objects that store non-confidential key-value pairs. In the Interactive Tutorial you will learn how to use a ConfigMap to store the application's name. For more information regarding ConfigMaps, you can find the documentation [here](/docs/tasks/configure-pod-container/configure-pod-configmap/). -Although Secrets are also used to store key-value pairs, they differ from ConfigMaps in that they're intended for confidential/sensitive information and are stored using Base64 encoding. This makes secrets the appropriate choice for storing such things as credentials, keys, and tokens, the former of which you'll do in the Interactive Tutorial. For more information on Secrets, you can find the documentation [here](https://kubernetes.io/docs/concepts/configuration/secret/). +Although Secrets are also used to store key-value pairs, they differ from ConfigMaps in that they're intended for confidential/sensitive information and are stored using Base64 encoding. This makes secrets the appropriate choice for storing such things as credentials, keys, and tokens, the former of which you'll do in the Interactive Tutorial. For more information on Secrets, you can find the documentation [here](/docs/concepts/configuration/secret/). ### Externalizing Config from Code @@ -36,4 +36,4 @@ Many open source frameworks and runtimes implement and support MicroProfile Conf ## Example: Externalizing config using MicroProfile, ConfigMaps and Secrets -### [Start Interactive Tutorial](/docs/tutorials/configuration/configure-java-microservice/configure-java-microservice-interactive/) \ No newline at end of file +### [Start Interactive Tutorial](/docs/tutorials/configuration/configure-java-microservice/configure-java-microservice-interactive/) diff --git a/content/en/docs/tutorials/hello-minikube.md b/content/en/docs/tutorials/hello-minikube.md index 6cc03c2198..ba115b601c 100644 --- a/content/en/docs/tutorials/hello-minikube.md +++ b/content/en/docs/tutorials/hello-minikube.md @@ -16,24 +16,20 @@ card: This tutorial shows you how to run a sample app -on Kubernetes using [Minikube](/docs/setup/learning-environment/minikube) and Katacoda. +on Kubernetes using minikube and Katacoda. Katacoda provides a free, in-browser Kubernetes environment. {{< note >}} -You can also follow this tutorial if you've installed [Minikube locally](/docs/tasks/tools/install-minikube/). +You can also follow this tutorial if you've installed minikube locally. +See [minikube start](https://minikube.sigs.k8s.io/docs/start/) for installation instructions. {{< /note >}} - - ## {{% heading "objectives" %}} - -* Deploy a sample application to Minikube. +* Deploy a sample application to minikube. * Run the app. * View application logs. - - ## {{% heading "prerequisites" %}} @@ -43,14 +39,14 @@ This tutorial provides a container image that uses NGINX to echo back all the re -## Create a Minikube cluster +## Create a minikube cluster 1. Click **Launch Terminal** {{< kat-button >}} {{< note >}} - If you installed Minikube locally, run `minikube start`. +If you installed minikube locally, run `minikube start`. {{< /note >}} 2. Open the Kubernetes dashboard in a browser: @@ -155,7 +151,7 @@ Kubernetes [*Service*](/docs/concepts/services-networking/service/). ``` On cloud providers that support load balancers, - an external IP address would be provisioned to access the Service. On Minikube, + an external IP address would be provisioned to access the Service. On minikube, the `LoadBalancer` type makes the Service accessible through the `minikube service` command. @@ -173,7 +169,7 @@ Kubernetes [*Service*](/docs/concepts/services-networking/service/). ## Enable addons -Minikube has a set of built-in {{< glossary_tooltip text="addons" term_id="addons" >}} that can be enabled, disabled and opened in the local Kubernetes environment. +The minikube tool includes a set of built-in {{< glossary_tooltip text="addons" term_id="addons" >}} that can be enabled, disabled and opened in the local Kubernetes environment. 1. List the currently supported addons: diff --git a/content/en/docs/tutorials/stateful-application/cassandra.md b/content/en/docs/tutorials/stateful-application/cassandra.md index 6eb5e15dae..5b453f6594 100644 --- a/content/en/docs/tutorials/stateful-application/cassandra.md +++ b/content/en/docs/tutorials/stateful-application/cassandra.md @@ -50,7 +50,7 @@ To complete this tutorial, you should already have a basic familiarity with ### Additional Minikube setup instructions {{< caution >}} -[Minikube](/docs/setup/learning-environment/minikube/) defaults to 1024MiB of memory and 1 CPU. +[Minikube](https://minikube.sigs.k8s.io/docs/) defaults to 1024MiB of memory and 1 CPU. Running Minikube with the default resource configuration results in insufficient resource errors during this tutorial. To avoid these errors, start Minikube with the following settings: diff --git a/content/en/docs/tutorials/stateless-application/guestbook-logs-metrics-with-elk.md b/content/en/docs/tutorials/stateless-application/guestbook-logs-metrics-with-elk.md index de67b9eab3..d3a38c4df5 100644 --- a/content/en/docs/tutorials/stateless-application/guestbook-logs-metrics-with-elk.md +++ b/content/en/docs/tutorials/stateless-application/guestbook-logs-metrics-with-elk.md @@ -19,17 +19,14 @@ This tutorial builds upon the [PHP Guestbook with Redis](/docs/tutorials/statele * Metricbeat * Packetbeat - - ## {{% heading "objectives" %}} * Start up the PHP Guestbook with Redis. * Install kube-state-metrics. -* Create a Kubernetes secret. +* Create a Kubernetes Secret. * Deploy the Beats. * View dashboards of your logs and metrics. - ## {{% heading "prerequisites" %}} @@ -40,16 +37,18 @@ Additionally you need: * A running deployment of the [PHP Guestbook with Redis](/docs/tutorials/stateless-application/guestbook) tutorial. -* A running Elasticsearch and Kibana deployment. You can use [Elasticsearch Service in Elastic Cloud](https://cloud.elastic.co), run the [download files](https://www.elastic.co/guide/en/elastic-stack-get-started/current/get-started-elastic-stack.html) on your workstation or servers, or the [Elastic Helm Charts](https://github.com/elastic/helm-charts). - - +* A running Elasticsearch and Kibana deployment. You can use [Elasticsearch Service in Elastic Cloud](https://cloud.elastic.co), + run the [downloaded files](https://www.elastic.co/guide/en/elastic-stack-get-started/current/get-started-elastic-stack.html) + on your workstation or servers, or the [Elastic Helm Charts](https://github.com/elastic/helm-charts). ## Start up the PHP Guestbook with Redis + This tutorial builds on the [PHP Guestbook with Redis](/docs/tutorials/stateless-application/guestbook) tutorial. If you have the guestbook application running, then you can monitor that. If you do not have it running then follow the instructions to deploy the guestbook and do not perform the **Cleanup** steps. Come back to this page when you have the guestbook running. ## Add a Cluster role binding + Create a [cluster level role binding](/docs/reference/access-authn-authz/rbac/#rolebinding-and-clusterrolebinding) so that you can deploy kube-state-metrics and the Beats at the cluster level (in kube-system). ```shell @@ -60,31 +59,39 @@ kubectl create clusterrolebinding cluster-admin-binding \ ## Install kube-state-metrics Kubernetes [*kube-state-metrics*](https://github.com/kubernetes/kube-state-metrics) is a simple service that listens to the Kubernetes API server and generates metrics about the state of the objects. Metricbeat reports these metrics. Add kube-state-metrics to the Kubernetes cluster that the guestbook is running in. + ```shell git clone https://github.com/kubernetes/kube-state-metrics.git kube-state-metrics kubectl apply -f kube-state-metrics/examples/standard ``` ### Check to see if kube-state-metrics is running + ```shell kubectl get pods --namespace=kube-system -l app.kubernetes.io/name=kube-state-metrics ``` + Output: -```shell + +``` NAME READY STATUS RESTARTS AGE kube-state-metrics-89d656bf8-vdthm 1/1 Running 0 21s ``` + ## Clone the Elastic examples GitHub repo + ```shell git clone https://github.com/elastic/examples.git ``` The rest of the commands will reference files in the `examples/beats-k8s-send-anywhere` directory, so change dir there: + ```shell cd examples/beats-k8s-send-anywhere ``` ## Create a Kubernetes Secret + A Kubernetes {{< glossary_tooltip text="Secret" term_id="secret" >}} is an object that contains a small amount of sensitive data such as a password, a token, or a key. Such information might otherwise be put in a Pod specification or in an image; putting it in a Secret object allows for more control over how it is used, and reduces the risk of accidental exposure. {{< note >}} @@ -95,55 +102,70 @@ There are two sets of steps here, one for *self managed* Elasticsearch and Kiban {{% tab name="Self Managed" %}} ### Self managed + Switch to the **Managed service** tab if you are connecting to Elasticsearch Service in Elastic Cloud. ### Set the credentials + There are four files to edit to create a k8s secret when you are connecting to self managed Elasticsearch and Kibana (self managed is effectively anything other than the managed Elasticsearch Service in Elastic Cloud). The files are: -1. ELASTICSEARCH_HOSTS -1. ELASTICSEARCH_PASSWORD -1. ELASTICSEARCH_USERNAME -1. KIBANA_HOST +1. `ELASTICSEARCH_HOSTS` +1. `ELASTICSEARCH_PASSWORD` +1. `ELASTICSEARCH_USERNAME` +1. `KIBANA_HOST` Set these with the information for your Elasticsearch cluster and your Kibana host. Here are some examples (also see [*this configuration*](https://stackoverflow.com/questions/59892896/how-to-connect-from-minikube-to-elasticsearch-installed-on-host-local-developme/59892897#59892897)) #### `ELASTICSEARCH_HOSTS` + 1. A nodeGroup from the Elastic Elasticsearch Helm Chart: - ```shell - ["http://elasticsearch-master.default.svc.cluster.local:9200"] - ``` + ``` + ["http://elasticsearch-master.default.svc.cluster.local:9200"] + ``` + 1. A single Elasticsearch node running on a Mac where your Beats are running in Docker for Mac: - ```shell - ["http://host.docker.internal:9200"] - ``` + ``` + ["http://host.docker.internal:9200"] + ``` + 1. Two Elasticsearch nodes running in VMs or on physical hardware: - ```shell - ["http://host1.example.com:9200", "http://host2.example.com:9200"] - ``` -Edit `ELASTICSEARCH_HOSTS` + ``` + ["http://host1.example.com:9200", "http://host2.example.com:9200"] + ``` + +Edit `ELASTICSEARCH_HOSTS`: + ```shell vi ELASTICSEARCH_HOSTS ``` #### `ELASTICSEARCH_PASSWORD` -Just the password; no whitespace, quotes, or <>: - +Just the password; no whitespace, quotes, `<` or `>`: + +``` + +``` + +Edit `ELASTICSEARCH_PASSWORD`: -Edit `ELASTICSEARCH_PASSWORD` ```shell vi ELASTICSEARCH_PASSWORD ``` #### `ELASTICSEARCH_USERNAME` -Just the username; no whitespace, quotes, or <>: - +Just the username; no whitespace, quotes, `<` or `>`: + +``` + +``` + +Edit `ELASTICSEARCH_USERNAME`: -Edit `ELASTICSEARCH_USERNAME` ```shell vi ELASTICSEARCH_USERNAME ``` @@ -152,78 +174,98 @@ vi ELASTICSEARCH_USERNAME 1. The Kibana instance from the Elastic Kibana Helm Chart. The subdomain `default` refers to the default namespace. If you have deployed the Helm Chart using a different namespace, then your subdomain will be different: - ```shell - "kibana-kibana.default.svc.cluster.local:5601" - ``` + ``` + "kibana-kibana.default.svc.cluster.local:5601" + ``` + 1. A Kibana instance running on a Mac where your Beats are running in Docker for Mac: - ```shell - "host.docker.internal:5601" - ``` + ``` + "host.docker.internal:5601" + ``` 1. Two Elasticsearch nodes running in VMs or on physical hardware: - ```shell - "host1.example.com:5601" - ``` -Edit `KIBANA_HOST` + ``` + "host1.example.com:5601" + ``` + +Edit `KIBANA_HOST`: + ```shell vi KIBANA_HOST ``` -### Create a Kubernetes secret -This command creates a secret in the Kubernetes system level namespace (kube-system) based on the files you just edited: +### Create a Kubernetes Secret - kubectl create secret generic dynamic-logging \ - --from-file=./ELASTICSEARCH_HOSTS \ - --from-file=./ELASTICSEARCH_PASSWORD \ - --from-file=./ELASTICSEARCH_USERNAME \ - --from-file=./KIBANA_HOST \ - --namespace=kube-system +This command creates a Secret in the Kubernetes system level namespace (`kube-system`) based on the files you just edited: + +```shell +kubectl create secret generic dynamic-logging \ + --from-file=./ELASTICSEARCH_HOSTS \ + --from-file=./ELASTICSEARCH_PASSWORD \ + --from-file=./ELASTICSEARCH_USERNAME \ + --from-file=./KIBANA_HOST \ + --namespace=kube-system +``` {{% /tab %}} {{% tab name="Managed service" %}} ## Managed service -This tab is for Elasticsearch Service in Elastic Cloud only, if you have already created a secret for a self managed Elasticsearch and Kibana deployment, then continue with [Deploy the Beats](#deploy-the-beats). -### Set the credentials -There are two files to edit to create a k8s secret when you are connecting to the managed Elasticsearch Service in Elastic Cloud. The files are: -1. ELASTIC_CLOUD_AUTH -1. ELASTIC_CLOUD_ID +This tab is for Elasticsearch Service in Elastic Cloud only, if you have already created a secret for a self managed Elasticsearch and Kibana deployment, then continue with [Deploy the Beats](#deploy-the-beats). + +### Set the credentials + +There are two files to edit to create a Kubernetes Secret when you are connecting to the managed Elasticsearch Service in Elastic Cloud. The files are: + +1. `ELASTIC_CLOUD_AUTH` +1. `ELASTIC_CLOUD_ID` Set these with the information provided to you from the Elasticsearch Service console when you created the deployment. Here are some examples: -#### ELASTIC_CLOUD_ID -```shell +#### `ELASTIC_CLOUD_ID` + +``` devk8s:ABC123def456ghi789jkl123mno456pqr789stu123vwx456yza789bcd012efg345hijj678klm901nop345zEwOTJjMTc5YWQ0YzQ5OThlN2U5MjAwYTg4NTIzZQ== ``` -#### ELASTIC_CLOUD_AUTH +#### `ELASTIC_CLOUD_AUTH` + Just the username, a colon (`:`), and the password, no whitespace or quotes: -```shell + +``` elastic:VFxJJf9Tjwer90wnfTghsn8w ``` ### Edit the required files: + ```shell vi ELASTIC_CLOUD_ID vi ELASTIC_CLOUD_AUTH ``` -### Create a Kubernetes secret -This command creates a secret in the Kubernetes system level namespace (kube-system) based on the files you just edited: - kubectl create secret generic dynamic-logging \ - --from-file=./ELASTIC_CLOUD_ID \ - --from-file=./ELASTIC_CLOUD_AUTH \ - --namespace=kube-system +### Create a Kubernetes Secret + +This command creates a Secret in the Kubernetes system level namespace (`kube-system`) based on the files you just edited: + +```shell +kubectl create secret generic dynamic-logging \ + --from-file=./ELASTIC_CLOUD_ID \ + --from-file=./ELASTIC_CLOUD_AUTH \ + --namespace=kube-system +``` + +{{% /tab %}} - {{% /tab %}} {{< /tabs >}} ## Deploy the Beats + Manifest files are provided for each Beat. These manifest files use the secret created earlier to configure the Beats to connect to your Elasticsearch and Kibana servers. ### About Filebeat + Filebeat will collect logs from the Kubernetes nodes and the containers running in each pod running on those nodes. Filebeat is deployed as a {{< glossary_tooltip text="DaemonSet" term_id="daemonset" >}}. Filebeat can autodiscover applications running in your Kubernetes cluster. At startup Filebeat scans existing containers and launches the proper configurations for them, then it will watch for new start/stop events. Here is the autodiscover configuration that enables Filebeat to locate and parse Redis logs from the Redis containers deployed with the guestbook application. This configuration is in the file `filebeat-kubernetes.yaml`: @@ -242,20 +284,25 @@ Here is the autodiscover configuration that enables Filebeat to locate and parse enabled: true var.hosts: ["${data.host}:${data.port}"] ``` + This configures Filebeat to apply the Filebeat module `redis` when a container is detected with a label `app` containing the string `redis`. The redis module has the ability to collect the `log` stream from the container by using the docker input type (reading the file on the Kubernetes node associated with the STDOUT stream from this Redis container). Additionally, the module has the ability to collect Redis `slowlog` entries by connecting to the proper pod host and port, which is provided in the container metadata. ### Deploy Filebeat: + ```shell kubectl create -f filebeat-kubernetes.yaml ``` #### Verify + ```shell kubectl get pods -n kube-system -l k8s-app=filebeat-dynamic ``` ### About Metricbeat + Metricbeat autodiscover is configured in the same way as Filebeat. Here is the Metricbeat autodiscover configuration for the Redis containers. This configuration is in the file `metricbeat-kubernetes.yaml`: + ```yaml - condition.equals: kubernetes.labels.tier: backend @@ -267,22 +314,27 @@ Metricbeat autodiscover is configured in the same way as Filebeat. Here is the # Redis hosts hosts: ["${data.host}:${data.port}"] ``` + This configures Metricbeat to apply the Metricbeat module `redis` when a container is detected with a label `tier` equal to the string `backend`. The `redis` module has the ability to collect the `info` and `keyspace` metrics from the container by connecting to the proper pod host and port, which is provided in the container metadata. ### Deploy Metricbeat + ```shell kubectl create -f metricbeat-kubernetes.yaml ``` + #### Verify + ```shell kubectl get pods -n kube-system -l k8s-app=metricbeat ``` ### About Packetbeat + Packetbeat configuration is different than Filebeat and Metricbeat. Rather than specify patterns to match against container labels the configuration is based on the protocols and port numbers involved. Shown below is a subset of the port numbers. {{< note >}} -If you are running a service on a non-standard port add that port number to the appropriate type in `filebeat.yaml` and delete / create the Packetbeat DaemonSet. +If you are running a service on a non-standard port add that port number to the appropriate type in `filebeat.yaml` and delete/create the Packetbeat DaemonSet. {{< /note >}} ```yaml @@ -309,11 +361,13 @@ packetbeat.flows: ``` #### Deploy Packetbeat + ```shell kubectl create -f packetbeat-kubernetes.yaml ``` #### Verify + ```shell kubectl get pods -n kube-system -l k8s-app=packetbeat-dynamic ``` @@ -328,15 +382,17 @@ Similarly, view dashboards for Apache and Redis. You will see dashboards for lo To enable Metricbeat to retrieve the Apache metrics, enable server-status by adding a ConfigMap including a mod-status configuration file and re-deploy the guestbook. +## Scale your Deployments and see new pods being monitored + +List the existing Deployments: -## Scale your deployments and see new pods being monitored -List the existing deployments: ```shell kubectl get deployments ``` The output: -```shell + +``` NAME READY UP-TO-DATE AVAILABLE AGE frontend 3/3 3 3 3h27m redis-master 1/1 1 1 3h27m @@ -344,54 +400,56 @@ redis-slave 2/2 2 2 3h27m ``` Scale the frontend down to two pods: + ```shell kubectl scale --replicas=2 deployment/frontend ``` + The output: -```shell + +``` deployment.extensions/frontend scaled ``` + Scale the frontend back up to three pods: + ```shell kubectl scale --replicas=3 deployment/frontend ``` ## View the changes in Kibana + See the screenshot, add the indicated filters and then add the columns to the view. You can see the ScalingReplicaSet entry that is marked, following from there to the top of the list of events shows the image being pulled, the volumes mounted, the pod starting, etc. ![Kibana Discover](https://raw.githubusercontent.com/elastic/examples/master/beats-k8s-send-anywhere/scaling-up.png) - - ## {{% heading "cleanup" %}} Deleting the Deployments and Services also deletes any running Pods. Use labels to delete multiple resources with one command. 1. Run the following commands to delete all Pods, Deployments, and Services. - ```shell - kubectl delete deployment -l app=redis - kubectl delete service -l app=redis - kubectl delete deployment -l app=guestbook - kubectl delete service -l app=guestbook - kubectl delete -f filebeat-kubernetes.yaml - kubectl delete -f metricbeat-kubernetes.yaml - kubectl delete -f packetbeat-kubernetes.yaml - kubectl delete secret dynamic-logging -n kube-system - ``` + ```shell + kubectl delete deployment -l app=redis + kubectl delete service -l app=redis + kubectl delete deployment -l app=guestbook + kubectl delete service -l app=guestbook + kubectl delete -f filebeat-kubernetes.yaml + kubectl delete -f metricbeat-kubernetes.yaml + kubectl delete -f packetbeat-kubernetes.yaml + kubectl delete secret dynamic-logging -n kube-system + ``` 1. Query the list of Pods to verify that no Pods are running: - ```shell - kubectl get pods - ``` - - The response should be this: - - ``` - No resources found. - ``` + ```shell + kubectl get pods + ``` + The response should be this: + ``` + No resources found. + ``` ## {{% heading "whatsnext" %}} @@ -399,3 +457,4 @@ Deleting the Deployments and Services also deletes any running Pods. Use labels * Read more about [logging architecture](/docs/concepts/cluster-administration/logging/) * Read more about [application introspection and debugging](/docs/tasks/debug-application-cluster/) * Read more about [troubleshoot applications](/docs/tasks/debug-application-cluster/resource-usage-monitoring/) + diff --git a/content/en/examples/access/certificate-signing-request/clusterrole-approve.yaml b/content/en/examples/access/certificate-signing-request/clusterrole-approve.yaml new file mode 100644 index 0000000000..e5bcfdc44e --- /dev/null +++ b/content/en/examples/access/certificate-signing-request/clusterrole-approve.yaml @@ -0,0 +1,28 @@ +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: csr-approver +rules: +- apiGroups: + - certificates.k8s.io + resources: + - certificatesigningrequests + verbs: + - get + - list + - watch +- apiGroups: + - certificates.k8s.io + resources: + - certificatesigningrequests/approval + verbs: + - update +- apiGroups: + - certificates.k8s.io + resources: + - signers + resourceNames: + - example.com/my-signer-name # example.com/* can be used to authorize for all signers in the 'example.com' domain + verbs: + - approve + diff --git a/content/en/examples/access/certificate-signing-request/clusterrole-create.yaml b/content/en/examples/access/certificate-signing-request/clusterrole-create.yaml new file mode 100644 index 0000000000..def1b879d8 --- /dev/null +++ b/content/en/examples/access/certificate-signing-request/clusterrole-create.yaml @@ -0,0 +1,14 @@ +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: csr-creator +rules: +- apiGroups: + - certificates.k8s.io + resources: + - certificatesigningrequests + verbs: + - create + - get + - list + - watch diff --git a/content/en/examples/access/certificate-signing-request/clusterrole-sign.yaml b/content/en/examples/access/certificate-signing-request/clusterrole-sign.yaml new file mode 100644 index 0000000000..29bbc6a9cd --- /dev/null +++ b/content/en/examples/access/certificate-signing-request/clusterrole-sign.yaml @@ -0,0 +1,27 @@ +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: csr-signer +rules: +- apiGroups: + - certificates.k8s.io + resources: + - certificatesigningrequests + verbs: + - get + - list + - watch +- apiGroups: + - certificates.k8s.io + resources: + - certificatesigningrequests/status + verbs: + - update +- apiGroups: + - certificates.k8s.io + resources: + - signers + resourceName: + - example.com/my-signer-name # example.com/* can be used to authorize for all signers in the 'example.com' domain + verbs: + - sign diff --git a/content/en/examples/pods/init-containers.yaml b/content/en/examples/pods/init-containers.yaml index 35c393d7c7..667b03eccd 100644 --- a/content/en/examples/pods/init-containers.yaml +++ b/content/en/examples/pods/init-containers.yaml @@ -19,7 +19,7 @@ spec: - wget - "-O" - "/work-dir/index.html" - - http://kubernetes.io + - http://info.cern.ch volumeMounts: - name: workdir mountPath: "/work-dir" diff --git a/content/en/includes/task-tutorial-prereqs.md b/content/en/includes/task-tutorial-prereqs.md index d4546bdf62..93195d8b9c 100644 --- a/content/en/includes/task-tutorial-prereqs.md +++ b/content/en/includes/task-tutorial-prereqs.md @@ -1,7 +1,7 @@ You need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using -[Minikube](/docs/setup/learning-environment/minikube/), +[minikube](/docs/tasks/tools/#minikube) or you can use one of these Kubernetes playgrounds: * [Katacoda](https://www.katacoda.com/courses/kubernetes/playground) diff --git a/content/es/docs/concepts/workloads/controllers/deployment.md b/content/es/docs/concepts/workloads/controllers/deployment.md index 89563b3b72..17a42882a0 100644 --- a/content/es/docs/concepts/workloads/controllers/deployment.md +++ b/content/es/docs/concepts/workloads/controllers/deployment.md @@ -109,7 +109,7 @@ Para ver el estado del Deployment, ejecuta el comando `kubectl rollout status de ```shell Waiting for rollout to finish: 2 out of 3 new replicas have been updated... -deployment.apps/nginx-deployment successfully rolled out +deployment "nginx-deployment" successfully rolled out ``` Ejecuta de nuevo el comando `kubectl get deployments` unos segundos más tarde: @@ -195,7 +195,7 @@ kubectl rollout status deployment.v1.apps/nginx-deployment ``` ``` Waiting for rollout to finish: 2 out of 3 new replicas have been updated... -deployment.apps/nginx-deployment successfully rolled out +deployment "nginx-deployment" successfully rolled out ``` Cuando el despliegue funciona, puede que quieras `obtener` el Deployment: @@ -803,7 +803,7 @@ kubectl rollout status deployment.v1.apps/nginx-deployment ``` ``` Waiting for rollout to finish: 2 of 3 updated replicas are available... -deployment.apps/nginx-deployment successfully rolled out +deployment "nginx-deployment" successfully rolled out $ echo $? 0 ``` diff --git a/content/es/docs/concepts/workloads/pods/podpreset.md b/content/es/docs/concepts/workloads/pods/podpreset.md index 87cd6bb83f..d4406f56ea 100644 --- a/content/es/docs/concepts/workloads/pods/podpreset.md +++ b/content/es/docs/concepts/workloads/pods/podpreset.md @@ -7,7 +7,9 @@ weight: 50 --- -Esta página provee una descripción general de los Pod Presets, los cuales son +{{< feature-state for_k8s_version="v1.6" state="alpha" >}} + +Esta página provee una descripción general de los PodPresets, los cuales son los objetos que se utilizan para inyectar cierta información en los Pods en el momento de la creación. Esta información puede incluir secretos, volúmenes, montajes de volúmenes y variables de entorno. @@ -17,16 +19,35 @@ montajes de volúmenes y variables de entorno. ## Entendiendo los Pod Presets -Un `Pod Preset` es un recurso de la API utilizado para poder inyectar requerimientos +Un PodPreset es un recurso de la API utilizado para poder inyectar requerimientos adicionales de tiempo de ejecución en un Pod en el momento de la creación. Se utilizan los [selectores de etiquetas](/docs/concepts/overview/working-with-objects/labels/#label-selectors) -para especificar los Pods a los que se aplica un Pod Preset determinado. +para especificar los Pods a los que se aplica un PodPreset determinado. -El uso de un Pod Preset permite a los autores de plantillas de Pods no tener que proporcionar +El uso de un PodPreset permite a los autores de plantillas de Pods no tener que proporcionar explícitamente toda la información de cada Pod. De esta manera, los autores de plantillas de Pods que consuman un determinado servicio no tendrán que conocer todos los detalles de ese servicio. -Para más información sobre los detalles de los trasfondos, consulte la [propuesta de diseño de PodPreset](https://git.k8s.io/community/contributors/design-proposals/service-catalog/pod-preset.md). + +## Habilitando un PodPreset en su clúster + +Con el fin de utilizar los Pod Presets en un clúster debe asegurarse de lo siguiente: + +1. Que se ha configurado el tipo de API `settings.k8s.io/v1alpha1/podpreset`. Esto se puede hacer, + por ejemplo, incluyendo `settings.k8s.io/v1alpha1=true` como valor de la opción `--runtime-config` + en el servidor API. En minikube se debe añadir el flag + `--extra-config=apiserver.runtime-config=settings.k8s.io/v1alpha1=true` cuando el clúster + se está iniciando. +2. Que se ha habilitado el controlador de admisión `PodPreset`. Una forma de hacer esto es incluir + `PodPreset` como valor de la opción `--enable-admission-plugins` especificada + para el servidor API. En minikube se debe añadir el flag + + ```shell + --extra-config=apiserver.enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota,PodPreset + ``` + + cuando el clúster se está iniciando. + ## Cómo funciona @@ -46,51 +67,29 @@ Cuando se realiza una solicitud de creación de Pods, el sistema hace lo siguien indicar que ha sido modificada por un `PodPreset`. La nota descriptiva presenta la forma `podpreset.admission.kubernetes.io/podpreset-: ""`. -Cada Pod puede ser correspondido por cero o más Pod Presets; y cada `Pod Preset` puede ser -aplicado a cero o más Pods. Cuando se aplica un `Pod Preset` a una o más Pods, Kubernetes -modifica la especificación del Pod. Para los cambios a `Env`, `EnvFrom`, y `VolumeMounts`, +Cada Pod puede ser correspondido por cero o más Pod Presets; y cada `PodPreset` puede ser +aplicado a cero o más Pods. Cuando se aplica un `PodPreset` a una o más Pods, Kubernetes +modifica la especificación del Pod. Para los cambios a `env`, `envFrom`, y `volumeMounts`, Kubernetes modifica la especificación del Container para todos los Containers en el Pod; -para los cambios a `Volume`, Kubernetes modifica la especificación del Pod. +para los cambios a `volumes`, Kubernetes modifica la especificación del Pod. {{< note >}} Un Pod Preset es capaz de modificar los siguientes campos en las especificaciones de un Pod en caso de ser necesario: - El campo `.spec.containers`. -- El campo `initContainers` (requiere Kubernetes versión 1.14.0 o posterior). +- El campo `.spec.initContainers` {{< /note >}} ### Deshabilitar un Pod Preset para un Pod específico Puede haber casos en los que se desee que un Pod no se vea alterado por ninguna posible modificación del Pod Preset. En estos casos, se puede añadir una observación en el Pod -Spec de la siguiente forma: `podpreset.admission.kubernetes.io/exclude: "true"`. - -## Habilitando un Pod Preset - -Con el fin de utilizar los Pod Presets en un clúster debe asegurarse de lo siguiente: - -1. Que se ha configurado el tipo de API `settings.k8s.io/v1alpha1/podpreset`. Esto se puede hacer, - por ejemplo, incluyendo `settings.k8s.io/v1alpha1=true` como valor de la opción `--runtime-config` - en el servidor API. En minikube se debe añadir el flag - `--extra-config=apiserver.runtime-config=settings.k8s.io/v1alpha1=true` cuando el clúster - se está iniciando. -2. Que se ha habilitado el controlador de admisión `PodPreset`. Una forma de hacer esto es incluir - `PodPreset` como valor de la opción `--enable-admission-plugins` especificada - para el servidor API. En minikube se debe añadir el flag - - ```shell - --extra-config=apiserver.enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota,PodPreset - ``` - - cuando el clúster se está iniciando. -3. Que se han definido los Pod Presets mediante la creación de objetos `PodPreset` en el - namespace que se utilizará. - +`.spec` de la siguiente forma: `podpreset.admission.kubernetes.io/exclude: "true"`. ## {{% heading "whatsnext" %}} -* [Inyectando datos en un Pod usando PodPreset](/docs/tasks/inject-data-application/podpreset/) - +Ver [Inyectando datos en un Pod usando PodPreset](/docs/tasks/inject-data-application/podpreset/) +Para más información sobre los detalles de los trasfondos, consulte la [propuesta de diseño de PodPreset](https://git.k8s.io/community/contributors/design-proposals/service-catalog/pod-preset.md). diff --git a/content/fr/docs/tasks/access-application-cluster/configure-access-multiple-clusters.md b/content/fr/docs/tasks/access-application-cluster/configure-access-multiple-clusters.md new file mode 100644 index 0000000000..ba11fb3a06 --- /dev/null +++ b/content/fr/docs/tasks/access-application-cluster/configure-access-multiple-clusters.md @@ -0,0 +1,374 @@ +--- +title: Configurer l'accès à plusieurs clusters +content_type: task +weight: 30 +card: + name: tasks + weight: 40 +--- + + + + +Cette page montre comment configurer l'accès à plusieurs clusters à l'aide de fichiers de configuration. +Une fois vos clusters, utilisateurs et contextes définis dans un ou plusieurs fichiers de configuration, vous pouvez basculer rapidement entre les clusters en utilisant la commande `kubectl config use-context`. + +{{< note >}} +Un fichier utilisé pour configurer l'accès à un cluster est parfois appelé *fichier kubeconfig*. +C'est une manière générique de se référer aux fichiers de configuration. +Cela ne signifie pas qu'il existe un fichier nommé `kubeconfig`. +{{< /note >}} + +## {{% heading "prerequisites" %}} + +{{< include "task-tutorial-prereqs.md" >}} + +Pour vérifier que {{< glossary_tooltip text="kubectl" term_id="kubectl" >}} est installé, executez `kubectl version --client`. +La version kubectl doit être [dans une version mineure](/docs/setup/release/version-skew-policy/#kubectl) de votre +serveur API du cluster. + + + +## Définir des clusters, des utilisateurs et des contextes + +Supposons que vous ayez deux clusters, un pour le développement et un pour le travail `scratch`. +Dans le cluster `development`, vos développeurs frontend travaillent dans un espace de noms appelé `frontend`, et vos développeurs de stockage travaillent dans un espace de noms appelé `storage`. +Dans votre cluster `scratch`, les développeurs travaillent dans le namespace par défaut ou créent des namespaces auxiliaires comme bon leur semble. +L'accès au cluster `development` nécessite une authentification par certificat. +L'accès au cluster `scratch` nécessite une authentification par nom d'utilisateur et mot de passe. + +Créez un répertoire nommé `config-exercice`. +Dans votre répertoire `config-exercice`, créez un fichier nommé `config-demo` avec ce contenu: + +```shell +apiVersion: v1 +kind: Config +preferences: {} + +clusters: +- cluster: + name: development +- cluster: + name: scratch + +users: +- name: developer +- name: experimenter + +contexts: +- context: + name: dev-frontend +- context: + name: dev-storage +- context: + name: exp-scratch +``` + +Un fichier de configuration décrit les clusters, les utilisateurs et les contextes. +Votre fichier `config-demo` a le cadre pour décrire deux clusters, deux utilisateurs et trois contextes. + +Allez dans votre répertoire `config-exercice`. +Entrez ces commandes pour ajouter les détails du cluster à votre fichier de configuration: + +```shell +kubectl config --kubeconfig=config-demo set-cluster development --server=https://1.2.3.4 --certificate-authority=fake-ca-file +kubectl config --kubeconfig=config-demo set-cluster scratch --server=https://5.6.7.8 --insecure-skip-tls-verify +``` + +Ajoutez les détails de l'utilisateur à votre fichier de configuration: + +```shell +kubectl config --kubeconfig=config-demo set-credentials developer --client-certificate=fake-cert-file --client-key=fake-key-seefile +kubectl config --kubeconfig=config-demo set-credentials experimenter --username=exp --password=some-password +``` + +{{< note >}} + +- Pour supprimer un utilisateur, vous pouvez exécuter `kubectl --kubeconfig=config-demo config unset users.` +- Pour supprimer un cluster, vous pouvez exécuter `kubectl --kubeconfig=config-demo config unset clusters.` +- Pour supprimer un contexte, vous pouvez exécuter `kubectl --kubeconfig=config-demo config unset contexts.` +{{< /note >}} + +Ajoutez des détails de contexte à votre fichier de configuration: + +```shell +kubectl config --kubeconfig=config-demo set-context dev-frontend --cluster=development --namespace=frontend --user=developer +kubectl config --kubeconfig=config-demo set-context dev-storage --cluster=development --namespace=storage --user=developer +kubectl config --kubeconfig=config-demo set-context exp-scratch --cluster=scratch --namespace=default --user=experimenter +``` + +Ouvrez votre fichier `config-demo` pour voir les détails ajoutés. +Au lieu d'ouvrir le fichier `config-demo`, vous pouvez utiliser la commande `kubectl config view`. + +```shell +kubectl config --kubeconfig=config-demo view +``` + +La sortie montre les deux clusters, deux utilisateurs et trois contextes: + +```shell +apiVersion: v1 +clusters: +- cluster: + certificate-authority: fake-ca-file + server: https://1.2.3.4 + name: development +- cluster: + insecure-skip-tls-verify: true + server: https://5.6.7.8 + name: scratch +contexts: +- context: + cluster: development + namespace: frontend + user: developer + name: dev-frontend +- context: + cluster: development + namespace: storage + user: developer + name: dev-storage +- context: + cluster: scratch + namespace: default + user: experimenter + name: exp-scratch +current-context: "" +kind: Config +preferences: {} +users: +- name: developer + user: + client-certificate: fake-cert-file + client-key: fake-key-file +- name: experimenter + user: + password: some-password + username: exp +``` + +Le `fake-ca-file`, `fake-cert-file` et `fake-key-file` ci-dessus sont les espaces réservés pour les noms de chemin des fichiers de certificat. +Vous devez les remplacer par les noms de chemin réels des fichiers de certificat dans votre environnement. + +Parfois, vous souhaiterez peut-être utiliser des données encodées en Base64 incorporées ici au lieu de fichiers de certificat séparés; dans ce cas, vous devez ajouter le suffixe `-data` aux clés, par exemple, `certificate-Authority-data`, `client-certificate-data`, `client-key-data`. + +Chaque contexte est un triplet (cluster, utilisateur, namespace). +Par exemple, le contexte `dev-frontend` dit, "Utilisez les informations d'identification de l'utilisateur `developer` pour accéder au namespace `frontend` du cluster `development`". + +Définissez le contexte actuel: + +```shell +kubectl config --kubeconfig=config-demo use-context dev-frontend +``` + +Maintenant, chaque fois que vous entrez une commande `kubectl`, l'action s'appliquera au cluster et au namespace répertorié dans le contexte `dev-frontend`. +Et la commande utilisera les informations d'identification de l'utilisateur répertoriées dans le contexte `dev-frontend`. + +Pour voir uniquement les informations de configuration associées au contexte actuel, utilisez l'indicateur `--minify`. + +```shell +kubectl config --kubeconfig=config-demo view --minify +``` + +La sortie affiche les informations de configuration associées au contexte `dev-frontend`: + +```shell +apiVersion: v1 +clusters: +- cluster: + certificate-authority: fake-ca-file + server: https://1.2.3.4 + name: development +contexts: +- context: + cluster: development + namespace: frontend + user: developer + name: dev-frontend +current-context: dev-frontend +kind: Config +preferences: {} +users: +- name: developer + user: + client-certificate: fake-cert-file + client-key: fake-key-file +``` + +Supposons maintenant que vous souhaitiez travailler pendant un certain temps dans le cluster scratch. + +Changez le contexte actuel en `exp-scratch`: + +```shell +kubectl config --kubeconfig=config-demo use-context exp-scratch +``` + +Maintenant, toute commande `kubectl` que vous donnez s'appliquera au namespace par défaut du cluster `scratch`. +Et la commande utilisera les informations d'identification de l'utilisateur répertoriées dans le contexte `exp-scratch`. + +Afficher la configuration associée au nouveau contexte actuel, `exp-scratch`. + +```shell +kubectl config --kubeconfig=config-demo view --minify +``` + +Enfin, supposons que vous vouliez travailler pendant un certain temps dans le namespace `storage` du cluster `development`. + +Changez le contexte actuel en `dev-storage`: + +```shell +kubectl config --kubeconfig=config-demo use-context dev-storage +``` + +Afficher la configuration associée au nouveau contexte actuel, `dev-storage`. + +```shell +kubectl config --kubeconfig=config-demo view --minify +``` + +## Créer un deuxième fichier de configuration + +Dans votre répertoire `config-exercice`, créez un fichier nommé `config-demo-2` avec ce contenu: + +```shell +apiVersion: v1 +kind: Config +preferences: {} + +contexts: +- context: + cluster: development + namespace: ramp + user: developer + name: dev-ramp-up +``` + +Le fichier de configuration précédent définit un nouveau contexte nommé `dev-ramp-up`. + +## Définissez la variable d'environnement KUBECONFIG + +Vérifiez si vous avez une variable d'environnement nommée `KUBECONFIG`. +Si tel est le cas, enregistrez la valeur actuelle de votre variable d'environnement `KUBECONFIG`, afin de pouvoir la restaurer ultérieurement. +Par exemple: + +### Linux + +```shell +export KUBECONFIG_SAVED=$KUBECONFIG +``` + +### Windows PowerShell + +```shell +$Env:KUBECONFIG_SAVED=$ENV:KUBECONFIG +``` + +La variable d'environnement `KUBECONFIG` est une liste de chemins vers les fichiers de configuration. +La liste est délimitée par deux-points pour Linux et Mac et par des points-virgules pour Windows. +Si vous avez une variable d'environnement `KUBECONFIG`, familiarisez-vous avec les fichiers de configuration de la liste. + +Ajoutez temporairement deux chemins à votre variable d'environnement `KUBECONFIG`. +Par exemple: + +### Linux + +```shell +export KUBECONFIG=$KUBECONFIG:config-demo:config-demo-2 +``` + +### Windows PowerShell + +```shell +$Env:KUBECONFIG=("config-demo;config-demo-2") +``` + +Dans votre répertoire `config-exercice`, entrez cette commande: + +```shell +kubectl config view +``` + +La sortie affiche les informations fusionnées de tous les fichiers répertoriés dans votre variable d'environnement `KUBECONFIG`. +En particulier, notez que les informations fusionnées ont le contexte `dev-ramp-up` du fichier `config-demo-2` et les trois contextes du fichier `config-demo`: + +```shell +contexts: +- context: + cluster: development + namespace: frontend + user: developer + name: dev-frontend +- context: + cluster: development + namespace: ramp + user: developer + name: dev-ramp-up +- context: + cluster: development + namespace: storage + user: developer + name: dev-storage +- context: + cluster: scratch + namespace: default + user: experimenter + name: exp-scratch +``` + +Pour plus d'informations sur la manière dont les fichiers kubeconfig sont fusionnés, consultez [Organisation de l'accès au cluster à l'aide des fichiers kubeconfig](/docs/concepts/configuration/organize-cluster-access-kubeconfig/) + +## Explorez le répertoire $HOME/.kube + +Si vous avez déjà un cluster, et vous pouvez utiliser `kubectl` pour interagir avec le cluster, alors vous avez probablement un fichier nommé `config` dans le repertoire `$HOME/.kube`. + +Allez dans `$ HOME/.kube`, et voyez quels fichiers sont là. +En règle générale, il existe un fichier nommé `config`. +Il peut également y avoir d'autres fichiers de configuration dans ce répertoire. +Familiarisez-vous brièvement avec le contenu de ces fichiers. + +## Ajoutez $HOME/.kube/config à votre variable d'environnement KUBECONFIG + +Si vous avez un fichier `$ HOME/.kube/config`, et qu'il n'est pas déjà répertorié dans votre variable d'environnement `KUBECONFIG`, ajoutez-le maintenant à votre variable d'environnement `KUBECONFIG`. +Par exemple: + +### Linux + +```shell +export KUBECONFIG=$KUBECONFIG:$HOME/.kube/config +``` + +### Windows Powershell + +```shell +$Env:KUBECONFIG="$Env:KUBECONFIG;$HOME\.kube\config" +``` + +Affichez les informations de configuration fusionnées à partir de tous les fichiers qui sont maintenant répertoriés dans votre variable d'environnement `KUBECONFIG`. +Dans votre répertoire config-exercice, entrez: + +```shell +kubectl config view +``` + +## Nettoyage + +Remettez votre variable d'environnement `KUBECONFIG` à sa valeur d'origine. + +Par exemple: + +### Linux + +```shell +export KUBECONFIG=$KUBECONFIG_SAVED +``` + +### Windows PowerShell + +```shell +$Env:KUBECONFIG=$ENV:KUBECONFIG_SAVED +``` + +## {{% heading "whatsnext" %}} + +- [Organisation de l'accès au cluster à l'aide des fichiers kubeconfig](/docs/concepts/configuration/organize-cluster-access-kubeconfig/) +- [kubectl config](/docs/reference/generated/kubectl/kubectl-commands#config) diff --git a/content/fr/docs/tasks/access-application-cluster/list-all-running-container-images.md b/content/fr/docs/tasks/access-application-cluster/list-all-running-container-images.md new file mode 100644 index 0000000000..eea6e5cd3d --- /dev/null +++ b/content/fr/docs/tasks/access-application-cluster/list-all-running-container-images.md @@ -0,0 +1,97 @@ +--- +title: Lister toutes les images de conteneur exécutées dans un cluster +content_type: task +weight: 100 +--- + + + +Cette page montre comment utiliser kubectl pour répertorier toutes les images de conteneur pour les pods s'exécutant dans un cluster. + +## {{% heading "prerequisites" %}} + +{{< include "task-tutorial-prereqs.md" >}} {{< version-check >}} + + + +Dans cet exercice, vous allez utiliser kubectl pour récupérer tous les pods exécutés dans un cluster et formater la sortie pour extraire la liste des conteneurs pour chacun. + +## Répertorier toutes les images de conteneurs dans tous les namespaces + +- Récupérez tous les pods dans tous les namespace à l'aide de `kubectl get pods --all-namespaces` +- Formatez la sortie pour inclure uniquement la liste des noms d'image de conteneur à l'aide de `-o jsonpath={..image}`. + Cela analysera récursivement le champ `image` du json retourné. + - Voir la [reference jsonpath](/docs/reference/kubectl/jsonpath/) pour plus d'informations sur l'utilisation de jsonpath. +- Formatez la sortie à l'aide des outils standard: `tr`, `sort`, `uniq` + - Utilisez `tr` pour remplacer les espaces par des nouvelles lignes + - Utilisez `sort` pour trier les résultats + - Utilisez `uniq` pour agréger le nombre d'images + +```shell +kubectl get pods --all-namespaces -o jsonpath="{..image}" |\ +tr -s '[[:space:]]' '\n' |\ +sort |\ +uniq -c +``` + +La commande ci-dessus renverra récursivement tous les champs nommés `image` pour tous les éléments retournés. + +Comme alternative, il est possible d'utiliser le chemin absolu vers le champ d'image dans le Pod. +Cela garantit que le champ correct est récupéré même lorsque le nom du champ est répété, par ex. de nombreux champs sont appelés `name` dans un élément donné: + +```shell +kubectl get pods --all-namespaces -o jsonpath="{.items[*].spec.containers[*].image}" +``` + +Le jsonpath est interprété comme suit: + +- `.items[*]`: pour chaque valeur renvoyée +- `.spec`: obtenir les spécifications +- `.containers[*]`: pour chaque conteneur +- `.image`: obtenir l'image + +{{< note >}} +Lors de la récupération d'un seul pod par son nom, par exemple `kubectl get pod nginx`, la portion `.items[*]` du chemin doit être omis car un seul pod est renvoyé au lieu d'une liste d'éléments. +{{< /note >}} + +## Liste des images de conteneurs par pod + +Le formatage peut être contrôlé davantage en utilisant l'opération `range` pour parcourir les éléments individuellement. + +```shell +kubectl get pods --all-namespaces -o=jsonpath='{range .items[*]}{"\n"}{.metadata.name}{":\t"}{range .spec.containers[*]}{.image}{", "}{end}{end}' |\ +sort +``` + +## Filtrage des images de conteneur de liste par label de pod + +Pour cibler uniquement les pods correspondant à un label spécifique, utilisez l'indicateur -l. +Les éléments suivants correspondent uniquement aux pods avec les labels `app=nginx`. + +```shell +kubectl get pods --all-namespaces -o=jsonpath="{..image}" -l app=nginx +``` + +## Filtrage des images de conteneur de liste par namespace de pod + +Pour cibler uniquement les pods dans un namespace spécifique, utilisez l'indicateur de namespace. +Ce qui suit correspond uniquement aux pods du namespace `kube-system`. + +```shell +kubectl get pods --namespace kube-system -o jsonpath="{..image}" +``` + +## Répertorier les images de conteneurs en utilisant un go-template au lieu de jsonpath + +Comme alternative à jsonpath, Kubectl peut aussi utiliser les [go-templates](https://golang.org/pkg/text/template/) pour formater la sortie: + +```shell +kubectl get pods --all-namespaces -o go-template --template="{{range .items}}{{range .spec.containers}}{{.image}} {{end}}{{end}}" +``` + +## {{% heading "whatsnext" %}} + +### Reference + +- Guide de référence pour [Jsonpath](/docs/reference/kubectl/jsonpath/) +- Guide de référence pour les [Go template](https://golang.org/pkg/text/template/) diff --git a/content/fr/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes.md b/content/fr/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes.md index dc427699f6..1c45b2b071 100644 --- a/content/fr/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes.md +++ b/content/fr/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes.md @@ -1,10 +1,10 @@ --- title: Configurer les Liveness, Readiness et Startup Probes -content_template: templates/task +content_type: task weight: 110 --- -{{% capture overview %}} + Cette page montre comment configurer les liveness, readiness et startup probes pour les conteneurs. @@ -17,15 +17,11 @@ Le Kubelet utilise startup probes pour savoir quand une application d'un contene Si une telle probe est configurée, elle désactive les contrôles de liveness et readiness jusqu'à cela réussit, en s'assurant que ces probes n'interfèrent pas avec le démarrage de l'application. Cela peut être utilisé dans le cas des liveness checks sur les conteneurs à démarrage lent, en les évitant de se faire tuer par le Kubelet avant qu'ils ne soient opérationnels. -{{% /capture %}} - -{{% capture prerequisites %}} +## {{% heading "prerequisites" %}} {{< include "task-tutorial-prereqs.md" >}} {{< version-check >}} -{{% /capture %}} - -{{% capture steps %}} + ## Définir une commande de liveness @@ -281,9 +277,7 @@ Voici un scénario où vous le mettriez en place. Supposons que le conteneur éc Le Kubelet fait la connexion de la probe au noeud, pas dans le Pod, ce qui signifie que vous ne pouvez pas utiliser un nom de service dans le paramètre `host` puisque le Kubelet est incapable pour le résoudre. -{{% /capture %}} - -{{% capture whatsnext %}} +## {{% heading whatsnext %}} * Pour en savoir plus sur [Probes des Conteneurs](/docs/concepts/workloads/pods/pod-lifecycle/#container-probes). @@ -294,6 +288,4 @@ Le Kubelet fait la connexion de la probe au noeud, pas dans le Pod, ce qui signi * [Container](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#container-v1-core) * [Probe](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#probe-v1-core) -{{% /capture %}} - diff --git a/content/fr/docs/tasks/configure-pod-container/share-process-namespace.md b/content/fr/docs/tasks/configure-pod-container/share-process-namespace.md index f55431571d..cb5a99cbe3 100644 --- a/content/fr/docs/tasks/configure-pod-container/share-process-namespace.md +++ b/content/fr/docs/tasks/configure-pod-container/share-process-namespace.md @@ -1,11 +1,11 @@ --- title: Partager l'espace de nommage des processus entre les conteneurs d'un Pod min-kubernetes-server-version: v1.10 -content_template: templates/task +content_type: task weight: 160 --- -{{% capture overview %}} + {{< feature-state state="stable" for_k8s_version="v1.17" >}} @@ -13,15 +13,11 @@ Cette page montre comment configurer le partage de l'espace de noms d'un process Vous pouvez utiliser cette fonctionnalité pour configurer les conteneurs coopérants, comme un conteneur de sidecar de gestionnaire de journaux, ou pour dépanner les images de conteneurs qui n'incluent pas d'utilitaires de débogage comme un shell. -{{% /capture %}} - -{{% capture prerequisites %}} +## {{% heading prerequisites %}} {{< include "task-tutorial-prereqs.md" >}} {{< version-check >}} -{{% /capture %}} - -{{% capture steps %}} + ## Configurer un Pod @@ -83,10 +79,6 @@ events { worker_connections 1024; ``` -{{% /capture %}} - -{{% capture discussion %}} - ## Comprendre le processus de partage de l'espace de nommage Les pods partagent de nombreuses ressources, il est donc logique qu'elles partagent également un espace de noms des processus. Pour certaines images de conteneur, on peut envisager de les isoler les uns des autres. Il est donc important de comprendre ces différences : @@ -97,6 +89,3 @@ Les pods partagent de nombreuses ressources, il est donc logique qu'elles partag 1. **Les systèmes de fichiers des conteneurs sont visibles par les autres conteneurs du pod à travers le lien `/proc/$pid/root`.** Cela rend le débogage plus facile, mais cela signifie aussi que les secrets du système de fichiers ne sont protégés que par les permissions du système de fichiers. -{{% /capture %}} - - diff --git a/content/id/docs/concepts/overview/components.md b/content/id/docs/concepts/overview/components.md index aa2ee52152..43bba4d783 100644 --- a/content/id/docs/concepts/overview/components.md +++ b/content/id/docs/concepts/overview/components.md @@ -2,6 +2,9 @@ title: Komponen-Komponen Kubernetes content_type: concept weight: 20 +description: > + Sebuah klaster Kubernetes terdiri dari komponen yang merepresentasikan + bidang kontrol dan sepasang mesin yaitu _nodes_. card: name: concepts weight: 20 diff --git a/content/id/docs/concepts/overview/kubernetes-api.md b/content/id/docs/concepts/overview/kubernetes-api.md index 35bf3f67dc..336c48f60f 100644 --- a/content/id/docs/concepts/overview/kubernetes-api.md +++ b/content/id/docs/concepts/overview/kubernetes-api.md @@ -2,6 +2,9 @@ title: API Kubernetes content_type: concept weight: 30 +description: > + API Kubernetes membuatmu dapat melakukan _query_ dan memanipulasi keadaan objek dalam Kubernetes. + Inti dari bidang kontrol Kubernetes adalah _server_ API dan HTTP API yang diekspos. Pengguna, berbagai bagian klastermu, dan komponen eksternal semuanya berkomunikasi satu sama lain melalui server API. card: name: concepts weight: 30 diff --git a/content/id/docs/concepts/overview/object-management-kubectl/_index.md b/content/id/docs/concepts/overview/object-management-kubectl/_index.md deleted file mode 100644 index 909abd79ce..0000000000 --- a/content/id/docs/concepts/overview/object-management-kubectl/_index.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -title: "Pengelolaan Objek dengan kubectl" -weight: 50 ---- - diff --git a/content/id/docs/concepts/overview/what-is-kubernetes.md b/content/id/docs/concepts/overview/what-is-kubernetes.md index de35f65b29..a0e0982633 100644 --- a/content/id/docs/concepts/overview/what-is-kubernetes.md +++ b/content/id/docs/concepts/overview/what-is-kubernetes.md @@ -2,6 +2,16 @@ title: Apa itu Kubernetes? content_type: concept weight: 10 +description: > + Kubernetes merupakan _platform open-source_ yang digunakan untuk melakukan + manajemen _workloads_ aplikasi yang dikontainerisasi, serta menyediakan + konfigurasi dan otomatisasi secara deklaratif. Kubernetes berada di dalam ekosistem + yang besar dan berkembang cepat. _Service_, _support_, dan perkakas + Kubernetes tersedia secara meluas. Kubernetes merupakan _platform open-source_ + yang digunakan untuk melakukan manajemen _workloads_ aplikasi yang dikontainerisasi, + serta menyediakan konfigurasi dan otomatisasi secara deklaratif. Kubernetes berada + di dalam ekosistem yang besar dan berkembang cepat. _Service_, _support_, + dan perkakas Kubernetes tersedia secara meluas. card: name: concepts weight: 10 diff --git a/content/id/docs/concepts/overview/working-with-objects/_index.md b/content/id/docs/concepts/overview/working-with-objects/_index.md index 53a2d7c9b3..6849cbe870 100644 --- a/content/id/docs/concepts/overview/working-with-objects/_index.md +++ b/content/id/docs/concepts/overview/working-with-objects/_index.md @@ -1,5 +1,9 @@ --- title: "Menggunakan Objek-Objek Kubernetes" weight: 40 +description: > + Objek-objek Kubernetes adalah entitas yang tetap dalam sistem Kubernetes. + Kubernetes menggunakan entitas tersebut untuk merepresentasikan keadaan dari klastermu. + Pelajari tentang objek model Kubernetes dan bagaimana menggunakan objek tersebut. --- diff --git a/content/id/docs/concepts/services-networking/network-policies.md b/content/id/docs/concepts/services-networking/network-policies.md index fe510b846d..cba7ff9337 100644 --- a/content/id/docs/concepts/services-networking/network-policies.md +++ b/content/id/docs/concepts/services-networking/network-policies.md @@ -81,7 +81,7 @@ kecuali penyedia jaringan mendukung network policy. membutuhkan _field-field_ `apiVersion`, `kind`, dan `metadata`. Informasi generik mengenai bagaimana bekerja dengan _file_ `config`, dapat dilihat di [Konfigurasi Kontainer menggunakan `ConfigMap`](/id/docs/tasks/configure-pod-container/configure-pod-configmap/), -serta [Manajemen Objek](/docs/concepts/overview/object-management-kubectl/overview/). +serta [Manajemen Objek](/id/docs/tasks/manage-kubernetes-objects/). **spec**: `NetworkPolicy` [spec](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#spec-and-status) memiliki semua informasi yang harus diberikan untuk memberikan definisi _network policy_ yang ada pada _namespace_ tertentu. diff --git a/content/id/docs/concepts/storage/volume-snapshots.md b/content/id/docs/concepts/storage/volume-snapshots.md index 5ddfc2aaa6..96461e16a3 100644 --- a/content/id/docs/concepts/storage/volume-snapshots.md +++ b/content/id/docs/concepts/storage/volume-snapshots.md @@ -43,7 +43,7 @@ VolumeSnapshotContent merupakan bagian dari sumber daya klaster. VolumeSnapshot Ada dua cara untuk menyediakan _snapshot_: secara statis maupun dinamis. #### Statis -Seorang adminstrator klaster membuat beberapa VolumeSnapshotContent, yang masing-masing memiliki detail tentang penyimpanan sebenarnya yang dapat dipergunakan oleh para pengguna. VolumeSnapshotContent tersebut dapat dikonsumsi melalui API Kubernetes. +Seorang administrator klaster membuat beberapa VolumeSnapshotContent, yang masing-masing memiliki detail tentang penyimpanan sebenarnya yang dapat dipergunakan oleh para pengguna. VolumeSnapshotContent tersebut dapat dikonsumsi melalui API Kubernetes. #### Dinamis Ketika VolumeSnapshotContent yang dibuat oleh administrator tidak ada yang sesuai dengan VolumeSnapshot yang dibuat pengguna, klaster bisa saja diff --git a/content/id/docs/concepts/storage/volumes.md b/content/id/docs/concepts/storage/volumes.md index 8d593f1eba..fca45bbd74 100644 --- a/content/id/docs/concepts/storage/volumes.md +++ b/content/id/docs/concepts/storage/volumes.md @@ -16,7 +16,7 @@ Pengetahuan tentang [Pod](/docs/user-guide/pods) disarankan. ## Latar Belakang -Docker juga memiliki konsep _[volume]_(https://docs.docker.com/storage/), walaupun konsepnya Docker agak lebih fleksibel dan kurang dikelola. Pada Docker, sebuah volume adalah sesederhana sebuah direktori pada _disk_ atau di dalam Container lainnya. _Lifetime_ tidak dikelola dan hingga baru-baru ini hanya ada volume yang didukung _disk_ lokal. Docker sekarang menyediakan _driver_ untuk volume, namun fungsionalitasnya masih sangat terbatas (misalnya hingga Docker 1.7 hanya ada satu _driver_ volume yang diizinkan untuk setiap Container, dan tidak ada cara untuk menyampaikan parameter kepada volume). +Docker juga memiliki konsep _[volume](https://docs.docker.com/storage/)_, walaupun konsepnya Docker agak lebih fleksibel dan kurang dikelola. Pada Docker, sebuah volume adalah sesederhana sebuah direktori pada _disk_ atau di dalam Container lainnya. _Lifetime_ tidak dikelola dan hingga baru-baru ini hanya ada volume yang didukung _disk_ lokal. Docker sekarang menyediakan _driver_ untuk volume, namun fungsionalitasnya masih sangat terbatas (misalnya hingga Docker 1.7 hanya ada satu _driver_ volume yang diizinkan untuk setiap Container, dan tidak ada cara untuk menyampaikan parameter kepada volume). Sebaliknya, sebuah Volume Kubernetes memiliki _lifetime_ yang gamblang - sama dengan _lifetime_ Pod yang berisi Volume tersebut. Oleh karena itu, sebuah Volume bertahan lebih lama dari Container-container yang berjalan di dalam Pod tersebut, dan data di Volum tersebut juga dipertahankan melewati diulangnya Container. Tentu saja, saat sebuah Pod berakhir, Volume tersebut juga akan berakhir/terhapus. Dan mungkin lebih penting lagi, Kubernetes mendukung banyak jenis Volume, dan sebuah Pod dapat menggunakan sebanyak apapun Volume secara bersamaan. diff --git a/content/id/docs/concepts/overview/object-management-kubectl/declarative-config.md b/content/id/docs/tasks/manage-kubernetes-objects/declarative-config.md similarity index 98% rename from content/id/docs/concepts/overview/object-management-kubectl/declarative-config.md rename to content/id/docs/tasks/manage-kubernetes-objects/declarative-config.md index 46066769d4..3a4ae67de2 100644 --- a/content/id/docs/concepts/overview/object-management-kubectl/declarative-config.md +++ b/content/id/docs/tasks/manage-kubernetes-objects/declarative-config.md @@ -1,7 +1,7 @@ --- title: Pengelolaan Objek Kubernetes secara Deklaratif dengan Menggunakan File Konfigurasi content_type: concept -weight: 40 +weight: 10 --- @@ -18,15 +18,15 @@ Perintah `kubectl` memungkinkan tiga cara untuk mengelola objek: * Konfigurasi objek imperatif * Konfigurasi objek deklaratif -Lihat [Pengelolaan Objek Kubernetes](/docs/concepts/overview/object-management-kubectl/overview/) untuk menyimak diskusi mengenai kelebihan dan kekurangan dari tiap cara pengelolaan objek. +Lihat [Pengelolaan Objek Kubernetes](/id/docs/concepts/overview/working-with-objects/object-management/) untuk menyimak diskusi mengenai kelebihan dan kekurangan dari tiap cara pengelolaan objek. ## Sebelum kamu mulai Konfigurasi objek secara deklaratif membutuhkan pemahaman yang baik tentang definisi dan konfigurasi objek-objek Kubernetes. Jika belum pernah, kamu disarankan untuk membaca terlebih dulu dokumen-dokumen berikut: -- [Pengelolaan Objek Kubernetes Menggunakan Perintah Imperatif](/id/docs/concepts/overview/object-management-kubectl/imperative-command/) -- [Pengelolaan Objek Kubernetes Menggunakan File Konfigurasi Imperatif](/id/docs/concepts/overview/object-management-kubectl/imperative-config/) +- [Pengelolaan Objek Kubernetes Menggunakan Perintah Imperatif](/id/docs/tasks/manage-kubernetes-objects/imperative-command/) +- [Pengelolaan Objek Kubernetes Menggunakan File Konfigurasi Imperatif](/id/docs/tasks/manage-kubernetes-objects/imperative-config/) Berikut adalah beberapa defnisi dari istilah-istilah yang digunakan dalam dokumen ini: @@ -862,8 +862,8 @@ template: ## {{% heading "whatsnext" %}} -- [Pengelolaan Objek Kubernetes Menggunakan Perintah Imperatif](/id/docs/concepts/overview/object-management-kubectl/imperative-command/) -- [Pengelolaan Objek Kubernetes secara Imperatif Menggunakan File Konfigurasi](/id/docs/concepts/overview/object-management-kubectl/imperative-config/) -- [Rujukan Perintah Kubectl](/docs/reference/generated/kubectl/kubectl/) +- [Pengelolaan Objek Kubernetes Menggunakan Perintah Imperatif](/id/docs/tasks/manage-kubernetes-objects/imperative-command/) +- [Pengelolaan Objek Kubernetes secara Imperatif Menggunakan File Konfigurasi](/id/docs/tasks/manage-kubernetes-objects/imperative-config/) +- [Rujukan Perintah Kubectl](/docs/reference/generated/kubectl/kubectl-commands/) - [Rujukan API Kubernetes](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/) diff --git a/content/id/docs/concepts/overview/object-management-kubectl/imperative-command.md b/content/id/docs/tasks/manage-kubernetes-objects/imperative-command.md similarity index 95% rename from content/id/docs/concepts/overview/object-management-kubectl/imperative-command.md rename to content/id/docs/tasks/manage-kubernetes-objects/imperative-command.md index 23489efb59..b0b6c34b86 100644 --- a/content/id/docs/concepts/overview/object-management-kubectl/imperative-command.md +++ b/content/id/docs/tasks/manage-kubernetes-objects/imperative-command.md @@ -1,7 +1,7 @@ --- title: Pengelolaan Objek Kubernetes dengan Perintah Imperatif content_type: concept -weight: 20 +weight: 30 --- @@ -18,7 +18,7 @@ Perintah `kubectl` mendukung tiga cara pengelolaan objek: * Konfigurasi objek imperatif * Konfigurasi objek deklaratif -Lihat [Pengelolaan Objek Kubernetes](/id/docs/concepts/overview/object-management-kubectl/overview/) untuk mengenali lebih lanjut kelebihan dan kekurangan dari tiap pengelolaan objek. +Lihat [Pengelolaan Objek Kubernetes](/id/docs/concepts/overview/working-with-objects/object-management/) untuk mengenali lebih lanjut kelebihan dan kekurangan dari tiap pengelolaan objek. ## Cara membuat objek @@ -126,8 +126,8 @@ kubectl create --edit -f /tmp/srv.yaml ## {{% heading "whatsnext" %}} -- [Pengelolaan Objek Kubernetes secara Imperatif dengan Menggunakan Konfigurasi Objek](/id/docs/concepts/overview/object-management-kubectl/imperative-config/) -- [Pengelolaan Objek Kubernetes secara Deklaratif dengan Menggunakan File Konfigurasi](/id/docs/concepts/overview/object-management-kubectl/declarative-config/) -- [Rujukan Perintah Kubectl](/docs/reference/generated/kubectl/kubectl/) +- [Pengelolaan Objek Kubernetes secara Imperatif dengan Menggunakan Konfigurasi Objek](/id/docs/tasks/manage-kubernetes-objects/imperative-config/) +- [Pengelolaan Objek Kubernetes secara Deklaratif dengan Menggunakan Konfigurasi Objek](/id//docs/tasks/manage-kubernetes-objects/declarative-config/) +- [Rujukan Perintah Kubectl](/docs/reference/generated/kubectl/kubectl-commands/) - [Kubernetes API Reference](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/) diff --git a/content/id/docs/concepts/overview/object-management-kubectl/imperative-config.md b/content/id/docs/tasks/manage-kubernetes-objects/imperative-config.md similarity index 91% rename from content/id/docs/concepts/overview/object-management-kubectl/imperative-config.md rename to content/id/docs/tasks/manage-kubernetes-objects/imperative-config.md index 94f1082e35..e9f0cfe639 100644 --- a/content/id/docs/concepts/overview/object-management-kubectl/imperative-config.md +++ b/content/id/docs/tasks/manage-kubernetes-objects/imperative-config.md @@ -1,7 +1,7 @@ --- -title: Penglolaan Objek Kubernetes Secara Imperatif dengan Menggunakan File Konfigurasi +title: Pengelolaan Objek Kubernetes Secara Imperatif dengan Menggunakan File Konfigurasi content_type: concept -weight: 30 +weight: 40 --- @@ -18,7 +18,7 @@ Perintah `kubectl` mendukung tiga cara pengelolaan objek: * Konfigurasi objek imperatif * Konfigurasi objek deklaratif -Lihat [Pengelolaan Objek Kubernetes](/id/docs/concepts/overview/object-management-kubectl/overview/) untuk mengenali lebih lanjut kelebihan dan kekurangan dari tiap cara pengelolaan objek. +Lihat [Pengelolaan Objek Kubernetes](/id/docs/tasks/manage-kubernetes-objects/) untuk mengenali lebih lanjut kelebihan dan kekurangan dari tiap cara pengelolaan objek. ## Cara membuat objek @@ -108,9 +108,9 @@ template: ## {{% heading "whatsnext" %}} -- [Pengelolaan Objek Kubernetes Menggunakan Perintah Imperatif](/id/docs/concepts/overview/object-management-kubectl/imperative-command/) -- [Pengelolaan Objek Kubernetes secara Deklaratif dengan Menggunakan File Konfigurasi](/id/docs/concepts/overview/object-management-kubectl/declarative-config/) -- [Rujukan Perintah Kubectl](/docs/reference/generated/kubectl/kubectl/) +- [Pengelolaan Objek Kubernetes Menggunakan Perintah Imperatif](/id/docs/tasks/manage-kubernetes-objects/imperative-command/) +- [Pengelolaan Objek Kubernetes secara Deklaratif dengan Menggunakan File Konfigurasi](/id/docs/tasks/manage-kubernetes-objects/declarative-config/) +- [Rujukan Perintah Kubectl](/docs/reference/generated/kubectl/kubectl-commands/) - [Rujukan API Kubernetes](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/) diff --git a/content/id/docs/tasks/manage-kubernetes-objects/kustomization.md b/content/id/docs/tasks/manage-kubernetes-objects/kustomization.md index 680b20d371..2ca6535d90 100644 --- a/content/id/docs/tasks/manage-kubernetes-objects/kustomization.md +++ b/content/id/docs/tasks/manage-kubernetes-objects/kustomization.md @@ -835,7 +835,7 @@ deployment.apps "dev-my-nginx" deleted * [Kustomize](https://github.com/kubernetes-sigs/kustomize) * [Buku Kubectl](https://kubectl.docs.kubernetes.io) -* [Rujukan Perintah Kubectl](/id/docs/reference/generated/kubectl/kubectl/) -* [Rujukan API Kubernetes](/id/docs/reference/generated/kubernetes-api/{{< param "version" >}}/) +* [Rujukan Perintah Kubectl](/docs/reference/generated/kubectl/kubectl-commands/) +* [Rujukan API Kubernetes](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/) diff --git a/content/it/_index.html b/content/it/_index.html index a5347fa45a..1e5191a982 100644 --- a/content/it/_index.html +++ b/content/it/_index.html @@ -41,7 +41,6 @@ Kubernetes è open source, e ti offre la libertà di spostare i tuoi carichi di

    -
    Partecipa alla KubeCon ad Amsterdam (13-16 Agosto 2020)

    diff --git a/content/ja/docs/reference/access-authn-authz/rbac.md b/content/ja/docs/reference/access-authn-authz/rbac.md new file mode 100644 index 0000000000..fee25f323e --- /dev/null +++ b/content/ja/docs/reference/access-authn-authz/rbac.md @@ -0,0 +1,1102 @@ +--- +title: RBAC認可を使用する +content_type: concept +weight: 70 +--- + + +Role Based Access Control(RBAC)は、組織内の個々のユーザーのRoleをベースに、コンピューターまたはネットワークリソースへのアクセスを制御する方法です。 + + + +RBAC認可は{{< glossary_tooltip term_id="api-group" >}} `rbac.authorization.k8s.io`を使用して認可の決定を行い、Kubernetes APIを介して動的にポリシーを構成できるようにします。 + +RBACを有効にするには、以下の例のように{{< glossary_tooltip text="API server" term_id="kube-apiserver" >}}の`--authorization-mode` フラグをコンマ区切りの`RBAC`を含むリストでスタートします。 + +```shell +kube-apiserver --authorization-mode=Example,RBAC --other-options --more-options +``` + +## APIオブジェクト{#api-overview} + +RBAC APIは4種類のKubernetesオブジェクト(_Role_、 _ClusterRole_、 _RoleBinding_ そして _ClusterRoleBinding_)を宣言します。他のKubernetesオブジェクトのように`kubectl`のようなツールを使って、[オブジェクトを記述](https://kubernetes.io/ja/docs/concepts/overview/working-with-objects/kubernetes-objects/#kubernetes-objects)、または変更できます。 + +{{< caution >}} +これらのオブジェクトは設計上、アクセス制限を課します。学んできたように変更を行っている場合、 +[特権エスカレーション防止とブートストラップ](#特権昇格の防止とブートストラップ) +を参照し、これらの制限がどのようにいつかの変更を防止するのかを理解しましょう。 +{{< /caution >}} + +### RoleとClusterRole + +RBACの _Role_ または _ClusterRole_ には、一連の権限を表すルールが含まれて言います。 +権限は完全な追加方式です(「deny」のルールはありません)。 + +Roleは常に特定の{{< glossary_tooltip text="namespace" term_id="namespace" >}}で権限を設定します。 +つまり、Roleを作成する時は、Roleが属するNamespaceを指定する必要があります。 + +対照的にClusterRoleは、Namespaceに属さないリソースです。Kubernetesオブジェクトは常にNamespaceに属するか、属さないかのいずれかである必要があり、リソースは異なる名前(RoleとClusterRole)を持っています。つまり、両方であることは不可能です。 + +ClusterRolesにはいくつかの用途があります。ClusterRoleを利用して、以下のことができます。 + +1. Namespaceに属するリソースに対する権限を定義し、個々のNamespace内で付与する +2. Namespaceに属するリソースに対する権限を定義し、すべてのNamespaceにわたって付与する +3. クラスター単位でスコープされているリソースに対するアクセス許可を定義する + +NamespaceでRoleを定義する場合は、Roleを使用します。クラスター全体でRoleを定義する婆は、ClusterRoleを使用します + +#### Roleの例 + +以下はNamespace「default」にあるRoleの例で、 +{{< glossary_tooltip text="Pod" term_id="pod" >}}への読み取りアクセス権の付与に使用できます。 + +```yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + namespace: default + name: pod-reader +rules: +- apiGroups: [""] # "" はコアのAPIグループを示します + resources: ["pods"] + verbs: ["get", "watch", "list"] +``` + +#### ClusterRoleの例 + +ClusterRoleを使用してRoleと同じ権限を付与できます。 +ClusterRolesはクラスター単位でスコープされているため、以下へのアクセスの許可もできます。 + +* クラスター単位でスコープされているリソースに({{< glossary_tooltip text="node" term_id="node" >}}など) +* 非リソースエンドポイントに(`/healthz`など) +* すべてのNamespaceに渡ってNamespaceに属するリソースに(Podなど)。 + 例えば、ClusterRoleを使用して特定のユーザーに`kubectl get pods --all-namespaces`の実行を許可できます。 + +以下は特定のNamespace、またはすべてのNamespace([バインド](#rolebindingとclusterrolebinding)方法によります)で{{< glossary_tooltip text="secrets" term_id="secret" >}}への読み取りアクセス権を付与するClusterRoleの例です。 + + +```yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + # 「namespace」はClusterRolesがNamespaceに属していないため、省略されています + name: secret-reader +rules: +- apiGroups: [""] + # + # HTTPレベルでの、Secretにアクセスするリソースの名前 + # オブジェクトは"secrets" + resources: ["secrets"] + verbs: ["get", "watch", "list"] +``` + +RoleまたはClusterRoleオブジェクトの名前は有効な +[パスセグメント名](https://kubernetes.io/ja/docs/concepts/overview/working-with-objects/names/)である必要があります。 + +### RoleBindingとClusterRoleBinding + +RoleBindingはRoleで定義された権限をユーザーまたはユーザのセットに付与します。 +RoleBindingは*subjects* (ユーザー、グループ、サービスアカウント)のリストと、付与されるRoleへの参照を保持します。 +RoleBindingは特定のNamespace内の権限を付与しますが、ClusterRoleBindingはクラスター全体にアクセスする権限を付与します。 + +RoleBindingは、同じNamespace内の任意のRoleを参照できます。 +または、RoleBindingはClusterRoleを参照し、そのClusterRoleをRoleBindingのNamespaceにバインドできます。 +ClusterRoleをクラスター内のすべてのNamespaceにバインドする場合は、ClusterRoleBindingを使用します。 + +RoleBindingまたはClusterRoleBindingオブジェクトは有効な +[パスセグメント名](https://kubernetes.io/ja/docs/concepts/overview/working-with-objects/names/)である必要があります。 + +#### RoleBindingの例 {#rolebinding-example} + +以下はNamespace「default」内でユーザー「jane」に「pod-reader」のRoleを付与するRoleBindingの例です。 +これにより、「jane」にNamespace「default」のポッドの読み取り許可されます。 + +```yaml +apiVersion: rbac.authorization.k8s.io/v1 +# このRoleBindingは「jane」にNamespace「default」のポッドの読み取りを許可する +# そのNamespaceでRole「pod-reader」を既に持っている必要があります。 +kind: RoleBinding +metadata: + name: read-pods + namespace: default +subjects: +# 一つ以上の「subject」を指定する必要があります +- kind: User + name: jane # 「name」は大文字と小文字が区別されます + apiGroup: rbac.authorization.k8s.io +roleRef: + # 「roleRef」はRole/ClusterRoleへのバインドを指定します + kind: Role #RoleまたはClusterRoleである必要があります + name: pod-reader # これはバインドしたいRole名またはClusterRole名とマッチする必要があります + apiGroup: rbac.authorization.k8s.io +``` + +RoleBindingはClusterRoleを参照し、ClusterRoleで定義されている権限をRoleBinding内のNamespaceのリソースに権限付与もできます。この種類の参照を利用すると、クラスター全体で共通のRoleのセットを定義して、それらを複数のNamespace内での再利用できます。 + +例えば、以下のRoleBindingがClusterRoleを参照している場合でも、 +「dave」(大文字と小文字が区別されるsubject)はRoleBindingのNamespace(メタデータ内)が「development」のため、Namespace「development」のSecretsのみの読み取りができます。 + +```yaml +apiVersion: rbac.authorization.k8s.io/v1 +# このRoleBindingは「dave」にNamespace「development」のSecretsの読み取りを許可する +# ClusterRole「secret-reader」を既に持っている必要があります。 +kind: RoleBinding +metadata: + name: read-secrets + # + # RoleBindingのNamespaceが、どこの権限が付与されるかを決定する。 + # これはNamespace「development」内の権限のみ付与する。 + namespace: development +subjects: +- kind: User + name: dave # nameは大文字、小文字を区別する + apiGroup: rbac.authorization.k8s.io +roleRef: + kind: ClusterRole + name: secret-reader + apiGroup: rbac.authorization.k8s.io +``` + +#### ClusterRoleBindingの例 + +クラスター全体に権限を付与するには、ClusterRoleBindingを使用できます。 +以下のClusterRoleBindingはグループ「manager」のすべてのユーザーに +Secretsの読み取りを許可します。 + +```yaml +apiVersion: rbac.authorization.k8s.io/v1 +# このClusterRoleBindingはグループ「manager」のすべてのユーザーに任意のNamespaceのSecretsの読み取りを許可します。 +kind: ClusterRoleBinding +metadata: + name: read-secrets-global +subjects: +- kind: Group + name: manager # nameは大文字、小文字を区別します + apiGroup: rbac.authorization.k8s.io +roleRef: + kind: ClusterRole + name: secret-reader + apiGroup: rbac.authorization.k8s.io +``` + +Bindingを作成後は、それが参照するRoleまたはClusterRoleを変更できません。 +Bindingの`roleRef`を変更しようとすると、バリデーションエラーが発生します。Bindingの`roleRef`を変更する場合は、Bindingのオブジェクトを削除して、代わりのオブジェクトを作成する必要があります。 + +この制限には２つの理由があります。 + +1. `roleRef`をイミュータブルにすることで、誰かに既存のオブジェクトに対する`update`権限を付与します。それにより、subjectsに付与されたRoleの変更ができなくても、subjectsのリストを管理できるようになります。 +2. 異なるRoleへのBindingは根本的に異なるBindingです。 +`roleRef`を変更するためにBindingの削除/再作成を要求することによって、(すべての既存のsubjectsを確認せずに、roleRefだけを誤って変更できるようにするのとは対照的に)Binding内のsubjectsのリストのすべてが意図された新しいRoleが付与されることを担保します。 + +`kubectl auth reconcile` コマンドラインユーティリティーはRBACオブジェクトを含んだマニフェストファイルを作成または更新します。また、それらが参照しているRoleへの変更を要求されると、Bindingオブジェクトの削除と再作成を取り扱います。 +詳細は[command usage and examples](#kubectl-auth-reconcile)を確認してください。 + +### リソースを参照する + +KubernetesのAPIでは、ほとんどのリソースはPodであれば`pods`のように、オブジェクト名の文字列表現を使用して表されます。RBACは、関連するAPIエンドポイントのURLに表示されるものとまったく同じ名前を使用するリソースを参照します。 +一部のKubernetes APIには、Podのログなどの +_subresource_　が含まれます。Podのログのリクエストは次のようになります。 + +```http +GET /api/v1/namespaces/{namespace}/pods/{name}/log +``` + +この場合、`pods` はPodリソースのNamespaceに属するリソースであり、`log`は`pods`のサブリソースです。これをRBACRoleで表すには、スラッシュ(`/`)を使用してリソースとサブリソースを区切ります。サブジェクトへの`pods`の読み込みと各Podの`log`サブリソースへのアクセスを許可するには、次のように記述します。 + +```yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + namespace: default + name: pod-and-pod-logs-reader +rules: +- apiGroups: [""] + resources: ["pods", "pods/log"] + verbs: ["get", "list"] +``` + +`resourceNames`リストを通じて、特定のリクエストのリソースを名前で参照することもできます。 +指定すると、リクエストをリソースの個々のインスタンスに制限できます。 +以下は対象を`get`または`my-configmap`と名付けられた +{{< glossary_tooltip term_id="ConfigMap" >}}を`update`のみに制限する例です。 + +```yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + namespace: default + name: configmap-updater +rules: +- apiGroups: [""] + # + # HTTPレベルでの、ConfigMapにアクセスするリソースの名前 + # オブジェクトは"configmaps" + resources: ["configmaps"] + resourceNames: ["my-configmap"] + verbs: ["update", "get"] +``` + +{{< note >}} +`resourceNames`で`create`または`deletecollection`のリクエストを制限することはできません。この制限は`create`の場合、認証時にオブジェクト名がわからないためです。 +{{< /note >}} + + +### 集約ClusterRole + +複数のClusterRoleを一つのClusterRoleに _集約_ できます。 +クラスターコントロールプレーンの一部として実行されるコントローラーは、`aggregationRule`セットを持つClusterRoleオブジェクトを監視します。`aggregationRule`はコントローラーが、このオブジェクトの`rules`フィールドに結合する必要のある他のClusterRoleオブジェクトを一致させるために使用するラベル{{< glossary_tooltip text="selector" term_id="selector" >}}を定義します。 + +以下に、集約されたClusterRoleの例を示します。 + +```yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: monitoring +aggregationRule: + clusterRoleSelectors: + - matchLabels: + rbac.example.com/aggregate-to-monitoring: "true" +rules: [] # コントロールプレーンは自動的にルールを入力します +``` +既存の集約されたClusterRoleのラベルセレクターと一致する新しいClusterRoleを作成すると、その変更をトリガーに、集約されたClusterRoleに新しいルールが追加されます。 +`rbac.example.com/aggregate-to-monitoring: true`ラベルが付けられた別のClusterRoleを作成して、ClusterRole「monitoring」にルールを追加する例を以下に示します。 + +```yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: monitoring-endpoints + labels: + rbac.example.com/aggregate-to-monitoring: "true" +# ClusterRole「monitoring-endpoints」を作成すると、 +# 以下のルールがClusterRole「monitoring」に追加されます +rules: +- apiGroups: [""] + resources: ["services", "endpoints", "pods"] + verbs: ["get", "list", "watch"] +``` + +[デフォルトのユーザー向けRole](#デフォルトroleとclusterrolebinding)はClusterRoleの集約を使用します。これによりクラスター管理者として、 デフォルトroleを拡張するため、{{< glossary_tooltip term_id="CustomResourceDefinition" text="CustomResourceDefinitions" >}}または集約されたAPIサーバーなどによって提供されたルールをカスタムリソースに含めることができます。 + +例えば、次のClusterRoleでは、「admin」と「edit」のデフォルトのRoleでCronTabという名前のカスタムリソースを管理できますが、「view」のRoleではCronTabリソースに対して読み取りアクションのみを実行できます。CronTabオブジェクトは、APIサーバーから見たURLで`"crontabs"`と名前が付けられていると想定できます。 + +```yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: aggregate-cron-tabs-edit + labels: + # デフォルトRoleの「admin」と「edit」にこれらの権限を追加する。 + rbac.authorization.k8s.io/aggregate-to-admin: "true" + rbac.authorization.k8s.io/aggregate-to-edit: "true" +rules: +- apiGroups: ["stable.example.com"] + resources: ["crontabs"] + verbs: ["get", "list", "watch", "create", "update", "patch", "delete"] +--- +kind: ClusterRole +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: aggregate-cron-tabs-view + labels: + # デフォルトRoleの「view」にこれらの権限を追加します。 + rbac.authorization.k8s.io/aggregate-to-view: "true" +rules: +- apiGroups: ["stable.example.com"] + resources: ["crontabs"] + verbs: ["get", "list", "watch"] +``` + +#### Roleの例 + +次の例は、RoleオブジェクトまたはClusterRoleオブジェクトからの抜粋であり、`rules`セクションのみを示しています。 + +`"pods"`の読み取りを許可する +{{< glossary_tooltip text="API Group" term_id="api-group" >}}。 + +```yaml +rules: +- apiGroups: [""] + # + # HTTPレベルでの、Podにアクセスするリソースの名前 + # オブジェクトは"pods" + resources: ["pods"] + verbs: ["get", "list", "watch"] +``` + +APIグループ`" extensions "`と `" apps "` の両方で、Deploymentsへの読み取り/書き込みを許可します。 +(HTTPレベルでURLのリソース部分に`"deployments"`を持つオブジェクトで) + +```yaml +rules: +- apiGroups: ["extensions", "apps"] + # + # HTTPレベルでの、Deploymentにアクセスするリソースの名前 + # オブジェクトは"deployments" + resources: ["deployments"] + verbs: ["get", "list", "watch", "create", "update", "patch", "delete"] +``` + +コアAPIグループのPodの読み取り、および`"batch"`または`"extensions"`APIグループのJobリソースの読み取りまたは書き込みを許可します。 + + +```yaml +rules: +- apiGroups: [""] + # + # HTTPレベルでの、Podにアクセスするリソースの名前 + # オブジェクトは"pods" + resources: ["pods"] + verbs: ["get", "list", "watch"] +- apiGroups: ["batch", "extensions"] + # + # HTTPレベルでの、Jobにアクセスするリソースの名前 + # オブジェクトは"jobs" + resources: ["jobs"] + verbs: ["get", "list", "watch", "create", "update", "patch", "delete"] +``` + +「my-config」という名前のConfigMapの読み取りを許可します( +単一のNamespace内の単一のConfigMapに制限するRoleBinding) + +```yaml +rules: +- apiGroups: [""] + # + # HTTPレベルでの、ConfigMapにアクセスするリソースの名前 + # オブジェクトは"configmaps" + resources: ["configmaps"] + resourceNames: ["my-config"] + verbs: ["get"] +``` + + +コアグループ内のリソース `"nodes"`の読み取りを許可します(Nodeはクラスタースコープであり、ClusterRoleBindingが効果的であるため、ClusterRoleにバインドされている必要があります)。 + +```yaml +rules: +- apiGroups: [""] + # + # HTTPレベルでの、Nodeにアクセスするリソースの名前 + # オブジェクトは"nodes" + resources: ["nodes"] + verbs: ["get", "list", "watch"] +``` + +非リソースエンドポイント `/ healthz`およびすべてのサブパス(ClusterRoleBindingが効果的であるため、ClusterRoleにバインドされている必要があります)のGETおよびPOSTリクエストを許可します。 + +```yaml +rules: +- nonResourceURLs: ["/healthz", "/healthz/*"] # nonResourceURLの「*」はサフィックスグロブマッチです + verbs: ["get", "post"] +``` + +### subjectsを参照する + +RoleBindingまたはClusterRoleBindingは、Roleをsubjectsにバインドします。subjectsはグループ、ユーザー、または{{< glossary_tooltip text="ServiceAccounts" term_id="service-account" >}}にすることができます。 + +Kubernetesはユーザー名を文字列として表します。 +これらは次のようにできます。「alice」などの単純な名前。「bob@example.com」のような電子メール形式の名前。または文字列として表される数値のユーザーID。 認証が必要な形式のユーザー名を生成するように[認証モジュール](https://kubernetes.io/ja/docs/reference/access-authn-authz/authentication/)を構成するかどうかは、クラスター管理者が決定します。 + +{{< caution >}} +プレフィックス`system:`はKubernetesシステムで使用するために予約されているため、誤って`system:`で始まる名前のユーザーまたはグループが存在しないようにする必要があります。 +この特別なプレフィックスを除いて、RBAC承認システムでは、ユーザー名の形式は問いません。 +{{< /caution >}} + +Kubernetesでは、Authenticatorモジュールがグループ情報を提供します。 +ユーザーと同様に、グループは文字列として表され、その文字列には、プレフィックス`system:`が予約されていることを除いて、フォーマット要件はありません。 + +[ServiceAccount](/docs/tasks/configure-pod-container/configure-service-account/)の名前はプレフィックス`system:serviceaccount:`が付いており、名前のプレフィックス`system:serviceaccounts:`が付いているグループに属しています。 + +{{< note >}} +- `system:serviceaccount:` (単数)は、サービスアカウントのユーザー名のプレフィックスです。 +- `system:serviceaccounts:`(複数)は、サービスアカウントグループのプレフィックスです。 +{{< /note >}} + +#### RoleBindingの例 {#role-binding-examples} + +次の例は`RoleBinding`、`subjects`セクションのみを示す抜粋です。 + +`alice@example.com`という名前のユーザーの場合。 + +```yaml +subjects: +- kind: User + name: "alice@example.com" + apiGroup: rbac.authorization.k8s.io +``` + +`frontend-admins`という名前のグループの場合。 + +```yaml +subjects: +- kind: Group + name: "frontend-admins" + apiGroup: rbac.authorization.k8s.io +``` + +Namespace「kube-system」のデフォルトのサービスアカウントの場合。 + +```yaml +subjects: +- kind: ServiceAccount + name: default + namespace: kube-system +``` + +Namespace「qa」の全てのサービスアカウントの場合。 + +```yaml +subjects: +- kind: Group + name: system:serviceaccounts:qa + apiGroup: rbac.authorization.k8s.io +``` + +任意のNamespaceの全てのサービスアカウントの場合。 + +```yaml +subjects: +- kind: Group + name: system:serviceaccounts + apiGroup: rbac.authorization.k8s.io +``` + +すべての認証済みユーザーの場合。 + +```yaml +subjects: +- kind: Group + name: system:authenticated + apiGroup: rbac.authorization.k8s.io +``` + +認証されていないすべてのユーザーの場合。 + +```yaml +subjects: +- kind: Group + name: system:unauthenticated + apiGroup: rbac.authorization.k8s.io +``` + +すべてのユーザーの場合。 + +```yaml +subjects: +- kind: Group + name: system:authenticated + apiGroup: rbac.authorization.k8s.io +- kind: Group + name: system:unauthenticated + apiGroup: rbac.authorization.k8s.io +``` + +## デフォルトRoleとClusterRoleBinding + +APIサーバーは、デフォルトのClusterRoleオブジェクトとClusterRoleBindingオブジェクトのセットを作成します。 +これらの多くにはプレフィックス`system:`が付いています。これは、リソースがクラスターコントロールプレーンによって直接管理されることを示しています。 +デフォルトのすべてのClusterRoleおよびClusterRoleBindingには、ラベル`kubernetes.io/bootstrapping=rbac-defaults`が付いています。 + +{{< caution >}} +プレフィックスとして`system:`を含む名前で、ClusterRolesおよびClusterRoleBindingsを変更する場合は注意してください。 +これらのリソースを変更すると、クラスターが機能しなくなる可能性があります。 +{{< /caution >}} + +### 自動調整 + +起動するたびに、APIサーバーはデフォルトのClusterRoleを不足している権限で更新し、 +デフォルトのClusterRoleBindingを不足しているsubjectsで更新します。 +これにより、誤った変更をクラスタが修復できるようになり、新しいKubernetesリリースで権限とsubjectsが変更されても、 +RoleとRoleBindingを最新の状態に保つことができます。 + +この調整を無効化するには`rbac.authorization.kubernetes.io/autoupdate`をデフォルトのClusterRoleまたはRoleBindingのアノテーションを`false`に設定します。 +デフォルトの権限と subjectsがないと、クラスターが機能しなくなる可能性があることに注意してください。 + +RBAC authorizerが有効な場合、自動調整はデフォルトで有効になっています。 + +### APIディスカバリーRole {#discovery-roles} + +デフォルトのRoleBindingでは、認証されていないユーザーと認証されたユーザーが、パブリックアクセスが安全であると見なされるAPI情報(CustomResourceDefinitionを含む)の読み取りを認可しています。匿名の非認証アクセスを無効にするには、APIサーバー構成に`--anonymous-auth=false` 追加します。 + +`kubectl`の実行によってこれらのRoleの構成を表示するには。 + +```shell +kubectl get clusterroles system:discovery -o yaml +``` + +{{< note >}} +ClusterRoleを編集すると、変更が[自動調整](#自動調整)によるAPIサーバーの再起動時に上書きされます。この上書きを回避するにはRoleを手動で編集しないか、自動調整を無効にします。 +{{< /note >}} + + + + + + + + + + + + + + + + + + + + + + + + +
    Kubernetes RBAC APIディスカバリーRole
    デフォルトのClusterRoleデフォルトのClusterRoleBinding説明
    system:basic-usersystem:authenticated groupユーザーに、自身に関する基本情報への読み取り専用アクセスを許可します。v1.14より前は、このRoleはデフォルトでsystem:unauthenticatedにもバインドされていました。
    system:discoverysystem:authenticated groupAPIレベルのディスカバリーとネゴシエーションに必要なAPIディスカバリーエンドポイントへの読み取り専用アクセスを許可します。v1.14より前は、このRoleはデフォルトでsystem:unauthenticatedにもバインドされていました。
    system:public-info-viewersystem:authenticated and system:unauthenticated groupsクラスターに関する機密情報以外への読み取り専用アクセスを許可します。Kubernetes v1.14で導入されました。
    + +### ユーザー向けRole + +一部のデフォルトClusterRolesにはプレフィックス`system:`が付いていません。これらは、ユーザー向けのroleを想定しています。それらは、スーパーユーザのRole(`cluster-admin`)、ClusterRoleBindingsを使用してクラスタ全体に付与されることを意図しているRole、そしてRoleBindings(`admin`, `edit`, `view`)を使用して、特定のNamespace内に付与されることを意図しているRoleを含んでいます。 + +ユーザー向けのClusterRolesは[ClusterRoleの集約](#集約clusterrole)を使用して、管理者がこれらのClusterRolesにカスタムリソースのルールを含めることができるようにします。ルールを`admin`、`edit`、または`view` Roleに追加するには、次のラベルの一つ以上でClusterRoleを作成します。 + +```yaml +metadata: + labels: + rbac.authorization.k8s.io/aggregate-to-admin: "true" + rbac.authorization.k8s.io/aggregate-to-edit: "true" + rbac.authorization.k8s.io/aggregate-to-view: "true" +``` + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    デフォルトのClusterRoleデフォルトのClusterRoleBinding説明
    cluster-adminsystem:masters groupスーパーユーザーが任意のリソースで任意のアクションを実行できるようにします。 +ClusterRoleBindingで使用すると、クラスター内およびすべてのNamespace内のすべてのリソースを完全に制御できます。 +RoleBindingで使用すると、Namespace自体を含む、RoleBindingのNamespace内のすべてのリソースを完全に制御できます。
    adminNoneRoleBindingを使用してNamespace内で付与することを想定した、管理者アクセスを許可します。 +RoleBindingで使用した場合、Namespace内にRoleとRoleBindingを作成する機能を含め、Namespaceのほとんどのリソースへの読み取り/書き込みアクセスを許可します。 +このRoleは、リソースクォータまたはNamespace自体への書き込みアクセスを許可しません。
    editNoneNamespace内のほとんどのオブジェクトへの読み取り/書き込みアクセスを許可します。 + +このRoleは、RoleまたはRoleBindingの表示または変更を許可しません。 +ただし、このRoleでは、Secretsにアクセスして、Namespace内の任意のServiceAccountとしてPodsを実行できるため、Namespace内の任意のServiceAccountのAPIアクセスレベルを取得するために使用できます。
    viewNoneNamespace内のほとんどのオブジェクトを表示するための読み取り専用アクセスを許可します。 +RoleまたはRoleBindingは表示できません。 + +Secretsの内容を読み取るとNamespaceのServiceAccountのクレデンシャルにアクセスできるため、このRoleではSecretsの表示は許可されません。これにより、Namespace内の任意のServiceAccountとしてAPIアクセスが許可されます(特権昇格の形式)。
    + +### コアコンポーネントのRole + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    デフォルトのClusterRoleデフォルトのClusterRoleBinding説明
    system:kube-schedulersystem:kube-scheduler user{{< glossary_tooltip term_id="kube-scheduler" text="scheduler" >}}コンポーネントが必要とするリソースへのアクセスを許可します。
    system:volume-schedulersystem:kube-scheduler userkube-scheduleコンポーネントが必要とするリソースへのアクセスを許可します。
    system:kube-controller-managersystem:kube-controller-manager user{{< glossary_tooltip term_id="kube-controller-manager" text="controller manager" >}}コンポーネントが必要とするリソースへのアクセスを許可します。 +個々のコントローラーに必要な権限については、組み込みコントローラーのRoleで詳しく説明しています。
    system:nodeNoneすべてのsecretへの読み取りアクセス、すべてのポッドステータスオブジェクトへの書き込みアクセスなど、kubeletが必要とするリソースへのアクセスを許可します。 + +system:nodeRoleの代わりにNode authorizerと NodeRestriction admission pluginを使用し、それらで実行するようにスケジュールされたPodに基づいてkubeletへのAPIアクセスを許可する必要があります。 + +system:nodeのRoleは、V1.8より前のバージョンからアップグレードしたKubernetesクラスターとの互換性のためだけに存在します。 +
    system:node-proxiersystem:kube-proxy user{{< glossary_tooltip term_id="kube-proxy" text="kube-proxy" >}}コンポーネントが必要とするリソースへのアクセスを許可します。
    + +### 他のコンポーネントのRole + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    デフォルトのClusterRoleデフォルトのClusterRoleBinding説明
    system:auth-delegatorNone委任された認証と認可のチェックを許可します。 +これは一般に、認証と認可を統合するためにアドオンAPIサーバーで使用されます。
    system:heapsterNoneHeapsterコンポーネントのRole(非推奨)。
    system:kube-aggregatorNonekube-aggregatorコンポーネントのRole。
    system:kube-dnskube-systemNamespaceのサービスアカウントkube-dnskube-dnsコンポーネントのRole。
    system:kubelet-api-adminNonekubelet APIへのフルアクセスを許可します。
    system:node-bootstrapperNonekubelet TLS bootstrappingの実行に必要なリソースへのアクセスを許可します。
    system:node-problem-detectorNonenode-problem-detectorコンポーネントのRole。
    system:persistent-volume-provisionerNoneほとんどのdynamic volume provisionersが必要とするリソースへのアクセスを許可します。
    + +### 組み込みコントローラーのRole {#controller-roles} + +Kubernetes {{< glossary_tooltip term_id="kube-controller-manager" text="controller manager" >}}は、Kubernetesコントロールプレーンに組み込まれている{{< glossary_tooltip term_id="controller" text="controllers" >}}を実行します。 +`--use-service-account-credentials`を指定して呼び出すと、kube-controller-manager個別のサービスアカウントを使用して各コントローラーを起動します。 +組み込みコントローラーごとに、プレフィックス`system:controller:`付きの対応するRoleが存在します。 +コントローラーマネージャーが`--use-service-account-credentials`で開始されていない場合、コントローラマネージャーは、関連するすべてのRoleを付与する必要がある自身のクレデンシャルを使用して、すべてのコントロールループを実行します。 +これらのRoleは次のとおりです。 + +* `system:controller:attachdetach-controller` +* `system:controller:certificate-controller` +* `system:controller:clusterrole-aggregation-controller` +* `system:controller:cronjob-controller` +* `system:controller:daemon-set-controller` +* `system:controller:deployment-controller` +* `system:controller:disruption-controller` +* `system:controller:endpoint-controller` +* `system:controller:expand-controller` +* `system:controller:generic-garbage-collector` +* `system:controller:horizontal-pod-autoscaler` +* `system:controller:job-controller` +* `system:controller:namespace-controller` +* `system:controller:node-controller` +* `system:controller:persistent-volume-binder` +* `system:controller:pod-garbage-collector` +* `system:controller:pv-protection-controller` +* `system:controller:pvc-protection-controller` +* `system:controller:replicaset-controller` +* `system:controller:replication-controller` +* `system:controller:resourcequota-controller` +* `system:controller:root-ca-cert-publisher` +* `system:controller:route-controller` +* `system:controller:service-account-controller` +* `system:controller:service-controller` +* `system:controller:statefulset-controller` +* `system:controller:ttl-controller` + +## 特権昇格の防止とブートストラップ + +RBAC APIは、RoleまたはRoleBindingを編集することにより、ユーザーが特権を昇格するのを防ぎます。 +これはAPIレベルで適用されるため、RBAC authorizerが使用されていない場合でも適用されます。 + +### Roleの作成または更新に関する制限 + +次の項目1つ以上が当てはまる場合にのみ、Roleを作成/更新できます。 + +1. 変更対象のオブジェクトと同じスコープで、Roleに含まれるすべての権限を既に持っている(ClusterRoleの場合はクラスター全体。Roleの場合は、同じNamespace内またはクラスター全体)。 +2. `rbac.authorization.k8s.io`APIグループの` roles`または`clusterroles`リソースで` escalate` verbを実行する明示的な権限が付与されている。 + +たとえば、 `user-1`にクラスター全体でSecretsを一覧表示する権限がない場合、それらにその権限を含むClusterRoleを作成できません。 +ユーザーがRoleを作成/更新できるようにするには、以下のいずれかを実施します。 + +1. 必要に応じて、RoleオブジェクトまたはClusterRoleオブジェクトを作成/更新できるRoleを付与する。 +2. 作成/更新するRoleに特定の権限を含む権限を付与する。 + * 暗黙的に、これらの権限を付与することにより(自分自身が付与されていない権限でRoleまたはClusterRoleを作成または変更しようとすると、APIリクエストは禁止されます)。 + * または、`rbac.authorization.k8s.io`APIグループの` roles`または `clusterroles`リソースで` escalate` verbを実行する権限を与えることにより、 `Role`または` ClusterRole`で権限を指定することを明示的に許可する + +### RoleBindingの作成または更新に関する制限 + +参照されるRoleに含まれるすべての権限を(RoleBindingと同じスコープで)すでに持っている場合、 +*または*参照されたRoleで`bind` verbを実行する認可されている場合のみ、RoleBindingを作成/更新できます。 +たとえば、 `user-1`にクラスター全体でSecretsを一覧表示する権限がない場合、ClusterRoleBindingを作成してもRoleにその権限を付与できません。 +ユーザーがRoleBindingを作成/更新できるようにするには、以下のいずれかを実施します。 + +1. 必要に応じて、RoleBindingまたはClusterRoleBindingオブジェクトを作成/更新できるようにする役割を付与する。 +2. 特定の役割をバインドするために必要なアクセス許可を付与する。 + * 暗黙的に、Roleに含まれる権限を付与することによって。 + * 明示的に、特定のRole(またはClusterRole)で `bind` verbを実行する許可を与えることによって。 + +たとえば、このClusterRoleとRoleBindingを使用すると、 `user-1`は他のユーザーにNamespace` user-1-namespace`の `admin`、` edit`、および `view`Roleを付与します。 + +```yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: role-grantor +rules: +- apiGroups: ["rbac.authorization.k8s.io"] + resources: ["rolebindings"] + verbs: ["create"] +- apiGroups: ["rbac.authorization.k8s.io"] + resources: ["clusterroles"] + verbs: ["bind"] + resourceNames: ["admin","edit","view"] +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: role-grantor-binding + namespace: user-1-namespace +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: role-grantor +subjects: +- apiGroup: rbac.authorization.k8s.io + kind: User + name: user-1 +``` + +最初のRoleとRoleBindingをブートストラップするときは、最初のユーザーがまだ持っていない権限を付与する必要があります。 +初期RoleとRoleBindingをブートストラップするには、以下のいずれかを実施します。 + +* 「system：masters」グループのクレデンシャルを使用します。このグループは、デフォルトのBindingによって「cluster-admin」スーパーユーザーRoleにバインドされています。 +* APIサーバーが安全でないポート(`--insecure-port`)を有効にして実行されている場合、そのポートを介してのAPI呼び出しもできます。これにより、認証や認可が実行されません。 + +## コマンドラインユーティリティー + +### `kubectl create role` + +以下に、単一のNamespace内で権限を定義するRoleオブジェクトをいくつか例として作成します。 + +* ユーザーがポッドで `get`、` watch`、および `list`を実行できるように「pod-reader」という名前のRoleを作成します。 + + ```shell + kubectl create role pod-reader --verb=get --verb=list --verb=watch --resource=pods + ``` + +* resourceNamesを指定して、「pod-reader」という名前のRoleを作成します。 + + ```shell + kubectl create role pod-reader --verb=get --resource=pods --resource-name=readablepod --resource-name=anotherpod + ``` + +* apiGroupsを指定して「foo」という名前のRoleを作成します。 + + ```shell + kubectl create role foo --verb=get,list,watch --resource=replicasets.apps + ``` + +* サブリソースの権限を持つ「foo」という名前のRoleを作成します。 +* Create a Role named "foo" with subresource permissions: + + ```shell + kubectl create role foo --verb=get,list,watch --resource=pods,pods/status + ``` +* 特定の名前のリソースを取得/更新する権限を持つ「my-component-lease-holder」という名前のRoleを作成します。 + + ```shell + kubectl create role my-component-lease-holder --verb=get,list,watch,update --resource=lease --resource-name=my-component + ``` + +### `kubectl create clusterrole` + +以下にClusterRoleをいくつか例として作成します。 + +* ユーザーがポッドに対して`get`、` watch`、および `list`を実行できるようにする「pod-reader」という名前のClusterRoleを作成します。 + + ```shell + kubectl create clusterrole pod-reader --verb=get,list,watch --resource=pods + ``` + +* resourceNamesを指定して、「pod-reader」という名前のClusterRoleを作成します。 + + ```shell + kubectl create clusterrole pod-reader --verb=get --resource=pods --resource-name=readablepod --resource-name=anotherpod + ``` + +* apiGroupsを指定して「foo」という名前のClusterRoleを作成します。 + + ```shell + kubectl create clusterrole foo --verb=get,list,watch --resource=replicasets.apps + ``` + +* サブリソースの権限を持つ「foo」という名前のClusterRoleを作成します。 + + ```shell + kubectl create clusterrole foo --verb=get,list,watch --resource=pods,pods/status + ``` + +* nonResourceURLを指定して「foo」という名前のClusterRoleを作成します。 + + ```shell + kubectl create clusterrole "foo" --verb=get --non-resource-url=/logs/* + ``` + +* aggregationRuleを指定して、「monitoring」という名前のClusterRoleを作成します。 + + ```shell + kubectl create clusterrole monitoring --aggregation-rule="rbac.example.com/aggregate-to-monitoring=true" + ``` + +### `kubectl create rolebinding` + +以下に、特定のNamespace内でRoleまたはClusterRoleをいくつか例として付与します。 + +* Namespace「acme」内で、「admin」ClusterRoleの権限を「bob」という名前のユーザーに付与します。 + + ```shell + kubectl create rolebinding bob-admin-binding --clusterrole=admin --user=bob --namespace=acme + ``` + +* Namespace「acme」内で、ClusterRole「view」へのアクセス許可を「myapp」というNamespace「acme」のサービスアカウントに付与します。 + + ```shell + kubectl create rolebinding myapp-view-binding --clusterrole=view --serviceaccount=acme:myapp --namespace=acme + ``` + +* Namespace「acme」内で、ClusterRole「view」へのアクセス許可を「myapp」というNamespace「myappnamespace」のサービスアカウントに付与します。 + + ```shell + kubectl create rolebinding myappnamespace-myapp-view-binding --clusterrole=view --serviceaccount=myappnamespace:myapp --namespace=acme + ``` + +### `kubectl create clusterrolebinding` + +以下に、クラスタ全体(すべてのNamespace)にClusterRoleをいくつか例として付与します。 + +* クラスター全体で、ClusterRole「cluster-admin」へのアクセス許可を「root」という名前のユーザーに付与します。 + + ```shell + kubectl create clusterrolebinding root-cluster-admin-binding --clusterrole=cluster-admin --user=root + ``` + +* クラスター全体で、ClusterRole「system：node-proxier」へのアクセス許可を「system：kube-proxy」という名前のユーザーに付与します。 + + ```shell + kubectl create clusterrolebinding kube-proxy-binding --clusterrole=system:node-proxier --user=system:kube-proxy + ``` + +* クラスター全体で、ClusterRole「view」へのアクセス許可を、Namespace「acme」の「myapp」という名前のサービスアカウントに付与します。 + + ```shell + kubectl create clusterrolebinding myapp-view-binding --clusterrole=view --serviceaccount=acme:myapp + ``` + +### `kubectl auth reconcile` {#kubectl-auth-reconcile} + +マニフェストファイルから `rbac.authorization.k8s.io/v1`APIオブジェクトを作成または更新します。 + +欠落しているオブジェクトが作成され、必要に応じて、Namespaceに属するオブジェクト用にオブジェクトを含むNamespaceが作成されます。 + +既存のRoleが更新され、入力オブジェクトに権限が含まれるようになります。 +`--remove-extra-permissions`が指定されている場合は、余分な権限を削除します。 + +既存のBindingが更新され、入力オブジェクトにsubjectsが含まれるようになります。 +`--remove-extra-subjects`が指定されている場合は、余分な件名を削除します。 + +以下、例として。 + +* RBACオブジェクトのマニフェストファイルをテストとして適用し、行われる変更を表示します。 + + ``` + kubectl auth reconcile -f my-rbac-rules.yaml --dry-run=client + ``` + +* RBACオブジェクトのマニフェストファイルを適用し、(Role内の)追加のアクセス許可と(Binding内の)追加のsubjectsを保持します。 + + ```shell + kubectl auth reconcile -f my-rbac-rules.yaml + ``` + +* RBACオブジェクトのマニフェストファイルを適用し、(Role内の)余分なアクセス許可と(Binding内の)余分なsubjectsを削除します。 + +## ServiceAccount権限 {#service-account-permissions} + +デフォルトのRBACポリシーは、コントロールプレーンコンポーネント、ノード、 +およびコントローラーをスコープとして権限を付与しますが、 Namespace`kube-system`外のサービスアカウントには*no permissions*で付与します +(すべての認証されたユーザーに与えられたディスカバリー権限に関わらず)。 + +これにより、必要に応じて特定のServiceAccountに特定のRoleを付与できます。 +きめ細かいRoleBindingはセキュリティを強化しますが、管理にはより多くの労力が必要です。 +より広範な権限は、不必要な(そして潜在的にエスカレートする)APIアクセスをServiceAccountsに与える可能性がありますが、管理が簡単です。 + +アプローチを最も安全なものから最も安全でないものの順に並べると、次のとおりです。 + +1. アプリケーション固有のサービスアカウントにRoleを付与する(ベストプラクティス) + これには、アプリケーションがpodのspec、そして作成するサービスアカウント(API、アプリケーションマニフェスト、 `kubectl create serviceaccount`などを介して）で`serviceAccountName`を指定する必要があります。 + たとえば、「my-namespace」内の読み取り専用権限を「my-sa」サービスアカウントに付与します。 + + ```shell + kubectl create rolebinding my-sa-view \ + --clusterrole=view \ + --serviceaccount=my-namespace:my-sa \ + --namespace=my-namespace + ``` + +2. あるNamespaceのサービスアカウント「default」にRoleを付与します + + アプリケーションが `serviceAccountName`を指定しない場合、サービスアカウント「default」を使用します。 + + {{< note >}} + サービスアカウント「default」に付与された権限は、`serviceAccountName`を指定しないNamespace内のすべてのポッドで利用できます。 + {{< /note >}} + + たとえば、「my-namespace」内の読み取り専用権限をサービスアカウント「default」に付与します。 + + ```shell + kubectl create rolebinding default-view \ + --clusterrole=view \ + --serviceaccount=my-namespace:default \ + --namespace=my-namespace + ``` + + 多くの[アドオン](https://kubernetes.io/docs/concepts/cluster-administration/addons/)は、 + Namespace`kube-system`のサービスアカウント「default」として実行されます。 + これらのアドオンをスーパーユーザーアクセスでの実行を許可するには、Namespace`kube-system`のサービスアカウント「default」のcluster-admin権限を付与します。 + + {{< caution >}} + これを有効にすると、 Namespace`kube-systemにクラスターのAPIへのスーパーユーザーアクセス許可するSecretsが含まれます。 + {{< /caution >}} + + ```shell + kubectl create clusterrolebinding add-on-cluster-admin \ + --clusterrole=cluster-admin \ + --serviceaccount=kube-system:default + ``` + +3. Namespace内のすべてのサービスアカウントにRoleを付与します + + Namespace内のすべてのアプリケーションにRoleを持たせたい場合は、使用するサービスアカウントに関係なく、 + そのNamespaceのサービスアカウントグループにRoleを付与できます。 + + たとえば、「my-namespace」内の読み取り専用アクセス許可を、そのNamespace内のすべてのサービスアカウントに付与します。 + + ```shell + kubectl create rolebinding serviceaccounts-view \ + --clusterrole=view \ + --group=system:serviceaccounts:my-namespace \ + --namespace=my-namespace + ``` + +4. クラスター全体のすべてのサービスアカウントに制限されたRoleを付与します(お勧めしません) + + Namespaceごとのアクセス許可を管理したくない場合は、すべてのサービスアカウントにクラスター全体の役割を付与できます。 + + たとえば、クラスター内のすべてのサービスアカウントに、すべてのNamespaceで読み取り専用のアクセス許可を付与します。 + + ```shell + kubectl create clusterrolebinding serviceaccounts-view \ + --clusterrole=view \ + --group=system:serviceaccounts + ``` + +5. クラスター全体のすべてのサービスアカウントへのスーパーユーザーアクセスを許可します。(強くお勧めしません) + + 権限の分割をまったく考慮しない場合は、すべてのサービスアカウントにスーパーユーザーアクセスを許可できます。 + + {{< warning >}} + これにより、すべてのアプリケーションにクラスターへのフルアクセスが許可され、Secretsの読み取りアクセス権(または任意のポッドを作成する機能)を持つユーザーに、クラスターへのフルアクセスが許可されます。 + {{< /warning >}} + + ```shell + kubectl create clusterrolebinding serviceaccounts-cluster-admin \ + --clusterrole=cluster-admin \ + --group=system:serviceaccounts + ``` + +## ABACからアップグレードする + +以前は古いバージョンのKubernetesを実行していたクラスターは、すべてのサービスアカウントに完全なAPIアクセスを許可するなど、permissiveなABACポリシーを使用することがよくありました。 + +デフォルトのRBACポリシーは、コントロールプレーンコンポーネント、ノード、 +およびコントローラーをスコープとして権限を付与しますが、 Namespace`kube-system`外のサービスアカウントには*no permissions*で付与します +(すべての認証されたユーザーに与えられたディスカバリー権限に関わらず)。 + +これははるかに安全ですが、API権限を自動的に受け取ることを期待している既存のワークロードを混乱させる可能性があります。 +この移行を管理するための2つのアプローチは次のとおりです。 + +### 並行認可 + + +RBACとABACの両方のauthorizerを実行し、[legacy ABAC policy](/docs/reference/access-authn-authz/abac/#policy-file-format)を含むポリシーファイルを指定します。 + +``` +--authorization-mode=...,RBAC,ABAC --authorization-policy-file=mypolicy.json +``` + +最初のコマンドラインオプションを詳細に説明すると、Nodeなどの以前のauthorizerが +要求を拒否すると、RBAC authorizerはAPI要求を認可しようとします。 RBACの場合 +また、そのAPI要求を拒否すると、ABAC authorizerが実行されます。これにより、すべてのリクエストが +RBACまたはABACポリシーの*いずれか*で許可されます。 + +RBACコンポーネントのログレベルが5以上でkube-apiserverを実行した場合(`--vmodule = rbac * = 5`または` --v = 5`)、APIサーバーログでRBACの拒否を確認できます(プレフィックスは「RBAC」)。 +その情報を使用して、どのRoleをどのユーザー、グループ、またはサービスアカウントに付与する必要があるかを判断できます。 + +[サービスアカウントに付与されたRole](#service-account-permissions)を取得し、 +ワークロードがサーバーログにRBACの拒否メッセージがない状態で実行されている場合は、ABAC authorizerを削除できます。 + +### Permissive RBAC権限 + +RBACRoleBindingを使用して、permissive ABACポリシーを複製できます。 + +{{< warning >}} +次のポリシーでは、**すべて**のサービスアカウントがクラスター管理者としてふるまうことを許可しています。 +コンテナで実行されているアプリケーションは、サービスアカウントのクレデンシャルを自動的に受け取ります。 +secretsの表示や権限の変更など、APIに対して任意のアクションを実行できます。 +これは推奨されるポリシーではありません。 + +```shell +kubectl create clusterrolebinding permissive-binding \ + --clusterrole=cluster-admin \ + --user=admin \ + --user=kubelet \ + --group=system:serviceaccounts +``` +{{< /warning >}} + +RBACの使用に移行後、クラスターが情報セキュリティのニーズを確実に満たすように、アクセスコントロールを調整する必要があります。 diff --git a/content/ko/docs/concepts/architecture/control-plane-node-communication.md b/content/ko/docs/concepts/architecture/control-plane-node-communication.md index 7d5e86837e..2fdaf7599c 100644 --- a/content/ko/docs/concepts/architecture/control-plane-node-communication.md +++ b/content/ko/docs/concepts/architecture/control-plane-node-communication.md @@ -16,7 +16,7 @@ aliases: ## 노드에서 컨트롤 플레인으로의 통신 쿠버네티스에는 "허브 앤 스포크(hub-and-spoke)" API 패턴을 가지고 있다. 노드(또는 노드에서 실행되는 파드들)의 모든 API 사용은 API 서버에서 종료된다(다른 컨트롤 플레인 컴포넌트 중 어느 것도 원격 서비스를 노출하도록 설계되지 않았다). API 서버는 하나 이상의 클라이언트 [인증](/docs/reference/access-authn-authz/authentication/) 형식이 활성화된 보안 HTTPS 포트(일반적으로 443)에서 원격 연결을 수신하도록 구성된다. -특히 [익명의 요청](/docs/reference/access-authn-authz/authentication/#anonymous-requests) 또는 [서비스 어카운트 토큰](/docs/reference/access-authn-authz/authentication/#service-account-tokens)이 허용되는 경우, 하나 이상의 [권한 부여](/docs/reference/access-authn-authz/authorization/) 형식을 사용해야 한다. +특히 [익명의 요청](/docs/reference/access-authn-authz/authentication/#anonymous-requests) 또는 [서비스 어카운트 토큰](/docs/reference/access-authn-authz/authentication/#service-account-tokens)이 허용되는 경우, 하나 이상의 [권한 부여](/ko/docs/reference/access-authn-authz/authorization/) 형식을 사용해야 한다. 노드는 유효한 클라이언트 자격 증명과 함께 API 서버에 안전하게 연결할 수 있도록 클러스터에 대한 공개 루트 인증서로 프로비전해야 한다. 예를 들어, 기본 GKE 배포에서, kubelet에 제공되는 클라이언트 자격 증명은 클라이언트 인증서 형식이다. kubelet 클라이언트 인증서의 자동 프로비저닝은 [kubelet TLS 부트스트랩](/docs/reference/command-line-tools-reference/kubelet-tls-bootstrapping/)을 참고한다. diff --git a/content/ko/docs/concepts/cluster-administration/_index.md b/content/ko/docs/concepts/cluster-administration/_index.md index cde1b093b5..daa1dfd702 100755 --- a/content/ko/docs/concepts/cluster-administration/_index.md +++ b/content/ko/docs/concepts/cluster-administration/_index.md @@ -47,11 +47,11 @@ no_list: true * [쿠버네티스 컨테이너 환경](/ko/docs/concepts/containers/container-environment/)은 쿠버네티스 노드에서 Kubelet으로 관리하는 컨테이너에 대한 환경을 설명한다. -* [쿠버네티스 API에 대한 접근 제어](/docs/reference/access-authn-authz/controlling-access/)는 사용자와 서비스 어카운트에 대한 권한을 설정하는 방법을 설명한다. +* [쿠버네티스 API에 대한 접근 제어](/ko/docs/reference/access-authn-authz/controlling-access/)는 사용자와 서비스 어카운트에 대한 권한을 설정하는 방법을 설명한다. * [인증](/docs/reference/access-authn-authz/authentication/)은 다양한 인증 옵션을 포함한 쿠버네티스에서의 인증에 대해 설명한다. -* [인가](/docs/reference/access-authn-authz/authorization/)는 인증과는 별개로, HTTP 호출 처리 방법을 제어한다. +* [인가](/ko/docs/reference/access-authn-authz/authorization/)는 인증과는 별개로, HTTP 호출 처리 방법을 제어한다. * [어드미션 컨트롤러 사용하기](/docs/reference/access-authn-authz/admission-controllers/)는 인증과 권한 부여 후 쿠버네티스 API 서버에 대한 요청을 가로채는 플러그인에 대해 설명한다. diff --git a/content/ko/docs/concepts/configuration/configmap.md b/content/ko/docs/concepts/configuration/configmap.md index 7b1bf21b9b..0c89f83bdf 100644 --- a/content/ko/docs/concepts/configuration/configmap.md +++ b/content/ko/docs/concepts/configuration/configmap.md @@ -214,6 +214,7 @@ kubelet은 모든 주기적인 동기화에서 마운트된 컨피그맵이 최 전파 지연은 선택한 캐시 유형에 따라 달라질 수 있다(전파 지연을 지켜보거나, 캐시의 ttl 또는 0에 상응함). +환경 변수로 사용되는 컨피그맵은 자동으로 업데이트되지 않으며 파드를 다시 시작해야 한다. ## 변경할 수 없는(immutable) 컨피그맵 {#configmap-immutable} {{< feature-state for_k8s_version="v1.19" state="beta" >}} diff --git a/content/ko/docs/concepts/configuration/manage-resources-containers.md b/content/ko/docs/concepts/configuration/manage-resources-containers.md index 41e6422e75..1b8a3d1393 100644 --- a/content/ko/docs/concepts/configuration/manage-resources-containers.md +++ b/content/ko/docs/concepts/configuration/manage-resources-containers.md @@ -134,7 +134,6 @@ spec: containers: - name: app image: images.my-company.example/app:v4 - env: resources: requests: memory: "64Mi" diff --git a/content/ko/docs/concepts/configuration/pod-priority-preemption.md b/content/ko/docs/concepts/configuration/pod-priority-preemption.md index ec97dec798..ea978260f3 100644 --- a/content/ko/docs/concepts/configuration/pod-priority-preemption.md +++ b/content/ko/docs/concepts/configuration/pod-priority-preemption.md @@ -8,7 +8,7 @@ weight: 70 {{< feature-state for_k8s_version="v1.14" state="stable" >}} -[파드](/ko/docs/concepts/workloads/pods/pod/)는 _우선순위_ 를 가질 수 있다. 우선순위는 +[파드](/ko/docs/concepts/workloads/pods/)는 _우선순위_ 를 가질 수 있다. 우선순위는 다른 파드에 대한 상대적인 파드의 중요성을 나타낸다. 파드를 스케줄링할 수 없는 경우, 스케줄러는 우선순위가 낮은 파드를 선점(축출)하여 보류 중인 파드를 스케줄링할 수 있게 한다. diff --git a/content/ko/docs/concepts/configuration/secret.md b/content/ko/docs/concepts/configuration/secret.md index 59a7fca1ed..a934d173d1 100644 --- a/content/ko/docs/concepts/configuration/secret.md +++ b/content/ko/docs/concepts/configuration/secret.md @@ -35,6 +35,12 @@ weight: 30 - [컨테이너 환경 변수](#시크릿을-환경-변수로-사용하기)로써 사용. - 파드의 [이미지를 가져올 때 kubelet](#imagepullsecrets-사용하기)에 의해 사용. +시크릿 오브젝트의 이름은 유효한 +[DNS 서브도메인 이름](/docs/concepts/overview/working-with-objects/names#dns-subdomain-names)이어야 한다. + +`data` 와 `stringData` 의 키는 영숫자 및 `-`, `_` 또는 `.` 으로 +구성되어야 한다. + ### 빌트인 시크릿 #### 서비스 어카운트는 API 자격 증명으로 시크릿을 자동으로 생성하고 연결함 @@ -50,401 +56,15 @@ weight: 30 서비스 어카운트 작동 방식에 대한 자세한 내용은 [서비스어카운트(ServiceAccount)](/docs/tasks/configure-pod-container/configure-service-account/) 문서를 참고한다. -### 자신만의 시크릿 생성하기 +### 시크릿 생성하기 -#### `kubectl` 사용하여 시크릿 생성하기 +시크릿을 생성하기 위한 몇 가지 옵션이 있다. -시크릿에는 파드가 데이터베이스에 접근하는 데 필요한 사용자 자격 증명이 포함될 수 있다. -예를 들어, 데이터베이스 연결 문자열은 -사용자명(username)과 비밀번호(password)로 구성된다. 사용자명은 `./username.txt` 파일에 -저장하고 비밀번호는 로컬 시스템의 `./password.txt` 파일에 저장할 수 있다. +- [`kubectl` 명령을 사용하여 시크릿 생성하기](/docs/tasks/configmap-secret/managing-secret-using-kubectl/) +- [구성 파일로 시크릿 생성하기](/docs/tasks/configmap-secret/managing-secret-using-config-file/) +- [kustomize를 사용하여 시크릿 생성하기](/docs/tasks/configmap-secret/managing-secret-using-kustomize/) -```shell -# 예제를 위해서 필요한 파일들을 생성한다. -echo -n 'admin' > ./username.txt -echo -n '1f2d1e2e67df' > ./password.txt -``` - -`kubectl create secret` 명령은 이러한 파일을 시크릿으로 패키징하고 -API 서버에 오브젝트를 생성한다. -시크릿 오브젝트의 이름은 유효한 -[DNS 서브도메인 이름](/ko/docs/concepts/overview/working-with-objects/names/#dns-서브도메인-이름)이어야 한다. - -```shell -kubectl create secret generic db-user-pass --from-file=./username.txt --from-file=./password.txt -``` - -출력 결과는 다음과 비슷하다. - -``` -secret "db-user-pass" created -``` - -기본 키 이름은 파일명(filename)이다. 선택적으로 `--from-file=[key=]source]` 를 사용하여 키 이름을 설정할 수 있다. - -```shell -kubectl create secret generic db-user-pass --from-file=username=./username.txt --from-file=password=./password.txt -``` - -{{< note >}} -`$`, `\`, `*`, `=`, 그리고 `!` 등의 특수 문자는 [셸](https://ko.wikipedia.org/wiki/셸)에 의해 해석되고 이스케이핑이 필요하다. -대부분의 셸에서, 비밀번호를 이스케이프하는 가장 쉬운 방법은 작은 따옴표(`'`)로 묶는 것이다. -예를 들어, 실제 비밀번호가 `S!B\*d$zDsb=` 이면, 다음과 같은 명령을 실행해야 한다. - -```shell -kubectl create secret generic dev-db-secret --from-literal=username=devuser --from-literal=password='S!B\*d$zDsb=' -``` - -파일(`--from-file`)에서는 비밀번호의 특수 문자를 이스케이프할 필요가 없다. -{{< /note >}} - -다음의 명령으로 시크릿이 생성되었는지 확인할 수 있다. - -```shell -kubectl get secrets -``` - -출력 결과는 다음과 비슷하다. - -``` -NAME TYPE DATA AGE -db-user-pass Opaque 2 51s -``` - -시크릿에 대한 설명을 볼 수 있다. - -```shell -kubectl describe secrets/db-user-pass -``` - -출력 결과는 다음과 비슷하다. - -``` -Name: db-user-pass -Namespace: default -Labels: -Annotations: - -Type: Opaque - -Data -==== -password.txt: 12 bytes -username.txt: 5 bytes -``` - -{{< note >}} -`kubectl get` 명령과 `kubectl describe` 명령은 기본적으로 시크릿의 내용을 표시하지 -않는다. 이는 시크릿이 우연히 다른 사람에게 노출되거나, -터미널 로그에 저장되지 않도록 보호하기 위함이다. -{{< /note >}} - -시크릿 내용을 보는 방법을 익히기 위해서는 [시크릿 디코딩하기](#시크릿-디코딩하기)를 참고한다. - -#### 수동으로 시크릿 생성하기 - -먼저 JSON 또는 YAML 형식으로 파일에 시크릿을 생성한 -다음 해당 오브젝트를 생성할 수도 있다. -시크릿 오브젝트의 이름은 유효한 -[DNS 서브도메인 이름](/ko/docs/concepts/overview/working-with-objects/names/#dns-서브도메인-이름)이어야 한다. -[시크릿](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#secret-v1-core)은 -두 개의 맵(`data` 와 `stringData`)을 -포함한다. `data` 필드는 base64를 사용하여, 인코딩된 임의 데이터를 저장하는 데 -사용된다. `stringData` 필드는 편의를 위해 제공되며, 시크릿 데이터를 인코딩되지 않은 -문자열로 제공할 수 있다. - -예를 들어, `data` 필드를 사용하여 시크릿에 두 개의 문자열을 저장하려면, 다음과 같이 -문자열을 base64로 변환한다. - -```shell -echo -n 'admin' | base64 -``` - -출력 결과는 다음과 비슷하다. - -``` -YWRtaW4= -``` - -```shell -echo -n '1f2d1e2e67df' | base64 -``` - -출력 결과는 다음과 비슷하다. - -``` -MWYyZDFlMmU2N2Rm -``` - -다음과 같은 시크릿을 작성한다. - -```yaml -apiVersion: v1 -kind: Secret -metadata: - name: mysecret -type: Opaque -data: - username: YWRtaW4= - password: MWYyZDFlMmU2N2Rm -``` - -이제 [`kubectl apply`](/docs/reference/generated/kubectl/kubectl-commands#apply)를 사용하여 시크릿을 생성한다. - -```shell -kubectl apply -f ./secret.yaml -``` - -출력 결과는 다음과 비슷하다. - -``` -secret "mysecret" created -``` - -특정 시나리오의 경우, `stringData` 필드를 대신 사용할 수 있다. 이 -필드를 사용하면 base64로 인코딩되지 않은 문자열을 시크릿에 직접 넣을 수 있으며, -시크릿이 생성되거나 업데이트될 때 문자열이 인코딩된다. - -이에 대한 실질적인 예는 애플리케이션 배포 시에 -구성 파일 저장을 위해서 시크릿을 사용하되, 배포 프로세스 중에 해당 구성 파일의 -일부를 채우려는 경우이다. - -예를 들어, 애플리케이션이 다음 구성 파일을 사용하는 경우, - -```yaml -apiUrl: "https://my.api.com/api/v1" -username: "user" -password: "password" -``` - -다음 정의를 사용하여 이를 시크릿에 저장할 수 있다. - -```yaml -apiVersion: v1 -kind: Secret -metadata: - name: mysecret -type: Opaque -stringData: - config.yaml: |- - apiUrl: "https://my.api.com/api/v1" - username: {{username}} - password: {{password}} -``` - -그런 다음 `kubectl apply` 를 실행하기 전에 배포 도구로 -`{{username}}` 및 `{{password}}` 템플릿 변수를 바꿀 수 있다. - -`stringData` 필드는 쓰기 전용의 편의 필드이다. 이 내용은 시크릿을 -검색할 때 출력되지 않는다. 예를 들어, 다음 명령을 실행해본다. - -```shell -kubectl get secret mysecret -o yaml -``` - -출력 결과는 다음과 비슷하다. - -```yaml -apiVersion: v1 -kind: Secret -metadata: - creationTimestamp: 2018-11-15T20:40:59Z - name: mysecret - namespace: default - resourceVersion: "7225" - uid: c280ad2e-e916-11e8-98f2-025000000001 -type: Opaque -data: - config.yaml: YXBpVXJsOiAiaHR0cHM6Ly9teS5hcGkuY29tL2FwaS92MSIKdXNlcm5hbWU6IHt7dXNlcm5hbWV9fQpwYXNzd29yZDoge3twYXNzd29yZH19 -``` - -`username` 과 같은 필드가 `data` 와 `stringData` 모두에 지정되면, -`stringData` 의 값이 사용된다. 예를 들어, 다음의 시크릿 정의를 보자. - -```yaml -apiVersion: v1 -kind: Secret -metadata: - name: mysecret -type: Opaque -data: - username: YWRtaW4= -stringData: - username: administrator -``` - -아래의 시크릿에서 결과는 다음과 같다. - -```yaml -apiVersion: v1 -kind: Secret -metadata: - creationTimestamp: 2018-11-15T20:46:46Z - name: mysecret - namespace: default - resourceVersion: "7579" - uid: 91460ecb-e917-11e8-98f2-025000000001 -type: Opaque -data: - username: YWRtaW5pc3RyYXRvcg== -``` - -위의 `YWRtaW5pc3RyYXRvcg==` 를 디코딩하면 `administrator` 가 된다. - -`data` 와 `stringData` 의 키는 영숫자, -'-', '_' 또는 '.'로 구성되어야 한다. - -{{< note >}} -시크릿 데이터의 직렬화된 JSON 및 YAML 값은 -base64 문자열로 인코딩된다. 줄바꿈은 이러한 문자열 내에서 유효하지 않으므로 -생략해야 한다. Darwin/macOS에서 `base64` 유틸리티를 사용할 때, -사용자는 긴 라인을 분할하는 `-b` 옵션을 사용하면 안된다. 반대로, 리눅스 사용자는 -`-w` 옵션을 사용할 수 없는 경우 `base64` 명령이나 `base64 | tr -d '\n'` 파이프라인에 -`-w 0` 옵션을 추가 *해야 한다*. -{{< /note >}} - -#### 생성기를 통해 시크릿 생성하기 - -쿠버네티스 v1.14부터 `kubectl` 은 [Kustomize를 사용한 오브젝트 관리](/ko/docs/tasks/manage-kubernetes-objects/kustomization/)를 지원한다. Kustomize는 시크릿과 컨피그맵(ConfigMap)을 생성하기 위한 -리소스 생성기를 제공한다. Kustomize 생성기는 디렉터리 내의 -`kustomization.yaml` 파일에 지정되어야 한다. 시크릿을 생성한 후, -`kubectl apply` 를 사용하여 API 서버에서 시크릿을 만들 수 있다. - -#### 파일을 통해 시크릿 생성하기 - -./username.txt 와 ./password.txt 파일에서 -`secretGenerator` 를 정의하여 시크릿을 생성할 수 있다. - -```shell -cat <./kustomization.yaml -secretGenerator: -- name: db-user-pass - files: - - username.txt - - password.txt -EOF -``` - -`kustomization.yaml` 을 포함하는 디렉터리를 적용하여 시크릿을 생성한다. - -```shell -kubectl apply -k . -``` - -출력 결과는 다음과 비슷하다. - -``` -secret/db-user-pass-96mffmfh4k created -``` - -시크릿이 생성되었는지 다음의 명령으로 확인할 수 있다. - -```shell -kubectl get secrets -``` - -출력 결과는 다음과 비슷하다. - -``` -NAME TYPE DATA AGE -db-user-pass-96mffmfh4k Opaque 2 51s -``` - -시크릿에 대한 설명을 볼 수 있다. - -```shell -kubectl describe secrets/db-user-pass-96mffmfh4k -``` - -출력 결과는 다음과 비슷하다. - -``` -Name: db-user-pass -Namespace: default -Labels: -Annotations: - -Type: Opaque - -Data -==== -password.txt: 12 bytes -username.txt: 5 bytes -``` - -#### 문자열 리터럴(literals)을 통해 시크릿 생성하기 - -문자열 리터럴인 `username=admin` 과 `password=secret` 을 -`secretGenerator` 에 정의하여 시크릿을 만들 수 있다. - -```shell -cat <./kustomization.yaml -secretGenerator: -- name: db-user-pass - literals: - - username=admin - - password=secret -EOF -``` - -`kustomization.yaml` 을 포함하는 디렉터리를 적용하여 시크릿을 생성한다. - -```shell -kubectl apply -k . -``` - -출력 결과는 다음과 비슷하다. - -``` -secret/db-user-pass-dddghtt9b5 created -``` - -{{< note >}} -시크릿이 생성될 때, 시크릿의 이름은 시크릿 데이터를 -해시하고, 해시된 값을 이름에 추가하는 방식으로 만들어진다. 이 방식은 -데이터가 수정될 때마다 새로운 시크릿이 생성되도록 만든다. -{{< /note >}} - -#### 시크릿 디코딩하기 - -`kubectl get secret` 을 실행하여 시크릿을 검색할 수 있다. -예를 들어, 다음 명령을 실행하여 이전 섹션에서 -생성된 시크릿을 볼 수 있다. - -```shell -kubectl get secret mysecret -o yaml -``` - -출력 결과는 다음과 비슷하다. - -```yaml -apiVersion: v1 -kind: Secret -metadata: - creationTimestamp: 2016-01-22T18:41:56Z - name: mysecret - namespace: default - resourceVersion: "164619" - uid: cfee02d6-c137-11e5-8d73-42010af00002 -type: Opaque -data: - username: YWRtaW4= - password: MWYyZDFlMmU2N2Rm -``` - -`password` 필드를 디코딩한다. - -```shell -echo 'MWYyZDFlMmU2N2Rm' | base64 --decode -``` - -출력 결과는 다음과 비슷하다. - -``` -1f2d1e2e67df -``` - -#### 시크릿 편집하기 +### 시크릿 편집하기 기존 시크릿은 다음 명령을 사용하여 편집할 수 있다. @@ -1273,3 +893,10 @@ API 서버에서 kubelet으로의 통신은 SSL/TLS로 보호된다. API 서버에서 _모든_ 시크릿을 읽을 수 있다. 단일 노드에 대한 루트 취약점 공격의 영향을 제한하기 위해, 실제로 필요한 노드에만 시크릿을 보내는 것이 앞으로 계획된 기능이다. + + +## {{% heading "whatsnext" %}} + +- [`kubectl` 을 사용한 시크릿 관리](/docs/tasks/configmap-secret/managing-secret-using-kubectl/)하는 방법 배우기 +- [구성 파일을 사용한 시크릿 관리](/docs/tasks/configmap-secret/managing-secret-using-config-file/)하는 방법 배우기 +- [kustomize를 사용한 시크릿 관리](/docs/tasks/configmap-secret/managing-secret-using-kustomize/)하는 방법 배우기 diff --git a/content/ko/docs/concepts/containers/runtime-class.md b/content/ko/docs/concepts/containers/runtime-class.md index 3da41dfde0..743027b0d4 100644 --- a/content/ko/docs/concepts/containers/runtime-class.md +++ b/content/ko/docs/concepts/containers/runtime-class.md @@ -77,7 +77,7 @@ handler: myconfiguration # 상응하는 CRI 설정의 이름임 {{< note >}} 런타임클래스 쓰기 작업(create/update/patch/delete)은 클러스터 관리자로 제한할 것을 권장한다. 이것은 일반적으로 기본 설정이다. -더 자세한 정보는 [권한 개요](/docs/reference/access-authn-authz/authorization/)를 참고한다. +더 자세한 정보는 [권한 개요](/ko/docs/reference/access-authn-authz/authorization/)를 참고한다. {{< /note >}} ## 사용 diff --git a/content/ko/docs/concepts/extend-kubernetes/_index.md b/content/ko/docs/concepts/extend-kubernetes/_index.md index e6d64bdfb7..4bab0ad214 100644 --- a/content/ko/docs/concepts/extend-kubernetes/_index.md +++ b/content/ko/docs/concepts/extend-kubernetes/_index.md @@ -126,7 +126,7 @@ API를 추가해도 기존 API(예: 파드)의 동작에 직접 영향을 미치 ### API 접근 익스텐션 -요청이 쿠버네티스 API 서버에 도달하면 먼저 인증이 되고, 그런 다음 승인된 후 다양한 유형의 어드미션 컨트롤이 적용된다. 이 흐름에 대한 자세한 내용은 [쿠버네티스 API에 대한 접근 제어](/docs/reference/access-authn-authz/controlling-access/)를 참고하길 바란다. +요청이 쿠버네티스 API 서버에 도달하면 먼저 인증이 되고, 그런 다음 승인된 후 다양한 유형의 어드미션 컨트롤이 적용된다. 이 흐름에 대한 자세한 내용은 [쿠버네티스 API에 대한 접근 제어](/ko/docs/reference/access-authn-authz/controlling-access/)를 참고하길 바란다. 이러한 각 단계는 익스텐션 포인트를 제공한다. diff --git a/content/ko/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins.md b/content/ko/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins.md index 1966d2db5d..70a7d1ea27 100644 --- a/content/ko/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins.md +++ b/content/ko/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins.md @@ -253,6 +253,6 @@ pluginapi.Device{ID: "25102017", Health: pluginapi.Healthy, Topology:&pluginapi. * 장치 플러그인을 사용한 [GPU 리소스 스케줄링](/ko/docs/tasks/manage-gpus/scheduling-gpus/)에 대해 알아보기 * 노드에서의 [확장 리소스 알리기](/ko/docs/tasks/administer-cluster/extended-resource-node/)에 대해 배우기 * 쿠버네티스에서 [TLS 수신에 하드웨어 가속](https://kubernetes.io/blog/2019/04/24/hardware-accelerated-ssl/tls-termination-in-ingress-controllers-using-kubernetes-device-plugins-and-runtimeclass/) 사용에 대해 읽기 -* [토폴로지 관리자](/docs/tasks/adminster-cluster/topology-manager/)에 대해 알아보기 +* [토폴로지 관리자](/docs/tasks/administer-cluster/topology-manager/)에 대해 알아보기 diff --git a/content/ko/docs/concepts/extend-kubernetes/extend-cluster.md b/content/ko/docs/concepts/extend-kubernetes/extend-cluster.md index e4e2b9b96d..d214a2c0cf 100644 --- a/content/ko/docs/concepts/extend-kubernetes/extend-cluster.md +++ b/content/ko/docs/concepts/extend-kubernetes/extend-cluster.md @@ -127,7 +127,7 @@ API를 추가해도 기존 API(예: 파드)의 동작에 직접 영향을 미치 ### API 접근 익스텐션 -요청이 쿠버네티스 API 서버에 도달하면 먼저 인증이 되고, 그런 다음 승인된 후 다양한 유형의 어드미션 컨트롤이 적용된다. 이 흐름에 대한 자세한 내용은 [쿠버네티스 API에 대한 접근 제어](/docs/reference/access-authn-authz/controlling-access/)를 참고하길 바란다. +요청이 쿠버네티스 API 서버에 도달하면 먼저 인증이 되고, 그런 다음 승인된 후 다양한 유형의 어드미션 컨트롤이 적용된다. 이 흐름에 대한 자세한 내용은 [쿠버네티스 API에 대한 접근 제어](/ko/docs/reference/access-authn-authz/controlling-access/)를 참고하길 바란다. 이러한 각 단계는 익스텐션 포인트를 제공한다. diff --git a/content/ko/docs/concepts/overview/kubernetes-api.md b/content/ko/docs/concepts/overview/kubernetes-api.md index 7a02d46b13..b26efc8085 100644 --- a/content/ko/docs/concepts/overview/kubernetes-api.md +++ b/content/ko/docs/concepts/overview/kubernetes-api.md @@ -3,7 +3,7 @@ title: 쿠버네티스 API content_type: concept weight: 30 description: > - 쿠버네티스 API를 사용하면 쿠버네티스 오브젝트들의 상태를 쿼리하고 조작할 수 있다. + 쿠버네티스 API를 사용하면 쿠버네티스 오브젝트들의 상태를 쿼리하고 조작할 수 있다. 쿠버네티스 컨트롤 플레인의 핵심은 API 서버와 그것이 노출하는 HTTP API이다. 사용자와 클러스터의 다른 부분 및 모든 외부 컴포넌트는 API 서버를 통해 서로 통신한다. card: name: concepts @@ -20,24 +20,17 @@ card: 쿠버네티스 API를 사용하면 쿠버네티스 API 오브젝트(예: 파드(Pod), 네임스페이스(Namespace), 컨피그맵(ConfigMap) 그리고 이벤트(Event))를 질의하고 조작할 수 있다. -API 엔드포인트, 리소스 타입과 샘플은 [API Reference](/ko/docs/reference)에 기술되어 있다. +대부분의 작업은 [kubectl](/docs/reference/kubectl/overview/) +커맨드 라인 인터페이스 또는 API를 사용하는 +[kubeadm](/docs/reference/setup-tools/kubeadm/kubeadm/)과 +같은 다른 커맨드 라인 도구를 통해 수행할 수 있다. +그러나, REST 호출을 사용하여 API에 직접 접근할 수도 있다. + +쿠버네티스 API를 사용하여 애플리케이션을 작성하는 경우 +[클라이언트 라이브러리](/docs/reference/using-api/client-libraries/) 중 하나를 사용하는 것이 좋다. -## API 변경 - -새로운 유스케이스가 등장하거나 기존 시스템이 변경됨에 따라 성공적인 시스템은 성장하고 변경될 필요가 있다. -따라서, 쿠버네티스는 쿠버네티스 API를 지속적으로 변경하고 성장시킬 수 있는 디자인 기능을 가지고 있다. -쿠버네티스 프로젝트는 기존 클라이언트와의 호환성을 중단하지 _않고_, -다른 프로젝트가 적응할 수 있도록 오랫동안 호환성을 유지하는 것을 목표로 한다. - -일반적으로, 새로운 API 리소스와 새로운 리소스 필드가 주기적으로 추가될 것이다. -리소스나 필드를 없애는 일은 다음의 -[API 사용 중단 정책](/docs/reference/using-api/deprecation-policy/)을 따른다. - -호환되는 변경에 어떤 내용이 포함되는지, 어떻게 API를 변경하는지에 대한 자세한 내용은 -[API 변경](https://git.k8s.io/community/contributors/devel/sig-architecture/api_changes.md#readme)을 참고한다. - ## OpenAPI 명세 {#api-specification} 완전한 API 상세 내용은 [OpenAPI](https://www.openapis.org/)를 활용해서 문서화했다. @@ -76,96 +69,58 @@ OpenAPI 규격은 `/openapi/v2` 엔드포인트에서만 제공된다. Valid request header values for OpenAPI v2 queries -쿠버네티스는 주로 클러스터 내부 통신용 API를 위해 대안적인 Protobuf에 기반한 직렬화 형식을 구현한다. 해당 API는 [design proposal](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/api-machinery/protobuf.md) 문서와 IDL 파일에 문서화되어 있고 각각의 스키마를 담고 있는 IDL 파일은 API 오브젝트를 정의하는 Go 패키지에 들어있다. +쿠버네티스는 주로 클러스터 내부 통신을 위해 대안적인 +Protobuf에 기반한 직렬화 형식을 구현한다. 이 형식에 대한 +자세한 내용은 [쿠버네티스 Protobuf 직렬화](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/api-machinery/protobuf.md) 디자인 제안과 +API 오브젝트를 정의하는 Go 패키지에 들어있는 각각의 스키마에 대한 +IDL(인터페이스 정의 언어) 파일을 참고한다. -## API 버전 규칙 +## API 변경 사항 -필드를 없애거나 리소스 표현을 재구성하기 쉽도록, -쿠버네티스는 `/api/v1`이나 `/apis/rbac.authorization.k8s.io/v1alpha1`과 같이 -각각 다른 API 경로에서 복수의 API 버전을 지원한다. +성공적인 시스템은 새로운 유스케이스가 등장하거나 기존 사례가 변경됨에 따라 성장하고 변화해야 한다. +따라서, 쿠버네티스는 쿠버네티스 API가 지속적으로 변경되고 성장할 수 있도록 기능을 설계했다. +쿠버네티스 프로젝트는 기존 클라이언트와의 호환성을 깨지 _않고_ 다른 프로젝트가 +적응할 기회를 가질 수 있도록 장기간 해당 호환성을 유지하는 것을 목표로 한다. -버전 관리는 API가 시스템 리소스와 동작에 대해 명확하고 일관된 보기를 -제공하고 수명 종료(end-of-life)와 실험적인 API에 대한 접근을 제어할 수 있도록 -리소스 또는 필드 수준이 아닌 API 수준에서 수행된다. +일반적으로, 새 API 리소스와 새 리소스 필드는 자주 추가될 수 있다. +리소스 또는 필드를 제거하려면 +[API 지원 중단 정책](/docs/reference/using-api/deprecation-policy/)을 따라야 한다. -JSON과 Protobuf 직렬화 스키마는 스키마 변경에 대한 동일한 지침을 따르며 아래의 모든 설명은 두 형식을 모두 포함한다. +호환 가능한 변경 사항과 API 변경 방법은 +[API 변경 사항](https://git.k8s.io/community/contributors/devel/sig-architecture/api_changes.md#readme)에 자세히 설명되어 있다. -참고로 API 버전 관리와 소프트웨어 버전 관리는 간접적으로만 연관이 있다. -[쿠버네티스 릴리스 버전 관리](https://git.k8s.io/community/contributors/design-proposals/release/versioning.md) -제안은 API 버전 관리와 소프트웨어 버전 관리 사이의 관계를 설명한다. +## API 그룹과 버전 규칙 -API 버전이 다른 경우는 안정성이나 기술 지원의 수준이 다르다는 것을 암시한다. 각각의 수준에 대한 조건은 -[API 변경](https://git.k8s.io/community/contributors/devel/sig-architecture/api_changes.md#alpha-beta-and-stable-versions)에서 -상세히 다룬다. 요약하자면 다음과 같다. +필드를 쉽게 제거하거나 리소스 표현을 재구성하기 위해, +쿠버네티스는 각각 `/api/v1` 또는 `/apis/rbac.authorization.k8s.io/v1alpha1` 과 +같은 서로 다른 API 경로에서 여러 API 버전을 지원한다. -- 알파(Alpha) 수준: - - 버전 이름에 `alpha`가 포함된다. (예: `v1alpha1`) - - 버그가 있을 수도 있다. 이 기능을 활성화하면 버그가 노출될 수 있다. 기본적으로 비활성화되어 있다. - - 기능에 대한 기술 지원이 언제든 공지 없이 중단될 수 있다. - - 다음 소프트웨어를 릴리스할 때 공지 없이 API의 호환성이 깨지는 방식으로 변경될 수 있다. - - 버그의 위험이 높고 장기간 지원되지 않으므로 단기간 테스트 용도의 클러스터에서만 사용하기를 권장한다. -- 베타(Beta) 수준: - - 버전 이름에 `beta`가 포함된다. (예: `v2beta3`). - - 코드가 잘 테스트되었다. 이 기능을 활성화 시켜도 안전하다. 기본적으로 활성화되어 있다. - - 구체적인 내용이 바뀔 수는 있지만, 전반적인 기능에 대한 기술 지원이 중단되지 않는다. - - 오브젝트에 대한 스키마나 문법이 다음 베타 또는 안정화 릴리스에서 호환되지 않는 방식으로 바뀔 수도 있다. 이런 경우, - 다음 버전으로 이관할 수 있는 가이드를 제공할 것이다. - 이 때 API 오브젝트의 삭제, 편집 또는 재생성이 필요할 수도 있다. 편집 절차는 좀 생각해볼 필요가 있다. 이 기능에 의존하고 있는 애플리케이션은 다운타임이 필요할 수도 있다. - - 다음 릴리스에서 호환되지 않을 수도 있으므로 사업적으로 중요하지 않은 용도로만 사용하기를 권장한다. - 복수의 클러스터를 가지고 있어서 독립적으로 업그레이드할 수 있다면 이런 제약에서 안심이 될 수도 있겠다. - - **베타 기능을 사용하고 피드백을 주기를 바란다! 일단 베타가 끝나면, 실질적으로 더 많은 변경이 어렵다.** -- 안정화(stable) 수준: - - 버전 이름이 `vX`이고 `X` 는 정수다. - - 안정화 버전의 기능은 이후 여러 버전에 걸쳐서 소프트웨어 릴리스에 포함된다. +버전 규칙은 리소스나 필드 수준이 아닌 API 수준에서 수행되어 +API가 시스템 리소스 및 동작에 대한 명확하고 일관된 보기를 제공하고 +수명 종료 및/또는 실험적 API에 대한 접근을 +제어할 수 있도록 한다. -## API 그룹 +API 버전 수준 정의에 대한 자세한 내용은 +[API 버전 레퍼런스](/ko/docs/reference/using-api/api-overview/#api-버전-규칙)를 참조한다. -쿠버네티스 API를 보다 쉽게 확장하기 위해서, [*API 그룹*](https://git.k8s.io/community/contributors/design-proposals/api-machinery/api-group.md)을 구현했다. -API 그룹은 REST 경로와 직렬화된 객체의 `apiVersion` 필드에 명시된다. +보다 쉽게 발전하고 API를 확장하기 위해, 쿠버네티스는 +[활성화 또는 비활성화](/ko/docs/reference/using-api/api-overview/#api-그룹-활성화-또는-비활성화-하기)가 +가능한 [API 그룹](/ko/docs/reference/using-api/api-overview/#api-그룹)을 구현한다. -클러스터에 다양한 API 그룹이 있다. +## API 확장 -1. *레거시* 그룹이라고도 하는 *핵심* 그룹은 REST 경로인 `/api/v1/` 에 있고, `apiVersion: v1`을 사용한다. - -1. 이름이 있는 그룹은 REST 경로 `/apis/$GROUP_NAME/$VERSION`에 있으며 `apiVersion: $GROUP_NAME/$VERSION`을 사용한다 - (예: `apiVersion: batch/v1`). 사용 가능한 API 그룹의 전체의 목록은 - [쿠버네티스 API 참조](/ko/docs/reference/)에 있다. - - -[사용자 지정 리소스](/ko/docs/concepts/extend-kubernetes/api-extension/custom-resources/)로 API를 확장하는 두 가지 방법이 있다. - -1. [커스텀리소스데피니션(CustomResourceDefinition)](/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/)은 - API 서버가 선택한 리소스 API를 제공하는 방법을 선언적으로 정의할 수 있다. -1. 또한, [자신의 확장 API 서버 구현](/ko/docs/tasks/extend-kubernetes/setup-extension-api-server/)과 - [aggregator](/docs/tasks/extend-kubernetes/configure-aggregation-layer/)를 - 사용해서 클라이언트를 원활하게 만들 수 있다. - - -## API 그룹 활성화 또는 비활성화하기 - -특정 리소스와 API 그룹은 기본적으로 활성화되어 있다. kube-apiserver에서 커맨드 라인 옵션으로 `--runtime-config` 를 -설정해서 활성화하거나 비활성화할 수 있다. - -`--runtime-config`는 쉼표로 분리된 값을 허용한다. 예를 들어서 batch/v1을 비활성화시키려면, -`--runtime-config=batch/v1=false`와 같이 설정하고, batch/v2alpha1을 활성화시키려면, `--runtime-config=batch/v2alpha1`을 -설정한다. 이 플래그는 API 서버의 런타임 설정에 쉼표로 분리된 키=값 쌍의 집합을 허용한다. - -{{< note >}}그룹이나 리소스를 활성화 또는 비활성화하려면 kube-apiserver와 -controller-manager를 재시작해서 `--runtime-config` 변경 사항을 반영해야 한다. {{< /note >}} - -## 지속성 - -쿠버네티스는 API 리소스에 대한 직렬화된 상태를 {{< glossary_tooltip term_id="etcd" >}}에 -기록하고 저장한다. +쿠버네티스 API는 다음 두 가지 방법 중 하나로 확장할 수 있다. +1. [커스텀 리소스](/ko/docs/concepts/extend-kubernetes/api-extension/custom-resources/)를 + 사용하면 API 서버가 선택한 리소스 API를 제공하는 방법을 선언적으로 정의할 수 있다. +1. [애그리게이션 레이어(aggregation layer)](/ko/docs/concepts/extend-kubernetes/api-extension/apiserver-aggregation/)를 + 구현하여 쿠버네티스 API를 확장할 수도 있다. ## {{% heading "whatsnext" %}} -[API 접근 제어하기](/docs/reference/access-authn-authz/controlling-access/)는 클러스터가 -API 접근에 대한 인증과 권한을 관리하는 방법을 설명한다. - -전체 API 규약은 -[API 규약](https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#api-conventions) -문서에 설명되어 있다. - -API 엔드포인트, 리소스 타입과 샘플은 [API 참조](/docs/reference/kubernetes-api/)에 설명되어 있다. +- 자체 [CustomResourceDefinition](/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/)을 + 추가하여 쿠버네티스 API를 확장하는 방법에 대해 배우기. +- [API 접근 제어하기](/ko/docs/reference/access-authn-authz/controlling-access/)는 + 클러스터가 API 접근을 위한 인증 및 권한을 관리하는 방법을 설명한다. +- [API 레퍼런스](/docs/reference/kubernetes-api/)를 + 읽고 API 엔드포인트, 리소스 유형 및 샘플에 대해 배우기. diff --git a/content/ko/docs/concepts/overview/working-with-objects/annotations.md b/content/ko/docs/concepts/overview/working-with-objects/annotations.md index 96db884a4f..944d7340ba 100644 --- a/content/ko/docs/concepts/overview/working-with-objects/annotations.md +++ b/content/ko/docs/concepts/overview/working-with-objects/annotations.md @@ -63,7 +63,7 @@ weight: 50 ## 문법과 캐릭터 셋 -_어노테이션_ 은 키/값 쌍이다. 유효한 어노테이션 키에는 두 개의 세그먼트가 있다. 두 개의 세그먼트는 선택적인 접두사와 이름(name)이며, 슬래시(`/`)로 구분된다. 이름 세그먼트는 필수이며, 영문 숫자(`[a-z0-9A-Z]`)로 시작하고 끝나는 63자 이하이어야 하고, 사이에 대시(`-`), 밑줄(`_`), 점(`.`)이 들어갈 수 있다. 접두사는 선택적이다. 지정된 경우, 접두사는 DNS 하위 도메인이어야 한다. 점(`.`)으로 구분된 일련의 DNS 레이블은 총 253자를 넘지 않고, 뒤에 슬래시(`/`)가 붙는다. +_어노테이션_ 은 키/값 쌍이다. 유효한 어노테이션 키에는 두 개의 세그먼트가 있다. 두 개의 세그먼트는 선택적인 접두사와 이름(name)이며, 슬래시(`/`)로 구분된다. 이름 세그먼트는 필수이며, 영문 숫자(`[a-z0-9A-Z]`)로 시작하고 끝나는 63자 이하이어야 하고, 사이에 대시(`-`), 밑줄(`_`), 점(`.`)이 들어갈 수 있다. 접두사는 선택적이다. 지정된 경우, 접두사는 DNS 서브도메인이어야 한다. 점(`.`)으로 구분된 일련의 DNS 레이블은 총 253자를 넘지 않고, 뒤에 슬래시(`/`)가 붙는다. 접두사가 생략되면, 어노테이션 키는 사용자에게 비공개로 간주된다. 최종 사용자 오브젝트에 어노테이션을 추가하는 자동화된 시스템 구성 요소(예 :`kube-scheduler`, `kube-controller-manager`, `kube-apiserver`, `kubectl`, 또는 다른 써드파티 자동화)는 접두사를 지정해야 한다. diff --git a/content/ko/docs/concepts/overview/working-with-objects/common-labels.md b/content/ko/docs/concepts/overview/working-with-objects/common-labels.md index 8abdacb09d..b67a524a7f 100644 --- a/content/ko/docs/concepts/overview/working-with-objects/common-labels.md +++ b/content/ko/docs/concepts/overview/working-with-objects/common-labels.md @@ -58,15 +58,15 @@ metadata: ## 애플리케이션과 애플리케이션 인스턴스 -애플리케이션은 때에 따라 쿠버네티스 클러스터의 동일한 네임스페이스에 -한번 또는 그 이상 설치할 수 있다. 예를 들어 워드프레스는 다른 워드프레스가 -설치되어있는 웹사이트에 한번 한번 또는 그 이상 설치할 수 있다. +애플리케이션은 동일한 쿠버네티스 클러스터에, +심지어는 동일한 네임스페이스에도 한번 또는 그 이상 설치될 수 있다. 예를 들어, 하나의 쿠버네티스 클러스터에 +워드프레스가 여러 번 설치되어 각각 서로 다른 웹사이트를 서비스할 수 있다. -애플리케이션의 이름과 인스턴스 이름은 별도로 기록된다. -예를 들어 워드프레스는 `app.kubernetes.io/name` 에 `wordpress` 를 가지며 인스턴스 이름으로는 -`app.kubernetes.io/instance` 에 `wordpress-abcxzy` 의 값을 가진다. -이를 통해 애플리케이션과 애플리케이션 인스턴스를 식별할 수 있다. -모든 애플리케이션 인스턴스는 고유한 이름을 가져야 한다. +애플리케이션의 이름과 애플리케이션 인스턴스 이름은 별도로 기록된다. +예를 들어 워드프레스는 애플리케이션 이름으로 `app.kubernetes.io/name` 이라는 레이블에 `wordpress` 라는 값을 가지며, +애플리케이션 인스턴스 이름으로는 `app.kubernetes.io/instance` 라는 레이블에 +`wordpress-abcxzy` 라는 값을 가진다. 이를 통해 애플리케이션과 애플리케이션 인스턴스를 +식별할 수 있다. 모든 애플리케이션 인스턴스는 고유한 이름을 가져야 한다. ## 예시 @@ -170,4 +170,3 @@ metadata: MySQL `StatefulSet` 과 `Service` 로 MySQL과 WordPress가 더 큰 범위의 애플리케이션에 포함되어 있는 것을 알게 된다. - diff --git a/content/ko/docs/concepts/policy/pod-security-policy.md b/content/ko/docs/concepts/policy/pod-security-policy.md index a3d9b641e9..c1e7bc56a9 100644 --- a/content/ko/docs/concepts/policy/pod-security-policy.md +++ b/content/ko/docs/concepts/policy/pod-security-policy.md @@ -138,7 +138,7 @@ RBAC 바인딩에 대한 자세한 예는, ### 문제 해결 - [컨트롤러 관리자](/docs/reference/command-line-tools-reference/kube-controller-manager/)는 -[보안 API 포트](/docs/reference/access-authn-authz/controlling-access/)에 대해 실행해야 하며, +[보안 API 포트](/ko/docs/reference/access-authn-authz/controlling-access/)에 대해 실행해야 하며, 슈퍼유저 권한이 없어야 한다. 그렇지 않으면 요청이 인증 및 권한 부여 모듈을 우회하고, 모든 파드시큐리티폴리시 오브젝트가 허용되며 사용자는 특권있는 컨테이너를 만들 수 있다. 컨트롤러 관리자 권한 구성에 대한 자세한 diff --git a/content/ko/docs/concepts/policy/resource-quotas.md b/content/ko/docs/concepts/policy/resource-quotas.md index 526cf03e53..028209a07e 100644 --- a/content/ko/docs/concepts/policy/resource-quotas.md +++ b/content/ko/docs/concepts/policy/resource-quotas.md @@ -206,6 +206,19 @@ GPU 리소스를 다음과 같이 쿼터를 정의할 수 있다. 쿼터 스펙의 `scopeSelector`가 파드를 선택한 경우에만 쿼터가 일치하고 사용된다. +`scopeSelector` 필드를 사용하여 우선 순위 클래스의 쿼터 범위를 지정하면, 쿼터 오브젝트는 다음의 리소스만 추적하도록 제한된다. + +* `pods` +* `cpu` +* `memory` +* `ephemeral-storage` +* `limits.cpu` +* `limits.memory` +* `limits.ephemeral-storage` +* `requests.cpu` +* `requests.memory` +* `requests.ephemeral-storage` + 이 예에서는 쿼터 오브젝트를 생성하여 특정 우선 순위의 파드와 일치시킨다. 예제는 다음과 같이 작동한다. diff --git a/content/ko/docs/concepts/scheduling-eviction/assign-pod-node.md b/content/ko/docs/concepts/scheduling-eviction/assign-pod-node.md index 4694533315..5e8b935422 100644 --- a/content/ko/docs/concepts/scheduling-eviction/assign-pod-node.md +++ b/content/ko/docs/concepts/scheduling-eviction/assign-pod-node.md @@ -69,17 +69,7 @@ spec: ## 넘어가기 전에: 내장 노드 레이블들 {#built-in-node-labels} [붙인](#1-단계-노드에-레이블-붙이기) 레이블뿐만 아니라, 노드에는 -표준 레이블 셋이 미리 채워져 있다. 이 레이블들은 다음과 같다. - -* [`kubernetes.io/hostname`](/docs/reference/kubernetes-api/labels-annotations-taints/#kubernetes-io-hostname) -* [`failure-domain.beta.kubernetes.io/zone`](/docs/reference/kubernetes-api/labels-annotations-taints/#failure-domainbetakubernetesiozone) -* [`failure-domain.beta.kubernetes.io/region`](/docs/reference/kubernetes-api/labels-annotations-taints/#failure-domainbetakubernetesioregion) -* [`topology.kubernetes.io/zone`](/docs/reference/kubernetes-api/labels-annotations-taints/#topologykubernetesiozone) -* [`topology.kubernetes.io/region`](/docs/reference/kubernetes-api/labels-annotations-taints/#topologykubernetesiozone) -* [`beta.kubernetes.io/instance-type`](/docs/reference/kubernetes-api/labels-annotations-taints/#beta-kubernetes-io-instance-type) -* [`node.kubernetes.io/instance-type`](/docs/reference/kubernetes-api/labels-annotations-taints/#nodekubernetesioinstance-type) -* [`kubernetes.io/os`](/docs/reference/kubernetes-api/labels-annotations-taints/#kubernetes-io-os) -* [`kubernetes.io/arch`](/docs/reference/kubernetes-api/labels-annotations-taints/#kubernetes-io-arch) +표준 레이블 셋이 미리 채워져 있다. 이들 목록은 [잘 알려진 레이블, 어노테이션 및 테인트](/docs/reference/kubernetes-api/labels-annotations-taints/)를 참고한다. {{< note >}} 이 레이블들의 값은 클라우드 공급자에 따라 다르고 신뢰성이 보장되지 않는다. diff --git a/content/ko/docs/concepts/scheduling-eviction/scheduler-perf-tuning.md b/content/ko/docs/concepts/scheduling-eviction/scheduler-perf-tuning.md index a964fb21db..4ed5c58a63 100644 --- a/content/ko/docs/concepts/scheduling-eviction/scheduler-perf-tuning.md +++ b/content/ko/docs/concepts/scheduling-eviction/scheduler-perf-tuning.md @@ -48,17 +48,10 @@ kube-scheduler 의 `percentageOfNodesToScore` 설정을 통해 이를 변경한 후에 다음을 실행해서 ```bash -kubectl get componentstatuses +kubectl get pods -n kube-system | grep kube-scheduler ``` -kube-scheduler 컴포넌트가 정상인지 확인할 수 있다. 출력은 다음과 유사하다. - -``` -NAME STATUS MESSAGE ERROR -controller-manager Healthy ok -scheduler Healthy ok -... -``` +kube-scheduler 컴포넌트가 정상인지 확인할 수 있다. ## 노드 스코어링(scoring) 임계값 {#percentage-of-nodes-to-score} diff --git a/content/ko/docs/concepts/security/overview.md b/content/ko/docs/concepts/security/overview.md index 1a8316024c..af5c9ba574 100644 --- a/content/ko/docs/concepts/security/overview.md +++ b/content/ko/docs/concepts/security/overview.md @@ -147,7 +147,7 @@ TLS를 통한 접근 | 코드가 TCP를 통해 통신해야 한다면, 미리 * [파드 보안 표준](/docs/concepts/security/pod-security-standards/) * [파드에 대한 네트워크 정책](/ko/docs/concepts/services-networking/network-policies/) * [클러스터 보안](/docs/tasks/administer-cluster/securing-a-cluster/) -* [API 접근 통제](/docs/reference/access-authn-authz/controlling-access/) +* [API 접근 통제](/ko/docs/reference/access-authn-authz/controlling-access/) * 컨트롤 플레인을 위한 [전송 데이터 암호화](/docs/tasks/tls/managing-tls-in-a-cluster/) * [Rest에서 데이터 암호화](/docs/tasks/administer-cluster/encrypt-data/) * [쿠버네티스 시크릿](/ko/docs/concepts/configuration/secret/) diff --git a/content/ko/docs/concepts/services-networking/endpoint-slices.md b/content/ko/docs/concepts/services-networking/endpoint-slices.md index 900f771cee..22af37603a 100644 --- a/content/ko/docs/concepts/services-networking/endpoint-slices.md +++ b/content/ko/docs/concepts/services-networking/endpoint-slices.md @@ -21,8 +21,8 @@ _엔드포인트슬라이스_ 는 쿠버네티스 클러스터 내의 네트워 엔드포인트 API는 쿠버네티스에서 네트워크 엔드포인트를 추적하는 간단하고 직접적인 방법을 제공한다. 불행하게도 쿠버네티스 클러스터와 -{{< glossary_tooltip text="서비스" term_id="service" >}}가 점점 더 -커짐에 따라, 이 API의 한계가 더욱 눈에 띄게 +{{< glossary_tooltip text="서비스" term_id="service" >}}가 더 많은 수의 백엔드 파드로 +더 많은 트래픽을 처리하고 전송하는 방향으로 성장함에 따라, 이 API의 한계가 더욱 눈에 띄게 되었다. 특히나, 많은 수의 네트워크 엔드포인트로 확장하는 것에 어려움이 있었다. diff --git a/content/ko/docs/concepts/services-networking/ingress.md b/content/ko/docs/concepts/services-networking/ingress.md index 5656f4d125..7b552c424f 100644 --- a/content/ko/docs/concepts/services-networking/ingress.md +++ b/content/ko/docs/concepts/services-networking/ingress.md @@ -27,15 +27,26 @@ weight: 40 {{< link text="서비스" url="/docs/concepts/services-networking/service/" >}}로 HTTP와 HTTPS 경로를 노출한다. 트래픽 라우팅은 인그레스 리소스에 정의된 규칙에 의해 컨트롤된다. -```none - internet - | - [ Ingress ] - --|-----|-- - [ Services ] -``` +다음은 인그레스가 모든 트래픽을 하나의 서비스로 보내는 간단한 예시이다. +{{< mermaid >}} +graph LR; + client([클라이언트])-. 인그레스-매니지드
    로드 밸런서 .->ingress[인그레스]; + ingress-->|라우팅 규칙|service[서비스]; + subgraph 클러스터 + ingress; + service-->pod1[파드]; + service-->pod2[파드]; + end + classDef plain fill:#ddd,stroke:#fff,stroke-width:4px,color:#000; + classDef k8s fill:#326ce5,stroke:#fff,stroke-width:4px,color:#fff; + classDef cluster fill:#fff,stroke:#bbb,stroke-width:2px,color:#326ce5; + class ingress,service,pod1,pod2 k8s; + class client plain; + class cluster cluster; +{{}} -인그레스는 외부에서 서비스로 접속이 가능한 URL, 로드 밸런스 트래픽, SSL / TLS 종료 그리고 이름 기반의 가상 호스팅을 제공하도록 구성할 수 있다. [인그레스 컨트롤러](/ko/docs/concepts/services-networking/ingress-controllers)는 일반적으로 로드 밸런서를 사용해서 인그레스를 수행할 책임이 있으며, 트래픽을 처리하는데 도움이 되도록 에지 라우터 또는 추가 프런트 엔드를 구성할 수도 있다. + +인그레스는 외부에서 서비스로 접속이 가능한 URL, 로드 밸런스 트래픽, SSL / TLS 종료 그리고 이름-기반의 가상 호스팅을 제공하도록 구성할 수 있다. [인그레스 컨트롤러](/ko/docs/concepts/services-networking/ingress-controllers)는 일반적으로 로드 밸런서를 사용해서 인그레스를 수행할 책임이 있으며, 트래픽을 처리하는데 도움이 되도록 에지 라우터 또는 추가 프런트 엔드를 구성할 수도 있다. 인그레스는 임의의 포트 또는 프로토콜을 노출시키지 않는다. HTTP와 HTTPS 이외의 서비스를 인터넷에 노출하려면 보통 [Service.Type=NodePort](/ko/docs/concepts/services-networking/service/#nodeport) 또는 @@ -104,7 +115,7 @@ weight: 40 ### 리소스 백엔드 {#resource-backend} -`Resource` 백엔드는 인그레스 오브젝트의 동일한 네임스페이스 내에 있는 +`Resource` 백엔드는 인그레스 오브젝트와 동일한 네임스페이스 내에 있는 다른 쿠버네티스 리소스에 대한 ObjectRef이다. `Resource` 는 서비스와 상호 배타적인 설정이며, 둘 다 지정하면 유효성 검사에 실패한다. `Resource` 백엔드의 일반적인 용도는 정적 자산이 있는 오브젝트 스토리지 백엔드로 데이터를 @@ -272,10 +283,25 @@ test-ingress external-lb * 203.0.113.123 80 59s 트래픽을 라우팅 한다. 인그레스를 사용하면 로드 밸런서의 수를 최소로 유지할 수 있다. 예를 들어 다음과 같은 설정을 한다. -``` -foo.bar.com -> 178.91.123.132 -> / foo service1:4200 - / bar service2:8080 -``` +{{< mermaid >}} +graph LR; + client([클라이언트])-. 인그레스-매니지드
    로드 밸런서 .->ingress[인그레스, 178.91.123.132]; + ingress-->|/foo|service1[서비스 service1:4200]; + ingress-->|/bar|service2[서비스 service2:8080]; + subgraph 클러스터 + ingress; + service1-->pod1[파드]; + service1-->pod2[파드]; + service2-->pod3[파드]; + service2-->pod4[파드]; + end + classDef plain fill:#ddd,stroke:#fff,stroke-width:4px,color:#000; + classDef k8s fill:#326ce5,stroke:#fff,stroke-width:4px,color:#fff; + classDef cluster fill:#fff,stroke:#bbb,stroke-width:2px,color:#326ce5; + class ingress,service1,service2,pod1,pod2,pod3,pod4 k8s; + class client plain; + class cluster cluster; +{{}} 다음과 같은 인그레스가 필요하다. @@ -319,11 +345,26 @@ Events: 이름 기반의 가상 호스트는 동일한 IP 주소에서 여러 호스트 이름으로 HTTP 트래픽을 라우팅하는 것을 지원한다. -```none -foo.bar.com --| |-> foo.bar.com service1:80 - | 178.91.123.132 | -bar.foo.com --| |-> bar.foo.com service2:80 -``` +{{< mermaid >}} +graph LR; + client([클라이언트])-. 인그레스-매니지드
    로드 밸런서 .->ingress[인그레스, 178.91.123.132]; + ingress-->|호스트: foo.bar.com|service1[서비스 service1:80]; + ingress-->|호스트: bar.foo.com|service2[서비스 service2:80]; + subgraph 클러스터 + ingress; + service1-->pod1[파드]; + service1-->pod2[파드]; + service2-->pod3[파드]; + service2-->pod4[파드]; + end + classDef plain fill:#ddd,stroke:#fff,stroke-width:4px,color:#000; + classDef k8s fill:#326ce5,stroke:#fff,stroke-width:4px,color:#fff; + classDef cluster fill:#fff,stroke:#bbb,stroke-width:2px,color:#326ce5; + class ingress,service1,service2,pod1,pod2,pod3,pod4 k8s; + class client plain; + class cluster cluster; +{{}} + 다음 인그레스는 [호스트 헤더](https://tools.ietf.org/html/rfc7230#section-5.4)에 기반한 요청을 라우팅 하기 위해 뒷단의 로드 밸런서를 알려준다. diff --git a/content/ko/docs/concepts/services-networking/service.md b/content/ko/docs/concepts/services-networking/service.md index 9181e61d83..ee0553a467 100644 --- a/content/ko/docs/concepts/services-networking/service.md +++ b/content/ko/docs/concepts/services-networking/service.md @@ -22,8 +22,8 @@ weight: 10 ## 동기 -쿠버네티스 {{< glossary_tooltip term_id="pod" text="파드" >}}는 수명이 있다. -파드는 생성되고, 소멸된 후 부활하지 않는다. +쿠버네티스 {{< glossary_tooltip term_id="pod" text="파드" >}}는 클러스터 상태와 +일치하도록 생성되고 삭제된다. 파드는 비영구적 리소스이다. 만약 앱을 실행하기 위해 {{< glossary_tooltip term_id="deployment" text="디플로이먼트" >}}를 사용한다면, 동적으로 파드를 생성하고 제거할 수 있다. @@ -44,8 +44,8 @@ _서비스_ 로 들어가보자. 정책을 정의하는 추상적 개념이다. (때로는 이 패턴을 마이크로-서비스라고 한다.) 서비스가 대상으로 하는 파드 집합은 일반적으로 {{< glossary_tooltip text="셀렉터" term_id="selector" >}}가 결정한다. -(셀렉터가 _없는_ 서비스가 필요한 이유는 [아래](#셀렉터가-없는-서비스)를 -참조한다). +서비스 엔드포인트를 정의하는 다른 방법에 대한 자세한 내용은 +[셀렉터가 _없는_ 서비스](#셀렉터가-없는-서비스)를 참고한다. 예를 들어, 3개의 레플리카로 실행되는 스테이트리스 이미지-처리 백엔드를 생각해보자. 이러한 레플리카는 대체 가능하다. 즉, 프론트엔드는 그것들이 사용하는 백엔드를 @@ -127,12 +127,12 @@ spec: 다른 종류의 백엔드를 추상화할 수도 있다. 예를 들면 - * 프로덕션 환경에서는 외부 데이터베이스 클러스터를 사용하려고 하지만, - 테스트 환경에서는 자체 데이터베이스를 사용한다. - * 한 서비스에서 다른 - {{< glossary_tooltip term_id="namespace" text="네임스페이스">}} 또는 다른 클러스터의 서비스를 지정하려고 한다. - * 워크로드를 쿠버네티스로 마이그레이션하고 있다. 해당 방식을 평가하는 동안, - 쿠버네티스에서는 일정 비율의 백엔드만 실행한다. +* 프로덕션 환경에서는 외부 데이터베이스 클러스터를 사용하려고 하지만, + 테스트 환경에서는 자체 데이터베이스를 사용한다. +* 한 서비스에서 다른 + {{< glossary_tooltip term_id="namespace" text="네임스페이스">}} 또는 다른 클러스터의 서비스를 지정하려고 한다. +* 워크로드를 쿠버네티스로 마이그레이션하고 있다. 해당 방식을 평가하는 동안, + 쿠버네티스에서는 일정 비율의 백엔드만 실행한다. 이러한 시나리오 중에서 파드 셀렉터 _없이_ 서비스를 정의 할 수 있다. 예를 들면 @@ -150,7 +150,7 @@ spec: ``` 이 서비스에는 셀렉터가 없으므로, 해당 엔드포인트 오브젝트가 자동으로 -생성되지 *않는다*. 엔드포인트 오브젝트를 수동으로 추가하여, 서비스를 실행 중인 네트워크 주소 및 포트에 +생성되지 않는다. 엔드포인트 오브젝트를 수동으로 추가하여, 서비스를 실행 중인 네트워크 주소 및 포트에 서비스를 수동으로 매핑할 수 있다. ```yaml @@ -186,6 +186,7 @@ DNS명을 대신 사용하는 특수한 상황의 서비스이다. 자세한 내 이 문서 뒷부분의 [ExternalName](#externalname) 섹션을 참조한다. ### 엔드포인트슬라이스 + {{< feature-state for_k8s_version="v1.17" state="beta" >}} 엔드포인트슬라이스는 엔드포인트에 보다 확장 가능한 대안을 제공할 수 있는 @@ -202,9 +203,8 @@ API 리소스이다. 개념적으로 엔드포인트와 매우 유사하지만, {{< feature-state for_k8s_version="v1.19" state="beta" >}} -AppProtocol 필드는 각 서비스 포트에 사용될 애플리케이션 프로토콜을 -지정하는 방법을 제공한다. 이 필드의 값은 해당 엔드포인트와 엔드포인트슬라이스에 -의해 미러링된다. +`AppProtocol` 필드는 각 서비스 포트에 대한 애플리케이션 프로토콜을 지정하는 방법을 제공한다. +이 필드의 값은 해당 엔드포인트와 엔드포인트슬라이스 리소스에 의해 미러링된다. ## 가상 IP와 서비스 프록시 @@ -222,20 +222,19 @@ DNS 레코드를 구성하고, 라운드-로빈 이름 확인 방식을 서비스에 프록시를 사용하는 데는 몇 가지 이유가 있다. - * 레코드 TTL을 고려하지 않고, 만료된 이름 검색 결과를 - 캐싱하는 DNS 구현에 대한 오래된 역사가 있다. - * 일부 앱은 DNS 검색을 한 번만 수행하고 결과를 무기한으로 캐시한다. - * 앱과 라이브러리가 적절히 재-확인을 했다고 하더라도, DNS 레코드의 TTL이 - 낮거나 0이면 DNS에 부하가 높아 관리하기가 - 어려워 질 수 있다. +* 레코드 TTL을 고려하지 않고, 만료된 이름 검색 결과를 + 캐싱하는 DNS 구현에 대한 오래된 역사가 있다. +* 일부 앱은 DNS 검색을 한 번만 수행하고 결과를 무기한으로 캐시한다. +* 앱과 라이브러리가 적절히 재-확인을 했다고 하더라도, DNS 레코드의 TTL이 + 낮거나 0이면 DNS에 부하가 높아 관리하기가 + 어려워 질 수 있다. ### 유저 스페이스(User space) 프록시 모드 {#proxy-mode-userspace} 이 모드에서는, kube-proxy는 쿠버네티스 마스터의 서비스, 엔드포인트 오브젝트의 추가와 제거를 감시한다. 각 서비스는 로컬 노드에서 포트(임의로 선택됨)를 연다. 이 "프록시 포트"에 대한 모든 -연결은 (엔드포인트를 통해 보고된대로) 서비스의 백엔드 파드 중 하나로 -프록시된다. +연결은 (엔드포인트를 통해 보고된대로) 서비스의 백엔드 파드 중 하나로 프록시된다. kube-proxy는 사용할 백엔드 파드를 결정할 때 서비스의 `SessionAffinity` 설정을 고려한다. @@ -296,12 +295,12 @@ IPVS 프록시 모드는 iptables 모드와 유사한 넷필터 후크 기능을 IPVS는 트래픽을 백엔드 파드로 밸런싱하기 위한 추가 옵션을 제공한다. 다음과 같다. -- `rr`: 라운드-로빈 -- `lc`: 최소 연결 (가장 적은 수의 열려있는 연결) -- `dh`: 목적지 해싱 -- `sh`: 소스 해싱 -- `sed`: 최단 예상 지연 (shortest expected delay) -- `nq`: 큐 미사용 (never queue) +* `rr`: 라운드-로빈 +* `lc`: 최소 연결 (가장 적은 수의 열려있는 연결) +* `dh`: 목적지 해싱 +* `sh`: 소스 해싱 +* `sed`: 최단 예상 지연 (shortest expected delay) +* `nq`: 큐 미사용 (never queue) {{< note >}} IPVS 모드에서 kube-proxy를 실행하려면, kube-proxy를 시작하기 전에 노드에서 IPVS를 @@ -388,7 +387,7 @@ CIDR 범위 내의 유효한 IPv4 또는 IPv6 주소여야 한다. 이때 서비스 이름은 대문자이고 대시는 밑줄로 변환된다. 예를 들어, TCP 포트 6379를 개방하고 -클러스터 IP 주소 10.0.0.11이 할당된 서비스 `"redis-master"`는, +클러스터 IP 주소 10.0.0.11이 할당된 서비스 `redis-master`는, 다음 환경 변수를 생성한다. ```shell @@ -421,18 +420,18 @@ CoreDNS와 같은, 클러스터-인식 DNS 서버는 새로운 서비스를 위 모든 파드는 DNS 이름으로 서비스를 자동으로 확인할 수 있어야 한다. -예를 들면, 쿠버네티스 네임스페이스 `"my-ns"`에 `"my-service"`라는 +예를 들면, 쿠버네티스 네임스페이스 `my-ns`에 `my-service`라는 서비스가 있는 경우, 컨트롤 플레인과 DNS 서비스가 함께 작동하여 -`"my-service.my-ns"`에 대한 DNS 레코드를 만든다. `"my-ns"` 네임 스페이스의 파드들은 +`my-service.my-ns`에 대한 DNS 레코드를 만든다. `my-ns` 네임 스페이스의 파드들은 간단히 `my-service`에 대한 이름 조회를 수행하여 찾을 수 있어야 한다 -(`"my-service.my-ns"` 역시 동작함). +(`my-service.my-ns` 역시 동작함). 다른 네임스페이스의 파드들은 이름을 `my-service.my-ns`으로 사용해야 한다. 이 이름은 서비스에 할당된 클러스터 IP로 변환된다. 쿠버네티스는 또한 알려진 포트에 대한 DNS SRV (서비스) 레코드를 지원한다. -`"my-service.my-ns"` 서비스에 프로토콜이 `TCP`로 설정된 `"http"`라는 포트가 있는 경우, -IP 주소와 `"http"`에 대한 포트 번호를 검색하기 위해 `_http._tcp.my-service.my-ns` 에 대한 +`my-service.my-ns` 서비스에 프로토콜이 `TCP`로 설정된 `http`라는 포트가 있는 경우, +IP 주소와 `http`에 대한 포트 번호를 검색하기 위해 `_http._tcp.my-service.my-ns` 에 대한 DNS SRV 쿼리를 수행할 수 있다. 쿠버네티스 DNS 서버는 `ExternalName` 서비스에 접근할 수 있는 유일한 방법이다. @@ -465,9 +464,9 @@ API에서 `엔드포인트` 레코드를 생성하고, DNS 구성을 수정하 `엔드포인트` 레코드를 생성하지 않는다. 그러나 DNS 시스템은 다음 중 하나를 찾고 구성한다. - * [`ExternalName`](#externalname)-유형 서비스에 대한 CNAME 레코드 - * 다른 모든 유형에 대해, 서비스의 이름을 공유하는 모든 `엔드포인트`에 - 대한 레코드 +* [`ExternalName`](#externalname)-유형 서비스에 대한 CNAME 레코드 +* 다른 모든 유형에 대해, 서비스의 이름을 공유하는 모든 `엔드포인트`에 + 대한 레코드 ## 서비스 퍼블리싱 (ServiceTypes) {#publishing-services-service-types} @@ -479,26 +478,26 @@ API에서 `엔드포인트` 레코드를 생성하고, DNS 구성을 수정하 `Type` 값과 그 동작은 다음과 같다. - * `ClusterIP`: 서비스를 클러스터-내부 IP에 노출시킨다. 이 값을 선택하면 - 클러스터 내에서만 서비스에 도달할 수 있다. 이것은 - `ServiceTypes`의 기본 값이다. - * [`NodePort`](#nodeport): 고정 포트 (`NodePort`)로 각 노드의 IP에 서비스를 - 노출시킨다. `NodePort` 서비스가 라우팅되는 `ClusterIP` 서비스가 - 자동으로 생성된다. `:`를 요청하여, - 클러스터 외부에서 - `NodePort` 서비스에 접속할 수 있다. - * [`LoadBalancer`](#loadbalancer): 클라우드 공급자의 로드 밸런서를 사용하여 - 서비스를 외부에 노출시킨다. 외부 로드 밸런서가 라우팅되는 - `NodePort`와 `ClusterIP` 서비스가 자동으로 생성된다. - * [`ExternalName`](#externalname): 값과 함께 CNAME 레코드를 리턴하여, 서비스를 - `externalName` 필드의 콘텐츠 (예:`foo.bar.example.com`)에 - 매핑한다. - 어떤 종류의 프록시도 설정되어 있지 않다. - {{< note >}} - `ExternalName` 유형을 사용하려면 kube-dns 버전 1.7 또는 CoreDNS 버전 1.7 이상이 필요하다. - {{< /note >}} +* `ClusterIP`: 서비스를 클러스터-내부 IP에 노출시킨다. 이 값을 선택하면 + 클러스터 내에서만 서비스에 도달할 수 있다. 이것은 + `ServiceTypes`의 기본 값이다. +* [`NodePort`](#nodeport): 고정 포트 (`NodePort`)로 각 노드의 IP에 서비스를 + 노출시킨다. `NodePort` 서비스가 라우팅되는 `ClusterIP` 서비스가 + 자동으로 생성된다. `:`를 요청하여, + 클러스터 외부에서 + `NodePort` 서비스에 접속할 수 있다. +* [`LoadBalancer`](#loadbalancer): 클라우드 공급자의 로드 밸런서를 사용하여 + 서비스를 외부에 노출시킨다. 외부 로드 밸런서가 라우팅되는 + `NodePort`와 `ClusterIP` 서비스가 자동으로 생성된다. +* [`ExternalName`](#externalname): 값과 함께 CNAME 레코드를 리턴하여, 서비스를 + `externalName` 필드의 콘텐츠 (예:`foo.bar.example.com`)에 + 매핑한다. 어떤 종류의 프록시도 설정되어 있지 않다. + {{< note >}}`ExternalName` 유형을 사용하려면 kube-dns 버전 1.7 또는 + CoreDNS 버전 1.7 이상이 필요하다. + {{< /note >}} -[인그레스](/ko/docs/concepts/services-networking/ingress/)를 사용하여 서비스를 노출시킬 수도 있다. 인그레스는 서비스 유형이 아니지만, 클러스터의 진입점 역할을 한다. 동일한 IP 주소로 여러 서비스를 노출시킬 수 있기 때문에 라우팅 규칙을 단일 리소스로 통합할 수 있다. +[인그레스](/ko/docs/concepts/services-networking/ingress/)를 사용하여 서비스를 노출시킬 수도 있다. 인그레스는 서비스 유형이 아니지만, 클러스터의 진입점 역할을 한다. 동일한 IP 주소로 여러 서비스를 +노출시킬 수 있기 때문에 라우팅 규칙을 단일 리소스로 통합할 수 있다. ### NodePort 유형 {#nodeport} @@ -507,7 +506,6 @@ API에서 `엔드포인트` 레코드를 생성하고, DNS 구성을 수정하 각 노드는 해당 포트 (모든 노드에서 동일한 포트 번호)를 서비스로 프록시한다. 서비스는 할당된 포트를 `.spec.ports[*].nodePort` 필드에 나타낸다. - 포트를 프록시하기 위해 특정 IP를 지정하려면 kube-proxy의 `--nodeport-addresses` 플래그를 특정 IP 블록으로 설정할 수 있다. 이것은 쿠버네티스 v1.10부터 지원된다. 이 플래그는 쉼표로 구분된 IP 블록 목록 (예: 10.0.0.0/8, 192.0.2.0/25)을 사용하여 kube-proxy가 로컬 노드로 고려해야 하는 IP 주소 범위를 지정한다. @@ -528,6 +526,7 @@ NodePort를 사용하면 자유롭게 자체 로드 밸런싱 솔루션을 설 `.spec.clusterIP:spec.ports[*].port`로 표기된다. (kube-proxy에서 `--nodeport-addresses` 플래그가 설정되면, 는 NodeIP를 필터링한다.) 예를 들면 + ```yaml apiVersion: v1 kind: Service @@ -604,19 +603,21 @@ SCTP를 사용하는 경우, `LoadBalancer` 서비스 유형에 대한 아래의 {{< /note >}} #### 내부 로드 밸런서 {#internal-load-balancer} + 혼재된 환경에서는 서비스의 트래픽을 동일한 (가상) 네트워크 주소 블록 내로 -라우팅해야하는 경우가 있다. +라우팅해야 하는 경우가 있다. 수평 분할 DNS 환경에서는 외부와 내부 트래픽을 엔드포인트로 라우팅 할 수 있는 두 개의 서비스가 필요하다. -서비스에 다음 어노테이션 중 하나를 추가하여 이를 수행할 수 있다. -추가할 어노테이션은 사용 중인 클라우드 서비스 공급자에 따라 다르다. +내부 로드 밸런서를 설정하려면, 사용 중인 클라우드 서비스 공급자에 따라 +다음의 어노테이션 중 하나를 서비스에 추가한다. {{< tabs name="service_tabs" >}} {{% tab name="Default" %}} 탭 중 하나를 선택 {{% /tab %}} {{% tab name="GCP" %}} + ```yaml [...] metadata: @@ -625,8 +626,10 @@ metadata: cloud.google.com/load-balancer-type: "Internal" [...] ``` + {{% /tab %}} {{% tab name="AWS" %}} + ```yaml [...] metadata: @@ -635,8 +638,10 @@ metadata: service.beta.kubernetes.io/aws-load-balancer-internal: "true" [...] ``` + {{% /tab %}} {{% tab name="Azure" %}} + ```yaml [...] metadata: @@ -645,8 +650,10 @@ metadata: service.beta.kubernetes.io/azure-load-balancer-internal: "true" [...] ``` + {{% /tab %}} {{% tab name="IBM Cloud" %}} + ```yaml [...] metadata: @@ -655,8 +662,10 @@ metadata: service.kubernetes.io/ibm-load-balancer-cloud-provider-ip-type: "private" [...] ``` + {{% /tab %}} {{% tab name="OpenStack" %}} + ```yaml [...] metadata: @@ -665,8 +674,10 @@ metadata: service.beta.kubernetes.io/openstack-internal-load-balancer: "true" [...] ``` + {{% /tab %}} {{% tab name="Baidu Cloud" %}} + ```yaml [...] metadata: @@ -675,8 +686,10 @@ metadata: service.beta.kubernetes.io/cce-load-balancer-internal-vpc: "true" [...] ``` + {{% /tab %}} {{% tab name="Tencent Cloud" %}} + ```yaml [...] metadata: @@ -684,8 +697,10 @@ metadata: service.kubernetes.io/qcloud-loadbalancer-internal-subnetid: subnet-xxxxx [...] ``` + {{% /tab %}} {{% tab name="Alibaba Cloud" %}} + ```yaml [...] metadata: @@ -693,10 +708,10 @@ metadata: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type: "intranet" [...] ``` + {{% /tab %}} {{< /tabs >}} - #### AWS에서 TLS 지원 {#ssl-support-on-aws} AWS에서 실행되는 클러스터에서 부분적으로 TLS / SSL을 지원하기 위해, `LoadBalancer` 서비스에 세 가지 @@ -821,7 +836,6 @@ Classic ELB의 연결 드레이닝은 사용하여 인스턴스를 해제하기 전에, 기존 연결을 열어 두는 목적으로 최대 시간을 초 단위로 설정할 수도 있다. - ```yaml metadata: name: my-service @@ -989,6 +1003,7 @@ spec: type: ExternalName externalName: my.database.example.com ``` + {{< note >}} ExternalName은 IPv4 주소 문자열을 허용하지만, IP 주소가 아닌 숫자로 구성된 DNS 이름을 허용한다. IPv4 주소와 유사한 ExternalName은 CoreDNS 또는 ingress-nginx에 의해 확인되지 않는데, ExternalName은 정식(canonical) DNS 이름을 지정하기 때문이다. IP 주소를 하드 코딩하려면, @@ -1043,7 +1058,7 @@ spec: ## 단점 -VIP용으로 유저스페이스 프록시를 사용하면, 중소 급 스케일에서는 동작하지만, 수천 개의 +VIP용 유저스페이스 프록시를 사용하면 중소 규모의 스케일에서는 동작하지만, 수천 개의 서비스가 포함된 대규모 클러스터로는 확장되지 않는다. [포털에 대한 독창적인 설계 제안](https://github.com/kubernetes/kubernetes/issues/1107)에 이에 대한 자세한 내용이 있다. @@ -1171,7 +1186,7 @@ IPVS는 로드 밸런싱을 위해 설계되었고 커널-내부 해시 테이 {{< note >}} 서비스 대신 {{< glossary_tooltip term_id="ingress" text="인그레스" >}} 를 사용하여 -HTTP / HTTPS 서비스를 노출할 수도 있다. +HTTP/HTTPS 서비스를 노출할 수도 있다. {{< /note >}} ### PROXY 프로토콜 @@ -1187,6 +1202,7 @@ LoadBalancer 모드의 서비스를 사용하여 쿠버네티스 자체 외부 ``` PROXY TCP4 192.0.2.202 10.0.42.7 12345 7\r\n ``` + 클라이언트 데이터가 뒤따라온다. ### SCTP @@ -1225,11 +1241,8 @@ SCTP는 윈도우 기반 노드를 지원하지 않는다. kube-proxy는 유저스페이스 모드에 있을 때 SCTP 연결 관리를 지원하지 않는다. {{< /warning >}} - - ## {{% heading "whatsnext" %}} - * [서비스와 애플리케이션 연결](/ko/docs/concepts/services-networking/connect-applications-service/) 알아보기 * [인그레스](/ko/docs/concepts/services-networking/ingress/)에 대해 알아보기 * [엔드포인트슬라이스](/ko/docs/concepts/services-networking/endpoint-slices/)에 대해 알아보기 diff --git a/content/ko/docs/concepts/storage/persistent-volumes.md b/content/ko/docs/concepts/storage/persistent-volumes.md index d67ee4d993..12f35aa428 100644 --- a/content/ko/docs/concepts/storage/persistent-volumes.md +++ b/content/ko/docs/concepts/storage/persistent-volumes.md @@ -25,7 +25,7 @@ _퍼시스턴트볼륨클레임_ (PVC)은 사용자의 스토리지에 대한 퍼시스턴트볼륨클레임을 사용하면 사용자가 추상화된 스토리지 리소스를 사용할 수 있지만, 다른 문제들 때문에 성능과 같은 다양한 속성을 가진 퍼시스턴트볼륨이 필요한 경우가 일반적이다. 클러스터 관리자는 사용자에게 해당 볼륨의 구현 방법에 대한 세부 정보를 제공하지 않고 단순히 크기와 접근 모드와는 다른 방식으로 다양한 퍼시스턴트볼륨을 제공할 수 있어야 한다. 이러한 요구에는 _스토리지클래스_ 리소스가 있다. -[실습 예제와 함께 상세한 내용](/docs/tasks/configure-pod-container/configure-persistent-volume-storage/)을 참고하길 바란다. +[실습 예제와 함께 상세한 내용](/ko/docs/tasks/configure-pod-container/configure-persistent-volume-storage/)을 참고하길 바란다. ## 볼륨과 클레임 라이프사이클 @@ -755,8 +755,8 @@ spec: ## {{% heading "whatsnext" %}} -* [퍼시스턴트볼륨 생성](/docs/tasks/configure-pod-container/configure-persistent-volume-storage/#create-a-persistentvolume)에 대해 자세히 알아보기 -* [퍼시스턴트볼륨클레임 생성](/docs/tasks/configure-pod-container/configure-persistent-volume-storage/#create-a-persistentvolumeclaim)에 대해 자세히 알아보기 +* [퍼시스턴트볼륨 생성](/ko/docs/tasks/configure-pod-container/configure-persistent-volume-storage/#퍼시스턴트볼륨-생성하기)에 대해 자세히 알아보기 +* [퍼시스턴트볼륨클레임 생성](/ko/docs/tasks/configure-pod-container/configure-persistent-volume-storage/#퍼시스턴트볼륨클레임-생성하기)에 대해 자세히 알아보기 * [퍼시스턴트 스토리지 설계 문서](https://git.k8s.io/community/contributors/design-proposals/storage/persistent-storage.md) 읽어보기 ### 참고 diff --git a/content/ko/docs/concepts/storage/storage-classes.md b/content/ko/docs/concepts/storage/storage-classes.md index 69b74ddefe..38d59b6fb0 100644 --- a/content/ko/docs/concepts/storage/storage-classes.md +++ b/content/ko/docs/concepts/storage/storage-classes.md @@ -410,6 +410,21 @@ parameters: ### vSphere +vSphere 스토리지 클래스에는 두 가지 유형의 프로비저닝 도구가 있다. + +- [CSI 프로비저닝 도구](#csi-프로비저닝-도구): `csi.vsphere.vmware.com` +- [vCP 프로비저닝 도구](#vcp-프로비저닝-도구): `kubernetes.io/vsphere-volume` + +인-트리 프로비저닝 도구는 [사용 중단](/blog/2019/12/09/kubernetes-1-17-feature-csi-migration-beta/#why-are-we-migrating-in-tree-plugins-to-csi)되었다. CSI 프로비저닝 도구에 대한 자세한 내용은 [쿠버네티스 vSphere CSI 드라이버](https://vsphere-csi-driver.sigs.k8s.io/) 및 [vSphereVolume CSI 마이그레이션](/ko/docs/concepts/storage/volumes/#csi-마이그레이션)을 참고한다. + +#### CSI 프로비저닝 도구 {#vsphere-provisioner-csi} + +vSphere CSI 스토리지클래스 프로비저닝 도구는 Tanzu 쿠버네티스 클러스터에서 작동한다. 예시는 [vSphere CSI 리포지터리](https://raw.githubusercontent.com/kubernetes-sigs/vsphere-csi-driver/master/example/vanilla-k8s-file-driver/example-sc.yaml)를 참조한다. + +#### vCP 프로비저닝 도구 + +다음 예시에서는 VMware 클라우드 공급자(vCP) 스토리지클래스 프로비저닝 도구를 사용한다. + 1. 사용자 지정 디스크 형식으로 스토리지클래스를 생성한다. ```yaml @@ -575,7 +590,7 @@ parameters: 이 Quobyte 테넌트는 이미 Quobyte에 있어야 한다. 기본값은 "DEFAULT". -### Azure 디스크크 +### Azure 디스크 #### Azure 비관리 디스크 스토리지 클래스 {#azure-unmanaged-disk-storage-class} diff --git a/content/ko/docs/concepts/storage/volumes.md b/content/ko/docs/concepts/storage/volumes.md index 2ad8b36f90..9077eec5d2 100644 --- a/content/ko/docs/concepts/storage/volumes.md +++ b/content/ko/docs/concepts/storage/volumes.md @@ -751,7 +751,7 @@ spec: ### persistentVolumeClaim {#persistentvolumeclaim} `persistentVolumeClaim` 볼륨은 -[퍼시스턴트볼륨](/ko/docs/concepts/storage/persistent-volumes)을 파드에 마운트하는데 사용한다. 퍼시스턴트볼륨은 +[퍼시스턴트볼륨](/ko/docs/concepts/storage/persistent-volumes)을 파드에 마운트하는데 사용한다. 퍼시스턴트볼륨클레임은 사용자가 특정 클라우드 환경의 세부 내용을 몰라도 내구성이있는 스토리지 (GCE 퍼시스턴트디스크 또는 iSCSI 볼륨와 같은)를 "클레임" 할 수 있는 방법이다. diff --git a/content/ko/docs/concepts/workloads/controllers/cron-jobs.md b/content/ko/docs/concepts/workloads/controllers/cron-jobs.md index 44590ccdfc..fe46abd604 100644 --- a/content/ko/docs/concepts/workloads/controllers/cron-jobs.md +++ b/content/ko/docs/concepts/workloads/controllers/cron-jobs.md @@ -44,7 +44,7 @@ kube-controller-manager 컨테이너에 설정된 시간대는 {{< codenew file="application/job/cronjob.yaml" >}} -([크론잡으로 자동화된 작업 실행하기](/docs/tasks/job/automated-tasks-with-cron-jobs/)는 +([크론잡으로 자동화된 작업 실행하기](/ko/docs/tasks/job/automated-tasks-with-cron-jobs/)는 이 예시를 더 자세히 설명한다.) ## 크론잡의 한계 {#cron-job-limitations} @@ -85,4 +85,4 @@ Cannot determine if job needs to be started. Too many missed start time (> 100). 크론잡 `schedule` 필드의 포맷을 문서화 한다. 크론잡 생성과 작업에 대한 지침과 크론잡 매니페스트의 -예는 [크론잡으로 자동화된 작업 실행하기](/docs/tasks/job/automated-tasks-with-cron-jobs/)를 참조한다. +예는 [크론잡으로 자동화된 작업 실행하기](/ko/docs/tasks/job/automated-tasks-with-cron-jobs/)를 참조한다. diff --git a/content/ko/docs/concepts/workloads/controllers/job.md b/content/ko/docs/concepts/workloads/controllers/job.md index f7c42e9294..50d446a00e 100644 --- a/content/ko/docs/concepts/workloads/controllers/job.md +++ b/content/ko/docs/concepts/workloads/controllers/job.md @@ -331,7 +331,7 @@ spec: - 각 작업 항목에 대한 하나의 잡 오브젝트 vs 모든 작업 항목에 대한 단일 잡 오브젝트. 후자는 작업 항목 수가 많은 경우 더 적합하다. 전자는 사용자와 시스템이 많은 수의 잡 오브젝트를 관리해야 하는 약간의 오버헤드를 만든다. -- 작업 항목과 동일한 개수의 파드 생성 vs 각 파드에서 다수의 작업 항목을 처리 +- 작업 항목과 동일한 개수의 파드 생성 vs 각 파드에서 다수의 작업 항목을 처리. 전자는 일반적으로 기존 코드와 컨테이너를 거의 수정할 필요가 없다. 후자는 이전 글 머리표(-)와 비슷한 이유로 많은 수의 작업 항목에 적합하다. - 여러 접근 방식이 작업 큐를 사용한다. 이를 위해서는 큐 서비스를 실행하고, @@ -346,7 +346,7 @@ spec: | -------------------------------------------------------------------- |:-----------------:|:---------------------------:|:-------------------:|:-------------------:| | [잡 템플릿 확장](/ko/docs/tasks/job/parallel-processing-expansion/) | | | ✓ | ✓ | | [작업 항목 당 파드가 있는 큐](/docs/tasks/job/coarse-parallel-processing-work-queue/) | ✓ | | 때때로 | ✓ | -| [가변 파드 수를 가진 큐](/docs/tasks/job/fine-parallel-processing-work-queue/) | ✓ | ✓ | | ✓ | +| [가변 파드 수를 가진 큐](/ko/docs/tasks/job/fine-parallel-processing-work-queue/) | ✓ | ✓ | | ✓ | | 정적 작업이 할당된 단일 잡 | ✓ | | ✓ | | `.spec.completions` 로 완료를 지정할 때, 잡 컨트롤러에 의해 생성된 각 파드는 @@ -362,7 +362,7 @@ spec: | -------------------------------------------------------------------- |:-------------------:|:--------------------:| | [잡 템플릿 확장](/ko/docs/tasks/job/parallel-processing-expansion/) | 1 | 1이어야 함 | | [작업 항목 당 파드가 있는 큐](/docs/tasks/job/coarse-parallel-processing-work-queue/) | W | any | -| [가변 파드 수를 가진 큐](/docs/tasks/job/fine-parallel-processing-work-queue/) | 1 | any | +| [가변 파드 수를 가진 큐](/ko/docs/tasks/job/fine-parallel-processing-work-queue/) | 1 | any | | 정적 작업이 할당된 단일 잡 | W | any | diff --git a/content/ko/docs/concepts/workloads/pods/_index.md b/content/ko/docs/concepts/workloads/pods/_index.md index 6c025fc2a6..6eeabfd48e 100644 --- a/content/ko/docs/concepts/workloads/pods/_index.md +++ b/content/ko/docs/concepts/workloads/pods/_index.md @@ -120,7 +120,7 @@ term_id="deployment" >}} 또는 {{< glossary_tooltip text="잡(Job)" term_id="jo {{< /note >}} 파드 오브젝트에 대한 매니페스트를 만들 때, 지정된 이름이 유효한 -[DNS 하위 도메인 이름](/ko/docs/concepts/overview/working-with-objects/names/#dns-서브도메인-이름)인지 확인한다. +[DNS 서브도메인 이름](/ko/docs/concepts/overview/working-with-objects/names/#dns-서브도메인-이름)인지 확인한다. ### 파드와 컨트롤러 diff --git a/content/ko/docs/concepts/workloads/pods/pod-topology-spread-constraints.md b/content/ko/docs/concepts/workloads/pods/pod-topology-spread-constraints.md index c81db66021..31587a99e6 100644 --- a/content/ko/docs/concepts/workloads/pods/pod-topology-spread-constraints.md +++ b/content/ko/docs/concepts/workloads/pods/pod-topology-spread-constraints.md @@ -30,13 +30,23 @@ node4 Ready 2m43s v1.16.0 node=node4,zone=zoneB 그러면 클러스터는 논리적으로 다음과 같이 보이게 된다. -``` -+---------------+---------------+ -| zoneA | zoneB | -+-------+-------+-------+-------+ -| node1 | node2 | node3 | node4 | -+-------+-------+-------+-------+ -``` +{{}} +graph TB + subgraph "zoneB" + n3(Node3) + n4(Node4) + end + subgraph "zoneA" + n1(Node1) + n2(Node2) + end + + classDef plain fill:#ddd,stroke:#fff,stroke-width:4px,color:#000; + classDef k8s fill:#326ce5,stroke:#fff,stroke-width:4px,color:#fff; + classDef cluster fill:#fff,stroke:#bbb,stroke-width:2px,color:#326ce5; + class n1,n2,n3,n4 k8s; + class zoneA,zoneB cluster; +{{< /mermaid >}} 레이블을 수동으로 적용하는 대신에, 사용자는 대부분의 클러스터에서 자동으로 생성되고 채워지는 [잘-알려진 레이블](/docs/reference/kubernetes-api/labels-annotations-taints/)을 재사용할 수 있다. @@ -80,17 +90,25 @@ spec: ### 예시: 단수 토폴로지 분배 제약 조건 -4개 노드를 가지는 클러스터에 `foo:bar` 가 레이블된 3개의 파드가 node1, node2 그리고 node3에 각각 위치한다고 가정한다(`P`는 파드를 나타낸다). +4개 노드를 가지는 클러스터에 `foo:bar` 가 레이블된 3개의 파드가 node1, node2 그리고 node3에 각각 위치한다고 가정한다. -``` -+---------------+---------------+ -| zoneA | zoneB | -+-------+-------+-------+-------+ -| node1 | node2 | node3 | node4 | -+-------+-------+-------+-------+ -| P | P | P | | -+-------+-------+-------+-------+ -``` +{{}} +graph BT + subgraph "zoneB" + p3(Pod) --> n3(Node3) + n4(Node4) + end + subgraph "zoneA" + p1(Pod) --> n1(Node1) + p2(Pod) --> n2(Node2) + end + + classDef plain fill:#ddd,stroke:#fff,stroke-width:4px,color:#000; + classDef k8s fill:#326ce5,stroke:#fff,stroke-width:4px,color:#fff; + classDef cluster fill:#fff,stroke:#bbb,stroke-width:2px,color:#326ce5; + class n1,n2,n3,n4,p1,p2,p3 k8s; + class zoneA,zoneB cluster; +{{< /mermaid >}} 신규 파드가 기존 파드와 함께 영역에 걸쳐서 균등하게 분배되도록 하려면, 스펙(spec)은 다음과 같이 주어질 수 있다. @@ -100,15 +118,46 @@ spec: 만약 스케줄러가 이 신규 파드를 "zoneA"에 배치하면 파드 분포는 [3, 1]이 되며, 따라서 실제 차이(skew)는 2 (3 - 1)가 되어 `maxSkew: 1` 를 위반하게 된다. 이 예시에서는 들어오는 파드는 오직 "zoneB"에만 배치할 수 있다. -``` -+---------------+---------------+ +---------------+---------------+ -| zoneA | zoneB | | zoneA | zoneB | -+-------+-------+-------+-------+ +-------+-------+-------+-------+ -| node1 | node2 | node3 | node4 | OR | node1 | node2 | node3 | node4 | -+-------+-------+-------+-------+ +-------+-------+-------+-------+ -| P | P | P | P | | P | P | P P | | -+-------+-------+-------+-------+ +-------+-------+-------+-------+ -``` +{{}} +graph BT + subgraph "zoneB" + p3(Pod) --> n3(Node3) + p4(mypod) --> n4(Node4) + end + subgraph "zoneA" + p1(Pod) --> n1(Node1) + p2(Pod) --> n2(Node2) + end + + classDef plain fill:#ddd,stroke:#fff,stroke-width:4px,color:#000; + classDef k8s fill:#326ce5,stroke:#fff,stroke-width:4px,color:#fff; + classDef cluster fill:#fff,stroke:#bbb,stroke-width:2px,color:#326ce5; + class n1,n2,n3,n4,p1,p2,p3 k8s; + class p4 plain; + class zoneA,zoneB cluster; +{{< /mermaid >}} + +OR + +{{}} +graph BT + subgraph "zoneB" + p3(Pod) --> n3(Node3) + p4(mypod) --> n3 + n4(Node4) + end + subgraph "zoneA" + p1(Pod) --> n1(Node1) + p2(Pod) --> n2(Node2) + end + + classDef plain fill:#ddd,stroke:#fff,stroke-width:4px,color:#000; + classDef k8s fill:#326ce5,stroke:#fff,stroke-width:4px,color:#fff; + classDef cluster fill:#fff,stroke:#bbb,stroke-width:2px,color:#326ce5; + class n1,n2,n3,n4,p1,p2,p3 k8s; + class p4 plain; + class zoneA,zoneB cluster; +{{< /mermaid >}} 사용자는 파드 스펙을 조정해서 다음과 같은 다양한 요구사항을 충족할 수 있다. @@ -118,17 +167,26 @@ spec: ### 예시: 다중 토폴로지 분배 제약 조건 -4개 노드를 가지는 클러스터에 `foo:bar` 가 레이블된 3개의 파드가 node1, node2 그리고 node3에 각각 위치한다고 가정한다(`P`는 파드를 나타낸다). +4개 노드를 가지는 클러스터에 `foo:bar` 가 레이블된 3개의 파드가 node1, node2 그리고 node3에 각각 위치한다고 가정한다. -``` -+---------------+---------------+ -| zoneA | zoneB | -+-------+-------+-------+-------+ -| node1 | node2 | node3 | node4 | -+-------+-------+-------+-------+ -| P | P | P | | -+-------+-------+-------+-------+ -``` +{{}} +graph BT + subgraph "zoneB" + p3(Pod) --> n3(Node3) + n4(Node4) + end + subgraph "zoneA" + p1(Pod) --> n1(Node1) + p2(Pod) --> n2(Node2) + end + + classDef plain fill:#ddd,stroke:#fff,stroke-width:4px,color:#000; + classDef k8s fill:#326ce5,stroke:#fff,stroke-width:4px,color:#fff; + classDef cluster fill:#fff,stroke:#bbb,stroke-width:2px,color:#326ce5; + class n1,n2,n3,n4,p1,p2,p3 k8s; + class p4 plain; + class zoneA,zoneB cluster; +{{< /mermaid >}} 사용자는 2개의 TopologySpreadConstraints를 사용해서 영역과 노드에 파드를 분배하는 것을 제어할 수 있다. @@ -138,15 +196,24 @@ spec: 다중 제약 조건은 충돌로 이어질 수 있다. 3개의 노드를 가지는 클러스터 하나가 2개의 영역에 걸쳐 있다고 가정한다. -``` -+---------------+-------+ -| zoneA | zoneB | -+-------+-------+-------+ -| node1 | node2 | node3 | -+-------+-------+-------+ -| P P | P | P P | -+-------+-------+-------+ -``` +{{}} +graph BT + subgraph "zoneB" + p4(Pod) --> n3(Node3) + p5(Pod) --> n3 + end + subgraph "zoneA" + p1(Pod) --> n1(Node1) + p2(Pod) --> n1 + p3(Pod) --> n2(Node2) + end + + classDef plain fill:#ddd,stroke:#fff,stroke-width:4px,color:#000; + classDef k8s fill:#326ce5,stroke:#fff,stroke-width:4px,color:#fff; + classDef cluster fill:#fff,stroke:#bbb,stroke-width:2px,color:#326ce5; + class n1,n2,n3,n4,p1,p2,p3,p4,p5 k8s; + class zoneA,zoneB cluster; +{{< /mermaid >}} 만약 사용자가 "two-constraints.yaml" 을 이 클러스터에 적용하면, "mypod"가 `Pending` 상태로 유지되는 것을 알게 된다. 이러한 이유는, 첫 번째 제약 조건을 충족하기 위해 "mypod"는 오직 "zoneB"에만 놓을 수 있다. 두 번째 제약 조건에서는 "mypod"는 오직 "node2"에만 놓을 수 있다. 그러면 "zoneB"와 "node2"의 공동 결과는 아무것도 반환되지 않는다. @@ -169,15 +236,37 @@ spec: zoneA 에서 zoneC에 걸쳐있고, 5개의 노드를 가지는 클러스터가 있다고 가정한다. - ``` - +---------------+---------------+-------+ - | zoneA | zoneB | zoneC | - +-------+-------+-------+-------+-------+ - | node1 | node2 | node3 | node4 | node5 | - +-------+-------+-------+-------+-------+ - | P | P | P | | | - +-------+-------+-------+-------+-------+ - ``` + {{}} + graph BT + subgraph "zoneB" + p3(Pod) --> n3(Node3) + n4(Node4) + end + subgraph "zoneA" + p1(Pod) --> n1(Node1) + p2(Pod) --> n2(Node2) + end + + classDef plain fill:#ddd,stroke:#fff,stroke-width:4px,color:#000; + classDef k8s fill:#326ce5,stroke:#fff,stroke-width:4px,color:#fff; + classDef cluster fill:#fff,stroke:#bbb,stroke-width:2px,color:#326ce5; + class n1,n2,n3,n4,p1,p2,p3 k8s; + class p4 plain; + class zoneA,zoneB cluster; + {{< /mermaid >}} + + {{}} + graph BT + subgraph "zoneC" + n5(Node5) + end + + classDef plain fill:#ddd,stroke:#fff,stroke-width:4px,color:#000; + classDef k8s fill:#326ce5,stroke:#fff,stroke-width:4px,color:#fff; + classDef cluster fill:#fff,stroke:#bbb,stroke-width:2px,color:#326ce5; + class n5 k8s; + class zoneC cluster; + {{< /mermaid >}} 그리고 알다시피 "zoneC"는 제외해야 한다. 이 경우에, "mypod"가 "zoneC"가 아닌 "zoneB"에 배치되도록 yaml을 다음과 같이 구성할 수 있다. 마찬가지로 `spec.nodeSelector` 도 존중된다. diff --git a/content/ko/docs/contribute/suggesting-improvements.md b/content/ko/docs/contribute/suggesting-improvements.md index c7fe87ac07..7dd9f80a71 100644 --- a/content/ko/docs/contribute/suggesting-improvements.md +++ b/content/ko/docs/contribute/suggesting-improvements.md @@ -24,7 +24,7 @@ card: 기존 콘텐츠에 대한 개선을 제안하거나, 오류를 발견하면, 이슈를 연다. -1. 페이지 하단으로 이동하여 **이슈 생성하기** 버튼을 클릭한다. 그러면 헤더가 +1. 오른쪽 사이드바에서 **문서에 이슈 생성** 링크를 클릭한다. 그러면 헤더가 미리 채워진 GitHub 이슈 페이지로 리디렉션된다. 2. 개선을 위한 문제나 제안 사항을 설명한다. 가능한 한 많은 정보를 제공한다. 3. **Submit new issue** 를 클릭한다. @@ -61,5 +61,3 @@ card: - [행동 강령](/ko/community/code-of-conduct/)을 따른다. 동료 기여자를 존중한다. 예를 들어, "문서가 끔찍하다"는 도움이 되지 않거나 예의 바르지 않은 피드백이다. - - diff --git a/content/ko/docs/reference/access-authn-authz/authorization.md b/content/ko/docs/reference/access-authn-authz/authorization.md index 3de1de0276..f85b1635e6 100644 --- a/content/ko/docs/reference/access-authn-authz/authorization.md +++ b/content/ko/docs/reference/access-authn-authz/authorization.md @@ -11,7 +11,7 @@ weight: 60 쿠버네티스에서는 사용자의 요청이 인가(접근 권한을 부여) 받기 전에 사용자가 인증(로그인)되어야 한다. -인증에 대한 자세한 내용은 [쿠버네티스 API 접근 제어하기](/docs/reference/access-authn-authz/controlling-access/)를 +인증에 대한 자세한 내용은 [쿠버네티스 API 접근 제어하기](/ko/docs/reference/access-authn-authz/controlling-access/)를 참고한다. 쿠버네티스는 REST API 요청에 공통적인 속성을 요구한다. @@ -42,7 +42,7 @@ weight: 60 * **extra** - 인증 계층에서 제공하는 문자열 값에 대한 임의의 문자열 키 맵. * **API** - 요청이 API 리소스에 대한 것인지 여부. * **Request path** - `/api` 또는 `/healthz`와 같이 다양한 리소스가 아닌 엔드포인트의 경로. - * **API request verb** - `get`, `list`, `create`, `update`, `patch`, `watch`, `delete`, `deletecollection`과 같은 리소스 요청에 사용하는 API 동사. 리소스 API 엔드포인트의 요청 동사를 결정하려면 [요청 동사 결정](/docs/reference/access-authn-authz/authorization/#determine-the-request-verb)을 참고한다. + * **API request verb** - `get`, `list`, `create`, `update`, `patch`, `watch`, `delete`, `deletecollection`과 같은 리소스 요청에 사용하는 API 동사. 리소스 API 엔드포인트의 요청 동사를 결정하려면 [요청 동사 결정](/ko/docs/reference/access-authn-authz/authorization/#요청-동사-결정)을 참고한다. * **HTTP request verb** - `get`, `post`, `put`, `delete`처럼 소문자 HTTP 메서드는 리소스가 아닌 요청에 사용한다. * **Resource** - 접근 중인 리소스의 ID 또는 이름(리소스 요청만 해당) -- `get`, `update`, `patch`, `delete` 동사를 사용하는 리소스 요청의 경우 리소스 이름을 지정해야 한다. * **Subresource** - 접근 중인 하위 리소스(리소스 요청만 해당). @@ -197,5 +197,5 @@ status: ## {{% heading "whatsnext" %}} -* 인증에 대한 자세한 내용은 [쿠버네티스 API 접근 제어하기](/docs/reference/access-authn-authz/controlling-access/)에서 **인증**을 참조한다. +* 인증에 대한 자세한 내용은 [쿠버네티스 API 접근 제어하기](/ko/docs/reference/access-authn-authz/controlling-access/)에서 **인증**을 참조한다. * 어드미션 제어에 대한 자세한 내용은 [어드미션 컨트롤러 사용하기](/docs/reference/access-authn-authz/admission-controllers/)를 참조한다. diff --git a/content/ko/docs/reference/access-authn-authz/controlling-access.md b/content/ko/docs/reference/access-authn-authz/controlling-access.md index e823ba60a5..d9f34a202e 100644 --- a/content/ko/docs/reference/access-authn-authz/controlling-access.md +++ b/content/ko/docs/reference/access-authn-authz/controlling-access.md @@ -11,7 +11,7 @@ weight: 5 사용자는 `kubectl`, 클라이언트 라이브러리 또는 REST 요청을 통해 -[API에 접근한다](/docs/tasks/access-application-cluster/access-cluster/). +[API에 접근한다](/ko/docs/tasks/access-application-cluster/access-cluster/). 사용자와 쿠버네티스 서비스 어카운트 모두 API에 접근할 수 있다. 요청이 API에 도달하면, 다음 다이어그램에 설명된 몇 가지 단계를 거친다. @@ -100,7 +100,7 @@ Bob이 `projectCaribou` 네임스페이스에 있는 오브젝트에 쓰기(`cre 쿠버네티스는 ABAC 모드, RBAC 모드, 웹훅 모드와 같은 여러 개의 인가 모듈을 지원한다. 관리자가 클러스터를 생성할 때 API 서버에서 사용해야 하는 인가 모듈을 구성했다. 인가 모듈이 2개 이상 구성되면 쿠버네티스가 각 모듈을 확인하고, 어느 모듈이 요청을 승인하면 요청을 진행할 수 있다. 모든 모듈이 요청을 거부하면 요청이 거부된다(HTTP 상태 코드 403). -인가 모듈을 사용한 정책 생성을 포함해 쿠버네티스 인가에 대해 더 배우려면 [인가 개요](/docs/reference/access-authz/authorization/)를 참조한다. +인가 모듈을 사용한 정책 생성을 포함해 쿠버네티스 인가에 대해 더 배우려면 [인가 개요](/ko/docs/reference/access-authn-authz/authorization/)를 참조한다. ## 어드미션 제어 @@ -159,4 +159,3 @@ GCE(구글 컴퓨트 엔진) 및 다른 클라우드 제공자에서 `kube-up.sh API 서버는 포트 443에서 서비스한다. GCE에서는 외부 HTTPS가 API에 접근할 수 있도록 프로젝트에서 방화벽 규칙이 구성된다. 이외에 클러스터 설정 방법은 다양하다. - diff --git a/content/ko/docs/reference/glossary/index.md b/content/ko/docs/reference/glossary/index.md index c7a4b3a927..9af1b3d078 100755 --- a/content/ko/docs/reference/glossary/index.md +++ b/content/ko/docs/reference/glossary/index.md @@ -7,6 +7,6 @@ weight: 5 card: name: reference weight: 10 - title: Glossary + title: 용어집 --- diff --git a/content/ko/docs/reference/glossary/pod-lifecycle.md b/content/ko/docs/reference/glossary/pod-lifecycle.md index 52f107ecb9..cdec826e9d 100644 --- a/content/ko/docs/reference/glossary/pod-lifecycle.md +++ b/content/ko/docs/reference/glossary/pod-lifecycle.md @@ -16,4 +16,4 @@ short-description: > -[파드 라이프사이클](/ko/docs/concepts/workloads/pods/pod-lifecycle/)은 파드의 라이프사이클에 대한 고수준의 요약이다. 다섯 가지 파드 단계가 있다: Pending, Running, Succeeded, Failed, 그리고 Unknown. [파드 스테이터스](/docs/reference/generated/kubernetes-api/v1.13/#podstatus-v1-core)의 `phase` 필드에 파드 상태에 대한 자세한 설명이 요약되어 있다. +[파드 라이프사이클](/ko/docs/concepts/workloads/pods/pod-lifecycle/)은 파드의 라이프사이클에 대한 고수준의 요약이다. 다섯 가지 파드 단계가 있다: Pending, Running, Succeeded, Failed, 그리고 Unknown. [파드 스테이터스](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#podstatus-v1-core)의 `phase` 필드에 파드 상태에 대한 자세한 설명이 요약되어 있다. diff --git a/content/ko/docs/reference/kubectl/docker-cli-to-kubectl.md b/content/ko/docs/reference/kubectl/docker-cli-to-kubectl.md new file mode 100644 index 0000000000..e81dd65b4a --- /dev/null +++ b/content/ko/docs/reference/kubectl/docker-cli-to-kubectl.md @@ -0,0 +1,372 @@ +--- +title: 도커 사용자를 위한 kubectl +content_type: concept + + + +--- + + +당신은 쿠버네티스 커맨드 라인 도구인 kubectl을 사용하여 API 서버와 상호 작용할 수 있다. 만약 도커 커맨드 라인 도구에 익숙하다면 kubectl을 사용하는 것은 간단하다. 다음 섹션에서는 도커의 하위 명령을 보여주고 kubectl과 같은 명령어를 설명한다. + + + +## docker run + +nginx 디플로이먼트(Deployment)를 실행하고 해당 디플로이먼트를 노출시키려면, [kubectl create deployment](/docs/reference/generated/kubectl/kubectl-commands#-em-deployment-em-)을 참고한다. +docker: + +```shell +docker run -d --restart=always -e DOMAIN=cluster --name nginx-app -p 80:80 nginx +``` +``` +55c103fa129692154a7652490236fee9be47d70a8dd562281ae7d2f9a339a6db +``` + +```shell +docker ps +``` +``` +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +55c103fa1296 nginx "nginx -g 'daemon of…" 9 seconds ago Up 9 seconds 0.0.0.0:80->80/tcp nginx-app +``` + +kubectl: + +```shell +# nginx 실행하는 파드를 시작한다 +kubectl create deployment --image=nginx nginx-app +``` + +```shell +# nginx-app 에 env를 추가한다 +kubectl set env deployment/nginx-app DOMAIN=cluster +``` +``` +deployment.apps/nginx-app created +``` + +``` +# nginx-app 에 env를 추가한다 +kubectl set env deployment/nginx-app DOMAIN=cluster +``` +``` +deployment.apps/nginx-app env updated +``` + +{{< note >}} +`kubectl` 커맨드는 생성되거나 변경된 리소스의 유형과 이름을 출력하므로, 이를 후속 커맨드에 사용할 수 있다. 디플로이먼트가 생성된 후에는 새로운 서비스를 노출할 수 있다. +{{< /note >}} + +```shell +# 서비스를 통해 포트를 노출 +kubectl expose deployment nginx-app --port=80 --name=nginx-http +``` +``` +service "nginx-http" exposed +``` + +kubectl을 사용하면, N개의 파드가 nginx를 실행하도록 [디플로이먼트](/ko/docs/concepts/workloads/controllers/deployment/)를 생성할 수 있다. 여기서 N은 스펙에 명시된 레플리카 수이며, 기본값은 1이다. 또한 파드의 레이블과 셀럭터를 사용하여 서비스를 생성할 수 있다. 자세한 내용은 [클러스터 내 애플리케이션에 접근하기 위해 서비스 사용하기](/ko/docs/tasks/access-application-cluster/service-access-application-cluster)를 참고한다. + +기본적으로 이미지는 `docker run -d ...` 와 비슷하게 백그라운드로 실행된다. 포그라운드로 실행하려면 [`kubectl run`](/docs/reference/generated/kubectl/kubectl-commands/#run)을 이용하여 파드를 생성한다. +```shell +kubectl run [-i] [--tty] --attach --image= +``` + +`docker run ...` 과 달리 `--attach` 를 지정하면 `표준 입력(stdin)`, `표준 출력(stdout)` 및 `표준 오류(stderr)`가 붙는다. 연결된(attached) 스트림을 제어할 수 없다(`docker -a ...`). +해당 컨테이너에서 분리(detach)하려면 이스케이프 시퀀스(escape sequence) Ctrl+P를 입력한 다음 Ctrl+Q를 입력한다. + +## docker ps + +현재 실행 중인 목록을 보기 위해서는 [kubectl get](/docs/reference/generated/kubectl/kubectl-commands/#get)을 참고한다. + +docker: + +```shell +docker ps -a +``` +``` +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +14636241935f ubuntu:16.04 "echo test" 5 seconds ago Exited (0) 5 seconds ago cocky_fermi +55c103fa1296 nginx "nginx -g 'daemon of…" About a minute ago Up About a minute 0.0.0.0:80->80/tcp nginx-app +``` + +kubectl: + +```shell +kubectl get po +``` +``` +NAME READY STATUS RESTARTS AGE +nginx-app-8df569cb7-4gd89 1/1 Running 0 3m +ubuntu 0/1 Completed 0 20s +``` + +## docker attach + +이미 실행 중인 컨테이너에 연결하려면 [kubectl attach](/docs/reference/generated/kubectl/kubectl-commands/#attach)를 참고한다. + +docker: + +```shell +docker ps +``` +``` +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +55c103fa1296 nginx "nginx -g 'daemon of…" 5 minutes ago Up 5 minutes 0.0.0.0:80->80/tcp nginx-app +``` + +```shell +docker attach 55c103fa1296 +... +``` + +kubectl: + +```shell +kubectl get pods +``` +``` +NAME READY STATUS RESTARTS AGE +nginx-app-5jyvm 1/1 Running 0 10m +``` + +```shell +kubectl attach -it nginx-app-5jyvm +... +``` + +컨테이너에서 분리하려면 이스케이프 시퀀스 Ctrl+P를 입력한 다음 Ctrl+Q를 입력한다. + +## docker exec + +컨테이너에서 커맨드를 실행하려면 [kubectl exec](/docs/reference/generated/kubectl/kubectl-commands/#exec)를 참고한다. + +docker: + +```shell +docker ps +``` +``` +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +55c103fa1296 nginx "nginx -g 'daemon of…" 6 minutes ago Up 6 minutes 0.0.0.0:80->80/tcp nginx-app +``` +```shell +docker exec 55c103fa1296 cat /etc/hostname +``` +``` +55c103fa1296 +``` + +kubectl: + +```shell +kubectl get po +``` +``` +NAME READY STATUS RESTARTS AGE +nginx-app-5jyvm 1/1 Running 0 10m +``` + +```shell +kubectl exec nginx-app-5jyvm -- cat /etc/hostname +``` +``` +nginx-app-5jyvm +``` + +대화형 커맨드를 사용한다. + + +docker: + +```shell +docker exec -ti 55c103fa1296 /bin/sh +# exit +``` + +kubectl: + +```shell +kubectl exec -ti nginx-app-5jyvm -- /bin/sh +# exit +``` + +자세한 내용은 [실행 중인 컨테이너의 셸 얻기](/docs/tasks/debug-application-cluster/get-shell-running-container/)를 참고한다. + +## docker logs + +실행 중인 프로세스의 표준 입력(stdout)/표준 오류(stderr)를 수행하려면 [kubectl logs](/docs/reference/generated/kubectl/kubectl-commands/#logs)를 참고한다. + + +docker: + +```shell +docker logs -f a9e +``` +``` +192.168.9.1 - - [14/Jul/2015:01:04:02 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.35.0" "-" +192.168.9.1 - - [14/Jul/2015:01:04:03 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.35.0" "-" +``` + +kubectl: + +```shell +kubectl logs -f nginx-app-zibvs +``` +``` +10.240.63.110 - - [14/Jul/2015:01:09:01 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.26.0" "-" +10.240.63.110 - - [14/Jul/2015:01:09:02 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.26.0" "-" +``` + +파드와 컨테이너에는 근소한 차이가 있다. 기본적으로 파드는 프로세스가 종료되어도 종료되지 않는다. 대신 파드가 프로세스를 다시 시작한다. 이는 도커의 실행 옵션인 `--restart=always`와 유사하지만, 한 가지 큰 차이점이 있다. 도커에서는 프로세스의 각 호출에 대한 출력이 연결되지만, 쿠버네티스의 경우 각 호출은 별개다. 쿠버네티스에서 이전 실행의 출력 내용을 보려면 다음을 수행한다. + +```shell +kubectl logs --previous nginx-app-zibvs +``` +``` +10.240.63.110 - - [14/Jul/2015:01:09:01 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.26.0" "-" +10.240.63.110 - - [14/Jul/2015:01:09:02 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.26.0" "-" +``` + +자세한 정보는 [로깅 아키텍처](/ko/docs/concepts/cluster-administration/logging/)를 참고한다. + +## docker stop 과 docker rm + +실행 중인 프로세스를 중지하고 삭제하려면 [kubectl delete](/docs/reference/generated/kubectl/kubectl-commands/#delete)을 참고한다. + +docker: + +```shell +docker ps +``` +``` +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +a9ec34d98787 nginx "nginx -g 'daemon of" 22 hours ago Up 22 hours 0.0.0.0:80->80/tcp, 443/tcp nginx-app +``` + +```shell +docker stop a9ec34d98787 +``` +``` +a9ec34d98787 +``` + +```shell +docker rm a9ec34d98787 +``` +``` +a9ec34d98787 +``` + +kubectl: + +```shell +kubectl get deployment nginx-app +``` +``` +NAME READY UP-TO-DATE AVAILABLE AGE +nginx-app 1/1 1 1 2m +``` + +```shell +kubectl get po -l run=nginx-app +``` +``` +NAME READY STATUS RESTARTS AGE +nginx-app-2883164633-aklf7 1/1 Running 0 2m +``` +```shell +kubectl delete deployment nginx-app +``` +``` +deployment "nginx-app" deleted +``` + +```shell +kubectl get po -l run=nginx-app +# 아무것도 반환하지 않는다 +``` + +{{< note >}} +kubectl을 사용할 때는 파드를 직접 삭제하지 않는다. 먼저 파드를 소유한 디플로이먼트를 삭제해야 한다. 만약 파드를 직접 삭제하면 디플로이먼트가 파드를 재생성할 것이다. +{{< /note >}} + +## docker login + +kubectl은 `docker login`와 직접적인 유사점은 없다. 프라이빗 레지스트리와 함께 쿠버네티스를 사용하려면 [프라이빗 레지스트리 사용](/ko/docs/concepts/containers/images/#프라이빗-레지스트리-사용)을 참고한다. + +## docker version + +클라이언트와 서버의 버전을 가져오려면 [kubectl version](/docs/reference/generated/kubectl/kubectl-commands/#version)을 참고한다. + +docker: + +```shell +docker version +``` +``` +Client version: 1.7.0 +Client API version: 1.19 +Go version (client): go1.4.2 +Git commit (client): 0baf609 +OS/Arch (client): linux/amd64 +Server version: 1.7.0 +Server API version: 1.19 +Go version (server): go1.4.2 +Git commit (server): 0baf609 +OS/Arch (server): linux/amd64 +``` + +kubectl: + +```shell +kubectl version +``` +``` +Client Version: version.Info{Major:"1", Minor:"6", GitVersion:"v1.6.9+a3d1dfa6f4335", GitCommit:"9b77fed11a9843ce3780f70dd251e92901c43072", GitTreeState:"dirty", BuildDate:"2017-08-29T20:32:58Z", OpenPaasKubernetesVersion:"v1.03.02", GoVersion:"go1.7.5", Compiler:"gc", Platform:"linux/amd64"} +Server Version: version.Info{Major:"1", Minor:"6", GitVersion:"v1.6.9+a3d1dfa6f4335", GitCommit:"9b77fed11a9843ce3780f70dd251e92901c43072", GitTreeState:"dirty", BuildDate:"2017-08-29T20:32:58Z", OpenPaasKubernetesVersion:"v1.03.02", GoVersion:"go1.7.5", Compiler:"gc", Platform:"linux/amd64"} +``` + +## docker info + +환경 및 설정에 대한 자세한 정보는 [kubectl cluster-info](/docs/reference/generated/kubectl/kubectl-commands/#cluster-info)를 참고한다. + +docker: + +```shell +docker info +``` +``` +Containers: 40 +Images: 168 +Storage Driver: aufs + Root Dir: /usr/local/google/docker/aufs + Backing Filesystem: extfs + Dirs: 248 + Dirperm1 Supported: false +Execution Driver: native-0.2 +Logging Driver: json-file +Kernel Version: 3.13.0-53-generic +Operating System: Ubuntu 14.04.2 LTS +CPUs: 12 +Total Memory: 31.32 GiB +Name: k8s-is-fun.mtv.corp.google.com +ID: ADUV:GCYR:B3VJ:HMPO:LNPQ:KD5S:YKFQ:76VN:IANZ:7TFV:ZBF4:BYJO +WARNING: No swap limit support +``` + +kubectl: + +```shell +kubectl cluster-info +``` +``` +Kubernetes master is running at https://203.0.113.141 +KubeDNS is running at https://203.0.113.141/api/v1/namespaces/kube-system/services/kube-dns/proxy +kubernetes-dashboard is running at https://203.0.113.141/api/v1/namespaces/kube-system/services/kubernetes-dashboard/proxy +Grafana is running at https://203.0.113.141/api/v1/namespaces/kube-system/services/monitoring-grafana/proxy +Heapster is running at https://203.0.113.141/api/v1/namespaces/kube-system/services/monitoring-heapster/proxy +InfluxDB is running at https://203.0.113.141/api/v1/namespaces/kube-system/services/monitoring-influxdb/proxy +``` + diff --git a/content/ko/docs/reference/kubectl/overview.md b/content/ko/docs/reference/kubectl/overview.md index 04b3e6c1d3..6626947c2a 100644 --- a/content/ko/docs/reference/kubectl/overview.md +++ b/content/ko/docs/reference/kubectl/overview.md @@ -125,14 +125,15 @@ kubectl [command] [TYPE] [NAME] [flags] 다음 표에는 지원되는 모든 리소스 타입과 해당 약어가 나열되어 있다. -(이 출력은 `kubectl api-resources` 에서 확인할 수 있으며, 쿠버네티스 1.13.3 부터 일치했다.) +(이 출력은 `kubectl api-resources` 에서 확인할 수 있으며, 쿠버네티스 1.19.1 에서의 출력을 기준으로 한다.) -| 리소스 이름 | 짧은 이름 | API 그룹 | 네임스페이스 | 리소스 종류 | +| NAME | SHORTNAMES | APIGROUP | NAMESPACED | KIND | |---|---|---|---|---| -| `bindings` | | | true | Binding| +| `bindings` | | | true | Binding | | `componentstatuses` | `cs` | | false | ComponentStatus | | `configmaps` | `cm` | | true | ConfigMap | | `endpoints` | `ep` | | true | Endpoints | +| `events` | `ev` | | true | Event | | `limitranges` | `limits` | | true | LimitRange | | `namespaces` | `ns` | | false | Namespace | | `nodes` | `no` | | false | Node | @@ -140,14 +141,14 @@ kubectl [command] [TYPE] [NAME] [flags] | `persistentvolumes` | `pv` | | false | PersistentVolume | | `pods` | `po` | | true | Pod | | `podtemplates` | | | true | PodTemplate | -| `replicationcontrollers` | `rc` | | true| ReplicationController | +| `replicationcontrollers` | `rc` | | true | ReplicationController | | `resourcequotas` | `quota` | | true | ResourceQuota | | `secrets` | | | true | Secret | | `serviceaccounts` | `sa` | | true | ServiceAccount | | `services` | `svc` | | true | Service | | `mutatingwebhookconfigurations` | | admissionregistration.k8s.io | false | MutatingWebhookConfiguration | | `validatingwebhookconfigurations` | | admissionregistration.k8s.io | false | ValidatingWebhookConfiguration | -| `customresourcedefinitions` | `crd`, `crds` | apiextensions.k8s.io | false | CustomResourceDefinition | +| `customresourcedefinitions` | `crd,crds` | apiextensions.k8s.io | false | CustomResourceDefinition | | `apiservices` | | apiregistration.k8s.io | false | APIService | | `controllerrevisions` | | apps | true | ControllerRevision | | `daemonsets` | `ds` | apps | true | DaemonSet | @@ -164,9 +165,15 @@ kubectl [command] [TYPE] [NAME] [flags] | `jobs` | | batch | true | Job | | `certificatesigningrequests` | `csr` | certificates.k8s.io | false | CertificateSigningRequest | | `leases` | | coordination.k8s.io | true | Lease | +| `endpointslices` | | discovery.k8s.io | true | EndpointSlice | | `events` | `ev` | events.k8s.io | true | Event | | `ingresses` | `ing` | extensions | true | Ingress | +| `flowschemas` | | flowcontrol.apiserver.k8s.io | false | FlowSchema | +| `prioritylevelconfigurations` | | flowcontrol.apiserver.k8s.io | false | PriorityLevelConfiguration | +| `ingressclasses` | | networking.k8s.io | false | IngressClass | +| `ingresses` | `ing` | networking.k8s.io | true | Ingress | | `networkpolicies` | `netpol` | networking.k8s.io | true | NetworkPolicy | +| `runtimeclasses` | | node.k8s.io | false | RuntimeClass | | `poddisruptionbudgets` | `pdb` | policy | true | PodDisruptionBudget | | `podsecuritypolicies` | `psp` | policy | false | PodSecurityPolicy | | `clusterrolebindings` | | rbac.authorization.k8s.io | false | ClusterRoleBinding | @@ -176,7 +183,7 @@ kubectl [command] [TYPE] [NAME] [flags] | `priorityclasses` | `pc` | scheduling.k8s.io | false | PriorityClass | | `csidrivers` | | storage.k8s.io | false | CSIDriver | | `csinodes` | | storage.k8s.io | false | CSINode | -| `storageclasses` | `sc` | storage.k8s.io | false | StorageClass | +| `storageclasses` | `sc` | storage.k8s.io | false | StorageClass | | `volumeattachments` | | storage.k8s.io | false | VolumeAttachment | ## 출력 옵션 diff --git a/content/ko/docs/reference/scheduling/policies.md b/content/ko/docs/reference/scheduling/policies.md index 71ccef5e87..f2cae65b68 100644 --- a/content/ko/docs/reference/scheduling/policies.md +++ b/content/ko/docs/reference/scheduling/policies.md @@ -26,7 +26,7 @@ weight: 10 - `PodFitsResources`: 파드의 요구 사항을 충족할 만큼 노드에 사용할 수 있는 리소스(예: CPU 및 메모리)가 있는지 확인한다. -- `PodMatchNodeSelector`: 파드의 노드 {{< glossary_tooltip text="셀렉터" term_id="selector" >}}가 +- `MatchNodeSelector`: 파드의 노드 {{< glossary_tooltip text="셀렉터" term_id="selector" >}}가 노드의 {{< glossary_tooltip text="레이블" term_id="label" >}}과 일치하는지 확인한다. - `NoVolumeZoneConflict`: 해당 스토리지에 대한 장애 영역 제한이 주어지면 @@ -115,4 +115,4 @@ weight: 10 ## {{% heading "whatsnext" %}} * [스케줄링](/ko/docs/concepts/scheduling-eviction/kube-scheduler/)에 대해 배우기 -* [kube-scheduler 프로파일](/docs/reference/scheduling/profiles/)에 대해 배우기 \ No newline at end of file +* [kube-scheduler 프로파일](/docs/reference/scheduling/profiles/)에 대해 배우기 diff --git a/content/ko/docs/reference/using-api/api-overview.md b/content/ko/docs/reference/using-api/api-overview.md index 0ac842ccc6..4bf982fc1b 100644 --- a/content/ko/docs/reference/using-api/api-overview.md +++ b/content/ko/docs/reference/using-api/api-overview.md @@ -9,62 +9,61 @@ card: --- + 이 페이지는 쿠버네티스 API에 대한 개요를 제공한다. -REST API는 쿠버네티스의 근본적인 구조이다. 모든 조작, 컴포넌트 간의 통신과 외부 사용자의 명령은 API 서버에서 처리할 수 있는 REST API 호출이다. 따라서, 쿠버네티스 플랫폼 안의 모든 것은 + +REST API는 쿠버네티스의 근본적인 구조이다. 모든 조작, +컴포넌트 간의 통신과 외부 사용자의 명령은 API 서버에서 처리할 수 있는 +REST API 호출이다. 따라서, 쿠버네티스 플랫폼 안의 모든 것은 API 오브젝트로 취급되고, [API](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/)에 상응하는 항목이 있다. -대부분의 작업은 API에 의존하고 있는 -[kubectl](/ko/docs/reference/kubectl/overview/) 커맨드라인 인터페이스 또는 -[kubeadm](/docs/reference/setup-tools/kubeadm/kubeadm/)과 같은 다른 커맨드라인 툴을 통해 수행할 수 있다. -그러나, REST 호출 사용을 통해서 API에 직접 접근할 수도 있다. - -쿠버네티스 API를 사용하는 애플리케이션을 작성하는 경우 -[클라이언트 라이브러리](/ko/docs/reference/using-api/client-libraries/)중 하나의 사용을 고려한다. - ## API 버전 규칙 -필드를 없애거나 리소스 표현을 재구성하기 쉽도록, -쿠버네티스는 `/api/v1`이나 `/apis/rbac.authorization.k8s.io/v1alpha1`과 같이 -각각 다른 API 경로에서 복수의 API 버전을 지원한다. +JSON과 Protobuf 직렬화 스키마 모두 스키마 변경에 대해서 +동일한 가이드라인을 따른다. 이후 설명에서는 이 형식 모두를 다룬다. -아래를 위해 버전은 리소스나 필드 수준보다는 API 수준에서 설정된다. - -- API가 시스템 리소스와 동작에 대해 명확하고 일관성 있게 표현하는 것을 보장 -- 수명 종료(end-of-life) 또는 실험적인 API 접근 제어 활성화 - -JSON과 Protobuf 직렬화 스키마 모두 스키마 변경에 대해서 동일한 가이드라인을 따른다. 이후 설명에서는 이 형식 모두를 다룬다. - -{{< note >}} API 버전 규칙과 소프트웨어 버전 규칙은 간접적으로 연관된다. -[API와 릴리스 버전 부여에 관한 제안](https://git.k8s.io/community/contributors/design-proposals/release/versioning.md)에는 API 버전 규칙과 소프트웨어 버전 규칙 간의 관계가 기술되어 있다. -{{< /note >}} +[API와 릴리스 버전 부여에 관한 제안](https://git.k8s.io/community/contributors/design-proposals/release/versioning.md)에는 +API 버전 규칙과 소프트웨어 버전 규칙 간의 관계가 기술되어 있다. -API 버전의 차이는 수준의 안정성과 지원의 차이를 나타낸다. [API 변경 문서](https://git.k8s.io/community/contributors/devel/sig-architecture/api_changes.md#alpha-beta-and-stable-versions)에서 각 수준의 기준에 대한 더 많은 정보를 찾을 수 있다. +API 버전의 차이는 수준의 안정성과 지원의 차이를 나타낸다. +[API 변경 문서](https://git.k8s.io/community/contributors/devel/sig-architecture/api_changes.md#alpha-beta-and-stable-versions)에서 +각 수준의 기준에 대한 더 많은 정보를 찾을 수 있다. 아래는 각 수준의 기준에 대한 요약이다. - 알파(Alpha) 수준: - 버전 이름에 `alpha`가 포함된다. (예: `v1alpha1`) - - 버그가 있을 수도 있다. 이 기능을 활성화하면 버그가 노출될 수 있다. 기본적으로 비활성화되어 있다. + - 버그가 있을 수도 있다. 이 기능을 활성화하면 버그가 노출될 수 있다. + 기본적으로 비활성화되어 있다. - 기능에 대한 기술 지원이 언제든 공지 없이 중단될 수 있다. - 다음 소프트웨어를 릴리스할 때 공지 없이 API의 호환성이 깨지는 방식으로 변경될 수 있다. - - 버그의 위험이 높고 장기간 지원되지 않으므로 단기간 테스트 용도의 클러스터에서만 사용하기를 권장한다. + - 버그의 위험이 높고 장기간 지원되지 않으므로 + 단기간 테스트 용도의 클러스터에서만 사용하기를 권장한다. - 베타(Beta) 수준: - 버전 이름에 `beta`가 포함된다. (예: `v2beta3`). - - 코드가 잘 테스트되었다. 이 기능을 활성화 시켜도 안전하다. 기본적으로 활성화되어 있다. + - 코드가 잘 테스트되었다. 이 기능을 활성화 시켜도 안전하다. + 기본적으로 활성화되어 있다. - 구체적인 내용이 바뀔 수는 있지만, 전반적인 기능에 대한 기술 지원이 중단되지 않는다. - - 오브젝트에 대한 스키마나 문법이 다음 베타 또는 안정화 릴리스에서 호환되지 않는 방식으로 바뀔 수도 있다. 이런 경우, 다음 버전으로 이관할 수 있는 가이드가 제공된다. 이때 API 오브젝트의 삭제, 편집 또는 재생성이 - 필요할 수도 있다. 편집 절차는 좀 생각해볼 필요가 있다. 이 기능에 의존하고 있는 애플리케이션은 다운타임이 필요할 수도 있다. - - 이후 여러 버전에서 잠재적으로 호환되지 않을 수도 있으므로 사업적으로 중요하지 않은 용도로만 사용하기를 권장한다. 복수의 클러스터를 가지고 있어서 독립적으로 업그레이드할 수 있다면, 이런 제약에서 안심이 될 수도 있겠다. - {{< note >}} -베타 기능을 사용해보고 피드백을 제공하자. 일단 베타가 끝나면, 실질적으로 더 많은 변경이 어렵다. - {{< /note >}} + - 오브젝트에 대한 스키마나 문법이 다음 베타 또는 안정화 릴리스에서 + 호환되지 않는 방식으로 바뀔 수도 있다. 이런 경우, 다음 버전으로 + 이관할 수 있는 가이드가 제공된다. 스키마 변경은 API 오브젝트의 삭제, 편집 또는 재생성이 + 필요할 수도 있다. 편집 절차는 좀 생각해볼 필요가 있다. + 이 기능에 의존하고 있는 애플리케이션은 다운타임이 필요할 수도 있다. + - 이 소프트웨어는 프로덕션 용도로 권장되지 않는다. 이후 여러 버전에서 + 호환되지 않는 변경 사항이 적용될 수 있다. 복수의 클러스터를 가지고 있어서 + 독립적으로 업그레이드할 수 있다면, 이런 제약에서 벗어날 수도 있다. + + {{< note >}} + 베타 기능을 사용해보고 피드백을 제공하자. 기능이 베타 수준을 벗어난 이후에는 + 실질적으로 더 많은 변경이 어렵다. + {{< /note >}} - 안정화(stable) 수준: - 버전 이름이 `vX`이고 `X` 는 정수다. @@ -72,32 +71,43 @@ API 버전의 차이는 수준의 안정성과 지원의 차이를 나타낸다. ## API 그룹 -[*API 그룹*](https://git.k8s.io/community/contributors/design-proposals/api-machinery/api-group.md)은 쿠버네티스 API를 더 쉽게 확장하게 해준다. API 그룹은 REST 경로와 직렬화된 객체의 `apiVersion` 필드에 명시된다. +[API 그룹](https://git.k8s.io/community/contributors/design-proposals/api-machinery/api-group.md)은 +쿠버네티스 API를 더 쉽게 확장하게 해준다. +API 그룹은 REST 경로와 직렬화된 객체의 `apiVersion` 필드에 +명시된다. -현재 다음과 같은 다양한 API 그룹이 사용되고 있다: +현재 다음과 같은 다양한 API 그룹이 사용되고 있다. -* *핵심* (또는 *레거시* 라고 불리는) 그룹은 `apiVersion: v1`와 같이 `apiVersion` 필드에 명시되지 않고 REST 경로 `/api/v1`에 있다. -* 이름이 있는 그룹은 REST 경로 `/apis/$GROUP_NAME/$VERSION`에 있으며 `apiVersion: $GROUP_NAME/$VERSION`을 사용한다 - (예를 들어 `apiVersion: batch/v1`). 지원되는 API 그룹 전체의 목록은 [쿠버네티스 API 참조 문서](/ko/docs/reference/)에서 확인할 수 있다. +* *핵심* (또는 *레거시* 라고 불리는) 그룹은 REST 경로 `/api/v1`에 있다. + 핵심 그룹은 `apiVersion` 필드의 일부로 명시되지 않는다. 예를 + 들어, `apiVersion: v1` 와 같다. +* 이름이 있는 그룹은 REST 경로 `/apis/$GROUP_NAME/$VERSION`에 있으며 + `apiVersion: $GROUP_NAME/$VERSION`을 사용한다(예를 들어, `apiVersion: batch/v1`). + 지원되는 API 그룹 전체의 목록은 + [쿠버네티스 API 참조 문서](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/)에서 확인할 수 있다. -[사용자 정의 리소스](/ko/docs/concepts/extend-kubernetes/api-extension/custom-resources/)로 API를 확장하는 경우에는 다음 두 종류의 경로가 지원된다. +## API 그룹 활성화 또는 비활성화 {#enabling-or-disabling} - - 기본적인 CRUD 요구에는 - [커스텀리소스데피니션(CustomResourceDefinition)](/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/) - - 쿠버네티스 API의 의미론적 전체 집합으로 사용자만의 Apiserver를 구현하려는 경우에는 [aggregator](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/api-machinery/aggregated-api-servers.md) +특정 리소스 및 API 그룹은 기본적으로 활성화된다. API 서버에서 +`--runtime-config` 를 설정하여 활성화 또는 비활성화할 수 있다. +`--runtime-config` 플래그는 API 서버의 런타임 구성을 설명하는 +쉼표로 구분된 `=` 쌍을 허용한다. 예를 들면, 다음과 같다. - -## API 그룹 활성화 또는 비활성화 하기 - -특정 리소스와 API 그룹은 기본적으로 활성화되어 있다. 이들은 apiserver에서 `--runtime-config`를 설정해서 활성화하거나 -비활성화 시킬 수 있다. `--runtime-config`는 쉼표로 분리된 값을 허용한다. 예를 들어: - - - batch/v1을 비활성화하려면 `--runtime-config=batch/v1=false`로 설정 - - batch/v2alpha1을 활성화하려면 `--runtime-config=batch/v2alpha1`로 설정 - -이 플래그는 apiserver의 런타임 구성을 설명하는 쉼표로 분리된 키=값 쌍의 집합을 허용한다. + - `batch/v1` 을 비활성화하려면, `--runtime-config=batch/v1=false` 로 설정 + - `batch/v2alpha1` 을 활성화하려면, `--runtime-config=batch/v2alpha1` 으로 설정 {{< note >}} 그룹이나 리소스를 활성화 또는 비활성화하려면, apiserver와 controller-manager를 재시작하여 `--runtime-config` 변경을 반영해야 한다. {{< /note >}} + +## 지속성 + +쿠버네티스는 {{< glossary_tooltip term_id="etcd" >}}에 기록하여 API 리소스 측면에서 +직렬화된 상태를 저장한다. + +## {{% heading "whatsnext" %}} + +- [API 규칙](https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#api-conventions)에 대해 자세히 알아보기 +- [애그리게이터(aggregator)](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/api-machinery/aggregated-api-servers.md)에 + 대한 디자인 문서 읽기 diff --git a/content/ko/docs/setup/best-practices/node-conformance.md b/content/ko/docs/setup/best-practices/node-conformance.md index d4579e2f89..9e9a65bee9 100644 --- a/content/ko/docs/setup/best-practices/node-conformance.md +++ b/content/ko/docs/setup/best-practices/node-conformance.md @@ -10,12 +10,6 @@ weight: 30 테스트는 노드가 쿠버네티스를 위한 최소 요구조건을 만족하는지를 검증한다. 그리고 테스트를 통과한 노드는 쿠버네티스 클러스터에 참 여할 자격이 주어진다. -## 제한 사항 - -쿠버네티스 1.5에서는 노드 적합성 테스트가 아래의 제약이 있다. - -* 노드 적합성 테스트는 컨테이너 런타임으로 Docker만 지원한다. - ## 노드 필수 구성 요소 노드 적합성 테스트를 실행하기 위해서는, 해당 노드는 표준 쿠버네티스 노드로서 동일한 전제조건을 만족해야 한다. diff --git a/content/ko/docs/setup/production-environment/tools/kubespray.md b/content/ko/docs/setup/production-environment/tools/kubespray.md new file mode 100644 index 0000000000..c4373a15d3 --- /dev/null +++ b/content/ko/docs/setup/production-environment/tools/kubespray.md @@ -0,0 +1,119 @@ +--- +title: Kubespray로 쿠버네티스 설치하기 +content_type: concept +weight: 30 +--- + + + +이 가이드는 [Kubespray](https://github.com/kubernetes-sigs/kubespray)를 이용하여 GCE, Azure, OpenStack, AWS, vSphere, Packet(베어메탈), Oracle Cloud infrastructure(실험적) 또는 베어메탈 등에서 운영되는 쿠버네티스 클러스터를 설치하는 과정을 보여준다. + +Kubespray는 [Ansible](https://docs.ansible.com/) 플레이북, [인벤토리](https://github.com/kubernetes-sigs/kubespray/blob/master/docs/ansible.md), 프로비저닝 도구와 일반적인 운영체제, 쿠버네티스 클러스터의 설정 관리 작업에 대한 도메인 지식의 결합으로 만들어졌다. Kubespray는 아래와 같은 기능을 제공한다. + +* 고가용성을 지닌 클러스터 +* 구성할 수 있는 속성들 +* 대부분의 인기있는 리눅스 배포판들에 대한 지원 + * Ubuntu 16.04, 18.04, 20.04 + * CentOS/RHEL/Oracle Linux 7, 8 + * Debian Buster, Jessie, Stretch, Wheezy + * Fedora 31, 32 + * Fedora CoreOS + * openSUSE Leap 15 + * Flatcar Container Linux by Kinvolk +* 지속적인 통합 (CI) 테스트 + +클러스터를 설치해 줄 도구로 유스케이스와 가장 잘 맞는 것을 고르고 싶다면, kubespray를 [kubeadm](/docs/reference/setup-tools/kubeadm/kubeadm/), [kops](/docs/setup/production-environment/tools/kops/)와 [비교한 글](https://github.com/kubernetes-sigs/kubespray/blob/master/docs/comparisons.md)을 읽어보자. + + + +## 클러스터 생성하기 + + +### (1/5) 아래의 요건 충족하기 + +언더레이(underlay) [요건](https://github.com/kubernetes-sigs/kubespray#requirements)을 만족하는 프로비전 한다. + +* **Ansible의 명령어를 실행하기 위해 Ansible v 2.9와 Python netaddr 라이브러리가 머신에 설치되어 있어야 한다** +* **Ansible 플레이북을 실행하기 위해 2.11 (혹은 그 이상) 버전의 Jinja가 필요하다** +* 타겟 서버들은 docker 이미지를 풀(pull) 하기 위해 반드시 인터넷에 접속할 수 있어야 한다. 아니라면, 추가적인 설정을 해야 한다 ([오프라인 환경 확인하기](https://github.com/kubernetes-sigs/kubespray/blob/master/docs/offline-environment.md)) +* 타겟 서버들의 **IPv4 포워딩**이 활성화되어야 한다 +* **SSH 키**가 인벤토리의 모든 서버들에 복사되어야 한다 +* **방화벽은 관리되지 않는다**. 사용자가 예전 방식대로 고유한 규칙을 구현해야 한다. 디플로이먼트 과정에서의 문제를 방지하려면 방화벽을 비활성화해야 한다 +* 만약 kubespray가 루트가 아닌 사용자 계정에서 실행되었다면, 타겟 서버에서 알맞은 권한 확대 방법이 설정되어야 한다. 그 뒤 `ansible_become` 플래그나 커맨드 파라미터들, `--become` 또는 `-b` 가 명시되어야 한다 + +Kubespray는 환경에 맞는 프로비저닝을 돕기 위해 아래와 같은 서비스를 제공한다: + +* 아래 클라우드 제공 업체를 위한 [Terraform](https://www.terraform.io/) 스크립트: + * [AWS](https://github.com/kubernetes-sigs/kubespray/tree/master/contrib/terraform/aws) + * [OpenStack](https://github.com/kubernetes-sigs/kubespray/tree/master/contrib/terraform/openstack) + * [Packet](https://github.com/kubernetes-sigs/kubespray/tree/master/contrib/terraform/packet) + +### (2/5) 인벤토리 파일 구성하기 + +서버들을 프로비저닝 한 후, [Ansible의 인벤토리 파일](https://docs.ansible.com/ansible/intro_inventory.html)을 만들어야 한다. 수동으로 만들 수도 있고, 동적인 인벤토리 스크립트를 통해 만들 수도 있다. 더 많이 알고싶다면 " [나만의 인벤토리 만들기](https://github.com/kubernetes-sigs/kubespray/blob/master/docs/getting-started.md#building-your-own-inventory)" 글을 확인하자. + +### (3/5) 클러스터 디플로이먼트 계획하기 + +Kubespray에서는 디플로이먼트의 많은 속성들을 사용자가 정의(customize)할 수 있다: + +* 디플로이먼트 모드의 선택: kubeadm 또는 그 외 +* CNI(네트워킹) 플러그인 +* DNS 설정 +* 컨트롤 플레인 선택: 네이티브/바이너리 또는 컨테이너화 된 것 +* 컴포넌트 버전 +* Calico 라우터 리플렉터 +* 컴포넌트 런타임 옵션 + * {{< glossary_tooltip term_id="docker" >}} + * {{< glossary_tooltip term_id="containerd" >}} + * {{< glossary_tooltip term_id="cri-o" >}} +* 인증서 생성 방법 + +Kubespray의 [변수 파일들](https://docs.ansible.com/ansible/playbooks_variables.html)을 사용자가 정의할 수 있다. 만약 Kubespray를 막 시작한 경우, kubespray의 기본 설정값을 이용해 클러스터를 배포하고 Kubernetes를 탐색하는 것이 좋다. + +### (4/5) 클러스터 배포하기 + +다음으로, 클러스터를 배포한다. + +[Ansible-플레이북](https://github.com/kubernetes-sigs/kubespray/blob/master/docs/getting-started.md#starting-custom-deployment)을 이용한 클러스터 디플로이먼트 + +```shell +ansible-playbook -i your/inventory/inventory.ini cluster.yml -b -v \ + --private-key=~/.ssh/private_key +``` + +규모가 큰 디플로이먼트는 (100개 이상의 노드) 최적의 결과를 얻기 위해 [특정한 조정](https://github.com/kubernetes-sigs/kubespray/blob/master/docs/large-deployments.md)을 필요로 할 수도 있다. + +### (5/5) 디플로이먼트 검증하기 + +Kubespray는 Netchecker를 사용하여 파드 사이의 연결성과 DNS 해석을 검증할 방법을 제공한다. Netchecker는 netchecker-agents 파드들이 DNS 요청을 해석하고 기본(default) 네임스페이스 내부에서 서로에게 ping을 보낼 수 있도록 보장한다. 그 파드들은 나머지 워크로드의 유사한 동작을 모방하고 클러스터의 상태 표시기 역할을 한다. + +## 클러스터 동작 + +Kubespray는 클러스터를 관리하기 위한 추가적인 플레이북, _scale_ 과 _upgrade_ 를 제공한다. + +### 클러스터 스케일링하기 + +scale 플레이북을 실행해 클러스터에 워커 노드를 추가할 수 있다. 더 자세히 알고 싶다면, "[노드 추가하기](https://github.com/kubernetes-sigs/kubespray/blob/master/docs/getting-started.md#adding-nodes)" 문서를 확인하자. remove-node 플레이북을 실행하면 클러스터로부터 워커 노드를 제거할 수 있다. 더 알고 싶다면 "[노드 제거하기](https://github.com/kubernetes-sigs/kubespray/blob/master/docs/getting-started.md#remove-nodes)" 문서를 확인하자. + +### 클러스터 업그레이드 하기 + +upgrade-cluster 플레이북을 실행해 클러스터를 업그레이드 할 수 있다. 더 자세히 알고 싶다면 "[업그레이드](https://github.com/kubernetes-sigs/kubespray/blob/master/docs/upgrades.md)" 문서를 확인하자. + +## 클린업 + +[reset 플레이북](https://github.com/kubernetes-sigs/kubespray/blob/master/reset.yml)을 이용하여 노드들을 리셋하고 Kubespray로 설치된 모든 구성요소를 삭제할 수 있다. + +{{< caution >}} +reset 플레이북을 실행할 때, 실수로 프로덕션 클러스터를 타겟으로 삼지 않도록 해야 한다! +{{< /caution >}} + +## 피드백 + +* Slack 채널: [#kubespray](https://kubernetes.slack.com/messages/kubespray/) ([이 곳](https://slack.k8s.io/)에서 초대를 받을 수 있다) +* [GitHub Issues](https://github.com/kubernetes-sigs/kubespray/issues) + +## {{% heading "whatsnext" %}} + + +Kubespray의 [로드맵](https://github.com/kubernetes-sigs/kubespray/blob/master/docs/roadmap.md)에서 계획중인 작업을 확인해보자. + diff --git a/content/ko/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md b/content/ko/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md index a2e0a1852c..1aea3d6ccf 100644 --- a/content/ko/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md +++ b/content/ko/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md @@ -14,7 +14,7 @@ weight: 65 ## 쿠버네티스의 윈도우 컨테이너 -쿠버네티스에서 윈도우 컨테이너 오케스트레이션을 활성화하려면, 기존 리눅스 클러스터에 윈도우 노드를 포함하기만 하면 된다. 쿠버네티스의 [파드](/ko/docs/concepts/workloads/pods/pod-overview/)에서 윈도우 컨테이너를 스케줄링하는 것은 리눅스 기반 컨테이너를 스케줄링하는 것만큼 간단하고 쉽다. +쿠버네티스에서 윈도우 컨테이너 오케스트레이션을 활성화하려면, 기존 리눅스 클러스터에 윈도우 노드를 포함하기만 하면 된다. 쿠버네티스의 {{< glossary_tooltip text="파드" term_id="pod" >}}에서 윈도우 컨테이너를 스케줄링하는 것은 리눅스 기반 컨테이너를 스케줄링하는 것만큼 간단하고 쉽다. 윈도우 컨테이너를 실행하려면, 쿠버네티스 클러스터에 리눅스를 실행하는 컨트롤 플레인 노드와 사용자의 워크로드 요구에 따라 윈도우 또는 리눅스를 실행하는 워커가 있는 여러 운영 체제가 포함되어 있어야 한다. 윈도우 서버 2019는 윈도우에서 [쿠버네티스 노드](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/architecture/architecture.md#the-kubernetes-node)를 활성화하는 유일한 윈도우 운영 체제이다(kubelet, [컨테이너 런타임](https://docs.microsoft.com/ko-kr/virtualization/windowscontainers/deploy-containers/containerd) 및 kube-proxy 포함). 윈도우 배포 채널에 대한 자세한 설명은 [Microsoft 문서](https://docs.microsoft.com/ko-kr/windows-server/get-started-19/servicing-channels-19)를 참고한다. @@ -62,7 +62,7 @@ API 및 kubectl의 관점에서, 윈도우 컨테이너는 리눅스 기반 컨 윈도우에서 주요 쿠버네티스 요소는 리눅스와 동일한 방식으로 작동한다. 이 섹션에서는, 주요 워크로드 인에이블러(enabler) 일부와 이들이 윈도우에 매핑되는 방법에 대해 설명한다. -* [파드](/ko/docs/concepts/workloads/pods/pod-overview/) +* [파드](/ko/docs/concepts/workloads/pods/) 파드는 쿠버네티스의 기본 빌딩 블록이다 - 쿠버네티스 오브젝트 모델에서 생성하고 배포하는 가장 작고 간단한 단위. 동일한 파드에 윈도우 및 리눅스 컨테이너를 배포할 수 없다. 파드의 모든 컨테이너는 단일 노드로 스케줄되며 각 노드는 특정 플랫폼 및 아키텍처를 나타낸다. 다음과 같은 파드 기능, 속성 및 이벤트가 윈도우 컨테이너에서 지원된다. @@ -81,7 +81,7 @@ API 및 kubectl의 관점에서, 윈도우 컨테이너는 리눅스 기반 컨 * 레플리카셋(ReplicaSet) * 레플리케이션컨트롤러(ReplicationController) * 디플로이먼트(Deployment) - * 스테이트풀셋(StatefulSet) + * 스테이트풀셋(StatefulSet) * 데몬셋(DaemonSet) * 잡(Job) * 크론잡(CronJob) @@ -198,7 +198,7 @@ CSI 노드 플러그인(특히 블록 디바이스 또는 공유 파일시스템 윈도우에서는 다음 IPAM 옵션이 지원된다. * [호스트-로컬](https://github.com/containernetworking/plugins/tree/master/plugins/ipam/host-local) -* HNS IPAM(Inbox 플랫폼 IPAM, 이것은 IPAM이 설정되지 않은 경우 폴백(fallback)이다) +* HNS IPAM(Inbox 플랫폼 IPAM, 이것은 IPAM이 설정되지 않은 경우 폴백(fallback)이다) * [Azure-vnet-ipam](https://github.com/Azure/azure-container-networking/blob/master/docs/ipam.md)(azure-cni 전용) ##### 로드 밸런싱과 서비스 @@ -255,9 +255,9 @@ CSI 노드 플러그인(특히 블록 디바이스 또는 공유 파일시스템 윈도우에는 리눅스처럼 out-of-memory 프로세스 킬러가 없다. 윈도우는 항상 모든 사용자 모드 메모리 할당을 가상으로 처리하며 페이지 파일은 필수이다. 결과적으로 윈도우는 리눅스와 같은 방식으로 메모리 부족 상태에 도달하지 않고, 메모리 부족(OOM)으로 인한 종료 대신 페이지를 디스크로 처리한다. 메모리가 과도하게 프로비저닝되고 모든 실제 메모리가 고갈되면, 페이징으로 인해 성능이 저하될 수 있다. -2단계 프로세스를 통해 적절한 범위 내에서 메모리 사용량을 유지할 수 있다. 먼저, kubelet 파라미터 `--kubelet-reserve` 그리고/또는 `--system-reserve`를 사용하여 노드(컨테이너 외부)의 메모리 사용량을 고려한다. 이렇게 하면 [노드 할당(NodeAllocatable)](/docs/tasks/administer-cluster/reserve-compute-resources/#node-allocatable)이 줄어든다. 워크로드를 배포할 때 컨테이너에 리소스 제한을 사용(limits만 설정하거나 limits이 requests과 같아야 함)한다. 또한 NodeAllocatable에서 빼고 노드가 가득차면 스케줄러가 더 많은 파드를 추가하지 못하도록 한다. +2단계 프로세스를 통해 적절한 범위 내에서 메모리 사용량을 유지할 수 있다. 먼저, kubelet 파라미터 `--kubelet-reserve` 그리고/또는 `--system-reserve`를 사용하여 노드(컨테이너 외부)의 메모리 사용량을 고려한다. 이렇게 하면 [노드 할당(NodeAllocatable)](/docs/tasks/administer-cluster/reserve-compute-resources/#node-allocatable)이 줄어든다. 워크로드를 배포할 때 컨테이너에 리소스 제한을 사용(limits만 설정하거나 limits이 requests과 같아야 함)한다. 또한 NodeAllocatable에서 빼고 노드가 가득차면 스케줄러가 더 많은 파드를 추가하지 못하도록 한다. -오버 프로비저닝을 방지하는 모범 사례는 윈도우, 도커 및 쿠버네티스 프로세스를 고려하여 최소 2GB의 시스템 예약 메모리로 kubelet을 구성하는 것이다. +오버 프로비저닝을 방지하는 모범 사례는 윈도우, 도커 및 쿠버네티스 프로세스를 고려하여 최소 2GB의 시스템 예약 메모리로 kubelet을 구성하는 것이다. 플래그의 동작은 아래에 설명된대로 다르게 동작한다. @@ -581,7 +581,7 @@ PodSecurityContext 필드는 윈도우에서 작동하지 않는다. 참조를 1. DNS 확인(resolution)이 제대로 작동하지 않는다. 이 [섹션](#dns-limitations)에서 윈도우에 대한 DNS 제한을 확인한다. - + 1. `kubectl port-forward`가 "unable to do port forwarding: wincat not found"로 실패한다. 이는 쿠버네티스 1.15 및 pause 인프라 컨테이너 `mcr.microsoft.com/k8s/core/pause:1.2.0`에서 구현되었다. 해당 버전 또는 최신 버전을 사용해야 한다. @@ -666,15 +666,13 @@ spec: ### kubeadm 및 클러스터 API를 사용한 배포 -Kubeadm은 사용자가 쿠버네티스 클러스터를 배포하기 위한 사실상의 표준이 -되고 있다. kubeadm의 윈도우 노드 지원은 현재 작업 중이지만 -[여기](/docs/tasks/administer-cluster/kubeadm/adding-windows-nodes/)에서 가이드를 사용할 수 있다. -또한 윈도우 노드가 적절하게 프로비저닝되도록 클러스터 API에 +Kubeadm은 사용자가 쿠버네티스 클러스터를 배포하기 위한 사실상의 표준이 +되고 있다. kubeadm의 윈도우 노드 지원은 현재 작업 중이지만 +[여기](/ko/docs/tasks/administer-cluster/kubeadm/adding-windows-nodes/)에서 가이드를 사용할 수 있다. +또한 윈도우 노드가 적절하게 프로비저닝되도록 클러스터 API에 투자하고 있다. ### 몇 가지 기타 주요 기능 * 그룹 관리 서비스 어카운트(Service Accounts)에 대한 베타 지원 * 더 많은 CNI * 더 많은 스토리지 플러그인 - - diff --git a/content/ko/docs/tasks/access-application-cluster/access-cluster.md b/content/ko/docs/tasks/access-application-cluster/access-cluster.md index c433384f05..39226c89be 100644 --- a/content/ko/docs/tasks/access-application-cluster/access-cluster.md +++ b/content/ko/docs/tasks/access-application-cluster/access-cluster.md @@ -149,8 +149,8 @@ root 인증서를 사용하려면 특수한 설정을 필요로 할 것이다. localhost에서 제공되거나 방화벽으로 보호되는 몇몇 클러스터들에서는 apiserver가 인증을 요구하지 않지만 이는 표준이 아니다. -[Configuring Access to the API](/docs/reference/access-authn-authz/controlling-access/) -는 클러스터 관리자가 이를 어떻게 구성할 수 있는지를 설명한다. +[API에 대한 접근 구성](/ko/docs/reference/access-authn-authz/controlling-access/)은 +클러스터 관리자가 이를 어떻게 구성할 수 있는지를 설명한다. 이 방식들은 미래의 고가용성 지원과 충돌될 수 있다. ## API에 프로그래밍 방식으로 접근 @@ -218,7 +218,7 @@ apiserver의 인증서 제공을 검증하는데 사용되어야 한다. 이전 장은 쿠버네티스 API server 접속에 대한 내용을 다루었다. 이번 장은 쿠버네티스 클러스터 상에서 실행되는 다른 서비스로의 연결을 다룰 것이다. 쿠버네티스에서 [노드들](/ko/docs/concepts/architecture/nodes/), -[파드들](/ko/docs/concepts/workloads/pods/pod/), +[파드들](/ko/docs/concepts/workloads/pods/), [서비스들](/ko/docs/concepts/services-networking/service/)은 모두 자신의 IP들을 가진다. 당신의 데스크탑 PC와 같은 클러스터 외부 장비에서는 클러스터 상의 노드 IP들, 파드 IP들, 서비스 IP들로 라우팅되지 않아서 접근을 @@ -375,4 +375,3 @@ redirect 기능은 deprecated되고 제거 되었다. 대신 (아래의) proxy 일반적으로 쿠버네티스 사용자들은 처음 두 타입이 아닌 다른 방식은 고려할 필요가 없지만 클러스터 관리자는 나머지 타입을 적절하게 구성해줘야 한다. - diff --git a/content/ko/docs/tasks/access-application-cluster/configure-access-multiple-clusters.md b/content/ko/docs/tasks/access-application-cluster/configure-access-multiple-clusters.md index 9f99c6acbd..7312feec01 100644 --- a/content/ko/docs/tasks/access-application-cluster/configure-access-multiple-clusters.md +++ b/content/ko/docs/tasks/access-application-cluster/configure-access-multiple-clusters.md @@ -30,7 +30,7 @@ card: {{< glossary_tooltip text="kubectl" term_id="kubectl" >}}이 설치되었는지 확인하려면, `kubectl version --client`을 실행한다. kubectl 버전은 클러스터의 API 서버 버전과 -[마이너 버전 하나 차이 이내](/ko/docs/setup/release/version-skew-policy/#kubectl)여야 +[마이너 버전 하나 차이 이내](/docs/setup/release/version-skew-policy/#kubectl)여야 한다. diff --git a/content/ko/docs/tasks/access-application-cluster/web-ui-dashboard.md b/content/ko/docs/tasks/access-application-cluster/web-ui-dashboard.md index 664a049544..b352e6ffd0 100644 --- a/content/ko/docs/tasks/access-application-cluster/web-ui-dashboard.md +++ b/content/ko/docs/tasks/access-application-cluster/web-ui-dashboard.md @@ -5,7 +5,7 @@ weight: 10 card: name: tasks weight: 30 - title: Use the Web UI Dashboard + title: 웹 UI 대시보드 사용 --- @@ -52,7 +52,7 @@ kubectl 커맨드라인 도구를 이용해 다음 커맨드를 실행함으로 kubectl proxy ``` -kubectl은 http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/ 에 대시보드를 사용하는 것을 가능하게 해줄 것이다. +kubectl은 [http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/](http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/)에 대시보드를 사용하는 것을 가능하게 해줄 것이다. UI는 커맨드가 실행된 머신에서 _오직_ 접근 가능하다. 상세 내용은 `kubectl proxy --help` 옵션을 확인한다. @@ -170,7 +170,7 @@ Kubeconfig 인증 방법은 외부 아이덴티티 프로파이더 또는 x509 커맨드 옵션과 인자를 기본 옵션에 우선 적용하여 사용할 수 있다. - **특권을 가진(privileged) 상태로 실행**: 다음 세팅은 호스트에서 루트 권한을 가진 프로세스들이 - [특권을 가진 컨테이너](/ko/docs/concepts/workloads/pods/pod/#파드-컨테이너의-특권-privileged-모드)의 + [특권을 가진 컨테이너](/ko/docs/concepts/workloads/pods/#컨테이너에-대한-특권-모드)의 프로세스들과 동등한지 아닌지 정의한다. 특권을 가진(privileged) 컨테이너는 네트워크 스택과 디바이스에 접근하는 것을 조작하도록 활용할 수 있다. diff --git a/content/ko/docs/tasks/administer-cluster/access-cluster-api.md b/content/ko/docs/tasks/administer-cluster/access-cluster-api.md index e14c2a7166..ba4a794d23 100644 --- a/content/ko/docs/tasks/administer-cluster/access-cluster-api.md +++ b/content/ko/docs/tasks/administer-cluster/access-cluster-api.md @@ -148,7 +148,7 @@ http 클라이언트가 루트 인증서를 사용하도록 하려면 특별한 일부 클러스터에서, API 서버는 인증이 필요하지 않다. 로컬 호스트에서 제공되거나, 방화벽으로 보호될 수 있다. 이에 대한 표준은 -없다. [API에 대한 접근 구성](/docs/reference/access-authn-authz/controlling-access/)은 +없다. [API에 대한 접근 구성](/ko/docs/reference/access-authn-authz/controlling-access/)은 클러스터 관리자가 이를 구성하는 방법에 대해 설명한다. 이러한 접근 방식은 향후 고 가용성 지원과 충돌할 수 있다. diff --git a/content/ko/docs/tasks/administer-cluster/access-cluster-services.md b/content/ko/docs/tasks/administer-cluster/access-cluster-services.md index e65ec1cc1f..f8f707eb33 100644 --- a/content/ko/docs/tasks/administer-cluster/access-cluster-services.md +++ b/content/ko/docs/tasks/administer-cluster/access-cluster-services.md @@ -18,7 +18,7 @@ content_type: task ## 클러스터에서 실행되는 서비스에 접근 쿠버네티스에서, [노드](/ko/docs/concepts/architecture/nodes/), -[파드](/ko/docs/concepts/workloads/pods/pod/) 및 [서비스](/ko/docs/concepts/services-networking/service/)는 모두 +[파드](/ko/docs/concepts/workloads/pods/) 및 [서비스](/ko/docs/concepts/services-networking/service/)는 모두 고유한 IP를 가진다. 대부분의 경우, 클러스터의 노드 IP, 파드 IP 및 일부 서비스 IP는 라우팅할 수 없으므로, 데스크톱 시스템과 같은 클러스터 외부 시스템에서 도달할 수 없다. diff --git a/content/ko/docs/tasks/administer-cluster/network-policy-provider/cilium-network-policy.md b/content/ko/docs/tasks/administer-cluster/network-policy-provider/cilium-network-policy.md index f41b5cd716..78c19ad39b 100644 --- a/content/ko/docs/tasks/administer-cluster/network-policy-provider/cilium-network-policy.md +++ b/content/ko/docs/tasks/administer-cluster/network-policy-provider/cilium-network-policy.md @@ -69,7 +69,7 @@ L3/L4(예, IP 주소 + 포트) 모두의 보안 정책 뿐만 아니라 L7(예, ## 실리움을 실 서비스 용도로 배포하기 실리움을 실 서비스 용도의 배포에 관련한 자세한 방법은 -[실리움 쿠버네티스 설치 안내](https://docs.cilium.io/en/stable/kubernetes/intro/)를 살펴본다. +[실리움 쿠버네티스 설치 안내](https://docs.cilium.io/en/stable/concepts/kubernetes/intro/)를 살펴본다. 이 문서는 자세한 요구사항, 방법과 실제 데몬셋 예시를 포함한다. diff --git a/content/ko/docs/tasks/administer-cluster/network-policy-provider/weave-network-policy.md b/content/ko/docs/tasks/administer-cluster/network-policy-provider/weave-network-policy.md index b20c2e9890..736572c808 100644 --- a/content/ko/docs/tasks/administer-cluster/network-policy-provider/weave-network-policy.md +++ b/content/ko/docs/tasks/administer-cluster/network-policy-provider/weave-network-policy.md @@ -17,12 +17,12 @@ weight: 50 ## Weave Net 애드온을 설치한다 -[애드온을 통한 쿠버네티스 통합하기](https://www.weave.works/docs/net/latest/kube-addon/) 가이드를 따른다. +[애드온을 통한 쿠버네티스 통합하기](https://www.weave.works/docs/net/latest/kubernetes/kube-addon/) 가이드를 따른다. 쿠버네티스의 위브넷 애드온은 쿠버네티스의 모든 네임스페이스의 네크워크 정책 어노테이션을 자동으로 모니터링하며, 정책에 따라 트래픽을 허용하고 차단하는 `iptables` 규칙을 구성하는 -[네트워크 폴리시 컨트롤러](https://www.weave.works/docs/net/latest/kube-addon/#npc)와 함께 제공된다. +[네트워크 폴리시 컨트롤러](https://www.weave.works/docs/net/latest/kubernetes/kube-addon/#npc)와 함께 제공된다. ## 설치 시험 diff --git a/content/ko/docs/tasks/debug-application-cluster/resource-usage-monitoring.md b/content/ko/docs/tasks/debug-application-cluster/resource-usage-monitoring.md index 584afa5b19..87856b9cfe 100644 --- a/content/ko/docs/tasks/debug-application-cluster/resource-usage-monitoring.md +++ b/content/ko/docs/tasks/debug-application-cluster/resource-usage-monitoring.md @@ -7,7 +7,7 @@ title: 리소스 모니터링 도구 애플리케이션을 스케일하여 신뢰할 수 있는 서비스를 제공하려면, 애플리케이션이 배포되었을 때 애플리케이션이 어떻게 동작하는지를 이해해야 한다. -컨테이너, [파드](/ko/docs/concepts/workloads/pods/pod), +컨테이너, [파드](/ko/docs/concepts/workloads/pods/), [서비스](/ko/docs/concepts/services-networking/service), 그리고 전체 클러스터의 특성을 검사하여 쿠버네티스 클러스터 내의 애플리케이션 성능을 검사할 수 있다. 쿠버네티스는 각 레벨에서 diff --git a/content/ko/docs/tasks/job/coarse-parallel-processing-work-queue.md b/content/ko/docs/tasks/job/coarse-parallel-processing-work-queue.md new file mode 100644 index 0000000000..aeaac4803d --- /dev/null +++ b/content/ko/docs/tasks/job/coarse-parallel-processing-work-queue.md @@ -0,0 +1,328 @@ +--- +title: 작업 대기열을 사용한 거친 병렬 처리 +min-kubernetes-server-version: v1.8 +content_type: task +weight: 30 +--- + + + + +이 예제에서는, 여러 병렬 워커 프로세스를 활용해 쿠버네티스 잡(Job)을 +실행한다. + +이 예제에서는, 각 파드가 생성될 때 작업 대기열에서 하나의 작업 단위를 +선택하여, 완료하고, 대기열에서 삭제하고, 종료한다. + +이 예제에서의 단계에 대한 개요는 다음과 같다. + +1. **메시지 대기열 서비스를 시작한다.** 이 예에서는, RabbitMQ를 사용하지만, 다른 메시지 대기열을 이용해도 + 된다. 실제로 사용할 때는, 한 번 메시지 대기열 서비스를 구축하고서 이를 여러 잡을 위해 재사용하기도 한다. +1. **대기열을 만들고, 메시지로 채운다.** 각 메시지는 수행할 하나의 작업을 나타낸다. + 이 예제에서, 메시지는 긴 계산을 수행할 정수일 뿐이다. +1. **대기열에서 작업을 수행하는 잡을 시작한다.** 잡은 여러 파드를 시작한다. 각 파드는 + 메시지 대기열에서 하나의 작업을 가져와서, 처리한 다음, 대기열이 비워질 때까지 반복한다. + +## {{% heading "prerequisites" %}} + + +기본적이고, 병렬 작업이 아닌, +[잡](/ko/docs/concepts/workloads/controllers/job/)의 사용법에 대해 잘 알고 있어야 한다. + +{{< include "task-tutorial-prereqs.md" >}} + + + +## 메시지 대기열 서비스 시작 + +이 문서의 예시에서는 RabbitMQ를 사용하지만, 다른 AMQP 타입의 메시지 서비스에 적용하는데 문제가 없을 것이다. + +실제로 사용할 때는, 클러스터에 메시지 대기열 서비스를 한 번 +구축하고서, 여러 많은 잡이나 오래 동작하는 서비스에 재사용할 수 있다. + +다음과 같이 RabbitMQ를 시작한다. + +```shell +kubectl create -f https://raw.githubusercontent.com/kubernetes/kubernetes/release-1.3/examples/celery-rabbitmq/rabbitmq-service.yaml +``` +``` +service "rabbitmq-service" created +``` + +```shell +kubectl create -f https://raw.githubusercontent.com/kubernetes/kubernetes/release-1.3/examples/celery-rabbitmq/rabbitmq-controller.yaml +``` +``` +replicationcontroller "rabbitmq-controller" created +``` + +이 문서에서는 [celery-rabbitmq 예제](https://github.com/kubernetes/kubernetes/tree/release-1.3/examples/celery-rabbitmq)에 나오는 정도로만 rabbitmq를 사용한다. + +## 메시지 대기열 서비스 테스트하기 + +이제, 메시지 대기열을 이용해 실험할 수 있다. 임시 +대화형 파드를 만들어 그 위에 도구들을 설치하고, +대기열을 실험해본다. + +먼저 임시 대화형 파드를 만든다. + +```shell +# 임시 대화형 컨테이너를 만든다. +kubectl run -i --tty temp --image ubuntu:18.04 +``` +``` +Waiting for pod default/temp-loe07 to be running, status is Pending, pod ready: false +... [ previous line repeats several times .. hit return when it stops ] ... +``` + +참고로 파드 이름과 명령 프롬프트는 위와 다를 수 있다. + +다음으로 `amqp-tools`를 설치하여 메시지 대기열을 활용할 수 있게 한다. + +```shell +# 도구들을 설치한다. +root@temp-loe07:/# apt-get update +.... [ lots of output ] .... +root@temp-loe07:/# apt-get install -y curl ca-certificates amqp-tools python dnsutils +.... [ lots of output ] .... +``` + +후에, 이 패키지들을 포함하는 도커 이미지를 만든다. + +다음으로, rabbitmq 서비스를 발견할 수 있는지 확인한다. + +``` +# rabbitmq-service가 쿠버네티스로부터 주어진 DNS 이름을 갖는다. + +root@temp-loe07:/# nslookup rabbitmq-service +Server: 10.0.0.10 +Address: 10.0.0.10#53 + +Name: rabbitmq-service.default.svc.cluster.local +Address: 10.0.147.152 + +# 주소는 다를 수 있다. +``` + +만약 Kube-DNS가 적절히 구축되지 않았다면, 전 단계 작업이 작동하지 않을 수 있다. +환경 변수를 통해서도 서비스 IP를 찾을 수 있다. + +``` +# env | grep RABBIT | grep HOST +RABBITMQ_SERVICE_SERVICE_HOST=10.0.147.152 +# 주소는 다를 수 있다. +``` + +다음으로 대기열을 생성하고, 메시지를 발행하고 사용할 수 있는지 확인한다. + +```shell +# 다음 줄에서, rabbitmq-service는 rabbitmq-service에 접근할 수 있는 +# 호스트네임이다. 5672는 rabbitmq의 표준 포트이다. + +root@temp-loe07:/# export BROKER_URL=amqp://guest:guest@rabbitmq-service:5672 +# 만약 전 단계에서 "rabbitmq-service"가 주소로 변환되지 않는다면, +# 이 커맨드를 대신 사용하면 된다. +# root@temp-loe07:/# BROKER_URL=amqp://guest:guest@$RABBITMQ_SERVICE_SERVICE_HOST:5672 + +# 이제 대기열을 생성한다. + +root@temp-loe07:/# /usr/bin/amqp-declare-queue --url=$BROKER_URL -q foo -d +foo + +# 대기열에 메시지를 하나 발행한다. + +root@temp-loe07:/# /usr/bin/amqp-publish --url=$BROKER_URL -r foo -p -b Hello + +# 다시 메시지를 돌려받는다. + +root@temp-loe07:/# /usr/bin/amqp-consume --url=$BROKER_URL -q foo -c 1 cat && echo +Hello +root@temp-loe07:/# +``` + +마지막 커맨드에서, `amqp-consume` 도구는 대기열로부터 하나의 메시지를 +받고(`-c 1`), 그 메시지를 임의의 명령 표준입력으로 전달한다. 이 경우에는, `cat` 프로그램이 표준입력으로부터 +받은 값을 바로 출력하고 있고, echo가 캐리지 리턴을 더해주어 +출력 결과가 보여진다. + +## 작업으로 대기열 채우기 + +이제 몇 가지 "작업"으로 대기열을 채운다. 이 예제에서의 작업은 간단히 문자열을 +출력하는 것이다. + +실제로 사용할 때는, 메시지의 내용이 다음과 같을 수 있다. + +- 처리되어야 하는 파일들의 이름 +- 프로그램의 추가 플래그 +- 데이터베이스 테이블의 키(key) 범위 +- 시뮬레이션의 구성 파라미터 +- 렌더링해야 하는 씬(scene)의 프레임 번호 + +실제로는, 잡의 모든 파드에서 읽기-전용 모드로 필요한 큰 데이터가 +있다면, 일반적으로 그 데이터를 NFS와 같은 공유 파일시스템에 넣고 +모든 파드에 읽기 전용으로 마운트하거나, 파드 안에 있는 프로그램이 기본적으로 HDFS와 같은 +클러스터 파일시스템으로부터 데이터를 불러들인다. + +본 예제에서는, 대기열을 만들고 amqp 커맨드라인 도구를 이용해 대기열을 채울 것이다. +실제로는, amqp 라이브러리를 이용해 대기열을 채우는 프로그램을 작성하게 된다. + +```shell +/usr/bin/amqp-declare-queue --url=$BROKER_URL -q job1 -d +job1 +``` +```shell +for f in apple banana cherry date fig grape lemon melon +do + /usr/bin/amqp-publish --url=$BROKER_URL -r job1 -p -b $f +done +``` + +8개의 메시지로 대기열을 채웠다. + +## 이미지 생성 + +이제 잡으로 실행할 이미지를 만들 준비가 되었다. + +`amqp-consume` 유틸리티를 이용해 대기열로부터 메시지를 읽고, +실제 프로그램을 실행해 볼 것이다. +여기에 아주 간단한 예제 프로그램이 있다. + +{{< codenew language="python" file="application/job/rabbitmq/worker.py" >}} + +스크립트에 실행 권한을 준다. + +```shell +chmod +x worker.py +``` + +이제 이미지를 빌드한다. 만약 소스 트리 안에서 작업하고 +있다면, `examples/job/work-queue-1`로 디렉터리를 옮긴다. +아니면, 임시 디렉터리를 만들고, 그 디렉터리로 옮긴다. +[Dockerfile](/examples/application/job/rabbitmq/Dockerfile)과 +[worker.py](/examples/application/job/rabbitmq/worker.py)를 다운로드한다. +위 두 경우 모두, 다음의 명령을 이용해 이미지를 빌드한다. + +```shell +docker build -t job-wq-1 . +``` + +[도커 허브](https://hub.docker.com/)를 이용하기 위해, 앱 이미지를 +사용자의 username으로 태깅하고 아래의 명령어를 이용해 허브에 푸시한다. +``을 사용자의 허브 username으로 대체한다. + +```shell +docker tag job-wq-1 /job-wq-1 +docker push /job-wq-1 +``` + +만약 [구글 컨테이너 +레지스트리](https://cloud.google.com/tools/container-registry/)를 이용하고 있다면, +앱 이미지를 사용자의 프로젝트 ID를 이용해 태깅하고, GCR에 푸시한다. +`` 부분을 사용자의 프로젝트 ID로 대체한다. + +```shell +docker tag job-wq-1 gcr.io//job-wq-1 +gcloud docker -- push gcr.io//job-wq-1 +``` + +## 잡 정의 + +다음은 잡 정의이다. 잡의 사본을 만들고 위에서 정한 이름에 맞게 +이미지를 수정하고, 파일 이름을 `./job.yaml`이라 정한다. + + +{{< codenew file="application/job/rabbitmq/job.yaml" >}} + +이 예시에서는, 각 파드가 대기열로부터 얻은 하나의 아이템을 수행하고 종료한다. +그래서, 잡의 완료 횟수가 완료된 작업 아이템의 숫자에 대응한다. +예시에서 `.spec.completions: 8`이라 정한 것도, 대기열에 8개의 아이템을 넣었기 때문이다. + +## 잡 실행 + +이제 잡을 실행한다. + +```shell +kubectl apply -f ./job.yaml +``` + +이제 조금 기다린 다음, 잡을 확인한다. + +```shell +kubectl describe jobs/job-wq-1 +``` +``` +Name: job-wq-1 +Namespace: default +Selector: controller-uid=41d75705-92df-11e7-b85e-fa163ee3c11f +Labels: controller-uid=41d75705-92df-11e7-b85e-fa163ee3c11f + job-name=job-wq-1 +Annotations: +Parallelism: 2 +Completions: 8 +Start Time: Wed, 06 Sep 2017 16:42:02 +0800 +Pods Statuses: 0 Running / 8 Succeeded / 0 Failed +Pod Template: + Labels: controller-uid=41d75705-92df-11e7-b85e-fa163ee3c11f + job-name=job-wq-1 + Containers: + c: + Image: gcr.io/causal-jigsaw-637/job-wq-1 + Port: + Environment: + BROKER_URL: amqp://guest:guest@rabbitmq-service:5672 + QUEUE: job1 + Mounts: + Volumes: +Events: + FirstSeen LastSeen Count From SubobjectPath Type Reason Message + ───────── ──────── ───── ──── ───────────── ────── ────── ─────── + 27s 27s 1 {job } Normal SuccessfulCreate Created pod: job-wq-1-hcobb + 27s 27s 1 {job } Normal SuccessfulCreate Created pod: job-wq-1-weytj + 27s 27s 1 {job } Normal SuccessfulCreate Created pod: job-wq-1-qaam5 + 27s 27s 1 {job } Normal SuccessfulCreate Created pod: job-wq-1-b67sr + 26s 26s 1 {job } Normal SuccessfulCreate Created pod: job-wq-1-xe5hj + 15s 15s 1 {job } Normal SuccessfulCreate Created pod: job-wq-1-w2zqe + 14s 14s 1 {job } Normal SuccessfulCreate Created pod: job-wq-1-d6ppa + 14s 14s 1 {job } Normal SuccessfulCreate Created pod: job-wq-1-p17e0 +``` + +모든 파드가 성공했다. 야호. + + + + + +## 대안 + +이러한 접근은 "워커" 프로그램을 작업 대기열에 맞게 수정하지 않아도 +된다는 장점이 있다. + +이 접근을 이용하려면, 메시지 대기열 서비스를 실행해야만 한다. +만약 메시지 대기열 서비스를 실행하는 게 불편하다면, +다른 [잡 패턴](/ko/docs/concepts/workloads/controllers/job/#잡-패턴)을 고려해볼 수 있다. + +이 접근은 모든 작업 아이템에 대해 파드를 생성한다. 만약 작업 아이템이 오직 몇 초밖에 걸리지 않는 작업이라면, +매 작업마다 파드를 생성하는 것은 아주 큰 오버헤드를 더할 수 있다. 하나의 파드가 +여러 작업 아이템을 수행하는 이 [예제](/ko/docs/tasks/job/fine-parallel-processing-work-queue/)를 고려해보자. + +이 예제에서는, `amqp-consume` 유틸리티를 이용해 대기열로부터 메시지를 읽어 +실제 프로그램을 실행했다. 이러면 메시지 대기열을 이용하기 위해 프로그램을 수정하지 +않아도 된다는 장점이 있다. +[다른 예제](/ko/docs/tasks/job/fine-parallel-processing-work-queue/)는 +클라이언트 라이브러리를 이용해 작업 대기열과 소통하는 방법을 보여준다. + +## 주의 사항 + +만약 작업 완료 수가 대기열에 있는 아이템의 숫자보다 적게 설정되면, +모든 아이템 처리되지 않는다. + +만약 작업 완료 수가 큐에 있는 아이템의 숫자보다 많게 설정되면, +대기열에 있는 아이템이 모두 처리되어도, 잡이 완료됐다고 표시되지 +않고, 메시지를 기다리는 과정에서 막히는 파드를 +추가적으로 실행시킨다. + +이 패턴에서는 경쟁 상태(race)가 잘 나타나지 않는다. 만약 amqp-consume 명령으로부터 +메시지가 인정되는 시간과 컨테이너가 성공적으로 종료되는 +시간 사이에 컨테이너가 종료되거나, kubelet이 api-server에게 파드가 성공했음을 알리기 전에 +노드가 비정상적으로 종료되면, 대기열의 모든 아이템이 처리되었다 해도, +잡이 완료되었다고 표시되지 않는다. diff --git a/content/ko/docs/tasks/manage-kubernetes-objects/imperative-command.md b/content/ko/docs/tasks/manage-kubernetes-objects/imperative-command.md index a10cd29362..28655ed57b 100644 --- a/content/ko/docs/tasks/manage-kubernetes-objects/imperative-command.md +++ b/content/ko/docs/tasks/manage-kubernetes-objects/imperative-command.md @@ -37,7 +37,7 @@ weight: 30 지원한다. 쿠버네티스 오브젝트 타입에 익숙하지 않은 사용자가 인지할 수 있도록 커맨드 이름이 지어졌다. -- `run`: 하나 이상의 파드 내 컨테이너를 실행하도록 새로운 디플로이먼트 오브젝트를 생성한다. +- `run`: 컨테이너를 실행할 새로운 파드를 생성한다. - `expose`: 파드에 걸쳐 트래픽을 로드 밸런스하도록 새로운 서비스 오브젝트를 생성한다. - `autoscale`: 디플로이먼트와 같이, 하나의 컨트롤러에 대해 자동으로 수평적 스케일이 이루어 지도록 새로운 Autoscaler 오브젝트를 생성한다. diff --git a/content/ko/docs/tasks/manage-kubernetes-objects/imperative-config.md b/content/ko/docs/tasks/manage-kubernetes-objects/imperative-config.md index b9c90475d8..9e4f8de774 100644 --- a/content/ko/docs/tasks/manage-kubernetes-objects/imperative-config.md +++ b/content/ko/docs/tasks/manage-kubernetes-objects/imperative-config.md @@ -64,6 +64,18 @@ weight: 40 * `kubectl delete -f <파일명|url>` +{{< note >}} +구성 파일이 `metadata` 섹션에서 `name` 필드 대신 `generateName` +필드를 지정한 경우, `kubectl delete -f ` 을 사용하여 +오브젝트를 삭제할 수 없다. +오브젝트를 삭제하려면 다른 플래그를 사용해야 한다. 예를 들면, 다음과 같다. + +```shell +kubectl delete +kubectl delete -l