Merge pull request #31375 from sftim/20211207_mark_only_live_site_indexable_v1.19

Mark only live site indexable (v1.19 backport)
Mark live site in HTML
2022-01-27 11:05:50 -08:00 · 2022-01-17 22:43:24 +00:00 · 2022-01-17 22:43:15 +00:00 · 2021-12-07 15:59:22 -08:00 · 2021-12-06 11:57:49 -05:00 · 2021-08-04 16:05:11 -07:00
1946 changed files with 112382 additions and 64440 deletions
@@ -5,6 +5,9 @@ charset = utf-8
 max_line_length = 80
 trim_trailing_whitespace = true

+[*.md]
+trim_trailing_whitespace = false
+
 [*.{css,html,js,json,sass,md,mmark,toml,yaml}]
 indent_style = space
 indent_size = 2
@@ -2,7 +2,7 @@
 name: Support Request
 about: Support request or question relating to Kubernetes Website project
 labels:
- triage/support
+- kind/support
 ---
 **This is Support**

@@ -0,0 +1,7 @@
+# See the OWNERS docs at https://go.k8s.io/owners
+
+reviewers:
+- sig-docs-en-reviews # Defined in OWNERS_ALIASES
+
+approvers:
+- sig-docs-en-owners # Defined in OWNERS_ALIASES
@@ -14,9 +14,11 @@
 Use the default base branch, “master”, if you're documenting existing
 features in the English localization.

- If you're working on a different localization (not English), or you
- are documenting a feature that will be part of a future release, see
+ If you're working on a different localization (not English), see
 https://kubernetes.io/docs/contribute/new-content/overview/#choose-which-git-branch-to-use
 for advice.

+ If you're documenting a feature that will be part of a future release, see
+ https://kubernetes.io/docs/contribute/new-content/new-features/ for advice.
+
 -->
@@ -0,0 +1,11 @@
+# See the OWNERS docs at https://go.k8s.io/owners
+
+# When modifying this file, consider the security implications of
+# allowing listed reviewers / approvals to modify or remove any
+# configured GitHub Actions.
+
+reviewers:
+- sig-docs-leads
+
+approvers:
+- sig-docs-leads
@@ -18,7 +18,7 @@ The [Kubernetes SIG Docs Discussion Group](https://groups.google.com/forum/#!for

 ### Style Guides and Templates

-Before submitting a pull request to create new content, please review the [Kubernetes.io style guide](http://kubernetes.io/docs/home/contribute/style-guide/) and follow the [instructions for using page templates](http://kubernetes.io/docs/home/contribute/page-templates/).
+Before submitting a pull request to create new content, please review the [Kubernetes.io style guide](http://kubernetes.io/docs/home/contribute/style-guide/) and follow the [instructions for using page templates](https://kubernetes.io/docs/contribute/style/page-content-types/).


 ## Contributing to Documentation
@@ -35,3 +35,5 @@ Note that code issues should be filed against the main kubernetes repository, wh
 ### Submitting Documentation Pull Requests

 If you're fixing an issue in the existing documentation, you should submit a PR against the master branch.  Follow [these instructions to create a documentation pull request against the kubernetes.io repository](http://kubernetes.io/docs/home/contribute/create-pull-request/).
+
+For more information, see [contributing to Kubernetes docs](https://kubernetes.io/docs/contribute/).
@@ -16,7 +16,7 @@ RUN apk add --no-cache \
    build-base \
    libc6-compat \
    npm && \
-    npm install -G autoprefixer postcss-cli
+    npm install -D autoprefixer postcss-cli

 ARG HUGO_VERSION

@@ -29,4 +29,6 @@ RUN mkdir -p /usr/local/src && \

 WORKDIR /src

+USER hugo:hugo
+
 EXPOSE 1313
@@ -23,14 +23,14 @@ module-check:

 all: build ## Build site with production settings and put deliverables in ./public

-build: module-check ## Build site with production settings and put deliverables in ./public
-	hugo --minify
+build: module-check ## Build site with non-production settings and put deliverables in ./public
+	hugo --minify --environment development

 build-preview: module-check ## Build site with drafts and future posts enabled
-	hugo --buildDrafts --buildFuture
+	hugo --buildDrafts --buildFuture --environment preview

 deploy-preview: ## Deploy preview site via netlify
-	hugo --enableGitInfo --buildFuture -b $(DEPLOY_PRIME_URL)
+	hugo --enableGitInfo --buildFuture --environment preview -b $(DEPLOY_PRIME_URL)

 functions-build:
 	$(NETLIFY_FUNC) build functions-src
@@ -38,13 +38,15 @@ functions-build:
 check-headers-file:
 	scripts/check-headers-file.sh

-production-build: build check-headers-file ## Build the production site and ensure that noindex headers aren't added
+production-build: module-check ## Build the production site and ensure that noindex headers aren't added
+	hugo --minify --environment production
+	HUGO_ENV=production $(MAKE) check-headers-file

-non-production-build: ## Build the non-production site, which adds noindex headers to prevent indexing
-	hugo --enableGitInfo
+non-production-build: module-check ## Build the non-production site, which adds noindex headers to prevent indexing
+	hugo --enableGitInfo --environment nonprod

 serve: module-check ## Boot the development server.
-	hugo server --buildFuture
+	hugo server --buildFuture --environment development

 docker-image:
 	@echo -e "$(CCRED)**** The use of docker-image is deprecated. Use container-image instead. ****$(CCEND)"
@@ -65,10 +67,10 @@ container-image:
 		--build-arg HUGO_VERSION=$(HUGO_VERSION)

 container-build: module-check
-	$(CONTAINER_RUN) $(CONTAINER_IMAGE) hugo --minify
+	$(CONTAINER_RUN) --read-only --mount type=tmpfs,destination=/tmp,tmpfs-mode=01777 $(CONTAINER_IMAGE) sh -c "npm ci && hugo --minify --environment development"

 container-serve: module-check
-	$(CONTAINER_RUN) --mount type=tmpfs,destination=/src/resources,tmpfs-mode=0755 -p 1313:1313 $(CONTAINER_IMAGE) hugo server --buildFuture --bind 0.0.0.0
+	$(CONTAINER_RUN) --read-only --mount type=tmpfs,destination=/tmp,tmpfs-mode=01777 -p 1313:1313 $(CONTAINER_IMAGE) hugo server --buildFuture --environment development --bind 0.0.0.0 --destination /tmp/hugo --cleanDestinationDir

 test-examples:
 	scripts/test_examples.sh install
@@ -83,5 +85,5 @@ docker-internal-linkcheck:
 	$(MAKE) container-internal-linkcheck

 container-internal-linkcheck: link-checker-image-pull
-	$(CONTAINER_RUN) $(CONTAINER_IMAGE) hugo --config config.toml,linkcheck-config.toml --buildFuture
+	$(CONTAINER_RUN) $(CONTAINER_IMAGE) hugo --config config.toml,linkcheck-config.toml --buildFuture --environment test
 	$(CONTAINER_ENGINE) run --mount type=bind,source=$(CURDIR),target=/test --rm wjdp/htmltest htmltest
@@ -7,10 +7,11 @@ approvers:
 - sig-docs-en-owners # Defined in OWNERS_ALIASES

 emeritus_approvers:
-# chenopis, you're welcome to return when you're ready to resume PR wrangling
-# jaredbhatti, you're welcome to return when you're ready to resume PR wrangling
-# stewart-yu, you're welcome to return when you're ready to resume PR wrangling
+# - chenopis, commented out to disable PR assignments
+# - jaredbhatti, commented out to disable PR assignments
+# - steveperry-53, commented out to disable PR assignments
 - stewart-yu 
+- zacharysarah

 labels:
 - sig/docs
@@ -22,18 +22,18 @@ aliases:
    - mkorbi
    - rlenferink
  sig-docs-en-owners: # Admins for English content
+    - annajung
    - bradtopol
    - celestehorgan
+    - irvifa
    - jimangel
    - kbarnard10
    - kbhawkey
-    - makoscafee
    - onlydole
    - savitharaghunathan
    - sftim
    - steveperry-53
    - tengqm
-    - xiangpengzhao
    - zacharysarah
    - zparnold
  sig-docs-en-reviews: # PR reviews for English content
@@ -43,14 +43,11 @@ aliases:
    - jimangel
    - kbarnard10
    - kbhawkey
-    - makoscafee
    - onlydole
    - rajeshdeshpande02
    - sftim
    - steveperry-53
    - tengqm
-    - xiangpengzhao
-    - zacharysarah
    - zparnold
  sig-docs-es-owners: # Admins for Spanish content
    - raelga
@@ -123,17 +120,17 @@ aliases:
    - bells17
    # cstoku
    - inductor
+    - kakts
    - makocchi-git
    # MasayaAoyama
    - nasa9084
-    - oke-py
+    # oke-py
  sig-docs-ko-owners: # Admins for Korean content
    - ClaudiaJKang
    - gochist
    - ianychoi
    - seokho-son
    - ysyukr
-    - zacharysarah
  sig-docs-ko-reviews: # PR reviews for Korean content
    - ClaudiaJKang
    - gochist
@@ -142,35 +139,34 @@ aliases:
    - ysyukr
    - pjhwa
  sig-docs-leads: # Website chairs and tech leads
+    - irvifa
    - jimangel
    - kbarnard10
    - kbhawkey
    - onlydole
    - sftim
-    - zacharysarah
  sig-docs-zh-owners: # Admins for Chinese content
-    - chenopis
+    # chenopis
    - chenrui333
-    - dchen1107
-    - haibinxie
-    - hanjiayao
-    - lichuqiang
-    - SataQiu
-    - tengqm
-    - xiangpengzhao
-    - xichengliudui
-    - zacharysarah
-    - zhangxiaoyu-zidif
-  sig-docs-zh-reviews: # PR reviews for Chinese content
-    - chenrui333
-    - idealhack
+    # dchen1107
+    # haibinxie
+    # hanjiayao
+    # lichuqiang
    - SataQiu
    - tanjunchen
    - tengqm
-    - xiangpengzhao
    - xichengliudui
-    - zhangxiaoyu-zidif
+    # zhangxiaoyu-zidif
+  sig-docs-zh-reviews: # PR reviews for Chinese content
+    - chenrui333
+    - howieyuen
+    - idealhack
    - pigletfly
+    - SataQiu
+    - tanjunchen
+    - tengqm
+    - xichengliudui
+    # zhangxiaoyu-zidif
  sig-docs-pt-owners: # Admins for Portuguese content
    - femrtnz
    - jcjesus
@@ -40,13 +40,13 @@ Um die Kubernetes-Website lokal laufen zu lassen, empfiehlt es sich, ein speziel
 Wenn Sie Docker [installiert](https://www.docker.com/get-started) haben, erstellen Sie das Docker-Image `kubernetes-hugo` lokal:

 ```bash
-make docker-image
+make container-image
 ```

 Nachdem das Image erstellt wurde, können Sie die Site lokal ausführen:

 ```bash
-make docker-serve
+make container-serve
 ```

 Öffnen Sie Ihren Browser unter http://localhost:1313, um die Site anzuzeigen. Wenn Sie Änderungen an den Quelldateien vornehmen, aktualisiert Hugo die Site und erzwingt eine Browseraktualisierung.
@@ -33,13 +33,13 @@ El método recomendado para levantar una copia local del sitio web kubernetes.io
 Una vez tenga Docker [configurado en su máquina](https://www.docker.com/get-started), puede construir la imagen de Docker `kubernetes-hugo` localmente ejecutando el siguiente comando en la raíz del repositorio:

 ```bash
-make docker-image
+make container-image
 ```

 Una vez tenga la imagen construida, puede levantar el sitio web ejecutando:

 ```bash
-make docker-serve
+make container-serve
 ```

 Abra su navegador y visite http://localhost:1313 para acceder a su copia local del sitio. A medida que vaya haciendo cambios en el código fuente, Hugo irá actualizando la página y forzará la actualización en el navegador.
@@ -16,13 +16,13 @@ Faites tous les changements que vous voulez dans votre fork, et quand vous êtes
 Une fois votre pull request créée, un examinateur de Kubernetes se chargera de vous fournir une revue claire et exploitable.
 En tant que propriétaire de la pull request, **il est de votre responsabilité de modifier votre pull request pour tenir compte des commentaires qui vous ont été fournis par l'examinateur de Kubernetes.**
 Notez également que vous pourriez vous retrouver avec plus d'un examinateur de Kubernetes pour vous fournir des commentaires ou vous pourriez finir par recevoir des commentaires d'un autre examinateur que celui qui vous a été initialement affecté pour vous fournir ces commentaires.
-De plus, dans certains cas, l'un de vos examinateur peut demander un examen technique à un [examinateur technique de Kubernetes](https://github.com/kubernetes/website/wiki/Tech-reviewers) au besoin.
+De plus, dans certains cas, l'un de vos examinateurs peut demander un examen technique à un [examinateur technique de Kubernetes](https://github.com/kubernetes/website/wiki/Tech-reviewers) au besoin.
 Les examinateurs feront de leur mieux pour fournir une revue rapidement, mais le temps de réponse peut varier selon les circonstances.

 Pour plus d'informations sur la contribution à la documentation Kubernetes, voir :

 * [Commencez à contribuer](https://kubernetes.io/docs/contribute/start/)
-* [Apperçu des modifications apportées à votre documentation](http://kubernetes.io/docs/contribute/intermediate#view-your-changes-locally)
+* [Aperçu des modifications apportées à votre documentation](http://kubernetes.io/docs/contribute/intermediate#view-your-changes-locally)
 * [Utilisation des modèles de page](https://kubernetes.io/docs/contribute/style/page-content-types/)
 * [Documentation Style Guide](http://kubernetes.io/docs/contribute/style/style-guide/)
 * [Traduction de la documentation Kubernetes](https://kubernetes.io/docs/contribute/localization/)
@@ -38,13 +38,13 @@ La façon recommandée d'exécuter le site web Kubernetes localement est d'utili
 Si vous avez Docker [up and running](https://www.docker.com/get-started), construisez l'image Docker `kubernetes-hugo' localement:

 ```bash
-make docker-image
+make container-image
 ```

 Une fois l'image construite, vous pouvez exécuter le site localement :

 ```bash
-make docker-serve
+make container-serve
 ```

 Ouvrez votre navigateur à l'adresse: http://localhost:1313 pour voir le site.
@@ -17,7 +17,7 @@

 * [योगदान देना शुरू करें](https://kubernetes.io/docs/contribute/start/)
 * [परिवर्तनों को अंतिम चरण में लेजाएं](http://kubernetes.io/docs/contribute/intermediate#view-your-changes-locally)
-* [पेज टेम्पलेट](http://kubernetes.io/docs/contribute/style/page-templates/)
+* [पेज टेम्पलेट](https://kubernetes.io/docs/contribute/style/page-content-types/)
 * [प्रलेखन शैली गाइड](http://kubernetes.io/docs/contribute/style/style-guide/)
 * [स्थानीयकरण कुबरनेट्स प्रलेखन](https://kubernetes.io/docs/contribute/localization/)

@@ -41,13 +41,13 @@
 यदि आप [डॉकर](https://www.docker.com/get-started) चला रहे हैं, तो स्थानीय रूप से `कुबेरनेट्स-ह्यूगो` Docker image बनाएँ:

 ```bash
-make docker-image
+make container-image
 ```

 एक बार image बन जाने के बाद, आप साइट को स्थानीय रूप से चला सकते हैं:

 ```bash
-make docker-serve
+make container-serve
 ```

 साइट देखने के लिए अपने browser को `http://localhost:1313` पर खोलें। जैसा कि आप source फ़ाइलों में परिवर्तन करते हैं, Hugo साइट को अपडेट करता है और browser को refresh करने पर मजबूर करता है।
@@ -81,4 +81,4 @@ make serve

 कुबरनेट्स सामुदायिक भागीदारी पर पनपती है, और हम वास्तव में हमारी साइट और हमारे प्रलेखन में आपके योगदान की सराहना करते हैं!

-कुबरनेट्स आपकी भागीदारी पर निर्भर है, और हम हमारी साइट और प्रलेखन में आपके योगदान का मान करते हैं!
+कुबरनेट्स आपकी भागीदारी पर निर्भर है, और हम हमारी साइट और प्रलेखन में आपके योगदान का मान करते हैं!
@@ -15,7 +15,7 @@ Untuk informasi lebih lanjut mengenai tata cara melakukan kontribusi, kamu dapat

 * [Petunjuk Melakukan Kontribusi](https://kubernetes.io/docs/contribute/start/)
 * [Melakukan Tahap Staging pada Konten Dokumentasi yang telah Kamu Sediakan](http://kubernetes.io/docs/contribute/intermediate#view-your-changes-locally)
-* [Petunjuk Menggunakan Page Templates](http://kubernetes.io/docs/contribute/style/page-templates/)
+* [Petunjuk Menggunakan Page Templates](https://kubernetes.io/docs/contribute/style/page-content-types/)
 * [Petunjuk untuk Documentation Style](http://kubernetes.io/docs/contribute/style/style-guide/)
 * [Petunjuk untuk Melakukan Lokalisasi Dokumentasi Kubernetes](https://kubernetes.io/docs/contribute/localization/)

@@ -30,13 +30,13 @@ Petunjuk yang disarankan untuk menjalankan Dokumentasi Kubernetes pada mesin lok
 Jika kamu sudah memiliki **Docker** [yang sudah dapat digunakan](https://www.docker.com/get-started), kamu dapat melakukan **build** `kubernetes-hugo` **Docker image** secara lokal:

 ```bash
-make docker-image
+make container-image
 ```

 Setelah **image** berhasil di-**build**, kamu dapat menjalankan website tersebut pada mesin lokal-mu:

 ```bash
-make docker-serve
+make container-serve
 ```

 Buka **browser** kamu ke http://localhost:1313 untuk melihat laman dokumentasi. Selama kamu melakukan penambahan konten, **Hugo** akan secara otomatis melakukan perubahan terhadap laman dokumentasi apabila **browser** melakukan proses **refresh**.
@@ -15,7 +15,7 @@ Per maggiori informazioni su come contribuire alla documentazione Kubernetes, ve

 * [Cominciare a contribuire](https://kubernetes.io/docs/contribute/start/)
 * [Vedere le modifiche localmente](http://kubernetes.io/docs/contribute/intermediate#view-your-changes-locally)
-* [Utilizzare i template delle pagine](http://kubernetes.io/docs/contribute/style/page-templates/)
+* [Utilizzare i template delle pagine](https://kubernetes.io/docs/contribute/style/page-content-types/)
 * [Guida Stilistica per la documentazione di Kubernetes](http://kubernetes.io/docs/contribute/style/style-guide/)
 * [Tradurre la documentazione di Kubernetes](https://kubernetes.io/docs/contribute/localization/)

@@ -30,13 +30,13 @@ Il modo consigliato per eseguire localmente il sito Web Kubernetes prevede l'uti
 Se hai Docker [attivo e funzionante](https://www.docker.com/get-started), crea l'immagine Docker `kubernetes-hugo` localmente:

 ```bash
-make docker-image
+make container-image
 ```

 Dopo aver creato l'immagine, è possibile eseguire il sito Web localmente:

 ```bash
-make docker-serve
+make container-serve
 ```

 Apri il tuo browser su http://localhost:1313 per visualizzare il sito Web. Mentre modifichi i file sorgenti, Hugo aggiorna automaticamente il sito Web e forza un aggiornamento della pagina visualizzata nel browser.
@@ -4,34 +4,100 @@

 このリポジトリには、[KubernetesのWebサイトとドキュメント](https://kubernetes.io/)をビルドするために必要な全アセットが格納されています。貢献に興味を持っていただきありがとうございます！

-## Hugoを使ってローカル環境でWebサイトを動かす
+# リポジトリの使い方

-Hugoのインストール方法については[Hugoの公式ドキュメント](https://gohugo.io/getting-started/installing/)をご覧ください。このとき、[`netlify.toml`](netlify.toml#L10)ファイルに記述されている`HUGO_VERSION`と同じバージョンをインストールするようにしてください。
+Hugo(Extended version)を使用してWebサイトをローカルで実行することも、コンテナランタイムで実行することもできます。コンテナランタイムを使用することを強くお勧めします。これにより、本番Webサイトとのデプロイメントの一貫性が得られます。

-Hugoがインストールできたら、以下のコマンドを使ってWebサイトをローカル上で動かすことができます:
+## 前提条件

-```bash
+このリポジトリを使用するには、以下をローカルにインストールする必要があります。
+
+- [npm](https://www.npmjs.com/)
+- [Go](https://golang.org/)
+- [Hugo(Extended version)](https://gohugo.io/)
+- [Docker](https://www.docker.com/)などのコンテナランタイム
+
+開始する前に、依存関係をインストールしてください。リポジトリのクローンを作成し、ディレクトリに移動します。
+
+```
 git clone https://github.com/kubernetes/website.git
 cd website
+```
+
+KubernetesのWebサイトではDocsyというHugoテーマを使用しています。コンテナでWebサイトを実行する場合でも、以下を実行して、サブモジュールおよびその他の開発依存関係をプルすることを強くお勧めします。
+
+```
+# pull in the Docsy submodule
 git submodule update --init --recursive --depth 1
 ```

-**注意:** Kubernetesのウェブサイトでは[DocsyというHugoのテーマ](https://github.com/google/docsy#readme)を使用しています。リポジトリを更新していない場合、 `website/themes/docsy`ディレクトリは空です。 このサイトはテーマのローカルコピーなしでは構築できません。
+## コンテナを使ってウェブサイトを動かす

-テーマをアップデートするには以下のコマンドを実行します:
+コンテナ内でサイトを構築するには、以下を実行してコンテナイメージを構築し、実行します。

-```bash
-git submodule update --init --recursive --depth 1
+```
+make container-image
+make container-serve
 ```

-サイトをローカルでビルドしてテストするには以下のコマンドを実行します:
+お使いのブラウザにて http://localhost:1313 にアクセスしてください。リポジトリ内のソースファイルに変更を加えると、HugoがWebサイトの内容を更新してブラウザに反映します。
+
+## Hugoを使ってローカル環境でWebサイトを動かす
+
+[`netlify.toml`](netlify.toml#L10)ファイルに記述されている`HUGO_VERSION`と同じExtended versionのHugoをインストールするようにしてください。
+
+ローカルでサイトを構築してテストするには、次のコマンドを実行します。

 ```bash
-hugo server --buildFuture
+# install dependencies
+npm ci
+make serve
 ```

 これで、Hugoのサーバーが1313番ポートを使って開始します。お使いのブラウザにて http://localhost:1313 にアクセスしてください。リポジトリ内のソースファイルに変更を加えると、HugoがWebサイトの内容を更新してブラウザに反映します。

+## トラブルシューティング
+
+### error: failed to transform resource: TOCSS: failed to transform "scss/main.scss" (text/x-scss): this feature is not available in your current Hugo version
+
+Hugoは、技術的な理由から2種類のバイナリがリリースされています。現在のウェブサイトは**Hugo Extended**バージョンのみに基づいて運営されています。[リリースページ](https://github.com/gohugoio/hugo/releases)で名前に「extended」が含まれるアーカイブを探します。確認するには、`hugo version`を実行し、「extended」という単語を探します。
+
+### macOSにてtoo many open filesというエラーが表示される
+
+macOS上で`make serve`を実行した際に以下のエラーが表示される場合
+
+```
+ERROR 2020/08/01 19:09:18 Error: listen tcp 127.0.0.1:1313: socket: too many open files
+make: *** [serve] Error 1
+```
+
+OS上で同時に開けるファイルの上限を確認してください。
+
+`launchctl limit maxfiles`
+
+続いて、以下のコマンドを実行します(https://gist.github.com/tombigel/d503800a282fcadbee14b537735d202c より引用)。
+
+```
+#!/bin/sh
+
+# These are the original gist links, linking to my gists now.
+# curl -O https://gist.githubusercontent.com/a2ikm/761c2ab02b7b3935679e55af5d81786a/raw/ab644cb92f216c019a2f032bbf25e258b01d87f9/limit.maxfiles.plist
+# curl -O https://gist.githubusercontent.com/a2ikm/761c2ab02b7b3935679e55af5d81786a/raw/ab644cb92f216c019a2f032bbf25e258b01d87f9/limit.maxproc.plist
+
+curl -O https://gist.githubusercontent.com/tombigel/d503800a282fcadbee14b537735d202c/raw/ed73cacf82906fdde59976a0c8248cce8b44f906/limit.maxfiles.plist
+curl -O https://gist.githubusercontent.com/tombigel/d503800a282fcadbee14b537735d202c/raw/ed73cacf82906fdde59976a0c8248cce8b44f906/limit.maxproc.plist
+
+sudo mv limit.maxfiles.plist /Library/LaunchDaemons
+sudo mv limit.maxproc.plist /Library/LaunchDaemons
+
+sudo chown root:wheel /Library/LaunchDaemons/limit.maxfiles.plist
+sudo chown root:wheel /Library/LaunchDaemons/limit.maxproc.plist
+
+sudo launchctl load -w /Library/LaunchDaemons/limit.maxfiles.plist
+```
+
+こちらはmacOSのCatalinaとMojaveで動作を確認しています。
+
 ## SIG Docsに参加する

 [コミュニティのページ](https://github.com/kubernetes/community/tree/master/sig-docs#meetings)をご覧になることで、SIG Docs Kubernetesコミュニティとの関わり方を学ぶことができます。
@@ -1,69 +1,144 @@
 # 쿠버네티스 문서화

-[![Build Status](https://api.travis-ci.org/kubernetes/website.svg?branch=master)](https://travis-ci.org/kubernetes/website)
-[![GitHub release](https://img.shields.io/github/release/kubernetes/website.svg)](https://github.com/kubernetes/website/releases/latest)
+[![Netlify Status](https://api.netlify.com/api/v1/badges/be93b718-a6df-402a-b4a4-855ba186c97d/deploy-status)](https://app.netlify.com/sites/kubernetes-io-master-staging/deploys) [![GitHub release](https://img.shields.io/github/release/kubernetes/website.svg)](https://github.com/kubernetes/website/releases/latest)

-환영합니다! 이 저장소는 쿠버네티스 웹사이트 및 문서화를 만드는 데 필요로 하는 모든 asset에 대한 공간을 제공합니다. 여러분이 기여를 원한다는 사실에 매우 기쁩니다!
+이 저장소에는 [쿠버네티스 웹사이트 및 문서](https://kubernetes.io/)를 빌드하는 데 필요한 자산이 포함되어 있습니다. 기여해주셔서 감사합니다!

-## 문서에 기여하기
+# 저장소 사용하기

-이 저장소에 대한 복제본을 여러분의 GitHub 계정에 생성하기 위해 화면 오른쪽 위 영역에 있는 **Fork** 버튼을 클릭 가능합니다. 이 복제본은 *fork* 라고 부릅니다. 여러분의 fork에서 원하는 임의의 변경 사항을 만들고, 해당 변경 사항을 보낼 준비가 되었다면, 여러분의 fork로 이동하여 새로운 풀 리퀘스트를 만들어 우리에게 알려주시기 바랍니다.
+Hugo(확장 버전)를 사용하여 웹사이트를 로컬에서 실행하거나, 컨테이너 런타임에서 실행할 수 있습니다. 라이브 웹사이트와의 배포 일관성을 제공하므로, 컨테이너 런타임을 사용하는 것을 적극 권장합니다.

-여러분의 풀 리퀘스트가 생성된 이후에는, 쿠버네티스 리뷰어가 명료하고 실행 가능한 피드백을 제공하는 책임을 담당할 것입니다. 풀 리퀘스트의 오너로서, **쿠버네티스 리뷰어로부터 제공받은 피드백을 수용하기 위해 풀 리퀘스트를 수정하는 것은 여러분의 책임입니다.** 또한, 참고로 한 명 이상의 쿠버네티스 리뷰어가 여러분에게 피드백을 제공하는 상황에 처하거나, 또는 여러분에게 피드백을 제공하기로 원래 할당된 사람이 아닌 다른 쿠버네티스 리뷰어로부터 피드백을 받는 상황에 처할 수도 있습니다.  그뿐만 아니라, 몇몇 상황에서는, 필요에 따라 리뷰어 중 한 명이 [쿠버네티스 기술 리뷰어](https://github.com/kubernetes/website/wiki/Tech-reviewers)로부터의 기술 리뷰를 요청할지도 모릅니다. 리뷰어는 제시간에 피드백을 제공하기 위해 최선을 다할 것이지만, 응답 시간은 상황에 따라 달라질 수도 있습니다.
+## 사전 준비 사항

-쿠버네티스 문서화에 기여하기와 관련된 보다 자세한 정보는, 다음을 살펴봅니다:
+이 저장소를 사용하기 위해, 로컬에 다음의 소프트웨어들이 설치되어 있어야 합니다.

-* [기여 시작하기](https://kubernetes.io/docs/contribute/start/)
-* [문서화 변경 사항 스테이징하기](http://kubernetes.io/docs/contribute/intermediate#view-your-changes-locally)
-* [페이지 템플릿 사용하기](http://kubernetes.io/docs/contribute/style/page-templates/)
-* [문서화 스타일 가이드](http://kubernetes.io/docs/contribute/style/style-guide/)
-* [쿠버네티스 문서화 로컬라이징](https://kubernetes.io/docs/contribute/localization/)
+- [npm](https://www.npmjs.com/)
+- [Go](https://golang.org/)
+- [Hugo(확장 버전)](https://gohugo.io/)
+- [도커](https://www.docker.com/)와 같은 컨테이너 런타임.

-## `README.md`에 대한 쿠버네티스 문서화 번역
+시작하기 전에 의존성이 있는 소프트웨어를 설치합니다. 저장소를 복제(clone)하고 디렉터리로 이동합니다.

-### 한국어
-
-`README.md` 번역 및 한국어 기여자를 위한 보다 자세한 가이드를 [한국어 README](README-ko.md) 페이지 혹은 [쿠버네티스 문서 한글화 가이드](https://kubernetes.io/ko/docs/contribute/localization_ko/)에서 살펴봅니다.
-
-한국어 번역 메인테이너에게 다음을 통해 연락 가능합니다.
-
-* 이덕준 ([GitHub - @gochist](https://github.com/gochist))
-* [Slack channel](https://kubernetes.slack.com/messages/kubernetes-docs-ko)
-
-## 도커를 사용하여 사이트를 로컬에서 실행하기
-
-쿠버네티스 웹사이트를 로컬에서 실행하기 위한 추천하는 방식은 [Hugo](https://gohugo.io) 정적 사이트 생성기를 포함하는 특별한 [도커](https://docker.com) 이미지를 실행하는 것입니다.
-
-> Windows에서 실행하는 경우, [Chocolatey](https://chocolatey.org)로 설치할 수 있는 명명 추가 도구를 필요로 할 것입니다. `choco install make`
-
-> 도커를 사용하지 않고 웹사이트를 로컬에서 실행하기를 선호하는 경우에는, 아래 [Hugo를 사용한 로컬 사이트 실행하기](#hugo를-사용한-로컬-사이트-실행하기)를 살펴봅니다.
-
-도커 [동작 및 실행](https://www.docker.com/get-started) 환경이 있는 경우, 로컬에서 `kubernetes-hugo` 도커 이미지를 빌드 합니다:
-
-```bash
-make docker-image
+```
+git clone https://github.com/kubernetes/website.git
+cd website
 ```

-해당 이미지가 빌드 된 이후, 사이트를 로컬에서 실행할 수 있습니다:
+쿠버네티스 웹사이트는 [Docsy Hugo 테마](https://github.com/google/docsy#readme)를 사용합니다. 웹사이트를 컨테이너에서 실행하려는 경우에도, 다음을 실행하여 하위 모듈 및 기타 개발 종속성을 가져오는 것이 좋습니다.

-```bash
-make docker-serve
+```
+# Docsy 하위 모듈 가져오기
+git submodule update --init --recursive --depth 1
 ```

-브라우저에서 http://localhost:1313 를 열어 사이트를 살펴봅니다. 소스 파일에 변경 사항이 있을 때, Hugo는 사이트를 업데이트하고 브라우저를 강제로 새로고침합니다.
+## 컨테이너를 사용하여 웹사이트 실행하기

-## Hugo를 사용한 로컬 사이트 실행하기
+컨테이너에서 사이트를 빌드하려면, 다음을 실행하여 컨테이너 이미지를 빌드하고 실행합니다.

-Hugo 설치 안내를 위해서는 [공식 Hugo 문서화](https://gohugo.io/getting-started/installing/)를 살펴봅니다. [`netlify.toml`](netlify.toml#L9) 파일에 있는 `HUGO_VERSION` 환경 변수에서 지정된 Hugo 버전이 설치되었는지를 확인합니다.
+```
+make container-image
+make container-serve
+```

-Hugo가 설치되었을 때 로컬에서 사이트를 실행하기 위해 (다음을 실행합니다):
+웹사이트를 보려면 브라우저를 http://localhost:1313 으로 엽니다. 소스 파일을 변경하면 Hugo가 웹사이트를 업데이트하고 브라우저를 강제로 새로 고칩니다.
+
+## Hugo를 사용하여 로컬에서 웹사이트 실행하기
+
+[`netlify.toml`](netlify.toml#L10) 파일의 `HUGO_VERSION` 환경 변수에 지정된 Hugo 확장 버전을 설치해야 합니다.
+
+사이트를 로컬에서 빌드하고 테스트하려면, 다음을 실행합니다.

 ```bash
+# 의존성 있는 소프트웨어 설치
+npm ci
 make serve
 ```

-이를 통해 로컬 Hugo 서버를 1313번 포트에 시작합니다. 브라우저에서 http://localhost:1313 를 열어 사이트를 살펴봅니다. 소스 파일에 변경 사항이 있을 때, Hugo는 사이트를 업데이트하고 브라우저를 강제로 새로고침합니다.
+그러면 포트 1313에서 로컬 Hugo 서버가 시작됩니다. 웹사이트를 보려면 http://localhost:1313 으로 브라우저를 엽니다. 소스 파일을 변경하면, Hugo가 웹사이트를 업데이트하고 브라우저를 강제로 새로 고칩니다.

-## 감사합니다!
+## 문제 해결
+### error: failed to transform resource: TOCSS: failed to transform "scss/main.scss" (text/x-scss): this feature is not available in your current Hugo version

-쿠버네티스는 커뮤니티 참여와 함께 생존하며, 우리는 사이트 및 문서화에 대한 여러분의 컨트리뷰션에 대해 정말 감사하게 생각합니다!
+Hugo는 기술적인 이유로 2개의 바이너리 세트로 제공됩니다. 현재 웹사이트는 **Hugo 확장** 버전 기반에서만 실행됩니다. [릴리스 페이지](https://github.com/gohugoio/hugo/releases)에서 이름에 `extended` 가 포함된 아카이브를 찾습니다. 확인하려면, `hugo version` 을 실행하고 `extended` 라는 단어를 찾습니다.
+
+### too many open files 이슈에 대한 macOS 문제 해결
+
+macOS에서 `make serve` 를 실행하면 다음의 오류 메시지가 출력됩니다.
+
+```
+ERROR 2020/08/01 19:09:18 Error: listen tcp 127.0.0.1:1313: socket: too many open files
+make: *** [serve] Error 1
+```
+
+파일 오픈 개수에 대한 현재 제한값을 확인합니다.
+
+`launchctl limit maxfiles`
+
+그리고 다음의 명령을 실행합니다(https://gist.github.com/tombigel/d503800a282fcadbee14b537735d202c 를 참고하여 적용).
+
+```
+#!/bin/sh
+
+# 코멘트 처리한 것은 원래 gist 링크들이며, 그 아래는 수정된 tombigel의 gist 링크입니다.
+# curl -O https://gist.githubusercontent.com/a2ikm/761c2ab02b7b3935679e55af5d81786a/raw/ab644cb92f216c019a2f032bbf25e258b01d87f9/limit.maxfiles.plist
+# curl -O https://gist.githubusercontent.com/a2ikm/761c2ab02b7b3935679e55af5d81786a/raw/ab644cb92f216c019a2f032bbf25e258b01d87f9/limit.maxproc.plist
+
+curl -O https://gist.githubusercontent.com/tombigel/d503800a282fcadbee14b537735d202c/raw/ed73cacf82906fdde59976a0c8248cce8b44f906/limit.maxfiles.plist
+curl -O https://gist.githubusercontent.com/tombigel/d503800a282fcadbee14b537735d202c/raw/ed73cacf82906fdde59976a0c8248cce8b44f906/limit.maxproc.plist
+
+sudo mv limit.maxfiles.plist /Library/LaunchDaemons
+sudo mv limit.maxproc.plist /Library/LaunchDaemons
+
+sudo chown root:wheel /Library/LaunchDaemons/limit.maxfiles.plist
+sudo chown root:wheel /Library/LaunchDaemons/limit.maxproc.plist
+
+sudo launchctl load -w /Library/LaunchDaemons/limit.maxfiles.plist
+```
+
+이 내용은 Catalina와 Mojave macOS에서 작동합니다.
+
+
+# SIG Docs에 참여하기
+
+[커뮤니티 페이지](https://github.com/kubernetes/community/tree/master/sig-docs#meetings)에서 SIG Docs 쿠버네티스 커뮤니티 및 회의에 대한 자세한 내용을 확인합니다.
+
+이 프로젝트의 메인테이너에게 연락을 할 수도 있습니다.
+
+- [슬랙](https://kubernetes.slack.com/messages/sig-docs) [슬랙에 초대 받기](https://slack.k8s.io/)
+- [메일링 리스트](https://groups.google.com/forum/#!forum/kubernetes-sig-docs)
+
+# 문서에 기여하기
+
+이 저장소에 대한 복제본을 여러분의 GitHub 계정에 생성하기 위해 화면 오른쪽 위 영역에 있는 **Fork** 버튼을 클릭하면 됩니다. 이 복제본은 *fork* 라고 부릅니다. 여러분의 fork에서 원하는 임의의 변경 사항을 만들고, 해당 변경 사항을 보낼 준비가 되었다면, 여러분의 fork로 이동하여 새로운 풀 리퀘스트를 만들어 우리에게 알려주시기 바랍니다.
+
+여러분의 풀 리퀘스트가 생성된 이후에는, 쿠버네티스 리뷰어가 명료하고 실행 가능한 피드백을 제공하는 책임을 담당할 것입니다. 풀 리퀘스트의 오너로서, **쿠버네티스 리뷰어로부터 제공받은 피드백을 수용하기 위해 풀 리퀘스트를 수정하는 것은 여러분의 책임입니다.**
+
+또한, 참고로 한 명 이상의 쿠버네티스 리뷰어가 여러분에게 피드백을 제공하는 상황이거나, 또는 원래 여러분에게 피드백을 제공하기로 할당된 사람이 아닌 다른 쿠버네티스 리뷰어로부터 피드백을 받는 상황도 있습니다.
+
+그뿐만 아니라, 몇몇 상황에서는, 필요에 따라 리뷰어 중 한 명이 [쿠버네티스 기술 리뷰어](https://github.com/kubernetes/website/wiki/Tech-reviewers)로부터의 기술 리뷰를 요청할지도 모릅니다. 리뷰어는 제시간에 피드백을 제공하기 위해 최선을 다할 것이지만, 응답 시간은 상황에 따라 달라질 수도 있습니다.
+
+쿠버네티스 문서화에 기여하기와 관련된 보다 자세한 정보는, 다음을 참고합니다.
+
+* [쿠버네티스 문서에 기여하기](https://kubernetes.io/docs/contribute/)
+* [페이지 콘텐트 타입](https://kubernetes.io/docs/contribute/style/page-content-types/)
+* [문서화 스타일 가이드](http://kubernetes.io/docs/contribute/style/style-guide/)
+* [쿠버네티스 문서 현지화](https://kubernetes.io/docs/contribute/localization/)
+
+# `README.md`에 대한 쿠버네티스 문서 현지화(localization)
+
+## 한국어
+
+`README.md` 번역 및 한국어 기여자를 위한 보다 자세한 가이드는 [쿠버네티스 문서 한글화 가이드](https://kubernetes.io/ko/docs/contribute/localization_ko/)를 참고합니다.
+
+한국어 번역 메인테이너에게 다음을 통해 연락할 수 있습니다.
+
+* 손석호 ([GitHub - @seokho-son](https://github.com/seokho-son))
+* [슬랙 채널](https://kubernetes.slack.com/messages/kubernetes-docs-ko)
+
+# 행동 강령
+
+쿠버네티스 커뮤니티 참여는 [CNCF 행동 강령](https://github.com/cncf/foundation/blob/master/code-of-conduct-languages/ko.md)을 따릅니다.
+
+# 감사합니다!
+
+쿠버네티스는 커뮤니티 참여를 통해 번창하며, 우리는 웹사이트 및 문서화에 대한 당신의 기여에 감사드립니다!
@@ -17,7 +17,7 @@ Więcej informacji na temat współpracy przy tworzeniu dokumentacji znajdziesz

 * [Jak rozpocząć współpracę](https://kubernetes.io/docs/contribute/start/)
 * [Podgląd wprowadzanych zmian w dokumentacji](http://kubernetes.io/docs/contribute/intermediate#view-your-changes-locally)
-* [Szablony stron](http://kubernetes.io/docs/contribute/style/page-templates/)
+* [Szablony stron](https://kubernetes.io/docs/contribute/style/page-content-types/)
 * [Styl pisania dokumentacji](http://kubernetes.io/docs/contribute/style/style-guide/)
 * [Lokalizacja dokumentacji Kubernetes](https://kubernetes.io/docs/contribute/localization/)

@@ -49,13 +49,13 @@ choco install make
 Jeśli [zainstalowałeś i uruchomiłeś](https://www.docker.com/get-started) już Dockera, zbuduj obraz `kubernetes-hugo` lokalnie:

 ```bash
-make docker-image
+make container-image
 ```

 Po zbudowaniu obrazu, możesz uruchomić serwis lokalnie:

 ```bash
-make docker-serve
+make container-serve
 ```

 Aby obejrzeć zawartość serwisu otwórz w przeglądarce adres http://localhost:1313. Po każdej zmianie plików źródłowych, Hugo automatycznie aktualizuje stronę i odświeża jej widok w przeglądarce.
@@ -35,13 +35,13 @@ A maneira recomendada de executar o site do Kubernetes localmente é executar um
 Se você tiver o Docker [em funcionamento](https://www.docker.com/get-started), crie a imagem do Docker do `kubernetes-hugo` localmente:

 ```bash
-make docker-image
+make container-image
 ```

 Depois que a imagem foi criada, você pode executar o site localmente:

 ```bash
-make docker-serve
+make container-serve
 ```

 Abra seu navegador para http://localhost:1313 para visualizar o site. Conforme você faz alterações nos arquivos de origem, Hugo atualiza o site e força a atualização do navegador.
@@ -38,8 +38,8 @@ hugo server --buildFuture
 Узнать подробнее о том, как поучаствовать в документации Kubernetes, вы можете по ссылкам ниже:

 * [Начните вносить свой вклад](https://kubernetes.io/docs/contribute/)
-* [Использование шаблонов страниц](http://kubernetes.io/docs/contribute/style/page-templates/)
-* [Руководство по оформлению документации](http://kubernetes.io/docs/contribute/style/style-guide/)
+* [Использование шаблонов страниц](https://kubernetes.io/docs/contribute/style/page-content-types/)
+* [Руководство по оформлению документации](https://kubernetes.io/docs/contribute/style/style-guide/)
 * [Руководство по локализации Kubernetes](https://kubernetes.io/docs/contribute/localization/)

 ## Файл `README.md` на других языках
@@ -15,7 +15,7 @@ Một khi Pull Request của bạn được tạo, reviewer sẽ chịu trách n

 * [Bắt đầu đóng góp](https://kubernetes.io/docs/contribute/start/)
 * [Các giai đoạn thay đổi tài liệu](http://kubernetes.io/docs/contribute/intermediate#view-your-changes-locally)
-* [Sử dụng các trang templates](http://kubernetes.io/docs/contribute/style/page-templates/)
+* [Sử dụng các trang templates](https://kubernetes.io/docs/contribute/style/page-content-types/)
 * [Hướng dẫn biểu mẫu tài liệu](http://kubernetes.io/docs/contribute/style/style-guide/)
 * [Địa phương hóa tài liệu Kubernetes](https://kubernetes.io/docs/contribute/localization/)

@@ -31,13 +31,13 @@ Cách được đề xuất để chạy trang web Kubernetes cục bộ là dù
 Nếu bạn có Docker đang [up và running](https://www.docker.com/get-started), build `kubernetes-hugo` Docker image cục bộ:

 ```bash
-make docker-image
+make container-image
 ```

 Khi image đã được built, bạn có thể chạy website cục bộ:

 ```bash
-make docker-serve
+make container-serve
 ```

 Mở trình duyệt và đến địa chỉ http://localhost:1313 để xem website. Khi bạn thay đổi các file nguồn, Hugo cập nhật website và buộc làm mới trình duyệt.
@@ -13,52 +13,94 @@ This repository contains the assets required to build the [Kubernetes website an
 我们非常高兴您想要参与贡献！

 <!--
-## Running the website locally using Hugo
+# Using this repository

-See the [official Hugo documentation](https://gohugo.io/getting-started/installing/) for Hugo installation instructions. Make sure to install the Hugo extended version specified by the `HUGO_VERSION` environment variable in the [`netlify.toml`](netlify.toml#L10) file.
+You can run the website locally using Hugo (Extended version), or you can run it in a container runtime. We strongly recommend using the container runtime, as it gives deployment consistency with the live website.
 -->
-## 在本地使用 Hugo 来运行网站
+## 使用这个仓库

-请参考 [Hugo 的官方文档](https://gohugo.io/getting-started/installing/)了解 Hugo 的安装指令。
-请确保安装的是 [`netlify.toml`](netlify.toml#L10) 文件中环境变量 `HUGO_VERSION` 所指定的
-Hugo 扩展版本。
+可以使用 Hugo（扩展版）在本地运行网站，也可以在容器中运行它。强烈建议使用容器，因为这样可以和在线网站的部署保持一致。

 <!--
-Before building the site, clone the Kubernetes website repository:
-->
-在构造网站之前，先克隆 Kubernetes website 仓库：
+## Prerequisites

-```bash
+To use this repository, you need the following installed locally:
+
+- [npm](https://www.npmjs.com/)
+- [Go](https://golang.org/)
+- [Hugo (Extended version)](https://gohugo.io/)
+- A container runtime, like [Docker](https://www.docker.com/).
+
+-->
+## 前提条件
+
+使用这个仓库，需要在本地安装以下软件：
+
+- [npm](https://www.npmjs.com/)
+- [Go](https://golang.org/)
+- [Hugo (Extended version)](https://gohugo.io/)
+- 容器运行时，比如 [Docker](https://www.docker.com/).
+
+<!--
+Before you start, install the dependencies. Clone the repository and navigate to the directory:
+-->
+开始前，先安装这些依赖。克隆本仓库并进入对应目录：
+
+```
 git clone https://github.com/kubernetes/website.git
 cd website
-git submodule update --init --recursive
 ```

 <!--
-**Note:**  The Kubernetes website deploys the [Docsy Hugo theme](https://github.com/google/docsy#readme).
-If you have not updated your website repository, the `website/themes/docsy` directory is empty.
-The site cannot build without a local copy of the theme.
-
-Update the website theme:
+The Kubernetes website uses the [Docsy Hugo theme](https://github.com/google/docsy#readme). Even if you plan to run the website in a container, we strongly recommend pulling in the submodule and other development dependencies by running the following:
 -->
-**注意：** Kubernetes 网站要部署 [Docsy Hugo 主题](https://github.com/google/docsy#readme).
-如果你还没有更新你本地的 website 仓库，目录 `website/themes/docsy`
-会是空目录。
-在本地没有主题副本的情况下，网站无法正常构造。

-使用下面的命令更新网站主题：
+Kubernetes 网站使用的是 [Docsy Hugo 主题](https://github.com/google/docsy#readme)。 即使你打算在容器中运行网站，我们也强烈建议你通过运行以下命令来引入子模块和其他开发依赖项：

-```bash
+```
+# pull in the Docsy submodule
 git submodule update --init --recursive --depth 1
 ```

 <!--
+## Running the website using a container
+
+To build the site in a container, run the following to build the container image and run it:
+
+-->
+## 在容器中运行网站
+
+要在容器中构建网站，请通过以下命令来构建容器镜像并运行：
+
+```
+make container-image
+make container-serve
+```
+
+<!--
+Open up your browser to http://localhost:1313 to view the website. As you make changes to the source files, Hugo updates the website and forces a browser refresh.
+-->
+启动浏览器，打开 http://localhost:1313 来查看网站。
+当你对源文件作出修改时，Hugo 会更新网站并强制浏览器执行刷新操作。
+
+<!--
+## Running the website locally using Hugo
+
+Make sure to install the Hugo extended version specified by the `HUGO_VERSION` environment variable in the [`netlify.toml`](netlify.toml#L10) file.
+
 To build and test the site locally, run:
 -->
+## 在本地使用 Hugo 来运行网站
+
+请确保安装的是 [`netlify.toml`](netlify.toml#L10) 文件中环境变量 `HUGO_VERSION` 所指定的
+Hugo 扩展版本。
+
 若要在本地构造和测试网站，请运行：

 ```bash
-hugo server --buildFuture
+# install dependencies
+npm ci
+make serve
 ```

 <!--
@@ -68,6 +110,63 @@ This will start the local Hugo server on port 1313. Open up your browser to http
 启动浏览器，打开 http://localhost:1313 来查看网站。
 当你对源文件作出修改时，Hugo 会更新网站并强制浏览器执行刷新操作。

+<!--
+## Troubleshooting
+### error: failed to transform resource: TOCSS: failed to transform "scss/main.scss" (text/x-scss): this feature is not available in your current Hugo version
+
+Hugo is shipped in two set of binaries for technical reasons. The current website runs based on the **Hugo Extended** version only. In the [release page](https://github.com/gohugoio/hugo/releases) look for archives with `extended` in the name. To confirm, run `hugo version` and look for the word `extended`.
+
+-->
+## 故障排除
+
+###  error: failed to transform resource: TOCSS: failed to transform "scss/main.scss" (text/x-scss): this feature is not available in your current Hugo version
+
+由于技术原因，Hugo 会发布两套二进制文件。
+当前网站仅基于 **Hugo Extended** 版本运行。
+在 [发布页面](https://github.com/gohugoio/hugo/releases) 中查找名称为 `extended` 的归档。可以运行 `huge version` 查看是否有单词 `extended` 来确认。
+
+<!--
+### Troubleshooting macOS for too many open files
+
+If you run `make serve` on macOS and receive the following error:
+
+-->
+### 对 macOs 上打开太多文件的故障排除
+
+如果在 macOS 上运行 `make serve` 收到以下错误：
+
+```
+ERROR 2020/08/01 19:09:18 Error: listen tcp 127.0.0.1:1313: socket: too many open files
+make: *** [serve] Error 1
+```
+
+试着查看一下当前打开文件数的限制：
+
+`launchctl limit maxfiles`
+
+然后运行以下命令（参考https://gist.github.com/tombigel/d503800a282fcadbee14b537735d202c）：
+
+```
+#!/bin/sh
+
+# These are the original gist links, linking to my gists now.
+# curl -O https://gist.githubusercontent.com/a2ikm/761c2ab02b7b3935679e55af5d81786a/raw/ab644cb92f216c019a2f032bbf25e258b01d87f9/limit.maxfiles.plist
+# curl -O https://gist.githubusercontent.com/a2ikm/761c2ab02b7b3935679e55af5d81786a/raw/ab644cb92f216c019a2f032bbf25e258b01d87f9/limit.maxproc.plist
+
+curl -O https://gist.githubusercontent.com/tombigel/d503800a282fcadbee14b537735d202c/raw/ed73cacf82906fdde59976a0c8248cce8b44f906/limit.maxfiles.plist
+curl -O https://gist.githubusercontent.com/tombigel/d503800a282fcadbee14b537735d202c/raw/ed73cacf82906fdde59976a0c8248cce8b44f906/limit.maxproc.plist
+
+sudo mv limit.maxfiles.plist /Library/LaunchDaemons
+sudo mv limit.maxproc.plist /Library/LaunchDaemons
+
+sudo chown root:wheel /Library/LaunchDaemons/limit.maxfiles.plist
+sudo chown root:wheel /Library/LaunchDaemons/limit.maxproc.plist
+
+sudo launchctl load -w /Library/LaunchDaemons/limit.maxfiles.plist
+```
+
+这适用于 Catalina 和 Mojave macOS。
+
 <!--
 ## Get involved with SIG Docs

@@ -78,7 +177,7 @@ You can also reach the maintainers of this project at:
 - [Slack](https://kubernetes.slack.com/messages/sig-docs)
 - [Mailing List](https://groups.google.com/forum/#!forum/kubernetes-sig-docs)
 -->
-## 参与 SIG Docs 工作
+# 参与 SIG Docs 工作

 通过 [社区页面](https://github.com/kubernetes/community/tree/master/sig-docs#meetings)
 进一步了解 SIG Docs Kubernetes 社区和会议信息。
@@ -95,7 +194,7 @@ You can click the **Fork** button in the upper-right area of the screen to creat

 Once your pull request is created, a Kubernetes reviewer will take responsibility for providing clear, actionable feedback.  As the owner of the pull request, **it is your responsibility to modify your pull request to address the feedback that has been provided to you by the Kubernetes reviewer.**
 -->
-## 为文档做贡献
+# 为文档做贡献

 你也可以点击屏幕右上方区域的 **Fork** 按钮，在你自己的 GitHub
 账号下创建本仓库的拷贝。此拷贝被称作 *fork*。
@@ -133,7 +232,7 @@ For more information about contributing to the Kubernetes documentation, see:
 * [文档风格指南](http://kubernetes.io/docs/contribute/style/style-guide/)
 * [本地化 Kubernetes 文档](https://kubernetes.io/docs/contribute/localization/)

-## 中文本地化
+# 中文本地化

 可以通过以下方式联系中文本地化的维护人员：

@@ -146,15 +245,15 @@ For more information about contributing to the Kubernetes documentation, see:

 Participation in the Kubernetes community is governed by the [CNCF Code of Conduct](https://github.com/cncf/foundation/blob/master/code-of-conduct.md).
 -->
-### 行为准则
+# 行为准则

-参与 Kubernetes 社区受 [CNCF 行为准则](https://github.com/cncf/foundation/blob/master/code-of-conduct.md)约束。
+参与 Kubernetes 社区受 [CNCF 行为准则](https://github.com/cncf/foundation/blob/master/code-of-conduct.md) 约束。

 <!--
 ## Thank you!

 Kubernetes thrives on community participation, and we appreciate your contributions to our website and our documentation!
 -->
-## 感谢！
+# 感谢！

 Kubernetes 因为社区的参与而蓬勃发展，感谢您对我们网站和文档的贡献！
@@ -6,16 +6,15 @@ This repository contains the assets required to build the [Kubernetes website an

 # Using this repository

-You can run the website locally using Hugo, or you can run it in a container runtime. We strongly recommend using the container runtime, as it gives deployment consistency with the live website.
+You can run the website locally using Hugo (Extended version), or you can run it in a container runtime. We strongly recommend using the container runtime, as it gives deployment consistency with the live website.

 ## Prerequisites

 To use this repository, you need the following installed locally:

- [yarn](https://yarnpkg.com/)
 - [npm](https://www.npmjs.com/)
 - [Go](https://golang.org/)
- [Hugo](https://gohugo.io/)
+- [Hugo (Extended version)](https://gohugo.io/)
 - A container runtime, like [Docker](https://www.docker.com/).

 Before you start, install the dependencies. Clone the repository and navigate to the directory:
@@ -28,9 +27,6 @@ cd website
 The Kubernetes website uses the [Docsy Hugo theme](https://github.com/google/docsy#readme). Even if you plan to run the website in a container, we strongly recommend pulling in the submodule and other development dependencies by running the following:

 ```
-# install dependencies
-yarn
-
 # pull in the Docsy submodule
 git submodule update --init --recursive --depth 1
 ```
@@ -53,11 +49,18 @@ Make sure to install the Hugo extended version specified by the `HUGO_VERSION` e
 To build and test the site locally, run:

 ```bash
+# install dependencies
+npm ci
 make serve
 ```

 This will start the local Hugo server on port 1313. Open up your browser to http://localhost:1313 to view the website. As you make changes to the source files, Hugo updates the website and forces a browser refresh.

+## Troubleshooting
+### error: failed to transform resource: TOCSS: failed to transform "scss/main.scss" (text/x-scss): this feature is not available in your current Hugo version
+
+Hugo is shipped in two set of binaries for technical reasons. The current website runs based on the **Hugo Extended** version only. In the [release page](https://github.com/gohugoio/hugo/releases) look for archives with `extended` in the name. To confirm, run `hugo version` and look for the word `extended`.
+
 ### Troubleshooting macOS for too many open files

 If you run `make serve` on macOS and receive the following error:
@@ -71,7 +74,7 @@ Try checking the current limit for open files:

 `launchctl limit maxfiles`

-Then run the following commands:
+Then run the following commands (adapted from https://gist.github.com/tombigel/d503800a282fcadbee14b537735d202c):

 ```
 #!/bin/sh
@@ -101,7 +104,7 @@ Learn more about SIG Docs Kubernetes community and meetings on the [community pa

 You can also reach the maintainers of this project at:

- [Slack](https://kubernetes.slack.com/messages/sig-docs)
+- [Slack](https://kubernetes.slack.com/messages/sig-docs) [Get an invite for this Slack](https://slack.k8s.io/)
 - [Mailing List](https://groups.google.com/forum/#!forum/kubernetes-sig-docs)

 # Contributing to the docs
@@ -10,6 +10,7 @@
 # DO NOT REPORT SECURITY VULNERABILITIES DIRECTLY TO THESE NAMES, FOLLOW THE
 # INSTRUCTIONS AT https://kubernetes.io/security/

+irvifa
 jimangel
 kbarnard10
-zacharysarah
+sftim
@@ -17,14 +17,22 @@ limitations under the License.
  var Search = {
    init: function () {
      $(document).ready(function () {
+        // Fill the search input form with the current search keywords
+        const searchKeywords = new URLSearchParams(location.search).get('q');
+        if (searchKeywords !== null && searchKeywords !== '') {
+          const searchInput = document.querySelector('.td-search-input');
+          searchInput.focus();
+          searchInput.value = searchKeywords;
+        }
+
+        // Set a keydown event
        $(document).on("keypress", ".td-search-input", function (e) {
          if (e.keyCode !== 13) {
            return;
          }

          var query = $(this).val();
-          var searchPage = "{{ "docs/search/" | absURL }}?q=" + query;
-          document.location = searchPage;
+          document.location = "{{ "search/" | absURL }}?q=" + query;

          return false;
        });
@@ -42,6 +42,10 @@ $video-section-height: 200px;

 body {
  background-color: white;
+
+  a {
+    color: $blue;
+  }
 }

 section {
@@ -71,6 +75,7 @@ footer {
    background-color: $blue;
    text-decoration: none;
    font-size: 1rem;
+    border: 0px;
  }

  #cellophane {
@@ -83,6 +88,20 @@ footer {
  }
 }

+main {
+  .button {
+    display: inline-block;
+    border-radius: 6px;
+    padding: 6px 20px;
+    line-height: 1.3rem;
+    color: white;
+    background-color: $blue;
+    text-decoration: none;
+    font-size: 1rem;
+    border: 0px;
+  }
+}
+
 // HEADER

 #hamburger {
@@ -336,7 +355,6 @@ dd {
    width: 100%;
    height: 45px;
    line-height: 45px;
-    font-family: "Roboto", sans-serif;
    font-size: 20px;
    color: $blue;
  }
@@ -612,7 +630,6 @@ section#cncf {
  padding-top: 30px;
  padding-bottom: 80px;
  background-size: auto;
-  // font-family: "Roboto Mono", monospace !important;
  font-size: 24px;
  // font-weight: bold;

@@ -2,6 +2,7 @@

 hr {
  background-color: #999999;
+  margin-top: 0;
 }

 h2 {
@@ -10,7 +11,7 @@ h2 {

 .subhead {
  padding-bottom: 2% !important;
-  padding-top: 0% !important;
+  padding-top: 0.75em !important;
 }

 .details {
@@ -11,15 +11,19 @@ $announcement-size-adjustment: 8px;
      max-width: 100%;
    }

-    table {
-      margin-bottom: 1rem;
-      display: initial;
-    }
-
    @media only screen and (min-width: 768px) {
      padding-top: 2rem !important;
    }
  }
+
+  .ui-widget {
+    font-family: inherit;
+    font-size: inherit;
+  }
+
+  .ui-widget-content a {
+    color: $blue;
+  }
 }

 section {
@@ -34,6 +38,14 @@ section {
  padding: 0 !important;
 }

+
+/* Emphasize first paragraph of running text on site front page */
+body.td-home main[role="main"] > section:first-of-type .content p:first-child {
+  line-height: 1.3em;
+  font-size: 1.4em;
+  margin-bottom: 1.5em;
+}
+
 #desktopShowVideoButton {
  border: none
 }
@@ -44,6 +56,39 @@ section {
  }
 }

+body.td-404 main .error-details {
+  max-width: 1100px;
+  margin-left: auto;
+  margin-right: auto;
+  margin-top: 4em;
+  margin-bottom: 0;
+}
+
+/* Global - Mermaid.js diagrams */
+
+.mermaid {
+  overflow-x: auto;
+  max-width: 80%;
+  border: 1px solid rgb(222, 226, 230);
+  border-radius: 5px;
+  margin-bottom: 1rem;
+  padding-top: 1rem;
+  padding-bottom: 1rem;
+
+  // mermaid diagram - sequence diagram
+  .actor {
+    fill: #326ce5 !important;
+  }
+  text.actor {
+    font-size: 18px !important;
+    stroke: white !important;
+    fill: white !important;
+  }
+  .activation0 {
+    fill: #c9e9ec !important;
+  }
+}
+
 /* HEADER */

 .td-navbar {
@@ -268,22 +313,34 @@ main {

 // blockquotes and callouts

-blockquote {
-  padding: 0.4rem 0.4rem 0.4rem 1rem !important;
-}
+.td-content, body {
+  blockquote.callout {
+    padding: 0.4rem 0.4rem 0.4rem 1rem;
+    border: 1px solid #eee;
+    border-left-width: 0.5em;
+    background: #fff;
+    color: #000;
+    margin-top: 0.5em;
+    margin-bottom: 0.5em;
+  }
+  blockquote.callout {
+    border-radius: calc(1em/3);
+  }
+  .callout.caution {
+    border-left-color: #f0ad4e;
+  }

-// callouts are contained in static CSS as well. these require override.
+  .callout.note {
+    border-left-color: #428bca;
+  }

-.caution {
-  border-left-color: #f0ad4e !important;
-}
+  .callout.warning {
+    border-left-color: #d9534f;
+  }

-.note {
-  border-left-color: #428bca !important;
-}
-
-.warning {
-  border-left-color: #d9534f !important;
+  h1:first-of-type + blockquote.callout {
+    margin-top: 1.5em;
+  }
 }

 .deprecation-warning {
@@ -393,6 +450,7 @@ body.cid-community > #deprecation-warning > .deprecation-warning > * {
      }

      color: $blue;
+      margin: 1rem;
    }
  }
 }
@@ -512,3 +570,11 @@ body.td-documentation {
    }
  }
 }
+
+/* glossary tooltip */
+.glossary-tooltip {
+  display: inline-block;
+  border-bottom: 1px dotted black; /* If you want dots under the hoverable text */
+  color: black;
+  text-decoration: none !important;
+}
@@ -11,3 +11,17 @@ Add styles or override variables from the theme here. */
@import "base";
@import "tablet";
@import "desktop";
+
+$primary: #3371e3;
+
+// tooltip
+$tooltip-bg: #555;
+$tooltip-arrow-color: $tooltip-bg !default;
+$tooltip-arrow-width: 10px !default;
+$tooltip-opacity: 1 !default;
+$tooltip-color: #fff !default;
+$tooltip-max-width: 300px !default;
+$tooltip-font-size: 1rem;
+$tooltip-padding: 5px 8px;
+$tooltip-border-radius: 6px;
+$tooltip-font-weight: 400;
@@ -33,6 +33,23 @@ enableGitInfo = true
 # Hindi is disabled because it's currently in development.
 disableLanguages = ["hi", "no"]

+[caches]
+ [caches.assets]
+  dir = ":cacheDir/_gen"
+  maxAge = -1
+ [caches.getcsv]
+  dir = ":cacheDir/:project"
+  maxAge = "60s"
+ [caches.getjson]
+  dir = ":cacheDir/:project"
+  maxAge = "60s"
+ [caches.images]
+  dir = ":cacheDir/_images"
+  maxAge = -1
+ [caches.modules]
+  dir = ":cacheDir/modules"
+  maxAge = -1
+
 [markup]
 [markup.goldmark]
   [markup.goldmark.extensions]
@@ -66,6 +83,10 @@ date = ["date", ":filename", "publishDate", "lastmod"]
 [permalinks]
 blog = "/:section/:year/:month/:day/:slug/"

+[sitemap]
+  filename = "sitemap.xml"
+  priority = 0.75
+
 # Be explicit about the output formats. We (currently) only want an RSS feed for the home page.
 [outputs]
 home = [ "HTML", "RSS", "HEADERS" ]
@@ -112,24 +133,30 @@ copyright_linux =  "Copyright © 2020 The Linux Foundation ®."
 version_menu = "Versions"

 time_format_blog = "Monday, January 02, 2006"
+time_format_default = "January 02, 2006 at 3:04 PM PST"
+
 description = "Production-Grade Container Orchestration"
 showedit = true

-latest = "v1.18"
+latest = "v1.23"

-fullversion = "v1.18.0"
-version = "v1.18"
-githubbranch = "master"
-docsbranch = "master"
-deprecated = false
+fullversion = "v1.19.16"
+version = "v1.19"
+githubbranch = "v1.19.16"
+docsbranch = "release-1.19"
+deprecated = true
 currentUrl = "https://kubernetes.io/docs/home/"
 nextUrl = "https://kubernetes-io-vnext-staging.netlify.com/"
-githubWebsiteRepo = "github.com/kubernetes/website"
+
+# See codenew shortcode
 githubWebsiteRaw = "raw.githubusercontent.com/kubernetes/website"

+# GitHub repository link for editing a page and opening issues.
+github_repo = "https://github.com/kubernetes/website"
+
 # param for displaying an announcement block on every page.
 # See /i18n/en.toml for message text and title.
-announcement = true
+announcement = false
 announcement_bg = "#000000" #choose a dark color – text is white

 #Searching
@@ -149,47 +176,46 @@ offlineSearch = false
 [params.pushAssets]
 css = [
  "callouts",
-  "styles",
-  "custom-jekyll/tags"
+  "styles"
 ]
 js = [
  "script"
 ]

 [[params.versions]]
-fullversion = "v1.18.0"
-version = "v1.18"
-githubbranch = "v1.18.0"
-docsbranch = "release-1.18"
+fullversion = "v1.23.0"
+version = "v1.23"
+githubbranch = "v1.23.0"
+docsbranch = "main"
 url = "https://kubernetes.io"

 [[params.versions]]
-fullversion = "v1.17.4"
-version = "v1.17"
-githubbranch = "v1.17.4"
-docsbranch = "release-1.17"
-url = "https://v1-17.docs.kubernetes.io"
+fullversion = "v1.22.4"
+version = "v1.22"
+githubbranch = "v1.22.4"
+docsbranch = "release-1.22"
+url = "https://v1-22.docs.kubernetes.io"

 [[params.versions]]
-fullversion = "v1.16.8"
-version = "v1.16"
-githubbranch = "v1.16.8"
-docsbranch = "release-1.16"
-url = "https://v1-16.docs.kubernetes.io"
+fullversion = "v1.21.7"
+version = "v1.21"
+githubbranch = "v1.21.7"
+docsbranch = "release-1.21"
+url = "https://v1-21.docs.kubernetes.io"

 [[params.versions]]
-fullversion = "v1.15.11"
-version = "v1.15"
-githubbranch = "v1.15.11"
-docsbranch = "release-1.15"
-url = "https://v1-15.docs.kubernetes.io"
+fullversion = "v1.20.13"
+version = "v1.20"
+githubbranch = "v1.20.13"
+docsbranch = "release-1.20"
+url = "https://v1-20.docs.kubernetes.io"

 [[params.versions]]
-fullversion = "v1.14.10"
-version = "v1.14"
-githubbranch = "v1.14.10"
-docsbranch = "release-1.14"
-url = "https://v1-14.docs.kubernetes.io"
+fullversion = "v1.19.16"
+version = "v1.19"
+githubbranch = "v1.19.16"
+docsbranch = "release-1.19"
+url = "https://v1-19.docs.kubernetes.io"

 # User interface configuration
 [params.ui]
@@ -6,7 +6,7 @@ weight: 10

 <!-- overview -->

-Ein Knoten (Node in Englisch) ist eine Arbeitsmaschine in Kubernetes, früher als `minion` bekannt. Ein Node
+Ein Knoten (Node in Englisch) ist eine Arbeitsmaschine in Kubernetes. Ein Node
 kann je nach Cluster eine VM oder eine physische Maschine sein. Jeder Node enthält
 die für den Betrieb von [Pods](/docs/concepts/workloads/pods/pod/) notwendigen Dienste
 und wird von den Master-Komponenten verwaltet.
@@ -54,7 +54,8 @@ KUBECONFIG=~/.kube/config:~/.kube/kubconfig2 kubectl config view
 # Zeigen Sie das Passwort für den e2e-Benutzer an
 kubectl config view -o jsonpath='{.users[?(@.name == "e2e")].user.password}'

-kubectl config view -o jsonpath='{.users[].name}'    # eine Liste der Benutzer erhalten
+kubectl config view -o jsonpath='{.users[].name}'    # den ersten Benutzer anzeigen
+kubectl config view -o jsonpath='{.users[*].name}'    # eine Liste der Benutzer erhalten
 kubectl config current-context                       # den aktuellen Kontext anzeigen
 kubectl config use-context my-cluster-name           # Setzen Sie den Standardkontext auf my-cluster-name

@@ -334,7 +334,7 @@ Sie müssen nun sicherstellen, dass das kubectl-Abschlussskript in allen Ihren S
    ```

 {{< note >}}
-bash-completion bezieht alle Verfollständigungsskripte aus `/etc/bash_completion.d`.
+bash-completion bezieht alle Vervollständigungsskripte aus `/etc/bash_completion.d`.
 {{< /note >}}

 Beide Ansätze sind gleichwertig. Nach dem erneuten Laden der Shell sollte kubectl autocompletion funktionieren.
@@ -69,7 +69,7 @@ sudo mv minikube /usr/local/bin
 ### Linux

 {{< note >}}
-Dieses Dokument zeigt Ihnen, wie Sie Minikube mit einer statischen Binärdatei unter Linux installieren. Für alternative Linux-Installationsmethoden siehe [Andere Installationsmethoden](https://github.com/kubernetes/minikube#other-ways-to-install) im offiziellen Minikube-GitHub-Repository.
+Dieses Dokument zeigt Ihnen, wie Sie Minikube mit einer statischen Binärdatei unter Linux installieren. Für alternative Linux-Installationsmethoden siehe [Andere Installationsmethoden](https://minikube.sigs.k8s.io/docs/start/) im offiziellen Minikube-GitHub-Repository.
 {{< /note >}}

 Sie können Minikube unter Linux installieren, indem Sie eine statische Binärdatei herunterladen:
@@ -21,7 +21,7 @@ weight: 20
            <div class="katacoda__alert">
                Um mit dem Terminal zu interagieren, verwenden Sie bitte die Desktop- / Tablet-Version
            </div>
-            <div class="katacoda__box" id="inline-terminal-1"  data-katacoda-id="kubernetes-bootcamp/1" data-katacoda-color="326de6" data-katacoda-secondary="273d6d" data-katacoda-hideintro="false" data-katacoda-font="Roboto" data-katacoda-fontheader="Roboto Slab" data-katacoda-prompt="Kubernetes Bootcamp Terminal" style="height: 600px;"></div>
+            <div class="katacoda__box" id="inline-terminal-1"  data-katacoda-id="kubernetes-bootcamp/1" data-katacoda-color="326de6" data-katacoda-secondary="273d6d" data-katacoda-hideintro="false" data-katacoda-prompt="Kubernetes Bootcamp Terminal" style="height: 600px;"></div>
        </div>
        <div class="row">
            <div class="col-md-12">
@@ -23,7 +23,7 @@ weight: 20
                Um mit dem Terminal zu interagieren, verwenden Sie bitte die Desktop- / Tablet-Version
            </div>

-            <div class="katacoda__box" id="inline-terminal-1"  data-katacoda-id="kubernetes-bootcamp/7" data-katacoda-color="326de6" data-katacoda-secondary="273d6d" data-katacoda-hideintro="false" data-katacoda-font="Roboto" data-katacoda-fontheader="Roboto Slab" data-katacoda-prompt="Kubernetes Bootcamp Terminal" style="height: 600px;">
+            <div class="katacoda__box" id="inline-terminal-1"  data-katacoda-id="kubernetes-bootcamp/7" data-katacoda-color="326de6" data-katacoda-secondary="273d6d" data-katacoda-hideintro="false" data-katacoda-prompt="Kubernetes Bootcamp Terminal" style="height: 600px;">
            </div>

        </div>
@@ -24,7 +24,7 @@ weight: 20
                Um mit dem Terminal zu interagieren, verwenden Sie bitte die Desktop- / Tablet-Version
            </div>

-            <div class="katacoda__box" id="inline-terminal-1"  data-katacoda-id="kubernetes-bootcamp/4" data-katacoda-color="326de6" data-katacoda-secondary="273d6d" data-katacoda-hideintro="false" data-katacoda-font="Roboto" data-katacoda-fontheader="Roboto Slab" data-katacoda-prompt="Kubernetes Bootcamp Terminal" style="height: 600px;">
+            <div class="katacoda__box" id="inline-terminal-1"  data-katacoda-id="kubernetes-bootcamp/4" data-katacoda-color="326de6" data-katacoda-secondary="273d6d" data-katacoda-hideintro="false" data-katacoda-prompt="Kubernetes Bootcamp Terminal" style="height: 600px;">
            </div>
        </div>
        <div class="row">
@@ -21,7 +21,7 @@ weight: 20
            <div class="katacoda__alert">
                Um mit dem Terminal zu interagieren, verwenden Sie bitte die Desktop- / Tablet-Version
            </div>
-            <div class="katacoda__box" id="inline-terminal-1"  data-katacoda-id="kubernetes-bootcamp/8" data-katacoda-color="326de6" data-katacoda-secondary="273d6d" data-katacoda-hideintro="false" data-katacoda-font="Roboto" data-katacoda-fontheader="Roboto Slab" data-katacoda-prompt="Kubernetes Bootcamp Terminal" style="height: 600px;">
+            <div class="katacoda__box" id="inline-terminal-1"  data-katacoda-id="kubernetes-bootcamp/8" data-katacoda-color="326de6" data-katacoda-secondary="273d6d" data-katacoda-hideintro="false" data-katacoda-prompt="Kubernetes Bootcamp Terminal" style="height: 600px;">
            </div>
        </div>
        <div class="row">
@@ -21,7 +21,7 @@ weight: 20
            <div class="katacoda__alert">
                Um mit dem Terminal zu interagieren, verwenden Sie bitte die Desktop- / Tablet-Version
            </div>
-            <div class="katacoda__box" id="inline-terminal-1"  data-katacoda-id="kubernetes-bootcamp/5" data-katacoda-color="326de6" data-katacoda-secondary="273d6d" data-katacoda-hideintro="false" data-katacoda-font="Roboto" data-katacoda-fontheader="Roboto Slab" data-katacoda-prompt="Kubernetes Bootcamp Terminal" style="height: 600px;">
+            <div class="katacoda__box" id="inline-terminal-1"  data-katacoda-id="kubernetes-bootcamp/5" data-katacoda-color="326de6" data-katacoda-secondary="273d6d" data-katacoda-hideintro="false" data-katacoda-prompt="Kubernetes Bootcamp Terminal" style="height: 600px;">
            </div>
        </div>
        <div class="row">
@@ -21,7 +21,7 @@ weight: 20
            <div class="katacoda__alert">
                Um mit dem Terminal zu interagieren, verwenden Sie bitte die Desktop- / Tablet-Version
            </div>
-            <div class="katacoda__box" id="inline-terminal-1"  data-katacoda-id="kubernetes-bootcamp/6" data-katacoda-color="326de6" data-katacoda-secondary="273d6d" data-katacoda-hideintro="false" data-katacoda-font="Roboto" data-katacoda-fontheader="Roboto Slab" data-katacoda-prompt="Kubernetes Bootcamp Terminal" style="height: 600px;">
+            <div class="katacoda__box" id="inline-terminal-1"  data-katacoda-id="kubernetes-bootcamp/6" data-katacoda-color="326de6" data-katacoda-secondary="273d6d" data-katacoda-hideintro="false" data-katacoda-prompt="Kubernetes Bootcamp Terminal" style="height: 600px;">
            </div>
        </div>
        <div class="row">
@@ -2,11 +2,13 @@
 title: "Production-Grade Container Orchestration"
 abstract: "Automated container deployment, scaling, and management"
 cid: home
+sitemap:
+  priority: 1.0
 ---

 {{< blocks/section id="oceanNodes" >}}
 {{% blocks/feature image="flower" %}}
-### [Kubernetes (K8s)]({{< relref "/docs/concepts/overview/what-is-kubernetes" >}}) is an open-source system for automating deployment, scaling, and management of containerized applications.
+[Kubernetes]({{< relref "/docs/concepts/overview/what-is-kubernetes" >}}), also known as K8s, is an open-source system for automating deployment, scaling, and management of containerized applications.

 It groups containers that make up an application into logical units for easy management and discovery. Kubernetes builds upon [15 years of experience of running production workloads at Google](http://queue.acm.org/detail.cfm?id=2898444), combined with best-of-breed ideas and practices from the community.
 {{% /blocks/feature %}}
@@ -26,7 +28,7 @@ Whether testing locally or running a global enterprise, Kubernetes flexibility g
 {{% /blocks/feature %}}

 {{% blocks/feature image="suitcase" %}}
-#### Run Anywhere
+#### Run K8s Anywhere

 Kubernetes is open source giving you the freedom to take advantage of on-premises, hybrid, or public cloud infrastructure, letting you effortlessly move workloads to where it matters to you.

@@ -41,12 +43,12 @@ Kubernetes is open source giving you the freedom to take advantage of on-premise
        <button id="desktopShowVideoButton" onclick="kub.showVideo()">Watch Video</button>
        <br>
        <br>
-        <a href="https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/?utm_source=kubernetes.io&utm_medium=nav&utm_campaign=kccnceu20" button id="desktopKCButton">Attend KubeCon EU virtually on August 17-20, 2020</a>
-        <br>
-        <br>
-        <br>
-        <br>
        <a href="https://events.linuxfoundation.org/kubecon-cloudnativecon-north-america/?utm_source=kubernetes.io&utm_medium=nav&utm_campaign=kccncna20" button id="desktopKCButton">Attend KubeCon NA virtually on November 17-20, 2020</a>
+        <br>
+        <br>
+        <br>
+        <br>
+        <a href="https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/?utm_source=kubernetes.io&utm_medium=nav&utm_campaign=kccnceu21" button id="desktopKCButton">Attend KubeCon EU virtually on May 4 – 7, 2021</a>
 </div>
 <div id="videoPlayer">
    <iframe data-url="https://www.youtube.com/embed/H06qrNmGqyE?autoplay=1" frameborder="0" allowfullscreen></iframe>
@@ -9,57 +9,37 @@ url: /blog/2016/01/Kubernetes-Community-Meeting-Notes_28

 Note taker: Rob Hirshfeld
  - Use Case (10 min): [SFDC Paul Brown](https://docs.google.com/a/google.com/presentation/d/1MEI97efplr3f-GDX1GcWGfkEuGKKV-4niu27kHOeMLk/edit?usp=sharing_eid&ts=56a114f8)
-  - SIG Report - SIG-config and the story of [#18215](https://github.com/kubernetes/kubernetes/pull/18215)
+  
+  - SIG Report - SIG-config and the story of [#18215](https://github.com/kubernetes/kubernetes/pull/18215).
+    - Application config IN K8s not deployment of K8s
+    - Topic has been reuse of configuration,specifically parameterization(aka templates). Needs:
+      - include scoping(cluster namespace)
+      - slight customization (naming changes, but not major config)
+      - multiple positions on how todo this including allowing external or simple extensions
+    - PetSet creates instances w/stable namespace

-    - A[p](https://github.com/kubernetes/kubernetes/pull/18215)p[l](https://github.com/kubernetes/kubernetes/pull/18215)i[c](https://github.com/kubernetes/kubernetes/pull/18215)a[t](https://github.com/kubernetes/kubernetes/pull/18215)i[o](https://github.com/kubernetes/kubernetes/pull/18215)n[](https://github.com/kubernetes/kubernetes/pull/18215)c[o](https://github.com/kubernetes/kubernetes/pull/18215)n[f](https://github.com/kubernetes/kubernetes/pull/18215)i[g](https://github.com/kubernetes/kubernetes/pull/18215) [I](https://github.com/kubernetes/kubernetes/pull/18215)N[](https://github.com/kubernetes/kubernetes/pull/18215)K[8](https://github.com/kubernetes/kubernetes/pull/18215)s[,](https://github.com/kubernetes/kubernetes/pull/18215) [n](https://github.com/kubernetes/kubernetes/pull/18215)o[t](https://github.com/kubernetes/kubernetes/pull/18215) [d](https://github.com/kubernetes/kubernetes/pull/18215)e[p](https://github.com/kubernetes/kubernetes/pull/18215)l[o](https://github.com/kubernetes/kubernetes/pull/18215)y[m](https://github.com/kubernetes/kubernetes/pull/18215)e[n](https://github.com/kubernetes/kubernetes/pull/18215)t[](https://github.com/kubernetes/kubernetes/pull/18215)o[f](https://github.com/kubernetes/kubernetes/pull/18215) [K](https://github.com/kubernetes/kubernetes/pull/18215)8[s](https://github.com/kubernetes/kubernetes/pull/18215)
-    - [T](https://github.com/kubernetes/kubernetes/pull/18215)o[p](https://github.com/kubernetes/kubernetes/pull/18215)i[c](https://github.com/kubernetes/kubernetes/pull/18215) [h](https://github.com/kubernetes/kubernetes/pull/18215)a[s](https://github.com/kubernetes/kubernetes/pull/18215) [b](https://github.com/kubernetes/kubernetes/pull/18215)e[e](https://github.com/kubernetes/kubernetes/pull/18215)n[](https://github.com/kubernetes/kubernetes/pull/18215)r[e](https://github.com/kubernetes/kubernetes/pull/18215)u[s](https://github.com/kubernetes/kubernetes/pull/18215)e[](https://github.com/kubernetes/kubernetes/pull/18215)o[f](https://github.com/kubernetes/kubernetes/pull/18215) [c](https://github.com/kubernetes/kubernetes/pull/18215)o[n](https://github.com/kubernetes/kubernetes/pull/18215)f[i](https://github.com/kubernetes/kubernetes/pull/18215)g[u](https://github.com/kubernetes/kubernetes/pull/18215)r[a](https://github.com/kubernetes/kubernetes/pull/18215)t[i](https://github.com/kubernetes/kubernetes/pull/18215)o[n](https://github.com/kubernetes/kubernetes/pull/18215),[](https://github.com/kubernetes/kubernetes/pull/18215)s[p](https://github.com/kubernetes/kubernetes/pull/18215)e[c](https://github.com/kubernetes/kubernetes/pull/18215)i[f](https://github.com/kubernetes/kubernetes/pull/18215)i[c](https://github.com/kubernetes/kubernetes/pull/18215)a[l](https://github.com/kubernetes/kubernetes/pull/18215)l[y](https://github.com/kubernetes/kubernetes/pull/18215) [p](https://github.com/kubernetes/kubernetes/pull/18215)a[r](https://github.com/kubernetes/kubernetes/pull/18215)a[m](https://github.com/kubernetes/kubernetes/pull/18215)e[t](https://github.com/kubernetes/kubernetes/pull/18215)e[r](https://github.com/kubernetes/kubernetes/pull/18215)i[z](https://github.com/kubernetes/kubernetes/pull/18215)a[t](https://github.com/kubernetes/kubernetes/pull/18215)i[o](https://github.com/kubernetes/kubernetes/pull/18215)n[](https://github.com/kubernetes/kubernetes/pull/18215)([a](https://github.com/kubernetes/kubernetes/pull/18215)k[a](https://github.com/kubernetes/kubernetes/pull/18215) [t](https://github.com/kubernetes/kubernetes/pull/18215)e[m](https://github.com/kubernetes/kubernetes/pull/18215)p[l](https://github.com/kubernetes/kubernetes/pull/18215)a[t](https://github.com/kubernetes/kubernetes/pull/18215)e[s](https://github.com/kubernetes/kubernetes/pull/18215))[. Needs:](https://github.com/kubernetes/kubernetes/pull/18215)
+  - Workflow proposal
+    - Distributed Chron. Challenge is that configs need to create multiple objects in sequence
+    - Trying to figure out how balance the many config options out there (compose, terraform,ansible/etc)
+    - Goal is to “meet people where they are” to keep it simple
+    - Q: is there an opinion for the keystore sizing
+      - large size / data blob would not be appropriate
+      - you can pull data(config) from another store for larger objects

-      - n[e](https://github.com/kubernetes/kubernetes/pull/18215)e[d](https://github.com/kubernetes/kubernetes/pull/18215)s[](https://github.com/kubernetes/kubernetes/pull/18215)i[n](https://github.com/kubernetes/kubernetes/pull/18215)c[l](https://github.com/kubernetes/kubernetes/pull/18215)u[d](https://github.com/kubernetes/kubernetes/pull/18215)e[](https://github.com/kubernetes/kubernetes/pull/18215)s[c](https://github.com/kubernetes/kubernetes/pull/18215)o[p](https://github.com/kubernetes/kubernetes/pull/18215)i[n](https://github.com/kubernetes/kubernetes/pull/18215)g[](https://github.com/kubernetes/kubernetes/pull/18215)([c](https://github.com/kubernetes/kubernetes/pull/18215)l[u](https://github.com/kubernetes/kubernetes/pull/18215)s[t](https://github.com/kubernetes/kubernetes/pull/18215)e[r](https://github.com/kubernetes/kubernetes/pull/18215) [n](https://github.com/kubernetes/kubernetes/pull/18215)a[m](https://github.com/kubernetes/kubernetes/pull/18215)e[s](https://github.com/kubernetes/kubernetes/pull/18215)p[a](https://github.com/kubernetes/kubernetes/pull/18215)c[e](https://github.com/kubernetes/kubernetes/pull/18215))
-      - s[l](https://github.com/kubernetes/kubernetes/pull/18215)i[g](https://github.com/kubernetes/kubernetes/pull/18215)h[t](https://github.com/kubernetes/kubernetes/pull/18215) [c](https://github.com/kubernetes/kubernetes/pull/18215)u[s](https://github.com/kubernetes/kubernetes/pull/18215)t[o](https://github.com/kubernetes/kubernetes/pull/18215)m[i](https://github.com/kubernetes/kubernetes/pull/18215)z[a](https://github.com/kubernetes/kubernetes/pull/18215)t[i](https://github.com/kubernetes/kubernetes/pull/18215)o[n (naming changes, but not major config)](https://github.com/kubernetes/kubernetes/pull/18215)
-      - [m](https://github.com/kubernetes/kubernetes/pull/18215)u[l](https://github.com/kubernetes/kubernetes/pull/18215)t[i](https://github.com/kubernetes/kubernetes/pull/18215)p[l](https://github.com/kubernetes/kubernetes/pull/18215)e[](https://github.com/kubernetes/kubernetes/pull/18215)p[o](https://github.com/kubernetes/kubernetes/pull/18215)s[i](https://github.com/kubernetes/kubernetes/pull/18215)t[i](https://github.com/kubernetes/kubernetes/pull/18215)o[n](https://github.com/kubernetes/kubernetes/pull/18215)s[](https://github.com/kubernetes/kubernetes/pull/18215)o[n](https://github.com/kubernetes/kubernetes/pull/18215) [h](https://github.com/kubernetes/kubernetes/pull/18215)o[w](https://github.com/kubernetes/kubernetes/pull/18215) [t](https://github.com/kubernetes/kubernetes/pull/18215)o[](https://github.com/kubernetes/kubernetes/pull/18215)d[o](https://github.com/kubernetes/kubernetes/pull/18215) [t](https://github.com/kubernetes/kubernetes/pull/18215)h[i](https://github.com/kubernetes/kubernetes/pull/18215)s[](https://github.com/kubernetes/kubernetes/pull/18215)i[n](https://github.com/kubernetes/kubernetes/pull/18215)c[l](https://github.com/kubernetes/kubernetes/pull/18215)u[d](https://github.com/kubernetes/kubernetes/pull/18215)i[n](https://github.com/kubernetes/kubernetes/pull/18215)g[](https://github.com/kubernetes/kubernetes/pull/18215)
-
-        -
-a[l](https://github.com/kubernetes/kubernetes/pull/18215)l[o](https://github.com/kubernetes/kubernetes/pull/18215)w[i](https://github.com/kubernetes/kubernetes/pull/18215)n[g](https://github.com/kubernetes/kubernetes/pull/18215) [e](https://github.com/kubernetes/kubernetes/pull/18215)x[t](https://github.com/kubernetes/kubernetes/pull/18215)e[r](https://github.com/kubernetes/kubernetes/pull/18215)n[a](https://github.com/kubernetes/kubernetes/pull/18215)l[](https://github.com/kubernetes/kubernetes/pull/18215)o[r](https://github.com/kubernetes/kubernetes/pull/18215) [s](https://github.com/kubernetes/kubernetes/pull/18215)i[m](https://github.com/kubernetes/kubernetes/pull/18215)p[l](https://github.com/kubernetes/kubernetes/pull/18215)e[](https://github.com/kubernetes/kubernetes/pull/18215)e[x](https://github.com/kubernetes/kubernetes/pull/18215)t[e](https://github.com/kubernetes/kubernetes/pull/18215)n[s](https://github.com/kubernetes/kubernetes/pull/18215)i[o](https://github.com/kubernetes/kubernetes/pull/18215)n[s](https://github.com/kubernetes/kubernetes/pull/18215).
-        -
-P[e](https://github.com/kubernetes/kubernetes/pull/18215)t[S](https://github.com/kubernetes/kubernetes/pull/18215)e[t](https://github.com/kubernetes/kubernetes/pull/18215) [c](https://github.com/kubernetes/kubernetes/pull/18215)r[e](https://github.com/kubernetes/kubernetes/pull/18215)a[t](https://github.com/kubernetes/kubernetes/pull/18215)e[s](https://github.com/kubernetes/kubernetes/pull/18215) [i](https://github.com/kubernetes/kubernetes/pull/18215)n[s](https://github.com/kubernetes/kubernetes/pull/18215)t[a](https://github.com/kubernetes/kubernetes/pull/18215)n[c](https://github.com/kubernetes/kubernetes/pull/18215)e[s](https://github.com/kubernetes/kubernetes/pull/18215) [w](https://github.com/kubernetes/kubernetes/pull/18215)/[](https://github.com/kubernetes/kubernetes/pull/18215)s[t](https://github.com/kubernetes/kubernetes/pull/18215)a[b](https://github.com/kubernetes/kubernetes/pull/18215)l[e](https://github.com/kubernetes/kubernetes/pull/18215) [n](https://github.com/kubernetes/kubernetes/pull/18215)a[m](https://github.com/kubernetes/kubernetes/pull/18215)e[s](https://github.com/kubernetes/kubernetes/pull/18215)p[a](https://github.com/kubernetes/kubernetes/pull/18215)c[e](https://github.com/kubernetes/kubernetes/pull/18215)
-        -
-[W](https://github.com/kubernetes/kubernetes/pull/18215)o[r](https://github.com/kubernetes/kubernetes/pull/18215)k[f](https://github.com/kubernetes/kubernetes/pull/18215)l[o](https://github.com/kubernetes/kubernetes/pull/18215)w[](https://github.com/kubernetes/kubernetes/pull/18215)p[r](https://github.com/kubernetes/kubernetes/pull/18215)o[p](https://github.com/kubernetes/kubernetes/pull/18215)o[s](https://github.com/kubernetes/kubernetes/pull/18215)a[l](https://github.com/kubernetes/kubernetes/pull/18215)
-        -
-[D](https://github.com/kubernetes/kubernetes/pull/18215)i[s](https://github.com/kubernetes/kubernetes/pull/18215)t[r](https://github.com/kubernetes/kubernetes/pull/18215)i[b](https://github.com/kubernetes/kubernetes/pull/18215)u[t](https://github.com/kubernetes/kubernetes/pull/18215)e[d](https://github.com/kubernetes/kubernetes/pull/18215) [C](https://github.com/kubernetes/kubernetes/pull/18215)h[r](https://github.com/kubernetes/kubernetes/pull/18215)o[n](https://github.com/kubernetes/kubernetes/pull/18215).
-      -
-C[h](https://github.com/kubernetes/kubernetes/pull/18215)a[l](https://github.com/kubernetes/kubernetes/pull/18215)l[e](https://github.com/kubernetes/kubernetes/pull/18215)n[g](https://github.com/kubernetes/kubernetes/pull/18215)e[](https://github.com/kubernetes/kubernetes/pull/18215)i[s](https://github.com/kubernetes/kubernetes/pull/18215) [t](https://github.com/kubernetes/kubernetes/pull/18215)h[a](https://github.com/kubernetes/kubernetes/pull/18215)t[](https://github.com/kubernetes/kubernetes/pull/18215)c[o](https://github.com/kubernetes/kubernetes/pull/18215)n[f](https://github.com/kubernetes/kubernetes/pull/18215)i[g](https://github.com/kubernetes/kubernetes/pull/18215)s[](https://github.com/kubernetes/kubernetes/pull/18215)n[e](https://github.com/kubernetes/kubernetes/pull/18215)e[d](https://github.com/kubernetes/kubernetes/pull/18215) [t](https://github.com/kubernetes/kubernetes/pull/18215)o[](https://github.com/kubernetes/kubernetes/pull/18215)c[r](https://github.com/kubernetes/kubernetes/pull/18215)e[a](https://github.com/kubernetes/kubernetes/pull/18215)t[e](https://github.com/kubernetes/kubernetes/pull/18215) [m](https://github.com/kubernetes/kubernetes/pull/18215)u[l](https://github.com/kubernetes/kubernetes/pull/18215)t[i](https://github.com/kubernetes/kubernetes/pull/18215)p[l](https://github.com/kubernetes/kubernetes/pull/18215)e[](https://github.com/kubernetes/kubernetes/pull/18215)o[b](https://github.com/kubernetes/kubernetes/pull/18215)j[e](https://github.com/kubernetes/kubernetes/pull/18215)c[t](https://github.com/kubernetes/kubernetes/pull/18215)s[](https://github.com/kubernetes/kubernetes/pull/18215)i[n](https://github.com/kubernetes/kubernetes/pull/18215) [s](https://github.com/kubernetes/kubernetes/pull/18215)e[q](https://github.com/kubernetes/kubernetes/pull/18215)u[e](https://github.com/kubernetes/kubernetes/pull/18215)n[c](https://github.com/kubernetes/kubernetes/pull/18215)e
-      -
-T[r](https://github.com/kubernetes/kubernetes/pull/18215)y[i](https://github.com/kubernetes/kubernetes/pull/18215)n[g](https://github.com/kubernetes/kubernetes/pull/18215) [t](https://github.com/kubernetes/kubernetes/pull/18215)o[](https://github.com/kubernetes/kubernetes/pull/18215)f[i](https://github.com/kubernetes/kubernetes/pull/18215)g[u](https://github.com/kubernetes/kubernetes/pull/18215)r[e](https://github.com/kubernetes/kubernetes/pull/18215) [o](https://github.com/kubernetes/kubernetes/pull/18215)u[t](https://github.com/kubernetes/kubernetes/pull/18215) [h](https://github.com/kubernetes/kubernetes/pull/18215)o[w](https://github.com/kubernetes/kubernetes/pull/18215) [b](https://github.com/kubernetes/kubernetes/pull/18215)a[l](https://github.com/kubernetes/kubernetes/pull/18215)a[n](https://github.com/kubernetes/kubernetes/pull/18215)c[e](https://github.com/kubernetes/kubernetes/pull/18215) [t](https://github.com/kubernetes/kubernetes/pull/18215)h[e](https://github.com/kubernetes/kubernetes/pull/18215) [m](https://github.com/kubernetes/kubernetes/pull/18215)a[n](https://github.com/kubernetes/kubernetes/pull/18215)y[](https://github.com/kubernetes/kubernetes/pull/18215)c[o](https://github.com/kubernetes/kubernetes/pull/18215)n[f](https://github.com/kubernetes/kubernetes/pull/18215)i[g](https://github.com/kubernetes/kubernetes/pull/18215) [o](https://github.com/kubernetes/kubernetes/pull/18215)p[t](https://github.com/kubernetes/kubernetes/pull/18215)i[o](https://github.com/kubernetes/kubernetes/pull/18215)n[s](https://github.com/kubernetes/kubernetes/pull/18215) [o](https://github.com/kubernetes/kubernetes/pull/18215)u[t](https://github.com/kubernetes/kubernetes/pull/18215) [t](https://github.com/kubernetes/kubernetes/pull/18215)h[e](https://github.com/kubernetes/kubernetes/pull/18215)r[e](https://github.com/kubernetes/kubernetes/pull/18215) [(](https://github.com/kubernetes/kubernetes/pull/18215)c[o](https://github.com/kubernetes/kubernetes/pull/18215)m[p](https://github.com/kubernetes/kubernetes/pull/18215)o[s](https://github.com/kubernetes/kubernetes/pull/18215)e[,](https://github.com/kubernetes/kubernetes/pull/18215) [t](https://github.com/kubernetes/kubernetes/pull/18215)e[r](https://github.com/kubernetes/kubernetes/pull/18215)r[a](https://github.com/kubernetes/kubernetes/pull/18215)f[o](https://github.com/kubernetes/kubernetes/pull/18215)r[m](https://github.com/kubernetes/kubernetes/pull/18215),[](https://github.com/kubernetes/kubernetes/pull/18215)a[n](https://github.com/kubernetes/kubernetes/pull/18215)s[i](https://github.com/kubernetes/kubernetes/pull/18215)b[l](https://github.com/kubernetes/kubernetes/pull/18215)e[/](https://github.com/kubernetes/kubernetes/pull/18215)e[t](https://github.com/kubernetes/kubernetes/pull/18215)c[)](https://github.com/kubernetes/kubernetes/pull/18215)
-      -
-[G](https://github.com/kubernetes/kubernetes/pull/18215)o[a](https://github.com/kubernetes/kubernetes/pull/18215)l[](https://github.com/kubernetes/kubernetes/pull/18215)i[s](https://github.com/kubernetes/kubernetes/pull/18215) [t](https://github.com/kubernetes/kubernetes/pull/18215)o[](https://github.com/kubernetes/kubernetes/pull/18215)“[m](https://github.com/kubernetes/kubernetes/pull/18215)e[e](https://github.com/kubernetes/kubernetes/pull/18215)t[](https://github.com/kubernetes/kubernetes/pull/18215)p[e](https://github.com/kubernetes/kubernetes/pull/18215)o[p](https://github.com/kubernetes/kubernetes/pull/18215)l[e](https://github.com/kubernetes/kubernetes/pull/18215) [w](https://github.com/kubernetes/kubernetes/pull/18215)h[e](https://github.com/kubernetes/kubernetes/pull/18215)r[e](https://github.com/kubernetes/kubernetes/pull/18215) [t](https://github.com/kubernetes/kubernetes/pull/18215)h[e](https://github.com/kubernetes/kubernetes/pull/18215)y[](https://github.com/kubernetes/kubernetes/pull/18215)a[r](https://github.com/kubernetes/kubernetes/pull/18215)e[”](https://github.com/kubernetes/kubernetes/pull/18215) [t](https://github.com/kubernetes/kubernetes/pull/18215)o[](https://github.com/kubernetes/kubernetes/pull/18215)k[e](https://github.com/kubernetes/kubernetes/pull/18215)e[p](https://github.com/kubernetes/kubernetes/pull/18215) [i](https://github.com/kubernetes/kubernetes/pull/18215)t[](https://github.com/kubernetes/kubernetes/pull/18215)s[i](https://github.com/kubernetes/kubernetes/pull/18215)m[p](https://github.com/kubernetes/kubernetes/pull/18215)l[e](https://github.com/kubernetes/kubernetes/pull/18215)
-      -
-[Q](https://github.com/kubernetes/kubernetes/pull/18215): is there an opinion for the keystore sizing
-
-        -
-large size / data blob would not be appropriate
-        -
-you can pull data(config) from another store for larger objects
-  -
-[SIG Report - SIG-federation - progress on Ubernetes-Lite & Ubernetes design](https://github.com/kubernetes/kubernetes/pull/18215)
-
-    -
-[G](https://github.com/kubernetes/kubernetes/pull/18215)o[a](https://github.com/kubernetes/kubernetes/pull/18215)l[](https://github.com/kubernetes/kubernetes/pull/18215)i[t](https://github.com/kubernetes/kubernetes/pull/18215) [t](https://github.com/kubernetes/kubernetes/pull/18215)o[](https://github.com/kubernetes/kubernetes/pull/18215)b[e](https://github.com/kubernetes/kubernetes/pull/18215) [a](https://github.com/kubernetes/kubernetes/pull/18215)b[l](https://github.com/kubernetes/kubernetes/pull/18215)e[](https://github.com/kubernetes/kubernetes/pull/18215)t[o](https://github.com/kubernetes/kubernetes/pull/18215) [h](https://github.com/kubernetes/kubernetes/pull/18215)a[v](https://github.com/kubernetes/kubernetes/pull/18215)e[](https://github.com/kubernetes/kubernetes/pull/18215)a[](https://github.com/kubernetes/kubernetes/pull/18215)c[l](https://github.com/kubernetes/kubernetes/pull/18215)u[s](https://github.com/kubernetes/kubernetes/pull/18215)t[e](https://github.com/kubernetes/kubernetes/pull/18215)r[](https://github.com/kubernetes/kubernetes/pull/18215)m[a](https://github.com/kubernetes/kubernetes/pull/18215)n[a](https://github.com/kubernetes/kubernetes/pull/18215)g[e](https://github.com/kubernetes/kubernetes/pull/18215)r[,](https://github.com/kubernetes/kubernetes/pull/18215) [s](https://github.com/kubernetes/kubernetes/pull/18215)o[](https://github.com/kubernetes/kubernetes/pull/18215)y[o](https://github.com/kubernetes/kubernetes/pull/18215)u[](https://github.com/kubernetes/kubernetes/pull/18215)c[a](https://github.com/kubernetes/kubernetes/pull/18215)n[](https://github.com/kubernetes/kubernetes/pull/18215)f[e](https://github.com/kubernetes/kubernetes/pull/18215)d[e](https://github.com/kubernetes/kubernetes/pull/18215)r[a](https://github.com/kubernetes/kubernetes/pull/18215)t[e](https://github.com/kubernetes/kubernetes/pull/18215) [c](https://github.com/kubernetes/kubernetes/pull/18215)l[u](https://github.com/kubernetes/kubernetes/pull/18215)s[t](https://github.com/kubernetes/kubernetes/pull/18215)e[r](https://github.com/kubernetes/kubernetes/pull/18215)s[. &nbsp;They will automatically distribute the pods.](https://github.com/kubernetes/kubernetes/pull/18215)
-    -
-P[l](https://github.com/kubernetes/kubernetes/pull/18215)a[n](https://github.com/kubernetes/kubernetes/pull/18215) [i](https://github.com/kubernetes/kubernetes/pull/18215)s[](https://github.com/kubernetes/kubernetes/pull/18215)t[o](https://github.com/kubernetes/kubernetes/pull/18215) [u](https://github.com/kubernetes/kubernetes/pull/18215)s[e](https://github.com/kubernetes/kubernetes/pull/18215) [t](https://github.com/kubernetes/kubernetes/pull/18215)h[e](https://github.com/kubernetes/kubernetes/pull/18215) [s](https://github.com/kubernetes/kubernetes/pull/18215)a[m](https://github.com/kubernetes/kubernetes/pull/18215)e[](https://github.com/kubernetes/kubernetes/pull/18215)A[P](https://github.com/kubernetes/kubernetes/pull/18215)I[](https://github.com/kubernetes/kubernetes/pull/18215)f[o](https://github.com/kubernetes/kubernetes/pull/18215)r[](https://github.com/kubernetes/kubernetes/pull/18215)t[h](https://github.com/kubernetes/kubernetes/pull/18215)e[](https://github.com/kubernetes/kubernetes/pull/18215)m[a](https://github.com/kubernetes/kubernetes/pull/18215)s[t](https://github.com/kubernetes/kubernetes/pull/18215)e[r](https://github.com/kubernetes/kubernetes/pull/18215) [c](https://github.com/kubernetes/kubernetes/pull/18215)l[u](https://github.com/kubernetes/kubernetes/pull/18215)s[te](https://github.com/kubernetes/kubernetes/pull/18215)r
-    - Quinton's Ubernetes Talk: &nbsp;https://youtu.be/L2ZK24JojB4
-    - Design for Ubernetes: https://github.com/kubernetes/kubernetes/pull/19313
+  - SIG Report - SIG-federation - progress on Ubernetes-Lite & Ubernetes design
+   - Goal is to be able to have a cluster manager, so you can federate clusters. They will automatically distribute the pods.
+   - Plan is to use the same API for the master cluster
+   - [Quinton's Kubernetes Talk](https://youtu.be/L2ZK24JojB4)
+   - [Design for Kubernetes:](https://github.com/kubernetes/kubernetes/pull/19313)


-  - Conformance testing Q+A [Isaac Hollander McCreery]
-
+  - Conformance testing Q+A Isaac Hollander McCreery
    - status on conformance testing for release process
-
-      - expect to be forward compatible but not backwards
+    - expect to be forward compatible but not backwards
    - is there interest for a sig-testing meeting
    - testing needs to a higher priority for the project
    - lots of focus on trying to make this a higher priority
-To get involved in the Kubernetes community consider joining our [Slack channel](http://slack.k8s.io/), taking a look at the [Kubernetes project](https://github.com/kubernetes/) on GitHub, or join the [Kubernetes-dev Google group](https://groups.google.com/forum/#!forum/kubernetes-dev). If you’re really excited, you can do all of the above and join us for the next community conversation -- January 27th, 2016. Please add yourself or a topic you want to know about to the [agenda](https://docs.google.com/document/d/1VQDIAB0OqiSjIHI8AWMvSdceWhnz56jNpZrLs6o7NJY/edit) and get a calendar invitation by joining [this group](https://groups.google.com/forum/#!forum/kubernetes-community-video-chat).  
-
+To get involved in the Kubernetes community consider joining our [Slack channel](http://slack.k8s.io/), taking a look at the [Kubernetes project](https://github.com/kubernetes/) on GitHub, or join the [Kubernetes-dev Google group](https://groups.google.com/forum/#!forum/kubernetes-dev). If you’re really excited, you can do all of the above and join us for the next community conversation -- January 27th, 2016. Please add yourself or a topic you want to know about to [the agenda](https://docs.google.com/document/d/1VQDIAB0OqiSjIHI8AWMvSdceWhnz56jNpZrLs6o7NJY/edit) and get a calendar invitation by joining [this group](https://groups.google.com/forum/#!forum/kubernetes-community-video-chat).  


 Still want more Kubernetes? Check out the [recording](https://www.youtube.com/watch?v=izQLFx_6kwY&feature=youtu.be&list=PL69nYSiGNLP1pkHsbPjzAewvMgGUpkCnJ) of this meeting and the growing of the archive of [Kubernetes Community Meetings](https://www.youtube.com/playlist?list=PL69nYSiGNLP1pkHsbPjzAewvMgGUpkCnJ).
@@ -45,7 +45,7 @@ Support for [dynamic maximum volume count](https://github.com/kubernetes/feature

 The StorageObjectInUseProtection feature is now stable and prevents the removal of both [Persistent Volumes](https://github.com/kubernetes/features/issues/499) that are bound to a Persistent Volume Claim, and [Persistent Volume Claims](https://github.com/kubernetes/features/issues/498) that are being used by a pod. This safeguard will help prevent issues from deleting a PV or a PVC that is currently tied to an active pod.

-Each Special Interest Group (SIG) within the community continues to deliver the most-requested enhancements, fixes, and functionality for their respective specialty areas. For a complete list of inclusions by SIG, please visit the [release notes](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.11.md#111-release-notes).
+Each Special Interest Group (SIG) within the community continues to deliver the most-requested enhancements, fixes, and functionality for their respective specialty areas. For a complete list of inclusions by SIG, please visit the [release notes](https://github.com/kubernetes/kubernetes/blob/release-1.11/CHANGELOG-1.11.md#111-release-notes).

 ## Availability

@@ -88,7 +88,7 @@ Is Kubernetes helping your team? Share your story with the community.
 * The CNCF recently expanded its certification offerings to include a Certified Kubernetes Application Developer exam. The CKAD exam certifies an individual's ability to design, build, configure, and expose cloud native applications for Kubernetes. More information can be found [here](https://www.cncf.io/blog/2018/03/16/cncf-announces-ckad-exam/).
 * The CNCF recently added a new partner category, Kubernetes Training Partners (KTP). KTPs are a tier of vetted training providers who have deep experience in cloud native technology training. View partners and learn more [here](https://www.cncf.io/certification/training/).
 * CNCF also offers [online training](https://www.cncf.io/certification/training/) that teaches the skills needed to create and configure a real-world Kubernetes cluster.
-* Kubernetes documentation now features [user journeys](https://k8s.io/docs/home/): specific pathways for learning based on who readers are and what readers want to do. Learning Kubernetes is easier than ever for beginners, and more experienced users can find task journeys specific to cluster admins and application developers.  
+* Kubernetes documentation now features [user journeys](https://k8s.io/docs/home/): specific pathways for learning based on who readers are and what readers want to do. Learning Kubernetes is easier than ever for beginners, and more experienced users can find task journeys specific to cluster admins and application developers.

 ## KubeCon

@@ -12,7 +12,7 @@ Kubernetes is well-known for running scalable workloads. It scales your workload

 ## Guaranteed scheduling with controlled cost

-[Kubernetes Cluster Autoscaler](https://kubernetes.io/docs/tasks/administer-cluster/cluster-management/#cluster-autoscaling) is an excellent tool in the ecosystem which adds more nodes to your cluster when your applications need them. However, cluster autoscaler has some limitations and may not work for all users:
+[Kubernetes Cluster Autoscaler](https://github.com/kubernetes/autoscaler/) is an excellent tool in the ecosystem which adds more nodes to your cluster when your applications need them. However, cluster autoscaler has some limitations and may not work for all users:

 - It does not work in physical clusters.
 - Adding more nodes to the cluster costs more.
@@ -89,7 +89,7 @@ Volume cloning enables users to specify another PVC as a "DataSource" when provi
 - ExecutionHook API [designed to trigger hook commands](https://github.com/kubernetes/enhancements/issues/962) in the containers for different use cases is now Alpha.
 - Continued deprecation of extensions/v1beta1, apps/v1beta1, and apps/v1beta2 APIs; these extensions will be retired in 1.16!

-Check the [release notes](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.15.md#kubernetes-v115-release-notes) for a complete list of notable features and fixes.
+Check the [release notes](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.15.md#kubernetes-v115-release-notes) for a complete list of notable features and fixes.

 **Availability**

@@ -153,7 +153,7 @@ And as sources are always important to mention, we will follow (partially) the h

 ```bash
 # Download the latest version of KinD
-curl -Lo ./kind https://github.com/kubernetes-sigs/kind/releases/download/v0.7.0/kind-$(uname)-amd64
+curl -Lo ./kind https://github.com/kubernetes-sigs/kind/releases/download/v0.7.0/kind-linux-amd64
 # Make the binary executable
 chmod +x ./kind
 # Move the binary to your executable path
@@ -61,7 +61,7 @@ mind.
 ## Avoiding permanent beta

 For Kubernetes REST APIs, when a new feature's API reaches beta, that starts a countdown.
-The beta-quality API now has **nine calendar months** to either:
+The beta-quality API now has **three releases** (about nine calendar months) to either:
 - reach GA, and deprecate the beta, or
 - have a new beta version (_and deprecate the previous beta_).

@@ -69,9 +69,10 @@ To be clear, at this point **only REST APIs are affected**. For example, _APILis
 a beta feature but isn't itself a REST API. Right now there are no plans to automatically
 deprecate _APIListChunking_ nor any other features that aren't REST APIs.

-If a REST API reaches the end of that 9 month countdown, then the next Kubernetes release
-will deprecate that API version. There's no option for the REST API to stay at the same
-beta version beyond the first Kubernetes release to come out after the 9 month window.
+If a beta API has not graduated to GA after three Kubernetes releases, then the
+next Kubernetes release will deprecate that API version. There's no option for
+the REST API to stay at the same beta version beyond the first Kubernetes
+release to come out after the release window.

 ### What this means for you

@@ -0,0 +1,127 @@
+---
+layout: blog
+title: 'Kubernetes  1.19: Accentuate the Paw-sitive'
+date: 2020-08-26
+slug: kubernetes-release-1.19-accentuate-the-paw-sitive
+---
+
+**Authors:** [Kubernetes 1.19 Release Team](https://github.com/kubernetes/sig-release/blob/master/releases/release-1.19/release_team.md)
+
+Finally, we have arrived with Kubernetes 1.19, the second release for 2020, and by far the longest release cycle lasting 20 weeks in total. It consists of 34 enhancements: 10 enhancements are moving to stable, 15 enhancements in beta, and 9 enhancements in alpha.
+
+The 1.19 release was quite different from a regular release due to COVID-19, the George Floyd protests, and several other global events that we experienced as a release team. Due to these events, we made the decision to adjust our timeline and allow the SIGs, Working Groups, and contributors more time to get things done. The extra time also allowed for people to take time to focus on their lives outside of the Kubernetes project, and ensure their mental wellbeing was in a good place.
+
+Contributors are the heart of Kubernetes, not the other way around. The Kubernetes code of conduct asks that people be excellent to one another and despite the unrest in our world, we saw nothing but greatness and humility from the community.
+
+## Major Themes
+### Increase Kubernetes support window to one year
+
+A survey conducted in early 2019 by the [Long Term Support (LTS) working group](https://github.com/kubernetes/community/tree/master/wg-lts#readme) showed that a significant subset of Kubernetes end-users fail to upgrade within the current 9-month support period.
+This, and other responses from the survey, suggest that 30% of users would be able to keep their deployments on supported versions if the patch support period were extended to 12-14 months. This appears to be true regardless of whether the users are on self build or commercially vendored distributions. An extension would thus lead to more than 80% of users being on supported versions, instead of the 50-60% we have now.
+A yearly support period provides the cushion end-users appear to desire, and is more in harmony with familiar annual planning cycles.
+From Kubernetes version 1.19 on, the support window will be extended to one year.
+
+### Storage capacity tracking
+
+Traditionally, the Kubernetes scheduler was based on the assumptions that additional persistent storage is available everywhere in the cluster and has infinite capacity. Topology constraints addressed the first point, but up to now pod scheduling was still done without considering that the remaining storage capacity may not be enough to start a new pod. [Storage capacity tracking](/docs/concepts/storage/storage-capacity/), a new alpha feature, addresses that by adding an API for a CSI driver to report storage capacity and uses that information in the Kubernetes scheduler when choosing a node for a pod. This feature serves as a stepping stone for supporting dynamic provisioning for local volumes and other volume types that are more capacity constrained.
+
+#### Generic ephemeral volumes
+Kubernetes provides volume plugins whose lifecycle is tied to a pod and can be used as scratch space (e.g. the builtin `emptydir` volume type) or to load some data in to a pod (e.g. the builtin `configmap` and `secret` volume types, or “CSI inline volumes”). The new [generic ephemeral volumes](/docs/concepts/storage/ephemeral-volumes/#generic-ephemeral-volumes) alpha feature allows any existing storage driver that supports dynamic provisioning to be used as an ephemeral volume with the volume’s lifecycle bound to the Pod.
+It can be used to provide scratch storage that is different from the root disk, for example persistent memory, or a separate local disk on that node.
+All StorageClass parameters for volume provisioning are supported.
+All features supported with PersistentVolumeClaims are supported, such as storage capacity tracking, snapshots and restore, and volume resizing.
+
+#### CSI Volume Health Monitoring
+The alpha version of CSI health monitoring is being released with Kubernetes 1.19. This feature enables CSI Drivers to share abnormal volume conditions from the underlying storage systems with Kubernetes so that they can be reported as events on PVCs or Pods. This feature serves as a stepping stone towards programmatic detection and resolution of individual volume health issues by Kubernetes.
+
+### Ingress graduates to General Availability
+In terms of moving the Ingress API towards GA, the API itself has been available in beta for so long that it has attained de facto GA status through usage and adoption (both by users and by load balancer / ingress controller providers). Abandoning it without a full replacement is not a viable approach. It is clearly a useful API and captures a non-trivial set of use cases. At this point, it seems more prudent to declare the current API as something the community will support as a V1, codifying its status, while working on either a V2 Ingress API or an entirely different API with a superset of features.
+
+### Structured logging
+Before v1.19, logging in the Kubernetes control plane couldn't guarantee any uniform structure for log messages and references to Kubernetes objects in those logs. This makes parsing, processing, storing, querying and analyzing logs hard and forces administrators and developers to rely on ad-hoc solutions in most cases based on some regular expressions. Due to those problems any analytical solution based on those logs is hard to implement and maintain.
+
+#### New klog methods
+This Kubernetes release introduces new methods to the _klog_ library that provide a more structured interface for formatting log messages. Each existing formatted log method (`Infof`, `Errorf`) is now matched by a structured method (`InfoS`, `ErrorS`). The new logging methods accept log messages as a first argument and a list of key-values pairs as a variadic second argument. This approach allows incremental adoption of structured logging without converting **all** of Kubernetes to a new API at one time.
+
+### Client TLS certificate rotation for kubelet
+
+A kubelet authenticates the kubelet to the kube-apiserver using a private key and certificate. The certificate is supplied to the kubelet when it is first booted, via an out-of-cluster mechanism. Since Kubernetes v1.8, clusters have included a (beta) process for obtaining the initial cert/key pair and rotating it as expiration of the certificate approaches. In Kubernetes v1.19 this graduates to stable.
+
+During the kubelet start-up sequence, the filesystem is scanned for an existing cert/key pair, which is managed by the certificate manager. In the case that a cert/key is available it will be loaded. If not, the kubelet checks its config file for an encoded certificate value or a file reference in the kubeconfig. If the certificate is a bootstrap certificate, this will be used to generate a key, create a certificate signing request and request a signed certificate from the API server.
+
+When an expiration approaches the cert manager takes care of providing the correct certificate, generating new private keys and requesting new certificates. With the kubelet requesting certificates be signed as part of its boot sequence, and on an ongoing basis, certificate signing requests from the kubelet need to be auto approved to make cluster administration manageable.
+
+## Other Updates
+### Graduated to Stable
+* [Seccomp](https://github.com/kubernetes/enhancements/issues/135)
+* [Kubelet client TLS certificate rotation](https://github.com/kubernetes/enhancements/issues/266)
+* [Limit node access to API](https://github.com/kubernetes/enhancements/issues/279)
+* [Redesign Event API](https://github.com/kubernetes/enhancements/issues/383)
+* [Graduate Ingress to V1](https://github.com/kubernetes/enhancements/issues/1453)
+* [CertificateSigningRequest API](https://github.com/kubernetes/enhancements/issues/1513)
+* [Building Kubelet without Docker](https://github.com/kubernetes/enhancements/issues/1547)
+
+### Major Changes
+* [Node Topology Manager](https://github.com/kubernetes/enhancements/issues/693)
+* [New Endpoint API](https://github.com/kubernetes/enhancements/issues/752)
+* [Increase Kubernetes support window to one year](https://github.com/kubernetes/enhancements/issues/1498)
+
+### Other Notable Features
+* [Run multiple Scheduling Profiles](https://github.com/kubernetes/enhancements/issues/1451)
+* [CertificateSigningRequest API](https://github.com/kubernetes/enhancements/issues/1513)
+* [Immutable Secrets and ConfigMaps](https://github.com/kubernetes/enhancements/issues/1412)
+
+## Release Notes
+Check out the full details of the Kubernetes 1.19 release in our [release notes](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.19.md).
+
+## Availability
+
+Kubernetes 1.19 is available for download on [GitHub](https://github.com/kubernetes/kubernetes/releases/tag/v1.19.0). To get started with Kubernetes, check out these [interactive tutorials](https://kubernetes.io/docs/tutorials/) or run local Kubernetes clusters using Docker container “nodes” with [KinD](https://kind.sigs.k8s.io/) (Kubernetes in Docker). You can also easily install 1.19 using [kubeadm](https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/).
+
+## Release Team
+This release is made possible through the efforts of hundreds of individuals who contributed both technical and non-technical content. Special thanks to the [release team](https://github.com/kubernetes/sig-release/blob/master/releases/release-1.19/release_team.md) led by Taylor Dolezal, Senior Developer Advocate at HashiCorp. The 34 release team members coordinated many aspects of the release, from documentation to testing, validation, and feature completeness.
+
+As the Kubernetes community has grown, our release process represents an amazing demonstration of collaboration in open source software development. Kubernetes continues to gain new users at a rapid pace. This growth creates a positive feedback cycle where more contributors commit code creating a more vibrant ecosystem. Kubernetes has had over [49,000 individual contributors](https://k8s.devstats.cncf.io/d/24/overall-project-statistics?orgId=1) to date and an active community of more than 3,000 people.
+
+## Release Logo
+All of you inspired this Kubernetes 1.19 release logo! This release was a bit more of a marathon and a testament to when the world is a wild place, we can come together and do unbelievable things.
+
+![Kubernetes 1.19 Release Logo](/images/blog/2020-08-26-kubernetes-1.19-release-announcement/accentuate.png)
+
+
+"Accentuate the Paw-sitive" was chosen as the release theme because it captures the positive outlook that the release team had, despite the state of the world. The characters pictured in the 1.19 logo represent everyone's personalities on our release team, from emo to peppy, and beyond!
+
+About the designer: Hannabeth Lagerlof is a Visual Designer based in Los Angeles, California, and she has an extensive background in Environments and Graphic Design. Hannabeth creates art and user experiences that inspire connection. You can find Hannabeth on Twitter as @emanate_design.
+
+## The Long Run
+The release was also different from the enhancements side of things. Traditionally, we have had 3-4 weeks between the call for enhancements and Enhancements Freeze, which ends the phase in which contributors can acknowledge whether a particular feature will be part of the cycle. This release cycle, being unique, we had five weeks for the same milestone. The extended duration gave the contributors more time to plan and decide about the graduation of their respective features.
+
+The milestone until which contributors implement the features was extended from the usual five weeks to 7 weeks. Contributors were provided with 40% more time to work on their features, resulting in reduced fatigue and more to think through about the implementation. We also noticed a considerable reduction in last-minute hustles. There were also a lesser number of exception requests this cycle - 6 compared to 14 the previous release cycle.
+
+## User Highlights
+* The CNCF grants Zalando, Europe’s leading online platform for fashion and lifestyle, the [Top End User Award](https://www.cncf.io/announcement/2020/08/20/cloud-native-computing-foundation-grants-zalando-the-top-end-user-award/). Zalando leverages numerous CNCF projects and open sourced multiple of their own development.
+
+## Ecosystem Updates
+* The CNCF just concluded its very first Virtual KubeCon. All talks are [on-demand]( https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/) for anyone registered, it's not too late!
+* The [Certified Kubernetes Security Specialist](https://www.cncf.io/blog/2020/07/15/certified-kubernetes-security-specialist-cks-coming-in-november/) (CKS) coming in November! CKS focuses on cluster & system hardening, minimizing microservice vulnerabilities and the security of the supply chain.
+* CNCF published the second [State of Cloud Native Development](https://www.cncf.io/blog/2020/08/14/state-of-cloud-native-development/), showing the massively growing number of cloud native developer using container and serverless technology.
+* [Kubernetes.dev](https://www.kubernetes.dev), a Kubernetes contributor focused website has been launched. It brings the  contributor documentation, resources and project event information into one central location.
+
+## Project Velocity
+The [Kubernetes DevStats dashboard](https://k8s.devstats.cncf.io/d/12/dashboards?orgId=1) illustrates the breakdown of contributions from major company contributors, as well as an impressive set of preconfigured reports on everything from individual contributors to pull request lifecycle times. If you want to gather numbers, facts and figures from Kubernetes and the CNCF community it is the best place to start.
+
+During this release cycle from April till August, 382 different companies and over 2,464 individuals contributed to Kubernetes. [Check out DevStats](https://k8s.devstats.cncf.io/d/11/companies-contributing-in-repository-groups?orgId=1&var-period=m&var-repogroup_name=All&from=1585692000000&to=1598392799000) to learn more about the overall velocity of the Kubernetes project and community.
+
+## Upcoming release webinar
+Join the members of the Kubernetes 1.19 release team on September 25th, 2020 to learn about the major features in this release including storage capacity tracking, structured logging, Ingress V1 GA, and many more. Register here: https://www.cncf.io/webinars/kubernetes-1-19/.
+
+## Get Involved
+The simplest way to get involved with Kubernetes is by joining one of the many [Special Interest Groups](https://github.com/kubernetes/community/blob/master/sig-list.md) (SIGs) that align with your interests. Have something you’d like to broadcast to the Kubernetes community? Share your voice at our monthly [community meeting](https://github.com/kubernetes/community/tree/master/communication), and through the channels below. Thank you for your continued feedback and support.
+
+* Find out more about contributing to Kubernetes at the new [Kubernetes Contributor website](https://www.kubernetes.dev/)
+* Follow us on Twitter [@Kubernetesio](https://twitter.com/kubernetesio) for latest updates
+* Join the community discussion on [Discuss](https://discuss.kubernetes.io/)
+* Join the community on [Slack](http://slack.k8s.io/)
+* Share your Kubernetes [story](https://docs.google.com/a/linuxfoundation.org/forms/d/e/1FAIpQLScuI7Ye3VQHQTwBASrgkjQDSS5TP0g3AXfFhwSM9YpHgxRKFA/viewform)
+* Read more about what’s happening with Kubernetes on the [blog](https://kubernetes.io/blog/)
+* Learn more about the [Kubernetes Release Team](https://github.com/kubernetes/sig-release/tree/master/release-team)
@@ -0,0 +1,30 @@
+---
+layout: blog 
+title: 'Increasing the Kubernetes Support Window to One Year'
+date: 2020-08-31 
+slug: kubernetes-1-19-feature-one-year-support
+---
+
+**Authors:** Tim Pepper (VMware), Nick Young (VMware)
+
+Starting with Kubernetes 1.19, the support window for Kubernetes versions [will increase from 9 months to one year](https://github.com/kubernetes/enhancements/issues/1498). The longer support window is intended to allow organizations to perform major upgrades at a time of the year that works the best for them.
+
+This is a big change. For many years, the Kubernetes project has delivered a new minor release (e.g.: 1.13 or 1.14) every 3 months. The project provides bugfix support via patch releases (e.g.: 1.13.Y) for three parallel branches of the codebase. Combined, this led to each minor release (e.g.: 1.13) having a patch release stream of support for approximately 9 months. In the end, a cluster operator had to upgrade at least every 9 months to remain supported. 
+
+A survey conducted in early 2019 by the WG LTS showed that a significant subset of Kubernetes end-users fail to upgrade within the 9-month support period. 
+
+![Versions in Production](/images/blog/2020-08-31-increase-kubernetes-support-one-year/versions-in-production-text-2.png)
+
+This, and other responses from the survey, suggest that a considerable portion of our community would better be able to manage their deployments on supported versions if the patch support period were extended to 12-14 months. It appears to be true regardless of whether the users are on DIY builds or commercially vendored distributions. An extension in the patch support length of time would thus lead to a larger percentage of our user base running supported versions compared to what we have now.
+
+A yearly support period provides the cushion end-users appear to desire, and is more aligned with familiar annual planning cycles.
+There are many unknowns about changing the support windows for a project with as many moving parts as Kubernetes. Keeping the change relatively small (relatively being the important word), gives us the chance to find out what those unknowns are in detail and address them.
+From Kubernetes version 1.19 on, the support window will be extended to one year. For Kubernetes versions 1.16, 1.17, and 1.18, the story is more complicated.
+
+All of these versions still fall under the older “three releases support” model, and will drop out of support when 1.19, 1.20 and 1.21 are respectively released. However, because the 1.19 release has been delayed due to the events of 2020, they will end up with close to a year of support (depending on their exact release dates).
+
+For example, 1.19 was released on the 26th of August 2020, which is 11 months since the release of 1.16. Since 1.16 is still under the old release policy, this means that it is now out of support.
+
+![Support Timeline](/images/blog/2020-08-31-increase-kubernetes-support-one-year/support-timeline.png)
+
+If you’ve got thoughts or feedback, we’d love to hear them. Please contact us on [#wg-lts](https://kubernetes.slack.com/messages/wg-lts/) on the Kubernetes Slack, or to the [kubernetes-wg-lts mailing list](https://groups.google.com/g/kubernetes-wg-lts).
@@ -0,0 +1,394 @@
+---
+layout: blog
+title: 'Ephemeral volumes with storage capacity tracking: EmptyDir on steroids'
+date: 2020-09-01
+slug: ephemeral-volumes-with-storage-capacity-tracking
+---
+
+**Author:** Patrick Ohly (Intel)
+
+Some applications need additional storage but don't care whether that
+data is stored persistently across restarts. For example, caching
+services are often limited by memory size and can move infrequently
+used data into storage that is slower than memory with little impact
+on overall performance. Other applications expect some read-only input
+data to be present in files, like configuration data or secret keys.
+
+Kubernetes already supports several kinds of such [ephemeral
+volumes](/docs/concepts/storage/ephemeral-volumes), but the
+functionality of those is limited to what is implemented inside
+Kubernetes.
+
+[CSI ephemeral volumes](https://kubernetes.io/blog/2020/01/21/csi-ephemeral-inline-volumes/)
+made it possible to extend Kubernetes with CSI
+drivers that provide light-weight, local volumes. These [*inject
+arbitrary states, such as configuration, secrets, identity, variables
+or similar
+information*](https://github.com/kubernetes/enhancements/blob/master/keps/sig-storage/20190122-csi-inline-volumes.md#motivation).
+CSI drivers must be modified to support this Kubernetes feature,
+i.e. normal, standard-compliant CSI drivers will not work, and
+by design such volumes are supposed to be usable on whatever node
+is chosen for a pod.
+
+This is problematic for volumes which consume significant resources on
+a node or for special storage that is only available on some nodes.
+Therefore, Kubernetes 1.19 introduces two new alpha features for
+volumes that are conceptually more like the `EmptyDir` volumes:
+- [*generic* ephemeral volumes](/docs/concepts/storage/ephemeral-volumes#generic-ephemeral-volumes) and
+- [CSI storage capacity tracking](/docs/concepts/storage/storage-capacity).
+
+The advantages of the new approach are:
+- Storage can be local or network-attached.
+- Volumes can have a fixed size that applications are never able to exceed.
+- Works with any CSI driver that supports provisioning of persistent
+  volumes and (for capacity tracking) implements the CSI `GetCapacity` call.
+- Volumes may have some initial data, depending on the driver and
+  parameters.
+- All of the typical volume operations (snapshotting,
+  resizing, the future storage capacity tracking, etc.)
+  are supported.
+- The volumes are usable with any app controller that accepts
+  a Pod or volume specification.
+- The Kubernetes scheduler itself picks suitable nodes, i.e. there is
+  no need anymore to implement and configure scheduler extenders and
+  mutating webhooks.
+
+This makes generic ephemeral volumes a suitable solution for several
+use cases:
+
+# Use cases
+
+## Persistent Memory as DRAM replacement for memcached
+
+Recent releases of memcached added [support for using Persistent
+Memory](https://memcached.org/blog/persistent-memory/) (PMEM) instead
+of standard DRAM. When deploying memcached through one of the app
+controllers, generic ephemeral volumes make it possible to request a PMEM volume
+of a certain size from a CSI driver like
+[PMEM-CSI](https://intel.github.io/pmem-csi/).
+
+## Local LVM storage as scratch space
+
+Applications working with data sets that exceed the RAM size can
+request local storage with performance characteristics or size that is
+not met by the normal Kubernetes `EmptyDir` volumes. For example,
+[TopoLVM](https://github.com/cybozu-go/topolvm) was written for that
+purpose.
+
+## Read-only access to volumes with data
+
+Provisioning a volume might result in a non-empty volume:
+- [restore a snapshot](/docs/concepts/storage/persistent-volumes/#volume-snapshot-and-restore-volume-from-snapshot-support)
+- [cloning a volume](/docs/concepts/storage/volume-pvc-datasource)
+- [generic data populators](https://github.com/kubernetes/enhancements/blob/master/keps/sig-storage/20200120-generic-data-populators.md)
+
+Such volumes can be mounted read-only.
+
+# How it works
+
+## Generic ephemeral volumes
+
+The key idea behind generic ephemeral volumes is that a new volume
+source, the so-called
+[`EphemeralVolumeSource`](/docs/reference/generated/kubernetes-api/#ephemeralvolumesource-v1alpha1-core)
+contains all fields that are needed to created a volume claim
+(historically called persistent volume claim, PVC). A new controller
+in the `kube-controller-manager` waits for Pods which embed such a
+volume source and then creates a PVC for that pod. To a CSI driver
+deployment, that PVC looks like any other, so no special support is
+needed.
+
+As long as these PVCs exist, they can be used like any other volume claim. In
+particular, they can be referenced as data source in volume cloning or
+snapshotting. The PVC object also holds the current status of the
+volume.
+
+Naming of the automatically created PVCs is deterministic: the name is
+a combination of Pod name and volume name, with a hyphen (`-`) in the
+middle. This deterministic naming makes it easier to
+interact with the PVC because one does not have to search for it once
+the Pod name and volume name are known. The downside is that the name might
+be in use already. This is detected by Kubernetes and then blocks Pod
+startup.
+
+To ensure that the volume gets deleted together with the pod, the
+controller makes the Pod the owner of the volume claim. When the Pod
+gets deleted, the normal garbage-collection mechanism also removes the
+claim and thus the volume.
+
+Claims select the storage driver through the normal storage class
+mechanism. Although storage classes with both immediate and late
+binding (aka `WaitForFirstConsumer`) are supported, for ephemeral
+volumes it makes more sense to use `WaitForFirstConsumer`: then Pod
+scheduling can take into account both node utilization and
+availability of storage when choosing a node. This is where the other
+new feature comes in.
+
+## Storage capacity tracking
+
+Normally, the Kubernetes scheduler has no information about where a
+CSI driver might be able to create a volume. It also has no way of
+talking directly to a CSI driver to retrieve that information. It
+therefore tries different nodes until it finds one where all volumes
+can be made available (late binding) or leaves it entirely to the
+driver to choose a location (immediate binding).
+
+The new [`CSIStorageCapacity` alpha
+API](/docs/reference/generated/kubernetes-api/v1.19/#csistoragecapacity-v1alpha1-storage-k8s-io)
+allows storing the necessary information in etcd where it is available to the
+scheduler. In contrast to support for generic ephemeral volumes,
+storage capacity tracking must be [enabled when deploying a CSI
+driver](https://github.com/kubernetes-csi/external-provisioner/blob/master/README.md#capacity-support):
+the `external-provisioner` must be told to publish capacity
+information that it then retrieves from the CSI driver through the normal
+`GetCapacity` call.
+<!-- TODO: update the link with a revision once https://github.com/kubernetes-csi/external-provisioner/pull/450 is merged -->
+
+When the Kubernetes scheduler needs to choose a node for a Pod with an
+unbound volume that uses late binding and the CSI driver deployment
+has opted into the feature by setting the [`CSIDriver.storageCapacity`
+flag](/docs/reference/generated/kubernetes-api/v1.19/#csidriver-v1beta1-storage-k8s-io)
+flag, the scheduler automatically filters out nodes that do not have
+access to enough storage capacity. This works for generic ephemeral
+and persistent volumes but *not* for CSI ephemeral volumes because the
+parameters of those are opaque for Kubernetes.
+
+As usual, volumes with immediate binding get created before scheduling
+pods, with their location chosen by the storage driver. Therefore, the
+external-provisioner's default configuration skips storage
+classes with immediate binding as the information wouldn't be used anyway.
+
+Because the Kubernetes scheduler must act on potentially outdated
+information, it cannot be ensured that the capacity is still available
+when a volume is to be created. Still, the chances that it can be created
+without retries should be higher.
+
+# Security
+
+## CSIStorageCapacity
+
+CSIStorageCapacity objects are namespaced. When deploying each CSI
+drivers in its own namespace and, as recommended, limiting the RBAC
+permissions for CSIStorageCapacity to that namespace, it is
+always obvious where the data came from. However, Kubernetes does
+not check that and typically drivers get installed in the same
+namespace anyway, so ultimately drivers are *expected to behave* and
+not publish incorrect data.
+
+## Generic ephemeral volumes
+
+If users have permission to create a Pod (directly or indirectly),
+then they can also create generic ephemeral volumes even when they do
+not have permission to create a volume claim. That's because RBAC
+permission checks are applied to the controller which creates the
+PVC, not the original user. This is a fundamental change that must be
+[taken into
+account](/docs/concepts/storage/ephemeral-volumes#security) before
+enabling the feature in clusters where untrusted users are not
+supposed to have permission to create volumes.
+
+# Example
+
+A [special branch](https://github.com/intel/pmem-csi/commits/kubernetes-1-19-blog-post)
+in PMEM-CSI contains all the necessary changes to bring up a
+Kubernetes 1.19 cluster inside QEMU VMs with both alpha features
+enabled. The PMEM-CSI driver code is used unchanged, only the
+deployment was updated.
+
+On a suitable machine (Linux, non-root user can use Docker - see the
+[QEMU and
+Kubernetes](https://intel.github.io/pmem-csi/0.7/docs/autotest.html#qemu-and-kubernetes)
+section in the PMEM-CSI documentation), the following commands bring
+up a cluster and install the PMEM-CSI driver:
+
+```console
+git clone --branch=kubernetes-1-19-blog-post https://github.com/intel/pmem-csi.git
+cd pmem-csi
+export TEST_KUBERNETES_VERSION=1.19 TEST_FEATURE_GATES=CSIStorageCapacity=true,GenericEphemeralVolume=true TEST_PMEM_REGISTRY=intel
+make start && echo && test/setup-deployment.sh
+```
+
+If all goes well, the output contains the following usage
+instructions:
+
+```
+The test cluster is ready. Log in with [...]/pmem-csi/_work/pmem-govm/ssh.0, run
+kubectl once logged in.  Alternatively, use kubectl directly with the
+following env variable:
+   KUBECONFIG=[...]/pmem-csi/_work/pmem-govm/kube.config
+
+secret/pmem-csi-registry-secrets created
+secret/pmem-csi-node-secrets created
+serviceaccount/pmem-csi-controller created
+...
+To try out the pmem-csi driver ephemeral volumes:
+   cat deploy/kubernetes-1.19/pmem-app-ephemeral.yaml |
+   [...]/pmem-csi/_work/pmem-govm/ssh.0 kubectl create -f -
+```
+
+The CSIStorageCapacity objects are not meant to be human-readable, so
+some post-processing is needed. The following Golang template filters
+all objects by the storage class that the example uses and prints the
+name, topology and capacity:
+
+```console
+kubectl get \
+        -o go-template='{{range .items}}{{if eq .storageClassName "pmem-csi-sc-late-binding"}}{{.metadata.name}} {{.nodeTopology.matchLabels}} {{.capacity}}
+{{end}}{{end}}' \
+        csistoragecapacities
+```
+
+```
+csisc-2js6n map[pmem-csi.intel.com/node:pmem-csi-pmem-govm-worker2] 30716Mi
+csisc-sqdnt map[pmem-csi.intel.com/node:pmem-csi-pmem-govm-worker1] 30716Mi
+csisc-ws4bv map[pmem-csi.intel.com/node:pmem-csi-pmem-govm-worker3] 30716Mi
+```
+
+One individual object has the following content:
+
+```console
+kubectl describe csistoragecapacities/csisc-6cw8j
+```
+
+```
+Name:         csisc-sqdnt
+Namespace:    default
+Labels:       <none>
+Annotations:  <none>
+API Version:  storage.k8s.io/v1alpha1
+Capacity:     30716Mi
+Kind:         CSIStorageCapacity
+Metadata:
+  Creation Timestamp:  2020-08-11T15:41:03Z
+  Generate Name:       csisc-
+  Managed Fields:
+    ...
+  Owner References:
+    API Version:     apps/v1
+    Controller:      true
+    Kind:            StatefulSet
+    Name:            pmem-csi-controller
+    UID:             590237f9-1eb4-4208-b37b-5f7eab4597d1
+  Resource Version:  2994
+  Self Link:         /apis/storage.k8s.io/v1alpha1/namespaces/default/csistoragecapacities/csisc-sqdnt
+  UID:               da36215b-3b9d-404a-a4c7-3f1c3502ab13
+Node Topology:
+  Match Labels:
+    pmem-csi.intel.com/node:  pmem-csi-pmem-govm-worker1
+Storage Class Name:           pmem-csi-sc-late-binding
+Events:                       <none>
+```
+
+Now let's create the example app with one generic ephemeral
+volume. The `pmem-app-ephemeral.yaml` file contains:
+
+```yaml
+# This example Pod definition demonstrates
+# how to use generic ephemeral inline volumes
+# with a PMEM-CSI storage class.
+kind: Pod
+apiVersion: v1
+metadata:
+  name: my-csi-app-inline-volume
+spec:
+  containers:
+    - name: my-frontend
+      image: intel/pmem-csi-driver-test:v0.7.14
+      command: [ "sleep", "100000" ]
+      volumeMounts:
+      - mountPath: "/data"
+        name: my-csi-volume
+  volumes:
+  - name: my-csi-volume
+    ephemeral:
+      volumeClaimTemplate:
+        spec:
+          accessModes:
+          - ReadWriteOnce
+          resources:
+            requests:
+              storage: 4Gi
+          storageClassName: pmem-csi-sc-late-binding
+```
+
+After creating that as shown in the usage instructions above, we have one additional Pod and PVC:
+
+```console
+kubectl get pods/my-csi-app-inline-volume -o wide
+```
+
+```
+NAME                       READY   STATUS    RESTARTS   AGE     IP          NODE                         NOMINATED NODE   READINESS GATES
+my-csi-app-inline-volume   1/1     Running   0          6m58s   10.36.0.2   pmem-csi-pmem-govm-worker1   <none>           <none>
+```
+
+```console
+kubectl get pvc/my-csi-app-inline-volume-my-csi-volume
+```
+
+```
+NAME                                     STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS               AGE
+my-csi-app-inline-volume-my-csi-volume   Bound    pvc-c11eb7ab-a4fa-46fe-b515-b366be908823   4Gi        RWO            pmem-csi-sc-late-binding   9m21s
+```
+
+That PVC is owned by the Pod:
+
+```console
+kubectl get -o yaml pvc/my-csi-app-inline-volume-my-csi-volume
+```
+
+```
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  annotations:
+    pv.kubernetes.io/bind-completed: "yes"
+    pv.kubernetes.io/bound-by-controller: "yes"
+    volume.beta.kubernetes.io/storage-provisioner: pmem-csi.intel.com
+    volume.kubernetes.io/selected-node: pmem-csi-pmem-govm-worker1
+  creationTimestamp: "2020-08-11T15:44:57Z"
+  finalizers:
+  - kubernetes.io/pvc-protection
+  managedFields:
+    ...
+  name: my-csi-app-inline-volume-my-csi-volume
+  namespace: default
+  ownerReferences:
+  - apiVersion: v1
+    blockOwnerDeletion: true
+    controller: true
+    kind: Pod
+    name: my-csi-app-inline-volume
+    uid: 75c925bf-ca8e-441a-ac67-f190b7a2265f
+...
+```
+
+Eventually, the storage capacity information for `pmem-csi-pmem-govm-worker1` also gets updated:
+
+```
+csisc-2js6n map[pmem-csi.intel.com/node:pmem-csi-pmem-govm-worker2] 30716Mi
+csisc-sqdnt map[pmem-csi.intel.com/node:pmem-csi-pmem-govm-worker1] 26620Mi
+csisc-ws4bv map[pmem-csi.intel.com/node:pmem-csi-pmem-govm-worker3] 30716Mi
+```
+
+If another app needs more than 26620Mi, the Kubernetes
+scheduler will not pick `pmem-csi-pmem-govm-worker1` anymore.
+
+
+# Next steps
+
+Both features are under development. Several open questions were
+already raised during the alpha review process. The two enhancement
+proposals document the work that will be needed for migration to beta and what
+alternatives were already considered and rejected:
+
+* [KEP-1698: generic ephemeral inline
+volumes](https://github.com/kubernetes/enhancements/blob/9d7a75d/keps/sig-storage/1698-generic-ephemeral-volumes/README.md)
+* [KEP-1472: Storage Capacity
+Tracking](https://github.com/kubernetes/enhancements/tree/9d7a75d/keps/sig-storage/1472-storage-capacity-tracking)
+
+Your feedback is crucial for driving that development. SIG-Storage
+[meets
+regularly](https://github.com/kubernetes/community/tree/master/sig-storage#meetings)
+and can be reached via [Slack and a mailing
+list](https://github.com/kubernetes/community/tree/master/sig-storage#contact).
@@ -0,0 +1,46 @@
+---
+layout: blog
+title: 'Scaling Kubernetes Networking With EndpointSlices'
+date: 2020-09-02
+slug: scaling-kubernetes-networking-with-endpointslices
+---
+
+**Author:** Rob Scott (Google)
+
+EndpointSlices are an exciting new API that provides a scalable and extensible alternative to the Endpoints API. EndpointSlices track IP addresses, ports, readiness, and topology information for Pods backing a Service.
+
+In Kubernetes 1.19 this feature is enabled by default with kube-proxy reading from [EndpointSlices](/docs/concepts/services-networking/endpoint-slices/) instead of Endpoints. Although this will mostly be an invisible change, it should result in noticeable scalability improvements in large clusters. It also enables significant new features in future Kubernetes releases like [Topology Aware Routing](/docs/concepts/services-networking/service-topology/).
+
+## Scalability Limitations of the Endpoints API 
+With the Endpoints API, there was only one Endpoints resource for a Service. That meant that it needed to be able to store IP addresses and ports (network endpoints) for every Pod that was backing the corresponding Service. This resulted in huge API resources. To compound this problem, kube-proxy was running on every node and watching for any updates to Endpoints resources. If even a single network endpoint changed in an Endpoints resource, the whole object would have to be sent to each of those instances of kube-proxy.
+
+A further limitation of the Endpoints API is that it limits the number of network endpoints that can be tracked for a Service. The default size limit for an object stored in etcd is 1.5MB. In some cases that can limit an Endpoints resource to 5,000 Pod IPs. This is not an issue for most users, but it becomes a significant problem for users with Services approaching this size.
+
+To show just how significant these issues become at scale it helps to have a simple example. Think about a Service which has 5,000 Pods, it might end up with a 1.5MB Endpoints resource. If even a single network endpoint in that list changes, the full Endpoints resource will need to be distributed to each Node in the cluster. This becomes quite an issue in a large cluster with 3,000 Nodes. Each update would involve sending 4.5GB of data (1.5MB Endpoints * 3,000 Nodes) across the cluster. That's nearly enough to fill up a DVD, and it would happen for each Endpoints change. Imagine a rolling update that results in all 5,000 Pods being replaced - that's more than 22TB (or 5,000 DVDs) worth of data transferred.
+
+## Splitting endpoints up with the EndpointSlice API
+The EndpointSlice API was designed to address this issue with an approach similar to sharding. Instead of tracking all Pod IPs for a Service with a single Endpoints resource, we split them into multiple smaller EndpointSlices. 
+
+Consider an example where a Service is backed by 15 pods. We'd end up with a single Endpoints resource that tracked all of them. If EndpointSlices were configured to store 5 endpoints each, we'd end up with 3 different EndpointSlices:
+![EndpointSlices](/images/blog/2020-09-02-scaling-kubernetes-networking-endpointslices/endpoint-slices.png)
+
+By default, EndpointSlices  store as many as 100 endpoints each, though this can be configured with the `--max-endpoints-per-slice` flag on kube-controller-manager.
+
+## EndpointSlices provide 10x scalability improvements
+This API dramatically improves networking scalability. Now when a Pod is added or removed, only 1 small EndpointSlice needs to be updated. This difference becomes quite noticeable when hundreds or thousands of Pods are backing a single Service. 
+
+Potentially more significant, now that all Pod IPs for a Service don't need to be stored in a single resource, we don't have to worry about the size limit for objects stored in etcd. EndpointSlices have already been used to scale Services beyond 100,000 network endpoints.
+
+All of this is brought together with some significant performance improvements that have been made in kube-proxy. When using EndpointSlices at scale, significantly less data will be transferred for endpoints updates and kube-proxy should be faster to update iptables or ipvs rules. Beyond that, Services can now scale to at least 10 times beyond any previous limitations.
+
+## EndpointSlices enable new functionality
+Introduced as an alpha feature in Kubernetes v1.16, EndpointSlices were built to enable some exciting new functionality in future Kubernetes releases. This could include dual-stack Services, topology aware routing, and endpoint subsetting.
+
+Dual-Stack Services are an exciting new feature that has been in development alongside EndpointSlices. They will utilize both IPv4 and IPv6 addresses for Services and rely on the addressType field on EndpointSlices to track these addresses by IP family. 
+
+Topology aware routing will update kube-proxy to prefer routing requests within the same zone or region. This makes use of the topology fields stored for each endpoint in an EndpointSlice. As a further refinement of that, we're exploring the potential of endpoint subsetting. This would allow kube-proxy to only watch a subset of EndpointSlices. For example, this might be combined with topology aware routing so that kube-proxy would only need to watch EndpointSlices containing endpoints within the same zone. This would provide another very significant scalability improvement.
+
+## What does this mean for the Endpoints API?
+Although the EndpointSlice API is providing a newer and more scalable alternative to the Endpoints API, the Endpoints API will continue to be considered generally available and stable. The most significant change planned for the Endpoints API will involve beginning to truncate Endpoints that would otherwise run into scalability issues.
+
+The Endpoints API is not going away, but many new features will rely on the EndpointSlice API. To take advantage of the new scalability and functionality that EndpointSlices provide, applications that currently consume Endpoints will likely want to consider supporting EndpointSlices in the future.
@@ -0,0 +1,333 @@
+---
+layout: blog
+title: "Warning: Helpful Warnings Ahead"
+date: 2020-09-03
+slug: warnings
+---
+
+**Author**: Jordan Liggitt (Google)
+
+As Kubernetes maintainers, we're always looking for ways to improve usability while preserving compatibility.
+As we develop features, triage bugs, and answer support questions, we accumulate information that would be helpful for Kubernetes users to know.
+In the past, sharing that information was limited to out-of-band methods like release notes, announcement emails, documentation, and blog posts.
+Unless someone knew to seek out that information and managed to find it, they would not benefit from it.
+
+In Kubernetes v1.19, we added a feature that allows the Kubernetes API server to
+[send warnings to API clients](https://github.com/kubernetes/enhancements/tree/master/keps/sig-api-machinery/1693-warnings).
+The warning is sent using a [standard `Warning` response header](https://tools.ietf.org/html/rfc7234#section-5.5),
+so it does not change the status code or response body in any way.
+This allows the server to send warnings easily readable by any API client, while remaining compatible with previous client versions.
+
+Warnings are surfaced by `kubectl` v1.19+ in `stderr` output, and by the `k8s.io/client-go` client library v0.19.0+ in log output.
+The `k8s.io/client-go` behavior can be [overridden per-process or per-client](#customize-client-handling).
+
+## Deprecation Warnings
+
+The first way we are using this new capability is to send warnings for use of deprecated APIs.
+
+Kubernetes is a [big, fast-moving project](https://www.cncf.io/cncf-kubernetes-project-journey/#development-velocity).
+Keeping up with the [changes](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.19.md#changelog-since-v1180)
+in each release can be daunting, even for people who work on the project full-time. One important type of change is API deprecations.
+As APIs in Kubernetes graduate to GA versions, pre-release API versions are deprecated and eventually removed.
+
+Even though there is an [extended deprecation period](/docs/reference/using-api/deprecation-policy/),
+and deprecations are [included in release notes](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.19.md#deprecation),
+they can still be hard to track. During the deprecation period, the pre-release API remains functional,
+allowing several releases to transition to the stable API version. However, we have found that users often don't even realize
+they are depending on a deprecated API version until they upgrade to the release that stops serving it.
+
+Starting in v1.19, whenever a request is made to a deprecated REST API, a warning is returned along with the API response.
+This warning includes details about the release in which the API will no longer be available, and the replacement API version.
+
+Because the warning originates at the server, and is intercepted at the client level, it works for all kubectl commands,
+including high-level commands like `kubectl apply`, and low-level commands like `kubectl get --raw`:
+
+<img alt="kubectl applying a manifest file, then displaying a warning message 'networking.k8s.io/v1beta1 Ingress is deprecated in v1.19+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress'."
+     src="kubectl-warnings.png"
+     style="width:637px;max-width:100%;">
+
+This helps people affected by the deprecation to know the request they are making is deprecated,
+how long they have to address the issue, and what API they should use instead.
+This is especially helpful when the user is applying a manifest they didn't create,
+so they have time to reach out to the authors to ask for an updated version.
+
+We also realized that the person *using* a deprecated API is often not the same person responsible for upgrading the cluster,
+so we added two administrator-facing tools to help track use of deprecated APIs and determine when upgrades are safe.
+
+### Metrics 
+
+Starting in Kubernetes v1.19, when a request is made to a deprecated REST API endpoint,
+an `apiserver_requested_deprecated_apis` gauge metric is set to `1` in the kube-apiserver process.
+This metric has labels for the API `group`, `version`, `resource`, and `subresource`,
+and a `removed_version` label that indicates the Kubernetes release in which the API will no longer be served.
+
+This is an example query using `kubectl`, [prom2json](https://github.com/prometheus/prom2json),
+and [jq](https://stedolan.github.io/jq/) to determine which deprecated APIs have been requested
+from the current instance of the API server:
+
+```sh
+kubectl get --raw /metrics | prom2json | jq '
+  .[] | select(.name=="apiserver_requested_deprecated_apis").metrics[].labels
+'
+```
+
+Output:
+
+```json
+{
+  "group": "extensions",
+  "removed_release": "1.22",
+  "resource": "ingresses",
+  "subresource": "",
+  "version": "v1beta1"
+}
+{
+  "group": "rbac.authorization.k8s.io",
+  "removed_release": "1.22",
+  "resource": "clusterroles",
+  "subresource": "",
+  "version": "v1beta1"
+}
+```
+
+This shows the deprecated `extensions/v1beta1` Ingress and `rbac.authorization.k8s.io/v1beta1` ClusterRole APIs
+have been requested on this server, and will be removed in v1.22.
+
+We can join that information with the `apiserver_request_total` metrics to get more details about the requests being made to these APIs:
+
+```sh
+kubectl get --raw /metrics | prom2json | jq '
+  # set $deprecated to a list of deprecated APIs
+  [
+    .[] | 
+    select(.name=="apiserver_requested_deprecated_apis").metrics[].labels |
+    {group,version,resource}
+  ] as $deprecated 
+  
+  |
+  
+  # select apiserver_request_total metrics which are deprecated
+  .[] | select(.name=="apiserver_request_total").metrics[] |
+  select(.labels | {group,version,resource} as $key | $deprecated | index($key))
+'
+```
+
+Output:
+
+```json
+{
+  "labels": {
+    "code": "0",
+    "component": "apiserver",
+    "contentType": "application/vnd.kubernetes.protobuf;stream=watch",
+    "dry_run": "",
+    "group": "extensions",
+    "resource": "ingresses",
+    "scope": "cluster",
+    "subresource": "",
+    "verb": "WATCH",
+    "version": "v1beta1"
+  },
+  "value": "21"
+}
+{
+  "labels": {
+    "code": "200",
+    "component": "apiserver",
+    "contentType": "application/vnd.kubernetes.protobuf",
+    "dry_run": "",
+    "group": "extensions",
+    "resource": "ingresses",
+    "scope": "cluster",
+    "subresource": "",
+    "verb": "LIST",
+    "version": "v1beta1"
+  },
+  "value": "1"
+}
+{
+  "labels": {
+    "code": "200",
+    "component": "apiserver",
+    "contentType": "application/json",
+    "dry_run": "",
+    "group": "rbac.authorization.k8s.io",
+    "resource": "clusterroles",
+    "scope": "cluster",
+    "subresource": "",
+    "verb": "LIST",
+    "version": "v1beta1"
+  },
+  "value": "1"
+}
+```
+
+The output shows that only read requests are being made to these APIs, and the most requests have been made to watch the deprecated Ingress API.
+
+You can also find that information through the following Prometheus query,
+which returns information about requests made to deprecated APIs which will be removed in v1.22:
+
+```promql
+apiserver_requested_deprecated_apis{removed_version="1.22"} * on(group,version,resource,subresource)
+group_right() apiserver_request_total
+```
+
+### Audit Annotations
+
+Metrics are a fast way to check whether deprecated APIs are being used, and at what rate,
+but they don't include enough information to identify particular clients or API objects.
+Starting in Kubernetes v1.19, [audit events](/docs/tasks/debug-application-cluster/audit/)
+for requests to deprecated APIs include an audit annotation of `"k8s.io/deprecated":"true"`.
+Administrators can use those audit events to identify specific clients or objects that need to be updated.
+
+## Custom Resource Definitions
+
+Along with the API server ability to warn about deprecated API use, starting in v1.19, a CustomResourceDefinition can indicate a 
+[particular version of the resource it defines is deprecated](/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definition-versioning/#version-deprecation).
+When API requests to a deprecated version of a custom resource are made, a warning message is returned, matching the behavior of built-in APIs.
+
+The author of the CustomResourceDefinition can also customize the warning for each version if they want to.
+This allows them to give a pointer to a migration guide or other information if needed.
+
+```yaml
+apiVersion: apiextensions.k8s.io/v1
+kind: CustomResourceDefinition
+  name: crontabs.example.com
+spec:
+  versions:
+  - name: v1alpha1
+    # This indicates the v1alpha1 version of the custom resource is deprecated.
+    # API requests to this version receive a warning in the server response.
+    deprecated: true
+    # This overrides the default warning returned to clients making v1alpha1 API requests.
+    deprecationWarning: "example.com/v1alpha1 CronTab is deprecated; use example.com/v1 CronTab (see http://example.com/v1alpha1-v1)"
+    ...
+
+  - name: v1beta1
+    # This indicates the v1beta1 version of the custom resource is deprecated.
+    # API requests to this version receive a warning in the server response.
+    # A default warning message is returned for this version.
+    deprecated: true
+    ...
+
+  - name: v1
+    ...
+```
+
+## Admission Webhooks
+
+[Admission webhooks](/docs/reference/access-authn-authz/extensible-admission-controllers)
+are the primary way to integrate custom policies or validation with Kubernetes.
+Starting in v1.19, admission webhooks can [return warning messages](/docs/reference/access-authn-authz/extensible-admission-controllers/#response)
+that are passed along to the requesting API client. Warnings can be returned with allowed or rejected admission responses.
+
+As an example, to allow a request but warn about a configuration known not to work well, an admission webhook could send this response:
+
+```json
+{
+  "apiVersion": "admission.k8s.io/v1",
+  "kind": "AdmissionReview",
+  "response": {
+    "uid": "<value from request.uid>",
+    "allowed": true,
+    "warnings": [
+      ".spec.memory: requests >1GB do not work on Fridays"
+    ]
+  }
+}
+```
+
+If you are implementing a webhook that returns a warning message, here are some tips:
+
+* Don't include a "Warning:" prefix in the message (that is added by clients on output)
+* Use warning messages to describe problems the client making the API request should correct or be aware of
+* Be brief; limit warnings to 120 characters if possible
+
+There are many ways admission webhooks could use this new feature, and I'm looking forward to seeing what people come up with.
+Here are a couple ideas to get you started:
+
+* webhook implementations adding a "complain" mode, where they return warnings instead of rejections,
+  to allow trying out a policy to verify it is working as expected before starting to enforce it
+* "lint" or "vet"-style webhooks, inspecting objects and surfacing warnings when best practices are not followed
+
+## Customize Client Handling
+
+Applications that use the `k8s.io/client-go` library to make API requests can customize
+how warnings returned from the server are handled. By default, warnings are logged to
+stderr as they are received, but this behavior can be customized 
+[per-process](https://godoc.org/k8s.io/client-go/rest#SetDefaultWarningHandler)
+or [per-client](https://godoc.org/k8s.io/client-go/rest#Config).
+
+This example shows how to make your application behave like `kubectl`,
+overriding message handling process-wide to deduplicate warnings 
+and highlighting messages using colored output where supported:
+
+```go
+import (
+  "os"
+  "k8s.io/client-go/rest"
+  "k8s.io/kubectl/pkg/util/term"
+  ...
+)
+
+func main() {
+  rest.SetDefaultWarningHandler(
+    rest.NewWarningWriter(os.Stderr, rest.WarningWriterOptions{
+        // only print a given warning the first time we receive it
+        Deduplicate: true,
+        // highlight the output with color when the output supports it
+        Color: term.AllowsColorOutput(os.Stderr),
+      },
+    ),
+  )
+
+  ...
+```
+
+The next example shows how to construct a client that ignores warnings.
+This is useful for clients that operate on metadata for all resource types
+(found dynamically at runtime using the discovery API)
+and do not benefit from warnings about a particular resource being deprecated.
+Suppressing deprecation warnings is not recommended for clients that require use of particular APIs.
+
+```go
+import (
+  "k8s.io/client-go/rest"
+  "k8s.io/client-go/kubernetes"
+)
+
+func getClientWithoutWarnings(config *rest.Config) (kubernetes.Interface, error) {
+  // copy to avoid mutating the passed-in config
+  config = rest.CopyConfig(config)
+  // set the warning handler for this client to ignore warnings
+  config.WarningHandler = rest.NoWarnings{}
+  // construct and return the client
+  return kubernetes.NewForConfig(config)
+}
+```
+
+## Kubectl Strict Mode
+
+If you want to be sure you notice deprecations as soon as possible and get a jump start on addressing them,
+`kubectl` added a `--warnings-as-errors` option in v1.19. When invoked with this option,
+`kubectl` treats any warnings it receives from the server as errors and exits with a non-zero exit code:
+
+<img alt="kubectl applying a manifest file with a --warnings-as-errors flag, displaying a warning message and exiting with a non-zero exit code."
+     src="kubectl-warnings-as-errors.png"
+     style="width:637px;max-width:100%;">
+
+This could be used in a CI job to apply manifests to a current server,
+and required to pass with a zero exit code in order for the CI job to succeed.
+
+## Future Possibilities
+
+Now that we have a way to communicate helpful information to users in context,
+we're already considering other ways we can use this to improve people's experience with Kubernetes.
+A couple areas we're looking at next are warning about [known problematic values](http://issue.k8s.io/64841#issuecomment-395141013)
+we cannot reject outright for compatibility reasons, and warning about use of deprecated fields or field values
+(like selectors using beta os/arch node labels, [deprecated in v1.14](/docs/reference/kubernetes-api/labels-annotations-taints/#beta-kubernetes-io-arch-deprecated)).
+I'm excited to see progress in this area, continuing to make it easier to use Kubernetes.
+
+---
+
+_[Jordan Liggitt](https://twitter.com/liggitt) is a software engineer at Google, and helps lead Kubernetes authentication, authorization, and API efforts._
@@ -0,0 +1,56 @@
+---
+layout: blog
+title: 'Introducing Structured Logs'
+date: 2020-09-04
+slug: kubernetes-1-19-Introducing-Structured-Logs
+---
+
+**Authors:** Marek Siarkowicz (Google), Nathan Beach (Google)
+
+Logs are an essential aspect of observability and a critical tool for debugging. But Kubernetes logs have traditionally been unstructured strings, making any automated parsing difficult and any downstream processing, analysis, or querying challenging to do reliably.
+
+In Kubernetes 1.19, we are adding support for structured logs, which natively support (key, value) pairs and object references. We have also updated many logging calls such that over 99% of logging volume in a typical deployment are now migrated to the structured format.
+
+To maintain backwards compatibility, structured logs will still be outputted as a string where the string contains representations of those "key"="value" pairs. Starting in alpha in 1.19, logs can also be outputted in JSON format using the `--logging-format=json` flag.
+
+## Using Structured Logs
+
+We've added two new methods to the klog library: InfoS and ErrorS. For example, this invocation of InfoS:
+
+```golang
+klog.InfoS("Pod status updated", "pod", klog.KObj(pod), "status", status)
+```
+
+will result in this log:
+
+```
+I1025 00:15:15.525108       1 controller_utils.go:116] "Pod status updated" pod="kube-system/kubedns" status="ready"
+```
+
+Or, if the --logging-format=json flag is set, it will result in this output:
+
+```json
+{
+  "ts": 1580306777.04728,
+  "msg": "Pod status updated",
+  "pod": {
+    "name": "coredns",
+    "namespace": "kube-system"
+  },
+  "status": "ready"
+}
+```
+
+This means downstream logging tools can easily ingest structured logging data and instead of using regular expressions to parse unstructured strings. This also makes processing logs easier, querying logs more robust, and analyzing logs much faster.
+
+With structured logs, all references to Kubernetes objects are structured the same way, so you can filter the output and only log entries referencing the particular pod. You can also find logs indicating how the scheduler was scheduling the pod, how the pod was created, the health probes of the pod, and all other changes in the lifecycle of the pod.
+
+Suppose you are debugging an issue with a pod. With structured logs, you can filter to only those log entries referencing the pod of interest, rather than needing to scan through potentially thousands of log lines to find the relevant ones.
+
+Not only are structured logs more useful when manual debugging of issues, they also enable richer features like automated pattern recognition within logs or tighter correlation of log and trace data.
+
+Finally, structured logs can help reduce storage costs for logs because most storage systems are more efficiently able to compress structured key=value data than unstructured strings.
+
+## Get Involved
+
+While we have updated over 99% of the log entries by log volume in a typical deployment, there are still thousands of logs to be updated. Pick a file or directory that you would like to improve and [migrate existing log calls to use structured logs](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-instrumentation/migration-to-structured-logging.md). It's a great and easy way to make your first contribution to Kubernetes!
@@ -0,0 +1,118 @@
+---
+layout: blog
+title: "GSoC 2020 - Building operators for cluster addons"
+date: 2020-09-16
+slug: gsoc20-building-operators-for-cluster-addons
+---
+
+**Author**: Somtochi Onyekwere
+
+# Introduction
+
+[Google Summer of Code](https://summerofcode.withgoogle.com/) is a global program that is geared towards introducing students to open source. Students are matched with open-source organizations to work with them for three months during the summer.
+
+My name is Somtochi Onyekwere from the Federal University of Technology, Owerri (Nigeria) and this year, I was given the opportunity to work with Kubernetes (under the CNCF organization) and this led to an amazing summer spent learning, contributing and interacting with the community.
+
+Specifically, I worked on the _Cluster Addons: Package all the things!_ project. The project focused on building operators for better management of various cluster addons, extending the tooling for building these operators and making the creation of these operators a smooth process.
+
+# Background
+
+Kubernetes has progressed greatly in the past few years with a flourishing community and a large number of contributors. The codebase is gradually moving away from the monolith structure where all the code resides in the [kubernetes/kubernetes](https://github.com/kubernetes/kubernetes) repository to being split into multiple sub-projects. Part of the focus of cluster-addons is to make some of these sub-projects work together in an easy to assemble, self-monitoring, self-healing and Kubernetes-native way. It enables them to work seamlessly without human intervention.
+
+The community is exploring the use of operators as a mechanism to monitor various resources in the cluster and properly manage these resources. In addition to this, it provides self-healing and it is a kubernetes-native pattern that can encode how best these addons work and manage them properly.
+
+What are cluster addons? Cluster addons are a collection of resources (like Services and deployment) that are used to give a Kubernetes cluster additional functionalities. They range from things as simple as the Kubernetes dashboards (for visualization) to more complex ones like Calico (for networking). These addons are essential to different applications running in the cluster and the cluster itself. The addon operator provides a nicer way of managing these addons and understanding the health and status of the various resources that comprise the addon. You can get a deeper overview in this [article](https://kubernetes.io/docs/concepts/overview/components/#addons).
+
+Operators are custom controllers with custom resource definitions that encode application-specific knowledge and are used for managing complex stateful applications. It is a widely accepted pattern. Managing addons via operators, with these operators encoding knowledge of how best the addons work, introduces a lot of advantages while setting standards that will be easy to follow and scale. This [article](https://kubernetes.io/docs/concepts/extend-kubernetes/operator) does a good job of explaining operators.
+
+The addon operators can solve a lot of problems, but they have their challenges. Those under the [cluster-addons project](https://github.com/kubernetes-sigs/cluster-addons) had missing pieces and were still a proof of concept. Generating the RBAC configuration for the operators was a pain and sometimes the operators were given too much privilege. The operators weren’t very extensible as it only pulled manifests from local filesystems or HTTP(s) servers and a lot of simple addons were generating the same code.
+I spent the summer working on these issues, looking at them with fresh eyes and coming up with solutions for both the known and unknown issues.
+
+# Various additions to kubebuilder-declarative-pattern
+
+The [kubebuilder-declarative-pattern](https://github.com/kubernetes-sigs/kubebuilder-declarative-pattern) (from here on referred to as KDP) repo is an extra layer of addon specific tooling on top of the [kubebuilder](https://github.com/kubernetes-sigs/kubebuilder) SDK that is enabled by passing the experimental `--pattern=addon` flag to `kubebuilder create` command. Together, they create the base code for the addon operator. During the internship, I worked on a couple of features in KDP and cluster-addons.
+
+## Operator version checking
+Enabling version checks for operators helped in making upgrades/downgrades safer to different versions of the addon, even though the operator had complex logic. It is a way of matching the version of an addon to the version of the operator that knows how to manage it well. Most addons have different versions and these versions might need to be managed differently. This feature checks the custom resource for the `addons.k8s.io/min-operator-version` annotation which states the minimum operator version that is needed to manage the version against the version of the operator. If the operator version is below the minimum version required, the operator pauses with an error telling the user that the version of the operator is too low. This helps to ensure that the correct operator is being used for the addon.
+
+## Git repository for storing the manifests
+Previously, there was support for only local file directories and HTTPS repositories for storing manifests. Giving creators of addon operators the ability to store manifest in GitHub repository enables faster development and version control.  When starting the controller, you can pass a flag to specify the location of your channels directory. The channels directory contains the manifests for different versions, the controller pulls the manifest from this directory and applies it to the cluster. During the internship period, I extended it to include Git repositories.
+
+## Annotations to temporarily disable reconciliation
+The reconciliation loop that ensures that the desired state matches the actual state prevents modification of objects in the cluster. This makes it hard to experiment or investigate what might be wrong in the cluster as any changes made are promptly reverted. I resolved this by allowing users to place an `addons.k8s.io/ignore` annotation on the resource that they don’t want the controller to reconcile. The controller checks for this annotation and doesn’t reconcile that object. To resume reconciliation, the annotation can be removed from the resource.
+
+## Unstructured support in kubebuilder-declarative-pattern
+One of the operators that I worked on is a generic controller that could manage more than one cluster addon that did not require extra configuration. To do this, the operator couldn’t use a particular type and needed the kubebuilder-declarative-repo to support using the [unstructured.Unstructured](https://godoc.org/k8s.io/apimachinery/pkg/apis/meta/v1/unstructured#Unstructured) type. There were various functions in the kubebuilder-declarative-pattern that couldn’t handle this type and returned an error if the object passed in was not of type `addonsv1alpha1.CommonObject`. The functions were modified to handle both `unstructured.Unstructured` and `addonsv1alpha.CommonObject`.
+
+# Tools and CLI programs
+There were also some command-line programs I wrote that could be used to make working with addon operators easier. Most of them have uses outside the addon operators as they try to solve a specific problem that could surface anywhere while working with Kubernetes. I encourage you to [check them out](https://github.com/kubernetes-sigs/cluster-addons/tree/master/tools) when you have the chance!
+
+## RBAC Generator
+One of the biggest concerns with the operator was RBAC. You had to manually look through the manifest and add the RBAC rule for each resource as it needs to have RBAC permissions to create, get, update and delete the resources in the manifest when running in-cluster. Building the [RBAC generator](https://github.com/kubernetes-sigs/cluster-addons/blob/master/tools/rbac-gen) automated the process of writing the RBAC roles and role bindings. The function of the RBAC generator is simple. It accepts the file name of the manifest as a flag. Then, it parses the manifest and gets the API group and resource name of the resources and adds it to a role. It outputs the role and role binding to stdout or a file if the `--out` flag is parsed.
+
+Additionally, the tool enables you to split the RBAC by separating the cluster roles in the manifest. This lessened the security concern of an operator being over-privileged as it needed to have all the permissions that the clusterrole has. If you want to apply the clusterrole yourself and not give the operator these permissions, you can pass in a `--supervisory` boolean flag so that the generator does not add these permissions to the role. The CLI program resides [here](https://github.com/kubernetes-sigs/cluster-addons/blob/master/tools/rbac-gen).
+
+## Kubectl Ownerref
+It is hard to find out at a glance which objects were created by an addon custom resource. This kubectl plugin alleviates that pain by displaying all the objects in the cluster that a resource has ownerrefs on. You simply pass the kind and the name of the resource as arguments to the program and it checks the cluster for the objects and gives the kind, name, the namespace of such an object. It could be useful to get a general overview of all the objects that the controller is reconciling by passing in the name and kind of custom resource. The CLI program resides [here](https://github.com/kubernetes-sigs/cluster-addons/tree/master/tools/kubectl-ownerref).
+
+# Addon Operators
+To fully understand addons operators and make changes to how they are being created, you have to try creating and using them. Part of the summer was spent building operators for some popular addons like the Kubernetes dashboard, flannel, NodeLocalDNS and so on. Please check the [cluster-addons](https://github.com/kubernetes-sigs/cluster-addons) repository for the different addon operators. In this section, I will just highlight one that is a little different from the others.
+
+## Generic Controller
+The generic controller can be shared between addons that don’t require much configuration. This minimizes resource consumption on the cluster as it reduces the number of controllers that need to be run. Also instead of building your own operator, you can just use the generic controller and whenever you feel that your needs have grown and you need a more complex operator, you can always scaffold the code with kubebuilder and continue from where the generic operator stopped. To use the generic controller, you can generate the CustomResourceDefinition(CRD) using this tool ([generic-addon](https://github.com/kubernetes-sigs/cluster-addons/blob/master/tools/generic-addon/README.md)). You pass in the kind, group, and the location of your channels directory (it could be a Git repository too!). The tool generates the - CRD, RBAC manifest and two custom resources for you.
+
+The process is as follows:
+- Create the Generic CRD
+- Generate all the manifests needed with the [`generic-addon tool`](https://github.com/kubernetes-sigs/cluster-addons/blob/master/tools/generic-addon/README.md).
+
+This tool creates:
+1. The CRD for your addon
+2. The RBAC rules for the CustomResourceDefinitions
+3. The RBAC rules for applying the manifests
+4. The custom resource for your addon
+5. A Generic custom resource
+
+The Generic custom resource looks like this:
+
+```yaml
+apiVersion: addons.x-k8s.io/v1alpha1
+kind: Generic
+metadata:
+ 	name: generic-sample
+spec:
+  objectKind:
+  kind: NodeLocalDNS
+  version: "v1alpha1"
+  group: addons.x-k8s.io
+channel: "../nodelocaldns/channels"
+```
+
+Apply these manifests but ensure to apply the CRD before the CR.
+Then, run the Generic controller, either on your machine or in-cluster.
+
+
+If you are interested in building an operator, Please check out [this guide](https://github.com/kubernetes-sigs/cluster-addons/blob/master/dashboard/README.md).
+
+# Relevant Links
+- [Detailed breakdown of work done during the internship](https://github.com/SomtochiAma/gsoc-2020-meta-k8s)
+- [Addon Operator (KEP)](https://github.com/kubernetes/enhancements/blob/master/keps/sig-cluster-lifecycle/addons/0035-20190128-addons-via-operators.md)
+- [Original GSoC Issue](https://github.com/kubernetes-sigs/cluster-addons/issues/39)
+- [Proposal Submitted for GSoC](https://github.com/SomtochiAma/gsoc-2020-meta-k8s/blob/master/GSoC%202020%20PROPOSAL%20-%20PACKAGE%20ALL%20THINGS.pdf)
+- [All commits to kubernetes-sigs/cluster-addons](https://github.com/kubernetes-sigs/cluster-addons/commits?author=SomtochiAma)
+- [All commits to kubernetes-sigs/kubebuidler-declarative-pattern](https://github.com/kubernetes-sigs/kubebuilder-declarative-pattern/commits?author=SomtochiAma)
+
+# Further Work
+A lot of work was definitely done on the cluster addons during the GSoC period. But we need more people building operators and using them in the cluster. We need wider adoption in the community. Build operators for your favourite addons and tell us how it went and if you had any issues. Check out this [README.md](https://github.com/kubernetes-sigs/cluster-addons/blob/master/dashboard/README.md) to get started.
+
+# Appreciation
+I really want to appreciate my mentors [Justin Santa Barbara](https://github.com/justinsb) (Google) and [Leigh Capili](https://github.com/stealthybox) (Weaveworks). My internship was awesome because they were awesome. They set a golden standard for what mentorship should be. They were accessible and always available to clear any confusion. I think what I liked best was that they didn’t just dish out tasks, instead, we had open discussions about what was wrong and what could be improved. They are really the best and I hope I get to work with them again!
+  Also, I want to say a huge thanks to [Lubomir I. Ivanov](https://github.com/neolit123) for reviewing this blog post!
+
+# Conclusion
+So far I have learnt a lot about Go, the internals of Kubernetes, and operators. I want to conclude by encouraging people to contribute to open-source (especially Kubernetes :)) regardless of your level of experience. It has been a well-rounded experience for me and I have come to love the community. It is a great initiative and it is a great way to learn and meet awesome people. Special shoutout to Google for organizing this program.
+
+If you are interested in cluster addons and finding out more on addon operators, you are welcome to join our slack channel on the Kubernetes [#cluster-addons](https://kubernetes.slack.com/messages/cluster-addons).
+
+---
+
+_[Somtochi Onyekwere](https://twitter.com/SomtochiAma) is a software engineer that loves contributing to open-source and exploring cloud native solutions._
@@ -0,0 +1,107 @@
+---
+title: "Contributing to the Development Guide"
+linkTitle: "Contributing to the Development Guide"
+Author: Erik L. Arneson
+Description: "A new contributor describes the experience of writing and submitting changes to the Kubernetes Development Guide."
+date: 2020-10-01
+canonicalUrl: https://www.kubernetes.dev/blog/2020/09/28/contributing-to-the-development-guide/
+resources:
+- src: "jorge-castro-code-of-conduct.jpg"
+  title: "Jorge Castro announcing the Kubernetes Code of Conduct during a weekly SIG ContribEx meeting."
+---
+
+When most people think of contributing to an open source project, I suspect they probably think of
+contributing code changes, new features, and bug fixes. As a software engineer and a long-time open
+source user and contributor, that's certainly what I thought. Although I have written a good quantity
+of documentation in different workflows, the massive size of the Kubernetes community was a new kind 
+of "client." I just didn't know what to expect when Google asked my compatriots and me at
+[Lion's Way](https://lionswaycontent.com/) to make much-needed updates to the Kubernetes Development Guide.
+
+*This article originally appeared on the [Kubernetes Contributor Community blog](https://www.kubernetes.dev/blog/2020/09/28/contributing-to-the-development-guide/).*
+
+## The Delights of Working With a Community
+
+As professional writers, we are used to being hired to write very specific pieces. We specialize in
+marketing, training, and documentation for technical services and products, which can range anywhere from relatively fluffy marketing emails to deeply technical white papers targeted at IT and developers. With 
+this kind of professional service, every deliverable tends to have a measurable return on investment. 
+I knew this metric wouldn't be present when working on open source documentation, but I couldn't
+predict how it would change my relationship with the project.
+
+One of the primary traits of the relationship between our writing and our traditional clients is that we
+always have one or two primary points of contact inside a company. These contacts are responsible
+for reviewing our writing and making sure it matches the voice of the company and targets the
+audience they're looking for. It can be stressful -- which is why I'm so glad that my writing
+partner, eagle-eyed reviewer, and bloodthirsty editor [Joel](https://twitter.com/JoelByronBarker)
+handles most of the client contact.
+
+I was surprised and delighted that all of the stress of client contact went out the window when
+working with the Kubernetes community.
+
+"How delicate do I have to be? What if I screw up? What if I make a developer angry? What if I make
+enemies?" These were all questions that raced through my mind and made me feel like I was
+approaching a field of eggshells when I first joined the `#sig-contribex` channel on the Kubernetes
+Slack and announced that I would be working on the
+[Development Guide](https://github.com/kubernetes/community/blob/master/contributors/devel/development.md).
+
+{{< imgproc jorge-castro-code-of-conduct Fit "800x450" >}}
+"The Kubernetes Code of Conduct is in effect, so please be excellent to each other." &mdash; Jorge
+Castro, SIG ContribEx co-chair
+{{< /imgproc >}}
+
+My fears were unfounded. Immediately, I felt welcome. I like to think this isn't just because I was
+working on a much needed task, but rather because the Kubernetes community is filled
+with friendly, welcoming people. During the weekly SIG ContribEx meetings, our reports on progress
+with the Development Guide were included immediately. In addition, the leader of the meeting would
+always stress that the [Kubernetes Code of Conduct](https://www.kubernetes.dev/resources/code-of-conduct/) was in
+effect, and that we should, like Bill and Ted, be excellent to each other.
+
+## This Doesn't Mean It's All Easy
+
+The Development Guide needed a pretty serious overhaul. When we got our hands on it, it was already
+packed with information and lots of steps for new developers to go through, but it was getting dusty
+with age and neglect. Documentation can really require a global look, not just point fixes.
+As a result, I ended up submitting a gargantuan pull request to the
+[Community repo](https://github.com/kubernetes/community): 267 additions and 88 deletions.
+
+The life cycle of a pull request requires a certain number of Kubernetes organization members to review and approve changes
+before they can be merged. This is a great practice, as it keeps both documentation and code in
+pretty good shape, but it can be tough to cajole the right people into taking the time for such a hefty
+review. As a result, that massive PR took 26 days from my first submission to final merge. But in
+the end, [it was successful](https://github.com/kubernetes/community/pull/5003).
+
+Since Kubernetes is a pretty fast-moving project, and since developers typically aren't really
+excited about writing documentation, I also ran into the problem that sometimes, the secret jewels
+that describe the workings of a Kubernetes subsystem are buried deep within the [labyrinthine mind of
+a brilliant engineer](https://github.com/amwat), and not in plain English in a Markdown file. I ran headlong into this issue
+when it came time to update the getting started documentation for end-to-end (e2e) testing. 
+
+This portion of my journey took me out of documentation-writing territory and into the role of a
+brand new user of some unfinished software. I ended up working with one of the developers of the new
+[`kubetest2` framework](https://github.com/kubernetes-sigs/kubetest2) to document the latest process of
+getting up-and-running for e2e testing, but it required a lot of head scratching on my part. You can
+judge the results for yourself by checking out my
+[completed pull request](https://github.com/kubernetes/community/pull/5045).
+
+## Nobody Is the Boss, and Everybody Gives Feedback
+
+But while I secretly expected chaos, the process of contributing to the Kubernetes Development Guide
+and interacting with the amazing Kubernetes community went incredibly smoothly. There was no
+contention. I made no enemies. Everybody was incredibly friendly and welcoming. It was *enjoyable*.
+
+With an open source project, there is no one boss. The Kubernetes project, which approaches being
+gargantuan, is split into many different special interest groups (SIGs), working groups, and
+communities. Each has its own regularly scheduled meetings, assigned duties, and elected
+chairpersons. My work intersected with the efforts of both SIG ContribEx (who watch over and seek to
+improve the contributor experience) and SIG Testing (who are in charge of testing). Both of these
+SIGs proved easy to work with, eager for contributions, and populated with incredibly friendly and
+welcoming people.
+
+In an active, living project like Kubernetes, documentation continues to need maintenance, revision,
+and testing alongside the code base. The Development Guide will continue to be crucial to onboarding
+new contributors to the Kubernetes code base, and as our efforts have shown, it is important that
+this guide keeps pace with the evolution of the Kubernetes project.
+
+Joel and I really enjoy interacting with the Kubernetes community and contributing to
+the Development Guide. I really look forward to continuing to not only contributing more, but to
+continuing to build the new friendships I've made in this vast open source community over the past
+few months.
@@ -0,0 +1,43 @@
+---
+layout: blog
+title: "Announcing the 2020 Steering Committee Election Results"
+date: 2020-10-12
+slug: steering-committee-results-2020
+---
+
+**Author**: Kaslin Fields
+
+The [2020 Steering Committee Election](https://github.com/kubernetes/community/tree/master/events/elections/2020) is now complete. In 2019, the committee arrived at its final allocation of 7 seats, 3 of which were up for election in 2020. Incoming committee members serve a term of 2 years, and all members are elected by the Kubernetes Community.
+
+This community body is significant since it oversees the governance of the entire Kubernetes project. With that great power comes great responsibility. You can learn more about the steering committee’s role in their [charter](https://github.com/kubernetes/steering/blob/master/charter.md).
+
+## Results
+
+Congratulations to the elected committee members whose two year terms begin immediately (listed in alphabetical order by GitHub handle):
+
+* **Davanum Srinivas ([@dims](https://github.com/dims)), VMware**
+* **Jordan Liggitt ([@liggitt](https://github.com/liggitt)), Google**
+* **Bob Killen ([@mrbobbytables](https://github.com/mrbobbytables)), Google**
+
+They join continuing members Christoph Blecker ([@cblecker](https://github.com/cblecker)), Red Hat; Derek Carr ([@derekwaynecarr](https://github.com/derekwaynecarr)), Red Hat; Nikhita Raghunath ([@nikhita](https://github.com/nikhita)), VMware; and Paris Pittman ([@parispittman](https://github.com/parispittman)), Apple. Davanum Srinivas is returning for his second term on the committee.
+
+## Big Thanks!
+
+* Thank you and congratulations on a successful election to this round’s election officers:
+  * Jaice Singer DuMars ([@jdumars](https://github.com/jdumars)), Apple
+  * Ihor Dvoretskyi ([@idvoretskyi](https://github.com/idvoretskyi)), CNCF
+  * Josh Berkus ([@jberkus](https://github.com/jberkus)), Red Hat
+* Thanks to the Emeritus Steering Committee Members. Your prior service is appreciated by the community:
+  * Aaron Crickenberger ([@spiffxp](https://github.com/spiffxp)), Google
+  * and Lachlan Evenson([@lachie8e)](https://github.com/lachie8e)), Microsoft
+* And thank you to all the candidates who came forward to run for election. As [Jorge Castro put it](https://twitter.com/castrojo/status/1315718627639820288?s=20): we are spoiled with capable, kind, and selfless volunteers who put the needs of the project first.
+
+## Get Involved with the Steering Committee
+
+This governing body, like all of Kubernetes, is open to all. You can follow along with Steering Committee [backlog items](https://github.com/kubernetes/steering/projects/1) and weigh in by filing an issue or creating a PR against their [repo](https://github.com/kubernetes/steering). They have an open meeting on [the first Monday of the month at 6pm UTC](https://github.com/kubernetes/steering) and regularly attend Meet Our Contributors. They can also be contacted at their public mailing list steering@kubernetes.io.
+
+You can see what the Steering Committee meetings are all about by watching past meetings on the [YouTube Playlist](https://www.youtube.com/playlist?list=PL69nYSiGNLP1yP1B_nd9-drjoxp0Q14qM).
+
+----
+
+_This post was written by the [Upstream Marketing Working Group](https://github.com/kubernetes/community/tree/master/communication/marketing-team#contributor-marketing). If you want to write stories about the Kubernetes community, learn more about us._
@@ -0,0 +1,16 @@
+---
+layout: blog
+title: "Remembering Dan Kohn"
+date: 2020-11-02
+slug: remembering-dan-kohn
+---
+
+**Author**: The Kubernetes Steering Committee
+
+Dan Kohn was instrumental in getting Kubernetes and CNCF community to where it is today. He shared our values, motivations, enthusiasm, community spirit, and helped the Kubernetes community to become the best that it could be. Dan loved getting people together to solve problems big and small. He enabled people to grow their individual scope in the community which often helped launch their career in open source software.
+
+Dan built a coalition around the nascent Kubernetes project and turned that into a cornerstone to build the larger cloud native space. He loved challenges, especially ones where the payoff was great like building worldwide communities, spreading the love of open source, and helping diverse, underprivileged communities and students to get a head start in technology.
+
+Our heart goes out to his family. Thank you, Dan, for bringing your boys to events in India and elsewhere as we got to know how great you were as a father. Dan, your thoughts and ideas will help us make progress in our journey as a community. Thank you for your life's work!
+
+If Dan has made an impact on you in some way, please consider adding a memory of him in his [CNCF memorial](https://github.com/cncf/memorials/blob/master/dan-kohn.md).
@@ -0,0 +1,57 @@
+---
+layout: blog
+title: "Cloud native security for your clusters"
+date: 2020-11-18
+slug: cloud-native-security-for-your-clusters
+---
+
+**Author**: [Pushkar Joglekar](https://twitter.com/pudijoglekar)
+
+Over the last few years a small, security focused community has been working diligently to deepen our understanding of security, given the evolving cloud native infrastructure and corresponding iterative deployment practices. To enable sharing of this knowledge with the rest of the community, members of [CNCF SIG Security](https://github.com/cncf/sig-security) (a group which reports into [CNCF TOC](https://github.com/cncf/toc#sigs) and who are friends with [Kubernetes SIG Security](https://github.com/kubernetes/community/tree/master/sig-security)) led by Emily Fox, collaborated on a whitepaper outlining holistic cloud native security concerns and best practices. After over 1200 comments, changes, and discussions from 35 members across the world, we are proud to share [cloud native security whitepaper v1.0](https://www.cncf.io/blog/2020/11/18/announcing-the-cloud-native-security-white-paper) that serves as essential reading for security leadership in enterprises, financial and healthcare industries, academia, government, and non-profit organizations.
+
+The paper attempts to _not_ focus on any specific [cloud native project](https://www.cncf.io/projects/). Instead, the intent is to model and inject security into four logical phases of cloud native application lifecycle: _Develop, Distribute, Deploy, and Runtime_.
+
+<img alt="Cloud native application lifecycle phases"
+     src="cloud-native-app-lifecycle-phases.svg"
+     style="width:60em;max-width:100%;">
+
+
+## Kubernetes native security controls
+When using Kubernetes as a workload orchestrator, some of the security controls this version of the whitepaper recommends are:
+* [Pod Security Policies](/docs/concepts/policy/pod-security-policy/): Implement a single source of truth for “least privilege” workloads across the entire cluster
+* [Resource requests and limits](/docs/concepts/configuration/manage-resources-containers/#requests-and-limits): Apply requests (soft constraint) and limits (hard constraint) for shared resources such as memory and CPU
+* [Audit log analysis](/docs/tasks/debug-application-cluster/audit/): Enable Kubernetes API auditing and filtering for security relevant events
+* [Control plane authentication and certificate root of trust](/docs/concepts/architecture/control-plane-node-communication/): Enable mutual TLS authentication with a trusted CA for communication within the cluster
+* [Secrets management](/docs/concepts/configuration/secret/): Integrate with a built-in or external secrets store
+
+## Cloud native complementary security controls
+
+Kubernetes has direct involvement in the _deploy_ phase and to a lesser extent in the _runtime_ phase. Ensuring the artifacts are securely _developed_ and _distributed_ is necessary for, enabling workloads in Kubernetes to run “secure by default”. Throughout all phases of the Cloud native application life cycle, several complementary security controls exist for Kubernetes orchestrated workloads, which includes but are not limited to:
+* Develop:
+  - Image signing and verification
+  - Image vulnerability scanners
+* Distribute:
+  - Pre-deployment checks for detecting excessive privileges
+  - Enabling observability and logging
+* Deploy:
+  - Using a service mesh for workload authentication and authorization
+  - Enforcing “default deny” network policies for inter-workload communication via [network plugins](/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins/)
+* Runtime:
+  - Deploying security monitoring agents for workloads
+  - Isolating applications that run on the same node using SELinux, AppArmor, etc.
+  - Scanning configuration against recognized secure baselines for node, workload and orchestrator
+
+## Understand first, secure next
+The cloud native way, including containers, provides great security benefits for its users: immutability, modularity, faster upgrades and consistent state across the environment. Realizing this fundamental change in “the way things are done”, motivates us to look at security with a cloud native lens. One of the things that was evident for all the authors of the paper was the fact that it’s tough to make smarter decisions on how and what to secure in a cloud native ecosystem if you do not understand the tools, patterns, and frameworks at hand (in addition to knowing your own critical assets). Hence, for all the security practitioners out there who want to be partners rather than a gatekeeper for your friends in Operations, Product Development, and Compliance, let’s make an attempt to _learn more so we can secure better_.
+
+We recommend following this **7 step R.U.N.T.I.M.E. path** to get started on cloud native security:
+1. <b>R</b>ead the paper and any linked material in it
+2. <b>U</b>nderstand challenges and constraints for your environment
+3. <b>N</b>ote the content and controls that apply to your environment
+4. <b>T</b>alk about your observations with your peers
+5. <b>I</b>nvolve your leadership and ask for help
+6. <b>M</b>ake a risk profile based on existing and missing security controls
+7. <b>E</b>xpend time, money, and resources that improve security posture and reduce risk where appropriate.
+
+## Acknowledgements
+Huge shout out to _Emily Fox, Tim Bannister (The Scale Factory), Chase Pettet (Mirantis), and Wayne Haber (GitLab)_ for contributing with their wonderful suggestions for this blog post.
@@ -0,0 +1,180 @@
+---
+layout: blog
+title: "Dockershim Deprecation FAQ"
+date: 2020-12-02
+slug: dockershim-faq
+aliases: [ '/dockershim' ]
+---
+
+This document goes over some frequently asked questions regarding the Dockershim
+deprecation announced as a part of the Kubernetes v1.20 release. For more detail
+on the deprecation of Docker as a container runtime for Kubernetes kubelets, and
+what that means, check out the blog post
+[Don't Panic: Kubernetes and Docker](/blog/2020/12/02/dont-panic-kubernetes-and-docker/).
+
+### Why is dockershim being deprecated?
+
+Maintaining dockershim has become a heavy burden on the Kubernetes maintainers.
+The CRI standard was created to reduce this burden and allow smooth interoperability
+of different container runtimes. Docker itself doesn't currently implement CRI,
+thus the problem.
+
+Dockershim was always intended to be a temporary solution (hence the name: shim).
+You can read more about the community discussion and planning in the
+[Dockershim Removal Kubernetes Enhancement Proposal][drkep].
+
+Additionally, features that were largely incompatible with the dockershim, such
+as cgroups v2 and user namespaces are being implemented in these newer CRI
+runtimes. Removing support for the dockershim will allow further development in
+those areas.
+
+[drkep]: https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/1985-remove-dockershim
+
+### Can I still use Docker in Kubernetes 1.20?
+
+Yes, the only thing changing in 1.20 is a single warning log printed at [kubelet]
+startup if using Docker as the runtime.
+
+[kubelet]: /docs/reference/command-line-tools-reference/kubelet/
+
+
+### When will dockershim be removed?
+
+Given the impact of this change, we are using an extended deprecation timeline.
+It will not be removed before Kubernetes 1.22, meaning the earliest release without
+dockershim would be 1.23 in late 2021. We will be working closely with vendors
+and other ecosystem groups to ensure a smooth transition and will evaluate things
+as the situation evolves.
+
+
+### Will my existing Docker images still work?
+
+Yes, the images produced from `docker build` will work with all CRI implementations.
+All your existing images will still work exactly the same.
+
+
+### What about private images?
+
+Also yes. All CRI runtimes support the same pull secrets configuration used in
+Kubernetes, either via the PodSpec or ServiceAccount.
+
+
+### Are Docker and containers the same thing?
+
+Docker popularized the Linux containers pattern and has been instrumental in
+developing the underlying technology, however containers in Linux have existed
+for a long time. The container ecosystem has grown to be much broader than just
+Docker. Standards like OCI and CRI have helped many tools grow and thrive in our
+ecosystem, some replacing aspects of Docker while others enhance existing
+functionality.
+
+
+### Are there examples of folks using other runtimes in production today?
+
+All Kubernetes project produced artifacts (Kubernetes binaries) are validated
+with each release.
+
+Additionally, the [kind] project has been using containerd for some time and has
+seen an improvement in stability for its use case. Kind and containerd are leveraged
+multiple times every day to validate any changes to the Kubernetes codebase. Other
+related projects follow a similar pattern as well, demonstrating the stability and
+usability of other container runtimes. As an example, OpenShift 4.x has been
+using the [CRI-O] runtime in production since June 2019.
+
+For other examples and references you can look at the adopters of containerd and
+cri-o, two container runtimes under the Cloud Native Computing Foundation ([CNCF]).
+- [containerd](https://github.com/containerd/containerd/blob/master/ADOPTERS.md)
+- [CRI-O](https://github.com/cri-o/cri-o/blob/master/ADOPTERS.md)
+
+[CRI-O]: https://cri-o.io/
+[kind]: https://kind.sigs.k8s.io/
+[CNCF]: https://cncf.io
+
+
+### People keep referencing OCI, what is that?
+
+OCI stands for the [Open Container Initiative], which standardized many of the
+interfaces between container tools and technologies. They maintain a standard
+specification for packaging container images (OCI image-spec) and running containers
+(OCI runtime-spec). They also maintain an actual implementation of the runtime-spec
+in the form of [runc], which is the underlying default runtime for both
+[containerd] and [CRI-O]. The CRI builds on these low-level specifications to
+provide an end-to-end standard for managing containers.
+
+[Open Container Initiative]: https://opencontainers.org/about/overview/
+[runc]: https://github.com/opencontainers/runc
+[containerd]: https://containerd.io/
+
+
+### Which CRI implementation should I use?
+
+That’s a complex question and it depends on a lot of factors. If Docker is
+working for you, moving to containerd should be a relatively easy swap and
+has have strictly better performance and less overhead. However we encourage you
+to explore all the options from the [CNCF landscape] in case another would be an
+even better fit for your environment.
+
+[CNCF landscape]: https://landscape.cncf.io/category=container-runtime&format=card-mode&grouping=category
+
+
+### What should I look out for when changing CRI implementations?
+
+While the underlying containerization code is the same between Docker and most
+CRIs (including containerd), there are a few differences around the edges. Some
+common things to consider when migrating are:
+
+- Logging configuration
+- Runtime resource limitations
+- Node provisioning scripts that call docker or use docker via it's control socket
+- Kubectl plugins that require docker CLI or the control socket
+- Kubernetes tools that require direct access to Docker (e.g. kube-imagepuller)
+- Configuration of functionality like `registry-mirrors` and insecure registries 
+- Other support scripts or daemons that expect docker to be available and are run
+  outside of Kubernetes (e.g. monitoring or security agents)
+- GPUs or special hardware and how they integrate with your runtime and Kubernetes
+
+If you use Kubernetes resource requests/limits or file-based log collection
+DaemonSets then they will continue to work the same, but if you’ve customized
+your dockerd configuration, you’ll need to adapt that for your new container
+runtime where possible.
+
+Another thing to look out for is anything expecting to run for system maintenance
+or nested inside a container when building images will no longer work. For the
+former, you can use the [`crictl`][cr] tool as a drop-in replacement and for the
+latter you can use newer container build options like [img], [buildah], or
+[kaniko] that don’t require Docker.
+
+[cr]: https://github.com/kubernetes-sigs/cri-tools
+[img]: https://github.com/genuinetools/img
+[buildah]: https://github.com/containers/buildah
+[kaniko]: https://github.com/GoogleContainerTools/kaniko
+
+For containerd, you can start with their [documentation] to see what configuration
+options are available as you migrate things over.
+
+[documentation]: https://github.com/containerd/cri/blob/master/docs/registry.md
+
+For instructions on how to use containerd and CRI-O with Kubernetes, see the
+Kubernetes documentation on [Container Runtimes]
+
+[Container Runtimes]: /docs/setup/production-environment/container-runtimes
+
+
+### What if I have more questions?
+
+If you use a vendor-supported Kubernetes distribution, you can ask them about
+upgrade plans for their products. For end-user questions, please post them
+to our end user community forum: https://discuss.kubernetes.io/. 
+
+You can also check out the excellent blog post
+[Wait, Docker is deprecated in Kubernetes now?][dep] a more in-depth technical
+discussion of the changes.
+
+[dep]: https://dev.to/inductor/wait-docker-is-deprecated-in-kubernetes-now-what-do-i-do-e4m
+
+
+### Can I have a hug?
+
+Always and whenever you want!  🤗🤗
+
+
@@ -0,0 +1,104 @@
+---
+layout: blog
+title: "Don't Panic: Kubernetes and Docker"
+date: 2020-12-02
+slug: dont-panic-kubernetes-and-docker
+---
+
+**Authors:** Jorge Castro, Duffie Cooley, Kat Cosgrove, Justin Garrison, Noah Kantrowitz, Bob Killen, Rey Lejano, Dan “POP” Papandrea, Jeffrey Sica, Davanum “Dims” Srinivas
+
+Kubernetes is [deprecating
+Docker](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.20.md#deprecation)
+as a container runtime after v1.20.
+
+**You do not need to panic. It’s not as dramatic as it sounds.**
+
+tl;dr Docker as an underlying runtime is being deprecated in favor of runtimes
+that use the [Container Runtime Interface(CRI)](https://kubernetes.io/blog/2016/12/container-runtime-interface-cri-in-kubernetes/)
+created for Kubernetes. Docker-produced images will continue to work in your
+cluster with all runtimes, as they always have.
+
+If you’re an end-user of Kubernetes, not a whole lot will be changing for you.
+This doesn’t mean the death of Docker, and it doesn’t mean you can’t, or
+shouldn’t, use Docker as a development tool anymore. Docker is still a useful
+tool for building containers, and the images that result from running `docker
+build` can still run in your Kubernetes cluster. 
+ 
+If you’re using a managed Kubernetes service like GKE, EKS, or AKS (which [defaults to containerd](https://github.com/Azure/AKS/releases/tag/2020-11-16)) you will need to
+make sure your worker nodes are using a supported container runtime before
+Docker support is removed in a future version of Kubernetes. If you have node
+customizations you may need to update them based on your environment and runtime
+requirements. Please work with your service provider to ensure proper upgrade
+testing and planning. 
+
+If you’re rolling your own clusters, you will also need to make changes to avoid
+your clusters breaking. At v1.20, you will get a deprecation warning for Docker.
+When Docker runtime support is removed in a future release (currently planned
+for the 1.22 release in late 2021) of Kubernetes it will no longer be supported
+and you will need to switch to one of the other compliant container runtimes,
+like containerd or CRI-O. Just make sure that the runtime you choose supports
+the docker daemon configurations you currently use (e.g. logging).
+
+## So why the confusion and what is everyone freaking out about?
+
+We’re talking about two different environments here, and that’s creating
+confusion. Inside of your Kubernetes cluster, there’s a thing called a container
+runtime that’s responsible for pulling and running your container images. Docker
+is a popular choice for that runtime (other common options include containerd
+and CRI-O), but Docker was not designed to be embedded inside Kubernetes, and
+that causes a problem. 
+
+You see, the thing we call “Docker” isn’t actually one thing -- it’s an entire
+tech stack, and one part of it is a thing called “containerd,” which is a
+high-level container runtime by itself. Docker is cool and useful because it has
+a lot of UX enhancements that make it really easy for humans to interact with
+while we’re doing development work, but those UX enhancements aren’t necessary
+for Kubernetes, because it isn’t a human. 
+
+As a result of this human-friendly abstraction layer, your Kubernetes cluster
+has to use another tool called Dockershim to get at what it really needs, which
+is containerd. That’s not great, because it gives us another thing that has to
+be maintained and can possibly break. What’s actually happening here is that
+Dockershim is being removed from Kubelet as early as v1.23 release, which
+removes support for Docker as a container runtime as a result. You might be
+thinking to yourself, but if containerd is included in the Docker stack, why
+does Kubernetes need the Dockershim?
+
+Docker isn’t compliant with CRI, the [Container Runtime Interface](https://kubernetes.io/blog/2016/12/container-runtime-interface-cri-in-kubernetes/).
+If it were, we wouldn’t need the shim, and this wouldn’t be a thing. But it’s
+not the end of the world, and you don’t need to panic -- you just need to change
+your container runtime from Docker to another supported container runtime.
+
+One thing to note: If you are relying on the underlying docker socket
+(/var/run/docker.sock) as part of a workflow within your cluster today, moving
+to a different runtime will break your ability to use it. This pattern is often
+called Docker in Docker. There are lots of options out there for this specific
+use case including things like
+[kaniko](https://github.com/GoogleContainerTools/kaniko),
+[img](https://github.com/genuinetools/img), and
+[buildah](https://github.com/containers/buildah). 
+ 
+## What does this change mean for developers, though? Do we still write Dockerfiles? Do we still build things with Docker?
+ 
+This change addresses a different environment than most folks use to interact
+with Docker. The Docker installation you’re using in development is unrelated to
+the Docker runtime inside your Kubernetes cluster. It’s confusing, I know. As a
+developer, Docker is still useful to you in all the ways it was before this
+change was announced. The image that Docker produces isn’t really a
+Docker-specific image -- it’s an OCI ([Open Container Initiative](https://opencontainers.org/)) image. 
+Any OCI-compliant image, regardless of the tool you use to build it, will look
+the same to Kubernetes. Both [containerd](https://containerd.io/) and
+[CRI-O](https://cri-o.io/) know how to pull those images and run them. This is
+why we have a standard for what containers should look like.
+
+So, this change is coming. It’s going to cause issues for some, but it isn’t
+catastrophic, and generally it’s a good thing. Depending on how you interact
+with Kubernetes, this could mean nothing to you, or it could mean a bit of work.
+In the long run, it’s going to make things easier. If this is still confusing
+for you, that’s okay -- there’s a lot going on here, Kubernetes has a lot of
+moving parts, and nobody is an expert in 100% of it. We encourage any and all
+questions regardless of experience level or complexity! Our goal is to make sure
+everyone is educated as much as possible on the upcoming changes. `<3` We hope
+this has answered most of your questions and soothed some anxieties!
+
+Looking for more answers? Check out our accompanying [Dockershim Deprecation FAQ](/blog/2020/12/02/dockershim-faq/).
@@ -0,0 +1,92 @@
+---
+layout: blog
+title: "GSoD 2020: Improving the API Reference Experience"
+date: 2020-12-04
+slug: gsod-2020-improving-api-reference-experience
+---
+
+**Author**: [Philippe Martin](https://github.com/feloy)
+
+_Editor's note: Better API references have been my goal since I joined Kubernetes docs three and a half years ago. Philippe has succeeded fantastically. More than a better API reference, though, Philippe embodied the best of the Kubernetes community in this project: excellence through collaboration, and a process that made the community itself better. Thanks, Google Season of Docs, for making Philippe's work possible. —Zach Corleissen_
+
+## Introduction
+
+The [Google Season of Docs](https://developers.google.com/season-of-docs) project brings open source organizations and technical writers together to work closely on a specific documentation project.
+
+I was selected by the CNCF to work on Kubernetes documentation, specifically to make the API Reference documentation more accessible.
+
+I'm a software developer with a great interest in documentation systems. In the late 90's I started translating Linux-HOWTO documents into French. From one thing to another, I learned about documentation systems. Eventually, I wrote a Linux-HOWTO to help documentarians learn the language used at that time for writing documents, LinuxDoc/SGML.
+
+Shortly afterward, Linux documentation adopted the DocBook language. I helped some writers rewrite their documents in this format; for example, the Advanced Bash-Scripting Guide. I also worked on the GNU `makeinfo` program to add DocBook output, making it possible to transform *GNU Info* documentation into Docbook format.
+
+## Background
+
+The [Kubernetes website](https://kubernetes.io/docs/home/) is built with Hugo from documentation written in Markdown format in the [website repository](https://github.com/kubernetes/website), using the [Docsy Hugo theme](https://www.docsy.dev/about/).
+
+The existing API reference documentation is a large HTML file generated from the Kubernetes OpenAPI specification.
+
+On my side, I wanted for some time to make the API Reference more accessible, by:
+- building individual and autonomous pages for each Kubernetes resource
+- adapting the format to mobile reading
+- reusing the website's assets and theme to build, integrate, and display the reference pages
+- allowing the search engines to reference the content of the pages
+
+Around one year ago, I started to work on the generator building the current unique HTML page, to add a DocBook output, so the API Reference could be generated first in DocBook format, and after that in PDF or other formats supported by DocBook processors. The first result has been some [Ebook files for the API Reference](https://github.com/feloy/kubernetes-resources-reference/releases) and an auto-edited paper book.
+
+I decided later to add another output to this generator, to generate Markdown files and create [a website with the API Reference](https://web.archive.org/web/20201022201911/https://www.k8sref.io/docs/workloads/).
+
+When the CNCF proposed a project for the Google Season of Docs to work on the API Reference, I applied, and the match occurred.
+
+## The Project
+
+### swagger-ui
+
+The first idea of the CNCF members that proposed this project was to test the [`swagger-ui` tool](https://swagger.io/tools/swagger-ui/), to try and document the Kubernetes API Reference with this standard tool. 
+
+Because the Kubernetes API is much larger than many other APIs, it has been necessary to write a tool to split the complete API Reference by API Groups, and insert in the Documentation website several `swagger-ui` components, one for each API Group.
+
+Generally, APIs are used by developers by calling endpoints with a specific HTTP verb, with specific parameters and waiting for a response. The `swagger-ui` interface is built for this usage: the interface displays a list of endpoints and their associated verbs, and for each the parameters and responses formats. 
+
+The Kubernetes API is most of the time used differently: users create manifest files containing resources definitions in YAML format, and use the `kubectl` CLI to *apply* these manifests to the cluster. In this case, the most important information is the description of the structures used as parameters and responses (the Kubernetes Resources).
+
+Because of this specificity, we realized that it would be difficult to adapt the `swagger-ui` interface to satisfy the users of the Kubernetes API and this direction has been abandoned.
+
+### Markdown pages
+
+The second stage of the project has been to adapt the work I had done to create the k8sref.io website, to include it in the official documentation website.
+
+The main changes have been to:
+- use go-templates to represent the output pages, so non-developers can adapt the generated pages without having to edit the generator code
+- create a new custom [shortcode](https://gohugo.io/content-management/shortcodes/), to easily create links from inside the website to specific pages of the API reference
+- improve the navigation between the sections of the API reference
+- add the code of the generator to the Kubernetes GitHub repository containing the different reference generators
+
+All the discussions and work done can be found in website [pull request #23294](https://github.com/kubernetes/website/pull/23294).
+
+Adding the generator code to the Kubernetes project happened in [kubernetes-sigs/reference-docs#179](https://github.com/kubernetes-sigs/reference-docs/pull/179).
+
+Here are the features of the new API Reference to be included in the official documentation website:
+
+- the resources are categorized, in the categories Workloads, Services, Config & Storage, Authentication, Authorization, Policies, Extend, Cluster. This structure is configurable with a simple [`toc.yaml` file](https://github.com/kubernetes-sigs/reference-docs/blob/master/gen-resourcesdocs/config/v1.20/toc.yaml)
+- each page displays associated resources at the first level ; for example: Pod, PodSpec, PodStatus, PodList
+- most resource pages inline relevant definitions ; the exceptions are when those definitions are common to several resources, or are too complex to be displayed inline. With the old approach, you had to follow a hyperlink to read each extra detail.
+- some widely used definitions, such as `ObjectMeta`, are documented in a specific page
+- required fields are indicated, and placed first
+- fields of a resource can be categorized and ordered, with the help of a [`fields.yaml` file](https://github.com/kubernetes-sigs/reference-docs/blob/master/gen-resourcesdocs/config/v1.20/fields.yaml)
+- `map` fields are indicated. For example the `.spec.nodeSelector` for a `Pod` is `map[string]string`, instead of `object`, using the value of `x-kubernetes-list-type`
+- patch strategies are indicated
+- `apiVersion` and `kind` display the value, not the `string` type
+- At the top of a reference page, the page displays the Go import necessary to use these resources from a Go program.
+
+The work is currently on hold pending the 1.20 release. When the release finishes and the work is integrated, the API reference will be available at https://kubernetes.io/docs/reference/.
+
+### Future Work
+
+There are points to improve, particularly:
+
+- Some Kubernetes resources are deeply nested. Inlining the definition of these resources makes them difficult to understand.
+- The created `shortcode` uses the URL of the page to reference a Resource page. It would be easier for documentarians if they could reference a Resource by its group and name.
+
+## Appreciation
+
+I would like to thank my mentor [Zach Corleissen](https://github.com/zacharysarah) and the lead writers [Karen Bradshaw](https://github.com/kbhawkey), [Celeste Horgan](https://github.com/celestehorgan), [Tim Bannister](https://github.com/sftim) and [Qiming Teng](https://github.com/tengqm) who supervised me during all the season. They all have been very encouraging and gave me tons of great advice.
@@ -3,116 +3,84 @@ title: Adform Case Study
 linkTitle: Adform
 case_study_styles: true
 cid: caseStudies
-css: /css/style_case_studies.css
 logo: adform_featured_logo.png
 draft: false
 featured: true
 weight: 47
 quote: >
  Kubernetes enabled the self-healing and immutable infrastructure. We can do faster releases, so our developers are really happy. They can ship our features faster than before, and that makes our clients happier.
+
+new_case_study_styles: true
+heading_background: /images/case-studies/adform/banner1.jpg
+heading_title_logo: /images/adform_logo.png
+subheading: >
+  Improving Performance and Morale with Cloud Native
+case_study_details:
+  - Company: AdForm
+  - Location: Copenhagen, Denmark
+  - Industry: Adtech
 ---

-<div class="banner1 desktop" style="background-image: url('/images/case-studies/adform/banner1.jpg')">
-  <h1> CASE STUDY:<img src="/images/adform_logo.png" style="width:15%;margin-bottom:0%" class="header_logo"><br> <div class="subhead">Improving Performance and Morale with Cloud Native
+<h2>Challenge</h2>

-</div></h1>
+<p><a href="https://site.adform.com/">Adform's</a> mission is to provide a secure and transparent full stack of advertising technology to enable digital ads across devices. The company has a large infrastructure: <a href="https://www.openstack.org/">OpenStack</a>-based private clouds running on 1,100 physical servers in 7 data centers around the world, 3 of which were opened in the past year. With the company's growth, the infrastructure team felt that "our private cloud was not really flexible enough," says IT System Engineer Edgaras Apšega. "The biggest pain point is that our developers need to maintain their virtual machines, so rolling out technology and new software takes time. We were really struggling with our releases, and we didn't have self-healing infrastructure."</p>

-</div>
+<h2>Solution</h2>

-<div class="details">
-    Company &nbsp;<b>AdForm</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location &nbsp;<b>Copenhagen, Denmark</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Industry &nbsp;<b>Adtech</b>
-</div>
-
-<hr>
-<section class="section1">
-<div class="cols">
-  <div class="col1">
-    <h2>Challenge</h2>
-    <a href="https://site.adform.com/">Adform’s</a> mission is to provide a secure and transparent full stack of advertising technology to enable digital ads across devices. The company has a large infrastructure: <a href="https://www.openstack.org/">OpenStack</a>-based private clouds running on 1,100 physical servers in 7 data centers around the world, 3 of which were opened in the past year. With the company’s growth, the infrastructure team felt that "our private cloud was not really flexible enough," says IT System Engineer Edgaras Apšega. "The biggest pain point is that our developers need to maintain their virtual machines, so rolling out technology and new software takes time. We were really struggling with our releases, and we didn’t have self-healing infrastructure."
-
-
-<br>
-
- <h2>Solution</h2>
-    The team, which had already been using <a href="https://prometheus.io/">Prometheus</a> for monitoring, embraced <a href="https://kubernetes.io/">Kubernetes</a> and cloud native practices in 2017. "To start our Kubernetes journey, we had to adapt all our software, so we had to choose newer frameworks," says Apšega. "We also adopted the microservices way, so observability is much better because you can inspect the bug or the services separately."
-
-
-</div>
-
-<div class="col2">
+<p>The team, which had already been using <a href="https://prometheus.io/">Prometheus</a> for monitoring, embraced <a href="https://kubernetes.io/">Kubernetes</a> and cloud native practices in 2017. "To start our Kubernetes journey, we had to adapt all our software, so we had to choose newer frameworks," says Apšega. "We also adopted the microservices way, so observability is much better because you can inspect the bug or the services separately."</p>

 <h2>Impact</h2>
- "Kubernetes helps our business a lot because our features are coming to market faster," says Apšega. The release process went from several hours to several minutes. Autoscaling has been at least 6 times faster than the semi-manual VM bootstrapping and application deployment required before. The team estimates that the company has experienced cost savings of 4-5x due to less hardware and fewer man hours needed to set up the hardware and virtual machines, metrics, and logging. Utilization of the hardware resources has been reduced as well, with containers notching 2-3 times more efficiency over virtual machines. "The deployments are very easy because developers just push the code and it automatically appears on Kubernetes," says Apšega. Prometheus has also had a positive impact: "It provides high availability for metrics and alerting. We monitor everything starting from hardware to applications. Having all the metrics in <a href="https://grafana.com/">Grafana</a> dashboards provides great insight on your systems."

+<p>"Kubernetes helps our business a lot because our features are coming to market faster," says Apšega. The release process went from several hours to several minutes. Autoscaling has been at least 6 times faster than the semi-manual VM bootstrapping and application deployment required before. The team estimates that the company has experienced cost savings of 4-5x due to less hardware and fewer man hours needed to set up the hardware and virtual machines, metrics, and logging. Utilization of the hardware resources has been reduced as well, with containers notching 2-3 times more efficiency over virtual machines. "The deployments are very easy because developers just push the code and it automatically appears on Kubernetes," says Apšega. Prometheus has also had a positive impact: "It provides high availability for metrics and alerting. We monitor everything starting from hardware to applications. Having all the metrics in <a href="https://grafana.com/">Grafana</a> dashboards provides great insight on your systems."</p>

-</div>
+{{< case-studies/quote author="Edgaras Apšega, IT Systems Engineer, Adform" >}}
+"Kubernetes enabled the self-healing and immutable infrastructure. We can do faster releases, so our developers are really happy. They can ship our features faster than before, and that makes our clients happier."  
+{{< /case-studies/quote >}}

-</div>
-</section>
-<div class="banner2">
-  <div class="banner2text">
-"Kubernetes enabled the self-healing and immutable infrastructure. We can do faster releases, so our developers are really happy. They can ship our features faster than before, and that makes our clients happier."<span style="font-size:14px;letter-spacing:0.12em;padding-top:20px;text-transform:uppercase;line-height:14px"><br><br>— Edgaras Apšega, IT Systems Engineer, Adform</span>
+{{< case-studies/lead >}}
+Adform made <a href="https://www.wsj.com/articles/fake-ad-operation-used-to-steal-from-publishers-is-uncovered-1511290981">headlines</a> last year when it detected the HyphBot ad fraud network that was costing some businesses hundreds of thousands of dollars a day.
+{{< /case-studies/lead >}}

-  </div>
-</div>
+<p>With its mission to provide a secure and transparent full stack of advertising technology to enable an open internet, Adform published a <a href="https://site.adform.com/media/85132/hyphbot_whitepaper_.pdf">white paper</a> revealing what it did—and others could too—to limit customers' exposure to the scam.</p>

+<p>In that same spirit, Adform is sharing its cloud native journey. "When you see that everyone shares their best practices, it inspires you to contribute back to the project," says IT Systems Engineer Edgaras Apšega.</p>

-<section class="section2">
-<div class="fullcol">
-<h2>Adform made <a href="https://www.wsj.com/articles/fake-ad-operation-used-to-steal-from-publishers-is-uncovered-1511290981">headlines</a> last year when it detected the HyphBot ad fraud network that was costing some businesses hundreds of thousands of dollars a day.</h2>With its mission to provide a secure and transparent full stack of advertising technology to enable an open internet, Adform published a <a href="https://site.adform.com/media/85132/hyphbot_whitepaper_.pdf">white paper</a> revealing what it did—and others could too—to limit customers’ exposure to the scam. <br><br>
-In that same spirit, Adform is sharing its cloud native journey. "When you see that everyone shares their best practices, it inspires you to contribute back to the project," says IT Systems Engineer Edgaras Apšega.<br><br>
-The company has a large infrastructure: <a href="https://www.openstack.org/">OpenStack</a>-based private clouds running on 1,100 physical servers in their own seven data centers around the world, three of which were opened in the past year. With the company’s growth, the infrastructure team felt that "our private cloud was not really flexible enough," says Apšega. "The biggest pain point is that our developers need to maintain their virtual machines, so rolling out technology and new software really takes time. We were really struggling with our releases, and we didn’t have self-healing infrastructure."
+<p>The company has a large infrastructure: <a href="https://www.openstack.org/">OpenStack</a>-based private clouds running on 1,100 physical servers in their own seven data centers around the world, three of which were opened in the past year. With the company's growth, the infrastructure team felt that "our private cloud was not really flexible enough," says Apšega. "The biggest pain point is that our developers need to maintain their virtual machines, so rolling out technology and new software really takes time. We were really struggling with our releases, and we didn't have self-healing infrastructure."</p>

+{{< case-studies/quote
+  image="/images/case-studies/adform/banner3.jpg"
+  author="Edgaras Apšega, IT Systems Engineer, Adform"
+>}}
+"The fact that Cloud Native Computing Foundation incubated Kubernetes was a really big point for us because it was vendor neutral. And we can see that a community really gathers around it. Everyone shares their experiences, their knowledge, and the fact that it's open source, you can contribute."
+{{< /case-studies/quote >}}

-</div>
-</section>
-<div class="banner3" style="background-image: url('/images/case-studies/adform/banner3.jpg')">
-  <div class="banner3text">
-    "The fact that Cloud Native Computing Foundation incubated Kubernetes was a really big point for us because it was vendor neutral. And we can see that a community really gathers around it. Everyone shares their experiences, their knowledge, and the fact that it’s open source, you can contribute."<span style="font-size:14px;letter-spacing:0.12em;padding-top:20px;text-transform:uppercase;line-height:14px"><br><br>— Edgaras Apšega, IT Systems Engineer, Adform</span>
-  </div>
-</div>
-<section class="section3">
-<div class="fullcol">
+<p>The team, which had already been using Prometheus for monitoring, embraced Kubernetes, microservices, and cloud native practices. "The fact that Cloud Native Computing Foundation incubated Kubernetes was a really big point for us because it was vendor neutral," says Apšega. "And we can see that a community really gathers around it."</p>

-The team, which had already been using Prometheus for monitoring, embraced Kubernetes, microservices, and cloud native practices. "The fact that Cloud Native Computing Foundation incubated Kubernetes was a really big point for us because it was vendor neutral," says Apšega. "And we can see that a community really gathers around it."<br><br>
-A proof of concept project was started, with a Kubernetes cluster running on bare metal in the data center. When developers saw how quickly containers could be spun up compared to the virtual machine process, "they wanted to ship their containers in production right away, and we were still doing proof of concept," says IT Systems Engineer Andrius Cibulskis.
-Of course, a lot of work still had to be done. "First of all, we had to learn Kubernetes, see all of the moving parts, how they glue together," says Apšega. "Second of all, the whole CI/CD part had to be redone, and our DevOps team had to invest more man hours to implement it. And third is that developers had to rewrite the code, and they’re still doing it."
-<br><br>
-The first production cluster was launched in the spring of 2018, and is now up to 20 physical machines dedicated for pods throughout three data centers, with plans for separate clusters in the other four data centers. The user-facing Adform application platform, data distribution platform, and back ends are now all running on Kubernetes. "Many APIs for critical applications are being developed for Kubernetes," says Apšega. "Teams are rewriting their applications to .NET core, because it supports containers, and preparing to move to Kubernetes. And new applications, by default, go in containers."
+<p>A proof of concept project was started, with a Kubernetes cluster running on bare metal in the data center. When developers saw how quickly containers could be spun up compared to the virtual machine process, "they wanted to ship their containers in production right away, and we were still doing proof of concept," says IT Systems Engineer Andrius Cibulskis.</p>

+<p>Of course, a lot of work still had to be done. "First of all, we had to learn Kubernetes, see all of the moving parts, how they glue together," says Apšega. "Second of all, the whole CI/CD part had to be redone, and our DevOps team had to invest more man hours to implement it. And third is that developers had to rewrite the code, and they're still doing it."</p>

-</div>
-</section>
-<div class="banner4" style="background-image: url('/images/case-studies/adform/banner4.jpg')">
-  <div class="banner4text">
-"Releases are really nice for them, because they just push their code to Git and that’s it. They don’t have to worry about their virtual machines anymore." <span style="font-size:14px;letter-spacing:0.12em;padding-top:20px;text-transform:uppercase;line-height:14px"><br><br>— Andrius Cibulskis, IT Systems Engineer, Adform</span>
-  </div>
-</div>
+<p>The first production cluster was launched in the spring of 2018, and is now up to 20 physical machines dedicated for pods throughout three data centers, with plans for separate clusters in the other four data centers. The user-facing Adform application platform, data distribution platform, and back ends are now all running on Kubernetes. "Many APIs for critical applications are being developed for Kubernetes," says Apšega. "Teams are rewriting their applications to .NET core, because it supports containers, and preparing to move to Kubernetes. And new applications, by default, go in containers."</p>

-<section class="section5">
-<div class="fullcol">
-This big push has been driven by the real impact that these new practices have had. "Kubernetes helps our business a lot because our features are coming to market faster," says Apšega. "The deployments are very easy because developers just push the code and it automatically appears on Kubernetes." The release process went from several hours to several minutes. Autoscaling is at least six times faster than the semi-manual VM bootstrapping and application deployment required before. <br><br>
-The team estimates that the company has experienced cost savings of 4-5x due to less hardware and fewer man hours needed to set up the hardware and virtual machines, metrics, and logging. Utilization of the hardware resources has been reduced as well, with containers notching two to three times more efficiency over virtual machines. <br><br>
-Prometheus has also had a positive impact: "It provides high availability for metrics and alerting," says Apšega. "We monitor everything starting from hardware to applications. Having all the metrics in Grafana dashboards provides great insight on our systems."
+{{< case-studies/quote
+  image="/images/case-studies/adform/banner4.jpg"
+  author="Andrius Cibulskis, IT Systems Engineer, Adform"
+>}}
+"Releases are really nice for them, because they just push their code to Git and that's it. They don't have to worry about their virtual machines anymore."  
+{{< /case-studies/quote >}}

+<p>This big push has been driven by the real impact that these new practices have had. "Kubernetes helps our business a lot because our features are coming to market faster," says Apšega. "The deployments are very easy because developers just push the code and it automatically appears on Kubernetes." The release process went from several hours to several minutes. Autoscaling is at least six times faster than the semi-manual VM bootstrapping and application deployment required before.</p>

+<p>The team estimates that the company has experienced cost savings of 4-5x due to less hardware and fewer man hours needed to set up the hardware and virtual machines, metrics, and logging. Utilization of the hardware resources has been reduced as well, with containers notching two to three times more efficiency over virtual machines.</p>

-</div>
+<p>Prometheus has also had a positive impact: "It provides high availability for metrics and alerting," says Apšega. "We monitor everything starting from hardware to applications. Having all the metrics in Grafana dashboards provides great insight on our systems."</p>

-<div class="banner5">
-  <div class="banner5text">
- "I think that our company just started our cloud native journey. It seems like a huge road ahead, but we’re really happy that we joined it." <span style="font-size:14px;letter-spacing:0.12em;padding-top:20px;text-transform:uppercase;line-height:14px"><br><br>— Edgaras Apšega, IT Systems Engineer, Adform</span>
-  </div>
-</div>
+{{< case-studies/quote author="Edgaras Apšega, IT Systems Engineer, Adform" >}}
+"I think that our company just started our cloud native journey. It seems like a huge road ahead, but we're really happy that we joined it."  
+{{< /case-studies/quote >}}

-<div class="fullcol">
-All of these benefits have trickled down to individual team members, whose working lives have been changed for the better. "They used to have to get up at night to re-start some services, and now Kubernetes handles all of that," says Apšega. Adds Cibulskis: "Releases are really nice for them, because they just push their code to Git and that’s it. They don’t have to worry about their virtual machines anymore." Even the security teams have been impacted. "Security teams are always not happy," says Apšega, "and now they’re happy because they can easily inspect the containers."
-The company plans to remain in the data centers for now, "mostly because we want to keep all the data, to not share it in any way," says Cibulskis, "and it’s cheaper at our scale." But, Apšega says, the possibility of using a hybrid cloud for computing is intriguing: "One of the projects we’re interested in is the <a href="https://github.com/virtual-kubelet/virtual-kubelet">Virtual Kubelet</a> that lets you spin up the working nodes on different clouds to do some computing."
-<br><br>
-Apšega, Cibulskis and their colleagues are keeping tabs on how the cloud native ecosystem develops, and are excited to contribute where they can. "I think that our company just started our cloud native journey," says Apšega. "It seems like a huge road ahead, but we’re really happy that we joined it."
+<p>All of these benefits have trickled down to individual team members, whose working lives have been changed for the better. "They used to have to get up at night to re-start some services, and now Kubernetes handles all of that," says Apšega. Adds Cibulskis: "Releases are really nice for them, because they just push their code to Git and that's it. They don't have to worry about their virtual machines anymore." Even the security teams have been impacted. "Security teams are always not happy," says Apšega, "and now they're happy because they can easily inspect the containers."</p>

+<p>The company plans to remain in the data centers for now, "mostly because we want to keep all the data, to not share it in any way," says Cibulskis, "and it's cheaper at our scale." But, Apšega says, the possibility of using a hybrid cloud for computing is intriguing: "One of the projects we're interested in is the <a href="https://github.com/virtual-kubelet/virtual-kubelet">Virtual Kubelet</a> that lets you spin up the working nodes on different clouds to do some computing."</p>

-
-</div>
-
-</section>
+<p>Apšega, Cibulskis and their colleagues are keeping tabs on how the cloud native ecosystem develops, and are excited to contribute where they can. "I think that our company just started our cloud native journey," says Apšega. "It seems like a huge road ahead, but we're really happy that we joined it."</p>
@@ -3,106 +3,76 @@ title: adidas Case Study
 linkTitle: adidas
 case_study_styles: true
 cid: caseStudies
-css: /css/case-studies-gradient.css
 featured: false
+
+new_case_study_styles: true
+heading_background: /images/case-studies/adidas/banner1.png
+heading_title_text: adidas
+use_gradient_overlay: true
+subheading: >
+  Staying True to Its Culture, adidas Got 40% of Its Most Impactful Systems Running on Kubernetes in a Year
+case_study_details:
+  - Company: adidas
+  - Location: Herzogenaurach, Germany
+  - Industry: Fashion
 ---
-

+<h2>Challenge</h2>
+
+<p>In recent years, the adidas team was happy with its software choices from a technology perspective—but accessing all of the tools was a problem. For instance, "just to get a developer VM, you had to send a request form, give the purpose, give the title of the project, who's responsible, give the internal cost center a call so that they can do recharges," says Daniel Eichten, Senior Director of Platform Engineering. "The best case is you got your machine in half an hour. Worst case is half a week or sometimes even a week."</p>
+
+<h2>Solution</h2>
+
+<p>To improve the process, "we started from the developer point of view," and looked for ways to shorten the time it took to get a project up and running and into the adidas infrastructure, says Senior Director of Platform Engineering Fernando Cornago. They found the solution with containerization, agile development, continuous delivery, and a cloud native platform that includes Kubernetes and Prometheus.</p>

-<div class="article image overlay" style="background-image: url('/images/case-studies/adidas/banner1.png')">
-  <h1> CASE STUDY: adidas</h1>
-  <div class="subhead">Staying True to Its Culture, adidas Got 40% of Its Most Impactful Systems Running on Kubernetes in a Year</div>
-</div>
-
-
-<div class="details">
-    Company &nbsp;<b>adidas</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location &nbsp;<b>Herzogenaurach, Germany</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Industry &nbsp;<b>Fashion</b>
-</div>
-
-<hr>
-<section class="section1">
-<div class="colsgradient">
-  <div class="col1">
-    <h2>Challenge</h2>
-    In recent years, the adidas team was happy with its software choices from a technology perspective—but accessing all of the tools was a problem. For instance, “just to get a developer VM, you had to send a request form, give the purpose, give the title of the project, who’s responsible, give the internal cost center a call so that they can do recharges,” says Daniel Eichten, Senior Director of Platform Engineering. “The best case is you got your machine in half an hour. Worst case is half a week or sometimes even a week.”
-    <h2>Solution</h2>
-    To improve the process, “we started from the developer point of view,” and looked for ways to shorten the time it took to get a project up and running and into the adidas infrastructure, says Senior Director of Platform Engineering Fernando Cornago. They found the solution with containerization, agile development, continuous delivery, and a cloud native platform that includes Kubernetes and Prometheus.
-
 <h2>Impact</h2>
-    Just six months after the project began, 100% of the adidas e-commerce site was running on Kubernetes. Load time for the e-commerce site was reduced by half. Releases went from every 4-6 weeks to 3-4 times a day. With 4,000 pods, 200 nodes, and 80,000 builds per month, adidas is now running 40% of its most critical, impactful systems on its cloud native platform.
-</div>
-</div>
-</section>
-<div class="article image overlay" style="background-image: url('/images/case-studies/adidas/banner2.png');">
-  <div class="quotetext">
-    "For me, Kubernetes is a platform made by engineers for engineers. It’s relieving the development team from tasks that they don’t want to do, but at the same time giving the visibility of what is behind the curtain, so they can also control it."
-<p><div class="quoteauthortext">- FERNANDO CORNAGO, SENIOR DIRECTOR OF PLATFORM ENGINEERING AT ADIDAS</div></p>
-  </div>
-</div>
-
-
-<section class="section2">
-<div class="fullcol">
-  <h4>In recent years, the adidas team was happy with its software choices from a technology perspective—but accessing all of the tools was a problem.</h4>
-  <p>For engineers at adidas, says Daniel Eichten, Senior Director of Platform Engineering, "it felt like being an artist with your hands tied behind your back, and you’re supposed to paint something."</p>
-<p>
-  For instance, "just to get a developer VM, you had to send a request form, give the purpose, give the title of the project, who’s responsible, give the internal cost center a call so that they can do recharges," says Eichten. "Eventually, after a ton of approvals, then the provisioning of the machine happened within minutes, and then the best case is you got your machine in half an hour. Worst case is half a week or sometimes even a week."
-  </p>
-  <p>
-    To improve the process, "we started from the developer point of view," and looked for ways to shorten the time it took to get a project up and running and into the adidas infrastructure, says Senior Director of Platform Engineering Fernando Cornago.</p>
-</div>
-</section>
-
-<div class="greybanner">
-  <div class="greyquotetext">
-     "I call our cloud native platform the field of dreams. We built it, and we never anticipated that people would come and just love it."<p><div class="quoteauthortext">- DANIEL EICHTEN, SENIOR DIRECTOR OF PLATFORM ENGINEERING AT ADIDAS</div></p>
-  </div>
-</div>
-
-<section class="section3">
-<div class="fullcol">
-  <p>
-  "We were engineers before," adds Eichten. "We know what a typical engineer needs, is craving for, what he or she doesn’t want to take care of. For us it was pretty clear. We filled the gaps that no one wants to take care of, and we make the stuff that is usually painful as painless as possible." The goals: to improve speed, operability, and observability.
-  </p>
-<p>
-  Cornago and Eichten found the solution with containerization, agile development, continuous delivery, and a cloud native platform that includes Kubernetes and Prometheus. "Choosing Kubernetes was pretty clear," says Eichten. "Day zero, deciding, easy. Day one, installing, configuring, easy. Day two, keeping it up and running even with small workloads, if something goes wrong, you don’t know how these things work in detail, you’re lost. For day two problems, we needed a partner who’s helping us."
-</p>
-<p>
-  In early 2017, adidas chose Giant Swarm to consult, install, configure, and run all of its Kubernetes clusters in AWS and on premise. "There is no competitive edge over our competitors like Puma or Nike in running and operating a Kubernetes cluster," says Eichten. "Our competitive edge is that we teach our internal engineers how to build cool e-comm stores that are fast, that are resilient, that are running perfectly."
-  </p>
-</div>
-</section>
-
-
-  <div class="article image overlay" style="background-image: url('/images/case-studies/adidas/banner3.png');">
-  <div class="quotetext">
-  “There is no competitive edge over our competitors like Puma or Nike in running and operating a Kubernetes cluster. Our competitive edge is that we teach our internal engineers how to build cool e-comm stores that are fast, that are resilient, that are running perfectly.” <p><div class="quoteauthortext">- DANIEL EICHTEN, SENIOR DIRECTOR OF PLATFORM ENGINEERING AT ADIDAS</div></p>
-  </div>
-</div>
-
-<section class="section5">
-<div class="fullcol">
-  <p>
-  Adds Cornago: “For me, our Kubernetes platform is made by engineers for engineers. It’s relieving the development team from tasks that they don’t want to do, but at the same time giving the visibility of what is behind the curtain, so they can also control it.”
-</p>
-<p>
-  Case in point: For Cyber Week, the team has to create a lot of custom metrics. In November 2017, “because we used the same Prometheus that we use for monitoring the cluster, we really filled the Prometheus database, and we were not able to reduce the retention period [enough],” says Cornago. So during the freeze period before the peak shopping week, five engineers from the platform team worked with five engineers from the e-comm team to figure out a federated solution that was implemented in two days.
-</p>
-<p>
-  In addition to being ready for Cyber Week—100% of the adidas e-commerce site was running on Kubernetes then, just six months after the project began—the cloud native stack has had other impressive results. Load time for the e-commerce site was reduced by half. Releases went from every 4-6 weeks to 3-4 times a day. With 4,000 pods, 200 nodes, and 80,000 builds per month, adidas is now running 40% of its most critical, impactful systems on its cloud native platform.
-  </p>
-<p>
-  And adoption has spread quickly among adidas’s 300-strong engineering corps. “I call our cloud native platform the field of dreams,” says Eichten. “We built it, and we never anticipated that people would come and just love it.”
-</p>
-<p>
-  For one thing, “everybody who can touch a line of code” has spent one full week onboarding and learning the platform with members of the 35-person platform engineering team, says Cornago. “We try to spend 50% of our time sitting with the teams, because this is the only way to understand how our platform is being used. And this is how the teams will feel safe that there is someone on the other side of the wall, also feeling the pain.”
-  </p>
-<p>
-  Additionally, Cornago and Eichten took advantage of the fact that as a fashion athletic wear brand, adidas has sports and competition in its DNA. “Top-down mandates don’t work at adidas, but gamification works,” says Cornago. “So this year we had a DevOps Cup competition. Every team created new technical capabilities and had a hypothesis of how this affected business value. We announced the winner at a big internal tech summit with more than 600 people. It’s been really, really useful for the teams.”
-</p>
-<p>
-  So if they had any advice for other companies looking to start a cloud native journey, it would be this: “There is no one-size-fits-all for all companies,” says Cornago. “Apply your company’s culture to everything that you do.”
-  </p>
-</div>
-</section>
-</body>
+
+<p>Just six months after the project began, 100% of the adidas e-commerce site was running on Kubernetes. Load time for the e-commerce site was reduced by half. Releases went from every 4-6 weeks to 3-4 times a day. With 4,000 pods, 200 nodes, and 80,000 builds per month, adidas is now running 40% of its most critical, impactful systems on its cloud native platform.</p>
+
+{{< case-studies/quote 
+  image="/images/case-studies/adidas/banner2.png"
+  author="FERNANDO CORNAGO, SENIOR DIRECTOR OF PLATFORM ENGINEERING AT ADIDAS"
+>}}
+"For me, Kubernetes is a platform made by engineers for engineers. It's relieving the development team from tasks that they don't want to do, but at the same time giving the visibility of what is behind the curtain, so they can also control it."
+{{< /case-studies/quote >}}
+
+{{< case-studies/lead >}}
+In recent years, the adidas team was happy with its software choices from a technology perspective—but accessing all of the tools was a problem.
+{{< /case-studies/lead >}}
+
+<p>For engineers at adidas, says Daniel Eichten, Senior Director of Platform Engineering, "it felt like being an artist with your hands tied behind your back, and you're supposed to paint something."</p>
+
+<p>For instance, "just to get a developer VM, you had to send a request form, give the purpose, give the title of the project, who's responsible, give the internal cost center a call so that they can do recharges," says Eichten. "Eventually, after a ton of approvals, then the provisioning of the machine happened within minutes, and then the best case is you got your machine in half an hour. Worst case is half a week or sometimes even a week."</p>
+
+<p>To improve the process, "we started from the developer point of view," and looked for ways to shorten the time it took to get a project up and running and into the adidas infrastructure, says Senior Director of Platform Engineering Fernando Cornago.</p>
+
+{{< case-studies/quote author="DANIEL EICHTEN, SENIOR DIRECTOR OF PLATFORM ENGINEERING AT ADIDAS" >}}
+"I call our cloud native platform the field of dreams. We built it, and we never anticipated that people would come and just love it."
+{{< /case-studies/quote >}}
+
+<p>"We were engineers before," adds Eichten. "We know what a typical engineer needs, is craving for, what he or she doesn't want to take care of. For us it was pretty clear. We filled the gaps that no one wants to take care of, and we make the stuff that is usually painful as painless as possible." The goals: to improve speed, operability, and observability.</p>
+
+<p>Cornago and Eichten found the solution with containerization, agile development, continuous delivery, and a cloud native platform that includes Kubernetes and Prometheus. "Choosing Kubernetes was pretty clear," says Eichten. "Day zero, deciding, easy. Day one, installing, configuring, easy. Day two, keeping it up and running even with small workloads, if something goes wrong, you don't know how these things work in detail, you're lost. For day two problems, we needed a partner who's helping us."</p>
+
+<p>In early 2017, adidas chose Giant Swarm to consult, install, configure, and run all of its Kubernetes clusters in AWS and on premise. "There is no competitive edge over our competitors like Puma or Nike in running and operating a Kubernetes cluster," says Eichten. "Our competitive edge is that we teach our internal engineers how to build cool e-comm stores that are fast, that are resilient, that are running perfectly."</p>
+
+{{< case-studies/quote 
+  image="/images/case-studies/adidas/banner3.png"
+  author="DANIEL EICHTEN, SENIOR DIRECTOR OF PLATFORM ENGINEERING AT ADIDAS"
+>}}
+"There is no competitive edge over our competitors like Puma or Nike in running and operating a Kubernetes cluster. Our competitive edge is that we teach our internal engineers how to build cool e-comm stores that are fast, that are resilient, that are running perfectly."
+{{< /case-studies/quote >}}
+
+<p>Adds Cornago: "For me, our Kubernetes platform is made by engineers for engineers. It's relieving the development team from tasks that they don't want to do, but at the same time giving the visibility of what is behind the curtain, so they can also control it."</p>
+
+<p>Case in point: For Cyber Week, the team has to create a lot of custom metrics. In November 2017, "because we used the same Prometheus that we use for monitoring the cluster, we really filled the Prometheus database, and we were not able to reduce the retention period [enough]," says Cornago. So during the freeze period before the peak shopping week, five engineers from the platform team worked with five engineers from the e-comm team to figure out a federated solution that was implemented in two days.</p>
+
+<p>In addition to being ready for Cyber Week—100% of the adidas e-commerce site was running on Kubernetes then, just six months after the project began—the cloud native stack has had other impressive results. Load time for the e-commerce site was reduced by half. Releases went from every 4-6 weeks to 3-4 times a day. With 4,000 pods, 200 nodes, and 80,000 builds per month, adidas is now running 40% of its most critical, impactful systems on its cloud native platform.</p>
+
+<p>And adoption has spread quickly among adidas's 300-strong engineering corps. "I call our cloud native platform the field of dreams," says Eichten. "We built it, and we never anticipated that people would come and just love it."</p>
+
+<p>For one thing, "everybody who can touch a line of code" has spent one full week onboarding and learning the platform with members of the 35-person platform engineering team, says Cornago. "We try to spend 50% of our time sitting with the teams, because this is the only way to understand how our platform is being used. And this is how the teams will feel safe that there is someone on the other side of the wall, also feeling the pain."</p>
+
+<p>Additionally, Cornago and Eichten took advantage of the fact that as a fashion athletic wear brand, adidas has sports and competition in its DNA. "Top-down mandates don't work at adidas, but gamification works," says Cornago. "So this year we had a DevOps Cup competition. Every team created new technical capabilities and had a hypothesis of how this affected business value. We announced the winner at a big internal tech summit with more than 600 people. It's been really, really useful for the teams."</p>
+
+<p>So if they had any advice for other companies looking to start a cloud native journey, it would be this: "There is no one-size-fits-all for all companies," says Cornago. "Apply your company's culture to everything that you do."</p>
@@ -1,105 +1,84 @@
 ---
 title: Amadeus Case Study
-
 case_study_styles: true
 cid: caseStudies
-css: /css/style_amadeus.css
+
+new_case_study_styles: true
+heading_background: /images/case-studies/amadeus/banner1.jpg
+heading_title_logo: /images/amadeus_logo.png
+subheading: >
+  Another Technical Evolution for a 30-Year-Old Company
+case_study_details:
+  - Company: Amadeus IT Group
+  - Location: Madrid, Spain
+  - Industry: Travel Technology
 ---

-<div class="banner1">
-  <h1> CASE STUDY:<img src="/images/amadeus_logo.png" class="header_logo"><br> <div class="subhead">Another Technical Evolution for a 30-Year-Old Company
-</div></h1>
-</div>
-<div class="details">
-    Company &nbsp;<b>Amadeus IT Group</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location &nbsp;<b>Madrid, Spain</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Industry &nbsp;<b>Travel Technology</b>
-</div>
-<hr>
-<section class="section1">
-<div class="cols">
-  <div class="col1">
-    <h2>Challenge</h2>
-      In the past few years, Amadeus, which provides IT solutions to the travel industry around the world, found itself in need of a new platform for the 5,000 services supported by its service-oriented architecture. The 30-year-old company operates its own data center in Germany, and there were growing demands internally and externally for solutions that needed to be geographically dispersed. And more generally, "we had objectives of being even more highly available," says Eric Mountain, Senior Expert, Distributed Systems at Amadeus. Among the company’s goals: to increase automation in managing its infrastructure, optimize the distribution of workloads, use data center resources more efficiently, and adopt new technologies more easily.
-      </div>
-<div class="col2">
-  <h2>Solution</h2>
-      Mountain has been overseeing the company’s migration to <a href="http://kubernetes.io/">Kubernetes</a>, using <a href="https://www.openshift.org/">OpenShift</a> Container Platform, <a href="https://www.redhat.com/en">Red Hat</a>’s enterprise container platform.
-<br><br>
+<h2>Challenge</h2>
+
+<p>In the past few years, Amadeus, which provides IT solutions to the travel industry around the world, found itself in need of a new platform for the 5,000 services supported by its service-oriented architecture. The 30-year-old company operates its own data center in Germany, and there were growing demands internally and externally for solutions that needed to be geographically dispersed. And more generally, "we had objectives of being even more highly available," says Eric Mountain, Senior Expert, Distributed Systems at Amadeus. Among the company's goals: to increase automation in managing its infrastructure, optimize the distribution of workloads, use data center resources more efficiently, and adopt new technologies more easily.</p>
+
+<h2>Solution</h2>
+
+<p>Mountain has been overseeing the company's migration to <a href="http://kubernetes.io/">Kubernetes</a>, using <a href="https://www.openshift.org/">OpenShift</a> Container Platform, <a href="https://www.redhat.com/en">Red Hat</a>'s enterprise container platform.</p>
+
 <h2>Impact</h2>
-      One of the first projects the team deployed in Kubernetes was the Amadeus Airline Cloud Availability solution, which helps manage ever-increasing flight-search volume. "It’s now handling in production several thousand transactions per second, and it’s deployed in multiple data centers throughout the world," says Mountain. "It’s not a migration of an existing workload; it’s a whole new workload that we couldn’t have done otherwise. [This platform] gives us access to market opportunities that we didn’t have before."
-</div>
-</div>
-</section>

-<div class="banner2">
-  <div class="banner2text">
-    "We want multi-data center capabilities, and we want them for our mainstream system as well. We didn’t think that we could achieve them with our existing system. We need new automation, things that Kubernetes and OpenShift bring."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Eric Mountain, Senior Expert, Distributed Systems at Amadeus IT Group</span>
-  </div>
-</div>
-<section class="section2">
+<p>One of the first projects the team deployed in Kubernetes was the Amadeus Airline Cloud Availability solution, which helps manage ever-increasing flight-search volume. "It's now handling in production several thousand transactions per second, and it's deployed in multiple data centers throughout the world," says Mountain. "It's not a migration of an existing workload; it's a whole new workload that we couldn't have done otherwise. [This platform] gives us access to market opportunities that we didn't have before."</p>

-<div class="fullcol">
-    <h2>In his two decades at Amadeus, Eric Mountain has been the migrations guy. </h2>
-    Back in the day, he worked on the company’s move from Unix to Linux, and now he’s overseeing the journey to cloud native. "Technology just keeps changing, and we embrace it," he says. "We are celebrating our 30 years this year, and we continue evolving and innovating to stay cost-efficient and enhance everyone’s travel experience, without interrupting workflows for the customers who depend on our technology."<br><br>
-    That was the challenge that Amadeus—which provides IT solutions to the travel industry around the world, from flight searches to hotel bookings to customer feedback—faced in 2014. The technology team realized it was in need of a new platform for the 5,000 services supported by its service-oriented architecture.<br><br>
-    The tipping point occurred when they began receiving many requests, internally and externally, for solutions that needed to be geographically outside the company’s main data center in Germany. "Some requests were for running our applications on customer premises," Mountain says. "There were also new services we were looking to offer that required response time to the order of a few hundred milliseconds, which we couldn’t achieve with transatlantic traffic. Or at least, not without eating into a considerable portion of the time available to our applications for them to process individual queries."<br><br>
-    More generally, the company was interested in leveling up on high availability, increasing automation in managing infrastructure, optimizing the distribution of workloads and using data center resources more efficiently. "We have thousands and thousands of servers," says Mountain. "These servers are assigned roles, so even if the setup is highly automated, the machine still has a given role. It’s wasteful on many levels. For instance, an application doesn’t necessarily use the machine very optimally. Virtualization can help a bit, but it’s not a silver bullet. If that machine breaks, you still want to repair it because it has that role and you can’t simply say, ‘Well, I’ll bring in another machine and give it that role.’ It’s not fast. It’s not efficient. So we wanted the next level of automation."<br><br>
-</div>
-</section>
+{{< case-studies/quote author="Eric Mountain, Senior Expert, Distributed Systems at Amadeus IT Group" >}}
+"We want multi-data center capabilities, and we want them for our mainstream system as well. We didn't think that we could achieve them with our existing system. We need new automation, things that Kubernetes and OpenShift bring."
+{{< /case-studies/quote >}}

-<div class="banner3">
-  <div class="banner3text">
-    "We hope that if we build on what others have built, what we do might actually be upstream-able. As Kubernetes and OpenShift progress, we see that we are indeed able to remove some of the additional layers we implemented to compensate for gaps we perceived earlier."
-  </div>
-</div>
+{{< case-studies/lead >}}
+In his two decades at Amadeus, Eric Mountain has been the migrations guy.
+{{< /case-studies/lead >}}

-<section class="section3">
-<div class="fullcol">
-  While mainly a C++ and Java shop, Amadeus also wanted to be able to adopt new technologies more easily. Some of its developers had started using languages like <a href="https://www.python.org/">Python</a> and databases like <a href="https://www.couchbase.com/">Couchbase</a>, but Mountain wanted still more options, he says, "in order to better adapt our technical solutions to the products we offer, and open up entirely new possibilities to our developers." Working with recent technologies and cool new things would also make it easier to attract new talent.
-  <br><br>
-  All of those needs led Mountain and his team on a search for a new platform. "We did a set of studies and proofs of concept over a fairly short period, and we considered many technologies," he says. "In the end, we were left with three choices: build everything on premise, build on top of <a href="http://kubernetes.io/">Kubernetes</a> whatever happens to be missing from our point of view, or go with <a href="https://www.openshift.com/">OpenShift</a> and build whatever remains there."
-<br><br>
-  The team decided against building everything themselves—though they’d done that sort of thing in the past—because "people were already inventing things that looked good," says Mountain.
-<br><br>
-  Ultimately, they went with OpenShift Container Platform, <a href="https://www.redhat.com/en">Red Hat</a>’s Kubernetes-based enterprise offering, instead of building on top of Kubernetes because "there was a lot of synergy between what we wanted and the way Red Hat was anticipating going with OpenShift," says Mountain. "They were clearly developing Kubernetes, and developing certain things ahead of time in OpenShift, which were important to us, such as more security."
-<br><br>
-  The hope was that those particular features would eventually be built into Kubernetes, and, in the case of security, Mountain feels that has happened. "We realize that there’s always a certain amount of automation that we will probably have to develop ourselves to compensate for certain gaps," says Mountain. "The less we do that, the better for us. We hope that if we build on what others have built, what we do might actually be upstream-able. As Kubernetes and OpenShift progress, we see that we are indeed able to remove some of the additional layers we implemented to compensate for gaps we perceived earlier."
-</div>
-</section>
+<p>Back in the day, he worked on the company's move from Unix to Linux, and now he's overseeing the journey to cloud native. "Technology just keeps changing, and we embrace it," he says. "We are celebrating our 30 years this year, and we continue evolving and innovating to stay cost-efficient and enhance everyone's travel experience, without interrupting workflows for the customers who depend on our technology."</p>

-<div class="banner4">
-  <div class="banner4text">
-    "It’s not a migration of an existing workload; it’s a whole new workload that we couldn’t have done otherwise. [This platform] gives us access to market opportunities that we didn’t have before."
-  </div>
-</div>
+<p>That was the challenge that Amadeus—which provides IT solutions to the travel industry around the world, from flight searches to hotel bookings to customer feedback—faced in 2014. The technology team realized it was in need of a new platform for the 5,000 services supported by its service-oriented architecture.</p>

-<section class="section4">
-<div class="fullcol">
-  The first project the team tackled was one that they knew had to run outside the data center in Germany. Because of the project’s needs, "We couldn’t rely only on the built-in Kubernetes service discovery; we had to layer on top of that an extra service discovery level that allows us to load balance at the operation level within our system," says Mountain. They also built a stream dedicated to monitoring, which at the time wasn’t offered in the Kubernetes or OpenShift ecosystem. Now that <a href="https://www.prometheus.io/">Prometheus</a> and other products are available, Mountain says the company will likely re-evaluate their monitoring system: "We obviously always like to leverage what Kubernetes and OpenShift can offer."
-<br><br>
-  The second project ended up going into production first: the Amadeus Airline Cloud Availability solution, which helps manage ever-increasing flight-search volume and was deployed in public cloud. Launched in early 2016, it is "now handling in production several thousand transactions per second, and it’s deployed in multiple data centers throughout the world," says Mountain. "It’s not a migration of an existing workload; it’s a whole new workload that we couldn’t have done otherwise. [This platform] gives us access to market opportunities that we didn’t have before."
-<br><br>
-  Having been through this kind of technical evolution more than once, Mountain has advice on how to handle the cultural changes. "That’s one aspect that we can tackle progressively," he says. "We have to go on supplying our customers with new features on our pre-existing products, and we have to keep existing products working. So we can’t simply do absolutely everything from one day to the next. And we mustn’t sell it that way."
-<br><br>
-  The first order of business, then, is to pick one or two applications to demonstrate that the technology works. Rather than choosing a high-impact, high-risk project, Mountain’s team selected a smaller application that was representative of all the company’s other applications in its complexity: "We just made sure we picked something that’s complex enough, and we showed that it can be done."
-</div>
-</section>
+<p>The tipping point occurred when they began receiving many requests, internally and externally, for solutions that needed to be geographically outside the company's main data center in Germany. "Some requests were for running our applications on customer premises," Mountain says. "There were also new services we were looking to offer that required response time to the order of a few hundred milliseconds, which we couldn't achieve with transatlantic traffic. Or at least, not without eating into a considerable portion of the time available to our applications for them to process individual queries."</p>

-<div class="banner5">
-  <div class="banner5text">
-    "The bottom line is we want these multi-data center capabilities, and we want them as well for our mainstream system," he says. "And we don’t think that we can implement them with our previous system. We need the new automation, homogeneity, and scale that Kubernetes and OpenShift bring."
-  </div>
-</div>
+<p>More generally, the company was interested in leveling up on high availability, increasing automation in managing infrastructure, optimizing the distribution of workloads and using data center resources more efficiently. "We have thousands and thousands of servers," says Mountain. "These servers are assigned roles, so even if the setup is highly automated, the machine still has a given role. It's wasteful on many levels. For instance, an application doesn't necessarily use the machine very optimally. Virtualization can help a bit, but it's not a silver bullet. If that machine breaks, you still want to repair it because it has that role and you can't simply say, 'Well, I'll bring in another machine and give it that role.' It's not fast. It's not efficient. So we wanted the next level of automation."</p>

-<section class="section5">
-<div class="fullcol">
-  Next comes convincing people. "On the operations side and on the R&D side, there will be people who say quite rightly, ‘There is a system, and it works, so why change?’" Mountain says. "The only thing that really convinces people is showing them the value." For Amadeus, people realized that the Airline Cloud Availability product could not have been made available on the public cloud with the company’s existing system. The question then became, he says, "Do we go into a full-blown migration? Is that something that is justified?"
-<br><br>
-  "The bottom line is we want these multi-data center capabilities, and we want them as well for our mainstream system," he says. "And we don’t think that we can implement them with our previous system. We need the new automation, homogeneity, and scale that Kubernetes and OpenShift bring."
-<br><br>
-  So how do you get everyone on board? "Make sure you have good links between your R&D and your operations," he says. "Also make sure you’re going to talk early on to the investors and stakeholders. Figure out what it is that they will be expecting from you, that will convince them or not, that this is the right way for your company."
-<br><br>
-  His other advice is simply to make the technology available for people to try it. "Kubernetes and OpenShift Origin are open source software, so there’s no complicated license key for the evaluation period and you’re not limited to 30 days," he points out. "Just go and get it running." Along with that, he adds, "You’ve got to be prepared to rethink how you do things. Of course making your applications as cloud native as possible is how you’ll reap the most benefits: 12 factors, CI/CD, which is continuous integration, continuous delivery, but also continuous deployment."
-<br><br>
-  And while they explore that aspect of the technology, Mountain and his team will likely be practicing what he preaches to others taking the cloud native journey. "See what happens when you break it, because it’s important to understand the limits of the system," he says. Or rather, he notes, the advantages of it. "Breaking things on Kube is actually one of the nice things about it—it recovers. It’s the only real way that you’ll see that you might be able to do things."
-</div>
-</section>
+{{< case-studies/quote image="/images/case-studies/amadeus/banner3.jpg" >}}
+"We hope that if we build on what others have built, what we do might actually be upstream-able. As Kubernetes and OpenShift progress, we see that we are indeed able to remove some of the additional layers we implemented to compensate for gaps we perceived earlier."
+{{< /case-studies/quote >}}
+
+<p>While mainly a C++ and Java shop, Amadeus also wanted to be able to adopt new technologies more easily. Some of its developers had started using languages like <a href="https://www.python.org/">Python</a> and databases like <a href="https://www.couchbase.com/">Couchbase</a>, but Mountain wanted still more options, he says, "in order to better adapt our technical solutions to the products we offer, and open up entirely new possibilities to our developers." Working with recent technologies and cool new things would also make it easier to attract new talent.</p>
+
+<p>All of those needs led Mountain and his team on a search for a new platform. "We did a set of studies and proofs of concept over a fairly short period, and we considered many technologies," he says. "In the end, we were left with three choices: build everything on premise, build on top of <a href="http://kubernetes.io/">Kubernetes</a> whatever happens to be missing from our point of view, or go with <a href="https://www.openshift.com/">OpenShift</a> and build whatever remains there."</p>
+
+<p>The team decided against building everything themselves—though they'd done that sort of thing in the past—because "people were already inventing things that looked good," says Mountain.</p>
+
+<p>Ultimately, they went with OpenShift Container Platform, <a href="https://www.redhat.com/en">Red Hat</a>'s Kubernetes-based enterprise offering, instead of building on top of Kubernetes because "there was a lot of synergy between what we wanted and the way Red Hat was anticipating going with OpenShift," says Mountain. "They were clearly developing Kubernetes, and developing certain things ahead of time in OpenShift, which were important to us, such as more security."</p>
+
+<p>The hope was that those particular features would eventually be built into Kubernetes, and, in the case of security, Mountain feels that has happened. "We realize that there's always a certain amount of automation that we will probably have to develop ourselves to compensate for certain gaps," says Mountain. "The less we do that, the better for us. We hope that if we build on what others have built, what we do might actually be upstream-able. As Kubernetes and OpenShift progress, we see that we are indeed able to remove some of the additional layers we implemented to compensate for gaps we perceived earlier."</p>
+
+{{< case-studies/quote image="/images/case-studies/amadeus/banner4.jpg" >}}
+"It's not a migration of an existing workload; it's a whole new workload that we couldn't have done otherwise. [This platform] gives us access to market opportunities that we didn't have before."
+{{< /case-studies/quote >}}
+
+<p>The first project the team tackled was one that they knew had to run outside the data center in Germany. Because of the project's needs, "We couldn't rely only on the built-in Kubernetes service discovery; we had to layer on top of that an extra service discovery level that allows us to load balance at the operation level within our system," says Mountain. They also built a stream dedicated to monitoring, which at the time wasn't offered in the Kubernetes or OpenShift ecosystem. Now that <a href="https://www.prometheus.io/">Prometheus</a> and other products are available, Mountain says the company will likely re-evaluate their monitoring system: "We obviously always like to leverage what Kubernetes and OpenShift can offer."
+</p>
+
+<p>The second project ended up going into production first: the Amadeus Airline Cloud Availability solution, which helps manage ever-increasing flight-search volume and was deployed in public cloud. Launched in early 2016, it is "now handling in production several thousand transactions per second, and it's deployed in multiple data centers throughout the world," says Mountain. "It's not a migration of an existing workload; it's a whole new workload that we couldn't have done otherwise. [This platform] gives us access to market opportunities that we didn't have before."</p>
+
+<p>Having been through this kind of technical evolution more than once, Mountain has advice on how to handle the cultural changes. "That's one aspect that we can tackle progressively," he says. "We have to go on supplying our customers with new features on our pre-existing products, and we have to keep existing products working. So we can't simply do absolutely everything from one day to the next. And we mustn't sell it that way."</p>
+
+<p>The first order of business, then, is to pick one or two applications to demonstrate that the technology works. Rather than choosing a high-impact, high-risk project, Mountain's team selected a smaller application that was representative of all the company's other applications in its complexity: "We just made sure we picked something that's complex enough, and we showed that it can be done."</p>
+
+{{< case-studies/quote >}}
+"The bottom line is we want these multi-data center capabilities, and we want them as well for our mainstream system," he says. "And we don't think that we can implement them with our previous system. We need the new automation, homogeneity, and scale that Kubernetes and OpenShift bring."
+{{< /case-studies/quote >}}
+
+<p>Next comes convincing people. "On the operations side and on the R&D side, there will be people who say quite rightly, 'There is a system, and it works, so why change?'" Mountain says. "The only thing that really convinces people is showing them the value." For Amadeus, people realized that the Airline Cloud Availability product could not have been made available on the public cloud with the company's existing system. The question then became, he says, "Do we go into a full-blown migration? Is that something that is justified?"</p>
+
+<p>"The bottom line is we want these multi-data center capabilities, and we want them as well for our mainstream system," he says. "And we don't think that we can implement them with our previous system. We need the new automation, homogeneity, and scale that Kubernetes and OpenShift bring."</p>
+
+<p>So how do you get everyone on board? "Make sure you have good links between your R&D and your operations," he says. "Also make sure you're going to talk early on to the investors and stakeholders. Figure out what it is that they will be expecting from you, that will convince them or not, that this is the right way for your company."</p>
+
+<p>His other advice is simply to make the technology available for people to try it. "Kubernetes and OpenShift Origin are open source software, so there's no complicated license key for the evaluation period and you're not limited to 30 days," he points out. "Just go and get it running." Along with that, he adds, "You've got to be prepared to rethink how you do things. Of course making your applications as cloud native as possible is how you'll reap the most benefits: 12 factors, CI/CD, which is continuous integration, continuous delivery, but also continuous deployment."</p>
+
+<p>And while they explore that aspect of the technology, Mountain and his team will likely be practicing what he preaches to others taking the cloud native journey. "See what happens when you break it, because it's important to understand the limits of the system," he says. Or rather, he notes, the advantages of it. "Breaking things on Kube is actually one of the nice things about it—it recovers. It's the only real way that you'll see that you might be able to do things."</p>
@@ -1,117 +1,92 @@
 ---
 title: Ancestry Case Study
-
 case_study_styles: true
 cid: caseStudies
-css: /css/style_ancestry.css
+
+new_case_study_styles: true
+heading_background: /images/case-studies/ancestry/banner1.jpg
+heading_title_logo: /images/ancestry_logo.png
+subheading: >
+  Digging Into the Past With New Technology
+case_study_details:
+  - Company: Ancestry
+  - Location: Lehi, Utah
+  - Industry: Internet Company, Online Services
 ---

-<div class="banner1">
-  <h1> CASE STUDY:<img src="/images/ancestry_logo.png" width="22%" style="margin-bottom:-12px;margin-left:3px;"><br> <div class="subhead">Digging Into the Past With New Technology</div></h1>
-
-</div>
-
-<div class="details">
-  Company &nbsp;<b>Ancestry</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location &nbsp;<b>Lehi, Utah</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Industry &nbsp;<b>Internet Company, Online Services</b>
-</div>
-
-<hr>
-
-<section class="section1">
-<div class="cols">
-<div class="col1">
-
 <h2>Challenge</h2>
-Ancestry, the global leader in family history and consumer genomics, uses sophisticated engineering and technology to help everyone, everywhere discover the story of what led to them. The company has spent more than 30 years innovating and building products and technologies that at their core, result in real and emotional human responses. <a href="https://www.ancestry.com">Ancestry</a> currently serves more than 2.6 million paying subscribers, holds 20 billion historical records, 90 million family trees and more than four million people are in its AncestryDNA network, making it the largest consumer genomics DNA network in the world. The company's popular website, <a href="https://www.ancestry.com">ancestry.com</a>, has been working with big data long before the term was popularized. The site was built on hundreds of services, technologies and a traditional deployment methodology. "It's worked well for us in the past," says Paul MacKay, software engineer and architect at Ancestry, "but had become quite cumbersome in its processing and is time-consuming. As a primarily online service, we are constantly looking for ways to accelerate to be more agile in delivering our solutions and our&nbsp;products."

-<br>
+<p>Ancestry, the global leader in family history and consumer genomics, uses sophisticated engineering and technology to help everyone, everywhere discover the story of what led to them. The company has spent more than 30 years innovating and building products and technologies that at their core, result in real and emotional human responses. <a href="https://www.ancestry.com">Ancestry</a> currently serves more than 2.6 million paying subscribers, holds 20 billion historical records, 90 million family trees and more than four million people are in its AncestryDNA network, making it the largest consumer genomics DNA network in the world. The company's popular website, <a href="https://www.ancestry.com">ancestry.com</a>, has been working with big data long before the term was popularized. The site was built on hundreds of services, technologies and a traditional deployment methodology. "It's worked well for us in the past," says Paul MacKay, software engineer and architect at Ancestry, "but had become quite cumbersome in its processing and is time-consuming. As a primarily online service, we are constantly looking for ways to accelerate to be more agile in delivering our solutions and our&nbsp;products."</p>

-</div>
+<h2>Solution</h2>

-<div class="col2">
-  <h2>Solution</h2>
+<p>The company is transitioning to cloud native infrastructure, using <a href="https://www.docker.com">Docker</a> containerization, <a href="https://kubernetes.io">Kubernetes</a> orchestration and <a href="https://prometheus.io">Prometheus</a> for cluster monitoring.</p>

-  The company is transitioning to cloud native infrastructure, using <a href="https://www.docker.com">Docker</a> containerization, <a href="https://kubernetes.io">Kubernetes</a> orchestration and <a href="https://prometheus.io">Prometheus</a> for cluster monitoring.<br>
-<br>
 <h2>Impact</h2>
-  "Every single product, every decision we make at Ancestry, focuses on delighting our customers with intimate, sometimes life-changing discoveries about themselves and their families," says MacKay. "As the company continues to grow, the increased productivity gains from using Kubernetes has helped Ancestry make customer discoveries faster. With the move to Dockerization for example, instead of taking between 20 to 50 minutes to deploy a new piece of code, we can now deploy in under a minute for much of our code. We’ve truly experienced significant time savings in addition to the various features and benefits from cloud native and Kubernetes-type technologies."
-</div>
-</div>
-</section>

-<div class="banner2">
-  <div class="banner2text">
-    "At a certain point, you have to step back if you're going to push a new technology and get key thought leaders with engineers within the organization to become your champions for new technology adoption. At training sessions, the development teams were always the ones that were saying, 'Kubernetes saved our time tremendously; it's an enabler. It really is incredible.'"<br><br><span style="font-size:16px">- PAUL MACKAY, SOFTWARE ENGINEER AND ARCHITECT AT ANCESTRY</span>
-  </div>
-</div>
+<p>"Every single product, every decision we make at Ancestry, focuses on delighting our customers with intimate, sometimes life-changing discoveries about themselves and their families," says MacKay. "As the company continues to grow, the increased productivity gains from using Kubernetes has helped Ancestry make customer discoveries faster. With the move to Dockerization for example, instead of taking between 20 to 50 minutes to deploy a new piece of code, we can now deploy in under a minute for much of our code. We've truly experienced significant time savings in addition to the various features and benefits from cloud native and Kubernetes-type technologies."</p>

-<section class="section2">
-<div class="fullcol">
-<h2>It started with a Shaky Leaf.</h2>
+{{< case-studies/quote author="PAUL MACKAY, SOFTWARE ENGINEER AND ARCHITECT AT ANCESTRY" >}}
+"At a certain point, you have to step back if you're going to push a new technology and get key thought leaders with engineers within the organization to become your champions for new technology adoption. At training sessions, the development teams were always the ones that were saying, 'Kubernetes saved our time tremendously; it's an enabler. It really is incredible.'"  
+{{< /case-studies/quote >}}

-    Since its introduction a decade ago, the Shaky Leaf icon has become one of Ancestry's signature features, which signals to users that there's a helpful hint you can use to find out more about your family tree.<br><br>
-    So when the company decided to begin moving its infrastructure to cloud native technology, the first service that was launched on <a href="https://kubernetes.io">Kubernetes</a>, the open source platform for managing application containers across clusters of hosts, was this hint system. Think of it as Amazon's recommended products, but instead of recommending products the company recommends records, stories, or familial connections. "It was a very important part of the site," says Ancestry software engineer and architect Paul MacKay, "but also small enough for a pilot project that we knew we could handle in a very appropriate, secure way."<br><br>
-    And when it went live smoothly in early 2016, "our deployment time for this service literally was cut down from 50 minutes to 2 or 5 minutes," MacKay adds. "The development team was just thrilled because we're focused on supplying a great experience for our customers. And that means features, it means stability, it means all those things that we need for a first-in-class type operation."<br><br>
-    The stability of that Shaky Leaf was a signal for MacKay and his team that their decision to embrace cloud native technologies was the right one for the company. With a private data center, Ancestry built its website (which launched in 1996) on hundreds of services and technologies and a traditional deployment methodology. "It worked well for us in the past, but the sum of the legacy systems became quite cumbersome in its processing and was time-consuming," says MacKay. "We were looking for other ways to accelerate, to be more agile in delivering our solutions and our products."
-</div>
-</section>
+{{< case-studies/lead >}}
+It started with a Shaky Leaf.
+{{< /case-studies/lead >}}

-<div class="banner3">
-  <div class="banner3text">
+<p>Since its introduction a decade ago, the Shaky Leaf icon has become one of Ancestry's signature features, which signals to users that there's a helpful hint you can use to find out more about your family tree.</p>
+
+<p>So when the company decided to begin moving its infrastructure to cloud native technology, the first service that was launched on <a href="https://kubernetes.io">Kubernetes</a>, the open source platform for managing application containers across clusters of hosts, was this hint system. Think of it as Amazon's recommended products, but instead of recommending products the company recommends records, stories, or familial connections. "It was a very important part of the site," says Ancestry software engineer and architect Paul MacKay, "but also small enough for a pilot project that we knew we could handle in a very appropriate, secure way."</p>
+
+<p>And when it went live smoothly in early 2016, "our deployment time for this service literally was cut down from 50 minutes to 2 or 5 minutes," MacKay adds. "The development team was just thrilled because we're focused on supplying a great experience for our customers. And that means features, it means stability, it means all those things that we need for a first-in-class type operation."</p>
+
+<p>The stability of that Shaky Leaf was a signal for MacKay and his team that their decision to embrace cloud native technologies was the right one for the company. With a private data center, Ancestry built its website (which launched in 1996) on hundreds of services and technologies and a traditional deployment methodology. "It worked well for us in the past, but the sum of the legacy systems became quite cumbersome in its processing and was time-consuming," says MacKay. "We were looking for other ways to accelerate, to be more agile in delivering our solutions and our products."</p>
+
+{{< case-studies/quote image="/images/case-studies/ancestry/banner3.jpg" >}}
 "And when it [Kubernetes] went live smoothly in early 2016, 'our deployment time for this service literally was cut down from 50 minutes to 2 or 5 minutes,' MacKay adds. 'The development team was just thrilled because we're focused on supplying a great experience for our customers. And that means features, it means stability, it means all those things that we need for a first-in-class type operation.'"
-  </div>
-</div>
+{{< /case-studies/quote >}}

-<section class="section3">
-<div class="fullcol">
-  That need led them in 2015 to explore containerization. Ancestry engineers had already been using technology like <a href="https://www.java.com/en/">Java</a> and <a href="https://www.python.org">Python</a> on Linux, so part of the decision was about making the infrastructure more Linux-friendly. They quickly decided that they wanted to go with Docker for containerization, "but it always comes down to the orchestration part of it to make it really work," says MacKay.<br><br>
-  His team looked at orchestration platforms offered by <a href="https://docs.docker.com/compose/">Docker Compose</a>, <a href="http://mesos.apache.org">Mesos</a> and <a href="https://www.openstack.org/software/">OpenStack</a>, and even started to prototype some homegrown solutions. And then they started hearing rumblings of the imminent release of Kubernetes v1.0. "At the forefront, we were looking at the secret store, so we didn't have to manage that all ourselves, the config maps, the methodology of seamless deployment strategy," he says. "We found that how Kubernetes had done their resources, their types, their labels and just their interface was so much further advanced than the other things we had seen. It was a feature fit."<br><br>
-  <div class="quote">
-  Plus, MacKay says, "I just believed in the confidence that comes with the history that Google has with containerization. So we started out right on the leading edge of it. And we haven't looked back since."</div><br>
-  Which is not to say that adopting a new technology hasn't come with some challenges. "Change is hard," says MacKay. "Not because the technology is hard or that the technology is not good. It's just that people like to do things like they had done [before]. You have the early adopters and you have those who are coming in later. It was a learning experience on both sides."<br><br>
-  Figuring out the best deployment operations for Ancestry was a big part of the work it took to adopt cloud native infrastructure. "We want to make sure the process is easy and also controlled in the manner that allows us the highest degree of security that we demand and our customers demand," says MacKay. "With Kubernetes and other products, there are some good solutions, but a little bit of glue is needed to bring it into corporate processes and governances. It's like having a set of gloves that are generic, but when you really do want to grab something you have to make it so it's customized to you. That's what we had to do."<br><br>
-  Their best practices include allowing their developers to deploy into development stage and production, but then controlling the aspects that need governance and auditing, such as secrets. They found that having one namespace per service is useful for achieving that containment of secrets and config maps. And for their needs, having one container per pod makes it easier to manage and to have a smaller unit of deployment.
-<br><br>
-</div>
-</section>
+<p>That need led them in 2015 to explore containerization. Ancestry engineers had already been using technology like <a href="https://www.java.com/en/">Java</a> and <a href="https://www.python.org">Python</a> on Linux, so part of the decision was about making the infrastructure more Linux-friendly. They quickly decided that they wanted to go with Docker for containerization, "but it always comes down to the orchestration part of it to make it really work," says MacKay.</p>

-<div class="banner4">
-<div class="banner4text">
+<p>His team looked at orchestration platforms offered by <a href="https://docs.docker.com/compose/">Docker Compose</a>, <a href="http://mesos.apache.org">Mesos</a> and <a href="https://www.openstack.org/software/">OpenStack</a>, and even started to prototype some homegrown solutions. And then they started hearing rumblings of the imminent release of Kubernetes v1.0. "At the forefront, we were looking at the secret store, so we didn't have to manage that all ourselves, the config maps, the methodology of seamless deployment strategy," he says. "We found that how Kubernetes had done their resources, their types, their labels and just their interface was so much further advanced than the other things we had seen. It was a feature fit."</p>

+{{< case-studies/lead >}}
+Plus, MacKay says, "I just believed in the confidence that comes with the history that Google has with containerization. So we started out right on the leading edge of it. And we haven't looked back since."
+{{< /case-studies/lead >}}
+
+<p>Which is not to say that adopting a new technology hasn't come with some challenges. "Change is hard," says MacKay. "Not because the technology is hard or that the technology is not good. It's just that people like to do things like they had done [before]. You have the early adopters and you have those who are coming in later. It was a learning experience on both sides."</p>
+
+<p>Figuring out the best deployment operations for Ancestry was a big part of the work it took to adopt cloud native infrastructure. "We want to make sure the process is easy and also controlled in the manner that allows us the highest degree of security that we demand and our customers demand," says MacKay. "With Kubernetes and other products, there are some good solutions, but a little bit of glue is needed to bring it into corporate processes and governances. It's like having a set of gloves that are generic, but when you really do want to grab something you have to make it so it's customized to you. That's what we had to do."</p>
+
+<p>Their best practices include allowing their developers to deploy into development stage and production, but then controlling the aspects that need governance and auditing, such as secrets. They found that having one namespace per service is useful for achieving that containment of secrets and config maps. And for their needs, having one container per pod makes it easier to manage and to have a smaller unit of deployment.
+</p>
+
+{{< case-studies/quote image="/images/case-studies/ancestry/banner4.jpg" >}}
 "The success of Ancestry's first deployment of the hint system on Kubernetes helped create momentum for greater adoption of the technology."
+{{< /case-studies/quote >}}

-</div>
-</div>
+<p>With that process established, the time spent on deployment was cut down to under a minute for some services. "As programmers, we have what's called REPL: read, evaluate, print, and loop, but with Kubernetes, we have CDEL: compile, deploy, execute, and loop," says MacKay. "It's a very quick loop back and a great benefit to understand that when our services are deployed in production, they're the same as what we tested in the pre-production environments. The approach of cloud native for Ancestry provides us a better ability to scale and to accommodate the business needs as work loads occur."</p>

-<section class="section4">
-<div class="fullcol">
-  With that process established, the time spent on deployment was cut down to under a minute for some services. "As programmers, we have what's called REPL: read, evaluate, print, and loop, but with Kubernetes, we have CDEL: compile, deploy, execute, and loop," says MacKay. "It's a very quick loop back and a great benefit to understand that when our services are deployed in production, they're the same as what we tested in the pre-production environments. The approach of cloud native for Ancestry provides us a better ability to scale and to accommodate the business needs as work loads occur."<br><br>
-  The success of Ancestry's first deployment of the hint system on Kubernetes helped create momentum for greater adoption of the technology. "Engineers like to code, they like to do features, they don't like to sit around waiting for things to be deployed and worrying about scaling up and out and down," says MacKay. "After a while the engineers became our champions. At training sessions, the development teams were always the ones saying, 'Kubernetes saved our time tremendously; it's an enabler; it really is incredible.' Over time, we were able to convince our management that this was a transition that the industry is making and that we needed to be a part of it."<br><br>
-  A year later, Ancestry has transitioned a good number of applications to Kubernetes. "We have many different services that make up the rich environment that [the website] has from both the DNA side and the family history side," says MacKay. "We have front-end stacks, back-end stacks and back-end processing type stacks that are in the cluster."<br><br>
-  The company continues to weigh which services it will move forward to Kubernetes, which ones will be kept as is, and which will be replaced in the future and thus don't have to be moved over. MacKay estimates that the company is "approaching halfway on those features that are going forward. We don't have to do a lot of convincing anymore. It's more of an issue of timing with getting product management and engineering staff the knowledge and information that they need."
-</div>
-</section>
+<p>The success of Ancestry's first deployment of the hint system on Kubernetes helped create momentum for greater adoption of the technology. "Engineers like to code, they like to do features, they don't like to sit around waiting for things to be deployed and worrying about scaling up and out and down," says MacKay. "After a while the engineers became our champions. At training sessions, the development teams were always the ones saying, 'Kubernetes saved our time tremendously; it's an enabler; it really is incredible.' Over time, we were able to convince our management that this was a transition that the industry is making and that we needed to be a part of it."</p>

-<div class="banner5">
-  <div class="banner5text">
-    "... 'I believe in Kubernetes. I believe in containerization. I think
-    if we can get there and establish ourselves in that world, we will be further along and far better off being agile and all the things we talk about,
-    and it'll&nbsp;go&nbsp;forward.'"
-  </div>
-</div>
+<p>A year later, Ancestry has transitioned a good number of applications to Kubernetes. "We have many different services that make up the rich environment that [the website] has from both the DNA side and the family history side," says MacKay. "We have front-end stacks, back-end stacks and back-end processing type stacks that are in the cluster."</p>

-<section class="section5">
-<div class="fullcol">
+<p>The company continues to weigh which services it will move forward to Kubernetes, which ones will be kept as is, and which will be replaced in the future and thus don't have to be moved over. MacKay estimates that the company is "approaching halfway on those features that are going forward. We don't have to do a lot of convincing anymore. It's more of an issue of timing with getting product management and engineering staff the knowledge and information that they need."</p>

+{{< case-studies/quote >}}
+"... 'I believe in Kubernetes. I believe in containerization. I think if we can get there and establish ourselves in that world, we will be further along and far better off being agile and all the things we talk about, and it'll&nbsp;go&nbsp;forward.'"
+{{< /case-studies/quote >}}

-Looking ahead, MacKay sees Ancestry maximizing the benefits of Kubernetes in 2017. "We're very close to having everything that should be or could be in a Linux-friendly world in Kubernetes by the end of the year," he says, adding that he's looking forward to features such as federation and horizontal pod autoscaling that are currently in the works. "Kubernetes has been very wonderful for us and we continue to ride the wave."<br><br>
-That wave, he points out, has everything to do with the vibrant Kubernetes community, which has grown by leaps and bounds since Ancestry joined it as an early adopter. "This is just a very rough way of judging it, but on Slack in June 2015, there were maybe 500 on there," MacKay says. "The last time I looked there were maybe 8,500 just on the Slack channel. There are so many major companies and different kinds of companies involved now. It's the variety of contributors, the number of contributors, the incredibly competent and friendly community."<br><br>
-As much as he and his team at Ancestry have benefited from what he calls "the goodness and the technical abilities of many" in the community, they've also contributed information about best practices, logged bug issues and participated in the open source conversation. And they've been active in attending <a href="https://www.meetup.com/Utah-Kubernetes-Meetup/">meetups</a> to help educate and give back to the local tech community in Utah. Says MacKay: "We're trying to give back as far as our experience goes, rather than just code."
-<br><br>When he meets with companies considering adopting cloud native infrastructure, the best advice he has to give from Ancestry's Kubernetes journey is this: "Start small, but with hard problems," he says. And "you need a patron who understands the vision of containerization, to help you tackle the political as well as other technical roadblocks that can occur when change is needed."<br><br>
-With the changes that MacKay's team has led over the past year and a half, cloud native will be part of Ancestry's technological genealogy for years to come. MacKay has been such a champion of the technology that he says people have jokingly accused him of having a Kubernetes tattoo.<br><br>
-"I really don't," he says with a laugh. "But I'm passionate. I'm not exclusive to any technology; I use whatever I need that's out there that makes us great. If it's something else, I'll use it. But right now I believe in Kubernetes. I believe in containerization. I think if we can get there and establish ourselves in that world, we will be further along and far better off being agile and all the things we talk about, and it'll go forward."<br><br>
-He pauses. "So, yeah, I guess you can say I'm an evangelist for Kubernetes," he says. "But I'm not getting a tattoo!"
+<p>Looking ahead, MacKay sees Ancestry maximizing the benefits of Kubernetes in 2017. "We're very close to having everything that should be or could be in a Linux-friendly world in Kubernetes by the end of the year," he says, adding that he's looking forward to features such as federation and horizontal pod autoscaling that are currently in the works. "Kubernetes has been very wonderful for us and we continue to ride the wave."</p>

+<p>That wave, he points out, has everything to do with the vibrant Kubernetes community, which has grown by leaps and bounds since Ancestry joined it as an early adopter. "This is just a very rough way of judging it, but on Slack in June 2015, there were maybe 500 on there," MacKay says. "The last time I looked there were maybe 8,500 just on the Slack channel. There are so many major companies and different kinds of companies involved now. It's the variety of contributors, the number of contributors, the incredibly competent and friendly community."</p>

-</div>
-</section>
+<p>As much as he and his team at Ancestry have benefited from what he calls "the goodness and the technical abilities of many" in the community, they've also contributed information about best practices, logged bug issues and participated in the open source conversation. And they've been active in attending <a href="https://www.meetup.com/Utah-Kubernetes-Meetup/">meetups</a> to help educate and give back to the local tech community in Utah. Says MacKay: "We're trying to give back as far as our experience goes, rather than just code."</p>
+
+<p>When he meets with companies considering adopting cloud native infrastructure, the best advice he has to give from Ancestry's Kubernetes journey is this: "Start small, but with hard problems," he says. And "you need a patron who understands the vision of containerization, to help you tackle the political as well as other technical roadblocks that can occur when change is needed."</p>
+
+<p>With the changes that MacKay's team has led over the past year and a half, cloud native will be part of Ancestry's technological genealogy for years to come. MacKay has been such a champion of the technology that he says people have jokingly accused him of having a Kubernetes tattoo.</p>
+
+<p>"I really don't," he says with a laugh. "But I'm passionate. I'm not exclusive to any technology; I use whatever I need that's out there that makes us great. If it's something else, I'll use it. But right now I believe in Kubernetes. I believe in containerization. I think if we can get there and establish ourselves in that world, we will be further along and far better off being agile and all the things we talk about, and it'll go forward."</p>
+
+<p>He pauses. "So, yeah, I guess you can say I'm an evangelist for Kubernetes," he says. "But I'm not getting a tattoo!"</p>
@@ -3,94 +3,80 @@ title: Ant Financial Case Study
 linkTitle: ant-financial
 case_study_styles: true
 cid: caseStudies
-css: /css/style_case_studies.css
 featured: false
+
+new_case_study_styles: true
+heading_background: /images/case-studies/antfinancial/banner1.jpg
+heading_title_logo: /images/antfinancial_logo.png
+subheading: >
+  Ant Financial's Hypergrowth Strategy Using Kubernetes
+case_study_details:
+  - Company: Ant Financial
+  - Location: Hangzhou, China
+  - Industry: Financial Services
 ---

-<div class="banner1" style="background-image: url('/images/case-studies/antfinancial/banner1.jpg')">
-  <h1> CASE STUDY:<img src="/images/antfinancial_logo.png" class="header_logo" style="width:20%;margin-bottom:-2.5%"><br> <div class="subhead" style="margin-top:1%">Ant Financial’s Hypergrowth Strategy Using Kubernetes
+<h2>Challenge</h2>

-</div></h1>
+<p>Officially founded in October 2014, <a href="https://www.antfin.com/index.htm?locale=en_us">Ant Financial</a> originated from <a href="https://global.alipay.com/">Alipay</a>, the world's largest online payment platform that launched in 2004. The company also offers numerous other services leveraging technology innovation. With the volume of transactions Alipay handles for its 900+ million users worldwide (through its local and global partners)—256,000 transactions per second at the peak of Double 11 Singles Day 2017, and total gross merchandise value of $31 billion for Singles Day 2018—not to mention that of its other services, Ant Financial faces "data processing challenge in a whole new way," says Haojie Hang, who is responsible for Product Management for the Storage and Compute Group. "We see three major problems of operating at that scale: how to provide real-time compute, storage, and processing capability, for instance to make real-time recommendations for fraud detection; how to provide intelligence on top of this data, because there's too much data and then we're not getting enough insight; and how to apply security in the application level, in the middleware level, the system level, even the chip level." In order to provide reliable and consistent services to its customers, Ant Financial embraced containers in early 2014, and soon needed an orchestration solution for the tens-of-thousands-of-node clusters in its data centers.</p>

-</div>
+<h2>Solution</h2>

-<div class="details">
-    Company &nbsp;<b>Ant Financial</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location &nbsp;<b>Hangzhou, China</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Industry &nbsp;<b>Financial Services</b>
-</div>
+<p>After investigating several technologies, the team chose <a href="https://kubernetes.io/">Kubernetes</a> for orchestration, as well as a number of other CNCF projects, including <a href="https://prometheus.io/">Prometheus</a>, <a href="https://opentracing.io/">OpenTracing</a>, <a href="https://coreos.com/etcd/">etcd</a> and <a href="https://coredns.io/">CoreDNS</a>. "In late 2016, we decided that Kubernetes will be the de facto standard," says Hang. "Looking back, we made the right bet on the right technology. But then we needed to move the production workload from the legacy infrastructure to the latest Kubernetes-enabled platform, and that took some time, because we are very careful in terms of reliability and consistency." All core financial systems were containerized by November 2017, and the migration to Kubernetes is ongoing.</p>

-<hr>
-<section class="section1">
-<div class="cols">
-  <div class="col1" style="width:100%"">
-    <h2>Challenge</h2>
-    Officially founded in October 2014, <a href="https://www.antfin.com/index.htm?locale=en_us">Ant Financial</a> originated from <a href="https://global.alipay.com/">Alipay</a>, the world’s largest online payment platform that launched in 2004. The company also offers numerous other services leveraging technology innovation. With the volume of transactions Alipay handles for its 900+ million users worldwide (through its local and global partners)—256,000 transactions per second at the peak of Double 11 Singles Day 2017, and total gross merchandise value of $31 billion for Singles Day 2018—not to mention that of its other services, Ant Financial faces “data processing challenge in a whole new way,” says Haojie Hang, who is responsible for Product Management for the Storage and Compute Group. “We see three major problems of operating at that scale: how to provide real-time compute, storage, and processing capability, for instance to make real-time recommendations for fraud detection; how to provide intelligence on top of this data, because there’s too much data and then we’re not getting enough insight; and how to apply security in the application level, in the middleware level, the system level, even the chip level.” In order to provide reliable and consistent services to its customers, Ant Financial embraced containers in early 2014, and soon needed an orchestration solution for the tens-of-thousands-of-node clusters in its data centers.
-
-    <h2>Solution</h2>
-    After investigating several technologies, the team chose <a href="https://kubernetes.io/">Kubernetes</a> for orchestration, as well as a number of other CNCF projects, including <a href="https://prometheus.io/">Prometheus</a>, <a href="https://opentracing.io/">OpenTracing</a>, <a href="https://coreos.com/etcd/">etcd</a> and <a href="https://coredns.io/">CoreDNS</a>. “In late 2016, we decided that Kubernetes will be the de facto standard,” says Hang. “Looking back, we made the right bet on the right technology. But then we needed to move the production workload from the legacy infrastructure to the latest Kubernetes-enabled platform, and that took some time, because we are very careful in terms of reliability and consistency.” All core financial systems were containerized by November 2017, and the migration to Kubernetes is ongoing.
-<br>
 <h2>Impact</h2>
-    “We’ve seen at least tenfold in improvement in terms of the operations with cloud native technology, which means you can have tenfold increase in terms of output,” says Hang. Ant also provides its fully integrated financial cloud platform to business partners around the world, and hopes to power the next generation of digital banking with deep experience in service innovation and technology expertise. Hang says the team hasn’t begun to focus on optimizing the Kubernetes platform, either: “Because we’re still in the hyper growth stage, we’re not in a mode where we do cost saving yet.”
-</div>

-</div>
-</section>
-<div class="banner2">
-  <div class="banner2text">
-    "In late 2016, we decided that Kubernetes will be the de facto standard. Looking back, we made the right bet on the right technology."
-<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- HAOJIE HANG, PRODUCT MANAGEMENT, ANT FINANCIAL</span>
-  </div>
-</div>
-<section class="section2">
-<div class="fullcol">
-  <h2>A spinoff of the multinational conglomerate Alibaba, Ant Financial boasts a $150+ billion valuation and the scale to match. The fintech startup, launched in 2014, is comprised of Alipay, the world’s largest online payment platform, and numerous other services leveraging technology innovation.</h2>
-  And the volume of transactions that Alipay handles for over 900 million users worldwide (through its local and global partners) is staggering: 256,000 per second at the peak of Double 11 Singles Day 2017, and total gross merchandise value of $31 billion for Singles Day 2018. With the mission of “bringing the world equal opportunities,” Ant Financial is dedicated to creating an open, shared credit system and financial services platform through technology innovations.
-<br><br>
-  Combine that with the operations of its other properties—such as the Huabei online credit system, Jiebei lending service, and the 350-million-user <a href="https://en.wikipedia.org/wiki/Ant_Forest">Ant Forest</a> green energy mobile app—and Ant Financial faces “data processing challenge in a whole new way,” says Haojie Hang, who is responsible for Product Management for the Storage and Compute Group. “We see three major problems of operating at that scale: how to provide real-time compute, storage, and processing capability, for instance to make real-time recommendations for fraud detection; how to provide intelligence on top of this data, because there’s too much data and we’re not getting enough insight; and how to apply security in the application level, in the middleware level, the system level, even the chip level.”
-<br><br>
-  To address those challenges and provide reliable and consistent services to its customers, Ant Financial embraced <a href="https://www.docker.com/">Docker</a> containerization in 2014. But they soon realized that they needed an orchestration solution for some tens-of-thousands-of-node clusters in the company’s data centers.
-</div>
-</section>
-<div class="banner3" style="background-image: url('/images/case-studies/antfinancial/banner3.jpg')">
-  <div class="banner3text">
-    "On Double 11 this year, we had plenty of nodes on Kubernetes, but compared to the whole scale of our infrastructure, this is still in progress."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- RANGER YU, GLOBAL TECHNOLOGY PARTNERSHIP & DEVELOPMENT, ANT FINANCIAL</span>
+<p>"We've seen at least tenfold in improvement in terms of the operations with cloud native technology, which means you can have tenfold increase in terms of output," says Hang. Ant also provides its fully integrated financial cloud platform to business partners around the world, and hopes to power the next generation of digital banking with deep experience in service innovation and technology expertise. Hang says the team hasn't begun to focus on optimizing the Kubernetes platform, either: "Because we're still in the hyper growth stage, we're not in a mode where we do cost saving yet."</p>

-  </div>
-</div>
-<section class="section3">
-<div class="fullcol">
-  The team investigated several technologies, including Docker Swarm and Mesos. “We did a lot of POCs, but we’re very careful in terms of production systems, because we want to make sure we don’t lose any data,” says Hang. “You cannot afford to have a service downtime for one minute; even one second has a very, very big impact. We operate every day under pressure to provide reliable and consistent services to consumers and businesses in China and globally.”
-<br><br>
-  Ultimately, Hang says Ant chose Kubernetes because it checked all the boxes: a strong community, technology that “will be relevant in the next three to five years,” and a good match for the company’s engineering talent. “In late 2016, we decided that Kubernetes will be the de facto standard,” says Hang. “Looking back, we made the right bet on the right technology. But then we needed to move the production workload from the legacy infrastructure to the latest Kubernetes-enabled platform. We spent a lot of time learning and then training our people to build applications on Kubernetes well.”
-<br><br>
-  All core financial systems were containerized by November 2017, and the migration to Kubernetes is ongoing. Ant’s platform also leverages a number of other CNCF projects, including <a href="https://prometheus.io/">Prometheus</a>, <a href="https://opentracing.io/">OpenTracing</a>, <a href="https://coreos.com/etcd/">etcd</a> and <a href="https://coredns.io/">CoreDNS</a>.  “On Double 11 this year, we had plenty of nodes on Kubernetes, but compared to the whole scale of our infrastructure, this is still in progress,” says Ranger Yu, Global Technology Partnership & Development.
-</div>
-</section>
-<div class="banner4" style="background-image: url('/images/case-studies/antfinancial/banner4.jpg')">
-  <div class="banner4text">
-  "We’re very grateful for CNCF and this amazing technology, which we need as we continue to scale globally. We’re definitely embracing the community and open source more in the future." <br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- HAOJIE HANG, PRODUCT MANAGEMENT, ANT FINANCIAL</span>
-  </div>
-</div>
+{{< case-studies/quote author="HAOJIE HANG, PRODUCT MANAGEMENT, ANT FINANCIAL" >}}
+"In late 2016, we decided that Kubernetes will be the de facto standard. Looking back, we made the right bet on the right technology."
+{{< /case-studies/quote >}}

-<section class="section5" style="padding:0px !important">
-<div class="fullcol">
-  Still, there has already been an impact. “Cloud native technology has benefited us greatly in terms of efficiency,” says Hang. “In general, we want to make sure our infrastructure is nimble and flexible enough for the work that could happen tomorrow. That’s the goal. And with cloud native technology, we’ve seen at least tenfold improvement in operations, which means you can have tenfold increase in terms of output. Let’s say you are operating 10 nodes with one person. With cloud native, tomorrow you can have 100 nodes.”
-<br><br>
-  Ant also provides its financial cloud platform to partners around the world, and hopes to power the next generation of digital banking with deep experience in service innovation and technology expertise. Hang says the team hasn’t begun to focus on optimizing the Kubernetes platform, either: “Because we’re still in the hyper growth stage, we’re not in a mode where we do cost-saving yet.”
-<br><br>
-  The CNCF community has also been a valuable asset during Ant Financial’s move to cloud native. “If you are applying a new technology, it’s very good to have a community to discuss technical problems with other users,” says Hang. “We’re very grateful for CNCF and this amazing technology, which we need as we continue to scale globally. We’re definitely embracing the community and open sourcing more in the future.”
-</div>
+{{< case-studies/lead >}}
+A spinoff of the multinational conglomerate Alibaba, Ant Financial boasts a $150+ billion valuation and the scale to match. The fintech startup, launched in 2014, is comprised of Alipay, the world's largest online payment platform, and numerous other services leveraging technology innovation.
+{{< /case-studies/lead >}}

-<div class="banner5" >
-  <div class="banner5text">
-"In China, we are the North Star in terms of innovation in financial and other related services,” says Hang. “We definitely want to make sure we’re still leading in the next 5 to 10 years with our investment in technology."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- RANGER YU, GLOBAL TECHNOLOGY PARTNERSHIP & DEVELOPMENT, ANT FINANCIAL</span></div>
-</div>
+<p>And the volume of transactions that Alipay handles for over 900 million users worldwide (through its local and global partners) is staggering: 256,000 per second at the peak of Double 11 Singles Day 2017, and total gross merchandise value of $31 billion for Singles Day 2018. With the mission of "bringing the world equal opportunities," Ant Financial is dedicated to creating an open, shared credit system and financial services platform through technology innovations.</p>

-<div class="fullcol">
-  In fact, the company has already started to open source some of its <a href="https://github.com/alipay">cloud native middleware</a>. “We are going to be very proactive about that,” says Yu. “CNCF provided a platform so everyone can plug in or contribute components. This is very good open source governance.”
-<br><br>
-  Looking ahead, the Ant team will continue to evaluate many other CNCF projects. Building a service mesh community in China, the team has brought together many China-based companies and developers to discuss the potential of that technology. “Service mesh is very attractive for Chinese developers and end users because we have a lot of legacy systems running now, and it’s an ideal mid-layer to glue everything together, both new and legacy,” says Hang. “For new technologies, we look very closely at whether they will last.”
-<br><br>
-  At Ant, Kubernetes passed that test with flying colors, and the team hopes other companies will follow suit. “In China, we are the North Star in terms of innovation in financial and other related services,” says Hang. “We definitely want to make sure we’re still leading in the next 5 to 10 years with our investment in technology.”
+<p>Combine that with the operations of its other properties—such as the Huabei online credit system, Jiebei lending service, and the 350-million-user <a href="https://en.wikipedia.org/wiki/Ant_Forest">Ant Forest</a> green energy mobile app—and Ant Financial faces "data processing challenge in a whole new way," says Haojie Hang, who is responsible for Product Management for the Storage and Compute Group. "We see three major problems of operating at that scale: how to provide real-time compute, storage, and processing capability, for instance to make real-time recommendations for fraud detection; how to provide intelligence on top of this data, because there's too much data and we're not getting enough insight; and how to apply security in the application level, in the middleware level, the system level, even the chip level."</p>

-</div>
-</section>
+<p>To address those challenges and provide reliable and consistent services to its customers, Ant Financial embraced <a href="https://www.docker.com/">Docker</a> containerization in 2014. But they soon realized that they needed an orchestration solution for some tens-of-thousands-of-node clusters in the company's data centers.</p>
+
+{{< case-studies/quote 
+  image="/images/case-studies/antfinancial/banner3.jpg"
+  author="RANGER YU, GLOBAL TECHNOLOGY PARTNERSHIP & DEVELOPMENT, ANT FINANCIAL"
+>}}
+"On Double 11 this year, we had plenty of nodes on Kubernetes, but compared to the whole scale of our infrastructure, this is still in progress."
+{{< /case-studies/quote >}}
+
+<p>The team investigated several technologies, including Docker Swarm and Mesos. "We did a lot of POCs, but we're very careful in terms of production systems, because we want to make sure we don't lose any data," says Hang. "You cannot afford to have a service downtime for one minute; even one second has a very, very big impact. We operate every day under pressure to provide reliable and consistent services to consumers and businesses in China and globally."</p>
+
+<p>Ultimately, Hang says Ant chose Kubernetes because it checked all the boxes: a strong community, technology that "will be relevant in the next three to five years," and a good match for the company's engineering talent. "In late 2016, we decided that Kubernetes will be the de facto standard," says Hang. "Looking back, we made the right bet on the right technology. But then we needed to move the production workload from the legacy infrastructure to the latest Kubernetes-enabled platform. We spent a lot of time learning and then training our people to build applications on Kubernetes well."</p>
+
+<p>All core financial systems were containerized by November 2017, and the migration to Kubernetes is ongoing. Ant's platform also leverages a number of other CNCF projects, including <a href="https://prometheus.io/">Prometheus</a>, <a href="https://opentracing.io/">OpenTracing</a>, <a href="https://coreos.com/etcd/">etcd</a> and <a href="https://coredns.io/">CoreDNS</a>.  "On Double 11 this year, we had plenty of nodes on Kubernetes, but compared to the whole scale of our infrastructure, this is still in progress," says Ranger Yu, Global Technology Partnership & Development.</p>
+
+{{< case-studies/quote 
+  image="/images/case-studies/antfinancial/banner4.jpg"
+  author="HAOJIE HANG, PRODUCT MANAGEMENT, ANT FINANCIAL"
+>}}
+"We're very grateful for CNCF and this amazing technology, which we need as we continue to scale globally. We're definitely embracing the community and open source more in the future."
+{{< /case-studies/quote >}}
+
+<p>Still, there has already been an impact. "Cloud native technology has benefited us greatly in terms of efficiency," says Hang. "In general, we want to make sure our infrastructure is nimble and flexible enough for the work that could happen tomorrow. That's the goal. And with cloud native technology, we've seen at least tenfold improvement in operations, which means you can have tenfold increase in terms of output. Let's say you are operating 10 nodes with one person. With cloud native, tomorrow you can have 100 nodes."</p>
+
+<p>Ant also provides its financial cloud platform to partners around the world, and hopes to power the next generation of digital banking with deep experience in service innovation and technology expertise. Hang says the team hasn't begun to focus on optimizing the Kubernetes platform, either: "Because we're still in the hyper growth stage, we're not in a mode where we do cost-saving yet."</p>
+
+<p>The CNCF community has also been a valuable asset during Ant Financial's move to cloud native. "If you are applying a new technology, it's very good to have a community to discuss technical problems with other users," says Hang. "We're very grateful for CNCF and this amazing technology, which we need as we continue to scale globally. We're definitely embracing the community and open sourcing more in the future."</p>
+
+{{< case-studies/quote 
+  image="/images/case-studies/antfinancial/banner4.jpg"
+  author="RANGER YU, GLOBAL TECHNOLOGY PARTNERSHIP & DEVELOPMENT, ANT FINANCIAL"
+>}}
+"In China, we are the North Star in terms of innovation in financial and other related services," says Hang. "We definitely want to make sure we're still leading in the next 5 to 10 years with our investment in technology."
+{{< /case-studies/quote >}}
+
+<p>In fact, the company has already started to open source some of its <a href="https://github.com/alipay">cloud native middleware</a>. "We are going to be very proactive about that," says Yu. "CNCF provided a platform so everyone can plug in or contribute components. This is very good open source governance."</p>
+
+<p>Looking ahead, the Ant team will continue to evaluate many other CNCF projects. Building a service mesh community in China, the team has brought together many China-based companies and developers to discuss the potential of that technology. "Service mesh is very attractive for Chinese developers and end users because we have a lot of legacy systems running now, and it's an ideal mid-layer to glue everything together, both new and legacy," says Hang. "For new technologies, we look very closely at whether they will last."</p>
+
+<p>At Ant, Kubernetes passed that test with flying colors, and the team hopes other companies will follow suit. "In China, we are the North Star in terms of innovation in financial and other related services," says Hang. "We definitely want to make sure we're still leading in the next 5 to 10 years with our investment in technology."</p>
@@ -1,99 +1,85 @@
 ---
 title: AppDirect Case Study
-
 linkTitle: AppDirect
 case_study_styles: true
 cid: caseStudies
-css: /css/style_case_studies.css
 logo: appdirect_featured_logo.png
 featured: true
 weight: 4
 quote: >
   We made the right decisions at the right time. Kubernetes and the cloud native technologies are now seen as the de facto ecosystem.
+
+new_case_study_styles: true
+heading_background: /images/case-studies/appdirect/banner1.jpg
+heading_title_logo: /images/appdirect_logo.png
+subheading: >
+  AppDirect: How AppDirect Supported the 10x Growth of Its Engineering Staff with Kubernetess
+case_study_details:
+  - Company: AppDirect
+  - Location: San Francisco, California
+  - Industry: Software
 ---

-<div class="banner1" style="background-image: url('/images/case-studies/appdirect/banner1.jpg')">
-  <h1> CASE STUDY:<img src="/images/appdirect_logo.png" class="header_logo" style="margin-bottom:-2%"><br> <div class="subhead" style="margin-top:1%;font-size:0.5em">AppDirect: How AppDirect Supported the 10x Growth of Its Engineering Staff with Kubernetess
-</div></h1>
+<h2>Challenge</h2>

-</div>
+<p><a href="https://www.appdirect.com/">AppDirect</a> provides an end-to-end commerce platform for cloud-based products and services. When Director of Software Development Pierre-Alexandre Lacerte began working there in 2014, the company had a monolith application deployed on a "tomcat infrastructure, and the whole release process was complex for what it should be," he says. "There were a lot of manual steps involved, with one engineer building a feature, then another team picking up the change. So you had bottlenecks in the pipeline to ship a feature to production." At the same time, the engineering team was growing, and the company realized it needed a better infrastructure to both support that growth and increase velocity.</p>

-<div class="details">
-    Company &nbsp;<b>AppDirect</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location &nbsp;<b>San Francisco, California
-</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Industry &nbsp;<b>Software</b>
-</div>
+<h2>Solution</h2>
+
+<p>"My idea was: Let's create an environment where teams can deploy their services faster, and they will say, 'Okay, I don't want to build in the monolith anymore. I want to build a service,'" says Lacerte. They considered and prototyped several different technologies before deciding to adopt <a href="https://kubernetes.io/">Kubernetes</a> in early 2016. Lacerte's team has also integrated <a href="https://prometheus.io/">Prometheus</a> monitoring into the platform; tracing is next. Today, AppDirect has more than 50 microservices in production and 15 Kubernetes clusters deployed on <a href="https://aws.amazon.com/">AWS</a> and on premise around the world.</p>

-<hr>
-<section class="section1">
-<div class="cols">
-  <div class="col1" style="width:100%"">
-    <h2>Challenge</h2>
-    <a href="https://www.appdirect.com/">AppDirect</a> provides an end-to-end commerce platform for cloud-based products and services. When Director of Software Development Pierre-Alexandre Lacerte began working there in 2014, the company had a monolith application deployed on a "tomcat infrastructure, and the whole release process was complex for what it should be," he says. "There were a lot of manual steps involved, with one engineer building a feature, then another team picking up the change. So you had bottlenecks in the pipeline to ship a feature to production." At the same time, the engineering team was growing, and the company realized it needed a better infrastructure to both support that growth and increase velocity.
-<br><br>
-    <h2>Solution</h2>
-    "My idea was: Let’s create an environment where teams can deploy their services faster, and they will say, ‘Okay, I don’t want to build in the monolith anymore. I want to build a service,’" says Lacerte. They considered and prototyped several different technologies before deciding to adopt <a href="https://kubernetes.io/">Kubernetes</a> in early 2016. Lacerte’s team has also integrated <a href="https://prometheus.io/">Prometheus</a> monitoring into the platform; tracing is next. Today, AppDirect has more than 50 microservices in production and 15 Kubernetes clusters deployed on <a href="https://aws.amazon.com/">AWS</a> and on premise around the world.
-<br><br>
 <h2>Impact</h2>
-    The Kubernetes platform has helped support the engineering team’s 10x growth over the past few years. Coupled with the fact that they were continually adding new features, Lacerte says, "I think our velocity would have slowed down a lot if we didn’t have this new infrastructure." Moving to Kubernetes and services has meant that deployments have become much faster due to less dependency on custom-made, brittle shell scripts with SCP commands. Time to deploy a new version has shrunk from 4 hours to a few minutes. Additionally, the company invested a lot of effort to make things self-service for developers. "Onboarding a new service doesn’t require <a href="https://www.atlassian.com/software/jira">Jira</a> tickets or meeting with three different teams," says Lacerte. Today, the company sees 1,600 deployments per week, compared to 1-30 before. The company also achieved cost savings by moving its marketplace and billing monoliths to Kubernetes from legacy EC2 hosts as well as by leveraging autoscaling, as traffic is higher during business hours.
-</div>
-</div>
-</section>
-<div class="banner2">
-  <div class="banner2text">
-    "It was an immense engineering culture shift, but the benefits are undeniable in terms of scale and speed."
-<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Alexandre Gervais, Staff Software Developer, AppDirect</span>
-  </div>
-</div>
-<section class="section2">
-<div class="fullcol">
-  <h2>With its end-to-end commerce platform for cloud-based products and services, <a href="https://www.appdirect.com/">AppDirect</a> has been helping organizations such as Comcast and GoDaddy simplify the digital supply chain since 2009. </h2>
-<br>
-  When Director of Software Development Pierre-Alexandre Lacerte started working there in 2014, the company had a monolith application deployed on a "tomcat infrastructure, and the whole release process was complex for what it should be," he says. "There were a lot of manual steps involved, with one engineer building a feature then creating a pull request, and a QA or another engineer validating the feature. Then it gets merged and someone else will take care of the deployment. So we had bottlenecks in the pipeline to ship a feature to production."  <br><br>
-  At the same time, the engineering team of 40 was growing, and the company wanted to add an increasing number of features to its products. As a member of the platform team, Lacerte began hearing from multiple teams that wanted to deploy applications using different frameworks and languages, from <a href="https://nodejs.org/">Node.js</a> to <a href="http://spring.io/projects/spring-boot">Spring Boot Java</a>. He soon realized that in order to both support growth and increase velocity, the company needed a better infrastructure, and a system in which teams are autonomous, can do their own deploys, and be responsible for their services in production.

-</div>
-</section>
-<div class="banner3" style="background-image: url('/images/case-studies/appdirect/banner3.jpg')">
-  <div class="banner3text">
-  "We made the right decisions at the right time. Kubernetes and the cloud native technologies are now seen as the de facto ecosystem. We know where to focus our efforts in order to tackle the new wave of challenges we face as we scale out. The community is so active and vibrant, which is a great complement to our awesome internal team."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Alexandre Gervais, Staff Software Developer, AppDirect
-</span>
+<p>The Kubernetes platform has helped support the engineering team's 10x growth over the past few years. Coupled with the fact that they were continually adding new features, Lacerte says, "I think our velocity would have slowed down a lot if we didn't have this new infrastructure." Moving to Kubernetes and services has meant that deployments have become much faster due to less dependency on custom-made, brittle shell scripts with SCP commands. Time to deploy a new version has shrunk from 4 hours to a few minutes. Additionally, the company invested a lot of effort to make things self-service for developers. "Onboarding a new service doesn't require <a href="https://www.atlassian.com/software/jira">Jira</a> tickets or meeting with three different teams," says Lacerte. Today, the company sees 1,600 deployments per week, compared to 1-30 before. The company also achieved cost savings by moving its marketplace and billing monoliths to Kubernetes from legacy EC2 hosts as well as by leveraging autoscaling, as traffic is higher during business hours.</p>

-  </div>
-</div>
-<section class="section3">
-<div class="fullcol">
-  From the beginning, Lacerte says, "My idea was: Let’s create an environment where teams can deploy their services faster, and they will say, ‘Okay, I don’t want to build in the monolith anymore. I want to build a service.’" (Lacerte left the company in 2019.)<br><br>
-  Working with the operations team, Lacerte’s group got more control and access to the company’s <a href="https://aws.amazon.com/">AWS infrastructure</a>, and started prototyping several orchestration technologies. "Back then, Kubernetes was a little underground, unknown," he says. "But we looked at the community, the number of pull requests, the velocity on GitHub, and we saw it was getting traction. And we found that it was much easier for us to manage than the other technologies."
-  They spun up the first few services on Kubernetes using <a href="https://www.chef.io/">Chef</a> and <a href="https://www.terraform.io/">Terraform</a> provisioning, and as more services were added, more automation was, too. "We have clusters around the world—in Korea, in Australia, in Germany, and in the U.S.," says Lacerte. "Automation is critical for us." They’re now largely using <a href="https://github.com/kubernetes/kops">Kops</a>, and are looking at managed Kubernetes offerings from several cloud providers.<br><br>
-  Today, though the monolith still exists, there are fewer and fewer commits and features. All teams are deploying on the new infrastructure, and services are the norm. AppDirect now has more than 50 microservices in production and 15 Kubernetes clusters deployed on AWS and on premise around the world.<br><br>
-  Lacerte’s strategy ultimately worked because of the very real impact the Kubernetes platform has had to deployment time. Due to less dependency on custom-made, brittle shell scripts with SCP commands, time to deploy a new version has shrunk from 4 hours to a few minutes. Additionally, the company invested a lot of effort to make things self-service for developers. "Onboarding a new service doesn’t require <a href="https://www.atlassian.com/software/jira">Jira</a> tickets or meeting with three different teams," says Lacerte. Today, the company sees 1,600 deployments per week, compared to 1-30 before.
-</div>
-</section>
-<div class="banner4" style="background-image: url('/images/case-studies/appdirect/banner4.jpg');width:100%;">
-  <div class="banner4text">
-  "I think our velocity would have slowed down a lot if we didn’t have this new infrastructure."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Pierre-Alexandre Lacerte, Director of Software Development, AppDirect</span>
-  </div>
-</div>
+{{< case-studies/quote author="Alexandre Gervais, Staff Software Developer, AppDirect" >}}
+"It was an immense engineering culture shift, but the benefits are undeniable in terms of scale and speed."
+{{< /case-studies/quote >}}

-<section class="section5" style="padding:0px !important">
+{{< case-studies/lead >}}
+With its end-to-end commerce platform for cloud-based products and services, <a href="https://www.appdirect.com/">AppDirect</a> has been helping organizations such as Comcast and GoDaddy simplify the digital supply chain since 2009.
+{{< /case-studies/lead >}}

-<div class="fullcol">
-  Additionally, the Kubernetes platform has helped support the engineering team’s 10x growth over the past few years. "Ownership, a core value of AppDirect, reflects in our ability to ship services independently of our monolith code base," says Staff Software Developer Alexandre Gervais, who worked with Lacerte on the initiative. "Small teams now own critical parts of our business domain model, and they operate in their decoupled domain of expertise, with limited knowledge of the entire codebase. This reduces and isolates some of the complexity." Coupled with the fact that they were continually adding new features, Lacerte says, "I think our velocity would have slowed down a lot if we didn’t have this new infrastructure."
-  The company also achieved cost savings by moving its marketplace and billing monoliths to Kubernetes from legacy EC2 hosts as well as by leveraging autoscaling, as traffic is higher during business hours.<br><br>
-  AppDirect’s cloud native stack also includes <a href="https://grpc.io/">gRPC</a> and <a href="https://www.fluentd.org/">Fluentd</a>, and the team is currently working on setting up <a href="https://opencensus.io/">OpenCensus</a>. The platform already has <a href="https://prometheus.io/">Prometheus</a> integrated, so "when teams deploy their service, they have their notifications, alerts and configurations," says Lacerte. "For example, in the test environment, I want to get a message on Slack, and in production, I want a <a href="https://slack.com/">Slack</a> message and I also want to get paged. We have integration with pager duty. Teams have more ownership on their services."
+<p>When Director of Software Development Pierre-Alexandre Lacerte started working there in 2014, the company had a monolith application deployed on a "tomcat infrastructure, and the whole release process was complex for what it should be," he says. "There were a lot of manual steps involved, with one engineer building a feature then creating a pull request, and a QA or another engineer validating the feature. Then it gets merged and someone else will take care of the deployment. So we had bottlenecks in the pipeline to ship a feature to production."</p>

-</div>
+<p>At the same time, the engineering team of 40 was growing, and the company wanted to add an increasing number of features to its products. As a member of the platform team, Lacerte began hearing from multiple teams that wanted to deploy applications using different frameworks and languages, from <a href="https://nodejs.org/">Node.js</a> to <a href="http://spring.io/projects/spring-boot">Spring Boot Java</a>. He soon realized that in order to both support growth and increase velocity, the company needed a better infrastructure, and a system in which teams are autonomous, can do their own deploys, and be responsible for their services in production.</p>

-<div class="banner5" >
-  <div class="banner5text">
-"We moved from a culture limited to ‘pushing code in a branch’ to exciting new responsibilities outside of the code base: deployment of features and configurations; monitoring of application and business metrics; and on-call support in case of outages. It was an immense engineering culture shift, but the benefits are undeniable in terms of scale and speed."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Pierre-Alexandre Lacerte, Director of Software Development, AppDirect</span></div>
-</div>
+{{< case-studies/quote 
+  image="/images/case-studies/appdirect/banner3.jpg"
+  author="Alexandre Gervais, Staff Software Developer, AppDirect"
+>}}
+"We made the right decisions at the right time. Kubernetes and the cloud native technologies are now seen as the de facto ecosystem. We know where to focus our efforts in order to tackle the new wave of challenges we face as we scale out. The community is so active and vibrant, which is a great complement to our awesome internal team."
+{{< /case-studies/quote >}}

-<div class="fullcol">
-  That of course also means more responsibility. "We asked engineers to expand their horizons," says Gervais. "We moved from a culture limited to ‘pushing code in a branch’ to exciting new responsibilities outside of the code base: deployment of features and configurations; monitoring of application and business metrics; and on-call support in case of outages. It was an immense engineering culture shift, but the benefits are undeniable in terms of scale and speed." <br><br>
-  As the engineering ranks continue to grow, the platform team has a new challenge, of making sure that the Kubernetes platform is accessible and easily utilized by everyone. "How can we make sure that when we add more people to our team that they are efficient, productive, and know how to ramp up on the platform?" Lacerte says. So we have the evangelists, the documentation, some project examples. We do demos, we have AMA sessions. We’re trying different strategies to get everyone’s attention."<br><br>
-  Three and a half years into their Kubernetes journey, Gervais feels AppDirect "made the right decisions at the right time," he says. "Kubernetes and the cloud native technologies are now seen as the de facto ecosystem. We know where to focus our efforts in order to tackle the new wave of challenges we face as we scale out. The community is so active and vibrant, which is a great complement to our awesome internal team. Going forward, our focus will really be geared towards benefiting from the ecosystem by providing added business value in our day-to-day operations."
+<p>From the beginning, Lacerte says, "My idea was: Let's create an environment where teams can deploy their services faster, and they will say, 'Okay, I don't want to build in the monolith anymore. I want to build a service.'" (Lacerte left the company in 2019.)</p>

+<p>Working with the operations team, Lacerte's group got more control and access to the company's <a href="https://aws.amazon.com/">AWS infrastructure</a>, and started prototyping several orchestration technologies. "Back then, Kubernetes was a little underground, unknown," he says. "But we looked at the community, the number of pull requests, the velocity on GitHub, and we saw it was getting traction. And we found that it was much easier for us to manage than the other technologies."</p>

-</div>
-</section>
+<p>They spun up the first few services on Kubernetes using <a href="https://www.chef.io/">Chef</a> and <a href="https://www.terraform.io/">Terraform</a> provisioning, and as more services were added, more automation was, too. "We have clusters around the world—in Korea, in Australia, in Germany, and in the U.S.," says Lacerte. "Automation is critical for us." They're now largely using <a href="https://github.com/kubernetes/kops">Kops</a>, and are looking at managed Kubernetes offerings from several cloud providers.</p>
+
+<p>Today, though the monolith still exists, there are fewer and fewer commits and features. All teams are deploying on the new infrastructure, and services are the norm. AppDirect now has more than 50 microservices in production and 15 Kubernetes clusters deployed on AWS and on premise around the world.</p>
+
+<p>Lacerte's strategy ultimately worked because of the very real impact the Kubernetes platform has had to deployment time. Due to less dependency on custom-made, brittle shell scripts with SCP commands, time to deploy a new version has shrunk from 4 hours to a few minutes. Additionally, the company invested a lot of effort to make things self-service for developers. "Onboarding a new service doesn't require <a href="https://www.atlassian.com/software/jira">Jira</a> tickets or meeting with three different teams," says Lacerte. Today, the company sees 1,600 deployments per week, compared to 1-30 before.</p>
+
+{{< case-studies/quote 
+  image="/images/case-studies/appdirect/banner4.jpg"
+  author="Pierre-Alexandre Lacerte, Director of Software Development, AppDirect"
+>}}
+"I think our velocity would have slowed down a lot if we didn't have this new infrastructure."
+{{< /case-studies/quote >}}
+
+<p>Additionally, the Kubernetes platform has helped support the engineering team's 10x growth over the past few years. "Ownership, a core value of AppDirect, reflects in our ability to ship services independently of our monolith code base," says Staff Software Developer Alexandre Gervais, who worked with Lacerte on the initiative. "Small teams now own critical parts of our business domain model, and they operate in their decoupled domain of expertise, with limited knowledge of the entire codebase. This reduces and isolates some of the complexity." Coupled with the fact that they were continually adding new features, Lacerte says, "I think our velocity would have slowed down a lot if we didn't have this new infrastructure."</p>
+
+<p>The company also achieved cost savings by moving its marketplace and billing monoliths to Kubernetes from legacy EC2 hosts as well as by leveraging autoscaling, as traffic is higher during business hours.</p>
+
+<p>AppDirect's cloud native stack also includes <a href="https://grpc.io/">gRPC</a> and <a href="https://www.fluentd.org/">Fluentd</a>, and the team is currently working on setting up <a href="https://opencensus.io/">OpenCensus</a>. The platform already has <a href="https://prometheus.io/">Prometheus</a> integrated, so "when teams deploy their service, they have their notifications, alerts and configurations," says Lacerte. "For example, in the test environment, I want to get a message on Slack, and in production, I want a <a href="https://slack.com/">Slack</a> message and I also want to get paged. We have integration with pager duty. Teams have more ownership on their services."</p>
+
+{{< case-studies/quote author="Pierre-Alexandre Lacerte, Director of Software Development, AppDirect" >}}
+"We moved from a culture limited to 'pushing code in a branch' to exciting new responsibilities outside of the code base: deployment of features and configurations; monitoring of application and business metrics; and on-call support in case of outages. It was an immense engineering culture shift, but the benefits are undeniable in terms of scale and speed."
+{{< /case-studies/quote >}}
+
+<p>That of course also means more responsibility. "We asked engineers to expand their horizons," says Gervais. "We moved from a culture limited to 'pushing code in a branch' to exciting new responsibilities outside of the code base: deployment of features and configurations; monitoring of application and business metrics; and on-call support in case of outages. It was an immense engineering culture shift, but the benefits are undeniable in terms of scale and speed."</p>
+
+<p>As the engineering ranks continue to grow, the platform team has a new challenge, of making sure that the Kubernetes platform is accessible and easily utilized by everyone. "How can we make sure that when we add more people to our team that they are efficient, productive, and know how to ramp up on the platform?" Lacerte says. So we have the evangelists, the documentation, some project examples. We do demos, we have AMA sessions. We're trying different strategies to get everyone's attention."</p>
+
+<p>Three and a half years into their Kubernetes journey, Gervais feels AppDirect "made the right decisions at the right time," he says. "Kubernetes and the cloud native technologies are now seen as the de facto ecosystem. We know where to focus our efforts in order to tackle the new wave of challenges we face as we scale out. The community is so active and vibrant, which is a great complement to our awesome internal team. Going forward, our focus will really be geared towards benefiting from the ecosystem by providing added business value in our day-to-day operations."</p>
@@ -3,107 +3,82 @@ title: Babylon Case Study
 linkTitle: Babylon
 case_study_styles: true
 cid: caseStudies
-css: /css/case-studies-gradient.css
 logo: babylon_featured_logo.svg
 featured: true
 weight: 1
 quote: >
   Kubernetes is a great platform for machine learning because it comes with all the scheduling and scalability that you need.
+
+new_case_study_styles: true
+heading_background: /images/case-studies/babylon/banner4.jpg
+heading_title_text: Babylon
+use_gradient_overlay: true
+subheading: >
+  AppDirect: How Cloud Native Is Enabling Babylon's Medical AI Innovations
+case_study_details:
+  - Company: Babylon
+  - Location: United Kingdom
+  - Industry: AI, Healthcare
 ---

+<h2>Challenge</h2>

-<div class="article image overlay" style="background-image: url('/images/case-studies/babylon/banner4.jpg')">
-  <h1> CASE STUDY: Babylon</h1>
-  <div class="subhead">How Cloud Native Is Enabling Babylon’s Medical AI Innovations</div>
-</div>
+<p>A large number of Babylon's products leverage machine learning and artificial intelligence, and in 2019, there wasn't enough computing power in-house to run a particular experiment. The company was also growing (from 100 to 1,600 in three years) and planning expansion into other countries.</p>

-<div class="details">
-    Company &nbsp;<b>Babylon</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location &nbsp;<b>United Kingdom</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Industry &nbsp;<b>AI, Healthcare</b>
-</div>
+<h2>Solution</h2>

-<hr>
-<section class="section1">
-<div class="colsgradient">
-  <div class="col1">
-    <h2>Challenge</h2>
-   A large number of Babylon’s products leverage machine learning and artificial intelligence, and in 2019, there wasn’t enough computing power in-house to run a particular experiment. The company was also growing (from 100 to 1,600 in three years) and planning expansion into other countries.
-
-
-    <h2>Solution</h2>
-    Babylon had migrated its user-facing applications to a Kubernetes platform in 2018, so the infrastructure team turned to Kubeflow, a toolkit for machine learning on Kubernetes. “We tried to create a Kubernetes core server, we deployed Kubeflow, and we orchestrated the whole experiment, which ended up being a really good success,” says AI Infrastructure Lead Jérémie Vallée. The team began building a self-service AI training platform on top of Kubernetes.
+<p>Babylon had migrated its user-facing applications to a Kubernetes platform in 2018, so the infrastructure team turned to Kubeflow, a toolkit for machine learning on Kubernetes. "We tried to create a Kubernetes core server, we deployed Kubeflow, and we orchestrated the whole experiment, which ended up being a really good success," says AI Infrastructure Lead Jérémie Vallée. The team began building a self-service AI training platform on top of Kubernetes.</p>

 <h2>Impact</h2>
-    Instead of waiting hours or days to be able to compute, teams can get access instantaneously. Clinical validations used to take 10 hours; now they are done in under 20 minutes. The portability of the cloud native platform has also enabled Babylon to expand into other countries.</div>
-</div>
-</section>
-<div class="article image overlay" style="background-image: url('/images/case-studies/babylon/banner1.jpg');">
-  <div class="quotetext">
-    “Kubernetes is a great platform for machine learning because it comes with all the scheduling and scalability that you need.”
-<p><div class="quoteauthortext">- JÉRÉMIE VALLÉE, AI INFRASTRUCTURE LEAD AT BABYLON</div></p>
-  </div>
-</div>

+<p>Instead of waiting hours or days to be able to compute, teams can get access instantaneously. Clinical validations used to take 10 hours; now they are done in under 20 minutes. The portability of the cloud native platform has also enabled Babylon to expand into other countries.</p>

-<section class="section2">
-<div class="fullcol">
-  <h4>Babylon’s mission is to put accessible and affordable healthcare services in the hands of every person on earth.</h4>
+{{< case-studies/quote
+  image="/images/case-studies/babylon/banner1.jpg"
+  author="JÉRÉMIE VALLÉE, AI INFRASTRUCTURE LEAD AT BABYLON"
+>}}
+"Kubernetes is a great platform for machine learning because it comes with all the scheduling and scalability that you need."
+{{< /case-studies/quote >}}

-  <p>Since its launch in the U.K. in 2013, the startup has facilitated millions of digital consultations around the world. In the U.K., patients were typically waiting a week or two for a doctor’s appointment. Through Babylon’s NHS service, GP at Hand—which has more than 75,000 registered patients—39% get an appointment through their phone within 30 minutes, and 89% within 6 hours.  </p>
+{{< case-studies/lead >}}
+Babylon's mission is to put accessible and affordable healthcare services in the hands of every person on earth.
+{{< /case-studies/lead >}}

-  <p>That’s just the start. “We try to combine different types of technology with the medical expertise that we have in-house to build products that will help patients manage and understand their health, and also help doctors be more efficient at what they do,” says Jérémie Vallée, AI Infrastructure Lead at Babylon. </p>
+<p>Since its launch in the U.K. in 2013, the startup has facilitated millions of digital consultations around the world. In the U.K., patients were typically waiting a week or two for a doctor's appointment. Through Babylon's NHS service, GP at Hand—which has more than 75,000 registered patients—39% get an appointment through their phone within 30 minutes, and 89% within 6 hours.</p>

-<p>A large number of these products leverage machine learning and artificial intelligence, and in 2019, researchers hit a pain point. “We have some servers in-house where our researchers were doing a lot of AI experiments and some training of models, and we came to a point where we didn’t have enough compute in-house to run a particular experiment,” says Vallée. </p>
- <p>
-    Babylon had migrated its user-facing applications to a Kubernetes platform in 2018, “and we had a lot of Kubernetes knowledge thanks to the migration,” he adds. To optimize some of the models that had been created, the team turned to Kubeflow, a toolkit for machine learning on Kubernetes. “We tried to create a Kubernetes core server, we deployed Kubeflow, and we orchestrated the whole experiment, which ended up being a really good success,” he says.</p>
-  <p>
-   Based on that experience, Vallée’s team was tasked with building a self-service platform to help Babylon’s AI teams become more efficient, and by extension help get products to market faster. The main requirements: (1) the ability to give researchers and engineers access to the compute they needed, regardless of the size of the experiments they may need to run; (2) a way to provide teams with the best tools that they needed to do their work, on demand and in a centralized way; and (3) the training platform had to be close to the data that was being managed, because of the company’s expansion into different countries.</p>
+<p>That's just the start. "We try to combine different types of technology with the medical expertise that we have in-house to build products that will help patients manage and understand their health, and also help doctors be more efficient at what they do," says Jérémie Vallée, AI Infrastructure Lead at Babylon. </p>

+<p>A large number of these products leverage machine learning and artificial intelligence, and in 2019, researchers hit a pain point. "We have some servers in-house where our researchers were doing a lot of AI experiments and some training of models, and we came to a point where we didn't have enough compute in-house to run a particular experiment," says Vallée. </p>

-</div>
-</section>
+<p>Babylon had migrated its user-facing applications to a Kubernetes platform in 2018, "and we had a lot of Kubernetes knowledge thanks to the migration," he adds. To optimize some of the models that had been created, the team turned to Kubeflow, a toolkit for machine learning on Kubernetes. "We tried to create a Kubernetes core server, we deployed Kubeflow, and we orchestrated the whole experiment, which ended up being a really good success," he says.</p>

-<div class="greybanner">
-  <div class="greyquotetext">
-     “Delivering a self-service platform where users are empowered to run their own workload has enabled our data scientist community to do hyper parameter tuning and general algorithm development without any cloud skill and without the help of platform engineers, thus accelerating our innovation.”<p><div class="quoteauthortext">- CAROLINE HARGROVE, CHIEF TECHNOLOGY OFFICER AT BABYLON</div></p>
-  </div>
-</div>
+<p>Based on that experience, Vallée's team was tasked with building a self-service platform to help Babylon's AI teams become more efficient, and by extension help get products to market faster. The main requirements: (1) the ability to give researchers and engineers access to the compute they needed, regardless of the size of the experiments they may need to run; (2) a way to provide teams with the best tools that they needed to do their work, on demand and in a centralized way; and (3) the training platform had to be close to the data that was being managed, because of the company's expansion into different countries.</p>

-<section class="section3">
-<div class="fullcol">
-   <p>
-    Kubernetes was an enabler on every count. “Kubernetes is a great platform for machine learning because it comes with all the scheduling and scalability that you need,” says Vallée. The need to keep data in every country in which Babylon operates requires a multi-region, multi-cloud strategy, and some countries might not even have a public cloud provider at all. “We wanted to make this platform portable so that we can run training jobs anywhere,” he says. “Kubernetes offered a base layer that allows you to deploy the platform outside of the cloud provider, and then deploy whatever tooling you need. That was a very good selling point for us.” </p>
- <p>
-   Once the team decided to build the Babylon AI Research platform on top of Kubernetes, they referred to the Cloud Native Landscape to build out the stack: Prometheus and Grafana for monitoring; an Istio service mesh to control the network on the training platform and control what access all of the workflows would have; Helm to deploy the stack; and Flux to manage the GitOps part of the pipeline. </p>
-  <p>
-  The cloud native AI platform has had a huge impact at Babylon. The first research projects run on the platform mostly involved machine learning and natural language processing. These experiments required a huge amount of compute—1600 CPU, 3.2 TB RAM—which was much more than Babylon had in-house. Plus, access to compute used to take hours, or sometimes even days, depending on how busy the platform team was. “Now, with Kubernetes and the self-service platform that we provide, it’s pretty much instantaneous,” says Vallée. </p>
-  <p>
-  Another important type of work that’s done on the platform is clinical validation for new applications such as Babylon’s Symptom Checker, which calculates the probability of a disease given the evidence input by the user. “Being in healthcare, we want all of our models to be safe before they’re going to hit production,” says Vallée. Using Argo for GitOps “enabled us to scale the process massively.”  </p>
+{{< case-studies/quote author="CAROLINE HARGROVE, CHIEF TECHNOLOGY OFFICER AT BABYLON" >}}
+"Delivering a self-service platform where users are empowered to run their own workload has enabled our data scientist community to do hyper parameter tuning and general algorithm development without any cloud skill and without the help of platform engineers, thus accelerating our innovation."
+{{< /case-studies/quote >}}

-  </p>
-</div>
-</section>
+<p>Kubernetes was an enabler on every count. "Kubernetes is a great platform for machine learning because it comes with all the scheduling and scalability that you need," says Vallée. The need to keep data in every country in which Babylon operates requires a multi-region, multi-cloud strategy, and some countries might not even have a public cloud provider at all. "We wanted to make this platform portable so that we can run training jobs anywhere," he says. "Kubernetes offered a base layer that allows you to deploy the platform outside of the cloud provider, and then deploy whatever tooling you need. That was a very good selling point for us."</p>

+<p>Once the team decided to build the Babylon AI Research platform on top of Kubernetes, they referred to the Cloud Native Landscape to build out the stack: Prometheus and Grafana for monitoring; an Istio service mesh to control the network on the training platform and control what access all of the workflows would have; Helm to deploy the stack; and Flux to manage the GitOps part of the pipeline.</p>

-  <div class="article image overlay" style="background-image: url('/images/case-studies/babylon/banner2.jpg');">
-  <div class="quotetext">
- “Giving a Kubernetes-based platform to our data scientists has meant increased security, increased innovation through empowerment, and a more affordable health service as our cloud engineers are building an experience that is used by hundreds on a daily basis, rather than supporting specific bespoke use cases.” <p><div class="quoteauthortext">- JEAN MARIE FERDEGUE, DIRECTOR OF PLATFORM OPERATIONS AT BABYLON</div></p>
-  </div>
-</div>
+<p>The cloud native AI platform has had a huge impact at Babylon. The first research projects run on the platform mostly involved machine learning and natural language processing. These experiments required a huge amount of compute—1600 CPU, 3.2 TB RAM—which was much more than Babylon had in-house. Plus, access to compute used to take hours, or sometimes even days, depending on how busy the platform team was. "Now, with Kubernetes and the self-service platform that we provide, it's pretty much instantaneous," says Vallée.</p>

-<section class="section5">
-<div class="fullcol">
-   <p>
-  Researchers used to have to wait up to 10 hours to get results on new versions of their models. With Kubernetes, that time is now down to under 20 minutes. Plus, previously they could only run one clinical validation at a time, now they can run many parallel ones if they need to—a huge benefit considering that in the past three years, Babylon has grown from 100 to 1,600 employees. </p>
-  <p>
-  “Delivering a self-service platform where users are empowered to run their own workload has enabled our data scientist community to do hyper parameter tuning and general algorithm development without any cloud skill and without the help of platform engineers, thus accelerating our innovation,” says Chief Technology Officer Caroline Hargrove.</p>
-  <p>
- Adds Director of Platform Operations Jean Marie Ferdegue: “Giving a Kubernetes-based platform to our data scientists has meant increased security, increased innovation through empowerment, and a more affordable health service as our cloud engineers are building an experience that is used by hundreds on a daily basis, rather than supporting specific bespoke use cases.” </p>
-  <p>
- Plus, as Babylon continues to expand, “it will be very easy to onboard new countries,” says Vallée. “Fifteen months ago when we deployed this platform, we had one big environment in the U.K., but now we have one in Canada, we have one in Asia, and we have one coming in the U.S. This is one of the things that Kubernetes and the other cloud native projects have enabled for us.”
-  </p>
-  <p>
-  Babylon’s road map for cloud native involves onboarding all of the company’s AI efforts to the platform. Increasingly, that includes AI services of care. “I think this is going to be an interesting field where AI and healthcare meet,” Vallée says. “It’s kind of a complex problem and there’s a lot of issues around this. So with our platform, we want to say, ‘What can we do to make this less painful for our developers and machine learning engineers?’”
-</p>
-</div>
-</section>
-</body>
+<p>Another important type of work that's done on the platform is clinical validation for new applications such as Babylon's Symptom Checker, which calculates the probability of a disease given the evidence input by the user. "Being in healthcare, we want all of our models to be safe before they're going to hit production," says Vallée. Using Argo for GitOps "enabled us to scale the process massively."</p>
+
+{{< case-studies/quote 
+  image="/images/case-studies/babylon/banner2.jpg"
+  author="JEAN MARIE FERDEGUE, DIRECTOR OF PLATFORM OPERATIONS AT BABYLON"
+>}}
+"Giving a Kubernetes-based platform to our data scientists has meant increased security, increased innovation through empowerment, and a more affordable health service as our cloud engineers are building an experience that is used by hundreds on a daily basis, rather than supporting specific bespoke use cases."
+{{< /case-studies/quote >}}
+
+<p>Researchers used to have to wait up to 10 hours to get results on new versions of their models. With Kubernetes, that time is now down to under 20 minutes. Plus, previously they could only run one clinical validation at a time, now they can run many parallel ones if they need to—a huge benefit considering that in the past three years, Babylon has grown from 100 to 1,600 employees.</p>
+
+<p>"Delivering a self-service platform where users are empowered to run their own workload has enabled our data scientist community to do hyper parameter tuning and general algorithm development without any cloud skill and without the help of platform engineers, thus accelerating our innovation," says Chief Technology Officer Caroline Hargrove.</p>
+
+<p>Adds Director of Platform Operations Jean Marie Ferdegue: "Giving a Kubernetes-based platform to our data scientists has meant increased security, increased innovation through empowerment, and a more affordable health service as our cloud engineers are building an experience that is used by hundreds on a daily basis, rather than supporting specific bespoke use cases."</p>
+
+<p>Plus, as Babylon continues to expand, "it will be very easy to onboard new countries," says Vallée. "Fifteen months ago when we deployed this platform, we had one big environment in the U.K., but now we have one in Canada, we have one in Asia, and we have one coming in the U.S. This is one of the things that Kubernetes and the other cloud native projects have enabled for us."</p>
+
+<p>Babylon's road map for cloud native involves onboarding all of the company's AI efforts to the platform. Increasingly, that includes AI services of care. "I think this is going to be an interesting field where AI and healthcare meet," Vallée says. "It's kind of a complex problem and there's a lot of issues around this. So with our platform, we want to say, 'What can we do to make this less painful for our developers and machine learning engineers?'"</p>
@@ -1,98 +1,85 @@
 ---
 title: BlaBlaCar Case Study
-
 case_study_styles: true
 cid: caseStudies
-css: /css/style_blablacar.css
+
+new_case_study_styles: true
+heading_background: /images/case-studies/blablacar/banner1.jpg
+heading_title_logo: /images/blablacar_logo.png
+subheading: >
+  Turning to Containerization to Support Millions of Rideshares
+case_study_details:
+  - Company: BlaBlaCar
+  - Location: Paris, France
+  - Industry: Ridesharing Company
 ---

-<div class="banner1">
-  <h1> CASE STUDY:<img src="/images/blablacar_logo.png" class="header_logo"><br /> <div class="subhead">Turning to Containerization to Support Millions of Rideshares</div></h1>
+<h2>Challenge</h2>

-</div>
+<p>The world's largest long-distance carpooling community, <a href="https://www.blablacar.com/">BlaBlaCar</a>, connects 40 million members across 22 countries. The company has been experiencing exponential growth since 2012 and needed its infrastructure to keep up. "When you're thinking about doubling the number of servers, you start thinking, 'What should I do to be more efficient?'" says Simon Lallemand, Infrastructure Engineer at BlaBlaCar. "The answer is not to hire more and more people just to deal with the servers and installation." The team knew they had to scale the platform, but wanted to stay on their own bare metal servers.</p>

-<div class="details">
-    Company &nbsp;<b>BlaBlaCar</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location &nbsp;<b>Paris, France</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Industry &nbsp;<b>Ridesharing Company</b>
-</div>
+<h2>Solution</h2>

-<hr />
-<section class="section1">
-<div class="cols">
-  <div class="col1">
-    <h2>Challenge</h2>
-        The world’s largest long-distance carpooling community, <a href="https://www.blablacar.com/">BlaBlaCar</a>, connects 40 million members across 22 countries. The company has been experiencing exponential growth since 2012 and needed its infrastructure to keep up. "When you’re thinking about doubling the number of servers, you start thinking, ‘What should I do to be more efficient?’" says Simon Lallemand, Infrastructure Engineer at BlaBlaCar. "The answer is not to hire more and more people just to deal with the servers and installation." The team knew they had to scale the platform, but wanted to stay on their own bare metal servers.
-    <br />
-    <br />
-      <h2>Solution</h2>
-        Opting not to shift to cloud virtualization or use a private cloud on their own servers, the BlaBlaCar team became early adopters of containerization, using the CoreOs runtime <a href="https://coreos.com/rkt">rkt</a>, initially deployed using <a href="https://coreos.com/fleet/docs/latest/launching-containers-fleet.html">fleet</a> cluster manager. Last year, the company switched to <a href="http://kubernetes.io/">Kubernetes</a> orchestration, and now also uses <a href="https://prometheus.io/">Prometheus</a> for monitoring.
-      </div>
+<p>Opting not to shift to cloud virtualization or use a private cloud on their own servers, the BlaBlaCar team became early adopters of containerization, using the CoreOs runtime <a href="https://coreos.com/rkt">rkt</a>, initially deployed using <a href="https://coreos.com/fleet/docs/latest/launching-containers-fleet.html">fleet</a> cluster manager. Last year, the company switched to <a href="http://kubernetes.io/">Kubernetes</a> orchestration, and now also uses <a href="https://prometheus.io/">Prometheus</a> for monitoring.</p>

-<div class="col2">
 <h2>Impact</h2>
-        "Before using containers, it would take sometimes a day, sometimes two, just to create a new service," says Lallemand. "With all the tooling that we made around the containers, copying a new service now is a matter of minutes. It’s really a huge gain. We are better at capacity planning in our data center because we have fewer constraints due to this abstraction between the services and the hardware we run on. For the developers, it also means they can focus only on the features that they’re developing, and not on the infrastructure."
-</div>
-</div>
-</section>
-<div class="banner2">
-  <div class="banner2text">
-    "When you’re switching to this cloud-native model and running everything in containers, you have to make sure that at any moment you can reboot without any downtime and without losing traffic. [With Kubernetes] our infrastructure is much more resilient and we have better availability than before."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br />- Simon Lallemand, Infrastructure Engineer at BlaBlaCar</span>
-  </div>
-</div>

-<section class="section2">
-<div class="fullcol">
-  <h2>For the 40 million users of <a href="https://www.blablacar.com/">BlaBlaCar</a>, it’s easy to find strangers headed in the same direction to share rides and costs. You can even choose how much "bla bla" chatter you want from a long-distance ride mate.</h2>
-  Behind the scenes, though, the infrastructure was falling woefully behind the rider community’s exponential growth. Founded in 2006, the company hit its current stride around 2012. "Our infrastructure was very traditional," says Infrastructure Engineer Simon Lallemand, who began working at the company in 2014. "In the beginning, it was a bit chaotic because we had to [grow] fast. But then comes the time when you have to design things to make it manageable."<br /><br />
-  By 2015, the company had about 50 bare metal servers. The team was using a <a href="https://www.mysql.com/">MySQL</a> database and <a href="http://php.net/">PHP</a>, but, Lallemand says, "it was a very static way." They also utilized the configuration management system, <a href="https://www.chef.io/chef/">Chef</a>, but had little automation in its process. "When you’re thinking about doubling the number of servers, you start thinking, ‘What should I do to be more efficient?’" says Lallemand. "The answer is not to hire more and more people just to deal with the servers and installation."<br /><br />
-  Instead, BlaBlaCar began its cloud-native journey but wasn’t sure which route to take. "We could either decide to go into cloud virtualization or even use a private cloud on our own servers," says Lallemand. "But going into the cloud meant we had to make a lot of changes in our application work, and we were just not ready to make the switch from on premise to the cloud." They wanted to keep the great performance they got on bare metal, so they didn’t want to go to virtualization on premise.<br /><br />
-  The solution: containerization. This was early 2015 and containers were still relatively new. "It was a bold move at the time," says Lallemand. "We decided that the next servers that we would buy in the new data center would all be the same model, so we could outsource the maintenance of the servers. And we decided to go with containers and with <a href="https://coreos.com/">CoreOS</a> Container Linux as an abstraction for this hardware. It seemed future-proof to go with containers because we could see what companies were already doing with containers."
-</div>
-</section>
+<p>"Before using containers, it would take sometimes a day, sometimes two, just to create a new service," says Lallemand. "With all the tooling that we made around the containers, copying a new service now is a matter of minutes. It's really a huge gain. We are better at capacity planning in our data center because we have fewer constraints due to this abstraction between the services and the hardware we run on. For the developers, it also means they can focus only on the features that they're developing, and not on the infrastructure."</p>

-<div class="banner3">
-  <div class="banner3text">
-    "With all the tooling that we made around the containers, copying a new service is a matter of minutes. It’s a huge gain. For the developers, it means they can focus only on the features that they’re developing and not on the infrastructure or the hour they would test their code, or the hour that it would get deployed."
-  </div>
-</div>
+{{< case-studies/quote author="Simon Lallemand, Infrastructure Engineer at BlaBlaCar" >}}
+"When you're switching to this cloud-native model and running everything in containers, you have to make sure that at any moment you can reboot without any downtime and without losing traffic. [With Kubernetes] our infrastructure is much more resilient and we have better availability than before."
+{{< /case-studies/quote >}}

-<section class="section3">
-<div class="fullcol">
-    Next, they needed to choose a runtime for the containers, but "there were very few deployments in production at that time," says Lallemand. They experimented with <a href="https://www.docker.com/">Docker</a> but decided to go with <a href="https://coreos.com/rkt">rkt</a>. Lallemand explains that for BlaBlaCar, it was "much simpler to integrate things that are on rkt." At the time,  the project was still pre-v1.0, so "we could speak with the developers of rkt and give them feedback. It was an advantage." Plus, he notes, rkt was very stable, even at this early stage.<br /><br />
-    Once those decisions were made that summer, the company came up with a plan for implementation. First, they formed a task force to create a workflow that would be tested by three of the 10 members on Lallemand’s team. But they took care to run regular workshops with all 10 members to make sure everyone was on board. "When you’re focused on your product sometimes you forget if it’s really user friendly, whether other people can manage to create containers too," Lallemand says. "So we did a lot of iterations to find a good workflow."<br /><br />
-    After establishing the workflow, Lallemand says with a smile that "we had this strange idea that we should try the most difficult thing first. Because if it works, it will work for everything." So the first project the team decided to containerize was the database. "Nobody did that at the time, and there were really no existing tools for what we wanted to do, including building container images," he says. So the team created their own tools, such as <a href="https://github.com/blablacar/dgr">dgr</a>, which builds container images so that the whole team has a common framework to build on the same images with the same standards. They also revamped the service-discovery tools <a href="https://github.com/airbnb/nerve">Nerve</a> and <a href="http://airbnb.io/projects/synapse/">Synapse</a>; their versions, <a href="https://github.com/blablacar/go-nerve">Go-Nerve</a> and <a href="https://github.com/blablacar/go-synapse">Go-Synapse</a>, were written in Go and built to be more efficient and include new features. All of these tools were open-sourced.<br /><br />
-    At the same time, the company was working to migrate its entire platform to containers with a deadline set for Christmas 2015. With all the work being done in parallel, BlaBlaCar was able to get about 80 percent of its production into containers by its deadline with live traffic running on containers during December. (It’s now at 100 percent.) "It’s a really busy time for traffic," says Lallemand. "We knew that by using those new servers with containers, it would help us handle the traffic."<br /><br />
-    In the middle of that peak season for carpooling, everything worked well. "The biggest impact that we had was for the deployment of new services," says Lallemand. "Before using containers, we had to first deploy a new server and create configurations with Chef. It would take sometimes a day, sometimes two, just to create a new service. And with all the tooling that we made around the containers, copying a new service is a matter of minutes. So it’s really a huge gain. For the developers, it means they can focus only on the features that they’re developing and not on the infrastructure or the hour they would test their code, or the hour that it would get deployed."
-</div>
-</section>
+{{< case-studies/lead >}}
+For the 40 million users of <a href="https://www.blablacar.com/">BlaBlaCar</a>, it's easy to find strangers headed in the same direction to share rides and costs. You can even choose how much "bla bla" chatter you want from a long-distance ride mate.
+{{< /case-studies/lead >}}

-<div class="banner4">
-  <div class="banner4text">
-    "We realized that there was a really strong community around it [Kubernetes], which meant we would not have to maintain a lot of tools of our own," says Lallemand. "It was better if we could contribute to some bigger project like Kubernetes."
-  </div>
-</div>
+<p>Behind the scenes, though, the infrastructure was falling woefully behind the rider community's exponential growth. Founded in 2006, the company hit its current stride around 2012. "Our infrastructure was very traditional," says Infrastructure Engineer Simon Lallemand, who began working at the company in 2014. "In the beginning, it was a bit chaotic because we had to [grow] fast. But then comes the time when you have to design things to make it manageable."</p>

-<section class="section4">
-<div class="fullcol">
-    In order to meet their self-imposed deadline, one of the decisions they made was to not do any "orchestration magic" for containers in the first production alignment. Instead, they used the basic <a href="https://coreos.com/fleet/docs/latest/launching-containers-fleet.html">fleet</a> tool from CoreOS to deploy their containers. (They did build a tool called <a href="https://github.com/blablacar/ggn">GGN</a>, which they’ve open-sourced, to make it more manageable for their system engineers to use.)<br /><br />
-    Still, the team knew that they’d want more orchestration. "Our tool was doing a pretty good job, but at some point you want to give more autonomy to the developer team," Lallemand says. "We also realized that we don’t want to be the single point of contact for developers when they want to launch new services." By the summer of 2016, they found their answer in <a href="http://kubernetes.io/">Kubernetes</a>, which had just begun supporting rkt implementation.<br /><br />
-    After discussing their needs with their contacts at CoreOS and Google, they were convinced that Kubernetes would work for BlaBlaCar. "We realized that there was a really strong community around it, which meant we would not have to maintain a lot of tools of our own," says Lallemand. "It was better if we could contribute to some bigger project like Kubernetes." They also started using <a href="https://prometheus.io/">Prometheus</a>, as they were looking for "service-oriented monitoring that could be updated nightly." Production on Kubernetes began in December 2016. "We like to do crazy stuff around Christmas," he adds with a laugh.<br /><br />
-    BlaBlaCar now has about 3,000 pods, with 1200 of them running on Kubernetes. Lallemand leads a "foundations team" of 25 members who take care of the networks, databases and systems for about 100 developers. There have been some challenges getting to this point. "The rkt implementation is still not 100 percent finished," Lallemand points out. "It’s really good, but there are some features still missing. We have questions about how we do things with stateful services, like databases. We know how we will be migrating some of the services; some of the others are a bit more complicated to deal with. But the Kubernetes community is making a lot of progress on that part."<br /><br />
-    The team is particularly happy that they’re now able to plan capacity better in the company’s data center. "We have fewer constraints since we have this abstraction between the services and the hardware we run on," says Lallemand. "If we lose a server because there’s a hardware problem on it, we just move the containers onto another server. It’s much more efficient. We do that by just changing a line in the configuration file. And with Kubernetes, it should be automatic, so we would have nothing to do."
-</div>
-</section>
+<p>By 2015, the company had about 50 bare metal servers. The team was using a <a href="https://www.mysql.com/">MySQL</a> database and <a href="http://php.net/">PHP</a>, but, Lallemand says, "it was a very static way." They also utilized the configuration management system, <a href="https://www.chef.io/chef/">Chef</a>, but had little automation in its process. "When you're thinking about doubling the number of servers, you start thinking, 'What should I do to be more efficient?'" says Lallemand. "The answer is not to hire more and more people just to deal with the servers and installation."</p>

-<div class="banner5">
-  <div class="banner5text">
-    "If we lose a server because there’s a hardware problem on it, we just move the containers onto another server. It’s much more efficient. We do that by just changing a line in the configuration file. With Kubernetes, it should be automatic, so we would have nothing to do."
-  </div>
-</div>
+<p>Instead, BlaBlaCar began its cloud-native journey but wasn't sure which route to take. "We could either decide to go into cloud virtualization or even use a private cloud on our own servers," says Lallemand. "But going into the cloud meant we had to make a lot of changes in our application work, and we were just not ready to make the switch from on premise to the cloud." They wanted to keep the great performance they got on bare metal, so they didn't want to go to virtualization on premise.</p>

-<section class="section5">
-<div class="fullcol">
-    And these advances ultimately trickle down to BlaBlaCar’s users. "We have improved availability overall on our website," says Lallemand. "When you’re switching to this cloud-native model with running everything in containers, you have to make sure that you can at any moment reboot a server or a data container without any downtime, without losing traffic. So now our infrastructure is much more resilient and we have better availability than before."<br /><br />
-    Within BlaBlaCar’s technology department, the cloud-native journey has created some profound changes. Lallemand thinks that the regular meetings during the conception stage and the training sessions during implementation helped. "After that everybody took part in the migration process," he says. "Then we split the organization into different ‘tribes’—teams that gather developers, product managers, data analysts, all the different jobs, to work on a specific part of the product. Before, they were organized by function. The idea is to give all these tribes access to the infrastructure directly in a self-service way without having to ask. These people are really autonomous. They have responsibility of that part of the product, and they can make decisions faster." <br /><br />
-    This DevOps transformation turned out to be a positive one for the company’s staffers. "The team was very excited about the DevOps transformation because it was new, and we were working to make things more reliable, more future-proof," says Lallemand. "We like doing things that very few people are doing, other than the internet giants." <br /><br />
-    With these changes already making an impact, BlaBlaCar is looking to split up more and more of its application into services. "I don’t say microservices because they’re not so micro," Lallemand says. "If we can split the responsibilities between the development teams, it would be easier to manage and more reliable, because we can easily add and remove services if one fails. You can handle it easily, instead of adding a big monolith that we still have." <br /><br />
-    When Lallemand speaks to other European companies curious about what BlaBlaCar has done with its infrastructure, he tells them to come along for the ride. "I tell them that it’s such a pleasure to deal with the infrastructure that we have today compared to what we had before," he says. "They just need to keep in mind their real motive, whether it’s flexibility in development or reliability or so on, and then go step by step towards reaching those objectives. That’s what we’ve done. It’s important not to do technology for the sake of technology. Do it for a purpose. Our focus was on helping the developers."
-</div>
-</section>
+<p>The solution: containerization. This was early 2015 and containers were still relatively new. "It was a bold move at the time," says Lallemand. "We decided that the next servers that we would buy in the new data center would all be the same model, so we could outsource the maintenance of the servers. And we decided to go with containers and with <a href="https://coreos.com/">CoreOS</a> Container Linux as an abstraction for this hardware. It seemed future-proof to go with containers because we could see what companies were already doing with containers."</p>
+
+{{< case-studies/quote image="/images/case-studies/blablacar/banner3.jpg">}}
+"With all the tooling that we made around the containers, copying a new service is a matter of minutes. It's a huge gain. For the developers, it means they can focus only on the features that they're developing and not on the infrastructure or the hour they would test their code, or the hour that it would get deployed."
+{{< /case-studies/quote >}}
+
+<p>Next, they needed to choose a runtime for the containers, but "there were very few deployments in production at that time," says Lallemand. They experimented with <a href="https://www.docker.com/">Docker</a> but decided to go with <a href="https://coreos.com/rkt">rkt</a>. Lallemand explains that for BlaBlaCar, it was "much simpler to integrate things that are on rkt." At the time,  the project was still pre-v1.0, so "we could speak with the developers of rkt and give them feedback. It was an advantage." Plus, he notes, rkt was very stable, even at this early stage.</p>
+
+<p>Once those decisions were made that summer, the company came up with a plan for implementation. First, they formed a task force to create a workflow that would be tested by three of the 10 members on Lallemand's team. But they took care to run regular workshops with all 10 members to make sure everyone was on board. "When you're focused on your product sometimes you forget if it's really user friendly, whether other people can manage to create containers too," Lallemand says. "So we did a lot of iterations to find a good workflow."</p>
+
+<p>After establishing the workflow, Lallemand says with a smile that "we had this strange idea that we should try the most difficult thing first. Because if it works, it will work for everything." So the first project the team decided to containerize was the database. "Nobody did that at the time, and there were really no existing tools for what we wanted to do, including building container images," he says. So the team created their own tools, such as <a href="https://github.com/blablacar/dgr">dgr</a>, which builds container images so that the whole team has a common framework to build on the same images with the same standards. They also revamped the service-discovery tools <a href="https://github.com/airbnb/nerve">Nerve</a> and <a href="http://airbnb.io/projects/synapse/">Synapse</a>; their versions, <a href="https://github.com/blablacar/go-nerve">Go-Nerve</a> and <a href="https://github.com/blablacar/go-synapse">Go-Synapse</a>, were written in Go and built to be more efficient and include new features. All of these tools were open-sourced.</p>
+
+<p>At the same time, the company was working to migrate its entire platform to containers with a deadline set for Christmas 2015. With all the work being done in parallel, BlaBlaCar was able to get about 80 percent of its production into containers by its deadline with live traffic running on containers during December. (It's now at 100 percent.) "It's a really busy time for traffic," says Lallemand. "We knew that by using those new servers with containers, it would help us handle the traffic."</p>
+
+<p>In the middle of that peak season for carpooling, everything worked well. "The biggest impact that we had was for the deployment of new services," says Lallemand. "Before using containers, we had to first deploy a new server and create configurations with Chef. It would take sometimes a day, sometimes two, just to create a new service. And with all the tooling that we made around the containers, copying a new service is a matter of minutes. So it's really a huge gain. For the developers, it means they can focus only on the features that they're developing and not on the infrastructure or the hour they would test their code, or the hour that it would get deployed."</p>
+
+{{< case-studies/quote image="/images/case-studies/blablacar/banner4.jpg" >}}
+"We realized that there was a really strong community around it [Kubernetes], which meant we would not have to maintain a lot of tools of our own," says Lallemand. "It was better if we could contribute to some bigger project like Kubernetes."
+{{< /case-studies/quote >}}
+
+<p>In order to meet their self-imposed deadline, one of the decisions they made was to not do any "orchestration magic" for containers in the first production alignment. Instead, they used the basic <a href="https://coreos.com/fleet/docs/latest/launching-containers-fleet.html">fleet</a> tool from CoreOS to deploy their containers. (They did build a tool called <a href="https://github.com/blablacar/ggn">GGN</a>, which they've open-sourced, to make it more manageable for their system engineers to use.)</p>
+
+<p>Still, the team knew that they'd want more orchestration. "Our tool was doing a pretty good job, but at some point you want to give more autonomy to the developer team," Lallemand says. "We also realized that we don't want to be the single point of contact for developers when they want to launch new services." By the summer of 2016, they found their answer in <a href="http://kubernetes.io/">Kubernetes</a>, which had just begun supporting rkt implementation.</p>
+
+<p>After discussing their needs with their contacts at CoreOS and Google, they were convinced that Kubernetes would work for BlaBlaCar. "We realized that there was a really strong community around it, which meant we would not have to maintain a lot of tools of our own," says Lallemand. "It was better if we could contribute to some bigger project like Kubernetes." They also started using <a href="https://prometheus.io/">Prometheus</a>, as they were looking for "service-oriented monitoring that could be updated nightly." Production on Kubernetes began in December 2016. "We like to do crazy stuff around Christmas," he adds with a laugh.</p>
+
+<p>BlaBlaCar now has about 3,000 pods, with 1200 of them running on Kubernetes. Lallemand leads a "foundations team" of 25 members who take care of the networks, databases and systems for about 100 developers. There have been some challenges getting to this point. "The rkt implementation is still not 100 percent finished," Lallemand points out. "It's really good, but there are some features still missing. We have questions about how we do things with stateful services, like databases. We know how we will be migrating some of the services; some of the others are a bit more complicated to deal with. But the Kubernetes community is making a lot of progress on that part."</p>
+
+<p>The team is particularly happy that they're now able to plan capacity better in the company's data center. "We have fewer constraints since we have this abstraction between the services and the hardware we run on," says Lallemand. "If we lose a server because there's a hardware problem on it, we just move the containers onto another server. It's much more efficient. We do that by just changing a line in the configuration file. And with Kubernetes, it should be automatic, so we would have nothing to do."</p>
+
+{{< case-studies/quote >}}
+"If we lose a server because there's a hardware problem on it, we just move the containers onto another server. It's much more efficient. We do that by just changing a line in the configuration file. With Kubernetes, it should be automatic, so we would have nothing to do."
+{{< /case-studies/quote >}}
+
+<p>And these advances ultimately trickle down to BlaBlaCar's users. "We have improved availability overall on our website," says Lallemand. "When you're switching to this cloud-native model with running everything in containers, you have to make sure that you can at any moment reboot a server or a data container without any downtime, without losing traffic. So now our infrastructure is much more resilient and we have better availability than before."</p>
+
+<p>Within BlaBlaCar's technology department, the cloud-native journey has created some profound changes. Lallemand thinks that the regular meetings during the conception stage and the training sessions during implementation helped. "After that everybody took part in the migration process," he says. "Then we split the organization into different 'tribes'—teams that gather developers, product managers, data analysts, all the different jobs, to work on a specific part of the product. Before, they were organized by function. The idea is to give all these tribes access to the infrastructure directly in a self-service way without having to ask. These people are really autonomous. They have responsibility of that part of the product, and they can make decisions faster."</p>
+
+<p>This DevOps transformation turned out to be a positive one for the company's staffers. "The team was very excited about the DevOps transformation because it was new, and we were working to make things more reliable, more future-proof," says Lallemand. "We like doing things that very few people are doing, other than the internet giants."</p>
+
+<p>With these changes already making an impact, BlaBlaCar is looking to split up more and more of its application into services. "I don't say microservices because they're not so micro," Lallemand says. "If we can split the responsibilities between the development teams, it would be easier to manage and more reliable, because we can easily add and remove services if one fails. You can handle it easily, instead of adding a big monolith that we still have."</p>
+
+<p>When Lallemand speaks to other European companies curious about what BlaBlaCar has done with its infrastructure, he tells them to come along for the ride. "I tell them that it's such a pleasure to deal with the infrastructure that we have today compared to what we had before," he says. "They just need to keep in mind their real motive, whether it's flexibility in development or reliability or so on, and then go step by step towards reaching those objectives. That's what we've done. It's important not to do technology for the sake of technology. Do it for a purpose. Our focus was on helping the developers."</p>
@@ -1,112 +1,83 @@
 ---
 title: BlackRock Case Study
-
 case_study_styles: true
 cid: caseStudies
-css: /css/style_blackrock.css
+
+new_case_study_styles: true
+heading_background: /images/case-studies/blackrock/banner1.jpg
+heading_title_logo: /images/blackrock_logo.png
+subheading: >
+  Rolling Out Kubernetes in Production in 100 Days
+case_study_details:
+  - Company: BlackRock
+  - Location: New York, NY
+  - Industry: Financial Services
 ---

-<div class="banner1">
-  <h1> CASE STUDY: <img src="/images/blackrock_logo.png" class="header_logo"><br>
-    <div class="subhead">Rolling Out Kubernetes in Production in 100 Days</div>
-  </h1>
+<h2>Challenge</h2>

-</div>
+<p>The world's largest asset manager, <a href="https://www.blackrock.com/investing">BlackRock</a> operates a very controlled static deployment scheme, which has allowed for scalability over the years. But in their data science division, there was a need for more dynamic access to resources. "We want to be able to give every investor access to data science, meaning <a href="https://www.python.org">Python</a> notebooks, or even something much more advanced, like a MapReduce engine based on <a href="https://spark.apache.org">Spark</a>," says Michael Francis, a Managing Director in BlackRock's  Product Group, which runs the company's investment management platform. "Managing complex Python installations on users' desktops is really hard because everyone ends up with slightly different environments. We have existing environments that do these things, but we needed to make it real, expansive and scalable. Being able to spin that up on demand, tear it down, make that much more dynamic, became a critical thought process for us. It's not so much that we had to solve our main core production problem, it's how do we extend that? How do we evolve?"</p>

-<div class="details">
-    Company &nbsp;<b>BlackRock</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location &nbsp;<b>New York, NY</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Industry &nbsp;<b>Financial Services</b>
-</div>
+<h2>Solution</h2>

-<hr>
+<p>Drawing from what they learned during a pilot done last year using <a href="https://www.docker.com">Docker</a> environments, Francis put together a cross-sectional team of 20 to build an investor research web app using <a href="https://kubernetes.io">Kubernetes</a> with the goal of getting it into production within one quarter.</p>

-<section class="section1">
-
-<div class="cols">
-  <div class="col1">
-    <h2>Challenge</h2>
-        The world’s largest asset manager, <a href="https://www.blackrock.com/investing">BlackRock</a> operates a very controlled static deployment scheme, which has allowed for scalability over the years. But in their data science division, there was a need for more dynamic access to resources. "We want to be able to give every investor access to data science, meaning <a href="https://www.python.org">Python</a> notebooks, or even something much more advanced, like a MapReduce engine based on <a href="https://spark.apache.org">Spark</a>," says Michael Francis, a Managing Director in BlackRock’s  Product Group, which runs the company’s investment management platform. "Managing complex Python installations on users’ desktops is really hard because everyone ends up with slightly different environments. We have existing environments that do these things, but we needed to make it real, expansive and scalable. Being able to spin that up on demand, tear it down, make that much more dynamic, became a critical thought process for us. It’s not so much that we had to solve our main core production problem, it’s how do we extend that? How do we evolve?"
-      </div>
-
-<div class="col2">
-  <h2>Solution</h2>
-    Drawing from what they learned during a pilot done last year using <a href="https://www.docker.com">Docker</a> environments, Francis put together a cross-sectional team of 20 to build an investor research web app using <a href="https://kubernetes.io">Kubernetes</a> with the goal of getting it into production within one quarter.
-<br><br>
 <h2>Impact</h2>
-        "Our goal was: How do you give people tools rapidly without having to install them on their desktop?" says Francis. And the team hit the goal within 100 days. Francis is pleased with the results and says, "We’re going to use this infrastructure for lots of other application workloads as time goes on. It’s not just data science; it’s this style of application that needs the dynamism. But I think we’re 6-12 months away from making a [large scale] decision. We need to gain experience of running the system in production, we need to understand failure modes and how best to manage operational issues. What’s interesting is that just having this technology there is changing the way our developers are starting to think about their future development."

-</div>
-</div>
+<p>"Our goal was: How do you give people tools rapidly without having to install them on their desktop?" says Francis. And the team hit the goal within 100 days. Francis is pleased with the results and says, "We're going to use this infrastructure for lots of other application workloads as time goes on. It's not just data science; it's this style of application that needs the dynamism. But I think we're 6-12 months away from making a [large scale] decision. We need to gain experience of running the system in production, we need to understand failure modes and how best to manage operational issues. What's interesting is that just having this technology there is changing the way our developers are starting to think about their future development."</p>

-</section>
+{{< case-studies/quote author="Michael Francis, Managing Director, BlackRock">}}
+"My message to other enterprises like us is you can actually integrate Kubernetes into an existing, well-orchestrated machinery. You don't have to throw out everything you do. And using Kubernetes made a complex problem significantly easier."
+{{< /case-studies/quote >}}

-<div class="banner2">
-  <div class="banner2text">
-    "My message to other enterprises like us is you can actually integrate Kubernetes into an existing, well-orchestrated machinery. You don’t have to throw out everything you do. And using Kubernetes made a complex problem significantly easier."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Michael Francis, Managing Director, BlackRock</span>
-  </div>
-</div>
+<p>One of the management objectives for BlackRock's  Product Group employees in 2017 was to "build cool stuff." Led by Managing Director Michael Francis, a cross-sectional group of 20 did just that: They rolled out a full production Kubernetes environment and released a new investor research web app on it. In 100 days.</p>

-<section class="section2">
+<p>For a company that's the world's largest asset manager, "just equipment procurement can take 100 days sometimes, let alone from inception to delivery," says Karl Wieman, a Senior System Administrator. "It was an aggressive schedule. But it moved the dial." In fact, the project achieved two goals: It solved a business problem (creating the needed web app) as well as provided real-world, in-production experience with Kubernetes, a cloud-native technology that the company was eager to explore. "It's not so much that we had to solve our main core production problem, it's how do we extend that? How do we evolve?" says Francis. The ultimate success of this project, beyond delivering the app, lies in the fact that "we've managed to integrate a radically new thought process into a controlled infrastructure that we didn't want to change."</p>

-<div class="fullcol">
-  One of the management objectives for BlackRock’s  Product Group employees in 2017 was to "build cool stuff." Led by Managing Director Michael Francis, a cross-sectional group of 20 did just that: They rolled out a full production Kubernetes environment and released a new investor research web app on it. In 100 days.<br><br>
-  For a company that’s the world’s largest asset manager, "just equipment procurement can take 100 days sometimes, let alone from inception to delivery," says Karl Wieman, a Senior System Administrator. "It was an aggressive schedule. But it moved the dial."
-  In fact, the project achieved two goals: It solved a business problem (creating the needed web app) as well as provided real-world, in-production experience with Kubernetes, a cloud-native technology that the company was eager to explore. "It’s not so much that we had to solve our main core production problem, it’s how do we extend that? How do we evolve?" says Francis. The ultimate success of this project, beyond delivering the app, lies in the fact that "we’ve managed to integrate a radically new thought process into a controlled infrastructure that we didn’t want to change."<br><br>
-  After all, in its three decades of existence, BlackRock has "a very well-established environment for managing our compute resources," says Francis. "We manage large cluster processes on machines, so we do a lot of orchestration and management for our main production processes in a way that’s very cloudish in concept. We’re able to manage them in a very controlled, static deployment scheme, and that has given us a huge amount of scalability."<br><br>
-  Though that works well for the core production, the company has found that some data science workloads require more dynamic access to resources. "It’s a very bursty process," says Francis, who is head of data for the company’s Aladdin investment management platform division.<br><br>
-  Aladdin, which connects the people, information and technology needed for money management in real time, is used internally and is also sold as a platform to other asset managers and insurance companies. "We want to be able to give every investor access to data science, meaning <a href="https://www.python.org">Python</a> notebooks, or even something much more advanced, like a MapReduce engine based on <a href="https://spark.apache.org">Spark</a>," says Francis. But "managing complex Python installations on users’ desktops is really hard because everyone ends up with slightly different environments. Docker allows us to flatten that environment."
-</div>
-</section>
+<p>After all, in its three decades of existence, BlackRock has "a very well-established environment for managing our compute resources," says Francis. "We manage large cluster processes on machines, so we do a lot of orchestration and management for our main production processes in a way that's very cloudish in concept. We're able to manage them in a very controlled, static deployment scheme, and that has given us a huge amount of scalability."</p>

-<div class="banner3">
-  <div class="banner3text">
-    "We manage large cluster processes on machines, so we do a lot of orchestration and management for our main production processes in a way that’s very cloudish in concept. We’re able to manage them in a very controlled, static deployment scheme, and that has given us a huge amount of scalability."
-  </div>
-</div>
+<p>Though that works well for the core production, the company has found that some data science workloads require more dynamic access to resources. "It's a very bursty process," says Francis, who is head of data for the company's Aladdin investment management platform division.</p>

-<section class="section3">
-<div class="fullcol">
-  Still, challenges remain. "If you have a shared cluster, you get this storming herd problem where everyone wants to do the same thing at the same time," says Francis. "You could put limits on it, but you’d have to build an infrastructure to define limits for our processes, and the Python notebooks weren’t really designed for that. We have existing environments that do these things, but we needed to make it real, expansive, and scalable. Being able to spin that up on demand, tear it down, and make that much more dynamic, became a critical thought process for us."<br><br>
-  Made up of managers from technology, infrastructure, production operations, development and information security, Francis’s team was able to look at the problem holistically and come up with a solution that made sense for BlackRock. "Our initial straw man was that we were going to build everything using <a href="https://www.ansible.com">Ansible</a> and run it all using some completely different distributed environment," says Francis. "That would have been absolutely the wrong thing to do. Had we gone off on our own as the dev team and developed this solution, it would have been a very different product. And it would have been very expensive. We would not have gone down the route of running under our existing orchestration system. Because we don’t understand it. These guys [in operations and infrastructure] understand it. Having the multidisciplinary team allowed us to get to the right solutions and that actually meant we didn’t build anywhere near the amount we thought we were going to end up building."<br><br>
-  In search of a solution in which they could manage usage on a user-by-user level, Francis’s team gravitated to Red Hat’s <a href="https://www.openshift.com">OpenShift</a> Kubernetes offering. The company had already experimented with other cloud-native environments, but the team liked that Kubernetes was open source, and "we felt the winds were blowing in the direction of Kubernetes long term," says Francis. "Typically we make technology choices that we believe are going to be here in 5-10 years’ time, in some form. And right now, in this space, Kubernetes feels like the one that’s going to be there." Adds Uri Morris, Vice President of Production Operations: "When you see that the non-Google committers to Kubernetes overtook the Google committers, that’s an indicator of the momentum."<br><br>
-  Once that decision was made, the major challenge was figuring out how to make Kubernetes work within BlackRock’s existing framework. "It’s about understanding how we can operate, manage and support a platform like this, in addition to tacking it onto our existing technology platform," says Project Manager Michael Maskallis. "All the controls we have in place, the change management process, the software development lifecycle, onboarding processes we go through—how can we do all these things?"<br><br>
-  The first (anticipated) speed bump was working around issues behind BlackRock’s corporate firewalls. "One of our challenges is there are no firewalls in most open source software," says Francis. "So almost all install scripts fail in some bizarre way, and pulling down packages doesn’t necessarily work." The team ran into these types of problems using <a href="/docs/getting-started-guides/minikube/">Minikube</a> and did a few small pushes back to the open source project.
+<p>Aladdin, which connects the people, information and technology needed for money management in real time, is used internally and is also sold as a platform to other asset managers and insurance companies. "We want to be able to give every investor access to data science, meaning <a href="https://www.python.org">Python</a> notebooks, or even something much more advanced, like a MapReduce engine based on <a href="https://spark.apache.org">Spark</a>," says Francis. But "managing complex Python installations on users' desktops is really hard because everyone ends up with slightly different environments. Docker allows us to flatten that environment."</p>

+{{< case-studies/quote image="/images/case-studies/blackrock/banner3.jpg">}}
+"We manage large cluster processes on machines, so we do a lot of orchestration and management for our main production processes in a way that's very cloudish in concept. We're able to manage them in a very controlled, static deployment scheme, and that has given us a huge amount of scalability."
+{{< /case-studies/quote >}}

-</div>
-</section>
+<p>Still, challenges remain. "If you have a shared cluster, you get this storming herd problem where everyone wants to do the same thing at the same time," says Francis. "You could put limits on it, but you'd have to build an infrastructure to define limits for our processes, and the Python notebooks weren't really designed for that. We have existing environments that do these things, but we needed to make it real, expansive, and scalable. Being able to spin that up on demand, tear it down, and make that much more dynamic, became a critical thought process for us."</p>

-<div class="banner4">
-  <div class="banner4text">
-    "Typically we make technology choices that we believe are going to be here in 5-10 years’ time, in some form. And right now, in this space, Kubernetes feels like the one that’s going to be there."
-  </div>
-</div>
+<p>Made up of managers from technology, infrastructure, production operations, development and information security, Francis's team was able to look at the problem holistically and come up with a solution that made sense for BlackRock. "Our initial straw man was that we were going to build everything using <a href="https://www.ansible.com">Ansible</a> and run it all using some completely different distributed environment," says Francis. "That would have been absolutely the wrong thing to do. Had we gone off on our own as the dev team and developed this solution, it would have been a very different product. And it would have been very expensive. We would not have gone down the route of running under our existing orchestration system. Because we don't understand it. These guys [in operations and infrastructure] understand it. Having the multidisciplinary team allowed us to get to the right solutions and that actually meant we didn't build anywhere near the amount we thought we were going to end up building."</p>

-<section class="section4">
-<div class="fullcol">
-  There were also questions about service discovery. "You can think of Aladdin as a cloud of services with APIs between them that allows us to build applications rapidly," says Francis. "It’s all on a proprietary message bus, which gives us all sorts of advantages but at the same time, how does that play in a third party [platform]?"<br><br>
-  Another issue they had to navigate was that in BlackRock’s existing system, the messaging protocol has different instances in the different development, test and production environments. While Kubernetes enables a more DevOps-style model, it didn’t make sense for BlackRock. "I think what we are very proud of is that the ability for us to push into production is still incredibly rapid in this [new] infrastructure, but we have the control points in place, and we didn’t have to disrupt everything," says Francis. "A lot of the cost of this development was thinking how best to leverage our internal tools. So it was less costly than we actually thought it was going to be."<br><br>
-  The project leveraged tools associated with the messaging bus, for example. "The way that the Kubernetes cluster will talk to our internal messaging platform is through a gateway program, and this gateway program already has built-in checks and throttles," says Morris. "We can use them to control and potentially throttle the requests coming in from Kubernetes’s very elastic infrastructure to the production infrastructure. We’ll continue to go in that direction. It enables us to scale as we need to from the operational perspective."<br><br>
-  The solution also had to be complementary with BlackRock’s centralized operational support team structure. "The core infrastructure components of Kubernetes are hooked into our existing orchestration framework, which means that anyone in our support team has both control and visibility to the cluster using the existing operational tools," Morris explains. "That means that I don’t need to hire more people."<br><br>
-  With those points established, the team created a procedure for the project: "We rolled this out first to a development environment, then moved on to a testing environment and then eventually to two production environments, in that sequential order," says Maskallis. "That drove a lot of our learning curve. We have all these moving parts, the software components on the infrastructure side, the software components with Kubernetes directly, the interconnectivity with the rest of the environment that we operate here at BlackRock, and how we connect all these pieces. If we came across issues, we fixed them, and then moved on to the different environments to replicate that until we eventually ended up in our production environment where this particular cluster is supposed to live."<br><br>
-  The team had weekly one-hour working sessions with all the members (who are located around the world) participating, and smaller breakout or deep-dive meetings focusing on specific technical details. Possible solutions would be reported back to the group and debated the following week. "I think what made it a successful experiment was people had to work to learn, and they shared their experiences with others," says Vice President and Software Developer Fouad Semaan. Then, Francis says, "We gave our engineers the space to do what they’re good at. This hasn’t been top-down."
+<p>In search of a solution in which they could manage usage on a user-by-user level, Francis's team gravitated to Red Hat's <a href="https://www.openshift.com">OpenShift</a> Kubernetes offering. The company had already experimented with other cloud-native environments, but the team liked that Kubernetes was open source, and "we felt the winds were blowing in the direction of Kubernetes long term," says Francis. "Typically we make technology choices that we believe are going to be here in 5-10 years' time, in some form. And right now, in this space, Kubernetes feels like the one that's going to be there." Adds Uri Morris, Vice President of Production Operations: "When you see that the non-Google committers to Kubernetes overtook the Google committers, that's an indicator of the momentum."</p>

+<p>Once that decision was made, the major challenge was figuring out how to make Kubernetes work within BlackRock's existing framework. "It's about understanding how we can operate, manage and support a platform like this, in addition to tacking it onto our existing technology platform," says Project Manager Michael Maskallis. "All the controls we have in place, the change management process, the software development lifecycle, onboarding processes we go through—how can we do all these things?"</p>

-</div>
-</section>
+<p>The first (anticipated) speed bump was working around issues behind BlackRock's corporate firewalls. "One of our challenges is there are no firewalls in most open source software," says Francis. "So almost all install scripts fail in some bizarre way, and pulling down packages doesn't necessarily work." The team ran into these types of problems using <a href="/docs/getting-started-guides/minikube/">Minikube</a> and did a few small pushes back to the open source project.</p>

-<div class="banner5">
-  <div class="banner5text">
-    "The core infrastructure components of Kubernetes are hooked into our existing orchestration framework, which means that anyone in our support team has both control and visibility to the cluster using the existing operational tools. That means that I don’t need to hire more people."
+{{< case-studies/quote image="/images/case-studies/blackrock/banner4.jpg">}}
+"Typically we make technology choices that we believe are going to be here in 5-10 years' time, in some form. And right now, in this space, Kubernetes feels like the one that's going to be there."
+{{< /case-studies/quote >}}

-  </div>
-</div>
+<p>There were also questions about service discovery. "You can think of Aladdin as a cloud of services with APIs between them that allows us to build applications rapidly," says Francis. "It's all on a proprietary message bus, which gives us all sorts of advantages but at the same time, how does that play in a third party [platform]?"</p>

-<section class="section5">
-<div class="fullcol">
-  They were led by one key axiom: To stay focused and avoid scope creep. This meant that they wouldn’t use features that weren’t in the core of Kubernetes and Docker. But if there was a real need, they’d build the features themselves. Luckily, Francis says, "Because of the rapidity of the development, a lot of things we thought we would have to build ourselves have been rolled into the core product. [The package manager<a href="https://helm.sh"> Helm</a> is one example]. People have similar problems."<br><br>
-  By the end of the 100 days, the app was up and running for internal BlackRock users. The initial capacity of 30 users was hit within hours, and quickly increased to 150. "People were immediately all over it," says Francis. In the next phase of this project, they are planning to scale up the cluster to have more capacity.<br><br>
-  Even more importantly, they now have in-production experience with Kubernetes that they can continue to build on—and a complete framework for rolling out new applications. "We’re going to use this infrastructure for lots of other application workloads as time goes on. It’s not just data science; it’s this style of application that needs the dynamism," says Francis. "Is it the right place to move our core production processes onto? It might be. We’re not at a point where we can say yes or no, but we felt that having real production experience with something like Kubernetes at some form and scale would allow us to understand that. I think we’re 6-12 months away from making a [large scale] decision. We need to gain experience of running the system in production, we need to understand failure modes and how best to manage operational issues."<br><br>
-  For other big companies considering a project like this, Francis says commitment and dedication are key: "We got the signoff from [senior management] from day one, with the commitment that we were able to get the right people. If I had to isolate what makes something complex like this succeed, I would say senior hands-on people who can actually drive it make a huge difference." With that in place, he adds, "My message to other enterprises like us is you can actually integrate Kubernetes into an existing, well-orchestrated machinery. You don’t have to throw out everything you do. And using Kubernetes made a complex problem significantly easier."
+<p>Another issue they had to navigate was that in BlackRock's existing system, the messaging protocol has different instances in the different development, test and production environments. While Kubernetes enables a more DevOps-style model, it didn't make sense for BlackRock. "I think what we are very proud of is that the ability for us to push into production is still incredibly rapid in this [new] infrastructure, but we have the control points in place, and we didn't have to disrupt everything," says Francis. "A lot of the cost of this development was thinking how best to leverage our internal tools. So it was less costly than we actually thought it was going to be."</p>

-</div>
-</section>
+<p>The project leveraged tools associated with the messaging bus, for example. "The way that the Kubernetes cluster will talk to our internal messaging platform is through a gateway program, and this gateway program already has built-in checks and throttles," says Morris. "We can use them to control and potentially throttle the requests coming in from Kubernetes's very elastic infrastructure to the production infrastructure. We'll continue to go in that direction. It enables us to scale as we need to from the operational perspective."</p>
+
+<p>The solution also had to be complementary with BlackRock's centralized operational support team structure. "The core infrastructure components of Kubernetes are hooked into our existing orchestration framework, which means that anyone in our support team has both control and visibility to the cluster using the existing operational tools," Morris explains. "That means that I don't need to hire more people."</p>
+
+<p>With those points established, the team created a procedure for the project: "We rolled this out first to a development environment, then moved on to a testing environment and then eventually to two production environments, in that sequential order," says Maskallis. "That drove a lot of our learning curve. We have all these moving parts, the software components on the infrastructure side, the software components with Kubernetes directly, the interconnectivity with the rest of the environment that we operate here at BlackRock, and how we connect all these pieces. If we came across issues, we fixed them, and then moved on to the different environments to replicate that until we eventually ended up in our production environment where this particular cluster is supposed to live."</p>
+
+<p>The team had weekly one-hour working sessions with all the members (who are located around the world) participating, and smaller breakout or deep-dive meetings focusing on specific technical details. Possible solutions would be reported back to the group and debated the following week. "I think what made it a successful experiment was people had to work to learn, and they shared their experiences with others," says Vice President and Software Developer Fouad Semaan. Then, Francis says, "We gave our engineers the space to do what they're good at. This hasn't been top-down."</p>
+
+{{< case-studies/quote >}}
+"The core infrastructure components of Kubernetes are hooked into our existing orchestration framework, which means that anyone in our support team has both control and visibility to the cluster using the existing operational tools. That means that I don't need to hire more people."
+{{< /case-studies/quote >}}
+
+<p>They were led by one key axiom: To stay focused and avoid scope creep. This meant that they wouldn't use features that weren't in the core of Kubernetes and Docker. But if there was a real need, they'd build the features themselves. Luckily, Francis says, "Because of the rapidity of the development, a lot of things we thought we would have to build ourselves have been rolled into the core product. [The package manager<a href="https://helm.sh"> Helm</a> is one example]. People have similar problems."</p>
+
+<p>By the end of the 100 days, the app was up and running for internal BlackRock users. The initial capacity of 30 users was hit within hours, and quickly increased to 150. "People were immediately all over it," says Francis. In the next phase of this project, they are planning to scale up the cluster to have more capacity.</p>
+
+<p>Even more importantly, they now have in-production experience with Kubernetes that they can continue to build on—and a complete framework for rolling out new applications. "We're going to use this infrastructure for lots of other application workloads as time goes on. It's not just data science; it's this style of application that needs the dynamism," says Francis. "Is it the right place to move our core production processes onto? It might be. We're not at a point where we can say yes or no, but we felt that having real production experience with something like Kubernetes at some form and scale would allow us to understand that. I think we're 6-12 months away from making a [large scale] decision. We need to gain experience of running the system in production, we need to understand failure modes and how best to manage operational issues."</p>
+
+<p>For other big companies considering a project like this, Francis says commitment and dedication are key: "We got the signoff from [senior management] from day one, with the commitment that we were able to get the right people. If I had to isolate what makes something complex like this succeed, I would say senior hands-on people who can actually drive it make a huge difference." With that in place, he adds, "My message to other enterprises like us is you can actually integrate Kubernetes into an existing, well-orchestrated machinery. You don't have to throw out everything you do. And using Kubernetes made a complex problem significantly easier."</p>
@@ -3,115 +3,84 @@ title: Booking.com Case Study
 linkTitle: Booking.com
 case_study_styles: true
 cid: caseStudies
-css: /css/case-studies-gradient.css
 logo: booking.com_featured_logo.png
 featured: true
 weight: 3
 quote: >
  We realized that we needed to learn Kubernetes better in order to fully use the potential of it. At that point, we made the shift to build our own Kubernetes platform.
+
+new_case_study_styles: true
+heading_background: /images/case-studies/booking/banner1.jpg
+heading_title_text: Booking.com
+use_gradient_overlay: true
+subheading: >
+  After Learning the Ropes with a Kubernetes Distribution, Booking.com Built a Platform of Its Own
+case_study_details:
+  - Company: Booking.com
+  - Location: Netherlands
+  - Industry: Travel
 ---

-
+<h2>Challenge</h2>

+<p>In 2016, Booking.com migrated to an OpenShift platform, which gave product developers faster access to infrastructure. But because Kubernetes was abstracted away from the developers, the infrastructure team became a "knowledge bottleneck" when challenges arose. Trying to scale that support wasn't sustainable.</p>

-<div class="article image overlay" style="background-image: url('/images/case-studies/booking/banner1.jpg')">
-  <h1> CASE STUDY: Booking.com</h1>
-  <div class="subhead">After Learning the Ropes with a Kubernetes Distribution, Booking.com Built a Platform of Its Own</div>
-</div>
-
-
-<div class="details">
-    Company &nbsp;<b>Booking.com</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location &nbsp;<b>Netherlands</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Industry &nbsp;<b>Travel</b>
-</div>
-
-<hr>
-<section class="section1">
-<div class="colsgradient">
-  <div class="col1">
-    <h2>Challenge</h2>
-  In 2016, Booking.com migrated to an OpenShift platform, which gave product developers faster access to infrastructure. But because Kubernetes was abstracted away from the developers, the infrastructure team became a “knowledge bottleneck” when challenges arose. Trying to scale that support wasn’t sustainable.
+<h2>Solution</h2>

+<p>After a year operating OpenShift, the platform team decided to build its own vanilla Kubernetes platform—and ask developers to learn some Kubernetes in order to use it. "This is not a magical platform," says Ben Tyler, Principal Developer, B Platform Track. "We're not claiming that you can just use it with your eyes closed. Developers need to do some learning, and we're going to do everything we can to make sure they have access to that knowledge."</p>

-    <h2>Solution</h2>
-    After a year operating OpenShift, the platform team decided to build its own vanilla Kubernetes platform—and ask developers to learn some Kubernetes in order to use it. “This is not a magical platform,” says Ben Tyler, Principal Developer, B Platform Track. “We’re not claiming that you can just use it with your eyes closed. Developers need to do some learning, and we’re going to do everything we can to make sure they have access to that knowledge.”
-
 <h2>Impact</h2>
-   Despite the learning curve, there’s been a great uptick in adoption of the new Kubernetes platform. Before containers, creating a new service could take a couple of days if the developers understood Puppet, or weeks if they didn’t. On the new platform, it can take as few as 10 minutes. About 500 new services were built on the platform in the first 8 months.
-</div>
-</div>
-</section>
-<div class="article image overlay" style="background-image: url('/images/case-studies/booking/banner2.jpg');">
-  <div class="quotetext">
-    “As our users learn Kubernetes and become more sophisticated Kubernetes users, they put pressure on us to provide a better, more native Kubernetes experience, which is great. It’s a super healthy dynamic.”
-<p><div class="quoteauthortext">- BEN TYLER, PRINCIPAL DEVELOPER, B PLATFORM TRACK AT BOOKING.COM</div></p>
-  </div>
-</div>
-
-
-<section class="section2">
-<div class="fullcol">
-  <h4>Booking.com has a long history with Kubernetes: In 2015, a team at the travel platform prototyped a container platform based on Mesos and Marathon.
-</h4>

-  <p>Impressed by what the technology offered, but in need of enterprise features at its scale—the site handles more than 1.5 million room-night reservations a day on average—the team decided to adopt an OpenShift platform.</p>
+<p>Despite the learning curve, there's been a great uptick in adoption of the new Kubernetes platform. Before containers, creating a new service could take a couple of days if the developers understood Puppet, or weeks if they didn't. On the new platform, it can take as few as 10 minutes. About 500 new services were built on the platform in the first 8 months.</p>

-  <p>This platform, which was wrapped in a Heroku-style, high-level CLI interface, “was definitely popular with our product developers,” says Ben Tyler, Principal Developer, B Platform Track. “We gave them faster access to infrastructure.”</p>
+{{< case-studies/quote
+  image="/images/case-studies/booking/banner2.jpg"
+  author="BEN TYLER, PRINCIPAL DEVELOPER, B PLATFORM TRACK AT BOOKING.COM"
+>}}
+"As our users learn Kubernetes and become more sophisticated Kubernetes users, they put pressure on us to provide a better, more native Kubernetes experience, which is great. It's a super healthy dynamic."
+{{< /case-studies/quote >}}

-<p>But, he adds, “anytime something went slightly off the rails, developers didn’t have any of the knowledge required to support themselves.”</p>
+{{< case-studies/lead >}}
+Booking.com has a long history with Kubernetes: In 2015, a team at the travel platform prototyped a container platform based on Mesos and Marathon.
+{{< /case-studies/lead >}}

- <p>And after a year of operating this platform, the infrastructure team found that it had become “a knowledge bottleneck,” he says. “Most of the developers who used it did not know it was Kubernetes underneath. An application failure and a platform failure both looked like failures of that Heroku-style tool.”</p>
-  <p>
-   Scaling the necessary support did not seem feasible or sustainable, so the platform team needed a new solution. The understanding of Kubernetes that they had gained operating the OpenShift platform gave them confidence to build a vanilla Kubernetes platform of their own and customize it to suit the company’s needs.</p>
+<p>Impressed by what the technology offered, but in need of enterprise features at its scale—the site handles more than 1.5 million room-night reservations a day on average—the team decided to adopt an OpenShift platform.</p>

+<p>This platform, which was wrapped in a Heroku-style, high-level CLI interface, "was definitely popular with our product developers," says Ben Tyler, Principal Developer, B Platform Track. "We gave them faster access to infrastructure."</p>

-</div>
-</section>
-
-<div class="greybanner">
-  <div class="greyquotetext">
-     “For entering the landscape, OpenShift was definitely very helpful. It shows you what the technology can do, and it makes it easy for you to use it. After we spent some time on it, we realized that we needed to learn Kubernetes better in order to fully use the potential of it. At that point, we made the shift to build our own Kubernetes platform. We definitely benefit in the long term for taking that step and investing the time in gaining that knowledge.”<p><div class="quoteauthortext">- EDUARD IACOBOAIA, SENIOR SYSTEM ADMINISTRATOR, B PLATFORM TRACK AT BOOKING.COM</div></p>
-  </div>
-</div>
-
-<section class="section3">
-<div class="fullcol">
-   <p>
-    “For entering the landscape, OpenShift was definitely very helpful,” says Eduard Iacoboaia, Senior System Administrator, B Platform Track. “It shows you what the technology can do, and it makes it easy for you to use it. After we spent some time on it, we realized that we needed to learn Kubernetes better in order to fully use the potential of it. At that point, we made the shift to build our own Kubernetes platform. We definitely benefit in the long term for taking that step and investing the time in gaining that knowledge.” </p>
- <p>
-   Iacoboaia’s team had customized a lot of OpenShift tools to make them work at Booking.com, and “those integrations points were kind of fragile,” he says. “We spent much more time understanding all the components of Kubernetes, how they work, how they interact with each other.” That research led the team to switch from OpenShift’s built-in Ansible playbooks to Puppet deployments, which are used for the rest of Booking’s infrastructure. The control plane was also moved from inside the cluster onto bare metal, as the company runs tens of thousands of bare-metal servers and a large infrastructure for running applications on bare metal. (Booking runs Kubernetes in multiple clusters in multiple data centers across the various regions where it has compute.) “We decided to keep it as simple as possible and to also use the tools that we know best,” says Iacoboaia. </p>
-   <p>
-  The other big change was that product engineers would have to learn Kubernetes in order to onboard. “This is not a magical platform,” says Tyler. “We’re not claiming that you can just use it with your eyes closed. Developers need to do some learning, and we’re going to do everything we can to make sure they have access to that knowledge.” That includes trainings, blog posts, videos, and Udemy courses. </p>
-  <p>
-  Despite the learning curve, there’s been a great uptick in adoption of the new Kubernetes platform. “I think the reason we’ve been able to strike this bargain successfully is that we’re not asking them to learn a proprietary app system,” says Tyler. “We’re asking them to learn something that’s open source, where the knowledge is transferable. They’re investing in their own careers by learning Kubernetes.”</p>
-  <p>
- One clear sign that this strategy has been a success is that in the support channel, when users have questions, other product engineers are jumping in to respond. “I haven’t seen that kind of community engagement around a particular platform product internally before,” says Tyler. “It helps a lot that it’s visibly an ecosystem standard outside of the company, so people feel value in investing in that knowledge and sharing it with others, which is really, really powerful.” </p>
+<p>But, he adds, "anytime something went slightly off the rails, developers didn't have any of the knowledge required to support themselves."</p>

+<p>And after a year of operating this platform, the infrastructure team found that it had become "a knowledge bottleneck," he says. "Most of the developers who used it did not know it was Kubernetes underneath. An application failure and a platform failure both looked like failures of that Heroku-style tool."</p>

-</div>
-</section>
-
-
-  <div class="article image overlay" style="background-image: url('/images/case-studies/booking/banner3.jpg');">
-  <div class="quotetext">
- “We have a tutorial. You follow the tutorial. Your code is running. Then, it’s business-logic time. The time to gain access to resources is decreased enormously.” <p><div class="quoteauthortext">- BEN TYLER, PRINCIPAL DEVELOPER, B PLATFORM TRACK AT BOOKING.COM</div></p>
-  </div>
-</div>
-
-<section class="section5">
-<div class="fullcol">
-  <p>
- There’s other quantifiable evidence too: Before containers, creating a new service could take a couple of days if the developers understood Puppet, or weeks if they didn’t. On the new platform, it takes 10 minutes. “We have a tutorial. You follow the tutorial. Your code is running. Then, it’s business-logic time,” says Tyler. “The time to gain access to resources is decreased enormously.” About 500 new services were built in the first 8 months on the platform, with hundreds of releases per day.  </p>
-  <p>
-  The platform offers different “layers of contracts, so to speak,” says Tyler. “At the very base, it’s just Kubernetes. If you’re a pro Kubernetes user, here’s a Kubernetes API, just like you get from GKE or AKS. We’re trying to be a provider on that same level. But our whole job inside the company is to be a bigger value add than just vanilla infrastructure, so we provide a set of base images for our main stacks, Perl and Java.”
-</p>
-<p>
-  And “as our users learn Kubernetes and become more sophisticated Kubernetes users, they put pressure on us to provide a better more native Kubernetes experience, which is great,” says Tyler. “It’s a super healthy dynamic.”</p>
-<p>
-  The platform also includes other CNCF technologies, such as Envoy, Helm, and Prometheus. Most of the critical service traffic for Booking.com is routed through Envoy, and Prometheus is used primarily to monitor infrastructure components. Helm is consumed as a packaging standard. The team also developed and open sourced Shipper, an extension for Kubernetes to add more complex rollout strategies and multi-cluster orchestration.
-</p>
-<p>
-  To be sure, there have been internal discussions about the wisdom of building a Kubernetes platform from the ground up. “This is not really our core competency—Kubernetes and travel, they’re kind of far apart, right?” says Tyler. “But we’ve made a couple of bets on CNCF components that have worked out really well for us. Envoy and Kubernetes, in particular, have been really beneficial to our organization. We were able to customize them, either because we could look at the source code or because they had extension points, and we were able to get value out of them very quickly without having to change any paradigms internally.”
-</p>
-</div>
-</section>
-</body>
+<p>Scaling the necessary support did not seem feasible or sustainable, so the platform team needed a new solution. The understanding of Kubernetes that they had gained operating the OpenShift platform gave them confidence to build a vanilla Kubernetes platform of their own and customize it to suit the company's needs.</p>
+
+{{< case-studies/quote author="EDUARD IACOBOAIA, SENIOR SYSTEM ADMINISTRATOR, B PLATFORM TRACK AT BOOKING.COM" >}}
+"For entering the landscape, OpenShift was definitely very helpful. It shows you what the technology can do, and it makes it easy for you to use it. After we spent some time on it, we realized that we needed to learn Kubernetes better in order to fully use the potential of it. At that point, we made the shift to build our own Kubernetes platform. We definitely benefit in the long term for taking that step and investing the time in gaining that knowledge."
+{{< /case-studies/quote >}}
+
+<p>"For entering the landscape, OpenShift was definitely very helpful," says Eduard Iacoboaia, Senior System Administrator, B Platform Track. "It shows you what the technology can do, and it makes it easy for you to use it. After we spent some time on it, we realized that we needed to learn Kubernetes better in order to fully use the potential of it. At that point, we made the shift to build our own Kubernetes platform. We definitely benefit in the long term for taking that step and investing the time in gaining that knowledge."</p>
+
+<p>Iacoboaia's team had customized a lot of OpenShift tools to make them work at Booking.com, and "those integrations points were kind of fragile," he says. "We spent much more time understanding all the components of Kubernetes, how they work, how they interact with each other." That research led the team to switch from OpenShift's built-in Ansible playbooks to Puppet deployments, which are used for the rest of Booking's infrastructure. The control plane was also moved from inside the cluster onto bare metal, as the company runs tens of thousands of bare-metal servers and a large infrastructure for running applications on bare metal. (Booking runs Kubernetes in multiple clusters in multiple data centers across the various regions where it has compute.) "We decided to keep it as simple as possible and to also use the tools that we know best," says Iacoboaia.</p>
+
+<p>The other big change was that product engineers would have to learn Kubernetes in order to onboard. "This is not a magical platform," says Tyler. "We're not claiming that you can just use it with your eyes closed. Developers need to do some learning, and we're going to do everything we can to make sure they have access to that knowledge." That includes trainings, blog posts, videos, and Udemy courses.</p>
+
+<p>Despite the learning curve, there's been a great uptick in adoption of the new Kubernetes platform. "I think the reason we've been able to strike this bargain successfully is that we're not asking them to learn a proprietary app system," says Tyler. "We're asking them to learn something that's open source, where the knowledge is transferable. They're investing in their own careers by learning Kubernetes."</p>
+
+<p>One clear sign that this strategy has been a success is that in the support channel, when users have questions, other product engineers are jumping in to respond. "I haven't seen that kind of community engagement around a particular platform product internally before," says Tyler. "It helps a lot that it's visibly an ecosystem standard outside of the company, so people feel value in investing in that knowledge and sharing it with others, which is really, really powerful."</p>
+
+{{< case-studies/quote
+  image="/images/case-studies/booking/banner3.jpg"
+  author="BEN TYLER, PRINCIPAL DEVELOPER, B PLATFORM TRACK AT BOOKING.COM"
+>}}
+"We have a tutorial. You follow the tutorial. Your code is running. Then, it's business-logic time. The time to gain access to resources is decreased enormously."
+{{< /case-studies/quote >}}
+
+<p>There's other quantifiable evidence too: Before containers, creating a new service could take a couple of days if the developers understood Puppet, or weeks if they didn't. On the new platform, it takes 10 minutes. "We have a tutorial. You follow the tutorial. Your code is running. Then, it's business-logic time," says Tyler. "The time to gain access to resources is decreased enormously." About 500 new services were built in the first 8 months on the platform, with hundreds of releases per day.</p>
+
+<p>The platform offers different "layers of contracts, so to speak," says Tyler. "At the very base, it's just Kubernetes. If you're a pro Kubernetes user, here's a Kubernetes API, just like you get from GKE or AKS. We're trying to be a provider on that same level. But our whole job inside the company is to be a bigger value add than just vanilla infrastructure, so we provide a set of base images for our main stacks, Perl and Java."</p>
+
+<p>And "as our users learn Kubernetes and become more sophisticated Kubernetes users, they put pressure on us to provide a better more native Kubernetes experience, which is great," says Tyler. "It's a super healthy dynamic."</p>
+
+<p>The platform also includes other CNCF technologies, such as Envoy, Helm, and Prometheus. Most of the critical service traffic for Booking.com is routed through Envoy, and Prometheus is used primarily to monitor infrastructure components. Helm is consumed as a packaging standard. The team also developed and open sourced Shipper, an extension for Kubernetes to add more complex rollout strategies and multi-cluster orchestration.</p>
+
+<p>To be sure, there have been internal discussions about the wisdom of building a Kubernetes platform from the ground up. "This is not really our core competency—Kubernetes and travel, they're kind of far apart, right?" says Tyler. "But we've made a couple of bets on CNCF components that have worked out really well for us. Envoy and Kubernetes, in particular, have been really beneficial to our organization. We were able to customize them, either because we could look at the source code or because they had extension points, and we were able to get value out of them very quickly without having to change any paradigms internally."</p>
@@ -3,101 +3,78 @@ title: Booz Allen Case Study
 linkTitle: Booz Allen Hamilton
 case_study_styles: true
 cid: caseStudies
-css: /css/case-studies-gradient.css
 logo: booz-allen-featured-logo.svg
 featured: true
 weight: 2
 quote: >
-  Kubernetes is a great solution for us. It allows us to rapidly iterate on our clients’ demands.
+  Kubernetes is a great solution for us. It allows us to rapidly iterate on our clients' demands.
+
+new_case_study_styles: true
+heading_background: /images/case-studies/booz-allen/banner4.jpg
+heading_title_text: Booz Allen Hamilton
+use_gradient_overlay: true
+subheading: >
+  How Booz Allen Hamilton Is Helping Modernize the Federal Government with Kubernetes
+case_study_details:
+  - Company: Booz Allen Hamilton
+  - Location: United States
+  - Industry: Government
 ---
-

+<h2>Challenge</h2>

-<div class="article image overlay" style="background-image: url('/images/case-studies/booz-allen/banner4.jpg')">
-  <h1> CASE STUDY: Booz Allen Hamilton</h1>
-  <div class="subhead">How Booz Allen Hamilton Is Helping Modernize the Federal Government with Kubernetes</div>
-</div>
-
-
-<div class="details">
-    Company &nbsp;<b>Booz Allen Hamilton</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location &nbsp;<b>United States</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Industry &nbsp;<b>Government</b>
-</div>
-
-<hr>
-<section class="section1">
-<div class="colsgradient">
-  <div class="col1">
-    <h2>Challenge</h2>
-    In 2017, Booz Allen Hamilton’s Strategic Innovation Group worked with the federal government to relaunch the decade-old recreation.gov website, which provides information and real-time booking for more than 100,000 campsites and facilities on federal lands across the country. The infrastructure needed to be agile, reliable, and scalable—as well as repeatable for the other federal agencies that are among Booz Allen Hamilton’s customers.
+<p>In 2017, Booz Allen Hamilton's Strategic Innovation Group worked with the federal government to relaunch the decade-old recreation.gov website, which provides information and real-time booking for more than 100,000 campsites and facilities on federal lands across the country. The infrastructure needed to be agile, reliable, and scalable—as well as repeatable for the other federal agencies that are among Booz Allen Hamilton's customers.</p>
+
+<h2>Solution</h2>
+
+<p>"The only way that we thought we could be successful with this problem across all the different agencies is to create a microservice architecture and containers, so that we could be very dynamic and very agile to any given agency for whatever requirements that they may have," says Booz Allen Hamilton Senior Lead Technologist Martin Folkoff. To meet those requirements, Folkoff's team looked to Kubernetes for orchestration.</p>

-    <h2>Solution</h2>
-    “The only way that we thought we could be successful with this problem across all the different agencies is to create a microservice architecture and containers, so that we could be very dynamic and very agile to any given agency for whatever requirements that they may have,” says Booz Allen Hamilton Senior Lead Technologist Martin Folkoff. To meet those requirements, Folkoff’s team looked to Kubernetes for orchestration.
-
 <h2>Impact</h2>
-    With the recreation.gov Kubernetes platform, changes can be implemented in about 30 minutes, compared to the multiple hours or even days legacy government applications require to review the code, get approval, and deploy the fix. Recreation.gov deploys to production on average 10 times a day. With monitoring, security, and logging built in, developers can create and publish new services to production within a week. Additionally, Folkoff says, “supporting the large, existing monoliths in the government is extremely expensive,” and migrating into a more modern platform has resulted in perhaps 50% cost savings.
-</div>
-</div>
-</section>
-<div class="article image overlay" style="background-image: url('/images/case-studies/booz-allen/banner2.jpg');">
-  <div class="quotetext">
-    "When there’s a regulatory change in an agency, or a legislative change in Congress, or an executive order that changes the way you do business, how do I deploy that and get that out to the people who need it rapidly? At the end of the day, that’s the problem we’re trying to help the government solve with tools like Kubernetes."
-<p><div class="quoteauthortext">- JOSH BOYD, CHIEF TECHNOLOGIST AT BOOZ ALLEN HAMILTON</div></p>
-  </div>
-</div>
-
-
-<section class="section2">
-<div class="fullcol">
-  <h4>The White House launched an IT modernization effort in 2017, and in addition to improving cybersecurity and shifting to the public cloud and a consolidated IT model, “the federal government is looking to provide a better experience to citizens in every way that we interact with the government through every channel,” says Booz Allen Hamilton Senior Lead Technologist Martin Folkoff.</h4>

-  <p>To that end, Folkoff’s Strategic Innovation Group worked with the federal government last year to relaunch the decade-old recreation.gov website, which provides information and real-time booking for more than 100,000 campsites and facilities on federal lands across the country.</p>
+<p>With the recreation.gov Kubernetes platform, changes can be implemented in about 30 minutes, compared to the multiple hours or even days legacy government applications require to review the code, get approval, and deploy the fix. Recreation.gov deploys to production on average 10 times a day. With monitoring, security, and logging built in, developers can create and publish new services to production within a week. Additionally, Folkoff says, "supporting the large, existing monoliths in the government is extremely expensive," and migrating into a more modern platform has resulted in perhaps 50% cost savings.</p>

-  <p>The infrastructure needed to be agile, reliable, and scalable—as well as repeatable for the other federal agencies that are among Booz Allen Hamilton’s customers. “The only way that we thought we could be successful with this problem across all the different agencies is to create a microservice architecture, so that we could be very dynamic and very agile to any given agency for whatever requirements that they may have,” says Folkoff.</p>
-</div>
-</section>
+{{< case-studies/quote
+  image="/images/case-studies/booz-allen/banner2.jpg"
+  author="JOSH BOYD, CHIEF TECHNOLOGIST AT BOOZ ALLEN HAMILTON"
+>}}
+"When there's a regulatory change in an agency, or a legislative change in Congress, or an executive order that changes the way you do business, how do I deploy that and get that out to the people who need it rapidly? At the end of the day, that's the problem we're trying to help the government solve with tools like Kubernetes."
+{{< /case-studies/quote >}}

-<div class="greybanner">
-  <div class="greyquotetext">
-     "With CNCF, there’s a lot of focus on scale, and so there’s a lot of comfort knowing that as the project grows, we’re going to be comfortable using that tool set."<p><div class="quoteauthortext">- MARTIN FOLKOFF, SENIOR LEAD TECHNOLOGIST AT BOOZ ALLEN HAMILTON</div></p>
-  </div>
-</div>
+{{< case-studies/lead >}}
+The White House launched an IT modernization effort in 2017, and in addition to improving cybersecurity and shifting to the public cloud and a consolidated IT model, "the federal government is looking to provide a better experience to citizens in every way that we interact with the government through every channel," says Booz Allen Hamilton Senior Lead Technologist Martin Folkoff.
+{{< /case-studies/lead >}}
+
+<p>To that end, Folkoff's Strategic Innovation Group worked with the federal government last year to relaunch the decade-old recreation.gov website, which provides information and real-time booking for more than 100,000 campsites and facilities on federal lands across the country.</p>
+
+<p>The infrastructure needed to be agile, reliable, and scalable—as well as repeatable for the other federal agencies that are among Booz Allen Hamilton's customers. "The only way that we thought we could be successful with this problem across all the different agencies is to create a microservice architecture, so that we could be very dynamic and very agile to any given agency for whatever requirements that they may have," says Folkoff.</p>
+
+{{< case-studies/quote author="MARTIN FOLKOFF, SENIOR LEAD TECHNOLOGIST AT BOOZ ALLEN HAMILTON" >}}
+"With CNCF, there's a lot of focus on scale, and so there's a lot of comfort knowing that as the project grows, we're going to be comfortable using that tool set."
+{{< /case-studies/quote >}}
+
+<p>Booz Allen Hamilton, which has provided consulting services to the federal government for more than a century, introduced microservices, Docker containers, and AWS to its federal agency clients about five years ago. The next logical step was Kubernetes for orchestration. "Knowing that we had to be really agile and really reliable and scalable, we felt that the only technology that we know that can enable those kinds of things are the ones the CNCF provides," Folkoff says. "One of the things that is always important for the government is to make sure that the things that we build really endure. Using technology that is supported across multiple different companies and has strong governance gives people a lot of confidence."</p>
+
+<p>Kubernetes was also aligned with the government's open source and IT modernization initiatives, so there has been an uptick in its usage at federal agencies over the past two years. "Now that Kubernetes is becoming offered as a service by the cloud providers like AWS and Microsoft, we're starting to see even more interest," says Chief Technologist Josh Boyd. Adds Folkoff: "With CNCF, there's a lot of focus on scale, and so there's a lot of comfort knowing that as the project grows, we're going to be comfortable using that tool set."</p>
+
+<p>The greenfield recreation.gov project allowed the team to build a new Kubernetes-enabled site running on AWS, and the migration lasted only a week, when the old site didn't take bookings. "For the actual transition, we just swapped a DNS server, and it only took about 35 seconds between the old site being down and our new site being up and available," Folkoff adds. </p>

-<section class="section3">
-<div class="fullcol">
-  <p>
-    Booz Allen Hamilton, which has provided consulting services to the federal government for more than a century, introduced microservices, Docker containers, and AWS to its federal agency clients about five years ago. The next logical step was Kubernetes for orchestration. “Knowing that we had to be really agile and really reliable and scalable, we felt that the only technology that we know that can enable those kinds of things are the ones the CNCF provides,” Folkoff says. “One of the things that is always important for the government is to make sure that the things that we build really endure. Using technology that is supported across multiple different companies and has strong governance gives people a lot of confidence.”</p>
-  <p>
-    Kubernetes was also aligned with the government’s open source and IT modernization initiatives, so there has been an uptick in its usage at federal agencies over the past two years. “Now that Kubernetes is becoming offered as a service by the cloud providers like AWS and Microsoft, we’re starting to see even more interest,” says Chief Technologist Josh Boyd. Adds Folkoff: “With CNCF, there’s a lot of focus on scale, and so there’s a lot of comfort knowing that as the project grows, we’re going to be comfortable using that tool set.” </p>
-  <p>
-    The greenfield recreation.gov project allowed the team to build a new Kubernetes-enabled site running on AWS, and the migration lasted only a week, when the old site didn’t take bookings. “For the actual transition, we just swapped a DNS server, and it only took about 35 seconds between the old site being down and our new site being up and available,” Folkoff adds. </p>
-  </p>
-</div>
-</section>
-
-
-  <div class="article image overlay" style="background-image: url('/images/case-studies/booz-allen/banner1.png');">
-  <div class="quotetext">
-  "Kubernetes alone enables a dramatic reduction in cost as resources are prioritized to the day’s event" <p><div class="quoteauthortext">- MARTIN FOLKOFF, SENIOR LEAD TECHNOLOGIST AT BOOZ ALLEN HAMILTON</div></p>
-  </div>
-</div>
-
-<section class="section5">
-<div class="fullcol">
-  <p>
-    In addition to its work with the Department of Interior for recreation.gov, Booz Allen Hamilton has brought Kubernetes to various Defense, Intelligence, and civilian agencies. Says Boyd: “When there’s a regulatory change in an agency, or a legislative change in Congress, or an executive order that changes the way you do business, how do I deploy that and get that out to the people who need it rapidly? At the end of the day, that’s the problem we’re trying to help the government solve with tools like Kubernetes.”</p>
-  <p>
-  For recreation.gov, the impact was clear and immediate. With the Kubernetes platform, Folkoff says, “if a new requirement for a permit comes out, we have the ability to design and develop and implement that completely independently of reserving a campsite. It provides a much better experience to users.” Today, changes can be implemented in about 30 minutes, compared to the multiple hours or even days legacy government applications require to review the code, get approval, and deploy the fix. Recreation.gov deploys to production on average 10 times a day. </p>
-  <p>
-  Developer velocity has been improved. “When I want to do monitoring or security or logging, I don’t have to do anything to my services or my application to enable that anymore,” says Boyd. “I get all of this magic just by being on the Kubernetes platform.” With all of those things built in, developers can create and publish new services to production within one week. </p>
-  <p>
-  Additionally, Folkoff says, “supporting the large, existing monoliths in the government is extremely expensive,” and migrating into a more modern platform has resulted in perhaps 50% cost savings. “Kubernetes alone enables a dramatic reduction in cost as resources are prioritized to the day’s event,” he says. “For example, during a popular campsite release, camping-related services are scaled out while permit services are scaled down.”</p>
-  <p>
-  So far, “Kubernetes is a great solution for us,” says Folkoff. “It allows us to rapidly iterate on our clients’ demands.” Looking ahead, the team sees further adoption of the Kubernetes platform across federal agencies. Says Boyd: “You get the ability for the rapid delivery of business value for your customers. You now have observability into everything that you’re doing. You don’t have these onesies and twosies unicorn servers anymore. Now everything that you deploy is deployed in the same way, it’s all instrumented the same way, and it’s all built and deployed the same way through our CI/CD processes.”</p>
-  <p>
-  They also see a push toward re-platforming. “There’s still a lot of legacy workloads out there,” says Boyd. “We’ve got the new challenges of greenfield development and integration with legacy systems, but also that brown field of ‘Hey, how do I take this legacy monolith and get it onto a platform where now it’s instrumented with all the magic of the Kubernetes platform without having to do a whole lot to my application?’ I think re-platforming is a pretty big use case for the government right now.”</p>
-  <p>
-  And given the success that they’ve had with Kubernetes so far, Boyd says, “I think at this point that technology is becoming pretty easy to sell.” Adds Folkoff: “People are really excited about being able to deploy, scale, be reliable, and do cheaper maintenance of all of this.”
-  </p>
-</div>
-</section>
-</body>
+{{< case-studies/quote
+  image="/images/case-studies/booz-allen/banner1.png"
+  author="MARTIN FOLKOFF, SENIOR LEAD TECHNOLOGIST AT BOOZ ALLEN HAMILTON"
+>}}
+"Kubernetes alone enables a dramatic reduction in cost as resources are prioritized to the day's event"
+{{< /case-studies/quote >}}
+
+<p>In addition to its work with the Department of Interior for recreation.gov, Booz Allen Hamilton has brought Kubernetes to various Defense, Intelligence, and civilian agencies. Says Boyd: "When there's a regulatory change in an agency, or a legislative change in Congress, or an executive order that changes the way you do business, how do I deploy that and get that out to the people who need it rapidly? At the end of the day, that's the problem we're trying to help the government solve with tools like Kubernetes."</p>
+
+<p>For recreation.gov, the impact was clear and immediate. With the Kubernetes platform, Folkoff says, "if a new requirement for a permit comes out, we have the ability to design and develop and implement that completely independently of reserving a campsite. It provides a much better experience to users." Today, changes can be implemented in about 30 minutes, compared to the multiple hours or even days legacy government applications require to review the code, get approval, and deploy the fix. Recreation.gov deploys to production on average 10 times a day.</p>
+
+<p>Developer velocity has been improved. "When I want to do monitoring or security or logging, I don't have to do anything to my services or my application to enable that anymore," says Boyd. "I get all of this magic just by being on the Kubernetes platform." With all of those things built in, developers can create and publish new services to production within one week. </p>
+
+<p>Additionally, Folkoff says, "supporting the large, existing monoliths in the government is extremely expensive," and migrating into a more modern platform has resulted in perhaps 50% cost savings. "Kubernetes alone enables a dramatic reduction in cost as resources are prioritized to the day's event," he says. "For example, during a popular campsite release, camping-related services are scaled out while permit services are scaled down."</p>
+
+<p>So far, "Kubernetes is a great solution for us," says Folkoff. "It allows us to rapidly iterate on our clients' demands." Looking ahead, the team sees further adoption of the Kubernetes platform across federal agencies. Says Boyd: "You get the ability for the rapid delivery of business value for your customers. You now have observability into everything that you're doing. You don't have these onesies and twosies unicorn servers anymore. Now everything that you deploy is deployed in the same way, it's all instrumented the same way, and it's all built and deployed the same way through our CI/CD processes."</p>
+
+<p>They also see a push toward re-platforming. "There's still a lot of legacy workloads out there," says Boyd. "We've got the new challenges of greenfield development and integration with legacy systems, but also that brown field of 'Hey, how do I take this legacy monolith and get it onto a platform where now it's instrumented with all the magic of the Kubernetes platform without having to do a whole lot to my application?' I think re-platforming is a pretty big use case for the government right now."</p>
+
+<p>And given the success that they've had with Kubernetes so far, Boyd says, "I think at this point that technology is becoming pretty easy to sell." Adds Folkoff: "People are really excited about being able to deploy, scale, be reliable, and do cheaper maintenance of all of this."</p>
@@ -3,101 +3,85 @@ title: Bose Case Study
 linkTitle: Bose
 case_study_styles: true
 cid: caseStudies
-css: /css/style_case_studies.css
 logo: bose_featured_logo.png
 featured: false
 weight: 2
 quote: >
-   The CNCF Landscape quickly explains what’s going on in all the different areas from storage to cloud providers to automation and so forth. This is our shopping cart to build a cloud infrastructure. We can go choose from the different aisles.
+   The CNCF Landscape quickly explains what's going on in all the different areas from storage to cloud providers to automation and so forth. This is our shopping cart to build a cloud infrastructure. We can go choose from the different aisles.
+
+new_case_study_styles: true
+heading_background: /images/case-studies/bose/banner1.jpg
+heading_title_logo: /images/bose_logo.png
+subheading: >
+  Bose: Supporting Rapid Development for Millions of IoT Products With Kubernetes
+case_study_details:
+  - Company: Bose Corporation
+  - Location: Framingham, Massachusetts
+  - Industry: Consumer Electronics
 ---

-<div class="banner1" style="background-image: url('/images/case-studies/bose/banner1.jpg')">
-  <h1> CASE STUDY:<img src="/images/bose_logo.png" class="header_logo" style="width:20%;margin-bottom:-1.2%"><br> <div class="subhead" style="margin-top:1%">Bose: Supporting Rapid Development for Millions of IoT Products With Kubernetes
+<h2>Challenge</h2>

-</div></h1>
+<p>A household name in high-quality audio equipment, <a href="https://www.bose.com/en_us/index.html">Bose</a> has offered connected products for more than five years, and as that demand grew, the infrastructure had to change to support it. "We needed to provide a mechanism for developers to rapidly prototype and deploy services all the way to production pretty fast," says Lead Cloud Engineer Josh West. In 2016, the company decided to start building a platform from scratch. The primary goal: "To be one to two steps ahead of the different product groups so that we are never scrambling to catch up with their scale," says Cloud Architecture Manager Dylan O'Mahony.</p>

-</div>
+<h2>Solution</h2>

-<div class="details">
-    Company &nbsp;<b>Bose Corporation</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location &nbsp;<b>Framingham, Massachusetts
-</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Industry &nbsp;<b>Consumer Electronics</b>
-</div>
+<p>From the beginning, the team knew it wanted a microservices architecture. After evaluating and prototyping a couple of orchestration solutions, the team decided to adopt <a href="https://kubernetes.io/">Kubernetes</a> for its scaled IoT Platform-as-a-Service running on AWS. The platform, which also incorporated Prometheus monitoring, launched in production in 2017, serving over 3 million connected products from the get-go. Bose has since adopted a number of other CNCF technologies, including <a href="https://www.fluentd.org/">Fluentd</a>, <a href="https://coredns.io/">CoreDNS</a>, <a href="https://www.jaegertracing.io/">Jaeger</a>, and <a href="https://opentracing.io/">OpenTracing</a>.</p>

-<hr>
-<section class="section1">
-<div class="cols">
-  <div class="col1" style="width:100%"">
-    <h2>Challenge</h2>
-    A household name in high-quality audio equipment, <a href="https://www.bose.com/en_us/index.html">Bose</a> has offered connected products for more than five years, and as that demand grew, the infrastructure had to change to support it. "We needed to provide a mechanism for developers to rapidly prototype and deploy services all the way to production pretty fast,” says Lead Cloud Engineer Josh West. In 2016, the company decided to start building a platform from scratch. The primary goal: "To be one to two steps ahead of the different product groups so that we are never scrambling to catch up with their scale,” says Cloud Architecture Manager Dylan O’Mahony.
-<br><br>
-    <h2>Solution</h2>
-    From the beginning, the team knew it wanted a microservices architecture. After evaluating and prototyping a couple of orchestration solutions, the team decided to adopt <a href="https://kubernetes.io/">Kubernetes</a> for its scaled IoT Platform-as-a-Service running on AWS. The platform, which also incorporated Prometheus monitoring, launched in production in 2017, serving over 3 million connected products from the get-go. Bose has since adopted a number of other CNCF technologies, including <a href="https://www.fluentd.org/">Fluentd</a>, <a href="https://coredns.io/">CoreDNS</a>, <a href="https://www.jaegertracing.io/">Jaeger</a>, and <a href="https://opentracing.io/">OpenTracing</a>.
-<br><br>
 <h2>Impact</h2>
-    With about 100 engineers onboarded, the platform is now enabling 30,000 non-production deployments across dozens of microservices per year. In 2018, there were 1250+ production deployments. Just one production cluster holds 1,800 namespaces and 340 worker nodes. "We had a brand new service taken from concept through coding and deployment all the way to production, including hardening, security testing and so forth, in less than two and a half weeks,” says O’Mahony.
-</div>

-</div>
-</section>
-<div class="banner2">
-  <div class="banner2text">
-    "At Bose we’re building an IoT platform that has enabled our physical products. If it weren’t for Kubernetes and the rest of the CNCF projects being free open source software with such a strong community, we would never have achieved scale, or even gotten to launch&nbsp;on&nbsp;schedule."
-<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Josh West, Lead Cloud Engineer, Bose</span>
-  </div>
-</div>
-<section class="section2">
-<div class="fullcol">
-  <h2>A household name in high-quality audio equipment, <a href="https://www.bose.com/en_us/index.html">Bose</a> has offered connected products for more than five years, and as that demand grew, the infrastructure had to change to support it. </h2>
-  "We needed to provide a mechanism for developers to rapidly prototype and deploy services all the way to production pretty fast,” says Lead Cloud Engineer Josh West. "There were a lot of cloud capabilities we wanted to provide to support our audio equipment and experiences.”<br><br>
-In 2016, the company decided to start building an IoT platform from scratch. The primary goal: "To be one to two steps ahead of the different product groups so that we are never scrambling to catch up with their scale,” says Cloud Architecture Manager Dylan O’Mahony. "If they release a new connected product, we want to be already well ahead of being able to handle whatever scale that they’re going to throw at us.”<br><br>
-From the beginning, the team knew it wanted a microservices architecture and platform as a service. After evaluating and prototyping orchestration solutions, including Mesos and Docker Swarm, the team decided to adopt <a href="https://kubernetes.io/">Kubernetes</a> for its platform running on AWS. Kubernetes was still in 1.5, but already the technology could do much of what the team wanted and needed for the present and the future. For West, that meant having storage and network handled. O’Mahony points to Kubernetes’ portability in case Bose decides to go multi-cloud.<br><br>
-"Bose is a company that looks out for the long term,” says West. "Going with a quick commercial off-the-shelf solution might’ve worked for that point in time, but it would not have carried us forward, which is what we needed from Kubernetes and the CNCF.”
+<p>With about 100 engineers onboarded, the platform is now enabling 30,000 non-production deployments across dozens of microservices per year. In 2018, there were 1250+ production deployments. Just one production cluster holds 1,800 namespaces and 340 worker nodes. "We had a brand new service taken from concept through coding and deployment all the way to production, including hardening, security testing and so forth, in less than two and a half weeks," says O'Mahony.</p>

+{{< case-studies/quote author="Josh West, Lead Cloud Engineer, Bose" >}}
+"At Bose we're building an IoT platform that has enabled our physical products. If it weren't for Kubernetes and the rest of the CNCF projects being free open source software with such a strong community, we would never have achieved scale, or even gotten to launch on schedule."
+{{< /case-studies/quote >}}

-</div>
-</section>
-<div class="banner3" style="background-image: url('/images/case-studies/bose/banner3.jpg')">
-  <div class="banner3text">
-    "Everybody on the team thinks in terms of automation, leaning out the processes, getting things done as quickly as possible. When you step back and look at what it means for a 50-plus-year-old speaker company to have that sort of culture, it really is quite incredible, and I think the tools that we use and the foundation that we’ve built with them is a huge piece of that."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Dylan O’Mahony, Cloud Architecture Manager, Bose</span>
+{{< case-studies/lead >}}
+A household name in high-quality audio equipment, <a href="https://www.bose.com/en_us/index.html">Bose</a> has offered connected products for more than five years, and as that demand grew, the infrastructure had to change to support it. 
+{{< /case-studies/lead >}}

-  </div>
-</div>
-<section class="section3">
-<div class="fullcol">
-  The team spent time working on choosing tooling to make the experience easier for developers. "Our developers interact with tools provided by our Ops team, and the Ops team run all of their tooling on top of Kubernetes,” says O’Mahony. "We try not to make direct Kubernetes access the only way. In fact, ideally, our developers wouldn’t even need to know that they’re running on Kubernetes.”<br><br>
-  The platform, which also incorporated <a href="https://prometheus.io/">Prometheus</a> monitoring from the beginning, backdoored its way into production in 2017, serving over 3 million connected products from the get-go. "Even though the speakers and the products that we were designing this platform for were still quite a ways away from being launched, we did have some connected speakers on the market,” says O’Mahony. "We basically started to point certain features of those speakers and the apps that go with those speakers to this platform.”<br><br>
-  Today, just one of Bose’s production clusters holds 1,800 namespaces/discrete services and 340 nodes. With about 100 engineers now onboarded, the platform infrastructure is now enabling 30,000 non-production deployments across dozens of microservices per year. In 2018, there were 1250+ production deployments.. It’s a staggering improvement over some of Bose’s previous deployment processes, which supported far fewer deployments and services.
+<p>"We needed to provide a mechanism for developers to rapidly prototype and deploy services all the way to production pretty fast," says Lead Cloud Engineer Josh West. "There were a lot of cloud capabilities we wanted to provide to support our audio equipment and experiences."</p>

-</div>
-</section>
-<div class="banner4" style="background-image: url('/images/case-studies/bose/banner4.jpg');width:100%">
-  <div class="banner4text">
-  "The CNCF Landscape quickly explains what’s going on in all the different areas from storage to cloud providers to automation and so forth. This is our shopping cart to build a cloud infrastructure. We can go choose from the different aisles." <br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Josh West, Lead Cloud Engineer, Bose</span>
-  </div>
-</div>
+<p>In 2016, the company decided to start building an IoT platform from scratch. The primary goal: "To be one to two steps ahead of the different product groups so that we are never scrambling to catch up with their scale," says Cloud Architecture Manager Dylan O'Mahony. "If they release a new connected product, we want to be already well ahead of being able to handle whatever scale that they're going to throw at us."</p>

-<section class="section5" style="padding:0px !important">
-<div class="fullcol">
-  "We had a brand new service deployed from concept through coding and deployment all the way to production, including hardening, security testing and so forth, in less than two and a half weeks,” says O’Mahony. "Everybody thinks in terms of automation, leaning out the processes, getting things done as quickly as possible. When you step back and look at what it means for a 50-plus-year-old speaker company to have that sort of culture, it really is quite incredible, and I think the tools that we use and the foundation that we’ve built is a huge piece of that.”<br><br>
-  Many of those technologies—such as <a href="https://www.fluentd.org/">Fluentd</a>, <a href="https://coredns.io/">CoreDNS</a>, <a href="https://www.jaegertracing.io/">Jaeger</a>, and <a href="https://opentracing.io/">OpenTracing</a>—come from the <a href="https://landscape.cncf.io/">CNCF Landscape</a>, which West and O’Mahony have relied upon throughout Bose’s cloud native journey. "The CNCF Landscape quickly explains what’s going on in all the different areas from storage to cloud providers to automation and so forth,” says West. "This is our shopping cart to build a cloud infrastructure. We can go choose from the different aisles.” <br><br>
-  And, he adds, "If it weren’t for Kubernetes and the rest of the CNCF projects being free open source software with such a strong community, we would never have achieved scale, or even gotten to launch on schedule.”<br><br>
-  Another benefit of going cloud native: "We are even attracting much more talent into Bose because we’re so involved with the <a href="http://careers.bose.com">CNCF Landscape</a>,” says West. (Yes, they’re hiring.) "It’s just enabled so many people to do so many great things and really brought Bose into the future of cloud.”
+<p>From the beginning, the team knew it wanted a microservices architecture and platform as a service. After evaluating and prototyping orchestration solutions, including Mesos and Docker Swarm, the team decided to adopt <a href="https://kubernetes.io/">Kubernetes</a> for its platform running on AWS. Kubernetes was still in 1.5, but already the technology could do much of what the team wanted and needed for the present and the future. For West, that meant having storage and network handled. O'Mahony points to Kubernetes' portability in case Bose decides to go multi-cloud.</p>

+<p>"Bose is a company that looks out for the long term," says West. "Going with a quick commercial off-the-shelf solution might've worked for that point in time, but it would not have carried us forward, which is what we needed from Kubernetes and the CNCF."</p>

-</div>
+{{< case-studies/quote
+  image="/images/case-studies/bose/banner3.jpg"
+  author="Dylan O'Mahony, Cloud Architecture Manager, Bose"
+>}}
+"Everybody on the team thinks in terms of automation, leaning out the processes, getting things done as quickly as possible. When you step back and look at what it means for a 50-plus-year-old speaker company to have that sort of culture, it really is quite incredible, and I think the tools that we use and the foundation that we've built with them is a huge piece of that."
+{{< /case-studies/quote >}}

-<div class="banner5" >
-  <div class="banner5text">
-"We have a lot going on to support many more of our business units at Bose in addition to the consumer electronics division, which we currently do. It’s only because of the cloud native landscape and the tools and the features that are available that we can provide such a fantastic cloud platform for all the developers and divisions that are trying to enable some pretty amazing experiences."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Dylan O’Mahony, Cloud Architecture Manager, Bose</span></div>
-</div>
+<p>The team spent time working on choosing tooling to make the experience easier for developers. "Our developers interact with tools provided by our Ops team, and the Ops team run all of their tooling on top of Kubernetes," says O'Mahony. "We try not to make direct Kubernetes access the only way. In fact, ideally, our developers wouldn't even need to know that they're running on Kubernetes."</p>

-<div class="fullcol">
-  In the coming year, the team wants to work on service mesh and serverless, as well as expansion around the world. "Getting our latency down by going multi-region is going to be a big focus for us,” says O’Mahony. "In order to make sure that our customers in Japan, Australia, and everywhere else are having a good experience, we want to have points of presence closer to them. It’s never been done at Bose before.”<br><br>
-  That won’t stop them, because the team is all about lofty goals. "We want to get to billions of connected products!” says West. "We have a lot going on to support many more of our business units at Bose in addition to the consumer electronics division, which we currently do. It’s only because of the cloud native landscape and the tools and the features that are available that we can provide such a fantastic cloud platform for all the developers and divisions that are trying to enable some pretty amazing experiences.”<br><br>
-  In fact, given the scale the platform is already supporting, says O’Mahony, "doing anything other than Kubernetes, I think, would be folly at this point.”
+<p>The platform, which also incorporated <a href="https://prometheus.io/">Prometheus</a> monitoring from the beginning, backdoored its way into production in 2017, serving over 3 million connected products from the get-go. "Even though the speakers and the products that we were designing this platform for were still quite a ways away from being launched, we did have some connected speakers on the market," says O'Mahony. "We basically started to point certain features of those speakers and the apps that go with those speakers to this platform."</p>

+<p>Today, just one of Bose's production clusters holds 1,800 namespaces/discrete services and 340 nodes. With about 100 engineers now onboarded, the platform infrastructure is now enabling 30,000 non-production deployments across dozens of microservices per year. In 2018, there were 1250+ production deployments.. It's a staggering improvement over some of Bose's previous deployment processes, which supported far fewer deployments and services.</p>

-</div>
-</section>
-</body>
-</html>
+{{< case-studies/quote
+  image="/images/case-studies/bose/banner4.jpg"
+  author="Josh West, Lead Cloud Engineer, Bose"
+>}}
+"The CNCF Landscape quickly explains what's going on in all the different areas from storage to cloud providers to automation and so forth. This is our shopping cart to build a cloud infrastructure. We can go choose from the different aisles."
+{{< /case-studies/quote >}}
+
+<p>"We had a brand new service deployed from concept through coding and deployment all the way to production, including hardening, security testing and so forth, in less than two and a half weeks," says O'Mahony. "Everybody thinks in terms of automation, leaning out the processes, getting things done as quickly as possible. When you step back and look at what it means for a 50-plus-year-old speaker company to have that sort of culture, it really is quite incredible, and I think the tools that we use and the foundation that we've built is a huge piece of that."</p>
+
+<p>Many of those technologies—such as <a href="https://www.fluentd.org/">Fluentd</a>, <a href="https://coredns.io/">CoreDNS</a>, <a href="https://www.jaegertracing.io/">Jaeger</a>, and <a href="https://opentracing.io/">OpenTracing</a>—come from the <a href="https://landscape.cncf.io/">CNCF Landscape</a>, which West and O'Mahony have relied upon throughout Bose's cloud native journey. "The CNCF Landscape quickly explains what's going on in all the different areas from storage to cloud providers to automation and so forth," says West. "This is our shopping cart to build a cloud infrastructure. We can go choose from the different aisles."</p>
+
+<p>And, he adds, "If it weren't for Kubernetes and the rest of the CNCF projects being free open source software with such a strong community, we would never have achieved scale, or even gotten to launch on schedule."</p>
+
+<p>Another benefit of going cloud native: "We are even attracting much more talent into Bose because we're so involved with the <a href="http://careers.bose.com">CNCF Landscape</a>," says West. (Yes, they're hiring.) "It's just enabled so many people to do so many great things and really brought Bose into the future of cloud."</p>
+
+{{< case-studies/quote author="Dylan O'Mahony, Cloud Architecture Manager, Bose" >}}
+"We have a lot going on to support many more of our business units at Bose in addition to the consumer electronics division, which we currently do. It's only because of the cloud native landscape and the tools and the features that are available that we can provide such a fantastic cloud platform for all the developers and divisions that are trying to enable some pretty amazing experiences."
+{{< /case-studies/quote >}}
+
+<p>In the coming year, the team wants to work on service mesh and serverless, as well as expansion around the world. "Getting our latency down by going multi-region is going to be a big focus for us," says O'Mahony. "In order to make sure that our customers in Japan, Australia, and everywhere else are having a good experience, we want to have points of presence closer to them. It's never been done at Bose before."</p>
+
+<p>That won't stop them, because the team is all about lofty goals. "We want to get to billions of connected products!" says West. "We have a lot going on to support many more of our business units at Bose in addition to the consumer electronics division, which we currently do. It's only because of the cloud native landscape and the tools and the features that are available that we can provide such a fantastic cloud platform for all the developers and divisions that are trying to enable some pretty amazing experiences."</p>
+
+<p>In fact, given the scale the platform is already supporting, says O'Mahony, "doing anything other than Kubernetes, I think, would be folly at this point."</p>
@@ -2,113 +2,99 @@
 title: Box Case Study
 case_study_styles: true
 cid: caseStudies
-css: /css/style_box.css
 video: https://www.youtube.com/embed/of45hYbkIZs?autoplay=1
 quote: >
  Kubernetes has the opportunity to be the new cloud platform. The amount of innovation that's going to come from being able to standardize on Kubernetes as a platform is incredibly exciting - more exciting than anything I've seen in the last 10 years of working on the cloud.

+new_case_study_styles: true
+heading_background: /images/case-studies/box/banner1.jpg
+heading_title_logo: /images/box_logo.png
+subheading: >
+  An Early Adopter Envisions a New Cloud Platform
+case_study_details:
+  - Company: Box
+  - Location: Redwood City, California
+  - Industry: Technology
 ---

-<div class="banner1">
-  <h1>CASE STUDY: <img src="/images/box_logo.png" width="10%" style="margin-bottom:-6px"><br>
-    <div class="subhead">An Early Adopter Envisions
-      a New Cloud Platform</div>
-  </h1>
-</div>
+<h2>Challenge</h2>

+<p>Founded in 2005, the enterprise content management company allows its more than 50 million users to manage content in the cloud. <a href="https://www.box.com/home">Box</a> was built primarily with bare metal inside the company's own data centers, with a monolithic PHP code base. As the company was expanding globally, it needed to focus on "how we run our workload across many different cloud infrastructures from bare metal to public cloud," says Sam Ghods, Cofounder and Services Architect of Box. "It's been a huge challenge because of different clouds, especially bare metal, have very different interfaces."</p>

-<div class="details">
-    Company &nbsp;<b>Box</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location &nbsp;<b>Redwood City, California</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Industry &nbsp;<b>Technology</b>
-</div>
+<h2>Solution</h2>

-<hr>
+<p>Over the past couple of years, Box has been decomposing its infrastructure into microservices, and became an early adopter of, as well as contributor to, <a href="http://kubernetes.io/">Kubernetes</a> container orchestration. Kubernetes, Ghods says, has allowed Box's developers to "target a universal set of concepts that are portable across all clouds."</p>

-<section class="section1">
+<h2>Impact</h2>

-<div class="cols">
-  <div class="col1">
+<p>"Before Kubernetes," Ghods says, "our infrastructure was so antiquated it was taking us more than six months to deploy a new microservice. Today, a new microservice takes less than five days to deploy. And we're working on getting it to an hour."</p>

-      <h2>Challenge</h2>
-        Founded in 2005, the enterprise content management company allows its more than 50 million users to manage content in the cloud. <a href="https://www.box.com/home">Box</a> was built primarily with bare metal inside the company’s own data centers, with a monolithic PHP code base. As the company was expanding globally, it needed to focus on "how we run our workload across many different cloud infrastructures from bare metal to public cloud," says Sam Ghods, Cofounder and Services Architect of Box. "It’s been a huge challenge because of different clouds, especially bare metal, have very different interfaces."
-        <br>
-  </div>
+{{< case-studies/quote author="SAM GHOUDS, CO-FOUNDER AND SERVICES ARCHITECT OF BOX" >}}
+"We looked at a lot of different options, but Kubernetes really stood out....the fact that on day one it was designed to run on bare metal just as well as Google Cloud meant that we could actually migrate to it inside of our data centers, and then use those same tools and concepts to run across public cloud providers as&nbsp;well."
+{{< /case-studies/quote >}}

-  <div class="col2">
-    <h2>Solution</h2>
-      Over the past couple of years, Box has been decomposing its infrastructure into microservices, and became an early adopter of, as well as contributor to, <a href="http://kubernetes.io/">Kubernetes</a> container orchestration. Kubernetes, Ghods says, has allowed Box’s developers to "target a universal set of concepts that are portable across all clouds."<br><br>
+{{< case-studies/lead >}}
+In the summer of 2014, Box was feeling the pain of a decade's worth of hardware and software infrastructure that wasn't keeping up with the company's needs.
+{{< /case-studies/lead >}}

-    <h2>Impact</h2>
-      "Before Kubernetes," Ghods says, "our infrastructure was so antiquated it was taking us more than six months to deploy a new microservice. Today, a new microservice takes less than five days to deploy. And we’re working on getting it to an hour."
-  </div>
-</div>
+<p>A platform that allows its more than 50 million users (including governments and big businesses like <a href="https://www.ge.com/">General Electric</a>) to manage and share content in the cloud, Box was originally a <a href="http://php.net/">PHP</a> monolith of millions of lines of code built exclusively with bare metal inside of its own data centers. It had already begun to slowly chip away at the monolith, decomposing it into microservices. And "as we've been expanding into regions around the globe, and as the public cloud wars have been heating up, we've been focusing a lot more on figuring out how we run our workload across many different environments and many different cloud infrastructure providers," says Box Cofounder and Services Architect Sam Ghods. "It's been a huge challenge thus far because of all these different providers, especially bare metal, have very different interfaces and ways in which you work with them."</p>

-</section>
+<p>Box's cloud native journey accelerated that June, when Ghods attended <a href="https://www.docker.com/events/dockercon">DockerCon</a>. The company had come to the realization that it could no longer run its applications only off bare metal, and was researching containerizing with Docker, virtualizing with OpenStack, and supporting public cloud.</p>

-<div class="banner2">
-  <div class="banner2text">
-      "We looked at a lot of different options, but Kubernetes really stood out....the fact that on day one it was designed to run on bare metal just as well as Google Cloud meant that we could actually migrate to it inside of our data centers, and then use those same tools and concepts to run across public cloud providers as&nbsp;well."<br><br><span style="font-size:15px;letter-spacing:0.08em">- SAM GHOUDS, CO-FOUNDER AND SERVICES ARCHITECT OF BOX</span>
-  </div>
-</div>
+<p>At that conference, Google announced the release of its Kubernetes container management system, and Ghods was won over. "We looked at a lot of different options, but Kubernetes really stood out, especially because of the incredibly strong team of <a href="https://research.google.com/pubs/pub43438.html">Borg</a> veterans and the vision of having a completely infrastructure-agnostic way of being able to run cloud software," he says, referencing Google's internal container orchestrator Borg. "The fact that on day one it was designed to run on bare metal just as well as <a href="https://cloud.google.com/">Google Cloud</a> meant that we could actually migrate to it inside of our data centers, and then use those same tools and concepts to run across public cloud providers as well."</p>

-<section class="section2">
+<p>Another plus: Ghods liked that <a href="https://kubernetes.io/">Kubernetes</a> has a universal set of API objects like pod, service, replica set and deployment object, which created a consistent surface to build tooling against. "Even PaaS layers like <a href="https://www.openshift.com/">OpenShift</a> or <a href="http://deis.io/">Deis</a> that build on top of Kubernetes still treat those objects as first-class principles," he says. "We were excited about having these abstractions shared across the entire ecosystem, which would result in a lot more momentum than we saw in other potential solutions."</p>

-  <div class="fullcol">
-      <h2>In the summer of 2014, Box was feeling the pain of a decade’s worth of hardware and software infrastructure that wasn’t keeping up with the company’s needs.</h2>
+<p>Box deployed Kubernetes in a cluster in a production data center just six months later. Kubernetes was then still pre-beta, on version 0.11. They started small: The very first thing Ghods's team ran on Kubernetes was a Box API checker that confirms Box is up. "That was just to write and deploy some software to get the whole pipeline functioning," he says. Next came some daemons that process jobs, which was "nice and safe because if they experienced any interruptions, we wouldn't fail synchronous incoming requests from customers."</p>

-      A platform that allows its more than 50 million users (including governments and big businesses like <a href="https://www.ge.com/">General Electric</a>) to manage and share content in the cloud, Box was originally a <a href="http://php.net/">PHP</a> monolith of millions of lines of code built exclusively with bare metal inside of its own data centers. It had already begun to slowly chip away at the monolith, decomposing it into microservices. And "as we’ve been expanding into regions around the globe, and as the public cloud wars have been heating up, we’ve been focusing a lot more on figuring out how we run our workload across many different environments and many different cloud infrastructure providers," says Box Cofounder and Services Architect Sam Ghods. "It’s been a huge challenge thus far because of all these different providers, especially bare metal, have very different interfaces and ways in which you work with them."<br><br>
-      Box’s cloud native journey accelerated that June, when Ghods attended <a href="https://www.docker.com/events/dockercon">DockerCon</a>. The company had come to the realization that it could no longer run its applications only off bare metal, and was researching containerizing with Docker, virtualizing with OpenStack, and supporting public cloud.<br><br>
-      At that conference, Google announced the release of its Kubernetes container management system, and Ghods was won over. "We looked at a lot of different options, but Kubernetes really stood out, especially because of the incredibly strong team of <a href="https://research.google.com/pubs/pub43438.html">Borg</a> veterans and the vision of having a completely infrastructure-agnostic way of being able to run cloud software," he says, referencing Google’s internal container orchestrator Borg. "The fact that on day one it was designed to run on bare metal just as well as <a href="https://cloud.google.com/">Google Cloud</a> meant that we could actually migrate to it inside of our data centers, and then use those same tools and concepts to run across public cloud providers as well."<br><br>
-      Another plus: Ghods liked that <a href="https://kubernetes.io/">Kubernetes</a> has a universal set of API objects like pod, service, replica set and deployment object, which created a consistent surface to build tooling against. "Even PaaS layers like <a href="https://www.openshift.com/">OpenShift</a> or <a href="http://deis.io/">Deis</a> that build on top of Kubernetes still treat those objects as first-class principles," he says. "We were excited about having these abstractions shared across the entire ecosystem, which would result in a lot more momentum than we saw in other potential solutions."<br><br>
-      Box deployed Kubernetes in a cluster in a production data center just six months later. Kubernetes was then still pre-beta, on version 0.11. They started small: The very first thing Ghods’s team ran on Kubernetes was a Box API checker that confirms Box is up. "That was just to write and deploy some software to get the whole pipeline functioning," he says. Next came some daemons that process jobs, which was "nice and safe because if they experienced any interruptions, we wouldn’t fail synchronous incoming requests from customers."
+{{< case-studies/quote image="/images/case-studies/box/banner3.jpg">}}
+"As we've been expanding into regions around the globe, and as the public cloud wars have been heating up, we've been focusing a lot more on figuring out how we [can have Kubernetes help] run our workload across many different environments and many different cloud infrastructure providers."
+{{< /case-studies/quote >}}

-  </div>
-</section>
+<p>The first live service, which the team could route to and ask for information, was launched a few months later. At that point, Ghods says, "We were comfortable with the stability of the Kubernetes cluster. We started to port some services over, then we would increase the cluster size and port a few more, and that's ended up to about 100 servers in each data center that are dedicated purely to Kubernetes. And that's going to be expanding a lot over the next 12 months, probably too many hundreds if not thousands."</p>

-<div class="banner3">
-  <div class="banner3text">
-    "As we’ve been expanding into regions around the globe, and as the public cloud wars have been heating up, we’ve been focusing a lot more on figuring out how we [can have Kubernetes help] run our workload across many different environments and many different cloud infrastructure providers."
-  </div>
-</div>
+<p>While observing teams who began to use Kubernetes for their microservices, "we immediately saw an uptick in the number of microservices being released," Ghods&nbsp;notes. "There was clearly a pent-up demand for a better way of building software through microservices, and the increase in agility helped our developers be more productive and make better architectural choices."</p>

-<section class="section3">
-  <div class="fullcol">
-    The first live service, which the team could route to and ask for information, was launched a few months later. At that point, Ghods says, "We were comfortable with the stability of the Kubernetes cluster. We started to port some services over, then we would increase the cluster size and port a few more, and that’s ended up to about 100 servers in each data center that are dedicated purely to Kubernetes. And that’s going to be expanding a lot over the next 12 months, probably too many hundreds if not thousands."<br><br>
-    While observing teams who began to use Kubernetes for their microservices, "we immediately saw an uptick in the number of microservices being released," Ghods&nbsp;notes. "There was clearly a pent-up demand for a better way of building software through microservices, and the increase in agility helped our developers be more productive and make better architectural choices."
-    <br><br><div class="quote">"There was clearly a pent-up demand for a better way of building software through microservices, and the increase in agility helped our developers be more productive and make better architectural choices."</div><br>
-    Ghods reflects that as early adopters, Box had a different journey from what companies experience now. "We were definitely lock step with waiting for certain things to stabilize or features to get released," he says. "In the early days we were doing a lot of contributions [to components such as kubectl apply] and waiting for Kubernetes to release each of them, and then we’d upgrade, contribute more, and go back and forth several times. The entire project took about 18 months from our first real deployment on Kubernetes to having general availability. If we did that exact same thing today, it would probably be no more than six."<br><br>
-    In any case, Box didn’t have to make too many modifications to Kubernetes for it to work for the company. "The vast majority of the work our team has done to implement Kubernetes at Box has been making it work inside of our existing (and often legacy) infrastructure," says Ghods, "such as upgrading our base operating system from RHEL6 to RHEL7 or integrating it into <a href="https://www.nagios.org/">Nagios</a>, our monitoring infrastructure. But overall Kubernetes has been remarkably flexible with fitting into many of our constraints, and we’ve been running it very successfully on our bare metal infrastructure."<br><br>
-    Perhaps the bigger challenge for Box was a cultural one. "Kubernetes, and cloud native in general, represents a pretty big paradigm shift, and it’s not very incremental," Ghods says. "We’re essentially making this pitch that Kubernetes is going to solve everything because it does things the right way and everything is just suddenly better. But it’s important to keep in mind that it’s not nearly as proven as many other solutions out there. You can’t say how long this or that company took to do it because there just aren’t that many yet. Our team had to really fight for resources because our project was a bit of a moonshot."
-  </div>
-</section>
+{{< case-studies/lead >}}
+"There was clearly a pent-up demand for a better way of building software through microservices, and the increase in agility helped our developers be more productive and make better architectural choices."
+{{< /case-studies/lead >}}

-  <div class="banner4">
-    <div class="banner4text">
-    "The vast majority of the work our team has done to implement Kubernetes at Box has been making it work inside of our existing [and often legacy] infrastructure....overall Kubernetes has been remarkably flexible with fitting into many of our constraints, and we’ve been running it very successfully on our bare metal infrastructure."
-    </div>
-  </div>
+<p>Ghods reflects that as early adopters, Box had a different journey from what companies experience now. "We were definitely lock step with waiting for certain things to stabilize or features to get released," he says. "In the early days we were doing a lot of contributions [to components such as kubectl apply] and waiting for Kubernetes to release each of them, and then we'd upgrade, contribute more, and go back and forth several times. The entire project took about 18 months from our first real deployment on Kubernetes to having general availability. If we did that exact same thing today, it would probably be no more than six."</p>

-<section class="section4">
-  <div class="fullcol">
-    Having learned from experience, Ghods offers these two pieces of advice for companies going through similar challenges:
-    <h2>1. Deliver early and often.</h2> Service discovery was a huge problem for Box, and the team had to decide whether to build an interim solution or wait for Kubernetes to natively satisfy Box’s unique requirements. After much debate, "we just started focusing on delivering something that works, and then dealing with potentially migrating to a more native solution later," Ghods says. "The above-all-else target for the team should always be to serve real production use cases on the infrastructure, no matter how trivial. This helps keep the momentum going both for the team itself and for the organizational perception of the project." </br></br>
-    <h2>2. Keep an open mind about what your company has to abstract away from developers and what it&nbsp;doesn’t.</h2> Early on, the team built an abstraction on top of Docker files to help ensure that images had the right security updates.
-    This turned out to be superfluous work, since container images are considered immutable and you can easily scan them post-build to ensure they do not contain vulnerabilities. Because managing infrastructure through containerization is such a discontinuous leap, it’s better to start by interacting directly with the native tools and learning their unique advantages and caveats. An abstraction should be built only after a practical need for it arises.</br></br>
-    In the end, the impact has been powerful. "Before Kubernetes," Ghods says, "our infrastructure was so antiquated it was taking us more than six months to deploy a new microservice. Now a new microservice takes less than five days to deploy. And we’re working on getting it to an hour. Granted, much of that six months was due to how broken our systems were, but bare metal is intrinsically a difficult platform to support unless you have a system like Kubernetes to help manage&nbsp;it."</br></br>
-    By Ghods’s estimate, Box is still several years away from his goal of being a 90-plus percent Kubernetes shop. "We’re very far along on having a mission-critical, stable Kubernetes deployment that provides a lot of value," he says. "Right now about five percent of all of our compute runs on Kubernetes, and I think in the next six months we’ll likely be between 20 to 50 percent. We’re working hard on enabling all stateless service use cases, and shift our focus to stateful services after&nbsp;that."
-  </div>
-</section>
+<p>In any case, Box didn't have to make too many modifications to Kubernetes for it to work for the company. "The vast majority of the work our team has done to implement Kubernetes at Box has been making it work inside of our existing (and often legacy) infrastructure," says Ghods, "such as upgrading our base operating system from RHEL6 to RHEL7 or integrating it into <a href="https://www.nagios.org/">Nagios</a>, our monitoring infrastructure. But overall Kubernetes has been remarkably flexible with fitting into many of our constraints, and we've been running it very successfully on our bare metal infrastructure."</p>

-  <div class="banner5">
-    <div class="banner5text">
-    "Ghods predicts that Kubernetes has the opportunity to be the new cloud platform. '...because it’s a never-before-seen level of automation and intelligence surrounding infrastructure that is portable and agnostic to every way you can run your infrastructure.'"
-    </div>
-  </div>
+<p>Perhaps the bigger challenge for Box was a cultural one. "Kubernetes, and cloud native in general, represents a pretty big paradigm shift, and it's not very incremental," Ghods says. "We're essentially making this pitch that Kubernetes is going to solve everything because it does things the right way and everything is just suddenly better. But it's important to keep in mind that it's not nearly as proven as many other solutions out there. You can't say how long this or that company took to do it because there just aren't that many yet. Our team had to really fight for resources because our project was a bit of a moonshot."</p>

-<section class="section5">
-  <div class="fullcol">
-    In fact, that’s what he envisions across the industry: Ghods predicts that Kubernetes has the opportunity to be the new cloud platform. Kubernetes provides an API consistent across different cloud platforms including bare metal, and "I don’t think people have seen the full potential of what’s possible when you can program against one single interface," he says. "The same way <a href="https://aws.amazon.com/">AWS</a> changed infrastructure so that you don’t have to think about servers or cabinets or networking equipment anymore, Kubernetes enables you to focus exclusively on the containers that you’re running, which is pretty exciting. That’s the vision."</br></br>
-    Ghods points to projects that are already in development or recently released for Kubernetes as a cloud platform: cluster federation, the Dashboard UI, and <a href="https://coreos.com/">CoreOS</a>’s etcd operator. "I honestly believe it’s the most exciting thing I’ve seen in cloud infrastructure," he says, "because it’s a never-before-seen level of automation and intelligence surrounding infrastructure that is portable and agnostic to every way you can run your infrastructure."</br></br>
-    Box, with its early decision to use bare metal, embarked on its Kubernetes journey out of necessity. But Ghods says that even if companies don’t have to be agnostic about cloud providers today, Kubernetes may soon become the industry standard, as more and more tooling and extensions are built around the API.</br></br>
-    "The same way it doesn’t make sense to deviate from Linux because it’s such a standard," Ghods says, "I think Kubernetes is going down the same path. It is still early days—the documentation still needs work and the user experience for writing and publishing specs to the Kubernetes clusters is still rough. When you’re on the cutting edge you can expect to bleed a little. But the bottom line is, this is where the industry is going. Three to five years from now it’s really going to be shocking if you run your infrastructure any other way."
-  </div>
-</section>
+{{< case-studies/quote image="/images/case-studies/box/banner4.jpg">}}
+"The vast majority of the work our team has done to implement Kubernetes at Box has been making it work inside of our existing [and often legacy] infrastructure....overall Kubernetes has been remarkably flexible with fitting into many of our constraints, and we've been running it very successfully on our bare metal infrastructure."
+{{< /case-studies/quote >}}
+
+<p>Having learned from experience, Ghods offers these two pieces of advice for companies going through similar challenges:</p>
+
+{{< case-studies/lead >}}
+1. Deliver early and often.
+{{< /case-studies/lead >}}
+
+<p>Service discovery was a huge problem for Box, and the team had to decide whether to build an interim solution or wait for Kubernetes to natively satisfy Box's unique requirements. After much debate, "we just started focusing on delivering something that works, and then dealing with potentially migrating to a more native solution later," Ghods says. "The above-all-else target for the team should always be to serve real production use cases on the infrastructure, no matter how trivial. This helps keep the momentum going both for the team itself and for the organizational perception of the project."</p>
+
+{{< case-studies/lead >}}
+2. Keep an open mind about what your company has to abstract away from developers and what it doesn't.
+{{< /case-studies/lead >}}
+
+<p>Early on, the team built an abstraction on top of Docker files to help ensure that images had the right security updates. This turned out to be superfluous work, since container images are considered immutable and you can easily scan them post-build to ensure they do not contain vulnerabilities. Because managing infrastructure through containerization is such a discontinuous leap, it's better to start by interacting directly with the native tools and learning their unique advantages and caveats. An abstraction should be built only after a practical need for it arises.</p>
+
+<p>In the end, the impact has been powerful. "Before Kubernetes," Ghods says, "our infrastructure was so antiquated it was taking us more than six months to deploy a new microservice. Now a new microservice takes less than five days to deploy. And we're working on getting it to an hour. Granted, much of that six months was due to how broken our systems were, but bare metal is intrinsically a difficult platform to support unless you have a system like Kubernetes to help manage it."</p>
+
+<p>By Ghods's estimate, Box is still several years away from his goal of being a 90-plus percent Kubernetes shop. "We're very far along on having a mission-critical, stable Kubernetes deployment that provides a lot of value," he says. "Right now about five percent of all of our compute runs on Kubernetes, and I think in the next six months we'll likely be between 20 to 50 percent. We're working hard on enabling all stateless service use cases, and shift our focus to stateful services after that."</p>
+
+{{< case-studies/quote >}}
+"Ghods predicts that Kubernetes has the opportunity to be the new cloud platform. '...because it's a never-before-seen level of automation and intelligence surrounding infrastructure that is portable and agnostic to every way you can run your infrastructure.'"
+{{< /case-studies/quote >}}
+
+<p>In fact, that's what he envisions across the industry: Ghods predicts that Kubernetes has the opportunity to be the new cloud platform. Kubernetes provides an API consistent across different cloud platforms including bare metal, and "I don't think people have seen the full potential of what's possible when you can program against one single interface," he says. "The same way <a href="https://aws.amazon.com/">AWS</a> changed infrastructure so that you don't have to think about servers or cabinets or networking equipment anymore, Kubernetes enables you to focus exclusively on the containers that you're running, which is pretty exciting. That's the vision."</p>
+
+<p>Ghods points to projects that are already in development or recently released for Kubernetes as a cloud platform: cluster federation, the Dashboard UI, and <a href="https://coreos.com/">CoreOS</a>'s etcd operator. "I honestly believe it's the most exciting thing I've seen in cloud infrastructure," he says, "because it's a never-before-seen level of automation and intelligence surrounding infrastructure that is portable and agnostic to every way you can run your infrastructure."</p>
+
+<p>Box, with its early decision to use bare metal, embarked on its Kubernetes journey out of necessity. But Ghods says that even if companies don't have to be agnostic about cloud providers today, Kubernetes may soon become the industry standard, as more and more tooling and extensions are built around the API.</p>
+
+<p>"The same way it doesn't make sense to deviate from Linux because it's such a standard," Ghods says, "I think Kubernetes is going down the same path. It is still early days—the documentation still needs work and the user experience for writing and publishing specs to the Kubernetes clusters is still rough. When you're on the cutting edge you can expect to bleed a little. But the bottom line is, this is where the industry is going. Three to five years from now it's really going to be shocking if you run your infrastructure any other way."</p>
@@ -1,112 +1,83 @@
 ---
 title: Buffer Case Study
-
 case_study_styles: true
 cid: caseStudies
-css: /css/style_buffer.css
+
+new_case_study_styles: true
+heading_background: /images/case-studies/buffer/banner3.jpg
+heading_title_logo: /images/buffer.png
+subheading: >
+  Making Deployments Easy for a Small, Distributed Team
+case_study_details:
+  - Company: Buffer
+  - Location: Around the World
+  - Industry: Social Media Technology
 ---

-<div class="banner1">
-  <h1>CASE STUDY: <img src="/images/buffer.png" width="18%" style="margin-bottom:-5px;margin-left:10px;"><br>
-  <div class="subhead">Making Deployments Easy for a Small, Distributed Team</div>
-</h1>
-</div>
+<h2>Challenge</h2>

-<div class="details">
-    Company&nbsp;<b>Buffer</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location &nbsp;<b>Around the World</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Industry &nbsp;<b>Social Media Technology</b>
-</div>
+<p>With a small but fully distributed team of 80 working across almost a dozen time zones, Buffer—which offers social media management to agencies and marketers—was looking to solve its "classic monolithic code base problem," says Architect Dan Farrelly. "We wanted to have the kind of liquid infrastructure where a developer could create an app and deploy it and scale it horizontally as necessary."</p>

-<hr>
+<h2>Solution</h2>

-<section class="section1">
-  <div class="cols">
-    <div class="col1">
+<p>Embracing containerization, Buffer moved its infrastructure from Amazon Web Services' Elastic Beanstalk to Docker on AWS, orchestrated with Kubernetes.</p>

-    <h2>Challenge</h2>
-      With a small but fully distributed team of 80 working across almost a dozen time zones, Buffer—which offers social media management to agencies and marketers—was looking to solve its "classic monolithic code base problem," says Architect Dan Farrelly. "We wanted to have the kind of liquid infrastructure where a developer could create an app and deploy it and scale it horizontally as&nbsp;necessary."
-</div>
-
-<div class="col2">
-  <h2>Solution</h2>
-      Embracing containerization, Buffer moved its infrastructure from Amazon Web Services’ Elastic Beanstalk to Docker on AWS, orchestrated with&nbsp;Kubernetes.
-      <br>
-      <br>
 <h2>Impact</h2>
-      The new system "leveled up our ability with deployment and rolling out new changes," says Farrelly. "Building something on your computer and knowing that it’s going to work has shortened things up a lot. Our feedback cycles are a lot faster now&nbsp;too."
-    </div>
-</div>
-</section>

-<div class="banner2">
-  <div class="banner2text">
-    "It’s amazing that we can use the Kubernetes solution off the shelf with our team. And it just keeps getting better. Before we even know that we need something, it’s there in the next release or it’s coming in the next few months."<br><br><span style="font-size:16px;letter-spacing:2px;">- DAN FARRELLY, BUFFER ARCHITECT</span>
-  </div>
-</div>
+<p>The new system "leveled up our ability with deployment and rolling out new changes," says Farrelly. "Building something on your computer and knowing that it's going to work has shortened things up a lot. Our feedback cycles are a lot faster now too."</p>

-<section class="section2">
-<div class="fullcol">
-<h2>Dan Farrelly uses a carpentry analogy to explain the problem his company, <a href="https://buffer.com">Buffer</a>, began having as its team of developers grew over the past few years.</h2>
+{{< case-studies/quote author="DAN FARRELLY, BUFFER ARCHITECT" >}}
+"It's amazing that we can use the Kubernetes solution off the shelf with our team. And it just keeps getting better. Before we even know that we need something, it's there in the next release or it's coming in the next few months."
+{{< /case-studies/quote >}}

-    "If you’re building a table by yourself, it’s fine," the company’s architect says. "If you bring in a second person to work on the table, maybe that person can start sanding the legs while you’re sanding the top. But when you bring a third or fourth person in, someone should probably work on a different table." Needing to work on more and more different tables led Buffer on a path toward microservices and containerization made possible by Kubernetes.<br><br>
-    Since around 2012, Buffer had already been using <a href="https://aws.amazon.com/elasticbeanstalk/">Elastic Beanstalk</a>, the orchestration service for deploying infrastructure offered by <a href="https://aws.amazon.com">Amazon Web Services</a>. "We were deploying a single monolithic <a href="http://php.net/manual/en/intro-whatis.php">PHP</a> application, and it was the same application across five or six environments," says Farrelly. "We were very much a product-driven company. It was all about shipping new features quickly and getting things out the door, and if something was not broken, we didn’t spend too much time on it. If things were getting a little bit slow, we’d maybe use a faster server or just scale up one instance, and it would be good enough. We’d move on."<br><br>
-    But things came to a head in 2016. With the growing number of committers on staff, Farrelly and Buffer’s then-CTO, Sunil Sadasivan, decided it was time to re-architect and rethink their infrastructure. "It was a classic monolithic code base problem," says Farrelly.<br><br>Some of the company’s team was already successfully using <a href="https://www.docker.com">Docker</a> in their development environment, but the only application running on Docker in production was a marketing website that didn’t see real user traffic. They wanted to go further with Docker, and the next step was looking at options for&nbsp;orchestration.
-</div>
-</section>
+{{< case-studies/lead >}}
+Dan Farrelly uses a carpentry analogy to explain the problem his company, <a href="https://buffer.com">Buffer</a>, began having as its team of developers grew over the past few years.
+{{< /case-studies/lead >}}

-<div class="banner3">
-  <div class="banner3text">
-    And all the things Kubernetes did well suited Buffer’s needs. "We wanted to have the kind of liquid infrastructure where a developer could create an app and deploy it and scale it horizontally as necessary," says Farrelly. "We quickly used some scripts to set up a couple of test clusters, we built some small proof-of-concept applications in containers, and we deployed things within an hour. We had very little experience in running containers in production. It was amazing how quickly we could get a handle on it&nbsp;[Kubernetes]."
-  </div>
-</div>
+<p>"If you're building a table by yourself, it's fine," the company's architect says. "If you bring in a second person to work on the table, maybe that person can start sanding the legs while you're sanding the top. But when you bring a third or fourth person in, someone should probably work on a different table." Needing to work on more and more different tables led Buffer on a path toward microservices and containerization made possible by Kubernetes.</p>

-<section class="section3">
-<div class="fullcol">
-    First they considered <a href="https://mesosphere.com">Mesosphere</a>, <a href="https://dcos.io">DC/OS</a> and <a href="https://aws.amazon.com/ecs/">Amazon Elastic Container Service</a> (which their data systems team was already using for some data pipeline jobs). While they were impressed by these offerings, they ultimately went with Kubernetes. "We run on AWS still, so spinning up, creating services and creating load balancers on demand for us without having to configure them manually was a great way for our team to get into this," says Farrelly. "We didn’t need to figure out how to configure this or that, especially coming from a former Elastic Beanstalk environment that gave us an automatically-configured load balancer. I really liked Kubernetes’ controls of the command line. It just took care of ports. It was a lot more flexible. Kubernetes was designed for doing what it does, so it does it very well."<br><br>
-    And all the things Kubernetes did well suited Buffer’s needs. "We wanted to have the kind of liquid infrastructure where a developer could create an app and deploy it and scale it horizontally as necessary," says Farrelly. "We quickly used some scripts to set up a couple of test clusters, we built some small proof-of-concept applications in containers, and we deployed things within an hour. We had very little experience in running containers in production. It was amazing how quickly we could get a handle on it [Kubernetes]."<br><br>
-    Above all, it provided a powerful solution for one of the company’s most distinguishing characteristics: their remote team that’s spread across a dozen different time zones. "The people with deep knowledge of our infrastructure live in time zones different from our peak traffic time zones, and most of our product engineers live in other places," says Farrelly. "So we really wanted something where anybody could get a grasp of the system early on and utilize it, and not have to worry that the deploy engineer is asleep. Otherwise people would sit around for 12 to 24 hours for something. It’s been really cool to see people moving much faster."
-    <br><br>
-    With a relatively small engineering team—just 25 people, and only a handful working on infrastructure, with the majority front-end developers—Buffer needed "something robust for them to deploy whatever they wanted," says Farrelly. Before, "it was only a couple of people who knew how to set up everything in the old way. With this system, it was easy to review documentation and get something out extremely quickly. It lowers the bar for us to get everything in production. We don't have the big team to build all these tools or manage the infrastructure like other larger companies&nbsp;might."
-</div>
-</section>
+<p>Since around 2012, Buffer had already been using <a href="https://aws.amazon.com/elasticbeanstalk/">Elastic Beanstalk</a>, the orchestration service for deploying infrastructure offered by <a href="https://aws.amazon.com">Amazon Web Services</a>. "We were deploying a single monolithic <a href="http://php.net/manual/en/intro-whatis.php">PHP</a> application, and it was the same application across five or six environments," says Farrelly. "We were very much a product-driven company. It was all about shipping new features quickly and getting things out the door, and if something was not broken, we didn't spend too much time on it. If things were getting a little bit slow, we'd maybe use a faster server or just scale up one instance, and it would be good enough. We'd move on."</p>

-<div class="banner4">
-  <div class="banner4text">
-    "In our old way of working, the feedback loop was a lot longer, and it was delicate because if you deployed something, the risk was high to potentially break something else," Farrelly says. "With the kind of deploys that we built around Kubernetes, we were able to detect bugs and fix them, and get them deployed super fast. The second someone is fixing [a bug], it’s out the&nbsp;door."
-  </div>
-</div>
+<p>But things came to a head in 2016. With the growing number of committers on staff, Farrelly and Buffer's then-CTO, Sunil Sadasivan, decided it was time to re-architect and rethink their infrastructure. "It was a classic monolithic code base problem," says Farrelly.<br><br>Some of the company's team was already successfully using <a href="https://www.docker.com">Docker</a> in their development environment, but the only application running on Docker in production was a marketing website that didn't see real user traffic. They wanted to go further with Docker, and the next step was looking at options for orchestration.</p>

-<section class="section4">
-<div class="fullcol">
-    To help with this, Buffer developers wrote a deploy bot that wraps the Kubernetes deploy process and can be used by every team. "Before, our data analysts would update, say, a <a href="https://www.python.org">Python</a> analysis script and have to wait for the lead on that team to click the button and deploy it," Farrelly explains. "Now our data analysts can make a change, enter a <a href="https://slack.com">Slack</a> command, ‘/deploy,’ and it goes out instantly. They don’t need to wait on these slow turnaround times. They don’t even know where it’s running; it doesn’t matter."
-    <br><br>
-    One of the first applications the team built from scratch using Kubernetes was a new image resizing service. As a social media management tool that allows marketing teams to collaborate on posts and send updates across multiple social media profiles and networks, Buffer has to be able to resize photographs as needed to meet the varying limitations of size and format posed by different social networks. "We always had these hacked together solutions," says Farrelly.
-    <br><br>
-    To create this new service, one of the senior product engineers was assigned to learn Docker and Kubernetes, then build the service, test it, deploy it and monitor it—which he was able to do relatively quickly. "In our old way of working, the feedback loop was a lot longer, and it was delicate because if you deployed something, the risk was high to potentially break something else," Farrelly says. "With the kind of deploys that we built around Kubernetes, we were able to detect bugs and fix them, and get them deployed super fast. The second someone is fixing [a bug], it’s out the door."
-    <br><br>
-    Plus, unlike with their old system, they could scale things horizontally with one command. "As we rolled it out," Farrelly says, "we could anticipate and just click a button. This allowed us to deal with the demand that our users were placing on the system and easily scale it to handle it."
-    <br><br>
-    Another thing they weren’t able to do before was a canary deploy. This new capability "made us so much more confident in deploying big changes," says Farrelly. "Before, it took a lot of testing, which is still good, but it was also a lot of ‘fingers crossed.’ And this is something that gets run 800,000 times a day, the core of our business. If it doesn’t work, our business doesn’t work. In a Kubernetes world, I can do a canary deploy to test it for 1 percent and I can shut it down very quickly if it isn’t working. This has leveled up our ability to deploy and roll out new changes quickly while reducing&nbsp;risk."
+{{< case-studies/quote image="/images/case-studies/buffer/banner1.jpg" >}}
+And all the things Kubernetes did well suited Buffer's needs. "We wanted to have the kind of liquid infrastructure where a developer could create an app and deploy it and scale it horizontally as necessary," says Farrelly. "We quickly used some scripts to set up a couple of test clusters, we built some small proof-of-concept applications in containers, and we deployed things within an hour. We had very little experience in running containers in production. It was amazing how quickly we could get a handle on it [Kubernetes]."
+{{< /case-studies/quote >}}

-</div>
-</section>
+<p>First they considered <a href="https://mesosphere.com">Mesosphere</a>, <a href="https://dcos.io">DC/OS</a> and <a href="https://aws.amazon.com/ecs/">Amazon Elastic Container Service</a> (which their data systems team was already using for some data pipeline jobs). While they were impressed by these offerings, they ultimately went with Kubernetes. "We run on AWS still, so spinning up, creating services and creating load balancers on demand for us without having to configure them manually was a great way for our team to get into this," says Farrelly. "We didn't need to figure out how to configure this or that, especially coming from a former Elastic Beanstalk environment that gave us an automatically-configured load balancer. I really liked Kubernetes' controls of the command line. It just took care of ports. It was a lot more flexible. Kubernetes was designed for doing what it does, so it does it very well."</p>

-<div class="banner5">
-  <div class="banner5text">
-    "If you want to run containers in production, with nearly the power that Google uses internally, this [Kubernetes] is a great way to do that," Farrelly says. "We’re a relatively small team that’s actually running Kubernetes, and we’ve never run anything like it before. So it’s more approachable than you might think. That’s the one big thing that I tell people who are experimenting with it. Pick a couple of things, roll it out, kick the tires on this for a couple of months and see how much it can handle. You start learning a lot this&nbsp;way."
-  </div>
-</div>
+<p>And all the things Kubernetes did well suited Buffer's needs. "We wanted to have the kind of liquid infrastructure where a developer could create an app and deploy it and scale it horizontally as necessary," says Farrelly. "We quickly used some scripts to set up a couple of test clusters, we built some small proof-of-concept applications in containers, and we deployed things within an hour. We had very little experience in running containers in production. It was amazing how quickly we could get a handle on it [Kubernetes]."</p>

-<section class="section5">
-<div class="fullcol">
-    By October 2016, 54 percent of Buffer’s traffic was going through their Kubernetes cluster. "There’s a lot of our legacy functionality that still runs alright, and those parts might move to Kubernetes or stay in our old setup forever," says Farrelly. But the company made the commitment at that time that going forward, "all new development, all new features, will be running on Kubernetes."
-    <br><br>
-    The plan for 2017 is to move all the legacy applications to a new Kubernetes cluster, and run everything they’ve pulled out of their old infrastructure, plus the new services they’re developing in Kubernetes, on another cluster. "I want to bring all the benefits that we’ve seen on our early services to everyone on the team," says Farrelly.
-    <br><br>
-    <h2>For Buffer’s engineers, it’s an exciting process. "Every time we’re deploying a new service, we need to figure out: OK, what’s the architecture? How do these services communicate? What’s the best way to build this service?" Farrelly says. "And then we use the different features that Kubernetes has to glue all the pieces together. It’s enabling us to experiment as we’re learning how to design a service-oriented architecture. Before, we just wouldn’t have been able to do it. This is actually giving us a blank white board so we can do whatever we want on it."
-    </h2>
-    Part of that blank slate is the flexibility that Kubernetes offers should the time come when Buffer may want or need to change its cloud. "It’s cloud agnostic so maybe one day we could switch to Google or somewhere else," Farrelly says. "We’re very deep in Amazon but it’s nice to know we could move away if we need to."
-    <br><br>
-    At this point, the team at Buffer can’t imagine running their infrastructure any other way—and they’re happy to spread the word. "If you want to run containers in production, with nearly the power that Google uses internally, this [Kubernetes] is a great way to do that," Farrelly says. "We’re a relatively small team that’s actually running Kubernetes, and we’ve never run anything like it before. So it’s more approachable than you might think. That’s the one big thing that I tell people who are experimenting with it. Pick a couple of things, roll it out, kick the tires on this for a couple of months and see how much it can handle. You start learning a lot this&nbsp;way."
-  <br><br>
-</div>
-</section>
+<p>Above all, it provided a powerful solution for one of the company's most distinguishing characteristics: their remote team that's spread across a dozen different time zones. "The people with deep knowledge of our infrastructure live in time zones different from our peak traffic time zones, and most of our product engineers live in other places," says Farrelly. "So we really wanted something where anybody could get a grasp of the system early on and utilize it, and not have to worry that the deploy engineer is asleep. Otherwise people would sit around for 12 to 24 hours for something. It's been really cool to see people moving much faster."</p>
+
+<p>With a relatively small engineering team—just 25 people, and only a handful working on infrastructure, with the majority front-end developers—Buffer needed "something robust for them to deploy whatever they wanted," says Farrelly. Before, "it was only a couple of people who knew how to set up everything in the old way. With this system, it was easy to review documentation and get something out extremely quickly. It lowers the bar for us to get everything in production. We don't have the big team to build all these tools or manage the infrastructure like other larger companies might."</p>
+
+{{< case-studies/quote image="/images/case-studies/buffer/banner4.jpg" >}}
+"In our old way of working, the feedback loop was a lot longer, and it was delicate because if you deployed something, the risk was high to potentially break something else," Farrelly says. "With the kind of deploys that we built around Kubernetes, we were able to detect bugs and fix them, and get them deployed super fast. The second someone is fixing [a bug], it's out the door."
+{{< /case-studies/quote >}}
+
+<p>To help with this, Buffer developers wrote a deploy bot that wraps the Kubernetes deploy process and can be used by every team. "Before, our data analysts would update, say, a <a href="https://www.python.org">Python</a> analysis script and have to wait for the lead on that team to click the button and deploy it," Farrelly explains. "Now our data analysts can make a change, enter a <a href="https://slack.com">Slack</a> command, '/deploy,' and it goes out instantly. They don't need to wait on these slow turnaround times. They don't even know where it's running; it doesn't matter."</p>
+
+<p>One of the first applications the team built from scratch using Kubernetes was a new image resizing service. As a social media management tool that allows marketing teams to collaborate on posts and send updates across multiple social media profiles and networks, Buffer has to be able to resize photographs as needed to meet the varying limitations of size and format posed by different social networks. "We always had these hacked together solutions," says Farrelly.</p>
+
+<p>To create this new service, one of the senior product engineers was assigned to learn Docker and Kubernetes, then build the service, test it, deploy it and monitor it—which he was able to do relatively quickly. "In our old way of working, the feedback loop was a lot longer, and it was delicate because if you deployed something, the risk was high to potentially break something else," Farrelly says. "With the kind of deploys that we built around Kubernetes, we were able to detect bugs and fix them, and get them deployed super fast. The second someone is fixing [a bug], it's out the door."</p>
+
+<p>Plus, unlike with their old system, they could scale things horizontally with one command. "As we rolled it out," Farrelly says, "we could anticipate and just click a button. This allowed us to deal with the demand that our users were placing on the system and easily scale it to handle it."</p>
+
+<p>Another thing they weren't able to do before was a canary deploy. This new capability "made us so much more confident in deploying big changes," says Farrelly. "Before, it took a lot of testing, which is still good, but it was also a lot of 'fingers crossed.' And this is something that gets run 800,000 times a day, the core of our business. If it doesn't work, our business doesn't work. In a Kubernetes world, I can do a canary deploy to test it for 1 percent and I can shut it down very quickly if it isn't working. This has leveled up our ability to deploy and roll out new changes quickly while reducing risk."</p>
+
+{{< case-studies/quote >}}
+"If you want to run containers in production, with nearly the power that Google uses internally, this [Kubernetes] is a great way to do that," Farrelly says. "We're a relatively small team that's actually running Kubernetes, and we've never run anything like it before. So it's more approachable than you might think. That's the one big thing that I tell people who are experimenting with it. Pick a couple of things, roll it out, kick the tires on this for a couple of months and see how much it can handle. You start learning a lot this way."
+{{< /case-studies/quote >}}
+
+<p>By October 2016, 54 percent of Buffer's traffic was going through their Kubernetes cluster. "There's a lot of our legacy functionality that still runs alright, and those parts might move to Kubernetes or stay in our old setup forever," says Farrelly. But the company made the commitment at that time that going forward, "all new development, all new features, will be running on Kubernetes."</p>
+
+<p>The plan for 2017 is to move all the legacy applications to a new Kubernetes cluster, and run everything they've pulled out of their old infrastructure, plus the new services they're developing in Kubernetes, on another cluster. "I want to bring all the benefits that we've seen on our early services to everyone on the team," says Farrelly.</p>
+
+{{< case-studies/lead >}}
+For Buffer's engineers, it's an exciting process. "Every time we're deploying a new service, we need to figure out: OK, what's the architecture? How do these services communicate? What's the best way to build this service?" Farrelly says. "And then we use the different features that Kubernetes has to glue all the pieces together. It's enabling us to experiment as we're learning how to design a service-oriented architecture. Before, we just wouldn't have been able to do it. This is actually giving us a blank white board so we can do whatever we want on it."
+{{< /case-studies/lead >}}
+
+<p>Part of that blank slate is the flexibility that Kubernetes offers should the time come when Buffer may want or need to change its cloud. "It's cloud agnostic so maybe one day we could switch to Google or somewhere else," Farrelly says. "We're very deep in Amazon but it's nice to know we could move away if we need to."</p>
+
+<p>At this point, the team at Buffer can't imagine running their infrastructure any other way—and they're happy to spread the word. "If you want to run containers in production, with nearly the power that Google uses internally, this [Kubernetes] is a great way to do that," Farrelly says. "We're a relatively small team that's actually running Kubernetes, and we've never run anything like it before. So it's more approachable than you might think. That's the one big thing that I tell people who are experimenting with it. Pick a couple of things, roll it out, kick the tires on this for a couple of months and see how much it can handle. You start learning a lot this way."</p>
@@ -2,95 +2,61 @@
 title: Capital One Case Study
 case_study_styles: true
 cid: caseStudies
-css: /css/style_case_studies.css
+
+new_case_study_styles: true
+heading_background: /images/case-studies/capitalone/banner1.jpg
+heading_title_logo: /images/capitalone-logo.png
+subheading: >
+  Supporting Fast Decisioning Applications with Kubernetes
+case_study_details:
+  - Company: Capital One
+  - Location: McLean, Virginia
+  - Industry: Retail banking
 ---

-<div class="banner1 desktop" style="background-image: url('/images/case-studies/capitalone/banner1.jpg')">
-  <h1> CASE STUDY:<img src="/images/capitalone-logo.png" style="margin-bottom:-2%" class="header_logo"><br> <div class="subhead">Supporting Fast Decisioning Applications with Kubernetes
+<h2>Challenge</h2>

-</div></h1>
+<p>The team set out to build a provisioning platform for <a href="https://www.capitalone.com/">Capital One</a> applications deployed on AWS that use streaming, big-data decisioning, and machine learning. One of these applications handles millions of transactions a day; some deal with critical functions like fraud detection and credit decisioning. The key considerations: resilience and speed—as well as full rehydration of the cluster from base AMIs.</p>

-</div>
+<h2>Solution</h2>

-<div class="details">
-    Company &nbsp;<b>Capital One</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location &nbsp;<b>McLean, Virginia</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Industry &nbsp;<b>Retail banking</b>
-</div>
-
-<hr>
-<section class="section1">
-<div class="cols">
-  <div class="col1">
-    <h2>Challenge</h2>
-    The team set out to build a provisioning platform for <a href="https://www.capitalone.com/">Capital One</a> applications deployed on AWS that use streaming, big-data decisioning, and machine learning. One of these applications handles millions of transactions a day; some deal with critical functions like fraud detection and credit decisioning. The key considerations: resilience and speed—as well as full rehydration of the cluster from base AMIs.
-
-<br>
-    <h2>Solution</h2>
-    The decision to run <a href="https://kubernetes.io/">Kubernetes</a> "is very strategic for us," says John Swift, Senior Director Software Engineering. "We use Kubernetes as a substrate or an operating system, if you will. There’s a degree of affinity in our product development."
-</div>
-
-<div class="col2">
+<p>The decision to run <a href="https://kubernetes.io/">Kubernetes</a> "is very strategic for us," says John Swift, Senior Director Software Engineering. "We use Kubernetes as a substrate or an operating system, if you will. There's a degree of affinity in our product development."</p>

 <h2>Impact</h2>
-  "Kubernetes is a significant productivity multiplier," says Lead Software Engineer Keith Gasser, adding that to run the platform without Kubernetes would "easily see our costs triple, quadruple what they are now for the amount of pure AWS expense." Time to market has been improved as well: "Now, a team can come to us and we can have them up and running with a basic decisioning app in a fortnight, which before would have taken a whole quarter, if not longer." Deployments increased by several orders of magnitude. Plus, the rehydration/cluster-rebuild process, which took a significant part of a day to do manually, now takes a couple hours with Kubernetes automation and declarative configuration.
+
+<p>"Kubernetes is a significant productivity multiplier," says Lead Software Engineer Keith Gasser, adding that to run the platform without Kubernetes would "easily see our costs triple, quadruple what they are now for the amount of pure AWS expense." Time to market has been improved as well: "Now, a team can come to us and we can have them up and running with a basic decisioning app in a fortnight, which before would have taken a whole quarter, if not longer." Deployments increased by several orders of magnitude. Plus, the rehydration/cluster-rebuild process, which took a significant part of a day to do manually, now takes a couple hours with Kubernetes automation and declarative configuration.</p>
+
+{{< case-studies/quote author="Jamil Jadallah, Scrum Master" >}}
+<iframe width="560" height="315" src="https://www.youtube.com/embed/UHVW01ksg-s" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>
+<br>
+"With the scalability, the management, the coordination, Kubernetes really empowers us and gives us more time back than we had before."
+{{< /case-studies/quote >}}
+
+<p>As a top 10 U.S. retail bank, Capital One has applications that handle millions of transactions a day. Big-data decisioning—for fraud detection, credit approvals and beyond—is core to the business. To support the teams that build applications with those functions for the bank, the cloud team led by Senior Director Software Engineering John Swift embraced Kubernetes for its provisioning platform. "Kubernetes and its entire ecosystem are very strategic for us," says Swift. "We use Kubernetes as a substrate or an operating system, if you will. There's a degree of affinity in our product development."</p>
+
+<p>Almost two years ago, the team embarked on this journey by first working with Docker. Then came Kubernetes. "We wanted to put streaming services into Kubernetes as one feature of the workloads for fast decisioning, and to be able to do batch alongside it," says Lead Software Engineer Keith Gasser. "Once the data is streamed and batched, there are so many tool sets in <a href="https://flink.apache.org/">Flink</a> that we use for decisioning. We want to provide the tools in the same ecosystem, in a consistent way, rather than have a large custom snowflake ecosystem where every tool needs its own custom deployment. Kubernetes gives us the ability to bring all of these together, so the richness of the open source and even the license community dealing with big data can be corralled."</p>
+
+{{< case-studies/quote image="/images/case-studies/capitalone/banner3.jpg" >}}
+"We want to provide the tools in the same ecosystem, in a consistent way, rather than have a large custom snowflake ecosystem where every tool needs its own custom deployment. Kubernetes gives us the ability to bring all of these together, so the richness of the open source and even the license community dealing with big data can be corralled."
+{{< /case-studies/quote >}}
+
+<p>In this first year, the impact has already been great. "Time to market is really huge for us," says Gasser. "Especially with fraud, you have to be very nimble in the way you respond to threats in the marketplace—being able to add and push new rules, detect new patterns of behavior, detect anomalies in account and transaction flows." With Kubernetes, "a team can come to us and we can have them up and running with a basic decisioning app in a fortnight, which before would have taken a whole quarter, if not longer. Kubernetes is a manifold productivity multiplier."</p>
+
+<p>Teams now have the tools to be autonomous in their deployments, and as a result, deployments have increased by two orders of magnitude. "And that was with just seven dedicated resources, without needing a whole group sitting there watching everything," says Scrum Master Jamil Jadallah. "That's a huge cost savings. With the scalability, the management, the coordination, Kubernetes really empowers us and gives us more time back than we had before."</p>
+
+{{< case-studies/quote image="/images/case-studies/capitalone/banner4.jpg" >}}
+With Kubernetes, "a team can come to us and we can have them up and running with a basic decisioning app in a fortnight, which before would have taken a whole quarter, if not longer. Kubernetes is a manifold productivity multiplier."
+{{< /case-studies/quote >}}
+
+<p>Kubernetes has also been a great time-saver for Capital One's required period "rehydration" of clusters from base AMIs. To minimize the attack vulnerability profile for applications in the cloud, "Our entire clusters get rebuilt from scratch periodically, with new fresh instances and virtual server images that are patched with the latest and greatest security patches," says Gasser. This process used to take the better part of a day, and personnel, to do manually. It's now a quick Kubernetes job.</p>
+
+<p>Savings extend to both capital and operating expenses. "It takes very little to get into Kubernetes because it's all open source," Gasser points out. "We went the DIY route for building our cluster, and we definitely like the flexibility of being able to embrace the latest from the community immediately without waiting for a downstream company to do it. There's capex related to those licenses that we don't have to pay for. Moreover, there's capex savings for us from some of the proprietary software that we get to sunset in our particular domain. So that goes onto our ledger in a positive way as well." (Some of those open source technologies include Prometheus, Fluentd, gRPC, Istio, CNI, and Envoy.)</p>
+
+{{< case-studies/quote >}}
+"If we had to do all of this without Kubernetes, on underlying cloud services, I could easily see our costs triple, quadruple what they are now for the amount of pure AWS expense. That doesn't account for personnel to deploy and maintain all the additional infrastructure."
+{{< /case-studies/quote >}}


-</div>
+<p>And on the opex side, Gasser says, the savings are high. "We run dozens of services, we have scores of pods, many daemon sets, and since we're data-driven, we take advantage of EBS-backed volume claims for all of our stateful services. If we had to do all of this without Kubernetes, on underlying cloud services, I could easily see our costs triple, quadruple what they are now for the amount of pure AWS expense. That doesn't account for personnel to deploy and maintain all the additional infrastructure."</p>

-</div>
-</section>
-<div class="banner2">
-  <div class="banner2text">
-<iframe width="560" height="315" src="https://www.youtube.com/embed/UHVW01ksg-s" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe><br><br>
-"With the scalability, the management, the coordination, Kubernetes really empowers us and gives us more time back than we had before." <span style="font-size:16px;text-transform:uppercase">— Jamil Jadallah, Scrum Master</span>
-</div>
-  </div>
-</div>
-<section class="section2">
-<div class="fullcol">
-  <h2></h2>
-  As a top 10 U.S. retail bank, Capital One has applications that handle millions of transactions a day. Big-data decisioning—for fraud detection, credit approvals and beyond—is core to the business. To support the teams that build applications with those functions for the bank, the cloud team led by Senior Director Software Engineering John Swift embraced Kubernetes for its provisioning platform. "Kubernetes and its entire ecosystem are very strategic for us," says Swift. "We use Kubernetes as a substrate or an operating system, if you will. There’s a degree of affinity in our product development."<br><br>
-  Almost two years ago, the team embarked on this journey by first working with Docker. Then came Kubernetes. "We wanted to put streaming services into Kubernetes as one feature of the workloads for fast decisioning, and to be able to do batch alongside it," says Lead Software Engineer Keith Gasser. "Once the data is streamed and batched, there are so many tool sets in <a href="https://flink.apache.org/">Flink</a> that we use for decisioning. We want to provide the tools in the same ecosystem, in a consistent way, rather than have a large custom snowflake ecosystem where every tool needs its own custom deployment. Kubernetes gives us the ability to bring all of these together, so the richness of the open source and even the license community dealing with big data can be corralled."
-
-
-
-</div>
-</section>
-<div class="banner3" style="background-image: url('/images/case-studies/capitalone/banner3.jpg')">
-  <div class="banner3text">
-    "We want to provide the tools in the same ecosystem, in a consistent way, rather than have a large custom snowflake ecosystem where every tool needs its own custom deployment. Kubernetes gives us the ability to bring all of these together, so the richness of the open source and even the license community dealing with big data can be corralled."
-
-
-  </div>
-</div>
-<section class="section3">
-<div class="fullcol">
-  In this first year, the impact has already been great. "Time to market is really huge for us," says Gasser. "Especially with fraud, you have to be very nimble in the way you respond to threats in the marketplace—being able to add and push new rules, detect new patterns of behavior, detect anomalies in account and transaction flows." With Kubernetes, "a team can come to us and we can have them up and running with a basic decisioning app in a fortnight, which before would have taken a whole quarter, if not longer. Kubernetes is a manifold productivity multiplier."<br><br>
-  Teams now have the tools to be autonomous in their deployments, and as a result, deployments have increased by two orders of magnitude. "And that was with just seven dedicated resources, without needing a whole group sitting there watching everything," says Scrum Master Jamil Jadallah. "That’s a huge cost savings. With the scalability, the management, the coordination, Kubernetes really empowers us and gives us more time back than we had before."
-
-</div>
-</section>
-<div class="banner4" style="background-image: url('/images/case-studies/capitalone/banner4.jpg')">
-  <div class="banner4text">
-    With Kubernetes, "a team can come to us and we can have them up and running with a basic decisioning app in a fortnight, which before would have taken a whole quarter, if not longer. Kubernetes is a manifold productivity multiplier."
-  </div>
-</div>
-
-<section class="section5" style="padding:0px !important">
-<div class="fullcol">
-    Kubernetes has also been a great time-saver for Capital One’s required period "rehydration" of clusters from base AMIs. To minimize the attack vulnerability profile for applications in the cloud, "Our entire clusters get rebuilt from scratch periodically, with new fresh instances and virtual server images that are patched with the latest and greatest security patches," says Gasser. This process used to take the better part of a day, and personnel, to do manually. It’s now a quick Kubernetes job.<br><br>
-  Savings extend to both capital and operating expenses. "It takes very little to get into Kubernetes because it’s all open source," Gasser points out. "We went the DIY route for building our cluster, and we definitely like the flexibility of being able to embrace the latest from the community immediately without waiting for a downstream company to do it. There’s capex related to those licenses that we don’t have to pay for. Moreover, there’s capex savings for us from some of the proprietary software that we get to sunset in our particular domain. So that goes onto our ledger in a positive way as well." (Some of those open source technologies include Prometheus, Fluentd, gRPC, Istio, CNI, and Envoy.)
-
-</div>
-
-<div class="banner5">
-  <div class="banner5text">
-   "If we had to do all of this without Kubernetes, on underlying cloud services, I could easily see our costs triple, quadruple what they are now for the amount of pure AWS expense. That doesn’t account for personnel to deploy and maintain all the additional infrastructure."
-  </div>
-</div>
-
-<div class="fullcol">
-  And on the opex side, Gasser says, the savings are high. "We run dozens of services, we have scores of pods, many daemon sets, and since we’re data-driven, we take advantage of EBS-backed volume claims for all of our stateful services. If we had to do all of this without Kubernetes, on underlying cloud services, I could easily see our costs triple, quadruple what they are now for the amount of pure AWS expense. That doesn’t account for personnel to deploy and maintain all the additional infrastructure."<br><br>
-  The team is confident that the benefits will continue to multiply—without a steep learning curve for the engineers being exposed to the new technology. "As we onboard additional tenants in this ecosystem, I think the need for folks to understand Kubernetes may not necessarily go up. In fact, I think it goes down, and that’s good," says Gasser. "Because that really demonstrates the scalability of the technology. You start to reap the benefits, and they can concentrate on all the features they need to build for great decisioning in the business— fraud decisions, credit decisions—and not have to worry about, ‘Is my AWS server broken? Is my pod not running?’"
-</div>
-
-</section>
+<p>The team is confident that the benefits will continue to multiply—without a steep learning curve for the engineers being exposed to the new technology. "As we onboard additional tenants in this ecosystem, I think the need for folks to understand Kubernetes may not necessarily go up. In fact, I think it goes down, and that's good," says Gasser. "Because that really demonstrates the scalability of the technology. You start to reap the benefits, and they can concentrate on all the features they need to build for great decisioning in the business— fraud decisions, credit decisions—and not have to worry about, 'Is my AWS server broken? Is my pod not running?'"</p>
@@ -3,91 +3,79 @@ title: CERN Case Study
 linkTitle: cern
 case_study_styles: true
 cid: caseStudies
-css: /css/style_case_studies.css
 logo: cern_featured_logo.png
+
+new_case_study_styles: true
+heading_background: /images/case-studies/cern/banner1.jpg
+heading_title_text: CERN
+subheading: >
+  CERN: Processing Petabytes of Data More Efficiently with Kubernetes
+case_study_details:
+  - Company: CERN
+  - Location: Geneva, Switzerland
+  - Industry: Particle physics research
 ---

-<div class="banner1" style="background-image: url('/images/case-studies/cern/banner1.jpg')">
-  <h1> CASE STUDY: CERN<br> <div class="subhead" style="margin-top:1%">CERN: Processing Petabytes of Data More Efficiently with Kubernetes
+<h2>Challenge</h2>

-</div></h1>
+<p>At CERN, the European Organization for Nuclear Research, physicists conduct experiments to learn about fundamental science. In its particle accelerators, "we accelerate protons to very high energy, close to the speed of light, and we make the two beams of protons collide," says CERN Software Engineer Ricardo Rocha. "The end result is a lot of data that we have to process." CERN currently stores 330 petabytes of data in its data centers, and an upgrade of its accelerators expected in the next few years will drive that number up by 10x. Additionally, the organization experiences extreme peaks in its workloads during periods prior to big conferences, and needs its infrastructure to scale to those peaks. "We want to have a more hybrid infrastructure, where we have our on premise infrastructure but can make use of public clouds temporarily when these peaks come up," says Rocha. "We've been looking to new technologies that can help improve our efficiency in our infrastructure so that we can dedicate more of our resources to the actual processing of the data."</p>

-</div>
+<h2>Solution</h2>

-<div class="details">
-    Company &nbsp;<b>CERN</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location &nbsp;<b>Geneva, Switzerland
-</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Industry &nbsp;<b>Particle physics research</b>
-</div>
+<p>CERN's technology team embraced containerization and cloud native practices, choosing Kubernetes for orchestration, Helm for deployment, Prometheus for monitoring, and CoreDNS for DNS resolution inside the clusters. Kubernetes federation has allowed the organization to run some production workloads both on premise and in public clouds.</p>

-<hr>
-<section class="section1">
-<div class="cols">
-  <div class="col1" style="width:100%"">
-    <h2>Challenge</h2>
-    At CERN, the European Organization for Nuclear Research, physicists conduct experiments to learn about fundamental science. In its particle accelerators, "we accelerate protons to very high energy, close to the speed of light, and we make the two beams of protons collide," says CERN Software Engineer Ricardo Rocha. "The end result is a lot of data that we have to process." CERN currently stores 330 petabytes of data in its data centers, and an upgrade of its accelerators expected in the next few years will drive that number up by 10x. Additionally, the organization experiences extreme peaks in its workloads during periods prior to big conferences, and needs its infrastructure to scale to those peaks. "We want to have a more hybrid infrastructure, where we have our on premise infrastructure but can make use of public clouds temporarily when these peaks come up," says Rocha. "We’ve been looking to new technologies that can help improve our efficiency in our infrastructure so that we can dedicate more of our resources to the actual processing of the data."
-<br><br>
-    <h2>Solution</h2>
-    CERN’s technology team embraced containerization and cloud native practices, choosing Kubernetes for orchestration, Helm for deployment, Prometheus for monitoring, and CoreDNS for DNS resolution inside the clusters. Kubernetes federation has allowed the organization to run some production workloads both on premise and in public clouds.
-<br><br>
 <h2>Impact</h2>
-  "Kubernetes gives us the full automation of the application," says Rocha. "It comes with built-in monitoring and logging for all the applications and the workloads that deploy in Kubernetes. This is a massive simplification of our current deployments." The time to deploy a new cluster for a complex distributed storage system has gone from more than 3 hours to less than 15 minutes. Adding new nodes to a cluster used to take more than an hour; now it takes less than 2 minutes. The time it takes to autoscale replicas for system components has decreased from more than an hour to less than 2 minutes. Initially, virtualization gave 20% overhead, but with tuning this was reduced to ~5%. Moving to Kubernetes on bare metal would get this to 0%. Not having to host virtual machines is expected to also get 10% of memory capacity back.

-</div>
+<p>"Kubernetes gives us the full automation of the application," says Rocha. "It comes with built-in monitoring and logging for all the applications and the workloads that deploy in Kubernetes. This is a massive simplification of our current deployments." The time to deploy a new cluster for a complex distributed storage system has gone from more than 3 hours to less than 15 minutes. Adding new nodes to a cluster used to take more than an hour; now it takes less than 2 minutes. The time it takes to autoscale replicas for system components has decreased from more than an hour to less than 2 minutes. Initially, virtualization gave 20% overhead, but with tuning this was reduced to ~5%. Moving to Kubernetes on bare metal would get this to 0%. Not having to host virtual machines is expected to also get 10% of memory capacity back.</p>

-</div>
-</section>
-<div class="banner2">
-  <div class="banner2text">
-    "Kubernetes is something we can relate to very much because it’s naturally distributed. What it gives us is a uniform API across heterogeneous resources to define our workloads. This is something we struggled with a lot in the past when we want to expand our resources outside our infrastructure."
-<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Ricardo Rocha, Software Engineer, CERN</span>
-  </div>
-</div>
-<section class="section2">
-<div class="fullcol">
-  <h2>With a mission of researching fundamental science, and a stable of extremely large machines, the European Organization for Nuclear Research (CERN) operates at what can only be described as hyperscale.  </h2>
-  Experiments are conducted in particle accelerators, the biggest of which is 27 kilometers in circumference. "We accelerate protons to very high energy, to close to the speed of light, and we make the two beams of protons collide in well-defined places," says CERN Software Engineer Ricardo Rocha. "We build experiments around these places where we do the collisions. The end result is a lot of data that we have to process."<br><br>
-  And he does mean a lot: CERN currently stores and processes 330 petabytes of data—gathered from 4,300 projects and 3,300 users—using 10,000 hypervisors and 320,000 cores in its data centers. <br><br>
-  Over the years, the CERN technology department has built a large computing infrastructure, based on OpenStack private clouds, to help the organization’s physicists analyze and treat all this data. The organization experiences extreme peaks in its workloads. "Very often, just before conferences, physicists want to do an enormous amount of extra analysis to publish their papers, and we have to scale to these peaks, which means overcommitting resources in some cases," says Rocha. "We want to have a more hybrid infrastructure, where we have our on premise infrastructure but can make use of public clouds temporarily when these peaks come up."<br><br>
-  Additionally, few years ago, CERN announced that it would be doing a big upgrade of its accelerators, which will mean a ten-fold increase in the amount of data that can be collected. "So we’ve been looking to new technologies that can help improve our efficiency in our infrastructure, so that we can dedicate more of our resources to the actual processing of the data," says Rocha.
+{{< case-studies/quote author="Ricardo Rocha, Software Engineer, CERN" >}}
+"Kubernetes is something we can relate to very much because it's naturally distributed. What it gives us is a uniform API across heterogeneous resources to define our workloads. This is something we struggled with a lot in the past when we want to expand our resources outside our infrastructure."
+{{< /case-studies/quote >}}

-</div>
-</section>
-<div class="banner3" style="background-image: url('/images/case-studies/cern/banner3.jpg')">
-  <div class="banner3text">
-    "Before, the tendency was always: ‘I need this, I get a couple of developers, and I implement it.’ Right now it’s ‘I need this, I’m sure other people also need this, so I’ll go and ask around.’ The CNCF is a good source because there’s a very large catalog of applications available. It’s very hard right now to justify developing a new product in-house. There is really no real reason to keep doing that. It’s much easier for us to try it out, and if we see it’s a good solution, we try to reach out to the community and start working with that community." <br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Ricardo Rocha, Software Engineer, CERN</span>
+{{< case-studies/lead >}}
+With a mission of researching fundamental science, and a stable of extremely large machines, the European Organization for Nuclear Research (CERN) operates at what can only be described as hyperscale.
+{{< /case-studies/lead >}}

-  </div>
-</div>
-<section class="section3">
-<div class="fullcol">
-  Rocha’s team started looking at Kubernetes and containerization in the second half of 2015. "We’ve been using distributed infrastructures for decades now," says Rocha. "Kubernetes is something we can relate to very much because it’s naturally distributed. What it gives us is a uniform API across heterogeneous resources to define our workloads. This is something we struggled with a lot in the past when we want to expand our resources outside our infrastructure."<br><br>
-  The team created a prototype system for users to deploy their own Kubernetes cluster in CERN’s infrastructure, and spent six months validating the use cases and making sure that Kubernetes integrated with CERN’s internal systems. The main use case is batch workloads, which represent more than 80% of resource usage at CERN. (One single project that does most of the physics data processing and analysis alone consumes 250,000 cores.) "This is something where the investment in simplification of the deployment, logging, and monitoring pays off very quickly," says Rocha. Other use cases include Spark-based data analysis and machine learning to improve physics analysis. "The fact that most of these technologies integrate very well with Kubernetes makes our lives easier," he adds.<br><br>
-  The system went into production in October 2016, also using Helm for deployment, Prometheus for monitoring, and CoreDNS for DNS resolution within the cluster. "One thing that Kubernetes gives us is the full automation of the application," says Rocha. "So it comes with built-in monitoring and logging for all the applications and the workloads that deploy in Kubernetes. This is a massive simplification of our current deployments." The time to deploy a new cluster for a complex distributed storage system has gone from more than 3 hours to less than 15 minutes.<br><br> Adding new nodes to a cluster used to take more than an hour; now it takes less than 2 minutes. The time it takes to autoscale replicas for system components has decreased from more than an hour to less than 2 minutes.
+<p>Experiments are conducted in particle accelerators, the biggest of which is 27 kilometers in circumference. "We accelerate protons to very high energy, to close to the speed of light, and we make the two beams of protons collide in well-defined places," says CERN Software Engineer Ricardo Rocha. "We build experiments around these places where we do the collisions. The end result is a lot of data that we have to process."</p>

-</div>
-</section>
-<div class="banner4" style="background-image: url('/images/case-studies/cern/banner4.jpg')">
-  <div class="banner4text">
-  "With Kubernetes, there’s a well-established technology and a big community that we can contribute to. It allows us to do our physics analysis without having to focus so much on the lower level software. This is just exciting. We are looking forward to keep contributing to the community and collaborating with everyone."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Ricardo Rocha, Software Engineer, CERN</span>
-  </div>
-</div>
+<p>And he does mean a lot: CERN currently stores and processes 330 petabytes of data—gathered from 4,300 projects and 3,300 users—using 10,000 hypervisors and 320,000 cores in its data centers.</p>

-<section class="section5" style="padding:0px !important">
-<div class="fullcol">
-  Rocha points out that the metric used in the particle accelerators may be events per second, but in reality "it’s how fast and how much of the data we can process that actually counts." And efficiency has certainly been improved with Kubernetes. Initially, virtualization gave 20% overhead, but with tuning this was reduced to ~5%. Moving to Kubernetes on bare metal would get this to 0%. Not having to host virtual machines is expected to also get 10% of memory capacity back.<br><br>
-  Kubernetes federation, which CERN has been using for a portion of its production workloads since February 2018, has allowed the organization to adopt a hybrid cloud strategy. And it was remarkably simple to do. "We had a summer intern working on federation," says Rocha. "For many years, I’ve been developing distributed computing software, which took like a decade and a lot of effort from a lot of people to stabilize and make sure it works. And for our intern, in a couple of days he was able to demo to me and my team that we had a cluster at CERN and a few clusters outside in public clouds that were federated together and that we could submit workloads to. This was shocking for us. It really shows the power of using this kind of well-established technologies." <br><br>
-  With such results, adoption of Kubernetes has made rapid gains at CERN, and the team is eager to give back to the community. "If we look back into the ’90s and early 2000s, there were not a lot of companies focusing on systems that have to scale to this kind of size, storing petabytes of data, analyzing petabytes of data," says Rocha. "The fact that Kubernetes is supported by such a wide community and different backgrounds, it motivates us to contribute back."
+<p>Over the years, the CERN technology department has built a large computing infrastructure, based on OpenStack private clouds, to help the organization's physicists analyze and treat all this data. The organization experiences extreme peaks in its workloads. "Very often, just before conferences, physicists want to do an enormous amount of extra analysis to publish their papers, and we have to scale to these peaks, which means overcommitting resources in some cases," says Rocha. "We want to have a more hybrid infrastructure, where we have our on premise infrastructure but can make use of public clouds temporarily when these peaks come up."</p>

-</div>
+<p>Additionally, few years ago, CERN announced that it would be doing a big upgrade of its accelerators, which will mean a ten-fold increase in the amount of data that can be collected. "So we've been looking to new technologies that can help improve our efficiency in our infrastructure, so that we can dedicate more of our resources to the actual processing of the data," says Rocha.</p>

-<div class="banner5" >
-  <div class="banner5text">
-This means that the physicist can build his or her analysis and publish it in a repository, share it with colleagues, and in 10 years redo the same analysis with new data. If we looked back even 10 years, this was just a dream."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Ricardo Rocha, Software Engineer, CERN</span></div>
-</div>
+{{< case-studies/quote
+  image="/images/case-studies/cern/banner3.jpg"
+  author="Ricardo Rocha, Software Engineer, CERN"
+>}}
+"Before, the tendency was always: 'I need this, I get a couple of developers, and I implement it.' Right now it's 'I need this, I'm sure other people also need this, so I'll go and ask around.' The CNCF is a good source because there's a very large catalog of applications available. It's very hard right now to justify developing a new product in-house. There is really no real reason to keep doing that. It's much easier for us to try it out, and if we see it's a good solution, we try to reach out to the community and start working with that community."
+{{< /case-studies/quote >}}

-<div class="fullcol">
-  These new technologies aren’t just enabling infrastructure improvements. CERN also uses the Kubernetes-based <a href="https://github.com/recast-hep">Reana/Recast</a> platform for reusable analysis, which is "the ability to define physics analysis as a set of workflows that are fully containerized in one single entry point," says Rocha. "This means that the physicist can build his or her analysis and publish it in a repository, share it with colleagues, and in 10 years redo the same analysis with new data. If we looked back even 10 years, this was just a dream."<br><br>
-  All of these things have changed the culture at CERN considerably. A decade ago, "The tendency was always: ‘I need this, I get a couple of developers, and I implement it,’" says Rocha. "Right now it’s ‘I need this, I’m sure other people also need this, so I’ll go and ask around.’ The CNCF is a good source because there’s a very large catalog of applications available. It’s very hard right now to justify developing a new product in-house. There is really no real reason to keep doing that. It’s much easier for us to try it out, and if we see it’s a good solution, we try to reach out to the community and start working with that community."
+<p>Rocha's team started looking at Kubernetes and containerization in the second half of 2015. "We've been using distributed infrastructures for decades now," says Rocha. "Kubernetes is something we can relate to very much because it's naturally distributed. What it gives us is a uniform API across heterogeneous resources to define our workloads. This is something we struggled with a lot in the past when we want to expand our resources outside our infrastructure."</p>

-</div>
-</section>
+<p>The team created a prototype system for users to deploy their own Kubernetes cluster in CERN's infrastructure, and spent six months validating the use cases and making sure that Kubernetes integrated with CERN's internal systems. The main use case is batch workloads, which represent more than 80% of resource usage at CERN. (One single project that does most of the physics data processing and analysis alone consumes 250,000 cores.) "This is something where the investment in simplification of the deployment, logging, and monitoring pays off very quickly," says Rocha. Other use cases include Spark-based data analysis and machine learning to improve physics analysis. "The fact that most of these technologies integrate very well with Kubernetes makes our lives easier," he adds.</p>
+
+<p>The system went into production in October 2016, also using Helm for deployment, Prometheus for monitoring, and CoreDNS for DNS resolution within the cluster. "One thing that Kubernetes gives us is the full automation of the application," says Rocha. "So it comes with built-in monitoring and logging for all the applications and the workloads that deploy in Kubernetes. This is a massive simplification of our current deployments." The time to deploy a new cluster for a complex distributed storage system has gone from more than 3 hours to less than 15 minutes.</p>
+
+<p>Adding new nodes to a cluster used to take more than an hour; now it takes less than 2 minutes. The time it takes to autoscale replicas for system components has decreased from more than an hour to less than 2 minutes.</p>
+
+{{< case-studies/quote
+  image="/images/case-studies/cern/banner4.jpg"
+  author="Ricardo Rocha, Software Engineer, CERN"
+>}}
+"With Kubernetes, there's a well-established technology and a big community that we can contribute to. It allows us to do our physics analysis without having to focus so much on the lower level software. This is just exciting. We are looking forward to keep contributing to the community and collaborating with everyone."
+{{< /case-studies/quote >}}
+
+<p>Rocha points out that the metric used in the particle accelerators may be events per second, but in reality "it's how fast and how much of the data we can process that actually counts." And efficiency has certainly been improved with Kubernetes. Initially, virtualization gave 20% overhead, but with tuning this was reduced to ~5%. Moving to Kubernetes on bare metal would get this to 0%. Not having to host virtual machines is expected to also get 10% of memory capacity back.</p>
+
+<p>Kubernetes federation, which CERN has been using for a portion of its production workloads since February 2018, has allowed the organization to adopt a hybrid cloud strategy. And it was remarkably simple to do. "We had a summer intern working on federation," says Rocha. "For many years, I've been developing distributed computing software, which took like a decade and a lot of effort from a lot of people to stabilize and make sure it works. And for our intern, in a couple of days he was able to demo to me and my team that we had a cluster at CERN and a few clusters outside in public clouds that were federated together and that we could submit workloads to. This was shocking for us. It really shows the power of using this kind of well-established technologies."</p>
+
+<p>With such results, adoption of Kubernetes has made rapid gains at CERN, and the team is eager to give back to the community. "If we look back into the '90s and early 2000s, there were not a lot of companies focusing on systems that have to scale to this kind of size, storing petabytes of data, analyzing petabytes of data," says Rocha. "The fact that Kubernetes is supported by such a wide community and different backgrounds, it motivates us to contribute back."</p>
+
+{{< case-studies/quote author="Ricardo Rocha, Software Engineer, CERN" >}}
+This means that the physicist can build his or her analysis and publish it in a repository, share it with colleagues, and in 10 years redo the same analysis with new data. If we looked back even 10 years, this was just a dream."
+{{< /case-studies/quote >}}
+
+<p>These new technologies aren't just enabling infrastructure improvements. CERN also uses the Kubernetes-based <a href="https://github.com/recast-hep">Reana/Recast</a> platform for reusable analysis, which is "the ability to define physics analysis as a set of workflows that are fully containerized in one single entry point," says Rocha. "This means that the physicist can build his or her analysis and publish it in a repository, share it with colleagues, and in 10 years redo the same analysis with new data. If we looked back even 10 years, this was just a dream."</p>
+
+<p>All of these things have changed the culture at CERN considerably. A decade ago, "The tendency was always: 'I need this, I get a couple of developers, and I implement it,'" says Rocha. "Right now it's 'I need this, I'm sure other people also need this, so I'll go and ask around.' The CNCF is a good source because there's a very large catalog of applications available. It's very hard right now to justify developing a new product in-house. There is really no real reason to keep doing that. It's much easier for us to try it out, and if we see it's a good solution, we try to reach out to the community and start working with that community."</p>
@@ -1,95 +1,77 @@
 ---
 title: China Unicom Case Study
-
 linkTitle: chinaunicom
 case_study_styles: true
 cid: caseStudies
-css: /css/style_case_studies.css
 featured: false
+
+new_case_study_styles: true
+heading_background: /images/case-studies/chinaunicom/banner1.jpg
+heading_title_logo: /images/chinaunicom_logo.png
+subheading: >
+  China Unicom: How China Unicom Leveraged Kubernetes to Boost Efficiency and Lower IT Costs
+case_study_details:
+  - Company: China Unicom
+  - Location: Beijing, China
+  - Industry: Telecom
 ---

-<div class="banner1" style="background-image: url('/images/case-studies/chinaunicom/banner1.jpg')">
-  <h1> CASE STUDY:<img src="/images/chinaunicom_logo.png" class="header_logo" style="width:25%;margin-bottom:-1%"><br> <div class="subhead" style="margin-top:1%;line-height:1.4em">China Unicom: How China Unicom Leveraged Kubernetes to Boost Efficiency<br>and Lower IT&nbsp;Costs
+<h2>Challenge</h2>

-</div></h1>
+<p>China Unicom is one of the top three telecom operators in China, and to serve its 300 million users, the company runs several data centers with thousands of servers in each, using <a href="https://www.docker.com/">Docker</a> containerization and <a href="https://www.vmware.com/">VMWare</a> and <a href="https://www.openstack.org/">OpenStack</a> infrastructure since 2016. Unfortunately, "the resource utilization rate was relatively low," says Chengyu Zhang, Group Leader of Platform Technology R&D, "and we didn't have a cloud platform to accommodate our hundreds of applications." Formerly an entirely state-owned company, China Unicom has in recent years taken private investment from BAT (Baidu, Alibaba, Tencent) and JD.com, and is now focusing on internal development using open source technology, rather than commercial products. As such, Zhang's China Unicom Lab team began looking for open source orchestration for its cloud infrastructure.</p>

-</div>
+<h2>Solution</h2>

-<div class="details">
-    Company &nbsp;<b>China Unicom</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location &nbsp;<b>Beijing, China</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Industry &nbsp;<b>Telecom</b>
-</div>
+<p>Because of its rapid growth and mature open source community, Kubernetes was a natural choice for China Unicom. The company's Kubernetes-enabled cloud platform now hosts 50 microservices and all new development going forward. "Kubernetes has improved our experience using cloud infrastructure," says Zhang. "There is currently no alternative technology that can replace it." China Unicom also uses <a href="https://istio.io/">Istio</a> for its microservice framework, <a href="https://www.envoyproxy.io/">Envoy</a>, <a href="https://coredns.io/">CoreDNS</a>, and <a href="https://www.fluentd.org/">Fluentd</a>.</p>

-<hr>
-<section class="section1">
-<div class="cols">
-  <div class="col1" style="width:100%"">
-    <h2>Challenge</h2>
-    China Unicom is one of the top three telecom operators in China, and to serve its 300 million users, the company runs several data centers with thousands of servers in each, using <a href="https://www.docker.com/">Docker</a> containerization and <a href="https://www.vmware.com/">VMWare</a> and <a href="https://www.openstack.org/">OpenStack</a> infrastructure since 2016. Unfortunately, "the resource utilization rate was relatively low," says Chengyu Zhang, Group Leader of Platform Technology R&D, "and we didn’t have a cloud platform to accommodate our hundreds of applications." Formerly an entirely state-owned company, China Unicom has in recent years taken private investment from BAT (Baidu, Alibaba, Tencent) and JD.com, and is now focusing on internal development using open source technology, rather than commercial products. As such, Zhang’s China Unicom Lab team began looking for open source orchestration for its cloud infrastructure.
-<br><br>
-    <h2>Solution</h2>
-    Because of its rapid growth and mature open source community, Kubernetes was a natural choice for China Unicom. The company’s Kubernetes-enabled cloud platform now hosts 50 microservices and all new development going forward. "Kubernetes has improved our experience using cloud infrastructure," says Zhang. "There is currently no alternative technology that can replace it." China Unicom also uses <a href="https://istio.io/">Istio</a> for its microservice framework, <a href="https://www.envoyproxy.io/">Envoy</a>, <a href="https://coredns.io/">CoreDNS</a>, and <a href="https://www.fluentd.org/">Fluentd</a>.
-<br><br>
 <h2>Impact</h2>
-    At China Unicom, Kubernetes has improved both operational and development efficiency. Resource utilization has increased by 20-50%, lowering IT infrastructure costs, and deployment time has gone from a couple of hours to 5-10 minutes. "This is mainly because of the self-healing and scalability, so we can increase our efficiency in operation and maintenance," Zhang says. "For example, we currently have only five people maintaining our multiple systems. We could never imagine we can achieve this scalability in such a short time."

-</div>
+<p>At China Unicom, Kubernetes has improved both operational and development efficiency. Resource utilization has increased by 20-50%, lowering IT infrastructure costs, and deployment time has gone from a couple of hours to 5-10 minutes. "This is mainly because of the self-healing and scalability, so we can increase our efficiency in operation and maintenance," Zhang says. "For example, we currently have only five people maintaining our multiple systems. We could never imagine we can achieve this scalability in such a short time."</p>

-</div>
-</section>
-<div class="banner2">
-  <div class="banner2text">
-    "Kubernetes has improved our experience using cloud infrastructure. There is currently no alternative technology that can replace it."
-<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Chengyu Zhang, Group Leader of Platform Technology R&D, China Unicom</span>
-  </div>
-</div>
-<section class="section2">
-<div class="fullcol">
-  <h2>With more than 300 million users, China Unicom is one of the country’s top three telecom operators. </h2>
+{{< case-studies/quote author="Chengyu Zhang, Group Leader of Platform Technology R&D, China Unicom" >}}
+"Kubernetes has improved our experience using cloud infrastructure. There is currently no alternative technology that can replace it."
+{{< /case-studies/quote >}}

-  Behind the scenes, the company runs multiple data centers with thousands of servers in each, using Docker containerization and VMWare and OpenStack infrastructure since 2016. Unfortunately, "the resource utilization rate was relatively low," says Chengyu Zhang,  Group Leader of Platform Technology R&D, "and we didn’t have a cloud platform to accommodate our hundreds of applications." <br><br>
-  Zhang’s team, which is responsible for new technology, R&D and platforms, set out to find an IT management solution. Formerly an entirely state-owned company, China Unicom has in recent years taken private investment from BAT (Baidu, Alibaba, Tencent) and JD.com, and is now focusing on homegrown development using open source technology, rather than commercial products. For that reason, the team began looking for open source orchestration for its cloud infrastructure.
+{{< case-studies/lead >}}
+With more than 300 million users, China Unicom is one of the country's top three telecom operators.
+{{< /case-studies/lead >}}

-</div>
-</section>
-<div class="banner3" style="background-image: url('/images/case-studies/chinaunicom/banner3.jpg');width:100%;padding-left:0;">
-  <div class="banner3text">
-    "We could never imagine we can achieve this scalability in such a short time."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Chengyu Zhang, Group Leader of Platform Technology R&D, China Unicom</span>
+<p>Behind the scenes, the company runs multiple data centers with thousands of servers in each, using Docker containerization and VMWare and OpenStack infrastructure since 2016. Unfortunately, "the resource utilization rate was relatively low," says Chengyu Zhang,  Group Leader of Platform Technology R&D, "and we didn't have a cloud platform to accommodate our hundreds of applications."</p>

-  </div>
-</div>
-<section class="section3">
-<div class="fullcol">
-  Though China Unicom was already using Mesos for a core telecom operator system, the team felt that Kubernetes was a natural choice for the new cloud platform. "The main reason was that it has a mature community," says Zhang. "It grows very rapidly, and so we can learn a lot from others’ best practices." China Unicom also uses Istio for its microservice framework, Envoy, CoreDNS, and Fluentd.<br><br>
-  The company’s Kubernetes-enabled cloud platform now hosts 50 microservices and all new development going forward. China Unicom developers can easily leverage the technology through APIs, without doing the development work themselves. The cloud platform provides 20-30 services connected to the company’s data center PaaS platform, as well as supports things such as big data analysis for internal users in the branch offices across the 31 provinces in China.<br><br>
-  "Kubernetes has improved our experience using cloud infrastructure," says Zhang. "There is currently no alternative technology that can replace it."
+<p>Zhang's team, which is responsible for new technology, R&D and platforms, set out to find an IT management solution. Formerly an entirely state-owned company, China Unicom has in recent years taken private investment from BAT (Baidu, Alibaba, Tencent) and JD.com, and is now focusing on homegrown development using open source technology, rather than commercial products. For that reason, the team began looking for open source orchestration for its cloud infrastructure.</p>

-</div>
-</section>
-<div class="banner4" style="background-image: url('/images/case-studies/chinaunicom/banner4.jpg');width:100%">
-  <div class="banner4text">
-  "This technology is relatively complicated, but as long as developers get used to it, they can enjoy all the benefits." <br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Jie Jia, Member of Platform Technology R&D, China Unicom</span>
-  </div>
-</div>
+{{< case-studies/quote
+  image="/images/case-studies/chinaunicom/banner3.jpg"
+  author="Chengyu Zhang, Group Leader of Platform Technology R&D, China Unicom"
+>}}
+"We could never imagine we can achieve this scalability in such a short time."
+{{< /case-studies/quote >}}

-<section class="section5" style="padding:0px !important">
-<div class="fullcol">
-  In fact, Kubernetes has boosted both operational and development efficiency at China Unicom. Resource utilization has increased by 20-50%, lowering IT infrastructure costs, and deployment time has gone from a couple of hours to 5-10 minutes. "This is mainly because of the self-healing and scalability of Kubernetes, so we can increase our efficiency in operation and maintenance," Zhang says. "For example, we currently have only five people maintaining our multiple systems."<br><br>
-  With the wins China Unicom has experienced with Kubernetes, Zhang and his team are eager to give back to the community. That starts with participating in meetups and conferences, and offering advice to other companies that are considering a similar path. "Especially for those companies who have had traditional cloud computing system, I really recommend them to join the cloud native computing community," says Zhang.
+<p>Though China Unicom was already using Mesos for a core telecom operator system, the team felt that Kubernetes was a natural choice for the new cloud platform. "The main reason was that it has a mature community," says Zhang. "It grows very rapidly, and so we can learn a lot from others' best practices." China Unicom also uses Istio for its microservice framework, Envoy, CoreDNS, and Fluentd.</p>

-</div>
+<p>The company's Kubernetes-enabled cloud platform now hosts 50 microservices and all new development going forward. China Unicom developers can easily leverage the technology through APIs, without doing the development work themselves. The cloud platform provides 20-30 services connected to the company's data center PaaS platform, as well as supports things such as big data analysis for internal users in the branch offices across the 31 provinces in China.</p>

-<div class="banner5" >
-  <div class="banner5text">
-"Companies can use the managed services offered by companies like Rancher, because they have already customized this technology, you can easily leverage this technology."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Jie Jia, Member of Platform Technology R&D, China Unicom</span></div>
-</div>
+<p>"Kubernetes has improved our experience using cloud infrastructure," says Zhang. "There is currently no alternative technology that can replace it."</p>

-<div class="fullcol">
-  Platform Technology R&D team member Jie Jia adds that though "this technology is relatively complicated, as long as developers get used to it, they can enjoy all the benefits." And Zhang points out that in his own experience with virtual machine cloud, "Kubernetes and these cloud native technologies are relatively simpler."<br><br>
-  Plus, "companies can use the managed services offered by companies like <a href="https://rancher.com/">Rancher</a>, because they have already customized this technology," says Jia. "You can easily leverage this technology."<br><br>
-  Looking ahead, China Unicom plans to develop more applications on Kubernetes, focusing on big data and machine learning. The team is continuing to optimize the cloud platform that it built, and hopes to pass the conformance test to join CNCF’s <a href="https://www.cncf.io/announcement/2017/11/13/cloud-native-computing-foundation-launches-certified-kubernetes-program-32-conformant-distributions-platforms/">Certified Kubernetes Conformance Program</a>. They’re also hoping to someday contribute code back to the community. <br><br>
-  If that sounds ambitious, it’s because the results they’ve gotten from adopting Kubernetes have been beyond even their greatest expectations. Says Zhang: "We could never imagine we can achieve this scalability in such a short time."
+{{< case-studies/quote 
+  image="/images/case-studies/chinaunicom/banner4.jpg"
+  author="Jie Jia, Member of Platform Technology R&D, China Unicom"
+>}}
+"This technology is relatively complicated, but as long as developers get used to it, they can enjoy all the benefits."
+{{< /case-studies/quote >}}

-</div>
-</section>
-</body>
-</html>
+<p>In fact, Kubernetes has boosted both operational and development efficiency at China Unicom. Resource utilization has increased by 20-50%, lowering IT infrastructure costs, and deployment time has gone from a couple of hours to 5-10 minutes. "This is mainly because of the self-healing and scalability of Kubernetes, so we can increase our efficiency in operation and maintenance," Zhang says. "For example, we currently have only five people maintaining our multiple systems."</p>
+
+<p>With the wins China Unicom has experienced with Kubernetes, Zhang and his team are eager to give back to the community. That starts with participating in meetups and conferences, and offering advice to other companies that are considering a similar path. "Especially for those companies who have had traditional cloud computing system, I really recommend them to join the cloud native computing community," says Zhang.</p>
+
+{{< case-studies/quote author="Jie Jia, Member of Platform Technology R&D, China Unicom" >}}
+"Companies can use the managed services offered by companies like Rancher, because they have already customized this technology, you can easily leverage this technology."
+{{< /case-studies/quote >}}
+
+<p>Platform Technology R&D team member Jie Jia adds that though "this technology is relatively complicated, as long as developers get used to it, they can enjoy all the benefits." And Zhang points out that in his own experience with virtual machine cloud, "Kubernetes and these cloud native technologies are relatively simpler."</p>
+
+<p>Plus, "companies can use the managed services offered by companies like <a href="https://rancher.com/">Rancher</a>, because they have already customized this technology," says Jia. "You can easily leverage this technology."</p>
+
+<p>Looking ahead, China Unicom plans to develop more applications on Kubernetes, focusing on big data and machine learning. The team is continuing to optimize the cloud platform that it built, and hopes to pass the conformance test to join CNCF's <a href="https://www.cncf.io/announcement/2017/11/13/cloud-native-computing-foundation-launches-certified-kubernetes-program-32-conformant-distributions-platforms/">Certified Kubernetes Conformance Program</a>. They're also hoping to someday contribute code back to the community.</p>
+
+<p>If that sounds ambitious, it's because the results they've gotten from adopting Kubernetes have been beyond even their greatest expectations. Says Zhang: "We could never imagine we can achieve this scalability in such a short time."</p>
@@ -3,97 +3,79 @@ title: City of Montreal Case Study
 linkTitle: city-of-montreal
 case_study_styles: true
 cid: caseStudies
-css: /css/style_case_studies.css
 featured: false
+
+new_case_study_styles: true
+heading_background: /images/case-studies/montreal/banner1.jpg
+heading_title_logo: /images/montreal_logo.png
+subheading: >
+  City of Montréal - How the City of Montréal Is Modernizing Its 30-Year-Old, Siloed Architecture with Kubernetes
+case_study_details:
+  - Company: City of Montréal
+  - Location: Montréal, Québec, Canada
+  - Industry: Government
 ---

-<div class="banner1" style="background-image: url('/images/case-studies/montreal/banner1.jpg')">
-  <h1> CASE STUDY:<img src="/images/montreal_logo.png" class="header_logo" style="width:20%;margin-bottom:-1.2%"><br> <div class="subhead" style="margin-top:1%">City of Montréal - How the City of Montréal Is Modernizing Its 30-Year-Old, Siloed&nbsp;Architecture&nbsp;with&nbsp;Kubernetes
+<h2>Challenge</h2>

-</div></h1>
+<p>Like many governments, Montréal has a number of legacy systems, and "we have systems that are older than some developers working here," says the city's CTO, Jean-Martin Thibault. "We have mainframes, all flavors of Windows, various flavors of Linux, old and new Oracle systems, Sun servers, all kinds of databases. Like all big corporations, some of the most important systems, like Budget and Human Resources, were developed on mainframes in-house over the past 30 years." There are over 1,000 applications in all, and most of them were running on different ecosystems. In 2015, a new management team decided to break down those silos, and invest in IT in order to move toward a more integrated governance for the city. They needed to figure out how to modernize the architecture.</p>

-</div>
+<h2>Solution</h2>

-<div class="details">
-    Company &nbsp;<b>City of Montréal</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location &nbsp;<b>Montréal, Québec, Canada</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Industry &nbsp;<b>Government</b>
-</div>
+<p>The first step was containerization. The team started with a small Docker farm with four or five servers, with Rancher for providing access to the Docker containers and their logs and Jenkins to deploy. "We based our effort on the new trends; we understood the benefits of immutability and deployments without downtime and such things," says Solutions Architect Marc Khouzam. They soon realized they needed orchestration as well, and opted for Kubernetes. Says Enterprise Architect Morgan Martinet: "Kubernetes offered concepts on how you would describe an architecture for any kind of application, and based on those concepts, deploy what's required to run the infrastructure. It was becoming a de facto standard."</p>

-<hr>
-<section class="section1">
-<div class="cols">
-  <div class="col1" style="width:100%"">
-    <h2>Challenge</h2>
-    Like many governments, Montréal has a number of legacy systems, and “we have systems that are older than some developers working here,” says the city’s CTO, Jean-Martin Thibault. “We have mainframes, all flavors of Windows, various flavors of Linux, old and new Oracle systems, Sun servers, all kinds of databases. Like all big corporations, some of the most important systems, like Budget and Human Resources, were developed on mainframes in-house over the past 30 years.” There are over 1,000 applications in all, and most of them were running on different ecosystems. In 2015, a new management team decided to break down those silos, and invest in IT in order to move toward a more integrated governance for the city. They needed to figure out how to modernize the architecture.
-
-    <h2>Solution</h2>
-    The first step was containerization. The team started with a small Docker farm with four or five servers, with Rancher for providing access to the Docker containers and their logs and Jenkins to deploy. “We based our effort on the new trends; we understood the benefits of immutability and deployments without downtime and such things,” says Solutions Architect Marc Khouzam. They soon realized they needed orchestration as well, and opted for Kubernetes. Says Enterprise Architect Morgan Martinet: “Kubernetes offered concepts on how you would describe an architecture for any kind of application, and based on those concepts, deploy what’s required to run the infrastructure. It was becoming a de facto standard.”
-<br>
 <h2>Impact</h2>
-    The time to market has improved drastically, from many months to a few weeks. Deployments went from months to hours. “In the past, you would have to ask for virtual machines, and that alone could take weeks, easily,” says Thibault. “Now you don’t even have to ask for anything. You just create your project and it gets deployed.” Kubernetes has also improved the efficiency of how the city uses its compute resources: “Before, the 200 application components we currently run on Kubernetes would have required hundreds of virtual machines, and now, if we’re talking about a single environment of production, we are able to run them on 8 machines, counting the masters of Kubernetes,” says Martinet. And it’s all done with a small team of just 5 people operating the Kubernetes clusters.
-</div>

-</div>
-</section>
-<div class="banner2">
-  <div class="banner2text">
-    "We realized the limitations of having a non-orchestrated Docker environment. Kubernetes came to the rescue, bringing in all these features that make it a lot easier to manage and give a lot more benefits to the users."
-<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- JEAN-MARTIN THIBAULT, CTO, CITY OF MONTRÉAL</span>
-  </div>
-</div>
-<section class="section2">
-<div class="fullcol">
-  <h2>The second biggest municipality in Canada, Montréal has a large number of legacy systems keeping the government running. And while they don’t quite date back to the city’s founding in 1642, “we have systems that are older than some developers working here,” jokes the city’s CTO, Jean-Martin Thibault.</h2>
-  “We have mainframes, all flavors of Windows, various flavors of Linux, old and new Oracle systems, Sun servers, all kinds of databases. Some of the most important systems, like Budget and Human Resources, were developed on mainframes in-house over the past 30 years.”
-<br><br>
-  In recent years, that fact became a big pain point. There are over 1,000 applications in all, running on almost as many different ecosystems. In 2015, a new city management team decided to break down those silos, and invest in IT in order to move toward a more integrated governance. “The organization was siloed, so as a result the architecture was siloed,” says Thibault. “Once we got integrated into one IT team, we decided to redo an overall enterprise architecture.”
-<br><br>
-  The first step to modernize the architecture was containerization. “We based our effort on the new trends; we understood the benefits of immutability and deployments without downtime and such things,” says Solutions Architect Marc Khouzam. The team started with a small Docker farm with four or five servers, with Rancher for providing access to the Docker containers and their logs and Jenkins for deployment.
-</div>
-</section>
-<div class="banner3" style="background-image: url('/images/case-studies/montreal/banner3.jpg')">
-  <div class="banner3text">
-    "Getting a project running in Kubernetes is entirely dependent on how long you need to program the actual software. It’s no longer dependent on deployment. Deployment is so fast that it’s negligible."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- MARC KHOUZAM, SOLUTIONS ARCHITECT, CITY OF MONTRÉAL</span>
+<p>The time to market has improved drastically, from many months to a few weeks. Deployments went from months to hours. "In the past, you would have to ask for virtual machines, and that alone could take weeks, easily," says Thibault. "Now you don't even have to ask for anything. You just create your project and it gets deployed." Kubernetes has also improved the efficiency of how the city uses its compute resources: "Before, the 200 application components we currently run on Kubernetes would have required hundreds of virtual machines, and now, if we're talking about a single environment of production, we are able to run them on 8 machines, counting the masters of Kubernetes," says Martinet. And it's all done with a small team of just 5 people operating the Kubernetes clusters.</p>

-  </div>
-</div>
-<section class="section3">
-<div class="fullcol">
-  But this Docker farm setup had some limitations, including the lack of self-healing and dynamic scaling based on traffic, and the effort required to optimize server resources and scale to multiple instances of the same container. The team soon realized they needed orchestration as well. “Kubernetes came to the rescue,” says Thibault, “bringing in all these features that make it a lot easier to manage and give a lot more benefits to the users.”
-<br><br>
-  The team had evaluated several orchestration solutions, but Kubernetes stood out because it addressed all of the pain points. (They were also inspired by Yahoo! Japan’s use case, which the team members felt came close to their vision.) “Kubernetes offered concepts on how you would describe an architecture for any kind of application, and based on those concepts, deploy what’s required to run the infrastructure,” says Enterprise Architect Morgan Martinet. “It was becoming a de facto standard. It also promised portability across cloud providers. The choice of Kubernetes now gives us many options such as running clusters in-house or in any IaaS provider, or even using Kubernetes-as-a-service in any of the major cloud providers.”
-<br><br>
-  Another important factor in the decision was vendor neutrality. “As a government entity, it is essential for us to be neutral in our selection of products and providers,” says Thibault. “The independence of the Cloud Native Computing Foundation from any company provides this.”
-</div>
-</section>
-<div class="banner4" style="background-image: url('/images/case-studies/montreal/banner4.jpg')">
-  <div class="banner4text">
-  "Kubernetes has been great. It’s been stable, and it provides us with elasticity, resilience, and robustness. While re-architecting for Kubernetes, we also benefited from the monitoring and logging aspects, with centralized logging, Prometheus logging, and Grafana dashboards. We have enhanced visibility of what’s being deployed." <br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- MORGAN MARTINET, ENTERPRISE ARCHITECT, CITY OF MONTRÉAL</span>
-  </div>
-</div>
+{{< case-studies/quote author="JEAN-MARTIN THIBAULT, CTO, CITY OF MONTRÉAL" >}}
+"We realized the limitations of having a non-orchestrated Docker environment. Kubernetes came to the rescue, bringing in all these features that make it a lot easier to manage and give a lot more benefits to the users."
+{{< /case-studies/quote >}}

-<section class="section5" style="padding:0px !important">
-<div class="fullcol">
-  The Kubernetes implementation began with the deployment of a small cluster using an internal Ansible playbook, which was soon replaced by the Kismatic distribution. Given the complexity they saw in operating a Kubernetes platform, they decided to provide development groups with an automated CI/CD solution based on Helm. “An integrated CI/CD solution on Kubernetes standardized how the various development teams designed and deployed their solutions, but allowed them to remain independent,” says Khouzam.
-<br><br>
-  During the re-architecting process, the team also added Prometheus for monitoring and alerting, Fluentd for logging, and Grafana for visualization. “We have enhanced visibility of what’s being deployed,” says Martinet. Adds Khouzam: “The big benefit is we can track anything, even things that don’t run inside the Kubernetes cluster. It’s our way to unify our monitoring effort.”
-<br><br>
-  All together, the cloud native solution has had a positive impact on velocity as well as administrative overhead. With standardization, code generation, automatic deployments into Kubernetes, and standardized monitoring through Prometheus, the time to market has improved drastically, from many months to a few weeks. Deployments went from months and weeks of planning down  to hours. “In the past, you would have to ask for virtual machines, and that alone could take weeks to properly provision,” says Thibault. Plus, for dedicated systems, experts often had to be brought in to install them with their own recipes, which could take weeks and months.
-<br><br>
-  Now, says Khouzam, “we can deploy pretty much any application that’s been Dockerized without any help from anybody. Getting a project running in Kubernetes is entirely dependent on how long you need to program the actual software. It’s no longer dependent on deployment. Deployment is so fast that it’s negligible.”
+{{< case-studies/lead >}}
+The second biggest municipality in Canada, Montréal has a large number of legacy systems keeping the government running. And while they don't quite date back to the city's founding in 1642, "we have systems that are older than some developers working here," jokes the city's CTO, Jean-Martin Thibault.
+{{< /case-studies/lead >}}

-</div>
+<p>"We have mainframes, all flavors of Windows, various flavors of Linux, old and new Oracle systems, Sun servers, all kinds of databases. Some of the most important systems, like Budget and Human Resources, were developed on mainframes in-house over the past 30 years."</p>

-<div class="banner5" >
-  <div class="banner5text">
-"We’re working with the market when possible, to put pressure on our vendors to support Kubernetes, because it’s a much easier solution to manage"<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- MORGAN MARTINET, ENTERPRISE ARCHITECT, CITY OF MONTRÉAL</span></div>
-</div>
+<p>In recent years, that fact became a big pain point. There are over 1,000 applications in all, running on almost as many different ecosystems. In 2015, a new city management team decided to break down those silos, and invest in IT in order to move toward a more integrated governance. "The organization was siloed, so as a result the architecture was siloed," says Thibault. "Once we got integrated into one IT team, we decided to redo an overall enterprise architecture."</p>

-<div class="fullcol">
-  Kubernetes has also improved the efficiency of how the city uses its compute resources: “Before, the 200 application components we currently run in Kubernetes would have required hundreds of virtual machines, and now, if we’re talking about a single environment of production, we are able to run them on 8 machines, counting the masters of Kubernetes,” says Martinet. And it’s all done with a small team of just five people operating the Kubernetes clusters. Adds Martinet: “It’s a dramatic improvement no matter what you measure.”
-<br><br>
-  So it should come as no surprise that the team’s strategy going forward is to target Kubernetes as much as they can. “If something can’t run inside Kubernetes, we’ll wait for it,” says Thibault. That means they haven’t moved any of the city’s Windows systems onto Kubernetes, though it’s something they would like to do. “We’re working with the market when possible, to put pressure on our vendors to support Kubernetes, because it’s a much easier solution to manage,” says Martinet.
-<br><br>
-  Thibault sees a near future where 60% of the city’s workloads are running on a Kubernetes platform—basically any and all of the use cases that they can get to work there. “It’s so much more efficient than the way we used to do things,” he says. “There’s no looking back.”
+<p>The first step to modernize the architecture was containerization. "We based our effort on the new trends; we understood the benefits of immutability and deployments without downtime and such things," says Solutions Architect Marc Khouzam. The team started with a small Docker farm with four or five servers, with Rancher for providing access to the Docker containers and their logs and Jenkins for deployment.</p>

-</div>
-</section>
+{{< case-studies/quote
+  image="/images/case-studies/montreal/banner3.jpg"
+  author="MARC KHOUZAM, SOLUTIONS ARCHITECT, CITY OF MONTRÉAL"
+>}}
+"Getting a project running in Kubernetes is entirely dependent on how long you need to program the actual software. It's no longer dependent on deployment. Deployment is so fast that it's negligible."
+{{< /case-studies/quote >}}
+
+<p>But this Docker farm setup had some limitations, including the lack of self-healing and dynamic scaling based on traffic, and the effort required to optimize server resources and scale to multiple instances of the same container. The team soon realized they needed orchestration as well. "Kubernetes came to the rescue," says Thibault, "bringing in all these features that make it a lot easier to manage and give a lot more benefits to the users."</p>
+
+<p>The team had evaluated several orchestration solutions, but Kubernetes stood out because it addressed all of the pain points. (They were also inspired by Yahoo! Japan's use case, which the team members felt came close to their vision.) "Kubernetes offered concepts on how you would describe an architecture for any kind of application, and based on those concepts, deploy what's required to run the infrastructure," says Enterprise Architect Morgan Martinet. "It was becoming a de facto standard. It also promised portability across cloud providers. The choice of Kubernetes now gives us many options such as running clusters in-house or in any IaaS provider, or even using Kubernetes-as-a-service in any of the major cloud providers."</p>
+
+<p>Another important factor in the decision was vendor neutrality. "As a government entity, it is essential for us to be neutral in our selection of products and providers," says Thibault. "The independence of the Cloud Native Computing Foundation from any company provides this."</p>
+
+{{< case-studies/quote
+  image="/images/case-studies/montreal/banner4.jpg"
+  author="MORGAN MARTINET, ENTERPRISE ARCHITECT, CITY OF MONTRÉAL"
+>}}
+"Kubernetes has been great. It's been stable, and it provides us with elasticity, resilience, and robustness. While re-architecting for Kubernetes, we also benefited from the monitoring and logging aspects, with centralized logging, Prometheus logging, and Grafana dashboards. We have enhanced visibility of what's being deployed."
+{{< /case-studies/quote >}}
+
+<p>The Kubernetes implementation began with the deployment of a small cluster using an internal Ansible playbook, which was soon replaced by the Kismatic distribution. Given the complexity they saw in operating a Kubernetes platform, they decided to provide development groups with an automated CI/CD solution based on Helm. "An integrated CI/CD solution on Kubernetes standardized how the various development teams designed and deployed their solutions, but allowed them to remain independent," says Khouzam.</p>
+
+<p>During the re-architecting process, the team also added Prometheus for monitoring and alerting, Fluentd for logging, and Grafana for visualization. "We have enhanced visibility of what's being deployed," says Martinet. Adds Khouzam: "The big benefit is we can track anything, even things that don't run inside the Kubernetes cluster. It's our way to unify our monitoring effort."</p>
+
+<p>All together, the cloud native solution has had a positive impact on velocity as well as administrative overhead. With standardization, code generation, automatic deployments into Kubernetes, and standardized monitoring through Prometheus, the time to market has improved drastically, from many months to a few weeks. Deployments went from months and weeks of planning down  to hours. "In the past, you would have to ask for virtual machines, and that alone could take weeks to properly provision," says Thibault. Plus, for dedicated systems, experts often had to be brought in to install them with their own recipes, which could take weeks and months.</p>
+
+<p>Now, says Khouzam, "we can deploy pretty much any application that's been Dockerized without any help from anybody. Getting a project running in Kubernetes is entirely dependent on how long you need to program the actual software. It's no longer dependent on deployment. Deployment is so fast that it's negligible."</p>
+
+{{< case-studies/quote author="MORGAN MARTINET, ENTERPRISE ARCHITECT, CITY OF MONTRÉAL">}}
+"We're working with the market when possible, to put pressure on our vendors to support Kubernetes, because it's a much easier solution to manage"
+{{< /case-studies/quote >}}
+
+<p>Kubernetes has also improved the efficiency of how the city uses its compute resources: "Before, the 200 application components we currently run in Kubernetes would have required hundreds of virtual machines, and now, if we're talking about a single environment of production, we are able to run them on 8 machines, counting the masters of Kubernetes," says Martinet. And it's all done with a small team of just five people operating the Kubernetes clusters. Adds Martinet: "It's a dramatic improvement no matter what you measure."</p>
+
+<p>So it should come as no surprise that the team's strategy going forward is to target Kubernetes as much as they can. "If something can't run inside Kubernetes, we'll wait for it," says Thibault. That means they haven't moved any of the city's Windows systems onto Kubernetes, though it's something they would like to do. "We're working with the market when possible, to put pressure on our vendors to support Kubernetes, because it's a much easier solution to manage," says Martinet.</p>
+
+<p>Thibault sees a near future where 60% of the city's workloads are running on a Kubernetes platform—basically any and all of the use cases that they can get to work there. "It's so much more efficient than the way we used to do things," he says. "There's no looking back."</p>
@@ -1,101 +1,85 @@
 ---
 title: Crowdfire Case Study
-
 case_study_styles: true
 cid: caseStudies
-css: /css/style_crowdfire.css
+
+new_case_study_styles: true
+heading_background: /images/case-studies/crowdfire/banner1.jpg
+heading_title_logo: /images/crowdfire_logo.png
+subheading: >
+  How to Keep Iterating a Fast-Growing App With a Cloud-Native Approach
+case_study_details:
+  - Company: Crowdfire
+  - Location: Mumbai, India
+  - Industry: Social Media Software
 ---

-<div class="banner1">
-  <h1> CASE STUDY:<img src="/images/crowdfire_logo.png" class="header_logo"><br> <div class="subhead">How to Keep Iterating a Fast-Growing App With a Cloud-Native Approach</div></h1>
+<h2>Challenge</h2>

-</div>
+<p><a href="https://www.crowdfireapp.com/">Crowdfire</a> helps content creators create their content anywhere on the Internet and publish it everywhere else in the right format. Since its launch in 2010, it has grown to 16 million users. The product began as a monolith app running on <a href="https://cloud.google.com/appengine/">Google App Engine</a>, and in 2015, the company began a transformation to microservices running on Amazon Web Services <a href="https://aws.amazon.com/elasticbeanstalk/">Elastic Beanstalk</a>. "It was okay for our use cases initially, but as the number of services, development teams and scale increased, the deploy times, self-healing capabilities and resource utilization started to become problems for us," says Software Engineer Amanpreet Singh, who leads the infrastructure team for Crowdfire.</p>

-<div class="details">
-    Company &nbsp;<b>Crowdfire</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location &nbsp;<b>Mumbai, India</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Industry &nbsp;<b>Social Media Software</b>
-</div>
+<h2>Solution</h2>

-<hr>
-<section class="section1">
-<div class="cols">
-  <div class="col1">
-    <h2>Challenge</h2>
-        <a href="https://www.crowdfireapp.com/">Crowdfire</a> helps content creators create their content anywhere on the Internet and publish it everywhere else in the right format. Since its launch in 2010, it has grown to 16 million users. The product began as a monolith app running on <a href="https://cloud.google.com/appengine/">Google App Engine</a>, and in 2015, the company began a transformation to microservices running on Amazon Web Services <a href="https://aws.amazon.com/elasticbeanstalk/">Elastic Beanstalk</a>. "It was okay for our use cases initially, but as the number of services, development teams and scale increased, the deploy times, self-healing capabilities and resource utilization started to become problems for us," says Software Engineer Amanpreet Singh, who leads the infrastructure team for Crowdfire.<br>
-        <h2>Solution</h2>
-            "We realized that we needed a more cloud-native approach to deal with these issues," says Singh. The team decided to implement a custom setup of Kubernetes based on <a href="https://www.terraform.io/">Terraform</a> and <a href="https://www.ansible.com/">Ansible</a>.
-      <br>
-      </div>
-
-<div class="col2">
+<p>"We realized that we needed a more cloud-native approach to deal with these issues," says Singh. The team decided to implement a custom setup of Kubernetes based on <a href="https://www.terraform.io/">Terraform</a> and <a href="https://www.ansible.com/">Ansible</a>.</p>

 <h2>Impact</h2>
-      "Kubernetes has helped us reduce the deployment time from 15 minutes to less than a minute," says Singh. "Due to Kubernetes’s self-healing nature, the operations team doesn’t need to do any manual intervention in case of a node or pod failure." Plus, he says, "Dev-Prod parity has improved since developers can experiment with options in dev/staging clusters, and when it’s finalized, they just commit the config changes in the respective code repositories. These changes automatically get replicated on the production cluster via CI/CD pipelines."
-</div>

-</div>
-</section>
-<div class="banner2">
-  <div class="banner2text">
-  "In the 15 months that we’ve been using Kubernetes, it has been amazing for us. It enabled us to iterate quickly, increase development speed, and continuously deliver new features and bug fixes to our users, while keeping our operational costs and infrastructure management overhead under control."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Amanpreet Singh, Software Engineer at Crowdfire</span>
-  </div>
-</div>
-<section class="section2">
-<div class="fullcol">
-    <h2>"If you build it, they will come."</h2>
-    For most content creators, only half of that movie quote may ring true. Sure, platforms like Wordpress, YouTube and Shopify have made it simple for almost anyone to start publishing new content online, but attracting an audience isn’t as easy. Crowdfire "helps users publish their content to all possible places where their audience exists," says Amanpreet Singh, a Software Engineer at the company based in Mumbai, India. Crowdfire has gained more than 16 million users—from bloggers and artists to makers and small businesses—since its launch in 2010.<br><br>
-    With that kind of growth—and a high demand from users for new features and continuous improvements—the Crowdfire team struggled to keep up behind the scenes. In 2015, they moved their monolith Java application to Amazon Web Services <a href="https://aws.amazon.com/elasticbeanstalk/">Elastic Beanstalk</a> and started breaking it down into microservices.<br><br>
-    It was a good first step, but the team soon realized they needed to go further down the cloud-native path, which would lead them to Kubernetes. "It was okay for our use cases initially, but as the number of services and development teams increased and we scaled further, deploy times, self-healing capabilities and resource utilization started to become problematic," says Singh, who leads the infrastructure team at Crowdfire. "We realized that we needed a more cloud-native approach to deal with these issues."<br><br>
-    As he looked around for solutions, Singh had a checklist of what Crowdfire needed. "We wanted to keep some things separate so they could be shipped independent of other things; this would help remove blockers and let different teams work at their own pace," he says. "We also make a lot of data-driven decisions, so shipping a feature and its iterations quickly was a must."<br><br>
-    Kubernetes checked all the boxes and then some. "One of the best things was the built-in service discovery," he says. "When you have a bunch of microservices that need to call each other, having internal DNS readily available and service IPs and ports automatically set as environment variables help a lot." Plus, he adds, "Kubernetes’s opinionated approach made it easier to get started."
+<p>"Kubernetes has helped us reduce the deployment time from 15 minutes to less than a minute," says Singh. "Due to Kubernetes's self-healing nature, the operations team doesn't need to do any manual intervention in case of a node or pod failure." Plus, he says, "Dev-Prod parity has improved since developers can experiment with options in dev/staging clusters, and when it's finalized, they just commit the config changes in the respective code repositories. These changes automatically get replicated on the production cluster via CI/CD pipelines."</p>

-</div>
-</section>
-<div class="banner3">
-  <div class="banner3text">
-    "We realized that we needed a more cloud-native approach to deal with these issues," says Singh. The team decided to implement a custom setup of Kubernetes based on Terraform and Ansible."
-  </div>
-</div>
-<section class="section3">
-<div class="fullcol">
-  There was another compelling business reason for the cloud-native approach. "In today’s world of ever-changing business requirements, using cloud native technology provides a variety of options to choose from—even the ability to run services in a hybrid cloud environment," says Singh. "Businesses can keep services in a region closest to the users, and thus benefit from high-availability and resiliency."<br><br>
-  So in February 2016, Singh set up a test Kubernetes cluster using the kube-up scripts provided. "I explored the features and was able to deploy an application pretty easily," he says. "However, it seemed like a black box since I didn’t understand the components completely, and had no idea what the kube-up script did under the hood. So when it broke, it was hard to find the issue and fix it." <br><br>
-  To get a better understanding, Singh dove into the internals of Kubernetes, reading the docs and even some of the code. And he looked to the Kubernetes community for more insight. "I used to stay up a little late every night (a lot of users were active only when it’s night here in India) and would try to answer questions on the Kubernetes community Slack from users who were getting started," he says. "I would also follow other conversations closely. I must admit I was able to avoid a lot of issues in our setup because I knew others had faced the same issues."<br><br>
-  Based on the knowledge he gained, Singh decided to implement a custom setup of Kubernetes based on <a href="https://www.terraform.io/">Terraform</a> and <a href="https://www.ansible.com/">Ansible</a>. "I wrote Terraform to launch Kubernetes master and nodes (Auto Scaling Groups) and an Ansible playbook to install the required components," he says. (The company recently switched to using prebaked <a href="http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIs.html">AMIs</a> to make the node bringup faster, and is planning to change its networking layer.) <br><br>
+{{< case-studies/quote author="Amanpreet Singh, Software Engineer at Crowdfire" >}}
+"In the 15 months that we've been using Kubernetes, it has been amazing for us. It enabled us to iterate quickly, increase development speed, and continuously deliver new features and bug fixes to our users, while keeping our operational costs and infrastructure management overhead under control."
+{{< /case-studies/quote >}}

-</div>
-</section>
-<div class="banner4">
-  <div class="banner4text">
-    "Kubernetes helped us reduce the deployment time from 15 minutes to less than a minute. Due to Kubernetes’s self-healing nature, the operations team doesn’t need to do any manual intervention in case of a node or pod failure."
-  </div>
-</div>
+{{< case-studies/lead >}}
+"If you build it, they will come."
+{{< /case-studies/lead >}}

-<section class="section4">
-<div class="fullcol">
-  First, the team migrated a few staging services from Elastic Beanstalk to the new Kubernetes staging cluster, and then set up a production cluster a month later to deploy some services. The results were convincing. "By the end of March 2016, we established that all the new services must be deployed on Kubernetes," says Singh. "Kubernetes helped us reduce the deployment time from 15 minutes to less than a minute. Due to Kubernetes’s self-healing nature, the operations team doesn’t need to do any manual intervention in case of a node or pod failure." On top of that, he says, "Dev-Prod parity has improved since developers can experiment with options in dev/staging clusters, and when it’s finalized, they just commit the config changes in the respective code repositories. These changes automatically get replicated on the production cluster via CI/CD pipelines. This brings more visibility into the changes being made, and keeping an audit trail."<br><br>
-  Over the next six months, the team worked on migrating all the services from Elastic Beanstalk to Kubernetes, except for the few that were deprecated and would soon be terminated anyway. The services were moved one at a time, and their performance was monitored for two to three days each. Today, "We’re completely migrated and we run all new services on Kubernetes," says Singh. <br><br>
-  The impact has been considerable: With Kubernetes, the company has experienced a 90% cost savings on Elastic Load Balancer, which is now only used for their public, user-facing services. Their EC2 operating expenses have been decreased by as much as 50%.<br><br>
-  All 30 engineers at Crowdfire were onboarded at once. "I gave an internal talk where I shared the basic components and demoed the usage of kubectl," says Singh. "Everyone was excited and happy about using Kubernetes. Developers have more control and visibility into their applications running in production now. Most of all, they’re happy with the low deploy times and self-healing services." <br><br>
-  And they’re much more productive, too. "Where we used to do about 5 deployments per day," says Singh, "now we’re doing 30+ production and 50+ staging deployments almost every day."
+<p>For most content creators, only half of that movie quote may ring true. Sure, platforms like Wordpress, YouTube and Shopify have made it simple for almost anyone to start publishing new content online, but attracting an audience isn't as easy. Crowdfire "helps users publish their content to all possible places where their audience exists," says Amanpreet Singh, a Software Engineer at the company based in Mumbai, India. Crowdfire has gained more than 16 million users—from bloggers and artists to makers and small businesses—since its launch in 2010.</p>

+<p>With that kind of growth—and a high demand from users for new features and continuous improvements—the Crowdfire team struggled to keep up behind the scenes. In 2015, they moved their monolith Java application to Amazon Web Services <a href="https://aws.amazon.com/elasticbeanstalk/">Elastic Beanstalk</a> and started breaking it down into microservices.</p>

-</div>
-</section>
-<div class="banner5">
-  <div class="banner5text">
-    The impact has been considerable: With Kubernetes, the company has experienced a 90% cost savings on Elastic Load Balancer, which is now only used for their public, user-facing services. Their EC2 operating expenses have been decreased by as much as 50%.
+<p>It was a good first step, but the team soon realized they needed to go further down the cloud-native path, which would lead them to Kubernetes. "It was okay for our use cases initially, but as the number of services and development teams increased and we scaled further, deploy times, self-healing capabilities and resource utilization started to become problematic," says Singh, who leads the infrastructure team at Crowdfire. "We realized that we needed a more cloud-native approach to deal with these issues."</p>

-  </div>
-</div>
-<section class="section5">
-<div class="fullcol">
+<p>As he looked around for solutions, Singh had a checklist of what Crowdfire needed. "We wanted to keep some things separate so they could be shipped independent of other things; this would help remove blockers and let different teams work at their own pace," he says. "We also make a lot of data-driven decisions, so shipping a feature and its iterations quickly was a must."</p>

-   Singh notes that almost all of the engineers interact with the staging cluster on a daily basis, and that has created a cultural change at Crowdfire. "Developers are more aware of the cloud infrastructure now," he says. "They’ve started following cloud best practices like better health checks, structured logs to stdout [standard output], and config via files or environment variables."<br><br>
-  With Crowdfire’s commitment to Kubernetes, Singh is looking to expand the company’s cloud-native stack. The team already uses <a href="https://prometheus.io/">Prometheus</a> for monitoring, and he says he is evaluating <a href="https://linkerd.io/">Linkerd</a> and <a href="https://envoyproxy.github.io/">Envoy Proxy</a> as a way to "get more metrics about request latencies and failures, and handle them better." Other CNCF projects, including <a href="http://opentracing.io/">OpenTracing</a> and <a href="https://grpc.io/">gRPC</a> are also on his radar.<br><br>
-  Singh has found that the cloud-native community is growing in India, too, particularly in Bangalore. "A lot of startups and new companies are starting to run their infrastructure on Kubernetes," he says. <br><br>
-  And when people ask him about Crowdfire’s experience, he has this advice to offer: "Kubernetes is a great piece of technology, but it might not be right for you, especially if you have just one or two services or your app isn’t easy to run in a containerized environment," he says. "Assess your situation and the value that Kubernetes provides before going all in. If you do decide to use Kubernetes, make sure you understand the components that run under the hood and what role they play in smoothly running the cluster. Another thing to consider is if your apps are ‘Kubernetes-ready,’ meaning if they have proper health checks and handle termination signals to shut down gracefully."<br><br>
-  And if your company fits that profile, go for it. Crowdfire clearly did—and is now reaping the benefits. "In the 15 months that we’ve been using Kubernetes, it has been amazing for us," says Singh. "It enabled us to iterate quickly, increase development speed and continuously deliver new features and bug fixes to our users, while keeping our operational costs and infrastructure management overhead under control."
+<p>Kubernetes checked all the boxes and then some. "One of the best things was the built-in service discovery," he says. "When you have a bunch of microservices that need to call each other, having internal DNS readily available and service IPs and ports automatically set as environment variables help a lot." Plus, he adds, "Kubernetes's opinionated approach made it easier to get started."</p>

+{{< case-studies/quote image="/images/case-studies/crowdfire/banner3.jpg" >}}
+"We realized that we needed a more cloud-native approach to deal with these issues," says Singh. The team decided to implement a custom setup of Kubernetes based on Terraform and Ansible."
+{{< /case-studies/quote >}}

-</div>
-</section>
+<p>There was another compelling business reason for the cloud-native approach. "In today's world of ever-changing business requirements, using cloud native technology provides a variety of options to choose from—even the ability to run services in a hybrid cloud environment," says Singh. "Businesses can keep services in a region closest to the users, and thus benefit from high-availability and resiliency."</p>
+
+<p>So in February 2016, Singh set up a test Kubernetes cluster using the kube-up scripts provided. "I explored the features and was able to deploy an application pretty easily," he says. "However, it seemed like a black box since I didn't understand the components completely, and had no idea what the kube-up script did under the hood. So when it broke, it was hard to find the issue and fix it."</p>
+
+<p>To get a better understanding, Singh dove into the internals of Kubernetes, reading the docs and even some of the code. And he looked to the Kubernetes community for more insight. "I used to stay up a little late every night (a lot of users were active only when it's night here in India) and would try to answer questions on the Kubernetes community Slack from users who were getting started," he says. "I would also follow other conversations closely. I must admit I was able to avoid a lot of issues in our setup because I knew others had faced the same issues."</p>
+
+<p>Based on the knowledge he gained, Singh decided to implement a custom setup of Kubernetes based on <a href="https://www.terraform.io/">Terraform</a> and <a href="https://www.ansible.com/">Ansible</a>. "I wrote Terraform to launch Kubernetes master and nodes (Auto Scaling Groups) and an Ansible playbook to install the required components," he says. (The company recently switched to using prebaked <a href="http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIs.html">AMIs</a> to make the node bringup faster, and is planning to change its networking layer.)</p>
+
+{{< case-studies/quote image="/images/case-studies/crowdfire/banner4.jpg" >}}
+"Kubernetes helped us reduce the deployment time from 15 minutes to less than a minute. Due to Kubernetes's self-healing nature, the operations team doesn't need to do any manual intervention in case of a node or pod failure."
+{{< /case-studies/quote >}}
+
+<p>First, the team migrated a few staging services from Elastic Beanstalk to the new Kubernetes staging cluster, and then set up a production cluster a month later to deploy some services. The results were convincing. "By the end of March 2016, we established that all the new services must be deployed on Kubernetes," says Singh. "Kubernetes helped us reduce the deployment time from 15 minutes to less than a minute. Due to Kubernetes's self-healing nature, the operations team doesn't need to do any manual intervention in case of a node or pod failure." On top of that, he says, "Dev-Prod parity has improved since developers can experiment with options in dev/staging clusters, and when it's finalized, they just commit the config changes in the respective code repositories. These changes automatically get replicated on the production cluster via CI/CD pipelines. This brings more visibility into the changes being made, and keeping an audit trail."</p>
+
+<p>Over the next six months, the team worked on migrating all the services from Elastic Beanstalk to Kubernetes, except for the few that were deprecated and would soon be terminated anyway. The services were moved one at a time, and their performance was monitored for two to three days each. Today, "We're completely migrated and we run all new services on Kubernetes," says Singh.</p>
+
+<p>The impact has been considerable: With Kubernetes, the company has experienced a 90% cost savings on Elastic Load Balancer, which is now only used for their public, user-facing services. Their EC2 operating expenses have been decreased by as much as 50%.</p>
+
+<p>All 30 engineers at Crowdfire were onboarded at once. "I gave an internal talk where I shared the basic components and demoed the usage of kubectl," says Singh. "Everyone was excited and happy about using Kubernetes. Developers have more control and visibility into their applications running in production now. Most of all, they're happy with the low deploy times and self-healing services."</p>
+
+<p>And they're much more productive, too. "Where we used to do about 5 deployments per day," says Singh, "now we're doing 30+ production and 50+ staging deployments almost every day."</p>
+
+{{< case-studies/quote >}}
+The impact has been considerable: With Kubernetes, the company has experienced a 90% cost savings on Elastic Load Balancer, which is now only used for their public, user-facing services. Their EC2 operating expenses have been decreased by as much as 50%.
+{{< /case-studies/quote >}}
+
+<p>Singh notes that almost all of the engineers interact with the staging cluster on a daily basis, and that has created a cultural change at Crowdfire. "Developers are more aware of the cloud infrastructure now," he says. "They've started following cloud best practices like better health checks, structured logs to stdout [standard output], and config via files or environment variables."</p>
+
+<p>With Crowdfire's commitment to Kubernetes, Singh is looking to expand the company's cloud-native stack. The team already uses <a href="https://prometheus.io/">Prometheus</a> for monitoring, and he says he is evaluating <a href="https://linkerd.io/">Linkerd</a> and <a href="https://envoyproxy.github.io/">Envoy Proxy</a> as a way to "get more metrics about request latencies and failures, and handle them better." Other CNCF projects, including <a href="http://opentracing.io/">OpenTracing</a> and <a href="https://grpc.io/">gRPC</a> are also on his radar.</p>
+
+<p>Singh has found that the cloud-native community is growing in India, too, particularly in Bangalore. "A lot of startups and new companies are starting to run their infrastructure on Kubernetes," he says.</p>
+
+<p>And when people ask him about Crowdfire's experience, he has this advice to offer: "Kubernetes is a great piece of technology, but it might not be right for you, especially if you have just one or two services or your app isn't easy to run in a containerized environment," he says. "Assess your situation and the value that Kubernetes provides before going all in. If you do decide to use Kubernetes, make sure you understand the components that run under the hood and what role they play in smoothly running the cluster. Another thing to consider is if your apps are 'Kubernetes-ready,' meaning if they have proper health checks and handle termination signals to shut down gracefully."</p>
+
+<p>And if your company fits that profile, go for it. Crowdfire clearly did—and is now reaping the benefits. "In the 15 months that we've been using Kubernetes, it has been amazing for us," says Singh. "It enabled us to iterate quickly, increase development speed and continuously deliver new features and bug fixes to our users, while keeping our operational costs and infrastructure management overhead under control."</p>
@@ -3,108 +3,80 @@ title: Denso Case Study
 linkTitle: Denso
 case_study_styles: true
 cid: caseStudies
-css: /css/case-studies-gradient.css
 logo: denso_featured_logo.svg
 featured: true
 weight: 4
 quote: >
  We got Kubernetes experts involved on our team, and it dramatically accelerated development speed.
+
+new_case_study_styles: true
+heading_background: /images/case-studies/denso/banner2.jpg
+heading_title_text: Denso
+use_gradient_overlay: true
+subheading: >
+  How DENSO Is Fueling Development on the Vehicle Edge with Kubernetes
+case_study_details:
+  - Company: Denso
+  - Location: Japan
+  - Industry: Automotive, Edge
 ---

+<h2>Challenge</h2>

-<div class="article image overlay" style="background-image: url('/images/case-studies/denso/banner2.jpg')">
-  <h1> CASE STUDY: Denso</h1>
-  <div class="subhead">How DENSO Is Fueling Development on the Vehicle Edge with Kubernetes</div>
-</div>
+<p>DENSO Corporation is one of the biggest automotive components suppliers in the world. With the advent of connected cars, the company launched a Digital Innovation Department to expand into software, working on vehicle edge and vehicle cloud products. But there were several technical challenges to creating an integrated vehicle edge/cloud platform: "the amount of computing resources, the occasional lack of mobile signal, and an enormous number of distributed vehicles," says R&D Product Manager Seiichi Koizumi.</p>

-<div class="details">
-    Company &nbsp;<b>Denso</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location &nbsp;<b>Japan</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Industry &nbsp;<b>Automotive, Edge</b>
-</div>
+<h2>Solution</h2>

-<hr>
-<section class="section1">
-<div class="colsgradient">
-  <div class="col1">
-    <h2>Challenge</h2>
-    DENSO Corporation is one of the biggest automotive components suppliers in the world. With the advent of connected cars, the company launched a Digital Innovation Department to expand into software, working on vehicle edge and vehicle cloud products. But there were several technical challenges to creating an integrated vehicle edge/cloud platform: "the amount of computing resources, the occasional lack of mobile signal, and an enormous number of distributed vehicles," says R&D Product Manager Seiichi Koizumi.
-
-
-    <h2>Solution</h2>
-    Koizumi’s team realized that because mobility services evolve every day, they needed the flexibility of the cloud native ecosystem for their platform. After considering other orchestrators, DENSO went with Kubernetes for orchestration and added Prometheus, Fluentd, Envoy, Istio, and Helm to the platform. Today, DENSO is using a vehicle edge computer, a private Kubernetes cloud, and managed Kubernetes (GKE, EKS, AKS).
+<p>Koizumi's team realized that because mobility services evolve every day, they needed the flexibility of the cloud native ecosystem for their platform. After considering other orchestrators, DENSO went with Kubernetes for orchestration and added Prometheus, Fluentd, Envoy, Istio, and Helm to the platform. Today, DENSO is using a vehicle edge computer, a private Kubernetes cloud, and managed Kubernetes (GKE, EKS, AKS).</p>

 <h2>Impact</h2>
-    Critical layer features can take 2-3 years to implement in the traditional, waterfall model of development at DENSO. With the Kubernetes platform and agile methods, there’s a 2-month development cycle for non-critical software. Now, ten new applications are released a year, and a new prototype is introduced every week. "By utilizing Kubernetes managed services, such as GKE/EKS/AKS, we can unify the environment and simplify our maintenance operation," says Koizumi.
-</div>
-</section>
-<div class="article image overlay" style="background-image: url('/images/case-studies/denso/banner1.png');">
-  <div class="quotetext">
-    "Another disruptive innovation is coming, so to survive in this situation, we need to change our culture."
-<p><div class="quoteauthortext">- SEIICHI KOIZUMI, R&D PRODUCT MANAGER, DIGITAL INNOVATION DEPARTMENT AT DENSO</div></p>
-  </div>
-</div>

+<p>Critical layer features can take 2-3 years to implement in the traditional, waterfall model of development at DENSO. With the Kubernetes platform and agile methods, there's a 2-month development cycle for non-critical software. Now, ten new applications are released a year, and a new prototype is introduced every week. "By utilizing Kubernetes managed services, such as GKE/EKS/AKS, we can unify the environment and simplify our maintenance operation," says Koizumi.</p>

-<section class="section2">
-<div class="fullcol">
-  <h4>Spun off from Toyota in 1949, DENSO Corporation is one of the top automotive suppliers in the world today, with consolidated net revenue of $48.3 billion.
-</h4>
+{{< case-studies/quote
+  image="/images/case-studies/denso/banner1.png"
+  author="SEIICHI KOIZUMI, R&D PRODUCT MANAGER, DIGITAL INNOVATION DEPARTMENT AT DENSO"
+>}}
+"Another disruptive innovation is coming, so to survive in this situation, we need to change our culture."
+{{< /case-studies/quote >}}

-  <p>The company’s mission is "contributing to a better world by creating value together with a vision for the future"—and part of that vision in recent years has been development on the vehicle edge and vehicle cloud.</p>
+{{< case-studies/lead >}}
+Spun off from Toyota in 1949, DENSO Corporation is one of the top automotive suppliers in the world today, with consolidated net revenue of $48.3 billion.
+{{< /case-studies/lead >}}

-  <p>With the advent of connected cars, DENSO established a Digital Innovation Department to expand its business beyond the critical layer of the engine, braking systems, and other automotive parts into the non-critical analytics and entertainment layer. Comparing connected cars to smartphones, R&D Product Manager Seiichi Koizumi says DENSO wants the ability to quickly and easily develop and install apps for the "blank slate" of the car, and iterate them based on the driver’s preferences. Thus "we need a flexible application platform," he says. </p>
+<p>The company's mission is "contributing to a better world by creating value together with a vision for the future"—and part of that vision in recent years has been development on the vehicle edge and vehicle cloud.</p>
+
+<p>With the advent of connected cars, DENSO established a Digital Innovation Department to expand its business beyond the critical layer of the engine, braking systems, and other automotive parts into the non-critical analytics and entertainment layer. Comparing connected cars to smartphones, R&D Product Manager Seiichi Koizumi says DENSO wants the ability to quickly and easily develop and install apps for the "blank slate" of the car, and iterate them based on the driver's preferences. Thus "we need a flexible application platform," he says.</p>

 <p>But working on vehicle edge and vehicle cloud products meant there were several technical challenges: "the amount of computing resources, the occasional lack of mobile signal, and an enormous number of distributed vehicles," says Koizumi. "We are tackling these challenges to create an integrated vehicle edge/cloud platform."</p>

+{{< case-studies/quote author="SEIICHI KOIZUMI, R&D PRODUCT MANAGER, DIGITAL INNOVATION DEPARTMENT AT DENSO" >}}
+"We got Kubernetes experts involved on our team, and it dramatically accelerated development speed."
+{{< /case-studies/quote >}}

-</div>
-</section>
+<p>Koizumi's team realized that because mobility services evolve every day, they needed the flexibility of the cloud native ecosystem for their platform. As they evaluated technologies, they were led by these criteria: Because their service-enabler business needed to support multiple cloud and on-premise environments, the solution needed to be cloud agnostic, with no vendor lock-in and open governance. It also had to support an edge-cloud integrated environment.</p>

-<div class="greybanner">
-  <div class="greyquotetext">
-    "We got Kubernetes experts involved on our team, and it dramatically accelerated development speed."<p><div class="quoteauthortext">— SEIICHI KOIZUMI, R&D PRODUCT MANAGER, DIGITAL INNOVATION DEPARTMENT AT DENSO</div></p>
-  </div>
-</div>
+<p>After considering other orchestrators, DENSO went with Kubernetes for orchestration and added Prometheus, Fluentd, Envoy, Istio, and Helm to the platform. During implementation, the team used "design thinking to clarify use cases and their value proposition," says Koizumi. Next, an agile development team worked on a POC, then an MVP, in DevOps style. "Even in the development phase, we are keeping a channel to end users," he adds.</p>

-<section class="section3">
-<div class="fullcol">
-   <p>
-    Koizumi’s team realized that because mobility services evolve every day, they needed the flexibility of the cloud native ecosystem for their platform. As they evaluated technologies, they were led by these criteria: Because their service-enabler business needed to support multiple cloud and on-premise environments, the solution needed to be cloud agnostic, with no vendor lock-in and open governance. It also had to support an edge-cloud integrated environment.</p>
- <p>
-   After considering other orchestrators, DENSO went with Kubernetes for orchestration and added Prometheus, Fluentd, Envoy, Istio, and Helm to the platform. During implementation, the team used "design thinking to clarify use cases and their value proposition," says Koizumi. Next, an agile development team worked on a POC, then an MVP, in DevOps style. "Even in the development phase, we are keeping a channel to end users," he adds. </p>
-  <p>
-  One lesson learned during this process was the value of bringing in experts. "We tried to learn Kubernetes and cloud native technologies from scratch, but it took more time than expected," says Koizumi. "We got Kubernetes experts involved on our team, and it dramatically accelerated development speed."</p>
+<p>One lesson learned during this process was the value of bringing in experts. "We tried to learn Kubernetes and cloud native technologies from scratch, but it took more time than expected," says Koizumi. "We got Kubernetes experts involved on our team, and it dramatically accelerated development speed."</p>

-  </p>
-</div>
-</section>
+{{< case-studies/quote
+  image="/images/case-studies/denso/banner4.jpg"
+  author="SEIICHI KOIZUMI, R&D PRODUCT MANAGER, DIGITAL INNOVATION DEPARTMENT AT DENSO"
+>}}
+"By utilizing Kubernetes managed services, such as GKE/EKS/AKS, we can unify the environment and simplify our maintenance operation."
+{{< /case-studies/quote >}}

+<p>Today, DENSO is using a vehicle edge computer, a private Kubernetes cloud, and managed Kubernetes on GKE, EKS, and AKS. "We are developing a vehicle edge/cloud integrated platform based on a microservice and service mesh architecture," says Koizumi. "We extend cloud into multiple vehicle edges and manage it as a unified platform."</p>

-  <div class="article image overlay" style="background-image: url('/images/case-studies/denso/banner4.jpg');">
-  <div class="quotetext">
- "By utilizing Kubernetes managed services, such as GKE/EKS/AKS, we can unify the environment and simplify our maintenance operation." <p><div class="quoteauthortext">- SEIICHI KOIZUMI, R&D PRODUCT MANAGER, DIGITAL INNOVATION DEPARTMENT AT DENSO</div></p>
-  </div>
-</div>
+<p>Cloud native has enabled DENSO to deliver applications via its new dash cam, which has a secure connection that collects data to the cloud. "It's like a smartphone," he says. "We are installing new applications and getting the data through the cloud, and we can keep updating new applications all through the dash cam."</p>

-<section class="section5">
-<div class="fullcol">
-   <p>
-  Today, DENSO is using a vehicle edge computer, a private Kubernetes cloud, and managed Kubernetes on GKE, EKS, and AKS. "We are developing a vehicle edge/cloud integrated platform based on a microservice and service mesh architecture," says Koizumi. "We extend cloud into multiple vehicle edges and manage it as a unified platform."</p>
-  <p>
-  Cloud native has enabled DENSO to deliver applications via its new dash cam, which has a secure connection that collects data to the cloud. "It’s like a smartphone," he says. "We are installing new applications and getting the data through the cloud, and we can keep updating new applications all through the dash cam."</p>
-  <p>
- The unified cloud native platform, combined with agile development, has had a positive impact on productivity. Critical layer features—those involving engines or braking systems, for example—can take 2-3 years to implement at DENSO, because of the time needed to test safety, but also because of the traditional, waterfall model of development. With the Kubernetes platform and agile methods, there’s a 2-month development cycle for non-critical software. Now, ten new applications are released a year, and with the department’s scrum-style development, a new prototype is introduced every week. </p>
-  <p>
- Application portability has also led to greater developer efficiency. "There’s no need to care about differences in the multi-cloud platform anymore," says Koizumi. Now, "we are also trying to have the same portability between vehicle edge and cloud platform."
-  </p>
-  <p>
-  Another improvement: Automotive Tier-1 suppliers like DENSO always have multiple Tier-2 suppliers. "To provide automotive-grade high-availability services, we tried to do the same thing on a multi-cloud platform," says Koizumi. Before Kubernetes, maintaining two different systems simultaneously was difficult. "By utilizing Kubernetes managed services, such as GKE/EKS/AKS, we can unify the environment and simplify our maintenance operation," he says.
-</p>
-<p>
-Cloud native has also profoundly changed the culture at DENSO. The Digital Innovation Department is known as "Noah’s Ark," and it has grown from 2 members to 70—with plans to more than double in the next year. The way they operate is completely different from the traditional Japanese automotive culture. But just as the company embraced change brought by hybrid cars in the past decade, Koizumi says, they’re doing it again now, as technology companies have moved into the connected car space. "Another disruptive innovation is coming," he says, "so to survive in this situation, we need to change our culture."
-</p>
-<p>
-Looking ahead, Koizumi and his team are expecting serverless and zero-trust security architecture to be important enhancements of Kubernetes. They are glad DENSO has come along for the ride. "Mobility service businesses require agility and flexibility," he says. "DENSO is trying to bring cloud native flexibility into the vehicle infrastructure."
-</p>
-</div>
-</section>
-</body>
+<p>The unified cloud native platform, combined with agile development, has had a positive impact on productivity. Critical layer features—those involving engines or braking systems, for example—can take 2-3 years to implement at DENSO, because of the time needed to test safety, but also because of the traditional, waterfall model of development. With the Kubernetes platform and agile methods, there's a 2-month development cycle for non-critical software. Now, ten new applications are released a year, and with the department's scrum-style development, a new prototype is introduced every week.</p>
+
+<p>Application portability has also led to greater developer efficiency. "There's no need to care about differences in the multi-cloud platform anymore," says Koizumi. Now, "we are also trying to have the same portability between vehicle edge and cloud platform."</p>
+
+<p>Another improvement: Automotive Tier-1 suppliers like DENSO always have multiple Tier-2 suppliers. "To provide automotive-grade high-availability services, we tried to do the same thing on a multi-cloud platform," says Koizumi. Before Kubernetes, maintaining two different systems simultaneously was difficult. "By utilizing Kubernetes managed services, such as GKE/EKS/AKS, we can unify the environment and simplify our maintenance operation," he says.</p>
+
+<p>Cloud native has also profoundly changed the culture at DENSO. The Digital Innovation Department is known as "Noah's Ark," and it has grown from 2 members to 70—with plans to more than double in the next year. The way they operate is completely different from the traditional Japanese automotive culture. But just as the company embraced change brought by hybrid cars in the past decade, Koizumi says, they're doing it again now, as technology companies have moved into the connected car space. "Another disruptive innovation is coming," he says, "so to survive in this situation, we need to change our culture."</p>
+
+<p>Looking ahead, Koizumi and his team are expecting serverless and zero-trust security architecture to be important enhancements of Kubernetes. They are glad DENSO has come along for the ride. "Mobility service businesses require agility and flexibility," he says. "DENSO is trying to bring cloud native flexibility into the vehicle infrastructure."</p>
@@ -1,125 +1,89 @@
 ---
 title: GolfNow Case Study
-
 case_study_styles: true
 cid: caseStudies
-css: /css/style_golfnow.css
+
+new_case_study_styles: true
+heading_background: /images/case-studies/golfnow/banner1.jpg
+heading_title_logo: /images/golfnow_logo.png
+subheading: >
+  Saving Time and Money with Cloud Native Infrastructure
+case_study_details:
+  - Company: GolfNow
+  - Location: Orlando, Florida
+  - Industry: Golf Industry Technology and Services Provider
 ---

-<div class="banner1">
-  <h1>CASE STUDY: <img src="/images/golfnow_logo.png" width="20%" style="margin-bottom:-6px"><br>
-    <div class="subhead">Saving Time and Money with Cloud Native Infrastructure</div>
-  </h1>
-</div>
+<h2>Challenge</h2>

-<div class="details">
-    Company&nbsp;<b>GolfNow</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location&nbsp;<b>Orlando, Florida</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Industry&nbsp;<b>Golf Industry Technology and Services Provider</b>
-</div>
+<p>A member of the <a href="http://www.nbcunicareers.com/our-businesses/nbc-sports-group">NBC Sports Group</a>, <a href="https://www.golfnow.com/">GolfNow</a> is the golf industry's technology and services leader, managing 10 different products, as well as the largest e-commerce tee time marketplace in the world. As its business began expanding rapidly and globally, GolfNow's monolithic application became problematic. "We kept growing our infrastructure vertically rather than horizontally, and the cost of doing business became problematic," says Sheriff Mohamed, GolfNow's Director, Architecture. "We wanted the ability to more easily expand globally."</p>

-<hr>
+<h2>Solution</h2>

-<section class="section1">
-  <div class="cols">
-    <div class="col1">
+<p>Turning to microservices and containerization, GolfNow began moving its applications and databases from third-party services to its own clusters running on <a href="https://www.docker.com/">Docker</a> and <a href="http://kubernetes.io/">Kubernetes.</a></p>

-    <h2>Challenge</h2>
-      A member of the <a href="http://www.nbcunicareers.com/our-businesses/nbc-sports-group">NBC Sports Group</a>, <a href="https://www.golfnow.com/">GolfNow</a> is the golf industry’s technology and services leader, managing 10 different products, as well as the largest e-commerce tee time marketplace in the world. As its business began expanding rapidly and globally, GolfNow’s monolithic application became problematic. "We kept growing our infrastructure vertically rather than horizontally, and the cost of doing business became problematic," says Sheriff Mohamed, GolfNow’s Director, Architecture. "We wanted the ability to more easily expand globally."
-      <br>
-  </div>
+<h2>Impact</h2>

-  <div class="col2">
-    <h2>Solution</h2>
-      Turning to microservices and containerization, GolfNow began moving its applications and databases from third-party services to its own clusters running on <a href="https://www.docker.com/">Docker</a> and <a href="http://kubernetes.io/">Kubernetes.</a><br><br>
+<p>The results were immediate. While maintaining the same capacity—and beyond, during peak periods—GolfNow saw its infrastructure costs for the first application virtually cut in half.</p>

-    <h2>Impact</h2>
-      The results were immediate. While maintaining the same capacity—and beyond, during peak periods—GolfNow saw its infrastructure costs for the first application virtually cut in half.
-    </div>
-  </div>
-</section>
+{{< case-studies/quote author="SHERIFF MOHAMED, DIRECTOR, ARCHITECTURE AT GOLFNOW" >}}
+"With our growth we obviously needed to expand our infrastructure, and we kept growing vertically rather than horizontally. We were basically wasting money and doubling the cost of our infrastructure."
+{{< /case-studies/quote >}}

+{{< case-studies/lead >}}
+It's not every day that you can say you've slashed an operating expense by half.
+{{< /case-studies/lead >}}

-<div class="banner2">
-  <div class="banner2text">
-      "With our growth we obviously needed to expand our infrastructure, and we kept growing vertically rather than horizontally. We were basically wasting money and doubling the cost of our infrastructure."<br><br><span style="font-size:15px;letter-spacing:0.08em">- SHERIFF MOHAMED, DIRECTOR, ARCHITECTURE AT GOLFNOW</span>
-  </div>
-</div>
+<p>But Sheriff Mohamed and Josh Chandler did just that when they helped lead their company, <a href="https://www.golfnow.com/">GolfNow</a>, on a journey from a monolithic to a containerized, cloud native infrastructure managed by Kubernetes.</p>

-<section class="section2">
-  <div class="fullcol">
-      <h2>It’s not every day that you can say you’ve slashed an operating expense by half.</h2>
+<p>A top-performing business within the NBC Sports Group, GolfNow is a technology and services company with the largest tee time marketplace in the world. GolfNow serves 5 million active golfers across 10 different products. In recent years, the business had grown so fast that the infrastructure supporting their giant monolithic application (written in C#.NET and backed by SQL Server database management system) could not keep up. "With our growth we obviously needed to expand our infrastructure, and we kept growing vertically rather than horizontally," says Sheriff, GolfNow's Director, Architecture. "Our costs were growing exponentially. And on top of that, we had to build a Disaster Recovery (DR) environment, which then meant we'd have to copy exactly what we had in our original data center to another data center that was just the standby. We were basically wasting money and doubling the cost of our infrastructure."</p>

-      But Sheriff Mohamed and Josh Chandler did just that when they helped lead their company, <a href="https://www.golfnow.com/">GolfNow</a>, on a journey from a monolithic to a containerized, cloud native infrastructure managed by Kubernetes.
-      <br> <br>
-      A top-performing business within the NBC Sports Group, GolfNow is a technology and services company with the largest tee time marketplace in the world. GolfNow serves 5 million active golfers across 10 different products. In recent years, the business had grown so fast that the infrastructure supporting their giant monolithic application (written in C#.NET and backed by SQL Server database management system) could not keep up. "With our growth we obviously needed to expand our infrastructure, and we kept growing vertically rather than horizontally," says Sheriff, GolfNow’s Director, Architecture. "Our costs were growing exponentially. And on top of that, we had to build a Disaster Recovery (DR) environment, which then meant we’d have to copy exactly what we had in our original data center to another data center that was just the standby. We were basically wasting money and doubling the cost of our infrastructure."
-      <br> <br>
-      In moving just the first of GolfNow’s important applications—a booking engine for golf courses and B2B marketing platform—from third-party services to their own Kubernetes environment, "our bill went down drastically," says Sheriff.
-      <br> <br>
-      The path to those stellar results began in late 2014. In order to support GolfNow’s global growth, the team decided that the company needed to have multiple data centers and the ability to quickly and easily re-route traffic as needed. "From there we knew that we needed to go in a direction of breaking things apart, microservices, and containerization," says Sheriff. "At the time we were trying to get away from <a href="https://www.microsoft.com/net">C#.NET</a> and <a href="https://www.microsoft.com/en-cy/sql-server/sql-server-2016">SQL Server</a> since it didn’t run very well on Linux, where everything container was running smoothly."
-      <br> <br>
-      To that end, the team shifted to working with <a href="https://nodejs.org/">Node.js</a>, the open-source, cross-platform JavaScript runtime environment for developing tools and applications, and <a href="https://www.mongodb.com/">MongoDB</a>, the open-source database program. At the time, <a href="https://www.docker.com/">Docker</a>, the platform for deploying applications in containers, was still new. But once the team began experimenting with it, Sheriff says, "we realized that was the way we wanted to go, especially since that’s the way the industry is heading."
-  </div>
-</section>
+<p>In moving just the first of GolfNow's important applications—a booking engine for golf courses and B2B marketing platform—from third-party services to their own Kubernetes environment, "our bill went down drastically," says Sheriff.</p>

-<div class="banner3">
-  <div class="banner3text">
-    "The team migrated the rest of the application into their Kubernetes cluster. And the impact was immediate: On top of cutting monthly costs by a large percentage, says Sheriff, 'Running at the same capacity and during our peak time, we were able to horizontally grow. Since we were using our VMs more efficiently with containers, we didn’t have to pay extra money at all.'"
-  </div>
-</div>
+<p>The path to those stellar results began in late 2014. In order to support GolfNow's global growth, the team decided that the company needed to have multiple data centers and the ability to quickly and easily re-route traffic as needed. "From there we knew that we needed to go in a direction of breaking things apart, microservices, and containerization," says Sheriff. "At the time we were trying to get away from <a href="https://www.microsoft.com/net">C#.NET</a> and <a href="https://www.microsoft.com/en-cy/sql-server/sql-server-2016">SQL Server</a> since it didn't run very well on Linux, where everything container was running smoothly."</p>

-<section class="section3">
-  <div class="fullcol">
-    GolfNow’s dev team ran an "internal, low-key" proof of concept and were won over. "We really liked how easy it was to be able to pass containers around to each other and have them up and running in no time, exactly the way it was running on my machine," says Sheriff. "Because that is always the biggest gripe that Ops has with developers, right? ‘It worked on my machine!’ But then we started getting to the point of, ‘How do we make sure that these things stay up and running?’" <br><br>
-    That led the team on a quest to find the right orchestration system for the company’s needs. Sheriff says the first few options they tried were either too heavy or "didn’t feel quite right." In late summer 2015, they discovered the just-released <a href="http://kubernetes.io/">Kubernetes</a>, which Sheriff immediately liked for its ease of use. "We did another proof of concept," he says, "and Kubernetes won because of the fact that the community backing was there, built on top of what Google had already done."
-    <br><br>
-    But before they could go with Kubernetes, <a href="http://www.nbc.com/">NBC</a>, GolfNow’s parent company, also asked them to comparison shop with another company. Sheriff and his team liked the competing company’s platform user interface, but didn’t like that its platform would not allow containers to run natively on Docker. With no clear decision in sight, Sheriff’s VP at GolfNow, Steve McElwee, set up a three-month trial during which a GolfNow team (consisting of Sheriff and Josh, who’s now Lead Architect, Open Platforms) would build out a Kubernetes environment, and a large NBC team would build out one with the other company’s platform.
-    <br><br>
-    "We spun up the cluster and we tried to get everything to run the way we wanted it to run," Sheriff says. "The biggest thing that we took away from it is that not only did we want our applications to run within Kubernetes and Docker, we also wanted our databases to run there. We literally wanted our entire infrastructure to run within Kubernetes."
-    <br><br>
-    At the time there was nothing in the community to help them get Kafka and MongoDB clusters running within a Kubernetes and Docker environment, so Sheriff and Josh figured it out on their own, taking a full month to get it right. "Everything started rolling from there," Sheriff says. "We were able to get all our applications connected, and we finished our side of the proof of concept a month in advance. My VP was like, ‘Alright, it’s over. Kubernetes wins.’"
-    <br><br>
-    The next step, beginning in January 2016, was getting everything working in production. The team focused first on one application that was already written in Node.js and MongoDB. A booking engine for golf courses and B2B marketing platform, the application was already going in the microservice direction but wasn’t quite finished yet. At the time, it was running in <a href="https://devcenter.heroku.com/articles/mongohq">Heroku Compose</a> and other third-party services—resulting in a large monthly bill.
+<p>To that end, the team shifted to working with <a href="https://nodejs.org/">Node.js</a>, the open-source, cross-platform JavaScript runtime environment for developing tools and applications, and <a href="https://www.mongodb.com/">MongoDB</a>, the open-source database program. At the time, <a href="https://www.docker.com/">Docker</a>, the platform for deploying applications in containers, was still new. But once the team began experimenting with it, Sheriff says, "we realized that was the way we wanted to go, especially since that's the way the industry is heading."</p>

-  </div>
-</section>
+{{< case-studies/quote image="/images/case-studies/golfnow/banner3.jpg" >}}
+"The team migrated the rest of the application into their Kubernetes cluster. And the impact was immediate: On top of cutting monthly costs by a large percentage, says Sheriff, 'Running at the same capacity and during our peak time, we were able to horizontally grow. Since we were using our VMs more efficiently with containers, we didn't have to pay extra money at all.'"
+{{< /case-studies/quote >}}

-  <div class="banner4">
-    <div class="banner4text">
-    "'The time I spent actually moving the applications was under 30 seconds! We can move data centers in just incredible amounts of time. If you haven’t come from the Kubernetes world you wouldn’t believe me.' Sheriff puts it in these terms: 'Before Kubernetes I wasn’t sleeping at night, literally. I was woken up all the time, because things were down. After Kubernetes, I’ve been sleeping at night.'"
-    </div>
-  </div>
+<p>GolfNow's dev team ran an "internal, low-key" proof of concept and were won over. "We really liked how easy it was to be able to pass containers around to each other and have them up and running in no time, exactly the way it was running on my machine," says Sheriff. "Because that is always the biggest gripe that Ops has with developers, right? 'It worked on my machine!' But then we started getting to the point of, 'How do we make sure that these things stay up and running?'"</p>

-<section class="section4">
-  <div class="fullcol">
-    "The goal was to take all of that out and put it within this new platform we’ve created with Kubernetes on <a href="https://cloud.google.com/compute/">Google Compute Engine (GCE)</a>," says Sheriff. "So we ended up building piece by piece, in parallel, what was out in Heroku and Compose, in our Kubernetes cluster. Then, literally, just switched configs in the background. So in Heroku we had the app running hitting a Compose database. We’d take the config, change it and make it hit the database that was running in our cluster."
-    <br><br>
-    Using this procedure, they were able to migrate piecemeal, without any downtime. The first migration was done during off hours, but to test the limits, the team migrated the second database in the middle of the day, when lots of users were running the application. "We did it," Sheriff says, "and again it was successful. Nobody noticed."
-    <br><br>
-    After three weeks of monitoring to make sure everything was running stable, the team migrated the rest of the application into their Kubernetes cluster. And the impact was immediate: On top of cutting monthly costs by a large percentage, says Sheriff, "Running at the same capacity and during our peak time, we were able to horizontally grow. Since we were using our VMs more efficiently with containers, we didn’t have to pay extra money at all."
-    <br><br>
-    Not only were they saving money, but they were also saving time. "I had a meeting this morning about migrating some applications from one cluster to another," says Josh. "I spent about 2 hours explaining the process. The time I spent actually moving the applications was under 30 seconds! We can move data centers in just incredible amounts of time. If you haven’t come from the Kubernetes world you wouldn’t believe me." Sheriff puts it in these terms: "Before Kubernetes I wasn’t sleeping at night, literally. I was woken up all the time, because things were down. After Kubernetes, I’ve been sleeping at night."
-    <br><br>
-    A small percentage of the applications on GolfNow have been migrated over to the Kubernetes environment. "Our Core Team is rewriting a lot of the .NET applications into <a href="https://www.microsoft.com/net/core">.NET Core</a> [which is compatible with Linux and Docker] so that we can run them within containers," says Sheriff.
-    <br><br>
-    Looking ahead, Sheriff and his team want to spend 2017 continuing to build a whole platform around Kubernetes with <a href="https://github.com/drone/drone">Drone</a>, an open-source continuous delivery platform, to make it more developer-centric. "Now they’re able to manage configuration, they’re able to manage their deployments and things like that, making all these subteams that are now creating all these microservices, be self sufficient," he says. "So it can pull us away from applications and allow us to just make sure the cluster is running and healthy, and then actually migrate that over to our Ops team."
+<p>That led the team on a quest to find the right orchestration system for the company's needs. Sheriff says the first few options they tried were either too heavy or "didn't feel quite right." In late summer 2015, they discovered the just-released <a href="http://kubernetes.io/">Kubernetes</a>, which Sheriff immediately liked for its ease of use. "We did another proof of concept," he says, "and Kubernetes won because of the fact that the community backing was there, built on top of what Google had already done."</p>

-  </div>
-</section>
+<p>But before they could go with Kubernetes, <a href="http://www.nbc.com/">NBC</a>, GolfNow's parent company, also asked them to comparison shop with another company. Sheriff and his team liked the competing company's platform user interface, but didn't like that its platform would not allow containers to run natively on Docker. With no clear decision in sight, Sheriff's VP at GolfNow, Steve McElwee, set up a three-month trial during which a GolfNow team (consisting of Sheriff and Josh, who's now Lead Architect, Open Platforms) would build out a Kubernetes environment, and a large NBC team would build out one with the other company's platform.</p>

-  <div class="banner5">
-    <div class="banner5text">
-    "Having gone from complete newbies to production-ready in three months, the GolfNow team is eager to encourage other companies to follow their lead. 'This is The Six Million Dollar Man of the cloud right now,' adds Josh. 'Just try it out, watch it happen. I feel like the proof is in the pudding when you look at these kinds of application stacks. They’re faster, they’re more resilient.'"
-    </div>
-  </div>
+<p>"We spun up the cluster and we tried to get everything to run the way we wanted it to run," Sheriff says. "The biggest thing that we took away from it is that not only did we want our applications to run within Kubernetes and Docker, we also wanted our databases to run there. We literally wanted our entire infrastructure to run within Kubernetes."</p>

-<section class="section5">
-  <div class="fullcol">
-    And long-term, Sheriff has an even bigger goal for getting more people into the Kubernetes fold. "We’re actually trying to make this platform generic enough so that any of our sister companies can use it if they wish," he says. "Most definitely I think it can be used as a model. I think the way we migrated into it, the way we built it out, are all ways that I think other companies can learn from, and should not be afraid of."
-    <br><br>
-    The GolfNow team is also giving back to the Kubernetes community by open-sourcing a bot framework that Josh built. "We noticed that the dashboard user interface is actually moving a lot faster than when we started," says Sheriff. "However we realized what we needed was something that’s more of a bot that really helps us administer Kubernetes as a whole through Slack." Josh explains: "With the Kubernetes-Slack integration, you can essentially hook into a cluster and the issue commands and edit configurations. We’ve tried to simplify the security configuration as much as possible. We hope this will be our major thank you to Kubernetes, for everything you’ve given us."
-    <br><br>
-    Having gone from complete newbies to production-ready in three months, the GolfNow team is eager to encourage other companies to follow their lead. The lessons they’ve learned: "You’ve got to have buy-in from your boss," says Sheriff. "Another big deal is having two to three people dedicated to this type of endeavor. You can’t have people who are half in, half out." And if you don’t have buy-in from the get go, proving it out will get you there.
-    <br><br>
-    "This is The Six Million Dollar Man of the cloud right now," adds Josh. "Just try it out, watch it happen. I feel like the proof is in the pudding when you look at these kinds of application stacks. They’re faster, they’re more resilient."
+<p>At the time there was nothing in the community to help them get Kafka and MongoDB clusters running within a Kubernetes and Docker environment, so Sheriff and Josh figured it out on their own, taking a full month to get it right. "Everything started rolling from there," Sheriff says. "We were able to get all our applications connected, and we finished our side of the proof of concept a month in advance. My VP was like, 'Alright, it's over. Kubernetes wins.'"</p>

-  </div>
-</section>
+<p>The next step, beginning in January 2016, was getting everything working in production. The team focused first on one application that was already written in Node.js and MongoDB. A booking engine for golf courses and B2B marketing platform, the application was already going in the microservice direction but wasn't quite finished yet. At the time, it was running in <a href="https://devcenter.heroku.com/articles/mongohq">Heroku Compose</a> and other third-party services—resulting in a large monthly bill.</p>
+
+{{< case-studies/quote image="/images/case-studies/golfnow/banner4.jpg" >}}
+"'The time I spent actually moving the applications was under 30 seconds! We can move data centers in just incredible amounts of time. If you haven't come from the Kubernetes world you wouldn't believe me.' Sheriff puts it in these terms: 'Before Kubernetes I wasn't sleeping at night, literally. I was woken up all the time, because things were down. After Kubernetes, I've been sleeping at night.'"
+{{< /case-studies/quote >}}
+
+<p>"The goal was to take all of that out and put it within this new platform we've created with Kubernetes on <a href="https://cloud.google.com/compute/">Google Compute Engine (GCE)</a>," says Sheriff. "So we ended up building piece by piece, in parallel, what was out in Heroku and Compose, in our Kubernetes cluster. Then, literally, just switched configs in the background. So in Heroku we had the app running hitting a Compose database. We'd take the config, change it and make it hit the database that was running in our cluster."</p>
+
+<p>Using this procedure, they were able to migrate piecemeal, without any downtime. The first migration was done during off hours, but to test the limits, the team migrated the second database in the middle of the day, when lots of users were running the application. "We did it," Sheriff says, "and again it was successful. Nobody noticed."</p>
+
+<p>After three weeks of monitoring to make sure everything was running stable, the team migrated the rest of the application into their Kubernetes cluster. And the impact was immediate: On top of cutting monthly costs by a large percentage, says Sheriff, "Running at the same capacity and during our peak time, we were able to horizontally grow. Since we were using our VMs more efficiently with containers, we didn't have to pay extra money at all."</p>
+
+<p>Not only were they saving money, but they were also saving time. "I had a meeting this morning about migrating some applications from one cluster to another," says Josh. "I spent about 2 hours explaining the process. The time I spent actually moving the applications was under 30 seconds! We can move data centers in just incredible amounts of time. If you haven't come from the Kubernetes world you wouldn't believe me." Sheriff puts it in these terms: "Before Kubernetes I wasn't sleeping at night, literally. I was woken up all the time, because things were down. After Kubernetes, I've been sleeping at night."</p>
+
+<p>A small percentage of the applications on GolfNow have been migrated over to the Kubernetes environment. "Our Core Team is rewriting a lot of the .NET applications into <a href="https://www.microsoft.com/net/core">.NET Core</a> [which is compatible with Linux and Docker] so that we can run them within containers," says Sheriff.</p>
+
+<p>Looking ahead, Sheriff and his team want to spend 2017 continuing to build a whole platform around Kubernetes with <a href="https://github.com/drone/drone">Drone</a>, an open-source continuous delivery platform, to make it more developer-centric. "Now they're able to manage configuration, they're able to manage their deployments and things like that, making all these subteams that are now creating all these microservices, be self sufficient," he says. "So it can pull us away from applications and allow us to just make sure the cluster is running and healthy, and then actually migrate that over to our Ops team."</p>
+
+{{< case-studies/quote >}}
+"Having gone from complete newbies to production-ready in three months, the GolfNow team is eager to encourage other companies to follow their lead. 'This is The Six Million Dollar Man of the cloud right now,' adds Josh. 'Just try it out, watch it happen. I feel like the proof is in the pudding when you look at these kinds of application stacks. They're faster, they're more resilient.'"
+{{< /case-studies/quote >}}
+
+<p>And long-term, Sheriff has an even bigger goal for getting more people into the Kubernetes fold. "We're actually trying to make this platform generic enough so that any of our sister companies can use it if they wish," he says. "Most definitely I think it can be used as a model. I think the way we migrated into it, the way we built it out, are all ways that I think other companies can learn from, and should not be afraid of."</p>
+
+<p>The GolfNow team is also giving back to the Kubernetes community by open-sourcing a bot framework that Josh built. "We noticed that the dashboard user interface is actually moving a lot faster than when we started," says Sheriff. "However we realized what we needed was something that's more of a bot that really helps us administer Kubernetes as a whole through Slack." Josh explains: "With the Kubernetes-Slack integration, you can essentially hook into a cluster and the issue commands and edit configurations. We've tried to simplify the security configuration as much as possible. We hope this will be our major thank you to Kubernetes, for everything you've given us."</p>
+
+<p>Having gone from complete newbies to production-ready in three months, the GolfNow team is eager to encourage other companies to follow their lead. The lessons they've learned: "You've got to have buy-in from your boss," says Sheriff. "Another big deal is having two to three people dedicated to this type of endeavor. You can't have people who are half in, half out." And if you don't have buy-in from the get go, proving it out will get you there.</p>
+
+<p>"This is The Six Million Dollar Man of the cloud right now," adds Josh. "Just try it out, watch it happen. I feel like the proof is in the pudding when you look at these kinds of application stacks. They're faster, they're more resilient."</p>
@@ -1,112 +1,85 @@
 ---
 title: Haufe Group Case Study
-
 case_study_styles: true
 cid: caseStudies
-css: /css/style_haufegroup.css
+
+new_case_study_styles: true
+heading_background: /images/case-studies/haufegroup/banner1.jpg
+heading_title_logo: /images/haufegroup_logo.png
+subheading: >
+  Paving the Way for Cloud Native for Midsize Companies
+case_study_details:
+  - Company: Haufe Group
+  - Location: Freiburg, Germany
+  - Industry: Media and Software
 ---

+<h2>Challenge</h2>

-<div class="banner1">
-  <h1> CASE STUDY:<img src="/images/haufegroup_logo.png" class="header_logo"><br> <div class="subhead">Paving the Way for Cloud Native for Midsize Companies</div></h1>
+<p>Founded in 1930 as a traditional publisher, Haufe Group has grown into a media and software company with 95 percent of its sales from digital products. Over the years, the company has gone from having "hardware in the basement" to outsourcing its infrastructure operations and IT. More recently, the development of new products, from Internet portals for tax experts to personnel training software, has created demands for increased speed, reliability and scalability. "We need to be able to move faster," says Solution Architect Martin Danielsson. "Adapting workloads is something that we really want to be able to do."</p>

-</div>
+<h2>Solution</h2>
+<p>Haufe Group began its cloud-native journey when <a href="https://azure.microsoft.com/">Microsoft Azure</a> became available in Europe; the company needed cloud deployments for its desktop apps with bandwidth-heavy download services. "After that, it has been different projects trying out different things," says Danielsson. Two years ago, Holger Reinhardt joined Haufe Group as CTO and rapidly re-oriented the traditional host provider-based approach toward a cloud and API-first strategy.</p>

-<div class="details">
-    Company &nbsp;<b>Haufe Group</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location &nbsp;<b>Freiburg, Germany</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Industry &nbsp;<b>Media and Software</b>
-</div>
+<p>A core part of this strategy was a strong mandate to embrace infrastructure-as-code across the entire software deployment lifecycle via Docker. The company is now getting ready to go live with two services in production using <a href="https://kubernetes.io/">Kubernetes</a> orchestration on <a href="https://azure.microsoft.com/">Microsoft Azure</a> and <a href="https://aws.amazon.com/">Amazon Web Services</a>. The team is also working on breaking up one of their core Java Enterprise desktop products into microservices to allow for better evolvability and dynamic scaling in the cloud.</p>

-<hr>
-
-<section class="section1">
-
-<div class="cols">
-  <div class="col1">
-    <h2>Challenge</h2>
-        Founded in 1930 as a traditional publisher, Haufe Group has grown into a media and software company with 95 percent of its sales from digital products. Over the years, the company has gone from having "hardware in the basement" to outsourcing its infrastructure operations and IT. More recently, the development of new products, from Internet portals for tax experts to personnel training software, has created demands for increased speed, reliability and scalability. "We need to be able to move faster," says Solution Architect Martin Danielsson. "Adapting workloads is something that we really want to be able to do."
-    <br>
-    <br>
-      <h2>Solution</h2>
-      Haufe Group began its cloud-native journey when <a href="https://azure.microsoft.com/">Microsoft Azure</a> became available in Europe; the company needed cloud deployments for its desktop apps with bandwidth-heavy download services. "After that, it has been different projects trying out different things," says Danielsson. Two years ago, Holger Reinhardt joined Haufe Group as CTO and rapidly re-oriented the traditional host provider-based approach toward a cloud and API-first strategy.
-      </div>
-<div class="col2">
-  A core part of this strategy was a strong mandate to embrace infrastructure-as-code across the entire software deployment lifecycle via Docker. The company is now getting ready to go live with two services in production using <a href="https://kubernetes.io/">Kubernetes</a> orchestration on <a href="https://azure.microsoft.com/">Microsoft Azure</a> and <a href="https://aws.amazon.com/">Amazon Web Services</a>. The team is also working on breaking up one of their core Java Enterprise desktop products into microservices to allow for better evolvability and dynamic scaling in the cloud.
-<br>
-<br>
 <h2>Impact</h2>
-        With the ability to adapt workloads, Danielsson says, teams "will be able to scale down to around half the capacity at night, saving 30 percent of the hardware cost." Plus, shorter release times have had a major impact. "Before, we had to announce at least a week in advance when we wanted to do a release because there was a huge checklist of things that you had to do," he says. "By going cloud native, we have the infrastructure in place to be able to automate all of these things. Now we can get a new release done in half an hour instead of days."
+<p>With the ability to adapt workloads, Danielsson says, teams "will be able to scale down to around half the capacity at night, saving 30 percent of the hardware cost." Plus, shorter release times have had a major impact. "Before, we had to announce at least a week in advance when we wanted to do a release because there was a huge checklist of things that you had to do," he says. "By going cloud native, we have the infrastructure in place to be able to automate all of these things. Now we can get a new release done in half an hour instead of days."</p>

-</div>
-</div>
+{{< case-studies/quote author="Martin Danielsson, Solution Architect, Haufe Group" >}}
+"Over the next couple of years, people won't even think that much about it when they want to run containers. Kubernetes is going to be the go-to solution."
+{{< /case-studies/quote >}}

-</section>
+{{< case-studies/lead >}}
+More than 80 years ago, Haufe Group was founded as a traditional publishing company, printing books and commentary on paper.
+{{< /case-studies/lead >}}

-<div class="banner2">
-  <div class="banner2text">
-     "Over the next couple of years, people won’t even think that much about it when they want to run containers. Kubernetes is going to be the go-to solution."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Martin Danielsson, Solution Architect, Haufe Group</span>
-  </div>
-</div>
+<p>By the 1990s, though, the company's leaders recognized that the future was digital, and to their credit, were able to transform Haufe Group into a media and software business that now gets 95 percent of its sales from digital products. "Among the German companies doing this, we were one of the early adopters," says Martin Danielsson, Solution Architect for Haufe Group.</p>

-<section class="section2">
+<p>And now they're leading the way for midsize companies embracing cloud-native technology like Kubernetes. "The really big companies like Ticketmaster and Google get it right, and the startups get it right because they're faster," says Danielsson. "We're in this big lump of companies in the middle with a lot of legacy, a lot of structure, a lot of culture that does not easily fit the cloud technologies. We're just 1,500 people, but we have hundreds of customer-facing applications. So we're doing things that will be relevant for many companies of our size or even smaller."</p>

-<div class="fullcol">
-    <h2>More than 80 years ago, Haufe Group was founded as a traditional publishing company, printing books and commentary on paper.</h2> By the 1990s, though, the company’s leaders recognized that the future was digital, and to their credit, were able to transform Haufe Group into a media and software business that now gets 95 percent of its sales from digital products. "Among the German companies doing this, we were one of the early adopters," says Martin Danielsson, Solution Architect for Haufe Group.<br><br>
-    And now they’re leading the way for midsize companies embracing cloud-native technology like Kubernetes. "The really big companies like Ticketmaster and Google get it right, and the startups get it right because they’re faster," says Danielsson. "We’re in this big lump of companies in the middle with a lot of legacy, a lot of structure, a lot of culture that does not easily fit the cloud technologies. We’re just 1,500 people, but we have hundreds of customer-facing applications. So we’re doing things that will be relevant for many companies of our size or even smaller."<br><br>
-    Many of those legacy challenges stemmed from simply following the technology trends of the times. "We used to do full DevOps," he says. In the 1990s and 2000s, "that meant that you had your hardware in the basement. And then 10 years ago, the hype of the moment was to outsource application operations, outsource everything, and strip down your IT department to take away the distraction of all these hardware things. That’s not our area of expertise. We didn’t want to be an infrastructure provider. And now comes the backlash of that."<br><br>
-    Haufe Group began feeling the pain as they were developing more new products, from Internet portals for tax experts to personnel training software, that have created demands for increased speed, reliability and scalability. "Right now, we have this break in workflows, where we go from writing concepts to developing, handing it over to production and then handing that over to your host provider," he says. "And then when things go bad we have no clue what went wrong. We definitely want to take back control, and we want to move a lot faster. Adapting workloads is something that we really want to be able to do."<br><br>
-    Those needs led them to explore cloud-native technology. Their first foray into the cloud was doing deployments in <a href="https://azure.microsoft.com/">Microsoft Azure</a>, once it became available in Europe, for desktop products that had built-in download services. Hosting expenses for such bandwidth-heavy services were too high, so the company turned to the cloud. "After that, it has been different projects trying out different things," says Danielsson.
-</div>
-</section>
+<p>Many of those legacy challenges stemmed from simply following the technology trends of the times. "We used to do full DevOps," he says. In the 1990s and 2000s, "that meant that you had your hardware in the basement. And then 10 years ago, the hype of the moment was to outsource application operations, outsource everything, and strip down your IT department to take away the distraction of all these hardware things. That's not our area of expertise. We didn't want to be an infrastructure provider. And now comes the backlash of that."</p>

-<div class="banner3">
-  <div class="banner3text">
-    "We have been doing containers for the last two years, and we really got the hang of how they work," says Danielsson. "But it was always for development and test, never in production, because we didn’t fully understand how that would work. And to me, Kubernetes was definitely the technology that solved that."
-  </div>
-</div>
+<p>Haufe Group began feeling the pain as they were developing more new products, from Internet portals for tax experts to personnel training software, that have created demands for increased speed, reliability and scalability. "Right now, we have this break in workflows, where we go from writing concepts to developing, handing it over to production and then handing that over to your host provider," he says. "And then when things go bad we have no clue what went wrong. We definitely want to take back control, and we want to move a lot faster. Adapting workloads is something that we really want to be able to do."</p>

-<section class="section3">
-<div class="fullcol">
+<p>Those needs led them to explore cloud-native technology. Their first foray into the cloud was doing deployments in <a href="https://azure.microsoft.com/">Microsoft Azure</a>, once it became available in Europe, for desktop products that had built-in download services. Hosting expenses for such bandwidth-heavy services were too high, so the company turned to the cloud. "After that, it has been different projects trying out different things," says Danielsson.</p>

-      Two years ago, Holger Reinhardt joined Haufe Group as CTO and rapidly re-oriented the traditional host provider-based approach toward a cloud and API-first strategy. A core part of this strategy was a strong mandate to embrace infrastructure-as-code across the entire software deployment lifecycle via Docker.
-      Some experiments went further than others; German regulations about sensitive data proved to be a road block in moving some workloads to Azure and Amazon Web Services. "Due to our history, Germany is really strict with things like personally identifiable data," Danielsson says.<br><br>
-      These experiments took on new life with the arrival of the Azure Sovereign Cloud for Germany (an Azure clone run by the German T-Systems provider). With the availability of Azure.de—which conforms to Germany’s privacy regulations—teams started to seriously consider deploying production loads in Docker into the cloud. "We have been doing containers for the last two years, and we really got the hang of how they work," says Danielsson. "But it was always for development and test, never in production, because we didn’t fully understand how that would work. And to me, Kubernetes was definitely the technology that solved that."<br><br>
-      In parallel, Danielsson had built an API management system with the aim of supporting CI/CD scenarios, aspects of which were missing in off-the-shelf API management products. With a foundation based on <a href="https://getkong.org/">Mashape’s Kong</a> gateway, it is open-sourced as <a href="http://wicked.haufe.io/">wicked.haufe.io</a>. He put wicked.haufe.io to use with his product team.<br><br> Otherwise, Danielsson says his philosophy was "don’t try to reinvent the wheel all the time. Go for what’s there and 99 percent of the time it will be enough. And if you think you really need something custom or additional, think perhaps once or twice again. One of the things that I find so amazing with this cloud-native framework is that everything ties in."<br><br>
-      Currently, Haufe Group is working on two projects using Kubernetes in production. One is a new mobile application for researching legislation and tax laws. "We needed a way to take out functionality from a legacy core and put an application on top of that with an API gateway—a lot of moving parts that screams containers," says Danielsson. So the team moved the build pipeline away from "deploying to some old, huge machine that you could deploy anything to" and onto a Kubernetes cluster where there would be automatic CI/CD "with feature branches and all these things that were a bit tedious in the past."
-</div>
-</section>
+{{< case-studies/quote image="/images/case-studies/haufegroup/banner3.jpg" >}}
+"We have been doing containers for the last two years, and we really got the hang of how they work," says Danielsson. "But it was always for development and test, never in production, because we didn't fully understand how that would work. And to me, Kubernetes was definitely the technology that solved that."
+{{< /case-studies/quote >}}

-<div class="banner4">
-  <div class="banner4text">
-     "Before, we had to announce at least a week in advance when we wanted to do a release because there was a huge checklist of things that you had to do," says Danielsson. "By going cloud native, we have the infrastructure in place to be able to automate all of these things. Now we can get a new release done in half an hour instead of days."
-  </div>
-</div>
+<p>Two years ago, Holger Reinhardt joined Haufe Group as CTO and rapidly re-oriented the traditional host provider-based approach toward a cloud and API-first strategy. A core part of this strategy was a strong mandate to embrace infrastructure-as-code across the entire software deployment lifecycle via Docker. Some experiments went further than others; German regulations about sensitive data proved to be a road block in moving some workloads to Azure and Amazon Web Services. "Due to our history, Germany is really strict with things like personally identifiable data," Danielsson says.</p>

-<section class="section4">
-<div class="fullcol">
-      It was a proof of concept effort, and the proof was in the pudding. "Everyone was really impressed at what we accomplished in a week," says Danielsson. "We did these kinds of integrations just to make sure that we got a handle on how Kubernetes works. If you can create optimism and buzz around something, it’s half won. And if the developers and project managers know this is working, you’re more or less done." Adds Reinhardt: "You need to create some very visible, quick wins in order to overcome the status quo."<br><br>
-      The impact on the speed of deployment was clear: "Before, we had to announce at least a week in advance when we wanted to do a release because there was a huge checklist of things that you had to do," says Danielsson. "By going cloud native, we have the infrastructure in place to be able to automate all of these things. Now we can get a new release done in half an hour instead of days." <br><br>
-      The potential impact on cost was another bonus. "Hosting applications is quite expensive, so moving to the cloud is something that we really want to be able to do," says Danielsson. With the ability to adapt workloads, teams "will be able to scale down to around half the capacity at night, saving 30 percent of the hardware cost." <br><br>
-      Just as importantly, Danielsson says, there’s added flexibility: "When we try to move or rework applications that are really crucial, it’s often tricky to validate whether the path we want to take is going to work out well. In order to validate that, we would need to reproduce the environment and really do testing, and that’s prohibitively expensive and simply not doable with traditional host providers. Cloud native gives us the ability to do risky changes and validate them in a cost-effective way."<br><br>
-      As word of the two successful test projects spread throughout the company, interest in Kubernetes has grown. "We want to be able to support our developers in running Kubernetes clusters but we’re not there yet, so we allow them to do it as long as they’re aware that they are on their own," says Danielsson. "So that’s why we are also looking at things like [the managed Kubernetes platform] <a href="https://coreos.com/tectonic/">CoreOS Tectonic</a>, <a href="https://azure.microsoft.com/en-us/services/container-service/">Azure Container Service</a>, <a href="https://aws.amazon.com/ecs/">ECS</a>, etc. These kinds of services will be a lot more relevant to midsize companies that want to leverage cloud native but don’t have the IT departments or the structure around that."<br><br>
-      In the next year and a half, Danielsson says the company will be working on moving one of their legacy desktop products, a web app for researching legislation and tax laws originally built in Java Enterprise, onto cloud-native technology. "We’re doing a microservice split out right now so that we can independently deploy the different parts," he says. The main website, which provides free content for customers, is also moving to cloud native.
+<p>These experiments took on new life with the arrival of the Azure Sovereign Cloud for Germany (an Azure clone run by the German T-Systems provider). With the availability of Azure.de—which conforms to Germany's privacy regulations—teams started to seriously consider deploying production loads in Docker into the cloud. "We have been doing containers for the last two years, and we really got the hang of how they work," says Danielsson. "But it was always for development and test, never in production, because we didn't fully understand how that would work. And to me, Kubernetes was definitely the technology that solved that."</p>

-</div>
-</section>
+<p>In parallel, Danielsson had built an API management system with the aim of supporting CI/CD scenarios, aspects of which were missing in off-the-shelf API management products. With a foundation based on <a href="https://getkong.org/">Mashape's Kong</a> gateway, it is open-sourced as <a href="http://wicked.haufe.io/">wicked.haufe.io</a>. He put wicked.haufe.io to use with his product team.<br><br> Otherwise, Danielsson says his philosophy was "don't try to reinvent the wheel all the time. Go for what's there and 99 percent of the time it will be enough. And if you think you really need something custom or additional, think perhaps once or twice again. One of the things that I find so amazing with this cloud-native framework is that everything ties in."</p>

-<div class="banner5">
-  <div class="banner5text">
-     "the execution of a strategy requires alignment of culture, structure and technology. Only if those three dimensions are aligned can you successfully execute a transformation into microservices and cloud-native architectures. And it is only then that the Cloud will pay the dividends in much faster speeds in product innovation and much lower operational costs."
+<p>Currently, Haufe Group is working on two projects using Kubernetes in production. One is a new mobile application for researching legislation and tax laws. "We needed a way to take out functionality from a legacy core and put an application on top of that with an API gateway—a lot of moving parts that screams containers," says Danielsson. So the team moved the build pipeline away from "deploying to some old, huge machine that you could deploy anything to" and onto a Kubernetes cluster where there would be automatic CI/CD "with feature branches and all these things that were a bit tedious in the past."</p>

-  </div>
-</div>
+{{< case-studies/quote image="/images/case-studies/haufegroup/banner4.jpg" >}}
+"Before, we had to announce at least a week in advance when we wanted to do a release because there was a huge checklist of things that you had to do," says Danielsson. "By going cloud native, we have the infrastructure in place to be able to automate all of these things. Now we can get a new release done in half an hour instead of days."
+{{< /case-studies/quote >}}

-<section class="section5">
-<div class="fullcol">
-  But with these goals, Danielsson believes there are bigger cultural challenges that need to be constantly addressed. The move to new technology, not to mention a shift toward DevOps, means a lot of change for employees. "The roles were rather fixed in the past," he says. "You had developers, you had project leads, you had testers. And now you get into these really, really important things like test automation. Testers aren’t actually doing click testing anymore, and they have to write automated testing. And if you really want to go full-blown CI/CD, all these little pieces have to work together so that you get the confidence to do a check in, and know this check in is going to land in production, because if I messed up, some test is going to break. This is a really powerful thing because whatever you do, whenever you merge something into the trunk or to the master, this is going live. And that’s where you either get the people or they run away screaming."
-  Danielsson understands that it may take some people much longer to get used to the new ways.<br><br>
-  "Culture is nothing that you can force on people," he says. "You have to live it for yourself. You have to evangelize. You have to show the advantages time and time again: This is how you can do it, this is what you get from it." To that end, his team has scheduled daylong workshops for the staff, bringing in outside experts to talk about everything from API to Devops to cloud. <br><br>
-  For every person who runs away screaming, many others get drawn in. "Get that foot in the door and make them really interested in this stuff," says Danielsson. "Usually it catches on. We have people you never would have expected chanting, ‘Docker Docker Docker’ now. It’s cool to see them realize that there is a world outside of their Python libraries. It’s awesome to see them really work with Kubernetes."<br><br>
-  Ultimately, Reinhardt says, "the execution of a strategy requires alignment of culture, structure and technology. Only if those three dimensions are aligned can you successfully execute a transformation into microservices and cloud-native architectures. And it is only then that the Cloud will pay the dividends in much faster speeds in product innovation and much lower operational costs."
+<p>It was a proof of concept effort, and the proof was in the pudding. "Everyone was really impressed at what we accomplished in a week," says Danielsson. "We did these kinds of integrations just to make sure that we got a handle on how Kubernetes works. If you can create optimism and buzz around something, it's half won. And if the developers and project managers know this is working, you're more or less done." Adds Reinhardt: "You need to create some very visible, quick wins in order to overcome the status quo."</p>

-</div>
-</section>
+<p>The impact on the speed of deployment was clear: "Before, we had to announce at least a week in advance when we wanted to do a release because there was a huge checklist of things that you had to do," says Danielsson. "By going cloud native, we have the infrastructure in place to be able to automate all of these things. Now we can get a new release done in half an hour instead of days."</p>
+
+<p>The potential impact on cost was another bonus. "Hosting applications is quite expensive, so moving to the cloud is something that we really want to be able to do," says Danielsson. With the ability to adapt workloads, teams "will be able to scale down to around half the capacity at night, saving 30 percent of the hardware cost."</p>
+
+<p>Just as importantly, Danielsson says, there's added flexibility: "When we try to move or rework applications that are really crucial, it's often tricky to validate whether the path we want to take is going to work out well. In order to validate that, we would need to reproduce the environment and really do testing, and that's prohibitively expensive and simply not doable with traditional host providers. Cloud native gives us the ability to do risky changes and validate them in a cost-effective way."</p>
+
+<p>As word of the two successful test projects spread throughout the company, interest in Kubernetes has grown. "We want to be able to support our developers in running Kubernetes clusters but we're not there yet, so we allow them to do it as long as they're aware that they are on their own," says Danielsson. "So that's why we are also looking at things like [the managed Kubernetes platform] <a href="https://coreos.com/tectonic/">CoreOS Tectonic</a>, <a href="https://azure.microsoft.com/en-us/services/container-service/">Azure Container Service</a>, <a href="https://aws.amazon.com/ecs/">ECS</a>, etc. These kinds of services will be a lot more relevant to midsize companies that want to leverage cloud native but don't have the IT departments or the structure around that."</p>
+
+<p>In the next year and a half, Danielsson says the company will be working on moving one of their legacy desktop products, a web app for researching legislation and tax laws originally built in Java Enterprise, onto cloud-native technology. "We're doing a microservice split out right now so that we can independently deploy the different parts," he says. The main website, which provides free content for customers, is also moving to cloud native.</p>
+
+{{< case-studies/quote >}}
+"the execution of a strategy requires alignment of culture, structure and technology. Only if those three dimensions are aligned can you successfully execute a transformation into microservices and cloud-native architectures. And it is only then that the Cloud will pay the dividends in much faster speeds in product innovation and much lower operational costs."
+{{< /case-studies/quote >}}
+
+<p>But with these goals, Danielsson believes there are bigger cultural challenges that need to be constantly addressed. The move to new technology, not to mention a shift toward DevOps, means a lot of change for employees. "The roles were rather fixed in the past," he says. "You had developers, you had project leads, you had testers. And now you get into these really, really important things like test automation. Testers aren't actually doing click testing anymore, and they have to write automated testing. And if you really want to go full-blown CI/CD, all these little pieces have to work together so that you get the confidence to do a check in, and know this check in is going to land in production, because if I messed up, some test is going to break. This is a really powerful thing because whatever you do, whenever you merge something into the trunk or to the master, this is going live. And that's where you either get the people or they run away screaming." Danielsson understands that it may take some people much longer to get used to the new ways.</p>
+
+<p>"Culture is nothing that you can force on people," he says. "You have to live it for yourself. You have to evangelize. You have to show the advantages time and time again: This is how you can do it, this is what you get from it." To that end, his team has scheduled daylong workshops for the staff, bringing in outside experts to talk about everything from API to Devops to cloud.</p>
+
+<p>For every person who runs away screaming, many others get drawn in. "Get that foot in the door and make them really interested in this stuff," says Danielsson. "Usually it catches on. We have people you never would have expected chanting, 'Docker Docker Docker' now. It's cool to see them realize that there is a world outside of their Python libraries. It's awesome to see them really work with Kubernetes."</p>
+
+<p>Ultimately, Reinhardt says, "the execution of a strategy requires alignment of culture, structure and technology. Only if those three dimensions are aligned can you successfully execute a transformation into microservices and cloud-native architectures. And it is only then that the Cloud will pay the dividends in much faster speeds in product innovation and much lower operational costs."</p>
@@ -1,101 +1,73 @@
 ---
 title: Huawei Case Study
-
 case_study_styles: true
 cid: caseStudies
-css: /css/style_huawei.css
+
+new_case_study_styles: true
+heading_background: /images/case-studies/huawei/banner1.jpg
+heading_title_logo: /images/huawei_logo.png
+subheading: >
+  Embracing Cloud Native as a User – and a Vendor
+case_study_details:
+  - Company: Huawei
+  - Location: Shenzhen, China
+  - Industry: Telecommunications Equipment
 ---
-<div class="banner1">
-  <h1> CASE STUDY:<img src="/images/huawei_logo.png" class="header_logo"><br> <div class="subhead">Embracing Cloud Native as a User – and a Vendor</div></h1>

-</div>
+<h2>Challenge</h2>

-<div class="details">
-    Company &nbsp;<b>Huawei</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location &nbsp;<b>Shenzhen, China</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Industry &nbsp;<b>Telecommunications Equipment</b>
-</div>
+<p>A multinational company that's the largest telecommunications equipment manufacturer in the world, Huawei has more than 180,000 employees. In order to support its fast business development around the globe, <a href="http://www.huawei.com/">Huawei</a> has eight data centers for its internal I.T. department, which have been running 800+ applications in 100K+ VMs to serve these 180,000 users. With the rapid increase of new applications, the cost and efficiency of management and deployment of VM-based apps all became critical challenges for business agility. "It's very much a distributed system so we found that managing all of the tasks in a more consistent way is always a challenge," says Peixin Hou, the company's Chief Software Architect and Community Director for Open Source. "We wanted to move into a more agile and decent practice."</p>

-<hr>
+<h2>Solution</h2>

-<section class="section1">
+<p>After deciding to use container technology, Huawei began moving the internal I.T. department's applications to run on <a href="http://kubernetes.io/">Kubernetes</a>. So far, about 30 percent of these applications have been transferred to cloud native.</p>

-<div class="cols">
-  <div class="col1">
-    <h2>Challenge</h2>
-        A multinational company that’s the largest telecommunications equipment manufacturer in the world, Huawei has more than 180,000 employees. In order to support its fast business development around the globe, <a href="http://www.huawei.com/">Huawei</a> has eight data centers for its internal I.T. department, which have been running 800+ applications in 100K+ VMs to serve these 180,000 users. With the rapid increase of new applications, the cost and efficiency of management and deployment of VM-based apps all became critical challenges for business agility. "It’s very much a distributed system so we found that managing all of the tasks in a more consistent way is always a challenge," says Peixin Hou, the company’s Chief Software Architect and Community Director for Open Source. "We wanted to move into a more agile and decent practice."
-      </div>
-
-<div class="col2">
-  <h2>Solution</h2>
-      After deciding to use container technology, Huawei began moving the internal I.T. department’s applications to run on <a href="http://kubernetes.io/">Kubernetes</a>. So far, about 30 percent of these applications have been transferred to cloud native.
-<br>
-<br>
 <h2>Impact</h2>
-      "By the end of 2016, Huawei’s internal I.T. department managed more than 4,000 nodes with tens of thousands containers using a Kubernetes-based Platform as a Service (PaaS) solution," says Hou. "The global deployment cycles decreased from a week to minutes, and the efficiency of application delivery has been improved 10 fold." For the bottom line, he says, "We also see significant operating expense spending cut, in some circumstances 20-30 percent, which we think is very helpful for our business." Given the results Huawei has had internally – and the demand it is seeing externally – the company has also built the technologies into <a href="http://developer.huawei.com/ict/en/site-paas">FusionStage™</a>, the PaaS solution it offers its customers.
-</div>

-</div>
+<p>"By the end of 2016, Huawei's internal I.T. department managed more than 4,000 nodes with tens of thousands containers using a Kubernetes-based Platform as a Service (PaaS) solution," says Hou. "The global deployment cycles decreased from a week to minutes, and the efficiency of application delivery has been improved 10 fold." For the bottom line, he says, "We also see significant operating expense spending cut, in some circumstances 20-30 percent, which we think is very helpful for our business." Given the results Huawei has had internally – and the demand it is seeing externally – the company has also built the technologies into <a href="http://developer.huawei.com/ict/en/site-paas">FusionStage™</a>, the PaaS solution it offers its customers.</p>

-</section>
+{{< case-studies/quote author="Peixin Hou, chief software architect and community director for open source" >}}
+"If you're a vendor, in order to convince your customer, you should use it yourself. Luckily because Huawei has a lot of employees, we can demonstrate the scale of cloud we can build using this technology."
+{{< /case-studies/quote >}}

-<div class="banner2">
-  <div class="banner2text">
-    "If you’re a vendor, in order to convince your customer, you should use it yourself. Luckily because Huawei has a lot of employees, we can demonstrate the scale of cloud we can build using this technology."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Peixin Hou, chief software architect and community director for open source</span>
-  </div>
-</div>
+<p>Huawei's Kubernetes journey began with one developer. Over two years ago, one of the engineers employed by the networking and telecommunications giant became interested in <a href="http://kubernetes.io/">Kubernetes</a>, the technology for managing application containers across clusters of hosts, and started contributing to its open source community. As the technology developed and the community grew, he kept telling his managers about it.</p>

-<section class="section2">
+<p>And as fate would have it, at the same time, Huawei was looking for a better orchestration system for its internal enterprise I.T. department, which supports every business flow processing. "We have more than 180,000 employees worldwide, and a complicated internal procedure, so probably every week this department needs to develop some new applications," says Peixin Hou, Huawei's Chief Software Architect and Community Director for Open Source. "Very often our I.T. departments need to launch tens of thousands of containers, with tasks running across thousands of nodes across the world. It's very much a distributed system, so we found that managing all of the tasks in a more consistent way is always a challenge."</p>

-<div class="fullcol">
-      Huawei’s Kubernetes journey began with one developer.
-    Over two years ago, one of the engineers employed by the networking and telecommunications giant became interested in <a href="http://kubernetes.io/">Kubernetes</a>, the technology for managing application containers across clusters of hosts, and started contributing to its open source community. As the technology developed and the community grew, he kept telling his managers about it.<br><br>
-    And as fate would have it, at the same time, Huawei was looking for a better orchestration system for its internal enterprise I.T. department, which supports every business flow processing. "We have more than 180,000 employees worldwide, and a complicated internal procedure, so probably every week this department needs to develop some new applications," says Peixin Hou, Huawei’s Chief Software Architect and Community Director for Open Source. "Very often our I.T. departments need to launch tens of thousands of containers, with tasks running across thousands of nodes across the world. It’s very much a distributed system, so we found that managing all of the tasks in a more consistent way is always a challenge."<br><br>
-    In the past, Huawei had used virtual machines to encapsulate applications, but "every time when we start a VM," Hou says, "whether because it’s a new service or because it was a service that was shut down because of some abnormal node functioning, it takes a lot of time."  Huawei turned to containerization, so the timing was right to try Kubernetes. It took a year to adopt that engineer’s suggestion – the process "is not overnight," says Hou – but once in use, he says, "Kubernetes basically solved most of our problems. Before, the time of deployment took about a week, now it only takes minutes. The developers are happy. That department is also quite happy."<br><br>
-    Hou sees great benefits to the company that come with using this technology: "Kubernetes brings agility, scale-out capability, and DevOps practice to the cloud-based applications," he says. "It provides us with the ability to customize the scheduling architecture, which makes possible the affinity between container tasks that gives greater efficiency. It supports multiple container formats. It has extensive support for various container networking solutions and container storage."
-</div>
-</section>
+<p>In the past, Huawei had used virtual machines to encapsulate applications, but "every time when we start a VM," Hou says, "whether because it's a new service or because it was a service that was shut down because of some abnormal node functioning, it takes a lot of time."  Huawei turned to containerization, so the timing was right to try Kubernetes. It took a year to adopt that engineer's suggestion – the process "is not overnight," says Hou – but once in use, he says, "Kubernetes basically solved most of our problems. Before, the time of deployment took about a week, now it only takes minutes. The developers are happy. That department is also quite happy."</p>

-<div class="banner3">
-  <div class="banner3text">
-    "Kubernetes basically solved most of our problems. Before, the time of deployment took about a week, now it only takes minutes. The developers are happy. That department is also quite happy."
-  </div>
-</div>
+<p>Hou sees great benefits to the company that come with using this technology: "Kubernetes brings agility, scale-out capability, and DevOps practice to the cloud-based applications," he says. "It provides us with the ability to customize the scheduling architecture, which makes possible the affinity between container tasks that gives greater efficiency. It supports multiple container formats. It has extensive support for various container networking solutions and container storage."</p>

-<section class="section3">
-<div class="fullcol">
-      And not least of all, there’s an impact on the bottom line. Says Hou: "We also see significant operating expense spending cut in some circumstances 20-30 percent, which is very helpful for our business."<br><br>
-    Pleased with those initial results, and seeing a demand for cloud native technologies from its customers, Huawei doubled down on Kubernetes. In the spring of 2016, the company became not only a user but also a vendor.<br><br>
-    "We built the Kubernetes technologies into our solutions," says Hou, referring to Huawei’s <a href="http://developer.huawei.com/ict/en/site-paas">FusionStage™</a> PaaS offering. "Our customers, from very big telecommunications operators to banks, love the idea of cloud native. They like Kubernetes technology. But they need to spend a lot of time to decompose their applications to turn them into microservice architecture, and as a solution provider, we help them. We’ve started to work with some Chinese banks, and we see a lot of interest from our customers like <a href="http://www.chinamobileltd.com/">China Mobile</a> and <a href="https://www.telekom.com/en">Deutsche Telekom</a>."<br><br>
-    "If you’re just a user, you’re just a user," adds Hou. "But if you’re a vendor, in order to even convince your customers, you should use it yourself. Luckily because Huawei has a lot of employees, we can demonstrate the scale of cloud we can build using this technology. We provide customer wisdom." While Huawei has its own private cloud, many of its customers run cross-cloud applications using Huawei’s solutions. It’s a big selling point that most of the public cloud providers now support Kubernetes. "This makes the cross-cloud transition much easier than with other solutions," says Hou.<br><br>
-</div>
-</section>
+{{< case-studies/quote image="/images/case-studies/huawei/banner3.jpg" >}}
+"Kubernetes basically solved most of our problems. Before, the time of deployment took about a week, now it only takes minutes. The developers are happy. That department is also quite happy."
+{{< /case-studies/quote >}}

-<div class="banner4">
-  <div class="banner4text">
-    "Our customers, from very big telecommunications operators to banks, love the idea of cloud native. They like Kubernetes technology. But they need to spend a lot of time to decompose their applications to turn them into microservice architecture, and as a solution provider, we help them."
-  </div>
-</div>
+<p>And not least of all, there's an impact on the bottom line. Says Hou: "We also see significant operating expense spending cut in some circumstances 20-30 percent, which is very helpful for our business."</p>

-<section class="section4">
-<div class="fullcol">
-  Within Huawei itself, once his team completes the transition of the internal business procedure department to Kubernetes, Hou is looking to convince more departments to move over to the cloud native development cycle and practice. "We have a lot of software developers, so we will provide them with our platform as a service solution, our own product," he says. "We would like to see significant cuts in their iteration cycle."<br><br>
-    Having overseen the initial move to Kubernetes at Huawei, Hou has advice for other companies considering the technology: "When you start to design the architecture of your application, think about cloud native, think about microservice architecture from the beginning," he says. "I think you will benefit from that."<br><br>
-  But if you already have legacy applications, "start from some microservice-friendly part of those applications first, parts that are relatively easy to be decomposed into simpler pieces and are relatively lightweight," Hou says. "Don’t think from day one that within how many days I want to move the whole architecture, or move everything into microservices. Don’t put that as a kind of target. You should do it in a gradual manner. And I would say for legacy applications, not every piece would be suitable for microservice architecture. No need to force it."<br><br>
-  After all, as enthusiastic as Hou is about Kubernetes at Huawei, he estimates that "in the next 10 years, maybe 80 percent of the workload can be distributed, can be run on the cloud native environments. There’s still 20 percent that’s not, but it’s fine. If we can make 80 percent of our workload really be cloud native, to have agility, it’s a much better world at the end of the day."
+<p>Pleased with those initial results, and seeing a demand for cloud native technologies from its customers, Huawei doubled down on Kubernetes. In the spring of 2016, the company became not only a user but also a vendor.</p>

-</div>
-</section>
+<p>"We built the Kubernetes technologies into our solutions," says Hou, referring to Huawei's <a href="http://developer.huawei.com/ict/en/site-paas">FusionStage™</a> PaaS offering. "Our customers, from very big telecommunications operators to banks, love the idea of cloud native. They like Kubernetes technology. But they need to spend a lot of time to decompose their applications to turn them into microservice architecture, and as a solution provider, we help them. We've started to work with some Chinese banks, and we see a lot of interest from our customers like <a href="http://www.chinamobileltd.com/">China Mobile</a> and <a href="https://www.telekom.com/en">Deutsche Telekom</a>."</p>

-<div class="banner5">
-  <div class="banner5text">
-    "In the next 10 years, maybe 80 percent of the workload can be distributed, can be run on the cloud native environments. There’s still 20 percent that’s not, but it’s fine. If we can make 80 percent of our workload really be cloud native, to have agility, it’s a much better world at the end of the day."
+<p>"If you're just a user, you're just a user," adds Hou. "But if you're a vendor, in order to even convince your customers, you should use it yourself. Luckily because Huawei has a lot of employees, we can demonstrate the scale of cloud we can build using this technology. We provide customer wisdom." While Huawei has its own private cloud, many of its customers run cross-cloud applications using Huawei's solutions. It's a big selling point that most of the public cloud providers now support Kubernetes. "This makes the cross-cloud transition much easier than with other solutions," says Hou.</p>

-  </div>
-</div>
-<section class="section5">
-<div class="fullcol">
-  In the nearer future, Hou is looking forward to new features that are being developed around Kubernetes, not least of all the ones that Huawei is contributing to. Huawei engineers have worked on the federation feature (which puts multiple Kubernetes clusters in a single framework to be managed seamlessly), scheduling, container networking and storage, and a just-announced technology called <a href="http://containerops.org/">Container Ops</a>, which is a DevOps pipeline engine. "This will put every DevOps job into a container," he explains. "And then this container mechanism is running using Kubernetes, but is also used to test Kubernetes. With that mechanism, we can make the containerized DevOps jobs be created, shared and managed much more easily than before."<br><br>
-  Still, Hou sees this technology as only halfway to its full potential. First and foremost, he’d like to expand the scale it can orchestrate, which is important for supersized companies like Huawei –  as well as some of its customers.<br><br>
-  Hou proudly notes that two years after that first Huawei engineer became a contributor to and evangelist for Kubernetes, Huawei is now a top contributor to the community. "We’ve learned that the more you contribute to the community," he says, "the more you get back."
+{{< case-studies/quote image="/images/case-studies/huawei/banner4.jpg" >}}
+"Our customers, from very big telecommunications operators to banks, love the idea of cloud native. They like Kubernetes technology. But they need to spend a lot of time to decompose their applications to turn them into microservice architecture, and as a solution provider, we help them."
+{{< /case-studies/quote >}}

-</div>
-</section>
+<p>Within Huawei itself, once his team completes the transition of the internal business procedure department to Kubernetes, Hou is looking to convince more departments to move over to the cloud native development cycle and practice. "We have a lot of software developers, so we will provide them with our platform as a service solution, our own product," he says. "We would like to see significant cuts in their iteration cycle."</p>
+
+<p>Having overseen the initial move to Kubernetes at Huawei, Hou has advice for other companies considering the technology: "When you start to design the architecture of your application, think about cloud native, think about microservice architecture from the beginning," he says. "I think you will benefit from that."</p>
+
+<p>But if you already have legacy applications, "start from some microservice-friendly part of those applications first, parts that are relatively easy to be decomposed into simpler pieces and are relatively lightweight," Hou says. "Don't think from day one that within how many days I want to move the whole architecture, or move everything into microservices. Don't put that as a kind of target. You should do it in a gradual manner. And I would say for legacy applications, not every piece would be suitable for microservice architecture. No need to force it."</p>
+
+<p>After all, as enthusiastic as Hou is about Kubernetes at Huawei, he estimates that "in the next 10 years, maybe 80 percent of the workload can be distributed, can be run on the cloud native environments. There's still 20 percent that's not, but it's fine. If we can make 80 percent of our workload really be cloud native, to have agility, it's a much better world at the end of the day."</p>
+
+{{< case-studies/quote >}}
+"In the next 10 years, maybe 80 percent of the workload can be distributed, can be run on the cloud native environments. There's still 20 percent that's not, but it's fine. If we can make 80 percent of our workload really be cloud native, to have agility, it's a much better world at the end of the day."
+{{< /case-studies/quote >}}
+
+<p>In the nearer future, Hou is looking forward to new features that are being developed around Kubernetes, not least of all the ones that Huawei is contributing to. Huawei engineers have worked on the federation feature (which puts multiple Kubernetes clusters in a single framework to be managed seamlessly), scheduling, container networking and storage, and a just-announced technology called <a href="http://containerops.org/">Container Ops</a>, which is a DevOps pipeline engine. "This will put every DevOps job into a container," he explains. "And then this container mechanism is running using Kubernetes, but is also used to test Kubernetes. With that mechanism, we can make the containerized DevOps jobs be created, shared and managed much more easily than before."</p>
+
+<p>Still, Hou sees this technology as only halfway to its full potential. First and foremost, he'd like to expand the scale it can orchestrate, which is important for supersized companies like Huawei –  as well as some of its customers.</p>
+
+<p>Hou proudly notes that two years after that first Huawei engineer became a contributor to and evangelist for Kubernetes, Huawei is now a top contributor to the community. "We've learned that the more you contribute to the community," he says, "the more you get back."</p>
@@ -1,108 +1,80 @@
 ---
 title: IBM Case Study
-
 linkTitle: IBM
 case_study_styles: true
 cid: caseStudies
-css: /css/style_case_studies.css
 logo: ibm_featured_logo.svg
 featured: false
+
+new_case_study_styles: true
+heading_background: /images/case-studies/ibm/banner1.jpg
+heading_title_logo: /images/ibm_logo.png
+subheading: >
+  Building an Image Trust Service on Kubernetes with Notary and TUF
+case_study_details:
+  - Company: IBM
+  - Location: Armonk, New York
+  - Industry: Cloud Computing
 ---

-<div class="banner1" style="background-image: url('/images/case-studies/ibm/banner1.jpg')">
-  <h1> CASE STUDY:<img src="/images/ibm_logo.png" class="header_logo" style="width:10%"><br> <div class="subhead">Building an Image Trust Service on Kubernetes with Notary and TUF</div></h1>
+<h2>Challenge</h2>

-</div>
+<p><a href="https://www.ibm.com/cloud/">IBM Cloud</a> offers public, private, and hybrid cloud functionality across a diverse set of runtimes from its OpenWhisk-based function as a service (FaaS) offering, managed <a href="https://kubernetes.io">Kubernetes</a> and containers, to <a href="https://www.cloudfoundry.org">Cloud Foundry</a> platform as a service (PaaS). These runtimes are combined with the power of the company's enterprise technologies, such as MQ and DB2, its modern artificial intelligence (AI) Watson, and data analytics services. Users of IBM Cloud can exploit capabilities from more than 170 different cloud native services in its catalog, including capabilities such as IBM's Weather Company API and data services. In the later part of 2017, the IBM Cloud Container Registry team wanted to build out an image trust service.</p>

-<div class="details">
-    Company &nbsp;<b>IBM</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location &nbsp;<b>Armonk, New York</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Industry &nbsp;<b>Cloud Computing</b>
-</div>
-
-<hr>
-<section class="section1">
-<div class="cols">
-  <div class="col1" style="width:95%">
-    <h2>Challenge</h2>
-        <a href="https://www.ibm.com/cloud/">IBM Cloud</a> offers public, private, and hybrid cloud functionality across a diverse set of runtimes from its OpenWhisk-based function as a service (FaaS) offering, managed <a href="https://kubernetes.io">Kubernetes</a> and containers, to <a href="https://www.cloudfoundry.org">Cloud Foundry</a> platform as a service (PaaS). These runtimes are combined with the power of the company’s enterprise technologies, such as MQ and DB2, its modern artificial intelligence (AI) Watson, and data analytics services. Users of IBM Cloud can exploit capabilities from more than 170 different cloud native services in its catalog, including capabilities such as IBM’s Weather Company API and data services. In the later part of 2017, the IBM Cloud Container Registry team wanted to build out an image trust service.
-<br><br>
- <h2>Solution</h2>
-      The work on this new service culminated with its public availability in the IBM Cloud in February 2018. The image trust service, called Portieris, is fully based on the <a href="https://www.cncf.io">Cloud Native Computing Foundation (CNCF)</a> open source project <a href="https://github.com/theupdateframework/notary">Notary</a>, according to Michael Hough, a software developer with the IBM Cloud Container Registry team. Portieris is a Kubernetes admission controller for enforcing content trust. Users can create image security policies for each Kubernetes namespace, or at the cluster level, and enforce different levels of trust for different images. Portieris is a key part of IBM’s trust story, since it makes it possible for users to consume the company’s Notary offering from within their IKS clusters. The offering is that Notary server runs in IBM’s cloud, and then Portieris runs inside the IKS cluster. This enables users to be able to have their IKS cluster verify that the image they're loading containers from contains exactly what they expect it to, and Portieris is what allows an IKS cluster to apply that verification.
-
-      </div>
-
-<div class="col2" style="width:95%">
+<h2>Solution</h2>

+<p>The work on this new service culminated with its public availability in the IBM Cloud in February 2018. The image trust service, called Portieris, is fully based on the <a href="https://www.cncf.io">Cloud Native Computing Foundation (CNCF)</a> open source project <a href="https://github.com/theupdateframework/notary">Notary</a>, according to Michael Hough, a software developer with the IBM Cloud Container Registry team. Portieris is a Kubernetes admission controller for enforcing content trust. Users can create image security policies for each Kubernetes namespace, or at the cluster level, and enforce different levels of trust for different images. Portieris is a key part of IBM's trust story, since it makes it possible for users to consume the company's Notary offering from within their IKS clusters. The offering is that Notary server runs in IBM's cloud, and then Portieris runs inside the IKS cluster. This enables users to be able to have their IKS cluster verify that the image they're loading containers from contains exactly what they expect it to, and Portieris is what allows an IKS cluster to apply that verification.</p>

 <h2>Impact</h2>
-     IBM's intention in offering a managed Kubernetes container service and image registry is to provide a fully secure end-to-end platform for its enterprise customers. "Image signing is one key part of that offering, and our container registry team saw Notary as the de facto way to implement that capability in the current Docker and container ecosystem," Hough says. The company had not been offering image signing before, and Notary is the tool it used to implement that capability. "We had a multi-tenant Docker Registry with private image hosting," Hough says. "The Docker Registry uses hashes to ensure that image content is correct, and data is encrypted both in flight and at rest. But it does not provide any guarantees of who pushed an image. We used Notary to enable users to sign images in their private registry namespaces if they so choose."

+<p>IBM's intention in offering a managed Kubernetes container service and image registry is to provide a fully secure end-to-end platform for its enterprise customers. "Image signing is one key part of that offering, and our container registry team saw Notary as the de facto way to implement that capability in the current Docker and container ecosystem," Hough says. The company had not been offering image signing before, and Notary is the tool it used to implement that capability. "We had a multi-tenant Docker Registry with private image hosting," Hough says. "The Docker Registry uses hashes to ensure that image content is correct, and data is encrypted both in flight and at rest. But it does not provide any guarantees of who pushed an image. We used Notary to enable users to sign images in their private registry namespaces if they so choose."</p>

-</div>
+{{< case-studies/quote author="Michael Hough, a software developer with the IBM Container Registry team" >}}
+"We see CNCF as a safe haven for cloud native open source, providing stability, longevity, and expected maintenance for member projects—no matter the originating vendor or project."
+{{< /case-studies/quote >}}

-</div>
-</section>
-<div class="banner2">
-  <div class="banner2text">
-   "We see CNCF as a safe haven for cloud native open source, providing stability, longevity, and expected maintenance for member projects—no matter the originating vendor or project."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Michael Hough, a software developer with the IBM Container Registry team</span>
-  </div>
-</div>
-<section class="section2">
-<div class="fullcol">
-  <h2>Docker had already created the Notary project as an implementation of <a href="https://github.com/theupdateframework/specification" style="text-decoration:underline">The Update Framework (TUF)</a>, and this implementation of TUF provided the capabilities for Docker Content Trust.</h2> "After contribution to CNCF of both TUF and Notary, we perceived that it was becoming the de facto standard for image signing in the container ecosystem", says Michael Hough, a software developer with the IBM Cloud Container Registry team.
-<br><br>
-The key reason for selecting Notary was that it was already compatible with the existing authentication stack IBM’s container registry was using. So was the design of TUF, which does not require the registry team to have to enter the business of key management. Both of these were "attractive design decisions that confirmed our choice of Notary," he says.
-<br><br>
-The introduction of Notary to implement image signing capability in IBM Cloud encourages increased security across IBM's cloud platform, "where we expect it will include both the signing of official IBM images as well as expected use by security-conscious enterprise customers," Hough says. "When combined with security policy implementations, we expect an increased use of deployment policies in CI/CD pipelines that allow for fine-grained control of service deployment based on image signers."
-The availability of image signing "is a huge benefit to security-conscious customers who require this level of image provenance and security," Hough says. "With our IBM Cloud Kubernetes as-a-service offering and the admission controller we have made available, it allows both IBM services as well as customers of the IBM public cloud to use security policies to control service deployment."
+{{< case-studies/lead >}}
+Docker had already created the Notary project as an implementation of <a href="https://github.com/theupdateframework/specification">The Update Framework (TUF)</a>, and this implementation of TUF provided the capabilities for Docker Content Trust.
+{{< /case-studies/lead >}}

+<p>"After contribution to CNCF of both TUF and Notary, we perceived that it was becoming the de facto standard for image signing in the container ecosystem", says Michael Hough, a software developer with the IBM Cloud Container Registry team.</p>

-</div>
-</section>
-<div class="banner3" style="background-image: url('/images/case-studies/ibm/banner3.jpg')">
-  <div class="banner3text">
-    "Image signing is one key part of our Kubernetes container service offering, and our container registry team saw Notary as the de facto way to implement that capability in the current Docker and container ecosystem"<span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br><br>- Michael Hough, a software developer with the IBM Cloud Container Registry team</span>
-  </div>
-</div>
-<section class="section3">
-<div class="fullcol">
-    Now that the Notary-implemented service is generally available in IBM’s public cloud as a component of its existing IBM Cloud Container Registry, it is deployed as a highly available service across five IBM Cloud regions. This high-availability deployment has three instances across two zones in each of the five regions, load balanced with failover support. "We have also deployed it with end-to-end TLS support through to our back-end IBM Cloudant persistence storage service," Hough says.
-<br><br>
-    The IBM team has created and open sourced a Kubernetes admission controller called Portieris, which uses Notary signing information combined with customer-defined security policies to control image deployment into their cluster. "We are hoping to drive adoption of Portieris through its use of our Notary offering," Hough says.
-    <br><br>
-    IBM has been a key player in the creation and support of open source foundations, including CNCF. Todd Moore, IBM's vice president of Open Technology, is the current CNCF governing board chair and a number of IBMers are active across many of the CNCF member projects.
+<p>The key reason for selecting Notary was that it was already compatible with the existing authentication stack IBM's container registry was using. So was the design of TUF, which does not require the registry team to have to enter the business of key management. Both of these were "attractive design decisions that confirmed our choice of Notary," he says.</p>

+<p>The introduction of Notary to implement image signing capability in IBM Cloud encourages increased security across IBM's cloud platform, "where we expect it will include both the signing of official IBM images as well as expected use by security-conscious enterprise customers," Hough says. "When combined with security policy implementations, we expect an increased use of deployment policies in CI/CD pipelines that allow for fine-grained control of service deployment based on image signers."</p>

+<p>The availability of image signing "is a huge benefit to security-conscious customers who require this level of image provenance and security," Hough says. "With our IBM Cloud Kubernetes as-a-service offering and the admission controller we have made available, it allows both IBM services as well as customers of the IBM public cloud to use security policies to control service deployment."</p>

-</div>
-</section>
-<div class="banner4" style="background-image: url('/images/case-studies/ibm/banner4.jpg')">
-  <div class="banner4text">
-     "With our IBM Cloud Kubernetes as-a-service offering and the admission controller we have made available, it allows both IBM services as well as customers of the IBM public cloud to use security policies to control service deployment."<span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br><br>- Michael Hough, a software developer with the IBM Cloud Container Registry team</span>
-  </div>
-  </div>
-</div>
+{{< case-studies/quote
+  image="/images/case-studies/ibm/banner3.jpg"
+  author="Michael Hough, a software developer with the IBM Cloud Container Registry team"
+>}}
+"Image signing is one key part of our Kubernetes container service offering, and our container registry team saw Notary as the de facto way to implement that capability in the current Docker and container ecosystem"
+{{< /case-studies/quote >}}

-<section class="section4">
-<div class="fullcol">
-  "Given that, we see CNCF as a safe haven for cloud native open source, providing stability, longevity, and expected maintenance for member projects—no matter the originating vendor or project," Hough says. Because the entire cloud native world is a fast-moving area with many competing vendors and solutions, "we see the CNCF model as an arbiter of openness and fair play across the ecosystem," he says.
-<br><br>
-With both TUF and Notary as part of CNCF, IBM expects there to be standardization around these capabilities beyond just de facto standards for signing and provenance. IBM has determined to not simply consume Notary, but also to contribute to the open source project where applicable. "IBMers have contributed a CouchDB backend to support our use of IBM Cloudant as the persistent store; and are working on generalization of the pkcs11 provider, allowing support of other security hardware devices beyond Yubikey," Hough says.
+<p>Now that the Notary-implemented service is generally available in IBM's public cloud as a component of its existing IBM Cloud Container Registry, it is deployed as a highly available service across five IBM Cloud regions. This high-availability deployment has three instances across two zones in each of the five regions, load balanced with failover support. "We have also deployed it with end-to-end TLS support through to our back-end IBM Cloudant persistence storage service," Hough says.</p>

-</div>
-</section>
-<div class="banner5">
-  <div class="banner5text">
-   "There are new projects addressing these challenges, including within CNCF. We will definitely be following these advancements with interest. We found the Notary community to be an active and friendly community open to changes, such as our addition of a CouchDB backend for persistent storage." <span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br><br>- Michael Hough, a software developer with the IBM Cloud Container Registry team</span>
-  </div>
-</div>
-<section class="section5" style="padding:0px !important">
-<div class="fullcol">
-The company has used other CNCF projects <a href="https://containerd.io">containerd</a>, <a href="https://www.envoyproxy.io">Envoy</a>, <a href="https://prometheus.io">Prometheus</a>, <a href="https://grpc.io">gRPC</a>, and <a href="https://github.com/containernetworking">CNI</a>, and is looking into <a href="https://github.com/spiffe">SPIFFE</a> and <a href="https://github.com/spiffe/spire">SPIRE</a> as well for potential future use.
-<br><br>
-What advice does Hough have for other companies that are looking to deploy Notary or a cloud native infrastructure?
-<br><br>
-"While this is true for many areas of cloud native infrastructure software, we found that a high-availability, multi-region deployment of Notary requires a solid implementation to handle certificate management and rotation," he says. "There are new projects addressing these challenges, including within CNCF. We will definitely be following these advancements with interest. We found the Notary community to be an active and friendly community open to changes, such as our addition of a CouchDB backend for persistent storage."
+<p>The IBM team has created and open sourced a Kubernetes admission controller called Portieris, which uses Notary signing information combined with customer-defined security policies to control image deployment into their cluster. "We are hoping to drive adoption of Portieris through its use of our Notary offering," Hough says.</p>

-</div>
+<p>IBM has been a key player in the creation and support of open source foundations, including CNCF. Todd Moore, IBM's vice president of Open Technology, is the current CNCF governing board chair and a number of IBMers are active across many of the CNCF member projects.</p>

-</section>
+{{< case-studies/quote
+  image="/images/case-studies/ibm/banner4.jpg"
+  author="Michael Hough, a software developer with the IBM Cloud Container Registry team"
+>}}
+"With our IBM Cloud Kubernetes as-a-service offering and the admission controller we have made available, it allows both IBM services as well as customers of the IBM public cloud to use security policies to control service deployment."
+{{< /case-studies/quote >}}
+
+<p>"Given that, we see CNCF as a safe haven for cloud native open source, providing stability, longevity, and expected maintenance for member projects—no matter the originating vendor or project," Hough says. Because the entire cloud native world is a fast-moving area with many competing vendors and solutions, "we see the CNCF model as an arbiter of openness and fair play across the ecosystem," he says.</p>
+
+<p>With both TUF and Notary as part of CNCF, IBM expects there to be standardization around these capabilities beyond just de facto standards for signing and provenance. IBM has determined to not simply consume Notary, but also to contribute to the open source project where applicable. "IBMers have contributed a CouchDB backend to support our use of IBM Cloudant as the persistent store; and are working on generalization of the pkcs11 provider, allowing support of other security hardware devices beyond Yubikey," Hough says.</p>
+
+{{< case-studies/quote author="Michael Hough, a software developer with the IBM Cloud Container Registry team" >}}
+"There are new projects addressing these challenges, including within CNCF. We will definitely be following these advancements with interest. We found the Notary community to be an active and friendly community open to changes, such as our addition of a CouchDB backend for persistent storage."
+{{< /case-studies/quote >}}
+
+<p>The company has used other CNCF projects <a href="https://containerd.io">containerd</a>, <a href="https://www.envoyproxy.io">Envoy</a>, <a href="https://prometheus.io">Prometheus</a>, <a href="https://grpc.io">gRPC</a>, and <a href="https://github.com/containernetworking">CNI</a>, and is looking into <a href="https://github.com/spiffe">SPIFFE</a> and <a href="https://github.com/spiffe/spire">SPIRE</a> as well for potential future use.</p>
+
+<p>What advice does Hough have for other companies that are looking to deploy Notary or a cloud native infrastructure?</p>
+
+<p>"While this is true for many areas of cloud native infrastructure software, we found that a high-availability, multi-region deployment of Notary requires a solid implementation to handle certificate management and rotation," he says. "There are new projects addressing these challenges, including within CNCF. We will definitely be following these advancements with interest. We found the Notary community to be an active and friendly community open to changes, such as our addition of a CouchDB backend for persistent storage."</p>
@@ -5,95 +5,74 @@ case_study_styles: true
 cid: caseStudies
 weight: 50
 featured: true
-css: /css/style_case_studies.css
 quote: >
-  The big cloud native promise to our business is the ability to go from idea to production within 48 hours. We are some years away from this, but that’s quite feasible to us.
+  The big cloud native promise to our business is the ability to go from idea to production within 48 hours. We are some years away from this, but that's quite feasible to us.
+
+new_case_study_styles: true
+heading_background: /images/case-studies/ing/banner1.jpg
+heading_title_logo: /images/ing_logo.png
+subheading: >
+  Driving Banking Innovation with Cloud Native
+case_study_details:
+  - Company: ING
+  - Location: Amsterdam, Netherlands
+  - Industry: Finance
 ---

+<h2>Challenge</h2>

-<div class="banner1" style="background-image: url('/images/case-studies/ing/banner1.jpg')">
-  <h1> CASE STUDY:<img src="/images/ing_logo.png" style="margin-bottom:-1.5%;" class="header_logo"><br> <div class="subhead"> Driving Banking Innovation with Cloud Native
-</div></h1>
+<p>After undergoing an agile transformation, <a href="https://www.ing.com/">ING</a> realized it needed a standardized platform to support the work their developers were doing. "Our DevOps teams got empowered to be autonomous," says Infrastructure Architect Thijs Ebbers. "It has benefits; you get all kinds of ideas. But a lot of teams are going to devise the same wheel. Teams started tinkering with <a href="https://www.docker.com/">Docker</a>, Docker Swarm, <a href="https://kubernetes.io/">Kubernetes</a>, <a href="https://mesosphere.com/">Mesos</a>. Well, it's not really useful for a company to have one hundred wheels, instead of one good wheel.</p>

-</div>
+<h2>Solution</h2>

-<div class="details">
-    Company &nbsp;<b>ING</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location &nbsp;<b>Amsterdam, Netherlands
-</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Industry &nbsp;<b>Finance</b>
-</div>
+<p>Using Kubernetes for container orchestration and Docker for containerization, the ING team began building an internal public cloud for its CI/CD pipeline and green-field applications. The pipeline, which has been built on Mesos Marathon, will be migrated onto Kubernetes. The bank-account management app <a href="https://www.yolt.com/">Yolt</a> in the U.K. (and soon France and Italy) market already is live hosted on a Kubernetes framework. At least two greenfield projects currently on the Kubernetes framework will be going into production later this year. By the end of 2018, the company plans to have converted a number of APIs used in the banking customer experience to cloud native APIs and host these on the Kubernetes-based platform.</p>

-<hr>
-<section class="section1">
-<div class="cols" style="width:85% !important;margin-left:10% !important">
-  <div class="col1" style="width:100% !important;">
-    <h2>Challenge</h2>
-    After undergoing an agile transformation, <a href="https://www.ing.com/">ING</a> realized it needed a standardized platform to support the work their developers were doing. "Our DevOps teams got empowered to be autonomous," says Infrastructure Architect Thijs Ebbers. "It has benefits; you get all kinds of ideas. But a lot of teams are going to devise the same wheel. Teams started tinkering with <a href="https://www.docker.com/">Docker</a>, Docker Swarm, <a href="https://kubernetes.io/">Kubernetes</a>, <a href="https://mesosphere.com/">Mesos</a>. Well, it’s not really useful for a company to have one hundred wheels, instead of one good wheel.
-<br>
-    <br>
-    <h2>Solution</h2>
-    Using Kubernetes for container orchestration and Docker for containerization, the ING team began building an internal public cloud for its CI/CD pipeline and green-field applications. The pipeline, which has been built on Mesos Marathon, will be migrated onto Kubernetes. The bank-account management app <a href="https://www.yolt.com/">Yolt</a> in the U.K. (and soon France and Italy) market already is live hosted on a Kubernetes framework. At least two greenfield projects currently on the Kubernetes framework will be going into production later this year. By the end of 2018, the company plans to have converted a number of APIs used in the banking customer experience to cloud native APIs and host these on the Kubernetes-based&nbsp;platform.
+<h2>Impact</h2>

-    <br>
-    </div>
+<p>"Cloud native technologies are helping our speed, from getting an application to test to acceptance to production," says Infrastructure Architect Onno Van der Voort. "If you walk around ING now, you see all these DevOps teams, doing stand-ups, demoing. They try to get new functionality out there really fast. We held a hackathon for one of our existing components and basically converted it to cloud native within 2.5 days, though of course the tail takes more time before code is fully production ready."</p>

-<div class="col2"  style="width:100% !important;padding-right:8%">
-<br>
-  <h2>Impact</h2>
-    "Cloud native technologies are helping our speed, from getting an application to test to acceptance to production," says Infrastructure Architect Onno Van der Voort. "If you walk around ING now, you see all these DevOps teams, doing stand-ups, demoing. They try to get new functionality out there really fast. We held a hackathon for one of our existing components and basically converted it to cloud native within 2.5 days, though of course the tail takes more time before code is fully production ready."
-  <br>
-</div>
-</div>
-</section>
-  <div class="banner2" style="padding-top:% !important">
-    <div class="banner2text" style="width:70% !important">
-      "The big cloud native promise to our business is the ability to go from idea to production within 48 hours. We are some years away from this, but that’s quite feasible to us."
-      <span style="font-size:16px;text-transform:uppercase;letter-spacing:0.1em;"><br><br>— Thijs Ebbers, Infrastructure Architect, ING</span>
-    </div>
-  </div>
-<section class="section2">
-<div class="fullcol">
-  <h2>ING has long embraced innovation in banking, launching the internet-based ING Direct in&nbsp;1997. </h2>In that same spirit, the company underwent an agile transformation a few years ago. "Our DevOps teams got empowered to be autonomous," says Infrastructure Architect Thijs Ebbers. "It has benefits; you get all kinds of ideas. But a lot of teams are going to devise the same wheel. Teams started tinkering with Docker, Docker Swarm, Kubernetes, Mesos. Well, it’s not really useful for a company to have one hundred wheels, instead of one good wheel." <br><br>
-  Looking to standardize the deployment process within the company’s strict security guidelines, the team looked at several solutions and found that in the past year, "Kubernetes won the container management framework wars," says Ebbers. "We decided to standardize ING on a Kubernetes framework." Everything is run on premise due to banking regulations, he adds, but "we will be building an internal public cloud. We are trying to get on par with what public clouds are doing. That’s one of the reasons we got Kubernetes."<br><br>
-  They also embraced Docker to address a major pain point in ING’s CI/CD pipeline. Before containerization, "Every development team had to order a VM, and it was quite a heavy delivery model for them," says Infrastructure Architect Onno Van der Voort. "Another use case for containerization is when the application travels through the pipeline, they fire up Docker containers to do test work against the applications and after they’ve done the work, the containers get killed again."
+{{< case-studies/quote author="Thijs Ebbers, Infrastructure Architect, ING">}}
+"The big cloud native promise to our business is the ability to go from idea to production within 48 hours. We are some years away from this, but that's quite feasible to us."
+{{< /case-studies/quote >}}

-</div>
-</section>
-<div class="banner3" style="background-image: url('/images/case-studies/ing/banner3.jpg')">
-  <div class="banner3text">
-  "We decided to standardize ING on a Kubernetes framework." Everything is run on premise due to banking regulations, he adds, but "we will be building an internal public cloud. We are trying to get on par with what public clouds are doing. That’s one of the reasons we got Kubernetes."
-  <span style="font-size:16px;text-transform:uppercase;letter-spacing:0.1em;"><br><br>— Thijs Ebbers, Infrastructure Architect, ING</span>
-  </div>
-</div>
-<section class="section3">
-<div class="fullcol">
-  Because of industry regulations, applications are only allowed to go through the pipeline, where compliance is enforced, rather than be deployed directly into a container. "We have to run the complete platform of services we need, many routing from different places," says Van der Voort. "We need this Kubernetes framework for deploying the containers, with all those components, monitoring, logging. It’s complex." For that reason, ING has chosen to start on the <a href="https://www.openshift.org/">OpenShift Origin</a> Kubernetes distribution. <br><br>
-  Already, "cloud native technologies are helping our speed, from getting an application to test to acceptance to production," says Van der Voort. "If you walk around ING now, you see all these DevOps teams, doing stand-ups, demoing. They try to get new functionality out there really fast. We held a hackathon for one of our existing components and basically converted it to cloud native within 2.5 days, though of course the tail takes more time before code is fully production ready."<br><br>
-  The pipeline, which has been built on Mesos Marathon, will be migrated onto Kubernetes. Some legacy applications are also being rewritten as cloud native in order to run on the framework. At least two smaller greenfield projects built on Kubernetes will go into production this year. By the end of 2018, the company plans to have converted a number of APIs used in the banking customer experience to cloud native APIs and host these on the Kubernetes-based platform.
+{{< case-studies/lead >}}
+ING has long embraced innovation in banking, launching the internet-based ING Direct in 1997.
+{{< /case-studies/lead >}}

-</div>
-</section>
-<div class="banner4" style="background-image: url('/images/case-studies/ing/banner4.jpg')">
-  <div class="banner4text">
-    "We have to run the complete platform of services we need, many routing from different places. We need this Kubernetes framework for deploying the containers, with all those components, monitoring, logging. It’s complex." <span style="font-size:16px;text-transform:uppercase;letter-spacing:0.1em;"><br><br>— Onno Van der Voort,  Infrastructure Architect, ING</span>
-  </div>
-</div>
+<p>In that same spirit, the company underwent an agile transformation a few years ago. "Our DevOps teams got empowered to be autonomous," says Infrastructure Architect Thijs Ebbers. "It has benefits; you get all kinds of ideas. But a lot of teams are going to devise the same wheel. Teams started tinkering with Docker, Docker Swarm, Kubernetes, Mesos. Well, it's not really useful for a company to have one hundred wheels, instead of one good wheel."</p>

+<p>Looking to standardize the deployment process within the company's strict security guidelines, the team looked at several solutions and found that in the past year, "Kubernetes won the container management framework wars," says Ebbers. "We decided to standardize ING on a Kubernetes framework." Everything is run on premise due to banking regulations, he adds, but "we will be building an internal public cloud. We are trying to get on par with what public clouds are doing. That's one of the reasons we got Kubernetes."</p>

-<div class="fullcol">
-  The team, however, doesn’t see the bank’s back-end systems going onto the Kubernetes platform. "Our philosophy is it only makes sense to move things to cloud if they are cloud native," says Van der Voort. "If you have traditional architecture, build traditional patterns, it doesn’t hold any value to go to the cloud." Adds Cloud Platform Architect Alfonso Fernandez-Barandiaran: "ING has a strategy about where we will go, in order to improve our agility. So it’s not about how cool this technology is, it’s about finding the right technology and the right approach."<br><br>
-  The Kubernetes framework will be hosting some greenfield projects that are high priority for ING: applications the company is developing in response to <a href="https://ec.europa.eu/info/law/payment-services-psd-2-directive-eu-2015-2366_en">PSD2</a>, the European Commission directive requiring more innovative online and mobile payments that went into effect at the beginning of 2018. For example, a bank-account management app called <a href="https://www.yolt.com/">Yolt</a>, serving the U.K. market (and soon France and Italy), was built on a Kubernetes platform and has gone into production. ING is also developing blockchain-enabled applications that will live on the Kubernetes platform. "We’ve been contacted by a lot of development teams that have ideas with what they want to do with containers," says Ebbers.
+<p>They also embraced Docker to address a major pain point in ING's CI/CD pipeline. Before containerization, "Every development team had to order a VM, and it was quite a heavy delivery model for them," says Infrastructure Architect Onno Van der Voort. "Another use case for containerization is when the application travels through the pipeline, they fire up Docker containers to do test work against the applications and after they've done the work, the containers get killed again."</p>

-</div>
-<section class="section5" style="padding:0px !important">
-<div class="banner5">
-  <div class="banner5text">
-Even with the particular requirements that come in banking, ING has managed to take a lead in technology and innovation. "Every time we have constraints, we look for maybe a better way that we can use this technology." <span style="font-size:16px;text-transform:uppercase;letter-spacing:0.1em;"><br><br>— Alfonso Fernandez-Barandiaran, Cloud Platform Architect, ING</span></div>
-</div>
+{{< case-studies/quote
+  image="/images/case-studies/ing/banner3.jpg"
+  author="Thijs Ebbers, Infrastructure Architect, ING"
+>}}
+"We decided to standardize ING on a Kubernetes framework." Everything is run on premise due to banking regulations, he adds, but "we will be building an internal public cloud. We are trying to get on par with what public clouds are doing. That's one of the reasons we got Kubernetes."
+{{< /case-studies/quote >}}

-<div class="fullcol">
-  Even with the particular requirements that come in banking, ING has managed to take a lead in technology and innovation. "Every time we have constraints, we look for maybe a better way that we can use this technology," says Fernandez-Barandiaran. <br><br>
-  The results, after all, are worth the effort. "The big cloud native promise to our business is the ability to go from idea to production within 48 hours," says Ebbers. "That would require all these projects to be mature. We are some years away from this, but that’s quite feasible to us."
+<p>Because of industry regulations, applications are only allowed to go through the pipeline, where compliance is enforced, rather than be deployed directly into a container. "We have to run the complete platform of services we need, many routing from different places," says Van der Voort. "We need this Kubernetes framework for deploying the containers, with all those components, monitoring, logging. It's complex." For that reason, ING has chosen to start on the <a href="https://www.openshift.org/">OpenShift Origin</a> Kubernetes distribution.</p>

-</div>
+<p>Already, "cloud native technologies are helping our speed, from getting an application to test to acceptance to production," says Van der Voort. "If you walk around ING now, you see all these DevOps teams, doing stand-ups, demoing. They try to get new functionality out there really fast. We held a hackathon for one of our existing components and basically converted it to cloud native within 2.5 days, though of course the tail takes more time before code is fully production ready."</p>

-</section>
+<p>The pipeline, which has been built on Mesos Marathon, will be migrated onto Kubernetes. Some legacy applications are also being rewritten as cloud native in order to run on the framework. At least two smaller greenfield projects built on Kubernetes will go into production this year. By the end of 2018, the company plans to have converted a number of APIs used in the banking customer experience to cloud native APIs and host these on the Kubernetes-based platform.</p>
+
+{{< case-studies/quote
+  image="/images/case-studies/ing/banner4.jpg"
+  author="Onno Van der Voort,  Infrastructure Architect, ING"
+>}}
+"We have to run the complete platform of services we need, many routing from different places. We need this Kubernetes framework for deploying the containers, with all those components, monitoring, logging. It's complex."
+{{< /case-studies/quote >}}
+
+<p>The team, however, doesn't see the bank's back-end systems going onto the Kubernetes platform. "Our philosophy is it only makes sense to move things to cloud if they are cloud native," says Van der Voort. "If you have traditional architecture, build traditional patterns, it doesn't hold any value to go to the cloud." Adds Cloud Platform Architect Alfonso Fernandez-Barandiaran: "ING has a strategy about where we will go, in order to improve our agility. So it's not about how cool this technology is, it's about finding the right technology and the right approach."</p>
+
+<p>The Kubernetes framework will be hosting some greenfield projects that are high priority for ING: applications the company is developing in response to <a href="https://ec.europa.eu/info/law/payment-services-psd-2-directive-eu-2015-2366_en">PSD2</a>, the European Commission directive requiring more innovative online and mobile payments that went into effect at the beginning of 2018. For example, a bank-account management app called <a href="https://www.yolt.com/">Yolt</a>, serving the U.K. market (and soon France and Italy), was built on a Kubernetes platform and has gone into production. ING is also developing blockchain-enabled applications that will live on the Kubernetes platform. "We've been contacted by a lot of development teams that have ideas with what they want to do with containers," says Ebbers.</p>
+
+{{< case-studies/quote author="Alfonso Fernandez-Barandiaran, Cloud Platform Architect, ING" >}}
+Even with the particular requirements that come in banking, ING has managed to take a lead in technology and innovation. "Every time we have constraints, we look for maybe a better way that we can use this technology."
+{{< /case-studies/quote >}}
+
+<p>Even with the particular requirements that come in banking, ING has managed to take a lead in technology and innovation. "Every time we have constraints, we look for maybe a better way that we can use this technology," says Fernandez-Barandiaran.</p>
+
+<p>The results, after all, are worth the effort. "The big cloud native promise to our business is the ability to go from idea to production within 48 hours," says Ebbers. "That would require all these projects to be mature. We are some years away from this, but that's quite feasible to us."</p>
@@ -3,95 +3,77 @@ title: JD.com Case Study
 linkTitle: jd-com
 case_study_styles: true
 cid: caseStudies
-css: /css/style_case_studies.css
 featured: false
+
+new_case_study_styles: true
+heading_background: /images/case-studies/jdcom/banner1.jpg
+heading_title_logo: /images/jdcom_logo.png
+subheading: >
+  JD.com: How JD.com Pioneered Kubernetes for E-Commerce at Hyperscale
+case_study_details:
+  - Company: JD.com
+  - Location: Beijing, China
+  - Industry: eCommerce
 ---

-<div class="banner1" style="background-image: url('/images/case-studies/jdcom/banner1.jpg')">
-  <h1> CASE STUDY:<img src="/images/jdcom_logo.png" class="header_logo" style="width:17%;margin-bottom:-1%"><br> <div class="subhead" style="margin-top:1%">JD.com: How JD.com Pioneered Kubernetes for E-Commerce at Hyperscale
+<h2>Challenge</h2>

-</div></h1>
+<p>With more than 300 million active users and total 2017 revenue of more than $55 billion, <a href="https://corporate.JD.com/home">JD.com</a> is China's largest retailer, and its operations are the epitome of hyperscale. For example, there are more than a trillion images in JD.com's product databases—with 100 million being added daily—and this enormous amount of data needs to be instantly accessible. In 2014, JD.com moved its applications to containers running on bare metal machines using OpenStack and Docker to "speed up the delivery of our computing resources and make the operations much simpler," says Haifeng Liu, JD.com's Chief Architect. But by the end of 2015, with tens of thousands of nodes running in multiple data centers, "we encountered a lot of problems because our platform was not strong enough, and we suffered from bottlenecks and scalability issues," says Liu. "We needed infrastructure for the next five years of development, now."</p>

-</div>
+<h2>Solution</h2>

-<div class="details">
-    Company &nbsp;<b>JD.com</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location &nbsp;<b>Beijing, China</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Industry &nbsp;<b>eCommerce</b>
-</div>
+<p>JD.com turned to Kubernetes to accommodate its clusters. At the beginning of 2016, the company began to transition from OpenStack to Kubernetes, and today, JD.com runs the world's largest Kubernetes cluster. "Kubernetes has provided a strong foundation on top of which we have customized the solution to suit our needs as China's largest retailer."</p>

-<hr>
-<section class="section1">
-<div class="cols">
-  <div class="col1" style="width:100%"">
-    <h2>Challenge</h2>
-    With more than 300 million active users and total 2017 revenue of more than $55 billion, <a href="https://corporate.JD.com/home">JD.com</a> is China’s largest retailer, and its operations are the epitome of hyperscale. For example, there are more than a trillion images in JD.com’s product databases—with 100 million being added daily—and this enormous amount of data needs to be instantly accessible. In 2014, JD.com moved its applications to containers running on bare metal machines using OpenStack and Docker to "speed up the delivery of our computing resources and make the operations much simpler," says Haifeng Liu, JD.com’s Chief Architect. But by the end of 2015, with tens of thousands of nodes running in multiple data centers, "we encountered a lot of problems because our platform was not strong enough, and we suffered from bottlenecks and scalability issues," says Liu. "We needed infrastructure for the next five years of development, now."
-
-    <h2>Solution</h2>
-    JD.com turned to Kubernetes to accommodate its clusters. At the beginning of 2016, the company began to transition from OpenStack to Kubernetes, and today, JD.com runs the world’s largest Kubernetes cluster. "Kubernetes has provided a strong foundation on top of which we have customized the solution to suit our needs as China’s largest retailer."
-<br>
 <h2>Impact</h2>
-    "We have greater data center efficiency, better managed resources, and smarter deployment with the Kubernetes platform," says Liu. Deployment time went from several hours to tens of seconds. Efficiency has improved by 20-30%, measured in IT costs. With the further optimizations the team is working on, Liu believes there is the potential to save hundreds of millions of dollars a year. But perhaps the best indication of success was the annual Singles Day shopping event, which ran on the Kubernetes platform for the first time in 2018. Over 11 days, transaction volume on JD.com was $23 billion, and "our e-commerce platforms did great," says Liu. "Infrastructure led the way to prep for 11.11. We took the approach of predicting volume, emulating the behavior of customers to prepare beforehand, and drilled for malfunctions. Because of Kubernetes’s scalability, we were able to handle an extremely high level of demand."
-</div>

-</div>
-</section>
-<div class="banner2">
-  <div class="banner2text">
-    "Kubernetes helped us reduce the complexity of operations to make distributed systems stable and scalable. Most importantly, we can leverage Kubernetes for scheduling resources to reduce hardware costs. That’s the&nbsp;big&nbsp;win."
-<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- HAIFENG LIU, CHIEF ARCHITECT, JD.com</span>
-  </div>
-</div>
-<section class="section2">
-<div class="fullcol">
-  <h2>With more than 300 million active users and $55.7 billion in annual revenues last year, JD.com is China’s largest retailer, and its operations are the epitome of hyperscale.</h2>
-  For example, there are more than a trillion images in JD.com’s product databases for customers, with 100 million being added daily. And this enormous amount of data needs to be instantly accessible to enable a smooth online customer experience.
-<br><br>
-  In 2014, JD.com moved its applications to containers running on bare metal machines using OpenStack and Docker to "speed up the delivery of our computing resources and make the operations much simpler," says Haifeng Liu, JD.com’s Chief Architect. But by the end of 2015, with hundreds of thousands of nodes in multiple data centers, "we encountered a lot of problems because our platform was not strong enough, and we suffered from bottlenecks and scalability issues," Liu adds. "We needed infrastructure for the next five years of development, now."
-<br><br>
-  After considering a number of orchestration technologies, JD.com decided to adopt Kubernetes to accommodate its ever-growing clusters. "The main reason is because Kubernetes can give us more efficient, scalable and much simpler application deployments, plus we can leverage it to do flexible platform scheduling," says Liu.
+<p>"We have greater data center efficiency, better managed resources, and smarter deployment with the Kubernetes platform," says Liu. Deployment time went from several hours to tens of seconds. Efficiency has improved by 20-30%, measured in IT costs. With the further optimizations the team is working on, Liu believes there is the potential to save hundreds of millions of dollars a year. But perhaps the best indication of success was the annual Singles Day shopping event, which ran on the Kubernetes platform for the first time in 2018. Over 11 days, transaction volume on JD.com was $23 billion, and "our e-commerce platforms did great," says Liu. "Infrastructure led the way to prep for 11.11. We took the approach of predicting volume, emulating the behavior of customers to prepare beforehand, and drilled for malfunctions. Because of Kubernetes's scalability, we were able to handle an extremely high level of demand."</p>

-</div>
-</section>
-<div class="banner3" style="background-image: url('/images/case-studies/jdcom/banner3.jpg')">
-  <div class="banner3text">
-    "We customized Kubernetes and built a modern system on top of it. This entire ecosystem of Kubernetes plus our own optimizations have helped us save costs and time."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- HAIFENG LIU, CHIEF ARCHITECT, JD.com</span>
+{{< case-studies/quote author="HAIFENG LIU, CHIEF ARCHITECT, JD.com" >}}
+"Kubernetes helped us reduce the complexity of operations to make distributed systems stable and scalable. Most importantly, we can leverage Kubernetes for scheduling resources to reduce hardware costs. That's the big win."
+{{< /case-studies/quote >}}

-  </div>
-</div>
-<section class="section3">
-<div class="fullcol">
-  The fact that Kubernetes is based on Google’s Borg also gave the company confidence. The team liked that Kubernetes has a clear and simple architecture, and that it’s developed mostly in Go, which is a popular language within JD.com.  Though he felt that at the time Kubernetes "was not mature enough," Liu says, "we adopted it anyway."
-<br><br>
-  The team spent a year developing the new container engine platform based on Kubernetes, and at the end of 2016, began promoting it within the company. "We wanted the cluster to be the default way for creating services, so scalability is easier," says Liu. "We talked to developers, interest grew, and we solved problems together." Some of these problems included networking performance and etcd scalability. "But during the past two years, Kubernetes has become more mature and very stable," he adds.
-<br><br>
-  Today, the company runs the world’s largest Kubernetes cluster. "We customized Kubernetes and built a modern system on top of it," says Liu. "This entire ecosystem of Kubernetes plus our own optimizations have helped us save costs and time. We have greater data center efficiency, better managed resources, and smarter deployment with the Kubernetes platform."
+{{< case-studies/lead >}}
+With more than 300 million active users and $55.7 billion in annual revenues last year, JD.com is China's largest retailer, and its operations are the epitome of hyperscale.
+{{< /case-studies/lead >}}

-</div>
-</section>
-<div class="banner4" style="background-image: url('/images/case-studies/jdcom/banner4.jpg');width:100%">
-  <div class="banner4text">
-  "My advice is first you need to combine this technology with your own businesses, and the second is you need clear goals. You cannot just use the technology because others are using it. You need to consider your own objectives." <br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- HAIFENG LIU, CHIEF ARCHITECT, JD.com</span>
-  </div>
-</div>
+<p>For example, there are more than a trillion images in JD.com's product databases for customers, with 100 million being added daily. And this enormous amount of data needs to be instantly accessible to enable a smooth online customer experience.</p>

-<section class="section5" style="padding:0px !important">
-<div class="fullcol">
-  The results are clear: Deployment time went from several hours to tens of seconds. Efficiency has improved by 20-30%, measured in IT costs. But perhaps the best indication of success was the annual <a href="https://JD.comcorporateblog.com/shoppers-snap-up-quality-and-imported-products-on-JD.com-com-for-record-breaking-singles-day-festival/">Singles Day</a> shopping event, which ran on the Kubernetes platform for the first time in 2018. Over 11 days, transaction volume on JD.com was $23 billion, and "our e-commerce platforms did great," says Liu. "Infrastructure led the way to prep for 11.11. We took the approach of predicting volume, emulating the behavior of customers to prepare beforehand, and drilled for malfunctions. Because of Kubernetes’s scalability, we were able to handle an extremely high level of demand."
-<br><br>
-  JD.com is now in its second stage with Kubernetes: The platform is already stable, scalable, and flexible, so the focus is on how to run things much more efficiently to further reduce costs. With the optimizations the team is working on with resource management, Liu believes there is the potential to save hundreds of millions of dollars a year.
-<br><br>
-  "We run Kubernetes and container clusters on roughly tens of thousands of physical bare metal nodes," he says. "Using Kubernetes and leveraging our own machine learning pipeline to predict how many resources we need for each application we use, and our own intelligent scaling algorithm, we can improve our resource usage. If we boost the resource usage, for example, by several percent, that means we can reduce huge hardware costs. Then we don’t need that many servers to get that same amount of workload. That can save us a lot of resources."
-</div>
+<p>In 2014, JD.com moved its applications to containers running on bare metal machines using OpenStack and Docker to "speed up the delivery of our computing resources and make the operations much simpler," says Haifeng Liu, JD.com's Chief Architect. But by the end of 2015, with hundreds of thousands of nodes in multiple data centers, "we encountered a lot of problems because our platform was not strong enough, and we suffered from bottlenecks and scalability issues," Liu adds. "We needed infrastructure for the next five years of development, now."</p>

-<div class="banner5" style="width:100%">
-  <div class="banner5text">
-"We can share our successful experience with the community, and we also receive good feedback from others. So it’s mutually beneficial."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- HAIFENG LIU, CHIEF ARCHITECT, JD.com</span></div>
-</div>
+<p>After considering a number of orchestration technologies, JD.com decided to adopt Kubernetes to accommodate its ever-growing clusters. "The main reason is because Kubernetes can give us more efficient, scalable and much simpler application deployments, plus we can leverage it to do flexible platform scheduling," says Liu.</p>

-<div class="fullcol">
-  JD.com, which won CNCF’s 2018 End User Award, is also using <a href="https://helm.sh/">Helm</a>, <a href="https://github.com/containernetworking">CNI</a>, <a href="https://goharbor.io/">Harbor</a>, and <a href="https://vitess.io/">Vitess</a> on its platform. JD.com developers have made considerable contributions to Vitess, the CNCF project for scalable MySQL cluster management, and the company hopes to donate its own project to CNCF in the near future. Community participation is a priority for JD.com. "We have a good partnership with this community," says Liu. "We can share our successful experience with the community, and we also receive good feedback from others. So it’s mutually beneficial."
-<br><br>
-  To that end, Liu offers this advice for other companies considering adopting cloud native technology. "First you need to combine this technology with your own businesses, and the second is you need clear goals," he says. "You cannot just use the technology because others are using it. You need to consider your own objectives."
-<br><br>
-  For JD.com’s objectives, these cloud native technologies have been an ideal fit with the company’s own homegrown innovation. "Kubernetes helped us reduce the complexity of operations to make distributed systems stable and scalable," says Liu. "Most importantly, we can leverage Kubernetes for scheduling resources to reduce hardware costs. That’s the big win."
-</div>
-</section>
+{{< case-studies/quote
+  image="/images/case-studies/jdcom/banner3.jpg"
+  author="HAIFENG LIU, CHIEF ARCHITECT, JD.com"
+>}}
+"We customized Kubernetes and built a modern system on top of it. This entire ecosystem of Kubernetes plus our own optimizations have helped us save costs and time."
+{{< /case-studies/quote >}}
+
+<p>The fact that Kubernetes is based on Google's Borg also gave the company confidence. The team liked that Kubernetes has a clear and simple architecture, and that it's developed mostly in Go, which is a popular language within JD.com.  Though he felt that at the time Kubernetes "was not mature enough," Liu says, "we adopted it anyway."</p>
+
+<p>The team spent a year developing the new container engine platform based on Kubernetes, and at the end of 2016, began promoting it within the company. "We wanted the cluster to be the default way for creating services, so scalability is easier," says Liu. "We talked to developers, interest grew, and we solved problems together." Some of these problems included networking performance and etcd scalability. "But during the past two years, Kubernetes has become more mature and very stable," he adds.</p>
+
+<p>Today, the company runs the world's largest Kubernetes cluster. "We customized Kubernetes and built a modern system on top of it," says Liu. "This entire ecosystem of Kubernetes plus our own optimizations have helped us save costs and time. We have greater data center efficiency, better managed resources, and smarter deployment with the Kubernetes platform."</p>
+
+{{< case-studies/quote
+  image="/images/case-studies/jdcom/banner4.jpg"
+  author="HAIFENG LIU, CHIEF ARCHITECT, JD.com"
+>}}
+"My advice is first you need to combine this technology with your own businesses, and the second is you need clear goals. You cannot just use the technology because others are using it. You need to consider your own objectives."
+{{< /case-studies/quote >}}
+
+<p>The results are clear: Deployment time went from several hours to tens of seconds. Efficiency has improved by 20-30%, measured in IT costs. But perhaps the best indication of success was the annual <a href="https://JD.comcorporateblog.com/shoppers-snap-up-quality-and-imported-products-on-JD.com-com-for-record-breaking-singles-day-festival/">Singles Day</a> shopping event, which ran on the Kubernetes platform for the first time in 2018. Over 11 days, transaction volume on JD.com was $23 billion, and "our e-commerce platforms did great," says Liu. "Infrastructure led the way to prep for 11.11. We took the approach of predicting volume, emulating the behavior of customers to prepare beforehand, and drilled for malfunctions. Because of Kubernetes's scalability, we were able to handle an extremely high level of demand."</p>
+
+<p>JD.com is now in its second stage with Kubernetes: The platform is already stable, scalable, and flexible, so the focus is on how to run things much more efficiently to further reduce costs. With the optimizations the team is working on with resource management, Liu believes there is the potential to save hundreds of millions of dollars a year.</p>
+
+<p>"We run Kubernetes and container clusters on roughly tens of thousands of physical bare metal nodes," he says. "Using Kubernetes and leveraging our own machine learning pipeline to predict how many resources we need for each application we use, and our own intelligent scaling algorithm, we can improve our resource usage. If we boost the resource usage, for example, by several percent, that means we can reduce huge hardware costs. Then we don't need that many servers to get that same amount of workload. That can save us a lot of resources."</p>
+
+{{< case-studies/quote author="HAIFENG LIU, CHIEF ARCHITECT, JD.com" >}}
+"We can share our successful experience with the community, and we also receive good feedback from others. So it's mutually beneficial."
+{{< /case-studies/quote >}}
+
+<p>JD.com, which won CNCF's 2018 End User Award, is also using <a href="https://helm.sh/">Helm</a>, <a href="https://github.com/containernetworking">CNI</a>, <a href="https://goharbor.io/">Harbor</a>, and <a href="https://vitess.io/">Vitess</a> on its platform. JD.com developers have made considerable contributions to Vitess, the CNCF project for scalable MySQL cluster management, and the company hopes to donate its own project to CNCF in the near future. Community participation is a priority for JD.com. "We have a good partnership with this community," says Liu. "We can share our successful experience with the community, and we also receive good feedback from others. So it's mutually beneficial."</p>
+
+<p>To that end, Liu offers this advice for other companies considering adopting cloud native technology. "First you need to combine this technology with your own businesses, and the second is you need clear goals," he says. "You cannot just use the technology because others are using it. You need to consider your own objectives."</p>
+
+<p>For JD.com's objectives, these cloud native technologies have been an ideal fit with the company's own homegrown innovation. "Kubernetes helped us reduce the complexity of operations to make distributed systems stable and scalable," says Liu. "Most importantly, we can leverage Kubernetes for scheduling resources to reduce hardware costs. That's the big win."</p>
@@ -1,113 +1,87 @@
 ---
 title: NAIC Case Study
-
 linkTitle: NAIC
 case_study_styles: true
 cid: caseStudies
-css: /css/style_case_studies.css
 logo: naic_featured_logo.png
 featured: false
+
+new_case_study_styles: true
+heading_background: /images/case-studies/naic/banner1.jpg
+heading_title_logo: /images/naic_logo.png
+subheading: >
+  A Culture and Technology Transition Enabled by Kubernetes
+case_study_details:
+  - Company: NAIC
+  - Location: Washington, DC
+  - Industry: Regulatory
 ---

-<div class="banner1" style="background-image: url('/images/case-studies/naic/banner1.jpg')">
-  <h1> CASE STUDY:<img src="/images/naic_logo.png" class="header_logo" style="width:18%"><br> <div class="subhead" style="margin-top:1%">A Culture and Technology Transition Enabled by Kubernetes</div></h1>
+<h2>Challenge</h2>

-</div>
+<p>The <a href="http://www.naic.org/">National Association of Insurance Commissioners (NAIC)</a>, the U.S. standard-setting and regulatory support organization, was looking for a way to deliver new services faster to provide more value for members and staff. It also needed greater agility to improve productivity internally.</p>

-<div class="details" style="font-size:1em">
-    Company &nbsp;<b>National Association of Insurance Commissioners (NAIC)</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location &nbsp;<b>Washington, DC</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Industry &nbsp;<b>Regulatory</b>
-</div>
-
-<hr>
-<section class="section1">
-<div class="cols">
-  <div class="col1">
-    <h2>Challenge</h2>
-        The <a href="http://www.naic.org/">National Association of Insurance Commissioners (NAIC)</a>, the U.S. standard-setting and regulatory support organization, was looking for a way to deliver new services faster to provide more value for members and staff. It also needed greater agility to improve productivity internally.
-<br><br>
- <h2>Solution</h2>
-     Beginning in 2016, they started using <a href="https://www.cncf.io/">Cloud Native Computing Foundation (CNCF)</a> tools such as <a href="https://prometheus.io/">Prometheus</a>. NAIC began hosting internal systems and development systems on <a href="https://kubernetes.io/">Kubernetes</a> at the beginning of 2018, as part of a broad move toward the public cloud. "Our culture and technology transition is a strategy embraced by our top leaders," says Dan Barker, Chief Enterprise Architect. "It has already proven successful by allowing us to accelerate our value pipeline by more than double while decreasing our costs by more than half. We are also seeing customer satisfaction increase as we add more and more applications to these new technologies."
-
-
-
-      </div>
-
-<div class="col2">
+<h2>Solution</h2>

+<p>Beginning in 2016, they started using <a href="https://www.cncf.io/">Cloud Native Computing Foundation (CNCF)</a> tools such as <a href="https://prometheus.io/">Prometheus</a>. NAIC began hosting internal systems and development systems on <a href="https://kubernetes.io/">Kubernetes</a> at the beginning of 2018, as part of a broad move toward the public cloud. "Our culture and technology transition is a strategy embraced by our top leaders," says Dan Barker, Chief Enterprise Architect. "It has already proven successful by allowing us to accelerate our value pipeline by more than double while decreasing our costs by more than half. We are also seeing customer satisfaction increase as we add more and more applications to these new technologies."</p>

 <h2>Impact</h2>
-     Leveraging Kubernetes, "our development teams can create rapid prototypes far faster than they used to," Barker said. Applications running on Kubernetes are more resilient than those running in other environments. The deployment of open source solutions is helping influence company culture, as NAIC becomes a more open and transparent organization.
-    <br><br>
-    "We completed a small prototype in two days that would have previously taken at least a month," Barker says. Resiliency is currently measured in how much downtime systems have. "They’ve basically had none, and the occasional issue is remedied in minutes," he says.

+<p>Leveraging Kubernetes, "our development teams can create rapid prototypes far faster than they used to," Barker said. Applications running on Kubernetes are more resilient than those running in other environments. The deployment of open source solutions is helping influence company culture, as NAIC becomes a more open and transparent organization.</p>

-</div>
+<p>"We completed a small prototype in two days that would have previously taken at least a month," Barker says. Resiliency is currently measured in how much downtime systems have. "They've basically had none, and the occasional issue is remedied in minutes," he says.</p>

-</div>
-</section>
-<div class="banner2">
-  <div class="banner2text">
-   "Our culture and technology transition is a strategy embraced by our top leaders. It has already proven successful by allowing us to accelerate our value pipeline by more than double while decreasing our costs by more than half. We are also seeing customer satisfaction increase as we add more and more applications to these new technologies." <br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Dan Barker, Chief Enterprise Architect, NAIC</span>
-  </div>
-</div>
-<section class="section2">
-<div class="fullcol">
- NAIC—which was created and overseen by the chief insurance regulators from the 50 states, the District of Columbia and five U.S. territories—provides a means through which state insurance regulators establish standards and best practices, conduct peer reviews, and coordinate their regulatory oversight. Their staff supports these efforts and represents the collective views of regulators in the United States and internationally. NAIC members, together with the organization’s central resources, form the national system of state-based insurance regulation in the United States.<br><br>
-The organization has been using the cloud for years, and wanted to find more ways to quickly deliver new services that provide more value for members and staff. They looked to Kubernetes for a solution. Within NAIC, several groups are leveraging Kubernetes, one being the Platform Engineering Team. "The team building out these tools are not only deploying and operating Kubernetes, but they’re also using them," Barker says. "In fact, we’re using GitLab to deploy Kubernetes with a pipeline using <a href="https://github.com/kubernetes/kops">kops</a>. This team was created from developers, operators, and quality engineers from across the company, so their jobs have changed quite a bit."<br><br>
-In addition, NAIC is onboarding teams to the new platform, and those teams have seen a lot of change in how they work and what they can do. "They now have more power in creating their own infrastructure and deploying their own applications," Barker says. They also use pipelines to facilitate their currently manual processes. NAIC has consumers who are using GitLab heavily, and they’re starting to use Kubernetes to deploy simple applications that help their internal processes.
+{{< case-studies/quote author="Dan Barker, Chief Enterprise Architect, NAIC" >}}
+"Our culture and technology transition is a strategy embraced by our top leaders. It has already proven successful by allowing us to accelerate our value pipeline by more than double while decreasing our costs by more than half. We are also seeing customer satisfaction increase as we add more and more applications to these new technologies."
+{{< /case-studies/quote >}}

+<p>NAIC—which was created and overseen by the chief insurance regulators from the 50 states, the District of Columbia and five U.S. territories—provides a means through which state insurance regulators establish standards and best practices, conduct peer reviews, and coordinate their regulatory oversight. Their staff supports these efforts and represents the collective views of regulators in the United States and internationally. NAIC members, together with the organization's central resources, form the national system of state-based insurance regulation in the United States.</p>

-</div>
-</section>
-<div class="banner3" style="background-image: url('/images/case-studies/naic/banner3.jpg')">
-  <div class="banner3text">
- "In our experience, vendor lock-in and tooling that is highly specific results in less resilient technology with fewer minds working to solve problems and grow the community." <br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Dan Barker, Chief Enterprise Architect, NAIC</span>
-  </div>
-</div>
-<section class="section3">
-<div class="fullcol">
-    "We needed greater agility to enable our own productivity internally," he says. "We decided it was right for us to move everything to the public cloud [Amazon Web Services] to help with that process and be able to access many of the native tools that allows us to move faster by not needing to build everything."
-The NAIC also wanted to be cloud-agnostic, "and Kubernetes helps with this for our compute layer," Barker says. "Compute is pretty standard across the clouds, and now we can take advantage of any of them while getting all of the other features Kubernetes offers."<br><br>
-The NAIC currently hosts internal systems and development systems on Kubernetes, and has already seen how impactful it can be. "Our development teams can create rapid prototypes in minutes instead of weeks," Barker says. "This recently happened with an internal tool that had no measurable wait time on the infrastructure. It was solely development bound. There is now a central shared resource that lives in AWS, which means it can grow as needed."
-The native integrations into Kubernetes at NAIC has made it easy to write code and have it running in minutes instead of weeks. Applications running on Kubernetes have also proven to be more resilient than those running in other environments. "We even have teams using this to create more internal tools to help with communication or automating some of their current tasks," Barker says.
-<br><br>
-"We knew that Kubernetes had become the de facto standard for container orchestration," he says. "Two major factors for selecting this were the three major cloud vendors hosting their own versions and having it hosted in a neutral party as fully open source."
-<br><br>
-As for other CNCF projects, NAIC is using Prometheus on a small scale and hopes to continue using it moving forward because of the seamless integration with Kubernetes. The Association also is considering <a href="https://grpc.io/">gRPC</a> as its internal communications standard, <a href="https://www.envoyproxy.io/">Envoy</a> in conjunction with Istio for service mesh, <a href="http://opentracing.io/">OpenTracing</a> and <a href="https://www.jaegertracing.io">Jaeger</a> for tracing aggregation, and <a href="https://www.fluentd.org/">Fluentd</a> with its Elasticsearch cluster.
+<p>The organization has been using the cloud for years, and wanted to find more ways to quickly deliver new services that provide more value for members and staff. They looked to Kubernetes for a solution. Within NAIC, several groups are leveraging Kubernetes, one being the Platform Engineering Team. "The team building out these tools are not only deploying and operating Kubernetes, but they're also using them," Barker says. "In fact, we're using GitLab to deploy Kubernetes with a pipeline using <a href="https://github.com/kubernetes/kops">kops</a>. This team was created from developers, operators, and quality engineers from across the company, so their jobs have changed quite a bit."</p>

-</div>
-</section>
-<div class="banner4" style="background-image: url('/images/case-studies/naic/banner4.jpg')">
-  <div class="banner4text">
-     "We knew that Kubernetes had become the de facto standard for container orchestration. Two major factors for selecting this were the three major cloud vendors hosting their own versions and having it hosted in a neutral party as fully open source."<span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br><br>- Dan Barker, Chief Enterprise Architect, NAIC</span>
-  </div>
-  </div>
-</div>
+<p>In addition, NAIC is onboarding teams to the new platform, and those teams have seen a lot of change in how they work and what they can do. "They now have more power in creating their own infrastructure and deploying their own applications," Barker says. They also use pipelines to facilitate their currently manual processes. NAIC has consumers who are using GitLab heavily, and they're starting to use Kubernetes to deploy simple applications that help their internal processes.</p>

-<section class="section5" style="padding:0px !important">
-<div class="fullcol">
+{{< case-studies/quote
+  image="/images/case-studies/naic/banner3.jpg"
+  author="Dan Barker, Chief Enterprise Architect, NAIC"
+>}}
+"In our experience, vendor lock-in and tooling that is highly specific results in less resilient technology with fewer minds working to solve problems and grow the community."
+{{< /case-studies/quote >}}

-The open governance and broad industry participation in CNCF provided a comfort level with the technology, Barker says. "We also see it as helping to influence our own company culture," he says. "We’re moving to be a more open and transparent company, and we are encouraging our staff to get involved with the different working groups and codebases. We recently became CNCF members to help further our commitment to community contribution and transparency."<br><br>
-Factors such as vendor-neutrality and cross-industry investment were important in the selection. "In our experience, vendor lock-in and tooling that is highly specific results in less resilient technology with fewer minds working to solve problems and grow the community," Barker says.<br><br>
-NAIC is a largely Oracle shop, Barker says, and has been running mostly Java on JBoss. "However, we have years of history with other applications," he says. "Some of these have been migrated by completely rewriting the application, while others are just being modified slightly to fit into this new paradigm."<br><br>
-Running on AWS cloud, the Association has not specifically taken a microservices approach. "We are moving to microservices where practical, but we haven’t found that it’s a necessity to operate them within Kubernetes," Barker says<br><br>
-All of its databases are currently running within public cloud services, but they have explored eventually running those in Kubernetes, as it makes sense. "We’re doing this to get more reuse from common components and to limit our failure domains to something more manageable and observable," Barker says.
+<p>"We needed greater agility to enable our own productivity internally," he says. "We decided it was right for us to move everything to the public cloud [Amazon Web Services] to help with that process and be able to access many of the native tools that allows us to move faster by not needing to build everything."
+The NAIC also wanted to be cloud-agnostic, "and Kubernetes helps with this for our compute layer," Barker says. "Compute is pretty standard across the clouds, and now we can take advantage of any of them while getting all of the other features Kubernetes offers."</p>

+<p>The NAIC currently hosts internal systems and development systems on Kubernetes, and has already seen how impactful it can be. "Our development teams can create rapid prototypes in minutes instead of weeks," Barker says. "This recently happened with an internal tool that had no measurable wait time on the infrastructure. It was solely development bound. There is now a central shared resource that lives in AWS, which means it can grow as needed."</p>

-</div>
+<p>The native integrations into Kubernetes at NAIC has made it easy to write code and have it running in minutes instead of weeks. Applications running on Kubernetes have also proven to be more resilient than those running in other environments. "We even have teams using this to create more internal tools to help with communication or automating some of their current tasks," Barker says.</p>

-<div class="banner5">
-  <div class="banner5text">
-    "We have been able to move much faster at lower cost than we were able to in the past," Barker says. "We were able to complete one of our projects in a year, when the previous version took over two years. And the new project cost $500,000 while the original required $3 million, and with fewer defects. We are also able to push out new features much faster."<span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br><br>- Dan Barker, Chief Enterprise Architect, NAIC</span>
+<p>"We knew that Kubernetes had become the de facto standard for container orchestration," he says. "Two major factors for selecting this were the three major cloud vendors hosting their own versions and having it hosted in a neutral party as fully open source."</p>

-  </div>
-</div>
-<div class="fullcol">
+<p>As for other CNCF projects, NAIC is using Prometheus on a small scale and hopes to continue using it moving forward because of the seamless integration with Kubernetes. The Association also is considering <a href="https://grpc.io/">gRPC</a> as its internal communications standard, <a href="https://www.envoyproxy.io/">Envoy</a> in conjunction with Istio for service mesh, <a href="http://opentracing.io/">OpenTracing</a> and <a href="https://www.jaegertracing.io">Jaeger</a> for tracing aggregation, and <a href="https://www.fluentd.org/">Fluentd</a> with its Elasticsearch cluster.</p>

-NAIC has seen a significant business impact from its efforts. "We have been able to move much faster at lower cost than we were able to in the past," Barker says. "We were able to complete one of our projects in a year, when the previous version took over two years. And the new project cost $500,000 while the original required $3 million, and with fewer defects. We are also able to push out new features much faster."
-He says the organization is moving toward continuous deployment "because the business case makes sense. The research is becoming very hard to argue with. We want to reduce our batch sizes and optimize on delivering value to customers and not feature count. This is requiring a larger cultural shift than just a technology shift."
-NAIC is "becoming more open and transparent, as well as more resilient to failure," Barker says. "Even our customers are wanting more and more of this and trying to figure out how they can work with us to accomplish our mutual goals faster. Members of the insurance industry have reached out so that we can better learn together and grow as an industry."
+{{< case-studies/quote
+  image="/images/case-studies/naic/banner4.jpg"
+  author="Dan Barker, Chief Enterprise Architect, NAIC"
+>}}
+"We knew that Kubernetes had become the de facto standard for container orchestration. Two major factors for selecting this were the three major cloud vendors hosting their own versions and having it hosted in a neutral party as fully open source."
+{{< /case-studies/quote >}}

-</div>
+<p>The open governance and broad industry participation in CNCF provided a comfort level with the technology, Barker says. "We also see it as helping to influence our own company culture," he says. "We're moving to be a more open and transparent company, and we are encouraging our staff to get involved with the different working groups and codebases. We recently became CNCF members to help further our commitment to community contribution and transparency."</p>

-</section>
+<p>Factors such as vendor-neutrality and cross-industry investment were important in the selection. "In our experience, vendor lock-in and tooling that is highly specific results in less resilient technology with fewer minds working to solve problems and grow the community," Barker says.</p>
+
+<p>NAIC is a largely Oracle shop, Barker says, and has been running mostly Java on JBoss. "However, we have years of history with other applications," he says. "Some of these have been migrated by completely rewriting the application, while others are just being modified slightly to fit into this new paradigm."</p>
+
+<p>Running on AWS cloud, the Association has not specifically taken a microservices approach. "We are moving to microservices where practical, but we haven't found that it's a necessity to operate them within Kubernetes," Barker says.</p>
+
+<p>All of its databases are currently running within public cloud services, but they have explored eventually running those in Kubernetes, as it makes sense. "We're doing this to get more reuse from common components and to limit our failure domains to something more manageable and observable," Barker says.</p>
+
+{{< case-studies/quote author="Dan Barker, Chief Enterprise Architect, NAIC" >}}
+"We have been able to move much faster at lower cost than we were able to in the past," Barker says. "We were able to complete one of our projects in a year, when the previous version took over two years. And the new project cost $500,000 while the original required $3 million, and with fewer defects. We are also able to push out new features much faster."
+{{< /case-studies/quote >}}
+
+<p>NAIC has seen a significant business impact from its efforts. "We have been able to move much faster at lower cost than we were able to in the past," Barker says. "We were able to complete one of our projects in a year, when the previous version took over two years. And the new project cost $500,000 while the original required $3 million, and with fewer defects. We are also able to push out new features much faster."</p>
+
+<p>He says the organization is moving toward continuous deployment "because the business case makes sense. The research is becoming very hard to argue with. We want to reduce our batch sizes and optimize on delivering value to customers and not feature count. This is requiring a larger cultural shift than just a technology shift."</p>
+
+<p>NAIC is "becoming more open and transparent, as well as more resilient to failure," Barker says. "Even our customers are wanting more and more of this and trying to figure out how they can work with us to accomplish our mutual goals faster. Members of the insurance industry have reached out so that we can better learn together and grow as an industry."</p>
@@ -1,95 +1,81 @@
 ---
 title: Nav Case Study
-
 linkTitle: Nav
 case_study_styles: true
 cid: caseStudies
-css: /css/style_case_studies.css
 featured: false
+
+new_case_study_styles: true
+heading_background: /images/case-studies/nav/banner1.jpg
+heading_title_logo: /images/nav_logo.png
+subheading: >
+  How A Startup Reduced Its Infrastructure Costs by 50% With Kubernetes
+case_study_details:
+  - Company: Nav
+  - Location: Salt Lake City, Utah, and San Mateo, California
+  - Industry: Financial services for businesses
 ---

-<div class="banner1" style="background-image: url('/images/case-studies/nav/banner1.jpg')">
-  <h1> CASE STUDY:<img src="/images/nav_logo.png" class="header_logo" style="width:15%"><br> <div class="subhead" style="margin-top:1%">How A Startup Reduced Its Infrastructure Costs by 50% With Kubernetes
-</div></h1>
+<h2>Challenge</h2>

-</div>
+<p>Founded in 2012, <a href="https://www.nav.com/">Nav</a> provides small business owners with access to their business credit scores from all three major commercial credit bureaus—Equifax, Experian and Dun & Bradstreet—and financing options that best fit their needs. Five years in, the startup was growing rapidly, and "our cloud environments were getting very large, and our usage of those environments was extremely low, like under 1%," says Director of Engineering Travis Jeppson. "We wanted our usage of cloud environments to be more tightly coupled with what we actually needed, so we started looking at containerization and orchestration to help us be able to run workloads that were distinct from one another but could share a similar resource pool."</p>

-<div class="details">
-    Company &nbsp;<b>Nav</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location &nbsp;<b>Salt Lake City, Utah, and San Mateo, California</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Industry &nbsp;<b>Financial services for businesses</b>
-</div>
+<h2>Solution</h2>
+
+<p>After evaluating a number of orchestration solutions, the Nav team decided to adopt <a href="https://kubernetes.io/">Kubernetes</a> running on <a href="https://aws.amazon.com/">AWS</a>. The strength of the community around Kubernetes was a strong draw, as well as its Google provenance. Plus, "the other solutions tended to be fairly heavy-handed, really complex, really large, and really hard to manage just off the bat," says Jeppson. "Kubernetes gave us a very simple way to be able to step into an orchestration solution that fit our needs at the time, but also the extensibility of it allowed us to be able to grow with it and be able to build in more features and functionality later on."</p>

-<hr>
-<section class="section1">
-<div class="cols">
-  <div class="col1" style="width:100%;margin-left:-1.5%">
-<br><br>
-    <h2>Challenge</h2>
-    Founded in 2012, <a href="https://www.nav.com/">Nav</a> provides small business owners with access to their business credit scores from all three major commercial credit bureaus—Equifax, Experian and Dun & Bradstreet—and financing options that best fit their needs. Five years in, the startup was growing rapidly, and “our cloud environments were getting very large, and our usage of those environments was extremely low, like under 1%,” says Director of Engineering Travis Jeppson. “We wanted our usage of cloud environments to be more tightly coupled with what we actually needed, so we started looking at containerization and orchestration to help us be able to run workloads that were distinct from one another but could share a similar resource pool.”
-    <br><br>
-    <h2>Solution</h2>
-    After evaluating a number of orchestration solutions, the Nav team decided to adopt <a href="https://kubernetes.io/">Kubernetes</a> running on <a href="https://aws.amazon.com/">AWS</a>. The strength of the community around Kubernetes was a strong draw, as well as its Google provenance. Plus, “the other solutions tended to be fairly heavy-handed, really complex, really large, and really hard to manage just off the bat,” says Jeppson. “Kubernetes gave us a very simple way to be able to step into an orchestration solution that fit our needs at the time, but also the extensibility of it allowed us to be able to grow with it and be able to build in more features and functionality later on.”
-    <br><br>
 <h2>Impact</h2>
-    The four-person team got Kubernetes up and running in six months, and the full migration of Nav’s 25 microservices was completed in another six months. The results have been impressive: Resource utilization, which led the company on this path in the first place, has increased from 1% to 40%. Launching a new service used to take two developers two weeks; now it takes only one developer less than 10 minutes. Deployments have increased 5x. And the company is saving 50% in infrastructure costs.

-</div>
-</div>
-</section>
-<div class="banner2">
-  <div class="banner2text">
-        <iframe width="500" height="320" src="https://www.youtube.com/embed/0IbOk1_H-f0" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
-        <br><br>
-    "Kubernetes gave us a very simple way to be able to step into an orchestration solution that fit our needs at the time, but also the extensibility of it allowed us to be able to grow with it and be able to build in more features and functionality&nbsp;later&nbsp;on."
-<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Travis Jeppson, Director of Engineering, Nav</span>
-  </div>
-</div>
-<section class="section2">
-<div class="fullcol">
-  <h2>Founded in 2012, <a href="https://www.nav.com/">Nav</a> provides small business owners with access to their business credit scores from all three major commercial credit bureaus—Equifax, Experian and Dun & Bradstreet—as well as details on their businesses’ financial health and financing options that best fit their needs. Its mission boils down to this, says Director of Engineering Travis Jeppson: “to increase the success rate of small businesses.”</h2>
-  A couple of years ago, Nav recognized an obstacle in its own path to success. The business was growing rapidly, and “our cloud environments were getting very large, and our usage of those environments was extremely low, like under 1%,” says Jeppson. “Most of the problem was around the ability to scale. We were just throwing money at it. ‘Let’s just spin up more servers. Let’s just do more things in order to handle an increased load.’ And with us being a startup, that could lead to our demise. We don’t have the money to burn on that kind of stuff.”<br><br>
-  Plus, every new service had to go through 10 different people, taking an unacceptably long two weeks to launch. “All of the patch management and the server management was done very manually, and so we all had to watch it and maintain it really well,” adds Jeppson. “It was just a very troublesome system.”
+<p>The four-person team got Kubernetes up and running in six months, and the full migration of Nav's 25 microservices was completed in another six months. The results have been impressive: Resource utilization, which led the company on this path in the first place, has increased from 1% to 40%. Launching a new service used to take two developers two weeks; now it takes only one developer less than 10 minutes. Deployments have increased 5x. And the company is saving 50% in infrastructure costs.</p>

+{{< case-studies/quote author="Travis Jeppson, Director of Engineering, Nav" >}}
+<iframe width="500" height="320" src="https://www.youtube.com/embed/0IbOk1_H-f0" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
+<br>
+"Kubernetes gave us a very simple way to be able to step into an orchestration solution that fit our needs at the time, but also the extensibility of it allowed us to be able to grow with it and be able to build in more features and functionality later on."
+{{< /case-studies/quote >}}

-</div>
-</section>
-<div class="banner3" style="background-image: url('/images/case-studies/nav/banner3.jpg')">
-  <div class="banner3text">
-    "The community is absolutely vital: being able to pass ideas around, talk about a lot of the similar challenges that we’re all facing, and just get help. I like that we’re able to tackle the same problems for different reasons but help each other along the way."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Travis Jeppson, Director of Engineering, Nav</span>
+{{< case-studies/lead >}}
+Founded in 2012, <a href="https://www.nav.com/">Nav</a> provides small business owners with access to their business credit scores from all three major commercial credit bureaus—Equifax, Experian and Dun & Bradstreet—as well as details on their businesses' financial health and financing options that best fit their needs. Its mission boils down to this, says Director of Engineering Travis Jeppson: "to increase the success rate of small businesses."
+{{< /case-studies/lead >}}

-  </div>
-</div>
-<section class="section3">
-<div class="fullcol">
-  Jeppson had worked with containers at his previous job, and pitched that technology to Nav’s management as a solution to these problems. He got the green light in early 2017. “We wanted our usage of cloud environments to be more tightly coupled with what we actually needed, so we started looking at containerization and orchestration to help us be able to run workloads that were distinct from one another but could share a similar resource pool,” he says.<br><br>
-  After evaluating a number of orchestration solutions, the company decided to adopt <a href="https://kubernetes.io/">Kubernetes</a> running on <a href="https://aws.amazon.com/">AWS</a>. The strength of the community around Kubernetes was a strong draw, as was its Google origins. Additionally, “the other solutions tended to be fairly heavy-handed, really complex, really large, and really hard to manage just off the bat,” says Jeppson. “Kubernetes gave us a very simple way to be able to step into an orchestration solution that fit our needs at the time, but the extensibility of it would also allow us to grow with it and build in more features and functionality later on.”<br><br>
-  Jeppson’s four-person Engineering Services team got Kubernetes up and running in six months (they decided to use <a href="http://kubespray.io/">Kubespray</a> to spin up clusters), and the full migration of Nav’s 25 microservices and one primary monolith was completed in another six months. “We couldn’t rewrite everything; we couldn’t stop,” he says. “We had to stay up, we had to stay available, and we had to have minimal amount of downtime. So we got really comfortable around our building pipeline, our metrics and logging, and then around Kubernetes itself: how to launch it, how to upgrade it, how to service it. And we moved little by little.”
-</div>
-</section>
-<div class="banner4" style="background-image: url('/images/case-studies/nav/banner4.jpg')" style="width:100%">
-  <div class="banner4text">
-  “Kubernetes has brought so much value to Nav by allowing all of these new freedoms that we had just never had before.” <br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Travis Jeppson, Director of Engineering, Nav</span>
-  </div>
-</div>
+<p>A couple of years ago, Nav recognized an obstacle in its own path to success. The business was growing rapidly, and "our cloud environments were getting very large, and our usage of those environments was extremely low, like under 1%," says Jeppson. "Most of the problem was around the ability to scale. We were just throwing money at it. 'Let's just spin up more servers. Let's just do more things in order to handle an increased load.' And with us being a startup, that could lead to our demise. We don't have the money to burn on that kind of stuff."</p>

-<section class="section5" style="padding:0px !important">
+<p>Plus, every new service had to go through 10 different people, taking an unacceptably long two weeks to launch. "All of the patch management and the server management was done very manually, and so we all had to watch it and maintain it really well," adds Jeppson. "It was just a very troublesome system."</p>

-<div class="fullcol">
-  A crucial part of the process involved educating Nav’s 50 engineers and being transparent regarding the new workflow as well as the roadmap for the migration. Jeppson did regular presentations along the way, and a week of four-hours-a-day labs for the entire staff of engineers. He then created a repository in <a href="https://gitlab.com/">GitLab</a> to house all of the information. “We showed all the frontend and backend developers how to go in, create their own namespace using <a href="https://kubernetes.io/docs/tasks/tools/install-kubectl/">kubectl</a>, all themselves,” he says. “Now, a lot of times, they just come to us and say, ‘This is ready.’ We click a little button in GitLab to allow it to release into production, and they’re off to the races.”<br><br>
-  Since the migration was completed in early 2018, the results have been impressive: Resource utilization, which led the company on this path in the first place, has increased from 1% to 40%. Launching a new service used to take two developers two weeks; now it takes only one developer less than 10 minutes. Deployments have increased 5x, from 10 a day to 50 a day. And the company is saving 50% in infrastructure costs on the computational side. “Next we want to go in to address the database side, and once we do that, then we’re going to continue to drop that cost quite a bit more,” says Jeppson.<br><br>
-  Kubernetes has also helped Nav with its compliance needs. Before, “we had to map one application to one server, mostly due to different compliance regulations around data,” Jeppson says. “With the Kubernetes API, we could add in network policies and segregate that data and restrict it if needed.” The company segregates its cluster into an unrestricted zone and a restricted zone, which has its own set of nodes where data protection happens. The company also uses the <a href="https://www.twistlock.com/">Twistlock</a> tool to ensure security, “and that makes it a lot easier to sleep at night,” he adds.
+{{< case-studies/quote
+  image="/images/case-studies/nav/banner3.jpg"
+  author="Travis Jeppson, Director of Engineering, Nav"
+>}}
+"The community is absolutely vital: being able to pass ideas around, talk about a lot of the similar challenges that we're all facing, and just get help. I like that we're able to tackle the same problems for different reasons but help each other along the way."
+{{< /case-studies/quote >}}

-</div>
+<p>Jeppson had worked with containers at his previous job, and pitched that technology to Nav's management as a solution to these problems. He got the green light in early 2017. "We wanted our usage of cloud environments to be more tightly coupled with what we actually needed, so we started looking at containerization and orchestration to help us be able to run workloads that were distinct from one another but could share a similar resource pool," he says.</p>

-<div class="banner5" >
-  <div class="banner5text">
-"We’re talking four to 10 times the amount of traffic that we handle now, and it’s just like, 'Oh, yeah. We’re good. Kubernetes handles this for us.'"<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Travis Jeppson, Director of Engineering, Nav</span></div>
-</div>
+<p>After evaluating a number of orchestration solutions, the company decided to adopt <a href="https://kubernetes.io/">Kubernetes</a> running on <a href="https://aws.amazon.com/">AWS</a>. The strength of the community around Kubernetes was a strong draw, as was its Google origins. Additionally, "the other solutions tended to be fairly heavy-handed, really complex, really large, and really hard to manage just off the bat," says Jeppson. "Kubernetes gave us a very simple way to be able to step into an orchestration solution that fit our needs at the time, but the extensibility of it would also allow us to grow with it and build in more features and functionality later on."</p>

-<div class="fullcol">
-  With Kubernetes in place, the Nav team also started improving the system’s metrics and logging by adopting <a href="https://prometheus.io/">Prometheus</a>. “Prometheus created a standard around metrics that was really easy for a developer to adopt,” says Jeppson. “They have the freedom to display what they want, to do what they need, and keep their codebase clean, and that to us was absolutely a must.”<br><br>
-  Next up for Nav in the coming year: looking at tracing, storage, and service mesh. They’re currently evaluating <a href="https://www.envoyproxy.io/">Envoy</a>, <a href="https://opentracing.io/">OpenTracing</a>, and <a href="https://www.jaegertracing.io/">Jaeger</a> after spending much of KubeCon talking to other companies. “The community is absolutely vital: being able to pass ideas around, talk about a lot of the similar challenges that we’re all facing, and just get help. I like that we’re able to tackle the same problems for different reasons but help each other along the way,” says Jeppson. “There’s still so, so much to do around scalability, around being able to really fully adopt a cloud native solution.”<br><br>
-  Of course, it all starts with Kubernetes. With that technology, Jeppson’s team has built a platform that allows Nav to scale, and that “has brought so much value to Nav by allowing all of these new freedoms that we had just never had before,” he says. <br><br>
-  Conversations about new products used to be bogged down by the fact they’d have to wait six months to get an environment set up with isolation and then figure out how to handle spikes of traffic. “But now it’s just nothing to us,” says Jeppson. “We’re talking four to 10 times the amount of traffic that we handle now, and it’s just like, ‘Oh, yeah. We’re good. Kubernetes handles this for us.’”
+<p>Jeppson's four-person Engineering Services team got Kubernetes up and running in six months (they decided to use <a href="http://kubespray.io/">Kubespray</a> to spin up clusters), and the full migration of Nav's 25 microservices and one primary monolith was completed in another six months. "We couldn't rewrite everything; we couldn't stop," he says. "We had to stay up, we had to stay available, and we had to have minimal amount of downtime. So we got really comfortable around our building pipeline, our metrics and logging, and then around Kubernetes itself: how to launch it, how to upgrade it, how to service it. And we moved little by little."</p>

-</div>
-</section>
+{{< case-studies/quote
+  image="/images/case-studies/nav/banner4.jpg"
+  author="Travis Jeppson, Director of Engineering, Nav"
+>}}
+"Kubernetes has brought so much value to Nav by allowing all of these new freedoms that we had just never had before."
+{{< /case-studies/quote >}}
+
+<p>A crucial part of the process involved educating Nav's 50 engineers and being transparent regarding the new workflow as well as the roadmap for the migration. Jeppson did regular presentations along the way, and a week of four-hours-a-day labs for the entire staff of engineers. He then created a repository in <a href="https://gitlab.com/">GitLab</a> to house all of the information. "We showed all the frontend and backend developers how to go in, create their own namespace using <a href="https://kubernetes.io/docs/tasks/tools/install-kubectl/">kubectl</a>, all themselves," he says. "Now, a lot of times, they just come to us and say, 'This is ready.' We click a little button in GitLab to allow it to release into production, and they're off to the races."</p>
+
+<p>Since the migration was completed in early 2018, the results have been impressive: Resource utilization, which led the company on this path in the first place, has increased from 1% to 40%. Launching a new service used to take two developers two weeks; now it takes only one developer less than 10 minutes. Deployments have increased 5x, from 10 a day to 50 a day. And the company is saving 50% in infrastructure costs on the computational side. "Next we want to go in to address the database side, and once we do that, then we're going to continue to drop that cost quite a bit more," says Jeppson.</p>
+
+<p>Kubernetes has also helped Nav with its compliance needs. Before, "we had to map one application to one server, mostly due to different compliance regulations around data," Jeppson says. "With the Kubernetes API, we could add in network policies and segregate that data and restrict it if needed." The company segregates its cluster into an unrestricted zone and a restricted zone, which has its own set of nodes where data protection happens. The company also uses the <a href="https://www.twistlock.com/">Twistlock</a> tool to ensure security, "and that makes it a lot easier to sleep at night," he adds.</p>
+
+{{< case-studies/quote author="Travis Jeppson, Director of Engineering, Nav" >}}
+"We're talking four to 10 times the amount of traffic that we handle now, and it's just like, 'Oh, yeah. We're good. Kubernetes handles this for us.'"
+{{< /case-studies/quote >}}
+
+<p>With Kubernetes in place, the Nav team also started improving the system's metrics and logging by adopting <a href="https://prometheus.io/">Prometheus</a>. "Prometheus created a standard around metrics that was really easy for a developer to adopt," says Jeppson. "They have the freedom to display what they want, to do what they need, and keep their codebase clean, and that to us was absolutely a must."</p>
+
+<p>Next up for Nav in the coming year: looking at tracing, storage, and service mesh. They're currently evaluating <a href="https://www.envoyproxy.io/">Envoy</a>, <a href="https://opentracing.io/">OpenTracing</a>, and <a href="https://www.jaegertracing.io/">Jaeger</a> after spending much of KubeCon talking to other companies. "The community is absolutely vital: being able to pass ideas around, talk about a lot of the similar challenges that we're all facing, and just get help. I like that we're able to tackle the same problems for different reasons but help each other along the way," says Jeppson. "There's still so, so much to do around scalability, around being able to really fully adopt a cloud native solution."</p>
+
+<p>Of course, it all starts with Kubernetes. With that technology, Jeppson's team has built a platform that allows Nav to scale, and that "has brought so much value to Nav by allowing all of these new freedoms that we had just never had before," he says.</p>
+
+<p>Conversations about new products used to be bogged down by the fact they'd have to wait six months to get an environment set up with isolation and then figure out how to handle spikes of traffic. "But now it's just nothing to us," says Jeppson. "We're talking four to 10 times the amount of traffic that we handle now, and it's just like, 'Oh, yeah. We're good. Kubernetes handles this for us.'"</p>
@@ -3,94 +3,79 @@ title: Prowise Case Study
 linkTitle: prowise
 case_study_styles: true
 cid: caseStudies
-css: /css/style_case_studies.css
 featured: false
+
+new_case_study_styles: true
+heading_background: /images/case-studies/nerdalize/banner1.jpg
+heading_title_logo: /images/nerdalize_logo.png
+subheading: >
+  Nerdalize: Providing Affordable and Sustainable Cloud Hosting with Kubernetes
+case_study_details:
+  - Company: Nerdalize
+  - Location: Delft, Netherlands
+  - Industry: Cloud Provider
 ---
-<div class="banner1" style="background-image: url('/images/case-studies/nerdalize/banner1.jpg')">
-  <h1> CASE STUDY:<img src="/images/nerdalize_logo.png" class="header_logo" style="width:25%;margin-bottom:-1%"><br> <div class="subhead" style="margin-top:1%">Nerdalize: Providing Affordable and Sustainable Cloud Hosting with Kubernetes
-</div></h1>

-</div>
+<h2>Challenge</h2>

-<div class="details">
-    Company &nbsp;<b>Nerdalize</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location &nbsp;<b>Delft, Netherlands  </b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Industry &nbsp;<b>Cloud Provider</b>
-</div>
+<p>Nerdalize offers affordable cloud hosting for customers—and free heat and hot water for people who sign up to house the heating devices that contain the company's servers. The savings Nerdalize realizes by not running data centers are passed on to its customers. When the team began using Docker to make its software more portable, it realized it also needed a container orchestration solution. "As a cloud provider, we have internal services for hosting our backends and billing our customers, but we also need to offer our compute to our end users," says Digital Product Engineer Ad van der Veer. "Since we have these heating devices spread across the Netherlands, we need some way of tying that all together."</p>

-<hr>
-<section class="section1">
-<div class="cols">
-  <div class="col1" style="width:100%"">
+<h2>Solution</h2>
+
+<p>After briefly using a basic scheduling setup with another open source tool, Nerdalize switched to Kubernetes. "On top of our heating devices throughout the Netherlands, we have a virtual machine layer, and on top of that we run Kubernetes clusters for our customers," says van der Veer. "As a small company, we have to provide a rock solid story in terms of the technology. Kubernetes allows us to offer a hybrid solution: 'You can run this on our cloud, but you can run it on other clouds as well. It runs in your internal hardware if you like.' And together with the Docker image standard and our multi-cloud dashboard, that allows them peace of mind."</p>

-    <h2>Challenge</h2>
-    Nerdalize offers affordable cloud hosting for customers—and free heat and hot water for people who sign up to house the heating devices that contain the company’s servers. The savings Nerdalize realizes by not running data centers are passed on to its customers. When the team began using Docker to make its software more portable, it realized it also needed a container orchestration solution. “As a cloud provider, we have internal services for hosting our backends and billing our customers, but we also need to offer our compute to our end users,” says Digital Product Engineer Ad van der Veer. “Since we have these heating devices spread across the Netherlands, we need some way of tying that all together.”
-    <h2>Solution</h2>
-    After briefly using a basic scheduling setup with another open source tool, Nerdalize switched to Kubernetes. “On top of our heating devices throughout the Netherlands, we have a virtual machine layer, and on top of that we run Kubernetes clusters for our customers,” says van der Veer. “As a small company, we have to provide a rock solid story in terms of the technology. Kubernetes allows us to offer a hybrid solution: ‘You can run this on our cloud, but you can run it on other clouds as well. It runs in your internal hardware if you like.’ And together with the Docker image standard and our multi-cloud dashboard, that allows them peace of mind.”
 <h2>Impact</h2>
-    Nerdalize prides itself on being a Kubernetes-native cloud provider that charges its customers prices 40% below that of other cloud providers. “Every euro that we have to invest for licensing of software that’s not open source comes from that 40%,” says van der Veer. If they had used a non-open source orchestration platform instead of Kubernetes, “that would reduce this proposition that we have of 40% less cost to like 30%. Kubernetes directly allows us to have this business model and this strategic advantage.” Nerdalize customers also benefit from time savings: One went from spending a day to set up VMs, network, and software, to spinning up a Kubernetes cluster in minutes. And for households using the heating devices, they save an average of 200 euro a year on their heating bill. The environmental impact? The annual reduction in CO2 emissions comes out to be 2 tons per Nerdalize household, which is equivalent to a car driving 8,000 km.
-</div>
-</div>
-</section>
-<div class="banner2">
-  <div class="banner2text">
-  “We can walk into a boardroom and put a Kubernetes logo up, and people accept it as an established technology. It becomes this centerpiece where other cloud native projects can tie in, so there’s a network effect that each project empowers each other. This is something that has a lot of value when we have to talk to customers and convince them that our cloud fits their&nbsp;needs.”
-<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>— AD VAN DER VEER, PRODUCT ENGINEER, NERDALIZE</span>
-  </div>
-</div>
-<section class="section2">
-<div class="fullcol">
-  <h2>Nerdalize is a cloud hosting provider that has no data centers. Instead, the four-year-old startup places its servers in homes across the Netherlands, inside heating devices it developed to turn the heat produced by the servers into heating and hot water for the residents.
- </h2>
- “Households save on their gas bills, and cloud users have a much more sustainable cloud solution,” says Maaike Stoops, Customer Experience Queen at Nerdalize. “And we don’t have the overhead of building a data center, so our cloud is up to 40% more affordable.”
-<br><br>
-  That business model has been enabled by the company’s adoption of containerization and Kubernetes. “When we just got started, Docker was just introduced,” says Digital Product Engineer Ad van der Veer. “We began with a very basic bare metal setup, but once we developed the business, we saw that containerization technology was super useful to help our customers. As a cloud provider, we have internal services for hosting our backends and billing our customers, but we also need to offer our compute to our end users. Since we have these heating devices spread across the Netherlands, we need some way of tying that all together.”
-<br><br>
-  After trying to develop its own scheduling system using another open source tool, Nerdalize found Kubernetes. “Kubernetes provided us with more functionality out of the gate,” says van der Veer.
-</div>
-</section>
-<div class="banner3" style="background-image: url('/images/case-studies/nerdalize/banner3.jpg')">
-  <div class="banner3text">
-    “We always try to get a working version online first, like minimal viable products, and then move to stabilize that,” says van der Veer. “And I think that these kinds of day-two problems are now immediately solved. The rapid prototyping we saw internally is a very valuable aspect of Kubernetes.”<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>— AD VAN DER VEER, PRODUCT ENGINEER, NERDALIZE</span>

-  </div>
-</div>
-<section class="section3">
-<div class="fullcol">
-  The team first experimented with a basic use case to run customers’ workloads on Kubernetes. “Getting the data working was kind of difficult, and at the time the installation wasn’t that simple,” says van der Veer. “Then CNCF started, we saw the community grow, these problems got solved, and from there it became a very easy decision.”
-<br><br>
-  The first Nerdalize product that was launched in 2017 was “100% containerized and Kubernetes native,” says van der Veer. “On top of our heating devices throughout the Netherlands, we have a virtual machine layer, and on top of that we run Kubernetes clusters for our customers. As a small company, we have to provide a rock solid story in terms of the technology. Kubernetes allows us to offer a hybrid solution: ‘You can run this on our cloud, but you can run it on other clouds as well. It runs in your internal hardware if you like.’ And together with the Docker image standard and our multi-cloud dashboard, that gives them peace of mind.”
-<br><br>
-  Not to mention the 40% cost savings. “Every euro that we have to invest for licensing of software that’s not open source comes from that 40%,” says van der Veer. If Nerdalize had used a non-open source orchestration platform instead of Kubernetes, “that would reduce our cost savings proposition to like 30%. Kubernetes directly allows us to have this business model and this strategic advantage.”
-</div>
-</section>
-<div class="banner4" style="background-image: url('/images/case-studies/nerdalize/banner4.jpg')" style="width:100%">
-  <div class="banner4text">
-    “One of our customers used to spend up to a day setting up the virtual machines, network and software every time they wanted to run a project in the cloud. On our platform, with Docker and Kubernetes, customers can have their projects running in a couple of minutes.”
-<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- MAAIKE STOOPS, CUSTOMER EXPERIENCE QUEEN, NERDALIZE</span>
-  </div>
-</div>
-<section class="section5" style="padding:0px !important">
-<div class="fullcol">
-  Nerdalize now has customers, from individual engineers to data-intensive startups and established companies, all around the world. (For the time being, though, the heating devices are exclusive to the Netherlands.) One of the most common use cases is batch workloads used by data scientists and researchers, and the time savings for these end users is profound. “One of our customers used to spend up to a day setting up the virtual machines, network and software every time they wanted to run a project in the cloud,” says Stoops. “On our platform, with Docker and Kubernetes, customers can have their projects running in a couple of minutes.”
-<br><br>
-  As for households using the heating devices, they save an average of 200 euro a year on their heating bill. The environmental impact? The annual reduction in CO2 emissions comes out to 2 tons per Nerdalize household, which is equivalent to a car driving 8,000 km.
-<br><br>
-  For the Nerdalize team, feature development—such as the accessible command line interface called Nerd, which recently went live—has also been sped up by Kubernetes. “We always try to get a working version online first, like minimal viable products, and then move to stabilize that,” says van der Veer. “And I think that these kinds of day-two problems are now immediately solved. The rapid prototyping we saw internally is a very valuable aspect of Kubernetes.”
-<br><br>
-  Another unexpected benefit has been the growing influence and reputation of Kubernetes. “We can walk into a boardroom and put a Kubernetes logo up, and people accept it as an established technology,” says van der Veer. “It becomes this centerpiece where other cloud native projects can tie in, so there’s a network effect that each project empowers each other. This is something that has a lot of value when we have to convince customers that our cloud fits their needs.”
-</div>
+<p>Nerdalize prides itself on being a Kubernetes-native cloud provider that charges its customers prices 40% below that of other cloud providers. "Every euro that we have to invest for licensing of software that's not open source comes from that 40%," says van der Veer. If they had used a non-open source orchestration platform instead of Kubernetes, "that would reduce this proposition that we have of 40% less cost to like 30%. Kubernetes directly allows us to have this business model and this strategic advantage." Nerdalize customers also benefit from time savings: One went from spending a day to set up VMs, network, and software, to spinning up a Kubernetes cluster in minutes. And for households using the heating devices, they save an average of 200 euro a year on their heating bill. The environmental impact? The annual reduction in CO2 emissions comes out to be 2 tons per Nerdalize household, which is equivalent to a car driving 8,000 km.</p>

-<div class="banner5" >
-  <div class="banner5text">
-“It shouldn’t be too big of a hassle and too large of a commitment. It should be fun and easy for end users. So we really love Kubernetes in that way.”<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- MAAIKE STOOPS, CUSTOMER EXPERIENCE QUEEN, NERDALIZE</span></div>
-</div>
+{{< case-studies/quote author="AD VAN DER VEER, PRODUCT ENGINEER, NERDALIZE" >}}
+"We can walk into a boardroom and put a Kubernetes logo up, and people accept it as an established technology. It becomes this centerpiece where other cloud native projects can tie in, so there's a network effect that each project empowers each other. This is something that has a lot of value when we have to talk to customers and convince them that our cloud fits their needs."
+{{< /case-studies/quote >}}

-<div class="fullcol">
+{{< case-studies/lead >}}
+Nerdalize is a cloud hosting provider that has no data centers. Instead, the four-year-old startup places its servers in homes across the Netherlands, inside heating devices it developed to turn the heat produced by the servers into heating and hot water for the residents.
+{{< /case-studies/lead >}}

-  In fact, Nerdalize is currently looking into implementing other CNCF projects, such as <a href="https://prometheus.io/">Prometheus</a> for monitoring and <a href="https://rook.io/">Rook</a>, “which should help us with some of the data problems that we want to solve for our customers,” says van der Veer.
-  <br><br>
-  In the coming year, Nerdalize will scale up the number of households running its hardware to 50, or the equivalent of a small scale data center. Geographic redundancy and greater server ability for customers are two main goals. Spreading the word about Kubernetes is also in the game plan. “We offer a free namespace on our sandbox, multi-tenant Kubernetes cluster for anyone to try,” says van der Veer. “What’s more cool than trying your first Kubernetes project on houses, to warm a shower?”
-  <br><br>
-  Ultimately, this ties into Nerdalize’s mission of supporting affordable and sustainable cloud hosting. “We want to be the disrupter of the cloud space, showing organizations that running in the cloud is easy and affordable,” says Stoops. “It shouldn’t be too big of a hassle and too large of a commitment. It should be fun and easy for end users. So we really love Kubernetes in that way.”
-</div>
+<p>"Households save on their gas bills, and cloud users have a much more sustainable cloud solution," says Maaike Stoops, Customer Experience Queen at Nerdalize. "And we don't have the overhead of building a data center, so our cloud is up to 40% more affordable."</p>

-</section>
+<p>That business model has been enabled by the company's adoption of containerization and Kubernetes. "When we just got started, Docker was just introduced," says Digital Product Engineer Ad van der Veer. "We began with a very basic bare metal setup, but once we developed the business, we saw that containerization technology was super useful to help our customers. As a cloud provider, we have internal services for hosting our backends and billing our customers, but we also need to offer our compute to our end users. Since we have these heating devices spread across the Netherlands, we need some way of tying that all together."</p>
+
+<p>After trying to develop its own scheduling system using another open source tool, Nerdalize found Kubernetes. "Kubernetes provided us with more functionality out of the gate," says van der Veer.</p>
+
+{{< case-studies/quote
+  image="/images/case-studies/nerdalize/banner3.jpg"
+  author="AD VAN DER VEER, PRODUCT ENGINEER, NERDALIZE"
+>}}
+"We always try to get a working version online first, like minimal viable products, and then move to stabilize that," says van der Veer. "And I think that these kinds of day-two problems are now immediately solved. The rapid prototyping we saw internally is a very valuable aspect of Kubernetes."
+{{< /case-studies/quote >}}
+
+<p>The team first experimented with a basic use case to run customers' workloads on Kubernetes. "Getting the data working was kind of difficult, and at the time the installation wasn't that simple," says van der Veer. "Then CNCF started, we saw the community grow, these problems got solved, and from there it became a very easy decision."</p>
+
+<p>The first Nerdalize product that was launched in 2017 was "100% containerized and Kubernetes native," says van der Veer. "On top of our heating devices throughout the Netherlands, we have a virtual machine layer, and on top of that we run Kubernetes clusters for our customers. As a small company, we have to provide a rock solid story in terms of the technology. Kubernetes allows us to offer a hybrid solution: 'You can run this on our cloud, but you can run it on other clouds as well. It runs in your internal hardware if you like.' And together with the Docker image standard and our multi-cloud dashboard, that gives them peace of mind."</p>
+
+<p>Not to mention the 40% cost savings. "Every euro that we have to invest for licensing of software that's not open source comes from that 40%," says van der Veer. If Nerdalize had used a non-open source orchestration platform instead of Kubernetes, "that would reduce our cost savings proposition to like 30%. Kubernetes directly allows us to have this business model and this strategic advantage."</p>
+
+{{< case-studies/quote
+  image="/images/case-studies/nerdalize/banner4.jpg"
+  author="MAAIKE STOOPS, CUSTOMER EXPERIENCE QUEEN, NERDALIZE"
+>}}
+"One of our customers used to spend up to a day setting up the virtual machines, network and software every time they wanted to run a project in the cloud. On our platform, with Docker and Kubernetes, customers can have their projects running in a couple of minutes."
+{{< /case-studies/quote >}}
+
+<p>Nerdalize now has customers, from individual engineers to data-intensive startups and established companies, all around the world. (For the time being, though, the heating devices are exclusive to the Netherlands.) One of the most common use cases is batch workloads used by data scientists and researchers, and the time savings for these end users is profound. "One of our customers used to spend up to a day setting up the virtual machines, network and software every time they wanted to run a project in the cloud," says Stoops. "On our platform, with Docker and Kubernetes, customers can have their projects running in a couple of minutes."</p>
+
+<p>As for households using the heating devices, they save an average of 200 euro a year on their heating bill. The environmental impact? The annual reduction in CO2 emissions comes out to 2 tons per Nerdalize household, which is equivalent to a car driving 8,000 km.</p>
+
+<p>For the Nerdalize team, feature development—such as the accessible command line interface called Nerd, which recently went live—has also been sped up by Kubernetes. "We always try to get a working version online first, like minimal viable products, and then move to stabilize that," says van der Veer. "And I think that these kinds of day-two problems are now immediately solved. The rapid prototyping we saw internally is a very valuable aspect of Kubernetes."</p>
+
+<p>Another unexpected benefit has been the growing influence and reputation of Kubernetes. "We can walk into a boardroom and put a Kubernetes logo up, and people accept it as an established technology," says van der Veer. "It becomes this centerpiece where other cloud native projects can tie in, so there's a network effect that each project empowers each other. This is something that has a lot of value when we have to convince customers that our cloud fits their needs."</p>
+
+{{< case-studies/quote author="MAAIKE STOOPS, CUSTOMER EXPERIENCE QUEEN, NERDALIZE" >}}
+"It shouldn't be too big of a hassle and too large of a commitment. It should be fun and easy for end users. So we really love Kubernetes in that way."
+{{< /case-studies/quote >}}
+
+<p>In fact, Nerdalize is currently looking into implementing other CNCF projects, such as <a href="https://prometheus.io/">Prometheus</a> for monitoring and <a href="https://rook.io/">Rook</a>, "which should help us with some of the data problems that we want to solve for our customers," says van der Veer.</p>
+
+<p>In the coming year, Nerdalize will scale up the number of households running its hardware to 50, or the equivalent of a small scale data center. Geographic redundancy and greater server ability for customers are two main goals. Spreading the word about Kubernetes is also in the game plan. "We offer a free namespace on our sandbox, multi-tenant Kubernetes cluster for anyone to try," says van der Veer. "What's more cool than trying your first Kubernetes project on houses, to warm a shower?"</p>
+
+<p>Ultimately, this ties into Nerdalize's mission of supporting affordable and sustainable cloud hosting. "We want to be the disrupter of the cloud space, showing organizations that running in the cloud is easy and affordable," says Stoops. "It shouldn't be too big of a hassle and too large of a commitment. It should be fun and easy for end users. So we really love Kubernetes in that way."</p>
@@ -3,87 +3,74 @@ title: NetEase Case Study
 linkTitle: NetEase
 case_study_styles: true
 cid: caseStudies
-css: /css/style_case_studies.css
 logo: netease_featured_logo.png
 featured: false
+
+new_case_study_styles: true
+heading_background: /images/case-studies/netease/banner1.jpg
+heading_title_logo: /images/netease_logo.png
+subheading: >
+  How NetEase Leverages Kubernetes to Support Internet Business Worldwide
+case_study_details:
+  - Company: NetEase
+  - Location: Hangzhou, China
+  - Industry: Internet technology
 ---

+<h2>Challenge</h2>

-<div class="banner1" style="background-image: url('/images/case-studies/netease/banner1.jpg')">
-  <h1> CASE STUDY:<img src="/images/netease_logo.png" class="header_logo" style="width:22%;margin-bottom:-1%"><br> <div class="subhead" style="margin-top:1%"> How NetEase Leverages Kubernetes to Support Internet Business Worldwide</div></h1>
+<p>Its gaming business is one of the largest in the world, but that's not all that <a href="https://netease-na.com/">NetEase</a> provides to Chinese consumers. The company also operates e-commerce, advertising, music streaming, online education, and email platforms; the last of which serves almost a billion users with free email services through sites like <a href="https://www.163.com/">163.com</a>. In 2015, the NetEase Cloud team providing the infrastructure for all of these systems realized that their R&D process was slowing down developers. "Our users needed to prepare all of the infrastructure by themselves," says Feng Changjian, Architect for NetEase Cloud and Container Service. "We were eager to provide the infrastructure and tools for our users automatically via serverless container service."</p>

-</div>
+<h2>Solution</h2>

-<div class="details" style="font-size:1em">
-    Company &nbsp;<b>NetEase</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location &nbsp;<b>Hangzhou, China</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Industry &nbsp;<b>Internet technology</b>
-</div>
+<p>After considering building its own orchestration solution, NetEase decided to base its private cloud platform on <a href="https://kubernetes.io/">Kubernetes</a>. The fact that the technology came out of Google gave the team confidence that it could keep up with NetEase's scale. "After our 2-to-3-month evaluation, we believed it could satisfy our needs," says Feng. The team started working with Kubernetes in 2015, before it was even 1.0. Today, the NetEase internal cloud platform—which also leverages the CNCF projects <a href="https://prometheus.io/">Prometheus</a>, <a href="https://www.envoyproxy.io/">Envoy</a>, <a href="https://goharbor.io/">Harbor</a>, <a href="https://grpc.io/">gRPC</a>, and <a href="https://helm.sh/">Helm</a>—runs 10,000 nodes in a production cluster and can support up to 30,000 nodes in a cluster. Based on its learnings from its internal platform, the company introduced a Kubernetes-based cloud and microservices-oriented PaaS product, <a href="https://landscape.cncf.io/selected=netease-qingzhou-microservice">NetEase Qingzhou Microservice</a>, to outside customers.</p>

-<hr>
-<section class="section1">
-<div class="cols">
-  <div class="col1" style="margin-left:-1.5% !important">
-    <h2>Challenge</h2>
-         Its gaming business is one of the largest in the world, but that’s not all that <a href="https://netease-na.com/">NetEase</a> provides to Chinese consumers. The company also operates e-commerce, advertising, music streaming, online education, and email platforms; the last of which serves almost a billion users with free email services through sites like <a href="https://www.163.com/">163.com</a>. In 2015, the NetEase Cloud team providing the infrastructure for all of these systems realized that their R&D process was slowing down developers. “Our users needed to prepare all of the infrastructure by themselves,” says Feng Changjian, Architect for NetEase Cloud and Container Service. “We were eager to provide the infrastructure and tools for our users automatically via serverless container service.”
-<br><br>
- <h2>Solution</h2>
-      After considering building its own orchestration solution, NetEase decided to base its private cloud platform on <a href="https://kubernetes.io/">Kubernetes</a>. The fact that the technology came out of Google gave the team confidence that it could keep up with NetEase’s scale. “After our 2-to-3-month evaluation, we believed it could satisfy our needs,” says Feng. The team started working with Kubernetes in 2015, before it was even 1.0. Today, the NetEase internal cloud platform—which also leverages the CNCF projects <a href="https://prometheus.io/">Prometheus</a>, <a href="https://www.envoyproxy.io/">Envoy</a>, <a href="https://goharbor.io/">Harbor</a>, <a href="https://grpc.io/">gRPC</a>, and <a href="https://helm.sh/">Helm</a>—runs 10,000 nodes in a production cluster and can support up to 30,000 nodes in a cluster. Based on its learnings from its internal platform, the company introduced a Kubernetes-based cloud and microservices-oriented PaaS product, <a href="https://landscape.cncf.io/selected=netease-qingzhou-microservice">NetEase Qingzhou Microservice</a>, to outside customers.
-
-
-<br><br>
 <h2>Impact</h2>
-      The NetEase team reports that Kubernetes has increased R&D efficiency by more than 100%. Deployment efficiency has improved by 280%. “In the past, if we wanted to do upgrades, we needed to work with other teams, even in other departments,” says Feng. “We needed special staff to prepare everything, so it took about half an hour. Now we can do it in only 5 minutes.” The new platform also allows for mixed deployments using GPU and CPU resources. “Before, if we put all the resources toward the GPU, we won’t have spare resources for the CPU. But now we have improvements thanks to the mixed deployments,” he says. Those improvements have also brought an increase in resource utilization.
-</div>
-</div>

-</section>
-<div class="banner2">
-  <div class="banner2text">
-    "The system can support 30,000 nodes in a single cluster. In production, we have gotten the data of 10,000 nodes in a single cluster. The whole internal system is using this system for development, test, and production."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>— Zeng Yuxing, Architect, NetEase</span>
-  </div>
-</div>
-<section class="section2">
-<div class="fullcol">
- <h2>Its gaming business is the <a href="https://newzoo.com/insights/rankings/top-25-companies-game-revenues/">fifth-largest</a> in the world, but that’s not all that <a href="https://netease-na.com/">NetEase</a> provides consumers.</h2>The company also operates e-commerce, advertising, music streaming, online education, and email platforms in China; the last of which serves almost a billion users with free email services through popular sites like <a href="https://www.163.com/">163.com</a> and <a href="https://www.126.com/">126.com</a>. With that kind of scale, the NetEase Cloud team providing the infrastructure for all of these systems realized in 2015 that their R&D process was making it hard for developers to keep up with demand. “Our users needed to prepare all of the infrastructure by themselves,” says Feng Changjian, Architect for NetEase Cloud and Container Service. “We were eager to provide the infrastructure and tools for our users automatically via serverless container service.”<br><br>
- After considering building its own orchestration solution, NetEase decided to base its private cloud platform on <a href="https://kubernetes.io/">Kubernetes</a>. The fact that the technology came out of Google gave the team confidence that it could keep up with NetEase’s scale. “After our 2-to-3-month evaluation, we believed it could satisfy our needs,” says Feng.
-</div>
-</section>
-<div class="banner3" style="background-image: url('/images/case-studies/netease/banner3.jpg')">
-  <div class="banner3text">
-    "We leveraged the programmability of Kubernetes so that we can build a platform to satisfy the needs of our internal customers for upgrades and deployment."
-<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Feng Changjian, Architect for NetEase Cloud and Container Service, NetEase</span>
-  </div>
-</div>
-<section class="section3">
-<div class="fullcol">
-  The team started adopting Kubernetes in 2015, before it was even 1.0, because it was relatively easy to use and enabled DevOps at the company. “We abandoned some of the concepts of Kubernetes; we only wanted to use the standardized framework,” says Feng. “We leveraged the programmability of Kubernetes so that we can build a platform to satisfy the needs of our internal customers for upgrades and deployment.”<br><br>
-  The team first focused on building the container platform to manage resources better, and then turned their attention to improving its support of microservices by adding internal systems such as monitoring. That has meant integrating the CNCF projects <a href="https://prometheus.io/">Prometheus</a>, <a href="https://www.envoyproxy.io/">Envoy</a>, <a href="https://goharbor.io/">Harbor</a>, <a href="https://grpc.io/">gRPC</a>, and <a href="https://helm.sh/">Helm</a>. “We are trying to provide a simplified and standardized process, so our users and customers can leverage our best practices,” says Feng.<br><br>
-  And the team is continuing to make improvements. For example, the e-commerce part of the business needs to leverage mixed deployments, which in the past required using two separate platforms: the infrastructure-as-a-service platform and the Kubernetes platform. More recently, NetEase has created a cross-platform application that enables using both with one-command deployment.
-</div>
-</section>
-<div class="banner4" style="background-image: url('/images/case-studies/netease/banner4.jpg')">
-  <div class="banner4text">
-    "As long as a company has a mature team and enough developers, I think Kubernetes is a very good technology that can help them."
-<span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br><br>- Li Lanqing, Kubernetes Developer, NetEase</span>
-  </div>
-  </div>
-</div>
+<p>The NetEase team reports that Kubernetes has increased R&D efficiency by more than 100%. Deployment efficiency has improved by 280%. "In the past, if we wanted to do upgrades, we needed to work with other teams, even in other departments," says Feng. "We needed special staff to prepare everything, so it took about half an hour. Now we can do it in only 5 minutes." The new platform also allows for mixed deployments using GPU and CPU resources. "Before, if we put all the resources toward the GPU, we won't have spare resources for the CPU. But now we have improvements thanks to the mixed deployments," he says. Those improvements have also brought an increase in resource utilization.</p>

-<section class="section5" style="padding:0px !important">
-<div class="fullcol">
-  Today, the NetEase internal cloud platform “can support 30,000 nodes in a single cluster,” says Architect Zeng Yuxing. “In production, we have gotten the data of 10,000 nodes in a single cluster. The whole internal system is using this system for development, test, and production.” <br><br>
-  The NetEase team reports that Kubernetes has increased R&D efficiency by more than 100%. Deployment efficiency has improved by 280%. “In the past, if we wanted to do upgrades, we needed to work with other teams, even in other departments,” says Feng. “We needed special staff to prepare everything, so it took about half an hour. Now we can do it in only 5 minutes.” The new platform also allows for mixed deployments using GPU and CPU resources. “Before, if we put all the resources toward the GPU, we won’t have spare resources for the CPU. But now we have improvements thanks to the mixed deployments.” Those improvements have also brought an increase in resource utilization.
+{{< case-studies/quote author="Zeng Yuxing, Architect, NetEase" >}}
+"The system can support 30,000 nodes in a single cluster. In production, we have gotten the data of 10,000 nodes in a single cluster. The whole internal system is using this system for development, test, and production."
+{{< /case-studies/quote >}}

-</div>
+{{< case-studies/lead >}}
+Its gaming business is the <a href="https://newzoo.com/insights/rankings/top-25-companies-game-revenues/">fifth-largest</a> in the world, but that's not all that <a href="https://netease-na.com/">NetEase</a> provides consumers.
+{{< /case-studies/lead >}}

-<div class="banner5">
-  <div class="banner5text">
-  "By engaging with this community, we can gain some experience from it and we can also benefit from it. We can see what are the concerns and the challenges faced by the community, so we can get involved."<span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br><br>- Li Lanqing, Kubernetes Developer, NetEase</span>
+<p>The company also operates e-commerce, advertising, music streaming, online education, and email platforms in China; the last of which serves almost a billion users with free email services through popular sites like <a href="https://www.163.com/">163.com</a> and <a href="https://www.126.com/">126.com</a>. With that kind of scale, the NetEase Cloud team providing the infrastructure for all of these systems realized in 2015 that their R&D process was making it hard for developers to keep up with demand. "Our users needed to prepare all of the infrastructure by themselves," says Feng Changjian, Architect for NetEase Cloud and Container Service. "We were eager to provide the infrastructure and tools for our users automatically via serverless container service."</p>

-  </div>
-</div>
-<div class="fullcol">
-  Based on the results and learnings from using its internal platform, the company introduced a Kubernetes-based cloud and microservices-oriented PaaS product, <a href="https://landscape.cncf.io/selected=netease-qingzhou-microservice">NetEase Qingzhou Microservice</a>, to outside customers. “The idea is that we can find the problems encountered by our game and e-commerce and cloud music providers, so we can integrate their experiences and provide a platform to satisfy the needs of our users,” says Zeng. <br><br>
-  With or without the use of the NetEase product, the team encourages other companies to try Kubernetes. “As long as a company has a mature team and enough developers, I think Kubernetes is a very good technology that can help them,” says Kubernetes developer Li Lanqing.<br><br>
-  As an end user as well as a vendor, NetEase has become more involved in the community, learning from other companies and sharing what they’ve done. The team has been contributing to the Harbor and Envoy projects, providing feedback as the technologies are being tested at NetEase scale. “We are a team focusing on addressing the challenges of microservices architecture,” says Feng. “By engaging with this community, we can gain some experience from it and we can also benefit from it. We can see what are the concerns and the challenges faced by the community, so we can get involved.”
-</div>
-</section>
+<p>After considering building its own orchestration solution, NetEase decided to base its private cloud platform on <a href="https://kubernetes.io/">Kubernetes</a>. The fact that the technology came out of Google gave the team confidence that it could keep up with NetEase's scale. "After our 2-to-3-month evaluation, we believed it could satisfy our needs," says Feng.</p>
+
+{{< case-studies/quote
+  image="/images/case-studies/netease/banner3.jpg"
+  author="Feng Changjian, Architect for NetEase Cloud and Container Service, NetEase"
+>}}
+"We leveraged the programmability of Kubernetes so that we can build a platform to satisfy the needs of our internal customers for upgrades and deployment."
+{{< /case-studies/quote >}}
+
+<p>The team started adopting Kubernetes in 2015, before it was even 1.0, because it was relatively easy to use and enabled DevOps at the company. "We abandoned some of the concepts of Kubernetes; we only wanted to use the standardized framework," says Feng. "We leveraged the programmability of Kubernetes so that we can build a platform to satisfy the needs of our internal customers for upgrades and deployment."</p>
+
+<p>The team first focused on building the container platform to manage resources better, and then turned their attention to improving its support of microservices by adding internal systems such as monitoring. That has meant integrating the CNCF projects <a href="https://prometheus.io/">Prometheus</a>, <a href="https://www.envoyproxy.io/">Envoy</a>, <a href="https://goharbor.io/">Harbor</a>, <a href="https://grpc.io/">gRPC</a>, and <a href="https://helm.sh/">Helm</a>. "We are trying to provide a simplified and standardized process, so our users and customers can leverage our best practices," says Feng.</p>
+
+<p>And the team is continuing to make improvements. For example, the e-commerce part of the business needs to leverage mixed deployments, which in the past required using two separate platforms: the infrastructure-as-a-service platform and the Kubernetes platform. More recently, NetEase has created a cross-platform application that enables using both with one-command deployment.</p>
+
+{{< case-studies/quote
+  image="/images/case-studies/netease/banner4.jpg"
+  author="Li Lanqing, Kubernetes Developer, NetEase"
+>}}
+"As long as a company has a mature team and enough developers, I think Kubernetes is a very good technology that can help them."
+{{< /case-studies/quote >}}
+
+<p>Today, the NetEase internal cloud platform "can support 30,000 nodes in a single cluster," says Architect Zeng Yuxing. "In production, we have gotten the data of 10,000 nodes in a single cluster. The whole internal system is using this system for development, test, and production."</p>
+
+<p>The NetEase team reports that Kubernetes has increased R&D efficiency by more than 100%. Deployment efficiency has improved by 280%. "In the past, if we wanted to do upgrades, we needed to work with other teams, even in other departments," says Feng. "We needed special staff to prepare everything, so it took about half an hour. Now we can do it in only 5 minutes." The new platform also allows for mixed deployments using GPU and CPU resources. "Before, if we put all the resources toward the GPU, we won't have spare resources for the CPU. But now we have improvements thanks to the mixed deployments." Those improvements have also brought an increase in resource utilization.</p>
+
+{{< case-studies/quote author="Li Lanqing, Kubernetes Developer, NetEase">}}
+"By engaging with this community, we can gain some experience from it and we can also benefit from it. We can see what are the concerns and the challenges faced by the community, so we can get involved."
+{{< /case-studies/quote >}}
+
+<p>Based on the results and learnings from using its internal platform, the company introduced a Kubernetes-based cloud and microservices-oriented PaaS product, <a href="https://landscape.cncf.io/selected=netease-qingzhou-microservice">NetEase Qingzhou Microservice</a>, to outside customers. "The idea is that we can find the problems encountered by our game and e-commerce and cloud music providers, so we can integrate their experiences and provide a platform to satisfy the needs of our users," says Zeng.</p>
+
+<p>With or without the use of the NetEase product, the team encourages other companies to try Kubernetes. "As long as a company has a mature team and enough developers, I think Kubernetes is a very good technology that can help them," says Kubernetes developer Li Lanqing.</p>
+
+<p>As an end user as well as a vendor, NetEase has become more involved in the community, learning from other companies and sharing what they've done. The team has been contributing to the Harbor and Envoy projects, providing feedback as the technologies are being tested at NetEase scale. "We are a team focusing on addressing the challenges of microservices architecture," says Feng. "By engaging with this community, we can gain some experience from it and we can also benefit from it. We can see what are the concerns and the challenges faced by the community, so we can get involved."</p>
@@ -2,107 +2,72 @@
 title: New York Times Case Study
 case_study_styles: true
 cid: caseStudies
-css: /css/style_case_studies.css
+
+new_case_study_styles: true
+heading_background: /images/case-studies/newyorktimes/banner1.jpg
+heading_title_logo: /images/newyorktimes_logo.png
+subheading: >
+  The New York Times: From Print to the Web to Cloud Native
+case_study_details:
+  - Company: New York Times
+  - Location: New York, N.Y.
+  - Industry: News Media
 ---

-<div class="banner1" style="padding-left:8% !important;background-image: url('/images/case-studies/newyorktimes/banner1.jpg')">
-  <h1> CASE STUDY:<img src="/images/newyorktimes_logo.png" class="header_logo"><br> <div class="subhead">The New York Times: From Print to the Web to Cloud Native
+<h2>Challenge</h2>

-</div></h1>
+<p>When the company decided a few years ago to move out of its data centers, its first deployments on the public cloud were smaller, less critical applications managed on virtual machines. "We started building more and more tools, and at some point we realized that we were doing a disservice by treating Amazon as another data center," says Deep Kapadia, Executive Director, Engineering at The New York Times. Kapadia was tapped to lead a Delivery Engineering Team that would "design for the abstractions that cloud providers offer us."</p>

-</div>
+<h2>Solution</h2>

-<div class="details" style="padding-left:6% !important">
-    Company &nbsp;<b>New York Times</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location &nbsp;<b>New York, N.Y.
-</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Industry &nbsp;<b>News Media</b>
-</div>
+<p>The team decided to use <a href="https://cloud.google.com/">Google Cloud Platform</a> and its Kubernetes-as-a-service offering, <a href="https://cloud.google.com/kubernetes-engine/">GKE</a>.</p>

-<hr />
-<section class="section1">
-<div class="cols">
-  <div class="col1">
-    <h2>Challenge</h2>
-    When the company decided a few years ago to move out of its data centers, its first deployments on the public cloud were smaller, less critical applications managed on virtual machines. "We started building more and more tools, and at some point we realized that we were doing a disservice by treating Amazon as another data center," says Deep Kapadia, Executive Director, Engineering at The New York Times. Kapadia was tapped to lead a Delivery Engineering Team that would "design for the abstractions that cloud providers offer us."
+<h2>Impact</h2>

-    <br><br>
-    <h2>Solution</h2>
-    The team decided to use <a href="https://cloud.google.com/">Google Cloud Platform</a> and its Kubernetes-as-a-service offering, <a href="https://cloud.google.com/kubernetes-engine/">GKE</a>.
-    <br />
-    </div>
+<p>Speed of delivery increased. Some of the legacy VM-based deployments took 45 minutes; with Kubernetes, that time was "just a few seconds to a couple of minutes," says Engineering Manager Brian Balser. Adds Li: "Teams that used to deploy on weekly schedules or had to coordinate schedules with the infrastructure team now deploy their updates independently, and can do it daily when necessary." Adopting Cloud Native Computing Foundation technologies allows for a more unified approach to deployment across the engineering staff, and portability for the company.</p>

-<div class="col2">
+{{< case-studies/quote author="Deep Kapadia, Executive Director, Engineering at The New York Times" >}}
+{{< youtube DqS_IPw-c6o youtube-quote-sm >}}
+{{< youtube Tm4VfJtOHt8 youtube-quote-sm >}}
+"I think once you get over the initial hump, things get a lot easier and actually a lot faster."
+{{< /case-studies/quote >}}

-  <h2>Impact</h2>
-  Speed of delivery increased. Some of the legacy VM-based deployments took 45 minutes; with Kubernetes, that time was "just a few seconds to a couple of minutes," says Engineering Manager Brian Balser. Adds Li: "Teams that used to deploy on weekly schedules or had to coordinate schedules with the infrastructure team now deploy their updates independently, and can do it daily when necessary." Adopting Cloud Native Computing Foundation technologies allows for a more unified approach to deployment across the engineering staff, and portability for the company.
-  <br />
-</div>
+<p>Founded in 1851 and known as the newspaper of record, The New York Times is a digital pioneer: Its first website launched in 1996, before Google even existed. After the company decided a few years ago to move out of its private data centers—including one located in the pricy real estate of Manhattan. It recently took another step into the future by going cloud native.</p>

-</div>
-</section>
-<div class="banner2" style="padding-bottom:0% !important;padding-left:10% !important;">
-    <div class="banner2text" style="width:100% !important;padding-left:5% !important;padding-right:5% !important">
-    <div class="video" style="float:left;width:90%;">
-    <iframe style="padding:1%:" width="380" height="215" src="https://www.youtube.com/embed/DqS_IPw-c6o" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>
-      <iframe style="padding:1%:" width="380" height="215" src="https://www.youtube.com/embed/Tm4VfJtOHt8" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>
-    </div>
-  </div>
-  </div>
-  <br><br>
-  <div class="banner2" style="padding-top:10% !important;padding-left:5% !important;">
-    <div class="banner2text" style="padding-top:5% !important">
-<br>
-      "I think once you get over the initial hump, things get a lot easier and actually a lot faster." — Deep Kapadia, Executive Director, Engineering at The&nbsp;New&nbsp;York&nbsp;Times</div>
-  </div>
-</div>
-<section class="section2">
-<div class="fullcol">
-  Founded in 1851 and known as the newspaper of record, The New York Times is a digital pioneer: Its first website launched in 1996, before Google even existed. After the company decided a few years ago to move out of its private data centers—including one located in the pricy real estate of Manhattan. It recently took another step into the future by going cloud native.<br /><br />
-  At first, the infrastructure team "managed the virtual machines in the Amazon cloud, and they deployed more critical applications in our data centers and the less critical ones on <a href="https://aws.amazon.com/">AWS</a> as an experiment," says Deep Kapadia, Executive Director, Engineering at The New York Times. "We started building more and more tools, and at some point we realized that we were doing a disservice by treating Amazon as another data center."<br /><br />
-  To get the most out of the cloud, Kapadia was tapped to lead a new Delivery Engineering Team that would "design for the abstractions that cloud providers offer us." In mid-2016, they began looking at the <a href="https://cloud.google.com/">Google Cloud Platform</a> and its Kubernetes-as-a-service offering, <a href="https://cloud.google.com/kubernetes-engine/">GKE</a>.<br /><br />
-  At the time, says team member Tony Li, a Site Reliability Engineer, "We had some internal tooling that attempted to do what Kubernetes does for containers, but for VMs. We asked why are we building and maintaining these tools ourselves?"<br /><br />
-  In early 2017, the first production application—the nytimes.com mobile homepage—began running on Kubernetes, serving just 1% of the traffic. Today, almost 100% of the nytimes.com site’s end-user facing applications run on GCP, with the majority on Kubernetes.
+<p>At first, the infrastructure team "managed the virtual machines in the Amazon cloud, and they deployed more critical applications in our data centers and the less critical ones on <a href="https://aws.amazon.com/">AWS</a> as an experiment," says Deep Kapadia, Executive Director, Engineering at The New York Times. "We started building more and more tools, and at some point we realized that we were doing a disservice by treating Amazon as another data center."</p>

-</div>
-</section>
-<div class="banner3" style="background-image: url('/images/case-studies/newyorktimes/banner3.jpg')">
-  <div class="banner3text">
-  "We had some internal tooling that attempted to do what Kubernetes does for containers, but for VMs. We asked why are we building and maintaining these tools ourselves?"
-  </div>
-</div>
-<section class="section3">
-<div class="fullcol">
+<p>To get the most out of the cloud, Kapadia was tapped to lead a new Delivery Engineering Team that would "design for the abstractions that cloud providers offer us." In mid-2016, they began looking at the <a href="https://cloud.google.com/">Google Cloud Platform</a> and its Kubernetes-as-a-service offering, <a href="https://cloud.google.com/kubernetes-engine/">GKE</a>.</p>

-  The team found that the speed of delivery was immediately impacted. "Deploying Docker images versus spinning up VMs was quite a lot faster," says Engineering Manager Brian Balser. Some of the legacy VM-based deployments took 45 minutes; with Kubernetes, that time was "just a few seconds to a couple of minutes."<br /><br />
-  The plan is to get as much as possible, not just the website, running on Kubernetes, and beyond that, moving toward serverless deployments. For instance, The New York Times crossword app was built on <a href="https://cloud.google.com/appengine/">Google App Engine</a>, which has been the main platform for the company’s experimentation with serverless. "The hardest part was getting the engineers over the hurdle of how little they had to do," Chief Technology Officer Nick Rockwell recently told The CTO Advisor. "Our experience has been very, very good. We have invested a lot of work into deploying apps on container services, and I’m really excited about experimenting with deploying those on App Engine Flex and <a href="https://aws.amazon.com/fargate/">AWS Fargate</a> and seeing how that feels, because that’s a great migration path."<br /><br />
-  There are some exceptions to the move to cloud native, of course. "We have the print publishing business as well," says Kapadia. "A lot of that is definitely not going down the cloud-native path because they’re using vendor software and even special machinery that prints the physical paper. But even those teams are looking at things like App Engine and Kubernetes if they can."<br /><br />
-  Kapadia acknowledges that there was a steep learning curve for some engineers, but "I think once you get over the initial hump, things get a lot easier and actually a lot faster."
+<p>At the time, says team member Tony Li, a Site Reliability Engineer, "We had some internal tooling that attempted to do what Kubernetes does for containers, but for VMs. We asked why are we building and maintaining these tools ourselves?"</p>

-</div>
-</section>
-<div class="banner4" style="background-image: url('/images/case-studies/newyorktimes/banner4.jpg')">
-  <div class="banner4text">
-    "Right now, every team is running a small Kubernetes cluster, but it would be nice if we could all live in a larger ecosystem," says Kapadia. "Then we can harness the power of things like service mesh proxies that can actually do a lot of instrumentation between microservices, or service-to-service orchestration. Those are the new things that we want to experiment with as we go forward."
+<p>In early 2017, the first production application—the nytimes.com mobile homepage—began running on Kubernetes, serving just 1% of the traffic. Today, almost 100% of the nytimes.com site's end-user facing applications run on GCP, with the majority on Kubernetes.</p>

-  </div>
-</div>
-<section class="section5" style="padding:0px !important">
-<div class="fullcol">
-  At The New York Times, they did. As teams started sharing their own best practices with each other, "We’re no longer the bottleneck for figuring out certain things," Kapadia says. "Most of the infrastructure and systems were managed by a centralized function. We’ve sort of blown that up, partly because Google and Amazon have tools that allow us to do that. We provide teams with complete ownership of their Google Cloud Platform projects, and give them a set of sensible defaults or standards. We let them know, ‘If this works for you as is, great! If not, come talk to us and we’ll figure out how to make it work for you.’"<br /><br />
-  As a result, "It’s really allowed teams to move at a much more rapid pace than they were able to in the past," says Kapadia. Adds Li: "The use of GKE means each team can get their own compute cluster, reducing the number of individual instances they have to care about since developers can treat the cluster as a whole. Because the ticket-based workflow was removed from requesting resources and connections, developers can just call an API to get what they want. Teams that used to deploy on weekly schedules or had to coordinate schedules with the infrastructure team now deploy their updates independently, and can do it daily when necessary."<br /><br />
-  Another benefit to adopting Kubernetes: allowing for a more unified approach to deployment across the engineering staff. "Before, many teams were building their own tools for deployment," says Balser. With Kubernetes—as well as the other CNCF projects The New York Times uses, including Fluentd to collect logs for all of its AWS servers, gRPC for its <a href="https://open.nytimes.com/publishing-with-apache-kafka-at-the-new-york-times-7f0e3b7d2077">Publishing Pipeline</a>, Prometheus, and Envoy—"we can benefit from the advances that each of these technologies make, instead of trying to catch up."
+{{< case-studies/quote image="/images/case-studies/newyorktimes/banner3.jpg" >}}
+"We had some internal tooling that attempted to do what Kubernetes does for containers, but for VMs. We asked why are we building and maintaining these tools ourselves?"
+{{< /case-studies/quote >}}

-</div>
+<p>The team found that the speed of delivery was immediately impacted. "Deploying Docker images versus spinning up VMs was quite a lot faster," says Engineering Manager Brian Balser. Some of the legacy VM-based deployments took 45 minutes; with Kubernetes, that time was "just a few seconds to a couple of minutes."</p>

-<div class="banner5">
-  <div class="banner5text">
+<p>The plan is to get as much as possible, not just the website, running on Kubernetes, and beyond that, moving toward serverless deployments. For instance, The New York Times crossword app was built on <a href="https://cloud.google.com/appengine/">Google App Engine</a>, which has been the main platform for the company's experimentation with serverless. "The hardest part was getting the engineers over the hurdle of how little they had to do," Chief Technology Officer Nick Rockwell recently told The CTO Advisor. "Our experience has been very, very good. We have invested a lot of work into deploying apps on container services, and I'm really excited about experimenting with deploying those on App Engine Flex and <a href="https://aws.amazon.com/fargate/">AWS Fargate</a> and seeing how that feels, because that's a great migration path."</p>

-Li calls the Cloud Native Computing Foundation’s projects "a northern star that we can all look at and follow."
-  </div>
-</div>
+<p>There are some exceptions to the move to cloud native, of course. "We have the print publishing business as well," says Kapadia. "A lot of that is definitely not going down the cloud-native path because they're using vendor software and even special machinery that prints the physical paper. But even those teams are looking at things like App Engine and Kubernetes if they can."</p>

-<div class="fullcol">
-  These open-source technologies have given the company more portability. "CNCF has enabled us to follow an industry standard," says Kapadia. "It allows us to think about whether we want to move away from our current service providers. Most of our applications are connected to Fluentd. If we wish to switch our logging provider from provider A to provider B we can do that. We’re running Kubernetes in GCP today, but if we want to run it in Amazon or Azure, we could potentially look into that as well."<br /><br />
-  Li calls the Cloud Native Computing Foundation’s projects "a northern star that we can all look at and follow." Led by that star, the team is looking ahead to a year of onboarding the remaining half of the 40 or so product engineering teams to extract even more value out of the technology. "Right now, every team is running a small Kubernetes cluster, but it would be nice if we could all live in a larger ecosystem," says Kapadia. "Then we can harness the power of things like service mesh proxies that can actually do a lot of instrumentation between microservices, or service-to-service orchestration. Those are the new things that we want to experiment with as we go forward."
+<p>Kapadia acknowledges that there was a steep learning curve for some engineers, but "I think once you get over the initial hump, things get a lot easier and actually a lot faster."</p>

-</div>
-</section>
+{{< case-studies/quote image="/images/case-studies/newyorktimes/banner4.jpg" >}}
+"Right now, every team is running a small Kubernetes cluster, but it would be nice if we could all live in a larger ecosystem," says Kapadia. "Then we can harness the power of things like service mesh proxies that can actually do a lot of instrumentation between microservices, or service-to-service orchestration. Those are the new things that we want to experiment with as we go forward."
+{{< /case-studies/quote >}}
+
+<p>At The New York Times, they did. As teams started sharing their own best practices with each other, "We're no longer the bottleneck for figuring out certain things," Kapadia says. "Most of the infrastructure and systems were managed by a centralized function. We've sort of blown that up, partly because Google and Amazon have tools that allow us to do that. We provide teams with complete ownership of their Google Cloud Platform projects, and give them a set of sensible defaults or standards. We let them know, 'If this works for you as is, great! If not, come talk to us and we'll figure out how to make it work for you.'"</p>
+
+<p>As a result, "It's really allowed teams to move at a much more rapid pace than they were able to in the past," says Kapadia. Adds Li: "The use of GKE means each team can get their own compute cluster, reducing the number of individual instances they have to care about since developers can treat the cluster as a whole. Because the ticket-based workflow was removed from requesting resources and connections, developers can just call an API to get what they want. Teams that used to deploy on weekly schedules or had to coordinate schedules with the infrastructure team now deploy their updates independently, and can do it daily when necessary."</p>
+
+<p>Another benefit to adopting Kubernetes: allowing for a more unified approach to deployment across the engineering staff. "Before, many teams were building their own tools for deployment," says Balser. With Kubernetes—as well as the other CNCF projects The New York Times uses, including Fluentd to collect logs for all of its AWS servers, gRPC for its <a href="https://open.nytimes.com/publishing-with-apache-kafka-at-the-new-york-times-7f0e3b7d2077">Publishing Pipeline</a>, Prometheus, and Envoy—"we can benefit from the advances that each of these technologies make, instead of trying to catch up."</p>
+
+{{< case-studies/quote >}}
+Li calls the Cloud Native Computing Foundation's projects "a northern star that we can all look at and follow."
+{{< /case-studies/quote >}}
+
+<p>These open-source technologies have given the company more portability. "CNCF has enabled us to follow an industry standard," says Kapadia. "It allows us to think about whether we want to move away from our current service providers. Most of our applications are connected to Fluentd. If we wish to switch our logging provider from provider A to provider B we can do that. We're running Kubernetes in GCP today, but if we want to run it in Amazon or Azure, we could potentially look into that as well."</p>
+
+<p>Li calls the Cloud Native Computing Foundation's projects "a northern star that we can all look at and follow." Led by that star, the team is looking ahead to a year of onboarding the remaining half of the 40 or so product engineering teams to extract even more value out of the technology. "Right now, every team is running a small Kubernetes cluster, but it would be nice if we could all live in a larger ecosystem," says Kapadia. "Then we can harness the power of things like service mesh proxies that can actually do a lot of instrumentation between microservices, or service-to-service orchestration. Those are the new things that we want to experiment with as we go forward."</p>
@@ -3,91 +3,75 @@ title: Nokia Case Study
 linkTitle: Nokia
 case_study_styles: true
 cid: caseStudies
-css: /css/style_case_studies.css
 logo: nokia_featured_logo.png
+
+new_case_study_styles: true
+heading_background: /images/case-studies/nokia/banner1.jpg
+heading_title_logo: /images/nokia_logo.png
+subheading: >
+  Nokia: Enabling 5G and DevOps at a Telecom Company with Kubernetes
+case_study_details:
+  - Company: Nokia
+  - Location: Espoo, Finland
+  - Industry: Telecommunications
 ---

+<h2>Challenge</h2>

-<div class="banner1" style="background-image: url('/images/case-studies/nokia/banner1.jpg')">
-  <h1> CASE STUDY:<img src="/images/nokia_logo.png" class="header_logo" style="width:20%;margin-bottom:-2.2%"><br> <div class="subhead" style="margin-top:1%">Nokia: Enabling 5G and DevOps at a Telecom Company with Kubernetes
+<p><a href="https://www.nokia.com/en_int">Nokia</a>'s core business is building telecom networks end-to-end; its main products are related to the infrastructure, such as antennas, switching equipment, and routing equipment. "As telecom vendors, we have to deliver our software to several telecom operators and put the software into their infrastructure, and each of the operators have a bit different infrastructure," says Gergely Csatari, Senior Open Source Engineer. "There are operators who are running on bare metal. There are operators who are running on virtual machines. There are operators who are running on <a href="https://cloud.vmware.com/">VMware Cloud</a> and <a href="https://www.openstack.org/">OpenStack</a> Cloud. We want to run the same product on all of these different infrastructures without changing the product itself."</p>

-</div></h1>
+<h2>Solution</h2>

-</div>
+<p>The company decided that moving to cloud native technologies would allow teams to have infrastructure-agnostic behavior in their products. Teams at Nokia began experimenting with Kubernetes in pre-1.0 versions. "The simplicity of the label-based scheduling of Kubernetes was a sign that showed us this architecture will scale, will be stable, and will be good for our purposes," says Csatari. The first Kubernetes-based product, the <a href="https://networks.nokia.com/products/telecom-application-server">Nokia Telephony Application Server</a>, went live in early 2018. "Now, all the products are doing some kind of re-architecture work, and they're moving to Kubernetes."</p>

-<div class="details">
-    Company &nbsp;<b>Nokia</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location &nbsp;<b>Espoo, Finland
-</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Industry &nbsp;<b>Telecommunications</b>
-</div>
-
-<hr>
-<section class="section1">
-<div class="cols">
-  <div class="col1" style="width:100%"">
-    <h2>Challenge</h2>
-    <a href="https://www.nokia.com/en_int">Nokia</a>’s core business is building telecom networks end-to-end; its main products are related to the infrastructure, such as antennas, switching equipment, and routing equipment. "As telecom vendors, we have to deliver our software to several telecom operators and put the software into their infrastructure, and each of the operators have a bit different infrastructure," says Gergely Csatari, Senior Open Source Engineer. "There are operators who are running on bare metal. There are operators who are running on virtual machines. There are operators who are running on <a href="https://cloud.vmware.com/">VMware Cloud</a> and <a href="https://www.openstack.org/">OpenStack</a> Cloud. We want to run the same product on all of these different infrastructures without changing the product itself."
-
-    <h2>Solution</h2>
-    The company decided that moving to cloud native technologies would allow teams to have infrastructure-agnostic behavior in their products. Teams at Nokia began experimenting with Kubernetes in pre-1.0 versions. "The simplicity of the label-based scheduling of Kubernetes was a sign that showed us this architecture will scale, will be stable, and will be good for our purposes," says Csatari. The first Kubernetes-based product, the <a href="https://networks.nokia.com/products/telecom-application-server">Nokia Telephony Application Server</a>, went live in early 2018. "Now, all the products are doing some kind of re-architecture work, and they’re moving to Kubernetes."
-<br>
 <h2>Impact</h2>
-    Kubernetes has enabled Nokia’s foray into 5G. "When you develop something that is part of the operator’s infrastructure, you have to develop it for the future, and Kubernetes and containers are the forward-looking technologies," says Csatari. The teams using Kubernetes are already seeing clear benefits. "By separating the infrastructure and the application layer, we have less dependencies in the system, which means that it’s easier to implement features in the application layer," says Csatari. And because teams can test the exact same binary artifact independently of the target execution environment, "we find more errors in early phases of the testing, and we do not need to run the same tests on different target environments, like VMware, OpenStack, or bare metal," he adds. As a result, "we save several hundred hours in every release."

-</div>
+<p>Kubernetes has enabled Nokia's foray into 5G. "When you develop something that is part of the operator's infrastructure, you have to develop it for the future, and Kubernetes and containers are the forward-looking technologies," says Csatari. The teams using Kubernetes are already seeing clear benefits. "By separating the infrastructure and the application layer, we have less dependencies in the system, which means that it's easier to implement features in the application layer," says Csatari. And because teams can test the exact same binary artifact independently of the target execution environment, "we find more errors in early phases of the testing, and we do not need to run the same tests on different target environments, like VMware, OpenStack, or bare metal," he adds. As a result, "we save several hundred hours in every release."</p>

-</div>
-</section>
-<div class="banner2">
-  <div class="banner2text">
-    "When people are picking up their phones and making a call on Nokia networks, they are creating containers in the background with Kubernetes."
-<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Gergely Csatari, Senior Open Source Engineer, Nokia</span>
-  </div>
-</div>
-<section class="section2">
-<div class="fullcol">
-  <h2>Nokia was the first name in mobile phones when they were becoming ubiquitous in the late 1990s and early 2000s. But by 2014, the company had sold off its mobile device division and was focusing its core business not on the handhelds used for calls, but on the networks.</h2>
-  Today, Nokia is building telecom networks end-to-end—from antennas to switching and routing equipment—serving operators in more than 120 countries. "As telecom vendors, we have to deliver our software to several telecom operators and put the software into their infrastructure, and each of the operators have a bit different infrastructure," says Gergely Csatari, Senior Open Source Engineer at Nokia. "There are operators who are running on bare metal. There are operators who are running on virtual machines. There are operators who are running on VMware Cloud and OpenStack Cloud. We want to run the same product on all of these different infrastructures without changing the product itself."<br><br>
-  Looking for a way to allow its teams to build products with infrastructure-agnostic behavior, the company decided to embrace containerization, Kubernetes, and other cloud native technologies, a move that is being made across the telecom industry. Since early 2018, "when people are picking up their phones and making a call on Nokia networks, they are creating containers in the background with Kubernetes," says Csatari. "Now, all the products are doing some kind of re-architecture work, and they’re moving to Kubernetes."
+{{< case-studies/quote author="Gergely Csatari, Senior Open Source Engineer, Nokia" >}}
+"When people are picking up their phones and making a call on Nokia networks, they are creating containers in the background with Kubernetes."
+{{< /case-studies/quote >}}

-</div>
-</section>
-<div class="banner3" style="background-image: url('/images/case-studies/nokia/banner3.jpg')">
-  <div class="banner3text">
-    "Having the community and CNCF around Kubernetes is not only important for having a connection to other companies who are using Kubernetes and a forum where you can ask or discuss features of Kubernetes. But as a company who would like to contribute to Kubernetes, it was very important to have a CLA (Contributors License Agreement) which is connected to the CNCF and not to a particular company. That was a critical step for us to start contributing to Kubernetes and Helm."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Gergely Csatari, Senior Open Source Engineer, Nokia</span>
+{{< case-studies/lead >}}
+Nokia was the first name in mobile phones when they were becoming ubiquitous in the late 1990s and early 2000s. But by 2014, the company had sold off its mobile device division and was focusing its core business not on the handhelds used for calls, but on the networks.
+{{< /case-studies/lead >}}

-  </div>
-</div>
-<section class="section3">
-<div class="fullcol">
-  Nokia’s cloud native journey began about two years ago, when Csatari’s team was building the company’s Telephony Application Server (TAS). "We wanted to have a service execution engine in the product, which was a totally separate function from all other parts," he says. "There, we had the possibility to think about new architectures and new tools that we could use. We created this particular product based on Kubernetes, and we liked the work, so we started to talk about cloud native and containers and all of these things. We did a very extensive research of different container orchestration tools. We knew that we have some, let’s say, strange or different requirements because of the special environment that our software is running on."<br><br>
-  For one thing, Nokia’s software serves millions of people, and is required to have the carrier-grade "five nines" availability: to be up 99.999% of the time. "If you turn it to minutes, this means we’re allowed to have only 10 minutes of downtime in a whole year," says Csatari. "Downtime here means that you are not able to serve the person to full capacity, which means that we cannot fail. This includes software upgrades, everything, because when you call 911, you’re using our software, and you expect that it will work."<br><br>
-  That meant that they needed to be able to set affinity and anti-affinity rules in their orchestration tools. "You cannot put all of the functions to the same physical host because physical hosts are failing," Csatari explains. "If you fail with one physical host, then you lose all of the core processing processes. Then there are no calls going through. So we have to divide them among the different physical hosts. At that time, only Kubernetes was able to provide these features. The simplicity of the label-based scheduling of Kubernetes was a sign that showed us this architecture will scale, will be stable, and will be good for our purposes."
+<p>Today, Nokia is building telecom networks end-to-end—from antennas to switching and routing equipment—serving operators in more than 120 countries. "As telecom vendors, we have to deliver our software to several telecom operators and put the software into their infrastructure, and each of the operators have a bit different infrastructure," says Gergely Csatari, Senior Open Source Engineer at Nokia. "There are operators who are running on bare metal. There are operators who are running on virtual machines. There are operators who are running on VMware Cloud and OpenStack Cloud. We want to run the same product on all of these different infrastructures without changing the product itself."</p>

-</div>
-</section>
-<div class="banner4" style="background-image: url('/images/case-studies/nokia/banner4.jpg')" style="width:100%">
-  <div class="banner4text">
-  "Kubernetes opened the window to all of these open source projects instead of implementing everything in house. Our engineers can focus more on the application level, which is actually the thing what we are selling, and not on the infrastructure level. For us, the most important thing about Kubernetes is it allows us to focus on value creation of our business." <br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Gergely Csatari, Senior Open Source Engineer, Nokia</span>
-  </div>
-</div>
+<p>Looking for a way to allow its teams to build products with infrastructure-agnostic behavior, the company decided to embrace containerization, Kubernetes, and other cloud native technologies, a move that is being made across the telecom industry. Since early 2018, "when people are picking up their phones and making a call on Nokia networks, they are creating containers in the background with Kubernetes," says Csatari. "Now, all the products are doing some kind of re-architecture work, and they're moving to Kubernetes."</p>

-<section class="section5" style="padding:0px !important">
-<div class="fullcol">
-  The TAS went live in early 2018, and now Kubernetes is also enabling Nokia’s foray into 5G. The company is introducing microservices architecture and Kubernetes while adding 5G features to existing products. And all new 5G product development will be on top of Kubernetes. "When you develop something that is part of the operator’s infrastructure, you have to develop it for the future, and Kubernetes and containers are the forward-looking technologies," says Csatari. <br><br>
-  There have been real time savings thanks to Kubernetes. "By separating the infrastructure and the application layer, we have less dependencies in the system, which means that it’s easier to implement features in the application layer," says Csatari. Because teams can test the exact same binary artifact independently of the target execution environment, "we find more errors in early phases of the testing, and we do not need to run the same tests on different target environments, like VMware, OpenStack or bare metal," he adds. As a result, "we save several hundred hours in every release."<br><br>
-  Moving from Nokia’s legacy cluster management system, which had been built in-house more than thirty years ago, to a Kubernetes platform also meant that "we started using Linux as a base operating system, so we just opened the window to all of these open source projects instead of implementing everything in house," says Csatari. (From CNCF’s ecosystem, the team is already using <a href="https://helm.sh/">Helm</a>, <a href="https://grpc.io/">gRPC</a>, <a href="https://github.com/containernetworking">CNI</a>, <a href="https://prometheus.io/">Prometheus</a>, and <a href="https://www.envoyproxy.io/">Envoy</a>, and plans to implement <a href="https://coredns.io/">CoreDNS</a>.) "Our engineers can focus more on the application level, which is actually the thing what we are selling, and not on the infrastructure level. For us, the most important thing about Kubernetes is it allows us to focus on value creation of our business."
+{{< case-studies/quote
+  image="/images/case-studies/nokia/banner3.jpg"
+  author="Gergely Csatari, Senior Open Source Engineer, Nokia"
+>}}
+"Having the community and CNCF around Kubernetes is not only important for having a connection to other companies who are using Kubernetes and a forum where you can ask or discuss features of Kubernetes. But as a company who would like to contribute to Kubernetes, it was very important to have a CLA (Contributors License Agreement) which is connected to the CNCF and not to a particular company. That was a critical step for us to start contributing to Kubernetes and Helm."
+{{< /case-studies/quote >}}

-</div>
+<p>Nokia's cloud native journey began about two years ago, when Csatari's team was building the company's Telephony Application Server (TAS). "We wanted to have a service execution engine in the product, which was a totally separate function from all other parts," he says. "There, we had the possibility to think about new architectures and new tools that we could use. We created this particular product based on Kubernetes, and we liked the work, so we started to talk about cloud native and containers and all of these things. We did a very extensive research of different container orchestration tools. We knew that we have some, let's say, strange or different requirements because of the special environment that our software is running on."</p>

-<div class="banner5" style="width:100%">
-  <div class="banner5text">
-"I had some discussions at KubeCon with people from the networking SIG and the resource management working group, to work together on our requirements, and that’s very exciting for me and my colleagues,"<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Gergely Csatari, Senior Open Source Engineer, Nokia</span></div>
-</div>
+<p>For one thing, Nokia's software serves millions of people, and is required to have the carrier-grade "five nines" availability: to be up 99.999% of the time. "If you turn it to minutes, this means we're allowed to have only 10 minutes of downtime in a whole year," says Csatari. "Downtime here means that you are not able to serve the person to full capacity, which means that we cannot fail. This includes software upgrades, everything, because when you call 911, you're using our software, and you expect that it will work."</p>

-<div class="fullcol">
-  The company has a long-term goal of moving the entire product portfolio into the Kubernetes platform. To that end, Nokia teams are working together with other companies to add the features needed to use Kubernetes with the real-time, nanosecond-sensitive applications close to the edge of the radio network. <br><br>
-  And the CNCF community is proving to be a great forum for that collaboration. "I had some discussions at KubeCon with people from the networking SIG and the resource management working group, to work together on our requirements, and that’s very exciting for me and my colleagues," says Csatari. "Previously, everybody had the same problem, but everybody just did it in his own, and now we are trying to solve the same problem together."<br><br>
-  Perhaps the biggest impact that Kubernetes is having on Nokia, Csatari believes, is that people are starting to think about how a telecom company can do DevOps. "We are building a DevOps pipeline, which reaches from the actual developer to the customers, and thinking about new ways how can we digitally deliver our software to our customers and get feedback from the customers right to the engineers," he says. "This is something that will fundamentally change how telecom companies are delivering software, and how quickly can we develop new features. This is because of the usage of containers and, of course, the usage of Kubernetes."
+<p>That meant that they needed to be able to set affinity and anti-affinity rules in their orchestration tools. "You cannot put all of the functions to the same physical host because physical hosts are failing," Csatari explains. "If you fail with one physical host, then you lose all of the core processing processes. Then there are no calls going through. So we have to divide them among the different physical hosts. At that time, only Kubernetes was able to provide these features. The simplicity of the label-based scheduling of Kubernetes was a sign that showed us this architecture will scale, will be stable, and will be good for our purposes."</p>

-</div>
-</section>
+{{< case-studies/quote 
+  image="/images/case-studies/nokia/banner4.jpg"
+  author="Gergely Csatari, Senior Open Source Engineer, Nokia"
+>}}
+"Kubernetes opened the window to all of these open source projects instead of implementing everything in house. Our engineers can focus more on the application level, which is actually the thing what we are selling, and not on the infrastructure level. For us, the most important thing about Kubernetes is it allows us to focus on value creation of our business."
+{{< /case-studies/quote >}}
+
+<p>The TAS went live in early 2018, and now Kubernetes is also enabling Nokia's foray into 5G. The company is introducing microservices architecture and Kubernetes while adding 5G features to existing products. And all new 5G product development will be on top of Kubernetes. "When you develop something that is part of the operator's infrastructure, you have to develop it for the future, and Kubernetes and containers are the forward-looking technologies," says Csatari.</p>
+
+<p>There have been real time savings thanks to Kubernetes. "By separating the infrastructure and the application layer, we have less dependencies in the system, which means that it's easier to implement features in the application layer," says Csatari. Because teams can test the exact same binary artifact independently of the target execution environment, "we find more errors in early phases of the testing, and we do not need to run the same tests on different target environments, like VMware, OpenStack or bare metal," he adds. As a result, "we save several hundred hours in every release."</p>
+
+<p>Moving from Nokia's legacy cluster management system, which had been built in-house more than thirty years ago, to a Kubernetes platform also meant that "we started using Linux as a base operating system, so we just opened the window to all of these open source projects instead of implementing everything in house," says Csatari. (From CNCF's ecosystem, the team is already using <a href="https://helm.sh/">Helm</a>, <a href="https://grpc.io/">gRPC</a>, <a href="https://github.com/containernetworking">CNI</a>, <a href="https://prometheus.io/">Prometheus</a>, and <a href="https://www.envoyproxy.io/">Envoy</a>, and plans to implement <a href="https://coredns.io/">CoreDNS</a>.) "Our engineers can focus more on the application level, which is actually the thing what we are selling, and not on the infrastructure level. For us, the most important thing about Kubernetes is it allows us to focus on value creation of our business."</p>
+
+{{< case-studies/quote author="Gergely Csatari, Senior Open Source Engineer, Nokia" >}}
+"I had some discussions at KubeCon with people from the networking SIG and the resource management working group, to work together on our requirements, and that's very exciting for me and my colleagues,"
+{{< /case-studies/quote >}}
+
+<p>The company has a long-term goal of moving the entire product portfolio into the Kubernetes platform. To that end, Nokia teams are working together with other companies to add the features needed to use Kubernetes with the real-time, nanosecond-sensitive applications close to the edge of the radio network.</p>
+
+<p>And the CNCF community is proving to be a great forum for that collaboration. "I had some discussions at KubeCon with people from the networking SIG and the resource management working group, to work together on our requirements, and that's very exciting for me and my colleagues," says Csatari. "Previously, everybody had the same problem, but everybody just did it in his own, and now we are trying to solve the same problem together."</p>
+
+<p>Perhaps the biggest impact that Kubernetes is having on Nokia, Csatari believes, is that people are starting to think about how a telecom company can do DevOps. "We are building a DevOps pipeline, which reaches from the actual developer to the customers, and thinking about new ways how can we digitally deliver our software to our customers and get feedback from the customers right to the engineers," he says. "This is something that will fundamentally change how telecom companies are delivering software, and how quickly can we develop new features. This is because of the usage of containers and, of course, the usage of Kubernetes."</p>
@@ -2,109 +2,74 @@
 title: Nordstrom Case Study
 case_study_styles: true
 cid: caseStudies
-css: /css/style_case_studies.css
+
+new_case_study_styles: true
+heading_background: /images/case-studies/nordstrom/banner1.jpg
+heading_title_logo: /images/nordstrom_logo.png
+subheading: >
+  Finding Millions in Potential Savings in a Tough Retail Climate
+case_study_details:
+  - Company: Nordstrom
+  - Location: Seattle, Washington
+  - Industry: Retail
 ---

-<div class="banner1" style="background-image: url('/images/case-studies/nordstrom/banner1.jpg')">
-  <h1> CASE STUDY:<img src="/images/nordstrom_logo.png" class="header_logo" style="margin-bottom:-1.5% !important;width:20% !important;"><br> <div class="subhead">Finding Millions in Potential Savings in a Tough Retail Climate
+<h2>Challenge</h2>

+<p>Nordstrom wanted to increase the efficiency and speed of its technology operations, which includes the Nordstrom.com e-commerce site. At the same time, Nordstrom Technology was looking for ways to tighten its technology operational costs.</p>

-</div></h1>
+<h2>Solution</h2>

-</div>
+<p>After embracing a DevOps transformation and launching a continuous integration/continuous deployment (CI/CD) project four years ago, the company reduced its deployment time from three months to 30 minutes. But they wanted to go even faster across environments, so they began their cloud native journey, adopting Docker containers orchestrated with <a href="http://kubernetes.io/">Kubernetes</a>.</p>

-<div class="details">
-    Company &nbsp;<b>Nordstrom</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location &nbsp;<b>Seattle, Washington</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Industry &nbsp;<b>Retail</b>
-</div>
+<h2>Impact</h2>

-<hr>
-<section class="section1">
-<div class="cols">
-  <div class="col1">
-    <h2>Challenge</h2>
-    Nordstrom wanted to increase the efficiency and speed of its technology operations, which includes the Nordstrom.com e-commerce site. At the same time, Nordstrom Technology was looking for ways to tighten its technology operational costs.
+<p>Nordstrom Technology developers using Kubernetes now deploy faster and can "just focus on writing applications," says Dhawal Patel, a senior engineer on the team building a Kubernetes enterprise platform for Nordstrom. Furthermore, the team has increased Ops efficiency, improving CPU utilization from 5x to 12x depending on the workload. "We run thousands of virtual machines (VMs), but aren't effectively using all those resources," says Patel. "With Kubernetes, without even trying to make our cluster efficient, we are currently at a 10x increase."</p>

-<br>
-    <h2>Solution</h2>
-    After embracing a DevOps transformation and launching a continuous integration/continuous deployment (CI/CD) project four years ago, the company reduced its deployment time from three months to 30 minutes. But they wanted to go even faster across environments, so they began their cloud native journey, adopting Docker containers orchestrated with <a href="http://kubernetes.io/">Kubernetes</a>.
+{{< case-studies/quote author="Dhawal Patel, senior engineer at Nordstrom" >}}
+"We are always looking for ways to optimize and provide more value through technology. With Kubernetes we are showcasing two types of efficiency that we can bring: Dev efficiency and Ops efficiency. It's a win-win."
+{{< /case-studies/quote >}}

-  <br>
+<p>When Dhawal Patel joined <a href="http://shop.nordstrom.com/">Nordstrom</a> five years ago as an application developer for the retailer's website, he realized there was an opportunity to help speed up development cycles.</p>

+<p>In those early DevOps days, Nordstrom Technology still followed a traditional model of silo teams and functions. "As a developer, I was spending more time fixing environments than writing code and adding value to business," Patel says. "I was passionate about that—so I was given the opportunity to help fix it."</p>

-      </div>
+<p>The company was eager to move faster, too, and in 2013 launched the first continuous integration/continuous deployment (CI/CD) project. That project was the first step in Nordstrom's cloud native journey.</p>

-<div class="col2">
+<p>Dev and Ops team members built a CI/CD pipeline, working with the company's servers on premise. The team chose <a href="https://www.chef.io/chef/">Chef</a>, and wrote cookbooks that automated virtual IP creation, servers, and load balancing. "After we completed the project, deployment went from three months to 30 minutes," says Patel. "We still had multiple environments—dev, test, staging, then production—so with each environment running the Chef cookbooks, it took 30 minutes. It was a huge achievement at that point."</p>

-  <h2>Impact</h2>
-  Nordstrom Technology developers using Kubernetes now deploy faster and can "just focus on writing applications," says Dhawal Patel, a senior engineer on the team building a Kubernetes enterprise platform for Nordstrom. Furthermore, the team has increased Ops efficiency, improving CPU utilization from 5x to 12x depending on the workload. "We run thousands of virtual machines (VMs), but aren’t effectively using all those resources," says Patel. "With Kubernetes, without even trying to make our cluster efficient, we are currently at a 10x increase."
+<p>But new environments still took too long to turn up, so the next step was working in the cloud. Today, Nordstrom Technology has built an enterprise platform that allows the company's 1,500 developers to deploy applications running as Docker containers in the cloud, orchestrated with Kubernetes.</p>

-</div>
-</div>
-</section>
-<div class="banner2">
-  <div class="banner2text">
-    "We are always looking for ways to optimize and provide more value through technology. With Kubernetes we are showcasing two types of efficiency that we can bring: Dev efficiency and Ops efficiency. It’s a win-win."
-<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>-— Dhawal Patel, senior engineer at Nordstrom</span>
-  </div>
-</div>
-<section class="section2">
-<div class="fullcol">
-  When Dhawal Patel joined <a href="http://shop.nordstrom.com/">Nordstrom</a> five years ago as an application developer for the retailer’s website, he realized there was an opportunity to help speed up development cycles.
-<br><br>
-    In those early DevOps days, Nordstrom Technology still followed a traditional model of silo teams and functions. "As a developer, I was spending more time fixing environments than writing code and adding value to business," Patel says. "I was passionate about that—so I was given the opportunity to help fix it."
-<br><br>
-    The company was eager to move faster, too, and in 2013 launched the first continuous integration/continuous deployment (CI/CD) project. That project was the first step in Nordstrom’s cloud native journey.
-<br><br>
-    Dev and Ops team members built a CI/CD pipeline, working with the company’s servers on premise. The team chose <a href="https://www.chef.io/chef/">Chef</a>, and wrote cookbooks that automated virtual IP creation, servers, and load balancing. "After we completed the project, deployment went from three months to 30 minutes," says Patel. "We still had multiple environments—dev, test, staging, then production—so with each environment running the Chef cookbooks, it took 30 minutes. It was a huge achievement at that point."
-    <br><br>But new environments still took too long to turn up, so the next step was working in the cloud. Today, Nordstrom Technology has built an enterprise platform that allows the company’s 1,500 developers to deploy applications running as Docker containers in the cloud, orchestrated with Kubernetes.
+{{< case-studies/quote image="/images/case-studies/nordstrom/banner3.jpg" >}}
+"We made a bet that Kubernetes was going to take off, informed by early indicators of community support and project velocity, so we rebuilt our system with Kubernetes at the core,"
+{{< /case-studies/quote >}}

-</div>
-</section>
-<div class="banner3" style="background-image: url('/images/case-studies/nordstrom/banner3.jpg')">
-  <div class="banner3text">
-    "We made a bet that Kubernetes was going to take off, informed by early indicators of community support and project velocity, so we rebuilt our system with Kubernetes at the core,"
-  </div>
-</div>
-<section class="section3">
-<div class="fullcol">
+<p>"The cloud provided faster access to resources, because it took weeks for us to get a virtual machine (VM) on premises," says Patel. "But now we can do the same thing in only five minutes."</p>

-"The cloud provided faster access to resources, because it took weeks for us to get a virtual machine (VM) on premises," says Patel. "But now we can do the same thing in only five minutes."
-<br><br>
-Nordstrom’s first foray into scheduling containers on a cluster was a homegrown system based on CoreOS fleet. They began doing a few proofs of concept projects with that system until Kubernetes 1.0 was released when they made the switch. "We made a bet that Kubernetes was going to take off, informed by early indicators of community support and project velocity, so we rebuilt our system with Kubernetes at the core," says Marius Grigoriu, Sr. Manager of the Kubernetes team at Nordstrom.
-While Kubernetes is often thought as a platform for microservices, the first application to launch on Kubernetes in a critical production role at Nordstrom was Jira. "It was not the ideal microservice we were hoping to get as our first application," Patel admits, "but the team that was working on it was really passionate about Docker and Kubernetes, and they wanted to try it out. They had their application running on premises, and wanted to move it to Kubernetes."
-<br><br>
-The benefits were immediate for the teams that came on board. "Teams running on our Kubernetes cluster loved the fact that they had fewer issues to worry about. They didn’t need to manage infrastructure or operating systems," says Grigoriu. "Early adopters loved the declarative nature of Kubernetes. They loved the reduced surface area they had to deal with."
+<p>Nordstrom's first foray into scheduling containers on a cluster was a homegrown system based on CoreOS fleet. They began doing a few proofs of concept projects with that system until Kubernetes 1.0 was released when they made the switch. "We made a bet that Kubernetes was going to take off, informed by early indicators of community support and project velocity, so we rebuilt our system with Kubernetes at the core," says Marius Grigoriu, Sr. Manager of the Kubernetes team at Nordstrom.</p>

-</div>
-</section>
-<div class="banner4" style="background-image: url('/images/case-studies/nordstrom/banner4.jpg')">
-  <div class="banner4text">
- "Teams running on our Kubernetes cluster loved the fact that they had fewer issues to worry about. They didn’t need to manage infrastructure or operating systems," says Grigoriu. "Early adopters loved the declarative nature of Kubernetes. They loved the reduced surface area they had to deal with."
-  </div>
-</div>
+<p>While Kubernetes is often thought as a platform for microservices, the first application to launch on Kubernetes in a critical production role at Nordstrom was Jira. "It was not the ideal microservice we were hoping to get as our first application," Patel admits, "but the team that was working on it was really passionate about Docker and Kubernetes, and they wanted to try it out. They had their application running on premises, and wanted to move it to Kubernetes."</p>

-<section class="section5">
-<div class="fullcol">
-  To support these early adopters, Patel’s team began growing the cluster and building production-grade services. "We integrated with <a href="https://prometheus.io/">Prometheus</a> for monitoring, with a <a href="https://grafana.com/">Grafana</a> front end; we used <a href="http://www.fluentd.org/">Fluentd</a> to push logs to <a href="https://www.elastic.co/">Elasticsearch</a>, so that gives us log aggregation," says Patel. The team also added dozens of open-source components, including CNCF projects and has made contributions to Kubernetes, Terraform, and kube2iam.
- <br><br>
-There are now more than 60 development teams running Kubernetes in Nordstrom Technology, and as success stories have popped up, more teams have gotten on board. "Our initial customer base, the ones who were willing to try this out, are now going and evangelizing to the next set of users," says Patel. "One early adopter had Docker containers and he was not sure how to run it in production. We sat with him and within 15 minutes we deployed it in production. He thought it was amazing, and more people in his org started coming in."
- <br><br>
-For Nordstrom Technology, going cloud-native has vastly improved development and operational efficiency. The developers using Kubernetes now deploy faster and can focus on building value in their applications.  One such team started with a 25-minute merge to deploy by launching virtual machines in the cloud. Switching to Kubernetes was a 5x speedup in their process, improving their merge to deploy time to 5 minutes.
-</div>
+<p>The benefits were immediate for the teams that came on board. "Teams running on our Kubernetes cluster loved the fact that they had fewer issues to worry about. They didn't need to manage infrastructure or operating systems," says Grigoriu. "Early adopters loved the declarative nature of Kubernetes. They loved the reduced surface area they had to deal with."</p>

-<div class="banner5">
-  <div class="banner5text">
-    "With Kubernetes, without even trying to make our cluster efficient, we are currently at 40 percent CPU utilization—a 10x increase. we are running 2600+ customer pods that would have been 2600+ VMs if they had gone directly to the cloud. We are running them on 40 VMs now, so that’s a huge reduction in operational overhead."
-  </div>
-</div>
+{{< case-studies/quote image="/images/case-studies/nordstrom/banner4.jpg">}}
+"Teams running on our Kubernetes cluster loved the fact that they had fewer issues to worry about. They didn't need to manage infrastructure or operating systems," says Grigoriu. "Early adopters loved the declarative nature of Kubernetes. They loved the reduced surface area they had to deal with."
+{{< /case-studies/quote >}}

-<div class="fullcol">
-  Speed is great, and easily demonstrated, but perhaps the bigger impact lies in the operational efficiency. "We run thousands of VMs on AWS, and their overall average CPU utilization is about four percent," says Patel. "With Kubernetes, without even trying to make our cluster efficient, we are currently at 40 percent CPU utilization—a 10x increase. We are running 2600+ customer pods that would have been 2600+ VMs if they had gone directly to the cloud. We are running them on 40 VMs now, so that’s a huge reduction in operational overhead."
-<br><br>
-  Nordstrom Technology is also exploring running Kubernetes on bare metal on premises. "If we can build an on-premises Kubernetes cluster," says Patel, "we could bring the power of cloud to provision resources fast on-premises. Then for the developer, their interface is Kubernetes; they might not even realize or care that their services are now deployed on premises because they’re only working with Kubernetes."
-  For that reason, Patel is eagerly following Kubernetes’ development of multi-cluster capabilities. "With cluster federation, we can have our on-premise as the primary cluster and the cloud as a secondary burstable cluster," he says. "So, when there is an anniversary sale or Black Friday sale, and we need more containers - we can go to the cloud."
-<br><br>
-  That kind of possibility—as well as the impact that Grigoriu and Patel’s team has already delivered using Kubernetes—is what led Nordstrom on its cloud native journey in the first place. "The way the retail environment is today, we are trying to build responsiveness and flexibility where we can," says Grigoriu.  "Kubernetes makes it easy to: bring efficiency to both the Dev and Ops side of the equation. It’s a win-win."
+<p>To support these early adopters, Patel's team began growing the cluster and building production-grade services. "We integrated with <a href="https://prometheus.io/">Prometheus</a> for monitoring, with a <a href="https://grafana.com/">Grafana</a> front end; we used <a href="http://www.fluentd.org/">Fluentd</a> to push logs to <a href="https://www.elastic.co/">Elasticsearch</a>, so that gives us log aggregation," says Patel. The team also added dozens of open-source components, including CNCF projects and has made contributions to Kubernetes, Terraform, and kube2iam.</p>

-</div>
-</section>
+<p>There are now more than 60 development teams running Kubernetes in Nordstrom Technology, and as success stories have popped up, more teams have gotten on board. "Our initial customer base, the ones who were willing to try this out, are now going and evangelizing to the next set of users," says Patel. "One early adopter had Docker containers and he was not sure how to run it in production. We sat with him and within 15 minutes we deployed it in production. He thought it was amazing, and more people in his org started coming in."</p>
+
+<p>For Nordstrom Technology, going cloud-native has vastly improved development and operational efficiency. The developers using Kubernetes now deploy faster and can focus on building value in their applications.  One such team started with a 25-minute merge to deploy by launching virtual machines in the cloud. Switching to Kubernetes was a 5x speedup in their process, improving their merge to deploy time to 5 minutes.</p>
+
+{{< case-studies/quote >}}
+"With Kubernetes, without even trying to make our cluster efficient, we are currently at 40 percent CPU utilization—a 10x increase. we are running 2600+ customer pods that would have been 2600+ VMs if they had gone directly to the cloud. We are running them on 40 VMs now, so that's a huge reduction in operational overhead."
+{{< /case-studies/quote >}}
+
+<p>Speed is great, and easily demonstrated, but perhaps the bigger impact lies in the operational efficiency. "We run thousands of VMs on AWS, and their overall average CPU utilization is about four percent," says Patel. "With Kubernetes, without even trying to make our cluster efficient, we are currently at 40 percent CPU utilization—a 10x increase. We are running 2600+ customer pods that would have been 2600+ VMs if they had gone directly to the cloud. We are running them on 40 VMs now, so that's a huge reduction in operational overhead."</p>
+
+<p>Nordstrom Technology is also exploring running Kubernetes on bare metal on premises. "If we can build an on-premises Kubernetes cluster," says Patel, "we could bring the power of cloud to provision resources fast on-premises. Then for the developer, their interface is Kubernetes; they might not even realize or care that their services are now deployed on premises because they're only working with Kubernetes."</p>
+
+<p>For that reason, Patel is eagerly following Kubernetes' development of multi-cluster capabilities. "With cluster federation, we can have our on-premise as the primary cluster and the cloud as a secondary burstable cluster," he says. "So, when there is an anniversary sale or Black Friday sale, and we need more containers - we can go to the cloud."</p>
+
+<p>That kind of possibility—as well as the impact that Grigoriu and Patel's team has already delivered using Kubernetes—is what led Nordstrom on its cloud native journey in the first place. "The way the retail environment is today, we are trying to build responsiveness and flexibility where we can," says Grigoriu.  "Kubernetes makes it easy to: bring efficiency to both the Dev and Ops side of the equation. It's a win-win."</p>
--- a/Show More
+++ b/Show More