[Do Not Merge] Release 1.12 (#10292 )

* Update docs for fields allowed at root of CRD schema (#9973) * add plugin docs and examples (#10053) * docs update to promote TaintNodesByCondition to beta (#9626) * HPA Specificity Improvements (#8757) Updated the HPA docs to reference the `autoscaling/v2beta2` API version, and added documentation about the new fields. * adjust docs for pod ready++ (#10049) * Remove --cadvisor-port - has been deprecated since v1.10 (#10023) Change-Id: Id2a685473a243aef492a98ff450759f39e362557 * Add Documentation for Snapshot Feature (#9948) * Add documentation for snapshot feature * Update volume-snapshots.md * Add dry-run to api-concepts (#10033) * kubeadm-init: Update the offline support section (#10062) The update includes the following things (in mind with Kubernetes 1.12): - Remove the 1.8 image versions - Add the 1.10 image versions that were missing until now - Include a comment for the missing arch suffixes in 1.12 Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com> * Say bye to `DynamicProvisioningScheduling` (#10157) The mentioned feature gate is now collapsed into `VolumeScheduling`. xref: kubernetes/kubernetes#67432 * Update ResourceQuota per PriorityClass state for 1.12 (#10229) * TokenRequest and TokenRequestProjection now beta (#10161) xref: kubernetes/kubernetes#67349 * Change feature state for kms provider to beta. (#10230) KMS Provider will be graduating to beta in v1.12, reflecting this change on the website. * coredns default (#10200) * Promote ShareProcessNamespace to beta in docs (#9996) * Add CoreDNS details to DNS Debug docs (#10201) * add coredns details * address nits, add query logging section * Update docs with topology aware dynamic provisioning (#9939) * Document topology aware volume binding feature * update for readability * Update storage-classes.md * comma splice * don't abbreviate * HPA Algorithm Information Improvements (#9780) * Update HPA docs with more algorithm details The HPA docs pointed to an out-of-date document for information on the algorithm details, which users were finding confusing. This sticks a section on the algorithm in the HPA docs instead, documenting both general behavior and corner cases. * Add glossary info, HPA docs on quantities People often ask about the quantity notation when working with the metrics APIs, so this adds a glossary entry on quantities (since they're used elsewhere in the system), and a short explantation in the HPA walkthough. * Information about HPA readiness and stabilization This adds information about the new changes to HPA readiness and stabilization from kubernetes/features#591, and other minor changes that landed in Kubernetes 1.12. * Update horizontal-pod-autoscale.md * Audit 1.12 doc (#9953) * audit 1.12 document * remove legacy audit feature https://github.com/kubernetes/kubernetes/pull/65862 * update feature gate doc * MountPropagation is now GA (#10090) * RuntimeClass documentation (#10102) * RuntimeClass documentation * Update runtime-class.md * Add documentation for Scheduler performance tuning (#10048) * Add documentation for Scheduler performance tuning * Update scheduler-perf-tuning.md * TTL controller for cleaning up finished resources (#10064) * TTL controller for cleaning up finished resources * Address comments * Update ttlafterfinished.md * Bump quota configuration api version (#10217) * Incremental update from master (#10278) * fix invalid href of cloud controller manager (#10240) * fix invalid yaml format (#10238) * update storage-limits doc with Azure disk part (#10224) update storage-limits doc with Azure disk part fix comments * Update kubelet-config-file.md (#10222) Update link to KubeletConfiguration struct. * fix a trivial misspelling (#10244) * Fix cassandra-statefulset.yaml indent level (#10243) * Mention minimum etcd versions (#10208) Source: https://groups.google.com/d/msg/kubernetes-dev/jMPA4JzKiY4/HIx2ugvLBAAJ * fix 404 error (#10250) * Small verb tweak (#10190) Present participle, ftw. * Add AnchorJS logic for header links (#10155) * Add AnchorJS JavaScript * Remove existing inpage_heading logic * Remove underline from anchor tags * Use single icon and add touch visibility * Use paragraph link icon for AnchorJS * Update Sass to use code formatting in docsContent headers * Update header size coverage to H3-H6 * fix broken link in kubefed.md (#10254) * Update the version numbers for the X-Remote-Extra- and Impersonate-Extra- key fixes (#9827) The fix was cherry picked into 1.11.3, 1.10.7, and 1.9.11: https://github.com/kubernetes/kubernetes/pull/67162 https://github.com/kubernetes/kubernetes/pull/67163 https://github.com/kubernetes/kubernetes/pull/67164 * fix typo (#10168) * fix typo * addressing comments. * Update setup-ha-etcd-with-kubeadm.md * fix typos (#10252) * fix description of contribute guide (#10253) * describe truncate feature about advanced audit (#10236) * describe truncate feature about advanced audit * Update audit.md * docs update to promote ScheduleDaemonSetPods to beta (#9923) * Dynamic volume limit updates for 1.12 (#10211) * add a placeholder commit * Update docs for csi volume limits * Update storage-limits.md * Add "MayRunAs" value among other GroupStrategies (#9888) * Add CoreDNS details to the customize DNS doc (#10228) * Add CoreDNS details to the customize DNS doc Rewrite the document to include more details about CoreDNS, since it's now the default from v1.12 * Address comments * Improve doc wording * Fix link * Update dns-custom-nameservers.md * Update dns-custom-nameservers.md * Fix secrets docs in 1.12 branch (#10056) * Fix secrets docs * Update secret.md * Revert CoreDNS Docs (#10319) * Revert "Add CoreDNS details to DNS Debug docs (#10201)" This reverts commit 462817a674. * Revert "Add CoreDNS details to the customize DNS doc (#10228)" This reverts commit e7319eeb8c. * Revert "coredns default (#10200)" This reverts commit 698e93b441. * Add CRI installation instructions page Added cri-installation page with CRI installation instructions Referenced it from kubeadm-init and install-kubeadm pages. * kubeadm: update API types documentation for 1.12 (#10283) v1alpha2 -> v1alpha3 MasterConfiguration -> [new-api-types] * TokenRequest feature documentation (#10295) * AdvancedAuditing is now GA (#10156) xref: kubernetes/kubernetes#65862 `AdvancedAuditing` feature is GA in 1.12. This PR adjusts the related docs. * update runtime-class.md (#10332) * update runtime-class.md * Update runtime-class.md * Document cross-authorizer permissions for creating RBAC roles (#10015) * Document cross-authorizer permissions for creating RBAC roles * Update rbac.md * kubeadm: update authored content for 1.12 (reference docs and cluster creation) (#10348) * kubeadm: update authored content in reference docs for 1.12 * kubeadm: add time frame in create-cluster-kubeadm for 1.12 * add AllowedProcMountTypes and ProcMountType to docs (#9911) Signed-off-by: Jess Frazelle <acidburn@microsoft.com> * kubeadm: add new command line reference (#10306) Add: - placeholder files - include place holder files - include "renew" sub command - add missing tabs for "alpha phase kubelet" * Documenting SCTP support in Kubernetes (#10279) * Documenting SCTP support in Kubernetes Service, Endpoint, NetworkPolicy and Pod * Updates based on comments on the PR * kubectl expose update with SCTP support * Updated according to comments in the PR * Revert "kubectl expose update with SCTP support" This reverts commit 0d5a1e6720a012390cf100c83e16b4a8c0782356. * TLS Bootstrap and Server Cert Rotation feature documentation (#10232) * TokenRequest feature documentation * line wrapping to make review not insane * update content for GA without major refactor * Update kubelet-tls-bootstrapping.md * Add clarifications for volume snapshots (#10296) * Update kubadm ha installation for 1.12 (#10264) * Update kubadm ha installation for 1.12 Signed-off-by: Chuck Ha <ha.chuck@gmail.com> * update stable version Signed-off-by: Chuck Ha <ha.chuck@gmail.com> * Update stacked control plane for v1.12 (#2) * use v1alpha3 Signed-off-by: Chuck Ha <ha.chuck@gmail.com> * more v1alpha3 (#4) * updates Signed-off-by: Chuck Ha <ha.chuck@gmail.com> * Document how to run in-tree cloud providers with kubeadm (#10357) Change-Id: Iab6b996a830503d74a6eb0c507c5f8ca7a39235b * kubeadm reference doc for release 1.12 (#10359) * Revert "Revert "Add CoreDNS details to DNS Debug docs (#10201)"" This reverts commit bb30f4d1fcd6fba2fe6190778ead99f8010033b7. * Revert "Revert "Add CoreDNS details to the customize DNS doc (#10228)"" This reverts commit bc23d45c09d7b83cac130fe22a0bd91e72435862. * Revert "Revert "coredns default (#10200)"" This reverts commit 7f4350d6ab7fc554ee53126d3875e845d2e43d1f. * add missing instruction for ha guide (#10374) Signed-off-by: Chuck Ha <ha.chuck@gmail.com> * kubeadm - Ha upgrade updates (#10340) * Update HA upgrade docs * Adds external etcd HA upgrade guide Signed-off-by: Chuck Ha <ha.chuck@gmail.com> * copyedit * more edits * add runasgroup in psp (#10076) * update KubeletPluginsWatcher feature gate (#10205) * generated 1.12 docs * Building Multi-arch images with Manifests (#10379) In 1.12, a variety of images used in a typical kubernetes installation have started to using manifests to better support environments with arm or ppc64le architectures. For example all images used with kubeadm by default have manifests, another would be all the tests in the conformance test suite. Here we capture the best practices for everyone to start using manifests in their own workflows. Change-Id: I5ba4c5fe55ffc9486a8251760f3352be4f2e1494 * Upgrade docs for v1.12 (#10344) * generated assets and docs * remove 1.7 * update 1.12 * update plugin documentation under docs>tasks>extend-kubectl (#10259) * update plugin documentation under docs>tasks>extend-kubectl * Update kubectl-plugins.md
Update config.toml (#10396 )
2018-09-27 16:41:39 -07:00 · 2018-09-27 12:18:56 -07:00 · 2018-09-27 07:08:57 -07:00 · 2018-09-25 15:10:18 -07:00 · 2018-09-25 09:04:23 -07:00 · 2018-09-25 06:34:23 -07:00
1038 changed files with 72189 additions and 126517 deletions
@@ -0,0 +1 @@
+**
@@ -1,4 +1,5 @@
 <!-- Thanks for filing an issue! Before submitting, please fill in the following information. -->
+<!-- See https://kubernetes.io/docs/contribute/start/ for guidance on writing an actionable issue description. -->

 <!--Required Information-->

@@ -1,8 +1,9 @@
 >^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-> For 1.11 Features: set Milestone to 1.11 and Base Branch to release-1.11
->^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-> NOTE: After opening the PR, please *un-check and re-check* the ["Allow edits from maintainers"](https://help.github.com/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork/) box so that maintainers can work on your patch and speed up the review process. This is a temporary workaround to address a known issue with GitHub.> 
->
 > Please delete this note before submitting the pull request.
+> For 1.12 Features: set Milestone to 1.12 and Base Branch to release-1.12
+> Help editing and submitting pull requests:  https://deploy-preview-9510--kubernetes-io-master-staging.netlify.com/docs/contribute/start/#submit-a-pull-request.
+> Help choosing which branch to use, see
+> https://kubernetes.io/docs/contribute/start#choose-which-git-branch-to-use.
+>^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+>

-![Allow edits from maintainers checkbox](https://help.github.com/assets/images/help/pull_requests/allow-maintainers-to-make-edits-sidebar-checkbox.png)
@@ -29,3 +29,6 @@ nohup.out

 # Hugo output
 public/
+
+# User-specific editorconfig files
+.editorconfig
@@ -1,22 +1,21 @@
 language: go
 go:
-  - 1.9.3
+  - 1.10.2

 # Don't want default ./... here:
 install:
 - export PATH=$GOPATH/bin:$PATH
 - mkdir -p $HOME/gopath/src/k8s.io
 - mv $TRAVIS_BUILD_DIR $HOME/gopath/src/k8s.io/website && cd $HOME/gopath/src/k8s.io/website
-# Fetch dependencies for us to run the tests in test/examples_test.go
- go get -t -v k8s.io/website/test
-# Make sure we are testing against the correct branch
- pushd $GOPATH/src/k8s.io/kubernetes && git checkout release-1.10 && popd

-# Simplified deduplication of dependencies.
+# Make sure we are testing against the correct branch
+- pushd $GOPATH/src/k8s.io && git clone https://github.com/kubernetes/kubernetes && popd
+- pushd $GOPATH/src/k8s.io/kubernetes && git checkout release-1.11 && popd
 - cp -L -R $GOPATH/src/k8s.io/kubernetes/vendor/ $GOPATH/src/
 - rm -r $GOPATH/src/k8s.io/kubernetes/vendor/

+# Fetch additional dependencies to run the tests in examples/examples_test.go
+- go get -t -v k8s.io/website/content/en/examples
+
 script:
-# TODO(bep)
- go test -v k8s.io/website/test #fixed by https://github.com/kubernetes/website/pull/8388
-#- ./verify-docs-format.sh
+- go test -v k8s.io/website/content/en/examples
@@ -12,13 +12,15 @@ RUN apk add --no-cache \
    curl \
    git \
    openssh-client \
-    rsync
+    rsync \
+    build-base \
+    libc6-compat

 ARG HUGO_VERSION

 RUN mkdir -p /usr/local/src && \
    cd /usr/local/src && \
-    curl -L https://github.com/gohugoio/hugo/releases/download/v${HUGO_VERSION}/hugo_${HUGO_VERSION}_linux-64bit.tar.gz | tar -xz && \
+    curl -L https://github.com/gohugoio/hugo/releases/download/v${HUGO_VERSION}/hugo_extended_${HUGO_VERSION}_linux-64bit.tar.gz | tar -xz && \
    mv hugo /usr/local/bin/hugo && \
    curl -L https://bin.equinox.io/c/dhgbqpS8Bvy/minify-stable-linux-amd64.tgz | tar -xz && \
    mv minify /usr/local/bin && \
@@ -1,5 +1,5 @@
 DOCKER       = docker
-HUGO_VERSION = 0.40.3
+HUGO_VERSION = 0.47.1
 DOCKER_IMAGE = kubernetes-hugo
 DOCKER_RUN   = $(DOCKER) run --rm --interactive --tty --volume $(PWD):/src

@@ -8,20 +8,24 @@ DOCKER_RUN   = $(DOCKER) run --rm --interactive --tty --volume $(PWD):/src
 help: ## Show this help.
 	@awk 'BEGIN {FS = ":.*?## "} /^[a-zA-Z_-]+:.*?## / {sub("\\\\n",sprintf("\n%22c"," "), $$2);printf "\033[36m%-20s\033[0m %s\n", $$1, $$2}' $(MAKEFILE_LIST)

-all: build ## Build site with production settings and put deliverables in _site.
+all: build ## Build site with production settings and put deliverables in ./public

-sass: # Rebuild the SASS source into CSS
-	node-sass --output-style compact ./src/sass/styles.sass ./static/css/styles.css
-	node-sass --output-style compact ./src/sass/case_study_styles.sass ./static/css/case_study_styles.css
-
-build: ## Build site with production settings and put deliverables in _site.
+build: ## Build site with production settings and put deliverables in ./public
 	hugo

-build-preview: ## Build site with drafts and future posts enabled.
+build-preview: ## Build site with drafts and future posts enabled
 	hugo -D -F

+check-headers-file:
+	scripts/check-headers-file.sh
+
+production-build: build check-headers-file ## Build the production site and ensure that noindex headers aren't added
+
+non-production-build: ## Build the non-production site, which adds noindex headers to prevent indexing
+	hugo --enableGitInfo
+
 serve: ## Boot the development server.
-	hugo server
+	hugo server --ignoreCache --disableFastRender

 docker-image:
 	$(DOCKER) build . --tag $(DOCKER_IMAGE) --build-arg HUGO_VERSION=$(HUGO_VERSION)
@@ -29,5 +33,5 @@ docker-image:
 docker-build:
 	$(DOCKER_RUN) $(DOCKER_IMAGE) hugo

-stage:
+docker-serve:
 	$(DOCKER_RUN) -p 1313:1313 $(DOCKER_IMAGE) hugo server --watch --bind 0.0.0.0
@@ -1,19 +1,23 @@
 # Reviewers can /lgtm /approve but not sufficient for auto-merge without an
 # approver
 reviewers:
- zhangxiaoyu-zidif
- xiangpengzhao
+- Rajakavitha1
 - stewart-yu
- Rajakavitha1 
+- xiangpengzhao
+- zhangxiaoyu-zidif
+
 # Approvers have all the ability of reviewers but their /approve makes
 # auto-merge happen if a /lgtm exists, or vice versa, or they can do both
 # No need for approvers to also be listed as reviewers
 approvers:
- heckj
 - bradamant3
 - bradtopol
- steveperry-53
- zacharysarah
 - chenopis
+- kbarnard10
 - mistyhacks
+- ryanmcginnis
+- steveperry-53
 - tengqm
+- tfogo
+- zacharysarah
+- zparnold
@@ -41,9 +41,11 @@ aliases:
    - chrislovecnm
    - mfburnett
  sig-azure: #Microsoft Azure
-    - slack
-    - colemickens
-    - jdumars
+    - andyzhangx
+    - feiskyer
+    - justaugustus
+    - karataliu
+    - khenidak
  sig-big-data: #GH: sig-big-data-pr-reviews
    - foxish
  sig-cli: #Team: CLI; GH: sig-cli-pr-reviews; e.g. kubectl
@@ -61,11 +63,25 @@ aliases:
    - smarterclayton
    - soltysh
    - sttts
-  sig-cluster-lifecycle: #GH: sig-cluster-lifecycle-pr-reviews
+  sig-cluster-lifecycle-kubeadm-approvers: # Approving changes to kubeadm documentation
    - timothysc
+    - lukemarsden
    - luxas
    - roberthbailey
-    - fabriziopandini 
+    - fabriziopandini
+  sig-cluster-lifecycle-kubeadm-reviewers: # Reviewing kubeadm documentation
+    - timothysc
+    - lukemarsden
+    - luxas
+    - roberthbailey
+    - fabriziopandini
+    - kad
+    - xiangpengzhao
+    - stealthybox
+    - liztio
+    - chuckha
+    - detiber
+    - dixudx
  sig-cluster-ops:
    - zehicle
    - jdumars
@@ -83,10 +99,19 @@ aliases:
    - spxtr
  sig-docs: #Team: documentation; GH: sig-docs-pr-reviews
    - bradamant3
-    - steveperry-53
-    - zacharysarah
    - bradtopol
-    - heckj
+    - chenopis
+    - kbarnard10
+    - mistyhacks
+    - rajakavitha1
+    - ryanmcginnis
+    - steveperry-53
+    - stewart-yu
+    - tengqm
+    - xiangpengzhao
+    - zacharysarah
+    - zhangxiaoyu-zidif
+    - zparnold
  sig-federation: #Team: Federation; e.g. Federated Clusters
    - csbell
  sig-gcp: #Google Cloud Platform; GH: sig-gcp-pr-reviews
@@ -1,53 +1,50 @@
-## Instructions for Contributing to the Kubernetes Documentation
+# The Kubernetes documentation

-Welcome! We are very pleased you want to contribute to the Kubernetes documentation.
+Welcome! This repository houses all of the assets required to build the Kubernetes website and documentation. We're very pleased that you want to contribute!

-You can click the **Fork** button in the upper-right area of the screen to create a copy of this repository in your GitHub account called a *fork*. Make any changes you want in your fork, and when you are ready to send those changes to us, go to your fork and create a new pull request to let us know about it.
+## Contributing to the docs
+
+You can click the **Fork** button in the upper-right area of the screen to create a copy of this repository in your GitHub account. This copy is called a *fork*. Make any changes you want in your fork, and when you are ready to send those changes to us, go to your fork and create a new pull request to let us know about it.

 Once your pull request is created, a Kubernetes reviewer will take responsibility for providing clear, actionable feedback.  As the owner of the pull request, **it is your responsibility to modify your pull request to address the feedback that has been provided to you by the Kubernetes reviewer.**  Also note that you may end up having more than one Kubernetes reviewer provide you feedback or you may end up getting feedback from a Kubernetes reviewer that is different than the one originally assigned to provide you feedback. Furthermore, in some cases, one of your reviewers might ask for a technical review from a [Kubernetes tech reviewer](https://github.com/kubernetes/website/wiki/Tech-reviewers) when needed.  Reviewers will do their best to provide feedback in a timely fashion but response time can vary based on circumstances.

 For more information about contributing to the Kubernetes documentation, see:

-* [Contributing to the Kubernetes Documentation](http://kubernetes.io/editdocs/)
-* [Creating a Documentation Pull Request](http://kubernetes.io/docs/home/contribute/create-pull-request/)
-* [Writing a New Topic](http://kubernetes.io/docs/home/contribute/write-new-topic/)
-* [Review Issues](http://kubernetes.io/docs/home/contribute/review-issues/)
-* [Staging Your Documentation Changes](http://kubernetes.io/docs/home/contribute/stage-documentation-changes/)
-* [Using Page Templates](http://kubernetes.io/docs/home/contribute/page-templates/)
-* [Documentation Style Guide](http://kubernetes.io/docs/home/contribute/style-guide/)
+* [Start contributing](https://kubernetes.io/docs/contribute/start/)
+* [Staging Your Documentation Changes](http://kubernetes.io/docs/contribute/intermediate#view-your-changes-locally)
+* [Using Page Templates](http://kubernetes.io/docs/contribute/style/page-templates/)
+* [Documentation Style Guide](http://kubernetes.io/docs/contribute/style/style-guide/)
+
+## Running the site locally using Hugo
+
+The Kubernetes documentation is built using the [Hugo](https://gohugo.io) static site generator. See the [official Hugo documentation](https://gohugo.io/getting-started/installing/) for Hugo installation instructions.
+
+> Building and running the site requires the Hugo version specified by the `HUGO_VERSION` environment variable in the [`netlify.toml`](netlify.toml#L9) file.
+
+To run the site locally when you have Hugo installed:
+
+```bash
+make serve
+```
+
+This will start the local Hugo server on port 1313. Open up your browser to http://localhost:1313 to view the site. As you make changes to the source files, Hugo updates immediately and forces a browser refresh.

 ## Building the site using Docker

-If you'd like, you can build the Kubernetes docs using Docker. To get started, build the image locally:
+You can build the Kubernetes docs using [Docker](https://docker.com). To get started, make sure that you have Docker running and build the image locally:

 ```bash
-$ make docker-image
-
-# The underlying command:
-$ docker build . \
-  --tag kubernetes-hugo \
-  --build-arg HUGO_VERSION=0.40.3
+make docker-image
 ```

-You can create an image for a different version of Hugo by changing the value of the `HUGO_VERSION` argument for the build. You *must* specify a version or the image will not build.
-Once the `kubernetes-hugo` image has been built locally, you can build the site:
+Once the `kubernetes-hugo` image has been built locally, you can run the site locally:

 ```bash
-$ make docker-serve
-
-# The underlying command:
-$ docker run \
-  --rm \
-  --interactive \
-  --tty \
-  --volume $(PWD):/src \
-  kubernetes-hugo:latest \
-  hugo
+make docker-serve
 ```

-As when building without using a Docker container, the results of the build will be published to the `public` directory (the default output directory for [Hugo](https://gohugo.io), the static site generator used to build this site).
+Open up your browser to http://localhost:1313 to view the site.

 ## Thank you!

-Kubernetes thrives on community participation, and we really appreciate your
-contributions to our site and our documentation!
+Kubernetes thrives on community participation, and we really appreciate your contributions to our site and our documentation!
@@ -800,8 +800,6 @@ dd
 	&:after
 		transform: rotate(-45deg)

-
-
 #docsContent
 	position: relative
 	float: right
@@ -895,7 +893,6 @@ dd
 		overflow-x: auto

 	h1 code, h2 code, h3 code, h4 code, h5 code, h6 code
-		font-family: inherit
 		font-size: inherit
 		background-color: transparent

@@ -999,10 +996,6 @@ dd
 	img
 		max-width: 100%

-	a
-		//font-weight: 700
-		text-decoration: underline
-
 	#TableOfContents > ul > li { list-style: none; }
 	#TableOfContents
 		ul, li
@@ -1133,7 +1126,6 @@ $feature-box-div-margin-bottom: 40px
 #video
 	width: 100%
 	position: relative
-	background-image: url(/images/kub_video_banner_box.jpg)
 	background-position: center center
 	background-size: cover

@@ -1238,8 +1230,15 @@ $feature-box-div-margin-bottom: 40px
 		&:hover
 			border-color: white

-
-
+// CNCF
+#cncf
+	padding-top: 60px
+	padding-bottom: 140px
+	background-color: $light-grey
+	background-image: url(/images/cncf-color.png)
+	background-position: center 100px
+	background-repeat: no-repeat
+	background-size: 300px

 // Features
 #features
@@ -1721,35 +1720,25 @@ $feature-box-div-margin-bottom: 40px
 		text-align: center
 		margin-bottom: 30px

+		img
+			padding-bottom: 1rem;
+
 		div
 			position: relative
 			display: inline-block
 			vertical-align: top
 			width: 100%
 			min-height: 230px
-			padding: 125px 10px 15px
-			margin-bottom: 30px
+			margin-bottom: 60px
+			padding-right: 1rem;
 			background-position: top center
-			background-repeat: no-repeat
-
-		div:nth-child(1)
-			background-image: url(/images/community_logos/pearson_logo.png)
-
-		div:nth-child(2)
-			background-image: url(/images/community_logos/box_logo.png)
-
-		div:nth-child(3)
-			background-image: url(/images/community_logos/ebay_logo.png)
-
-		div:nth-child(4)
-			background-image: url(/images/community_logos/wikimedia_foundation_logo.png)
-
+	
 		p
 			font-size: 20px

 		a
 			position: absolute
-			bottom: 0
+			bottom: -30px;
 			left: 50%
 			transform: translateX(-50%)
 			color: $blue
@@ -1768,3 +1757,37 @@ $feature-box-div-margin-bottom: 40px

 .gsc-above-wrapper-area
 	border-bottom: 0;
+
+
+
+/* Bing Search */
+#bing-results-container
+    margin-top: 30px
+    margin-left: 20px
+
+.bing-result
+    margin-bottom: 20px
+
+.bing-result-name a
+    font-size: 16px
+    color: #0000CC
+
+
+.bing-result-url
+    color: #008000
+    font-size: 13px
+
+.bing-result-snippet
+    color: #000
+    font-size: 11px
+
+#bing-pagination-container
+	margin: 10px
+	margin-left: 20px
+
+.bing-page-anchor
+    text-decoration: none!important
+    cursor: pointer
+    color: #0000CC
+    margin-right: 8px
+
@@ -138,7 +138,6 @@ $video-section-height: 550px
 	#video
 		height: $video-section-height
 		position: relative
-		background-image: url(../images/kub_video_banner_box.jpg)
 		background-position: center center
 		background-size: cover

@@ -4,4 +4,4 @@
 // media queries
@import "base"
@import "tablet"
-@import "desktop"
+@import "desktop"
@@ -7,7 +7,7 @@ enableRobotsTXT = true

 disableKinds = ["taxonomy", "taxonomyTerm"]

-ignoreFiles = [ "^OWNERS$", "README.md", "^node_modules$" ]
+ignoreFiles = [ "^OWNERS$", "README.md", "^node_modules$", "content/en/docs/doc-contributor-tools" ]

 contentDir = "content/en"

@@ -19,6 +19,12 @@ pygmentsUseClasses = false
 # See https://help.farbox.com/pygments.html
 pygmentsStyle = "emacs"

+# Enable Git variables like commit, lastmod
+enableGitInfo = true
+
+# This is currently used for testing.
+disableLanguages = ["no"]
+
 [blackfriday]
 hrefTargetBlank = true
 fractions = false
@@ -27,49 +33,79 @@ fractions = false
 date = ["date", ":filename", "publishDate", "lastmod"]

 [permalinks]
-  blog = "/:section/:year/:month/:day/:slug/"
+blog = "/:section/:year/:month/:day/:slug/"

 # Be explicit about the output formats. We (currently) only want an RSS feed for the home page.
 [outputs]
-home = [ "HTML", "RSS"]
+home = [ "HTML", "RSS", "HEADERS" ]
 page = [ "HTML"]
 section = [ "HTML"]

+# Add a "text/netlify" media type for auto-generating the _headers file
+[mediaTypes]
+[mediaTypes."text/netlify"]
+delimiter = ""
+
 [outputFormats]
 [outputFormats.RSS]
 baseName = "feed"

+# _headers file output (uses the template at layouts/index.headers)
+[outputFormats.HEADERS]
+mediatype = "text/netlify"
+baseName = "_headers"
+isPlainText = true
+notAlternative = true
+
 [params]

 time_format_blog = "Monday, January 02, 2006"
 description = "Production-Grade Container Orchestration"
 showedit = true

-latest = "v1.11"
+latest = "v1.12"

-fullversion = "v1.11.0"
-version = "v1.11"
+fullversion = "v1.12.0"
+version = "v1.12"
 githubbranch = "master"
 docsbranch = "master"
 deprecated = false
 currentUrl = "https://kubernetes.io/docs/home/"
-nextUrl = "http://kubernetes-io-vnext-staging.netlify.com/"
+nextUrl = "https://kubernetes-io-vnext-staging.netlify.com/"
 githubWebsiteRepo = "github.com/kubernetes/website"
 githubWebsiteRaw = "raw.githubusercontent.com/kubernetes/website"

 [[params.versions]]
-fullversion = "v1.11.0"
-version = "v1.11"
-githubbranch = "v1.11.0"
-docsbranch = "release-1.11"
+fullversion = "v1.12.0"
+version = "v1.12"
+githubbranch = "v1.12.0"
+docsbranch = "release-1.12"
 url = "https://kubernetes.io"

+[params.pushAssets]
+css = [
+  "callouts",
+  "styles",
+  "custom-jekyll/tags"
+]
+js = [
+  "custom-jekyll/tags",
+  "script"
+]
+
+[[params.versions]]
+fullversion = "v1.11.3"
+version = "v1.11"
+githubbranch = "v1.11.3"
+docsbranch = "release-1.11"
+url = "https://v1-11.docs.kubernetes.io"
+
 [[params.versions]]
 fullversion = "v1.10.3"
 version = "v1.10"
 githubbranch = "v1.10.3"
 docsbranch = "release-1.10"
-url = "https://kubernetes.io"
+url = "https://v1-10.docs.kubernetes.io"

 [[params.versions]]
 fullversion = "v1.9.7"
@@ -85,13 +121,6 @@ githubbranch = "v1.8.4"
 docsbranch = "release-1.8"
 url = "https://v1-8.docs.kubernetes.io"

-[[params.versions]]
-fullversion = "v1.7.6"
-version = "v1.7"
-githubbranch = "v1.7.6"
-docsbranch = "release-1.7"
-url = "https://v1-7.docs.kubernetes.io"
-

 # Language definitions.

@@ -105,8 +134,15 @@ weight = 1
 [languages.cn]
 title = "Kubernetes"
 description = "Production-Grade Container Orchestration"
-languageName ="Chinese"
+languageName = "Chinese"
 weight = 2
 contentDir = "content/cn"
-
+[languages.no]
+title = "Kubernetes"
+description = "Production-Grade Container Orchestration"
+languageName ="Norsk"
+weight = 3
+contentDir = "content/no"
+# A list of language codes to look for untranslated content, ordered from left to right.
+language_alternatives = ["en"]

@@ -104,7 +104,7 @@ $ kubectl create -f ./secret.yaml
 secret "mysecret" created
 ```

-**编码注意：** secret 数据的序列化 JSON 和 YAML 值使用 base64 编码成字符串。换行符在这些字符串中无效，必须省略。当在 Darwin/OS X 上使用 `base64` 实用程序时，用户应避免使用 `-b` 选项来拆分长行。另外，对于 Linux 用户如果 `-w` 选项不可用的话，应该添加选项 `-w 0` 到 `base64` 命令或管道 `base64 | tr -d '\n' ` 。
+**编码注意：** secret 数据的序列化 JSON 和 YAML 值使用 base64 编码成字符串。换行符在这些字符串中无效，必须省略。当在 Darwin/macOS 上使用 `base64` 实用程序时，用户应避免使用 `-b` 选项来拆分长行。另外，对于 Linux 用户如果 `-w` 选项不可用的话，应该添加选项 `-w 0` 到 `base64` 命令或管道 `base64 | tr -d '\n' ` 。

 #### 解码 Secret

@@ -34,7 +34,7 @@ content_template: templates/tutorial

 * 本任务使用 [外部负载均衡服务](/docs/tasks/access-application-cluster/create-external-load-balancer/)，
  所以需要对应的可支持此功能的环境。如果你的环境不能支持，你可以使用
-  [NodePort](/docs/user-guide/services/#type-nodeport) 类型的服务代替。
+  [NodePort](/docs/user-guide/services/#nodeport) 类型的服务代替。

 {{% /capture %}}

@@ -4,7 +4,7 @@ metadata:
  name: default-mem-demo-2
 spec:
  containers:
-  - name: defalt-mem-demo-2-ctr
+  - name: default-mem-demo-2-ctr
    image: nginx
    resources:
      limits:
@@ -116,7 +116,7 @@ command=GET
 ## Type=NodePort 类型 Services 的 Source IP


-对于 Kubernetes 1.5，发送给类型为 [Type=NodePort](/docs/user-guide/services/#type-nodeport) Services 的数据包默认进行源地址 NAT。你可以创建一个 `NodePort` Service 来进行测试：
+对于 Kubernetes 1.5，发送给类型为 [Type=NodePort](/docs/user-guide/services/#nodeport) Services 的数据包默认进行源地址 NAT。你可以创建一个 `NodePort` Service 来进行测试：

 ```console
 $ kubectl expose deployment source-ip-app --name=nodeport --port=80 --target-port=8080 --type=NodePort
@@ -210,7 +210,7 @@ client_address=104.132.1.79
 ## Type=LoadBalancer 类型 Services 的 Source IP


-对于 Kubernetes 1.5，发送给类型为 [Type=LoadBalancer](/docs/user-guide/services/#type-nodeport) Services 的数据包默认进行源地址 NAT，这是由于所有处于 `Ready` 状态的 Kubernetes 节点对于负载均衡的流量都是符合条件的。所以如果数据包到达一个没有 endpoint 的节点，系统将把这个包代理到*有* endpoint 的节点，并替换数据包的源 IP 为节点的 IP（如前面章节所述）。
+对于 Kubernetes 1.5，发送给类型为 [Type=LoadBalancer](/docs/user-guide/services/#nodeport) Services 的数据包默认进行源地址 NAT，这是由于所有处于 `Ready` 状态的 Kubernetes 节点对于负载均衡的流量都是符合条件的。所以如果数据包到达一个没有 endpoint 的节点，系统将把这个包代理到*有* endpoint 的节点，并替换数据包的源 IP 为节点的 IP（如前面章节所述）。


 你可以通过在一个 loadbalancer 上暴露这个 source-ip-app 来进行测试。
@@ -15,7 +15,7 @@ approvers:

 * [Persistent Volumes](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) 定义持久化磁盘（磁盘生命周期不和 Pods 绑定）。
 * [Services](https://kubernetes.io/docs/concepts/services-networking/service/) 使得 Pods 能够找到其它 Pods。
-* [External Load Balancers](https://kubernetes.io/docs/concepts/services-networking/service/#type-loadbalancer) 对外暴露 Services。
+* [External Load Balancers](https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer) 对外暴露 Services。
 * [Deployments](http://kubernetes.io/docs/user-guide/deployments/) 确保 Pods 持续运行。
 * [Secrets](http://kubernetes.io/docs/user-guide/secrets/) 保存敏感密码信息。

@@ -66,7 +66,7 @@ kubectl create -f https://raw.githubusercontent.com/kubernetes/examples/master/m
 Kubernetes本质是模块化的，可以在各种环境中运行。但并不是所有集群都相同。此处是本示例的一些要求：
 * 需要 1.2 版本以上的 Kubernetes，以使用更新的特性，例如 PV Claims 和 Deployments。运行 `kubectl version` 来查看你的集群版本。
 * [Cluster DNS](https://github.com/kubernetes/dns) 将被用于服务发现。
-* 一个 [external load balancer](https://kubernetes.io/docs/concepts/services-networking/service/#type-loadbalancer) 将被用于接入 WordPress。
+* 一个 [external load balancer](https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer) 将被用于接入 WordPress。
 * 使用了 [Persistent Volume Claims](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#persistentvolumeclaims)。你必须创建集群中需要的 Persistent Volumes。本示例将展示两种类型的 volume 的创建方法，但是任何类型的 volume 都是足够使用的。


@@ -0,0 +1,3 @@
+---
+headless: true
+---
@@ -6,168 +6,57 @@ cid: home

 {{< deprecationwarning >}}

-<!--  OCEAN NODES  -->
-<section id="oceanNodes">
-    <main>
-        <div class="image-wrapper"><img src="images/flower.png"></div>
-        <div class="content">
-            <h3><a href="/docs/concepts/overview/what-is-kubernetes/">Kubernetes</a> is an open-source system for automating deployment, scaling, and management of containerized applications.</h3>
-            <p>It groups containers that make up an application into logical units for easy management and discovery. Kubernetes builds
-            upon <a href="http://queue.acm.org/detail.cfm?id=2898444">15 years of experience of running production workloads at Google</a>,
-            combined with best-of-breed ideas and practices from the community.</p>
-        </div>
-    </main>
-    <main>
-        <div class="image-wrapper"><img src="images/scalable.png"></div>
-        <div class="content">
-            <h4>Planet Scale</h4>
-            <p>Designed on the same principles that allows Google to run billions of containers a week, Kubernetes can scale without increasing your ops team.</p>
-        </div>
-    </main>
-    <main>
-        <div class="image-wrapper"><img src="images/blocks.png"></div>
-        <div class="content">
-            <h4>Never Outgrow</h4>
-            <p>Whether testing locally or running a global enterprise, Kubernetes flexibility grows with you to deliver your applications consistently and easily no matter how complex your need is.</p>
-        </div>
-    </main>
-    <main>
-        <div class="image-wrapper"><img src="images/suitcase.png"></div>
-        <div class="content">
-            <h4>Run Anywhere</h4>
-            <p>Kubernetes is open source giving you the freedom to take advantage of on-premises, hybrid, or public cloud infrastructure, letting you effortlessly move workloads to where it matters to you.</p>
-        </div>
-    </main>
-</section>
+{{< blocks/section id="oceanNodes" >}}
+{{% blocks/feature image="flower" %}}
+### [Kubernetes]({{< relref "/docs/concepts/overview/what-is-kubernetes" >}}) is an open-source system for automating deployment, scaling, and management of containerized applications.

-<section id="video">
-    <div class="light-text">
-        <h2>The Challenges of Migrating 150+ Microservices to Kubernetes</h2>
+It groups containers that make up an application into logical units for easy management and discovery. Kubernetes builds upon [15 years of experience of running production workloads at Google](http://queue.acm.org/detail.cfm?id=2898444), combined with best-of-breed ideas and practices from the community.
+{{% /blocks/feature %}}
+
+{{% blocks/feature image="scalable" %}}
+#### Planet Scale
+
+Designed on the same principles that allows Google to run billions of containers a week, Kubernetes can scale without increasing your ops team.
+
+{{% /blocks/feature %}}
+
+{{% blocks/feature image="blocks" %}}
+#### Never Outgrow
+
+Whether testing locally or running a global enterprise, Kubernetes flexibility grows with you to deliver your applications consistently and easily no matter how complex your need is.
+
+{{% /blocks/feature %}}
+
+{{% blocks/feature image="suitcase" %}}
+#### Run Anywhere
+
+Kubernetes is open source giving you the freedom to take advantage of on-premises, hybrid, or public cloud infrastructure, letting you effortlessly move workloads to where it matters to you.
+
+{{% /blocks/feature %}}
+
+{{< /blocks/section >}}
+
+{{< blocks/section id="video" background-image="kub_video_banner_homepage" >}}
+<div class="light-text">
+<h2>The Challenges of Migrating 150+ Microservices to Kubernetes</h2>
        <p>By Sarah Wells, Technical Director for Operations and Reliability, Financial Times</p>
        <button id="desktopShowVideoButton" onclick="kub.showVideo()">Watch Video</button>
        <br>
        <br>
-        <a href="https://www.lfasiallc.com/events/kubecon-cloudnativecon-china-2018/" button id= "desktopKCButton">Attend KubeCon in Shanghai on Nov. 14-15, 2018</a>
+        <br>
+        <a href="https://www.lfasiallc.com/events/kubecon-cloudnativecon-china-2018/" button id="desktopKCButton">Attend KubeCon in Shanghai on Nov. 13-15, 2018</a>
        <br>
        <br>
        <br>
        <br>
-        <a href="https://events.linuxfoundation.org/events/kubecon-cloudnativecon-north-america-2018/" button id= "desktopKCButton">Attend KubeCon in Seattle on Dec. 11-13, 2018</a>
-  </div>
-  <button id="mobileShowVideoButton" onclick="kub.showVideo()"></button>
-</section>
-
-<section id="features">
-    <main>
-        <h3 class="center">Kubernetes Features</h3>
-
-        <div class="feature-box">
-            <div>
-                <h4><a href="/docs/concepts/configuration/manage-compute-resources-container/">Automatic binpacking</a></h4>
-                <p>Automatically places containers based on their resource requirements and other constraints, while not
-                    sacrificing availability. Mix critical and best-effort workloads in order to drive up utilization and save even more resources.</p>
-            </div>
-            <div>
-                <h4><a href="/docs/concepts/workloads/controllers/replicationcontroller/#what-is-a-replicationcontroller">Self-healing</a></h4>
-                <p>Restarts containers that fail, replaces and reschedules containers when nodes die, kills containers
-                    that don't respond to your user-defined health check, and doesn't advertise them to clients until they are ready to serve.</p>
-            </div>
-        </div>
-
-        <div class="feature-box">
-            <div>
-                <h4><a href="/docs/tasks/run-application/horizontal-pod-autoscale/">Horizontal scaling</a></h4>
-                <p>Scale your application up and down with a simple command, with a UI, or automatically based on CPU usage.</p>
-            </div>
-            <div>
-                <h4><a href="/docs/concepts/services-networking/service/">Service discovery and load balancing</a></h4>
-                <p>No need to modify your application to use an unfamiliar service discovery mechanism. Kubernetes gives
-                    containers their own IP addresses and a single DNS name for a set of containers, and can load-balance across them.</p>
-            </div>
-        </div>
-
-        <div class="feature-box">
-            <div>
-                <h4><a href="/docs/concepts/workloads/controllers/deployment/#what-is-a-deployment">Automated rollouts and rollbacks</a></h4>
-                <p>Kubernetes progressively rolls out changes to your application or its configuration, while monitoring
-                    application health to ensure it doesn't kill all your instances at the same time. If something goes
-                    wrong, Kubernetes will rollback the change for you. Take advantage of a growing ecosystem of deployment solutions.</p>
-            </div>
-            <div>
-                <h4><a href="/docs/concepts/configuration/secret/">Secret</a> and <a href="/docs/tasks/configure-pod-container/configure-pod-configmap/">configuration</a> management</h4>
-                <p>Deploy and update secrets and application configuration without rebuilding your image and without
-                    exposing secrets in your stack configuration.</p>
-            </div>
-        </div>
-
-        <div class="feature-box">
-            <div>
-                <h4><a href="/docs/concepts/storage/persistent-volumes/">Storage orchestration</a></h4>
-                <p>Automatically mount the storage system of your choice, whether from local storage, a public cloud provider
-                such as <a href="https://cloud.google.com/storage/">GCP</a> or <a href="https://aws.amazon.com/products/storage/">AWS</a>, or a network storage system such as NFS, iSCSI,
-                Gluster, Ceph, Cinder, or Flocker.</p>
-            </div>
-            <div>
-                <h4><a href="/docs/concepts/workloads/controllers/jobs-run-to-completion/">Batch execution</a></h4>
-                <p>In addition to services, Kubernetes can manage your batch and CI workloads, replacing containers that fail, if desired.</p>
-            </div>
-        </div>
-
-    </main>
-</section>
-
-<section id="talkToUs">
-    <main>
-        <h3>Case Studies</h3>
-        <div id="caseStudiesWrapper">
-            <div>
-                <p>Using Kubernetes to reinvent the world's largest educational company</p>
-                <a href="/case-studies/pearson/">Read more</a>
-            </div>
-            <div>
-                <p>Kubernetes at Box: Microservices at Maximum Velocity</p>
-                <a href="https://blog.box.com/blog/kubernetes-box-microservices-maximum-velocity/">Read more</a>
-            </div>
-            <div>
-                <p>Inside eBay's shift to Kubernetes and containers atop OpenStack</p>
-                <a href="https://www.nextplatform.com/2015/11/12/inside-ebays-shift-to-kubernetes-and-containers-atop-openstack/">Read more</a>
-            </div>
-            <div>
-                <p>Migrating from a homegrown 'cluster' to Kubernetes</p>
-                <a href="https://www.youtube.com/watch?v=6XGUTu3WhBw">Watch the video</a>
-            </div>
-        </div>
-
-        <!--<div id="bigSocial">-->
-            <!--&lt;!&ndash;<div>&ndash;&gt;-->
-                <!--&lt;!&ndash;<a href="https://youtu.be/OLwzSHBtxhI">Watch the video</a>&ndash;&gt;-->
-                <!--&lt;!&ndash;<p>Creating Next Generation Kubernetes Infrastructure at Viacom</p>&ndash;&gt;-->
-            <!--&lt;!&ndash;</div>&ndash;&gt;-->
-            <!--<div>-->
-                <!--<p>Using Kubernetes to reinvent the world's largest educational company</p>-->
-                <!--<a href="/case-studies/pearson/">Read more</a>-->
-            <!--</div>-->
-            <!--<div>-->
-                <!--<p>Kubernetes at Box: Microservices at Maximum Velocity</p>-->
-                <!--<a href="https://blog.box.com/blog/kubernetes-box-microservices-maximum-velocity/">Read more</a>-->
-            <!--</div>-->
-            <!--<div>-->
-                <!--<p>Inside eBay's shift to Kubernetes and containers atop OpenStack</p>-->
-                <!--<a href="http://www.nextplatform.com/2015/11/12/inside-ebays-shift-to-kubernetes-and-containers-atop-openstack/">Read the story</a>-->
-            <!--</div>-->
-            <!--<div>-->
-                <!--<p>Migrating from a homegrown 'cluster' to Kubernetes</p>-->
-                <!--<a href="https://www.youtube.com/watch?v=6XGUTu3WhBw">Watch the video</a>-->
-            <!--</div>-->
-        <!--</div>-->
-        <h5 style="text-align: center"><a href="/case-studies/" style="color: #3371E3; font-weight: 400">View all case studies</a></h5>
-    </main>
-</section>
-
-
-
-
+        <a href="https://events.linuxfoundation.org/events/kubecon-cloudnativecon-north-america-2018/" button id="desktopKCButton">Attend KubeCon in Seattle on Dec. 11-13, 2018</a>
+</div>
 <div id="videoPlayer">
    <iframe data-url="https://www.youtube.com/embed/H06qrNmGqyE?autoplay=1" frameborder="0" allowfullscreen></iframe>
    <button id="closeButton"></button>
 </div>
+{{< /blocks/section >}}
+
+{{< blocks/kubernetes-features >}}
+
+{{< blocks/case-studies >}}
@@ -1,3 +1,10 @@
 ---
-title: Blog
+title: Kubernetes Blog
+linkTitle: Blog
+menu:
+  main:
+    title: "Blog"
+    weight: 40
+    post: >
+       <p>Read the latest news for Kubernetes and the containers space in general, and get technical how-tos hot off the presses.</p>
 ---
@@ -7,7 +7,7 @@ url: /blog/2015/05/Kubernetes-On-Openstack



-[![](http://3.bp.blogspot.com/-EOrCHChZJZE/VVZzq43g6CI/AAAAAAAAF-E/JUilRHk369E/s400/Untitled%2Bdrawing.jpg)](http://3.bp.blogspot.com/-EOrCHChZJZE/VVZzq43g6CI/AAAAAAAAF-E/JUilRHk369E/s1600/Untitled%2Bdrawing.jpg)
+[![](https://3.bp.blogspot.com/-EOrCHChZJZE/VVZzq43g6CI/AAAAAAAAF-E/JUilRHk369E/s400/Untitled%2Bdrawing.jpg)](https://3.bp.blogspot.com/-EOrCHChZJZE/VVZzq43g6CI/AAAAAAAAF-E/JUilRHk369E/s1600/Untitled%2Bdrawing.jpg)



@@ -121,7 +121,7 @@ CONTAINER ID  IMAGE           COMMAND           CREATED        STATUS
 docker\_cr.sh
  ```

-Since the command line arguments to CRIU were long, a helper script called docker\_cr.sh was provided in the CRIU source tree to simplify the proces.  So, for the above container, one would simply C/R the container as follows (for details see [http://criu.org/Docker](http://criu.org/Docker)):
+Since the command line arguments to CRIU were long, a helper script called docker\_cr.sh was provided in the CRIU source tree to simplify the process.  So, for the above container, one would simply C/R the container as follows (for details see [http://criu.org/Docker](http://criu.org/Docker)):

 ```  
 $ sudo docker\_cr.sh -c 4397   
@@ -17,7 +17,7 @@ In the spirit of minimum viability, the first version of Jenkins-on-Kubernetes I

 Here’s a visual of that first version:

-[![](http://1.bp.blogspot.com/-ccmpTmulrng/VaVxOs7gysI/AAAAAAAAAU8/bCEzgGGm-pE/s400/0.png)](http://1.bp.blogspot.com/-ccmpTmulrng/VaVxOs7gysI/AAAAAAAAAU8/bCEzgGGm-pE/s1600/0.png)
+[![](https://1.bp.blogspot.com/-ccmpTmulrng/VaVxOs7gysI/AAAAAAAAAU8/bCEzgGGm-pE/s400/0.png)](https://1.bp.blogspot.com/-ccmpTmulrng/VaVxOs7gysI/AAAAAAAAAU8/bCEzgGGm-pE/s1600/0.png)



@@ -156,7 +156,7 @@ The pod will have a service exposing TCP 80 and 443 to a public load balancer. H

 And here’s an overview with the SSL termination proxy in place. Notice that Jenkins is no longer directly exposed to the public Internet:

-[![](http://3.bp.blogspot.com/-0B1BEQo_fWc/VaVxVUBkf3I/AAAAAAAAAVE/5yCCnA29C88/s400/0%2B%25281%2529.png)](http://3.bp.blogspot.com/-0B1BEQo_fWc/VaVxVUBkf3I/AAAAAAAAAVE/5yCCnA29C88/s1600/0%2B%25281%2529.png)
+[![](https://3.bp.blogspot.com/-0B1BEQo_fWc/VaVxVUBkf3I/AAAAAAAAAVE/5yCCnA29C88/s400/0%2B%25281%2529.png)](https://3.bp.blogspot.com/-0B1BEQo_fWc/VaVxVUBkf3I/AAAAAAAAAVE/5yCCnA29C88/s1600/0%2B%25281%2529.png)



@@ -8,7 +8,7 @@ At Devoxx Belgium and Devoxx Morocco, Ray Tsang and I showed a Raspberry Pi clus

 ### Wait! Why the heck build a Raspberry Pi cluster running Kubernetes?&nbsp;

-We had two big reasons to build the Pi cluster at Quintor. First of all we wanted to experiment with container technology at scale on real hardware. You can try out container technology using virtual machines, but Kubernetes runs great on on bare metal too. To explore what that’d be like, we built a Raspberry Pi cluster just like we would build a cluster of machines in a production datacenter. This allowed us to understand and simulate how Kubernetes would work when we move it to our data centers.  
+We had two big reasons to build the Pi cluster at Quintor. First of all we wanted to experiment with container technology at scale on real hardware. You can try out container technology using virtual machines, but Kubernetes runs great on bare metal too. To explore what that’d be like, we built a Raspberry Pi cluster just like we would build a cluster of machines in a production datacenter. This allowed us to understand and simulate how Kubernetes would work when we move it to our data centers.  

 Secondly, we did not want to blow the budget to do this exploration. And what is cheaper than a Raspberry Pi! If you want to build a cluster comprising many nodes, each node should have a good cost to performance ratio. Our Pi cluster has 20 CPU cores, which is more than many servers, yet cost us less than $400. Additionally, the total power consumption is low and the form factor is small, which is great for these kind of demo systems.  

@@ -6,7 +6,7 @@ url: /blog/2015/11/Kubernetes-As-Foundation-For-Cloud-Native-Paas
 ---
 With Kubernetes continuing to gain momentum as a critical tool for building and scaling container based applications, we’ve been thrilled to see a growing number of platform as a service (PaaS) offerings adopt it as a foundation. PaaS developers have been drawn to Kubernetes by its rapid rate of maturation, the soundness of its core architectural concepts, and the strength of its contributor community. The [Kubernetes ecosystem](https://kubernetes.io/blog/2015/07/the-growing-kubernetes-ecosystem) continues to grow, and these PaaS projects are great additions to it.

-[![](http://1.bp.blogspot.com/-xX93tnoIlGo/Vjj2fSc_CDI/AAAAAAAAAi0/lvTkT9jyFog/s400/k8%2Bipaas%2B1.png)](http://1.bp.blogspot.com/-xX93tnoIlGo/Vjj2fSc_CDI/AAAAAAAAAi0/lvTkT9jyFog/s1600/k8%2Bipaas%2B1.png)
+[![](https://1.bp.blogspot.com/-xX93tnoIlGo/Vjj2fSc_CDI/AAAAAAAAAi0/lvTkT9jyFog/s400/k8%2Bipaas%2B1.png)](https://1.bp.blogspot.com/-xX93tnoIlGo/Vjj2fSc_CDI/AAAAAAAAAi0/lvTkT9jyFog/s1600/k8%2Bipaas%2B1.png)



@@ -23,7 +23,7 @@ With Kubernetes continuing to gain momentum as a critical tool for building and



-[![](http://1.bp.blogspot.com/-1XZFGRHGb34/Vjj2wUtA6pI/AAAAAAAAAi8/SD-qRhVIiIs/s400/k8%2Bipaas%2B2.png)](http://1.bp.blogspot.com/-1XZFGRHGb34/Vjj2wUtA6pI/AAAAAAAAAi8/SD-qRhVIiIs/s1600/k8%2Bipaas%2B2.png)
+[![](https://1.bp.blogspot.com/-1XZFGRHGb34/Vjj2wUtA6pI/AAAAAAAAAi8/SD-qRhVIiIs/s400/k8%2Bipaas%2B2.png)](https://1.bp.blogspot.com/-1XZFGRHGb34/Vjj2wUtA6pI/AAAAAAAAAi8/SD-qRhVIiIs/s1600/k8%2Bipaas%2B2.png)



@@ -38,7 +38,7 @@ With Kubernetes continuing to gain momentum as a critical tool for building and



-[![](http://2.bp.blogspot.com/-t3L1CANyhUs/Vjj28Zpf9WI/AAAAAAAAAjE/Ef-PLLmHGvU/s400/k8%2Bipaas%2B3.png)](http://2.bp.blogspot.com/-t3L1CANyhUs/Vjj28Zpf9WI/AAAAAAAAAjE/Ef-PLLmHGvU/s1600/k8%2Bipaas%2B3.png)
+[![](https://2.bp.blogspot.com/-t3L1CANyhUs/Vjj28Zpf9WI/AAAAAAAAAjE/Ef-PLLmHGvU/s400/k8%2Bipaas%2B3.png)](https://2.bp.blogspot.com/-t3L1CANyhUs/Vjj28Zpf9WI/AAAAAAAAAjE/Ef-PLLmHGvU/s1600/k8%2Bipaas%2B3.png)



@@ -64,7 +64,7 @@ Huawei has made Kubernetes the core runtime engine for container based applicati



-[![](http://2.bp.blogspot.com/-Ys0Zn4IQzn0/Vjj3JIE0BVI/AAAAAAAAAjM/ktwltzVa1GE/s400/k8%2Bipaas%2B4.png)](http://2.bp.blogspot.com/-Ys0Zn4IQzn0/Vjj3JIE0BVI/AAAAAAAAAjM/ktwltzVa1GE/s1600/k8%2Bipaas%2B4.png)
+[![](https://2.bp.blogspot.com/-Ys0Zn4IQzn0/Vjj3JIE0BVI/AAAAAAAAAjM/ktwltzVa1GE/s400/k8%2Bipaas%2B4.png)](https://2.bp.blogspot.com/-Ys0Zn4IQzn0/Vjj3JIE0BVI/AAAAAAAAAjM/ktwltzVa1GE/s1600/k8%2Bipaas%2B4.png)



@@ -27,7 +27,7 @@ The easiest way to take advantage of sysdig’s Kubernetes support is by launchi
 Now that csysdig is running, hit F2 to bring up the views panel, and you'll notice the presence of a bunch of new views. The **k8s Namespaces** view can be used to see the list of namespaces and observe the amount of CPU, memory, network and disk resources each of them is using on this machine:  


-[![](http://2.bp.blogspot.com/-9kXfpo76r0k/Vkz8AkpctEI/AAAAAAAAAss/yvf9oc759Wg/s640/sisdig%2B6.png)](http://2.bp.blogspot.com/-9kXfpo76r0k/Vkz8AkpctEI/AAAAAAAAAss/yvf9oc759Wg/s1600/sisdig%2B6.png)
+[![](https://2.bp.blogspot.com/-9kXfpo76r0k/Vkz8AkpctEI/AAAAAAAAAss/yvf9oc759Wg/s640/sisdig%2B6.png)](https://2.bp.blogspot.com/-9kXfpo76r0k/Vkz8AkpctEI/AAAAAAAAAss/yvf9oc759Wg/s1600/sisdig%2B6.png)



@@ -40,7 +40,7 @@ Now that csysdig is running, hit F2 to bring up the views panel, and you'll noti
 Similarly, you can select **k8s Services** to see the same information broken up by service:  


-[![](http://2.bp.blogspot.com/-Ya1W3Z_ETcs/Vkz8AN3XtfI/AAAAAAAAAs8/HNv_TvHpfHU/s640/sisdig%2B2.png)](http://2.bp.blogspot.com/-Ya1W3Z_ETcs/Vkz8AN3XtfI/AAAAAAAAAs8/HNv_TvHpfHU/s1600/sisdig%2B2.png)
+[![](https://2.bp.blogspot.com/-Ya1W3Z_ETcs/Vkz8AN3XtfI/AAAAAAAAAs8/HNv_TvHpfHU/s640/sisdig%2B2.png)](https://2.bp.blogspot.com/-Ya1W3Z_ETcs/Vkz8AN3XtfI/AAAAAAAAAs8/HNv_TvHpfHU/s1600/sisdig%2B2.png)



@@ -53,7 +53,7 @@ Similarly, you can select **k8s Services** to see the same information broken up
 or **k8s Controllers** to see the replication controllers:  


-[![](http://3.bp.blogspot.com/-gGkgXRC5P6g/Vkz8A1RVyAI/AAAAAAAAAtQ/SFlHQeNrDjQ/s640/sysdig%2B1.png)](http://3.bp.blogspot.com/-gGkgXRC5P6g/Vkz8A1RVyAI/AAAAAAAAAtQ/SFlHQeNrDjQ/s1600/sysdig%2B1.png)
+[![](https://3.bp.blogspot.com/-gGkgXRC5P6g/Vkz8A1RVyAI/AAAAAAAAAtQ/SFlHQeNrDjQ/s640/sysdig%2B1.png)](https://3.bp.blogspot.com/-gGkgXRC5P6g/Vkz8A1RVyAI/AAAAAAAAAtQ/SFlHQeNrDjQ/s1600/sysdig%2B1.png)



@@ -66,7 +66,7 @@ or **k8s Controllers** to see the replication controllers:
 or **k8s Pods** to see the list of pods running on this machine and the resources they use:  


-[![](http://3.bp.blogspot.com/-PrDfWzi9F3c/Vkz8H6rPlII/AAAAAAAAAtc/f46tE6EKvoo/s640/sisdig%2B7.png)](http://3.bp.blogspot.com/-PrDfWzi9F3c/Vkz8H6rPlII/AAAAAAAAAtc/f46tE6EKvoo/s1600/sisdig%2B7.png)
+[![](https://3.bp.blogspot.com/-PrDfWzi9F3c/Vkz8H6rPlII/AAAAAAAAAtc/f46tE6EKvoo/s640/sisdig%2B7.png)](https://3.bp.blogspot.com/-PrDfWzi9F3c/Vkz8H6rPlII/AAAAAAAAAtc/f46tE6EKvoo/s1600/sisdig%2B7.png)



@@ -74,7 +74,7 @@ or **k8s Pods** to see the list of pods running on this machine and the resource
 A cool feature in csysdig is the ability to drill down: just select an element, click on enter and&nbsp;–&nbsp;boom&nbsp;–&nbsp;now you're looking inside it. Drill down is also aware of the Kubernetes hierarchy, which means I&nbsp;can start from a service, get the list of its pods, see which containers run inside one of the pods, and go inside one of the containers to explore files, network connections, processes or even threads. Check out the video below.  


-[![](http://1.bp.blogspot.com/-lQ-P2gLywlY/Vkz9MOoTgGI/AAAAAAAAAtk/UB6pW7sUbQA/s640/image09.gif)](http://1.bp.blogspot.com/-lQ-P2gLywlY/Vkz9MOoTgGI/AAAAAAAAAtk/UB6pW7sUbQA/s1600/image09.gif)
+[![](https://1.bp.blogspot.com/-lQ-P2gLywlY/Vkz9MOoTgGI/AAAAAAAAAtk/UB6pW7sUbQA/s640/image09.gif)](https://1.bp.blogspot.com/-lQ-P2gLywlY/Vkz9MOoTgGI/AAAAAAAAAtk/UB6pW7sUbQA/s1600/image09.gif)


 ### Actions!&nbsp;
@@ -90,7 +90,7 @@ Let’s start with a quick review of Kubernetes’ architecture. From the physic

 On the other hand, from the logical/application point of view, Kubernetes clusters are arranged in the hierarchical fashion shown in this picture:  

-[![](http://1.bp.blogspot.com/-p_x0bLRdFJo/Vkz8IPR5q4I/AAAAAAAAAtg/D9UU2MfPmcI/s640/sisdig%2B4.png)](http://1.bp.blogspot.com/-p_x0bLRdFJo/Vkz8IPR5q4I/AAAAAAAAAtg/D9UU2MfPmcI/s1600/sisdig%2B4.png)
+[![](https://1.bp.blogspot.com/-p_x0bLRdFJo/Vkz8IPR5q4I/AAAAAAAAAtg/D9UU2MfPmcI/s640/sisdig%2B4.png)](https://1.bp.blogspot.com/-p_x0bLRdFJo/Vkz8IPR5q4I/AAAAAAAAAtg/D9UU2MfPmcI/s1600/sisdig%2B4.png)



@@ -123,7 +123,7 @@ One of the core features of Sysdig Cloud is groups, which allow you to define th

 If you’re interested in the utilization of your underlying physical resource&nbsp;– e.g., identifying noisy neighbors&nbsp;– then the physical hierarchy is great. But if you’re looking to explore the performance of your applications and microservices, then the logical hierarchy is often the best place to start.&nbsp;

-[![](http://4.bp.blogspot.com/-80u3oSEi_Fw/Vkz8AZgE6eI/AAAAAAAAAtE/3iRDMJKBNmc/s640/sisdig%2B5.png)](http://4.bp.blogspot.com/-80u3oSEi_Fw/Vkz8AZgE6eI/AAAAAAAAAtE/3iRDMJKBNmc/s1600/sisdig%2B5.png)
+[![](https://4.bp.blogspot.com/-80u3oSEi_Fw/Vkz8AZgE6eI/AAAAAAAAAtE/3iRDMJKBNmc/s640/sisdig%2B5.png)](https://4.bp.blogspot.com/-80u3oSEi_Fw/Vkz8AZgE6eI/AAAAAAAAAtE/3iRDMJKBNmc/s1600/sisdig%2B5.png)



@@ -148,7 +148,7 @@ If you’re interested in the utilization of your underlying physical resource&n

 For example: here you can see the overall performance of our WordPress service:&nbsp;

-[![](http://4.bp.blogspot.com/-QAsedrM2UxI/Vkz8Aas-26I/AAAAAAAAAtM/9B7Z33vUQrg/s640/sisdig%2B3.png)](http://4.bp.blogspot.com/-QAsedrM2UxI/Vkz8Aas-26I/AAAAAAAAAtM/9B7Z33vUQrg/s1600/sisdig%2B3.png)
+[![](https://4.bp.blogspot.com/-QAsedrM2UxI/Vkz8Aas-26I/AAAAAAAAAtM/9B7Z33vUQrg/s640/sisdig%2B3.png)](https://4.bp.blogspot.com/-QAsedrM2UxI/Vkz8Aas-26I/AAAAAAAAAtM/9B7Z33vUQrg/s1600/sisdig%2B3.png)

 Keep in mind that the pods implementing this service are scattered across multiple machines, but we can still total request counts, response times and URL statistics aggregated together for this service. And don’t forget: this doesn’t require any configuration or instrumentation of wordpress, apache, or the underlying containers!&nbsp;

@@ -162,7 +162,7 @@ And from this view, I can now easily create alerts for these service-level metri

 We’ve also included Kubernetes awareness in Sysdig Cloud’s famous topology view, at both the physical and logical level.&nbsp;

-[![](http://2.bp.blogspot.com/-2is-UJatmPk/Vk0AtdfvYvI/AAAAAAAAAt0/9SEsl2LCpYI/s640/image02.gif)](http://2.bp.blogspot.com/-2is-UJatmPk/Vk0AtdfvYvI/AAAAAAAAAt0/9SEsl2LCpYI/s1600/image02.gif)
+[![](https://2.bp.blogspot.com/-2is-UJatmPk/Vk0AtdfvYvI/AAAAAAAAAt0/9SEsl2LCpYI/s640/image02.gif)](https://2.bp.blogspot.com/-2is-UJatmPk/Vk0AtdfvYvI/AAAAAAAAAt0/9SEsl2LCpYI/s1600/image02.gif)



@@ -188,7 +188,7 @@ We’ve also included Kubernetes awareness in Sysdig Cloud’s famous topology v



-[![](http://2.bp.blogspot.com/-hGQtaIV9XTA/Vk0RnwtlcGI/AAAAAAAAAuM/7ndiyAWpSvU/s640/image08.gif)](http://2.bp.blogspot.com/-hGQtaIV9XTA/Vk0RnwtlcGI/AAAAAAAAAuM/7ndiyAWpSvU/s1600/image08.gif)
+[![](https://2.bp.blogspot.com/-hGQtaIV9XTA/Vk0RnwtlcGI/AAAAAAAAAuM/7ndiyAWpSvU/s640/image08.gif)](https://2.bp.blogspot.com/-hGQtaIV9XTA/Vk0RnwtlcGI/AAAAAAAAAuM/7ndiyAWpSvU/s1600/image08.gif)



@@ -13,13 +13,13 @@ Fortunately, for developers, our infrastructure is evolving along with the requi

 To demonstrate exactly what we mean by this, I've developed a simple demo of a Container Engine cluster serving 1 million HTTP requests per second. In all honesty, serving 1 million requests per second isn’t really that exciting. In fact, it’s really so very [2013](http://googlecloudplatform.blogspot.com/2013/11/compute-engine-load-balancing-hits-1-million-requests-per-second.html).

-[![](http://4.bp.blogspot.com/-eACCKAzuQFQ/VkO1rwW1DRI/AAAAAAAAAko/zKu-19QCCBU/s640/image01.gif)](http://4.bp.blogspot.com/-eACCKAzuQFQ/VkO1rwW1DRI/AAAAAAAAAko/zKu-19QCCBU/s1600/image01.gif)
+[![](https://4.bp.blogspot.com/-eACCKAzuQFQ/VkO1rwW1DRI/AAAAAAAAAko/zKu-19QCCBU/s640/image01.gif)](https://4.bp.blogspot.com/-eACCKAzuQFQ/VkO1rwW1DRI/AAAAAAAAAko/zKu-19QCCBU/s1600/image01.gif)


 What _is_ exciting is that while successfully handling 1 million HTTP requests per second with uninterrupted availability, we have Kubernetes perform a zero-downtime rolling upgrade of the service to a new version of the software _while we're&nbsp; **still** serving 1 million requests per second_.  


-[![](http://2.bp.blogspot.com/-_96_QwNRHLo/VkO1oDAyLLI/AAAAAAAAAkk/B_y5Uh5ngPU/s640/image00.gif)](http://2.bp.blogspot.com/-_96_QwNRHLo/VkO1oDAyLLI/AAAAAAAAAkk/B_y5Uh5ngPU/s1600/image00.gif)
+[![](https://2.bp.blogspot.com/-_96_QwNRHLo/VkO1oDAyLLI/AAAAAAAAAkk/B_y5Uh5ngPU/s640/image00.gif)](https://2.bp.blogspot.com/-_96_QwNRHLo/VkO1oDAyLLI/AAAAAAAAAkk/B_y5Uh5ngPU/s1600/image00.gif)


 This is only possible due to a large number of performance tweaks and enhancements that have gone into the [Kubernetes 1.1 release](https://kubernetes.io/blog/2015/11/Kubernetes-1-1-Performance-upgrades-improved-tooling-and-a-growing-community). I’m incredibly proud of all of the features that our community has built into this release. Indeed in addition to making it possible to serve 1 million requests per second, we’ve also added an auto-scaler, so that you won’t even have to wake up in the middle of the night to scale your service in response to load or memory pressures.  
@@ -57,7 +57,7 @@ While we could have decreased the “pod startup time” substantially by exclud

 ### Metrics from Kubernetes 1.2&nbsp;

-So what was the result?We run our tests on Google Compute Engine, setting the size of the master VM based on on the size of the Kubernetes cluster. In particular for 1000-node clusters we use a n1-standard-32 VM for the master (32 cores, 120GB RAM).  
+So what was the result?We run our tests on Google Compute Engine, setting the size of the master VM based on the size of the Kubernetes cluster. In particular for 1000-node clusters we use a n1-standard-32 VM for the master (32 cores, 120GB RAM).  


 #### API responsiveness&nbsp;
@@ -148,7 +148,7 @@ Highly available apps, here we come.


 ### Deploying a multi-zone application&nbsp;
-Create the guestbook-go example, which includes a ReplicationController of size 3, running a simple web app. Download all the files from [here](https://github.com/kubernetes/kubernetes/tree/master/examples/guestbook-go), and execute the following command (the command assumes you downloaded them to a directory named “guestbook-go”:  
+Create the guestbook-go example, which includes a ReplicationController of size 3, running a simple web app. Download all the files from [here](https://github.com/kubernetes/examples/tree/master/guestbook-go), and execute the following command (the command assumes you downloaded them to a directory named “guestbook-go”:  

 ```
 kubectl create -f guestbook-go/
@@ -28,7 +28,7 @@ Use your existing tools of choice to actually build your containers

 Use ElasticKube to run, deploy and manage your containers and services

-[![](http://cl.ly/0i3M2L3Q030z/Image%202016-03-11%20at%209.49.12%20AM.png)](http://cl.ly/0i3M2L3Q030z/Image%202016-03-11%20at%209.49.12%20AM.png)
+[![](https://cl.ly/0i3M2L3Q030z/Image%202016-03-11%20at%209.49.12%20AM.png)](http://cl.ly/0i3M2L3Q030z/Image%202016-03-11%20at%209.49.12%20AM.png)



@@ -10,7 +10,7 @@ While Kubernetes is one of the best tools for managing containerized application

 For the past several months, several of us from the Kubernetes community have been working to fix this in the [Minikube](http://github.com/kubernetes/minikube) repository on GitHub. Our goal is to build an easy-to-use, high-fidelity Kubernetes distribution that can be run locally on Mac, Linux and Windows workstations and laptops with a single command.  

-Thanks to lots of help from members of the community, we're proud to announce the official release of Minikube. This release comes with support for [Kubernetes 1.3](https://kubernetes.io/blog/2016/07/kubernetes-1.3-bridging-cloud-native-and-enterprise-workloads), new commands to make interacting with your local cluster easier and experimental drivers for xhyve (on Mac OSX) and KVM (on Linux).  
+Thanks to lots of help from members of the community, we're proud to announce the official release of Minikube. This release comes with support for [Kubernetes 1.3](https://kubernetes.io/blog/2016/07/kubernetes-1.3-bridging-cloud-native-and-enterprise-workloads), new commands to make interacting with your local cluster easier and experimental drivers for xhyve (on macOS) and KVM (on Linux).  

 **Using Minikube**  

@@ -107,8 +107,8 @@ Here's a list of some of the things we're hoping to add to Minikube soon:



- Native hypervisor support for OSX and Windows
- We're planning to remove the dependency on Virtualbox, and integrate with the native hypervisors included in OSX and Windows (Hypervisor.framework and Hyper-v, respectively).
+- Native hypervisor support for macOS and Windows
+- We're planning to remove the dependency on Virtualbox, and integrate with the native hypervisors included in macOS and Windows (Hypervisor.framework and Hyper-v, respectively).
 - Improved support for Kubernetes features
 - We're planning to increase the range of supported Kubernetes features, to include things like Ingress.
 - Configurable versions of Kubernetes
@@ -30,7 +30,7 @@ There is work in progress being done in Kubernetes for image authorization plugi
 **Limit Direct Access to Kubernetes Nodes**  
 You should limit SSH access to Kubernetes nodes, reducing the risk for unauthorized access to host resource. Instead you should ask users to use "kubectl exec", which will provide direct access to the container environment without the ability to access the host.  

-You can use Kubernetes [Authorization Plugins](http://kubernetes.io/docs/admin/authorization/) to further control user access to resources. This allows defining fine-grained-access control rules for specific namespace, containers and operations.  
+You can use Kubernetes [Authorization Plugins](http://kubernetes.io/docs/reference/access-authn-authz/authorization/) to further control user access to resources. This allows defining fine-grained-access control rules for specific namespace, containers and operations.

 **Create Administrative Boundaries between Resources**  
 Limiting the scope of user permissions can reduce the impact of mistakes or malicious activities. A Kubernetes namespace allows you to partition created resources into logically named groups. Resources created in one namespace can be hidden from other namespaces. By default, each resource created by a user in Kubernetes cluster runs in a default namespace, called default. You can create additional namespaces and attach resources and users to them. You can use Kubernetes Authorization plugins to create policies that segregate access to namespace resources between different users.  
@@ -205,7 +205,7 @@ Reference [here](http://kubernetes.io/docs/api-reference/v1/definitions/#_v1_pod



-In case you are running containers with elevated privileges (--privileged) you should consider using the “DenyEscalatingExec” admission control. This control denies exec and attach commands to pods that run with escalated privileges that allow host access. This includes pods that run as privileged, have access to the host IPC namespace, and have access to the host PID namespace. For more details on admission controls, see the Kubernetes [documentation](http://kubernetes.io/docs/admin/admission-controllers/).
+In case you are running containers with elevated privileges (--privileged) you should consider using the “DenyEscalatingExec” admission control. This control denies exec and attach commands to pods that run with escalated privileges that allow host access. This includes pods that run as privileged, have access to the host IPC namespace, and have access to the host PID namespace. For more details on admission controls, see the Kubernetes [documentation](http://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/).



@@ -6,7 +6,7 @@ url: /blog/2017/03/Kubernetes-1.6-Multi-User-Multi-Workloads-At-Scale
 ---
 Today we’re announcing the release of Kubernetes 1.6.  

-In this release the community’s focus is on scale and automation, to help you deploy multiple workloads to multiple users on a cluster. We are announcing that 5,000 node clusters are supported. We moved dynamic storage provisioning to _stable_. Role-based access control ([RBAC](https://kubernetes.io/docs/admin/authorization/rbac/)), [kubefed](https://kubernetes.io/docs/tutorials/federation/set-up-cluster-federation-kubefed/), [kubeadm](https://kubernetes.io/docs/getting-started-guides/kubeadm/), and several scheduling features are moving to _beta_. We have also added intelligent defaults throughout to enable greater automation out of the box.  
+In this release the community’s focus is on scale and automation, to help you deploy multiple workloads to multiple users on a cluster. We are announcing that 5,000 node clusters are supported. We moved dynamic storage provisioning to _stable_. Role-based access control ([RBAC](https://kubernetes.io/docs/reference/access-authn-authz/rbac/)), [kubefed](https://kubernetes.io/docs/tutorials/federation/set-up-cluster-federation-kubefed/), [kubeadm](https://kubernetes.io/docs/getting-started-guides/kubeadm/), and several scheduling features are moving to _beta_. We have also added intelligent defaults throughout to enable greater automation out of the box.  

 **What’s New**  

@@ -14,9 +14,9 @@ In this release the community’s focus is on scale and automation, to help you

 For users who want to scale beyond 5,000 nodes or spread across multiple regions or clouds, [federation](https://kubernetes.io/docs/concepts/cluster-administration/federation/) lets you combine multiple Kubernetes clusters and address them through a single API endpoint. In this release, the [kubefed](https://kubernetes.io//docs/tutorials/federation/set-up-cluster-federation-kubefed) command line utility graduated to _beta_ - with improved support for on-premise clusters. kubefed now [automatically configures](https://kubernetes.io//docs/tutorials/federation/set-up-cluster-federation-kubefed.md#kube-dns-configuration) kube-dns on joining clusters and can pass arguments to federated components.  

-**Security and Setup** : Users concerned with security will find that [RBAC](https://kubernetes.io//docs/admin/authorization/rbac), now _beta_ adds a significant security benefit through more tightly scoped default roles for system components. The default RBAC policies in 1.6 grant scoped permissions to control-plane components, nodes, and controllers. RBAC allows cluster administrators to selectively grant particular users or service accounts fine-grained access to specific resources on a per-namespace basis. RBAC users upgrading from 1.5 to 1.6 should view the guidance [here](https://kubernetes.io//docs/admin/authorization/rbac.md#upgrading-from-15).&nbsp;  
+**Security and Setup** : Users concerned with security will find that [RBAC](https://kubernetes.io/docs/reference/access-authn-authz/rbac), now _beta_ adds a significant security benefit through more tightly scoped default roles for system components. The default RBAC policies in 1.6 grant scoped permissions to control-plane components, nodes, and controllers. RBAC allows cluster administrators to selectively grant particular users or service accounts fine-grained access to specific resources on a per-namespace basis. RBAC users upgrading from 1.5 to 1.6 should view the guidance [here](https://kubernetes.io/docs/reference/access-authn-authz/rbac#upgrading-from-1-5).&nbsp;

-Users looking for an easy way to provision a secure cluster on physical or cloud servers can use [kubeadm](https://kubernetes.io/docs/getting-started-guides/kubeadm/), which is now _beta_. kubeadm has been enhanced with a set of command line flags and a base feature set that includes RBAC setup, use of the [Bootstrap Token system](http://kubernetes.io/docs/admin/bootstrap-tokens/) and an enhanced [Certificates API](https://kubernetes.io/docs/tasks/tls/managing-tls-in-a-cluster/).  
+Users looking for an easy way to provision a secure cluster on physical or cloud servers can use [kubeadm](https://kubernetes.io/docs/getting-started-guides/kubeadm/), which is now _beta_. kubeadm has been enhanced with a set of command line flags and a base feature set that includes RBAC setup, use of the [Bootstrap Token system](http://kubernetes.io/docs/reference/access-authn-authz/bootstrap-tokens/) and an enhanced [Certificates API](https://kubernetes.io/docs/tasks/tls/managing-tls-in-a-cluster/).  

 **Advanced Scheduling** : This release adds a set of [powerful and versatile scheduling constructs](https://kubernetes.io/docs/user-guide/node-selection/) to give you greater control over how pods are scheduled, including rules to restrict pods to particular nodes in heterogeneous clusters, and rules to spread or pack pods across failure domains such as nodes, racks, and zones.  

@@ -13,7 +13,7 @@ The focus of this post is to highlight some of the interesting new capabilities

 **RBAC vs ABAC**  

-Currently there are several [authorization mechanisms](https://kubernetes.io/docs/admin/authorization/) available for use with Kubernetes. Authorizers are the mechanisms that decide who is permitted to make what changes to the cluster using the Kubernetes API. This affects things like kubectl, system components, and also certain applications that run in the cluster and manipulate the state of the cluster, like Jenkins with the Kubernetes plugin, or [Helm](https://github.com/kubernetes/helm) that runs in the cluster and uses the Kubernetes API to install applications in the cluster. Out of the available authorization mechanisms, ABAC and RBAC are the mechanisms local to a Kubernetes cluster that allow configurable permissions policies.  
+Currently there are several [authorization mechanisms](https://kubernetes.io/docs/reference/access-authn-authz/authorization/) available for use with Kubernetes. Authorizers are the mechanisms that decide who is permitted to make what changes to the cluster using the Kubernetes API. This affects things like kubectl, system components, and also certain applications that run in the cluster and manipulate the state of the cluster, like Jenkins with the Kubernetes plugin, or [Helm](https://github.com/kubernetes/helm) that runs in the cluster and uses the Kubernetes API to install applications in the cluster. Out of the available authorization mechanisms, ABAC and RBAC are the mechanisms local to a Kubernetes cluster that allow configurable permissions policies.

 ABAC, Attribute Based Access Control, is a powerful concept. However, as implemented in Kubernetes, ABAC is difficult to manage and understand. It requires ssh and root filesystem access on the master VM of the cluster to make authorization policy changes. For permission changes to take effect the cluster API server must be restarted.  

@@ -42,11 +42,11 @@ A RoleBinding maps a Role to a user or set of users, granting that Role's permis
 [![](https://1.bp.blogspot.com/-ixDe91-cnqw/WOa0auxC0mI/AAAAAAAABBs/4LxVsr6shEgTYqUapt5QPISUeuTuztVwwCEw/s640/rbac2.png)](https://1.bp.blogspot.com/-ixDe91-cnqw/WOa0auxC0mI/AAAAAAAABBs/4LxVsr6shEgTYqUapt5QPISUeuTuztVwwCEw/s1600/rbac2.png)  


-Additionally there are cluster roles and cluster role bindings to consider. Cluster roles and cluster role bindings function like roles and role bindings except they have wider scope. The exact differences and how cluster roles and cluster role bindings interact with roles and role bindings are covered in the [Kubernetes documentation](https://kubernetes.io/docs/admin/authorization/rbac/#rolebinding-and-clusterrolebinding).  
+Additionally there are cluster roles and cluster role bindings to consider. Cluster roles and cluster role bindings function like roles and role bindings except they have wider scope. The exact differences and how cluster roles and cluster role bindings interact with roles and role bindings are covered in the [Kubernetes documentation](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#rolebinding-and-clusterrolebinding).  

 **RBAC in Kubernetes**  

-RBAC is now deeply integrated into Kubernetes and used by the system components to grant the permissions necessary for them to function. [System roles](https://kubernetes.io/docs/admin/authorization/rbac/#default-roles-and-role-bindings) are typically prefixed with system: so they can be easily recognized.  
+RBAC is now deeply integrated into Kubernetes and used by the system components to grant the permissions necessary for them to function. [System roles](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#default-roles-and-role-bindings) are typically prefixed with system: so they can be easily recognized.  


 ```
@@ -76,7 +76,7 @@ system:controller:certificate-controller ClusterRole.v1beta1.rbac.authorization.

 The RBAC system roles have been expanded to cover the necessary permissions for running a Kubernetes cluster with RBAC only.  

-During the permission translation from ABAC to RBAC, some of the permissions that were enabled by default in many deployments of ABAC authorized clusters were identified as unnecessarily broad and were [scoped down](https://kubernetes.io/docs/admin/authorization/rbac/#upgrading-from-15) in RBAC. The area most likely to impact workloads on a cluster is the permissions available to service accounts. With the permissive ABAC configuration, requests from a pod using the pod mounted token to authenticate to the API server have broad authorization. As a concrete example, the curl command at the end of this sequence will return a JSON formatted result when ABAC is enabled and an error when only RBAC is enabled.  
+During the permission translation from ABAC to RBAC, some of the permissions that were enabled by default in many deployments of ABAC authorized clusters were identified as unnecessarily broad and were [scoped down](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#upgrading-from-1-5) in RBAC. The area most likely to impact workloads on a cluster is the permissions available to service accounts. With the permissive ABAC configuration, requests from a pod using the pod mounted token to authenticate to the API server have broad authorization. As a concrete example, the curl command at the end of this sequence will return a JSON formatted result when ABAC is enabled and an error when only RBAC is enabled.


 ```
@@ -96,13 +96,13 @@ During the permission translation from ABAC to RBAC, some of the permissions tha

 Any applications you run in your Kubernetes cluster that interact with the Kubernetes API have the potential to be affected by the permissions changes when transitioning from ABAC to RBAC.  

-To smooth the transition from ABAC to RBAC, you can create Kubernetes 1.6 clusters with both [ABAC and RBAC authorizers](https://kubernetes.io/docs/admin/authorization/rbac/#parallel-authorizers) enabled. When both ABAC and RBAC are enabled, authorization for a resource is granted if either authorization policy grants access. However, under that configuration the most permissive authorizer is used and it will not be possible to use RBAC to fully control permissions.  
+To smooth the transition from ABAC to RBAC, you can create Kubernetes 1.6 clusters with both [ABAC and RBAC authorizers](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#parallel-authorizers) enabled. When both ABAC and RBAC are enabled, authorization for a resource is granted if either authorization policy grants access. However, under that configuration the most permissive authorizer is used and it will not be possible to use RBAC to fully control permissions.  

 At this point, RBAC is complete enough that ABAC support should be considered deprecated going forward. It will still remain in Kubernetes for the foreseeable future but development attention is focused on RBAC.  



-Two different talks at the at the Google Cloud Next conference touched on RBAC related changes in Kubernetes 1.6, jump to the relevant parts [here](https://www.youtube.com/watch?v=Cd4JU7qzYbE#t=8m01s) and [here](https://www.youtube.com/watch?v=18P7cFc6nTU#t=41m06s). For more detailed information about using RBAC in Kubernetes 1.6 read the full [RBAC documentation](https://kubernetes.io/docs/admin/authorization/rbac/).
+Two different talks at the at the Google Cloud Next conference touched on RBAC related changes in Kubernetes 1.6, jump to the relevant parts [here](https://www.youtube.com/watch?v=Cd4JU7qzYbE#t=8m01s) and [here](https://www.youtube.com/watch?v=18P7cFc6nTU#t=41m06s). For more detailed information about using RBAC in Kubernetes 1.6 read the full [RBAC documentation](https://kubernetes.io/docs/reference/access-authn-authz/rbac/).


 **Get Involved**  
@@ -16,7 +16,7 @@ Also, for power users, API aggregation in this release allows user-provided apis
 Security:  

 - [The Network Policy API](https://kubernetes.io/docs/concepts/services-networking/network-policies/) is promoted to stable. Network policy, implemented through a network plug-in, allows users to set and enforce rules governing which pods can communicate with each other.&nbsp;
- [Node authorizer](https://kubernetes.io/docs/admin/authorization/node/) and admission control plugin are new additions that restrict kubelet’s access to secrets, pods and other objects based on its node.
+- [Node authorizer](https://kubernetes.io/docs/reference/access-authn-authz/node/) and admission control plugin are new additions that restrict kubelet’s access to secrets, pods and other objects based on its node.
 - [Encryption for Secrets](https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/), and other resources in etcd, is now available as alpha.&nbsp;
 - [Kubelet TLS bootstrapping](https://kubernetes.io/docs/admin/kubelet-tls-bootstrapping/) now supports client and server certificate rotation.
 - [Audit logs](https://kubernetes.io/docs/tasks/debug-application-cluster/audit/) stored by the API server are now more customizable and extensible with support for event filtering and webhooks. They also provide richer data for system audit.
@@ -36,7 +36,7 @@ Extensibility:

 Additional Features:  

- Alpha support for [external admission controllers](https://kubernetes.io/docs/admin/extensible-admission-controllers/) is introduced, providing two options for adding custom business logic to the API server for modifying objects as they are created and validating policy.&nbsp;
+- Alpha support for [external admission controllers](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/) is introduced, providing two options for adding custom business logic to the API server for modifying objects as they are created and validating policy.&nbsp;
 - [Policy-based Federated Resource Placement](https://kubernetes.io/docs/tasks/federation/set-up-placement-policies-federation/) is introduced as Alpha providing placement policies for the federated clusters, based on custom requirements such as regulation, pricing or performance.

 Deprecation:&nbsp;  
@@ -12,7 +12,7 @@ We’re pleased to announce the delivery of Kubernetes 1.8, our third release th

 ## Spotlight on security

-Kubernetes 1.8 graduates support for [role based access control](https://en.wikipedia.org/wiki/Role-based_access_control) (RBAC) to stable. RBAC allows cluster administrators to [dynamically define roles](https://kubernetes.io/docs/admin/authorization/rbac/) to enforce access policies through the Kubernetes API. Beta support for filtering outbound traffic through [network policies](https://kubernetes.io/docs/concepts/services-networking/network-policies/) augments existing support for filtering inbound traffic to a pod. RBAC and Network Policies are two powerful tools for enforcing organizational and regulatory security requirements within Kubernetes.  
+Kubernetes 1.8 graduates support for [role based access control](https://en.wikipedia.org/wiki/Role-based_access_control) (RBAC) to stable. RBAC allows cluster administrators to [dynamically define roles](https://kubernetes.io/docs/reference/access-authn-authz/rbac/) to enforce access policies through the Kubernetes API. Beta support for filtering outbound traffic through [network policies](https://kubernetes.io/docs/concepts/services-networking/network-policies/) augments existing support for filtering inbound traffic to a pod. RBAC and Network Policies are two powerful tools for enforcing organizational and regulatory security requirements within Kubernetes.  


 Transport Layer Security (TLS) [certificate rotation](https://kubernetes.io/docs/admin/kubelet-tls-bootstrapping/) for the Kubelet graduates to beta. Automatic certificate rotation eases secure cluster operation.  
@@ -6,7 +6,7 @@ url: /blog/2017/10/Using-Rbac-Generally-Available-18
 ---
 **_Editor's note: this post is part of a [series of in-depth articles](https://kubernetes.io/blog/2017/10/five-days-of-kubernetes-18) on what's new in Kubernetes 1.8. Today’s post comes from Eric Chiang, software engineer, CoreOS, and SIG-Auth co-lead._**  

-Kubernetes 1.8 represents a significant milestone for the [role-based access control (RBAC) authorizer](https://kubernetes.io/docs/admin/authorization/rbac/), which was promoted to GA in this release. RBAC is a mechanism for controlling access to the Kubernetes API, and since its [beta in 1.6](https://kubernetes.io/blog/2017/04/rbac-support-in-kubernetes), many Kubernetes clusters and provisioning strategies have enabled it by default.  
+Kubernetes 1.8 represents a significant milestone for the [role-based access control (RBAC) authorizer](https://kubernetes.io/docs/reference/access-authn-authz/rbac/), which was promoted to GA in this release. RBAC is a mechanism for controlling access to the Kubernetes API, and since its [beta in 1.6](https://kubernetes.io/blog/2017/04/rbac-support-in-kubernetes), many Kubernetes clusters and provisioning strategies have enabled it by default.  

 Going forward, we expect to see RBAC become a fundamental building block for securing Kubernetes clusters. This post explores using RBAC to manage user and application access to the Kubernetes API.  

@@ -5,7 +5,7 @@ slug: certified-kubernetes-conformance
 url: /blog/2017/11/Certified-Kubernetes-Conformance
 ---

-[![](https://1.bp.blogspot.com/-YasPeoIh8tA/Wg28rH4dzXI/AAAAAAAAAHg/Hfk2dnUoav4XMefGyjzMWdJMZbu1QJFagCK4BGAYYCw/s200/certified_kubernetes_color.png)](http://1.bp.blogspot.com/-YasPeoIh8tA/Wg28rH4dzXI/AAAAAAAAAHg/Hfk2dnUoav4XMefGyjzMWdJMZbu1QJFagCK4BGAYYCw/s1600/certified_kubernetes_color.png)This week the CNCFⓇ [certified the first group](https://www.cncf.io/announcement/2017/11/13/cloud-native-computing-foundation-launches-certified-kubernetes-program-32-conformant-distributions-platforms/) of KubernetesⓇ&nbsp;offerings under the [Certified Kubernetes Conformance Program](https://www.cncf.io/certification/software-conformance/). These first certifications follow a [beta phase](https://kubernetes.io/blog/2017/10/software-conformance-certification) during which we invited participants to submit conformance results. The community response was overwhelming: CNCF certified offerings from 32 vendors!  
+[![](https://1.bp.blogspot.com/-YasPeoIh8tA/Wg28rH4dzXI/AAAAAAAAAHg/Hfk2dnUoav4XMefGyjzMWdJMZbu1QJFagCK4BGAYYCw/s200/certified_kubernetes_color.png)](https://1.bp.blogspot.com/-YasPeoIh8tA/Wg28rH4dzXI/AAAAAAAAAHg/Hfk2dnUoav4XMefGyjzMWdJMZbu1QJFagCK4BGAYYCw/s1600/certified_kubernetes_color.png)This week the CNCFⓇ [certified the first group](https://www.cncf.io/announcement/2017/11/13/cloud-native-computing-foundation-launches-certified-kubernetes-program-32-conformant-distributions-platforms/) of KubernetesⓇ&nbsp;offerings under the [Certified Kubernetes Conformance Program](https://www.cncf.io/certification/software-conformance/). These first certifications follow a [beta phase](https://kubernetes.io/blog/2017/10/software-conformance-certification) during which we invited participants to submit conformance results. The community response was overwhelming: CNCF certified offerings from 32 vendors!  

 The new Certified Kubernetes Conformance Program gives enterprise organizations the confidence that workloads running on any Certified Kubernetes distribution or platform will work correctly on other Certified Kubernetes distributions or platforms. A Certified Kubernetes product guarantees that the complete Kubernetes API functions as specified, so users can rely on a seamless, stable experience.  

@@ -71,7 +71,7 @@ Once you log in, all of your clusters are available within Codefresh.


 ### Add Cluster
-To add your cluster, click the down arrow, and then click **add cluste** r, select the project and cluster name. You can now deploy images!
+To add your cluster, click the down arrow, and then click **add cluster**, select the project and cluster name. You can now deploy images!



@@ -13,7 +13,7 @@ On October 12th, Google and partners [announced](https://cloudplatform.googleblo
 Grafeas allows build, auditing and compliance tools to exchange comprehensive metadata on container images using a central API. This allows enforcing policies that provide central control over the software supply process.  


-[![](https://2.bp.blogspot.com/-TDD4slMA7gg/WfzDeKVLr2I/AAAAAAAAAGw/dhfWOrCMdmogSNhGr5RrA2ovr02K5nn8ACK4BGAYYCw/s400/Screen%2BShot%2B2017-11-03%2Bat%2B12.28.13%2BPM.png)](http://2.bp.blogspot.com/-TDD4slMA7gg/WfzDeKVLr2I/AAAAAAAAAGw/dhfWOrCMdmogSNhGr5RrA2ovr02K5nn8ACK4BGAYYCw/s1600/Screen%2BShot%2B2017-11-03%2Bat%2B12.28.13%2BPM.png)  
+[![](https://2.bp.blogspot.com/-TDD4slMA7gg/WfzDeKVLr2I/AAAAAAAAAGw/dhfWOrCMdmogSNhGr5RrA2ovr02K5nn8ACK4BGAYYCw/s400/Screen%2BShot%2B2017-11-03%2Bat%2B12.28.13%2BPM.png)](https://2.bp.blogspot.com/-TDD4slMA7gg/WfzDeKVLr2I/AAAAAAAAAGw/dhfWOrCMdmogSNhGr5RrA2ovr02K5nn8ACK4BGAYYCw/s1600/Screen%2BShot%2B2017-11-03%2Bat%2B12.28.13%2BPM.png)  


 ## Example application: PaymentProcessor
@@ -24,13 +24,13 @@ Let’s consider a simple application, _PaymentProcessor_, that retrieves, proce
 Due to the sensitive nature of the payment data, the developers and DevOps team really want to make sure that the code meets certain security and compliance requirements, with detailed records on the provenance of this code. There are CI/CD stages that validate the quality of the PaymentProcessor release, but there is no easy way to centrally view/manage this information:


-[![](https://1.bp.blogspot.com/-WeI6zpGd42A/WfzDkkIonFI/AAAAAAAAAG4/wKUaNaXYvaQ-an9p4_9T9J3EQB_zHkRXwCK4BGAYYCw/s1600/Screen%2BShot%2B2017-11-03%2Bat%2B12.28.23%2BPM.png)](http://1.bp.blogspot.com/-WeI6zpGd42A/WfzDkkIonFI/AAAAAAAAAG4/wKUaNaXYvaQ-an9p4_9T9J3EQB_zHkRXwCK4BGAYYCw/s1600/Screen%2BShot%2B2017-11-03%2Bat%2B12.28.23%2BPM.png)  
+[![](https://1.bp.blogspot.com/-WeI6zpGd42A/WfzDkkIonFI/AAAAAAAAAG4/wKUaNaXYvaQ-an9p4_9T9J3EQB_zHkRXwCK4BGAYYCw/s1600/Screen%2BShot%2B2017-11-03%2Bat%2B12.28.23%2BPM.png)](https://1.bp.blogspot.com/-WeI6zpGd42A/WfzDkkIonFI/AAAAAAAAAG4/wKUaNaXYvaQ-an9p4_9T9J3EQB_zHkRXwCK4BGAYYCw/s1600/Screen%2BShot%2B2017-11-03%2Bat%2B12.28.23%2BPM.png)  


 ## Visibility and governance over the PaymentProcessor Code
 Grafeas provides an API for customers to centrally manage metadata created by various CI/CD components and enables deploy time policy enforcement through a Kritis implementation.  

-[![](https://4.bp.blogspot.com/-SRMfm5z606M/WfzDpHqlz-I/AAAAAAAAAHA/y2suaInhr9E0hU0u78PacBT_kZj2D7DKgCK4BGAYYCw/s1600/Screen%2BShot%2B2017-11-03%2Bat%2B12.28.34%2BPM.png)](http://4.bp.blogspot.com/-SRMfm5z606M/WfzDpHqlz-I/AAAAAAAAAHA/y2suaInhr9E0hU0u78PacBT_kZj2D7DKgCK4BGAYYCw/s1600/Screen%2BShot%2B2017-11-03%2Bat%2B12.28.34%2BPM.png)  
+[![](https://4.bp.blogspot.com/-SRMfm5z606M/WfzDpHqlz-I/AAAAAAAAAHA/y2suaInhr9E0hU0u78PacBT_kZj2D7DKgCK4BGAYYCw/s1600/Screen%2BShot%2B2017-11-03%2Bat%2B12.28.34%2BPM.png)](https://4.bp.blogspot.com/-SRMfm5z606M/WfzDpHqlz-I/AAAAAAAAAHA/y2suaInhr9E0hU0u78PacBT_kZj2D7DKgCK4BGAYYCw/s1600/Screen%2BShot%2B2017-11-03%2Bat%2B12.28.34%2BPM.png)  


 Let’s consider a basic example of how Grafeas can provide deploy time control for the PaymentProcessor app using a demo verification pipeline.  
@@ -11,9 +11,9 @@ The admission stage of API server processing is one of the most powerful tools f


 ## What is Admission?
-[Admission](https://kubernetes.io/docs/admin/admission-controllers/#what-are-they) is the phase of [handling an API server request](https://blog.openshift.com/kubernetes-deep-dive-api-server-part-1/) that happens before a resource is persisted, but after authorization. Admission gets access to the same information as authorization (user, URL, etc) and the complete body of an API request (for most requests).  
+[Admission](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#what-are-they) is the phase of [handling an API server request](https://blog.openshift.com/kubernetes-deep-dive-api-server-part-1/) that happens before a resource is persisted, but after authorization. Admission gets access to the same information as authorization (user, URL, etc) and the complete body of an API request (for most requests).  

-[![](https://2.bp.blogspot.com/-p8WGg2BATsY/WlfywbD_tAI/AAAAAAAAAJw/mDqZV0dB4_Y0gXXQp_1tQ7CtMRSd6lHVwCK4BGAYYCw/s640/Screen%2BShot%2B2018-01-11%2Bat%2B3.22.07%2BPM.png)](http://2.bp.blogspot.com/-p8WGg2BATsY/WlfywbD_tAI/AAAAAAAAAJw/mDqZV0dB4_Y0gXXQp_1tQ7CtMRSd6lHVwCK4BGAYYCw/s1600/Screen%2BShot%2B2018-01-11%2Bat%2B3.22.07%2BPM.png)  
+[![](https://2.bp.blogspot.com/-p8WGg2BATsY/WlfywbD_tAI/AAAAAAAAAJw/mDqZV0dB4_Y0gXXQp_1tQ7CtMRSd6lHVwCK4BGAYYCw/s640/Screen%2BShot%2B2018-01-11%2Bat%2B3.22.07%2BPM.png)](https://2.bp.blogspot.com/-p8WGg2BATsY/WlfywbD_tAI/AAAAAAAAAJw/mDqZV0dB4_Y0gXXQp_1tQ7CtMRSd6lHVwCK4BGAYYCw/s1600/Screen%2BShot%2B2018-01-11%2Bat%2B3.22.07%2BPM.png)  

 The admission phase is composed of individual plugins, each of which are narrowly focused and have semantic knowledge of what they are inspecting. Examples include: PodNodeSelector (influences scheduling decisions), PodSecurityPolicy (prevents escalating containers), and ResourceQuota (enforces resource allocation per namespace).  

@@ -17,7 +17,7 @@ This blog post is one of a number of efforts to make client-go more accessible t
 The following API group promotions are part of Kubernetes 1.9:  

 - Workload objects (Deployments, DaemonSets, ReplicaSets, and StatefulSets) have been [promoted to the apps/v1 API group in Kubernetes 1.9](https://kubernetes.io/docs/reference/workloads-18-19/). client-go follows this transition and allows developers to use the latest version by importing the k8s.io/api/apps/v1 package instead of k8s.io/api/apps/v1beta1 and by using Clientset.AppsV1().
- Admission Webhook Registration has been promoted to the admissionregistration.k8s.io/v1beta1 API group in Kubernetes 1.9. The former ExternalAdmissionHookConfiguration type has been replaced by the incompatible ValidatingWebhookConfiguration and MutatingWebhookConfiguration types. Moreover, the webhook admission payload type AdmissionReview in admission.k8s.io has been promoted to v1beta1. Note that versioned objects are now passed to webhooks. Refer to the admission webhook [documentation](https://kubernetes.io/docs/admin/extensible-admission-controllers/#external-admission-webhooks) for details.
+- Admission Webhook Registration has been promoted to the admissionregistration.k8s.io/v1beta1 API group in Kubernetes 1.9. The former ExternalAdmissionHookConfiguration type has been replaced by the incompatible ValidatingWebhookConfiguration and MutatingWebhookConfiguration types. Moreover, the webhook admission payload type AdmissionReview in admission.k8s.io has been promoted to v1beta1. Note that versioned objects are now passed to webhooks. Refer to the admission webhook [documentation](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#admission-webhooks) for details.



@@ -84,12 +84,12 @@ spec.version in body should be one of [v1.0.0 v1.0.1]



-Note that with [Admission Webhooks](https://kubernetes.io/docs/admin/extensible-admission-controllers/#external-admission-webhooks), Kubernetes 1.9 provides another beta feature to validate objects before they are created or updated. Starting with 1.9, these webhooks also allow mutation of objects (for example, to set defaults or to inject values). Of course, webhooks work with CRDs as well. Moreover, webhooks can be used to implement validations that are not easily expressible with CRD validation. Note that webhooks are harder to implement than CRD validation, so for many purposes, CRD validation is the right tool.
+Note that with [Admission Webhooks](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#admission-webhooks), Kubernetes 1.9 provides another beta feature to validate objects before they are created or updated. Starting with 1.9, these webhooks also allow mutation of objects (for example, to set defaults or to inject values). Of course, webhooks work with CRDs as well. Moreover, webhooks can be used to implement validations that are not easily expressible with CRD validation. Note that webhooks are harder to implement than CRD validation, so for many purposes, CRD validation is the right tool.



 ## Creating namespaced informers
-Often objects in one namespace or only with certain labels are to be processed in a controller. Informers [now allow](https://github.com/kubernetes/kubernetes/pull/54660) you to tweak the ListOptions used to query the API server to list and watch objects. Uninitialized objects (for consumption by [initializers](https://kubernetes.io/docs/admin/extensible-admission-controllers/#what-are-initializers)) can be made visible by setting IncludeUnitialized to true. All this can be done using the new NewFilteredSharedInformerFactory constructor for shared informers:  
+Often objects in one namespace or only with certain labels are to be processed in a controller. Informers [now allow](https://github.com/kubernetes/kubernetes/pull/54660) you to tweak the ListOptions used to query the API server to list and watch objects. Uninitialized objects (for consumption by [initializers](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#what-are-initializers)) can be made visible by setting IncludeUnitialized to true. All this can be done using the new NewFilteredSharedInformerFactory constructor for shared informers:  

 ```

@@ -26,7 +26,7 @@ The number of respondents in the past year and 10 months increased at a rate of

 ![Minikube operating system usage](/images/blog/survey-results/2018-application-survey/minikube-os-usage.png)

-Minikube is used primarily be people on MacOS and Linux. Yet, according to the 2018 Stack Overflow survey, [almost half of developers use Windows as their primary operating system](https://insights.stackoverflow.com/survey/2018/#technology-developers-primary-operating-systems). This is where Minikube would run.
+Minikube is used primarily by people on macOS and Linux. Yet, according to the 2018 Stack Overflow survey, [almost half of developers use Windows as their primary operating system](https://insights.stackoverflow.com/survey/2018/#technology-developers-primary-operating-systems). This is where Minikube would run.

 Seeing differences from other data sets is worth looking more deeply at to better understand our audience, where Kubernetes is at, and where it is on the journey it's headed.

@@ -0,0 +1,24 @@
+---
+layout: blog
+title: Meet Our Contributors - Monthly Streaming YouTube Mentoring Series
+date: 2018-07-10
+---
+
+**Author**: Paris Pittman (Google)
+
+![meet_our_contributors](/images/blog/2018-06-05-meet-our-contributors-youtube-mentoring-series/meet-our-contributors.png)
+
+July 11th at 2:30pm and 8pm UTC kicks off our next installment of Meet Our Contributors YouTube series. This month is special: members of the steering committee will be on to answer any and all questions from the community on the first 30 minutes of the 8pm UTC session. More on submitting questions below.
+
+[Meet Our Contributors](https://github.com/kubernetes/community/blob/master/mentoring/meet-our-contributors.md) was created to give an opportunity to new and current contributors alike to get time in front of our upstream community to ask questions that you would typically ask a mentor. We have 3-6 contributors on each session (an AM and PM session depending on where you are in the world!) answer questions [live on a YouTube stream](https://www.youtube.com/c/KubernetesCommunity/live). If you miss it, don’t stress, the recording is up after it’s over. Check out a past episode [here](https://www.youtube.com/watch?v=EVsXi3Zhlo0&list=PL69nYSiGNLP3QpQrhZq_sLYo77BVKv09F).
+
+As you can imagine, the questions span broadly from introductory - “what’s a SIG?” to more advanced - “why’s my test flaking?” You’ll also hear growth related advice questions such as “what’s my best path to becoming an approver?” We’re happy to do a live code/docs review or explain part of the codebase as long as we have a few days notice.
+
+We answer at least 10 questions per session and have helped 500+ people to date. This is a scalable mentoring initiative that makes it easy for all parties to share information, get advice, and get going with what they are trying to accomplish. We encourage you to submit questions for our next session:
+
+- Join the Kubernetes Slack channel - #meet-our-contributors - to ask your question or for more detailed information. DM paris@ if you would like to remain anonymous.
+- Twitter works, too, with the hashtag #k8smoc
+
+If you are contributor reading this that has wanted to mentor but just can’t find the time - this is for you! [Reach out to us](https://goo.gl/forms/ZcnFiqNR5EQH03zm2).
+
+You can join us live on June 6th at 2:30pm and 8pm UTC, and every first Wednesday of the month, on the [Kubernetes Community live stream](https://www.youtube.com/c/KubernetesCommunity/live). We look forward to seeing you there!
@@ -49,19 +49,18 @@ Each Special Interest Group (SIG) within the community continues to deliver the

 ## Availability

-Kubernetes 1.11 is available for [download on GitHub](https://github.com/kubernetes/kubernetes/releases/tag/v1.11.0). To get started with Kubernetes, check out these [interactive tutorials](https://kubernetes.io/docs/tutorials/). 
+Kubernetes 1.11 is available for [download on GitHub](https://github.com/kubernetes/kubernetes/releases/tag/v1.11.0). To get started with Kubernetes, check out these [interactive tutorials](https://kubernetes.io/docs/tutorials/).

 You can also install 1.11 using Kubeadm. Version 1.11.0 will be available as Deb and RPM packages, installable using the [Kubeadm cluster installer](https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/) sometime on June 28th.

-## 5 Day Features Blog Series
+## 4 Day Features Blog Series

-If you’re interested in exploring these features more in depth, check back in two weeks for our 5 Days of Kubernetes series where we’ll highlight detailed walkthroughs of the following features:
+If you’re interested in exploring these features more in depth, check back in two weeks for our 4 Days of Kubernetes series where we’ll highlight detailed walkthroughs of the following features:

-* Day 1: IPVS-Based In-Cluster Service Load Balancing Graduates to General Availability
-* Day 2: CoreDNS Promoted to General Availability
-* Day 3: Dynamic Kubelet Configuration Moves to Beta
-* Day 4: Custom Resource Definitions Can Now Define Multiple Versions
-* Day 5: Overview of CSI Enhancements
+* Day 1: [IPVS-Based In-Cluster Service Load Balancing Graduates to General Availability](/blog/2018/07/09/ipvs-based-in-cluster-load-balancing-deep-dive/)
+* Day 2: [CoreDNS Promoted to General Availability](/blog/2018/07/10/coredns-ga-for-kubernetes-cluster-dns/)
+* Day 3: [Dynamic Kubelet Configuration Moves to Beta](/blog/2018/07/11/dynamic-kubelet-configuration/)
+* Day 4: [Resizing Persistent Volumes using Kubernetes](/blog/2018/07/12/resizing-persistent-volumes-using-kubernetes/)

 ## Release team

@@ -93,7 +92,7 @@ Is Kubernetes helping your team? Share your story with the community.

 ## KubeCon

-The world’s largest Kubernetes gathering, KubeCon + CloudNativeCon is coming to [Shanghai](https://www.lfasiallc.com/events/kubecon-cloudnativecon-china-2018/) from November 14-15, 2018 and [Seattle](https://events.linuxfoundation.org/events/kubecon-cloudnativecon-north-america-2018/) from December 11-13, 2018. This conference will feature technical sessions, case studies, developer deep dives, salons and more! The CFP for both event is currently open. [Submit your talk](https://events.linuxfoundation.org/events/kubecon-cloudnativecon-north-america-2018/program/call-for-proposals-cfp/) and [register](https://events.linuxfoundation.org/events/kubecon-cloudnativecon-europe-2018/attend/register/) today!
+The world’s largest Kubernetes gathering, KubeCon + CloudNativeCon is coming to [Shanghai](https://events.linuxfoundation.cn/events/kubecon-cloudnativecon-china-2018/ from November 14-15, 2018 and [Seattle](https://events.linuxfoundation.org/events/kubecon-cloudnativecon-north-america-2018/) from December 11-13, 2018. This conference will feature technical sessions, case studies, developer deep dives, salons and more! The CFP for both event is currently open. [Submit your talk](https://events.linuxfoundation.org/events/kubecon-cloudnativecon-north-america-2018/program/call-for-proposals-cfp/) and [register](https://www.regonline.com/registration/Checkin.aspx?EventID=2246960) today!

 ## Webinar

@@ -0,0 +1,204 @@
+---
+layout: blog
+title:  'Airflow on Kubernetes (Part 1): A Different Kind of Operator'
+date:   2018-06-28
+---
+
+**Author**: Daniel Imberman (Bloomberg LP)
+
+## Introduction
+
+As part of Bloomberg's [continued commitment to developing the Kubernetes ecosystem](https://www.techatbloomberg.com/blog/bloomberg-awarded-first-cncf-end-user-award-contributions-kubernetes/), we are excited to announce the Kubernetes Airflow Operator; a mechanism for [Apache Airflow](https://airflow.apache.org/), a popular workflow orchestration framework to natively launch arbitrary Kubernetes Pods using the Kubernetes API.
+
+## What Is Airflow?
+
+Apache Airflow is one realization of the DevOps philosophy of "Configuration As Code." Airflow allows users to launch multi-step pipelines using a simple Python object DAG (Directed Acyclic Graph). You can define dependencies, programmatically construct complex workflows, and monitor scheduled jobs in an easy to read UI.
+
+<img src="/images/blog/2018-05-25-Airflow-Kubernetes-Operator/2018-05-25-airflow_dags.png" width="85%" alt="Airflow DAGs" />
+<img src="/images/blog/2018-05-25-Airflow-Kubernetes-Operator/2018-05-25-airflow.png" width="85%" alt="Airflow UI" />
+
+## Why Airflow on Kubernetes?
+
+Since its inception, Airflow's greatest strength has been its flexibility. Airflow offers a wide range of integrations for services ranging from Spark and HBase, to services on various cloud providers. Airflow also offers easy extensibility through its plug-in framework. However, one limitation of the project is that Airflow users are confined to the frameworks and clients that exist on the Airflow worker at the moment of execution. A single organization can have varied Airflow workflows ranging from data science pipelines to application deployments. This difference in use-case creates issues in dependency management as both teams might use vastly different libraries for their workflows.
+
+To address this issue, we've utilized Kubernetes to allow users to launch arbitrary Kubernetes pods and configurations. Airflow users can now have full power over their run-time environments, resources, and secrets, basically turning Airflow into an "any job you want" workflow orchestrator.
+
+
+## The Kubernetes Operator
+
+Before we move any further, we should clarify that an [Operator](https://airflow.apache.org/concepts.html#operators) in Airflow is a task definition. When a user creates a DAG, they would use an operator like the "SparkSubmitOperator" or the "PythonOperator" to submit/monitor a Spark job or a Python function respectively. Airflow comes with built-in operators for frameworks like Apache Spark, BigQuery, Hive, and EMR. It also offers a Plugins entrypoint that allows DevOps engineers to develop their own connectors.
+
+Airflow users are always looking for ways to make deployments and ETL pipelines simpler to manage. Any opportunity to decouple pipeline steps, while increasing monitoring, can reduce future outages and fire-fights. The following is a list of benefits provided by the Airflow Kubernetes Operator:
+
+ * **Increased flexibility for deployments:**  
+Airflow's plugin API has always offered a significant boon to engineers wishing to test new functionalities within their DAGs. On the downside, whenever a developer wanted to create a new operator, they had to develop an entirely new plugin. Now, any task that can be run within a Docker container is accessible through the exact same operator, with no extra Airflow code to maintain.
+
+ * **Flexibility of configurations and dependencies:**
+For operators that are run within static Airflow workers, dependency management can become quite difficult. If a developer wants to run one task that requires [SciPy](https://www.scipy.org) and another that requires [NumPy](http://www.numpy.org), the developer would have to either maintain both dependencies within all Airflow workers or offload the task to an external machine (which can cause bugs if that external machine changes in an untracked manner). Custom Docker images allow users to ensure that the tasks environment, configuration, and dependencies are completely idempotent.  
+
+ * **Usage of kubernetes secrets for added security:**
+Handling sensitive data is a core responsibility of any DevOps engineer. At every opportunity, Airflow users want to isolate any API keys, database passwords, and login credentials on a strict need-to-know basis. With the Kubernetes operator, users can utilize the Kubernetes Vault technology to store all sensitive data. This means that the Airflow workers will never have access to this information, and can simply request that pods be built with only the secrets they need.
+
+
+# Architecture
+
+<img src="/images/blog/2018-05-25-Airflow-Kubernetes-Operator/2018-05-25-airflow-architecture.png" width="85%" alt="Airflow Architecture" />
+
+The Kubernetes Operator uses the [Kubernetes Python Client](https://github.com/kubernetes-client/Python) to generate a request that is processed by the APIServer (1). Kubernetes will then launch your pod with whatever specs you've defined (2). Images will be loaded with all the necessary environment variables, secrets and dependencies, enacting a single command. Once the job is launched, the operator only needs to monitor the health of track logs (3). Users will have the choice of gathering logs locally to the scheduler or to any distributed logging service currently in their Kubernetes cluster.
+
+# Using the Kubernetes Operator
+
+## A Basic Example
+
+The following DAG is probably the simplest example we could write to show how the Kubernetes Operator works. This DAG creates two pods on Kubernetes: a Linux distro with Python and a base Ubuntu distro without it. The Python pod will run the Python request correctly, while the one without Python will report a failure to the user. If the Operator is working correctly, the `passing-task` pod should complete, while the `failing-task` pod returns a failure to the Airflow webserver.
+
+
+
+```Python
+from airflow import DAG
+from datetime import datetime, timedelta
+from airflow.contrib.operators.kubernetes_pod_operator import KubernetesPodOperator
+from airflow.operators.dummy_operator import DummyOperator
+
+
+default_args = {
+    'owner': 'airflow',
+    'depends_on_past': False,
+    'start_date': datetime.utcnow(),
+    'email': ['airflow@example.com'],
+    'email_on_failure': False,
+    'email_on_retry': False,
+    'retries': 1,
+    'retry_delay': timedelta(minutes=5)
+}
+
+dag = DAG(
+    'kubernetes_sample', default_args=default_args, schedule_interval=timedelta(minutes=10))
+
+
+start = DummyOperator(task_id='run_this_first', dag=dag)
+
+passing = KubernetesPodOperator(namespace='default',
+                          image="Python:3.6",
+                          cmds=["Python","-c"],
+                          arguments=["print('hello world')"],
+                          labels={"foo": "bar"},
+                          name="passing-test",
+                          task_id="passing-task",
+                          get_logs=True,
+                          dag=dag
+                          )
+
+failing = KubernetesPodOperator(namespace='default',
+                          image="ubuntu:1604",
+                          cmds=["Python","-c"],
+                          arguments=["print('hello world')"],
+                          labels={"foo": "bar"},
+                          name="fail",
+                          task_id="failing-task",
+                          get_logs=True,
+                          dag=dag
+                          )
+
+passing.set_upstream(start)
+failing.set_upstream(start)
+```
+<img src="/images/blog/2018-05-25-Airflow-Kubernetes-Operator/2018-05-25-basic-dag-run.png" width="85%" alt="Basic DAG Run" />
+
+## But how does this relate to my workflow?
+
+While this example only uses basic images, the magic of Docker is that this same DAG will work for any image/command pairing you want. The following is a recommended CI/CD pipeline to run production-ready code on an Airflow DAG.
+
+### 1: PR in github
+Use Travis or Jenkins to run unit and integration tests, bribe your favorite team-mate into PR'ing your code, and merge to the master branch to trigger an automated CI build.
+
+### 2: CI/CD via Jenkins -> Docker Image
+
+[Generate your Docker images and bump release version within your Jenkins build](https://getintodevops.com/blog/building-your-first-docker-image-with-jenkins-2-guide-for-developers).
+
+### 3: Airflow launches task
+
+Finally, update your DAGs to reflect the new release version and you should be ready to go!
+
+```Python
+production_task = KubernetesPodOperator(namespace='default',
+                          # image="my-production-job:release-1.0.1", <-- old release
+                          image="my-production-job:release-1.0.2",
+                          cmds=["Python","-c"],
+                          arguments=["print('hello world')"],
+                          name="fail",
+                          task_id="failing-task",
+                          get_logs=True,
+                          dag=dag
+                          )
+```
+
+# Launching a test deployment
+
+Since the Kubernetes Operator is not yet released, we haven't released an official [helm](https://helm.sh/) chart or operator (however both are currently in progress). However, we are including instructions for a basic deployment below  and are actively looking for foolhardy beta testers to try this new feature. To try this system out please follow these steps:
+
+## Step 1: Set your kubeconfig to point to a kubernetes cluster
+
+## Step 2: Clone the Airflow Repo:
+
+Run `git clone https://github.com/apache/incubator-airflow.git` to clone the official Airflow repo.
+
+## Step 3: Run
+
+To run this basic deployment, we are co-opting the integration testing script that we currently use for the Kubernetes Executor (which will be explained in the next article of this series). To launch this deployment, run these three commands:
+
+```
+sed -ie "s/KubernetesExecutor/LocalExecutor/g" scripts/ci/kubernetes/kube/configmaps.yaml
+./scripts/ci/kubernetes/Docker/build.sh
+./scripts/ci/kubernetes/kube/deploy.sh
+```
+
+Before we move on, let's discuss what these commands are doing:
+
+### sed -ie "s/KubernetesExecutor/LocalExecutor/g" scripts/ci/kubernetes/kube/configmaps.yaml
+
+The Kubernetes Executor is another Airflow feature that allows for dynamic allocation  of tasks as idempotent pods. The reason we are switching this to the LocalExecutor is simply to introduce one feature at a time. You are more then welcome to skip this step if you would like to try the Kubernetes Executor, however we will go into more detail in a future article.
+
+### ./scripts/ci/kubernetes/Docker/build.sh
+
+This script will tar the Airflow master source code build a Docker container based on the Airflow distribution
+
+### ./scripts/ci/kubernetes/kube/deploy.sh
+
+Finally, we create a full Airflow deployment on your cluster. This includes Airflow configs, a postgres backend, the webserver + scheduler, and all necessary services between. One thing to note is that the role binding supplied is a cluster-admin, so if you do not have that level of permission on the cluster, you can modify this at scripts/ci/kubernetes/kube/airflow.yaml
+
+## Step 4: Log into your webserver
+
+Now that your Airflow instance is running let's take a look at the UI! The UI lives in port 8080 of the Airflow pod, so simply run
+
+```
+WEB=$(kubectl get pods -o go-template --template '{{range .items}}{{.metadata.name}}{{"\n"}}{{end}}' | grep "airflow" | head -1)
+kubectl port-forward $WEB 8080:8080
+ ```
+
+ Now the Airflow UI will exist on http://localhost:8080. To log in simply enter `airflow`/`airflow` and you should have full access to the Airflow web UI.
+
+## Step 5: Upload a test document
+
+To modify/add your own DAGs, you can use `kubectl cp` to upload local files into the DAG folder of the Airflow scheduler. Airflow will then read the new DAG and automatically upload it to its system. The following command will upload any local file into the correct directory:
+
+`kubectl cp <local file> <namespace>/<pod>:/root/airflow/dags -c scheduler`
+
+## Step 6: Enjoy!
+
+# So when will I be able to use this?
+
+ While this feature is still in the early stages, we hope to see it released for wide release in the next few months.
+
+# Get Involved
+
+This feature is just the beginning of multiple major efforts to improves Apache Airflow integration into Kubernetes. The Kubernetes Operator has been merged into the [1.10 release branch of Airflow](https://github.com/apache/incubator-airflow/tree/v1-10-test) (the executor in experimental mode), along with a fully k8s native scheduler called the Kubernetes Executor (article to come). These features are still in a stage where early adopters/contributers can have a huge influence on the future of these features.
+
+For those interested in joining these efforts, I'd recommend checkint out these steps:
+
+ * Join the airflow-dev mailing list at dev@airflow.apache.org.
+ * File an issue in [Apache Airflow JIRA](https://issues.apache.org/jira/projects/AIRFLOW/issues/)
+ * Join our SIG-BigData meetings on Wednesdays at 10am PST.
+ * Reach us on slack at #sig-big-data on kubernetes.slack.com
+
+Special thanks to the Apache Airflow and Kubernetes communities, particularly Grant Nicholas, Ben Goldberg, Anirudh Ramanathan, Fokko Dreisprong, and Bolke de Bruin, for your awesome help on these features as well as our future efforts.
@@ -0,0 +1,181 @@
+---
+layout: blog
+title:  'IPVS-Based In-Cluster Load Balancing Deep Dive'
+date:   2018-07-09
+---
+
+**Author**: Jun Du(Huawei), Haibin Xie(Huawei), Wei Liang(Huawei)
+
+**Editor’s note: this post is part of a [series of in-depth articles](https://kubernetes.io/blog/2018/06/27/kubernetes-1.11-release-announcement/) on what’s new in Kubernetes 1.11**
+
+## Introduction
+
+Per [the Kubernetes 1.11 release blog post ](https://kubernetes.io/blog/2018/06/27/kubernetes-1.11-release-announcement/), we announced that IPVS-Based In-Cluster Service Load Balancing graduates to General Availability. In this blog, we will take you through a deep dive of the feature.
+
+## What Is IPVS?
+
+**IPVS** (**IP Virtual Server**) is built on top of the Netfilter and implements transport-layer load balancing as part of the Linux kernel.
+
+IPVS is incorporated into the LVS (Linux Virtual Server), where it runs on a host and acts as a load balancer in front of a cluster of real servers. IPVS can direct requests for TCP- and UDP-based services to the real servers, and make services of the real servers appear as virtual services on a single IP address. Therefore, IPVS naturally supports Kubernetes Service.
+
+## Why IPVS for Kubernetes?
+
+As Kubernetes grows in usage, the scalability of its resources becomes more and more important. In particular, the scalability of services is paramount to the adoption of Kubernetes by developers/companies running large workloads.
+
+Kube-proxy, the building block of service routing has relied on the battle-hardened iptables to implement the core supported Service types such as ClusterIP and NodePort. However, iptables struggles to scale to tens of thousands of Services because it is designed purely for firewalling purposes and is based on in-kernel rule lists.
+
+Even though Kubernetes already support 5000 nodes in release v1.6, the kube-proxy with iptables is actually a bottleneck to scale the cluster to 5000 nodes. One example is that with NodePort Service in a 5000-node cluster, if we have 2000 services and each services have 10 pods, this will cause at least 20000 iptable records on each worker node, and this can make the kernel pretty busy.
+
+On the other hand, using IPVS-based in-cluster service load balancing can help a lot for such cases. IPVS is specifically designed for load balancing and uses more efficient data structures (hash tables) allowing for almost unlimited scale under the hood.
+
+## IPVS-based Kube-proxy
+
+### Parameter Changes
+
+**Parameter: --proxy-mode** In addition to existing userspace and iptables modes, IPVS mode is configured via `--proxy-mode=ipvs`. It implicitly uses IPVS NAT mode for service port mapping.
+
+**Parameter: --ipvs-scheduler**
+
+A new kube-proxy parameter has been added to specify the IPVS load balancing algorithm, with the parameter being `--ipvs-scheduler`. If it’s not configured, then round-robin (rr) is the default value.
+
+- rr: round-robin
+- lc: least connection
+- dh: destination hashing
+- sh: source hashing
+- sed: shortest expected delay
+- nq: never queue
+
+In the future, we can implement Service specific scheduler (potentially via annotation), which has higher priority and overwrites the value.
+
+**Parameter: `--cleanup-ipvs`** Similar to the `--cleanup-iptables` parameter, if true, cleanup IPVS configuration and IPTables rules that are created in IPVS mode.
+
+**Parameter: `--ipvs-sync-period`** Maximum interval of how often IPVS rules are refreshed (e.g. '5s', '1m'). Must be greater than 0.
+
+**Parameter: `--ipvs-min-sync-period`** Minimum interval of how often the IPVS rules are refreshed (e.g. '5s', '1m'). Must be greater than 0.
+
+**Parameter: `--ipvs-exclude-cidrs`**  A comma-separated list of CIDR's which the IPVS proxier should not touch when cleaning up IPVS rules because IPVS proxier can't distinguish kube-proxy created IPVS rules from user original IPVS rules. If you are using IPVS proxier with your own IPVS rules in the environment, this parameter should be specified, otherwise your original rule will be cleaned.
+
+### Design Considerations
+
+#### IPVS Service Network Topology
+
+When creating a ClusterIP type Service, IPVS proxier will do the following three things:
+
+* Make sure a dummy interface exists in the node, defaults to kube-ipvs0
+* Bind Service IP addresses to the dummy interface
+* Create IPVS virtual servers for each Service IP address respectively
+
+Here comes an example:
+
+```
+# kubectl describe svc nginx-service
+Name:			nginx-service
+...
+Type:			ClusterIP
+IP:			    10.102.128.4
+Port:			http	3080/TCP
+Endpoints:		10.244.0.235:8080,10.244.1.237:8080
+Session Affinity:	None
+
+# ip addr
+...
+73: kube-ipvs0: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN qlen 1000
+    link/ether 1a:ce:f5:5f:c1:4d brd ff:ff:ff:ff:ff:ff
+    inet 10.102.128.4/32 scope global kube-ipvs0
+       valid_lft forever preferred_lft forever
+
+# ipvsadm -ln
+IP Virtual Server version 1.2.1 (size=4096)
+Prot LocalAddress:Port Scheduler Flags
+  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn     
+TCP  10.102.128.4:3080 rr
+  -> 10.244.0.235:8080            Masq    1      0          0         
+  -> 10.244.1.237:8080            Masq    1      0          0   
+```
+
+Please note that the relationship between a Kubernetes Service and IPVS virtual servers is `1:N`. For example, consider a Kubernetes Service that has more than one IP address. An External IP type Service has two IP addresses - ClusterIP and External IP. Then the IPVS proxier will create 2 IPVS virtual servers - one for Cluster IP and another one for External IP. The relationship between a Kubernetes Endpoint (each IP+Port pair) and an IPVS virtual server is `1:1`.
+
+Deleting of a Kubernetes service will trigger deletion of the corresponding IPVS virtual server, IPVS real servers and its IP addresses bound to the dummy interface.
+
+####  Port Mapping
+
+There are three proxy modes in IPVS: NAT (masq), IPIP and DR. Only NAT mode supports port mapping. Kube-proxy leverages NAT mode for port mapping. The following example shows IPVS mapping Service port 3080 to Pod port 8080.
+
+```
+TCP  10.102.128.4:3080 rr
+  -> 10.244.0.235:8080            Masq    1      0          0         
+  -> 10.244.1.237:8080            Masq    1      0       
+```
+
+#### Session Affinity
+
+IPVS supports client IP session affinity (persistent connection). When a Service specifies session affinity, the IPVS proxier will set a timeout value (180min=10800s by default) in the IPVS virtual server. For example:
+
+```
+# kubectl describe svc nginx-service
+Name:			nginx-service
+...
+IP:			    10.102.128.4
+Port:			http	3080/TCP
+Session Affinity:	ClientIP
+
+# ipvsadm -ln
+IP Virtual Server version 1.2.1 (size=4096)
+Prot LocalAddress:Port Scheduler Flags
+  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
+TCP  10.102.128.4:3080 rr persistent 10800
+```
+
+#### Iptables & Ipset in IPVS Proxier
+
+IPVS is for load balancing and it can't handle other workarounds in kube-proxy, e.g. packet filtering, hairpin-masquerade tricks, SNAT, etc.
+
+IPVS proxier leverages iptables in the above scenarios. Specifically, ipvs proxier will fall back on iptables in the following 4 scenarios:
+
+- kube-proxy start with --masquerade-all=true
+- Specify cluster CIDR in kube-proxy startup
+- Support Loadbalancer type service
+- Support NodePort type service
+
+However, we don't want to create too many iptables rules. So we adopt ipset for the sake of decreasing iptables rules. The following is the table of ipset sets that IPVS proxier maintains:
+
+| set name                       | members                                  | usage                                    |
+| ------------------------------ | ---------------------------------------- | ---------------------------------------- |
+| KUBE-CLUSTER-IP                | All Service IP + port                    | masquerade for cases that `masquerade-all=true` or `clusterCIDR` specified |
+| KUBE-LOOP-BACK                 | All Service IP + port + IP               | masquerade for resolving hairpin issue   |
+| KUBE-EXTERNAL-IP               | Service External IP + port               | masquerade for packets to external IPs   |
+| KUBE-LOAD-BALANCER             | Load Balancer ingress IP + port          | masquerade for packets to Load Balancer type service |
+| KUBE-LOAD-BALANCER-LOCAL       | Load Balancer ingress IP + port with `externalTrafficPolicy=local` | accept packets to Load Balancer with `externalTrafficPolicy=local` |
+| KUBE-LOAD-BALANCER-FW          | Load Balancer ingress IP + port with `loadBalancerSourceRanges` | Drop packets for Load Balancer type Service with `loadBalancerSourceRanges` specified |
+| KUBE-LOAD-BALANCER-SOURCE-CIDR | Load Balancer ingress IP + port + source CIDR | accept packets for Load Balancer type Service with `loadBalancerSourceRanges` specified |
+| KUBE-NODE-PORT-TCP             | NodePort type Service TCP port           | masquerade for packets to NodePort(TCP)  |
+| KUBE-NODE-PORT-LOCAL-TCP       | NodePort type Service TCP port with `externalTrafficPolicy=local` | accept packets to NodePort Service with `externalTrafficPolicy=local` |
+| KUBE-NODE-PORT-UDP             | NodePort type Service UDP port           | masquerade for packets to NodePort(UDP)  |
+| KUBE-NODE-PORT-LOCAL-UDP       | NodePort type service UDP port with `externalTrafficPolicy=local` | accept packets to NodePort Service with `externalTrafficPolicy=local` |
+
+
+In general, for IPVS proxier, the number of iptables rules is static, no matter how many Services/Pods we have.
+
+### Run kube-proxy in IPVS Mode
+
+Currently, local-up scripts, GCE scripts, and kubeadm support switching IPVS proxy mode via exporting environment variables (`KUBE_PROXY_MODE=ipvs`) or specifying flag (`--proxy-mode=ipvs`). Before running IPVS proxier, please ensure IPVS required kernel modules are already installed.
+
+```
+ip_vs
+ip_vs_rr
+ip_vs_wrr
+ip_vs_sh
+nf_conntrack_ipv4
+```
+
+Finally, for Kubernetes v1.10, feature gate `SupportIPVSProxyMode` is set to `true` by default. For Kubernetes v1.11, the feature gate is entirely removed. However, you need to enable `--feature-gates=SupportIPVSProxyMode=true` explicitly for Kubernetes before v1.10.
+
+## Get Involved
+The simplest way to get involved with Kubernetes is by joining one of the many [Special Interest Groups](https://github.com/kubernetes/community/blob/master/sig-list.md) (SIGs) that align with your interests. Have something you’d like to broadcast to the Kubernetes community? Share your voice at our weekly [community meeting](https://github.com/kubernetes/community/blob/master/communication.md#weekly-meeting), and through the channels below.
+
+Thank you for your continued feedback and support.
+Post questions (or answer questions) on [Stack Overflow](http://stackoverflow.com/questions/tagged/kubernetes)
+Join the community portal for advocates on [K8sPort](http://k8sport.org/)
+Follow us on Twitter [@Kubernetesio](https://twitter.com/kubernetesio) for latest updates
+Chat with the community on [Slack](http://slack.k8s.io/)
+Share your Kubernetes [story](https://docs.google.com/a/linuxfoundation.org/forms/d/e/1FAIpQLScuI7Ye3VQHQTwBASrgkjQDSS5TP0g3AXfFhwSM9YpHgxRKFA/viewform)
@@ -0,0 +1,187 @@
+---
+layout: blog
+title: "CoreDNS GA for Kubernetes Cluster DNS"
+date: 2018-07-10
+---
+
+**Author**: John Belamaric (Infoblox)
+
+**Editor’s note: this post is part of a [series of in-depth articles](https://kubernetes.io/blog/2018/06/27/kubernetes-1.11-release-announcement/) on what’s new in Kubernetes 1.11**
+
+## Introduction
+
+In Kubernetes 1.11, [CoreDNS](https://coredns.io) has reached General Availability (GA) for DNS-based service discovery, as an alternative to the kube-dns addon. This means that CoreDNS will be offered as an option in upcoming versions of the various installation tools. In fact, the kubeadm team chose to make it the default option starting with Kubernetes 1.11.
+
+DNS-based service discovery has been part of Kubernetes for a long time with the kube-dns cluster addon. This has generally worked pretty well, but there have been some concerns around the reliability, flexibility and security of the implementation.
+
+CoreDNS is a general-purpose, authoritative DNS server that provides a backwards-compatible, but extensible, integration with Kubernetes. It resolves the issues seen with kube-dns, and offers a number of unique features that solve a wider variety of use cases.
+
+In this article, you will learn about the differences in the implementations of kube-dns and CoreDNS, and some of the helpful extensions offered by CoreDNS.
+
+## We appreciate your feedback
+
+We are conducting a survey to evaluate the adoption of CoreDNS as the DNS for Kubernetes's cluster. 
+If you are currently using CoreDNS inside a Kubernetes cluster, please, [take 5 minutes to provide us some feedback by filling this survey](https://www.surveymonkey.com/r/SKZQSLK).
+
+Thank you, we appreciate your collaboration here.
+
+## Implementation differences
+
+In kube-dns, several containers are used within a single pod: `kubedns`, `dnsmasq`, and `sidecar`. The `kubedns`
+container watches the Kubernetes API and serves DNS records based on the [Kubernetes DNS specification](https://github.com/kubernetes/dns/blob/master/docs/specification.md), `dnsmasq` provides caching and stub domain support, and `sidecar` provides metrics and health checks.
+
+This setup leads to a few issues that have been seen over time. For one, security vulnerabilities in `dnsmasq` have led to the need
+for a security-patch release of Kubernetes in the past. Additionally, because `dnsmasq` handles the stub domains,
+but `kubedns` handles the External Services, you cannot use a stub domain in an external service, which is very
+limiting to that functionality (see [dns#131](https://github.com/kubernetes/dns/issues/131)).
+
+All of these functions are done in a single container in CoreDNS, which is running a process written in Go. The
+different plugins that are enabled replicate (and enhance) the functionality found in kube-dns.
+
+## Configuring CoreDNS
+
+In kube-dns, you can [modify a ConfigMap](https://kubernetes.io/blog/2017/04/configuring-private-dns-zones-upstream-nameservers-kubernetes/) to change the behavior of your service discovery. This allows the addition of
+features such as serving stub domains, modifying upstream nameservers, and enabling federation.
+
+In CoreDNS, you similarly can modify the ConfigMap for the CoreDNS [Corefile](https://coredns.io/2017/07/23/corefile-explained/) to change how service discovery
+works. This Corefile configuration offers many more options than you will find in kube-dns, since it is the
+primary configuration file that CoreDNS uses for configuration of all of its features, even those that are not
+Kubernetes related.
+
+When upgrading from kube-dns to CoreDNS using `kubeadm`, your existing ConfigMap will be used to generate the
+customized Corefile for you, including all of the configuration for stub domains, federation, and upstream nameservers. See [Using CoreDNS for Service Discovery](https://kubernetes.io/docs/tasks/administer-cluster/coredns/) for more details.
+
+
+## Bug fixes and enhancements
+
+There are several open issues with kube-dns that are resolved in CoreDNS, either in default configuration or with some customized configurations.
+
+  * [dns#55 - Custom DNS entries for kube-dns](https://github.com/kubernetes/dns/issues/55) may be handled using the "fallthrough" mechanism in the [kubernetes plugin](https://coredns.io/plugins/kubernetes), using the [rewrite plugin](https://coredns.io/plugins/rewrite), or simply serving a subzone with a different plugin such as the [file plugin](https://coredns.io/plugins/file).
+
+  * [dns#116 - Only one A record set for headless service with pods having single hostname](https://github.com/kubernetes/dns/issues/116). This issue is fixed without any additional configuration.
+  * [dns#131 - externalName not using stubDomains settings](https://github.com/kubernetes/dns/issues/131). This issue is fixed without any additional configuration.
+  * [dns#167 - enable skyDNS round robin A/AAAA records](https://github.com/kubernetes/dns/issues/167). The equivalent functionality can be configured using the [load balance plugin](https://coredns.io/plugins/loadbalance).
+  * [dns#190 - kube-dns cannot run as non-root user](https://github.com/kubernetes/dns/issues/190). This issue is solved today by using a non-default image, but it will be made the default CoreDNS behavior in a future release.
+  * [dns#232 - fix pod hostname to be podname for dns srv records](https://github.com/kubernetes/dns/issues/232) is an enhancement that is supported through the "endpoint_pod_names" feature described below.
+
+
+## Metrics
+
+The functional behavior of the default CoreDNS configuration is the same as kube-dns. However,
+one difference you need to be aware of is that the published metrics are not the same. In kube-dns,
+you get separate metrics for `dnsmasq` and `kubedns` (skydns). In CoreDNS there is a completely
+different set of metrics, since it is all a single process. You can find more details on these
+metrics on the CoreDNS [Prometheus plugin](https://coredns.io/plugins/metrics/) page.
+
+## Some special features
+
+The standard CoreDNS Kubernetes configuration is designed to be backwards compatible with the prior
+kube-dns behavior. But with some configuration changes, CoreDNS can allow you to modify how the
+DNS service discovery works in your cluster. A number of these features are intended to still be
+compliant with the [Kubernetes DNS specification](https://github.com/kubernetes/dns/blob/master/docs/specification.md);
+they enhance functionality but remain backward compatible. Since CoreDNS is not
+*only* made for Kubernetes, but is instead a general-purpose DNS server, there are many things you
+can do beyond that specification.
+
+### Pods verified mode
+
+In kube-dns, pod name records are "fake". That is, any "a-b-c-d.namespace.pod.cluster.local" query will
+return the IP address "a.b.c.d". In some cases, this can weaken the identity guarantees offered by TLS. So,
+CoreDNS offers a "pods verified" mode, which will only return the IP address if there is a pod in the
+specified namespace with that IP address.
+
+### Endpoint names based on pod names
+
+In kube-dns, when using a headless service, you can use an SRV request to get a list of
+all endpoints for the service:
+
+```
+dnstools# host -t srv headless
+headless.default.svc.cluster.local has SRV record 10 33 0 6234396237313665.headless.default.svc.cluster.local.
+headless.default.svc.cluster.local has SRV record 10 33 0 6662363165353239.headless.default.svc.cluster.local.
+headless.default.svc.cluster.local has SRV record 10 33 0 6338633437303230.headless.default.svc.cluster.local.
+dnstools#
+```
+
+However, the endpoint DNS names are (for practical purposes) random. In CoreDNS, by default, you get endpoint
+DNS names based upon the endpoint IP address:
+
+```
+dnstools# host -t srv headless
+headless.default.svc.cluster.local has SRV record 0 25 443 172-17-0-14.headless.default.svc.cluster.local.
+headless.default.svc.cluster.local has SRV record 0 25 443 172-17-0-18.headless.default.svc.cluster.local.
+headless.default.svc.cluster.local has SRV record 0 25 443 172-17-0-4.headless.default.svc.cluster.local.
+headless.default.svc.cluster.local has SRV record 0 25 443 172-17-0-9.headless.default.svc.cluster.local.
+```
+
+For some applications, it is desirable to have the pod name for this, rather than the pod IP
+address (see for example [kubernetes#47992](https://github.com/kubernetes/kubernetes/issues/47992) and [coredns#1190](https://github.com/coredns/coredns/pull/1190)). To enable this in CoreDNS, you specify the "endpoint_pod_names" option in your Corefile, which results in this:
+
+```
+dnstools# host -t srv headless
+headless.default.svc.cluster.local has SRV record 0 25 443 headless-65bb4c479f-qv84p.headless.default.svc.cluster.local.
+headless.default.svc.cluster.local has SRV record 0 25 443 headless-65bb4c479f-zc8lx.headless.default.svc.cluster.local.
+headless.default.svc.cluster.local has SRV record 0 25 443 headless-65bb4c479f-q7lf2.headless.default.svc.cluster.local.
+headless.default.svc.cluster.local has SRV record 0 25 443 headless-65bb4c479f-566rt.headless.default.svc.cluster.local.
+```
+
+### Autopath
+
+CoreDNS also has a special feature to improve latency in DNS requests for external names. In Kubernetes, the
+DNS search path for pods specifies a long list of suffixes. This enables the use of short names when requesting
+services in the cluster - for example, "headless" above, rather than "headless.default.svc.cluster.local". However,
+when requesting an external name  - "infoblox.com", for example - several invalid DNS queries are made by the client,
+requiring a roundtrip from the client to kube-dns each time (actually to `dnsmasq` and then to `kubedns`, since [negative caching is disabled](https://github.com/kubernetes/dns/issues/121)):
+
+  * infoblox.com.default.svc.cluster.local -> NXDOMAIN
+  * infoblox.com.svc.cluster.local -> NXDOMAIN
+  * infoblox.com.cluster.local -> NXDOMAIN
+  * infoblox.com.your-internal-domain.com -> NXDOMAIN
+  * infoblox.com -> returns a valid record
+
+In CoreDNS, an optional feature called [autopath](https://coredns.io/plugins/autopath) can be enabled that will cause this search path to be followed
+*in the server*. That is, CoreDNS will figure out from the source IP address which namespace the client pod is in,
+and it will walk this search list until it gets a valid answer. Since the first 3 of these are resolved internally
+within CoreDNS itself, it cuts out all of the back and forth between the client and server, reducing latency.
+
+### A few other Kubernetes specific features
+
+In CoreDNS, you can use standard DNS zone transfer to export the entire DNS record set. This is useful for
+debugging your services as well as importing the cluster zone into other DNS servers.
+
+You can also filter by namespaces or a label selector. This can allow you to run specific CoreDNS instances that will only server records that match the filters, exposing only a limited set of your services via DNS.
+
+## Extensibility
+
+In addition to the features described above, CoreDNS is easily extended. It is possible to build custom versions
+of CoreDNS that include your own features. For example, this ability has been used to extend CoreDNS to do recursive resolution
+with the [unbound plugin](https://https://coredns.io/explugins/unbound), to server records directly from a database with the [pdsql plugin](https://coredns.io/explugins/pdsql), and to allow multiple CoreDNS instances to share a common level 2 cache with the [redisc plugin](https://coredns.io/explugins/redisc).
+
+Many other interesting extensions have been added, which you will find on the [External Plugins](https://coredns.io/explugins/) page of the CoreDNS site. One that is really interesting for Kubernetes and Istio users is the [kubernetai plugin](https://coredns.io/explugins/kubernetai), which allows a single CoreDNS instance to connect to multiple Kubernetes clusters and provide service discovery across all of them.
+
+## What's Next?
+
+CoreDNS is an independent project, and as such is developing many features that are not directly
+related to Kubernetes. However, a number of these will have applications within Kubernetes. For example,
+the upcoming integration with policy engines will allow CoreDNS to make intelligent choices about which endpoint
+to return when a headless service is requested. This could be used to route traffic to a local pod, or
+to a more responsive pod. Many other features are in development, and of course as an open source project, we welcome you to suggest and contribute your own features!
+
+The features and differences described above are a few examples. There is much more you can do with CoreDNS.
+You can find out more on the [CoreDNS Blog](https://coredns.io/blog).
+
+### Get involved with CoreDNS
+
+CoreDNS is an incubated [CNCF](https:://cncf.io) project.
+
+We're most active on Slack (and Github):
+
+- Slack: #coredns on <https://slack.cncf.io>
+- Github: <https://github.com/coredns/coredns>
+
+More resources can be found:
+
+- Website: <https://coredns.io>
+- Blog: <https://blog.coredns.io>
+- Twitter: [@corednsio](https://twitter.com/corednsio)
+- Mailing list/group: <coredns-discuss@googlegroups.com>
@@ -0,0 +1,48 @@
+---
+layout: blog
+title: 'Dynamic Kubelet Configuration'
+date: 2018-07-11
+---
+
+**Author**: Michael Taufen (Google)
+
+**Editor’s note: this post is part of a [series of in-depth articles](https://kubernetes.io/blog/2018/06/27/kubernetes-1.11-release-announcement/) on what’s new in Kubernetes 1.11**
+
+## Why Dynamic Kubelet Configuration?
+
+Kubernetes provides API-centric tooling that significantly improves workflows for managing applications and infrastructure. Most Kubernetes installations, however, run the Kubelet as a native process on each host, outside the scope of standard Kubernetes APIs.
+
+In the past, this meant that cluster administrators and service providers could not rely on Kubernetes APIs to reconfigure Kubelets in a live cluster. In practice, this required operators to either ssh into machines to perform manual reconfigurations, use third-party configuration management automation tools, or create new VMs with the desired configuration already installed, then migrate work to the new machines. These approaches are environment-specific and can be expensive.
+
+Dynamic Kubelet configuration gives cluster administrators and service providers the ability to reconfigure Kubelets in a live cluster via Kubernetes APIs.
+
+## What is Dynamic Kubelet Configuration?
+
+Kubernetes v1.10 made it possible to configure the Kubelet via a beta [config file](https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/) API. Kubernetes already provides the ConfigMap abstraction for storing arbitrary file data in the API server.
+
+Dynamic Kubelet configuration extends the Node object so that a Node can refer to a ConfigMap that contains the same type of config file. When a Node is updated to refer to a new ConfigMap, the associated Kubelet will attempt to use the new configuration.
+
+## How does it work?
+
+Dynamic Kubelet configuration provides the following core features:
+
+* Kubelet attempts to use the dynamically assigned configuration.
+* Kubelet "checkpoints" configuration to local disk, enabling restarts without API server access.
+* Kubelet reports assigned, active, and last-known-good configuration sources in the Node status.
+* When invalid configuration is dynamically assigned, Kubelet automatically falls back to a last-known-good configuration and reports errors in the Node status.
+
+To use the dynamic Kubelet configuration feature, a cluster administrator or service provider will first post a ConfigMap containing the desired configuration, then set each Node.Spec.ConfigSource.ConfigMap reference to refer to the new ConfigMap. Operators can update these references at their preferred rate, giving them the ability to perform controlled rollouts of new configurations.
+
+Each Kubelet watches its associated Node object for changes. When the Node.Spec.ConfigSource.ConfigMap reference is updated, the Kubelet will "checkpoint" the new ConfigMap by writing the files it contains to local disk. The Kubelet will then exit, and the OS-level process manager will restart it. Note that if the Node.Spec.ConfigSource.ConfigMap reference is not set, the Kubelet uses the set of flags and config files local to the machine it is running on.
+
+Once restarted, the Kubelet will attempt to use the configuration from the new checkpoint. If the new configuration passes the Kubelet's internal validation, the Kubelet will update Node.Status.Config to reflect that it is using the new configuration. If the new configuration is invalid, the Kubelet will fall back to its last-known-good configuration and report an error in Node.Status.Config.
+
+Note that the default last-known-good configuration is the combination of Kubelet command-line flags with the Kubelet's local configuration file. Command-line flags that overlap with the config file always take precedence over both the local configuration file and dynamic configurations, for backwards-compatibility.
+
+See the following diagram for a high-level overview of a configuration update for a single Node:
+
+![kubelet-diagram](/images/blog/2018-07-11-dynamic-kubelet-configuration/kubelet-diagram.png)
+
+## How can I learn more?
+
+Please see the official tutorial at https://kubernetes.io/docs/tasks/administer-cluster/reconfigure-kubelet/, which contains more in-depth details on user workflow, how a configuration becomes "last-known-good," how the Kubelet "checkpoints" config, and possible failure modes.
@@ -0,0 +1,86 @@
+---
+layout: blog
+title: 'Resizing Persistent Volumes using Kubernetes'
+date: 2018-07-12
+---
+
+**Author**: Hemant Kumar (Red Hat)
+
+**Editor’s note: this post is part of a [series of in-depth articles](https://kubernetes.io/blog/2018/06/27/kubernetes-1.11-release-announcement/) on what’s new in Kubernetes 1.11**
+
+In Kubernetes v1.11 the persistent volume expansion feature is being promoted to beta. This feature allows users to easily resize an existing volume by editing the `PersistentVolumeClaim` (PVC) object. Users no longer have to manually interact with the storage backend or delete and recreate PV and PVC objects to increase the size of a volume. Shrinking persistent volumes is not supported.
+
+Volume expansion was introduced in v1.8 as an Alpha feature, and versions prior to v1.11 required enabling the feature gate, `ExpandPersistentVolumes`, as well as the admission controller, `PersistentVolumeClaimResize` (which prevents expansion of PVCs whose underlying storage provider does not support resizing). In Kubernetes v1.11+, both the feature gate and admission controller are enabled by default.
+
+Although the feature is enabled by default, a cluster admin must opt-in to allow users to resize their volumes. Kubernetes v1.11 ships with volume expansion support for the following in-tree volume plugins: AWS-EBS, GCE-PD, Azure Disk, Azure File, Glusterfs, Cinder, Portworx, and Ceph RBD. Once the admin has determined that volume expansion is supported for the underlying provider, they can make the feature available to users by setting the `allowVolumeExpansion` field to `true` in their `StorageClass` object(s). Only PVCs created from that `StorageClass` will be allowed to trigger volume expansion.
+
+```
+~> cat standard.yaml
+apiVersion: storage.k8s.io/v1
+kind: StorageClass
+metadata:
+  name: standard
+parameters:
+  type: pd-standard
+provisioner: kubernetes.io/gce-pd
+allowVolumeExpansion: true
+reclaimPolicy: Delete
+```
+
+Any PVC created from this `StorageClass` can be edited (as illustrated below) to request more space. Kubernetes will interpret a change to the storage field as a request for more space, and will trigger automatic volume resizing.
+
+![PVC StorageClass](/images/blog/2018-07-12-resizing-persistent-volumes-using-kubernetes/pvc-storageclass.png)
+
+## File System Expansion
+
+Block storage volume types such as GCE-PD, AWS-EBS, Azure Disk, Cinder, and Ceph RBD typically require a file system expansion before the additional space of an expanded volume is usable by pods. Kubernetes takes care of this automatically whenever the pod(s) referencing your volume are restarted.
+
+Network attached file systems (like Glusterfs and Azure File) can be expanded without having to restart the referencing Pod, because these systems do not require special file system expansion.
+
+File system expansion must be triggered by terminating the pod using the volume. More specifically:
+
+* Edit the PVC to request more space.
+* Once underlying volume has been expanded by the storage provider, then the PersistentVolume object will reflect the updated size and the PVC will have the `FileSystemResizePending` condition.
+
+You can verify this by running `kubectl get pvc <pvc_name> -o yaml`
+
+```
+~> kubectl get pvc myclaim -o yaml
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: myclaim
+  namespace: default
+  uid: 02d4aa83-83cd-11e8-909d-42010af00004
+spec:
+  accessModes:
+  - ReadWriteOnce
+  resources:
+    requests:
+      storage: 14Gi
+  storageClassName: standard
+  volumeName: pvc-xxx
+status:
+  capacity:
+    storage: 9G
+  conditions:
+  - lastProbeTime: null
+    lastTransitionTime: 2018-07-11T14:51:10Z
+    message: Waiting for user to (re-)start a pod to finish file system resize of
+      volume on node.
+    status: "True"
+    type: FileSystemResizePending
+  phase: Bound
+```
+* Once the PVC has the condition `FileSystemResizePending` then pod that uses the PVC can be restarted to finish file system resizing on the node. Restart can be achieved by deleting and recreating the pod or by scaling down the deployment and then scaling it up again.
+* Once file system resizing is done, the PVC will automatically be updated to reflect new size.
+
+Any errors encountered while expanding file system should be available as events on pod.
+
+## Online File System Expansion
+
+Kubernetes v1.11 also introduces an alpha feature called online file system expansion. This feature enables file system expansion while a volume is still in-use by a pod. Because this feature is alpha, it requires enabling the feature gate, `ExpandInUsePersistentVolumes`. It is supported by the in-tree volume plugins GCE-PD, AWS-EBS, Cinder, and Ceph RBD. When this feature is enabled, pod referencing the resized volume do not need to be restarted. Instead, the file system will automatically be resized while in use as part of volume expansion. File system expansion does not happen until a pod references the resized volume, so if no pods referencing the volume are running file system expansion will not happen.
+
+## How can I learn more?
+
+Check out additional documentation on this feature here: http://k8s.io/docs/concepts/storage/persistent-volumes.
@@ -0,0 +1,275 @@
+---
+layout: blog
+title: "How the sausage is made: the Kubernetes 1.11 release interview, from the Kubernetes Podcast"
+date: 2018-07-16
+---
+
+<b>Author</b>: Craig Box (Google)
+
+At KubeCon EU, my colleague Adam Glick and I were pleased to announce the [Kubernetes Podcast from Google](https://kubernetespodcast.com/). In this weekly conversation, we focus on all the great things that are happening in the world of Kubernetes and Cloud Native. From the news of the week, to interviews with people in the community, we help you stay up to date on everything Kubernetes.
+
+We [recently had the pleasure of speaking](https://kubernetespodcast.com/episode/010-kubernetes-1.11/) to the release manager for Kubernetes 1.11, Josh Berkus from Red Hat, and the release manager for the upcoming 1.12, Tim Pepper from VMware.  
+
+In this conversation we learned about the release process, the impact of quarterly releases on end users, and how Kubernetes is like baking.
+
+I encourage you to listen to [the podcast version](https://kubernetespodcast.com/episode/010-kubernetes-1.11/) if you have a commute, or a dog to walk. If you like what you hear, [we encourage you to subscribe](https://kubernetespodcast.com/subscribe)! In case you're short on time, or just want to browse quickly, we are delighted to share the transcript with you.
+
+<hr/>
+
+<b>CRAIG BOX: First of all, congratulations both, and thank you.</b>
+
+JOSH BERKUS: Well, thank you. Congratulations for me, because my job is done. 
+
+[LAUGHTER] 
+
+Congratulations and sympathy for Tim. 
+
+[LAUGH] 
+
+TIM PEPPER: Thank you, and I guess thank you? 
+
+[LAUGH] 
+
+<b>ADAM GLICK: For those that don't know a lot about the process, why don't you help people understand — what is it like to be the release manager? What's the process that a release goes through to get to the point when everyone just sees, OK, it's released — 1.11 is available? What does it take to get up to that?</b>
+
+JOSH BERKUS: We have a quarterly release cycle. So every three months, we're releasing. And ideally and fortunately, this is actually now how we are doing things. Somewhere around two, three weeks before the previous release, somebody volunteers to be the release lead. That person is confirmed by [SIG Release](https://github.com/kubernetes/sig-release). So far, we've never had more than one volunteer, so there hasn't been really a fight about it. 
+
+And then that person starts working with others to put together [a team](https://github.com/kubernetes/sig-release/blob/master/releases/release-1.11/release_team.md) called the release team. Tim's just gone through this with Stephen Augustus and [picking out a whole bunch of people](https://github.com/kubernetes/sig-release/blob/master/releases/release-1.12/release_team.md). And then after or a little before— probably after, because we want to wait for the retrospective from the previous release— the release lead then sets [a schedule](https://github.com/kubernetes/sig-release/blob/master/releases/release-1.11/release-1.11.md) for the upcoming release, as in when all the deadlines will be. 
+
+And this is a thing, because we're still tinkering with relative deadlines, and how long should code freeze be, and how should we track features? Because we don't feel that we've gotten down that sort of cadence perfectly yet. I mean, like, we've done pretty well, but we don't feel like we want to actually set [in stone], this is the schedule for each and every release. 
+
+Also, we have to adjust the schedule because of holidays, right? Because you can't have the code freeze deadline starting on July 4 or in the middle of design or sometime else when we're going to have a large group of contributors who are out on vacation. 
+
+TIM PEPPER: This is something I've had to spend some time looking at, thinking about 1.12. Going back to early June as we were tinkering with the code freeze date, starting to think about, well, what are the implications going to be on 1.12? When would these things start falling on the calendar? And then also for 1.11, we had one complexity. If we slipped the release past this week, we start running into the US 4th of July holiday, and we're not likely to get a lot done. 
+
+So much of a slip would mean slipping into the middle of July before we'd really know that we were successfully triaging things. And worst case maybe, we're quite a bit later into July. 
+
+So instead of quarterly with a three-month sort of cadence, well, maybe we've accidentally ended up chopping out one month out of the next release or pushing it quite a bit into the end of the year. And that made the deliberation around things quite complex, but thankfully this week, everything's gone smoothly in the end. 
+
+<b>CRAIG BOX: All the releases so far have been one quarter — they've been a 12-week release cycle, give or take. Is that something that you think will continue going forward, or is the release team thinking about different ways they could run releases?</b>
+
+TIM PEPPER: The whole community is thinking about this. There are voices who'd like the cadence to be faster, and there are voices who'd like it to be shorter. And there's good arguments for both. 
+
+<b>ADAM GLICK: Because it's interesting. It sounds like it is a date-driven release cycle versus a feature-driven release cycle.</b>
+
+JOSH BERKUS: Yeah, certainly. I really honestly think everybody in the world of software recognizes that feature-driven release cycles just don't work. And a big part of the duties of the release team collectively— several members of the team do this— is yanking things out of the release that are not ready. And the hard part of that is figuring out which things aren't ready, right? Because the person who's working on it tends to be super optimistic about what they can get done and what they can get fixed before the deadline. 
+
+<b>ADAM GLICK: Of course.</b>
+
+TIM PEPPER: And this is one of the things I think that's useful about the process we have in place on the release team for having shadows who spend some time on the release team, working their way up into more of a lead position and gaining some experience, starting to get some exposure to see that optimism and see the processes for vetting. 
+
+And it's even an overstatement to say the process. Just see the way that we build the intuition for how to vet and understand and manage the risk, and really go after and chase things down proactively and early to get resolution in a timely way versus continuing to just all be optimistic and letting things maybe languish and put a release at risk. 
+
+<b>CRAIG BOX: I've been reading this week about the introduction of [feature branches](https://github.com/kubernetes/community/issues/566) to Kubernetes. The new server-side apply feature, for example, is being built in a branch so that it didn't have to be half-built in master and then ripped out again as the release approached, if the feature wasn't ready. That seems to me like something that's a normal part of software development? Is there a reason it's taken so long to bring that to core Kubernetes?</b>
+
+JOSH BERKUS: I don't actually know the history of why we're not using feature branches. I mean, the reason why we're not using feature branches pervasively now is that we have to transition from a different system. And I'm not really clear on how we adopted that linear development system. But it's certainly something we discussed on the release team, because there were issues of features that we thought were going to be ready, and then developed major problems. And we're like, if we have to back this out, that's going to be painful. And we did actually have to back one feature out, which involved not pulling out a Git commit, but literally reversing the line changes, which is really not how you want to be doing things. 
+
+<b>CRAIG BOX: No.</b>
+
+TIM PEPPER: The other big benefit, I think, to the release branches if they are well integrated with the CI system for continuous integration and testing, you really get the feedback, and you can demonstrate, this set of stuff is ready. And then you can do deferred commitment on the master branch. And what comes in to a particular release on the timely cadence that users are expecting is stuff that's ready. You don't have potentially destabilizing things, because you can get a lot more proof and evidence of readiness. 
+
+<b>ADAM GLICK: What are you looking at in terms of the tool chain that you're using to do this? You mentioned a couple of things, and I know it's obviously run through GitHub. But I imagine you have a number of other tools that you're using in order to manage the release, to make sure that you understand what's ready, what's not. You mentioned balancing between people who are very optimistic about the feature they're working on making it in versus the time-driven deadline, and balancing those two. Is that just a manual process, or do you have a set of tools that help you do that?</b>
+
+JOSH BERKUS: Well, there's code review, obviously. So just first of all, process was somebody wants to actually put in a feature, commit, or any kind of merge really, right? So that has to be assigned to one of the SIGs, one of these Special Interest Groups. Possibly more than one, depending on what areas it touches. 
+
+And then two generally overlapping groups of people have to approve that. One would be the SIG that it's assigned to, and the second would be anybody represented in the OWNERS files in the code tree of the directories which get touched. 
+
+Now sometimes those are the same group of people. I'd say often, actually. But sometimes they're not completely the same group of people, because sometimes you're making a change to the network, but that also happens to touch GCP support and OpenStack support, and so they need to review it as well. 
+
+So the first part is the human part, which is a bunch of other people need to look at this. And possibly they're going to comment "Hey. This is a really weird way to do this. Do you have a reason for it?"
+
+Then the second part of it is the automated testing that happens, the automated acceptance testing that happens via webhook on there. And actually, one of the things that we did that was a significant advancement in this release cycle— and by we, I actually mean not me, but the great folks at [SIG Scalability](https://github.com/kubernetes/community/tree/master/sig-scalability) did— was add an [additional acceptance test](https://k8s-testgrid.appspot.com/sig-release-master-blocking#gce-scale-performance) that does a mini performance test. 
+
+Because one of the problems we've had historically is our major performance tests are large and take a long time to run, and so by the time we find out that we're failing the performance tests, we've already accumulated, you know, 40, 50 commits. And so now we're having to do git bisect to find out which of those commits actually caused the performance regression, which can make them very slow to address. 
+
+And so adding that performance pre-submit, the performance acceptance test really has helped stabilize performance in terms of new commits. So then we have that level of testing that you have to get past. 
+
+And then when we're done with that level of testing, we run a whole large battery of larger tests— end-to-end tests, performance tests, upgrade and downgrade tests. And one of the things that we've added recently and we're integrating to the process something called conformance tests. And the conformance test is we're testing whether or not you broke backwards compatibility, because it's obviously a big deal for Kubernetes users if you do that when you weren't intending to. 
+
+One of the busiest roles in the release team is a role called [CI Signal](https://github.com/kubernetes/sig-release#ci-signal-lead). And it's that person's job just to watch all of the tests for new things going red and then to try to figure out why they went red and bring it to people's attention. 
+
+<b>ADAM GLICK: I've often heard what you're referring to kind of called a breaking change, because it breaks the existing systems that are running. How do you identify those to people so when they see, hey, there's a new version of Kubernetes out there, I want to try it out, is that just going to release notes? Or is there a special way that you identify breaking changes as opposed to new features?</b>
+
+JOSH BERKUS: That goes into release notes. I mean, keep in mind that one of the things that happens with Kubernetes' features is we go through this alpha, beta, general availability phase, right? So a feature's alpha for a couple of releases and then becomes beta for a release or two, and then it becomes generally available. And part of the idea of having this that may require a feature to go through that cycle for a year or more before its general availability is by the time it's general availability, we really want it to be, we are not going to change the API for this. 
+
+However, stuff happens, and we do occasionally have to do those. And so far, our main way to identify that to people actually is in the release notes. If you look at [the current release notes](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.11.md#no-really-you-must-do-this-before-you-upgrade), there are actually two things in there right now that are sort of breaking changes. 
+
+One of them is the bit with [priority and preemption](https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/) in that preemption being on by default now allows badly behaved users of the system to cause trouble in new ways. I'd actually have to look at the release notes to see what the second one was...
+
+TIM PEPPER: The [JSON capitalization case sensitivity](https://github.com/kubernetes/kubernetes/issues/64612). 
+
+JOSH BERKUS: Right. Yeah. And that was one of those cases where you have to break backwards compatibility, because due to a library switch, we accidentally enabled people using JSON in a case-insensitive way in certain APIs, which was never supposed to be the case. But because we didn't have a specific test for that, we didn't notice that we'd done it. 
+
+And so for three releases, people could actually shove in malformed JSON, and Kubernetes would accept it. Well, we have to fix that now. But that does mean that there are going to be users out in the field who have malformed JSON in their configuration management that is now going to break. 
+
+<b>CRAIG BOX: But at least the good news is Kubernetes was always outputting correct formatted JSON during this period, I understand.</b>
+
+JOSH BERKUS: Mm-hmm. 
+
+TIM PEPPER: I think that also kind of reminds of one of the other areas— so kind of going back to the question of, well, how do you share word of breaking changes? Well, one of the ways you do that is to have as much quality CI that you can to catch these things that are important. Give the feedback to the developer who's making the breaking change, such that they don't make the breaking change. And then you don't actually have to communicate it out to users. 
+
+So some of this is bound to happen, because you always have test escapes. But it's also a reminder of the need to ensure that you're also really building and maintaining your test cases and the quality and coverage of your CI system over time. 
+
+<b>ADAM GLICK: What do you mean when you say test escapes?</b>
+
+TIM PEPPER: So I guess it's a term in the art, but for those who aren't familiar with it, you have intended behavior that wasn't covered by test, and as a result, an unintended change happens to that. And instead of your intended behavior being shipped, you're shipping something else. 
+
+JOSH BERKUS: The JSON change is a textbook example of this, which is we were testing that the API would continue to accept correct JSON. We were not testing adequately that it wouldn't accept incorrect JSON. 
+
+TIM PEPPER: A test escape, another way to think of it as you shipped a bug because there was not a test case highlighting the possibility of the bug. 
+
+<b>ADAM GLICK: It's the classic, we tested to make sure the feature worked. We didn't test to make sure that breaking things didn't work.</b>
+
+TIM PEPPER: It's common for us to focus on "I've created this feature and I'm testing the positive cases". And this also comes to thinking about things like secure by default and having a really robust system. A harder piece of engineering often is to think about the failure cases and really actively manage those well. 
+
+JOSH BERKUS: I had a conversation with a contributor recently where it became apparent that that contributor had never worked on a support team, because their conception of a badly behaved user was, like, a hacker, right? An attacker who comes from outside. 
+
+And I'm like, no, no, no. You're stable of badly behaved users is your own staff. You know, they will do bad things, not necessarily intending to do bad things, but because they're trying to take a shortcut. And that is actually your primary concern in terms of preventing breaking the system. 
+
+<b>CRAIG BOX: Josh, what was your preparation to be release manager for 1.11?</b>
+
+JOSH BERKUS: I was on the release team for two cycles, plus I was kind of auditing the release team for half a cycle before that. So in 1.9, I originally joined to be the shadow for bug triage, except I ended up not being the shadow, because the person who was supposed to be the lead for bug triage then dropped out. Then I ended up being the bug triage lead, and had to kind of improvise it because there wasn't documentation on what was involved in the role at the time. 
+
+And then I was [bug triage lead](https://github.com/kubernetes/sig-release/blob/master/README.md#bug-triage-lead) for a second cycle, for the 1.10 cycle, and then took over as release lead for the cycle. And one of the things on my to-do list is to update the requirements to be release lead, because we actually do have written requirements, and to say that the expectation now is that you spend at least two cycles on the release team, one of them either as a lead or as a shadow to the release lead. 
+
+<b>CRAIG BOX: And is bug triage lead just what it sounds like?</b>
+
+JOSH BERKUS: Yeah. Pretty much. There's more tracking involved than triage. Part of it is just deficiencies in tooling, something we're looking to address. But things like GitHub API limitations make it challenging to build automated tools that help us intelligently track issues. And we are actually working with GitHub on that. Like, they've been helpful. It's just, they have their own scaling problems. 
+
+But then beyond that, you know, a lot of that, it's what you would expect it to be in terms of what triage says, right? Which is looking at every issue and saying, first of all, is this a real issue? Second, is it a serious issue? Third, who needs to address this? 
+
+And that's a lot of the work, because for anybody who is a regular contributor to Kubernetes, the number of GitHub notifications that they receive per day means that most of us turn our GitHub notifications off. 
+
+<b>CRAIG BOX: Indeed.</b>
+
+JOSH BERKUS: Because it's just this fire hose. And as a result, when somebody really needs to pay attention to something right now, that generally requires a human to go and track them down by email or Slack or whatever they prefer. Twitter in some cases. I've done that. And say, hey. We really need you to look at this issue, because it's about to hold up the beta release. 
+
+<b>ADAM GLICK: When you look at the process that you're doing now, what are the changes that are coming in the future that will make the release process even better and easier?</b>
+
+JOSH BERKUS: Well, we just went through this whole retro, and I put in some recommendations for things. Obviously, some additional automation, which I'm going to be looking at doing now that I'm cycling off of the release team for a quarter and can actually look at more longer term goals, will help, particularly now that we've addressed actually some of our GitHub data flow issues. 
+
+Beyond that, I put in a whole bunch of recommendations in the retro, but it's actually up to Tim which recommendations he's going to try to implement. So I'll let him [comment]. 
+
+TIM PEPPER: I think one of the biggest changes that happened in the 1.11 cycle is this emphasis on trying to keep our continuous integration test status always green. That is huge for software development and keeping velocity. If you have this more, I guess at this point antiquated notion of waterfall development, where you do feature development for a while and are accepting of destabilization, and somehow later you're going to come back and spend a period on stabilization and fixing, that really elongates the feedback loop for developers. 
+
+And they don't realize what was broken, and the problems become much more complex to sort out as time goes by. One, developers aren't thinking about what it was that they'd been working on anymore. They've lost the context to be able to efficiently solve the problem. 
+
+But then you start also getting interactions. Maybe a bug was introduced, and other people started working around it or depending on it, and you get complex dependencies then that are harder to fix. And when you're trying to do that type of complex resolution late in the cycle, it becomes untenable over time. So I think continuing on that and building on it, I'm seeing a little bit more focus on test cases and meaningful test coverage. I think that's a great cultural change to have happening. 
+
+And maybe because I'm following Josh into this role from a bug triage position and in his mentions earlier of just the communications and tracking involved with that versus triage, I do have a bit of a concern that at times, email and Slack are relatively quiet. Some of the SIG meeting notes are a bit sparse or YouTube videos slow to upload. So the general artifacts around choice making I think is an area where we need a little more rigor. So I'm hoping to see some of that. 
+
+And that can be just as subtle as commenting on issues like, hey, this commit doesn't say what it's doing. And for that reason on the release team, we can't assess its risk versus value. So could you give a little more information here? Things like that that give more information both to the release team and the development community as well, because this is open source. And to collaborate, you really do need to communicate in depth. 
+
+<b>CRAIG BOX: Speaking of cultural changes, professional baker to Kubernetes' release lead sounds like quite a journey.</b>
+
+JOSH BERKUS: There was a lot of stuff in between. 
+
+<b>CRAIG BOX: Would you say there are a lot of similarities?</b>
+
+JOSH BERKUS: You know, believe it or not, there actually are similarities. And here's where it's similar, because I was actually thinking about this earlier. So when I was a professional baker, one of the things that I had to do was morning pastry. Like, I was actually in charge of doing several other things for custom orders, but since I had to come to work at 3:00 AM anyway— which also distressingly has similarities with some of this process. Because I had to come to work at 3:00 AM anyway, one of my secondary responsibilities was traying the morning pastry. 
+
+And one of the parts of that is you have this great big gas-fired oven with 10 rotating racks in it that are constantly rotating. Like, you get things in and out in the oven by popping them in and out while the racks are moving. That takes a certain amount of skill. You get burn marks on your wrists for your first couple of weeks of work. And then different pastries require a certain number of rotations to be done. 
+
+And there's a lot of similarities to the release cadence, because what you're doing is you're popping something in the oven or you're seeing something get kicked off, and then you have a certain amount of time before you need to check on it or you need to pull it out. And you're doing that in parallel with a whole bunch of other things. You know, with 40 other trays. 
+
+<b>CRAIG BOX: And with presumably a bunch of colleagues who are all there at the same time.</b>
+
+JOSH BERKUS: Yeah. And the other thing is that these deadlines are kind of absolute, right? You can't say, oh, well, I was reading a magazine article, and I didn't have time to pull that tray out. It's too late. The pastry is burned, and you're going to have to throw it away, and they're not going to have enough pastry in the front case for the morning rush. And the customers are not interested in your excuses for that. 
+
+So from that perspective, from the perspective of saying, hey, we have a bunch of things that need to happen in parallel, they have deadlines and those deadlines are hard deadlines, there it's actually fairly similar. 
+
+<b>CRAIG BOX: Tim, do you have any other history that helped get you to where you are today?</b>
+
+TIM PEPPER: I think in some ways I'm more of a traditional journey. I've got a computer engineering bachelor's degree. But I'm also maybe a bit of an outlier. In the late '90s, I found a passion for open source and Linux. Maybe kind of an early adopter, early believer in that. 
+
+And was working in the industry in the Bay Area for a while. Got involved in the Silicon Valley and Bay Area Linux users groups a bit, and managed to find work as a Linux sysadmin, and then doing device driver and kernel work and on up into distro. So that was all kind of standard in a way. And then I also did some other work around hardware enablement, high-performance computing, non-uniform memory access. Things that are really, really systems work. 
+
+And then about three years ago, my boss was really bending my ear and trying to get me to work on this cloud-related project. And that just felt so abstract and different from the low-level bits type of stuff that I'd been doing. 
+
+But kind of grudgingly, I eventually came around to the realization that the cloud is interesting, and it's so much more complex than local machine-only systems work, the type of things that I'd been doing before. It's massively distributed and you have a high-latency, low-reliability interconnect on all the nodes in the distributed network. So it's wildly complex engineering problems that need solved. 
+
+And so that got me interested. Started working then on this open source orchestrator for virtual machines and containers. It was written in Go and was having a lot of fun. But it wasn't Kubernetes, and it was becoming clear that Kubernetes was taking off. So about a year ago, I made the deliberate choice to move over to Kubernetes work. 
+
+<b>ADAM GLICK: Previously, Josh, you spoke a little bit about your preparation for becoming a release manager. For other folks that are interested in getting involved in the community and maybe getting involved in release management, should they follow the same path that you did? Or what are ways that would be good for them to get involved? And for you, Tim, how you've approached the preparation for taking on the next release.</b>
+
+JOSH BERKUS: The great thing with the release team is that we have this formal mentorship path. And it's fast, right? That's the advantage of releasing quarterly, right? Is that within six months, you can go from joining the team as a shadow to being the release lead if you have the time. And you know, by the time you work your way up to release time, you better have support from your boss about this, because you're going to end up spending a majority of your work time towards the end of the release on release management. 
+
+So the answer is to sign up to look when we're getting into the latter half of release cycle, to sign up as a shadow. Or at the beginning of a release cycle, to sign up as a shadow. Some positions actually can reasonably use more than one shadow. There's some position that just require a whole ton of legwork like release notes. And as a result, could actually use more than one shadow meaningfully. So there's probably still places where people could sign up for 1.12. Is that true, Tim? 
+
+TIM PEPPER: Definitely. I think— gosh, right now we have 34 volunteers on the release team, which is— 
+
+<b>ADAM GLICK: Wow.</b>
+
+JOSH BERKUS: OK. OK. Maybe not then. 
+
+[LAUGH] 
+
+TIM PEPPER: It's potentially becoming a lot of cats to herd. But I think even outside of that formal volunteering to be a named shadow, anybody is welcome to show up to the release team meetings, follow the release team activities on [Slack](http://slack.k8s.io), start understanding how the process works. And really, this is the case all across open source. It doesn't even have to be the release team. If you're passionate about networking, start following what SIG Network is doing. It's the same sort of path, I think, into any area on the project. 
+
+Each of the SIGs [has] a channel. So it would be #SIG-whatever the name is. [In our] case, #SIG-Release. 
+
+I'd also maybe give a plug for a [talk I did at KubeCon](https://youtu.be/goAph8A20gQ) in Copenhagen this spring, talking about how the release team specifically can be a path for new contributors coming in. And had some ideas and suggestions there for newcomers. 
+
+<b>CRAIG BOX: There's three questions in the Google SRE postmortem template that I really like. And I'm sure you will have gone through these in the retrospective process as you released 1.11, so I'd like to ask them now one at a time. 
+
+First of all, what went well?</b>
+
+JOSH BERKUS: Two things, I think, really improved things, both for contributors and for the release team. Thing number one was putting a strong emphasis on getting the test grid green well ahead of code freeze. 
+
+TIM PEPPER: Definitely. 
+
+JOSH BERKUS: Now partly that went well because we had a spectacular CI lead, [Aish Sundar](https://github.com/aishsundar), who's now in training to become the release lead. 
+
+TIM PEPPER: And I'd count that partly as one of the "Where were you lucky?" areas. We happened upon a wonderful person who just popped up and volunteered. 
+
+JOSH BERKUS: Yes. And then but part of that was also that we said, hey. You know, we're not going to do what we've done before which is not really care about these tests until code slush. We're going to care about these tests now. 
+
+And importantly— this is really important to the Kubernetes community— when we went to the various SIGs, the SIG Cluster Lifecycle and SIG Scalability and SIG Node and the other ones who were having test failures, and we said this to them. They didn't say, get lost. I'm busy. They said, what's failing? 
+
+<b>CRAIG BOX: Great.</b>
+
+JOSH BERKUS: And so that made a big difference. And the second thing that was pretty much allowed by the first thing was to shorten the code freeze period. Because the code freeze period is frustrating for developers, because if they don't happen to be working on a 1.11 feature, even if they worked on one before, and they delivered it early in the cycle, and it's completely done, they're kind of paralyzed, and they can't do anything during code freeze. And so it's very frustrating for them, and we want to make that period as short as possible. And we did that this time, and I think it helped everybody. 
+
+<b>CRAIG BOX: What went poorly?</b>
+
+JOSH BERKUS: We had a lot of problems with flaky tests. We have a lot of old tests that are not all that well maintained, and they're testing very complicated things like upgrading a cluster that has 40 nodes. And as a result, these tests have high failure rates that have very little to do with any change in the code. 
+
+And so one of the things that happened, and the reason we had a one-day delay in the release is, you know, we're a week out from release, and just by random luck of the draw, a bunch of these tests all at once got a run of failures. And it turned out that that run of failures didn't actually mean anything, having anything to do with Kubernetes. But there was no way for us to tell that without a lot of research, and we were not going to have enough time for that research without delaying the release. 
+
+So one of the things we're looking to address in the 1.12 cycle is to actually move some of those flaky tests out. Either fix them or move them out of the release blocking category. 
+
+TIM PEPPER: In a way, I think this also highlights one of the things that Josh mentioned that went well, the emphasis early on getting the test results green, it allows us to see the extent to which these flakes are such a problem. And then the unlucky occurrence of them all happening to overlap on a failure, again, highlights that these flakes have been called out in the community for quite some time. I mean, at least a year. I know one contributor who was really concerned about them. 
+
+But they became a second order concern versus just getting things done in the short term, getting features and proving that the features worked, and kind of accepting in a risk management way on the release team that, yes, those are flakes. We don't have time to do something about them, and it's OK. But because of the emphasis on keeping the test always green now, we have the luxury maybe to focus on improving these flakes, and really get to where we have truly high quality CI signal, and can really believe in the results that we have on an ongoing basis. 
+
+JOSH BERKUS: And having solved some of the more basic problems, we're now seeing some of the other problems like coordination between related features. Like we right now have a feature where— and this is one of the sort of backwards compatibility release notes— where the feature went into beta, and is on by default. 
+
+And the second feature that was supposed to provide access control for the first feature did not go in as beta, and is not on by default. And the team for the first feature did not realize the second feature was being held up until two days before the release. So it's going to result in us actually patching something in 11.1. 
+
+And so like, we put that into something that didn't go well. But on the other hand, as Tim points out, a few release cycles ago, we wouldn't even have identified that as a problem, because we were still struggling with just individual features having a clear idea well ahead of the release of what was going in and what wasn't going in. 
+
+TIM PEPPER: I think something like this also is a case that maybe advocates for the use of feature branches. If these things are related, we might have seen it and done more pre-testing within that branch and pre-integration, and decide maybe to merge a couple of what initially had been disjoint features into a single feature branch, and really convince ourselves that together they were good. And cross all the Ts, dot all the Is on them, and not have something that's gated on an alpha feature that's possibly falling away. 
+
+<b>CRAIG BOX: And then the final question, which I think you've both touched on a little. Where did you get lucky, or unlucky perhaps?</b>
+
+JOSH BERKUS: I would say number one where I got lucky is truly having a fantastic team. I mean, we just had a lot of terrific people who were very good and very energetic and very enthusiastic about taking on their release responsibilities including Aish and Tim and Ben and Nick and Misty who took over Docs four weeks into the release. And then went crazy with it and said, well, I'm new here, so I'm going to actually change a bunch of things we've been doing that didn't work in the first place. So that was number one. I mean, that really made honestly all the difference. 
+
+And then the second thing, like I said, is that we didn't have sort of major, unexpected monkey wrenches thrown at us. So in the 1.10 cycle, we actually had two of those, which is why I still count Jace as heroic for pulling off a release that was only a week late. 
+
+You know, number one was having the scalability tests start failing for unrelated reasons for a long period, which then masked the fact that they were actually failing for real reasons when we actually got them working again. And as a result, ending up debugging a major and super complicated scalability issue within days of what was supposed to be the original release date. So that was monkey wrench number one for the 1.10 cycle. 
+
+Monkey wrench number two for the 1.10 cycle was we got a security hole that needed to be patched. And so again, a week out from what was supposed to be the original release date, we were releasing a security update, and that security update required patching the release branch. And it turns out that that patch against the release branch broke a bunch of incoming features. And we didn't get anything of that magnitude in the 1.11 release, and I'm thankful for that. 
+
+TIM PEPPER: Also, I would maybe argue in a way that a portion of that wasn't just luck. The extent to which this community has a good team, not just the release team but beyond, some of this goes to active work that folks all across the project, but especially in the contributor experience SIG are doing to cultivate a positive and inclusive culture here. And you really see that. When problems crop up, you're seeing people jump on and really try to constructively tackle them. And it's really fun to be a part of that. 
+
+
+<hr/>
+
+<i>Thanks to Josh Berkus and Tim Pepper for talking to the Kubernetes Podcast from Google.
+
+[Josh Berkus](https://github.com/jberkus) hangs out in #sig-release on the [Kubernetes Slack](slack.k8s.io). He maintains a newsletter called "[Last Week in Kubernetes Development](http://lwkd.info/)", with Noah Kantrowitz. You can read him on Twitter at [@fuzzychef](https://twitter.com/fuzzychef), but he does warn you that there's a lot of politics there as well.
+
+
+[Tim Pepper](https://github.com/tpepper) is also on Slack - he's always open to folks reaching out with a question, looking for help or advice.  On Twitter you'll find him at [@pythomit](https://twitter.com/pythomit), which is "Timothy P" backwards. Tim is an avid soccer fan and season ticket holder for the [Portland Timbers](https://portlandtimbers.com/) and the [Portland Thorns](https://portlandthorns.com/), so you'll get all sorts of opinions on soccer in addition to technology!
+
+You can find the [Kubernetes Podcast from Google](http://www.kubernetespodcast.com/) at [@kubernetespod](https://twitter.com/KubernetesPod) on Twitter, and you can [subscribe](https://kubernetespodcast.com/subscribe/) so you never miss an episode.</i>
@@ -0,0 +1,310 @@
+---
+layout: blog
+title: "11 Ways (Not) to Get Hacked"
+date:  2018-07-18
+---
+
+**Author**: Andrew Martin (ControlPlane)
+
+Kubernetes security has come a long way since the project&#39;s inception, but still contains some gotchas. Starting with the control plane, building up through workload and network security, and finishing with a projection into the future of security, here is a list of handy tips to help harden your clusters and increase their resilience if compromised.
+
+- [Part One: The Control Plane](#part-one-the-control-plane)
+  * [1. TLS Everywhere](#1-tls-everywhere)
+  * [2. Enable RBAC with Least Privilege, Disable ABAC, and Monitor Logs](#2-enable-rbac-with-least-privilege-disable-abac-and-monitor-logs)
+  * [3. Use Third Party Auth for API Server](#3-use-third-party-auth-for-api-server)
+  * [4. Separate and Firewall your etcd Cluster](#4-separate-and-firewall-your-etcd-cluster)
+  * [5. Rotate Encryption Keys](#5-rotate-encryption-keys)
+- [Part Two: Workloads](#part-two-workloads)
+  * [6. Use Linux Security Features and PodSecurityPolicies](#6-use-linux-security-features-and-podsecuritypolicies)
+  * [7. Statically Analyse YAML](#7-statically-analyse-yaml)
+  * [8. Run Containers as a Non-Root User](#8-run-containers-as-a-non-root-user)
+  * [9. Use Network Policies](#9-use-network-policies)
+  * [10. Scan Images and Run IDS](#10-scan-images-and-run-ids)
+- [Part Three: The Future](#part-three-the-future)
+  * [11. Run a Service Mesh](#11-run-a-service-mesh)
+- [Conclusion](#conclusion)
+
+# Part One: The Control Plane
+
+The control plane is Kubernetes&#39; brain. It has an overall view of every container and pod running on the cluster, can schedule new pods (which can include containers with root access to their parent node), and can read all the secrets stored in the cluster. This valuable cargo needs protecting from accidental leakage and malicious intent: when it&#39;s accessed, when it&#39;s at rest, and when it&#39;s being transported across the network.
+
+## 1. TLS Everywhere
+
+**TLS should be enabled for every component that supports it to prevent traffic sniffing, verify the identity of the server, and (for mutual TLS) verify the identity of the client.**
+
+> Note that some components and installation methods may enable local ports over HTTP and administrators should familiarize themselves with the settings of each component to identify potentially unsecured traffic.
+
+[Source](https://kubernetes.io/docs/tasks/administer-cluster/securing-a-cluster/#use-transport-level-security-tls-for-all-api-traffic)
+
+This network diagram by [Lucas Käldström](https://docs.google.com/presentation/d/1Gp-2blk5WExI_QR59EUZdwfO2BWLJqa626mK2ej-huo/edit#slide=id.g1e639c415b_0_56) demonstrates some of the places TLS should ideally be applied: between every component on the master, and between the Kubelet and API server. [Kelsey Hightower](https://twitter.com/kelseyhightower/)&#39;s canonical [Kubernetes The Hard Way](https://github.com/kelseyhightower/kubernetes-the-hard-way/blob/1.9.0/docs/04-certificate-authority.md) provides detailed manual instructions, as does [etcd&#39;s security model](https://coreos.com/etcd/docs/latest/op-guide/security.html) documentation.
+
+<img src="/images/blog/2018-06-05-11-ways-not-to-get-hacked/kubernetes-control-plane.png" width="800" />
+
+Autoscaling Kubernetes nodes was historically difficult, as each node requires a TLS key to connect to the master, and baking secrets into base images is not good practice. [Kubelet TLS bootstrapping](https://medium.com/@toddrosner/kubernetes-tls-bootstrapping-cf203776abc7) provides the ability for a new kubelet to create a certificate signing request so that certificates are generated at boot time.
+
+<img src="/images/blog/2018-06-05-11-ways-not-to-get-hacked/node-tls-bootstrap.png" width="800" />
+
+## 2. Enable RBAC with Least Privilege, Disable ABAC, and Monitor Logs
+
+**Role-based access control provides fine-grained policy management for user access to resources, such as access to namespaces.**
+
+<img src="/images/blog/2018-06-05-11-ways-not-to-get-hacked/rbac2.png" width="800" />
+
+Kubernetes' ABAC (Attribute Based Access Control) has been [superseded by RBAC](http://blog.kubernetes.io/2017/04/rbac-support-in-kubernetes.html) since release 1.6, and should not be enabled on the API server. Use RBAC instead:
+
+```
+--authorization-mode=RBAC
+```
+
+Or use this flag to disable it in GKE:
+
+```
+--no-enable-legacy-authorization
+```
+
+There are plenty of [good examples](https://docs.bitnami.com/kubernetes/how-to/configure-rbac-in-your-kubernetes-cluster/) of [RBAC policies for cluster services](https://github.com/uruddarraju/kubernetes-rbac-policies), as well as [the docs](https://kubernetes.io/docs/admin/authorization/rbac/#role-binding-examples). And it doesn&#39;t have to stop there - fine-grained RBAC policies can be extracted from audit logs with [audit2rbac](https://github.com/liggitt/audit2rbac).
+
+Incorrect or excessively permissive RBAC policies are a security threat in case of a compromised pod. Maintaining least privilege, and continuously reviewing and improving RBAC rules, should be considered part of the "technical debt hygiene" that teams build into their development lifecycle.
+
+[Audit Logging](https://kubernetes.io/docs/tasks/debug-application-cluster/audit/) (beta in 1.10) provides customisable API logging at the payload (e.g. request and response), and also metadata levels. Log levels can be tuned to your organisation&#39;s security policy - [GKE](https://cloud.google.com/kubernetes-engine/docs/how-to/audit-logging#audit_policy) provides sane defaults to get you started.
+
+For read requests such as get, list, and watch, only the request object is saved in the audit logs; the response object is not. For requests involving sensitive data such as Secret and ConfigMap, only the metadata is exported. For all other requests, both request and response objects are saved in audit logs.
+
+Don&#39;t forget: keeping these logs inside the cluster is a security threat in case of compromise. These, like all other security-sensitive logs, should be transported outside the cluster to prevent tampering in the event of a breach.
+
+## 3. Use Third Party Auth for API Server
+
+**Centralising authentication and authorisation across an organisation (aka Single Sign On) helps onboarding, offboarding, and consistent permissions for users**.
+
+Integrating Kubernetes with third party auth providers (like Google or Github) uses the remote platform&#39;s identity guarantees (backed up by things like 2FA) and prevents administrators having to reconfigure the Kubernetes API server to add or remove users.
+
+[Dex](https://github.com/coreos/dex) is an OpenID Connect Identity (OIDC) and OAuth 2.0 provider with pluggable connectors. Pusher takes this a stage further with [some custom tooling](https://thenewstack.io/kubernetes-single-sign-one-less-identity/), and there are some [other](https://github.com/negz/kuberos) [helpers](https://github.com/micahhausler/k8s-oidc-helper) available with slightly different use cases.
+
+## 4. Separate and Firewall your etcd Cluster
+
+**etcd stores information on state and secrets, and is a critical Kubernetes component - it should be protected differently from the rest of your cluster.**
+
+Write access to the API server&#39;s etcd is equivalent to gaining root on the entire cluster, and even read access can be used to escalate privileges fairly easily.
+
+The Kubernetes scheduler will search etcd for pod definitions that do not have a node. It then sends the pods it finds to an available kubelet for scheduling. Validation for submitted pods is performed by the API server before it writes them to etcd, so malicious users writing directly to etcd can bypass many security mechanisms - e.g. PodSecurityPolicies.
+
+etcd should be configured with [peer and client TLS certificates](https://github.com/coreos/etcd/blob/master/Documentation/op-guide/security.md), and deployed on dedicated nodes. To mitigate against private keys being stolen and used from worker nodes, the cluster can also be firewalled to the API server.
+
+## 5. Rotate Encryption Keys
+
+**A security best practice is to regularly rotate encryption keys and certificates, in order to limit the &quot;blast radius&quot; of a key compromise.**
+
+Kubernetes will [rotate some certificates automatically](https://kubernetes.io/docs/tasks/tls/certificate-rotation/) (notably, the kubelet client and server certs) by creating new CSRs as its existing credentials expire.
+
+However, the [symmetric encryption keys](https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/) that the API server uses to encrypt etcd values are not automatically rotated - they must be [rotated manually](https://www.twistlock.com/2017/08/02/kubernetes-secrets-encryption/). Master access is required to do this, so managed services (such as GKE or AKS) abstract this problem from an operator.
+
+
+# Part Two: Workloads
+
+With minimum viable security on the control plane the cluster is able to operate securely. But, like a ship carrying potentially dangerous cargo, the ship's containers must be protected to contain that cargo in the event of an unexpected accident or breach. The same is true for Kubernetes workloads (pods, deployments, jobs, sets, etc.) - they may be trusted at deployment time, but if they&#39;re internet-facing there&#39;s always a risk of later exploitation. Running workloads with minimal privileges and hardening their runtime configuration can help to mitigate this risk.
+
+## 6. Use Linux Security Features and PodSecurityPolicies
+
+**The Linux kernel has a number of overlapping security extensions (capabilities, SELinux, AppArmor, seccomp-bpf) that can be configured to provide least privilege to applications**.
+
+Tools like [bane](https://github.com/genuinetools/bane) can help to generate AppArmor profiles, and [docker-slim](https://github.com/docker-slim/docker-slim#quick-seccomp-example) for seccomp profiles, but beware - a comprehensive test suite it required to exercise all code paths in your application when verifying the side effects of applying these policies.
+
+[PodSecurityPolicies](https://kubernetes.io/docs/concepts/policy/pod-security-policy/) can be used to mandate the use of security extensions and other Kubernetes security directives. They provide a minimum contract that a pod must fulfil to be submitted to the API server - including security profiles, the privileged flag, and the sharing of host network, process, or IPC namespaces.
+
+These directives are important, as they help to prevent containerised processes from escaping their isolation boundaries, and [Tim Allclair](https://twitter.com/tallclair)&#39;s [example PodSecurityPolicy](https://gist.github.com/tallclair/11981031b6bfa829bb1fb9dcb7e026b0) is a comprehensive resource that you can customise to your use case.
+
+## 7. Statically Analyse YAML
+
+**Where PodSecurityPolicies deny access to the API server, static analysis can also be used in the development workflow to model an organisation&#39;s compliance requirements or risk appetite.**
+
+Sensitive information should not be stored in pod-type YAML resource (deployments, pods, sets, etc.), and sensitive configmaps and secrets should be encrypted with tools such as [vault](https://github.com/coreos/vault-operator) (with CoreOS&#39;s operator), [git-crypt](https://github.com/AGWA/git-crypt), [sealed secrets](https://github.com/bitnami-labs/sealed-secrets), or [cloud provider KMS](https://cloud.google.com/kms/).
+
+Static analysis of YAML configuration can be used to establish a baseline for runtime security. [kubesec](https://kubesec.io/) generates risk scores for resources:
+
+```json
+{
+  "score": -30,
+  "scoring": {
+    "critical": [{
+      "selector": "containers[] .securityContext .privileged == true",
+      "reason": "Privileged containers can allow almost completely unrestricted host access"
+    }],
+    "advise": [{
+      "selector": "containers[] .securityContext .runAsNonRoot == true",
+      "reason": "Force the running image to run as a non-root user to ensure least privilege"
+    }, {
+      "selector": "containers[] .securityContext .capabilities .drop",
+      "reason": "Reducing kernel capabilities available to a container limits its attack surface",
+      "href": "https://kubernetes.io/docs/tasks/configure-pod-container/security-context/"
+    }]
+  }
+}
+```
+
+And [kubetest](https://github.com/garethr/kubetest) is a unit test framework for Kubernetes configurations:
+
+```python
+#// vim: set ft=python:
+def test_for_team_label():
+    if spec["kind"] == "Deployment":
+        labels = spec["spec"]["template"]["metadata"]["labels"]
+        assert_contains(labels, "team", "should indicate which team owns the deployment")
+
+test_for_team_label()
+```
+
+These tools &quot;[shift left](https://en.wikipedia.org/wiki/Shift_left_testing)&quot; (moving checks and verification earlier in the development cycle). Security testing in the development phase gives users fast feedback about code and configuration that may be rejected by a later manual or automated check, and can reduce the friction of introducing more secure practices.
+
+## 8. Run Containers as a Non-Root User
+
+**Containers that run as root frequently have far more permissions than their workload requires which, in case of compromise, could help an attacker further their attack.**
+
+Containers still rely on the traditional Unix security model (called [discretionary access control](https://www.linux.com/learn/overview-linux-kernel-security-features) or DAC) - everything is a file, and permissions are granted to users and groups.
+
+User namespaces are not enabled in Kubernetes. This means that a container&#39;s user ID table maps to the host&#39;s user table, and running a process as the root user inside a container runs it as root on the host. Although we have layered security mechanisms to prevent container breakouts, running as root inside the container is still not recommended.
+
+Many container images use the root user to run PID 1 - if that process is compromised, the attacker has root in the container, and any mis-configurations become much easier to exploit.
+
+[Bitnami has done a lot of work](https://engineering.bitnami.com/articles/running-non-root-containers-on-openshift.html) moving their container images to [non-root users](https://github.com/bitnami/bitnami-docker-nginx/blob/b068b8bd01eb2f5a7314c09466724f86aa4548f9/1.12/Dockerfile#L28) (especially as OpenShift requires this by default), which may ease a migration to non-root container images.
+
+This PodSecurityPolicy snippet prevents running processes as root inside a container, and also escalation to root:
+
+```yaml
+# Required to prevent escalations to root.
+allowPrivilegeEscalation: false
+runAsUser:
+  # Require the container to run without root privileges.
+  rule: 'MustRunAsNonRoot'
+```
+
+Non-root containers cannot bind to the privileged ports under 1024 (this is gated by the CAP\_NET\_BIND\_SERVICE kernel capability), but services can be used to disguise this fact. In this example the fictional MyApp application is bound to port 8443 in its container, but the service exposes it on 443 by proxying the request to the targetPort:
+
+```yaml
+kind: Service
+apiVersion: v1
+metadata:
+  name: my-service
+spec:
+  selector:
+    app: MyApp
+  ports:
+  - protocol: TCP
+    port: 443
+    targetPort: 8443
+```
+
+Having to run workloads as a non-root user is not going to change until user namespaces are usable, or the ongoing work to [run containers without root](https://rootlesscontaine.rs/) lands in container runtimes.
+
+## 9. Use Network Policies
+
+**By default, Kubernetes networking allows all pod to pod traffic; this can be restricted using a** [**Network Policy**](https://kubernetes.io/docs/concepts/services-networking/network-policies/) **.**
+
+<img src="/images/blog/2018-06-05-11-ways-not-to-get-hacked/kubernetes-networking.png" width="800" />
+
+Traditional services are restricted with firewalls, which use static IP and port ranges for each service. As these IPs very rarely change they have historically been used as a form of identity. Containers rarely have static IPs - they are built to fail fast, be rescheduled quickly, and use service discovery instead of static IP addresses. These properties mean that firewalls become much more difficult to configure and review.
+
+As Kubernetes stores all its system state in etcd it can configure dynamic firewalling - if it is supported by the CNI networking plugin. Calico, Cilium, kube-router, Romana, and Weave Net all support network policy.
+
+It should be noted that these policies fail-closed, so the absence of a podSelector here defaults to a wildcard:
+
+```yaml
+apiVersion: networking.k8s.io/v1
+kind: NetworkPolicy
+metadata:
+  name: default-deny
+spec:
+  podSelector:
+```
+
+Here&#39;s an example NetworkPolicy that denies all egress except UDP 53 (DNS), which also prevents inbound connections to your application. [NetworkPolicies are stateful](https://www.weave.works/blog/securing-microservices-kubernetes/), so the replies to outbound requests still reach the application.
+
+```yaml
+apiVersion: networking.k8s.io/v1
+kind: NetworkPolicy
+metadata:
+  name: myapp-deny-external-egress
+spec:
+  podSelector:
+    matchLabels:
+      app: myapp
+  policyTypes:
+  - Egress
+  egress:
+  - ports:
+    - port: 53
+      protocol: UDP
+  - to:
+    - namespaceSelector: {}
+```
+
+Kubernetes network policies can not be applied to DNS names. This is because DNS can resolve round-robin to many IPs, or dynamically based on the calling IP, so network policies can be applied to a fixed IP or podSelector (for dynamic Kubernetes IPs) only.
+
+Best practice is to start by denying all traffic for a namespace and incrementally add routes to allow an application to pass its acceptance test suite. This can become complex, so ControlPlane hacked together [netassert](https://github.com/controlplaneio/netassert) - network security testing for DevSecOps workflows with highly parallelised nmap:
+
+```yaml
+k8s: # used for Kubernetes pods
+  deployment: # only deployments currently supported
+    test-frontend: # pod name, defaults to `default` namespace
+      test-microservice: 80  # `test-microservice` is the DNS name of the target service
+      test-database: -80     # `test-frontend` should not be able to access test-database’s port 80
+      169.254.169.254: -80, -443           # AWS metadata API
+      metadata.google.internal: -80, -443  # GCP metadata API
+
+    new-namespace:test-microservice:  # `new-namespace` is the namespace name
+      test-database.new-namespace: 80 # longer DNS names can be used for other namespaces
+      test-frontend.default: 80
+      169.254.169.254: -80, -443           # AWS metadata API
+      metadata.google.internal: -80, -443  # GCP metadata API
+```
+
+Cloud provider metadata APIs are a constant source of escalation (as the recent [Shopify](https://hackerone.com/reports/341876) [bug bounty](https://hackerone.com/reports/341876) demonstrates), so specific tests to confirm that the APIs are blocked on the container network helps to guard against accidental misconfiguration.
+
+## 10. Scan Images and Run IDS
+
+**Web servers present an attack surface to the network they&#39;re attached to: scanning an image&#39;s installed files ensures the absence of known vulnerabilities that an attacker could exploit to gain remote access to the container. An IDS (Intrusion Detection System) detects them if they do.**
+
+Kubernetes permits pods into the cluster through a series of [admission controller](https://kubernetes.io/docs/admin/admission-controllers/) gates, which are applied to pods and other resources like deployments. These gates can validate each pod for admission or change its contents, and they now support backend webhooks.
+
+<img src="/images/blog/2018-06-05-11-ways-not-to-get-hacked/admission-controllers.png" width="800" />
+
+These webhooks can be used by container image scanning tools to validate images before they are deployed to the cluster. Images that have failed checks can be refused admission.
+
+Scanning container images for known vulnerabilities can reduce the window of time that an attacker can exploit a disclosed CVE. Free tools such as CoreOS&#39;s [Clair](https://github.com/coreos/clair) and Aqua&#39;s [Micro Scanner](https://github.com/aquasecurity/microscanner) should be used in a deployment pipeline to prevent the deployment of images with critical, exploitable vulnerabilities.
+
+Tools such as [Grafeas](https://grafeas.io/) can store image metadata for constant compliance and vulnerability checks against a container&#39;s unique signature (a [content addressable](https://en.wikipedia.org/wiki/Content-addressable_storage) hash). This means that scanning a container image with that hash is the same as scanning the images deployed in production, and can be done continually without requiring access to production environments.
+
+Unknown Zero Day vulnerabilities will always exist, and so intrusion detection tools such as [Twistlock](https://www.twistlock.com/), [Aqua](https://www.aquasec.com/), and [Sysdig Secure](https://sysdig.com/product/secure/) should be deployed in Kubernetes. IDS detects unusual behaviours in a container and pauses or kills it - [Sysdig&#39;s Falco](https://github.com/draios/falco) is a an Open Source rules engine, and an entrypoint to this ecosystem.
+
+# Part Three: The Future
+
+The next stage of security&#39;s &quot;cloud native evolution&quot; looks to be the service mesh, although adoption may take time - migration involves shifting complexity from applications to the mesh infrastructure, and organisations will be keen to understand best-practice.
+
+<img src="/images/blog/2018-06-05-11-ways-not-to-get-hacked/service-mesh-@sebiwicb.png" width="800" />
+
+## 11. Run a Service Mesh
+
+**A service mesh is a web of encrypted persistent connections, made between high performance &quot;sidecar&quot; proxy servers like Envoy and Linkerd. It adds traffic management, monitoring, and policy - all without microservice changes.**
+
+Offloading microservice security and networking code to a shared, battle tested set of libraries was already possible with [Linkerd](https://linkerd.io/), and the introduction of [Istio](https://istio.io/) by Google, IBM, and Lyft, has added an alternative in this space. With the addition of [SPIFFE](https://spiffe.io) for per-pod cryptographic identity and a plethora of [other features](https://istio.io/docs/concepts/what-is-istio/overview.html), Istio could simplify the deployment of the next generation of network security.
+
+In &quot;Zero Trust&quot; networks there may be no need for traditional firewalling or Kubernetes network policy, as every interaction occurs over mTLS (mutual TLS), ensuring that both parties are not only communicating securely, but that the identity of both services is known.
+
+This shift from traditional networking to Cloud Native security principles is not one we expect to be easy for those with a traditional security mindset, and the [Zero Trust Networking book](https://amzn.to/2Gg6Pav) from SPIFFE&#39;s [Evan Gilman](https://twitter.com/evan2645) is a highly recommended introduction to this brave new world.
+
+Istio [0.8 LTS](https://istio.io/about/notes/0.8/) is out, and the project is rapidly approaching a 1.0 release. Its stability versioning is the same as the Kubernetes model: a stable core, with individual APIs identifying themselves under their own alpha/beta stability namespace. Expect to see an uptick in Istio adoption over the coming months.
+
+# Conclusion
+
+Cloud Native applications have a more fine-grained set of lightweight security primitives to lock down workloads and infrastructure. The power and flexibility of these tools is both a blessing and curse - with insufficient automation it has become easier to expose insecure workloads which permit breakouts from the container or its isolation model.
+
+There are more defensive tools available than ever, but caution must be taken to reduce attack surfaces and the potential for misconfiguration.
+
+However if security slows down an organisation&#39;s pace of feature delivery it will never be a first-class citizen. Applying Continuous Delivery principles to the software supply chain allows an organisation to achieve compliance, continuous audit, and enforced governance without impacting the business&#39;s bottom line.
+
+Iteratating quickly on security is easiest when supported by a comprehensive test suite. This is achieved with Continuous Security - an alternative to point-in-time penetration tests, with constant pipeline validation ensuring an organisation&#39;s attack surface is known, and the risk constantly understood and managed.
+
+This is ControlPlane&#39;s modus operandi: if we can help kickstart a Continuous Security discipline, deliver Kubernetes security and operations training, or co-implement a secure cloud native evolution for you, please [get in touch](https://control-plane.io).
+
+---
+
+Andrew Martin is a co-founder at [@controlplaneio](https://twitter.com/controlplaneio) and tweets about cloud native security at [@sublimino](https://twitter.com/sublimino)
@@ -0,0 +1,28 @@
+---
+layout: blog
+title: "Kubernetes Wins the 2018 OSCON Most Impact Award"
+date:  2018-07-19
+slug: kubernetes-wins-2018-oscon-most-impact-award
+---
+
+**Authors**: Brian Grant (Principal Engineer, Google) and Tim Hockin (Principal Engineer, Google)
+
+We are humbled to be recognized by the community with this award.
+
+We had high hopes when we created Kubernetes. We wanted to change the way cloud applications were deployed and managed. Whether we’d succeed or not was very uncertain. And look how far we’ve come in such a short time.
+
+The core technology behind Kubernetes was informed by [lessons learned from Google’s internal infrastructure](https://ai.google/research/pubs/pub44843), but nobody can deny the enormous role of the Kubernetes community in the success of the project. [The community, of which Google is a part](https://k8s.devstats.cncf.io/d/8/company-statistics-by-repository-group?orgId=1), now drives every aspect of the project: the design, development, testing, documentation, releases, and more. That is what makes Kubernetes fly.
+
+While we actively sought partnerships and community engagement, none of us anticipated just how important the open-source community would be, how fast it would grow, or how large it would become. Honestly,  we really didn’t have much of a plan.
+
+We looked to other open-source projects for inspiration and advice: Docker (now Moby), other open-source projects at Google such as Angular and Go, the Apache Software Foundation, OpenStack, Node.js, Linux, and others. But it became clear that there was no clear-cut recipe we could follow. So we winged it.
+
+Rather than rehashing history, we thought we’d share two high-level lessons we learned along the way.
+
+First, in order to succeed, community health and growth needs to be treated as a top priority. It’s hard, and it is time-consuming. It requires attention to both internal project dynamics and outreach, as well as constant vigilance to build and sustain relationships, be inclusive, maintain open communication, and remain responsive to contributors and users. Growing existing contributors and onboarding new ones is critical to sustaining project growth, but that takes time and energy that might otherwise be spent on development. These things have to become core values in order for contributors to keep them going.
+
+Second, start simple with how the project is organized and operated, but be ready to adopt to more scalable approaches as it grows. Over time, Kubernetes has transitioned from what was effectively a single team and git repository to many subgroups (Special Interest Groups and Working Groups), sub-projects, and repositories. From manual processes to fully automated ones. From informal policies to formal governance.
+
+We certainly didn’t get everything right or always adapt quickly enough, and we constantly struggle with scale. [At this point](https://k8s.devstats.cncf.io/d/24/overall-project-statistics?orgId=1), Kubernetes has more than 20,000 contributors and is approaching one million comments on its issues and pull requests, [making it one of the fastest moving projects in the history of open source](https://www.cncf.io/blog/2017/02/27/measuring-popularity-kubernetes-using-bigquery/).
+
+Thank you to all our contributors and to all the users who’ve stuck with us on the sometimes bumpy journey. This project would not be what it is today without the community.
@@ -0,0 +1,62 @@
+---
+layout: blog
+title: "The History of Kubernetes & the Community Behind It"
+date:  2018-07-20
+---
+
+**Authors**: Brendan Burns (Distinguished Engineer, Microsoft)
+
+![oscon award](/images/blog/2018-07-20-history-kubernetes-community.png)
+
+It is remarkable to me to return to Portland and OSCON to stand on stage with members of the Kubernetes community and accept this award for Most Impactful Open Source Project. It was scarcely three years ago, that on this very same stage we declared Kubernetes 1.0 and the project was added to the newly formed Cloud Native Computing Foundation.
+
+To think about how far we have come in that short period of time and to see the ways in which this project has shaped the cloud computing landscape is nothing short of amazing. The success is a testament to the power and contributions of this amazing open source community. And the daily passion and quality contributions of our endlessly engaged, world-wide community is nothing short of humbling.
+
+<blockquote class="twitter-tweet" data-lang="en"><p lang="en" dir="ltr">Congratulations <a href="https://twitter.com/kubernetesio?ref_src=twsrc%5Etfw">@kubernetesio</a> for winning the &quot;most impact&quot; award at <a href="https://twitter.com/hashtag/OSCON?src=hash&amp;ref_src=twsrc%5Etfw">#OSCON</a> I&#39;m so proud to be a part of this amazing community! <a href="https://twitter.com/CloudNativeFdn?ref_src=twsrc%5Etfw">@CloudNativeFdn</a> <a href="https://t.co/5sRUYyefAK">pic.twitter.com/5sRUYyefAK</a></p>&mdash; Jaice Singer DuMars (@jaydumars) <a href="https://twitter.com/jaydumars/status/1019993233487613952?ref_src=twsrc%5Etfw">July 19, 2018</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
+
+<blockquote class="twitter-tweet" data-lang="en"><p lang="en" dir="ltr">👏 congrats <a href="https://twitter.com/kubernetesio?ref_src=twsrc%5Etfw">@kubernetesio</a> community on winning the <a href="https://twitter.com/hashtag/oscon?src=hash&amp;ref_src=twsrc%5Etfw">#oscon</a> Most Impact Award, we are proud of you! <a href="https://t.co/5ezDphi6J6">pic.twitter.com/5ezDphi6J6</a></p>&mdash; CNCF (@CloudNativeFdn) <a href="https://twitter.com/CloudNativeFdn/status/1019996928296095744?ref_src=twsrc%5Etfw">July 19, 2018</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
+
+
+At a meetup in Portland this week, I had a chance to tell the story of Kubernetes’ past, its present and some thoughts about its future, so I thought I would write down some pieces of what I said for those of you who couldn’t be there in person.
+
+It all began in the fall of 2013, with three of us: Craig McLuckie, Joe Beda and I were working on public cloud infrastructure. If you cast your mind back to the world of cloud in 2013, it was a vastly different place than it is today. Imperative bash scripts were only just starting to give way to declarative configuration of IaaS with systems. Netflix was popularizing the idea of immutable infrastructure but doing it with heavy-weight full VM images. The notion of orchestration, and certainly container orchestration existed in a few internet scale companies, but not in cloud and certainly not in the enterprise.
+
+Docker changed all of that. By popularizing a lightweight container runtime and providing a simple way to package, distributed and deploy applications onto a machine, the Docker tooling and experience popularized a brand-new cloud native approach to application packaging and maintenance. Were it not for Docker’s shifting of the cloud developer’s perspective, Kubernetes simply would not exist.
+
+I think that it was Joe who first suggested that we look at Docker in the summer of 2013, when Craig, Joe and I were all thinking about how we could bring a cloud native application experience to a broader audience. And for all three of us, the implications of this new tool were immediately obvious. We knew it was a critical component in the development of cloud native infrastructure.
+
+But as we thought about it, it was equally obvious that Docker, with its focus on a single machine, was not the complete solution. While Docker was great at building and packaging individual containers and running them on individual machines, there was a clear need for an orchestrator that could deploy and manage large numbers of containers across a fleet of machines.
+
+As we thought about it some more, it became increasingly obvious to Joe, Craig and I, that not only was such an orchestrator necessary, it was also inevitable, and it was equally inevitable that this orchestrator would be open source. This realization crystallized for us in the late fall of 2013, and thus began the rapid development of first a prototype, and then the system that would eventually become known as Kubernetes. As 2013 turned into 2014 we were lucky to be joined by some incredibly talented developers including Ville Aikas, Tim Hockin, Dawn Chen, Brian Grant and Daniel Smith.
+
+<blockquote class="twitter-tweet" data-lang="en"><p lang="en" dir="ltr">Happy to see k8s team members winning the “most impact” award. <a href="https://twitter.com/hashtag/oscon?src=hash&amp;ref_src=twsrc%5Etfw">#oscon</a> <a href="https://t.co/D6mSIiDvsU">pic.twitter.com/D6mSIiDvsU</a></p>&mdash; Bridget Kromhout (@bridgetkromhout) <a href="https://twitter.com/bridgetkromhout/status/1019992441825341440?ref_src=twsrc%5Etfw">July 19, 2018</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
+
+<blockquote class="twitter-tweet" data-lang="en"><p lang="en" dir="ltr">Kubernetes won the O&#39;Reilly Most Impact Award. Thanks to our contributors and users! <a href="https://t.co/T6Co1wpsAh">pic.twitter.com/T6Co1wpsAh</a></p>&mdash; Brian Grant (@bgrant0607) <a href="https://twitter.com/bgrant0607/status/1019995276235325440?ref_src=twsrc%5Etfw">July 19, 2018</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
+
+
+The initial goal of this small team was to develop a “minimally viable orchestrator.” From experience we knew that the basic feature set for such an orchestrator was:
+
+  * Replication to deploy multiple instances of an application
+  * Load balancing and service discovery to route traffic to these replicated containers
+  * Basic health checking and repair to ensure a self-healing system
+  * Scheduling to group many machines into a single pool and distribute work to them
+
+Along the way, we also spent a significant chunk of our time convincing executive leadership that open sourcing this project was a good idea. I’m endlessly grateful to Craig for writing numerous whitepapers and to Eric Brewer, for the early and vocal support that he lent us to ensure that Kubernetes could see the light of day.
+
+In June of 2014 when Kubernetes was released to the world, the list above was the sum total of its basic feature set. As an early stage open source community, we then spent a year building, expanding, polishing and fixing this initial minimally viable orchestrator into the product that we released as a 1.0 in OSCON in 2015. We were very lucky to be joined early on by the very capable OpenShift team which lent significant engineering and real world enterprise expertise to the project. Without their perspective and contributions, I don’t think we would be standing here today.
+
+Three years later, the Kubernetes community has grown exponentially, and Kubernetes has become synonymous with cloud native container orchestration. There are more than 1700 people who have contributed to Kubernetes, there are more than 500 Kubernetes meetups worldwide and more than 42000 users have joined the #kubernetes-dev channel. What’s more, the community that we have built works successfully across geographic, language and corporate boundaries. It is a truly open, engaged and collaborative community, and in-and-of-itself and amazing achievement. Many thanks to everyone who has helped make it what it is today. Kubernetes is a commodity in the public cloud because of you.
+
+But if Kubernetes is a commodity, then what is the future? Certainly, there are an endless array of tweaks, adjustments and improvements to the core codebase that will occupy us for years to come, but the true future of Kubernetes are the applications and experiences that are being built on top of this new, ubiquitous platform.  
+
+Kubernetes has dramatically reduced the complexity to build new developer experiences, and a myriad of new experiences have been developed or are in the works that provide simplified or targeted developer experiences like Functions-as-a-Service, on top of core Kubernetes-as-a-Service.  
+
+The Kubernetes cluster itself is being extended with custom resource definitions (which I first described to Kelsey Hightower on a walk from OSCON to a nearby restaurant in 2015), these new resources allow cluster operators to enable new plugin functionality that extend and enhance the APIs that their users have access to.
+
+By embedding core functionality like logging and monitoring in the cluster itself and enabling developers to take advantage of such services simply by deploying their application into the cluster, Kubernetes has reduced the learning necessary for developers to build scalable reliable applications.
+
+Finally, Kubernetes has provided a new, common vocabulary for expressing the patterns and paradigms of distributed system development. This common vocabulary means that we can more easily describe and discuss the common ways in which our distributed systems are built, and furthermore we can build standardized, re-usable implementations of such systems. The net effect of this is the development of higher quality, reliable distributed systems, more quickly.
+
+It’s truly amazing to see how far Kubernetes has come, from a rough idea in the minds of three people in Seattle to a phenomenon that has redirected the way we think about cloud native development across the world. It has been an amazing journey, but what’s truly amazing to me, is that I think we’re only just now scratching the surface of the impact that Kubernetes will have. Thank you to everyone who has enabled us to get this far, and thanks to everyone who will take us further.
+
+Brendan
@@ -0,0 +1,160 @@
+---
+layout: blog
+title: "Feature Highlight: CPU Manager"
+date:  2018-07-24
+---
+
+**Authors**: Balaji Subramaniam ([Intel](mailto:balaji.subramaniam@intel.com)), Connor Doyle ([Intel](mailto:connor.p.doyle@intel.com))
+
+This blog post describes the [CPU Manager](https://kubernetes.io/docs/tasks/administer-cluster/cpu-management-policies/), a beta feature in [Kubernetes](https://kubernetes.io/). The CPU manager feature enables better placement of workloads in the [Kubelet](https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/), the Kubernetes node agent, by allocating exclusive CPUs to certain pod containers.
+
+![cpu manager](/images/blog/2018-07-24-cpu-manager/cpu-manager.png)
+
+## Sounds Good! But Does the CPU Manager Help Me?
+
+It depends on your workload. A single compute node in a Kubernetes cluster can run many [pods](https://kubernetes.io/docs/concepts/workloads/pods/pod/) and some of these pods could be running CPU-intensive workloads. In such a scenario, the pods might contend for the CPU resources available in that compute node. When this contention intensifies, the workload can move to different CPUs depending on whether the pod is throttled and the availability of CPUs at scheduling time. There might also be cases where the workload could be sensitive to context switches. In all the above scenarios, the performance of the workload might be affected.
+
+If your workload is sensitive to such scenarios, then CPU Manager can be enabled to provide better performance isolation by allocating exclusive CPUs for your workload.
+
+CPU manager might help workloads with the following characteristics:
+
+* Sensitive to CPU throttling effects.
+* Sensitive to context switches.
+* Sensitive to processor cache misses.
+* Benefits from sharing a processor resources (e.g., data and instruction caches).
+* Sensitive to cross-socket memory traffic.
+* Sensitive or requires hyperthreads from the same physical CPU core.
+
+## Ok! How Do I use it?
+
+Using the CPU manager is simple. First, [enable CPU manager with the Static policy](https://kubernetes.io/docs/tasks/administer-cluster/cpu-management-policies/#cpu-management-policies) in the Kubelet running on the compute nodes of your cluster. Then configure your pod to be in the [Guaranteed Quality of Service (QoS) class](https://kubernetes.io/docs/tasks/configure-pod-container/quality-service-pod/#create-a-pod-that-gets-assigned-a-qos-class-of-guaranteed). Request whole numbers of CPU cores (e.g., `1000m`, `4000m`) for containers that need exclusive cores. Create your pod in the same way as before (e.g., `kubectl create -f pod.yaml`). And _voilà_, the CPU manager will assign exclusive CPUs to each of container in the pod according to their CPU requests.
+
+```
+apiVersion: v1
+kind: Pod
+metadata:
+  name: exclusive-2
+spec:
+  containers:
+  - image: quay.io/connordoyle/cpuset-visualizer
+    name: exclusive-2
+    resources:
+      # Pod is in the Guaranteed QoS class because requests == limits
+      requests:
+        # CPU request is an integer
+        cpu: 2
+        memory: "256M"
+      limits:
+        cpu: 2
+        memory: "256M"
+```      
+_Pod specification requesting two exclusive CPUs._
+
+## Hmm … How Does the CPU Manager Work?
+
+For Kubernetes, and the purposes of this blog post, we will discuss three kinds of CPU resource controls available in most Linux distributions. The first two are CFS shares (what's my weighted fair share of CPU time on this system) and CFS quota (what's my hard cap of CPU time over a period). The CPU manager uses a third control called CPU affinity (on what logical CPUs am I allowed to execute).
+
+By default, all the pods and the containers running on a compute node of your Kubernetes cluster can execute on any available cores in the system. The total amount of allocatable shares and quota are limited by the CPU resources explicitly [reserved for kubernetes and system daemons](https://kubernetes.io/docs/tasks/administer-cluster/reserve-compute-resources/). However, limits on the CPU time being used can be specified using [CPU limits in the pod spec](https://kubernetes.io/docs/tasks/configure-pod-container/assign-cpu-resource/#specify-a-cpu-request-and-a-cpu-limit). Kubernetes uses [CFS quota](https://www.kernel.org/doc/Documentation/scheduler/sched-bwc.txt) to enforce CPU limits on pod containers.
+
+When CPU manager is enabled with the "static" policy, it manages a shared pool of CPUs. Initially this shared pool contains all the CPUs in the compute node. When a container with integer CPU request in a Guaranteed pod is created by the Kubelet, CPUs for that container are removed from the shared pool and assigned exclusively for the lifetime of the container. Other containers are migrated off these exclusively allocated CPUs.
+
+All non-exclusive-CPU containers (Burstable, BestEffort and Guaranteed with non-integer CPU) run on the CPUs remaining in the shared pool. When a container with exclusive CPUs terminates, its CPUs are added back to the shared CPU pool.
+
+## More Details Please ...
+
+![cpu manager](/images/blog/2018-07-24-cpu-manager/cpu-manager-anatomy.png)
+
+The figure above shows the anatomy of the CPU manager. The CPU Manager uses the Container Runtime Interface's `UpdateContainerResources` method to modify the CPUs on which containers can run. The Manager periodically reconciles the current State of the CPU resources of each running container with `cgroupfs`.
+
+The CPU Manager uses [Policies](https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/cm/cpumanager/policy.go#L25) to decide the allocation of CPUs. There are two policies implemented: None and Static. By default, the CPU manager is enabled with the None policy from Kubernetes version 1.10.
+
+The Static policy allocates exclusive CPUs to pod containers in the Guaranteed QoS class which request integer CPUs. On a best-effort basis, the Static policy tries to allocate CPUs topologically in the following order:
+
+1. Allocate all the CPUs in the same processor socket if available and the container requests at least an entire socket worth of CPUs.
+2. Allocate all the logical CPUs (hyperthreads) from the same physical CPU core if available and the container requests an entire core worth of CPUs.
+3. Allocate any available logical CPU, preferring to acquire CPUs from the same socket.
+
+## How is Performance Isolation Improved by CPU Manager?
+
+With CPU manager static policy enabled, the workloads might perform better due to one of the following reasons:
+
+1. Exclusive CPUs can be allocated for the workload container but not the other containers. These containers do not share the CPU resources. As a result, we expect better performance due to isolation when an aggressor or a co-located workload is involved.
+2. There is a reduction in interference between the resources used by the workload since we can partition the CPUs among workloads. These resources might also include the cache hierarchies and memory bandwidth and not just the CPUs. This helps improve the performance of workloads in general.
+3. CPU Manager allocates CPUs in a topological order on a best-effort basis. If a whole socket is free, the CPU Manager will exclusively allocate the CPUs from the free socket to the workload. This boosts the performance of the workload by avoiding any cross-socket traffic.
+4. Containers in Guaranteed QoS pods are subject to CFS quota. Very bursty workloads may get scheduled, burn through their quota before the end of the period, and get throttled. During this time, there may or may not be meaningful work to do with those CPUs. Because of how the resource math lines up between CPU quota and number of exclusive CPUs allocated by the static policy, these containers are not subject to CFS throttling (quota is equal to the maximum possible cpu-time over the quota period).
+
+## Ok! Ok! Do You Have Any Results?
+
+Glad you asked! To understand the performance improvement and isolation provided by enabling the CPU Manager feature in the Kubelet, we ran experiments on a dual-socket compute node (Intel Xeon CPU E5-2680 v3) with hyperthreading enabled. The node consists of 48 logical CPUs (24 physical cores each with 2-way hyperthreading). Here we demonstrate the performance benefits and isolation provided by the CPU Manager feature using benchmarks and real-world workloads for three different scenarios.
+
+### How Do I Interpret the Plots?
+
+For each scenario, we show box plots that illustrates the normalized execution time and its variability of running a benchmark or real-world workload with and without CPU Manager enabled. The execution time of the runs are normalized to the best-performing run (1.00 on y-axis represents the best performing run and lower is better). The height of the box plot shows the variation in performance. For example if the box plot is a line, then there is no variation in performance across runs. In the box, middle line is the median, upper line is 75th percentile and lower line is 25th percentile. The height of the box (i.e., difference between 75th and 25th percentile) is defined as the interquartile range (IQR). Whiskers shows data outside that range and the points show outliers. The outliers are defined as any data 1.5x IQR below or above the lower or upper quartile respectively. Every experiment is run ten times.
+
+### Protection from Aggressor Workloads
+
+We ran six benchmarks from the [PARSEC benchmark suite](http://parsec.cs.princeton.edu/) (the victim workloads) co-located with a CPU stress container (the aggressor workload) with and without the CPU Manager feature enabled. The CPU stress container is run [as a pod](https://gist.github.com/balajismaniam/7c2d57b2f526a56bb79cf870c122a34c) in the Burstable QoS class requesting 23 CPUs with `--cpus 48` flag. [The benchmarks are run as pods](https://gist.github.com/balajismaniam/fac7923f6ee44f1f36969c29354e3902) in the Guaranteed QoS class requesting a full socket worth of CPUs (24 CPUs on this system). The figure below plots the normalized execution time of running a benchmark pod co-located with the stress pod, with and without the CPU Manager static policy enabled. We see improved performance and reduced performance variability when static policy is enabled for all test cases.
+
+![execution time](/images/blog/2018-07-24-cpu-manager/execution-time.png)
+
+### Performance Isolation for Co-located Workloads
+
+In this section, we demonstrate how CPU manager can be beneficial to multiple workloads in a co-located workload scenario. In the box plots below we show the performance of two benchmarks (Blackscholes and Canneal) from the PARSEC benchmark suite run in the Guaranteed (Gu) and Burstable (Bu) QoS classes co-located with each other, with and without the CPU manager static policy enabled.
+
+Starting from the top left and proceeding clockwise, we show the performance of Blackscholes in the Bu QoS class (top left), Canneal in the Bu QoS class (top right), Canneal in Gu QoS class (bottom right) and Blackscholes in the Gu QoS class (bottom left, respectively. In each case, they are co-located with Canneal in the Gu QoS class (top left), Blackscholes in the Gu QoS class (top right), Blackscholes in the Bu QoS class (bottom right) and Canneal in the Bu QoS class (bottom left) going clockwise from top left, respectively. For example, Bu-blackscholes-Gu-canneal plot (top left) is showing the performance of Blackscholes running in the Bu QoS class when co-located with Canneal running in the Gu QoS class. In each case, the pod in Gu QoS class requests cores worth a whole socket (i.e., 24 CPUs) and the pod in Bu QoS class request 23 CPUs.
+
+There is better performance and less performance variation for both the co-located workloads in all the tests. For example, consider the case of Bu-blackscholes-Gu-canneal (top left) and Gu-canneal-Bu-blackscholes (bottom right). They show the performance of Blackscholes and Canneal run simultaneously with and without the CPU manager enabled. In this particular case, Canneal gets exclusive cores due to CPU manager since it is in the Gu QoS class and requesting integer number of CPU cores. But Blackscholes also gets exclusive set of CPUs as it is the only workload in the shared pool. As a result, both Blackscholes and Canneal get some performance isolation benefits due to the CPU manager.
+
+![performance comparison](/images/blog/2018-07-24-cpu-manager/performance-comparison.png)
+
+### Performance Isolation for Stand-Alone Workloads
+
+This section shows the performance improvement and isolation provided by the CPU manager for stand-alone real-world workloads. We use two workloads from the [TensorFlow official models](https://github.com/tensorflow/models/tree/master/official): [wide and deep](https://github.com/tensorflow/models/tree/master/official/wide_deep) and [ResNet](https://github.com/tensorflow/models/tree/master/official/resnet). We use the census and CIFAR10 dataset for the wide and deep and ResNet models respectively. In each case the [pods](https://gist.github.com/balajismaniam/941db0d0ec14e2bc93b7dfe04d1f6c58) ([wide and deep](https://gist.github.com/balajismaniam/9953b54dd240ecf085b35ab1bc283f3c), [ResNet](https://gist.github.com/balajismaniam/a1919010fe9081ca37a6e1e7b01f02e3) request 24 CPUs which corresponds to a whole socket worth of cores. As shown in the plots, CPU manager enables better performance isolation in both cases.
+
+![performance comparison](/images/blog/2018-07-24-cpu-manager/performance-comparison-2.png)
+
+## Limitations
+
+Users might want to get CPUs allocated on the socket near to the bus which connects to an external device, such as an accelerator or high-performance network card, in order to avoid cross-socket traffic. This type of alignment is not yet supported by CPU manager.
+Since the CPU manager provides a best-effort allocation of CPUs belonging to a socket and physical core, it is susceptible to corner cases and might lead to fragmentation.
+The CPU manager does not take the isolcpus Linux kernel boot parameter into account, although this is reportedly common practice for some low-jitter use cases.
+
+## Acknowledgements
+
+We thank the members of the community who have contributed to this feature or given feedback including members of WG-Resource-Management and SIG-Node.
+cmx.io (for the fun drawing tool).
+
+#### Notices and Disclaimers
+
+Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more information go to www.intel.com/benchmarks.
+
+Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. No computer system can be absolutely secure. Check with your system manufacturer or retailer or learn more at intel.com.
+
+Workload Configuration:
+https://gist.github.com/balajismaniam/fac7923f6ee44f1f36969c29354e3902
+https://gist.github.com/balajismaniam/7c2d57b2f526a56bb79cf870c122a34c
+https://gist.github.com/balajismaniam/941db0d0ec14e2bc93b7dfe04d1f6c58
+https://gist.github.com/balajismaniam/a1919010fe9081ca37a6e1e7b01f02e3
+https://gist.github.com/balajismaniam/9953b54dd240ecf085b35ab1bc283f3c
+
+System Configuration:
+CPU
+ Architecture:          x86_64
+ CPU op-mode(s):        32-bit, 64-bit
+ Byte Order:            Little Endian
+ CPU(s):                48
+ On-line CPU(s) list:   0-47
+ Thread(s) per core:    2
+ Core(s) per socket:    12
+ Socket(s):             2
+ NUMA node(s):          2
+ Vendor ID:             GenuineIntel
+ Model name:            Intel(R) Xeon(R) CPU E5-2680 v3
+Memory
+ 256 GB
+OS/Kernel
+ Linux 3.10.0-693.21.1.el7.x86_64
+
+Intel, the Intel logo, Xeon are trademarks of Intel Corporation or its subsidiaries in the U.S. and/or other countries.  
+*Other names and brands may be claimed as the property of others.
+© Intel Corporation.
@@ -0,0 +1,107 @@
+---
+layout: blog
+title:  'KubeVirt: Extending Kubernetes with CRDs for Virtualized Workloads'
+date:   2018-07-27
+---
+
+**Author**: David Vossel (Red Hat)
+
+## What is KubeVirt?
+
+[KubeVirt](https://github.com/kubevirt/kubevirt) is a Kubernetes addon that provides users the ability to schedule traditional virtual machine workloads side by side with container workloads. Through the use of [Custom Resource Definitions](https://Kubernetes.io/docs/concepts/extend-Kubernetes/api-extension/custom-resources/) (CRDs) and other Kubernetes features, KubeVirt seamlessly extends existing Kubernetes clusters to provide a set of virtualization APIs that can be used to manage virtual machines.
+
+## Why Use CRDs Over an Aggregated API Server?
+
+Back in the middle of 2017, those of us working on KubeVirt were at a crossroads. We had to make a decision whether or not to extend Kubernetes using an aggregated API server or to make use of the new Custom Resource Definitions (CRDs) feature.
+
+At the time, CRDs lacked much of the functionality we needed to deliver our feature set. The ability to create our own aggregated API server gave us all the flexibility we needed, but it had one major flaw. **An aggregated API server significantly increased the complexity involved with installing and operating KubeVirt.**
+
+The crux of the issue for us was that aggregated API servers required access to etcd for object persistence. This meant that cluster admins would have to either accept that KubeVirt needs a separate etcd deployment which increases complexity, or provide KubeVirt with shared access to the Kubernetes etcd store which introduces risk.
+
+We weren’t okay with this tradeoff. Our goal wasn’t to just extend Kubernetes to run virtualization workloads, it was to do it in the most seamless and effortless way possible. We felt that the added complexity involved with an aggregated API server sacrificed the part of the user experience involved with installing and operating KubeVirt.
+ 
+**Ultimately we chose to go with CRDs and trust that the Kubernetes ecosystem would grow with us to meet the needs of our use case.** Our bets were well placed. At this point there are either solutions in place or solutions under discussion that solve every feature gap we encountered back in 2017 when were evaluating CRDs vs an aggregated API server. 
+
+## Building Layered “Kubernetes like” APIs with CRDs
+
+We designed KubeVirt’s API to follow the same patterns users are already familiar with in the Kubernetes core API. 
+
+For example, in Kubernetes the lowest level unit that users create to perform work is a Pod. Yes, Pods do have multiple containers but logically the Pod is the unit at the bottom of the stack. A Pod represents a mortal workload. The Pod gets scheduled, eventually the Pod’s workload terminates, and that’s the end of the Pod’s lifecycle.
+
+Workload controllers such as the ReplicaSet and StatefulSet are layered on top of the Pod abstraction to help manage scale out and stateful applications. From there we have an even higher level controller called a Deployment which is layered on top of ReplicaSets help manage things like rolling updates.
+
+In KubeVirt, this concept of layering controllers is at the very center of our design. The KubeVirt VirtualMachineInstance (VMI) object is the lowest level unit at the very bottom of the KubeVirt stack. Similar in concept to a Pod, a VMI represents a single mortal virtualized workload that executes once until completion (powered off).
+
+Layered on top of VMIs we have a workload controller called a VirtualMachine (VM). The VM controller is where we really begin to see the differences between how users manage virtualized workloads vs containerized workloads. Within the context of existing Kubernetes functionality, the best way to describe the VM controller’s behavior is to compare it to a StatefulSet of size one. This is because the VM controller represents a single stateful (immortal) virtual machine capable of persisting state across both node failures and multiple restarts of its underlying VMI. This object behaves in the way that is familiar to users who have managed virtual machines in AWS, GCE, OpenStack or any other similar IaaS cloud platform. The user can shutdown a VM, then choose to start that exact same VM up again at a later time. 
+
+In addition to VMs, we also have a VirtualMachineInstanceReplicaSet (VMIRS) workload controller which manages scale out of identical VMI objects. This controller behaves nearly identically to the Kubernetes ReplicSet controller. The primary difference being that the VMIRS manages VMI objects and the ReplicaSet manages Pods. Wouldn’t it be nice if we could come up with a way to [use the Kubernetes ReplicaSet controller to scale out CRDs?](https://github.com/kubernetes/kubernetes/issues/65622)
+
+Each one of these KubeVirt objects (VMI, VM, VMIRS) are registered with Kubernetes as a CRD when the KubeVirt install manifest is posted to the cluster. By registering our APIs as CRDs with Kubernetes, all the tooling involved with managing Kubernetes clusters (like kubectl) have access to the KubeVirt APIs just as if they are native Kubernetes objects.
+
+## Dynamic Webhooks for API Validation
+
+One of the responsibilities of the Kubernetes API server is to intercept and validate requests prior to allowing objects to be persisted into etcd. For example, if someone tries to create a Pod using a malformed Pod specification, the Kubernetes API server immediately catches the error and rejects the POST request. This all occurs before the object is persistent into etcd preventing the malformed Pod specification from making its way into the cluster.
+
+This validation occurs during a process called admission control. Until recently, it was not possible to extend the default Kubernetes admission controllers without altering code and compiling/deploying an entirely new Kubernetes API server. This meant that if we wanted to perform admission control on KubeVirt’s CRD objects while they are posted to the cluster, we’d have to build our own version of the Kubernetes API server and convince our users to use that instead. That was not a viable solution for us.
+
+Using the new [Dynamic Admission Control](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/) feature that first landed in Kubernetes 1.9, we now have a path for performing custom validation on KubeVirt API through the use of a [ValidatingAdmissionWebhook](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#external-admission-webhooks). This feature allows KubeVirt to dynamically register an HTTPS webhook with Kubernetes at KubeVirt install time. After registering the custom webhook, all requests related to KubeVirt API objects are forwarded from the Kubernetes API server to our HTTPS endpoint for validation. If our endpoint rejects a request for any reason, the object will not be persisted into etcd and the client receives our response outlining the reason for the rejection.
+
+For example, if someone posts a malformed VirtualMachine object, they’ll receive an error indicating what the problem is. 
+
+```
+$ kubectl create -f my-vm.yaml 
+Error from server: error when creating "my-vm.yaml": admission webhook "virtualmachine-validator.kubevirt.io" denied the request: spec.template.spec.domain.devices.disks[0].volumeName 'registryvolume' not found.
+```
+
+In the example output above, that error response is coming directly from KubeVirt’s admission control webhook. 
+
+## CRD OpenAPIv3 Validation
+
+In addition to the validating webhook, KubeVirt also uses the ability to provide an [OpenAPIv3 validation schema](https://kubernetes.io/docs/tasks/access-kubernetes-API/extend-api-custom-resource-definitions/#advanced-topics) when registering a CRD with the cluster. While the OpenAPIv3 schema does not let us express some of the more advanced validation checks that the validation webhook provides, it does offer the ability to enforce simple validation checks involving things like required fields, max/min value lengths, and verifying that values are formatted in a way that matches a regular expression string.
+
+## Dynamic Webhooks for “PodPreset Like” Behavior
+
+The Kubernetes Dynamic Admission Control feature is not only limited to validation logic, it also provides the ability for applications like KubeVirt to both intercept and mutate requests as they enter the cluster. This is achieved through the use of a **MutatingAdmissionWebhook** object. In KubeVirt, we are looking to use a mutating webhook to support our VirtualMachinePreset (VMPreset) feature.
+
+A VMPreset acts in a similar way to a PodPreset. Just like a PodPreset allows users to define values that should automatically be injected into pods at creation time, a VMPreset allows users to define values that should be injected into VMs at creation time. Through the use of a mutating webhook, KubeVirt can intercept a request to create a VM, apply VMPresets to the VM spec, and then validate that the resulting VM object. This all occurs before the VM object is persisted into etcd which allows KubeVirt to immediately notify the user of any conflicts at the time the request is made.
+
+## Subresources for CRDs
+
+When comparing the use of CRDs to an aggregated API server, one of the features CRDs lack is the ability to support subresources. Subresources are used to provide additional resource functionality. For example, the `pod/logs` and `pod/exec` subresource endpoints are used behind the scenes to provide the `kubectl logs` and `kubectl exec` command functionality.
+
+Just like Kubernetes uses the `pod/exec` subresource to provide access to a pod’s environment, in KubeVirt we want subresources to provide serial-console, VNC, and SPICE access to a virtual machine. By adding virtual machine guest access through subresources, we can leverage RBAC to provide access control for these features.
+
+So, given that the KubeVirt team decided to use CRD’s instead of an aggregated API server for custom resource support, how can we have subresources for CRDs when the CRD feature expiclity does not support subresources?
+
+We created a workaround for this limitation by implementing a stateless aggregated API server that exists only to serve subresource requests. With no state, we don’t have to worry about any of the issues we identified earlier with regards to access to etcd. This means the KubeVirt API is actually supported through a combination of both CRDs for resources and an aggregated API server for stateless subresources.
+
+This isn’t a perfect solution for us. Both aggregated API servers and CRDs require us to register an API GroupName with Kubernetes. This API GroupName field essentially namespaces the API’s REST path in a way that prevents API naming conflicts between other third party applications. Because CRDs and aggregated API servers can’t share the same GroupName, we have to register two separate GroupNames. One is used by our CRDs and the other is used by the aggregated API server for subresource requests. 
+
+Having two GroupNames in our API is slightly inconvenient because it means the REST path for the endpoints that serve the KubeVirt subresource requests have a slightly different base path than the resources.
+
+For example, the endpoint to create a VMI object is as follows.
+
+**/apis/kubevirt.io/v1alpha2/namespaces/my-namespace/virtualmachineinstances/my-vm**
+
+However, the subresource endpoint to access graphical VNC looks like this.
+
+**/apis/subresources.kubevirt.io/v1alpha2/namespaces/my-namespace/virtualmachineinstances/my-vm/vnc**
+
+Notice that the first request uses **kubevirt.io** and the second request uses **subresource.kubevirt.io**. We don’t like that, but that’s how we’ve managed to combine CRDs with a stateless aggregated API server for subresources.
+
+One thing worth noting is that in Kubernetes 1.10 a very basic form of CRD subresource support was added in the form of the `/status` and `/scale` subresources. This support does not help us deliver the virtualization features we want subresources for. However, there have been discussions about exposing custom CRD subresources as webhooks in a future Kubernetes version. If this functionality lands, we will gladly transition away from our stateless aggregated API server workaround to use a subresource webhook feature.
+
+## CRD Finalizers
+
+A [CRD finalizer](https://kubernetes.io/docs/tasks/access-kubernetes-API/extend-api-custom-resource-definitions/#advanced-topics) is a feature that lets us provide a pre-delete hook in order to perform actions before allowing a CRD object to be removed from persistent storage. In KubeVirt, we use finalizers to guarantee a virtual machine has completely terminated before we allow the corresponding VMI object to be removed from etcd.
+
+## API Versioning for CRDs
+
+The Kubernetes core APIs have the ability to support multiple versions for a single object type and perform conversions between those versions. This gives the Kubernetes core APIs a path for advancing the `v1alpha1` version of an object to a `v1beta1` version and so forth.
+
+Prior to Kubernetes 1.11, CRDs did not not have support for multiple versions. This meant when we wanted to progress a CRD from `kubevirt.io/v1alpha1` to `kubevirt.io/v1beta1`, the only path available to was to backup our CRD objects, delete the registered CRD from Kubernetes, register a new CRD with the updated version, convert the backed up CRD objects to the new version, and finally post the migrated CRD objects back to the cluster.
+
+That strategy was not exactly a viable option for us.
+
+Fortunately thanks to some recent [work to rectify this issue in Kubernetes](https://github.com/kubernetes/features/issues/544), the latest Kubernetes v1.11 now supports [CRDs with multiple versions](https://github.com/kubernetes/kubernetes/pull/63830). Note however that this initial multi version support is limited. While a CRD can now have multiple versions, the feature does not currently contain a path for performing conversions between versions. In KubeVirt, the lack of conversion makes it difficult us to evolve our API as we progress versions. Luckily, support for conversions between versions is underway and we look forward to taking advantage of that feature once it lands in a future Kubernetes release.
+
@@ -0,0 +1,161 @@
+---
+layout: blog
+title:  'Dynamically Expand Volume with CSI and Kubernetes'
+date:   2018-08-02
+---
+
+**Author**: Orain Xiong (Co-Founder, WoquTech)
+
+_There is a very powerful storage subsystem within Kubernetes itself, covering a fairly broad spectrum of use cases. Whereas, when planning to build a product-grade relational database platform with Kubernetes, we face a big challenge: coming up with storage. This article describes how to extend latest Container Storage Interface 0.2.0 and integrate with Kubernetes, and demonstrates the essential facet of dynamically expanding volume capacity._
+
+## Introduction
+
+As we focalize our customers, especially in financial space, there is a huge upswell in the adoption of container orchestration technology.
+
+They are looking forward to open source solutions to redesign already existing monolithic applications, which have been running for several years on virtualization infrastructure or bare metal.
+
+Considering extensibility and the extent of technical maturity, Kubernetes and Docker are at the very top of the list. But migrating monolithic applications to a distributed orchestration like Kubernetes is challenging, the relational database is critical for the migration.
+
+With respect to the relational database, we should pay attention to storage. There is a very powerful storage subsystem within Kubernetes itself. It is very useful and covers a fairly broad spectrum of use cases. When planning to run a relational database with Kubernetes in production, we face a big challenge: coming up with storage. There are still some fundamental functionalities which are left unimplemented. Specifically, dynamically expanding volume. It sounds boring but is highly required, except for actions like create and delete and mount and unmount.
+
+Currently, expanding volume is only available with those storage provisioners:
+
+* gcePersistentDisk
+* awsElasticBlockStore
+* OpenStack Cinder
+* glusterfs
+* rbd
+
+In order to enable this feature, we should set feature gate `ExpandPersistentVolumes` true and turn on the `PersistentVolumeClaimResize` admission plugin. Once `PersistentVolumeClaimResize` has been enabled, resizing will be allowed by a Storage Class whose `allowVolumeExpansion` field is set to true.
+
+Unfortunately, dynamically expanding volume through the Container Storage Interface (CSI) and Kubernetes is unavailable, even though the underlying storage providers have this feature.
+
+This article will give a simplified view of CSI, followed by a walkthrough of how to introduce a new expanding volume feature on the existing CSI and Kubernetes. Finally, the article will demonstrate how to dynamically expand volume capacity.
+
+## Container Storage Interface (CSI)
+
+To have a better understanding of what we're going to do, the first thing we need to know is what the Container Storage Interface is. Currently, there are still some problems for already existing storage subsystem within Kubernetes. Storage driver code is maintained in the Kubernetes core repository which is difficult to test. But beyond that, Kubernetes needs to give permissions to storage vendors to check code into the Kubernetes core repository. Ideally, that should be implemented externally.
+
+CSI is designed to define an industry standard that will enable storage providers who enable CSI to be available across container orchestration systems that support CSI.
+
+This diagram depicts a kind of high-level Kubernetes archetypes integrated with CSI:
+
+![csi diagram](/images/blog/2018-08-02-dynamically-expand-volume-csi/csi-diagram.png)
+
+* Three new external components are introduced to decouple Kubernetes and Storage Provider logic
+* Blue arrows present the conventional way to call against API Server
+* Red arrows present gRPC to call against Volume Driver
+
+For more details, please visit: https://github.com/container-storage-interface/spec/blob/master/spec.md
+
+## Extend CSI and Kubernetes
+
+In order to enable the feature of expanding volume atop Kubernetes, we should extend several components including CSI specification, “in-tree” volume plugin, external-provisioner and external-attacher.
+
+## Extend CSI spec
+
+The feature of expanding volume is still undefined in latest CSI 0.2.0. The new 3 RPCs, including `RequiresFSResize` and `ControllerResizeVolume` and `NodeResizeVolume`, should be introduced.
+
+```
+service Controller {
+ rpc CreateVolume (CreateVolumeRequest)
+   returns (CreateVolumeResponse) {}
+……
+ rpc RequiresFSResize (RequiresFSResizeRequest)
+   returns (RequiresFSResizeResponse) {}
+ rpc ControllerResizeVolume (ControllerResizeVolumeRequest)
+   returns (ControllerResizeVolumeResponse) {}
+}
+
+service Node {
+ rpc NodeStageVolume (NodeStageVolumeRequest)
+   returns (NodeStageVolumeResponse) {}
+……
+ rpc NodeResizeVolume (NodeResizeVolumeRequest)
+   returns (NodeResizeVolumeResponse) {}
+}
+```
+
+## Extend “In-Tree” Volume Plugin
+
+In addition to the extend CSI specification, the `csiPlugin` interface within Kubernetes should also implement `expandablePlugin`. The `csiPlugin` interface will expand `PersistentVolumeClaim` representing for `ExpanderController`.
+
+
+```go
+type ExpandableVolumePlugin interface {
+VolumePlugin
+ExpandVolumeDevice(spec Spec, newSize resource.Quantity, oldSize resource.Quantity) (resource.Quantity, error)
+RequiresFSResize() bool
+}
+```
+
+### Implement Volume Driver
+
+Finally, to abstract complexity of the implementation, we should hard code the separate storage provider management logic into the following functions which is well-defined in the CSI specification:
+
+* CreateVolume
+* DeleteVolume
+* ControllerPublishVolume
+* ControllerUnpublishVolume
+* ValidateVolumeCapabilities
+* ListVolumes
+* GetCapacity
+* ControllerGetCapabilities
+* RequiresFSResize
+* ControllerResizeVolume
+
+## Demonstration
+
+Let’s demonstrate this feature with a concrete user case.
+
+* Create storage class for CSI storage provisioner
+
+```yaml
+allowVolumeExpansion: true
+apiVersion: storage.k8s.io/v1
+kind: StorageClass
+metadata:
+  name: csi-qcfs
+parameters:
+  csiProvisionerSecretName: orain-test
+  csiProvisionerSecretNamespace: default
+provisioner: csi-qcfsplugin
+reclaimPolicy: Delete
+volumeBindingMode: Immediate
+```
+
+* Deploy CSI Volume Driver including storage provisioner `csi-qcfsplugin` across Kubernetes cluster
+
+* Create PVC `qcfs-pvc` which will be dynamically provisioned by storage class `csi-qcfs`
+
+```yaml
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: qcfs-pvc
+  namespace: default
+....
+spec:
+  accessModes:
+  - ReadWriteOnce
+  resources:
+    requests:
+      storage: 300Gi
+  storageClassName: csi-qcfs
+```
+
+* Create MySQL 5.7 instance to use PVC `qcfs-pvc`
+* In order to mirror the exact same production-level scenario, there are actually two different types of workloads including:
+    * Batch insert to make MySQL consuming more file system capacity
+    * Surge query request
+* Dynamically expand volume capacity through edit pvc `qcfs-pvc` configuration
+
+The Prometheus and Grafana integration allows us to visualize corresponding critical metrics.
+
+![prometheus grafana](/images/blog/2018-08-02-dynamically-expand-volume-csi/prometheus-grafana.png)
+
+We notice that the middle reading shows MySQL datafile size increasing slowly during bulk inserting. At the same time, the bottom reading shows file system expanding twice in about 20 minutes, from 300 GiB to 400 GiB and then 500 GiB. Meanwhile, the upper reading shows the whole process of expanding volume immediately completes and hardly impacts MySQL QPS.
+
+## Conclusion
+
+Regardless of whatever infrastructure applications have been running on, the database is always a critical resource. It is essential to have a more advanced storage subsystem out there to fully support database requirements. This will help drive the more broad adoption of cloud native technology.
@@ -0,0 +1,193 @@
+---
+layout: blog
+title:  'Out of the Clouds onto the Ground: How to Make Kubernetes Production Grade Anywhere'
+date:   2018-08-03
+---
+
+**Authors**: Steven Wong (VMware), Michael Gasch (VMware)
+
+This blog offers some guidelines for running a production grade Kubernetes cluster in an environment like an on-premise data center or edge location.
+
+What does it mean to be “production grade”?
+
+* The installation is secure
+* The deployment is managed with a repeatable and recorded process
+* Performance is predictable and consistent
+* Updates and configuration changes can be safely applied
+* Logging and monitoring is in place to detect and diagnose failures and resource shortages
+* Service is “highly available enough” considering available resources, including constraints on money, physical space, power, etc.
+* A recovery process is available, documented, and tested for use in the event of failures
+
+In short, production grade means anticipating accidents and preparing for recovery with minimal pain and delay.
+
+This article is directed at on-premise Kubernetes deployments on a hypervisor or bare-metal platform, facing finite backing resources compared to the expansibility of the major public clouds. However, some of these recommendations may also be useful in a public cloud if budget constraints limit the resources you choose to consume.
+
+A single node bare-metal Minikube deployment may be cheap and easy, but is not production grade. Conversely, you’re not likely to achieve Google’s Borg experience in a retail store, branch office, or edge location, nor are you likely to need it.
+
+This blog offers some guidance on achieving a production worthy Kubernetes deployment, even when dealing with some resource constraints.
+
+![without incidence](/images/blog/2018-08-03-make-kubernetes-production-grade-anywhere/without-incidence.png)
+
+## Critical components in a Kubernetes cluster
+
+Before we dive into the details, it is critical to understand the overall Kubernetes architecture.
+
+A Kubernetes cluster is a highly distributed system based on a control plane and clustered worker node architecture as depicted below.
+
+![api server](/images/blog/2018-08-03-make-kubernetes-production-grade-anywhere/api-server.png)
+
+Typically the API server, Controller Manager and Scheduler components are co-located within multiple instances of control plane (aka Master) nodes. Master nodes usually include etcd too, although there are high availability and large cluster scenarios that call for running etcd on independent hosts. The components can be run as containers, and optionally be supervised by Kubernetes, i.e. running as statics pods.
+
+For high availability, redundant instances of these components are used. The importance and required degree of redundancy varies.
+
+### Kubernetes components from an HA perspective
+
+![kubernetes components HA](/images/blog/2018-08-03-make-kubernetes-production-grade-anywhere/kubernetes-components-ha.png)
+
+Risks to these components include hardware failures, software bugs, bad updates, human errors, network outages, and overloaded systems resulting in resource exhaustion. Redundancy can mitigate the impact of many of these hazards. In addition, the resource scheduling and high availability features of a hypervisor platform can be useful to surpass what can be achieved using the Linux operating system, Kubernetes, and a container runtime alone.
+
+The API Server uses multiple instances behind a load balancer to achieve scale and availability. The load balancer is a critical component for purposes of high availability. Multiple DNS API Server ‘A’ records might be an alternative if you don’t have a load balancer.
+
+The kube-scheduler and kube-controller-manager engage in a leader election process, rather than utilizing a load balancer. Since a [cloud-controller-manager](https://kubernetes.io/docs/tasks/administer-cluster/running-cloud-controller/) is used for selected types of hosting infrastructure, and these have implementation variations, they will not be discussed, beyond indicating that they are a control plane component.
+
+Pods running on Kubernetes worker nodes are managed by the kubelet agent. Each worker instance runs the kubelet agent and a [CRI-compatible](https://kubernetes.io/blog/2016/12/container-runtime-interface-cri-in-kubernetes/) container runtime. Kubernetes itself is designed to monitor and recover from worker node outages. But for critical workloads, hypervisor resource management, workload isolation and availability features can be used to enhance availability and make performance more predictable.
+
+## etcd
+
+etcd is the persistent store for all Kubernetes objects. The availability and recoverability of the etcd cluster should be the first consideration in a production-grade Kubernetes deployment.
+
+A five-node etcd cluster is a best practice if you can afford it. Why? Because you could engage in maintenance on one and still tolerate a failure. A three-node cluster is the minimum [recommendation](https://coreos.com/etcd/docs/latest/v2/admin_guide.html#optimal-cluster-size) for production-grade service, even if only a single hypervisor host is available. More than seven nodes is not recommended except for [very large installations](https://monzo.com/blog/2017/11/29/very-robust-etcd/) straddling multiple availability zones.
+
+The minimum recommendation for hosting an etcd cluster node is 2GB of RAM with 8GB of SSD-backed disk. Usually, 8GB RAM and a 20GB disk will be enough. Disk performance affects failed node recovery time. See https://coreos.com/etcd/docs/latest/op-guide/hardware.html for more on this.
+
+### Consider multiple etcd clusters in special situations
+
+For very large Kubernetes clusters, consider using a separate etcd cluster for Kubernetes events so that event storms do not impact the main Kubernetes API service. If you use flannel networking, it retains configuration in etcd and may have differing version requirements than Kubernetes, which can complicate etcd backup -- consider using a dedicated etcd cluster for flannel.
+
+## Single host deployments
+
+The availability risk list includes hardware, software and people. If you are limited to a single host, the use of redundant storage, error-correcting memory and dual power supplies can reduce hardware failure exposure. Running a hypervisor on the physical host will allow operation of redundant software components and add operational advantages related to deployment, upgrade, and resource consumption governance, with predictable and repeatable performance under stress. For example, even if you can only afford to run singletons of the master services, they need to be protected from overload and resource exhaustion while competing with your application workload. A hypervisor can be more effective and easier to manage than configuring Linux scheduler priorities, cgroups, Kubernetes flags, etc.
+
+If resources on the host permit, you can deploy three etcd VMs. Each of the etcd VMs should be backed by a different physical storage device, or they should use separate partitions of a backing store using redundancy (mirroring, RAID, etc).
+
+Dual redundant instances of the API server, scheduler and controller manager would be the next upgrade, if your single host has the resources.
+
+### Single host deployment options, least production worthy to better
+
+![single host deployment](/images/blog/2018-08-03-make-kubernetes-production-grade-anywhere/single-host-deployment.png)
+
+## Dual host deployments
+
+With two hosts, storage concerns for etcd are the same as a single host, you want redundancy. And you would preferably run 3 etcd instances. Although possibly counter-intuitive, it is better to concentrate all etcd nodes on a single host. You do not gain reliability by doing a 2+1 split across two hosts -  because loss of the node holding the majority of etcd instances results in an outage, whether that majority is 2 or 3. If the hosts are not identical, put the whole etcd cluster on the most reliable host.
+
+Running redundant API Servers, kube-schedulers, and kube-controller-managers is recommended. These should be split across hosts to minimize risk due to container runtime, OS and hardware failures.
+
+Running a hypervisor layer on the physical hosts will allow operation of redundant software components with resource consumption governance, and can have planned maintenance operational advantages.
+
+### Dual host deployment options, least production worthy to better
+
+![dual host deployment](/images/blog/2018-08-03-make-kubernetes-production-grade-anywhere/dual-host-deployment.png)
+
+Triple (or larger) host deployments -- Moving into uncompromised production-grade service
+Splitting etcd across three hosts is recommended. A single hardware failure will reduce application workload capacity, but should not result in a complete service outage.
+
+With very large clusters, more etcd instances will be required.
+
+Running a hypervisor layer offers operational advantages and better workload isolation. It is beyond the scope of this article, but at the three-or-more host level, advanced features may be available (clustered redundant shared storage, resource governance with dynamic load balancing, automated health monitoring with live migration or failover).
+
+### Triple (or more) host options, least production worthy to better
+
+![triple host deployment](/images/blog/2018-08-03-make-kubernetes-production-grade-anywhere/triple-host-deployment.png)
+
+## Kubernetes configuration settings
+Master and Worker nodes should be protected from overload and resource exhaustion. Hypervisor features can be used to isolate critical components and reserve resources. There are also Kubernetes configuration settings that can throttle things like API call rates and pods per node. Some install suites and commercial distributions take care of this, but if you are performing a custom Kubernetes deployment, you may find that the defaults are not appropriate, particularly if your resources are small or your cluster is large.
+
+Resource consumption by the control plane will correlate with the number of pods and the pod churn rate. Very large and very small clusters will benefit from non-default [settings](https://kubernetes.io/docs/reference/command-line-tools-reference/kube-apiserver/) of kube-apiserver request throttling and memory. Having these too high can lead to request limit exceeded and out of memory errors.
+
+On worker nodes, [Node Allocatable](https://kubernetes.io/docs/tasks/administer-cluster/reserve-compute-resources/) should be configured based on a reasonable supportable workload density at each node. Namespaces can be created to subdivide the worker node cluster into multiple virtual clusters with resource CPU and memory [quotas](https://kubernetes.io/docs/tasks/administer-cluster/manage-resources/memory-default-namespace/). Kubelet handling of [out of resource](https://kubernetes.io/docs/tasks/administer-cluster/out-of-resource/) conditions can be configured.
+
+## Security
+
+Every Kubernetes cluster has a cluster root Certificate Authority (CA). The Controller Manager, API Server, Scheduler, kubelet client, kube-proxy and administrator certificates need to be generated and installed. If you use an install tool or a distribution this may be handled for you. A manual process is described [here](https://github.com/kelseyhightower/kubernetes-the-hard-way/blob/master/docs/04-certificate-authority.md). You should be prepared to reinstall certificates in the event of node replacements or expansions.
+
+As Kubernetes is entirely API driven, controlling and limiting who can access the cluster and what actions they are allowed to perform is essential. Encryption and authentication options are addressed in this [documentation](https://kubernetes.io/docs/tasks/administer-cluster/securing-a-cluster/).
+
+Kubernetes application workloads are based on container images. You want the source and content of these images to be trustworthy. This will almost always mean that you will host a local container image repository. Pulling images from the public Internet can present both reliability and security issues. You should choose a repository that supports image signing, security scanning, access controls on pushing and pulling images, and logging of activity.
+
+Processes must be in place to support applying updates for host firmware, hypervisor, OS, Kubernetes, and other dependencies. Version monitoring should be in place to support audits.
+
+Recommendations:
+
+* Tighten security settings on the control plane components beyond defaults (e.g., [locking down worker nodes](http://blog.kontena.io/locking-down-kubernetes-workers/))
+* Utilize [Pod Security Policies](https://kubernetes.io/docs/concepts/policy/pod-security-policy/)
+* Consider the [NetworkPolicy](https://kubernetes.io/docs/concepts/services-networking/network-policies/) integration available with your networking solution, including how you will accomplish tracing, monitoring and troubleshooting.
+* Use RBAC to drive authorization decisions and enforcement.
+* Consider physical security, especially when deploying to edge or remote office locations that may be unattended. Include storage encryption to limit exposure from stolen devices and protection from attachment of malicious devices like USB keys.
+* Protect Kubernetes plain-text cloud provider credentials (access keys, tokens, passwords, etc.)
+
+Kubernetes [secret](https://kubernetes.io/docs/concepts/configuration/secret/) objects are appropriate for holding small amounts of sensitive data. These are retained within etcd. These can be readily used to hold credentials for the Kubernetes API but there are times when a workload or an extension of the cluster itself needs a more full-featured solution. The HashiCorp Vault project is a popular solution if you need more than the built-in secret objects can provide.
+
+## Disaster Recovery and Backup
+
+![disaster recovery](/images/blog/2018-08-03-make-kubernetes-production-grade-anywhere/disaster-recovery.png)
+
+Utilizing redundancy through the use of multiple hosts and VMs helps reduce some classes of outages, but scenarios such as a sitewide natural disaster, a bad update, getting hacked, software bugs, or human error could still result in an outage.
+
+A critical part of a production deployment is anticipating a possible future recovery.
+
+It’s also worth noting that some of your investments in designing, documenting, and automating a recovery process might also be re-usable if you need to do large-scale replicated deployments at multiple sites.
+
+Elements of a DR plan include backups (and possibly replicas), replacements, a planned process, people who can carry out the process, and recurring training. Regular test exercises and [chaos engineering principles](https://github.com/dastergon/awesome-chaos-engineering) can be used to audit your readiness.
+
+Your availability requirements might demand that you retain local copies of the OS, Kubernetes components, and container images to allow recovery even during an Internet outage. The ability to deploy replacement hosts and nodes in an “air-gapped” scenario can also offer security and speed of deployment advantages.
+
+All Kubernetes objects are stored on etcd. Periodically backing up the etcd cluster data is important to recover Kubernetes clusters under disaster scenarios, such as losing all master nodes.
+
+Backing up an etcd cluster can be accomplished with etcd’s [built-in](https://coreos.com/etcd/docs/latest/op-guide/recovery.html) snapshot mechanism, and copying the resulting file to storage in a different failure domain. The snapshot file contains all the Kubernetes states and critical information. In order to keep the sensitive Kubernetes data safe, encrypt the snapshot files.
+
+Using disk volume based snapshot recovery of etcd can have issues; see [#40027](https://github.com/kubernetes/kubernetes/issues/40027). API-based backup solutions (e.g., [Ark](https://github.com/heptio/ark)) can offer more granular recovery than a etcd snapshot, but also can be slower. You could utilize both snapshot and API-based backups, but you should do one form of etcd backup as a minimum.
+
+Be aware that some Kubernetes extensions may maintain state in independent etcd clusters, on persistent volumes, or through other mechanisms. If this state is critical, it should have a backup and recovery plan.
+
+Some critical state is held outside etcd. Certificates, container images, and other configuration- and operation-related state may be managed by your automated install/update tooling. Even if these items can be regenerated, backup or replication might allow for faster recovery after a failure. Consider backups with a recovery plan for these items:
+
+* Certificate and key pairs
+    * CA
+    * API Server
+    * Apiserver-kubelet-client
+    * ServiceAccount signing
+    * “Front proxy”
+    * Front proxy client
+* Critical DNS records
+* IP/subnet assignments and reservations
+* External load-balancers
+* kubeconfig files
+* LDAP or other authentication details
+* Cloud provider specific account and configuration data
+
+## Considerations for your production workloads
+Anti-affinity specifications can be used to split clustered services across backing hosts, but at this time the settings are used only when the pod is scheduled. This means that Kubernetes can restart a failed node of your clustered application, but does not have a native mechanism to rebalance after a fail back. This is a topic worthy of a separate blog, but supplemental logic might be useful to achieve optimal workload placements after host or worker node recoveries or expansions. The [Pod Priority and Preemption feature](https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/) can be used to specify a preferred triage in the event of resource shortages caused by failures or bursting workloads.
+
+For stateful services, external attached volume mounts are the standard Kubernetes recommendation for a non-clustered service (e.g., a typical SQL database). At this time Kubernetes managed snapshots of these external volumes is in the category of a [roadmap feature request](https://docs.google.com/presentation/d/1dgxfnroRAu0aF67s-_bmeWpkM1h2LCxe6lB1l1oS0EQ/edit#slide=id.g3ca07c98c2_0_47), likely to align with the Container Storage Interface (CSI) integration. Thus performing backups of such a service would involve application specific, in-pod activity that is beyond the scope of this document. While awaiting better Kubernetes support for a snapshot and backup workflow, running your database service in a VM rather than a container, and exposing it to your Kubernetes workload may be worth considering.
+
+Cluster-distributed stateful services (e.g., Cassandra) can benefit from splitting across hosts, using [local persistent volumes](https://kubernetes.io/blog/2018/04/13/local-persistent-volumes-beta/#disclaimer) if resources allow. This would require deploying multiple Kubernetes worker nodes (could be VMs on hypervisor hosts) to preserve a quorum under single point failures.
+
+## Other considerations
+
+[Logs](https://kubernetes.io/docs/concepts/cluster-administration/logging/) and [metrics](https://kubernetes.io/docs/tasks/debug-application-cluster/resource-usage-monitoring/) (if collected and persistently retained) are valuable to diagnose outages, but given the variety of technologies available it will not be addressed in this blog. If Internet connectivity is available, it may be desirable to retain logs and metrics externally at a central location.
+
+Your production deployment should utilize an automated installation, configuration and update tool (e.g., [Ansible](https://github.com/kubernetes-incubator/kubespray), [BOSH](https://github.com/cloudfoundry-incubator/kubo-deployment), [Chef](https://github.com/chef-cookbooks/kubernetes), [Juju](https://kubernetes.io/docs/getting-started-guides/ubuntu/installation/), [kubeadm](https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm/), [Puppet](https://forge.puppet.com/puppetlabs/kubernetes), etc.). A manual process will have repeatability issues, be labor intensive, error prone, and difficult to scale. [Certified distributions](https://www.cncf.io/certification/software-conformance/#logos) are likely to include a facility for retaining configuration settings across updates, but if you implement your own install and config toolchain, then retention, backup and recovery of the configuration artifacts is essential. Consider keeping your deployment components and settings under a version control system such as Git.
+
+## Outage recovery
+
+[Runbooks](https://en.wikipedia.org/wiki/Runbook) documenting recovery procedures should be tested and retained offline -- perhaps even printed. When an on-call staff member is called up at 2 am on a Friday night, it may not be a great time to improvise. Better to execute from a pre-planned, tested checklist -- with shared access by remote and onsite personnel.
+
+## Final thoughts
+
+![airplane](/images/blog/2018-08-03-make-kubernetes-production-grade-anywhere/airplane.png)
+
+Buying a ticket on a commercial airline is convenient and safe. But when you travel to a remote location with a short runway, that commercial Airbus A320 flight isn’t an option. This doesn’t mean that air travel is off the table. It does mean that some compromises are necessary.
+
+The adage in aviation is that on a single engine aircraft, an engine failure means you crash. With twin engines, at the very least, you get more choices of where you crash. Kubernetes on a small number of hosts is similar, and if your business case justifies it, you might scale up to a larger fleet of mixed large and small vehicles (e.g., FedEx, Amazon).
+
+Those designing a production-grade Kubernetes solution have a lot of options and decisions. A blog-length article can’t provide all the answers, and can’t know your specific priorities. We do hope this offers a checklist of things to consider, along with some useful guidance. Some options were left “on the cutting room floor” (e.g., running Kubernetes components using [self-hosting](https://github.com/kubernetes/kubeadm/blob/master/docs/design/design_v1.9.md#optional-and-alpha-in-v19-self-hosting) instead of static pods). These might be covered in a follow up if there is interest. Also, Kubernetes’ high enhancement rate means that if your search engine found this article after 2019, some content might be past the “sell by” date.
@@ -0,0 +1,51 @@
+---
+layout: blog
+title:  'Introducing Kubebuilder: an SDK for building Kubernetes APIs using CRDs'
+date:   2018-08-10
+---
+
+**Author**: Phillip Wittrock (Google), Sunil Arora (Google)
+
+[kubebuilder-repo]: https://github.com/kubernetes-sigs/kubebuilder
+[controller-runtime]: https://github.com/kubernetes-sigs/controller-runtime
+[SIG-APIMachinery]: https://github.com/kubernetes/community/tree/master/sig-api-machinery
+[mailing-list]: https://groups.google.com/forum/#!forum/kubernetes-sig-api-machinery
+[slack-channel]: https://slack.k8s.io/#kubebuilder 
+[kubebuilder-book]: https://book.kubebuilder.io
+[open-an-issue]: https://github.com/kubernetes-sigs/kubebuilder/issues/new
+
+
+How can we enable applications such as MySQL, Spark and Cassandra to manage themselves just like Kubernetes Deployments and Pods do? How do we configure these applications as their own first class APIs instead of a collection of StatefulSets, Services, and ConfigMaps?
+
+We have been working on a solution and are happy to introduce [*kubebuilder*][kubebuilder-repo], a comprehensive development kit for rapidly building and publishing Kubernetes APIs and Controllers using CRDs. Kubebuilder scaffolds projects and API definitions and is built on top of the [controller-runtime][controller-runtime] libraries.
+
+### Why Kubebuilder and Kubernetes APIs?
+Applications and cluster resources typically require some operational work - whether it is replacing failed replicas with new ones, or scaling replica counts while resharding data. Running the MySQL application may require scheduling backups, reconfiguring replicas after scaling, setting up failure detection and remediation, etc.
+
+With the Kubernetes API model, management logic is embedded directly into an application specific Kubernetes API, e.g. a “MySQL” API. Users then declaratively manage the application through YAML configuration using tools such as kubectl, just like they do for Kubernetes objects. This approach is referred to as an Application Controller, also known as an Operator. Controllers are a powerful technique backing the core Kubernetes APIs that may be used to build many kinds of solutions in addition to Applications; such as Autoscalers, Workload APIs, Configuration APIs, CI/CD systems, and more.
+
+However, while it has been possible for trailblazers to build new Controllers on top of the raw API machinery, doing so has been a DIY “from scratch” experience, requiring developers to learn low level details about how Kubernetes libraries are implemented, handwrite boilerplate code, and wrap their own solutions for integration testing, RBAC configuration, documentation, etc. Kubebuilder makes this experience simple and easy by applying the lessons learned from building the core Kubernetes APIs.
+
+### Getting Started Building Application Controllers and Kubernetes APIs
+
+By providing an opinionated and structured solution for creating Controllers and Kubernetes APIs, developers have a working “out of the box” experience that uses the lessons and best practices learned from developing the core Kubernetes APIs. Creating a new "Hello World" Controller with `kubebuilder` is as simple as:
+
+ 1. Create a project with `kubebuilder init`
+ 2. Define a new API with `kubebuilder create api`
+ 3. Build and run the provided main function with `make install & make run`
+
+This will scaffold the API and Controller for users to modify, as well as scaffold integration tests, RBAC rules, Dockerfiles, Makefiles, etc.
+After adding their implementation to the project, users create the artifacts to publish their API through:
+
+ 1. Build and push the container image from the provided Dockerfile using `make docker-build` and `make docker-push` commands
+ 2. Deploy the API using `make deploy` command
+
+Whether you are already a Controller aficionado or just want to learn what the buzz is about, check out the [kubebuilder repo][kubebuilder-repo] or take a look at an example in the [kubebuilder book][kubebuilder-book] to learn about how simple and easy it is to build Controllers.
+
+### Get Involved
+Kubebuilder is a project under [SIG API Machinery][SIG-APIMachinery] and is being actively developed by contributors from many companies such as Google, Red Hat, VMware, Huawei and others. Get involved by giving us feedback through these channels:
+ 
+ - Kubebuilder [chat room on Slack][slack-channel]
+ - SIG [mailing list][mailing-list]
+ - [Github issues][open-an-issue]
+ - Send a pull request in the [kubebuilder repo][kubebuilder-repo]
@@ -0,0 +1,91 @@
+---
+layout: blog
+title:  'The Machines Can Do the Work, a Story of Kubernetes Testing, CI, and Automating the Contributor Experience'
+date:   2018-08-29
+---
+
+**Author**: Aaron Crickenberger (Google) and Benjamin Elder (Google)
+
+_“Large projects have a lot of less exciting, yet, hard work. We value time spent automating repetitive work more highly than toil. Where that work cannot be automated, it is our culture to recognize and reward all types of contributions. However, heroism is not sustainable.”_ - [Kubernetes Community Values](https://github.com/kubernetes/steering/blob/master/values.md#automation-over-process)
+
+Like many open source projects, Kubernetes is hosted on GitHub. We felt the barrier to participation would be lowest if the project lived where developers already worked, using tools and processes developers already knew. Thus the project embraced the service fully: it was the basis of our workflow, our issue tracker, our documentation, our blog platform, our team structure, and more.
+
+This strategy worked. It worked so well that the project quickly scaled past its contributors’ capacity as humans. What followed was an incredible journey of automation and innovation. We didn’t just need to rebuild our airplane mid-flight without crashing, we needed to convert it into a rocketship and launch into orbit. We needed machines to do the work.
+
+## The Work
+
+Initially, we focused on the fact that we needed to support the sheer volume of tests mandated by a complex distributed system such as Kubernetes. Real world failure scenarios had to be exercised via end-to-end (e2e) tests to ensure proper functionality. Unfortunately, e2e tests were susceptible to flakes (random failures) and took anywhere from an hour to a day to complete.
+
+Further experience revealed other areas where machines could do the work for us:
+
+* PR Workflow
+  * Did the contributor sign our CLA?
+  * Did the PR pass tests?
+  * Is the PR mergeable?
+  * Did the merge commit pass tests?
+* Triage
+  * Who should be reviewing PRs?
+  * Is there enough information to route an issue to the right people?
+  * Is an issue still relevant?
+* Project Health
+  * What is happening in the project?
+  * What should we be paying attention to?
+
+As we developed automation to improve our situation, we followed a few guiding principles:
+
+* Follow the push/poll control loop patterns that worked well for Kubernetes
+* Prefer stateless loosely coupled services that do one thing well
+* Prefer empowering the entire community over empowering a few core contributors
+* Eat our own dogfood and avoid reinventing wheels
+
+## Enter Prow
+
+This led us to create [Prow](https://git.k8s.io/test-infra/prow) as the central component for our automation. Prow is sort of like an [If This, Then That](https://ifttt.com/) for GitHub events, with a built-in library of [commands](https://prow.k8s.io/command-help), [plugins](https://prow.k8s.io/plugins), and utilities. We built Prow on top of Kubernetes to free ourselves from worrying about resource management and scheduling, and ensure a more pleasant operational experience.
+
+Prow lets us do things like:
+
+* Allow our community to triage issues/PRs by commenting commands such as “/priority critical-urgent”, “/assign mary” or “/close”
+* Auto-label PRs based on how much code they change, or which files they touch
+* Age out issues/PRs that have remained inactive for too long
+* Auto-merge PRs that meet our PR workflow requirements
+* Run CI jobs defined as [Knative Builds](https://github.com/knative/build), Kubernetes Pods, or Jenkins jobs
+* Enforce org-wide and per-repo GitHub policies like [branch protection](https://github.com/kubernetes/test-infra/tree/master/prow/cmd/branchprotector) and [GitHub labels](https://github.com/kubernetes/test-infra/tree/master/label_sync)
+
+Prow was initially developed by the engineering productivity team building Google Kubernetes Engine, and is actively contributed to by multiple members of Kubernetes SIG Testing. Prow has been adopted by several other open source projects, including Istio, JetStack, Knative and OpenShift. [Getting started with Prow](https://github.com/kubernetes/test-infra/blob/master/prow/getting_started.md) takes a Kubernetes cluster and `kubectl apply starter.yaml` (running pods on a Kubernetes cluster).
+
+Once we had Prow in place, we began to hit other scaling bottlenecks, and so produced additional tooling to support testing at the scale required by Kubernetes, including:
+
+- [Boskos](https://github.com/kubernetes/test-infra/tree/master/boskos): manages job resources (such as GCP projects) in pools, checking them out for jobs and cleaning them up automatically ([with monitoring](http://velodrome.k8s.io/dashboard/db/boskos-dashboard?orgId=1))
+- [ghProxy](https://github.com/kubernetes/test-infra/tree/master/ghproxy): a reverse proxy HTTP cache optimized for use with the GitHub API, to ensure our token usage doesn’t hit API limits ([with monitoring](http://velodrome.k8s.io/dashboard/db/github-cache?refresh=1m&orgId=1))
+- [Greenhouse](https://github.com/kubernetes/test-infra/tree/master/greenhouse): allows us to use a remote bazel cache to provide faster build and test results for PRs ([with monitoring](http://velodrome.k8s.io/dashboard/db/bazel-cache?orgId=1))
+- [Splice](https://github.com/kubernetes/test-infra/tree/master/prow/cmd/splice): allows us to test and merge PRs in a batch, ensuring our merge velocity is not limited to our test velocity
+- [Tide](https://github.com/kubernetes/test-infra/tree/master/prow/cmd/tide): allows us to merge PRs selected via GitHub queries rather than ordered in a queue, allowing for significantly higher merge velocity in tandem with splice
+
+## Scaling Project Health
+
+With workflow automation addressed, we turned our attention to project health. We chose to use Google Cloud Storage (GCS) as our source of truth for all test data, allowing us to lean on established infrastructure, and allowed the community to contribute results. We then built a variety of tools to help individuals and the project as a whole make sense of this data, including:
+
+* [Gubernator](https://github.com/kubernetes/test-infra/tree/master/gubernator): display the results and test history for a given PR
+* [Kettle](https://github.com/kubernetes/test-infra/tree/master/kettle): transfer data from GCS to a publicly accessible bigquery dataset
+* [PR dashboard](https://k8s-gubernator.appspot.com/pr): a workflow-aware dashboard that allows contributors to understand which PRs require attention and why
+* [Triage](https://storage.googleapis.com/k8s-gubernator/triage/index.html): identify common failures that happen across all jobs and tests
+* [Testgrid](https://k8s-testgrid.appspot.com/): display test results for a given job across all runs, summarize test results across groups of jobs
+
+We approached the Cloud Native Computing Foundation (CNCF) to develop DevStats to glean insights from our GitHub events such as:
+
+* [Which prow commands are people most actively using](https://k8s.devstats.cncf.io/d/5/bot-commands-repository-groups?orgId=1)
+* [PR reviews by contributor over time](https://k8s.devstats.cncf.io/d/46/pr-reviews-by-contributor?orgId=1&var-period=d7&var-repo_name=All&var-reviewers=All)
+* [Time spent in each phase of our PR workflow](https://k8s.devstats.cncf.io/d/44/pr-time-to-approve-and-merge?orgId=1)
+
+## Into the Beyond
+
+Today, the Kubernetes project spans over 125 repos across five orgs. There are 31 Special Interests Groups and 10 Working Groups coordinating development within the project. In the last year the project has had [participation from over 13,800 unique developers](https://k8s.devstats.cncf.io/d/13/developer-activity-counts-by-repository-group?orgId=1&var-period_name=Last%20year&var-metric=contributions&var-repogroup_name=All) on GitHub.
+
+On any given weekday our Prow instance [runs over 10,000 CI jobs](http://velodrome.k8s.io/dashboard/db/bigquery-metrics?panelId=10&fullscreen&orgId=1&from=now-6M&to=now); from March 2017 to March 2018 it ran 4.3 million jobs. Most of these jobs involve standing up an entire Kubernetes cluster, and exercising it using real world scenarios. They allow us to ensure all supported releases of Kubernetes work across cloud providers, container engines, and networking plugins. They make sure the latest releases of Kubernetes work with various optional features enabled, upgrade safely, meet performance requirements, and work across architectures.
+
+With today’s [announcement from CNCF](https://www.cncf.io/announcement/2018/08/29/cncf-receives-9-million-cloud-credit-grant-from-google) – noting that Google Cloud has begun transferring ownership and management of the Kubernetes project’s cloud resources to CNCF community contributors, we are excited to embark on another journey. One that allows the project infrastructure to be owned and operated by the community of contributors, following the same open governance model that has worked for the rest of the project. Sound exciting to you? Come talk to us at #sig-testing on kubernetes.slack.com.
+
+Want to find out more? Come check out these resources:
+
+* [Prow: Testing the way to Kubernetes Next](https://bentheelder.io/posts/prow)
+* [Automation and the Kubernetes Contributor Experience](https://www.youtube.com/watch?v=BsIC7gPkH5M)
@@ -0,0 +1,36 @@
+---
+layout: blog
+title:  '2018 Steering Committee Election Cycle Kicks Off'
+date:   2018-09-06
+---
+
+**Author**: Paris Pittman (Google), Jorge Castro (Heptio), Ihor Dvoretskyi (CNCF)
+
+Having a clear, definable governance model is crucial for the health of open source projects. For one of the highest velocity projects in the open source world, governance is critical especially for one as large and active as Kubernetes, which is one of the most high-velocity projects in the open source world. A clear structure helps users trust that the project will be nurtured and progress forward. Initially, this structure was laid by the former 7 member bootstrap committee composed of founders and senior contributors with a goal to create the foundational governance building blocks.  
+
+The [initial charter](https://git.k8s.io/steering/charter.md) and establishment of an election process to seat a full Steering Committee was a part of those first building blocks. Last year, the bootstrap committee [kicked off](https://groups.google.com/d/msg/kubernetes-dev/piPuoqFkJwA/mCjwLH81BgAJ) the first Kubernetes Steering Committee election which brought forth 6 new members from the community as voted on by contributors. These new members plus the bootstrap committee formed the [Steering Committee that we know today](https://github.com/kubernetes/steering). This yearly election cycle will continue to ensure that new representatives get cycled through to add different voices and thoughts on the Kubernetes project strategy.
+
+The committee has worked hard on topics that will streamline the project and how we operate. SIG (Special Interest Group) governance was an overarching recurring theme this year: Kubernetes community is not a monolithic organization, but a huge, distributed community, where [Special Interest Groups (SIGs) and Working Groups (WGs)](https://github.com/kubernetes/community/blob/master/sig-list.md) are the atomic community units, that are making Kubernetes so successful from the ground.
+
+### Contributors - this is where you come in.
+
+There are three seats up for election this year. The [voters guide](https://git.k8s.io/community/events/elections/2018) will get you up to speed on the specifics of this years election including candidate bios as they are updated in real time. The [elections process doc](https://github.com/kubernetes/steering/blob/master/elections.md) will steer you towards eligibility, operations, and the fine print.
+
+1) Nominate yourself, someone else, and/or put your support to others.
+
+Want to help chart our course? Interested in governance and community topics? Add your name! _The nomination process is optional_.
+
+2) Vote.
+
+On September 19th, eligible voters will receive an email poll invite conducted by [CIVS](https://civs.cs.cornell.edu). The newly elected will be announced at the [weekly community meeting](https://github.com/kubernetes/community/tree/master/communication#weekly-meeting) on Thursday, October 4th at 5pm UTC.
+
+To those who are running:
+
+<img src="/images/blog/2018-09-06-2018-steering-committee-election-cycle-kicks-off/sc-elections.png" width="400">
+
+### Helpful resources
+
+* [Steering Committee](https://github.com/kubernetes/steering) - who sits on the committee and terms, their projects and meetings info  
+* [Steering Committee Charter](https://git.k8s.io/steering/charter.md) - this is a great read if you’re interested in running (or assessing for the best candidates!)
+* [Election Process](https://git.k8s.io/steering/elections.md)
+* [Voters Guide!](https://git.k8s.io/community/events/elections/2018) - Updated on a rolling basis. This guide will always have the latest information throughout the election cycle. The complete schedule of events and candidate bios will be housed here.
@@ -0,0 +1,140 @@
+---
+layout: blog
+title:  'Hands On With Linkerd 2.0'
+date:   2018-09-18
+---
+
+**Author**: Thomas Rampelberg (Buoyant) 
+
+Linkerd 2.0 was recently announced as generally available (GA), signaling its readiness for production use. In this tutorial, we’ll walk you through how to get Linkerd 2.0 up and running on your Kubernetes cluster in a matter seconds.
+
+But first, what is Linkerd and why should you care? Linkerd is a service sidecar that augments a Kubernetes service, providing zero-config dashboards and UNIX-style CLI tools for runtime debugging, diagnostics, and reliability. Linkerd is also a service mesh, applied to multiple (or all) services in a cluster to provide a uniform layer of telemetry, security, and control across them.
+
+Linkerd works by installing ultralight proxies into each pod of a service. These proxies report telemetry data to, and receive signals from, a control plane. This means that using Linkerd doesn’t require any code changes, and can even be installed live on a running service. Linkerd is fully open source, Apache v2 licensed, and is hosted by the Cloud Native Computing Foundation (just like Kubernetes itself!)
+
+Without further ado, let’s see just how quickly you can get Linkerd running on your Kubernetes cluster. In this tutorial, we’ll walk you through how to deploy Linkerd on any Kubernetes 1.9+ cluster and how to use it to debug failures in a sample gRPC application.
+
+## Step 1: Install the demo app 🚀
+
+Before we install Linkerd, let’s start by installing a basic gRPC demo application called Emojivoto onto your Kubernetes cluster. To install Emojivoto, run:
+
+`curl https://run.linkerd.io/emojivoto.yml | kubectl apply -f -`
+
+This command downloads the Kubernetes manifest for Emojivoto, and uses kubectl to apply it to your Kubernetes cluster. Emojivoto is comprised of several services that run in the “emojivoto” namespace. You can see the services by running:
+
+`kubectl get -n emojivoto deployments`
+
+You can also see the app live by running
+
+`minikube -n emojivoto service web-svc --url # if you’re on minikube`
+
+… or:
+
+`kubectl get svc web-svc -n emojivoto -o jsonpath="{.status.loadBalancer.ingress[0].*}" #`
+
+… if you’re somewhere else
+
+Click around. You might notice that some parts of the application are broken! If you were to inspect your handly local Kubernetes dashboard, you wouldn’t see very much interesting---as far as Kubernetes is concerned, the app is running just fine. This is a very common situation! Kubernetes understands whether your pods are running, but not whether they are responding properly.
+
+In the next few steps, we’ll walk you through how to use Linkerd to diagnose the problem.
+
+## Step 2: Install Linkerd’s CLI
+
+We’ll start by installing Linkerd’s command-line interface (CLI) onto your local machine.  Visit the [Linkerd releases page](https://github.com/linkerd/linkerd2/releases/), or simply run:
+
+`curl -sL https://run.linkerd.io/install | sh`
+
+Once installed, add the `linkerd` command to your path with:
+
+`export PATH=$PATH:$HOME/.linkerd2/bin`
+
+You should now be able to run the command `linkerd version`, which should display:
+
+```
+Client version: v2.0
+Server version: unavailable
+```
+
+“Server version: unavailable” means that we need to add Linkerd’s control plane to the cluster, which we’ll do next. But first, let’s validate that your cluster is prepared for Linkerd by running:
+
+`linkerd check --pre`
+
+This handy command will report any problems that will interfere with your ability to install Linkerd. Hopefully everything looks OK and you’re ready to move on to the next step.
+
+## Step 3: Install Linkerd’s control plane onto the cluster
+
+In this step, we’ll install Linkerd’s lightweight control plane into its own namespace (“linkerd”) on your cluster. To do this, run:
+
+`linkerd install | kubectl apply -f -`
+
+This command generates a Kubernetes manifest and uses `kubectl` command to apply it to your Kubernetes cluster. (Feel free to inspect the manifest before you apply it.)
+
+(Note: if your Kubernetes cluster is on GKE with RBAC enabled, you’ll need an extra step: you must grant a ClusterRole of cluster-admin to your Google Cloud account first, in order to install certain telemetry features in the control plane. To do that, run: `kubectl create clusterrolebinding cluster-admin-binding-$USER --clusterrole=cluster-admin --user=$(gcloud config get-value account)`.)
+
+Depending on the speed of your internet connection, it may take a minute or two for your Kubernetes cluster to pull the Linkerd images. While that’s happening, we can validate that everything’s happening correctly by running:
+
+`linkerd check`
+
+This command will patiently wait until Linkerd has been installed and is running.
+
+Finally, we’re ready to view Linkerd’s dashboard! Just run:
+
+`linkerd dashboard`
+
+If you see something like below, Linkerd is now running on your cluster. 🎉
+
+<center><img src="/images/blog/2018-09-18-2018-linkerd-2.0/1-dashboard.png" width="700"></center>
+
+## Step 4: Add Linkerd to the web service
+
+At this point we have the Linkerd control plane installed in the “linkerd” namespace, and we have our emojivoto demo app installed in the “emojivoto” namespace. But we haven’t actually added Linkerd to our service yet. So let’s do that.
+
+In this example, let’s pretend we are the owners of the “web” service. Other services, like “emoji” and “voting”, are owned by other teams--so we don’t want to touch them.
+
+There are a couple ways to add Linkerd to our service. For demo purposes, the easiest is to do something like this:
+
+`kubectl get -n emojivoto deploy/web -o yaml | linkerd inject - | kubectl apply -f -`
+
+This command retrieves the manifest of the “web” service from Kubernetes, runs this manifest through `linkerd inject`, and finally reapplies it to the Kubernetes cluster. The `linkerd inject` command augments the manifest to include Linkerd’s data plane proxies. As with `linkerd install`, `linkerd inject` is a pure text operation, meaning that you can inspect the input and output before you use it. Since “web” is a Deployment, Kubernetes is kind enough to slowly roll the service one pod at a time--meaning that “web” can be serving traffic live while we add Linkerd to it!
+
+We now have a service sidecar running on the “web” service!
+
+## Step 5: Debugging for Fun and for Profit
+
+Congratulations! You now have a full gRPC application running on your Kubernetes cluster with Linkerd installed on the “web” service. Of course, that application is failing when you use it--so now let’s use Linkerd to track down those errors.
+
+If you glance at the Linkerd dashboard (the `linkerd dashboard` command), you should see all services in the “emojivoto” namespace show up. Since “web” has the Linkerd service sidecar installed on it, you’ll also see success rate, requests per second, and latency percentiles show up.
+
+<center><img src="/images/blog/2018-09-18-2018-linkerd-2.0/2-web-overview.png" width="700"></center>
+
+That’s pretty neat, but the first thing you might notice is that success rate is well below 100%! Click on “web” and let’s dig in.
+
+You should now be looking at the Deployment page for the web service. The first thing you’ll see here is that web is taking traffic from vote-bot (a service included in the Emojivoto manifest to continually generate a low level of live traffic), and has two outgoing dependencies, emoji and voting.
+
+<center><img src="/images/blog/2018-09-18-2018-linkerd-2.0/3-web-detail.png" width="700"></center>
+
+The emoji service is operating at 100%, but the voting service is failing! A failure in a dependent service may be exactly what’s causing the errors that web is returning.
+
+Let’s scroll a little further down the page, we’ll see a live list of all traffic endpoints that “web” is receiving. This is interesting:
+
+<center><img src="/images/blog/2018-09-18-2018-linkerd-2.0/4-web-top.png" width="700"></center>
+
+There are two calls that are not at 100%: the first is vote-bot’s call the “/api/vote” endpoint. The second is the “VotePoop” call from the web service to the voting service. Very interesting! Since /api/vote is an incoming call, and “/VotePoop” is an outgoing call, this is a good clue that  that the failure of the vote service’s VotePoop endpoint is what’s causing the problem!
+
+Finally, if we click on the “tap” icon for that row in the far right column, we’ll be taken to live list of requests that match this endpoint. This allows us to confirm that the requests are failing (they all have [gRPC status code 2](https://godoc.org/google.golang.org/grpc/codes#Code), indicating an error).
+
+<center><img src="/images/blog/2018-09-18-2018-linkerd-2.0/5-web-tap.png" width="700"></center>
+
+At this point we have the ammunition we need to talk to the owners of the vote “voting” service. We’ve identified an endpoint on their service that consistently returns an error, and have found no other obvious sources of failures in the system.
+
+We hope you’ve enjoyed this journey through Linkerd 2.0. There is much more for you to explore. For example, everything we did above using the web UI can also be accomplished via pure CLI commands, e.g. `linkerd top`, `linkerd stat`, and `linkerd tap`.
+
+Also, did you notice the little Grafana icon on the very first page we looked at? Linkerd ships with automatic Grafana dashboards for all those metrics, allowing you to view everything you’re seeing in the Linkerd dashboard in a time series format. Check it out!
+
+<center><img src="/images/blog/2018-09-18-2018-linkerd-2.0/6-grafana.png" width="700"></center>
+
+## Want more?
+
+In this tutorial, we’ve shown you how to install Linkerd on a cluster, add it as a service sidecar to just one service--while the service is receiving live traffic!---and use it to debug a runtime issue. But this is just the tip of the iceberg. We haven’t even touched any of Linkerd’s reliability or security features!
+
+Linkerd has a thriving community of adopters and contributors, and we’d love for YOU to be a part of it. For more, check out the [docs](https://linkerd.io/docs) and [GitHub](https://github.com/linkerd/linkerd) repo, join the [Linkerd Slack](https://slack.linkerd.io/) and mailing lists ([users](https://lists.cncf.io/g/cncf-linkerd-users), [developers](https://lists.cncf.io/g/cncf-linkerd-dev), [announce](https://lists.cncf.io/g/cncf-linkerd-announce)), and, of course, follow [@linkerd](https://twitter.com/linkerd) on Twitter! We can’t wait to have you aboard!
@@ -1,5 +1,6 @@
 ---
 title: Case Studies
+linkTitle: Case Studies
 bigheader: Kubernetes User Case Studies
 abstract: A collection of users running Kubernetes in production.
 layout: basic
@@ -7,108 +8,3 @@ class: gridPage
 cid: caseStudies
 ---

-<section id="mainContent">
-    <main>
-        <div class="content">
-            <div class="case-studies">
-
-              <div class="case-study">
-                  <img src="/images/case_studies/openai_feature.png" alt="openAI">
-                  <p class="quote">"Research teams can now take advantage of the frameworks we’ve built on top of Kubernetes, which make it easy to launch experiments, scale them by 10x or 50x, and take little effort to manage."</p>
-                  <!--<p class="attrib">—  Christopher Berner, Head of Infrastructure for OpenAI</p>-->
-                  <a href="/case-studies/openai/">Read about OpenAI</a>
-              </div>
-
-
-              <div class="case-study">
-                  <img src="/images/case_studies/newyorktimes_feature.png" alt="the new york times">
-                  <p class="quote">"I think once you get over the initial hump, things get a lot easier and actually a lot faster."</p>
-                  <!--<p class="attrib">—  Deep Kapadia, Executive Director, Engineering at The New York Times</p>-->
-                  <a href="/case-studies/newyorktimes/">Read about The New York Times</a>
-              </div>
-
-              <div class="case-study">
-                  <img src="/images/case_studies/nordstrom_feature.png" alt="nordstrom">
-                  <p class="quote">"We are always looking for ways to optimize and provide more value through technology. With Kubernetes we are showcasing two types of efficiency that we can bring: Dev efficiency and Ops efficiency. It’s a win-win."</p>
-                  <!--<p class="attrib">—  Dhawal Patel, senior engineer at Nordstrom</p>-->
-                  <a href="/case-studies/nordstrom/">Read about Nordstrom</a>
-              </div>
-
-              <div class="case-study">
-                  <img src="/images/case_studies/squarespace_feature.png" alt="squarespace">
-                  <p class="quote">"Once you prove that Kubernetes solves one problem, everyone immediately starts solving other problems without you even having to evangelize it."</p>
-                  <!--<p class="attrib">—  Kevin Lynch, Staff Engineer on the Site Reliability team at Squarespace</p>-->
-                  <a href="/case-studies/squarespace/">Read about Squarespace</a>
-              </div>
-
-
-            </div>
-        </div>
-    </main>
-</section>
-
-<section id="video">
-    <main>
-        <!--<div><button onclick="kub.showVideo()"><img src="/images/case_studies/video_thumb.jpg" alt="Zulily video"><h6>Kubernetes at zulily</h6></button></div>-->
-        <div>
-            <!--<h3>Box</h3>-->
-            <h4><i>"Kubernetes has the opportunity to be the new cloud platform. The amount of innovation that's going to come from being able to standardize on Kubernetes as a platform is incredibly exciting - more exciting than anything I've seen in the last 10 years of working on the cloud."</i></h4>
-            <img src="/images/case_studies/box-small.png" id="boxlogo" alt="Box">
-        </div>
-        <div style= "padding-left: 40px;"><button onclick="kub.showVideo()"><img src="/images/case_studies/Video-Clip-Box.png" alt="Box video"><h6>Kubernetes at Box</h6></button></div>
-    </main>
-</section>
-
-<section id="users">
-    <main>
-        <h3>Kubernetes Users</h3>
-        <div id="usersGrid">
-          <a target="_blank" href="/case-studies/openai/"><img src="/images/case_studies/openai_feature.png" alt="OpenAI"></a>
-          <a target="_blank" href="/case-studies/newyorktimes/"><img src="/images/case_studies/newyorktimes_feature.png" alt="The New York Times"></a>
-          <a target="_blank" href="/case-studies/nordstrom/"><img src="/images/case_studies/nordstrom_feature.png" alt="Nordstrom"></a>
-          <a target="_blank" href="/case-studies/crowdfire/"><img src="/images/case_studies/crowdfire_feature.png" alt="Crowdfire"></a>
-          <a target="_blank" href="/case-studies/squarespace/"><img src="/images/case_studies/squarespace_feature.png" alt="Squarespace"></a>
-          <a target="_blank" href="/case-studies/zalando/"><img src="/images/case_studies/zalando_feature.png" alt="Zalando"></a>
-          <a target="_blank" href="/case-studies/amadeus/"><img src="/images/case_studies/amadeus.png" alt="Amadeus"></a>
-          <a target="_blank" href="/case-studies/ancestry/"><img src="/images/case_studies/ancestry.png" alt="Ancestry.com"></a>
-          <a target="_blank" href="/case-studies/blablacar/"><img src="/images/case_studies/blablacar.png" alt="BlaBlaCar"></a>
-          <a target="_blank" href="/case-studies/blackrock/"><img src="/images/case_studies/blackrock2.png" alt="BlackRock"></a>
-          <a target="_blank" href="/case-studies/box/"><img src="/images/case_studies/box_logo.png" alt="box"></a>
-          <a target="_blank" href="/case-studies/buffer/"><img src="/images/case_studies/buffer_logo.png" alt="Buffer"></a>
-          <a target="_blank" href="https://cloud.google.com/customers/ccp-games/"><img src="/images/case_studies/ccp.png" alt="CCP Games"></a>
-          <a target="_blank" href="https://youtu.be/lmeFkH-rHII"><img src="/images/case_studies/comcast_logo.png" alt="Comcast"></a>
-          <a target="_blank" href="http://searchitoperations.techtarget.com/news/450297178/Tech-firms-roll-out-Kubernetes-in-production"><img src="/images/case_studies/concur.png" alt="Concur"></a>
-          <a target="_blank" href="http://www.nextplatform.com/2015/11/12/inside-ebays-shift-to-kubernetes-and-containers-atop-openstack/"><img src="/images/case_studies/ebay_logo.png" alt="Ebay"></a>
-          <a target="_blank" href="http://blogs.wsj.com/cio/2016/02/24/big-changes-in-goldmans-software-emerge-from-small-containers/"><img src="/images/case_studies/gs.png" alt="Goldman Sachs"></a>
-          <a target="_blank" href="/case-studies/golfnow/"><img src="/images/case_studies/golfnow_logo.png" alt="GolfNow"></a>
-          <a target="_blank" href="/case-studies/haufegroup/"><img src="/images/case_studies/haufegroup_logo.png" alt="Haufe Group"></a>
-          <a target="_blank" href="/case-studies/huawei/"><img src="/images/case_studies/huawei.png" alt="Huawei"></a>
-          <a target="_blank" href="https://kubernetes.io/blog/2017/02/inside-jd-com-shift-to-kubernetes-from-openstack"><img src="/images/case_studies/jd.png" alt="JD.COM"></a>
-          <a target="_blank" href="https://www.openstack.org/videos/video/running-kubernetes-on-openstack-at-liveperson"><img src="/images/case_studies/liveperson.png" alt="LivePerson"></a>
-          <a target="_blank" href="https://youtu.be/YkOY7DgXKyw"><img src="/images/case_studies/monzo_logo.png" alt="monzo"></a>
-          <a target="_blank" href="https://www.youtube.com/watch?v=P5qfyv_zGcU"><img src="/images/case_studies/nyt.png" alt="New York Times"></a>
-          <a target="_blank" href="https://openai.com/blog/infrastructure-for-deep-learning"><img src="/images/case_studies/openai.png" alt="OpenAI"></a>
-          <a target="_blank" href="/case-studies/peardeck/"><img src="/images/case_studies/peardeck_logo.png" alt="peardeck"></a>
-          <a href="/case-studies/pearson/"><img src="/images/case_studies/pearson_logo.png" alt="Pearson"></a>
-          <a target="_blank" href="https://cloud.google.com/customers/philips/"><img src="/images/case_studies/philips_logo.png" alt="Philips"></a>
-          <a target="_blank" href="https://cloudplatform.googleblog.com/2016/09/bringing-Pokemon-GO-to-life-on-Google-Cloud.html"><img src="/images/case_studies/pokemon_go_logo.png" alt="Pokemon GO"></a>
-          <a target="_blank" href="http://www.nextplatform.com/2016/05/24/samsung-experts-put-kubernetes-paces/"><img src="/images/case_studies/sds.png" alt="Samsung SDS"></a>
-          <a target="_blank" href="https://youtu.be/4gyeixJLabo"><img src="/images/case_studies/sap.png" alt="SAP"></a>
-          <a target="_blank" href="https://www.youtube.com/watch?v=5378N5iLb2Q"><img src="/images/case_studies/soundcloud.png" alt="SoundCloud"></a>
-          <a target="_blank" href="https://www.youtube.com/watch?v=F3iMkz_NSvU"><img src="/images/case_studies/homeoffice.png" alt="UK Home Office"></a>
-          <a target="_blank" href="http://thenewstack.io/wepay-kubernetes-changed-business/"><img src="/images/case_studies/wepay.png" alt="WePay"></a>
-          <a target="_blank" href="/case-studies/wink/"><img src="/images/case_studies/wink.png" alt="Wink"></a>
-          <a href="/case-studies/wikimedia/"><img src="/images/case_studies/wikimedia_logo.png" alt="Wikimedia"></a>
-          <a target="_blank" href="https://kubernetes.io/blog/2016/10/kubernetes-and-openstack-at-yahoo-japan"><img src="/images/case_studies/yahooJapan_logo.png" alt="Yahoo! Japan"></a>
-          <a target="_blank" href="#" onclick="event.preventDefault(); kub.showVideo()"><img src="/images/case_studies/zulily_logo.png" alt="zulily"></a>
-          <a target="_blank" href="https://docs.google.com/a/google.com/forms/d/e/1FAIpQLScuI7Ye3VQHQTwBASrgkjQDSS5TP0g3AXfFhwSM9YpHgxRKFA/viewform" class="tell-your-story"><img src="/images/case_studies/story.png" alt="Tell your story"></a>
-        </div>
-    </main>
-</section>
-
-
-<div id="videoPlayer">
-    <!--<iframe data-url="https://www.youtube.com/watch?v=of45hYbkIZs" frameborder="0" allowfullscreen></iframe>-->
-    <iframe data-url="https://www.youtube.com/embed/of45hYbkIZs?autoplay=1" frameborder="0" allowfullscreen="true"></iframe>
-    <button id="closeButton"></button>
-</div>
@@ -0,0 +1,118 @@
+---
+title: Adform Case Study
+linkTitle: Adform
+case_study_styles: true
+cid: caseStudies
+css: /css/style_case_studies.css
+logo: adform_featured_logo.png
+draft: false
+featured: true
+weight: 47
+quote: >
+  Kubernetes enabled the self-healing and immutable infrastructure. We can do faster releases, so our developers are really happy. They can ship our features faster than before, and that makes our clients happier.
+---
+
+<div class="banner1 desktop" style="background-image: url('/images/CaseStudy_adform_banner1.jpg')">
+  <h1> CASE STUDY:<img src="/images/adform_logo.png" style="width:15%;margin-bottom:0%" class="header_logo"><br> <div class="subhead">Improving Performance and Morale with Cloud Native
+
+</div></h1>
+
+</div>
+
+<div class="details">
+    Company &nbsp;<b>AdForm</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Location &nbsp;<b>Copenhagen, Denmark</b>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Industry &nbsp;<b>Adtech</b>
+</div>
+
+<hr>
+<section class="section1">
+<div class="cols">
+  <div class="col1">
+    <h2>Challenge</h2>
+    <a href="https://site.adform.com/">Adform’s</a> mission is to provide a secure and transparent full stack of advertising technology to enable digital ads across devices. The company has a large infrastructure: <a href="https://www.openstack.org/">OpenStack</a>-based private clouds running on 1,100 physical servers in 7 data centers around the world, 3 of which were opened in the past year. With the company’s growth, the infrastructure team felt that "our private cloud was not really flexible enough," says IT System Engineer Edgaras Apšega. "The biggest pain point is that our developers need to maintain their virtual machines, so rolling out technology and new software takes time. We were really struggling with our releases, and we didn’t have self-healing infrastructure."
+
+
+<br>
+
+ <h2>Solution</h2>
+    The team, which had already been using <a href="https://prometheus.io/">Prometheus</a> for monitoring, embraced <a href="https://kubernetes.io/">Kubernetes</a> and cloud native practices in 2017. "To start our Kubernetes journey, we had to adapt all our software, so we had to choose newer frameworks," says Apšega. "We also adopted the microservices way, so observability is much better because you can inspect the bug or the services separately."
+
+
+</div>
+
+<div class="col2">
+
+<h2>Impact</h2>
+ "Kubernetes helps our business a lot because our features are coming to market faster," says Apšega. The release process went from several hours to several minutes. Autoscaling has been at least 6 times faster than the semi-manual VM bootstrapping and application deployment required before. The team estimates that the company has experienced cost savings of 4-5x due to less hardware and fewer man hours needed to set up the hardware and virtual machines, metrics, and logging. Utilization of the hardware resources has been reduced as well, with containers notching 2-3 times more efficiency over virtual machines. "The deployments are very easy because developers just push the code and it automatically appears on Kubernetes," says Apšega. Prometheus has also had a positive impact: "It provides high availability for metrics and alerting. We monitor everything starting from hardware to applications. Having all the metrics in <a href="https://grafana.com/">Grafana</a> dashboards provides great insight on your systems."
+
+
+</div>
+
+</div>
+</section>
+<div class="banner2">
+  <div class="banner2text">
+"Kubernetes enabled the self-healing and immutable infrastructure. We can do faster releases, so our developers are really happy. They can ship our features faster than before, and that makes our clients happier."<span style="font-size:14px;letter-spacing:0.12em;padding-top:20px;text-transform:uppercase;line-height:14px"><br><br>— Edgaras Apšega, IT Systems Engineer, Adform</span>
+
+  </div>
+</div>
+
+
+<section class="section2">
+<div class="fullcol">
+<h2>Adform made <a href="https://www.wsj.com/articles/fake-ad-operation-used-to-steal-from-publishers-is-uncovered-1511290981">headlines</a> last year when it detected the HyphBot ad fraud network that was costing some businesses hundreds of thousands of dollars a day.</h2>With its mission to provide a secure and transparent full stack of advertising technology to enable an open internet, Adform published a <a href="https://site.adform.com/media/85132/hyphbot_whitepaper_.pdf">white paper</a> revealing what it did—and others could too—to limit customers’ exposure to the scam. <br><br>
+In that same spirit, Adform is sharing its cloud native journey. "When you see that everyone shares their best practices, it inspires you to contribute back to the project," says IT Systems Engineer Edgaras Apšega.<br><br>
+The company has a large infrastructure: <a href="https://www.openstack.org/">OpenStack</a>-based private clouds running on 1,100 physical servers in their own seven data centers around the world, three of which were opened in the past year. With the company’s growth, the infrastructure team felt that "our private cloud was not really flexible enough," says Apšega. "The biggest pain point is that our developers need to maintain their virtual machines, so rolling out technology and new software really takes time. We were really struggling with our releases, and we didn’t have self-healing infrastructure."
+
+
+</div>
+</section>
+<div class="banner3" style="background-image: url('/images/CaseStudy_adform_banner3.jpg')">
+  <div class="banner3text">
+    "The fact that Cloud Native Computing Foundation incubated Kubernetes was a really big point for us because it was vendor neutral. And we can see that a community really gathers around it. Everyone shares their experiences, their knowledge, and the fact that it’s open source, you can contribute."<span style="font-size:14px;letter-spacing:0.12em;padding-top:20px;text-transform:uppercase;line-height:14px"><br><br>— Edgaras Apšega, IT Systems Engineer, Adform</span>
+  </div>
+</div>
+<section class="section3">
+<div class="fullcol">
+
+The team, which had already been using Prometheus for monitoring, embraced Kubernetes, microservices, and cloud native practices. "The fact that Cloud Native Computing Foundation incubated Kubernetes was a really big point for us because it was vendor neutral," says Apšega. "And we can see that a community really gathers around it."<br><br>
+A proof of concept project was started, with a Kubernetes cluster running on bare metal in the data center. When developers saw how quickly containers could be spun up compared to the virtual machine process, "they wanted to ship their containers in production right away, and we were still doing proof of concept," says IT Systems Engineer Andrius Cibulskis.
+Of course, a lot of work still had to be done. "First of all, we had to learn Kubernetes, see all of the moving parts, how they glue together," says Apšega. "Second of all, the whole CI/CD part had to be redone, and our DevOps team had to invest more man hours to implement it. And third is that developers had to rewrite the code, and they’re still doing it."
+<br><br>
+The first production cluster was launched in the spring of 2018, and is now up to 20 physical machines dedicated for pods throughout three data centers, with plans for separate clusters in the other four data centers. The user-facing Adform application platform, data distribution platform, and back ends are now all running on Kubernetes. "Many APIs for critical applications are being developed for Kubernetes," says Apšega. "Teams are rewriting their applications to .NET core, because it supports containers, and preparing to move to Kubernetes. And new applications, by default, go in containers."
+
+
+</div>
+</section>
+<div class="banner4" style="background-image: url('/images/CaseStudy_adform_banner4.jpg')">
+  <div class="banner4text">
+"Releases are really nice for them, because they just push their code to Git and that’s it. They don’t have to worry about their virtual machines anymore." <span style="font-size:14px;letter-spacing:0.12em;padding-top:20px;text-transform:uppercase;line-height:14px"><br><br>— Andrius Cibulskis, IT Systems Engineer, Adform</span>
+  </div>
+</div>
+
+<section class="section5">
+<div class="fullcol">
+This big push has been driven by the real impact that these new practices have had. "Kubernetes helps our business a lot because our features are coming to market faster," says Apšega. "The deployments are very easy because developers just push the code and it automatically appears on Kubernetes." The release process went from several hours to several minutes. Autoscaling is at least six times faster than the semi-manual VM bootstrapping and application deployment required before. <br><br>
+The team estimates that the company has experienced cost savings of 4-5x due to less hardware and fewer man hours needed to set up the hardware and virtual machines, metrics, and logging. Utilization of the hardware resources has been reduced as well, with containers notching two to three times more efficiency over virtual machines. <br><br>
+Prometheus has also had a positive impact: "It provides high availability for metrics and alerting," says Apšega. "We monitor everything starting from hardware to applications. Having all the metrics in Grafana dashboards provides great insight on our systems."
+
+
+
+</div>
+
+<div class="banner5">
+  <div class="banner5text">
+ "I think that our company just started our cloud native journey. It seems like a huge road ahead, but we’re really happy that we joined it." <span style="font-size:14px;letter-spacing:0.12em;padding-top:20px;text-transform:uppercase;line-height:14px"><br><br>— Edgaras Apšega, IT Systems Engineer, Adform</span>
+  </div>
+</div>
+
+<div class="fullcol">
+All of these benefits have trickled down to individual team members, whose working lives have been changed for the better. "They used to have to get up at night to re-start some services, and now Kubernetes handles all of that," says Apšega. Adds Cibulskis: "Releases are really nice for them, because they just push their code to Git and that’s it. They don’t have to worry about their virtual machines anymore." Even the security teams have been impacted. "Security teams are always not happy," says Apšega, "and now they’re happy because they can easily inspect the containers."
+The company plans to remain in the data centers for now, "mostly because we want to keep all the data, to not share it in any way," says Cibulskis, "and it’s cheaper at our scale." But, Apšega says, the possibility of using a hybrid cloud for computing is intriguing: "One of the projects we’re interested in is the <a href="https://github.com/virtual-kubelet/virtual-kubelet">Virtual Kubelet</a> that lets you spin up the working nodes on different clouds to do some computing."
+<br><br>
+Apšega, Cibulskis and their colleagues are keeping tabs on how the cloud native ecosystem develops, and are excited to contribute where they can. "I think that our company just started our cloud native journey," says Apšega. "It seems like a huge road ahead, but we’re really happy that we joined it."
+
+
+
+</div>
+
+</section>
@@ -7,7 +7,7 @@ css: /css/style_ancestry.css
 ---

 <div class="banner1">
-  <h1> CASE STUDY:<img src="http://c.mfcreative.com/i/logo/ancestry-on-dark.svg" width="22%" style="margin-bottom:-12px;margin-left:3px;"><br> <div class="subhead">Digging Into the Past With New Technology</div></h1>
+  <h1> CASE STUDY:<img src="https://c.mfcreative.com/i/logo/ancestry-on-dark.svg" width="22%" style="margin-bottom:-12px;margin-left:3px;"><br> <div class="subhead">Digging Into the Past With New Technology</div></h1>

 </div>

@@ -1,9 +1,12 @@
 ---
 title: Box Case Study
-
 case_study_styles: true
 cid: caseStudies
 css: /css/style_box.css
+video: https://www.youtube.com/embed/of45hYbkIZs?autoplay=1
+quote: >
+  Kubernetes has the opportunity to be the new cloud platform. The amount of innovation that's going to come from being able to standardize on Kubernetes as a platform is incredibly exciting - more exciting than anything I've seen in the last 10 years of working on the cloud.
+
 ---

 <div class="banner1">
--- a/Show More
+++ b/Show More