update with master content resolving merge conflicts
This commit is contained in:
MAKOSCAFEE
@@ -44,12 +44,12 @@ Kubernetes is open source giving you the freedom to take advantage of on-premise
|
||||
<br>
|
||||
<br>
|
||||
<br>
|
||||
<a href="https://events.linuxfoundation.org/events/kubecon-cloudnativecon-europe-2019" button id="desktopKCButton">Attend KubeCon in Barcelona on May 20-23, 2019</a>
|
||||
<br>
|
||||
<br>
|
||||
<br>
|
||||
<br>
|
||||
<a href="https://www.lfasiallc.com/events/kubecon-cloudnativecon-china-2019" button id="desktopKCButton">Attend KubeCon in Shanghai on June 24-26, 2019</a>
|
||||
<br>
|
||||
<br>
|
||||
<br>
|
||||
<br>
|
||||
<a href="https://events.linuxfoundation.org/events/kubecon-cloudnativecon-north-america-2019" button id="desktopKCButton">Attend KubeCon in San Diego on Nov. 18-21, 2019</a>
|
||||
</div>
|
||||
<div id="videoPlayer">
|
||||
<iframe data-url="https://www.youtube.com/embed/H06qrNmGqyE?autoplay=1" frameborder="0" allowfullscreen></iframe>
|
||||
|
||||
+15
-6
@@ -1,11 +1,20 @@
|
||||
# See the OWNERS docs at https://go.k8s.io/owners
|
||||
|
||||
# Owned by Kubernetes Blog reviewers.
|
||||
# Owned by Kubernetes Blog reviewers
|
||||
|
||||
options:
|
||||
no_parent_owners: false
|
||||
reviewers:
|
||||
- kbarnard10
|
||||
|
||||
approvers:
|
||||
- bobsky
|
||||
- natekartchner
|
||||
- sarahkconway
|
||||
- kbarnard10
|
||||
- zacharysarah
|
||||
|
||||
reviewers:
|
||||
- vonguard
|
||||
- castrojo
|
||||
- mrbobbytables
|
||||
- cody-clark
|
||||
- parispittman
|
||||
|
||||
labels:
|
||||
- area/blog
|
||||
|
||||
@@ -140,7 +140,7 @@ Release Notes:
|
||||
* Update Exec and Portforward client to use pod subresource [#7715][127] (csrwng)
|
||||
* Added NFS to PV structs [#7564][128] (markturansky)
|
||||
* Fix environment variable error in Vagrant docs [#7904][129] (posita)
|
||||
* Adds a simple release-note builder that scrapes the Github API for recent PRs [#7616][130](brendandburns)
|
||||
* Adds a simple release-note builder that scrapes the GitHub API for recent PRs [#7616][130](brendandburns)
|
||||
* Scheduler ignores nodes that are in a bad state [#7668][131] (bprashanth)
|
||||
* Set GOMAXPROCS for etcd [#7863][132] (fgrzadkowski)
|
||||
* Auto-generated conversion methods calling one another [#7556][133] (wojtek-t)
|
||||
@@ -444,7 +444,7 @@ Simple theme. Powered by [Blogger][385].
|
||||
[127]: https://github.com/GoogleCloudPlatform/kubernetes/pull/7715 "Update Exec and Portforward client to use pod subresource"
|
||||
[128]: https://github.com/GoogleCloudPlatform/kubernetes/pull/7564 "Added NFS to PV structs"
|
||||
[129]: https://github.com/GoogleCloudPlatform/kubernetes/pull/7904 "Fix environment variable error in Vagrant docs"
|
||||
[130]: https://github.com/GoogleCloudPlatform/kubernetes/pull/7616 "Adds a simple release-note builder that scrapes the Github API for recent PRs"
|
||||
[130]: https://github.com/GoogleCloudPlatform/kubernetes/pull/7616 "Adds a simple release-note builder that scrapes the GitHub API for recent PRs"
|
||||
[131]: https://github.com/GoogleCloudPlatform/kubernetes/pull/7668 "Scheduler ignores nodes that are in a bad state"
|
||||
[132]: https://github.com/GoogleCloudPlatform/kubernetes/pull/7863 "Set GOMAXPROCS for etcd"
|
||||
[133]: https://github.com/GoogleCloudPlatform/kubernetes/pull/7556 "Auto-generated conversion methods calling one another"
|
||||
|
||||
@@ -169,7 +169,7 @@ The LPC demo was done with a simple container that did not require network conne
|
||||
In May 2015, the criu branch of libcontainer was merged into master. Using the newly-introduced lightweight [runC](https://blog.docker.com/2015/06/runc/) container runtime, container migration was demo’ed at DockerCon15. In this
|
||||
[](https://www.youtube.com/watch?v=?mL9AFkJJAq0) (minute 23:00), a container running Quake was checkpointed and restored on a different machine, effectively implementing container migration.
|
||||
|
||||
At the time of this writing, there are two repos on Github that have native C/R support in Docker:
|
||||
At the time of this writing, there are two repos on GitHub that have native C/R support in Docker:
|
||||
- [Docker 1.5](https://github.com/SaiedKazemi/docker/tree/cr) (old libcontainer, relatively stable)
|
||||
- [Docker 1.7](https://github.com/boucher/docker/tree/cr-combined) (newer, less stable)
|
||||
|
||||
|
||||
@@ -12,7 +12,7 @@ Now you got your Raspberry Pi Cluster all setup, it is time to run some software
|
||||
First step is to make sure every Pi has Hypriot running, if not yet please check the [getting started guide](http://blog.hypriot.com/getting-started-with-docker-on-your-arm-device/) of them. Also hook up the cluster switch to a network so that Internet is available and every Pi get an IP-address assigned via DHCP. Because we will be running multiple Pi’s it is practical to give each Pi a unique hostname. I renamed my Pi’s to rpi-master, rpi-node-1, rpi-node-2, etc for my convenience. Note that on Hypriot the hostname is set by editing the /boot/occidentalis.txt file, not the /etc/hostname. You could also set the hostname using the Hypriot flash tool.
|
||||
|
||||
|
||||
The most important thing about running software on a Pi is the availability of an ARM distribution. Thanks to [Brendan Burns](https://twitter.com/brendandburns), there are Kubernetes components for ARM available in the [Google Cloud Registry](https://cloud.google.com/container-registry/docs/). That’s great. The second hurdle is how to install Kubernetes. There are two ways; directly on the system or in a Docker container. Although the container support has an experimental status, I choose to go for that because it makes it easier to install Kubernetes for you. Kubernetes requires several processes (etcd, flannel, kubectl, etc) to run on a node, which should be started in a specific order. To ease that, systemd services are made available to start the necessary processes in the right way. Also the systemd services make sure that Kubernetes is spun up when a node is (re)booted. To make the installation real easy I created an simple install script for the master node and the worker nodes. All is available at [Github](https://github.com/awassink/k8s-on-rpi). So let’s get started now!
|
||||
The most important thing about running software on a Pi is the availability of an ARM distribution. Thanks to [Brendan Burns](https://twitter.com/brendandburns), there are Kubernetes components for ARM available in the [Google Cloud Registry](https://cloud.google.com/container-registry/docs/). That’s great. The second hurdle is how to install Kubernetes. There are two ways; directly on the system or in a Docker container. Although the container support has an experimental status, I choose to go for that because it makes it easier to install Kubernetes for you. Kubernetes requires several processes (etcd, flannel, kubectl, etc) to run on a node, which should be started in a specific order. To ease that, systemd services are made available to start the necessary processes in the right way. Also the systemd services make sure that Kubernetes is spun up when a node is (re)booted. To make the installation real easy I created an simple install script for the master node and the worker nodes. All is available at [GitHub](https://github.com/awassink/k8s-on-rpi). So let’s get started now!
|
||||
|
||||
### Installing the Kubernetes master node
|
||||
|
||||
|
||||
@@ -22,7 +22,7 @@ Note taker: Rob Hirschfeld
|
||||
* *SIG Report:*
|
||||
* Release Automation and an introduction to David McMahon
|
||||
* Docs and k8s website redesign proposal and an introduction to John Mulhausen
|
||||
* This will allow the system to build docs correctly from Github w/ minimal effort
|
||||
* This will allow the system to build docs correctly from GitHub w/ minimal effort
|
||||
* Will be check-in triggered
|
||||
* Getting website style updates
|
||||
* Want to keep authoring really light
|
||||
@@ -33,7 +33,7 @@ Note taker: Rob Hirschfeld
|
||||
* discussion about release criteria: we will hold release date for bugs
|
||||
* Testing flake surge is over (one time event and then maintain test stability)
|
||||
* 1.3 Planning (time +40 minutes)
|
||||
* working to cleanup the Github milestones — they should be a source of truth. you can use Github for bug reporting
|
||||
* working to cleanup the GitHub milestones — they should be a source of truth. you can use GitHub for bug reporting
|
||||
* push off discussion while 1.2 crunch is under
|
||||
* Framework
|
||||
* dates
|
||||
|
||||
@@ -14,7 +14,7 @@ _--Justin_
|
||||
|
||||
You started with an excellent foundation - good declarative functionality, built around a solid API with a well defined schema and the machinery so that we could evolve going forwards. And sure enough, over your first year you grew so fast: autoscaling, HTTP load-balancing support (Ingress), support for persistent workloads including clustered databases (PetSets). You’ve made friends with more clouds (welcome Azure & OpenStack to the family), and even started to span zones and clusters (Federation). And these are just some of the most visible changes - there’s so much happening inside that brain of yours!
|
||||
|
||||
I think it’s wonderful you’ve remained so open in all that you do - you seem to write down everything on Github - for better or worse. I think we’ve all learned a lot about that on the way, like the perils of having engineers make scaling statements that are then weighed against claims made without quite the same framework of precision and rigor. But I’m proud that you chose not to lower your standards, but rose to the challenge and just ran faster instead - it might not be the most realistic approach, but it is the only way to move mountains!
|
||||
I think it’s wonderful you’ve remained so open in all that you do - you seem to write down everything on GitHub - for better or worse. I think we’ve all learned a lot about that on the way, like the perils of having engineers make scaling statements that are then weighed against claims made without quite the same framework of precision and rigor. But I’m proud that you chose not to lower your standards, but rose to the challenge and just ran faster instead - it might not be the most realistic approach, but it is the only way to move mountains!
|
||||
|
||||
And yet, somehow, you’ve managed to avoid a lot of the common dead-ends that other open source software has fallen into, particularly as those projects got bigger and the developers end up working on it more than they use it directly. How did you do that? There’s a probably-apocryphal story of an employee at IBM that makes a huge mistake, and is summoned to meet with the big boss, expecting to be fired, only to be told “We just spent several million dollars training you. Why would we want to fire you?”. Despite all the investment google is pouring into you (along with Redhat and others), I sometimes wonder if the mistakes we are avoiding could be worth even more. There is a very open development process, yet there’s also an “oracle” that will sometimes course-correct by telling us what happens two years down the road if we make a particular design decision. This is a parent you should probably listen to!
|
||||
|
||||
|
||||
@@ -34,7 +34,7 @@ With interest growing in stateful apps, work has already started to further enha
|
||||
Join the conversation and contribute! Here are some places to get started:
|
||||
|
||||
|
||||
- Product Management [group](https://groups.google.com/forum/#!forum/kubernetes-pm)
|
||||
- Product Management [group](https://groups.google.com/forum/#!forum/kubernetes-sig-pm)
|
||||
- Kubernetes [Storage SIG](https://groups.google.com/forum/#!forum/kubernetes-sig-storage)
|
||||
- Kubernetes [Cluster Federation SIG](https://groups.google.com/forum/#!forum/kubernetes-sig-federation)
|
||||
|
||||
|
||||
@@ -52,7 +52,7 @@ We’re done with step 1. In my experience it usually takes less than 5 minutes
|
||||
|
||||
|
||||
## Deploy an Application to Kubernetes
|
||||
First go to [Codefresh and create an account using Github, Bitbucket, or Gitlab](https://codefresh.io/kubernetes-deploy/). As mentioned previously, Codefresh is free for both open source and smaller private projects. We’ll use it to create the configuration Yaml necessary to deploy our application to Kubernetes. Then we'll deploy our application and automate the process to happen every time we commit code changes. Here are the steps:
|
||||
First go to [Codefresh and create an account using GitHub, Bitbucket, or Gitlab](https://codefresh.io/kubernetes-deploy/). As mentioned previously, Codefresh is free for both open source and smaller private projects. We’ll use it to create the configuration Yaml necessary to deploy our application to Kubernetes. Then we'll deploy our application and automate the process to happen every time we commit code changes. Here are the steps:
|
||||
|
||||
1. 1.Create a Codefresh account
|
||||
2. 2.Connect to Google Cloud (or other cluster)
|
||||
|
||||
@@ -5,7 +5,7 @@ slug: introducing-kubeflow-composable
|
||||
url: /blog/2017/12/Introducing-Kubeflow-Composable
|
||||
---
|
||||
|
||||
**_Today’s post is by David Aronchick and Jeremy Lewi, a PM and Engineer on the Kubeflow project, a new open source Github repo dedicated to making using machine learning (ML) stacks on Kubernetes easy, fast and extensible._**
|
||||
**_Today’s post is by David Aronchick and Jeremy Lewi, a PM and Engineer on the Kubeflow project, a new open source GitHub repo dedicated to making using machine learning (ML) stacks on Kubernetes easy, fast and extensible._**
|
||||
|
||||
|
||||
|
||||
@@ -19,7 +19,7 @@ Worse, these deployments are so tied to the clusters they have been deployed to
|
||||
|
||||
|
||||
## Introducing Kubeflow
|
||||
To address these concerns, we’re announcing the creation of the Kubeflow project, a new open source Github repo dedicated to making using ML stacks on Kubernetes easy, fast and extensible. This repository contains:
|
||||
To address these concerns, we’re announcing the creation of the Kubeflow project, a new open source GitHub repo dedicated to making using ML stacks on Kubernetes easy, fast and extensible. This repository contains:
|
||||
|
||||
- JupyterHub to create & manage interactive Jupyter notebooks
|
||||
- A Tensorflow [Custom Resource](/docs/concepts/api-extension/custom-resources/) (CRD) that can be configured to use CPUs or GPUs, and adjusted to the size of a cluster with a single setting
|
||||
|
||||
@@ -86,14 +86,14 @@ It’d be impossible to have gotten where we are without enormous help from ever
|
||||
* [PyTorch operator](https://github.com/kubeflow/pytorch-operator) for running PyTorch jobs
|
||||
* [Seldon Core](https://github.com/kubeflow/kubeflow/tree/master/kubeflow/seldon) for running complex model deployments and non-TensorFlow serving
|
||||
|
||||
It’s difficult to overstate how much the community has helped bring all these projects (and more) to fruition. Just a few of the contributing companies include: Alibaba Cloud, Ant Financial, Caicloud, Canonical, Cisco, Datawire, Dell, Github, Google, Heptio, Huawei, Intel, Microsoft, Momenta, One Convergence, Pachyderm, Project Jupyter, Red Hat, Seldon, Uber and Weaveworks.
|
||||
It’s difficult to overstate how much the community has helped bring all these projects (and more) to fruition. Just a few of the contributing companies include: Alibaba Cloud, Ant Financial, Caicloud, Canonical, Cisco, Datawire, Dell, GitHub, Google, Heptio, Huawei, Intel, Microsoft, Momenta, One Convergence, Pachyderm, Project Jupyter, Red Hat, Seldon, Uber and Weaveworks.
|
||||
|
||||
# Learning More
|
||||
|
||||
If you’d like to try out Kubeflow, we have a number of options for you:
|
||||
|
||||
1. You can use sample walkthroughs hosted on [Katacoda](https://www.katacoda.com/kubeflow)
|
||||
2. You can follow a guided tutorial with existing models from the [examples repository](https://github.com/kubeflow/examples). These include the [Github Issue Summarization](https://github.com/kubeflow/examples/tree/master/github_issue_summarization), [MNIST](https://github.com/kubeflow/examples/tree/master/mnist) and [Reinforcement Learning with Agents](https://github.com/kubeflow/examples/tree/master/agents).
|
||||
2. You can follow a guided tutorial with existing models from the [examples repository](https://github.com/kubeflow/examples). These include the [GitHub Issue Summarization](https://github.com/kubeflow/examples/tree/master/github_issue_summarization), [MNIST](https://github.com/kubeflow/examples/tree/master/mnist) and [Reinforcement Learning with Agents](https://github.com/kubeflow/examples/tree/master/agents).
|
||||
3. You can start a cluster on your own and try your own model. Any Kubernetes conformant cluster will support Kubeflow including those from contributors [Caicloud](https://www.prnewswire.com/news-releases/caicloud-releases-its-kubernetes-based-cluster-as-a-service-product-claas-20-and-the-first-tensorflow-as-a-service-taas-11-while-closing-6m-series-a-funding-300418071.html), [Canonical](https://jujucharms.com/canonical-kubernetes/), [Google](https://cloud.google.com/kubernetes-engine/docs/how-to/creating-a-container-cluster), [Heptio](https://heptio.com/products/kubernetes-subscription/), [Mesosphere](https://github.com/mesosphere/dcos-kubernetes-quickstart), [Microsoft](https://docs.microsoft.com/en-us/azure/aks/kubernetes-walkthrough), [IBM](https://cloud.ibm.com/docs/containers?topic=containers-cs_cluster_tutorial#cs_cluster_tutorial), [Red Hat/Openshift ](https://docs.openshift.com/container-platform/3.3/install_config/install/quick_install.html#install-config-install-quick-install)and [Weaveworks](https://www.weave.works/product/cloud/).
|
||||
|
||||
There were also a number of sessions at KubeCon + CloudNativeCon EU 2018 covering Kubeflow. The links to the talks are here; the associated videos will be posted in the coming days.
|
||||
|
||||
@@ -23,6 +23,6 @@ The version of Kubernetes at that point was really just a shadow of what it was
|
||||
<center><iframe width="560" height="315" src="https://www.youtube.com/embed/YrxnVKZeqK8" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe></center>
|
||||
|
||||
|
||||
But, however raw, that modest start was enough to pique the interest of a community that started strong and has only gotten stronger. Over the past four years Kubernetes has exceeded the expectations of all of us that were there early on. We owe the Kubernetes community a huge debt. The success the project has seen is based not just on code and technology but also the way that an amazing group of people have come together to create something special. The best expression of this is the [set of Kubernetes values](https://github.com/kubernetes/steering/blob/master/values.md) that Sarah Novotny helped curate.
|
||||
But, however raw, that modest start was enough to pique the interest of a community that started strong and has only gotten stronger. Over the past four years Kubernetes has exceeded the expectations of all of us that were there early on. We owe the Kubernetes community a huge debt. The success the project has seen is based not just on code and technology but also the way that an amazing group of people have come together to create something special. The best expression of this is the [set of Kubernetes values](https://git.k8s.io/community/values.md) that Sarah Novotny helped curate.
|
||||
|
||||
Here is to another 4 years and beyond! 🎉🎉🎉
|
||||
|
||||
@@ -167,10 +167,10 @@ You can find out more on the [CoreDNS Blog](https://coredns.io/blog).
|
||||
|
||||
CoreDNS is an incubated [CNCF](https:://cncf.io) project.
|
||||
|
||||
We're most active on Slack (and Github):
|
||||
We're most active on Slack (and GitHub):
|
||||
|
||||
- Slack: #coredns on <https://slack.cncf.io>
|
||||
- Github: <https://github.com/coredns/coredns>
|
||||
- GitHub: <https://github.com/coredns/coredns>
|
||||
|
||||
More resources can be found:
|
||||
|
||||
|
||||
@@ -76,7 +76,7 @@ Don't forget: keeping these logs inside the cluster is a security threat in
|
||||
|
||||
**Centralising authentication and authorisation across an organisation (aka Single Sign On) helps onboarding, offboarding, and consistent permissions for users**.
|
||||
|
||||
Integrating Kubernetes with third party auth providers (like Google or Github) uses the remote platform's identity guarantees (backed up by things like 2FA) and prevents administrators having to reconfigure the Kubernetes API server to add or remove users.
|
||||
Integrating Kubernetes with third party auth providers (like Google or GitHub) uses the remote platform's identity guarantees (backed up by things like 2FA) and prevents administrators having to reconfigure the Kubernetes API server to add or remove users.
|
||||
|
||||
[Dex](https://github.com/coreos/dex) is an OpenID Connect Identity (OIDC) and OAuth 2.0 provider with pluggable connectors. Pusher takes this a stage further with [some custom tooling](https://thenewstack.io/kubernetes-single-sign-one-less-identity/), and there are some [other](https://github.com/negz/kuberos) [helpers](https://github.com/micahhausler/k8s-oidc-helper) available with slightly different use cases.
|
||||
|
||||
|
||||
@@ -47,5 +47,5 @@ Kubebuilder is a project under [SIG API Machinery][SIG-APIMachinery] and is bein
|
||||
|
||||
- Kubebuilder [chat room on Slack][slack-channel]
|
||||
- SIG [mailing list][mailing-list]
|
||||
- [Github issues][open-an-issue]
|
||||
- [GitHub issues][open-an-issue]
|
||||
- Send a pull request in the [kubebuilder repo][kubebuilder-repo]
|
||||
|
||||
+1
-1
@@ -6,7 +6,7 @@ date: 2018-08-29
|
||||
|
||||
**Author**: Aaron Crickenberger (Google) and Benjamin Elder (Google)
|
||||
|
||||
_“Large projects have a lot of less exciting, yet, hard work. We value time spent automating repetitive work more highly than toil. Where that work cannot be automated, it is our culture to recognize and reward all types of contributions. However, heroism is not sustainable.”_ - [Kubernetes Community Values](https://github.com/kubernetes/steering/blob/master/values.md#automation-over-process)
|
||||
_“Large projects have a lot of less exciting, yet, hard work. We value time spent automating repetitive work more highly than toil. Where that work cannot be automated, it is our culture to recognize and reward all types of contributions. However, heroism is not sustainable.”_ - [Kubernetes Community Values](https://git.k8s.io/community/values.md#automation-over-process)
|
||||
|
||||
Like many open source projects, Kubernetes is hosted on GitHub. We felt the barrier to participation would be lowest if the project lived where developers already worked, using tools and processes developers already knew. Thus the project embraced the service fully: it was the basis of our workflow, our issue tracker, our documentation, our blog platform, our team structure, and more.
|
||||
|
||||
|
||||
@@ -10,7 +10,7 @@ It was May 2018 in Copenhagen, and the Kubernetes community was enjoying the con
|
||||
|
||||
**This all led to an effort called the [Non-Code Contributor’s Guide](https://github.com/kubernetes/community/blob/master/contributors/guide/non-code-contributions.md).**
|
||||
|
||||
Now, it’s important to note that Kubernetes is rare, if not unique, in the open source world, in that it was defined very early on as both a project and a community. While the project itself is focused on the codebase, it is the community of people driving it forward that makes the project successful. The community works together with an explicit set of [community values](https://github.com/kubernetes/steering/blob/master/values.md), guiding the day-to-day behavior of contributors whether on GitHub, Slack, Discourse, or sitting together over tea or coffee.
|
||||
Now, it’s important to note that Kubernetes is rare, if not unique, in the open source world, in that it was defined very early on as both a project and a community. While the project itself is focused on the codebase, it is the community of people driving it forward that makes the project successful. The community works together with an explicit set of [community values](https://git.k8s.io/community/values.md), guiding the day-to-day behavior of contributors whether on GitHub, Slack, Discourse, or sitting together over tea or coffee.
|
||||
|
||||
By having a community that values people first, and explicitly values a diversity of people, the Kubernetes project is building a product to serve people with diverse needs. The different backgrounds of the contributors bring different approaches to the problem solving, with different methods of collaboration, and all those different viewpoints ultimately create a better project.
|
||||
|
||||
@@ -28,7 +28,7 @@ Some of the ways that you can contribute to the Kubernetes community without wri
|
||||
- Project, program, and [product management](https://github.com/kubernetes/community/blob/master/sig-pm/README.md)
|
||||
- And many more!
|
||||
|
||||
The guide to get started with Kubernetes project contribution is [documented on Github](https://github.com/kubernetes/community/tree/master/contributors/guide), and as the Non-Code Contributors Guide is a part of that Kubernetes Contributors Guide, it can be found [here](https://github.com/kubernetes/community/blob/master/contributors/guide/non-code-contributions.md). As stated earlier, this list is not exhaustive and will continue to be a work in progress.
|
||||
The guide to get started with Kubernetes project contribution is [documented on GitHub](https://github.com/kubernetes/community/tree/master/contributors/guide), and as the Non-Code Contributors Guide is a part of that Kubernetes Contributors Guide, it can be found [here](https://github.com/kubernetes/community/blob/master/contributors/guide/non-code-contributions.md). As stated earlier, this list is not exhaustive and will continue to be a work in progress.
|
||||
|
||||
To date, the typical Non-Code contributions fall into the following categories:
|
||||
|
||||
|
||||
@@ -139,8 +139,7 @@ logs-web-1 us-central1-a
|
||||
|
||||
## How can I learn more?
|
||||
|
||||
Official documentation on the topology-aware dynamic provisioning feature is available here:
|
||||
/docs/concepts/storage/storage-classes/#volume-binding-mode
|
||||
Official documentation on the topology-aware dynamic provisioning feature is available [here](/docs/concepts/storage/storage-classes/#volume-binding-mode)
|
||||
|
||||
Documentation for CSI drivers is available at https://kubernetes-csi.github.io/docs/
|
||||
|
||||
|
||||
@@ -0,0 +1,32 @@
|
||||
---
|
||||
title: 'Process ID Limiting for Stability Improvements in Kubernetes 1.14'
|
||||
date: 2019-04-15
|
||||
---
|
||||
|
||||
**Author: Derek Carr**
|
||||
|
||||
Have you ever seen someone take more than their fair share of the cookies? The one person who reaches in and grabs a half dozen fresh baked chocolate chip chunk morsels and skitters off like Cookie Monster exclaiming “Om nom nom nom.”
|
||||
|
||||
In some rare workloads, a similar occurrence was taking place inside Kubernetes clusters. With each Pod and Node, there comes a finite number of possible process IDs (PIDs) for all applications to share. While it is rare for any one process or pod to reach in and grab all the PIDs, some users were experiencing resource starvation due to this type of behavior. So in Kubernetes 1.14, we introduced an enhancement to mitigate the risk of a single pod monopolizing all of the PIDs available.
|
||||
|
||||
## Can You Spare Some PIDs?
|
||||
|
||||
Here, we’re talking about the greed of certain containers. Outside the ideal, runaway processes occur from time to time, particularly in clusters where testing is taking place. Thus, some wildly non-production-ready activity is happening.
|
||||
|
||||
In such a scenario, it’s possible for something akin to a fork bomb taking place inside a node. As resources slowly erode, being taken over by some zombie-like process that continually spawns children, other legitimate workloads begin to get bumped in favor of this inflating balloon of wasted processing power. This could result in other processes on the same pod being starved of their needed PIDs. It could also lead to interesting side effects as a node could fail and a replica of that pod is scheduled to a new machine where the process repeats across your entire cluster.
|
||||
|
||||
## Fixing the Problem
|
||||
|
||||
Thus, in Kubernetes 1.14, we have added a feature that allows for the configuration of a kubelet to limit the number of PIDs a given pod can consume. If that machine supports 32,768 PIDs and 100 pods, one can give each pod a budget of 300 PIDs to prevent total exhaustion of PIDs. If the admin wants to overcommit PIDs similar to cpu or memory, they may do so as well with some additional risks. Either way, no one pod can bring the whole machine down. This will generally prevent against simple fork bombs from taking over your cluster.
|
||||
|
||||
This change allows administrators to protect one pod from another, but does not ensure if all pods on the machine can protect the node, and the node agents themselves from falling over. Thus, we’ve introduced a feature in this release in alpha form that provides isolation of PIDs from end user workloads on a pod from the node agents (kubelet, runtime, etc.). The admin is able to reserve a specific number of PIDs--similar to how one reserves CPU or memory today--and ensure they are never consumed by pods on that machine. Once that graduates from alpha, to beta, then stable in future releases of Kubernetes, we’ll have protection against an easily starved Linux resource.
|
||||
|
||||
Get started with [Kubernetes 1.14](https://github.com/kubernetes/kubernetes/releases/tag/v1.14.0).
|
||||
|
||||
## Get Involved
|
||||
|
||||
If you have feedback for this feature or are interested in getting involved with the design and development, join the [Node Special Interest Group](https://github.com/kubernetes/community/tree/master/sig-node).
|
||||
|
||||
### About the author:
|
||||
Derek Carr is Senior Principal Software Engineer at Red Hat. He is a Kubernetes contributor and member of the Kubernetes Community Steering Committee.
|
||||
|
||||
+174
@@ -0,0 +1,174 @@
|
||||
---
|
||||
title: 'Running Kubernetes locally on Linux with Minikube - now with Kubernetes 1.14 support'
|
||||
date: 2019-03-28
|
||||
---
|
||||
**Author**: [Ihor Dvoretskyi](https://twitter.com/idvoretskyi), Developer Advocate, Cloud Native Computing Foundation
|
||||
|
||||
<center>{{<figure width="600" src="/images/blog/2019-03-28-running-kubernetes-locally-on-linux-with-minikube/ihor-dvoretskyi-1470985-unsplash.jpg">}}</center>
|
||||
|
||||
*A few days ago, the Kubernetes community announced [Kubernetes 1.14](https://kubernetes.io/blog/2019/03/25/kubernetes-1-14-release-announcement/), the most recent version of Kubernetes. Alongside it, Minikube, a part of the Kubernetes project, recently hit the [1.0 milestone](https://github.com/kubernetes/minikube/releases/tag/v1.0.0), which supports [Kubernetes 1.14](https://kubernetes.io/blog/2019/03/25/kubernetes-1-14-release-announcement/) by default.*
|
||||
|
||||
Kubernetes is a real winner (and a de facto standard) in the world of distributed Cloud Native computing. While it can handle up to [5000 nodes](https://kubernetes.io/blog/2017/03/scalability-updates-in-kubernetes-1.6) in a single cluster, local deployment on a single machine (e.g. a laptop, a developer workstation, etc.) is an increasingly common scenario for using Kubernetes.
|
||||
|
||||
A few weeks ago I ran a poll on Twitter asking the community to specify their preferred option for running Kubernetes locally on Linux:
|
||||
|
||||
<center>{{< tweet 1093154369040773120 >}}</center>
|
||||
|
||||
This is post #1 in a series about the local deployment options on Linux, and it will cover Minikube, the most popular community-built solution for running Kubernetes on a local machine.
|
||||
|
||||
[Minikube](https://github.com/kubernetes/minikube) is a cross-platform, community-driven [Kubernetes](https://kubernetes.io/) distribution, which is targeted to be used primarily in local environments. It deploys a single-node cluster, which is an excellent option for having a simple Kubernetes cluster up and running on localhost.
|
||||
|
||||
Minikube is designed to be used as a virtual machine (VM), and the default VM runtime is [VirtualBox](https://www.virtualbox.org/). At the same time, extensibility is one of the critical benefits of Minikube, so it's possible to use it with [drivers](https://github.com/kubernetes/minikube/blob/master/docs/drivers.md) outside of VirtualBox.
|
||||
|
||||
By default, Minikube uses Virtualbox as a runtime for running the virtual machine. Virtualbox is a cross-platform solution, which can be used on a variety of operating systems, including GNU/Linux, Windows, and macOS.
|
||||
|
||||
At the same time, QEMU/KVM is a Linux-native virtualization solution, which may offer benefits compared to Virtualbox. For example, it's much easier to use KVM on a GNU/Linux server, so you can run a single-node Minikube cluster not only on a Linux workstation or laptop with GUI, but also on a remote headless server.
|
||||
|
||||
Unfortunately, Virtualbox and KVM can't be used simultaneously, so if you are already running KVM workloads on a machine and want to run Minikube there as well, using the KVM minikube driver is the preferred way to go.
|
||||
|
||||
In this guide, we'll focus on running Minikube with the KVM driver on Ubuntu 18.04 (I am using a bare metal machine running on [packet.com](https://www.packet.com).)
|
||||
|
||||
<center>{{<figure width="600" src="/images/blog/2019-03-28-running-kubernetes-locally-on-linux-with-minikube/module_01_cluster.png" caption="Minikube architecture (source: kubernetes.io)">}}</center>
|
||||
|
||||
## Disclaimer
|
||||
|
||||
This is not an official guide to Minikube. You may find detailed information on running and using Minikube on it's official [webpage](https://github.com/kubernetes/minikube), where different use cases, operating systems, environments, etc. are covered. Instead, the purpose of this guide is to provide clear and easy guidelines for running Minikube with KVM on Linux.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Any Linux you like (in this tutorial we'll use Ubuntu 18.04 LTS, and all the instructions below are applicable to it. If you prefer using a different Linux distribution, please check out the relevant documentation)
|
||||
- `libvirt` and QEMU-KVM installed and properly configured
|
||||
- The Kubernetes CLI (`kubectl`) for operating the Kubernetes cluster
|
||||
|
||||
### QEMU/KVM and libvirt installation
|
||||
|
||||
*NOTE: skip if already installed*
|
||||
|
||||
Before we proceed, we have to verify if our host can run KVM-based virtual machines. This can be easily checked using the [kvm-ok](https://manpages.ubuntu.com/manpages/bionic/man1/kvm-ok.1.html) tool, available on Ubuntu.
|
||||
|
||||
```shell
|
||||
sudo apt install cpu-checker && sudo kvm-ok
|
||||
```
|
||||
If you receive the following output after running `kvm-ok`, you can use KVM on your machine (otherwise, please check out your configuration):
|
||||
|
||||
```shell
|
||||
$ sudo kvm-ok
|
||||
INFO: /dev/kvm exists
|
||||
KVM acceleration can be used
|
||||
```
|
||||
|
||||
Now let's install KVM and libvirt and add our current user to the `libvirt` group to grant sufficient permissions:
|
||||
|
||||
```shell
|
||||
sudo apt install libvirt-clients libvirt-daemon-system qemu-kvm \
|
||||
&& sudo usermod -a -G libvirt $(whoami) \
|
||||
&& newgrp libvirt
|
||||
```
|
||||
|
||||
After installing libvirt, you may verify the host validity to run the virtual machines with `virt-host-validate` tool, which is a part of libvirt.
|
||||
|
||||
```shell
|
||||
sudo virt-host-validate
|
||||
```
|
||||
|
||||
### kubectl (Kubernetes CLI) installation
|
||||
|
||||
*NOTE: skip if already installed*
|
||||
|
||||
In order to manage the Kubernetes cluster, we need to install [kubectl](https://kubernetes.io/docs/reference/kubectl/overview/), the Kubernetes CLI tool.
|
||||
|
||||
The recommended way to install it on Linux is to download the pre-built binary and move it to a directory under the `$PATH`.
|
||||
|
||||
```shell
|
||||
curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl \
|
||||
&& sudo install kubectl /usr/local/bin && rm kubectl
|
||||
```
|
||||
|
||||
Alternatively, kubectl can be installed with a big variety of different methods (eg. as a .deb or snap package - check out the [kubectl documentation](https://kubernetes.io/docs/tasks/tools/install-kubectl/) to find the best one for you).
|
||||
|
||||
## Minikube installation
|
||||
|
||||
|
||||
### Minikube KVM driver installation
|
||||
|
||||
A VM driver is an essential requirement for local deployment of Minikube. As we've chosen to use KVM as the Minikube driver in this tutorial, let's install the KVM driver with the following command:
|
||||
|
||||
```shell
|
||||
curl -LO https://storage.googleapis.com/minikube/releases/latest/docker-machine-driver-kvm2 \
|
||||
&& sudo install docker-machine-driver-kvm2 /usr/local/bin/ && rm docker-machine-driver-kvm2
|
||||
```
|
||||
|
||||
### Minikube installation
|
||||
|
||||
Now let's install Minikube itself:
|
||||
|
||||
```shell
|
||||
curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64 \
|
||||
&& sudo install minikube-linux-amd64 /usr/local/bin/minikube && rm minikube-linux-amd64
|
||||
```
|
||||
|
||||
### Verify the Minikube installation
|
||||
|
||||
Before we proceed, we need to verify that Minikube is correctly installed. The simplest way to do this is to check Minikube’s status.
|
||||
|
||||
```shell
|
||||
minikube version
|
||||
```
|
||||
|
||||
### To use the KVM2 driver:
|
||||
|
||||
Now let's run the local Kubernetes cluster with Minikube and KVM:
|
||||
|
||||
```shell
|
||||
minikube start --vm-driver kvm2
|
||||
```
|
||||
|
||||
### Set KVM2 as a default VM driver for Minikube
|
||||
|
||||
If KVM is used as the single driver for Minikube on our machine, it's more convenient to set it as a default driver and run Minikube with fewer command-line arguments. The following command sets the KVM driver as the default:
|
||||
|
||||
```shell
|
||||
minikube config set vm-driver kvm2
|
||||
```
|
||||
|
||||
So now let's run Minikube as usual:
|
||||
|
||||
```shell
|
||||
minikube start
|
||||
```
|
||||
|
||||
## Verify the Kubernetes installation
|
||||
|
||||
Let's check if the Kubernetes cluster is up and running:
|
||||
|
||||
```shell
|
||||
kubectl get nodes
|
||||
```
|
||||
|
||||
Now let's run a simple sample app (nginx in our case):
|
||||
|
||||
```shell
|
||||
kubectl create deployment nginx --image=nginx
|
||||
```
|
||||
|
||||
Let’s also check that the Kubernetes pods are correctly provisioned:
|
||||
|
||||
```shell
|
||||
kubectl get pods
|
||||
```
|
||||
|
||||
## Screencast
|
||||
|
||||
<center>[](https://asciinema.org/a/237106)</center>
|
||||
|
||||
## Next steps
|
||||
|
||||
At this point, a Kubernetes cluster with Minikube and KVM is adequately set up and configured on your local machine.
|
||||
|
||||
To proceed, you may check out the Kubernetes tutorials on the project website:
|
||||
|
||||
- [Hello Minikube](https://kubernetes.io/docs/tutorials/hello-minikube/)
|
||||
|
||||
It’s also worth checking out the "Introduction to Kubernetes" course by The Linux Foundation/Cloud Native Computing Foundation, available for free on EDX:
|
||||
|
||||
- [Introduction to Kubernetes](https://www.edx.org/course/introduction-to-kubernetes#)
|
||||
+193
@@ -0,0 +1,193 @@
|
||||
---
|
||||
title: 'kube-proxy Subtleties: Debugging an Intermittent Connection Reset'
|
||||
date: 2019-03-29
|
||||
---
|
||||
|
||||
**Author:** [Yongkun Gui](ygui@google.com), Google
|
||||
|
||||
I recently came across a bug that causes intermittent connection resets. After
|
||||
some digging, I found it was caused by a subtle combination of several different
|
||||
network subsystems. It helped me understand Kubernetes networking better, and I
|
||||
think it’s worthwhile to share with a wider audience who are interested in the same
|
||||
topic.
|
||||
|
||||
## The symptom
|
||||
|
||||
We received a user report claiming they were getting connection resets while using a
|
||||
Kubernetes service of type ClusterIP to serve large files to pods running in the
|
||||
same cluster. Initial debugging of the cluster did not yield anything
|
||||
interesting: network connectivity was fine and downloading the files did not hit
|
||||
any issues. However, when we ran the workload in parallel across many clients,
|
||||
we were able to reproduce the problem. Adding to the mystery was the fact that
|
||||
the problem could not be reproduced when the workload was run using VMs without
|
||||
Kubernetes. The problem, which could be easily reproduced by [a simple
|
||||
app](https://github.com/tcarmet/k8s-connection-reset), clearly has something to
|
||||
do with Kubernetes networking, but what?
|
||||
|
||||
## Kubernetes networking basics
|
||||
|
||||
Before digging into this problem, let’s talk a little bit about some basics of
|
||||
Kubernetes networking, as Kubernetes handles network traffic from a pod
|
||||
very differently depending on different destinations.
|
||||
|
||||
### Pod-to-Pod
|
||||
|
||||
In Kubernetes, every pod has its own IP address. The benefit is that the
|
||||
applications running inside pods could use their canonical port, instead of
|
||||
remapping to a different random port. Pods have L3 connectivity between each
|
||||
other. They can ping each other, and send TCP or UDP packets to each other.
|
||||
[CNI](https://github.com/containernetworking/cni) is the standard that solves
|
||||
this problem for containers running on different hosts. There are tons of
|
||||
different plugins that support CNI.
|
||||
|
||||
### Pod-to-external
|
||||
|
||||
For the traffic that goes from pod to external addresses, Kubernetes simply uses
|
||||
[SNAT](https://en.wikipedia.org/wiki/Network_address_translation). What it does
|
||||
is replace the pod’s internal source IP:port with the host’s IP:port. When
|
||||
the return packet comes back to the host, it rewrites the pod’s IP:port as the
|
||||
destination and sends it back to the original pod. The whole process is transparent
|
||||
to the original pod, who doesn’t know the address translation at all.
|
||||
|
||||
### Pod-to-Service
|
||||
|
||||
Pods are mortal. Most likely, people want reliable service. Otherwise, it’s
|
||||
pretty much useless. So Kubernetes has this concept called "service" which is
|
||||
simply a L4 load balancer in front of pods. There are several different types of
|
||||
services. The most basic type is called ClusterIP. For this type of service, it
|
||||
has a unique VIP address that is only routable inside the cluster.
|
||||
|
||||
The component in Kubernetes that implements this feature is called kube-proxy.
|
||||
It sits on every node, and programs complicated iptables rules to do all kinds
|
||||
of filtering and NAT between pods and services. If you go to a Kubernetes node
|
||||
and type `iptables-save`, you’ll see the rules that are inserted by Kubernetes
|
||||
or other programs. The most important chains are `KUBE-SERVICES`, `KUBE-SVC-*`
|
||||
and `KUBE-SEP-*`.
|
||||
|
||||
- `KUBE-SERVICES` is the entry point for service packets. What it does is to
|
||||
match the destination IP:port and dispatch the packet to the corresponding
|
||||
`KUBE-SVC-*` chain.
|
||||
- `KUBE-SVC-*` chain acts as a load balancer, and distributes the packet to
|
||||
`KUBE-SEP-*` chain equally. Every `KUBE-SVC-*` has the same number of
|
||||
`KUBE-SEP-*` chains as the number of endpoints behind it.
|
||||
- `KUBE-SEP-*` chain represents a Service EndPoint. It simply does DNAT,
|
||||
replacing service IP:port with pod's endpoint IP:Port.
|
||||
|
||||
For DNAT, conntrack kicks in and tracks the connection state using a state
|
||||
machine. The state is needed because it needs to remember the destination
|
||||
address it changed to, and changed it back when the returning packet came back.
|
||||
Iptables could also rely on the conntrack state (ctstate) to decide the destiny
|
||||
of a packet. Those 4 conntrack states are especially important:
|
||||
|
||||
- *NEW*: conntrack knows nothing about this packet, which happens when the SYN
|
||||
packet is received.
|
||||
- *ESTABLISHED*: conntrack knows the packet belongs to an established connection,
|
||||
which happens after handshake is complete.
|
||||
- *RELATED*: The packet doesn’t belong to any connection, but it is affiliated
|
||||
to another connection, which is especially useful for protocols like FTP.
|
||||
- *INVALID*: Something is wrong with the packet, and conntrack doesn’t know how
|
||||
to deal with it. This state plays a centric role in this Kubernetes issue.
|
||||
|
||||
Here is a diagram of how a TCP connection works between pod and service. The
|
||||
sequence of events are:
|
||||
|
||||
- Client pod from left hand side sends a packet to a
|
||||
service: 192.168.0.2:80
|
||||
- The packet is going through iptables rules in client
|
||||
node and the destination is changed to pod IP, 10.0.1.2:80
|
||||
- Server pod handles the packet and sends back a packet with destination 10.0.0.2
|
||||
- The packet is going back to the client node, conntrack recognizes the packet and rewrites the source
|
||||
address back to 192.169.0.2:80
|
||||
- Client pod receives the response packet
|
||||
|
||||
{{<figure width="100%"
|
||||
src="/images/blog/2019-03-26-kube-proxy-subtleties-debugging-an-intermittent-connection-resets/good-packet-flow.png"
|
||||
caption="Good packet flow">}}
|
||||
|
||||
## What caused the connection reset?
|
||||
|
||||
Enough of the background, so what really went wrong and caused the unexpected
|
||||
connection reset?
|
||||
|
||||
As the diagram below shows, the problem is packet 3. When conntrack cannot
|
||||
recognize a returning packet, and mark it as *INVALID*. The most common
|
||||
reasons include: conntrack cannot keep track of a connection because it is out
|
||||
of capacity, the packet itself is out of a TCP window, etc. For those packets
|
||||
that have been marked as *INVALID* state by conntrack, we don’t have the
|
||||
iptables rule to drop it, so it will be forwarded to client pod, with source IP
|
||||
address not rewritten (as shown in packet 4)! Client pod doesn’t recognize this
|
||||
packet because it has a different source IP, which is pod IP, not service IP. As
|
||||
a result, client pod says, "Wait a second, I don't recall this connection to
|
||||
this IP ever existed, why does this dude keep sending this packet to me?" Basically,
|
||||
what the client does is simply send a RST packet to the server pod IP, which
|
||||
is packet 5. Unfortunately, this is a totally legit pod-to-pod packet, which can
|
||||
be delivered to server pod. Server pod doesn’t know all the address translations
|
||||
that happened on the client side. From its view, packet 5 is a totally legit
|
||||
packet, like packet 2 and 3. All server pod knows is, "Well, client pod doesn’t
|
||||
want to talk to me, so let’s close the connection!" Boom! Of course, in order
|
||||
for all these to happen, the RST packet has to be legit too, with the right TCP
|
||||
sequence number, etc. But when it happens, both parties agree to close the
|
||||
connection.
|
||||
|
||||
{{<figure width="100%"
|
||||
src="/images/blog/2019-03-26-kube-proxy-subtleties-debugging-an-intermittent-connection-resets/connection-reset-packet-flow.png"
|
||||
caption="Connection reset packet flow">}}
|
||||
|
||||
## How to address it?
|
||||
|
||||
Once we understand the root cause, the fix is not hard. There are at least 2
|
||||
ways to address it.
|
||||
|
||||
- Make conntrack more liberal on packets, and don’t mark the packets as
|
||||
*INVALID*. In Linux, you can do this by `echo 1 >
|
||||
/proc/sys/net/ipv4/netfilter/ip_conntrack_tcp_be_liberal`.
|
||||
- Specifically add an iptables rule to drop the packets that are marked as
|
||||
*INVALID*, so it won’t reach to client pod and cause harm.
|
||||
|
||||
The fix is drafted (https://github.com/kubernetes/kubernetes/pull/74840), but
|
||||
unfortunately it didn’t catch the v1.14 release window. However, for the users
|
||||
that are affected by this bug, there is a way to mitigate the problem by applying
|
||||
the following rule in your cluster.
|
||||
|
||||
```yaml
|
||||
apiVersion: extensions/v1beta1
|
||||
kind: DaemonSet
|
||||
metadata:
|
||||
name: startup-script
|
||||
labels:
|
||||
app: startup-script
|
||||
spec:
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: startup-script
|
||||
spec:
|
||||
hostPID: true
|
||||
containers:
|
||||
- name: startup-script
|
||||
image: gcr.io/google-containers/startup-script:v1
|
||||
imagePullPolicy: IfNotPresent
|
||||
securityContext:
|
||||
privileged: true
|
||||
env:
|
||||
- name: STARTUP_SCRIPT
|
||||
value: |
|
||||
#! /bin/bash
|
||||
echo 1 > /proc/sys/net/ipv4/netfilter/ip_conntrack_tcp_be_liberal
|
||||
echo done
|
||||
```
|
||||
|
||||
## Summary
|
||||
|
||||
Obviously, the bug has existed almost forever. I am surprised that it
|
||||
hasn’t been noticed until recently. I believe the reasons could be: (1) this
|
||||
happens more in a congested server serving large payloads, which might not be a
|
||||
common use case; (2) the application layer handles the retry to be tolerant of
|
||||
this kind of reset. Anyways, regardless of how fast Kubernetes has been growing,
|
||||
it’s still a young project. There are no other secrets than listening closely to
|
||||
customers’ feedback, not taking anything for granted but digging deep, we can
|
||||
make it the best platform to run applications.
|
||||
|
||||
Special thanks to [bowei](https://github.com/bowei) for the consulting for both
|
||||
debugging process and the blog, to [tcarmet](https://github.com/tcarmet) for
|
||||
reporting the issue and providing a reproduction.
|
||||
+81
@@ -0,0 +1,81 @@
|
||||
---
|
||||
title: 'Kubernetes v1.14 delivers production-level support for Windows nodes and Windows containers'
|
||||
date: 2019-04-01
|
||||
---
|
||||
|
||||
**Authors:** Michael Michael (VMware), Patrick Lang (Microsoft)
|
||||
|
||||
The first release of Kubernetes in 2019 brings a highly anticipated feature - production-level support for Windows workloads. Up until now Windows node support in Kubernetes has been in beta, allowing many users to experiment and see the value of Kubernetes for Windows containers. While in beta, developers in the Kubernetes community and Windows Server team worked together to improve the container runtime, build a continuous testing process, and complete features needed for a good user experience. Kubernetes now officially supports adding Windows nodes as worker nodes and scheduling Windows containers, enabling a vast ecosystem of Windows applications to leverage the power of our platform.
|
||||
|
||||
As Windows developers and devops engineers have been adopting containers over the last few years, they've been looking for a way to manage all their workloads with a common interface. Kubernetes has taken the lead for container orchestration, and this gives users a consistent way to manage their container workloads whether they need to run on Linux or Windows.
|
||||
|
||||
The journey to a stable release of Windows in Kubernetes was not a walk in the park. The community has been working on Windows support for 3 years, delivering an alpha release with v1.5, a beta with v1.9, and now a stable release with v1.14. We would not be here today without rallying broad support and getting significant contributions from companies including Microsoft, Docker, VMware, Pivotal, Cloudbase Solutions, Google and Apprenda. During this journey, there were 3 critical points in time that significantly advanced our progress.
|
||||
|
||||
1. Advancements in Windows Server container networking that provided the infrastructure to create CNI (Container Network Interface) plugins
|
||||
1. Enhancements shipped in Windows Server semi-annual channel releases enabled Kubernetes development to move forward - culminating with Windows Server 2019 on the Long-Term Servicing Channel. This is the best release of Windows Server for running containers.
|
||||
1. The adoption of the KEP (Kubernetes Enhancement Proposals) [process](https://github.com/kubernetes/enhancements/blob/master/keps/README.md). The Windows KEP outlined a clear and agreed upon set of goals, expectations, and deliverables based on review and feedback from stakeholders across multiple SIGs. This created a clear plan that SIG-Windows could follow, paving the path towards this stable release.
|
||||
|
||||
With v1.14, we're declaring that Windows node support is stable, well-tested, and ready for adoption in production scenarios. This is a huge milestone for many reasons. For Kubernetes, it strengthens its position in the industry, enabling a vast ecosystem of Windows-based applications to be deployed on the platform. For Windows operators and developers, this means they can use the same tools and processes to manage their Windows and Linux workloads, taking full advantage of the efficiencies of the cloud-native ecosystem powered by Kubernetes. Let’s dig in a little bit into these.
|
||||
|
||||
## Operator Advantages
|
||||
|
||||
- Gain operational efficiencies by leveraging existing investments in solutions, tools, and technologies to manage Windows containers the same way as Linux containers
|
||||
- Knowledge, training and expertise on container orchestration transfers to Windows container support
|
||||
- IT can deliver a scalable self-service container platform to Linux and Windows developers
|
||||
|
||||
## Developer Advantages
|
||||
|
||||
- Containers simplify packaging and deploying applications during development and test. Now you also get to take advantage of Kubernetes’ benefits in creating reliable, secure, and scalable distributed applications.
|
||||
- Windows developers can now take advantage of the growing ecosystem of cloud and container-native tools to build and deploy faster, resulting in a faster time to market for their applications
|
||||
- Taking advantage of Kubernetes as the leader in container orchestration, developers only need to learn how to use Kubernetes and that skillset will transfer across development environments and across clouds
|
||||
|
||||
## CIO Advantages
|
||||
|
||||
- Leverage the operational and cost efficiencies that are introduced with Kubernetes
|
||||
- Containerize existing.NET applications or Windows-based workloads to eliminate old hardware or underutilized virtual machines, and streamline migration from end-of-support OS versions. You retain the benefit your application brings to the business, but decrease the cost of keeping it running
|
||||
|
||||
“Using Kubernetes on Windows allows us to run our internal web applications as microservices. This provides quick scaling in response to load, smoother upgrades, and allows for different development groups to build without worry of other group's version dependencies. We save money because development times are shorter and operation's time is not spent maintaining multiple virtual machine environments,” said Jeremy, a lead devops engineer working for a top multinational legal firm, one of the early adopters of Windows on Kubernetes.
|
||||
|
||||
There are many features that are surfaced with this release. We want to turn your attention to a few key features and enablers of Windows support in Kubernetes. For a detailed list of supported functionality, you can read our [documentation](https://kubernetes.io/docs/setup/windows/intro-windows-in-kubernetes/#supported-functionality).
|
||||
|
||||
- You can now add Windows Server 2019 worker nodes
|
||||
- You can now schedule Windows containers utilizing deployments, pods, services, and workload controllers
|
||||
- Out of tree CNI plugins are provided for Azure, OVN-Kubernetes, and Flannel
|
||||
- Containers can utilize a variety of in and out-of-tree storage plugins
|
||||
- Improved support for metrics/quotas closely matches the capabilities offered for Linux containers
|
||||
|
||||
When looking at Windows support in Kubernetes, many start drawing comparisons to Linux containers. Although some of the comparisons that highlight limitations are fair, it is important to distinguish between **operational limitations and differences between the Windows and Linux operating systems**. From a container management standpoint, we must strike a balance between preserving OS-specific behaviors required for application compatibility, and reaching operational consistency in Kubernetes across multiple operating systems. For example, some Linux-specific file system features, user IDs and permissions exposed through Kubernetes will not work on Windows today, and users are familiar with these fundamental differences. We will also be adding support for Windows-specific configurations to meet the needs of Windows customers that may not exist on Linux. The alpha support for Windows Group Managed Service Accounts is one example. Other areas such as memory reservations for Windows pods and the Windows kubelet are a work in progress and highlight an operational limitation. We will continue working on operational limitations based on what’s important to our community in future releases.
|
||||
|
||||
Today, Kubernetes master components will continue to run on Linux. That way users can add Windows nodes without having to create a separate Kubernetes cluster. As always, our future direction is set by the community, so more components, features and deployment methods will come over time. Users should understand the differences between Windows and Linux and utilize the advantages of each platform. Our goal with this release is not to make Windows interchangeable with Linux or to answer the question of Windows vs Linux. We offer consistency in management. Managing workloads without automation is tedious and expensive. Rewriting or re-architecting workloads is even more expensive. Containers provide a clear path forward whether your app runs on Linux or Windows, and Kubernetes brings an IT organization operational consistency.
|
||||
|
||||
As a community, our work is not complete. As already mentioned , we still have a fair bit of [limitations](https://kubernetes.io/docs/setup/windows/intro-windows-in-kubernetes/#limitations) and a healthy [roadmap](https://kubernetes.io/docs/setup/windows/intro-windows-in-kubernetes/#what-s-next). We will continue making progress and enhancing Windows container support in Kubernetes, with some notable upcoming features including:
|
||||
|
||||
- Support for CRI-ContainerD and Hyper-V isolation, bringing hypervisor-level isolation between pods for additional security and extending our container-to-node compatibility matrix
|
||||
- Additional network plugins, including the stable release of Flannel overlay support
|
||||
- Simple heterogeneous cluster creation using kubeadm on Windows
|
||||
|
||||
We welcome you to get involved and join our community to share feedback and deployment stories, and contribute to code, docs, and improvements of any kind.
|
||||
|
||||
- Read our getting started and contributor guides, which include links to the community meetings and past recordings, at https://github.com/kubernetes/community/tree/master/sig-windows
|
||||
- Explore our documentation at https://kubernetes.io/docs/setup/windows
|
||||
- Join us on [Slack](https://kubernetes.slack.com/messages/sig-windows) or the [Kubernetes Community Forums](https://discuss.kubernetes.io/c/general-discussions/windows) to chat about Windows containers on Kubernetes.
|
||||
|
||||
Thank you and feel free to reach us individually if you have any questions.
|
||||
|
||||
Michael Michael
|
||||
<br>
|
||||
SIG-Windows Chair
|
||||
<br>
|
||||
Director of Product Management, VMware
|
||||
<br>
|
||||
@michmike77 on Twitter
|
||||
<br>
|
||||
@m2 on Slack
|
||||
|
||||
Patrick Lang
|
||||
<br>
|
||||
SIG-Windows Chair
|
||||
<br>
|
||||
Senior Software Engineer, Microsoft
|
||||
<br>
|
||||
@PatrickLang on Slack
|
||||
@@ -0,0 +1,336 @@
|
||||
---
|
||||
layout: blog
|
||||
title: 'Kubernetes 1.14: Local Persistent Volumes GA'
|
||||
date: 2019-04-04
|
||||
---
|
||||
|
||||
**Authors**: Michelle Au (Google), Matt Schallert (Uber), Celina Ward (Uber)
|
||||
|
||||
The [Local Persistent Volumes](https://kubernetes.io/docs/concepts/storage/volumes/#local)
|
||||
feature has been promoted to GA in Kubernetes 1.14.
|
||||
It was first introduced as alpha in Kubernetes 1.7, and then
|
||||
[beta](https://kubernetes.io/blog/2018/04/13/local-persistent-volumes-beta/) in Kubernetes
|
||||
1.10. The GA milestone indicates that Kubernetes users may depend on the feature
|
||||
and its API for production use. GA features are protected by the Kubernetes
|
||||
[deprecation
|
||||
policy](https://kubernetes.io/docs/reference/using-api/deprecation-policy/).
|
||||
|
||||
## What is a Local Persistent Volume?
|
||||
|
||||
A local persistent volume represents a local disk directly-attached to a single
|
||||
Kubernetes Node.
|
||||
|
||||
Kubernetes provides a powerful volume plugin system that enables Kubernetes
|
||||
workloads to use a [wide
|
||||
variety](https://kubernetes.io/docs/concepts/storage/volumes/#types-of-volumes)
|
||||
of block and file storage to persist data. Most
|
||||
of these plugins enable remote storage -- these remote storage systems persist
|
||||
data independent of the Kubernetes node where the data originated. Remote
|
||||
storage usually can not offer the consistent high performance guarantees of
|
||||
local directly-attached storage. With the Local Persistent Volume plugin,
|
||||
Kubernetes workloads can now consume high performance local storage using the
|
||||
same volume APIs that app developers have become accustomed to.
|
||||
|
||||
## How is it different from a HostPath Volume?
|
||||
|
||||
To better understand the benefits of a Local Persistent Volume, it is useful to
|
||||
compare it to a [HostPath volume](https://kubernetes.io/docs/concepts/storage/volumes/#hostpath).
|
||||
HostPath volumes mount a file or directory from
|
||||
the host node’s filesystem into a Pod. Similarly a Local Persistent Volume
|
||||
mounts a local disk or partition into a Pod.
|
||||
|
||||
The biggest difference is that the Kubernetes scheduler understands which node a
|
||||
Local Persistent Volume belongs to. With HostPath volumes, a pod referencing a
|
||||
HostPath volume may be moved by the scheduler to a different node resulting in
|
||||
data loss. But with Local Persistent Volumes, the Kubernetes scheduler ensures
|
||||
that a pod using a Local Persistent Volume is always scheduled to the same node.
|
||||
|
||||
While HostPath volumes may be referenced via a Persistent Volume Claim (PVC) or
|
||||
directly inline in a pod definition, Local Persistent Volumes can only be
|
||||
referenced via a PVC. This provides additional security benefits since
|
||||
Persistent Volume objects are managed by the administrator, preventing Pods from
|
||||
being able to access any path on the host.
|
||||
|
||||
Additional benefits include support for formatting of block devices during
|
||||
mount, and volume ownership using fsGroup.
|
||||
|
||||
## What's New With GA?
|
||||
|
||||
Since 1.10, we have mainly focused on improving stability and scalability of the
|
||||
feature so that it is production ready.
|
||||
|
||||
The only major feature addition is the ability to specify a raw block device and
|
||||
have Kubernetes automatically format and mount the filesystem. This reduces the
|
||||
previous burden of having to format and mount devices before giving it to
|
||||
Kubernetes.
|
||||
|
||||
## Limitations of GA
|
||||
|
||||
At GA, Local Persistent Volumes do not support [dynamic volume
|
||||
provisioning](https://kubernetes.io/docs/concepts/storage/dynamic-provisioning/).
|
||||
However there is an [external
|
||||
controller](https://github.com/kubernetes-sigs/sig-storage-local-static-provisioner)
|
||||
available to help manage the local
|
||||
PersistentVolume lifecycle for individual disks on your nodes. This includes
|
||||
creating the PersistentVolume objects, cleaning up and reusing disks once they
|
||||
have been released by the application.
|
||||
|
||||
## How to Use a Local Persistent Volume?
|
||||
|
||||
Workloads can request a local persistent volume using the same
|
||||
PersistentVolumeClaim interface as remote storage backends. This makes it easy
|
||||
to swap out the storage backend across clusters, clouds, and on-prem
|
||||
environments.
|
||||
|
||||
First, a StorageClass should be created that sets `volumeBindingMode:
|
||||
WaitForFirstConsumer` to enable [volume topology-aware
|
||||
scheduling](https://kubernetes.io/docs/concepts/storage/storage-classes/#volume-binding-mode).
|
||||
This mode instructs Kubernetes to wait to bind a PVC until a Pod using it is scheduled.
|
||||
|
||||
```
|
||||
kind: StorageClass
|
||||
apiVersion: storage.k8s.io/v1
|
||||
metadata:
|
||||
name: local-storage
|
||||
provisioner: kubernetes.io/no-provisioner
|
||||
volumeBindingMode: WaitForFirstConsumer
|
||||
```
|
||||
|
||||
Then, the external static provisioner can be [configured and
|
||||
run](https://github.com/kubernetes-sigs/sig-storage-local-static-provisioner#user-guide) to create PVs
|
||||
for all the local disks on your nodes.
|
||||
|
||||
```
|
||||
$ kubectl get pv
|
||||
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
|
||||
local-pv-27c0f084 368Gi RWO Delete Available local-storage 8s
|
||||
local-pv-3796b049 368Gi RWO Delete Available local-storage 7s
|
||||
local-pv-3ddecaea 368Gi RWO Delete Available local-storage 7s
|
||||
```
|
||||
|
||||
Afterwards, workloads can start using the PVs by creating a PVC and Pod or a
|
||||
StatefulSet with volumeClaimTemplates.
|
||||
|
||||
```
|
||||
apiVersion: apps/v1
|
||||
kind: StatefulSet
|
||||
metadata:
|
||||
name: local-test
|
||||
spec:
|
||||
serviceName: "local-service"
|
||||
replicas: 3
|
||||
selector:
|
||||
matchLabels:
|
||||
app: local-test
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: local-test
|
||||
spec:
|
||||
containers:
|
||||
- name: test-container
|
||||
image: k8s.gcr.io/busybox
|
||||
command:
|
||||
- "/bin/sh"
|
||||
args:
|
||||
- "-c"
|
||||
- "sleep 100000"
|
||||
volumeMounts:
|
||||
- name: local-vol
|
||||
mountPath: /usr/test-pod
|
||||
volumeClaimTemplates:
|
||||
- metadata:
|
||||
name: local-vol
|
||||
spec:
|
||||
accessModes: [ "ReadWriteOnce" ]
|
||||
storageClassName: "local-storage"
|
||||
resources:
|
||||
requests:
|
||||
storage: 368Gi
|
||||
```
|
||||
|
||||
Once the StatefulSet is up and running, the PVCs are all bound:
|
||||
|
||||
```
|
||||
$ kubectl get pvc
|
||||
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
|
||||
local-vol-local-test-0 Bound local-pv-27c0f084 368Gi RWO local-storage 3m45s
|
||||
local-vol-local-test-1 Bound local-pv-3ddecaea 368Gi RWO local-storage 3m40s
|
||||
local-vol-local-test-2 Bound local-pv-3796b049 368Gi RWO local-storage 3m36s
|
||||
```
|
||||
|
||||
When the disk is no longer needed, the PVC can be deleted. The external static provisioner
|
||||
will clean up the disk and make the PV available for use again.
|
||||
|
||||
```
|
||||
$ kubectl patch sts local-test -p '{"spec":{"replicas":2}}'
|
||||
statefulset.apps/local-test patched
|
||||
|
||||
$ kubectl delete pvc local-vol-local-test-2
|
||||
persistentvolumeclaim "local-vol-local-test-2" deleted
|
||||
|
||||
$ kubectl get pv
|
||||
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
|
||||
local-pv-27c0f084 368Gi RWO Delete Bound default/local-vol-local-test-0 local-storage 11m
|
||||
local-pv-3796b049 368Gi RWO Delete Available local-storage 7s
|
||||
local-pv-3ddecaea 368Gi RWO Delete Bound default/local-vol-local-test-1 local-storage 19m
|
||||
```
|
||||
|
||||
You can find full [documentation](https://kubernetes.io/docs/concepts/storage/volumes/#local)
|
||||
for the feature on the Kubernetes website.
|
||||
|
||||
## What Are Suitable Use Cases?
|
||||
|
||||
The primary benefit of Local Persistent Volumes over remote persistent storage
|
||||
is performance: local disks usually offer higher IOPS and throughput and lower
|
||||
latency compared to remote storage systems.
|
||||
|
||||
However, there are important limitations and caveats to consider when using
|
||||
Local Persistent Volumes:
|
||||
|
||||
* Using local storage ties your application to a specific node, making your
|
||||
application harder to schedule. Applications which use local storage should
|
||||
specify a high priority so that lower priority pods, that don’t require local
|
||||
storage, can be preempted if necessary.
|
||||
* If that node or local volume encounters a failure and becomes inaccessible, then
|
||||
that pod also becomes inaccessible. Manual intervention, external controllers,
|
||||
or operators may be needed to recover from these situations.
|
||||
* While most remote storage systems implement synchronous replication, most local
|
||||
disk offerings do not provide data durability guarantees. Meaning loss of the
|
||||
disk or node may result in loss of all the data on that disk
|
||||
|
||||
For these reasons, local persistent storage should only be considered for
|
||||
workloads that handle data replication and backup at the application layer, thus
|
||||
making the applications resilient to node or data failures and unavailability
|
||||
despite the lack of such guarantees at the individual disk level.
|
||||
|
||||
Examples of good workloads include software defined storage systems and
|
||||
replicated databases. Other types of applications should continue to use highly
|
||||
available, remotely accessible, durable storage.
|
||||
|
||||
## How Uber Uses Local Storage
|
||||
|
||||
[M3](https://eng.uber.com/m3/), Uber’s in-house metrics platform,
|
||||
piloted Local Persistent Volumes at scale
|
||||
in an effort to evaluate [M3DB](https://m3db.io/) —
|
||||
an open-source, distributed timeseries database
|
||||
created by Uber. One of M3DB’s notable features is its ability to shard its
|
||||
metrics into partitions, replicate them by a factor of three, and then evenly
|
||||
disperse the replicas across separate failure domains.
|
||||
|
||||
Prior to the pilot with local persistent volumes, M3DB ran exclusively in
|
||||
Uber-managed environments. Over time, internal use cases arose that required the
|
||||
ability to run M3DB in environments with fewer dependencies. So the team began
|
||||
to explore options. As an open-source project, we wanted to provide the
|
||||
community with a way to run M3DB as easily as possible, with an open-source
|
||||
stack, while meeting M3DB’s requirements for high throughput, low-latency
|
||||
storage, and the ability to scale itself out.
|
||||
|
||||
The Kubernetes Local Persistent Volume interface, with its high-performance,
|
||||
low-latency guarantees, quickly emerged as the perfect abstraction to build on
|
||||
top of. With Local Persistent Volumes, individual M3DB instances can comfortably
|
||||
handle up to 600k writes per-second. This leaves plenty of headroom for spikes
|
||||
on clusters that typically process a few million metrics per-second.
|
||||
|
||||
Because M3DB also gracefully handles losing a single node or volume, the limited
|
||||
data durability guarantees of Local Persistent Volumes are not an issue. If a
|
||||
node fails, M3DB finds a suitable replacement and the new node begins streaming
|
||||
data from its two peers.
|
||||
|
||||
Thanks to the Kubernetes scheduler’s intelligent handling of volume topology,
|
||||
M3DB is able to programmatically evenly disperse its replicas across multiple
|
||||
local persistent volumes in all available cloud zones, or, in the case of
|
||||
on-prem clusters, across all available server racks.
|
||||
|
||||
## Uber's Operational Experience
|
||||
|
||||
As mentioned above, while Local Persistent Volumes provide many benefits, they
|
||||
also require careful planning and careful consideration of constraints before
|
||||
committing to them in production. When thinking about our local volume strategy
|
||||
for M3DB, there were a few things Uber had to consider.
|
||||
|
||||
For one, we had to take into account the hardware profiles of the nodes in our
|
||||
Kubernetes cluster. For example, how many local disks would each node cluster
|
||||
have? How would they be partitioned?
|
||||
|
||||
The local static provisioner provides
|
||||
[guidance](https://github.com/kubernetes-sigs/sig-storage-local-static-provisioner/blob/master/docs/best-practices.md)
|
||||
to help answer these questions. It’s best to be able to dedicate a full disk to each local volume
|
||||
(for IO isolation) and a full partition per-volume (for capacity isolation).
|
||||
This was easier in our cloud environments where we could mix and match local
|
||||
disks. However, if using local volumes on-prem, hardware constraints may be a
|
||||
limiting factor depending on the number of disks available and their
|
||||
characteristics.
|
||||
|
||||
When first testing local volumes, we wanted to have a thorough understanding of
|
||||
the effect
|
||||
[disruptions](https://kubernetes.io/docs/concepts/workloads/pods/disruptions/)
|
||||
(voluntary and involuntary) would have on pods using
|
||||
local storage, and so we began testing some failure scenarios. We found that
|
||||
when a local volume becomes unavailable while the node remains available (such
|
||||
as when performing maintenance on the disk), a pod using the local volume will
|
||||
be stuck in a ContainerCreating state until it can mount the volume. If a node
|
||||
becomes unavailable, for example if it is removed from the cluster or is
|
||||
[drained](https://kubernetes.io/docs/tasks/administer-cluster/safely-drain-node/),
|
||||
then pods using local volumes on that node are stuck in an Unknown or
|
||||
Pending state depending on whether or not the node was removed gracefully.
|
||||
|
||||
Recovering pods from these interim states means having to delete the PVC binding
|
||||
the pod to its local volume and then delete the pod in order for it to be
|
||||
rescheduled (or wait until the node and disk are available again). We took this
|
||||
into account when building our [operator](https://github.com/m3db/m3db-operator)
|
||||
for M3DB, which makes changes to the
|
||||
cluster topology when a pod is rescheduled such that the new one gracefully
|
||||
streams data from the remaining two peers. Eventually we plan to automate the
|
||||
deletion and rescheduling process entirely.
|
||||
|
||||
Alerts on pod states can help call attention to stuck local volumes, and
|
||||
workload-specific controllers or operators can remediate them automatically.
|
||||
Because of these constraints, it’s best to exclude nodes with local volumes from
|
||||
automatic upgrades or repairs, and in fact some cloud providers explicitly
|
||||
mention this as a best practice.
|
||||
|
||||
## Portability Between On-Prem and Cloud
|
||||
|
||||
Local Volumes played a big role in Uber’s decision to build orchestration for
|
||||
M3DB using Kubernetes, in part because it is a storage abstraction that works
|
||||
the same across on-prem and cloud environments. Remote storage solutions have
|
||||
different characteristics across cloud providers, and some users may prefer not
|
||||
to use networked storage at all in their own data centers. On the other hand,
|
||||
local disks are relatively ubiquitous and provide more predictable performance
|
||||
characteristics.
|
||||
|
||||
By orchestrating M3DB using local disks in the cloud, where it was easier to get
|
||||
up and running with Kubernetes, we gained confidence that we could still use our
|
||||
operator to run M3DB in our on-prem environment without any modifications. As we
|
||||
continue to work on how we’d run Kubernetes on-prem, having solved such an
|
||||
important pending question is a big relief.
|
||||
|
||||
## What's Next for Local Persistent Volumes?
|
||||
|
||||
As we’ve seen with Uber’s M3DB, local persistent volumes have successfully been
|
||||
used in production environments. As adoption of local persistent volumes
|
||||
continues to increase, SIG Storage continues to seek feedback for ways to
|
||||
improve the feature.
|
||||
|
||||
One of the most frequent asks has been for a controller that can help with
|
||||
recovery from failed nodes or disks, which is currently a manual process (or
|
||||
something that has to be built into an operator). SIG Storage is investigating
|
||||
creating a common controller that can be used by workloads with simple and
|
||||
similar recovery processes.
|
||||
|
||||
Another popular ask has been to support dynamic provisioning using lvm. This can
|
||||
simplify disk management, and improve disk utilization. SIG Storage is
|
||||
evaluating the performance tradeoffs for the viability of this feature.
|
||||
|
||||
## Getting Invovled
|
||||
|
||||
If you have feedback for this feature or are interested in getting involved with
|
||||
the design and development, join the [Kubernetes Storage
|
||||
Special-Interest-Group](https://github.com/kubernetes/community/blob/master/sig-storage/README.md)
|
||||
(SIG). We’re rapidly growing and always welcome new contributors.
|
||||
|
||||
Special thanks to all the contributors that helped bring this feature to GA,
|
||||
including Chuqiang Li (lichuqiang), Dhiraj Hedge (dhirajh), Ian Chakeres
|
||||
(ianchakeres), Jan Šafránek (jsafrane), Michelle Au (msau42), Saad Ali
|
||||
(saad-ali), Yecheng Fu (cofyc) and Yuquan Ren (nickrenren).
|
||||
@@ -0,0 +1,35 @@
|
||||
---
|
||||
layout: blog
|
||||
title: 'Pod Priority and Preemption in Kubernetes'
|
||||
date: 2019-04-16
|
||||
---
|
||||
|
||||
**Author**: Bobby Salamat
|
||||
|
||||
Kubernetes is well-known for running scalable workloads. It scales your workloads based on their resource usage. When a workload is scaled up, more instances of the application get created. When the application is critical for your product, you want to make sure that these new instances are scheduled even when your cluster is under resource pressure. One obvious solution to this problem is to over-provision your cluster resources to have some amount of slack resources available for scale-up situations. This approach often works, but costs more as you would have to pay for the resources that are idle most of the time.
|
||||
|
||||
[Pod priority and preemption](https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/) is a scheduler feature made generally available in Kubernetes 1.14 that allows you to achieve high levels of scheduling confidence for your critical workloads without overprovisioning your clusters. It also provides a way to improve resource utilization in your clusters without sacrificing the reliability of your essential workloads.
|
||||
|
||||
## Guaranteed scheduling with controlled cost
|
||||
|
||||
[Kubernetes Cluster Autoscaler](https://kubernetes.io/docs/tasks/administer-cluster/cluster-management/#cluster-autoscaling) is an excellent tool in the ecosystem which adds more nodes to your cluster when your applications need them. However, cluster autoscaler has some limitations and may not work for all users:
|
||||
|
||||
- It does not work in physical clusters.
|
||||
- Adding more nodes to the cluster costs more.
|
||||
- Adding nodes is not instantaneous and could take minutes before those nodes become available for scheduling.
|
||||
|
||||
An alternative is Pod Priority and Preemption. In this approach, you combine multiple workloads in a single cluster. For example, you may run your CI/CD pipeline, ML workloads, and your critical service in the same cluster. When multiple workloads run in the same cluster, the size of your cluster is larger than a cluster that you would use to run only your critical service. If you give your critical service the highest priority and your CI/CD and ML workloads lower priority, when your service needs more computing resources, the scheduler preempts (evicts) enough pods of your lower priority workloads, e.g., ML workload, to allow all your higher priority pods to schedule.
|
||||
|
||||
With pod priority and preemption you can set a maximum size for your cluster in the Autoscaler configuration to ensure your costs get controlled without sacrificing availability of your service. Moreover, preemption is much faster than adding new nodes to the cluster. Within seconds your high priority pods are scheduled, which is critical for latency sensitive services.
|
||||
|
||||
## Improve cluster resource utilization
|
||||
|
||||
Cluster operators who run critical services learn over time a rough estimate of the number of nodes that they need in their clusters to achieve high service availability. The estimate is usually conservative. Such estimates take bursts of traffic into account to find the number of required nodes. Cluster autoscaler can be configured never to reduce the size of the cluster below this level. The only problem is that such estimates are often conservative and cluster resources may remain underutilized most of the time. Pod priority and preemption allows you to improve resource utilization significantly by running a non-critical workload in the cluster.
|
||||
|
||||
The non-critical workload may have many more pods that can fit in the cluster. If you give a negative priority to your non-critical workload, Cluster Autoscaler does not add more nodes to your cluster when the non-critical pods are pending. Therefore, you won’t incur higher expenses. When your critical workload requires more computing resources, the scheduler preempts non-critical pods and schedules critical ones.
|
||||
|
||||
The non-critical pods fill the “holes” in your cluster resources which improves resource utilization without raising your costs.
|
||||
|
||||
## Get Involved
|
||||
|
||||
If you have feedback for this feature or are interested in getting involved with the design and development, join the [Scheduling Special Interest Group](https://github.com/kubernetes/community/tree/master/sig-scheduling).
|
||||
+174
@@ -0,0 +1,174 @@
|
||||
---
|
||||
layout: blog
|
||||
title: 'Hardware Accelerated SSL/TLS Termination in Ingress Controllers using Kubernetes Device Plugins and RuntimeClass'
|
||||
date: 2019-04-24
|
||||
---
|
||||
|
||||
**Authors:** Mikko Ylinen (Intel)
|
||||
|
||||
## Abstract
|
||||
|
||||
A Kubernetes Ingress is a way to connect cluster services to the world outside the cluster. In order
|
||||
to correctly route the traffic to service backends, the cluster needs an Ingress controller. The
|
||||
Ingress controller is responsible for setting the right destinations to backends based on the
|
||||
Ingress API objects’ information. The actual traffic is routed through a proxy server that
|
||||
is responsible for tasks such as load balancing and SSL/TLS (later “SSL” refers to both SSL
|
||||
or TLS ) termination. The SSL termination is a CPU heavy operation due to the crypto operations
|
||||
involved. To offload some of the CPU intensive work away from the CPU, OpenSSL based proxy
|
||||
servers can take the benefit of OpenSSL Engine API and dedicated crypto hardware. This frees
|
||||
CPU cycles for other things and improves the overall throughput of the proxy server.
|
||||
|
||||
In this blog post, we will show how easy it is to make hardware accelerated crypto available
|
||||
for containers running the Ingress controller proxy using some of the recently created Kubernetes
|
||||
building blocks: Device plugin framework and RuntimeClass. At the end, a reference setup is given
|
||||
using an HAproxy based Ingress controller accelerated using Intel® QuickAssist Technology cards.
|
||||
|
||||
## About Proxies, OpenSSL Engine and Crypto Hardware
|
||||
|
||||
The proxy server plays a vital role in a Kubernetes Ingress Controller function. It proxies
|
||||
the traffic to the backends per Ingress objects routes. Under heavy traffic load, the performance
|
||||
becomes critical especially if the proxying involves CPU intensive operations like SSL crypto.
|
||||
|
||||
The OpenSSL project provides the widely adopted library for implementing the SSL protocol. Of
|
||||
the commonly known proxy servers used by Kubernetes Ingress controllers, Nginx and HAproxy use
|
||||
OpenSSL. The CNCF graduated Envoy proxy uses BoringSSL but there seems to be [community interest
|
||||
in having OpenSSL as the alternative](https://github.com/envoyproxy/envoy/pull/5161#issuecomment-446374130) for it too.
|
||||
|
||||
The OpenSSL SSL protocol library relies on libcrypto that implements the cryptographic functions.
|
||||
For quite some time now (first introduced in 0.9.6 release), OpenSSL has provided an [ENGINE
|
||||
concept](https://github.com/openssl/openssl/blob/master/README.ENGINE) that allows these cryptographic operations to be offloaded to a dedicated crypto
|
||||
acceleration hardware. Later, a special *dynamic* ENGINE enabled the crypto hardware specific
|
||||
pieces to be implemented in an independent loadable module that can be developed outside the
|
||||
OpenSSL code base and distributed separately. From the application’s perspective, this is also
|
||||
ideal because they don’t need to know the details of how to use the hardware, and the hardware
|
||||
specific module can be loaded/used when the hardware is available.
|
||||
|
||||
Hardware based crypto can greatly improve Cloud applications’ performance due to hardware
|
||||
accelerated processing in SSL operations as discussed, and can provide other crypto
|
||||
services like key/random number generation. Clouds can make the hardware easily available
|
||||
using the dynamic ENGINE and several loadable module implementations exist, for
|
||||
example, [CloudHSM](https://docs.aws.amazon.com/cloudhsm/latest/userguide/openssl-library.html), [IBMCA](https://github.com/opencryptoki/openssl-ibmca), or [QAT Engine](https://github.com/intel/QAT_Engine/).
|
||||
|
||||
For Cloud deployments, the ideal scenario is for these modules to be shipped as part of
|
||||
the container workload. The workload would get scheduled on a node that provides the
|
||||
underlying hardware that the module needs to access. On the other hand, the workloads
|
||||
should run the same way and without code modifications regardless of the crypto acceleration
|
||||
hardware being available or not. The OpenSSL dynamic engine enables this. Figure 1 below
|
||||
illustrates these two scenarios using a typical Ingress Controller container as an example.
|
||||
The red colored boxes indicate the differences between a container with a crypto hardware
|
||||
engine enabled container vs. a “standard” one. It’s worth pointing out that the configuration
|
||||
changes shown do not necessarily require another version of the container since the configurations
|
||||
could be managed, e.g., using ConfigMaps.
|
||||
|
||||
{{<figure width="600" src="/images/blog/2019-04-23-hardware-accelerated-tls-termination/k8s-blog-fig1.png" caption="Figure 1. Examples of Ingress controller containers">}}
|
||||
|
||||
## Hardware Resources and Isolation
|
||||
|
||||
To be able to deploy workloads with hardware dependencies, Kubernetes provides excellent extension
|
||||
and configurability mechanisms. Let’s take a closer look into Kubernetes the [device plugin framework](https://kubernetes.io/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins/)
|
||||
(beta in 1.14) and [RuntimeClass](https://kubernetes.io/docs/concepts/containers/runtime-class/) (beta in 1.14) and learn how they can be leveraged to expose crypto
|
||||
hardware to workloads.
|
||||
|
||||
The device plugin framework, first introduced in Kubernetes 1.8, provides a way for hardware vendors
|
||||
to register and allocate node hardware resources to Kubelets. The plugins implement the hardware
|
||||
specific initialization logic and resource management. The pods can request hardware resources in
|
||||
their PodSpec, which also guarantees the pod is scheduled on a node that can provide those resources.
|
||||
|
||||
The device resource allocation for containers is non-trivial. For applications dealing with security,
|
||||
the hardware level isolation is critical. The PCIe based crypto acceleration device functions can
|
||||
benefit from IO hardware virtualization, through an I/O Memory Management Unit (IOMMU), to provide
|
||||
the isolation: an *IOMMU group* the device belongs to provides the isolated resource for a workload
|
||||
(assuming the crypto cards do not share the IOMMU group with other devices). The number of isolated
|
||||
resources can be further increased if the PCIe device supports the Single-Root I/O Virtualization
|
||||
(SR-IOV) specification. SR-IOV allows the PCIe device to be split further to *virtual functions* (VF),
|
||||
derived from *physical function* (PF) devices, and each belonging to their own IOMMU group. To expose
|
||||
these IOMMU isolated device functions to user space and containers, the host kernel should bind
|
||||
them to a specific device driver. In Linux, this driver is vfio-pci and it makes each device
|
||||
available through a character device in user space. The kernel vfio-pci driver provides user space
|
||||
applications with a direct, IOMMU backed access to PCIe devices and functions, using a mechanism
|
||||
called *PCI passthrough*. The interface can be leveraged by user space frameworks, such as the
|
||||
Data Plane Development Kit (DPDK). Additionally, virtual machine (VM) hypervisors can provide
|
||||
these user space device nodes to VMs and expose them as PCI devices to the guest kernel.
|
||||
Assuming support from the guest kernel, the VM gets close to native performant direct access to the
|
||||
underlying host devices.
|
||||
|
||||
To advertise these device resources to Kubernetes, we can have a simple Kubernetes device plugin
|
||||
that runs the initialization (i.e., binding), calls kubelet’s `Registration` gRPC service, and
|
||||
implements the DevicePlugin gRPC service that kubelet calls to, e.g., to `Allocate` the resources
|
||||
upon Pod creation.
|
||||
|
||||
## Device Assignment and Pod Deployment
|
||||
|
||||
At this point, you may ask what the container could do with a VFIO device node? The answer comes
|
||||
after we first take a quick look into the Kubernetes RuntimeClass.
|
||||
|
||||
The Kubernetes RuntimeClass was created to provide better control and configurability
|
||||
over a variety of *runtimes* (an earlier [blog post](https://kubernetes.io/blog/2018/10/10/kubernetes-v1.12-introducing-runtimeclass/) goes into the details of the needs,
|
||||
status and roadmap for it) that are available in the cluster. In essence, the RuntimeClass
|
||||
provides cluster users better tools to pick and use the runtime that best suits for the pod use case.
|
||||
|
||||
The OCI compatible [Kata Containers runtime](https://katacontainers.io/) provides workloads with a hardware virtualized
|
||||
isolation layer. In addition to workload isolation, the Kata Containers VM has the added
|
||||
side benefit that the VFIO devices, as `Allocate`’d by the device plugin, can be passed
|
||||
through to the container as hardware isolated devices. The only requirement is that the
|
||||
Kata Containers kernel has driver for the exposed device enabled.
|
||||
|
||||
That’s all it really takes to enable hardware accelerated crypto for container workloads. To summarize:
|
||||
|
||||
1. Cluster needs a device plugin running on the node that provides the hardware
|
||||
2. Device plugin exposes the hardware to user space using the VFIO driver
|
||||
3. Pod requests the device resources and Kata Containers as the RuntimeClass in the PodSpec
|
||||
4. The container has the hardware adaptation library and the OpenSSL engine module
|
||||
|
||||
Figure 2 shows the overall setup using the Container A illustrated earlier.
|
||||
|
||||
{{<figure width="600" src="/images/blog/2019-04-23-hardware-accelerated-tls-termination/k8s-blog-fig2.png" caption="Figure 2. Deployment overview">}}
|
||||
|
||||
## Reference Setup
|
||||
|
||||
Finally, we describe the necessary building blocks and steps to build a functional
|
||||
setup described in Figure 2 that enables hardware accelerated SSL termination in
|
||||
an Ingress Controller using an Intel® QuickAssist Technology (QAT) PCIe device.
|
||||
It should be noted that the use cases are not limited to Ingress controllers, but
|
||||
any OpenSSL based workload can be accelerated.
|
||||
|
||||
### Cluster configuration:
|
||||
* Kubernetes 1.14 (`RuntimeClass` and `DevicePlugin` feature gates enabled (both are `true` in 1.14)
|
||||
* RuntimeClass ready runtime and Kata Containers configured
|
||||
|
||||
### Host configuration:
|
||||
* Intel® QAT driver release with the kernel drivers installed for both host kernel and Kata Containers kernel (or on a rootfs as loadable modules)
|
||||
* [QAT device plugin](https://github.com/intel/intel-device-plugins-for-kubernetes/tree/master/cmd/qat_plugin) DaemonSet deployed
|
||||
|
||||
### Ingress controller configuration and deployment:
|
||||
* [HAproxy-ingress](https://github.com/jcmoraisjr/haproxy-ingress) ingress controller in a modified container that has
|
||||
* the QAT HW HAL user space library (part of Intel® QAT SW release) and
|
||||
* the [OpenSSL QAT Engine](https://github.com/intel/QAT_Engine/) built in
|
||||
* Haproxy-ingress ConfigMap to enable QAT engine usage
|
||||
* `ssl-engine=”qat”`
|
||||
* `ssl-mode-async=true`
|
||||
* Haproxy-ingress deployment `.yaml` to
|
||||
* Request `qat.intel.com: n` resources
|
||||
* Request `runtimeClassName: kata-containers` (name value depends on cluster config)
|
||||
* (QAT device config file for each requested device resource with OpenSSL engine configured available in the container)
|
||||
|
||||
Once the building blocks are available, the hardware accelerated SSL/TLS can be tested by following the [TLS termination
|
||||
example](https://github.com/jcmoraisjr/haproxy-ingress/tree/master/examples/tls-termination) steps. In order to verify the hardware is used, you can check `/sys/kernel/debug/*/fw_counters` files on host as they
|
||||
get updated by the Intel® QAT firmware.
|
||||
|
||||
Haproxy-ingress and HAproxy are used because HAproxy can be directly configured to use the OpenSSL engine using
|
||||
`ssl-engine <name> [algo ALGOs]` configuration flag without modifications to the global openssl configuration file.
|
||||
Moreover, HAproxy can offload configured algorithms using asynchronous calls (with `ssl-mode-async`) to further improve performance.
|
||||
|
||||
## Call to Action
|
||||
|
||||
In this blog post we have shown how Kubernetes Device Plugins and RuntimeClass can be used to provide isolated hardware
|
||||
access for applications in pods to offload crypto operations to hardware accelerators. Hardware accelerators can be used
|
||||
to speed up crypto operations and also save CPU cycles to other tasks. We demonstrated the setup using HAproxy that already
|
||||
supports asynchronous crypto offload with OpenSSL.
|
||||
|
||||
The next steps for our team is to repeat the same for Envoy (with an OpenSSL based TLS transport socket built
|
||||
as an extension). Furthermore, we are working to enhance Envoy to be able to [offload BoringSSL asynchronous
|
||||
private key operations](https://github.com/envoyproxy/envoy/issues/6248) to a crypto acceleration hardware. Any review feedback or help is appreciated!
|
||||
|
||||
How many CPU cycles can your crypto application save for other tasks when offloading crypto processing to a dedicated accelerator?
|
||||
@@ -0,0 +1,55 @@
|
||||
---
|
||||
layout: blog
|
||||
title: 'How You Can Help Localize Kubernetes Docs'
|
||||
date: 2019-04-26
|
||||
---
|
||||
|
||||
**Author: Zach Corleissen (Linux Foundation)**
|
||||
|
||||
Last year we optimized the Kubernetes website for [hosting multilingual content](/blog/2018/11/08/kubernetes-docs-updates-international-edition/). Contributors responded by adding multiple new localizations: as of April 2019, Kubernetes docs are partially available in nine different languages, with six added in 2019 alone. You can see a list of available languages in the language selector at the top of each page.
|
||||
|
||||
By _partially available_, I mean that localizations are ongoing projects. They range from mostly complete ([Chinese docs for 1.12](https://v1-12.docs.kubernetes.io/zh/)) to brand new (1.14 docs in [Portuguese](https://kubernetes.io/pt/)). If you're interested in helping an existing localization, read on!
|
||||
|
||||
## What is a localization?
|
||||
|
||||
Translation is about words and meaning. Localization is about words, meaning, process, and design.
|
||||
|
||||
A localization is like a translation, but more thorough. Instead of just translating words, a localization optimizes the framework for writing and publishing words. For example, most site navigation features (button text) on kubernetes.io are strings contained in a [single file](https://github.com/kubernetes/website/tree/master/i18n). Part of creating a new localization involves adding a language-specific version of that file and translating the strings it contains.
|
||||
|
||||
Localization matters because it reduces barriers to adoption and support. When we can read Kubernetes docs in our own language, it's easier to get started using Kubernetes and contributing to its development.
|
||||
|
||||
## How do localizations happen?
|
||||
|
||||
The availability of docs in different languages is a feature—and like all Kubernetes features, contributors develop localized docs in a SIG, share them for review, and add them to the project.
|
||||
|
||||
Contributors work in teams to localize content. Because folks can't approve their own PRs, localization teams have a minimum size of two—for example, the Italian localization has two contributors. Teams can also be quite large: the Chinese team has several dozen contributors.
|
||||
|
||||
Each team has its own workflow. Some teams localize all content manually; others use editors with translation plugins and review machine output for accuracy. SIG Docs focuses on standards of output; this leaves teams free to adopt the workflow that works best for them. That said, teams frequently collaborate with each other on best practices, and sharing abounds in the best spirit of the Kubernetes community.
|
||||
|
||||
## Helping with localizations
|
||||
|
||||
If you're interested in starting a new localization for Kubernetes docs, the [Kubernetes contribution guide](https://kubernetes.io/docs/contribute/localization/) shows you how.
|
||||
|
||||
Existing localizations also need help. If you'd like to contribute to an existing project, join the localization team's Slack channel and introduce yourself. Folks on that team can help you get started.
|
||||
|
||||
Localization | Slack channel
|
||||
---|---
|
||||
Chinese (中文) | [#kubernetes-docs-zh](https://kubernetes.slack.com/messages/CE3LNFYJ1/)
|
||||
English | [#sig-docs](https://kubernetes.slack.com/messages/C1J0BPD2M/)
|
||||
French (Français) | [#kubernetes-docs-fr](https://kubernetes.slack.com/messages/CG838BFT9/)
|
||||
German (Deutsch) | [#kubernetes-docs-de](https://kubernetes.slack.com/messages/CH4UJ2BAL/)
|
||||
Hindi | [#kubernetes-docs-hi](https://kubernetes.slack.com/messages/CJ14B9BDJ/)
|
||||
Indonesian | [#kubernetes-docs-id](https://kubernetes.slack.com/messages/CJ1LUCUHM/)
|
||||
Italian | [#kubernetes-docs-it](https://kubernetes.slack.com/messages/CGB1MCK7X/)
|
||||
Japanese (日本語) | [#kubernetes-docs-ja](https://kubernetes.slack.com/messages/CAG2M83S8/)
|
||||
Korean (한국어) | [#kubernetes-docs-ko](https://kubernetes.slack.com/messages/CA1MMR86S/)
|
||||
Portuguese (Português) | [#kubernetes-docs-pt](https://kubernetes.slack.com/messages/CJ21AS0NA/)
|
||||
Spanish (Español) | [#kubernetes-docs-es](https://kubernetes.slack.com/messages/CH7GB2E3B/)
|
||||
|
||||
|
||||
## What's next?
|
||||
|
||||
There's a new [Hindi localization](https://kubernetes.slack.com/messages/CJ14B9BDJ/) beginning. Why not add your language, too?
|
||||
|
||||
As a chair of SIG Docs, I'd love to see localization spread beyond the docs and into Kubernetes components. Is there a Kubernetes component you'd like to see supported in a different language? Consider making a [Kubernetes Enhancement Proposal](https://github.com/kubernetes/enhancements/tree/master/keps) to support the change.
|
||||
|
||||
@@ -0,0 +1,39 @@
|
||||
---
|
||||
title: "Join us for the 2019 KubeCon Diversity Lunch & Hack"
|
||||
date: 2019-05-02
|
||||
slug: kubecon-diversity-lunch-and-hack
|
||||
---
|
||||
|
||||
**Authors:** Kiran Oliver, Podcast Producer, The New Stack
|
||||
|
||||
Join us for the 2019 KubeCon Diversity Lunch & Hack: Building Tech Skills & An Inclusive Community - Sponsored by Google Cloud and VMware
|
||||
|
||||
Registration for the Diversity Lunch opens today, May 2nd, 2019. To register, go to the main <a href="https://events.linuxfoundation.org/events/kubecon-cloudnativecon-europe-2019/schedule/">KubeCon + CloudNativeCon EU schedule</a>, then log in to your Sched account, and confirm your attendance to the Diversity Lunch. Please sign up ASAP once the link is live, as spaces will fill quickly. We filled the event in just a few days last year, and anticipate doing so again this year.
|
||||
|
||||
The 2019 KubeCon Diversity Lunch & Hack will be held at the Fira Gran Via Barcelona Hall 8.0 Room F1 on May 22nd, 2019 from 12:30-14:00.
|
||||
|
||||
If you’ve never attended a Diversity Lunch before, not to worry. All are welcome, and there’s a variety of things to experience and discuss.
|
||||
|
||||
First things first, let’s establish some ground rules:
|
||||
|
||||
This is a safe space. What does that mean? Simple:
|
||||
|
||||
1. Asking for and using people’s pronouns
|
||||
2. Absolutely no photography
|
||||
3. Awareness of your actions towards others. Do your best to ensure that you contribute towards making this environment welcoming, safe, and inclusive for all.
|
||||
4. Please avoid tech-heavy arbitrary community slang/jargon [keep in mind that not all of us are developers, many are tech-adjacent and/or new to the community]
|
||||
5. Act with care and empathy towards your fellow community members at all times.
|
||||
|
||||
This event also follows the <a href="https://events.linuxfoundation.org/events/kubecon-cloudnativecon-europe-2019/attend/code-of-conduct/">Code of Conduct</a> for all CNCF events.
|
||||
|
||||
We have run a very successful diversity lunch event before. This isn’t a trial run, nor is it a proof of concept. We had a fun, productive, and educational conversation last year in Seattle, and hope to do so again this year. As 2018’s KubeCon + CloudNativeCon in Seattle marked our first Diversity Lunch with pair programming, we hammered out a lot of kinks post-mortem, using that feedback to inform and improve upon our decision making, planning, and organizational process moving forward, to bring you an improved experience at the 2019 KubeCon + CloudNativeCon Diversity Lunch.
|
||||
|
||||
Tables not related to pair-programming or hands-on Kubernetes will be led by a moderator, where notes and feedback will then be taken and shared at the end of the lunch and in a post-mortem discussion after KubeCon+CloudNativeCon Barcelona ends, as part of our continuous improvement process. Some of last year’s tables were dedicated to topics that were submitted at registration, such as: security, D&I, service meshes, and more. You can suggest your own table topic on the registration form this year as well, and we highly encourage you to do so, particularly if you do not see your preferred topic or activity of choice listed. Your suggestions will then be used to determine the discussion table tracks that will be available at this year’s Diversity Lunch & Hack.
|
||||
|
||||
We hope you are also excited to participate in the ‘Hack’ portion of this ‘Lunch and Hack.’ This breakout track will include a variety of peer-programming exercises led by your fellow Kubernetes community members, with discussion leads working together with attendees hands-on to solve their Kubernetes-related problems in a welcoming, safe environment.
|
||||
|
||||
To make this all possible, we need you. Yes, you, to register. As much as we love having groups of diverse people all gather in the same room, we also need allies. If you’re a member of a privileged group or majority, you are welcome and encouraged to join us. Most importantly, we want you to take what you learn and experience at the Diversity Lunch back to both your companies and your open source communities, so that you can help us make positive changes not only within our industry, but beyond. No-one lives [or works] in a bubble. We hope that the things you learn here will carry over and bring about positive change in the world as a whole.
|
||||
|
||||
We look forward to seeing you!
|
||||
|
||||
_Special thanks to [Leah Petersen](https://www.linkedin.com/in/leahstunts/), [Sarah Conway](https://www.linkedin.com/in/sarah-conway-6166151/) and [Paris Pittman](https://www.linkedin.com/in/parispittman/) for their help in editing this post._
|
||||
@@ -0,0 +1,235 @@
|
||||
---
|
||||
layout: blog
|
||||
title: "Cat shirts and Groundhog Day: the Kubernetes 1.14 release interview"
|
||||
date: 2019-05-13
|
||||
---
|
||||
|
||||
<b>Author</b>: Craig Box (Google)
|
||||
|
||||
Last week we celebrated one year of the [Kubernetes Podcast from Google](https://kubernetespodcast.com/). In this weekly show, my co-host Adam Glick and I focus on all the great things that are happening in the world of Kubernetes and Cloud Native. From the news of the week, to interviews with people in the community, we help you stay up to date on everything Kubernetes.
|
||||
|
||||
Every few cycles we check in on the release process for Kubernetes itself. Last year we [interviewed the release managers for Kubernetes 1.11](https://kubernetespodcast.com/episode/010-kubernetes-1.11/), and shared that transcript on the Kubernetes blog. We got such great feedback that we wanted to share the transcript of our recent conversation with Aaron Crickenberger, the release manager for Kubernetes 1.14.
|
||||
|
||||
As always, the canonical version can be enjoyed by listening to [the podcast version](https://kubernetespodcast.com/episode/046-kubernetes-1.14/). If you like what you hear, [we encourage you to subscribe](https://kubernetespodcast.com/subscribe/)!
|
||||
|
||||
<hr/>
|
||||
|
||||
<b>CRAIG BOX: We like to start with our guests into digging into their backgrounds a little bit. Kubernetes is built from contributors from many different companies. You worked on Kubernetes at Samsung SDS before joining Google. Does anything change in your position in the community and the work you do, when you change companies?</b>
|
||||
|
||||
AARON CRICKENBERGER: Largely, no. I think the food's a little bit better at the current company! But by and large, I have gotten to work with basically the same people doing basically the same thing. I cared about the community first and Google second before I joined Google, and I kind of still operate that way mostly because I believe that Google's success depends upon the community's success, as does everybody else who depends upon Kubernetes. A good and healthy upstream makes a good and healthy downstream.
|
||||
|
||||
So that was largely why Samsung had me working on Kubernetes in the first place was because we thought the technology was legit. But we needed to make sure that the community and project as a whole was also legit. And so that's why you've seen me continue to advocate for transparency and community empowerment throughout my tenure in Kubernetes.
|
||||
|
||||
<b>ADAM GLICK: You co-founded the [Testing SIG](https://github.com/kubernetes/community/tree/master/sig-testing). How did you decide that that was needed, and at what stage in the process did you come to that?</b>
|
||||
|
||||
AARON CRICKENBERGER: This was very early on in the Kubernetes project. I'm actually a little hazy on specifically when it happened. But at the time, my boss, Bob Wise, worked with some folks within Google to co-found the Scalability SIG.
|
||||
|
||||
If you remember way, way back when Kubernetes first started, there was concern over whether or not Kubernetes was performance enough. Like, I believe it officially supported something on the order of 100 nodes. And there were some who thought, that's silly. I mean, come on, Google can do way more than that. And who in their right mind is going to use a container orchestrator that only supports 100 nodes?
|
||||
|
||||
And of course the thing is we're being super-conservative. We're trying to iterate, ship early and often. And so we helped push the boundaries to make sure that Kubernetes could prove that it worked up to a thousand nodes before it was even officially supported to say, look, it already does this, we're just trying to make sure we have all of the nuts and bolts tightened.
|
||||
|
||||
OK, so great. We decided we needed to create a thing called a SIG in the very first place to talk about these things and make sure that we were moving in the right direction. I then turned my personal attention to testing as the next thing that I believe needed a SIG. So I believe that testing was the second SIG ever to be created for Kubernetes. It was co-founded initially with [Ike McCreary](https://github.com/ihmccreery) who, at the time I believe, was an SRE for Google, and then eventually it was handed over to some folks who work in the engineering productivity part of Google where I think it aligned really well with testing's interests.
|
||||
|
||||
It is like "I don't know what you people are trying to write here with Kubernetes, but I want to help you write it better, faster, and stronger". And so I want to make sure we, as a community and as a project, are making it easier for you to write tests, easier for you to run tests, and most importantly, easier for you to act based on those test results.
|
||||
|
||||
That came down to, let's make sure that Kubernetes gets tested on more than just Google Cloud. That was super important to me, as somebody who operated not in Google Cloud but in other clouds. I think it really helped sell the story and build confidence in Kubernetes as something that worked effectively on multiple clouds. And I also thought it was really helpful to see SIG Testing in the community's advocacy move us to a world today we can use test grids so that everybody see the same set of test results to understand what is allowed to prevent Kubernetes from going out the door.
|
||||
|
||||
The process was basically just saying, let's do it. The process was finding people who were motivated and suggesting that we meet on a recurring basis and we try to rally around a common set of work. This was sort of well before SIG governance was an official thing. And we gradually, after about a year, I think, settled on the pattern that most SIGs follow where you try to make sure you have a meeting agenda, you have a Slack channel, you have a mailing list, you discuss everything out in the open, you try to use sort a consistent set of milestones and move forward.
|
||||
|
||||
<b>CRAIG BOX: A couple of things I wanted to ask about your life before Kubernetes. Why is there a [Black Hawk flight simulator in a shipping container?](https://www.rockwellcollins.com/Products-and-Services/Defense/Simulation-and-Training/Training-Systems/Transportable-Black-Hawk-Operations-Simulator.aspx)</b>
|
||||
|
||||
AARON CRICKENBERGER: As you may imagine, Black Hawk helicopters are flown in a variety of places around the world, not just next to a building that happens to have a parking lot next to it. And so in order to keep your pilots fresh, you may want to make sure they have good training hours and flight time, without spending fuel to fly an actual helicopter.
|
||||
|
||||
I was involved in helping make what's called a operation simulator, to train pilots on a bunch of the procedures using the same exact hardware that was deployed in Black Hawk helicopters, complete with motion seats that would shake to simulate movement and a full-fidelity visual system. This was all packed up in two shipping containers so that the simulator could be deployed wherever needed.
|
||||
|
||||
I definitely had a really fun experience working on this simulator in the field at an Air Force base prior to a conference where I got to experience F-16s doing takeoff drills, which was amazing. They would get off the runway, and then just slam the afterburners to max and go straight up into the air. And I got to work on graphic simulation bugs. It was really cool.
|
||||
|
||||
<b>CRAIG BOX: And for a lot of people, when you click on the web page they have listed in the GitHub link, you get their resume, or you get the list of open source projects they work on. In your case, there is [a SoundCloud page](https://soundcloud.com/spiffxp). What do people find on that page?</b>
|
||||
|
||||
AARON CRICKENBERGER: They get to see me living my whole life. I find that music is a very important part of my life. It's a non-verbal voice that I have developed over time. I needed some place to host that. And then it came down between SoundCloud and Bandcamp, and SoundCloud was a much easier place to host my recordings.
|
||||
|
||||
So you get to hear the results of me having picked up a guitar and noodling with that about five years ago. You get to hear what I've learned messing around with Ableton Live. You get to hear some mixes that I've done of ambient music. And I haven't posted anything in a while there because I'm trying to get my recording of drums just right.
|
||||
|
||||
So if you go to [my YouTube channel](https://www.youtube.com/channel/UCfnUO-9Q_gMraUXjbk4p50g), mostly what you'll see are recordings of the various SIG meetings that I've participated in. But if you go back a little bit earlier than that, you'll see that I do, in fact, play the drums. I'm trying to get those folded into my next songs.
|
||||
|
||||
<b>CRAIG BOX: Do you know who [Hugh Padgham](https://en.wikipedia.org/wiki/Hugh_Padgham#The_%22gated_drum%22_sound) is?</b>
|
||||
|
||||
AARON CRICKENBERGER: I do not.
|
||||
|
||||
<b>CRAIG BOX: Hugh Padgham was the recording engineer who did the gated reverb drum sound that basically defined Phil Collins in the 1980s. I think you should call him up if you're having problems with your drum sound.</b>
|
||||
|
||||
AARON CRICKENBERGER: That is awesome.
|
||||
|
||||
<b>ADAM GLICK: You mentioned you can also find videos of the work that you're doing with the SIG. How did you become the release manager for 1.14?</b>
|
||||
|
||||
AARON CRICKENBERGER: I've been involved in the Kubernetes release process way back in the 1.4 days. I started out as somebody who tried to help figure out, how do you write release notes for this thing? How do you take this whole mess and try to describe it in a sane way that makes sense to end users and developers? And I gradually became involved in other aspects of the release over time.
|
||||
|
||||
I helped out with CI Signal. I helped out with issue triage. When I helped out with CI Signal, I wrote the [very first playbook](https://github.com/kubernetes/sig-release/blob/master/release-team/role-handbooks/ci-signal/README.md) to describe what it is I do around here. That's the model that has since been used for the rest of the release team, where every role describes what they do in a playbook that is used not just for their own benefit, but to help them train other people.
|
||||
|
||||
Formally how I became release lead was I served as release shadow in 1.13. And when release leads are looking to figure out who's going to lead the next release, they turn around and they look at their shadows, because those are who they have been helping out and training.
|
||||
|
||||
<b>CRAIG BOX: If they don't have a shadow, do they have to wait another three months and do a release again?</b>
|
||||
|
||||
AARON CRICKENBERGER: They do not. The way it works is the release lead can look at their shadows, then they take a look at the rest of their release team leads to see if there is sufficient experience there. And then if not, they consult with the chairs of SIG release.
|
||||
|
||||
So for example, for Kubernetes v1.15, I ended up in an unfortunate situation where neither of my shadows were available to step up and become the leads for 1.15. I consulted with [Claire Lawrence](https://github.com/claurence), who was my enhancements lead for 1.14 and who was on the release team for two quarters, and so met the requirements to become a release lead that way. So she will be the release lead for v1.15.
|
||||
|
||||
<b>CRAIG BOX: That was a fantastic answer to a throwaway [Groundhog Day](https://en.wikipedia.org/wiki/Groundhog_Day) joke. I appreciate that.</b>
|
||||
|
||||
AARON CRICKENBERGER: [LAUGHS]
|
||||
|
||||
<b>ADAM GLICK: You can ask it again and see what the answer is, and then another time, and see how it evolves over time.</b>
|
||||
|
||||
AARON CRICKENBERGER: I'm short on my Groundhog Day riffs. I'll come back to you.
|
||||
|
||||
<b>ADAM GLICK: What are your responsibilities as the release lead?</b>
|
||||
|
||||
AARON CRICKENBERGER: Don't Panic. I mean, essentially, a release lead's job is to make the final call, and then hold the line by making the final call. So what you shouldn't be doing as a release lead is attempting to dive in and fix all of the things, or do all of the things, or second-guess anybody else's work. You are there principally and primarily to listen to everybody else's advice and help them make the best decision. And only in the situations where there's not a clear consensus do you wade in and make the call yourself.
|
||||
|
||||
I feel like I was helped out by a very capable team in this regard, this release cycle. So it was super helpful. But as somebody who has what I like to call an "accomplishment monkey" on my back, it can be very difficult to resist the urge to dive right in and help out, because I have been there before. I have the boots-on-the-ground experience.
|
||||
|
||||
The release lead's job is not to be the boots on the ground, but to help make sure that everybody who is boots on the ground is actually doing what they need to do and unblocked in doing what they need to do. It also involves doing songs and dances and making funny pictures. So I view it more as like it's about effective communication. And doing a lot of songs and dances, and funny pictures, and memes is one way that I do that.
|
||||
|
||||
So one way that I thought it would help people pay attention to the release updates that I gave every week at the Kubernetes community meeting was to make sure that I wore a different cat T-shirt each week. After people riffed and joked out my first cat T-shirt where I said, I really need coffee right "meow", and somebody asked if I got that coffee from a "purr-colator", I decided to up the ante.
|
||||
|
||||
And I've heard that people will await those cat T-shirts. They want to know what the latest one is. I even got a special cat T-shirt just to signify that code freeze was coming.
|
||||
|
||||
We also decided that instead of imposing this crazy process that involved a lot of milestones, and labels, and whatnot that would cause the machinery to impose a bunch of additional friction, I would just post a lot of memes to Twitter about code freeze coming. And that seems to have worked out really well. So by and large, the release lead's job is communication, unblocking, and then doing nothing for as much as possible.
|
||||
|
||||
It's really kind of difficult and terrifying because you always have this feeling that you may have missed something, or that you're just not seeing something that's out there. So I'm sitting in this position with a release that has been extremely stable, and I spent a lot of time thinking, OK, what am I missing? Like, this looks too good. This is too quiet. There's usually something that blows up. Come on, what is it, what is it, what is it? And it's an exercise in keeping that all in and not sharing it with everybody until the release is over.
|
||||
|
||||
<b>ADAM GLICK: [He is here in a cat T-shirt](https://twitter.com/KubernetesPod/status/1110611630180597760/photo/1), as well.
|
||||
|
||||
When a new US President takes over the office, it's customary that the outgoing president leaves them a note with advice in it. Aside from the shadow team, is there something similar that exists with Kubernetes release management?</b>
|
||||
|
||||
AARON CRICKENBERGER: Yeah, I would say there's a very special-- I don't know what the word is I'm looking for here-- bond, relationship, or something where people who have been release leads in the past are very empathetic and very supportive of those who step into the role as release lead.
|
||||
|
||||
You know, I talked about release lead being a lot of uncertainty and second-guessing yourself, while on the outside you have to pretend like everything is OK. And having the support of people who have been there and who have gone through that experience is tremendously helpful.
|
||||
|
||||
So I was able to reach out to a previous release lead. Not to pull the game with-- what is it, like two envelopes? The first envelope, you blame the outgoing president. The second envelope, you write two letters. It's not quite like that.
|
||||
|
||||
I am totally happy to be blamed for all of the changes we made to the release process that didn't go well, but I'm also happy to help support my successor. I feel like my job as a release lead is, number one, make sure the release gets out the door, number two, make sure I set up my successor for success.
|
||||
|
||||
So I've already been meeting with Claire to describe what I would do as the introductory steps. And I plan on continuing to consult with Claire throughout the release process to make sure that things are going well.
|
||||
|
||||
<b>CRAIG BOX: If you want to hear the perspective from some previous release leads, check out [episode 10](http://kubernetespodcast.com/episode/010-kubernetes-1.11/), where we interview Josh Berkus and Tim Pepper.</b>
|
||||
|
||||
<b>ADAM GLICK: What do you plan to put into that set of notes for Claire?</b>
|
||||
|
||||
AARON CRICKENBERGER: That's a really good question. I would tell Claire to trust her team first and trust her gut second. Like I said, I think it is super important to establish trust with your team, because the release is this superhuman effort that involves consuming, or otherwise fielding, or shepherding the work of hundreds of contributors.
|
||||
|
||||
And your team is made up of at least 13 people. You could go all the way up to 40 or 50, if you include all of the people that are being trained by those people. There's so much work out there. It's just more work than any one person can possibly handle.
|
||||
|
||||
It's honestly the same thing I will tell new contributors to Kubernetes is that there's no way you can possibly understand all of it. You will not understand the shape of Kubernetes. You will never be the expert who knows literally all of the things, and that's OK. The important part is to make sure that you have people who, when you don't know the answer, you know who to ask for the answer. And it is really helpful if your team are those people.
|
||||
|
||||
<b>CRAIG BOX: The specific version that you've been working on and the release that's just come out is Kubernetes 1.14. What are some of the new things in this release?</b>
|
||||
|
||||
AARON CRICKENBERGER: This release of Kubernetes contains more stable enhancements than any other release of Kubernetes ever. And I'm pretty proud of that fact. I know in the past you may have heard other release leads talk about, like, this is the stability release, or this time we're really making things a little more mature. But I feel a lot of confidence in saying that this time around.
|
||||
|
||||
Like, I stood in a room, and it was a leadership summit, I think, back in 2017 where we said, look, we're really going to try and make Kubernetes more stable. And we're going to focus on sort of hardening the core of Kubernetes and defining what the core of Kubernetes is. And we're not going to accept a bunch of new features. And then we kind of went and accepted a bunch of new features. And that was a while ago. And here we are today.
|
||||
|
||||
But I think we are finally starting to see the results of work that was started back then. [Windows Server Container Support](https://github.com/kubernetes/enhancements/blob/master/keps/sig-windows/20190103-windows-node-support.md) is probably the biggest one. You can hear Michael Michael tell stories about how SIG Windows was started about three years ago. And today, they can finally announce that Windows Server containers have gone GA. That's a huge accomplishment.
|
||||
|
||||
A lot of the heavy lifting for this, I believe, came at the end. It started with a conversation in Kubernetes 1.13, and was really wrapped up this release where we define, what are Windows Server containers, exactly? How do they differ from Docker containers or other container runtimes that run on Linux?
|
||||
|
||||
Because today so much of the assumptions people make about the functionality that Kubernetes offers are also baked in with the functionality that Linux-based containers offer. And so we wanted to enable people to use the awesome Kubernetes orchestration capabilities that they have come to love, but to also use that to orchestrate some applications or capabilities that are only available on Windows.
|
||||
|
||||
So we put together what's called a [Kubernetes Enhancement Proposal](https://github.com/kubernetes/enhancements/tree/master/keps) process, or a KEP, for short. And we said that we're going to use these KEPs to describe exactly what the criteria are to call something alpha, or beta, or stable. And so the Windows feature allowed us to use a KEP-- or in getting Windows in here, we used the KEP to describe everything that would and would not work for Windows Server containers. That was super huge. And that really, I think, helped us better understand or define what Kubernetes is in that context.
|
||||
|
||||
But OK, I've spent most of the time answering your question with just one single stable feature.
|
||||
|
||||
<b>CRAIG BOX: Well, let's dig a little bit in to the KEP process then, because this is the first release where there's a new rule. It says, all proposed enhancements for this release must have an associated KEP. So that's a Kubernetes Enhancement Proposal, a one-page document that describes it. What has the process been like of A, getting engineers on-board with using that, and then B, building something based on these documents?</b>
|
||||
|
||||
AARON CRICKENBERGER: It is a process of continued improvement. So it is by no means done, but it honestly required a lot of talking, and saying the same thing over and over to the same people or to different people, as is often the case when it comes to things that involve communication and process changes. But by and large, everybody was pretty much on-board with this.
|
||||
|
||||
There was a little bit of confusion, though, over how high the bar would be set and how rigorously or rigidly we would be enforcing these criteria. And that's where I feel like we have room to iterate and improve on. But we have collectively agreed that, yeah, we do like having all of the information about a particular enhancement in one place. Right?
|
||||
|
||||
The way the world used to operate before is we would throw around Google Docs, that were these design proposals, and then we'd comment on those a bunch. And then eventually, those were turned into markdown files. And those would end up in the community repo,
|
||||
|
||||
And then we'd have a bunch of associated issues that talked about that. And then maybe somebody would open up another issue that they'd call an umbrella issue. And then a bunch of comments would be put there. And then there's lots of discussion that goes on in the PRs. There's like seven different things that I just rattled off there.
|
||||
|
||||
So KEPs are about focusing all of the discussion about the design and implementation and reasoning behind enhancements in one single place. And I think there, we are fully on board. Do we have room to improve? Absolutely. Humans are involved, and it's a messy process. We could definitely find places to automate this better, structure it better. And I look forward to seeing those improvements happen.
|
||||
|
||||
You know, I think another one of the big things was a lot of these KEPs were mired across three different SIGs. There was sort of SIG architecture who had the technical vision for these. There was SIG PM, who-- you know, pick your P of choice-- product, project, process, program, people who are better about how to shepherd things forward, and then SIG release, who just wanted to figure out, what's landing in the release, and why, and how, and why is it important? And so taking the responsibilities across all of those three SIGs and putting it in the right place, which is SIG PM, I think really will help us iterate properly, moving forward.
|
||||
|
||||
<b>CRAIG BOX: The other change in this release is that there is no code slush. [What is a code slush, and why don't we have one anymore?](https://github.com/kubernetes/sig-release/issues/269)</b>
|
||||
|
||||
AARON CRICKENBERGER: That's a really good question. I had 10 different people ask me that question over the past couple of months, quarters, years. Take your pick. And so I finally decided, if nobody knows what a code slush is, why do we even have it?
|
||||
|
||||
<b>CRAIG BOX: It's like a thawed freeze, but possibly with sugar?</b>
|
||||
|
||||
AARON CRICKENBERGER: [LAUGHING] So code slush is about-- we want to slow the rate of change prior to code freeze. Like, let's accept code freeze as this big deadline where nothing's going to happen after a code freeze.
|
||||
|
||||
So while I really want to assume and aspire to live in a world where developers are super productive, and start their changes early, and get them done when they're done, today, I happen to live in a world where developers are driven by deadlines. And they get distracted. And there's other stuff going on. And then suddenly, they realize there's a code freeze ahead of them.
|
||||
|
||||
And this wonderful feature that they've been thinking about implementing over the past two months, they now have to get done in two weeks. And so suddenly, all sorts of code starts to fly in super fast and super quickly. And OK, that's great. I love empowering people to be productive.
|
||||
|
||||
But what we don't want to have happen is somebody decide to land some massive feature or enhancement that changes absolutely everything. Or maybe they decided they want to refactor the world. And if they do that, then they make everybody else's life super difficult because of merge conflicts and rebases. Or maybe all of the test signal that we had kind-of grown accustomed to and gotten used to, completely changes.
|
||||
|
||||
So code slush was about reminding people, hey, don't be jerks. Be kind of responsible. Please try not to land anything super huge at the last minute. But the way that we enforced this was with, like, make sure your PR has a milestone. And make sure that it has priority critical/urgent. In times past, we were like, make sure there is a label called status approved for milestone.
|
||||
|
||||
We were like, what do all these things even mean? People became obsessed with all the labels, and the milestones, and the process. And they never really paid attention to why we're asking people to pay attention to the fact that code freeze was coming soon.
|
||||
|
||||
<b>ADAM GLICK: Process for process sake, they could start to build on top of each other. You mentioned that there is a number of other things in the release. Do you want to talk about some of the other pieces that are in there?</b>
|
||||
|
||||
AARON CRICKENBERGER: Sure. I think two of the other stable features that I believe other people will find to be exciting are [readiness gates](https://github.com/kubernetes/enhancements/blob/master/keps/sig-network/0007-pod-ready%2B%2B.md) and [Pod priority and preemption](https://github.com/kubernetes/enhancements/blob/master/keps/sig-scheduling/20190131-pod-priority-preemption.md). Today, Pods have the concept of liveliness and readiness. A live Pod has an application running in it, but it might not be ready to do anything. And so when a Pod is ready, that means it's ready to receive traffic.
|
||||
|
||||
So if you're thinking of some big application that's scaled out everywhere, you want to make sure your Pods are only handling traffic when they're good and ready to do so. But prior to 1.14, the only ways you could verify that were by using either TCP probes, HTTP probes, or exec probes. Either make sure that ports are open inside of the container, or run a command inside of the container and see what that command says.
|
||||
|
||||
And then you can definitely customize a fair amount there, but that requires that you put all of that information inside of the Pod. And it might be really useful for some cluster operators to signify some more overarching concerns that they have before a Pod could be ready. So just-- I don't know-- make sure a Pod has registered with some other system to make sure that it is authorized to serve traffic, or something of that nature. Pod readiness gates allow that sort of capability to happen-- to transparently extend the conditions that you use to figure out whether a Pod is ready for traffic. We believe this will enable more sophisticated orchestration and deployment mechanisms for people who are trying to manage their applications and services.
|
||||
|
||||
I feel like Pod priority and preemption will be interesting to consumers who like to oversubscribe their Kubernetes clusters. Instead of assuming everything is the same size and is the same priority, and first Pods win, you can now say that certain Pods are more important than other Pods. They get scheduled before other Pods, and maybe even so that they kick out other Pods to make room for the really important Pods.
|
||||
|
||||
You could think of it as if you have any super important agents or daemons that have to run on your cluster. Those should always be there. Now, you can describe them as high-priority to make sure that they are definitely always there and always scheduled before anything else is.
|
||||
|
||||
<b>ADAM GLICK: Are there any other new features that are in alpha or beta that you're keeping your eye on?</b>
|
||||
|
||||
AARON CRICKENBERGER: Yeah. So I feel like, on the beta side of things, a lot of what I am interested in-- if I go back to my theme of maturity, and stability, and defining the core of Kubernetes, I think that the storage SIG has been doing amazing work. They continue to ship out, quarter, after quarter, after quarter, after quarter, new and progressive enhancements to storage-- mostly these days through the CSI, Container Storage Interface project, which is fantastic. It allows you to plug in arbitrary pieces of storage functionality.
|
||||
|
||||
They have a number of things related to that that are in beta this time around, such as topology support. So you're going to be able to more accurately express how and where your CSI volumes need to live relative to your application. Block storage support is something I've heard a number of people asking for, as well as the ability to define [durable local volumes](https://github.com/kubernetes/enhancements/blob/master/keps/sig-storage/20190124-local-persistent-volumes.md).
|
||||
|
||||
Let's say you're running a Pod on a node, and you want to make sure it's writing directly to the node's local volumes. And that way, it could be super performant. Cool. Give it an emptydir. It'll be fine.
|
||||
|
||||
But if you destroy the Pod, then you lose all the data that the Pod wrote. And so again, I go back to the example of maybe it's an agent, and it's writing a bunch of useful, stateful information to disk. And you'd love for the agent to be able to go away and something to replace it, and be able to get all of that information off of disk. Local durable volumes allow you to do that. And you get to do that in the same way that you're used to specifying durable or persistent volumes that are given to you by a cloud provider, for example.
|
||||
|
||||
Since I did co-found SIG testing, I think I have to call out a testing feature that I like. It's really tiny and silly, but it has always bugged me that when you try to download the tests, you download something that's over a gigabyte in size. That's the way things used work for Kubernetes back in the old days for Kubernetes client and server stuff as well. And we have since broken that up into-- you only need to download the binaries that makes sense for your platform.
|
||||
|
||||
So say I'm developing Kubernetes on my MacBook. I probably don't need to download the Linux test binaries, or the Windows test binaries, or the ARM64 test binaries, or the s390x test binaries. Did I mention Kubernetes supports a lot of different architectures?
|
||||
|
||||
<b>CRAIG BOX: I hadn't noticed s390 was a supported platform until now.</b>
|
||||
|
||||
AARON CRICKENBERGER: It is definitely something that we build binaries for. I'm not sure if we've actually seen a certified conformant Kubernetes that runs on s390, but it is definitely one of the things we build Kubernetes against.
|
||||
|
||||
Not having to download an entire gigabyte plus of binaries just to run some tests is super great. I like to live in a world where I don't have to build the tests from scratch. Can I please just run a program that has all the tests? Maybe I can use that to soak test or sanity test my cluster to make sure that everything is OK. And downloading just the thing that I need is super great.
|
||||
|
||||
<b>CRAIG BOX: You're talking about the idea of Kubernetes having a core and the idea of releases and stability. If you think back to Linux distributions maybe even 10 years ago, we didn't care so much about the version number releases of the kernel anymore, but we cared when there was a new feature in a Red Hat release. Do you think we're getting to that point with Kubernetes at the moment?</b>
|
||||
|
||||
AARON CRICKENBERGER: I think that is one model that people really hope to see Kubernetes move toward. I'm not sure if it is the model that we will move toward, but I think it is an ongoing discussion. So you know, we've created a working group called [WG LTS](https://github.com/kubernetes/community/tree/master/wg-lts). I like to call it by its longer name-- WG "to LTS, or not to LTS". What does LTS even mean? What are we trying to release and support?
|
||||
|
||||
Because I think that when people think about distributions, they do naturally gravitate towards some distributions have higher velocity release cadences, and others have slower release cadences. And that's cool and great for people who want to live on a piece of software that never ever changes. But those of us who run software at scale find that you can't actually prevent change from happening. There will always be pieces of your infrastructure, or your environment, or your software, that are not under your control.
|
||||
|
||||
And so anything we can do to achieve what I like to call a dynamic stability is probably better for everybody involved. Make the cost of change as low as you possibly can. Make the pain of changing and upgrade as low as you possibly can, and accept that everything will always be changing all the time.
|
||||
|
||||
So yeah. Maybe that's where Linux lives, where the Kernel is always changing. And you can either care about that, or not. And you can go with a distribution that is super up-to-date with the Linux Kernel, or maybe has a slightly longer upgrade cadence. But I think it's about enabling both of those options. Because I think if we try to live in a world where there are only distributions and nothing else, that's going to actually harm everybody in the long term and maybe bring us away from all of these cloud-native ideals that we have, trying to accept change as a constant.
|
||||
|
||||
<b>ADAM GLICK: We can't let you go without talking about the Beard. What is SIG Beard, and how critical was it in you becoming the 1.14 release manager?</b>
|
||||
|
||||
AARON CRICKENBERGER: I feel like it's a new requirement for all release leads to be a member of SIG Beard. SIG Beard happened because, one day, I realized I had gotten lazy, and I had this just ginormous and magnificent beard. It was really flattering to have Brendan Burns up on stage at KubeCon Seattle compliment my beard in front of an audience of thousands of people. I cannot tell you what that feels like.
|
||||
|
||||
But to be serious for a moment, like OK, I'm a dude. I have a beard. There are a lot of dudes who work in tech, and many dudes are bearded. And this is by no means a way of being exclusionary, or calling that out, or anything like that. It was just noticing that while I was on camera, there seemed to be more beard than face at times. And what is that about?
|
||||
|
||||
And I had somebody start referring to me as "The Beard" in my company. It turns out they read Neil Stevenson's "[Cryptonomicon](https://en.wikipedia.org/wiki/Cryptonomicon)," if you're familiar with that book at all.
|
||||
|
||||
<b>ADAM GLICK: It's a great book.</b>
|
||||
|
||||
AARON CRICKENBERGER: Yeah. It talks about how you have the beard, and you have the suit. The suit is the person who's responsible for doing all the talking, and the beard is responsible for doing all the walking. And I guess I have gained a reputation for doing an awful lot of walking and showing up in an awful lot of places. And so I thought I would embrace that.
|
||||
|
||||
When I showed up to Google my first day at work where I was looking for the name tag that shows what desk is mine, and my name tag was SIG Beard. And I don't know who did it, but I was like, all right, I'm running with it. And so I referred to myself as "Aaron of SIG Beard" from then on.
|
||||
|
||||
And so to me, the beard is not so much about being bearded on my face, but being bearded at heart-- being welcoming, being fun, embracing this community for all of the awesomeness that it has, and encouraging other people to do the same. So in that regard, I would like to see more people be members of SIG Beard. I'm trying to figure out ways to make that happen. And yeah, it's great.
|
||||
|
||||
<hr/>
|
||||
|
||||
<i>[Aaron Crickenberger](http://twitter.com/spiffxp) is a senior test engineer with [Google Cloud](https://cloud.google.com/). He co-founded the Kubernetes Testing SIG, has participated in every Kubernetes release since version 1.4, has served on the [Kubernetes steering committee](https://github.com/kubernetes/steering) since its inception in 2017, and most recently served as the Kubernetes 1.14 release lead.
|
||||
|
||||
You can find the [Kubernetes Podcast from Google](http://www.kubernetespodcast.com/) at [@kubernetespod](https://twitter.com/KubernetesPod) on Twitter, and you can [subscribe](https://kubernetespodcast.com/subscribe/) so you never miss an episode. Please come and say Hello to us at KubeCon EU!</i>
|
||||
@@ -0,0 +1,77 @@
|
||||
---
|
||||
layout: blog
|
||||
title: "Expanding our Contributor Workshops"
|
||||
date: 2019-05-14
|
||||
slug: expanding-our-contributor-workshops
|
||||
---
|
||||
|
||||
**Authors:** Guinevere Saenger (GitHub) and Paris Pittman (Google)
|
||||
|
||||
**tl;dr** - learn about the contributor community with us and land your first
|
||||
PR! We have spots available in [Barcelona][eu] (registration **closes** on
|
||||
Wednesday May 15, so grab your spot!) and the upcoming [Shanghai][cn] Summit.
|
||||
The Barcelona event is poised to be our biggest one yet, with more registered
|
||||
attendees than ever before!
|
||||
|
||||
Have you always wanted to contribute to Kubernetes, but not sure where to begin?
|
||||
Have you seen our community’s many code bases and seen places to improve? We
|
||||
have a workshop for you!
|
||||
|
||||
|
||||
KubeCon + CloudNativeCon Barcelona’s [new contributor workshop][ncw] will be the
|
||||
fourth one of its kind, and we’re really looking forward to it! The workshop was
|
||||
kickstarted last year at KubeConEU in Copenhagen, and so far we have taken it to
|
||||
Shanghai and Seattle, and now Barcelona, as well as some non-KubeCon locations.
|
||||
We are constantly updating and improving the workshop content based on feedback
|
||||
from past sessions. This time, we’re breaking up the participants by their
|
||||
experience and comfort level with open source and Kubernetes. We’ll have
|
||||
developer setup and project workflow support for folks entirely new to open
|
||||
source and Kubernetes as part of the 101 track, and hope to set up each
|
||||
participant with their very own first issue to work on. In the 201 track, we
|
||||
will have a codebase walkthrough and local development and test demonstration
|
||||
for folks who have a bit more experience in open source but may be unfamiliar
|
||||
with our community’s development tools. For both tracks, you will have a chance
|
||||
to get your hands dirty and have some fun. Because not every contributor works
|
||||
with code, and not every contribution is technical, we will spend the beginning
|
||||
of the workshop learning how our project is structured and organized, where to
|
||||
find the right people, and where to get help when stuck.
|
||||
|
||||
|
||||
## Mentoring Opportunities
|
||||
|
||||
We will also bring back the SIG Meet-and-Greet where new contributors will have
|
||||
a chance to mingle with current contributors, perhaps find their dream SIG,
|
||||
learn what exciting areas they can help with, gain mentors, and make friends.
|
||||
|
||||
PS - there are also two mentoring sessions DURING KubeCon + CloudNativeCon on
|
||||
Thursday, May 23. [Sign up here][mentor]. 60% of the attendees during the
|
||||
Seattle event asked contributor questions.
|
||||
|
||||
## Past Attendee Story - Vallery Lancy, Engineer at Lyft
|
||||
|
||||
We talked to a few of our past participants in a series of interviews that we
|
||||
will publish throughout the course of the year. In our first two clips, we meet
|
||||
Vallery Lancy, an Engineer at Lyft and one of 75 attendees at our recent Seattle
|
||||
edition of the workshop. She was poking around in the community for a while to
|
||||
see where she could jump in.
|
||||
|
||||
Watch Vallery talk about her experience here:
|
||||
<center><iframe width="560" height="315" src="https://www.youtube.com/embed/uKg5WUcl6WU" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe></center>
|
||||
|
||||
|
||||
What does Vallery say to folks curious about the workshops, or those attending
|
||||
the Barcelona edition?
|
||||
|
||||
<center><iframe width="560" height="315" src="https://www.youtube.com/embed/niHiem7JmPA" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe></center>
|
||||
|
||||
|
||||
Be like Vallery and hundreds of previous New Contributor Workshop attendees:
|
||||
join us in Barcelona (or Shanghai - or San Diego!) for a unique experience
|
||||
without digging into our documentation! Have the opportunity to meet with the
|
||||
experts and go step by step into your journey with your peers around you. We’re
|
||||
looking forward to seeing you there! [Register here][ncw]
|
||||
|
||||
[eu]: https://events.linuxfoundation.org/events/contributor-summit-europe-2019/
|
||||
[cn]: https://www.lfasiallc.com/events/contributors-summit-china-2019/
|
||||
[ncw]: https://events.linuxfoundation.org/events/contributor-summit-europe-2019/
|
||||
[mentor]: http://bit.ly/mentor-bcn
|
||||
@@ -0,0 +1,126 @@
|
||||
---
|
||||
layout: blog
|
||||
title: 'Kyma - extend and build on Kubernetes with ease'
|
||||
date: 2019-05-23
|
||||
---
|
||||
|
||||
**Authors:** Lukasz Gornicki (SAP)
|
||||
|
||||
According to this recently completed [CNCF Survey](https://www.cncf.io/blog/2018/08/29/cncf-survey-use-of-cloud-native-technologies-in-production-has-grown-over-200-percent/), the adoption rate of Cloud Native technologies in production is growing rapidly. Kubernetes is at the heart of this technological revolution. Naturally, the growth of cloud native technologies has been accompanied by the growth of the ecosystem that surrounds it. Of course, the complexity of cloud native technologies have increased as well. Just google for the phrase “Kubernetes is hard”, and you’ll get plenty of articles that explain this complexity problem. The best thing about the CNCF community is that problems like this can be solved by smart people building new tools to enable Kubernetes users: Projects like Knative and its [Build resource](https://github.com/knative/build) extension, for example, serve to reduce complexity across a range of scenarios. Even though increasing complexity might seem like the most important issue to tackle, it is not the only challenge you face when transitioning to Cloud Native.
|
||||
|
||||
## Problems to solve
|
||||
|
||||
### Picking the right technology is hard
|
||||
|
||||
Now that you understand Kubernetes, your teams are trained and you’ve started building applications on top, it’s time to face a new layer of challenges. Cloud native doesn’t just mean deploying a platform for developers to build on top of. Developers also need storage, backup, monitoring, logging and a service mesh to enforce policies upon data in transit. Each of these individual systems must be properly configured and deployed, as well as logged, monitored and backed up on its own. The CNCF is here to help. We provide a [landscape](https://landscape.cncf.io/) overview of all cloud-native technologies, but the list is huge and can be overwhelming.
|
||||
|
||||
This is where [Kyma](http://kyma-project.io) will make your life easier. Its mission statement is to enable a flexible and easy way of extending applications.
|
||||
|
||||
<img src="/images/blog/2019-05-23-Kyma-extend-and-build-on-kubernetes-with-ease/kyma-center.png" width="40%" alt="Kyma in center" />
|
||||
|
||||
This project is designed to give you the tools you need to be able to write an end-to-end, production-grade cloud native application. [Kyma](https://github.com/kyma-project/kyma/) was donated to the open-source community by [SAP](https://www.sap.com); a company with great experience in writing production-grade cloud native applications. That’s why we’re so excited to -- [announce](https://twitter.com/kymaproject/status/1121426458243678209) the first major release of [Kyma 1.0](https://github.com/kyma-project/kyma/releases/tag/1.0.0)!
|
||||
|
||||
### Deciding on the path from monolith to cloud-native is hard
|
||||
|
||||
Try Googling `monolith to cloud native` or `monolith to microservices` and you’ll get a list of plenty of talks and papers that tackle this challenge. There are many different paths available for migrating a monolith to the cloud, and our experience has taught us to be quite opinionated in this area. First, let's answer the question of why you’d want to move from monolith to cloud native. The goals driving this move are typically:
|
||||
|
||||
- Increased scalability.
|
||||
- Faster implementation of new features.
|
||||
- More flexible approach to extensibility.
|
||||
|
||||
You do not have to rewrite your monolith to achieve these goals. Why spend all that time rewriting functionality that you already have? Just focus on enabling your monolith to support [event-driven architecture](https://en.wikipedia.org/wiki/Event-driven_architecture).
|
||||
|
||||
## How does Kyma solve your challenges?
|
||||
|
||||
### What is Kyma?
|
||||
|
||||
[Kyma](https://kyma-project.io/docs/root/kyma/#overview-overview) runs on Kubernetes and consists of a number of different components, three of which are:
|
||||
|
||||
- [Application connector](https://kyma-project.io/docs/components/application-connector/) that you can use to connect any application with a Kubernetes cluster and expose its APIs and Events through the [Kubernetes Service Catalog](https://github.com/kubernetes-incubator/service-catalog).
|
||||
- [Serverless](https://kyma-project.io/docs/components/serverless/) which enables you to easily write extensions for your application. Your function code can be triggered by API calls and also by events coming from external system. You can also securely call back the integrated system from your function.
|
||||
- [Service Catalog](https://kyma-project.io/docs/components/service-catalog/) is here to expose integrated systems. This integration also enables you to use services from hyperscalers like Azure, AWS or Google Cloud. [Kyma](https://kyma-project.io/docs/components/service-catalog/#service-brokers-service-brokers) allows for easy integration of official service brokers maintained by Microsoft and Google.
|
||||
|
||||
|
||||
|
||||
You can watch [this video](https://www.youtube.com/watch?v=wJzVWFGkiKk) for a short overview of Kyma key features that is based on a real demo scenario.
|
||||
|
||||
### We picked the right technologies for you
|
||||
|
||||
You can provide reliable extensibility in a project like Kyma only if it is properly monitored and configured. We decided not to reinvent the wheel. There are many great projects in the CNCF landscape, most with huge communities behind them. We decided to pick the best ones and glue them all together in Kyma. You can see the same architecture diagram that is above but with a focus on the projects we put together to create Kyma:
|
||||
|
||||
<img src="/images/blog/2019-05-23-Kyma-extend-and-build-on-kubernetes-with-ease/arch.png" width="70%" alt="Kyma architecture" />
|
||||
|
||||
- Monitoring and alerting is based on [Prometheus](https://prometheus.io/) and [Grafana](https://grafana.com/)
|
||||
- Logging is based on [Loki](https://grafana.com/loki)
|
||||
- Eventing uses [Knative](https://github.com/knative/eventing/) and [NATS](https://nats.io/)
|
||||
- Asset management uses [Minio](https://min.io/) as a storage
|
||||
- Service Mesh is based on [Istio](https://istio.io/)
|
||||
- Tracing is done with [Jaeger](https://www.jaegertracing.io/)
|
||||
- Authentication is supported by [dex](https://github.com/dexidp/dex)
|
||||
|
||||
You don't have to integrate these tools: We made sure they all play together well, and are always up to date ( Kyma is already using Istio 1.1). With our custom [Installer](https://github.com/kyma-project/kyma/tree/master/components/installer) and [Helm](https://helm.sh/) charts, we enabled easy installation and easy upgrades to new versions of Kyma.
|
||||
|
||||
### Do not rewrite your monoliths
|
||||
|
||||
Rewriting is hard, costs a fortune, and in most cases is not needed. At the end of the day, what you need is to be able to write and put new features into production quicker. You can do it by connecting your monolith to Kyma using the [Application Connector](https://kyma-project.io/docs/components/application-connector). In short, this component makes sure that:
|
||||
|
||||
- You can securely call back the registered monolith without the need to take care of authorization, as the Application Connector handles this.
|
||||
- Events sent from your monolith get securely to the Kyma Event Bus.
|
||||
|
||||
At the moment, your monolith can consume three different types of services: REST (with [OpenAPI](https://www.openapis.org/) specification) and OData (with Entity Data Model specification) for synchronous communication, and for asynchronous communication you can register a catalog of events based on [AsyncAPI](https://www.asyncapi.com/) specification. Your events are later delivered internally using [NATS Streaming](https://nats.io/) channel with [Knative eventing](https://github.com/knative/eventing/).
|
||||
|
||||
Once your monolith's services are connected, you can provision them in selected Namespaces thanks to the previously mentioned [Service Catalog](https://kyma-project.io/docs/components/service-catalog/) integration. You, as a developer, can go to the catalog and see a list of all the services you can consume. There are services from your monolith, and services from other 3rd party providers thanks to registered Service Brokers, like [Azure's OSBA](https://github.com/Azure/open-service-broker-azure). It is the one single place with everything you need. If you want to stand up a new application, everything you need is already available in Kyma.
|
||||
|
||||
### Finally some code
|
||||
|
||||
Check out some code I had to write to integrate a monolith with Azure services. I wanted to understand the sentiments shared by customers under the product's review section. On every event with a review comment, I wanted to use machine learning to call a sentiments analysis service, and in the case of a negative comment, I wanted to store it in a database for later review. This is the code of a function created thanks to our [Serverless](https://kyma-project.io/docs/components/serverless) component. Pay attention to my code comments:
|
||||
|
||||
> You can watch [this](https://www.youtube.com/watch?v=wJzVWFGkiKk) short video for a full demo of sentiment analysis function.
|
||||
|
||||
```js
|
||||
/* It is a function powered by NodeJS runtime so I have to import some necessary dependencies. I choosed Azure's CosmoDB that is a Mongo-like database, so I could use a MongoClient */
|
||||
const axios = require("axios");
|
||||
const MongoClient = require('mongodb').MongoClient;
|
||||
|
||||
module.exports = { main: async function (event, context) {
|
||||
/* My function was triggered because it was subscribed to customer review event. I have access to the payload of the event. */
|
||||
let negative = await isNegative(event.data.comment)
|
||||
|
||||
if (negative) {
|
||||
console.log("Customer sentiment is negative:", event.data)
|
||||
await mongoInsert(event.data)
|
||||
} else {
|
||||
console.log("This positive comment was not saved:", event.data)
|
||||
}
|
||||
}}
|
||||
|
||||
/* Like in case of isNegative function, I focus of usage of the MongoClient API. The necessary information about the database location and an authorization needed to call it is injected into my function and I just need to pick a proper environment variable. */
|
||||
async function mongoInsert(data) {
|
||||
|
||||
try {
|
||||
client = await MongoClient.connect(process.env.connectionString, { useNewUrlParser: true });
|
||||
db = client.db('mycommerce');
|
||||
const collection = db.collection('comments');
|
||||
return await collection.insertOne(data);
|
||||
} finally {
|
||||
client.close();
|
||||
}
|
||||
}
|
||||
/* This function calls Azure's Text Analytics service to get information about the sentiment. Notice process.env.textAnalyticsEndpoint and process.env.textAnalyticsKey part. When I wrote this function I didn't have to go to Azure's console to get these details. I had these variables automatically injected into my function thanks to our integration with Service Catalog and our Service Binding Usage controller that pairs the binding with a function. */
|
||||
async function isNegative(comment) {
|
||||
let response = await axios.post(`${process.env.textAnalyticsEndpoint}/sentiment`,
|
||||
{ documents: [{ id: '1', text: comment }] }, {headers:{ 'Ocp-Apim-Subscription-Key': process.env.textAnalyticsKey }})
|
||||
return response.data.documents[0].score < 0.5
|
||||
}
|
||||
```
|
||||
Thanks to Kyma, I don't have to worry about the infrastructure around my function. As I mentioned, I have all the tools needed in Kyma, and they are integrated together. I can quickly get access to my logs through [Loki](https://grafana.com/loki), and I can quickly get access to a preconfigured Grafana dashboard to see the metrics of my Lambda delivered thanks to [Prometheus](https://prometheus.io/) and [Istio](https://istio.io/).
|
||||
|
||||
<img src="/images/blog/2019-05-23-Kyma-extend-and-build-on-kubernetes-with-ease/grafana-lambda.png" width="70%" alt="Grafana with preconfigured lambda dashboard" />
|
||||
|
||||
Such an approach gives you a lot of flexibility in adding new functionality. It also gives you time to rethink the need to rewrite old functions.
|
||||
|
||||
## Contribute and give feedback
|
||||
|
||||
Kyma is an open source project, and we would love help it grow. The way that happens is with your help. After reading this post, you already know that we don't want to reinvent the wheel. We stay true to this approach in our work model, which enables community contributors. We work in [Special Interest Groups](
|
||||
https://github.com/kyma-project/community/tree/master/sig-and-wg) and have publicly recorded meeting that you can join any time, so we have a setup similar to what you know from Kubernetes itself.
|
||||
Feel free to share also your feedback with us, through [Twitter](https://twitter.com/kymaproject) or [Slack](http://slack.kyma-project.io).
|
||||
@@ -0,0 +1,48 @@
|
||||
---
|
||||
layout: blog
|
||||
title: 'Join us at the Contributor Summit in Shanghai'
|
||||
date: 2019-06-12
|
||||
---
|
||||
|
||||
**Author**: Josh Berkus (Red Hat)
|
||||
|
||||
|
||||
|
||||
For the second year, we will have [a Contributor Summit event](https://www.lfasiallc.com/events/contributors-summit-china-2019/) the day before [KubeCon China](https://events.linuxfoundation.cn/events/kubecon-cloudnativecon-china-2019/) in Shanghai. If you already contribute to Kubernetes or would like to contribute, please consider attending and [register](https://www.lfasiallc.com/events/contributors-summit-china-2019/register/). The Summit will be held June 24th, at the Shanghai Expo Center (the same location where KubeCon will take place), and will include a Current Contributor Day as well as the New Contributor Workshop and the Documentation Sprints.
|
||||
|
||||
### Current Contributor Day
|
||||
|
||||
After last year's Contributor Day, our team received feedback that many of our contributors in Asia and Oceania would like content for current contributors as well. As such, we have added a Current Contributor track to the schedule.
|
||||
|
||||
While we do not yet have a full schedule up, the topics covered in the current contributor track will include:
|
||||
|
||||
* How to write a KEP (Kubernetes Enhancement Proposal)
|
||||
* Codebase and repository review
|
||||
* Local Build & Test troubleshooting session
|
||||
* Guide to Non-Code Contribution opportunities
|
||||
* SIG-Azure face-to-face meeting
|
||||
* SIG-Scheduling face-to-face meeting
|
||||
* Other SIG face-to-face meetings as we confirm them
|
||||
|
||||
The schedule will be on [the Community page](https://github.com/kubernetes/community/tree/master/events/2019/06-contributor-summit) once it is complete.
|
||||
|
||||
If your SIG wants to have a face-to-face meeting at Kubecon Shanghai, please contact [Josh Berkus](mailto:jberkus@redhat.com).
|
||||
|
||||
### New Contributor Workshop
|
||||
|
||||
Students at [last year's New Contributor Workshop](/blog/2018/12/05/new-contributor-workshop-shanghai/) (NCW) found it to be extremely valuable, and the event helped to orient a few of the many Asian and Pacific developers looking to participate in the Kubernetes community.
|
||||
|
||||
> "It's a one-stop-shop for becoming familiar with the community." said one participant.
|
||||
|
||||
If you have not contributed to Kubernetes before, or have only done one or two things, please consider [enrolling](https://www.lfasiallc.com/events/contributors-summit-china-2019/register/) in the NCW.
|
||||
|
||||
> "Got to know the process from signing CLA to PR and made friends with other contributors." said another.
|
||||
|
||||
### Documentation Sprints
|
||||
|
||||
Both old and new contributors on our Docs Team will spend a day both improving our documentation and translating it into other languages. If you are interested in having better documentation, fully localized into Chinese and other languages, please [sign up](https://www.lfasiallc.com/events/contributors-summit-china-2019/register/) to help with the Doc Sprints.
|
||||
|
||||
### Before you attend
|
||||
|
||||
Regardless of where you participate, everyone at the Contributor Summit should [sign the Kubernetes Contributor License Agreement](https://git.k8s.io/community/CLA.md#the-contributor-license-agreement) (CLA) before coming to the conference. You should also bring a laptop suitable for working on documentation or code development.
|
||||
+48
@@ -0,0 +1,48 @@
|
||||
---
|
||||
title: "Introducing kube-iptables-tailer: Better Networking Visibility in Kubernetes Clusters"
|
||||
date: 2019-04-19
|
||||
slug: introducing-kube-iptables-tailer
|
||||
---
|
||||
|
||||
**Authors:** Saifuding Diliyaer, Software Engineer, Box
|
||||
|
||||
At Box, we use Kubernetes to empower our engineers to own the whole lifecycle of their microservices. When it comes to networking, our engineers use Tigera’s [Project Calico](https://www.tigera.io/tigera-calico/) to declaratively manage network policies for their apps running in our Kubernetes clusters. App owners define a Calico policy in order to enable their Pods to send/receive network traffic, which is instantiated as iptables rules.
|
||||
|
||||
There may be times, however, when such network policy is missing or declared incorrectly by app owners. In this situation, the iptables rules will cause network packet drops between the affected Pods, which get logged in a file that is inaccessible to app owners. We needed a mechanism to seamlessly deliver alerts about those iptables packet drops based on their network policies to help app owners quickly diagnose the corresponding issues. To solve this, we developed a service called [kube-iptables-tailer](https://github.com/box/kube-iptables-tailer) to detect packet drops from iptables logs and report them as Kubernetes events. We are proud to open-source kube-iptables-tailer for you to utilize in your own cluster, regardless of whether you use Calico or other network policy tools.
|
||||
|
||||
## Improved Experience for App Owners
|
||||
App owners do not have to apply any additional changes to utilize kube-iptables-tailer. They can simply run `kubectl describe pods` to check if any of their Pods' traffic has been dropped due to iptables rules. All the results sent from kube-iptables-tailer will be shown under the *Events* section, which is a much better experience for developers when compared to reading through raw iptables logs.
|
||||
|
||||
```shell
|
||||
$ kubectl describe pods --namespace=YOUR_NAMESPACE
|
||||
|
||||
...
|
||||
Events:
|
||||
Type Reason Age From Message
|
||||
---- ------ ---- ---- -------
|
||||
Warning PacketDrop 5s kube-iptables-tailer Packet dropped when receiving traffic from example-service-2 (IP: 22.222.22.222).
|
||||
|
||||
Warning PacketDrop 10m kube-iptables-tailer Packet dropped when sending traffic to example-service-1 (IP: 11.111.11.111).
|
||||
```
|
||||
*\* output of events sent from kube-iptables-tailer to Kubernetes Pods having networking issues*
|
||||
|
||||
|
||||
## Process behind kube-iptables-tailer
|
||||
Before we had kube-iptables-tailer, the only way for Box’s engineers to get information about packet drops related to their network policies was parsing through the raw iptables logs and matching their service IPs. This was a suboptimal experience because iptables logs only contain basic IP address information. Mapping these IPs to specific Pods could be painful, especially in the Kubernetes world where Pods and containers are ephemeral and IPs are frequently changing. This process involved a bunch of manual commands for our engineers. Additionally, iptables logs could be noisy due to a number of drops, and if IP addresses were being reused, the app owners might even have some stale data. With the help of kube-iptables-tailer, life now becomes much easier for our developers. As shown in the following diagram, the principle of this service can be divided into three steps:
|
||||
|
||||
|
||||
*\* sequence diagram for kube-iptables-tailer*
|
||||
|
||||
### 1. Watch changes on iptables log file
|
||||
Instead of requiring human engineers to manually decipher the raw iptables logs, we now use kube-iptables-tailer to help identify changes in that file. We run the service as a **DaemonSet** on every host node in our cluster, and it tails the iptables log file periodically. The service itself is written in Go, and it has multiple goroutines for the different service components running concurrently. We use channels to share information among those various components. In this step, for instance, the service will send out any changes it detected in iptables log file to a Go channel to be parsed later.
|
||||
|
||||
### 2. Parse iptables logs based on log prefix
|
||||
Once the parser receives a new log message through a particular Go channel, it will first check whether the log message includes any network policy related packet drop information by parsing the log prefix. Packet drops based on our Calico policies will be logged containing “calico-drop:” as the log prefix in iptables log file. In this case, an object will be created by the parser with the data from the log message being stored as the object’s fields. These handy objects will be later used to locate the relevant Pods running in Kubernetes and post notifications directly to them. The parser is also able to identify duplicate logs and filter them to avoid causing confusion and consuming extra resources. After the parsing process, it will come to the final step for kube-iptables-tailer to send out the results.
|
||||
|
||||
### 3. Locate pods and send out events
|
||||
Using the Kubernetes API, kube-iptables-tailer will try locating both senders and receivers in our cluster by matching the IPs stored in objects parsed from the previous step. As a result, an event will be posted to these affected Pods if they are located successfully. Kubernetes events are objects designed to provide information about what is happening inside a Kubernetes component. At Box, one of the use cases for Kubernetes events is to report errors directly to the corresponding applications (for more details, please refer to this [blog post](https://kubernetes.io/blog/2018/01/reporting-errors-using-kubernetes-events/)). The event generated by kube-iptables-tailer includes useful information such as traffic direction, IPs and the namespace of Pods from the other side. We have added DNS lookup as well because our Pods also send and receive traffic from services running on bare-metal hosts and VMs. Besides, exponential backoff is implemented to avoid overwhelming the Kubernetes API server.
|
||||
|
||||
## Summary
|
||||
At Box, kube-iptables-tailer has saved time as well as made life happier for many developers across various teams. Instead of flying blind with regards to packet drops based on network policies, the service is able to help detect changes in iptables log file and get the corresponding information delivered right to the Pods inside Kubernetes clusters. If you’re not using Calico, you can still apply any other log prefix (configured as an environment variable in the service) to match whatever is defined in your iptables rules and get notified about the network policy related packet drops. You may also find other cases where it is useful to make information from host systems available to Pods via the Kubernetes API. As an open-sourced project, every contribution is more than welcome to help improve the project together. You can find this project hosted on Github at https://github.com/box/kube-iptables-tailer
|
||||
|
||||
*Special thanks to [Kunal Parmar](https://www.linkedin.com/in/kunalparmar/), [Greg Lyons](https://www.linkedin.com/in/greg-lyons-8277a188/) and [Shrenik Dedhia](https://www.linkedin.com/in/shrenikd/) for contributing to this project.*
|
||||
@@ -0,0 +1,54 @@
|
||||
---
|
||||
title: 'Kubernetes, Cloud Native, and the Future of Software'
|
||||
date: 2019-05-17
|
||||
---
|
||||
|
||||
**Authors:** Brian Grant (Google), Jaice Singer DuMars (Google)
|
||||
|
||||
# Kubernetes, Cloud Native, and the Future of Software
|
||||
|
||||
Five years ago this June, Google Cloud announced a new application management technology called Kubernetes. It began with a [simple open source commit](https://github.com/kubernetes/kubernetes/commit/2c4b3a562ce34cddc3f8218a2c4d11c7310e6d56), followed the next day by a [one-paragraph blog mention](https://cloudplatform.googleblog.com/2014/06/an-update-on-container-support-on-google-cloud-platform.html) around container support. Later in the week, Eric Brewer [talked about Kubernetes for the first time](https://www.youtube.com/watch?v=YrxnVKZeqK8) at DockerCon. And soon the world was watching.
|
||||
|
||||
We’re delighted to see Kubernetes become core to the creation and operation of modern software, and thereby a key part of the global economy. To us, the success of Kubernetes represents even more: A business transition with truly worldwide implications, thanks to the unprecedented cooperation afforded by the open source software movement.
|
||||
|
||||
Like any important technology, Kubernetes has become about more than just itself; it has positively affected the environment in which it arose, changing how software is deployed at scale, how work is done, and how corporations engage with big open-source projects.
|
||||
|
||||
Let’s take a look at how this happened, since it tells us a lot about where we are today, and what might be happening next.
|
||||
|
||||
**Beginnings**
|
||||
|
||||
The most important precursor to Kubernetes was the rise of application containers. Docker, the first tool to really make containers usable by a broad audience, began as an open source project in 2013. By containerizing an application, developers could achieve easier language runtime management, deployment, and scalability. This triggered a sea change in the application ecosystem. Containers made stateless applications easily scalable and provided an immutable deployment artifact that drastically reduced the number of variables previously encountered between test and production systems.
|
||||
|
||||
While containers presented strong stand-alone value for developers, the next challenge was how to deliver and manage services, applications, and architectures that spanned multiple containers and multiple hosts.
|
||||
|
||||
Google had already encountered similar issues within its own IT infrastructure. Running the world’s most popular search engine (and several other products with millions of users) lead to early innovation around, and adoption of, containers. Kubernetes was inspired by Borg, Google’s internal platform for scheduling and managing the hundreds of millions, and eventually billions, of containers that implement all of our services.
|
||||
|
||||
Kubernetes is more than just “Borg, for everyone” It distills the most successful architectural and API patterns of prior systems and couples them with load balancing, authorization policies, and other features needed to run and manage applications at scale. This in turn provides the groundwork for cluster-wide abstractions that allow true portability across clouds.
|
||||
|
||||
The November 2014 [alpha launch](https://cloudplatform.googleblog.com/2014/11/google-cloud-platform-live-introducing-container-engine-cloud-networking-and-much-more.html) of Google Cloud’s [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine/) introduced managed Kubernetes. There was an explosion of innovation around Kubernetes, and companies from the enterprise down to the startup saw barriers to adoption fall away. Google, Red Hat, and others in the community increased their investment of people, experience, and architectural know-how to ensure it was ready for increasingly mission-critical workloads. The response was a wave of adoption that swept it to the forefront of the crowded container management space.
|
||||
|
||||
**The Rise of Cloud Native**
|
||||
|
||||
Every enterprise, regardless of its core business, is embracing more digital technology. The ability to rapidly adapt is fundamental to continued growth and competitiveness. Cloud-native technologies, and especially Kubernetes, arose to meet this need, providing the automation and observability necessary to manage applications at scale and with high velocity. Organizations previously constrained to quarterly deployments of critical applications can now deploy safely multiple times a day.
|
||||
|
||||
Kubernetes’s declarative, API-driven infrastructure empowers teams to operate independently, and enables them to focus on their business objectives. An inevitable cultural shift in the workplace has come from enabling greater autonomy and productivity and reducing the toil of development teams.
|
||||
|
||||
**Increased engagement with open source**
|
||||
|
||||
The ability for teams to rapidly develop and deploy new software creates a virtuous cycle of success for companies and technical practitioners alike. Companies have started to recognize that contributing back to the software projects they use not only improves the performance of the software for their use cases, but also builds critical skills and creates challenging opportunities that help them attract and retain new developers.
|
||||
|
||||
The Kubernetes project in particular curates a collaborative culture that encourages contribution and sharing of learning and development with the community. This fosters a positive-sum ecosystem that benefits both contributors and end-users equally.
|
||||
|
||||
**What’s Next?**
|
||||
|
||||
Where Kubernetes is concerned, five years seems like an eternity. That says much about the collective innovation we’ve seen in the community, and the rapid adoption of the technology.
|
||||
|
||||
In other ways, it is just the start. New applications such as machine learning, edge computing, and the Internet of Things are finding their way into the cloud native ecosystem via projects like Kubeflow. Kubernetes is almost certain to be at the heart of their success.
|
||||
|
||||
Kubernetes may be most successful if it becomes an invisible essential of daily life, like urban plumbing or electrical grids. True standards are dramatic, but they are also taken for granted. As Googler and KubeCon co-chair Janet Kuo said in a [recent keynote](https://www.youtube.com/watch?v=LAO7RuWwfzA), Kubernetes is going to become boring, and that’s a good thing, at least for the majority of people who don’t have to care about container management.
|
||||
|
||||
At Google Cloud, we’re still excited about the project, and we go to work on it every day. Yet it’s all of the solutions and extensions that expand from Kubernetes that will dramatically change the world as we know it.
|
||||
|
||||
So, as we all celebrate the continued success of Kubernetes, remember to take the time and thank someone you see helping make the community better. It’s up to all of us to foster a cloud-native ecosystem that prizes the efforts of everyone who helps maintain and nurture the work we do together.
|
||||
|
||||
And, to everyone who has been a part of the global success of Kubernetes, thank you. You have changed the world.
|
||||
@@ -47,6 +47,9 @@ Vagrant.configure("2") do |config|
|
||||
master.vm.hostname = "k8s-master"
|
||||
master.vm.provision "ansible" do |ansible|
|
||||
ansible.playbook = "kubernetes-setup/master-playbook.yml"
|
||||
ansible.extra_vars = {
|
||||
node_ip: "192.168.50.10",
|
||||
}
|
||||
end
|
||||
end
|
||||
|
||||
@@ -57,6 +60,9 @@ Vagrant.configure("2") do |config|
|
||||
node.vm.hostname = "node-#{i}"
|
||||
node.vm.provision "ansible" do |ansible|
|
||||
ansible.playbook = "kubernetes-setup/node-playbook.yml"
|
||||
ansible.extra_vars = {
|
||||
node_ip: "192.168.50.#{i + 10}",
|
||||
}
|
||||
end
|
||||
end
|
||||
end
|
||||
@@ -162,6 +168,17 @@ We will be installing the following packages, and then adding a user named “va
|
||||
- kubelet
|
||||
- kubeadm
|
||||
- kubectl
|
||||
|
||||
- name: Configure node ip
|
||||
lineinfile:
|
||||
path: /etc/default/kubelet
|
||||
line: KUBELET_EXTRA_ARGS=--node-ip={{ node_ip }}
|
||||
|
||||
- name: Restart kubelet
|
||||
service:
|
||||
name: kubelet
|
||||
daemon_reload: yes
|
||||
state: restarted
|
||||
```
|
||||
|
||||
#### Step 2.3: Initialize the Kubernetes cluster with kubeadm using the below code (applicable only on master node).
|
||||
|
||||
@@ -0,0 +1,36 @@
|
||||
---
|
||||
title: 'The Future of Cloud Providers in Kubernetes'
|
||||
date: 2019-04-17
|
||||
---
|
||||
|
||||
**Authors:** Andrew Sy Kim (VMware), Mike Crute (AWS), Walter Fender (Google)
|
||||
|
||||
Approximately 9 months ago, the Kubernetes community agreed to form the Cloud Provider Special Interest Group (SIG). The justification was to have a single governing SIG to own and shape the integration points between Kubernetes and the many cloud providers it supported. A lot has been in motion since then and we’re here to share with you what has been accomplished so far and what we hope to see in the future.
|
||||
|
||||
## The Mission
|
||||
|
||||
First and foremost, I want to share what the mission of the SIG is, because we use it to guide our present & future work. Taken straight from our [charter](https://github.com/kubernetes/community/blob/master/sig-cloud-provider/CHARTER.md), the mission of the SIG is to simplify, develop and maintain cloud provider integrations as extensions, or add-ons, to Kubernetes clusters. The motivation behind this is two-fold: to ensure Kubernetes remains extensible and cloud agnostic.
|
||||
|
||||
## The Current State of Cloud Providers
|
||||
|
||||
In order to gain a forward looking perspective to our work, I think it’s important to take a step back to look at the current state of cloud providers. Today, each core Kubernetes component (except the scheduler and kube-proxy) has a --cloud-provider flag you can configure to enable a set of functionalities that integrate with the underlying infrastructure provider, a.k.a the cloud provider. Enabling this integration unlocks a wide set of features for your clusters such as: node address & zone discovery, cloud load balancers for Services with Type=LoadBalancer, IP address management, and cluster networking via VPC routing tables. Today, the cloud provider integrations can be done either in-tree or out-of-tree.
|
||||
|
||||
## In-Tree & Out-of-Tree Providers
|
||||
|
||||
In-tree cloud providers are the providers we develop & release in the [main Kubernetes repository](https://github.com/kubernetes/kubernetes/tree/master/pkg/cloudprovider/providers). This results in embedding the knowledge and context of each cloud provider into most of the Kubernetes components. This enables more native integrations such as the kubelet requesting information about itself via a metadata service from the cloud provider.
|
||||
|
||||
<center>{{<figure width="600" src="/images/docs/pre-ccm-arch.png" caption="In-Tree Cloud Provider Architecture (source: kubernetes.io)">}}</center>
|
||||
|
||||
Out-of-tree cloud providers are providers that can be developed, built, and released independent of Kubernetes core. This requires deploying a new component called the cloud-controller-manager which is responsible for running all the cloud specific controllers that were previously run in the kube-controller-manager.
|
||||
|
||||
<center>{{<figure width="600" src="/images/docs/post-ccm-arch.png" caption="Out-of-Tree Cloud Provider Architecture (source: kubernetes.io)">}}</center>
|
||||
|
||||
|
||||
When cloud provider integrations were initially developed, they were developed natively (in-tree). We integrated each provider close to the core of Kubernetes and within the monolithic repository that is k8s.io/kubernetes today. As Kubernetes became more ubiquitous and more infrastructure providers wanted to support Kubernetes natively, we realized that this model was not going to scale. Each provider brings along a large set of dependencies which increases potential vulnerabilities in our code base and significantly increases the binary size of each component. In addition to this, more of the Kubernetes release notes started to focus on provider specific changes rather than core changes that impacted all Kubernetes users.
|
||||
|
||||
In late 2017, we developed a way for cloud providers to build integrations without adding them to the main Kubernetes tree (out-of-tree). This became the de-facto way for new infrastructure providers in the ecosystem to integrate with Kubernetes. Since then, we’ve been actively working towards migrating all cloud providers to use the out-of-tree architecture as most clusters today are still using the in-tree cloud providers.
|
||||
|
||||
## Looking Ahead
|
||||
|
||||
Looking ahead, the goal of the SIG is to remove all existing in-tree cloud providers in favor of their out-of-tree equivalents with minimal impact to users. In addition to the core cloud provider integration mentioned above, there are more extension points for cloud integrations like CSI and the image credential provider that are actively being worked on for v1.15. Getting to this point would mean that Kubernetes is truly cloud-agnostic with no native integrations for any cloud provider. By doing this work we empower each cloud provider to develop and release new versions at their own cadence independent of Kubernetes. We’ve learned by now that this is a large feat with a unique set of challenges. Migrating workloads is never easy, especially when it’s an essential part of the control plane. Providing a safe and easy migration path between in-tree and out-of-tree cloud providers is of the highest priority for our SIG in the upcoming releases. If any of this sounds interesting to you, I encourage you to check out of some of our [KEPs](https://github.com/kubernetes/enhancements/tree/master/keps/sig-cloud-provider) and get in touch with our SIG by joining the [mailing list](https://groups.google.com/forum/#!forum/kubernetes-sig-cloud-provider) or our slack channel (#sig-cloud-provider in Kubernetes slack).
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 10 KiB |
@@ -0,0 +1,96 @@
|
||||
---
|
||||
title: Ant Financial Case Study
|
||||
linkTitle: ant-financial
|
||||
case_study_styles: true
|
||||
cid: caseStudies
|
||||
css: /css/style_case_studies.css
|
||||
featured: false
|
||||
---
|
||||
|
||||
<div class="banner1" style="background-image: url('/images/CaseStudy_antfinancial_banner1.jpg')">
|
||||
<h1> CASE STUDY:<img src="/images/antfinancial_logo.png" class="header_logo" style="width:20%;margin-bottom:-2.5%"><br> <div class="subhead" style="margin-top:1%">Ant Financial’s Hypergrowth Strategy Using Kubernetes
|
||||
|
||||
</div></h1>
|
||||
|
||||
</div>
|
||||
|
||||
<div class="details">
|
||||
Company <b>Ant Financial</b> Location <b>Hangzhou, China</b> Industry <b>Financial Services</b>
|
||||
</div>
|
||||
|
||||
<hr>
|
||||
<section class="section1">
|
||||
<div class="cols">
|
||||
<div class="col1" style="width:100%"">
|
||||
<h2>Challenge</h2>
|
||||
Officially founded in October 2014, <a href="https://www.antfin.com/index.htm?locale=en_us">Ant Financial</a> originated from <a href="https://global.alipay.com/">Alipay</a>, the world’s largest online payment platform that launched in 2004. The company also offers numerous other services leveraging technology innovation. With the volume of transactions Alipay handles for its 900+ million users worldwide (through its local and global partners)—256,000 transactions per second at the peak of Double 11 Singles Day 2017, and total gross merchandise value of $31 billion for Singles Day 2018—not to mention that of its other services, Ant Financial faces “data processing challenge in a whole new way,” says Haojie Hang, who is responsible for Product Management for the Storage and Compute Group. “We see three major problems of operating at that scale: how to provide real-time compute, storage, and processing capability, for instance to make real-time recommendations for fraud detection; how to provide intelligence on top of this data, because there’s too much data and then we’re not getting enough insight; and how to apply security in the application level, in the middleware level, the system level, even the chip level.” In order to provide reliable and consistent services to its customers, Ant Financial embraced containers in early 2014, and soon needed an orchestration solution for the tens-of-thousands-of-node clusters in its data centers.
|
||||
|
||||
<h2>Solution</h2>
|
||||
After investigating several technologies, the team chose <a href="https://kubernetes.io/">Kubernetes</a> for orchestration, as well as a number of other CNCF projects, including <a href="https://prometheus.io/">Prometheus</a>, <a href="https://opentracing.io/">OpenTracing</a>, <a href="https://coreos.com/etcd/">etcd</a> and <a href="https://coredns.io/">CoreDNS</a>. “In late 2016, we decided that Kubernetes will be the de facto standard,” says Hang. “Looking back, we made the right bet on the right technology. But then we needed to move the production workload from the legacy infrastructure to the latest Kubernetes-enabled platform, and that took some time, because we are very careful in terms of reliability and consistency.” All core financial systems were containerized by November 2017, and the migration to Kubernetes is ongoing.
|
||||
<br>
|
||||
<h2>Impact</h2>
|
||||
“We’ve seen at least tenfold in improvement in terms of the operations with cloud native technology, which means you can have tenfold increase in terms of output,” says Hang. Ant also provides its fully integrated financial cloud platform to business partners around the world, and hopes to power the next generation of digital banking with deep experience in service innovation and technology expertise. Hang says the team hasn’t begun to focus on optimizing the Kubernetes platform, either: “Because we’re still in the hyper growth stage, we’re not in a mode where we do cost saving yet.”
|
||||
</div>
|
||||
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner2">
|
||||
<div class="banner2text">
|
||||
"In late 2016, we decided that Kubernetes will be the de facto standard. Looking back, we made the right bet on the right technology."
|
||||
<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- HAOJIE HANG, PRODUCT MANAGEMENT, ANT FINANCIAL</span>
|
||||
</div>
|
||||
</div>
|
||||
<section class="section2">
|
||||
<div class="fullcol">
|
||||
<h2>A spinoff of the multinational conglomerate Alibaba, Ant Financial boasts a $150+ billion valuation and the scale to match. The fintech startup, launched in 2014, is comprised of Alipay, the world’s largest online payment platform, and numerous other services leveraging technology innovation.</h2>
|
||||
And the volume of transactions that Alipay handles for over 900 million users worldwide (through its local and global partners) is staggering: 256,000 per second at the peak of Double 11 Singles Day 2017, and total gross merchandise value of $31 billion for Singles Day 2018. With the mission of “bringing the world equal opportunities,” Ant Financial is dedicated to creating an open, shared credit system and financial services platform through technology innovations.
|
||||
<br><br>
|
||||
Combine that with the operations of its other properties—such as the Huabei online credit system, Jiebei lending service, and the 350-million-user <a href="https://en.wikipedia.org/wiki/Ant_Forest">Ant Forest</a> green energy mobile app—and Ant Financial faces “data processing challenge in a whole new way,” says Haojie Hang, who is responsible for Product Management for the Storage and Compute Group. “We see three major problems of operating at that scale: how to provide real-time compute, storage, and processing capability, for instance to make real-time recommendations for fraud detection; how to provide intelligence on top of this data, because there’s too much data and we’re not getting enough insight; and how to apply security in the application level, in the middleware level, the system level, even the chip level.”
|
||||
<br><br>
|
||||
To address those challenges and provide reliable and consistent services to its customers, Ant Financial embraced <a href="https://www.docker.com/">Docker</a> containerization in 2014. But they soon realized that they needed an orchestration solution for some tens-of-thousands-of-node clusters in the company’s data centers.
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner3" style="background-image: url('/images/CaseStudy_antfinancial_banner3.jpg')">
|
||||
<div class="banner3text">
|
||||
"On Double 11 this year, we had plenty of nodes on Kubernetes, but compared to the whole scale of our infrastructure, this is still in progress."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- RANGER YU, GLOBAL TECHNOLOGY PARTNERSHIP & DEVELOPMENT, ANT FINANCIAL</span>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
<section class="section3">
|
||||
<div class="fullcol">
|
||||
The team investigated several technologies, including Docker Swarm and Mesos. “We did a lot of POCs, but we’re very careful in terms of production systems, because we want to make sure we don’t lose any data,” says Hang. “You cannot afford to have a service downtime for one minute; even one second has a very, very big impact. We operate every day under pressure to provide reliable and consistent services to consumers and businesses in China and globally.”
|
||||
<br><br>
|
||||
Ultimately, Hang says Ant chose Kubernetes because it checked all the boxes: a strong community, technology that “will be relevant in the next three to five years,” and a good match for the company’s engineering talent. “In late 2016, we decided that Kubernetes will be the de facto standard,” says Hang. “Looking back, we made the right bet on the right technology. But then we needed to move the production workload from the legacy infrastructure to the latest Kubernetes-enabled platform. We spent a lot of time learning and then training our people to build applications on Kubernetes well.”
|
||||
<br><br>
|
||||
All core financial systems were containerized by November 2017, and the migration to Kubernetes is ongoing. Ant’s platform also leverages a number of other CNCF projects, including <a href="https://prometheus.io/">Prometheus</a>, <a href="https://opentracing.io/">OpenTracing</a>, <a href="https://coreos.com/etcd/">etcd</a> and <a href="https://coredns.io/">CoreDNS</a>. “On Double 11 this year, we had plenty of nodes on Kubernetes, but compared to the whole scale of our infrastructure, this is still in progress,” says Ranger Yu, Global Technology Partnership & Development.
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner4" style="background-image: url('/images/CaseStudy_antfinancial_banner4.jpg')">
|
||||
<div class="banner4text">
|
||||
"We’re very grateful for CNCF and this amazing technology, which we need as we continue to scale globally. We’re definitely embracing the community and open source more in the future." <br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- HAOJIE HANG, PRODUCT MANAGEMENT, ANT FINANCIAL</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<section class="section5" style="padding:0px !important">
|
||||
<div class="fullcol">
|
||||
Still, there has already been an impact. “Cloud native technology has benefited us greatly in terms of efficiency,” says Hang. “In general, we want to make sure our infrastructure is nimble and flexible enough for the work that could happen tomorrow. That’s the goal. And with cloud native technology, we’ve seen at least tenfold improvement in operations, which means you can have tenfold increase in terms of output. Let’s say you are operating 10 nodes with one person. With cloud native, tomorrow you can have 100 nodes.”
|
||||
<br><br>
|
||||
Ant also provides its financial cloud platform to partners around the world, and hopes to power the next generation of digital banking with deep experience in service innovation and technology expertise. Hang says the team hasn’t begun to focus on optimizing the Kubernetes platform, either: “Because we’re still in the hyper growth stage, we’re not in a mode where we do cost-saving yet.”
|
||||
<br><br>
|
||||
The CNCF community has also been a valuable asset during Ant Financial’s move to cloud native. “If you are applying a new technology, it’s very good to have a community to discuss technical problems with other users,” says Hang. “We’re very grateful for CNCF and this amazing technology, which we need as we continue to scale globally. We’re definitely embracing the community and open sourcing more in the future.”
|
||||
</div>
|
||||
|
||||
<div class="banner5" >
|
||||
<div class="banner5text">
|
||||
"In China, we are the North Star in terms of innovation in financial and other related services,” says Hang. “We definitely want to make sure we’re still leading in the next 5 to 10 years with our investment in technology."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- RANGER YU, GLOBAL TECHNOLOGY PARTNERSHIP & DEVELOPMENT, ANT FINANCIAL</span></div>
|
||||
</div>
|
||||
|
||||
<div class="fullcol">
|
||||
In fact, the company has already started to open source some of its <a href="https://github.com/alipay">cloud native middleware</a>. “We are going to be very proactive about that,” says Yu. “CNCF provided a platform so everyone can plug in or contribute components. This is very good open source governance.”
|
||||
<br><br>
|
||||
Looking ahead, the Ant team will continue to evaluate many other CNCF projects. Building a service mesh community in China, the team has brought together many China-based companies and developers to discuss the potential of that technology. “Service mesh is very attractive for Chinese developers and end users because we have a lot of legacy systems running now, and it’s an ideal mid-layer to glue everything together, both new and legacy,” says Hang. “For new technologies, we look very closely at whether they will last.”
|
||||
<br><br>
|
||||
At Ant, Kubernetes passed that test with flying colors, and the team hopes other companies will follow suit. “In China, we are the North Star in terms of innovation in financial and other related services,” says Hang. “We definitely want to make sure we’re still leading in the next 5 to 10 years with our investment in technology.”
|
||||
|
||||
</div>
|
||||
</section>
|
||||
@@ -1,11 +1,17 @@
|
||||
---
|
||||
title: AppDirect Case Study
|
||||
|
||||
linkTitle: AppDirect
|
||||
case_study_styles: true
|
||||
cid: caseStudies
|
||||
css: /css/style_case_studies.css
|
||||
logo: appdirect_featured_logo.png
|
||||
featured: true
|
||||
weight: 4
|
||||
quote: >
|
||||
We made the right decisions at the right time. Kubernetes and the cloud native technologies are now seen as the de facto ecosystem.
|
||||
---
|
||||
|
||||
|
||||
<div class="banner1" style="background-image: url('/images/CaseStudy_appdirect_banner1.jpg')">
|
||||
<h1> CASE STUDY:<img src="/images/appdirect_logo.png" class="header_logo" style="margin-bottom:-2%"><br> <div class="subhead" style="margin-top:1%;font-size:0.5em">AppDirect: How AppDirect Supported the 10x Growth of Its Engineering Staff with Kubernetess
|
||||
</div></h1>
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 3.5 KiB |
@@ -0,0 +1,103 @@
|
||||
---
|
||||
title: Bose Case Study
|
||||
linkTitle: Bose
|
||||
case_study_styles: true
|
||||
cid: caseStudies
|
||||
css: /css/style_case_studies.css
|
||||
logo: bose_featured_logo.png
|
||||
featured: false
|
||||
weight: 2
|
||||
quote: >
|
||||
The CNCF Landscape quickly explains what’s going on in all the different areas from storage to cloud providers to automation and so forth. This is our shopping cart to build a cloud infrastructure. We can go choose from the different aisles.
|
||||
---
|
||||
|
||||
<div class="banner1" style="background-image: url('/images/CaseStudy_bose_banner1.jpg')">
|
||||
<h1> CASE STUDY:<img src="/images/bose_logo.png" class="header_logo" style="width:20%;margin-bottom:-1.2%"><br> <div class="subhead" style="margin-top:1%">Bose: Supporting Rapid Development for Millions of IoT Products With Kubernetes
|
||||
|
||||
</div></h1>
|
||||
|
||||
</div>
|
||||
|
||||
<div class="details">
|
||||
Company <b>Bose Corporation</b> Location <b>Framingham, Massachusetts
|
||||
</b> Industry <b>Consumer Electronics</b>
|
||||
</div>
|
||||
|
||||
<hr>
|
||||
<section class="section1">
|
||||
<div class="cols">
|
||||
<div class="col1" style="width:100%"">
|
||||
<h2>Challenge</h2>
|
||||
A household name in high-quality audio equipment, <a href="https://www.bose.com/en_us/index.html">Bose</a> has offered connected products for more than five years, and as that demand grew, the infrastructure had to change to support it. "We needed to provide a mechanism for developers to rapidly prototype and deploy services all the way to production pretty fast,” says Lead Cloud Engineer Josh West. In 2016, the company decided to start building a platform from scratch. The primary goal: "To be one to two steps ahead of the different product groups so that we are never scrambling to catch up with their scale,” says Cloud Architecture Manager Dylan O’Mahony.
|
||||
<br><br>
|
||||
<h2>Solution</h2>
|
||||
From the beginning, the team knew it wanted a microservices architecture. After evaluating and prototyping a couple of orchestration solutions, the team decided to adopt <a href="https://kubernetes.io/">Kubernetes</a> for its scaled IoT Platform-as-a-Service running on AWS. The platform, which also incorporated Prometheus monitoring, launched in production in 2017, serving over 3 million connected products from the get-go. Bose has since adopted a number of other CNCF technologies, including <a href="https://www.fluentd.org/">Fluentd</a>, <a href="https://coredns.io/">CoreDNS</a>, <a href="https://www.jaegertracing.io/">Jaeger</a>, and <a href="https://opentracing.io/">OpenTracing</a>.
|
||||
<br><br>
|
||||
<h2>Impact</h2>
|
||||
With about 100 engineers onboarded, the platform is now enabling 30,000 non-production deployments across dozens of microservices per year. In 2018, there were 1250+ production deployments. Just one production cluster holds 1,800 namespaces and 340 worker nodes. "We had a brand new service taken from concept through coding and deployment all the way to production, including hardening, security testing and so forth, in less than two and a half weeks,” says O’Mahony.
|
||||
</div>
|
||||
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner2">
|
||||
<div class="banner2text">
|
||||
"At Bose we’re building an IoT platform that has enabled our physical products. If it weren’t for Kubernetes and the rest of the CNCF projects being free open source software with such a strong community, we would never have achieved scale, or even gotten to launch on schedule."
|
||||
<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Josh West, Lead Cloud Engineer, Bose</span>
|
||||
</div>
|
||||
</div>
|
||||
<section class="section2">
|
||||
<div class="fullcol">
|
||||
<h2>A household name in high-quality audio equipment, <a href="https://www.bose.com/en_us/index.html">Bose</a> has offered connected products for more than five years, and as that demand grew, the infrastructure had to change to support it. </h2>
|
||||
"We needed to provide a mechanism for developers to rapidly prototype and deploy services all the way to production pretty fast,” says Lead Cloud Engineer Josh West. "There were a lot of cloud capabilities we wanted to provide to support our audio equipment and experiences.”<br><br>
|
||||
In 2016, the company decided to start building an IoT platform from scratch. The primary goal: "To be one to two steps ahead of the different product groups so that we are never scrambling to catch up with their scale,” says Cloud Architecture Manager Dylan O’Mahony. "If they release a new connected product, we want to be already well ahead of being able to handle whatever scale that they’re going to throw at us.”<br><br>
|
||||
From the beginning, the team knew it wanted a microservices architecture and platform as a service. After evaluating and prototyping orchestration solutions, including Mesos and Docker Swarm, the team decided to adopt <a href="https://kubernetes.io/">Kubernetes</a> for its platform running on AWS. Kubernetes was still in 1.5, but already the technology could do much of what the team wanted and needed for the present and the future. For West, that meant having storage and network handled. O’Mahony points to Kubernetes’ portability in case Bose decides to go multi-cloud.<br><br>
|
||||
"Bose is a company that looks out for the long term,” says West. "Going with a quick commercial off-the-shelf solution might’ve worked for that point in time, but it would not have carried us forward, which is what we needed from Kubernetes and the CNCF.”
|
||||
|
||||
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner3" style="background-image: url('/images/CaseStudy_bose_banner3.jpg')">
|
||||
<div class="banner3text">
|
||||
"Everybody on the team thinks in terms of automation, leaning out the processes, getting things done as quickly as possible. When you step back and look at what it means for a 50-plus-year-old speaker company to have that sort of culture, it really is quite incredible, and I think the tools that we use and the foundation that we’ve built with them is a huge piece of that."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Dylan O’Mahony, Cloud Architecture Manager, Bose</span>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
<section class="section3">
|
||||
<div class="fullcol">
|
||||
The team spent time working on choosing tooling to make the experience easier for developers. "Our developers interact with tools provided by our Ops team, and the Ops team run all of their tooling on top of Kubernetes,” says O’Mahony. "We try not to make direct Kubernetes access the only way. In fact, ideally, our developers wouldn’t even need to know that they’re running on Kubernetes.”<br><br>
|
||||
The platform, which also incorporated <a href="https://prometheus.io/">Prometheus</a> monitoring from the beginning, backdoored its way into production in 2017, serving over 3 million connected products from the get-go. "Even though the speakers and the products that we were designing this platform for were still quite a ways away from being launched, we did have some connected speakers on the market,” says O’Mahony. "We basically started to point certain features of those speakers and the apps that go with those speakers to this platform.”<br><br>
|
||||
Today, just one of Bose’s production clusters holds 1,800 namespaces/discrete services and 340 nodes. With about 100 engineers now onboarded, the platform infrastructure is now enabling 30,000 non-production deployments across dozens of microservices per year. In 2018, there were 1250+ production deployments.. It’s a staggering improvement over some of Bose’s previous deployment processes, which supported far fewer deployments and services.
|
||||
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner4" style="background-image: url('/images/CaseStudy_bose_banner4.jpg');width:100%">
|
||||
<div class="banner4text">
|
||||
"The CNCF Landscape quickly explains what’s going on in all the different areas from storage to cloud providers to automation and so forth. This is our shopping cart to build a cloud infrastructure. We can go choose from the different aisles." <br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Josh West, Lead Cloud Engineer, Bose</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<section class="section5" style="padding:0px !important">
|
||||
<div class="fullcol">
|
||||
"We had a brand new service deployed from concept through coding and deployment all the way to production, including hardening, security testing and so forth, in less than two and a half weeks,” says O’Mahony. "Everybody thinks in terms of automation, leaning out the processes, getting things done as quickly as possible. When you step back and look at what it means for a 50-plus-year-old speaker company to have that sort of culture, it really is quite incredible, and I think the tools that we use and the foundation that we’ve built is a huge piece of that.”<br><br>
|
||||
Many of those technologies—such as <a href="https://www.fluentd.org/">Fluentd</a>, <a href="https://coredns.io/">CoreDNS</a>, <a href="https://www.jaegertracing.io/">Jaeger</a>, and <a href="https://opentracing.io/">OpenTracing</a>—come from the <a href="https://landscape.cncf.io/">CNCF Landscape</a>, which West and O’Mahony have relied upon throughout Bose’s cloud native journey. "The CNCF Landscape quickly explains what’s going on in all the different areas from storage to cloud providers to automation and so forth,” says West. "This is our shopping cart to build a cloud infrastructure. We can go choose from the different aisles.” <br><br>
|
||||
And, he adds, "If it weren’t for Kubernetes and the rest of the CNCF projects being free open source software with such a strong community, we would never have achieved scale, or even gotten to launch on schedule.”<br><br>
|
||||
Another benefit of going cloud native: "We are even attracting much more talent into Bose because we’re so involved with the <a href="http://careers.bose.com">CNCF Landscape</a>,” says West. (Yes, they’re hiring.) "It’s just enabled so many people to do so many great things and really brought Bose into the future of cloud.”
|
||||
|
||||
|
||||
</div>
|
||||
|
||||
<div class="banner5" >
|
||||
<div class="banner5text">
|
||||
"We have a lot going on to support many more of our business units at Bose in addition to the consumer electronics division, which we currently do. It’s only because of the cloud native landscape and the tools and the features that are available that we can provide such a fantastic cloud platform for all the developers and divisions that are trying to enable some pretty amazing experiences."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Dylan O’Mahony, Cloud Architecture Manager, Bose</span></div>
|
||||
</div>
|
||||
|
||||
<div class="fullcol">
|
||||
In the coming year, the team wants to work on service mesh and serverless, as well as expansion around the world. "Getting our latency down by going multi-region is going to be a big focus for us,” says O’Mahony. "In order to make sure that our customers in Japan, Australia, and everywhere else are having a good experience, we want to have points of presence closer to them. It’s never been done at Bose before.”<br><br>
|
||||
That won’t stop them, because the team is all about lofty goals. "We want to get to billions of connected products!” says West. "We have a lot going on to support many more of our business units at Bose in addition to the consumer electronics division, which we currently do. It’s only because of the cloud native landscape and the tools and the features that are available that we can provide such a fantastic cloud platform for all the developers and divisions that are trying to enable some pretty amazing experiences.”<br><br>
|
||||
In fact, given the scale the platform is already supporting, says O’Mahony, "doing anything other than Kubernetes, I think, would be folly at this point.”
|
||||
|
||||
|
||||
</div>
|
||||
</section>
|
||||
</body>
|
||||
</html>
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 2.8 KiB |
@@ -0,0 +1,93 @@
|
||||
---
|
||||
title: CERN Case Study
|
||||
linkTitle: cern
|
||||
case_study_styles: true
|
||||
cid: caseStudies
|
||||
css: /css/style_case_studies.css
|
||||
logo: cern_featured_logo.png
|
||||
---
|
||||
|
||||
<div class="banner1" style="background-image: url('/images/CaseStudy_cern_banner1.jpg')">
|
||||
<h1> CASE STUDY: CERN<br> <div class="subhead" style="margin-top:1%">CERN: Processing Petabytes of Data More Efficiently with Kubernetes
|
||||
|
||||
</div></h1>
|
||||
|
||||
</div>
|
||||
|
||||
<div class="details">
|
||||
Company <b>CERN</b> Location <b>Geneva, Switzerland
|
||||
</b> Industry <b>Particle physics research</b>
|
||||
</div>
|
||||
|
||||
<hr>
|
||||
<section class="section1">
|
||||
<div class="cols">
|
||||
<div class="col1" style="width:100%"">
|
||||
<h2>Challenge</h2>
|
||||
At CERN, the European Organization for Nuclear Research, physicists conduct experiments to learn about fundamental science. In its particle accelerators, "we accelerate protons to very high energy, close to the speed of light, and we make the two beams of protons collide," says CERN Software Engineer Ricardo Rocha. "The end result is a lot of data that we have to process." CERN currently stores 330 petabytes of data in its data centers, and an upgrade of its accelerators expected in the next few years will drive that number up by 10x. Additionally, the organization experiences extreme peaks in its workloads during periods prior to big conferences, and needs its infrastructure to scale to those peaks. "We want to have a more hybrid infrastructure, where we have our on premise infrastructure but can make use of public clouds temporarily when these peaks come up," says Rocha. "We’ve been looking to new technologies that can help improve our efficiency in our infrastructure so that we can dedicate more of our resources to the actual processing of the data."
|
||||
<br><br>
|
||||
<h2>Solution</h2>
|
||||
CERN’s technology team embraced containerization and cloud native practices, choosing Kubernetes for orchestration, Helm for deployment, Prometheus for monitoring, and CoreDNS for DNS resolution inside the clusters. Kubernetes federation has allowed the organization to run some production workloads both on premise and in public clouds.
|
||||
<br><br>
|
||||
<h2>Impact</h2>
|
||||
"Kubernetes gives us the full automation of the application," says Rocha. "It comes with built-in monitoring and logging for all the applications and the workloads that deploy in Kubernetes. This is a massive simplification of our current deployments." The time to deploy a new cluster for a complex distributed storage system has gone from more than 3 hours to less than 15 minutes. Adding new nodes to a cluster used to take more than an hour; now it takes less than 2 minutes. The time it takes to autoscale replicas for system components has decreased from more than an hour to less than 2 minutes. Initially, virtualization gave 20% overhead, but with tuning this was reduced to ~5%. Moving to Kubernetes on bare metal would get this to 0%. Not having to host virtual machines is expected to also get 10% of memory capacity back.
|
||||
|
||||
</div>
|
||||
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner2">
|
||||
<div class="banner2text">
|
||||
"Kubernetes is something we can relate to very much because it’s naturally distributed. What it gives us is a uniform API across heterogeneous resources to define our workloads. This is something we struggled with a lot in the past when we want to expand our resources outside our infrastructure."
|
||||
<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Ricardo Rocha, Software Engineer, CERN</span>
|
||||
</div>
|
||||
</div>
|
||||
<section class="section2">
|
||||
<div class="fullcol">
|
||||
<h2>With a mission of researching fundamental science, and a stable of extremely large machines, the European Organization for Nuclear Research (CERN) operates at what can only be described as hyperscale. </h2>
|
||||
Experiments are conducted in particle accelerators, the biggest of which is 27 kilometers in circumference. "We accelerate protons to very high energy, to close to the speed of light, and we make the two beams of protons collide in well-defined places," says CERN Software Engineer Ricardo Rocha. "We build experiments around these places where we do the collisions. The end result is a lot of data that we have to process."<br><br>
|
||||
And he does mean a lot: CERN currently stores and processes 330 petabytes of data—gathered from 4,300 projects and 3,300 users—using 10,000 hypervisors and 320,000 cores in its data centers. <br><br>
|
||||
Over the years, the CERN technology department has built a large computing infrastructure, based on OpenStack private clouds, to help the organization’s physicists analyze and treat all this data. The organization experiences extreme peaks in its workloads. "Very often, just before conferences, physicists want to do an enormous amount of extra analysis to publish their papers, and we have to scale to these peaks, which means overcommitting resources in some cases," says Rocha. "We want to have a more hybrid infrastructure, where we have our on premise infrastructure but can make use of public clouds temporarily when these peaks come up."<br><br>
|
||||
Additionally, few years ago, CERN announced that it would be doing a big upgrade of its accelerators, which will mean a ten-fold increase in the amount of data that can be collected. "So we’ve been looking to new technologies that can help improve our efficiency in our infrastructure, so that we can dedicate more of our resources to the actual processing of the data," says Rocha.
|
||||
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner3" style="background-image: url('/images/CaseStudy_cern_banner3.jpg')">
|
||||
<div class="banner3text">
|
||||
"Before, the tendency was always: ‘I need this, I get a couple of developers, and I implement it.’ Right now it’s ‘I need this, I’m sure other people also need this, so I’ll go and ask around.’ The CNCF is a good source because there’s a very large catalog of applications available. It’s very hard right now to justify developing a new product in-house. There is really no real reason to keep doing that. It’s much easier for us to try it out, and if we see it’s a good solution, we try to reach out to the community and start working with that community." <br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Ricardo Rocha, Software Engineer, CERN</span>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
<section class="section3">
|
||||
<div class="fullcol">
|
||||
Rocha’s team started looking at Kubernetes and containerization in the second half of 2015. "We’ve been using distributed infrastructures for decades now," says Rocha. "Kubernetes is something we can relate to very much because it’s naturally distributed. What it gives us is a uniform API across heterogeneous resources to define our workloads. This is something we struggled with a lot in the past when we want to expand our resources outside our infrastructure."<br><br>
|
||||
The team created a prototype system for users to deploy their own Kubernetes cluster in CERN’s infrastructure, and spent six months validating the use cases and making sure that Kubernetes integrated with CERN’s internal systems. The main use case is batch workloads, which represent more than 80% of resource usage at CERN. (One single project that does most of the physics data processing and analysis alone consumes 250,000 cores.) "This is something where the investment in simplification of the deployment, logging, and monitoring pays off very quickly," says Rocha. Other use cases include Spark-based data analysis and machine learning to improve physics analysis. "The fact that most of these technologies integrate very well with Kubernetes makes our lives easier," he adds.<br><br>
|
||||
The system went into production in October 2016, also using Helm for deployment, Prometheus for monitoring, and CoreDNS for DNS resolution within the cluster. "One thing that Kubernetes gives us is the full automation of the application," says Rocha. "So it comes with built-in monitoring and logging for all the applications and the workloads that deploy in Kubernetes. This is a massive simplification of our current deployments." The time to deploy a new cluster for a complex distributed storage system has gone from more than 3 hours to less than 15 minutes.<br><br> Adding new nodes to a cluster used to take more than an hour; now it takes less than 2 minutes. The time it takes to autoscale replicas for system components has decreased from more than an hour to less than 2 minutes.
|
||||
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner4" style="background-image: url('/images/CaseStudy_cern_banner4.jpg')">
|
||||
<div class="banner4text">
|
||||
"With Kubernetes, there’s a well-established technology and a big community that we can contribute to. It allows us to do our physics analysis without having to focus so much on the lower level software. This is just exciting. We are looking forward to keep contributing to the community and collaborating with everyone."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Ricardo Rocha, Software Engineer, CERN</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<section class="section5" style="padding:0px !important">
|
||||
<div class="fullcol">
|
||||
Rocha points out that the metric used in the particle accelerators may be events per second, but in reality "it’s how fast and how much of the data we can process that actually counts." And efficiency has certainly been improved with Kubernetes. Initially, virtualization gave 20% overhead, but with tuning this was reduced to ~5%. Moving to Kubernetes on bare metal would get this to 0%. Not having to host virtual machines is expected to also get 10% of memory capacity back.<br><br>
|
||||
Kubernetes federation, which CERN has been using for a portion of its production workloads since February 2018, has allowed the organization to adopt a hybrid cloud strategy. And it was remarkably simple to do. "We had a summer intern working on federation," says Rocha. "For many years, I’ve been developing distributed computing software, which took like a decade and a lot of effort from a lot of people to stabilize and make sure it works. And for our intern, in a couple of days he was able to demo to me and my team that we had a cluster at CERN and a few clusters outside in public clouds that were federated together and that we could submit workloads to. This was shocking for us. It really shows the power of using this kind of well-established technologies." <br><br>
|
||||
With such results, adoption of Kubernetes has made rapid gains at CERN, and the team is eager to give back to the community. "If we look back into the ’90s and early 2000s, there were not a lot of companies focusing on systems that have to scale to this kind of size, storing petabytes of data, analyzing petabytes of data," says Rocha. "The fact that Kubernetes is supported by such a wide community and different backgrounds, it motivates us to contribute back."
|
||||
|
||||
</div>
|
||||
|
||||
<div class="banner5" >
|
||||
<div class="banner5text">
|
||||
This means that the physicist can build his or her analysis and publish it in a repository, share it with colleagues, and in 10 years redo the same analysis with new data. If we looked back even 10 years, this was just a dream."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Ricardo Rocha, Software Engineer, CERN</span></div>
|
||||
</div>
|
||||
|
||||
<div class="fullcol">
|
||||
These new technologies aren’t just enabling infrastructure improvements. CERN also uses the Kubernetes-based <a href="https://github.com/recast-hep">Reana/Recast</a> platform for reusable analysis, which is "the ability to define physics analysis as a set of workflows that are fully containerized in one single entry point," says Rocha. "This means that the physicist can build his or her analysis and publish it in a repository, share it with colleagues, and in 10 years redo the same analysis with new data. If we looked back even 10 years, this was just a dream."<br><br>
|
||||
All of these things have changed the culture at CERN considerably. A decade ago, "The tendency was always: ‘I need this, I get a couple of developers, and I implement it,’" says Rocha. "Right now it’s ‘I need this, I’m sure other people also need this, so I’ll go and ask around.’ The CNCF is a good source because there’s a very large catalog of applications available. It’s very hard right now to justify developing a new product in-house. There is really no real reason to keep doing that. It’s much easier for us to try it out, and if we see it’s a good solution, we try to reach out to the community and start working with that community."
|
||||
|
||||
</div>
|
||||
</section>
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 11 KiB |
@@ -0,0 +1,99 @@
|
||||
---
|
||||
title: China Unicom Case Study
|
||||
|
||||
linkTitle: chinaunicom
|
||||
case_study_styles: true
|
||||
cid: caseStudies
|
||||
css: /css/style_case_studies.css
|
||||
logo: chinaunicom_featured_logo.png
|
||||
featured: true
|
||||
weight: 1
|
||||
quote: >
|
||||
Kubernetes has improved our experience using cloud infrastructure. There is currently no alternative technology that can replace it.
|
||||
---
|
||||
|
||||
<div class="banner1" style="background-image: url('/images/CaseStudy_chinaunicom_banner1.jpg')">
|
||||
<h1> CASE STUDY:<img src="/images/chinaunicom_logo.png" class="header_logo" style="width:25%;margin-bottom:-1%"><br> <div class="subhead" style="margin-top:1%;line-height:1.4em">China Unicom: How China Unicom Leveraged Kubernetes to Boost Efficiency<br>and Lower IT Costs
|
||||
|
||||
</div></h1>
|
||||
|
||||
</div>
|
||||
|
||||
<div class="details">
|
||||
Company <b>China Unicom</b> Location <b>Beijing, China</b> Industry <b>Telecom</b>
|
||||
</div>
|
||||
|
||||
<hr>
|
||||
<section class="section1">
|
||||
<div class="cols">
|
||||
<div class="col1" style="width:100%"">
|
||||
<h2>Challenge</h2>
|
||||
China Unicom is one of the top three telecom operators in China, and to serve its 300 million users, the company runs several data centers with thousands of servers in each, using <a href="https://www.docker.com/">Docker</a> containerization and <a href="https://www.vmware.com/">VMWare</a> and <a href="https://www.openstack.org/">OpenStack</a> infrastructure since 2016. Unfortunately, "the resource utilization rate was relatively low," says Chengyu Zhang, Group Leader of Platform Technology R&D, "and we didn’t have a cloud platform to accommodate our hundreds of applications." Formerly an entirely state-owned company, China Unicom has in recent years taken private investment from BAT (Baidu, Alibaba, Tencent) and JD.com, and is now focusing on internal development using open source technology, rather than commercial products. As such, Zhang’s China Unicom Lab team began looking for open source orchestration for its cloud infrastructure.
|
||||
<br><br>
|
||||
<h2>Solution</h2>
|
||||
Because of its rapid growth and mature open source community, Kubernetes was a natural choice for China Unicom. The company’s Kubernetes-enabled cloud platform now hosts 50 microservices and all new development going forward. "Kubernetes has improved our experience using cloud infrastructure," says Zhang. "There is currently no alternative technology that can replace it." China Unicom also uses <a href="https://istio.io/">Istio</a> for its microservice framework, <a href="https://www.envoyproxy.io/">Envoy</a>, <a href="https://coredns.io/">CoreDNS</a>, and <a href="https://www.fluentd.org/">Fluentd</a>.
|
||||
<br><br>
|
||||
<h2>Impact</h2>
|
||||
At China Unicom, Kubernetes has improved both operational and development efficiency. Resource utilization has increased by 20-50%, lowering IT infrastructure costs, and deployment time has gone from a couple of hours to 5-10 minutes. "This is mainly because of the self-healing and scalability, so we can increase our efficiency in operation and maintenance," Zhang says. "For example, we currently have only five people maintaining our multiple systems. We could never imagine we can achieve this scalability in such a short time."
|
||||
|
||||
</div>
|
||||
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner2">
|
||||
<div class="banner2text">
|
||||
"Kubernetes has improved our experience using cloud infrastructure. There is currently no alternative technology that can replace it."
|
||||
<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Chengyu Zhang, Group Leader of Platform Technology R&D, China Unicom</span>
|
||||
</div>
|
||||
</div>
|
||||
<section class="section2">
|
||||
<div class="fullcol">
|
||||
<h2>With more than 300 million users, China Unicom is one of the country’s top three telecom operators. </h2>
|
||||
|
||||
Behind the scenes, the company runs multiple data centers with thousands of servers in each, using Docker containerization and VMWare and OpenStack infrastructure since 2016. Unfortunately, "the resource utilization rate was relatively low," says Chengyu Zhang, Group Leader of Platform Technology R&D, "and we didn’t have a cloud platform to accommodate our hundreds of applications." <br><br>
|
||||
Zhang’s team, which is responsible for new technology, R&D and platforms, set out to find an IT management solution. Formerly an entirely state-owned company, China Unicom has in recent years taken private investment from BAT (Baidu, Alibaba, Tencent) and JD.com, and is now focusing on homegrown development using open source technology, rather than commercial products. For that reason, the team began looking for open source orchestration for its cloud infrastructure.
|
||||
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner3" style="background-image: url('/images/CaseStudy_chinaunicom_banner3.jpg');width:100%;padding-left:0;">
|
||||
<div class="banner3text">
|
||||
"We could never imagine we can achieve this scalability in such a short time."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Chengyu Zhang, Group Leader of Platform Technology R&D, China Unicom</span>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
<section class="section3">
|
||||
<div class="fullcol">
|
||||
Though China Unicom was already using Mesos for a core telecom operator system, the team felt that Kubernetes was a natural choice for the new cloud platform. "The main reason was that it has a mature community," says Zhang. "It grows very rapidly, and so we can learn a lot from others’ best practices." China Unicom also uses Istio for its microservice framework, Envoy, CoreDNS, and Fluentd.<br><br>
|
||||
The company’s Kubernetes-enabled cloud platform now hosts 50 microservices and all new development going forward. China Unicom developers can easily leverage the technology through APIs, without doing the development work themselves. The cloud platform provides 20-30 services connected to the company’s data center PaaS platform, as well as supports things such as big data analysis for internal users in the branch offices across the 31 provinces in China.<br><br>
|
||||
"Kubernetes has improved our experience using cloud infrastructure," says Zhang. "There is currently no alternative technology that can replace it."
|
||||
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner4" style="background-image: url('/images/CaseStudy_chinaunicom_banner4.jpg');width:100%">
|
||||
<div class="banner4text">
|
||||
"This technology is relatively complicated, but as long as developers get used to it, they can enjoy all the benefits." <br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Jie Jia, Member of Platform Technology R&D, China Unicom</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<section class="section5" style="padding:0px !important">
|
||||
<div class="fullcol">
|
||||
In fact, Kubernetes has boosted both operational and development efficiency at China Unicom. Resource utilization has increased by 20-50%, lowering IT infrastructure costs, and deployment time has gone from a couple of hours to 5-10 minutes. "This is mainly because of the self-healing and scalability of Kubernetes, so we can increase our efficiency in operation and maintenance," Zhang says. "For example, we currently have only five people maintaining our multiple systems."<br><br>
|
||||
With the wins China Unicom has experienced with Kubernetes, Zhang and his team are eager to give back to the community. That starts with participating in meetups and conferences, and offering advice to other companies that are considering a similar path. "Especially for those companies who have had traditional cloud computing system, I really recommend them to join the cloud native computing community," says Zhang.
|
||||
|
||||
</div>
|
||||
|
||||
<div class="banner5" >
|
||||
<div class="banner5text">
|
||||
"Companies can use the managed services offered by companies like Rancher, because they have already customized this technology, you can easily leverage this technology."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Jie Jia, Member of Platform Technology R&D, China Unicom</span></div>
|
||||
</div>
|
||||
|
||||
<div class="fullcol">
|
||||
Platform Technology R&D team member Jie Jia adds that though "this technology is relatively complicated, as long as developers get used to it, they can enjoy all the benefits." And Zhang points out that in his own experience with virtual machine cloud, "Kubernetes and these cloud native technologies are relatively simpler."<br><br>
|
||||
Plus, "companies can use the managed services offered by companies like <a href="https://www.docker.com/">Rancher</a>, because they have already customized this technology," says Jia. "You can easily leverage this technology."<br><br>
|
||||
Looking ahead, China Unicom plans to develop more applications on Kubernetes, focusing on big data and machine learning. The team is continuing to optimize the cloud platform that it built, and hopes to pass the conformance test to join CNCF’s <a href="https://www.cncf.io/announcement/2017/11/13/cloud-native-computing-foundation-launches-certified-kubernetes-program-32-conformant-distributions-platforms/">Certified Kubernetes Conformance Program</a>. They’re also hoping to someday contribute code back to the community. <br><br>
|
||||
If that sounds ambitious, it’s because the results they’ve gotten from adopting Kubernetes have been beyond even their greatest expectations. Says Zhang: "We could never imagine we can achieve this scalability in such a short time."
|
||||
|
||||
</div>
|
||||
</section>
|
||||
</body>
|
||||
</html>
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 9.4 KiB |
@@ -0,0 +1,99 @@
|
||||
---
|
||||
title: City of Montreal Case Study
|
||||
linkTitle: city-of-montreal
|
||||
case_study_styles: true
|
||||
cid: caseStudies
|
||||
css: /css/style_case_studies.css
|
||||
featured: false
|
||||
---
|
||||
|
||||
<div class="banner1" style="background-image: url('/images/CaseStudy_montreal_banner1.jpg')">
|
||||
<h1> CASE STUDY:<img src="/images/montreal_logo.png" class="header_logo" style="width:20%;margin-bottom:-1.2%"><br> <div class="subhead" style="margin-top:1%">City of Montréal - How the City of Montréal Is Modernizing Its 30-Year-Old, Siloed Architecture with Kubernetes
|
||||
|
||||
</div></h1>
|
||||
|
||||
</div>
|
||||
|
||||
<div class="details">
|
||||
Company <b>City of Montréal</b> Location <b>Montréal, Québec, Canada</b> Industry <b>Government</b>
|
||||
</div>
|
||||
|
||||
<hr>
|
||||
<section class="section1">
|
||||
<div class="cols">
|
||||
<div class="col1" style="width:100%"">
|
||||
<h2>Challenge</h2>
|
||||
Like many governments, Montréal has a number of legacy systems, and “we have systems that are older than some developers working here,” says the city’s CTO, Jean-Martin Thibault. “We have mainframes, all flavors of Windows, various flavors of Linux, old and new Oracle systems, Sun servers, all kinds of databases. Like all big corporations, some of the most important systems, like Budget and Human Resources, were developed on mainframes in-house over the past 30 years.” There are over 1,000 applications in all, and most of them were running on different ecosystems. In 2015, a new management team decided to break down those silos, and invest in IT in order to move toward a more integrated governance for the city. They needed to figure out how to modernize the architecture.
|
||||
|
||||
<h2>Solution</h2>
|
||||
The first step was containerization. The team started with a small Docker farm with four or five servers, with Rancher for providing access to the Docker containers and their logs and Jenkins to deploy. “We based our effort on the new trends; we understood the benefits of immutability and deployments without downtime and such things,” says Solutions Architect Marc Khouzam. They soon realized they needed orchestration as well, and opted for Kubernetes. Says Enterprise Architect Morgan Martinet: “Kubernetes offered concepts on how you would describe an architecture for any kind of application, and based on those concepts, deploy what’s required to run the infrastructure. It was becoming a de facto standard.”
|
||||
<br>
|
||||
<h2>Impact</h2>
|
||||
The time to market has improved drastically, from many months to a few weeks. Deployments went from months to hours. “In the past, you would have to ask for virtual machines, and that alone could take weeks, easily,” says Thibault. “Now you don’t even have to ask for anything. You just create your project and it gets deployed.” Kubernetes has also improved the efficiency of how the city uses its compute resources: “Before, the 200 application components we currently run on Kubernetes would have required hundreds of virtual machines, and now, if we’re talking about a single environment of production, we are able to run them on 8 machines, counting the masters of Kubernetes,” says Martinet. And it’s all done with a small team of just 5 people operating the Kubernetes clusters.
|
||||
</div>
|
||||
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner2">
|
||||
<div class="banner2text">
|
||||
"We realized the limitations of having a non-orchestrated Docker environment. Kubernetes came to the rescue, bringing in all these features that make it a lot easier to manage and give a lot more benefits to the users."
|
||||
<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- JEAN-MARTIN THIBAULT, CTO, CITY OF MONTRÉAL</span>
|
||||
</div>
|
||||
</div>
|
||||
<section class="section2">
|
||||
<div class="fullcol">
|
||||
<h2>The second biggest municipality in Canada, Montréal has a large number of legacy systems keeping the government running. And while they don’t quite date back to the city’s founding in 1642, “we have systems that are older than some developers working here,” jokes the city’s CTO, Jean-Martin Thibault.</h2>
|
||||
“We have mainframes, all flavors of Windows, various flavors of Linux, old and new Oracle systems, Sun servers, all kinds of databases. Some of the most important systems, like Budget and Human Resources, were developed on mainframes in-house over the past 30 years.”
|
||||
<br><br>
|
||||
In recent years, that fact became a big pain point. There are over 1,000 applications in all, running on almost as many different ecosystems. In 2015, a new city management team decided to break down those silos, and invest in IT in order to move toward a more integrated governance. “The organization was siloed, so as a result the architecture was siloed,” says Thibault. “Once we got integrated into one IT team, we decided to redo an overall enterprise architecture.”
|
||||
<br><br>
|
||||
The first step to modernize the architecture was containerization. “We based our effort on the new trends; we understood the benefits of immutability and deployments without downtime and such things,” says Solutions Architect Marc Khouzam. The team started with a small Docker farm with four or five servers, with Rancher for providing access to the Docker containers and their logs and Jenkins for deployment.
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner3" style="background-image: url('/images/CaseStudy_montreal_banner3.jpg')">
|
||||
<div class="banner3text">
|
||||
"Getting a project running in Kubernetes is entirely dependent on how long you need to program the actual software. It’s no longer dependent on deployment. Deployment is so fast that it’s negligible."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- MARC KHOUZAM, SOLUTIONS ARCHITECT, CITY OF MONTRÉAL</span>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
<section class="section3">
|
||||
<div class="fullcol">
|
||||
But this Docker farm setup had some limitations, including the lack of self-healing and dynamic scaling based on traffic, and the effort required to optimize server resources and scale to multiple instances of the same container. The team soon realized they needed orchestration as well. “Kubernetes came to the rescue,” says Thibault, “bringing in all these features that make it a lot easier to manage and give a lot more benefits to the users.”
|
||||
<br><br>
|
||||
The team had evaluated several orchestration solutions, but Kubernetes stood out because it addressed all of the pain points. (They were also inspired by Yahoo! Japan’s use case, which the team members felt came close to their vision.) “Kubernetes offered concepts on how you would describe an architecture for any kind of application, and based on those concepts, deploy what’s required to run the infrastructure,” says Enterprise Architect Morgan Martinet. “It was becoming a de facto standard. It also promised portability across cloud providers. The choice of Kubernetes now gives us many options such as running clusters in-house or in any IaaS provider, or even using Kubernetes-as-a-service in any of the major cloud providers.”
|
||||
<br><br>
|
||||
Another important factor in the decision was vendor neutrality. “As a government entity, it is essential for us to be neutral in our selection of products and providers,” says Thibault. “The independence of the Cloud Native Computing Foundation from any company provides this.”
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner4" style="background-image: url('/images/CaseStudy_montreal_banner4.jpg')">
|
||||
<div class="banner4text">
|
||||
"Kubernetes has been great. It’s been stable, and it provides us with elasticity, resilience, and robustness. While re-architecting for Kubernetes, we also benefited from the monitoring and logging aspects, with centralized logging, Prometheus logging, and Grafana dashboards. We have enhanced visibility of what’s being deployed." <br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- MORGAN MARTINET, ENTERPRISE ARCHITECT, CITY OF MONTRÉAL</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<section class="section5" style="padding:0px !important">
|
||||
<div class="fullcol">
|
||||
The Kubernetes implementation began with the deployment of a small cluster using an internal Ansible playbook, which was soon replaced by the Kismatic distribution. Given the complexity they saw in operating a Kubernetes platform, they decided to provide development groups with an automated CI/CD solution based on Helm. “An integrated CI/CD solution on Kubernetes standardized how the various development teams designed and deployed their solutions, but allowed them to remain independent,” says Khouzam.
|
||||
<br><br>
|
||||
During the re-architecting process, the team also added Prometheus for monitoring and alerting, Fluentd for logging, and Grafana for visualization. “We have enhanced visibility of what’s being deployed,” says Martinet. Adds Khouzam: “The big benefit is we can track anything, even things that don’t run inside the Kubernetes cluster. It’s our way to unify our monitoring effort.”
|
||||
<br><br>
|
||||
All together, the cloud native solution has had a positive impact on velocity as well as administrative overhead. With standardization, code generation, automatic deployments into Kubernetes, and standardized monitoring through Prometheus, the time to market has improved drastically, from many months to a few weeks. Deployments went from months and weeks of planning down to hours. “In the past, you would have to ask for virtual machines, and that alone could take weeks to properly provision,” says Thibault. Plus, for dedicated systems, experts often had to be brought in to install them with their own recipes, which could take weeks and months.
|
||||
<br><br>
|
||||
Now, says Khouzam, “we can deploy pretty much any application that’s been Dockerized without any help from anybody. Getting a project running in Kubernetes is entirely dependent on how long you need to program the actual software. It’s no longer dependent on deployment. Deployment is so fast that it’s negligible.”
|
||||
|
||||
</div>
|
||||
|
||||
<div class="banner5" >
|
||||
<div class="banner5text">
|
||||
"We’re working with the market when possible, to put pressure on our vendors to support Kubernetes, because it’s a much easier solution to manage"<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- MORGAN MARTINET, ENTERPRISE ARCHITECT, CITY OF MONTRÉAL</span></div>
|
||||
</div>
|
||||
|
||||
<div class="fullcol">
|
||||
Kubernetes has also improved the efficiency of how the city uses its compute resources: “Before, the 200 application components we currently run in Kubernetes would have required hundreds of virtual machines, and now, if we’re talking about a single environment of production, we are able to run them on 8 machines, counting the masters of Kubernetes,” says Martinet. And it’s all done with a small team of just five people operating the Kubernetes clusters. Adds Martinet: “It’s a dramatic improvement no matter what you measure.”
|
||||
<br><br>
|
||||
So it should come as no surprise that the team’s strategy going forward is to target Kubernetes as much as they can. “If something can’t run inside Kubernetes, we’ll wait for it,” says Thibault. That means they haven’t moved any of the city’s Windows systems onto Kubernetes, though it’s something they would like to do. “We’re working with the market when possible, to put pressure on our vendors to support Kubernetes, because it’s a much easier solution to manage,” says Martinet.
|
||||
<br><br>
|
||||
Thibault sees a near future where 60% of the city’s workloads are running on a Kubernetes platform—basically any and all of the use cases that they can get to work there. “It’s so much more efficient than the way we used to do things,” he says. “There’s no looking back.”
|
||||
|
||||
</div>
|
||||
</section>
|
||||
@@ -6,10 +6,7 @@ case_study_styles: true
|
||||
cid: caseStudies
|
||||
css: /css/style_case_studies.css
|
||||
logo: ibm_featured_logo.svg
|
||||
featured: true
|
||||
weight: 2
|
||||
quote: >
|
||||
We see CNCF as a safe haven for cloud native open source, providing stability, longevity, and expected maintenance for member projects—no matter the originating vendor or project.
|
||||
featured: false
|
||||
---
|
||||
|
||||
<div class="banner1" style="background-image: url('/images/CaseStudy_ibm_banner1.jpg')">
|
||||
@@ -34,7 +31,7 @@ quote: >
|
||||
</div>
|
||||
|
||||
<div class="col2" style="width:95%">
|
||||
|
||||
|
||||
|
||||
<h2>Impact</h2>
|
||||
IBM's intention in offering a managed Kubernetes container service and image registry is to provide a fully secure end-to-end platform for its enterprise customers. "Image signing is one key part of that offering, and our container registry team saw Notary as the de facto way to implement that capability in the current Docker and container ecosystem," Hough says. The company had not been offering image signing before, and Notary is the tool it used to implement that capability. "We had a multi-tenant Docker Registry with private image hosting," Hough says. "The Docker Registry uses hashes to ensure that image content is correct, and data is encrypted both in flight and at rest. But it does not provide any guarantees of who pushed an image. We used Notary to enable users to sign images in their private registry namespaces if they so choose."
|
||||
|
||||
@@ -0,0 +1,97 @@
|
||||
---
|
||||
title: JD.com Case Study
|
||||
linkTitle: jd-com
|
||||
case_study_styles: true
|
||||
cid: caseStudies
|
||||
css: /css/style_case_studies.css
|
||||
featured: false
|
||||
---
|
||||
|
||||
<div class="banner1" style="background-image: url('/images/CaseStudy_jdcom_banner1.jpg')">
|
||||
<h1> CASE STUDY:<img src="/images/jdcom_logo.png" class="header_logo" style="width:17%;margin-bottom:-1%"><br> <div class="subhead" style="margin-top:1%">JD.com: How JD.com Pioneered Kubernetes for E-Commerce at Hyperscale
|
||||
|
||||
</div></h1>
|
||||
|
||||
</div>
|
||||
|
||||
<div class="details">
|
||||
Company <b>JD.com</b> Location <b>Beijing, China</b> Industry <b>eCommerce</b>
|
||||
</div>
|
||||
|
||||
<hr>
|
||||
<section class="section1">
|
||||
<div class="cols">
|
||||
<div class="col1" style="width:100%"">
|
||||
<h2>Challenge</h2>
|
||||
With more than 300 million active users and total 2017 revenue of more than $55 billion, <a href="https://corporate.JD.com/home">JD.com</a> is China’s largest retailer, and its operations are the epitome of hyperscale. For example, there are more than a trillion images in JD.com’s product databases—with 100 million being added daily—and this enormous amount of data needs to be instantly accessible. In 2014, JD.com moved its applications to containers running on bare metal machines using OpenStack and Docker to "speed up the delivery of our computing resources and make the operations much simpler," says Haifeng Liu, JD.com’s Chief Architect. But by the end of 2015, with tens of thousands of nodes running in multiple data centers, "we encountered a lot of problems because our platform was not strong enough, and we suffered from bottlenecks and scalability issues," says Liu. "We needed infrastructure for the next five years of development, now."
|
||||
|
||||
<h2>Solution</h2>
|
||||
JD.com turned to Kubernetes to accommodate its clusters. At the beginning of 2016, the company began to transition from OpenStack to Kubernetes, and today, JD.com runs the world’s largest Kubernetes cluster. "Kubernetes has provided a strong foundation on top of which we have customized the solution to suit our needs as China’s largest retailer."
|
||||
<br>
|
||||
<h2>Impact</h2>
|
||||
"We have greater data center efficiency, better managed resources, and smarter deployment with the Kubernetes platform," says Liu. Deployment time went from several hours to tens of seconds. Efficiency has improved by 20-30%, measured in IT costs. With the further optimizations the team is working on, Liu believes there is the potential to save hundreds of millions of dollars a year. But perhaps the best indication of success was the annual Singles Day shopping event, which ran on the Kubernetes platform for the first time in 2018. Over 11 days, transaction volume on JD.com was $23 billion, and "our e-commerce platforms did great," says Liu. "Infrastructure led the way to prep for 11.11. We took the approach of predicting volume, emulating the behavior of customers to prepare beforehand, and drilled for malfunctions. Because of Kubernetes’s scalability, we were able to handle an extremely high level of demand."
|
||||
</div>
|
||||
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner2">
|
||||
<div class="banner2text">
|
||||
"Kubernetes helped us reduce the complexity of operations to make distributed systems stable and scalable. Most importantly, we can leverage Kubernetes for scheduling resources to reduce hardware costs. That’s the big win."
|
||||
<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- HAIFENG LIU, CHIEF ARCHITECT, JD.com</span>
|
||||
</div>
|
||||
</div>
|
||||
<section class="section2">
|
||||
<div class="fullcol">
|
||||
<h2>With more than 300 million active users and $55.7 billion in annual revenues last year, JD.com is China’s largest retailer, and its operations are the epitome of hyperscale.</h2>
|
||||
For example, there are more than a trillion images in JD.com’s product databases for customers, with 100 million being added daily. And this enormous amount of data needs to be instantly accessible to enable a smooth online customer experience.
|
||||
<br><br>
|
||||
In 2014, JD.com moved its applications to containers running on bare metal machines using OpenStack and Docker to "speed up the delivery of our computing resources and make the operations much simpler," says Haifeng Liu, JD.com’s Chief Architect. But by the end of 2015, with hundreds of thousands of nodes in multiple data centers, "we encountered a lot of problems because our platform was not strong enough, and we suffered from bottlenecks and scalability issues," Liu adds. "We needed infrastructure for the next five years of development, now."
|
||||
<br><br>
|
||||
After considering a number of orchestration technologies, JD.com decided to adopt Kubernetes to accommodate its ever-growing clusters. "The main reason is because Kubernetes can give us more efficient, scalable and much simpler application deployments, plus we can leverage it to do flexible platform scheduling," says Liu.
|
||||
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner3" style="background-image: url('/images/CaseStudy_jdcom_banner3.jpg')">
|
||||
<div class="banner3text">
|
||||
"We customized Kubernetes and built a modern system on top of it. This entire ecosystem of Kubernetes plus our own optimizations have helped us save costs and time."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- HAIFENG LIU, CHIEF ARCHITECT, JD.com</span>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
<section class="section3">
|
||||
<div class="fullcol">
|
||||
The fact that Kubernetes is based on Google’s Borg also gave the company confidence. The team liked that Kubernetes has a clear and simple architecture, and that it’s developed mostly in Go, which is a popular language within JD.com. Though he felt that at the time Kubernetes "was not mature enough," Liu says, "we adopted it anyway."
|
||||
<br><br>
|
||||
The team spent a year developing the new container engine platform based on Kubernetes, and at the end of 2016, began promoting it within the company. "We wanted the cluster to be the default way for creating services, so scalability is easier," says Liu. "We talked to developers, interest grew, and we solved problems together." Some of these problems included networking performance and etcd scalability. "But during the past two years, Kubernetes has become more mature and very stable," he adds.
|
||||
<br><br>
|
||||
Today, the company runs the world’s largest Kubernetes cluster. "We customized Kubernetes and built a modern system on top of it," says Liu. "This entire ecosystem of Kubernetes plus our own optimizations have helped us save costs and time. We have greater data center efficiency, better managed resources, and smarter deployment with the Kubernetes platform."
|
||||
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner4" style="background-image: url('/images/CaseStudy_jdcom_banner4.jpg');width:100%">
|
||||
<div class="banner4text">
|
||||
"My advice is first you need to combine this technology with your own businesses, and the second is you need clear goals. You cannot just use the technology because others are using it. You need to consider your own objectives." <br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- HAIFENG LIU, CHIEF ARCHITECT, JD.com</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<section class="section5" style="padding:0px !important">
|
||||
<div class="fullcol">
|
||||
The results are clear: Deployment time went from several hours to tens of seconds. Efficiency has improved by 20-30%, measured in IT costs. But perhaps the best indication of success was the annual <a href="https://JD.comcorporateblog.com/shoppers-snap-up-quality-and-imported-products-on-JD.com-com-for-record-breaking-singles-day-festival/">Singles Day</a> shopping event, which ran on the Kubernetes platform for the first time in 2018. Over 11 days, transaction volume on JD.com was $23 billion, and "our e-commerce platforms did great," says Liu. "Infrastructure led the way to prep for 11.11. We took the approach of predicting volume, emulating the behavior of customers to prepare beforehand, and drilled for malfunctions. Because of Kubernetes’s scalability, we were able to handle an extremely high level of demand."
|
||||
<br><br>
|
||||
JD.com is now in its second stage with Kubernetes: The platform is already stable, scalable, and flexible, so the focus is on how to run things much more efficiently to further reduce costs. With the optimizations the team is working on with resource management, Liu believes there is the potential to save hundreds of millions of dollars a year.
|
||||
<br><br>
|
||||
"We run Kubernetes and container clusters on roughly tens of thousands of physical bare metal nodes," he says. "Using Kubernetes and leveraging our own machine learning pipeline to predict how many resources we need for each application we use, and our own intelligent scaling algorithm, we can improve our resource usage. If we boost the resource usage, for example, by several percent, that means we can reduce huge hardware costs. Then we don’t need that many servers to get that same amount of workload. That can save us a lot of resources."
|
||||
</div>
|
||||
|
||||
<div class="banner5" style="width:100%">
|
||||
<div class="banner5text">
|
||||
"We can share our successful experience with the community, and we also receive good feedback from others. So it’s mutually beneficial."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- HAIFENG LIU, CHIEF ARCHITECT, JD.com</span></div>
|
||||
</div>
|
||||
|
||||
<div class="fullcol">
|
||||
JD.com, which won CNCF’s 2018 End User Award, is also using <a href="https://helm.sh/">Helm</a>, <a href="https://github.com/containernetworking">CNI</a>, <a href="https://goharbor.io/">Harbor</a>, and <a href="https://vitess.io/">Vitess</a> on its platform. JD.com developers have made considerable contributions to Vitess, the CNCF project for scalable MySQL cluster management, and the company hopes to donate its own project to CNCF in the near future. Community participation is a priority for JD.com. "We have a good partnership with this community," says Liu. "We can share our successful experience with the community, and we also receive good feedback from others. So it’s mutually beneficial."
|
||||
<br><br>
|
||||
To that end, Liu offers this advice for other companies considering adopting cloud native technology. "First you need to combine this technology with your own businesses, and the second is you need clear goals," he says. "You cannot just use the technology because others are using it. You need to consider your own objectives."
|
||||
<br><br>
|
||||
For JD.com’s objectives, these cloud native technologies have been an ideal fit with the company’s own homegrown innovation. "Kubernetes helped us reduce the complexity of operations to make distributed systems stable and scalable," says Liu. "Most importantly, we can leverage Kubernetes for scheduling resources to reduce hardware costs. That’s the big win."
|
||||
</div>
|
||||
</section>
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 6.4 KiB |
@@ -6,10 +6,7 @@ case_study_styles: true
|
||||
cid: caseStudies
|
||||
css: /css/style_case_studies.css
|
||||
logo: naic_featured_logo.png
|
||||
featured: true
|
||||
weight: 3
|
||||
quote: >
|
||||
Our culture and technology transition is a strategy embraced by our top leaders. It has already proven successful by allowing us to accelerate our value pipeline by more than double while decreasing our costs by more than half.
|
||||
featured: false
|
||||
---
|
||||
|
||||
<div class="banner1" style="background-image: url('/images/CaseStudy_naic_banner1.jpg')">
|
||||
@@ -36,7 +33,7 @@ quote: >
|
||||
</div>
|
||||
|
||||
<div class="col2">
|
||||
|
||||
|
||||
|
||||
<h2>Impact</h2>
|
||||
Leveraging Kubernetes, "our development teams can create rapid prototypes far faster than they used to," Barker said. Applications running on Kubernetes are more resilient than those running in other environments. The deployment of open source solutions is helping influence company culture, as NAIC becomes a more open and transparent organization.
|
||||
@@ -57,7 +54,7 @@ quote: >
|
||||
<div class="fullcol">
|
||||
NAIC—which was created and overseen by the chief insurance regulators from the 50 states, the District of Columbia and five U.S. territories—provides a means through which state insurance regulators establish standards and best practices, conduct peer reviews, and coordinate their regulatory oversight. Their staff supports these efforts and represents the collective views of regulators in the United States and internationally. NAIC members, together with the organization’s central resources, form the national system of state-based insurance regulation in the United States.<br><br>
|
||||
The organization has been using the cloud for years, and wanted to find more ways to quickly deliver new services that provide more value for members and staff. They looked to Kubernetes for a solution. Within NAIC, several groups are leveraging Kubernetes, one being the Platform Engineering Team. "The team building out these tools are not only deploying and operating Kubernetes, but they’re also using them," Barker says. "In fact, we’re using GitLab to deploy Kubernetes with a pipeline using <a href="https://github.com/kubernetes/kops">kops</a>. This team was created from developers, operators, and quality engineers from across the company, so their jobs have changed quite a bit."<br><br>
|
||||
In addition, NAIC is onboarding teams to the new platform, and those teams have seen a lot of change in how they work and what they can do. "They now have more power in creating their own infrastructure and deploying their own applications," Barker says. They also use pipelines to facilitate their currently manual processes. NAIC has consumers who are using GitLab heavily, and they’re starting to use Kubernetes to deploy simple applications that help their internal processes.
|
||||
In addition, NAIC is onboarding teams to the new platform, and those teams have seen a lot of change in how they work and what they can do. "They now have more power in creating their own infrastructure and deploying their own applications," Barker says. They also use pipelines to facilitate their currently manual processes. NAIC has consumers who are using GitLab heavily, and they’re starting to use Kubernetes to deploy simple applications that help their internal processes.
|
||||
|
||||
|
||||
</div>
|
||||
@@ -71,7 +68,7 @@ In addition, NAIC is onboarding teams to the new platform, and those teams have
|
||||
<div class="fullcol">
|
||||
"We needed greater agility to enable our own productivity internally," he says. "We decided it was right for us to move everything to the public cloud [Amazon Web Services] to help with that process and be able to access many of the native tools that allows us to move faster by not needing to build everything."
|
||||
The NAIC also wanted to be cloud-agnostic, "and Kubernetes helps with this for our compute layer," Barker says. "Compute is pretty standard across the clouds, and now we can take advantage of any of them while getting all of the other features Kubernetes offers."<br><br>
|
||||
The NAIC currently hosts internal systems and development systems on Kubernetes, and has already seen how impactful it can be. "Our development teams can create rapid prototypes in minutes instead of weeks," Barker says. "This recently happened with an internal tool that had no measurable wait time on the infrastructure. It was solely development bound. There is now a central shared resource that lives in AWS, which means it can grow as needed."
|
||||
The NAIC currently hosts internal systems and development systems on Kubernetes, and has already seen how impactful it can be. "Our development teams can create rapid prototypes in minutes instead of weeks," Barker says. "This recently happened with an internal tool that had no measurable wait time on the infrastructure. It was solely development bound. There is now a central shared resource that lives in AWS, which means it can grow as needed."
|
||||
The native integrations into Kubernetes at NAIC has made it easy to write code and have it running in minutes instead of weeks. Applications running on Kubernetes have also proven to be more resilient than those running in other environments. "We even have teams using this to create more internal tools to help with communication or automating some of their current tasks," Barker says.
|
||||
<br><br>
|
||||
"We knew that Kubernetes had become the de facto standard for container orchestration," he says. "Two major factors for selecting this were the three major cloud vendors hosting their own versions and having it hosted in a neutral party as fully open source."
|
||||
@@ -89,7 +86,7 @@ As for other CNCF projects, NAIC is using Prometheus on a small scale and hopes
|
||||
|
||||
<section class="section5" style="padding:0px !important">
|
||||
<div class="fullcol">
|
||||
|
||||
|
||||
The open governance and broad industry participation in CNCF provided a comfort level with the technology, Barker says. "We also see it as helping to influence our own company culture," he says. "We’re moving to be a more open and transparent company, and we are encouraging our staff to get involved with the different working groups and codebases. We recently became CNCF members to help further our commitment to community contribution and transparency."<br><br>
|
||||
Factors such as vendor-neutrality and cross-industry investment were important in the selection. "In our experience, vendor lock-in and tooling that is highly specific results in less resilient technology with fewer minds working to solve problems and grow the community," Barker says.<br><br>
|
||||
NAIC is a largely Oracle shop, Barker says, and has been running mostly Java on JBoss. "However, we have years of history with other applications," he says. "Some of these have been migrated by completely rewriting the application, while others are just being modified slightly to fit into this new paradigm."<br><br>
|
||||
@@ -106,7 +103,7 @@ All of its databases are currently running within public cloud services, but the
|
||||
</div>
|
||||
</div>
|
||||
<div class="fullcol">
|
||||
|
||||
|
||||
NAIC has seen a significant business impact from its efforts. "We have been able to move much faster at lower cost than we were able to in the past," Barker says. "We were able to complete one of our projects in a year, when the previous version took over two years. And the new project cost $500,000 while the original required $3 million, and with fewer defects. We are also able to push out new features much faster."
|
||||
He says the organization is moving toward continuous deployment "because the business case makes sense. The research is becoming very hard to argue with. We want to reduce our batch sizes and optimize on delivering value to customers and not feature count. This is requiring a larger cultural shift than just a technology shift."
|
||||
NAIC is "becoming more open and transparent, as well as more resilient to failure," Barker says. "Even our customers are wanting more and more of this and trying to figure out how they can work with us to accomplish our mutual goals faster. Members of the insurance industry have reached out so that we can better learn together and grow as an industry."
|
||||
|
||||
@@ -1,8 +1,15 @@
|
||||
---
|
||||
title: Nav Case Study
|
||||
|
||||
linkTitle: Nav
|
||||
case_study_styles: true
|
||||
cid: caseStudies
|
||||
css: /css/style_case_studies.css
|
||||
logo: nav_featured_logo.png
|
||||
featured: true
|
||||
weight: 3
|
||||
quote: >
|
||||
The community is absolutely vital: being able to pass ideas around, talk about a lot of the similar challenges that we’re all facing, and just get help. I like that we’re able to tackle the same problems for different reasons but help each other along the way.
|
||||
---
|
||||
|
||||
<div class="banner1" style="background-image: url('/images/CaseStudy_nav_banner1.jpg')">
|
||||
|
||||
@@ -0,0 +1,96 @@
|
||||
---
|
||||
title: Prowise Case Study
|
||||
linkTitle: prowise
|
||||
case_study_styles: true
|
||||
cid: caseStudies
|
||||
css: /css/style_case_studies.css
|
||||
featured: false
|
||||
---
|
||||
<div class="banner1" style="background-image: url('/images/CaseStudy_nerdalize_banner1.jpg')">
|
||||
<h1> CASE STUDY:<img src="/images/nerdalize_logo.png" class="header_logo" style="width:25%;margin-bottom:-1%"><br> <div class="subhead" style="margin-top:1%">Nerdalize: Providing Affordable and Sustainable Cloud Hosting with Kubernetes
|
||||
</div></h1>
|
||||
|
||||
</div>
|
||||
|
||||
<div class="details">
|
||||
Company <b>Nerdalize</b> Location <b>Delft, Netherlands </b> Industry <b>Cloud Provider</b>
|
||||
</div>
|
||||
|
||||
<hr>
|
||||
<section class="section1">
|
||||
<div class="cols">
|
||||
<div class="col1" style="width:100%"">
|
||||
|
||||
<h2>Challenge</h2>
|
||||
Nerdalize offers affordable cloud hosting for customers—and free heat and hot water for people who sign up to house the heating devices that contain the company’s servers. The savings Nerdalize realizes by not running data centers are passed on to its customers. When the team began using Docker to make its software more portable, it realized it also needed a container orchestration solution. “As a cloud provider, we have internal services for hosting our backends and billing our customers, but we also need to offer our compute to our end users,” says Digital Product Engineer Ad van der Veer. “Since we have these heating devices spread across the Netherlands, we need some way of tying that all together.”
|
||||
<h2>Solution</h2>
|
||||
After briefly using a basic scheduling setup with another open source tool, Nerdalize switched to Kubernetes. “On top of our heating devices throughout the Netherlands, we have a virtual machine layer, and on top of that we run Kubernetes clusters for our customers,” says van der Veer. “As a small company, we have to provide a rock solid story in terms of the technology. Kubernetes allows us to offer a hybrid solution: ‘You can run this on our cloud, but you can run it on other clouds as well. It runs in your internal hardware if you like.’ And together with the Docker image standard and our multi-cloud dashboard, that allows them peace of mind.”
|
||||
<h2>Impact</h2>
|
||||
Nerdalize prides itself on being a Kubernetes-native cloud provider that charges its customers prices 40% below that of other cloud providers. “Every euro that we have to invest for licensing of software that’s not open source comes from that 40%,” says van der Veer. If they had used a non-open source orchestration platform instead of Kubernetes, “that would reduce this proposition that we have of 40% less cost to like 30%. Kubernetes directly allows us to have this business model and this strategic advantage.” Nerdalize customers also benefit from time savings: One went from spending a day to set up VMs, network, and software, to spinning up a Kubernetes cluster in minutes. And for households using the heating devices, they save an average of 200 euro a year on their heating bill. The environmental impact? The annual reduction in CO2 emissions comes out to be 2 tons per Nerdalize household, which is equivalent to a car driving 8,000 km.
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner2">
|
||||
<div class="banner2text">
|
||||
“We can walk into a boardroom and put a Kubernetes logo up, and people accept it as an established technology. It becomes this centerpiece where other cloud native projects can tie in, so there’s a network effect that each project empowers each other. This is something that has a lot of value when we have to talk to customers and convince them that our cloud fits their needs.”
|
||||
<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>— AD VAN DER VEER, PRODUCT ENGINEER, NERDALIZE</span>
|
||||
</div>
|
||||
</div>
|
||||
<section class="section2">
|
||||
<div class="fullcol">
|
||||
<h2>Nerdalize is a cloud hosting provider that has no data centers. Instead, the four-year-old startup places its servers in homes across the Netherlands, inside heating devices it developed to turn the heat produced by the servers into heating and hot water for the residents.
|
||||
</h2>
|
||||
“Households save on their gas bills, and cloud users have a much more sustainable cloud solution,” says Maaike Stoops, Customer Experience Queen at Nerdalize. “And we don’t have the overhead of building a data center, so our cloud is up to 40% more affordable.”
|
||||
<br><br>
|
||||
That business model has been enabled by the company’s adoption of containerization and Kubernetes. “When we just got started, Docker was just introduced,” says Digital Product Engineer Ad van der Veer. “We began with a very basic bare metal setup, but once we developed the business, we saw that containerization technology was super useful to help our customers. As a cloud provider, we have internal services for hosting our backends and billing our customers, but we also need to offer our compute to our end users. Since we have these heating devices spread across the Netherlands, we need some way of tying that all together.”
|
||||
<br><br>
|
||||
After trying to develop its own scheduling system using another open source tool, Nerdalize found Kubernetes. “Kubernetes provided us with more functionality out of the gate,” says van der Veer.
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner3" style="background-image: url('/images/CaseStudy_nerdalize_banner3.jpg')">
|
||||
<div class="banner3text">
|
||||
“We always try to get a working version online first, like minimal viable products, and then move to stabilize that,” says van der Veer. “And I think that these kinds of day-two problems are now immediately solved. The rapid prototyping we saw internally is a very valuable aspect of Kubernetes.”<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>— AD VAN DER VEER, PRODUCT ENGINEER, NERDALIZE</span>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
<section class="section3">
|
||||
<div class="fullcol">
|
||||
The team first experimented with a basic use case to run customers’ workloads on Kubernetes. “Getting the data working was kind of difficult, and at the time the installation wasn’t that simple,” says van der Veer. “Then CNCF started, we saw the community grow, these problems got solved, and from there it became a very easy decision.”
|
||||
<br><br>
|
||||
The first Nerdalize product that was launched in 2017 was “100% containerized and Kubernetes native,” says van der Veer. “On top of our heating devices throughout the Netherlands, we have a virtual machine layer, and on top of that we run Kubernetes clusters for our customers. As a small company, we have to provide a rock solid story in terms of the technology. Kubernetes allows us to offer a hybrid solution: ‘You can run this on our cloud, but you can run it on other clouds as well. It runs in your internal hardware if you like.’ And together with the Docker image standard and our multi-cloud dashboard, that gives them peace of mind.”
|
||||
<br><br>
|
||||
Not to mention the 40% cost savings. “Every euro that we have to invest for licensing of software that’s not open source comes from that 40%,” says van der Veer. If Nerdalize had used a non-open source orchestration platform instead of Kubernetes, “that would reduce our cost savings proposition to like 30%. Kubernetes directly allows us to have this business model and this strategic advantage.”
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner4" style="background-image: url('/images/CaseStudy_nerdalize_banner4.jpg')" style="width:100%">
|
||||
<div class="banner4text">
|
||||
“One of our customers used to spend up to a day setting up the virtual machines, network and software every time they wanted to run a project in the cloud. On our platform, with Docker and Kubernetes, customers can have their projects running in a couple of minutes.”
|
||||
<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- MAAIKE STOOPS, CUSTOMER EXPERIENCE QUEEN, NERDALIZE</span>
|
||||
</div>
|
||||
</div>
|
||||
<section class="section5" style="padding:0px !important">
|
||||
<div class="fullcol">
|
||||
Nerdalize now has customers, from individual engineers to data-intensive startups and established companies, all around the world. (For the time being, though, the heating devices are exclusive to the Netherlands.) One of the most common use cases is batch workloads used by data scientists and researchers, and the time savings for these end users is profound. “One of our customers used to spend up to a day setting up the virtual machines, network and software every time they wanted to run a project in the cloud,” says Stoops. “On our platform, with Docker and Kubernetes, customers can have their projects running in a couple of minutes.”
|
||||
<br><br>
|
||||
As for households using the heating devices, they save an average of 200 euro a year on their heating bill. The environmental impact? The annual reduction in CO2 emissions comes out to 2 tons per Nerdalize household, which is equivalent to a car driving 8,000 km.
|
||||
<br><br>
|
||||
For the Nerdalize team, feature development—such as the accessible command line interface called Nerd, which recently went live—has also been sped up by Kubernetes. “We always try to get a working version online first, like minimal viable products, and then move to stabilize that,” says van der Veer. “And I think that these kinds of day-two problems are now immediately solved. The rapid prototyping we saw internally is a very valuable aspect of Kubernetes.”
|
||||
<br><br>
|
||||
Another unexpected benefit has been the growing influence and reputation of Kubernetes. “We can walk into a boardroom and put a Kubernetes logo up, and people accept it as an established technology,” says van der Veer. “It becomes this centerpiece where other cloud native projects can tie in, so there’s a network effect that each project empowers each other. This is something that has a lot of value when we have to convince customers that our cloud fits their needs.”
|
||||
</div>
|
||||
|
||||
<div class="banner5" >
|
||||
<div class="banner5text">
|
||||
“It shouldn’t be too big of a hassle and too large of a commitment. It should be fun and easy for end users. So we really love Kubernetes in that way.”<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- MAAIKE STOOPS, CUSTOMER EXPERIENCE QUEEN, NERDALIZE</span></div>
|
||||
</div>
|
||||
|
||||
<div class="fullcol">
|
||||
|
||||
In fact, Nerdalize is currently looking into implementing other CNCF projects, such as <a href="https://prometheus.io/">Prometheus</a> for monitoring and <a href="https://rook.io/">Rook</a>, “which should help us with some of the data problems that we want to solve for our customers,” says van der Veer.
|
||||
<br><br>
|
||||
In the coming year, Nerdalize will scale up the number of households running its hardware to 50, or the equivalent of a small scale data center. Geographic redundancy and greater server ability for customers are two main goals. Spreading the word about Kubernetes is also in the game plan. “We offer a free namespace on our sandbox, multi-tenant Kubernetes cluster for anyone to try,” says van der Veer. “What’s more cool than trying your first Kubernetes project on houses, to warm a shower?”
|
||||
<br><br>
|
||||
Ultimately, this ties into Nerdalize’s mission of supporting affordable and sustainable cloud hosting. “We want to be the disrupter of the cloud space, showing organizations that running in the cloud is easy and affordable,” says Stoops. “It shouldn’t be too big of a hassle and too large of a commitment. It should be fun and easy for end users. So we really love Kubernetes in that way.”
|
||||
</div>
|
||||
|
||||
</section>
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 10 KiB |
@@ -1,8 +1,11 @@
|
||||
---
|
||||
title: NetEase Case Study
|
||||
linkTitle: NetEase
|
||||
case_study_styles: true
|
||||
cid: caseStudies
|
||||
css: /css/style_case_studies.css
|
||||
logo: netease_featured_logo.png
|
||||
featured: false
|
||||
---
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,93 @@
|
||||
---
|
||||
title: Nokia Case Study
|
||||
linkTitle: Nokia
|
||||
case_study_styles: true
|
||||
cid: caseStudies
|
||||
css: /css/style_case_studies.css
|
||||
logo: nokia_featured_logo.png
|
||||
---
|
||||
|
||||
|
||||
<div class="banner1" style="background-image: url('/images/CaseStudy_nokia_banner1.jpg')">
|
||||
<h1> CASE STUDY:<img src="/images/nokia_logo.png" class="header_logo" style="width:20%;margin-bottom:-2.2%"><br> <div class="subhead" style="margin-top:1%">Nokia: Enabling 5G and DevOps at a Telecom Company with Kubernetes
|
||||
|
||||
</div></h1>
|
||||
|
||||
</div>
|
||||
|
||||
<div class="details">
|
||||
Company <b>Nokia</b> Location <b>Espoo, Finland
|
||||
</b> Industry <b>Telecommunications</b>
|
||||
</div>
|
||||
|
||||
<hr>
|
||||
<section class="section1">
|
||||
<div class="cols">
|
||||
<div class="col1" style="width:100%"">
|
||||
<h2>Challenge</h2>
|
||||
<a href="https://www.nokia.com/en_int">Nokia</a>’s core business is building telecom networks end-to-end; its main products are related to the infrastructure, such as antennas, switching equipment, and routing equipment. "As telecom vendors, we have to deliver our software to several telecom operators and put the software into their infrastructure, and each of the operators have a bit different infrastructure," says Gergely Csatari, Senior Open Source Engineer. "There are operators who are running on bare metal. There are operators who are running on virtual machines. There are operators who are running on <a href="https://cloud.vmware.com/">VMware Cloud</a> and <a href="https://www.openstack.org/">OpenStack</a> Cloud. We want to run the same product on all of these different infrastructures without changing the product itself."
|
||||
|
||||
<h2>Solution</h2>
|
||||
The company decided that moving to cloud native technologies would allow teams to have infrastructure-agnostic behavior in their products. Teams at Nokia began experimenting with Kubernetes in pre-1.0 versions. "The simplicity of the label-based scheduling of Kubernetes was a sign that showed us this architecture will scale, will be stable, and will be good for our purposes," says Csatari. The first Kubernetes-based product, the <a href="https://networks.nokia.com/products/telecom-application-server">Nokia Telephony Application Server</a>, went live in early 2018. "Now, all the products are doing some kind of re-architecture work, and they’re moving to Kubernetes."
|
||||
<br>
|
||||
<h2>Impact</h2>
|
||||
Kubernetes has enabled Nokia’s foray into 5G. "When you develop something that is part of the operator’s infrastructure, you have to develop it for the future, and Kubernetes and containers are the forward-looking technologies," says Csatari. The teams using Kubernetes are already seeing clear benefits. "By separating the infrastructure and the application layer, we have less dependencies in the system, which means that it’s easier to implement features in the application layer," says Csatari. And because teams can test the exact same binary artifact independently of the target execution environment, "we find more errors in early phases of the testing, and we do not need to run the same tests on different target environments, like VMware, OpenStack, or bare metal," he adds. As a result, "we save several hundred hours in every release."
|
||||
|
||||
</div>
|
||||
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner2">
|
||||
<div class="banner2text">
|
||||
"When people are picking up their phones and making a call on Nokia networks, they are creating containers in the background with Kubernetes."
|
||||
<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Gergely Csatari, Senior Open Source Engineer, Nokia</span>
|
||||
</div>
|
||||
</div>
|
||||
<section class="section2">
|
||||
<div class="fullcol">
|
||||
<h2>Nokia was the first name in mobile phones when they were becoming ubiquitous in the late 1990s and early 2000s. But by 2014, the company had sold off its mobile device division and was focusing its core business not on the handhelds used for calls, but on the networks.</h2>
|
||||
Today, Nokia is building telecom networks end-to-end—from antennas to switching and routing equipment—serving operators in more than 120 countries. "As telecom vendors, we have to deliver our software to several telecom operators and put the software into their infrastructure, and each of the operators have a bit different infrastructure," says Gergely Csatari, Senior Open Source Engineer at Nokia. "There are operators who are running on bare metal. There are operators who are running on virtual machines. There are operators who are running on VMware Cloud and OpenStack Cloud. We want to run the same product on all of these different infrastructures without changing the product itself."<br><br>
|
||||
Looking for a way to allow its teams to build products with infrastructure-agnostic behavior, the company decided to embrace containerization, Kubernetes, and other cloud native technologies, a move that is being made across the telecom industry. Since early 2018, "when people are picking up their phones and making a call on Nokia networks, they are creating containers in the background with Kubernetes," says Csatari. "Now, all the products are doing some kind of re-architecture work, and they’re moving to Kubernetes."
|
||||
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner3" style="background-image: url('/images/CaseStudy_nokia_banner3.jpg')">
|
||||
<div class="banner3text">
|
||||
"Having the community and CNCF around Kubernetes is not only important for having a connection to other companies who are using Kubernetes and a forum where you can ask or discuss features of Kubernetes. But as a company who would like to contribute to Kubernetes, it was very important to have a CLA (Contributors License Agreement) which is connected to the CNCF and not to a particular company. That was a critical step for us to start contributing to Kubernetes and Helm."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Gergely Csatari, Senior Open Source Engineer, Nokia</span>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
<section class="section3">
|
||||
<div class="fullcol">
|
||||
Nokia’s cloud native journey began about two years ago, when Csatari’s team was building the company’s Telephony Application Server (TAS). "We wanted to have a service execution engine in the product, which was a totally separate function from all other parts," he says. "There, we had the possibility to think about new architectures and new tools that we could use. We created this particular product based on Kubernetes, and we liked the work, so we started to talk about cloud native and containers and all of these things. We did a very extensive research of different container orchestration tools. We knew that we have some, let’s say, strange or different requirements because of the special environment that our software is running on."<br><br>
|
||||
For one thing, Nokia’s software serves millions of people, and is required to have the carrier-grade "five nines" availability: to be up 99.999% of the time. "If you turn it to minutes, this means we’re allowed to have only 10 minutes of downtime in a whole year," says Csatari. "Downtime here means that you are not able to serve the person to full capacity, which means that we cannot fail. This includes software upgrades, everything, because when you call 911, you’re using our software, and you expect that it will work."<br><br>
|
||||
That meant that they needed to be able to set affinity and anti-affinity rules in their orchestration tools. "You cannot put all of the functions to the same physical host because physical hosts are failing," Csatari explains. "If you fail with one physical host, then you lose all of the core processing processes. Then there are no calls going through. So we have to divide them among the different physical hosts. At that time, only Kubernetes was able to provide these features. The simplicity of the label-based scheduling of Kubernetes was a sign that showed us this architecture will scale, will be stable, and will be good for our purposes."
|
||||
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner4" style="background-image: url('/images/CaseStudy_nokia_banner4.jpg')" style="width:100%">
|
||||
<div class="banner4text">
|
||||
"Kubernetes opened the window to all of these open source projects instead of implementing everything in house. Our engineers can focus more on the application level, which is actually the thing what we are selling, and not on the infrastructure level. For us, the most important thing about Kubernetes is it allows us to focus on value creation of our business." <br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Gergely Csatari, Senior Open Source Engineer, Nokia</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<section class="section5" style="padding:0px !important">
|
||||
<div class="fullcol">
|
||||
The TAS went live in early 2018, and now Kubernetes is also enabling Nokia’s foray into 5G. The company is introducing microservices architecture and Kubernetes while adding 5G features to existing products. And all new 5G product development will be on top of Kubernetes. "When you develop something that is part of the operator’s infrastructure, you have to develop it for the future, and Kubernetes and containers are the forward-looking technologies," says Csatari. <br><br>
|
||||
There have been real time savings thanks to Kubernetes. "By separating the infrastructure and the application layer, we have less dependencies in the system, which means that it’s easier to implement features in the application layer," says Csatari. Because teams can test the exact same binary artifact independently of the target execution environment, "we find more errors in early phases of the testing, and we do not need to run the same tests on different target environments, like VMware, OpenStack or bare metal," he adds. As a result, "we save several hundred hours in every release."<br><br>
|
||||
Moving from Nokia’s legacy cluster management system, which had been built in-house more than thirty years ago, to a Kubernetes platform also meant that "we started using Linux as a base operating system, so we just opened the window to all of these open source projects instead of implementing everything in house," says Csatari. (From CNCF’s ecosystem, the team is already using <a href="https://helm.sh/">Helm</a>, <a href="https://grpc.io/">gRPC</a>, <a href="https://github.com/containernetworking">CNI</a>, <a href="https://prometheus.io/">Prometheus</a>, and <a href="https://www.envoyproxy.io/">Envoy</a>, and plans to implement <a href="https://coredns.io/">CoreDNS</a>.) "Our engineers can focus more on the application level, which is actually the thing what we are selling, and not on the infrastructure level. For us, the most important thing about Kubernetes is it allows us to focus on value creation of our business."
|
||||
|
||||
</div>
|
||||
|
||||
<div class="banner5" style="width:100%">
|
||||
<div class="banner5text">
|
||||
"I had some discussions at KubeCon with people from the networking SIG and the resource management working group, to work together on our requirements, and that’s very exciting for me and my colleagues,"<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Gergely Csatari, Senior Open Source Engineer, Nokia</span></div>
|
||||
</div>
|
||||
|
||||
<div class="fullcol">
|
||||
The company has a long-term goal of moving the entire product portfolio into the Kubernetes platform. To that end, Nokia teams are working together with other companies to add the features needed to use Kubernetes with the real-time, nanosecond-sensitive applications close to the edge of the radio network. <br><br>
|
||||
And the CNCF community is proving to be a great forum for that collaboration. "I had some discussions at KubeCon with people from the networking SIG and the resource management working group, to work together on our requirements, and that’s very exciting for me and my colleagues," says Csatari. "Previously, everybody had the same problem, but everybody just did it in his own, and now we are trying to solve the same problem together."<br><br>
|
||||
Perhaps the biggest impact that Kubernetes is having on Nokia, Csatari believes, is that people are starting to think about how a telecom company can do DevOps. "We are building a DevOps pipeline, which reaches from the actual developer to the customers, and thinking about new ways how can we digitally deliver our software to our customers and get feedback from the customers right to the engineers," he says. "This is something that will fundamentally change how telecom companies are delivering software, and how quickly can we develop new features. This is because of the usage of containers and, of course, the usage of Kubernetes."
|
||||
|
||||
</div>
|
||||
</section>
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 4.2 KiB |
@@ -0,0 +1,96 @@
|
||||
---
|
||||
title: pingcap Case Study
|
||||
linkTitle: pingcap
|
||||
case_study_styles: true
|
||||
cid: caseStudies
|
||||
css: /css/style_case_studies.css
|
||||
featured: false
|
||||
---
|
||||
|
||||
<div class="banner1" style="background-image: url('/images/CaseStudy_pingcap_banner1.jpg')">
|
||||
<h1> CASE STUDY:<img src="/images/pingcap_logo.png" class="header_logo" style="width:20%;margin-bottom:-1.5%"><br> <div class="subhead" style="margin-top:1%">PingCAP Bets on Cloud Native for Its TiDB Database Platform
|
||||
|
||||
</div></h1>
|
||||
|
||||
</div>
|
||||
|
||||
<div class="details">
|
||||
Company <b>PingCAP</b> Location <b>Beijing, China, and San Mateo, CA</b> Industry <b>Software</b>
|
||||
</div>
|
||||
|
||||
<hr>
|
||||
<section class="section1">
|
||||
<div class="cols">
|
||||
<div class="col1" style="width:100%"">
|
||||
<h2>Challenge</h2>
|
||||
PingCAP is the company leading the development of the popular open source NewSQL database <a href="https://github.com/pingcap/tidb">TiDB</a>, which is MySQL-compatible, can handle hybrid transactional and analytical processing (HTAP) workloads, and has a cloud native architectural design. "Having a hybrid multi-cloud product is an important part of our global go-to-market strategy," says Kevin Xu, General Manager of Global Strategy and Operations. In order to achieve that, the team had to address two challenges: "how to deploy, run, and manage a distributed stateful application, such as a distributed database like TiDB, in a containerized world," Xu says, and "how to deliver an easy-to-use, consistent, and reliable experience for our customers when they use TiDB in the cloud, any cloud, whether that’s one cloud provider or a combination of different cloud environments." Knowing that using a distributed system isn’t easy, they began looking for the right orchestration layer to help reduce some of that complexity for end users.
|
||||
<h2>Solution</h2>
|
||||
The team started looking at Kubernetes for orchestration early on. "We knew Kubernetes had the promise of helping us solve our problems," says Xu. "We were just waiting for it to mature." In early 2018, PingCAP began integrating Kubernetes into its internal development as well as in its TiDB product. At that point, the team has already had experience using other cloud native technologies, having integrated both <a href="https://prometheus.io/">Prometheus</a> and <a href="https://grpc.io/">gRPC</a> as parts of the TiDB platform earlier on.
|
||||
<br>
|
||||
<h2>Impact</h2>
|
||||
Xu says that PingCAP customers have had a "very positive" response so far to Kubernetes being the tool to deploy and manage TiDB. Prometheus, with <a href="https://grafana.com/">Grafana</a> as the dashboard, is installed by default when customers deploy TiDB, so that they can monitor performance and make any adjustments needed to reach their target before and while deploying TiDB in production. That monitoring layer "makes the evaluation process and communication much smoother," says Xu.
|
||||
<br><br>
|
||||
With the company’s <a href="https://github.com/pingcap/tidb-operator">Kubernetes-based Operator implementation</a>, which is open sourced, customers are now able to deploy, run, manage, upgrade, and maintain their TiDB clusters in the cloud with no downtime, and reduced workload, burden and overhead. And internally, says Xu, "we’ve completely switched to Kubernetes for our own development and testing, including our data center infrastructure and <a href="https://thenewstack.io/chaos-tools-and-techniques-for-testing-the-tidb-distributed-newsql-database/">Schrodinger</a>, an automated testing platform for TiDB. With Kubernetes, our resource usage is greatly improved. Our developers can allocate and deploy clusters themselves, and the deploying process has gone from hours to minutes, so we can devote fewer people to manage IDC resources. The productivity improvement is about 15%, and as we gain more Kubernetes knowledge on the debugging and diagnosis front, the productivity should improve to more than 20%."
|
||||
|
||||
</div>
|
||||
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner2">
|
||||
<div class="banner2text">
|
||||
"We knew Kubernetes had the promise of helping us solve our problems. We were just waiting for it to mature, so we can fold it into our own development and product roadmap."
|
||||
<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- KEVIN XU, GENERAL MANAGER OF GLOBAL STRATEGY AND OPERATIONS, PINGCAP</span>
|
||||
</div>
|
||||
</div>
|
||||
<section class="section2">
|
||||
<div class="fullcol">
|
||||
<h2>Since it was introduced in 2015, the open source NewSQL database TiDB has gained a following for its compatibility with MySQL, its ability to handle hybrid transactional and analytical processing (HTAP) workloads—and its cloud native architectural design.</h2>
|
||||
PingCAP, the company behind TiDB, designed the platform with cloud in mind from day one, says Kevin Xu, General Manager of Global Strategy and Operations, and "having a hybrid multi-cloud product is an important part of our global go-to-market strategy."
|
||||
<br><br>
|
||||
In order to achieve that, the team had to address two challenges: "how to deploy, run, and manage a distributed stateful application, such as a distributed database like TiDB, in a containerized world," Xu says, and "how to deliver an easy-to-use, consistent, and reliable experience for our customers when they use TiDB in the cloud, any cloud, whether that’s one cloud provider or a combination of different cloud environments."
|
||||
<br><br>
|
||||
Knowing that using a distributed system isn’t easy, the PingCAP team began looking for the right orchestration layer to help reduce some of that complexity for end users. Kubernetes had been on their radar for quite some time. "We knew Kubernetes had the promise of helping us solve our problems," says Xu. "We were just waiting for it to mature."
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner3" style="background-image: url('/images/CaseStudy_pingcap_banner3.jpg')">
|
||||
<div class="banner3text">
|
||||
"With the governance process being so open, it’s not hard to find out what’s the latest development in the technology and community, or figure out who to reach out to if we have problems or issues."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- KEVIN XU, GENERAL MANAGER OF GLOBAL STRATEGY AND OPERATIONS, PINGCAP</span>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
<section class="section3">
|
||||
<div class="fullcol">
|
||||
That time came in early 2018, when PingCAP began integrating Kubernetes into its internal development as well as in its TiDB product. "Having Kubernetes be part of the CNCF, as opposed to having only the backing of one individual company, was valuable in having confidence in the longevity of the technology," says Xu. Plus, "with the governance process being so open, it’s not hard to find out what’s the latest development in the technology and community, or figure out who to reach out to if we have problems or issues."
|
||||
<br><br>
|
||||
TiDB’s cloud native architecture consists of a stateless SQL layer (also called TiDB) and a persistent key-value storage layer that supports distributed transactions (<a href="https://github.com/tikv/tikv">TiKV</a>, which is now in the CNCF Sandbox), which are loosely coupled. "You can scale both out or in depending on your computation and storage needs, and the two scaling processes can happen independent of each other," says Xu. The PingCAP team also built the <a href="https://github.com/pingcap/tidb-operator">TiDB Operator</a> based on Kubernetes, which helps bootstrap a TiDB cluster on any cloud environment and simplifies and automates deployment, scaling, scheduling, upgrades, and maintenance. The company also recently previewed its fully-managed <a href="https://www.pingcap.com/blog/announcing-tidb-cloud-managed-as-a-service-and-in-the-marketplace/">TiDB Cloud</a> offering.
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner4" style="background-image: url('/images/CaseStudy_pingcap_banner4.jpg')">
|
||||
<div class="banner4text">
|
||||
"A cloud native infrastructure will not only save you money and allow you to be more in control of the infrastructure resources you consume, but also empower new product innovation, new experience for your users, and new business possibilities. It’s both a cost reducer and a money maker." <br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- KEVIN XU, GENERAL MANAGER OF GLOBAL STRATEGY AND OPERATIONS, PINGCAP</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<section class="section5" style="padding:0px !important">
|
||||
<div class="fullcol">
|
||||
The entire TiDB platform leverages Kubernetes and other cloud native technologies, including <a href="https://prometheus.io/">Prometheus</a> for monitoring and <a href="https://grpc.io/">gRPC</a> for interservice communication.
|
||||
<br><br>
|
||||
So far, the customer response to the Kubernetes-enabled platform has been "very positive." Prometheus, with <a href="https://grafana.com/">Grafana</a> as the dashboard, is installed by default when customers deploy TiDB, so that they can monitor and make any adjustments needed to reach their performance requirements before deploying TiDB in production. That monitoring layer "makes the evaluation process and communication much smoother," says Xu. With the company’s Kubernetes-based Operator implementation, customers are now able to deploy, run, manage, upgrade, and maintain their TiDB clusters in the cloud with no downtime, and reduced workload, burden and overhead.
|
||||
<br><br>
|
||||
These technologies have also had an impact internally. "We’ve completely switched to Kubernetes for our own development and testing, including our data center infrastructure and <a href="https://thenewstack.io/chaos-tools-and-techniques-for-testing-the-tidb-distributed-newsql-database/">Schrodinger</a>, an automated testing platform for TiDB," says Xu. "With Kubernetes, our resource usage is greatly improved. Our developers can allocate and deploy clusters themselves, and the deploying process takes less time, so we can devote fewer people to manage IDC resources.
|
||||
</div>
|
||||
|
||||
<div class="banner5" >
|
||||
<div class="banner5text">
|
||||
"The entire cloud native community, whether it’s Kubernetes, CNCF in general, or cloud native vendors like us, have all gained enough experience—and have the battle scars to prove it—and are ready to help you succeed."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- KEVIN XU, GENERAL MANAGER OF GLOBAL STRATEGY AND OPERATIONS, PINGCAP</span></div>
|
||||
</div>
|
||||
|
||||
<div class="fullcol">
|
||||
The productivity improvement is about 15%, and as we gain more Kubernetes knowledge on the debugging and diagnosis front, the productivity should improve to more than 20%."
|
||||
<br><br>
|
||||
Kubernetes is now a crucial part of PingCAP’s product roadmap. For anyone else considering going cloud native, Xu has this advice: "There’s no better time to get started," he says. "The entire cloud native community, whether it’s Kubernetes, CNCF in general, or cloud native vendors like us, have all gained enough experience—and have the battle scars to prove it—and are ready to help you succeed."
|
||||
<br><br>
|
||||
In fact, the PingCAP team has seen more and more customers moving toward a cloud native approach, and for good reason. "IT infrastructure is quickly evolving from a cost-center and afterthought, to the core competency and competitiveness of any company," says Xu. "A cloud native infrastructure will not only save you money and allow you to be more in control of the infrastructure resources you consume, but also empower new product innovation, new experience for your users, and new business possibilities. It’s both a cost reducer and a money maker."
|
||||
|
||||
</div>
|
||||
</section>
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 6.9 KiB |
@@ -0,0 +1,99 @@
|
||||
---
|
||||
title: Prowise Case Study
|
||||
linkTitle: prowise
|
||||
case_study_styles: true
|
||||
cid: caseStudies
|
||||
css: /css/style_case_studies.css
|
||||
featured: false
|
||||
---
|
||||
|
||||
|
||||
<div class="banner1" style="background-image: url('/images/CaseStudy_prowise_banner1.jpg')">
|
||||
<h1> CASE STUDY:<img src="/images/prowise_logo.png" class="header_logo" style="width:25%;margin-bottom:-1%"><br> <div class="subhead" style="margin-top:1%">Prowise: How Kubernetes is Enabling the Edtech Solution’s Global Expansion
|
||||
</div></h1>
|
||||
|
||||
</div>
|
||||
|
||||
<div class="details">
|
||||
Company <b>Prowise</b> Location <b>Budel, The Netherlands </b> Industry <b>Edtech</b>
|
||||
</div>
|
||||
|
||||
<hr>
|
||||
<section class="section1">
|
||||
<div class="cols">
|
||||
<div class="col1" style="width:100%"">
|
||||
|
||||
<h2>Challenge</h2>
|
||||
A Dutch company that produces educational devices and software used around the world, <a href="https://www.prowise.com/en/">Prowise</a> had an infrastructure based on Linux services with multiple availability zones in Europe, Australia, and the U.S. “We’ve grown a lot in the past couple of years, and we started to encounter problems with versioning and flexible scaling,” says Senior DevOps Engineer Victor van den Bosch, “not only scaling in demands, but also in being able to deploy multiple products which all have their own versions, their own development teams, and their own problems that they’re trying to solve. To be able to put that all on the same platform without much resistance is what we were looking for. We wanted to future proof our infrastructure, and also solve some of the problems that are associated with just running a normal Linux service.”
|
||||
<h2>Solution</h2>
|
||||
The Prowise team adopted containerization, spent time improving its CI/CD pipelines, and chose Microsoft Azure’s managed Kubernetes service, <a href="https://azure.microsoft.com/en-us/services/kubernetes-service/">AKS</a>, for orchestration. “Kubernetes solves things like networking really well, in a way that fits our business model,” says van den Bosch. “We want to focus on our core products, and that’s the software that runs on it and not necessarily the infrastructure itself.”
|
||||
<h2>Impact</h2>
|
||||
With its first web-based applications now running in beta on Prowise’s Kubernetes platform, the team is seeing the benefits of rapid and smooth deployments. “The old way of deploying took half an hour of preparations and half an hour deploying it. With Kubernetes, it’s a couple of seconds,” says Senior Developer Bart Haalstra. As a result, adds van den Bosch, “We’ve gone from quarterly releases to a release every month in production. We’re pretty much deploying every hour or just when we find that a feature is ready for production; before, our releases were mostly done on off-hours, where it couldn’t impact our customers, as our confidence in the process was relatively low. Kubernetes has also enabled us to follow up quickly on bugs and implement tweaks to our users with zero downtime between versions. For some bugs we’ve pushed code fixes to production minutes after detection.” Recently, the team launched a new single sign-on solution for use in an internal application. “Due to the resource based architecture of the Kubernetes platform, we were able to bring that application into an entirely new production environment in less than a day, most of that time used for testing after applying the already well-known resource definitions from staging to the new environment,” says van den Bosch. “On a traditional VM this would have likely cost a day or two, and then probably a few weeks to iron out the kinks in our provisioning scripts as we apply updates.”
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner2">
|
||||
<div class="banner2text">
|
||||
"Because of Kubernetes, things have been much easier, our individual applications are better, and we can spend more time on functional implementation. We do not want to go back."
|
||||
<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>— VICTOR VAN DEN BOSCH, SENIOR DEVOPS ENGINEER, PROWISE</span>
|
||||
</div>
|
||||
</div>
|
||||
<section class="section2">
|
||||
<div class="fullcol">
|
||||
<h2>If you haven’t set foot in a school in awhile, you might be surprised by what you’d see in a digitally connected classroom these days: touchscreen monitors, laptops, tablets, touch tables, and more.</h2>
|
||||
One of the leaders in the space, the Dutch company Prowise, offers an integrated solution of hardware and software to help educators create a more engaging learning environment.
|
||||
<br><br>
|
||||
As the company expanded its offerings beyond the Netherlands in recent years—creating multiple availability zones in Europe, Australia, and the U.S., with as many as nine servers per zone—its Linux service-based infrastructure struggled to keep up. “We’ve grown a lot in the past couple of years, and we started to encounter problems with versioning and flexible scaling,” says Senior DevOps Engineer Victor van den Bosch, who was hired by the company in late 2017 to build a new platform.
|
||||
<br><br>
|
||||
Prowise’s products support ten languages, so the problem wasn’t just scaling in demands, he adds, “but also in being able to deploy multiple products which all have their own versions, their own development teams, and their own problems that they’re trying to solve. To be able to put that all on the same platform without much resistance is what we were looking for. We wanted to future proof our infrastructure, and also solve some of the problems that are associated with just running a normal Linux service.”
|
||||
<br><br>
|
||||
The company’s existing infrastructure on Microsoft Azure Cloud was all on virtual machines, “a pretty traditional setup,” van den Bosch says. “We decided that we want some features in our software that requires being able to scale quickly, being able to deploy new applications and versions on different versions of different programming languages quickly. And we didn’t really want the hassle of trying to keep those servers in a particular state.”
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner3" style="background-image: url('/images/CaseStudy_prowise_banner3.jpg')">
|
||||
<div class="banner3text">
|
||||
"You don’t have to go all-in immediately. You can just take a few projects, a service, run it alongside your more traditional stack, and build it up from there. Kubernetes scales, so as you add applications and services to it, it will scale with you. You don’t have to do it all at once, and that’s really a secret to everything, but especially true to Kubernetes."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>— VICTOR VAN DEN BOSCH, SENIOR DEVOPS ENGINEER, PROWISE</span>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
<section class="section3">
|
||||
<div class="fullcol">
|
||||
After researching possible solutions, he opted for containerization and Kubernetes orchestration. “Containerization is the future,” van den Bosch says. “Kubernetes solves things like networking really well, in a way that fits our business model. We want to focus on our core products, and that’s the software that runs on it and not necessarily the infrastructure itself.” Plus, the Prowise team liked that there was no vendor lock-in. “We don’t want to be limited to one platform,” he says. “We try not to touch products that are very proprietary and can’t be ported easily to another vendor.”
|
||||
<br><br>
|
||||
The time to market with Kubernetes was very short: The first web-based applications on the platform went into beta within a few months. That was largely made possible by van den Bosch’s decision to use Azure’s managed Kubernetes service, AKS. The team then had to figure out which components to keep and which to replace. Monitoring tools like New Relic were taken out “because they tend to become very expensive when you scale it to different availability zones, and it’s just not very maintainable,” he says.
|
||||
<br><br>
|
||||
A lot of work also went into improving Prowise’s CI/CD pipelines. “We wanted to make sure that the pipelines are automated and easy to use,” he says. “We have a lot of settings and configurations figured out for the pipelines, and it’s just applying those scripts and those configurations to new projects from here on out.”
|
||||
<br><br>
|
||||
With its first web-based applications now running in beta on Prowise’s Kubernetes platform, the team is seeing the benefits of rapid and smooth deployments. “The old way of deploying took half an hour of preparations and half an hour deploying it. With Kubernetes, it’s a couple of seconds,” says Senior Developer Bart Haalstra. As a result, adds van den Bosch, “We’ve gone from quarterly releases to a release every month in production. We’re pretty much deploying every hour or just when we find that a feature is ready for production. Before, our releases were mostly done on off-hours, where it couldn’t impact our customers, as our confidence the process itself was relatively low. With Kubernetes, we dare to deploy in the middle of a busy day with high confidence the deployment will succeed.”
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner4" style="background-image: url('/images/CaseStudy_prowise_banner4.jpg')" style="width:100%">
|
||||
<div class="banner4text">
|
||||
"Kubernetes allows us to really consider the best tools for a problem. Want to have a full-fledged analytics application developed by a third party that is just right for your use case? Run it. Dabbling in machine learning and AI algorithms but getting tired of waiting days for training to complete? It takes only seconds to scale it. Got a stubborn developer that wants to use a programming language no one has heard of? Let him, if it runs in a container, of course. And all of that while your operations team/DevOps get to sleep at night." <br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- VICTOR VAN DEN BOSCH, SENIOR DEVOPS ENGINEER, PROWISE</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<section class="section5" style="padding:0px !important">
|
||||
|
||||
<div class="fullcol">
|
||||
Plus, van den Bosch says, “Kubernetes has enabled us to follow up quickly on bugs and implement tweaks to our users with zero downtime between versions. For some bugs we’ve pushed code fixes to production minutes after detection.”
|
||||
<br><br>
|
||||
Recently, the team launched a new single sign-on solution for use in an internal application. “Due to the resource based architecture of the Kubernetes platform, we were able to bring that application into an entirely new production environment in less than a day, most of that time used for testing after applying the already well-known resource definitions from staging to the new environment,” says van den Bosch. “On a traditional VM this would have likely cost a day or two, and then probably a few weeks to iron out the kinks in our provisioning scripts as we apply updates.”
|
||||
<br><br>
|
||||
Legacy applications are also being moved to Kubernetes. Not long ago, the team needed to set up a Java-based application for compiling and running a frontend. “On a traditional VM, it would have taken quite a bit of time to set it up and keep it up to date, not to mention maintenance for that setup down the line,” says van den Bosch. Instead, it took less than half a day to Dockerize it and get it running on Kubernetes. “It was much easier, and we were able to save costs too because we didn’t have to spin up new VMs specially for it.”
|
||||
</div>
|
||||
|
||||
<div class="banner5" >
|
||||
<div class="banner5text">
|
||||
"We’re really trying to deliver integrated solutions with our hardware and software and making it as easy as possible for users to use and collaborate from different places,” says van den Bosch. And, says Haalstra, “We cannot do it without Kubernetes."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- VICTOR VAN DEN BOSCH, SENIOR DEVOPS ENGINEER, PROWISE</span></div>
|
||||
</div>
|
||||
|
||||
<div class="fullcol">
|
||||
Perhaps most importantly, van den Bosch says, “Kubernetes allows us to really consider the best tools for a problem and take full advantage of microservices architecture. Got a library in Node.js that excels at solving a certain problem? Use it. Want to have a full-fledged analytics application developed by a third party that is just right for your use case? Run it. Dabbling in machine learning and AI algorithms but getting tired of waiting days for training to complete? It takes only seconds to scale it. Got a stubborn developer that wants to use a programming language no one has heard of? Let him, if it runs in a container, of course. And all of that while your operations team/DevOps get to sleep at night.”
|
||||
<br><br>
|
||||
Looking ahead, all new web development, platforms, and APIs at Prowise will be on Kubernetes. One of the big greenfield projects is a platform for teachers and students that is launching for back-to-school season in September. Users will be able to log in and access a wide variety of educational applications. With the <a href="https://www.prowise.com/en/press-release-largest-dutch-education-innovators-join-forces/">recent acquisition</a> of the software company Oefenweb, Prowise plans to provide adaptive software that allows teachers to get an accurate view of their students’ progress and weak points, and automatically adjusts the difficulty level of assignments to suit individual students. “We will be leveraging Kubernetes’ power to integrate, supplement, and support our combined application portfolio and bring our solutions to more classrooms,” says van den Bosch.
|
||||
<br><br>
|
||||
Collaborative software is also a priority. With the single sign-in software, users’ settings and credentials are saved in the cloud and can be used on any screen in the world. “We’re really trying to deliver integrated solutions with our hardware and software and making it as easy as possible for users to use and collaborate from different places,” says van den Bosch. And, says Haalstra, “We cannot do it without Kubernetes.”
|
||||
</div>
|
||||
|
||||
</section>
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 7.4 KiB |
@@ -0,0 +1,98 @@
|
||||
---
|
||||
title: ricardo.ch Case Study
|
||||
linkTitle: ricardo-ch
|
||||
case_study_styles: true
|
||||
cid: caseStudies
|
||||
css: /css/style_case_studies.css
|
||||
featured: false
|
||||
---
|
||||
|
||||
<div class="banner1" style="background-image: url('/images/CaseStudy_ricardoch_banner1.png')">
|
||||
<h1> CASE STUDY:<img src="/images/ricardoch_logo.png" class="/images/header_logo" style="width:25%;margin-bottom:-1%"><br> <div class="subhead" style="margin-top:1%">ricardo.ch: How Kubernetes Improved Velocity and DevOps Harmony
|
||||
</div></h1>
|
||||
|
||||
</div>
|
||||
|
||||
<div class="details">
|
||||
Company <b>ricardo.ch</b> Location <b>Zurich, Switzerland </b> Industry <b>E-commerce</b>
|
||||
</div>
|
||||
|
||||
<hr>
|
||||
<section class="section1">
|
||||
<div class="cols">
|
||||
<div class="col1" style="width:100%"">
|
||||
|
||||
<h2>Challenge</h2>
|
||||
A Swiss online marketplace, <a href="https://www.ricardo.ch/de/">ricardo.ch</a> was experiencing problems with velocity, as well as a "classic gap" between Development and Operations, with the two sides unable to work well together. "They wanted to, but they didn’t have common ground," says Cedric Meury, Head of Platform Engineering. "This was one of the root causes that slowed us down." The company began breaking down the legacy monolith into microservices, and needed orchestration to support the new architecture in its own data centers—as well as bring together Dev and Ops.
|
||||
<h2>Solution</h2>
|
||||
The company adopted <a href="https://kubernetes.io/">Kubernetes</a> for cluster management, <a href="https://prometheus.io/">Prometheus</a> for monitoring, and <a href="https://www.fluentd.org/">Fluentd</a> for logging. The first cluster was deployed on premise in December 2016, with the first service in production three months later. The migration is about half done, and the company plans to move completely to <a href="https://cloud.google.com/">Google Cloud Platform</a> by the end of 2018.
|
||||
<h2>Impact</h2>
|
||||
Splitting up the monolith into microservices "allowed higher velocity, and Kubernetes was crucial to support that," says Meury. The number of deployments to production has gone from fewer than 10 a week to 30-60 per day. Before, "when there was a problem with something in production, tickets or complaints would be thrown over the wall to operations, the classical problem. Now, people have the chance to look into operations and troubleshoot for themselves first because everything is deployed in a standardized way," says Meury. He sees the impact in everyday interactions: "A couple of weeks ago, I saw a product manager doing a pull request for a JSON file that contains some variables, and someone else accepted it. And it was deployed after a couple of minutes or seconds even, which was unthinkable before. There used to be quite a chain of things that needed to happen, the whole monolith was difficult to understand, even for engineers. So, previously requests would go into large, inefficient Kanban boards and hopefully someone will have done the change after weeks and months." Before, infrastructure- and platform-related projects took months or years to complete; now developers and operators can work together to deploy infrastructure parts via Kubernetes in a matter of weeks and sometimes days. In the long run, the company also expects to notch 50% cost savings going from custom data center and virtual machines to containerized infrastructure and cloud services.
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner2">
|
||||
<div class="banner2text">
|
||||
"Splitting up the monolith allowed higher velocity, and Kubernetes was crucial to support that. Containerization and orchestration by Kubernetes helped us to drastically reduce the conflict between Dev and Ops and also allowed us to speak the same language on both sides of the aisle."
|
||||
<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>— CEDRIC MEURY, HEAD OF PLATFORM ENGINEERING, RICARDO.CH</span>
|
||||
</div>
|
||||
</div>
|
||||
<section class="section3">
|
||||
<div class="fullcol">
|
||||
<h2>When Cedric Meury joined ricardo.ch in 2016, he saw a clear divide between Operations and Development. In fact, there was literal distance between them: The engineering team worked in France, while the rest of the org was based in Switzerland.
|
||||
</h2><br><br>
|
||||
"It was a classic gap between those departments and even some anger and frustration here and there," says Meury. "They wanted to work together, but they didn’t have common ground. This was one of the root causes that slowed us down."
|
||||
<br><br>
|
||||
That gap was hurting velocity at ricardo.ch, a Swiss online marketplace. The website processes up to 2.6 million searches on a peak day from both web and mobile apps, serving 3.2 million members with its live auctions. The technology team’s main challenge was to make sure that "the bids for items come in the right order, and before the auction is finished, and that this works in a fair way," says Meury. "We have a real-time requirement. We also provide an automated system to bid, and it needs to be accurate and correct. With a distributed system, you have the challenge of making sure that the ordering is right. And that’s one of the things we’re currently dealing with."
|
||||
<br><br>
|
||||
To address the velocity issue, ricardo.ch CTO Jeremy Seitz established a new software factory called EPD, which consists of 65 engineers, 7 product managers and 2 designers. "We brought these three departments together so that they can kind of streamline this and talk to each other much more closely," says Meury.
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner3" style="background-image: url('/images/CaseStudy_ricardoch_banner3.png')">
|
||||
<div class="banner3text">
|
||||
"Being in the End User Community demonstrates that we stand behind these technologies. In Switzerland, if all the companies see that ricardo.ch’s using it, I think that will help adoption. I also like that we’re connected to the other end users, so if there is a really heavy problem, I could go to the Slack channel, and say, ‘Hey, you guys…’ Like Reddit, Github and New York Times or whoever can give a recommendation on what to use here or how to solve that. So that’s kind of a superpower."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>— CEDRIC MEURY, HEAD OF PLATFORM ENGINEERING, RICARDO.CH</span>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
<section class="section3">
|
||||
<div class="fullcol">
|
||||
|
||||
The company also began breaking down the legacy monolith into more than 100 microservices, and needed orchestration to support the new architecture in its own data centers. "Splitting up the monolith allowed higher velocity, and Kubernetes was crucial to support that," says Meury. "Containerization and orchestration by Kubernetes helped us to drastically reduce the conflict between Dev and Ops and also allowed us to speak the same language on both sides of the aisle."
|
||||
<br><br>
|
||||
Meury put together a platform engineering team to choose the tools—including Fluentd for logging and Prometheus for monitoring, with Grafana visualization—and lay the groundwork for the first Kubernetes cluster, which was installed on premise in December 2016. Within a few weeks, the new platform was available to teams, who were given training sessions and documentation. The platform engineering team then embedded with engineers to help them deploy their applications on the new platform. The first service in production was the ricardo.ch jobs page. "It was an exercise in front-end development, so the developers could experiment with a new stack," says Meury.
|
||||
<br><br>
|
||||
Meury estimates that half of the application has been migrated to Kubernetes. And the plan is to move everything to the Google Cloud Platform by the end of 2018. "We are still running some servers in our own data centers, but all of the containerization efforts and describing our services as Kubernetes manifests will allow us to quite easily make that shift," says Meury.
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner4" style="background-image: url('/images/CaseStudy_ricardoch_banner4.png')" style="width:100%">
|
||||
<div class="banner4text">
|
||||
"One of the core moments was when a front-end developer asked me how to do a port forward from his laptop to a front-end application to debug, and I told him the command. And he was like, ‘Wow, that’s all I need to do?’ He was super excited and happy about it. That showed me that this power in the right hands can just accelerate development."
|
||||
<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- CEDRIC MEURY, HEAD OF PLATFORM ENGINEERING, RICARDO.CH</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<section class="section5" style="padding:0px !important">
|
||||
<div class="fullcol">
|
||||
The impact has been great. Moving from custom data center and virtual machines to containerized infrastructure and cloud services is expected to result in 50% cost savings for the company. The number of deployments to production has gone from fewer than 10 a week to 30-60 per day. Before, "when there was a problem with something in production, tickets or complaints would be thrown over the wall to operations, the classical problem," says Meury. "Now, people have the chance to look into operations and troubleshoot for themselves first because everything is deployed in a standardized way. That reduces time and uncertainty."
|
||||
<br><br>
|
||||
Meury also sees the impact in everyday interactions: "A couple of weeks ago, I saw a product manager doing a pull request for a JSON file that contains some variables, and someone else accepted it. And it was deployed after a couple of minutes or seconds even, which was unthinkable before. There used to be quite a chain of things that needed to happen, the whole monolith was difficult to understand, even for engineers. So, previously requests would go into large, inefficient Kanban boards and hopefully someone will have done the change after weeks and months."
|
||||
<br><br>
|
||||
The divide between Dev and Ops has also diminished. "After a couple of months, I got requests by people saying, ‘Hey, could you help me install the Kubernetes client? I want to actually look at what’s going on,’" says Meury. "People were directly looking at the state of the system, bringing them much, much closer to the operations." Before, infrastructure- and platform-related projects took months or years to complete; now developers and operators can work together to deploy infrastructure parts via Kubernetes in a matter of weeks and sometimes days.
|
||||
</div>
|
||||
|
||||
<div class="banner5" >
|
||||
<div class="banner5text">
|
||||
"One of my colleagues was listening to all the talks at KubeCon, and he was overwhelmed by all the tools, technologies, frameworks out there that are currently lacking on our platform, but at the same time, he’s very happy to know that in the future there is so much that we can still explore and we can improve and we can work on."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- CEDRIC MEURY, HEAD OF PLATFORM ENGINEERING, RICARDO.CH</span></div>
|
||||
</div>
|
||||
|
||||
<div class="fullcol">
|
||||
|
||||
|
||||
The ability to have insight into the system has extended to other parts of the company, too. "I found out that one of our customer support representatives looks at Grafana metrics to find out whether the system is running fine, which is fantastic," says Meury. "Prometheus is directly hooked into customer care."
|
||||
<br><br>
|
||||
The ricardo.ch cloud native journey has perhaps had the most impact on the Ops team. "We have an operations team that comes from a hardware-based background, and right now they are relearning how to operate in a more virtualized and cloud native world, with great success so far," says Meury. "So besides still operating on-site data center firewalls, they learn to code in Go or do some Python scripting at the same time. Former network administrators are writing Go code. It’s just really cool.
|
||||
<br><br>
|
||||
For Meury, the journey boils down to this. "One of my colleagues was listening to all the talks at KubeCon, and he was overwhelmed by all the tools, technologies, frameworks out there that are currently lacking on our platform," says Meury. "But at the same time, he’s very happy to know that in the future there is so much that we can still explore and we can improve and we can work on. We’re transitioning from seeing problems everywhere—like, ‘This is broken’ or ‘This is down, and we have to fix it’—more to, ‘How can we actually improve and automate more, and make it nicer for developers and ultimately for the end users?’"
|
||||
</div>
|
||||
|
||||
</section>
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 5.1 KiB |
@@ -0,0 +1,88 @@
|
||||
---
|
||||
title: Slamtec Case Study
|
||||
linkTitle: slamtec
|
||||
case_study_styles: true
|
||||
cid: caseStudies
|
||||
css: /css/style_case_studies.css
|
||||
featured: false
|
||||
---
|
||||
|
||||
<div class="banner1" style="background-image: url('/images/CaseStudy_slamtec_banner1.jpg')">
|
||||
<h1> CASE STUDY:<img src="/images/slamtec_logo.png" class="header_logo" style="width:17%;margin-bottom:%"><br> </h1>
|
||||
<br><br>
|
||||
</div>
|
||||
|
||||
<div class="details">
|
||||
Company <b>Slamtec</b> Location <b>Shanghai, China</b> Industry <b>Robotics</b>
|
||||
</div>
|
||||
|
||||
<hr>
|
||||
<section class="section1">
|
||||
<div class="cols">
|
||||
<div class="col1" style="width:100%"">
|
||||
<h2>Challenge</h2>
|
||||
Founded in 2013, SLAMTEC provides service robot autonomous localization and navigation solutions. The company’s strength lies in its R&D team’s ability to quickly introduce, and continually iterate on, its core products. In the past few years, the company, which had a legacy infrastructure based on Alibaba Cloud and VMware vSphere, began looking to build its own stable and reliable container cloud platform to host its Internet of Things applications. "Our needs for the cloud platform included high availability, scalability and security; multi-granularity monitoring alarm capability; friendliness to containers and microservices; and perfect CI/CD support," says Benniu Ji, Director of Cloud Computing Business Division.
|
||||
|
||||
<h2>Solution</h2>
|
||||
Ji’s team chose Kubernetes for orchestration. "CNCF brings quality assurance and a complete ecosystem for <a href="https://kubernetes.io/">Kubernetes</a>, which is very important for the wide application of Kubernetes," says Ji. Thus Slamtec decided to adopt other CNCF projects as well: <a href="https://prometheus.io/">Prometheus</a> monitoring, <a href="https://www.fluentd.org/">Fluentd</a> logging, <a href="https://goharbor.io/">Harbor</a> registry, and <a href="https://helm.sh/">Helm</a> package manager.
|
||||
<br>
|
||||
<h2>Impact</h2>
|
||||
With the new platform, Ji reports that Slamtec has experienced "18+ months of 100% stability!" For users, there is now zero service downtime and seamless upgrades. "Kubernetes with third-party service mesh integration (Istio, along with Jaeger and Envoy) significantly reduced the microservice configuration and maintenance efforts by 50%," he adds. With centralized metrics monitoring and log aggregation provided by Prometheus on Fluentd, teams are saving 50% of time spent on troubleshooting and debugging. Harbor replication has allowed production/staging/testing environments to cross public cloud and the private Kubernetes cluster to share the same container registry, resulting in 30% savings of CI/CD efforts. Plus, Ji says, "Helm has accelerated prototype development and environment setup with its rich sharing charts."
|
||||
</div>
|
||||
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner2">
|
||||
<div class="banner2text">
|
||||
"Cloud native technology helps us ensure high availability of our business, while improving development and testing efficiency, shortening the research and development cycle and enabling rapid product delivery."
|
||||
<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- BENNIU JI, DIRECTOR OF CLOUD COMPUTING BUSINESS DIVISION</span>
|
||||
</div>
|
||||
</div>
|
||||
<section class="section2">
|
||||
<div class="fullcol">
|
||||
<h2>Founded in 2013, Slamtec provides service robot autonomous localization and navigation solutions. In this fast-moving space, the company built its success on the ability of its R&D team to quickly introduce, and continually iterate on, its core products.
|
||||
</h2>
|
||||
To sustain that development velocity, the company over the past few years began looking to build its own stable and reliable container cloud platform to host its Internet of Things applications. With a legacy infrastructure based on <a href="https://www.alibabacloud.com/">Alibaba Cloud</a> and <a href="https://www.vmware.com/products/vsphere.html">VMware vSphere</a>, Slamtec teams had already adopted microservice architecture and continuous delivery, for "fine granularity on-demand scaling, fault isolation, ease of development, testing, and deployment, and for facilitating high-speed iteration," says Benniu Ji, Director of Cloud Computing Business Division. So "our needs for the cloud platform included high availability, scalability and security; multi-granularity monitoring alarm capability; friendliness to containers and microservices; and perfect CI/CD support."
|
||||
<br><br>
|
||||
After an evaluation of existing technologies, Ji’s team chose <a href="https://kubernetes.io/">Kubernetes</a> for orchestration. "CNCF brings quality assurance and a complete ecosystem for Kubernetes, which is very important for the wide application of Kubernetes," says Ji. Plus, "avoiding binding to an infrastructure technology or provider can help us ensure that our business is deployed and migrated in cross-regional environments, and can serve users all over the world."
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner3" style="background-image: url('/images/CaseStudy_slamtec_banner3.jpg')">
|
||||
<div class="banner3text">
|
||||
"CNCF brings quality assurance and a complete ecosystem for Kubernetes, which is very important for the wide application of Kubernetes."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- BENNIU JI, DIRECTOR OF CLOUD COMPUTING BUSINESS DIVISION</span>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
<section class="section3">
|
||||
<div class="fullcol">
|
||||
Thus Slamtec decided to adopt other CNCF projects as well. "We built a monitoring and logging system based on <a href="https://prometheus.io/">Prometheus</a> and <a href="https://www.fluentd.org/">Fluentd</a>," says Ji. "The integration between Prometheus/Fluentd and Kubernetes is convenient, with multiple dimensions of data monitoring and log collection capabilities."
|
||||
<br><br>
|
||||
The company uses <a href="https://goharbor.io/">Harbor</a> as a container image repository. "Harbor’s replication function helps us implement CI/CD on both private and public clouds," says Ji. "In addition, multi-project support, certification and policy configuration, and integration with Kubernetes are also excellent functions." <a href="https://helm.sh/">Helm</a> is also being used as a package manager, and the team is evaluating the Istio framework. "We’re very pleased that Kubernetes and these frameworks can be seamlessly integrated," Ji adds.
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner4" style="background-image: url('/images/CaseStudy_slamtec_banner4.jpg')">
|
||||
<div class="banner4text">
|
||||
"Cloud native is suitable for microservice architecture, it’s suitable for fast iteration and agile development, and it has a relatively perfect ecosystem and active community." <br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- BENNIU JI, DIRECTOR OF CLOUD COMPUTING BUSINESS DIVISION</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<section class="section5" style="padding:0px !important">
|
||||
<div class="fullcol">
|
||||
With the new platform, Ji reports that Slamtec has experienced "18+ months of 100% stability!" For users, there is now zero service downtime and seamless upgrades. "We benefit from the abstraction of Kubernetes from network and storage," says Ji. "The dependence on external services can be decoupled from the service and placed under unified management in the cluster."
|
||||
<br><br>
|
||||
Using Kubernetes and Istio "significantly reduced the microservice configuration and maintenance efforts by 50%," he adds. With centralized metrics monitoring and log aggregation provided by Prometheus on Fluentd, teams are saving 50% of time spent on troubleshooting and debugging. Harbor replication has allowed production/staging/testing environments to cross public cloud and the private Kubernetes cluster to share the same container registry, resulting in 30% savings of CI/CD efforts. Plus, Ji adds, "Helm has accelerated prototype development and environment setup with its rich sharing charts."
|
||||
<br><br>
|
||||
In short, Ji says, Slamtec’s new platform is helping it achieve one of its primary goals: the quick and easy release of products. With multiple release models and a centralized control interface, the platform is changing developers’ lives for the better. Slamtec also offers a unified API for the development of automated deployment tools according to users’ specific needs.
|
||||
</div>
|
||||
|
||||
<div class="banner5" style="width:100%">
|
||||
<div class="banner5text">
|
||||
"We benefit from the abstraction of Kubernetes from network and storage, the dependence on external services can be decoupled from the service and placed under unified management in the cluster."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- BENNIU JI, DIRECTOR OF CLOUD COMPUTING BUSINESS DIVISION</span></div>
|
||||
</div>
|
||||
|
||||
<div class="fullcol">
|
||||
Given its own success with cloud native, Slamtec has just one piece of advice for organizations considering making the leap. "For already containerized services, you should migrate them to the cloud native architecture as soon as possible and enjoy the advantages brought by the cloud native ecosystem," Ji says. "To migrate traditional, non-containerized services, in addition to the architecture changes of the service itself, you need to fully consider the operation and maintenance workload required to build the cloud native architecture."
|
||||
<br><br>
|
||||
That said, the cost-benefit analysis has been simple for Slamtec. "Cloud native technology is suitable for microservice architecture, it’s suitable for fast iteration and agile development, and it has a relatively perfect ecosystem and active community," says Ji. "It helps us ensure high availability of our business, while improving development and testing efficiency, shortening the research and development cycle and enabling rapid product delivery."
|
||||
</div>
|
||||
</section>
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 16 KiB |
@@ -0,0 +1,99 @@
|
||||
---
|
||||
title: SOS International Case Study
|
||||
linkTitle: SOS International
|
||||
case_study_styles: true
|
||||
cid: caseStudies
|
||||
css: /css/style_case_studies.css
|
||||
logo: sos_featured_logo.png
|
||||
---
|
||||
|
||||
|
||||
<div class="banner1" style="background-image: url('/images/CaseStudy_sos_banner1.jpg')">
|
||||
<h1> CASE STUDY:<img src="/images/sos_logo.png" class="header_logo" style="width:20%;margin-bottom:-1.2%"><br> <div class="subhead" style="margin-top:1%">SOS International: Using Kubernetes to Provide Emergency Assistance in a Connected World
|
||||
|
||||
</div></h1>
|
||||
|
||||
</div>
|
||||
|
||||
<div class="details">
|
||||
Company <b>SOS International</b> Location <b>Frederiksberg, Denmark
|
||||
</b> Industry <b>Medical and Travel Assistance</b>
|
||||
</div>
|
||||
|
||||
<hr>
|
||||
<section class="section1">
|
||||
<div class="cols">
|
||||
<div class="col1" style="width:100%"">
|
||||
<h2>Challenge</h2>
|
||||
For the past six decades, SOS International has been providing reliable medical and travel assistance in the Nordic region. In recent years, the company’s business strategy has required increasingly intense development in the digital space, but when it came to its IT systems, "SOS has a very fragmented legacy," with three traditional monoliths (Java, .NET, and IBM’s AS/400) and a waterfall approach, says Martin Ahrentsen, Head of Enterprise Architecture. "We have been forced to institute both new technology and new ways of working, so we could be more efficient with a shorter time to market. It was a much more agile approach, and we needed to have a platform that can help us deliver that to the business."
|
||||
|
||||
<br><br>
|
||||
<h2>Solution</h2>
|
||||
After an unsuccessful search for a standard system, the company decided to take a platform approach and look for a solution that rolls up Kubernetes and the container technology. <a href="https://www.openshift.com/">RedHat OpenShift</a> proved to be a perfect fit for SOS’s fragmented systems. "We have a lot of different technologies that we use, both code languages and others, and all of them could use the resources on the new platform," says Ahrentsen. Of the company’s three monoliths, "we can provide this new bleeding edge technology to two of them (.NET and Java)." The platform went live in the spring of 2018; there are now six greenfield projects based on microservices architecture underway, plus all of the company’s Java applications are currently going through a "lift and shift" migration.
|
||||
|
||||
<br><br>
|
||||
<h2>Impact</h2>
|
||||
Kubernetes has delivered "improved time to market, agility, and the ability to adapt to changes and new technologies," says Ahrentsen. "Just the time between when the software is ready for release and when it can be released has dramatically been improved." The way of thinking at SOS International has also changed for the better: "Since we have Kubernetes and easy access to scripts that can help us automate, creating CI/CD pipelines easily, that has spawned a lot of internal interest in how to do this fully automated, all the way. It creates a very good climate in order to start the journey," he says. Moreover, being part of the cloud native community has helped the company attract talent. "They want to work with the cool, new technologies," says Ahrentsen. "During our onboarding, we could see that we were chosen by IT professionals because we provided the new technologies."
|
||||
|
||||
</div>
|
||||
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner2">
|
||||
<div class="banner2text">
|
||||
"The speed of the changes that cloud native software and technologies drive right now is amazing, and following and adopting it is very crucial for us. The amazing technology provided by Kubernetes and cloud native has started the change for SOS towards a digital future."
|
||||
<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Martin Ahrentsen, Head of Enterprise Architecture, SOS International</span>
|
||||
</div>
|
||||
</div>
|
||||
<section class="section2">
|
||||
<div class="fullcol">
|
||||
<h2>For six decades, SOS International has provided reliable emergency medical and travel assistance for customers in the Nordic countries.</h2>
|
||||
SOS operators handle a million cases and over a million phone calls a year. But in the past four years, the company’s business strategy has required increasingly intense development in the digital space.<br><br>
|
||||
When it comes to its IT systems, "SOS has a very fragmented legacy," with three traditional monoliths running in the company’s own data centers and a waterfall approach, says Martin Ahrentsen, Head of Enterprise Architecture. "We had to institute both new technology and new ways of working so we could be more efficient, with a shorter time to market. It was a much more agile approach, and we needed to have a platform that can help us deliver that to the business."<br><br>
|
||||
For a long time, Ahrentsen and his team searched for a standard solution that could work at SOS. "There aren’t that many assistance companies like us, so you cannot get a standard system that fits for that; there is no perfect match," he says. "We would have to take a standard system and twist it too much so it is not standard anymore. Based on that, we decided to find a technology platform instead, with some common components that we could use to build the new digital systems and core systems."
|
||||
|
||||
|
||||
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner3" style="background-image: url('/images/CaseStudy_sos_banner3.jpg')">
|
||||
<div class="banner3text">
|
||||
"We have to deliver new digital services, but we also have to migrate the old stuff, and we have to transform our core systems into new systems built on top of this platform. One of the reasons why we chose this technology is that we could build new digital services while changing the old one."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Martin Ahrentsen, Head of Enterprise Architecture, SOS International</span>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
<section class="section3">
|
||||
<div class="fullcol">
|
||||
Sold on what Kubernetes could do, Ahrentsen zeroed in on platforms that could meet the business’s needs right away. The company opted to use RedHat’s OpenShift container platform, which incorporates Docker containers and Kubernetes, as well as a whole stack of technologies, including RedHat Hyperconverged Infrastructure and some midware components, all from the open source community.<br><br>
|
||||
Based on the company’s criteria—technology fit, agility fit, legal requirements, and competencies—the OpenShift solution seemed like a perfect fit for SOS’s fragmented systems. "We have a lot of different technologies that we use, both code languages and others, and all of them could use the resources on the new platform," says Ahrentsen. Of the company’s three monoliths, "we can provide this new bleeding edge technology to two of them (.NET and Java)."<br><br>
|
||||
The platform went live in the spring of 2018; six greenfield projects based on microservices architecture were initially launched, plus all of the company’s Java applications are currently going through a "lift and shift" migration. One of the first Kubernetes-based projects to go live is Remote Medical Treatment, a solution in which customers can contact the SOS alarm center via voice, chat, or video. "We managed to develop it in quite a short timeframe with focus on full CI/CD pipelining and a modern microservice architecture all running in a dual OpenShift cluster setup," says Ahrentsen. Onsite, which is used for dispatching rescue trucks around the Nordic countries, and Follow Your Truck, which allows customers to track tow trucks, are also being rolled out.
|
||||
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner4" style="background-image: url('/images/CaseStudy_sos_banner4.jpg');width:100%">
|
||||
<div class="banner4text">
|
||||
"During our onboarding, we could see that we were chosen by IT professionals because we provided the new technologies." <br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Martin Ahrentsen, Head of Enterprise Architecture, SOS International</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<section class="section5" style="padding:0px !important">
|
||||
<div class="fullcol">
|
||||
The platform is still running on premise, because some of SOS’s customers in the insurance industry, for whom the company handles data, don’t yet have a cloud strategy. Kubernetes is allowing SOS to start in the data center and move to the cloud when the business is ready. "Over the next three to five years, all of them will have a strategy, and we could probably take the data and go to the cloud," says Ahrentsen. There’s also the possibility of moving to a hybrid cloud setup for sensitive and non-sensitive data. <br><br>
|
||||
SOS’s technology is certainly in a state of transition. "We have to deliver new digital services, but we also have to migrate the old stuff, and we have to transform our core systems into new systems built on top of this platform," says Ahrentsen. "One of the reasons why we chose this technology is that we could build new digital services while changing the old one."<br><br>
|
||||
But already, Kubernetes has delivered improved time to market, as evidenced by how quickly the greenfield projects were developed and released. "Just the time between when the software is ready for release and when it can be released has dramatically been improved," says Ahrentsen.<br><br>
|
||||
Moreover, being part of the cloud native community has helped the company attract talent as it pursues a goal of growing the ranks of engineers, operators, and architects from 60 to 100 this year. "They want to work with the cool, new technologies," says Ahrentsen. "During our onboarding, we could see that we were chosen by IT professionals because we provided the new technologies."
|
||||
|
||||
</div>
|
||||
|
||||
<div class="banner5" >
|
||||
<div class="banner5text">
|
||||
"The future world where everything is connected and sends data will create a big potential for us in terms of new market opportunities. But it will also set a big demand on the IT platform and what we need to deliver."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- Martin Ahrentsen, Head of Enterprise Architecture, SOS International</span></div>
|
||||
</div>
|
||||
|
||||
<div class="fullcol">
|
||||
The way of thinking at SOS International has also changed dramatically: "Since we have Kubernetes and easy access to scripts that can help us automate, creating CI/CD pipelines easily, that has spawned a lot of internal interest in how to do this fully automated, all the way. It creates a very good climate in order to start the journey." <br><br>
|
||||
For this journey at SOS, digitalization and optimization are the key words. "For IT to deliver this, we need to improve, and that is not just on the way of using Kubernetes and the platform," says Ahrentsen. "It’s also a way of building the systems to be ready for automation, and afterwards, machine learning and other interesting technologies that are on the way."<br><br>
|
||||
Case in point: the introduction of the internet of things into automobiles. The European Commission now mandates all new cars to be equipped with <a href="https://ec.europa.eu/transport/themes/its/road/action_plan/ecall_en">eCall</a>, which transmits location and other data in case of a serious traffic accident. SOS provides this service as smart auto assistance. "We receive the call and find out if an emergency response team needs to be sent, or if it’s not heavy impact," says Ahrentsen. "The future world where everything is connected and sends data will create a big potential for us in terms of new market opportunities. But it will also set a big demand on the IT platform and what we need to deliver."<br><br>
|
||||
Ahrentsen feels that SOS is well equipped for the challenge, given the technology choices the company has made. "The speed of the changes that cloud native software and technologies drive right now is amazing, and following it and adopting it is very crucial for us," he says. "The amazing technology provided by Kubernetes and cloud native has started the change for SOS towards a digital future."
|
||||
</div>
|
||||
</section>
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 6.5 KiB |
@@ -0,0 +1,94 @@
|
||||
---
|
||||
title: ThredUp Case Study
|
||||
linkTitle: thredup
|
||||
case_study_styles: true
|
||||
cid: caseStudies
|
||||
css: /css/style_case_studies.css
|
||||
featured: false
|
||||
---
|
||||
|
||||
<div class="banner1" style="background-image: url('/images/CaseStudy_thredup_banner1.jpg')">
|
||||
<h1> CASE STUDY:<img src="/images/thredup_logo.png" class="header_logo" style="width:17%;margin-bottom:%"><br> </h1>
|
||||
<br><br>
|
||||
</div>
|
||||
|
||||
<div class="details">
|
||||
Company <b>ThredUp</b> Location <b>San Francisco, CA</b> Industry <b>eCommerce</b>
|
||||
</div>
|
||||
|
||||
<hr>
|
||||
<section class="section1">
|
||||
<div class="cols">
|
||||
<div class="col1" style="width:100%"">
|
||||
<h2>Challenge</h2>
|
||||
The largest online consignment store for women’s and children’s clothes, ThredUP launched in 2009 with a monolithic application running on Amazon Web Services. Though the company began breaking up the monolith into microservices a few years ago, the infrastructure team was still dealing with handcrafted servers, which hampered productivity. "We’ve configured them just to get them out as fast as we could, but there was no standardization, and as we kept growing, that became a bigger and bigger chore to manage," says Cofounder/CTO Chris Homer. The infrastructure, they realized, needed to be modernized to enable the velocity the company needed. "It’s really important to a company like us who’s disrupting the retail industry to make sure that as we’re building software and getting it out in front of our users, we can do it on a fast cycle and learn a ton as we experiment," adds Homer. "We wanted to make sure that our engineers could embrace the DevOps mindset as they built software. It was really important to us that they could own the life cycle from end to end, from conception at design, through shipping it and running it in production, from marketing to ecommerce, the user experience and our internal distribution center operations."
|
||||
<br><br>
|
||||
|
||||
<h2>Solution</h2>
|
||||
In early 2017, the company adopted Kubernetes for container orchestration, and in the course of a year, the entire infrastructure was moved to Kubernetes.
|
||||
<br><br>
|
||||
<h2>Impact</h2>
|
||||
Before, "even considering that we already have all the infrastructure in the cloud, databases and services, and all these good things," says Infrastructure Engineer Oleksandr Snagovskyi, setting up a new service meant waiting 2-4 weeks just to get the environment. With Kubernetes, new application roll-out time has decreased from several days or weeks to minutes or hours. Now, says Infrastructure Engineer Oleksii Asiutin, "our developers can experiment with existing applications and create new services, and do it all blazingly fast." In fact, deployment time has decreased about 50% on average for key services. "Lead time" for all applications is under 20 minutes, enabling engineers to deploy multiple times a day. Plus, 3200+ ansible scripts have been deprecated in favor of helm charts. And impressively, hardware cost has decreased 56% while the number of services ThredUP runs has doubled.
|
||||
</div>
|
||||
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner2">
|
||||
<div class="banner2text">
|
||||
<iframe width="504" height="296" src="https://www.youtube.com/embed/t0csOf-uDrk" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe><br><br>
|
||||
"Moving towards cloud native technologies like Kubernetes really unlocks our ability to experiment quickly and learn from customers along the way."
|
||||
<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- CHRIS HOMER, COFOUNDER/CTO, THREDUP</span>
|
||||
</div>
|
||||
</div>
|
||||
<section class="section2">
|
||||
<div class="fullcol">
|
||||
<h2>The largest online consignment store for women’s and children’s clothes, ThredUP is focused on getting consumers to think second-hand first. "We’re disrupting the retail industry, and it’s really important to us to make sure that as we’re building software and getting it out in front of our users, we can do it on a fast cycle and learn a ton as we experiment," says Cofounder/CTO Chris Homer.
|
||||
</h2>
|
||||
But over the past few years, ThredUP, which was launched in 2009 with a monolithic application running on Amazon Web Services, was feeling growing pains as its user base passed the 20- million mark. Though the company had begun breaking up the monolith into microservices, the infrastructure team was still dealing with handcrafted servers, which hampered productivity. "We’ve configured them just to get them out as fast as we could, but there was no standardization, and as we kept growing, that became a bigger and bigger chore to manage," says Homer. The infrastructure, Homer realized, needed to be modernized to enable the velocity—and the culture—the company wanted.
|
||||
<br><br>
|
||||
"We wanted to make sure that our engineers could embrace the DevOps mindset as they built software," Homer says. "It was really important to us that they could own the life cycle from end to end, from conception at design, through shipping it and running it in production, from marketing to ecommerce, the user experience and our internal distribution center operations."
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner3" style="background-image: url('/images/CaseStudy_thredup_banner3.jpg')">
|
||||
<div class="banner3text">
|
||||
"Kubernetes enabled auto scaling in a seamless and easily manageable way on days like Black Friday. We no longer have to sit there adding instances, monitoring the traffic, doing a lot of manual work."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- CHRIS HOMER, COFOUNDER/CTO, THREDUP</span>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
<section class="section3">
|
||||
<div class="fullcol">
|
||||
In early 2017, Homer found the solution with Kubernetes container orchestration. In the course of a year, the company migrated its entire infrastructure to Kubernetes, starting with its website applications and concluding with its operations backend. Teams are now also using Fluentd and Helm. "Initially there were skeptics about the value that this move to cloud native technologies would bring, but as we went through the process, people very quickly started to realize the benefit of having seamless upgrades and easy rollbacks without having to worry about what was happening," says Homer. "It unlocks the developers’ confidence in being able to deploy quickly, learn, and if you make a mistake, you can roll it back without any issue."
|
||||
<br><br>
|
||||
According to the infrastructure team, the key improvement was the consistent experience Kubernetes enabled for developers. "It lets developers work in the same environment that their application will be running in production," says Infrastructure Engineer Oleksandr Snagovskyi. Plus, "It became easier to test, easier to refine, and easier to deploy, because everything’s done automatically," says Infrastructure Engineer Oleksii Asiutin. "One of the main goals of our team is to make developers’ lives more comfortable, and we are achieving this with Kubernetes. They can experiment with existing applications and create new services, and do it all blazingly fast."
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner4" style="background-image: url('/images/CaseStudy_thredup_banner4.jpg')">
|
||||
<div class="banner4text">
|
||||
"One of the main goals of our team is to make developers’ lives more comfortable, and we are achieving this with Kubernetes. They can experiment with existing applications and create new services, and do it all blazingly fast." <br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- OLEKSII ASIUTIN, INFRASTRUCTURE ENGINEER, THREDUP</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<section class="section5" style="padding:0px !important">
|
||||
<div class="fullcol">
|
||||
Before, "even considering that we already have all the infrastructure in the cloud, databases and services, and all these good things," says Snagovskyi, setting up a new service meant waiting 2-4 weeks just to get the environment. With Kubernetes, because of simple configuration and minimal dependency on the infrastructure team, the roll-out time for new applications has decreased from several days or weeks to minutes or hours.
|
||||
<br><br>
|
||||
In fact, deployment time has decreased about 50% on average for key services. "Fast deployment and parallel test execution in Kubernetes keep a ‘lead time’ for all applications under 20 minutes," allowing engineers to do multiple releases a day, says Director of Infrastructure Roman Chepurnyi. The infrastructure team’s jobs, he adds, have become less burdensome, too: "We can execute seamless upgrades frequently and keep cluster performance and security up-to-date because OS-level hardening and upgrades of a Kubernetes cluster is a non-blocking activity for production operations and does not involve coordination with multiple engineering teams."
|
||||
<br><br>
|
||||
More than 3,200 ansible scripts have been deprecated in favor of Helm charts. And impressively, hardware cost has decreased 56% while the number of services ThredUP runs has doubled.
|
||||
</div>
|
||||
|
||||
<div class="banner5" style="width:100%">
|
||||
<div class="banner5text">
|
||||
"Our future’s all about automation, and behind that, cloud native technologies are going to unlock our ability to embrace that and go full force towards the future."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- CHRIS HOMER, COFOUNDER/CTO, THREDUP</span></div>
|
||||
</div>
|
||||
|
||||
<div class="fullcol">
|
||||
Perhaps the impact is most evident on the busiest days in retail. "Kubernetes enabled auto scaling in a seamless and easily manageable way on days like Black Friday," says Homer. "We no longer have to sit there adding instances, monitoring the traffic, doing a lot of manual work. That’s handled for us, and instead we can actually have some turkey, drink some wine and enjoy our families."
|
||||
<br><br>
|
||||
For ThredUP, Kubernetes fits perfectly with the company’s vision for how it’s changing retail. Some of what ThredUP does is still very manual: "As our customers send bags of items to our distribution centers, they’re photographed, inspected, tagged, and put online today," says Homer.
|
||||
<br><br>
|
||||
But in every other aspect, "we use different forms of technology to drive everything we do," Homer says. "We have machine learning algorithms to help predict the likelihood of sale for items, which drives our pricing algorithm. We have personalization algorithms that look at the images and try to determine style and match users’ preferences across our systems."
|
||||
<br><br>
|
||||
Count Kubernetes as one of those drivers. "Our future’s all about automation," says Homer, "and behind that, cloud native technologies are going to unlock our ability to embrace that and go full force towards the future."
|
||||
</div>
|
||||
</section>
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 4.6 KiB |
@@ -0,0 +1,97 @@
|
||||
---
|
||||
title: vsco Case Study
|
||||
linkTitle: vsco
|
||||
case_study_styles: true
|
||||
cid: caseStudies
|
||||
css: /css/style_case_studies.css
|
||||
featured: false
|
||||
---
|
||||
|
||||
<div class="banner1" style="background-image: url('/images/CaseStudy_vsco_banner1.jpg')">
|
||||
<h1> CASE STUDY:<img src="/images/vsco_logo.png" class="header_logo" style="width:17%;margin-bottom:-2%"><br> <div class="subhead" style="margin-top:1%">VSCO: How a Mobile App Saved 70% on Its EC2 Bill with Cloud Native
|
||||
</div></h1>
|
||||
|
||||
</div>
|
||||
|
||||
<div class="details">
|
||||
Company <b>VSCO</b> Location <b>Oakland, CA</b> Industry <b>Photo Mobile App</b>
|
||||
</div>
|
||||
|
||||
<hr>
|
||||
<section class="section1">
|
||||
<div class="cols">
|
||||
<div class="col1" style="width:100%"">
|
||||
<h2>Challenge</h2>
|
||||
After moving from <a href="https://www.rackspace.com/">Rackspace</a> to <a href="https://aws.amazon.com/">AWS</a> in 2015, <a href="https://vsco.co/">VSCO</a> began building <a href="https://nodejs.org/en/">Node.js</a> and <a href="https://golang.org/">Go</a> microservices in addition to running its <a href="http://php.net/">PHP monolith</a>. The team containerized the microservices using <a href="https://www.docker.com/">Docker</a>, but "they were all in separate groups of <a href="https://aws.amazon.com/ec2/">EC2</a> instances that were dedicated per service," says Melinda Lu, Engineering Manager for the Machine Learning Team. Adds Naveen Gattu, Senior Software Engineer on the Community Team: "That yielded a lot of wasted resources. We started looking for a way to consolidate and be more efficient in the AWS EC2 instances."
|
||||
<h2>Solution</h2>
|
||||
The team began exploring the idea of a scheduling system, and looked at several solutions including Mesos and Swarm before deciding to go with <a href="https://kubernetes.io/">Kubernetes</a>. VSCO also uses <a href="https://grpc.io/">gRPC</a> and <a href="https://www.envoyproxy.io/">Envoy</a> in their cloud native stack.
|
||||
<br>
|
||||
<h2>Impact</h2>
|
||||
Before, deployments required "a lot of manual tweaking, in-house scripting that we wrote, and because of our disparate EC2 instances, Operations had to babysit the whole thing from start to finish," says Senior Software Engineer Brendan Ryan. "We didn't really have a story around testing in a methodical way, and using reusable containers or builds in a standardized way." There's a faster onboarding process now. Before, the time to first deploy was two days' hands-on setup time; now it's two hours. By moving to continuous integration, containerization, and Kubernetes, velocity was increased dramatically. The time from code-complete to deployment in production on real infrastructure went from one to two weeks to two to four hours for a typical service. Adds Gattu: "In man hours, that's one person versus a developer and a DevOps individual at the same time." With an 80% decrease in time for a single deployment to happen in production, the number of deployments has increased as well, from 1200/year to 3200/year. There have been real dollar savings too: With Kubernetes, VSCO is running at 2x to 20x greater EC2 efficiency, depending on the service, adding up to about 70% overall savings on the company's EC2 bill. Ryan points to the company's ability to go from managing one large monolithic application to 50+ microservices with "the same size developer team, more or less. And we've only been able to do that because we have increased trust in our tooling and a lot more flexibility, so we don't need to employ a DevOps engineer to tune every service." With Kubernetes, gRPC, and Envoy in place, VSCO has seen an 88% reduction in total minutes of outage time, mainly due to the elimination of JSON-schema errors and service-specific infrastructure provisioning errors, and an increased speed in fixing outages.
|
||||
|
||||
</div>
|
||||
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner2">
|
||||
<div class="banner2text">
|
||||
"I've been really impressed seeing how our engineers have come up with creative solutions to things by just combining a lot of Kubernetes primitives. Exposing Kubernetes constructs as a service to our engineers as opposed to exposing higher order constructs has worked well for us. It lets you get familiar with the technology and do more interesting things with it."
|
||||
<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- MELINDA LU, ENGINEERING MANAGER FOR VSCO'S MACHINE LEARNING TEAM</span>
|
||||
</div>
|
||||
</div>
|
||||
<section class="section2">
|
||||
<div class="fullcol">
|
||||
<h2>A photography app for mobile, VSCO was born in the cloud in 2011. In the beginning, "we were using Rackspace and had one PHP monolith application talking to MySQL database, with FTP deployments, no containerization, no orchestration," says Software Engineer Brendan Ryan, "which was sufficient at the time."</h2>
|
||||
After VSCO moved to AWS in 2015 and its user base passed the 30 million mark, the team quickly realized that set-up wouldn't work anymore. Developers had started building some Node and Go microservices, which the team tried containerizing with Docker. But "they were all in separate groups of EC2 instances that were dedicated per service," says Melinda Lu, Engineering Manager for the Machine Learning Team. Adds Naveen Gattu, Senior Software Engineer on the Community Team: "That yielded a lot of wasted resources. We started looking for a way to consolidate and be more efficient in the EC2 instances."
|
||||
<br><br>
|
||||
With a checklist that included ease of use and implementation, level of support, and whether it was open source, the team evaluated a few scheduling solutions, including Mesos and Swarm, before deciding to go with Kubernetes. "Kubernetes seemed to have the strongest open source community around it," says Lu. Plus, "We had started to standardize on a lot of the Google stack, with Go as a language, and gRPC for almost all communication between our own services inside the data center. So it seemed pretty natural for us to choose Kubernetes."
|
||||
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner3" style="background-image: url('/images/CaseStudy_vsco_banner2.jpg')">
|
||||
<div class="banner3text">
|
||||
"Kubernetes seemed to have the strongest open source community around it, plus, we had started to standardize on a lot of the Google stack, with Go as a language, and gRPC for almost all communication between our own services inside the data center. So it seemed pretty natural for us to choose Kubernetes."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- MELINDA LU, ENGINEERING MANAGER FOR VSCO'S MACHINE LEARNING TEAM</span>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
<section class="section3">
|
||||
<div class="fullcol">
|
||||
At the time, there were few managed Kubernetes offerings and less tooling available in the ecosystem, so the team stood up its own cluster and built some custom components for its specific deployment needs, such as an automatic ingress controller and policy constructs for canary deploys. "We had already begun breaking up the monolith, so we moved things one by one, starting with pretty small, low-risk services," says Lu. "Every single new service was deployed there." The first service was migrated at the end of 2016, and after one year, 80% of the entire stack was on Kubernetes, including the rest of the monolith.
|
||||
<br><br>
|
||||
The impact has been great. Deployments used to require "a lot of manual tweaking, in-house scripting that we wrote, and because of our disparate EC2 instances, Operations had to babysit the whole thing from start to finish," says Ryan. "We didn't really have a story around testing in a methodical way, and using reusable containers or builds in a standardized way." There's a faster onboarding process now. Before, the time to first deploy was two days' hands-on setup time; now it's two hours.
|
||||
<br><br>
|
||||
By moving to continuous integration, containerization, and Kubernetes, velocity was increased dramatically. The time from code-complete to deployment in production on real infrastructure went from one to two weeks to two to four hours for a typical service. Plus, says Gattu, "In man hours, that's one person versus a developer and a DevOps individual at the same time." With an 80% decrease in time for a single deployment to happen in production, the number of deployments has increased as well, from 1200/year to 3200/year.
|
||||
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner4" style="background-image: url('/images/CaseStudy_vsco_banner4.jpg')">
|
||||
<div class="banner4text">
|
||||
"I've been really impressed seeing how our engineers have come up with really creative solutions to things by just combining a lot of Kubernetes primitives, exposing Kubernetes constructs as a service to our engineers as opposed to exposing higher order constructs has worked well for us. It lets you get familiar with the technology and do more interesting things with it." <br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- MELINDA LU, ENGINEERING MANAGER FOR VSCO’S MACHINE LEARNING TEAM</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<section class="section5" style="padding:0px !important">
|
||||
<div class="fullcol">
|
||||
There have been real dollar savings too: With Kubernetes, VSCO is running at 2x to 20x greater EC2 efficiency, depending on the service, adding up to about 70% overall savings on the company’s EC2 bill.
|
||||
<br><br>
|
||||
Ryan points to the company’s ability to go from managing one large monolithic application to 50+ microservices with “the same size developer team, more or less. And we’ve only been able to do that because we have increased trust in our tooling and a lot more flexibility when there are stress points in our system. You can increase CPU memory requirements of a service without having to bring up and tear down instances, and read through AWS pages just to be familiar with a lot of jargon, which isn’t really tenable for a company at our scale.”
|
||||
<br><br>
|
||||
Envoy and gRPC have also had a positive impact at VSCO. “We get many benefits from gRPC out of the box: type safety across multiple languages, ease of defining services with the gRPC IDL, built-in architecture like interceptors, and performance improvements over HTTP/1.1 and JSON,” says Lu.
|
||||
<br><br>
|
||||
VSCO was one of the first users of Envoy, getting it in production five days after it was open sourced. “We wanted to serve gRPC and HTTP/2 directly to mobile clients through our edge load balancers, and Envoy was our only reasonable solution,” says Lu. “The ability to send consistent and detailed stats by default across all services has made observability and standardization of dashboards much easier.” The metrics that come built in with Envoy have also “greatly helped with debugging,” says DevOps Engineer Ryan Nguyen.
|
||||
</div>
|
||||
|
||||
<div class="banner5" style="width:100%">
|
||||
<div class="banner5text">
|
||||
"Because there’s now an organization that supports Kubernetes, does that build confidence? The answer is a resounding yes."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- NAVEEN GATTU, SENIOR SOFTWARE ENGINEER ON VSCO’S COMMUNITY TEAM</span></div>
|
||||
</div>
|
||||
|
||||
<div class="fullcol">
|
||||
With Kubernetes, gRPC, and Envoy in place, VSCO has seen an 88% reduction in total minutes of outage time, mainly due to the elimination of JSON-schema errors and service-specific infrastructure provisioning errors, and an increased speed in fixing outages.
|
||||
<br><br>
|
||||
Given its success using CNCF projects, VSCO is starting to experiment with others, including <a href="https://github.com/containernetworking">CNI</a> and Prometheus. “To have a large organization backing these technologies, we have a lot more confidence trying this software and deploying to production,” says Nguyen.
|
||||
<br><br>
|
||||
The team has made contributions to gRPC and Envoy, and is hoping to be even more active in the CNCF community. “I’ve been really impressed seeing how our engineers have come up with really creative solutions to things by just combining a lot of Kubernetes primitives,” says Lu. “Exposing Kubernetes constructs as a service to our engineers as opposed to exposing higher order constructs has worked well for us. It lets you get familiar with the technology and do more interesting things with it.”
|
||||
|
||||
</div>
|
||||
</section>
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 11 KiB |
@@ -0,0 +1,96 @@
|
||||
---
|
||||
title: Woorank Case Study
|
||||
linkTitle: woorank
|
||||
case_study_styles: true
|
||||
cid: caseStudies
|
||||
css: /css/style_case_studies.css
|
||||
featured: false
|
||||
---
|
||||
|
||||
|
||||
<div class="banner1" style="background-image: url('/images/CaseStudy_woorank_banner1.jpg')">
|
||||
<h1> CASE STUDY:<img src="/images/woorank_logo.png" class="header_logo" style="width:25%;margin-bottom:-1%"><br> <div class="subhead" style="margin-top:1%">Woorank: How Kubernetes Helped a Startup Manage 50 Microservices with<br>12 Engineers—At 30% Less Cost
|
||||
</div></h1>
|
||||
|
||||
</div>
|
||||
|
||||
<div class="details">
|
||||
Company <b>Woorank</b> Location <b>Brussels, Belgium</b> Industry <b>Digital marketing tool</b>
|
||||
</div>
|
||||
|
||||
<hr>
|
||||
<section class="section1">
|
||||
<div class="cols">
|
||||
<div class="col1" style="width:100%"">
|
||||
|
||||
<h2>Challenge</h2>
|
||||
Founded in 2011, Woorank embraced microservices and containerization early on, so its core product, a tool that helps digital marketers improve their websites’ visibility on the internet, consists of 50 applications developed and maintained by a technical team of 12. For two years, the infrastructure ran smoothly on Mesos, but “there were still lots of our own libraries that we had to roll and applications that we had to bring in, so it was very cumbersome for us as a small team to keep those things alive and to update them,” says CTO/Cofounder Nils De Moor. So he began looking for a new solution with more automation and self-healing built in, that would better suit the company’s human resources.
|
||||
<h2>Solution</h2>
|
||||
De Moor decided to switch to <a href="https://kubernetes.io/">Kubernetes</a> running on <a href="https://aws.amazon.com/">AWS</a>, which “allows us to just define applications, how they need to run, how scalable they need to be, and it takes pain away from the developers thinking about that,” he says. “When things fail and errors pop up, the system tries to heal itself, and that’s really, for us, the key reason to work with Kubernetes.” The company now also uses <a href="https://www.fluentd.org/">Fluentd</a>, <a href="https://prometheus.io/">Prometheus</a>, and <a href="http://opentracing.io/">OpenTracing</a>.
|
||||
<h2>Impact</h2>
|
||||
The company’s number one concern was immediately erased: Maintaining Kubernetes takes just one person on staff, and it’s not a fulltime job. Infrastructure updates used to take two active working days; now it’s just a matter of “a few hours of passively following the process,” says De Moor. Implementing new tools—which once took weeks of planning, installing, and onboarding—now only takes a few days. “We were already pretty flexible in our costs and taking on traffic peaks and higher load in general,” adds De Moor, “but with Kubernetes and the other CNCF tools we use, we have achieved about 30% in cost savings.” Plus, the rate of deployments per day has nearly doubled.
|
||||
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner2">
|
||||
<div class="banner2text">
|
||||
“It was definitely important for us to have CNCF as an umbrella above everything. We’ve always been working with open source libraries and tools and technologies. It works very well for us, but sometimes things can drift, maintainers drop out, and projects go haywire. For us, it was indeed important to know that whatever project gets taken under this umbrella, it’s taken very seriously. Our way of contributing back is also by joining this community. It’s, for us, a way to show our appreciation for what’s going on in this framework.”
|
||||
<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>— NILS DE MOOR, CTO/COFOUNDER, WOORANK</span>
|
||||
</div>
|
||||
</div>
|
||||
<section class="section2">
|
||||
<div class="fullcol">
|
||||
<h2>Woorank’s core product is a tool that enables digital marketers to improve their websites’ visibility on the internet.</h2>
|
||||
“We help them acquire lots of data and then present it to them in meaningful ways so they can work with it,” says CTO/Cofounder Nils De Moor. In its seven years as a startup, the company followed a familiar technological path to build that product: starting with a monolithic application, breaking it down into microservices, and then embracing containerization. “That’s where our modern infrastructure started out,” says De Moor.
|
||||
<br><br>
|
||||
As new features have been added to the product, it has grown to consist of 50 applications under the hood. Though Docker had made things easier to deploy, and the team had been using Mesos as an orchestration framework on AWS since 2015, De Moor realized there was still too much overhead to managing the infrastructure, especially with a technical team of just 12.
|
||||
<br><br>
|
||||
“The pain point was that there were still lots of our own libraries that we had to roll and applications that we had to bring in, so it was very cumbersome for us as a small team to keep those things alive and to update them,” says De Moor. “When things went wrong during deployment, someone manually had to come in and figure it out. It wasn’t necessarily that the technology or anything was wrong with Mesos; it was just not really fitting our model of being a small company, not having the human resources to make sure it all works and can be updated.”
|
||||
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner3" style="background-image: url('/images/CaseStudy_woorank_banner3.jpg')">
|
||||
<div class="banner3text">
|
||||
"Cloud native technologies have brought to us a transparency on everything going on in our system, from the code to the server. It has brought huge cost savings and a better way of dealing with those costs and keeping them under control. And performance-wise, it has helped our team understand how we can make our code work better on the cloud native infrastructure."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>— NILS DE MOOR, CTO/COFOUNDER, WOORANK</span>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<section class="section3">
|
||||
<div class="fullcol">
|
||||
Around the time Woorank was grappling with these issues, Kubernetes was emerging as a technology. De Moor knew that he wanted a platform that would be more automated and self-healing, and when he began experimenting with Kubernetes, he found that it checked all those boxes. “Kubernetes allows us to just define applications, how they need to run, how scalable they need to be, and it takes pain away from the developers thinking about that,” he says. “When things fail and errors pop up, the system tries to heal itself, and that’s really, for us, the key reason to work with Kubernetes. It allowed us to set up certain testing frameworks to just be alerted when things go wrong, instead of having to look at whether everything went right. It’s made people’s lives much easier. It’s quite a big mindset change.”
|
||||
<br><br>
|
||||
Once one small Kubernetes cluster was up and running, the team began moving over a few applications at a time, gradually increasing the load over the course of several months. By early 2017, Woorank was 100% deployed on Kubernetes.
|
||||
<br><br>
|
||||
The company’s number one concern was immediately erased: Maintaining Kubernetes is the responsibility of just one person on staff, and it’s not his fulltime job. Updating the old infrastructure “was always a pain,” says De Moor: It used to take two active working days, “and it was always a bit scary when we did that.” With Kubernetes, it’s just a matter of “a few hours of passively following the process.”
|
||||
</div>
|
||||
</section>
|
||||
<div class="banner4" style="background-image: url('/images/CaseStudy_woorank_banner4.jpg')" style="width:100%">
|
||||
<div class="banner4text">
|
||||
"When things fail and errors pop up, the system tries to heal itself, and that’s really, for us, the key reason to work with Kubernetes. It allowed us to set up certain testing frameworks to just be alerted when things go wrong, instead of having to look at whether everything went right. It’s made people’s lives much easier. It’s quite a big mindset change." <br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- NILS DE MOOR, CTO/COFOUNDER, WOORANK</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<section class="section5" style="padding:0px !important">
|
||||
<div class="fullcol">
|
||||
Transparency on all levels, from the code to the servers, has also been a byproduct of the move to Kubernetes. “It’s easier for the entire team to get a better understanding of the infrastructure, how it’s working, how it looks like, what’s going on,” says De Moor. “It’s not that thing that’s running, and no one really knows how it works except this one person. Now it’s really a team effort of everyone knowing, ‘Okay, when something goes wrong, it’s probably in this area or we need to check this.’”
|
||||
<br><br>
|
||||
To that end, Woorank has begun implementing other cloud native tools that help with visibility, such as Fluentd for logging, Prometheus for monitoring, and OpenTracing for distributed tracing. Implementing these new tools—which once took weeks of planning, installing, and onboarding—now only takes a few days. “With all the tools and projects under the CNCF umbrella, it’s easier for us to test and play with technology than it used to be,” says De Moor. “With Prometheus, we used it fairly early and couldn’t get it fairly stable. A couple of months ago, the question reappeared, so we set it up in two days, and now everyone is using it.”
|
||||
<br><br>
|
||||
Deployments, too, have been impacted: The rate has more than doubled, which De Moor partly attributes to the transparency of the new process. “With Kubernetes, you see that these three containers didn’t start for this reason,” he says. Plus, “now we bring deployment messages into Slack. If you see deployments rolling by every day, it does somehow indirectly enforce you, okay, I need to be part of this train, so I also need to deploy.”
|
||||
</div>
|
||||
|
||||
<div class="banner5" >
|
||||
<div class="banner5text">
|
||||
"We can plan those things over a certain timeline, try to fit our resource usage to that, and then bring in spot instances, which will hopefully drive the costs down more."<br style="height:25px"><span style="font-size:14px;letter-spacing:2px;text-transform:uppercase;margin-top:5% !important;"><br>- NILS DE MOOR, CTO/COFOUNDER, WOORANK</span></div>
|
||||
</div>
|
||||
|
||||
<div class="fullcol">
|
||||
Perhaps the biggest impact, though, has been on the bottom line. “We were already pretty flexible in our costs and taking on traffic peaks and higher load in general, but with Kubernetes and the other CNCF tools we use, we have achieved about 30% in cost savings,” says De Moor.
|
||||
<br><br>
|
||||
And there’s room for even greater savings. Currently, most of Woorank’s infrastructure is running on AWS on demand; the company pays a fixed price and makes some reservations for its planned amount of resources needed. De Moor is planning to experiment more with spot instances with certain resource-heavy workloads such as web crawls: “We can plan those things over a certain timeline, try to fit our resource usage to that, and then bring in spot instances, which will hopefully drive the costs down more.”
|
||||
<br><br>
|
||||
Moving to Kubernetes has been so beneficial to Woorank that the company is doubling down on both cloud native technologies and the community. “It was definitely important for us to have CNCF as an umbrella above everything,” says De Moor. “We’ve always been working with open source libraries and tools and technologies. It works very well for us, but sometimes things can drift, maintainers drop out, and projects go haywire. For us, it was indeed important to know that whatever project gets taken under this umbrella, it’s taken very seriously. Our way of contributing back is also by joining this community. It’s, for us, a way to show our appreciation for what’s going on in this framework.”
|
||||
</div>
|
||||
</section>
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 6.3 KiB |
@@ -1,69 +1,236 @@
|
||||
---
|
||||
title: Community
|
||||
layout: basic
|
||||
cid: community
|
||||
---
|
||||
---
|
||||
title: Community
|
||||
layout: basic
|
||||
cid: community
|
||||
---
|
||||
|
||||
<div class="newcommunitywrapper">
|
||||
<div class="banner1">
|
||||
<img src="/images/community/kubernetes-community-final-02.jpg" style="width:100%" class="desktop">
|
||||
<img src="/images/community/kubernetes-community-02-mobile.jpg" style="width:100%" class="mobile">
|
||||
</div>
|
||||
|
||||
<div class="intro">
|
||||
<br class="mobile">
|
||||
<p>The Kubernetes community -- users, contributors, and the culture we've built together -- is one of the biggest reasons for the meteoric rise of this open source project. Our culture and values continue to grow and change as the project itself grows and changes. We all work together toward constant improvement of the project and the ways we work on it.
|
||||
<br><br>We are the people who file issues and pull requests, attend SIG meetings, Kubernetes meetups, and KubeCon, advocate for it's adoption and innovation, run <code>kubectl get pods</code>, and contribute in a thousand other vital ways. Read on to learn how you can get involved and become part of this amazing community.</p>
|
||||
<br class="mobile">
|
||||
</div>
|
||||
|
||||
<div class="navbar">
|
||||
|
||||
<a href="#conduct">Code of conduct </a>
|
||||
<a href="#videos">Videos</a>
|
||||
<a href="#discuss">Discussions</a>
|
||||
<a href="#events">Events and meetups</a>
|
||||
<a href="#news">News</a>
|
||||
|
||||
<section id="mainContent">
|
||||
<main>
|
||||
<div class="content">
|
||||
<h3>Ensuring Kubernetes works well everywhere and for everyone.</h3>
|
||||
<p>Connect with the Kubernetes community on our <a href="http://slack.k8s.io/">Slack channel</a>, <a href="https://discuss.kubernetes.io/">discussion board</a>, or join the
|
||||
<a href="https://groups.google.com/forum/#!forum/kubernetes-dev">Kubernetes-dev Google group</a>. A weekly
|
||||
community meeting takes place via video conference to discuss the state of affairs, see
|
||||
<a href="https://github.com/kubernetes/community/blob/master/events/community-meeting.md">these instructions</a> for information
|
||||
on how to participate.</p>
|
||||
<p>You can also join Kubernetes all around the world through our
|
||||
<a href="https://www.meetup.com/topics/kubernetes/">Kubernetes Meetup Community</a> and the
|
||||
<a href="https://www.meetup.com/Kubernetes-Cloud-Native-Online-Meetup/">Kubernetes Cloud Native Meetup Community</a>.</p>
|
||||
</div>
|
||||
<div class="content">
|
||||
<h3>Special Interest Groups (SIGs)</h3>
|
||||
<p>Have a special interest in how Kubernetes works with another technology? See our ever growing
|
||||
<a href="https://git.k8s.io/community/sig-list.md">lists of SIGs</a>,
|
||||
from AWS and Openstack to Big Data and Scalability, there's a place for you to contribute and instructions
|
||||
for forming a new SIG if your special interest isn't covered (yet).</p>
|
||||
</div>
|
||||
<br class="mobile"><br class="mobile">
|
||||
<div class="imagecols">
|
||||
<br class="mobile">
|
||||
<div class="imagecol">
|
||||
<img src="/images/community/kubernetes-community-final-03.jpg" style="width:100%" class="desktop">
|
||||
</div>
|
||||
|
||||
<div class="imagecol">
|
||||
<img src="/images/community/kubernetes-community-final-04.jpg" style="width:100%" class="desktop">
|
||||
</div>
|
||||
|
||||
<div class="imagecol" style="margin-right:0% important">
|
||||
<img src="/images/community/kubernetes-community-final-05.jpg" style="width:100%;margin-right:0% important" class="desktop">
|
||||
</div>
|
||||
<img src="/images/community/kubernetes-community-04-mobile.jpg" style="width:100%;margin-bottom:3%" class="mobile">
|
||||
|
||||
<a name="conduct"></a>
|
||||
</div>
|
||||
|
||||
|
||||
|
||||
<div class="conduct">
|
||||
<div class="conducttext">
|
||||
<br class="mobile"><br class="mobile">
|
||||
<br class="tablet"><br class="tablet">
|
||||
<div class="conducttextnobutton" style="margin-bottom:2%"><h1>Code of Conduct</h1>
|
||||
The Kubernetes community values respect and inclusiveness, and enforces a Code of Conduct in all interactions. If you notice a violation of the Code of Conduct at an event or meeting, in Slack, or in another communication mechanism, reach out to the Kubernetes Code of Conduct Committee at <a href="mailto:conduct@kubernetes.io" style="color:#0662EE;font-weight:300">conduct@kubernetes.io</a>. All reports are kept confidential. You can read about the committee <a href="https://github.com/kubernetes/community/tree/master/committee-code-of-conduct" style="color:#0662EE;font-weight:300">here</a>.
|
||||
<br>
|
||||
<a href="https://kubernetes.io/community/code-of-conduct/">
|
||||
<br class="mobile"><br class="mobile">
|
||||
|
||||
<span class="fullbutton">
|
||||
READ MORE
|
||||
</span>
|
||||
</a>
|
||||
</div><a name="videos"></a>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
|
||||
<div class="videos">
|
||||
<br class="mobile"><br class="mobile">
|
||||
<br class="tablet"><br class="tablet">
|
||||
<h1 style="margin-top:0px">Videos</h1>
|
||||
|
||||
<div style="margin-bottom:4%;font-weight:300;text-align:center;padding-left:10%;padding-right:10%">We're on YouTube, a lot. Subscribe for a wide range of topics.</div>
|
||||
|
||||
<div class="videocontainer">
|
||||
|
||||
<div class="video">
|
||||
|
||||
<iframe width="100%" height="250" src="https://www.youtube.com/embed/videoseries?list=PL69nYSiGNLP3azFUvYJjGn45YbF6C-uIg" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>
|
||||
|
||||
<a href="https://www.youtube.com/playlist?list=PL69nYSiGNLP3azFUvYJjGn45YbF6C-uIg">
|
||||
<div class="videocta">
|
||||
Watch monthly office hours ▶</div>
|
||||
</a>
|
||||
</div>
|
||||
|
||||
<div class="video">
|
||||
<iframe width="100%" height="250" src="https://www.youtube.com/embed/videoseries?list=PL69nYSiGNLP1pkHsbPjzAewvMgGUpkCnJ" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>
|
||||
<a href="https://www.youtube.com/playlist?list=PL69nYSiGNLP1pkHsbPjzAewvMgGUpkCnJ">
|
||||
<div class="videocta">
|
||||
Watch weekly community meetings ▶
|
||||
</div>
|
||||
</a>
|
||||
</div>
|
||||
|
||||
<div class="video">
|
||||
|
||||
<iframe width="100%" height="250" src="https://www.youtube.com/embed/videoseries?list=PL69nYSiGNLP3QpQrhZq_sLYo77BVKv09F" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>
|
||||
|
||||
<a href="https://www.youtube.com/playlist?list=PL69nYSiGNLP3QpQrhZq_sLYo77BVKv09F">
|
||||
<div class="videocta">
|
||||
Watch a talk from a community member ▶
|
||||
</div>
|
||||
|
||||
</a>
|
||||
<a name="discuss"></a>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
|
||||
<div class="resources">
|
||||
<br class="mobile"><br class="mobile">
|
||||
<br class="tablet"><br class="tablet">
|
||||
<h1 style="padding-top:1%">Discussions</h1>
|
||||
|
||||
<div style="font-weight:300;text-align:center">We talk a lot. Find us and join the conversation on any of these platforms.</div>
|
||||
|
||||
<div class="resourcecontainer">
|
||||
|
||||
<div class="resourcebox">
|
||||
<img src="/images/community/discuss.png" style="width:80%;padding-bottom:2%">
|
||||
<a href="https://discuss.kubernetes.io/" style="color:#0662EE;display:block;margin-top:1%">
|
||||
forum ▶
|
||||
</a>
|
||||
<div class="resourceboxtext" style="font-size:12px;text-transform:none !important;font-weight:300;line-height:1.4em;color:#333333;margin-top:4%">
|
||||
Topic-based technical discussions that bridge docs, StackOverflow, and so much more
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="resourcebox">
|
||||
<img src="/images/community/twitter.png" style="width:80%;padding-bottom:2%">
|
||||
<a href="https://twitter.com/kubernetesio" style="color:#0662EE;display:block;margin-top:1%">
|
||||
twitter ▶
|
||||
</a>
|
||||
<div class="resourceboxtext" style="font-size:12px;text-transform:none !important;font-weight:300;line-height:1.4em;color:#333333;margin-top:4%">Real-time announcements of blog posts, events, news, ideas
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="resourcebox">
|
||||
<img src="/images/community/github.png" style="width:80%;padding-bottom:2%">
|
||||
<a href="https://github.com/kubernetes/kubernetes" style="color:#0662EE;display:block;margin-top:1%">
|
||||
github ▶
|
||||
</a>
|
||||
<div class="resourceboxtext" style="font-size:12px;text-transform:none !important;font-weight:300;line-height:1.4em;color:#333333;margin-top:4%">
|
||||
All the project and issue tracking, plus of course code
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="resourcebox">
|
||||
<img src="/images/community/stack.png" style="width:80%;padding-bottom:2%">
|
||||
<a href="https://stackoverflow.com/search?q=kubernetes" style="color:#0662EE;display:block;margin-top:1%">
|
||||
stack overflow ▶
|
||||
</a>
|
||||
<div class="resourceboxtext" style="font-size:12px;text-transform:none !important;font-weight:300;line-height:1.4em;color:#333333;margin-top:4%">
|
||||
Technical troubleshooting for any use case
|
||||
<a name="events"></a>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!--
|
||||
<div class="resourcebox">
|
||||
|
||||
<img src="/images/community/slack.png" style="width:80%">
|
||||
|
||||
slack ▶
|
||||
|
||||
<div class="resourceboxtext" style="font-size:11px;text-transform:none !important;font-weight:200;line-height:1.4em;color:#333333;margin-top:4%">
|
||||
With 170+ channels, you'll find one that fits your needs.
|
||||
</div>
|
||||
|
||||
</div>-->
|
||||
|
||||
</div>
|
||||
</div>
|
||||
<div class="events">
|
||||
<br class="mobile"><br class="mobile">
|
||||
<br class="tablet"><br class="tablet">
|
||||
<div class="eventcontainer">
|
||||
<h1 style="color:white !important">Upcoming Events</h1>
|
||||
{{< upcoming-events >}}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="meetups">
|
||||
<div class="meetupcol">
|
||||
<div class="meetuptext">
|
||||
<h1 style="text-align:left">Global Community</h1>
|
||||
With over 150 meetups in the world and growing, go find your local kube people. If one isn't near, take charge and create your own.
|
||||
</div>
|
||||
<a href="https://www.meetup.com/topics/kubernetes/">
|
||||
<div class="button">
|
||||
FIND A MEETUP
|
||||
</div>
|
||||
</a>
|
||||
<a name="news"></a>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<p>As a member of the Kubernetes community, you are welcome to join any of the SIG meetings
|
||||
you are interested in. No registration required.</p>
|
||||
|
||||
</div>
|
||||
<!--
|
||||
<div class="contributor">
|
||||
<div class="contributortext">
|
||||
<br>
|
||||
<h1 style="text-align:left">
|
||||
New Contributors Site
|
||||
</h1>
|
||||
Text about new contributors site.
|
||||
|
||||
<br><br>
|
||||
|
||||
<div class="button">
|
||||
VISIT SITE
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="content">
|
||||
<h3>Code of Conduct</h3>
|
||||
<p>The Kubernetes community values respect and inclusiveness, and
|
||||
enforces a <a href="code-of-conduct/">Code of Conduct</a> in all
|
||||
interactions. If you notice a violation of the Code of Conduct at
|
||||
an event or meeting, in Slack, or in another communication
|
||||
mechanism, reach out to the <a href="https://github.com/kubernetes/community/tree/master/committee-code-of-conduct">Kubernetes Code of Conduct Committee</a>
|
||||
<a href="mailto:conduct@kubernetes.io">conduct@kubernetes.io</a>.
|
||||
Your anonymity will be protected.
|
||||
</p>
|
||||
</div>
|
||||
</main>
|
||||
</section>
|
||||
-->
|
||||
|
||||
<div class="news">
|
||||
<br class="mobile"><br class="mobile">
|
||||
<br class="tablet"><br class="tablet">
|
||||
<h1 style="margin-bottom:2%">Recent News</h1>
|
||||
|
||||
<br>
|
||||
<div class="twittercol1">
|
||||
<a class="twitter-timeline" data-tweet-limit="1" href="https://twitter.com/kubernetesio?ref_src=twsrc%5Etfw">Tweets by kubernetesio</a> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
|
||||
</div>
|
||||
|
||||
<section id="talkToUs">
|
||||
<main>
|
||||
<h3>Talk to Us!</h3>
|
||||
<h4>We would love to hear from you, how you are using Kubernetes,<br> and what we can do to make it better.</h4>
|
||||
<div id="bigSocial">
|
||||
<div>
|
||||
<a href="https://twitter.com/kubernetesio">@kubernetesio</a>
|
||||
<p>Get the latest news and updates.</p>
|
||||
</div>
|
||||
<div>
|
||||
<a href="https://github.com/kubernetes/kubernetes">Github Project</a>
|
||||
<p>Check out the project and consider contributing.</p>
|
||||
</div>
|
||||
<div>
|
||||
<a href="http://slack.k8s.io/">#kubernetes-users</a>
|
||||
<p>Our Slack channel is the best way to contact our engineers and share your ideas with them.</p>
|
||||
</div>
|
||||
<div>
|
||||
<a href="http://stackoverflow.com/questions/tagged/kubernetes">Stack Overflow</a>
|
||||
<p>Our user forum is a great place to go for community support.</p>
|
||||
</div>
|
||||
</div>
|
||||
</main>
|
||||
</section>
|
||||
<br>
|
||||
<br><br><br><br>
|
||||
</div>
|
||||
|
||||
</div>
|
||||
|
||||
@@ -11,7 +11,7 @@ css: /css/community.css
|
||||
Kubernetes follows the
|
||||
<a href="https://github.com/cncf/foundation/blob/master/code-of-conduct.md">CNCF Code of Conduct</a>.
|
||||
The text of the CNCF CoC is replicated below, as of
|
||||
<a href="https://github.com/cncf/foundation/blob/0ce4694e5103c0c24ca90c189da81e5408a46632/code-of-conduct.md">commit 0ce4694</a>.
|
||||
<a href="https://github.com/cncf/foundation/blob/214585e24aab747fb85c2ea44fbf4a2442e30de6/code-of-conduct.md">commit 214585e</a>.
|
||||
If you notice that this is out of date, please
|
||||
<a href="https://github.com/kubernetes/website/issues/new">file an issue</a>.
|
||||
|
||||
|
||||
@@ -34,8 +34,7 @@ Conduct may be permanently removed from the project team.
|
||||
This code of conduct applies both within project spaces and in public spaces
|
||||
when an individual is representing the project or its community.
|
||||
|
||||
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting
|
||||
the [Kubernetes Code of Conduct Committee](https://github.com/kubernetes/community/tree/master/committee-code-of-conduct) <conduct@kubernetes.io>.
|
||||
Instances of abusive, harassing, or otherwise unacceptable behavior in Kubernetes may be reported by contacting the [Kubernetes Code of Conduct Committee](https://git.k8s.io/community/committee-code-of-conduct) via <conduct@kubernetes.io>. For other projects, please contact a CNCF project maintainer or our mediator, Mishi Choudhary <mishi@linux.com>.
|
||||
|
||||
This Code of Conduct is adapted from the Contributor Covenant
|
||||
(http://contributor-covenant.org), version 1.2.0, available at
|
||||
@@ -43,4 +42,4 @@ http://contributor-covenant.org/version/1/2/0/
|
||||
|
||||
### CNCF Events Code of Conduct
|
||||
|
||||
CNCF events are governed by the Linux Foundation [Code of Conduct](http://events.linuxfoundation.org/events/cloudnativecon/attend/code-of-conduct) available on the event page. This is designed to be compatible with the above policy and also includes more details on responding to incidents.
|
||||
CNCF events are governed by the Linux Foundation [Code of Conduct](https://events.linuxfoundation.org/code-of-conduct/) available on the event page. This is designed to be compatible with the above policy and also includes more details on responding to incidents.
|
||||
|
||||
@@ -17,7 +17,7 @@ The Concepts section helps you learn about the parts of the Kubernetes system an
|
||||
|
||||
To work with Kubernetes, you use *Kubernetes API objects* to describe your cluster's *desired state*: what applications or other workloads you want to run, what container images they use, the number of replicas, what network and disk resources you want to make available, and more. You set your desired state by creating objects using the Kubernetes API, typically via the command-line interface, `kubectl`. You can also use the Kubernetes API directly to interact with the cluster and set or modify your desired state.
|
||||
|
||||
Once you've set your desired state, the *Kubernetes Control Plane* works to make the cluster's current state match the desired state. To do so, Kubernetes performs a variety of tasks automatically--such as starting or restarting containers, scaling the number of replicas of a given application, and more. The Kubernetes Control Plane consists of a collection of processes running on your cluster:
|
||||
Once you've set your desired state, the *Kubernetes Control Plane* makes the cluster's current state match the desired state via the Pod Lifecycle Event Generator (PLEG). To do so, Kubernetes performs a variety of tasks automatically--such as starting or restarting containers, scaling the number of replicas of a given application, and more. The Kubernetes Control Plane consists of a collection of processes running on your cluster:
|
||||
|
||||
* The **Kubernetes Master** is a collection of three processes that run on a single node in your cluster, which is designated as the master node. Those processes are: [kube-apiserver](/docs/admin/kube-apiserver/), [kube-controller-manager](/docs/admin/kube-controller-manager/) and [kube-scheduler](/docs/admin/kube-scheduler/).
|
||||
* Each individual non-master node in your cluster runs two processes:
|
||||
@@ -47,13 +47,13 @@ In addition, Kubernetes contains a number of higher-level abstractions called Co
|
||||
|
||||
The various parts of the Kubernetes Control Plane, such as the Kubernetes Master and kubelet processes, govern how Kubernetes communicates with your cluster. The Control Plane maintains a record of all of the Kubernetes Objects in the system, and runs continuous control loops to manage those objects' state. At any given time, the Control Plane's control loops will respond to changes in the cluster and work to make the actual state of all the objects in the system match the desired state that you provided.
|
||||
|
||||
For example, when you use the Kubernetes API to create a Deployment object, you provide a new desired state for the system. The Kubernetes Control Plane records that object creation, and carries out your instructions by starting the required applications and scheduling them to cluster nodes--thus making the cluster's actual state match the desired state.
|
||||
For example, when you use the Kubernetes API to create a Deployment, you provide a new desired state for the system. The Kubernetes Control Plane records that object creation, and carries out your instructions by starting the required applications and scheduling them to cluster nodes--thus making the cluster's actual state match the desired state.
|
||||
|
||||
### Kubernetes Master
|
||||
|
||||
The Kubernetes master is responsible for maintaining the desired state for your cluster. When you interact with Kubernetes, such as by using the `kubectl` command-line interface, you're communicating with your cluster's Kubernetes master.
|
||||
|
||||
> The "master" refers to a collection of processes managing the cluster state. Typically these processes are all run on a single node in the cluster, and this node is also referred to as the master. The master can also be replicated for availability and redundancy.
|
||||
> The "master" refers to a collection of processes managing the cluster state. Typically all these processes run on a single node in the cluster, and this node is also referred to as the master. The master can also be replicated for availability and redundancy.
|
||||
|
||||
### Kubernetes Nodes
|
||||
|
||||
|
||||
@@ -48,8 +48,6 @@ In version 1.9, the CCM runs the following controllers from the preceding list:
|
||||
* Route controller
|
||||
* Service controller
|
||||
|
||||
Additionally, it runs another controller called the PersistentVolumeLabels controller. This controller is responsible for setting the zone and region labels on PersistentVolumes created in GCP and AWS clouds.
|
||||
|
||||
{{< note >}}
|
||||
Volume controller was deliberately chosen to not be a part of CCM. Due to the complexity involved and due to the existing efforts to abstract away vendor specific volume logic, it was decided that volume controller will not be moved to CCM.
|
||||
{{< /note >}}
|
||||
@@ -69,7 +67,6 @@ The majority of the CCM's functions are derived from the KCM. As mentioned in th
|
||||
* Node controller
|
||||
* Route controller
|
||||
* Service controller
|
||||
* PersistentVolumeLabels controller
|
||||
|
||||
#### Node controller
|
||||
|
||||
@@ -87,15 +84,7 @@ The Route controller is responsible for configuring routes in the cloud appropri
|
||||
|
||||
#### Service Controller
|
||||
|
||||
The Service controller is responsible for listening to service create, update, and delete events. Based on the current state of the services in Kubernetes, it configures cloud load balancers (such as ELB or Google LB) to reflect the state of the services in Kubernetes. Additionally, it ensures that service backends for cloud load balancers are up to date.
|
||||
|
||||
#### PersistentVolumeLabels controller
|
||||
|
||||
The PersistentVolumeLabels controller applies labels on AWS EBS/GCE PD volumes when they are created. This removes the need for users to manually set the labels on these volumes.
|
||||
|
||||
These labels are essential for the scheduling of pods as these volumes are constrained to work only within the region/zone that they are in. Any Pod using these volumes needs to be scheduled in the same region/zone.
|
||||
|
||||
The PersistentVolumeLabels controller was created specifically for the CCM; that is, it did not exist before the CCM was created. This was done to move the PV labelling logic in the Kubernetes API server (it was an admission controller) to the CCM. It does not run on the KCM.
|
||||
The Service controller is responsible for listening to service create, update, and delete events. Based on the current state of the services in Kubernetes, it configures cloud load balancers (such as ELB , Google LB, or Oracle Cloud Infrastructure LB) to reflect the state of the services in Kubernetes. Additionally, it ensures that service backends for cloud load balancers are up to date.
|
||||
|
||||
### 2. Kubelet
|
||||
|
||||
@@ -103,10 +92,6 @@ The Node controller contains the cloud-dependent functionality of the kubelet. P
|
||||
|
||||
In this new model, the kubelet initializes a node without cloud-specific information. However, it adds a taint to the newly created node that makes the node unschedulable until the CCM initializes the node with cloud-specific information. It then removes this taint.
|
||||
|
||||
### 3. Kubernetes API server
|
||||
|
||||
The PersistentVolumeLabels controller moves the cloud-dependent functionality of the Kubernetes API server to the CCM as described in the preceding sections.
|
||||
|
||||
## Plugin mechanism
|
||||
|
||||
The cloud controller manager uses Go interfaces to allow implementations from any cloud to be plugged in. Specifically, it uses the CloudProvider Interface defined [here](https://github.com/kubernetes/cloud-provider/blob/9b77dc1c384685cb732b3025ed5689dd597a5971/cloud.go#L42-L62).
|
||||
@@ -157,17 +142,6 @@ v1/Service:
|
||||
- Patch
|
||||
- Update
|
||||
|
||||
### PersistentVolumeLabels controller
|
||||
|
||||
The PersistentVolumeLabels controller listens on PersistentVolume (PV) create events and then updates them. This controller requires access to get and update PVs.
|
||||
|
||||
v1/PersistentVolume:
|
||||
|
||||
- Get
|
||||
- List
|
||||
- Watch
|
||||
- Update
|
||||
|
||||
### Others
|
||||
|
||||
The implementation of the core of CCM requires access to create events, and to ensure secure operation, it requires access to create ServiceAccounts.
|
||||
@@ -252,9 +226,9 @@ The following cloud providers have implemented CCMs:
|
||||
|
||||
* [Digital Ocean](https://github.com/digitalocean/digitalocean-cloud-controller-manager)
|
||||
* [Oracle](https://github.com/oracle/oci-cloud-controller-manager)
|
||||
* [Azure](https://github.com/kubernetes/kubernetes/tree/master/pkg/cloudprovider/providers/azure)
|
||||
* [GCE](https://github.com/kubernetes/kubernetes/tree/master/pkg/cloudprovider/providers/gce)
|
||||
* [AWS](https://github.com/kubernetes/kubernetes/tree/master/pkg/cloudprovider/providers/aws)
|
||||
* [Azure](https://github.com/kubernetes/cloud-provider-azure)
|
||||
* [GCP](https://github.com/kubernetes/cloud-provider-gcp)
|
||||
* [AWS](https://github.com/kubernetes/cloud-provider-aws)
|
||||
* [BaiduCloud](https://github.com/baidu/cloud-provider-baiducloud)
|
||||
* [Linode](https://github.com/linode/linode-cloud-controller-manager)
|
||||
|
||||
|
||||
@@ -26,8 +26,8 @@ architecture design doc for more details.
|
||||
A node's status contains the following information:
|
||||
|
||||
* [Addresses](#addresses)
|
||||
* [Condition](#condition)
|
||||
* [Capacity](#capacity)
|
||||
* [Conditions](#condition)
|
||||
* [Capacity and Allocatable](#capacity)
|
||||
* [Info](#info)
|
||||
|
||||
Each section is described in detail below.
|
||||
@@ -41,9 +41,9 @@ The usage of these fields varies depending on your cloud provider or bare metal
|
||||
* InternalIP: Typically the IP address of the node that is routable only within the cluster.
|
||||
|
||||
|
||||
### Condition
|
||||
### Conditions {#condition}
|
||||
|
||||
The `conditions` field describes the status of all `Running` nodes.
|
||||
The `conditions` field describes the status of all `Running` nodes. Examples of conditions include:
|
||||
|
||||
| Node Condition | Description |
|
||||
|----------------|-------------|
|
||||
@@ -60,7 +60,11 @@ The node condition is represented as a JSON object. For example, the following r
|
||||
"conditions": [
|
||||
{
|
||||
"type": "Ready",
|
||||
"status": "True"
|
||||
"status": "True",
|
||||
"reason": "KubeletReady",
|
||||
"message": "kubelet is posting ready status",
|
||||
"lastHeartbeatTime": "2019-06-05T18:38:35Z",
|
||||
"lastTransitionTime": "2019-06-05T11:41:27Z"
|
||||
}
|
||||
]
|
||||
```
|
||||
@@ -88,11 +92,19 @@ Enabling this feature creates a small delay between the
|
||||
time when a condition is observed and when a taint is created. This delay is usually less than one second, but it can increase the number of Pods that are successfully scheduled but rejected by the kubelet.
|
||||
{{< /caution >}}
|
||||
|
||||
### Capacity
|
||||
### Capacity and Allocatable {#capacity}
|
||||
|
||||
Describes the resources available on the node: CPU, memory and the maximum
|
||||
number of pods that can be scheduled onto the node.
|
||||
|
||||
The fields in the capacity block indicate the total amount of resources that a
|
||||
Node has. The allocatable block indicates the amount of resources that on a
|
||||
Node that are available to be consumed by normal Pods.
|
||||
|
||||
You may read more about capacity and allocatable resources while learning how
|
||||
to [reserve compute resources](/docs/tasks/administer-cluster/reserve-compute-resources/#node-allocatable)
|
||||
on a Node.
|
||||
|
||||
### Info
|
||||
|
||||
General information about the node, such as kernel version, Kubernetes version
|
||||
|
||||
@@ -26,7 +26,7 @@ Add-ons in each section are sorted alphabetically - the ordering does not imply
|
||||
* [Cilium](https://github.com/cilium/cilium) is a L3 network and network policy plugin that can enforce HTTP/API/L7 policies transparently. Both routing and overlay/encapsulation mode are supported.
|
||||
* [CNI-Genie](https://github.com/Huawei-PaaS/CNI-Genie) enables Kubernetes to seamlessly connect to a choice of CNI plugins, such as Calico, Canal, Flannel, Romana, or Weave.
|
||||
* [Contiv](http://contiv.github.io) provides configurable networking (native L3 using BGP, overlay using vxlan, classic L2, and Cisco-SDN/ACI) for various use cases and a rich policy framework. Contiv project is fully [open sourced](http://github.com/contiv). The [installer](http://github.com/contiv/install) provides both kubeadm and non-kubeadm based installation options.
|
||||
* [Contrail](http://www.juniper.net/us/en/products-services/sdn/contrail/contrail-networking/), based on [Tungsten Fabric](https://tungsten.io), is a open source, multi-cloud network virtualization and policy management platform. Contrail and Tungsten Fabric are integrated with orchestration systems such as Kubernetes, OpenShift, OpenStack and Mesos, and provide isolation modes for virtual machines, containers/pods and bare metal workloads.
|
||||
* [Contrail](http://www.juniper.net/us/en/products-services/sdn/contrail/contrail-networking/), based on [Tungsten Fabric](https://tungsten.io), is an open source, multi-cloud network virtualization and policy management platform. Contrail and Tungsten Fabric are integrated with orchestration systems such as Kubernetes, OpenShift, OpenStack and Mesos, and provide isolation modes for virtual machines, containers/pods and bare metal workloads.
|
||||
* [Flannel](https://github.com/coreos/flannel/blob/master/Documentation/kubernetes.md) is an overlay network provider that can be used with Kubernetes.
|
||||
* [Knitter](https://github.com/ZTE/Knitter/) is a network solution supporting multiple networking in Kubernetes.
|
||||
* [Multus](https://github.com/Intel-Corp/multus-cni) is a Multi plugin for multiple network support in Kubernetes to support all CNI plugins (e.g. Calico, Cilium, Contiv, Flannel), in addition to SRIOV, DPDK, OVS-DPDK and VPP based workloads in Kubernetes.
|
||||
|
||||
@@ -49,12 +49,15 @@ The in-tree cloud providers typically need both `--cloud-provider` and `--cloud-
|
||||
for the [kube-apiserver](/docs/admin/kube-apiserver/), [kube-controller-manager](/docs/admin/kube-controller-manager/) and the
|
||||
[kubelet](/docs/admin/kubelet/). The contents of the file specified in `--cloud-config` for each provider is documented below as well.
|
||||
|
||||
For all external cloud providers, please follow the instructions on the individual repositories.
|
||||
For all external cloud providers, please follow the instructions on the individual repositories,
|
||||
which are listed under their headings below, or one may view [the list of all repositories](https://github.com/kubernetes?q=cloud-provider-&type=&language=)
|
||||
|
||||
## AWS
|
||||
This section describes all the possible configurations which can
|
||||
be used when running Kubernetes on Amazon Web Services.
|
||||
|
||||
If you wish to use the external cloud provider, its repository is [kubernetes/cloud-provider-aws](https://github.com/kubernetes/cloud-provider-aws#readme)
|
||||
|
||||
### Node Name
|
||||
|
||||
The AWS cloud provider uses the private DNS name of the AWS instance as the name of the Kubernetes Node object.
|
||||
@@ -101,10 +104,12 @@ Different settings can be applied to a load balancer service in AWS using _annot
|
||||
* `service.beta.kubernetes.io/aws-load-balancer-proxy-protocol`: Used on the service to enable the proxy protocol on an ELB. Right now we only accept the value `*` which means enabling the proxy protocol on all ELB backends. In the future we could adjust this to allow setting the proxy protocol only on certain backends.
|
||||
* `service.beta.kubernetes.io/aws-load-balancer-ssl-ports`: Used on the service to specify a comma-separated list of ports that will use SSL/HTTPS listeners. Defaults to `*` (all)
|
||||
|
||||
The information for the annotations for AWS is taken from the comments on [aws.go](https://github.com/kubernetes/kubernetes/blob/master/pkg/cloudprovider/providers/aws/aws.go)
|
||||
The information for the annotations for AWS is taken from the comments on [aws.go](https://github.com/kubernetes/cloud-provider-aws/blob/master/pkg/cloudprovider/providers/aws/aws.go)
|
||||
|
||||
## Azure
|
||||
|
||||
If you wish to use the external cloud provider, its repository is [kubernetes/cloud-provider-azure](https://github.com/kubernetes/cloud-provider-azure#readme)
|
||||
|
||||
### Node Name
|
||||
|
||||
The Azure cloud provider uses the hostname of the node (as determined by the kubelet or overridden with `--hostname-override`) as the name of the Kubernetes Node object.
|
||||
@@ -112,6 +117,8 @@ Note that the Kubernetes Node name must match the Azure VM name.
|
||||
|
||||
## CloudStack
|
||||
|
||||
If you wish to use the external cloud provider, its repository is [kubernetes/cloud-provider-openstack](https://github.com/kubernetes/cloud-provider-openstack#readme)
|
||||
|
||||
### Node Name
|
||||
|
||||
The CloudStack cloud provider uses the hostname of the node (as determined by the kubelet or overridden with `--hostname-override`) as the name of the Kubernetes Node object.
|
||||
@@ -119,6 +126,8 @@ Note that the Kubernetes Node name must match the CloudStack VM name.
|
||||
|
||||
## GCE
|
||||
|
||||
If you wish to use the external cloud provider, its repository is [kubernetes/cloud-provider-gcp](https://github.com/kubernetes/cloud-provider-gcp#readme)
|
||||
|
||||
### Node Name
|
||||
|
||||
The GCE cloud provider uses the hostname of the node (as determined by the kubelet or overridden with `--hostname-override`) as the name of the Kubernetes Node object.
|
||||
@@ -240,7 +249,7 @@ file:
|
||||
respective network to get its subnets.
|
||||
* `floating-network-id` (Optional): If specified, will create a floating IP for
|
||||
the load balancer.
|
||||
* `lb-method` (Optional): Used to specify algorithm by which load will be
|
||||
* `lb-method` (Optional): Used to specify an algorithm by which load will be
|
||||
distributed amongst members of the load balancer pool. The value can be
|
||||
`ROUND_ROBIN`, `LEAST_CONNECTIONS`, or `SOURCE_IP`. The default behavior if
|
||||
none is specified is `ROUND_ROBIN`.
|
||||
@@ -285,6 +294,8 @@ and should appear in the `[BlockStorage]` section of the `cloud.conf` file:
|
||||
there are many Nova availability zones but only one Cinder availability zone.
|
||||
The default value is `false` to preserve the behavior used in earlier
|
||||
releases, but may change in the future.
|
||||
* `node-volume-attach-limit` (Optional): Maximum number of Volumes that can be
|
||||
attached to the node, default is 256 for cinder.
|
||||
|
||||
If deploying Kubernetes versions <= 1.8 on an OpenStack deployment that uses
|
||||
paths rather than ports to differentiate between endpoints it may be necessary
|
||||
|
||||
@@ -15,7 +15,7 @@ It assumes some familiarity with core Kubernetes [concepts](/docs/concepts/).
|
||||
{{% capture body %}}
|
||||
## Planning a cluster
|
||||
|
||||
See the guides in [Picking the Right Solution](/docs/setup/pick-right-solution/) for examples of how to plan, set up, and configure Kubernetes clusters. The solutions listed in this article are called *distros*.
|
||||
See the guides in [Setup](/docs/setup/) for examples of how to plan, set up, and configure Kubernetes clusters. The solutions listed in this article are called *distros*.
|
||||
|
||||
Before choosing a guide, here are some considerations:
|
||||
|
||||
|
||||
@@ -170,7 +170,7 @@ users in the event of a cluster failure), then you need to have `R * (U + 1)` cl
|
||||
|
||||
Finally, if any of your clusters would need more than the maximum recommended number of nodes for a Kubernetes cluster, then
|
||||
you may need even more clusters. Kubernetes v1.3 supports clusters up to 1000 nodes in size. Kubernetes v1.8 supports
|
||||
clusters up to 5000 nodes. See [Building Large Clusters](/docs/setup/cluster-large/) for more guidance.
|
||||
clusters up to 5000 nodes. See [Building Large Clusters](/docs/setup/best-practices/cluster-large/) for more guidance.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
|
||||
@@ -209,7 +209,7 @@ an application that writes to a single file, it's generally better to set
|
||||
container approach.
|
||||
|
||||
Sidecar containers can also be used to rotate log files that cannot be
|
||||
rotated by the application itself. [An example](https://github.com/samsung-cnct/logrotate)
|
||||
rotated by the application itself. An example
|
||||
of this approach is a small container running logrotate periodically.
|
||||
However, it's recommended to use `stdout` and `stderr` directly and leave rotation
|
||||
and retention policies to the kubelet.
|
||||
|
||||
@@ -307,7 +307,7 @@ kubectl annotate pods my-nginx-v4-9gw19 description='my frontend running nginx'
|
||||
kubectl get pods my-nginx-v4-9gw19 -o yaml
|
||||
```
|
||||
```shell
|
||||
apiversion: v1
|
||||
apiVersion: v1
|
||||
kind: pod
|
||||
metadata:
|
||||
annotations:
|
||||
@@ -366,7 +366,6 @@ This command will compare the version of the configuration that you're pushing w
|
||||
|
||||
```shell
|
||||
kubectl apply -f https://k8s.io/examples/application/nginx/nginx-deployment.yaml
|
||||
```shell
|
||||
deployment.apps/my-nginx configured
|
||||
```
|
||||
|
||||
|
||||
@@ -64,7 +64,7 @@ Kubernetes IP addresses exist at the `Pod` scope - containers within a `Pod`
|
||||
share their network namespaces - including their IP address. This means that
|
||||
containers within a `Pod` can all reach each other's ports on `localhost`. This
|
||||
also means that containers within a `Pod` must coordinate port usage, but this
|
||||
is no different than processes in a VM. This is called the "IP-per-pod" model.
|
||||
is no different from processes in a VM. This is called the "IP-per-pod" model.
|
||||
|
||||
How this is implemented is a detail of the particular container runtime in use.
|
||||
|
||||
@@ -102,7 +102,7 @@ Details on how the AOS system works can be accessed here: http://www.apstra.com/
|
||||
|
||||
[Big Cloud Fabric](https://www.bigswitch.com/container-network-automation) is a cloud native networking architecture, designed to run Kubernetes in private cloud/on-premises environments. Using unified physical & virtual SDN, Big Cloud Fabric tackles inherent container networking problems such as load balancing, visibility, troubleshooting, security policies & container traffic monitoring.
|
||||
|
||||
With the help of the Big Cloud Fabric's virtual pod multi-tenant architecture, container orchestration systems such as Kubernetes, RedHat OpenShift, Mesosphere DC/OS & Docker Swarm will be natively integrated along side with VM orchestration systems such as VMware, OpenStack & Nutanix. Customers will be able to securely inter-connect any number of these clusters and enable inter-tenant communication between them if needed.
|
||||
With the help of the Big Cloud Fabric's virtual pod multi-tenant architecture, container orchestration systems such as Kubernetes, RedHat OpenShift, Mesosphere DC/OS & Docker Swarm will be natively integrated alongside with VM orchestration systems such as VMware, OpenStack & Nutanix. Customers will be able to securely inter-connect any number of these clusters and enable inter-tenant communication between them if needed.
|
||||
|
||||
BCF was recognized by Gartner as a visionary in the latest [Magic Quadrant](http://go.bigswitch.com/17GatedDocuments-MagicQuadrantforDataCenterNetworking_Reg.html). One of the BCF Kubernetes on-premises deployments (which includes Kubernetes, DC/OS & VMware running on multiple DCs across different geographic regions) is also referenced [here](https://portworx.com/architects-corner-kubernetes-satya-komala-nio/).
|
||||
|
||||
|
||||
@@ -256,6 +256,7 @@ be co-located in the same defined topology, eg., the same node.
|
||||
##### Always co-located in the same node
|
||||
|
||||
In a three node cluster, a web application has in-memory cache such as redis. We want the web-servers to be co-located with the cache as much as possible.
|
||||
|
||||
Here is the yaml snippet of a simple redis deployment with three replicas and selector label `app=store`. The deployment has `PodAntiAffinity` configured to ensure the scheduler does not co-locate replicas on a single node.
|
||||
|
||||
```yaml
|
||||
@@ -339,7 +340,10 @@ If we create the above two deployments, our three node cluster should look like
|
||||
As you can see, all the 3 replicas of the `web-server` are automatically co-located with the cache as expected.
|
||||
|
||||
```
|
||||
$ kubectl get pods -o wide
|
||||
kubectl get pods -o wide
|
||||
```
|
||||
The output is similar to this:
|
||||
```
|
||||
NAME READY STATUS RESTARTS AGE IP NODE
|
||||
redis-cache-1450370735-6dzlj 1/1 Running 0 8m 10.192.4.2 kube-node-3
|
||||
redis-cache-1450370735-j2j96 1/1 Running 0 8m 10.192.2.2 kube-node-1
|
||||
|
||||
@@ -3,7 +3,7 @@ title: Managing Compute Resources for Containers
|
||||
content_template: templates/concept
|
||||
weight: 20
|
||||
feature:
|
||||
title: Automatic binpacking
|
||||
title: Automatic bin packing
|
||||
description: >
|
||||
Automatically places containers based on their resource requirements and other constraints, while not sacrificing availability. Mix critical and best-effort workloads in order to drive up utilization and save even more resources.
|
||||
---
|
||||
|
||||
@@ -34,7 +34,7 @@ Kubernetes Version | Priority and Preemption State | Enabled by default
|
||||
1.9 | alpha | no
|
||||
1.10 | alpha | no
|
||||
1.11 | beta | yes
|
||||
1.14 | GA | yes
|
||||
1.14 | stable | yes
|
||||
|
||||
{{< warning >}}In a cluster where not all users are trusted, a
|
||||
malicious user could create pods at the highest possible priorities, causing
|
||||
@@ -143,7 +143,7 @@ cluster when they should use this PriorityClass.
|
||||
### Example PriorityClass
|
||||
|
||||
```yaml
|
||||
apiVersion: scheduling.k8s.io/v1beta1
|
||||
apiVersion: scheduling.k8s.io/v1
|
||||
kind: PriorityClass
|
||||
metadata:
|
||||
name: high-priority
|
||||
@@ -334,9 +334,8 @@ preempted. Here's an example:
|
||||
If Pod Q were removed from its Node, the Pod anti-affinity violation would be
|
||||
gone, and Pod P could possibly be scheduled on Node N.
|
||||
|
||||
We may consider adding cross Node preemption in future versions if we find an
|
||||
algorithm with reasonable performance. We cannot promise anything at this point,
|
||||
and cross Node preemption will not be considered a blocker for Beta or GA.
|
||||
We may consider adding cross Node preemption in future versions if there is
|
||||
enough demand and if we find an algorithm with reasonable performance.
|
||||
|
||||
## Debugging Pod Priority and Preemption
|
||||
|
||||
|
||||
@@ -43,7 +43,7 @@ smaller than 5.
|
||||
Below is an example configuration that sets `percentageOfNodesToScore` to 50%.
|
||||
|
||||
```yaml
|
||||
apiVersion: componentconfig/v1alpha1
|
||||
apiVersion: kubescheduler.config.k8s.io/v1alpha1
|
||||
kind: KubeSchedulerConfiguration
|
||||
algorithmSource:
|
||||
provider: DefaultProvider
|
||||
|
||||
@@ -75,11 +75,11 @@ kubectl create secret generic db-user-pass --from-file=./username.txt --from-fil
|
||||
```
|
||||
secret "db-user-pass" created
|
||||
```
|
||||
{{< note >}}
|
||||
Special characters such as `$`, `\*`, and `!` require escaping.
|
||||
If the password you are using has special characters, you need to escape them using the `\\` character. For example, if your actual password is `S!B\*d$zDsb`, you should execute the command this way:
|
||||
kubectl create secret generic dev-db-secret --from-literal=username=devuser --from-literal=password=S\\!B\\\*d\\$zDsb
|
||||
You do not need to escape special characters in passwords from files (`--from-file`).
|
||||
{{< note >}}
|
||||
Special characters such as `$`, `\*`, and `!` require escaping.
|
||||
If the password you are using has special characters, you need to escape them using the `\\` character. For example, if your actual password is `S!B\*d$zDsb`, you should execute the command this way:
|
||||
kubectl create secret generic dev-db-secret --from-literal=username=devuser --from-literal=password=S\\!B\\\\*d\\$zDsb
|
||||
You do not need to escape special characters in passwords from files (`--from-file`).
|
||||
{{< /note >}}
|
||||
|
||||
You can check that the secret was created like this:
|
||||
@@ -203,8 +203,6 @@ The output will be similar to:
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
data:
|
||||
config.yaml: YXBpVXJsOiAiaHR0cHM6Ly9teS5hcGkuY29tL2FwaS92MSIKdXNlcm5hbWU6IHt7dXNlcm5hbWV9fQpwYXNzd29yZDoge3twYXNzd29yZH19
|
||||
kind: Secret
|
||||
metadata:
|
||||
creationTimestamp: 2018-11-15T20:40:59Z
|
||||
@@ -214,6 +212,8 @@ metadata:
|
||||
selfLink: /api/v1/namespaces/default/secrets/mysecret
|
||||
uid: c280ad2e-e916-11e8-98f2-025000000001
|
||||
type: Opaque
|
||||
data:
|
||||
config.yaml: YXBpVXJsOiAiaHR0cHM6Ly9teS5hcGkuY29tL2FwaS92MSIKdXNlcm5hbWU6IHt7dXNlcm5hbWV9fQpwYXNzd29yZDoge3twYXNzd29yZH19
|
||||
```
|
||||
|
||||
If a field is specified in both data and stringData, the value from stringData
|
||||
@@ -235,8 +235,6 @@ Results in the following secret:
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
data:
|
||||
username: YWRtaW5pc3RyYXRvcg==
|
||||
kind: Secret
|
||||
metadata:
|
||||
creationTimestamp: 2018-11-15T20:46:46Z
|
||||
@@ -246,6 +244,8 @@ metadata:
|
||||
selfLink: /api/v1/namespaces/default/secrets/mysecret
|
||||
uid: 91460ecb-e917-11e8-98f2-025000000001
|
||||
type: Opaque
|
||||
data:
|
||||
username: YWRtaW5pc3RyYXRvcg==
|
||||
```
|
||||
|
||||
Where `YWRtaW5pc3RyYXRvcg==` decodes to `administrator`.
|
||||
@@ -261,9 +261,9 @@ the option `-w 0` to `base64` commands or the pipeline `base64 | tr -d '\n'` if
|
||||
`-w` option is not available.
|
||||
|
||||
#### Creating a Secret from Generator
|
||||
Kubectl supports [managing objects using Kustomize](/docs/concepts/overview/object-management-kubectl/kustomization/)
|
||||
Kubectl supports [managing objects using Kustomize](/docs/tasks/manage-kubernetes-objects/kustomization/)
|
||||
since 1.14. With this new feature,
|
||||
you can also create a Secret from generators and then apply it to create the object on
|
||||
you can also create a Secret from generators and then apply it to create the object on
|
||||
the Apiserver. The generators
|
||||
should be specified in a `kustomization.yaml` inside a directory.
|
||||
|
||||
@@ -306,7 +306,7 @@ username.txt: 5 bytes
|
||||
```
|
||||
|
||||
For example, to generate a Secret from literals `username=admin` and `password=secret`,
|
||||
you can specify the secret generator in `kusotmization.yaml` as
|
||||
you can specify the secret generator in `kustomization.yaml` as
|
||||
```shell
|
||||
# Create a kustomization.yaml file with SecretGenerator
|
||||
$ cat <<EOF >./kustomization.yaml
|
||||
@@ -336,9 +336,6 @@ kubectl get secret mysecret -o yaml
|
||||
```
|
||||
```
|
||||
apiVersion: v1
|
||||
data:
|
||||
username: YWRtaW4=
|
||||
password: MWYyZDFlMmU2N2Rm
|
||||
kind: Secret
|
||||
metadata:
|
||||
creationTimestamp: 2016-01-22T18:41:56Z
|
||||
@@ -348,6 +345,9 @@ metadata:
|
||||
selfLink: /api/v1/namespaces/default/secrets/mysecret
|
||||
uid: cfee02d6-c137-11e5-8d73-42010af00002
|
||||
type: Opaque
|
||||
data:
|
||||
username: YWRtaW4=
|
||||
password: MWYyZDFlMmU2N2Rm
|
||||
```
|
||||
|
||||
Decode the password field:
|
||||
@@ -359,7 +359,7 @@ echo 'MWYyZDFlMmU2N2Rm' | base64 --decode
|
||||
1f2d1e2e67df
|
||||
```
|
||||
|
||||
### Using Secrets
|
||||
## Using Secrets
|
||||
|
||||
Secrets can be mounted as data volumes or be exposed as
|
||||
{{< glossary_tooltip text="environment variables" term_id="container-env-variables" >}}
|
||||
@@ -368,7 +368,7 @@ system, without being directly exposed to the pod. For example, they can hold
|
||||
credentials that other parts of the system should use to interact with external
|
||||
systems on your behalf.
|
||||
|
||||
#### Using Secrets as Files from a Pod
|
||||
### Using Secrets as Files from a Pod
|
||||
|
||||
To consume a Secret in a volume in a Pod:
|
||||
|
||||
@@ -521,12 +521,15 @@ ls /etc/foo/
|
||||
username
|
||||
password
|
||||
```
|
||||
|
||||
```shell
|
||||
cat /etc/foo/username
|
||||
```
|
||||
```
|
||||
admin
|
||||
```
|
||||
|
||||
|
||||
```shell
|
||||
cat /etc/foo/password
|
||||
```
|
||||
@@ -557,7 +560,7 @@ A container using a Secret as a
|
||||
Secret updates.
|
||||
{{< /note >}}
|
||||
|
||||
#### Using Secrets as Environment Variables
|
||||
### Using Secrets as Environment Variables
|
||||
|
||||
To use a secret in an {{< glossary_tooltip text="environment variable" term_id="container-env-variables" >}}
|
||||
in a pod:
|
||||
@@ -610,7 +613,7 @@ echo $SECRET_PASSWORD
|
||||
1f2d1e2e67df
|
||||
```
|
||||
|
||||
#### Using imagePullSecrets
|
||||
### Using imagePullSecrets
|
||||
|
||||
An imagePullSecret is a way to pass a secret that contains a Docker (or other) image registry
|
||||
password to the Kubelet so it can pull a private image on behalf of your Pod.
|
||||
@@ -694,9 +697,11 @@ start until all the pod's volumes are mounted.
|
||||
### Use-Case: Pod with ssh keys
|
||||
|
||||
Create a kustomization.yaml with SecretGenerator containing some ssh keys:
|
||||
|
||||
```shell
|
||||
kubectl create secret generic ssh-key-secret --from-file=ssh-privatekey=/path/to/.ssh/id_rsa --from-file=ssh-publickey=/path/to/.ssh/id_rsa.pub
|
||||
```
|
||||
|
||||
```
|
||||
secret "ssh-key-secret" created
|
||||
```
|
||||
@@ -710,8 +715,8 @@ Now we can create a pod which references the secret with the ssh key and
|
||||
consumes it in a volume:
|
||||
|
||||
```yaml
|
||||
kind: Pod
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: secret-test-pod
|
||||
labels:
|
||||
@@ -746,6 +751,7 @@ credentials and another pod which consumes a secret with test environment
|
||||
credentials.
|
||||
|
||||
Make the kustomization.yaml with SecretGenerator
|
||||
|
||||
```shell
|
||||
kubectl create secret generic prod-db-secret --from-literal=username=produser --from-literal=password=Y4nys7f11
|
||||
```
|
||||
@@ -753,6 +759,23 @@ kubectl create secret generic prod-db-secret --from-literal=username=produser --
|
||||
secret "prod-db-secret" created
|
||||
```
|
||||
|
||||
```shell
|
||||
kubectl create secret generic test-db-secret --from-literal=username=testuser --from-literal=password=iluvtests
|
||||
```
|
||||
```
|
||||
secret "test-db-secret" created
|
||||
```
|
||||
{{< note >}}
|
||||
Special characters such as `$`, `\*`, and `!` require escaping.
|
||||
If the password you are using has special characters, you need to escape them using the `\\` character. For example, if your actual password is `S!B\*d$zDsb`, you should execute the command this way:
|
||||
|
||||
```shell
|
||||
kubectl create secret generic dev-db-secret --from-literal=username=devuser --from-literal=password=S\\!B\\\*d\\$zDsb
|
||||
```
|
||||
|
||||
You do not need to escape special characters in passwords from files (`--from-file`).
|
||||
{{< /note >}}
|
||||
|
||||
Now make the pods:
|
||||
|
||||
```shell
|
||||
@@ -828,8 +851,8 @@ one called, say, `prod-user` with the `prod-db-secret`, and one called, say,
|
||||
`test-user` with the `test-db-secret`. Then, the pod spec can be shortened to, for example:
|
||||
|
||||
```yaml
|
||||
kind: Pod
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: prod-db-client-pod
|
||||
labels:
|
||||
@@ -847,15 +870,15 @@ In order to make piece of data 'hidden' (i.e., in a file whose name begins with
|
||||
make that key begin with a dot. For example, when the following secret is mounted into a volume:
|
||||
|
||||
```yaml
|
||||
kind: Secret
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: dotfile-secret
|
||||
data:
|
||||
.secret-file: dmFsdWUtMg0KDQo=
|
||||
---
|
||||
kind: Pod
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: secret-dotfiles-pod
|
||||
spec:
|
||||
|
||||
@@ -63,6 +63,8 @@ Credentials can be provided in several ways:
|
||||
- Using AWS EC2 Container Registry (ECR)
|
||||
- use IAM roles and policies to control access to ECR repositories
|
||||
- automatically refreshes ECR login credentials
|
||||
- Using Oracle Cloud Infrastructure Registry (OCIR)
|
||||
- use IAM roles and policies to control access to OCIR repositories
|
||||
- Using Azure Container Registry (ACR)
|
||||
- Using IBM Cloud Container Registry
|
||||
- Configuring Nodes to Authenticate to a Private Registry
|
||||
@@ -279,19 +281,7 @@ Kubernetes supports specifying registry keys on a pod.
|
||||
Run the following command, substituting the appropriate uppercase values:
|
||||
|
||||
```shell
|
||||
cat <<EOF > ./kustomization.yaml
|
||||
secretGenerator:
|
||||
- name: myregistrykey
|
||||
type: docker-registry
|
||||
literals:
|
||||
- docker-server=DOCKER_REGISTRY_SERVER
|
||||
- docker-username=DOCKER_USER
|
||||
- docker-password=DOCKER_PASSWORD
|
||||
- docker-email=DOCKER_EMAIL
|
||||
EOF
|
||||
|
||||
kubectl apply -k .
|
||||
secret/myregistrykey-66h7d4d986 created
|
||||
kubectl create secret docker-registry <name> --docker-server=DOCKER_REGISTRY_SERVER --docker-username=DOCKER_USER --docker-password=DOCKER_PASSWORD --docker-email=DOCKER_EMAIL
|
||||
```
|
||||
|
||||
If you already have a Docker credentials file then, rather than using the above
|
||||
@@ -360,7 +350,7 @@ common use cases and suggested solutions.
|
||||
- Or, when on GCE/Google Kubernetes Engine, use the project's Google Container Registry.
|
||||
- It will work better with cluster autoscaling than manual node configuration.
|
||||
- Or, on a cluster where changing the node configuration is inconvenient, use `imagePullSecrets`.
|
||||
1. Cluster with a proprietary images, a few of which require stricter access control.
|
||||
1. Cluster with proprietary images, a few of which require stricter access control.
|
||||
- Ensure [AlwaysPullImages admission controller](/docs/reference/access-authn-authz/admission-controllers/#alwayspullimages) is active. Otherwise, all Pods potentially have access to all images.
|
||||
- Move sensitive data into a "Secret" resource, instead of packaging it in an image.
|
||||
1. A multi-tenant cluster where each tenant needs own private registry.
|
||||
|
||||
@@ -29,6 +29,18 @@ Beta](#upgrading-runtimeclass-from-alpha-to-beta).
|
||||
RuntimeClass is a feature for selecting the container runtime configuration. The container runtime
|
||||
configuration is used to run a Pod's containers.
|
||||
|
||||
## Motivation
|
||||
|
||||
You can set a different RuntimeClass between different Pods to provide a balance of
|
||||
performance versus security. For example, if part of your workload deserves a high
|
||||
level of information security assurance, you might choose to schedule those Pods so
|
||||
that they run in a container runtime that uses hardware virtualization. You'd then
|
||||
benefit from the extra isolation of the alternative runtime, at the expense of some
|
||||
additional overhead.
|
||||
|
||||
You can also use RuntimeClass to run different Pods with the same container runtime
|
||||
but with different settings.
|
||||
|
||||
### Set Up
|
||||
|
||||
Ensure the RuntimeClass feature gate is enabled (it is by default). See [Feature
|
||||
@@ -41,7 +53,7 @@ feature gates. The `RuntimeClass` feature gate must be enabled on apiservers _an
|
||||
#### 1. Configure the CRI implementation on nodes
|
||||
|
||||
The configurations available through RuntimeClass are Container Runtime Interface (CRI)
|
||||
implementation dependent. See the corresponding documentation ([below](#cri-documentation)) for your
|
||||
implementation dependent. See the corresponding documentation ([below](#cri-configuration)) for your
|
||||
CRI implementation for how to configure.
|
||||
|
||||
{{< note >}}
|
||||
@@ -103,7 +115,7 @@ to the behavior when the RuntimeClass feature is disabled.
|
||||
|
||||
### CRI Configuration
|
||||
|
||||
For more details on setting up CRI runtimes, see [CRI installation](/docs/setup/cri/).
|
||||
For more details on setting up CRI runtimes, see [CRI installation](/docs/setup/production-environment/container-runtimes/).
|
||||
|
||||
#### dockershim
|
||||
|
||||
|
||||
@@ -125,8 +125,8 @@ must be mounted as a
|
||||
in the plugin's
|
||||
[PodSpec](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#podspec-v1-core).
|
||||
|
||||
Kubernetes device plugin support is still in alpha. As development continues, its API version can
|
||||
change in incompatible ways. We recommend that device plugin developers do the following:
|
||||
Kubernetes device plugin support is in beta. As development continues, its API version can
|
||||
change. We recommend that device plugin developers do the following:
|
||||
|
||||
* Watch for changes in future releases.
|
||||
* Support multiple versions of the device plugin API for backward/forward compatibility.
|
||||
@@ -148,16 +148,16 @@ The kubelet provides a gRPC service to enable discovery of in-use devices, and t
|
||||
for these devices:
|
||||
|
||||
```gRPC
|
||||
// PodResources is a service provided by the kubelet that provides information about the
|
||||
// PodResourcesLister is a service provided by the kubelet that provides information about the
|
||||
// node resources consumed by pods and containers on the node
|
||||
service PodResources {
|
||||
service PodResourcesLister {
|
||||
rpc List(ListPodResourcesRequest) returns (ListPodResourcesResponse) {}
|
||||
}
|
||||
```
|
||||
|
||||
The gRPC service is served over a unix socket at `/var/lib/kubelet/pod-resources/kubelet.sock`.
|
||||
Monitoring agents for device plugin resources can be deployed as a daemon, or as a DaemonSet.
|
||||
The cannonical directory `/var/lib/kubelet/pod-resources` requires privileged access, so monitoring
|
||||
The canonical directory `/var/lib/kubelet/pod-resources` requires privileged access, so monitoring
|
||||
agents must run in a privileged security context. If a device monitoring agent is running as a
|
||||
DaemonSet, `/var/lib/kubelet/pod-resources` must be mounted as a
|
||||
[Volume](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#volume-v1-core)
|
||||
@@ -179,6 +179,7 @@ For examples of device plugin implementations, see:
|
||||
* The [AMD GPU device plugin](https://github.com/RadeonOpenCompute/k8s-device-plugin)
|
||||
* The [SRIOV Network device plugin](https://github.com/intel/sriov-network-device-plugin)
|
||||
* The [Intel device plugins](https://github.com/intel/intel-device-plugins-for-kubernetes) for GPU, FPGA and QuickAssist devices
|
||||
* The [Xilinx FPGA device plugins](https://github.com/Xilinx/FPGA_as_a_Service/tree/master/k8s-fpga-device-plugin/trunk) for Xilinx FPGA devices
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
|
||||
@@ -156,7 +156,6 @@ After a request is authorized, if it is a write operation, it also goes through
|
||||
|
||||
* The [Image Policy webhook](/docs/reference/access-authn-authz/admission-controllers/#imagepolicywebhook) restricts what images can be run in containers.
|
||||
* To make arbitrary admission control decisions, a general [Admission webhook](/docs/reference/access-authn-authz/extensible-admission-controllers/#admission-webhooks) can be used. Admission Webhooks can reject creations or updates.
|
||||
* [Initializers](/docs/reference/access-authn-authz/extensible-admission-controllers/#initializers) are controllers that can modify objects before they are created. Initializers can modify initial object creations but cannot affect updates to objects. Initializers can also reject objects.
|
||||
|
||||
## Infrastructure Extensions
|
||||
|
||||
@@ -210,7 +209,3 @@ the nodes chosen for a pod.
|
||||
* [List of Operators](https://github.com/operator-framework/awesome-operators)
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
---
|
||||
title: "Overview"
|
||||
weight: 20
|
||||
---
|
||||
|
||||
---
|
||||
@@ -18,7 +18,7 @@ deliver a functioning Kubernetes cluster.
|
||||
## Master Components
|
||||
|
||||
Master components provide the cluster's control plane. Master components make global decisions about the
|
||||
cluster (for example, scheduling), and detecting and responding to cluster events (starting up a new pod when a replication controller's 'replicas' field is unsatisfied).
|
||||
cluster (for example, scheduling), and they detect and respond to cluster events (for example, starting up a new pod when a replication controller's `replicas` field is unsatisfied).
|
||||
|
||||
Master components can be run on any machine in the cluster. However,
|
||||
for simplicity, set up scripts typically start all master components on
|
||||
@@ -55,7 +55,7 @@ These controllers include:
|
||||
|
||||
cloud-controller-manager runs cloud-provider-specific controller loops only. You must disable these controller loops in the kube-controller-manager. You can disable the controller loops by setting the `--cloud-provider` flag to `external` when starting the kube-controller-manager.
|
||||
|
||||
cloud-controller-manager allows cloud vendors code and the Kubernetes code to evolve independent of each other. In prior releases, the core Kubernetes code was dependent upon cloud-provider-specific code for functionality. In future releases, code specific to cloud vendors should be maintained by the cloud vendor themselves, and linked to cloud-controller-manager while running Kubernetes.
|
||||
cloud-controller-manager allows the cloud vendor's code and the Kubernetes code to evolve independently of each other. In prior releases, the core Kubernetes code was dependent upon cloud-provider-specific code for functionality. In future releases, code specific to cloud vendors should be maintained by the cloud vendor themselves, and linked to cloud-controller-manager while running Kubernetes.
|
||||
|
||||
The following controllers have cloud provider dependencies:
|
||||
|
||||
@@ -74,13 +74,11 @@ Node components run on every node, maintaining running pods and providing the Ku
|
||||
|
||||
### kube-proxy
|
||||
|
||||
[kube-proxy](/docs/admin/kube-proxy/) enables the Kubernetes service abstraction by maintaining
|
||||
network rules on the host and performing connection forwarding.
|
||||
{{< glossary_definition term_id="kube-proxy" length="all" >}}
|
||||
|
||||
### Container Runtime
|
||||
|
||||
The container runtime is the software that is responsible for running containers.
|
||||
Kubernetes supports several runtimes: [Docker](http://www.docker.com), [containerd](https://containerd.io), [cri-o](https://cri-o.io/), [rktlet](https://github.com/kubernetes-incubator/rktlet) and any implementation of the [Kubernetes CRI (Container Runtime Interface)](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-node/container-runtime-interface.md).
|
||||
{{< glossary_definition term_id="container-runtime" length="all" >}}
|
||||
|
||||
## Addons
|
||||
|
||||
|
||||
@@ -4,14 +4,14 @@ reviewers:
|
||||
title: The Kubernetes API
|
||||
content_template: templates/concept
|
||||
weight: 30
|
||||
card:
|
||||
card:
|
||||
name: concepts
|
||||
weight: 30
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
Overall API conventions are described in the [API conventions doc](https://git.k8s.io/community/contributors/devel/api-conventions.md).
|
||||
Overall API conventions are described in the [API conventions doc](https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md).
|
||||
|
||||
API endpoints, resource types and samples are described in [API Reference](/docs/reference).
|
||||
|
||||
@@ -32,7 +32,7 @@ Kubernetes itself is decomposed into multiple components, which interact through
|
||||
|
||||
In our experience, any system that is successful needs to grow and change as new use cases emerge or existing ones change. Therefore, we expect the Kubernetes API to continuously change and grow. However, we intend to not break compatibility with existing clients, for an extended period of time. In general, new API resources and new resource fields can be expected to be added frequently. Elimination of resources or fields will require following the [API deprecation policy](/docs/reference/using-api/deprecation-policy/).
|
||||
|
||||
What constitutes a compatible change and how to change the API are detailed by the [API change document](https://git.k8s.io/community/contributors/devel/api_changes.md).
|
||||
What constitutes a compatible change and how to change the API are detailed by the [API change document](https://git.k8s.io/community/contributors/devel/sig-architecture/api_changes.md).
|
||||
|
||||
## OpenAPI and Swagger definitions
|
||||
|
||||
@@ -47,7 +47,7 @@ Accept | `application/json`, `application/com.github.proto-openapi.spec.v2@v1.0+
|
||||
Accept-Encoding | `gzip` (not passing this header is acceptable)
|
||||
|
||||
Prior to 1.14, format-separated endpoints (`/swagger.json`, `/swagger-2.0.0.json`, `/swagger-2.0.0.pb-v1`, `/swagger-2.0.0.pb-v1.gz`)
|
||||
serve the OpenAPI spec in different formats. These endpoints are deprecated, and will be removed in Kubernetes 1.14.
|
||||
serve the OpenAPI spec in different formats. These endpoints are deprecated, and are removed in Kubernetes 1.14.
|
||||
|
||||
**Examples of getting OpenAPI spec**:
|
||||
|
||||
@@ -59,7 +59,7 @@ GET /swagger-2.0.0.pb-v1.gz | GET /openapi/v2 **Accept**: application/com.github
|
||||
|
||||
Kubernetes implements an alternative Protobuf based serialization format for the API that is primarily intended for intra-cluster communication, documented in the [design proposal](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/api-machinery/protobuf.md) and the IDL files for each schema are located in the Go packages that define the API objects.
|
||||
|
||||
Prior to 1.14, the Kubernetes apiserver also exposes an API that can be used to retrieve
|
||||
Prior to 1.14, the Kubernetes apiserver also exposes an API that can be used to retrieve
|
||||
the [Swagger v1.2](http://swagger.io/) Kubernetes API spec at `/swaggerapi`.
|
||||
This endpoint is deprecated, and will be removed in Kubernetes 1.14.
|
||||
|
||||
@@ -69,7 +69,7 @@ To make it easier to eliminate fields or restructure resource representations, K
|
||||
multiple API versions, each at a different API path, such as `/api/v1` or
|
||||
`/apis/extensions/v1beta1`.
|
||||
|
||||
We chose to version at the API level rather than at the resource or field level to ensure that the API presents a clear, consistent view of system resources and behavior, and to enable controlling access to end-of-lifed and/or experimental APIs. The JSON and Protobuf serialization schemas follow the same guidelines for schema changes - all descriptions below cover both formats.
|
||||
We chose to version at the API level rather than at the resource or field level to ensure that the API presents a clear, consistent view of system resources and behavior, and to enable controlling access to end-of-life and/or experimental APIs. The JSON and Protobuf serialization schemas follow the same guidelines for schema changes - all descriptions below cover both formats.
|
||||
|
||||
Note that API versioning and Software versioning are only indirectly related. The [API and release
|
||||
versioning proposal](https://git.k8s.io/community/contributors/design-proposals/release/versioning.md) describes the relationship between API versioning and
|
||||
@@ -77,7 +77,7 @@ software versioning.
|
||||
|
||||
|
||||
Different API versions imply different levels of stability and support. The criteria for each level are described
|
||||
in more detail in the [API Changes documentation](https://git.k8s.io/community/contributors/devel/api_changes.md#alpha-beta-and-stable-versions). They are summarized here:
|
||||
in more detail in the [API Changes documentation](https://git.k8s.io/community/contributors/devel/sig-architecture/api_changes.md#alpha-beta-and-stable-versions). They are summarized here:
|
||||
|
||||
- Alpha level:
|
||||
- The version names contain `alpha` (e.g. `v1alpha1`).
|
||||
|
||||
@@ -1,5 +0,0 @@
|
||||
---
|
||||
title: "Object Management Using kubectl"
|
||||
weight: 50
|
||||
---
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
reviewers:
|
||||
- bgrant0607
|
||||
- mikedanese
|
||||
title: What is Kubernetes?
|
||||
title: What is Kubernetes
|
||||
content_template: templates/concept
|
||||
weight: 10
|
||||
card:
|
||||
@@ -26,7 +26,7 @@ production workloads at
|
||||
scale](https://research.google.com/pubs/pub43438.html), combined with
|
||||
best-of-breed ideas and practices from the community.
|
||||
|
||||
## Why do I need Kubernetes and what can it do?
|
||||
## Why do I need Kubernetes and what can it do
|
||||
|
||||
Kubernetes has a number of features. It can be thought of as:
|
||||
|
||||
@@ -42,7 +42,7 @@ Platform as a Service (PaaS) with the flexibility of Infrastructure as
|
||||
a Service (IaaS), and enables portability across infrastructure
|
||||
providers.
|
||||
|
||||
## How is Kubernetes a platform?
|
||||
## How Kubernetes is a platform
|
||||
|
||||
Even though Kubernetes provides a lot of functionality, there are
|
||||
always new scenarios that would benefit from new
|
||||
@@ -122,7 +122,7 @@ shouldn't matter how you get from A to C. Centralized control is also
|
||||
not required. This results in a system that is easier to use and more
|
||||
powerful, robust, resilient, and extensible.
|
||||
|
||||
## Why containers?
|
||||
## Why containers
|
||||
|
||||
Looking for reasons why you should be using containers?
|
||||
|
||||
@@ -192,7 +192,7 @@ Summary of container benefits:
|
||||
* **Resource utilization**:
|
||||
High efficiency and density.
|
||||
|
||||
## What does Kubernetes mean? K8s?
|
||||
## What Kubernetes and K8s mean
|
||||
|
||||
The name **Kubernetes** originates from Greek, meaning *helmsman* or
|
||||
*pilot*, and is the root of *governor* and
|
||||
|
||||
@@ -69,6 +69,25 @@ If the prefix is omitted, the annotation Key is presumed to be private to the us
|
||||
|
||||
The `kubernetes.io/` and `k8s.io/` prefixes are reserved for Kubernetes core components.
|
||||
|
||||
For example, here’s the configuration file for a Pod that has the annotation `imageregistry: https://hub.docker.com/` :
|
||||
|
||||
```yaml
|
||||
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: annotations-demo
|
||||
annotations:
|
||||
imageregistry: "https://hub.docker.com/"
|
||||
spec:
|
||||
containers:
|
||||
- name: nginx
|
||||
image: nginx:1.7.9
|
||||
ports:
|
||||
- containerPort: 80
|
||||
|
||||
```
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture whatsnext %}}
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user