CASE STUDY: Booking.com
+After Learning the Ropes with a Kubernetes Distribution, Booking.com Built a Platform of Its Own
+Read the latest news for Kubernetes and the containers space in general, and get technical how-tos hot off the presses.
--- +{{< comment >}} + +For information about contributing to the blog, see +https://kubernetes.io/docs/contribute/new-content/blogs-case-studies/#write-a-blog-post + +{{< /comment >}} diff --git a/content/en/blog/_posts/2018-04-13-local-persistent-volumes-beta.md b/content/en/blog/_posts/2018-04-13-local-persistent-volumes-beta.md index 42f9b60e66..71a0fa26d9 100644 --- a/content/en/blog/_posts/2018-04-13-local-persistent-volumes-beta.md +++ b/content/en/blog/_posts/2018-04-13-local-persistent-volumes-beta.md @@ -144,7 +144,7 @@ The local persistent volume beta feature is not complete by far. Some notable en [Pod disruption budget](/docs/concepts/workloads/pods/disruptions/) is also very important for those workloads that must maintain quorum. Setting a disruption budget for your workload ensures that it does not drop below quorum due to voluntary disruption events, such as node drains during upgrade. -[Pod affinity and anti-affinity](/docs/concepts/configuration/assign-pod-node/#inter-pod-affinity-and-anti-affinity-beta-feature) ensures that your workloads stay either co-located or spread out across failure domains. If you have multiple local persistent volumes available on a single node, it may be preferable to specify an pod anti-affinity policy to spread your workload across nodes. Note that if you want multiple pods to share the same local persistent volume, you do not need to specify a pod affinity policy. The scheduler understands the locality constraints of the local persistent volume and schedules your pod to the correct node. +[Pod affinity and anti-affinity](/docs/concepts/scheduling-eviction/assign-pod-node/#inter-pod-affinity-and-anti-affinity-beta-feature) ensures that your workloads stay either co-located or spread out across failure domains. If you have multiple local persistent volumes available on a single node, it may be preferable to specify an pod anti-affinity policy to spread your workload across nodes. Note that if you want multiple pods to share the same local persistent volume, you do not need to specify a pod affinity policy. The scheduler understands the locality constraints of the local persistent volume and schedules your pod to the correct node. ## Getting involved diff --git a/content/en/blog/_posts/2018-10-10-runtimeclass.md b/content/en/blog/_posts/2018-10-10-runtimeclass.md index 6a889ada70..5c2ba7208b 100644 --- a/content/en/blog/_posts/2018-10-10-runtimeclass.md +++ b/content/en/blog/_posts/2018-10-10-runtimeclass.md @@ -27,7 +27,7 @@ Why is RuntimeClass a pod level concept? The Kubernetes resource model expects c ## What's next? -The RuntimeClass resource is an important foundation for surfacing runtime properties to the control plane. For example, to implement scheduler support for clusters with heterogeneous nodes supporting different runtimes, we might add [NodeAffinity](/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity) terms to the RuntimeClass definition. Another area to address is managing the variable resource requirements to run pods of different runtimes. The [Pod Overhead proposal](https://docs.google.com/document/d/1EJKT4gyl58-kzt2bnwkv08MIUZ6lkDpXcxkHqCvvAp4/preview) was an early take on this that aligns nicely with the RuntimeClass design, and may be pursued further. +The RuntimeClass resource is an important foundation for surfacing runtime properties to the control plane. For example, to implement scheduler support for clusters with heterogeneous nodes supporting different runtimes, we might add [NodeAffinity](/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity) terms to the RuntimeClass definition. Another area to address is managing the variable resource requirements to run pods of different runtimes. The [Pod Overhead proposal](https://docs.google.com/document/d/1EJKT4gyl58-kzt2bnwkv08MIUZ6lkDpXcxkHqCvvAp4/preview) was an early take on this that aligns nicely with the RuntimeClass design, and may be pursued further. Many other RuntimeClass extensions have also been proposed, and will be revisited as the feature continues to develop and mature. A few more extensions that are being considered include: diff --git a/content/en/case-studies/babylon/babylon-featured.svg b/content/en/case-studies/babylon/babylon-featured.svg deleted file mode 100644 index fc2431a270..0000000000 --- a/content/en/case-studies/babylon/babylon-featured.svg +++ /dev/null @@ -1,172 +0,0 @@ - - - diff --git a/content/en/case-studies/babylon/babylon_featured.svg b/content/en/case-studies/babylon/babylon_featured_logo.svg similarity index 100% rename from content/en/case-studies/babylon/babylon_featured.svg rename to content/en/case-studies/babylon/babylon_featured_logo.svg diff --git a/content/en/case-studies/babylon/index.html b/content/en/case-studies/babylon/index.html index ab7874a2fb..afdc005411 100644 --- a/content/en/case-studies/babylon/index.html +++ b/content/en/case-studies/babylon/index.html @@ -4,7 +4,7 @@ linkTitle: Babylon case_study_styles: true cid: caseStudies css: /css/case-studies-gradient.css -logo: babylon_featured_logo.png +logo: babylon_featured_logo.svg featured: true weight: 1 quote: > diff --git a/content/en/case-studies/booking-com/booking.com_featured_logo.png b/content/en/case-studies/booking-com/booking.com_featured_logo.png new file mode 100644 index 0000000000..623ca67345 Binary files /dev/null and b/content/en/case-studies/booking-com/booking.com_featured_logo.png differ diff --git a/content/en/case-studies/booking-com/booking.com_featured_logo.svg b/content/en/case-studies/booking-com/booking.com_featured_logo.svg new file mode 100644 index 0000000000..298c77c773 --- /dev/null +++ b/content/en/case-studies/booking-com/booking.com_featured_logo.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/content/en/case-studies/booking-com/index.html b/content/en/case-studies/booking-com/index.html new file mode 100644 index 0000000000..ffeb3f2707 --- /dev/null +++ b/content/en/case-studies/booking-com/index.html @@ -0,0 +1,117 @@ +--- +title: Booking.com Case Study +linkTitle: Booking.com +case_study_styles: true +cid: caseStudies +css: /css/case-studies-gradient.css +logo: booking.com_featured_logo.png +featured: true +weight: 3 +quote: > + We realized that we needed to learn Kubernetes better in order to fully use the potential of it. At that point, we made the shift to build our own Kubernetes platform. +--- + + + + + + + +Impressed by what the technology offered, but in need of enterprise features at its scale—the site handles more than 1.5 million room-night reservations a day on average—the team decided to adopt an OpenShift platform.
+ +This platform, which was wrapped in a Heroku-style, high-level CLI interface, “was definitely popular with our product developers,” says Ben Tyler, Principal Developer, B Platform Track. “We gave them faster access to infrastructure.”
+ +But, he adds, “anytime something went slightly off the rails, developers didn’t have any of the knowledge required to support themselves.”
+ +And after a year of operating this platform, the infrastructure team found that it had become “a knowledge bottleneck,” he says. “Most of the developers who used it did not know it was Kubernetes underneath. An application failure and a platform failure both looked like failures of that Heroku-style tool.”
++ Scaling the necessary support did not seem feasible or sustainable, so the platform team needed a new solution. The understanding of Kubernetes that they had gained operating the OpenShift platform gave them confidence to build a vanilla Kubernetes platform of their own and customize it to suit the company’s needs.
+ + ++ “For entering the landscape, OpenShift was definitely very helpful,” says Eduard Iacoboaia, Senior System Administrator, B Platform Track. “It shows you what the technology can do, and it makes it easy for you to use it. After we spent some time on it, we realized that we needed to learn Kubernetes better in order to fully use the potential of it. At that point, we made the shift to build our own Kubernetes platform. We definitely benefit in the long term for taking that step and investing the time in gaining that knowledge.”
++ Iacoboaia’s team had customized a lot of OpenShift tools to make them work at Booking.com, and “those integrations points were kind of fragile,” he says. “We spent much more time understanding all the components of Kubernetes, how they work, how they interact with each other.” That research led the team to switch from OpenShift’s built-in Ansible playbooks to Puppet deployments, which are used for the rest of Booking’s infrastructure. The control plane was also moved from inside the cluster onto bare metal, as the company runs tens of thousands of bare-metal servers and a large infrastructure for running applications on bare metal. (Booking runs Kubernetes in multiple clusters in multiple data centers across the various regions where it has compute.) “We decided to keep it as simple as possible and to also use the tools that we know best,” says Iacoboaia.
++ The other big change was that product engineers would have to learn Kubernetes in order to onboard. “This is not a magical platform,” says Tyler. “We’re not claiming that you can just use it with your eyes closed. Developers need to do some learning, and we’re going to do everything we can to make sure they have access to that knowledge.” That includes trainings, blog posts, videos, and Udemy courses.
++ Despite the learning curve, there’s been a great uptick in adoption of the new Kubernetes platform. “I think the reason we’ve been able to strike this bargain successfully is that we’re not asking them to learn a proprietary app system,” says Tyler. “We’re asking them to learn something that’s open source, where the knowledge is transferable. They’re investing in their own careers by learning Kubernetes.”
++ One clear sign that this strategy has been a success is that in the support channel, when users have questions, other product engineers are jumping in to respond. “I haven’t seen that kind of community engagement around a particular platform product internally before,” says Tyler. “It helps a lot that it’s visibly an ecosystem standard outside of the company, so people feel value in investing in that knowledge and sharing it with others, which is really, really powerful.”
+ + ++ There’s other quantifiable evidence too: Before containers, creating a new service could take a couple of days if the developers understood Puppet, or weeks if they didn’t. On the new platform, it takes 10 minutes. “We have a tutorial. You follow the tutorial. Your code is running. Then, it’s business-logic time,” says Tyler. “The time to gain access to resources is decreased enormously.” About 500 new services were built in the first 8 months on the platform, with hundreds of releases per day.
++ The platform offers different “layers of contracts, so to speak,” says Tyler. “At the very base, it’s just Kubernetes. If you’re a pro Kubernetes user, here’s a Kubernetes API, just like you get from GKE or AKS. We’re trying to be a provider on that same level. But our whole job inside the company is to be a bigger value add than just vanilla infrastructure, so we provide a set of base images for our main stacks, Perl and Java.” +
++ And “as our users learn Kubernetes and become more sophisticated Kubernetes users, they put pressure on us to provide a better more native Kubernetes experience, which is great,” says Tyler. “It’s a super healthy dynamic.”
++ The platform also includes other CNCF technologies, such as Envoy, Helm, and Prometheus. Most of the critical service traffic for Booking.com is routed through Envoy, and Prometheus is used primarily to monitor infrastructure components. Helm is consumed as a packaging standard. The team also developed and open sourced Shipper, an extension for Kubernetes to add more complex rollout strategies and multi-cluster orchestration. +
++ To be sure, there have been internal discussions about the wisdom of building a Kubernetes platform from the ground up. “This is not really our core competency—Kubernetes and travel, they’re kind of far apart, right?” says Tyler. “But we’ve made a couple of bets on CNCF components that have worked out really well for us. Envoy and Kubernetes, in particular, have been really beneficial to our organization. We were able to customize them, either because we could look at the source code or because they had extension points, and we were able to get value out of them very quickly without having to change any paradigms internally.” +
+