Merge remote-tracking branch 'upstream/master' into dev-1.19
This commit is contained in:
Savitha Raghunathan
@@ -3,8 +3,6 @@ title: "Production-Grade Container Orchestration"
|
||||
abstract: "Automated container deployment, scaling, and management"
|
||||
cid: home
|
||||
---
|
||||
{{< announcement >}}
|
||||
|
||||
{{< deprecationwarning >}}
|
||||
|
||||
{{< blocks/section id="oceanNodes" >}}
|
||||
@@ -60,4 +58,4 @@ Kubernetes is open source giving you the freedom to take advantage of on-premise
|
||||
|
||||
{{< blocks/kubernetes-features >}}
|
||||
|
||||
{{< blocks/case-studies >}}
|
||||
{{< blocks/case-studies >}}
|
||||
@@ -60,9 +60,7 @@ using `kube-apiserver --feature-gates DryRun=true`.
|
||||
If you have dynamic admission controllers, you might have to fix them to:
|
||||
|
||||
- Remove any side-effects when the dry-run parameter is specified on the webhook request,
|
||||
- Specify in the [`sideEffects`](/docs/reference/generated/kubernetes-api/v1.13/#webhook-v1beta1-admissionregistration)
|
||||
field of the `admissionregistration.k8s.io/v1beta1.Webhook` object to indicate that the object doesn't
|
||||
have side-effects on dry-run (or at all).
|
||||
- Specify in the [`sideEffects`](https://v1-13.docs.kubernetes.io/docs/reference/generated/kubernetes-api/v1.13/#webhook-v1beta1-admissionregistration) field of the `admissionregistration.k8s.io/v1beta1.Webhook` object to indicate that the object doesn't have side-effects on dry-run (or at all).
|
||||
|
||||
### How to use it
|
||||
|
||||
|
||||
@@ -71,7 +71,7 @@ Introducing CSI plugin support for out-of-tree providers, enabling Windows nodes
|
||||
|
||||
## Introducing Endpoint Slices
|
||||
|
||||
The release of Kubernetes 1.16 includes an exciting new alpha feature: the EndpointSlice API. This API provides a scalable and extensible alternative to the [Endpoints](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.16/#endpoints-v1-core) resource, which dates back to the very first versions of Kubernetes. Behind the scenes, Endpoints play a big role in network routing within Kubernetes. Each Service endpoint is tracked within these resources - kube-proxy uses them for generating proxy rules that allow pods to communicate with each other so easily in Kubernetes, and many ingress controllers use them to route HTTP traffic directly to pods.
|
||||
The release of Kubernetes 1.16 includes an exciting new alpha feature: the EndpointSlice API. This API provides a scalable and extensible alternative to the [Endpoints](https://v1-16.docs.kubernetes.io/docs/reference/generated/kubernetes-api/v1.16/#endpoints-v1-core) resource, which dates back to the very first versions of Kubernetes. Behind the scenes, Endpoints play a big role in network routing within Kubernetes. Each Service endpoint is tracked within these resources - kube-proxy uses them for generating proxy rules that allow pods to communicate with each other so easily in Kubernetes, and many ingress controllers use them to route HTTP traffic directly to pods.
|
||||
|
||||
### Providing Greater Scalability
|
||||
|
||||
|
||||
@@ -43,14 +43,14 @@ Upon arrival at the handler, a request is assigned to exactly one priority level
|
||||
|
||||
* FlowSchema: FlowSchema will identify a PriorityLevelConfiguration object and the way to compute the request’s “flow identifier”. Currently we support matching requests according to: the identity making the request, the verb, and the target object. The identity can match in terms of: a username, a user group name, or a ServiceAccount. And as for the target objects, we can match by apiGroup, resource[/subresource], and namespace.
|
||||
* The flow identifier is used for shuffle sharding, so it’s important that requests have the same flow identifier if they are from the same source! We like to consider scenarios with “elephants” (which send many/heavy requests) vs “mice” (which send few/light requests): it is important to make sure the elephant’s requests all get the same flow identifier, otherwise they will look like many different mice to the system!
|
||||
* See the API Documentation [here](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.18/#flowschema-v1alpha1-flowcontrol-apiserver-k8s-io)!
|
||||
* See the API Documentation [here](https://kubernetes.io/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#flowschema-v1alpha1-flowcontrol-apiserver-k8s-io)!
|
||||
|
||||
* PriorityLevelConfiguration: Defines a priority level.
|
||||
* For apiserver self requests, and any reentrant traffic (e.g., admission webhooks which themselves make API requests), a Priority Level can be marked “exempt”, which means that no queueing or limiting of any sort is done. This is to prevent priority inversions.
|
||||
* Each non-exempt Priority Level is configured with a number of "concurrency shares" and gets an isolated pool of concurrency to use. Requests of that Priority Level run in that pool when it is not full, never anywhere else. Each apiserver is configured with a total concurrency limit (taken to be the sum of the old limits on mutating and readonly requests), and this is then divided among the Priority Levels in proportion to their concurrency shares.
|
||||
* A non-exempt Priority Level may select a number of queues and a "hand size" to use for the shuffle sharding. Shuffle sharding maps flows to queues in a way that is better than consistent hashing. A given flow has access to a small collection of queues, and for each incoming request the shortest queue is chosen. When a Priority Level has queues, it also sets a limit on queue length. There is also a limit placed on how long a request can wait in its queue; this is a fixed fraction of the apiserver's request timeout. A request that cannot be executed and cannot be queued (any longer) is rejected.
|
||||
* Alternatively, a non-exempt Priority Level may select immediate rejection instead of waiting in a queue.
|
||||
* See the [API documentation](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.18/#prioritylevelconfiguration-v1alpha1-flowcontrol-apiserver-k8s-io) for this feature.
|
||||
* See the [API documentation](https://kubernetes.io/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#prioritylevelconfiguration-v1alpha1-flowcontrol-apiserver-k8s-io) for this feature.
|
||||
|
||||
## What’s missing? When will there be a beta?
|
||||
We’re already planning a few enhancements based on alpha and there will be more as users send feedback to our community. Here’s a list of them:
|
||||
|
||||
+67
@@ -0,0 +1,67 @@
|
||||
---
|
||||
layout: blog
|
||||
title: "An Introduction to the K8s-Infrastructure Working Group"
|
||||
date: 2020-05-27
|
||||
slug: an-introduction-to-the-k8s-infrastructure-working-group
|
||||
---
|
||||
|
||||
**Author**: [Kiran "Rin" Oliver](https://twitter.com/kiran_oliver) Storyteller, Kubernetes Upstream Marketing Team
|
||||
|
||||
# An Introduction to the K8s-Infrastructure Working Group
|
||||
|
||||
*Welcome to part one of a new series introducing the K8s-Infrastructure working group!*
|
||||
|
||||
When Kubernetes was formed in 2014, Google undertook the task of building and maintaining the infrastructure necessary for keeping the project running smoothly. The tools itself were open source, but the Google Cloud Platform project used to run the infrastructure was internal-only, preventing contributors from being able to help out. In August 2018, Google granted the Cloud Native Computing Foundation [$9M in credits for the operation of Kubernetes](https://cloud.google.com/blog/products/gcp/google-cloud-grants-9m-in-credits-for-the-operation-of-the-kubernetes-project). The sentiment behind this was that a project such as Kubernetes should be both maintained and operated by the community itself rather than by a single vendor.
|
||||
|
||||
A group of community members enthusiastically undertook the task of collaborating on the path forward, realizing that there was a [more formal infrastructure necessary](https://github.com/kubernetes/community/issues/2715). They joined together as a cross-team working group with ownership spanning across multiple Kubernetes SIGs (Architecture, Contributor Experience, Release, and Testing). [Aaron Crickenberger](https://twitter.com/spiffxp) worked with the Kubernetes Steering Committee to enable the formation of the working group, co-drafting a charter alongside long-time collaborator [Davanum Srinivas](https://twitter.com/dims), and by 2019 the working group was official.
|
||||
|
||||
## What Issues Does the K8s-Infrastructure Working Group Tackle?
|
||||
|
||||
The team took on the complex task of managing the many moving parts of the infrastructure that sustains Kubernetes as a project.
|
||||
|
||||
The need started with necessity: the first problem they took on was a complete migration of all of the project's infrastructure from Google-owned infrastructure to the Cloud Native Computing Foundation (CNCF). This is being done so that the project is self-sustainable without the need of any direct assistance from individual vendors. This breaks down in the following ways:
|
||||
|
||||
* Identifying what infrastructure the Kubernetes project depends on.
|
||||
* What applications are running?
|
||||
* Where does it run?
|
||||
* Where is its source code?
|
||||
* What is custom built?
|
||||
* What is off-the-shelf?
|
||||
* What services depend on each other?
|
||||
* How is it administered?
|
||||
* Documenting guidelines and policies for how to run the infrastructure as a community.
|
||||
* What are our access policies?
|
||||
* How do we keep track of billing?
|
||||
* How do we ensure privacy and security?
|
||||
* Migrating infrastructure over to the CNCF as-is.
|
||||
* What is the path of least resistance to migration?
|
||||
* Improving the state of the infrastructure for sustainability.
|
||||
* Moving from humans running scripts to a more automated GitOps model (YAML all the things!)
|
||||
* Supporting community members who wish to develop new infrastructure
|
||||
* Documenting the state of our efforts, better defining goals, and completeness indicators.
|
||||
* The project and program management necessary to communicate this work to our [massive community of contributors](https://kubernetes.io/blog/2020/04/21/contributor-communication/)
|
||||
|
||||
## The challenge of K8s-Infrastructure is documentation
|
||||
|
||||
The most crucial problem the working group is trying to tackle is that the project is all volunteer-led. This leads to contributors, chairs, and others involved in the project quickly becoming overscheduled. As a result of this, certain areas such as documentation and organization often lack information, and efforts to progress are taking longer than the group would like to complete.
|
||||
|
||||
Some of the infrastructure that is being migrated over hasn't been updated in a while, and its original authors or directly responsible individuals have moved on from working on Kubernetes. While this is great from the perspective of the fact that the code was able to run untouched for a long period of time, from the perspective of trying to migrate, this makes it difficult to identify how to operate these components, and how to move these infrastructure pieces where they need to be effectively.
|
||||
|
||||
The lack of documentation is being addressed head-on by group member [Bart Smykla](https://twitter.com/bartsmykla), but there is a definite need for others to support. If you're looking for a way to [get involved](https://github.com/kubernetes/community/labels/wg%2Fk8s-infra) and learn the infrastructure, you can become a new contributor to the working group!
|
||||
|
||||
## Celebrating some Working Group wins
|
||||
|
||||
The team has made progress in the last few months that is well worth celebrating.
|
||||
|
||||
- The K8s-Infrastructure Working Group released an automated billing report that they start every meeting off by reviewing as a group.
|
||||
- DNS for k8s.io and kubernetes.io are also fully [community-owned](https://groups.google.com/g/kubernetes-dev/c/LZTYJorGh7c/m/u-ydk-yNEgAJ), with community members able to [file issues](https://github.com/kubernetes/k8s.io/issues/new?assignees=&labels=wg%2Fk8s-infra&template=dns-request.md&title=DNS+REQUEST%3A+%3Cyour-dns-record%3E) to manage records.
|
||||
- The container registry [k8s.gcr.io](https://github.com/kubernetes/k8s.io/tree/master/k8s.gcr.io) is also fully community-owned and available for all Kubernetes subprojects to use.
|
||||
- The Kubernetes [publishing-bot](https://github.com/kubernetes/publishing-bot) responsible for keeping k8s.io/kubernetes/staging repositories published to their own top-level repos (For example: [kubernetes/api](https://github.com/kubernetes/api)) runs on a community-owned cluster.
|
||||
- The gcsweb.k8s.io service used to provide anonymous access to GCS buckets for kubernetes artifacts runs on a community-owned cluster.
|
||||
- There is also an automated process of promoting all our container images. This includes a fully documented infrastructure, managed by the Kubernetes community, with automated processes for provisioning permissions.
|
||||
|
||||
These are just a few of the things currently happening in the K8s Infrastructure working group.
|
||||
|
||||
If you're interested in getting involved, be sure to join the [#wg-K8s-infra Slack Channel](https://app.slack.com/client/T09NY5SBT/CCK68P2Q2). Meetings are 60 minutes long, and are held every other Wednesday at 8:30 AM PT/16:30 UTC.
|
||||
|
||||
Join to help with the documentation, stay to learn about the amazing infrastructure supporting the Kubernetes community.
|
||||
@@ -0,0 +1,655 @@
|
||||
---
|
||||
title: "My exciting journey into Kubernetes’ history"
|
||||
date: 2020-05-28
|
||||
slug: kubernetes-history
|
||||
url: /blog/2020/05/my-exciting-journey-into-kubernetes-history
|
||||
---
|
||||
|
||||
**Author:** Sascha Grunert, SUSE Software Solutions
|
||||
|
||||
_Editor's note: Sascha is part of [SIG Release][0] and is working on many other
|
||||
different container runtime related topics. Feel free to reach him out on
|
||||
Twitter [@saschagrunert][1]._
|
||||
|
||||
[0]: https://github.com/kubernetes/sig-release
|
||||
[1]: https://twitter.com/saschagrunert
|
||||
|
||||
---
|
||||
|
||||
> A story of data science-ing 90,000 GitHub issues and pull requests by using
|
||||
> Kubeflow, TensorFlow, Prow and a fully automated CI/CD pipeline.
|
||||
|
||||
- [Introduction](#introduction)
|
||||
- [Getting the Data](#getting-the-data)
|
||||
- [Exploring the Data](#exploring-the-data)
|
||||
- [Labels, Labels, Labels](#labels-labels-labels)
|
||||
- [Building the Machine Learning Model](#building-the-machine-learning-model)
|
||||
- [Doing some first Natural Language Processing (NLP)](#doing-some-first-natural-language-processing-nlp)
|
||||
- [Creating the Multi-Layer Perceptron (MLP) Model](#creating-the-multi-layer-perceptron-mlp-model)
|
||||
- [Training the Model](#training-the-model)
|
||||
- [A first Prediction](#a-first-prediction)
|
||||
- [Automate Everything](#automate-everything)
|
||||
- [Automatic Labeling of new PRs](#automatic-labeling-of-new-prs)
|
||||
- [Summary](#summary)
|
||||
|
||||
# Introduction
|
||||
|
||||
Choosing the right steps when working in the field of data science is truly no
|
||||
silver bullet. Most data scientists might have their custom workflow, which
|
||||
could be more or less automated, depending on their area of work. Using
|
||||
[Kubernetes][10] can be a tremendous enhancement when trying to automate
|
||||
workflows on a large scale. In this blog post, I would like to take you on my
|
||||
journey of doing data science while integrating the overall workflow into
|
||||
Kubernetes.
|
||||
|
||||
The target of the research I did in the past few months was to find any
|
||||
useful information about all those thousands of GitHub issues and pull requests
|
||||
(PRs) we have in the [Kubernetes repository][11]. What I ended up with was a
|
||||
fully automated, in Kubernetes running Continuous Integration (CI) and
|
||||
Deployment (CD) data science workflow powered by [Kubeflow][12] and [Prow][13].
|
||||
You may not know both of them, but we get to the point where I explain what
|
||||
they’re doing in detail. The source code of my work can be found in the
|
||||
[kubernetes-analysis GitHub repository][14], which contains everything source
|
||||
code-related as well as the raw data. But how to retrieve this data I’m talking
|
||||
about? Well, this is where the story begins.
|
||||
|
||||
[10]: https://kubernetes.io
|
||||
[11]: https://github.com/kubernetes/kubernetes
|
||||
[12]: https://www.kubeflow.org
|
||||
[13]: https://github.com/kubernetes/test-infra/tree/master/prow
|
||||
[14]: https://github.com/kubernetes-analysis/kubernetes-analysis
|
||||
|
||||
# Getting the Data
|
||||
|
||||
The foundation for my experiments is the raw GitHub API data in plain [JSON][23]
|
||||
format. The necessary data can be retrieved via the [GitHub issues
|
||||
endpoint][20], which returns all pull requests as well as regular issues in the
|
||||
[REST][21] API. I exported roughly **91000** issues and pull requests in
|
||||
the first iteration into a massive **650 MiB** data blob. This took me about **8
|
||||
hours** of data retrieval time because for sure, the GitHub API is [rate
|
||||
limited][22]. To be able to put this data into a GitHub repository, I’d chosen
|
||||
to compress it via [`xz(1)`][24]. The result was a roundabout [25 MiB sized
|
||||
tarball][25], which fits well into the repository.
|
||||
|
||||
[20]: https://developer.github.com/v3/issues
|
||||
[21]: https://en.wikipedia.org/wiki/Representational_state_transfer
|
||||
[22]: https://developer.github.com/apps/building-github-apps/understanding-rate-limits-for-github-apps/
|
||||
[23]: https://en.wikipedia.org/wiki/JSON
|
||||
[24]: https://linux.die.net/man/1/xz
|
||||
[25]: https://github.com/kubernetes-analysis/kubernetes-analysis/blob/master/data/api.tar.xz
|
||||
|
||||
I had to find a way to regularly update the dataset because the Kubernetes
|
||||
issues and pull requests are updated by the users over time as well as new ones
|
||||
are created. To achieve the continuous update without having to wait 8 hours
|
||||
over and over again, I now fetch the delta GitHub API data between the
|
||||
[last update][31] and the current time. This way, a Continuous Integration job
|
||||
can update the data on a regular basis, whereas I can continue my research with
|
||||
the latest available set of data.
|
||||
|
||||
From a tooling perspective, I’ve written an [all-in-one Python executable][30],
|
||||
which allows us to trigger the different steps during the data science
|
||||
experiments separately via dedicated subcommands. For example, to run an export
|
||||
of the whole data set, we can call:
|
||||
|
||||
[30]: https://github.com/kubernetes-analysis/kubernetes-analysis/blob/master/main
|
||||
[31]: https://github.com/kubernetes-analysis/kubernetes-analysis/blob/master/.update
|
||||
|
||||
```
|
||||
> export GITHUB_TOKEN=<MY-SECRET-TOKEN>
|
||||
> ./main export
|
||||
INFO | Getting GITHUB_TOKEN from environment variable
|
||||
INFO | Dumping all issues
|
||||
INFO | Pulling 90929 items
|
||||
INFO | 1: Unit test coverage in Kubelet is lousy. (~30%)
|
||||
INFO | 2: Better error messages if go isn't installed, or if gcloud is old.
|
||||
INFO | 3: Need real cluster integration tests
|
||||
INFO | 4: kubelet should know which containers it is managing
|
||||
… [just wait 8 hours] …
|
||||
```
|
||||
|
||||
To update the data between the last time stamp stored in the repository we can
|
||||
run:
|
||||
|
||||
```
|
||||
> ./main export --update-api
|
||||
INFO | Getting GITHUB_TOKEN from environment variable
|
||||
INFO | Retrieving issues and PRs
|
||||
INFO | Updating API
|
||||
INFO | Got update timestamp: 2020-05-09T10:57:40.854151
|
||||
INFO | 90786: Automated cherry pick of #90749: fix: azure disk dangling attach issue
|
||||
INFO | 90674: Switch core master base images from debian to distroless
|
||||
INFO | 90086: Handling error returned by request.Request.ParseForm()
|
||||
INFO | 90544: configurable weight on the CPU and memory
|
||||
INFO | 87746: Support compiling Kubelet w/o docker/docker
|
||||
INFO | Using already extracted data from data/data.pickle
|
||||
INFO | Loading pickle dataset
|
||||
INFO | Parsed 34380 issues and 55832 pull requests (90212 items)
|
||||
INFO | Updating data
|
||||
INFO | Updating issue 90786 (updated at 2020-05-09T10:59:43Z)
|
||||
INFO | Updating issue 90674 (updated at 2020-05-09T10:58:27Z)
|
||||
INFO | Updating issue 90086 (updated at 2020-05-09T10:58:26Z)
|
||||
INFO | Updating issue 90544 (updated at 2020-05-09T10:57:51Z)
|
||||
INFO | Updating issue 87746 (updated at 2020-05-09T11:01:51Z)
|
||||
INFO | Saving data
|
||||
```
|
||||
|
||||
This gives us an idea of how fast the project is actually moving: On a Saturday
|
||||
at noon (European time), 5 issues and pull requests got updated within literally 5
|
||||
minutes!
|
||||
|
||||
Funnily enough, [Joe Beda][32], one of the founders of Kubernetes, created the
|
||||
first GitHub issue [mentioning that the unit test coverage is too low][33]. The
|
||||
issue has no further description than the title, and no enhanced labeling
|
||||
applied, like we know from more recent issues and pull requests. But now we have
|
||||
to explore the exported data more deeply to do something useful with it.
|
||||
|
||||
[32]: https://github.com/jbeda
|
||||
[33]: https://github.com/kubernetes/kubernetes/issues/1
|
||||
|
||||
# Exploring the Data
|
||||
|
||||
Before we can start creating machine learning models and train them, we have to
|
||||
get an idea about how our data is structured and what we want to achieve in
|
||||
general.
|
||||
|
||||
To get a better feeling about the amount of data, let’s look at how many issues
|
||||
and pull requests have been created over time inside the Kubernetes repository:
|
||||
|
||||
```
|
||||
> ./main analyze --created
|
||||
INFO | Using already extracted data from data/data.pickle
|
||||
INFO | Loading pickle dataset
|
||||
INFO | Parsed 34380 issues and 55832 pull requests (90212 items)
|
||||
```
|
||||
|
||||
The Python [matplotlib][40] module should pop up a graph which looks like this:
|
||||
|
||||
|
||||
|
||||
[40]: https://matplotlib.org
|
||||
|
||||
Okay, this looks not that spectacular but gives us an impression on how the
|
||||
project has grown over the past 6 years. To get a better idea about the speed of
|
||||
development of the project, we can look at the _created-vs-closed_ metric. This
|
||||
means on our timeline, we add one to the y-axis if an issue or pull request got
|
||||
created and subtracts one if closed. Now the chart looks like this:
|
||||
|
||||
```
|
||||
> ./main analyze --created-vs-closed
|
||||
```
|
||||
|
||||
|
||||
|
||||
At the beginning of 2018, the Kubernetes projects introduced some more enhanced
|
||||
life-cycle management via the glorious [fejta-bot][41]. This automatically
|
||||
closes issues and pull requests after they got stale over a longer period of
|
||||
time. This resulted in a massive closing of issues, which does not apply to pull
|
||||
requests in the same amount. For example, if we look at the _created-vs-closed_
|
||||
metric only for pull requests.
|
||||
|
||||
[41]: https://github.com/fejta-bot
|
||||
|
||||
```
|
||||
> ./main analyze --created-vs-closed --pull-requests
|
||||
```
|
||||
|
||||
|
||||
|
||||
The overall impact is not that obvious. What we can see is that the increasing
|
||||
number of peaks in the PR chart indicates that the project is moving faster over
|
||||
time. Usually, a candlestick chart would be a better choice for showing this kind
|
||||
of volatility-related information. I’d also like to highlight that it looks like
|
||||
the development of the project slowed down a bit in the beginning of 2020.
|
||||
|
||||
Parsing raw JSON in every analysis iteration is not the fastest approach to do
|
||||
in Python. This means that I decided to parse the more important information,
|
||||
for example the content, title and creation time into dedicated [issue][50] and
|
||||
[PR classes][51]. This data will be [pickle][58] serialized into the repository
|
||||
as well, which allows an overall faster startup independently of the JSON blob.
|
||||
|
||||
A pull request is more or less the same as an issue in my analysis, except that
|
||||
it contains a release note.
|
||||
|
||||
[50]: https://github.com/kubernetes-analysis/kubernetes-analysis/blob/master/src/issue.py
|
||||
[51]: https://github.com/kubernetes-analysis/kubernetes-analysis/blob/master/src/pull_request.py
|
||||
[58]: https://docs.python.org/3/library/pickle.html
|
||||
|
||||
Release notes in Kubernetes are written in the PRs description into a separate
|
||||
`release-note` block like this:
|
||||
|
||||
````
|
||||
```release-note
|
||||
I changed something extremely important and you should note that.
|
||||
```
|
||||
````
|
||||
|
||||
Those release notes are parsed by [dedicated Release Engineering Tools like
|
||||
`krel`][52] during the release creation process and will be part of the various
|
||||
[CHANGELOG.md][53] files and the [Release Notes Website][54]. That seems like a
|
||||
lot of magic, but in the end, the quality of the overall release notes is much
|
||||
higher because they’re easy to edit, and the PR reviewers can ensure that we
|
||||
only document real user-facing changes and nothing else.
|
||||
|
||||
[52]: https://github.com/kubernetes/release#tools
|
||||
[53]: https://github.com/kubernetes/kubernetes/tree/master/CHANGELOG
|
||||
[54]: https://relnotes.k8s.io
|
||||
|
||||
The quality of the input data is a key aspect when doing data science. I decided
|
||||
to focus on the release notes because they seem to have the highest amount of
|
||||
overall quality when comparing them to the plain descriptions in issues and PRs.
|
||||
Besides that, they’re easy to parse, and we would not need to strip away
|
||||
the [various issue][55] and [PR template][56] text noise.
|
||||
|
||||
[55]: https://github.com/kubernetes/kubernetes/tree/master/.github/ISSUE_TEMPLATE
|
||||
[56]: https://github.com/kubernetes/kubernetes/blob/master/.github/PULL_REQUEST_TEMPLATE.md
|
||||
|
||||
## Labels, Labels, Labels
|
||||
|
||||
Issues and pull requests in Kubernetes get different labels applied during its
|
||||
life-cycle. They are usually grouped via a single slash (`/`). For example, we
|
||||
have `kind/bug` and `kind/api-change` or `sig/node` and `sig/network`. An easy
|
||||
way to understand which label groups exist and how they’re distributed across
|
||||
the repository is to plot them into a bar chart:
|
||||
|
||||
```
|
||||
> ./main analyze --labels-by-group
|
||||
```
|
||||
|
||||
|
||||
|
||||
It looks like that `sig/`, `kind/` and `area/` labels are pretty common.
|
||||
Something like `size/` can be ignored for now because these labels are
|
||||
automatically applied based on the amount of the code changes for a pull
|
||||
request. We said that we want to focus on release notes as input data, which
|
||||
means that we have to check out the distribution of the labels for the PRs. This
|
||||
means that the top 25 labels on pull requests are:
|
||||
|
||||
```
|
||||
> ./main analyze --labels-by-name --pull-requests
|
||||
```
|
||||
|
||||
|
||||
|
||||
Again, we can ignore labels like `lgtm` (looks good to me), because every PR
|
||||
which now should get merged has to look good. Pull requests containing release
|
||||
notes automatically get the `release-note` label applied, which enables further
|
||||
filtering more easily. This does not mean that every PR containing that label
|
||||
also contains the release notes block. The label could have been applied
|
||||
manually and the parsing of the release notes block did not exist since the
|
||||
beginning of the project. This means we will probably loose a decent amount of
|
||||
input data on one hand. On the other hand we can focus on the highest possible
|
||||
data quality, because applying labels the right way needs some enhanced maturity
|
||||
of the project and its contributors.
|
||||
|
||||
From a label group perspective I have chosen to focus on the `kind/` labels.
|
||||
Those labels are something which has to be applied manually by the author of the
|
||||
PR, they are available on a good amount of pull requests and they’re related to
|
||||
user-facing changes as well. Besides that, the `kind/` choice has to be done for
|
||||
every pull request because it is part of the PR template.
|
||||
|
||||
Alright, how does the distribution of those labels look like when focusing only
|
||||
on pull requests which have release notes?
|
||||
|
||||
```
|
||||
> ./main analyze --release-notes-stats
|
||||
```
|
||||
|
||||
|
||||
|
||||
Interestingly, we have approximately 7,000 overall pull requests containing
|
||||
release notes, but only ~5,000 have a `kind/` label applied. The distribution of
|
||||
the labels is not equal, and one-third of them are labeled as `kind/bug`. This
|
||||
brings me to the next decision in my data science journey: I will build a binary
|
||||
classifier which, for the sake of simplicity, is only able to distinguish between
|
||||
bugs (via `kind/bug`) and non-bugs (where the label is not applied).
|
||||
|
||||
The main target is now to be able to classify newly incoming release notes if
|
||||
they are related to a bug or not, based on the historical data we already have
|
||||
from the community.
|
||||
|
||||
Before doing that, I recommend that you play around with the `./main analyze -h`
|
||||
subcommand as well to explore the latest set of data. You can also check out all
|
||||
the [continuously updated assets][57] I provide within the analysis repository.
|
||||
For example, those are the top 25 PR creators inside the Kubernetes repository:
|
||||
|
||||
|
||||
|
||||
[57]: https://github.com/kubernetes-analysis/kubernetes-analysis/tree/master/assets
|
||||
|
||||
# Building the Machine Learning Model
|
||||
|
||||
Now we have an idea what the data set is about, and we can start building a first
|
||||
machine learning model. Before actually building the model, we have to
|
||||
pre-process all the extracted release notes from the PRs. Otherwise, the model
|
||||
would not be able to understand our input.
|
||||
|
||||
## Doing some first Natural Language Processing (NLP)
|
||||
|
||||
In the beginning, we have to define a vocabulary for which we want to train. I
|
||||
decided to choose the [TfidfVectorizer][60] from the Python scikit-learn machine
|
||||
learning library. This vectorizer is able to take our input texts and create a
|
||||
single huge vocabulary out of it. This is our so-called [bag-of-words][61],
|
||||
which has a chosen n-gram range of `(1, 2)` (unigrams and bigrams). Practically
|
||||
this means that we always use the first word and the next one as a single
|
||||
vocabulary entry (bigrams). We also use the single word as vocabulary entry
|
||||
(unigram). The TfidfVectorizer is able to skip words that occur multiple times
|
||||
(`max_df`), and requires a minimum amount (`min_df`) to add a word to the
|
||||
vocabulary. I decided not to change those values in the first place, just
|
||||
because I had the intuition that release notes are something unique to a
|
||||
project.
|
||||
|
||||
Parameters like `min_df`, `max_df` and the n-gram range can be seen as some of
|
||||
our hyperparameters. Those parameters have to be optimized in a dedicated step
|
||||
after the machine learning model has been built. This step is called
|
||||
hyperparameter tuning and basically means that we train multiple times with
|
||||
different parameters and compare the accuracy of the model. Afterwards, we choose
|
||||
the parameters with the best accuracy.
|
||||
|
||||
[60]: https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfVectorizer.html
|
||||
[61]: https://en.wikipedia.org/wiki/Bag-of-words_model
|
||||
|
||||
During the training, the vectorizer will produce a `data/features.json` which
|
||||
contains the whole vocabulary. This gives us a good understanding of how such a
|
||||
vocabulary may look like:
|
||||
|
||||
```json
|
||||
[
|
||||
…
|
||||
"hostname",
|
||||
"hostname address",
|
||||
"hostname and",
|
||||
"hostname as",
|
||||
"hostname being",
|
||||
"hostname bug",
|
||||
…
|
||||
]
|
||||
```
|
||||
|
||||
This produces round about 50,000 entries in the overall bag-of-words, which is
|
||||
pretty much. Previous analyses between different data sets showed that it is
|
||||
simply not necessary to take so many features into account. Some general data
|
||||
sets state that an overall vocabulary of 20,000 is enough and higher amounts do
|
||||
not influence the accuracy any more. To do so we can use the [SelectKBest][62]
|
||||
feature selector to strip down the vocabulary to only choose the top features.
|
||||
Anyway, I still decided to stick to the top 50,000 to not negatively influence
|
||||
the model accuracy. We have a relatively low amount of data (appr. 7,000
|
||||
samples) and a low number of words per sample (~15) which already made me wonder
|
||||
if we have enough data at all.
|
||||
|
||||
[62]: https://scikit-learn.org/stable/modules/generated/sklearn.feature_selection.SelectKBest.html
|
||||
|
||||
The vectorizer is not only able to create our bag-of-words, but it is also able to
|
||||
encode the features in [term frequency–inverse document frequency (tf-idf)][63]
|
||||
format. That is where the vectorizer got its name, whereas the output of that
|
||||
encoding is something the machine learning model can directly consume. All the
|
||||
details of the vectorization process can be found in the [source code][64].
|
||||
|
||||
[63]: https://en.wikipedia.org/wiki/Tf%e2%80%93idf
|
||||
[64]: https://github.com/kubernetes-analysis/kubernetes-analysis/blob/f419ff4a3462bafc0cb067aa6973dc7280409699/src/nlp.py#L193-L235
|
||||
|
||||
## Creating the Multi-Layer Perceptron (MLP) Model
|
||||
|
||||
I decided to choose a simple MLP based model which is built with the help of the
|
||||
popular [TensorFlow][70] framework. Because we do not have that much input data,
|
||||
we just use two hidden layers, so that the model basically looks like this:
|
||||
|
||||
|
||||
|
||||
[70]: https://www.tensorflow.org/api_docs/python/tf/keras
|
||||
|
||||
There have to be [multiple other][71] hyperparameters to be taken into account
|
||||
when creating the model. I will not discuss them in detail here, but they’re
|
||||
important to be optimized also in relation to the number of classes we want to
|
||||
have in the model (only two in our case).
|
||||
|
||||
[71]: https://github.com/kubernetes-analysis/kubernetes-analysis/blob/f419ff4a3462bafc0cb067aa6973dc7280409699/src/nlp.py#L95-L100
|
||||
|
||||
## Training the Model
|
||||
|
||||
Before starting the actual training, we have to split up our input data into
|
||||
training and validation data sets. I’ve chosen to use ~80% of the data for
|
||||
training and 20% for validation purposes. We have to shuffle our input data as
|
||||
well to ensure that the model is not affected by ordering issues. The technical
|
||||
details of the training process can be found in the [GitHub sources][80]. So now
|
||||
we’re ready to finally start the training:
|
||||
|
||||
```
|
||||
> ./main train
|
||||
INFO | Using already extracted data from data/data.pickle
|
||||
INFO | Loading pickle dataset
|
||||
INFO | Parsed 34380 issues and 55832 pull requests (90212 items)
|
||||
INFO | Training for label 'kind/bug'
|
||||
INFO | 6980 items selected
|
||||
INFO | Using 5584 training and 1395 testing texts
|
||||
INFO | Number of classes: 2
|
||||
INFO | Vocabulary len: 51772
|
||||
INFO | Wrote features to file data/features.json
|
||||
INFO | Using units: 1
|
||||
INFO | Using activation function: sigmoid
|
||||
INFO | Created model with 2 layers and 64 units
|
||||
INFO | Compiling model
|
||||
INFO | Starting training
|
||||
Train on 5584 samples, validate on 1395 samples
|
||||
Epoch 1/1000
|
||||
5584/5584 - 3s - loss: 0.6895 - acc: 0.6789 - val_loss: 0.6856 - val_acc: 0.6860
|
||||
Epoch 2/1000
|
||||
5584/5584 - 2s - loss: 0.6822 - acc: 0.6827 - val_loss: 0.6782 - val_acc: 0.6860
|
||||
Epoch 3/1000
|
||||
…
|
||||
Epoch 68/1000
|
||||
5584/5584 - 2s - loss: 0.2587 - acc: 0.9257 - val_loss: 0.4847 - val_acc: 0.7728
|
||||
INFO | Confusion matrix:
|
||||
[[920 32]
|
||||
[291 152]]
|
||||
INFO | Confusion matrix normalized:
|
||||
[[0.966 0.034]
|
||||
[0.657 0.343]]
|
||||
INFO | Saving model to file data/model.h5
|
||||
INFO | Validation accuracy: 0.7727598547935486, loss: 0.48470408514836355
|
||||
```
|
||||
|
||||
The output of the [Confusion Matrix][81] shows us that we’re pretty good on
|
||||
training accuracy, but the validation accuracy could be a bit higher. We now
|
||||
could start a hyperparameter tuning to see if we can optimize the output of the
|
||||
model even further. I will leave that experiment up to you with the hint to the
|
||||
`./main train --tune` flag.
|
||||
|
||||
We saved the model (`data/model.h5`), the vectorizer (`data/vectorizer.pickle`)
|
||||
and the feature selector (`data/selector.pickle`) to disk to be able to use them
|
||||
later on for prediction purposes without having a need for additional training
|
||||
steps.
|
||||
|
||||
[80]: https://github.com/kubernetes-analysis/kubernetes-analysis/blob/f419ff4a3462bafc0cb067aa6973dc7280409699/src/nlp.py#L91-L170
|
||||
[81]: https://en.wikipedia.org/wiki/Confusion_matrix
|
||||
|
||||
## A first Prediction
|
||||
|
||||
We are now able to test the model by loading it from disk and predicting some
|
||||
input text:
|
||||
|
||||
```
|
||||
> ./main predict --test
|
||||
INFO | Testing positive text:
|
||||
|
||||
Fix concurrent map access panic
|
||||
Don't watch .mount cgroups to reduce number of inotify watches
|
||||
Fix NVML initialization race condition
|
||||
Fix brtfs disk metrics when using a subdirectory of a subvolume
|
||||
|
||||
INFO | Got prediction result: 0.9940581321716309
|
||||
INFO | Matched expected positive prediction result
|
||||
INFO | Testing negative text:
|
||||
|
||||
action required
|
||||
1. Currently, if users were to explicitly specify CacheSize of 0 for
|
||||
KMS provider, they would end-up with a provider that caches up to
|
||||
1000 keys. This PR changes this behavior.
|
||||
Post this PR, when users supply 0 for CacheSize this will result in
|
||||
a validation error.
|
||||
2. CacheSize type was changed from int32 to *int32. This allows
|
||||
defaulting logic to differentiate between cases where users
|
||||
explicitly supplied 0 vs. not supplied any value.
|
||||
3. KMS Provider's endpoint (path to Unix socket) is now validated when
|
||||
the EncryptionConfiguration files is loaded. This used to be handled
|
||||
by the GRPCService.
|
||||
|
||||
INFO | Got prediction result: 0.1251964420080185
|
||||
INFO | Matched expected negative prediction result
|
||||
```
|
||||
|
||||
Both tests are real-world examples which already exist. We could also try
|
||||
something completely different, like this random tweet I found a couple of
|
||||
minutes ago:
|
||||
|
||||
```
|
||||
./main predict "My dudes, if you can understand SYN-ACK, you can understand consent"
|
||||
INFO | Got prediction result: 0.1251964420080185
|
||||
ERROR | Result is lower than selected threshold 0.6
|
||||
```
|
||||
|
||||
Looks like it is not classified as bug for a release note, which seems to work.
|
||||
Selecting a good threshold is also not that easy, but sticking to something >
|
||||
50% should be the bare minimum.
|
||||
|
||||
# Automate Everything
|
||||
|
||||
The next step is to find some way of automation to continuously update the model
|
||||
with new data. If I change any source code within my repository, then I’d like
|
||||
to get feedback about the test results of the model without having a need to run
|
||||
the training on my own machine. I would like to utilize the GPUs in my
|
||||
Kubernetes cluster to train faster and automatically update the data set if a PR
|
||||
got merged.
|
||||
|
||||
With the help of [Kubeflow pipelines][90] we can fulfill most of these
|
||||
requirements. The pipeline I built looks like this:
|
||||
|
||||
[90]: https://www.kubeflow.org/docs/pipelines/overview/pipelines-overview
|
||||
|
||||
|
||||
|
||||
First, we check out the source code of the PR, which will be passed on as output
|
||||
artifact to all other steps. Then we incrementally update the API and internal
|
||||
data before we run the training on an always up-to-date data set. The prediction
|
||||
test verifies after the training that we did not badly influence the model with
|
||||
our changes.
|
||||
|
||||
We also built a container image within our pipeline. [This container image][91]
|
||||
copies the previously built model, vectorizer, and selector into a container and
|
||||
runs `./main serve`. When doing this, we spin up a [kfserving][92] web server,
|
||||
which can be used for prediction purposes. Do you want to try it out by yourself? Simply
|
||||
do a JSON POST request like this and run the prediction against the endpoint:
|
||||
|
||||
```
|
||||
> curl https://kfserving.k8s.saschagrunert.de/v1/models/kubernetes-analysis:predict \
|
||||
-d '{"text": "my test text"}'
|
||||
{"result": 0.1251964420080185}
|
||||
```
|
||||
|
||||
The [custom kfserving][93] implementation is pretty straightforward, whereas the
|
||||
deployment utilizes [Knative Serving][95] and an [Istio][94] ingress gateway
|
||||
under the hood to correctly route the traffic into the cluster and provide the
|
||||
right set of services.
|
||||
|
||||
[91]: https://github.com/kubernetes-analysis/kubernetes-analysis/blob/master/Dockerfile-deploy
|
||||
[92]: https://www.kubeflow.org/docs/components/serving/kfserving
|
||||
[93]: https://github.com/kubernetes-analysis/kubernetes-analysis/blob/master/src/kfserver.py
|
||||
[94]: https://istio.io
|
||||
[95]: https://knative.dev/docs/serving
|
||||
|
||||
The `commit-changes` and `rollout` step will only run if the pipeline runs on
|
||||
the `master` branch. Those steps make sure that we always have the latest data
|
||||
set available on the master branch as well as in the kfserving deployment. The
|
||||
[rollout step][96] creates a new canary deployment, which only accepts 50% of the
|
||||
incoming traffic in the first place. After the canary got deployed successfully,
|
||||
it will be promoted as the new main instance of the service. This is a great way
|
||||
to ensure that the deployment works as intended and allows additional testing
|
||||
after rolling out the canary.
|
||||
|
||||
[96]: https://github.com/kubernetes-analysis/kubernetes-analysis/blob/f419ff4a3462bafc0cb067aa6973dc7280409699/src/rollout.py#L30-L51
|
||||
|
||||
But how to trigger Kubeflow pipelines when creating a pull request? Kubeflow has
|
||||
no feature for that right now. That’s why I decided to use [Prow][100],
|
||||
Kubernetes test-infrastructure project for CI/CD purposes.
|
||||
|
||||
First of all, a [24h periodic job][101] ensures that we have at least daily
|
||||
up-to-date data available within the repository. Then, if we create a pull
|
||||
request, Prow will run the whole Kubeflow pipeline without committing or rolling
|
||||
out any changes. If we merge the pull request via Prow, another job runs on the
|
||||
master and updates the data as well as the deployment. That’s pretty neat, isn’t
|
||||
it?
|
||||
|
||||
[100]: https://github.com/kubernetes/test-infra/tree/master/prow
|
||||
[101]: https://github.com/kubernetes-analysis/kubernetes-analysis/blob/f419ff4a3462bafc0cb067aa6973dc7280409699/ci/config.yaml#L45-L61
|
||||
|
||||
# Automatic Labeling of new PRs
|
||||
|
||||
The prediction API is nice for testing, but now we need a real-world use case.
|
||||
Prow supports external plugins which can be used to take action on any GitHub
|
||||
event. I wrote [a plugin][110] which uses the kfserving API to make predictions
|
||||
based on new pull requests. This means if we now create a new pull request in
|
||||
the kubernetes-analysis repository, we will see the following:
|
||||
|
||||
[110]: https://github.com/kubernetes-analysis/kubernetes-analysis/tree/master/pkg
|
||||
|
||||
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
Okay cool, so now let’s change the release note based on a real bug from the
|
||||
already existing dataset:
|
||||
|
||||
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
The bot edits its own comment, predicts it with round about 90% as `kind/bug`
|
||||
and automatically adds the correct label! Now, if we change it back to some
|
||||
different - obviously wrong - release note:
|
||||
|
||||
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
The bot does the work for us, removes the label and informs us what it did!
|
||||
Finally, if we change the release note to `None`:
|
||||
|
||||
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
The bot removed the comment, which is nice and reduces the text noise on the PR.
|
||||
Everything I demonstrated is running inside a single Kubernetes cluster, which
|
||||
would make it unnecessary at all to expose the kfserving API to the public. This
|
||||
introduces an indirect API rate limiting because the only usage would be
|
||||
possible via the Prow bot user.
|
||||
|
||||
If you want to try it out for yourself, feel free to open a [new test
|
||||
issue][111] in `kubernetes-analysis`. This works because I enabled the plugin
|
||||
also for issues rather than only for pull requests.
|
||||
|
||||
[111]: https://github.com/kubernetes-analysis/kubernetes-analysis/issues/new?&template=release-notes-test.md
|
||||
|
||||
So then, we have a running CI bot which is able to classify new release notes
|
||||
based on a machine learning model. If the bot would run in the official
|
||||
Kubernetes repository, then we could correct wrong label predictions manually.
|
||||
This way, the next training iteration would pick up the correction and result in
|
||||
a continuously improved model over time. All totally automated!
|
||||
|
||||
# Summary
|
||||
|
||||
Thank you for reading down to here! This was my little data science journey
|
||||
through the Kubernetes GitHub repository. There are a lot of other things to
|
||||
optimize, for example introducing more classes (than just `kind/bug` or nothing)
|
||||
or automatic hyperparameter tuning with Kubeflows [Katib][120]. If you have any
|
||||
questions or suggestions, then feel free to get in touch with me anytime. See you
|
||||
soon!
|
||||
|
||||
[120]: https://www.kubeflow.org/docs/components/hyperparameter-tuning/hyperparameter
|
||||
@@ -0,0 +1,201 @@
|
||||
---
|
||||
layout: blog
|
||||
title: "K8s KPIs with Kuberhealthy"
|
||||
date: 2020-05-29
|
||||
---
|
||||
|
||||
**Authors:** Joshulyne Park (Comcast), Eric Greer (Comcast)
|
||||
|
||||
### Building Onward from Kuberhealthy v2.0.0
|
||||
|
||||
Last November at KubeCon San Diego 2019, we announced the release of
|
||||
[Kuberhealthy 2.0.0](https://www.youtube.com/watch?v=aAJlWhBtzqY) - transforming Kuberhealthy into a Kubernetes operator
|
||||
for synthetic monitoring. This new ability granted developers the means to create their own Kuberhealthy check
|
||||
containers to synthetically monitor their applications and clusters. The community was quick to adopt this new feature and we're grateful for everyone who implemented and tested Kuberhealthy 2.0.0 in their clusters. Thanks to all of you who reported
|
||||
issues and contributed to discussions on the #kuberhealthy Slack channel. We quickly set to work to address all your feedback
|
||||
with a newer version of Kuberhealthy. Additionally, we created a guide on how to easily install and use Kuberhealthy in order to capture some helpful synthetic [KPIs](https://kpi.org/KPI-Basics).
|
||||
|
||||
### Deploying Kuberhealthy
|
||||
|
||||
To install Kuberhealthy, make sure you have [Helm 3](https://helm.sh/docs/intro/install/) installed. If not, you can use the generated flat spec files located
|
||||
in this [deploy folder](https://github.com/Comcast/kuberhealthy/tree/master/deploy). You should use [kuberhealthy-prometheus.yaml](https://github.com/Comcast/kuberhealthy/blob/master/deploy/kuberhealthy-prometheus.yaml) if you don't use the [Prometheus Operator](https://github.com/coreos/prometheus-operator), and [kuberhealthy-prometheus-operator.yaml](https://github.com/Comcast/kuberhealthy/blob/master/deploy/kuberhealthy-prometheus-operator.yaml) if you do. If you don't use Prometheus at all, you can still use Kuberhealthy with a JSON status page and/or InfluxDB integration using [this spec](https://github.com/Comcast/kuberhealthy/blob/master/deploy/kuberhealthy.yaml).
|
||||
|
||||
#### To install using Helm 3:
|
||||
##### 1. Create namespace "kuberhealthy" in the desired Kubernetes cluster/context:
|
||||
```
|
||||
kubectl create namespace kuberhealthy
|
||||
```
|
||||
##### 2. Set your current namespace to "kuberhealthy":
|
||||
```
|
||||
kubectl config set-context --current --namespace=kuberhealthy
|
||||
```
|
||||
##### 3. Add the kuberhealthy repo to Helm:
|
||||
```
|
||||
helm repo add kuberhealthy https://comcast.github.io/kuberhealthy/helm-repos
|
||||
```
|
||||
##### 4. Depending on your Prometheus implementation, install Kuberhealthy using the appropriate command for your cluster:
|
||||
|
||||
- If you use the [Prometheus Operator](https://github.com/coreos/prometheus-operator):
|
||||
```
|
||||
helm install kuberhealthy kuberhealthy/kuberhealthy --set prometheus.enabled=true,prometheus.enableAlerting=true,prometheus.enableScraping=true,prometheus.serviceMonitor=true
|
||||
```
|
||||
|
||||
- If you use Prometheus, but NOT Prometheus Operator:
|
||||
```
|
||||
helm install kuberhealthy kuberhealthy/kuberhealthy --set prometheus.enabled=true,prometheus.enableAlerting=true,prometheus.enableScraping=true
|
||||
```
|
||||
See additional details about configuring the appropriate scrape annotations in the section [Prometheus Integration Details](#prometheus-integration-details) below.
|
||||
|
||||
- Finally, if you don't use Prometheus:
|
||||
```
|
||||
helm install kuberhealthy kuberhealthy/kuberhealthy
|
||||
```
|
||||
|
||||
Running the Helm command should automatically install the newest version of Kuberhealthy (v2.2.0) along with a few basic checks. If you run `kubectl get pods`, you should see two Kuberhealthy pods. These are the pods that create, coordinate, and track test pods. These two Kuberhealthy pods also serve a JSON status page as well as a `/metrics` endpoint. Every other pod you see created is a checker pod designed to execute and shut down when done.
|
||||
|
||||
### Configuring Additional Checks
|
||||
|
||||
Next, you can run `kubectl get khchecks`. You should see three Kuberhealthy checks installed by default:
|
||||
- [daemonset](https://github.com/Comcast/kuberhealthy/tree/master/cmd/daemonset-check): Deploys and tears down a daemonset to ensure all nodes in the cluster are functional.
|
||||
- [deployment](https://github.com/Comcast/kuberhealthy/tree/master/cmd/deployment-check): Creates a deployment and then triggers a rolling update. Tests that the deployment is reachable via a service and then deletes everything. Any problem in this process will cause this check to report a failure.
|
||||
- [dns-status-internal](https://github.com/Comcast/kuberhealthy/tree/master/cmd/dns-resolution-check): Validates that internal cluster DNS is functioning as expected.
|
||||
|
||||
To view other available external checks, check out the [external checks registry](https://github.com/Comcast/kuberhealthy/blob/master/docs/EXTERNAL_CHECKS_REGISTRY.md) where you can find other yaml files you can apply to your cluster to enable various checks.
|
||||
|
||||
Kuberhealthy check pods should start running shortly after Kuberhealthy starts running (1-2 minutes). Additionally, the check-reaper cronjob runs every few minutes to ensure there are no more than 5 completed checker pods left lying around at a time.
|
||||
|
||||
To get status page view of these checks, you'll need to either expose the `kuberhealthy` service externally by editing the service `kuberhealthy` and setting `Type: LoadBalancer` or use `kubectl port-forward service/kuberhealthy 8080:80`. When viewed, the service endpoint will display a JSON status page that looks like this:
|
||||
|
||||
```json
|
||||
{
|
||||
"OK": true,
|
||||
"Errors": [],
|
||||
"CheckDetails": {
|
||||
"kuberhealthy/daemonset": {
|
||||
"OK": true,
|
||||
"Errors": [],
|
||||
"RunDuration": "22.512278967s",
|
||||
"Namespace": "kuberhealthy",
|
||||
"LastRun": "2020-04-06T23:20:31.7176964Z",
|
||||
"AuthoritativePod": "kuberhealthy-67bf8c4686-mbl2j",
|
||||
"uuid": "9abd3ec0-b82f-44f0-b8a7-fa6709f759cd"
|
||||
},
|
||||
"kuberhealthy/deployment": {
|
||||
"OK": true,
|
||||
"Errors": [],
|
||||
"RunDuration": "29.142295647s",
|
||||
"Namespace": "kuberhealthy",
|
||||
"LastRun": "2020-04-06T23:20:31.7176964Z",
|
||||
"AuthoritativePod": "kuberhealthy-67bf8c4686-mbl2j",
|
||||
"uuid": "5f0d2765-60c9-47e8-b2c9-8bc6e61727b2"
|
||||
},
|
||||
"kuberhealthy/dns-status-internal": {
|
||||
"OK": true,
|
||||
"Errors": [],
|
||||
"RunDuration": "2.43940936s",
|
||||
"Namespace": "kuberhealthy",
|
||||
"LastRun": "2020-04-06T23:20:44.6294547Z",
|
||||
"AuthoritativePod": "kuberhealthy-67bf8c4686-mbl2j",
|
||||
"uuid": "c85f95cb-87e2-4ff5-b513-e02b3d25973a"
|
||||
}
|
||||
},
|
||||
"CurrentMaster": "kuberhealthy-7cf79bdc86-m78qr"
|
||||
}
|
||||
```
|
||||
|
||||
This JSON page displays all Kuberhealthy checks running in your cluster. If you have Kuberhealthy checks running in different namespaces, you can filter them by adding the `GET` variable `namespace` parameter: `?namespace=kuberhealthy,kube-system` onto the status page URL.
|
||||
|
||||
|
||||
### Writing Your Own Checks
|
||||
|
||||
Kuberhealthy is designed to be extended with custom check containers that can be written by anyone to check anything. These checks can be written in any language as long as they are packaged in a container. This makes Kuberhealthy an excellent platform for creating your own synthetic checks!
|
||||
|
||||
Creating your own check is a great way to validate your client library, simulate real user workflow, and create a high level of confidence in your service or system uptime.
|
||||
|
||||
To learn more about writing your own checks, along with simple examples, check the [custom check creation](https://github.com/Comcast/kuberhealthy/blob/master/docs/EXTERNAL_CHECK_CREATION.md) documentation.
|
||||
|
||||
|
||||
### Prometheus Integration Details
|
||||
|
||||
When enabling Prometheus (not the operator), the Kuberhealthy service gets the following annotations added:
|
||||
```.env
|
||||
prometheus.io/path: /metrics
|
||||
prometheus.io/port: "80"
|
||||
prometheus.io/scrape: "true"
|
||||
```
|
||||
|
||||
In your prometheus configuration, add the following example scrape_config that scrapes the Kuberhealthy service given the added prometheus annotation:
|
||||
|
||||
```yaml
|
||||
- job_name: 'kuberhealthy'
|
||||
scrape_interval: 1m
|
||||
honor_labels: true
|
||||
metrics_path: /metrics
|
||||
kubernetes_sd_configs:
|
||||
- role: service
|
||||
namespaces:
|
||||
names:
|
||||
- kuberhealthy
|
||||
relabel_configs:
|
||||
- source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scrape]
|
||||
action: keep
|
||||
regex: true
|
||||
```
|
||||
|
||||
You can also specify the target endpoint to be scraped using this example job:
|
||||
```yaml
|
||||
- job_name: kuberhealthy
|
||||
scrape_interval: 1m
|
||||
honor_labels: true
|
||||
metrics_path: /metrics
|
||||
static_configs:
|
||||
- targets:
|
||||
- kuberhealthy.kuberhealthy.svc.cluster.local:80
|
||||
```
|
||||
|
||||
Once the appropriate prometheus configurations are applied, you should be able to see the following Kuberhealthy metrics:
|
||||
- `kuberhealthy_check`
|
||||
- `kuberhealthy_check_duration_seconds`
|
||||
- `kuberhealthy_cluster_states`
|
||||
- `kuberhealthy_running`
|
||||
|
||||
### Creating Key Performance Indicators
|
||||
|
||||
Using these Kuberhealthy metrics, our team has been able to collect KPIs based on the following definitions, calculations, and PromQL queries.
|
||||
|
||||
*Availability*
|
||||
|
||||
We define availability as the K8s cluster control plane being up and functioning as expected. This is measured by our ability to create a deployment, do a rolling update, and delete the deployment within a set period of time.
|
||||
|
||||
We calculate this by measuring Kuberhealthy's [deployment check](https://github.com/Comcast/kuberhealthy/tree/master/cmd/deployment-check) successes and failures.
|
||||
- Availability = Uptime / (Uptime * Downtime)
|
||||
- Uptime = Number of Deployment Check Passes * Check Run Interval
|
||||
- Downtime = Number of Deployment Check Fails * Check Run Interval
|
||||
- Check Run Interval = how often the check runs (`runInterval` set in your KuberhealthyCheck Spec)
|
||||
|
||||
- PromQL Query (Availability % over the past 30 days):
|
||||
```promql
|
||||
1 - (sum(count_over_time(kuberhealthy_check{check="kuberhealthy/deployment", status="0"}[30d])) OR vector(0))/(sum(count_over_time(kuberhealthy_check{check="kuberhealthy/deployment", status="1"}[30d])) * 100)
|
||||
```
|
||||
|
||||
*Utilization*
|
||||
|
||||
We define utilization as user uptake of product (k8s) and its resources (pods, services, etc.). This is measured by how many nodes, deployments, statefulsets, persistent volumes, services, pods, and jobs are being utilized by our customers.
|
||||
We calculate this by counting the total number of nodes, deployments, statefulsets, persistent volumes, services, pods, and jobs.
|
||||
|
||||
*Duration (Latency)*
|
||||
|
||||
We define duration as the control plane's capacity and utilization of throughput. We calculate this by capturing the average run duration of a Kuberhealthy [deployment check](https://github.com/Comcast/kuberhealthy/tree/master/cmd/deployment-check) run.
|
||||
|
||||
- PromQL Query (Deployment check average run duration):
|
||||
```promql
|
||||
avg(kuberhealthy_check_duration_seconds{check="kuberhealthy/deployment"})
|
||||
```
|
||||
|
||||
*Errors / Alerts*
|
||||
|
||||
We define errors as all k8s cluster and Kuberhealthy related alerts. Every time one of our Kuberhealthy check fails, we are alerted of this failure.
|
||||
|
||||
### Thank You!
|
||||
|
||||
Thanks again to everyone in the community for all of your contributions and help! We are excited to see what you build. As always, if you find an issue, have a feature request, or need to open a pull request, please [open an issue](https://github.com/Comcast/kuberhealthy/issues) on the Github project.
|
||||
+56
@@ -0,0 +1,56 @@
|
||||
---
|
||||
layout: blog
|
||||
title: Supporting the Evolving Ingress Specification in Kubernetes 1.18
|
||||
date: 2020-06-05
|
||||
slug: Supporting-the-Evolving-Ingress-Specification-in-Kubernetes-1.18
|
||||
---
|
||||
|
||||
**Authors:** Alex Gervais (Datawire.io)
|
||||
|
||||
Earlier this year, the Kubernetes team released [Kubernetes 1.18](https://kubernetes.io/blog/2020/03/25/kubernetes-1-18-release-announcement/), which extended Ingress. In this blog post, we’ll walk through what’s new in the new Ingress specification, what it means for your applications, and how to upgrade to an ingress controller that supports this new specification.
|
||||
|
||||
### What is Kubernetes Ingress
|
||||
When deploying your applications in Kubernetes, one of the first challenges many people encounter is how to get traffic into their cluster. [Kubernetes ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/) is a collection of routing rules that govern how external users access services running in a Kubernetes cluster. There are [three general approaches](https://blog.getambassador.io/kubernetes-ingress-nodeport-load-balancers-and-ingress-controllers-6e29f1c44f2d) for exposing your application:
|
||||
|
||||
* Using a `NodePort` to expose your application on a port across each of your nodes
|
||||
* Using a `LoadBalancer` service to create an external load balancer that points to a Kubernetes service in your cluster
|
||||
* Using a Kubernetes Ingress resource
|
||||
|
||||
### What’s new in Kubernetes 1.18 Ingress
|
||||
There are three significant additions to the Ingress API in Kubernetes 1.18:
|
||||
|
||||
* A new `pathType` field
|
||||
* A new `IngressClass` resource
|
||||
* Support for wildcards in hostnames
|
||||
|
||||
The new `pathType` field allows you to specify how Ingress paths should match.
|
||||
The field supports three types: `ImplementationSpecific` (default), `exact`, and `prefix`. Explicitly defining the expected behavior of path matching will allow every ingress-controller to support a user’s needs and will increase portability between ingress-controller implementation solutions.
|
||||
|
||||
The `IngressClass` resource specifies how Ingresses should be implemented by controllers. This was added to formalize the commonly used but never standardized `kubernetes.io/ingress.class` annotation and allow for implementation-specific extensions and configuration.
|
||||
|
||||
You can read more about these changes, as well as the support for wildcards in hostnames in more detail in [a previous blog post](https://kubernetes.io/blog/2020/04/02/improvements-to-the-ingress-api-in-kubernetes-1.18/).
|
||||
|
||||
## Supporting Kubernetes ingress
|
||||
[Ambassador](https://www.getambassador.io) is an open-source Envoy-based ingress controller. We believe strongly in supporting common standards such as Kubernetes ingress, which we adopted and [announced our initial support for back in 2019](https://blog.getambassador.io/ambassador-ingress-controller-better-config-reporting-updated-envoy-proxy-99dc9139e28f).
|
||||
|
||||
Every Ambassador release goes through rigorous testing. Therefore, we also contributed an [open conformance test suite](https://github.com/kubernetes-sigs/ingress-controller-conformance), supporting Kubernetes ingress. We wrote the initial bits of test code and will keep iterating over the newly added features and different versions of the Ingress specification as it evolves to a stable v1 GA release. Documentation and usage samples, is one of our top priorities. We understand how complex usage can be, especially when transitioning from a previous version of an API.
|
||||
|
||||
Following a test-driven development approach, the first step we took in supporting Ingress improvements in Ambassador was to translate the revised specification -- both in terms of API and behavior -- into a comprehensible test suite. The test suite, although still under heavy development and going through multiple iterations, was rapidly added to the Ambassador CI infrastructure and acceptance criteria. This means every change to the Ambassador codebase going forward will be compliant with the Ingress API and be tested end-to-end in a lightweight [KIND cluster](https://kind.sigs.k8s.io/). Using KIND allowed us to make rapid improvements while limiting our cloud provider infrastructure bill and testing out unreleased Kubernetes features with pre-release builds.
|
||||
|
||||
### Adopting a new specification
|
||||
With a global comprehension of additions to Ingress introduced in Kubernetes 1.18 and a test suite on hand, we tackled the task of adapting the Ambassador code so that it would support translating the high-level Ingress API resources into Envoy configurations and constructs. Luckily Ambassador already supported previous versions of ingress functionalities so the development effort was incremental.
|
||||
|
||||
We settled on a controller name of `getambassador.io/ingress-controller`. This value, consistent with Ambassador's domain and CRD versions, must be used to tie in an IngressClass `spec.controller` with an Ambassador deployment. The new IngressClass resource allows for extensibility by setting a `spec.parameters` field. At the moment Ambassador makes no use of this field and its usage is reserved for future development.
|
||||
|
||||
Paths can now define different matching behaviors using the `pathType` field. The field will default to a value of `ImplementationSpecific`, which uses the same matching rules as the [Ambassador Mappings](https://www.getambassador.io/docs/latest/topics/using/mappings/) prefix field and previous Ingress specification for backward compatibility reasons.
|
||||
|
||||
### Kubernetes Ingress Controllers
|
||||
A comprehensive [list of Kubernetes ingress controllers](https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/) is available in the Kubernetes documentation. Currently, Ambassador is the only ingress controller that supports these new additions to the ingress specification. Powered by the [Envoy Proxy](https://www.envoyproxy.io), Ambassador is the fastest way for you to try out the new ingress specification today.
|
||||
|
||||
Check out the following resources:
|
||||
|
||||
* Ambassador on [GitHub](https://www.github.com/datawire/ambassador)
|
||||
* The Ambassador [documentation](https://www.getambassador.io/docs)
|
||||
* [Improvements to the Ingress API](https://kubernetes.io/blog/2020/04/02/improvements-to-the-ingress-api-in-kubernetes-1.18/)
|
||||
|
||||
Or join the community on [Slack](http://d6e.co/slack)!
|
||||
@@ -212,4 +212,4 @@ The cloud controller manager uses Go interfaces to allow implementations from an
|
||||
The implementation of the shared controllers highlighted in this document (Node, Route, and Service), and some scaffolding along with the shared cloudprovider interface, is part of the Kubernetes core. Implementations specific to cloud providers are outside the core of Kubernetes and implement the `CloudProvider` interface.
|
||||
|
||||
For more information about developing plugins, see [Developing Cloud Controller Manager](/docs/tasks/administer-cluster/developing-cloud-controller-manager/).
|
||||
{{% /capture %}}
|
||||
{{% /capture %}}
|
||||
@@ -1,7 +1,6 @@
|
||||
---
|
||||
reviewers:
|
||||
- dchen1107
|
||||
- roberthbailey
|
||||
- liggitt
|
||||
title: Control Plane-Node Communication
|
||||
content_template: templates/concept
|
||||
|
||||
@@ -400,9 +400,9 @@ By using the IBM Cloud Kubernetes Service provider, you can create clusters with
|
||||
The name of the Kubernetes Node object is the private IP address of the IBM Cloud Kubernetes Service worker node instance.
|
||||
|
||||
### Networking
|
||||
The IBM Cloud Kubernetes Service provider provides VLANs for quality network performance and network isolation for nodes. You can set up custom firewalls and Calico network policies to add an extra layer of security for your cluster, or connect your cluster to your on-prem data center via VPN. For more information, see [Planning in-cluster and private networking](https://cloud.ibm.com/docs/containers?topic=containers-cs_network_cluster#cs_network_cluster).
|
||||
The IBM Cloud Kubernetes Service provider provides VLANs for quality network performance and network isolation for nodes. You can set up custom firewalls and Calico network policies to add an extra layer of security for your cluster, or connect your cluster to your on-prem data center via VPN. For more information, see [Planning your cluster network setup](https://cloud.ibm.com/docs/containers?topic=containers-plan_clusters).
|
||||
|
||||
To expose apps to the public or within the cluster, you can leverage NodePort, LoadBalancer, or Ingress services. You can also customize the Ingress application load balancer with annotations. For more information, see [Planning to expose your apps with external networking](https://cloud.ibm.com/docs/containers?topic=containers-cs_network_planning#cs_network_planning).
|
||||
To expose apps to the public or within the cluster, you can leverage NodePort, LoadBalancer, or Ingress services. You can also customize the Ingress application load balancer with annotations. For more information, see [Choosing an app exposure service](https://cloud.ibm.com/docs/containers?topic=containers-cs_network_planning#cs_network_planning).
|
||||
|
||||
### Storage
|
||||
The IBM Cloud Kubernetes Service provider leverages Kubernetes-native persistent volumes to enable users to mount file, block, and cloud object storage to their apps. You can also use database-as-a-service and third-party add-ons for persistent storage of your data. For more information, see [Planning highly available persistent storage](https://cloud.ibm.com/docs/containers?topic=containers-storage_planning#storage_planning).
|
||||
|
||||
@@ -136,7 +136,7 @@ classes:
|
||||
controllers.
|
||||
|
||||
* The `workload-low` priority level is for requests from any other service
|
||||
account, which will typically include all requests from controllers runing in
|
||||
account, which will typically include all requests from controllers running in
|
||||
Pods.
|
||||
|
||||
* The `global-default` priority level handles all other traffic, e.g.
|
||||
@@ -375,4 +375,4 @@ the [enhancement proposal](https://github.com/kubernetes/enhancements/blob/maste
|
||||
You can make suggestions and feature requests via [SIG API
|
||||
Machinery](https://github.com/kubernetes/community/tree/master/sig-api-machinery).
|
||||
|
||||
{{% /capture %}}
|
||||
{{% /capture %}}
|
||||
|
||||
@@ -402,7 +402,7 @@ For more information, please see [kubectl edit](/docs/reference/generated/kubect
|
||||
|
||||
You can use `kubectl patch` to update API objects in place. This command supports JSON patch,
|
||||
JSON merge patch, and strategic merge patch. See
|
||||
[Update API Objects in Place Using kubectl patch](/docs/tasks/run-application/update-api-object-kubectl-patch/)
|
||||
[Update API Objects in Place Using kubectl patch](/docs/tasks/manage-kubernetes-objects/update-api-object-kubectl-patch/)
|
||||
and
|
||||
[kubectl patch](/docs/reference/generated/kubectl/kubectl-commands/#patch).
|
||||
|
||||
|
||||
@@ -157,6 +157,91 @@ or {{< glossary_tooltip text="operators" term_id="operator-pattern" >}} that
|
||||
adjust their behavior based on a ConfigMap.
|
||||
{{< /note >}}
|
||||
|
||||
## Using ConfigMaps
|
||||
|
||||
ConfigMaps can be mounted as data volumes. ConfigMaps can also be used by other
|
||||
parts of the system, without being directly exposed to the Pod. For example,
|
||||
ConfigMaps can hold data that other parts of the system should use for configuration.
|
||||
|
||||
### Using ConfigMaps as files from a Pod
|
||||
|
||||
To consume a ConfigMap in a volume in a Pod:
|
||||
|
||||
1. Create a config map or use an existing one. Multiple Pods can reference the same config map.
|
||||
1. Modify your Pod definition to add a volume under `.spec.volumes[]`. Name the volume anything, and have a `.spec.volumes[].configmap.localObjectReference` field set to reference your ConfigMap object.
|
||||
1. Add a `.spec.containers[].volumeMounts[]` to each container that needs the config map. Specify `.spec.containers[].volumeMounts[].readOnly = true` and `.spec.containers[].volumeMounts[].mountPath` to an unused directory name where you would like the config map to appear.
|
||||
1. Modify your image or command line so that the program looks for files in that directory. Each key in the config map `data` map becomes the filename under `mountPath`.
|
||||
|
||||
This is an example of a Pod that mounts a ConfigMap in a volume:
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: mypod
|
||||
spec:
|
||||
containers:
|
||||
- name: mypod
|
||||
image: redis
|
||||
volumeMounts:
|
||||
- name: foo
|
||||
mountPath: "/etc/foo"
|
||||
readOnly: true
|
||||
volumes:
|
||||
- name: foo
|
||||
configMap:
|
||||
name: myconfigmap
|
||||
```
|
||||
|
||||
Each ConfigMap you want to use needs to be referred to in `.spec.volumes`.
|
||||
|
||||
If there are multiple containers in the Pod, then each container needs its
|
||||
own `volumeMounts` block, but only one `.spec.volumes` is needed per ConfigMap.
|
||||
|
||||
#### Mounted ConfigMaps are updated automatically
|
||||
|
||||
When a config map currently consumed in a volume is updated, projected keys are eventually updated as well.
|
||||
The kubelet checks whether the mounted config map is fresh on every periodic sync.
|
||||
However, the kubelet uses its local cache for getting the current value of the ConfigMap.
|
||||
The type of the cache is configurable using the `ConfigMapAndSecretChangeDetectionStrategy` field in
|
||||
the [KubeletConfiguration struct](https://github.com/kubernetes/kubernetes/blob/{{< param "docsbranch" >}}/staging/src/k8s.io/kubelet/config/v1beta1/types.go).
|
||||
A ConfigMap can be either propagated by watch (default), ttl-based, or simply redirecting
|
||||
all requests directly to the API server.
|
||||
As a result, the total delay from the moment when the ConfigMap is updated to the moment
|
||||
when new keys are projected to the Pod can be as long as the kubelet sync period + cache
|
||||
propagation delay, where the cache propagation delay depends on the chosen cache type
|
||||
(it equals to watch propagation delay, ttl of cache, or zero correspondingly).
|
||||
|
||||
{{< feature-state for_k8s_version="v1.18" state="alpha" >}}
|
||||
|
||||
The Kubernetes alpha feature _Immutable Secrets and ConfigMaps_ provides an option to set
|
||||
individual Secrets and ConfigMaps as immutable. For clusters that extensively use ConfigMaps
|
||||
(at least tens of thousands of unique ConfigMap to Pod mounts), preventing changes to their
|
||||
data has the following advantages:
|
||||
|
||||
- protects you from accidental (or unwanted) updates that could cause applications outages
|
||||
- improves performance of your cluster by significantly reducing load on kube-apiserver, by
|
||||
closing watches for config maps marked as immutable.
|
||||
|
||||
To use this feature, enable the `ImmutableEmphemeralVolumes`
|
||||
[feature gate](/docs/reference/command-line-tools-reference/feature-gates/) and set
|
||||
your Secret or ConfigMap `immutable` field to `true`. For example:
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
...
|
||||
data:
|
||||
...
|
||||
immutable: true
|
||||
```
|
||||
|
||||
{{< note >}}
|
||||
Once a ConfigMap or Secret is marked as immutable, it is _not_ possible to revert this change
|
||||
nor to mutate the contents of the `data` field. You can only delete and recreate the ConfigMap.
|
||||
Existing Pods maintain a mount point to the deleted ConfigMap - it is recommended to recreate
|
||||
these pods.
|
||||
{{< /note >}}
|
||||
|
||||
{{% /capture %}}
|
||||
{{% capture whatsnext %}}
|
||||
|
||||
@@ -725,7 +725,7 @@ data has the following advantages:
|
||||
- improves performance of your cluster by significantly reducing load on kube-apiserver, by
|
||||
closing watches for secrets marked as immutable.
|
||||
|
||||
To use this feature, enable the `ImmutableEmphemeralVolumes`
|
||||
To use this feature, enable the `ImmutableEphemeralVolumes`
|
||||
[feature gate](/docs/reference/command-line-tools-reference/feature-gates/) and set
|
||||
your Secret or ConfigMap `immutable` field to `true`. For example:
|
||||
```yaml
|
||||
|
||||
@@ -151,7 +151,7 @@ Once you have those variables filled in you can
|
||||
### Using IBM Cloud Container Registry
|
||||
IBM Cloud Container Registry provides a multi-tenant private image registry that you can use to safely store and share your images. By default, images in your private registry are scanned by the integrated Vulnerability Advisor to detect security issues and potential vulnerabilities. Users in your IBM Cloud account can access your images, or you can use IAM roles and policies to grant access to IBM Cloud Container Registry namespaces.
|
||||
|
||||
To install the IBM Cloud Container Registry CLI plug-in and create a namespace for your images, see [Getting started with IBM Cloud Container Registry](https://cloud.ibm.com/docs/Registry?topic=registry-getting-started).
|
||||
To install the IBM Cloud Container Registry CLI plug-in and create a namespace for your images, see [Getting started with IBM Cloud Container Registry](https://cloud.ibm.com/docs/Registry?topic=Registry-getting-started).
|
||||
|
||||
If you are using the same account and region, you can deploy images that are stored in IBM Cloud Container Registry into the default namespace of your IBM Cloud Kubernetes Service cluster without any additional configuration, see [Building containers from images](https://cloud.ibm.com/docs/containers?topic=containers-images). For other configuration options, see [Understanding how to authorize your cluster to pull images from a registry](https://cloud.ibm.com/docs/containers?topic=containers-registry#cluster_registry_auth).
|
||||
|
||||
|
||||
@@ -186,7 +186,7 @@ Aggregated APIs offer more advanced API features and customization of other feat
|
||||
| Scale Subresource | Allows systems like HorizontalPodAutoscaler and PodDisruptionBudget interact with your new resource | [Yes](/docs/tasks/access-kubernetes-api/extend-api-custom-resource-definitions/#scale-subresource) | Yes |
|
||||
| Status Subresource | Allows fine-grained access control where user writes the spec section and the controller writes the status section. Allows incrementing object Generation on custom resource data mutation (requires separate spec and status sections in the resource) | [Yes](/docs/tasks/access-kubernetes-api/extend-api-custom-resource-definitions/#status-subresource) | Yes |
|
||||
| Other Subresources | Add operations other than CRUD, such as "logs" or "exec". | No | Yes |
|
||||
| strategic-merge-patch | The new endpoints support PATCH with `Content-Type: application/strategic-merge-patch+json`. Useful for updating objects that may be modified both locally, and by the server. For more information, see ["Update API Objects in Place Using kubectl patch"](/docs/tasks/run-application/update-api-object-kubectl-patch/) | No | Yes |
|
||||
| strategic-merge-patch | The new endpoints support PATCH with `Content-Type: application/strategic-merge-patch+json`. Useful for updating objects that may be modified both locally, and by the server. For more information, see ["Update API Objects in Place Using kubectl patch"](/docs/tasks/manage-kubernetes-objects/update-api-object-kubectl-patch/) | No | Yes |
|
||||
| Protocol Buffers | The new resource supports clients that want to use Protocol Buffers | No | Yes |
|
||||
| OpenAPI Schema | Is there an OpenAPI (swagger) schema for the types that can be dynamically fetched from the server? Is the user protected from misspelling field names by ensuring only allowed fields are set? Are types enforced (in other words, don't put an `int` in a `string` field?) | Yes, based on the [OpenAPI v3.0 validation](/docs/tasks/access-kubernetes-api/extend-api-custom-resource-definitions/#validation) schema (GA in 1.16). | Yes |
|
||||
|
||||
|
||||
@@ -11,73 +11,94 @@ card:
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
Overall API conventions are described in the [API conventions doc](https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md).
|
||||
The core of Kubernetes' {{< glossary_tooltip text="control plane" term_id="control-plane" >}}
|
||||
is the {{< glossary_tooltip text="API server" term_id="kube-apiserver" >}}. The API server
|
||||
exposes an HTTP API that lets end users, different parts of your cluster, and external components
|
||||
communicate with one another.
|
||||
|
||||
API endpoints, resource types and samples are described in [API Reference](/docs/reference).
|
||||
The Kubernetes API lets you query and manipulate the state of objects in the Kubernetes API
|
||||
(for example: Pods, Namespaces, ConfigMaps, and Events).
|
||||
|
||||
Remote access to the API is discussed in the [Controlling API Access doc](/docs/reference/access-authn-authz/controlling-access/).
|
||||
|
||||
The Kubernetes API also serves as the foundation for the declarative configuration schema for the system. The [kubectl](/docs/reference/kubectl/overview/) command-line tool can be used to create, update, delete, and get API objects.
|
||||
|
||||
Kubernetes also stores its serialized state (currently in [etcd](https://coreos.com/docs/distributed-configuration/getting-started-with-etcd/)) in terms of the API resources.
|
||||
|
||||
Kubernetes itself is decomposed into multiple components, which interact through its API.
|
||||
API endpoints, resource types and samples are described in the [API Reference](/docs/reference/kubernetes-api/).
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
|
||||
{{% capture body %}}
|
||||
|
||||
## API changes
|
||||
|
||||
In our experience, any system that is successful needs to grow and change as new use cases emerge or existing ones change. Therefore, we expect the Kubernetes API to continuously change and grow. However, we intend to not break compatibility with existing clients, for an extended period of time. In general, new API resources and new resource fields can be expected to be added frequently. Elimination of resources or fields will require following the [API deprecation policy](/docs/reference/using-api/deprecation-policy/).
|
||||
Any system that is successful needs to grow and change as new use cases emerge or existing ones change.
|
||||
Therefore, Kubernetes has design features to allow the Kubernetes API to continuously change and grow.
|
||||
The Kubernetes project aims to _not_ break compatibility with existing clients, and to maintain that
|
||||
compatibility for a length of time so that other projects have an opportunity to adapt.
|
||||
|
||||
What constitutes a compatible change and how to change the API are detailed by the [API change document](https://git.k8s.io/community/contributors/devel/sig-architecture/api_changes.md).
|
||||
In general, new API resources and new resource fields can be added often and frequently.
|
||||
Elimination of resources or fields requires following the
|
||||
[API deprecation policy](/docs/reference/using-api/deprecation-policy/).
|
||||
|
||||
## OpenAPI and Swagger definitions
|
||||
What constitutes a compatible change, and how to change the API, are detailed in
|
||||
[API changes](https://git.k8s.io/community/contributors/devel/sig-architecture/api_changes.md#readme).
|
||||
|
||||
## OpenAPI specification {#api-specification}
|
||||
|
||||
Complete API details are documented using [OpenAPI](https://www.openapis.org/).
|
||||
|
||||
Starting with Kubernetes 1.10, the Kubernetes API server serves an OpenAPI spec via the `/openapi/v2` endpoint.
|
||||
The requested format is specified by setting HTTP headers:
|
||||
The Kubernetes API server serves an OpenAPI spec via the `/openapi/v2` endpoint.
|
||||
You can request the response format using request headers as follows:
|
||||
|
||||
Header | Possible Values
|
||||
------ | ---------------
|
||||
Accept | `application/json`, `application/com.github.proto-openapi.spec.v2@v1.0+protobuf` (the default content-type is `application/json` for `*/*` or not passing this header)
|
||||
Accept-Encoding | `gzip` (not passing this header is acceptable)
|
||||
|
||||
Prior to 1.14, format-separated endpoints (`/swagger.json`, `/swagger-2.0.0.json`, `/swagger-2.0.0.pb-v1`, `/swagger-2.0.0.pb-v1.gz`)
|
||||
serve the OpenAPI spec in different formats. These endpoints are deprecated, and are removed in Kubernetes 1.14.
|
||||
|
||||
**Examples of getting OpenAPI spec**:
|
||||
|
||||
Before 1.10 | Starting with Kubernetes 1.10
|
||||
----------- | -----------------------------
|
||||
GET /swagger.json | GET /openapi/v2 **Accept**: application/json
|
||||
GET /swagger-2.0.0.pb-v1 | GET /openapi/v2 **Accept**: application/com.github.proto-openapi.spec.v2@v1.0+protobuf
|
||||
GET /swagger-2.0.0.pb-v1.gz | GET /openapi/v2 **Accept**: application/com.github.proto-openapi.spec.v2@v1.0+protobuf **Accept-Encoding**: gzip
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Header</th>
|
||||
<th style="min-width: 50%;">Possible values</th>
|
||||
<th>Notes</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>Accept-Encoding</code></td>
|
||||
<td><code>gzip</code></td>
|
||||
<td><em>not supplying this header is also acceptable</em></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="3"><code>Accept</code></td>
|
||||
<td><code>application/com.github.proto-openapi.spec.v2@v1.0+protobuf</code></td>
|
||||
<td><em>mainly for intra-cluster use</em></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>application/json</code></td>
|
||||
<td><em>default</em></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>*</code></td>
|
||||
<td><em>serves </em><code>application/json</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
<caption>Valid request header values for OpenAPI v2 queries</caption>
|
||||
</table>
|
||||
|
||||
Kubernetes implements an alternative Protobuf based serialization format for the API that is primarily intended for intra-cluster communication, documented in the [design proposal](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/api-machinery/protobuf.md) and the IDL files for each schema are located in the Go packages that define the API objects.
|
||||
|
||||
Prior to 1.14, the Kubernetes apiserver also exposes an API that can be used to retrieve
|
||||
the [Swagger v1.2](http://swagger.io/) Kubernetes API spec at `/swaggerapi`.
|
||||
This endpoint is deprecated, and was removed in Kubernetes 1.14.
|
||||
|
||||
## API versioning
|
||||
|
||||
To make it easier to eliminate fields or restructure resource representations, Kubernetes supports
|
||||
multiple API versions, each at a different API path, such as `/api/v1` or
|
||||
`/apis/extensions/v1beta1`.
|
||||
|
||||
We chose to version at the API level rather than at the resource or field level to ensure that the API presents a clear, consistent view of system resources and behavior, and to enable controlling access to end-of-life and/or experimental APIs. The JSON and Protobuf serialization schemas follow the same guidelines for schema changes - all descriptions below cover both formats.
|
||||
Versioning is done at the API level rather than at the resource or field level to ensure that the
|
||||
API presents a clear, consistent view of system resources and behavior, and to enable controlling
|
||||
access to end-of-life and/or experimental APIs.
|
||||
|
||||
Note that API versioning and Software versioning are only indirectly related. The [API and release
|
||||
versioning proposal](https://git.k8s.io/community/contributors/design-proposals/release/versioning.md) describes the relationship between API versioning and
|
||||
software versioning.
|
||||
The JSON and Protobuf serialization schemas follow the same guidelines for schema changes - all descriptions below cover both formats.
|
||||
|
||||
Note that API versioning and Software versioning are only indirectly related. The
|
||||
[Kubernetes Release Versioning](https://git.k8s.io/community/contributors/design-proposals/release/versioning.md)
|
||||
proposal describes the relationship between API versioning and software versioning.
|
||||
|
||||
Different API versions imply different levels of stability and support. The criteria for each level are described
|
||||
in more detail in the [API Changes documentation](https://git.k8s.io/community/contributors/devel/sig-architecture/api_changes.md#alpha-beta-and-stable-versions). They are summarized here:
|
||||
in more detail in the
|
||||
[API Changes](https://git.k8s.io/community/contributors/devel/sig-architecture/api_changes.md#alpha-beta-and-stable-versions)
|
||||
documentation. They are summarized here:
|
||||
|
||||
- Alpha level:
|
||||
- The version names contain `alpha` (e.g. `v1alpha1`).
|
||||
@@ -101,35 +122,36 @@ in more detail in the [API Changes documentation](https://git.k8s.io/community/c
|
||||
|
||||
## API groups
|
||||
|
||||
To make it easier to extend the Kubernetes API, we implemented [*API groups*](https://git.k8s.io/community/contributors/design-proposals/api-machinery/api-group.md).
|
||||
To make it easier to extend its API, Kubernetes implements [*API groups*](https://git.k8s.io/community/contributors/design-proposals/api-machinery/api-group.md).
|
||||
The API group is specified in a REST path and in the `apiVersion` field of a serialized object.
|
||||
|
||||
Currently there are several API groups in use:
|
||||
There are several API groups in a cluster:
|
||||
|
||||
1. The *core* group, often referred to as the *legacy group*, is at the REST path `/api/v1` and uses `apiVersion: v1`.
|
||||
1. The *core* group, also referred to as the *legacy* group, is at the REST path `/api/v1` and uses `apiVersion: v1`.
|
||||
|
||||
1. The named groups are at REST path `/apis/$GROUP_NAME/$VERSION`, and use `apiVersion: $GROUP_NAME/$VERSION`
|
||||
(e.g. `apiVersion: batch/v1`). Full list of supported API groups can be seen in [Kubernetes API reference](/docs/reference/).
|
||||
1. *Named* groups are at REST path `/apis/$GROUP_NAME/$VERSION`, and use `apiVersion: $GROUP_NAME/$VERSION`
|
||||
(e.g. `apiVersion: batch/v1`). The Kubernetes [API reference](/docs/reference/kubernetes-api/) has a
|
||||
full list of available API groups.
|
||||
|
||||
|
||||
There are two supported paths to extending the API with [custom resources](/docs/concepts/api-extension/custom-resources/):
|
||||
There are two paths to extending the API with [custom resources](/docs/concepts/api-extension/custom-resources/):
|
||||
|
||||
1. [CustomResourceDefinition](/docs/tasks/access-kubernetes-api/extend-api-custom-resource-definitions/)
|
||||
is for users with very basic CRUD needs.
|
||||
1. Users needing the full set of Kubernetes API semantics can implement their own apiserver
|
||||
lets you declaratively define how the API server should provide your chosen resource API.
|
||||
1. You can also [implement your own extension API server](/docs/tasks/access-kubernetes-api/setup-extension-api-server/)
|
||||
and use the [aggregator](/docs/tasks/access-kubernetes-api/configure-aggregation-layer/)
|
||||
to make it seamless for clients.
|
||||
|
||||
|
||||
## Enabling or disabling API groups
|
||||
|
||||
Certain resources and API groups are enabled by default. They can be enabled or disabled by setting `--runtime-config`
|
||||
on apiserver. `--runtime-config` accepts comma separated values. For example: to disable batch/v1, set
|
||||
`--runtime-config=batch/v1=false`, to enable batch/v2alpha1, set `--runtime-config=batch/v2alpha1`.
|
||||
The flag accepts comma separated set of key=value pairs describing runtime configuration of the apiserver.
|
||||
Certain resources and API groups are enabled by default. They can be enabled or disabled by setting `--runtime-config`
|
||||
as a command line option to the kube-apiserver.
|
||||
|
||||
{{< note >}}Enabling or disabling groups or resources requires restarting apiserver and controller-manager
|
||||
to pick up the `--runtime-config` changes.{{< /note >}}
|
||||
`--runtime-config` accepts comma separated values. For example: to disable batch/v1, set
|
||||
`--runtime-config=batch/v1=false`; to enable batch/v2alpha1, set `--runtime-config=batch/v2alpha1`.
|
||||
The flag accepts comma separated set of key=value pairs describing runtime configuration of the API server.
|
||||
|
||||
{{< note >}}Enabling or disabling groups or resources requires restarting the kube-apiserver and the
|
||||
kube-controller-manager to pick up the `--runtime-config` changes.{{< /note >}}
|
||||
|
||||
## Enabling specific resources in the extensions/v1beta1 group
|
||||
|
||||
@@ -139,4 +161,20 @@ For example: to enable deployments and daemonsets, set
|
||||
|
||||
{{< note >}}Individual resource enablement/disablement is only supported in the `extensions/v1beta1` API group for legacy reasons.{{< /note >}}
|
||||
|
||||
## Persistence
|
||||
|
||||
Kubernetes stores its serialized state in terms of the API resources by writing them into
|
||||
{{< glossary_tooltip term_id="etcd" >}}.
|
||||
|
||||
{{% /capture %}}
|
||||
{{% capture whatsnext %}}
|
||||
[Controlling API Access](/docs/reference/access-authn-authz/controlling-access/) describes
|
||||
how the cluster manages authentication and authorization for API access.
|
||||
|
||||
Overall API conventions are described in the
|
||||
[API conventions](https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#api-conventions)
|
||||
document.
|
||||
|
||||
API endpoints, resource types and samples are described in the [API Reference](/docs/reference/kubernetes-api/).
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
@@ -16,12 +16,7 @@ kubectl get pods --field-selector status.phase=Running
|
||||
```
|
||||
|
||||
{{< note >}}
|
||||
Field selectors are essentially resource *filters*. By default, no selectors/filters are applied, meaning that all resources of the specified type are selected. This makes the following `kubectl` queries equivalent:
|
||||
|
||||
```shell
|
||||
kubectl get pods
|
||||
kubectl get pods --field-selector ""
|
||||
```
|
||||
Field selectors are essentially resource *filters*. By default, no selectors/filters are applied, meaning that all resources of the specified type are selected. This makes the `kubectl` queries `kubectl get pods` and `kubectl get pods --field-selector ""` equivalent.
|
||||
{{< /note >}}
|
||||
|
||||
## Supported fields
|
||||
|
||||
@@ -56,8 +56,8 @@ developers of non-critical applications. The following listed controls should be
|
||||
enforced/disallowed:
|
||||
|
||||
<table>
|
||||
<caption style="display:none">Baseline policy specification</caption>
|
||||
<tbody>
|
||||
<caption style="display:none">Baseline policy specification</caption>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><strong>Control</strong></td>
|
||||
<td><strong>Policy</strong></td>
|
||||
@@ -115,10 +115,10 @@ enforced/disallowed:
|
||||
<tr>
|
||||
<td>AppArmor <em>(optional)</em></td>
|
||||
<td>
|
||||
On supported hosts, the `runtime/default` AppArmor profile is applied by default. The default policy should prevent overriding or disabling the policy, or restrict overrides to a whitelisted set of profiles.<br>
|
||||
On supported hosts, the 'runtime/default' AppArmor profile is applied by default. The default policy should prevent overriding or disabling the policy, or restrict overrides to a whitelisted set of profiles.<br>
|
||||
<br><b>Restricted Fields:</b><br>
|
||||
metadata.annotations['container.apparmor.security.beta.kubernetes.io/*']<br>
|
||||
<br><b>Allowed Values:</b> runtime/default, undefined<br>
|
||||
<br><b>Allowed Values:</b> 'runtime/default', undefined<br>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
@@ -132,6 +132,31 @@ enforced/disallowed:
|
||||
<br><b>Allowed Values:</b> undefined/nil<br>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>/proc Mount Type</td>
|
||||
<td>
|
||||
The default /proc masks are set up to reduce attack surface, and should be required.<br>
|
||||
<br><b>Restricted Fields:</b><br>
|
||||
spec.containers[*].securityContext.procMount<br>
|
||||
spec.initContainers[*].securityContext.procMount<br>
|
||||
<br><b>Allowed Values:</b> undefined/nil, 'Default'<br>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Sysctls</td>
|
||||
<td>
|
||||
Sysctls can disable security mechanisms or affect all containers on a host, and should be disallowed except for a whitelisted "safe" subset.
|
||||
A sysctl is considered safe if it is namespaced in the container or the Pod, and it is isolated from other Pods or processes on the same Node.<br>
|
||||
<br><b>Restricted Fields:</b><br>
|
||||
spec.securityContext.sysctls<br>
|
||||
<br><b>Allowed Values:</b><br>
|
||||
kernel.shm_rmid_forced<br>
|
||||
net.ipv4.ip_local_port_range<br>
|
||||
net.ipv4.tcp_syncookies<br>
|
||||
net.ipv4.ping_group_range<br>
|
||||
undefined/empty<br>
|
||||
</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
@@ -143,7 +168,7 @@ well as lower-trust users.The following listed controls should be enforced/disal
|
||||
|
||||
|
||||
<table>
|
||||
<caption style="display:none">Restricted policy specification</caption>
|
||||
<caption style="display:none">Restricted policy specification</caption>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><strong>Control</strong></td>
|
||||
@@ -184,7 +209,7 @@ well as lower-trust users.The following listed controls should be enforced/disal
|
||||
<tr>
|
||||
<td>Privilege Escalation</td>
|
||||
<td>
|
||||
Privilege escalation to root should not be allowed.<br>
|
||||
Privilege escalation to root should not be allowed.<br>
|
||||
<br><b>Restricted Fields:</b><br>
|
||||
spec.containers[*].securityContext.privileged<br>
|
||||
spec.initContainers[*].securityContext.privileged<br>
|
||||
@@ -194,7 +219,7 @@ well as lower-trust users.The following listed controls should be enforced/disal
|
||||
<tr>
|
||||
<td>Running as Non-root</td>
|
||||
<td>
|
||||
Containers must be required to run as non-root users.<br>
|
||||
Containers must be required to run as non-root users.<br>
|
||||
<br><b>Restricted Fields:</b><br>
|
||||
spec.securityContext.runAsNonRoot<br>
|
||||
spec.containers[*].securityContext.runAsNonRoot<br>
|
||||
@@ -205,7 +230,7 @@ well as lower-trust users.The following listed controls should be enforced/disal
|
||||
<tr>
|
||||
<td>Non-root groups <em>(optional)</em></td>
|
||||
<td>
|
||||
Containers should be forbidden from running with a root primary or supplementary GID.<br>
|
||||
Containers should be forbidden from running with a root primary or supplementary GID.<br>
|
||||
<br><b>Restricted Fields:</b><br>
|
||||
spec.securityContext.runAsGroup<br>
|
||||
spec.securityContext.supplementalGroups[*]<br>
|
||||
@@ -224,12 +249,12 @@ well as lower-trust users.The following listed controls should be enforced/disal
|
||||
<tr>
|
||||
<td>Seccomp</td>
|
||||
<td>
|
||||
The runtime/default seccomp profile must be required, or allow additional whitelisted values.<br>
|
||||
The 'runtime/default' seccomp profile must be required, or allow additional whitelisted values.<br>
|
||||
<br><b>Restricted Fields:</b><br>
|
||||
metadata.annotations['seccomp.security.alpha.kubernetes.io/pod']<br>
|
||||
metadata.annotations['container.seccomp.security.alpha.kubernetes.io/*']<br>
|
||||
<br><b>Allowed Values:</b><br>
|
||||
runtime/default<br>
|
||||
'runtime/default'<br>
|
||||
undefined (container annotation)<br>
|
||||
</td>
|
||||
</tr>
|
||||
|
||||
+1
-1
@@ -93,7 +93,7 @@ hostaliases-pod 0/1 Completed 0 6s 10.200
|
||||
The `hosts` file content would look like this:
|
||||
|
||||
```shell
|
||||
kubectl exec hostaliases-pod -- cat /etc/hosts
|
||||
kubectl logs hostaliases-pod
|
||||
```
|
||||
|
||||
```none
|
||||
|
||||
@@ -456,15 +456,13 @@ spec:
|
||||
```
|
||||
|
||||
#### Regional Persistent Disks
|
||||
{{< feature-state for_k8s_version="v1.10" state="beta" >}}
|
||||
|
||||
The [Regional Persistent Disks](https://cloud.google.com/compute/docs/disks/#repds) feature allows the creation of Persistent Disks that are available in two zones within the same region. In order to use this feature, the volume must be provisioned as a PersistentVolume; referencing the volume directly from a pod is not supported.
|
||||
|
||||
#### Manually provisioning a Regional PD PersistentVolume
|
||||
Dynamic provisioning is possible using a [StorageClass for GCE PD](/docs/concepts/storage/storage-classes/#gce).
|
||||
Before creating a PersistentVolume, you must create the PD:
|
||||
```shell
|
||||
gcloud beta compute disks create --size=500GB my-data-disk
|
||||
gcloud compute disks create --size=500GB my-data-disk
|
||||
--region us-central1
|
||||
--replica-zones us-central1-a,us-central1-b
|
||||
```
|
||||
@@ -475,8 +473,6 @@ apiVersion: v1
|
||||
kind: PersistentVolume
|
||||
metadata:
|
||||
name: test-volume
|
||||
labels:
|
||||
failure-domain.beta.kubernetes.io/zone: us-central1-a__us-central1-b
|
||||
spec:
|
||||
capacity:
|
||||
storage: 400Gi
|
||||
@@ -485,6 +481,15 @@ spec:
|
||||
gcePersistentDisk:
|
||||
pdName: my-data-disk
|
||||
fsType: ext4
|
||||
nodeAffinity:
|
||||
required:
|
||||
nodeSelectorTerms:
|
||||
- matchExpressions:
|
||||
- key: failure-domain.beta.kubernetes.io/zone
|
||||
operator: In
|
||||
values:
|
||||
- us-central1-a
|
||||
- us-central1-b
|
||||
```
|
||||
|
||||
#### CSI Migration
|
||||
|
||||
@@ -18,9 +18,9 @@ collected. Deleting a DaemonSet will clean up the Pods it created.
|
||||
|
||||
Some typical uses of a DaemonSet are:
|
||||
|
||||
- running a cluster storage daemon, such as `glusterd`, `ceph`, on each node.
|
||||
- running a logs collection daemon on every node, such as `fluentd` or `filebeat`.
|
||||
- running a node monitoring daemon on every node, such as [Prometheus Node Exporter](https://github.com/prometheus/node_exporter), [Flowmill](https://github.com/Flowmill/flowmill-k8s/), [Sysdig Agent](https://docs.sysdig.com), `collectd`, [Dynatrace OneAgent](https://www.dynatrace.com/technologies/kubernetes-monitoring/), [AppDynamics Agent](https://docs.appdynamics.com/display/CLOUD/Container+Visibility+with+Kubernetes), [Datadog agent](https://docs.datadoghq.com/agent/kubernetes/daemonset_setup/), [New Relic agent](https://docs.newrelic.com/docs/integrations/kubernetes-integration/installation/kubernetes-installation-configuration), Ganglia `gmond`, [Instana Agent](https://www.instana.com/supported-integrations/kubernetes-monitoring/) or [Elastic Metricbeat](https://www.elastic.co/guide/en/beats/metricbeat/current/running-on-kubernetes.html).
|
||||
- running a cluster storage daemon on every node
|
||||
- running a logs collection daemon on every node
|
||||
- running a node monitoring daemon on every node
|
||||
|
||||
In a simple case, one DaemonSet, covering all nodes, would be used for each type of daemon.
|
||||
A more complex setup might use multiple DaemonSets for a single type of daemon, but with
|
||||
@@ -95,7 +95,7 @@ another DaemonSet, or via another workload resource such as ReplicaSet. Otherwi
|
||||
Kubernetes will not stop you from doing this. One case where you might want to do this is manually
|
||||
create a Pod with a different value on a node for testing.
|
||||
|
||||
### Running Pods on Only Some Nodes
|
||||
### Running Pods on select Nodes
|
||||
|
||||
If you specify a `.spec.template.spec.nodeSelector`, then the DaemonSet controller will
|
||||
create Pods on nodes which match that [node
|
||||
@@ -103,7 +103,7 @@ selector](/docs/concepts/scheduling-eviction/assign-pod-node/). Likewise if you
|
||||
then DaemonSet controller will create Pods on nodes which match that [node affinity](/docs/concepts/scheduling-eviction/assign-pod-node/).
|
||||
If you do not specify either, then the DaemonSet controller will create Pods on all nodes.
|
||||
|
||||
## How Daemon Pods are Scheduled
|
||||
## How Daemon Pods are scheduled
|
||||
|
||||
### Scheduled by default scheduler
|
||||
|
||||
@@ -144,7 +144,6 @@ In addition, `node.kubernetes.io/unschedulable:NoSchedule` toleration is added
|
||||
automatically to DaemonSet Pods. The default scheduler ignores
|
||||
`unschedulable` Nodes when scheduling DaemonSet Pods.
|
||||
|
||||
|
||||
### Taints and Tolerations
|
||||
|
||||
Although Daemon Pods respect
|
||||
@@ -152,17 +151,14 @@ Although Daemon Pods respect
|
||||
the following tolerations are added to DaemonSet Pods automatically according to
|
||||
the related features.
|
||||
|
||||
| Toleration Key | Effect | Version | Description |
|
||||
| ---------------------------------------- | ---------- | ------- | ------------------------------------------------------------ |
|
||||
| `node.kubernetes.io/not-ready` | NoExecute | 1.13+ | DaemonSet pods will not be evicted when there are node problems such as a network partition. |
|
||||
| `node.kubernetes.io/unreachable` | NoExecute | 1.13+ | DaemonSet pods will not be evicted when there are node problems such as a network partition. |
|
||||
| `node.kubernetes.io/disk-pressure` | NoSchedule | 1.8+ | |
|
||||
| `node.kubernetes.io/memory-pressure` | NoSchedule | 1.8+ | |
|
||||
| `node.kubernetes.io/unschedulable` | NoSchedule | 1.12+ | DaemonSet pods tolerate unschedulable attributes by default scheduler. |
|
||||
| `node.kubernetes.io/network-unavailable` | NoSchedule | 1.12+ | DaemonSet pods, who uses host network, tolerate network-unavailable attributes by default scheduler. |
|
||||
|
||||
|
||||
|
||||
| Toleration Key | Effect | Version | Description |
|
||||
| ---------------------------------------- | ---------- | ------- | ----------- |
|
||||
| `node.kubernetes.io/not-ready` | NoExecute | 1.13+ | DaemonSet pods will not be evicted when there are node problems such as a network partition. |
|
||||
| `node.kubernetes.io/unreachable` | NoExecute | 1.13+ | DaemonSet pods will not be evicted when there are node problems such as a network partition. |
|
||||
| `node.kubernetes.io/disk-pressure` | NoSchedule | 1.8+ | |
|
||||
| `node.kubernetes.io/memory-pressure` | NoSchedule | 1.8+ | |
|
||||
| `node.kubernetes.io/unschedulable` | NoSchedule | 1.12+ | DaemonSet pods tolerate unschedulable attributes by default scheduler. |
|
||||
| `node.kubernetes.io/network-unavailable` | NoSchedule | 1.12+ | DaemonSet pods, who uses host network, tolerate network-unavailable attributes by default scheduler. |
|
||||
|
||||
## Communicating with Daemon Pods
|
||||
|
||||
@@ -195,7 +191,7 @@ You can [perform a rolling update](/docs/tasks/manage-daemon/update-daemon-set/)
|
||||
|
||||
## Alternatives to DaemonSet
|
||||
|
||||
### Init Scripts
|
||||
### Init scripts
|
||||
|
||||
It is certainly possible to run daemon processes by directly starting them on a node (e.g. using
|
||||
`init`, `upstartd`, or `systemd`). This is perfectly fine. However, there are several advantages to
|
||||
|
||||
@@ -140,19 +140,19 @@ See section [specifying your own pod selector](#specifying-your-own-pod-selector
|
||||
There are three main types of task suitable to run as a Job:
|
||||
|
||||
1. Non-parallel Jobs
|
||||
- normally, only one Pod is started, unless the Pod fails.
|
||||
- the Job is complete as soon as its Pod terminates successfully.
|
||||
- normally, only one Pod is started, unless the Pod fails.
|
||||
- the Job is complete as soon as its Pod terminates successfully.
|
||||
1. Parallel Jobs with a *fixed completion count*:
|
||||
- specify a non-zero positive value for `.spec.completions`.
|
||||
- the Job represents the overall task, and is complete when there is one successful Pod for each value in the range 1 to `.spec.completions`.
|
||||
- **not implemented yet:** Each Pod is passed a different index in the range 1 to `.spec.completions`.
|
||||
- specify a non-zero positive value for `.spec.completions`.
|
||||
- the Job represents the overall task, and is complete when there is one successful Pod for each value in the range 1 to `.spec.completions`.
|
||||
- **not implemented yet:** Each Pod is passed a different index in the range 1 to `.spec.completions`.
|
||||
1. Parallel Jobs with a *work queue*:
|
||||
- do not specify `.spec.completions`, default to `.spec.parallelism`.
|
||||
- the Pods must coordinate amongst themselves or an external service to determine what each should work on. For example, a Pod might fetch a batch of up to N items from the work queue.
|
||||
- each Pod is independently capable of determining whether or not all its peers are done, and thus that the entire Job is done.
|
||||
- when _any_ Pod from the Job terminates with success, no new Pods are created.
|
||||
- once at least one Pod has terminated with success and all Pods are terminated, then the Job is completed with success.
|
||||
- once any Pod has exited with success, no other Pod should still be doing any work for this task or writing any output. They should all be in the process of exiting.
|
||||
- do not specify `.spec.completions`, default to `.spec.parallelism`.
|
||||
- the Pods must coordinate amongst themselves or an external service to determine what each should work on. For example, a Pod might fetch a batch of up to N items from the work queue.
|
||||
- each Pod is independently capable of determining whether or not all its peers are done, and thus that the entire Job is done.
|
||||
- when _any_ Pod from the Job terminates with success, no new Pods are created.
|
||||
- once at least one Pod has terminated with success and all Pods are terminated, then the Job is completed with success.
|
||||
- once any Pod has exited with success, no other Pod should still be doing any work for this task or writing any output. They should all be in the process of exiting.
|
||||
|
||||
For a _non-parallel_ Job, you can leave both `.spec.completions` and `.spec.parallelism` unset. When both are
|
||||
unset, both are defaulted to 1.
|
||||
|
||||
@@ -322,7 +322,7 @@ reasons:
|
||||
|
||||
{{% capture whatsnext %}}
|
||||
|
||||
* Read about [creating a Pod that has an init container](/docs/tasks/configure-pod-container/configure-pod-initialization/#creating-a-pod-that-has-an-init-container)
|
||||
* Read about [creating a Pod that has an init container](/docs/tasks/configure-pod-container/configure-pod-initialization/#create-a-pod-that-has-an-init-container)
|
||||
* Learn how to [debug init containers](/docs/tasks/debug-application-cluster/debug-init-containers/)
|
||||
|
||||
{{% /capture %}}
|
||||
{{% /capture %}}
|
||||
|
||||
@@ -138,19 +138,19 @@ git status
|
||||
The output is similar to:
|
||||
|
||||
```
|
||||
static/docs/reference/generated/kubernetes-api/v1.17/css/bootstrap.min.css
|
||||
static/docs/reference/generated/kubernetes-api/v1.17/css/font-awesome.min.css
|
||||
static/docs/reference/generated/kubernetes-api/v1.17/css/stylesheet.css
|
||||
static/docs/reference/generated/kubernetes-api/v1.17/fonts/FontAwesome.otf
|
||||
static/docs/reference/generated/kubernetes-api/v1.17/fonts/fontawesome-webfont.eot
|
||||
static/docs/reference/generated/kubernetes-api/v1.17/fonts/fontawesome-webfont.svg
|
||||
static/docs/reference/generated/kubernetes-api/v1.17/fonts/fontawesome-webfont.ttf
|
||||
static/docs/reference/generated/kubernetes-api/v1.17/fonts/fontawesome-webfont.woff
|
||||
static/docs/reference/generated/kubernetes-api/v1.17/fonts/fontawesome-webfont.woff2
|
||||
static/docs/reference/generated/kubernetes-api/v1.17/index.html
|
||||
static/docs/reference/generated/kubernetes-api/v1.17/js/jquery.scrollTo.min.js
|
||||
static/docs/reference/generated/kubernetes-api/v1.17/js/navData.js
|
||||
static/docs/reference/generated/kubernetes-api/v1.17/js/scroll.js
|
||||
static/docs/reference/generated/kubernetes-api/{{< param "version" >}}/css/bootstrap.min.css
|
||||
static/docs/reference/generated/kubernetes-api/{{< param "version" >}}/css/font-awesome.min.css
|
||||
static/docs/reference/generated/kubernetes-api/{{< param "version" >}}/css/stylesheet.css
|
||||
static/docs/reference/generated/kubernetes-api/{{< param "version" >}}/fonts/FontAwesome.otf
|
||||
static/docs/reference/generated/kubernetes-api/{{< param "version" >}}/fonts/fontawesome-webfont.eot
|
||||
static/docs/reference/generated/kubernetes-api/{{< param "version" >}}/fonts/fontawesome-webfont.svg
|
||||
static/docs/reference/generated/kubernetes-api/{{< param "version" >}}/fonts/fontawesome-webfont.ttf
|
||||
static/docs/reference/generated/kubernetes-api/{{< param "version" >}}/fonts/fontawesome-webfont.woff
|
||||
static/docs/reference/generated/kubernetes-api/{{< param "version" >}}/fonts/fontawesome-webfont.woff2
|
||||
static/docs/reference/generated/kubernetes-api/{{< param "version" >}}/index.html
|
||||
static/docs/reference/generated/kubernetes-api/{{< param "version" >}}/js/jquery.scrollTo.min.js
|
||||
static/docs/reference/generated/kubernetes-api/{{< param "version" >}}/js/navData.js
|
||||
static/docs/reference/generated/kubernetes-api/{{< param "version" >}}/js/scroll.js
|
||||
```
|
||||
|
||||
## Updating the API reference index pages
|
||||
@@ -177,7 +177,7 @@ version number.
|
||||
## Locally test the API reference
|
||||
|
||||
Publish a local version of the API reference.
|
||||
Verify the [local preview](http://localhost:1313/docs/reference/generated/kubernetes-api/v1.17/).
|
||||
Verify the [local preview](http://localhost:1313/docs/reference/generated/kubernetes-api/{{< param "version">}}/).
|
||||
|
||||
```shell
|
||||
cd <web-base>
|
||||
|
||||
@@ -219,19 +219,19 @@ static/docs/reference/generated/kubectl/css/font-awesome.min.css
|
||||
### Generated Kubernetes API reference directories and files
|
||||
|
||||
```
|
||||
static/docs/reference/generated/kubernetes-api/v1.17/index.html
|
||||
static/docs/reference/generated/kubernetes-api/v1.17/js/navData.js
|
||||
static/docs/reference/generated/kubernetes-api/v1.17/js/scroll.js
|
||||
static/docs/reference/generated/kubernetes-api/v1.17/js/query.scrollTo.min.js
|
||||
static/docs/reference/generated/kubernetes-api/v1.17/css/font-awesome.min.css
|
||||
static/docs/reference/generated/kubernetes-api/v1.17/css/bootstrap.min.css
|
||||
static/docs/reference/generated/kubernetes-api/v1.17/css/stylesheet.css
|
||||
static/docs/reference/generated/kubernetes-api/v1.17/fonts/FontAwesome.otf
|
||||
static/docs/reference/generated/kubernetes-api/v1.17/fonts/fontawesome-webfont.eot
|
||||
static/docs/reference/generated/kubernetes-api/v1.17/fonts/fontawesome-webfont.svg
|
||||
static/docs/reference/generated/kubernetes-api/v1.17/fonts/fontawesome-webfont.ttf
|
||||
static/docs/reference/generated/kubernetes-api/v1.17/fonts/fontawesome-webfont.woff
|
||||
static/docs/reference/generated/kubernetes-api/v1.17/fonts/fontawesome-webfont.woff2
|
||||
static/docs/reference/generated/kubernetes-api/{{< param "version" >}}/index.html
|
||||
static/docs/reference/generated/kubernetes-api/{{< param "version" >}}/js/navData.js
|
||||
static/docs/reference/generated/kubernetes-api/{{< param "version" >}}/js/scroll.js
|
||||
static/docs/reference/generated/kubernetes-api/{{< param "version" >}}/js/query.scrollTo.min.js
|
||||
static/docs/reference/generated/kubernetes-api/{{< param "version" >}}/css/font-awesome.min.css
|
||||
static/docs/reference/generated/kubernetes-api/{{< param "version" >}}/css/bootstrap.min.css
|
||||
static/docs/reference/generated/kubernetes-api/{{< param "version" >}}/css/stylesheet.css
|
||||
static/docs/reference/generated/kubernetes-api/{{< param "version" >}}/fonts/FontAwesome.otf
|
||||
static/docs/reference/generated/kubernetes-api/{{< param "version" >}}/fonts/fontawesome-webfont.eot
|
||||
static/docs/reference/generated/kubernetes-api/{{< param "version" >}}/fonts/fontawesome-webfont.svg
|
||||
static/docs/reference/generated/kubernetes-api/{{< param "version" >}}/fonts/fontawesome-webfont.ttf
|
||||
static/docs/reference/generated/kubernetes-api/{{< param "version" >}}/fonts/fontawesome-webfont.woff
|
||||
static/docs/reference/generated/kubernetes-api/{{< param "version" >}}/fonts/fontawesome-webfont.woff2
|
||||
```
|
||||
|
||||
Run `git add` and `git commit` to commit the files.
|
||||
|
||||
@@ -54,5 +54,8 @@ was wrong, you (and only you, the submitter) can change it.
|
||||
|
||||
Limit pull requests to one language per PR. If you need to make an identical change to the same code sample in multiple languages, open a separate PR for each language.
|
||||
|
||||
## Tools for contributors
|
||||
|
||||
The [doc contributors tools](https://github.com/kubernetes/website/tree/master/content/en/docs/doc-contributor-tools) directory in the `kubernetes/website` repository contains tools to help your contribution journey go more smoothly.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
@@ -32,9 +32,14 @@ project](https://github.com/kubernetes/kubernetes).
|
||||
|
||||
## What's allowed
|
||||
|
||||
Kubernetes docs permit only some kinds of content.
|
||||
Kubernetes docs allow content for third-party projects only when:
|
||||
|
||||
- Content documents software in the Kubernetes project
|
||||
- Content documents software that's out of project but necessary for Kubernetes to function
|
||||
- Content is canonical on kubernetes.io, or links to canonical content elsewhere
|
||||
|
||||
### Third party content
|
||||
|
||||
Kubernetes documentation includes applied examples of projects in the Kubernetes project—projects that live in the [kubernetes](https://github.com/kubernetes) and
|
||||
[kubernetes-sigs](https://github.com/kubernetes-sigs) GitHub organizations.
|
||||
|
||||
@@ -43,7 +48,7 @@ Links to active content in the Kubernetes project are always allowed.
|
||||
Kubernetes requires some third party content to function. Examples include container runtimes (containerd, CRI-O, Docker),
|
||||
[networking policy](/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins/) (CNI plugins), [Ingress controllers](https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/), and [logging](https://kubernetes.io/docs/concepts/cluster-administration/logging/).
|
||||
|
||||
Docs can link to third-party open source software (OSS) outside the Kubernetes project if it's necessary for Kubernetes to function.
|
||||
Docs can link to third-party open source software (OSS) outside the Kubernetes project only if it's necessary for Kubernetes to function.
|
||||
|
||||
### Dual sourced content
|
||||
|
||||
|
||||
@@ -0,0 +1,76 @@
|
||||
# Internal link checking tool
|
||||
|
||||
You can use [htmltest](https://github.com/wjdp/htmltest) to check for broken links in [`/content/en/`](https://git.k8s.io/website/content/en/). This is useful when refactoring sections of content, moving pages around, or renaming files or page headers.
|
||||
|
||||
## How the tool works
|
||||
|
||||
`htmltest` scans links in the generated HTML files of the kubernetes website repository. It runs using a `make` command which does the following:
|
||||
|
||||
- Builds the site and generates output HTML in the `/public` directory of your local `kubernetes/website` repository
|
||||
- Pulls the `wdjp/htmltest` Docker image
|
||||
- Mounts your local `kubernetes/website` repository to the Docker image
|
||||
- Scans the files generated in the `/public` directory and provides command line output when it encounters broken internal links
|
||||
|
||||
## What it does and doesn't check
|
||||
|
||||
The link checker scans generated HTML files, not raw Markdown. The htmltest tool depends on a configuration file, [`.htmltest.yml`](https://git.k8s.io/website/.htmltest.yml), to determine which content to examine.
|
||||
|
||||
The link checker scans the following:
|
||||
|
||||
- All content generated from Markdown in [`/content/en/docs`](https://git.k8s.io/website/content/en/docs/) directory, excluding:
|
||||
- Generated API references, for example https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.18/
|
||||
- All internal links, excluding:
|
||||
- Empty hashes (`<a href="#">` or `[title](#)`) and empty hrefs (`<a href="">` or `[title]()`)
|
||||
- Internal links to images and other media files
|
||||
|
||||
The link checker does not scan the following:
|
||||
|
||||
- Links included in the top and side nav bars, footer links, or links in a page's `<head>` section, such as links to CSS stylesheets, scripts, and meta information
|
||||
- Top level pages and their children, for example: `/training`, `/community`, `/case-studies/adidas`
|
||||
- Blog posts
|
||||
- API Reference documentation, for example: https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.18/
|
||||
- Localizations
|
||||
|
||||
## Prerequisites and installation
|
||||
|
||||
You must install
|
||||
* [Docker](https://docs.docker.com/get-docker/)
|
||||
* [make](https://www.gnu.org/software/make/)
|
||||
|
||||
## Running the link checker
|
||||
|
||||
To run the link checker:
|
||||
|
||||
1. Navigate to the root directory of your local `kubernetes/website` repository.
|
||||
|
||||
2. Run the following command:
|
||||
|
||||
```
|
||||
make docker-internal-linkcheck
|
||||
```
|
||||
|
||||
## Understanding the output
|
||||
|
||||
If the link checker finds broken links, the output is similar to the following:
|
||||
|
||||
```
|
||||
tasks/access-kubernetes-api/custom-resources/index.html
|
||||
hash does not exist --- tasks/access-kubernetes-api/custom-resources/index.html --> #preserving-unknown-fields
|
||||
hash does not exist --- tasks/access-kubernetes-api/custom-resources/index.html --> #preserving-unknown-fields
|
||||
```
|
||||
|
||||
This is one set of broken links. The log adds an output for each page with broken links.
|
||||
|
||||
In this output, the file with broken links is `tasks/access-kubernetes-api/custom-resources.md`.
|
||||
|
||||
The tool gives a reason: `hash does not exist`. In most cases, you can ignore this.
|
||||
|
||||
The target URL is `#preserving-unknown-fields`.
|
||||
|
||||
One way to fix this is to:
|
||||
|
||||
1. Navigate to the Markdown file with a broken link.
|
||||
2. Using a text editor, do a full-text search (usually Ctrl+F or Command+F) for the broken link's URL, `#preserving-unknown-fields`.
|
||||
3. Fix the link. For a broken page hash (or _anchor_) link, check whether the topic was renamed or removed.
|
||||
|
||||
Run htmltest to verify that broken links are fixed.
|
||||
@@ -15,7 +15,7 @@ menu:
|
||||
title: "Documentation"
|
||||
weight: 20
|
||||
post: >
|
||||
<p>Learn how to use Kubernetes with conceptual, tutorial, and reference documentation. You can even <a href="/editdocs/" data-auto-burger-exclude>help contribute to the docs</a>!</p>
|
||||
<p>Learn how to use Kubernetes with conceptual, tutorial, and reference documentation. You can even <a href="/editdocs/" data-auto-burger-exclude data-proofer-ignore>help contribute to the docs</a>!</p>
|
||||
description: >
|
||||
Kubernetes is an open source container orchestration engine for automating deployment, scaling, and management of containerized applications. The open source project is hosted by the Cloud Native Computing Foundation.
|
||||
overview: >
|
||||
@@ -38,7 +38,7 @@ cards:
|
||||
button_path: "/docs/setup"
|
||||
- name: tasks
|
||||
title: "Learn how to use Kubernetes"
|
||||
description: "Look up common tasks and how to perform them using a short sequence of steps."
|
||||
description: "Look up common tasks and how to perform them using a short sequence of steps."
|
||||
button: "View Tasks"
|
||||
button_path: "/docs/tasks"
|
||||
- name: training
|
||||
@@ -62,4 +62,4 @@ cards:
|
||||
- name: about
|
||||
title: About the documentation
|
||||
description: This website contains documentation for the current and previous 4 versions of Kubernetes.
|
||||
---
|
||||
---
|
||||
@@ -18,6 +18,7 @@ This page describes how to build, configure, use, and monitor admission webhooks
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture body %}}
|
||||
|
||||
## What are admission webhooks?
|
||||
|
||||
Admission webhooks are HTTP callbacks that receive admission requests and do
|
||||
@@ -549,7 +550,7 @@ Example of a minimal response from a webhook to forbid a request:
|
||||
|
||||
When rejecting a request, the webhook can customize the http code and message returned to the user using the `status` field.
|
||||
The specified status object is returned to the user.
|
||||
See the [API documentation](/docs/reference/generated/kubernetes-api/v1.14/#status-v1-meta) for details about the status type.
|
||||
See the [API documentation](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#status-v1-meta) for details about the status type.
|
||||
Example of a response to forbid a request, customizing the HTTP status code and message presented to the user:
|
||||
{{< tabs name="AdmissionReview_response_forbid_details" >}}
|
||||
{{% tab name="admission.k8s.io/v1" %}}
|
||||
|
||||
@@ -5,7 +5,7 @@ reviewers:
|
||||
- liggitt
|
||||
title: Using RBAC Authorization
|
||||
content_template: templates/concept
|
||||
aliases: [../../../rbac/]
|
||||
aliases: [/rbac/]
|
||||
weight: 70
|
||||
---
|
||||
|
||||
@@ -1210,4 +1210,4 @@ kubectl create clusterrolebinding permissive-binding \
|
||||
After you have transitioned to use RBAC, you should adjust the access controls
|
||||
for your cluster to ensure that these meet your information security needs.
|
||||
|
||||
{{% /capture %}}
|
||||
{{% /capture %}}
|
||||
@@ -105,6 +105,7 @@ different Kubernetes components.
|
||||
| `HugePageStorageMediumSize` | `false` | Alpha | 1.18 | |
|
||||
| `HyperVContainer` | `false` | Alpha | 1.10 | |
|
||||
| `ImmutableEphemeralVolumes` | `false` | Alpha | 1.18 | |
|
||||
| `IPv6DualStack` | `false` | Alpha | 1.16 | |
|
||||
| `KubeletPodResources` | `false` | Alpha | 1.13 | 1.14 |
|
||||
| `KubeletPodResources` | `true` | Beta | 1.15 | |
|
||||
| `LegacyNodeRoleBehavior` | `true` | Alpha | 1.16 | |
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
title: Container Environment Variables
|
||||
id: container-env-variables
|
||||
date: 2018-04-12
|
||||
full_link: /docs/concepts/containers/container-environment-variables/
|
||||
full_link: /docs/concepts/containers/container-environment/
|
||||
short_description: >
|
||||
Container environment variables are name=value pairs that provide useful information into containers running in a Pod.
|
||||
|
||||
|
||||
@@ -29,7 +29,7 @@ To make a report, submit your vulnerability to the [Kubernetes bug bounty progra
|
||||
|
||||
You can also email the private [security@kubernetes.io](mailto:security@kubernetes.io) list with the security details and the details expected for [all Kubernetes bug reports](https://git.k8s.io/kubernetes/.github/ISSUE_TEMPLATE/bug-report.md).
|
||||
|
||||
You may encrypt your email to this list using the GPG keys of the [Product Security Committee members](https://git.k8s.io/security/security-release-process.md#product-security-committee-psc). Encryption using GPG is NOT required to make a disclosure.
|
||||
You may encrypt your email to this list using the GPG keys of the [Product Security Committee members](https://git.k8s.io/security/README.md#product-security-committee-psc). Encryption using GPG is NOT required to make a disclosure.
|
||||
|
||||
### When Should I Report a Vulnerability?
|
||||
|
||||
|
||||
@@ -32,11 +32,11 @@ where `command`, `TYPE`, `NAME`, and `flags` are:
|
||||
|
||||
* `TYPE`: Specifies the [resource type](#resource-types). Resource types are case-insensitive and you can specify the singular, plural, or abbreviated forms. For example, the following commands produce the same output:
|
||||
|
||||
```shell
|
||||
kubectl get pod pod1
|
||||
kubectl get pods pod1
|
||||
kubectl get po pod1
|
||||
```
|
||||
```shell
|
||||
kubectl get pod pod1
|
||||
kubectl get pods pod1
|
||||
kubectl get po pod1
|
||||
```
|
||||
|
||||
* `NAME`: Specifies the name of the resource. Names are case-sensitive. If the name is omitted, details for all resources are displayed, for example `kubectl get pods`.
|
||||
|
||||
@@ -424,7 +424,7 @@ kubectl hello
|
||||
hello world
|
||||
```
|
||||
|
||||
```
|
||||
```shell
|
||||
# we can "uninstall" a plugin, by simply removing it from our PATH
|
||||
sudo rm /usr/local/bin/kubectl-hello
|
||||
```
|
||||
@@ -442,7 +442,7 @@ The following kubectl-compatible plugins are available:
|
||||
/usr/local/bin/kubectl-foo
|
||||
/usr/local/bin/kubectl-bar
|
||||
```
|
||||
```
|
||||
```shell
|
||||
# this command can also warn us about plugins that are
|
||||
# not executable, or that are overshadowed by other
|
||||
# plugins, for example
|
||||
|
||||
@@ -357,7 +357,7 @@ The modifying verbs (`POST`, `PUT`, `PATCH`, and `DELETE`) can accept requests i
|
||||
|
||||
Dry-run is triggered by setting the `dryRun` query parameter. This parameter is a string, working as an enum, and the only accepted values are:
|
||||
|
||||
* `All`: Every stage runs as normal, except for the final storage stage. Admission controllers are run to check that the request is valid, mutating controllers mutate the request, merge is performed on `PATCH`, fields are defaulted, and schema validation occurs. The changes are not persisted to the underlying storage, but the final object which would have been persisted is still returned to the user, along with the normal status code. If the request would trigger an admission controller which would have side effects, the request will be failed rather than risk an unwanted side effect. All built in admission control plugins support dry-run. Additionally, admission webhooks can declare in their [configuration object](/docs/reference/generated/kubernetes-api/v1.13/#webhook-v1beta1-admissionregistration-k8s-io) that they do not have side effects by setting the sideEffects field to "None". If a webhook actually does have side effects, then the sideEffects field should be set to "NoneOnDryRun", and the webhook should also be modified to understand the `DryRun` field in AdmissionReview, and prevent side effects on dry-run requests.
|
||||
* `All`: Every stage runs as normal, except for the final storage stage. Admission controllers are run to check that the request is valid, mutating controllers mutate the request, merge is performed on `PATCH`, fields are defaulted, and schema validation occurs. The changes are not persisted to the underlying storage, but the final object which would have been persisted is still returned to the user, along with the normal status code. If the request would trigger an admission controller which would have side effects, the request will be failed rather than risk an unwanted side effect. All built in admission control plugins support dry-run. Additionally, admission webhooks can declare in their [configuration object](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#webhook-v1beta1-admissionregistration-k8s-io) that they do not have side effects by setting the sideEffects field to "None". If a webhook actually does have side effects, then the sideEffects field should be set to "NoneOnDryRun", and the webhook should also be modified to understand the `DryRun` field in AdmissionReview, and prevent side effects on dry-run requests.
|
||||
* Leave the value empty, which is also the default: Keep the default modifying behavior.
|
||||
|
||||
For example:
|
||||
@@ -450,7 +450,7 @@ request (if not forced, see [Conflicts](#conflicts)).
|
||||
|
||||
Field management is stored in a newly introduced `managedFields` field that is
|
||||
part of an object's
|
||||
[`metadata`](/docs/reference/generated/kubernetes-api/v1.16/#objectmeta-v1-meta).
|
||||
[`metadata`](/docs/reference/generated/kubernetes-api/{{< latest-version >}}/#objectmeta-v1-meta).
|
||||
|
||||
A simple example of an object created by Server Side Apply could look like this:
|
||||
|
||||
@@ -490,7 +490,7 @@ Nevertheless it is possible to change `metadata.managedFields` through an
|
||||
option to try if, for example, the `managedFields` get into an inconsistent
|
||||
state (which clearly should not happen).
|
||||
|
||||
The format of the `managedFields` is described in the [API](/docs/reference/generated/kubernetes-api/v1.16/#fieldsv1-v1-meta).
|
||||
The format of the `managedFields` is described in the [API](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#fieldsv1-v1-meta).
|
||||
|
||||
### Conflicts
|
||||
|
||||
@@ -706,9 +706,9 @@ Resource versions are strings that identify the server's internal version of an
|
||||
|
||||
Clients find resource versions in resources, including the resources in watch events, and list responses returned from the server:
|
||||
|
||||
[v1.meta/ObjectMeta](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.16/#objectmeta-v1-meta) - The `metadata.resourceVersion` of a resource instance identifies the resource version the instance was last modified at.
|
||||
[v1.meta/ObjectMeta](https://kubernetes.io/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#objectmeta-v1-meta) - The `metadata.resourceVersion` of a resource instance identifies the resource version the instance was last modified at.
|
||||
|
||||
[v1.meta/ListMeta](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.16/#listmeta-v1-meta) - The `metadata.resourceVersion` of a resource collection (i.e. a list response) identifies the resource version at which the list response was constructed.
|
||||
[v1.meta/ListMeta](https://kubernetes.io/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#listmeta-v1-meta) - The `metadata.resourceVersion` of a resource collection (i.e. a list response) identifies the resource version at which the list response was constructed.
|
||||
|
||||
### The ResourceVersion Parameter
|
||||
|
||||
|
||||
@@ -49,6 +49,7 @@ their authors, not the Kubernetes team.
|
||||
| Go | [github.com/ericchiang/k8s](https://github.com/ericchiang/k8s) |
|
||||
| Java (OSGi) | [bitbucket.org/amdatulabs/amdatu-kubernetes](https://bitbucket.org/amdatulabs/amdatu-kubernetes) |
|
||||
| Java (Fabric8, OSGi) | [github.com/fabric8io/kubernetes-client](https://github.com/fabric8io/kubernetes-client) |
|
||||
| Java | [github.com/manusa/yakc](https://github.com/manusa/yakc) |
|
||||
| Lisp | [github.com/brendandburns/cl-k8s](https://github.com/brendandburns/cl-k8s) |
|
||||
| Lisp | [github.com/xh4/cube](https://github.com/xh4/cube) |
|
||||
| Node.js (TypeScript) | [github.com/Goyoo/node-k8s-client](https://github.com/Goyoo/node-k8s-client) |
|
||||
|
||||
@@ -1,101 +0,0 @@
|
||||
---
|
||||
title: Installing Kubernetes with KRIB
|
||||
krib-version: 2.4
|
||||
author: Rob Hirschfeld (zehicle)
|
||||
weight: 20
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
This guide helps to install a Kubernetes cluster hosted on bare metal with [Digital Rebar Provision](https://github.com/digitalrebar/provision) using only its Content packages and *kubeadm*.
|
||||
|
||||
Digital Rebar Provision (DRP) is an integrated Golang DHCP, bare metal provisioning (PXE/iPXE) and workflow automation platform. While [DRP can be used to invoke](https://provision.readthedocs.io/en/tip/doc/integrations/ansible.html) [kubespray](/docs/setup/custom-cloud/kubespray), it also offers a self-contained Kubernetes installation known as [KRIB (Kubernetes Rebar Integrated Bootstrap)](https://github.com/digitalrebar/provision-content/tree/master/krib).
|
||||
|
||||
{{< note >}}
|
||||
KRIB is not a _stand-alone_ installer: Digital Rebar templates drive a standard *[kubeadm](/docs/admin/kubeadm/)* configuration that manages the Kubernetes installation with the [Digital Rebar cluster pattern](https://provision.readthedocs.io/en/tip/doc/arch/cluster.html#rs-cluster-pattern) to elect leaders _without external supervision_.
|
||||
{{< /note >}}
|
||||
|
||||
|
||||
KRIB features:
|
||||
|
||||
* zero-touch, self-configuring cluster without pre-configuration or inventory
|
||||
* very fast, no-ssh required automation
|
||||
* bare metal, on-premises focused platform
|
||||
* highly available cluster options (including splitting etcd from the controllers)
|
||||
* dynamic generation of a TLS infrastructure
|
||||
* composable attributes and automatic detection of hardware by profile
|
||||
* options for persistent, immutable and image-based deployments
|
||||
* support for Ubuntu 18.04, CentOS/RHEL 7, CoreOS, RancherOS and others
|
||||
|
||||
## Creating a cluster
|
||||
|
||||
Review [Digital Rebar documentation](https://provision.readthedocs.io/en/tip/README.html) for details about installing the platform.
|
||||
|
||||
The Digital Rebar Provision Golang binary should be installed on a Linux-like system with 16 GB of RAM or larger (Packet.net Tiny and Rasberry Pi are also acceptable).
|
||||
|
||||
### (1/5) Discover servers
|
||||
|
||||
Following the [Digital Rebar installation](https://provision.readthedocs.io/en/tip/doc/quickstart.html), allow one or more servers to boot through the _Sledgehammer_ discovery process to register with the API. This will automatically install the Digital Rebar runner and to allow for next steps.
|
||||
|
||||
### (2/5) Install KRIB Content and Certificate Plugin
|
||||
|
||||
Upload the KRIB Content bundle (or build from [source](https://github.com/digitalrebar/provision-content/tree/master/krib)) and the Cert Plugin for your DRP platform. Both are freely available via the [RackN UX](https://portal.rackn.io) or using the upload from catalog feature of the DRPCLI (shown below).
|
||||
|
||||
```
|
||||
drpcli plugin_providers upload certs from catalog:certs-stable
|
||||
drpcli contents upload catalog:krib-stable
|
||||
```
|
||||
|
||||
### (3/5) Start your cluster deployment
|
||||
|
||||
{{< note >}}
|
||||
KRIB documentation is dynamically generated from the source and will be more up to date than this guide.
|
||||
{{< /note >}}
|
||||
|
||||
Following the [KRIB documentation](https://provision.readthedocs.io/en/tip/doc/content-packages/krib.html), create a Profile for your cluster and assign your target servers into the cluster Profile. The Profile must set `krib\cluster-name` and `etcd\cluster-name` Params to be the name of the Profile. Cluster configuration choices can be made by adding additional Params to the Profile; however, safe defaults are provided for all Params.
|
||||
|
||||
Once all target servers are assigned to the cluster Profile, start a KRIB installation Workflow by assigning one of the included Workflows to all cluster servers. For example, selecting `krib-live-cluster` will perform an immutable deployment into the Sledgehammer discovery operating system. You may use one of the pre-created read-only Workflows or choose to build your own custom variation.
|
||||
|
||||
For basic installs, no further action is required. Advanced users may choose to assign the controllers, etcd servers or other configuration values in the relevant Params.
|
||||
|
||||
### (4/5) Monitor your cluster deployment
|
||||
|
||||
Digital Rebar Provision provides detailed logging and live updates during the installation process. Workflow events are available via a websocket connection or monitoring the Jobs list.
|
||||
|
||||
During the installation, KRIB writes cluster configuration data back into the cluster Profile.
|
||||
|
||||
### (5/5) Access your cluster
|
||||
|
||||
The cluster is available for access via *kubectl* once the `krib/cluster-admin-conf` Param has been set. This Param contains the `kubeconfig` information necessary to access the cluster.
|
||||
|
||||
For example, if you named the cluster Profile `krib` then the following commands would allow you to connect to the installed cluster from your local terminal.
|
||||
|
||||
::
|
||||
|
||||
drpcli profiles get krib params krib/cluster-admin-conf > admin.conf
|
||||
export KUBECONFIG=admin.conf
|
||||
kubectl get nodes
|
||||
|
||||
|
||||
The installation continues after the `krib/cluster-admin-conf` is set to install the Kubernetes UI and Helm. You may interact with the cluster as soon as the `admin.conf` file is available.
|
||||
|
||||
## Cluster operations
|
||||
|
||||
KRIB provides additional Workflows to manage your cluster. Please see the [KRIB documentation](https://provision.readthedocs.io/en/tip/doc/content-packages/krib.html) for an updated list of advanced cluster operations.
|
||||
|
||||
### Scale your cluster
|
||||
|
||||
You can add servers into your cluster by adding the cluster Profile to the server and running the appropriate Workflow.
|
||||
|
||||
### Cleanup your cluster (for developers)
|
||||
|
||||
You can reset your cluster and wipe out all configuration and TLS certificates using the `krib-reset-cluster` Workflow on any of the servers in the cluster.
|
||||
|
||||
{{< caution >}}
|
||||
When running the reset Workflow, be sure not to accidentally target your production cluster!
|
||||
{{< /caution >}}
|
||||
|
||||
## Feedback
|
||||
|
||||
* Slack Channel: [#community](https://rackn.slack.com/messages/community/)
|
||||
* [GitHub Issues](https://github.com/digitalrebar/provision/issues)
|
||||
@@ -17,6 +17,11 @@ You can set up an HA cluster:
|
||||
|
||||
You should carefully consider the advantages and disadvantages of each topology before setting up an HA cluster.
|
||||
|
||||
{{< note >}}
|
||||
kubeadm bootstraps the etcd cluster statically. Read the etcd [Clustering Guide](https://github.com/etcd-io/etcd/blob/release-3.4/Documentation/op-guide/clustering.md#static)
|
||||
for more details.
|
||||
{{< /note >}}
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture body %}}
|
||||
|
||||
@@ -77,11 +77,12 @@ option. Your cluster requirements may need a different configuration.
|
||||
on the apiserver port. It must also allow incoming traffic on its
|
||||
listening port.
|
||||
|
||||
- [HAProxy](http://www.haproxy.org/) can be used as a load balancer.
|
||||
|
||||
- Make sure the address of the load balancer always matches
|
||||
the address of kubeadm's `ControlPlaneEndpoint`.
|
||||
|
||||
- Read the [Options for Software Load Balancing](https://github.com/kubernetes/kubeadm/blob/master/docs/ha-considerations.md#options-for-software-load-balancing)
|
||||
guide for more details.
|
||||
|
||||
1. Add the first control plane nodes to the load balancer and test the
|
||||
connection:
|
||||
|
||||
|
||||
@@ -1,340 +0,0 @@
|
||||
---
|
||||
title: Running Kubernetes on CenturyLink Cloud
|
||||
---
|
||||
|
||||
|
||||
These scripts handle the creation, deletion and expansion of Kubernetes clusters on CenturyLink Cloud.
|
||||
|
||||
You can accomplish all these tasks with a single command. We have made the Ansible playbooks used to perform these tasks available [here](https://github.com/CenturyLinkCloud/adm-kubernetes-on-clc/blob/master/ansible/README.md).
|
||||
|
||||
## Find Help
|
||||
|
||||
If you run into any problems or want help with anything, we are here to help. Reach out to use via any of the following ways:
|
||||
|
||||
- Submit a github issue
|
||||
- Send an email to Kubernetes AT ctl DOT io
|
||||
- Visit [http://info.ctl.io/kubernetes](http://info.ctl.io/kubernetes)
|
||||
|
||||
## Clusters of VMs or Physical Servers, your choice.
|
||||
|
||||
- We support Kubernetes clusters on both Virtual Machines or Physical Servers. If you want to use physical servers for the worker nodes (minions), simple use the --minion_type=bareMetal flag.
|
||||
- For more information on physical servers, visit: [https://www.ctl.io/bare-metal/](https://www.ctl.io/bare-metal/)
|
||||
- Physical serves are only available in the VA1 and GB3 data centers.
|
||||
- VMs are available in all 13 of our public cloud locations
|
||||
|
||||
## Requirements
|
||||
|
||||
The requirements to run this script are:
|
||||
|
||||
- A linux administrative host (tested on ubuntu and macOS)
|
||||
- python 2 (tested on 2.7.11)
|
||||
- pip (installed with python as of 2.7.9)
|
||||
- git
|
||||
- A CenturyLink Cloud account with rights to create new hosts
|
||||
- An active VPN connection to the CenturyLink Cloud from your linux host
|
||||
|
||||
## Script Installation
|
||||
|
||||
After you have all the requirements met, please follow these instructions to install this script.
|
||||
|
||||
1) Clone this repository and cd into it.
|
||||
|
||||
```shell
|
||||
git clone https://github.com/CenturyLinkCloud/adm-kubernetes-on-clc
|
||||
```
|
||||
|
||||
2) Install all requirements, including
|
||||
|
||||
* Ansible
|
||||
* CenturyLink Cloud SDK
|
||||
* Ansible Modules
|
||||
|
||||
```shell
|
||||
sudo pip install -r ansible/requirements.txt
|
||||
```
|
||||
|
||||
3) Create the credentials file from the template and use it to set your ENV variables
|
||||
|
||||
```shell
|
||||
cp ansible/credentials.sh.template ansible/credentials.sh
|
||||
vi ansible/credentials.sh
|
||||
source ansible/credentials.sh
|
||||
|
||||
```
|
||||
|
||||
4) Grant your machine access to the CenturyLink Cloud network by using a VM inside the network or [ configuring a VPN connection to the CenturyLink Cloud network.](https://www.ctl.io/knowledge-base/network/how-to-configure-client-vpn/)
|
||||
|
||||
|
||||
#### Script Installation Example: Ubuntu 14 Walkthrough
|
||||
|
||||
If you use an ubuntu 14, for your convenience we have provided a step by step
|
||||
guide to install the requirements and install the script.
|
||||
|
||||
```shell
|
||||
# system
|
||||
apt-get update
|
||||
apt-get install -y git python python-crypto
|
||||
curl -O https://bootstrap.pypa.io/get-pip.py
|
||||
python get-pip.py
|
||||
|
||||
# installing this repository
|
||||
mkdir -p ~home/k8s-on-clc
|
||||
cd ~home/k8s-on-clc
|
||||
git clone https://github.com/CenturyLinkCloud/adm-kubernetes-on-clc.git
|
||||
cd adm-kubernetes-on-clc/
|
||||
pip install -r requirements.txt
|
||||
|
||||
# getting started
|
||||
cd ansible
|
||||
cp credentials.sh.template credentials.sh; vi credentials.sh
|
||||
source credentials.sh
|
||||
```
|
||||
|
||||
|
||||
|
||||
## Cluster Creation
|
||||
|
||||
To create a new Kubernetes cluster, simply run the ```kube-up.sh``` script. A complete
|
||||
list of script options and some examples are listed below.
|
||||
|
||||
```shell
|
||||
CLC_CLUSTER_NAME=[name of kubernetes cluster]
|
||||
cd ./adm-kubernetes-on-clc
|
||||
bash kube-up.sh -c="$CLC_CLUSTER_NAME"
|
||||
```
|
||||
|
||||
It takes about 15 minutes to create the cluster. Once the script completes, it
|
||||
will output some commands that will help you setup kubectl on your machine to
|
||||
point to the new cluster.
|
||||
|
||||
When the cluster creation is complete, the configuration files for it are stored
|
||||
locally on your administrative host, in the following directory
|
||||
|
||||
```shell
|
||||
> CLC_CLUSTER_HOME=$HOME/.clc_kube/$CLC_CLUSTER_NAME/
|
||||
```
|
||||
|
||||
|
||||
#### Cluster Creation: Script Options
|
||||
|
||||
```shell
|
||||
Usage: kube-up.sh [OPTIONS]
|
||||
Create servers in the CenturyLinkCloud environment and initialize a Kubernetes cluster
|
||||
Environment variables CLC_V2_API_USERNAME and CLC_V2_API_PASSWD must be set in
|
||||
order to access the CenturyLinkCloud API
|
||||
|
||||
All options (both short and long form) require arguments, and must include "="
|
||||
between option name and option value.
|
||||
|
||||
-h (--help) display this help and exit
|
||||
-c= (--clc_cluster_name=) set the name of the cluster, as used in CLC group names
|
||||
-t= (--minion_type=) standard -> VM (default), bareMetal -> physical]
|
||||
-d= (--datacenter=) VA1 (default)
|
||||
-m= (--minion_count=) number of kubernetes minion nodes
|
||||
-mem= (--vm_memory=) number of GB ram for each minion
|
||||
-cpu= (--vm_cpu=) number of virtual cps for each minion node
|
||||
-phyid= (--server_conf_id=) physical server configuration id, one of
|
||||
physical_server_20_core_conf_id
|
||||
physical_server_12_core_conf_id
|
||||
physical_server_4_core_conf_id (default)
|
||||
-etcd_separate_cluster=yes create a separate cluster of three etcd nodes,
|
||||
otherwise run etcd on the master node
|
||||
```
|
||||
|
||||
## Cluster Expansion
|
||||
|
||||
To expand an existing Kubernetes cluster, run the ```add-kube-node.sh```
|
||||
script. A complete list of script options and some examples are listed [below](#cluster-expansion-script-options).
|
||||
This script must be run from the same host that created the cluster (or a host
|
||||
that has the cluster artifact files stored in ```~/.clc_kube/$cluster_name```).
|
||||
|
||||
```shell
|
||||
cd ./adm-kubernetes-on-clc
|
||||
bash add-kube-node.sh -c="name_of_kubernetes_cluster" -m=2
|
||||
```
|
||||
|
||||
#### Cluster Expansion: Script Options
|
||||
|
||||
```shell
|
||||
Usage: add-kube-node.sh [OPTIONS]
|
||||
Create servers in the CenturyLinkCloud environment and add to an
|
||||
existing CLC kubernetes cluster
|
||||
|
||||
Environment variables CLC_V2_API_USERNAME and CLC_V2_API_PASSWD must be set in
|
||||
order to access the CenturyLinkCloud API
|
||||
|
||||
-h (--help) display this help and exit
|
||||
-c= (--clc_cluster_name=) set the name of the cluster, as used in CLC group names
|
||||
-m= (--minion_count=) number of kubernetes minion nodes to add
|
||||
```
|
||||
|
||||
## Cluster Deletion
|
||||
|
||||
There are two ways to delete an existing cluster:
|
||||
|
||||
1) Use our python script:
|
||||
|
||||
```shell
|
||||
python delete_cluster.py --cluster=clc_cluster_name --datacenter=DC1
|
||||
```
|
||||
|
||||
2) Use the CenturyLink Cloud UI. To delete a cluster, log into the CenturyLink
|
||||
Cloud control portal and delete the parent server group that contains the
|
||||
Kubernetes Cluster. We hope to add a scripted option to do this soon.
|
||||
|
||||
## Examples
|
||||
|
||||
Create a cluster with name of k8s_1, 1 master node and 3 worker minions (on physical machines), in VA1
|
||||
|
||||
```shell
|
||||
bash kube-up.sh --clc_cluster_name=k8s_1 --minion_type=bareMetal --minion_count=3 --datacenter=VA1
|
||||
```
|
||||
|
||||
Create a cluster with name of k8s_2, an ha etcd cluster on 3 VMs and 6 worker minions (on VMs), in VA1
|
||||
|
||||
```shell
|
||||
bash kube-up.sh --clc_cluster_name=k8s_2 --minion_type=standard --minion_count=6 --datacenter=VA1 --etcd_separate_cluster=yes
|
||||
```
|
||||
|
||||
Create a cluster with name of k8s_3, 1 master node, and 10 worker minions (on VMs) with higher mem/cpu, in UC1:
|
||||
|
||||
```shell
|
||||
bash kube-up.sh --clc_cluster_name=k8s_3 --minion_type=standard --minion_count=10 --datacenter=VA1 -mem=6 -cpu=4
|
||||
```
|
||||
|
||||
|
||||
|
||||
## Cluster Features and Architecture
|
||||
|
||||
We configure the Kubernetes cluster with the following features:
|
||||
|
||||
* KubeDNS: DNS resolution and service discovery
|
||||
* Heapster/InfluxDB: For metric collection. Needed for Grafana and auto-scaling.
|
||||
* Grafana: Kubernetes/Docker metric dashboard
|
||||
* KubeUI: Simple web interface to view Kubernetes state
|
||||
* Kube Dashboard: New web interface to interact with your cluster
|
||||
|
||||
We use the following to create the Kubernetes cluster:
|
||||
|
||||
* Kubernetes 1.1.7
|
||||
* Ubuntu 14.04
|
||||
* Flannel 0.5.4
|
||||
* Docker 1.9.1-0~trusty
|
||||
* Etcd 2.2.2
|
||||
|
||||
## Optional add-ons
|
||||
|
||||
* Logging: We offer an integrated centralized logging ELK platform so that all
|
||||
Kubernetes and docker logs get sent to the ELK stack. To install the ELK stack
|
||||
and configure Kubernetes to send logs to it, follow [the log
|
||||
aggregation documentation](https://github.com/CenturyLinkCloud/adm-kubernetes-on-clc/blob/master/log_aggregration.md). Note: We don't install this by default as
|
||||
the footprint isn't trivial.
|
||||
|
||||
## Cluster management
|
||||
|
||||
The most widely used tool for managing a Kubernetes cluster is the command-line
|
||||
utility ```kubectl```. If you do not already have a copy of this binary on your
|
||||
administrative machine, you may run the script ```install_kubectl.sh``` which will
|
||||
download it and install it in ```/usr/bin/local```.
|
||||
|
||||
The script requires that the environment variable ```CLC_CLUSTER_NAME``` be defined. ```install_kubectl.sh``` also writes a configuration file which will embed the necessary
|
||||
authentication certificates for the particular cluster. The configuration file is
|
||||
written to the ```${CLC_CLUSTER_HOME}/kube``` directory
|
||||
|
||||
|
||||
```shell
|
||||
export KUBECONFIG=${CLC_CLUSTER_HOME}/kube/config
|
||||
kubectl version
|
||||
kubectl cluster-info
|
||||
```
|
||||
|
||||
### Accessing the cluster programmatically
|
||||
|
||||
It's possible to use the locally stored client certificates to access the apiserver. For example, you may want to use any of the [Kubernetes API client libraries](/docs/reference/using-api/client-libraries/) to program against your Kubernetes cluster in the programming language of your choice.
|
||||
|
||||
To demonstrate how to use these locally stored certificates, we provide the following example of using ```curl``` to communicate to the master apiserver via https:
|
||||
|
||||
```shell
|
||||
curl \
|
||||
--cacert ${CLC_CLUSTER_HOME}/pki/ca.crt \
|
||||
--key ${CLC_CLUSTER_HOME}/pki/kubecfg.key \
|
||||
--cert ${CLC_CLUSTER_HOME}/pki/kubecfg.crt https://${MASTER_IP}:6443
|
||||
```
|
||||
|
||||
But please note, this *does not* work out of the box with the ```curl``` binary
|
||||
distributed with macOS.
|
||||
|
||||
### Accessing the cluster with a browser
|
||||
|
||||
We install [the kubernetes dashboard](/docs/tasks/web-ui-dashboard/). When you
|
||||
create a cluster, the script should output URLs for these interfaces like this:
|
||||
|
||||
kubernetes-dashboard is running at ```https://${MASTER_IP}:6443/api/v1/namespaces/kube-system/services/kubernetes-dashboard/proxy```.
|
||||
|
||||
Note on Authentication to the UIs:
|
||||
|
||||
The cluster is set up to use basic authentication for the user _admin_.
|
||||
Hitting the url at ```https://${MASTER_IP}:6443``` will
|
||||
require accepting the self-signed certificate
|
||||
from the apiserver, and then presenting the admin
|
||||
password written to file at: ```> _${CLC_CLUSTER_HOME}/kube/admin_password.txt_```
|
||||
|
||||
|
||||
### Configuration files
|
||||
|
||||
Various configuration files are written into the home directory *CLC_CLUSTER_HOME* under ```.clc_kube/${CLC_CLUSTER_NAME}``` in several subdirectories. You can use these files
|
||||
to access the cluster from machines other than where you created the cluster from.
|
||||
|
||||
* ```config/```: Ansible variable files containing parameters describing the master and minion hosts
|
||||
* ```hosts/```: hosts files listing access information for the Ansible playbooks
|
||||
* ```kube/```: ```kubectl``` configuration files, and the basic-authentication password for admin access to the Kubernetes API
|
||||
* ```pki/```: public key infrastructure files enabling TLS communication in the cluster
|
||||
* ```ssh/```: SSH keys for root access to the hosts
|
||||
|
||||
|
||||
## ```kubectl``` usage examples
|
||||
|
||||
There are a great many features of _kubectl_. Here are a few examples
|
||||
|
||||
List existing nodes, pods, services and more, in all namespaces, or in just one:
|
||||
|
||||
```shell
|
||||
kubectl get nodes
|
||||
kubectl get --all-namespaces pods
|
||||
kubectl get --all-namespaces services
|
||||
kubectl get --namespace=kube-system replicationcontrollers
|
||||
```
|
||||
|
||||
The Kubernetes API server exposes services on web URLs, which are protected by requiring
|
||||
client certificates. If you run a kubectl proxy locally, ```kubectl``` will provide
|
||||
the necessary certificates and serve locally over http.
|
||||
|
||||
```shell
|
||||
kubectl proxy -p 8001
|
||||
```
|
||||
|
||||
Then, you can access urls like ```http://127.0.0.1:8001/api/v1/namespaces/kube-system/services/kubernetes-dashboard/proxy/``` without the need for client certificates in your browser.
|
||||
|
||||
|
||||
## What Kubernetes features do not work on CenturyLink Cloud
|
||||
|
||||
These are the known items that don't work on CenturyLink cloud but do work on other cloud providers:
|
||||
|
||||
- At this time, there is no support services of the type [LoadBalancer](/docs/tasks/access-application-cluster/create-external-load-balancer/). We are actively working on this and hope to publish the changes sometime around April 2016.
|
||||
|
||||
- At this time, there is no support for persistent storage volumes provided by
|
||||
CenturyLink Cloud. However, customers can bring their own persistent storage
|
||||
offering. We ourselves use Gluster.
|
||||
|
||||
|
||||
## Ansible Files
|
||||
|
||||
If you want more information about our Ansible files, please [read this file](https://github.com/CenturyLinkCloud/adm-kubernetes-on-clc/blob/master/ansible/README.md)
|
||||
|
||||
## Further reading
|
||||
|
||||
Please see the [Kubernetes docs](/docs/) for more details on administering
|
||||
and using a Kubernetes cluster.
|
||||
|
||||
|
||||
|
||||
@@ -66,4 +66,4 @@ You can install IBM Cloud Private on VMware with either Ubuntu or RHEL images. F
|
||||
|
||||
The IBM Cloud Private Hosted service automatically deploys IBM Cloud Private Hosted on your VMware vCenter Server instances. This service brings the power of microservices and containers to your VMware environment on IBM Cloud. With this service, you can extend the same familiar VMware and IBM Cloud Private operational model and tools from on-premises into the IBM Cloud.
|
||||
|
||||
For more information, see [IBM Cloud Private Hosted service](https://cloud.ibm.com/docs/services/vmwaresolutions/vmonic?topic=vmware-solutions-prod_overview#ibm-cloud-private-hosted).
|
||||
For more information, see [IBM Cloud Private Hosted service](https://cloud.ibm.com/docs/vmwaresolutions?topic=vmwaresolutions-icp_overview).
|
||||
|
||||
@@ -269,8 +269,8 @@ public class KubeConfigFileClientExample {
|
||||
CoreV1Api api = new CoreV1Api();
|
||||
|
||||
// invokes the CoreV1Api client
|
||||
V1PodList list = api.listPodForAllNamespaces(null, null, null, null, null, null, null, null);
|
||||
System.out.Println("Listing all pods: ");
|
||||
V1PodList list = api.listPodForAllNamespaces(null, null, null, null, null, null, null, null, null);
|
||||
System.out.println("Listing all pods: ");
|
||||
for (V1Pod item : list.getItems()) {
|
||||
System.out.println(item.getMetadata().getName());
|
||||
}
|
||||
|
||||
@@ -54,8 +54,7 @@ inheriting DNS. Set it to a valid file path to specify a file other than
|
||||
|
||||
## CoreDNS
|
||||
|
||||
CoreDNS is a general-purpose authoritative DNS server that can serve as cluster DNS, complying with the [dns specifications]
|
||||
(https://github.com/kubernetes/dns/blob/master/docs/specification.md).
|
||||
CoreDNS is a general-purpose authoritative DNS server that can serve as cluster DNS, complying with the [dns specifications](https://github.com/kubernetes/dns/blob/master/docs/specification.md).
|
||||
|
||||
### CoreDNS ConfigMap options
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@ reviewers:
|
||||
title: Using NodeLocal DNSCache in Kubernetes clusters
|
||||
content_template: templates/task
|
||||
---
|
||||
|
||||
|
||||
{{% capture overview %}}
|
||||
{{< feature-state for_k8s_version="v1.18" state="stable" >}}
|
||||
This page provides an overview of NodeLocal DNSCache feature in Kubernetes.
|
||||
@@ -27,7 +27,7 @@ NodeLocal DNSCache improves Cluster DNS performance by running a dns caching age
|
||||
|
||||
## Motivation
|
||||
|
||||
* With the current DNS architecture, it is possible that Pods with the highest DNS QPS have to reach out to a different node, if there is no local kube-dns/CoreDNS instance.
|
||||
* With the current DNS architecture, it is possible that Pods with the highest DNS QPS have to reach out to a different node, if there is no local kube-dns/CoreDNS instance.
|
||||
Having a local cache will help improve the latency in such scenarios.
|
||||
|
||||
* Skipping iptables DNAT and connection tracking will help reduce [conntrack races](https://github.com/kubernetes/kubernetes/issues/56903) and avoid UDP DNS entries filling up conntrack table.
|
||||
@@ -45,7 +45,7 @@ Having a local cache will help improve the latency in such scenarios.
|
||||
This is the path followed by DNS Queries after NodeLocal DNSCache is enabled:
|
||||
|
||||
|
||||
{{< figure src="/images/docs/nodelocaldns.jpg" alt="NodeLocal DNSCache flow" title="Nodelocal DNSCache flow" caption="This image shows how NodeLocal DNSCache handles DNS queries." >}}
|
||||
{{< figure src="/images/docs/nodelocaldns.svg" alt="NodeLocal DNSCache flow" title="Nodelocal DNSCache flow" caption="This image shows how NodeLocal DNSCache handles DNS queries." >}}
|
||||
|
||||
## Configuration
|
||||
{{< note >}} The local listen IP address for NodeLocal DNSCache can be any IP in the 169.254.20.0/16 space or any other IP address that can be guaranteed to not collide with any existing IP. This document uses 169.254.20.10 as an example.
|
||||
@@ -54,33 +54,33 @@ This is the path followed by DNS Queries after NodeLocal DNSCache is enabled:
|
||||
This feature can be enabled using the following steps:
|
||||
|
||||
* Prepare a manifest similar to the sample [`nodelocaldns.yaml`](https://github.com/kubernetes/kubernetes/blob/master/cluster/addons/dns/nodelocaldns/nodelocaldns.yaml) and save it as `nodelocaldns.yaml.`
|
||||
* Substitute the variables in the manifest with the right values:
|
||||
* Substitute the variables in the manifest with the right values:
|
||||
|
||||
* kubedns=`kubectl get svc kube-dns -n kube-system -o jsonpath={.spec.clusterIP}`
|
||||
|
||||
* domain=`<cluster-domain>`
|
||||
|
||||
* kubedns=`kubectl get svc kube-dns -n kube-system -o jsonpath={.spec.clusterIP}`
|
||||
|
||||
* domain=`<cluster-domain>`
|
||||
|
||||
* localdns=`<node-local-address>`
|
||||
|
||||
|
||||
`<cluster-domain>` is "cluster.local" by default. `<node-local-address>` is the local listen IP address chosen for NodeLocal DNSCache.
|
||||
|
||||
* If kube-proxy is running in IPTABLES mode:
|
||||
|
||||
* If kube-proxy is running in IPTABLES mode:
|
||||
|
||||
``` bash
|
||||
sed -i "s/__PILLAR__LOCAL__DNS__/$localdns/g; s/__PILLAR__DNS__DOMAIN__/$domain/g; s/__PILLAR__DNS__SERVER__/$kubedns/g" nodelocaldns.yaml
|
||||
```
|
||||
|
||||
`__PILLAR__CLUSTER__DNS__` and `__PILLAR__UPSTREAM__SERVERS__` will be populated by the node-local-dns pods.
|
||||
|
||||
`__PILLAR__CLUSTER__DNS__` and `__PILLAR__UPSTREAM__SERVERS__` will be populated by the node-local-dns pods.
|
||||
In this mode, node-local-dns pods listen on both the kube-dns service IP as well as `<node-local-address>`, so pods can lookup DNS records using either IP address.
|
||||
|
||||
* If kube-proxy is running in IPVS mode:
|
||||
|
||||
* If kube-proxy is running in IPVS mode:
|
||||
|
||||
``` bash
|
||||
sed -i "s/__PILLAR__LOCAL__DNS__/$localdns/g; s/__PILLAR__DNS__DOMAIN__/$domain/g; s/__PILLAR__DNS__SERVER__//g; s/__PILLAR__CLUSTER__DNS__/$kubedns/g" nodelocaldns.yaml
|
||||
```
|
||||
In this mode, node-local-dns pods listen only on `<node-local-address>`. The node-local-dns interface cannot bind the kube-dns cluster IP since the interface used for IPVS loadbalancing already uses this address.
|
||||
In this mode, node-local-dns pods listen only on `<node-local-address>`. The node-local-dns interface cannot bind the kube-dns cluster IP since the interface used for IPVS loadbalancing already uses this address.
|
||||
`__PILLAR__UPSTREAM__SERVERS__` will be populated by the node-local-dns pods.
|
||||
|
||||
|
||||
* Run `kubectl create -f nodelocaldns.yaml`
|
||||
* If using kube-proxy in IPVS mode, `--cluster-dns` flag to kubelet needs to be modified to use `<node-local-address>` that NodeLocal DNSCache is listening on.
|
||||
Otherwise, there is no need to modify the value of the `--cluster-dns` flag, since NodeLocal DNSCache listens on both the kube-dns service IP as well as `<node-local-address>`.
|
||||
|
||||
@@ -119,6 +119,7 @@ request to `/apis/batch/v1/namespaces/some-namespace/jobs/some-job-name`.
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
{{< /note >}}
|
||||
|
||||
### Log backend
|
||||
@@ -234,7 +235,7 @@ spec:
|
||||
url: "https://audit.app"
|
||||
```
|
||||
|
||||
For the complete API definition, see [AuditSink](/docs/reference/generated/kubernetes-api/v1.13/#auditsink-v1alpha1-auditregistration). Multiple objects will exist as independent solutions.
|
||||
For the complete API definition, see [AuditSink](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#auditsink-v1alpha1-auditregistration). Multiple objects will exist as independent solutions.
|
||||
The name of an AuditSink object must be a valid
|
||||
[DNS subdomain name](/docs/concepts/overview/working-with-objects/names#dns-subdomain-names).
|
||||
|
||||
@@ -339,12 +340,12 @@ audit policies.
|
||||
[Fluentd][fluentd] is an open source data collector for unified logging layer.
|
||||
In this example, we will use fluentd to split audit events by different namespaces.
|
||||
|
||||
1. install [fluentd][fluentd_install_doc], fluent-plugin-forest and fluent-plugin-rewrite-tag-filter in the kube-apiserver node
|
||||
{{< note >}}
|
||||
Fluent-plugin-forest and fluent-plugin-rewrite-tag-filter are plugins for fluentd. You can get details about plugin installation from [fluentd plugin-management][fluentd_plugin_management_doc].
|
||||
{{< note >}}Fluent-plugin-forest and fluent-plugin-rewrite-tag-filter are plugins for fluentd. You can get details about plugin installation from [fluentd plugin-management][fluentd_plugin_management_doc].
|
||||
{{< /note >}}
|
||||
|
||||
1. create a config file for fluentd
|
||||
1. Install [fluentd][fluentd_install_doc], fluent-plugin-forest and fluent-plugin-rewrite-tag-filter in the kube-apiserver node
|
||||
|
||||
1. Create a config file for fluentd
|
||||
|
||||
```
|
||||
cat <<'EOF' > /etc/fluentd/config
|
||||
@@ -399,19 +400,19 @@ Fluent-plugin-forest and fluent-plugin-rewrite-tag-filter are plugins for fluent
|
||||
EOF
|
||||
```
|
||||
|
||||
1. start fluentd
|
||||
1. Start fluentd
|
||||
|
||||
```shell
|
||||
fluentd -c /etc/fluentd/config -vv
|
||||
```
|
||||
|
||||
1. start kube-apiserver with the following options:
|
||||
1. Start kube-apiserver with the following options:
|
||||
|
||||
```shell
|
||||
--audit-policy-file=/etc/kubernetes/audit-policy.yaml --audit-log-path=/var/log/kube-audit --audit-log-format=json
|
||||
```
|
||||
|
||||
1. check audits for different namespaces in `/var/log/audit-*.log`
|
||||
1. Check audits for different namespaces in `/var/log/audit-*.log`
|
||||
|
||||
### Use logstash to collect and distribute audit events from webhook backend
|
||||
|
||||
@@ -490,8 +491,7 @@ Note that in addition to file output plugin, logstash has a variety of outputs t
|
||||
let users route data where they want. For example, users can emit audit events to elasticsearch
|
||||
plugin which supports full-text search and analytics.
|
||||
|
||||
|
||||
[kube-apiserver]: /docs/admin/kube-apiserver
|
||||
[kube-apiserver]: /docs/reference/command-line-tools-reference/kube-apiserver/
|
||||
[auditing-proposal]: https://github.com/kubernetes/community/blob/master/contributors/design-proposals/api-machinery/auditing.md
|
||||
[auditing-api]: https://github.com/kubernetes/kubernetes/blob/{{< param "githubbranch" >}}/staging/src/k8s.io/apiserver/pkg/apis/audit/v1/types.go
|
||||
[configure-helper]: https://github.com/kubernetes/kubernetes/blob/{{< param "githubbranch" >}}/cluster/gce/gci/configure-helper.sh
|
||||
|
||||
@@ -61,8 +61,7 @@ kubectl scale deployment hostnames --replicas=3
|
||||
deployment.apps/hostnames scaled
|
||||
```
|
||||
|
||||
{{< note >}}
|
||||
This is the same as if you had started the Deployment with the following
|
||||
Note that this is the same as if you had started the Deployment with the following
|
||||
YAML:
|
||||
|
||||
```yaml
|
||||
@@ -89,7 +88,6 @@ spec:
|
||||
|
||||
The label "app" is automatically set by `kubectl create deployment` to the name of the
|
||||
Deployment.
|
||||
{{< /note >}}
|
||||
|
||||
You can confirm your Pods are running:
|
||||
|
||||
@@ -195,7 +193,6 @@ hostnames ClusterIP 10.0.1.175 <none> 80/TCP 5s
|
||||
|
||||
Now you know that the Service exists.
|
||||
|
||||
{{< note >}}
|
||||
As before, this is the same as if you had started the Service with YAML:
|
||||
|
||||
```yaml
|
||||
@@ -218,7 +215,6 @@ spec:
|
||||
In order to highlight the full range of configuration, the Service you created
|
||||
here uses a different port number than the Pods. For many real-world
|
||||
Services, these values might be the same.
|
||||
{{< /note >}}
|
||||
|
||||
## Does the Service work by DNS name?
|
||||
|
||||
|
||||
@@ -82,6 +82,11 @@ value of `/dev/termination-log`. By customizing this field, you can tell Kuberne
|
||||
to use a different file. Kubernetes use the contents from the specified file to
|
||||
populate the Container's status message on both success and failure.
|
||||
|
||||
The termination message is intended to be brief final status, such as an assertion failure message.
|
||||
The kubelet truncates messages that are longer than 4096 bytes. The total message length across all
|
||||
containers will be limited to 12KiB. The default termination message path is `/dev/termination-log`.
|
||||
You cannot set the termination message path after a Pod is launched
|
||||
|
||||
In the following example, the container writes termination messages to
|
||||
`/tmp/my-log` for Kubernetes to retrieve:
|
||||
|
||||
|
||||
+15
-7
@@ -14,7 +14,7 @@ in a Kubernetes Pod.
|
||||
|
||||
{{% capture prerequisites %}}
|
||||
|
||||
{{< include "task-tutorial-prereqs.md" >}} {{< version-check >}}
|
||||
{{< include "task-tutorial-prereqs.md" >}}
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
@@ -29,12 +29,12 @@ that run in the Pod. To set environment variables, include the `env` or
|
||||
|
||||
In this exercise, you create a Pod that runs one container. The configuration
|
||||
file for the Pod defines an environment variable with name `DEMO_GREETING` and
|
||||
value `"Hello from the environment"`. Here is the configuration file for the
|
||||
value `"Hello from the environment"`. Here is the configuration manifest for the
|
||||
Pod:
|
||||
|
||||
{{< codenew file="pods/inject/envars.yaml" >}}
|
||||
|
||||
1. Create a Pod based on the YAML configuration file:
|
||||
1. Create a Pod based on that manifest:
|
||||
|
||||
```shell
|
||||
kubectl apply -f https://k8s.io/examples/pods/inject/envars.yaml
|
||||
@@ -46,7 +46,7 @@ Pod:
|
||||
kubectl get pods -l purpose=demonstrate-envars
|
||||
```
|
||||
|
||||
The output is similar to this:
|
||||
The output is similar to:
|
||||
|
||||
```
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
@@ -62,7 +62,8 @@ Pod:
|
||||
1. In your shell, run the `printenv` command to list the environment variables.
|
||||
|
||||
```shell
|
||||
root@envar-demo:/# printenv
|
||||
# Run this in the shell inside the container
|
||||
printenv
|
||||
```
|
||||
|
||||
The output is similar to this:
|
||||
@@ -80,12 +81,19 @@ Pod:
|
||||
|
||||
{{< note >}}
|
||||
The environment variables set using the `env` or `envFrom` field
|
||||
will override any environment variables specified in the container image.
|
||||
override any environment variables specified in the container image.
|
||||
{{< /note >}}
|
||||
|
||||
## Using environment variables inside of your config
|
||||
|
||||
Environment variables that you define in a Pod's configuration can be used elsewhere in the configuration, for example in commands and arguments that you set for the Pod's containers. In the example configuration below, the `GREETING`, `HONORIFIC`, and `NAME` environment variables are set to `Warm greetings to`, `The Most Honorable`, and `Kubernetes`, respectively. Those environment variables are then used in the CLI arguments passed to the `env-print-demo` container.
|
||||
Environment variables that you define in a Pod's configuration can be used
|
||||
elsewhere in the configuration, for example in commands and arguments that
|
||||
you set for the Pod's containers.
|
||||
In the example configuration below, the `GREETING`, `HONORIFIC`, and
|
||||
`NAME` environment variables are set to `Warm greetings to`, `The Most
|
||||
Honorable`, and `Kubernetes`, respectively. Those environment variables
|
||||
are then used in the CLI arguments passed to the `env-print-demo`
|
||||
container.
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
|
||||
+1
-1
@@ -2,7 +2,7 @@
|
||||
title: Update API Objects in Place Using kubectl patch
|
||||
description: Use kubectl patch to update Kubernetes API objects in place. Do a strategic merge patch or a JSON merge patch.
|
||||
content_template: templates/task
|
||||
weight: 40
|
||||
weight: 50
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
Reference in New Issue
Block a user