Merge remote-tracking branch 'upstream/main' into dev-1.24
This commit is contained in:
Nate W
@@ -48,7 +48,7 @@ Kubernetes is open source giving you the freedom to take advantage of on-premise
|
||||
<br>
|
||||
<br>
|
||||
<br>
|
||||
<a href="https://events.linuxfoundation.org/kubecon-cloudnativecon-north-america/?utm_source=kubernetes.io&utm_medium=nav&utm_campaign=kccncna21" button id="desktopKCButton">Attend KubeCon North America on October 24-28, 2022</a>
|
||||
<a href="https://events.linuxfoundation.org/kubecon-cloudnativecon-north-america/?utm_source=kubernetes.io&utm_medium=nav&utm_campaign=kccncna22" button id="desktopKCButton">Attend KubeCon North America on October 24-28, 2022</a>
|
||||
</div>
|
||||
<div id="videoPlayer">
|
||||
<iframe data-url="https://www.youtube.com/embed/H06qrNmGqyE?autoplay=1" frameborder="0" allowfullscreen></iframe>
|
||||
|
||||
+5
-2
@@ -67,8 +67,11 @@ As you can see in the above event messages, the affected Pod is not evicted imme
|
||||
|
||||
For our production clusters, we specify a lower time limit so as to avoid the impacted Pods serving traffic abidingly. The *kube-exec-controller* internally sets and tracks a timer for each Pod that matches the associated TTL. Once the timer is up, the controller evicts that Pod using K8s API. The eviction (rather than deletion) is to ensure service availability, since the cluster respects any configured [PodDisruptionBudget](/docs/concepts/workloads/pods/disruptions/) (PDB). Let's say if a user has defined *x* number of Pods as critical in their PDB, the eviction (as requested by *kube-exec-controller*) does not continue when the target workload has fewer than *x* Pods running.
|
||||
|
||||
Here comes a sequence diagram of the entire workflow mentioned above:
|
||||
{{< figure src="workflow-diagram.svg" alt="Workflow Diagram" class="diagram-medium" >}}
|
||||
Here comes a sequence diagram of the entire workflow mentioned above:
|
||||
|
||||
<!-- Mermaid Live Editor link - https://mermaid-js.github.io/mermaid-live-editor/edit/#pako:eNp9kjFPAzEMhf-KlalIbWd0QpUQdGJB3JrFTUyJmjhHzncFof53nGtpqYTYEuu958-Wv4zLnkxjenofiB09BtwWTJbRSS6QCLCHu01ZPdJIMXdUYNZTGYOjRd4zlRvLHRYJLnTIArvbtozV83TbAnZhUcVUrkXo04OU2I6uKu99Cn0fMsNDZik5Rm3SHntYTrRYrabUBl4GBmt2w4acRKAPcrBcLq0Bl1NC9pYnoRouHZopX9RX9aotddJeADaf4DDGwFuQN4IRY_Ao9bunzVvOO13COeYCcR9j3k-OCQDP9KfgC8TJsFbZIHSxnGljzp1lgKs2v9HXugMBwe2WPHTZ94CvottB6Ap5eg2s9cBaUnrLVEP_Yp5ynrOf3fxPV2V1lBOhmZtEJWHweiFfldQa1SWyptGnAuAQxRrLB5UOna6P1j7o4ZhGykBzg4Pk9pPdz_-oOR3ZsXj4BjrP5rU-->
|
||||
|
||||
|
||||
|
||||
## A new kubectl plugin for better user experience
|
||||
Our admission controller component works great for solving the container drift issue we had on the platform. It is also able to submit all related Events to the target Pod that has been affected. However, K8s clusters don't retain Events very long (the default retention period is one hour). We need to provide other ways for developers to get their Pod interaction activity. A [kubectl plugin](/docs/tasks/extend-kubectl/kubectl-plugins/) is a perfect choice for us to expose this information. We named our plugin `kubectl pi` (short for `pod-interaction`) and provide two subcommands: `get` and `extend`.
|
||||
|
||||
@@ -0,0 +1,63 @@
|
||||
---
|
||||
layout: blog
|
||||
title: "Spotlight on SIG Multicluster"
|
||||
date: 2022-02-07
|
||||
slug: sig-multicluster-spotlight-2022
|
||||
canonicalUrl: https://www.kubernetes.dev/blog/2022/02/04/sig-multicluster-spotlight-2022/
|
||||
---
|
||||
|
||||
**Authors:** Dewan Ahmed (Aiven) and Chris Short (AWS)
|
||||
|
||||
## Introduction
|
||||
|
||||
[SIG Multicluster](https://github.com/kubernetes/community/tree/master/sig-multicluster) is the SIG focused on how Kubernetes concepts are expanded and used beyond the cluster boundary. Historically, Kubernetes resources only interacted within that boundary - KRU or Kubernetes Resource Universe (not an actual Kubernetes concept). Kubernetes clusters, even now, don't really know anything about themselves or, about other clusters. Absence of cluster identifiers is a case in point. With the growing adoption of multicloud and multicluster deployments, the work SIG Multicluster doing is gaining a lot of attention. In this blog, [Jeremy Olmsted-Thompson, Google](https://twitter.com/jeremyot) and [Chris Short, AWS](https://twitter.com/ChrisShort) discuss the interesting problems SIG Multicluster is solving and how you can get involved. Their initials **JOT** and **CS** will be used for brevity.
|
||||
|
||||
## A summary of their conversation
|
||||
|
||||
**CS**: How long has the SIG Multicluster existed and how was the SIG in its infancy? How long have you been with this SIG?
|
||||
|
||||
**JOT**: I've been around for almost two years in the SIG Multicluster. All I know about the infancy years is from the lore but even in the early days, it was always about solving this same problem. Early efforts have been things like [KubeFed](https://github.com/kubernetes-sigs/kubefed). I think there are still folks using KubeFed but it's a smaller slice. Back then, I think people out there deploying large numbers of Kubernetes clusters were really not at a point where we had a ton of real concrete use cases. Projects like KubeFed and [Cluster Registry](https://github.com/kubernetes-retired/cluster-registry) were developed around that time and the need back then can be associated to these projects. The motivation for these projects were how do we solve the problems that we think people are **going to have**, when they start expanding to multiple clusters. Honestly, in some ways, it was trying to do too much at that time.
|
||||
|
||||
**CS**: How does KubeFed differ from the current state of SIG Multicluster? How does the **lore** differ from the **now**?
|
||||
|
||||
**JOT**: Yeah, it was like trying to get ahead of potential problems instead of addressing specific problems. I think towards the end of 2019, there was a slow down in SIG multicluster work and we kind of picked it back up with one of the most active recent projects that is the [SIG Multicluster services (MCS)](https://github.com/kubernetes-sigs/mcs-api).
|
||||
|
||||
Now this is the shift to solving real specific problems. For example,
|
||||
|
||||
> I've got workloads that are spread across multiple clusters and I need them to talk to each other.
|
||||
|
||||
Okay, that's very straightforward and we know that we need to solve that. To get started, let's make sure that these projects can work together on a common API so you get the same kind of portability that you get with Kubernetes.
|
||||
|
||||
There's a few implementations of the MCS API out there and more are being developed. But, we didn't build an implementation because depending on how you're deploying things there could be hundreds of implementations. As long as you only need the basic Multicluster service functionality, it'll just work on whatever background you want, whether it's Submariner, GKE, or a service mesh.
|
||||
|
||||
My favorite example of "then vs. now" is cluster ID. A few years ago, there was an effort to define a cluster ID. A lot of really good thought went into this concept, for example, how do we make a cluster ID is unique across multiple clusters. How do we make this ID globally unique so it'll work in every contact? Let's say, there's an acquisition or merger of teams - does the cluster IDs still remain unique for those teams?
|
||||
|
||||
With Multicluster services, we found the need for an actual cluster ID, and it has a very specific need. To address this specific need, we're no longer considering every single Kubernetes cluster out there rather the ClusterSets - a grouping of clusters that work together in some kind of bounds. That's a much narrower scope than considering clusters everywhere in time and space. It also leaves flexibility for an implementer to define the boundary (a ClusterSet) beyond which this cluster ID will no longer be unique.
|
||||
|
||||
|
||||
**CS**: How do you feel about the current state of SIG Multicluster versus where you're hoping to be in future?
|
||||
|
||||
**JOT**: There's a few projects that are kind of getting started, for example, Work API. In the future, I think that some common practices around how do we deploy things across clusters are going to develop.
|
||||
> If I have clusters deployed in a bunch of different regions; what's the best way to actually do that?
|
||||
|
||||
The answer is, almost always, "it depends". Why are you doing this? Is it because there's some kind of compliance that makes you care about locality? Is it performance? Is it availability?
|
||||
|
||||
I think revisiting registry patterns will probably be a natural step after we have cluster IDs, that is, how do you actually associate these clusters together? Maybe you've got a distributed deployment that you run in your own data centers all over the world. I imagine that expanding the API in that space is going to be important as more multi cluster features develop. It really depends on what the community starts doing with these tools.
|
||||
|
||||
**CS**: In the early days of Kubernetes, we used to have a few large Kubernetes clusters and now we're dealing with many small Kubernetes clusters - even multiple clusters for our own dev environments. How has this shift from a few large clusters to many small clusters affected the SIG? Has it accelerated the work or make it challenging in any way?
|
||||
|
||||
**JOT**: I think that it has created a lot of ambiguity that needs solving. Originally, you'd have a dev cluster, a staging cluster, and a prod cluster. When the multi region thing came in, we started needing dev/staging/prod clusters, per region. And then, sometimes clusters really need more isolation due to compliance or some regulations issues. Thus, we're ending up with a lot of clusters. I think figuring out the right balance on how many clusters should you actually have is important. The power of Kubernetes is being able to deploy a lot of things managed by a single control plane. So, it's not like every single workload that gets deployed should be in its own cluster. But I think it's pretty clear that we can't put every single workload in a single cluster.
|
||||
|
||||
**CS**: What are some of your favorite things about this SIG?
|
||||
|
||||
**JOT**: The complexity of the problems, the people and the newness of the space. We don't have right answers and we have to figure this out. At the beginning, we couldn't even think about multi clusters because there was no way to connect services across clusters. Now there is and we're starting to go tackle those problems, I think that this is a really fun place to be in because I expect that the SIG is going to get a lot busier the next couple of years. It's a very collaborative group and we definitely would like more people to come join us, get involved, raise their problems and bring their ideas.
|
||||
|
||||
**CS**: What do you think keeps people in this group? How has the pandemic affected you?
|
||||
|
||||
**JOT**: I think it definitely got a little bit quieter during the pandemic. But for the most part; it's a very distributed group so whether you're calling in to our weekly meetings from a conference room or from your home, it doesn't make that huge of a difference. During the pandemic, a lot of people had time to focus on what's next for their scale and growth. I think that's what keeps people in the group - we have real problems that need to be solved which are very new in this space. And it's fun :)
|
||||
|
||||
## Wrap up
|
||||
|
||||
**CS**: That's all we have for today. Thanks Jeremy for your time.
|
||||
|
||||
**JOT**: Thanks Chris. Everybody is welcome at our [bi-weekly meetings](https://github.com/kubernetes/community/tree/master/sig-multicluster#meetings). We love as many people to come as possible and welcome all questions and all ideas. It's a new space and it'd be great to grow the community.
|
||||
@@ -42,21 +42,21 @@ Fairness feature enabled.
|
||||
## Enabling/Disabling API Priority and Fairness
|
||||
|
||||
The API Priority and Fairness feature is controlled by a feature gate
|
||||
and is enabled by default. See
|
||||
[Feature Gates](/docs/reference/command-line-tools-reference/feature-gates/)
|
||||
and is enabled by default. See [Feature
|
||||
Gates](/docs/reference/command-line-tools-reference/feature-gates/)
|
||||
for a general explanation of feature gates and how to enable and
|
||||
disable them. The name of the feature gate for APF is
|
||||
"APIPriorityAndFairness". This feature also involves an {{<
|
||||
glossary_tooltip term_id="api-group" text="API Group" >}} with: (a) a
|
||||
`v1alpha1` version, disabled by default, and (b) a `v1beta1`
|
||||
version, enabled by default. You can disable the feature
|
||||
gate and API group v1beta1 version by adding the following
|
||||
`v1alpha1` version, disabled by default, and (b) `v1beta1` and
|
||||
`v1beta2` versions, enabled by default. You can disable the feature
|
||||
gate and API group beta versions by adding the following
|
||||
command-line flags to your `kube-apiserver` invocation:
|
||||
|
||||
```shell
|
||||
kube-apiserver \
|
||||
--feature-gates=APIPriorityAndFairness=false \
|
||||
--runtime-config=flowcontrol.apiserver.k8s.io/v1beta1=false \
|
||||
--runtime-config=flowcontrol.apiserver.k8s.io/v1beta1=false,flowcontrol.apiserver.k8s.io/v1beta2=false \
|
||||
# …and other flags as usual
|
||||
```
|
||||
|
||||
@@ -127,86 +127,13 @@ any of the limitations imposed by this feature. These exemptions prevent an
|
||||
improperly-configured flow control configuration from totally disabling an API
|
||||
server.
|
||||
|
||||
## Defaults
|
||||
|
||||
The Priority and Fairness feature ships with a suggested configuration that
|
||||
should suffice for experimentation; if your cluster is likely to
|
||||
experience heavy load then you should consider what configuration will work
|
||||
best. The suggested configuration groups requests into five priority
|
||||
classes:
|
||||
|
||||
* The `system` priority level is for requests from the `system:nodes` group,
|
||||
i.e. Kubelets, which must be able to contact the API server in order for
|
||||
workloads to be able to schedule on them.
|
||||
|
||||
* The `leader-election` priority level is for leader election requests from
|
||||
built-in controllers (in particular, requests for `endpoints`, `configmaps`,
|
||||
or `leases` coming from the `system:kube-controller-manager` or
|
||||
`system:kube-scheduler` users and service accounts in the `kube-system`
|
||||
namespace). These are important to isolate from other traffic because failures
|
||||
in leader election cause their controllers to fail and restart, which in turn
|
||||
causes more expensive traffic as the new controllers sync their informers.
|
||||
|
||||
* The `workload-high` priority level is for other requests from built-in
|
||||
controllers.
|
||||
|
||||
* The `workload-low` priority level is for requests from any other service
|
||||
account, which will typically include all requests from controllers running in
|
||||
Pods.
|
||||
|
||||
* The `global-default` priority level handles all other traffic, e.g.
|
||||
interactive `kubectl` commands run by nonprivileged users.
|
||||
|
||||
Additionally, there are two PriorityLevelConfigurations and two FlowSchemas that
|
||||
are built in and may not be overwritten:
|
||||
|
||||
* The special `exempt` priority level is used for requests that are not subject
|
||||
to flow control at all: they will always be dispatched immediately. The
|
||||
special `exempt` FlowSchema classifies all requests from the `system:masters`
|
||||
group into this priority level. You may define other FlowSchemas that direct
|
||||
other requests to this priority level, if appropriate.
|
||||
|
||||
* The special `catch-all` priority level is used in combination with the special
|
||||
`catch-all` FlowSchema to make sure that every request gets some kind of
|
||||
classification. Typically you should not rely on this catch-all configuration,
|
||||
and should create your own catch-all FlowSchema and PriorityLevelConfiguration
|
||||
(or use the `global-default` configuration that is installed by default) as
|
||||
appropriate. To help catch configuration errors that miss classifying some
|
||||
requests, the mandatory `catch-all` priority level only allows one concurrency
|
||||
share and does not queue requests, making it relatively likely that traffic
|
||||
that only matches the `catch-all` FlowSchema will be rejected with an HTTP 429
|
||||
error.
|
||||
|
||||
## Health check concurrency exemption
|
||||
|
||||
The suggested configuration gives no special treatment to the health
|
||||
check requests on kube-apiservers from their local kubelets --- which
|
||||
tend to use the secured port but supply no credentials. With the
|
||||
suggested config, these requests get assigned to the `global-default`
|
||||
FlowSchema and the corresponding `global-default` priority level,
|
||||
where other traffic can crowd them out.
|
||||
|
||||
If you add the following additional FlowSchema, this exempts those
|
||||
requests from rate limiting.
|
||||
|
||||
{{< caution >}}
|
||||
Making this change also allows any hostile party to then send
|
||||
health-check requests that match this FlowSchema, at any volume they
|
||||
like. If you have a web traffic filter or similar external security
|
||||
mechanism to protect your cluster's API server from general internet
|
||||
traffic, you can configure rules to block any health check requests
|
||||
that originate from outside your cluster.
|
||||
{{< /caution >}}
|
||||
|
||||
{{< codenew file="priority-and-fairness/health-for-strangers.yaml" >}}
|
||||
|
||||
## Resources
|
||||
|
||||
The flow control API involves two kinds of resources.
|
||||
[PriorityLevelConfigurations](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#prioritylevelconfiguration-v1beta1-flowcontrol-apiserver-k8s-io)
|
||||
[PriorityLevelConfigurations](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#prioritylevelconfiguration-v1beta2-flowcontrol-apiserver-k8s-io)
|
||||
define the available isolation classes, the share of the available concurrency
|
||||
budget that each can handle, and allow for fine-tuning queuing behavior.
|
||||
[FlowSchemas](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#flowschema-v1beta1-flowcontrol-apiserver-k8s-io)
|
||||
[FlowSchemas](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#flowschema-v1beta2-flowcontrol-apiserver-k8s-io)
|
||||
are used to classify individual inbound requests, matching each to a
|
||||
single PriorityLevelConfiguration. There is also a `v1alpha1` version
|
||||
of the same API group, and it has the same Kinds with the same syntax and
|
||||
@@ -329,6 +256,153 @@ omitted entirely), in which case all requests matched by this FlowSchema will be
|
||||
considered part of a single flow. The correct choice for a given FlowSchema
|
||||
depends on the resource and your particular environment.
|
||||
|
||||
## Defaults
|
||||
|
||||
Each kube-apiserver maintains two sorts of APF configuration objects:
|
||||
mandatory and suggested.
|
||||
|
||||
### Mandatory Configuration Objects
|
||||
|
||||
The four mandatory configuration objects reflect fixed built-in
|
||||
guardrail behavior. This is behavior that the servers have before
|
||||
those objects exist, and when those objects exist their specs reflect
|
||||
this behavior. The four mandatory objects are as follows.
|
||||
|
||||
* The mandatory `exempt` priority level is used for requests that are
|
||||
not subject to flow control at all: they will always be dispatched
|
||||
immediately. The mandatory `exempt` FlowSchema classifies all
|
||||
requests from the `system:masters` group into this priority
|
||||
level. You may define other FlowSchemas that direct other requests
|
||||
to this priority level, if appropriate.
|
||||
|
||||
* The mandatory `catch-all` priority level is used in combination with
|
||||
the mandatory `catch-all` FlowSchema to make sure that every request
|
||||
gets some kind of classification. Typically you should not rely on
|
||||
this catch-all configuration, and should create your own catch-all
|
||||
FlowSchema and PriorityLevelConfiguration (or use the suggested
|
||||
`global-default` priority level that is installed by default) as
|
||||
appropriate. Because it is not expected to be used normally, the
|
||||
mandatory `catch-all` priority level has a very small concurrency
|
||||
share and does not queue requests.
|
||||
|
||||
### Suggested Configuration Objects
|
||||
|
||||
The suggested FlowSchemas and PriorityLevelConfigurations constitute a
|
||||
reasonable default configuration. You can modify these and/or create
|
||||
additional configuration objects if you want. If your cluster is
|
||||
likely to experience heavy load then you should consider what
|
||||
configuration will work best.
|
||||
|
||||
The suggested configuration groups requests into six priority levels:
|
||||
|
||||
* The `node-high` priority level is for health updates from nodes.
|
||||
|
||||
* The `system` priority level is for non-health requests from the
|
||||
`system:nodes` group, i.e. Kubelets, which must be able to contact
|
||||
the API server in order for workloads to be able to schedule on
|
||||
them.
|
||||
|
||||
* The `leader-election` priority level is for leader election requests from
|
||||
built-in controllers (in particular, requests for `endpoints`, `configmaps`,
|
||||
or `leases` coming from the `system:kube-controller-manager` or
|
||||
`system:kube-scheduler` users and service accounts in the `kube-system`
|
||||
namespace). These are important to isolate from other traffic because failures
|
||||
in leader election cause their controllers to fail and restart, which in turn
|
||||
causes more expensive traffic as the new controllers sync their informers.
|
||||
|
||||
* The `workload-high` priority level is for other requests from built-in
|
||||
controllers.
|
||||
|
||||
* The `workload-low` priority level is for requests from any other service
|
||||
account, which will typically include all requests from controllers running in
|
||||
Pods.
|
||||
|
||||
* The `global-default` priority level handles all other traffic, e.g.
|
||||
interactive `kubectl` commands run by nonprivileged users.
|
||||
|
||||
The suggested FlowSchemas serve to steer requests into the above
|
||||
priority levels, and are not enumerated here.
|
||||
|
||||
### Maintenance of the Mandatory and Suggested Configuration Objects
|
||||
|
||||
Each `kube-apiserver` independently maintains the mandatory and
|
||||
suggested configuration objects, using initial and periodic behavior.
|
||||
Thus, in a situation with a mixture of servers of different versions
|
||||
there may be thrashing as long as different servers have different
|
||||
opinions of the proper content of these objects.
|
||||
|
||||
Each `kube-apiserver` makes an inital maintenance pass over the
|
||||
mandatory and suggested configuration objects, and after that does
|
||||
periodic maintenance (once per minute) of those objects.
|
||||
|
||||
For the mandatory configuration objects, maintenance consists of
|
||||
ensuring that the object exists and, if it does, has the proper spec.
|
||||
The server refuses to allow a creation or update with a spec that is
|
||||
inconsistent with the server's guardrail behavior.
|
||||
|
||||
Maintenance of suggested configuration objects is designed to allow
|
||||
their specs to be overridden. Deletion, on the other hand, is not
|
||||
respected: maintenance will restore the object. If you do not want a
|
||||
suggested configuration object then you need to keep it around but set
|
||||
its spec to have minimal consequences. Maintenance of suggested
|
||||
objects is also designed to support automatic migration when a new
|
||||
version of the `kube-apiserver` is rolled out, albeit potentially with
|
||||
thrashing while there is a mixed population of servers.
|
||||
|
||||
Maintenance of a suggested configuration object consists of creating
|
||||
it --- with the server's suggested spec --- if the object does not
|
||||
exist. OTOH, if the object already exists, maintenance behavior
|
||||
depends on whether the `kube-apiservers` or the users control the
|
||||
object. In the former case, the server ensures that the object's spec
|
||||
is what the server suggests; in the latter case, the spec is left
|
||||
alone.
|
||||
|
||||
The question of who controls the object is answered by first looking
|
||||
for an annotation with key `apf.kubernetes.io/autoupdate-spec`. If
|
||||
there is such an annotation and its value is `true` then the
|
||||
kube-apiservers control the object. If there is such an annotation
|
||||
and its value is `false` then the users control the object. If
|
||||
neither of those condtions holds then the `metadata.generation` of the
|
||||
object is consulted. If that is 1 then the kube-apiservers control
|
||||
the object. Otherwise the users control the object. These rules were
|
||||
introduced in release 1.22 and their consideration of
|
||||
`metadata.generation` is for the sake of migration from the simpler
|
||||
earlier behavior. Users who wish to control a suggested configuration
|
||||
object should set its `apf.kubernetes.io/autoupdate-spec` annotation
|
||||
to `false`.
|
||||
|
||||
Maintenance of a mandatory or suggested configuration object also
|
||||
includes ensuring that it has an `apf.kubernetes.io/autoupdate-spec`
|
||||
annotation that accurately reflects whether the kube-apiservers
|
||||
control the object.
|
||||
|
||||
Maintenance also includes deleting objects that are neither mandatory
|
||||
nor suggested but are annotated
|
||||
`apf.kubernetes.io/autoupdate-spec=true`.
|
||||
|
||||
## Health check concurrency exemption
|
||||
|
||||
The suggested configuration gives no special treatment to the health
|
||||
check requests on kube-apiservers from their local kubelets --- which
|
||||
tend to use the secured port but supply no credentials. With the
|
||||
suggested config, these requests get assigned to the `global-default`
|
||||
FlowSchema and the corresponding `global-default` priority level,
|
||||
where other traffic can crowd them out.
|
||||
|
||||
If you add the following additional FlowSchema, this exempts those
|
||||
requests from rate limiting.
|
||||
|
||||
{{< caution >}}
|
||||
Making this change also allows any hostile party to then send
|
||||
health-check requests that match this FlowSchema, at any volume they
|
||||
like. If you have a web traffic filter or similar external security
|
||||
mechanism to protect your cluster's API server from general internet
|
||||
traffic, you can configure rules to block any health check requests
|
||||
that originate from outside your cluster.
|
||||
{{< /caution >}}
|
||||
|
||||
{{< codenew file="priority-and-fairness/health-for-strangers.yaml" >}}
|
||||
|
||||
## Diagnostics
|
||||
|
||||
Every HTTP response from an API server with the priority and fairness feature
|
||||
|
||||
@@ -709,13 +709,13 @@ Allocated resources:
|
||||
680m (34%) 400m (20%) 920Mi (11%) 1070Mi (13%)
|
||||
```
|
||||
|
||||
In the preceding output, you can see that if a Pod requests more than 1.120 CPUs,
|
||||
In the preceding output, you can see that if a Pod requests more than 1.120 CPUs
|
||||
or more than 6.23Gi of memory, that Pod will not fit on the node.
|
||||
|
||||
By looking at the “Pods” section, you can see which Pods are taking up space on
|
||||
the node.
|
||||
|
||||
The amount of resources available to Pods is less than the node capacity, because
|
||||
The amount of resources available to Pods is less than the node capacity because
|
||||
system daemons use a portion of the available resources. Within the Kubernetes API,
|
||||
each Node has a `.status.allocatable` field
|
||||
(see [NodeStatus](/docs/reference/kubernetes-api/cluster-resources/node-v1/#NodeStatus)
|
||||
@@ -736,7 +736,7 @@ prevent one team from using so much of any resource that this over-use affects o
|
||||
|
||||
You should also consider what access you grant to that namespace:
|
||||
**full** write access to a namespace allows someone with that access to remove any
|
||||
resource, include a configured ResourceQuota.
|
||||
resource, including a configured ResourceQuota.
|
||||
|
||||
### My container is terminated
|
||||
|
||||
|
||||
@@ -35,13 +35,12 @@ The Pod name and namespace are available as environment variables through the
|
||||
[downward API](/docs/tasks/inject-data-application/downward-api-volume-expose-pod-information/).
|
||||
|
||||
User defined environment variables from the Pod definition are also available to the Container,
|
||||
as are any environment variables specified statically in the Docker image.
|
||||
as are any environment variables specified statically in the container image.
|
||||
|
||||
### Cluster information
|
||||
|
||||
A list of all services that were running when a Container was created is available to that Container as environment variables.
|
||||
This list is limited to services within the same namespace as the new Container's Pod and Kubernetes control plane services.
|
||||
Those environment variables match the syntax of Docker links.
|
||||
|
||||
For a service named *foo* that maps to a Container named *bar*,
|
||||
the following variables are defined:
|
||||
|
||||
@@ -13,11 +13,9 @@ weight: 30
|
||||
|
||||
## The Kubernetes model for connecting containers
|
||||
|
||||
Now that you have a continuously running, replicated application you can expose it on a network. Before discussing the Kubernetes approach to networking, it is worthwhile to contrast it with the "normal" way networking works with Docker.
|
||||
Now that you have a continuously running, replicated application you can expose it on a network.
|
||||
|
||||
By default, Docker uses host-private networking, so containers can talk to other containers only if they are on the same machine. In order for Docker containers to communicate across nodes, there must be allocated ports on the machine's own IP address, which are then forwarded or proxied to the containers. This obviously means that containers must either coordinate which ports they use very carefully or ports must be allocated dynamically.
|
||||
|
||||
Coordinating port allocations across multiple developers or teams that provide containers is very difficult to do at scale, and exposes users to cluster-level issues outside of their control. Kubernetes assumes that pods can communicate with other pods, regardless of which host they land on. Kubernetes gives every pod its own cluster-private IP address, so you do not need to explicitly create links between pods or map container ports to host ports. This means that containers within a Pod can all reach each other's ports on localhost, and all pods in a cluster can see each other without NAT. The rest of this document elaborates on how you can run reliable services on such a networking model.
|
||||
Kubernetes assumes that pods can communicate with other pods, regardless of which host they land on. Kubernetes gives every pod its own cluster-private IP address, so you do not need to explicitly create links between pods or map container ports to host ports. This means that containers within a Pod can all reach each other's ports on localhost, and all pods in a cluster can see each other without NAT. The rest of this document elaborates on how you can run reliable services on such a networking model.
|
||||
|
||||
This guide uses a simple nginx server to demonstrate proof of concept.
|
||||
|
||||
@@ -52,7 +50,7 @@ kubectl get pods -l run=my-nginx -o yaml | grep podIP
|
||||
podIP: 10.244.2.5
|
||||
```
|
||||
|
||||
You should be able to ssh into any node in your cluster and curl both IPs. Note that the containers are *not* using port 80 on the node, nor are there any special NAT rules to route traffic to the pod. This means you can run multiple nginx pods on the same node all using the same containerPort and access them from any other pod or node in your cluster using IP. Like Docker, ports can still be published to the host node's interfaces, but the need for this is radically diminished because of the networking model.
|
||||
You should be able to ssh into any node in your cluster and use a tool such as `curl` to make queries against both IPs. Note that the containers are *not* using port 80 on the node, nor are there any special NAT rules to route traffic to the pod. This means you can run multiple nginx pods on the same node all using the same `containerPort`, and access them from any other pod or node in your cluster using the assigned IP address for the Service. If you want to arrange for a specific port on the host Node to be forwarded to backing Pods, you can - but the networking model should mean that you do not need to do so.
|
||||
|
||||
You can read more about the [Kubernetes Networking Model](/docs/concepts/cluster-administration/networking/#the-kubernetes-network-model) if you're curious.
|
||||
|
||||
|
||||
@@ -9,7 +9,7 @@ weight: 45
|
||||
|
||||
<!-- overview -->
|
||||
|
||||
{{< feature-state for_k8s_version="v1.21" state="alpha" >}}
|
||||
{{< feature-state for_k8s_version="v1.23" state="beta" >}}
|
||||
|
||||
_Service Internal Traffic Policy_ enables internal traffic restrictions to only route
|
||||
internal traffic to endpoints within the node the traffic originated from. The
|
||||
@@ -20,9 +20,9 @@ cluster. This can help to reduce costs and improve performance.
|
||||
|
||||
## Using Service Internal Traffic Policy
|
||||
|
||||
Once you have enabled the `ServiceInternalTrafficPolicy`
|
||||
[feature gate](/docs/reference/command-line-tools-reference/feature-gates/),
|
||||
you can enable an internal-only traffic policy for a
|
||||
The `ServiceInternalTrafficPolicy` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/)
|
||||
is a Beta feature and enabled by default.
|
||||
When the feature is enabled, you can enable the internal-only traffic policy for a
|
||||
{{< glossary_tooltip text="Services" term_id="service" >}}, by setting its
|
||||
`.spec.internalTrafficPolicy` to `Local`.
|
||||
This tells kube-proxy to only use node local endpoints for cluster internal traffic.
|
||||
|
||||
@@ -450,10 +450,7 @@ variables and DNS.
|
||||
### Environment variables
|
||||
|
||||
When a Pod is run on a Node, the kubelet adds a set of environment variables
|
||||
for each active Service. It supports both [Docker links
|
||||
compatible](https://docs.docker.com/userguide/dockerlinks/) variables (see [makeLinkVariables](https://github.com/kubernetes/kubernetes/blob/dd2d12f6dc0e654c15d5db57a5f9f6ba61192726/pkg/kubelet/envvars/envvars.go#L72))
|
||||
and simpler `{SVCNAME}_SERVICE_HOST` and `{SVCNAME}_SERVICE_PORT` variables,
|
||||
where the Service name is upper-cased and dashes are converted to underscores.
|
||||
for each active Service. It adds `{SVCNAME}_SERVICE_HOST` and `{SVCNAME}_SERVICE_PORT` variables, where the Service name is upper-cased and dashes are converted to underscores. It also supports variables (see [makeLinkVariables](https://github.com/kubernetes/kubernetes/blob/dd2d12f6dc0e654c15d5db57a5f9f6ba61192726/pkg/kubelet/envvars/envvars.go#L72)) that are compatible with Docker Engine's "_[legacy container links](https://docs.docker.com/network/links/)_" feature.
|
||||
|
||||
For example, the Service `redis-master` which exposes TCP port 6379 and has been
|
||||
allocated cluster IP address 10.0.0.11, produces the following environment
|
||||
@@ -687,21 +684,28 @@ The set of protocols that can be used for LoadBalancer type of Services is still
|
||||
|
||||
#### Disabling load balancer NodePort allocation {#load-balancer-nodeport-allocation}
|
||||
|
||||
{{< feature-state for_k8s_version="v1.20" state="alpha" >}}
|
||||
{{< feature-state for_k8s_version="v1.22" state="beta" >}}
|
||||
|
||||
Starting in v1.20, you can optionally disable node port allocation for a Service Type=LoadBalancer by setting
|
||||
You can optionally disable node port allocation for a Service of `type=LoadBalancer`, by setting
|
||||
the field `spec.allocateLoadBalancerNodePorts` to `false`. This should only be used for load balancer implementations
|
||||
that route traffic directly to pods as opposed to using node ports. By default, `spec.allocateLoadBalancerNodePorts`
|
||||
is `true` and type LoadBalancer Services will continue to allocate node ports. If `spec.allocateLoadBalancerNodePorts`
|
||||
is set to `false` on an existing Service with allocated node ports, those node ports will NOT be de-allocated automatically.
|
||||
is set to `false` on an existing Service with allocated node ports, those node ports will **not** be de-allocated automatically.
|
||||
You must explicitly remove the `nodePorts` entry in every Service port to de-allocate those node ports.
|
||||
You must enable the `ServiceLBNodePortControl` feature gate to use this field.
|
||||
Your cluster must have the `ServiceLBNodePortControl`
|
||||
[feature gate](/docs/reference/command-line-tools-reference/feature-gates/)
|
||||
enabled to use this field.
|
||||
For Kubernetes v{{< skew currentVersion >}}, this feature gate is enabled by default,
|
||||
and you can use the `spec.allocateLoadBalancerNodePorts` field. For clusters running
|
||||
other versions of Kubernetes, check the documentation for that release.
|
||||
|
||||
#### Specifying class of load balancer implementation {#load-balancer-class}
|
||||
|
||||
{{< feature-state for_k8s_version="v1.22" state="beta" >}}
|
||||
|
||||
`spec.loadBalancerClass` enables you to use a load balancer implementation other than the cloud provider default. This feature is available from v1.21, you must enable the `ServiceLoadBalancerClass` feature gate to use this field in v1.21, and the feature gate is enabled by default from v1.22 onwards.
|
||||
`spec.loadBalancerClass` enables you to use a load balancer implementation other than the cloud provider default.
|
||||
Your cluster must have the `ServiceLoadBalancerClass` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/) enabled to use this field. For Kubernetes v{{< skew currentVersion >}}, this feature gate is enabled by default. For clusters running
|
||||
other versions of Kubernetes, check the documentation for that release.
|
||||
By default, `spec.loadBalancerClass` is `nil` and a `LoadBalancer` type of Service uses
|
||||
the cloud provider's default load balancer implementation if the cluster is configured with
|
||||
a cloud provider using the `--cloud-provider` component flag.
|
||||
|
||||
@@ -266,7 +266,7 @@ Note that we recommend using Deployments instead of directly using Replica Sets,
|
||||
|
||||
### Deployment (Recommended)
|
||||
|
||||
[`Deployment`](/docs/concepts/workloads/controllers/deployment/) is a higher-level API object that updates its underlying Replica Sets and their Pods. Deployments are recommended if you want this rolling update functionality because, they are declarative, server-side, and have additional features.
|
||||
[`Deployment`](/docs/concepts/workloads/controllers/deployment/) is a higher-level API object that updates its underlying Replica Sets and their Pods. Deployments are recommended if you want the rolling update functionality because, they are declarative, server-side, and have additional features.
|
||||
|
||||
### Bare Pods
|
||||
|
||||
|
||||
@@ -12,11 +12,13 @@ Read more about shortcodes in the [Hugo documentation](https://gohugo.io/content
|
||||
|
||||
## Feature state
|
||||
|
||||
In a Markdown page (`.md` file) on this site, you can add a shortcode to display version and state of the documented feature.
|
||||
In a Markdown page (`.md` file) on this site, you can add a shortcode to
|
||||
display version and state of the documented feature.
|
||||
|
||||
### Feature state demo
|
||||
|
||||
Below is a demo of the feature state snippet, which displays the feature as stable in the latest Kubernetes version.
|
||||
Below is a demo of the feature state snippet, which displays the feature as
|
||||
stable in the latest Kubernetes version.
|
||||
|
||||
```
|
||||
{{</* feature-state state="stable" */>}}
|
||||
@@ -50,16 +52,22 @@ Renders to:
|
||||
|
||||
There are two glossary shortcodes: `glossary_tooltip` and `glossary_definition`.
|
||||
|
||||
You can reference glossary terms with an inclusion that automatically updates and replaces content with the relevant links from [our glossary](/docs/reference/glossary/). When the glossary term is moused-over, the glossary entry displays a tooltip. The glossary term also displays as a link.
|
||||
You can reference glossary terms with an inclusion that automatically updates
|
||||
and replaces content with the relevant links from [our glossary](/docs/reference/glossary/).
|
||||
When the glossary term is moused-over, the glossary entry displays a tooltip.
|
||||
The glossary term also displays as a link.
|
||||
|
||||
As well as inclusions with tooltips, you can reuse the definitions from the glossary in
|
||||
page content.
|
||||
|
||||
The raw data for glossary terms is stored at [https://github.com/kubernetes/website/tree/main/content/en/docs/reference/glossary](https://github.com/kubernetes/website/tree/main/content/en/docs/reference/glossary), with a content file for each glossary term.
|
||||
The raw data for glossary terms is stored at
|
||||
[the glossary directory](https://github.com/kubernetes/website/tree/main/content/en/docs/reference/glossary),
|
||||
with a content file for each glossary term.
|
||||
|
||||
### Glossary demo
|
||||
|
||||
For example, the following include within the Markdown renders to {{< glossary_tooltip text="cluster" term_id="cluster" >}} with a tooltip:
|
||||
For example, the following include within the Markdown renders to
|
||||
{{< glossary_tooltip text="cluster" term_id="cluster" >}} with a tooltip:
|
||||
|
||||
```
|
||||
{{</* glossary_tooltip text="cluster" term_id="cluster" */>}}
|
||||
@@ -85,7 +93,9 @@ which renders as:
|
||||
|
||||
## Links to API Reference
|
||||
|
||||
You can link to a page of the Kubernetes API reference using the `api-reference` shortcode, for example to the {{< api-reference page="workload-resources/pod-v1" >}} reference:
|
||||
You can link to a page of the Kubernetes API reference using the
|
||||
`api-reference` shortcode, for example to the
|
||||
{{< api-reference page="workload-resources/pod-v1" >}} reference:
|
||||
|
||||
```
|
||||
{{</* api-reference page="workload-resources/pod-v1" */>}}
|
||||
@@ -94,7 +104,10 @@ You can link to a page of the Kubernetes API reference using the `api-reference`
|
||||
The content of the `page` parameter is the suffix of the URL of the API reference page.
|
||||
|
||||
|
||||
You can link to a specific place into a page by specifying an `anchor` parameter, for example to the {{< api-reference page="workload-resources/pod-v1" anchor="PodSpec" >}} reference or the {{< api-reference page="workload-resources/pod-v1" anchor="environment-variables" >}} section of the page:
|
||||
You can link to a specific place into a page by specifying an `anchor`
|
||||
parameter, for example to the {{< api-reference page="workload-resources/pod-v1" anchor="PodSpec" >}}
|
||||
reference or the {{< api-reference page="workload-resources/pod-v1" anchor="environment-variables" >}}
|
||||
section of the page:
|
||||
|
||||
```
|
||||
{{</* api-reference page="workload-resources/pod-v1" anchor="PodSpec" */>}}
|
||||
@@ -102,17 +115,20 @@ You can link to a specific place into a page by specifying an `anchor` parameter
|
||||
```
|
||||
|
||||
|
||||
You can change the text of the link by specifying a `text` parameter, for example by linking to the {{< api-reference page="workload-resources/pod-v1" anchor="environment-variables" text="Environment Variables">}} section of the page:
|
||||
You can change the text of the link by specifying a `text` parameter, for
|
||||
example by linking to the
|
||||
{{< api-reference page="workload-resources/pod-v1" anchor="environment-variables" text="Environment Variables">}}
|
||||
section of the page:
|
||||
|
||||
```
|
||||
{{</* api-reference page="workload-resources/pod-v1" anchor="environment-variables" text="Environment Variable" */>}}
|
||||
```
|
||||
|
||||
|
||||
|
||||
## Table captions
|
||||
|
||||
You can make tables more accessible to screen readers by adding a table caption. To add a [caption](https://www.w3schools.com/tags/tag_caption.asp) to a table, enclose the table with a `table` shortcode and specify the caption with the `caption` parameter.
|
||||
You can make tables more accessible to screen readers by adding a table caption. To add a
|
||||
[caption](https://www.w3schools.com/tags/tag_caption.asp) to a table,
|
||||
enclose the table with a `table` shortcode and specify the caption with the `caption` parameter.
|
||||
|
||||
{{< note >}}
|
||||
Table captions are visible to screen readers but invisible when viewed in standard HTML.
|
||||
@@ -138,7 +154,8 @@ Parameter | Description | Default
|
||||
`logLevel` | The log level for log output | `INFO`
|
||||
{{< /table >}}
|
||||
|
||||
If you inspect the HTML for the table, you should see this element immediately after the opening `<table>` element:
|
||||
If you inspect the HTML for the table, you should see this element immediately
|
||||
after the opening `<table>` element:
|
||||
|
||||
```html
|
||||
<caption style="display: none;">Configuration parameters</caption>
|
||||
@@ -146,14 +163,25 @@ If you inspect the HTML for the table, you should see this element immediately a
|
||||
|
||||
## Tabs
|
||||
|
||||
In a markdown page (`.md` file) on this site, you can add a tab set to display multiple flavors of a given solution.
|
||||
In a markdown page (`.md` file) on this site, you can add a tab set to display
|
||||
multiple flavors of a given solution.
|
||||
|
||||
The `tabs` shortcode takes these parameters:
|
||||
|
||||
* `name`: The name as shown on the tab.
|
||||
* `codelang`: If you provide inner content to the `tab` shortcode, you can tell Hugo what code language to use for highlighting.
|
||||
* `include`: The file to include in the tab. If the tab lives in a Hugo [leaf bundle](https://gohugo.io/content-management/page-bundles/#leaf-bundles), the file -- which can be any MIME type supported by Hugo -- is looked up in the bundle itself. If not, the content page that needs to be included is looked up relative to the current page. Note that with the `include`, you do not have any shortcode inner content and must use the self-closing syntax. For example, <code>{{</* tab name="Content File #1" include="example1" /*/>}}</code>. The language needs to be specified under `codelang` or the language is taken based on the file name. Non-content files are code-highlighted by default.
|
||||
* If your inner content is markdown, you must use the `%`-delimiter to surround the tab. For example, `{{%/* tab name="Tab 1" %}}This is **markdown**{{% /tab */%}}`
|
||||
* `codelang`: If you provide inner content to the `tab` shortcode, you can tell Hugo
|
||||
what code language to use for highlighting.
|
||||
* `include`: The file to include in the tab. If the tab lives in a Hugo
|
||||
[leaf bundle](https://gohugo.io/content-management/page-bundles/#leaf-bundles),
|
||||
the file -- which can be any MIME type supported by Hugo -- is looked up in the bundle itself.
|
||||
If not, the content page that needs to be included is looked up relative to the current page.
|
||||
Note that with the `include`, you do not have any shortcode inner content and must use the
|
||||
self-closing syntax. For example,
|
||||
`{{</* tab name="Content File #1" include="example1" /*/>}}`. The language needs to be specified
|
||||
under `codelang` or the language is taken based on the file name.
|
||||
Non-content files are code-highlighted by default.
|
||||
* If your inner content is markdown, you must use the `%`-delimiter to surround the tab.
|
||||
For example, `{{%/* tab name="Tab 1" %}}This is **markdown**{{% /tab */%}}`
|
||||
* You can combine the variations mentioned above inside a tab set.
|
||||
|
||||
Below is a demo of the tabs shortcode.
|
||||
@@ -288,13 +316,17 @@ The two most commonly used version parameters are `latest` and `version`.
|
||||
|
||||
### `{{</* param "version" */>}}`
|
||||
|
||||
The `{{</* param "version" */>}}` shortcode generates the value of the current version of
|
||||
the Kubernetes documentation from the `version` site parameter. The `param` shortcode accepts the name of one site parameter, in this case: `version`.
|
||||
The `{{</* param "version" */>}}` shortcode generates the value of the current
|
||||
version of the Kubernetes documentation from the `version` site parameter. The
|
||||
`param` shortcode accepts the name of one site parameter, in this case:
|
||||
`version`.
|
||||
|
||||
{{< note >}}
|
||||
In previously released documentation, `latest` and `version` parameter values are not equivalent.
|
||||
After a new version is released, `latest` is incremented and the value of `version` for the documentation set remains unchanged. For example, a previously released version of the documentation displays `version` as
|
||||
`v1.19` and `latest` as `v1.20`.
|
||||
In previously released documentation, `latest` and `version` parameter values
|
||||
are not equivalent. After a new version is released, `latest` is incremented
|
||||
and the value of `version` for the documentation set remains unchanged. For
|
||||
example, a previously released version of the documentation displays `version`
|
||||
as `v1.19` and `latest` as `v1.20`.
|
||||
{{< /note >}}
|
||||
|
||||
Renders to:
|
||||
@@ -313,7 +345,8 @@ Renders to:
|
||||
|
||||
### `{{</* latest-semver */>}}`
|
||||
|
||||
The `{{</* latest-semver */>}}` shortcode generates the value of `latest` without the "v" prefix.
|
||||
The `{{</* latest-semver */>}}` shortcode generates the value of `latest`
|
||||
without the "v" prefix.
|
||||
|
||||
Renders to:
|
||||
|
||||
@@ -330,8 +363,9 @@ Renders to:
|
||||
|
||||
### `{{</* latest-release-notes */>}}`
|
||||
|
||||
The `{{</* latest-release-notes */>}}` shortcode generates a version string from `latest` and removes
|
||||
the "v" prefix. The shortcode prints a new URL for the release note CHANGELOG page with the modified version string.
|
||||
The `{{</* latest-release-notes */>}}` shortcode generates a version string
|
||||
from `latest` and removes the "v" prefix. The shortcode prints a new URL for
|
||||
the release note CHANGELOG page with the modified version string.
|
||||
|
||||
Renders to:
|
||||
|
||||
@@ -344,3 +378,4 @@ Renders to:
|
||||
* Learn about [page content types](/docs/contribute/style/page-content-types/).
|
||||
* Learn about [opening a pull request](/docs/contribute/new-content/open-a-pr/).
|
||||
* Learn about [advanced contributing](/docs/contribute/advanced/).
|
||||
|
||||
|
||||
@@ -25,7 +25,8 @@ on each Kubernetes component.
|
||||
Each Kubernetes component lets you enable or disable a set of feature gates that
|
||||
are relevant to that component.
|
||||
Use `-h` flag to see a full set of feature gates for all components.
|
||||
To set feature gates for a component, such as kubelet, use the `--feature-gates` flag assigned to a list of feature pairs:
|
||||
To set feature gates for a component, such as kubelet, use the `--feature-gates`
|
||||
flag assigned to a list of feature pairs:
|
||||
|
||||
```shell
|
||||
--feature-gates="...,GracefulNodeShutdown=true"
|
||||
@@ -575,7 +576,7 @@ Each feature gate is designed for enabling/disabling a specific feature:
|
||||
- `AnyVolumeDataSource`: Enable use of any custom resource as the `DataSource` of a
|
||||
{{< glossary_tooltip text="PVC" term_id="persistent-volume-claim" >}}.
|
||||
- `AppArmor`: Enable use of AppArmor mandatory access control for Pods running on Linux nodes.
|
||||
See [AppArmor Tutorial](/docs/tutorials/clusters/apparmor/) for more details.
|
||||
See [AppArmor Tutorial](/docs/tutorials/security/apparmor/) for more details.
|
||||
- `AttachVolumeLimit`: Enable volume plugins to report limits on number of volumes
|
||||
that can be attached to a node.
|
||||
See [dynamic volume limits](/docs/concepts/storage/storage-limits/#dynamic-volume-limits) for more details.
|
||||
@@ -769,12 +770,12 @@ Each feature gate is designed for enabling/disabling a specific feature:
|
||||
- `EnableEquivalenceClassCache`: Enable the scheduler to cache equivalence of
|
||||
nodes when scheduling Pods.
|
||||
- `EndpointSlice`: Enables EndpointSlices for more scalable and extensible
|
||||
network endpoints. See [Enabling EndpointSlices](/docs/tasks/administer-cluster/enabling-endpointslices/).
|
||||
network endpoints. See [Enabling EndpointSlices](/docs/concepts/services-networking/endpoint-slices/).
|
||||
- `EndpointSliceNodeName`: Enables EndpointSlice `nodeName` field.
|
||||
- `EndpointSliceProxying`: When enabled, kube-proxy running
|
||||
on Linux will use EndpointSlices as the primary data source instead of
|
||||
Endpoints, enabling scalability and performance improvements. See
|
||||
[Enabling Endpoint Slices](/docs/tasks/administer-cluster/enabling-endpointslices/).
|
||||
[Enabling Endpoint Slices](/docs/concepts/services-networking/endpoint-slices/).
|
||||
- `EndpointSliceTerminatingCondition`: Enables EndpointSlice `terminating` and `serving`
|
||||
condition fields.
|
||||
- `EphemeralContainers`: Enable the ability to add
|
||||
@@ -1089,7 +1090,7 @@ Each feature gate is designed for enabling/disabling a specific feature:
|
||||
- `WindowsEndpointSliceProxying`: When enabled, kube-proxy running on Windows
|
||||
will use EndpointSlices as the primary data source instead of Endpoints,
|
||||
enabling scalability and performance improvements. See
|
||||
[Enabling Endpoint Slices](/docs/tasks/administer-cluster/enabling-endpointslices/).
|
||||
[Enabling Endpoint Slices](/docs/concepts/services-networking/endpoint-slices/).
|
||||
- `WindowsGMSA`: Enables passing of GMSA credential specs from pods to container runtimes.
|
||||
- `WindowsHostProcessContainers`: Enables support for Windows HostProcess containers.
|
||||
- `WindowsRunAsUserName` : Enable support for running applications in Windows containers
|
||||
|
||||
@@ -44,16 +44,16 @@ kubelet [flags]
|
||||
<td colspan="2">--add-dir-header</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">If true, adds the file directory to the header of the log messages</td>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">If true, adds the file directory to the header of the log messages (DEPRECATED: will be removed in a future release, see https://github.com/kubernetes/enhancements/tree/master/keps/sig-instrumentation/2845-deprecate-klog-specific-flags-in-k8s-components)</td>
|
||||
</tr>
|
||||
|
||||
|
||||
<tr>
|
||||
<td colspan="2">--address string Default: 0.0.0.0 </td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">The IP address for the Kubelet to serve on (set to <code>0.0.0.0</code> or <code>::</code> for listening in all interfaces and IP families) (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's <code>--config</code> flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)</td>
|
||||
</tr>
|
||||
|
||||
|
||||
<tr>
|
||||
<td colspan="2">--allowed-unsafe-sysctls strings</td>
|
||||
</tr>
|
||||
@@ -65,7 +65,7 @@ kubelet [flags]
|
||||
<td colspan="2">--alsologtostderr</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">Log to standard error as well as files</td>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">Log to standard error as well as files (DEPRECATED: will be removed in a future release, see https://github.com/kubernetes/enhancements/tree/master/keps/sig-instrumentation/2845-deprecate-klog-specific-flags-in-k8s-components)</td>
|
||||
</tr>
|
||||
|
||||
<tr>
|
||||
@@ -90,10 +90,10 @@ kubelet [flags]
|
||||
</tr>
|
||||
|
||||
<tr>
|
||||
<td colspan="2">--authorization-mode string</td>
|
||||
<td colspan="2">--authorization-mode string Default: <code>AlwaysAllow</code></td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">Authorization mode for Kubelet server. Valid options are <code>AlwaysAllow</code> or <code>Webhook</code>. <code>Webhook</code> mode uses the <code>SubjectAccessReview</code> API to determine authorization. Default <code>AlwaysAllow</code> when <code>--config</code> flag is not provided; <code>Webhook</code> when <code>--config</code> flag presents. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)</td>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">Authorization mode for Kubelet server. Valid options are AlwaysAllow or Webhook. Webhook mode uses the SubjectAccessReview API to determine authorization. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)</td>
|
||||
</tr>
|
||||
|
||||
<tr>
|
||||
@@ -243,7 +243,7 @@ kubelet [flags]
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">[Experimental] The endpoint of remote runtime service. Currently unix socket endpoint is supported on Linux, while npipe and tcp endpoints are supported on windows. Examples: <code>unix:///var/run/dockershim.sock</code>, <code>npipe:////./pipe/dockershim</code>.</td>
|
||||
</tr>
|
||||
|
||||
|
||||
|
||||
<tr>
|
||||
<td colspan="2">--contention-profiling</td>
|
||||
</tr>
|
||||
@@ -273,7 +273,7 @@ kubelet [flags]
|
||||
</tr>
|
||||
|
||||
<tr>
|
||||
<td colspan="2">--cpu-manager-policy-options strings</td>
|
||||
<td colspan="2">--cpu-manager-policy-options mapStringString</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">Comma-separated list of options to fine-tune the behavior of the selected CPU Manager policy. If not supplied, keep the default behaviour. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's <code>--config</code> flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)</td>
|
||||
@@ -290,14 +290,14 @@ kubelet [flags]
|
||||
<td colspan="2">--docker-endpoint string Default: <code>unix:///var/run/docker.sock</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">Use this for the <code>docker</code> endpoint to communicate with. This docker-specific flag only works when container-runtime is set to <code>docker</code>.</td>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">Use this for the <code>docker</code> endpoint to communicate with. This docker-specific flag only works when container-runtime is set to <code>docker</code>. (DEPRECATED: will be removed along with dockershim.)</td>
|
||||
</tr>
|
||||
|
||||
<tr>
|
||||
<td colspan="2">--dynamic-config-dir string</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">The Kubelet will use this directory for checkpointing downloaded configurations and tracking configuration health. The Kubelet will create this directory if it does not already exist. The path may be absolute or relative; relative paths start at the Kubelet's current working directory. Providing this flag enables dynamic Kubelet configuration. The <code>DynamicKubeletConfig</code> feature gate must be enabled to pass this flag. (DEPRECATED: Feature DynamicKubeletConfig is deprecated in 1.22 and will not move to GA. It is planned to be removed from Kubernetes in the version 1.23. Please use alternative ways to update kubelet configuration.)</td>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">The Kubelet will use this directory for checkpointing downloaded configurations and tracking configuration health. The Kubelet will create this directory if it does not already exist. The path may be absolute or relative; relative paths start at the Kubelet's current working directory. Providing this flag enables dynamic Kubelet configuration. The <code>DynamicKubeletConfig</code> feature gate must be enabled to pass this flag. (DEPRECATED: Feature DynamicKubeletConfig is deprecated in 1.22 and will not move to GA. It is planned to be removed from Kubernetes in the version 1.23. Please use alternative ways to update kubelet configuration.)</td>
|
||||
</tr>
|
||||
|
||||
<tr>
|
||||
@@ -397,13 +397,6 @@ kubelet [flags]
|
||||
<tr>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">When set to <code>true</code>, hard eviction thresholds will be ignored while calculating node allocatable. See https://kubernetes.io/docs/tasks/administer-cluster/reserve-compute-resources/ for more details. (DEPRECATED: will be removed in 1.23)</td>
|
||||
</tr>
|
||||
|
||||
<tr>
|
||||
<td colspan="2">--experimental-bootstrap-kubeconfig string</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">DEPRECATED: Use <code>--bootstrap-kubeconfig</code></td>
|
||||
</tr>
|
||||
|
||||
<tr>
|
||||
<td colspan="2">--experimental-check-node-capabilities-before-mount</td>
|
||||
@@ -416,7 +409,7 @@ kubelet [flags]
|
||||
<td colspan="2">--experimental-kernel-memcg-notification</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">If enabled, the kubelet will integrate with the kernel memcg notification to determine if memory eviction thresholds are crossed rather than polling. This flag will be removed in 1.23. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's <code>--config</code> flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)</td>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">Use kernelMemcgNotification configuration, this flag will be removed in 1.23. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's <code>--config</code> flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)</td>
|
||||
</tr>
|
||||
|
||||
<tr>
|
||||
@@ -455,55 +448,63 @@ AllBeta=true|false (BETA - default=false)<br/>
|
||||
AnyVolumeDataSource=true|false (ALPHA - default=false)<br/>
|
||||
AppArmor=true|false (BETA - default=true)<br/>
|
||||
CPUManager=true|false (BETA - default=true)<br/>
|
||||
CPUManagerPolicyAlphaOptions=true|false (ALPHA - default=false)<br/>
|
||||
CPUManagerPolicyBetaOptions=true|false (BETA - default=true)<br/>
|
||||
CPUManagerPolicyOptions=true|false (ALPHA - default=false)<br/>
|
||||
CSIInlineVolume=true|false (BETA - default=true)<br/>
|
||||
CSIMigration=true|false (BETA - default=true)<br/>
|
||||
CSIMigrationAWS=true|false (BETA - default=false)<br/>
|
||||
CSIMigrationAzureDisk=true|false (BETA - default=false)<br/>
|
||||
CSIMigrationAzureDisk=true|false (BETA - default=true)<br/>
|
||||
CSIMigrationAzureFile=true|false (BETA - default=false)<br/>
|
||||
CSIMigrationGCE=true|false (BETA - default=false)<br/>
|
||||
CSIMigrationGCE=true|false (BETA - default=true)<br/>
|
||||
CSIMigrationOpenStack=true|false (BETA - default=true)<br/>
|
||||
CSIMigrationPortworx=true|false (ALPHA - default=false)<br/>
|
||||
CSIMigrationvSphere=true|false (BETA - default=false)<br/>
|
||||
CSIStorageCapacity=true|false (BETA - default=true)<br/>
|
||||
CSIVolumeFSGroupPolicy=true|false (BETA - default=true)<br/>
|
||||
CSIVolumeHealth=true|false (ALPHA - default=false)<br/>
|
||||
CSRDuration=true|false (BETA - default=true)<br/>
|
||||
ConfigurableFSGroupPolicy=true|false (BETA - default=true)<br/>
|
||||
ControllerManagerLeaderMigration=true|false (BETA - default=true)<br/>
|
||||
CustomCPUCFSQuotaPeriod=true|false (ALPHA - default=false)<br/>
|
||||
CustomResourceValidationExpressions=true|false (ALPHA - default=false)<br/>
|
||||
DaemonSetUpdateSurge=true|false (BETA - default=true)<br/>
|
||||
DefaultPodTopologySpread=true|false (BETA - default=true)<br/>
|
||||
DelegateFSGroupToCSIDriver=true|false (ALPHA - default=false)<br/>
|
||||
DelegateFSGroupToCSIDriver=true|false (BETA - default=true)<br/>
|
||||
DevicePlugins=true|false (BETA - default=true)<br/>
|
||||
DisableAcceleratorUsageMetrics=true|false (BETA - default=true)<br/>
|
||||
DisableCloudProviders=true|false (ALPHA - default=false)<br/>
|
||||
DownwardAPIHugePages=true|false (BETA - default=false)<br/>
|
||||
DisableKubeletCloudCredentialProviders=true|false (ALPHA - default=false)<br/>
|
||||
DownwardAPIHugePages=true|false (BETA - default=true)<br/>
|
||||
EfficientWatchResumption=true|false (BETA - default=true)<br/>
|
||||
EndpointSliceTerminatingCondition=true|false (BETA - default=true)<br/>
|
||||
EphemeralContainers=true|false (ALPHA - default=false)<br/>
|
||||
EphemeralContainers=true|false (BETA - default=true)<br/>
|
||||
ExpandCSIVolumes=true|false (BETA - default=true)<br/>
|
||||
ExpandInUsePersistentVolumes=true|false (BETA - default=true)<br/>
|
||||
ExpandPersistentVolumes=true|false (BETA - default=true)<br/>
|
||||
ExpandedDNSConfig=true|false (ALPHA - default=false)<br/>
|
||||
ExperimentalHostUserNamespaceDefaulting=true|false (BETA - default=false)<br/>
|
||||
GenericEphemeralVolume=true|false (BETA - default=true)<br/>
|
||||
GRPCContainerProbe=true|false (ALPHA - default=false)<br/>
|
||||
GracefulNodeShutdown=true|false (BETA - default=true)<br/>
|
||||
GracefulNodeShutdownBasedOnPodPriority=true|false (ALPHA - default=false)<br/>
|
||||
HPAContainerMetrics=true|false (ALPHA - default=false)<br/>
|
||||
HPAScaleToZero=true|false (ALPHA - default=false)<br/>
|
||||
IPv6DualStack=true|false (BETA - default=true)<br/>
|
||||
HonorPVReclaimPolicy=true|false (ALPHA - default=false)<br/>
|
||||
IdentifyPodOS=true|false (ALPHA - default=false)<br/>
|
||||
InTreePluginAWSUnregister=true|false (ALPHA - default=false)<br/>
|
||||
InTreePluginAzureDiskUnregister=true|false (ALPHA - default=false)<br/>
|
||||
InTreePluginAzureFileUnregister=true|false (ALPHA - default=false)<br/>
|
||||
InTreePluginGCEUnregister=true|false (ALPHA - default=false)<br/>
|
||||
InTreePluginOpenStackUnregister=true|false (ALPHA - default=false)<br/>
|
||||
InTreePluginPortworxUnregister=true|false (ALPHA - default=false)<br/>
|
||||
InTreePluginRBDUnregister=true|false (ALPHA - default=false)<br>
|
||||
InTreePluginvSphereUnregister=true|false (ALPHA - default=false)<br/>
|
||||
IndexedJob=true|false (BETA - default=true)<br/>
|
||||
IngressClassNamespacedParams=true|false (BETA - default=true)<br/>
|
||||
JobTrackingWithFinalizers=true|false (ALPHA - default=false)<br/>
|
||||
JobMutableNodeSchedulingDirectives=true|false (BETA - default=true)<br/>
|
||||
JobReadyPods=true|false (ALPHA - default=false)<br/>
|
||||
JobTrackingWithFinalizers=true|false (BETA - default=true)<br/>
|
||||
KubeletCredentialProviders=true|false (ALPHA - default=false)<br/>
|
||||
KubeletInUserNamespace=true|false (ALPHA - default=false)<br/>
|
||||
KubeletPodResources=true|false (BETA - default=true)<br/>
|
||||
KubeletPodResourcesGetAllocatable=true|false (ALPHA - default=false)<br/>
|
||||
KubeletPodResourcesGetAllocatable=true|false (BETA - default=true)<br/>
|
||||
LocalStorageCapacityIsolation=true|false (BETA - default=true)<br/>
|
||||
LocalStorageCapacityIsolationFSQuotaMonitoring=true|false (ALPHA - default=false)<br/>
|
||||
LogarithmicScaleDown=true|false (BETA - default=true)<br/>
|
||||
@@ -513,16 +514,20 @@ MixedProtocolLBService=true|false (ALPHA - default=false)<br/>
|
||||
NetworkPolicyEndPort=true|false (BETA - default=true)<br/>
|
||||
NodeSwap=true|false (ALPHA - default=false)<br/>
|
||||
NonPreemptingPriority=true|false (BETA - default=true)<br/>
|
||||
OpenAPIEnums=true|false (ALPHA - default=false)<br/>
|
||||
OpenAPIV3=true|false (ALPHA - default=false)<br/>
|
||||
PodAffinityNamespaceSelector=true|false (BETA - default=true)<br/>
|
||||
PodAndContainerStatsFromCRI=true|false (ALPHA - default=false)<br/>
|
||||
PodDeletionCost=true|false (BETA - default=true)<br/>
|
||||
PodOverhead=true|false (BETA - default=true)<br/>
|
||||
PodSecurity=true|false (ALPHA - default=false)<br/>
|
||||
PodSecurity=true|false (BETA - default=true)<br/>
|
||||
PreferNominatedNode=true|false (BETA - default=true)<br/>
|
||||
ProbeTerminationGracePeriod=true|false (BETA - default=false)<br/>
|
||||
ProcMountType=true|false (ALPHA - default=false)<br/>
|
||||
ProxyTerminatingEndpoints=true|false (ALPHA - default=false)<br/>
|
||||
QOSReserved=true|false (ALPHA - default=false)<br/>
|
||||
ReadWriteOncePod=true|false (ALPHA - default=false)<br/>
|
||||
RecoverVolumeExpansionFailure=true|false (ALPHA - default=false)<br/>
|
||||
RemainingItemCount=true|false (BETA - default=true)<br/>
|
||||
RemoveSelfLink=true|false (BETA - default=true)<br/>
|
||||
RotateKubeletServerCertificate=true|false (BETA - default=true)<br/>
|
||||
@@ -531,17 +536,18 @@ ServiceInternalTrafficPolicy=true|false (BETA - default=true)<br/>
|
||||
ServiceLBNodePortControl=true|false (BETA - default=true)<br/>
|
||||
ServiceLoadBalancerClass=true|false (BETA - default=true)<br/>
|
||||
SizeMemoryBackedVolumes=true|false (BETA - default=true)<br/>
|
||||
StatefulSetMinReadySeconds=true|false (ALPHA - default=false)<br/>
|
||||
StatefulSetAutoDeletePVC=true|false (ALPHA - default=false)<br/>
|
||||
StatefulSetMinReadySeconds=true|false (BETA - default=true)<br/>
|
||||
StorageVersionAPI=true|false (ALPHA - default=false)<br/>
|
||||
StorageVersionHash=true|false (BETA - default=true)<br/>
|
||||
SuspendJob=true|false (BETA - default=true)<br/>
|
||||
TTLAfterFinished=true|false (BETA - default=true)<br/>
|
||||
TopologyAwareHints=true|false (ALPHA - default=false)<br/>
|
||||
TopologyAwareHints=true|false (BETA - default=true)<br/>
|
||||
TopologyManager=true|false (BETA - default=true)<br/>
|
||||
VolumeCapacityPriority=true|false (ALPHA - default=false)<br/>
|
||||
WinDSR=true|false (ALPHA - default=false)<br/>
|
||||
WinOverlay=true|false (BETA - default=true)<br/>
|
||||
WindowsHostProcessContainers=true|false (ALPHA - default=false)<br/>
|
||||
WindowsHostProcessContainers=true|false (BETA - default=true)<br/>
|
||||
csiMigrationRBD=true|false (ALPHA - default=false)<br/>
|
||||
(DEPRECATED: This parameter should be set via the config file specified by the Kubelet's <code>--config</code> flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)</td>
|
||||
</tr>
|
||||
|
||||
@@ -635,21 +641,21 @@ WindowsHostProcessContainers=true|false (ALPHA - default=false)<br/>
|
||||
<tr>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">[Experimental] The endpoint of remote image service. If not specified, it will be the same with <code>--container-runtime-endpoint</code> by default. Currently UNIX socket endpoint is supported on Linux, while npipe and TCP endpoints are supported on Windows. Examples: <code>unix:///var/run/dockershim.sock</code>, <code>npipe:////./pipe/dockershim</code></td>
|
||||
</tr>
|
||||
|
||||
|
||||
<tr>
|
||||
<td colspan="2">--iptables-drop-bit int32 Default: 15</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">The bit of the <code>fwmark</code> space to mark packets for dropping. Must be within the range [0, 31]. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's <code>--config</code> flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)</td>
|
||||
</tr>
|
||||
|
||||
|
||||
<tr>
|
||||
<td colspan="2">--iptables-masquerade-bit int32 Default: 14</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">The bit of the <code>fwmark</code> space to mark packets for SNAT. Must be within the range [0, 31]. Please match this parameter with corresponding parameter in <code>kube-proxy</code>. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's <code>--config</code> flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)</td>
|
||||
</tr>
|
||||
|
||||
|
||||
<tr>
|
||||
<td colspan="2">--keep-terminated-pod-volumes</td>
|
||||
</tr>
|
||||
@@ -682,7 +688,7 @@ WindowsHostProcessContainers=true|false (ALPHA - default=false)<br/>
|
||||
<td colspan="2">--kube-api-qps int32 Default: 5</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">QPS to use while talking with kubernetes API server. The number must be >= 0. If 0 will use default QPS (5). (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's <code>--config</code> flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)</td>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">QPS to use while talking with kubernetes API server. The number must be >= 0. If 0 will use default QPS (5). Doesn't cover events and node heartbeat apis which rate limiting is controlled by a different set of flags. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's <code>--config</code> flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)</td>
|
||||
</tr>
|
||||
|
||||
<tr>
|
||||
@@ -724,28 +730,28 @@ WindowsHostProcessContainers=true|false (ALPHA - default=false)<br/>
|
||||
<td colspan="2">--log-backtrace-at <A string of format 'file:line'> Default: <code>":0"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">When logging hits line <code><file>:<N></code>, emit a stack trace.</td>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">When logging hits line <code><file>:<N></code>, emit a stack trace. (DEPRECATED: will be removed in a future release, see https://github.com/kubernetes/enhancements/tree/master/keps/sig-instrumentation/2845-deprecate-klog-specific-flags-in-k8s-components)</td>
|
||||
</tr>
|
||||
|
||||
<tr>
|
||||
<td colspan="2">--log-dir string</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">If non-empty, write log files in this directory</td>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">If non-empty, write log files in this directory. (DEPRECATED: will be removed in a future release, see https://github.com/kubernetes/enhancements/tree/master/keps/sig-instrumentation/2845-deprecate-klog-specific-flags-in-k8s-components)</td>
|
||||
</tr>
|
||||
|
||||
<tr>
|
||||
<td colspan="2">--log-file string</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">If non-empty, use this log file</td>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">If non-empty, use this log file. (DEPRECATED: will be removed in a future release, see https://github.com/kubernetes/enhancements/tree/master/keps/sig-instrumentation/2845-deprecate-klog-specific-flags-in-k8s-components)</td>
|
||||
</tr>
|
||||
|
||||
<tr>
|
||||
<td colspan="2">--log-file-max-size uint Default: 1800</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">Defines the maximum size a log file can grow to. Unit is megabytes. If the value is 0, the maximum file size is unlimited.</td>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">Defines the maximum size a log file can grow to. Unit is megabytes. If the value is 0, the maximum file size is unlimited. (DEPRECATED: will be removed in a future release, see https://github.com/kubernetes/enhancements/tree/master/keps/sig-instrumentation/2845-deprecate-klog-specific-flags-in-k8s-components)</td>
|
||||
</tr>
|
||||
|
||||
<tr>
|
||||
@@ -755,6 +761,20 @@ WindowsHostProcessContainers=true|false (ALPHA - default=false)<br/>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">Maximum number of seconds between log flushes.</td>
|
||||
</tr>
|
||||
|
||||
<tr>
|
||||
<td colspan="2">--log-json-info-buffer-size string Default: <code>'0'</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">[Experimental] In JSON format with split output streams, the info messages can be buffered for a while to increase performance. The default value of zero bytes disables buffering. The size can be specified as number of bytes (512), multiples of 1000 (1K), multiples of 1024 (2Ki), or powers of those (3M, 4G, 5Mi, 6Gi). (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's <code>--config</code> flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)</td>
|
||||
</tr>
|
||||
|
||||
<tr>
|
||||
<td colspan="2">--log-json-split-stream</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">[Experimental] In JSON format, write error messages to stderr and info messages to stdout. The default is to write a single stream to stdout. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's <code>--config</code> flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)</td>
|
||||
</tr>
|
||||
|
||||
<tr>
|
||||
<td colspan="2">--logging-format string Default: <code>text</code></td>
|
||||
</tr>
|
||||
@@ -766,7 +786,7 @@ WindowsHostProcessContainers=true|false (ALPHA - default=false)<br/>
|
||||
<td colspan="2">--logtostderr Default: <code>true</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">log to standard error instead of files.</td>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">log to standard error instead of files. (DEPRECATED: will be removed in a future release, see https://github.com/kubernetes/enhancements/tree/master/keps/sig-instrumentation/2845-deprecate-klog-specific-flags-in-k8s-components)</td>
|
||||
</tr>
|
||||
|
||||
<tr>
|
||||
@@ -789,13 +809,14 @@ WindowsHostProcessContainers=true|false (ALPHA - default=false)<br/>
|
||||
<tr>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">Comma-separated list of HTTP headers to use when accessing the URL provided to <code>--manifest-url</code>. Multiple headers with the same name will be added in the same order provided. This flag can be repeatedly invoked. For example: <code>--manifest-url-header 'a:hello,b:again,c:world' --manifest-url-header 'b:beautiful'</code> (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's <code>--config</code> flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)</td>
|
||||
</tr>
|
||||
|
||||
<tr>
|
||||
<td colspan="2">--master-service-namespace string Default: <code>default</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">The namespace from which the kubernetes master services should be injected into pods. (DEPRECATED: This flag will be removed in a future version.)</td>
|
||||
</tr>
|
||||
|
||||
|
||||
<tr>
|
||||
<td colspan="2">--max-open-files int Default: 1000000</td>
|
||||
</tr>
|
||||
@@ -898,7 +919,7 @@ WindowsHostProcessContainers=true|false (ALPHA - default=false)<br/>
|
||||
<td colspan="2">--one-output</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">If true, only write logs to their native severity level (vs also writing to each lower severity level).</td>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">If true, only write logs to their native severity level (vs also writing to each lower severity level). (DEPRECATED: will be removed in a future release, see https://github.com/kubernetes/enhancements/tree/master/keps/sig-instrumentation/2845-deprecate-klog-specific-flags-in-k8s-components)</td>
|
||||
</tr>
|
||||
|
||||
<tr>
|
||||
@@ -989,7 +1010,7 @@ WindowsHostProcessContainers=true|false (ALPHA - default=false)<br/>
|
||||
<td colspan="2">--register-node Default: <code>true</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">Register the node with the API server. If <code>--kubeconfig</code> is not provided, this flag is irrelevant, as the Kubelet won't have an API server to register with.</td>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">Register the node with the API server. If <code>--kubeconfig</code> is not provided, this flag is irrelevant, as the Kubelet won't have an API server to register with. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)</td>
|
||||
</tr>
|
||||
|
||||
<tr>
|
||||
@@ -1003,7 +1024,7 @@ WindowsHostProcessContainers=true|false (ALPHA - default=false)<br/>
|
||||
<td colspan="2">--register-with-taints mapStringString</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">Register the node with the given list of taints (comma separated <code><key>=<value>:<effect></code>). No-op if <code>--register-node</code> is <code>false</code>.</td>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">Register the node with the given list of taints (comma separated <code><key>=<value>:<effect></code>). No-op if <code>--register-node</code> is <code>false</code>. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)</td>
|
||||
</tr>
|
||||
|
||||
<tr>
|
||||
@@ -1090,14 +1111,6 @@ WindowsHostProcessContainers=true|false (ALPHA - default=false)<br/>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;"><Warning: Alpha feature> Enable the use of <code>RuntimeDefault</code> as the default seccomp profile for all workloads. The <code>SeccompDefault</code> feature gate must be enabled to allow this flag, which is disabled by default.</td>
|
||||
</tr>
|
||||
|
||||
<tr>
|
||||
<td colspan="2">--seccomp-profile-root string Default: <code>/var/lib/kubelet/seccomp</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;"><Warning: Alpha feature> Directory path for seccomp profiles. (DEPRECATED: will be removed in 1.23, in favor of using the <code><root-dir>/seccomp</code> directory)
|
||||
</td>
|
||||
</tr>
|
||||
|
||||
<tr>
|
||||
<td colspan="2">--serialize-image-pulls Default: <code>true</code></td>
|
||||
</tr>
|
||||
@@ -1109,28 +1122,28 @@ WindowsHostProcessContainers=true|false (ALPHA - default=false)<br/>
|
||||
<td colspan="2">--skip-headers</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">If <code>true</code>, avoid header prefixes in the log messages</td>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">If <code>true</code>, avoid header prefixes in the log messages. (DEPRECATED: will be removed in a future release, see https://github.com/kubernetes/enhancements/tree/master/keps/sig-instrumentation/2845-deprecate-klog-specific-flags-in-k8s-components)</td>
|
||||
</tr>
|
||||
|
||||
<tr>
|
||||
<td colspan="2">--skip-log-headers</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">If <code>true</code>, avoid headers when opening log files</td>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">If <code>true</code>, avoid headers when opening log files. (DEPRECATED: will be removed in a future release, see https://github.com/kubernetes/enhancements/tree/master/keps/sig-instrumentation/2845-deprecate-klog-specific-flags-in-k8s-components)</td>
|
||||
</tr>
|
||||
|
||||
<tr>
|
||||
<td colspan="2">--stderrthreshold int Default: 2</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">logs at or above this threshold go to stderr.</td>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">logs at or above this threshold go to stderr. (DEPRECATED: will be removed in a future release, see https://github.com/kubernetes/enhancements/tree/master/keps/sig-instrumentation/2845-deprecate-klog-specific-flags-in-k8s-components)</td>
|
||||
</tr>
|
||||
|
||||
<tr>
|
||||
<td colspan="2">--streaming-connection-idle-timeout duration Default: <code>4h0m0s</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">Maximum time a streaming connection can be idle before the connection is automatically closed. <code>0</code> indicates no timeout. Example: <code>5m</code>. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's <code>--config</code> flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)</td>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">Maximum time a streaming connection can be idle before the connection is automatically closed. <code>0</code> indicates no timeout. Example: <code>5m</code>. Note: All connections to the kubelet server have a maximum duration of 4 hours. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's <code>--config</code> flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)</td>
|
||||
</tr>
|
||||
|
||||
<tr>
|
||||
@@ -1174,7 +1187,7 @@ WindowsHostProcessContainers=true|false (ALPHA - default=false)<br/>
|
||||
<tr>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;">Comma-separated list of cipher suites for the server. If omitted, the default Go cipher suites will be used.<br/>
|
||||
Preferred values:
|
||||
TLS_AES_128_GCM_SHA256, TLS_AES_256_GCM_SHA384, TLS_CHACHA20_POLY1305_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256, TLS_RSA_WITH_3DES_EDE_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_GCM_SHA384.<br/>
|
||||
TLS_AES_128_GCM_SHA256, TLS_AES_256_GCM_SHA384, TLS_CHACHA20_POLY1305_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_GCM_SHA384<br/>
|
||||
Insecure values:
|
||||
TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_RC4_128_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_RC4_128_SHA, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_RC4_128_SHA.
|
||||
(DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.)
|
||||
|
||||
@@ -489,6 +489,12 @@ PodSpec is a description of a pod.
|
||||
### Beta level
|
||||
|
||||
|
||||
- **ephemeralContainers** ([]<a href="{{< ref "../workload-resources/pod-v1#EphemeralContainer" >}}">EphemeralContainer</a>)
|
||||
|
||||
*Patch strategy: merge on key `name`*
|
||||
|
||||
List of ephemeral containers run in this pod. Ephemeral containers may be run in an existing pod to perform user-initiated actions such as debugging. This list cannot be specified when creating a pod, and it cannot be modified by updating the pod spec. In order to add an ephemeral container to an existing pod, use the pod's ephemeralcontainers subresource. This field is beta-level and available on clusters that haven't disabled the EphemeralContainers feature gate.
|
||||
|
||||
- **preemptionPolicy** (string)
|
||||
|
||||
PreemptionPolicy is the Policy for preempting pods with lower priority. One of Never, PreemptLowerPriority. Defaults to PreemptLowerPriority if unset. This field is beta-level, gated by the NonPreemptingPriority feature-gate.
|
||||
@@ -497,15 +503,6 @@ PodSpec is a description of a pod.
|
||||
|
||||
Overhead represents the resource overhead associated with running a pod for a given RuntimeClass. This field will be autopopulated at admission time by the RuntimeClass admission controller. If the RuntimeClass admission controller is enabled, overhead must not be set in Pod create requests. The RuntimeClass admission controller will reject Pod create requests which have the overhead already set. If RuntimeClass is configured and selected in the PodSpec, Overhead will be set to the value defined in the corresponding RuntimeClass, otherwise it will remain unset and treated as zero. More info: https://git.k8s.io/enhancements/keps/sig-node/688-pod-overhead/README.md This field is beta-level as of Kubernetes v1.18, and is only honored by servers that enable the PodOverhead feature.
|
||||
|
||||
### Alpha level
|
||||
|
||||
|
||||
- **ephemeralContainers** ([]<a href="{{< ref "../workload-resources/pod-v1#EphemeralContainer" >}}">EphemeralContainer</a>)
|
||||
|
||||
*Patch strategy: merge on key `name`*
|
||||
|
||||
List of ephemeral containers run in this pod. Ephemeral containers may be run in an existing pod to perform user-initiated actions such as debugging. This list cannot be specified when creating a pod, and it cannot be modified by updating the pod spec. In order to add an ephemeral container to an existing pod, use the pod's ephemeralcontainers subresource. This field is beta-level and available on clusters that haven't disabled the EphemeralContainers feature gate.
|
||||
|
||||
### Deprecated
|
||||
|
||||
|
||||
@@ -1220,83 +1217,9 @@ This is a beta feature available on clusters that haven't disabled the Ephemeral
|
||||
|
||||
Whether this container should allocate a TTY for itself, also requires 'stdin' to be true. Default is false.
|
||||
|
||||
### Not allowed
|
||||
### Security context
|
||||
|
||||
|
||||
- **ports** ([]ContainerPort)
|
||||
|
||||
*Patch strategy: merge on key `containerPort`*
|
||||
|
||||
*Map: unique values on keys `containerPort, protocol` will be kept during a merge*
|
||||
|
||||
Ports are not allowed for ephemeral containers.
|
||||
|
||||
<a name="ContainerPort"></a>
|
||||
*ContainerPort represents a network port in a single container.*
|
||||
|
||||
- **ports.containerPort** (int32), required
|
||||
|
||||
Number of port to expose on the pod's IP address. This must be a valid port number, 0 \< x \< 65536.
|
||||
|
||||
- **ports.hostIP** (string)
|
||||
|
||||
What host IP to bind the external port to.
|
||||
|
||||
- **ports.hostPort** (int32)
|
||||
|
||||
Number of port to expose on the host. If specified, this must be a valid port number, 0 \< x \< 65536. If HostNetwork is specified, this must match ContainerPort. Most containers do not need this.
|
||||
|
||||
- **ports.name** (string)
|
||||
|
||||
If specified, this must be an IANA_SVC_NAME and unique within the pod. Each named port in a pod must have a unique name. Name for the port that can be referred to by services.
|
||||
|
||||
- **ports.protocol** (string)
|
||||
|
||||
Protocol for port. Must be UDP, TCP, or SCTP. Defaults to "TCP".
|
||||
|
||||
Possible enum values:
|
||||
- `"SCTP"` is the SCTP protocol.
|
||||
- `"TCP"` is the TCP protocol.
|
||||
- `"UDP"` is the UDP protocol.
|
||||
|
||||
- **resources** (ResourceRequirements)
|
||||
|
||||
Resources are not allowed for ephemeral containers. Ephemeral containers use spare resources already allocated to the pod.
|
||||
|
||||
<a name="ResourceRequirements"></a>
|
||||
*ResourceRequirements describes the compute resource requirements.*
|
||||
|
||||
- **resources.limits** (map[string]<a href="{{< ref "../common-definitions/quantity#Quantity" >}}">Quantity</a>)
|
||||
|
||||
Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
|
||||
|
||||
- **resources.requests** (map[string]<a href="{{< ref "../common-definitions/quantity#Quantity" >}}">Quantity</a>)
|
||||
|
||||
Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
|
||||
|
||||
- **lifecycle** (Lifecycle)
|
||||
|
||||
Lifecycle is not allowed for ephemeral containers.
|
||||
|
||||
<a name="Lifecycle"></a>
|
||||
*Lifecycle describes actions that the management system should take in response to container lifecycle events. For the PostStart and PreStop lifecycle handlers, management of the container blocks until the action is complete, unless the container process fails, in which case the handler is aborted.*
|
||||
|
||||
- **lifecycle.postStart** (<a href="{{< ref "../workload-resources/pod-v1#LifecycleHandler" >}}">LifecycleHandler</a>)
|
||||
|
||||
PostStart is called immediately after a container is created. If the handler fails, the container is terminated and restarted according to its restart policy. Other management of the container blocks until the hook completes. More info: https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/#container-hooks
|
||||
|
||||
- **lifecycle.preStop** (<a href="{{< ref "../workload-resources/pod-v1#LifecycleHandler" >}}">LifecycleHandler</a>)
|
||||
|
||||
PreStop is called immediately before a container is terminated due to an API request or management event such as liveness/startup probe failure, preemption, resource contention, etc. The handler is not called if the container crashes or exits. The Pod's termination grace period countdown begins before the PreStop hook is executed. Regardless of the outcome of the handler, the container will eventually terminate within the Pod's termination grace period (unless delayed by finalizers). Other management of the container blocks until the hook completes or until the termination grace period is reached. More info: https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/#container-hooks
|
||||
|
||||
- **livenessProbe** (<a href="{{< ref "../workload-resources/pod-v1#Probe" >}}">Probe</a>)
|
||||
|
||||
Probes are not allowed for ephemeral containers.
|
||||
|
||||
- **readinessProbe** (<a href="{{< ref "../workload-resources/pod-v1#Probe" >}}">Probe</a>)
|
||||
|
||||
Probes are not allowed for ephemeral containers.
|
||||
|
||||
- **securityContext** (SecurityContext)
|
||||
|
||||
Optional: SecurityContext defines the security options the ephemeral container should be run with. If set, the fields of SecurityContext override the equivalent fields of PodSecurityContext.
|
||||
@@ -1415,6 +1338,83 @@ This is a beta feature available on clusters that haven't disabled the Ephemeral
|
||||
|
||||
The UserName in Windows to run the entrypoint of the container process. Defaults to the user specified in image metadata if unspecified. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.
|
||||
|
||||
### Not allowed
|
||||
|
||||
|
||||
- **ports** ([]ContainerPort)
|
||||
|
||||
*Patch strategy: merge on key `containerPort`*
|
||||
|
||||
*Map: unique values on keys `containerPort, protocol` will be kept during a merge*
|
||||
|
||||
Ports are not allowed for ephemeral containers.
|
||||
|
||||
<a name="ContainerPort"></a>
|
||||
*ContainerPort represents a network port in a single container.*
|
||||
|
||||
- **ports.containerPort** (int32), required
|
||||
|
||||
Number of port to expose on the pod's IP address. This must be a valid port number, 0 \< x \< 65536.
|
||||
|
||||
- **ports.hostIP** (string)
|
||||
|
||||
What host IP to bind the external port to.
|
||||
|
||||
- **ports.hostPort** (int32)
|
||||
|
||||
Number of port to expose on the host. If specified, this must be a valid port number, 0 \< x \< 65536. If HostNetwork is specified, this must match ContainerPort. Most containers do not need this.
|
||||
|
||||
- **ports.name** (string)
|
||||
|
||||
If specified, this must be an IANA_SVC_NAME and unique within the pod. Each named port in a pod must have a unique name. Name for the port that can be referred to by services.
|
||||
|
||||
- **ports.protocol** (string)
|
||||
|
||||
Protocol for port. Must be UDP, TCP, or SCTP. Defaults to "TCP".
|
||||
|
||||
Possible enum values:
|
||||
- `"SCTP"` is the SCTP protocol.
|
||||
- `"TCP"` is the TCP protocol.
|
||||
- `"UDP"` is the UDP protocol.
|
||||
|
||||
- **resources** (ResourceRequirements)
|
||||
|
||||
Resources are not allowed for ephemeral containers. Ephemeral containers use spare resources already allocated to the pod.
|
||||
|
||||
<a name="ResourceRequirements"></a>
|
||||
*ResourceRequirements describes the compute resource requirements.*
|
||||
|
||||
- **resources.limits** (map[string]<a href="{{< ref "../common-definitions/quantity#Quantity" >}}">Quantity</a>)
|
||||
|
||||
Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
|
||||
|
||||
- **resources.requests** (map[string]<a href="{{< ref "../common-definitions/quantity#Quantity" >}}">Quantity</a>)
|
||||
|
||||
Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
|
||||
|
||||
- **lifecycle** (Lifecycle)
|
||||
|
||||
Lifecycle is not allowed for ephemeral containers.
|
||||
|
||||
<a name="Lifecycle"></a>
|
||||
*Lifecycle describes actions that the management system should take in response to container lifecycle events. For the PostStart and PreStop lifecycle handlers, management of the container blocks until the action is complete, unless the container process fails, in which case the handler is aborted.*
|
||||
|
||||
- **lifecycle.postStart** (<a href="{{< ref "../workload-resources/pod-v1#LifecycleHandler" >}}">LifecycleHandler</a>)
|
||||
|
||||
PostStart is called immediately after a container is created. If the handler fails, the container is terminated and restarted according to its restart policy. Other management of the container blocks until the hook completes. More info: https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/#container-hooks
|
||||
|
||||
- **lifecycle.preStop** (<a href="{{< ref "../workload-resources/pod-v1#LifecycleHandler" >}}">LifecycleHandler</a>)
|
||||
|
||||
PreStop is called immediately before a container is terminated due to an API request or management event such as liveness/startup probe failure, preemption, resource contention, etc. The handler is not called if the container crashes or exits. The Pod's termination grace period countdown begins before the PreStop hook is executed. Regardless of the outcome of the handler, the container will eventually terminate within the Pod's termination grace period (unless delayed by finalizers). Other management of the container blocks until the hook completes or until the termination grace period is reached. More info: https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/#container-hooks
|
||||
|
||||
- **livenessProbe** (<a href="{{< ref "../workload-resources/pod-v1#Probe" >}}">Probe</a>)
|
||||
|
||||
Probes are not allowed for ephemeral containers.
|
||||
|
||||
- **readinessProbe** (<a href="{{< ref "../workload-resources/pod-v1#Probe" >}}">Probe</a>)
|
||||
|
||||
Probes are not allowed for ephemeral containers.
|
||||
|
||||
- **startupProbe** (<a href="{{< ref "../workload-resources/pod-v1#Probe" >}}">Probe</a>)
|
||||
|
||||
Probes are not allowed for ephemeral containers.
|
||||
|
||||
@@ -12,16 +12,17 @@ Kubernetes contains several tools to help you work with the Kubernetes system.
|
||||
|
||||
<!-- body -->
|
||||
|
||||
## Minikube
|
||||
## crictl
|
||||
|
||||
[`minikube`](https://minikube.sigs.k8s.io/docs/) is a tool that
|
||||
runs a single-node Kubernetes cluster locally on your workstation for
|
||||
development and testing purposes.
|
||||
[`crictl`](https://github.com/kubernetes-sigs/cri-tools) is a command-line
|
||||
interface for inspecting and debugging {{<glossary_tooltip term_id="cri" text="CRI">}}-compatible
|
||||
container runtimes.
|
||||
|
||||
## Dashboard
|
||||
|
||||
[`Dashboard`](/docs/tasks/access-application-cluster/web-ui-dashboard/), the web-based user interface of Kubernetes, allows you to deploy containerized applications
|
||||
to a Kubernetes cluster, troubleshoot them, and manage the cluster and its resources itself.
|
||||
to a Kubernetes cluster, troubleshoot them, and manage the cluster and its
|
||||
resources itself.
|
||||
|
||||
## Helm
|
||||
{{% thirdparty-content single="true" %}}
|
||||
@@ -64,4 +65,10 @@ Kui lets you:
|
||||
* Type in `kubectl` commands and see them execute, even sometimes faster than `kubectl` itself
|
||||
* Query a {{< glossary_tooltip text="Job" term_id="job">}} and see its execution rendered
|
||||
as a waterfall diagram
|
||||
* Click through resources in your cluster using a tabbed UI
|
||||
* Click through resources in your cluster using a tabbed UI
|
||||
|
||||
## Minikube
|
||||
|
||||
[`minikube`](https://minikube.sigs.k8s.io/docs/) is a tool that
|
||||
runs a single-node Kubernetes cluster locally on your workstation for
|
||||
development and testing purposes.
|
||||
@@ -0,0 +1,76 @@
|
||||
---
|
||||
title: Mapping from dockercli to crictl
|
||||
content_type: reference
|
||||
---
|
||||
|
||||
{{% thirdparty-content %}}
|
||||
|
||||
{{<note>}}
|
||||
This page is deprecated and will be removed in Kubernetes 1.27.
|
||||
{{</note>}}
|
||||
|
||||
`crictl` is a command-line interface for {{<glossary_tooltip term_id="cri" text="CRI">}}-compatible container runtimes.
|
||||
You can use it to inspect and debug container runtimes and applications on a
|
||||
Kubernetes node. `crictl` and its source are hosted in the
|
||||
[cri-tools](https://github.com/kubernetes-sigs/cri-tools) repository.
|
||||
|
||||
This page provides a reference for mapping common commands for the `docker`
|
||||
command-line tool into the equivalent commands for `crictl`.
|
||||
|
||||
## Mapping from docker CLI to crictl
|
||||
|
||||
The exact versions for the mapping table are for `docker` CLI v1.40 and `crictl`
|
||||
v1.19.0. This list is not exhaustive. For example, it doesn't include
|
||||
experimental `docker` CLI commands.
|
||||
|
||||
{{< note >}}
|
||||
The output format of `crictl` is similar to `docker` CLI, despite some missing
|
||||
columns for some CLI. Make sure to check output for the specific command if your
|
||||
command output is being parsed programmatically.
|
||||
{{< /note >}}
|
||||
|
||||
### Retrieve debugging information
|
||||
|
||||
{{< table caption="mapping from docker cli to crictl - retrieve debugging information" >}}
|
||||
docker cli | crictl | Description | Unsupported Features
|
||||
-- | -- | -- | --
|
||||
`attach` | `attach` | Attach to a running container | `--detach-keys`, `--sig-proxy`
|
||||
`exec` | `exec` | Run a command in a running container | `--privileged`, `--user`, `--detach-keys`
|
||||
`images` | `images` | List images |
|
||||
`info` | `info` | Display system-wide information |
|
||||
`inspect` | `inspect`, `inspecti` | Return low-level information on a container, image or task |
|
||||
`logs` | `logs` | Fetch the logs of a container | `--details`
|
||||
`ps` | `ps` | List containers |
|
||||
`stats` | `stats` | Display a live stream of container(s) resource usage statistics | Column: NET/BLOCK I/O, PIDs
|
||||
`version` | `version` | Show the runtime (Docker, ContainerD, or others) version information |
|
||||
{{< /table >}}
|
||||
|
||||
### Perform Changes
|
||||
|
||||
{{< table caption="mapping from docker cli to crictl - perform changes" >}}
|
||||
docker cli | crictl | Description | Unsupported Features
|
||||
-- | -- | -- | --
|
||||
`create` | `create` | Create a new container |
|
||||
`kill` | `stop` (timeout = 0) | Kill one or more running container | `--signal`
|
||||
`pull` | `pull` | Pull an image or a repository from a registry | `--all-tags`, `--disable-content-trust`
|
||||
`rm` | `rm` | Remove one or more containers |
|
||||
`rmi` | `rmi` | Remove one or more images |
|
||||
`run` | `run` | Run a command in a new container |
|
||||
`start` | `start` | Start one or more stopped containers | `--detach-keys`
|
||||
`stop` | `stop` | Stop one or more running containers |
|
||||
`update` | `update` | Update configuration of one or more containers | `--restart`, `--blkio-weight` and some other resource limit not supported by CRI.
|
||||
{{< /table >}}
|
||||
|
||||
### Supported only in crictl
|
||||
|
||||
{{< table caption="mapping from docker cli to crictl - supported only in crictl" >}}
|
||||
crictl | Description
|
||||
-- | --
|
||||
`imagefsinfo` | Return image filesystem info
|
||||
`inspectp` | Display the status of one or more pods
|
||||
`port-forward` | Forward local port to a pod
|
||||
`pods` | List pods
|
||||
`runp` | Run a new pod
|
||||
`rmp` | Remove one or more pods
|
||||
`stopp` | Stop one or more running pods
|
||||
{{< /table >}}
|
||||
@@ -83,13 +83,17 @@ If systemd doesn't use cgroup v2 by default, you can configure the system to use
|
||||
`systemd.unified_cgroup_hierarchy=1` to the kernel command line.
|
||||
|
||||
```shell
|
||||
# dnf install -y grubby && \
|
||||
# This example is for a Linux OS that uses the DNF package manager
|
||||
# Your system might use a different method for setting the command line
|
||||
# that the Linux kernel uses.
|
||||
sudo dnf install -y grubby && \
|
||||
sudo grubby \
|
||||
--update-kernel=ALL \
|
||||
--args="systemd.unified_cgroup_hierarchy=1"
|
||||
```
|
||||
|
||||
To apply the configuration, it is necessary to reboot the node.
|
||||
If you change the command line for the kernel, you must reboot the node before your
|
||||
change takes effect.
|
||||
|
||||
There should not be any noticeable difference in the user experience when switching to cgroup v2, unless
|
||||
users are accessing the cgroup file system directly, either on the node or from within the containers.
|
||||
@@ -168,7 +172,7 @@ installing the `containerd.io` package can be found at
|
||||
{{% /tab %}}
|
||||
{{% tab name="Windows (PowerShell)" %}}
|
||||
|
||||
Start a Powershell session, set `$Version` to the desired version (ex: `$Version=1.4.3`),
|
||||
Start a Powershell session, set `$Version` to the desired version (ex: `$Version="1.4.3"`),
|
||||
and then run the following commands:
|
||||
|
||||
1. Download containerd:
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
reviewers:
|
||||
- sig-cluster-lifecycle
|
||||
title: Options for Highly Available topology
|
||||
title: Options for Highly Available Topology
|
||||
content_type: concept
|
||||
weight: 50
|
||||
---
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
reviewers:
|
||||
- sig-cluster-lifecycle
|
||||
title: Creating Highly Available clusters with kubeadm
|
||||
title: Creating Highly Available Clusters with kubeadm
|
||||
content_type: task
|
||||
weight: 60
|
||||
---
|
||||
@@ -12,17 +12,17 @@ This page explains two different approaches to setting up a highly available Kub
|
||||
cluster using kubeadm:
|
||||
|
||||
- With stacked control plane nodes. This approach requires less infrastructure. The etcd members
|
||||
and control plane nodes are co-located.
|
||||
and control plane nodes are co-located.
|
||||
- With an external etcd cluster. This approach requires more infrastructure. The
|
||||
control plane nodes and etcd members are separated.
|
||||
control plane nodes and etcd members are separated.
|
||||
|
||||
Before proceeding, you should carefully consider which approach best meets the needs of your applications
|
||||
and environment. [This comparison topic](/docs/setup/production-environment/tools/kubeadm/ha-topology/) outlines the advantages and disadvantages of each.
|
||||
and environment. [Options for Highly Available topology](/docs/setup/production-environment/tools/kubeadm/ha-topology/) outlines the advantages and disadvantages of each.
|
||||
|
||||
If you encounter issues with setting up the HA cluster, please provide us with feedback
|
||||
If you encounter issues with setting up the HA cluster, please report these
|
||||
in the kubeadm [issue tracker](https://github.com/kubernetes/kubeadm/issues/new).
|
||||
|
||||
See also [The upgrade documentation](/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/).
|
||||
See also the [upgrade documentation](/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/).
|
||||
|
||||
{{< caution >}}
|
||||
This page does not address running your cluster on a cloud provider. In a cloud
|
||||
@@ -32,22 +32,80 @@ LoadBalancer, or with dynamic PersistentVolumes.
|
||||
|
||||
## {{% heading "prerequisites" %}}
|
||||
|
||||
The prerequisites depend on which topology you have selected for your cluster's
|
||||
control plane:
|
||||
|
||||
For both methods you need this infrastructure:
|
||||
{{< tabs name="prerequisite_tabs" >}}
|
||||
{{% tab name="Stacked etcd" %}}
|
||||
<!--
|
||||
note to reviewers: these prerequisites should match the start of the
|
||||
external etc tab
|
||||
-->
|
||||
|
||||
- Three machines that meet [kubeadm's minimum requirements](/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#before-you-begin) for
|
||||
the control-plane nodes
|
||||
- Three machines that meet [kubeadm's minimum
|
||||
You need:
|
||||
|
||||
- Three or more machines that meet [kubeadm's minimum requirements](/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#before-you-begin) for
|
||||
the control-plane nodes. Having an odd number of control plane nodes can help
|
||||
with leader selection in the case of machine or zone failure.
|
||||
- including a {{< glossary_tooltip text="container runtime" term_id="container-runtime" >}}, already set up and working
|
||||
- Three or more machines that meet [kubeadm's minimum
|
||||
requirements](/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#before-you-begin) for the workers
|
||||
- including a container runtime, already set up and working
|
||||
- Full network connectivity between all machines in the cluster (public or
|
||||
private network)
|
||||
- sudo privileges on all machines
|
||||
- Superuser privileges on all machines using `sudo`
|
||||
- You can use a different tool; this guide uses `sudo` in the examples.
|
||||
- SSH access from one device to all nodes in the system
|
||||
- `kubeadm` and `kubelet` installed on all machines. `kubectl` is optional.
|
||||
- `kubeadm` and `kubelet` already installed on all machines.
|
||||
|
||||
For the external etcd cluster only, you also need:
|
||||
_See [Stacked etcd topology](/docs/setup/production-environment/tools/kubeadm/ha-topology/#stacked-etcd-topology) for context._
|
||||
|
||||
- Three additional machines for etcd members
|
||||
{{% /tab %}}
|
||||
{{% tab name="External etcd" %}}
|
||||
<!--
|
||||
note to reviewers: these prerequisites should match the start of the
|
||||
stacked etc tab
|
||||
-->
|
||||
You need:
|
||||
|
||||
- Three or more machines that meet [kubeadm's minimum requirements](/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#before-you-begin) for
|
||||
the control-plane nodes. Having an odd number of control plane nodes can help
|
||||
with leader selection in the case of machine or zone failure.
|
||||
- including a {{< glossary_tooltip text="container runtime" term_id="container-runtime" >}}, already set up and working
|
||||
- Three or more machines that meet [kubeadm's minimum
|
||||
requirements](/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#before-you-begin) for the workers
|
||||
- including a container runtime, already set up and working
|
||||
- Full network connectivity between all machines in the cluster (public or
|
||||
private network)
|
||||
- Superuser privileges on all machines using `sudo`
|
||||
- You can use a different tool; this guide uses `sudo` in the examples.
|
||||
- SSH access from one device to all nodes in the system
|
||||
- `kubeadm` and `kubelet` already installed on all machines.
|
||||
|
||||
<!-- end of shared prerequisites -->
|
||||
|
||||
And you also need:
|
||||
- Three or more additional machines, that will become etcd cluster members.
|
||||
Having an odd number of members in the etcd cluster is a requirement for achieving
|
||||
optimal voting quorum.
|
||||
- These machines again need to have `kubeadm` and `kubelet` installed.
|
||||
- These machines also require a container runtime, that is already set up and working.
|
||||
|
||||
_See [External etcd topology](/docs/setup/production-environment/tools/kubeadm/ha-topology/#external-etcd-topology) for context._
|
||||
{{% /tab %}}
|
||||
{{< /tabs >}}
|
||||
|
||||
### Container images
|
||||
|
||||
Each host should have access read and fetch images from the Kubernetes container image registry, `k8s.gcr.io`.
|
||||
If you want to deploy a highly-available cluster where the hosts do not have access to pull images, this is possible. You must ensure by some other means that the correct container images are already available on the relevant hosts.
|
||||
|
||||
### Command line interface {#kubectl}
|
||||
|
||||
To manage Kubernetes once your cluster is set up, you should
|
||||
[install kubectl](/docs/tasks/tools/#kubectl) on your PC. It is also useful
|
||||
to install the `kubectl` tool on each control plane node, as this can be
|
||||
helpful for troubleshooting.
|
||||
|
||||
<!-- steps -->
|
||||
|
||||
@@ -60,147 +118,146 @@ There are many configurations for load balancers. The following example is only
|
||||
option. Your cluster requirements may need a different configuration.
|
||||
{{< /note >}}
|
||||
|
||||
1. Create a kube-apiserver load balancer with a name that resolves to DNS.
|
||||
1. Create a kube-apiserver load balancer with a name that resolves to DNS.
|
||||
|
||||
- In a cloud environment you should place your control plane nodes behind a TCP
|
||||
forwarding load balancer. This load balancer distributes traffic to all
|
||||
healthy control plane nodes in its target list. The health check for
|
||||
an apiserver is a TCP check on the port the kube-apiserver listens on
|
||||
(default value `:6443`).
|
||||
- In a cloud environment you should place your control plane nodes behind a TCP
|
||||
forwarding load balancer. This load balancer distributes traffic to all
|
||||
healthy control plane nodes in its target list. The health check for
|
||||
an apiserver is a TCP check on the port the kube-apiserver listens on
|
||||
(default value `:6443`).
|
||||
|
||||
- It is not recommended to use an IP address directly in a cloud environment.
|
||||
- It is not recommended to use an IP address directly in a cloud environment.
|
||||
|
||||
- The load balancer must be able to communicate with all control plane nodes
|
||||
on the apiserver port. It must also allow incoming traffic on its
|
||||
listening port.
|
||||
- The load balancer must be able to communicate with all control plane nodes
|
||||
on the apiserver port. It must also allow incoming traffic on its
|
||||
listening port.
|
||||
|
||||
- Make sure the address of the load balancer always matches
|
||||
the address of kubeadm's `ControlPlaneEndpoint`.
|
||||
- Make sure the address of the load balancer always matches
|
||||
the address of kubeadm's `ControlPlaneEndpoint`.
|
||||
|
||||
- Read the [Options for Software Load Balancing](https://git.k8s.io/kubeadm/docs/ha-considerations.md#options-for-software-load-balancing)
|
||||
guide for more details.
|
||||
- Read the [Options for Software Load Balancing](https://git.k8s.io/kubeadm/docs/ha-considerations.md#options-for-software-load-balancing)
|
||||
guide for more details.
|
||||
|
||||
1. Add the first control plane nodes to the load balancer and test the
|
||||
connection:
|
||||
1. Add the first control plane node to the load balancer, and test the
|
||||
connection:
|
||||
|
||||
```sh
|
||||
nc -v LOAD_BALANCER_IP PORT
|
||||
```
|
||||
```shell
|
||||
nc -v <LOAD_BALANCER_IP> <PORT>
|
||||
```
|
||||
|
||||
- A connection refused error is expected because the apiserver is not yet
|
||||
running. A timeout, however, means the load balancer cannot communicate
|
||||
with the control plane node. If a timeout occurs, reconfigure the load
|
||||
balancer to communicate with the control plane node.
|
||||
A connection refused error is expected because the API server is not yet
|
||||
running. A timeout, however, means the load balancer cannot communicate
|
||||
with the control plane node. If a timeout occurs, reconfigure the load
|
||||
balancer to communicate with the control plane node.
|
||||
|
||||
1. Add the remaining control plane nodes to the load balancer target group.
|
||||
1. Add the remaining control plane nodes to the load balancer target group.
|
||||
|
||||
## Stacked control plane and etcd nodes
|
||||
|
||||
### Steps for the first control plane node
|
||||
|
||||
1. Initialize the control plane:
|
||||
1. Initialize the control plane:
|
||||
|
||||
```sh
|
||||
sudo kubeadm init --control-plane-endpoint "LOAD_BALANCER_DNS:LOAD_BALANCER_PORT" --upload-certs
|
||||
```
|
||||
```sh
|
||||
sudo kubeadm init --control-plane-endpoint "LOAD_BALANCER_DNS:LOAD_BALANCER_PORT" --upload-certs
|
||||
```
|
||||
|
||||
- You can use the `--kubernetes-version` flag to set the Kubernetes version to use.
|
||||
It is recommended that the versions of kubeadm, kubelet, kubectl and Kubernetes match.
|
||||
- The `--control-plane-endpoint` flag should be set to the address or DNS and port of the load balancer.
|
||||
- You can use the `--kubernetes-version` flag to set the Kubernetes version to use.
|
||||
It is recommended that the versions of kubeadm, kubelet, kubectl and Kubernetes match.
|
||||
- The `--control-plane-endpoint` flag should be set to the address or DNS and port of the load balancer.
|
||||
|
||||
- The `--upload-certs` flag is used to upload the certificates that should be shared
|
||||
across all the control-plane instances to the cluster. If instead, you prefer to copy certs across
|
||||
control-plane nodes manually or using automation tools, please remove this flag and refer to [Manual
|
||||
certificate distribution](#manual-certs) section below.
|
||||
- The `--upload-certs` flag is used to upload the certificates that should be shared
|
||||
across all the control-plane instances to the cluster. If instead, you prefer to copy certs across
|
||||
control-plane nodes manually or using automation tools, please remove this flag and refer to [Manual
|
||||
certificate distribution](#manual-certs) section below.
|
||||
|
||||
{{< note >}}
|
||||
The `kubeadm init` flags `--config` and `--certificate-key` cannot be mixed, therefore if you want
|
||||
to use the [kubeadm configuration](/docs/reference/config-api/kubeadm-config.v1beta3/)
|
||||
you must add the `certificateKey` field in the appropriate config locations
|
||||
(under `InitConfiguration` and `JoinConfiguration: controlPlane`).
|
||||
{{< /note >}}
|
||||
{{< note >}}
|
||||
The `kubeadm init` flags `--config` and `--certificate-key` cannot be mixed, therefore if you want
|
||||
to use the [kubeadm configuration](/docs/reference/config-api/kubeadm-config.v1beta3/)
|
||||
you must add the `certificateKey` field in the appropriate config locations
|
||||
(under `InitConfiguration` and `JoinConfiguration: controlPlane`).
|
||||
{{< /note >}}
|
||||
|
||||
{{< note >}}
|
||||
Some CNI network plugins require additional configuration, for example specifying the pod IP CIDR, while others do not.
|
||||
See the [CNI network documentation](/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/#pod-network).
|
||||
To add a pod CIDR pass the flag `--pod-network-cidr`, or if you are using a kubeadm configuration file
|
||||
set the `podSubnet` field under the `networking` object of `ClusterConfiguration`.
|
||||
{{< /note >}}
|
||||
{{< note >}}
|
||||
Some CNI network plugins require additional configuration, for example specifying the pod IP CIDR, while others do not.
|
||||
See the [CNI network documentation](/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/#pod-network).
|
||||
To add a pod CIDR pass the flag `--pod-network-cidr`, or if you are using a kubeadm configuration file
|
||||
set the `podSubnet` field under the `networking` object of `ClusterConfiguration`.
|
||||
{{< /note >}}
|
||||
|
||||
- The output looks similar to:
|
||||
The output looks similar to:
|
||||
|
||||
```sh
|
||||
...
|
||||
You can now join any number of control-plane node by running the following command on each as a root:
|
||||
kubeadm join 192.168.0.200:6443 --token 9vr73a.a8uxyaju799qwdjv --discovery-token-ca-cert-hash sha256:7c2e69131a36ae2a042a339b33381c6d0d43887e2de83720eff5359e26aec866 --control-plane --certificate-key f8902e114ef118304e561c3ecd4d0b543adc226b7a07f675f56564185ffe0c07
|
||||
```sh
|
||||
...
|
||||
You can now join any number of control-plane node by running the following command on each as a root:
|
||||
kubeadm join 192.168.0.200:6443 --token 9vr73a.a8uxyaju799qwdjv --discovery-token-ca-cert-hash sha256:7c2e69131a36ae2a042a339b33381c6d0d43887e2de83720eff5359e26aec866 --control-plane --certificate-key f8902e114ef118304e561c3ecd4d0b543adc226b7a07f675f56564185ffe0c07
|
||||
|
||||
Please note that the certificate-key gives access to cluster sensitive data, keep it secret!
|
||||
As a safeguard, uploaded-certs will be deleted in two hours; If necessary, you can use kubeadm init phase upload-certs to reload certs afterward.
|
||||
Please note that the certificate-key gives access to cluster sensitive data, keep it secret!
|
||||
As a safeguard, uploaded-certs will be deleted in two hours; If necessary, you can use kubeadm init phase upload-certs to reload certs afterward.
|
||||
|
||||
Then you can join any number of worker nodes by running the following on each as root:
|
||||
kubeadm join 192.168.0.200:6443 --token 9vr73a.a8uxyaju799qwdjv --discovery-token-ca-cert-hash sha256:7c2e69131a36ae2a042a339b33381c6d0d43887e2de83720eff5359e26aec866
|
||||
```
|
||||
Then you can join any number of worker nodes by running the following on each as root:
|
||||
kubeadm join 192.168.0.200:6443 --token 9vr73a.a8uxyaju799qwdjv --discovery-token-ca-cert-hash sha256:7c2e69131a36ae2a042a339b33381c6d0d43887e2de83720eff5359e26aec866
|
||||
```
|
||||
|
||||
- Copy this output to a text file. You will need it later to join control plane and worker nodes to the cluster.
|
||||
- When `--upload-certs` is used with `kubeadm init`, the certificates of the primary control plane
|
||||
are encrypted and uploaded in the `kubeadm-certs` Secret.
|
||||
- To re-upload the certificates and generate a new decryption key, use the following command on a control plane
|
||||
node that is already joined to the cluster:
|
||||
- Copy this output to a text file. You will need it later to join control plane and worker nodes to
|
||||
the cluster.
|
||||
- When `--upload-certs` is used with `kubeadm init`, the certificates of the primary control plane
|
||||
are encrypted and uploaded in the `kubeadm-certs` Secret.
|
||||
- To re-upload the certificates and generate a new decryption key, use the following command on a
|
||||
control plane
|
||||
node that is already joined to the cluster:
|
||||
|
||||
```sh
|
||||
sudo kubeadm init phase upload-certs --upload-certs
|
||||
```
|
||||
```sh
|
||||
sudo kubeadm init phase upload-certs --upload-certs
|
||||
```
|
||||
|
||||
- You can also specify a custom `--certificate-key` during `init` that can later be used by `join`.
|
||||
To generate such a key you can use the following command:
|
||||
- You can also specify a custom `--certificate-key` during `init` that can later be used by `join`.
|
||||
To generate such a key you can use the following command:
|
||||
|
||||
```sh
|
||||
kubeadm certs certificate-key
|
||||
```
|
||||
```sh
|
||||
kubeadm certs certificate-key
|
||||
```
|
||||
|
||||
{{< note >}}
|
||||
The `kubeadm-certs` Secret and decryption key expire after two hours.
|
||||
{{< /note >}}
|
||||
{{< note >}}
|
||||
The `kubeadm-certs` Secret and decryption key expire after two hours.
|
||||
{{< /note >}}
|
||||
|
||||
{{< caution >}}
|
||||
As stated in the command output, the certificate key gives access to cluster sensitive data, keep it secret!
|
||||
{{< /caution >}}
|
||||
{{< caution >}}
|
||||
As stated in the command output, the certificate key gives access to cluster sensitive data, keep it secret!
|
||||
{{< /caution >}}
|
||||
|
||||
1. Apply the CNI plugin of your choice:
|
||||
[Follow these instructions](/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/#pod-network)
|
||||
to install the CNI provider. Make sure the configuration corresponds to the Pod CIDR specified in the kubeadm configuration file if applicable.
|
||||
|
||||
{{< note >}}
|
||||
You must pick a network plugin that suits your use case and deploy it before you move on to next step.
|
||||
If you don't do this, you will not be able to launch your cluster properly.
|
||||
{{< /note >}}
|
||||
1. Apply the CNI plugin of your choice:
|
||||
[Follow these instructions](/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/#pod-network)
|
||||
to install the CNI provider. Make sure the configuration corresponds to the Pod CIDR specified in the
|
||||
kubeadm configuration file (if applicable).
|
||||
|
||||
1. Type the following and watch the pods of the control plane components get started:
|
||||
{{< note >}}
|
||||
You must pick a network plugin that suits your use case and deploy it before you move on to next step.
|
||||
If you don't do this, you will not be able to launch your cluster properly.
|
||||
{{< /note >}}
|
||||
|
||||
```sh
|
||||
kubectl get pod -n kube-system -w
|
||||
```
|
||||
1. Type the following and watch the pods of the control plane components get started:
|
||||
|
||||
```sh
|
||||
kubectl get pod -n kube-system -w
|
||||
```
|
||||
|
||||
### Steps for the rest of the control plane nodes
|
||||
|
||||
{{< note >}}
|
||||
Since kubeadm version 1.15 you can join multiple control-plane nodes in parallel.
|
||||
Prior to this version, you must join new control plane nodes sequentially, only after
|
||||
the first node has finished initializing.
|
||||
{{< /note >}}
|
||||
|
||||
For each additional control plane node you should:
|
||||
|
||||
1. Execute the join command that was previously given to you by the `kubeadm init` output on the first node.
|
||||
It should look something like this:
|
||||
1. Execute the join command that was previously given to you by the `kubeadm init` output on the first node.
|
||||
It should look something like this:
|
||||
|
||||
```sh
|
||||
sudo kubeadm join 192.168.0.200:6443 --token 9vr73a.a8uxyaju799qwdjv --discovery-token-ca-cert-hash sha256:7c2e69131a36ae2a042a339b33381c6d0d43887e2de83720eff5359e26aec866 --control-plane --certificate-key f8902e114ef118304e561c3ecd4d0b543adc226b7a07f675f56564185ffe0c07
|
||||
```
|
||||
```sh
|
||||
sudo kubeadm join 192.168.0.200:6443 --token 9vr73a.a8uxyaju799qwdjv --discovery-token-ca-cert-hash sha256:7c2e69131a36ae2a042a339b33381c6d0d43887e2de83720eff5359e26aec866 --control-plane --certificate-key f8902e114ef118304e561c3ecd4d0b543adc226b7a07f675f56564185ffe0c07
|
||||
```
|
||||
|
||||
- The `--control-plane` flag tells `kubeadm join` to create a new control plane.
|
||||
- The `--certificate-key ...` will cause the control plane certificates to be downloaded
|
||||
from the `kubeadm-certs` Secret in the cluster and be decrypted using the given key.
|
||||
- The `--control-plane` flag tells `kubeadm join` to create a new control plane.
|
||||
- The `--certificate-key ...` will cause the control plane certificates to be downloaded
|
||||
from the `kubeadm-certs` Secret in the cluster and be decrypted using the given key.
|
||||
|
||||
You can join multiple control-plane nodes in parallel.
|
||||
|
||||
## External etcd nodes
|
||||
|
||||
@@ -210,64 +267,69 @@ in the kubeadm config file.
|
||||
|
||||
### Set up the etcd cluster
|
||||
|
||||
1. Follow [these instructions](/docs/setup/production-environment/tools/kubeadm/setup-ha-etcd-with-kubeadm/) to set up the etcd cluster.
|
||||
1. Follow these [instructions](/docs/setup/production-environment/tools/kubeadm/setup-ha-etcd-with-kubeadm/) to set up the etcd cluster.
|
||||
|
||||
1. Setup SSH as described [here](#manual-certs).
|
||||
1. Setup SSH as described [here](#manual-certs).
|
||||
|
||||
1. Copy the following files from any etcd node in the cluster to the first control plane node:
|
||||
1. Copy the following files from any etcd node in the cluster to the first control plane node:
|
||||
|
||||
```sh
|
||||
export CONTROL_PLANE="ubuntu@10.0.0.7"
|
||||
scp /etc/kubernetes/pki/etcd/ca.crt "${CONTROL_PLANE}":
|
||||
scp /etc/kubernetes/pki/apiserver-etcd-client.crt "${CONTROL_PLANE}":
|
||||
scp /etc/kubernetes/pki/apiserver-etcd-client.key "${CONTROL_PLANE}":
|
||||
```
|
||||
```sh
|
||||
export CONTROL_PLANE="ubuntu@10.0.0.7"
|
||||
scp /etc/kubernetes/pki/etcd/ca.crt "${CONTROL_PLANE}":
|
||||
scp /etc/kubernetes/pki/apiserver-etcd-client.crt "${CONTROL_PLANE}":
|
||||
scp /etc/kubernetes/pki/apiserver-etcd-client.key "${CONTROL_PLANE}":
|
||||
```
|
||||
|
||||
- Replace the value of `CONTROL_PLANE` with the `user@host` of the first control-plane node.
|
||||
- Replace the value of `CONTROL_PLANE` with the `user@host` of the first control-plane node.
|
||||
|
||||
### Set up the first control plane node
|
||||
|
||||
1. Create a file called `kubeadm-config.yaml` with the following contents:
|
||||
1. Create a file called `kubeadm-config.yaml` with the following contents:
|
||||
|
||||
apiVersion: kubeadm.k8s.io/v1beta3
|
||||
kind: ClusterConfiguration
|
||||
kubernetesVersion: stable
|
||||
controlPlaneEndpoint: "LOAD_BALANCER_DNS:LOAD_BALANCER_PORT"
|
||||
etcd:
|
||||
external:
|
||||
endpoints:
|
||||
- https://ETCD_0_IP:2379
|
||||
- https://ETCD_1_IP:2379
|
||||
- https://ETCD_2_IP:2379
|
||||
caFile: /etc/kubernetes/pki/etcd/ca.crt
|
||||
certFile: /etc/kubernetes/pki/apiserver-etcd-client.crt
|
||||
keyFile: /etc/kubernetes/pki/apiserver-etcd-client.key
|
||||
|
||||
{{< note >}}
|
||||
The difference between stacked etcd and external etcd here is that the external etcd setup requires
|
||||
a configuration file with the etcd endpoints under the `external` object for `etcd`.
|
||||
In the case of the stacked etcd topology this is managed automatically.
|
||||
{{< /note >}}
|
||||
```yaml
|
||||
---
|
||||
apiVersion: kubeadm.k8s.io/v1beta3
|
||||
kind: ClusterConfiguration
|
||||
kubernetesVersion: stable
|
||||
controlPlaneEndpoint: "LOAD_BALANCER_DNS:LOAD_BALANCER_PORT" # change this (see below)
|
||||
etcd:
|
||||
external:
|
||||
endpoints:
|
||||
- https://ETCD_0_IP:2379 # change ETCD_0_IP appropriately
|
||||
- https://ETCD_1_IP:2379 # change ETCD_1_IP appropriately
|
||||
- https://ETCD_2_IP:2379 # change ETCD_2_IP appropriately
|
||||
caFile: /etc/kubernetes/pki/etcd/ca.crt
|
||||
certFile: /etc/kubernetes/pki/apiserver-etcd-client.crt
|
||||
keyFile: /etc/kubernetes/pki/apiserver-etcd-client.key
|
||||
```
|
||||
|
||||
- Replace the following variables in the config template with the appropriate values for your cluster:
|
||||
{{< note >}}
|
||||
The difference between stacked etcd and external etcd here is that the external etcd setup requires
|
||||
a configuration file with the etcd endpoints under the `external` object for `etcd`.
|
||||
In the case of the stacked etcd topology, this is managed automatically.
|
||||
{{< /note >}}
|
||||
|
||||
- `LOAD_BALANCER_DNS`
|
||||
- `LOAD_BALANCER_PORT`
|
||||
- `ETCD_0_IP`
|
||||
- `ETCD_1_IP`
|
||||
- `ETCD_2_IP`
|
||||
- Replace the following variables in the config template with the appropriate values for your cluster:
|
||||
|
||||
- `LOAD_BALANCER_DNS`
|
||||
- `LOAD_BALANCER_PORT`
|
||||
- `ETCD_0_IP`
|
||||
- `ETCD_1_IP`
|
||||
- `ETCD_2_IP`
|
||||
|
||||
The following steps are similar to the stacked etcd setup:
|
||||
|
||||
1. Run `sudo kubeadm init --config kubeadm-config.yaml --upload-certs` on this node.
|
||||
1. Run `sudo kubeadm init --config kubeadm-config.yaml --upload-certs` on this node.
|
||||
|
||||
1. Write the output join commands that are returned to a text file for later use.
|
||||
1. Write the output join commands that are returned to a text file for later use.
|
||||
|
||||
1. Apply the CNI plugin of your choice. The given example is for Weave Net:
|
||||
1. Apply the CNI plugin of your choice.
|
||||
|
||||
```sh
|
||||
kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"
|
||||
```
|
||||
{{< note >}}
|
||||
You must pick a network plugin that suits your use case and deploy it before you move on to next step.
|
||||
If you don't do this, you will not be able to launch your cluster properly.
|
||||
{{< /note >}}
|
||||
|
||||
### Steps for the rest of the control plane nodes
|
||||
|
||||
@@ -275,7 +337,7 @@ The steps are the same as for the stacked etcd setup:
|
||||
|
||||
- Make sure the first control plane node is fully initialized.
|
||||
- Join each control plane node with the join command you saved to a text file. It's recommended
|
||||
to join the control plane nodes one at a time.
|
||||
to join the control plane nodes one at a time.
|
||||
- Don't forget that the decryption key from `--certificate-key` expires after two hours, by default.
|
||||
|
||||
## Common tasks after bootstrapping control plane
|
||||
@@ -295,79 +357,81 @@ If you choose to not use `kubeadm init` with the `--upload-certs` flag this mean
|
||||
you are going to have to manually copy the certificates from the primary control plane node to the
|
||||
joining control plane nodes.
|
||||
|
||||
There are many ways to do this. In the following example we are using `ssh` and `scp`:
|
||||
There are many ways to do this. The following example uses `ssh` and `scp`:
|
||||
|
||||
SSH is required if you want to control all nodes from a single machine.
|
||||
|
||||
1. Enable ssh-agent on your main device that has access to all other nodes in
|
||||
the system:
|
||||
1. Enable ssh-agent on your main device that has access to all other nodes in
|
||||
the system:
|
||||
|
||||
```
|
||||
eval $(ssh-agent)
|
||||
```
|
||||
eval $(ssh-agent)
|
||||
```
|
||||
|
||||
1. Add your SSH identity to the session:
|
||||
|
||||
```
|
||||
ssh-add ~/.ssh/path_to_private_key
|
||||
```
|
||||
|
||||
1. SSH between nodes to check that the connection is working correctly.
|
||||
|
||||
- When you SSH to any node, add the `-A` flag. This flag allows the node that you
|
||||
have logged into via SSH to access the SSH agent on your PC. Consider alternative
|
||||
methods if you do not fully trust the security of your user session on the node.
|
||||
|
||||
```
|
||||
ssh -A 10.0.0.7
|
||||
```
|
||||
|
||||
- When using sudo on any node, make sure to preserve the environment so SSH
|
||||
forwarding works:
|
||||
|
||||
```
|
||||
sudo -E -s
|
||||
```
|
||||
|
||||
1. After configuring SSH on all the nodes you should run the following script on the first
|
||||
control plane node after running `kubeadm init`. This script will copy the certificates from
|
||||
the first control plane node to the other control plane nodes:
|
||||
|
||||
In the following example, replace `CONTROL_PLANE_IPS` with the IP addresses of the
|
||||
other control plane nodes.
|
||||
```sh
|
||||
USER=ubuntu # customizable
|
||||
CONTROL_PLANE_IPS="10.0.0.7 10.0.0.8"
|
||||
for host in ${CONTROL_PLANE_IPS}; do
|
||||
scp /etc/kubernetes/pki/ca.crt "${USER}"@$host:
|
||||
scp /etc/kubernetes/pki/ca.key "${USER}"@$host:
|
||||
scp /etc/kubernetes/pki/sa.key "${USER}"@$host:
|
||||
scp /etc/kubernetes/pki/sa.pub "${USER}"@$host:
|
||||
scp /etc/kubernetes/pki/front-proxy-ca.crt "${USER}"@$host:
|
||||
scp /etc/kubernetes/pki/front-proxy-ca.key "${USER}"@$host:
|
||||
scp /etc/kubernetes/pki/etcd/ca.crt "${USER}"@$host:etcd-ca.crt
|
||||
# Skip the next line if you are using external etcd
|
||||
scp /etc/kubernetes/pki/etcd/ca.key "${USER}"@$host:etcd-ca.key
|
||||
done
|
||||
```
|
||||
|
||||
1. Add your SSH identity to the session:
|
||||
|
||||
```
|
||||
ssh-add ~/.ssh/path_to_private_key
|
||||
```
|
||||
|
||||
1. SSH between nodes to check that the connection is working correctly.
|
||||
|
||||
- When you SSH to any node, make sure to add the `-A` flag:
|
||||
|
||||
```
|
||||
ssh -A 10.0.0.7
|
||||
```
|
||||
|
||||
- When using sudo on any node, make sure to preserve the environment so SSH
|
||||
forwarding works:
|
||||
|
||||
```
|
||||
sudo -E -s
|
||||
```
|
||||
|
||||
1. After configuring SSH on all the nodes you should run the following script on the first control plane node after
|
||||
running `kubeadm init`. This script will copy the certificates from the first control plane node to the other
|
||||
control plane nodes:
|
||||
|
||||
In the following example, replace `CONTROL_PLANE_IPS` with the IP addresses of the
|
||||
other control plane nodes.
|
||||
```sh
|
||||
USER=ubuntu # customizable
|
||||
CONTROL_PLANE_IPS="10.0.0.7 10.0.0.8"
|
||||
for host in ${CONTROL_PLANE_IPS}; do
|
||||
scp /etc/kubernetes/pki/ca.crt "${USER}"@$host:
|
||||
scp /etc/kubernetes/pki/ca.key "${USER}"@$host:
|
||||
scp /etc/kubernetes/pki/sa.key "${USER}"@$host:
|
||||
scp /etc/kubernetes/pki/sa.pub "${USER}"@$host:
|
||||
scp /etc/kubernetes/pki/front-proxy-ca.crt "${USER}"@$host:
|
||||
scp /etc/kubernetes/pki/front-proxy-ca.key "${USER}"@$host:
|
||||
scp /etc/kubernetes/pki/etcd/ca.crt "${USER}"@$host:etcd-ca.crt
|
||||
# Quote this line if you are using external etcd
|
||||
scp /etc/kubernetes/pki/etcd/ca.key "${USER}"@$host:etcd-ca.key
|
||||
done
|
||||
```
|
||||
|
||||
{{< caution >}}
|
||||
Copy only the certificates in the above list. kubeadm will take care of generating the rest of the certificates
|
||||
with the required SANs for the joining control-plane instances. If you copy all the certificates by mistake,
|
||||
the creation of additional nodes could fail due to a lack of required SANs.
|
||||
{{< /caution >}}
|
||||
{{< caution >}}
|
||||
Copy only the certificates in the above list. kubeadm will take care of generating the rest of the certificates
|
||||
with the required SANs for the joining control-plane instances. If you copy all the certificates by mistake,
|
||||
the creation of additional nodes could fail due to a lack of required SANs.
|
||||
{{< /caution >}}
|
||||
|
||||
1. Then on each joining control plane node you have to run the following script before running `kubeadm join`.
|
||||
This script will move the previously copied certificates from the home directory to `/etc/kubernetes/pki`:
|
||||
|
||||
```sh
|
||||
USER=ubuntu # customizable
|
||||
mkdir -p /etc/kubernetes/pki/etcd
|
||||
mv /home/${USER}/ca.crt /etc/kubernetes/pki/
|
||||
mv /home/${USER}/ca.key /etc/kubernetes/pki/
|
||||
mv /home/${USER}/sa.pub /etc/kubernetes/pki/
|
||||
mv /home/${USER}/sa.key /etc/kubernetes/pki/
|
||||
mv /home/${USER}/front-proxy-ca.crt /etc/kubernetes/pki/
|
||||
mv /home/${USER}/front-proxy-ca.key /etc/kubernetes/pki/
|
||||
mv /home/${USER}/etcd-ca.crt /etc/kubernetes/pki/etcd/ca.crt
|
||||
# Quote this line if you are using external etcd
|
||||
mv /home/${USER}/etcd-ca.key /etc/kubernetes/pki/etcd/ca.key
|
||||
```
|
||||
```sh
|
||||
USER=ubuntu # customizable
|
||||
mkdir -p /etc/kubernetes/pki/etcd
|
||||
mv /home/${USER}/ca.crt /etc/kubernetes/pki/
|
||||
mv /home/${USER}/ca.key /etc/kubernetes/pki/
|
||||
mv /home/${USER}/sa.pub /etc/kubernetes/pki/
|
||||
mv /home/${USER}/sa.key /etc/kubernetes/pki/
|
||||
mv /home/${USER}/front-proxy-ca.crt /etc/kubernetes/pki/
|
||||
mv /home/${USER}/front-proxy-ca.key /etc/kubernetes/pki/
|
||||
mv /home/${USER}/etcd-ca.crt /etc/kubernetes/pki/etcd/ca.crt
|
||||
# Skip the next line if you are using external etcd
|
||||
mv /home/${USER}/etcd-ca.key /etc/kubernetes/pki/etcd/ca.key
|
||||
```
|
||||
|
||||
+3
-2
@@ -1,7 +1,7 @@
|
||||
---
|
||||
reviewers:
|
||||
- sig-cluster-lifecycle
|
||||
title: Set up a High Availability etcd cluster with kubeadm
|
||||
title: Set up a High Availability etcd Cluster with kubeadm
|
||||
content_type: task
|
||||
weight: 70
|
||||
---
|
||||
@@ -19,7 +19,8 @@ aspects.
|
||||
By default, kubeadm runs a local etcd instance on each control plane node.
|
||||
It is also possible to treat the etcd cluster as external and provision
|
||||
etcd instances on separate hosts. The differences between the two approaches are covered in the
|
||||
[Options for Highly Available topology][/docs/setup/production-environment/tools/kubeadm/ha-topology] page.
|
||||
[Options for Highly Available topology](/docs/setup/production-environment/tools/kubeadm/ha-topology) page.
|
||||
|
||||
This task walks through the process of creating a high availability external
|
||||
etcd cluster of three members that can be used by kubeadm during cluster creation.
|
||||
|
||||
|
||||
@@ -27,6 +27,11 @@ The `kube-apiserver` process accepts an argument `--encryption-provider-config`
|
||||
that controls how API data is encrypted in etcd. An example configuration
|
||||
is provided below.
|
||||
|
||||
{{< caution >}}
|
||||
**IMPORTANT:** For multi-master configurations (with two or more control plane nodes) the encryption configuration file must be the same!
|
||||
Otherwise, the kube-apiserver can't decrypt data stored inside the key-value store.
|
||||
{{< /caution >}}
|
||||
|
||||
## Understanding the encryption at rest configuration.
|
||||
|
||||
```yaml
|
||||
|
||||
@@ -100,4 +100,31 @@ shown in [the example](/docs/tasks/administer-cluster/dns-custom-nameservers/#ex
|
||||
The `node-local-dns` ConfigMap can also be modified directly with the stubDomain configuration
|
||||
in the Corefile format. Some cloud providers might not allow modifying `node-local-dns` ConfigMap directly.
|
||||
In those cases, the `kube-dns` ConfigMap can be updated.
|
||||
|
||||
|
||||
## Setting memory limits
|
||||
|
||||
node-local-dns pods use memory for storing cache entries and processing queries. Since they do not watch Kubernetes objects, the cluster size or the number of Services/Endpoints do not directly affect memory usage. Memory usage is influenced by the DNS query pattern.
|
||||
From [CoreDNS docs](https://github.com/coredns/deployment/blob/master/kubernetes/Scaling_CoreDNS.md),
|
||||
> The default cache size is 10000 entries, which uses about 30 MB when completely filled.
|
||||
|
||||
This would be the memory usage for each server block (if the cache gets completely filled).
|
||||
Memory usage can be reduced by specifying smaller cache sizes.
|
||||
|
||||
The number of concurrent queries is linked to the memory demand, because each extra
|
||||
goroutine used for handling a query requires an amount of memory. You can set an upper limit
|
||||
using the `max_concurrent` option in the forward plugin.
|
||||
|
||||
If a node-local-dns pod attempts to use more memory than is available (because of total system
|
||||
resources, or because of a configured
|
||||
[resource limit](/docs/concepts/configuration/manage-resources-containers/)), the operating system
|
||||
may shut down that pod's container.
|
||||
If this happens, the container that is terminated (“OOMKilled”) does not clean up the custom
|
||||
packet filtering rules that it previously added during startup.
|
||||
The node-local-dns container should get restarted (since managed as part of a DaemonSet), but this
|
||||
will lead to a brief DNS downtime each time that the container fails: the packet filtering rules direct
|
||||
DNS queries to a local Pod that is unhealthy.
|
||||
|
||||
You can determine a suitable memory limit by running node-local-dns pods without a limit and
|
||||
measuring the peak usage. You can also set up and use a
|
||||
[VerticalPodAutoscaler](https://github.com/kubernetes/autoscaler/tree/master/vertical-pod-autoscaler)
|
||||
in _recommender mode_, and then check its recommendations.
|
||||
|
||||
@@ -70,7 +70,7 @@ The following sysctls are supported in the _safe_ set:
|
||||
- `kernel.shm_rmid_forced`,
|
||||
- `net.ipv4.ip_local_port_range`,
|
||||
- `net.ipv4.tcp_syncookies`,
|
||||
- `net.ipv4.ping_group_range` (since Kubernetes 1.18).
|
||||
- `net.ipv4.ping_group_range` (since Kubernetes 1.18),
|
||||
- `net.ipv4.ip_unprivileged_port_start` (since Kubernetes 1.22).
|
||||
|
||||
{{< note >}}
|
||||
|
||||
@@ -8,10 +8,12 @@ card:
|
||||
---
|
||||
|
||||
<!-- overview -->
|
||||
Many applications rely on configuration which is used during either application initialization or runtime.
|
||||
Most of the times there is a requirement to adjust values assigned to configuration parameters.
|
||||
ConfigMaps is the kubernetes way to inject application pods with configuration data.
|
||||
ConfigMaps allow you to decouple configuration artifacts from image content to keep containerized applications portable. This page provides a series of usage examples demonstrating how to create ConfigMaps and configure Pods using data stored in ConfigMaps.
|
||||
|
||||
|
||||
|
||||
## {{% heading "prerequisites" %}}
|
||||
|
||||
|
||||
|
||||
@@ -290,10 +290,18 @@ To enable and use token request projection, you must specify each of the followi
|
||||
command line arguments to `kube-apiserver`:
|
||||
|
||||
* `--service-account-issuer`
|
||||
|
||||
It can be used as the Identifier of the service account token issuer. You can specify the `--service-account-issuer` argument multiple times, this can be useful to enable a non-disruptive change of the issuer. When this flag is specified multiple times, the first is used to generate tokens and all are used to determine which issuers are accepted. You must be running running Kubernetes v1.22 or later to be able to specify `--service-account-issuer` multiple times.
|
||||
* `--service-account-key-file`
|
||||
|
||||
File containing PEM-encoded x509 RSA or ECDSA private or public keys, used to verify ServiceAccount tokens. The specified file can contain multiple keys, and the flag can be specified multiple times with different files. If specified multiple times, tokens signed by any of the specified keys are considered valid by the Kubernetes API server.
|
||||
* `--service-account-signing-key-file`
|
||||
|
||||
Path to the file that contains the current private key of the service account token issuer. The issuer signs issued ID tokens with this private key.
|
||||
* `--api-audiences` (can be omitted)
|
||||
|
||||
The service account token authenticator validates that tokens used against the API are bound to at least one of these audiences. If `api-audiences` is specified multiple times, tokens for any of the specified audiences are considered valid by the Kubernetes API server. If the `--service-account-issuer` flag is configured and this flag is not, this field defaults to a single element list containing the issuer URL.
|
||||
|
||||
{{< /note >}}
|
||||
|
||||
The kubelet can also project a service account token into a Pod. You can
|
||||
|
||||
@@ -31,7 +31,7 @@ as Windows server containers, meaning that the version of the base images does n
|
||||
to match that of the host. It is, however, recommended that you use the same base image
|
||||
version as your Windows Server container workloads to ensure you do not have any unused
|
||||
images taking up space on the node. HostProcess containers also support
|
||||
[volume mounts](./create-hostprocess-pod#volume-mounts) within the container volume.
|
||||
[volume mounts](#volume-mounts) within the container volume.
|
||||
|
||||
### When should I use a Windows HostProcess container?
|
||||
|
||||
@@ -73,19 +73,20 @@ documentation for more details.
|
||||
These limitations are relevant for Kubernetes v{{< skew currentVersion >}}:
|
||||
|
||||
- HostProcess containers require containerd 1.6 or higher
|
||||
{{< glossary_tooltip text="container runtime" term_id="container-runtime" >}}.
|
||||
{{< glossary_tooltip text="container runtime" term_id="container-runtime" >}}.
|
||||
- HostProcess pods can only contain HostProcess containers. This is a current limitation
|
||||
of the Windows OS; non-privileged Windows containers cannot share a vNIC with the host IP namespace.
|
||||
of the Windows OS; non-privileged Windows containers cannot share a vNIC with the host IP namespace.
|
||||
- HostProcess containers run as a process on the host and do not have any degree of
|
||||
isolation other than resource constraints imposed on the HostProcess user account. Neither
|
||||
filesystem or Hyper-V isolation are supported for HostProcess containers.
|
||||
- Volume mounts are supported and are mounted under the container volume. See [Volume Mounts](#volume-mounts)
|
||||
isolation other than resource constraints imposed on the HostProcess user account. Neither
|
||||
filesystem or Hyper-V isolation are supported for HostProcess containers.
|
||||
- Volume mounts are supported and are mounted under the container volume. See
|
||||
[Volume Mounts](#volume-mounts)
|
||||
- A limited set of host user accounts are available for HostProcess containers by default.
|
||||
See [Choosing a User Account](#choosing-a-user-account).
|
||||
- Resource limits (disk, memory, cpu count) are supported in the same fashion as processes
|
||||
on the host.
|
||||
on the host.
|
||||
- Both Named pipe mounts and Unix domain sockets are **not** supported and should instead
|
||||
be accessed via their path on the host (e.g. \\\\.\\pipe\\\*)
|
||||
be accessed via their path on the host (e.g. \\\\.\\pipe\\\*)
|
||||
|
||||
## HostProcess Pod configuration requirements
|
||||
|
||||
|
||||
@@ -17,15 +17,11 @@ You can use it to inspect and debug container runtimes and applications on a
|
||||
Kubernetes node. `crictl` and its source are hosted in the
|
||||
[cri-tools](https://github.com/kubernetes-sigs/cri-tools) repository.
|
||||
|
||||
|
||||
|
||||
## {{% heading "prerequisites" %}}
|
||||
|
||||
|
||||
`crictl` requires a Linux operating system with a CRI runtime.
|
||||
|
||||
|
||||
|
||||
<!-- steps -->
|
||||
|
||||
## Installing crictl
|
||||
@@ -41,27 +37,37 @@ of Kubernetes. Extract it and move it to a location on your system path, such as
|
||||
The `crictl` command has several subcommands and runtime flags. Use
|
||||
`crictl help` or `crictl <subcommand> help` for more details.
|
||||
|
||||
`crictl` connects to `unix:///var/run/dockershim.sock` by default. For other
|
||||
runtimes, you can set the endpoint in multiple different ways:
|
||||
You can set the endpoint for `crictl` by doing one of the following:
|
||||
|
||||
- By setting flags `--runtime-endpoint` and `--image-endpoint`
|
||||
- By setting environment variables `CONTAINER_RUNTIME_ENDPOINT` and `IMAGE_SERVICE_ENDPOINT`
|
||||
- By setting the endpoint in the config file `--config=/etc/crictl.yaml`
|
||||
* Set the `--runtime-endpoint` and `--image-endpoint` flags.
|
||||
* Set the `CONTAINER_RUNTIME_ENDPOINT` and `IMAGE_SERVICE_ENDPOINT` environment
|
||||
variables.
|
||||
* Set the endpoint in the configuration file `/etc/crictl.yaml`. To specify a
|
||||
different file, use the `--config=PATH_TO_FILE` flag when you run `crictl`.
|
||||
|
||||
{{<note>}}
|
||||
If you don't set an endpoint, `crictl` attempts to connect to a list of known
|
||||
endpoints, which might result in an impact to performance.
|
||||
{{</note>}}
|
||||
|
||||
You can also specify timeout values when connecting to the server and enable or
|
||||
disable debugging, by specifying `timeout` or `debug` values in the configuration
|
||||
file or using the `--timeout` and `--debug` command-line flags.
|
||||
|
||||
To view or edit the current configuration, view or edit the contents of `/etc/crictl.yaml`.
|
||||
To view or edit the current configuration, view or edit the contents of
|
||||
`/etc/crictl.yaml`. For example, the configuration when using the `containerd`
|
||||
container runtime would be similar to this:
|
||||
|
||||
```shell
|
||||
cat /etc/crictl.yaml
|
||||
runtime-endpoint: unix:///var/run/dockershim.sock
|
||||
image-endpoint: unix:///var/run/dockershim.sock
|
||||
```
|
||||
runtime-endpoint: unix:///var/run/containerd/containerd.sock
|
||||
image-endpoint: unix:///var/run/containerd/containerd.sock
|
||||
timeout: 10
|
||||
debug: true
|
||||
```
|
||||
|
||||
To learn more about `crictl`, refer to the [`crictl`
|
||||
documentation](https://github.com/kubernetes-sigs/cri-tools/blob/master/docs/crictl.md).
|
||||
|
||||
## Example crictl commands
|
||||
|
||||
The following examples show some `crictl` commands and example output.
|
||||
@@ -348,64 +354,9 @@ CONTAINER ID IMAGE CREATED STATE
|
||||
3e025dd50a72d busybox About a minute ago Running busybox 0
|
||||
```
|
||||
|
||||
## {{% heading "whatsnext" %}}
|
||||
|
||||
* [Learn more about `crictl`](https://github.com/kubernetes-sigs/cri-tools).
|
||||
* [Map `docker` CLI commands to `crictl`](/reference/tools/map-crictl-dockercli/).
|
||||
|
||||
|
||||
<!-- discussion -->
|
||||
|
||||
See [kubernetes-sigs/cri-tools](https://github.com/kubernetes-sigs/cri-tools)
|
||||
for more information.
|
||||
|
||||
## Mapping from docker cli to crictl
|
||||
|
||||
The exact versions for below mapping table are for docker cli v1.40 and crictl v1.19.0. Please note that the list is not exhaustive. For example, it doesn't include experimental commands of docker cli.
|
||||
|
||||
{{< note >}}
|
||||
The output format of CRICTL is similar to Docker CLI, despite some missing columns for some CLI. Make sure to check output for the specific command if your script output parsing.
|
||||
{{< /note >}}
|
||||
|
||||
### Retrieve Debugging Information
|
||||
|
||||
{{< table caption="mapping from docker cli to crictl - retrieve debugging information" >}}
|
||||
docker cli | crictl | Description | Unsupported Features
|
||||
-- | -- | -- | --
|
||||
`attach` | `attach` | Attach to a running container | `--detach-keys`, `--sig-proxy`
|
||||
`exec` | `exec` | Run a command in a running container | `--privileged`, `--user`, `--detach-keys`
|
||||
`images` | `images` | List images |
|
||||
`info` | `info` | Display system-wide information |
|
||||
`inspect` | `inspect`, `inspecti` | Return low-level information on a container, image or task |
|
||||
`logs` | `logs` | Fetch the logs of a container | `--details`
|
||||
`ps` | `ps` | List containers |
|
||||
`stats` | `stats` | Display a live stream of container(s) resource usage statistics | Column: NET/BLOCK I/O, PIDs
|
||||
`version` | `version` | Show the runtime (Docker, ContainerD, or others) version information |
|
||||
{{< /table >}}
|
||||
|
||||
### Perform Changes
|
||||
|
||||
{{< table caption="mapping from docker cli to crictl - perform changes" >}}
|
||||
docker cli | crictl | Description | Unsupported Features
|
||||
-- | -- | -- | --
|
||||
`create` | `create` | Create a new container |
|
||||
`kill` | `stop` (timeout = 0) | Kill one or more running container | `--signal`
|
||||
`pull` | `pull` | Pull an image or a repository from a registry | `--all-tags`, `--disable-content-trust`
|
||||
`rm` | `rm` | Remove one or more containers |
|
||||
`rmi` | `rmi` | Remove one or more images |
|
||||
`run` | `run` | Run a command in a new container |
|
||||
`start` | `start` | Start one or more stopped containers | `--detach-keys`
|
||||
`stop` | `stop` | Stop one or more running containers |
|
||||
`update` | `update` | Update configuration of one or more containers | `--restart`, `--blkio-weight` and some other resource limit not supported by CRI.
|
||||
{{< /table >}}
|
||||
|
||||
### Supported only in crictl
|
||||
|
||||
{{< table caption="mapping from docker cli to crictl - supported only in crictl" >}}
|
||||
crictl | Description
|
||||
-- | --
|
||||
`imagefsinfo` | Return image filesystem info
|
||||
`inspectp` | Display the status of one or more pods
|
||||
`port-forward` | Forward local port to a pod
|
||||
`pods` | List pods
|
||||
`runp` | Run a new pod
|
||||
`rmp` | Remove one or more pods
|
||||
`stopp` | Stop one or more running pods
|
||||
{{< /table >}}
|
||||
<!-- discussion -->
|
||||
@@ -321,7 +321,7 @@ object:
|
||||
metric:
|
||||
name: requests-per-second
|
||||
describedObject:
|
||||
apiVersion: networking.k8s.io/v1beta1
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
name: main-route
|
||||
target:
|
||||
@@ -367,7 +367,7 @@ spec:
|
||||
metric:
|
||||
name: requests-per-second
|
||||
describedObject:
|
||||
apiVersion: networking.k8s.io/v1beta1
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
name: main-route
|
||||
target:
|
||||
@@ -390,7 +390,7 @@ status:
|
||||
metric:
|
||||
name: requests-per-second
|
||||
describedObject:
|
||||
apiVersion: networking.k8s.io/v1beta1
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
name: main-route
|
||||
current:
|
||||
|
||||
@@ -56,8 +56,9 @@ Kubernetes implements horizontal pod autoscaling as a control loop that runs int
|
||||
(and the default interval is 15 seconds).
|
||||
|
||||
Once during each period, the controller manager queries the resource utilization against the
|
||||
metrics specified in each HorizontalPodAutoscaler definition. The controller manager
|
||||
obtains the metrics from either the resource metrics API (for per-pod resource metrics),
|
||||
metrics specified in each HorizontalPodAutoscaler definition. The controller manager
|
||||
finds the target resource defined by the `scaleTargetRef`,
|
||||
then selects the pods based on the target resource's `.spec.selector` labels, and obtains the metrics from either the resource metrics API (for per-pod resource metrics),
|
||||
or the custom metrics API (for all other metrics).
|
||||
|
||||
* For per-pod resource metrics (like CPU), the controller fetches the metrics
|
||||
|
||||
@@ -76,16 +76,11 @@ cat <<EOF | cfssl genkey - | cfssljson -bare server
|
||||
"192.0.2.24",
|
||||
"10.0.34.2"
|
||||
],
|
||||
"CN": "system:node:my-pod.my-namespace.pod.cluster.local",
|
||||
"CN": "my-pod.my-namespace.pod.cluster.local",
|
||||
"key": {
|
||||
"algo": "ecdsa",
|
||||
"size": 256
|
||||
},
|
||||
"names": [
|
||||
{
|
||||
"O": "system:nodes"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
EOF
|
||||
```
|
||||
@@ -93,13 +88,13 @@ EOF
|
||||
Where `192.0.2.24` is the service's cluster IP,
|
||||
`my-svc.my-namespace.svc.cluster.local` is the service's DNS name,
|
||||
`10.0.34.2` is the pod's IP and `my-pod.my-namespace.pod.cluster.local`
|
||||
is the pod's DNS name. You should see the following output:
|
||||
is the pod's DNS name. You should see the output similar to:
|
||||
|
||||
```
|
||||
2017/03/21 06:48:17 [INFO] generate received request
|
||||
2017/03/21 06:48:17 [INFO] received CSR
|
||||
2017/03/21 06:48:17 [INFO] generating key: ecdsa-256
|
||||
2017/03/21 06:48:17 [INFO] encoded CSR
|
||||
2022/02/01 11:45:32 [INFO] generate received request
|
||||
2022/02/01 11:45:32 [INFO] received CSR
|
||||
2022/02/01 11:45:32 [INFO] generating key: ecdsa-256
|
||||
2022/02/01 11:45:32 [INFO] encoded CSR
|
||||
```
|
||||
|
||||
This command generates two files; it generates `server.csr` containing the PEM
|
||||
@@ -120,7 +115,7 @@ metadata:
|
||||
name: my-svc.my-namespace
|
||||
spec:
|
||||
request: $(cat server.csr | base64 | tr -d '\n')
|
||||
signerName: kubernetes.io/kubelet-serving
|
||||
signerName: example.com/serving
|
||||
usages:
|
||||
- digital signature
|
||||
- key encipherment
|
||||
@@ -131,7 +126,7 @@ EOF
|
||||
Notice that the `server.csr` file created in step 1 is base64 encoded
|
||||
and stashed in the `.spec.request` field. We are also requesting a
|
||||
certificate with the "digital signature", "key encipherment", and "server
|
||||
auth" key usages, signed by the `kubernetes.io/kubelet-serving` signer.
|
||||
auth" key usages, signed by an example `example.com/serving` signer.
|
||||
A specific `signerName` must be requested.
|
||||
View documentation for [supported signer names](/docs/reference/access-authn-authz/certificate-signing-requests/#signers)
|
||||
for more information.
|
||||
@@ -147,14 +142,16 @@ kubectl describe csr my-svc.my-namespace
|
||||
Name: my-svc.my-namespace
|
||||
Labels: <none>
|
||||
Annotations: <none>
|
||||
CreationTimestamp: Tue, 21 Mar 2017 07:03:51 -0700
|
||||
CreationTimestamp: Tue, 01 Feb 2022 11:49:15 -0500
|
||||
Requesting User: yourname@example.com
|
||||
Signer: example.com/serving
|
||||
Status: Pending
|
||||
Subject:
|
||||
Common Name: my-svc.my-namespace.svc.cluster.local
|
||||
Common Name: my-pod.my-namespace.pod.cluster.local
|
||||
Serial Number:
|
||||
Subject Alternative Names:
|
||||
DNS Names: my-svc.my-namespace.svc.cluster.local
|
||||
DNS Names: my-pod.my-namespace.pod.cluster.local
|
||||
my-svc.my-namespace.svc.cluster.local
|
||||
IP Addresses: 192.0.2.24
|
||||
10.0.34.2
|
||||
Events: <none>
|
||||
@@ -175,30 +172,136 @@ kubectl certificate approve my-svc.my-namespace
|
||||
certificatesigningrequest.certificates.k8s.io/my-svc.my-namespace approved
|
||||
```
|
||||
|
||||
|
||||
## Download the Certificate and Use It
|
||||
|
||||
Once the CSR is signed and approved you should see the following:
|
||||
You should now see the following:
|
||||
|
||||
```shell
|
||||
kubectl get csr
|
||||
```
|
||||
|
||||
```none
|
||||
NAME AGE REQUESTOR CONDITION
|
||||
my-svc.my-namespace 10m yourname@example.com Approved,Issued
|
||||
NAME AGE SIGNERNAME REQUESTOR REQUESTEDDURATION CONDITION
|
||||
my-svc.my-namespace 10m example.com/serving yourname@example.com <none> Approved
|
||||
```
|
||||
|
||||
You can download the issued certificate and save it to a `server.crt` file
|
||||
by running the following:
|
||||
This means the certificate request has been approved and is waiting for the
|
||||
requested signer to sign it.
|
||||
|
||||
## Sign the Certificate Signing Request
|
||||
|
||||
Next, you'll play the part of a certificate signer, issue the certificate, and upload it to the API.
|
||||
|
||||
A signer would typically watch the Certificate Signing Request API for objects with its `signerName`,
|
||||
check that they have been approved, sign certificates for those requests,
|
||||
and update the API object status with the issued certificate.
|
||||
|
||||
### Create a Certificate Authority
|
||||
|
||||
First, create a signing certificate by running the following:
|
||||
|
||||
```shell
|
||||
cat <<EOF | cfssl gencert -initca - | cfssljson -bare ca
|
||||
{
|
||||
"CN": "My Example Signer",
|
||||
"key": {
|
||||
"algo": "rsa",
|
||||
"size": 2048
|
||||
}
|
||||
}
|
||||
EOF
|
||||
```
|
||||
|
||||
You should see the output similar to:
|
||||
|
||||
```none
|
||||
2022/02/01 11:50:39 [INFO] generating a new CA key and certificate from CSR
|
||||
2022/02/01 11:50:39 [INFO] generate received request
|
||||
2022/02/01 11:50:39 [INFO] received CSR
|
||||
2022/02/01 11:50:39 [INFO] generating key: rsa-2048
|
||||
2022/02/01 11:50:39 [INFO] encoded CSR
|
||||
2022/02/01 11:50:39 [INFO] signed certificate with serial number 263983151013686720899716354349605500797834580472
|
||||
```
|
||||
|
||||
This produces a certificate authority key file (`ca-key.pem`) and certificate (`ca.pem`).
|
||||
|
||||
### Issue a Certificate
|
||||
|
||||
{{< codenew file="tls/server-signing-config.json" >}}
|
||||
|
||||
Use a `server-signing-config.json` signing configuration and the certificate authority key file
|
||||
and certificate to sign the certificate request:
|
||||
|
||||
```shell
|
||||
kubectl get csr my-svc.my-namespace -o jsonpath='{.spec.request}' | \
|
||||
base64 --decode | \
|
||||
cfssl sign -ca ca.pem -ca-key ca-key.pem -config server-signing-config.json - | \
|
||||
cfssljson -bare ca-signed-server
|
||||
```
|
||||
|
||||
You should see the output similar to:
|
||||
|
||||
```
|
||||
2022/02/01 11:52:26 [INFO] signed certificate with serial number 576048928624926584381415936700914530534472870337
|
||||
```
|
||||
|
||||
This produces a signed serving certificate file, `ca-signed-server.pem`.
|
||||
|
||||
### Upload the Signed Certificate
|
||||
|
||||
Finally, populate the signed certificate in the API object's status:
|
||||
|
||||
```shell
|
||||
kubectl get csr my-svc.my-namespace -o json | \
|
||||
jq '.status.certificate = "'$(base64 ca-signed-server.pem | tr -d '\n')'"' | \
|
||||
kubectl replace --raw /apis/certificates.k8s.io/v1/certificatesigningrequests/my-svc.my-namespace/status -f -
|
||||
```
|
||||
|
||||
{{< note >}}
|
||||
This uses the command line tool [jq](https://stedolan.github.io/jq/) to populate the base64-encoded content in the `.status.certificate` field.
|
||||
If you do not have `jq`, you can also save the JSON output to a file, populate this field manually, and upload the resulting file.
|
||||
{{< /note >}}
|
||||
|
||||
Once the CSR is approved and the signed certificate is uploaded you should see the following:
|
||||
|
||||
```shell
|
||||
kubectl get csr
|
||||
```
|
||||
|
||||
```none
|
||||
NAME AGE SIGNERNAME REQUESTOR REQUESTEDDURATION CONDITION
|
||||
my-svc.my-namespace 20m example.com/serving yourname@example.com <none> Approved,Issued
|
||||
```
|
||||
|
||||
## Download the Certificate and Use It
|
||||
|
||||
Now, as the requesting user, you can download the issued certificate
|
||||
and save it to a `server.crt` file by running the following:
|
||||
|
||||
```shell
|
||||
kubectl get csr my-svc.my-namespace -o jsonpath='{.status.certificate}' \
|
||||
| base64 --decode > server.crt
|
||||
```
|
||||
|
||||
Now you can use `server.crt` and `server-key.pem` as the keypair to start
|
||||
your HTTPS server.
|
||||
Now you can populate `server.crt` and `server-key.pem` in a secret and mount
|
||||
it into a pod to use as the keypair to start your HTTPS server:
|
||||
|
||||
```shell
|
||||
kubectl create secret tls server --cert server.crt --key server-key.pem
|
||||
```
|
||||
|
||||
```none
|
||||
secret/server created
|
||||
```
|
||||
|
||||
Finally, you can populate `ca.pem` in a configmap and use it as the trust root
|
||||
to verify the serving certificate:
|
||||
|
||||
```shell
|
||||
kubectl create configmap example-serving-ca --from-file ca.crt=ca.pem
|
||||
```
|
||||
|
||||
```none
|
||||
configmap/example-serving-ca created
|
||||
```
|
||||
|
||||
## Approving Certificate Signing Requests
|
||||
|
||||
|
||||
@@ -30,7 +30,6 @@ roleRef:
|
||||
name: system:volume-scheduler
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
---
|
||||
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
@@ -44,7 +43,6 @@ data:
|
||||
- schedulerName: my-scheduler
|
||||
leaderElection:
|
||||
leaderElect: false
|
||||
|
||||
---
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
@@ -76,13 +74,15 @@ spec:
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /healthz
|
||||
port: 10251
|
||||
port: 10259
|
||||
scheme: HTTPS
|
||||
initialDelaySeconds: 15
|
||||
name: kube-second-scheduler
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /healthz
|
||||
port: 10251
|
||||
port: 10259
|
||||
scheme: HTTPS
|
||||
resources:
|
||||
requests:
|
||||
cpu: '0.1'
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
apiVersion: flowcontrol.apiserver.k8s.io/v1beta1
|
||||
apiVersion: flowcontrol.apiserver.k8s.io/v1beta2
|
||||
kind: FlowSchema
|
||||
metadata:
|
||||
name: health-for-strangers
|
||||
|
||||
@@ -0,0 +1,15 @@
|
||||
{
|
||||
"signing": {
|
||||
"default": {
|
||||
"usages": [
|
||||
"digital signature",
|
||||
"key encipherment",
|
||||
"server auth"
|
||||
],
|
||||
"expiry": "876000h",
|
||||
"ca_constraint": {
|
||||
"is_ca": false
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -78,9 +78,10 @@ releases may also occur in between these.
|
||||
|
||||
| Monthly Patch Release | Cherry Pick Deadline | Target date |
|
||||
| --------------------- | -------------------- | ----------- |
|
||||
| January 2022 | 2022-01-14 | 2022-01-19 |
|
||||
| February 2022 | 2022-02-11 | 2022-02-16 |
|
||||
| March 2022 | 2022-03-11 | 2022-03-16 |
|
||||
| April 2022 | 2022-04-08 | 2022-04-13 |
|
||||
| May 2022 | 2022-05-13 | 2022-05-18 |
|
||||
|
||||
## Detailed Release History for Active Branches
|
||||
|
||||
@@ -92,6 +93,7 @@ End of Life for **1.23** is **2023-02-28**.
|
||||
|
||||
| Patch Release | Cherry Pick Deadline | Target Date | Note |
|
||||
|---------------|----------------------|-------------|------|
|
||||
| 1.23.4 | 2022-02-11 | 2022-02-16 | |
|
||||
| 1.23.3 | 2022-01-24 | 2022-01-25 | [Out-of-Band Release](https://groups.google.com/u/2/a/kubernetes.io/g/dev/c/Xl1sm-CItaY) |
|
||||
| 1.23.2 | 2022-01-14 | 2022-01-19 | |
|
||||
| 1.23.1 | 2021-12-14 | 2021-12-16 | |
|
||||
@@ -104,6 +106,7 @@ End of Life for **1.22** is **2022-10-28**
|
||||
|
||||
| Patch Release | Cherry Pick Deadline | Target Date | Note |
|
||||
|---------------|----------------------|-------------|------|
|
||||
| 1.22.7 | 2022-02-11 | 2022-02-16 | |
|
||||
| 1.22.6 | 2022-01-14 | 2022-01-19 | |
|
||||
| 1.22.5 | 2021-12-10 | 2021-12-15 | |
|
||||
| 1.22.4 | 2021-11-12 | 2021-11-17 | |
|
||||
@@ -119,6 +122,7 @@ End of Life for **1.21** is **2022-06-28**
|
||||
|
||||
| Patch Release | Cherry Pick Deadline | Target Date | Note |
|
||||
| ------------- | -------------------- | ----------- | ---------------------------------------------------------------------- |
|
||||
| 1.21.10 | 2022-02-11 | 2022-02-16 | |
|
||||
| 1.21.9 | 2022-01-14 | 2022-01-19 | |
|
||||
| 1.21.8 | 2021-12-10 | 2021-12-15 | |
|
||||
| 1.21.7 | 2021-11-12 | 2021-11-17 | |
|
||||
@@ -137,6 +141,7 @@ End of Life for **1.20** is **2022-02-28**
|
||||
|
||||
| Patch Release | Cherry Pick Deadline | Target Date | Note |
|
||||
| ------------- | -------------------- | ----------- | ----------------------------------------------------------------------------------- |
|
||||
| 1.20.16 | 2022-02-11 | 2022-02-16 | If there is critical/blocker patches to be released |
|
||||
| 1.20.15 | 2022-01-14 | 2022-01-19 | |
|
||||
| 1.20.14 | 2021-12-10 | 2021-12-15 | |
|
||||
| 1.20.13 | 2021-11-12 | 2021-11-17 | |
|
||||
|
||||
Reference in New Issue
Block a user