Merge branch 'master' into issue#28062
This commit is contained in:
hardikshah197
@@ -526,6 +526,6 @@ When you enable the API Priority and Fairness feature, the kube-apiserver serves
|
||||
|
||||
|
||||
For background information on design details for API priority and fairness, see
|
||||
the [enhancement proposal](https://github.com/kubernetes/enhancements/blob/master/keps/sig-api-machinery/20190228-priority-and-fairness.md).
|
||||
the [enhancement proposal](https://github.com/kubernetes/enhancements/tree/master/keps/sig-api-machinery/1040-priority-and-fairness).
|
||||
You can make suggestions and feature requests via [SIG API Machinery](https://github.com/kubernetes/community/tree/master/sig-api-machinery)
|
||||
or the feature's [slack channel](http://kubernetes.slack.com/messages/api-priority-and-fairness).
|
||||
|
||||
@@ -253,7 +253,7 @@ message AllocatableResourcesResponse {
|
||||
|
||||
`ContainerDevices` do expose the topology information declaring to which NUMA cells the device is affine.
|
||||
The NUMA cells are identified using a opaque integer ID, which value is consistent to what device
|
||||
plugins report [when they register themselves to the kubelet](https://kubernetes.io/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins/#device-plugin-integration-with-the-topology-manager).
|
||||
plugins report [when they register themselves to the kubelet](/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins/#device-plugin-integration-with-the-topology-manager).
|
||||
|
||||
|
||||
The gRPC service is served over a unix socket at `/var/lib/kubelet/pod-resources/kubelet.sock`.
|
||||
|
||||
@@ -16,7 +16,7 @@ During a node-pressure eviction, the kubelet sets the `PodPhase` for the
|
||||
selected pods to `Failed`. This terminates the pods.
|
||||
|
||||
Node-pressure eviction is not the same as
|
||||
[API-initiated eviction](/docs/concepts/scheduling-eviction/eviction/#api-eviction).
|
||||
[API-initiated eviction](/docs/concepts/scheduling-eviction/api-eviction/).
|
||||
|
||||
The kubelet does not respect your configured `PodDisruptionBudget` or the pod's
|
||||
`terminationGracePeriodSeconds`. If you use [soft eviction thresholds](#soft-eviction-thresholds),
|
||||
@@ -408,4 +408,4 @@ to estimate or measure an optimal memory limit value for that container.
|
||||
* Learn about [Pod Priority and Preemption](/docs/concepts/scheduling-eviction/pod-priority-preemption/)
|
||||
* Learn about [PodDisruptionBudgets](/docs/tasks/run-application/configure-pdb/)
|
||||
* Learn about [Quality of Service](/docs/tasks/configure-pod-container/quality-service-pod/) (QoS)
|
||||
* Check out the [Eviction API](/docs/reference/generated/kubernetes-api/{{<param "version">}}/#create-eviction-pod-v1-core)
|
||||
* Check out the [Eviction API](/docs/reference/generated/kubernetes-api/{{<param "version">}}/#create-eviction-pod-v1-core)
|
||||
|
||||
@@ -26,40 +26,41 @@ each resource to score nodes based on the request to capacity ratio. This
|
||||
allows users to bin pack extended resources by using appropriate parameters
|
||||
and improves the utilization of scarce resources in large clusters. The
|
||||
behavior of the `RequestedToCapacityRatioResourceAllocation` priority function
|
||||
can be controlled by a configuration option called
|
||||
`requestedToCapacityRatioArguments`. This argument consists of two parameters
|
||||
`shape` and `resources`. The `shape` parameter allows the user to tune the
|
||||
function as least requested or most requested based on `utilization` and
|
||||
`score` values. The `resources` parameter consists of `name` of the resource
|
||||
to be considered during scoring and `weight` specify the weight of each
|
||||
resource.
|
||||
can be controlled by a configuration option called `RequestedToCapacityRatioArgs`.
|
||||
This argument consists of two parameters `shape` and `resources`. The `shape`
|
||||
parameter allows the user to tune the function as least requested or most
|
||||
requested based on `utilization` and `score` values. The `resources` parameter
|
||||
consists of `name` of the resource to be considered during scoring and `weight`
|
||||
specify the weight of each resource.
|
||||
|
||||
Below is an example configuration that sets
|
||||
`requestedToCapacityRatioArguments` to bin packing behavior for extended
|
||||
resources `intel.com/foo` and `intel.com/bar`.
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Policy
|
||||
apiVersion: kubescheduler.config.k8s.io/v1beta1
|
||||
kind: KubeSchedulerConfiguration
|
||||
profiles:
|
||||
# ...
|
||||
priorities:
|
||||
# ...
|
||||
- name: RequestedToCapacityRatioPriority
|
||||
weight: 2
|
||||
argument:
|
||||
requestedToCapacityRatioArguments:
|
||||
shape:
|
||||
- utilization: 0
|
||||
score: 0
|
||||
- utilization: 100
|
||||
score: 10
|
||||
resources:
|
||||
- name: intel.com/foo
|
||||
weight: 3
|
||||
- name: intel.com/bar
|
||||
weight: 5
|
||||
pluginConfig:
|
||||
- name: RequestedToCapacityRatio
|
||||
args:
|
||||
shape:
|
||||
- utilization: 0
|
||||
score: 10
|
||||
- utilization: 100
|
||||
score: 0
|
||||
resources:
|
||||
- name: intel.com/foo
|
||||
weight: 3
|
||||
- name: intel.com/bar
|
||||
weight: 5
|
||||
```
|
||||
|
||||
Referencing the `KubeSchedulerConfiguration` file with the kube-scheduler
|
||||
flag `--config=/path/to/config/file` will pass the configuration to the
|
||||
scheduler.
|
||||
|
||||
**This feature is disabled by default**
|
||||
|
||||
### Tuning the Priority Function
|
||||
|
||||
@@ -212,9 +212,9 @@ This ensures that even pods that aren't selected by any other NetworkPolicy will
|
||||
|
||||
## SCTP support
|
||||
|
||||
{{< feature-state for_k8s_version="v1.19" state="beta" >}}
|
||||
{{< feature-state for_k8s_version="v1.20" state="stable" >}}
|
||||
|
||||
As a beta feature, this is enabled by default. To disable SCTP at a cluster level, you (or your cluster administrator) will need to disable the `SCTPSupport` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/) for the API server with `--feature-gates=SCTPSupport=false,…`.
|
||||
As a stable feature, this is enabled by default. To disable SCTP at a cluster level, you (or your cluster administrator) will need to disable the `SCTPSupport` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/) for the API server with `--feature-gates=SCTPSupport=false,…`.
|
||||
When the feature gate is enabled, you can set the `protocol` field of a NetworkPolicy to `SCTP`.
|
||||
|
||||
{{< note >}}
|
||||
|
||||
@@ -13,7 +13,7 @@ weight: 45
|
||||
|
||||
_Topology Aware Hints_ enable topology aware routing by including suggestions
|
||||
for how clients should consume endpoints. This approach adds metadata to enable
|
||||
consumers of EndpointSlice and / or and Endpoints objects, so that traffic to
|
||||
consumers of EndpointSlice and / or Endpoints objects, so that traffic to
|
||||
those network endpoints can be routed closer to where it originated.
|
||||
|
||||
For example, you can route traffic within a locality to reduce
|
||||
|
||||
@@ -86,7 +86,7 @@ rolling out node software updates can cause voluntary disruptions. Also, some im
|
||||
of cluster (node) autoscaling may cause voluntary disruptions to defragment and compact nodes.
|
||||
Your cluster administrator or hosting provider should have documented what level of voluntary
|
||||
disruptions, if any, to expect. Certain configuration options, such as
|
||||
[using PriorityClasses](https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/)
|
||||
[using PriorityClasses](/docs/concepts/configuration/pod-priority-preemption/)
|
||||
in your pod spec can also cause voluntary (and involuntary) disruptions.
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,25 @@
|
||||
---
|
||||
title: Viewing Site Analytics
|
||||
content_type: concept
|
||||
weight: 100
|
||||
card:
|
||||
name: contribute
|
||||
weight: 100
|
||||
---
|
||||
|
||||
<!-- overview -->
|
||||
|
||||
This page contains information about the kubernetes.io analytics dashboard.
|
||||
|
||||
|
||||
<!-- body -->
|
||||
|
||||
[View the dashboard](https://datastudio.google.com/reporting/fede2672-b2fd-402a-91d2-7473bdb10f04).
|
||||
|
||||
This dashboard is built using Google Data Studio and shows information collected on kubernetes.io using Google Analytics.
|
||||
|
||||
### Using the dashboard
|
||||
|
||||
By default, the dashboard shows all collected analytics for the past 30 days. Use the date selector to see data from a different date range. Other filtering options allow you to view data based on user location, the device used to access the site, the translation of the docs used, and more.
|
||||
|
||||
If you notice an issue with this dashboard, or would like to request any improvements, please [open an issue](https://github.com/kubernetes/website/issues/new/choose).
|
||||
@@ -134,7 +134,6 @@ Go to `<k8s-base>` and run these scripts:
|
||||
hack/update-generated-swagger-docs.sh
|
||||
hack/update-openapi-spec.sh
|
||||
hack/update-generated-protobuf.sh
|
||||
hack/update-api-reference-docs.sh
|
||||
```
|
||||
|
||||
Run `git status` to see what was generated.
|
||||
@@ -143,8 +142,6 @@ Run `git status` to see what was generated.
|
||||
On branch master
|
||||
...
|
||||
modified: api/openapi-spec/swagger.json
|
||||
modified: api/swagger-spec/apps_v1.json
|
||||
modified: docs/api-reference/apps/v1/definitions.html
|
||||
modified: staging/src/k8s.io/api/apps/v1/generated.proto
|
||||
modified: staging/src/k8s.io/api/apps/v1/types.go
|
||||
modified: staging/src/k8s.io/api/apps/v1/types_swagger_doc_generated.go
|
||||
|
||||
@@ -38,7 +38,7 @@ client libraries:
|
||||
- [Kubernetes Python client library](https://github.com/kubernetes-client/python)
|
||||
- [Kubernetes Java client library](https://github.com/kubernetes-client/java)
|
||||
- [Kubernetes JavaScript client library](https://github.com/kubernetes-client/javascript)
|
||||
- [Kubernetes Dotnet client library](https://github.com/kubernetes-client/csharp)
|
||||
- [Kubernetes C# client library](https://github.com/kubernetes-client/csharp)
|
||||
- [Kubernetes Haskell Client library](https://github.com/kubernetes-client/haskell)
|
||||
|
||||
## CLI
|
||||
|
||||
@@ -58,7 +58,7 @@ It acts synchronously to modify pods as they are created or updated. When this p
|
||||
1. It ensures that the `ServiceAccount` referenced by the pod exists, and otherwise rejects it.
|
||||
1. It adds a `volume` to the pod which contains a token for API access if neither the ServiceAccount `automountServiceAccountToken` nor the Pod's `automountServiceAccountToken` is set to `false`.
|
||||
1. It adds a `volumeSource` to each container of the pod mounted at `/var/run/secrets/kubernetes.io/serviceaccount`, if the previous step has created a volume for ServiceAccount token.
|
||||
1. If the pod does not contain any `ImagePullSecrets`, then `ImagePullSecrets` of the `ServiceAccount` are added to the pod.
|
||||
1. If the pod does not contain any `imagePullSecrets`, then `imagePullSecrets` of the `ServiceAccount` are added to the pod.
|
||||
|
||||
#### Bound Service Account Token Volume
|
||||
|
||||
@@ -91,14 +91,14 @@ add the following projected volume instead of a Secret-based volume for the non-
|
||||
This projected volume consists of three sources:
|
||||
|
||||
1. A ServiceAccountToken acquired from kube-apiserver via TokenRequest API. It will expire after 1 hour by default or when the pod is deleted. It is bound to the pod and has kube-apiserver as the audience.
|
||||
1. A ConfigMap containing a CA bundle used for verifying connections to the kube-apiserver. This feature depends on the `RootCAConfigMap` feature gate being enabled, which publishes a "kube-root-ca.crt" ConfigMap to every namespace. `RootCAConfigMap` is enabled by default in 1.20, and always enabled in 1.21+.
|
||||
1. A ConfigMap containing a CA bundle used for verifying connections to the kube-apiserver. This feature depends on the `RootCAConfigMap` feature gate, which publishes a "kube-root-ca.crt" ConfigMap to every namespace. `RootCAConfigMap` feature gate is graduated to GA in 1.21 and default to true. (This flag will be removed from --feature-gate arg in 1.22)
|
||||
1. A DownwardAPI that references the namespace of the pod.
|
||||
|
||||
See more details about [projected volumes](/docs/tasks/configure-pod-container/configure-projected-volume-storage/).
|
||||
|
||||
You can manually migrate a secret-based service account volume to a projected volume when
|
||||
You can manually migrate a Secret-based service account volume to a projected volume when
|
||||
the `BoundServiceAccountTokenVolume` feature gate is not enabled by adding the above
|
||||
projected volume to the pod spec. However, `RootCAConfigMap` needs to be enabled.
|
||||
projected volume to the pod spec.
|
||||
|
||||
### Token Controller
|
||||
|
||||
|
||||
@@ -844,7 +844,7 @@ Each feature gate is designed for enabling/disabling a specific feature:
|
||||
- `ValidateProxyRedirects`: This flag controls whether the API server should
|
||||
validate that redirects are only followed to the same host. Only used if the
|
||||
`StreamingProxyRedirects` flag is enabled.
|
||||
- 'VolumeCapacityPriority`: Enable support for prioritizing nodes in different
|
||||
- `VolumeCapacityPriority`: Enable support for prioritizing nodes in different
|
||||
topologies based on available PV capacity.
|
||||
- `VolumePVCDataSource`: Enable support for specifying an existing PVC as a DataSource.
|
||||
- `VolumeScheduling`: Enable volume topology aware scheduling and make the
|
||||
|
||||
@@ -64,7 +64,7 @@ kube-apiserver [flags]
|
||||
<td colspan="2">--allow-metric-labels stringToString Default: []</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>The map from metric-label to value allow-list of this label. The key's format is <!-- raw HTML omitted -->,<!-- raw HTML omitted -->. The value's format is <allowed_value>,<allowed_value>...e.g. metric1,label1='v1,v2,v3', metric1,label2='v1,v2,v3' metric2,label1='v1,v2,v3'.</p></td>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>The map from metric-label to value allow-list of this label. The key's format is <MetricName>,<LabelName>. The value's format is <allowed_value>,<allowed_value>...e.g. metric1,label1='v1,v2,v3', metric1,label2='v1,v2,v3' metric2,label1='v1,v2,v3'.</p></td>
|
||||
</tr>
|
||||
|
||||
<tr>
|
||||
@@ -967,7 +967,7 @@ kube-apiserver [flags]
|
||||
<td colspan="2">--runtime-config <comma-separated 'key=value' pairs></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>A set of key=value pairs that enable or disable built-in APIs. Supported options are:<br/>v1=true|false for the core API group<br/><!-- raw HTML omitted -->/<!-- raw HTML omitted -->=true|false for a specific API group and version (e.g. apps/v1=true)<br/>api/all=true|false controls all API versions<br/>api/ga=true|false controls all API versions of the form v[0-9]+<br/>api/beta=true|false controls all API versions of the form v[0-9]+beta[0-9]+<br/>api/alpha=true|false controls all API versions of the form v[0-9]+alpha[0-9]+<br/>api/legacy is deprecated, and will be removed in a future version</p></td>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>A set of key=value pairs that enable or disable built-in APIs. Supported options are:<br/>v1=true|false for the core API group<br/><group>/<version>=true|false for a specific API group and version (e.g. apps/v1=true)<br/>api/all=true|false controls all API versions<br/>api/ga=true|false controls all API versions of the form v[0-9]+<br/>api/beta=true|false controls all API versions of the form v[0-9]+beta[0-9]+<br/>api/alpha=true|false controls all API versions of the form v[0-9]+alpha[0-9]+<br/>api/legacy is deprecated, and will be removed in a future version</p></td>
|
||||
</tr>
|
||||
|
||||
<tr>
|
||||
@@ -1044,7 +1044,7 @@ kube-apiserver [flags]
|
||||
<td colspan="2">--show-hidden-metrics-for-version string</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>The previous version for which you want to show hidden metrics. Only the previous minor version is meaningful, other values will not be allowed. The format is <!-- raw HTML omitted -->.<!-- raw HTML omitted -->, e.g.: '1.16'. The purpose of this format is make sure you have the opportunity to notice if the next release hides additional metrics, rather than being surprised when they are permanently removed in the release after that.</p></td>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>The previous version for which you want to show hidden metrics. Only the previous minor version is meaningful, other values will not be allowed. The format is <major>.<minor>, e.g.: '1.16'. The purpose of this format is make sure you have the opportunity to notice if the next release hides additional metrics, rather than being surprised when they are permanently removed in the release after that.</p></td>
|
||||
</tr>
|
||||
|
||||
<tr>
|
||||
|
||||
@@ -61,7 +61,7 @@ kube-controller-manager [flags]
|
||||
<td colspan="2">--allow-metric-labels stringToString Default: []</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>The map from metric-label to value allow-list of this label. The key's format is <!-- raw HTML omitted -->,<!-- raw HTML omitted -->. The value's format is <allowed_value>,<allowed_value>...e.g. metric1,label1='v1,v2,v3', metric1,label2='v1,v2,v3' metric2,label1='v1,v2,v3'.</p></td>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>The map from metric-label to value allow-list of this label. The key's format is <MetricName>,<LabelName>. The value's format is <allowed_value>,<allowed_value>...e.g. metric1,label1='v1,v2,v3', metric1,label2='v1,v2,v3' metric2,label1='v1,v2,v3'.</p></td>
|
||||
</tr>
|
||||
|
||||
<tr>
|
||||
@@ -943,7 +943,7 @@ kube-controller-manager [flags]
|
||||
<td colspan="2">--show-hidden-metrics-for-version string</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>The previous version for which you want to show hidden metrics. Only the previous minor version is meaningful, other values will not be allowed. The format is <!-- raw HTML omitted -->.<!-- raw HTML omitted -->, e.g.: '1.16'. The purpose of this format is make sure you have the opportunity to notice if the next release hides additional metrics, rather than being surprised when they are permanently removed in the release after that.</p></td>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>The previous version for which you want to show hidden metrics. Only the previous minor version is meaningful, other values will not be allowed. The format is <major>.<minor>, e.g.: '1.16'. The purpose of this format is make sure you have the opportunity to notice if the next release hides additional metrics, rather than being surprised when they are permanently removed in the release after that.</p></td>
|
||||
</tr>
|
||||
|
||||
<tr>
|
||||
|
||||
@@ -424,7 +424,7 @@ kube-proxy [flags]
|
||||
<td colspan="2">--show-hidden-metrics-for-version string</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>The previous version for which you want to show hidden metrics. Only the previous minor version is meaningful, other values will not be allowed. The format is <!-- raw HTML omitted -->.<!-- raw HTML omitted -->, e.g.: '1.16'. The purpose of this format is make sure you have the opportunity to notice if the next release hides additional metrics, rather than being surprised when they are permanently removed in the release after that.</p></td>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>The previous version for which you want to show hidden metrics. Only the previous minor version is meaningful, other values will not be allowed. The format is <major>.<minor>, e.g.: '1.16'. The purpose of this format is make sure you have the opportunity to notice if the next release hides additional metrics, rather than being surprised when they are permanently removed in the release after that.</p></td>
|
||||
</tr>
|
||||
|
||||
<tr>
|
||||
|
||||
@@ -27,7 +27,7 @@ each Pod in the scheduling queue according to constraints and available
|
||||
resources. The scheduler then ranks each valid Node and binds the Pod to a
|
||||
suitable Node. Multiple different schedulers may be used within a cluster;
|
||||
kube-scheduler is the reference implementation.
|
||||
See [scheduling](https://kubernetes.io/docs/concepts/scheduling-eviction/)
|
||||
See [scheduling](/docs/concepts/scheduling-eviction/)
|
||||
for more information about scheduling and the kube-scheduler component.
|
||||
|
||||
```
|
||||
@@ -68,7 +68,7 @@ kube-scheduler [flags]
|
||||
<td colspan="2">--allow-metric-labels stringToString Default: []</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>The map from metric-label to value allow-list of this label. The key's format is <!-- raw HTML omitted -->,<!-- raw HTML omitted -->. The value's format is <allowed_value>,<allowed_value>...e.g. metric1,label1='v1,v2,v3', metric1,label2='v1,v2,v3' metric2,label1='v1,v2,v3'.</p></td>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>The map from metric-label to value allow-list of this label. The key's format is <MetricName>,<LabelName>. The value's format is <allowed_value>,<allowed_value>...e.g. metric1,label1='v1,v2,v3', metric1,label2='v1,v2,v3' metric2,label1='v1,v2,v3'.</p></td>
|
||||
</tr>
|
||||
|
||||
<tr>
|
||||
@@ -474,7 +474,7 @@ kube-scheduler [flags]
|
||||
<td colspan="2">--show-hidden-metrics-for-version string</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>The previous version for which you want to show hidden metrics. Only the previous minor version is meaningful, other values will not be allowed. The format is <!-- raw HTML omitted -->.<!-- raw HTML omitted -->, e.g.: '1.16'. The purpose of this format is make sure you have the opportunity to notice if the next release hides additional metrics, rather than being surprised when they are permanently removed in the release after that.</p></td>
|
||||
<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>The previous version for which you want to show hidden metrics. Only the previous minor version is meaningful, other values will not be allowed. The format is <major>.<minor>, e.g.: '1.16'. The purpose of this format is make sure you have the opportunity to notice if the next release hides additional metrics, rather than being surprised when they are permanently removed in the release after that.</p></td>
|
||||
</tr>
|
||||
|
||||
<tr>
|
||||
|
||||
@@ -222,7 +222,9 @@ When a single IngressClass resource has this annotation set to `"true"`, new Ing
|
||||
|
||||
## kubernetes.io/ingress.class (deprecated)
|
||||
|
||||
{{< note >}} Starting in v1.18, this annotation is deprecated in favor of `spec.ingressClassName`. {{< /note >}}
|
||||
{{< note >}}
|
||||
Starting in v1.18, this annotation is deprecated in favor of `spec.ingressClassName`.
|
||||
{{< /note >}}
|
||||
|
||||
## storageclass.kubernetes.io/is-default-class
|
||||
|
||||
@@ -230,7 +232,8 @@ Example: `storageclass.kubernetes.io/is-default-class=true`
|
||||
|
||||
Used on: StorageClass
|
||||
|
||||
When a single StorageClass resource has this annotation set to `"true"`, new Physical Volume Claim resource without a class specified will be assigned this default class.
|
||||
When a single StorageClass resource has this annotation set to `"true"`, new PersistentVolumeClaim
|
||||
resource without a class specified will be assigned this default class.
|
||||
|
||||
## alpha.kubernetes.io/provided-node-ip
|
||||
|
||||
|
||||
@@ -250,7 +250,7 @@ only has one pending pods queue.
|
||||
|
||||
## {{% heading "whatsnext" %}}
|
||||
|
||||
* Read the [kube-scheduler reference](https://kubernetes.io/docs/reference/command-line-tools-reference/kube-scheduler/)
|
||||
* Read the [kube-scheduler reference](/docs/reference/command-line-tools-reference/kube-scheduler/)
|
||||
* Learn about [scheduling](/docs/concepts/scheduling-eviction/kube-scheduler/)
|
||||
* Read the [kube-scheduler configuration (v1beta1)](/docs/reference/config-api/kube-scheduler-config.v1beta1/) reference
|
||||
|
||||
|
||||
+1
-1
@@ -17,7 +17,7 @@ Generate keys and certificate signing requests
|
||||
|
||||
Generates keys and certificate signing requests (CSRs) for all the certificates required to run the control plane. This command also generates partial kubeconfig files with private key data in the "users > user > client-key-data" field, and for each kubeconfig file an accompanying ".csr" file is created.
|
||||
|
||||
This command is designed for use in [Kubeadm External CA Mode](https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-certs/#external-ca-mode). It generates CSRs which you can then submit to your external certificate authority for signing.
|
||||
This command is designed for use in [Kubeadm External CA Mode](/docs/tasks/administer-cluster/kubeadm/kubeadm-certs/#external-ca-mode). It generates CSRs which you can then submit to your external certificate authority for signing.
|
||||
|
||||
The PEM encoded signed certificates should then be saved alongside the key files, using ".crt" as the file extension, or in the case of kubeconfig files, the PEM encoded signed certificate should be base64 encoded and added to the kubeconfig file in the "users > user > client-certificate-data" field.
|
||||
|
||||
|
||||
@@ -49,7 +49,7 @@ Some resource types will have one or more sub-resources, represented as sub path
|
||||
* Cluster-scoped subresource: `GET /apis/GROUP/VERSION/RESOURCETYPE/NAME/SUBRESOURCE`
|
||||
* Namespace-scoped subresource: `GET /apis/GROUP/VERSION/namespaces/NAMESPACE/RESOURCETYPE/NAME/SUBRESOURCE`
|
||||
|
||||
The verbs supported for each subresource will differ depending on the object - see the API documentation more information. It is not possible to access sub-resources across multiple resources - generally a new virtual resource type would be used if that becomes necessary.
|
||||
The verbs supported for each subresource will differ depending on the object - see the API documentation for more information. It is not possible to access sub-resources across multiple resources - generally a new virtual resource type would be used if that becomes necessary.
|
||||
|
||||
|
||||
## Efficient detection of changes
|
||||
@@ -442,7 +442,7 @@ feature, see the section on
|
||||
|
||||
## Resource Versions
|
||||
|
||||
Resource versions are strings that identify the server's internal version of an object. Resource versions can be used by clients to determine when objects have changed, or to express data consistency requirements when getting, listing and watching resources. Resource versions must be treated as opaque by clients and passed unmodified back to the server. For example, clients must not assume resource versions are numeric, and may only compare two resource version for equality (i.e. must not compare resource versions for greater-than or less-than relationships).
|
||||
Resource versions are strings that identify the server's internal version of an object. Resource versions can be used by clients to determine when objects have changed, or to express data consistency requirements when getting, listing and watching resources. Resource versions must be treated as opaque by clients and passed unmodified back to the server. For example, clients must not assume resource versions are numeric, and may only compare two resource versions for equality (i.e. must not compare resource versions for greater-than or less-than relationships).
|
||||
|
||||
### ResourceVersion in metadata
|
||||
|
||||
@@ -454,7 +454,7 @@ Clients find resource versions in resources, including the resources in watch ev
|
||||
|
||||
### The ResourceVersion Parameter
|
||||
|
||||
The get, list and watch operations support the `resourceVersion` parameter.
|
||||
The get, list, and watch operations support the `resourceVersion` parameter.
|
||||
|
||||
The exact meaning of this parameter differs depending on the operation and the value of `resourceVersion`.
|
||||
|
||||
|
||||
@@ -74,7 +74,7 @@ The **policy/v1beta1** API version of PodDisruptionBudget will no longer be serv
|
||||
PodSecurityPolicy in the **policy/v1beta1** API version will no longer be served in v1.25, and the PodSecurityPolicy admission controller will be removed.
|
||||
|
||||
PodSecurityPolicy replacements are still under discussion, but current use can be migrated to
|
||||
[3rd-party admission webhooks](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/) now.
|
||||
[3rd-party admission webhooks](/docs/reference/access-authn-authz/extensible-admission-controllers/) now.
|
||||
|
||||
#### RuntimeClass {#runtimeclass-v125}
|
||||
|
||||
|
||||
@@ -245,7 +245,7 @@ field tags.
|
||||
|
||||
### Compatibility across topology changes
|
||||
|
||||
On rare occurences, a CRD or built-in type author may want to change the
|
||||
On rare occurrences, a CRD or built-in type author may want to change the
|
||||
specific topology of a field in their resource without incrementing its
|
||||
version. Changing the topology of types, by upgrading the cluster or
|
||||
updating the CRD, has different consequences when updating existing
|
||||
@@ -253,7 +253,7 @@ objects. There are two categories of changes: when a field goes from
|
||||
`map`/`set`/`granular` to `atomic` and the other way around.
|
||||
|
||||
When the `listType`, `mapType`, or `structType` changes from
|
||||
`map`/`set`/`granular` to `atomic`, the whole list, map or struct of
|
||||
`map`/`set`/`granular` to `atomic`, the whole list, map, or struct of
|
||||
existing objects will end-up being owned by actors who owned an element
|
||||
of these types. This means that any further change to these objects
|
||||
would cause a conflict.
|
||||
@@ -310,7 +310,7 @@ simplify the update logic of your controller. The main differences with a
|
||||
read-modify-write and/or patch are the following:
|
||||
|
||||
* the applied object must contain all the fields that the controller cares about.
|
||||
* there are no way to remove fields that haven't been applied by the controller
|
||||
* there is no way to remove fields that haven't been applied by the controller
|
||||
before (controller can still send a PATCH/UPDATE for these use-cases).
|
||||
* the object doesn't have to be read beforehand, `resourceVersion` doesn't have
|
||||
to be specified.
|
||||
@@ -473,7 +473,7 @@ have an opinion about.
|
||||
## Clearing ManagedFields
|
||||
|
||||
It is possible to strip all managedFields from an object by overwriting them
|
||||
using `MergePatch`, `StrategicMergePatch`, `JSONPatch` or `Update`, so every
|
||||
using `MergePatch`, `StrategicMergePatch`, `JSONPatch`, or `Update`, so every
|
||||
non-apply operation. This can be done by overwriting the managedFields field
|
||||
with an empty entry. Two examples are:
|
||||
|
||||
|
||||
@@ -66,8 +66,8 @@ When creating a cluster, you can (using custom tooling):
|
||||
* start and configure additional etcd instance
|
||||
* configure the {{< glossary_tooltip term_id="kube-apiserver" text="API server" >}} to use it for storing events
|
||||
|
||||
See [Operating etcd clusters for Kubernetes](https://kubernetes.io/docs/tasks/administer-cluster/configure-upgrade-etcd/) and
|
||||
[Set up a High Availability etcd cluster with kubeadm](docs/setup/production-environment/tools/kubeadm/setup-ha-etcd-with-kubeadm/)
|
||||
See [Operating etcd clusters for Kubernetes](/docs/tasks/administer-cluster/configure-upgrade-etcd/) and
|
||||
[Set up a High Availability etcd cluster with kubeadm](/docs/setup/production-environment/tools/kubeadm/setup-ha-etcd-with-kubeadm/)
|
||||
for details on configuring and managing etcd for a large cluster.
|
||||
|
||||
## Addon resources
|
||||
|
||||
@@ -49,7 +49,7 @@ access cluster resources. You can use role-based access control
|
||||
security mechanisms to make sure that users and workloads can get access to the
|
||||
resources they need, while keeping workloads, and the cluster itself, secure.
|
||||
You can set limits on the resources that users and workloads can access
|
||||
by managing [policies](https://kubernetes.io/docs/concepts/policy/) and
|
||||
by managing [policies](/docs/concepts/policy/) and
|
||||
[container resources](/docs/concepts/configuration/manage-resources-containers/).
|
||||
|
||||
Before building a Kubernetes production environment on your own, consider
|
||||
@@ -286,8 +286,8 @@ and the
|
||||
deployment methods.
|
||||
- Configure user management by determining your
|
||||
[Authentication](/docs/reference/access-authn-authz/authentication/) and
|
||||
[Authorization](docs/reference/access-authn-authz/authorization/) methods.
|
||||
[Authorization](/docs/reference/access-authn-authz/authorization/) methods.
|
||||
- Prepare for application workloads by setting up
|
||||
[resource limits](docs/tasks/administer-cluster/manage-resources/),
|
||||
[resource limits](/docs/tasks/administer-cluster/manage-resources/),
|
||||
[DNS autoscaling](/docs/tasks/administer-cluster/dns-horizontal-autoscaling/)
|
||||
and [service accounts](/docs/reference/access-authn-authz/service-accounts-admin/).
|
||||
|
||||
+4
-4
@@ -23,7 +23,7 @@ of Containers for each.
|
||||
|
||||
- Fetch all Pods in all namespaces using `kubectl get pods --all-namespaces`
|
||||
- Format the output to include only the list of Container image names
|
||||
using `-o jsonpath={..image}`. This will recursively parse out the
|
||||
using `-o jsonpath={.items[*].spec.containers[*].image}`. This will recursively parse out the
|
||||
`image` field from the returned json.
|
||||
- See the [jsonpath reference](/docs/reference/kubectl/jsonpath/)
|
||||
for further information on how to use jsonpath.
|
||||
@@ -33,7 +33,7 @@ of Containers for each.
|
||||
- Use `uniq` to aggregate image counts
|
||||
|
||||
```shell
|
||||
kubectl get pods --all-namespaces -o jsonpath="{..image}" |\
|
||||
kubectl get pods --all-namespaces -o jsonpath="{.items[*].spec.containers[*].image}" |\
|
||||
tr -s '[[:space:]]' '\n' |\
|
||||
sort |\
|
||||
uniq -c
|
||||
@@ -80,7 +80,7 @@ To target only Pods matching a specific label, use the -l flag. The
|
||||
following matches only Pods with labels matching `app=nginx`.
|
||||
|
||||
```shell
|
||||
kubectl get pods --all-namespaces -o=jsonpath="{..image}" -l app=nginx
|
||||
kubectl get pods --all-namespaces -o=jsonpath="{.items[*].spec.containers[*].image}" -l app=nginx
|
||||
```
|
||||
|
||||
## List Container images filtering by Pod namespace
|
||||
@@ -89,7 +89,7 @@ To target only pods in a specific namespace, use the namespace flag. The
|
||||
following matches only Pods in the `kube-system` namespace.
|
||||
|
||||
```shell
|
||||
kubectl get pods --namespace kube-system -o jsonpath="{..image}"
|
||||
kubectl get pods --namespace kube-system -o jsonpath="{.items[*].spec.containers[*].image}"
|
||||
```
|
||||
|
||||
## List Container images using a go-template instead of jsonpath
|
||||
|
||||
+54
-46
@@ -1,16 +1,17 @@
|
||||
---
|
||||
reviewers:
|
||||
- jszczepkowski
|
||||
title: Set up High-Availability Kubernetes Masters
|
||||
title: Set up a High-Availability Control Plane
|
||||
content_type: task
|
||||
aliases: [ '/docs/tasks/administer-cluster/highly-available-master/' ]
|
||||
---
|
||||
|
||||
<!-- overview -->
|
||||
|
||||
{{< feature-state for_k8s_version="v1.5" state="alpha" >}}
|
||||
|
||||
You can replicate Kubernetes masters in `kube-up` or `kube-down` scripts for Google Compute Engine.
|
||||
This document describes how to use kube-up/down scripts to manage highly available (HA) masters and how HA masters are implemented for use with GCE.
|
||||
You can replicate Kubernetes control plane nodes in `kube-up` or `kube-down` scripts for Google Compute Engine.
|
||||
This document describes how to use kube-up/down scripts to manage a highly available (HA) control plane and how HA control planes are implemented for use with GCE.
|
||||
|
||||
|
||||
|
||||
@@ -28,17 +29,17 @@ This document describes how to use kube-up/down scripts to manage highly availab
|
||||
|
||||
To create a new HA-compatible cluster, you must set the following flags in your `kube-up` script:
|
||||
|
||||
* `MULTIZONE=true` - to prevent removal of master replicas kubelets from zones different than server's default zone.
|
||||
Required if you want to run master replicas in different zones, which is recommended.
|
||||
* `MULTIZONE=true` - to prevent removal of control plane kubelets from zones different than server's default zone.
|
||||
Required if you want to run control plane nodes in different zones, which is recommended.
|
||||
|
||||
* `ENABLE_ETCD_QUORUM_READ=true` - to ensure that reads from all API servers will return most up-to-date data.
|
||||
If true, reads will be directed to leader etcd replica.
|
||||
Setting this value to true is optional: reads will be more reliable but will also be slower.
|
||||
|
||||
Optionally, you can specify a GCE zone where the first master replica is to be created.
|
||||
Optionally, you can specify a GCE zone where the first control plane node is to be created.
|
||||
Set the following flag:
|
||||
|
||||
* `KUBE_GCE_ZONE=zone` - zone where the first master replica will run.
|
||||
* `KUBE_GCE_ZONE=zone` - zone where the first control plane node will run.
|
||||
|
||||
The following sample command sets up a HA-compatible cluster in the GCE zone europe-west1-b:
|
||||
|
||||
@@ -46,50 +47,52 @@ The following sample command sets up a HA-compatible cluster in the GCE zone eur
|
||||
MULTIZONE=true KUBE_GCE_ZONE=europe-west1-b ENABLE_ETCD_QUORUM_READS=true ./cluster/kube-up.sh
|
||||
```
|
||||
|
||||
Note that the commands above create a cluster with one master;
|
||||
however, you can add new master replicas to the cluster with subsequent commands.
|
||||
Note that the commands above create a cluster with one control plane node;
|
||||
however, you can add new control plane nodes to the cluster with subsequent commands.
|
||||
|
||||
## Adding a new master replica
|
||||
## Adding a new control plane node
|
||||
|
||||
After you have created an HA-compatible cluster, you can add master replicas to it.
|
||||
You add master replicas by using a `kube-up` script with the following flags:
|
||||
After you have created an HA-compatible cluster, you can add control plane nodes to it.
|
||||
You add control plane nodes by using a `kube-up` script with the following flags:
|
||||
|
||||
* `KUBE_REPLICATE_EXISTING_MASTER=true` - to create a replica of an existing
|
||||
master.
|
||||
* `KUBE_REPLICATE_EXISTING_MASTER=true` - to create a replica of an existing control plane
|
||||
node.
|
||||
|
||||
* `KUBE_GCE_ZONE=zone` - zone where the master replica will run.
|
||||
Must be in the same region as other replicas' zones.
|
||||
* `KUBE_GCE_ZONE=zone` - zone where the control plane node will run.
|
||||
Must be in the same region as other control plane nodes' zones.
|
||||
|
||||
You don't need to set the `MULTIZONE` or `ENABLE_ETCD_QUORUM_READS` flags,
|
||||
as those are inherited from when you started your HA-compatible cluster.
|
||||
|
||||
The following sample command replicates the master on an existing HA-compatible cluster:
|
||||
The following sample command replicates the control plane node on an existing
|
||||
HA-compatible cluster:
|
||||
|
||||
```shell
|
||||
KUBE_GCE_ZONE=europe-west1-c KUBE_REPLICATE_EXISTING_MASTER=true ./cluster/kube-up.sh
|
||||
```
|
||||
|
||||
## Removing a master replica
|
||||
## Removing a control plane node
|
||||
|
||||
You can remove a master replica from an HA cluster by using a `kube-down` script with the following flags:
|
||||
You can remove a control plane node from an HA cluster by using a `kube-down` script with the following flags:
|
||||
|
||||
* `KUBE_DELETE_NODES=false` - to restrain deletion of kubelets.
|
||||
|
||||
* `KUBE_GCE_ZONE=zone` - the zone from where master replica will be removed.
|
||||
* `KUBE_GCE_ZONE=zone` - the zone from where the control plane node will be removed.
|
||||
|
||||
* `KUBE_REPLICA_NAME=replica_name` - (optional) the name of master replica to remove.
|
||||
If empty: any replica from the given zone will be removed.
|
||||
* `KUBE_REPLICA_NAME=replica_name` - (optional) the name of control plane node to
|
||||
remove. If empty: any replica from the given zone will be removed.
|
||||
|
||||
The following sample command removes a master replica from an existing HA cluster:
|
||||
The following sample command removes a control plane node from an existing HA cluster:
|
||||
|
||||
```shell
|
||||
KUBE_DELETE_NODES=false KUBE_GCE_ZONE=europe-west1-c ./cluster/kube-down.sh
|
||||
```
|
||||
|
||||
## Handling master replica failures
|
||||
## Handling control plane node failures
|
||||
|
||||
If one of the master replicas in your HA cluster fails,
|
||||
the best practice is to remove the replica from your cluster and add a new replica in the same zone.
|
||||
If one of the control plane nodes in your HA cluster fails,
|
||||
the best practice is to remove the node from your cluster and add a new control plane
|
||||
node in the same zone.
|
||||
The following sample commands demonstrate this process:
|
||||
|
||||
1. Remove the broken replica:
|
||||
@@ -98,26 +101,31 @@ The following sample commands demonstrate this process:
|
||||
KUBE_DELETE_NODES=false KUBE_GCE_ZONE=replica_zone KUBE_REPLICA_NAME=replica_name ./cluster/kube-down.sh
|
||||
```
|
||||
|
||||
<ol start="2"><li>Add a new replica in place of the old one:</li></ol>
|
||||
<ol start="2"><li>Add a new node in place of the old one:</li></ol>
|
||||
|
||||
```shell
|
||||
KUBE_GCE_ZONE=replica-zone KUBE_REPLICATE_EXISTING_MASTER=true ./cluster/kube-up.sh
|
||||
```
|
||||
|
||||
## Best practices for replicating masters for HA clusters
|
||||
## Best practices for replicating control plane nodes for HA clusters
|
||||
|
||||
* Try to place master replicas in different zones. During a zone failure, all masters placed inside the zone will fail.
|
||||
* Try to place control plane nodes in different zones. During a zone failure, all
|
||||
control plane nodes placed inside the zone will fail.
|
||||
To survive zone failure, also place nodes in multiple zones
|
||||
(see [multiple-zones](/docs/setup/best-practices/multiple-zones/) for details).
|
||||
|
||||
* Do not use a cluster with two master replicas. Consensus on a two-replica cluster requires both replicas running when changing persistent state.
|
||||
As a result, both replicas are needed and a failure of any replica turns cluster into majority failure state.
|
||||
A two-replica cluster is thus inferior, in terms of HA, to a single replica cluster.
|
||||
* Do not use a cluster with two control plane nodes. Consensus on a two-node
|
||||
control plane requires both nodes running when changing persistent state.
|
||||
As a result, both nodes are needed and a failure of any node turns the cluster
|
||||
into majority failure state.
|
||||
A two-node control plane is thus inferior, in terms of HA, to a cluster with
|
||||
one control plane node.
|
||||
|
||||
* When you add a master replica, cluster state (etcd) is copied to a new instance.
|
||||
* When you add a control plane node, cluster state (etcd) is copied to a new instance.
|
||||
If the cluster is large, it may take a long time to duplicate its state.
|
||||
This operation may be sped up by migrating etcd data directory, as described [here](https://coreos.com/etcd/docs/latest/admin_guide.html#member-migration)
|
||||
(we are considering adding support for etcd data dir migration in future).
|
||||
This operation may be sped up by migrating the etcd data directory, as described in
|
||||
the [etcd administration guide](https://etcd.io/docs/v2.3/admin_guide/#member-migration)
|
||||
(we are considering adding support for etcd data dir migration in the future).
|
||||
|
||||
|
||||
|
||||
@@ -129,7 +137,7 @@ This operation may be sped up by migrating etcd data directory, as described [he
|
||||
|
||||
### Overview
|
||||
|
||||
Each of master replicas will run the following components in the following mode:
|
||||
Each of the control plane nodes will run the following components in the following mode:
|
||||
|
||||
* etcd instance: all instances will be clustered together using consensus;
|
||||
|
||||
@@ -143,9 +151,9 @@ In addition, there will be a load balancer in front of API servers that will rou
|
||||
|
||||
### Load balancing
|
||||
|
||||
When starting the second master replica, a load balancer containing the two replicas will be created
|
||||
When starting the second control plane node, a load balancer containing the two replicas will be created
|
||||
and the IP address of the first replica will be promoted to IP address of load balancer.
|
||||
Similarly, after removal of the penultimate master replica, the load balancer will be removed and its IP address will be assigned to the last remaining replica.
|
||||
Similarly, after removal of the penultimate control plane node, the load balancer will be removed and its IP address will be assigned to the last remaining replica.
|
||||
Please note that creation and removal of load balancer are complex operations and it may take some time (~20 minutes) for them to propagate.
|
||||
|
||||
### Master service & kubelets
|
||||
@@ -153,17 +161,17 @@ Please note that creation and removal of load balancer are complex operations an
|
||||
Instead of trying to keep an up-to-date list of Kubernetes apiserver in the Kubernetes service,
|
||||
the system directs all traffic to the external IP:
|
||||
|
||||
* in one master cluster the IP points to the single master,
|
||||
* in case of a single node control plane, the IP points to the control plane node,
|
||||
|
||||
* in multi-master cluster the IP points to the load balancer in-front of the masters.
|
||||
* in case of an HA control plane, the IP points to the load balancer in-front of the masters.
|
||||
|
||||
Similarly, the external IP will be used by kubelets to communicate with master.
|
||||
Similarly, the external IP will be used by kubelets to communicate with the control plane.
|
||||
|
||||
### Master certificates
|
||||
### Control plane node certificates
|
||||
|
||||
Kubernetes generates Master TLS certificates for the external public IP and local IP for each replica.
|
||||
There are no certificates for the ephemeral public IP for replicas;
|
||||
to access a replica via its ephemeral public IP, you must skip TLS verification.
|
||||
Kubernetes generates TLS certificates for the external public IP and local IP for each control plane node.
|
||||
There are no certificates for the ephemeral public IP for control plane nodes;
|
||||
to access a control plane node via its ephemeral public IP, you must skip TLS verification.
|
||||
|
||||
### Clustering etcd
|
||||
|
||||
@@ -172,7 +180,7 @@ To make such deployment secure, communication between etcd instances is authoriz
|
||||
|
||||
### API server identity
|
||||
|
||||
{{< feature-state state="alpha" for_k8s_version="v1.20" >}}
|
||||
{{< feature-state state="alpha" for_k8s_version="v1.20" >}}
|
||||
|
||||
The API Server Identity feature is controlled by a
|
||||
[feature gate](/docs/reference/command-line-tools-reference/feature-gates/)
|
||||
@@ -188,7 +188,7 @@ To install a specific version of containerD specify the version with -ContainerD
|
||||
|
||||
```powershell
|
||||
# Example
|
||||
.\Install-Containerd.ps1 -ContainerDVersion v1.4.1
|
||||
.\Install-Containerd.ps1 -ContainerDVersion 1.4.1
|
||||
```
|
||||
|
||||
{{< /note >}}
|
||||
|
||||
@@ -239,7 +239,7 @@ The field `serverTLSBootstrap: true` will enable the bootstrap of kubelet servin
|
||||
certificates by requesting them from the `certificates.k8s.io` API. One known limitation
|
||||
is that the CSRs (Certificate Signing Requests) for these certificates cannot be automatically
|
||||
approved by the default signer in the kube-controller-manager -
|
||||
[`kubernetes.io/kubelet-serving`](https://kubernetes.io/docs/reference/access-authn-authz/certificate-signing-requests/#kubernetes-signers).
|
||||
[`kubernetes.io/kubelet-serving`](/docs/reference/access-authn-authz/certificate-signing-requests/#kubernetes-signers).
|
||||
This will require action from the user or a third party controller.
|
||||
|
||||
These CSRs can be viewed using:
|
||||
|
||||
@@ -167,8 +167,8 @@ The output is similar to this:
|
||||
Name: build-robot-secret
|
||||
Namespace: default
|
||||
Labels: <none>
|
||||
Annotations: kubernetes.io/service-account.name=build-robot
|
||||
kubernetes.io/service-account.uid=da68f9c6-9d26-11e7-b84e-002dc52800da
|
||||
Annotations: kubernetes.io/service-account.name: build-robot
|
||||
kubernetes.io/service-account.uid: da68f9c6-9d26-11e7-b84e-002dc52800da
|
||||
|
||||
Type: kubernetes.io/service-account-token
|
||||
|
||||
|
||||
@@ -54,7 +54,7 @@ If you use a Docker credentials store, you won't see that `auth` entry but a `cr
|
||||
|
||||
## Create a Secret based on existing Docker credentials {#registry-secret-existing-credentials}
|
||||
|
||||
A Kubernetes cluster uses the Secret of `docker-registry` type to authenticate with
|
||||
A Kubernetes cluster uses the Secret of `kubernetes.io/dockerconfigjson` type to authenticate with
|
||||
a container registry to pull a private image.
|
||||
|
||||
If you already ran `docker login`, you can copy that credential into Kubernetes:
|
||||
|
||||
@@ -45,8 +45,12 @@ kubectl create namespace qos-example
|
||||
|
||||
For a Pod to be given a QoS class of Guaranteed:
|
||||
|
||||
* Every Container, including init containers, in the Pod must have a memory limit and a memory request, and they must be the same.
|
||||
* Every Container, including init containers, in the Pod must have a CPU limit and a CPU request, and they must be the same.
|
||||
* Every Container in the Pod must have a memory limit and a memory request.
|
||||
* For every Container in the Pod, the memory limit must equal the memory request.
|
||||
* Every Container in the Pod must have a CPU limit and a CPU request.
|
||||
* For every Container in the Pod, the CPU limit must equal the CPU request.
|
||||
|
||||
These restrictions apply to init containers and app containers equally.
|
||||
|
||||
Here is the configuration file for a Pod that has one Container. The Container has a memory limit and a
|
||||
memory request, both equal to 200 MiB. The Container has a CPU limit and a CPU request, both equal to 700 milliCPU:
|
||||
@@ -272,5 +276,3 @@ kubectl delete namespace qos-example
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -176,7 +176,6 @@ securityContext:
|
||||
fsGroupChangePolicy: "OnRootMismatch"
|
||||
```
|
||||
|
||||
This is an alpha feature. To use it, enable the [feature gate](/docs/reference/command-line-tools-reference/feature-gates/) `ConfigurableFSGroupPolicy` for the kube-api-server, the kube-controller-manager, and for the kubelet.
|
||||
|
||||
{{< note >}}
|
||||
This field has no effect on ephemeral volume types such as
|
||||
|
||||
@@ -1,371 +0,0 @@
|
||||
---
|
||||
reviewers:
|
||||
- piosz
|
||||
- x13n
|
||||
title: Logging Using Stackdriver
|
||||
content_type: concept
|
||||
---
|
||||
|
||||
<!-- overview -->
|
||||
|
||||
Before reading this page, it's highly recommended to familiarize yourself
|
||||
with the [overview of logging in Kubernetes](/docs/concepts/cluster-administration/logging).
|
||||
|
||||
{{< note >}}
|
||||
By default, Stackdriver logging collects only your container's standard output and
|
||||
standard error streams. To collect any logs your application writes to a file (for example),
|
||||
see the [sidecar approach](/docs/concepts/cluster-administration/logging#sidecar-container-with-a-logging-agent)
|
||||
in the Kubernetes logging overview.
|
||||
{{< /note >}}
|
||||
|
||||
|
||||
|
||||
|
||||
<!-- body -->
|
||||
|
||||
## Deploying
|
||||
|
||||
To ingest logs, you must deploy the Stackdriver Logging agent to each node in your cluster.
|
||||
The agent is a configured `fluentd` instance, where the configuration is stored in a `ConfigMap`
|
||||
and the instances are managed using a Kubernetes `DaemonSet`. The actual deployment of the
|
||||
`ConfigMap` and `DaemonSet` for your cluster depends on your individual cluster setup.
|
||||
|
||||
### Deploying to a new cluster
|
||||
|
||||
#### Google Kubernetes Engine
|
||||
|
||||
Stackdriver is the default logging solution for clusters deployed on Google Kubernetes Engine.
|
||||
Stackdriver Logging is deployed to a new cluster by default unless you explicitly opt-out.
|
||||
|
||||
#### Other platforms
|
||||
|
||||
To deploy Stackdriver Logging on a *new* cluster that you're
|
||||
creating using `kube-up.sh`, do the following:
|
||||
|
||||
1. Set the `KUBE_LOGGING_DESTINATION` environment variable to `gcp`.
|
||||
1. **If not running on GCE**, include the `beta.kubernetes.io/fluentd-ds-ready=true`
|
||||
in the `KUBE_NODE_LABELS` variable.
|
||||
|
||||
Once your cluster has started, each node should be running the Stackdriver Logging agent.
|
||||
The `DaemonSet` and `ConfigMap` are configured as addons. If you're not using `kube-up.sh`,
|
||||
consider starting a cluster without a pre-configured logging solution and then deploying
|
||||
Stackdriver Logging agents to the running cluster.
|
||||
|
||||
{{< warning >}}
|
||||
The Stackdriver logging daemon has known issues on platforms other
|
||||
than Google Kubernetes Engine. Proceed at your own risk.
|
||||
{{< /warning >}}
|
||||
|
||||
### Deploying to an existing cluster
|
||||
|
||||
1. Apply a label on each node, if not already present.
|
||||
|
||||
The Stackdriver Logging agent deployment uses node labels to determine to which nodes
|
||||
it should be allocated. These labels were introduced to distinguish nodes with the
|
||||
Kubernetes version 1.6 or higher. If the cluster was created with Stackdriver Logging
|
||||
configured and node has version 1.5.X or lower, it will have fluentd as static pod. Node
|
||||
cannot have more than one instance of fluentd, therefore only apply labels to the nodes
|
||||
that don't have fluentd pod allocated already. You can ensure that your node is labelled
|
||||
properly by running `kubectl describe` as follows:
|
||||
|
||||
```
|
||||
kubectl describe node $NODE_NAME
|
||||
```
|
||||
|
||||
The output should be similar to this:
|
||||
|
||||
```
|
||||
Name: NODE_NAME
|
||||
Role:
|
||||
Labels: beta.kubernetes.io/fluentd-ds-ready=true
|
||||
...
|
||||
```
|
||||
|
||||
Ensure that the output contains the label `beta.kubernetes.io/fluentd-ds-ready=true`. If it
|
||||
is not present, you can add it using the `kubectl label` command as follows:
|
||||
|
||||
```
|
||||
kubectl label node $NODE_NAME beta.kubernetes.io/fluentd-ds-ready=true
|
||||
```
|
||||
|
||||
{{< note >}}
|
||||
If a node fails and has to be recreated, you must re-apply the label to
|
||||
the recreated node. To make this easier, you can use Kubelet's command-line parameter
|
||||
for applying node labels in your node startup script.
|
||||
{{< /note >}}
|
||||
|
||||
1. Deploy a `ConfigMap` with the logging agent configuration by running the following command:
|
||||
|
||||
```
|
||||
kubectl apply -f https://k8s.io/examples/debug/fluentd-gcp-configmap.yaml
|
||||
```
|
||||
|
||||
The command creates the `ConfigMap` in the `default` namespace. You can download the file
|
||||
manually and change it before creating the `ConfigMap` object.
|
||||
|
||||
1. Deploy the logging agent `DaemonSet` by running the following command:
|
||||
|
||||
```
|
||||
kubectl apply -f https://k8s.io/examples/debug/fluentd-gcp-ds.yaml
|
||||
```
|
||||
|
||||
You can download and edit this file before using it as well.
|
||||
|
||||
## Verifying your Logging Agent Deployment
|
||||
|
||||
After Stackdriver `DaemonSet` is deployed, you can discover logging agent deployment status
|
||||
by running the following command:
|
||||
|
||||
```shell
|
||||
kubectl get ds --all-namespaces
|
||||
```
|
||||
|
||||
If you have 3 nodes in the cluster, the output should looks similar to this:
|
||||
|
||||
```
|
||||
NAMESPACE NAME DESIRED CURRENT READY NODE-SELECTOR AGE
|
||||
...
|
||||
default fluentd-gcp-v2.0 3 3 3 beta.kubernetes.io/fluentd-ds-ready=true 5m
|
||||
...
|
||||
```
|
||||
|
||||
To understand how logging with Stackdriver works, consider the following
|
||||
synthetic log generator pod specification [counter-pod.yaml](/examples/debug/counter-pod.yaml):
|
||||
|
||||
{{< codenew file="debug/counter-pod.yaml" >}}
|
||||
|
||||
This pod specification has one container that runs a bash script
|
||||
that writes out the value of a counter and the datetime once per
|
||||
second, and runs indefinitely. Let's create this pod in the default namespace.
|
||||
|
||||
```shell
|
||||
kubectl apply -f https://k8s.io/examples/debug/counter-pod.yaml
|
||||
```
|
||||
|
||||
You can observe the running pod:
|
||||
|
||||
```shell
|
||||
kubectl get pods
|
||||
```
|
||||
```
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
counter 1/1 Running 0 5m
|
||||
```
|
||||
|
||||
For a short period of time you can observe the 'Pending' pod status, because the kubelet
|
||||
has to download the container image first. When the pod status changes to `Running`
|
||||
you can use the `kubectl logs` command to view the output of this counter pod.
|
||||
|
||||
```shell
|
||||
kubectl logs counter
|
||||
```
|
||||
```
|
||||
0: Mon Jan 1 00:00:00 UTC 2001
|
||||
1: Mon Jan 1 00:00:01 UTC 2001
|
||||
2: Mon Jan 1 00:00:02 UTC 2001
|
||||
...
|
||||
```
|
||||
|
||||
As described in the logging overview, this command fetches log entries
|
||||
from the container log file. If the container is killed and then restarted by
|
||||
Kubernetes, you can still access logs from the previous container. However,
|
||||
if the pod is evicted from the node, log files are lost. Let's demonstrate this
|
||||
by deleting the currently running counter container:
|
||||
|
||||
```shell
|
||||
kubectl delete pod counter
|
||||
```
|
||||
```
|
||||
pod "counter" deleted
|
||||
```
|
||||
|
||||
and then recreating it:
|
||||
|
||||
```shell
|
||||
kubectl create -f https://k8s.io/examples/debug/counter-pod.yaml
|
||||
```
|
||||
```
|
||||
pod/counter created
|
||||
```
|
||||
|
||||
After some time, you can access logs from the counter pod again:
|
||||
|
||||
```shell
|
||||
kubectl logs counter
|
||||
```
|
||||
```
|
||||
0: Mon Jan 1 00:01:00 UTC 2001
|
||||
1: Mon Jan 1 00:01:01 UTC 2001
|
||||
2: Mon Jan 1 00:01:02 UTC 2001
|
||||
...
|
||||
```
|
||||
|
||||
As expected, only recent log lines are present. However, for a real-world
|
||||
application you will likely want to be able to access logs from all containers,
|
||||
especially for the debug purposes. This is exactly when the previously enabled
|
||||
Stackdriver Logging can help.
|
||||
|
||||
## Viewing logs
|
||||
|
||||
Stackdriver Logging agent attaches metadata to each log entry, for you to use later
|
||||
in queries to select only the messages you're interested in: for example,
|
||||
the messages from a particular pod.
|
||||
|
||||
The most important pieces of metadata are the resource type and log name.
|
||||
The resource type of a container log is `container`, which is named
|
||||
`GKE Containers` in the UI (even if the Kubernetes cluster is not on Google Kubernetes Engine).
|
||||
The log name is the name of the container, so that if you have a pod with
|
||||
two containers, named `container_1` and `container_2` in the spec, their logs
|
||||
will have log names `container_1` and `container_2` respectively.
|
||||
|
||||
System components have resource type `compute`, which is named
|
||||
`GCE VM Instance` in the interface. Log names for system components are fixed.
|
||||
For a Google Kubernetes Engine node, every log entry from a system component has one of the following
|
||||
log names:
|
||||
|
||||
* docker
|
||||
* kubelet
|
||||
* kube-proxy
|
||||
|
||||
You can learn more about viewing logs on [the dedicated Stackdriver page](https://cloud.google.com/logging/docs/view/logs_viewer).
|
||||
|
||||
One of the possible ways to view logs is using the
|
||||
[`gcloud logging`](https://cloud.google.com/logging/docs/api/gcloud-logging)
|
||||
command line interface from the [Google Cloud SDK](https://cloud.google.com/sdk/).
|
||||
It uses Stackdriver Logging [filtering syntax](https://cloud.google.com/logging/docs/view/advanced_filters)
|
||||
to query specific logs. For example, you can run the following command:
|
||||
|
||||
```none
|
||||
gcloud beta logging read 'logName="projects/$YOUR_PROJECT_ID/logs/count"' --format json | jq '.[].textPayload'
|
||||
```
|
||||
```
|
||||
...
|
||||
"2: Mon Jan 1 00:01:02 UTC 2001\n"
|
||||
"1: Mon Jan 1 00:01:01 UTC 2001\n"
|
||||
"0: Mon Jan 1 00:01:00 UTC 2001\n"
|
||||
...
|
||||
"2: Mon Jan 1 00:00:02 UTC 2001\n"
|
||||
"1: Mon Jan 1 00:00:01 UTC 2001\n"
|
||||
"0: Mon Jan 1 00:00:00 UTC 2001\n"
|
||||
```
|
||||
|
||||
As you can see, it outputs messages for the count container from both
|
||||
the first and second runs, despite the fact that the kubelet already deleted
|
||||
the logs for the first container.
|
||||
|
||||
### Exporting logs
|
||||
|
||||
You can export logs to [Google Cloud Storage](https://cloud.google.com/storage/)
|
||||
or to [BigQuery](https://cloud.google.com/bigquery/) to run further
|
||||
analysis. Stackdriver Logging offers the concept of sinks, where you can
|
||||
specify the destination of log entries. More information is available on
|
||||
the Stackdriver [Exporting Logs page](https://cloud.google.com/logging/docs/export/configure_export_v2).
|
||||
|
||||
## Configuring Stackdriver Logging Agents
|
||||
|
||||
Sometimes the default installation of Stackdriver Logging may not suit your needs, for example:
|
||||
|
||||
* You may want to add more resources because default performance doesn't suit your needs.
|
||||
* You may want to introduce additional parsing to extract more metadata from your log messages,
|
||||
like severity or source code reference.
|
||||
* You may want to send logs not only to Stackdriver or send it to Stackdriver only partially.
|
||||
|
||||
In this case you need to be able to change the parameters of `DaemonSet` and `ConfigMap`.
|
||||
|
||||
### Prerequisites
|
||||
|
||||
If you're using GKE and Stackdriver Logging is enabled in your cluster, you
|
||||
cannot change its configuration, because it's managed and supported by GKE.
|
||||
However, you can disable the default integration and deploy your own.
|
||||
|
||||
{{< note >}}
|
||||
You will have to support and maintain a newly deployed configuration
|
||||
yourself: update the image and configuration, adjust the resources and so on.
|
||||
{{< /note >}}
|
||||
|
||||
To disable the default logging integration, use the following command:
|
||||
|
||||
```
|
||||
gcloud beta container clusters update --logging-service=none CLUSTER
|
||||
```
|
||||
|
||||
You can find notes on how to then install Stackdriver Logging agents into
|
||||
a running cluster in the [Deploying section](#deploying).
|
||||
|
||||
### Changing `DaemonSet` parameters
|
||||
|
||||
When you have the Stackdriver Logging `DaemonSet` in your cluster, you can modify the
|
||||
`template` field in its spec. The DaemonSet controller manages the pods for you.
|
||||
For example, assume you've installed the Stackdriver Logging as described above. Now you want to
|
||||
change the memory limit to give fluentd more memory to safely process more logs.
|
||||
|
||||
Get the spec of `DaemonSet` running in your cluster:
|
||||
|
||||
```shell
|
||||
kubectl get ds fluentd-gcp-v2.0 --namespace kube-system -o yaml > fluentd-gcp-ds.yaml
|
||||
```
|
||||
|
||||
Then edit resource requirements in the spec file and update the `DaemonSet` object
|
||||
in the apiserver using the following command:
|
||||
|
||||
```shell
|
||||
kubectl replace -f fluentd-gcp-ds.yaml
|
||||
```
|
||||
|
||||
After some time, Stackdriver Logging agent pods will be restarted with the new configuration.
|
||||
|
||||
### Changing fluentd parameters
|
||||
|
||||
Fluentd configuration is stored in the `ConfigMap` object. It is effectively a set of configuration
|
||||
files that are merged together. You can learn about fluentd configuration on the
|
||||
[official site](https://docs.fluentd.org).
|
||||
|
||||
Imagine you want to add a new parsing logic to the configuration, so that fluentd can understand
|
||||
default Python logging format. An appropriate fluentd filter looks similar to this:
|
||||
|
||||
```
|
||||
<filter reform.**>
|
||||
type parser
|
||||
format /^(?<severity>\w):(?<logger_name>\w):(?<log>.*)/
|
||||
reserve_data true
|
||||
suppress_parse_error_log true
|
||||
key_name log
|
||||
</filter>
|
||||
```
|
||||
|
||||
Now you have to put it in the configuration and make Stackdriver Logging agents pick it up.
|
||||
Get the current version of the Stackdriver Logging `ConfigMap` in your cluster
|
||||
by running the following command:
|
||||
|
||||
```shell
|
||||
kubectl get cm fluentd-gcp-config --namespace kube-system -o yaml > fluentd-gcp-configmap.yaml
|
||||
```
|
||||
|
||||
Then in the value of the key `containers.input.conf` insert a new filter right after
|
||||
the `source` section.
|
||||
|
||||
{{< note >}}
|
||||
Order is important.
|
||||
{{< /note >}}
|
||||
|
||||
Updating `ConfigMap` in the apiserver is more complicated than updating `DaemonSet`. It's better
|
||||
to consider `ConfigMap` to be immutable. Then, in order to update the configuration, you should
|
||||
create `ConfigMap` with a new name and then change `DaemonSet` to point to it
|
||||
using [guide above](#changing-daemonset-parameters).
|
||||
|
||||
### Adding fluentd plugins
|
||||
|
||||
Fluentd is written in Ruby and allows to extend its capabilities using
|
||||
[plugins](https://www.fluentd.org/plugins). If you want to use a plugin, which is not included
|
||||
in the default Stackdriver Logging container image, you have to build a custom image. Imagine
|
||||
you want to add Kafka sink for messages from a particular container for additional processing.
|
||||
You can re-use the default [container image sources](https://git.k8s.io/contrib/fluentd/fluentd-gcp-image)
|
||||
with minor changes:
|
||||
|
||||
* Change Makefile to point to your container repository, for example `PREFIX=gcr.io/<your-project-id>`.
|
||||
* Add your dependency to the Gemfile, for example `gem 'fluent-plugin-kafka'`.
|
||||
|
||||
Then run `make build push` from this directory. After updating `DaemonSet` to pick up the
|
||||
new image, you can use the plugin you installed in the fluentd configuration.
|
||||
|
||||
|
||||
@@ -31,6 +31,7 @@ The following methods exist for installing kubectl on macOS:
|
||||
{{< tabs name="download_binary_macos" >}}
|
||||
{{< tab name="Intel" codelang="bash" >}}
|
||||
curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/darwin/amd64/kubectl"
|
||||
chmod +x kubectl
|
||||
{{< /tab >}}
|
||||
{{< tab name="Apple Silicon" codelang="bash" >}}
|
||||
curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/darwin/arm64/kubectl"
|
||||
|
||||
Reference in New Issue
Block a user