Qiming Teng
ca36ac797c
The `TracingConfiguration` struct we reference from the system traces page only exists in the v1alpha1 version of the API server configuration. This PR fixes the problem.
90 lines
3.2 KiB
Markdown
90 lines
3.2 KiB
Markdown
---
|
|
title: Traces For Kubernetes System Components
|
|
reviewers:
|
|
- logicalhan
|
|
- lilic
|
|
content_type: concept
|
|
weight: 60
|
|
---
|
|
|
|
<!-- overview -->
|
|
|
|
{{< feature-state for_k8s_version="v1.22" state="alpha" >}}
|
|
|
|
System component traces record the latency of and relationships between operations in the cluster.
|
|
|
|
Kubernetes components emit traces using the
|
|
[OpenTelemetry Protocol](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/protocol/otlp.md#opentelemetry-protocol-specification)
|
|
with the gRPC exporter and can be collected and routed to tracing backends using an
|
|
[OpenTelemetry Collector](https://github.com/open-telemetry/opentelemetry-collector#-opentelemetry-collector).
|
|
|
|
<!-- body -->
|
|
|
|
## Trace Collection
|
|
|
|
For a complete guide to collecting traces and using the collector, see
|
|
[Getting Started with the OpenTelemetry Collector](https://opentelemetry.io/docs/collector/getting-started/).
|
|
However, there are a few things to note that are specific to Kubernetes components.
|
|
|
|
By default, Kubernetes components export traces using the grpc exporter for OTLP on the
|
|
[IANA OpenTelemetry port](https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.xhtml?search=opentelemetry), 4317.
|
|
As an example, if the collector is running as a sidecar to a Kubernetes component,
|
|
the following receiver configuration will collect spans and log them to standard output:
|
|
|
|
```yaml
|
|
receivers:
|
|
otlp:
|
|
protocols:
|
|
grpc:
|
|
exporters:
|
|
# Replace this exporter with the exporter for your backend
|
|
logging:
|
|
logLevel: debug
|
|
service:
|
|
pipelines:
|
|
traces:
|
|
receivers: [otlp]
|
|
exporters: [logging]
|
|
```
|
|
|
|
## Component traces
|
|
|
|
### kube-apiserver traces
|
|
|
|
The kube-apiserver generates spans for incoming HTTP requests, and for outgoing requests
|
|
to webhooks, etcd, and re-entrant requests. It propagates the
|
|
[W3C Trace Context](https://www.w3.org/TR/trace-context/) with outgoing requests
|
|
but does not make use of the trace context attached to incoming requests,
|
|
as the kube-apiserver is often a public endpoint.
|
|
|
|
#### Enabling tracing in the kube-apiserver
|
|
|
|
To enable tracing, enable the `APIServerTracing`
|
|
[feature gate](/docs/reference/command-line-tools-reference/feature-gates/)
|
|
on the kube-apiserver. Also, provide the kube-apiserver with a tracing configration file
|
|
with `--tracing-config-file=<path-to-config>`. This is an example config that records
|
|
spans for 1 in 10000 requests, and uses the default OpenTelemetry endpoint:
|
|
|
|
```yaml
|
|
apiVersion: apiserver.config.k8s.io/v1alpha1
|
|
kind: TracingConfiguration
|
|
# default value
|
|
#endpoint: localhost:4317
|
|
samplingRatePerMillion: 100
|
|
```
|
|
|
|
For more information about the `TracingConfiguration` struct, see
|
|
[API server config API (v1alpha1)](/docs/reference/config-api/apiserver-config.v1alpha1/#apiserver-k8s-io-v1alpha1-TracingConfiguration).
|
|
|
|
## Stability
|
|
|
|
Tracing instrumentation is still under active development, and may change
|
|
in a variety of ways. This includes span names, attached attributes,
|
|
instrumented endpoints, etc. Until this feature graduates to stable,
|
|
there are no guarantees of backwards compatibility for tracing instrumentation.
|
|
|
|
## {{% heading "whatsnext" %}}
|
|
|
|
* Read about [Getting Started with the OpenTelemetry Collector](https://opentelemetry.io/docs/collector/getting-started/)
|
|
|