Use inline links when possible

This is in prep for link checker. By using inline links everywhere, we:

- reduce the risk of dangling, missing, unused links as you can see from the PR;
- simplify the link checker logic (under development).
This commit is contained in:
Qiming Teng
2020-07-15 17:09:32 +08:00
parent 896034cbc2
commit 31730ce174
8 changed files with 75 additions and 126 deletions
@@ -82,7 +82,8 @@ and the former approach is used in any other environment. In both cases, by
default rotation is configured to take place when log file exceeds 10MB. default rotation is configured to take place when log file exceeds 10MB.
As an example, you can find detailed information about how `kube-up.sh` sets As an example, you can find detailed information about how `kube-up.sh` sets
up logging for COS image on GCP in the corresponding [script][cosConfigureHelper]. up logging for COS image on GCP in the corresponding
[script](https://github.com/kubernetes/kubernetes/blob/{{< param "githubbranch" >}}/cluster/gce/gci/configure-helper.sh)
When you run [`kubectl logs`](/docs/reference/generated/kubectl/kubectl-commands#logs) as in When you run [`kubectl logs`](/docs/reference/generated/kubectl/kubectl-commands#logs) as in
the basic logging example, the kubelet on the node handles the request and the basic logging example, the kubelet on the node handles the request and
@@ -96,8 +97,6 @@ the rotation and there are two files, one 10MB in size and one empty,
`kubectl logs` will return an empty response. `kubectl logs` will return an empty response.
{{< /note >}} {{< /note >}}
[cosConfigureHelper]: https://github.com/kubernetes/kubernetes/blob/{{< param "githubbranch" >}}/cluster/gce/gci/configure-helper.sh
### System component logs ### System component logs
There are two types of system components: those that run in a container and those There are two types of system components: those that run in a container and those
@@ -109,7 +108,7 @@ that do not run in a container. For example:
On machines with systemd, the kubelet and container runtime write to journald. If On machines with systemd, the kubelet and container runtime write to journald. If
systemd is not present, they write to `.log` files in the `/var/log` directory. systemd is not present, they write to `.log` files in the `/var/log` directory.
System components inside containers always write to the `/var/log` directory, System components inside containers always write to the `/var/log` directory,
bypassing the default logging mechanism. They use the [klog][klog] bypassing the default logging mechanism. They use the [klog](https://github.com/kubernetes/klog)
logging library. You can find the conventions for logging severity for those logging library. You can find the conventions for logging severity for those
components in the [development docs on logging](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-instrumentation/logging.md). components in the [development docs on logging](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-instrumentation/logging.md).
@@ -118,8 +117,6 @@ directory should be rotated. In Kubernetes clusters brought up by
the `kube-up.sh` script, those logs are configured to be rotated by the `kube-up.sh` script, those logs are configured to be rotated by
the `logrotate` tool daily or once the size exceeds 100MB. the `logrotate` tool daily or once the size exceeds 100MB.
[klog]: https://github.com/kubernetes/klog
## Cluster-level logging architectures ## Cluster-level logging architectures
While Kubernetes does not provide a native solution for cluster-level logging, there are several common approaches you can consider. Here are some options: While Kubernetes does not provide a native solution for cluster-level logging, there are several common approaches you can consider. Here are some options:
@@ -138,9 +138,7 @@ table](https://github.com/cri-o/cri-o/blob/master/docs/crio.conf.5.md#crioruntim
runtime_path = "${PATH_TO_BINARY}" runtime_path = "${PATH_TO_BINARY}"
``` ```
See CRI-O's [config documentation][100] for more details. See CRI-O's [config documentation](https://raw.githubusercontent.com/cri-o/cri-o/9f11d1d/docs/crio.conf.5.md) for more details.
[100]: https://raw.githubusercontent.com/cri-o/cri-o/9f11d1d/docs/crio.conf.5.md
## Scheduling ## Scheduling
@@ -149,7 +147,8 @@ See CRI-O's [config documentation][100] for more details.
As of Kubernetes v1.16, RuntimeClass includes support for heterogenous clusters through its As of Kubernetes v1.16, RuntimeClass includes support for heterogenous clusters through its
`scheduling` fields. Through the use of these fields, you can ensure that pods running with this `scheduling` fields. Through the use of these fields, you can ensure that pods running with this
RuntimeClass are scheduled to nodes that support it. To use the scheduling support, you must have RuntimeClass are scheduled to nodes that support it. To use the scheduling support, you must have
the [RuntimeClass admission controller][] enabled (the default, as of 1.16). the [RuntimeClass admission controller](/docs/reference/access-authn-authz/admission-controllers/#runtimeclass)
enabled (the default, as of 1.16).
To ensure pods land on nodes supporting a specific RuntimeClass, that set of nodes should have a To ensure pods land on nodes supporting a specific RuntimeClass, that set of nodes should have a
common label which is then selected by the `runtimeclass.scheduling.nodeSelector` field. The common label which is then selected by the `runtimeclass.scheduling.nodeSelector` field. The
@@ -165,8 +164,6 @@ by each.
To learn more about configuring the node selector and tolerations, see [Assigning Pods to To learn more about configuring the node selector and tolerations, see [Assigning Pods to
Nodes](/docs/concepts/scheduling-eviction/assign-pod-node/). Nodes](/docs/concepts/scheduling-eviction/assign-pod-node/).
[RuntimeClass admission controller]: /docs/reference/access-authn-authz/admission-controllers/#runtimeclass
### Pod Overhead ### Pod Overhead
{{< feature-state for_k8s_version="v1.18" state="beta" >}} {{< feature-state for_k8s_version="v1.18" state="beta" >}}
@@ -28,7 +28,7 @@ Kubernetes requires PKI for the following operations:
* Client certificate for the API server to talk to etcd * Client certificate for the API server to talk to etcd
* Client certificate/kubeconfig for the controller manager to talk to the API server * Client certificate/kubeconfig for the controller manager to talk to the API server
* Client certificate/kubeconfig for the scheduler to talk to the API server. * Client certificate/kubeconfig for the scheduler to talk to the API server.
* Client and server certificates for the [front-proxy][proxy] * Client and server certificates for the [front-proxy](/docs/tasks/extend-kubernetes/configure-aggregation-layer/)
{{< note >}} {{< note >}}
`front-proxy` certificates are required only if you run kube-proxy to support [an extension API server](/docs/tasks/extend-kubernetes/setup-extension-api-server/). `front-proxy` certificates are required only if you run kube-proxy to support [an extension API server](/docs/tasks/extend-kubernetes/setup-extension-api-server/).
@@ -54,7 +54,7 @@ Required CAs:
|------------------------|---------------------------|----------------------------------| |------------------------|---------------------------|----------------------------------|
| ca.crt,key | kubernetes-ca | Kubernetes general CA | | ca.crt,key | kubernetes-ca | Kubernetes general CA |
| etcd/ca.crt,key | etcd-ca | For all etcd-related functions | | etcd/ca.crt,key | etcd-ca | For all etcd-related functions |
| front-proxy-ca.crt,key | kubernetes-front-proxy-ca | For the [front-end proxy][proxy] | | front-proxy-ca.crt,key | kubernetes-front-proxy-ca | For the [front-end proxy](/docs/tasks/extend-kubernetes/configure-aggregation-layer/) |
On top of the above CAs, it is also necessary to get a public/private key pair for service account management, `sa.key` and `sa.pub`. On top of the above CAs, it is also necessary to get a public/private key pair for service account management, `sa.key` and `sa.pub`.
@@ -74,10 +74,11 @@ Required certificates:
| kube-apiserver-kubelet-client | kubernetes-ca | system:masters | client | | | kube-apiserver-kubelet-client | kubernetes-ca | system:masters | client | |
| front-proxy-client | kubernetes-front-proxy-ca | | client | | | front-proxy-client | kubernetes-front-proxy-ca | | client | |
[1]: any other IP or DNS name you contact your cluster on (as used by [kubeadm][kubeadm] the load balancer stable IP and/or DNS name, `kubernetes`, `kubernetes.default`, `kubernetes.default.svc`, [1]: any other IP or DNS name you contact your cluster on (as used by [kubeadm](/docs/reference/setup-tools/kubeadm/kubeadm/)
the load balancer stable IP and/or DNS name, `kubernetes`, `kubernetes.default`, `kubernetes.default.svc`,
`kubernetes.default.svc.cluster`, `kubernetes.default.svc.cluster.local`) `kubernetes.default.svc.cluster`, `kubernetes.default.svc.cluster.local`)
where `kind` maps to one or more of the [x509 key usage][usage] types: where `kind` maps to one or more of the [x509 key usage](https://godoc.org/k8s.io/api/certificates/v1beta1#KeyUsage) types:
| kind | Key usage | | kind | Key usage |
|--------|---------------------------------------------------------------------------------| |--------|---------------------------------------------------------------------------------|
@@ -99,7 +100,8 @@ For kubeadm users only:
### Certificate paths ### Certificate paths
Certificates should be placed in a recommended path (as used by [kubeadm][kubeadm]). Paths should be specified using the given argument regardless of location. Certificates should be placed in a recommended path (as used by [kubeadm](/docs/reference/setup-tools/kubeadm/kubeadm/)).
Paths should be specified using the given argument regardless of location.
| Default CN | recommended key path | recommended cert path | command | key argument | cert argument | | Default CN | recommended key path | recommended cert path | command | key argument | cert argument |
|------------------------------|------------------------------|-----------------------------|----------------|------------------------------|-------------------------------------------| |------------------------------|------------------------------|-----------------------------|----------------|------------------------------|-------------------------------------------|
@@ -160,8 +162,4 @@ These files are used as follows:
| controller-manager.conf | kube-controller-manager | Must be added to manifest in `manifests/kube-controller-manager.yaml` | | controller-manager.conf | kube-controller-manager | Must be added to manifest in `manifests/kube-controller-manager.yaml` |
| scheduler.conf | kube-scheduler | Must be added to manifest in `manifests/kube-scheduler.yaml` | | scheduler.conf | kube-scheduler | Must be added to manifest in `manifests/kube-scheduler.yaml` |
[usage]: https://godoc.org/k8s.io/api/certificates/v1beta1#KeyUsage
[kubeadm]: /docs/reference/setup-tools/kubeadm/kubeadm/
[proxy]: /docs/tasks/extend-kubernetes/configure-aggregation-layer/
@@ -23,22 +23,15 @@ becoming unavailable. This task walks through the process of creating a high
availability etcd cluster of three members that can be used as an external etcd availability etcd cluster of three members that can be used as an external etcd
when using kubeadm to set up a kubernetes cluster. when using kubeadm to set up a kubernetes cluster.
## {{% heading "prerequisites" %}} ## {{% heading "prerequisites" %}}
* Three hosts that can talk to each other over ports 2379 and 2380. This * Three hosts that can talk to each other over ports 2379 and 2380. This
document assumes these default ports. However, they are configurable through document assumes these default ports. However, they are configurable through
the kubeadm config file. the kubeadm config file.
* Each host must [have docker, kubelet, and kubeadm installed][toolbox]. * Each host must [have docker, kubelet, and kubeadm installed](/docs/setup/production-environment/tools/kubeadm/install-kubeadm/).
* Some infrastructure to copy files between hosts. For example `ssh` and `scp` * Some infrastructure to copy files between hosts. For example `ssh` and `scp`
can satisfy this requirement. can satisfy this requirement.
[toolbox]: /docs/setup/production-environment/tools/kubeadm/install-kubeadm/
<!-- steps --> <!-- steps -->
## Setting up the cluster ## Setting up the cluster
+1 -3
View File
@@ -63,11 +63,9 @@ filename | sha512 hash
## Changelog since v1.17.0 ## Changelog since v1.17.0
A complete changelog for the release notes is now hosted in a customizable A complete changelog for the release notes is now hosted in a customizable
format at [https://relnotes.k8s.io][1]. Check it out and please give us your format at [https://relnotes.k8s.io](https://relnotes.k8s.io/?releaseVersions=1.18.0). Check it out and please give us your
feedback! feedback!
[1]: https://relnotes.k8s.io/?releaseVersions=1.18.0
## Whats New (Major Themes) ## Whats New (Major Themes)
### Kubernetes Topology Manager Moves to Beta - Align Up! ### Kubernetes Topology Manager Moves to Beta - Align Up!
@@ -12,15 +12,11 @@ weight: 10
Client certificates generated by [kubeadm](/docs/reference/setup-tools/kubeadm/kubeadm/) expire after 1 year. This page explains how to manage certificate renewals with kubeadm. Client certificates generated by [kubeadm](/docs/reference/setup-tools/kubeadm/kubeadm/) expire after 1 year. This page explains how to manage certificate renewals with kubeadm.
## {{% heading "prerequisites" %}} ## {{% heading "prerequisites" %}}
You should be familiar with [PKI certificates and requirements in Kubernetes](/docs/setup/best-practices/certificates/). You should be familiar with [PKI certificates and requirements in Kubernetes](/docs/setup/best-practices/certificates/).
<!-- steps --> <!-- steps -->
## Using custom certificates {#custom-certificates} ## Using custom certificates {#custom-certificates}
@@ -155,13 +151,13 @@ These are advanced topics for users who need to integrate their organization's c
### Set up a signer ### Set up a signer
The Kubernetes Certificate Authority does not work out of the box. The Kubernetes Certificate Authority does not work out of the box.
You can configure an external signer such as [cert-manager][cert-manager-issuer], or you can use the built-in signer. You can configure an external signer such as [cert-manager](https://docs.cert-manager.io/en/latest/tasks/issuers/setup-ca.html), or you can use the built-in signer.
The built-in signer is part of [`kube-controller-manager`][kcm]. The built-in signer is part of [`kube-controller-manager`](/docs/reference/command-line-tools-reference/kube-controller-manager/).
To activate the built-in signer, you must pass the `--cluster-signing-cert-file` and `--cluster-signing-key-file` flags. To activate the built-in signer, you must pass the `--cluster-signing-cert-file` and `--cluster-signing-key-file` flags.
If you're creating a new cluster, you can use a kubeadm [configuration file][config]: If you're creating a new cluster, you can use a kubeadm [configuration file](https://godoc.org/k8s.io/kubernetes/cmd/kubeadm/app/apis/kubeadm/v1beta2):
```yaml ```yaml
apiVersion: kubeadm.k8s.io/v1beta2 apiVersion: kubeadm.k8s.io/v1beta2
@@ -172,16 +168,12 @@ If you're creating a new cluster, you can use a kubeadm [configuration file][con
cluster-signing-key-file: /etc/kubernetes/pki/ca.key cluster-signing-key-file: /etc/kubernetes/pki/ca.key
``` ```
[cert-manager-issuer]: https://docs.cert-manager.io/en/latest/tasks/issuers/setup-ca.html
[kcm]: /docs/reference/command-line-tools-reference/kube-controller-manager/
[config]: https://godoc.org/k8s.io/kubernetes/cmd/kubeadm/app/apis/kubeadm/v1beta2
### Create certificate signing requests (CSR) ### Create certificate signing requests (CSR)
You can create the certificate signing requests for the Kubernetes certificates API with `kubeadm alpha certs renew --use-api`. You can create the certificate signing requests for the Kubernetes certificates API with `kubeadm alpha certs renew --use-api`.
If you set up an external signer such as [cert-manager][cert-manager], certificate signing requests (CSRs) are automatically approved. If you set up an external signer such as [cert-manager](https://github.com/jetstack/cert-manager), certificate signing requests (CSRs) are automatically approved.
Otherwise, you must manually approve certificates with the [`kubectl certificate`][certs] command. Otherwise, you must manually approve certificates with the [`kubectl certificate`](/docs/setup/best-practices/certificates/) command.
The following kubeadm command outputs the name of the certificate to approve, then blocks and waits for approval to occur: The following kubeadm command outputs the name of the certificate to approve, then blocks and waits for approval to occur:
```shell ```shell
@@ -197,7 +189,7 @@ The output is similar to this:
If you set up an external signer, certificate signing requests (CSRs) are automatically approved. If you set up an external signer, certificate signing requests (CSRs) are automatically approved.
Otherwise, you must manually approve certificates with the [`kubectl certificate`][certs] command. e.g. Otherwise, you must manually approve certificates with the [`kubectl certificate`](/docs/setup/best-practices/certificates/) command. e.g.
```shell ```shell
kubectl certificate approve kubeadm-cert-kube-apiserver-ld526 kubectl certificate approve kubeadm-cert-kube-apiserver-ld526
@@ -229,20 +221,16 @@ Certificates can be renewed with `kubeadm alpha certs renew --csr-only`.
As with `kubeadm init`, an output directory can be specified with the `--csr-dir` flag. As with `kubeadm init`, an output directory can be specified with the `--csr-dir` flag.
A CSR contains a certificate's name, domains, and IPs, but it does not specify usages. A CSR contains a certificate's name, domains, and IPs, but it does not specify usages.
It is the responsibility of the CA to specify [the correct cert usages][cert-table] when issuing a certificate. It is the responsibility of the CA to specify [the correct cert usages](/docs/setup/best-practices/certificates/#all-certificates)
when issuing a certificate.
* In `openssl` this is done with the [`openssl ca` command][openssl-ca]. * In `openssl` this is done with the
* In `cfssl` you specify [usages in the config file][cfssl-usages] [`openssl ca` command](https://superuser.com/questions/738612/openssl-ca-keyusage-extension).
* In `cfssl` you specify
[usages in the config file](https://github.com/cloudflare/cfssl/blob/master/doc/cmd/cfssl.txt#L170).
After a certificate is signed using your preferred method, the certificate and the private key must be copied to the PKI directory (by default `/etc/kubernetes/pki`). After a certificate is signed using your preferred method, the certificate and the private key must be copied to the PKI directory (by default `/etc/kubernetes/pki`).
[cert-manager]: https://github.com/jetstack/cert-manager
[openssl-ca]: https://superuser.com/questions/738612/openssl-ca-keyusage-extension
[cfssl-usages]: https://github.com/cloudflare/cfssl/blob/master/doc/cmd/cfssl.txt#L170
[certs]: /docs/setup/best-practices/certificates/
[cert-cas]: /docs/setup/best-practices/certificates/#single-root-ca
[cert-table]: /docs/setup/best-practices/certificates/#all-certificates
## Certificate authority (CA) rotation {#certificate-authority-rotation} ## Certificate authority (CA) rotation {#certificate-authority-rotation}
Kubeadm does not support rotation or replacement of CA certificates out of the box. Kubeadm does not support rotation or replacement of CA certificates out of the box.
@@ -22,12 +22,10 @@ answer the following questions:
- from where was it initiated? - from where was it initiated?
- to where was it going? - to where was it going?
<!-- body --> <!-- body -->
[Kube-apiserver][kube-apiserver] performs auditing. Each request on each stage [Kube-apiserver](/docs/reference/command-line-tools-reference/kube-apiserver/)
performs auditing. Each request on each stage
of its execution generates an event, which is then pre-processed according to of its execution generates an event, which is then pre-processed according to
a certain policy and written to a backend. The policy determines what's recorded a certain policy and written to a backend. The policy determines what's recorded
and the backends persist the records. The current backend implementations and the backends persist the records. The current backend implementations
@@ -55,7 +53,8 @@ Additionally, memory consumption depends on the audit logging configuration.
Audit policy defines rules about what events should be recorded and what data Audit policy defines rules about what events should be recorded and what data
they should include. The audit policy object structure is defined in the they should include. The audit policy object structure is defined in the
[`audit.k8s.io` API group][auditing-api]. When an event is processed, it's [`audit.k8s.io` API group](https://github.com/kubernetes/kubernetes/blob/{{< param "githubbranch" >}}/staging/src/k8s.io/apiserver/pkg/apis/audit/v1/types.go).
When an event is processed, it's
compared against the list of rules in order. The first matching rule sets the compared against the list of rules in order. The first matching rule sets the
"audit level" of the event. The known audit levels are: "audit level" of the event. The known audit levels are:
@@ -67,7 +66,7 @@ compared against the list of rules in order. The first matching rule sets the
- `RequestResponse` - log event metadata, request and response bodies. - `RequestResponse` - log event metadata, request and response bodies.
This does not apply for non-resource requests. This does not apply for non-resource requests.
You can pass a file with the policy to [kube-apiserver][kube-apiserver] You can pass a file with the policy to `kube-apiserver`
using the `--audit-policy-file` flag. If the flag is omitted, no events are logged. using the `--audit-policy-file` flag. If the flag is omitted, no events are logged.
Note that the `rules` field __must__ be provided in the audit policy file. Note that the `rules` field __must__ be provided in the audit policy file.
A policy with no (0) rules is treated as illegal. A policy with no (0) rules is treated as illegal.
@@ -86,12 +85,14 @@ rules:
- level: Metadata - level: Metadata
``` ```
The audit profile used by GCE should be used as reference by admins constructing their own audit profiles. You can check the [configure-helper.sh][configure-helper] script, which generates the audit policy file. You can see most of the audit policy file by looking directly at the script. The audit profile used by GCE should be used as reference by admins constructing their own audit profiles. You can check the
[configure-helper.sh](https://github.com/kubernetes/kubernetes/blob/{{< param "githubbranch" >}}/cluster/gce/gci/configure-helper.sh)
script, which generates the audit policy file. You can see most of the audit policy file by looking directly at the script.
## Audit backends ## Audit backends
Audit backends persist audit events to an external storage. Audit backends persist audit events to an external storage.
[Kube-apiserver][kube-apiserver] out of the box provides three backends: `Kube-apiserver` out of the box provides three backends:
- Log backend, which writes events to a disk - Log backend, which writes events to a disk
- Webhook backend, which sends events to an external API - Webhook backend, which sends events to an external API
@@ -99,7 +100,7 @@ Audit backends persist audit events to an external storage.
In all cases, audit events structure is defined by the API in the In all cases, audit events structure is defined by the API in the
`audit.k8s.io` API group. The current version of the API is `audit.k8s.io` API group. The current version of the API is
[`v1`][auditing-api]. [`v1`](https://github.com/kubernetes/kubernetes/blob/{{< param "githubbranch" >}}/staging/src/k8s.io/apiserver/pkg/apis/audit/v1/types.go).
{{< note >}} {{< note >}}
In case of patches, request body is a JSON array with patch operations, not a JSON object In case of patches, request body is a JSON array with patch operations, not a JSON object
@@ -125,7 +126,7 @@ request to `/apis/batch/v1/namespaces/some-namespace/jobs/some-job-name`.
### Log backend ### Log backend
Log backend writes audit events to a file in JSON format. You can configure Log backend writes audit events to a file in JSON format. You can configure
log audit backend using the following [kube-apiserver][kube-apiserver] flags: log audit backend using the following `kube-apiserver` flags:
- `--audit-log-path` specifies the log file path that log backend uses to write - `--audit-log-path` specifies the log file path that log backend uses to write
audit events. Not specifying this flag disables log backend. `-` means standard out audit events. Not specifying this flag disables log backend. `-` means standard out
@@ -136,11 +137,12 @@ log audit backend using the following [kube-apiserver][kube-apiserver] flags:
### Webhook backend ### Webhook backend
Webhook backend sends audit events to a remote API, which is assumed to be the Webhook backend sends audit events to a remote API, which is assumed to be the
same API as [kube-apiserver][kube-apiserver] exposes. You can configure webhook same API as `kube-apiserver` exposes. You can configure webhook
audit backend using the following kube-apiserver flags: audit backend using the following kube-apiserver flags:
- `--audit-webhook-config-file` specifies the path to a file with a webhook - `--audit-webhook-config-file` specifies the path to a file with a webhook
configuration. Webhook configuration is effectively a [kubeconfig][kubeconfig]. configuration. Webhook configuration is effectively a
[kubeconfig](/docs/tasks/access-application-cluster/configure-access-multiple-clusters).
- `--audit-webhook-initial-backoff` specifies the amount of time to wait after the first failed - `--audit-webhook-initial-backoff` specifies the amount of time to wait after the first failed
request before retrying. Subsequent requests are retried with exponential backoff. request before retrying. Subsequent requests are retried with exponential backoff.
@@ -327,23 +329,29 @@ Currently, this feature has performance implications for the apiserver in the fo
## Setup for multiple API servers ## Setup for multiple API servers
If you're extending the Kubernetes API with the [aggregation layer][kube-aggregator], you can also If you're extending the Kubernetes API with the [aggregation
set up audit logging for the aggregated apiserver. To do this, pass the configuration options in the layer](/docs/concepts/extend-kubernetes/api-extension/apiserver-aggregation/),
same format as described above to the aggregated apiserver and set up the log ingesting pipeline y ou can also set up audit logging for the aggregated apiserver. To do this,
to pick up audit logs. Different apiservers can have different audit configurations and different pass the configuration options in the same format as described above to the
audit policies. aggregated apiserver and set up the log ingesting pipeline to pick up audit
logs. Different apiservers can have different audit configurations and
different audit policies.
## Log Collector Examples ## Log Collector Examples
### Use fluentd to collect and distribute audit events from log file ### Use fluentd to collect and distribute audit events from log file
[Fluentd][fluentd] is an open source data collector for unified logging layer. [Fluentd](http://www.fluentd.org/) is an open source data collector for unified logging layer.
In this example, we will use fluentd to split audit events by different namespaces. In this example, we will use fluentd to split audit events by different namespaces.
{{< note >}}Fluent-plugin-forest and fluent-plugin-rewrite-tag-filter are plugins for fluentd. You can get details about plugin installation from [fluentd plugin-management][fluentd_plugin_management_doc]. {{< note >}}
The `fluent-plugin-forest` and `fluent-plugin-rewrite-tag-filter` are plugins for fluentd.
You can get details about plugin installation from
[fluentd plugin-management](https://docs.fluentd.org/v1.0/articles/plugin-management).
{{< /note >}} {{< /note >}}
1. Install [fluentd][fluentd_install_doc], fluent-plugin-forest and fluent-plugin-rewrite-tag-filter in the kube-apiserver node 1. Install [`fluentd`](https://docs.fluentd.org/v1.0/articles/quickstart#step-1:-installing-fluentd),
`fluent-plugin-forest` and `fluent-plugin-rewrite-tag-filter` in the kube-apiserver node
1. Create a config file for fluentd 1. Create a config file for fluentd
@@ -416,11 +424,12 @@ In this example, we will use fluentd to split audit events by different namespac
### Use logstash to collect and distribute audit events from webhook backend ### Use logstash to collect and distribute audit events from webhook backend
[Logstash][logstash] is an open source, server-side data processing tool. In this example, [Logstash](https://www.elastic.co/products/logstash)
is an open source, server-side data processing tool. In this example,
we will use logstash to collect audit events from webhook backend, and save events of we will use logstash to collect audit events from webhook backend, and save events of
different users into different files. different users into different files.
1. install [logstash][logstash_install_doc] 1. install [logstash](https://www.elastic.co/guide/en/logstash/current/installing-logstash.html)
1. create config file for logstash 1. create config file for logstash
@@ -491,19 +500,6 @@ Note that in addition to file output plugin, logstash has a variety of outputs t
let users route data where they want. For example, users can emit audit events to elasticsearch let users route data where they want. For example, users can emit audit events to elasticsearch
plugin which supports full-text search and analytics. plugin which supports full-text search and analytics.
[kube-apiserver]: /docs/reference/command-line-tools-reference/kube-apiserver/
[auditing-proposal]: https://github.com/kubernetes/community/blob/master/contributors/design-proposals/api-machinery/auditing.md
[auditing-api]: https://github.com/kubernetes/kubernetes/blob/{{< param "githubbranch" >}}/staging/src/k8s.io/apiserver/pkg/apis/audit/v1/types.go
[configure-helper]: https://github.com/kubernetes/kubernetes/blob/{{< param "githubbranch" >}}/cluster/gce/gci/configure-helper.sh
[kubeconfig]: /docs/tasks/access-application-cluster/configure-access-multiple-clusters/
[fluentd]: http://www.fluentd.org/
[fluentd_install_doc]: https://docs.fluentd.org/v1.0/articles/quickstart#step-1:-installing-fluentd
[fluentd_plugin_management_doc]: https://docs.fluentd.org/v1.0/articles/plugin-management
[logstash]: https://www.elastic.co/products/logstash
[logstash_install_doc]: https://www.elastic.co/guide/en/logstash/current/installing-logstash.html
[kube-aggregator]: /docs/concepts/api-extension/apiserver-aggregation
## {{% heading "whatsnext" %}} ## {{% heading "whatsnext" %}}
@@ -13,18 +13,15 @@ title: Auditing with Falco
[Falco](https://falco.org/) is an open source project for intrusion and abnormality detection for Cloud Native platforms. [Falco](https://falco.org/) is an open source project for intrusion and abnormality detection for Cloud Native platforms.
This section describes how to set up Falco, how to send audit events to the Kubernetes Audit endpoint exposed by Falco, and how Falco applies a set of rules to automatically detect suspicious behavior. This section describes how to set up Falco, how to send audit events to the Kubernetes Audit endpoint exposed by Falco, and how Falco applies a set of rules to automatically detect suspicious behavior.
<!-- body --> <!-- body -->
#### Install Falco #### Install Falco
Install Falco by using one of the following methods: Install Falco by using one of the following methods:
- [Standalone Falco][falco_installation] - [Standalone Falco](https://falco.org/docs/installation)
- [Kubernetes DaemonSet][falco_installation] - [Kubernetes DaemonSet](https://falco.org/docs/installation)
- [Falco Helm Chart][falco_helm_chart] - [Falco Helm Chart](https://github.com/falcosecurity/charts/tree/master/falco)
Once Falco is installed make sure it is configured to expose the Audit webhook. To do so, use the following configuration: Once Falco is installed make sure it is configured to expose the Audit webhook. To do so, use the following configuration:
@@ -41,7 +38,8 @@ This configuration is typically found in the `/etc/falco/falco.yaml` file. If Fa
#### Configure Kubernetes Audit #### Configure Kubernetes Audit
1. Create a [kubeconfig file](/docs/concepts/configuration/organize-cluster-access-kubeconfig/) for the [kube-apiserver][kube-apiserver] webhook audit backend. 1. Create a [kubeconfig file](/docs/concepts/configuration/organize-cluster-access-kubeconfig/)
for the [kube-apiserver](/docs/reference/generated/kube-apiserver/) webhook audit backend.
cat <<EOF > /etc/kubernetes/audit-webhook-kubeconfig cat <<EOF > /etc/kubernetes/audit-webhook-kubeconfig
apiVersion: v1 apiVersion: v1
@@ -60,7 +58,7 @@ This configuration is typically found in the `/etc/falco/falco.yaml` file. If Fa
users: [] users: []
EOF EOF
1. Start [kube-apiserver][kube-apiserver] with the following options: 1. Start `kube-apiserver` with the following options:
```shell ```shell
--audit-policy-file=/etc/kubernetes/audit-policy.yaml --audit-webhook-config-file=/etc/kubernetes/audit-webhook-kubeconfig --audit-policy-file=/etc/kubernetes/audit-policy.yaml --audit-webhook-config-file=/etc/kubernetes/audit-webhook-kubeconfig
@@ -68,7 +66,8 @@ This configuration is typically found in the `/etc/falco/falco.yaml` file. If Fa
#### Audit Rules #### Audit Rules
Rules devoted to Kubernetes Audit Events can be found in [k8s_audit_rules.yaml][falco_k8s_audit_rules]. If Audit Rules is installed as a native package or using the official Docker images, Falco copies the rules file to `/etc/falco/`, so they are available for use. Rules devoted to Kubernetes Audit Events can be found in [k8s_audit_rules.yaml](https://github.com/falcosecurity/falco/blob/master/rules/k8s_audit_rules.yaml).
If Audit Rules is installed as a native package or using the official Docker images, Falco copies the rules file to `/etc/falco/`, so they are available for use.
There are three classes of rules. There are three classes of rules.
@@ -99,23 +98,6 @@ A second class of rules tracks resources being created or destroyed, including:
The final class of rules simply displays any Audit Event received by Falco. This rule is disabled by default, as it can be quite noisy. The final class of rules simply displays any Audit Event received by Falco. This rule is disabled by default, as it can be quite noisy.
For further details, see [Kubernetes Audit Events][falco_ka_docs] in the Falco documentation. For further details, see [Kubernetes Audit Events](https://falco.org/docs/event-sources/kubernetes-audit) in the Falco documentation.
[kube-apiserver]: /docs/admin/kube-apiserver
[auditing-proposal]: https://github.com/kubernetes/community/blob/master/contributors/design-proposals/api-machinery/auditing.md
[auditing-api]: https://github.com/kubernetes/kubernetes/blob/{{< param "githubbranch" >}}/staging/src/k8s.io/apiserver/pkg/apis/audit/v1/types.go
[gce-audit-profile]: https://github.com/kubernetes/kubernetes/blob/{{< param "githubbranch" >}}/cluster/gce/gci/configure-helper.sh#L735
[kubeconfig]: /docs/tasks/access-application-cluster/configure-access-multiple-clusters/
[fluentd]: http://www.fluentd.org/
[fluentd_install_doc]: https://docs.fluentd.org/v1.0/articles/quickstart#step-1:-installing-fluentd
[fluentd_plugin_management_doc]: https://docs.fluentd.org/v1.0/articles/plugin-management
[logstash]: https://www.elastic.co/products/logstash
[logstash_install_doc]: https://www.elastic.co/guide/en/logstash/current/installing-logstash.html
[kube-aggregator]: /docs/concepts/api-extension/apiserver-aggregation
[falco_website]: https://www.falco.org
[falco_k8s_audit_rules]: https://github.com/falcosecurity/falco/blob/master/rules/k8s_audit_rules.yaml
[falco_ka_docs]: https://falco.org/docs/event-sources/kubernetes-audit
[falco_installation]: https://falco.org/docs/installation
[falco_helm_chart]: https://github.com/falcosecurity/charts/tree/master/falco