whentimeslows/website
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Release 1.9 (#5978)

Browse Source 
* Trivial change to open release branch

* Undo trivial change

* add service ipvs overview

* Add instructions on how to setup kubectl

* Document conntrack dependency for kube-proxy

* Add an a

This is kind of jarring / missing an article. I'm guessing it should either be ' to a rack of bare metal servers.' or '...to racks of bare metal servers.'.

* adding example responses for common issues

 - support request
 - code bug report

* Trivial change to open release branch

* Undo trivial change

* Signed-off-by: Ziqi Zhao <zhaoziqi@qiniu.com> (#5366)

Fix the not-working test case yaml for /doc/concepts/storage/volumes.md

* kubectl-overview

* temp fix for broken pod and deployment links

* Update Table of Solutions for Juju

* Revise certificates documentation (#5965)

* Update review-issues.md

Some edits for clarity and condensed language.

* Update init-containers.md

Fix leading spaces in commands.

* Update kubectl-overview.md

Fix format.

* Update clc.md

Fix format.

* Update openstack-heat.md

The url no need. just  highlight.

* Typo

I believe this should be "users" not "uses"

* making explicit hostname uniq requirement

* Update scheduling-hugepages.md

* Update update-daemon-set.md

* fix redirection of PersistentVolume

* Update hpa.md

* update kubectl instruction

* Use the format of kubeadm init

* fix spelling error

guarnatees  to guarantees

* add matchLabels description (#6020)

* search and replace for k8s.github.io to website (#6019)

* fix scale command of object-management (#6011)

* Update replicaset.md (#6009)

* Update secret.md (#6008)

* specify password for mysql image (#5990)

* specify password for mysql image

* specify password for mysql image

* link error for run-stateless-application-deployment.md (#5985)

* link error for run-stateless-application-deployment.md

* link error for run-stateless-application-deployment.md

* Add performance implications of inter-pod affinity/anti-affinity (#5979)

* 404 monthly maintenance - October 2017 (#5977)

* Updated redirects

* More redirects

* Add conjure-up to Turnkey Cloud Solutions list (#5973)

* Add conjure-up to Turnkey Cloud Solutions list

* Changed wording slightly

* change the StatefulSet to ReplicaSet in reference (#5968)

* Clarification of failureThreshold of probes (#5963)

* Mention usage of block storage version param (#5925)

Mention usage of block storage version (bs-version) parameter to
workaround attachment issues using older K8S versions on an OpenStack
cloud with path-based endpoints.

Resolves: https://github.com/kubernetes/kubernetes.github.io/issues/5924

* Update sysctl-cluster.md (#5894)

Include guide on enabling unsafe sysctls in minikube

* Avoid Latin phrases & format note (#5889)

* Avoid Latin phrases & format note

according the Documentation Style Guide

* Update scratch.md

* Update scratch.md

* resolves jekyll rendering error (#5976)

- chinese isn't understood for keys in YAML frontmatter in jekyll, so
   replaced it with the english equivalent that doesn't throw the
following error on rendering:

Error reading file src/kubernetes.github.io/cn/docs/concepts/cluster-administration/device-plugins.md: (<unknown>): could not find expected ':' while scanning a simple key at line 4 column 1

* Change VM to pod. (#6022)

* Add link to custom metrics. (#6023)

* Rephrase core group. (#6024)

* Added explanation on context to when joining (#6018)

* Update create-cluster-kubeadm.md (#5761)

Update Canal version in pod network apply commands

* Fixes issue #5620 (#5869)

* Fixes issue #5620

Signed-off-by: Brad Topol <btopol@us.ibm.com>

* Restructured so that review process is for both current and upcoming
releases.  Added content describing the use of tech reviewers.

* Removed incorrect Kubernetes reviewer link.

* Fixed tech reviewer URL to now use website

* Update pod-priority-preemption.md

fix-wrong-link-to-pod-preemption

* pod-security-policy.md: add links to the page about admission plugins.

* Adding all files for BlaBlaCar case study (#5857)

* Adding all files for BlaBlaCar case study

* Update blablacar.html

* Fix changed URL for google containers

* Add /docs/reference/auto-generated directory

* correct the downwardapi redirect

* Remove links using "here"

* Rename to /docs/reference/generated directory

* add Concept template

* Change title to just Ingress

* Link mistake (#6038)

* link mistake

* link mistake

* skip title check for skip_title_check.txt

* skip title check for skip_title_check.txt

* remove doesn't exist link.

* Fix podpreset task (#5705)

* Add a simple pod manifest to pod overview (#5986)

* Split PodPreset concept out from task doc (#5984)

* Add selector spec description (#5789)

* Add selector spec description

* Fix selector field explanation

* Put orphaned topics in TOC. (#6051)

* static-pod example bad format in the final page (#6050)

* static-pod example bad format in the final page

* static-pod example bad format in the final page

* static-pod example bad format in the final page

* static-pod example bad format in the final page

* static-pod example bad format in the final page

* Fix `backoffLimit` field misplacement (#6042)

It should be placed in JobSpec according to:
https://github.com/kubernetes/kubernetes/blob/master/api/swagger-spec/batch_v1.json#L1488-L1514

* Update addons.md (#6061)

* add info about VMware NSX-T CNI plugin (#5987)

* add info about VMware NSX-T CNI plugin

Hello,

I'm VMware Networking and Security Architect and would like to include short information about our CNI plugin implementation similar to what other vendors did

Best regards

Emil Gagala

* Update networking.md

* Update networking.md

* Update networking.md

* Update: Using universal zsh configuration (#5669)

* Update install-kubectl.md

Zsh is not only oh-my-zsh, so I added universal configuration for zsh that also can be used in prezto.

* fix merge error after rebase

* Operating etcd cluster for Kubernetes bad format in the final page (#6056)

* Operating etcd cluster for Kubernetes bad format in the final page

* Update configure-upgrade-etcd.md

* Update configure-upgrade-etcd.md

* Usage note and warning tags. (#6053)

* Usage note and warning tags.

* Update configure-upgrade-etcd.md

* Update configure-upgrade-etcd.md

* Document jekyll includes snippets

* Add jekyll includes to docs home toc

- Remove extra kubernetes home in toc

* document docker cgroupdriver req (#5937)

* Update test blacklists (#6063)

* Update toc check blacklist

* Update title check blacklist

* wip

* wip

* Fix typo

* Document unconfined apparmor profile

* Revert "Document the unconfined profile for AppArmor" (#6268)

* CRD Validation: remove alpha warning, change enable instructions to (#6066)

disable

* Documented service annotation for AWS ELB SSL policy

* kubeadm: add a note about the new `--print-join-command` flag.

This is a new flag for the `kubeadm token create` command.

* Add a note to PDB page

* Improve Kubeadm reference doc (#6103)

* automatically-generated kubeadm reference doc

* user-mantained kubeadm reference doc

* Documentation for CSIPersistentVolume

* change replicaset documentation to use apps/v1 APIs

* Update service.md

ipvs alpha version -> beta version

* Updated Deployment concept docs (#6494)

* Updated Deployment concept docs

* Addressed comments

* Documentation for volume scheduling alpha feature

* Update admission control docs for webhooks

* Improve DNS documentation (#6479)

* update ds for 1.9

* Update service.md

* Update service.md

* Revert "begin updating webhook documentation" (#6575)

* Update version numbers to include 1.9 (#6518)

* Update site versions for 1.9

* Removed 1.4 docs

* Update _config.yml

* Update _config.yml

* updates for raw block devices

* rbac: docs for aggregated cluster roles (#6474)

* Added IPv6 information for Kubelet arguments (#6498)

* Added IPv6 info to kube-proxy arguments

* Added IPv6 information for argument for kubelet

* Update PVC resizing documentation (#6487)

* Updates for Windows Server version 1709 with K8s v1.8 (#6180)

* Updated for WSv1709 and K8s v1.8

* Updated picture and CNI config

* Fixed formatting on CNI Config

* Updated docs to reference Microsoft/SDN GitHub docs

* fix typo

* Workaround for Jekyllr frontmatter

* Added section on features and limitations, with example yaml files.

* Update index.md

* Added kubeadm section, few other small fixes

* Few minor grammar fixes

* Update access-cluster.md with a comment that for IPv6
the user should use [::1] for the localhost

* Addressed a number of issues brought up against the base PR

* Fixed windows-host-setup link

* Rewrite PodSecurityPolicy guide

* Update index.md

Signed-off-by: Alin Balutoiu <abalutoiu@cloudbasesolutions.com>
Signed-off-by: Alin Gabriel Serdean <aserdean@ovn.org>

* Spelling correction and sentence capitalization.

- Corrected the spelling error for storing, was put in as 'stoing'.
- Capitalized list items.
- Added '.' at end of sentences in the list items.

* Update index.md

* Update index.md

* Addressed comments and rebased

* Fixed formatting

* Fixed formatting

* Updated header link

* Updated hyperlinks

* Updated warning

* formatting

* formatting

* formatting

* Revert "Update access-cluster.md with a comment that for IPv6"

This reverts commit 31e4dbdc25.

* Revert "fix typo"

This reverts commit c05678752d.

* Revert "Workaround for Jekyllr frontmatter"

This reverts commit b84ac59624.

* Fixed grammatical issues and reverted non-related commits

* Revert "Rewrite PodSecurityPolicy guide"

This reverts commit 5d39cfeae4.

* Revert "Spelling correction and sentence capitalization."

This reverts commit 47eed4346e.

* Fixed auto-numbering

* Minor formatting updates

* CoreDNS feature documentation (#6463)

* Initial placeholder PR for CoreDNS feature documentation

* Remove from admin, add content

* Fix missing endcapture

* Add to tasks.yml

* Review feedback

* Postpone Deletion of a Persistent Volume Claim in case It Is Used by a Pod (#6415)

* Postpone Deletion of a Persistent Volume Claim in case It Is Used by a Pod

A new feature PVC Protection was added into K8s 1.9 that's why this documentation change is needed.

* Added tag at the top of each new area.

* Fix typo

* Fix: switched on in (all kubelets) -> (all K8s components).

* Added link to admission controller

* Moved PVC Protection configuration into Before you begin section.

* Added steps how to verify PVC Protection feature.

* Fixes for admission controller plugin description and for PVC Protection description in PVC lifecycle.

* Testing official rendering of enumerations (1., 2., 3., etc.)

* Re-write to address comments from review.

* Fixed definition when a PVC is in active use by a pod.

* Change auditing docs page for 1.9 release (#6427)

* Change auditing docs page for 1.9 release

Signed-off-by: Mik Vyatskov <vmik@google.com>

* Address review comments

Signed-off-by: Mik Vyatskov <vmik@google.com>

* Address review comments

Signed-off-by: Mik Vyatskov <vmik@google.com>

* Address review comments

Signed-off-by: Mik Vyatskov <vmik@google.com>

* Fix broken link

Signed-off-by: Mik Vyatskov <vmik@google.com>

* short circuit deny docs (#6536)

* line wrap

* short circuit deny

* address comments

* Add kubeadm 1.9 upgrade docs (#6485)

* kubeadm: Improve kubeadm documentation for v1.9 (#6645)

* Update admission control docs for webhooks (re-send #6368) (#6650)

* Update admission control docs for webhooks

* update in response to comments

* Revamp rkt and add CRI-O as alternative runtime (#6371)

Signed-off-by: Lorenzo Fontana <lo@linux.com>

* Documented NLB for Kubernetes 1.9 (#6260)

* Added IPV6 information to setup cluster using kubeadm (#6465)

* Added IPV6 information to setup cluster using kubeadm

* Updated kubeadm.md & create-cluster-kubeadm.md with IPv6 related information

* Added IPv6 options for kubeadm --init  & automated address binding for kube-proxy based on version of IP configured for API server)

* Changes to kubeadm.md as per comments

* Modified kubeadm.md and create-cluster-kubeadm.md

* Implemented changes requested by zacharysarah

* Removed autogenerated kubeadm.md changes

* StatefulSet 1.9 updates. (#6550)

* updates sts concept and tutorials to use 1.9 apps/v1

* Update statefulset.md

* clarify pod name label

* Garbage collection updates for 1.9 (#6555)

* 1.9 gc policy update

* carify deletion

* Couple nits for dnsConfig doc (#6652)

* Add doc for AllowedFlexVolume (#6563)

* Update OpenStack Cloud Provider API support for v1.9 (#6638)

* Flex volume is GA. Remove alpha notation. (#6666)

* Update generated ref docs for Kubernetes and Federation components. (#6658)

* Update generated ref docs for Kubernetes and Federation components.

* Rename kubectl-options to kubectl.

* Add title to kubectl.

* Fix double synopsis.

* Update Federation API ref docs for 1.9. (#6636)

* Update federation API ref docs.

* Move and redirect.

* Move generated Federation docs to the generated directory.

* Fix titles.

* Type

* Fix titles

* Update auto-generated Kubernetes APi ref docs. (#6646)

* Update kubectl commands for 1.9 (#6635)

* add ExtendedResourceToleration admission controller (#6618)

* Update API reference paths for v1.9 (#6681)
This commit is contained in:
Zach Corleissen
2017-12-15 17:36:13 -06:00
committed by GitHub
parent 0e628e7895
commit 08af95d42e
893 changed files with 131178 additions and 105058 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/tasks/debug-application-cluster/audit.md
+100 -269
View File
@@ -9,7 +9,9 @@ title: Auditing
* TOC
{:toc}
Kubernetes Audit provides a security-relevant chronological set of records documenting
{% include feature-state-beta.md %}
Kubernetes auditing provides a security-relevant chronological set of records documenting
the sequence of activities that have affected system by individual users, administrators
or other components of the system. It allows cluster administrator to
answer the following questions:
@@ -22,55 +24,18 @@ answer the following questions:
- from where was it initiated?
- to where was it going?
## Legacy Audit
[Kube-apiserver][kube-apiserver] performs auditing. Each request on each stage
of its execution generates an event, which is then pre-processed according to
a certain policy and written to a backend. You can find more details about the
pipeline in the [design proposal][auditing-proposal].
Kubernetes audit is part of [Kube-apiserver][kube-apiserver] logging all requests
processed by the server. Each audit log entry contains two lines:
1. The request line containing a unique ID to match the response and request metadata, such as the source IP, requesting user, impersonation information, resource being requested, etc.
2. The response line containing a unique ID matching the request line and the response code.
Example output for `admin` user listing pods in the `default` namespace:
```
2017-03-21T03:57:09.106841886-04:00 AUDIT: id="c939d2a7-1c37-4ef1-b2f7-4ba9b1e43b53" ip="127.0.0.1" method="GET" user="admin" groups="\"system:masters\",\"system:authenticated\"" as="<self>" asgroups="<lookup>" namespace="default" uri="/api/v1/namespaces/default/pods"
2017-03-21T03:57:09.108403639-04:00 AUDIT: id="c939d2a7-1c37-4ef1-b2f7-4ba9b1e43b53" response="200"
```
Note that Kubernetes 1.8 has switched to use the advanced structured audit log by default.
To fallback to this legacy audit, disable the advanced auditing feature
using the `AdvancedAuditing` feature gate on the [kube-apiserver][kube-apiserver]:
```
--feature-gates=AdvancedAuditing=false
```
### Configuration
[Kube-apiserver][kube-apiserver] provides the following options which are responsible
for configuring where and how audit logs are handled:
- `audit-log-path` - enables the audit log pointing to a file where the requests are being logged to, '-' means standard out.
- `audit-log-maxage` - specifies maximum number of days to retain old audit log files based on the timestamp encoded in their filename.
- `audit-log-maxbackup` - specifies maximum number of old audit log files to retain.
- `audit-log-maxsize` - specifies maximum size in megabytes of the audit log file before it gets rotated. Defaults to 100MB.
If an audit log file already exists, Kubernetes appends new audit logs to that file.
Otherwise, Kubernetes creates an audit log file at the location you specified in
`audit-log-path`. If the audit log file exceeds the size you specify in `audit-log-maxsize`,
Kubernetes will rename the current log file by appending the current timestamp on
the file name (before the file extension) and create a new audit log file.
Kubernetes may delete old log files when creating a new log file; you can configure
how many files are retained and how old they can be by specifying the `audit-log-maxbackup`
and `audit-log-maxage` options.
## Advanced audit
Kubernetes 1.7 expands auditing with experimental functionality such as event
filtering and a webhook for integration with external systems. Kubernetes 1.8
upgrades the advanced audit feature to beta, and some backward incompatible changes
have been committed.
## Audit Policy
Audit policy defines rules about what events should be recorded and what data
they should include. When an event is processed, it's compared against the list
of rules in order. The first matching rule sets the [audit level][auditing-level]
of the event. The audit policy object structure is defined in the
[`audit.k8s.io` API group][auditing-api].
`AdvancedAuditing` is customizable in two ways. Policy, which determines what's recorded,
and backends, which persist records. Backend implementations include logs files and
@@ -117,83 +82,7 @@ a valid `apiVersion` and `kind` value will be treated as illegal.
Some example audit policy files:
```yaml
apiVersion: audit.k8s.io/v1beta1 #this is required in Kubernetes 1.8
kind: Policy
rules:
# Don't log watch requests by the "system:kube-proxy" on endpoints or services
- level: None
users: ["system:kube-proxy"]
verbs: ["watch"]
resources:
- group: "" # core API group
resources: ["endpoints", "services"]
# Don't log authenticated requests to certain non-resource URL paths.
- level: None
userGroups: ["system:authenticated"]
nonResourceURLs:
- "/api*" # Wildcard matching.
- "/version"
# Log the request body of configmap changes in kube-system.
- level: Request
resources:
- group: "" # core API group
resources: ["configmaps"]
# This rule only applies to resources in the "kube-system" namespace.
# The empty string "" can be used to select non-namespaced resources.
namespaces: ["kube-system"]
# Log configmap and secret changes in all other namespaces at the Metadata level.
- level: Metadata
resources:
- group: "" # core API group
resources: ["secrets", "configmaps"]
# Log all other resources in core and extensions at the Request level.
- level: Request
resources:
- group: "" # core API group
- group: "extensions" # Version of group should NOT be included.
# A catch-all rule to log all other requests at the Metadata level.
- level: Metadata
```
The next audit policy file shows new features introduced in Kubernetes 1.8:
```yaml
apiVersion: audit.k8s.io/v1beta1
kind: Policy
rules:
# Log pod changes at Request level
- level: Request
resources:
- group: ""
# Resource "pods" no longer matches requests to any subresource of pods,
# This behavior is consistent with the RBAC policy.
resources: ["pods"]
# Log "pods/log", "pods/status" at Metadata level
- level: Metadata
resources:
- group: ""
resources: ["pods/log", "pods/status"]
# Don't log requests to a configmap called "controller-leader"
- level: None
resources:
- group: ""
resources: ["configmaps"]
resourceNames: ["controller-leader"]
# A catch-all rule to log all other requests at the Metadata level.
# For this rule we use "omitStages" to omit events at "ReqeustReceived" stage.
# Events in this stage will not be sent to backend.
- level: Metadata
omitStages:
- "RequestReceived"
```
{% include code.html language="yaml" file="audit-policy.yaml" ghlink="/docs/tasks/debug-application-cluster/audit-policy.yaml" %}
You can use a minimal audit policy file to log all requests at the `Metadata` level:
@@ -208,153 +97,51 @@ rules:
The [audit profile used by GCE][gce-audit-profile] should be used as reference by
admins constructing their own audit profiles.
### Audit backends
## Audit backends
Audit backends implement strategies for emitting events. The [kube-apiserver][kube-apiserver]
provides a logging and webhook backend.
Audit backends implement exporting audit events to an external storage.
[Kube-apiserver][kube-apiserver] out of the box provides two backends:
Each request to the API server can generate multiple events, one when the request is received,
another when the response is sent, and additional events for long running requests (such as
watches). The ID of events will be the same if they were generated from the same request.
- Log backend, which writes events to a disk
- Webhook backend, which sends events to an external API
The event format is defined by the `audit.k8s.io` API group. The `v1alpha1` format of this
API can be found [here][audit-api] with more details about the exact fields captured.
In both cases, audit events structure is defined by the API in the
`audit.k8s.io` API group. The current version of the API is
[`v1beta1`][auditing-api].
#### Log backend
### Log backend
The behavior of the `--audit-log-path` flag changes when enabling the `AdvancedAuditing`
feature flag. All generated events defined by `--audit-policy-file` are recorded in structured
json format:
Log backend writes audit events to a file in JSON format. You can configure
log audit backend using the following [kube-apiserver][kube-apiserver] flags:
```
{"kind":"Event","apiVersion":"audit.k8s.io/v1beta1","metadata":{"creationTimestamp":null},"level":"Metadata","timestamp":"2017-09-05T10:04:55Z","auditID":"77e58433-d345-40ac-b2d8-9866bd355cea","stage":"RequestReceived","requestURI":"/apis/rbac.authorization.k8s.io/v1/namespaces/default/roles","verb":"list","user":{"username":"kubecfg","groups":["system:masters","system:authenticated"]},"sourceIPs":["172.16.116.128"],"objectRef":{"resource":"roles","namespace":"default","apiGroup":"rbac.authorization.k8s.io","apiVersion":"v1"}}
{"kind":"Event","apiVersion":"audit.k8s.io/v1beta1","metadata":{"creationTimestamp":null},"level":"Metadata","timestamp":"2017-09-05T10:04:55Z","auditID":"77e58433-d345-40ac-b2d8-9866bd355cea","stage":"ResponseComplete","requestURI":"/apis/rbac.authorization.k8s.io/v1/namespaces/default/roles","verb":"list","user":{"username":"kubecfg","groups":["system:masters","system:authenticated"]},"sourceIPs":["172.16.116.128"],"objectRef":{"resource":"roles","namespace":"default","apiGroup":"rbac.authorization.k8s.io","apiVersion":"v1"},"responseStatus":{"metadata":{},"code":200}}
```
- `--audit-log-path` specifies the log file path that log backend uses to write
audit events. Not specifying this flag disables log backend. `-` means standard out
- `--audit-log-maxage` defined the maximum number of days to retain old audit log files
- `--audit-log-maxbackup` defines the maximum number of audit log files to retain
- `--audit-log-maxsize` defines the maximum size in megabytes of the audit log file before it gets rotated
In alpha version, objectRef.apiVersion holds both the api group and version.
In beta version these were break out into objectRef.apiGroup and objectRef.apiVersion.
### Webhook backend
Starting from Kubernetes 1.8, structured json format is used for log backend by default.
Use the following option to switch log to legacy format:
Webhook backend sends audit events to a remote API, which is assumed to be the
same API as [kube-apiserver][kube-apiserver] exposes. You can configure webhook
audit backend using the following kube-apiserver flags:
```
--audit-log-format=legacy
```
With legacy format, events are formatted as follows:
```
2017-09-05T06:08:19.885328047-04:00 AUDIT: id="c28a95ad-f9dd-47e1-a617-b6dc152db95f" stage="RequestReceived" ip="172.16.116.128" method="list" user="kubecfg" groups="\"system:masters\",\"system:authenticated\"" as="<self>" asgroups="<lookup>" namespace="default" uri="/apis/rbac.authorization.k8s.io/v1/namespaces/default/roles" response="<deferred>"
2017-09-05T06:08:19.885328047-04:00 AUDIT: id="c28a95ad-f9dd-47e1-a617-b6dc152db95f" stage="ResponseComplete" ip="172.16.116.128" method="list" user="kubecfg" groups="\"system:masters\",\"system:authenticated\"" as="<self>" asgroups="<lookup>" namespace="default" uri="/apis/rbac.authorization.k8s.io/v1/namespaces/default/roles" response="200"
```
Logged events omit the request and response bodies. The `Request` and
`RequestResponse` levels are equivalent to `Metadata` for legacy format. This legacy format
of advanced audit is different from the [Legacy Audit](# Legacy Audit) discussed above, such
as changes to the method values and the introduction of a "stage" for each event.
#### Webhook backend
The audit webhook backend can be used to have [kube-apiserver][kube-apiserver]
send audit events to a remote service. The webhook requires the `AdvancedAuditing`
feature flag and is configured using the following command line flags:
```
--audit-webhook-config-file=/etc/kubernetes/audit-webhook-kubeconfig
--audit-webhook-mode=batch
```
`audit-webhook-mode` controls buffering strategies used by the webhook. Known modes are:
- `batch` - buffer events and asynchronously send the set of events to the external service.
- `blocking` - block API server responses on sending each event to the external service.
- `--audit-webhook-config-file` specifies the path to a file with a webhook
configuration. Webhook configuration is effectively a [kubeconfig][kubeconfig].
- `--audit-webhook-mode` define the buffering strategy, one of the following:
- `batch` - buffer events and asynchronously send the set of events to the external service
This is the default
- `blocking` - block API server responses on sending each event to the external service
The webhook config file uses the kubeconfig format to specify the remote address of
the service and credentials used to connect to it.
```
# clusters refers to the remote service.
clusters:
- name: name-of-remote-audit-service
cluster:
certificate-authority: /path/to/ca.pem # CA for verifying the remote service.
server: https://audit.example.com/audit # URL of remote service to query. Must use 'https'.
## Log Collector Examples
# users refers to the API server's webhook configuration.
users:
- name: name-of-api-server
user:
client-certificate: /path/to/cert.pem # cert for the webhook plugin to use
client-key: /path/to/key.pem # key matching the cert
# kubeconfig files require a context. Provide one for the API server.
current-context: webhook
contexts:
- context:
cluster: name-of-remote-audit-service
user: name-of-api-sever
name: webhook
```
Events are POSTed as a JSON serialized `EventList`. An example payload:
```json
{
"apiVersion": "audit.k8s.io/v1beta1",
"items": [
{
"auditID": "24f30caf-d7d4-45d5-b7bd-e7af300d7886",
"level": "Metadata",
"metadata": {
"creationTimestamp": null
},
"objectRef": {
"apiGroup": "rbac.authorization.k8s.io",
"apiVersion": "v1",
"name": "jane",
"namespace": "default",
"resource": "roles"
},
"requestURI": "/apis/rbac.authorization.k8s.io/v1/namespaces/default/roles/jane",
"responseStatus": {
"code": 200,
"metadata": {}
},
"sourceIPs": [
"172.16.116.128"
],
"stage": "ResponseComplete",
"timestamp": "2017-09-05T10:20:24Z",
"user": {
"groups": [
"system:masters",
"system:authenticated"
],
"username": "kubecfg"
},
"verb": "get"
}
],
"kind": "EventList",
"metadata": {}
}
```
### Audit-Id
Audit-Id is a unique ID for each http request to kube-apiserver. The ID of events will be the
same if they were generated from the same request. Starting from Kubernetes 1.8, if an audit
event is generated for the request, kube-apiserver will respond with an Audit-Id in the HTTP header.
Note that for some special requests like `kubectl exec`, `kubectl attach`, kube-apiserver works
like a proxy, no Audit-Id will be returned even if audit events are recorded.
### Log Collector Examples
#### Use fluentd to collect and distribute audit events from log file
### Use fluentd to collect and distribute audit events from log file
[Fluentd][fluentd] is an open source data collector for unified logging layer.
In this example, we will use fluentd to split audit events by different namespaces.
Note that this example requries json format output support in Kubernetes 1.8.
1. install [fluentd, fluent-plugin-forest and fluent-plugin-rewrite-tag-filter][fluentd_install_doc] in the kube-apiserver node
1. create a config file for fluentd
@@ -372,7 +159,7 @@ Note that this example requries json format output support in Kubernetes 1.8.
time_format %Y-%m-%dT%H:%M:%S.%N%z
tag audit
</source>
<filter audit>
#https://github.com/fluent/fluent-plugin-rewrite-tag-filter/issues/13
type record_transformer
@@ -381,18 +168,18 @@ Note that this example requries json format output support in Kubernetes 1.8.
namespace ${record["objectRef"].nil? ? "none":(record["objectRef"]["namespace"].nil? ? "none":record["objectRef"]["namespace"])}
</record>
</filter>
<match audit>
# route audit according to namespace element in context
@type rewrite_tag_filter
rewriterule1 namespace ^(.+) ${tag}.$1
</match>
<filter audit.**>
@type record_transformer
remove_keys namespace
</filter>
<match audit.**>
@type forest
subtype file
@@ -406,22 +193,22 @@ Note that this example requries json format output support in Kubernetes 1.8.
</template>
</match>
```
1. start fluentd
```shell
$ fluentd -c /etc/fluentd/config -vv
```
1. start kube-apiserver with the following options:
```shell
--audit-policy-file=/etc/kubernetes/audit-policy.yaml --audit-log-path=/var/log/kube-audit --audit-log-format=json
```
1. check audits for different namespaces in /var/log/audit-*.log
#### Use logstash to collect and distribute audit events from webhook backend
### Use logstash to collect and distribute audit events from webhook backend
[Logstash][logstash] is an open source, server-side data processing tool. In this example,
we will use logstash to collect audit events from webhook backend, and save events of
@@ -458,13 +245,13 @@ different users into different files.
}
}
```
1. start logstash
```shell
$ bin/logstash -f /etc/logstash/config --path.settings /etc/logstash/
```
1. create a [kubeconfig file](/docs/tasks/access-application-cluster/authenticate-across-clusters-kubeconfig/) for kube-apiserver webhook audit backend
```shell
@@ -485,22 +272,66 @@ different users into different files.
users: []
EOF
```
1. start kube-apiserver with the following options:
```shell
--audit-policy-file=/etc/kubernetes/audit-policy.yaml --audit-webhook-config-file=/etc/kubernetes/audit-webhook-kubeconfig
```
1. check audits in logstash node's directories /var/log/kube-audit-*/audit
Note that in addition to file output plugin, logstash has a variety of outputs that
let users route data where they want. For example, users can emit audit events to elasticsearch
plugin which supports full-text search and analytics.
[audit-api]: https://github.com/kubernetes/kubernetes/blob/v1.8.0-beta.1/staging/src/k8s.io/apiserver/pkg/apis/audit/v1beta1/types.go
## Legacy Audit
__Note:__ Legacy Audit is deprecated and is disabled by default since Kubernetes 1.8.
To fallback to this legacy audit, disable the advanced auditing feature
using the `AdvancedAuditing` feature gate in [kube-apiserver][kube-apiserver]:
```
--feature-gates=AdvancedAuditing=false
```
In legacy format, each audit log entry contains two lines:
1. The request line containing a unique ID to match the response and request metadata, such as the source IP, requesting user, impersonation information, resource being requested, etc.
2. The response line containing a unique ID matching the request line and the response code.
Example output for `admin` user listing pods in the `default` namespace:
```
2017-03-21T03:57:09.106841886-04:00 AUDIT: id="c939d2a7-1c37-4ef1-b2f7-4ba9b1e43b53" ip="127.0.0.1" method="GET" user="admin" groups="\"system:masters\",\"system:authenticated\"" as="<self>" asgroups="<lookup>" namespace="default" uri="/api/v1/namespaces/default/pods"
2017-03-21T03:57:09.108403639-04:00 AUDIT: id="c939d2a7-1c37-4ef1-b2f7-4ba9b1e43b53" response="200"
```
### Configuration
[Kube-apiserver][kube-apiserver] provides the following options which are responsible
for configuring where and how audit logs are handled:
- `audit-log-path` - enables the audit log pointing to a file where the requests are being logged to, '-' means standard out.
- `audit-log-maxage` - specifies maximum number of days to retain old audit log files based on the timestamp encoded in their filename.
- `audit-log-maxbackup` - specifies maximum number of old audit log files to retain.
- `audit-log-maxsize` - specifies maximum size in megabytes of the audit log file before it gets rotated. Defaults to 100MB.
If an audit log file already exists, Kubernetes appends new audit logs to that file.
Otherwise, Kubernetes creates an audit log file at the location you specified in
`audit-log-path`. If the audit log file exceeds the size you specify in `audit-log-maxsize`,
Kubernetes will rename the current log file by appending the current timestamp on
the file name (before the file extension) and create a new audit log file.
Kubernetes may delete old log files when creating a new log file; you can configure
how many files are retained and how old they can be by specifying the `audit-log-maxbackup`
and `audit-log-maxage` options.
[kube-apiserver]: /docs/admin/kube-apiserver
[gce-audit-profile]: https://github.com/kubernetes/kubernetes/blob/v1.8.0-beta.0/cluster/gce/gci/configure-helper.sh#L532
[auditing-proposal]: https://github.com/kubernetes/community/blob/master/contributors/design-proposals/api-machinery/auditing.md
[auditing-level]: https://github.com/kubernetes/community/blob/master/contributors/design-proposals/api-machinery/auditing.md#levels
[auditing-api]: https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/staging/src/k8s.io/apiserver/pkg/apis/audit/v1beta1/types.go
[gce-audit-profile]: https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/cluster/gce/gci/configure-helper.sh#L532
[kubeconfig]: https://kubernetes.io/docs/tasks/access-application-cluster/configure-access-multiple-clusters/
[fluentd]: http://www.fluentd.org/
[fluentd_install_doc]: http://docs.fluentd.org/v0.12/articles/quickstart#step1-installing-fluentd
[logstash]: https://www.elastic.co/products/logstash