Merge branch 'master' into release-1.8
This commit is contained in:
steveperry-53
@@ -316,7 +316,7 @@ Highly Available database statefulset has one master and three replicas, one may
|
|||||||
|:--------------------:|:-------------------:|:------------------:|:------------------:|
|
|:--------------------:|:-------------------:|:------------------:|:------------------:|
|
||||||
| *DB-MASTER* | *DB-REPLICA-1* | *DB-REPLICA-2* | *DB-REPLICA-3* |
|
| *DB-MASTER* | *DB-REPLICA-1* | *DB-REPLICA-2* | *DB-REPLICA-3* |
|
||||||
|
|
||||||
[Here](https://kubernetes.io/docs/tutorials/stateful-application/zookeeper/#tolerating-node-failure) is an example of zookeper statefulset configured with anti-affinity for high availability.
|
[Here](https://kubernetes.io/docs/tutorials/stateful-application/zookeeper/#tolerating-node-failure) is an example of Zookeeper statefulset configured with anti-affinity for high availability.
|
||||||
|
|
||||||
For more information on inter-pod affinity/anti-affinity, see the design doc
|
For more information on inter-pod affinity/anti-affinity, see the design doc
|
||||||
[here](https://git.k8s.io/community/contributors/design-proposals/scheduling/podaffinity.md).
|
[here](https://git.k8s.io/community/contributors/design-proposals/scheduling/podaffinity.md).
|
||||||
|
|||||||
@@ -113,7 +113,7 @@ Some new fields are supported in beta version, like `resourceNames` and `omitSta
|
|||||||
|
|
||||||
In Kubernetes 1.8 `kind` and `apiVersion` along with `rules` __must__ be provided in
|
In Kubernetes 1.8 `kind` and `apiVersion` along with `rules` __must__ be provided in
|
||||||
the audit policy file. A policy file with 0 rules, or a policy file that doesn't provide
|
the audit policy file. A policy file with 0 rules, or a policy file that doesn't provide
|
||||||
a valid `apiVersion` and `kind` value will be treated as illgal.
|
a valid `apiVersion` and `kind` value will be treated as illegal.
|
||||||
|
|
||||||
Some example audit policy files:
|
Some example audit policy files:
|
||||||
|
|
||||||
@@ -359,57 +359,66 @@ Note that this example requries json format output support in Kubernetes 1.8.
|
|||||||
1. install [fluentd, fluent-plugin-forest and fluent-plugin-rewrite-tag-filter][fluentd_install_doc] in the kube-apiserver node
|
1. install [fluentd, fluent-plugin-forest and fluent-plugin-rewrite-tag-filter][fluentd_install_doc] in the kube-apiserver node
|
||||||
1. create a config file for fluentd
|
1. create a config file for fluentd
|
||||||
|
|
||||||
$ cat <<EOF > /etc/fluentd/config
|
```shell
|
||||||
# fluentd conf runs in the same host with kube-apiserver
|
$ cat <<EOF > /etc/fluentd/config
|
||||||
<source>
|
# fluentd conf runs in the same host with kube-apiserver
|
||||||
@type tail
|
<source>
|
||||||
# audit log path of kube-apiserver
|
@type tail
|
||||||
path /var/log/audit
|
# audit log path of kube-apiserver
|
||||||
pos_file /var/log/audit.pos
|
path /var/log/audit
|
||||||
format json
|
pos_file /var/log/audit.pos
|
||||||
time_key time
|
format json
|
||||||
time_format %Y-%m-%dT%H:%M:%S.%N%z
|
time_key time
|
||||||
tag audit
|
time_format %Y-%m-%dT%H:%M:%S.%N%z
|
||||||
</source>
|
tag audit
|
||||||
|
</source>
|
||||||
|
|
||||||
<filter audit>
|
<filter audit>
|
||||||
#https://github.com/fluent/fluent-plugin-rewrite-tag-filter/issues/13
|
#https://github.com/fluent/fluent-plugin-rewrite-tag-filter/issues/13
|
||||||
type record_transformer
|
type record_transformer
|
||||||
enable_ruby
|
enable_ruby
|
||||||
<record>
|
<record>
|
||||||
namespace ${record["objectRef"].nil? ? "none":(record["objectRef"]["namespace"].nil? ? "none":record["objectRef"]["namespace"])}
|
namespace ${record["objectRef"].nil? ? "none":(record["objectRef"]["namespace"].nil? ? "none":record["objectRef"]["namespace"])}
|
||||||
</record>
|
</record>
|
||||||
</filter>
|
</filter>
|
||||||
|
|
||||||
<match audit>
|
<match audit>
|
||||||
# route audit according to namespace element in context
|
# route audit according to namespace element in context
|
||||||
@type rewrite_tag_filter
|
@type rewrite_tag_filter
|
||||||
rewriterule1 namespace ^(.+) ${tag}.$1
|
rewriterule1 namespace ^(.+) ${tag}.$1
|
||||||
</match>
|
</match>
|
||||||
|
|
||||||
<filter audit.**>
|
<filter audit.**>
|
||||||
@type record_transformer
|
@type record_transformer
|
||||||
remove_keys namespace
|
remove_keys namespace
|
||||||
</filter>
|
</filter>
|
||||||
|
|
||||||
<match audit.**>
|
<match audit.**>
|
||||||
@type forest
|
@type forest
|
||||||
subtype file
|
subtype file
|
||||||
remove_prefix audit
|
remove_prefix audit
|
||||||
<template>
|
<template>
|
||||||
time_slice_format %Y%m%d%H
|
time_slice_format %Y%m%d%H
|
||||||
compress gz
|
compress gz
|
||||||
path /var/log/audit-${tag}.*.log
|
path /var/log/audit-${tag}.*.log
|
||||||
format json
|
format json
|
||||||
include_time_key true
|
include_time_key true
|
||||||
</template>
|
</template>
|
||||||
</match>
|
</match>
|
||||||
|
```
|
||||||
|
|
||||||
1. start fluentd
|
1. start fluentd
|
||||||
|
|
||||||
$ fluentd -c /etc/fluentd/config -vv
|
```shell
|
||||||
|
$ fluentd -c /etc/fluentd/config -vv
|
||||||
|
```
|
||||||
|
|
||||||
1. start kube-apiserver with the following options:
|
1. start kube-apiserver with the following options:
|
||||||
|
|
||||||
--audit-policy-file=/etc/kubernetes/audit-policy.yaml --audit-log-path=/var/log/kube-audit --audit-log-format=json
|
```shell
|
||||||
|
--audit-policy-file=/etc/kubernetes/audit-policy.yaml --audit-log-path=/var/log/kube-audit --audit-log-format=json
|
||||||
|
```
|
||||||
|
|
||||||
1. check audits for different namespaces in /var/log/audit-*.log
|
1. check audits for different namespaces in /var/log/audit-*.log
|
||||||
|
|
||||||
#### Use logstash to collect and distribute audit events from webhook backend
|
#### Use logstash to collect and distribute audit events from webhook backend
|
||||||
@@ -421,56 +430,68 @@ different users into different files.
|
|||||||
1. install [logstash][logstash_install_doc]
|
1. install [logstash][logstash_install_doc]
|
||||||
1. create config file for logstash
|
1. create config file for logstash
|
||||||
|
|
||||||
$ cat <<EOF > /etc/logstash/config
|
```shell
|
||||||
input{
|
$ cat <<EOF > /etc/logstash/config
|
||||||
http{
|
input{
|
||||||
#TODO, figure out a way to use kubeconfig file to authenticate to logstash
|
http{
|
||||||
#https://www.elastic.co/guide/en/logstash/current/plugins-inputs-http.html#plugins-inputs-http-ssl
|
#TODO, figure out a way to use kubeconfig file to authenticate to logstash
|
||||||
port=>8888
|
#https://www.elastic.co/guide/en/logstash/current/plugins-inputs-http.html#plugins-inputs-http-ssl
|
||||||
}
|
port=>8888
|
||||||
}
|
}
|
||||||
filter{
|
}
|
||||||
split{
|
filter{
|
||||||
# Webhook audit backend sends several events together with EventList
|
split{
|
||||||
# split each event here.
|
# Webhook audit backend sends several events together with EventList
|
||||||
field=>[items]
|
# split each event here.
|
||||||
# We only need event subelement, remove others.
|
field=>[items]
|
||||||
remove_field=>[headers, metadata, apiVersion, "@timestamp", kind, "@version", host]
|
# We only need event subelement, remove others.
|
||||||
}
|
remove_field=>[headers, metadata, apiVersion, "@timestamp", kind, "@version", host]
|
||||||
mutate{
|
}
|
||||||
rename => {items=>event}
|
mutate{
|
||||||
}
|
rename => {items=>event}
|
||||||
}
|
}
|
||||||
output{
|
}
|
||||||
file{
|
output{
|
||||||
# Audit events from different users will be saved into different files.
|
file{
|
||||||
path=>"/var/log/kube-audit-%{[event][user][username]}/audit"
|
# Audit events from different users will be saved into different files.
|
||||||
}
|
path=>"/var/log/kube-audit-%{[event][user][username]}/audit"
|
||||||
}
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
1. start logstash
|
1. start logstash
|
||||||
|
|
||||||
$ bin/logstash -f /etc/logstash/config --path.settings /etc/logstash/
|
```shell
|
||||||
|
$ bin/logstash -f /etc/logstash/config --path.settings /etc/logstash/
|
||||||
|
```
|
||||||
|
|
||||||
1. create a [kubeconfig file](/docs/tasks/access-application-cluster/authenticate-across-clusters-kubeconfig/) for kube-apiserver webhook audit backend
|
1. create a [kubeconfig file](/docs/tasks/access-application-cluster/authenticate-across-clusters-kubeconfig/) for kube-apiserver webhook audit backend
|
||||||
|
|
||||||
$ cat <<EOF > /etc/kubernetes/audit-webhook-kubeconfig
|
```shell
|
||||||
apiVersion: v1
|
$ cat <<EOF > /etc/kubernetes/audit-webhook-kubeconfig
|
||||||
clusters:
|
apiVersion: v1
|
||||||
- cluster:
|
clusters:
|
||||||
server: http://<ip_of_logstash>:8888
|
- cluster:
|
||||||
name: logstash
|
server: http://<ip_of_logstash>:8888
|
||||||
contexts:
|
name: logstash
|
||||||
- context:
|
contexts:
|
||||||
cluster: logstash
|
- context:
|
||||||
user: ""
|
cluster: logstash
|
||||||
name: default-context
|
user: ""
|
||||||
current-context: default-context
|
name: default-context
|
||||||
kind: Config
|
current-context: default-context
|
||||||
preferences: {}
|
kind: Config
|
||||||
users: []
|
preferences: {}
|
||||||
EOF
|
users: []
|
||||||
|
EOF
|
||||||
|
```
|
||||||
|
|
||||||
1. start kube-apiserver with the following options:
|
1. start kube-apiserver with the following options:
|
||||||
|
|
||||||
--audit-policy-file=/etc/kubernetes/audit-policy.yaml --audit-webhook-config-file=/etc/kubernetes/audit-webhook-kubeconfig
|
```shell
|
||||||
|
--audit-policy-file=/etc/kubernetes/audit-policy.yaml --audit-webhook-config-file=/etc/kubernetes/audit-webhook-kubeconfig
|
||||||
|
```
|
||||||
|
|
||||||
1. check audits in logstash node's directories /var/log/kube-audit-*/audit
|
1. check audits in logstash node's directories /var/log/kube-audit-*/audit
|
||||||
|
|
||||||
Note that in addition to file output plugin, logstash has a variety of outputs that
|
Note that in addition to file output plugin, logstash has a variety of outputs that
|
||||||
|
|||||||
Reference in New Issue
Block a user