whentimeslows/website
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Release 1.9 (#5978)

Browse Source 
* Trivial change to open release branch

* Undo trivial change

* add service ipvs overview

* Add instructions on how to setup kubectl

* Document conntrack dependency for kube-proxy

* Add an a

This is kind of jarring / missing an article. I'm guessing it should either be ' to a rack of bare metal servers.' or '...to racks of bare metal servers.'.

* adding example responses for common issues

 - support request
 - code bug report

* Trivial change to open release branch

* Undo trivial change

* Signed-off-by: Ziqi Zhao <zhaoziqi@qiniu.com> (#5366)

Fix the not-working test case yaml for /doc/concepts/storage/volumes.md

* kubectl-overview

* temp fix for broken pod and deployment links

* Update Table of Solutions for Juju

* Revise certificates documentation (#5965)

* Update review-issues.md

Some edits for clarity and condensed language.

* Update init-containers.md

Fix leading spaces in commands.

* Update kubectl-overview.md

Fix format.

* Update clc.md

Fix format.

* Update openstack-heat.md

The url no need. just  highlight.

* Typo

I believe this should be "users" not "uses"

* making explicit hostname uniq requirement

* Update scheduling-hugepages.md

* Update update-daemon-set.md

* fix redirection of PersistentVolume

* Update hpa.md

* update kubectl instruction

* Use the format of kubeadm init

* fix spelling error

guarnatees  to guarantees

* add matchLabels description (#6020)

* search and replace for k8s.github.io to website (#6019)

* fix scale command of object-management (#6011)

* Update replicaset.md (#6009)

* Update secret.md (#6008)

* specify password for mysql image (#5990)

* specify password for mysql image

* specify password for mysql image

* link error for run-stateless-application-deployment.md (#5985)

* link error for run-stateless-application-deployment.md

* link error for run-stateless-application-deployment.md

* Add performance implications of inter-pod affinity/anti-affinity (#5979)

* 404 monthly maintenance - October 2017 (#5977)

* Updated redirects

* More redirects

* Add conjure-up to Turnkey Cloud Solutions list (#5973)

* Add conjure-up to Turnkey Cloud Solutions list

* Changed wording slightly

* change the StatefulSet to ReplicaSet in reference (#5968)

* Clarification of failureThreshold of probes (#5963)

* Mention usage of block storage version param (#5925)

Mention usage of block storage version (bs-version) parameter to
workaround attachment issues using older K8S versions on an OpenStack
cloud with path-based endpoints.

Resolves: https://github.com/kubernetes/kubernetes.github.io/issues/5924

* Update sysctl-cluster.md (#5894)

Include guide on enabling unsafe sysctls in minikube

* Avoid Latin phrases & format note (#5889)

* Avoid Latin phrases & format note

according the Documentation Style Guide

* Update scratch.md

* Update scratch.md

* resolves jekyll rendering error (#5976)

- chinese isn't understood for keys in YAML frontmatter in jekyll, so
   replaced it with the english equivalent that doesn't throw the
following error on rendering:

Error reading file src/kubernetes.github.io/cn/docs/concepts/cluster-administration/device-plugins.md: (<unknown>): could not find expected ':' while scanning a simple key at line 4 column 1

* Change VM to pod. (#6022)

* Add link to custom metrics. (#6023)

* Rephrase core group. (#6024)

* Added explanation on context to when joining (#6018)

* Update create-cluster-kubeadm.md (#5761)

Update Canal version in pod network apply commands

* Fixes issue #5620 (#5869)

* Fixes issue #5620

Signed-off-by: Brad Topol <btopol@us.ibm.com>

* Restructured so that review process is for both current and upcoming
releases.  Added content describing the use of tech reviewers.

* Removed incorrect Kubernetes reviewer link.

* Fixed tech reviewer URL to now use website

* Update pod-priority-preemption.md

fix-wrong-link-to-pod-preemption

* pod-security-policy.md: add links to the page about admission plugins.

* Adding all files for BlaBlaCar case study (#5857)

* Adding all files for BlaBlaCar case study

* Update blablacar.html

* Fix changed URL for google containers

* Add /docs/reference/auto-generated directory

* correct the downwardapi redirect

* Remove links using "here"

* Rename to /docs/reference/generated directory

* add Concept template

* Change title to just Ingress

* Link mistake (#6038)

* link mistake

* link mistake

* skip title check for skip_title_check.txt

* skip title check for skip_title_check.txt

* remove doesn't exist link.

* Fix podpreset task (#5705)

* Add a simple pod manifest to pod overview (#5986)

* Split PodPreset concept out from task doc (#5984)

* Add selector spec description (#5789)

* Add selector spec description

* Fix selector field explanation

* Put orphaned topics in TOC. (#6051)

* static-pod example bad format in the final page (#6050)

* static-pod example bad format in the final page

* static-pod example bad format in the final page

* static-pod example bad format in the final page

* static-pod example bad format in the final page

* static-pod example bad format in the final page

* Fix `backoffLimit` field misplacement (#6042)

It should be placed in JobSpec according to:
https://github.com/kubernetes/kubernetes/blob/master/api/swagger-spec/batch_v1.json#L1488-L1514

* Update addons.md (#6061)

* add info about VMware NSX-T CNI plugin (#5987)

* add info about VMware NSX-T CNI plugin

Hello,

I'm VMware Networking and Security Architect and would like to include short information about our CNI plugin implementation similar to what other vendors did

Best regards

Emil Gagala

* Update networking.md

* Update networking.md

* Update networking.md

* Update: Using universal zsh configuration (#5669)

* Update install-kubectl.md

Zsh is not only oh-my-zsh, so I added universal configuration for zsh that also can be used in prezto.

* fix merge error after rebase

* Operating etcd cluster for Kubernetes bad format in the final page (#6056)

* Operating etcd cluster for Kubernetes bad format in the final page

* Update configure-upgrade-etcd.md

* Update configure-upgrade-etcd.md

* Usage note and warning tags. (#6053)

* Usage note and warning tags.

* Update configure-upgrade-etcd.md

* Update configure-upgrade-etcd.md

* Document jekyll includes snippets

* Add jekyll includes to docs home toc

- Remove extra kubernetes home in toc

* document docker cgroupdriver req (#5937)

* Update test blacklists (#6063)

* Update toc check blacklist

* Update title check blacklist

* wip

* wip

* Fix typo

* Document unconfined apparmor profile

* Revert "Document the unconfined profile for AppArmor" (#6268)

* CRD Validation: remove alpha warning, change enable instructions to (#6066)

disable

* Documented service annotation for AWS ELB SSL policy

* kubeadm: add a note about the new `--print-join-command` flag.

This is a new flag for the `kubeadm token create` command.

* Add a note to PDB page

* Improve Kubeadm reference doc (#6103)

* automatically-generated kubeadm reference doc

* user-mantained kubeadm reference doc

* Documentation for CSIPersistentVolume

* change replicaset documentation to use apps/v1 APIs

* Update service.md

ipvs alpha version -> beta version

* Updated Deployment concept docs (#6494)

* Updated Deployment concept docs

* Addressed comments

* Documentation for volume scheduling alpha feature

* Update admission control docs for webhooks

* Improve DNS documentation (#6479)

* update ds for 1.9

* Update service.md

* Update service.md

* Revert "begin updating webhook documentation" (#6575)

* Update version numbers to include 1.9 (#6518)

* Update site versions for 1.9

* Removed 1.4 docs

* Update _config.yml

* Update _config.yml

* updates for raw block devices

* rbac: docs for aggregated cluster roles (#6474)

* Added IPv6 information for Kubelet arguments (#6498)

* Added IPv6 info to kube-proxy arguments

* Added IPv6 information for argument for kubelet

* Update PVC resizing documentation (#6487)

* Updates for Windows Server version 1709 with K8s v1.8 (#6180)

* Updated for WSv1709 and K8s v1.8

* Updated picture and CNI config

* Fixed formatting on CNI Config

* Updated docs to reference Microsoft/SDN GitHub docs

* fix typo

* Workaround for Jekyllr frontmatter

* Added section on features and limitations, with example yaml files.

* Update index.md

* Added kubeadm section, few other small fixes

* Few minor grammar fixes

* Update access-cluster.md with a comment that for IPv6
the user should use [::1] for the localhost

* Addressed a number of issues brought up against the base PR

* Fixed windows-host-setup link

* Rewrite PodSecurityPolicy guide

* Update index.md

Signed-off-by: Alin Balutoiu <abalutoiu@cloudbasesolutions.com>
Signed-off-by: Alin Gabriel Serdean <aserdean@ovn.org>

* Spelling correction and sentence capitalization.

- Corrected the spelling error for storing, was put in as 'stoing'.
- Capitalized list items.
- Added '.' at end of sentences in the list items.

* Update index.md

* Update index.md

* Addressed comments and rebased

* Fixed formatting

* Fixed formatting

* Updated header link

* Updated hyperlinks

* Updated warning

* formatting

* formatting

* formatting

* Revert "Update access-cluster.md with a comment that for IPv6"

This reverts commit 31e4dbdc25.

* Revert "fix typo"

This reverts commit c05678752d.

* Revert "Workaround for Jekyllr frontmatter"

This reverts commit b84ac59624.

* Fixed grammatical issues and reverted non-related commits

* Revert "Rewrite PodSecurityPolicy guide"

This reverts commit 5d39cfeae4.

* Revert "Spelling correction and sentence capitalization."

This reverts commit 47eed4346e.

* Fixed auto-numbering

* Minor formatting updates

* CoreDNS feature documentation (#6463)

* Initial placeholder PR for CoreDNS feature documentation

* Remove from admin, add content

* Fix missing endcapture

* Add to tasks.yml

* Review feedback

* Postpone Deletion of a Persistent Volume Claim in case It Is Used by a Pod (#6415)

* Postpone Deletion of a Persistent Volume Claim in case It Is Used by a Pod

A new feature PVC Protection was added into K8s 1.9 that's why this documentation change is needed.

* Added tag at the top of each new area.

* Fix typo

* Fix: switched on in (all kubelets) -> (all K8s components).

* Added link to admission controller

* Moved PVC Protection configuration into Before you begin section.

* Added steps how to verify PVC Protection feature.

* Fixes for admission controller plugin description and for PVC Protection description in PVC lifecycle.

* Testing official rendering of enumerations (1., 2., 3., etc.)

* Re-write to address comments from review.

* Fixed definition when a PVC is in active use by a pod.

* Change auditing docs page for 1.9 release (#6427)

* Change auditing docs page for 1.9 release

Signed-off-by: Mik Vyatskov <vmik@google.com>

* Address review comments

Signed-off-by: Mik Vyatskov <vmik@google.com>

* Address review comments

Signed-off-by: Mik Vyatskov <vmik@google.com>

* Address review comments

Signed-off-by: Mik Vyatskov <vmik@google.com>

* Fix broken link

Signed-off-by: Mik Vyatskov <vmik@google.com>

* short circuit deny docs (#6536)

* line wrap

* short circuit deny

* address comments

* Add kubeadm 1.9 upgrade docs (#6485)

* kubeadm: Improve kubeadm documentation for v1.9 (#6645)

* Update admission control docs for webhooks (re-send #6368) (#6650)

* Update admission control docs for webhooks

* update in response to comments

* Revamp rkt and add CRI-O as alternative runtime (#6371)

Signed-off-by: Lorenzo Fontana <lo@linux.com>

* Documented NLB for Kubernetes 1.9 (#6260)

* Added IPV6 information to setup cluster using kubeadm (#6465)

* Added IPV6 information to setup cluster using kubeadm

* Updated kubeadm.md & create-cluster-kubeadm.md with IPv6 related information

* Added IPv6 options for kubeadm --init  & automated address binding for kube-proxy based on version of IP configured for API server)

* Changes to kubeadm.md as per comments

* Modified kubeadm.md and create-cluster-kubeadm.md

* Implemented changes requested by zacharysarah

* Removed autogenerated kubeadm.md changes

* StatefulSet 1.9 updates. (#6550)

* updates sts concept and tutorials to use 1.9 apps/v1

* Update statefulset.md

* clarify pod name label

* Garbage collection updates for 1.9 (#6555)

* 1.9 gc policy update

* carify deletion

* Couple nits for dnsConfig doc (#6652)

* Add doc for AllowedFlexVolume (#6563)

* Update OpenStack Cloud Provider API support for v1.9 (#6638)

* Flex volume is GA. Remove alpha notation. (#6666)

* Update generated ref docs for Kubernetes and Federation components. (#6658)

* Update generated ref docs for Kubernetes and Federation components.

* Rename kubectl-options to kubectl.

* Add title to kubectl.

* Fix double synopsis.

* Update Federation API ref docs for 1.9. (#6636)

* Update federation API ref docs.

* Move and redirect.

* Move generated Federation docs to the generated directory.

* Fix titles.

* Type

* Fix titles

* Update auto-generated Kubernetes APi ref docs. (#6646)

* Update kubectl commands for 1.9 (#6635)

* add ExtendedResourceToleration admission controller (#6618)

* Update API reference paths for v1.9 (#6681)
This commit is contained in:
Zach Corleissen
2017-12-15 17:36:13 -06:00
committed by GitHub
parent 0e628e7895
commit 08af95d42e
893 changed files with 131178 additions and 105058 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/tasks/access-application-cluster/access-cluster.md
+1 -2
View File
@@ -55,8 +55,7 @@ $ kubectl proxy --port=8080 &
See [kubectl proxy](/docs/user-guide/kubectl/{{page.version}}/#proxy) for more details.
Then you can explore the API with curl, wget, or a browser, replacing localhost
with [::1] for IPv6, like so:
Then you can explore the API with curl, wget, or a browser, like so:
```shell
$ curl http://localhost:8080/api/
docs/tasks/access-application-cluster/frontend.yaml
+1 -1
View File
@@ -1,5 +1,5 @@
apiVersion: v1
kind: Service
apiVersion: v1
metadata:
name: frontend
spec:
docs/tasks/access-kubernetes-api/extend-api-custom-resource-definitions.md
+4 -4
View File
@@ -207,12 +207,12 @@ Additionally, the following restrictions are applied to the schema:
- The field `uniqueItems` cannot be set to true.
- The field `additionalProperties` cannot be set to false.
This feature is __alpha__ in v1.8 and may change in backward incompatible ways.
Enable this feature using the `CustomResourceValidation` feature gate on
This feature is __beta__ in v1.9.
You can disable this feature using the `CustomResourceValidation` feature gate on
the [kube-apiserver](/docs/admin/kube-apiserver):
```
--feature-gates=CustomResourceValidation=true
```
--feature-gates=CustomResourceValidation=false
```
The schema is defined in the CustomResourceDefinition. In the following example, the
docs/tasks/administer-cluster/busybox.yaml
+14
View File
@@ -0,0 +1,14 @@
apiVersion: v1
kind: Pod
metadata:
name: busybox
namespace: default
spec:
containers:
- name: busybox
image: busybox
command:
- sleep
- "3600"
imagePullPolicy: IfNotPresent
restartPolicy: Always
docs/tasks/administer-cluster/coredns.md
+42
View File
@@ -0,0 +1,42 @@
---
approvers:
- johnbelamaric
title: Using CoreDNS for Service Discovery
min-kubernetes-server-version: v1.9
---
{% include feature-state-alpha.md %}
{% capture overview %}
This page describes how to enable CoreDNS instead of kube-dns for service
discovery.
{% endcapture %}
{% capture prerequisites %}
{% include task-tutorial-prereqs.md %}
{% endcapture %}
{% capture steps %}
## Installing CoreDNS with kubeadm
In Kubernetes 1.9, [CoreDNS](https://coredns.io) is available as an alpha feature and
may be installed by setting the `CoreDNS` feature gate to `true` during `kubeadm init`:
```
kubeadm init --feature-gates=CoreDNS=true
```
This installs CoreDNS instead of kube-dns.
{% endcapture %}
{% capture whatsnext %}
You can configure [CoreDNS](https://coredns.io) to support many more use cases than
kube-dns by modifying the `Corefile`. For more information, see the
[CoreDNS site]https://coredns.io/2017/05/08/custom-dns-entries-for-kubernetes/).
{% endcapture %}
{% include templates/task.md %}
docs/tasks/administer-cluster/dns-custom-nameservers.md
+245 -27
View File
@@ -2,12 +2,12 @@
approvers:
- bowei
- zihongz
title: Configure private DNS zones and upstream nameservers in Kubernetes
title: Configure DNS Service
---
{% capture overview %}
This page shows how to add custom private DNS zones (stub domains) and upstream
nameservers.
This page provides hints on configuring DNS Pod and guidance on customizing the
DNS resolution process and diagnosing DNS problems.
{% endcapture %}
{% capture prerequisites %}
@@ -18,6 +18,45 @@ nameservers.
{% capture steps %}
## Introduction
Starting from Kubernetes v1.3, DNS is a built-in service launched automatically
using the addon manager
[cluster add-on](http://releases.k8s.io/{{page.githubbranch}}/cluster/addons/README.md).
The running Kubernetes DNS pod holds 3 containers:
- "`kubedns`": The `kubedns` process watches the Kubernetes master for changes
in Services and Endpoints, and maintains in-memory lookup structures to serve
DNS requests.
- "`dnsmasq`": The `dnsmasq` container adds DNS caching to improve performance.
- "`healthz`": The `healthz` container provides a single health check endpoint
while performing dual healthchecks (for `dnsmasq` and `kubedns`).
The DNS pod is exposed as a Kubernetes Service with a static IP. Once assigned
the kubelet passes DNS configured using the `--cluster-dns=<dns-service-ip>`
flag to each container.
DNS names also need domains. The local domain is configurable in the kubelet
using the flag `--cluster-domain=<default-local-domain>`.
The Kubernetes cluster DNS server is based off the
[SkyDNS](https://github.com/skynetservices/skydns) library. It supports forward
lookups (A records), service lookups (SRV records) and reverse IP address
lookups (PTR records).
## Inheriting DNS from the node
When running a pod, kubelet will prepend the cluster DNS server and search
paths to the node's own DNS settings. If the node is able to resolve DNS names
specific to the larger environment, pods should be able to, also.
See [Known issues](#known-issues) below for a caveat.
If you don't want this, or if you want a different DNS config for pods, you can
use the kubelet's `--resolv-conf` flag. Setting it to "" means that pods will
not inherit DNS. Setting it to a valid file path means that kubelet will use
this file instead of `/etc/resolv.conf` for DNS inheritance.
## Configure stub-domain and upstream DNS servers
Cluster administrators can specify custom stub domains and upstream nameservers
@@ -43,7 +82,8 @@ As specified, DNS requests with the “.acme.local” suffix
are forwarded to a DNS listening at 1.2.3.4. Google Public DNS
serves the upstream queries.
The table below describes how queries with certain domain names would map to their destination DNS servers:
The table below describes how queries with certain domain names would map to
their destination DNS servers:
| Domain name | Server answering the query |
| ----------- | -------------------------- |
@@ -58,36 +98,37 @@ details about the configuration option format.
{% capture discussion %}
## Understanding name resolution in Kubernetes
### Impacts on Pods
DNS policies can be set on a per-pod basis. Currently Kubernetes supports two pod-specific DNS policies: “Default” and “ClusterFirst”. These policies are specified with the `dnsPolicy` flag.
Custom upstream nameservers and stub domains won't impact Pods that have their
`dnsPolicy` set to "`Default`" or "`None`".
*NOTE: "Default" is not the default DNS policy. If `dnsPolicy` is not
explicitly specified, then “ClusterFirst” is used.*
If a Pod's `dnsPolicy` is set to "`ClusterFirst`", its name resolution is
handled differently, depending on whether stub-domain and upstream DNS servers
are configured.
### "Default" DNS Policy
**Without custom configurations**: Any query that does not match the configured
cluster domain suffix, such as "www.kubernetes.io", is forwarded to the upstream
nameserver inherited from the node.
If `dnsPolicy` is set to “Default”, then the name resolution configuration is
inherited from the node that the pods run on. Custom upstream nameservers and stub domains cannot be used in conjunction with this policy.
### "ClusterFirst" DNS Policy
If the `dnsPolicy` is set to "ClusterFirst", name resolution is handled differently, *depending on whether stub-domain and upstream DNS servers are configured*.
**Without custom configurations**: Any query that does not match the configured cluster domain suffix, such as "www.kubernetes.io", is forwarded to the upstream nameserver inherited from the node.
**With custom configurations**: If stub domains and upstream DNS servers are configured (as in the [previous example](#configuring-stub-domain-and-upstream-dns-servers)), DNS queries will be
routed according to the following flow:
**With custom configurations**: If stub domains and upstream DNS servers are
configured (as in the [previous example](#configuring-stub-domain-and-upstream-dns-servers)),
DNS queries will be routed according to the following flow:
1. The query is first sent to the DNS caching layer in kube-dns.
1. From the caching layer, the suffix of the request is examined and then forwarded to the appropriate DNS, based on the following cases:
1. From the caching layer, the suffix of the request is examined and then
forwarded to the appropriate DNS, based on the following cases:
* *Names with the cluster suffix* (e.g.".cluster.local"): The request is sent to kube-dns.
* *Names with the cluster suffix* (e.g.".cluster.local"):
The request is sent to kube-dns.
* *Names with the stub domain suffix* (e.g. ".acme.local"): The request is sent to the configured custom DNS resolver (e.g. listening at 1.2.3.4).
* *Names with the stub domain suffix* (e.g. ".acme.local"):
The request is sent to the configured custom DNS resolver (e.g. listening at 1.2.3.4).
* *Names without a matching suffix* (e.g."widget.com"): The request is forwarded to the upstream DNS (e.g. Google public DNS servers at 8.8.8.8 and 8.8.4.4).
* *Names without a matching suffix* (e.g."widget.com"):
The request is forwarded to the upstream DNS
(e.g. Google public DNS servers at 8.8.8.8 and 8.8.4.4).
![DNS lookup flow](/docs/tasks/administer-cluster/dns-custom-nameservers/dns.png)
@@ -100,9 +141,9 @@ Options for the kube-dns `kube-system:kube-dns` ConfigMap:
| `stubDomains` (optional) | A JSON map using a DNS suffix key (e.g. “acme.local”) and a value consisting of a JSON array of DNS IPs. | The target nameserver may itself be a Kubernetes service. For instance, you can run your own copy of dnsmasq to export custom DNS names into the ClusterDNS namespace. |
| `upstreamNameservers` (optional) | A JSON array of DNS IPs. | Note: If specified, then the values specified replace the nameservers taken by default from the node’s `/etc/resolv.conf`. Limits: a maximum of three upstream nameservers can be specified. |
## Additional examples
### Examples
### Example: Stub domain
#### Example: Stub domain
In this example, the user has a Consul DNS service discovery system that they wish to
integrate with kube-dns. The consul domain server is located at 10.150.0.1, and
@@ -124,7 +165,7 @@ Note that the cluster administrator did not wish to override the node’s
upstream nameservers, so they did not specify the optional
`upstreamNameservers` field.
### Example: Upstream nameserver
#### Example: Upstream nameserver
In this example the cluster administrator wants to explicitly force all
non-cluster DNS lookups to go through their own nameserver at 172.16.0.1.
@@ -142,6 +183,183 @@ data:
[“172.16.0.1”]
```
## Debugging DNS resolution
### Create a simple Pod to use as a test environment
Create a file named busybox.yaml with the following contents:
{% include code.html language="yaml" file="busybox.yaml" ghlink="/docs/tasks/administer-cluster/busybox.yaml" %}
Then create a pod using this file and verify its status:
```shell
$ kubectl create -f busybox.yaml
pod "busybox" created
$ kubectl get pods busybox
NAME READY STATUS RESTARTS AGE
busybox 1/1 Running 0 <some-time>
```
Once that pod is running, you can exec `nslookup` in that environment.
If you see something like the following, DNS is working correctly.
```shell
$ kubectl exec -ti busybox -- nslookup kubernetes.default
Server: 10.0.0.10
Address 1: 10.0.0.10
Name: kubernetes.default
Address 1: 10.0.0.1
```
If the `nslookup` command fails, check the following:
### Check the local DNS configuration first
Take a look inside the resolv.conf file.
(See [Inheriting DNS from the node](#inheriting-dns-from-the-node) and
[Known issues](#known-issues) below for more information)
```shell
$ kubectl exec busybox cat /etc/resolv.conf
```
Verify that the search path and name server are set up like the following
(note that search path may vary for different cloud providers):
```
search default.svc.cluster.local svc.cluster.local cluster.local google.internal c.gce_project_id.internal
nameserver 10.0.0.10
options ndots:5
```
Errors such as the following indicate a problem with the kube-dns add-on or
associated Services:
```
$ kubectl exec -ti busybox -- nslookup kubernetes.default
Server: 10.0.0.10
Address 1: 10.0.0.10
nslookup: can't resolve 'kubernetes.default'
```
or
```
$ kubectl exec -ti busybox -- nslookup kubernetes.default
Server: 10.0.0.10
Address 1: 10.0.0.10 kube-dns.kube-system.svc.cluster.local
nslookup: can't resolve 'kubernetes.default'
```
### Check if the DNS pod is running
Use the `kubectl get pods` command to verify that the DNS pod is running.
```shell
$ kubectl get pods --namespace=kube-system -l k8s-app=kube-dns
NAME READY STATUS RESTARTS AGE
...
kube-dns-v19-ezo1y 3/3 Running 0 1h
...
```
If you see that no pod is running or that the pod has failed/completed, the DNS
add-on may not be deployed by default in your current environment and you will
have to deploy it manually.
### Check for Errors in the DNS pod
Use `kubectl logs` command to see logs for the DNS daemons.
```shell
$ kubectl logs --namespace=kube-system $(kubectl get pods --namespace=kube-system -l k8s-app=kube-dns -o name) -c kubedns
$ kubectl logs --namespace=kube-system $(kubectl get pods --namespace=kube-system -l k8s-app=kube-dns -o name) -c dnsmasq
$ kubectl logs --namespace=kube-system $(kubectl get pods --namespace=kube-system -l k8s-app=kube-dns -o name) -c sidecar
```
See if there is any suspicious log. Letter '`W`', '`E`', '`F`' at the beginning
represent Warning, Error and Failure. Please search for entries that have these
as the logging level and use
[kubernetes issues](https://github.com/kubernetes/kubernetes/issues)
to report unexpected errors.
### Is DNS service up?
Verify that the DNS service is up by using the `kubectl get service` command.
```shell
$ kubectl get svc --namespace=kube-system
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
...
kube-dns 10.0.0.10 <none> 53/UDP,53/TCP 1h
...
```
If you have created the service or in the case it should be created by default
but it does not appear, see
[debugging services](/docs/tasks/debug-application-cluster/debug-service/) for
more information.
### Are DNS endpoints exposed?
You can verify that DNS endpoints are exposed by using the `kubectl get endpoints`
command.
```shell
$ kubectl get ep kube-dns --namespace=kube-system
NAME ENDPOINTS AGE
kube-dns 10.180.3.17:53,10.180.3.17:53 1h
```
If you do not see the endpoints, see endpoints section in the
[debugging services](/docs/tasks/debug-application-cluster/debug-service/) documentation.
For additional Kubernetes DNS examples, see the
[cluster-dns examples](https://github.com/kubernetes/examples/tree/master/staging/cluster-dns)
in the Kubernetes GitHub repository.
## Known issues
Kubernetes installs do not configure the nodes' resolv.conf files to use the
cluster DNS by default, because that process is inherently distro-specific.
This should probably be implemented eventually.
Linux's libc is impossibly stuck ([see this bug from
2005](https://bugzilla.redhat.com/show_bug.cgi?id=168253)) with limits of just
3 DNS `nameserver` records and 6 DNS `search` records. Kubernetes needs to
consume 1 `nameserver` record and 3 `search` records. This means that if a
local installation already uses 3 `nameserver`s or uses more than 3 `search`es,
some of those settings will be lost. As a partial workaround, the node can run
`dnsmasq` which will provide more `nameserver` entries, but not more `search`
entries. You can also use kubelet's `--resolv-conf` flag.
If you are using Alpine version 3.3 or earlier as your base image, DNS may not
work properly owing to a known issue with Alpine.
Check [here](https://github.com/kubernetes/kubernetes/issues/30215)
for more information.
## Kubernetes Federation (Multiple Zone support)
Release 1.3 introduced Cluster Federation support for multi-site Kubernetes
installations. This required some minor (backward-compatible) changes to the
way the Kubernetes cluster DNS server processes DNS queries, to facilitate
the lookup of federated services (which span multiple Kubernetes clusters).
See the [Cluster Federation Administrators' Guide](/docs/concepts/cluster-administration/federation/)
for more details on Cluster Federation and multi-site support.
## References
- [DNS for Services and Pods](/docs/concepts/services-networking/dns-pod-service/)
- [Docs for the DNS cluster addon](http://releases.k8s.io/{{page.githubbranch}}/cluster/addons/dns/README.md)
## What's next
- [Autoscaling the DNS Service in a Cluster](/docs/tasks/administer-cluster/dns-horizontal-autoscaling/).
{% endcapture %}
{% include templates/task.md %}
docs/tasks/administer-cluster/kubeadm-upgrade-1-9.md
+244
View File
@@ -0,0 +1,244 @@
---
approvers:
- pipejakob
- luxas
- roberthbailey
- jbeda
title: Upgrading/downgrading kubeadm clusters between v1.8 to v1.9
---
{% capture overview %}
This guide is for upgrading `kubeadm` clusters from version 1.8.x to 1.9.x, as well as 1.8.x to 1.8.y and 1.9.x to 1.9.y where `y > x`.
See also [upgrading kubeadm clusters from 1.7 to 1.8](/docs/tasks/administer-cluster/kubeadm-upgrade-1-8/) if you're on a 1.7 cluster currently.
{% endcapture %}
{% capture prerequisites %}
Before proceeding:
- You need to have a functional `kubeadm` Kubernetes cluster running version 1.8.0 or higher in order to use the process described here. Swap also needs to be disabled.
- Make sure you read the [release notes](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.9.md) carefully.
- `kubeadm upgrade` now allows you to upgrade etcd. `kubeadm upgrade` will also upgrade of etcd to 3.1.10 as part of upgrading from v1.8 to v1.9 by default. This is due to the fact that etcd 3.1.10 is the officially validated etcd version for Kubernetes v1.9. The upgrade is handled automatically by kubeadm for you.
- Note that `kubeadm upgrade` will not touch any of your workloads, only Kubernetes-internal components. As a best-practice you should back up what's important to you. For example, any app-level state, such as a database an app might depend on (like MySQL or MongoDB) must be backed up beforehand.
Also, note that only one minor version upgrade is supported. For example, you can only upgrade from 1.8 to 1.9, not from 1.7 to 1.9.
{% endcapture %}
{% capture steps %}
## Upgrading your control plane
Execute these commands on your master node:
1. Install the most recent version of `kubeadm` using `curl` like so:
```shell
$ export VERSION=$(curl -sSL https://dl.k8s.io/release/stable.txt) # or manually specify a released Kubernetes version
$ export ARCH=amd64 # or: arm, arm64, ppc64le, s390x
$ curl -sSL https://dl.k8s.io/release/${VERSION}/bin/linux/${ARCH}/kubeadm > /usr/bin/kubeadm
$ chmod a+rx /usr/bin/kubeadm
```
**Caution:** Upgrading the `kubeadm` package on your system prior to upgrading the control plane causes a failed upgrade.
Even though `kubeadm` ships in the Kubernetes repositories, it's important to install `kubeadm` manually. The kubeadm
team is working on fixing this limitation.
{: .caution}
Verify that this download of kubeadm works and has the expected version:
```shell
$ kubeadm version
```
2. On the master node, run the following:
```shell
$ kubeadm upgrade plan
[preflight] Running pre-flight checks
[upgrade] Making sure the cluster is healthy:
[upgrade/health] Checking API Server health: Healthy
[upgrade/health] Checking Node health: All Nodes are healthy
[upgrade/health] Checking Static Pod manifests exists on disk: All manifests exist on disk
[upgrade/config] Making sure the configuration is correct:
[upgrade/config] Reading configuration from the cluster...
[upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[upgrade] Fetching available versions to upgrade to:
[upgrade/versions] Cluster version: v1.8.1
[upgrade/versions] kubeadm version: v1.9.0
[upgrade/versions] Latest stable version: v1.9.0
[upgrade/versions] Latest version in the v1.8 series: v1.8.6
Components that must be upgraded manually after you've upgraded the control plane with 'kubeadm upgrade apply':
COMPONENT CURRENT AVAILABLE
Kubelet 1 x v1.8.1 v1.8.6
Upgrade to the latest version in the v1.8 series:
COMPONENT CURRENT AVAILABLE
API Server v1.8.1 v1.8.6
Controller Manager v1.8.1 v1.8.6
Scheduler v1.8.1 v1.8.6
Kube Proxy v1.8.1 v1.8.6
Kube DNS 1.14.4 1.14.5
You can now apply the upgrade by executing the following command:
kubeadm upgrade apply v1.8.6
_____________________________________________________________________
Components that must be upgraded manually after you've upgraded the control plane with 'kubeadm upgrade apply':
COMPONENT CURRENT AVAILABLE
Kubelet 1 x v1.8.1 v1.9.0
Upgrade to the latest experimental version:
COMPONENT CURRENT AVAILABLE
API Server v1.8.1 v1.9.0
Controller Manager v1.8.1 v1.9.0
Scheduler v1.8.1 v1.9.0
Kube Proxy v1.8.1 v1.9.0
Kube DNS 1.14.5 1.14.7
You can now apply the upgrade by executing the following command:
kubeadm upgrade apply v1.9.0
Note: Before you do can perform this upgrade, you have to update kubeadm to v1.9.0
_____________________________________________________________________
```
The `kubeadm upgrade plan` checks that your cluster is upgradeable and fetches the versions available to upgrade to in an user-friendly way.
3. Pick a version to upgrade to and run. For example:
```shell
$ kubeadm upgrade apply v1.9.0
[preflight] Running pre-flight checks.
[upgrade] Making sure the cluster is healthy:
[upgrade/config] Making sure the configuration is correct:
[upgrade/config] Reading configuration from the cluster...
[upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
[upgrade/version] You have chosen to upgrade to version "v1.9.0"
[upgrade/versions] Cluster version: v1.8.1
[upgrade/versions] kubeadm version: v1.9.0
[upgrade/confirm] Are you sure you want to proceed with the upgrade? [y/N]: y
[upgrade/prepull] Will prepull images for components [kube-apiserver kube-controller-manager kube-scheduler]
[upgrade/apply] Upgrading your Static Pod-hosted control plane to version "v1.9.0"...
[etcd] Wrote Static Pod manifest for a local etcd instance to "/etc/kubernetes/tmp/kubeadm-upgraded-manifests802453804/etcd.yaml"
[upgrade/staticpods] Moved upgraded manifest to "/etc/kubernetes/manifests/etcd.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests502223003/etcd.yaml"
[upgrade/staticpods] Waiting for the kubelet to restart the component
[apiclient] Found 1 Pods for label selector component=etcd
[upgrade/staticpods] Component "etcd" upgraded successfully!
[upgrade/staticpods] Writing upgraded Static Pod manifests to "/etc/kubernetes/tmp/kubeadm-upgraded-manifests802453804"
[controlplane] Wrote Static Pod manifest for component kube-apiserver to "/etc/kubernetes/tmp/kubeadm-upgraded-manifests802453804/kube-apiserver.yaml"
[controlplane] Wrote Static Pod manifest for component kube-controller-manager to "/etc/kubernetes/tmp/kubeadm-upgraded-manifests802453804/kube-controller-manager.yaml"
[controlplane] Wrote Static Pod manifest for component kube-scheduler to "/etc/kubernetes/tmp/kubeadm-upgraded-manifests802453804/kube-scheduler.yaml"
[upgrade/staticpods] Moved upgraded manifest to "/etc/kubernetes/manifests/kube-apiserver.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests502223003/kube-apiserver.yaml"
[upgrade/staticpods] Waiting for the kubelet to restart the component
[apiclient] Found 1 Pods for label selector component=kube-apiserver
[upgrade/staticpods] Component "kube-apiserver" upgraded successfully!
[upgrade/staticpods] Moved upgraded manifest to "/etc/kubernetes/manifests/kube-controller-manager.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests502223003/kube-controller-manager.yaml"
[upgrade/staticpods] Waiting for the kubelet to restart the component
[apiclient] Found 1 Pods for label selector component=kube-controller-manager
[upgrade/staticpods] Component "kube-controller-manager" upgraded successfully!
[upgrade/staticpods] Moved upgraded manifest to "/etc/kubernetes/manifests/kube-scheduler.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests502223003/kube-scheduler.yaml"
[upgrade/staticpods] Waiting for the kubelet to restart the component
[apiclient] Found 1 Pods for label selector component=kube-scheduler
[upgrade/staticpods] Component "kube-scheduler" upgraded successfully!
[uploadconfig] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[bootstraptoken] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstraptoken] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstraptoken] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[addons] Applied essential addon: kube-dns
[addons] Applied essential addon: kube-proxy
[upgrade/successful] SUCCESS! Your cluster was upgraded to "v1.9.0". Enjoy!
[upgrade/kubelet] Now that your control plane is upgraded, please proceed with upgrading your kubelets in turn.
```
`kubeadm upgrade apply` does the following:
- Checks that your cluster is in an upgradeable state:
- The API server is reachable,
- All nodes are in the `Ready` state
- The control plane is healthy
- Enforces the version skew policies.
- Makes sure the control plane images are available or available to pull to the machine.
- Upgrades the control plane components or rollbacks if any of them fails to come up.
- Applies the new `kube-dns` and `kube-proxy` manifests and enforces that all necessary RBAC rules are created.
4. Manually upgrade your Software Defined Network (SDN).
Your Container Network Interface (CNI) provider may have its own upgrade instructions to follow.
Check the [addons](/docs/concepts/cluster-administration/addons/) page to
find your CNI provider and see if there are additional upgrade steps
necessary.
## Upgrading your master and node packages
For each host (referred to as `$HOST` below) in your cluster, upgrade `kubelet` by executing the following commands:
1. Prepare the host for maintenance, marking it unschedulable and evicting the workload:
```shell
$ kubectl drain $HOST --ignore-daemonsets
```
When running this command against the master host, this error is expected and can be safely ignored (since there are static pods running on the master):
```shell
node "master" already cordoned
error: pods not managed by ReplicationController, ReplicaSet, Job, DaemonSet or StatefulSet (use --force to override): etcd-kubeadm, kube-apiserver-kubeadm, kube-controller-manager-kubeadm, kube-scheduler-kubeadm
```
2. Upgrade the Kubernetes package versions on the `$HOST` node by using a Linux distribution-specific package manager:
If the host is running a Debian-based distro such as Ubuntu, run:
```shell
$ apt-get update
$ apt-get upgrade
```
If the host is running CentOS or the like, run:
```shell
$ yum update
```
Now the new version of the `kubelet` should be running on the host. Verify this using the following command on `$HOST`:
```shell
$ systemctl status kubelet
```
3. Bring the host back online by marking it schedulable:
```shell
$ kubectl uncordon $HOST
```
4. After upgrading `kubelet` on each host in your cluster, verify that all nodes are available again by executing the following (from anywhere, for example, from outside the cluster):
```shell
$ kubectl get nodes
```
If the `STATUS` column of the above command shows `Ready` for all of your hosts, you are done.
## Recovering from a failure state
If `kubeadm upgrade` somehow fails and fails to roll back, for example due to an unexpected shutdown during execution,
you can run `kubeadm upgrade` again as it is idempotent and should eventually make sure the actual state is the desired state you are declaring.
You can use `kubeadm upgrade` to change a running cluster with `x.x.x --> x.x.x` with `--force`, which can be used to recover from a bad state.
{% endcapture %}
{% include templates/task.md %}
docs/tasks/administer-cluster/pvc-protection.md
+144
View File
@@ -0,0 +1,144 @@
---
approvers:
- msau42
- jsafrane
title: Persistent Volume Claim Protection
---
{% capture overview %}
{% assign for_k8s_version="v1.9" %}{% include feature-state-alpha.md %}
As of Kubernetes 1.9, persistent volume claims (PVCs) that are in active use by a pod can be protected from pre-mature removal.
{% endcapture %}
{% capture prerequisites %}
- A v1.9 or higher Kubernetes must be installed.
- As PVC Protection is a Kubernetes v1.9 alpha feature it must be enabled:
1. [Admission controller](/docs/admin/admission-controllers/) must be started with the [PVC Protection plugin](/docs/admin/admission-controllers/#persistent-volume-claim-protection-alpha).
2. All Kubernetes components must be started with the `PVCProtection` alpha features enabled.
{% endcapture %}
{% capture steps %}
## PVC Protection Verification
The example below uses a GCE PD `StorageClass`, however, similar steps can be performed for any volume type.
Create a `StorageClass` for convenient storage provisioning:
```yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: slow
provisioner: kubernetes.io/gce-pd
parameters:
type: pd-standard
```
There are two scenarios: a PVC deleted by a user is either in active use or not in active use by a pod.
### Scenario 1: The PVC is not in active use by a pod
- Create a PVC:
```yaml
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: slzc
spec:
accessModes:
- ReadWriteOnce
storageClassName: slow
resources:
requests:
storage: 3.7Gi
```
- Check that the PVC has the finalizer `kubernetes.io/pvc-protection` set:
```shell
$ kubectl describe pvc slzc
Name: slzc
Namespace: default
StorageClass: slow
Status: Bound
Volume: pvc-bee8c30a-d6a3-11e7-9af0-42010a800002
Labels: <none>
Annotations: pv.kubernetes.io/bind-completed=yes
pv.kubernetes.io/bound-by-controller=yes
volume.beta.kubernetes.io/storage-provisioner=kubernetes.io/gce-pd
Finalizers: [kubernetes.io/pvc-protection]
Capacity: 4Gi
Access Modes: RWO
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ProvisioningSucceeded 2m persistentvolume-controller Successfully provisioned volume pvc-bee8c30a-d6a3-11e7-9af0-42010a800002 using kubernetes.io/gce-pd
```
- Delete the PVC and check that the PVC (not in active use by a pod) was removed successfully.
### Scenario 2: The PVC is in active use by a pod
- Again, create the same PVC.
- Create a pod that uses the PVC:
```yaml
kind: Pod
apiVersion: v1
metadata:
name: app1
spec:
containers:
- name: test-pod
image: gcr.io/google_containers/busybox:1.24
command:
- "/bin/sh"
args:
- "-c"
- "date > /mnt/app1.txt; sleep 60 && exit 0 || exit 1"
volumeMounts:
- name: path-pvc
mountPath: "/mnt"
restartPolicy: "Never"
volumes:
- name: path-pvc
persistentVolumeClaim:
claimName: slzc
```
- Wait until the pod status is `Running`, i.e. the PVC becomes in active use.
- Delete the PVC that is now in active use by a pod and verify that the PVC is not removed but its status is `Terminating`:
```shell
Name: slzc
Namespace: default
StorageClass: slow
Status: Terminating (since Fri, 01 Dec 2017 14:47:55 +0000)
Volume: pvc-803a1f4d-d6a6-11e7-9af0-42010a800002
Labels: <none>
Annotations: pv.kubernetes.io/bind-completed=yes
pv.kubernetes.io/bound-by-controller=yes
volume.beta.kubernetes.io/storage-provisioner=kubernetes.io/gce-pd
Finalizers: [kubernetes.io/pvc-protection]
Capacity: 4Gi
Access Modes: RWO
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ProvisioningSucceeded 52s persistentvolume-controller Successfully provisioned volume pvc-803a1f4d-d6a6-11e7-9af0-42010a800002 using kubernetes.io/gce-pd
```
- Wait until the pod status is `Terminated` (either delete the pod or wait until it finishes). Afterwards, check that the PVC is removed.
{% endcapture %}
{% capture discussion %}
{% endcapture %}
{% include templates/task.md %}
docs/tasks/debug-application-cluster/audit-policy.yaml
+68
View File
@@ -0,0 +1,68 @@
apiVersion: audit.k8s.io/v1beta1 # This is required.
kind: Policy
# Don't generate audit events for all requests in RequestReceived stage.
omitStages:
- "RequestReceived"
rules:
# Log pod changes at RequestResponse level
- level: RequestResponse
resources:
- group: ""
# Resource "pods" doesn't match requests to any subresource of pods,
# which is consistent with the RBAC policy.
resources: ["pods"]
# Log "pods/log", "pods/status" at Metadata level
- level: Metadata
resources:
- group: ""
resources: ["pods/log", "pods/status"]
# Don't log requests to a configmap called "controller-leader"
- level: None
resources:
- group: ""
resources: ["configmaps"]
resourceNames: ["controller-leader"]
# Don't log watch requests by the "system:kube-proxy" on endpoints or services
- level: None
users: ["system:kube-proxy"]
verbs: ["watch"]
resources:
- group: "" # core API group
resources: ["endpoints", "services"]
# Don't log authenticated requests to certain non-resource URL paths.
- level: None
userGroups: ["system:authenticated"]
nonResourceURLs:
- "/api*" # Wildcard matching.
- "/version"
# Log the request body of configmap changes in kube-system.
- level: Request
resources:
- group: "" # core API group
resources: ["configmaps"]
# This rule only applies to resources in the "kube-system" namespace.
# The empty string "" can be used to select non-namespaced resources.
namespaces: ["kube-system"]
# Log configmap and secret changes in all other namespaces at the Metadata level.
- level: Metadata
resources:
- group: "" # core API group
resources: ["secrets", "configmaps"]
# Log all other resources in core and extensions at the Request level.
- level: Request
resources:
- group: "" # core API group
- group: "extensions" # Version of group should NOT be included.
# A catch-all rule to log all other requests at the Metadata level.
- level: Metadata
# Long-running requests like watches that fall under this rule will not
# generate an audit event in RequestReceived.
omitStages:
- "RequestReceived"
docs/tasks/debug-application-cluster/audit.md
+100 -269
View File
@@ -9,7 +9,9 @@ title: Auditing
* TOC
{:toc}
Kubernetes Audit provides a security-relevant chronological set of records documenting
{% include feature-state-beta.md %}
Kubernetes auditing provides a security-relevant chronological set of records documenting
the sequence of activities that have affected system by individual users, administrators
or other components of the system. It allows cluster administrator to
answer the following questions:
@@ -22,55 +24,18 @@ answer the following questions:
- from where was it initiated?
- to where was it going?
## Legacy Audit
[Kube-apiserver][kube-apiserver] performs auditing. Each request on each stage
of its execution generates an event, which is then pre-processed according to
a certain policy and written to a backend. You can find more details about the
pipeline in the [design proposal][auditing-proposal].
Kubernetes audit is part of [Kube-apiserver][kube-apiserver] logging all requests
processed by the server. Each audit log entry contains two lines:
1. The request line containing a unique ID to match the response and request metadata, such as the source IP, requesting user, impersonation information, resource being requested, etc.
2. The response line containing a unique ID matching the request line and the response code.
Example output for `admin` user listing pods in the `default` namespace:
```
2017-03-21T03:57:09.106841886-04:00 AUDIT: id="c939d2a7-1c37-4ef1-b2f7-4ba9b1e43b53" ip="127.0.0.1" method="GET" user="admin" groups="\"system:masters\",\"system:authenticated\"" as="<self>" asgroups="<lookup>" namespace="default" uri="/api/v1/namespaces/default/pods"
2017-03-21T03:57:09.108403639-04:00 AUDIT: id="c939d2a7-1c37-4ef1-b2f7-4ba9b1e43b53" response="200"
```
Note that Kubernetes 1.8 has switched to use the advanced structured audit log by default.
To fallback to this legacy audit, disable the advanced auditing feature
using the `AdvancedAuditing` feature gate on the [kube-apiserver][kube-apiserver]:
```
--feature-gates=AdvancedAuditing=false
```
### Configuration
[Kube-apiserver][kube-apiserver] provides the following options which are responsible
for configuring where and how audit logs are handled:
- `audit-log-path` - enables the audit log pointing to a file where the requests are being logged to, '-' means standard out.
- `audit-log-maxage` - specifies maximum number of days to retain old audit log files based on the timestamp encoded in their filename.
- `audit-log-maxbackup` - specifies maximum number of old audit log files to retain.
- `audit-log-maxsize` - specifies maximum size in megabytes of the audit log file before it gets rotated. Defaults to 100MB.
If an audit log file already exists, Kubernetes appends new audit logs to that file.
Otherwise, Kubernetes creates an audit log file at the location you specified in
`audit-log-path`. If the audit log file exceeds the size you specify in `audit-log-maxsize`,
Kubernetes will rename the current log file by appending the current timestamp on
the file name (before the file extension) and create a new audit log file.
Kubernetes may delete old log files when creating a new log file; you can configure
how many files are retained and how old they can be by specifying the `audit-log-maxbackup`
and `audit-log-maxage` options.
## Advanced audit
Kubernetes 1.7 expands auditing with experimental functionality such as event
filtering and a webhook for integration with external systems. Kubernetes 1.8
upgrades the advanced audit feature to beta, and some backward incompatible changes
have been committed.
## Audit Policy
Audit policy defines rules about what events should be recorded and what data
they should include. When an event is processed, it's compared against the list
of rules in order. The first matching rule sets the [audit level][auditing-level]
of the event. The audit policy object structure is defined in the
[`audit.k8s.io` API group][auditing-api].
`AdvancedAuditing` is customizable in two ways. Policy, which determines what's recorded,
and backends, which persist records. Backend implementations include logs files and
@@ -117,83 +82,7 @@ a valid `apiVersion` and `kind` value will be treated as illegal.
Some example audit policy files:
```yaml
apiVersion: audit.k8s.io/v1beta1 #this is required in Kubernetes 1.8
kind: Policy
rules:
# Don't log watch requests by the "system:kube-proxy" on endpoints or services
- level: None
users: ["system:kube-proxy"]
verbs: ["watch"]
resources:
- group: "" # core API group
resources: ["endpoints", "services"]
# Don't log authenticated requests to certain non-resource URL paths.
- level: None
userGroups: ["system:authenticated"]
nonResourceURLs:
- "/api*" # Wildcard matching.
- "/version"
# Log the request body of configmap changes in kube-system.
- level: Request
resources:
- group: "" # core API group
resources: ["configmaps"]
# This rule only applies to resources in the "kube-system" namespace.
# The empty string "" can be used to select non-namespaced resources.
namespaces: ["kube-system"]
# Log configmap and secret changes in all other namespaces at the Metadata level.
- level: Metadata
resources:
- group: "" # core API group
resources: ["secrets", "configmaps"]
# Log all other resources in core and extensions at the Request level.
- level: Request
resources:
- group: "" # core API group
- group: "extensions" # Version of group should NOT be included.
# A catch-all rule to log all other requests at the Metadata level.
- level: Metadata
```
The next audit policy file shows new features introduced in Kubernetes 1.8:
```yaml
apiVersion: audit.k8s.io/v1beta1
kind: Policy
rules:
# Log pod changes at Request level
- level: Request
resources:
- group: ""
# Resource "pods" no longer matches requests to any subresource of pods,
# This behavior is consistent with the RBAC policy.
resources: ["pods"]
# Log "pods/log", "pods/status" at Metadata level
- level: Metadata
resources:
- group: ""
resources: ["pods/log", "pods/status"]
# Don't log requests to a configmap called "controller-leader"
- level: None
resources:
- group: ""
resources: ["configmaps"]
resourceNames: ["controller-leader"]
# A catch-all rule to log all other requests at the Metadata level.
# For this rule we use "omitStages" to omit events at "ReqeustReceived" stage.
# Events in this stage will not be sent to backend.
- level: Metadata
omitStages:
- "RequestReceived"
```
{% include code.html language="yaml" file="audit-policy.yaml" ghlink="/docs/tasks/debug-application-cluster/audit-policy.yaml" %}
You can use a minimal audit policy file to log all requests at the `Metadata` level:
@@ -208,153 +97,51 @@ rules:
The [audit profile used by GCE][gce-audit-profile] should be used as reference by
admins constructing their own audit profiles.
### Audit backends
## Audit backends
Audit backends implement strategies for emitting events. The [kube-apiserver][kube-apiserver]
provides a logging and webhook backend.
Audit backends implement exporting audit events to an external storage.
[Kube-apiserver][kube-apiserver] out of the box provides two backends:
Each request to the API server can generate multiple events, one when the request is received,
another when the response is sent, and additional events for long running requests (such as
watches). The ID of events will be the same if they were generated from the same request.
- Log backend, which writes events to a disk
- Webhook backend, which sends events to an external API
The event format is defined by the `audit.k8s.io` API group. The `v1alpha1` format of this
API can be found [here][audit-api] with more details about the exact fields captured.
In both cases, audit events structure is defined by the API in the
`audit.k8s.io` API group. The current version of the API is
[`v1beta1`][auditing-api].
#### Log backend
### Log backend
The behavior of the `--audit-log-path` flag changes when enabling the `AdvancedAuditing`
feature flag. All generated events defined by `--audit-policy-file` are recorded in structured
json format:
Log backend writes audit events to a file in JSON format. You can configure
log audit backend using the following [kube-apiserver][kube-apiserver] flags:
```
{"kind":"Event","apiVersion":"audit.k8s.io/v1beta1","metadata":{"creationTimestamp":null},"level":"Metadata","timestamp":"2017-09-05T10:04:55Z","auditID":"77e58433-d345-40ac-b2d8-9866bd355cea","stage":"RequestReceived","requestURI":"/apis/rbac.authorization.k8s.io/v1/namespaces/default/roles","verb":"list","user":{"username":"kubecfg","groups":["system:masters","system:authenticated"]},"sourceIPs":["172.16.116.128"],"objectRef":{"resource":"roles","namespace":"default","apiGroup":"rbac.authorization.k8s.io","apiVersion":"v1"}}
{"kind":"Event","apiVersion":"audit.k8s.io/v1beta1","metadata":{"creationTimestamp":null},"level":"Metadata","timestamp":"2017-09-05T10:04:55Z","auditID":"77e58433-d345-40ac-b2d8-9866bd355cea","stage":"ResponseComplete","requestURI":"/apis/rbac.authorization.k8s.io/v1/namespaces/default/roles","verb":"list","user":{"username":"kubecfg","groups":["system:masters","system:authenticated"]},"sourceIPs":["172.16.116.128"],"objectRef":{"resource":"roles","namespace":"default","apiGroup":"rbac.authorization.k8s.io","apiVersion":"v1"},"responseStatus":{"metadata":{},"code":200}}
```
- `--audit-log-path` specifies the log file path that log backend uses to write
audit events. Not specifying this flag disables log backend. `-` means standard out
- `--audit-log-maxage` defined the maximum number of days to retain old audit log files
- `--audit-log-maxbackup` defines the maximum number of audit log files to retain
- `--audit-log-maxsize` defines the maximum size in megabytes of the audit log file before it gets rotated
In alpha version, objectRef.apiVersion holds both the api group and version.
In beta version these were break out into objectRef.apiGroup and objectRef.apiVersion.
### Webhook backend
Starting from Kubernetes 1.8, structured json format is used for log backend by default.
Use the following option to switch log to legacy format:
Webhook backend sends audit events to a remote API, which is assumed to be the
same API as [kube-apiserver][kube-apiserver] exposes. You can configure webhook
audit backend using the following kube-apiserver flags:
```
--audit-log-format=legacy
```
With legacy format, events are formatted as follows:
```
2017-09-05T06:08:19.885328047-04:00 AUDIT: id="c28a95ad-f9dd-47e1-a617-b6dc152db95f" stage="RequestReceived" ip="172.16.116.128" method="list" user="kubecfg" groups="\"system:masters\",\"system:authenticated\"" as="<self>" asgroups="<lookup>" namespace="default" uri="/apis/rbac.authorization.k8s.io/v1/namespaces/default/roles" response="<deferred>"
2017-09-05T06:08:19.885328047-04:00 AUDIT: id="c28a95ad-f9dd-47e1-a617-b6dc152db95f" stage="ResponseComplete" ip="172.16.116.128" method="list" user="kubecfg" groups="\"system:masters\",\"system:authenticated\"" as="<self>" asgroups="<lookup>" namespace="default" uri="/apis/rbac.authorization.k8s.io/v1/namespaces/default/roles" response="200"
```
Logged events omit the request and response bodies. The `Request` and
`RequestResponse` levels are equivalent to `Metadata` for legacy format. This legacy format
of advanced audit is different from the [Legacy Audit](# Legacy Audit) discussed above, such
as changes to the method values and the introduction of a "stage" for each event.
#### Webhook backend
The audit webhook backend can be used to have [kube-apiserver][kube-apiserver]
send audit events to a remote service. The webhook requires the `AdvancedAuditing`
feature flag and is configured using the following command line flags:
```
--audit-webhook-config-file=/etc/kubernetes/audit-webhook-kubeconfig
--audit-webhook-mode=batch
```
`audit-webhook-mode` controls buffering strategies used by the webhook. Known modes are:
- `batch` - buffer events and asynchronously send the set of events to the external service.
- `blocking` - block API server responses on sending each event to the external service.
- `--audit-webhook-config-file` specifies the path to a file with a webhook
configuration. Webhook configuration is effectively a [kubeconfig][kubeconfig].
- `--audit-webhook-mode` define the buffering strategy, one of the following:
- `batch` - buffer events and asynchronously send the set of events to the external service
This is the default
- `blocking` - block API server responses on sending each event to the external service
The webhook config file uses the kubeconfig format to specify the remote address of
the service and credentials used to connect to it.
```
# clusters refers to the remote service.
clusters:
- name: name-of-remote-audit-service
cluster:
certificate-authority: /path/to/ca.pem # CA for verifying the remote service.
server: https://audit.example.com/audit # URL of remote service to query. Must use 'https'.
## Log Collector Examples
# users refers to the API server's webhook configuration.
users:
- name: name-of-api-server
user:
client-certificate: /path/to/cert.pem # cert for the webhook plugin to use
client-key: /path/to/key.pem # key matching the cert
# kubeconfig files require a context. Provide one for the API server.
current-context: webhook
contexts:
- context:
cluster: name-of-remote-audit-service
user: name-of-api-sever
name: webhook
```
Events are POSTed as a JSON serialized `EventList`. An example payload:
```json
{
"apiVersion": "audit.k8s.io/v1beta1",
"items": [
{
"auditID": "24f30caf-d7d4-45d5-b7bd-e7af300d7886",
"level": "Metadata",
"metadata": {
"creationTimestamp": null
},
"objectRef": {
"apiGroup": "rbac.authorization.k8s.io",
"apiVersion": "v1",
"name": "jane",
"namespace": "default",
"resource": "roles"
},
"requestURI": "/apis/rbac.authorization.k8s.io/v1/namespaces/default/roles/jane",
"responseStatus": {
"code": 200,
"metadata": {}
},
"sourceIPs": [
"172.16.116.128"
],
"stage": "ResponseComplete",
"timestamp": "2017-09-05T10:20:24Z",
"user": {
"groups": [
"system:masters",
"system:authenticated"
],
"username": "kubecfg"
},
"verb": "get"
}
],
"kind": "EventList",
"metadata": {}
}
```
### Audit-Id
Audit-Id is a unique ID for each http request to kube-apiserver. The ID of events will be the
same if they were generated from the same request. Starting from Kubernetes 1.8, if an audit
event is generated for the request, kube-apiserver will respond with an Audit-Id in the HTTP header.
Note that for some special requests like `kubectl exec`, `kubectl attach`, kube-apiserver works
like a proxy, no Audit-Id will be returned even if audit events are recorded.
### Log Collector Examples
#### Use fluentd to collect and distribute audit events from log file
### Use fluentd to collect and distribute audit events from log file
[Fluentd][fluentd] is an open source data collector for unified logging layer.
In this example, we will use fluentd to split audit events by different namespaces.
Note that this example requries json format output support in Kubernetes 1.8.
1. install [fluentd, fluent-plugin-forest and fluent-plugin-rewrite-tag-filter][fluentd_install_doc] in the kube-apiserver node
1. create a config file for fluentd
@@ -372,7 +159,7 @@ Note that this example requries json format output support in Kubernetes 1.8.
time_format %Y-%m-%dT%H:%M:%S.%N%z
tag audit
</source>
<filter audit>
#https://github.com/fluent/fluent-plugin-rewrite-tag-filter/issues/13
type record_transformer
@@ -381,18 +168,18 @@ Note that this example requries json format output support in Kubernetes 1.8.
namespace ${record["objectRef"].nil? ? "none":(record["objectRef"]["namespace"].nil? ? "none":record["objectRef"]["namespace"])}
</record>
</filter>
<match audit>
# route audit according to namespace element in context
@type rewrite_tag_filter
rewriterule1 namespace ^(.+) ${tag}.$1
</match>
<filter audit.**>
@type record_transformer
remove_keys namespace
</filter>
<match audit.**>
@type forest
subtype file
@@ -406,22 +193,22 @@ Note that this example requries json format output support in Kubernetes 1.8.
</template>
</match>
```
1. start fluentd
```shell
$ fluentd -c /etc/fluentd/config -vv
```
1. start kube-apiserver with the following options:
```shell
--audit-policy-file=/etc/kubernetes/audit-policy.yaml --audit-log-path=/var/log/kube-audit --audit-log-format=json
```
1. check audits for different namespaces in /var/log/audit-*.log
#### Use logstash to collect and distribute audit events from webhook backend
### Use logstash to collect and distribute audit events from webhook backend
[Logstash][logstash] is an open source, server-side data processing tool. In this example,
we will use logstash to collect audit events from webhook backend, and save events of
@@ -458,13 +245,13 @@ different users into different files.
}
}
```
1. start logstash
```shell
$ bin/logstash -f /etc/logstash/config --path.settings /etc/logstash/
```
1. create a [kubeconfig file](/docs/tasks/access-application-cluster/authenticate-across-clusters-kubeconfig/) for kube-apiserver webhook audit backend
```shell
@@ -485,22 +272,66 @@ different users into different files.
users: []
EOF
```
1. start kube-apiserver with the following options:
```shell
--audit-policy-file=/etc/kubernetes/audit-policy.yaml --audit-webhook-config-file=/etc/kubernetes/audit-webhook-kubeconfig
```
1. check audits in logstash node's directories /var/log/kube-audit-*/audit
Note that in addition to file output plugin, logstash has a variety of outputs that
let users route data where they want. For example, users can emit audit events to elasticsearch
plugin which supports full-text search and analytics.
[audit-api]: https://github.com/kubernetes/kubernetes/blob/v1.8.0-beta.1/staging/src/k8s.io/apiserver/pkg/apis/audit/v1beta1/types.go
## Legacy Audit
__Note:__ Legacy Audit is deprecated and is disabled by default since Kubernetes 1.8.
To fallback to this legacy audit, disable the advanced auditing feature
using the `AdvancedAuditing` feature gate in [kube-apiserver][kube-apiserver]:
```
--feature-gates=AdvancedAuditing=false
```
In legacy format, each audit log entry contains two lines:
1. The request line containing a unique ID to match the response and request metadata, such as the source IP, requesting user, impersonation information, resource being requested, etc.
2. The response line containing a unique ID matching the request line and the response code.
Example output for `admin` user listing pods in the `default` namespace:
```
2017-03-21T03:57:09.106841886-04:00 AUDIT: id="c939d2a7-1c37-4ef1-b2f7-4ba9b1e43b53" ip="127.0.0.1" method="GET" user="admin" groups="\"system:masters\",\"system:authenticated\"" as="<self>" asgroups="<lookup>" namespace="default" uri="/api/v1/namespaces/default/pods"
2017-03-21T03:57:09.108403639-04:00 AUDIT: id="c939d2a7-1c37-4ef1-b2f7-4ba9b1e43b53" response="200"
```
### Configuration
[Kube-apiserver][kube-apiserver] provides the following options which are responsible
for configuring where and how audit logs are handled:
- `audit-log-path` - enables the audit log pointing to a file where the requests are being logged to, '-' means standard out.
- `audit-log-maxage` - specifies maximum number of days to retain old audit log files based on the timestamp encoded in their filename.
- `audit-log-maxbackup` - specifies maximum number of old audit log files to retain.
- `audit-log-maxsize` - specifies maximum size in megabytes of the audit log file before it gets rotated. Defaults to 100MB.
If an audit log file already exists, Kubernetes appends new audit logs to that file.
Otherwise, Kubernetes creates an audit log file at the location you specified in
`audit-log-path`. If the audit log file exceeds the size you specify in `audit-log-maxsize`,
Kubernetes will rename the current log file by appending the current timestamp on
the file name (before the file extension) and create a new audit log file.
Kubernetes may delete old log files when creating a new log file; you can configure
how many files are retained and how old they can be by specifying the `audit-log-maxbackup`
and `audit-log-maxage` options.
[kube-apiserver]: /docs/admin/kube-apiserver
[gce-audit-profile]: https://github.com/kubernetes/kubernetes/blob/v1.8.0-beta.0/cluster/gce/gci/configure-helper.sh#L532
[auditing-proposal]: https://github.com/kubernetes/community/blob/master/contributors/design-proposals/api-machinery/auditing.md
[auditing-level]: https://github.com/kubernetes/community/blob/master/contributors/design-proposals/api-machinery/auditing.md#levels
[auditing-api]: https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/staging/src/k8s.io/apiserver/pkg/apis/audit/v1beta1/types.go
[gce-audit-profile]: https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/cluster/gce/gci/configure-helper.sh#L532
[kubeconfig]: https://kubernetes.io/docs/tasks/access-application-cluster/configure-access-multiple-clusters/
[fluentd]: http://www.fluentd.org/
[fluentd_install_doc]: http://docs.fluentd.org/v0.12/articles/quickstart#step1-installing-fluentd
[logstash]: https://www.elastic.co/products/logstash
docs/tasks/run-application/configure-pdb.md
+5
View File
@@ -88,6 +88,11 @@ of the evicted pod. `minAvailable` can be either an absolute number or a percent
of the number of pods from that set that can be unavailable after the eviction.
It can be either an absolute number or a percentage.
**Note:** For versions 1.8 and earlier: When creating a `PodDisruptionBudget`
object using the `kubectl` command line tool, the `minAvailable` field has a
default value of 1 if neither `minAvailable` nor `maxAvailable` is specified.
{: .note}
You can specify only one of `maxUnavailable` and `minAvailable` in a single `PodDisruptionBudget`.
`maxUnavailable` can only be used to control the eviction of pods
that have an associated controller managing them. In the examples below, "desired replicas"
docs/tasks/run-application/mysql-statefulset.yaml
+1 -1
View File
@@ -1,4 +1,4 @@
apiVersion: apps/v1beta2 # for versions before 1.8.0 use apps/v1beta1
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: mysql