Merge pull request #21166 from sftim/20200524_revise_dns_debugging_task

Revise cluster DNS task pages
This commit is contained in:
Kubernetes Prow Robot
2020-06-12 22:55:56 -07:00
committed by GitHub
2 changed files with 150 additions and 279 deletions
@@ -4,49 +4,53 @@ reviewers:
- zihongz - zihongz
title: Customizing DNS Service title: Customizing DNS Service
content_type: task content_type: task
min-kubernetes-server-version: v1.12
--- ---
<!-- overview --> <!-- overview -->
This page explains how to configure your DNS Pod and customize the This page explains how to configure your DNS
DNS resolution process. In Kubernetes version 1.11 and later, CoreDNS is at GA {{< glossary_tooltip text="Pod(s)" term_id="pod" >}} and customize the
and is installed by default with kubeadm. See [CoreDNS ConfigMap options](#coredns-configmap-options) DNS resolution process in your cluster.
and [Using CoreDNS for Service Discovery](/docs/tasks/administer-cluster/coredns/).
## {{% heading "prerequisites" %}} ## {{% heading "prerequisites" %}}
* {{< include "task-tutorial-prereqs.md" >}} {{< version-check >}} {{< include "task-tutorial-prereqs.md" >}}
* Kubernetes version 1.6 or later. To work with CoreDNS, version 1.9 or later.
* The appropriate add-on: kube-dns or CoreDNS. To install with kubeadm,
see [the kubeadm reference documentation](/docs/reference/setup-tools/kubeadm/kubeadm-alpha/#cmd-phase-addon).
Your cluster must be running the CoreDNS add-on.
[Migrating to CoreDNS](https://kubernetes.io/docs/tasks/administer-cluster/coredns/#migrating-to-coredns)
explains how to use `kubeadm` to migrate from `kube-dns`.
{{% version-check %}}
<!-- steps --> <!-- steps -->
## Introduction ## Introduction
DNS is a built-in Kubernetes service launched automatically DNS is a built-in Kubernetes service launched automatically
using the addon manager using the _addon manager_
[cluster add-on](http://releases.k8s.io/{{< param "githubbranch" >}}/cluster/addons/README.md). [cluster add-on](http://releases.k8s.io/{{< param "githubbranch" >}}/cluster/addons/README.md).
As of Kubernetes v1.12, CoreDNS is the recommended DNS Server, replacing kube-dns. However, kube-dns may still be installed by As of Kubernetes v1.12, CoreDNS is the recommended DNS Server, replacing kube-dns. If your cluster
default with certain Kubernetes installer tools. Refer to the documentation provided by your installer to know which DNS server is installed by default. originally used kube-dns, you may still have `kube-dns` deployed rather than CoreDNS.
{{< note >}}
Both the CoreDNS and kube-dns Service are named `kube-dns` in the `metadata.name` field.
This is so that there is greater interoperability with workloads that relied on the legacy `kube-dns` Service name to resolve addresses internal to the cluster. Using a Service named `kube-dns` abstracts away the implementation detail of which DNS provider is running behind that common name.
{{< /note >}}
The CoreDNS Deployment is exposed as a Kubernetes Service with a static IP. If you are running CoreDNS as a Deployment, it will typically be exposed as a Kubernetes Service with a static IP address.
Both the CoreDNS and kube-dns Service are named `kube-dns` in the `metadata.name` field. This is done so that there is greater interoperability with workloads that relied on the legacy `kube-dns` Service name to resolve addresses internal to the cluster. It abstracts away the implementation detail of which DNS provider is running behind that common endpoint. The kubelet passes DNS resolver information to each container with the `--cluster-dns=<dns-service-ip>` flag.
The kubelet passes DNS to each container with the `--cluster-dns=<dns-service-ip>` flag.
DNS names also need domains. You configure the local domain in the kubelet DNS names also need domains. You configure the local domain in the kubelet
with the flag `--cluster-domain=<default-local-domain>`. with the flag `--cluster-domain=<default-local-domain>`.
The DNS server supports forward lookups (A records), port lookups (SRV records), reverse IP address lookups (PTR records), The DNS server supports forward lookups (A and AAAA records), port lookups (SRV records), reverse IP address lookups (PTR records),
and more. For more information see [DNS for Services and Pods] (/docs/concepts/services-networking/dns-pod-service/). and more. For more information, see [DNS for Services and Pods](/docs/concepts/services-networking/dns-pod-service/).
If a Pod's `dnsPolicy` is set to "`default`", it inherits the name resolution If a Pod's `dnsPolicy` is set to `default`, it inherits the name resolution
configuration from the node that the Pod runs on. The Pod's DNS resolution configuration from the node that the Pod runs on. The Pod's DNS resolution
should behave the same as the node. should behave the same as the node.
But see [Known issues](/docs/tasks/administer-cluster/dns-debugging-resolution/#known-issues). But see [Known issues](/docs/tasks/debug-application-cluster/dns-debugging-resolution/#known-issues).
If you don't want this, or if you want a different DNS config for pods, you can If you don't want this, or if you want a different DNS config for pods, you can
use the kubelet's `--resolv-conf` flag. Set this flag to "" to prevent Pods from use the kubelet's `--resolv-conf` flag. Set this flag to "" to prevent Pods from
@@ -61,9 +65,11 @@ CoreDNS is a general-purpose authoritative DNS server that can serve as cluster
CoreDNS is a DNS server that is modular and pluggable, and each plugin adds new functionality to CoreDNS. CoreDNS is a DNS server that is modular and pluggable, and each plugin adds new functionality to CoreDNS.
This can be configured by maintaining a [Corefile](https://coredns.io/2017/07/23/corefile-explained/), which is the CoreDNS This can be configured by maintaining a [Corefile](https://coredns.io/2017/07/23/corefile-explained/), which is the CoreDNS
configuration file. A cluster administrator can modify the ConfigMap for the CoreDNS Corefile to change how service discovery works. configuration file. As a cluster administrator, you can modify the
{{< glossary_tooltip text="ConfigMap" term_id="configmap" >}} for the CoreDNS Corefile to change how DNS service discovery
behaves for that cluster.
In Kubernetes, CoreDNS is installed with the following default Corefile configuration. In Kubernetes, CoreDNS is installed with the following default Corefile configuration:
```yaml ```yaml
apiVersion: v1 apiVersion: v1
@@ -92,16 +98,15 @@ data:
loadbalance loadbalance
} }
``` ```
The Corefile configuration includes the following [plugins](https://coredns.io/plugins/) of CoreDNS: The Corefile configuration includes the following [plugins](https://coredns.io/plugins/) of CoreDNS:
* [errors](https://coredns.io/plugins/errors/): Errors are logged to stdout. * [errors](https://coredns.io/plugins/errors/): Errors are logged to stdout.
* [health](https://coredns.io/plugins/health/): Health of CoreDNS is reported to http://localhost:8080/health. In this extended syntax `lameduck` will make the process unhealthy then wait for 5 seconds before the process is shut down. * [health](https://coredns.io/plugins/health/): Health of CoreDNS is reported to `http://localhost:8080/health`. In this extended syntax `lameduck` will make the process unhealthy then wait for 5 seconds before the process is shut down.
* [ready](https://coredns.io/plugins/ready/): An HTTP endpoint on port 8181 will return 200 OK, when all plugins that are able to signal readiness have done so. * [ready](https://coredns.io/plugins/ready/): An HTTP endpoint on port 8181 will return 200 OK, when all plugins that are able to signal readiness have done so.
* [kubernetes](https://coredns.io/plugins/kubernetes/): CoreDNS will reply to DNS queries based on IP of the services and pods of Kubernetes. You can find more details [here](https://coredns.io/plugins/kubernetes/). `ttl` allows you to set a custom TTL for responses. The default is 5 seconds. The minimum TTL allowed is 0 seconds, and the maximum is capped at 3600 seconds. Setting TTL to 0 will prevent records from being cached. * [kubernetes](https://coredns.io/plugins/kubernetes/): CoreDNS will reply to DNS queries based on IP of the services and pods of Kubernetes. You can find [more details](https://coredns.io/plugins/kubernetes/) about that plugin on the CoreDNS website. `ttl` allows you to set a custom TTL for responses. The default is 5 seconds. The minimum TTL allowed is 0 seconds, and the maximum is capped at 3600 seconds. Setting TTL to 0 will prevent records from being cached.
The `pods insecure` option is provided for backward compatibility with _kube-dns_. You can use the `pods verified` option, which returns an A record only if there exists a pod in same namespace with matching IP. The `pods disabled` option can be used if you don't use pod records.
> The `pods insecure` option is provided for backward compatibility with kube-dns. You can use the `pods verified` option, which returns an A record only if there exists a pod in same namespace with matching IP. The `pods disabled` option can be used if you don't use pod records. * [prometheus](https://coredns.io/plugins/metrics/): Metrics of CoreDNS are available at `http://localhost:9153/metrics` in [Prometheus](https://prometheus.io/) format (also known as OpenMetrics).
* [prometheus](https://coredns.io/plugins/metrics/): Metrics of CoreDNS are available at http://localhost:9153/metrics in [Prometheus](https://prometheus.io/) format.
* [forward](https://coredns.io/plugins/forward/): Any queries that are not within the cluster domain of Kubernetes will be forwarded to predefined resolvers (/etc/resolv.conf). * [forward](https://coredns.io/plugins/forward/): Any queries that are not within the cluster domain of Kubernetes will be forwarded to predefined resolvers (/etc/resolv.conf).
* [cache](https://coredns.io/plugins/cache/): This enables a frontend cache. * [cache](https://coredns.io/plugins/cache/): This enables a frontend cache.
* [loop](https://coredns.io/plugins/loop/): Detects simple forwarding loops and halts the CoreDNS process if a loop is found. * [loop](https://coredns.io/plugins/loop/): Detects simple forwarding loops and halts the CoreDNS process if a loop is found.
@@ -161,149 +166,14 @@ data:
forward . 10.150.0.1 forward . 10.150.0.1
} }
``` ```
In Kubernetes version 1.10 and later, kubeadm supports automatic translation of the CoreDNS ConfigMap from the kube-dns ConfigMap.
***Note: While kube-dns accepts an FQDN for stubdomain and nameserver (eg: ns.foo.com), CoreDNS does not support this feature.
During translation, all FQDN nameservers will be omitted from the CoreDNS config.***
## Kube-dns
Kube-dns is now available as an optional DNS server since CoreDNS is now the default.
The running DNS Pod holds 3 containers:
- "`kubedns`": watches the Kubernetes master for changes
in Services and Endpoints, and maintains in-memory lookup structures to serve
DNS requests.
- "`dnsmasq`": adds DNS caching to improve performance.
- "`sidecar`": provides a single health check endpoint
to perform healthchecks for `dnsmasq` and `kubedns`.
### Configure stub-domain and upstream DNS servers
Cluster administrators can specify custom stub domains and upstream nameservers
by providing a ConfigMap for kube-dns (`kube-system:kube-dns`).
For example, the following ConfigMap sets up a DNS configuration with a single stub domain and two
upstream nameservers:
```yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: kube-dns
namespace: kube-system
data:
stubDomains: |
{"acme.local": ["1.2.3.4"]}
upstreamNameservers: |
["8.8.8.8", "8.8.4.4"]
```
DNS requests with the “.acme.local” suffix
are forwarded to a DNS listening at 1.2.3.4. Google Public DNS
serves the upstream queries.
The table below describes how queries with certain domain names map to
their destination DNS servers:
| Domain name | Server answering the query |
| ----------- | -------------------------- |
| kubernetes.default.svc.cluster.local| kube-dns |
| foo.acme.local| custom DNS (1.2.3.4) |
| widget.com | upstream DNS (one of 8.8.8.8, 8.8.4.4) |
See [ConfigMap options](#configmap-options) for
details about the configuration option format.
<!-- discussion -->
#### Effects on Pods
Custom upstream nameservers and stub domains do not affect Pods with a
`dnsPolicy` set to "`Default`" or "`None`".
If a Pod's `dnsPolicy` is set to "`ClusterFirst`", its name resolution is
handled differently, depending on whether stub-domain and upstream DNS servers
are configured.
**Without custom configurations**: Any query that does not match the configured
cluster domain suffix, such as "www.kubernetes.io", is forwarded to the upstream
nameserver inherited from the node.
**With custom configurations**: If stub domains and upstream DNS servers are
configured,
DNS queries are routed according to the following flow:
1. The query is first sent to the DNS caching layer in kube-dns.
1. From the caching layer, the suffix of the request is examined and then
forwarded to the appropriate DNS, based on the following cases:
* *Names with the cluster suffix*, for example ".cluster.local":
The request is sent to kube-dns.
* *Names with the stub domain suffix*, for example ".acme.local":
The request is sent to the configured custom DNS resolver, listening for example at 1.2.3.4.
* *Names without a matching suffix*, for example "widget.com":
The request is forwarded to the upstream DNS,
for example Google public DNS servers at 8.8.8.8 and 8.8.4.4.
![DNS lookup flow](/docs/tasks/administer-cluster/dns-custom-nameservers/dns.png)
### ConfigMap options
Options for the kube-dns `kube-system:kube-dns` ConfigMap:
| Field | Format | Description |
| ----- | ------ | ----------- |
| `stubDomains` (optional) | A JSON map using a DNS suffix key such as “acme.local”, and a value consisting of a JSON array of DNS IPs. | The target nameserver can itself be a Kubernetes Service. For instance, you can run your own copy of dnsmasq to export custom DNS names into the ClusterDNS namespace. |
| `upstreamNameservers` (optional) | A JSON array of DNS IPs. | If specified, the values replace the nameservers taken by default from the nodes `/etc/resolv.conf`. Limits: a maximum of three upstream nameservers can be specified. |
#### Examples
##### Example: Stub domain
In this example, the user has a Consul DNS service discovery system they want to
integrate with kube-dns. The consul domain server is located at 10.150.0.1, and
all consul names have the suffix `.consul.local`. To configure Kubernetes, the
cluster administrator creates the following ConfigMap:
```yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: kube-dns
namespace: kube-system
data:
stubDomains: |
{"consul.local": ["10.150.0.1"]}
```
Note that the cluster administrator does not want to override the nodes
upstream nameservers, so they did not specify the optional
`upstreamNameservers` field.
##### Example: Upstream nameserver
In this example the cluster administrator wants to explicitly force all
non-cluster DNS lookups to go through their own nameserver at 172.16.0.1.
In this case, they create a ConfigMap with the
`upstreamNameservers` field specifying the desired nameserver:
```yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: kube-dns
namespace: kube-system
data:
upstreamNameservers: |
["172.16.0.1"]
```
The `kubeadm` tool supports automatic translation from the kube-dns ConfigMap
to the equivalent CoreDNS ConfigMap.
{{< note >}}
While kube-dns accepts an FQDN for stubdomain and nameserver (eg: ns.foo.com), CoreDNS does not support this feature.
During translation, all FQDN nameservers will be omitted from the CoreDNS config.
{{< /note >}}
## CoreDNS configuration equivalent to kube-dns ## CoreDNS configuration equivalent to kube-dns
@@ -313,7 +183,7 @@ Similarly, the `Federations` plugin in kube-dns translates to the `federation` p
### Example ### Example
This example ConfigMap for kubedns specifies federations, stubdomains and upstreamnameservers: This example ConfigMap for kube-dns specifies federations, stubdomains and upstreamnameservers:
```yaml ```yaml
apiVersion: v1 apiVersion: v1
@@ -330,59 +200,65 @@ kind: ConfigMap
The equivalent configuration in CoreDNS creates a Corefile: The equivalent configuration in CoreDNS creates a Corefile:
* For federations: * For federations:
```yaml ```
federation cluster.local { federation cluster.local {
foo foo.feddomain.com foo foo.feddomain.com
} }
``` ```
* For stubDomains: * For stubDomains:
```yaml ```yaml
abc.com:53 { abc.com:53 {
errors errors
cache 30 cache 30
forward . 1.2.3.4 forward . 1.2.3.4
} }
my.cluster.local:53 { my.cluster.local:53 {
errors errors
cache 30 cache 30
forward . 2.3.4.5 forward . 2.3.4.5
} }
``` ```
The complete Corefile with the default plugins: The complete Corefile with the default plugins:
```yaml ```
.:53 { .:53 {
errors errors
health health
kubernetes cluster.local in-addr.arpa ip6.arpa { kubernetes cluster.local in-addr.arpa ip6.arpa {
pods insecure pods insecure
fallthrough in-addr.arpa ip6.arpa fallthrough in-addr.arpa ip6.arpa
}
federation cluster.local {
foo foo.feddomain.com
}
prometheus :9153
forward . 8.8.8.8 8.8.4.4
cache 30
} }
abc.com:53 { federation cluster.local {
errors foo foo.feddomain.com
cache 30
forward . 1.2.3.4
}
my.cluster.local:53 {
errors
cache 30
forward . 2.3.4.5
} }
prometheus :9153
forward . 8.8.8.8 8.8.4.4
cache 30
}
abc.com:53 {
errors
cache 30
forward . 1.2.3.4
}
my.cluster.local:53 {
errors
cache 30
forward . 2.3.4.5
}
``` ```
## Migration to CoreDNS ## Migration to CoreDNS
To migrate from kube-dns to CoreDNS, [a detailed blog](https://coredns.io/2018/05/21/migration-from-kube-dns-to-coredns/) is available to help users adapt CoreDNS in place of kube-dns. To migrate from kube-dns to CoreDNS, a detailed
A cluster administrator can also migrate using [the deploy script](https://github.com/coredns/deployment/blob/master/kubernetes/deploy.sh). [blog article](https://coredns.io/2018/05/21/migration-from-kube-dns-to-coredns/)
is available to help users adapt CoreDNS in place of kube-dns.
## What's next You can also migrate using the offical CoreDNS
- [Debugging DNS Resolution](/docs/tasks/administer-cluster/dns-debugging-resolution/). [deploy script](https://github.com/coredns/deployment/blob/master/kubernetes/deploy.sh).
## {{% heading "whatsnext" %}}
- Read [Debugging DNS Resolution](/docs/tasks/debug-application-cluster/dns-debugging-resolution/)
@@ -4,43 +4,51 @@ reviewers:
- zihongz - zihongz
title: Debugging DNS Resolution title: Debugging DNS Resolution
content_type: task content_type: task
min-kubernetes-server-version: v1.6
--- ---
<!-- overview --> <!-- overview -->
This page provides hints on diagnosing DNS problems. This page provides hints on diagnosing DNS problems.
## {{% heading "prerequisites" %}} ## {{% heading "prerequisites" %}}
* {{< include "task-tutorial-prereqs.md" >}} {{< version-check >}} {{< include "task-tutorial-prereqs.md" >}}
* Kubernetes version 1.6 and above. Your cluster must be configured to use the CoreDNS
* The cluster must be configured to use the `coredns` (or `kube-dns`) addons. {{< glossary_tooltip text="addon" term_id="addons" >}} or its precursor,
kube-dns.
{{% version-check %}}
<!-- steps --> <!-- steps -->
### Create a simple Pod to use as a test environment ### Create a simple Pod to use as a test environment
Create a file named dnsutils.yaml with the following contents:
{{< codenew file="admin/dns/dnsutils.yaml" >}} {{< codenew file="admin/dns/dnsutils.yaml" >}}
Then create a pod using this file and verify its status: Use that manifest to create a Pod:
```shell ```shell
kubectl apply -f https://k8s.io/examples/admin/dns/dnsutils.yaml kubectl apply -f https://k8s.io/examples/admin/dns/dnsutils.yaml
```
```
pod/dnsutils created pod/dnsutils created
```
…and verify its status:
```shell
kubectl get pods dnsutils kubectl get pods dnsutils
```
```
NAME READY STATUS RESTARTS AGE NAME READY STATUS RESTARTS AGE
dnsutils 1/1 Running 0 <some-time> dnsutils 1/1 Running 0 <some-time>
``` ```
Once that pod is running, you can exec `nslookup` in that environment. Once that Pod is running, you can exec `nslookup` in that environment.
If you see something like the following, DNS is working correctly. If you see something like the following, DNS is working correctly.
```shell ```shell
kubectl exec -ti dnsutils -- nslookup kubernetes.default kubectl exec -i -t dnsutils -- nslookup kubernetes.default
```
```
Server: 10.0.0.10 Server: 10.0.0.10
Address 1: 10.0.0.10 Address 1: 10.0.0.10
@@ -69,11 +77,13 @@ nameserver 10.0.0.10
options ndots:5 options ndots:5
``` ```
Errors such as the following indicate a problem with the coredns/kube-dns add-on or Errors such as the following indicate a problem with the CoreDNS (or kube-dns)
associated Services: add-on or with associated Services:
```shell
kubectl exec -i -t dnsutils -- nslookup kubernetes.default
```
``` ```
kubectl exec -ti dnsutils -- nslookup kubernetes.default
Server: 10.0.0.10 Server: 10.0.0.10
Address 1: 10.0.0.10 Address 1: 10.0.0.10
@@ -82,8 +92,10 @@ nslookup: can't resolve 'kubernetes.default'
or or
```shell
kubectl exec -i -t dnsutils -- nslookup kubernetes.default
```
``` ```
kubectl exec -ti dnsutils -- nslookup kubernetes.default
Server: 10.0.0.10 Server: 10.0.0.10
Address 1: 10.0.0.10 kube-dns.kube-system.svc.cluster.local Address 1: 10.0.0.10 kube-dns.kube-system.svc.cluster.local
@@ -94,9 +106,10 @@ nslookup: can't resolve 'kubernetes.default'
Use the `kubectl get pods` command to verify that the DNS pod is running. Use the `kubectl get pods` command to verify that the DNS pod is running.
For CoreDNS:
```shell ```shell
kubectl get pods --namespace=kube-system -l k8s-app=kube-dns kubectl get pods --namespace=kube-system -l k8s-app=kube-dns
```
```
NAME READY STATUS RESTARTS AGE NAME READY STATUS RESTARTS AGE
... ...
coredns-7b96bf9f76-5hsxb 1/1 Running 0 1h coredns-7b96bf9f76-5hsxb 1/1 Running 0 1h
@@ -104,26 +117,22 @@ coredns-7b96bf9f76-mvmmt 1/1 Running 0 1h
... ...
``` ```
Or for kube-dns: {{< note >}}
```shell The value for label `k8s-app` is `kube-dns` for both CoreDNS and kube-dns deployments.
kubectl get pods --namespace=kube-system -l k8s-app=kube-dns {{< /note >}}
NAME READY STATUS RESTARTS AGE
...
kube-dns-v19-ezo1y 3/3 Running 0 1h
...
```
If you see that no pod is running or that the pod has failed/completed, the DNS
add-on may not be deployed by default in your current environment and you will
have to deploy it manually.
### Check for Errors in the DNS pod If you see that no CoreDNS Pod is running or that the Pod has failed/completed,
the DNS add-on may not be deployed by default in your current environment and you
will have to deploy it manually.
Use `kubectl logs` command to see logs for the DNS containers. ### Check for errors in the DNS pod
Use the `kubectl logs` command to see logs for the DNS containers.
For CoreDNS: For CoreDNS:
```shell ```shell
for p in $(kubectl get pods --namespace=kube-system -l k8s-app=kube-dns -o name); do kubectl logs --namespace=kube-system $p; done kubectl logs --namespace=kube-system -l k8s-app=kube-dns
``` ```
Here is an example of a healthy CoreDNS log: Here is an example of a healthy CoreDNS log:
@@ -137,21 +146,7 @@ linux/amd64, go1.10.3, 2e322f6
2018/08/15 14:37:17 [INFO] plugin/reload: Running configuration MD5 = 24e6c59e83ce706f07bcc82c31b1ea1c 2018/08/15 14:37:17 [INFO] plugin/reload: Running configuration MD5 = 24e6c59e83ce706f07bcc82c31b1ea1c
``` ```
See if there are any suspicious or unexpected messages in the logs.
For kube-dns, there are 3 sets of logs:
```shell
kubectl logs --namespace=kube-system $(kubectl get pods --namespace=kube-system -l k8s-app=kube-dns -o name | head -1) -c kubedns
kubectl logs --namespace=kube-system $(kubectl get pods --namespace=kube-system -l k8s-app=kube-dns -o name | head -1) -c dnsmasq
kubectl logs --namespace=kube-system $(kubectl get pods --namespace=kube-system -l k8s-app=kube-dns -o name | head -1) -c sidecar
```
See if there are any suspicious error messages in the logs. In kube-dns, a '`W`', '`E`' or '`F`' at the beginning
of a line represents a Warning, Error or Failure. Please search for entries that have these
as the logging level and use
[kubernetes issues](https://github.com/kubernetes/kubernetes/issues)
to report unexpected errors.
### Is DNS service up? ### Is DNS service up?
@@ -159,17 +154,23 @@ Verify that the DNS service is up by using the `kubectl get service` command.
```shell ```shell
kubectl get svc --namespace=kube-system kubectl get svc --namespace=kube-system
```
```
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
... ...
kube-dns ClusterIP 10.0.0.10 <none> 53/UDP,53/TCP 1h kube-dns ClusterIP 10.0.0.10 <none> 53/UDP,53/TCP 1h
... ...
``` ```
{{< note >}}
The service name is `kube-dns` for both CoreDNS and kube-dns deployments.
{{< /note >}}
Note that the service name will be "kube-dns" for both CoreDNS and kube-dns deployments.
If you have created the service or in the case it should be created by default
If you have created the Service or in the case it should be created by default
but it does not appear, see but it does not appear, see
[debugging services](/docs/tasks/debug-application-cluster/debug-service/) for [debugging Services](/docs/tasks/debug-application-cluster/debug-service/) for
more information. more information.
### Are DNS endpoints exposed? ### Are DNS endpoints exposed?
@@ -178,31 +179,32 @@ You can verify that DNS endpoints are exposed by using the `kubectl get endpoint
command. command.
```shell ```shell
kubectl get ep kube-dns --namespace=kube-system kubectl get endpoints kube-dns --namespace=kube-system
```
```
NAME ENDPOINTS AGE NAME ENDPOINTS AGE
kube-dns 10.180.3.17:53,10.180.3.17:53 1h kube-dns 10.180.3.17:53,10.180.3.17:53 1h
``` ```
If you do not see the endpoints, see endpoints section in the If you do not see the endpoints, see the endpoints section in the
[debugging services](/docs/tasks/debug-application-cluster/debug-service/) documentation. [debugging Services](/docs/tasks/debug-application-cluster/debug-service/) documentation.
For additional Kubernetes DNS examples, see the For additional Kubernetes DNS examples, see the
[cluster-dns examples](https://github.com/kubernetes/examples/tree/master/staging/cluster-dns) [cluster-dns examples](https://github.com/kubernetes/examples/tree/master/staging/cluster-dns)
in the Kubernetes GitHub repository. in the Kubernetes GitHub repository.
### Are DNS queries being received/processed? ### Are DNS queries being received/processed?
You can verify if queries are being received by CoreDNS by adding the `log` plugin to the CoreDNS configuration (aka Corefile). You can verify if queries are being received by CoreDNS by adding the `log` plugin to the CoreDNS configuration (aka Corefile).
The CoreDNS Corefile is held in a ConfigMap named `coredns`. To edit it, use the command ... The CoreDNS Corefile is held in a {{< glossary_tooltip text="ConfigMap" term_id="configmap" >}} named `coredns`. To edit it, use the command:
``` ```
kubectl -n kube-system edit configmap coredns kubectl -n kube-system edit configmap coredns
``` ```
Then add `log` in the Corefile section per the example below. Then add `log` in the Corefile section per the example below:
``` ```yaml
apiVersion: v1 apiVersion: v1
kind: ConfigMap kind: ConfigMap
metadata: metadata:
@@ -226,14 +228,13 @@ data:
reload reload
loadbalance loadbalance
} }
``` ```
After saving the changes, it may take up to minute or two for Kubernetes to propagate these changes to the CoreDNS pods. After saving the changes, it may take up to minute or two for Kubernetes to propagate these changes to the CoreDNS pods.
Next, make some queries and view the logs per the sections above in this document. If CoreDNS pods are receiving the queries, you should see them in the logs. Next, make some queries and view the logs per the sections above in this document. If CoreDNS pods are receiving the queries, you should see them in the logs.
Here is an example of a query in the log. Here is an example of a query in the log:
``` ```
.:53 .:53
@@ -244,7 +245,6 @@ linux/amd64, go1.10.3, 2e322f6
2018/09/07 15:29:04 [INFO] plugin/reload: Running configuration MD5 = 162475cdf272d8aa601e6fe67a6ad42f 2018/09/07 15:29:04 [INFO] plugin/reload: Running configuration MD5 = 162475cdf272d8aa601e6fe67a6ad42f
2018/09/07 15:29:04 [INFO] Reloading complete 2018/09/07 15:29:04 [INFO] Reloading complete
172.17.0.18:41675 - [07/Sep/2018:15:29:11 +0000] 59925 "A IN kubernetes.default.svc.cluster.local. udp 54 false 512" NOERROR qr,aa,rd,ra 106 0.000066649s 172.17.0.18:41675 - [07/Sep/2018:15:29:11 +0000] 59925 "A IN kubernetes.default.svc.cluster.local. udp 54 false 512" NOERROR qr,aa,rd,ra 106 0.000066649s
``` ```
## Known issues ## Known issues
@@ -253,7 +253,7 @@ Some Linux distributions (e.g. Ubuntu) use a local DNS resolver by default (syst
Systemd-resolved moves and replaces `/etc/resolv.conf` with a stub file that can cause a fatal forwarding Systemd-resolved moves and replaces `/etc/resolv.conf` with a stub file that can cause a fatal forwarding
loop when resolving names in upstream servers. This can be fixed manually by using kubelet's `--resolv-conf` flag loop when resolving names in upstream servers. This can be fixed manually by using kubelet's `--resolv-conf` flag
to point to the correct `resolv.conf` (With `systemd-resolved`, this is `/run/systemd/resolve/resolv.conf`). to point to the correct `resolv.conf` (With `systemd-resolved`, this is `/run/systemd/resolve/resolv.conf`).
kubeadm (>= 1.11) automatically detects `systemd-resolved`, and adjusts the kubelet flags accordingly. kubeadm automatically detects `systemd-resolved`, and adjusts the kubelet flags accordingly.
Kubernetes installs do not configure the nodes' `resolv.conf` files to use the Kubernetes installs do not configure the nodes' `resolv.conf` files to use the
cluster DNS by default, because that process is inherently distribution-specific. cluster DNS by default, because that process is inherently distribution-specific.
@@ -263,16 +263,11 @@ Linux's libc (a.k.a. glibc) has a limit for the DNS `nameserver` records to 3 by
If you are using Alpine version 3.3 or earlier as your base image, DNS may not If you are using Alpine version 3.3 or earlier as your base image, DNS may not
work properly due to a known issue with Alpine. work properly due to a known issue with Alpine.
Check [here](https://github.com/kubernetes/kubernetes/issues/30215) Kubernetes [issue 30215](https://github.com/kubernetes/kubernetes/issues/30215)
for more information. details more information on this.
## References
- [DNS for Services and Pods](/docs/concepts/services-networking/dns-pod-service/)
- [Docs for the kube-dns DNS cluster addon](http://releases.k8s.io/{{< param "githubbranch" >}}/cluster/addons/dns/kube-dns/README.md)
## What's next
- [Autoscaling the DNS Service in a Cluster](/docs/tasks/administer-cluster/dns-horizontal-autoscaling/).
## {{% heading "whatsnext" %}}
- See [Autoscaling the DNS Service in a Cluster](/docs/tasks/administer-cluster/dns-horizontal-autoscaling/).
- Read [DNS for Services and Pods](/docs/concepts/services-networking/dns-pod-service/)