[zh] sync docs/tasks/debug

This commit is contained in:
howieyuen
2022-05-01 17:18:08 +08:00
parent 8d6a818a23
commit 08bacd030c
20 changed files with 1063 additions and 3977 deletions
+220
View File
@@ -0,0 +1,220 @@
---
title: 监控、日志和调试
description: 设置监控和日志记录以对集群进行故障排除或调试容器化应用程序。
weight: 20
content_type: concept
no_list: true
---
<!--
title: "Monitoring, Logging, and Debugging"
description: Set up monitoring and logging to troubleshoot a cluster, or debug a containerized application.
weight: 20
reviewers:
- brendandburns
- davidopp
content_type: concept
no_list: true
-->
<!-- overview -->
<!--
Sometimes things go wrong. This guide is aimed at making them right. It has
two sections:
-->
有时候事情会出错。本指南旨在解决这些问题。它包含两个部分:
<!--
* [Debugging your application](/docs/tasks/debug/debug-application/) - Useful
for users who are deploying code into Kubernetes and wondering why it is not working.
* [Debugging your cluster](/docs/tasks/debug/debug-cluster/) - Useful
for cluster administrators and people whose Kubernetes cluster is unhappy.
-->
* [应用排错](/zh/docs/tasks/debug/debug-application/) -
针对部署代码到 Kubernetes 并想知道代码为什么不能正常运行的用户。
* [集群排错](/zh/docs/tasks/debug/debug-cluster/) -
针对集群管理员以及 Kubernetes 集群表现异常的用户。
<!--
You should also check the known issues for the [release](https://github.com/kubernetes/kubernetes/releases)
you're using.
-->
你也应该查看所用[发行版本](https://github.com/kubernetes/kubernetes/releases)的已知问题。
<!-- body -->
<!--
## Getting help
If your problem isn't answered by any of the guides above, there are variety of
ways for you to get help from the Kubernetes team.
-->
## 获取帮助 {#getting-help}
如果你的问题在上述指南中没有得到答案,你还有另外几种方式从 Kubernetes 团队获得帮助。
<!--
### Questions
The documentation on this site has been structured to provide answers to a wide
range of questions. [Concepts](/docs/concepts/) explain the Kubernetes
architecture and how each component works, while [Setup](/docs/setup/) provides
practical instructions for getting started. [Tasks](/docs/tasks/) show how to
accomplish commonly used tasks, and [Tutorials](/docs/tutorials/) are more
comprehensive walkthroughs of real-world, industry-specific, or end-to-end
development scenarios. The [Reference](/docs/reference/) section provides
detailed documentation on the [Kubernetes API](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/)
and command-line interfaces (CLIs), such as [`kubectl`](/docs/reference/kubectl/).
-->
### 问题 {#questions}
本网站上的文档针对回答各类问题进行了结构化组织和分类。
[概念](/zh/docs/concepts/)部分解释 Kubernetes 体系结构以及每个组件的工作方式,
[安装](/zh/docs/setup/)部分提供了安装的实用说明。
[任务](/zh/docs/tasks/)部分展示了如何完成常用任务,
[教程](/zh/docs/tutorials/)部分则提供对现实世界、特定行业或端到端开发场景的更全面的演练。
[参考](/zh/docs/reference/)部分提供了详细的
[Kubernetes API](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/) 文档
和命令行 (CLI) 接口的文档,例如[`kubectl`](/zh/docs/reference/kubectl/)。
<!--
## Help! My question isn't covered! I need help now!
-->
## 求救!我的问题还没有解决!我现在需要帮助!
<!--
### Stack Overflow
Someone else from the community may have already asked a similar question or may
be able to help with your problem. The Kubernetes team will also monitor
[posts tagged Kubernetes](https://stackoverflow.com/questions/tagged/kubernetes).
If there aren't any existing questions that help, **please [ensure that your question is on-topic on Stack Overflow](https://stackoverflow.com/help/on-topic)
and that you read through the guidance on [how to ask a new question](https://stackoverflow.com/help/how-to-ask)**,
before [asking a new one](https://stackoverflow.com/questions/ask?tags=kubernetes)!
-->
### Stack Overflow {#stack-overflow}
社区中的其他人可能已经问过和你类似的问题,也可能能够帮助解决你的问题。
Kubernetes 团队还会监视[带有 Kubernetes 标签的帖子](https://stackoverflow.com/questions/tagged/kubernetes)。
如果现有的问题对你没有帮助,在[问一个新问题](https://stackoverflow.com/questions/ask?tags=kubernetes)
之前,**请[确保你的问题是关于 Stack Overflow 的主题](https://stackoverflow.com/help/on-topic)
并且你需要阅读关于[如何提出新问题](https://stackoverflow.com/help/how-to-ask)
的指南。**
<!--
### Slack
Many people from the Kubernetes community hang out on Kubernetes Slack in the `#kubernetes-users` channel.
Slack requires registration; you can [request an invitation](https://slack.kubernetes.io),
and registration is open to everyone). Feel free to come and ask any and all questions.
Once registered, access the [Kubernetes organisation in Slack](https://kubernetes.slack.com)
via your web browser or via Slack's own dedicated app.
-->
### Slack
Kubernetes 社区中有很多人在 `#kubernetes-users` 这一 Slack 频道聚集。
Slack 需要注册;你可以[请求一份邀请](https://slack.kubernetes.io)
并且注册是对所有人开放的。欢迎你随时来问任何问题。
一旦注册了,就可以访问通过 Web 浏览器或者 Slack 专用的应用访问
[Slack 上的 Kubernetes 组织](https://kubernetes.slack.com)。
<!--
Once you are registered, browse the growing list of channels for various subjects of
interest. For example, people new to Kubernetes may also want to join the
[`#kubernetes-novice`](https://kubernetes.slack.com/messages/kubernetes-novice) channel. As another example, developers should join the
[`#kubernetes-dev`](https://kubernetes.slack.com/messages/kubernetes-dev) channel.
-->
一旦你完成了注册,就可以浏览各种感兴趣主题的频道列表(一直在增长)。
例如,Kubernetes 新人可能还想加入
[`#kubernetes-novice`](https://kubernetes.slack.com/messages/kubernetes-novice)
频道。又比如,开发人员应该加入
[`#kubernetes-dev`](https://kubernetes.slack.com/messages/kubernetes-dev)
频道。
<!--
There are also many country specific/local language channels. Feel free to join
these channels for localized support and info:
-->
还有许多国家/地区语言频道。请随时加入这些频道以获得本地化支持和信息:
{{< table caption="Country / language specific Slack channels" >}}
<!--
Country | Channels
:---------|:------------
China | [`#cn-users`](https://kubernetes.slack.com/messages/cn-users), [`#cn-events`](https://kubernetes.slack.com/messages/cn-events)
Finland | [`#fi-users`](https://kubernetes.slack.com/messages/fi-users)
France | [`#fr-users`](https://kubernetes.slack.com/messages/fr-users), [`#fr-events`](https://kubernetes.slack.com/messages/fr-events)
Germany | [`#de-users`](https://kubernetes.slack.com/messages/de-users), [`#de-events`](https://kubernetes.slack.com/messages/de-events)
India | [`#in-users`](https://kubernetes.slack.com/messages/in-users), [`#in-events`](https://kubernetes.slack.com/messages/in-events)
Italy | [`#it-users`](https://kubernetes.slack.com/messages/it-users), [`#it-events`](https://kubernetes.slack.com/messages/it-events)
Japan | [`#jp-users`](https://kubernetes.slack.com/messages/jp-users), [`#jp-events`](https://kubernetes.slack.com/messages/jp-events)
Korea | [`#kr-users`](https://kubernetes.slack.com/messages/kr-users)
Netherlands | [`#nl-users`](https://kubernetes.slack.com/messages/nl-users)
Norway | [`#norw-users`](https://kubernetes.slack.com/messages/norw-users)
Poland | [`#pl-users`](https://kubernetes.slack.com/messages/pl-users)
Russia | [`#ru-users`](https://kubernetes.slack.com/messages/ru-users)
Spain | [`#es-users`](https://kubernetes.slack.com/messages/es-users)
Sweden | [`#se-users`](https://kubernetes.slack.com/messages/se-users)
Turkey | [`#tr-users`](https://kubernetes.slack.com/messages/tr-users), [`#tr-events`](https://kubernetes.slack.com/messages/tr-events)
-->
国家 | 频道
:------|:------------
中国 | [`#cn-users`](https://kubernetes.slack.com/messages/cn-users), [`#cn-events`](https://kubernetes.slack.com/messages/cn-events)
芬兰 | [`#fi-users`](https://kubernetes.slack.com/messages/fi-users)
法国 | [`#fr-users`](https://kubernetes.slack.com/messages/fr-users), [`#fr-events`](https://kubernetes.slack.com/messages/fr-events)
德国 | [`#de-users`](https://kubernetes.slack.com/messages/de-users), [`#de-events`](https://kubernetes.slack.com/messages/de-events)
印度 | [`#in-users`](https://kubernetes.slack.com/messages/in-users), [`#in-events`](https://kubernetes.slack.com/messages/in-events)
意大利 | [`#it-users`](https://kubernetes.slack.com/messages/it-users), [`#it-events`](https://kubernetes.slack.com/messages/it-events)
日本 | [`#jp-users`](https://kubernetes.slack.com/messages/jp-users), [`#jp-events`](https://kubernetes.slack.com/messages/jp-events)
韩国 | [`#kr-users`](https://kubernetes.slack.com/messages/kr-users)
荷兰 | [`#nl-users`](https://kubernetes.slack.com/messages/nl-users)
挪威 | [`#norw-users`](https://kubernetes.slack.com/messages/norw-users)
波兰 | [`#pl-users`](https://kubernetes.slack.com/messages/pl-users)
俄罗斯 | [`#ru-users`](https://kubernetes.slack.com/messages/ru-users)
西班牙 | [`#es-users`](https://kubernetes.slack.com/messages/es-users)
瑞典 | [`#se-users`](https://kubernetes.slack.com/messages/se-users)
土耳其 | [`#tr-users`](https://kubernetes.slack.com/messages/tr-users), [`#tr-events`](https://kubernetes.slack.com/messages/tr-events)
{{< /table >}}
<!--
### Forum
You're welcome to join the official Kubernetes Forum: [discuss.kubernetes.io](https://discuss.kubernetes.io).
-->
### 论坛 {#forum}
欢迎你加入 Kubernetes 官方论坛
[discuss.kubernetes.io](https://discuss.kubernetes.io)。
<!--
### Bugs and Feature requests
If you have what looks like a bug, or you would like to make a feature request,
please use the [Github issue tracking system](https://github.com/kubernetes/kubernetes/issues).
-->
### Bugs 和功能请求 {#bugs-and-feature-requests}
如果你发现一个看起来像 Bug 的问题,或者你想提出一个功能请求,请使用
[Github 问题跟踪系统](https://github.com/kubernetes/kubernetes/issues)。
<!--
Before you file an issue, please search existing issues to see if your issue is
already covered.
If filing a bug, please include detailed information about how to reproduce the
problem, such as:
-->
在提交问题之前,请搜索现有问题列表以查看是否其中已涵盖你的问题。
如果提交 Bug,请提供如何重现问题的详细信息,例如:
<!--
* Kubernetes version: `kubectl version`
* Cloud provider, OS distro, network configuration, and Docker version
* Steps to reproduce the problem
-->
* Kubernetes 版本:`kubectl version`
* 云平台、OS 发行版、网络配置和 Docker 版本
* 重现问题的步骤
@@ -13,5 +13,6 @@ weight: 20
<!--
This doc contains a set of resources for fixing issues with containerized applications. It covers things like common issues with Kubernetes resources (like Pods, Services, or StatefulSets), advice on making sense of container termination messages, and ways to debug running containers.
-->
文档包含了一些用来解决容器应用问题的资源。它涵盖了 Kubernetes 资源(如 Pod、Service 或 StatefulSets
的常见问题,理解容器终止消息的建议以及调试正在运行的容器的方法。
文档包含一组用于解决容器应用程序问题的资源。
它涵盖了诸如 Kubernetes 资源(如 Pod、Service 或 StatefulSets)的常见问题、
关于理解容器终止消息的建议以及调试正在运行的容器的方法。
@@ -0,0 +1,194 @@
---
title: 调试 Init 容器
content_type: task
weight: 40
---
<!--
reviewers:
- bprashanth
- enisoc
- erictune
- foxish
- janetkuo
- kow3ns
- smarterclayton
title: Debug Init Containers
content_type: task
weight: 40
-->
<!-- overview -->
<!--
This page shows how to investigate problems related to the execution of
Init Containers. The example command lines below refer to the Pod as
`<pod-name>` and the Init Containers as `<init-container-1>` and
`<init-container-2>`.
-->
此页显示如何核查与 Init 容器执行相关的问题。
下面的示例命令行将 Pod 称为 `<pod-name>`,而 Init 容器称为 `<init-container-1>`
`<init-container-2>`
## {{% heading "prerequisites" %}}
{{< include "task-tutorial-prereqs.md" >}} {{< version-check >}}
<!--
* You should be familiar with the basics of
[Init Containers](/docs/concepts/workloads/pods/init-containers/).
* You should have [Configured an Init Container](/docs/tasks/configure-pod-container/configure-pod-initialization/#creating-a-pod-that-has-an-init-container/).
-->
* 你应该熟悉 [Init 容器](/zh/docs/concepts/workloads/pods/init-containers/)的基础知识。
* 你应该已经[配置好一个 Init 容器](/zh/docs/tasks/configure-pod-container/configure-pod-initialization/#creating-a-pod-that-has-an-init-container/)。
<!-- steps -->
<!--
## Checking the status of Init Containers
Display the status of your pod:
-->
## 检查 Init 容器的状态
显示你的 Pod 的状态:
```shell
kubectl get pod <pod-name>
```
<!--
For example, a status of `Init:1/2` indicates that one of two Init Containers
has completed successfully:
-->
例如,状态 `Init:1/2` 表明两个 Init 容器中的一个已经成功完成:
```
NAME READY STATUS RESTARTS AGE
<pod-name> 0/1 Init:1/2 0 7s
```
<!--
See [Understanding Pod status](#understanding-pod-status) for more examples of
status values and their meanings.
-->
更多状态值及其含义请参考[理解 Pod 的状态](#understanding-pod-status)。
<!--
## Getting details about Init Containers
View more detailed information about Init Container execution:
-->
## 获取 Init 容器详情 {#getting-details-about-init-containers}
查看 Init 容器运行的更多详情:
```shell
kubectl describe pod <pod-name>
```
<!--
For example, a Pod with two Init Containers might show the following:
-->
例如,对于包含两个 Init 容器的 Pod 可能显示如下信息:
```
Init Containers:
<init-container-1>:
Container ID: ...
...
State: Terminated
Reason: Completed
Exit Code: 0
Started: ...
Finished: ...
Ready: True
Restart Count: 0
...
<init-container-2>:
Container ID: ...
...
State: Waiting
Reason: CrashLoopBackOff
Last State: Terminated
Reason: Error
Exit Code: 1
Started: ...
Finished: ...
Ready: False
Restart Count: 3
...
```
<!--
You can also access the Init Container statuses programmatically by reading the
`status.initContainerStatuses` field on the Pod Spec:
-->
你还可以通过编程方式读取 Pod Spec 上的 `status.initContainerStatuses` 字段,了解 Init 容器的状态:
```shell
kubectl get pod nginx --template '{{.status.initContainerStatuses}}'
```
<!--
This command will return the same information as above in raw JSON.
-->
此命令将返回与原始 JSON 中相同的信息.
<!--
## Accessing logs from Init Containers
Pass the Init Container name along with the Pod name
to access its logs.
-->
## 通过 Init 容器访问日志 {#accessing-logs-from-init-containers}
与 Pod 名称一起传递 Init 容器名称,以访问容器的日志。
```shell
kubectl logs <pod-name> -c <init-container-2>
```
<!--
Init Containers that run a shell script print
commands as they're executed. For example, you can do this in Bash by running
`set -x` at the beginning of the script.
-->
运行 Shell 脚本的 Init 容器在执行 Shell 脚本时输出命令本身。
例如,你可以在 Bash 中通过在脚本的开头运行 `set -x` 来实现。
<!-- discussion -->
<!--
## Understanding Pod status
A Pod status beginning with `Init:` summarizes the status of Init Container
execution. The table below describes some example status values that you might
see while debugging Init Containers.
-->
## 理解 Pod 的状态 {#understanding-pod-status}
`Init:` 开头的 Pod 状态汇总了 Init 容器执行的状态。
下表介绍调试 Init 容器时可能看到的一些状态值示例。
<!--
Status | Meaning
------ | -------
`Init:N/M` | The Pod has `M` Init Containers, and `N` have completed so far.
`Init:Error` | An Init Container has failed to execute.
`Init:CrashLoopBackOff` | An Init Container has failed repeatedly.
`Pending` | The Pod has not yet begun executing Init Containers.
`PodInitializing` or `Running` | The Pod has already finished executing Init Containers.
-->
状态 | 含义
------ | -------
`Init:N/M` | Pod 包含 `M` 个 Init 容器,其中 `N` 个已经运行完成。
`Init:Error` | Init 容器已执行失败。
`Init:CrashLoopBackOff` | Init 容器执行总是失败。
`Pending` | Pod 还没有开始执行 Init 容器。
`PodInitializing` or `Running` | Pod 已经完成执行 Init 容器。
@@ -0,0 +1,305 @@
---
title: 调试 Pod
content_type: concept
weight: 10
---
<!--
reviewers:
- mikedanese
- thockin
title: Debug Pods
content_type: concept
weight: 10
-->
<!-- overview -->
<!--
This guide is to help users debug applications that are deployed into Kubernetes and not behaving correctly.
This is *not* a guide for people who want to debug their cluster. For that you should check out
[this guide](/docs/tasks/debug/debug-cluster).
-->
本指南帮助用户调试那些部署到 Kubernetes 上后没有正常运行的应用。
本指南 **并非** 指导用户如何调试集群。
如果想调试集群的话,请参阅[这里](/zh/docs/tasks/debug/debug-cluster)。
<!-- body -->
<!--
## Diagnosing the problem
The first step in troubleshooting is triage. What is the problem? Is it your Pods, your Replication Controller or
your Service?
* [Debugging Pods](#debugging-pods)
* [Debugging Replication Controllers](#debugging-replication-controllers)
* [Debugging Services](#debugging-services)
-->
## 诊断问题 {#diagnosing-the-problem}
故障排查的第一步是先给问题分类。问题是什么?是关于 Pod、Replication Controller 还是 Service
* [调试 Pod](#debugging-pods)
* [调试 Replication Controller](#debugging-replication-controllers)
* [调试 Service](#debugging-services)
<!--
### Debugging Pods
The first step in debugging a Pod is taking a look at it. Check the current state of the Pod and recent events with the following command:
-->
### 调试 Pod {#debugging-pods}
调试 Pod 的第一步是查看 Pod 信息。用如下命令查看 Pod 的当前状态和最近的事件:
```shell
kubectl describe pods ${POD_NAME}
```
<!--
Look at the state of the containers in the pod. Are they all `Running`? Have there been recent restarts?
Continue debugging depending on the state of the pods.
-->
查看一下 Pod 中的容器所处的状态。这些容器的状态都是 `Running` 吗?最近有没有重启过?
后面的调试都是要依靠 Pod 的状态的。
<!--
#### My pod stays pending
If a Pod is stuck in `Pending` it means that it can not be scheduled onto a node. Generally this is because
there are insufficient resources of one type or another that prevent scheduling. Look at the output of the
`kubectl describe ...` command above. There should be messages from the scheduler about why it can not schedule
your pod. Reasons include:
-->
#### Pod 停滞在 Pending 状态
如果一个 Pod 停滞在 `Pending` 状态,表示 Pod 没有被调度到节点上。通常这是因为
某种类型的资源不足导致无法调度。
查看上面的 `kubectl describe ...` 命令的输出,其中应该显示了为什么没被调度的原因。
常见原因如下:
<!--
* **You don't have enough resources**: You may have exhausted the supply of CPU or Memory in your cluster, in this case
you need to delete Pods, adjust resource requests, or add new nodes to your cluster. See
[Compute Resources document](/docs/concepts/configuration/manage-resources-containers/) for more information.
* **You are using `hostPort`**: When you bind a Pod to a `hostPort` there are a limited number of places that pod can be
scheduled. In most cases, `hostPort` is unnecessary, try using a Service object to expose your Pod. If you do require
`hostPort` then you can only schedule as many Pods as there are nodes in your Kubernetes cluster.
-->
* **资源不足**:
你可能耗尽了集群上所有的 CPU 或内存。此时,你需要删除 Pod、调整资源请求或者为集群添加节点。
更多信息请参阅[计算资源文档](/zh/docs/concepts/configuration/manage-resources-containers/)
* **使用了 `hostPort`**:
如果绑定 Pod 到 `hostPort`,那么能够运行该 Pod 的节点就有限了。
多数情况下,`hostPort` 是非必要的,而应该采用 Service 对象来暴露 Pod。
如果确实需要使用 `hostPort`,那么集群中节点的个数就是所能创建的 Pod
的数量上限。
<!--
#### My pod stays waiting
If a Pod is stuck in the `Waiting` state, then it has been scheduled to a worker node, but it can't run on that machine.
Again, the information from `kubectl describe ...` should be informative. The most common cause of `Waiting` pods is a failure to pull the image. There are three things to check:
* Make sure that you have the name of the image correct.
* Have you pushed the image to the registry?
* Try to manually pull the image to see if the image can be pulled. For example,
if you use Docker on your PC, run `docker pull <image>`.
-->
#### Pod 停滞在 Waiting 状态
如果 Pod 停滞在 `Waiting` 状态,则表示 Pod 已经被调度到某工作节点,但是无法在该节点上运行。
同样,`kubectl describe ...` 命令的输出可能很有用。
`Waiting` 状态的最常见原因是拉取镜像失败。要检查的有三个方面:
* 确保镜像名字拼写正确
* 确保镜像已被推送到镜像仓库
* 尝试手动是否能拉取镜像。例如,如果你在你的 PC 上使用 Docker,请运行 `docker pull <镜像>`
<!--
#### My pod is crashing or otherwise unhealthy
Once your pod has been scheduled, the methods described in [Debug Running Pods](
/docs/tasks/debug/debug-application/debug-running-pod/) are available for debugging.
-->
#### Pod 处于 Crashing 或别的不健康状态
一旦 Pod 被调度,就可以采用
[调试运行中的 Pod](/zh/docs/tasks/debug/debug-application/debug-running-pod/)
中的方法来进一步调试。
<!--
#### My pod is running but not doing what I told it to do
If your pod is not behaving as you expected, it may be that there was an error in your
pod description (e.g. `mypod.yaml` file on your local machine), and that the error
was silently ignored when you created the pod. Often a section of the pod description
is nested incorrectly, or a key name is typed incorrectly, and so the key is ignored.
For example, if you misspelled `command` as `commnd` then the pod will be created but
will not use the command line you intended it to use.
-->
#### Pod 处于 Running 态但是没有正常工作
如果 Pod 行为不符合预期,很可能 Pod 描述(例如你本地机器上的 `mypod.yaml`)中有问题,
并且该错误在创建 Pod 时被忽略掉,没有报错。
通常,Pod 的定义中节区嵌套关系错误、字段名字拼错的情况都会引起对应内容被忽略掉。
例如,如果你误将 `command` 写成 `commnd`Pod 虽然可以创建,但它不会执行
你期望它执行的命令行。
<!--
The first thing to do is to delete your pod and try creating it again with the `--validate` option.
For example, run `kubectl apply --validate -f mypod.yaml`.
If you misspelled `command` as `commnd` then will give an error like this:
-->
可以做的第一件事是删除你的 Pod,并尝试带有 `--validate` 选项重新创建。
例如,运行 `kubectl apply --validate -f mypod.yaml`
如果 `command` 被误拼成 `commnd`,你将会看到下面的错误信息:
```
I0805 10:43:25.129850 46757 schema.go:126] unknown field: commnd
I0805 10:43:25.129973 46757 schema.go:129] this may be a false alarm, see https://github.com/kubernetes/kubernetes/issues/6842
pods/mypod
```
<!-- TODO: Now that #11914 is merged, this advice may need to be updated -->
<!--
The next thing to check is whether the pod on the apiserver
matches the pod you meant to create (e.g. in a yaml file on your local machine).
For example, run `kubectl get pods/mypod -o yaml > mypod-on-apiserver.yaml` and then
manually compare the original pod description, `mypod.yaml` with the one you got
back from apiserver, `mypod-on-apiserver.yaml`. There will typically be some
lines on the "apiserver" version that are not on the original version. This is
expected. However, if there are lines on the original that are not on the apiserver
version, then this may indicate a problem with your pod spec.
-->
接下来就要检查的是 API 服务器上的 Pod 与你所期望创建的是否匹配
(例如,你原本使用本机上的一个 YAML 文件来创建 Pod)。
例如,运行 `kubectl get pods/mypod -o yaml > mypod-on-apiserver.yaml`,之后
手动比较 `mypod.yaml` 与从 API 服务器取回的 Pod 描述。
从 API 服务器处获得的 YAML 通常包含一些创建 Pod 所用的 YAML 中不存在的行,这是正常的。
不过,如果如果源文件中有些行在 API 服务器版本中不存在,则意味着
Pod 规约是有问题的。
<!--
### Debugging Replication Controllers
Replication controllers are fairly straightforward. They can either create Pods or they can't. If they can't
create pods, then please refer to the [instructions above](#debugging-pods) to debug your pods.
You can also use `kubectl describe rc ${CONTROLLER_NAME}` to introspect events related to the replication
controller.
-->
### 调试副本控制器 {#debugging-replication-controllers}
副本控制器相对比较简单直接。它们要么能创建 Pod,要么不能。
如果不能创建 Pod,请参阅[上述说明](#debugging-pods)调试 Pod。
你也可以使用 `kubectl describe rc ${CONTROLLER_NAME}` 命令来检视副本控制器相关的事件。
<!--
### Debugging Services
Services provide load balancing across a set of pods. There are several common problems that can make Services
not work properly. The following instructions should help debug Service problems.
First, verify that there are endpoints for the service. For every Service object, the apiserver makes an `endpoints` resource available.
You can view this resource with:
-->
### 调试 Service {#debugging-services}
服务支持在多个 Pod 间负载均衡。
有一些常见的问题可以造成服务无法正常工作。
以下说明将有助于调试服务的问题。
首先,验证服务是否有端点。对于每一个 Service 对象,API 服务器为其提供
对应的 `endpoints` 资源。
通过如下命令可以查看 endpoints 资源:
```shell
kubectl get endpoints ${SERVICE_NAME}
```
<!--
Make sure that the endpoints match up with the number of pods that you expect to be members of your service.
For example, if your Service is for an nginx container with 3 replicas, you would expect to see three different
IP addresses in the Service's endpoints.
-->
确保 Endpoints 与服务成员 Pod 个数一致。
例如,如果你的 Service 用来运行 3 个副本的 nginx 容器,你应该会在 Service 的 Endpoints
中看到 3 个不同的 IP 地址。
<!--
#### My service is missing endpoints
If you are missing endpoints, try listing pods using the labels that Service uses. Imagine that you have
a Service where the labels are:
-->
#### 服务缺少 Endpoints
如果没有 Endpoints,请尝试使用 Service 所使用的标签列出 Pod。
假定你的服务包含如下标签选择算符:
```yaml
...
spec:
- selector:
name: nginx
type: frontend
```
<!--
You can use:
```shell
kubectl get pods --selector=name=nginx,type=frontend
```
to list pods that match this selector. Verify that the list matches the Pods that you expect to provide your Service.
-->
你可以使用如下命令列出与选择算符相匹配的 Pod,并验证这些 Pod 是否归属于创建的服务:
```shell
kubectl get pods --selector=name=nginx,type=frontend
```
<!--
Verify that the pod's `containerPort` matches up with the Service's `targetPort`
-->
验证 Pod 的 `containerPort` 与服务的 `targetPort` 是否匹配。
<!--
#### Network traffic is not forwarded
Please see [debugging service](/docs/tasks/debug/debug-applications/debug-service/) for more information.
-->
#### 网络流量未被转发
请参阅[调试 Service](/zh/docs/tasks/debug/debug-applications/debug-service/) 了解更多信息。
## {{% heading "whatsnext" %}}
<!--
If none of the above solves your problem, follow the instructions in
[Debugging Service document](/docs/tasks/debug/debug-applications/debug-service/)
to make sure that your `Service` is running, has `Endpoints`, and your `Pods` are
actually serving; you have DNS working, iptables rules installed, and kube-proxy
does not seem to be misbehaving.
You may also visit [troubleshooting document](/docs/tasks/debug/overview/) for more information.
-->
如果上述方法都不能解决你的问题,
请按照[调试 Service 文档](/zh/docs/tasks/debug/debug-applications/debug-service/)中的介绍,
确保你的 `Service` 处于 Running 态,有 `Endpoints` 被创建,`Pod` 真的在提供服务;
DNS 服务已配置并正常工作,iptables 规则也以安装并且 `kube-proxy` 也没有异常行为。
你也可以访问[故障排查文档](/zh/docs/tasks/debug/overview/)来获取更多信息。
@@ -3,6 +3,14 @@ title: 调试运行中的 Pod
content_type: task
---
<!--
reviewers:
- verb
- soltysh
title: Debug Running Pods
content_type: task
-->
<!-- overview -->
<!--
This page explains how to debug Pods running (or crashing) on a Node.
@@ -13,24 +21,426 @@ This page explains how to debug Pods running (or crashing) on a Node.
<!--
* Your {{< glossary_tooltip text="Pod" term_id="pod" >}} should already be
scheduled and running. If your Pod is not yet running, start with [Troubleshoot
Applications](/docs/tasks/debug-application-cluster/debug-application/).
scheduled and running. If your Pod is not yet running, start with [Debugging
Pods](/docs/tasks/debug/debug-application/).
* For some of the advanced debugging steps you need to know on which Node the
Pod is running and have shell access to run commands on that Node. You don't
need that access to run the standard debug steps that use `kubectl`.
-->
* 你的 {{< glossary_tooltip text="Pod" term_id="pod" >}} 应该已经被调度并正在运行中,
如果你的 Pod 还没有运行,请参阅[应用问题排查](/zh/docs/tasks/debug-application-cluster/debug-application/)。
如果你的 Pod 还没有运行,请参阅[调试 Pod](/zh/docs/tasks/debug/debug-application/)。
* 对于一些高级调试步骤,你应该知道 Pod 具体运行在哪个节点上,在该节点上有权限去运行一些命令
* 对于一些高级调试步骤,你应该知道 Pod 具体运行在哪个节点上,并具有在该节点上运行命令的 shell 访问权限
你不需要任何访问权限就可以使用 `kubectl` 去运行一些标准调试步骤。
<!-- steps -->
<!--
## Using `kubectl describe pod` to fetch details about pods
-->
## 使用 `kubectl describe pod` 命令获取 Pod 详情
<!--
For this example we'll use a Deployment to create two pods, similar to the earlier example.
-->
与之前的例子类似,我们使用一个 Deployment 来创建两个 Pod。
{{< codenew file="application/nginx-with-request.yaml" >}}
<!--
Create deployment by running following command:
-->
使用如下命令创建 Deployment
```shell
kubectl apply -f https://k8s.io/examples/application/nginx-with-request.yaml
```
```
deployment.apps/nginx-deployment created
```
<!--
Check pod status by following command:
-->
使用如下命令查看 Pod 状态:
```shell
kubectl get pods
```
```
NAME READY STATUS RESTARTS AGE
nginx-deployment-67d4bdd6f5-cx2nz 1/1 Running 0 13s
nginx-deployment-67d4bdd6f5-w6kd7 1/1 Running 0 13s
```
<!--
We can retrieve a lot more information about each of these pods using `kubectl describe pod`. For example:
-->
我们可以使用 `kubectl describe pod` 命令来查询每个 Pod 的更多信息,比如:
```shell
kubectl describe pod nginx-deployment-67d4bdd6f5-w6kd7
```
```none
Name: nginx-deployment-67d4bdd6f5-w6kd7
Namespace: default
Priority: 0
Node: kube-worker-1/192.168.0.113
Start Time: Thu, 17 Feb 2022 16:51:01 -0500
Labels: app=nginx
pod-template-hash=67d4bdd6f5
Annotations: <none>
Status: Running
IP: 10.88.0.3
IPs:
IP: 10.88.0.3
IP: 2001:db8::1
Controlled By: ReplicaSet/nginx-deployment-67d4bdd6f5
Containers:
nginx:
Container ID: containerd://5403af59a2b46ee5a23fb0ae4b1e077f7ca5c5fb7af16e1ab21c00e0e616462a
Image: nginx
Image ID: docker.io/library/nginx@sha256:2834dc507516af02784808c5f48b7cbe38b8ed5d0f4837f16e78d00deb7e7767
Port: 80/TCP
Host Port: 0/TCP
State: Running
Started: Thu, 17 Feb 2022 16:51:05 -0500
Ready: True
Restart Count: 0
Limits:
cpu: 500m
memory: 128Mi
Requests:
cpu: 500m
memory: 128Mi
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-bgsgp (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
kube-api-access-bgsgp:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: Guaranteed
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 34s default-scheduler Successfully assigned default/nginx-deployment-67d4bdd6f5-w6kd7 to kube-worker-1
Normal Pulling 31s kubelet Pulling image "nginx"
Normal Pulled 30s kubelet Successfully pulled image "nginx" in 1.146417389s
Normal Created 30s kubelet Created container nginx
Normal Started 30s kubelet Started container nginx
```
<!--
Here you can see configuration information about the container(s) and Pod (labels, resource requirements, etc.), as well as status information about the container(s) and Pod (state, readiness, restart count, events, etc.).
-->
在这里,您可以看到有关容器和 Pod 的配置信息(标签、资源需求等),
以及有关容器和 Pod 的状态信息(状态、就绪、重启计数、事件等) 。
<!--
The container state is one of Waiting, Running, or Terminated. Depending on the state, additional information will be provided -- here you can see that for a container in Running state, the system tells you when the container started.
-->
容器状态是 Waiting、Running 和 Terminated 之一。
根据状态的不同,还有对应的额外的信息 —— 在这里你可以看到,
对于处于运行状态的容器,系统会告诉你容器的启动时间。
<!--
Ready tells you whether the container passed its last readiness probe. (In this case, the container does not have a readiness probe configured; the container is assumed to be ready if no readiness probe is configured.)
-->
Ready 指示是否通过了最后一个就绪态探测。
(在本例中,容器没有配置就绪态探测;如果没有配置就绪态探测,则假定容器已经就绪。)
<!--
Restart Count tells you how many times the container has been restarted; this information can be useful for detecting crash loops in containers that are configured with a restart policy of 'always.'
-->
Restart Count 告诉你容器已重启的次数;
这些信息对于定位配置了 “Always” 重启策略的容器持续崩溃问题非常有用。
<!--
Currently the only Condition associated with a Pod is the binary Ready condition, which indicates that the pod is able to service requests and should be added to the load balancing pools of all matching services.
-->
目前,唯一与 Pod 有关的状态是 Ready 状况,该状况表明 Pod 能够为请求提供服务,
并且应该添加到相应服务的负载均衡池中。
<!--
Lastly, you see a log of recent events related to your Pod. The system compresses multiple identical events by indicating the first and last time it was seen and the number of times it was seen. "From" indicates the component that is logging the event, "SubobjectPath" tells you which object (e.g. container within the pod) is being referred to, and "Reason" and "Message" tell you what happened.
-->
最后,你还可以看到与 Pod 相关的近期事件。
系统通过指示第一次和最后一次看到事件以及看到该事件的次数来压缩多个相同的事件。
“From” 标明记录事件的组件,
“SubobjectPath” 告诉你引用了哪个对象(例如 Pod 中的容器),
“Reason” 和 “Message” 告诉你发生了什么。
<!--
## Example: debugging Pending Pods
A common scenario that you can detect using events is when you've created a Pod that won't fit on any node. For example, the Pod might request more resources than are free on any node, or it might specify a label selector that doesn't match any nodes. Let's say we created the previous Deployment with 5 replicas (instead of 2) and requesting 600 millicores instead of 500, on a four-node cluster where each (virtual) machine has 1 CPU. In that case one of the Pods will not be able to schedule. (Note that because of the cluster addon pods such as fluentd, skydns, etc., that run on each node, if we requested 1000 millicores then none of the Pods would be able to schedule.)
-->
## 例子: 调试 Pending 状态的 Pod
可以使用事件来调试的一个常见的场景是,你创建 Pod 无法被调度到任何节点。
比如,Pod 请求的资源比较多,没有任何一个节点能够满足,或者它指定了一个标签,没有节点可匹配。
假定我们创建之前的 Deployment 时指定副本数是 5(不再是 2),并且请求 600 毫核(不再是 500),
对于一个 4 个节点的集群,若每个节点只有 1 个 CPU,这时至少有一个 Pod 不能被调度。
(需要注意的是,其他集群插件 Pod,比如 fluentd、skydns 等等会在每个节点上运行,
如果我们需求 1000 毫核,将不会有 Pod 会被调度。)
```shell
kubectl get pods
```
```
NAME READY STATUS RESTARTS AGE
nginx-deployment-1006230814-6winp 1/1 Running 0 7m
nginx-deployment-1006230814-fmgu3 1/1 Running 0 7m
nginx-deployment-1370807587-6ekbw 1/1 Running 0 1m
nginx-deployment-1370807587-fg172 0/1 Pending 0 1m
nginx-deployment-1370807587-fz9sd 0/1 Pending 0 1m
```
<!--
To find out why the nginx-deployment-1370807587-fz9sd pod is not running, we can use `kubectl describe pod` on the pending Pod and look at its events:
-->
为了查找 Pod nginx-deployment-1370807587-fz9sd 没有运行的原因,我们可以使用
`kubectl describe pod` 命令描述 Pod,查看其事件:
```shell
kubectl describe pod nginx-deployment-1370807587-fz9sd
```
```none
Name: nginx-deployment-1370807587-fz9sd
Namespace: default
Node: /
Labels: app=nginx,pod-template-hash=1370807587
Status: Pending
IP:
Controllers: ReplicaSet/nginx-deployment-1370807587
Containers:
nginx:
Image: nginx
Port: 80/TCP
QoS Tier:
memory: Guaranteed
cpu: Guaranteed
Limits:
cpu: 1
memory: 128Mi
Requests:
cpu: 1
memory: 128Mi
Environment Variables:
Volumes:
default-token-4bcbi:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-4bcbi
Events:
FirstSeen LastSeen Count From SubobjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
1m 48s 7 {default-scheduler } Warning FailedScheduling pod (nginx-deployment-1370807587-fz9sd) failed to fit in any node
fit failure on node (kubernetes-node-6ta5): Node didn't have enough resource: CPU, requested: 1000, used: 1420, capacity: 2000
fit failure on node (kubernetes-node-wul5): Node didn't have enough resource: CPU, requested: 1000, used: 1100, capacity: 2000
```
<!--
Here you can see the event generated by the scheduler saying that the Pod failed to schedule for reason `FailedScheduling` (and possibly others). The message tells us that there were not enough resources for the Pod on any of the nodes.
-->
这里你可以看到由调度器记录的事件,它表明了 Pod 不能被调度的原因是 `FailedScheduling`(也可能是其他值)。
其 message 部分表明没有任何节点拥有足够多的资源。
<!--
To correct this situation, you can use `kubectl scale` to update your Deployment to specify four or fewer replicas. (Or you could leave the one Pod pending, which is harmless.)
-->
要纠正这种情况,可以使用 `kubectl scale` 更新 Deployment,以指定 4 个或更少的副本。
(或者你可以让 Pod 继续保持这个状态,这是无害的。)
<!--
Events such as the ones you saw at the end of `kubectl describe pod` are persisted in etcd and provide high-level information on what is happening in the cluster. To list all events you can use
-->
你在 `kubectl describe pod` 结尾处看到的事件都保存在 etcd 中,
并提供关于集群中正在发生的事情的高级信息。
如果需要列出所有事件,可使用命令:
```shell
kubectl get events
```
<!--
but you have to remember that events are namespaced. This means that if you're interested in events for some namespaced object (e.g. what happened with Pods in namespace `my-namespace`) you need to explicitly provide a namespace to the command:
-->
但是,需要注意的是,事件是区分名字空间的。
如果你对某些名字空间域的对象(比如 `my-namespace` 名字下的 Pod)的事件感兴趣,
你需要显式地在命令行中指定名字空间:
```shell
kubectl get events --namespace=my-namespace
```
<!--
To see events from all namespaces, you can use the `--all-namespaces` argument.
-->
查看所有 namespace 的事件,可使用 `--all-namespaces` 参数。
<!--
In addition to `kubectl describe pod`, another way to get extra information about a pod (beyond what is provided by `kubectl get pod`) is to pass the `-o yaml` output format flag to `kubectl get pod`. This will give you, in YAML format, even more information than `kubectl describe pod`--essentially all of the information the system has about the Pod. Here you will see things like annotations (which are key-value metadata without the label restrictions, that is used internally by Kubernetes system components), restart policy, ports, and volumes.
-->
除了 `kubectl describe pod` 以外,另一种获取 Pod 额外信息(除了 `kubectl get pod`)的方法
是给 `kubectl get pod` 增加 `-o yaml` 输出格式参数。
该命令将以 YAML 格式为你提供比 `kubectl describe pod` 更多的信息 —— 实际上是系统拥有的关于 Pod 的所有信息。
在这里,你将看到注解(没有标签限制的键值元数据,由 Kubernetes 系统组件在内部使用)、
重启策略、端口和卷等。
```shell
kubectl get pod nginx-deployment-1006230814-6winp -o yaml
```
```yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: "2022-02-17T21:51:01Z"
generateName: nginx-deployment-67d4bdd6f5-
labels:
app: nginx
pod-template-hash: 67d4bdd6f5
name: nginx-deployment-67d4bdd6f5-w6kd7
namespace: default
ownerReferences:
- apiVersion: apps/v1
blockOwnerDeletion: true
controller: true
kind: ReplicaSet
name: nginx-deployment-67d4bdd6f5
uid: 7d41dfd4-84c0-4be4-88ab-cedbe626ad82
resourceVersion: "1364"
uid: a6501da1-0447-4262-98eb-c03d4002222e
spec:
containers:
- image: nginx
imagePullPolicy: Always
name: nginx
ports:
- containerPort: 80
protocol: TCP
resources:
limits:
cpu: 500m
memory: 128Mi
requests:
cpu: 500m
memory: 128Mi
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: kube-api-access-bgsgp
readOnly: true
dnsPolicy: ClusterFirst
enableServiceLinks: true
nodeName: kube-worker-1
preemptionPolicy: PreemptLowerPriority
priority: 0
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
serviceAccount: default
serviceAccountName: default
terminationGracePeriodSeconds: 30
tolerations:
- effect: NoExecute
key: node.kubernetes.io/not-ready
operator: Exists
tolerationSeconds: 300
- effect: NoExecute
key: node.kubernetes.io/unreachable
operator: Exists
tolerationSeconds: 300
volumes:
- name: kube-api-access-bgsgp
projected:
defaultMode: 420
sources:
- serviceAccountToken:
expirationSeconds: 3607
path: token
- configMap:
items:
- key: ca.crt
path: ca.crt
name: kube-root-ca.crt
- downwardAPI:
items:
- fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
path: namespace
status:
conditions:
- lastProbeTime: null
lastTransitionTime: "2022-02-17T21:51:01Z"
status: "True"
type: Initialized
- lastProbeTime: null
lastTransitionTime: "2022-02-17T21:51:06Z"
status: "True"
type: Ready
- lastProbeTime: null
lastTransitionTime: "2022-02-17T21:51:06Z"
status: "True"
type: ContainersReady
- lastProbeTime: null
lastTransitionTime: "2022-02-17T21:51:01Z"
status: "True"
type: PodScheduled
containerStatuses:
- containerID: containerd://5403af59a2b46ee5a23fb0ae4b1e077f7ca5c5fb7af16e1ab21c00e0e616462a
image: docker.io/library/nginx:latest
imageID: docker.io/library/nginx@sha256:2834dc507516af02784808c5f48b7cbe38b8ed5d0f4837f16e78d00deb7e7767
lastState: {}
name: nginx
ready: true
restartCount: 0
started: true
state:
running:
startedAt: "2022-02-17T21:51:05Z"
hostIP: 192.168.0.113
phase: Running
podIP: 10.88.0.3
podIPs:
- ip: 10.88.0.3
- ip: 2001:db8::1
qosClass: Guaranteed
startTime: "2022-02-17T21:51:01Z"
```
<!--
## Examining pod logs {#examine-pod-logs}
First, look at the logs of the affected container:
```shell
kubectl logs ${POD_NAME} ${CONTAINER_NAME}
```
If your container has previously crashed, you can access the previous container's crash log with:
```shell
kubectl logs --previous ${POD_NAME} ${CONTAINER_NAME}
```
-->
## 检查 Pod 的日志 {#examine-pod-logs}
@@ -40,10 +450,6 @@ First, look at the logs of the affected container:
kubectl logs ${POD_NAME} ${CONTAINER_NAME}
```
<!--
If your container has previously crashed, you can access the previous container's crash log with:
-->
如果你的容器之前崩溃过,你可以通过下面命令访问之前容器的崩溃日志:
```shell
@@ -57,6 +463,30 @@ If the {{< glossary_tooltip text="container image" term_id="image" >}} includes
debugging utilities, as is the case with images built from Linux and Windows OS
base images, you can run commands inside a specific container with
`kubectl exec`:
```shell
kubectl exec ${POD_NAME} -c ${CONTAINER_NAME} -- ${CMD} ${ARG1} ${ARG2} ... ${ARGN}
```
{{< note >}}
`-c ${CONTAINER_NAME}` is optional. You can omit it for Pods that only contain a single container.
{{< /note >}}
As an example, to look at the logs from a running Cassandra pod, you might run
```shell
kubectl exec cassandra -- cat /var/log/cassandra/system.log
```
You can run a shell that's connected to your terminal using the `-i` and `-t`
arguments to `kubectl exec`, for example:
```shell
kubectl exec -it cassandra -- sh
```
For more details, see [Get a Shell to a Running Container](
/docs/tasks/debug/debug-application/get-shell-running-container/).
-->
## 使用容器 exec 进行调试 {#container-exec}
@@ -67,41 +497,23 @@ base images, you can run commands inside a specific container with
```shell
kubectl exec ${POD_NAME} -c ${CONTAINER_NAME} -- ${CMD} ${ARG1} ${ARG2} ... ${ARGN}
```
<!--
`-c ${CONTAINER_NAME}` is optional. You can omit it for Pods that only contain a single container.
-->
{{< note >}}
`-c ${CONTAINER_NAME}` 是可选择的。如果Pod中仅包含一个容器,就可以忽略它。
{{< /note >}}
<!--
As an example, to look at the logs from a running Cassandra pod, you might run
-->
例如,要查看正在运行的 Cassandra Pod 中的日志,可以运行:
例如,要查看正在运行的 Cassandra pod中的日志,可以运行:
```shell
kubectl exec cassandra -- cat /var/log/cassandra/system.log
```
<!--
You can run a shell that's connected to your terminal using the `-i` and `-t`
arguments to `kubectl exec`, for example:
-->
你可以在 `kubectl exec` 命令后面加上 `-i``-t` 来运行一个连接到你的终端的 Shell,比如:
```shell
kubectl exec -it cassandra -- sh
```
<!--
For more details, see [Get a Shell to a Running Container](
/docs/tasks/debug-application-cluster/get-shell-running-container/).
-->
若要了解更多内容,可查看[获取正在运行容器的 Shell](/zh/docs/tasks/debug-application-cluster/get-shell-running-container/)。
若要了解更多内容,可查看[获取正在运行容器的 Shell](/zh/docs/tasks/debug/debug-application/get-shell-running-container/)。
<!--
## Debugging with an ephemeral debug container {#ephemeral-container}
@@ -127,14 +539,18 @@ https://github.com/GoogleContainerTools/distroless).
You can use the `kubectl debug` command to add ephemeral containers to a
running Pod. First, create a pod for the example:
This section use the `pause` container image in examples because it does not
```shell
kubectl run ephemeral-demo --image=k8s.gcr.io/pause:3.1 --restart=Never
```
The examples in this section use the `pause` container image because it does not
contain debugging utilities, but this method works with all container
images.
-->
## 使用临时容器来调试的例子 {#ephemeral-container-example}
你可以使用 `kubectl debug` 命令来给正在运行中的 Pod 增加一个临时容器。
首先,像例一样创建一个 Pod
首先,像例一样创建一个 pod
```shell
kubectl run ephemeral-demo --image=k8s.gcr.io/pause:3.1 --restart=Never
@@ -147,20 +563,6 @@ kubectl run ephemeral-demo --image=k8s.gcr.io/pause:3.1 --restart=Never
<!--
If you attempt to use `kubectl exec` to create a shell you will see an error
because there is no shell in this container image.
```
OCI runtime exec failed: exec failed: container_linux.go:346: starting container process caused "exec: \"sh\": executable file not found in $PATH": unknown
```
You can instead add a debugging container using `kubectl debug`. If you
specify the `-i`/`--interactive` argument, `kubectl` will automatically attach
to the console of the Ephemeral Container.
```
Defaulting debug container name to debugger-8xzrl.
If you don't see a command prompt, try pressing enter.
/ #
```
-->
如果你尝试使用 `kubectl exec` 来创建一个 shell,你将会看到一个错误,因为这个容器镜像中没有 shell。
@@ -172,6 +574,12 @@ kubectl exec -it ephemeral-demo -- sh
OCI runtime exec failed: exec failed: container_linux.go:346: starting container process caused "exec: \"sh\": executable file not found in $PATH": unknown
```
<!--
You can instead add a debugging container using `kubectl debug`. If you
specify the `-i`/`--interactive` argument, `kubectl` will automatically attach
to the console of the Ephemeral Container.
-->
你可以改为使用 `kubectl debug` 添加调试容器。
如果你指定 `-i` 或者 `--interactive` 参数,`kubectl` 将自动挂接到临时容器的控制台。
@@ -192,19 +600,22 @@ here because `kubectl run` does not enable [process namespace sharing](
/docs/tasks/configure-pod-container/share-process-namespace/) in the pod it
creates.
{{< note >}}
The `--target` parameter must be supported by the {{< glossary_tooltip
text="Container Runtime" term_id="container-runtime" >}}. When not supported,
the Ephemeral Container may not be started, or it may be started with an
isolated process namespace so that `ps` does not reveal processes in other containers.
isolated process namespace so that `ps` does not reveal processes in other
containers.
{{< /note >}}
You can view the state of the newly created ephemeral container using `kubectl describe`:
-->
此命令添加一个新的 busybox 容器并将其挂接到该容器。`--target` 参数指定另一个容器的进程命名空间。
这是必需的,因为 `kubectl run` 不能在它创建的 Pod
中启用[共享进程命名空间](/zh/docs/tasks/configure-pod-container/share-process-namespace/)。
这是必需的,因为 `kubectl run` 不能在它创建的pod中启用
[共享进程命名空间](/zh/docs/tasks/configure-pod-container/share-process-namespace/)。
{{< note >}}
{{< glossary_tooltip text="容器运行时" term_id="container-runtime" >}}必须支持`--target`参数。
{{< glossary_tooltip text="容器运行时" term_id="container-runtime" >}}必须支持 `--target` 参数。
如果不支持,则临时容器可能不会启动,或者可能使用隔离的进程命名空间启动,
以便 `ps` 不显示其他容器内的进程。
{{< /note >}}
@@ -236,7 +647,7 @@ Ephemeral Containers:
<!--
Use `kubectl delete` to remove the Pod when you're finished:
-->
完成后,使用 `kubectl delete` 来移除 Pod
使用 `kubectl delete` 来移除已经结束掉的 Pod
```shell
kubectl delete pod ephemeral-demo
@@ -255,7 +666,7 @@ crashes on startup. In these situations you can use `kubectl debug` to create a
copy of the Pod with configuration values changed to aid debugging.
-->
有些时候 Pod 的配置参数使得在某些情况下很难执行故障排查。
例如,在容器镜像中不包含 Shell 或者你的应用程序在启动时崩溃的情况下,
例如,在容器镜像中不包含 shell 或者你的应用程序在启动时崩溃的情况下,
就不能通过运行 `kubectl exec` 来排查容器故障。
在这些情况下,你可以使用 `kubectl debug` 来创建 Pod 的副本,通过更改配置帮助调试。
@@ -269,15 +680,15 @@ Adding a new container can be useful when your application is running but not
behaving as you expect and you'd like to add additional troubleshooting
utilities to the Pod.
-->
当应用程序正在运行但其表现不符合预期时,添加新容器可能会帮助你符合预期
并且你会希望在 Pod 中添加额外的调试工具
当应用程序正在运行但其表现不符合预期时,你会希望在 Pod 中添加额外的调试工具
这时添加新容器是很有用的
<!--
For example, maybe your application's container images are built on `busybox`
but you need debugging utilities not included in `busybox`. You can simulate
this scenario using `kubectl run`:
-->
例如,可能你的应用的容器镜像是基于 `busybox` 构造的,
例如,应用的容器镜像是建立在 `busybox`基础上
但是你需要 `busybox` 中并不包含的调试工具。
你可以使用 `kubectl run` 模拟这个场景:
@@ -312,19 +723,19 @@ root@myapp-debug:/#
/docs/tasks/configure-pod-container/share-process-namespace/).
-->
{{< note >}}
* 如果你没有使用 `--container` 指定新的容器名,`kubectl debug` 会自动生成容器名称
* 如果你没有使用 `--container` 指定新的容器名,`kubectl debug` 会自动生成
* 默认情况下,`-i` 标志使 `kubectl debug` 附加到新容器上。
你可以通过指定 `--attach=false` 来防止这种情况。
如果你的会话断开连接,你可以使用 `kubectl attach` 重新连接。
* `--share-processes` 允许在此 Pod 中的其他容器中查看该容器的进程。
参阅[在 Pod 中的容器之间共享进程命名空间](/zh/docs/tasks/configure-pod-container/share-process-namespace/)
参阅[在 Pod 中的容器之间共享进程命名空间](/zh/docs/tasks/configure-pod-container/share-process-namespace/)
获取更多信息。
{{< /note >}}
<!--
Don't forget to clean up the debugging Pod when you're finished with it:
-->
结束之后,不要忘了清理调试 Pod
不要忘了清理调试 Pod
```shell
kubectl delete pod myapp myapp-debug
@@ -393,18 +804,16 @@ checking filesystem paths or running the container command manually.
现在你有了一个可以执行类似检查文件系统路径或者手动运行容器命令的交互式 shell。
<!--
{{< note >}}
* To change the command of a specific container you must
specify its name using `--container` or `kubectl debug` will instead
create a new container to run the command you specified.
* The `-i` flag causes `kubectl debug` to attach to the container by default.
You can prevent this by specifying `--attach=false`. If your session becomes
disconnected you can reattach using `kubectl attach`.
{{< /note >}}
-->
{{< note >}}
* 要更改指定容器的命令,你必须用 `--container` 命令指定容器的名字,
否则 `kubectl debug`建一个新的容器运行你指定的命令。
否则 `kubectl debug` 将建一个新的容器运行你指定的命令。
* 默认情况下,标志 `-i` 使 `kubectl debug` 附加到容器。
你可通过指定 `--attach=false` 来防止这种情况。
如果你的断开连接,可以使用 `kubectl attach` 重新连接。
@@ -413,7 +822,7 @@ checking filesystem paths or running the container command manually.
<!--
Don't forget to clean up the debugging Pod when you're finished with it:
-->
结束之后,不要忘了清理调试 Pod
不要忘了清理调试 Pod
```shell
kubectl delete pod myapp myapp-debug
@@ -429,10 +838,10 @@ As an example, create a Pod using `kubectl run`:
-->
### 在更改容器镜像时创建 Pod 副本
在某些情况下,你可能想改变行为异常的 Pod
将其中的正常生产容器镜像更改为包含调试版本或者额外工具的镜像。
在某些情况下,你可能想从正常生产容器镜像中
把行为异常的 Pod 改变为包含调试版本或者附加应用的镜像。
作为示例,用 `kubectl run` 创建一个 Pod
下面的例子,用 `kubectl run`创建一个 Pod
```
kubectl run myapp --image=busybox:1.28 --restart=Never -- sleep 1d
@@ -441,7 +850,8 @@ kubectl run myapp --image=busybox:1.28 --restart=Never -- sleep 1d
Now use `kubectl debug` to make a copy and change its container image
to `ubuntu`:
-->
现在可以使用 `kubectl debug` 创建一个副本并改变容器镜像为 `ubuntu`
现在可以使用 `kubectl debug` 创建一个副本
并改变容器镜像为 `ubuntu`
```
kubectl debug myapp --copy-to=myapp-debug --set-image=*=ubuntu
@@ -454,7 +864,7 @@ to `ubuntu`.
Don't forget to clean up the debugging Pod when you're finished with it:
-->
`--set-image` 使用`kubectl set image` 相同的 `container_name=image` 语法。
`--set-image``container_name=image` 使用相同的 `kubectl set image` 语法。
`*=ubuntu` 表示把所有容器的镜像改为 `ubuntu`
```shell
@@ -468,11 +878,11 @@ If none of these approaches work, you can find the Node on which the Pod is
running and create a privileged Pod running in the host namespaces. To create
an interactive shell on a node using `kubectl debug`, run:
-->
## 通过节点上的 Shell 来进行调试 {#node-shell-session}
## 节点上通过 shell 来进行调试 {#node-shell-session}
如果这些方法都不起作用,你可以找到运行 Pod 的节点,然后在节点上部署一个运行在宿主名字空间的特权 Pod。
你可以通过 `kubectl debug` 在节点上创建一个交互式 Shell
你可以通过`kubectl debug` 在节点上创建一个交互式 shell
```shell
kubectl debug node/mynode -it --image=ubuntu
@@ -494,9 +904,9 @@ When creating a debugging session on a node, keep in mind that:
Don't forget to clean up the debugging Pod when you're finished with it:
-->
在节点上创建调试会话,注意以下要点:
在节点上创建调试会话,注意以下要点:
* `kubectl debug` 基于节点的名字自动生成新的 Pod 的名字。
* 新的调试容器运行在宿主 IPC、宿主网络、宿主 PID 名字空间
* 新的调试容器运行在宿主命名空间里(IPC, 网络 还有PID命名空间
* 节点的根文件系统会被挂载在 `/host`
当你完成节点调试时,不要忘记清理调试 Pod:
@@ -1,6 +1,7 @@
---
content_type: concept
title: 调试 Service
weight: 20
---
<!--
@@ -9,6 +10,7 @@ reviewers:
- bowei
content_type: concept
title: Debug Services
weight: 20
-->
<!-- overview -->
@@ -636,7 +638,7 @@ they are running fine and not crashing.
The "RESTARTS" column says that these pods are not crashing frequently or being
restarted. Frequent restarts could lead to intermittent connectivity issues.
If the restart count is high, read more about how to [debug pods](/docs/tasks/debug-application-cluster/debug-pod-replication-controller/#debugging-pods).
If the restart count is high, read more about how to [debug pods](/docs/tasks/debug/debug-application/debug-pods).
Inside the Kubernetes system is a control loop which evaluates the selector of
every Service and saves the results into a corresponding Endpoints object.
@@ -646,7 +648,7 @@ every Service and saves the results into a corresponding Endpoints object.
"AGE" 列表明这些 Pod 已经启动一个小时了,这意味着它们运行良好,而未崩溃。
"RESTARTS" 列表明 Pod 没有经常崩溃或重启。经常性崩溃可能导致间歇性连接问题。
如果重启次数过大,通过[调试 pod](/zh/docs/tasks/debug-application-cluster/debug-application/#debugging-pods)
如果重启次数过大,通过[调试 Pod](/zh/docs/tasks/debug/debug-application/debug-pods)
了解相关技术。
在 Kubernetes 系统中有一个控制回路,它评估每个 Service 的选择算符,并将结果保存到 Endpoints 对象中。
@@ -1091,7 +1093,7 @@ Service is not working. Please let us know what is going on, so we can help
investigate!
Contact us on
[Slack](/docs/tasks/debug-application-cluster/troubleshooting/#slack) or
[Slack](/docs/tasks/debug/overview/#slack) or
[Forum](https://discuss.kubernetes.io) or
[GitHub](https://github.com/kubernetes/kubernetes).
-->
@@ -1102,7 +1104,7 @@ Contact us on
然而 Service 还是没有正常工作。这种情况下,请告诉我们,以便我们可以帮助调查!
通过
[Slack](/zh/docs/tasks/debug-application-cluster/troubleshooting/#slack) 或者
[Slack](/zh/docs/tasks/debug/overview/#slack) 或者
[Forum](https://discuss.kubernetes.io) 或者
[GitHub](https://github.com/kubernetes/kubernetes)
联系我们。
@@ -1110,7 +1112,6 @@ Contact us on
## {{% heading "whatsnext" %}}
<!--
Visit [troubleshooting document](/docs/tasks/debug-application-cluster/troubleshooting/)
for more information.
Visit [troubleshooting document](/docs/tasks/debug/overview/) for more information.
-->
访问[故障排查文档](/zh/docs/tasks/debug-application-cluster/troubleshooting/) 获取更多信息。
@@ -0,0 +1,71 @@
---
title: 调试 StatefulSet
content_type: task
weight: 30
---
<!--
reviewers:
- bprashanth
- enisoc
- erictune
- foxish
- janetkuo
- kow3ns
- smarterclayton
title: Debug a StatefulSet
content_type: task
weight: 30
-->
<!-- overview -->
<!--
This task shows you how to debug a StatefulSet.
-->
此任务展示如何调试 StatefulSet。
## {{% heading "prerequisites" %}}
<!--
* You need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster.
* You should have a StatefulSet running that you want to investigate.
-->
* 你需要有一个 Kubernetes 集群,已配置好的 kubectl 命令行工具与你的集群进行通信。
* 你应该有一个运行中的 StatefulSet,以便用于调试。
<!-- steps -->
<!--
## Debugging a StatefulSet
In order to list all the pods which belong to a StatefulSet, which have a label `app=myapp` set on them,
you can use the following:
-->
## 调试 StatefulSet {#debuggin-a-statefulset}
StatefulSet 在创建 Pod 时为其设置了 `app=myapp` 标签,列出仅属于某 StatefulSet
的所有 Pod 时,可以使用以下命令:
```shell
kubectl get pods -l app=myapp
```
<!--
If you find that any Pods listed are in `Unknown` or `Terminating` state for an extended period of time,
refer to the [Deleting StatefulSet Pods](/docs/tasks/run-application/delete-stateful-set/) task for
instructions on how to deal with them.
You can debug individual Pods in a StatefulSet using the
[Debugging Pods](/docs/tasks/debug/debug-application/debug-pods/) guide.
-->
如果你发现列出的任何 Pod 长时间处于 `Unknown``Terminating` 状态,请参阅
[删除 StatefulSet Pod](/zh/docs/tasks/run-application/delete-stateful-set/)
了解如何处理它们的说明。
你可以参考[调试 Pod](/zh/docs/tasks/debug/debug-application/debug-pods/)
来调试 StatefulSet 中的各个 Pod。
## {{% heading "whatsnext" %}}
<!--
Learn more about [debugging an init-container](/docs/tasks/debug/debug-application/debug-init-containers/).
-->
* 进一步了解如何[调试 Init 容器](/zh/docs/tasks/debug/debug-application/debug-init-containers/)。
@@ -0,0 +1,175 @@
---
title: 确定 Pod 失败的原因
content_type: task
---
<!--
title: Determine the Reason for Pod Failure
content_type: task
-->
<!-- overview -->
<!--
This page shows how to write and read a Container
termination message.
-->
本文介绍如何编写和读取容器的终止消息。
<!--
Termination messages provide a way for containers to write
information about fatal events to a location where it can
be easily retrieved and surfaced by tools like dashboards
and monitoring software. In most cases, information that you
put in a termination message should also be written to
the general
[Kubernetes logs](/docs/concepts/cluster-administration/logging/).
-->
终止消息为容器提供了一种方法,可以将有关致命事件的信息写入某个位置,
在该位置可以通过仪表板和监控软件等工具轻松检索和显示致命事件。
在大多数情况下,您放入终止消息中的信息也应该写入
[常规 Kubernetes 日志](/zh/docs/concepts/cluster-administration/logging/)。
## {{% heading "prerequisites" %}}
{{< include "task-tutorial-prereqs.md" >}} {{< version-check >}}
<!-- steps -->
<!--
## Writing and reading a termination message
In this exercise, you create a Pod that runs one container.
The configuration file specifies a command that runs when
the container starts.
-->
## 读写终止消息
在本练习中,您将创建运行一个容器的 Pod。
配置文件指定在容器启动时要运行的命令。
{{< codenew file="debug/termination.yaml" >}}
<!-- 1. Create a Pod based on the YAML configuration file: -->
1. 基于 YAML 配置文件创建 Pod:
kubectl apply -f https://k8s.io/examples/debug/termination.yaml
<!--
In the YAML file, in the `command` and `args` fields, you can see that the
container sleeps for 10 seconds and then writes "Sleep expired" to
the `/dev/termination-log` file. After the container writes
the "Sleep expired" message, it terminates.
-->
YAML 文件中,在 `command``args` 字段,你可以看到容器休眠 10 秒然后将 "Sleep expired"
写入 `/dev/termination-log` 文件。
容器写完 "Sleep expired" 消息后就终止了。
<!-- 1. Display information about the Pod: -->
1. 显示 Pod 的信息:
kubectl get pod termination-demo
<!--Repeat the preceding command until the Pod is no longer running.-->
重复前面的命令直到 Pod 不再运行。
<!-- 1. Display detailed information about the Pod: -->
1. 显示 Pod 的详细信息:
kubectl get pod termination-demo --output=yaml
<!--The output includes the "Sleep expired" message:-->
输出结果包含 "Sleep expired" 消息:
apiVersion: v1
kind: Pod
...
lastState:
terminated:
containerID: ...
exitCode: 0
finishedAt: ...
message: |
Sleep expired
...
<!--
1. Use a Go template to filter the output so that it includes
only the termination message:
-->
1. 使用 Go 模板过滤输出结果,使其只含有终止消息:
kubectl get pod termination-demo -o go-template="{{range .status.containerStatuses}}{{.lastState.terminated.message}}{{end}}"
<!--
If you are running a multi-container pod, you can use a Go template to include the container's name. By doing so, you can discover which of the containers is failing:
-->
如果你正在运行多容器 Pod,则可以使用 Go 模板来包含容器的名称。这样,你可以发现哪些容器出现故障:
```shell
kubectl get pod multi-container-pod -o go-template='{{range .status.containerStatuses}}{{printf "%s:\n%s\n\n" .name .lastState.terminated.message}}{{end}}'
```
<!--
## Customizing the termination message
Kubernetes retrieves termination messages from the termination message file
specified in the `terminationMessagePath` field of a Container, which as a default
value of `/dev/termination-log`. By customizing this field, you can tell Kubernetes
to use a different file. Kubernetes use the contents from the specified file to
populate the Container's status message on both success and failure.
-->
## 定制终止消息
Kubernetes 从容器的 `terminationMessagePath` 字段中指定的终止消息文件中检索终止消息,
默认值为 `/dev/termination-log`
通过定制这个字段,您可以告诉 Kubernetes 使用不同的文件。
Kubernetes 使用指定文件中的内容在成功和失败时填充容器的状态消息。
<!--
In the following example, the container writes termination messages to
`/tmp/my-log` for Kubernetes to retrieve:
-->
在下例中,容器将终止消息写入 `/tmp/my-log` 给 Kubernetes 来接收:
```yaml
apiVersion: v1
kind: Pod
metadata:
name: msg-path-demo
spec:
containers:
- name: msg-path-demo-container
image: debian
terminationMessagePath: "/tmp/my-log"
```
<!--
Moreover, users can set the `terminationMessagePolicy` field of a Container for
further customization. This field defaults to "`File`" which means the termination
messages are retrieved only from the termination message file. By setting the
`terminationMessagePolicy` to "`FallbackToLogsOnError`", you can tell Kubernetes
to use the last chunk of container log output if the termination message file
is empty and the container exited with an error. The log output is limited to
2048 bytes or 80 lines, whichever is smaller.
-->
此外,用户可以设置容器的 `terminationMessagePolicy` 字段,以便进一步自定义。
此字段默认为 "`File`",这意味着仅从终止消息文件中检索终止消息。
通过将 `terminationMessagePolicy` 设置为 "`FallbackToLogsOnError`",你就可以告诉 Kubernetes,在容器因错误退出时,如果终止消息文件为空,则使用容器日志输出的最后一块作为终止消息。
日志输出限制为 2048 字节或 80 行,以较小者为准。
## {{% heading "whatsnext" %}}
<!--
* See the `terminationMessagePath` field in
[Container](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#container-v1-core).
* Learn about [retrieving logs](/docs/concepts/cluster-administration/logging/).
* Learn about [Go templates](https://golang.org/pkg/text/template/).
-->
* 参考 [Container](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#container-v1-core)
资源的 `terminationMessagePath` 字段。
* 了解[接收日志](/zh/docs/concepts/cluster-administration/logging/)。
* 了解 [Go 模版](https://golang.org/pkg/text/template/)。
@@ -0,0 +1,457 @@
---
title: 集群故障排查
description: 调试常见的集群问题。
weight: 20
no_list: true
---
<!--
reviewers:
- davidopp
title: "Troubleshooting Clusters"
description: Debugging common cluster issues.
weight: 20
no_list: true
-->
<!-- overview -->
<!--
This doc is about cluster troubleshooting; we assume you have already ruled out your application as the root cause of the
problem you are experiencing. See
the [application troubleshooting guide](/docs/tasks/debug/debug-application/) for tips on application debugging.
You may also visit the [troubleshooting overview document](/docs/tasks/debug/) for more information.
-->
本篇文档是介绍集群故障排查的;我们假设对于你碰到的问题,你已经排除了是由应用程序造成的。
对于应用的调试,请参阅[应用故障排查指南](/zh/docs/tasks/debug/debug-application/)。
你也可以访问[故障排查](/zh/docs/tasks/debug/)来获取更多的信息。
<!-- body -->
<!--
## Listing your cluster
The first thing to debug in your cluster is if your nodes are all registered correctly.
Run the following command:
-->
## 列举集群节点
调试的第一步是查看所有的节点是否都已正确注册。
运行以下命令:
```shell
kubectl get nodes
```
<!--
And verify that all of the nodes you expect to see are present and that they are all in the `Ready` state.
To get detailed information about the overall health of your cluster, you can run:
-->
验证你所希望看见的所有节点都能够显示出来,并且都处于 `Ready` 状态。
为了了解你的集群的总体健康状况详情,你可以运行:
```shell
kubectl cluster-info dump
```
<!--
### Example: debugging a down/unreachable node
Sometimes when debugging it can be useful to look at the status of a node -- for example, because you've noticed strange behavior of a Pod that's running on the node, or to find out why a Pod won't schedule onto the node. As with Pods, you can use `kubectl describe node` and `kubectl get node -o yaml` to retrieve detailed information about nodes. For example, here's what you'll see if a node is down (disconnected from the network, or kubelet dies and won't restart, etc.). Notice the events that show the node is NotReady, and also notice that the pods are no longer running (they are evicted after five minutes of NotReady status).
-->
### 示例:调试关闭/无法访问的节点
有时在调试时查看节点的状态很有用——例如,因为你注意到在节点上运行的 Pod 的奇怪行为,
或者找出为什么 Pod 不会调度到节点上。与 Pod 一样,你可以使用 `kubectl describe node`
`kubectl get node -o yaml` 来检索有关节点的详细信息。
例如,如果节点关闭(与网络断开连接,或者 kubelet 进程挂起并且不会重新启动等),
你将看到以下内容。请注意显示节点为 NotReady 的事件,并注意 Pod 不再运行(它们在 NotReady 状态五分钟后被驱逐)。
```shell
kubectl get nodes
```
```none
NAME STATUS ROLES AGE VERSION
kube-worker-1 NotReady <none> 1h v1.23.3
kubernetes-node-bols Ready <none> 1h v1.23.3
kubernetes-node-st6x Ready <none> 1h v1.23.3
kubernetes-node-unaj Ready <none> 1h v1.23.3
```
```shell
kubectl describe node kube-worker-1
```
```none
Name: kube-worker-1
Roles: <none>
Labels: beta.kubernetes.io/arch=amd64
beta.kubernetes.io/os=linux
kubernetes.io/arch=amd64
kubernetes.io/hostname=kube-worker-1
kubernetes.io/os=linux
Annotations: kubeadm.alpha.kubernetes.io/cri-socket: /run/containerd/containerd.sock
node.alpha.kubernetes.io/ttl: 0
volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp: Thu, 17 Feb 2022 16:46:30 -0500
Taints: node.kubernetes.io/unreachable:NoExecute
node.kubernetes.io/unreachable:NoSchedule
Unschedulable: false
Lease:
HolderIdentity: kube-worker-1
AcquireTime: <unset>
RenewTime: Thu, 17 Feb 2022 17:13:09 -0500
Conditions:
Type Status LastHeartbeatTime LastTransitionTime Reason Message
---- ------ ----------------- ------------------ ------ -------
NetworkUnavailable False Thu, 17 Feb 2022 17:09:13 -0500 Thu, 17 Feb 2022 17:09:13 -0500 WeaveIsUp Weave pod has set this
MemoryPressure Unknown Thu, 17 Feb 2022 17:12:40 -0500 Thu, 17 Feb 2022 17:13:52 -0500 NodeStatusUnknown Kubelet stopped posting node status.
DiskPressure Unknown Thu, 17 Feb 2022 17:12:40 -0500 Thu, 17 Feb 2022 17:13:52 -0500 NodeStatusUnknown Kubelet stopped posting node status.
PIDPressure Unknown Thu, 17 Feb 2022 17:12:40 -0500 Thu, 17 Feb 2022 17:13:52 -0500 NodeStatusUnknown Kubelet stopped posting node status.
Ready Unknown Thu, 17 Feb 2022 17:12:40 -0500 Thu, 17 Feb 2022 17:13:52 -0500 NodeStatusUnknown Kubelet stopped posting node status.
Addresses:
InternalIP: 192.168.0.113
Hostname: kube-worker-1
Capacity:
cpu: 2
ephemeral-storage: 15372232Ki
hugepages-2Mi: 0
memory: 2025188Ki
pods: 110
Allocatable:
cpu: 2
ephemeral-storage: 14167048988
hugepages-2Mi: 0
memory: 1922788Ki
pods: 110
System Info:
Machine ID: 9384e2927f544209b5d7b67474bbf92b
System UUID: aa829ca9-73d7-064d-9019-df07404ad448
Boot ID: 5a295a03-aaca-4340-af20-1327fa5dab5c
Kernel Version: 5.13.0-28-generic
OS Image: Ubuntu 21.10
Operating System: linux
Architecture: amd64
Container Runtime Version: containerd://1.5.9
Kubelet Version: v1.23.3
Kube-Proxy Version: v1.23.3
Non-terminated Pods: (4 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits Age
--------- ---- ------------ ---------- --------------- ------------- ---
default nginx-deployment-67d4bdd6f5-cx2nz 500m (25%) 500m (25%) 128Mi (6%) 128Mi (6%) 23m
default nginx-deployment-67d4bdd6f5-w6kd7 500m (25%) 500m (25%) 128Mi (6%) 128Mi (6%) 23m
kube-system kube-proxy-dnxbz 0 (0%) 0 (0%) 0 (0%) 0 (0%) 28m
kube-system weave-net-gjxxp 100m (5%) 0 (0%) 200Mi (10%) 0 (0%) 28m
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------
cpu 1100m (55%) 1 (50%)
memory 456Mi (24%) 256Mi (13%)
ephemeral-storage 0 (0%) 0 (0%)
hugepages-2Mi 0 (0%) 0 (0%)
Events:
...
```
```shell
kubectl get node kube-worker-1 -o yaml
```
```yaml
apiVersion: v1
kind: Node
metadata:
annotations:
kubeadm.alpha.kubernetes.io/cri-socket: /run/containerd/containerd.sock
node.alpha.kubernetes.io/ttl: "0"
volumes.kubernetes.io/controller-managed-attach-detach: "true"
creationTimestamp: "2022-02-17T21:46:30Z"
labels:
beta.kubernetes.io/arch: amd64
beta.kubernetes.io/os: linux
kubernetes.io/arch: amd64
kubernetes.io/hostname: kube-worker-1
kubernetes.io/os: linux
name: kube-worker-1
resourceVersion: "4026"
uid: 98efe7cb-2978-4a0b-842a-1a7bf12c05f8
spec: {}
status:
addresses:
- address: 192.168.0.113
type: InternalIP
- address: kube-worker-1
type: Hostname
allocatable:
cpu: "2"
ephemeral-storage: "14167048988"
hugepages-2Mi: "0"
memory: 1922788Ki
pods: "110"
capacity:
cpu: "2"
ephemeral-storage: 15372232Ki
hugepages-2Mi: "0"
memory: 2025188Ki
pods: "110"
conditions:
- lastHeartbeatTime: "2022-02-17T22:20:32Z"
lastTransitionTime: "2022-02-17T22:20:32Z"
message: Weave pod has set this
reason: WeaveIsUp
status: "False"
type: NetworkUnavailable
- lastHeartbeatTime: "2022-02-17T22:20:15Z"
lastTransitionTime: "2022-02-17T22:13:25Z"
message: kubelet has sufficient memory available
reason: KubeletHasSufficientMemory
status: "False"
type: MemoryPressure
- lastHeartbeatTime: "2022-02-17T22:20:15Z"
lastTransitionTime: "2022-02-17T22:13:25Z"
message: kubelet has no disk pressure
reason: KubeletHasNoDiskPressure
status: "False"
type: DiskPressure
- lastHeartbeatTime: "2022-02-17T22:20:15Z"
lastTransitionTime: "2022-02-17T22:13:25Z"
message: kubelet has sufficient PID available
reason: KubeletHasSufficientPID
status: "False"
type: PIDPressure
- lastHeartbeatTime: "2022-02-17T22:20:15Z"
lastTransitionTime: "2022-02-17T22:15:15Z"
message: kubelet is posting ready status. AppArmor enabled
reason: KubeletReady
status: "True"
type: Ready
daemonEndpoints:
kubeletEndpoint:
Port: 10250
nodeInfo:
architecture: amd64
bootID: 22333234-7a6b-44d4-9ce1-67e31dc7e369
containerRuntimeVersion: containerd://1.5.9
kernelVersion: 5.13.0-28-generic
kubeProxyVersion: v1.23.3
kubeletVersion: v1.23.3
machineID: 9384e2927f544209b5d7b67474bbf92b
operatingSystem: linux
osImage: Ubuntu 21.10
systemUUID: aa829ca9-73d7-064d-9019-df07404ad448
```
<!--
For now, digging deeper into the cluster requires logging into the relevant machines. Here are the locations
of the relevant log files. On systemd-based systems, you may need to use `journalctl` instead of examining log files.
-->
## 查看日志
目前,深入挖掘集群需要登录相关机器。以下是相关日志文件的位置。
在基于 systemd 的系统上,你可能需要使用 `journalctl` 而不是检查日志文件。
<!--
### Control Plane nodes
* `/var/log/kube-apiserver.log` - API Server, responsible for serving the API
* `/var/log/kube-scheduler.log` - Scheduler, responsible for making scheduling decisions
* `/var/log/kube-controller-manager.log` - a component that runs most Kubernetes built-in {{<glossary_tooltip text="controllers" term_id="controller">}}, with the notable exception of scheduling (the kube-scheduler handles scheduling).
-->
### 控制平面节点
* `/var/log/kube-apiserver.log` —— API 服务器 API
* `/var/log/kube-scheduler.log` —— 调度器,负责制定调度决策
* `/var/log/kube-controller-manager.log` —— 运行大多数 Kubernetes
内置{{<glossary_tooltip text="控制器" term_id="controller">}}的组件,除了调度(kube-scheduler 处理调度)。
<!--
### Worker Nodes
* `/var/log/kubelet.log` - logs from the kubelet, responsible for running containers on the node
* `/var/log/kube-proxy.log` - logs from `kube-proxy`, which is responsible for directing traffic to Service endpoints
-->
### 工作节点
* `/var/log/kubelet.log` —— 来自 `kubelet` 的日志,负责在节点运行容器
* `/var/log/kube-proxy.log` —— 来自 `kube-proxy` 的日志, 负责将流量转发到服务端点
<!--
## Cluster failure modes
This is an incomplete list of things that could go wrong, and how to adjust your cluster setup to mitigate the problems.
-->
## 集群故障模式
这是可能出错的事情的不完整列表,以及如何调整集群设置以缓解问题。
<!--
### Contributing causes
- VM(s) shutdown
- Network partition within cluster, or between cluster and users
- Crashes in Kubernetes software
- Data loss or unavailability of persistent storage (e.g. GCE PD or AWS EBS volume)
- Operator error, for example misconfigured Kubernetes software or application software
-->
### 贡献原因
- 虚拟机关闭
- 集群内或集群与用户之间的网络分区
- Kubernetes 软件崩溃
- 持久存储(例如 GCE PD 或 AWS EBS 卷)的数据丢失或不可用
- 操作员错误,例如配置错误的 Kubernetes 软件或应用程序软件
<!--
### Specific scenarios:
- Apiserver VM shutdown or apiserver crashing
- Results
- unable to stop, update, or start new pods, services, replication controller
- existing pods and services should continue to work normally, unless they depend on the Kubernetes API
- Apiserver backing storage lost
- Results
- apiserver should fail to come up
- kubelets will not be able to reach it but will continue to run the same pods and provide the same service proxying
- manual recovery or recreation of apiserver state necessary before apiserver is restarted
-->
### 具体情况
- API 服务器所在的 VM 关机或者 API 服务器崩溃
- 结果
- 不能停止、更新或者启动新的 Pod、服务或副本控制器
- 现有的 Pod 和服务在不依赖 Kubernetes API 的情况下应该能继续正常工作
- API 服务器的后端存储丢失
- 结果
- API 服务器应该不能启动
- kubelet 将不能访问 API 服务器,但是能够继续运行之前的 Pod 和提供相同的服务代理
- 在 API 服务器重启之前,需要手动恢复或者重建 API 服务器的状态
<!--
- Supporting services (node controller, replication controller manager, scheduler, etc) VM shutdown or crashes
- currently those are colocated with the apiserver, and their unavailability has similar consequences as apiserver
- in future, these will be replicated as well and may not be co-located
- they do not have their own persistent state
- Individual node (VM or physical machine) shuts down
- Results
- pods on that Node stop running
- Network partition
- Results
- partition A thinks the nodes in partition B are down; partition B thinks the apiserver is down. (Assuming the master VM ends up in partition A.)
-->
- Kubernetes 服务组件(节点控制器、副本控制器管理器、调度器等)所在的 VM 关机或者崩溃
- 当前,这些控制器是和 API 服务器在一起运行的,它们不可用的现象是与 API 服务器类似的
- 将来,这些控制器也会复制为多份,并且可能不在运行于同一节点上
- 它们没有自己的持久状态
- 单个节点(VM 或者物理机)关机
- 结果
- 此节点上的所有 Pod 都停止运行
- 网络分裂
- 结果
- 分区 A 认为分区 B 中所有的节点都已宕机;分区 B 认为 API 服务器宕机
(假定主控节点所在的 VM 位于分区 A 内)。
<!--
- Kubelet software fault
- Results
- crashing kubelet cannot start new pods on the node
- kubelet might delete the pods or not
- node marked unhealthy
- replication controllers start new pods elsewhere
- Cluster operator error
- Results
- loss of pods, services, etc
- lost of apiserver backing store
- users unable to read API
- etc.
-->
- kubelet 软件故障
- 结果
- 崩溃的 kubelet 就不能在其所在的节点上启动新的 Pod
- kubelet 可能删掉 Pod 或者不删
- 节点被标识为非健康态
- 副本控制器会在其它的节点上启动新的 Pod
- 集群操作错误
- 结果
- 丢失 Pod 或服务等等
- 丢失 API 服务器的后端存储
- 用户无法读取API
- 等等
<!--
### Mitigations:
- Action: Use IaaS provider's automatic VM restarting feature for IaaS VMs
- Mitigates: Apiserver VM shutdown or apiserver crashing
- Mitigates: Supporting services VM shutdown or crashes
- Action: Use IaaS providers reliable storage (e.g. GCE PD or AWS EBS volume) for VMs with apiserver+etcd
- Mitigates: Apiserver backing storage lost
- Action: Use [high-availability](/docs/setup/production-environment/tools/kubeadm/high-availability/) configuration
- Mitigates: Control plane node shutdown or control plane components (scheduler, API server, controller-manager) crashing
- Will tolerate one or more simultaneous node or component failures
- Mitigates: API server backing storage (i.e., etcd's data directory) lost
- Assumes HA (highly-available) etcd configuration
-->
### 缓解措施
- 措施:对于 IaaS 上的 VM,使用 IaaS 的自动 VM 重启功能
- 缓解:API 服务器 VM 关机或 API 服务器崩溃
- 缓解:Kubernetes 服务组件所在的 VM 关机或崩溃
- 措施: 对于运行 API 服务器和 etcd 的 VM,使用 IaaS 提供的可靠的存储(例如 GCE PD 或者 AWS EBS 卷)
- 缓解:API 服务器后端存储的丢失
- 措施:使用[高可用性](/zh/docs/setup/production-environment/tools/kubeadm/high-availability/)的配置
- 缓解:主控节点 VM 关机或者主控节点组件(调度器、API 服务器、控制器管理器)崩馈
- 将容许一个或多个节点或组件同时出现故障
- 缓解:API 服务器后端存储(例如 etcd 的数据目录)丢失
- 假定你使用了高可用的 etcd 配置
<!--
- Action: Snapshot apiserver PDs/EBS-volumes periodically
- Mitigates: Apiserver backing storage lost
- Mitigates: Some cases of operator error
- Mitigates: Some cases of Kubernetes software fault
- Action: use replication controller and services in front of pods
- Mitigates: Node shutdown
- Mitigates: Kubelet software fault
- Action: applications (containers) designed to tolerate unexpected restarts
- Mitigates: Node shutdown
- Mitigates: Kubelet software fault
-->
- 措施:定期对 API 服务器的 PDs/EBS 卷执行快照操作
- 缓解:API 服务器后端存储丢失
- 缓解:一些操作错误的场景
- 缓解:一些 Kubernetes 软件本身故障的场景
- 措施:在 Pod 的前面使用副本控制器或服务
- 缓解:节点关机
- 缓解:kubelet 软件故障
- 措施:应用(容器)设计成容许异常重启
- 缓解:节点关机
- 缓解:kubelet 软件故障
## {{% heading "whatsnext" %}}
<!--
* Learn about the metrics available in the [Resource Metrics Pipeline](resource-metrics-pipeline)
* Discover additional tools for [monitoring resource usage](resource-usage-monitoring)
* Use Node Problem Detector to [monitor node health](monitor-node-health)
* Use `crictl` to [debug Kubernetes nodes](crictl)
* Get more information about [Kubernetes auditing](audit)
* Use `telepresence` to [develop and debug services locally](local-debugging)
-->
* 了解 [资源指标管道](resource-metrics-pipeline) 中可用的指标
* 发现用于[监控资源使用](resource-usage-monitoring)的其他工具
* 使用节点问题检测器[监控节点健康](monitor-node-health)
* 使用 `crictl` 来[调试 Kubernetes 节点](crictl)
* 获取更多关于 [Kubernetes 审计](audit)的信息
* 使用 `telepresence` [本地开发和调试服务](local-debugging)
@@ -13,12 +13,12 @@ content_type: task
{{% thirdparty-content %}}
<!--
Kubernetes applications usually consist of multiple, separate services, each running in its own container. Developing and debugging these services on a remote Kubernetes cluster can be cumbersome, requiring you to [get a shell on a running container](/docs/tasks/debug-application-cluster/get-shell-running-container/) in order to run debugging tools.
Kubernetes applications usually consist of multiple, separate services, each running in its own container. Developing and debugging these services on a remote Kubernetes cluster can be cumbersome, requiring you to [get a shell on a running container](/docs/tasks/debug/debug-application/get-shell-running-container/) in order to run debugging tools.
-->
Kubernetes 应用程序通常由多个独立的服务组成,每个服务都在自己的容器中运行。
在远端的 Kubernetes 集群上开发和调试这些服务可能很麻烦,需要
[在运行的容器上打开 Shell](/zh/docs/tasks/debug-application-cluster/get-shell-running-container/)
在远端的 Kubernetes 集群上开发和调试这些服务可能很麻烦,
需要[在运行的容器上打开 Shell](/zh/docs/tasks/debug/debug-application/get-shell-running-container/)
以运行调试工具。
<!--
@@ -47,12 +47,12 @@ The Metrics API, and the metrics pipeline that it enables, only offers the minim
CPU and memory metrics to enable automatic scaling using HPA and / or VPA.
If you would like to provide a more complete set of metrics, you can complement
the simpler Metrics API by deploying a second
[metrics pipeline](/docs/tasks/debug-application-cluster/resource-usage-monitoring/#full-metrics-pipeline)
[metrics pipeline](/docs/tasks/debug/debug-cluster/resource-usage-monitoring/#full-metrics-pipeline)
that uses the _Custom Metrics API_.
-->
Metrics API 及其启用的指标管道仅提供最少的 CPU 和内存指标,以启用使用 HPA 和/或 VPA 的自动扩展。
如果你想提供更完整的指标集,你可以通过部署使用 _Custom Metrics API_ 的第二个
[指标管道](/zh/docs/tasks/debug-application-cluster/resource-usage-monitoring/#full-metrics-pipeline) 来作为简单的 Metrics API 的补充。
[指标管道](/zh/docs/tasks/debug/debug-cluster/resource-usage-monitoring/#full-metrics-pipeline)来作为简单的 Metrics API 的补充。
{{< /note >}}
<!--