Update debug-cluster and crictl files to get them consistent with the style guidelines (#13183)
This commit is contained in:
committed by
Kubernetes Prow Robot
parent
9c44e1ea2e
commit
d85ac2686f
@@ -1,5 +1,5 @@
|
|||||||
---
|
---
|
||||||
title: "Monitor, Log, and Debug"
|
title: "Monitoring, Logging, and Debugging"
|
||||||
weight: 80
|
weight: 80
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
@@ -51,10 +51,9 @@ You can also specify timeout values when connecting to the server and enable or
|
|||||||
disable debugging, by specifying `timeout` or `debug` values in the configuration
|
disable debugging, by specifying `timeout` or `debug` values in the configuration
|
||||||
file or using the `--timeout` and `--debug` command-line flags.
|
file or using the `--timeout` and `--debug` command-line flags.
|
||||||
|
|
||||||
To view or edit the current configuration, view or edit the contents of
|
To view or edit the current configuration, view or edit the contents of `/etc/crictl.yaml`.
|
||||||
`/etc/crictl.yaml`.
|
|
||||||
|
|
||||||
```sh
|
```shell
|
||||||
cat /etc/crictl.yaml
|
cat /etc/crictl.yaml
|
||||||
runtime-endpoint: unix:///var/run/dockershim.sock
|
runtime-endpoint: unix:///var/run/dockershim.sock
|
||||||
image-endpoint: unix:///var/run/dockershim.sock
|
image-endpoint: unix:///var/run/dockershim.sock
|
||||||
@@ -76,10 +75,12 @@ general purpose workflow tool, but a tool that is useful for debugging.
|
|||||||
|
|
||||||
List all pods:
|
List all pods:
|
||||||
|
|
||||||
```bash
|
```shell
|
||||||
crictl pods
|
crictl pods
|
||||||
```
|
```
|
||||||
```none
|
The output is similar to this:
|
||||||
|
|
||||||
|
```
|
||||||
POD ID CREATED STATE NAME NAMESPACE ATTEMPT
|
POD ID CREATED STATE NAME NAMESPACE ATTEMPT
|
||||||
926f1b5a1d33a About a minute ago Ready sh-84d7dcf559-4r2gq default 0
|
926f1b5a1d33a About a minute ago Ready sh-84d7dcf559-4r2gq default 0
|
||||||
4dccb216c4adb About a minute ago Ready nginx-65899c769f-wv2gp default 0
|
4dccb216c4adb About a minute ago Ready nginx-65899c769f-wv2gp default 0
|
||||||
@@ -89,20 +90,24 @@ a86316e96fa89 17 hours ago Ready kube-proxy-gblk4
|
|||||||
|
|
||||||
List pods by name:
|
List pods by name:
|
||||||
|
|
||||||
```bash
|
```shell
|
||||||
crictl pods --name nginx-65899c769f-wv2gp
|
crictl pods --name nginx-65899c769f-wv2gp
|
||||||
```
|
```
|
||||||
```none
|
The output is similar to this:
|
||||||
|
|
||||||
|
```
|
||||||
POD ID CREATED STATE NAME NAMESPACE ATTEMPT
|
POD ID CREATED STATE NAME NAMESPACE ATTEMPT
|
||||||
4dccb216c4adb 2 minutes ago Ready nginx-65899c769f-wv2gp default 0
|
4dccb216c4adb 2 minutes ago Ready nginx-65899c769f-wv2gp default 0
|
||||||
```
|
```
|
||||||
|
|
||||||
List pods by label:
|
List pods by label:
|
||||||
|
|
||||||
```bash
|
```shell
|
||||||
crictl pods --label run=nginx
|
crictl pods --label run=nginx
|
||||||
```
|
```
|
||||||
```none
|
The output is similar to this:
|
||||||
|
|
||||||
|
```
|
||||||
POD ID CREATED STATE NAME NAMESPACE ATTEMPT
|
POD ID CREATED STATE NAME NAMESPACE ATTEMPT
|
||||||
4dccb216c4adb 2 minutes ago Ready nginx-65899c769f-wv2gp default 0
|
4dccb216c4adb 2 minutes ago Ready nginx-65899c769f-wv2gp default 0
|
||||||
```
|
```
|
||||||
@@ -111,10 +116,12 @@ POD ID CREATED STATE NAME
|
|||||||
|
|
||||||
List all images:
|
List all images:
|
||||||
|
|
||||||
```bash
|
```shell
|
||||||
crictl images
|
crictl images
|
||||||
```
|
```
|
||||||
```none
|
The output is similar to this:
|
||||||
|
|
||||||
|
```
|
||||||
IMAGE TAG IMAGE ID SIZE
|
IMAGE TAG IMAGE ID SIZE
|
||||||
busybox latest 8c811b4aec35f 1.15MB
|
busybox latest 8c811b4aec35f 1.15MB
|
||||||
k8s-gcrio.azureedge.net/hyperkube-amd64 v1.10.3 e179bbfe5d238 665MB
|
k8s-gcrio.azureedge.net/hyperkube-amd64 v1.10.3 e179bbfe5d238 665MB
|
||||||
@@ -124,20 +131,24 @@ nginx latest cd5239a0906a6
|
|||||||
|
|
||||||
List images by repository:
|
List images by repository:
|
||||||
|
|
||||||
```bash
|
```shell
|
||||||
crictl images nginx
|
crictl images nginx
|
||||||
```
|
```
|
||||||
```none
|
The output is similar to this:
|
||||||
|
|
||||||
|
```
|
||||||
IMAGE TAG IMAGE ID SIZE
|
IMAGE TAG IMAGE ID SIZE
|
||||||
nginx latest cd5239a0906a6 109MB
|
nginx latest cd5239a0906a6 109MB
|
||||||
```
|
```
|
||||||
|
|
||||||
Only list image IDs:
|
Only list image IDs:
|
||||||
|
|
||||||
```bash
|
```shell
|
||||||
crictl images -q
|
crictl images -q
|
||||||
```
|
```
|
||||||
```none
|
The output is similar to this:
|
||||||
|
|
||||||
|
```
|
||||||
sha256:8c811b4aec35f259572d0f79207bc0678df4c736eeec50bc9fec37ed936a472a
|
sha256:8c811b4aec35f259572d0f79207bc0678df4c736eeec50bc9fec37ed936a472a
|
||||||
sha256:e179bbfe5d238de6069f3b03fccbecc3fb4f2019af741bfff1233c4d7b2970c5
|
sha256:e179bbfe5d238de6069f3b03fccbecc3fb4f2019af741bfff1233c4d7b2970c5
|
||||||
sha256:da86e6ba6ca197bf6bc5e9d900febd906b133eaa4750e6bed647b0fbe50ed43e
|
sha256:da86e6ba6ca197bf6bc5e9d900febd906b133eaa4750e6bed647b0fbe50ed43e
|
||||||
@@ -148,10 +159,12 @@ sha256:cd5239a0906a6ccf0562354852fae04bc5b52d72a2aff9a871ddb6bd57553569
|
|||||||
|
|
||||||
List all containers:
|
List all containers:
|
||||||
|
|
||||||
```bash
|
```shell
|
||||||
crictl ps -a
|
crictl ps -a
|
||||||
```
|
```
|
||||||
```none
|
The output is similar to this:
|
||||||
|
|
||||||
|
```
|
||||||
CONTAINER ID IMAGE CREATED STATE NAME ATTEMPT
|
CONTAINER ID IMAGE CREATED STATE NAME ATTEMPT
|
||||||
1f73f2d81bf98 busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 7 minutes ago Running sh 1
|
1f73f2d81bf98 busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 7 minutes ago Running sh 1
|
||||||
9c5951df22c78 busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 8 minutes ago Exited sh 0
|
9c5951df22c78 busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 8 minutes ago Exited sh 0
|
||||||
@@ -161,10 +174,12 @@ CONTAINER ID IMAGE
|
|||||||
|
|
||||||
List running containers:
|
List running containers:
|
||||||
|
|
||||||
```bash
|
```
|
||||||
crictl ps
|
crictl ps
|
||||||
```
|
```
|
||||||
```none
|
The output is similar to this:
|
||||||
|
|
||||||
|
```
|
||||||
CONTAINER ID IMAGE CREATED STATE NAME ATTEMPT
|
CONTAINER ID IMAGE CREATED STATE NAME ATTEMPT
|
||||||
1f73f2d81bf98 busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 6 minutes ago Running sh 1
|
1f73f2d81bf98 busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47 6 minutes ago Running sh 1
|
||||||
87d3992f84f74 nginx@sha256:d0a8828cccb73397acb0073bf34f4d7d8aa315263f1e7806bf8c55d8ac139d5f 7 minutes ago Running nginx 0
|
87d3992f84f74 nginx@sha256:d0a8828cccb73397acb0073bf34f4d7d8aa315263f1e7806bf8c55d8ac139d5f 7 minutes ago Running nginx 0
|
||||||
@@ -173,10 +188,12 @@ CONTAINER ID IMAGE
|
|||||||
|
|
||||||
### Execute a command in a running container
|
### Execute a command in a running container
|
||||||
|
|
||||||
```bash
|
```shell
|
||||||
crictl exec -i -t 1f73f2d81bf98 ls
|
crictl exec -i -t 1f73f2d81bf98 ls
|
||||||
```
|
```
|
||||||
```none
|
The output is similar to this:
|
||||||
|
|
||||||
|
```
|
||||||
bin dev etc home proc root sys tmp usr var
|
bin dev etc home proc root sys tmp usr var
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -184,10 +201,12 @@ bin dev etc home proc root sys tmp usr var
|
|||||||
|
|
||||||
Get all container logs:
|
Get all container logs:
|
||||||
|
|
||||||
```bash
|
```shell
|
||||||
crictl logs 87d3992f84f74
|
crictl logs 87d3992f84f74
|
||||||
```
|
```
|
||||||
```none
|
The output is similar to this:
|
||||||
|
|
||||||
|
```
|
||||||
10.240.0.96 - - [06/Jun/2018:02:45:49 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.47.0" "-"
|
10.240.0.96 - - [06/Jun/2018:02:45:49 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.47.0" "-"
|
||||||
10.240.0.96 - - [06/Jun/2018:02:45:50 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.47.0" "-"
|
10.240.0.96 - - [06/Jun/2018:02:45:50 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.47.0" "-"
|
||||||
10.240.0.96 - - [06/Jun/2018:02:45:51 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.47.0" "-"
|
10.240.0.96 - - [06/Jun/2018:02:45:51 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.47.0" "-"
|
||||||
@@ -195,10 +214,12 @@ crictl logs 87d3992f84f74
|
|||||||
|
|
||||||
Get only the latest `N` lines of logs:
|
Get only the latest `N` lines of logs:
|
||||||
|
|
||||||
```bash
|
```shell
|
||||||
crictl logs --tail=1 87d3992f84f74
|
crictl logs --tail=1 87d3992f84f74
|
||||||
```
|
```
|
||||||
```none
|
The output is similar to this:
|
||||||
|
|
||||||
|
```
|
||||||
10.240.0.96 - - [06/Jun/2018:02:45:51 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.47.0" "-"
|
10.240.0.96 - - [06/Jun/2018:02:45:51 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.47.0" "-"
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -226,7 +247,7 @@ deleted by the Kubelet.
|
|||||||
|
|
||||||
2. Use the `crictl runp` command to apply the JSON and run the sandbox.
|
2. Use the `crictl runp` command to apply the JSON and run the sandbox.
|
||||||
|
|
||||||
```bash
|
```shell
|
||||||
crictl runp pod-config.json
|
crictl runp pod-config.json
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -240,7 +261,7 @@ deleted by the Kubelet.
|
|||||||
|
|
||||||
1. Pull a busybox image
|
1. Pull a busybox image
|
||||||
|
|
||||||
```bash
|
```shell
|
||||||
crictl pull busybox
|
crictl pull busybox
|
||||||
Image is up to date for busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47
|
Image is up to date for busybox@sha256:141c253bc4c3fd0a201d32dc1f493bcf3fff003b6df416dea4f41046e0f37d47
|
||||||
```
|
```
|
||||||
@@ -284,17 +305,19 @@ deleted by the Kubelet.
|
|||||||
container config file, and the pod config file. The ID of the container is
|
container config file, and the pod config file. The ID of the container is
|
||||||
returned.
|
returned.
|
||||||
|
|
||||||
```bash
|
```shell
|
||||||
crictl create f84dd361f8dc51518ed291fbadd6db537b0496536c1d2d6c05ff943ce8c9a54f container-config.json pod-config.json
|
crictl create f84dd361f8dc51518ed291fbadd6db537b0496536c1d2d6c05ff943ce8c9a54f container-config.json pod-config.json
|
||||||
```
|
```
|
||||||
|
|
||||||
4. List all containers and verify that the newly-created container has its
|
4. List all containers and verify that the newly-created container has its
|
||||||
state set to `Created`.
|
state set to `Created`.
|
||||||
|
|
||||||
```bash
|
```shell
|
||||||
crictl ps -a
|
crictl ps -a
|
||||||
```
|
```
|
||||||
```none
|
The output is similar to this:
|
||||||
|
|
||||||
|
```
|
||||||
CONTAINER ID IMAGE CREATED STATE NAME ATTEMPT
|
CONTAINER ID IMAGE CREATED STATE NAME ATTEMPT
|
||||||
3e025dd50a72d busybox 32 seconds ago Created busybox 0
|
3e025dd50a72d busybox 32 seconds ago Created busybox 0
|
||||||
```
|
```
|
||||||
@@ -303,19 +326,23 @@ deleted by the Kubelet.
|
|||||||
|
|
||||||
To start a container, pass its ID to `crictl start`:
|
To start a container, pass its ID to `crictl start`:
|
||||||
|
|
||||||
```bash
|
```shell
|
||||||
crictl start 3e025dd50a72d956c4f14881fbb5b1080c9275674e95fb67f965f6478a957d60
|
crictl start 3e025dd50a72d956c4f14881fbb5b1080c9275674e95fb67f965f6478a957d60
|
||||||
```
|
```
|
||||||
```none
|
The output is similar to this:
|
||||||
|
|
||||||
|
```
|
||||||
3e025dd50a72d956c4f14881fbb5b1080c9275674e95fb67f965f6478a957d60
|
3e025dd50a72d956c4f14881fbb5b1080c9275674e95fb67f965f6478a957d60
|
||||||
```
|
```
|
||||||
|
|
||||||
Check the container has its state set to `Running`.
|
Check the container has its state set to `Running`.
|
||||||
|
|
||||||
```bash
|
```shell
|
||||||
crictl ps
|
crictl ps
|
||||||
```
|
```
|
||||||
```none
|
The output is similar to this:
|
||||||
|
|
||||||
|
```
|
||||||
CONTAINER ID IMAGE CREATED STATE NAME ATTEMPT
|
CONTAINER ID IMAGE CREATED STATE NAME ATTEMPT
|
||||||
3e025dd50a72d busybox About a minute ago Running busybox 0
|
3e025dd50a72d busybox About a minute ago Running busybox 0
|
||||||
```
|
```
|
||||||
|
|||||||
@@ -36,20 +36,20 @@ of the relevant log files. (note that on systemd-based systems, you may need to
|
|||||||
|
|
||||||
### Master
|
### Master
|
||||||
|
|
||||||
* /var/log/kube-apiserver.log - API Server, responsible for serving the API
|
* `/var/log/kube-apiserver.log` - API Server, responsible for serving the API
|
||||||
* /var/log/kube-scheduler.log - Scheduler, responsible for making scheduling decisions
|
* `/var/log/kube-scheduler.log` - Scheduler, responsible for making scheduling decisions
|
||||||
* /var/log/kube-controller-manager.log - Controller that manages replication controllers
|
* `/var/log/kube-controller-manager.log` - Controller that manages replication controllers
|
||||||
|
|
||||||
### Worker Nodes
|
### Worker Nodes
|
||||||
|
|
||||||
* /var/log/kubelet.log - Kubelet, responsible for running containers on the node
|
* `/var/log/kubelet.log` - Kubelet, responsible for running containers on the node
|
||||||
* /var/log/kube-proxy.log - Kube Proxy, responsible for service load balancing
|
* `/var/log/kube-proxy.log` - Kube Proxy, responsible for service load balancing
|
||||||
|
|
||||||
## A general overview of cluster failure modes
|
## A general overview of cluster failure modes
|
||||||
|
|
||||||
This is an incomplete list of things that could go wrong, and how to adjust your cluster setup to mitigate the problems.
|
This is an incomplete list of things that could go wrong, and how to adjust your cluster setup to mitigate the problems.
|
||||||
|
|
||||||
Root causes:
|
### Root causes:
|
||||||
|
|
||||||
- VM(s) shutdown
|
- VM(s) shutdown
|
||||||
- Network partition within cluster, or between cluster and users
|
- Network partition within cluster, or between cluster and users
|
||||||
@@ -57,7 +57,7 @@ Root causes:
|
|||||||
- Data loss or unavailability of persistent storage (e.g. GCE PD or AWS EBS volume)
|
- Data loss or unavailability of persistent storage (e.g. GCE PD or AWS EBS volume)
|
||||||
- Operator error, e.g. misconfigured Kubernetes software or application software
|
- Operator error, e.g. misconfigured Kubernetes software or application software
|
||||||
|
|
||||||
Specific scenarios:
|
### Specific scenarios:
|
||||||
|
|
||||||
- Apiserver VM shutdown or apiserver crashing
|
- Apiserver VM shutdown or apiserver crashing
|
||||||
- Results
|
- Results
|
||||||
@@ -91,7 +91,7 @@ Specific scenarios:
|
|||||||
- users unable to read API
|
- users unable to read API
|
||||||
- etc.
|
- etc.
|
||||||
|
|
||||||
Mitigations:
|
### Mitigations:
|
||||||
|
|
||||||
- Action: Use IaaS provider's automatic VM restarting feature for IaaS VMs
|
- Action: Use IaaS provider's automatic VM restarting feature for IaaS VMs
|
||||||
- Mitigates: Apiserver VM shutdown or apiserver crashing
|
- Mitigates: Apiserver VM shutdown or apiserver crashing
|
||||||
|
|||||||
Reference in New Issue
Block a user