ZH-trans: merge release1.16-temporary to master (#18217)

* zh-trans:/docs/docs/concepts/workloads/pods/ephemeral-containers.md (#16948)

* update zh-trans of define-environment-variable-container.md (#16999)

Signed-off-by: Yixiang2019 <wang.yixiang@zte.com.cn>

* update chinese docs (#16985)

* Fix ordered list (#16988)

Signed-off-by: PingWang <wang.ping5@zte.com.cn>

update

Signed-off-by: PingWang <wang.ping5@zte.com.cn>

* zh-trans:/docs/reference/setup-tools/kubeadm/kubeadm-upgrade-phase.md (#16951)

Signed-off-by: PingWang <wang.ping5@zte.com.cn>

update

Signed-off-by: PingWang <wang.ping5@zte.com.cn>

update

Signed-off-by: PingWang <wang.ping5@zte.com.cn>

* Remove redundant symbol and fix some ordered list (#17000)

Signed-off-by: PingWang <wang.ping5@zte.com.cn>

update

Signed-off-by: PingWang <wang.ping5@zte.com.cn>

* update-zh-translation/docs/reference/setup-tools/kubeadm/kubeadm-init.md (#16997)

* update zh translation kubeadm-reset.md kubeadm-upgrade.md (#16992)

* Create kubeadm_join_phase_control-plane-join_all.md (#16987)

* Update web-ui-dashboard.md (#16976)

* update format problem (#16956)

Signed-off-by: PingWang <wang.ping5@zte.com.cn>

update

Signed-off-by: PingWang <wang.ping5@zte.com.cn>

* zh-trans:/docs/docs/concepts/storage/volume-pvc-datasource.md (#17021)

* update zh translation /docs/reference/access-authn-authz/webhook.md (#16860)

* fix confict update zh translation (#16863)

* zh-trans:/docs/concepts/workloads/pods/disruptions.md (#16983)

* zh-trans:/docs/concepts/workloads/pods/disruptions.md

* Update content/zh/docs/concepts/workloads/pods/disruptions.md

Co-Authored-By: Qiming <tengqim@cn.ibm.com>

* update zh translation content/zh/docs/reference/command-line-tools-reference/kube-scheduler.md (#17006)

* Update RC's link (#16935)

Signed-off-by: PingWang <wang.ping5@zte.com.cn>

Update

Signed-off-by: PingWang <wang.ping5@zte.com.cn>

update the style

Signed-off-by: PingWang <wang.ping5@zte.com.cn>

* Update the links for /zh/docs/setup (#16938)

Signed-off-by: PingWang <wang.ping5@zte.com.cn>

* zh-trans:/docs/docs/concepts/services-networking/dual-stack.md (#17024)

* update zh translation /reference/setup-tools/kubeadm/generated/kubeadm.md (#17036)

* update zh tanslation /reference/setup-tools/kubeadm/generated/kubeadm_alpha_kubelet_config_download.md (#17037)

* update zh translation -/reference/setup-tools/kubeadm/generated/kubeadm_alpha.md (#17038)

* update zh translation /docs/contribute/participating.md (#17040)

* Fix cri-o's links to match English docs (#16936)

Signed-off-by: PingWang <wang.ping5@zte.com.cn>

* update zh translation content/zh/docs/reference/kubectl/jsonpath.md (#16862)

* update zh translation /docs/reference/setup-tools/kubeadm/generated/kubeadm_token_generate.md (#17049)

* update Unkown ->  Unknown (#17062)

* zh-translation:high-availability.md (#16960)

Signed-off-by: yuxiaobo <yuxiaobogo@163.com>

* update Runnning -> Running (#17061)

* zh-trans:/docs/docs/concepts/storage/volume-snapshots.md (#17054)

* update zh transation /docs/reference/setup-tools/kubeadm/generated/kubeadm_reset.md (#17060)

* update zh translation /docs/reference/setup-tools/kubeadm/generated/kubeadm_token_create.md (#17052)

* update zh translation /docs/reference/setup-tools/kubeadm/generated/kubeadm_token.md (#17055)

* update zh translation update-zh-translation-/docs/reference/setup-tools/kubeadm/generated/kubeadm_token_list.md (#17048)

* update zh-translation:ha-topology.md (#17099)

Signed-off-by: yuxiaobo <yuxiaobogo@163.com>

* zh-trans /reference/setup-tools/kubeadm/generated/kubeadm_config_images_list.md (#17093)

* update zh translation /reference/setup-tools/kubeadm/generated/kubeadm_init_phase_certs_all.md (#17083)

* Add Chinese translation for scheduler-perf-tuning (#17087)

Signed-off-by: GeorgeSen <wang.sen2@zte.com.cn>

Add Chinese translation for scheduler-perf-tuning

Signed-off-by: GeorgeSen <wang.sen2@zte.com.cn>

Add Chinese translation for scheduler-perf-tuning

Signed-off-by: GeorgeSen <wang.sen2@zte.com.cn>

Add Chinese translation for scheduler-perf-tuning

Signed-off-by: GeorgeSen <wang.sen2@zte.com.cn>

* zh trans content/zh/docs/setup/production-environment/tools/kubeadm/control-plane-flags.md (#17121)

* update zh trans content/zh/docs/tasks/access-application-cluster/service-access-application-cluster.md (#17122)

* zh-translation:content/zh/docs/tasks/administer-cluster/namespaces-walkthrough.md (#17105)

Signed-off-by: yuxiaobo <yuxiaobogo@163.com>

* update-zh-translation-/docs/reference/setup-tools/kubeadm/generated/kubeadm_config_migrate.md (#17091)

* update zh trans /docs/setup/learning-environment/minikube.md (#17143)

* update zh trans /zh/docs/reference/_index.md (#17146)

* update zh trans /docs/tasks/access-application-cluster/port-forward-access-application-cluster.md (#17134)

* update zh translation 20191020-update-zh-translation-/docs/contribute/localization.md (#17046)

* update zh trans /docs/reference/using-api/client-libraries.md (#17144)

* update zh trans /docs/reference/setup-tools/kubeadm/generated/kubeadm_version.md (#17145)

* update zh /docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade_node.md (#17131)

* update zh translation /reference/setup-tools/kubeadm/generated/kubeadm_init_phase_certs_front-proxy-client.md (#17081)

* update zh translation /docs/reference/setup-tools/kubeadm/generated/kubeadm_init_phase_kubeconfig.md (#17078)

* add zh translation /docs/reference/setup-tools/kubeadm/generated/kubeadm_join_phase_control-plane-join_update-status.md (#17076)

* update zh trans content/zh/docs/contribute/generate-ref-docs/kubectl.md (#17165)

* update zh translation /reference/command-line-tools-reference/kube-proxy.md (#17107)

* zh-trans:/docs/docs/concepts/workloads/pods/pod-topology-spread-const… (#16955)

* zh-trans:/docs/docs/concepts/workloads/pods/pod-topology-spread-constraints.md

* Update pod-topology-spread-constraints.md

* zh-trans:/docs/concepts/configuration/scheduling-framework.md (#17088)

* update zh translation /docs/reference/setup-tools/kubeadm/generated/kubeadm_init_phase_upload-config_kubelet.md (#17079)

* update zh translation /docs/reference/setup-tools/kubeadm/generated/kubeadm_init_phase_control-plane_apiserver.md (#17077)

* update zh translation /docs/reference/setup-tools/kubeadm/generated/kubeadm_join_phase_control-plane-join.md (#17075)

* update zh translation /docs/reference/setup-tools/kubeadm/generated/kubeadm_token_delete.md (#17050)

* update zh trans /docs/concepts/overview/working-with-objects/namespaces.md (#17205)

* update zh trans /docs/concepts/overview/what-is-kubernetes.md (#17203)

* pr_release-1.16_crictl (#17201)

* update zh docs/tasks/administer-cluster/dns-debugging-resolution.md (#17186)

* update zh trans content/zh/docs/concepts/overview/working-with-objects/field-selectors.md (#17191)

* translate configure_upgrade_etcd (#17160)

* translate kubeadm_upgrade_apply (#17159)

* update zh /docs/tasks/job/coarse-parallel-processing-work-queue.md (#17155)

* translate docs/setup/release/version-skew-policy.md to Chinese (#17142)

* update zh trans content/zh/docs/tasks/access-application-cluster/create-external-load-balancer.md (#17124)

* zh-trans replace the wrong translation (#17103)

zh-trans replace the wrong translation

* Merged 1.14~1.16 changes (#17117)

* zh-trans:/docs/concepts/configuration/assign-pod-node.md (#17129)

* update zh trans /docs/contribute/generate-ref-docs/kubernetes-api.md (#17161)

* pr_release-1.16_basic-ss (#17169)

* Add zh-trans of assign-cpu-resource.md (#17063)

Signed-off-by: heqg <he.qingguo@zte.com.cn>

Add zh-trans of assign-cpu-resource.md

Signed-off-by: heqg <he.qingguo@zte.com.cn>

Add zh-trans of assign-cpu-resource.md

Signed-off-by: heqg <he.qingguo@zte.com.cn>

Add zh-trans of assign-cpu-resource.md

Signed-off-by: heqg <he.qingguo@zte.com.cn>

Add zh-trans of assign-cpu-resource.md

Signed-off-by: heqg <he.qingguo@zte.com.cn>

* update zh trans content/zh/docs/concepts/overview/working-with-objects/common-labels.md (#17193)

* update zh translation /docs/contribute/intermediate.md (#17041)

* zh-trans:docs/setup/production-environment/turnkey/tencent.md (#17207)

* pr_release-1.16_mysql-wordpress-pv (#17202)

* pr_release-1.16_reconfig-kubelet (#17200)

* zh-trans:/docs/docs/concepts/workloads/controllers/jobs-run-completio… (#17020)

* zh-trans:/docs/docs/concepts/workloads/controllers/jobs-run-completion.md

* Update jobs-run-completion.md

* Update jobs-run-completion.md

* update zh trans content/zh/docs/concepts/extend-kubernetes/extend-cluster.md (#17212)

* update zh trans content/zh/docs/concepts/extend-kubernetes/api-extension/apiserver-aggregation.md (#17213)

* add zh trans /docs/reference/setup-tools/kubeadm/generated/kubeadm_join_phase_control-plane-prepare_kubeconfig.md (#17220)

kubeadm_join_phase_control-plane-prepare_download-certs.md

* add zh trans /reference/setup-tools/kubeadm/generated/kubeadm_init_phase_certs.md and /reference/setup-tools/kubeadm/generated/kubeadm_join_phase_control-plane-prepare.md (#17219)

* update zh trans content/zh/docs/concepts/containers/images.md (#17216)

* ZH-trans: add _index.md (#17214)

* ZH-trans: add _index.md

* add _index.md file

* add _index.md files

* pr_release-1.16_crd-versions (#17198)

* update zh translation /docs/reference/setup-tools/kubeadm/generated/kubeadm_config_images_pull.md (#17092)

* zh-trans /reference/setup-tools/kubeadm/generated/kubeadm_config_images.md (#17094)

* add zh trans content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_init_phase_addon_kube-proxy.md   and   content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_init_phase_certs_sa.md (#17222)

* add zh trans content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_join_phase_control-plane-prepare_all.md  content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_join_phase_control-plane-prepare_certs.md (#17221)

* add zh trans content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_init_phase_addon_all.md (#17223)

* 013 /docs/concepts/services networking/ingress.md (#17185)

* x

* update zh trans content/zh/docs/concepts/services-networking/ingress.md

* zh-trans:/reference/setup-tools/kubeadm/generated/kubeadm_completion.md and kubeadm_config.md (#17097)

* add zh trans reference/glossary/pod-lifecycle (#17226)

* update zh-translation document (#17096)

Signed-off-by: yuxiaobo <yuxiaobogo@163.com>

* update zh-translation:setup-ha-etcd-with-kubeadm.md (#17098)

Signed-off-by: yuxiaobo <yuxiaobogo@163.com>

* add zh trans /docs/reference/setup-tools/kubeadm/generated/kubeadm_alpha_selfhosting.md and /docs/reference/setup-tools/kubeadm/generated/kubeadm_alpha_kubelet_config_enable-dynamic.md (#17227)

* add zh trans /docs/reference/setup-tools/kubeadm/generated/kubeadm_alpha_kubeconfig_user.md and /docs/reference/setup-tools/kubeadm/generated/kubeadm_init_phase_upload-config_kubeadm.md (#17228)

* zh-translation:content/zh/docs/tasks/extend-kubectl/kubectl-plugins.md (#17089)

Signed-off-by: yuxiaobo <yuxiaobogo@163.com>

* update zh translation /docs/reference/setup-tools/kubeadm/generated/kubeadm_join.md (#17080)

* update zh translation /reference/setup-tools/kubeadm/generated/kubeadm_config_view.md (#17085)

* zh-translation:troubleshooting-kubeadm.md (#17069)

Signed-off-by: yuxiaobo <yuxiaobogo@163.com>

* zh-trans: docs/concepts/scheduling/kube-scheduler.md (#17067)

* Add Chinese translation for kube-scheduler

Signed-off-by: GeorgeSen <wang.sen2@zte.com.cn>

Add Chinese translation for kube-scheduler

Signed-off-by: GeorgeSen <wang.sen2@zte.com.cn>

Add Chinese translation for kube-scheduler

Signed-off-by: GeorgeSen <wang.sen2@zte.com.cn>

Add Chinese translation for kube-scheduler

Signed-off-by: GeorgeSen <wang.sen2@zte.com.cn>

Add Chinese translation for kube-scheduler

Signed-off-by: GeorgeSen <wang.sen2@zte.com.cn>

Add Chinese translation for kube-scheduler

Signed-off-by: GeorgeSen <wang.sen2@zte.com.cn>

Add Chinese translation for kube-scheduler

Signed-off-by: GeorgeSen <wang.sen2@zte.com.cn>

Add Chinese translation for kube-scheduler

Signed-off-by: GeorgeSen <wang.sen2@zte.com.cn>

* Update kube-scheduler.md

* Add Chinese translation for kube-scheduler

* Update kube-scheduler.md

* Update kube-scheduler.md

* Update kube-scheduler.md

* translate pods.md  and init-containers.md for branch release-1.16 (#17208)

* update zh translation /docs/contribute/start.md (#17039)

* zh-translation:2017-10-00-Five-Days-Of-Kubernetes-18.md (#17229)

Signed-off-by: yuxiaobo <yuxiaobogo@163.com>

* translate kubeadm-certs.md (#17090)

* update the Illegal comment such as : (<!--、<--) (#17266)

* add zh trans content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_init_phase.md and content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_init_phase_certs_etcd-ca.md (#17268)

* update zh /docs/concepts/architecture/cloud-controller.md (#17263)

* update zh /docs/concepts/cluster-administration/logging.md (#17247)

* modify the show of zh translation /concepts/overview/what-is-kubernetes.md /reference/setup-tools/kubeadm/generated/kubeadm_init.md  /reference/setup-tools/kubeadm/kubeadm-init.md (#17246)

* zh-translation:kubeadm_init_phase_control-plane_all.md (#17243)

Signed-off-by: yuxiaobo <yuxiaobogo@163.com>

* zh-translation:kubeadm_join_phase_control-plane-join_etcd.md (#17236)

Signed-off-by: yuxiaobo <yuxiaobogo@163.com>

* add zh trans content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_init_phase_kubelet-start.md and content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_init_phase_upload-certs.md (#17230)

* add content/zh/docs/reference/glossary/container-runtime.md file fix-up to pass the ci and trans content/zh/docs/reference/glossary/container-runtime.md、content/zh/docs/concepts/overview/components.md (#17211)

* zh-translation:mirror-pod.md (#17231)

Signed-off-by: yuxiaobo <yuxiaobogo@163.com>

* zh-translation:kubeadm_init_phase_upload-config.md (#17238)

Signed-off-by: yuxiaobo <yuxiaobogo@163.com>

* update zh /docs/concepts/workloads/pods/pod-overview.md (#17239)

* update the format of zh translation content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_reset.md content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade.md   content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade_apply.md  content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_upgrade_plan.md  content/zh/docs/reference/setup-tools/kubeadm/kubeadm-config.md  content/zh/docs/reference/setup-tools/kubeadm/kubeadm-reset.md  content/zh/docs/reference/setup-tools/kubeadm/kubeadm-token.md  content/zh/docs/reference/setup-tools/kubeadm/kubeadm-upgrade.md (#17248)

* Create advanced.md (#17256)

* Create advanced.md

* trans the advanced.md and fix the build bugs

* add zh trans content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_init_phase_control-plane.md and content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_init_phase_control-plane_scheduler.md (#17269)

* zh-translation:kubelet-integration.md (#17272)

Signed-off-by: yuxiaobo <yuxiaobogo@163.com>

* pr_release-1.16_out-of-resource (#17199)

* add zh trans content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_init_phase_certs_front-proxy-ca.md and content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_init_phase_control-plane_controller-manager.md (#17267)

* add zh trans content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_init_phase_certs_etcd-peer.md and content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_init_phase_kubeconfig_admin.md (#17271)

* pr_release-1.16_ext-admission-ctl (#17196)

* update zh translation /docs/contribute/advanced.md (#17042)

* add zh trans content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_init_phase_certs_apiserver-etcd-client.md and content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_init_phase_certs_ca.md (#17275)

* add zh trans content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_config_print.md update zh trans content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_config.md、content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_config_print_init-defaults.md (#17282)

* add zh trans  content/zh/docs/reference/setup-tools/kubeadm/generated… (#17287)

* add zh trans  content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_init_phase_etcd.md content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_init_phase_upload-config_all.md update zh trans content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_init_phase_control-plane.md

* Update kubeadm_init_phase_etcd.md

* add zh trans content/zh/docs/reference/setup-tools/kubeadm/generated/… (#17285)

* add zh trans content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_alpha_certs_renew_etcd-server.md content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_alpha_kubelet.md update zh trans content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_alpha_certs_renew_apiserver.md

* Update kubeadm_alpha_kubelet.md

* add zh trans content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_alpha_certs_renew_all.md and content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_init_phase_addon.md fix-up bad comment kubeadm_alpha_certs_renew.md、kubeadm_alpha_certs_renew_apiserver-etcd-client.md、kubeadm_init_phase_addon_all.md (#17276)

* add zh trans content/zh/docs/reference/setup-tools/kubeadm/generated/… (#17281)

* add zh trans content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_init_phase_etcd_local.md and content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_config_print_join-defaults.md

* Update kubeadm_init_phase_etcd_local.md

* zh trans update-daemon-set.md (#16872)

* zh trans update-daemon-set.md

* Update update-daemon-set.md

* managing-tls-in-a-cluster.md (#16874)

* update-api-object-kubectl-patch.md (#16875)

* improve the zh trans /kubeadm/generated/kubeadm_init_phase_.* 1 (#17295)

* improve the zh trans content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_config_.* (#17294)

* modify the zh translation content/zh/docs/reference/setup-tools/kubea… (#17293)

* modify the zh translation content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_alpha_.*

* Update kubeadm_alpha.md

* add zh trans content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_init_phase_kubeconfig_all.md and content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_init_phase_preflight.md (#17280)

* add zh /docs/reference/glossary/cgroup.md (#17291)

* add zh trans content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_alpha_kubelet_config.md、content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_init_phase_certs_apiserver-kubelet-client.md (#17288)

* improve zh trans of command in /kubeadm/generated/kubeadm_init_phase_.* files (#17297)

* improve zh command translation /kubeadm/generated/kubeadm_init_.* files (#17298)

* update zh trans in /kubeadm/generated/kubeadm_.* files (#17306)

* add zh trans /reference/setup-tools/kubeadm/generated/kubeadm_alpha_certs_renew_etcd-healthcheck-client.md、/reference/setup-tools/kubeadm/generated/kubeadm_alpha_certs_renew_etcd-peer.md、/reference/setup-tools/kubeadm/generated/kubeadm_alpha_kubeconfig.md (#17308)

* Improve previously translated documents (#17327)

Signed-off-by: yuxiaobo <yuxiaobogo@163.com>

* zh-trans: docs/setup/production-environment/turnkey/aws.md (#17320)

* Update zh.toml

* update zh trans /generated/kubeadm_join_phase_.* files (#17301)

* Update zh.toml

* add zh /docs/reference/glossary/pod-disruption-budget.md (#17344)

* pr_release-1.16_config-aggregation-layer (#17197)

* add zh trans content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_alpha_certs_renew_controller-manager.conf.md、content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_alpha_certs_renew_scheduler.conf.md (#17390)

* add zh trans content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_init_phase_kubeconfig_scheduler.md、content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_join_phase.md、content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_join_phase_control-plane-prepare_control-plane.md (#17381)

* add zh trans content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_alpha_certs_renew_apiserver-kubelet-client.md、content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_join_phase_control-plane-join_mark-control-plane.md、content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_join_phase_kubelet-start.md (#17380)

* add zh trans content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_init_phase_addon_coredns.md、content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_init_phase_bootstrap-token.md、content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_init_phase_certs_apiserver.md (#17383)

* add zh trans content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_alpha_certs_certificate-key.md、content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_reset_phase_preflight.md、content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_reset_phase_update-cluster-status.md (#17385)

* add zh trans content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_alpha_certs_renew_admin.conf.md、content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_reset_phase.md、content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_reset_phase_cleanup-node.md (#17387)

* add zh trans content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_alpha_certs_renew_apiserver-kubelet-client.md、content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_alpha_certs_renew_front-proxy-client.md、content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_alpha_selfhosting_pivot.md (#17384)

* add zh /docs/reference/glossary/limitrange.md (#17324)

* add zh trans content/zh/docs/setup/best-practices/cluster-large.md (#17321)

* add zh trans content/zh/docs/setup/best-practices/cluster-large.md

* Update cluster-large.md

* add zh trans /docs/reference/setup-tools/kubeadm/kubeadm-alpha.md、/do… (#17403)

* add zh trans /docs/reference/setup-tools/kubeadm/kubeadm-alpha.md、/docs/reference/setup-tools/kubeadm/generated/kubeadm_join_phase_preflight.md、/docs/reference/setup-tools/kubeadm/generated/kubeadm_init_phase_certs_etcd-server.md

* Update kubeadm_init_phase_certs_etcd-server.md

* add zh /docs/reference/glossary/cluster-operations.md (#17300)

* add zh /docs/reference/glossary/applications.md (#17419)

* update zh trans kubelet (#17379)

* update zh trans kubelet

* update the file according to feedback from reviewer tengqm

* update 1000-1757 lines

* update the advice zh trans

* add zh /docs/reference/glossary/static-pod.md (#17418)

* add zh /docs/reference/glossary/preemption.md (#17423)

* add zh /docs/reference/glossary/pod-priority.md (#17421)

* add zh /docs/reference/glossary/control-plane.md (#17425)

* add zh /docs/reference/glossary/cluster-infrastructure.md (#17424)

* pr_release-1.16_api-overview (#17444)

* pr_release-1.16_daemonset (#17435)

* pr_release-1.16_gc (#17445)

* pr_release-1.16_qos-class (#17442)

* fix QoS Class to QoS 类 (#17464)

* pr_release-1.16-abac (#17427)

* zh-trans:docs/setup/production-environment/turnkey/alibaba-cloud.md (#17345)

* pr_release-1.16_endpoint-slice (#17468)

* pr_release-1.16_taint (#17469)

* pr_release-1.16_operator-pattern (#17467)

* pr_release-1.16_ss (#17433)

* pr_release-1.16_admission-controller (#17440)

* pr_release-1.16_containerd (#17441)

* pr_release-1.16_app-container (#17466)

* add zh-trans content/zh/docs/setup/_index.md、content/zh/docs/setup/release/_index.md (#17503)

* pr-release-1.16_enabling-endpoint-slices (#17504)

* update zh trans content/zh/docs/reference/setup-tools/kubeadm/kubeadm… (#17495)

* update zh trans content/zh/docs/reference/setup-tools/kubeadm/kubeadm-upgrade-phase.md、content/zh/docs/reference/setup-tools/kubeadm/kubeadm-init-phase.md

* add content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_init_phase_kubeconfig_kubelet.md

* pr-release-1.16_logging (#17491)

* pr-release-1.16_cri (#17486)

* add zh-trans:docs/setup/production-environment/turnkey/azure.md (#17482)

* pattern translate into 模式 (#17485)

* fix zk affinity description in zh trans (#17393)

* pr-release-1.16_ephemeral-container (#17487)

* pr-release-1.16_data-plane (#17489)

* pr-release-1.16_cncf (#17490)

* zh-trans add content\zh\docs\tools\install-minikube.md (#16920)

* zh-trans add content\zh\docs\tools\install-minikube.md

* zh-trans update content\zh\docs\tools\install-minikube.md

* zh-trans update content\zh\docs\tools\install-minikube.md

* update \docs\tasks\tools\install-minikube.md

* update docs\concepts\workloads\controllers\deployment.md

* update deployment.md

* pr-release-1.16_extensions (#17492)

* pr-release-1.16_toleration (#17493)

* Revert "update zh trans content/zh/docs/reference/setup-tools/kubeadm/kubeadm… (#17495)" (#17521)

This reverts commit 1134c14e0a39bdc3d1a920ac9797139a6dcccf4b.

* motidy extensions in content/zh/docs/reference/glossary/extensions (#17524)

* motidy toleration in content/zh/docs/reference/glossary/toleration.md (#17523)

* motidy toleration in content/zh/docs/reference/glossary/toleration.md

* Update toleration.md

* improve zh-trans in content/zh/docs/setup/_index.md (#17526)

* add zh-trans /zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_init_phase_kubeconfig_kubelet.md (#17527)

* Broken Link (#17546)

Issue available at https://kubernetes.io/zh/docs/concepts/containers/runtime-class/
and introduced by original English documentation (see #17543)

* Update trans kubeadm_upgrade_plan.md (#17395)

* Update kubeadm_upgrade_plan.md

* Update kubeadm_upgrade_plan.md

* Update kubeadm_upgrade_plan.md

* Update kubeadm_upgrade_plan.md

* Update kubeadm_upgrade_plan.md

* Update kubeadm_upgrade_plan.md

* update zh-trans of define-command-argument-container.md (#17022)

Signed-off-by: Yixiang2019 <wang.yixiang@zte.com.cn>

update zh-trans of define-command-argument-container.md

Signed-off-by: Yixiang2019 <wang.yixiang@zte.com.cn>

Add back the Original English

Signed-off-by: Yixiang2019 <wang.yixiang@zte.com.cn>

update title of define-command-argument-container.md

Signed-off-by: Yixiang2019 <wang.yixiang@zte.com.cn>

update table title and reference

Signed-off-by: Yixiang2019 <wang.yixiang@zte.com.cn>

update reference of define-command-argument-container.md

Signed-off-by: Yixiang2019 <wang.yixiang@zte.com.cn>

* ZH-trans: fix multiple jump links and update files (#17603)

* ZH-trans: fix multiple jump links and update files

* Update _index.html

* update zh trans content/zh/docs/reference/setup-tools/kubeadm/kubeadm-init-phase.md (#17528)

* update zh trans /doc/concepts/architecture/nodes.md (#17617)

* update zh trans content/zh/docs/concepts/architecture/nodes.md

* fix-up content/zh/docs/concepts/architecture/nodes.md

* add zh-trans /docs/setup/release/notes.md (#17519)

* add zh-trans /docs/setup/release/notes.md

update-750

*  fix-up 1575 line and udpate 2483 line

* update to line 2980

* update to the last line 3160

* add zh-trans:docs/setup/production-environment/turnkey/icp.md (#17568)

* zh-trans: /docs/setup/production-environment/container-runtimes.md (#17646)

* zh-trs:container-runtimes.md

Signed-off-by: yuxiaobo <yuxiaobogo@163.com>

* Update container-runtimes.md

* Translate /docs/concepts/workloads/controllers/ttlafterfinished.md into Chinese (#17791)

* Translate /docs/concepts/cluster-administration/cloud-providers.md into Chinese

* Translate /docs/concepts/workloads/controllers/ttlafterfinished.md into Chinese

* Sorry, wrong commit, roll back...

* Translate /docs/concepts/workloads/controllers/ttlafterfinished.md

* Translate /docs/concepts/workloads/controllers/ttlafterfinished.md into Chinese

* Translate /docs/concepts/workloads/controllers/ttlafterfinished.md into Chinese

* Create trans kubeadm_upgrade_diff.md (#17392)

* Update kubeadm_upgrade_diff.md

* Update kubeadm_upgrade_diff.md

* Update kubeadm_upgrade_diff.md

* Create _index.md (#17831)

* zh-trans zh-trans-/docs/reference/command-line-tools-reference/feature-gates.md (#17658)

* Update volume-snapshots.md (#17829)

* Update volume-snapshots.md

* Update volume-snapshots.md

* Update volume-snapshots.md

* Create ovirt.md (#17849)

* Create ovirt.md

* Update ovirt.md

* Translate /docs/concepts/cluster-administration/cloud-providers.md into Chinese (#17761)

* Translate /docs/concepts/cluster-administration/cloud-providers.md into Chinese

* Translate /docs/concepts/workloads/controllers/ttlafterfinished.md into Chinese

* Sorry, wrong commit, roll back...

* Update translation after tengqm's review.

* Update cloud-providers.md

* Update dual-stack.md (#17836)

* Update dual-stack.md

* Update dual-stack.md

* Update dual-stack.md

* Update dual-stack.md

* Create validate-dual-stack.md (#17833)

* Create validate-dual-stack.md

* Update validate-dual-stack.md

* translation content/zh/docs/reference/setup-tools/kubeadm/ kubeadm-join-phase、kubeadm-reset-phase (#17881)

* zh-trans content/zh/docs/contribute/generate-ref-docs/contribute-upstream.md (#17882)

* Chinese translation /docs/tasks/administer-cluster/highly-available-master.md (#17884)

* Chinese translation /docs/tasks/administer-cluster/highly-available-master.md

* Apply suggestions from code review

Co-Authored-By: Qiming <tengqim@cn.ibm.com>

* Fix format issue (#17921)

* Create topology-manager.md (#17901)

* Create topology-manager.md

* Update topology-manager.md

* add zh-trans:docs/setup/production-environment/windows/user-guide-windows-containers.md (#17876)

* Update pod-overhead.md (#17931)

* Update scheduler-perf-tuning.md (#17934)

* Create dcos.md (#17932)

* Create dcos.md

* Update dcos.md

* Update object-management.md (#17937)

* zh-translation content/zh/docs/setup/production-environment/tools/kops.md (#17991)

* Create imperative-config.md (#17956)

* Create imperative-config.md

* Update imperative-config.md

* Create self-hosting.md (#17950)

* Create resource-bin-packing.md (#17935)

* Create nodelocaldns.md (#17938)

* Update config.toml(release-1.16) for 1.17 (#18025)

* Update config.toml(release-1.16) for 1.17

* Update config.toml

* Remove ru language

* Update the conflict and merge the two commits

* add nginx-deployment.yaml file

Co-authored-by: ZhongliangXiong <xiong.zhongliang@zte.com.cn>
Co-authored-by: Yixiang Wang <wang.yixiang@zte.com.cn>
Co-authored-by: li mengyang <hwdef97@gmail.com>
Co-authored-by: PingWang <wang.ping5@zte.com.cn>
Co-authored-by: chentanjun <tanjunchen20@gmail.com>
Co-authored-by: Sophy417 <53026875+Sophy417@users.noreply.github.com>
Co-authored-by: zhangx501 <zhang0000xun@gmail.com>
Co-authored-by: Qiming <tengqim@cn.ibm.com>
Co-authored-by: yuxiaobo96 <41496192+yuxiaobo96@users.noreply.github.com>
Co-authored-by: senwang <wang.sen2@zte.com.cn>
Co-authored-by: lichuqiang <lichuqiang@huawei.com>
Co-authored-by: lpf7551321 <liupengfei20@huawei.com>
Co-authored-by: Hongcai Ren <renhongcai@huawei.com>
Co-authored-by: jiajie <jiaj12@chinaunicom.cn>
Co-authored-by: heqg <56527988+heqg@users.noreply.github.com>
Co-authored-by: IreneByron <zhangbingqing7@huawei.com>
Co-authored-by: Wang Bing <wangbing.adam@gmail.com>
Co-authored-by: Damini Satya <daminisatya@gmail.com>
Co-authored-by: liufangwai <liufangwai@huawei.com>
Co-authored-by: wangcong <congfairy2536@gmail.com>
Co-authored-by: XuefeiWang2 <wangxuefei2@huawei.com>
Co-authored-by: Kubernetes Prow Robot <k8s-ci-robot@users.noreply.github.com>
Co-authored-by: Ziqiu Zhu <zzqshu@126.com>
Co-authored-by: ten2ton <50288981+ten2ton@users.noreply.github.com>
Co-authored-by: Oleg Butuzov <butuzov@users.noreply.github.com>
Co-authored-by: jiazxjason <52809535+jiazxjason@users.noreply.github.com>
Co-authored-by: Coffey Gao <coffiney@qq.com>
Co-authored-by: Bingshen Wang <bingshen.wbs@alibaba-inc.com>
This commit is contained in:
LiuDui
2019-12-25 08:51:29 +08:00
committed by Kubernetes Prow Robot
parent 9135bdbe6d
commit b8b2cc7c74
933 changed files with 113346 additions and 8088 deletions
@@ -0,0 +1,9 @@
---
title: "使用 kubeadm 引导集群"
weight: 10
---
<!--
title: "Bootstrapping clusters with kubeadm"
weight: 10
-->
@@ -0,0 +1,138 @@
---
title: 使用 kubeadm 定制控制平面配置
content_template: templates/concept
weight: 40
---
<!--
---
reviewers:
- sig-cluster-lifecycle
title: Customizing control plane configuration with kubeadm
content_template: templates/concept
weight: 40
---
-->
{{% capture overview %}}
{{< feature-state for_k8s_version="1.12" state="stable" >}}
<!--
The kubeadm `ClusterConfiguration` object exposes the field `extraArgs` that can override the default flags passed to control plane
components such as the APIServer, ControllerManager and Scheduler. The components are defined using the following fields:
-->
kubeadm `ClusterConfiguration` 对象公开了 `extraArgs` 字段,它可以覆盖传递给控制平面组件(如 APIServer、ControllerManager 和 Scheduler)的默认参数。各组件配置使用如下字段定义:
- `apiServer`
- `controllerManager`
- `scheduler`
<!--
The `extraArgs` field consist of `key: value` pairs. To override a flag for a control plane component:
-->
`extraArgs` 字段由 `key: value` 对组成。
要覆盖控制平面组件的参数:
<!--
1. Add the appropriate field to your configuration.
2. Add the flags to override to the field.
3. Run `kubeadm init` with `--config <YOUR CONFIG YAML>`.
-->
1. 将适当的字段添加到配置中。
2. 向字段添加要覆盖的参数值。
3.`--config <YOUR CONFIG YAML>` 运行 `kubeadm init`
<!--
For more details on each field in the configuration you can navigate to our
[API reference pages](https://godoc.org/k8s.io/kubernetes/cmd/kubeadm/app/apis/kubeadm/v1beta2#ClusterConfiguration).
-->
有关配置中的每个字段的详细信息,您可以导航到我们的 [API 参考页面](https://godoc.org/k8s.io/kubernetes/cmd/kubeadm/app/apis/kubeadm/v1beta2#ClusterConfiguration)。
{{< note >}}
<!--
You can generate a `ClusterConfiguration` object with default values by running `kubeadm config print init-defaults` and saving the output to a file of your choice.
-->
您可以通过运行 `kubeadm config print init-defaults` 并将输出保存到您选择的文件中,以默认值形式生成 `ClusterConfiguration` 对象。
{{< /note >}}
{{% /capture %}}
{{% capture body %}}
<!--
## APIServer flags
-->
## APIServer 参数
<!--
For details, see the [reference documentation for kube-apiserver](/docs/reference/command-line-tools-reference/kube-apiserver/).
-->
有关详细信息,请参阅 [kube-apiserver 参考文档](/docs/reference/command-line-tools-reference/kube-apiserver/)。
<!--
Example usage:
-->
使用示例:
```yaml
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
kubernetesVersion: v1.16.0
apiServer:
extraArgs:
advertise-address: 192.168.0.103
anonymous-auth: "false"
enable-admission-plugins: AlwaysPullImages,DefaultStorageClass
audit-log-path: /home/johndoe/audit.log
```
<!--
## ControllerManager flags
-->
## ControllerManager 参数
<!--
For details, see the [reference documentation for kube-controller-manager](/docs/reference/command-line-tools-reference/kube-controller-manager/).
-->
有关详细信息,请参阅 [kube-controller-manager 参考文档](/docs/reference/command-line-tools-reference/kube-controller-manager/)。
<!--
Example usage:
-->
使用示例:
```yaml
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
kubernetesVersion: v1.16.0
controllerManager:
extraArgs:
cluster-signing-key-file: /home/johndoe/keys/ca.key
bind-address: 0.0.0.0
deployment-controller-sync-period: "50"
```
<!--
## Scheduler flags
-->
## Scheduler 参数
<!--
For details, see the [reference documentation for kube-scheduler](/docs/reference/command-line-tools-reference/kube-scheduler/).
-->
有关详细信息,请参阅 [kube-scheduler 参考文档](/docs/reference/command-line-tools-reference/kube-scheduler/)。
<!--
Example usage:
-->
使用示例:
```yaml
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
kubernetesVersion: v1.16.0
scheduler:
extraArgs:
address: 0.0.0.0
config: /home/johndoe/schedconfig.yaml
kubeconfig: /home/johndoe/kubeconfig.yaml
```
{{% /capture %}}
@@ -0,0 +1,140 @@
---
reviewers:
- sig-cluster-lifecycle
title: 高可用拓扑选项
content_template: templates/concept
weight: 50
---
<!--
---
reviewers:
- sig-cluster-lifecycle
title: Options for Highly Available topology
content_template: templates/concept
weight: 50
---
-->
{{% capture overview %}}
<!--
This page explains the two options for configuring the topology of your highly available (HA) Kubernetes clusters.
-->
本页面介绍了配置高可用(HA) Kubernetes 集群拓扑的两个选项。
<!--
You can set up an HA cluster:
-->
您可以设置 HA 集群:
<!--
- With stacked control plane nodes, where etcd nodes are colocated with control plane nodes- With external etcd nodes, where etcd runs on separate nodes from the control plane
-->
- 使用堆叠(stacked)控制平面节点,其中 etcd 节点与控制平面节点共存
- 使用外部 etcd 节点,其中 etcd 在与控制平面不同的节点上运行
<!--
You should carefully consider the advantages and disadvantages of each topology before setting up an HA cluster.
-->
在设置 HA 集群之前,您应该仔细考虑每种拓扑的优缺点。
{{% /capture %}}
{{% capture body %}}
<!--
## Stacked etcd topology
-->
## 堆叠(Stacked etcd 拓扑
<!--
A stacked HA cluster is a [topology](https://en.wikipedia.org/wiki/Network_topology) where the distributeddata storage cluster provided by etcd is stacked on top of the cluster formed by the nodes managed by kubeadm that run control plane components.
-->
堆叠(Stacked) HA 集群是一种这样的[拓扑](https://en.wikipedia.org/wiki/Network_topology),其中 etcd 分布式数据存储集群堆叠在 kubeadm 管理的控制平面节点上,作为控制平面的一个组件运行。
<!--
Each control plane node runs an instance of the `kube-apiserver`, `kube-scheduler`, and `kube-controller-manager`.
-->
每个控制平面节点运行 `kube-apiserver``kube-scheduler``kube-controller-manager` 实例。
<!--
The `kube-apiserver` is exposed to worker nodes using a load balancer.
-->
`kube-apiserver` 使用负载均衡器暴露给工作节点。
<!--
Each control plane node creates a local etcd member and this etcd member communicates only with
the `kube-apiserver` of this node. The same applies to the local `kube-controller-manager`
and `kube-scheduler` instances.
-->
每个控制平面节点创建一个本地 etcd 成员(member),这个 etcd 成员只与该节点的 `kube-apiserver` 通信。这同样适用于本地 `kube-controller-manager``kube-scheduler` 实例。
<!--
This topology couples the control planes and etcd members on the same nodes. It is simpler to set up than a cluster with external etcd nodes, and simpler to manage for replication.
-->
这种拓扑将控制平面和 etcd 成员耦合在同一节点上。相对使用外部 etcd 集群,设置起来更简单,而且更易于副本管理。
<!--
However, a stacked cluster runs the risk of failed coupling. If one node goes down, both an etcd member and a controlplane instance are lost, and redundancy is compromised. You can mitigate this risk by adding more control plane nodes.
-->
然而,堆叠集群存在耦合失败的风险。如果一个节点发生故障,则 etcd 成员和控制平面实例都将丢失,并且冗余会受到影响。您可以通过添加更多控制平面节点来降低此风险。
<!--
You should therefore run a minimum of three stacked control plane nodes for an HA cluster.
-->
因此,您应该为 HA 集群运行至少三个堆叠的控制平面节点。
<!--
This is the default topology in kubeadm. A local etcd member is created automatically
on control plane nodes when using `kubeadm init` and `kubeadm join --control-plane`.
-->
这是 kubeadm 中的默认拓扑。当使用 `kubeadm init``kubeadm join --control-plane` 时,在控制平面节点上会自动创建本地 etcd 成员。
<!--
![Stacked etcd topology](/images/kubeadm/kubeadm-ha-topology-stacked-etcd.svg)
-->
[堆叠的 etcd 拓扑](/images/kubeadm/kubeadm-ha-topology-stacked-etcd.svg)
<!--
## External etcd topology
-->
## 外部 etcd 拓扑
<!--
An HA cluster with external etcd is a [topology](https://en.wikipedia.org/wiki/Network_topology) where the distributed data storage cluster provided by etcd is external to the cluster formed by the nodes that run control plane components.
-->
具有外部 etcd 的 HA 集群是一种这样的[拓扑](https://en.wikipedia.org/wiki/Network_topology),其中 etcd 分布式数据存储集群在独立于控制平面节点的其他节点上运行。
<!--
Like the stacked etcd topology, each control plane node in an external etcd topology runs an instance of the `kube-apiserver`, `kube-scheduler`, and `kube-controller-manager`. And the `kube-apiserver` is exposed to worker nodes using a load balancer. However, etcd members run on separate hosts, and each etcd host communicates with the `kube-apiserver` of each control plane node.
-->
就像堆叠的 etcd 拓扑一样,外部 etcd 拓扑中的每个控制平面节点都运行 `kube-apiserver``kube-scheduler``kube-controller-manager` 实例。同样, `kube-apiserver` 使用负载均衡器暴露给工作节点。但是,etcd 成员在不同的主机上运行,​​每个 etcd 主机与每个控制平面节点的 `kube-apiserver` 通信。
<!--
This topology decouples the control plane and etcd member. It therefore provides an HA setup wherelosing a control plane instance or an etcd member has less impact and does not affectthe cluster redundancy as much as the stacked HA topology.
-->
这种拓扑结构解耦了控制平面和 etcd 成员。因此,它提供了一种 HA 设置,其中失去控制平面实例或者 etcd 成员的影响较小,并且不会像堆叠的 HA 拓扑那样影响集群冗余。
<!--
However, this topology requires twice the number of hosts as the stacked HA topology.
-->
但是,此拓扑需要两倍于堆叠 HA 拓扑的主机数量。
<!--
A minimum of three hosts for control plane nodes and three hosts for etcd nodes are required for an HA cluster with this topology.
-->
具有此拓扑的 HA 集群至少需要三个用于控制平面节点的主机和三个用于 etcd 节点的主机。
<!--
![External etcd topology](/images/kubeadm/kubeadm-ha-topology-external-etcd.svg)
-->
[外部 etcd 拓扑](/images/kubeadm/kubeadm-ha-topology-external-etcd.svg)
{{% /capture %}}
{{% capture whatsnext %}}
<!--
- [Set up a highly available cluster with kubeadm](/docs/setup/production-environment/tools/kubeadm/high-availability/)
-->
- [使用 kubeadm 设置高可用集群](/docs/setup/production-environment/tools/kubeadm/high-availability/)
{{% /capture %}}
@@ -0,0 +1,727 @@
---
reviewers:
- sig-cluster-lifecycle
title: 利用 kubeadm 创建高可用集群
content_template: templates/task
weight: 60
---
<!--
---
reviewers:
- sig-cluster-lifecycle
title: Creating Highly Available clusters with kubeadm
content_template: templates/task
weight: 60
---
-->
{{% capture overview %}}
<!--
This page explains two different approaches to setting up a highly available Kubernetes
cluster using kubeadm:
- With stacked control plane nodes. This approach requires less infrastructure. The etcd members
and control plane nodes are co-located.
- With an external etcd cluster. This approach requires more infrastructure. The
control plane nodes and etcd members are separated.
-->
本文讲述了使用 kubeadm 设置一个高可用的 Kubernetes 集群的两种不同方式:
- 使用堆控制平面节点。这种方法所需基础设施较少。etcd 成员和控制平面节点位于同一位置。
- 使用外部集群。这种方法所需基础设施较多。控制平面的节点和 etcd 成员是分开的。
<!--
Before proceeding, you should carefully consider which approach best meets the needs of your applications
and environment. [This comparison topic](/docs/setup/production-environment/tools/kubeadm/ha-topology/) outlines the advantages and disadvantages of each.
If you encounter issues with setting up the HA cluster, please provide us with feedback
in the kubeadm [issue tracker](https://github.com/kubernetes/kubeadm/issues/new).
See also [The upgrade documentation](/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade-1-15).
-->
在下一步之前,您应该仔细考虑哪种方法更好的满足您的应用程序和环境的需求。 [这是对比文档](/docs/setup/production-environment/tools/kubeadm/ha-topology/) 讲述了每种方法的优缺点。
如果您在安装 HA 集群时遇到问题,请在 kubeadm [问题跟踪](https://github.com/kubernetes/kubeadm/issues/new)里向我们提供反馈。
您也可以阅读 [升级文件](/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade-1-15)。
{{< caution >}}
<!--
This page does not address running your cluster on a cloud provider. In a cloud
environment, neither approach documented here works with Service objects of type
LoadBalancer, or with dynamic PersistentVolumes.
-->
这篇文档没有讲述在云提供商上运行集群的问题。在云环境中,此处记录的方法不适用于类型为 LoadBalancer 的服务对象,或者具有动态的 PersistentVolumes。
{{< /caution >}}
{{% /capture %}}
{{% capture prerequisites %}}
<!--
For both methods you need this infrastructure:
- Three machines that meet [kubeadm's minimum requirements](/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#before-you-begin) for
the masters
- Three machines that meet [kubeadm's minimum
requirements](/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#before-you-begin) for the workers
- Full network connectivity between all machines in the cluster (public or
private network)
- sudo privileges on all machines
- SSH access from one device to all nodes in the system
- `kubeadm` and `kubelet` installed on all machines. `kubectl` is optional.
-->
对于这两种方法,您都需要以下基础设施:
- 配置三台机器[kubeadm 的最低要求](/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#before-you-begin)给主节点
- 配置三台机器 [kubeadm 的最低要求](/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#before-you-begin) 给工作节点
- 在集群中,所有计算机之间的完全网络连接(公网或私网)
- 所有机器上的 sudo 权限
- 每台设备对系统中所有节点的 SSH 访问
- 在所有机器上安装 `kubeadm``kubelet``kubectl` 是可选的。
<!--
For the external etcd cluster only, you also need:
- Three additional machines for etcd members
-->
仅对于外部 etcd 集群来说,您还需要:
- 给 etcd 成员使用的另外三台机器
{{% /capture %}}
{{% capture steps %}}
<!-- ## First steps for both methods -->
## 这两种方法的第一步
<!-- ### Create load balancer for kube-apiserver -->
### 为 kube-apiserver 创建负载均衡器
<!--
There are many configurations for load balancers. The following example is only one
option. Your cluster requirements may need a different configuration.
-->
{{< note >}}
**注意**:使用负载均衡器需要许多配置。您的集群搭建可能需要不同的配置。下面的例子只是其中的一方面配置。
{{< /note >}}
<!--
1. Create a kube-apiserver load balancer with a name that resolves to DNS.
- In a cloud environment you should place your control plane nodes behind a TCP
forwarding load balancer. This load balancer distributes traffic to all
healthy control plane nodes in its target list. The health check for
an apiserver is a TCP check on the port the kube-apiserver listens on
(default value `:6443`).
- It is not recommended to use an IP address directly in a cloud environment.
- The load balancer must be able to communicate with all control plane nodes
on the apiserver port. It must also allow incoming traffic on its
listening port.
- [HAProxy](http://www.haproxy.org/) can be used as a load balancer.
- Make sure the address of the load balancer always matches
the address of kubeadm's `ControlPlaneEndpoint`.
-->
1. 创建一个名为 kube-apiserver 的负载均衡器解析 DNS。
- 在云环境中,应该将控制平面节点放置在 TCP 后面转发负载平衡。 该负载均衡器将流量分配给目标列表中所有运行状况良好的控制平面节点。健康检查 apiserver 是在 kube-apiserver 监听端口(默认值 `:6443`)上的一个 TCP 检查。
- 不建议在云环境中直接使用 IP 地址。
- 负载均衡器必须能够在 apiserver 端口上与所有控制平面节点通信。它还必须允许其监听端口的传入流量。
- [HA代理](http://www.haproxy.org/)可以被用来做一个负载均衡器。
- 确保负载均衡器的地址始终匹配 kubeadm 的 `ControlPlaneEndpoint` 地址。
<!--
1. Add the first control plane nodes to the load balancer and test the
connection:
```sh
nc -v LOAD_BALANCER_IP PORT
```
- A connection refused error is expected because the apiserver is not yet
running. A timeout, however, means the load balancer cannot communicate
with the control plane node. If a timeout occurs, reconfigure the load
balancer to communicate with the control plane node.
1. Add the remaining control plane nodes to the load balancer target group.
-->
1. 添加第一个控制平面节点到负载均衡器并测试连接:
```sh
nc -v LOAD_BALANCER_IP PORT
```
- 由于 apiserver 尚未运行,预期会出现一个连接拒绝错误。然而超时意味着负载均衡器不能和控制平面节点通信。
如果发生超时,请重新配置负载均衡器与控制平面节点进行通信。
1. 将其余控制平面节点添加到负载均衡器目标组。
<!-- ## Stacked control plane and etcd nodes -->
## 使用堆控制平面和 etcd 节点
<!-- ### Steps for the first control plane node -->
### 控制平面节点的第一步
<!--
1. Initialize the control plane:
```sh
sudo kubeadm init --control-plane-endpoint "LOAD_BALANCER_DNS:LOAD_BALANCER_PORT" --upload-certs
```
- You can use the `--kubernetes-version` flag to set the Kubernetes version to use.
It is recommended that the versions of kubeadm, kubelet, kubectl and Kubernetes match.
- The `--control-plane-endpoint` flag should be set to the address or DNS and port of the load balancer.
- The `--upload-certs` flag is used to upload the certificates that should be shared
across all the control-plane instances to the cluster. If instead, you prefer to copy certs across
control-plane nodes manually or using automation tools, please remove this flag and refer to [Manual
certificate distribution](#manual-certs) section bellow.
-->
1. 初始化控制平面:
```sh
sudo kubeadm init --control-plane-endpoint "LOAD_BALANCER_DNS:LOAD_BALANCER_PORT" --upload-certs
```
- 您可以使用 `--kubernetes-version` 标志来设置要使用的 Kubernetes 版本。建议将 kubeadm、kebelet、kubectl 和 Kubernetes 的版本匹配。
- 这个 `--control-plane-endpoint` 标志应该被设置成负载均衡器的地址或 DNS 和端口。
- 这个 `--upload-certs` 标志用来将在所有控制平面实例之间的共享证书上传到集群。如果正好相反,你更喜欢手动地通过控制平面节点或者使用自动化
工具复制证书,请删除此标志并参考如下部分[证书分配手册](#manual-certs)。
{{< note >}}
<!--
The `kubeadm init` flags `--config` and `--certificate-key` cannot be mixed, therefore if you want
to use the [kubeadm configuration](https://godoc.org/k8s.io/kubernetes/cmd/kubeadm/app/apis/kubeadm/v1beta2) you must add the `certificateKey` field in the appropriate config locations (under `InitConfiguration` and `JoinConfiguration: controlPlane`).
-->
**注意**:标志 `kubeadm init`、`--config` 和 `--certificate-key` 不能混合使用,因此如果您要使用[kubeadm 配置](https://godoc.org/k8s.io/kubernetes/cmd/kubeadm/app/apis/kubeadm/v1beta2),您必须在相应的配置文件(位于 `InitConfiguration` 和 `JoinConfiguration: controlPlane`)添加 `certificateKey` 字段。
{{< /note >}}
{{< note >}}
<!--
Some CNI network plugins like Calico require a CIDR such as `192.168.0.0/16` and
some like Weave do not. See the [CNI network documentation](/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/#pod-network).
To add a pod CIDR pass the flag `--pod-network-cidr`, or if you are using a kubeadm configuration file
set the `podSubnet` field under the `networking` object of `ClusterConfiguration`.
-->
**注意**:一些 CNI 网络插件如 Calico 需要 CIDR 例如 `192.168.0.0/16` 和一些像 Weave 没有。参考 [CNI 网络文档](/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/#pod-network)。
通过传递 `--pod-network-cidr` 标志添加 pod CIDR,或者您可以使用 kubeadm 配置文件,在 `ClusterConfiguration` 的 `networking` 对象下设置 `podSubnet` 字段。
{{< note >}}
<!--
After the command completes you should see something like so:
```sh
...
You can now join any number of control-plane node by running the following command on each as a root:
kubeadm join 192.168.0.200:6443 --token 9vr73a.a8uxyaju799qwdjv --discovery-token-ca-cert-hash sha256:7c2e69131a36ae2a042a339b33381c6d0d43887e2de83720eff5359e26aec866 --control-plane --certificate-key f8902e114ef118304e561c3ecd4d0b543adc226b7a07f675f56564185ffe0c07
Please note that the certificate-key gives access to cluster sensitive data, keep it secret!
As a safeguard, uploaded-certs will be deleted in two hours; If necessary, you can use kubeadm init phase upload-certs to reload certs afterward.
Then you can join any number of worker nodes by running the following on each as root:
kubeadm join 192.168.0.200:6443 --token 9vr73a.a8uxyaju799qwdjv --discovery-token-ca-cert-hash sha256:7c2e69131a36ae2a042a339b33381c6d0d43887e2de83720eff5359e26aec866
```
- Copy this output to a text file. You will need it later to join control plane and worker nodes to the cluster.
- When `--upload-certs` is used with `kubeadm init`, the certificates of the primary control plane
are encrypted and uploaded in the `kubeadm-certs` Secret.
- To re-upload the certificates and generate a new decryption key, use the following command on a control plane
node that is already joined to the cluster:
```sh
sudo kubeadm init phase upload-certs --upload-certs
```
- You can also specify a custom `--certificate-key` during `init` that can later be used by `join`.
To generate such a key you can use the following command:
```sh
kubeadm alpha certs certificate-key
```
-->
命令完成后,您应该会看到类似以下内容:
```sh
...
现在,您可以通过在根目录上运行以下命令来加入任意数量的控制平面节点:
kubeadm join 192.168.0.200:6443 --token 9vr73a.a8uxyaju799qwdjv --discovery-token-ca-cert-hash sha256:7c2e69131a36ae2a042a339b33381c6d0d43887e2de83720eff5359e26aec866 --control-plane --certificate-key f8902e114ef118304e561c3ecd4d0b543adc226b7a07f675f56564185ffe0c07
请注意,证书密钥可以访问集群内敏感数据,请保密!
为了安全起见,将在两个小时内删除上传的证书; 如有必要,您可以使用 kubeadm 初始化上传证书阶段,之后重新加载证书。
然后,您可以通过在根目录上运行以下命令来加入任意数量的工作节点:
kubeadm join 192.168.0.200:6443 --token 9vr73a.a8uxyaju799qwdjv --discovery-token-ca-cert-hash sha256:7c2e69131a36ae2a042a339b33381c6d0d43887e2de83720eff5359e26aec866
```
- 将此输出复制到文本文件。 稍后您将需要它来将控制平面节点和辅助节点加入集群。
- 当 `--upload-certs` 与 `kubeadm init` 一起使用时,主控制平面的证书被加密并上传到 `kubeadm-certs` 密钥中。
- 要重新上传证书并生成新的解密密钥,请在已加入集群节点的控制平面上使用以下命令:
```sh
sudo kubeadm init phase upload-certs --upload-certs
```
- 您还可以在 `init` 期间指定自定义的 `--certificate-key`,以后可以由 `join` 使用。
要生成这样的密钥,可以使用以下命令:
```sh
kubeadm alpha certs certificate-key
```
{{< note >}}
<!--
The `kubeadm-certs` Secret and decryption key expire after two hours.
-->
**注意** `kubeadm-certs` 密钥和解密密钥会在两个小时后失效。
{{< /note >}}
{{< caution >}}
<!--
As stated in the command output, the certificate key gives access to cluster sensitive data, keep it secret!
-->
正如命令输出中所述,证书密钥可访问群集敏感数据,并将其保密!
{{< /caution >}}
<!--
1. Apply the CNI plugin of your choice:
[Follow these instructions](/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/#pod-network) to install the CNI provider. Make sure the configuration corresponds to the Pod CIDR specified in the kubeadm configuration file if applicable.
In this example we are using Weave Net:
```sh
kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"
```
1. Type the following and watch the pods of the control plane components get started:
```sh
kubectl get pod -n kube-system -w
```
-->
1. 应用您选择的 CNI 插件:
[请遵循以下指示](/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/#pod-network) 安装 CNI 提供程序。如果适用,请确保配置与 kubeadm 配置文件中指定的 Pod CIDR 相对应。
在此示例中,我们使用 Weave Net:
```sh
kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"
```
1. 输入以下内容,并查看 pods 的控制平面组件启动:
```sh
kubectl get pod -n kube-system -w
```
<!--
### Steps for the rest of the control plane nodes
-->
### 其余控制平面节点的步骤
{{< note >}}
<!--
Since kubeadm version 1.15 you can join multiple control-plane nodes in parallel.
Prior to this version, you must join new control plane nodes sequentially, only after
the first node has finished initializing.
-->
**注意**:从 kubeadm 1.15 版本开始,您可以并行加入多个控制平面节点。
在此版本之前,您必须在第一个节点初始化后才能依序的增加新的控制平面节点。
{{< /note >}}
<!--
For each additional control plane node you should:
1. Execute the join command that was previously given to you by the `kubeadm init` output on the first node.
It should look something like this:
```sh
sudo kubeadm join 192.168.0.200:6443 --token 9vr73a.a8uxyaju799qwdjv --discovery-token-ca-cert-hash sha256:7c2e69131a36ae2a042a339b33381c6d0d43887e2de83720eff5359e26aec866 --control-plane --certificate-key f8902e114ef118304e561c3ecd4d0b543adc226b7a07f675f56564185ffe0c07
```
- The `--control-plane` flag tells `kubeadm join` to create a new control plane.
- The `--certificate-key ...` will cause the control plane certificates to be downloaded
from the `kubeadm-certs` Secret in the cluster and be decrypted using the given key.
-->
对于每个其他控制平面节点,您应该:
1. 执行先前由第一个节点上的 `kubeadm init` 输出提供给您的 join 命令。
它看起来应该像这样:
```sh
sudo kubeadm join 192.168.0.200:6443 --token 9vr73a.a8uxyaju799qwdjv --discovery-token-ca-cert-hash sha256:7c2e69131a36ae2a042a339b33381c6d0d43887e2de83720eff5359e26aec866 --control-plane --certificate-key f8902e114ef118304e561c3ecd4d0b543adc226b7a07f675f56564185ffe0c07
```
- 这个 `--control-plane` 命令通知 `kubeadm join` 创建一个新的控制平面。
- `--certificate-key ...` 将导致从集群中的 `kubeadm-certs` 秘钥下载控制平面证书并使用给定的密钥进行解密。
<!--
## External etcd nodes
Setting up a cluster with external etcd nodes is similar to the procedure used for stacked etcd
with the exception that you should setup etcd first, and you should pass the etcd information
in the kubeadm config file.
-->
## 外部 etcd 节点
使用外部 etcd 节点设置集群类似于用于堆叠 etcd 的过程,
不同之处在于您应该首先设置 etcd,并在 kubeadm 配置文件中传递 etcd 信息。
<!--
### Set up the etcd cluster
1. Follow [these instructions](/docs/setup/production-environment/tools/kubeadm/setup-ha-etcd-with-kubeadm/) to set up the etcd cluster.
1. Setup SSH as described [here](#manual-certs).
1. Copy the following files from any etcd node in the cluster to the first control plane node:
```sh
export CONTROL_PLANE="ubuntu@10.0.0.7"
scp /etc/kubernetes/pki/etcd/ca.crt "${CONTROL_PLANE}":
scp /etc/kubernetes/pki/apiserver-etcd-client.crt "${CONTROL_PLANE}":
scp /etc/kubernetes/pki/apiserver-etcd-client.key "${CONTROL_PLANE}":
```
- Replace the value of `CONTROL_PLANE` with the `user@host` of the first control plane machine.
-->
### 设置 ectd 集群
1. 按照 [这些指示](/docs/setup/production-environment/tools/kubeadm/setup-ha-etcd-with-kubeadm/) 去设置 etcd 集群。
1. 设置 SSH 在 [这](#manual-certs)描述。
1. 将以下文件从集群中的任何 etcd 节点复制到第一个控制平面节点:
```sh
export CONTROL_PLANE="ubuntu@10.0.0.7"
scp /etc/kubernetes/pki/etcd/ca.crt "${CONTROL_PLANE}":
scp /etc/kubernetes/pki/apiserver-etcd-client.crt "${CONTROL_PLANE}":
scp /etc/kubernetes/pki/apiserver-etcd-client.key "${CONTROL_PLANE}":
```
- 用第一台控制平面机的 `user@host` 替换 `CONTROL_PLANE` 的值。
<!--
### Set up the first control plane node
1. Create a file called `kubeadm-config.yaml` with the following contents:
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
kubernetesVersion: stable
controlPlaneEndpoint: "LOAD_BALANCER_DNS:LOAD_BALANCER_PORT"
etcd:
external:
endpoints:
- https://ETCD_0_IP:2379
- https://ETCD_1_IP:2379
- https://ETCD_2_IP:2379
caFile: /etc/kubernetes/pki/etcd/ca.crt
certFile: /etc/kubernetes/pki/apiserver-etcd-client.crt
keyFile: /etc/kubernetes/pki/apiserver-etcd-client.key
-->
### 设置第一个控制平面节点
1. 用以下内容创建一个名为 `kubeadm-config.yaml` 的文件:
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
kubernetesVersion: stable
controlPlaneEndpoint: "LOAD_BALANCER_DNS:LOAD_BALANCER_PORT"
etcd:
external:
endpoints:
- https://ETCD_0_IP:2379
- https://ETCD_1_IP:2379
- https://ETCD_2_IP:2379
caFile: /etc/kubernetes/pki/etcd/ca.crt
certFile: /etc/kubernetes/pki/apiserver-etcd-client.crt
keyFile: /etc/kubernetes/pki/apiserver-etcd-client.key
{{< note >}}
<!--
The difference between stacked etcd and external etcd here is that the external etcd setup requires
a configuration file with the etcd endpoints under the `external` object for `etcd`.
In the case of the stacked etcd topology this is managed automatically.
-->
这里堆 etcd 和外部 etcd 之前的区别在于设置外部 etcd 需要一个 `etcd` 的 `external` 对象下带有 etcd 端点的配置文件。
如果是堆 etcd 技术,是自动管理的。
{{< /note >}}
<!--
- Replace the following variables in the config template with the appropriate values for your cluster:
- `LOAD_BALANCER_DNS`
- `LOAD_BALANCER_PORT`
- `ETCD_0_IP`
- `ETCD_1_IP`
- `ETCD_2_IP`
The following steps are similar to the stacked etcd setup:
1. Run `sudo kubeadm init --config kubeadm-config.yaml --upload-certs` on this node.
1. Write the output join commands that are returned to a text file for later use.
1. Apply the CNI plugin of your choice. The given example is for Weave Net:
```sh
kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"
```
-->
- 在您的集群中,将配置模板中的以下变量替换为适当值:
- `LOAD_BALANCER_DNS`
- `LOAD_BALANCER_PORT`
- `ETCD_0_IP`
- `ETCD_1_IP`
- `ETCD_2_IP`
以下的步骤与设置堆集群是相似的:
1. 在节点上运行 `sudo kubeadm init --config kubeadm-config.yaml --upload-certs` 命令。
1. 编写输出联接命令,这些命令将返回到文本文件以供以后使用。
1. 应用您选择的 CNI 插件。 给定以下示例适用于 Weave Net:
```sh
kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"
```
<!--
### Steps for the rest of the control plane nodes
The steps are the same as for the stacked etcd setup:
- Make sure the first control plane node is fully initialized.
- Join each control plane node with the join command you saved to a text file. It's recommended
to join the control plane nodes one at a time.
- Don't forget that the decryption key from `--certificate-key` expires after two hours, by default.
-->
### 其他控制平面节点的步骤
步骤与设置堆 etcd 相同:
- 确保第一个控制平面节点已完全初始化。
- 使用保存到文本文件的连接命令将每个控制平面节点连接在一起。建议一次加入一个控制平面节点。
- 不要忘记默认情况下,`--certificate-key` 中的解密秘钥会在两个小时后过期。
<!-- ## Common tasks after bootstrapping control plane -->
## 列举控制平面之后的常见任务
<!-- ### Install workers -->
### 安装工作节点
<!--
Worker nodes can be joined to the cluster with the command you stored previously
as the output from the `kubeadm init` command:
```sh
sudo kubeadm join 192.168.0.200:6443 --token 9vr73a.a8uxyaju799qwdjv --discovery-token-ca-cert-hash sha256:7c2e69131a36ae2a042a339b33381c6d0d43887e2de83720eff5359e26aec866
```
-->
您可以使用之前存储的命令将工作节点加入集群中
作为 `kubeadm init` 命令的输出:
```sh
sudo kubeadm join 192.168.0.200:6443 --token 9vr73a.a8uxyaju799qwdjv --discovery-token-ca-cert-hash sha256:7c2e69131a36ae2a042a339b33381c6d0d43887e2de83720eff5359e26aec866
```
<!--
## Manual certificate distribution {#manual-certs}
If you choose to not use `kubeadm init` with the `--upload-certs` flag this means that
you are going to have to manually copy the certificates from the primary control plane node to the
joining control plane nodes.
There are many ways to do this. In the following example we are using `ssh` and `scp`:
SSH is required if you want to control all nodes from a single machine.
-->
## 手动证书分发 {#manual-certs}
如果您选择不将 `kubeadm init` 与 `--upload-certs` 命令一起使用,
则意味着您将必须手动将证书从主控制平面节点复制到
将要加入的控制平面节点上。
有许多方法可以实现这种操作。在下面的例子中我们使用 `ssh` 和 `scp`
如果要在单独的一台计算机控制所有节点,则需要 SSH。
<!--
1. Enable ssh-agent on your main device that has access to all other nodes in
the system:
```
eval $(ssh-agent)
```
1. Add your SSH identity to the session:
```
ssh-add ~/.ssh/path_to_private_key
```
1. SSH between nodes to check that the connection is working correctly.
- When you SSH to any node, make sure to add the `-A` flag:
```
ssh -A 10.0.0.7
```
- When using sudo on any node, make sure to preserve the environment so SSH
forwarding works:
```
sudo -E -s
```
-->
1. 在您的主设备上启动 ssh-agent,要求该设备能访问系统中的所有其他节点:
```
eval $(ssh-agent)
```
1. 将 SSH 身份添加到会话中:
```
ssh-add ~/.ssh/path_to_private_key
```
1. 检查节点间的 SSH 以确保连接是正常运行的
- SSH 到任何节点时,请确保添加 `-A` 标志:
```
ssh -A 10.0.0.7
```
- 当在任何节点上使用 sudo 时,请确保环境完善,以便使用 SSH
转发任务:
```
sudo -E -s
```
<!--
1. After configuring SSH on all the nodes you should run the following script on the first control plane node after
running `kubeadm init`. This script will copy the certificates from the first control plane node to the other
control plane nodes:
In the following example, replace `CONTROL_PLANE_IPS` with the IP addresses of the
other control plane nodes.
```sh
USER=ubuntu # customizable
CONTROL_PLANE_IPS="10.0.0.7 10.0.0.8"
for host in ${CONTROL_PLANE_IPS}; do
scp /etc/kubernetes/pki/ca.crt "${USER}"@$host:
scp /etc/kubernetes/pki/ca.key "${USER}"@$host:
scp /etc/kubernetes/pki/sa.key "${USER}"@$host:
scp /etc/kubernetes/pki/sa.pub "${USER}"@$host:
scp /etc/kubernetes/pki/front-proxy-ca.crt "${USER}"@$host:
scp /etc/kubernetes/pki/front-proxy-ca.key "${USER}"@$host:
scp /etc/kubernetes/pki/etcd/ca.crt "${USER}"@$host:etcd-ca.crt
scp /etc/kubernetes/pki/etcd/ca.key "${USER}"@$host:etcd-ca.key
done
```
-->
1. 在所有节点上配置 SSH 之后,您应该在运行过 `kubeadm init` 命令的第一个控制平面节点上运行以下脚本。
该脚本会将证书从第一个控制平面节点复制到另一个控制平面节点:
在以下示例中,用其他控制平面节点的 IP 地址替换 `CONTROL_PLANE_IPS`。
```sh
USER=ubuntu # 可自己设置
CONTROL_PLANE_IPS="10.0.0.7 10.0.0.8"
for host in ${CONTROL_PLANE_IPS}; do
scp /etc/kubernetes/pki/ca.crt "${USER}"@$host:
scp /etc/kubernetes/pki/ca.key "${USER}"@$host:
scp /etc/kubernetes/pki/sa.key "${USER}"@$host:
scp /etc/kubernetes/pki/sa.pub "${USER}"@$host:
scp /etc/kubernetes/pki/front-proxy-ca.crt "${USER}"@$host:
scp /etc/kubernetes/pki/front-proxy-ca.key "${USER}"@$host:
scp /etc/kubernetes/pki/etcd/ca.crt "${USER}"@$host:etcd-ca.crt
scp /etc/kubernetes/pki/etcd/ca.key "${USER}"@$host:etcd-ca.key
done
```
{{< caution >}}
<!--
Copy only the certificates in the above list. kubeadm will take care of generating the rest of the certificates
with the required SANs for the joining control-plane instances. If you copy all the certificates by mistake,
the creation of additional nodes could fail due to a lack of required SANs.
-->
只需要复制上面列表中的证书。kubeadm 将负责生成其余证书以及加入控制平面实例所需的 SAN。
如果您错误地复制了所有证书,由于缺少所需的 SAN,创建其他节点可能会失败。
{{< /caution >}}
<!--
1. Then on each joining control plane node you have to run the following script before running `kubeadm join`.
This script will move the previously copied certificates from the home directory to `/etc/kubernetes/pki`:
```sh
USER=ubuntu # customizable
mkdir -p /etc/kubernetes/pki/etcd
mv /home/${USER}/ca.crt /etc/kubernetes/pki/
mv /home/${USER}/ca.key /etc/kubernetes/pki/
mv /home/${USER}/sa.pub /etc/kubernetes/pki/
mv /home/${USER}/sa.key /etc/kubernetes/pki/
mv /home/${USER}/front-proxy-ca.crt /etc/kubernetes/pki/
mv /home/${USER}/front-proxy-ca.key /etc/kubernetes/pki/
mv /home/${USER}/etcd-ca.crt /etc/kubernetes/pki/etcd/ca.crt
mv /home/${USER}/etcd-ca.key /etc/kubernetes/pki/etcd/ca.key
```
-->
1. 然后,在每个连接控制平面节点上,您必须先运行以下脚本,然后再运行 `kubeadm join`。
该脚本会将先前复制的证书从主目录移动到 `/etc/kubernetes/pki`
```sh
USER=ubuntu # 可自己设置
mkdir -p /etc/kubernetes/pki/etcd
mv /home/${USER}/ca.crt /etc/kubernetes/pki/
mv /home/${USER}/ca.key /etc/kubernetes/pki/
mv /home/${USER}/sa.pub /etc/kubernetes/pki/
mv /home/${USER}/sa.key /etc/kubernetes/pki/
mv /home/${USER}/front-proxy-ca.crt /etc/kubernetes/pki/
mv /home/${USER}/front-proxy-ca.key /etc/kubernetes/pki/
mv /home/${USER}/etcd-ca.crt /etc/kubernetes/pki/etcd/ca.crt
mv /home/${USER}/etcd-ca.key /etc/kubernetes/pki/etcd/ca.key
```
{{% /capture %}}
@@ -0,0 +1,381 @@
---
reviewers:
- sig-cluster-lifecycle
title: 使用 kubeadm 配置集群中的每个 kubelet
content_template: templates/concept
weight: 80
---
<!--
---
reviewers:
- sig-cluster-lifecycle
title: Configuring each kubelet in your cluster using kubeadm
content_template: templates/concept
weight: 80
---
-->
{{% capture overview %}}
{{< feature-state for_k8s_version="1.11" state="stable" >}}
<!--
The lifecycle of the kubeadm CLI tool is decoupled from the
[kubelet](/docs/reference/command-line-tools-reference/kubelet), which is a daemon that runs
on each node within the Kubernetes cluster. The kubeadm CLI tool is executed by the user when Kubernetes is
initialized or upgraded, where as the kubelet is always running in the background.
Since the kubelet is a daemon, it needs to be maintained by some kind of a init
system or service manager. When the kubelet is installed using DEBs or RPMs,
systemd is configured to manage the kubelet. You can use a different service
manager instead, but you need to configure it manually.
Some kubelet configuration details need to be the same across all kubelets involved in the cluster, while
other configuration aspects need to be set on a per-kubelet basis, to accommodate the different
characteristics of a given machine, such as OS, storage, and networking. You can manage the configuration
of your kubelets manually, but [kubeadm now provides a `KubeletConfiguration` API type for managing your
kubelet configurations centrally](#configure-kubelets-using-kubeadm).
-->
kubeadm CLI 工具的生命周期与 [kubelet](/docs/reference/command-line-tools-reference/kubelet)解耦,它是一个守护程序,在 Kubernetes 集群中的每个节点上运行。
当 Kubernetes 初始化或升级时,kubeadm CLI 工具由用户执行,而 kubelet 始终在后台运行。
由于kubelet是守护程序,因此需要通过某种初始化系统或服务管理器进行维护。
当使用 DEB 或 RPM 安装 kubelet 时,配置系统去管理 kubelet。
您可以改用其他服务管理器,但需要手动地配置。
集群中涉及的所有 kubelet 的一些配置细节都必须相同,而其他配置方面则需要基于每个 kubelet 进行设置,以适应给定机器的不同特性,例如操作系统、存储和网络。
您可以手动地管理 kubelet 的配置,但是 [kubeadm 现在提供一种 `KubeletConfiguration` API 类型,用于集中管理 kubelet 的配置](#configure-kubelets-using-kubeadm)。
{{% /capture %}}
{{% capture body %}}
<!--
## Kubelet configuration patterns
The following sections describe patterns to kubelet configuration that are simplified by
using kubeadm, rather than managing the kubelet configuration for each Node manually.
-->
## Kubelet 配置模式
以下各节讲述了通过使用 kubeadm 简化 kubelet 配置模式,而不是在每个节点上手动地管理 kubelet 配置。
<!--
### Propagating cluster-level configuration to each kubelet
You can provide the kubelet with default values to be used by `kubeadm init` and `kubeadm join`
commands. Interesting examples include using a different CRI runtime or setting the default subnet
used by services.
If you want your services to use the subnet `10.96.0.0/12` as the default for services, you can pass
the `--service-cidr` parameter to kubeadm:
```bash
kubeadm init --service-cidr 10.96.0.0/12
```
Virtual IPs for services are now allocated from this subnet. You also need to set the DNS address used
by the kubelet, using the `--cluster-dns` flag. This setting needs to be the same for every kubelet
on every manager and Node in the cluster. The kubelet provides a versioned, structured API object
that can configure most parameters in the kubelet and push out this configuration to each running
kubelet in the cluster. This object is called **the kubelet's ComponentConfig**.
The ComponentConfig allows the user to specify flags such as the cluster DNS IP addresses expressed as
a list of values to a camelCased key, illustrated by the following example:
```yaml
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
clusterDNS:
- 10.96.0.10
```
-->
### 将集群级配置传播到每个 kubelet 中
您可以通过使用 `kubeadm init``kubeadm join` 命令为 kubelet 提供默认值。
有趣的示例包括使用其他 CRI 运行时或通过服务器设置不同的默认子网。
如果您想使用子网 `10.96.0.0/12` 作为默认的服务,您可以给 kubeadm 传递 `--service-cidr` 参数:
```bash
kubeadm init --service-cidr 10.96.0.0/12
```
现在,可以从该子网分配服务的虚拟 IP。
您还需要通过 kubelet 使用 `--cluster-dns` 标志设置 DNS 地址。
在集群中的每个管理器和节点上的 kubelet 的设置需要相同。
kubelet 提供了一个版本化的结构化 API 对象,该对象可以配置 kubelet 中的大多数参数,并将此配置推送到集群中正在运行的每个 kubelet 上。
此对象被称为 **kubelet 的配置组件**
该配置组件允许用户指定标志,例如用骆峰值代表集群的 DNS IP 地址,如下所示:
```yaml
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
clusterDNS:
- 10.96.0.10
```
有关组件配置的更多详细信息,亲参阅 [本节](#configure-kubelets-using-kubeadm)。
<!--
### Providing instance-specific configuration details
Some hosts require specific kubelet configurations, due to differences in hardware, operating system,
networking, or other host-specific parameters. The following list provides a few examples.
- The path to the DNS resolution file, as specified by the `--resolv-conf` kubelet
configuration flag, may differ among operating systems, or depending on whether you are using
`systemd-resolved`. If this path is wrong, DNS resolution will fail on the Node whose kubelet
is configured incorrectly.
- The Node API object `.metadata.name` is set to the machine's hostname by default,
unless you are using a cloud provider. You can use the `--hostname-override` flag to override the
default behavior if you need to specify a Node name different from the machine's hostname.
- Currently, the kubelet cannot automatically detects the cgroup driver used by the CRI runtime,
but the value of `--cgroup-driver` must match the cgroup driver used by the CRI runtime to ensure
the health of the kubelet.
- Depending on the CRI runtime your cluster uses, you may need to specify different flags to the kubelet.
For instance, when using Docker, you need to specify flags such as `--network-plugin=cni`, but if you
are using an external runtime, you need to specify `--container-runtime=remote` and specify the CRI
endpoint using the `--container-runtime-path-endpoint=<path>`.
You can specify these flags by configuring an individual kubelet's configuration in your service manager,
such as systemd.
-->
### 提供指定实例的详细配置信息
由于硬件、操作系统、网络或者其他主机特定参数的差异。某些主机需要特定的 kubelet 配置。
以下列表提供了一些示例。
- 由 kubelet 配置标志 `--resolv-confkubelet` 指定的 DNS 解析文件的路径在操作系统之间可能有所不同,
它取决于您是否使用 `systemd-resolved`
如果此路径错误,则在其 kubelet 配置错误的节点上 DNS 解析也将失败。
- 除非您使用云提供商,否则默认情况下,Node API 对象 `.metadata.name` 被设置为计算机的主机名。
如果您需要指定一个节点的名称与机器的主机名不同,您可以是使用 `--hostname-override` 标志覆盖默认操作。
- 当前,kubelet 无法自动检测 CRI 运行时使用的 cgroup 驱动程序,
但是值 `--cgroup-driver` 必须与 CRI 运行时使用的 cgroup 驱动程序匹配,以确保 kubelet 的健康运行状况。
- 根据您集群使用的 CRI 运行时,您可能需要为 kubelet 指定不同的标志。
例如,当使用 Docker 时,你要需要指定标志如 `--network-plugin=cni`,但是如果您使用的是外部运行时,
则需要指定 `--container-runtime=remote` 并使用 `--container-runtime-path-endpoint=<path>` 指定 CRI端点。
您可以在服务管理器(例如系统)中通过对单个的 kubelet 配置来指定这些标志。
<!--
## Configure kubelets using kubeadm
It is possible to configure the kubelet that kubeadm will start if a custom `KubeletConfiguration`
API object is passed with a configuration file like so `kubeadm ... --config some-config-file.yaml`.
By calling `kubeadm config print init-defaults --component-configs KubeletConfiguration` you can
see all the default values for this structure.
Also have a look at the [API reference for the
kubelet ComponentConfig](https://godoc.org/k8s.io/kubernetes/pkg/kubelet/apis/config#KubeletConfiguration)
for more information on the individual fields.
-->
## 使用 kubeadm 配置 kubelet
如果自定义的 `KubeletConfiguration` API 对象使用像 `kubeadm ... --config some-config-file.yaml` 这样的配置文件进行传递,则可以配置 kubeadm 启动的 kubelet。
通过调用 `kubeadm config print init-defaults --component-configs KubeletConfiguration` 您可以看到此结构中的所有默认值。
也可以阅读 [kubelet 配置组件的 API 参考](https://godoc.org/k8s.io/kubernetes/pkg/kubelet/apis/config#KubeletConfiguration)来获取有关各个字段的更多信息。
<!--
### Workflow when using `kubeadm init`
When you call `kubeadm init`, the kubelet configuration is marshalled to disk
at `/var/lib/kubelet/config.yaml`, and also uploaded to a ConfigMap in the cluster. The ConfigMap
is named `kubelet-config-1.X`, where `.X` is the minor version of the Kubernetes version you are
initializing. A kubelet configuration file is also written to `/etc/kubernetes/kubelet.conf` with the
baseline cluster-wide configuration for all kubelets in the cluster. This configuration file
points to the client certificates that allow the kubelet to communicate with the API server. This
addresses the need to
[propagate cluster-level configuration to each kubelet](#propagating-cluster-level-configuration-to-each-kubelet).
To address the second pattern of
[providing instance-specific configuration details](#providing-instance-specific-configuration-details),
kubeadm writes an environment file to `/var/lib/kubelet/kubeadm-flags.env`, which contains a list of
flags to pass to the kubelet when it starts. The flags are presented in the file like this:
```bash
KUBELET_KUBEADM_ARGS="--flag1=value1 --flag2=value2 ..."
```
In addition to the flags used when starting the kubelet, the file also contains dynamic
parameters such as the cgroup driver and whether to use a different CRI runtime socket
(`--cri-socket`).
After marshalling these two files to disk, kubeadm attempts to run the following two
commands, if you are using systemd:
```bash
systemctl daemon-reload && systemctl restart kubelet
```
If the reload and restart are successful, the normal `kubeadm init` workflow continues.
-->
### 当使用 `kubeadm init`时的工作流程
当调用 `kubeadm init` 时,kubelet 配置被编组到磁盘上的 `/var/lib/kubelet/config.yaml` 中,
并且上传到集群中的 ConfigMap。
ConfigMap 名为 `kubelet-config-1.X`,其中 `.X` 是您正在初始化的 kubernetes 版本的次版本。
在集群中所有 kubelet 的基准集群范围内配置,将 kubelet 配置文件写入 `/etc/kubernetes/kubelet.conf` 中。
此配置文件指向允许 kubelet 与 API 服务器通信的客户端证书。
这解决了 [将集群级配置传播到每个 kubelet](#propagating-cluster-level-configuration-to-each-kubelet)的需求。
该文档 [提供特定实例的配置详细信息](#providing-instance-specific-configuration-details) 是第二种解决模式,
kubeadm 将环境文件写入 `/var/lib/kubelet/kubeadm-flags.env`,其中包含了一个标志列表,
当 kubelet 启动时,该标志列表会传递给 kubelet 标志在文件中的显示方式如下:
```bash
KUBELET_KUBEADM_ARGS="--flag1=value1 --flag2=value2 ..."
```
除了启动 kubelet 时使用该标志外,该文件还包含动态参数,例如 cgroup 驱动程序以及是否使用其他 CRI 运行时 socket(`--cri-socket`)。
将这两个文件编组到磁盘后,如果使用 systemd,则 kubeadm 尝试运行以下两个命令:
```bash
systemctl daemon-reload && systemctl restart kubelet
```
如果重新加载和重新启动成功,则正常的 `kubeadm init` 工作流程将继续。
<!--
### Workflow when using `kubeadm join`
When you run `kubeadm join`, kubeadm uses the Bootstrap Token credential to perform
a TLS bootstrap, which fetches the credential needed to download the
`kubelet-config-1.X` ConfigMap and writes it to `/var/lib/kubelet/config.yaml`. The dynamic
environment file is generated in exactly the same way as `kubeadm init`.
Next, `kubeadm` runs the following two commands to load the new configuration into the kubelet:
```bash
systemctl daemon-reload && systemctl restart kubelet
```
After the kubelet loads the new configuration, kubeadm writes the
`/etc/kubernetes/bootstrap-kubelet.conf` KubeConfig file, which contains a CA certificate and Bootstrap
Token. These are used by the kubelet to perform the TLS Bootstrap and obtain a unique
credential, which is stored in `/etc/kubernetes/kubelet.conf`. When this file is written, the kubelet
has finished performing the TLS Bootstrap.
-->
### 当使用 `kubeadm join`时的工作流程
当运行 `kubeadm join` 时,kubeadm 使用 Bootstrap Token 证书执行 TLS 引导,该引导会获取一份证书,该证书需要下载 `kubelet-config-1.X` ConfigMap 并把它写入 `/var/lib/kubelet/config.yaml` 中。
动态环境文件的生成方式恰好与 `kubeadm init` 相同。
接下来,kubeadm 运行以下两个命令将新配置加载到 kubelet 中:
```bash
systemctl daemon-reload && systemctl restart kubelet
```
在 kubelet 加载新配置后,kubeadm 将写入 `/etc/kubernetes/bootstrap-kubelet.conf` KubeConfig 文件中,
该文件包含 CA 证书和引导程序令牌。
kubelet 使用这些证书执行 TLS 引导程序并获取唯一的凭据,该凭据被存储在 `/etc/kubernetes/kubelet.conf` 中。
当此文件被写入后,kubelet 就完成了执行 TLS 引导程序。
<!--
## The kubelet drop-in file for systemd
kubeadm ships with configuration for how systemd should run the kubelet.
Note that the kubeadm CLI command never touches this drop-in file.
This configuration file installed by the `kubeadm` [DEB](https://github.com/kubernetes/kubernetes/blob/master/build/debs/10-kubeadm.conf) or [RPM package](https://github.com/kubernetes/kubernetes/blob/master/build/rpms/10-kubeadm.conf) is written to
`/etc/systemd/system/kubelet.service.d/10-kubeadm.conf` and is used by systemd.
It augments the basic [`kubelet.service` for RPM](https://github.com/kubernetes/kubernetes/blob/master/build/rpms/kubelet.service) (resp. [`kubelet.service` for DEB](https://github.com/kubernetes/kubernetes/blob/master/build/debs/kubelet.service))):
```none
[Service]
Environment="KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf
--kubeconfig=/etc/kubernetes/kubelet.conf"
Environment="KUBELET_CONFIG_ARGS=--config=/var/lib/kubelet/config.yaml"
# This is a file that "kubeadm init" and "kubeadm join" generate at runtime, populating
the KUBELET_KUBEADM_ARGS variable dynamically
EnvironmentFile=-/var/lib/kubelet/kubeadm-flags.env
# This is a file that the user can use for overrides of the kubelet args as a last resort.
# Preferably, the user should use the .NodeRegistration.KubeletExtraArgs object in the configuration files instead.
# KUBELET_EXTRA_ARGS should be sourced from this file.
EnvironmentFile=-/etc/default/kubelet
ExecStart=
ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS
```
This file specifies the default locations for all of the files managed by kubeadm for the kubelet.
- The KubeConfig file to use for the TLS Bootstrap is `/etc/kubernetes/bootstrap-kubelet.conf`,
but it is only used if `/etc/kubernetes/kubelet.conf` does not exist.
- The KubeConfig file with the unique kubelet identity is `/etc/kubernetes/kubelet.conf`.
- The file containing the kubelet's ComponentConfig is `/var/lib/kubelet/config.yaml`.
- The dynamic environment file that contains `KUBELET_KUBEADM_ARGS` is sourced from `/var/lib/kubelet/kubeadm-flags.env`.
- The file that can contain user-specified flag overrides with `KUBELET_EXTRA_ARGS` is sourced from
`/etc/default/kubelet` (for DEBs), or `/etc/sysconfig/kubelet` (for RPMs). `KUBELET_EXTRA_ARGS`
is last in the flag chain and has the highest priority in the event of conflicting settings.
-->
## 系统中的 kubelet 插件
kubeadm 中附带了有关系统如何运行 kubelet 的配置。
请注意 kubeadm CLI 命令不会触及此插件。
通过 `kubeadm` [DEB](https://github.com/kubernetes/kubernetes/blob/master/build/debs/10-kubeadm.conf) 或者 [RPM 包](https://github.com/kubernetes/kubernetes/blob/master/build/rpms/10-kubeadm.conf) 安装的配置文件已被写入 `/etc/systemd/system/kubelet.service.d/10-kubeadm.conf` 并由系统使用。
它加强了基础设施 [`kubelet.service` for RPM](https://github.com/kubernetes/kubernetes/blob/master/build/rpms/kubelet.service) (resp. [`kubelet.service` for DEB](https://github.com/kubernetes/kubernetes/blob/master/build/debs/kubelet.service)))
```none
[Service]
Environment="KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf
--kubeconfig=/etc/kubernetes/kubelet.conf"
Environment="KUBELET_CONFIG_ARGS=--config=/var/lib/kubelet/config.yaml"
# 这是 "kubeadm init" 和 "kubeadm join" 运行时生成的文件,动态地填充 KUBELET_KUBEADM_ARGS 变量
EnvironmentFile=-/var/lib/kubelet/kubeadm-flags.env
# 这是一个文件,用户在不得已下可以将其用作替代 kubelet args。
# 用户最好使用 .NodeRegistration.KubeletExtraArgs 对象在配置文件中替代。
# KUBELET_EXTRA_ARGS 应该从此文件中获取。
EnvironmentFile=-/etc/default/kubelet
ExecStart=
ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS
```
该文件为 kubelet 指定由 kubeadm 管理的所有文件的默认位置。
- 用于 TLS 引导程序的 KubeConfig 文件为 `/etc/kubernetes/bootstrap-kubelet.conf`
但仅当 `/etc/kubernetes/kubelet.conf` 不存在时才能使用。
- 具有唯一 kubelet 标识的 KubeConfig 文件为 `/etc/kubernetes/kubelet.conf`
- 包含 kubelet 的组件配置的文件为 `/var/lib/kubelet/config.yaml`
- 包含的动态环境的文件 `KUBELET_KUBEADM_ARGS` 是来源于 `/var/lib/kubelet/kubeadm-flags.env`
- 包含用户指定标志替代的文件 `KUBELET_EXTRA_ARGS` 是来源于
`/etc/default/kubelet`(对于 DEB),或者 `/etc/sysconfig/kubelet`(对于 RPM)。
`KUBELET_EXTRA_ARGS` 在标志链中排在最后,并且在设置冲突时具有最高优先级。
<!--
## Kubernetes binaries and package contents
The DEB and RPM packages shipped with the Kubernetes releases are:
| Package name | Description |
|--------------|-------------|
| `kubeadm` | Installs the `/usr/bin/kubeadm` CLI tool and the [kubelet drop-in file](#the-kubelet-drop-in-file-for-systemd) for the kubelet. |
| `kubelet` | Installs the `/usr/bin/kubelet` binary. |
| `kubectl` | Installs the `/usr/bin/kubectl` binary. |
| `kubernetes-cni` | Installs the official CNI binaries into the `/opt/cni/bin` directory. |
| `cri-tools` | Installs the `/usr/bin/crictl` binary from the [cri-tools git repository](https://github.com/kubernetes-incubator/cri-tools). |
-->
## Kubernetes 二进制文件和软件包内容
Kubernetes 版本对应的 DEB 和 RPM 软件包是:
| Package name | Description |
|--------------|-------------|
| `kubeadm` | 给 kubelet 安装 `/usr/bin/kubeadm` CLI 工具和 [kubelet 插件](#the-kubelet-drop-in-file-for-systemd)。 |
| `kubelet` | 安装 `/usr/bin/kubelet` 二进制文件。 |
| `kubectl` | 安装 `/usr/bin/kubectl` 二进制问价。 |
| `kubernetes-cni` | 将官方的 CNI 二进制文件安装到 `/opt/cni/bin` 目录中 |
| `cri-tools` | 从 [cri-tools git 仓库](https://github.com/kubernetes-incubator/cri-tools)中安装 `/usr/bin/crictl` 二进制文件。 |
{{% /capture %}}
@@ -0,0 +1,134 @@
---
reviewers:
- sig-cluster-lifecycle
title: 配置您的 kubernetes 集群以自托管控制平台
content_template: templates/concept
weight: 100
---
<!--
---
reviewers:
- sig-cluster-lifecycle
title: Configuring your kubernetes cluster to self-host the control plane
content_template: templates/concept
weight: 100
---
-->
{{% capture overview %}}
<!--
### Self-hosting the Kubernetes control plane {#self-hosting}
-->
### 自托管 Kubernetes 控制平台 {#self-hosting}
<!--
kubeadm allows you to experimentally create a _self-hosted_ Kubernetes control
plane. This means that key components such as the API server, controller
manager, and scheduler run as [DaemonSet pods](/docs/concepts/workloads/controllers/daemonset/)
configured via the Kubernetes API instead of [static pods](/docs/tasks/administer-cluster/static-pod/)
configured in the kubelet via static files.
-->
kubeadm 允许您实验性地创建 _self-hosted_ Kubernetes 控制平面。
这意味着 API 服务器,控制管理器和调度程序之类的关键组件将通过配置 Kubernetes API 以 [DaemonSet pods](/docs/concepts/workloads/controllers/daemonset/) 的身份运行,而不是通过静态文件将 [static pods](/docs/tasks/administer-cluster/static-pod/) 在 kubelet 中配置。
<!--
To create a self-hosted cluster see the
[kubeadm alpha selfhosting pivot](/docs/reference/setup-tools/kubeadm/kubeadm-alpha/#cmd-selfhosting) command.
-->
要创建自托管集群,请参见 [kubeadm alpha 自托管枢纽](/docs/reference/setup-tools/kubeadm/kubeadm-alpha/#cmd-selfhosting) 命令。
{{% /capture %}}
{{% capture body %}}
<!--
#### Caveats
-->
#### 警告
{{< caution >}}
<!--
This feature pivots your cluster into an unsupported state, rendering kubeadm unable
to manage you cluster any longer. This includes `kubeadm upgrade`.
-->
此功能将您的集群设置为不受支持的状态,从而使 kubeadm 无法再管理您的集群。
这包括 `kubeadm 升级`
{{< /caution >}}
<!--
1. Self-hosting in 1.8 and later has some important limitations. In particular, a
self-hosted cluster _cannot recover from a reboot of the control-plane node_
without manual intervention.
-->
1. 1.8及更高版本中的自托管功能有一些重要限制。
特别是,自托管集群在没有人工干预的情况下_无法从控制平面节点的重新启动中恢复_ 。
<!--
1. By default, self-hosted control plane Pods rely on credentials loaded from
[`hostPath`](/docs/concepts/storage/volumes/#hostpath)
volumes. Except for initial creation, these credentials are not managed by
kubeadm.
-->
1. 默认情况下,自托管的控制平面 Pod 依赖于从 [`hostPath`](/docs/concepts/storage/volumes/#hostpath) 卷加载的凭据。
除初始创建外,这些凭据不由 kubeadm 管理。
<!--
1. The self-hosted portion of the control plane does not include etcd,
which still runs as a static Pod.
-->
1. 控制平面的自托管部分不包括 etcd,后者仍作为静态 Pod 运行。
<!--
#### Process
-->
#### 处理
<!--
The self-hosting bootstrap process is documented in the [kubeadm design
document](https://github.com/kubernetes/kubeadm/blob/master/docs/design/design_v1.9.md#optional-self-hosting).
-->
自托管引导过程记录在 [kubeadm 设计文档](https://github.com/kubernetes/kubeadm/blob/master/docs/design/design_v1.9.md#optional-self-hosting) 中。
<!--
In summary, `kubeadm alpha selfhosting` works as follows:
-->
总而言之,`kubeadm alpha 自托管` 的工作原理如下:
<!--
1. Waits for this bootstrap static control plane to be running and
healthy. This is identical to the `kubeadm init` process without self-hosting.
-->
1. 等待此引导静态控制平面运行且良好。
这与没有自我托管的 `kubeadm init` 过程相同。
<!--
1. Uses the static control plane Pod manifests to construct a set of
DaemonSet manifests that will run the self-hosted control plane.
It also modifies these manifests where necessary, for example adding new volumes
for secrets.
-->
1. 使用静态控制平面 Pod 清单来构造一组 DaemonSet 清单,这些清单将运行自托管的控制平面。
它还会在必要时修改这些清单,例如添加新的秘密卷。
<!--
1. Creates DaemonSets in the `kube-system` namespace and waits for the
resulting Pods to be running.
-->
1.`kube-system` 名称空间中创建 DaemonSets ,并等待生成的 Pod 运行。
<!--
1. Once self-hosted Pods are operational, their associated static Pods are deleted
and kubeadm moves on to install the next component. This triggers kubelet to
stop those static Pods.
-->
1. 自托管 Pod 运行后,将删除其关联的静态 Pod,然后 kubeadm 继续安装下一个组件。
这将触发 kubelet 停止那些静态 Pod 。
<!--
1. When the original static control plane stops, the new self-hosted control
plane is able to bind to listening ports and become active.
-->
1. 当原始静态控制平面停止时,新的自托管控制平面能够绑定到侦听端口并变为活动状态。
{{% /capture %}}
@@ -0,0 +1,407 @@
---
title: 使用 kubeadm 创建一个高可用 etcd 集群
content_template: templates/task
weight: 60
---
<!--
---
reviewers:
- sig-cluster-lifecycle
title: Set up a High Availability etcd cluster with kubeadm
content_template: templates/task
weight: 60
---
-->
{{% capture overview %}}
<!--
Kubeadm defaults to running a single member etcd cluster in a static pod managed
by the kubelet on the control plane node. This is not a high availability setup
as the etcd cluster contains only one member and cannot sustain any members
becoming unavailable. This task walks through the process of creating a high
availability etcd cluster of three members that can be used as an external etcd
when using kubeadm to set up a kubernetes cluster.
-->
默认情况下,kubeadm 运行单成员的 etcd 集群,该集群由控制面节点上的 kubelet 以静态 Pod 的方式进行管理。由于 etcd 集群只包含一个成员且不能在任一成员不可用时保持运行,所以这不是一种高可用设置。本任务,将告诉您如何在使用 kubeadm 创建一个 kubernetes 集群时创建一个外部 etcd:有三个成员的高可用 etcd 集群。
{{% /capture %}}
{{% capture prerequisites %}}
<!--
* Three hosts that can talk to each other over ports 2379 and 2380. This document assumes these default ports. However, they are configurable through the kubeadm config file.
-->
* 三个可以通过 2379 和 2380 端口相互通信的主机。本文档使用这些作为默认端口。不过,它们可以通过 kubeadm 的配置文件进行自定义。
<!--
* Each host must [have docker, kubelet, and kubeadm installed][toolbox].
-->
* 每个主机必须 [安装有 docker、kubelet 和 kubeadm][工具箱]。
<!--
* Some infrastructure to copy files between hosts. For example `ssh` and `scp` can satisfy this requirement.
-->
* 一些可以用来在主机间复制文件的基础设施。例如 `ssh``scp` 就可以满足需求。
<!--
[toolbox]: /docs/setup/production-environment/tools/kubeadm/install-kubeadm/
-->
[工具箱]: /docs/setup/production-environment/tools/kubeadm/install-kubeadm/
{{% /capture %}}
{{% capture steps %}}
<!--
## Setting up the cluster
-->
## 建立集群
<!--
The general approach is to generate all certs on one node and only distribute the *necessary* files to the other nodes.
-->
一般来说,是在一个节点上生成所有证书并且只分发这些*必要*的文件到其它节点上。
{{< note >}}
<!--
kubeadm contains all the necessary crytographic machinery to generate the certificates described below; no other cryptographic tooling is required for this example.
-->
kubeadm 包含生成下述证书所需的所有必要的密码学工具;在这个例子中,不需要其他加密工具。
{{< /note >}}
<!--
1. Configure the kubelet to be a service manager for etcd.
Since etcd was created first, you must override the service priority by creating a new unit file
that has higher precedence than the kubeadm-provided kubelet unit file.
-->
1. 将 kubelet 配置为 etcd 的服务管理器。
由于 etcd 是首先创建的,因此您必须通过创建具有更高优先级的新文件来覆盖 kubeadm 提供的 kubelet 单元文件。
```sh
cat << EOF > /etc/systemd/system/kubelet.service.d/20-etcd-service-manager.conf
[Service]
ExecStart=
# Replace "systemd" with the cgroup driver of your container runtime. The default value in the kubelet is "cgroupfs".
ExecStart=/usr/bin/kubelet --address=127.0.0.1 --pod-manifest-path=/etc/kubernetes/manifests --cgroup-driver=systemd
Restart=always
EOF
systemctl daemon-reload
systemctl restart kubelet
```
<!--
1.Create configuration files for kubeadm.
Generate one kubeadm configuration file for each host that will have an etcd
member running on it using the following script.
-->
1. 为 kubeadm 创建配置文件。
使用以下脚本为每个将要运行 etcd 成员的主机生成一个 kubeadm 配置文件。
<!--
```sh
# Update HOST0, HOST1, and HOST2 with the IPs or resolvable names of your hosts
export HOST0=10.0.0.6
export HOST1=10.0.0.7
export HOST2=10.0.0.8
# Create temp directories to store files that will end up on other hosts.
mkdir -p /tmp/${HOST0}/ /tmp/${HOST1}/ /tmp/${HOST2}/
ETCDHOSTS=(${HOST0} ${HOST1} ${HOST2})
NAMES=("infra0" "infra1" "infra2")
for i in "${!ETCDHOSTS[@]}"; do
HOST=${ETCDHOSTS[$i]}
NAME=${NAMES[$i]}
cat << EOF > /tmp/${HOST}/kubeadmcfg.yaml
apiVersion: "kubeadm.k8s.io/v1beta2"
kind: ClusterConfiguration
etcd:
local:
serverCertSANs:
- "${HOST}"
peerCertSANs:
- "${HOST}"
extraArgs:
initial-cluster: infra0=https://${ETCDHOSTS[0]}:2380,infra1=https://${ETCDHOSTS[1]}:2380,infra2=https://${ETCDHOSTS[2]}:2380
initial-cluster-state: new
name: ${NAME}
listen-peer-urls: https://${HOST}:2380
listen-client-urls: https://${HOST}:2379
advertise-client-urls: https://${HOST}:2379
initial-advertise-peer-urls: https://${HOST}:2380
EOF
done
```
-->
```sh
# 使用 IP 或可解析的主机名替换 HOST0、HOST1 和 HOST2
export HOST0=10.0.0.6
export HOST1=10.0.0.7
export HOST2=10.0.0.8
# 创建临时目录来存储将被分发到其它主机上的文件
mkdir -p /tmp/${HOST0}/ /tmp/${HOST1}/ /tmp/${HOST2}/
ETCDHOSTS=(${HOST0} ${HOST1} ${HOST2})
NAMES=("infra0" "infra1" "infra2")
for i in "${!ETCDHOSTS[@]}"; do
HOST=${ETCDHOSTS[$i]}
NAME=${NAMES[$i]}
cat << EOF > /tmp/${HOST}/kubeadmcfg.yaml
apiVersion: "kubeadm.k8s.io/v1beta2"
kind: ClusterConfiguration
etcd:
local:
serverCertSANs:
- "${HOST}"
peerCertSANs:
- "${HOST}"
extraArgs:
initial-cluster: infra0=https://${ETCDHOSTS[0]}:2380,infra1=https://${ETCDHOSTS[1]}:2380,infra2=https://${ETCDHOSTS[2]}:2380
initial-cluster-state: new
name: ${NAME}
listen-peer-urls: https://${HOST}:2380
listen-client-urls: https://${HOST}:2379
advertise-client-urls: https://${HOST}:2379
initial-advertise-peer-urls: https://${HOST}:2380
EOF
done
```
<!--
1.Generate the certificate authority
If you already have a CA then the only action that is copying the CA's `crt` and
`key` file to `/etc/kubernetes/pki/etcd/ca.crt` and
`/etc/kubernetes/pki/etcd/ca.key`. After those files have been copied,
proceed to the next step, "Create certificates for each member".
-->
1. 生成证书颁发机构
如果您已经拥有 CA,那么唯一的操作是复制 CA 的 `crt` 和 `key` 文件到 `etc/kubernetes/pki/etcd/ca.crt` 和 `/etc/kubernetes/pki/etcd/ca.key`。复制完这些文件后继续下一步,“为每个成员创建证书”。
<!--
If you do not already have a CA then run this command on `$HOST0` (where you generated the configuration files for kubeadm).
-->
如果您还没有 CA,则在 `$HOST0`(您为 kubeadm 生成配置文件的位置)上运行此命令。
```
kubeadm alpha phase certs etcd-ca
```
<!--
This creates two files
-->
创建了如下两个文件
- `/etc/kubernetes/pki/etcd/ca.crt`
- `/etc/kubernetes/pki/etcd/ca.key`
<!--
1. Create certificates for each member
-->
1. 为每个成员创建证书
<!--
```sh
kubeadm alpha phase certs etcd-server --config=/tmp/${HOST2}/kubeadmcfg.yaml
kubeadm alpha phase certs etcd-peer --config=/tmp/${HOST2}/kubeadmcfg.yaml
kubeadm alpha phase certs etcd-healthcheck-client --config=/tmp/${HOST2}/kubeadmcfg.yaml
kubeadm alpha phase certs apiserver-etcd-client --config=/tmp/${HOST2}/kubeadmcfg.yaml
cp -R /etc/kubernetes/pki /tmp/${HOST2}/
# cleanup non-reusable certificates
find /etc/kubernetes/pki -not -name ca.crt -not -name ca.key -type f -delete
kubeadm alpha phase certs etcd-server --config=/tmp/${HOST1}/kubeadmcfg.yaml
kubeadm alpha phase certs etcd-peer --config=/tmp/${HOST1}/kubeadmcfg.yaml
kubeadm alpha phase certs etcd-healthcheck-client --config=/tmp/${HOST1}/kubeadmcfg.yaml
kubeadm alpha phase certs apiserver-etcd-client --config=/tmp/${HOST1}/kubeadmcfg.yaml
cp -R /etc/kubernetes/pki /tmp/${HOST1}/
find /etc/kubernetes/pki -not -name ca.crt -not -name ca.key -type f -delete
kubeadm alpha phase certs etcd-server --config=/tmp/${HOST0}/kubeadmcfg.yaml
kubeadm alpha phase certs etcd-peer --config=/tmp/${HOST0}/kubeadmcfg.yaml
kubeadm alpha phase certs etcd-healthcheck-client --config=/tmp/${HOST0}/kubeadmcfg.yaml
kubeadm alpha phase certs apiserver-etcd-client --config=/tmp/${HOST0}/kubeadmcfg.yaml
# No need to move the certs because they are for HOST0
# clean up certs that should not be copied off this host
find /tmp/${HOST2} -name ca.key -type f -delete
find /tmp/${HOST1} -name ca.key -type f -delete
```
-->
```sh
kubeadm alpha phase certs etcd-server --config=/tmp/${HOST2}/kubeadmcfg.yaml
kubeadm alpha phase certs etcd-peer --config=/tmp/${HOST2}/kubeadmcfg.yaml
kubeadm alpha phase certs etcd-healthcheck-client --config=/tmp/${HOST2}/kubeadmcfg.yaml
kubeadm alpha phase certs apiserver-etcd-client --config=/tmp/${HOST2}/kubeadmcfg.yaml
cp -R /etc/kubernetes/pki /tmp/${HOST2}/
# 清理不可重复使用的证书
find /etc/kubernetes/pki -not -name ca.crt -not -name ca.key -type f -delete
kubeadm alpha phase certs etcd-server --config=/tmp/${HOST1}/kubeadmcfg.yaml
kubeadm alpha phase certs etcd-peer --config=/tmp/${HOST1}/kubeadmcfg.yaml
kubeadm alpha phase certs etcd-healthcheck-client --config=/tmp/${HOST1}/kubeadmcfg.yaml
kubeadm alpha phase certs apiserver-etcd-client --config=/tmp/${HOST1}/kubeadmcfg.yaml
cp -R /etc/kubernetes/pki /tmp/${HOST1}/
find /etc/kubernetes/pki -not -name ca.crt -not -name ca.key -type f -delete
kubeadm alpha phase certs etcd-server --config=/tmp/${HOST0}/kubeadmcfg.yaml
kubeadm alpha phase certs etcd-peer --config=/tmp/${HOST0}/kubeadmcfg.yaml
kubeadm alpha phase certs etcd-healthcheck-client --config=/tmp/${HOST0}/kubeadmcfg.yaml
kubeadm alpha phase certs apiserver-etcd-client --config=/tmp/${HOST0}/kubeadmcfg.yaml
# 不需要移动 certs 因为它们是给 HOST0 使用的
# 清理不应从此主机复制的证书
find /tmp/${HOST2} -name ca.key -type f -delete
find /tmp/${HOST1} -name ca.key -type f -delete
```
<!--
1.Copy certificates and kubeadm configs
The certificates have been generated and now they must be moved to their
respective hosts.
-->
1. 复制证书和 kubeadm 配置
证书已生成,现在必须将它们移动到对应的主机。
```sh
USER=ubuntu
HOST=${HOST1}
scp -r /tmp/${HOST}/* ${USER}@${HOST}:
ssh ${USER}@${HOST}
USER@HOST $ sudo -Es
root@HOST $ chown -R root:root pki
root@HOST $ mv pki /etc/kubernetes/
```
<!--
1.Ensure all expected files exist
The complete list of required files on `$HOST0` is:
-->
1. 确保已经所有预期的文件都存在
`$HOST0` 所需文件的完整列表如下:
```
/tmp/${HOST0}
└── kubeadmcfg.yaml
---
/etc/kubernetes/pki
├── apiserver-etcd-client.crt
├── apiserver-etcd-client.key
└── etcd
├── ca.crt
├── ca.key
├── healthcheck-client.crt
├── healthcheck-client.key
├── peer.crt
├── peer.key
├── server.crt
└── server.key
```
<!--
On `$HOST1`:
-->
在 `$HOST1`:
```
$HOME
└── kubeadmcfg.yaml
---
/etc/kubernetes/pki
├── apiserver-etcd-client.crt
├── apiserver-etcd-client.key
└── etcd
├── ca.crt
├── healthcheck-client.crt
├── healthcheck-client.key
├── peer.crt
├── peer.key
├── server.crt
└── server.key
```
<!--
On `$HOST2`
-->
在 `$HOST2`
```
$HOME
└── kubeadmcfg.yaml
---
/etc/kubernetes/pki
├── apiserver-etcd-client.crt
├── apiserver-etcd-client.key
└── etcd
├── ca.crt
├── healthcheck-client.crt
├── healthcheck-client.key
├── peer.crt
├── peer.key
├── server.crt
└── server.key
```
<!--
1.Create the static pod manifests
Now that the certificates and configs are in place it's time to create the
manifests. On each host run the `kubeadm` command to generate a static manifest
for etcd.
-->
1. 创建静态 Pod 清单
既然证书和配置已经就绪,是时候去创建清单了。在每台主机上运行 `kubeadm` 命令来生成 etcd 使用的静态清单。
```sh
root@HOST0 $ kubeadm alpha phase etcd local --config=/tmp/${HOST0}/kubeadmcfg.yaml
root@HOST1 $ kubeadm alpha phase etcd local --config=/home/ubuntu/kubeadmcfg.yaml
root@HOST2 $ kubeadm alpha phase etcd local --config=/home/ubuntu/kubeadmcfg.yaml
```
<!--
1.Optional: Check the cluster health
-->
1. 可选:检查群集运行状况
```sh
docker run --rm -it \
--net host \
-v /etc/kubernetes:/etc/kubernetes quay.io/coreos/etcd:v3.2.18 etcdctl \
--cert-file /etc/kubernetes/pki/etcd/peer.crt \
--key-file /etc/kubernetes/pki/etcd/peer.key \
--ca-file /etc/kubernetes/pki/etcd/ca.crt \
--endpoints https://${HOST0}:2379 cluster-health
...
cluster is healthy
```
{{% /capture %}}
{{% capture whatsnext %}}
<!--
Once your have a working 3 member etcd cluster, you can continue setting up a
highly available control plane using the [external etcd method with
kubeadm](/docs/setup/independent/high-availability/).
-->
一旦拥有了一个正常工作的 3 成员的 etcd 集群,你就可以基于[使用 kubeadm 的外部 etcd 方法](/docs/setup/independent/high-availability/),继续部署一个高可用的控制平面。
{{% /capture %}}
@@ -0,0 +1,650 @@
---
title: 对 kubeadm 进行故障排查
content_template: templates/concept
weight: 20
---
<!--
---
title: Troubleshooting kubeadm
content_template: templates/concept
weight: 20
---
-->
{{% capture overview %}}
<!--
As with any program, you might run into an error installing or running kubeadm.
This page lists some common failure scenarios and have provided steps that can help you understand and fix the problem.
If your problem is not listed below, please follow the following steps:
- If you think your problem is a bug with kubeadm:
- Go to [github.com/kubernetes/kubeadm](https://github.com/kubernetes/kubeadm/issues) and search for existing issues.
- If no issue exists, please [open one](https://github.com/kubernetes/kubeadm/issues/new) and follow the issue template.
- If you are unsure about how kubeadm works, you can ask on [Slack](http://slack.k8s.io/) in #kubeadm, or open a question on [StackOverflow](https://stackoverflow.com/questions/tagged/kubernetes). Please include
relevant tags like `#kubernetes` and `#kubeadm` so folks can help you.
-->
与任何程序一样,您可能会在安装或者运行 kubeadm 时遇到错误。
本文列举了一些常见的故障场景,并提供可帮助您理解和解决这些问题的步骤。
如果您的问题未在下面列出,请执行以下步骤:
- 如果您认为问题是 kubeadm 的错误:
- 转到 [github.com/kubernetes/kubeadm](https://github.com/kubernetes/kubeadm/issues) 并搜索存在的问题。
- 如果没有问题,请 [打开](https://github.com/kubernetes/kubeadm/issues/new) 并遵循问题模板。
- 如果您对 kubeadm 的工作方式有疑问,可以在 [Slack](http://slack.k8s.io/) 上的 #kubeadm 频道提问,
或者在 [StackOverflow](https://stackoverflow.com/questions/tagged/kubernetes) 上提问。
请加入相关标签,例如 `#kubernetes``#kubeadm`,这样其他人可以帮助您。
{{% /capture %}}
{{% capture body %}}
<!--
## `ebtables` or some similar executable not found during installation
If you see the following warnings while running `kubeadm init`
```sh
[preflight] WARNING: ebtables not found in system path
[preflight] WARNING: ethtool not found in system path
```
Then you may be missing `ebtables`, `ethtool` or a similar executable on your node. You can install them with the following commands:
- For Ubuntu/Debian users, run `apt install ebtables ethtool`.
- For CentOS/Fedora users, run `yum install ebtables ethtool`.
-->
## 在安装过程中没有找到 `ebtables` 或者其他类似的可执行文件
如果在运行 `kubeadm init` 命令时,遇到以下的警告
```sh
[preflight] WARNING: ebtables not found in system path
[preflight] WARNING: ethtool not found in system path
```
那么或许在您的节点上缺失 `ebtables``ethtool` 或者类似的可执行文件。
您可以使用以下命令安装它们:
- 对于 Ubuntu/Debian 用户,运行 `apt install ebtables ethtool` 命令。
- 对于 CentOS/Fedora 用户,运行 `yum install ebtables ethtool` 命令。
<!--
## kubeadm blocks waiting for control plane during installation
If you notice that `kubeadm init` hangs after printing out the following line:
```sh
[apiclient] Created API client, waiting for the control plane to become ready
```
-->
## 在安装过程中,kubeadm 一直等待控制平面就绪
如果您注意到 `kubeadm init` 在打印以下行后挂起:
```sh
[apiclient] Created API client, waiting for the control plane to become ready
```
<!--
This may be caused by a number of problems. The most common are:
- network connection problems. Check that your machine has full network connectivity before continuing.
- the default cgroup driver configuration for the kubelet differs from that used by Docker.
Check the system log file (e.g. `/var/log/message`) or examine the output from `journalctl -u kubelet`. If you see something like the following:
```shell
error: failed to run Kubelet: failed to create kubelet:
misconfiguration: kubelet cgroup driver: "systemd" is different from docker cgroup driver: "cgroupfs"
```
There are two common ways to fix the cgroup driver problem:
1. Install Docker again following instructions
[here](/docs/setup/production-environment/container-runtimes/#docker).
1. Change the kubelet config to match the Docker cgroup driver manually, you can refer to
[Configure cgroup driver used by kubelet on Master Node](/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#configure-cgroup-driver-used-by-kubelet-on-master-node)
- control plane Docker containers are crashlooping or hanging. You can check this by running `docker ps` and investigating each container by running `docker logs`.
-->
这可能是由许多问题引起的。最常见的是:
- 网络连接问题。在继续之前,请检查您的计算机是否具有全部联通的网络连接。
- kubelet 的默认 cgroup 驱动程序配置不同于 Docker 使用的配置。
检查系统日志文件 (例如 `/var/log/message`) 或检查 `journalctl -u kubelet` 的输出。 如果您看见以下内容:
```shell
error: failed to run Kubelet: failed to create kubelet:
misconfiguration: kubelet cgroup driver: "systemd" is different from docker cgroup driver: "cgroupfs"
```
有两种常见方法可解决 cgroup 驱动程序问题:
1. 按照 [此处](/docs/setup/production-environment/container-runtimes/#docker) 的说明再次安装 Docker。
1. 更改 kubelet 配置以手动匹配 Docker cgroup 驱动程序,您可以参考
[在主节点上配置 kubelet 要使用的 cgroup 驱动程序](/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#configure-cgroup-driver-used-by-kubelet-on-master-node)
- 控制平面上的 Docker 容器持续进入崩溃状态或(因其他原因)挂起。您可以运行 `docker ps` 命令来检查以及 `docker logs` 命令来检视每个容器的运行日志。
<!--
## kubeadm blocks when removing managed containers
The following could happen if Docker halts and does not remove any Kubernetes-managed containers:
```bash
sudo kubeadm reset
[preflight] Running pre-flight checks
[reset] Stopping the kubelet service
[reset] Unmounting mounted directories in "/var/lib/kubelet"
[reset] Removing kubernetes-managed containers
(block)
```
A possible solution is to restart the Docker service and then re-run `kubeadm reset`:
```bash
sudo systemctl restart docker.service
sudo kubeadm reset
```
Inspecting the logs for docker may also be useful:
```sh
journalctl -ul docker
```
-->
## 当删除托管容器时 kubeadm 阻塞
如果 Docker 停止并且不删除 Kubernetes 所管理的所有容器,可能发生以下情况:
```bash
sudo kubeadm reset
[preflight] Running pre-flight checks
[reset] Stopping the kubelet service
[reset] Unmounting mounted directories in "/var/lib/kubelet"
[reset] Removing kubernetes-managed containers
(block)
```
一个可行的解决方案是重新启动 Docker 服务,然后重新运行 `kubeadm reset`
```bash
sudo systemctl restart docker.service
sudo kubeadm reset
```
检查 docker 的日志也可能有用:
```sh
journalctl -ul docker
```
<!--
## Pods in `RunContainerError`, `CrashLoopBackOff` or `Error` state
Right after `kubeadm init` there should not be any pods in these states.
- If there are pods in one of these states _right after_ `kubeadm init`, please open an
issue in the kubeadm repo. `coredns` (or `kube-dns`) should be in the `Pending` state
until you have deployed the network solution.
- If you see Pods in the `RunContainerError`, `CrashLoopBackOff` or `Error` state
after deploying the network solution and nothing happens to `coredns` (or `kube-dns`),
it's very likely that the Pod Network solution that you installed is somehow broken.
You might have to grant it more RBAC privileges or use a newer version. Please file
an issue in the Pod Network providers' issue tracker and get the issue triaged there.
- If you install a version of Docker older than 1.12.1, remove the `MountFlags=slave` option
when booting `dockerd` with `systemd` and restart `docker`. You can see the MountFlags in `/usr/lib/systemd/system/docker.service`.
MountFlags can interfere with volumes mounted by Kubernetes, and put the Pods in `CrashLoopBackOff` state.
The error happens when Kubernetes does not find `var/run/secrets/kubernetes.io/serviceaccount` files.
-->
## Pods 处于 `RunContainerError`、`CrashLoopBackOff` 或者 `Error` 状态
在 `kubeadm init` 命令运行后,系统中不应该有 pods 处于这类状态。
- 在 `kubeadm init` 命令执行完后,如果有 pods 处于这些状态之一,请在 kubeadm
仓库提起一个 issue。`coredns` (或者 `kube-dns`) 应该处于 `Pending` 状态,
直到您部署了网络解决方案为止。
- 如果在部署完网络解决方案之后,有 Pods 处于 `RunContainerError`、`CrashLoopBackOff`
或 `Error` 状态之一,并且`coredns` (或者 `kube-dns`)仍处于 `Pending` 状态,
那很可能是您安装的网络解决方案由于某种原因无法工作。您或许需要授予它更多的
RBAC 特权或使用较新的版本。请在 Pod Network 提供商的问题跟踪器中提交问题,
然后在此处分类问题。
- 如果您安装的 Docker 版本早于 1.12.1,请在使用 `systemd` 来启动 `dockerd` 和重启 `docker` 时,
删除 `MountFlags=slave` 选项。
您可以在 `/usr/lib/systemd/system/docker.service` 中看到 MountFlags。
MountFlags 可能会干扰 Kubernetes 挂载的卷, 并使 Pods 处于 `CrashLoopBackOff` 状态。
当 Kubernetes 不能找到 `var/run/secrets/kubernetes.io/serviceaccount` 文件时会发生错误。
<!--
## `coredns` (or `kube-dns`) is stuck in the `Pending` state
This is **expected** and part of the design. kubeadm is network provider-agnostic, so the admin
should [install the pod network solution](/docs/concepts/cluster-administration/addons/)
of choice. You have to install a Pod Network
before CoreDNS may be deployed fully. Hence the `Pending` state before the network is set up.
-->
## `coredns` (或 `kube-dns`)停滞在 `Pending` 状态
这一行为是 **预期之中** 的,因为系统就是这么设计的。
kubeadm 的网络供应商是中立的,因此管理员应该选择 [安装 pod 的网络解决方案](/docs/concepts/cluster-administration/addons/)。
您必须完成 Pod 的网络配置,然后才能完全部署 CoreDNS。
在网络被配置好之前,DNS 组件会一直处于 `Pending` 状态。
<!--
## `HostPort` services do not work
The `HostPort` and `HostIP` functionality is available depending on your Pod Network
provider. Please contact the author of the Pod Network solution to find out whether
`HostPort` and `HostIP` functionality are available.
Calico, Canal, and Flannel CNI providers are verified to support HostPort.
For more information, see the [CNI portmap documentation](https://github.com/containernetworking/plugins/blob/master/plugins/meta/portmap/README.md).
If your network provider does not support the portmap CNI plugin, you may need to use the [NodePort feature of
services](/docs/concepts/services-networking/service/#nodeport) or use `HostNetwork=true`.
-->
## `HostPort` 服务无法工作
此 `HostPort` 和 `HostIP` 功能是否可用取决于您的 Pod 网络配置。请联系 Pod 解决方案的作者,
以确认 `HostPort` 和 `HostIP` 功能是否可用。
已验证 Calico、Canal 和 Flannel CNI 驱动程序支持 HostPort。
有关更多信息,请参考 [CNI portmap 文档](https://github.com/containernetworking/plugins/blob/master/plugins/meta/portmap/README.md).
如果您的网络提供商不支持 portmap CNI 插件,您或许需要使用 [NodePort 服务的功能](/docs/concepts/services-networking/service/#nodeport) 或者使用 `HostNetwork=true`。
<!--
## Pods are not accessible via their Service IP
- Many network add-ons do not yet enable [hairpin mode](/docs/tasks/debug-application-cluster/debug-service/#a-pod-cannot-reach-itself-via-service-ip)
which allows pods to access themselves via their Service IP. This is an issue related to
[CNI](https://github.com/containernetworking/cni/issues/476). Please contact the network
add-on provider to get the latest status of their support for hairpin mode.
- If you are using VirtualBox (directly or via Vagrant), you will need to
ensure that `hostname -i` returns a routable IP address. By default the first
interface is connected to a non-routable host-only network. A work around
is to modify `/etc/hosts`, see this [Vagrantfile](https://github.com/errordeveloper/k8s-playground/blob/22dd39dfc06111235620e6c4404a96ae146f26fd/Vagrantfile#L11)
for an example.
-->
## 无法通过其服务 IP 访问 Pod
- 许多网络附加组件尚未启用 [hairpin 模式](/docs/tasks/debug-application-cluster/debug-service/#a-pod-cannot-reach-itself-via-service-ip)
该模式允许 Pod 通过其服务 IP 进行访问。这是与 [CNI](https://github.com/containernetworking/cni/issues/476) 有关的问题。请与网络附加组件提供商联系,以获取他们所提供的 hairpin 模式的最新状态。
- 如果您正在使用 VirtualBox (直接使用或者通过 Vagrant 使用),您需要
确保 `hostname -i` 返回一个可路由的 IP 地址。默认情况下,第一个接口连接不能路由的仅主机网络。
解决方法是修改 `/etc/hosts`,请参考示例 [Vagrantfile](https://github.com/errordeveloper/k8s-playground/blob/22dd39dfc06111235620e6c4404a96ae146f26fd/Vagrantfile#L11)。
<!--
## TLS certificate errors
The following error indicates a possible certificate mismatch.
```none
# kubectl get pods
Unable to connect to the server: x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubernetes")
```
- Verify that the `$HOME/.kube/config` file contains a valid certificate, and
regenerate a certificate if necessary. The certificates in a kubeconfig file
are base64 encoded. The `base64 -d` command can be used to decode the certificate
and `openssl x509 -text -noout` can be used for viewing the certificate information.
- Unset the `KUBECONFIG` environment variable using:
```sh
unset KUBECONFIG
```
Or set it to the default `KUBECONFIG` location:
```sh
export KUBECONFIG=/etc/kubernetes/admin.conf
```
- Another workaround is to overwrite the existing `kubeconfig` for the "admin" user:
```sh
mv $HOME/.kube $HOME/.kube.bak
mkdir $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
```
-->
## TLS 证书错误
以下错误指出证书可能不匹配。
```none
# kubectl get pods
Unable to connect to the server: x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubernetes")
```
- 验证 `$HOME/.kube/config` 文件是否包含有效证书,并
在必要时重新生成证书。在 kubeconfig 文件中的证书是 base64 编码的。
该 `base64 -d` 命令可以用来解码证书,`openssl x509 -text -noout` 命令
可以用于查看证书信息。
- 使用如下方法取消设置 `KUBECONFIG` 环境变量的值:
```sh
unset KUBECONFIG
```
或者将其设置为默认的 `KUBECONFIG` 位置:
```sh
export KUBECONFIG=/etc/kubernetes/admin.conf
```
- 另一个方法是覆盖 `kubeconfig` 的现有用户 "管理员"
```sh
mv $HOME/.kube $HOME/.kube.bak
mkdir $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
```
<!--
## Default NIC When using flannel as the pod network in Vagrant
The following error might indicate that something was wrong in the pod network:
```sh
Error from server (NotFound): the server could not find the requested resource
```
- If you're using flannel as the pod network inside Vagrant, then you will have to specify the default interface name for flannel.
Vagrant typically assigns two interfaces to all VMs. The first, for which all hosts are assigned the IP address `10.0.2.15`, is for external traffic that gets NATed.
This may lead to problems with flannel, which defaults to the first interface on a host. This leads to all hosts thinking they have the same public IP address. To prevent this, pass the `--iface eth1` flag to flannel so that the second interface is chosen.
-->
## 在 Vagrant 中使用 flannel 作为 pod 网络时的默认 NIC
以下错误可能表明 Pod 网络中出现问题:
```sh
Error from server (NotFound): the server could not find the requested resource
```
- 如果你正在 Vagrant 中使用 flannel 作为 pod 网络,则必须指定 flannel 的默认接口名称。
Vagrant 通常为所有 VM 分配两个接口。第一个为所有主机分配了 IP 地址 `10.0.2.15`,用于获得 NATed 的外部流量。
这可能会导致 flannel 出现问题,它默认为主机上的第一个接口。这导致所有主机认为它们具有
相同的公共 IP 地址。为防止这种情况,传递 `--iface eth1` 标志给 flannel 以便选择第二个接口。
<!--
## Non-public IP used for containers
In some situations `kubectl logs` and `kubectl run` commands may return with the following errors in an otherwise functional cluster:
```sh
Error from server: Get https://10.19.0.41:10250/containerLogs/default/mysql-ddc65b868-glc5m/mysql: dial tcp 10.19.0.41:10250: getsockopt: no route to host
```
- This may be due to Kubernetes using an IP that can not communicate with other IPs on the seemingly same subnet, possibly by policy of the machine provider.
- Digital Ocean assigns a public IP to `eth0` as well as a private one to be used internally as anchor for their floating IP feature, yet `kubelet` will pick the latter as the node's `InternalIP` instead of the public one.
Use `ip addr show` to check for this scenario instead of `ifconfig` because `ifconfig` will not display the offending alias IP address. Alternatively an API endpoint specific to Digital Ocean allows to query for the anchor IP from the droplet:
```sh
curl http://169.254.169.254/metadata/v1/interfaces/public/0/anchor_ipv4/address
```
The workaround is to tell `kubelet` which IP to use using `--node-ip`. When using Digital Ocean, it can be the public one (assigned to `eth0`) or the private one (assigned to `eth1`) should you want to use the optional private network. The [`KubeletExtraArgs` section of the kubeadm `NodeRegistrationOptions` structure](https://github.com/kubernetes/kubernetes/blob/release-1.13/cmd/kubeadm/app/apis/kubeadm/v1beta1/types.go) can be used for this.
Then restart `kubelet`:
```sh
systemctl daemon-reload
systemctl restart kubelet
```
-->
## 容器使用的非公共 IP
在某些情况下 `kubectl logs` 和 `kubectl run` 命令或许会返回以下错误,即便除此之外集群一切功能正常:
```sh
Error from server: Get https://10.19.0.41:10250/containerLogs/default/mysql-ddc65b868-glc5m/mysql: dial tcp 10.19.0.41:10250: getsockopt: no route to host
```
- 这或许是由于 Kubernetes 使用的 IP 无法与看似相同的子网上的其他 IP 进行通信的缘故,
可能是由机器提供商的政策所导致的。
- Digital Ocean 既分配一个共有 IP 给 `eth0`,也分配一个私有 IP 在内部用作其浮动 IP 功能的锚点,
然而 `kubelet` 将选择后者作为节点的 `InternalIP` 而不是公共 IP
使用 `ip addr show` 命令代替 `ifconfig` 命令去检查这种情况,因为 `ifconfig` 命令
不会显示有问题的别名 IP 地址。或者指定的 Digital Ocean 的 API 端口允许从 droplet 中
查询 anchor IP
```sh
curl http://169.254.169.254/metadata/v1/interfaces/public/0/anchor_ipv4/address
```
解决方法是通知 `kubelet` 使用哪个 `--node-ip`。当使用 Digital Ocean 时,可以是公网IP(分配给 `eth0`的),
或者是私网IP(分配给 `eth1` 的)。私网 IP 是可选的。
这个 [`KubeletExtraArgs` section of the kubeadm `NodeRegistrationOptions` structure](https://github.com/kubernetes/kubernetes/blob/release-1.13/cmd/kubeadm/app/apis/kubeadm/v1beta1/types.go) 被用来处理这种情况。
然后重启 `kubelet`
```sh
systemctl daemon-reload
systemctl restart kubelet
```
<!--
## `coredns` pods have `CrashLoopBackOff` or `Error` state
If you have nodes that are running SELinux with an older version of Docker you might experience a scenario
where the `coredns` pods are not starting. To solve that you can try one of the following options:
- Upgrade to a [newer version of Docker](/docs/setup/production-environment/container-runtimes/#docker).
- [Disable SELinux](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/security-enhanced_linux/sect-security-enhanced_linux-enabling_and_disabling_selinux-disabling_selinux).
- Modify the `coredns` deployment to set `allowPrivilegeEscalation` to `true`:
```bash
kubectl -n kube-system get deployment coredns -o yaml | \
sed 's/allowPrivilegeEscalation: false/allowPrivilegeEscalation: true/g' | \
kubectl apply -f -
```
Another cause for CoreDNS to have `CrashLoopBackOff` is when a CoreDNS Pod deployed in Kubernetes detects a loop. [A number of workarounds](https://github.com/coredns/coredns/tree/master/plugin/loop#troubleshooting-loops-in-kubernetes-clusters)
are available to avoid Kubernetes trying to restart the CoreDNS Pod every time CoreDNS detects the loop and exits.
-->
## `coredns` pods 有 `CrashLoopBackOff` 或者 `Error` 状态
如果有些节点运行的是旧版本的 Docker,同时启用了 SELinux,您或许会遇到 `coredns` pods 无法启动的情况。
要解决此问题,您可以尝试以下选项之一:
- 升级到 [Docker 的较新版本](/docs/setup/production-environment/container-runtimes/#docker)。
- [禁用 SELinux](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/security-enhanced_linux/sect-security-enhanced_linux-enabling_and_disabling_selinux-disabling_selinux).
- 修改 `coredns` 部署以设置 `allowPrivilegeEscalation` 为 `true`
```bash
kubectl -n kube-system get deployment coredns -o yaml | \
sed 's/allowPrivilegeEscalation: false/allowPrivilegeEscalation: true/g' | \
kubectl apply -f -
```
CoreDNS 处于 `CrashLoopBackOff` 时的另一个原因是当 Kubernetes 中部署的 CoreDNS Pod 检测
到环路时。[有许多解决方法](https://github.com/coredns/coredns/tree/master/plugin/loop#troubleshooting-loops-in-kubernetes-clusters)
可以避免在每次 CoreDNS 监测到循环并退出时,Kubernetes 尝试重启 CoreDNS Pod 的情况。
{{< warning >}}
<!--
Disabling SELinux or setting `allowPrivilegeEscalation` to `true` can compromise
the security of your cluster.
-->
**警告**:禁用 SELinux 或设置 `allowPrivilegeEscalation` 为 `true` 可能会损害集群的安全性。
{{< /warning >}}
<!--
## etcd pods restart continually
If you encounter the following error:
```
rpc error: code = 2 desc = oci runtime error: exec failed: container_linux.go:247: starting container process caused "process_linux.go:110: decoding init error from pipe caused \"read parent: connection reset by peer\""
```
this issue appears if you run CentOS 7 with Docker 1.13.1.84.
This version of Docker can prevent the kubelet from executing into the etcd container.
To work around the issue, choose one of these options:
- Roll back to an earlier version of Docker, such as 1.13.1-75
```
yum downgrade docker-1.13.1-75.git8633870.el7.centos.x86_64 docker-client-1.13.1-75.git8633870.el7.centos.x86_64 docker-common-1.13.1-75.git8633870.el7.centos.x86_64
```
- Install one of the more recent recommended versions, such as 18.06:
```bash
sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
yum install docker-ce-18.06.1.ce-3.el7.x86_64
```
-->
## etcd pods 持续重启
如果您遇到以下错误:
```
rpc error: code = 2 desc = oci runtime error: exec failed: container_linux.go:247: starting container process caused "process_linux.go:110: decoding init error from pipe caused \"read parent: connection reset by peer\""
```
如果您使用 Docker 1.13.1.84 运行 CentOS 7 就会出现这种问题。
此版本的 Docker 会阻止 kubelet 在 etcd 容器中执行。
为解决此问题,请选择以下选项之一:
- 回滚到早期版本的 Docker,例如 1.13.1-75
```
yum downgrade docker-1.13.1-75.git8633870.el7.centos.x86_64 docker-client-1.13.1-75.git8633870.el7.centos.x86_64 docker-common-1.13.1-75.git8633870.el7.centos.x86_64
```
- 安装较新的推荐版本之一,例如 18.06:
```bash
sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
yum install docker-ce-18.06.1.ce-3.el7.x86_64
```
<!--
## Not possible to pass a comma separated list of values to arguments inside a `--component-extra-args` flag
`kubeadm init` flags such as `--component-extra-args` allow you to pass custom arguments to a control-plane
component like the kube-apiserver. However, this mechanism is limited due to the underlying type used for parsing
the values (`mapStringString`).
If you decide to pass an argument that supports multiple, comma-separated values such as
`--apiserver-extra-args "enable-admission-plugins=LimitRanger,NamespaceExists"` this flag will fail with
`flag: malformed pair, expect string=string`. This happens because the list of arguments for
`--apiserver-extra-args` expects `key=value` pairs and in this case `NamespacesExists` is considered
as a key that is missing a value.
Alternatively, you can try separating the `key=value` pairs like so:
`--apiserver-extra-args "enable-admission-plugins=LimitRanger,enable-admission-plugins=NamespaceExists"`
but this will result in the key `enable-admission-plugins` only having the value of `NamespaceExists`.
A known workaround is to use the kubeadm [configuration file](/docs/setup/production-environment/tools/kubeadm/control-plane-flags/#apiserver-flags).
-->
## 无法将以逗号分隔的值列表传递给 `--component-extra-args` 标志内的参数
`kubeadm init` 标志例如 `--component-extra-args` 允许您将自定义参数传递给像 kube-apiserver 这样的控制平面组件。然而,由于解析 (`mapStringString`) 的基础类型值,此机制将受到限制。
如果您决定传递一个支持多个逗号分隔值(例如 `--apiserver-extra-args "enable-admission-plugins=LimitRanger,NamespaceExists"`)参数,将出现 `flag: malformed pair, expect string=string` 错误。
发生这种问题是因为参数列表 `--apiserver-extra-args` 预期的是 `key=value` 形式,而这里的 `NamespacesExists` 被误认为是缺少取值的键名。
一种解决方法是尝试分离 `key=value` 对,像这样:
`--apiserver-extra-args "enable-admission-plugins=LimitRanger,enable-admission-plugins=NamespaceExists"`
但这将导致键 `enable-admission-plugins` 仅有值 `NamespaceExists`。
已知的解决方法是使用 kubeadm [配置文件](/docs/setup/production-environment/tools/kubeadm/control-plane-flags/#apiserver-flags)。
<!--
## kube-proxy scheduled before node is initialized by cloud-controller-manager
In cloud provider scenarios, kube-proxy can end up being scheduled on new worker nodes before
the cloud-controller-manager has initialized the node addresses. This causes kube-proxy to fail
to pick up the node's IP address properly and has knock-on effects to the proxy function managing
load balancers.
The following error can be seen in kube-proxy Pods:
```
server.go:610] Failed to retrieve node IP: host IP unknown; known addresses: []
proxier.go:340] invalid nodeIP, initializing kube-proxy with 127.0.0.1 as nodeIP
```
A known solution is to patch the kube-proxy DaemonSet to allow scheduling it on control-plane
nodes regardless of their conditions, keeping it off of other nodes until their initial guarding
conditions abate:
```
kubectl -n kube-system patch ds kube-proxy -p='{ "spec": { "template": { "spec": { "tolerations": [ { "key": "CriticalAddonsOnly", "operator": "Exists" }, { "effect": "NoSchedule", "key": "node-role.kubernetes.io/master" } ] } } } }'
```
The tracking issue for this problem is [here](https://github.com/kubernetes/kubeadm/issues/1027).
-->
## 在节点被云控制管理器初始化之前,kube-proxy 就被调度了
在云环境场景中,可能出现在云控制管理器完成节点地址初始化之前,kube-proxy 就被调度到新节点了。
这会导致 kube-proxy 无法正确获取节点的 IP 地址,并对管理负载平衡器的代理功能产生连锁反应。
在 kube-proxy Pod 中可以看到以下错误:
```
server.go:610] Failed to retrieve node IP: host IP unknown; known addresses: []
proxier.go:340] invalid nodeIP, initializing kube-proxy with 127.0.0.1 as nodeIP
```
一种已知的解决方案是修补 kube-proxy DaemonSet,以允许在控制平面节点上调度它,而不管它们的条件如何,将其与其他节点保持隔离,直到它们的初始保护条件消除:
```
kubectl -n kube-system patch ds kube-proxy -p='{ "spec": { "template": { "spec": { "tolerations": [ { "key": "CriticalAddonsOnly", "operator": "Exists" }, { "effect": "NoSchedule", "key": "node-role.kubernetes.io/master" } ] } } } }'
```
此问题的跟踪 [在这里](https://github.com/kubernetes/kubeadm/issues/1027)。
<!--
## The NodeRegistration.Taints field is omitted when marshalling kubeadm configuration
*Note: This [issue](https://github.com/kubernetes/kubeadm/issues/1358) only applies to tools that marshal kubeadm types (e.g. to a YAML configuration file). It will be fixed in kubeadm API v1beta2.*
By default, kubeadm applies the `role.kubernetes.io/master:NoSchedule` taint to control-plane nodes.
If you prefer kubeadm to not taint the control-plane node, and set `InitConfiguration.NodeRegistration.Taints` to an empty slice,
the field will be omitted when marshalling. When the field is omitted, kubeadm applies the default taint.
There are at least two workarounds:
1. Use the `role.kubernetes.io/master:PreferNoSchedule` taint instead of an empty slice. [Pods will get scheduled on masters](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/), unless other nodes have capacity.
2. Remove the taint after kubeadm init exits:
```bash
kubectl taint nodes NODE_NAME role.kubernetes.io/master:NoSchedule-
```
-->
## NodeRegistration.Taints 字段在编组 kubeadm 配置时丢失
*注意:这个 [问题](https://github.com/kubernetes/kubeadm/issues/1358) 仅适用于操控 kubeadm 数据类型的工具(例如,YAML 配置文件)。它将在 kubeadm API v1beta2 修复。*
默认情况下,kubeadm 将 `role.kubernetes.io/master:NoSchedule` 污点应用于控制平面节点。
如果您希望 kubeadm 不污染控制平面节点,并将 `InitConfiguration.NodeRegistration.Taints` 设置成空切片,则应在编组时省略该字段。
如果省略该字段,则 kubeadm 将应用默认污点。
至少有两种解决方法:
1. 使用 `role.kubernetes.io/master:PreferNoSchedule` 污点代替空切片。
除非其他节点具有容量,[否则将在主节点上调度 Pods](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/)。
2. 在 kubeadm init 退出后删除污点:
```bash
kubectl taint nodes NODE_NAME role.kubernetes.io/master:NoSchedule-
```
{{% /capture %}}