Release 1.8 (#5659)
* GC now supports non-core resources * Add two examples about how to analysis audits of kube-apiserver (#4264) * Deprecate system:nodes binding * [1.8] StatefulSet `initialized` annotation is now ignored. * inits the kubeadm upgrade docs addresses kubernetes/kubernetes.github.io/issues/4689 * adds kubeadm upgrade cmd to ToC addresses kubernetes/kubernetes.github.io/issues/4689 * add workload placement docs * ScaleIO - document udpate for 1.8 * Add documentation on storageClass.mountOptions and PV.mountOptions (#5254) * Add documentation on storageClass.mountOptions and PV.mountOptions * convert notes into callouts * Add docs for CustomResource validation add info about supported fields * advanced audit beta features (#5300) * Update job workload doc with backoff failure policy (#5319) Add to the Jobs documentation how to use the new backoffLimit field that limit the number of Pod failure before considering the Job as failed. * Documented additional AWS Service annotations (#4864) * Add device plugin doc under concepts/cluster-administration. (#5261) * Add device plugin doc under concepts/cluster-administration. * Update device-plugins.md * Update device-plugins.md Add meta description. Fix typo. Change bare metal deployment to manual deployment. * Update device-plugins.md Fix typo again. * Update page.version. (#5341) * Add documentation on storageClass.reclaimPolicy (#5171) * [Advanced audit] use new herf for audit-api (#5349) This tag contains all the changes in v1beta1 version. Update it now. * Added documentation around creating the InitializerConfiguration for the persistent volume label controller in the cloud-controller-manager (#5255) * Documentation for kubectl plugins (#5294) * Documentation for kubectl plugins * Update kubectl-plugins.md * Update kubectl-plugins.md * Updated CPU manager docs to match implementation. (#5332) * Noted limitation of alpha static cpumanager. * Updated CPU manager docs to match implementation. - Removed references to CPU pressure node condition and evictions. - Added note about new --cpu-manager-reconcile-period flag. - Added note about node allocatable requirements for static policy. - Noted limitation of alpha static cpumanager. * Move cpu-manager task link to rsc mgmt section. * init containers annotation removed in 1.8 (#5390) * Add documentation for TaintNodesByCondition (#5352) * Add documentation for TaintNodesByCondition * Update nodes.md * Update taint-and-toleration.md * Update daemonset.md * Update nodes.md * Update taint-and-toleration.md * Update daemonset.md * Fix deployments (#5421) * Document extended resources and OIR deprecation. (#5399) * Document extended resources and OIR deprecation. * Updated extended resources doc per reviews. * reverts extra spacing in _data/tasks.yml * addresses `kubeadm upgrade` review comments Feedback from @chenopis, @luxas, and @steveperry-53 addressed with this commit * HugePages documentation (#5419) * Update cpu-management-policies.md (#5407) Fixed the bad link. Modified "cpu" to "CPU". Added more 'yaml' as supplement. * Update RBAC docs for v1 (#5445) * Add user docs for pod priority and preemption (#5328) * Add user docs for pod priority and preemption * Update pod-priority-preemption.md * More updates * Update docs/admin/kubeadm.md for 1.8 (#5440) - Made a couple of minor wording changes (not strictly 1.8 related). - Did some reformatting (not strictly 1.8 related). - Updated references to the default token TTL (was infinite, now 24 hours). - Documented the new `--discovery-token-ca-cert-hash` and `--discovery-token-unsafe-skip-ca-verification` flags for `kubeadm join`. - Added references to the new `--discovery-token-ca-cert-hash` flag in all the default examples. - Added a new _Security model_ section that describes the security tradeoffs of the various discovery modes. - Documented the new `--groups` flag for `kubeadm token create`. - Added a note of caution under _Automating kubeadm_ that references the _Security model_ section. - Updated the component version table to drop 1.6 and add 1.8. - Update `_data/reference.yml` to try to get the sidebar fixed up and more consistent with `kubefed`. * Update StatefulSet Basics for 1.8 release (#5398) * addresses `kubeadm upgrade` review comments 2nd iteration review comments by @luxas * adds kubelet upgrade section to kubeadm upgrade * Fix a bulleted list on docs/admin/kubeadm.md. (#5458) I updated this doc yesterday and I was absolutely sure I fixed this, but I just saw that this commit got lost somehow. This was introduced recently in https://github.com/kubernetes/kubernetes.github.io/pull/5440. * Clarify the API to check for device plugins * Moving Flexvolume to separate out-of-tree section * addresses `kubeadm upgrade` review comments CC: @luxas * fixes kubeadm upgrade index * Update Stackdriver Logging documentation (#5495) * Re-update WordPress and MySQL PV doc to use apps/v1beta2 APIs (#5526) * Update statefulset concepts doc to use apps/v1beta2 APIs (#5420) * add document on kubectl's behavior regarding initializers (#5505) * Update docs/admin/kubeadm.md to cover self-hosting in 1.8. (#5497) This is a new beta feature in 1.8. * Update kubectl patch doc to use apps/v1beta2 APIs (#5422) * [1.8] Update "Run Applications" tasks to apps/v1beta2. (#5525) * Update replicated stateful application task for 1.8. * Update single instance stateful app task for 1.8. * Update stateless app task for 1.8. * Update kubectl patch task for 1.8. * fix the link of persistent storage (#5515) * update the admission-controllers.md index.md what-is-kubernetes.md link * fix the link of persistent storage * Add quota support for local ephemeral storage (#5493) * Add quota support for local ephemeral storage update the doc to this alpha feature * Update resource-quotas.md * Updated Deployments concepts doc (#5491) * Updated Deployments concepts doc * Addressed comments * Addressed more comments * Modify allocatable storage to ephemeral-storage (#5490) Update the doc to use ephemeral-storage instead of storage * Revamped concepts doc for ReplicaSet (#5463) * Revamped concepts doc for ReplicaSet * Minor changes to call out specific versions for selector defaulting and immutability * Addressed doc review comments * Remove petset documentations (#5395) * Update docs to use batch/v1beta1 cronjobs (#5475) * add federation job doc (#5485) * add federation job doc * Update job.md Edits for clarity and consistency * Update job.md Fixed a typo * update DaemonSet concept for 1.8 release (#5397) * update DaemonSet concept for 1.8 release * Update daemonset.md Fix typo. than -> then * Update bootstrap tokens doc for 1.8. (#5479) * Update bootstrap tokens doc for 1.8. This has some changes I missed when I was updating the main kubeadm documention: - Bootstrap tokens are now beta, not alpha (https://github.com/kubernetes/features/issues/130) - The apiserver flag to enable the authenticator changedin 1.8 (https://github.com/kubernetes/kubernetes/pull/51198) - Added `auth-extra-groups` documentaion (https://github.com/kubernetes/kubernetes/pull/50933) - Updated the _Token Management with `kubeadm`_ section to link to the main kubeadm docs, since it was just duplicated information. * Update bootstrap-tokens.md * Updated the Cassandra tutorial to use apps/v1beta2 (#5548) * add docs for AllowPrivilegeEscalation (#5448) Signed-off-by: Jess Frazelle <acidburn@microsoft.com> * Add local ephemeral storage alpha feature in managing compute resource (#5522) * Add local ephemeral storage alpha feature in managing compute resource Since 1.8, we add the local ephemeral storage alpha feature as one resource type to manage. Add this feature into the doc. * Update manage-compute-resources-container.md * Update manage-compute-resources-container.md * Update manage-compute-resources-container.md * Update manage-compute-resources-container.md * Update manage-compute-resources-container.md * Update manage-compute-resources-container.md * Added documentation for Metrics Server (#5560) * authorization: improve authorization debugging docs (#5549) * Document mount propagation (#5544) * Update /docs/setup/independent/create-cluster-kubeadm.md for 1.8. (#5524) This introduction needed a couple of small tweaks to cover the `--discovery-token-ca-cert-hash` flag added in https://github.com/kubernetes/kubernetes/pull/49520 and some version bumps. * Add task doc for alpha dynamic kubelet configuration (#5523) * Fix input/output of selfsubjectaccess review (#5593) * Add docs for implementing resize (#5528) * Add docs for implementing resize * Update admission-controllers.md * Added link to PVC section * minor typo fixes * Update NetworkPolicy concept guide with egress and CIDR changes (#5529) * update zookeeper tutorial for 1.8 release * add doc for hostpath type (#5503) * Federated Hpa feature doc (#5487) * Federated Hpa feature doc * Federated Hpa feature doc review fixes * Update hpa.md * Update hpa.md * update cloud controller manager docs for v1.8 * Update cronjob with defaults information (#5556) * Kubernetes 1.8 reference docs (#5632) * Kubernetes 1.8 reference docs * Kubectl reference docs for 1.8 * Update side bar with 1.8 kubectl and api ref docs links * remove petset.md * update on state of HostAlias in 1.8 with hostNetwork Pod support (#5644) * Fix cron job deletion section (#5655) * update imported docs (#5656) * Add documentation for certificate rotation. (#5639) * Link to using kubeadm page * fix the command output fix the command output * fix typo in api/resources reference: "Worloads" * Add documentation for certificate rotation. * Create TOC entry for cloud controller manager. (#5662) * Updates for new versions of API types * Followup 5655: fix link to garbage collection (#5666) * Temporarily redirect resources-reference to api-reference. (#5668) * Update config for 1.8 release. (#5661) * Update config for 1.8 release. * Address reviewer comments. * Switch references in HPA docs from alpha to beta (#5671) The HPA docs still referenced the alpha version. This switches them to talk about v2beta1, which is the appropriate version for Kubernetes 1.8 * Deprecate openstack heat (#5670) * Fix typo in pod preset conflict example Move container port definition to the correct line. * Highlight openstack-heat provider deprecation The openstack-heat provider for kube-up is being deprecated and will be removed in a future release. * Temporarily fix broken links by redirecting. (#5672) * Fix broken links. (#5675) * Fix render of code block (#5674) * Fix broken links. (#5677) * Add a small note about auto-bootstrapped CSR ClusterRoles (#5660) * Update kubeadm install doc for v1.8 (#5676) * add draft workloads api content for 1.8 (#5650) * add draft workloads api content for 1.8 * edits per review, add tables, for 1.8 workloads api doc * fix typo * Minor fixes to kubeadm 1.8 upgrade guide. (#5678) - The kubelet upgrade instructions should be done on every host, not just worker nodes. - We should just upgrade all packages, instead of calling out kubelet specifically. This will also upgrade kubectl, kubeadm, and kubernetes-cni, if installed. - Draining nodes should also ignore daemonsets, and master errors can be ignored. - Make sure that the new kubeadm download is chmoded correctly. - Add a step to run `kubeadm version` to verify after downloading. - Manually approve new kubelet CSRs if rotation is enabled (known issue). * Release 1.8 (#5680) * Fix versions for 1.8 API ref docs * Updates for 1.8 kubectl reference docs * Kubeadm /docs/admin/kubeadm.md cleanup, editing. (#5681) * Update docs/admin/kubeadm.md (mostly 1.8 related). This is Fabrizio's work, which I'm committing along with my edits (in a commit on top of this). * A few of my own edits to clarify and clean up some Markdown.
This commit is contained in:
@@ -0,0 +1,470 @@
|
||||
---
|
||||
approvers:
|
||||
- mtaufen
|
||||
- dawnchen
|
||||
title: Reconfigure a Node's Kubelet in a Live Cluster
|
||||
---
|
||||
|
||||
{% capture overview %}
|
||||
{% include feature-state-alpha.md %}
|
||||
As of Kubernetes 1.8, the new
|
||||
[Dynamic Kubelet Configuration](https://github.com/kubernetes/features/issues/281)
|
||||
feature is available in alpha. This allows you to change the configuration of
|
||||
Kubelets in a live Kubernetes cluster via first-class Kubernetes concepts.
|
||||
Specifically, this feature allows you to configure individual Nodes' Kubelets
|
||||
via ConfigMaps.
|
||||
|
||||
**Warning:** All Kubelet configuration parameters may be changed dynamically,
|
||||
but not all parameters are safe to change dynamically. This feature is intended
|
||||
for system experts who have a strong understanding of how configuration changes
|
||||
will affect behavior. No documentation currently exists which plainly lists
|
||||
"safe to change" fields, but we plan to add it before this feature graduates
|
||||
from alpha.
|
||||
{% endcapture %}
|
||||
|
||||
{% capture prerequisites %}
|
||||
- A live Kubernetes cluster with both Master and Node at v1.8 or higher must be
|
||||
running, with the `DynamicKubeletConfig` feature gate enabled and the Kubelet's
|
||||
`--dynamic-config-dir` flag set to a writeable directory on the Node.
|
||||
This flag must be set to enable Dynamic Kubelet Configuration.
|
||||
- The kubectl command-line tool must be also be v1.8 or higher, and must be
|
||||
configured to communicate with the cluster.
|
||||
{% endcapture %}
|
||||
|
||||
{% capture steps %}
|
||||
|
||||
## Reconfiguring the Kubelet on a Live Node in your Cluster
|
||||
|
||||
### Basic Workflow Overview
|
||||
|
||||
The basic workflow for configuring a Kubelet in a live cluster is as follows:
|
||||
|
||||
1. Write a YAML or JSON configuration file containing the
|
||||
Kubelet's configuration.
|
||||
2. Wrap this file in a ConfigMap and save it to the Kubernetes control plane.
|
||||
3. Update the Kubelet's correspoinding Node object to use this ConfigMap.
|
||||
|
||||
Each Kubelet watches a configuration reference on its respective Node object.
|
||||
When this reference changes, the Kubelet downloads the new configuration and
|
||||
exits. For the feature to work correctly, you must be running a process manager
|
||||
(like systemd) which will restart the Kubelet when it exits. When the Kubelet is
|
||||
restarted, it will begin using the new configuration.
|
||||
|
||||
The new configuration completely overrides the old configuration; unspecified
|
||||
fields in the new configuration will receive their canonical default values.
|
||||
Some CLI flags do not have an associated configuration field, and will not be
|
||||
affected by the new configuration. These fields are defined by the KubeletFlags
|
||||
structure, [here](https://github.com/kubernetes/kubernetes/blob/master/cmd/kubelet/app/options/options.go).
|
||||
|
||||
The status of the Node's Kubelet configuration is reported via the `ConfigOK`
|
||||
condition in the Node status. Once you have updated a Node to use the new
|
||||
ConfigMap, you can observe this condition to confirm that the Node is using the
|
||||
intended configuration. A table describing the possible conditions can be found
|
||||
at the end of this article.
|
||||
|
||||
This document describes editing Nodes using `kubectl edit`.
|
||||
There are other ways to modify a Node's spec, including `kubectl patch`, for
|
||||
example, which facilitate scripted workflows.
|
||||
|
||||
This document only describes a single Node consuming each ConfigMap. Keep in
|
||||
mind that it is also valid for multiple Nodes to consume the same ConfigMap.
|
||||
|
||||
### Node Authorizer Workarounds
|
||||
|
||||
The Node Authorizer does not yet pay attention to which ConfigMaps are assigned
|
||||
to which Nodes. If you currently use the Node authorizer, your Kubelets will not
|
||||
be automatically granted permission to download their respective ConfigMaps.
|
||||
|
||||
The temporary workaround used in this document is to manually create the RBAC
|
||||
Roles and RoleBindings for each ConfigMap. The Node Authorizer will be extended
|
||||
before the Dynamic Kubelet Configuration feature graduates from alpha, so doing
|
||||
this in production should never be necessary.
|
||||
|
||||
### Generating a file that contains the current configuration
|
||||
|
||||
The Dynamic Kubelet Configuration feature allows you to provide an override for
|
||||
the entire configuration object, rather than a per-field overlay. This is a
|
||||
simpler model that makes it easier to trace the source of configuration values
|
||||
and debug issues. The compromise, however, is that you must start with knowledge
|
||||
of the existing configuration to ensure that you only change the fields you
|
||||
intend to change.
|
||||
|
||||
In the future, the Kubelet will be bootstrapped from a file on disk, and you
|
||||
will simply edit a copy of this file (which, as a best practice, should live in
|
||||
version control) while creating the first Kubelet ConfigMap. Today, however, the
|
||||
Kubelet is still bootstrapped with command-line flags. Fortunately, there is a
|
||||
dirty trick you can use to generate a config file containing a Node's current
|
||||
configuration. The trick involves hitting the Kubelet server's `configz`
|
||||
endpoint via the kubectl proxy. This endpoint, in its current implementation, is
|
||||
intended to be used only as a debugging aid, which is part of why this is a
|
||||
dirty trick. There is ongoing work to improve the endpoint, and in the future
|
||||
this will be a less "dirty" operation. This trick also requires the `jq` command
|
||||
to be installed on your machine, for unpacking and editing the JSON response
|
||||
from the endpoint.
|
||||
|
||||
Do the following to generate the file:
|
||||
|
||||
1. Pick a Node to reconfigure. We will refer to this Node's name as NODE_NAME.
|
||||
2. Start the kubectl proxy in the background with `kubectl proxy --port=8001 &`
|
||||
3. Run the following command to download and unpack the configuration from the
|
||||
configz endpoint:
|
||||
|
||||
```
|
||||
$ export NODE_NAME=the-name-of-the-node-you-are-reconfiguring
|
||||
$ curl -sSL http://localhost:8001/api/v1/proxy/nodes/${NODE_NAME}/configz | jq '.kubeletconfig|.kind="KubeletConfiguration"|.apiVersion="kubeletconfig/v1alpha1"' > kubelet_configz_${NODE_NAME}
|
||||
```
|
||||
|
||||
Note that we have to manually add the `kind` and `apiVersion` to the downloaded
|
||||
object, as these are not reported by the configz endpoint. This is one of the
|
||||
limitations of the endpoint that is planned to be fixed in the future.
|
||||
|
||||
### Edit the configuration file
|
||||
|
||||
Using your editor of choice, change one of the parameters in the
|
||||
`kubelet_configz_${NODE_NAME}` file from the previous step. A QPS parameter,
|
||||
`eventRecordQPS` for example, is a good candidate.
|
||||
|
||||
### Push the configuration file to the control plane
|
||||
|
||||
Push the edited configuration file to the control plane with the
|
||||
following command:
|
||||
|
||||
```
|
||||
$ kubectl -n kube-system create configmap my-node-config --from-file=kubelet=kubelet_configz_${NODE_NAME} --append-hash -o yaml
|
||||
```
|
||||
|
||||
You should see a response similar to:
|
||||
|
||||
```
|
||||
apiVersion: v1
|
||||
data:
|
||||
kubelet: |
|
||||
{...}
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
creationTimestamp: 2017-09-14T20:23:33Z
|
||||
name: my-node-config-gkt4c2m4b2
|
||||
namespace: kube-system
|
||||
resourceVersion: "119980"
|
||||
selfLink: /api/v1/namespaces/kube-system/configmaps/my-node-config-gkt4c2m4b2
|
||||
uid: 946d785e-998a-11e7-a8dd-42010a800006
|
||||
```
|
||||
|
||||
Note that the configuration data must appear under the ConfigMap's
|
||||
`kubelet` key.
|
||||
|
||||
We create the ConfigMap in the `kube-system` namespace, which is appropriate
|
||||
because this ConfigMap configures a Kubernetes system component - the Kubelet.
|
||||
|
||||
The `--append-hash` option appends a short checksum of the ConfigMap contents
|
||||
to the name. This is convenient for an edit->push workflow, as it will
|
||||
automatically, yet deterministically, generate new names for new ConfigMaps.
|
||||
|
||||
We use the `-o yaml` output format so that the name, namespace, and uid are all
|
||||
reported following creation. We will need these in the next step. We will refer
|
||||
to the name as CONFIG_MAP_NAME and the uid as CONFIG_MAP_UID.
|
||||
|
||||
### Authorize your Node to read the new ConfigMap
|
||||
|
||||
Now that you've created a new ConfigMap, you need to authorize your node to
|
||||
read it. First, create a Role for your new ConfigMap with the
|
||||
following commands:
|
||||
|
||||
```
|
||||
$ export CONFIG_MAP_NAME=name-from-previous-output
|
||||
$ kubectl -n kube-system create role ${CONFIG_MAP_NAME}-reader --verb=get --resource=configmap --resource-name=${CONFIG_MAP_NAME}
|
||||
```
|
||||
|
||||
Next, create a RoleBinding to associate your Node with the new Role:
|
||||
|
||||
```
|
||||
$ kubectl -n kube-system create rolebinding ${CONFIG_MAP_NAME}-reader --role=${CONFIG_MAP_NAME}-reader --user=system:node:${NODE_NAME}
|
||||
```
|
||||
|
||||
Once the Node Authorizer is updated to do this automatically, you will
|
||||
be able to skip this step.
|
||||
|
||||
### Set the Node to use the new configuration
|
||||
|
||||
Edit the Node's reference to point to the new ConfigMap with the
|
||||
following command:
|
||||
|
||||
```
|
||||
kubectl edit node ${NODE_NAME}
|
||||
```
|
||||
|
||||
Once in your editor, add the following YAML under `spec`:
|
||||
|
||||
```
|
||||
configSource:
|
||||
configMapRef:
|
||||
name: CONFIG_MAP_NAME
|
||||
namespace: kube-system
|
||||
uid: CONFIG_MAP_UID
|
||||
```
|
||||
|
||||
Be sure to specify all three of `name`, `namespace`, and `uid`.
|
||||
|
||||
### Observe that the Node begins using the new configuration
|
||||
|
||||
Retrieve the Node with `kubectl get node ${NODE_NAME} -o yaml`, and look for the
|
||||
`ConfigOK` condition in `status.conditions`. You should see the message
|
||||
`Using current (UID: CONFIG_MAP_UID)` when the Kubelet starts using the new
|
||||
configuration.
|
||||
|
||||
For convenience, you can use the following command (using `jq`) to filter down
|
||||
to the `ConfigOK` condition:
|
||||
|
||||
```
|
||||
$ kubectl get no ${NODE_NAME} -o json | jq '.status.conditions|map(select(.type=="ConfigOK"))'
|
||||
[
|
||||
{
|
||||
"lastHeartbeatTime": "2017-09-20T18:08:29Z",
|
||||
"lastTransitionTime": "2017-09-20T18:08:17Z",
|
||||
"message": "using current (UID: \"2ebc8d1a-9e2a-11e7-a8dd-42010a800006\")",
|
||||
"reason": "passing all checks",
|
||||
"status": "True",
|
||||
"type": "ConfigOK"
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
If something goes wrong, you may see one of several different error conditions,
|
||||
detailed in the Table of ConfigOK Conditions, below. When this happens, you
|
||||
should check the Kubelet's log for more details.
|
||||
|
||||
### Edit the configuration file again
|
||||
|
||||
To change the configuration again, we simply repeat the above workflow.
|
||||
Try editing the `kubelet` file, changing the previously changed parameter to a
|
||||
new value.
|
||||
|
||||
### Push the newly edited configuration to the control plane
|
||||
|
||||
Push the new configuration to the control plane in a new ConfigMap with the
|
||||
following command:
|
||||
|
||||
```
|
||||
$ kubectl create configmap my-node-config --namespace=kube-system --from-file=kubelet=kubelet_configz_${NODE_NAME} --append-hash -o yaml
|
||||
```
|
||||
|
||||
This new ConfigMap will get a new name, as we have changed the contents.
|
||||
We will refer to the new name as NEW_CONFIG_MAP_NAME and the new uid
|
||||
as NEW_CONFIG_MAP_UID.
|
||||
|
||||
### Authorize your Node to read the new ConfigMap
|
||||
|
||||
Now that you've created a new ConfigMap, you need to authorize your node to
|
||||
read it. First, create a Role for your new ConfigMap with the
|
||||
following commands:
|
||||
|
||||
```
|
||||
$ export NEW_CONFIG_MAP_NAME=name-from-previous-output
|
||||
$ kubectl -n kube-system create role ${NEW_CONFIG_MAP_NAME}-reader --verb=get --resource=configmap --resource-name=${NEW_CONFIG_MAP_NAME}
|
||||
```
|
||||
|
||||
Next, create a RoleBinding to associate your Node with the new Role:
|
||||
|
||||
```
|
||||
$ kubectl -n kube-system create rolebinding ${NEW_CONFIG_MAP_NAME}-reader --role=${NEW_CONFIG_MAP_NAME}-reader --user=system:node:${NODE_NAME}
|
||||
```
|
||||
|
||||
Once the Node Authorizer is updated to do this automatically, you will
|
||||
be able to skip this step.
|
||||
|
||||
### Configure the Node to use the new configuration
|
||||
|
||||
Once more, edit the Node's `spec.configSource` with
|
||||
`kubectl edit node ${NODE_NAME}`. Your new `spec.configSource` should look like
|
||||
the following, with `name` and `uid` substituted as necessary:
|
||||
|
||||
```
|
||||
configSource:
|
||||
configMapRef:
|
||||
name: NEW_CONFIG_MAP_NAME
|
||||
namespace: kube-system
|
||||
uid: NEW_CONFIG_MAP_UID
|
||||
```
|
||||
|
||||
### Observe that the Kubelet is using the new configuration
|
||||
|
||||
Once more, retrieve the Node with `kubectl get node ${NODE_NAME} -o yaml`, and
|
||||
look for the `ConfigOK` condition in `status.conditions`. You should the message
|
||||
`Using current (UID: NEW_CONFIG_MAP_UID)` when the Kubelet starts using the
|
||||
new configuration.
|
||||
|
||||
### Deauthorize your Node fom reading the old ConfigMap
|
||||
|
||||
Once you know your Node is using the new configuration and are confident that
|
||||
the new configuration has not caused any problems, it is a good idea to
|
||||
deauthorize the node from reading the old ConfigMap. Run the following
|
||||
commands to remove the RoleBinding and Role:
|
||||
|
||||
```
|
||||
$ kubectl -n kube-system delete rolebinding ${CONFIG_MAP_NAME}-reader
|
||||
$ kubectl -n kube-system delete role ${CONFIG_MAP_NAME}-reader
|
||||
```
|
||||
|
||||
Note that this does not necessarily prevent the Node from reverting to the old
|
||||
configuration, as it may locally cache the old ConfigMap for an indefinite
|
||||
period of time.
|
||||
|
||||
You may optionally also choose to remove the old ConfigMap:
|
||||
|
||||
```
|
||||
$ kubectl -n kube-system delete configmap ${CONFIG_MAP_NAME}
|
||||
```
|
||||
|
||||
Once the Node Authorizer is updated to do this automatically, you will
|
||||
be able to skip this step.
|
||||
|
||||
### Reset the Node to use its local default configuration
|
||||
|
||||
Finally, if you wish to reset the Node to use the configuration it was
|
||||
provisioned with, simply edit the Node with `kubectl edit node ${NODE_NAME}` and
|
||||
remove the `spec.configSource` subfield.
|
||||
|
||||
### Observe that the Node is using its local default configuration
|
||||
|
||||
After removing this subfield, you should eventually observe that the ConfigOK
|
||||
condition's message reverts to either `using current (default)` or
|
||||
`using current (init)`, depending on how the Node was provisioned.
|
||||
|
||||
### Deauthorize your Node fom reading the old ConfigMap
|
||||
|
||||
Once you know your Node is using the default configuraiton again, it is a good
|
||||
idea to deauthorize the node from reading the old ConfigMap. Run the following
|
||||
commands to remove the RoleBinding and Role:
|
||||
|
||||
```
|
||||
$ kubectl -n kube-system delete rolebinding ${NEW_CONFIG_MAP_NAME}-reader
|
||||
$ kubectl -n kube-system delete role ${NEW_CONFIG_MAP_NAME}-reader
|
||||
```
|
||||
|
||||
Note that this does not necessarily prevent the Node from reverting to the old
|
||||
ConfigMap, as it may locally cache the old ConfigMap for an indefinite
|
||||
period of time.
|
||||
|
||||
You may optionally also choose to remove the old ConfigMap:
|
||||
|
||||
```
|
||||
$ kubectl -n kube-system delete configmap ${NEW_CONFIG_MAP_NAME}
|
||||
```
|
||||
|
||||
Once the Node Authorizer is updated to do this automatically, you will
|
||||
be able to skip this step.
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
{% capture discussion %}
|
||||
## Kubectl Patch Example
|
||||
As mentioned above, there are many ways to change a Node's configSource.
|
||||
Here is an example command that uses `kubectl patch`:
|
||||
|
||||
```
|
||||
kubectl patch node ${NODE_NAME} -p "{\"spec\":{\"configSource\":{\"configMapRef\":{\"name\":\"${CONFIG_MAP_NAME}\",\"namespace\":\"kube-system\",\"uid\":\"${CONFIG_MAP_UID}\"}}}}"
|
||||
```
|
||||
|
||||
## Understanding ConfigOK Conditions
|
||||
|
||||
The following table describes several of the `ConfigOK` Node conditions you
|
||||
might encounter in a cluster that has Dynamic Kubelet Config enabled. If you
|
||||
observe a condition with `status=False`, you should check the Kubelet log for
|
||||
more error details by searching for the message or reason text.
|
||||
|
||||
<table>
|
||||
|
||||
|
||||
<table align="left">
|
||||
<tr>
|
||||
<th>Possible Messages</th>
|
||||
<th>Possible Reasons</th>
|
||||
<th>Status</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><p>using current (default)</p></td>
|
||||
<td><p>current is set to the local default, and no init config was provided</p></td>
|
||||
<td><p>True</p></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><p>using current (init)</p></td>
|
||||
<td><p>current is set to the local default, and an init config was provided</p></td>
|
||||
<td><p>True</p></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><p>using current (UID: CURRENT_CONFIG_MAP_UID)</p></td>
|
||||
<td><p>passing all checks</p></td>
|
||||
<td><p>True</p></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><p>using last-known-good (default)</p></td>
|
||||
<td>
|
||||
<ul>
|
||||
<li>failed to load current (UID: CURRENT_CONFIG_MAP_UID)</li>
|
||||
<li>failed to parse current (UID: CURRENT_CONFIG_MAP_UID)</li>
|
||||
<li>failed to validate current (UID: CURRENT_CONFIG_MAP_UID)</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><p>False</p></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><p>using last-known-good (init)</p></td>
|
||||
<td>
|
||||
<ul>
|
||||
<li>failed to load current (UID: CURRENT_CONFIG_MAP_UID)</li>
|
||||
<li>failed to parse current (UID: CURRENT_CONFIG_MAP_UID)</li>
|
||||
<li>failed to validate current (UID: CURRENT_CONFIG_MAP_UID)</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><p>False</p></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><p>using last-known-good (UID: LAST_KNOWN_GOOD_CONFIG_MAP_UID)</p></td>
|
||||
<td>
|
||||
<ul>
|
||||
<li>failed to load current (UID: CURRENT_CONFIG_MAP_UID)</li>
|
||||
<li>failed to parse current (UID: CURRENT_CONFIG_MAP_UID)</li>
|
||||
<li>failed to validate current (UID: CURRENT_CONFIG_MAP_UID)</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><p>False</p></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
<p>
|
||||
The reasons in the next column could potentially appear for any of
|
||||
the above messages.
|
||||
</p>
|
||||
<p>
|
||||
This condition indicates that the Kubelet is having trouble
|
||||
reconciling `spec.configSource`, and thus no change to the in-use
|
||||
configuration has occurred.
|
||||
</p>
|
||||
<p>
|
||||
The "failed to sync" reasons are specific to the failure that
|
||||
occurred, and the next column does not necessarily contain all
|
||||
possible failure reasons.
|
||||
</p>
|
||||
</td>
|
||||
<td>
|
||||
<p>failed to sync, reason:</p>
|
||||
<ul>
|
||||
<li>failed to read Node from informer object cache</li>
|
||||
<li>failed to reset to local (default or init) config</li>
|
||||
<li>invalid NodeConfigSource, exactly one subfield must be non-nil, but all were nil</li>
|
||||
<li>invalid ObjectReference, all of UID, Name, and Namespace must be specified</li>
|
||||
<li>invalid ObjectReference, UID SOME_UID does not match UID of downloaded ConfigMap SOME_OTHER_UID</li>
|
||||
<li>failed to determine whether object with UID SOME_UID was already checkpointed</li>
|
||||
<li>failed to download ConfigMap with name SOME_NAME from namespace SOME_NAMESPACE</li>
|
||||
<li>failed to save config checkpoint for object with UID SOME_UID</li>
|
||||
<li>failed to set current config checkpoint to default</li>
|
||||
<li>failed to set current config checkpoint to object with UID SOME_UID</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><p>False</p></td>
|
||||
</tr>
|
||||
</table>
|
||||
{% endcapture %}
|
||||
|
||||
|
||||
{% include templates/task.md %}
|
||||
Reference in New Issue
Block a user