Update docs/admin/kubeadm.md for 1.8 (#5440)
- Made a couple of minor wording changes (not strictly 1.8 related). - Did some reformatting (not strictly 1.8 related). - Updated references to the default token TTL (was infinite, now 24 hours). - Documented the new `--discovery-token-ca-cert-hash` and `--discovery-token-unsafe-skip-ca-verification` flags for `kubeadm join`. - Added references to the new `--discovery-token-ca-cert-hash` flag in all the default examples. - Added a new _Security model_ section that describes the security tradeoffs of the various discovery modes. - Documented the new `--groups` flag for `kubeadm token create`. - Added a note of caution under _Automating kubeadm_ that references the _Security model_ section. - Updated the component version table to drop 1.6 and add 1.8. - Update `_data/reference.yml` to try to get the sidebar fixed up and more consistent with `kubefed`.
This commit is contained in:
+1
-3
@@ -65,10 +65,8 @@ toc:
|
||||
|
||||
- title: Setup Tools
|
||||
section:
|
||||
- title: Kubeadm
|
||||
path: /docs/admin/kubeadm/
|
||||
- docs/admin/kubeadm.md
|
||||
- title: Kubefed
|
||||
path: /docs/admin/kubefed/
|
||||
section:
|
||||
- docs/admin/kubefed.md
|
||||
- docs/admin/kubefed_options.md
|
||||
|
||||
+302
-157
@@ -4,11 +4,16 @@ approvers:
|
||||
- luxas
|
||||
- errordeveloper
|
||||
- jbeda
|
||||
title: kubeadm Setup Tool Reference Guide
|
||||
title: Kubeadm
|
||||
notitle: true
|
||||
---
|
||||
# kubeadm Setup Tool Reference Guide
|
||||
|
||||
This document provides information on how to use kubeadm's advanced options.
|
||||
|
||||
* TOC
|
||||
{:toc}
|
||||
|
||||
Running `kubeadm init` bootstraps a Kubernetes cluster. This consists of the
|
||||
following steps:
|
||||
|
||||
@@ -27,8 +32,9 @@ following steps:
|
||||
dropping it in the cert directory (configured via `--cert-dir`, by default
|
||||
`/etc/kubernetes/pki`), this step is skipped.
|
||||
|
||||
1. Outputting a kubeconfig file for the kubelet to use to connect to the API
|
||||
server, as well as an additional kubeconfig file for administration.
|
||||
1. kubeadm outputs a kubeconfig file for the master's kubelet to use to connect
|
||||
to the API server, as well as an additional kubeconfig file for cluster
|
||||
administrators.
|
||||
|
||||
1. kubeadm generates Kubernetes static Pod manifests for the API server,
|
||||
controller manager and scheduler. It places them in
|
||||
@@ -46,8 +52,9 @@ following steps:
|
||||
Running `kubeadm join` on each node in the cluster consists of the following
|
||||
steps:
|
||||
|
||||
1. kubeadm downloads root CA information from the API server. It uses the token
|
||||
to verify the authenticity of that data.
|
||||
1. kubeadm downloads root CA information from the API server. By default, it
|
||||
uses the bootstrap token and the CA key hash to verify the authenticity of
|
||||
that data. The root CA can also be discovered directly via a file or URL.
|
||||
|
||||
1. kubeadm creates a local key pair. It prepares a certificate signing request
|
||||
(CSR) and sends that off to the API server for signing. The bootstrap token
|
||||
@@ -72,121 +79,123 @@ It is usually sufficient to run `kubeadm init` without any flags, but in some
|
||||
cases you might like to override the default behaviour. Here we specify all the
|
||||
flags that can be used to customise the Kubernetes installation.
|
||||
|
||||
**Options for `kubeadm init`:**
|
||||
|
||||
- `--apiserver-advertise-address`
|
||||
|
||||
This is the address the API Server will advertise to other members of the
|
||||
cluster. This is also the address used to construct the suggested `kubeadm
|
||||
join` line at the end of the init process. If not set (or set to 0.0.0.0) then
|
||||
IP for the default interface will be used.
|
||||
This is the address the API Server will advertise to other members of the
|
||||
cluster. This is also the address used to construct the suggested `kubeadm
|
||||
join` line at the end of the init process. If not set (or set to 0.0.0.0) then
|
||||
IP for the default interface will be used.
|
||||
|
||||
This address is also added to the certificate that the API Server uses.
|
||||
This address is also added to the certificate that the API Server uses.
|
||||
|
||||
- `--apiserver-bind-port`
|
||||
|
||||
The port that the API server will bind on. This defaults to 6443.
|
||||
The port that the API server will bind on. This defaults to 6443.
|
||||
|
||||
- `--apiserver-cert-extra-sans`
|
||||
|
||||
Additional hostnames or IP addresses that should be added to the Subject
|
||||
Alternate Name section for the certificate that the API Server will use. If you
|
||||
expose the API Server through a load balancer and public DNS you could specify
|
||||
this with
|
||||
|
||||
```
|
||||
--apiserver-cert-extra-sans=kubernetes.example.com,kube.example.com,10.100.245.1
|
||||
```
|
||||
Additional hostnames or IP addresses that should be added to the Subject
|
||||
Alternate Name section for the certificate that the API Server will use. If you
|
||||
expose the API Server through a load balancer and public DNS you could specify
|
||||
this with
|
||||
|
||||
```
|
||||
--apiserver-cert-extra-sans=kubernetes.example.com,kube.example.com,10.100.245.1
|
||||
```
|
||||
|
||||
- `--cert-dir`
|
||||
|
||||
The path where to save and store the certificates. The default is
|
||||
"/etc/kubernetes/pki".
|
||||
The path where to save and store the certificates. The default is
|
||||
"/etc/kubernetes/pki".
|
||||
|
||||
- `--config`
|
||||
|
||||
A kubeadm specific [config file](#config-file). This can be used to specify an
|
||||
extended set of options including passing arbitrary command line flags to the
|
||||
control plane components.
|
||||
|
||||
**Note**: When providing configuration values using _both_ a configuration file
|
||||
and flags, the file will take precedence. For example, if a file exists with:
|
||||
|
||||
```yaml
|
||||
apiVersion: kubeadm.k8s.io/v1alpha1
|
||||
kind: MasterConfiguration
|
||||
token: 1234
|
||||
```
|
||||
|
||||
and the user ran `kubeadm init --config file.yaml --token 5678`,
|
||||
the chosen token value will be `1234`.
|
||||
A kubeadm specific [config file](#config-file). This can be used to specify an
|
||||
extended set of options including passing arbitrary command line flags to the
|
||||
control plane components.
|
||||
|
||||
**Note**: When providing configuration values using _both_ a configuration file
|
||||
and flags, the file will take precedence. For example, if a file exists with:
|
||||
|
||||
```yaml
|
||||
apiVersion: kubeadm.k8s.io/v1alpha1
|
||||
kind: MasterConfiguration
|
||||
token: 1234
|
||||
```
|
||||
|
||||
and the user ran `kubeadm init --config file.yaml --token 5678`,
|
||||
the chosen token value will be `1234`.
|
||||
|
||||
|
||||
- `--kubernetes-version` (default 'latest') the kubernetes version to initialise
|
||||
|
||||
The **v1.6** version of kubeadm only supports building clusters that are at
|
||||
least **v1.6.0**. There are many reasons for this including kubeadm's use of
|
||||
RBAC, the Bootstrap Token system, and enhancements to the Certificates API. With
|
||||
this flag you can try any future version of Kubernetes. Check [releases
|
||||
page](https://github.com/kubernetes/kubernetes/releases) for a full list of
|
||||
available versions.
|
||||
The **v1.6** version of kubeadm only supports building clusters that are at
|
||||
least **v1.6.0**. There are many reasons for this including kubeadm's use of
|
||||
RBAC, the Bootstrap Token system, and enhancements to the Certificates API. With
|
||||
this flag you can try any future version of Kubernetes. Check [releases
|
||||
page](https://github.com/kubernetes/kubernetes/releases) for a full list of
|
||||
available versions.
|
||||
|
||||
- `--pod-network-cidr`
|
||||
|
||||
For certain networking solutions the Kubernetes master can also play a role in
|
||||
allocating network ranges (CIDRs) to each node. This includes many cloud
|
||||
providers and flannel. You can specify a subnet range that will be broken down
|
||||
and handed out to each node with the `--pod-network-cidr` flag. This should be a
|
||||
minimum of a /16 so controller-manager is able to assign /24 subnets to each
|
||||
node in the cluster. If you are using flannel with [this
|
||||
manifest](https://github.com/coreos/flannel/blob/master/Documentation/kube-flannel.yml)
|
||||
you should use `--pod-network-cidr=10.244.0.0/16`. Most CNI based networking
|
||||
solutions do not require this flag.
|
||||
For certain networking solutions the Kubernetes master can also play a role in
|
||||
allocating network ranges (CIDRs) to each node. This includes many cloud
|
||||
providers and flannel. You can specify a subnet range that will be broken down
|
||||
and handed out to each node with the `--pod-network-cidr` flag. This should be a
|
||||
minimum of a /16 so controller-manager is able to assign /24 subnets to each
|
||||
node in the cluster. If you are using flannel with [this
|
||||
manifest](https://github.com/coreos/flannel/blob/master/Documentation/kube-flannel.yml)
|
||||
you should use `--pod-network-cidr=10.244.0.0/16`. Most CNI based networking
|
||||
solutions do not require this flag.
|
||||
|
||||
- `--service-cidr` (default '10.96.0.0/12')
|
||||
|
||||
You can use the `--service-cidr` flag to override the subnet Kubernetes uses to
|
||||
assign pods IP addresses. If you do, you will also need to update the
|
||||
`/etc/systemd/system/kubelet.service.d/10-kubeadm.conf` file to reflect this
|
||||
change else DNS will not function correctly.
|
||||
You can use the `--service-cidr` flag to override the subnet Kubernetes uses to
|
||||
assign pods IP addresses. If you do, you will also need to update the
|
||||
`/etc/systemd/system/kubelet.service.d/10-kubeadm.conf` file to reflect this
|
||||
change else DNS will not function correctly.
|
||||
|
||||
- `--service-dns-domain` (default 'cluster.local')
|
||||
|
||||
By default, `kubeadm init` deploys a cluster that assigns services with DNS
|
||||
names `<service_name>.<namespace>.svc.cluster.local`. You can use the
|
||||
`--service-dns-domain` to change the DNS name suffix. Again, you will need to
|
||||
update the `/etc/systemd/system/kubelet.service.d/10-kubeadm.conf` file
|
||||
accordingly else DNS will not function correctly.
|
||||
|
||||
**Note**: This flag has an effect (it's needed for the kube-dns Deployment
|
||||
manifest and the API Server's serving certificate) but not as you might expect,
|
||||
since you will have to modify the arguments to the kubelets in the cluster for
|
||||
it to work fully. Specifying DNS parameters using this flag only is not enough.
|
||||
Rewriting the kubelet's CLI arguments is out of scope for kubeadm as it should
|
||||
be agnostic to how you run the kubelet. However, making all kubelets in the
|
||||
cluster pick up information dynamically via the API _is_ in scope and is a
|
||||
[planned feature](https://github.com/kubernetes/kubeadm/issues/28) for upcoming
|
||||
releases.
|
||||
By default, `kubeadm init` deploys a cluster that assigns services with DNS
|
||||
names `<service_name>.<namespace>.svc.cluster.local`. You can use the
|
||||
`--service-dns-domain` to change the DNS name suffix. Again, you will need to
|
||||
update the `/etc/systemd/system/kubelet.service.d/10-kubeadm.conf` file
|
||||
accordingly else DNS will not function correctly.
|
||||
|
||||
**Note**: This flag has an effect (it's needed for the kube-dns Deployment
|
||||
manifest and the API Server's serving certificate) but not as you might expect,
|
||||
since you will have to modify the arguments to the kubelets in the cluster for
|
||||
it to work fully. Specifying DNS parameters using this flag only is not enough.
|
||||
Rewriting the kubelet's CLI arguments is out of scope for kubeadm as it should
|
||||
be agnostic to how you run the kubelet. However, making all kubelets in the
|
||||
cluster pick up information dynamically via the API _is_ in scope and is a
|
||||
[planned feature](https://github.com/kubernetes/kubeadm/issues/28) for upcoming
|
||||
releases.
|
||||
|
||||
- `--skip-preflight-checks`
|
||||
|
||||
By default, kubeadm runs a series of preflight checks to validate the system
|
||||
before making any changes. Advanced users can use this flag to bypass these if
|
||||
necessary.
|
||||
By default, kubeadm runs a series of preflight checks to validate the system
|
||||
before making any changes. Advanced users can use this flag to bypass these if
|
||||
necessary.
|
||||
|
||||
- `--token`
|
||||
|
||||
By default, `kubeadm init` automatically generates the token used to initialise
|
||||
each new node. If you would like to manually specify this token, you can use the
|
||||
`--token` flag. The token must be of the format `[a-z0-9]{6}\.[a-z0-9]{16}`. A
|
||||
compatible random token can be generated `kubeadm token generate`. Tokens can
|
||||
be managed through the API after the cluster is created. See the [section on
|
||||
managing tokens](#manage-tokens) below.
|
||||
By default, `kubeadm init` automatically generates the token used to initialise
|
||||
each new node. If you would like to manually specify this token, you can use the
|
||||
`--token` flag. The token must be of the format `[a-z0-9]{6}\.[a-z0-9]{16}`. A
|
||||
compatible random token can be generated `kubeadm token generate`. Tokens can
|
||||
be managed through the API after the cluster is created. See the [section on
|
||||
managing tokens](#manage-tokens) below.
|
||||
|
||||
- `--token-ttl`
|
||||
|
||||
This sets an expiration time for the token. This is specified as a duration
|
||||
from the current time. After this time the token will no longer be valid and
|
||||
will be removed. A value of 0 specifies that the token never expires. 0 is the
|
||||
default. See the [section on managing tokens](#manage-tokens) below.
|
||||
This sets an expiration time for the token. This is specified as a duration
|
||||
from the current time. After this time the token will no longer be valid and
|
||||
will be removed. A value of 0 specifies that the token never expires. The
|
||||
default is 24 hours. See the [section on managing tokens](#manage-tokens) below.
|
||||
|
||||
### `kubeadm join`
|
||||
|
||||
@@ -194,15 +203,24 @@ When joining a kubeadm initialized cluster, we need to establish bidirectional
|
||||
trust. This is split into discovery (having the Node trust the Kubernetes
|
||||
master) and TLS bootstrap (having the Kubernetes master trust the Node).
|
||||
|
||||
There are 2 main schemes for discovery. The first is to use a shared token along
|
||||
with the IP address of the API server. The second is to provide a file (a subset
|
||||
of the standard kubeconfig file). This file can be a local file or downloaded
|
||||
via an HTTPS URL. The forms are `kubeadm join --discovery-token
|
||||
abcdef.1234567890abcdef 1.2.3.4:6443`, `kubeadm join --discovery-file
|
||||
path/to/file.conf` or `kubeadm join --discovery-file https://url/file.conf`.
|
||||
There are two main schemes for discovery:
|
||||
|
||||
- **Using a shared token** along with the IP address of the API server and a
|
||||
hash of the root CA key:
|
||||
|
||||
`kubeadm join --discovery-token abcdef.1234567890abcdef --discovery-token-ca-cert-hash sha256:1234..cdef 1.2.3.4:6443`
|
||||
|
||||
- **Using a file** (a subset of the standard kubeconfig file). This file can
|
||||
be a local file or downloaded via an HTTPS URL:
|
||||
|
||||
`kubeadm join --discovery-file path/to/file.conf`
|
||||
|
||||
`kubeadm join --discovery-file https://url/file.conf`
|
||||
|
||||
Only one form can be used. If the discovery information is loaded from a URL,
|
||||
HTTPS must be used and the host installed CA bundle is used to verify the
|
||||
connection.
|
||||
connection. For details on the security tradeoffs of these mechanisms, see the
|
||||
[security model](#security-model) section below.
|
||||
|
||||
The TLS bootstrap mechanism is also driven via a shared token. This is used to
|
||||
temporarily authenticate with the Kubernetes master to submit a certificate
|
||||
@@ -216,60 +234,83 @@ can be used instead of specifying the each token individually.
|
||||
|
||||
Here's an example on how to use it:
|
||||
|
||||
`kubeadm join --token=abcdef.1234567890abcdef 192.168.1.1:6443`
|
||||
`kubeadm join --token=abcdef.1234567890abcdef --discovery-token-ca-cert-hash sha256:1234..cdef 192.168.1.1:6443`
|
||||
|
||||
Specific options:
|
||||
**Options for `kubeadm join`:**
|
||||
|
||||
- `--config`
|
||||
|
||||
Extended options a specified in the [kubeadm specific config file](#config-file).
|
||||
Extended options are specified in the [kubeadm specific config file](#config-file).
|
||||
|
||||
- `--skip-preflight-checks`
|
||||
|
||||
By default, kubeadm runs a series of preflight checks to validate the system
|
||||
before making any changes. Advanced users can use this flag to bypass these if
|
||||
necessary.
|
||||
By default, kubeadm runs a series of preflight checks to validate the system
|
||||
before making any changes. Advanced users can use this flag to bypass these if
|
||||
necessary.
|
||||
|
||||
- `--discovery-file`
|
||||
|
||||
A local file path or HTTPS URL. The file specified must be a kubeconfig file
|
||||
with nothing but an unnamed cluster entry. This is used to find both the
|
||||
location of the API server to join along with a root CA bundle to use when
|
||||
talking to that server.
|
||||
|
||||
This might look something like this:
|
||||
|
||||
``` yaml
|
||||
apiVersion: v1
|
||||
clusters:
|
||||
- cluster:
|
||||
certificate-authority-data: <really long certificate data>
|
||||
server: https://10.138.0.2:6443
|
||||
name: ""
|
||||
contexts: []
|
||||
current-context: ""
|
||||
kind: Config
|
||||
preferences: {}
|
||||
users: []
|
||||
```
|
||||
A local file path or HTTPS URL. The file specified must be a kubeconfig file
|
||||
with nothing but an unnamed cluster entry. This is used to find both the
|
||||
location of the API server to join along with a root CA bundle to use when
|
||||
talking to that server.
|
||||
|
||||
This might look something like this:
|
||||
|
||||
``` yaml
|
||||
apiVersion: v1
|
||||
clusters:
|
||||
- cluster:
|
||||
certificate-authority-data: <really long certificate data>
|
||||
server: https://10.138.0.2:6443
|
||||
name: ""
|
||||
contexts: []
|
||||
current-context: ""
|
||||
kind: Config
|
||||
preferences: {}
|
||||
users: []
|
||||
```
|
||||
|
||||
- `--discovery-token`
|
||||
|
||||
The discovery token is used along with the address of the API server (as an
|
||||
unnamed argument) to download and verify information about the cluster. The
|
||||
most critical part of the cluster information is the root CA bundle used to
|
||||
verify the identity of the server during subsequent TLS connections.
|
||||
The discovery token is used along with the address of the API server (as an
|
||||
unnamed argument) to download and verify information about the cluster. The
|
||||
most critical part of the cluster information is the root CA bundle used to
|
||||
verify the identity of the server during subsequent TLS connections.
|
||||
|
||||
- `--discovery-token-ca-cert-hash`
|
||||
|
||||
The CA key hash is used to verify the full root CA certificate discovered during
|
||||
token-based bootstrapping. It has the format `sha256:<hex_encoded_hash>`. By
|
||||
default, the hash value is returned in the `kubeadm join` command printed at the
|
||||
end of `kubeadm init`. It is in a standard format (see
|
||||
[RFC7469](https://tools.ietf.org/html/rfc7469#section-2.4)) and can also be
|
||||
calculated by 3rd party tools or provisioning systems. For example, using the
|
||||
OpenSSL CLI:
|
||||
`openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>&/dev/null | openssl dgst -sha256 -hex`
|
||||
|
||||
_Skipping this flag is allowed in Kubernetes 1.8, but makes certain spoofing
|
||||
attacks possible._ See the [security model](#security-model) for details.
|
||||
Pass `--discovery-token-unsafe-skip-ca-verification` to silence warnings (which
|
||||
will become errors in Kubernetes 1.9).
|
||||
|
||||
- `--discovery-token-unsafe-skip-ca-verification`
|
||||
|
||||
Disable the warning/error that occurs when `--discovery-token-ca-cert-hash` is
|
||||
not provided. Passing this flag is an acknowledgement of the
|
||||
[security tradeoffs](#security-model) involved in skipping this verification
|
||||
(which may or may not be appropriate in your environment).
|
||||
|
||||
- `--tls-bootstrap-token`
|
||||
|
||||
The token used to authenticate to the API server for the purposes of TLS
|
||||
bootstrapping.
|
||||
The token used to authenticate to the API server for the purposes of TLS
|
||||
bootstrapping.
|
||||
|
||||
- `--token=<token>`
|
||||
|
||||
Often times the same token is used for both `--discovery-token` and
|
||||
`--tls-bootstrap-token`. This option specifies the same token for both. Other
|
||||
flags override this flag if present.
|
||||
Often times the same token is used for both `--discovery-token` and
|
||||
`--tls-bootstrap-token`. This option specifies the same token for both. Other
|
||||
flags override this flag if present.
|
||||
|
||||
## Using kubeadm with a configuration file {#config-file}
|
||||
|
||||
@@ -414,34 +455,53 @@ created cluster (`/etc/kubernetes/admin.conf`). You can specify an alternate
|
||||
kubeconfig file for credentials with the `--kubeconfig` to the following
|
||||
commands.
|
||||
|
||||
* `kubeadm token list` Lists the tokens along with when they expire and what the
|
||||
approved usages are.
|
||||
* `kubeadm token create` Creates a new token.
|
||||
* `--description` Set the description on the new token.
|
||||
* `--ttl duration` Set expiration time of the token as a delta from "now".
|
||||
Default is 0 for no expiration. The unit of the duration is seconds.
|
||||
* `--usages` Set the ways that the token can be used. The default is
|
||||
`signing,authentication`. These are the usages as described above.
|
||||
* `kubeadm token delete <token id>|<token id>.<token secret>` Delete a token.
|
||||
The token can either be identified with just an ID or with the entire token
|
||||
value. Only the ID is used; the token is still deleted if the secret does not
|
||||
match.
|
||||
* `kubeadm token list` - List tokens (along with their expirations, usages, and groups).
|
||||
|
||||
In addition, you can use the `kubeadm token generate` command to locally creates
|
||||
a new token. This token is of the correct form for specifying with the
|
||||
`--token` argument to `kubeadm init`.
|
||||
* `kubeadm token create` - Create a new token.
|
||||
|
||||
* `--description <description>`
|
||||
|
||||
Set the human-readable description for the new token.
|
||||
|
||||
* `--ttl <duration>`
|
||||
|
||||
Set the expiration time-to-live of the token relative to the current time.
|
||||
Default is 24 hours. A value of 0 means "never expire". The default unit
|
||||
of the duration is seconds but other units can be specified (e.g., `15m`, `1h`).
|
||||
|
||||
* `--usages <usage>[,<usage>...]`
|
||||
|
||||
Set the ways that the token can be used. The default is
|
||||
`signing,authentication`. These are the usages as described above.
|
||||
|
||||
* `--groups <group>[,<group>...]`
|
||||
|
||||
Add extra bootstrap groups that the new token will authenticate as. Can be
|
||||
specified multiple times. Each extra group must start with
|
||||
`system:bootstrappers:`. This is an advanced feature meant for custom
|
||||
bootstrap scenarios where you want to change how CSR approval works for
|
||||
different groups of nodes.
|
||||
|
||||
* `kubeadm token delete <token id>|<token id>.<token secret>` - Delete a token.
|
||||
|
||||
The token can either be identified with just an ID or with the
|
||||
entire token value. Only the ID is used; the token is still deleted if the
|
||||
secret does not match.
|
||||
|
||||
* `kubeadm token generate` - Generate a token locally.
|
||||
|
||||
Locally generate a token but do not create it on the server. This token is of
|
||||
the correct form for specifying with the `--token` argument to `kubeadm init`.
|
||||
|
||||
For the gory details on how the tokens are implemented (including managing them
|
||||
outside of kubeadm) see the [Bootstrap Token
|
||||
docs](/docs/admin/bootstrap-tokens/).
|
||||
outside of kubeadm) see the [Bootstrap Token docs](/docs/admin/bootstrap-tokens/).
|
||||
|
||||
## Automating kubeadm
|
||||
|
||||
Rather than copying the token you obtained from `kubeadm init` to each node, as
|
||||
in the [basic kubeadm tutorial](/docs/admin/kubeadm/), you can
|
||||
parallelize the token distribution for easier automation. To implement this
|
||||
automation, you must know the IP address that the master will have after it is
|
||||
started.
|
||||
in the [basic kubeadm tutorial](/docs/admin/kubeadm/), you can parallelize the
|
||||
token distribution for easier automation. To implement this automation, you must
|
||||
know the IP address that the master will have after it is started.
|
||||
|
||||
1. Generate a token. This token must have the form `<6 character string>.<16
|
||||
character string>`. More formally, it must match the regex:
|
||||
@@ -460,6 +520,91 @@ started.
|
||||
Once the cluster is up, you can grab the admin credentials from the master node
|
||||
at `/etc/kubernetes/admin.conf` and use that to talk to the cluster.
|
||||
|
||||
Note that this style of bootstrap has some relaxed security guarantees because
|
||||
it does not allow the root CA hash to be validated with
|
||||
`--discovery-token-ca-cert-hash` (since it's not generated when the nodes are
|
||||
provisioned). For details, see the [security model](#security-model).
|
||||
|
||||
## Security model
|
||||
The kubeadm discovery system has several options, each with security tradeoffs.
|
||||
The right method for your environment depends on how you provision nodes and the
|
||||
security expectations you have about your network and node lifecycles.
|
||||
|
||||
### Token-based discovery with CA pinning
|
||||
_This is the default mode in Kubernetes 1.8._ In this mode, kubeadm downloads
|
||||
the cluster configuration (including root CA) and validates it using the token
|
||||
as well as validating that the root CA public key matches the provided hash and
|
||||
that the API server certificate is valid under the root CA.
|
||||
|
||||
**Example `kubeadm join` command:**
|
||||
|
||||
- `kubeadm join --discovery-token abcdef.1234567890abcdef --discovery-token-ca-cert-hash sha256:1234..cdef 1.2.3.4:6443`
|
||||
|
||||
**Advantages:**
|
||||
|
||||
- Allows bootstrapping nodes to securely discover a root of trust for the
|
||||
master even if other worker nodes or the network are compromised.
|
||||
|
||||
- Convenient to execute manually since all of the information required fits
|
||||
into a single `kubeadm join` command that is easy to copy and paste.
|
||||
|
||||
**Disadvantages:**
|
||||
|
||||
- The CA hash is not normally known until the master has been provisioned,
|
||||
which can make it more difficult to build automated provisioning tools that
|
||||
use kubeadm.
|
||||
|
||||
### Token-based discovery without CA pinning
|
||||
_This was the default in Kubernetes 1.7 and earlier_, but comes with some
|
||||
important caveats. This mode relies only on the symmetric token to sign
|
||||
(HMAC-SHA256) the discovery information that establishes the root of trust for
|
||||
the master. It's still possible in Kubernetes 1.8 and above using the
|
||||
`--discovery-token-unsafe-skip-ca-verification` flag, but you should consider
|
||||
using one of the other modes if possible.
|
||||
|
||||
**Example `kubeadm join` command:**
|
||||
|
||||
- `kubeadm join --discovery-token abcdef.1234567890abcdef --discovery-token-unsafe-skip-ca-verification 1.2.3.4:6443`
|
||||
|
||||
**Advantages:**
|
||||
|
||||
- Still protects against many network-level attacks.
|
||||
|
||||
- The token can be generated ahead of time and shared with the master and
|
||||
worker nodes, which can then bootstrap in parallel without coordination. This
|
||||
allows it to be used in many provisioning scenarios.
|
||||
|
||||
**Disadvantages:**
|
||||
|
||||
- If an attacker is able to steal a bootstrap token via some vulnerability,
|
||||
they can use that token (along with network-level access) to impersonate the
|
||||
master to other bootstrapping nodes. This may or may not be an appropriate
|
||||
tradeoff in your environment.
|
||||
|
||||
### File or HTTPS-based discovery
|
||||
This provides an out-of-band way to establish a root of trust between the master
|
||||
and bootstrapping nodes. Consider using this mode if you are building automated provisioning
|
||||
using kubeadm.
|
||||
|
||||
**Example `kubeadm join` commands:**
|
||||
|
||||
- `kubeadm join --discovery-file path/to/file.conf` (local file)
|
||||
|
||||
- `kubeadm join --discovery-file https://url/file.conf` (remote HTTPS URL)
|
||||
|
||||
**Advantages:**
|
||||
- Allows bootstrapping nodes to securely discover a root of trust for the
|
||||
master even if other worker nodes or the network are compromised.
|
||||
|
||||
**Disadvantages:**
|
||||
- Requires that you have some way to carry the discovery information from
|
||||
the master to the bootstrapping nodes. This might be possible, for example,
|
||||
via your cloud provider or provisioning tool. The information in this file is
|
||||
not secret, but HTTPS or equivalent is required to ensure its integrity.
|
||||
|
||||
- Less convenient to use manually since the file is difficult to copy and paste
|
||||
between nodes.
|
||||
|
||||
## Use Kubeadm with other CRI runtimes
|
||||
|
||||
Since [Kubernetes 1.6 release](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG.md#node-components-1), Kubernetes container runtimes have been transferred to using CRI by default. Currently, the build-in container runtime is Docker which is enabled by build-in `dockershim` in `kubelet`.
|
||||
@@ -492,17 +637,17 @@ the following images are required for the cluster works will be automatically
|
||||
pulled by the kubelet if they don't exist locally while `kubeadm init` is initializing
|
||||
your master:
|
||||
|
||||
| Image Name | v1.6 release branch version | v1.7 release branch version
|
||||
| Image Name | v1.7 release branch version | v1.8 release branch version
|
||||
|---|---|---|
|
||||
| gcr.io/google_containers/kube-apiserver-${ARCH} | v1.6.x | v1.7.x
|
||||
| gcr.io/google_containers/kube-controller-manager-${ARCH} | v1.6.x | v1.7.x
|
||||
| gcr.io/google_containers/kube-scheduler-${ARCH} | v1.6.x | v1.7.x
|
||||
| gcr.io/google_containers/kube-proxy-${ARCH} | v1.6.x | v1.7.x
|
||||
| gcr.io/google_containers/kube-apiserver-${ARCH} | v1.7.x | v1.8.x
|
||||
| gcr.io/google_containers/kube-controller-manager-${ARCH} | v1.7.x | v1.8.x
|
||||
| gcr.io/google_containers/kube-scheduler-${ARCH} | v1.7.x | v1.8.x
|
||||
| gcr.io/google_containers/kube-proxy-${ARCH} | v1.7.x | v1.8.x
|
||||
| gcr.io/google_containers/etcd-${ARCH} | 3.0.17 | 3.0.17
|
||||
| gcr.io/google_containers/pause-${ARCH} | 3.0 | 3.0
|
||||
| gcr.io/google_containers/k8s-dns-sidecar-${ARCH} | 1.14.1 | 1.14.4
|
||||
| gcr.io/google_containers/k8s-dns-kube-dns-${ARCH} | 1.14.1 | 1.14.4
|
||||
| gcr.io/google_containers/k8s-dns-dnsmasq-nanny-${ARCH} | 1.14.1 | 1.14.4
|
||||
| gcr.io/google_containers/k8s-dns-sidecar-${ARCH} | 1.14.4 | 1.14.4
|
||||
| gcr.io/google_containers/k8s-dns-kube-dns-${ARCH} | 1.14.4 | 1.14.4
|
||||
| gcr.io/google_containers/k8s-dns-dnsmasq-nanny-${ARCH} | 1.14.4 | 1.14.4
|
||||
|
||||
Here `v1.7.x` means the "latest patch release of the v1.7 branch".
|
||||
|
||||
@@ -534,7 +679,7 @@ A breakdown of what/why:
|
||||
tells the kubelet where the API server is. This file also has the kubelet's
|
||||
credentials.
|
||||
* `--require-kubeconfig=true` the kubelet should fail fast if the kubeconfig file
|
||||
is not present. This makes the kubelet crashloop during the time the service is
|
||||
is not present. This makes the kubelet crash loop during the time the service is
|
||||
started until `kubeadm init` is run.
|
||||
* `--pod-manifest-path=/etc/kubernetes/manifests` specifies from where to read
|
||||
Static Pod manifests used for spinning up the control plane
|
||||
@@ -562,7 +707,7 @@ A breakdown of what/why:
|
||||
systemctl restart kubelet
|
||||
```
|
||||
|
||||
## Cloudprovider integrations (experimental)
|
||||
## Cloud provider integrations (experimental)
|
||||
|
||||
Enabling specific cloud providers is a common request. This currently requires
|
||||
manual configuration and is therefore not yet fully supported. If you wish to do
|
||||
@@ -573,7 +718,7 @@ installed on the host, for example for volume mounting/unmounting, install those
|
||||
packages.
|
||||
|
||||
Specify the `--cloud-provider` flag for the kubelet and set it to the cloud of your
|
||||
choice. If your cloudprovider requires a configuration file, create the file
|
||||
choice. If your cloud provider requires a configuration file, create the file
|
||||
`/etc/kubernetes/cloud-config` on every node. The exact format and content of
|
||||
that file depends on the requirements imposed by your cloud provider. If you use
|
||||
the `/etc/kubernetes/cloud-config` file, you must append it to the kubelet
|
||||
|
||||
Reference in New Issue
Block a user