update with master content resolving merge conflicts
This commit is contained in:
MAKOSCAFEE
@@ -4,92 +4,107 @@ reviewers:
|
||||
- erictune
|
||||
- mikedanese
|
||||
no_issue: true
|
||||
title: Setup
|
||||
title: Getting started
|
||||
main_menu: true
|
||||
weight: 30
|
||||
weight: 20
|
||||
content_template: templates/concept
|
||||
card:
|
||||
name: setup
|
||||
weight: 20
|
||||
anchors:
|
||||
- anchor: "#learning-environment"
|
||||
title: Learning environment
|
||||
- anchor: "#production-environment"
|
||||
title: Production environment
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
Use this page to find the type of solution that best fits your needs.
|
||||
This section covers different options to set up and run Kubernetes.
|
||||
|
||||
Deciding where to run Kubernetes depends on what resources you have available
|
||||
and how much flexibility you need. You can run Kubernetes almost anywhere,
|
||||
from your laptop to VMs on a cloud provider to a rack of bare metal servers.
|
||||
You can also set up a fully-managed cluster by running a single command or craft
|
||||
your own customized cluster on your bare metal servers.
|
||||
Different Kubernetes solutions meet different requirements: ease of maintenance, security, control, available resources, and expertise required to operate and manage a cluster.
|
||||
|
||||
You can deploy a Kubernetes cluster on a local machine, cloud, on-prem datacenter; or choose a managed Kubernetes cluster. You can also create custom solutions across a wide range of cloud providers, or bare metal environments.
|
||||
|
||||
More simply, you can create a Kubernetes cluster in learning and production environments.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture body %}}
|
||||
|
||||
## Local-machine Solutions
|
||||
## Learning environment
|
||||
|
||||
A local-machine solution is an easy way to get started with Kubernetes. You
|
||||
can create and test Kubernetes clusters without worrying about consuming cloud
|
||||
resources and quotas.
|
||||
If you're learning Kubernetes, use the Docker-based solutions: tools supported by the Kubernetes community, or tools in the ecosystem to set up a Kubernetes cluster on a local machine.
|
||||
|
||||
You should pick a local solution if you want to:
|
||||
{{< table caption="Local machine solutions table that lists the tools supported by the community and the ecosystem to deploy Kubernetes." >}}
|
||||
|
||||
* Try or start learning about Kubernetes
|
||||
* Develop and test clusters locally
|
||||
|
||||
Pick a [local-machine solution](/docs/setup/pick-right-solution/#local-machine-solutions).
|
||||
|
||||
## Hosted Solutions
|
||||
|
||||
Hosted solutions are a convenient way to create and maintain Kubernetes clusters. They
|
||||
manage and operate your clusters so you don’t have to.
|
||||
|
||||
You should pick a hosted solution if you:
|
||||
|
||||
* Want a fully-managed solution
|
||||
* Want to focus on developing your apps or services
|
||||
* Don’t have dedicated site reliability engineering (SRE) team but want high availability
|
||||
* Don't have resources to host and monitor your clusters
|
||||
|
||||
Pick a [hosted solution](/docs/setup/pick-right-solution/#hosted-solutions).
|
||||
|
||||
## Turnkey – Cloud Solutions
|
||||
|Community |Ecosystem |
|
||||
| ------------ | -------- |
|
||||
| [Minikube](/docs/setup/learning-environment/minikube/) | [CDK on LXD](https://www.ubuntu.com/kubernetes/docs/install-local) |
|
||||
| [Kubeadm-dind](https://github.com/kubernetes-sigs/kubeadm-dind-cluster) | [Docker Desktop](https://www.docker.com/products/docker-desktop)|
|
||||
| [Kubernetes IN Docker](https://github.com/kubernetes-sigs/kind) | [Minishift](https://docs.okd.io/latest/minishift/)|
|
||||
| | [MicroK8s](https://microk8s.io/)|
|
||||
| | [IBM Cloud Private-CE (Community Edition)](https://github.com/IBM/deploy-ibm-cloud-private) |
|
||||
| | [IBM Cloud Private-CE (Community Edition) on Linux Containers](https://github.com/HSBawa/icp-ce-on-linux-containers)|
|
||||
| | [k3s](https://k3s.io)|
|
||||
| | [Ubuntu on LXD](/docs/getting-started-guides/ubuntu/)|
|
||||
|
||||
|
||||
These solutions allow you to create Kubernetes clusters with only a few commands and
|
||||
are actively developed and have active community support. They can also be hosted on
|
||||
a range of Cloud IaaS providers, but they offer more freedom and flexibility in
|
||||
exchange for effort.
|
||||
## Production environment
|
||||
|
||||
You should pick a turnkey cloud solution if you:
|
||||
When evaluating a solution for a production environment, consider which aspects of operating a Kubernetes cluster (or _abstractions_) you want to manage yourself or offload to a provider.
|
||||
|
||||
* Want more control over your clusters than the hosted solutions allow
|
||||
* Want to take on more operations ownership
|
||||
Some possible abstractions of a Kubernetes cluster are {{< glossary_tooltip text="applications" term_id="applications" >}}, {{< glossary_tooltip text="data plane" term_id="data-plane" >}}, {{< glossary_tooltip text="control plane" term_id="control-plane" >}}, {{< glossary_tooltip text="cluster infrastructure" term_id="cluster-infrastructure" >}}, and {{< glossary_tooltip text="cluster operations" term_id="cluster-operations" >}}.
|
||||
|
||||
Pick a [turnkey cloud solution](/docs/setup/pick-right-solution/#turnkey-cloud-solutions)
|
||||
The following diagram lists the possible abstractions of a Kubernetes cluster and whether an abstraction is self-managed or managed by a provider.
|
||||
|
||||
## Turnkey – On-Premises Solutions
|
||||
Production environment solutions
|
||||
|
||||
These solutions allow you to create Kubernetes clusters on your internal, secure,
|
||||
cloud network with only a few commands.
|
||||
{{< table caption="Production environment solutions table lists the providers and the solutions." >}}
|
||||
The following production environment solutions table lists the providers and the solutions that they offer.
|
||||
|
||||
You should pick a on-prem turnkey cloud solution if you:
|
||||
|
||||
* Want to deploy clusters on your private cloud network
|
||||
* Have a dedicated SRE team
|
||||
* Have the resources to host and monitor your clusters
|
||||
|
||||
Pick an [on-prem turnkey cloud solution](/docs/setup/pick-right-solution/#on-premises-turnkey-cloud-solutions).
|
||||
|
||||
## Custom Solutions
|
||||
|
||||
Custom solutions give you the most freedom over your clusters but require the
|
||||
most expertise. These solutions range from bare-metal to cloud providers on
|
||||
different operating systems.
|
||||
|
||||
Pick a [custom solution](/docs/setup/pick-right-solution/#custom-solutions).
|
||||
|Providers | Managed | Turnkey cloud | On-prem datacenter | Custom (cloud) | Custom (On-premises VMs)| Custom (Bare Metal) |
|
||||
| --------- | ------ | ------ | ------ | ------ | ------ | ----- |
|
||||
| [Agile Stacks](https://www.agilestacks.com/products/kubernetes)| | ✔ | ✔ | | |
|
||||
| [Alibaba Cloud](https://www.alibabacloud.com/product/kubernetes)| | ✔ | | | |
|
||||
| [Amazon](https://aws.amazon.com) | [Amazon EKS](https://aws.amazon.com/eks/) |[Amazon EC2](https://aws.amazon.com/ec2/) | | | |
|
||||
| [AppsCode](https://appscode.com/products/pharmer/) | ✔ | | | | |
|
||||
| [APPUiO](https://appuio.ch/) | ✔ | ✔ | ✔ | | | |
|
||||
| [CenturyLink Cloud](https://www.ctl.io/) | | ✔ | | | |
|
||||
| [Cisco Container Platform](https://cisco.com/go/containers) | | | ✔ | | |
|
||||
| [Cloud Foundry Container Runtime (CFCR)](https://docs-cfcr.cfapps.io/) | | | | ✔ |✔ |
|
||||
| [CloudStack](https://cloudstack.apache.org/) | | | | | ✔|
|
||||
| [Canonical](https://www.ubuntu.com/kubernetes/docs/quickstart) | | ✔ | | ✔ |✔ | ✔
|
||||
| [Containership](https://containership.io/containership-platform) | ✔ |✔ | | | |
|
||||
| [Digital Rebar](https://provision.readthedocs.io/en/tip/README.html) | | | | | | ✔
|
||||
| [DigitalOcean](https://www.digitalocean.com/products/kubernetes/) | ✔ | | | | |
|
||||
| [Docker Enterprise](https://www.docker.com/products/docker-enterprise) | |✔ | ✔ | | | ✔
|
||||
| [Fedora (Multi Node)](https://kubernetes.io/docs/getting-started-guides/fedora/flannel_multi_node_cluster/) | | | | | ✔ | ✔
|
||||
| [Fedora (Single Node)](https://kubernetes.io/docs/getting-started-guides/fedora/fedora_manual_config/) | | | | | | ✔
|
||||
| [Gardner](https://gardener.cloud/) | |✔ | | ✔ | |
|
||||
| [Giant Swarm](https://giantswarm.io/) | ✔ | ✔ | ✔ | |
|
||||
| [Google](https://cloud.google.com/) | [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine/) | [Google Compute Engine (GCE)](https://cloud.google.com/compute/)|[GKE On-Prem](https://cloud.google.com/gke-on-prem/) | | | | | | | |
|
||||
| [IBM](https://www.ibm.com/in-en/cloud) | [IBM Cloud Kubernetes Service](https://cloud.ibm.com/kubernetes/catalog/cluster)| |[IBM Cloud Private](https://www.ibm.com/in-en/cloud/private) | |
|
||||
| [Kontena Pharos](https://www.kontena.io/pharos/) | |✔| ✔ | | |
|
||||
| [Kubermatic](https://www.loodse.com/) | ✔ | ✔ | ✔ | | |
|
||||
| [KubeSail](https://kubesail.com/) | ✔ | | | | |
|
||||
| [Kubespray](https://kubespray.io/#/) | | | |✔ | ✔ | ✔ |
|
||||
| [Kublr](https://kublr.com/) |✔ | ✔ |✔ |✔ |✔ |✔ |
|
||||
| [Microsoft Azure](https://azure.microsoft.com) | [Azure Kubernetes Service (AKS)](https://azure.microsoft.com/en-us/services/kubernetes-service/) | | | | |
|
||||
| [Mirantis Cloud Platform](https://www.mirantis.com/software/kubernetes/) | | | ✔ | | |
|
||||
| [Nirmata](https://www.nirmata.com/) | | ✔ | ✔ | | |
|
||||
| [Nutanix](https://www.nutanix.com/en) | [Nutanix Karbon](https://www.nutanix.com/products/karbon) | [Nutanix Karbon](https://www.nutanix.com/products/karbon) | | | [Nutanix AHV](https://www.nutanix.com/products/acropolis/virtualization) |
|
||||
| [OpenShift](https://www.openshift.com) |[OpenShift Dedicated](https://www.openshift.com/products/dedicated/) and [OpenShift Online](https://www.openshift.com/products/online/) | | [OpenShift Container Platform](https://www.openshift.com/products/container-platform/) | | [OpenShift Container Platform](https://www.openshift.com/products/container-platform/) |[OpenShift Container Platform](https://www.openshift.com/products/container-platform/)
|
||||
| [Oracle Cloud Infrastructure Container Engine for Kubernetes (OKE)](https://docs.cloud.oracle.com/iaas/Content/ContEng/Concepts/contengoverview.htm) | ✔ | ✔ | | | |
|
||||
| [oVirt](https://www.ovirt.org/) | | | | | ✔ |
|
||||
| [Pivotal](https://pivotal.io/) | | [Enterprise Pivotal Container Service (PKS)](https://pivotal.io/platform/pivotal-container-service) | [Enterprise Pivotal Container Service (PKS)](https://pivotal.io/platform/pivotal-container-service) | | |
|
||||
| [Platform9](https://platform9.com/) | ✔ | ✔ | ✔ | | ✔ |✔
|
||||
| [Rancher](https://rancher.com/) | | [Rancher 2.x](https://rancher.com/docs/rancher/v2.x/en/) | | [Rancher Kubernetes Engine (RKE)](https://rancher.com/docs/rke/latest/en/) | | [k3s](https://k3s.io/)
|
||||
| [StackPoint](https://stackpoint.io/) | ✔ | ✔ | | | |
|
||||
| [Supergiant](https://supergiant.io/) | |✔ | | | |
|
||||
| [SUSE](https://www.suse.com/) | | ✔ | | | |
|
||||
| [SysEleven](https://www.syseleven.io/) | ✔ | | | | |
|
||||
| [VEXXHOST](https://vexxhost.com/) | ✔ | ✔ | | | |
|
||||
| [VMware](https://cloud.vmware.com/) | [VMware Cloud PKS](https://cloud.vmware.com/vmware-cloud-pks) |[VMware Enterprise PKS](https://cloud.vmware.com/vmware-enterprise-pks) | [VMware Enterprise PKS](https://cloud.vmware.com/vmware-enterprise-pks) | [VMware Essential PKS](https://cloud.vmware.com/vmware-essential-pks) | |[VMware Essential PKS](https://cloud.vmware.com/vmware-essential-pks)
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture whatsnext %}}
|
||||
Go to [Picking the Right Solution](/docs/setup/pick-right-solution/) for a complete
|
||||
list of solutions.
|
||||
{{% /capture %}}
|
||||
|
||||
@@ -0,0 +1,4 @@
|
||||
---
|
||||
title: Best practices
|
||||
weight: 40
|
||||
---
|
||||
+12
-11
@@ -1,8 +1,9 @@
|
||||
---
|
||||
title: PKI Certificates and Requirements
|
||||
title: PKI certificates and requirements
|
||||
reviewers:
|
||||
- sig-cluster-lifecycle
|
||||
- sig-cluster-lifecycle
|
||||
content_template: templates/concept
|
||||
weight: 40
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
@@ -45,7 +46,7 @@ If you don't want kubeadm to generate the required certificates, you can create
|
||||
|
||||
### Single root CA
|
||||
|
||||
You can create a single root CA, controlled by an administrator. This root CA can then create multiple intermediate CAs, and delegate all further creation to Kubernetes itself.
|
||||
You can create a single root CA, controlled by an administrator. This root CA can then create multiple intermediate CAs, and delegate all further creation to Kubernetes itself.
|
||||
|
||||
Required CAs:
|
||||
|
||||
@@ -57,7 +58,7 @@ Required CAs:
|
||||
|
||||
### All certificates
|
||||
|
||||
If you don't wish to copy these private keys to your API servers, you can generate all certificates yourself.
|
||||
If you don't wish to copy these private keys to your API servers, you can generate all certificates yourself.
|
||||
|
||||
Required certificates:
|
||||
|
||||
@@ -84,17 +85,17 @@ where `kind` maps to one or more of the [x509 key usage][usage] types:
|
||||
|
||||
Certificates should be placed in a recommended path (as used by [kubeadm][kubeadm]). Paths should be specified using the given argument regardless of location.
|
||||
|
||||
| Default CN | recommend key path | recommended cert path | command | key argument | cert argument |
|
||||
| Default CN | recommended key path | recommended cert path | command | key argument | cert argument |
|
||||
|------------------------------|------------------------------|-----------------------------|----------------|------------------------------|-------------------------------------------|
|
||||
| etcd-ca | | etcd/ca.crt | kube-apiserver | | --etcd-cafile |
|
||||
| etcd-ca | etcd/ca.key | etcd/ca.crt | kube-apiserver | | --etcd-cafile |
|
||||
| etcd-client | apiserver-etcd-client.key | apiserver-etcd-client.crt | kube-apiserver | --etcd-keyfile | --etcd-certfile |
|
||||
| kubernetes-ca | | ca.crt | kube-apiserver | | --client-ca-file |
|
||||
| kubernetes-ca | ca.key | ca.crt | kube-apiserver | | --client-ca-file |
|
||||
| kube-apiserver | apiserver.key | apiserver.crt | kube-apiserver | --tls-private-key-file | --tls-cert-file |
|
||||
| apiserver-kubelet-client | | apiserver-kubelet-client.crt| kube-apiserver | | --kubelet-client-certificate |
|
||||
| front-proxy-ca | | front-proxy-ca.crt | kube-apiserver | | --requestheader-client-ca-file |
|
||||
| apiserver-kubelet-client | apiserver-kubelet-client.key | apiserver-kubelet-client.crt| kube-apiserver | | --kubelet-client-certificate |
|
||||
| front-proxy-ca | front-proxy-ca.key | front-proxy-ca.crt | kube-apiserver | | --requestheader-client-ca-file |
|
||||
| front-proxy-client | front-proxy-client.key | front-proxy-client.crt | kube-apiserver | --proxy-client-key-file | --proxy-client-cert-file |
|
||||
| | | | | | |
|
||||
| etcd-ca | | etcd/ca.crt | etcd | | --trusted-ca-file, --peer-trusted-ca-file |
|
||||
| etcd-ca | etcd/ca.key | etcd/ca.crt | etcd | | --trusted-ca-file, --peer-trusted-ca-file |
|
||||
| kube-etcd | etcd/server.key | etcd/server.crt | etcd | --key-file | --cert-file |
|
||||
| kube-etcd-peer | etcd/peer.key | etcd/peer.crt | etcd | --peer-key-file | --peer-cert-file |
|
||||
| etcd-ca | | etcd/ca.crt | etcdctl[2] | | --cacert |
|
||||
@@ -104,7 +105,7 @@ Certificates should be placed in a recommended path (as used by [kubeadm][kubead
|
||||
|
||||
## Configure certificates for user accounts
|
||||
|
||||
You must manually configure these administrator account and service accounts:
|
||||
You must manually configure these administrator account and service accounts:
|
||||
|
||||
| filename | credential name | Default CN | O (in Subject) |
|
||||
|-------------------------|----------------------------|--------------------------------|----------------|
|
||||
+2
-2
@@ -2,8 +2,8 @@
|
||||
reviewers:
|
||||
- davidopp
|
||||
- lavalamp
|
||||
title: Building Large Clusters
|
||||
weight: 80
|
||||
title: Building large clusters
|
||||
weight: 20
|
||||
---
|
||||
|
||||
## Support
|
||||
+4
-4
@@ -3,8 +3,8 @@ reviewers:
|
||||
- jlowdermilk
|
||||
- justinsb
|
||||
- quinton-hoole
|
||||
title: Running in Multiple Zones
|
||||
weight: 90
|
||||
title: Running in multiple zones
|
||||
weight: 10
|
||||
content_template: templates/concept
|
||||
---
|
||||
|
||||
@@ -191,8 +191,8 @@ Create a volume using the dynamic volume creation (only PersistentVolumes are su
|
||||
```json
|
||||
kubectl apply -f - <<EOF
|
||||
{
|
||||
"kind": "PersistentVolumeClaim",
|
||||
"apiVersion": "v1",
|
||||
"kind": "PersistentVolumeClaim",
|
||||
"metadata": {
|
||||
"name": "claim1",
|
||||
"annotations": {
|
||||
@@ -241,8 +241,8 @@ this means that this pod can only be created in the same zone as the volume:
|
||||
|
||||
```yaml
|
||||
kubectl apply -f - <<EOF
|
||||
kind: Pod
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: mypod
|
||||
spec:
|
||||
+2
-1
@@ -1,7 +1,8 @@
|
||||
---
|
||||
reviewers:
|
||||
- Random-Liu
|
||||
title: Validate Node Setup
|
||||
title: Validate node setup
|
||||
weight: 30
|
||||
---
|
||||
|
||||
{{< toc >}}
|
||||
@@ -1,4 +0,0 @@
|
||||
---
|
||||
title: Custom Cloud Solutions
|
||||
weight: 50
|
||||
---
|
||||
@@ -1,5 +0,0 @@
|
||||
---
|
||||
title: "Bootstrapping Clusters with kubeadm"
|
||||
weight: 30
|
||||
---
|
||||
|
||||
@@ -0,0 +1,4 @@
|
||||
---
|
||||
title: Learning environment
|
||||
weight: 20
|
||||
---
|
||||
+198
-167
@@ -3,13 +3,13 @@ reviewers:
|
||||
- dlorenc
|
||||
- balopat
|
||||
- aaron-prindle
|
||||
title: Running Kubernetes Locally via Minikube
|
||||
title: Installing Kubernetes with Minikube
|
||||
content_template: templates/concept
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
Minikube is a tool that makes it easy to run Kubernetes locally. Minikube runs a single-node Kubernetes cluster inside a VM on your laptop for users looking to try out Kubernetes or develop with it day-to-day.
|
||||
Minikube is a tool that makes it easy to run Kubernetes locally. Minikube runs a single-node Kubernetes cluster inside a Virtual Machine (VM) on your laptop for users looking to try out Kubernetes or develop with it day-to-day.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
@@ -17,14 +17,15 @@ Minikube is a tool that makes it easy to run Kubernetes locally. Minikube runs a
|
||||
|
||||
## Minikube Features
|
||||
|
||||
* Minikube supports Kubernetes features such as:
|
||||
* DNS
|
||||
* NodePorts
|
||||
* ConfigMaps and Secrets
|
||||
* Dashboards
|
||||
* Container Runtime: Docker, [rkt](https://github.com/rkt/rkt), [CRI-O](https://github.com/kubernetes-incubator/cri-o) and [containerd](https://github.com/containerd/containerd)
|
||||
* Enabling CNI (Container Network Interface)
|
||||
* Ingress
|
||||
Minikube supports the following Kubernetes features:
|
||||
|
||||
* DNS
|
||||
* NodePorts
|
||||
* ConfigMaps and Secrets
|
||||
* Dashboards
|
||||
* Container Runtime: Docker, [rkt](https://github.com/rkt/rkt), [CRI-O](https://github.com/kubernetes-incubator/cri-o), and [containerd](https://github.com/containerd/containerd)
|
||||
* Enabling CNI (Container Network Interface)
|
||||
* Ingress
|
||||
|
||||
## Installation
|
||||
|
||||
@@ -32,120 +33,184 @@ See [Installing Minikube](/docs/tasks/tools/install-minikube/).
|
||||
|
||||
## Quickstart
|
||||
|
||||
Here's a brief demo of Minikube usage.
|
||||
If you want to change the VM driver add the appropriate `--vm-driver=xxx` flag to `minikube start`. Minikube supports
|
||||
the following drivers:
|
||||
This brief demo guides you on how to start, use, and delete Minikube locally. Follow the steps given below to start and explore Minikube.
|
||||
|
||||
1. Start Minikube and create a cluster:
|
||||
```shell
|
||||
minikube start
|
||||
```
|
||||
The output is similar to this:
|
||||
|
||||
```
|
||||
Starting local Kubernetes cluster...
|
||||
Running pre-create checks...
|
||||
Creating machine...
|
||||
Starting local Kubernetes cluster...
|
||||
```
|
||||
For more information on starting your cluster on a specific Kubernetes version, VM, or container runtime, see [Starting a Cluster](#starting-a-cluster).
|
||||
|
||||
2. Now, you can interact with your cluster using kubectl. For more information, see [Interacting with Your Cluster](#interacting-with-your-cluster).
|
||||
|
||||
Let’s create a Kubernetes Deployment using an existing image named `echoserver`, which is a simple HTTP server and expose it on port 8080 using `--port`.
|
||||
```shell
|
||||
kubectl run hello-minikube --image=k8s.gcr.io/echoserver:1.10 --port=8080
|
||||
```
|
||||
The output is similar to this:
|
||||
```
|
||||
deployment.apps/hello-minikube created
|
||||
```
|
||||
3. To access the `hello-minikube` Deployment, expose it as a Service:
|
||||
```shell
|
||||
kubectl expose deployment hello-minikube --type=NodePort
|
||||
```
|
||||
The option `--type=NodePort` specifies the type of the Service.
|
||||
|
||||
The output is similar to this:
|
||||
```
|
||||
service/hello-minikube exposed
|
||||
```
|
||||
4. The `hello-minikube` Pod is now launched but you have to wait until the Pod is up before accessing it via the exposed Service.
|
||||
|
||||
Check if the Pod is up and running:
|
||||
```shell
|
||||
kubectl get pod
|
||||
```
|
||||
If the output shows the `STATUS` as `ContainerCreating`, the Pod is still being created:
|
||||
```
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
hello-minikube-3383150820-vctvh 0/1 ContainerCreating 0 3s
|
||||
```
|
||||
If the output shows the `STATUS` as `Running`, the Pod is now up and running:
|
||||
```
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
hello-minikube-3383150820-vctvh 1/1 Running 0 13s
|
||||
```
|
||||
5. Get the URL of the exposed Service to view the Service details:
|
||||
```shell
|
||||
minikube service hello-minikube --url
|
||||
```
|
||||
6. To view the details of your local cluster, copy and paste the URL you got as the output, on your browser.
|
||||
|
||||
The output is similar to this:
|
||||
```
|
||||
Hostname: hello-minikube-7c77b68cff-8wdzq
|
||||
|
||||
Pod Information:
|
||||
-no pod information available-
|
||||
|
||||
Server values:
|
||||
server_version=nginx: 1.13.3 - lua: 10008
|
||||
|
||||
Request Information:
|
||||
client_address=172.17.0.1
|
||||
method=GET
|
||||
real path=/
|
||||
query=
|
||||
request_version=1.1
|
||||
request_scheme=http
|
||||
request_uri=http://192.168.99.100:8080/
|
||||
|
||||
Request Headers:
|
||||
accept=*/*
|
||||
host=192.168.99.100:30674
|
||||
user-agent=curl/7.47.0
|
||||
|
||||
Request Body:
|
||||
-no body in request-
|
||||
```
|
||||
If you no longer want the Service and cluster to run, you can delete them.
|
||||
7. Delete the `hello-minikube` Service:
|
||||
```shell
|
||||
kubectl delete services hello-minikube
|
||||
```
|
||||
The output is similar to this:
|
||||
```
|
||||
service "hello-minikube" deleted
|
||||
```
|
||||
8. Delete the `hello-minikube` Deployment:
|
||||
```shell
|
||||
kubectl delete deployment hello-minikube
|
||||
```
|
||||
The output is similar to this:
|
||||
```
|
||||
deployment.extensions "hello-minikube" deleted
|
||||
```
|
||||
9. Stop the local Minikube cluster:
|
||||
```shell
|
||||
minikube stop
|
||||
```
|
||||
The output is similar to this:
|
||||
```
|
||||
Stopping "minikube"...
|
||||
"minikube" stopped.
|
||||
```
|
||||
For more information, see [Stopping a Cluster](#stopping-a-cluster).
|
||||
10. Delete the local Minikube cluster:
|
||||
```shell
|
||||
minikube delete
|
||||
```
|
||||
The output is similar to this:
|
||||
```
|
||||
Deleting "minikube" ...
|
||||
The "minikube" cluster has been deleted.
|
||||
```
|
||||
For more information, see [Deleting a cluster](#deleting-a-cluster).
|
||||
|
||||
## Managing your Cluster
|
||||
|
||||
### Starting a Cluster
|
||||
|
||||
The `minikube start` command can be used to start your cluster.
|
||||
This command creates and configures a Virtual Machine that runs a single-node Kubernetes cluster.
|
||||
This command also configures your [kubectl](/docs/user-guide/kubectl-overview/) installation to communicate with this cluster.
|
||||
|
||||
{{< note >}}
|
||||
If you are behind a web proxy, you need to pass this information to the `minikube start` command:
|
||||
|
||||
```shell
|
||||
https_proxy=<my proxy> minikube start --docker-env http_proxy=<my proxy> --docker-env https_proxy=<my proxy> --docker-env no_proxy=192.168.99.0/24
|
||||
```
|
||||
Unfortunately, setting the environment variables alone does not work.
|
||||
|
||||
Minikube also creates a "minikube" context, and sets it to default in kubectl.
|
||||
To switch back to this context, run this command: `kubectl config use-context minikube`.
|
||||
{{< /note >}}
|
||||
|
||||
#### Specifying the Kubernetes version
|
||||
|
||||
You can specify the version of Kubernetes for Minikube to use by
|
||||
adding the `--kubernetes-version` string to the `minikube start` command. For
|
||||
example, to run version {{< param "fullversion" >}}, you would run the following:
|
||||
|
||||
```
|
||||
minikube start --kubernetes-version {{< param "fullversion" >}}
|
||||
```
|
||||
#### Specifying the VM driver
|
||||
You can change the VM driver by adding the `--vm-driver=<enter_driver_name>` flag to `minikube start`.
|
||||
For example the command would be.
|
||||
```shell
|
||||
minikube start --vm-driver=<driver_name>
|
||||
```
|
||||
Minikube supports the following drivers:
|
||||
{{< note >}}
|
||||
See [DRIVERS](https://git.k8s.io/minikube/docs/drivers.md) for details on supported drivers and how to install
|
||||
plugins.
|
||||
{{< /note >}}
|
||||
|
||||
* virtualbox
|
||||
* vmwarefusion
|
||||
* kvm2 ([driver installation](https://git.k8s.io/minikube/docs/drivers.md#kvm2-driver))
|
||||
* kvm ([driver installation](https://git.k8s.io/minikube/docs/drivers.md#kvm-driver))
|
||||
* hyperkit ([driver installation](https://git.k8s.io/minikube/docs/drivers.md#hyperkit-driver))
|
||||
* xhyve ([driver installation](https://git.k8s.io/minikube/docs/drivers.md#xhyve-driver)) (deprecated)
|
||||
* hyperv ([driver installation](https://github.com/kubernetes/minikube/blob/master/docs/drivers.md#hyperv-driver))
|
||||
Note that the IP below is dynamic and can change. It can be retrieved with `minikube ip`.
|
||||
* vmware ([driver installation](https://github.com/kubernetes/minikube/blob/master/docs/drivers.md#vmware-unified-driver)) (VMware unified driver)
|
||||
* none (Runs the Kubernetes components on the host and not in a VM. Using this driver requires Docker ([docker install](https://docs.docker.com/install/linux/docker-ce/ubuntu/)) and a Linux environment)
|
||||
|
||||
```shell
|
||||
minikube start
|
||||
```
|
||||
```
|
||||
Starting local Kubernetes cluster...
|
||||
Running pre-create checks...
|
||||
Creating machine...
|
||||
Starting local Kubernetes cluster...
|
||||
```
|
||||
```shell
|
||||
kubectl run hello-minikube --image=k8s.gcr.io/echoserver:1.10 --port=8080
|
||||
```
|
||||
```
|
||||
deployment.apps/hello-minikube created
|
||||
```
|
||||
|
||||
```shell
|
||||
kubectl expose deployment hello-minikube --type=NodePort
|
||||
```
|
||||
```
|
||||
service/hello-minikube exposed
|
||||
```
|
||||
```
|
||||
# We have now launched an echoserver pod but we have to wait until the pod is up before curling/accessing it
|
||||
# via the exposed service.
|
||||
# To check whether the pod is up and running we can use the following:
|
||||
kubectl get pod
|
||||
```
|
||||
```
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
hello-minikube-3383150820-vctvh 0/1 ContainerCreating 0 3s
|
||||
```
|
||||
```
|
||||
# We can see that the pod is still being created from the ContainerCreating status
|
||||
kubectl get pod
|
||||
```
|
||||
```
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
hello-minikube-3383150820-vctvh 1/1 Running 0 13s
|
||||
```
|
||||
```
|
||||
# We can see that the pod is now Running and we will now be able to curl it:
|
||||
curl $(minikube service hello-minikube --url)
|
||||
```
|
||||
```
|
||||
|
||||
Hostname: hello-minikube-7c77b68cff-8wdzq
|
||||
|
||||
Pod Information:
|
||||
-no pod information available-
|
||||
|
||||
Server values:
|
||||
server_version=nginx: 1.13.3 - lua: 10008
|
||||
|
||||
Request Information:
|
||||
client_address=172.17.0.1
|
||||
method=GET
|
||||
real path=/
|
||||
query=
|
||||
request_version=1.1
|
||||
request_scheme=http
|
||||
request_uri=http://192.168.99.100:8080/
|
||||
|
||||
Request Headers:
|
||||
accept=*/*
|
||||
host=192.168.99.100:30674
|
||||
user-agent=curl/7.47.0
|
||||
|
||||
Request Body:
|
||||
-no body in request-
|
||||
```
|
||||
|
||||
```shell
|
||||
kubectl delete services hello-minikube
|
||||
```
|
||||
```
|
||||
service "hello-minikube" deleted
|
||||
```
|
||||
|
||||
```shell
|
||||
kubectl delete deployment hello-minikube
|
||||
```
|
||||
```
|
||||
deployment.extensions "hello-minikube" deleted
|
||||
```
|
||||
|
||||
```shell
|
||||
minikube stop
|
||||
```
|
||||
```
|
||||
Stopping local Kubernetes cluster...
|
||||
Stopping "minikube"...
|
||||
```
|
||||
|
||||
### Alternative Container Runtimes
|
||||
|
||||
#### containerd
|
||||
|
||||
#### Starting a cluster on alternative container runtimes
|
||||
You can start Minikube on the following container runtimes.
|
||||
{{< tabs name="container_runtimes" >}}
|
||||
{{% tab name="containerd" %}}
|
||||
To use [containerd](https://github.com/containerd/containerd) as the container runtime, run:
|
||||
|
||||
```bash
|
||||
minikube start \
|
||||
--network-plugin=cni \
|
||||
@@ -165,11 +230,9 @@ minikube start \
|
||||
--extra-config=kubelet.image-service-endpoint=unix:///run/containerd/containerd.sock \
|
||||
--bootstrapper=kubeadm
|
||||
```
|
||||
|
||||
#### CRI-O
|
||||
|
||||
{{% /tab %}}
|
||||
{{% tab name="CRI-O" %}}
|
||||
To use [CRI-O](https://github.com/kubernetes-incubator/cri-o) as the container runtime, run:
|
||||
|
||||
```bash
|
||||
minikube start \
|
||||
--network-plugin=cni \
|
||||
@@ -177,7 +240,6 @@ minikube start \
|
||||
--container-runtime=cri-o \
|
||||
--bootstrapper=kubeadm
|
||||
```
|
||||
|
||||
Or you can use the extended version:
|
||||
|
||||
```bash
|
||||
@@ -189,48 +251,47 @@ minikube start \
|
||||
--extra-config=kubelet.image-service-endpoint=/var/run/crio.sock \
|
||||
--bootstrapper=kubeadm
|
||||
```
|
||||
|
||||
#### rkt container engine
|
||||
|
||||
{{% /tab %}}
|
||||
{{% tab name="rkt container engine" %}}
|
||||
To use [rkt](https://github.com/rkt/rkt) as the container runtime run:
|
||||
|
||||
```shell
|
||||
minikube start \
|
||||
--network-plugin=cni \
|
||||
--enable-default-cni \
|
||||
--container-runtime=rkt
|
||||
```
|
||||
|
||||
This will use an alternative minikube ISO image containing both rkt, and Docker, and enable CNI networking.
|
||||
{{% /tab %}}
|
||||
{{< /tabs >}}
|
||||
|
||||
### Driver plugins
|
||||
#### Use local images by re-using the Docker daemon
|
||||
|
||||
See [DRIVERS](https://git.k8s.io/minikube/docs/drivers.md) for details on supported drivers and how to install
|
||||
plugins, if required.
|
||||
When using a single VM for Kubernetes, it's useful to reuse Minikube's built-in Docker daemon. Reusing the built-in daemon means you don't have to build a Docker registry on your host machine and push the image into it. Instead, you can build inside the same Docker daemon as Minikube, which speeds up local experiments.
|
||||
|
||||
### Use local images by re-using the Docker daemon
|
||||
{{< note >}}
|
||||
Be sure to tag your Docker image with something other than latest and use that tag to pull the image. Because `:latest` is the default value, with a corresponding default image pull policy of `Always`, an image pull error (`ErrImagePull`) eventually results if you do not have the Docker image in the default Docker registry (usually DockerHub).
|
||||
{{< /note >}}
|
||||
|
||||
When using a single VM of Kubernetes, it's really handy to reuse the Minikube's built-in Docker daemon; as this means you don't have to build a docker registry on your host machine and push the image into it - you can just build inside the same docker daemon as minikube which speeds up local experiments. Just make sure you tag your Docker image with something other than 'latest' and use that tag while you pull the image. Otherwise, if you do not specify version of your image, it will be assumed as `:latest`, with pull image policy of `Always` correspondingly, which may eventually result in `ErrImagePull` as you may not have any versions of your Docker image out there in the default docker registry (usually DockerHub) yet.
|
||||
|
||||
To be able to work with the docker daemon on your mac/linux host use the `docker-env command` in your shell:
|
||||
To work with the Docker daemon on your Mac/Linux host, use the `docker-env command` in your shell:
|
||||
|
||||
```shell
|
||||
eval $(minikube docker-env)
|
||||
```
|
||||
|
||||
You should now be able to use docker on the command line on your host mac/linux machine talking to the docker daemon inside the minikube VM:
|
||||
You can now use Docker at the command line of your host Mac/Linux machine to communicate with the Docker daemon inside the Minikube VM:
|
||||
|
||||
```shell
|
||||
docker ps
|
||||
```
|
||||
|
||||
On Centos 7, docker may report the following error:
|
||||
{{< note >}}
|
||||
On Centos 7, Docker may report the following error:
|
||||
|
||||
```shell
|
||||
```
|
||||
Could not read CA certificate "/etc/docker/ca.pem": open /etc/docker/ca.pem: no such file or directory
|
||||
```
|
||||
|
||||
The fix is to update /etc/sysconfig/docker to ensure that Minikube's environment changes are respected:
|
||||
You can fix this by updating /etc/sysconfig/docker to ensure that Minikube's environment changes are respected:
|
||||
|
||||
```shell
|
||||
< DOCKER_CERT_PATH=/etc/docker
|
||||
@@ -239,37 +300,7 @@ The fix is to update /etc/sysconfig/docker to ensure that Minikube's environment
|
||||
> DOCKER_CERT_PATH=/etc/docker
|
||||
> fi
|
||||
```
|
||||
|
||||
Remember to turn off the imagePullPolicy:Always, otherwise Kubernetes won't use images you built locally.
|
||||
|
||||
## Managing your Cluster
|
||||
|
||||
### Starting a Cluster
|
||||
|
||||
The `minikube start` command can be used to start your cluster.
|
||||
This command creates and configures a Virtual Machine that runs a single-node Kubernetes cluster.
|
||||
This command also configures your [kubectl](/docs/user-guide/kubectl-overview/) installation to communicate with this cluster.
|
||||
|
||||
If you are behind a web proxy, you will need to pass this information to the `minikube start` command:
|
||||
|
||||
```shell
|
||||
https_proxy=<my proxy> minikube start --docker-env http_proxy=<my proxy> --docker-env https_proxy=<my proxy> --docker-env no_proxy=192.168.99.0/24
|
||||
```
|
||||
|
||||
Unfortunately just setting the environment variables will not work.
|
||||
|
||||
Minikube will also create a "minikube" context, and set it to default in kubectl.
|
||||
To switch back to this context later, run this command: `kubectl config use-context minikube`.
|
||||
|
||||
#### Specifying the Kubernetes version
|
||||
|
||||
You can specify the specific version of Kubernetes for Minikube to use by
|
||||
adding the `--kubernetes-version` string to the `minikube start` command. For
|
||||
example, to run version `v1.7.3`, you would run the following:
|
||||
|
||||
```
|
||||
minikube start --kubernetes-version v1.7.3
|
||||
```
|
||||
{{< /note >}}
|
||||
|
||||
### Configuring Kubernetes
|
||||
|
||||
@@ -331,7 +362,7 @@ minikube dashboard
|
||||
|
||||
### Services
|
||||
|
||||
To access a service exposed via a node port, run this command in a shell after starting Minikube to get the address:
|
||||
To access a Service exposed via a node port, run this command in a shell after starting Minikube to get the address:
|
||||
|
||||
```shell
|
||||
minikube service [-n NAMESPACE] [--url] NAME
|
||||
@@ -1,314 +0,0 @@
|
||||
---
|
||||
reviewers:
|
||||
- brendandburns
|
||||
- erictune
|
||||
- mikedanese
|
||||
title: Picking the Right Solution
|
||||
weight: 10
|
||||
content_template: templates/concept
|
||||
card:
|
||||
name: setup
|
||||
weight: 20
|
||||
anchors:
|
||||
- anchor: "#hosted-solutions"
|
||||
title: Hosted Solutions
|
||||
- anchor: "#turnkey-cloud-solutions"
|
||||
title: Turnkey Cloud Solutions
|
||||
- anchor: "#on-premises-turnkey-cloud-solutions"
|
||||
title: On-Premises Solutions
|
||||
- anchor: "#custom-solutions"
|
||||
title: Custom Solutions
|
||||
- anchor: "#local-machine-solutions"
|
||||
title: Local Machine
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
Kubernetes can run on various platforms: from your laptop, to VMs on a cloud provider, to a rack of
|
||||
bare metal servers. The effort required to set up a cluster varies from running a single command to
|
||||
crafting your own customized cluster. Use this guide to choose a solution that fits your needs.
|
||||
|
||||
If you just want to "kick the tires" on Kubernetes, use the [local Docker-based solutions](#local-machine-solutions).
|
||||
|
||||
When you are ready to scale up to more machines and higher availability, a [hosted solution](#hosted-solutions) is the easiest to create and maintain.
|
||||
|
||||
[Turnkey cloud solutions](#turnkey-cloud-solutions) require only a few commands to create
|
||||
and cover a wide range of cloud providers. [On-Premises turnkey cloud solutions](#on-premises-turnkey-cloud-solutions) have the simplicity of the turnkey cloud solution combined with the security of your own private network.
|
||||
|
||||
If you already have a way to configure hosting resources, use [kubeadm](/docs/setup/independent/create-cluster-kubeadm/) to easily bring up a cluster with a single command per machine.
|
||||
|
||||
[Custom solutions](#custom-solutions) vary from step-by-step instructions to general advice for setting up
|
||||
a Kubernetes cluster from scratch.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture body %}}
|
||||
|
||||
## Local-machine Solutions
|
||||
|
||||
### Community Supported Tools
|
||||
|
||||
* [Minikube](/docs/setup/minikube/) is a method for creating a local, single-node Kubernetes cluster for development and testing. Setup is completely automated and doesn't require a cloud provider account.
|
||||
|
||||
* [Kubeadm-dind](https://github.com/kubernetes-sigs/kubeadm-dind-cluster) is a multi-node (while minikube is single-node) Kubernetes cluster which only requires a docker daemon. It uses docker-in-docker technique to spawn the Kubernetes cluster.
|
||||
|
||||
* [Kubernetes IN Docker](https://github.com/kubernetes-sigs/kind) is a tool for running local Kubernetes clusters using Docker container "nodes". It is primarily designed for testing Kubernetes 1.11+. You can use it to create multi-node or multi-control-plane Kubernetes clusters.
|
||||
|
||||
### Ecosystem Tools
|
||||
|
||||
* [Docker Desktop](https://www.docker.com/products/docker-desktop) is an
|
||||
easy-to-install application for your Mac or Windows environment that enables you to
|
||||
start coding and deploying in containers in minutes on a single-node Kubernetes
|
||||
cluster.
|
||||
|
||||
* [Minishift](https://docs.okd.io/latest/minishift/) installs the community version of the Kubernetes enterprise platform OpenShift for local development & testing. It offers an all-in-one VM (`minishift start`) for Windows, macOS, and Linux. The container start is based on `oc cluster up` (Linux only). You can also install [the included add-ons](https://github.com/minishift/minishift-addons/tree/master/add-ons).
|
||||
|
||||
* [MicroK8s](https://microk8s.io/) provides a single command installation of the latest Kubernetes release on a local machine for development and testing. Setup is quick, fast (~30 sec) and supports many plugins including Istio with a single command.
|
||||
|
||||
* [IBM Cloud Private-CE (Community Edition)](https://github.com/IBM/deploy-ibm-cloud-private) can use VirtualBox on your machine to deploy Kubernetes to one or more VMs for development and test scenarios. Scales to full multi-node cluster.
|
||||
|
||||
* [IBM Cloud Private-CE (Community Edition) on Linux Containers](https://github.com/HSBawa/icp-ce-on-linux-containers) is a Terraform/Packer/BASH based Infrastructure as Code (IaC) scripts to create a seven node (1 Boot, 1 Master, 1 Management, 1 Proxy and 3 Workers) LXD cluster on Linux Host.
|
||||
|
||||
* [Ubuntu on LXD](/docs/getting-started-guides/ubuntu/local/) supports a nine-instance deployment on localhost.
|
||||
|
||||
## Hosted Solutions
|
||||
|
||||
* [AppsCode.com](https://appscode.com/products/cloud-deployment/) provides managed Kubernetes clusters for various public clouds, including AWS and Google Cloud Platform.
|
||||
|
||||
* [APPUiO](https://appuio.ch) runs an OpenShift public cloud platform, supporting any Kubernetes workload. Additionally APPUiO offers Private Managed OpenShift Clusters, running on any public or private cloud.
|
||||
|
||||
* [Amazon Elastic Container Service for Kubernetes](https://aws.amazon.com/eks/) offers managed Kubernetes service.
|
||||
|
||||
* [Azure Kubernetes Service](https://azure.microsoft.com/services/container-service/) offers managed Kubernetes clusters.
|
||||
|
||||
* [Containership Kubernetes Engine (CKE)](https://containership.io/containership-platform) intuitive Kubernetes cluster provisioning and management on GCP, Azure, AWS, Packet, and DigitalOcean. Seamless version upgrades, autoscaling, metrics, workload creation, and more.
|
||||
|
||||
* [DigitalOcean Kubernetes](https://www.digitalocean.com/products/kubernetes/) offers managed Kubernetes service.
|
||||
|
||||
* [Giant Swarm](https://giantswarm.io/product/) offers managed Kubernetes clusters in their own datacenter, on-premises, or on public clouds.
|
||||
|
||||
* [Google Kubernetes Engine](https://cloud.google.com/kubernetes-engine/) offers managed Kubernetes clusters.
|
||||
|
||||
* [IBM Cloud Kubernetes Service](https://cloud.ibm.com/docs/containers?topic=containers-container_index#container_index) offers managed Kubernetes clusters with isolation choice, operational tools, integrated security insight into images and containers, and integration with Watson, IoT, and data.
|
||||
|
||||
* [Kubermatic](https://www.loodse.com) provides managed Kubernetes clusters for various public clouds, including AWS and Digital Ocean, as well as on-premises with OpenStack integration.
|
||||
|
||||
* [Kublr](https://kublr.com) offers enterprise-grade secure, scalable, highly reliable Kubernetes clusters on AWS, Azure, GCP, and on-premise. It includes out-of-the-box backup and disaster recovery, multi-cluster centralized logging and monitoring, and built-in alerting.
|
||||
|
||||
* [KubeSail](https://kubesail.com) is an easy, free way to try Kubernetes.
|
||||
|
||||
* [Madcore.Ai](https://madcore.ai) is devops-focused CLI tool for deploying Kubernetes infrastructure in AWS. Master, auto-scaling group nodes with spot-instances, ingress-ssl-lego, Heapster, and Grafana.
|
||||
|
||||
* [Nutanix Karbon](https://www.nutanix.com/products/karbon/) is a multi-cluster, highly available Kubernetes management and operational platform that simplifies the provisioning, operations, and lifecycle management of Kubernetes.
|
||||
|
||||
* [OpenShift Dedicated](https://www.openshift.com/dedicated/) offers managed Kubernetes clusters powered by OpenShift.
|
||||
|
||||
* [OpenShift Online](https://www.openshift.com/features/) provides free hosted access for Kubernetes applications.
|
||||
|
||||
* [Oracle Cloud Infrastructure Container Engine for Kubernetes (OKE)](https://docs.us-phoenix-1.oraclecloud.com/Content/ContEng/Concepts/contengoverview.htm) is a fully-managed, scalable, and highly available service that you can use to deploy your containerized applications to the cloud.
|
||||
|
||||
* [Platform9](https://platform9.com/products/kubernetes/) offers managed Kubernetes on-premises or on any public cloud, and provides 24/7 health monitoring and alerting. (Kube2go, a web-UI driven Kubernetes cluster deployment service Platform9 released, has been integrated to Platform9 Sandbox.)
|
||||
|
||||
* [Stackpoint.io](https://stackpoint.io) provides Kubernetes infrastructure automation and management for multiple public clouds.
|
||||
|
||||
* [SysEleven MetaKube](https://www.syseleven.io/products-services/managed-kubernetes/) offers managed Kubernetes as a service powered on our OpenStack public cloud. It includes lifecycle management, administration dashboards, monitoring, autoscaling and much more.
|
||||
|
||||
* [VEXXHOST](https://vexxhost.com/public-cloud/container-services/kubernetes/) VEXXHOST proudly offers Certified Kubernetes on our public cloud, which also happens to be the largest OpenStack public cloud in Canada.
|
||||
|
||||
* [VMware Cloud PKS](https://cloud.vmware.com/vmware-cloud-pks) is an enterprise Kubernetes-as-a-Service offering in the VMware Cloud Services portfolio that provides easy to use, secure by default, cost effective, SaaS-based Kubernetes clusters.
|
||||
|
||||
## Turnkey Cloud Solutions
|
||||
|
||||
These solutions allow you to create Kubernetes clusters on a range of Cloud IaaS providers with only a
|
||||
few commands. These solutions are actively developed and have active community support.
|
||||
|
||||
* [Agile Stacks](https://www.agilestacks.com/products/kubernetes)
|
||||
* [Alibaba Cloud](/docs/setup/turnkey/alibaba-cloud/)
|
||||
* [APPUiO](https://appuio.ch)
|
||||
* [AWS](/docs/setup/turnkey/aws/)
|
||||
* [Azure](/docs/setup/turnkey/azure/)
|
||||
* [CenturyLink Cloud](/docs/setup/turnkey/clc/)
|
||||
* [Conjure-up Kubernetes with Ubuntu on AWS, Azure, Google Cloud, Oracle Cloud](/docs/getting-started-guides/ubuntu/)
|
||||
* [Containership](https://containership.io/containership-platform)
|
||||
* [Docker Enterprise](https://www.docker.com/products/docker-enterprise)
|
||||
* [Gardener](https://gardener.cloud/)
|
||||
* [Giant Swarm](https://giantswarm.io)
|
||||
* [Google Compute Engine (GCE)](/docs/setup/turnkey/gce/)
|
||||
* [IBM Cloud](https://github.com/patrocinio/kubernetes-softlayer)
|
||||
* [Kontena Pharos](https://kontena.io/pharos/)
|
||||
* [Kubermatic](https://cloud.kubermatic.io)
|
||||
* [Kublr](https://kublr.com/)
|
||||
* [Madcore.Ai](https://madcore.ai/)
|
||||
* [Nirmata](https://nirmata.com/)
|
||||
* [Nutanix Karbon](https://www.nutanix.com/products/karbon/)
|
||||
* [Oracle Cloud Infrastructure Container Engine for Kubernetes (OKE)](https://docs.us-phoenix-1.oraclecloud.com/Content/ContEng/Concepts/contengprerequisites.htm)
|
||||
* [Pivotal Container Service](https://pivotal.io/platform/pivotal-container-service)
|
||||
* [Rancher 2.0](https://rancher.com/docs/rancher/v2.x/en/)
|
||||
* [Stackpoint.io](/docs/setup/turnkey/stackpoint/)
|
||||
* [Supergiant.io](https://supergiant.io/)
|
||||
* [VEXXHOST](https://vexxhost.com/private-cloud/)
|
||||
* [VMware Cloud PKS](https://cloud.vmware.com/vmware-cloud-pks)
|
||||
* [VMware Enterprise PKS](https://cloud.vmware.com/vmware-enterprise-pks)
|
||||
|
||||
## On-Premises turnkey cloud solutions
|
||||
These solutions allow you to create Kubernetes clusters on your internal, secure, cloud network with only a
|
||||
few commands.
|
||||
|
||||
* [Agile Stacks](https://www.agilestacks.com/products/kubernetes)
|
||||
* [APPUiO](https://appuio.ch)
|
||||
* [Docker Enterprise](https://www.docker.com/products/docker-enterprise)
|
||||
* [Giant Swarm](https://giantswarm.io)
|
||||
* [GKE On-Prem | Google Cloud](https://cloud.google.com/gke-on-prem/)
|
||||
* [IBM Cloud Private](https://www.ibm.com/cloud-computing/products/ibm-cloud-private/)
|
||||
* [Kontena Pharos](https://kontena.io/pharos/)
|
||||
* [Kubermatic](https://www.loodse.com)
|
||||
* [Kublr](www.kublr.com/kubernetes.io/setup-hosted-solution)
|
||||
* [Mirantis Cloud Platform](https://www.mirantis.com/software/kubernetes/)
|
||||
* [Nirmata](https://nirmata.com/)
|
||||
* [OpenShift Container Platform](https://www.openshift.com/products/container-platform/) (OCP) by [Red Hat](https://www.redhat.com)
|
||||
* [Pivotal Container Service](https://pivotal.io/platform/pivotal-container-service)
|
||||
* [Rancher 2.0](https://rancher.com/docs/rancher/v2.x/en/)
|
||||
* [SUSE CaaS Platform](https://www.suse.com/products/caas-platform)
|
||||
* [SUSE Cloud Application Platform](https://www.suse.com/products/cloud-application-platform/)
|
||||
* [VMware Enterprise PKS](https://cloud.vmware.com/vmware-enterprise-pks)
|
||||
|
||||
|
||||
## Custom Solutions
|
||||
|
||||
Kubernetes can run on a wide range of Cloud providers and bare-metal environments, and with many
|
||||
base operating systems.
|
||||
|
||||
If you can find a guide below that matches your needs, use it.
|
||||
|
||||
### Universal
|
||||
|
||||
If you already have a way to configure hosting resources, use
|
||||
[kubeadm](/docs/setup/independent/create-cluster-kubeadm/) to bring up a cluster
|
||||
with a single command per machine.
|
||||
|
||||
### Cloud
|
||||
|
||||
These solutions are combinations of cloud providers and operating systems not covered by the above solutions.
|
||||
|
||||
* [Cloud Foundry Container Runtime (CFCR)](https://docs-cfcr.cfapps.io/)
|
||||
* [Gardener](https://gardener.cloud/)
|
||||
* [Kublr](www.kublr.com/kubernetes.io/setup-hosted-solution)
|
||||
* [Kubernetes on Ubuntu](/docs/getting-started-guides/ubuntu/)
|
||||
* [Kubespray](/docs/setup/custom-cloud/kubespray/)
|
||||
* [Rancher Kubernetes Engine (RKE)](https://github.com/rancher/rke)
|
||||
* [VMware Essential PKS](https://cloud.vmware.com/vmware-essential-PKS)
|
||||
|
||||
### On-Premises VMs
|
||||
|
||||
* [Cloud Foundry Container Runtime (CFCR)](https://docs-cfcr.cfapps.io/)
|
||||
* [CloudStack](/docs/setup/on-premises-vm/cloudstack/) (uses Ansible)
|
||||
* [Fedora (Multi Node)](/docs/getting-started-guides/fedora/flannel_multi_node_cluster/) (uses Fedora and flannel)
|
||||
* [Nutanix AHV](https://www.nutanix.com/products/acropolis/virtualization/)
|
||||
* [OpenShift Container Platform](https://www.openshift.com/products/container-platform/) (OCP) Kubernetes platform by [Red Hat](https://www.redhat.com)
|
||||
* [oVirt](/docs/setup/on-premises-vm/ovirt/)
|
||||
* [VMware Essential PKS](https://cloud.vmware.com/vmware-essential-PKS)
|
||||
* [VMware vSphere](https://github.com/kubernetes/cloud-provider-vsphere)
|
||||
* [VMware vSphere, OpenStack, or Bare Metal](/docs/getting-started-guides/ubuntu/) (uses Juju, Ubuntu and flannel)
|
||||
|
||||
### Bare Metal
|
||||
|
||||
* [Digital Rebar](/docs/setup/on-premises-metal/krib/)
|
||||
* [Docker Enterprise](https://www.docker.com/products/docker-enterprise)
|
||||
* [Fedora (Single Node)](/docs/getting-started-guides/fedora/fedora_manual_config/)
|
||||
* [Fedora (Multi Node)](/docs/getting-started-guides/fedora/flannel_multi_node_cluster/)
|
||||
* [Kubernetes on Ubuntu](/docs/getting-started-guides/ubuntu/)
|
||||
* [OpenShift Container Platform](https://www.openshift.com/products/container-platform/) (OCP) Kubernetes platform by [Red Hat](https://www.redhat.com)
|
||||
* [VMware Essential PKS](https://cloud.vmware.com/vmware-essential-PKS)
|
||||
|
||||
### Integrations
|
||||
|
||||
These solutions provide integration with third-party schedulers, resource managers, and/or lower level platforms.
|
||||
|
||||
* [DCOS](/docs/setup/on-premises-vm/dcos/)
|
||||
* Community Edition DCOS uses AWS
|
||||
* Enterprise Edition DCOS supports cloud hosting, on-premises VMs, and bare metal
|
||||
|
||||
## Table of Solutions
|
||||
|
||||
Below is a table of all of the solutions listed above.
|
||||
|
||||
IaaS Provider | Config. Mgmt. | OS | Networking | Docs | Support Level
|
||||
-------------------- | ------------ | ------ | ---------- | --------------------------------------------- | ----------------------------
|
||||
Agile Stacks | Terraform | CoreOS | multi-support | [docs](https://www.agilestacks.com/products/kubernetes) | Commercial
|
||||
Alibaba Cloud Container Service For Kubernetes | ROS | CentOS | flannel/Terway | [docs](https://www.aliyun.com/product/containerservice) | Commercial
|
||||
any | any | multi-support | any CNI | [docs](/docs/setup/independent/create-cluster-kubeadm/) | Project ([SIG-cluster-lifecycle](https://git.k8s.io/community/sig-cluster-lifecycle))
|
||||
any | any | any | any | [docs](/docs/setup/scratch/) | Community ([@erictune](https://github.com/erictune))
|
||||
any | any | any | any | [docs](http://docs.projectcalico.org/v2.2/getting-started/kubernetes/installation/) | Commercial and Community
|
||||
any | RKE | multi-support | flannel or canal | [docs](https://rancher.com/docs/rancher/v2.x/en/quick-start-guide/) | [Commercial](https://rancher.com/what-is-rancher/overview/) and [Community](https://github.com/rancher/rancher)
|
||||
any | [Gardener Cluster-Operator](https://kubernetes.io/blog/2018/05/17/gardener/) | multi-support | multi-support | [docs](https://gardener.cloud) | [Project/Community](https://github.com/gardener) and [Commercial]( https://cloudplatform.sap.com/)
|
||||
AppsCode.com | Saltstack | Debian | multi-support | [docs](https://appscode.com/products/cloud-deployment/) | Commercial
|
||||
AWS | CoreOS | CoreOS | flannel | [docs](/docs/setup/turnkey/aws/) | Community
|
||||
AWS | Saltstack | Debian | AWS | [docs](/docs/setup/turnkey/aws/) | Community ([@justinsb](https://github.com/justinsb))
|
||||
AWS | kops | Debian | AWS | [docs](https://github.com/kubernetes/kops/) | Community ([@justinsb](https://github.com/justinsb))
|
||||
AWS | Juju | Ubuntu | flannel/calico/canal | [docs](/docs/getting-started-guides/ubuntu/) | [Commercial](https://www.ubuntu.com/kubernetes) and [Community](https://jujucharms.com/kubernetes)
|
||||
Azure | Juju | Ubuntu | flannel/calico/canal | [docs](/docs/getting-started-guides/ubuntu/) | [Commercial](https://www.ubuntu.com/kubernetes) and [Community](https://jujucharms.com/kubernetes)
|
||||
Azure (IaaS) | | Ubuntu | Azure | [docs](/docs/setup/turnkey/azure/) | [Community (Microsoft)](https://github.com/Azure/acs-engine)
|
||||
Azure Kubernetes Service | | Ubuntu | Azure | [docs](https://docs.microsoft.com/en-us/azure/aks/) | Commercial
|
||||
Bare-metal | custom | CentOS | flannel | [docs](/docs/getting-started-guides/centos/centos_manual_config/) | Community ([@coolsvap](https://github.com/coolsvap))
|
||||
Bare-metal | custom | Fedora | _none_ | [docs](/docs/getting-started-guides/fedora/fedora_manual_config/) | Project
|
||||
Bare-metal | custom | Fedora | flannel | [docs](/docs/getting-started-guides/fedora/flannel_multi_node_cluster/) | Community ([@aveshagarwal](https://github.com/aveshagarwal))
|
||||
Bare Metal | Juju | Ubuntu | flannel/calico/canal | [docs](/docs/getting-started-guides/ubuntu/) | [Commercial](https://www.ubuntu.com/kubernetes) and [Community](https://jujucharms.com/kubernetes)
|
||||
Bare-metal | custom | Ubuntu | flannel | [docs](/docs/getting-started-guides/ubuntu/) | Community ([@resouer](https://github.com/resouer), [@WIZARD-CXY](https://github.com/WIZARD-CXY))
|
||||
CloudStack | Ansible | CoreOS | flannel | [docs](/docs/getting-started-guides/cloudstack/) | Community ([@sebgoa](https://github.com/sebgoa))
|
||||
DCOS | Marathon | CoreOS/Alpine | custom | [docs](/docs/getting-started-guides/dcos/) | Community ([Kubernetes-Mesos Authors](https://github.com/mesosphere/kubernetes-mesos/blob/master/AUTHORS.md))
|
||||
Digital Rebar | kubeadm | any | metal | [docs](/docs/setup/on-premises-metal/krib/) | Community ([@digitalrebar](https://github.com/digitalrebar))
|
||||
Docker Enterprise | custom | [multi-support](https://success.docker.com/article/compatibility-matrix) | [multi-support](https://docs.docker.com/ee/ucp/kubernetes/install-cni-plugin/) | [docs](https://docs.docker.com/ee/) | Commercial
|
||||
Giant Swarm | | CoreOS | flannel and/or Calico | [docs](https://docs.giantswarm.io/) | Commercial
|
||||
GCE | CoreOS | CoreOS | flannel | [docs](/docs/getting-started-guides/coreos/) | Community ([@pires](https://github.com/pires))
|
||||
GCE | Juju | Ubuntu | flannel/calico/canal | [docs](/docs/getting-started-guides/ubuntu/) | [Commercial](https://www.ubuntu.com/kubernetes) and [Community](https://jujucharms.com/kubernetes)
|
||||
GCE | Saltstack | Debian | GCE | [docs](/docs/setup/turnkey/gce/) | Project
|
||||
Google Kubernetes Engine | | | GCE | [docs](https://cloud.google.com/kubernetes-engine/docs/) | Commercial
|
||||
IBM Cloud Kubernetes Service | | Ubuntu | IBM Cloud Networking + Calico | [docs](https://cloud.ibm.com/docs/containers?topic=containers-container_index#container_index) | Commercial
|
||||
IBM Cloud Kubernetes Service | | Ubuntu | calico | [docs](https://cloud.ibm.com/docs/containers?topic=containers-container_index#container_index) | Commercial
|
||||
IBM Cloud Private | Ansible | multi-support | multi-support | [docs](https://www.ibm.com/support/knowledgecenter/SSBS6K/product_welcome_cloud_private.html) | [Commercial](https://www.ibm.com/mysupport/s/topic/0TO500000001o0fGAA/ibm-cloud-private?language=en_US&productId=01t50000004X1PWAA0) and [Community](https://www.ibm.com/support/knowledgecenter/SSBS6K_3.1.2/troubleshoot/support_types.html) |
|
||||
Kublr | custom | multi-support | multi-support | [docs](http://docs.kublr.com/) | Commercial
|
||||
Kubermatic | | multi-support | multi-support | [docs](http://docs.kubermatic.io/) | Commercial
|
||||
KVM | custom | Fedora | flannel | [docs](/docs/getting-started-guides/fedora/flannel_multi_node_cluster/) | Community ([@aveshagarwal](https://github.com/aveshagarwal))
|
||||
libvirt | custom | Fedora | flannel | [docs](/docs/getting-started-guides/fedora/flannel_multi_node_cluster/) | Community ([@aveshagarwal](https://github.com/aveshagarwal))
|
||||
lxd | Juju | Ubuntu | flannel/canal | [docs](/docs/getting-started-guides/ubuntu/local/) | [Commercial](https://www.ubuntu.com/kubernetes) and [Community](https://jujucharms.com/kubernetes)
|
||||
Madcore.Ai | Jenkins DSL | Ubuntu | flannel | [docs](https://madcore.ai) | Community ([@madcore-ai](https://github.com/madcore-ai))
|
||||
Mirantis Cloud Platform | Salt | Ubuntu | multi-support | [docs](https://docs.mirantis.com/mcp/) | Commercial
|
||||
Oracle Cloud Infrastructure | Juju | Ubuntu | flannel/calico/canal | [docs](/docs/getting-started-guides/ubuntu/) | [Commercial](https://www.ubuntu.com/kubernetes) and [Community](https://jujucharms.com/kubernetes)
|
||||
Oracle Cloud Infrastructure Container Engine for Kubernetes (OKE) | | | multi-support | [docs](https://docs.cloud.oracle.com/iaas/Content/ContEng/Concepts/contengoverview.htm) | Commercial
|
||||
oVirt | | | | [docs](/docs/setup/on-premises-vm/ovirt/) | Community ([@simon3z](https://github.com/simon3z))
|
||||
Platform9 | | multi-support | multi-support | [docs](https://platform9.com/managed-kubernetes/) | Commercial
|
||||
Rackspace | custom | CoreOS | flannel/calico/canal | [docs](https://developer.rackspace.com/docs/rkaas/latest/) | [Commercial](https://www.rackspace.com/managed-kubernetes)
|
||||
Red Hat OpenShift | Ansible & CoreOS | RHEL & CoreOS | [multi-support](https://docs.openshift.com/container-platform/3.11/architecture/networking/network_plugins.html) | [docs](https://docs.openshift.com/container-platform/3.11/welcome/index.html) | Commercial
|
||||
Stackpoint.io | | multi-support | multi-support | [docs](https://stackpoint.io/) | Commercial
|
||||
Vagrant | CoreOS | CoreOS | flannel | [docs](/docs/getting-started-guides/coreos/) | Community ([@pires](https://github.com/pires), [@AntonioMeireles](https://github.com/AntonioMeireles))
|
||||
VMware vSphere | any | multi-support | multi-support | [docs](https://github.com/kubernetes/cloud-provider-vsphere/tree/master/docs) | [Community](https://vmware.github.io/vsphere-storage-for-kubernetes/documentation/contactus.html)
|
||||
VMware vSphere | Juju | Ubuntu | flannel/calico/canal | [docs](/docs/getting-started-guides/ubuntu/) | [Commercial](https://www.ubuntu.com/kubernetes) and [Community](https://jujucharms.com/kubernetes)
|
||||
VMware Cloud PKS | | Photon OS | Canal | [docs](https://docs.vmware.com/en/VMware-Kubernetes-Engine/index.html) | Commercial
|
||||
VMware Enterprise PKS | BOSH | Ubuntu | VMware NSX-T/flannel | [docs](https://docs.vmware.com/en/VMware-Enterprise-PKS/) | Commercial
|
||||
VMware Essential PKS | any | multi-support | multi-support | [docs](https://cloud.vmware.com/vmware-essential-PKS) | Commercial
|
||||
|
||||
### Definition of columns
|
||||
|
||||
* **IaaS Provider** is the product or organization which provides the virtual or physical machines (nodes) that Kubernetes runs on.
|
||||
* **OS** is the base operating system of the nodes.
|
||||
* **Config. Mgmt.** is the configuration management system that helps install and maintain Kubernetes on the
|
||||
nodes.
|
||||
* **Networking** is what implements the [networking model](/docs/concepts/cluster-administration/networking/). Those with networking type
|
||||
_none_ may not support more than a single node, or may support multiple VM nodes in a single physical node.
|
||||
* **Support Levels**
|
||||
* **Project**: Kubernetes committers regularly use this configuration, so it usually works with the latest release
|
||||
of Kubernetes.
|
||||
* **Commercial**: A commercial offering with its own support arrangements.
|
||||
* **Community**: Actively supported by community contributions. May not work with recent releases of Kubernetes.
|
||||
* **Inactive**: Not actively maintained. Not recommended for first-time Kubernetes users, and may be removed.
|
||||
* **Notes** has other relevant information, such as the version of Kubernetes used.
|
||||
|
||||
<!-- reference style links below here -->
|
||||
<!-- GCE conformance test result -->
|
||||
[1]: https://gist.github.com/erictune/4cabc010906afbcc5061
|
||||
<!-- Vagrant conformance test result -->
|
||||
[2]: https://gist.github.com/derekwaynecarr/505e56036cdf010bf6b6
|
||||
<!-- Google Kubernetes Engine conformance test result -->
|
||||
[3]: https://gist.github.com/erictune/2f39b22f72565365e59b
|
||||
|
||||
{{% /capture %}}
|
||||
@@ -0,0 +1,4 @@
|
||||
---
|
||||
title: Production environment
|
||||
weight: 30
|
||||
---
|
||||
+11
-4
@@ -2,9 +2,9 @@
|
||||
reviewers:
|
||||
- vincepri
|
||||
- bart0sh
|
||||
title: CRI installation
|
||||
title: Container runtimes
|
||||
content_template: templates/concept
|
||||
weight: 100
|
||||
weight: 10
|
||||
---
|
||||
{{% capture overview %}}
|
||||
{{< feature-state for_k8s_version="v1.6" state="stable" >}}
|
||||
@@ -53,6 +53,14 @@ for the rest of the processes running on the node becomes unstable under resourc
|
||||
Changing the settings such that your container runtime and kubelet use `systemd` as the cgroup driver
|
||||
stabilized the system. Please note the `native.cgroupdriver=systemd` option in the Docker setup below.
|
||||
|
||||
{{< caution >}}
|
||||
Changing the cgroup driver of a Node that has joined a cluster is highly unrecommended.
|
||||
If the kubelet has created Pods using the semantics of one cgroup driver, changing the container
|
||||
runtime to another cgroup driver can cause errors when trying to re-create the PodSandbox
|
||||
for such existing Pods. Restarting the kubelet may not solve such errors. The recommendation
|
||||
is to drain the Node from its workloads, remove it from the cluster and re-join it.
|
||||
{{< /caution >}}
|
||||
|
||||
## Docker
|
||||
|
||||
On each of your machines, install Docker.
|
||||
@@ -275,8 +283,7 @@ systemctl restart containerd
|
||||
|
||||
To use the `systemd` cgroup driver, set `plugins.cri.systemd_cgroup = true` in `/etc/containerd/config.toml`.
|
||||
When using kubeadm, manually configure the
|
||||
[cgroup driver for kubelet](/docs/setup/independent/install-kubeadm/#configure-cgroup-driver-used-by-kubelet-on-master-node)
|
||||
as well.
|
||||
[cgroup driver for kubelet](/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#configure-cgroup-driver-used-by-kubelet-on-master-node)
|
||||
|
||||
## Other CRI runtimes: frakti
|
||||
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
---
|
||||
title: On-Premises VMs
|
||||
weight: 60
|
||||
weight: 40
|
||||
---
|
||||
+2
-3
@@ -28,7 +28,7 @@ sudo apt-add-repository ppa:ansible/ansible
|
||||
sudo apt-get update
|
||||
sudo apt-get install ansible
|
||||
```
|
||||
|
||||
|
||||
On CloudStack server you also have to install libselinux-python :
|
||||
|
||||
```shell
|
||||
@@ -115,8 +115,7 @@ e9af8293... <node #2 IP> role=node
|
||||
|
||||
IaaS Provider | Config. Mgmt | OS | Networking | Docs | Conforms | Support Level
|
||||
-------------------- | ------------ | ------ | ---------- | --------------------------------------------- | ---------| ----------------------------
|
||||
CloudStack | Ansible | CoreOS | flannel | [docs](/docs/setup/on-premises-vm/cloudstack/) | | Community ([@Guiques](https://github.com/ltupin/))
|
||||
CloudStack | Ansible | CoreOS | flannel | [docs](/docs/setup/production-environment/on-premises-vm/cloudstack/) | | Community ([@Guiques](https://github.com/ltupin/))
|
||||
|
||||
For support level information on all solutions, see the [Table of solutions](/docs/setup/pick-right-solution/#table-of-solutions) chart.
|
||||
|
||||
{{% /capture %}}
|
||||
+1
-2
@@ -66,8 +66,7 @@ This short screencast demonstrates how the oVirt Cloud Provider can be used to d
|
||||
|
||||
IaaS Provider | Config. Mgmt | OS | Networking | Docs | Conforms | Support Level
|
||||
-------------------- | ------------ | ------ | ---------- | --------------------------------------------- | ---------| ----------------------------
|
||||
oVirt | | | | [docs](/docs/setup/on-premises-vm/ovirt/) | | Community ([@simon3z](https://github.com/simon3z))
|
||||
oVirt | | | | [docs](/docs/setup/production-environment/on-premises-vm/ovirt/) | | Community ([@simon3z](https://github.com/simon3z))
|
||||
|
||||
For support level information on all solutions, see the [Table of solutions](/docs/setup/pick-right-solution/#table-of-solutions) chart.
|
||||
|
||||
{{% /capture %}}
|
||||
@@ -0,0 +1,4 @@
|
||||
---
|
||||
title: Installing Kubernetes with deployment tools
|
||||
weight: 30
|
||||
---
|
||||
+2
-1
@@ -1,6 +1,7 @@
|
||||
---
|
||||
title: Installing Kubernetes on AWS with kops
|
||||
title: Installing Kubernetes with kops
|
||||
content_template: templates/concept
|
||||
weight: 20
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
+11
-5
@@ -1,12 +1,13 @@
|
||||
---
|
||||
title: Installing Kubernetes with Digital Rebar Provision (DRP) via KRIB
|
||||
title: Installing Kubernetes with KRIB
|
||||
krib-version: 2.4
|
||||
author: Rob Hirschfeld (zehicle)
|
||||
weight: 20
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
This guide helps to install a Kubernetes cluster hosted on bare metal with [Digital Rebar Provision](https://github.com/digitalrebar/provision) using only its Content packages and *kubeadm*.
|
||||
This guide helps to install a Kubernetes cluster hosted on bare metal with [Digital Rebar Provision](https://github.com/digitalrebar/provision) using only its Content packages and *kubeadm*.
|
||||
|
||||
Digital Rebar Provision (DRP) is an integrated Golang DHCP, bare metal provisioning (PXE/iPXE) and workflow automation platform. While [DRP can be used to invoke](https://provision.readthedocs.io/en/tip/doc/integrations/ansible.html) [kubespray](/docs/setup/custom-cloud/kubespray), it also offers a self-contained Kubernetes installation known as [KRIB (Kubernetes Rebar Integrated Bootstrap)](https://github.com/digitalrebar/provision-content/tree/master/krib).
|
||||
|
||||
@@ -24,7 +25,7 @@ KRIB features:
|
||||
* dynamic generation of a TLS infrastructure
|
||||
* composable attributes and automatic detection of hardware by profile
|
||||
* options for persistent, immutable and image-based deployments
|
||||
* support for Ubuntu 18.04, CentOS/RHEL 7 and others
|
||||
* support for Ubuntu 18.04, CentOS/RHEL 7, CoreOS, RancherOS and others
|
||||
|
||||
## Creating a cluster
|
||||
|
||||
@@ -38,7 +39,12 @@ Following the [Digital Rebar installation](https://provision.readthedocs.io/en/t
|
||||
|
||||
### (2/5) Install KRIB Content and Certificate Plugin
|
||||
|
||||
Upload the KRIB Content bundle (or build from [source](https://github.com/digitalrebar/provision-content/tree/master/krib)) and the Cert Plugin for your DRP platform (e.g.: [amd64 Linux v2.4.0](https://s3-us-west-2.amazonaws.com/rebar-catalog/certs/v2.4.0-0-02301d35f9f664d6c81d904c92a9c81d3fd41d2c/amd64/linux/certs)). Both are freely available via the [RackN UX](https://portal.rackn.io).
|
||||
Upload the KRIB Content bundle (or build from [source](https://github.com/digitalrebar/provision-content/tree/master/krib)) and the Cert Plugin for your DRP platform. Both are freely available via the [RackN UX](https://portal.rackn.io) or using the upload from catalog feature of the DRPCLI (shown below).
|
||||
|
||||
```
|
||||
drpcli plugin_providers upload certs from catalog:certs-stable
|
||||
drpcli contents upload catalog:krib-stable
|
||||
```
|
||||
|
||||
### (3/5) Start your cluster deployment
|
||||
|
||||
@@ -60,7 +66,7 @@ During the installation, KRIB writes cluster configuration data back into the cl
|
||||
|
||||
### (5/5) Access your cluster
|
||||
|
||||
The cluster is available for access via *kubectl* once the `krib/cluster-admin-conf` Param has been set. This Param contains the `kubeconfig` information necessary to access the cluster.
|
||||
The cluster is available for access via *kubectl* once the `krib/cluster-admin-conf` Param has been set. This Param contains the `kubeconfig` information necessary to access the cluster.
|
||||
|
||||
For example, if you named the cluster Profile `krib` then the following commands would allow you to connect to the installed cluster from your local terminal.
|
||||
|
||||
@@ -0,0 +1,4 @@
|
||||
---
|
||||
title: "Bootstrapping clusters with kubeadm"
|
||||
weight: 10
|
||||
---
|
||||
+87
@@ -0,0 +1,87 @@
|
||||
---
|
||||
reviewers:
|
||||
- sig-cluster-lifecycle
|
||||
title: Customizing control plane configuration with kubeadm
|
||||
content_template: templates/concept
|
||||
weight: 40
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
{{< feature-state for_k8s_version="1.12" state="stable" >}}
|
||||
|
||||
The kubeadm `ClusterConfiguration` object exposes the field `extraArgs` that can override the default flags passed to control plane
|
||||
components such as the APIServer, ControllerManager and Scheduler. The components are defined using the following fields:
|
||||
|
||||
- `apiServer`
|
||||
- `controllerManager`
|
||||
- `scheduler`
|
||||
|
||||
The `extraArgs` field consist of `key: value` pairs. To override a flag for a control plane component:
|
||||
|
||||
1. Add the appropriate fields to your configuration.
|
||||
2. Add the flags to override to the field.
|
||||
|
||||
For more details on each field in the configuration you can navigate to our
|
||||
[API reference pages](https://godoc.org/k8s.io/kubernetes/cmd/kubeadm/app/apis/kubeadm/v1beta2#ClusterConfiguration).
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture body %}}
|
||||
|
||||
## APIServer flags
|
||||
|
||||
For details, see the [reference documentation for kube-apiserver](/docs/reference/command-line-tools-reference/kube-apiserver/).
|
||||
|
||||
Example usage:
|
||||
```yaml
|
||||
apiVersion: kubeadm.k8s.io/v1beta2
|
||||
kind: ClusterConfiguration
|
||||
kubernetesVersion: v1.13.0
|
||||
metadata:
|
||||
name: 1.13-sample
|
||||
apiServer:
|
||||
extraArgs:
|
||||
advertise-address: 192.168.0.103
|
||||
anonymous-auth: false
|
||||
enable-admission-plugins: AlwaysPullImages,DefaultStorageClass
|
||||
audit-log-path: /home/johndoe/audit.log
|
||||
```
|
||||
|
||||
## ControllerManager flags
|
||||
|
||||
For details, see the [reference documentation for kube-controller-manager](/docs/reference/command-line-tools-reference/kube-controller-manager/).
|
||||
|
||||
Example usage:
|
||||
```yaml
|
||||
apiVersion: kubeadm.k8s.io/v1beta2
|
||||
kind: ClusterConfiguration
|
||||
kubernetesVersion: v1.13.0
|
||||
metadata:
|
||||
name: 1.13-sample
|
||||
controllerManager:
|
||||
extraArgs:
|
||||
cluster-signing-key-file: /home/johndoe/keys/ca.key
|
||||
bind-address: 0.0.0.0
|
||||
deployment-controller-sync-period: 50
|
||||
```
|
||||
|
||||
## Scheduler flags
|
||||
|
||||
For details, see the [reference documentation for kube-scheduler](/docs/reference/command-line-tools-reference/kube-scheduler/).
|
||||
|
||||
Example usage:
|
||||
```yaml
|
||||
apiVersion: kubeadm.k8s.io/v1beta2
|
||||
kind: ClusterConfiguration
|
||||
kubernetesVersion: v1.13.0
|
||||
metadata:
|
||||
name: 1.13-sample
|
||||
scheduler:
|
||||
extraArgs:
|
||||
address: 0.0.0.0
|
||||
config: /home/johndoe/schedconfig.yaml
|
||||
kubeconfig: /home/johndoe/kubeconfig.yaml
|
||||
```
|
||||
|
||||
{{% /capture %}}
|
||||
+26
-20
@@ -8,7 +8,7 @@ weight: 30
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cncf/artwork/master/kubernetes/certified-kubernetes/versionless/color/certified-kubernetes-color.png" align="right" width="150px">**kubeadm** helps you bootstrap a minimum viable Kubernetes cluster that conforms to best practices. With kubeadm, your cluster should pass [Kubernetes Conformance tests](https://kubernetes.io/blog/2017/10/software-conformance-certification). Kubeadm also supports other cluster
|
||||
<img src="https://raw.githubusercontent.com/cncf/artwork/master/projects/kubernetes/certified-kubernetes/versionless/color/certified-kubernetes-color.png" align="right" width="150px">**kubeadm** helps you bootstrap a minimum viable Kubernetes cluster that conforms to best practices. With kubeadm, your cluster should pass [Kubernetes Conformance tests](https://kubernetes.io/blog/2017/10/software-conformance-certification). Kubeadm also supports other cluster
|
||||
lifecycle functions, such as upgrades, downgrade, and managing [bootstrap tokens](/docs/reference/access-authn-authz/bootstrap-tokens/).
|
||||
|
||||
Because you can install kubeadm on various types of machine (e.g. laptop, server,
|
||||
@@ -89,7 +89,7 @@ timeframe; which also applies to `kubeadm`.
|
||||
|
||||
## Objectives
|
||||
|
||||
* Install a single control-plane Kubernetes cluster or [high-availability cluster](/docs/setup/independent/high-availability/)
|
||||
* Install a single master Kubernetes cluster or [high availability cluster](/docs/setup/production-environment/tools/kubeadm/high-availability/)
|
||||
* Install a Pod network on the cluster so that your Pods can
|
||||
talk to each other
|
||||
|
||||
@@ -97,7 +97,7 @@ timeframe; which also applies to `kubeadm`.
|
||||
|
||||
### Installing kubeadm on your hosts
|
||||
|
||||
See ["Installing kubeadm"](/docs/setup/independent/install-kubeadm/).
|
||||
See ["Installing kubeadm"](/docs/setup/production-environment/tools/kubeadm/install-kubeadm/).
|
||||
|
||||
{{< note >}}
|
||||
If you have already installed kubeadm, run `apt-get update &&
|
||||
@@ -105,7 +105,7 @@ apt-get upgrade` or `yum update` to get the latest version of kubeadm.
|
||||
|
||||
When you upgrade, the kubelet restarts every few seconds as it waits in a crashloop for
|
||||
kubeadm to tell it what to do. This crashloop is expected and normal.
|
||||
After you initialize your control-plane, the kubelet runs normally.
|
||||
After you initialize your master, the kubelet runs normally.
|
||||
{{< /note >}}
|
||||
|
||||
### Initializing your control-plane node
|
||||
@@ -121,14 +121,14 @@ a provider-specific value. See [Installing a pod network add-on](#pod-network).
|
||||
1. (Optional) Since version 1.14, kubeadm will try to detect the container runtime on Linux
|
||||
by using a list of well known domain socket paths. To use different container runtime or
|
||||
if there are more than one installed on the provisioned node, specify the `--cri-socket`
|
||||
argument to `kubeadm init`. See [Installing runtime](/docs/setup/independent/install-kubeadm/#installing-runtime).
|
||||
argument to `kubeadm init`. See [Installing runtime](/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#installing-runtime).
|
||||
1. (Optional) Unless otherwise specified, kubeadm uses the network interface associated
|
||||
with the default gateway to advertise the control-plane's IP. To use a different
|
||||
with the default gateway to advertise the master's IP. To use a different
|
||||
network interface, specify the `--apiserver-advertise-address=<ip-address>` argument
|
||||
to `kubeadm init`. To deploy an IPv6 Kubernetes cluster using IPv6 addressing, you
|
||||
must specify an IPv6 address, for example `--apiserver-advertise-address=fd00::101`
|
||||
1. (Optional) Run `kubeadm config images pull` prior to `kubeadm init` to verify
|
||||
connectivity to gcr.io registries.
|
||||
connectivity to gcr.io registries.
|
||||
|
||||
Now run:
|
||||
|
||||
@@ -289,8 +289,7 @@ For more information about using Calico, see [Quickstart for Calico on Kubernete
|
||||
For Calico to work correctly, you need to pass `--pod-network-cidr=192.168.0.0/16` to `kubeadm init` or update the `calico.yml` file to match your Pod network. Note that Calico works on `amd64`, `arm64`, and `ppc64le` only.
|
||||
|
||||
```shell
|
||||
kubectl apply -f https://docs.projectcalico.org/v3.3/getting-started/kubernetes/installation/hosted/rbac-kdd.yaml
|
||||
kubectl apply -f https://docs.projectcalico.org/v3.3/getting-started/kubernetes/installation/hosted/kubernetes-datastore/calico-networking/1.7/calico.yaml
|
||||
kubectl apply -f https://docs.projectcalico.org/v3.7/manifests/calico.yaml
|
||||
```
|
||||
|
||||
{{% /tab %}}
|
||||
@@ -300,8 +299,7 @@ Canal uses Calico for policy and Flannel for networking. Refer to the Calico doc
|
||||
For Canal to work correctly, `--pod-network-cidr=10.244.0.0/16` has to be passed to `kubeadm init`. Note that Canal works on `amd64` only.
|
||||
|
||||
```shell
|
||||
kubectl apply -f https://docs.projectcalico.org/v3.3/getting-started/kubernetes/installation/hosted/canal/rbac.yaml
|
||||
kubectl apply -f https://docs.projectcalico.org/v3.3/getting-started/kubernetes/installation/hosted/canal/canal.yaml
|
||||
kubectl apply -f https://docs.projectcalico.org/v3.7/manifests/canal.yaml
|
||||
```
|
||||
|
||||
{{% /tab %}}
|
||||
@@ -309,6 +307,8 @@ kubectl apply -f https://docs.projectcalico.org/v3.3/getting-started/kubernetes/
|
||||
{{% tab name="Cilium" %}}
|
||||
For more information about using Cilium with Kubernetes, see [Kubernetes Install guide for Cilium](https://docs.cilium.io/en/stable/kubernetes/).
|
||||
|
||||
For Cilium to work correctly, you must pass `--pod-network-cidr=10.217.0.0/16` to `kubeadm init`.
|
||||
|
||||
These commands will deploy Cilium with its own etcd managed by etcd operator.
|
||||
|
||||
_Note_: If you are running kubeadm in a single node please untaint it so that
|
||||
@@ -321,13 +321,16 @@ kubectl taint nodes <node-name> node-role.kubernetes.io/master:NoSchedule-
|
||||
To deploy Cilium you just need to run:
|
||||
|
||||
```shell
|
||||
kubectl create -f https://raw.githubusercontent.com/cilium/cilium/v1.4/examples/kubernetes/1.13/cilium.yaml
|
||||
kubectl create -f https://raw.githubusercontent.com/cilium/cilium/v1.5/examples/kubernetes/1.14/cilium.yaml
|
||||
```
|
||||
|
||||
Once all Cilium pods are marked as `READY`, you start using your cluster.
|
||||
|
||||
```shell
|
||||
$ kubectl get pods -n kube-system --selector=k8s-app=cilium
|
||||
kubectl get pods -n kube-system --selector=k8s-app=cilium
|
||||
```
|
||||
The output is similar to this:
|
||||
```
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
cilium-drxkl 1/1 Running 0 18m
|
||||
```
|
||||
@@ -341,11 +344,15 @@ Set `/proc/sys/net/bridge/bridge-nf-call-iptables` to `1` by running `sysctl net
|
||||
to pass bridged IPv4 traffic to iptables' chains. This is a requirement for some CNI plugins to work, for more information
|
||||
please see [here](/docs/concepts/cluster-administration/network-plugins/#network-plugin-requirements).
|
||||
|
||||
Make sure that your firewall rules allow UDP ports 8285 and 8472 traffic for all hosts participating in the overlay network.
|
||||
see [here
|
||||
](https://coreos.com/flannel/docs/latest/troubleshooting.html#firewalls).
|
||||
|
||||
Note that `flannel` works on `amd64`, `arm`, `arm64`, `ppc64le` and `s390x` under Linux.
|
||||
Windows (`amd64`) is claimed as supported in v0.11.0 but the usage is undocumented.
|
||||
|
||||
```shell
|
||||
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/a70459be0084506e4ec919aa1c114638878db11b/Documentation/kube-flannel.yml
|
||||
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/62e44c867a2846fefb68bd5f178daf4da3095ccb/Documentation/kube-flannel.yml
|
||||
```
|
||||
|
||||
For more information about `flannel`, see [the CoreOS flannel repository on GitHub
|
||||
@@ -420,8 +427,7 @@ Once a pod network has been installed, you can confirm that it is working by
|
||||
checking that the CoreDNS pod is Running in the output of `kubectl get pods --all-namespaces`.
|
||||
And once the CoreDNS pod is up and running, you can continue by joining your nodes.
|
||||
|
||||
If your network is not working or CoreDNS is not in the Running state, check
|
||||
out our [troubleshooting docs](/docs/setup/independent/troubleshooting-kubeadm/).
|
||||
If your network is not working or CoreDNS is not in the Running state, checkout our [troubleshooting docs](/docs/setup/production-environment/tools/kubeadm/troubleshooting-kubeadm/).
|
||||
|
||||
### Control plane node isolation
|
||||
|
||||
@@ -614,7 +620,7 @@ control of your Kubernetes cluster.
|
||||
|
||||
## Feedback {#feedback}
|
||||
|
||||
* For bugs, visit [kubeadm Github issue tracker](https://github.com/kubernetes/kubeadm/issues)
|
||||
* For bugs, visit [kubeadm GitHub issue tracker](https://github.com/kubernetes/kubeadm/issues)
|
||||
* For support, visit kubeadm Slack Channel:
|
||||
[#kubeadm](https://kubernetes.slack.com/messages/kubeadm/)
|
||||
* General SIG Cluster Lifecycle Development Slack Channel:
|
||||
@@ -635,8 +641,8 @@ v1.8.
|
||||
|
||||
These resources provide more information on supported version skew between kubelets and the control plane, and other Kubernetes components:
|
||||
|
||||
* Kubernetes [version and version-skew policy](/docs/setup/version-skew-policy/)
|
||||
* Kubeadm-specific [installation guide](/docs/setup/independent/install-kubeadm/#installing-kubeadm-kubelet-and-kubectl)
|
||||
* Kubernetes [version and version-skew policy](/docs/setup/release/version-skew-policy/)
|
||||
* Kubeadm-specific [installation guide](/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#installing-kubeadm-kubelet-and-kubectl)
|
||||
|
||||
## kubeadm works on multiple platforms {#multi-platform}
|
||||
|
||||
@@ -666,4 +672,4 @@ Workarounds:
|
||||
|
||||
## Troubleshooting {#troubleshooting}
|
||||
|
||||
If you are running into difficulties with kubeadm, please consult our [troubleshooting docs](/docs/setup/independent/troubleshooting-kubeadm/).
|
||||
If you are running into difficulties with kubeadm, please consult our [troubleshooting docs](/docs/setup/production-environment/tools/kubeadm/troubleshooting-kubeadm/).
|
||||
+675
@@ -0,0 +1,675 @@
|
||||
---
|
||||
reviewers:
|
||||
- sig-cluster-lifecycle
|
||||
title: Creating a single control-plane cluster with kubeadm
|
||||
content_template: templates/task
|
||||
weight: 30
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cncf/artwork/master/projects/kubernetes/certified-kubernetes/versionless/color/certified-kubernetes-color.png" align="right" width="150px">**kubeadm** helps you bootstrap a minimum viable Kubernetes cluster that conforms to best practices. With kubeadm, your cluster should pass [Kubernetes Conformance tests](https://kubernetes.io/blog/2017/10/software-conformance-certification). Kubeadm also supports other cluster
|
||||
lifecycle functions, such as upgrades, downgrade, and managing [bootstrap tokens](/docs/reference/access-authn-authz/bootstrap-tokens/).
|
||||
|
||||
Because you can install kubeadm on various types of machine (e.g. laptop, server,
|
||||
Raspberry Pi, etc.), it's well suited for integration with provisioning systems
|
||||
such as Terraform or Ansible.
|
||||
|
||||
kubeadm's simplicity means it can serve a wide range of use cases:
|
||||
|
||||
- New users can start with kubeadm to try Kubernetes out for the first time.
|
||||
- Users familiar with Kubernetes can spin up clusters with kubeadm and test their applications.
|
||||
- Larger projects can include kubeadm as a building block in a more complex system that can also include other installer tools.
|
||||
|
||||
kubeadm is designed to be a simple way for new users to start trying
|
||||
Kubernetes out, possibly for the first time, a way for existing users to
|
||||
test their application on and stitch together a cluster easily, and also to be
|
||||
a building block in other ecosystem and/or installer tool with a larger
|
||||
scope.
|
||||
|
||||
You can install _kubeadm_ very easily on operating systems that support
|
||||
installing deb or rpm packages. The responsible SIG for kubeadm,
|
||||
[SIG Cluster Lifecycle](https://github.com/kubernetes/community/tree/master/sig-cluster-lifecycle), provides these packages pre-built for you,
|
||||
but you may also build them from source for other OSes.
|
||||
|
||||
|
||||
### kubeadm maturity
|
||||
|
||||
| Area | Maturity Level |
|
||||
|---------------------------|--------------- |
|
||||
| Command line UX | GA |
|
||||
| Implementation | GA |
|
||||
| Config file API | Beta |
|
||||
| CoreDNS | GA |
|
||||
| kubeadm alpha subcommands | Alpha |
|
||||
| High availability | Beta |
|
||||
| DynamicKubeletConfig | Alpha |
|
||||
|
||||
|
||||
kubeadm's overall feature state is **GA**. Some sub-features, like the configuration
|
||||
file API are still under active development. The implementation of creating the cluster
|
||||
may change slightly as the tool evolves, but the overall implementation should be pretty stable.
|
||||
Any commands under `kubeadm alpha` are by definition, supported on an alpha level.
|
||||
|
||||
|
||||
### Support timeframes
|
||||
|
||||
Kubernetes releases are generally supported for nine months, and during that
|
||||
period a patch release may be issued from the release branch if a severe bug or
|
||||
security issue is found. Here are the latest Kubernetes releases and the support
|
||||
timeframe; which also applies to `kubeadm`.
|
||||
|
||||
| Kubernetes version | Release month | End-of-life-month |
|
||||
|--------------------|----------------|-------------------|
|
||||
| v1.6.x | March 2017 | December 2017 |
|
||||
| v1.7.x | June 2017 | March 2018 |
|
||||
| v1.8.x | September 2017 | June 2018 |
|
||||
| v1.9.x | December 2017 | September 2018 |
|
||||
| v1.10.x | March 2018 | December 2018 |
|
||||
| v1.11.x | June 2018 | March 2019 |
|
||||
| v1.12.x | September 2018 | June 2019 |
|
||||
| v1.13.x | December 2018 | September 2019 |
|
||||
| v1.14.x | March 2019 | December 2019 |
|
||||
| v1.15.x | June 2019 | March 2020 |
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture prerequisites %}}
|
||||
|
||||
- One or more machines running a deb/rpm-compatible OS, for example Ubuntu or CentOS
|
||||
- 2 GB or more of RAM per machine. Any less leaves little room for your
|
||||
apps.
|
||||
- 2 CPUs or more on the control-plane node
|
||||
- Full network connectivity among all machines in the cluster. A public or
|
||||
private network is fine.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture steps %}}
|
||||
|
||||
## Objectives
|
||||
|
||||
* Install a single master Kubernetes cluster or [high availability cluster](/docs/setup/production-environment/tools/kubeadm/high-availability/)
|
||||
* Install a Pod network on the cluster so that your Pods can
|
||||
talk to each other
|
||||
|
||||
## Instructions
|
||||
|
||||
### Installing kubeadm on your hosts
|
||||
|
||||
See ["Installing kubeadm"](/docs/setup/production-environment/tools/kubeadm/install-kubeadm/).
|
||||
|
||||
{{< note >}}
|
||||
If you have already installed kubeadm, run `apt-get update &&
|
||||
apt-get upgrade` or `yum update` to get the latest version of kubeadm.
|
||||
|
||||
When you upgrade, the kubelet restarts every few seconds as it waits in a crashloop for
|
||||
kubeadm to tell it what to do. This crashloop is expected and normal.
|
||||
After you initialize your master, the kubelet runs normally.
|
||||
{{< /note >}}
|
||||
|
||||
### Initializing your control-plane node
|
||||
|
||||
The control-plane node is the machine where the control plane components run, including
|
||||
etcd (the cluster database) and the API server (which the kubectl CLI
|
||||
communicates with).
|
||||
|
||||
1. Choose a pod network add-on, and verify whether it requires any arguments to
|
||||
be passed to kubeadm initialization. Depending on which
|
||||
third-party provider you choose, you might need to set the `--pod-network-cidr` to
|
||||
a provider-specific value. See [Installing a pod network add-on](#pod-network).
|
||||
1. (Optional) Since version 1.14, kubeadm will try to detect the container runtime on Linux
|
||||
by using a list of well known domain socket paths. To use different container runtime or
|
||||
if there are more than one installed on the provisioned node, specify the `--cri-socket`
|
||||
argument to `kubeadm init`. See [Installing runtime](/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#installing-runtime).
|
||||
1. (Optional) Unless otherwise specified, kubeadm uses the network interface associated
|
||||
with the default gateway to advertise the master's IP. To use a different
|
||||
network interface, specify the `--apiserver-advertise-address=<ip-address>` argument
|
||||
to `kubeadm init`. To deploy an IPv6 Kubernetes cluster using IPv6 addressing, you
|
||||
must specify an IPv6 address, for example `--apiserver-advertise-address=fd00::101`
|
||||
1. (Optional) Run `kubeadm config images pull` prior to `kubeadm init` to verify
|
||||
connectivity to gcr.io registries.
|
||||
|
||||
Now run:
|
||||
|
||||
```bash
|
||||
kubeadm init <args>
|
||||
```
|
||||
|
||||
### More information
|
||||
|
||||
For more information about `kubeadm init` arguments, see the [kubeadm reference guide](/docs/reference/setup-tools/kubeadm/kubeadm/).
|
||||
|
||||
For a complete list of configuration options, see the [configuration file documentation](/docs/reference/setup-tools/kubeadm/kubeadm-init/#config-file).
|
||||
|
||||
To customize control plane components, including optional IPv6 assignment to liveness probe for control plane components and etcd server, provide extra arguments to each component as documented in [custom arguments](/docs/admin/kubeadm#custom-args).
|
||||
|
||||
To run `kubeadm init` again, you must first [tear down the cluster](#tear-down).
|
||||
|
||||
If you join a node with a different architecture to your cluster, create a separate
|
||||
Deployment or DaemonSet for `kube-proxy` and `kube-dns` on the node. This is because the Docker images for these
|
||||
components do not currently support multi-architecture.
|
||||
|
||||
`kubeadm init` first runs a series of prechecks to ensure that the machine
|
||||
is ready to run Kubernetes. These prechecks expose warnings and exit on errors. `kubeadm init`
|
||||
then downloads and installs the cluster control plane components. This may take several minutes.
|
||||
The output should look like:
|
||||
|
||||
```none
|
||||
[init] Using Kubernetes version: vX.Y.Z
|
||||
[preflight] Running pre-flight checks
|
||||
[preflight] Pulling images required for setting up a Kubernetes cluster
|
||||
[preflight] This might take a minute or two, depending on the speed of your internet connection
|
||||
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
|
||||
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
|
||||
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
|
||||
[kubelet-start] Activating the kubelet service
|
||||
[certs] Using certificateDir folder "/etc/kubernetes/pki"
|
||||
[certs] Generating "etcd/ca" certificate and key
|
||||
[certs] Generating "etcd/server" certificate and key
|
||||
[certs] etcd/server serving cert is signed for DNS names [kubeadm-master localhost] and IPs [10.138.0.4 127.0.0.1 ::1]
|
||||
[certs] Generating "etcd/healthcheck-client" certificate and key
|
||||
[certs] Generating "etcd/peer" certificate and key
|
||||
[certs] etcd/peer serving cert is signed for DNS names [kubeadm-master localhost] and IPs [10.138.0.4 127.0.0.1 ::1]
|
||||
[certs] Generating "apiserver-etcd-client" certificate and key
|
||||
[certs] Generating "ca" certificate and key
|
||||
[certs] Generating "apiserver" certificate and key
|
||||
[certs] apiserver serving cert is signed for DNS names [kubeadm-master kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 10.138.0.4]
|
||||
[certs] Generating "apiserver-kubelet-client" certificate and key
|
||||
[certs] Generating "front-proxy-ca" certificate and key
|
||||
[certs] Generating "front-proxy-client" certificate and key
|
||||
[certs] Generating "sa" key and public key
|
||||
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
|
||||
[kubeconfig] Writing "admin.conf" kubeconfig file
|
||||
[kubeconfig] Writing "kubelet.conf" kubeconfig file
|
||||
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
|
||||
[kubeconfig] Writing "scheduler.conf" kubeconfig file
|
||||
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
|
||||
[control-plane] Creating static Pod manifest for "kube-apiserver"
|
||||
[control-plane] Creating static Pod manifest for "kube-controller-manager"
|
||||
[control-plane] Creating static Pod manifest for "kube-scheduler"
|
||||
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
|
||||
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
|
||||
[apiclient] All control plane components are healthy after 31.501735 seconds
|
||||
[uploadconfig] storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
|
||||
[kubelet] Creating a ConfigMap "kubelet-config-X.Y" in namespace kube-system with the configuration for the kubelets in the cluster
|
||||
[patchnode] Uploading the CRI Socket information "/var/run/dockershim.sock" to the Node API object "kubeadm-master" as an annotation
|
||||
[mark-control-plane] Marking the node kubeadm-master as control-plane by adding the label "node-role.kubernetes.io/master=''"
|
||||
[mark-control-plane] Marking the node kubeadm-master as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]
|
||||
[bootstrap-token] Using token: <token>
|
||||
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
|
||||
[bootstraptoken] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
|
||||
[bootstraptoken] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
|
||||
[bootstraptoken] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
|
||||
[bootstraptoken] creating the "cluster-info" ConfigMap in the "kube-public" namespace
|
||||
[addons] Applied essential addon: CoreDNS
|
||||
[addons] Applied essential addon: kube-proxy
|
||||
|
||||
Your Kubernetes master has initialized successfully!
|
||||
|
||||
To start using your cluster, you need to run the following as a regular user:
|
||||
|
||||
mkdir -p $HOME/.kube
|
||||
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
|
||||
sudo chown $(id -u):$(id -g) $HOME/.kube/config
|
||||
|
||||
You should now deploy a pod network to the cluster.
|
||||
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
|
||||
/docs/concepts/cluster-administration/addons/
|
||||
|
||||
You can now join any number of machines by running the following on each node
|
||||
as root:
|
||||
|
||||
kubeadm join <master-ip>:<master-port> --token <token> --discovery-token-ca-cert-hash sha256:<hash>
|
||||
```
|
||||
|
||||
To make kubectl work for your non-root user, run these commands, which are
|
||||
also part of the `kubeadm init` output:
|
||||
|
||||
```bash
|
||||
mkdir -p $HOME/.kube
|
||||
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
|
||||
sudo chown $(id -u):$(id -g) $HOME/.kube/config
|
||||
```
|
||||
|
||||
Alternatively, if you are the `root` user, you can run:
|
||||
|
||||
```bash
|
||||
export KUBECONFIG=/etc/kubernetes/admin.conf
|
||||
```
|
||||
|
||||
Make a record of the `kubeadm join` command that `kubeadm init` outputs. You
|
||||
need this command to [join nodes to your cluster](#join-nodes).
|
||||
|
||||
The token is used for mutual authentication between the control-plane node and the joining
|
||||
nodes. The token included here is secret. Keep it safe, because anyone with this
|
||||
token can add authenticated nodes to your cluster. These tokens can be listed,
|
||||
created, and deleted with the `kubeadm token` command. See the
|
||||
[kubeadm reference guide](/docs/reference/setup-tools/kubeadm/kubeadm-token/).
|
||||
|
||||
### Installing a pod network add-on {#pod-network}
|
||||
|
||||
{{< caution >}}
|
||||
This section contains important information about installation and deployment order. Read it carefully before proceeding.
|
||||
{{< /caution >}}
|
||||
|
||||
You must install a pod network add-on so that your pods can communicate with
|
||||
each other.
|
||||
|
||||
**The network must be deployed before any applications. Also, CoreDNS will not start up before a network is installed.
|
||||
kubeadm only supports Container Network Interface (CNI) based networks (and does not support kubenet).**
|
||||
|
||||
Several projects provide Kubernetes pod networks using CNI, some of which also
|
||||
support [Network Policy](/docs/concepts/services-networking/networkpolicies/). See the [add-ons page](/docs/concepts/cluster-administration/addons/) for a complete list of available network add-ons.
|
||||
- IPv6 support was added in [CNI v0.6.0](https://github.com/containernetworking/cni/releases/tag/v0.6.0).
|
||||
- [CNI bridge](https://github.com/containernetworking/plugins/blob/master/plugins/main/bridge/README.md) and [local-ipam](https://github.com/containernetworking/plugins/blob/master/plugins/ipam/host-local/README.md) are the only supported IPv6 network plugins in Kubernetes version 1.9.
|
||||
|
||||
Note that kubeadm sets up a more secure cluster by default and enforces use of [RBAC](/docs/reference/access-authn-authz/rbac/).
|
||||
Make sure that your network manifest supports RBAC.
|
||||
|
||||
Also, beware, that your Pod network must not overlap with any of the host networks as this can cause issues.
|
||||
If you find a collision between your network plugin’s preferred Pod network and some of your host networks, you should think of a suitable CIDR replacement and use that during `kubeadm init` with `--pod-network-cidr` and as a replacement in your network plugin’s YAML.
|
||||
|
||||
You can install a pod network add-on with the following command:
|
||||
|
||||
```bash
|
||||
kubectl apply -f <add-on.yaml>
|
||||
```
|
||||
|
||||
You can install only one pod network per cluster.
|
||||
|
||||
{{< tabs name="tabs-pod-install" >}}
|
||||
{{% tab name="Choose one..." %}}
|
||||
Please select one of the tabs to see installation instructions for the respective third-party Pod Network Provider.
|
||||
{{% /tab %}}
|
||||
|
||||
{{% tab name="Calico" %}}
|
||||
For more information about using Calico, see [Quickstart for Calico on Kubernetes](https://docs.projectcalico.org/latest/getting-started/kubernetes/), [Installing Calico for policy and networking](https://docs.projectcalico.org/latest/getting-started/kubernetes/installation/calico), and other related resources.
|
||||
|
||||
For Calico to work correctly, you need to pass `--pod-network-cidr=192.168.0.0/16` to `kubeadm init` or update the `calico.yml` file to match your Pod network. Note that Calico works on `amd64`, `arm64`, and `ppc64le` only.
|
||||
|
||||
```shell
|
||||
kubectl apply -f https://docs.projectcalico.org/v3.7/manifests/calico.yaml
|
||||
```
|
||||
|
||||
{{% /tab %}}
|
||||
{{% tab name="Canal" %}}
|
||||
Canal uses Calico for policy and Flannel for networking. Refer to the Calico documentation for the [official getting started guide](https://docs.projectcalico.org/latest/getting-started/kubernetes/installation/flannel).
|
||||
|
||||
For Canal to work correctly, `--pod-network-cidr=10.244.0.0/16` has to be passed to `kubeadm init`. Note that Canal works on `amd64` only.
|
||||
|
||||
```shell
|
||||
kubectl apply -f https://docs.projectcalico.org/v3.7/manifests/canal.yaml
|
||||
```
|
||||
|
||||
{{% /tab %}}
|
||||
|
||||
{{% tab name="Cilium" %}}
|
||||
For more information about using Cilium with Kubernetes, see [Kubernetes Install guide for Cilium](https://docs.cilium.io/en/stable/kubernetes/).
|
||||
|
||||
For Cilium to work correctly, you must pass `--pod-network-cidr=10.217.0.0/16` to `kubeadm init`.
|
||||
|
||||
These commands will deploy Cilium with its own etcd managed by etcd operator.
|
||||
|
||||
_Note_: If you are running kubeadm in a single node please untaint it so that
|
||||
etcd-operator pods can be scheduled in the control-plane node.
|
||||
|
||||
```shell
|
||||
kubectl taint nodes <node-name> node-role.kubernetes.io/master:NoSchedule-
|
||||
```
|
||||
|
||||
To deploy Cilium you just need to run:
|
||||
|
||||
```shell
|
||||
kubectl create -f https://raw.githubusercontent.com/cilium/cilium/v1.5/examples/kubernetes/1.14/cilium.yaml
|
||||
```
|
||||
|
||||
Once all Cilium pods are marked as `READY`, you start using your cluster.
|
||||
|
||||
```shell
|
||||
kubectl get pods -n kube-system --selector=k8s-app=cilium
|
||||
```
|
||||
The output is similar to this:
|
||||
```
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
cilium-drxkl 1/1 Running 0 18m
|
||||
```
|
||||
|
||||
{{% /tab %}}
|
||||
{{% tab name="Flannel" %}}
|
||||
|
||||
For `flannel` to work correctly, you must pass `--pod-network-cidr=10.244.0.0/16` to `kubeadm init`.
|
||||
|
||||
Set `/proc/sys/net/bridge/bridge-nf-call-iptables` to `1` by running `sysctl net.bridge.bridge-nf-call-iptables=1`
|
||||
to pass bridged IPv4 traffic to iptables' chains. This is a requirement for some CNI plugins to work, for more information
|
||||
please see [here](/docs/concepts/cluster-administration/network-plugins/#network-plugin-requirements).
|
||||
|
||||
Make sure that your firewall rules allow UDP ports 8285 and 8472 traffic for all hosts participating in the overlay network.
|
||||
see [here
|
||||
](https://coreos.com/flannel/docs/latest/troubleshooting.html#firewalls).
|
||||
|
||||
Note that `flannel` works on `amd64`, `arm`, `arm64`, `ppc64le` and `s390x` under Linux.
|
||||
Windows (`amd64`) is claimed as supported in v0.11.0 but the usage is undocumented.
|
||||
|
||||
```shell
|
||||
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/62e44c867a2846fefb68bd5f178daf4da3095ccb/Documentation/kube-flannel.yml
|
||||
```
|
||||
|
||||
For more information about `flannel`, see [the CoreOS flannel repository on GitHub
|
||||
](https://github.com/coreos/flannel).
|
||||
{{% /tab %}}
|
||||
|
||||
{{% tab name="Kube-router" %}}
|
||||
Set `/proc/sys/net/bridge/bridge-nf-call-iptables` to `1` by running `sysctl net.bridge.bridge-nf-call-iptables=1`
|
||||
to pass bridged IPv4 traffic to iptables' chains. This is a requirement for some CNI plugins to work, for more information
|
||||
please see [here](/docs/concepts/cluster-administration/network-plugins/#network-plugin-requirements).
|
||||
|
||||
Kube-router relies on kube-controller-manager to allocate pod CIDR for the nodes. Therefore, use `kubeadm init` with the `--pod-network-cidr` flag.
|
||||
|
||||
Kube-router provides pod networking, network policy, and high-performing IP Virtual Server(IPVS)/Linux Virtual Server(LVS) based service proxy.
|
||||
|
||||
For information on setting up Kubernetes cluster with Kube-router using kubeadm, please see official [setup guide](https://github.com/cloudnativelabs/kube-router/blob/master/docs/kubeadm.md).
|
||||
{{% /tab %}}
|
||||
|
||||
{{% tab name="Romana" %}}
|
||||
Set `/proc/sys/net/bridge/bridge-nf-call-iptables` to `1` by running `sysctl net.bridge.bridge-nf-call-iptables=1`
|
||||
to pass bridged IPv4 traffic to iptables' chains. This is a requirement for some CNI plugins to work, for more information
|
||||
please see [here](/docs/concepts/cluster-administration/network-plugins/#network-plugin-requirements).
|
||||
|
||||
The official Romana set-up guide is [here](https://github.com/romana/romana/tree/master/containerize#using-kubeadm).
|
||||
|
||||
Romana works on `amd64` only.
|
||||
|
||||
```shell
|
||||
kubectl apply -f https://raw.githubusercontent.com/romana/romana/master/containerize/specs/romana-kubeadm.yml
|
||||
```
|
||||
{{% /tab %}}
|
||||
|
||||
{{% tab name="Weave Net" %}}
|
||||
Set `/proc/sys/net/bridge/bridge-nf-call-iptables` to `1` by running `sysctl net.bridge.bridge-nf-call-iptables=1`
|
||||
to pass bridged IPv4 traffic to iptables' chains. This is a requirement for some CNI plugins to work, for more information
|
||||
please see [here](/docs/concepts/cluster-administration/network-plugins/#network-plugin-requirements).
|
||||
|
||||
The official Weave Net set-up guide is [here](https://www.weave.works/docs/net/latest/kube-addon/).
|
||||
|
||||
Weave Net works on `amd64`, `arm`, `arm64` and `ppc64le` without any extra action required.
|
||||
Weave Net sets hairpin mode by default. This allows Pods to access themselves via their Service IP address
|
||||
if they don't know their PodIP.
|
||||
|
||||
```shell
|
||||
kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"
|
||||
```
|
||||
{{% /tab %}}
|
||||
|
||||
{{% tab name="JuniperContrail/TungstenFabric" %}}
|
||||
Provides overlay SDN solution, delivering multicloud networking, hybrid cloud networking,
|
||||
simultaneous overlay-underlay support, network policy enforcement, network isolation,
|
||||
service chaining and flexible load balancing.
|
||||
|
||||
There are multiple, flexible ways to install JuniperContrail/TungstenFabric CNI.
|
||||
|
||||
Kindly refer to this quickstart: [TungstenFabric](https://tungstenfabric.github.io/website/)
|
||||
{{% /tab %}}
|
||||
|
||||
{{% tab name="Contiv-VPP" %}}
|
||||
[Contiv-VPP](https://contivpp.io/) employs a programmable CNF vSwitch based on [FD.io VPP](https://fd.io/),
|
||||
offering feature-rich & high-performance cloud-native networking and services.
|
||||
|
||||
It implements k8s services and network policies in the user space (on VPP).
|
||||
|
||||
Please refer to this installation guide: [Contiv-VPP Manual Installation](https://github.com/contiv/vpp/blob/master/docs/setup/MANUAL_INSTALL.md)
|
||||
{{% /tab %}}
|
||||
|
||||
{{< /tabs >}}
|
||||
|
||||
|
||||
Once a pod network has been installed, you can confirm that it is working by
|
||||
checking that the CoreDNS pod is Running in the output of `kubectl get pods --all-namespaces`.
|
||||
And once the CoreDNS pod is up and running, you can continue by joining your nodes.
|
||||
|
||||
If your network is not working or CoreDNS is not in the Running state, checkout our [troubleshooting docs](/docs/setup/production-environment/tools/kubeadm/troubleshooting-kubeadm/).
|
||||
|
||||
### Control plane node isolation
|
||||
|
||||
By default, your cluster will not schedule pods on the control-plane node for security
|
||||
reasons. If you want to be able to schedule pods on the control-plane node, e.g. for a
|
||||
single-machine Kubernetes cluster for development, run:
|
||||
|
||||
```bash
|
||||
kubectl taint nodes --all node-role.kubernetes.io/master-
|
||||
```
|
||||
|
||||
With output looking something like:
|
||||
|
||||
```
|
||||
node "test-01" untainted
|
||||
taint "node-role.kubernetes.io/master:" not found
|
||||
taint "node-role.kubernetes.io/master:" not found
|
||||
```
|
||||
|
||||
This will remove the `node-role.kubernetes.io/master` taint from any nodes that
|
||||
have it, including the control-plane node, meaning that the scheduler will then be able
|
||||
to schedule pods everywhere.
|
||||
|
||||
### Joining your nodes {#join-nodes}
|
||||
|
||||
The nodes are where your workloads (containers and pods, etc) run. To add new nodes to your cluster do the following for each machine:
|
||||
|
||||
* SSH to the machine
|
||||
* Become root (e.g. `sudo su -`)
|
||||
* Run the command that was output by `kubeadm init`. For example:
|
||||
|
||||
``` bash
|
||||
kubeadm join --token <token> <master-ip>:<master-port> --discovery-token-ca-cert-hash sha256:<hash>
|
||||
```
|
||||
|
||||
If you do not have the token, you can get it by running the following command on the control-plane node:
|
||||
|
||||
``` bash
|
||||
kubeadm token list
|
||||
```
|
||||
|
||||
The output is similar to this:
|
||||
|
||||
``` console
|
||||
TOKEN TTL EXPIRES USAGES DESCRIPTION EXTRA GROUPS
|
||||
8ewj1p.9r9hcjoqgajrj4gi 23h 2018-06-12T02:51:28Z authentication, The default bootstrap system:
|
||||
signing token generated by bootstrappers:
|
||||
'kubeadm init'. kubeadm:
|
||||
default-node-token
|
||||
```
|
||||
|
||||
By default, tokens expire after 24 hours. If you are joining a node to the cluster after the current token has expired,
|
||||
you can create a new token by running the following command on the control-plane node:
|
||||
|
||||
``` bash
|
||||
kubeadm token create
|
||||
```
|
||||
|
||||
The output is similar to this:
|
||||
|
||||
``` console
|
||||
5didvk.d09sbcov8ph2amjw
|
||||
```
|
||||
|
||||
If you don't have the value of `--discovery-token-ca-cert-hash`, you can get it by running the following command chain on the control-plane node:
|
||||
|
||||
``` bash
|
||||
openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | \
|
||||
openssl dgst -sha256 -hex | sed 's/^.* //'
|
||||
```
|
||||
|
||||
The output is similar to this:
|
||||
|
||||
``` console
|
||||
8cb2de97839780a412b93877f8507ad6c94f73add17d5d7058e91741c9d5ec78
|
||||
```
|
||||
|
||||
{{< note >}}
|
||||
To specify an IPv6 tuple for `<master-ip>:<master-port>`, IPv6 address must be enclosed in square brackets, for example: `[fd00::101]:2073`.
|
||||
{{< /note >}}
|
||||
|
||||
The output should look something like:
|
||||
|
||||
```
|
||||
[preflight] Running pre-flight checks
|
||||
|
||||
... (log output of join workflow) ...
|
||||
|
||||
Node join complete:
|
||||
* Certificate signing request sent to master and response
|
||||
received.
|
||||
* Kubelet informed of new secure connection details.
|
||||
|
||||
Run 'kubectl get nodes' on the master to see this machine join.
|
||||
```
|
||||
|
||||
A few seconds later, you should notice this node in the output from `kubectl get
|
||||
nodes` when run on the control-plane node.
|
||||
|
||||
### (Optional) Controlling your cluster from machines other than the control-plane node
|
||||
|
||||
In order to get a kubectl on some other computer (e.g. laptop) to talk to your
|
||||
cluster, you need to copy the administrator kubeconfig file from your control-plane node
|
||||
to your workstation like this:
|
||||
|
||||
``` bash
|
||||
scp root@<master ip>:/etc/kubernetes/admin.conf .
|
||||
kubectl --kubeconfig ./admin.conf get nodes
|
||||
```
|
||||
|
||||
{{< note >}}
|
||||
The example above assumes SSH access is enabled for root. If that is not the
|
||||
case, you can copy the `admin.conf` file to be accessible by some other user
|
||||
and `scp` using that other user instead.
|
||||
|
||||
The `admin.conf` file gives the user _superuser_ privileges over the cluster.
|
||||
This file should be used sparingly. For normal users, it's recommended to
|
||||
generate an unique credential to which you whitelist privileges. You can do
|
||||
this with the `kubeadm alpha kubeconfig user --client-name <CN>`
|
||||
command. That command will print out a KubeConfig file to STDOUT which you
|
||||
should save to a file and distribute to your user. After that, whitelist
|
||||
privileges by using `kubectl create (cluster)rolebinding`.
|
||||
{{< /note >}}
|
||||
|
||||
### (Optional) Proxying API Server to localhost
|
||||
|
||||
If you want to connect to the API Server from outside the cluster you can use
|
||||
`kubectl proxy`:
|
||||
|
||||
```bash
|
||||
scp root@<master ip>:/etc/kubernetes/admin.conf .
|
||||
kubectl --kubeconfig ./admin.conf proxy
|
||||
```
|
||||
|
||||
You can now access the API Server locally at `http://localhost:8001/api/v1`
|
||||
|
||||
## Tear down {#tear-down}
|
||||
|
||||
To undo what kubeadm did, you should first [drain the
|
||||
node](/docs/reference/generated/kubectl/kubectl-commands#drain) and make
|
||||
sure that the node is empty before shutting it down.
|
||||
|
||||
Talking to the control-plane node with the appropriate credentials, run:
|
||||
|
||||
```bash
|
||||
kubectl drain <node name> --delete-local-data --force --ignore-daemonsets
|
||||
kubectl delete node <node name>
|
||||
```
|
||||
|
||||
Then, on the node being removed, reset all kubeadm installed state:
|
||||
|
||||
```bash
|
||||
kubeadm reset
|
||||
```
|
||||
|
||||
The reset process does not reset or clean up iptables rules or IPVS tables. If you wish to reset iptables, you must do so manually:
|
||||
|
||||
```bash
|
||||
iptables -F && iptables -t nat -F && iptables -t mangle -F && iptables -X
|
||||
```
|
||||
|
||||
If you want to reset the IPVS tables, you must run the following command:
|
||||
|
||||
```bash
|
||||
ipvsadm -C
|
||||
```
|
||||
|
||||
If you wish to start over simply run `kubeadm init` or `kubeadm join` with the
|
||||
appropriate arguments.
|
||||
|
||||
More options and information about the
|
||||
[`kubeadm reset command`](/docs/reference/setup-tools/kubeadm/kubeadm-reset/).
|
||||
|
||||
## Maintaining a cluster {#lifecycle}
|
||||
|
||||
Instructions for maintaining kubeadm clusters (e.g. upgrades,downgrades, etc.) can be found [here.](/docs/tasks/administer-cluster/kubeadm)
|
||||
|
||||
## Explore other add-ons {#other-addons}
|
||||
|
||||
See the [list of add-ons](/docs/concepts/cluster-administration/addons/) to explore other add-ons,
|
||||
including tools for logging, monitoring, network policy, visualization &
|
||||
control of your Kubernetes cluster.
|
||||
|
||||
## What's next {#whats-next}
|
||||
|
||||
* Verify that your cluster is running properly with [Sonobuoy](https://github.com/heptio/sonobuoy)
|
||||
* Learn about kubeadm's advanced usage in the [kubeadm reference documentation](/docs/reference/setup-tools/kubeadm/kubeadm)
|
||||
* Learn more about Kubernetes [concepts](/docs/concepts/) and [`kubectl`](/docs/user-guide/kubectl-overview/).
|
||||
* Configure log rotation. You can use **logrotate** for that. When using Docker, you can specify log rotation options for Docker daemon, for example `--log-driver=json-file --log-opt=max-size=10m --log-opt=max-file=5`. See [Configure and troubleshoot the Docker daemon](https://docs.docker.com/engine/admin/) for more details.
|
||||
|
||||
## Feedback {#feedback}
|
||||
|
||||
* For bugs, visit [kubeadm GitHub issue tracker](https://github.com/kubernetes/kubeadm/issues)
|
||||
* For support, visit kubeadm Slack Channel:
|
||||
[#kubeadm](https://kubernetes.slack.com/messages/kubeadm/)
|
||||
* General SIG Cluster Lifecycle Development Slack Channel:
|
||||
[#sig-cluster-lifecycle](https://kubernetes.slack.com/messages/sig-cluster-lifecycle/)
|
||||
* SIG Cluster Lifecycle [SIG information](#TODO)
|
||||
* SIG Cluster Lifecycle Mailing List:
|
||||
[kubernetes-sig-cluster-lifecycle](https://groups.google.com/forum/#!forum/kubernetes-sig-cluster-lifecycle)
|
||||
|
||||
## Version skew policy {#version-skew-policy}
|
||||
|
||||
The kubeadm CLI tool of version vX.Y may deploy clusters with a control plane of version vX.Y or vX.(Y-1).
|
||||
kubeadm CLI vX.Y can also upgrade an existing kubeadm-created cluster of version vX.(Y-1).
|
||||
|
||||
Due to that we can't see into the future, kubeadm CLI vX.Y may or may not be able to deploy vX.(Y+1) clusters.
|
||||
|
||||
Example: kubeadm v1.8 can deploy both v1.7 and v1.8 clusters and upgrade v1.7 kubeadm-created clusters to
|
||||
v1.8.
|
||||
|
||||
These resources provide more information on supported version skew between kubelets and the control plane, and other Kubernetes components:
|
||||
|
||||
* Kubernetes [version and version-skew policy](/docs/setup/release/version-skew-policy/)
|
||||
* Kubeadm-specific [installation guide](/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#installing-kubeadm-kubelet-and-kubectl)
|
||||
|
||||
## kubeadm works on multiple platforms {#multi-platform}
|
||||
|
||||
kubeadm deb/rpm packages and binaries are built for amd64, arm (32-bit), arm64, ppc64le, and s390x
|
||||
following the [multi-platform
|
||||
proposal](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/multi-platform.md).
|
||||
|
||||
Multiplatform container images for the control plane and addons are also supported since v1.12.
|
||||
|
||||
Only some of the network providers offer solutions for all platforms. Please consult the list of
|
||||
network providers above or the documentation from each provider to figure out whether the provider
|
||||
supports your chosen platform.
|
||||
|
||||
## Limitations {#limitations}
|
||||
|
||||
The cluster created here has a single control-plane node, with a single etcd database
|
||||
running on it. This means that if the control-plane node fails, your cluster may lose
|
||||
data and may need to be recreated from scratch.
|
||||
|
||||
Workarounds:
|
||||
|
||||
* Regularly [back up etcd](https://coreos.com/etcd/docs/latest/admin_guide.html). The
|
||||
etcd data directory configured by kubeadm is at `/var/lib/etcd` on the control-plane node.
|
||||
|
||||
* Use multiple control-plane nodes by completing the
|
||||
[HA setup](/docs/setup/independent/ha-topology) instead.
|
||||
|
||||
## Troubleshooting {#troubleshooting}
|
||||
|
||||
If you are running into difficulties with kubeadm, please consult our [troubleshooting docs](/docs/setup/production-environment/tools/kubeadm/troubleshooting-kubeadm/).
|
||||
+3
-3
@@ -1,7 +1,7 @@
|
||||
---
|
||||
reviewers:
|
||||
- sig-cluster-lifecycle
|
||||
title: Options for Highly Available Topology
|
||||
title: Options for Highly Available topology
|
||||
content_template: templates/concept
|
||||
weight: 50
|
||||
---
|
||||
@@ -66,6 +66,6 @@ A minimum of three hosts for control plane nodes and three hosts for etcd nodes
|
||||
|
||||
{{% capture whatsnext %}}
|
||||
|
||||
- [Set up a highly available cluster with kubeadm](/docs/setup/independent/high-availability/)
|
||||
- [Set up a highly available cluster with kubeadm](/docs/setup/production-environment/tools/kubeadm/high-availability/)
|
||||
|
||||
{{% /capture %}}
|
||||
{{% /capture %}}
|
||||
+10
-18
@@ -1,7 +1,7 @@
|
||||
---
|
||||
reviewers:
|
||||
- sig-cluster-lifecycle
|
||||
title: Creating Highly Available Clusters with kubeadm
|
||||
title: Creating Highly Available clusters with kubeadm
|
||||
content_template: templates/task
|
||||
weight: 60
|
||||
---
|
||||
@@ -17,7 +17,7 @@ and control plane nodes are co-located.
|
||||
control plane nodes and etcd members are separated.
|
||||
|
||||
Before proceeding, you should carefully consider which approach best meets the needs of your applications
|
||||
and environment. [This comparison topic](/docs/setup/independent/ha-topology/) outlines the advantages and disadvantages of each.
|
||||
and environment. [This comparison topic](/docs/setup/production-environment/tools/kubeadm/ha-topology/) outlines the advantages and disadvantages of each.
|
||||
|
||||
If you encounter issues with setting up the HA cluster, please provide us with feedback
|
||||
in the kubeadm [issue tracker](https://github.com/kubernetes/kubeadm/issues/new).
|
||||
@@ -36,12 +36,10 @@ LoadBalancer, or with dynamic PersistentVolumes.
|
||||
|
||||
For both methods you need this infrastructure:
|
||||
|
||||
- Three machines that meet [kubeadm's minimum
|
||||
requirements](/docs/setup/independent/install-kubeadm/#before-you-begin) for
|
||||
- Three machines that meet [kubeadm's minimum requirements](/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#before-you-begin) for
|
||||
the masters
|
||||
- Three machines that meet [kubeadm's minimum
|
||||
requirements](/docs/setup/independent/install-kubeadm/#before-you-begin) for
|
||||
the workers
|
||||
requirements](/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#before-you-begin) for the workers
|
||||
- Full network connectivity between all machines in the cluster (public or
|
||||
private network)
|
||||
- sudo privileges on all machines
|
||||
@@ -116,8 +114,7 @@ option. Your cluster requirements may need a different configuration.
|
||||
|
||||
{{< note >}}
|
||||
Some CNI network plugins like Calico require a CIDR such as `192.168.0.0/16` and
|
||||
some like Weave do not. See the [CNI network
|
||||
documentation](/docs/setup/independent/create-cluster-kubeadm/#pod-network).
|
||||
some like Weave do not. See the [CNI network documentation](/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/#pod-network).
|
||||
To add a pod CIDR set the `podSubnet: 192.168.0.0/16` field under
|
||||
the `networking` object of `ClusterConfiguration`.
|
||||
{{< /note >}}
|
||||
@@ -127,7 +124,7 @@ the `networking` object of `ClusterConfiguration`.
|
||||
```sh
|
||||
sudo kubeadm init --config=kubeadm-config.yaml --upload-certs
|
||||
```
|
||||
- The `--upload-certs` flags is used to upload the certificates that should be shared
|
||||
- The `--experimental-upload-certs` flag is used to upload the certificates that should be shared
|
||||
across all the control-plane instances to the cluster. If instead, you prefer to copy certs across
|
||||
control-plane nodes manually or using automation tools, please remove this flag and refer to [Manual
|
||||
certificate distribution](#manual-certs) section bellow.
|
||||
@@ -137,7 +134,7 @@ the `networking` object of `ClusterConfiguration`.
|
||||
```sh
|
||||
...
|
||||
You can now join any number of control-plane node by running the following command on each as a root:
|
||||
kubeadm join 192.168.0.200:6443 --token 9vr73a.a8uxyaju799qwdjv --discovery-token-ca-cert-hash sha256:7c2e69131a36ae2a042a339b33381c6d0d43887e2de83720eff5359e26aec866 --control-plane --certificate-key f8902e114ef118304e561c3ecd4d0b543adc226b7a07f675f56564185ffe0c07
|
||||
kubeadm join 192.168.0.200:6443 --token 9vr73a.a8uxyaju799qwdjv --discovery-token-ca-cert-hash sha256:7c2e69131a36ae2a042a339b33381c6d0d43887e2de83720eff5359e26aec866 --experimental-control-plane --certificate-key f8902e114ef118304e561c3ecd4d0b543adc226b7a07f675f56564185ffe0c07
|
||||
|
||||
Please note that the certificate-key gives access to cluster sensitive data, keep it secret!
|
||||
As a safeguard, uploaded-certs will be deleted in two hours; If necessary, you can use kubeadm init phase upload-certs to reload certs afterward.
|
||||
@@ -177,10 +174,7 @@ As stated in the command output, the certificate-key gives access to cluster sen
|
||||
{{< /caution >}}
|
||||
|
||||
1. Apply the CNI plugin of your choice:
|
||||
|
||||
[Follow these instructions](/docs/setup/independent/create-cluster-kubeadm/#pod-network) to install
|
||||
the CNI provider. Make sure the configuration corresponds to the Pod CIDR specified in the kubeadm
|
||||
configuration file if applicable.
|
||||
[Follow these instructions](/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/#pod-network) to install the CNI provider. Make sure the configuration corresponds to the Pod CIDR specified in the kubeadm configuration file if applicable.
|
||||
|
||||
In this example we are using Weave Net:
|
||||
|
||||
@@ -223,8 +217,7 @@ in the kubeadm config file.
|
||||
|
||||
### Set up the etcd cluster
|
||||
|
||||
1. Follow [these instructions](/docs/setup/independent/setup-ha-etcd-with-kubeadm/)
|
||||
to set up the etcd cluster.
|
||||
1. Follow [these instructions](/docs/setup/production-environment/tools/kubeadm/setup-ha-etcd-with-kubeadm/) to set up the etcd cluster.
|
||||
|
||||
1. Setup SSH as described [here](#manual-certs).
|
||||
|
||||
@@ -369,7 +362,7 @@ the creation of additional nodes could fail due to a lack of required SANs.
|
||||
{{< /caution >}}
|
||||
|
||||
1. Then on each joining control plane node you have to run the following script before running `kubeadm join`.
|
||||
This script will move the previously copied certificates from the home directory to `/etc/kuberentes/pki`:
|
||||
This script will move the previously copied certificates from the home directory to `/etc/kubernetes/pki`:
|
||||
|
||||
```sh
|
||||
USER=ubuntu # customizable
|
||||
@@ -383,5 +376,4 @@ the creation of additional nodes could fail due to a lack of required SANs.
|
||||
mv /home/${USER}/etcd-ca.crt /etc/kubernetes/pki/etcd/ca.crt
|
||||
mv /home/${USER}/etcd-ca.key /etc/kubernetes/pki/etcd/ca.key
|
||||
```
|
||||
|
||||
{{% /capture %}}
|
||||
@@ -0,0 +1,379 @@
|
||||
---
|
||||
reviewers:
|
||||
- sig-cluster-lifecycle
|
||||
title: Creating Highly Available clusters with kubeadm
|
||||
content_template: templates/task
|
||||
weight: 60
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
This page explains two different approaches to setting up a highly available Kubernetes
|
||||
cluster using kubeadm:
|
||||
|
||||
- With stacked control plane nodes. This approach requires less infrastructure. The etcd members
|
||||
and control plane nodes are co-located.
|
||||
- With an external etcd cluster. This approach requires more infrastructure. The
|
||||
control plane nodes and etcd members are separated.
|
||||
|
||||
Before proceeding, you should carefully consider which approach best meets the needs of your applications
|
||||
and environment. [This comparison topic](/docs/setup/production-environment/tools/kubeadm/ha-topology/) outlines the advantages and disadvantages of each.
|
||||
|
||||
If you encounter issues with setting up the HA cluster, please provide us with feedback
|
||||
in the kubeadm [issue tracker](https://github.com/kubernetes/kubeadm/issues/new).
|
||||
|
||||
See also [The upgrade documentation](/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade-1-15).
|
||||
|
||||
{{< caution >}}
|
||||
This page does not address running your cluster on a cloud provider. In a cloud
|
||||
environment, neither approach documented here works with Service objects of type
|
||||
LoadBalancer, or with dynamic PersistentVolumes.
|
||||
{{< /caution >}}
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture prerequisites %}}
|
||||
|
||||
For both methods you need this infrastructure:
|
||||
|
||||
- Three machines that meet [kubeadm's minimum requirements](/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#before-you-begin) for
|
||||
the masters
|
||||
- Three machines that meet [kubeadm's minimum
|
||||
requirements](/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#before-you-begin) for the workers
|
||||
- Full network connectivity between all machines in the cluster (public or
|
||||
private network)
|
||||
- sudo privileges on all machines
|
||||
- SSH access from one device to all nodes in the system
|
||||
- `kubeadm` and `kubelet` installed on all machines. `kubectl` is optional.
|
||||
|
||||
For the external etcd cluster only, you also need:
|
||||
|
||||
- Three additional machines for etcd members
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture steps %}}
|
||||
|
||||
## First steps for both methods
|
||||
|
||||
### Create load balancer for kube-apiserver
|
||||
|
||||
{{< note >}}
|
||||
There are many configurations for load balancers. The following example is only one
|
||||
option. Your cluster requirements may need a different configuration.
|
||||
{{< /note >}}
|
||||
|
||||
1. Create a kube-apiserver load balancer with a name that resolves to DNS.
|
||||
|
||||
- In a cloud environment you should place your control plane nodes behind a TCP
|
||||
forwarding load balancer. This load balancer distributes traffic to all
|
||||
healthy control plane nodes in its target list. The health check for
|
||||
an apiserver is a TCP check on the port the kube-apiserver listens on
|
||||
(default value `:6443`).
|
||||
|
||||
- It is not recommended to use an IP address directly in a cloud environment.
|
||||
|
||||
- The load balancer must be able to communicate with all control plane nodes
|
||||
on the apiserver port. It must also allow incoming traffic on its
|
||||
listening port.
|
||||
|
||||
- [HAProxy](http://www.haproxy.org/) can be used as a load balancer.
|
||||
|
||||
- Make sure the address of the load balancer always matches
|
||||
the address of kubeadm's `ControlPlaneEndpoint`.
|
||||
|
||||
1. Add the first control plane nodes to the load balancer and test the
|
||||
connection:
|
||||
|
||||
```sh
|
||||
nc -v LOAD_BALANCER_IP PORT
|
||||
```
|
||||
|
||||
- A connection refused error is expected because the apiserver is not yet
|
||||
running. A timeout, however, means the load balancer cannot communicate
|
||||
with the control plane node. If a timeout occurs, reconfigure the load
|
||||
balancer to communicate with the control plane node.
|
||||
|
||||
1. Add the remaining control plane nodes to the load balancer target group.
|
||||
|
||||
## Stacked control plane and etcd nodes
|
||||
|
||||
### Steps for the first control plane node
|
||||
|
||||
1. On the first control plane node, create a configuration file called `kubeadm-config.yaml`:
|
||||
|
||||
apiVersion: kubeadm.k8s.io/v1beta2
|
||||
kind: ClusterConfiguration
|
||||
kubernetesVersion: stable
|
||||
controlPlaneEndpoint: "LOAD_BALANCER_DNS:LOAD_BALANCER_PORT"
|
||||
|
||||
- `kubernetesVersion` should be set to the Kubernetes version to use. This
|
||||
example uses `stable`.
|
||||
- `controlPlaneEndpoint` should match the address or DNS and port of the load balancer.
|
||||
- It's recommended that the versions of kubeadm, kubelet, kubectl and Kubernetes match.
|
||||
|
||||
{{< note >}}
|
||||
Some CNI network plugins like Calico require a CIDR such as `192.168.0.0/16` and
|
||||
some like Weave do not. See the [CNI network documentation](/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/#pod-network).
|
||||
To add a pod CIDR set the `podSubnet: 192.168.0.0/16` field under
|
||||
the `networking` object of `ClusterConfiguration`.
|
||||
{{< /note >}}
|
||||
|
||||
1. Initialize the control plane:
|
||||
|
||||
```sh
|
||||
sudo kubeadm init --config=kubeadm-config.yaml --upload-certs
|
||||
```
|
||||
- The `--experimental-upload-certs` flag is used to upload the certificates that should be shared
|
||||
across all the control-plane instances to the cluster. If instead, you prefer to copy certs across
|
||||
control-plane nodes manually or using automation tools, please remove this flag and refer to [Manual
|
||||
certificate distribution](#manual-certs) section bellow.
|
||||
|
||||
After the command completes you should see something like so:
|
||||
|
||||
```sh
|
||||
...
|
||||
You can now join any number of control-plane node by running the following command on each as a root:
|
||||
kubeadm join 192.168.0.200:6443 --token 9vr73a.a8uxyaju799qwdjv --discovery-token-ca-cert-hash sha256:7c2e69131a36ae2a042a339b33381c6d0d43887e2de83720eff5359e26aec866 --experimental-control-plane --certificate-key f8902e114ef118304e561c3ecd4d0b543adc226b7a07f675f56564185ffe0c07
|
||||
|
||||
Please note that the certificate-key gives access to cluster sensitive data, keep it secret!
|
||||
As a safeguard, uploaded-certs will be deleted in two hours; If necessary, you can use kubeadm init phase upload-certs to reload certs afterward.
|
||||
|
||||
Then you can join any number of worker nodes by running the following on each as root:
|
||||
kubeadm join 192.168.0.200:6443 --token 9vr73a.a8uxyaju799qwdjv --discovery-token-ca-cert-hash sha256:7c2e69131a36ae2a042a339b33381c6d0d43887e2de83720eff5359e26aec866
|
||||
```
|
||||
|
||||
- Copy this output to a text file. You will need it later to join control plane and worker nodes to the cluster.
|
||||
- When `--upload-certs` is used with `kubeadm init`, the certificates of the primary control plane
|
||||
are encrypted and uploaded in the `kubeadm-certs` Secret.
|
||||
- To re-upload the certificates and generate a new decryption key, use the following command on a control plane
|
||||
node that is already joined to the cluster:
|
||||
|
||||
```sh
|
||||
sudo kubeadm init phase upload-certs --upload-certs
|
||||
```
|
||||
|
||||
- You can also specify a custom `--certificate-key` during `init` that can later be used by `join`.
|
||||
To generate such a key you can use the following command:
|
||||
|
||||
```sh
|
||||
kubeadm alpha certs certificate-key
|
||||
```
|
||||
|
||||
{{< note >}}
|
||||
The `kubeadm init` flags `--config` and `--certificate-key` cannot be mixed, therefore if you want
|
||||
to use the [kubeadm configuration](https://godoc.org/k8s.io/kubernetes/cmd/kubeadm/app/apis/kubeadm/v1beta2) you must add the `certificateKey` field in the appropriate config locations (under `InitConfiguration` and `JoinConfiguration: controlPlane`).
|
||||
{{< /note >}}
|
||||
|
||||
{{< note >}}
|
||||
The `kubeadm-certs` Secret and decryption key expire after two hours.
|
||||
{{< /note >}}
|
||||
|
||||
{{< caution >}}
|
||||
As stated in the command output, the certificate-key gives access to cluster sensitive data, keep it secret!
|
||||
{{< /caution >}}
|
||||
|
||||
1. Apply the CNI plugin of your choice:
|
||||
[Follow these instructions](/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/#pod-network) to install the CNI provider. Make sure the configuration corresponds to the Pod CIDR specified in the kubeadm configuration file if applicable.
|
||||
|
||||
In this example we are using Weave Net:
|
||||
|
||||
```sh
|
||||
kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"
|
||||
```
|
||||
|
||||
1. Type the following and watch the pods of the control plane components get started:
|
||||
|
||||
```sh
|
||||
kubectl get pod -n kube-system -w
|
||||
```
|
||||
|
||||
### Steps for the rest of the control plane nodes
|
||||
|
||||
{{< note >}}
|
||||
Since kubeadm version 1.15 you can join multiple control-plane nodes in parallel.
|
||||
Prior to this version, you must join new control plane nodes sequentially, only after
|
||||
the first node has finished initializing.
|
||||
{{< /note >}}
|
||||
|
||||
For each additional control plane node you should:
|
||||
|
||||
1. Execute the join command that was previously given to you by the `kubeadm init` output on the first node.
|
||||
It should look something like this:
|
||||
|
||||
```sh
|
||||
sudo kubeadm join 192.168.0.200:6443 --token 9vr73a.a8uxyaju799qwdjv --discovery-token-ca-cert-hash sha256:7c2e69131a36ae2a042a339b33381c6d0d43887e2de83720eff5359e26aec866 --control-plane --certificate-key f8902e114ef118304e561c3ecd4d0b543adc226b7a07f675f56564185ffe0c07
|
||||
```
|
||||
|
||||
- The `--control-plane` flag tells `kubeadm join` to create a new control plane.
|
||||
- The `--certificate-key ...` will cause the control plane certificates to be downloaded
|
||||
from the `kubeadm-certs` Secret in the cluster and be decrypted using the given key.
|
||||
|
||||
## External etcd nodes
|
||||
|
||||
Setting up a cluster with external etcd nodes is similar to the procedure used for stacked etcd
|
||||
with the exception that you should setup etcd first, and you should pass the etcd information
|
||||
in the kubeadm config file.
|
||||
|
||||
### Set up the etcd cluster
|
||||
|
||||
1. Follow [these instructions](/docs/setup/production-environment/tools/kubeadm/setup-ha-etcd-with-kubeadm/) to set up the etcd cluster.
|
||||
|
||||
1. Setup SSH as described [here](#manual-certs).
|
||||
|
||||
1. Copy the following files from any etcd node in the cluster to the first control plane node:
|
||||
|
||||
```sh
|
||||
export CONTROL_PLANE="ubuntu@10.0.0.7"
|
||||
scp /etc/kubernetes/pki/etcd/ca.crt "${CONTROL_PLANE}":
|
||||
scp /etc/kubernetes/pki/apiserver-etcd-client.crt "${CONTROL_PLANE}":
|
||||
scp /etc/kubernetes/pki/apiserver-etcd-client.key "${CONTROL_PLANE}":
|
||||
```
|
||||
|
||||
- Replace the value of `CONTROL_PLANE` with the `user@host` of the first control plane machine.
|
||||
|
||||
### Set up the first control plane node
|
||||
|
||||
1. Create a file called `kubeadm-config.yaml` with the following contents:
|
||||
|
||||
apiVersion: kubeadm.k8s.io/v1beta2
|
||||
kind: ClusterConfiguration
|
||||
kubernetesVersion: stable
|
||||
controlPlaneEndpoint: "LOAD_BALANCER_DNS:LOAD_BALANCER_PORT"
|
||||
etcd:
|
||||
external:
|
||||
endpoints:
|
||||
- https://ETCD_0_IP:2379
|
||||
- https://ETCD_1_IP:2379
|
||||
- https://ETCD_2_IP:2379
|
||||
caFile: /etc/kubernetes/pki/etcd/ca.crt
|
||||
certFile: /etc/kubernetes/pki/apiserver-etcd-client.crt
|
||||
keyFile: /etc/kubernetes/pki/apiserver-etcd-client.key
|
||||
|
||||
{{< note >}}
|
||||
The difference between stacked etcd and external etcd here is that we are using
|
||||
the `external` field for `etcd` in the kubeadm config. In the case of the stacked
|
||||
etcd topology this is managed automatically.
|
||||
{{< /note >}}
|
||||
|
||||
- Replace the following variables in the config template with the appropriate values for your cluster:
|
||||
|
||||
- `LOAD_BALANCER_DNS`
|
||||
- `LOAD_BALANCER_PORT`
|
||||
- `ETCD_0_IP`
|
||||
- `ETCD_1_IP`
|
||||
- `ETCD_2_IP`
|
||||
|
||||
The following steps are exactly the same as described for stacked etcd setup:
|
||||
|
||||
1. Run `sudo kubeadm init --config kubeadm-config.yaml --upload-certs` on this node.
|
||||
|
||||
1. Write the output join commands that are returned to a text file for later use.
|
||||
|
||||
1. Apply the CNI plugin of your choice. The given example is for Weave Net:
|
||||
|
||||
```sh
|
||||
kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"
|
||||
```
|
||||
|
||||
### Steps for the rest of the control plane nodes
|
||||
|
||||
The steps are the same as for the stacked etcd setup:
|
||||
|
||||
- Make sure the first control plane node is fully initialized.
|
||||
- Join each control plane node with the join command you saved to a text file. It's recommended
|
||||
to join the control plane nodes one at a time.
|
||||
- Don't forget that the decryption key from `--certificate-key` expires after two hours, by default.
|
||||
|
||||
## Common tasks after bootstrapping control plane
|
||||
|
||||
### Install workers
|
||||
|
||||
Worker nodes can be joined to the cluster with the command you stored previously
|
||||
as the output from the `kubeadm init` command:
|
||||
|
||||
```sh
|
||||
sudo kubeadm join 192.168.0.200:6443 --token 9vr73a.a8uxyaju799qwdjv --discovery-token-ca-cert-hash sha256:7c2e69131a36ae2a042a339b33381c6d0d43887e2de83720eff5359e26aec866
|
||||
```
|
||||
|
||||
## Manual certificate distribution {#manual-certs}
|
||||
|
||||
If you choose to not use `kubeadm init` with the `--upload-certs` flag this means that
|
||||
you are going to have to manually copy the certificates from the primary control plane node to the
|
||||
joining control plane nodes.
|
||||
|
||||
There are many ways to do this. In the following example we are using `ssh` and `scp`:
|
||||
|
||||
SSH is required if you want to control all nodes from a single machine.
|
||||
|
||||
1. Enable ssh-agent on your main device that has access to all other nodes in
|
||||
the system:
|
||||
|
||||
```
|
||||
eval $(ssh-agent)
|
||||
```
|
||||
|
||||
1. Add your SSH identity to the session:
|
||||
|
||||
```
|
||||
ssh-add ~/.ssh/path_to_private_key
|
||||
```
|
||||
|
||||
1. SSH between nodes to check that the connection is working correctly.
|
||||
|
||||
- When you SSH to any node, make sure to add the `-A` flag:
|
||||
|
||||
```
|
||||
ssh -A 10.0.0.7
|
||||
```
|
||||
|
||||
- When using sudo on any node, make sure to preserve the environment so SSH
|
||||
forwarding works:
|
||||
|
||||
```
|
||||
sudo -E -s
|
||||
```
|
||||
|
||||
1. After configuring SSH on all the nodes you should run the following script on the first control plane node after
|
||||
running `kubeadm init`. This script will copy the certificates from the first control plane node to the other
|
||||
control plane nodes:
|
||||
|
||||
In the following example, replace `CONTROL_PLANE_IPS` with the IP addresses of the
|
||||
other control plane nodes.
|
||||
```sh
|
||||
USER=ubuntu # customizable
|
||||
CONTROL_PLANE_IPS="10.0.0.7 10.0.0.8"
|
||||
for host in ${CONTROL_PLANE_IPS}; do
|
||||
scp /etc/kubernetes/pki/ca.crt "${USER}"@$host:
|
||||
scp /etc/kubernetes/pki/ca.key "${USER}"@$host:
|
||||
scp /etc/kubernetes/pki/sa.key "${USER}"@$host:
|
||||
scp /etc/kubernetes/pki/sa.pub "${USER}"@$host:
|
||||
scp /etc/kubernetes/pki/front-proxy-ca.crt "${USER}"@$host:
|
||||
scp /etc/kubernetes/pki/front-proxy-ca.key "${USER}"@$host:
|
||||
scp /etc/kubernetes/pki/etcd/ca.crt "${USER}"@$host:etcd-ca.crt
|
||||
scp /etc/kubernetes/pki/etcd/ca.key "${USER}"@$host:etcd-ca.key
|
||||
done
|
||||
```
|
||||
|
||||
{{< caution >}}
|
||||
Copy only the certificates in the above list. kubeadm will take care of generating the rest of the certificates
|
||||
with the required SANs for the joining control-plane instances. If you copy all the certificates by mistake,
|
||||
the creation of additional nodes could fail due to a lack of required SANs.
|
||||
{{< /caution >}}
|
||||
|
||||
1. Then on each joining control plane node you have to run the following script before running `kubeadm join`.
|
||||
This script will move the previously copied certificates from the home directory to `/etc/kubernetes/pki`:
|
||||
|
||||
```sh
|
||||
USER=ubuntu # customizable
|
||||
mkdir -p /etc/kubernetes/pki/etcd
|
||||
mv /home/${USER}/ca.crt /etc/kubernetes/pki/
|
||||
mv /home/${USER}/ca.key /etc/kubernetes/pki/
|
||||
mv /home/${USER}/sa.pub /etc/kubernetes/pki/
|
||||
mv /home/${USER}/sa.key /etc/kubernetes/pki/
|
||||
mv /home/${USER}/front-proxy-ca.crt /etc/kubernetes/pki/
|
||||
mv /home/${USER}/front-proxy-ca.key /etc/kubernetes/pki/
|
||||
mv /home/${USER}/etcd-ca.crt /etc/kubernetes/pki/etcd/ca.crt
|
||||
mv /home/${USER}/etcd-ca.key /etc/kubernetes/pki/etcd/ca.key
|
||||
```
|
||||
{{% /capture %}}
|
||||
+11
-9
@@ -1,7 +1,7 @@
|
||||
---
|
||||
title: Installing kubeadm
|
||||
content_template: templates/task
|
||||
weight: 20
|
||||
weight: 10
|
||||
card:
|
||||
name: setup
|
||||
weight: 20
|
||||
@@ -10,9 +10,8 @@ card:
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cncf/artwork/master/kubernetes/certified-kubernetes/versionless/color/certified-kubernetes-color.png" align="right" width="150px">This page shows how to install the `kubeadm` toolbox.
|
||||
For information how to create a cluster with kubeadm once you have performed this installation process,
|
||||
see the [Using kubeadm to Create a Cluster](/docs/setup/independent/create-cluster-kubeadm/) page.
|
||||
<img src="https://raw.githubusercontent.com/cncf/artwork/master/projects/kubernetes/certified-kubernetes/versionless/color/certified-kubernetes-color.png" align="right" width="150px">This page shows how to install the `kubeadm` toolbox.
|
||||
For information how to create a cluster with kubeadm once you have performed this installation process, see the [Using kubeadm to Create a Cluster](/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/) page.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
@@ -134,16 +133,18 @@ kubelet and the control plane is supported, but the kubelet version may never ex
|
||||
server version. For example, kubelets running 1.7.0 should be fully compatible with a 1.8.0 API server,
|
||||
but not vice versa.
|
||||
|
||||
For information about installing `kubectl`, see [Install and set up kubectl](/docs/tasks/tools/install-kubectl/).
|
||||
|
||||
{{< warning >}}
|
||||
These instructions exclude all Kubernetes packages from any system upgrades.
|
||||
This is because kubeadm and Kubernetes require
|
||||
[special attention to upgrade](/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade-1-11/).
|
||||
[special attention to upgrade](/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade-1-14/).
|
||||
{{</ warning >}}
|
||||
|
||||
For more information on version skews, see:
|
||||
|
||||
* Kubernetes [version and version-skew policy](/docs/setup/version-skew-policy/)
|
||||
* Kubeadm-specific [version skew policy](/docs/setup/independent/create-cluster-kubeadm/#version-skew-policy)
|
||||
* Kubernetes [version and version-skew policy](/docs/setup/release/version-skew-policy/)
|
||||
* Kubeadm-specific [version skew policy](/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/#version-skew-policy)
|
||||
|
||||
{{< tabs name="k8s_install" >}}
|
||||
{{% tab name="Ubuntu, Debian or HypriotOS" %}}
|
||||
@@ -269,12 +270,13 @@ systemctl restart kubelet
|
||||
The automatic detection of cgroup driver for other container runtimes
|
||||
like CRI-O and containerd is work in progress.
|
||||
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
If you are running into difficulties with kubeadm, please consult our [troubleshooting docs](/docs/setup/independent/troubleshooting-kubeadm/).
|
||||
If you are running into difficulties with kubeadm, please consult our [troubleshooting docs](/docs/setup/production-environment/tools/kubeadm/troubleshooting-kubeadm/).
|
||||
|
||||
{{% capture whatsnext %}}
|
||||
|
||||
* [Using kubeadm to Create a Cluster](/docs/setup/independent/create-cluster-kubeadm/)
|
||||
* [Using kubeadm to Create a Cluster](/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/)
|
||||
|
||||
{{% /capture %}}
|
||||
@@ -0,0 +1,282 @@
|
||||
---
|
||||
title: Installing kubeadm
|
||||
content_template: templates/task
|
||||
weight: 10
|
||||
card:
|
||||
name: setup
|
||||
weight: 20
|
||||
title: Install the kubeadm setup tool
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cncf/artwork/master/projects/kubernetes/certified-kubernetes/versionless/color/certified-kubernetes-color.png" align="right" width="150px">This page shows how to install the `kubeadm` toolbox.
|
||||
For information how to create a cluster with kubeadm once you have performed this installation process, see the [Using kubeadm to Create a Cluster](/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/) page.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture prerequisites %}}
|
||||
|
||||
* One or more machines running one of:
|
||||
- Ubuntu 16.04+
|
||||
- Debian 9
|
||||
- CentOS 7
|
||||
- RHEL 7
|
||||
- Fedora 25/26 (best-effort)
|
||||
- HypriotOS v1.0.1+
|
||||
- Container Linux (tested with 1800.6.0)
|
||||
* 2 GB or more of RAM per machine (any less will leave little room for your apps)
|
||||
* 2 CPUs or more
|
||||
* Full network connectivity between all machines in the cluster (public or private network is fine)
|
||||
* Unique hostname, MAC address, and product_uuid for every node. See [here](#verify-the-mac-address-and-product-uuid-are-unique-for-every-node) for more details.
|
||||
* Certain ports are open on your machines. See [here](#check-required-ports) for more details.
|
||||
* Swap disabled. You **MUST** disable swap in order for the kubelet to work properly.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture steps %}}
|
||||
|
||||
## Verify the MAC address and product_uuid are unique for every node
|
||||
|
||||
* You can get the MAC address of the network interfaces using the command `ip link` or `ifconfig -a`
|
||||
* The product_uuid can be checked by using the command `sudo cat /sys/class/dmi/id/product_uuid`
|
||||
|
||||
It is very likely that hardware devices will have unique addresses, although some virtual machines may have
|
||||
identical values. Kubernetes uses these values to uniquely identify the nodes in the cluster.
|
||||
If these values are not unique to each node, the installation process
|
||||
may [fail](https://github.com/kubernetes/kubeadm/issues/31).
|
||||
|
||||
## Check network adapters
|
||||
|
||||
If you have more than one network adapter, and your Kubernetes components are not reachable on the default
|
||||
route, we recommend you add IP route(s) so Kubernetes cluster addresses go via the appropriate adapter.
|
||||
|
||||
## Check required ports
|
||||
|
||||
### Control-plane node(s)
|
||||
|
||||
| Protocol | Direction | Port Range | Purpose | Used By |
|
||||
|----------|-----------|------------|-------------------------|---------------------------|
|
||||
| TCP | Inbound | 6443* | Kubernetes API server | All |
|
||||
| TCP | Inbound | 2379-2380 | etcd server client API | kube-apiserver, etcd |
|
||||
| TCP | Inbound | 10250 | Kubelet API | Self, Control plane |
|
||||
| TCP | Inbound | 10251 | kube-scheduler | Self |
|
||||
| TCP | Inbound | 10252 | kube-controller-manager | Self |
|
||||
|
||||
### Worker node(s)
|
||||
|
||||
| Protocol | Direction | Port Range | Purpose | Used By |
|
||||
|----------|-----------|-------------|-----------------------|-------------------------|
|
||||
| TCP | Inbound | 10250 | Kubelet API | Self, Control plane |
|
||||
| TCP | Inbound | 30000-32767 | NodePort Services** | All |
|
||||
|
||||
** Default port range for [NodePort Services](/docs/concepts/services-networking/service/).
|
||||
|
||||
Any port numbers marked with * are overridable, so you will need to ensure any
|
||||
custom ports you provide are also open.
|
||||
|
||||
Although etcd ports are included in control-plane nodes, you can also host your own
|
||||
etcd cluster externally or on custom ports.
|
||||
|
||||
The pod network plugin you use (see below) may also require certain ports to be
|
||||
open. Since this differs with each pod network plugin, please see the
|
||||
documentation for the plugins about what port(s) those need.
|
||||
|
||||
## Installing runtime {#installing-runtime}
|
||||
|
||||
Since v1.6.0, Kubernetes has enabled the use of CRI, Container Runtime Interface, by default.
|
||||
|
||||
Since v1.14.0, kubeadm will try to automatically detect the container runtime on Linux nodes
|
||||
by scanning through a list of well known domain sockets. The detectable runtimes and the
|
||||
socket paths, that are used, can be found in the table below.
|
||||
|
||||
| Runtime | Domain Socket |
|
||||
|------------|----------------------------------|
|
||||
| Docker | /var/run/docker.sock |
|
||||
| containerd | /run/containerd/containerd.sock |
|
||||
| CRI-O | /var/run/crio/crio.sock |
|
||||
|
||||
If both Docker and containerd are detected together, Docker takes precedence. This is
|
||||
needed, because Docker 18.09 ships with containerd and both are detectable.
|
||||
If any other two or more runtimes are detected, kubeadm will exit with an appropriate
|
||||
error message.
|
||||
|
||||
On non-Linux nodes the container runtime used by default is Docker.
|
||||
|
||||
If the container runtime of choice is Docker, it is used through the built-in
|
||||
`dockershim` CRI implementation inside of the `kubelet`.
|
||||
|
||||
Other CRI-based runtimes include:
|
||||
|
||||
- [containerd](https://github.com/containerd/cri) (CRI plugin built into containerd)
|
||||
- [cri-o](https://github.com/kubernetes-incubator/cri-o)
|
||||
- [frakti](https://github.com/kubernetes/frakti)
|
||||
|
||||
Refer to the [CRI installation instructions](/docs/setup/cri) for more information.
|
||||
|
||||
## Installing kubeadm, kubelet and kubectl
|
||||
|
||||
You will install these packages on all of your machines:
|
||||
|
||||
* `kubeadm`: the command to bootstrap the cluster.
|
||||
|
||||
* `kubelet`: the component that runs on all of the machines in your cluster
|
||||
and does things like starting pods and containers.
|
||||
|
||||
* `kubectl`: the command line util to talk to your cluster.
|
||||
|
||||
kubeadm **will not** install or manage `kubelet` or `kubectl` for you, so you will
|
||||
need to ensure they match the version of the Kubernetes control plane you want
|
||||
kubeadm to install for you. If you do not, there is a risk of a version skew occurring that
|
||||
can lead to unexpected, buggy behaviour. However, _one_ minor version skew between the
|
||||
kubelet and the control plane is supported, but the kubelet version may never exceed the API
|
||||
server version. For example, kubelets running 1.7.0 should be fully compatible with a 1.8.0 API server,
|
||||
but not vice versa.
|
||||
|
||||
For information about installing `kubectl`, see [Install and set up kubectl](/docs/tasks/tools/install-kubectl/).
|
||||
|
||||
{{< warning >}}
|
||||
These instructions exclude all Kubernetes packages from any system upgrades.
|
||||
This is because kubeadm and Kubernetes require
|
||||
[special attention to upgrade](/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade-1-14/).
|
||||
{{</ warning >}}
|
||||
|
||||
For more information on version skews, see:
|
||||
|
||||
* Kubernetes [version and version-skew policy](/docs/setup/release/version-skew-policy/)
|
||||
* Kubeadm-specific [version skew policy](/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/#version-skew-policy)
|
||||
|
||||
{{< tabs name="k8s_install" >}}
|
||||
{{% tab name="Ubuntu, Debian or HypriotOS" %}}
|
||||
```bash
|
||||
apt-get update && apt-get install -y apt-transport-https curl
|
||||
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
|
||||
cat <<EOF >/etc/apt/sources.list.d/kubernetes.list
|
||||
deb https://apt.kubernetes.io/ kubernetes-xenial main
|
||||
EOF
|
||||
apt-get update
|
||||
apt-get install -y kubelet kubeadm kubectl
|
||||
apt-mark hold kubelet kubeadm kubectl
|
||||
```
|
||||
{{% /tab %}}
|
||||
{{% tab name="CentOS, RHEL or Fedora" %}}
|
||||
```bash
|
||||
cat <<EOF > /etc/yum.repos.d/kubernetes.repo
|
||||
[kubernetes]
|
||||
name=Kubernetes
|
||||
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
|
||||
enabled=1
|
||||
gpgcheck=1
|
||||
repo_gpgcheck=1
|
||||
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
|
||||
exclude=kube*
|
||||
EOF
|
||||
|
||||
# Set SELinux in permissive mode (effectively disabling it)
|
||||
setenforce 0
|
||||
sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config
|
||||
|
||||
yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes
|
||||
|
||||
systemctl enable --now kubelet
|
||||
```
|
||||
|
||||
**Note:**
|
||||
|
||||
- Setting SELinux in permissive mode by running `setenforce 0` and `sed ...` effectively disables it.
|
||||
This is required to allow containers to access the host filesystem, which is needed by pod networks for example.
|
||||
You have to do this until SELinux support is improved in the kubelet.
|
||||
- Some users on RHEL/CentOS 7 have reported issues with traffic being routed incorrectly due to iptables being bypassed. You should ensure
|
||||
`net.bridge.bridge-nf-call-iptables` is set to 1 in your `sysctl` config, e.g.
|
||||
|
||||
```bash
|
||||
cat <<EOF > /etc/sysctl.d/k8s.conf
|
||||
net.bridge.bridge-nf-call-ip6tables = 1
|
||||
net.bridge.bridge-nf-call-iptables = 1
|
||||
EOF
|
||||
sysctl --system
|
||||
```
|
||||
- Make sure that the `br_netfilter` module is loaded before this step. This can be done by running `lsmod | grep br_netfilter`. To load it explicitly call `modprobe br_netfilter`.
|
||||
{{% /tab %}}
|
||||
{{% tab name="Container Linux" %}}
|
||||
Install CNI plugins (required for most pod network):
|
||||
|
||||
```bash
|
||||
CNI_VERSION="v0.7.5"
|
||||
mkdir -p /opt/cni/bin
|
||||
curl -L "https://github.com/containernetworking/plugins/releases/download/${CNI_VERSION}/cni-plugins-amd64-${CNI_VERSION}.tgz" | tar -C /opt/cni/bin -xz
|
||||
```
|
||||
|
||||
Install crictl (required for kubeadm / Kubelet Container Runtime Interface (CRI))
|
||||
|
||||
```bash
|
||||
CRICTL_VERSION="v1.12.0"
|
||||
mkdir -p /opt/bin
|
||||
curl -L "https://github.com/kubernetes-incubator/cri-tools/releases/download/${CRICTL_VERSION}/crictl-${CRICTL_VERSION}-linux-amd64.tar.gz" | tar -C /opt/bin -xz
|
||||
```
|
||||
|
||||
Install `kubeadm`, `kubelet`, `kubectl` and add a `kubelet` systemd service:
|
||||
|
||||
```bash
|
||||
RELEASE="$(curl -sSL https://dl.k8s.io/release/stable.txt)"
|
||||
|
||||
mkdir -p /opt/bin
|
||||
cd /opt/bin
|
||||
curl -L --remote-name-all https://storage.googleapis.com/kubernetes-release/release/${RELEASE}/bin/linux/amd64/{kubeadm,kubelet,kubectl}
|
||||
chmod +x {kubeadm,kubelet,kubectl}
|
||||
|
||||
curl -sSL "https://raw.githubusercontent.com/kubernetes/kubernetes/${RELEASE}/build/debs/kubelet.service" | sed "s:/usr/bin:/opt/bin:g" > /etc/systemd/system/kubelet.service
|
||||
mkdir -p /etc/systemd/system/kubelet.service.d
|
||||
curl -sSL "https://raw.githubusercontent.com/kubernetes/kubernetes/${RELEASE}/build/debs/10-kubeadm.conf" | sed "s:/usr/bin:/opt/bin:g" > /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
|
||||
```
|
||||
|
||||
Enable and start `kubelet`:
|
||||
|
||||
```bash
|
||||
systemctl enable --now kubelet
|
||||
```
|
||||
{{% /tab %}}
|
||||
{{< /tabs >}}
|
||||
|
||||
|
||||
The kubelet is now restarting every few seconds, as it waits in a crashloop for
|
||||
kubeadm to tell it what to do.
|
||||
|
||||
## Configure cgroup driver used by kubelet on control-plane node
|
||||
|
||||
When using Docker, kubeadm will automatically detect the cgroup driver for the kubelet
|
||||
and set it in the `/var/lib/kubelet/kubeadm-flags.env` file during runtime.
|
||||
|
||||
If you are using a different CRI, you have to modify the file
|
||||
`/etc/default/kubelet` with your `cgroup-driver` value, like so:
|
||||
|
||||
```bash
|
||||
KUBELET_EXTRA_ARGS=--cgroup-driver=<value>
|
||||
```
|
||||
|
||||
This file will be used by `kubeadm init` and `kubeadm join` to source extra
|
||||
user defined arguments for the kubelet.
|
||||
|
||||
Please mind, that you **only** have to do that if the cgroup driver of your CRI
|
||||
is not `cgroupfs`, because that is the default value in the kubelet already.
|
||||
|
||||
Restarting the kubelet is required:
|
||||
|
||||
```bash
|
||||
systemctl daemon-reload
|
||||
systemctl restart kubelet
|
||||
```
|
||||
|
||||
The automatic detection of cgroup driver for other container runtimes
|
||||
like CRI-O and containerd is work in progress.
|
||||
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
If you are running into difficulties with kubeadm, please consult our [troubleshooting docs](/docs/setup/production-environment/tools/kubeadm/troubleshooting-kubeadm/).
|
||||
|
||||
{{% capture whatsnext %}}
|
||||
|
||||
* [Using kubeadm to Create a Cluster](/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/)
|
||||
|
||||
{{% /capture %}}
|
||||
+1
-1
@@ -96,7 +96,7 @@ such as systemd.
|
||||
It is possible to configure the kubelet that kubeadm will start if a custom `KubeletConfiguration`
|
||||
API object is passed with a configuration file like so `kubeadm ... --config some-config-file.yaml`.
|
||||
|
||||
By calling `kubeadm config print-default --api-objects KubeletConfiguration` you can
|
||||
By calling `kubeadm config print init-defaults --component-configs KubeletConfiguration` you can
|
||||
see all the default values for this structure.
|
||||
|
||||
Also have a look at the [API reference for the
|
||||
+2
-4
@@ -26,7 +26,7 @@ when using kubeadm to set up a kubernetes cluster.
|
||||
* Some infrastructure to copy files between hosts. For example `ssh` and `scp`
|
||||
can satisfy this requirement.
|
||||
|
||||
[toolbox]: /docs/setup/independent/install-kubeadm/
|
||||
[toolbox]: /docs/setup/production-environment/tools/kubeadm/install-kubeadm/
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
@@ -259,8 +259,6 @@ this example.
|
||||
|
||||
Once you have a working 3 member etcd cluster, you can continue setting up a
|
||||
highly available control plane using the [external etcd method with
|
||||
kubeadm](/docs/setup/independent/high-availability/).
|
||||
kubeadm](/docs/setup/production-environment/tools/kubeadm/high-availability/).
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
|
||||
+25
-9
@@ -1,7 +1,7 @@
|
||||
---
|
||||
title: Troubleshooting kubeadm
|
||||
content_template: templates/concept
|
||||
weight: 90
|
||||
weight: 20
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
@@ -58,10 +58,10 @@ This may be caused by a number of problems. The most common are:
|
||||
There are two common ways to fix the cgroup driver problem:
|
||||
|
||||
1. Install Docker again following instructions
|
||||
[here](/docs/setup/independent/install-kubeadm/#installing-docker).
|
||||
[here](/docs/setup/production-environment/container-runtimes/#docker).
|
||||
|
||||
1. Change the kubelet config to match the Docker cgroup driver manually, you can refer to
|
||||
[Configure cgroup driver used by kubelet on control-plane node](/docs/setup/independent/install-kubeadm/#configure-cgroup-driver-used-by-kubelet-on-master-node)
|
||||
for detailed instructions.
|
||||
[Configure cgroup driver used by kubelet on Master Node](/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#configure-cgroup-driver-used-by-kubelet-on-master-node)
|
||||
|
||||
- control plane Docker containers are crashlooping or hanging. You can check this by running `docker ps` and investigating each container by running `docker logs`.
|
||||
|
||||
@@ -113,7 +113,7 @@ Right after `kubeadm init` there should not be any pods in these states.
|
||||
This is **expected** and part of the design. kubeadm is network provider-agnostic, so the admin
|
||||
should [install the pod network solution](/docs/concepts/cluster-administration/addons/)
|
||||
of choice. You have to install a Pod Network
|
||||
before CoreDNS may deployed fully. Hence the `Pending` state before the network is set up.
|
||||
before CoreDNS may be deployed fully. Hence the `Pending` state before the network is set up.
|
||||
|
||||
## `HostPort` services do not work
|
||||
|
||||
@@ -219,7 +219,8 @@ Error from server: Get https://10.19.0.41:10250/containerLogs/default/mysql-ddc6
|
||||
If you have nodes that are running SELinux with an older version of Docker you might experience a scenario
|
||||
where the `coredns` pods are not starting. To solve that you can try one of the following options:
|
||||
|
||||
- Upgrade to a [newer version of Docker](/docs/setup/independent/install-kubeadm/#installing-docker).
|
||||
- Upgrade to a [newer version of Docker](/docs/setup/production-environment/container-runtimes/#docker).
|
||||
|
||||
- [Disable SELinux](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/security-enhanced_linux/sect-security-enhanced_linux-enabling_and_disabling_selinux-disabling_selinux).
|
||||
- Modify the `coredns` deployment to set `allowPrivilegeEscalation` to `true`:
|
||||
|
||||
@@ -273,12 +274,11 @@ If you decide to pass an argument that supports multiple, comma-separated values
|
||||
`--apiserver-extra-args` expects `key=value` pairs and in this case `NamespacesExists` is considered
|
||||
as a key that is missing a value.
|
||||
|
||||
Alternativelly, you can try separating the `key=value` pairs like so:
|
||||
Alternatively, you can try separating the `key=value` pairs like so:
|
||||
`--apiserver-extra-args "enable-admission-plugins=LimitRanger,enable-admission-plugins=NamespaceExists"`
|
||||
but this will result in the key `enable-admission-plugins` only having the value of `NamespaceExists`.
|
||||
|
||||
A known workaround is to use the kubeadm
|
||||
[configuration file](/docs/setup/independent/control-plane-flags/#apiserver-flags).
|
||||
A known workaround is to use the kubeadm [configuration file](/docs/setup/production-environment/tools/kubeadm/control-plane-flags/#apiserver-flags).
|
||||
|
||||
## kube-proxy scheduled before node is initialized by cloud-controller-manager
|
||||
|
||||
@@ -302,4 +302,20 @@ kubectl -n kube-system patch ds kube-proxy -p='{ "spec": { "template": { "spec":
|
||||
|
||||
The tracking issue for this problem is [here](https://github.com/kubernetes/kubeadm/issues/1027).
|
||||
|
||||
## The NodeRegistration.Taints field is omitted when marshalling kubeadm configuration
|
||||
|
||||
*Note: This [issue](https://github.com/kubernetes/kubeadm/issues/1358) only applies to tools that marshal kubeadm types (e.g. to a YAML configuration file). It will be fixed in kubeadm API v1beta2.*
|
||||
|
||||
By default, kubeadm applies the `role.kubernetes.io/master:NoSchedule` taint to control-plane nodes.
|
||||
If you prefer kubeadm to not taint the control-plane node, and set `InitConfiguration.NodeRegistration.Taints` to an empty slice,
|
||||
the field will be omitted when marshalling. When the field is omitted, kubeadm applies the default taint.
|
||||
|
||||
There are at least two workarounds:
|
||||
|
||||
1. Use the `role.kubernetes.io/master:PreferNoSchedule` taint instead of an empty slice. [Pods will get scheduled on masters](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/), unless other nodes have capacity.
|
||||
|
||||
2. Remove the taint after kubeadm init exits:
|
||||
```bash
|
||||
kubectl taint nodes NODE_NAME role.kubernetes.io/master:NoSchedule-
|
||||
```
|
||||
{{% /capture %}}
|
||||
+321
@@ -0,0 +1,321 @@
|
||||
---
|
||||
title: Troubleshooting kubeadm
|
||||
content_template: templates/concept
|
||||
weight: 20
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
As with any program, you might run into an error installing or running kubeadm.
|
||||
This page lists some common failure scenarios and have provided steps that can help you understand and fix the problem.
|
||||
|
||||
If your problem is not listed below, please follow the following steps:
|
||||
|
||||
- If you think your problem is a bug with kubeadm:
|
||||
- Go to [github.com/kubernetes/kubeadm](https://github.com/kubernetes/kubeadm/issues) and search for existing issues.
|
||||
- If no issue exists, please [open one](https://github.com/kubernetes/kubeadm/issues/new) and follow the issue template.
|
||||
|
||||
- If you are unsure about how kubeadm works, you can ask on [Slack](http://slack.k8s.io/) in #kubeadm, or open a question on [StackOverflow](https://stackoverflow.com/questions/tagged/kubernetes). Please include
|
||||
relevant tags like `#kubernetes` and `#kubeadm` so folks can help you.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture body %}}
|
||||
|
||||
## `ebtables` or some similar executable not found during installation
|
||||
|
||||
If you see the following warnings while running `kubeadm init`
|
||||
|
||||
```sh
|
||||
[preflight] WARNING: ebtables not found in system path
|
||||
[preflight] WARNING: ethtool not found in system path
|
||||
```
|
||||
|
||||
Then you may be missing `ebtables`, `ethtool` or a similar executable on your node. You can install them with the following commands:
|
||||
|
||||
- For Ubuntu/Debian users, run `apt install ebtables ethtool`.
|
||||
- For CentOS/Fedora users, run `yum install ebtables ethtool`.
|
||||
|
||||
## kubeadm blocks waiting for control plane during installation
|
||||
|
||||
If you notice that `kubeadm init` hangs after printing out the following line:
|
||||
|
||||
```sh
|
||||
[apiclient] Created API client, waiting for the control plane to become ready
|
||||
```
|
||||
|
||||
This may be caused by a number of problems. The most common are:
|
||||
|
||||
- network connection problems. Check that your machine has full network connectivity before continuing.
|
||||
- the default cgroup driver configuration for the kubelet differs from that used by Docker.
|
||||
Check the system log file (e.g. `/var/log/message`) or examine the output from `journalctl -u kubelet`. If you see something like the following:
|
||||
|
||||
```shell
|
||||
error: failed to run Kubelet: failed to create kubelet:
|
||||
misconfiguration: kubelet cgroup driver: "systemd" is different from docker cgroup driver: "cgroupfs"
|
||||
```
|
||||
|
||||
There are two common ways to fix the cgroup driver problem:
|
||||
|
||||
1. Install Docker again following instructions
|
||||
[here](/docs/setup/production-environment/container-runtimes/#docker).
|
||||
|
||||
1. Change the kubelet config to match the Docker cgroup driver manually, you can refer to
|
||||
[Configure cgroup driver used by kubelet on Master Node](/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#configure-cgroup-driver-used-by-kubelet-on-master-node)
|
||||
|
||||
- control plane Docker containers are crashlooping or hanging. You can check this by running `docker ps` and investigating each container by running `docker logs`.
|
||||
|
||||
## kubeadm blocks when removing managed containers
|
||||
|
||||
The following could happen if Docker halts and does not remove any Kubernetes-managed containers:
|
||||
|
||||
```bash
|
||||
sudo kubeadm reset
|
||||
[preflight] Running pre-flight checks
|
||||
[reset] Stopping the kubelet service
|
||||
[reset] Unmounting mounted directories in "/var/lib/kubelet"
|
||||
[reset] Removing kubernetes-managed containers
|
||||
(block)
|
||||
```
|
||||
|
||||
A possible solution is to restart the Docker service and then re-run `kubeadm reset`:
|
||||
|
||||
```bash
|
||||
sudo systemctl restart docker.service
|
||||
sudo kubeadm reset
|
||||
```
|
||||
|
||||
Inspecting the logs for docker may also be useful:
|
||||
|
||||
```sh
|
||||
journalctl -ul docker
|
||||
```
|
||||
|
||||
## Pods in `RunContainerError`, `CrashLoopBackOff` or `Error` state
|
||||
|
||||
Right after `kubeadm init` there should not be any pods in these states.
|
||||
|
||||
- If there are pods in one of these states _right after_ `kubeadm init`, please open an
|
||||
issue in the kubeadm repo. `coredns` (or `kube-dns`) should be in the `Pending` state
|
||||
until you have deployed the network solution.
|
||||
- If you see Pods in the `RunContainerError`, `CrashLoopBackOff` or `Error` state
|
||||
after deploying the network solution and nothing happens to `coredns` (or `kube-dns`),
|
||||
it's very likely that the Pod Network solution that you installed is somehow broken.
|
||||
You might have to grant it more RBAC privileges or use a newer version. Please file
|
||||
an issue in the Pod Network providers' issue tracker and get the issue triaged there.
|
||||
- If you install a version of Docker older than 1.12.1, remove the `MountFlags=slave` option
|
||||
when booting `dockerd` with `systemd` and restart `docker`. You can see the MountFlags in `/usr/lib/systemd/system/docker.service`.
|
||||
MountFlags can interfere with volumes mounted by Kubernetes, and put the Pods in `CrashLoopBackOff` state.
|
||||
The error happens when Kubernetes does not find `var/run/secrets/kubernetes.io/serviceaccount` files.
|
||||
|
||||
## `coredns` (or `kube-dns`) is stuck in the `Pending` state
|
||||
|
||||
This is **expected** and part of the design. kubeadm is network provider-agnostic, so the admin
|
||||
should [install the pod network solution](/docs/concepts/cluster-administration/addons/)
|
||||
of choice. You have to install a Pod Network
|
||||
before CoreDNS may be deployed fully. Hence the `Pending` state before the network is set up.
|
||||
|
||||
## `HostPort` services do not work
|
||||
|
||||
The `HostPort` and `HostIP` functionality is available depending on your Pod Network
|
||||
provider. Please contact the author of the Pod Network solution to find out whether
|
||||
`HostPort` and `HostIP` functionality are available.
|
||||
|
||||
Calico, Canal, and Flannel CNI providers are verified to support HostPort.
|
||||
|
||||
For more information, see the [CNI portmap documentation](https://github.com/containernetworking/plugins/blob/master/plugins/meta/portmap/README.md).
|
||||
|
||||
If your network provider does not support the portmap CNI plugin, you may need to use the [NodePort feature of
|
||||
services](/docs/concepts/services-networking/service/#nodeport) or use `HostNetwork=true`.
|
||||
|
||||
## Pods are not accessible via their Service IP
|
||||
|
||||
- Many network add-ons do not yet enable [hairpin mode](/docs/tasks/debug-application-cluster/debug-service/#a-pod-cannot-reach-itself-via-service-ip)
|
||||
which allows pods to access themselves via their Service IP. This is an issue related to
|
||||
[CNI](https://github.com/containernetworking/cni/issues/476). Please contact the network
|
||||
add-on provider to get the latest status of their support for hairpin mode.
|
||||
|
||||
- If you are using VirtualBox (directly or via Vagrant), you will need to
|
||||
ensure that `hostname -i` returns a routable IP address. By default the first
|
||||
interface is connected to a non-routable host-only network. A work around
|
||||
is to modify `/etc/hosts`, see this [Vagrantfile](https://github.com/errordeveloper/k8s-playground/blob/22dd39dfc06111235620e6c4404a96ae146f26fd/Vagrantfile#L11)
|
||||
for an example.
|
||||
|
||||
## TLS certificate errors
|
||||
|
||||
The following error indicates a possible certificate mismatch.
|
||||
|
||||
```none
|
||||
# kubectl get pods
|
||||
Unable to connect to the server: x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubernetes")
|
||||
```
|
||||
|
||||
- Verify that the `$HOME/.kube/config` file contains a valid certificate, and
|
||||
regenerate a certificate if necessary. The certificates in a kubeconfig file
|
||||
are base64 encoded. The `base64 -d` command can be used to decode the certificate
|
||||
and `openssl x509 -text -noout` can be used for viewing the certificate information.
|
||||
- Unset the `KUBECONFIG` environment variable using:
|
||||
|
||||
```sh
|
||||
unset KUBECONFIG
|
||||
```
|
||||
|
||||
Or set it to the default `KUBECONFIG` location:
|
||||
|
||||
```sh
|
||||
export KUBECONFIG=/etc/kubernetes/admin.conf
|
||||
```
|
||||
|
||||
- Another workaround is to overwrite the existing `kubeconfig` for the "admin" user:
|
||||
|
||||
```sh
|
||||
mv $HOME/.kube $HOME/.kube.bak
|
||||
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
|
||||
sudo chown $(id -u):$(id -g) $HOME/.kube/config
|
||||
```
|
||||
|
||||
## Default NIC When using flannel as the pod network in Vagrant
|
||||
|
||||
The following error might indicate that something was wrong in the pod network:
|
||||
|
||||
```sh
|
||||
Error from server (NotFound): the server could not find the requested resource
|
||||
```
|
||||
|
||||
- If you're using flannel as the pod network inside Vagrant, then you will have to specify the default interface name for flannel.
|
||||
|
||||
Vagrant typically assigns two interfaces to all VMs. The first, for which all hosts are assigned the IP address `10.0.2.15`, is for external traffic that gets NATed.
|
||||
|
||||
This may lead to problems with flannel, which defaults to the first interface on a host. This leads to all hosts thinking they have the same public IP address. To prevent this, pass the `--iface eth1` flag to flannel so that the second interface is chosen.
|
||||
|
||||
## Non-public IP used for containers
|
||||
|
||||
In some situations `kubectl logs` and `kubectl run` commands may return with the following errors in an otherwise functional cluster:
|
||||
|
||||
```sh
|
||||
Error from server: Get https://10.19.0.41:10250/containerLogs/default/mysql-ddc65b868-glc5m/mysql: dial tcp 10.19.0.41:10250: getsockopt: no route to host
|
||||
```
|
||||
|
||||
- This may be due to Kubernetes using an IP that can not communicate with other IPs on the seemingly same subnet, possibly by policy of the machine provider.
|
||||
- Digital Ocean assigns a public IP to `eth0` as well as a private one to be used internally as anchor for their floating IP feature, yet `kubelet` will pick the latter as the node's `InternalIP` instead of the public one.
|
||||
|
||||
Use `ip addr show` to check for this scenario instead of `ifconfig` because `ifconfig` will not display the offending alias IP address. Alternatively an API endpoint specific to Digital Ocean allows to query for the anchor IP from the droplet:
|
||||
|
||||
```sh
|
||||
curl http://169.254.169.254/metadata/v1/interfaces/public/0/anchor_ipv4/address
|
||||
```
|
||||
|
||||
The workaround is to tell `kubelet` which IP to use using `--node-ip`. When using Digital Ocean, it can be the public one (assigned to `eth0`) or the private one (assigned to `eth1`) should you want to use the optional private network. The [`KubeletExtraArgs` section of the kubeadm `NodeRegistrationOptions` structure](https://github.com/kubernetes/kubernetes/blob/release-1.13/cmd/kubeadm/app/apis/kubeadm/v1beta1/types.go) can be used for this.
|
||||
|
||||
Then restart `kubelet`:
|
||||
|
||||
```sh
|
||||
systemctl daemon-reload
|
||||
systemctl restart kubelet
|
||||
```
|
||||
|
||||
## `coredns` pods have `CrashLoopBackOff` or `Error` state
|
||||
|
||||
If you have nodes that are running SELinux with an older version of Docker you might experience a scenario
|
||||
where the `coredns` pods are not starting. To solve that you can try one of the following options:
|
||||
|
||||
- Upgrade to a [newer version of Docker](/docs/setup/production-environment/container-runtimes/#docker).
|
||||
|
||||
- [Disable SELinux](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/security-enhanced_linux/sect-security-enhanced_linux-enabling_and_disabling_selinux-disabling_selinux).
|
||||
- Modify the `coredns` deployment to set `allowPrivilegeEscalation` to `true`:
|
||||
|
||||
```bash
|
||||
kubectl -n kube-system get deployment coredns -o yaml | \
|
||||
sed 's/allowPrivilegeEscalation: false/allowPrivilegeEscalation: true/g' | \
|
||||
kubectl apply -f -
|
||||
```
|
||||
|
||||
Another cause for CoreDNS to have `CrashLoopBackOff` is when a CoreDNS Pod deployed in Kubernetes detects a loop. [A number of workarounds](https://github.com/coredns/coredns/tree/master/plugin/loop#troubleshooting-loops-in-kubernetes-clusters)
|
||||
are available to avoid Kubernetes trying to restart the CoreDNS Pod every time CoreDNS detects the loop and exits.
|
||||
|
||||
{{< warning >}}
|
||||
Disabling SELinux or setting `allowPrivilegeEscalation` to `true` can compromise
|
||||
the security of your cluster.
|
||||
{{< /warning >}}
|
||||
|
||||
## etcd pods restart continually
|
||||
|
||||
If you encounter the following error:
|
||||
|
||||
```
|
||||
rpc error: code = 2 desc = oci runtime error: exec failed: container_linux.go:247: starting container process caused "process_linux.go:110: decoding init error from pipe caused \"read parent: connection reset by peer\""
|
||||
```
|
||||
|
||||
this issue appears if you run CentOS 7 with Docker 1.13.1.84.
|
||||
This version of Docker can prevent the kubelet from executing into the etcd container.
|
||||
|
||||
To work around the issue, choose one of these options:
|
||||
|
||||
- Roll back to an earlier version of Docker, such as 1.13.1-75
|
||||
```
|
||||
yum downgrade docker-1.13.1-75.git8633870.el7.centos.x86_64 docker-client-1.13.1-75.git8633870.el7.centos.x86_64 docker-common-1.13.1-75.git8633870.el7.centos.x86_64
|
||||
```
|
||||
|
||||
- Install one of the more recent recommended versions, such as 18.06:
|
||||
```bash
|
||||
sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
|
||||
yum install docker-ce-18.06.1.ce-3.el7.x86_64
|
||||
```
|
||||
|
||||
## Not possible to pass a comma separated list of values to arguments inside a `--component-extra-args` flag
|
||||
|
||||
`kubeadm init` flags such as `--component-extra-args` allow you to pass custom arguments to a control-plane
|
||||
component like the kube-apiserver. However, this mechanism is limited due to the underlying type used for parsing
|
||||
the values (`mapStringString`).
|
||||
|
||||
If you decide to pass an argument that supports multiple, comma-separated values such as
|
||||
`--apiserver-extra-args "enable-admission-plugins=LimitRanger,NamespaceExists"` this flag will fail with
|
||||
`flag: malformed pair, expect string=string`. This happens because the list of arguments for
|
||||
`--apiserver-extra-args` expects `key=value` pairs and in this case `NamespacesExists` is considered
|
||||
as a key that is missing a value.
|
||||
|
||||
Alternatively, you can try separating the `key=value` pairs like so:
|
||||
`--apiserver-extra-args "enable-admission-plugins=LimitRanger,enable-admission-plugins=NamespaceExists"`
|
||||
but this will result in the key `enable-admission-plugins` only having the value of `NamespaceExists`.
|
||||
|
||||
A known workaround is to use the kubeadm [configuration file](/docs/setup/production-environment/tools/kubeadm/control-plane-flags/#apiserver-flags).
|
||||
|
||||
## kube-proxy scheduled before node is initialized by cloud-controller-manager
|
||||
|
||||
In cloud provider scenarios, kube-proxy can end up being scheduled on new worker nodes before
|
||||
the cloud-controller-manager has initialized the node addresses. This causes kube-proxy to fail
|
||||
to pick up the node's IP address properly and has knock-on effects to the proxy function managing
|
||||
load balancers.
|
||||
|
||||
The following error can be seen in kube-proxy Pods:
|
||||
```
|
||||
server.go:610] Failed to retrieve node IP: host IP unknown; known addresses: []
|
||||
proxier.go:340] invalid nodeIP, initializing kube-proxy with 127.0.0.1 as nodeIP
|
||||
```
|
||||
|
||||
A known solution is to patch the kube-proxy DaemonSet to allow scheduling it on control-plane
|
||||
nodes regardless of their conditions, keeping it off of other nodes until their initial guarding
|
||||
conditions abate:
|
||||
```
|
||||
kubectl -n kube-system patch ds kube-proxy -p='{ "spec": { "template": { "spec": { "tolerations": [ { "key": "CriticalAddonsOnly", "operator": "Exists" }, { "effect": "NoSchedule", "key": "node-role.kubernetes.io/master" } ] } } } }'
|
||||
```
|
||||
|
||||
The tracking issue for this problem is [here](https://github.com/kubernetes/kubeadm/issues/1027).
|
||||
|
||||
## The NodeRegistration.Taints field is omitted when marshalling kubeadm configuration
|
||||
|
||||
*Note: This [issue](https://github.com/kubernetes/kubeadm/issues/1358) only applies to tools that marshal kubeadm types (e.g. to a YAML configuration file). It will be fixed in kubeadm API v1beta2.*
|
||||
|
||||
By default, kubeadm applies the `role.kubernetes.io/master:NoSchedule` taint to control-plane nodes.
|
||||
If you prefer kubeadm to not taint the control-plane node, and set `InitConfiguration.NodeRegistration.Taints` to an empty slice,
|
||||
the field will be omitted when marshalling. When the field is omitted, kubeadm applies the default taint.
|
||||
|
||||
There are at least two workarounds:
|
||||
|
||||
1. Use the `role.kubernetes.io/master:PreferNoSchedule` taint instead of an empty slice. [Pods will get scheduled on masters](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/), unless other nodes have capacity.
|
||||
|
||||
2. Remove the taint after kubeadm init exits:
|
||||
```bash
|
||||
kubectl taint nodes NODE_NAME role.kubernetes.io/master:NoSchedule-
|
||||
```
|
||||
{{% /capture %}}
|
||||
+3
-2
@@ -1,6 +1,7 @@
|
||||
---
|
||||
title: Installing Kubernetes On-premises/Cloud Providers with Kubespray
|
||||
title: Installing Kubernetes with Kubespray
|
||||
content_template: templates/concept
|
||||
weight: 30
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
@@ -75,7 +76,7 @@ Next, deploy your cluster:
|
||||
Cluster deployment using [ansible-playbook](https://github.com/kubernetes-incubator/kubespray/blob/master/docs/getting-started.md#starting-custom-deployment).
|
||||
|
||||
```shell
|
||||
ansible-playbook -i your/inventory/hosts.ini cluster.yml -b -v \
|
||||
ansible-playbook -i your/inventory/inventory.ini cluster.yml -b -v \
|
||||
--private-key=~/.ssh/private_key
|
||||
```
|
||||
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
---
|
||||
title: Turnkey Cloud Solutions
|
||||
weight: 40
|
||||
weight: 30
|
||||
---
|
||||
+1
-1
@@ -9,7 +9,7 @@ title: Running Kubernetes on Alibaba Cloud
|
||||
|
||||
The [Alibaba Cloud Container Service](https://www.alibabacloud.com/product/container-service) lets you run and manage Docker applications on a cluster of Alibaba Cloud ECS instances. It supports the popular open source container orchestrators: Docker Swarm and Kubernetes.
|
||||
|
||||
To simplify cluster deployment and management, use [Kubernetes Support for Alibaba Cloud Container Service](https://www.alibabacloud.com/product/kubernetes). You can get started quickly by following the [Kubernetes walk-through](https://www.alibabacloud.com/help/doc-detail/86737.htm), and there are some [tutorials for Kubernetes Support on Alibaba Cloud](https://yq.aliyun.com/teams/11/type_blog-cid_200-page_1) in Chinese.
|
||||
To simplify cluster deployment and management, use [Kubernetes Support for Alibaba Cloud Container Service](https://www.alibabacloud.com/product/kubernetes). You can get started quickly by following the [Kubernetes walk-through](https://www.alibabacloud.com/help/doc-detail/86737.htm), and there are some [tutorials for Kubernetes Support on Alibaba Cloud](https://yq.aliyun.com/teams/11/type_blog-cid_200-page_1) in Chinese.
|
||||
|
||||
To use custom binaries or open source Kubernetes, follow the instructions below.
|
||||
|
||||
+3
-2
@@ -26,6 +26,8 @@ To create a Kubernetes cluster on AWS, you will need an Access Key ID and a Secr
|
||||
|
||||
* CoreOS originated and the Kubernetes Incubator maintains [a CLI tool, kube-aws](https://github.com/kubernetes-incubator/kube-aws), that creates and manages Kubernetes clusters with [Container Linux](https://coreos.com/why/) nodes, using AWS tools: EC2, CloudFormation and Autoscaling.
|
||||
|
||||
* [KubeOne](https://github.com/kubermatic/kubeone) is an open source cluster lifecycle management tool that creates, upgrades and manages Kubernetes Highly-Available clusters.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture steps %}}
|
||||
@@ -81,8 +83,7 @@ IaaS Provider | Config. Mgmt | OS | Networking | Docs
|
||||
AWS | kops | Debian | k8s (VPC) | [docs](https://github.com/kubernetes/kops) | | Community ([@justinsb](https://github.com/justinsb))
|
||||
AWS | CoreOS | CoreOS | flannel | [docs](/docs/getting-started-guides/aws) | | Community
|
||||
AWS | Juju | Ubuntu | flannel, calico, canal | [docs](/docs/getting-started-guides/ubuntu) | 100% | Commercial, Community
|
||||
|
||||
For support level information on all solutions, see the [Table of solutions](/docs/getting-started-guides/#table-of-solutions) chart.
|
||||
AWS | KubeOne | Ubuntu, CoreOS, CentOS | canal, weavenet | [docs](https://github.com/kubermatic/kubeone) | 100% | Commercial, Community
|
||||
|
||||
## Further reading
|
||||
|
||||
+9
-11
@@ -2,7 +2,6 @@
|
||||
title: Running Kubernetes on CenturyLink Cloud
|
||||
---
|
||||
|
||||
{: toc}
|
||||
|
||||
These scripts handle the creation, deletion and expansion of Kubernetes clusters on CenturyLink Cloud.
|
||||
|
||||
@@ -238,12 +237,11 @@ utility ```kubectl```. If you do not already have a copy of this binary on your
|
||||
administrative machine, you may run the script ```install_kubectl.sh``` which will
|
||||
download it and install it in ```/usr/bin/local```.
|
||||
|
||||
The script requires that the environment variable ```CLC_CLUSTER_NAME``` be defined
|
||||
|
||||
```install_kubectl.sh``` also writes a configuration file which will embed the necessary
|
||||
The script requires that the environment variable ```CLC_CLUSTER_NAME``` be defined. ```install_kubectl.sh``` also writes a configuration file which will embed the necessary
|
||||
authentication certificates for the particular cluster. The configuration file is
|
||||
written to the ```${CLC_CLUSTER_HOME}/kube``` directory
|
||||
|
||||
|
||||
```shell
|
||||
export KUBECONFIG=${CLC_CLUSTER_HOME}/kube/config
|
||||
kubectl version
|
||||
@@ -273,18 +271,18 @@ create a cluster, the script should output URLs for these interfaces like this:
|
||||
|
||||
kubernetes-dashboard is running at ```https://${MASTER_IP}:6443/api/v1/namespaces/kube-system/services/kubernetes-dashboard/proxy```.
|
||||
|
||||
Note on Authentication to the UIs: The cluster is set up to use basic
|
||||
authentication for the user _admin_. Hitting the url at
|
||||
```https://${MASTER_IP}:6443``` will require accepting the self-signed certificate
|
||||
from the apiserver, and then presenting the admin password written to file at:
|
||||
Note on Authentication to the UIs:
|
||||
|
||||
```> _${CLC_CLUSTER_HOME}/kube/admin_password.txt_```
|
||||
The cluster is set up to use basic authentication for the user _admin_.
|
||||
Hitting the url at ```https://${MASTER_IP}:6443``` will
|
||||
require accepting the self-signed certificate
|
||||
from the apiserver, and then presenting the admin
|
||||
password written to file at: ```> _${CLC_CLUSTER_HOME}/kube/admin_password.txt_```
|
||||
|
||||
|
||||
### Configuration files
|
||||
|
||||
Various configuration files are written into the home directory *CLC_CLUSTER_HOME* under
|
||||
```.clc_kube/${CLC_CLUSTER_NAME}``` in several subdirectories. You can use these files
|
||||
Various configuration files are written into the home directory *CLC_CLUSTER_HOME* under ```.clc_kube/${CLC_CLUSTER_NAME}``` in several subdirectories. You can use these files
|
||||
to access the cluster from machines other than where you created the cluster from.
|
||||
|
||||
* ```config/```: Ansible variable files containing parameters describing the master and minion hosts
|
||||
+2
-3
@@ -70,7 +70,7 @@ cluster/kube-up.sh
|
||||
|
||||
If you want more than one cluster running in your project, want to use a different name, or want a different number of worker nodes, see the `<kubernetes>/cluster/gce/config-default.sh` file for more fine-grained configuration before you start up your cluster.
|
||||
|
||||
If you run into trouble, please see the section on [troubleshooting](/docs/setup/turnkey/gce/#troubleshooting), post to the
|
||||
If you run into trouble, please see the section on [troubleshooting](/docs/setup/production-environment/turnkey/gce/#troubleshooting), post to the
|
||||
[Kubernetes Forum](https://discuss.kubernetes.io), or come ask questions on [Slack](/docs/troubleshooting/#slack).
|
||||
|
||||
The next few steps will show you:
|
||||
@@ -217,9 +217,8 @@ field values:
|
||||
|
||||
IaaS Provider | Config. Mgmt | OS | Networking | Docs | Conforms | Support Level
|
||||
-------------------- | ------------ | ------ | ---------- | --------------------------------------------- | ---------| ----------------------------
|
||||
GCE | Saltstack | Debian | GCE | [docs](/docs/setup/turnkey/gce/) | | Project
|
||||
GCE | Saltstack | Debian | GCE | [docs](/docs/setup/production-environment/turnkey/gce/) | | Project
|
||||
|
||||
For support level information on all solutions, see the [Table of solutions](/docs/getting-started-guides/#table-of-solutions) chart.
|
||||
|
||||
## Further reading
|
||||
|
||||
+3
-3
@@ -37,9 +37,9 @@ IBM Cloud Private can also run on the AWS cloud platform by using Terraform. To
|
||||
|
||||
## IBM Cloud Private on Azure
|
||||
|
||||
You can enable Microsoft Azure as a cloud provider for IBM Cloud Private deployment and take advantage of all the IBM Cloud Private features on the Azure public cloud. For more information, see [IBM Cloud Private on Azure](https://www.ibm.com/support/knowledgecenter/SSBS6K_3.1.2/supported_environments/azure_overview.html).
|
||||
You can enable Microsoft Azure as a cloud provider for IBM Cloud Private deployment and take advantage of all the IBM Cloud Private features on the Azure public cloud. For more information, see [IBM Cloud Private on Azure](https://www.ibm.com/support/knowledgecenter/SSBS6K_3.2.0/supported_environments/azure_overview.html).
|
||||
|
||||
## IBM Cloud Private on Red Hat OpenShift
|
||||
## IBM Cloud Private with Red Hat OpenShift
|
||||
|
||||
You can deploy IBM certified software containers that are running on IBM Cloud Private onto Red Hat OpenShift.
|
||||
|
||||
@@ -51,7 +51,7 @@ Integration capabilities:
|
||||
* Integrated core platform services, such as monitoring, metering, and logging
|
||||
* IBM Cloud Private uses the OpenShift image registry
|
||||
|
||||
For more information see, [IBM Cloud Private on OpenShift](https://www.ibm.com/support/knowledgecenter/SSBS6K_3.1.2/supported_environments/openshift/overview.html).
|
||||
For more information see, [IBM Cloud Private on OpenShift](https://www.ibm.com/support/knowledgecenter/SSBS6K_3.2.0/supported_environments/openshift/overview.html).
|
||||
|
||||
## IBM Cloud Private on VirtualBox
|
||||
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
---
|
||||
title: "Windows in Kubernetes"
|
||||
weight: 65
|
||||
weight: 50
|
||||
---
|
||||
|
Before Width: | Height: | Size: 109 KiB After Width: | Height: | Size: 109 KiB |
|
Before Width: | Height: | Size: 25 KiB After Width: | Height: | Size: 25 KiB |
+24
-24
@@ -9,7 +9,7 @@ weight: 65
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
Windows applications constitute a large portion of the services and applications that run in many organizations. [Windows containers](https://aka.ms/windowscontainers) provide a modern way to encapsulate processes and package dependencies, making it easier to use DevOps practices and follow cloud native patterns for Windows applications. Kubernetes has become the defacto standard container orchestrator, and the release of Kubernetes 1.14 includes production support for scheduling Windows containers on Windows nodes in a Kubernetes cluster, enabling a vast ecosystem of Windows applications to leverage the power of Kubernetes. Organizations with investments in Windows-based applications and Linux-based applications don't have to look for separate orchestrators to manage their workloads, leading to increased operational efficiencies across their deployments, regardless of operating system.
|
||||
Windows applications constitute a large portion of the services and applications that run in many organizations. [Windows containers](https://aka.ms/windowscontainers) provide a modern way to encapsulate processes and package dependencies, making it easier to use DevOps practices and follow cloud native patterns for Windows applications. Kubernetes has become the defacto standard container orchestrator, and the release of Kubernetes 1.14 includes production support for scheduling Windows containers on Windows nodes in a Kubernetes cluster, enabling a vast ecosystem of Windows applications to leverage the power of Kubernetes. Organizations with investments in Windows-based applications and Linux-based applications don't have to look for separate orchestrators to manage their workloads, leading to increased operational efficiencies across their deployments, regardless of operating system.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
@@ -26,7 +26,7 @@ The Kubernetes control plane, including the [master components](/docs/concepts/o
|
||||
{{< /note >}}
|
||||
|
||||
{{< note >}}
|
||||
In this document, when we talk about Windows containers we mean Windows containers with process isolation. Windows containers with [Hyper-V isolation](https://docs.microsoft.com/en-us/virtualization/windowscontainers/manage-containers/hyperv-container) is planned for a future release.
|
||||
In this document, when we talk about Windows containers we mean Windows containers with process isolation. Windows containers with [Hyper-V isolation](https://docs.microsoft.com/en-us/virtualization/windowscontainers/manage-containers/hyperv-container) is planned for a future release.
|
||||
{{< /note >}}
|
||||
|
||||
## Supported Functionality and Limitations
|
||||
@@ -45,7 +45,7 @@ Let's start with the operating system version. Refer to the following table for
|
||||
| *Kubernetes v1.14* | Not Supported | Not Supported| Supported for Windows Server containers Builds 17763.* with Docker EE-basic 18.09 |
|
||||
|
||||
{{< note >}}
|
||||
We don't expect all Windows customers to update the operating system for their apps frequently. Upgrading your applications is what dictates and necessitates upgrading or introducing new nodes to the cluster. For the customers that chose to upgrade their operating system for containers running on Kubernetes, we will offer guidance and step-by-step instructions when we add support for a new operating system version. This guidance will include recommended upgrade procedures for upgrading user applications together with cluster nodes. Windows nodes adhere to Kubernetes [version-skew policy](/docs/setup/version-skew-policy/) (node to control plane versioning) the same way as Linux nodes do today.
|
||||
We don't expect all Windows customers to update the operating system for their apps frequently. Upgrading your applications is what dictates and necessitates upgrading or introducing new nodes to the cluster. For the customers that chose to upgrade their operating system for containers running on Kubernetes, we will offer guidance and step-by-step instructions when we add support for a new operating system version. This guidance will include recommended upgrade procedures for upgrading user applications together with cluster nodes. Windows nodes adhere to Kubernetes [version-skew policy](/docs/setup/release/version-skew-policy/) (node to control plane versioning) the same way as Linux nodes do today.
|
||||
{{< /note >}}
|
||||
{{< note >}}
|
||||
The Windows Server Host Operating System is subject to the [Windows Server ](https://www.microsoft.com/en-us/cloud-platform/windows-server-pricing) licensing. The Windows Container images are subject to the [Supplemental License Terms for Windows containers](https://docs.microsoft.com/en-us/virtualization/windowscontainers/images-eula).
|
||||
@@ -61,7 +61,7 @@ Key Kubernetes elements work the same way in Windows as they do in Linux. In thi
|
||||
A Pod is the basic building block of Kubernetes–the smallest and simplest unit in the Kubernetes object model that you create or deploy. The following Pod capabilities, properties and events are supported with Windows containers:
|
||||
|
||||
* Single or multiple containers per Pod with process isolation and volume sharing
|
||||
* Pod status fields
|
||||
* Pod status fields
|
||||
* Readiness and Liveness probes
|
||||
* postStart & preStop container lifecycle events
|
||||
* ConfigMap, Secrets: as environment variables or volumes
|
||||
@@ -122,7 +122,7 @@ Networking for Windows containers is exposed through [CNI plugins](/docs/concept
|
||||
|
||||
The following service spec types are supported:
|
||||
|
||||
* NodePort
|
||||
* NodePort
|
||||
* ClusterIP
|
||||
* LoadBalancer
|
||||
* ExternalName
|
||||
@@ -142,7 +142,7 @@ As outlined above, the [Flannel](https://github.com/coreos/flannel) CNI [meta pl
|
||||
For the node, pod, and service objects, the following network flows are supported for TCP/UDP traffic:
|
||||
|
||||
* Pod -> Pod (IP)
|
||||
* Pod -> Pod (Name)
|
||||
* Pod -> Pod (Name)
|
||||
* Pod -> Service (Cluster IP)
|
||||
* Pod -> Service (PQDN, but only if there are no ".")
|
||||
* Pod -> Service (FQDN)
|
||||
@@ -198,7 +198,7 @@ The behavior of the flags behave differently as described below:
|
||||
* Eviction by using `--eviction-hard` and `--eviction-soft` are not implemented
|
||||
* MemoryPressure Condition is not implemented
|
||||
* There are no OOM eviction actions taken by the kubelet
|
||||
* Kubelet running on the windows node does not have memory restrictions. `--kubelet-reserve` and `--system-reserve` do not set limits on kubelet or processes running the host. This means kubelet or a process on the host could cause memory resource starvation outside the node-allocatable and scheduler
|
||||
* Kubelet running on the windows node does not have memory restrictions. `--kubelet-reserve` and `--system-reserve` do not set limits on kubelet or processes running on the host. This means kubelet or a process on the host could cause memory resource starvation outside the node-allocatable and scheduler
|
||||
|
||||
#### Storage
|
||||
|
||||
@@ -233,7 +233,7 @@ The following networking functionality is not supported on Windows nodes
|
||||
|
||||
* Host networking mode is not available for Windows pods
|
||||
* Local NodePort access from the node itself fails (works for other nodes or external clients)
|
||||
* Accessing service VIPs from nodes will be available with a future release of Windows Server
|
||||
* Accessing service VIPs from nodes will be available with a future release of Windows Server
|
||||
* Overlay networking support in kube-proxy is an alpha release. In addition, it requires [KB4482887](https://support.microsoft.com/en-us/help/4482887/windows-10-update-kb4482887) to be installed on Windows Server 2019
|
||||
* `kubectl port-forward`
|
||||
* Local Traffic Policy and DSR mode
|
||||
@@ -266,13 +266,13 @@ Secrets are written in clear text on the node's volume (as compared to tmpfs/in-
|
||||
|
||||
[RunAsUser ](/docs/concepts/policy/pod-security-policy/#users-and-groups)is not currently supported on Windows. The workaround is to create local accounts before packaging the container. The RunAsUsername capability may be added in a future release.
|
||||
|
||||
Linux specific pod security context privileges such as SELinux, AppArmor, Seccomp, Capabilities (POSIX Capabilities), and others are not supported.
|
||||
Linux specific pod security context privileges such as SELinux, AppArmor, Seccomp, Capabilities (POSIX Capabilities), and others are not supported.
|
||||
|
||||
In addition, as mentioned already, privileged containers are not supported on Windows.
|
||||
|
||||
#### API
|
||||
|
||||
There are no differences in how most of the Kubernetes APIs work for Windows. The subtleties around what's different come down to differences in the OS and container runtime. In certain situations, some properties on workload APIs such as Pod or Container were designed with an assumption that they are implemented on Linux, failing to run on Windows.
|
||||
There are no differences in how most of the Kubernetes APIs work for Windows. The subtleties around what's different come down to differences in the OS and container runtime. In certain situations, some properties on workload APIs such as Pod or Container were designed with an assumption that they are implemented on Linux, failing to run on Windows.
|
||||
|
||||
At a high level, these OS concepts are different:
|
||||
|
||||
@@ -288,7 +288,7 @@ Exit Codes follow the same convention where 0 is success, nonzero is failure. Th
|
||||
|
||||
##### V1.Container
|
||||
|
||||
* V1.Container.ResourceRequirements.limits.cpu and V1.Container.ResourceRequirements.limits.memory - Windows doesn't use hard limits for CPU allocations. Instead, a share system is used. The existing fields based on millicores are scaled into relative shares that are followed by the Windows scheduler. [see: kuberuntime/helpers_windows.go](https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/kuberuntime/helpers_windows.go), [see: resource controls in Microsoft docs](https://docs.microsoft.com/en-us/virtualization/windowscontainers/manage-containers/resource-controls)
|
||||
* V1.Container.ResourceRequirements.limits.cpu and V1.Container.ResourceRequirements.limits.memory - Windows doesn't use hard limits for CPU allocations. Instead, a share system is used. The existing fields based on millicores are scaled into relative shares that are followed by the Windows scheduler. [see: kuberuntime/helpers_windows.go](https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/kuberuntime/helpers_windows.go), [see: resource controls in Microsoft docs](https://docs.microsoft.com/en-us/virtualization/windowscontainers/manage-containers/resource-controls)
|
||||
* Huge pages are not implemented in the Windows container runtime, and are not available. They require [asserting a user privilege](https://docs.microsoft.com/en-us/windows/desktop/Memory/large-page-support) that's not configurable for containers.
|
||||
* V1.Container.ResourceRequirements.requests.cpu and V1.Container.ResourceRequirements.requests.memory - Requests are subtracted from node available resources, so they can be used to avoid overprovisioning a node. However, they cannot be used to guarantee resources in an overprovisioned node. They should be applied to all containers as a best practice if the operator wants to avoid overprovisioning entirely.
|
||||
* V1.Container.SecurityContext.allowPrivilegeEscalation - not possible on Windows, none of the capabilities are hooked up
|
||||
@@ -298,7 +298,7 @@ Exit Codes follow the same convention where 0 is success, nonzero is failure. Th
|
||||
* V1.Container.SecurityContext.readOnlyRootFilesystem - not possible on Windows, write access is required for registry & system processes to run inside the container
|
||||
* V1.Container.SecurityContext.runAsGroup - not possible on Windows, no GID support
|
||||
* V1.Container.SecurityContext.runAsNonRoot - Windows does not have a root user. The closest equivalent is ContainerAdministrator which is an identity that doesn't exist on the node.
|
||||
* V1.Container.SecurityContext.runAsUser - not possible on Windows, no UID support as int.
|
||||
* V1.Container.SecurityContext.runAsUser - not possible on Windows, no UID support as int.
|
||||
* V1.Container.SecurityContext.seLinuxOptions - not possible on Windows, no SELinux
|
||||
* V1.Container.terminationMessagePath - this has some limitations in that Windows doesn't support mapping single files. The default value is /dev/termination-log, which does work because it does not exist on Windows by default.
|
||||
|
||||
@@ -313,7 +313,7 @@ Exit Codes follow the same convention where 0 is success, nonzero is failure. Th
|
||||
* V1.Pod.volumeDevices - this is a beta feature, and is not implemented on Windows. Windows cannot attach raw block devices to pods.
|
||||
* V1.Pod.volumes - EmptyDir, Secret, ConfigMap, HostPath - all work and have tests in TestGrid
|
||||
* V1.emptyDirVolumeSource - the Node default medium is disk on Windows. Memory is not supported, as Windows does not have a built-in RAM disk.
|
||||
* V1.VolumeMount.mountPropagation - only MountPropagationHostToContainer is available. Windows cannot create mounts within a pod or project them back to the node.
|
||||
* V1.VolumeMount.mountPropagation - mount propagation is not supported on Windows.
|
||||
|
||||
##### V1.PodSecurityContext
|
||||
|
||||
@@ -365,26 +365,26 @@ Your main source of help for troubleshooting your Kubernetes cluster should star
|
||||
1. Using nssm.exe
|
||||
|
||||
You can also always use alternative service managers like [nssm.exe](https://nssm.cc/) to run these processes (flanneld, kubelet & kube-proxy) in the background for you. You can use this [sample script](https://github.com/Microsoft/SDN/tree/master/Kubernetes/flannel/register-svc.ps1), leveraging nssm.exe to register kubelet, kube-proxy, and flanneld.exe to run as Windows services in the background.
|
||||
|
||||
|
||||
```powershell
|
||||
register-svc.ps1 -NetworkMode <Network mode> -ManagementIP <Windows Node IP> -ClusterCIDR <Cluster subnet> -KubeDnsServiceIP <Kube-dns Service IP> -LogDir <Directory to place logs>
|
||||
|
||||
|
||||
# NetworkMode = The network mode l2bridge (flannel host-gw, also the default value) or overlay (flannel vxlan) chosen as a network solution
|
||||
# ManagementIP = The IP address assigned to the Windows node. You can use ipconfig to find this
|
||||
# ClusterCIDR = The cluster subnet range. (Default value 10.244.0.0/16)
|
||||
# KubeDnsServiceIP = The Kubernetes DNS service IP (Default value 10.96.0.10)
|
||||
# LogDir = The directory where kubelet and kube-proxy logs are redirected into their respective output files (Default value C:\k)
|
||||
```
|
||||
|
||||
|
||||
If the above referenced script is not suitable, you can manually configure nssm.exe using the following examples.
|
||||
```powershell
|
||||
# Register flanneld.exe
|
||||
nssm install flanneld C:\flannel\flanneld.exe
|
||||
nssm set flanneld AppParameters --kubeconfig-file=c:\k\config --iface=<ManagementIP> --ip-masq=1 --kube-subnet-mgr=1
|
||||
nssm set flanneld AppEnvironmentExtra NODE_NAME=<hostname>
|
||||
nssm set flanneld AppDirectory C:\flannel
|
||||
nssm set flanneld AppDirectory C:\flannel
|
||||
nssm start flanneld
|
||||
|
||||
|
||||
# Register kubelet.exe
|
||||
# Microsoft releases the pause infrastructure container at mcr.microsoft.com/k8s/core/pause:1.0.0
|
||||
# For more info search for "pause" in the "Guide for adding Windows Nodes in Kubernetes"
|
||||
@@ -392,7 +392,7 @@ Your main source of help for troubleshooting your Kubernetes cluster should star
|
||||
nssm set kubelet AppParameters --hostname-override=<hostname> --v=6 --pod-infra-container-image=mcr.microsoft.com/k8s/core/pause:1.0.0 --resolv-conf="" --allow-privileged=true --enable-debugging-handlers --cluster-dns=<DNS-service-IP> --cluster-domain=cluster.local --kubeconfig=c:\k\config --hairpin-mode=promiscuous-bridge --image-pull-progress-deadline=20m --cgroups-per-qos=false --log-dir=<log directory> --logtostderr=false --enforce-node-allocatable="" --network-plugin=cni --cni-bin-dir=c:\k\cni --cni-conf-dir=c:\k\cni\config
|
||||
nssm set kubelet AppDirectory C:\k
|
||||
nssm start kubelet
|
||||
|
||||
|
||||
# Register kube-proxy.exe (l2bridge / host-gw)
|
||||
nssm install kube-proxy C:\k\kube-proxy.exe
|
||||
nssm set kube-proxy AppDirectory c:\k
|
||||
@@ -400,7 +400,7 @@ Your main source of help for troubleshooting your Kubernetes cluster should star
|
||||
nssm.exe set kube-proxy AppEnvironmentExtra KUBE_NETWORK=cbr0
|
||||
nssm set kube-proxy DependOnService kubelet
|
||||
nssm start kube-proxy
|
||||
|
||||
|
||||
# Register kube-proxy.exe (overlay / vxlan)
|
||||
nssm install kube-proxy C:\k\kube-proxy.exe
|
||||
nssm set kube-proxy AppDirectory c:\k
|
||||
@@ -408,8 +408,8 @@ Your main source of help for troubleshooting your Kubernetes cluster should star
|
||||
nssm set kube-proxy DependOnService kubelet
|
||||
nssm start kube-proxy
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
For initial troubleshooting, you can use the following flags in [nssm.exe](https://nssm.cc/) to redirect stdout and stderr to a output file:
|
||||
|
||||
```powershell
|
||||
@@ -498,7 +498,7 @@ Your main source of help for troubleshooting your Kubernetes cluster should star
|
||||
1. My Pods are stuck at "Container Creating" or restarting over and over
|
||||
|
||||
Check that your pause image is compatible with your OS version. The [instructions](https://docs.microsoft.com/en-us/virtualization/windowscontainers/kubernetes/deploying-resources) assume that both the OS and the containers are version 1803. If you have a later version of Windows, such as an Insider build, you need to adjust the images accordingly. Please refer to the Microsoft's [Docker repository](https://hub.docker.com/u/microsoft/) for images. Regardless, both the pause image Dockerfile and the sample service expect the image to be tagged as :latest.
|
||||
|
||||
|
||||
Starting with Kubernetes v1.14, Microsoft releases the pause infrastructure container at `mcr.microsoft.com/k8s/core/pause:1.0.0`. For more information search for "pause" in the [Guide for adding Windows Nodes in Kubernetes](../user-guide-windows-nodes).
|
||||
|
||||
1. DNS resolution is not properly working
|
||||
@@ -515,7 +515,7 @@ If these steps don't resolve your problem, you can get help running Windows cont
|
||||
|
||||
## Reporting Issues and Feature Requests
|
||||
|
||||
If you have what looks like a bug, or you would like to make a feature request, please use the [Github issue tracking system](https://github.com/kubernetes/kubernetes/issues). You can open issues on [GitHub](https://github.com/kubernetes/kubernetes/issues/new/choose) and assign them to SIG-Windows. You should first search the list of issues in case it was reported previously and comment with your experience on the issue and add additional logs. SIG-Windows Slack is also a great avenue to get some initial support and troubleshooting ideas prior to creating a ticket.
|
||||
If you have what looks like a bug, or you would like to make a feature request, please use the [GitHub issue tracking system](https://github.com/kubernetes/kubernetes/issues). You can open issues on [GitHub](https://github.com/kubernetes/kubernetes/issues/new/choose) and assign them to SIG-Windows. You should first search the list of issues in case it was reported previously and comment with your experience on the issue and add additional logs. SIG-Windows Slack is also a great avenue to get some initial support and troubleshooting ideas prior to creating a ticket.
|
||||
|
||||
If filing a bug, please include detailed information about how to reproduce the problem, such as:
|
||||
|
||||
+7
-7
@@ -21,7 +21,8 @@ The Kubernetes platform can now be used to run both Linux and Windows containers
|
||||
## Before you begin
|
||||
|
||||
* Obtain a [Windows Server license](https://www.microsoft.com/en-us/cloud-platform/windows-server-pricing) in order to configure the Windows node that hosts Windows containers. You can use your organization's licenses for the cluster, or acquire one from Microsoft, a reseller, or via the major cloud providers such as GCP, AWS, and Azure by provisioning a virtual machine running Windows Server through their marketplaces. A [time-limited trial](https://www.microsoft.com/en-us/cloud-platform/windows-server-trial) is also available.
|
||||
* Build a Linux-based Kubernetes cluster in which you have access to the control plane (some examples include [Getting Started from Scratch](/docs/setup/scratch/), [kubeadm](/docs/setup/independent/create-cluster-kubeadm/), [AKS Engine](/docs/setup/turnkey/azure/), [GCE](/docs/setup/turnkey/gce/), [AWS](/docs/setup/turnkey/aws/)).
|
||||
|
||||
* Build a Linux-based Kubernetes cluster in which you have access to the control plane (some examples include [Getting Started from Scratch](https://github.com/kubernetes/kubernetes/tree/master/build/), [kubeadm/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/), [AKS Engine](/docs/setup/production-environment/turnkey/azure/), [GCE](/docs/setup/production-environment/turnkey/gce/), [AWS](/docs/setup/production-environment/turnkey/aws/).
|
||||
|
||||
## Getting Started: Adding a Windows Node to Your Cluster
|
||||
|
||||
@@ -88,17 +89,16 @@ Once you have a Linux-based Kubernetes master node you are ready to choose a net
|
||||
}
|
||||
```
|
||||
|
||||
{{< note >}}
|
||||
The VNI must be set to 4096 and port 4789 for Flannel on Linux to interoperate with Flannel on Windows. Support for other VNIs is coming soon. See the VXLAN documentation at https://github.com/coreos/flannel/blob/master/Documentation/backends.md#vxlan for an explanation of these fields.
|
||||
{{< /note >}}
|
||||
{{< note >}}The VNI must be set to 4096 and port 4789 for Flannel on Linux to interoperate with Flannel on Windows. Support for other VNIs is coming soon. See the [VXLAN documentation](https://github.com/coreos/flannel/blob/master/Documentation/backends.md#vxlan)
|
||||
for an explanation of these fields.{{< /note >}}
|
||||
|
||||
1. In the `net-conf.json` section of your `kube-flannel.yml`, double-check:
|
||||
1. The cluster subnet (e.g. "10.244.0.0/16") is set as per your IP plan.
|
||||
* VNI 4096 is set in the backend
|
||||
* Port 4789 is set in the backend
|
||||
2. In the `cni-conf.json` section of your `kube-flannel.yml`, change the network name to `vxlan0`.
|
||||
|
||||
|
||||
|
||||
|
||||
Your `cni-conf.json` should look as follows:
|
||||
|
||||
```json
|
||||
@@ -191,7 +191,7 @@ All code snippets in Windows sections are to be run in a PowerShell environment
|
||||
{{< note >}}
|
||||
The "pause" (infrastructure) image is hosted on Microsoft Container Registry (MCR). You can access it using "docker pull mcr.microsoft.com/k8s/core/pause:1.0.0". The DOCKERFILE is available at https://github.com/Microsoft/SDN/blob/master/Kubernetes/windows/Dockerfile.
|
||||
{{< /note >}}
|
||||
|
||||
|
||||
1. Prepare a Windows directory for Kubernetes
|
||||
|
||||
Create a "Kubernetes for Windows" directory to store Kubernetes binaries as well as any deployment scripts and config files.
|
||||
|
Before Width: | Height: | Size: 80 KiB After Width: | Height: | Size: 80 KiB |
@@ -1,5 +1,4 @@
|
||||
---
|
||||
title: "Downloading Kubernetes"
|
||||
weight: 20
|
||||
title: "Release notes and version skew"
|
||||
weight: 10
|
||||
---
|
||||
|
||||
|
||||
@@ -1,33 +0,0 @@
|
||||
---
|
||||
reviewers:
|
||||
- david-mcmahon
|
||||
- jbeda
|
||||
title: Building a release
|
||||
content_template: templates/concept
|
||||
card:
|
||||
name: download
|
||||
weight: 20
|
||||
title: Building a release
|
||||
---
|
||||
{{% capture overview %}}
|
||||
You can either build a release from source or download a pre-built release. If you do not plan on developing Kubernetes itself, we suggest using a pre-built version of the current release, which can be found in the [Release Notes](/docs/setup/release/notes/).
|
||||
|
||||
The Kubernetes source code can be downloaded from the [kubernetes/kubernetes](https://github.com/kubernetes/kubernetes) repo.
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture body %}}
|
||||
## Building from source
|
||||
|
||||
If you are simply building a release from source there is no need to set up a full golang environment as all building happens in a Docker container.
|
||||
|
||||
Building a release is simple.
|
||||
|
||||
```shell
|
||||
git clone https://github.com/kubernetes/kubernetes.git
|
||||
cd kubernetes
|
||||
make release
|
||||
```
|
||||
|
||||
For more details on the release process see the kubernetes/kubernetes [`build`](http://releases.k8s.io/{{< param "githubbranch" >}}/build/) directory.
|
||||
|
||||
{{% /capture %}}
|
||||
@@ -1,8 +1,9 @@
|
||||
---
|
||||
title: v1.14 Release Notes
|
||||
weight: 10
|
||||
card:
|
||||
name: download
|
||||
weight: 10
|
||||
weight: 20
|
||||
anchors:
|
||||
- anchor: "#"
|
||||
title: Current Release Notes
|
||||
@@ -628,7 +629,7 @@ filename | sha512 hash
|
||||
* Introduce a RuntimeClass v1beta1 API. This new beta API renames `runtimeHandler` to `handler`, makes it a required field, and cuts out the spec (handler is a top-level field).
|
||||
* Transition CSINodeInfo and CSIDriver alpha CRDs to in-tree CSINode and CSIDriver core storage v1beta1 APIs. ([#74283](https://github.com/kubernetes/kubernetes/pull/74283), [@xing-yang](https://github.com/xing-yang))
|
||||
* ACTION REQUIRED: the alpha CRDs are no longer used and drivers will need to be updated to use the beta APIs.
|
||||
* The support for `_` in the CSI driver name will be dropped as the CSI Spec does not allow that.
|
||||
* The support for `_` in the CSI driver name will be dropped as the CSI Spec does not allow that.
|
||||
|
||||
### Other notable changes
|
||||
|
||||
@@ -659,7 +660,7 @@ filename | sha512 hash
|
||||
* kube-apiserver now serves OpenAPI specs for registered CRDs with defined ([#71192](https://github.com/kubernetes/kubernetes/pull/71192), [@roycaihw](https://github.com/roycaihw))
|
||||
* validation schemata as an alpha feature, to be enabled via the "CustomResourcePublishOpenAPI" feature gate. Kubectl will validate client-side using those. Note that in
|
||||
* future, client-side validation in 1.14 kubectl against a 1.15 cluster will reject
|
||||
* unknown fields for CRDs with validation schema defined.
|
||||
* unknown fields for CRDs with validation schema defined.
|
||||
* Fix kubelet start failure issue on Azure Stack due to InstanceMetadata setting ([#74936](https://github.com/kubernetes/kubernetes/pull/74936), [@rjaini](https://github.com/rjaini))
|
||||
* add subcommand `kubectl create cronjob` ([#71651](https://github.com/kubernetes/kubernetes/pull/71651), [@Pingan2017](https://github.com/Pingan2017))
|
||||
* The CSIBlockVolume feature gate is now beta, and defaults to enabled. ([#74909](https://github.com/kubernetes/kubernetes/pull/74909), [@bswartz](https://github.com/bswartz))
|
||||
@@ -716,7 +717,7 @@ filename | sha512 hash
|
||||
* reflector_short_watches_total
|
||||
* reflector_watch_duration_seconds
|
||||
* reflector_watches_total
|
||||
* While this is a backwards-incompatible change, it would have been impossible to setup reliable monitoring around these metrics since the labels were not stable.
|
||||
* While this is a backwards-incompatible change, it would have been impossible to setup reliable monitoring around these metrics since the labels were not stable.
|
||||
* Add a configuration field to shorten the timeout of validating/mutating admission webhook call. The timeout value must be between 1 and 30 seconds. Default to 30 seconds when unspecified. ([#74562](https://github.com/kubernetes/kubernetes/pull/74562), [@roycaihw](https://github.com/roycaihw))
|
||||
* client-go: PortForwarder.GetPorts() now contain correct local port if no local port was initially specified when setting up the port forwarder ([#73676](https://github.com/kubernetes/kubernetes/pull/73676), [@martin-helmich](https://github.com/martin-helmich))
|
||||
* # Apply resources from a directory containing kustomization.yaml ([#74140](https://github.com/kubernetes/kubernetes/pull/74140), [@Liujingfang1](https://github.com/Liujingfang1))
|
||||
@@ -733,7 +734,7 @@ filename | sha512 hash
|
||||
* Image garbage collection no longer fails for images with only one tag but more than one repository associated. ([#70647](https://github.com/kubernetes/kubernetes/pull/70647), [@corvus-ch](https://github.com/corvus-ch))
|
||||
* - Fix liveness probe in fluentd-gcp cluster addon ([#74522](https://github.com/kubernetes/kubernetes/pull/74522), [@Pluies](https://github.com/Pluies))
|
||||
* The new test ``[sig-network] DNS should provide /etc/hosts entries for the cluster [LinuxOnly] [Conformance]`` will validate the host entries set in the ``/etc/hosts`` file (pod's FQDN and hostname), which should be managed by Kubelet. ([#72729](https://github.com/kubernetes/kubernetes/pull/72729), [@bclau](https://github.com/bclau))
|
||||
* The test has the tag ``[LinuxOnly]`` because individual files cannot be mounted in Windows Containers, which means that it cannot pass using Windows nodes.
|
||||
* The test has the tag ``[LinuxOnly]`` because individual files cannot be mounted in Windows Containers, which means that it cannot pass using Windows nodes.
|
||||
|
||||
|
||||
|
||||
@@ -903,11 +904,11 @@ filename | sha512 hash
|
||||
* Custom apiservers built with the latest apiserver library will have the 100MB limit on the body of resource requests as well. The limit can be altered via ServerRunOptions.MaxRequestBodyBytes.
|
||||
* The body size limit does not apply to subresources like pods/proxy that proxy request content to another server.
|
||||
* Kustomize is developed in its own repo https://github.com/kubernetes-sigs/kustomize ([#73033](https://github.com/kubernetes/kubernetes/pull/73033), [@Liujingfang1](https://github.com/Liujingfang1))
|
||||
* This PR added a new subcommand `kustomize` in kubectl.
|
||||
* This PR added a new subcommand `kustomize` in kubectl.
|
||||
* kubectl kustomize <somedir> has the same effect as kustomize build <somedir>
|
||||
* To build API resources from somedir with a kustomization.yaml file
|
||||
* kubectl kustomize <somedir>
|
||||
* This command can be piped to apply or delete
|
||||
* This command can be piped to apply or delete
|
||||
* kubectl kustomize <somedir> | kubectl apply -f -
|
||||
* kubectl kustomize <somedir> | kubectl delete -f -
|
||||
* kubeadm: all master components are now exclusively relying on the `PriorityClassName` pod spec for annotating them as cluster critical components. Since `scheduler.alpha.kubernetes.io/critical-pod` annotation is no longer supported by Kubernetes 1.14 this annotation is no longer added to master components. ([#73857](https://github.com/kubernetes/kubernetes/pull/73857), [@ereslibre](https://github.com/ereslibre))
|
||||
|
||||
+5
-5
@@ -6,9 +6,9 @@ reviewers:
|
||||
- sig-cluster-lifecycle
|
||||
- sig-node
|
||||
- sig-release
|
||||
title: Kubernetes Version and Version Skew Support Policy
|
||||
title: Kubernetes version and version skew support policy
|
||||
content_template: templates/concept
|
||||
weight: 70
|
||||
weight: 30
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
@@ -37,7 +37,7 @@ Minor releases occur approximately every 3 months, so each minor release branch
|
||||
|
||||
### kube-apiserver
|
||||
|
||||
In [highly-availabile (HA) clusters](/docs/setup/independent/high-availability/), the newest and oldest `kube-apiserver` instances must be within one minor version.
|
||||
In [highly-available (HA) clusters](/docs/setup/production-environment/tools/independent/high-availability/), the newest and oldest `kube-apiserver` instances must be within one minor version.
|
||||
|
||||
Example:
|
||||
|
||||
@@ -119,8 +119,8 @@ Pre-requisites:
|
||||
Upgrade `kube-apiserver` to **1.(n+1)**
|
||||
|
||||
{{< note >}}
|
||||
Project policies for [API deprecation](/docs/reference/using-api/deprecation-policy/) and
|
||||
[API change guidelines](https://github.com/kubernetes/community/blob/master/contributors/devel/api_changes.md)
|
||||
Project policies for [API deprecation](/docs/reference/using-api/deprecation-policy/) and
|
||||
[API change guidelines](https://github.com/kubernetes/community/blob/master/contributors/devel/api_changes.md)
|
||||
require `kube-apiserver` to not skip minor versions when upgrading, even in single-instance clusters.
|
||||
{{< /note >}}
|
||||
|
||||
@@ -0,0 +1,148 @@
|
||||
---
|
||||
reviewers:
|
||||
- sig-api-machinery
|
||||
- sig-architecture
|
||||
- sig-cli
|
||||
- sig-cluster-lifecycle
|
||||
- sig-node
|
||||
- sig-release
|
||||
title: Kubernetes version and version skew support policy
|
||||
content_template: templates/concept
|
||||
weight: 30
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
This document describes the maximum version skew supported between various Kubernetes components.
|
||||
Specific cluster deployment tools may place additional restrictions on version skew.
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture body %}}
|
||||
|
||||
## Supported versions
|
||||
|
||||
Kubernetes versions are expressed as **x.y.z**,
|
||||
where **x** is the major version, **y** is the minor version, and **z** is the patch version, following [Semantic Versioning](http://semver.org/) terminology.
|
||||
For more information, see [Kubernetes Release Versioning](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/release/versioning.md#kubernetes-release-versioning).
|
||||
|
||||
The Kubernetes project maintains release branches for the most recent three minor releases.
|
||||
|
||||
Applicable fixes, including security fixes, may be backported to those three release branches, depending on severity and feasibility.
|
||||
Patch releases are cut from those branches at a regular cadence, or as needed.
|
||||
This decision is owned by the [patch release manager](https://github.com/kubernetes/sig-release/blob/master/release-team/role-handbooks/patch-release-manager/README.md#release-timing).
|
||||
The patch release manager is a member of the [release team for each release](https://github.com/kubernetes/sig-release/tree/master/releases/).
|
||||
|
||||
Minor releases occur approximately every 3 months, so each minor release branch is maintained for approximately 9 months.
|
||||
|
||||
## Supported version skew
|
||||
|
||||
### kube-apiserver
|
||||
|
||||
In [highly-available (HA) clusters](/docs/setup/production-environment/tools/independent/high-availability/), the newest and oldest `kube-apiserver` instances must be within one minor version.
|
||||
|
||||
Example:
|
||||
|
||||
* newest `kube-apiserver` is at **1.13**
|
||||
* other `kube-apiserver` instances are supported at **1.13** and **1.12**
|
||||
|
||||
### kubelet
|
||||
|
||||
`kubelet` must not be newer than `kube-apiserver`, and may be up to two minor versions older.
|
||||
|
||||
Example:
|
||||
|
||||
* `kube-apiserver` is at **1.13**
|
||||
* `kubelet` is supported at **1.13**, **1.12**, and **1.11**
|
||||
|
||||
{{< note >}}
|
||||
If version skew exists between `kube-apiserver` instances in an HA cluster, this narrows the allowed `kubelet` versions.
|
||||
{{</ note >}}
|
||||
|
||||
Example:
|
||||
|
||||
* `kube-apiserver` instances are at **1.13** and **1.12**
|
||||
* `kubelet` is supported at **1.12**, and **1.11** (**1.13** is not supported because that would be newer than the `kube-apiserver` instance at version **1.12**)
|
||||
|
||||
### kube-controller-manager, kube-scheduler, and cloud-controller-manager
|
||||
|
||||
`kube-controller-manager`, `kube-scheduler`, and `cloud-controller-manager` must not be newer than the `kube-apiserver` instances they communicate with. They are expected to match the `kube-apiserver` minor version, but may be up to one minor version older (to allow live upgrades).
|
||||
|
||||
Example:
|
||||
|
||||
* `kube-apiserver` is at **1.13**
|
||||
* `kube-controller-manager`, `kube-scheduler`, and `cloud-controller-manager` are supported at **1.13** and **1.12**
|
||||
|
||||
{{< note >}}
|
||||
If version skew exists between `kube-apiserver` instances in an HA cluster, and these components can communicate with any `kube-apiserver` instance in the cluster (for example, via a load balancer), this narrows the allowed versions of these components.
|
||||
{{< /note >}}
|
||||
|
||||
Example:
|
||||
|
||||
* `kube-apiserver` instances are at **1.13** and **1.12**
|
||||
* `kube-controller-manager`, `kube-scheduler`, and `cloud-controller-manager` communicate with a load balancer that can route to any `kube-apiserver` instance
|
||||
* `kube-controller-manager`, `kube-scheduler`, and `cloud-controller-manager` are supported at **1.12** (**1.13** is not supported because that would be newer than the `kube-apiserver` instance at version **1.12**)
|
||||
|
||||
### kubectl
|
||||
|
||||
`kubectl` is supported within one minor version (older or newer) of `kube-apiserver`.
|
||||
|
||||
Example:
|
||||
|
||||
* `kube-apiserver` is at **1.13**
|
||||
* `kubectl` is supported at **1.14**, **1.13**, and **1.12**
|
||||
|
||||
{{< note >}}
|
||||
If version skew exists between `kube-apiserver` instances in an HA cluster, this narrows the supported `kubectl` versions.
|
||||
{{< /note >}}
|
||||
|
||||
Example:
|
||||
|
||||
* `kube-apiserver` instances are at **1.13** and **1.12**
|
||||
* `kubectl` is supported at **1.13** and **1.12** (other versions would be more than one minor version skewed from one of the `kube-apiserver` components)
|
||||
|
||||
## Supported component upgrade order
|
||||
|
||||
The supported version skew between components has implications on the order in which components must be upgraded.
|
||||
This section describes the order in which components must be upgraded to transition an existing cluster from version **1.n** to version **1.(n+1)**.
|
||||
|
||||
### kube-apiserver
|
||||
|
||||
Pre-requisites:
|
||||
|
||||
* In a single-instance cluster, the existing `kube-apiserver` instance is **1.n**
|
||||
* In an HA cluster, all `kube-apiserver` instances are at **1.n** or **1.(n+1)** (this ensures maximum skew of 1 minor version between the oldest and newest `kube-apiserver` instance)
|
||||
* The `kube-controller-manager`, `kube-scheduler`, and `cloud-controller-manager` instances that communicate with this server are at version **1.n** (this ensures they are not newer than the existing API server version, and are within 1 minor version of the new API server version)
|
||||
* `kubelet` instances on all nodes are at version **1.n** or **1.(n-1)** (this ensures they are not newer than the existing API server version, and are within 2 minor versions of the new API server version)
|
||||
* Registered admission webhooks are able to handle the data the new `kube-apiserver` instance will send them:
|
||||
* `ValidatingWebhookConfiguration` and `MutatingWebhookConfiguration` objects are updated to include any new versions of REST resources added in **1.(n+1)** (or use the [`matchPolicy: Equivalent` option](/docs/reference/access-authn-authz/extensible-admission-controllers/#matching-requests-matchpolicy) available in v1.15+)
|
||||
* The webhooks are able to handle any new versions of REST resources that will be sent to them, and any new fields added to existing versions in **1.(n+1)**
|
||||
|
||||
Upgrade `kube-apiserver` to **1.(n+1)**
|
||||
|
||||
{{< note >}}
|
||||
Project policies for [API deprecation](/docs/reference/using-api/deprecation-policy/) and
|
||||
[API change guidelines](https://github.com/kubernetes/community/blob/master/contributors/devel/api_changes.md)
|
||||
require `kube-apiserver` to not skip minor versions when upgrading, even in single-instance clusters.
|
||||
{{< /note >}}
|
||||
|
||||
### kube-controller-manager, kube-scheduler, and cloud-controller-manager
|
||||
|
||||
Pre-requisites:
|
||||
|
||||
* The `kube-apiserver` instances these components communicate with are at **1.(n+1)** (in HA clusters in which these control plane components can communicate with any `kube-apiserver` instance in the cluster, all `kube-apiserver` instances must be upgraded before upgrading these components)
|
||||
|
||||
Upgrade `kube-controller-manager`, `kube-scheduler`, and `cloud-controller-manager` to **1.(n+1)**
|
||||
|
||||
### kubelet
|
||||
|
||||
Pre-requisites:
|
||||
|
||||
* The `kube-apiserver` instances the `kubelet` communicates with are at **1.(n+1)**
|
||||
|
||||
Optionally upgrade `kubelet` instances to **1.(n+1)** (or they can be left at **1.n** or **1.(n-1)**)
|
||||
|
||||
{{< warning >}}
|
||||
Running a cluster with `kubelet` instances that are persistently two minor versions behind `kube-apiserver` is not recommended:
|
||||
|
||||
* they must be upgraded within one minor version of `kube-apiserver` before the control plane can be upgraded
|
||||
* it increases the likelihood of running `kubelet` versions older than the three maintained minor releases
|
||||
{{</ warning >}}
|
||||
Reference in New Issue
Block a user