Merge branch 'merged-master-dev-1.16' of github.com:simplytunde/website into merged-master-dev-1.16
This commit is contained in:
simplytunde
@@ -42,7 +42,7 @@ Example usage:
|
||||
```yaml
|
||||
apiVersion: kubeadm.k8s.io/v1beta2
|
||||
kind: ClusterConfiguration
|
||||
kubernetesVersion: v1.13.0
|
||||
kubernetesVersion: v1.16.0
|
||||
apiServer:
|
||||
extraArgs:
|
||||
advertise-address: 192.168.0.103
|
||||
@@ -59,7 +59,7 @@ Example usage:
|
||||
```yaml
|
||||
apiVersion: kubeadm.k8s.io/v1beta2
|
||||
kind: ClusterConfiguration
|
||||
kubernetesVersion: v1.13.0
|
||||
kubernetesVersion: v1.16.0
|
||||
controllerManager:
|
||||
extraArgs:
|
||||
cluster-signing-key-file: /home/johndoe/keys/ca.key
|
||||
@@ -75,7 +75,7 @@ Example usage:
|
||||
```yaml
|
||||
apiVersion: kubeadm.k8s.io/v1beta2
|
||||
kind: ClusterConfiguration
|
||||
kubernetesVersion: v1.13.0
|
||||
kubernetesVersion: v1.16.0
|
||||
scheduler:
|
||||
extraArgs:
|
||||
address: 0.0.0.0
|
||||
|
||||
+1
-18
@@ -35,17 +35,6 @@ but you may also build them from source for other OSes.
|
||||
|
||||
### kubeadm maturity
|
||||
|
||||
| Area | Maturity Level |
|
||||
|---------------------------|--------------- |
|
||||
| Command line UX | GA |
|
||||
| Implementation | GA |
|
||||
| Config file API | Beta |
|
||||
| CoreDNS | GA |
|
||||
| kubeadm alpha subcommands | Alpha |
|
||||
| High availability | Beta |
|
||||
| DynamicKubeletConfig | Alpha |
|
||||
|
||||
|
||||
kubeadm's overall feature state is **GA**. Some sub-features, like the configuration
|
||||
file API are still under active development. The implementation of creating the cluster
|
||||
may change slightly as the tool evolves, but the overall implementation should be pretty stable.
|
||||
@@ -61,16 +50,10 @@ timeframe; which also applies to `kubeadm`.
|
||||
|
||||
| Kubernetes version | Release month | End-of-life-month |
|
||||
|--------------------|----------------|-------------------|
|
||||
| v1.6.x | March 2017 | December 2017 |
|
||||
| v1.7.x | June 2017 | March 2018 |
|
||||
| v1.8.x | September 2017 | June 2018 |
|
||||
| v1.9.x | December 2017 | September 2018 |
|
||||
| v1.10.x | March 2018 | December 2018 |
|
||||
| v1.11.x | June 2018 | March 2019 |
|
||||
| v1.12.x | September 2018 | June 2019 |
|
||||
| v1.13.x | December 2018 | September 2019 |
|
||||
| v1.14.x | March 2019 | December 2019 |
|
||||
| v1.15.x | June 2019 | March 2020 |
|
||||
| v1.16.x | September 2019 | June 2020 |
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
|
||||
@@ -201,17 +201,17 @@ systemctl enable --now kubelet
|
||||
Install CNI plugins (required for most pod network):
|
||||
|
||||
```bash
|
||||
CNI_VERSION="v0.7.5"
|
||||
CNI_VERSION="v0.8.2"
|
||||
mkdir -p /opt/cni/bin
|
||||
curl -L "https://github.com/containernetworking/plugins/releases/download/${CNI_VERSION}/cni-plugins-amd64-${CNI_VERSION}.tgz" | tar -C /opt/cni/bin -xz
|
||||
curl -L "https://github.com/containernetworking/plugins/releases/download/${CNI_VERSION}/cni-plugins-linux-amd64-${CNI_VERSION}.tgz" | tar -C /opt/cni/bin -xz
|
||||
```
|
||||
|
||||
Install crictl (required for kubeadm / Kubelet Container Runtime Interface (CRI))
|
||||
|
||||
```bash
|
||||
CRICTL_VERSION="v1.12.0"
|
||||
CRICTL_VERSION="v1.16.0"
|
||||
mkdir -p /opt/bin
|
||||
curl -L "https://github.com/kubernetes-incubator/cri-tools/releases/download/${CRICTL_VERSION}/crictl-${CRICTL_VERSION}-linux-amd64.tar.gz" | tar -C /opt/bin -xz
|
||||
curl -L "https://github.com/kubernetes-sigs/cri-tools/releases/download/${CRICTL_VERSION}/crictl-${CRICTL_VERSION}-linux-amd64.tar.gz" | tar -C /opt/bin -xz
|
||||
```
|
||||
|
||||
Install `kubeadm`, `kubelet`, `kubectl` and add a `kubelet` systemd service:
|
||||
|
||||
@@ -10,7 +10,7 @@ weight: 100
|
||||
|
||||
### Self-hosting the Kubernetes control plane {#self-hosting}
|
||||
|
||||
As of 1.8, you can experimentally create a _self-hosted_ Kubernetes control
|
||||
kubeadm allows you to experimentally create a _self-hosted_ Kubernetes control
|
||||
plane. This means that key components such as the API server, controller
|
||||
manager, and scheduler run as [DaemonSet pods](/docs/concepts/workloads/controllers/daemonset/)
|
||||
configured via the Kubernetes API instead of [static pods](/docs/tasks/administer-cluster/static-pod/)
|
||||
|
||||
+39
-7
@@ -102,14 +102,32 @@ Pods, Controllers and Services are critical elements to managing Windows workloa
|
||||
|
||||
Docker EE-basic 18.09 is required on Windows Server 2019 / 1809 nodes for Kubernetes. This works with the dockershim code included in the kubelet. Additional runtimes such as CRI-ContainerD may be supported in later Kubernetes versions.
|
||||
|
||||
#### Storage
|
||||
#### Persistent Storage
|
||||
|
||||
Kubernetes Volumes enable complex applications with data persistence and Pod volume sharing requirements to be deployed on Kubernetes. Kubernetes on Windows supports the following types of [volumes](/docs/concepts/storage/volumes/):
|
||||
Kubernetes [volumes](/docs/concepts/storage/volumes/) enable complex applications, with data persistence and Pod volume sharing requirements, to be deployed on Kubernetes. Management of persistent volumes associated with a specific storage back-end or protocol includes actions such as: provisioning/de-provisioning/resizing of volumes, attaching/detaching a volume to/from a Kubernetes node and mounting/dismounting a volume to/from individual containers in a pod that needs to persist data. The code implementing these volume management actions for a specific storage back-end or protocol is shipped in the form of a Kubernetes volume [plugin](/docs/concepts/storage/volumes/#types-of-volumes). The following broad classes of Kubernetes volume plugins are supported on Windows:
|
||||
|
||||
##### In-tree Volume Plugins
|
||||
Code associated with in-tree volume plugins ship as part of the core Kubernetes code base. Deployment of in-tree volume plugins do not require installation of additional scripts or deployment of separate containerized plugin components. These plugins can handle: provisioning/de-provisioning and resizing of volumes in the storage backend, attaching/detaching of volumes to/from a Kubernetes node and mounting/dismounting a volume to/from individual containers in a pod. The following in-tree plugins support Windows nodes:
|
||||
|
||||
* FlexVolume out-of-tree plugin with [SMB and iSCSI](https://github.com/Microsoft/K8s-Storage-Plugins/tree/master/flexvolume/windows) support
|
||||
* [azureDisk](/docs/concepts/storage/volumes/#azuredisk)
|
||||
* [azureFile](/docs/concepts/storage/volumes/#azurefile)
|
||||
* [gcePersistentDisk](/docs/concepts/storage/volumes/#gcepersistentdisk)
|
||||
* [awsElasticBlockStore](/docs/concepts/storage/volumes/#awselasticblockstore)
|
||||
* [vsphereVolume](/docs/concepts/storage/volumes/#vspherevolume)
|
||||
|
||||
##### FlexVolume Plugins
|
||||
Code associated with [FlexVolume](/docs/concepts/storage/volumes/#flexVolume) plugins ship as out-of-tree scripts or binaries that need to be deployed directly on the host. FlexVolume plugins handle attaching/detaching of volumes to/from a Kubernetes node and mounting/dismounting a volume to/from individual containers in a pod. Provisioning/De-provisioning of persistent volumes associated with FlexVolume plugins may be handled through an external provisioner that is typically separate from the FlexVolume plugins. The following FlexVolume [plugins](https://github.com/Microsoft/K8s-Storage-Plugins/tree/master/flexvolume/windows), deployed as powershell scripts on the host, support Windows nodes:
|
||||
|
||||
* [SMB](https://github.com/microsoft/K8s-Storage-Plugins/tree/master/flexvolume/windows/plugins/microsoft.com~smb.cmd)
|
||||
* [iSCSI](https://github.com/microsoft/K8s-Storage-Plugins/tree/master/flexvolume/windows/plugins/microsoft.com~iscsi.cmd)
|
||||
|
||||
##### CSI Plugins
|
||||
|
||||
{{< feature-state for_k8s_version="v1.16" state="alpha" >}}
|
||||
|
||||
Code associated with {{< glossary_tooltip text="CSI" term_id="csi" >}} plugins ship as out-of-tree scripts and binaries that are typically distributed as container images and deployed using standard Kubernetes constructs like DaemonSets and StatefulSets. CSI plugins handle a wide range of volume management actions in Kubernetes: provisioning/de-provisioning/resizing of volumes, attaching/detaching of volumes to/from a Kubernetes node and mounting/dismounting a volume to/from individual containers in a pod, backup/restore of persistent data using snapshots and cloning. CSI plugins typically consist of node plugins (that run on each node as a DaemonSet) and controller plugins.
|
||||
|
||||
CSI node plugins (especially those associated with persistent volumes exposed as either block devices or over a shared file-system) need to perform various privileged operations like scanning of disk devices, mounting of file systems, etc. These operations differ for each host operating system. For Linux worker nodes, containerized CSI node plugins are typically deployed as privileged containers. For Windows worker nodes, privileged operations for containerized CSI node plugins is supported using [csi-proxy](https://github.com/kubernetes-csi/csi-proxy), a community-managed, stand-alone binary that needs to be pre-installed on each Windows node. Please refer to the deployment guide of the CSI plugin you wish to deploy for further details.
|
||||
|
||||
#### Networking
|
||||
|
||||
@@ -131,13 +149,13 @@ Windows supports five different networking drivers/modes: L2bridge, L2tunnel, Ov
|
||||
|
||||
| Network Driver | Description | Container Packet Modifications | Network Plugins | Network Plugin Characteristics |
|
||||
| -------------- | ----------- | ------------------------------ | --------------- | ------------------------------ |
|
||||
| L2bridge | Containers are attached to an external vSwitch. Containers are attached to the underlay network, although the physical network doesn't need to learn the container MACs because they are rewritten on ingress/egress. Inter-container traffic is bridged inside the container host. | MAC is rewritten to host MAC, IP remains the same. | [win-bridge](https://github.com/containernetworking/plugins/tree/master/plugins/main/windows/win-bridge), [Azure-CNI](https://github.com/Azure/azure-container-networking/blob/master/docs/cni.md), Flannel host-gateway uses win-bridge | win-bridge uses L2bridge network mode, connects containers to the underlay of hosts, offering best performance. Requires L2 adjacency between container hosts |
|
||||
| L2bridge | Containers are attached to an external vSwitch. Containers are attached to the underlay network, although the physical network doesn't need to learn the container MACs because they are rewritten on ingress/egress. Inter-container traffic is bridged inside the container host. | MAC is rewritten to host MAC, IP remains the same. | [win-bridge](https://github.com/containernetworking/plugins/tree/master/plugins/main/windows/win-bridge), [Azure-CNI](https://github.com/Azure/azure-container-networking/blob/master/docs/cni.md), Flannel host-gateway uses win-bridge | win-bridge uses L2bridge network mode, connects containers to the underlay of hosts, offering best performance. Requires user-defined routes (UDR) for inter-node connectivity. |
|
||||
| L2Tunnel | This is a special case of l2bridge, but only used on Azure. All packets are sent to the virtualization host where SDN policy is applied. | MAC rewritten, IP visible on the underlay network | [Azure-CNI](https://github.com/Azure/azure-container-networking/blob/master/docs/cni.md) | Azure-CNI allows integration of containers with Azure vNET, and allows them to leverage the set of capabilities that [Azure Virtual Network provides](https://azure.microsoft.com/en-us/services/virtual-network/). For example, securely connect to Azure services or use Azure NSGs. See [azure-cni for some examples](https://docs.microsoft.com/en-us/azure/aks/concepts-network#azure-cni-advanced-networking) |
|
||||
| Overlay (Overlay networking for Windows in Kubernetes is in *alpha* stage) | Containers are given a vNIC connected to an external vSwitch. Each overlay network gets its own IP subnet, defined by a custom IP prefix.The overlay network driver uses VXLAN encapsulation. | Encapsulated with an outer header, inner packet remains the same. | [Win-overlay](https://github.com/containernetworking/plugins/tree/master/plugins/main/windows/win-overlay), Flannel VXLAN (uses win-overlay) | win-overlay should be used when virtual container networks are desired to be isolated from underlay of hosts (e.g. for security reasons). Allows for IPs to be re-used for different overlay networks (which have different VNID tags) if you are restricted on IPs in your datacenter. This option may be used when the container hosts are not L2 adjacent but have L3 connectivity |
|
||||
| Overlay (Overlay networking for Windows in Kubernetes is in *alpha* stage) | Containers are given a vNIC connected to an external vSwitch. Each overlay network gets its own IP subnet, defined by a custom IP prefix.The overlay network driver uses VXLAN encapsulation. | Encapsulated with an outer header. | [Win-overlay](https://github.com/containernetworking/plugins/tree/master/plugins/main/windows/win-overlay), Flannel VXLAN (uses win-overlay) | win-overlay should be used when virtual container networks are desired to be isolated from underlay of hosts (e.g. for security reasons). Allows for IPs to be re-used for different overlay networks (which have different VNID tags) if you are restricted on IPs in your datacenter. This option requires [KB4489899](https://support.microsoft.com/help/4489899) on Windows Server 2019. |
|
||||
| Transparent (special use case for [ovn-kubernetes](https://github.com/openvswitch/ovn-kubernetes)) | Requires an external vSwitch. Containers are attached to an external vSwitch which enables intra-pod communication via logical networks (logical switches and routers). | Packet is encapsulated either via [GENEVE](https://datatracker.ietf.org/doc/draft-gross-geneve/) or [STT](https://datatracker.ietf.org/doc/draft-davie-stt/) tunneling to reach pods which are not on the same host. <br/> Packets are forwarded or dropped via the tunnel metadata information supplied by the ovn network controller. <br/> NAT is done for north-south communication. | [ovn-kubernetes](https://github.com/openvswitch/ovn-kubernetes) | [Deploy via ansible](https://github.com/openvswitch/ovn-kubernetes/tree/master/contrib). Distributed ACLs can be applied via Kubernetes policies. IPAM support. Load-balancing can be achieved without kube-proxy. NATing is done without using iptables/netsh. |
|
||||
| NAT (*not used in Kubernetes*) | Containers are given a vNIC connected to an internal vSwitch. DNS/DHCP is provided using an internal component called [WinNAT](https://blogs.technet.microsoft.com/virtualization/2016/05/25/windows-nat-winnat-capabilities-and-limitations/) | MAC and IP is rewritten to host MAC/IP. | [nat](https://github.com/Microsoft/windows-container-networking/tree/master/plugins/nat) | Included here for completeness |
|
||||
|
||||
As outlined above, the [Flannel](https://github.com/coreos/flannel) CNI [meta plugin](https://github.com/containernetworking/plugins/tree/master/plugins/meta/flannel) is also supported on [Windows](https://github.com/containernetworking/plugins/tree/master/plugins/meta/flannel#windows-support-experimental) via the [VXLAN network backend](https://github.com/coreos/flannel/blob/master/Documentation/backends.md#vxlan) (**alpha support** ; delegates to win-overlay) and [host-gateway network backend](https://github.com/coreos/flannel/blob/master/Documentation/backends.md#host-gw) (stable support; delegates to win-bridge). This plugin supports delegating to one of the reference CNI plugins (win-overlay, win-bridge), to work in conjunction with Flannel daemon on Windows (Flanneld) for automatic node subnet lease assignment and HNS network creation. This plugin reads in its own configuration file (net-conf.json), and aggregates it with the environment variables from the FlannelD generated subnet.env file. It then delegates to one of the reference CNI plugins for network plumbing, and sends the correct configuration containing the node-assigned subnet to the IPAM plugin (e.g. host-local).
|
||||
As outlined above, the [Flannel](https://github.com/coreos/flannel) CNI [meta plugin](https://github.com/containernetworking/plugins/tree/master/plugins/meta/flannel) is also supported on [Windows](https://github.com/containernetworking/plugins/tree/master/plugins/meta/flannel#windows-support-experimental) via the [VXLAN network backend](https://github.com/coreos/flannel/blob/master/Documentation/backends.md#vxlan) (**alpha support** ; delegates to win-overlay) and [host-gateway network backend](https://github.com/coreos/flannel/blob/master/Documentation/backends.md#host-gw) (stable support; delegates to win-bridge). This plugin supports delegating to one of the reference CNI plugins (win-overlay, win-bridge), to work in conjunction with Flannel daemon on Windows (Flanneld) for automatic node subnet lease assignment and HNS network creation. This plugin reads in its own configuration file (cni.conf), and aggregates it with the environment variables from the FlannelD generated subnet.env file. It then delegates to one of the reference CNI plugins for network plumbing, and sends the correct configuration containing the node-assigned subnet to the IPAM plugin (e.g. host-local).
|
||||
|
||||
For the node, pod, and service objects, the following network flows are supported for TCP/UDP traffic:
|
||||
|
||||
@@ -218,7 +236,6 @@ As a result, the following storage functionality is not supported on Windows nod
|
||||
* Read-only root filesystem. Mapped volumes still support readOnly
|
||||
* Block device mapping
|
||||
* Memory as the storage medium
|
||||
* CSI plugins which require privileged containers
|
||||
* File system features like uui/guid, per-user Linux filesystem permissions
|
||||
* NFS based storage/volume support
|
||||
* Expanding the mounted volume (resizefs)
|
||||
@@ -259,6 +276,7 @@ These features were added in Kubernetes v1.15:
|
||||
|
||||
* ClusterFirstWithHostNet is not supported for DNS. Windows treats all names with a '.' as a FQDN and skips PQDN resolution
|
||||
* On Linux, you have a DNS suffix list, which is used when trying to resolve PQDNs. On Windows, we only have 1 DNS suffix, which is the DNS suffix associated with that pod's namespace (mydns.svc.cluster.local for example). Windows can resolve FQDNs and services or names resolvable with just that suffix. For example, a pod spawned in the default namespace, will have the DNS suffix **default.svc.cluster.local**. On a Windows pod, you can resolve both **kubernetes.default.svc.cluster.local** and **kubernetes**, but not the in-betweens, like **kubernetes.default** or **kubernetes.default.svc**.
|
||||
* On Windows, there are multiple DNS resolvers that can be used. As these come with slightly different behaviors, using the `Resolve-DNSName` utility for name query resolutions is recommended.
|
||||
|
||||
##### Security
|
||||
|
||||
@@ -513,6 +531,20 @@ Your main source of help for troubleshooting your Kubernetes cluster should star
|
||||
This was implemented in Kubernetes 1.15, and the pause infrastructure container `mcr.microsoft.com/k8s/core/pause:1.2.0`. Be sure to use these versions or newer ones.
|
||||
If you would like to build your own pause infrastructure container, be sure to include [wincat](https://github.com/kubernetes-sigs/sig-windows-tools/tree/master/cmd/wincat)
|
||||
|
||||
1. My Kubernetes installation is failing because my Windows Server node is behind a proxy
|
||||
|
||||
If you are behind a proxy, the following PowerShell environment variables must be defined:
|
||||
```PowerShell
|
||||
[Environment]::SetEnvironmentVariable("HTTP_PROXY", "http://proxy.example.com:80/", [EnvironmentVariableTarget]::Machine)
|
||||
[Environment]::SetEnvironmentVariable("HTTPS_PROXY", "http://proxy.example.com:443/", [EnvironmentVariableTarget]::Machine)
|
||||
```
|
||||
|
||||
1. What is a `pause` container?
|
||||
|
||||
In a Kubernetes Pod, an infrastructure or "pause" container is first created to host the container endpoint. Containers that belong to the same pod, including infrastructure and worker containers, share a common network namespace and endpoint (same IP and port space). Pause containers are needed to accomodate worker containers crashing or restarting without losing any of the networking configuration.
|
||||
|
||||
The "pause" (infrastructure) image is hosted on Microsoft Container Registry (MCR). You can access it using `docker pull mcr.microsoft.com/k8s/core/pause:1.2.0`. For more details, see the [DOCKERFILE](https://github.com/kubernetes-sigs/sig-windows-tools/tree/master/cmd/wincat).
|
||||
|
||||
### Further investigation
|
||||
|
||||
If these steps don't resolve your problem, you can get help running Windows containers on Windows nodes in Kubernetes through:
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 6.9 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 3.5 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 3.4 MiB |
+163
-116
@@ -3,26 +3,36 @@ reviewers:
|
||||
- michmike
|
||||
- patricklang
|
||||
title: Guide for adding Windows Nodes in Kubernetes
|
||||
content_template: templates/concept
|
||||
min-kubernetes-server-version: v1.14
|
||||
content_template: templates/tutorial
|
||||
weight: 70
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
The Kubernetes platform can now be used to run both Linux and Windows containers. One or more Windows nodes can be registered to a cluster. This guide shows how to:
|
||||
|
||||
* Register a Windows node to the cluster
|
||||
* Configure networking so pods on Linux and Windows can communicate
|
||||
The Kubernetes platform can now be used to run both Linux and Windows containers. This page shows how one or more Windows nodes can be registered to a cluster.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture body %}}
|
||||
|
||||
## Before you begin
|
||||
{{% capture prerequisites %}}
|
||||
|
||||
* Obtain a [Windows Server license](https://www.microsoft.com/en-us/cloud-platform/windows-server-pricing) in order to configure the Windows node that hosts Windows containers. You can use your organization's licenses for the cluster, or acquire one from Microsoft, a reseller, or via the major cloud providers such as GCP, AWS, and Azure by provisioning a virtual machine running Windows Server through their marketplaces. A [time-limited trial](https://www.microsoft.com/en-us/cloud-platform/windows-server-trial) is also available.
|
||||
* Obtain a [Windows Server 2019 license](https://www.microsoft.com/en-us/cloud-platform/windows-server-pricing) (or higher) in order to configure the Windows node that hosts Windows containers. You can use your organization's licenses for the cluster, or acquire one from Microsoft, a reseller, or via the major cloud providers such as GCP, AWS, and Azure by provisioning a virtual machine running Windows Server through their marketplaces. A [time-limited trial](https://www.microsoft.com/en-us/cloud-platform/windows-server-trial) is also available.
|
||||
|
||||
* Build a Linux-based Kubernetes cluster in which you have access to the control plane (some examples include [Creating a single control-plane cluster with kubeadm](/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/), [AKS Engine](/docs/setup/production-environment/turnkey/azure/), [GCE](/docs/setup/production-environment/turnkey/gce/), [AWS](/docs/setup/production-environment/turnkey/aws/).
|
||||
* Build a Linux-based Kubernetes cluster in which you have access to the control-plane (some examples include [Creating a single control-plane cluster with kubeadm](/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/), [AKS Engine](/docs/setup/production-environment/turnkey/azure/), [GCE](/docs/setup/production-environment/turnkey/gce/), [AWS](/docs/setup/production-environment/turnkey/aws/).
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
|
||||
{{% capture objectives %}}
|
||||
|
||||
* Register a Windows node to the cluster
|
||||
* Configure networking so Pods and Services on Linux and Windows can communicate with each other
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
|
||||
{{% capture lessoncontent %}}
|
||||
|
||||
## Getting Started: Adding a Windows Node to Your Cluster
|
||||
|
||||
@@ -42,7 +52,7 @@ Review the networking options supported in 'Intro to Windows containers in Kuber
|
||||
|
||||
### Components that run on Windows
|
||||
|
||||
While the Kubernetes control plane runs on your Linux node(s), the following components are configured and run on your Windows node(s).
|
||||
While the Kubernetes control-plane runs on your Linux node(s), the following components are configured and run on your Windows node(s).
|
||||
|
||||
1. kubelet
|
||||
2. kube-proxy
|
||||
@@ -53,9 +63,9 @@ Get the latest binaries from [https://github.com/kubernetes/kubernetes/releases]
|
||||
|
||||
### Networking Configuration
|
||||
|
||||
Once you have a Linux-based Kubernetes master node you are ready to choose a networking solution. This guide illustrates using Flannel in VXLAN mode for simplicity.
|
||||
Once you have a Linux-based Kubernetes control-plane ("Master") node you are ready to choose a networking solution. This guide illustrates using Flannel in VXLAN mode for simplicity.
|
||||
|
||||
#### Configuring Flannel in VXLAN mode on the Linux controller
|
||||
#### Configuring Flannel in VXLAN mode on the Linux control-plane
|
||||
|
||||
1. Prepare Kubernetes master for Flannel
|
||||
|
||||
@@ -122,7 +132,7 @@ Once you have a Linux-based Kubernetes master node you are ready to choose a net
|
||||
}
|
||||
```
|
||||
|
||||
1. Apply the Flannel yaml and Validate
|
||||
1. Apply the Flannel manifest and validate
|
||||
|
||||
Let's apply the Flannel configuration:
|
||||
|
||||
@@ -169,120 +179,136 @@ Once you have a Linux-based Kubernetes master node you are ready to choose a net
|
||||
kube-proxy 2 2 2 2 2 beta.kubernetes.io/os=linux 26d
|
||||
```
|
||||
|
||||
#### Join Windows Worker
|
||||
|
||||
In this section we'll cover configuring a Windows node from scratch to join a cluster on-prem. If your cluster is on a cloud you'll likely want to follow the cloud specific guides in the next section.
|
||||
|
||||
### Join Windows Worker Node
|
||||
|
||||
In this section we'll cover configuring a Windows node from scratch to join a cluster on-prem. If your cluster is on a cloud you'll likely want to follow the cloud specific guides in the [public cloud providers section](#public-cloud-providers).
|
||||
|
||||
#### Preparing a Windows Node
|
||||
|
||||
{{< note >}}
|
||||
All code snippets in Windows sections are to be run in a PowerShell environment with elevated permissions (Admin).
|
||||
All code snippets in Windows sections are to be run in a PowerShell environment with elevated permissions (Administrator) on the Windows worker node.
|
||||
{{< /note >}}
|
||||
|
||||
1. Install Docker (requires a system reboot)
|
||||
1. Download the [SIG Windows tools](https://github.com/kubernetes-sigs/sig-windows-tools) repository containing install and join scripts
|
||||
```PowerShell
|
||||
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
|
||||
Start-BitsTransfer https://github.com/kubernetes-sigs/sig-windows-tools/archive/master.zip
|
||||
tar -xvf .\master.zip --strip-components 3 sig-windows-tools-master/kubeadm/v1.15.0/*
|
||||
Remove-Item .\master.zip
|
||||
```
|
||||
|
||||
Kubernetes uses [Docker](https://www.docker.com/) as its container engine, so we need to install it. You can follow the [official Docs instructions](https://docs.microsoft.com/en-us/virtualization/windowscontainers/manage-docker/configure-docker-daemon#install-docker), the [Docker instructions](https://store.docker.com/editions/enterprise/docker-ee-server-windows), or try the following *recommended* steps:
|
||||
|
||||
```PowerShell
|
||||
Enable-WindowsOptionalFeature -FeatureName Containers
|
||||
Restart-Computer -Force
|
||||
Install-Module -Name DockerMsftProvider -Repository PSGallery -Force
|
||||
Install-Package -Name Docker -ProviderName DockerMsftProvider
|
||||
```
|
||||
|
||||
If you are behind a proxy, the following PowerShell environment variables must be defined:
|
||||
|
||||
```PowerShell
|
||||
[Environment]::SetEnvironmentVariable("HTTP_PROXY", "http://proxy.example.com:80/", [EnvironmentVariableTarget]::Machine)
|
||||
[Environment]::SetEnvironmentVariable("HTTPS_PROXY", "http://proxy.example.com:443/", [EnvironmentVariableTarget]::Machine)
|
||||
```
|
||||
|
||||
After reboot, you can verify that the docker service is ready with the command below.
|
||||
|
||||
```PowerShell
|
||||
docker version
|
||||
```
|
||||
|
||||
If you see error message like the following, you need to start the docker service manually.
|
||||
1. Customize the Kubernetes [configuration file](https://github.com/kubernetes-sigs/sig-windows-tools/blob/master/kubeadm/v1.15.0/Kubeclustervxlan.json)
|
||||
|
||||
```
|
||||
Client:
|
||||
Version: 17.06.2-ee-11
|
||||
API version: 1.30
|
||||
Go version: go1.8.7
|
||||
Git commit: 06fc007
|
||||
Built: Thu May 17 06:14:39 2018
|
||||
OS/Arch: windows / amd64
|
||||
error during connect: Get http://%2F%2F.%2Fpipe%2Fdocker_engine/v1.30/version: open //./pipe/docker_engine: The system c
|
||||
annot find the file specified. In the default daemon configuration on Windows, the docker client must be run elevated to
|
||||
connect. This error may also indicate that the docker daemon is not running.
|
||||
```
|
||||
|
||||
You can start the docker service manually like below.
|
||||
|
||||
```PowerShell
|
||||
Start-Service docker
|
||||
```
|
||||
|
||||
{{< note >}}
|
||||
The "pause" (infrastructure) image is hosted on Microsoft Container Registry (MCR). You can access it using "docker pull mcr.microsoft.com/k8s/core/pause:1.2.0". The DOCKERFILE is available at https://github.com/kubernetes-sigs/sig-windows-tools/tree/master/cmd/wincat.
|
||||
{{< /note >}}
|
||||
|
||||
1. Prepare a Windows directory for Kubernetes
|
||||
|
||||
Create a "Kubernetes for Windows" directory to store Kubernetes binaries as well as any deployment scripts and config files.
|
||||
|
||||
```PowerShell
|
||||
mkdir c:\k
|
||||
```
|
||||
|
||||
1. Copy Kubernetes certificate
|
||||
|
||||
Copy the Kubernetes certificate file `$HOME/.kube/config` [from the Linux controller](https://docs.microsoft.com/en-us/virtualization/windowscontainers/kubernetes/creating-a-linux-master#collect-cluster-information) to this new `C:\k` directory on your Windows node.
|
||||
|
||||
Tip: You can use tools such as [xcopy](https://docs.microsoft.com/en-us/windows-server/administration/windows-commands/xcopy), [WinSCP](https://winscp.net/eng/download.php), or this [PowerShell wrapper for WinSCP](https://www.powershellgallery.com/packages/WinSCP/5.13.2.0) to transfer the config file between nodes.
|
||||
|
||||
1. Download Kubernetes binaries
|
||||
|
||||
To be able to run Kubernetes, you first need to download the `kubelet` and `kube-proxy` binaries. You download these from the Node Binaries links in the CHANGELOG.md file of the [latest releases](https://github.com/kubernetes/kubernetes/releases/). For example 'kubernetes-node-windows-amd64.tar.gz'. You may also optionally download `kubectl` to run on Windows which you can find under Client Binaries.
|
||||
|
||||
Use the [Expand-Archive](https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.archive/expand-archive?view=powershell-6) PowerShell command to extract the archive and place the binaries into `C:\k`.
|
||||
|
||||
#### Join the Windows node to the Flannel cluster
|
||||
|
||||
The Flannel overlay deployment scripts and documentation are available in [this repository](https://github.com/Microsoft/SDN/tree/master/Kubernetes/flannel/overlay). The following steps are a simple walkthrough of the more comprehensive instructions available there.
|
||||
|
||||
Download the [Flannel start.ps1](https://github.com/Microsoft/SDN/blob/master/Kubernetes/flannel/start.ps1) script, the contents of which should be extracted to `C:\k`:
|
||||
|
||||
```PowerShell
|
||||
cd c:\k
|
||||
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
|
||||
wget https://raw.githubusercontent.com/Microsoft/SDN/master/Kubernetes/flannel/start.ps1 -o c:\k\start.ps1
|
||||
```
|
||||
{
|
||||
"Cri" : { // Contains values for container runtime and base container setup
|
||||
"Name" : "dockerd", // Container runtime name
|
||||
"Images" : {
|
||||
"Pause" : "mcr.microsoft.com/k8s/core/pause:1.2.0", // Infrastructure container image
|
||||
"Nanoserver" : "mcr.microsoft.com/windows/nanoserver:1809", // Base Nanoserver container image
|
||||
"ServerCore" : "mcr.microsoft.com/windows/servercore:ltsc2019" // Base ServerCore container image
|
||||
}
|
||||
},
|
||||
"Cni" : { // Contains values for networking executables
|
||||
"Name" : "flannel", // Name of network fabric
|
||||
"Source" : [{ // Contains array of objects containing values for network daemon(s)
|
||||
"Name" : "flanneld", // Name of network daemon
|
||||
"Url" : "https://github.com/coreos/flannel/releases/download/v0.11.0/flanneld.exe" // Direct URL pointing to network daemon executable
|
||||
}
|
||||
],
|
||||
"Plugin" : { // Contains values for CNI network plugin
|
||||
"Name": "vxlan" // Backend network mechanism to use: ["vxlan" | "bridge"]
|
||||
},
|
||||
"InterfaceName" : "Ethernet" // Designated network interface name on Windows node to use as container network
|
||||
},
|
||||
"Kubernetes" : { // Contains values for Kubernetes node binaries
|
||||
"Source" : { // Contains values for Kubernetes node binaries
|
||||
"Release" : "1.15.0", // Version of Kubernetes node binaries
|
||||
"Url" : "https://dl.k8s.io/v1.15.0/kubernetes-node-windows-amd64.tar.gz" // Direct URL pointing to Kubernetes node binaries tarball
|
||||
},
|
||||
"ControlPlane" : { // Contains values associated with Kubernetes control-plane ("Master") node
|
||||
"IpAddress" : "kubemasterIP", // IP address of control-plane ("Master") node
|
||||
"Username" : "localadmin", // Username on control-plane ("Master") node with remote SSH access
|
||||
"KubeadmToken" : "token", // Kubeadm bootstrap token
|
||||
"KubeadmCAHash" : "discovery-token-ca-cert-hash" // Kubeadm CA key hash
|
||||
},
|
||||
"KubeProxy" : { // Contains values for Kubernetes network proxy configuration
|
||||
"Gates" : "WinOverlay=true" // Comma-separated key-value pairs passed to kube-proxy feature gate flag
|
||||
},
|
||||
"Network" : { // Contains values for IP ranges in CIDR notation for Kubernetes networking
|
||||
"ServiceCidr" : "10.96.0.0/12", // Service IP subnet used by Services in CIDR notation
|
||||
"ClusterCidr" : "10.244.0.0/16" // Cluster IP subnet used by Pods in CIDR notation
|
||||
}
|
||||
},
|
||||
"Install" : { // Contains values and configurations for Windows node installation
|
||||
"Destination" : "C:\\ProgramData\\Kubernetes" // Absolute DOS path where Kubernetes will be installed on the Windows node
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
{{< note >}}
|
||||
[start.ps1](https://github.com/Microsoft/SDN/blob/master/Kubernetes/flannel/start.ps1) references [install.ps1](https://github.com/Microsoft/SDN/blob/master/Kubernetes/windows/install.ps1), which downloads additional files such as the `flanneld` executable and the [Dockerfile for infrastructure pod](https://github.com/Microsoft/SDN/blob/master/Kubernetes/windows/Dockerfile) and install those for you. For overlay networking mode, the [firewall](https://github.com/Microsoft/SDN/blob/master/Kubernetes/windows/helper.psm1#L111) is opened for local UDP port 4789. There may be multiple powershell windows being opened/closed as well as a few seconds of network outage while the new external vSwitch for the pod network is being created the first time. Run the script using the arguments as specified below:
|
||||
Users can generate values for the `ControlPlane.KubeadmToken` and `ControlPlane.KubeadmCAHash` fields by running `kubeadm token create --print-join-command` on the Kubernetes control-plane ("Master") node.
|
||||
{{< /note >}}
|
||||
|
||||
```PowerShell
|
||||
cd c:\k
|
||||
.\start.ps1 -ManagementIP <Windows Node IP> `
|
||||
-NetworkMode overlay `
|
||||
-ClusterCIDR <Cluster CIDR> `
|
||||
-ServiceCIDR <Service CIDR> `
|
||||
-KubeDnsServiceIP <Kube-dns Service IP> `
|
||||
-LogDir <Log directory>
|
||||
```
|
||||
1. Install containers and Kubernetes (requires a system reboot)
|
||||
|
||||
| Parameter | Default Value | Notes |
|
||||
| --- | --- | --- |
|
||||
| -ManagementIP | N/A (required) | The IP address assigned to the Windows node. You can use `ipconfig` to find this. |
|
||||
| -NetworkMode | l2bridge | We're using `overlay` here |
|
||||
| -ClusterCIDR | 10.244.0.0/16 | Refer to your cluster IP plan |
|
||||
| -ServiceCIDR | 10.96.0.0/12 | Refer to your cluster IP plan |
|
||||
| -KubeDnsServiceIP | 10.96.0.10 | |
|
||||
| -InterfaceName | Ethernet | The name of the network interface of the Windows host. You can use <code>ipconfig</code> to find this. |
|
||||
| -LogDir | C:\k | The directory where kubelet and kube-proxy logs are redirected into their respective output files. |
|
||||
Use the previously downloaded [KubeCluster.ps1](https://github.com/kubernetes-sigs/sig-windows-tools/blob/master/kubeadm/v1.15.0/KubeCluster.ps1) script to install Kubernetes on the Windows Server container host:
|
||||
|
||||
```PowerShell
|
||||
.\KubeCluster.ps1 -ConfigFile .\Kubeclustervxlan.json -install
|
||||
```
|
||||
where `-ConfigFile` points to the path of the Kubernetes configuration file.
|
||||
|
||||
{{< note >}}
|
||||
In the example below, we are using overlay networking mode. This requires Windows Server version 2019 with [KB4489899](https://support.microsoft.com/help/4489899) and at least Kubernetes v1.14 or above. Users that cannot meet this requirement must use `L2bridge` networking instead by selecting `bridge` as the [plugin](https://github.com/kubernetes-sigs/sig-windows-tools/blob/master/kubeadm/v1.15.0/Kubeclusterbridge.json#L18) in the configuration file.
|
||||
{{< /note >}}
|
||||
|
||||
|
||||
|
||||
|
||||
On the Windows node you target, this step will:
|
||||
|
||||
1. Enable Windows Server containers role (and reboot)
|
||||
1. Download and install the chosen container runtime
|
||||
1. Download all needed container images
|
||||
1. Download Kubernetes binaries and add them to the `$PATH` environment variable
|
||||
1. Download CNI plugins based on the selection made in the Kubernetes Configuration file
|
||||
1. (Optionally) Generate a new SSH key which is required to connect to the control-plane ("Master") node during joining
|
||||
|
||||
{{< note >}}For the SSH key generation step, you also need to add the generated public SSH key to the `authorized_keys` file on your (Linux) control-plane node. You only need to do this once. The script prints out the steps you can follow to do this, at the end of its output.{{< /note >}}
|
||||
|
||||
Once installation is complete, any of the generated configuration files or binaries can be modified before joining the Windows node.
|
||||
|
||||
#### Join the Windows Node to the Kubernetes cluster
|
||||
This section covers how to join a [Windows node with Kubernetes installed](#preparing-a-windows-node) with an existing (Linux) control-plane, to form a cluster.
|
||||
|
||||
Use the previously downloaded [KubeCluster.ps1](https://github.com/kubernetes-sigs/sig-windows-tools/blob/master/kubeadm/v1.15.0/KubeCluster.ps1) script to join the Windows node to the cluster:
|
||||
|
||||
```PowerShell
|
||||
.\KubeCluster.ps1 -ConfigFile .\Kubeclustervxlan.json -join
|
||||
```
|
||||
where `-ConfigFile` points to the path of the Kubernetes configuration file.
|
||||
|
||||
|
||||
|
||||
{{< note >}}
|
||||
Should the script fail during the bootstrap or joining procedure for whatever reason, start a new PowerShell session before starting each consecutive join attempt.
|
||||
{{< /note >}}
|
||||
|
||||
This step will perform the following actions:
|
||||
|
||||
1. Connect to the control-plane ("Master") node via SSH, to retrieve the [Kubeconfig file](/docs/concepts/configuration/organize-cluster-access-kubeconfig/) file.
|
||||
1. Register kubelet as a Windows service
|
||||
1. Configure CNI network plugins
|
||||
1. Create an HNS network on top of the chosen network interface
|
||||
{{< note >}}
|
||||
This may cause a network blip for a few seconds while the vSwitch is being created.
|
||||
{{< /note >}}
|
||||
1. (If vxlan plugin is selected) Open up inbound firewall UDP port 4789 for overlay traffic
|
||||
1. Register flanneld as a Windows service
|
||||
1. Register kube-proxy as a Windows service
|
||||
|
||||
Now you can view the Windows nodes in your cluster by running the following:
|
||||
|
||||
@@ -290,9 +316,28 @@ Now you can view the Windows nodes in your cluster by running the following:
|
||||
kubectl get nodes
|
||||
```
|
||||
|
||||
{{< note >}}
|
||||
You may want to configure your Windows node components like kubelet and kube-proxy to run as services. View the services and background processes section under [troubleshooting](#troubleshooting) for additional instructions. Once you are running the node components as services, collecting logs becomes an important part of troubleshooting. View the [gathering logs](https://github.com/kubernetes/community/blob/master/sig-windows/CONTRIBUTING.md#gathering-logs) section of the contributing guide for further instructions.
|
||||
{{< /note >}}
|
||||
#### Remove the Windows Node from the Kubernetes cluster
|
||||
In this section we'll cover how to remove a Windows node from a Kubernetes cluster.
|
||||
|
||||
Use the previously downloaded [KubeCluster.ps1](https://github.com/kubernetes-sigs/sig-windows-tools/blob/master/kubeadm/v1.15.0/KubeCluster.ps1) script to remove the Windows node from the cluster:
|
||||
|
||||
```PowerShell
|
||||
.\KubeCluster.ps1 -ConfigFile .\Kubeclustervxlan.json -reset
|
||||
```
|
||||
where `-ConfigFile` points to the path of the Kubernetes configuration file.
|
||||
|
||||
|
||||
|
||||
This step will perform the following actions on the targeted Windows node:
|
||||
|
||||
1. Delete the Windows node from the Kubernetes cluster
|
||||
1. Stop all running containers
|
||||
1. Remove all container networking (HNS) resources
|
||||
1. Unregister all Kubernetes services (flanneld, kubelet, kube-proxy)
|
||||
1. Delete all Kubernetes binaries (kube-proxy.exe, kubelet.exe, flanneld.exe, kubeadm.exe)
|
||||
1. Delete all CNI network plugins binaries
|
||||
1. Delete [Kubeconfig file](https://kubernetes.io/docs/concepts/configuration/organize-cluster-access-kubeconfig/) used to access the Kubernetes cluster
|
||||
|
||||
|
||||
### Public Cloud Providers
|
||||
|
||||
@@ -306,10 +351,12 @@ Users can easily deploy a complete Kubernetes cluster on GCE following this step
|
||||
|
||||
#### Deployment with kubeadm and cluster API
|
||||
|
||||
Kubeadm is becoming the de facto standard for users to deploy a Kubernetes cluster. Windows node support in kubeadm will come in a future release. We are also making investments in cluster API to ensure Windows nodes are properly provisioned.
|
||||
Kubeadm is becoming the de facto standard for users to deploy a Kubernetes cluster. Windows node support in kubeadm is an alpha feature since Kubernetes release v1.16. We are also making investments in cluster API to ensure Windows nodes are properly provisioned. For more details, please consult the [kubeadm for Windows KEP](https://github.com/kubernetes/enhancements/blob/master/keps/sig-cluster-lifecycle/kubeadm/20190424-kubeadm-for-windows.md).
|
||||
|
||||
|
||||
### Next Steps
|
||||
|
||||
Now that you've configured a Windows worker in your cluster to run Windows containers you may want to add one or more Linux nodes as well to run Linux containers. You are now ready to schedule Windows containers on your cluster.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user