From 7181cb54cd616299cc83597d6cd0be311ddac1f2 Mon Sep 17 00:00:00 2001 From: marosset Date: Fri, 20 Nov 2020 23:02:30 +0000 Subject: [PATCH] Misc updates to intro-windows-in-kubernetes.md --- .../windows/intro-windows-in-kubernetes.md | 59 +++++-------------- 1 file changed, 15 insertions(+), 44 deletions(-) diff --git a/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md b/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md index b0f8e02e3e..ec06e070bd 100644 --- a/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md +++ b/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md @@ -132,6 +132,7 @@ There is a [known limitation](/docs/tasks/configure-pod-container/configure-gmsa Kubernetes [volumes](/docs/concepts/storage/volumes/) enable complex applications, with data persistence and Pod volume sharing requirements, to be deployed on Kubernetes. Management of persistent volumes associated with a specific storage back-end or protocol includes actions such as: provisioning/de-provisioning/resizing of volumes, attaching/detaching a volume to/from a Kubernetes node and mounting/dismounting a volume to/from individual containers in a pod that needs to persist data. The code implementing these volume management actions for a specific storage back-end or protocol is shipped in the form of a Kubernetes volume [plugin](/docs/concepts/storage/volumes/#types-of-volumes). The following broad classes of Kubernetes volume plugins are supported on Windows: ##### In-tree Volume Plugins + Code associated with in-tree volume plugins ship as part of the core Kubernetes code base. Deployment of in-tree volume plugins do not require installation of additional scripts or deployment of separate containerized plugin components. These plugins can handle: provisioning/de-provisioning and resizing of volumes in the storage backend, attaching/detaching of volumes to/from a Kubernetes node and mounting/dismounting a volume to/from individual containers in a pod. The following in-tree plugins support Windows nodes: * [awsElasticBlockStore](/docs/concepts/storage/volumes/#awselasticblockstore) @@ -141,6 +142,7 @@ Code associated with in-tree volume plugins ship as part of the core Kubernetes * [vsphereVolume](/docs/concepts/storage/volumes/#vspherevolume) ##### FlexVolume Plugins + Code associated with [FlexVolume](/docs/concepts/storage/volumes/#flexVolume) plugins ship as out-of-tree scripts or binaries that need to be deployed directly on the host. FlexVolume plugins handle attaching/detaching of volumes to/from a Kubernetes node and mounting/dismounting a volume to/from individual containers in a pod. Provisioning/De-provisioning of persistent volumes associated with FlexVolume plugins may be handled through an external provisioner that is typically separate from the FlexVolume plugins. The following FlexVolume [plugins](https://github.com/Microsoft/K8s-Storage-Plugins/tree/master/flexvolume/windows), deployed as powershell scripts on the host, support Windows nodes: * [SMB](https://github.com/microsoft/K8s-Storage-Plugins/tree/master/flexvolume/windows/plugins/microsoft.com~smb.cmd) @@ -148,7 +150,7 @@ Code associated with [FlexVolume](/docs/concepts/storage/volumes/#flexVolume) pl ##### CSI Plugins -{{< feature-state for_k8s_version="v1.16" state="alpha" >}} +{{< feature-state for_k8s_version="v1.19" state="beta" >}} Code associated with {{< glossary_tooltip text="CSI" term_id="csi" >}} plugins ship as out-of-tree scripts and binaries that are typically distributed as container images and deployed using standard Kubernetes constructs like DaemonSets and StatefulSets. CSI plugins handle a wide range of volume management actions in Kubernetes: provisioning/de-provisioning/resizing of volumes, attaching/detaching of volumes to/from a Kubernetes node and mounting/dismounting a volume to/from individual containers in a pod, backup/restore of persistent data using snapshots and cloning. CSI plugins typically consist of node plugins (that run on each node as a DaemonSet) and controller plugins. @@ -171,6 +173,7 @@ The following service spec types are supported: * ExternalName ##### Network modes + Windows supports five different networking drivers/modes: L2bridge, L2tunnel, Overlay, Transparent, and NAT. In a heterogeneous cluster with Windows and Linux worker nodes, you need to select a networking solution that is compatible on both Windows and Linux. The following out-of-tree plugins are supported on Windows, with recommendations on when to use each CNI: | Network Driver | Description | Container Packet Modifications | Network Plugins | Network Plugin Characteristics | @@ -196,6 +199,7 @@ For the node, pod, and service objects, the following network flows are supporte * Pod -> Node ##### IP address management (IPAM) {#ipam} + The following IPAM options are supported on Windows: * [Host-local](https://github.com/containernetworking/plugins/tree/master/plugins/ipam/host-local) @@ -216,10 +220,11 @@ On Windows, you can use the following settings to configure Services and load ba {{< /table >}} #### IPv4/IPv6 dual-stack + You can enable IPv4/IPv6 dual-stack networking for `l2bridge` networks using the `IPv6DualStack` [feature gate](https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/). See [enable IPv4/IPv6 dual stack](/docs/concepts/services-networking/dual-stack#enable-ipv4ipv6-dual-stack) for more details. {{< note >}} -On Windows, using IPv6 with Kubernetes require Windows Server vNext Insider Preview Build 19603 (or higher). +On Windows, using IPv6 with Kubernetes require Windows Server, version 2004 (kernel version 10.0.19041.610) or later. {{< /note >}} {{< note >}} @@ -232,7 +237,7 @@ Overlay (VXLAN) networks on Windows do not support dual-stack networking today. Windows is only supported as a worker node in the Kubernetes architecture and component matrix. This means that a Kubernetes cluster must always include Linux master nodes, zero or more Linux worker nodes, and zero or more Windows worker nodes. -#### Compute +#### Compute {compute-limitations} ##### Resource management and process isolation @@ -274,7 +279,7 @@ The behavior of the flags behave differently as described below: Windows has a layered filesystem driver to mount container layers and create a copy filesystem based on NTFS. All file paths in the container are resolved only within the context of that container. -* Volume mounts can only target a directory in the container, and not an individual file +* With Docker Volume mounts can only target a directory in the container, and not an individual file. This limitation does not exist with CRI-containerD. * Volume mounts cannot project files or directories back to the host filesystem * Read-only filesystems are not supported because write access is always required for the Windows registry and SAM database. However, read-only volumes are supported * Volume user-masks and permissions are not available. Because the SAM is not shared between the host & container, there's no mapping between them. All permissions are resolved within the context of the container @@ -292,7 +297,7 @@ As a result, the following storage functionality is not supported on Windows nod * NFS based storage/volume support * Expanding the mounted volume (resizefs) -#### Networking +#### Networking {networking-limitations} Windows Container Networking differs in some important ways from Linux networking. The [Microsoft documentation for Windows Container Networking](https://docs.microsoft.com/en-us/virtualization/windowscontainers/container-networking/architecture) contains additional details and background. @@ -331,10 +336,11 @@ These features were added in Kubernetes v1.15: * On Windows, there are multiple DNS resolvers that can be used. As these come with slightly different behaviors, using the `Resolve-DNSName` utility for name query resolutions is recommended. ##### IPv6 + Kubernetes on Windows does not support single-stack "IPv6-only" networking. However,dual-stack IPv4/IPv6 networking for pods and nodes with single-family services is supported. See [IPv4/IPv6 dual-stack networking](#ipv4ipv6-dual-stack) for more details. - ##### Session affinity + Setting the maximum session sticky time for Windows services using `service.spec.sessionAffinityConfig.clientIP.timeoutSeconds` is not supported. ##### Security @@ -344,7 +350,7 @@ Secrets are written in clear text on the node's volume (as compared to tmpfs/in- 1. Use file ACLs to secure the secrets file location 2. Use volume-level encryption using [BitLocker](https://docs.microsoft.com/en-us/windows/security/information-protection/bitlocker/bitlocker-how-to-deploy-on-windows-server) -[RunAsUser ](/docs/concepts/policy/pod-security-policy/#users-and-groups)is not currently supported on Windows. The workaround is to create local accounts before packaging the container. The RunAsUsername capability may be added in a future release. +[RunAsUsername](/docs/tasks/configure-pod-container/configure-runasusername) can be specified for Windows Pod's or Container's to execute the Container processes as a node-default user. This is roughly equivalent to [RunAsUser](/docs/concepts/policy/pod-security-policy/#users-and-groups). Linux specific pod security context privileges such as SELinux, AppArmor, Seccomp, Capabilities (POSIX Capabilities), and others are not supported. @@ -489,7 +495,6 @@ Your main source of help for troubleshooting your Kubernetes cluster should star nssm start kube-proxy ``` - For initial troubleshooting, you can use the following flags in [nssm.exe](https://nssm.cc/) to redirect stdout and stderr to a output file: ```powershell @@ -551,7 +556,7 @@ Your main source of help for troubleshooting your Kubernetes cluster should star 1. My Windows Pods cannot launch because of missing `/run/flannel/subnet.env` - This indicates that Flannel didn't launch correctly. You can either try to restart flanneld.exe or you can copy the files over manually from `/run/flannel/subnet.env` on the Kubernetes master to` C:\run\flannel\subnet.env` on the Windows worker node and modify the `FLANNEL_SUBNET` row to a different number. For example, if node subnet 10.244.4.1/24 is desired: + This indicates that Flannel didn't launch correctly. You can either try to restart flanneld.exe or you can copy the files over manually from `/run/flannel/subnet.env` on the Kubernetes master to `C:\run\flannel\subnet.env` on the Windows worker node and modify the `FLANNEL_SUBNET` row to a different number. For example, if node subnet 10.244.4.1/24 is desired: ```env FLANNEL_NETWORK=10.244.0.0/16 @@ -623,11 +628,8 @@ If filing a bug, please include detailed information about how to reproduce the * [Relevant logs](https://github.com/kubernetes/community/blob/master/sig-windows/CONTRIBUTING.md#gathering-logs) * Tag the issue sig/windows by commenting on the issue with `/sig windows` to bring it to a SIG-Windows member's attention - - ## {{% heading "whatsnext" %}} - We have a lot of features in our roadmap. An abbreviated high level list is included below, but we encourage you to view our [roadmap project](https://github.com/orgs/kubernetes/projects/8) and help us make Windows support better by [contributing](https://github.com/kubernetes/community/blob/master/sig-windows/). ### Hyper-V isolation @@ -639,31 +641,7 @@ Hyper-V isolation is requried to enable the following use cases for Windows cont * Specific CPU/NUMA settings for a pod * Memory isolation and reservations -The existing Hyper-V isolation support, an experimental feature as of v1.10, will be deprecated in the future in favor of the CRI-ContainerD and RuntimeClass features mentioned above. To use the current features and create a Hyper-V isolated container, the kubelet should be started with feature gates `HyperVContainer=true` and the Pod should include the annotation `experimental.windows.kubernetes.io/isolation-type=hyperv`. In the experiemental release, this feature is limited to 1 container per Pod. - -```yaml -apiVersion: apps/v1 -kind: Deployment -metadata: - name: iis -spec: - selector: - matchLabels: - app: iis - replicas: 3 - template: - metadata: - labels: - app: iis - annotations: - experimental.windows.kubernetes.io/isolation-type: hyperv - spec: - containers: - - name: iis - image: microsoft/iis - ports: - - containerPort: 80 -``` +Hyper-V isolation support will be added in a later release and will require CRI-Containerd. ### Deployment with kubeadm and cluster API @@ -672,10 +650,3 @@ cluster. Windows node support in kubeadm is currently a work-in-progress but a guide is available [here](/docs/tasks/administer-cluster/kubeadm/adding-windows-nodes/). We are also making investments in cluster API to ensure Windows nodes are properly provisioned. - -### A few other key features -* Beta support for Group Managed Service Accounts -* More CNIs -* More Storage Plugins - -