From e01349dc91777cb885d6bb6d42e3df3061f3fd23 Mon Sep 17 00:00:00 2001 From: David Schott Date: Fri, 5 Jun 2020 17:23:24 -0700 Subject: [PATCH 1/9] updating Windows networking section with new features --- .../windows/intro-windows-in-kubernetes.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md b/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md index 78e61d4588..3ce3914b6c 100644 --- a/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md +++ b/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md @@ -164,6 +164,7 @@ The following service spec types are supported: * LoadBalancer * ExternalName +##### Network modes Windows supports five different networking drivers/modes: L2bridge, L2tunnel, Overlay, Transparent, and NAT. In a heterogeneous cluster with Windows and Linux worker nodes, you need to select a networking solution that is compatible on both Windows and Linux. The following out-of-tree plugins are supported on Windows, with recommendations on when to use each CNI: | Network Driver | Description | Container Packet Modifications | Network Plugins | Network Plugin Characteristics | @@ -188,12 +189,22 @@ For the node, pod, and service objects, the following network flows are supporte * Node -> Pod * Pod -> Node +##### IPAM The following IPAM options are supported on Windows: * [Host-local](https://github.com/containernetworking/plugins/tree/master/plugins/ipam/host-local) * HNS IPAM (Inbox platform IPAM, this is a fallback when no IPAM is set) * [Azure-vnet-ipam](https://github.com/Azure/azure-container-networking/blob/master/docs/ipam.md) (for azure-cni only) +##### Services & Load Balancing +The following settings applicable to services and load balancing are available on Windows: +| Feature | Description | Supported Kubernetes version | Supported Windows OS build | How to enable | +| ------- | ----------- | ----------------------------- | -------------------------- | ------------- | +| Session affinity | Ensures that connections from a particular client are passed to the same Pod each time. | v1.19+ | [Windows Server vNext Insider Preview Build 19551](https://blogs.windows.com/windowsexperience/2020/01/28/announcing-windows-server-vnext-insider-preview-build-19551/) (or higher) | Set `service.spec.sessionAffinity` to "ClientIP" | +| DSR | Load balancing mode where the IP fixups and the LBNAT occurs at the container vSwitch port directly; service traffic arrives with the source IP set as the originating pod IP. Promises lower latency and scalability. | v1.15+ | Windows Server, version 2004 | Set the following flags in kube-proxy: `feature-gates="WinDSR=true" --enable-dsr=true` | +| Preserve-DIP | Skips DNAT of service traffic, thereby preserving the virtual IP of the target service in packets reaching the backend Pod. This setting will also ensure that the client IP of incoming packets get preserved. | v1.15+ | Windows Server, version 1903 (or higher) | Set `"preserve-destination": "true"` in service annotations and enable DSR. | + + ### Limitations #### Control Plane From b32a248618e483d1413ba017c210a52206bdbac2 Mon Sep 17 00:00:00 2001 From: David Schott Date: Tue, 16 Jun 2020 13:18:25 -0700 Subject: [PATCH 2/9] added IPv6 and minor networking fix-ups to Windows --- .../windows/intro-windows-in-kubernetes.md | 28 +++++++++++++++++-- 1 file changed, 25 insertions(+), 3 deletions(-) diff --git a/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md b/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md index 3ce3914b6c..0a5dca9ad6 100644 --- a/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md +++ b/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md @@ -169,7 +169,7 @@ Windows supports five different networking drivers/modes: L2bridge, L2tunnel, Ov | Network Driver | Description | Container Packet Modifications | Network Plugins | Network Plugin Characteristics | | -------------- | ----------- | ------------------------------ | --------------- | ------------------------------ | -| L2bridge | Containers are attached to an external vSwitch. Containers are attached to the underlay network, although the physical network doesn't need to learn the container MACs because they are rewritten on ingress/egress. Inter-container traffic is bridged inside the container host. | MAC is rewritten to host MAC, IP remains the same. | [win-bridge](https://github.com/containernetworking/plugins/tree/master/plugins/main/windows/win-bridge), [Azure-CNI](https://github.com/Azure/azure-container-networking/blob/master/docs/cni.md), Flannel host-gateway uses win-bridge | win-bridge uses L2bridge network mode, connects containers to the underlay of hosts, offering best performance. Requires user-defined routes (UDR) for inter-node connectivity. | +| L2bridge | Containers are attached to an external vSwitch. Containers are attached to the underlay network, although the physical network doesn't need to learn the container MACs because they are rewritten on ingress/egress. | MAC is rewritten to host MAC, IP may be rewritten to host IP using HNS OutboundNAT policy. | [win-bridge](https://github.com/containernetworking/plugins/tree/master/plugins/main/windows/win-bridge), [Azure-CNI](https://github.com/Azure/azure-container-networking/blob/master/docs/cni.md), Flannel host-gateway uses win-bridge | win-bridge uses L2bridge network mode, connects containers to the underlay of hosts, offering best performance. Requires user-defined routes (UDR) for inter-node connectivity. | | L2Tunnel | This is a special case of l2bridge, but only used on Azure. All packets are sent to the virtualization host where SDN policy is applied. | MAC rewritten, IP visible on the underlay network | [Azure-CNI](https://github.com/Azure/azure-container-networking/blob/master/docs/cni.md) | Azure-CNI allows integration of containers with Azure vNET, and allows them to leverage the set of capabilities that [Azure Virtual Network provides](https://azure.microsoft.com/en-us/services/virtual-network/). For example, securely connect to Azure services or use Azure NSGs. See [azure-cni for some examples](https://docs.microsoft.com/en-us/azure/aks/concepts-network#azure-cni-advanced-networking) | | Overlay (Overlay networking for Windows in Kubernetes is in *alpha* stage) | Containers are given a vNIC connected to an external vSwitch. Each overlay network gets its own IP subnet, defined by a custom IP prefix.The overlay network driver uses VXLAN encapsulation. | Encapsulated with an outer header. | [Win-overlay](https://github.com/containernetworking/plugins/tree/master/plugins/main/windows/win-overlay), Flannel VXLAN (uses win-overlay) | win-overlay should be used when virtual container networks are desired to be isolated from underlay of hosts (e.g. for security reasons). Allows for IPs to be re-used for different overlay networks (which have different VNID tags) if you are restricted on IPs in your datacenter. This option requires [KB4489899](https://support.microsoft.com/help/4489899) on Windows Server 2019. | | Transparent (special use case for [ovn-kubernetes](https://github.com/openvswitch/ovn-kubernetes)) | Requires an external vSwitch. Containers are attached to an external vSwitch which enables intra-pod communication via logical networks (logical switches and routers). | Packet is encapsulated either via [GENEVE](https://datatracker.ietf.org/doc/draft-gross-geneve/) or [STT](https://datatracker.ietf.org/doc/draft-davie-stt/) tunneling to reach pods which are not on the same host.
Packets are forwarded or dropped via the tunnel metadata information supplied by the ovn network controller.
NAT is done for north-south communication. | [ovn-kubernetes](https://github.com/openvswitch/ovn-kubernetes) | [Deploy via ansible](https://github.com/openvswitch/ovn-kubernetes/tree/master/contrib). Distributed ACLs can be applied via Kubernetes policies. IPAM support. Load-balancing can be achieved without kube-proxy. NATing is done without using iptables/netsh. | @@ -198,11 +198,26 @@ The following IPAM options are supported on Windows: ##### Services & Load Balancing The following settings applicable to services and load balancing are available on Windows: + | Feature | Description | Supported Kubernetes version | Supported Windows OS build | How to enable | | ------- | ----------- | ----------------------------- | -------------------------- | ------------- | | Session affinity | Ensures that connections from a particular client are passed to the same Pod each time. | v1.19+ | [Windows Server vNext Insider Preview Build 19551](https://blogs.windows.com/windowsexperience/2020/01/28/announcing-windows-server-vnext-insider-preview-build-19551/) (or higher) | Set `service.spec.sessionAffinity` to "ClientIP" | | DSR | Load balancing mode where the IP fixups and the LBNAT occurs at the container vSwitch port directly; service traffic arrives with the source IP set as the originating pod IP. Promises lower latency and scalability. | v1.15+ | Windows Server, version 2004 | Set the following flags in kube-proxy: `feature-gates="WinDSR=true" --enable-dsr=true` | -| Preserve-DIP | Skips DNAT of service traffic, thereby preserving the virtual IP of the target service in packets reaching the backend Pod. This setting will also ensure that the client IP of incoming packets get preserved. | v1.15+ | Windows Server, version 1903 (or higher) | Set `"preserve-destination": "true"` in service annotations and enable DSR. | +| Preserve-DIP | Skips DNAT of service traffic, thereby preserving the virtual IP of the target service in packets reaching the backend Pod. This setting will also ensure that the client IP of incoming packets get preserved. | v1.15+ | Windows Server, version 1903 (or higher) | Set `"preserve-destination": "true"` in service annotations and enable DSR. | +| IPv4/IPv6 dual-stack networking | Native IPv4-to-IPv4 in parallel with IPv6-to-IPv6 communications to, from, and within a cluster | v1.19+ | Windows Server vNext Insider Preview Build 19603 (or higher) | See [IPv4/IPv6 dual-stack](#ipv4ipv6-dual-stack) | + + +#### IPv4/IPv6 dual-stack +IPv4/IPv6 dual-stack networking on Windows can be enabled for `l2bridge` networks using the `IPv6DualStack` feature gate. For more details, see how to [enable IPv4/IPv6 dual stack](../../../../concepts/services-networking/dual-stack#enable-ipv4ipv6-dual-stack). + +{{< note >}} +Requires Windows Server vNext Insider Preview Build 19603 (or higher). + +Overlay (VXLAN) networks on Windows do not support dual-stack networking today. + +This feature will become available on Windows Server, version 2004 through a cumulative update tentatively scheduled to be released in 1.19 - 1.20 release timeframe. + +{{< /note >}} ### Limitations @@ -299,7 +314,7 @@ These features were added in Kubernetes v1.15: * Windows reference network plugins win-bridge and win-overlay do not currently implement [CNI spec](https://github.com/containernetworking/cni/blob/master/SPEC.md) v0.4.0 due to missing "CHECK" implementation. * The Flannel VXLAN CNI has the following limitations on Windows: -1. Node-pod connectivity isn't possible by design. It's only possible for local pods with Flannel [PR 1096](https://github.com/coreos/flannel/pull/1096) +1. Node-pod connectivity isn't possible by design. It's only possible for local pods with Flannel v0.12.0 (or higher). 2. We are restricted to using VNI 4096 and UDP port 4789. The VNI limitation is being worked on and will be overcome in a future release (open-source flannel changes). See the official [Flannel VXLAN](https://github.com/coreos/flannel/blob/master/Documentation/backends.md#vxlan) backend docs for more details on these parameters. ##### DNS {#dns-limitations} @@ -308,6 +323,13 @@ These features were added in Kubernetes v1.15: * On Linux, you have a DNS suffix list, which is used when trying to resolve PQDNs. On Windows, we only have 1 DNS suffix, which is the DNS suffix associated with that pod's namespace (mydns.svc.cluster.local for example). Windows can resolve FQDNs and services or names resolvable with just that suffix. For example, a pod spawned in the default namespace, will have the DNS suffix **default.svc.cluster.local**. On a Windows pod, you can resolve both **kubernetes.default.svc.cluster.local** and **kubernetes**, but not the in-betweens, like **kubernetes.default** or **kubernetes.default.svc**. * On Windows, there are multiple DNS resolvers that can be used. As these come with slightly different behaviors, using the `Resolve-DNSName` utility for name query resolutions is recommended. +##### IPv6 +Windows does not support "IPv6-only" networking. However, Windows does support dual-stack IPv4/IPv6 networking for pods and nodes with single-family services. See [IPv4/IPv6 dual-stack networking](#ipv4ipv6-dual-stack) for more details. + + +##### Session affinity +Setting the maximum session sticky time using `service.spec.sessionAffinityConfig.clientIP.timeoutSeconds` is not supported on Windows as this has not been implemented. + ##### Security Secrets are written in clear text on the node's volume (as compared to tmpfs/in-memory on linux). This means customers have to do two things From 4bc02fce4b3d2db491ce88ec89a76e1f58bd6eb4 Mon Sep 17 00:00:00 2001 From: David Schott <36676176+daschott@users.noreply.github.com> Date: Tue, 30 Jun 2020 11:53:24 -0700 Subject: [PATCH 3/9] Update content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md Co-authored-by: Tim Bannister --- .../windows/intro-windows-in-kubernetes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md b/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md index 0a5dca9ad6..bbb022d328 100644 --- a/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md +++ b/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md @@ -208,7 +208,7 @@ The following settings applicable to services and load balancing are available o #### IPv4/IPv6 dual-stack -IPv4/IPv6 dual-stack networking on Windows can be enabled for `l2bridge` networks using the `IPv6DualStack` feature gate. For more details, see how to [enable IPv4/IPv6 dual stack](../../../../concepts/services-networking/dual-stack#enable-ipv4ipv6-dual-stack). +You can enable IPv4/IPv6 dual-stack networking for `l2bridge` networks using the `IPv6DualStack` [feature gate](https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/). See [enable IPv4/IPv6 dual stack](/docs/concepts/services-networking/dual-stack#enable-ipv4ipv6-dual-stack) for more details. {{< note >}} Requires Windows Server vNext Insider Preview Build 19603 (or higher). From eae3119f89ae2af462a4f421ad7e3a29e6eb8f31 Mon Sep 17 00:00:00 2001 From: David Schott <36676176+daschott@users.noreply.github.com> Date: Tue, 30 Jun 2020 11:53:38 -0700 Subject: [PATCH 4/9] Update content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md Co-authored-by: Tim Bannister --- .../windows/intro-windows-in-kubernetes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md b/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md index bbb022d328..ff8c04d743 100644 --- a/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md +++ b/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md @@ -328,7 +328,7 @@ Windows does not support "IPv6-only" networking. However, Windows does support d ##### Session affinity -Setting the maximum session sticky time using `service.spec.sessionAffinityConfig.clientIP.timeoutSeconds` is not supported on Windows as this has not been implemented. +Setting the maximum session sticky time using `service.spec.sessionAffinityConfig.clientIP.timeoutSeconds` is not implemented for Windows nodes. ##### Security From 7686c208aefe241988d447ade64eeee34d25305b Mon Sep 17 00:00:00 2001 From: David Schott <36676176+daschott@users.noreply.github.com> Date: Tue, 30 Jun 2020 11:53:46 -0700 Subject: [PATCH 5/9] Update content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md Co-authored-by: Tim Bannister --- .../windows/intro-windows-in-kubernetes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md b/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md index ff8c04d743..55d0b2e2e1 100644 --- a/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md +++ b/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md @@ -189,7 +189,7 @@ For the node, pod, and service objects, the following network flows are supporte * Node -> Pod * Pod -> Node -##### IPAM +##### IP address management (IPAM) {#ipam} The following IPAM options are supported on Windows: * [Host-local](https://github.com/containernetworking/plugins/tree/master/plugins/ipam/host-local) From 90e2a3000510dc1b36060f04933494a142376377 Mon Sep 17 00:00:00 2001 From: David Schott <36676176+daschott@users.noreply.github.com> Date: Tue, 30 Jun 2020 11:53:55 -0700 Subject: [PATCH 6/9] Update content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md Co-authored-by: Tim Bannister --- .../windows/intro-windows-in-kubernetes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md b/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md index 55d0b2e2e1..98c73328d8 100644 --- a/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md +++ b/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md @@ -202,7 +202,7 @@ The following settings applicable to services and load balancing are available o | Feature | Description | Supported Kubernetes version | Supported Windows OS build | How to enable | | ------- | ----------- | ----------------------------- | -------------------------- | ------------- | | Session affinity | Ensures that connections from a particular client are passed to the same Pod each time. | v1.19+ | [Windows Server vNext Insider Preview Build 19551](https://blogs.windows.com/windowsexperience/2020/01/28/announcing-windows-server-vnext-insider-preview-build-19551/) (or higher) | Set `service.spec.sessionAffinity` to "ClientIP" | -| DSR | Load balancing mode where the IP fixups and the LBNAT occurs at the container vSwitch port directly; service traffic arrives with the source IP set as the originating pod IP. Promises lower latency and scalability. | v1.15+ | Windows Server, version 2004 | Set the following flags in kube-proxy: `feature-gates="WinDSR=true" --enable-dsr=true` | +| Direct Server Return | Load balancing mode where the IP address fixups and the LBNAT occurs at the container vSwitch port directly; service traffic arrives with the source IP set as the originating pod IP. Promises lower latency and scalability. | v1.15+ | Windows Server, version 2004 | Set the following flags in kube-proxy: `feature-gates="WinDSR=true" --enable-dsr=true` | | Preserve-DIP | Skips DNAT of service traffic, thereby preserving the virtual IP of the target service in packets reaching the backend Pod. This setting will also ensure that the client IP of incoming packets get preserved. | v1.15+ | Windows Server, version 1903 (or higher) | Set `"preserve-destination": "true"` in service annotations and enable DSR. | | IPv4/IPv6 dual-stack networking | Native IPv4-to-IPv4 in parallel with IPv6-to-IPv6 communications to, from, and within a cluster | v1.19+ | Windows Server vNext Insider Preview Build 19603 (or higher) | See [IPv4/IPv6 dual-stack](#ipv4ipv6-dual-stack) | From 0349514d11126c7d672ddc70ed13da772b73aedd Mon Sep 17 00:00:00 2001 From: David Schott <36676176+daschott@users.noreply.github.com> Date: Tue, 30 Jun 2020 11:54:33 -0700 Subject: [PATCH 7/9] Update content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md Co-authored-by: Tim Bannister --- .../windows/intro-windows-in-kubernetes.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md b/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md index 98c73328d8..9be4d1da36 100644 --- a/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md +++ b/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md @@ -196,8 +196,11 @@ The following IPAM options are supported on Windows: * HNS IPAM (Inbox platform IPAM, this is a fallback when no IPAM is set) * [Azure-vnet-ipam](https://github.com/Azure/azure-container-networking/blob/master/docs/ipam.md) (for azure-cni only) -##### Services & Load Balancing -The following settings applicable to services and load balancing are available on Windows: +##### Load balancing and Services + +On Windows, you can use the following settings to configure Services and load balancing behavior:``` + +This wording makes it clear that “Service” refers to the Kubernetes resource by that name. | Feature | Description | Supported Kubernetes version | Supported Windows OS build | How to enable | | ------- | ----------- | ----------------------------- | -------------------------- | ------------- | From ca3e813cb26e1e78641a01596cc87e5d0d30b2dc Mon Sep 17 00:00:00 2001 From: David Schott <36676176+daschott@users.noreply.github.com> Date: Tue, 30 Jun 2020 12:33:53 -0700 Subject: [PATCH 8/9] Update content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md Co-authored-by: Tim Bannister --- .../windows/intro-windows-in-kubernetes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md b/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md index 9be4d1da36..81273fe318 100644 --- a/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md +++ b/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md @@ -214,7 +214,7 @@ This wording makes it clear that “Service” refers to the Kubernetes resource You can enable IPv4/IPv6 dual-stack networking for `l2bridge` networks using the `IPv6DualStack` [feature gate](https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/). See [enable IPv4/IPv6 dual stack](/docs/concepts/services-networking/dual-stack#enable-ipv4ipv6-dual-stack) for more details. {{< note >}} -Requires Windows Server vNext Insider Preview Build 19603 (or higher). +On Windows, using IPv6 with Kubernetes requires Windows Server vNext Insider Preview Build 19603 (or higher). Overlay (VXLAN) networks on Windows do not support dual-stack networking today. From 4a746cf2c4e177d6d8191be12c45516521c82e14 Mon Sep 17 00:00:00 2001 From: David Schott Date: Tue, 30 Jun 2020 12:57:23 -0700 Subject: [PATCH 9/9] addressing feedback --- .../windows/intro-windows-in-kubernetes.md | 25 ++++++++----------- 1 file changed, 11 insertions(+), 14 deletions(-) diff --git a/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md b/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md index 81273fe318..6eca177609 100644 --- a/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md +++ b/content/en/docs/setup/production-environment/windows/intro-windows-in-kubernetes.md @@ -198,30 +198,27 @@ The following IPAM options are supported on Windows: ##### Load balancing and Services -On Windows, you can use the following settings to configure Services and load balancing behavior:``` - -This wording makes it clear that “Service” refers to the Kubernetes resource by that name. +On Windows, you can use the following settings to configure Services and load balancing behavior: +{{< table caption="Windows Service Settings" >}} | Feature | Description | Supported Kubernetes version | Supported Windows OS build | How to enable | | ------- | ----------- | ----------------------------- | -------------------------- | ------------- | | Session affinity | Ensures that connections from a particular client are passed to the same Pod each time. | v1.19+ | [Windows Server vNext Insider Preview Build 19551](https://blogs.windows.com/windowsexperience/2020/01/28/announcing-windows-server-vnext-insider-preview-build-19551/) (or higher) | Set `service.spec.sessionAffinity` to "ClientIP" | -| Direct Server Return | Load balancing mode where the IP address fixups and the LBNAT occurs at the container vSwitch port directly; service traffic arrives with the source IP set as the originating pod IP. Promises lower latency and scalability. | v1.15+ | Windows Server, version 2004 | Set the following flags in kube-proxy: `feature-gates="WinDSR=true" --enable-dsr=true` | -| Preserve-DIP | Skips DNAT of service traffic, thereby preserving the virtual IP of the target service in packets reaching the backend Pod. This setting will also ensure that the client IP of incoming packets get preserved. | v1.15+ | Windows Server, version 1903 (or higher) | Set `"preserve-destination": "true"` in service annotations and enable DSR. | +| Direct Server Return | Load balancing mode where the IP address fixups and the LBNAT occurs at the container vSwitch port directly; service traffic arrives with the source IP set as the originating pod IP. Promises lower latency and scalability. | v1.15+ | Windows Server, version 2004 | Set the following flags in kube-proxy: `--feature-gates="WinDSR=true" --enable-dsr=true` | +| Preserve-Destination | Skips DNAT of service traffic, thereby preserving the virtual IP of the target service in packets reaching the backend Pod. This setting will also ensure that the client IP of incoming packets get preserved. | v1.15+ | Windows Server, version 1903 (or higher) | Set `"preserve-destination": "true"` in service annotations and enable DSR flags in kube-proxy. | | IPv4/IPv6 dual-stack networking | Native IPv4-to-IPv4 in parallel with IPv6-to-IPv6 communications to, from, and within a cluster | v1.19+ | Windows Server vNext Insider Preview Build 19603 (or higher) | See [IPv4/IPv6 dual-stack](#ipv4ipv6-dual-stack) | - +{{< /table >}} #### IPv4/IPv6 dual-stack You can enable IPv4/IPv6 dual-stack networking for `l2bridge` networks using the `IPv6DualStack` [feature gate](https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/). See [enable IPv4/IPv6 dual stack](/docs/concepts/services-networking/dual-stack#enable-ipv4ipv6-dual-stack) for more details. {{< note >}} -On Windows, using IPv6 with Kubernetes requires Windows Server vNext Insider Preview Build 19603 (or higher). - -Overlay (VXLAN) networks on Windows do not support dual-stack networking today. - -This feature will become available on Windows Server, version 2004 through a cumulative update tentatively scheduled to be released in 1.19 - 1.20 release timeframe. - +On Windows, using IPv6 with Kubernetes require Windows Server vNext Insider Preview Build 19603 (or higher). {{< /note >}} +{{< note >}} +Overlay (VXLAN) networks on Windows do not support dual-stack networking today. +{{< /note >}} ### Limitations @@ -327,11 +324,11 @@ These features were added in Kubernetes v1.15: * On Windows, there are multiple DNS resolvers that can be used. As these come with slightly different behaviors, using the `Resolve-DNSName` utility for name query resolutions is recommended. ##### IPv6 -Windows does not support "IPv6-only" networking. However, Windows does support dual-stack IPv4/IPv6 networking for pods and nodes with single-family services. See [IPv4/IPv6 dual-stack networking](#ipv4ipv6-dual-stack) for more details. +Kubernetes on Windows does not support single-stack "IPv6-only" networking. However,dual-stack IPv4/IPv6 networking for pods and nodes with single-family services is supported. See [IPv4/IPv6 dual-stack networking](#ipv4ipv6-dual-stack) for more details. ##### Session affinity -Setting the maximum session sticky time using `service.spec.sessionAffinityConfig.clientIP.timeoutSeconds` is not implemented for Windows nodes. +Setting the maximum session sticky time for Windows services using `service.spec.sessionAffinityConfig.clientIP.timeoutSeconds` is not supported. ##### Security