Merge remote-tracking branch 'upstream/master' into HEAD
This commit is contained in:
vineeth
@@ -24,7 +24,7 @@ This section covers different options to set up and run Kubernetes.
|
||||
|
||||
Different Kubernetes solutions meet different requirements: ease of maintenance, security, control, available resources, and expertise required to operate and manage a cluster.
|
||||
|
||||
You can deploy a Kubernetes cluster on a local machine, cloud, on-prem datacenter; or choose a managed Kubernetes cluster. You can also create custom solutions across a wide range of cloud providers, or bare metal environments.
|
||||
You can deploy a Kubernetes cluster on a local machine, cloud, on-prem datacenter, or choose a managed Kubernetes cluster. You can also create custom solutions across a wide range of cloud providers, or bare metal environments.
|
||||
|
||||
More simply, you can create a Kubernetes cluster in learning and production environments.
|
||||
|
||||
@@ -53,63 +53,6 @@ If you're learning Kubernetes, use the Docker-based solutions: tools supported b
|
||||
|
||||
When evaluating a solution for a production environment, consider which aspects of operating a Kubernetes cluster (or _abstractions_) you want to manage yourself or offload to a provider.
|
||||
|
||||
Some possible abstractions of a Kubernetes cluster are {{< glossary_tooltip text="applications" term_id="applications" >}}, {{< glossary_tooltip text="data plane" term_id="data-plane" >}}, {{< glossary_tooltip text="control plane" term_id="control-plane" >}}, {{< glossary_tooltip text="cluster infrastructure" term_id="cluster-infrastructure" >}}, and {{< glossary_tooltip text="cluster operations" term_id="cluster-operations" >}}.
|
||||
|
||||
The following diagram lists the possible abstractions of a Kubernetes cluster and whether an abstraction is self-managed or managed by a provider.
|
||||
|
||||
Production environment solutions
|
||||
|
||||
{{< table caption="Production environment solutions table lists the providers and the solutions." >}}
|
||||
The following production environment solutions table lists the providers and the solutions that they offer.
|
||||
|
||||
|Providers | Managed | Turnkey cloud | On-prem datacenter | Custom (cloud) | Custom (On-premises VMs)| Custom (Bare Metal) |
|
||||
| --------- | ------ | ------ | ------ | ------ | ------ | ----- |
|
||||
| [Agile Stacks](https://www.agilestacks.com/products/kubernetes)| | ✔ | ✔ | | |
|
||||
| [Alibaba Cloud](https://www.alibabacloud.com/product/kubernetes)| | ✔ | | | |
|
||||
| [Amazon](https://aws.amazon.com) | [Amazon EKS](https://aws.amazon.com/eks/) |[Amazon EC2](https://aws.amazon.com/ec2/) | | | |
|
||||
| [AppsCode](https://appscode.com/products/pharmer/) | ✔ | | | | |
|
||||
| [APPUiO](https://appuio.ch/) | ✔ | ✔ | ✔ | | | |
|
||||
| [Banzai Cloud Pipeline Kubernetes Engine (PKE)](https://banzaicloud.com/products/pke/) | | ✔ | | ✔ | ✔ | ✔ |
|
||||
| [CenturyLink Cloud](https://www.ctl.io/) | | ✔ | | | |
|
||||
| [Cisco Container Platform](https://cisco.com/go/containers) | | | ✔ | | |
|
||||
| [Cloud Foundry Container Runtime (CFCR)](https://docs-cfcr.cfapps.io/) | | | | ✔ |✔ |
|
||||
| [CloudStack](https://cloudstack.apache.org/) | | | | | ✔|
|
||||
| [Canonical](https://ubuntu.com/kubernetes) | ✔ | ✔ | ✔ | ✔ |✔ | ✔
|
||||
| [Containership](https://containership.io) | ✔ |✔ | | | |
|
||||
| [D2iQ](https://d2iq.com/) | | [Kommander](https://d2iq.com/solutions/ksphere) | [Konvoy](https://d2iq.com/solutions/ksphere/konvoy) | [Konvoy](https://d2iq.com/solutions/ksphere/konvoy) | [Konvoy](https://d2iq.com/solutions/ksphere/konvoy) | [Konvoy](https://d2iq.com/solutions/ksphere/konvoy) |
|
||||
| [Digital Rebar](https://provision.readthedocs.io/en/tip/README.html) | | | | | | ✔
|
||||
| [DigitalOcean](https://www.digitalocean.com/products/kubernetes/) | ✔ | | | | |
|
||||
| [Docker Enterprise](https://www.docker.com/products/docker-enterprise) | |✔ | ✔ | | | ✔
|
||||
| [Gardener](https://gardener.cloud/) | ✔ | ✔ | ✔ | ✔ | ✔ | [Custom Extensions](https://github.com/gardener/gardener/blob/master/docs/extensions/overview.md) |
|
||||
| [Giant Swarm](https://www.giantswarm.io/) | ✔ | ✔ | ✔ | |
|
||||
| [Google](https://cloud.google.com/) | [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine/) | [Google Compute Engine (GCE)](https://cloud.google.com/compute/)|[GKE On-Prem](https://cloud.google.com/gke-on-prem/) | | | | | | | |
|
||||
| [Hidora](https://hidora.com/) | ✔ | ✔| ✔ | | | | | | | |
|
||||
| [IBM](https://www.ibm.com/in-en/cloud) | [IBM Cloud Kubernetes Service](https://cloud.ibm.com/kubernetes/catalog/cluster)| |[IBM Cloud Private](https://www.ibm.com/in-en/cloud/private) | |
|
||||
| [Ionos](https://www.ionos.com/enterprise-cloud) | [Ionos Managed Kubernetes](https://www.ionos.com/enterprise-cloud/managed-kubernetes) | [Ionos Enterprise Cloud](https://www.ionos.com/enterprise-cloud) | |
|
||||
| [Kontena Pharos](https://www.kontena.io/pharos/) | |✔| ✔ | | |
|
||||
| [KubeOne](https://kubeone.io/) | | ✔ | ✔ | ✔ | ✔ | ✔ |
|
||||
| [Kubermatic](https://kubermatic.io/) | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ |
|
||||
| [KubeSail](https://kubesail.com/) | ✔ | | | | |
|
||||
| [Kubespray](https://kubespray.io/#/) | | | |✔ | ✔ | ✔ |
|
||||
| [Kublr](https://kublr.com/) |✔ | ✔ |✔ |✔ |✔ |✔ |
|
||||
| [Microsoft Azure](https://azure.microsoft.com) | [Azure Kubernetes Service (AKS)](https://azure.microsoft.com/en-us/services/kubernetes-service/) | | | | |
|
||||
| [Mirantis Cloud Platform](https://www.mirantis.com/software/kubernetes/) | | | ✔ | | |
|
||||
| [NetApp Kubernetes Service (NKS)](https://cloud.netapp.com/kubernetes-service) | ✔ | ✔ | ✔ | | |
|
||||
| [Nirmata](https://www.nirmata.com/) | | ✔ | ✔ | | |
|
||||
| [Nutanix](https://www.nutanix.com/en) | [Nutanix Karbon](https://www.nutanix.com/products/karbon) | [Nutanix Karbon](https://www.nutanix.com/products/karbon) | | | [Nutanix AHV](https://www.nutanix.com/products/acropolis/virtualization) |
|
||||
| [OpenNebula](https://www.opennebula.org) |[OpenNebula Kubernetes](https://marketplace.opennebula.systems/docs/service/kubernetes.html) | | | | |
|
||||
| [OpenShift](https://www.openshift.com) |[OpenShift Dedicated](https://www.openshift.com/products/dedicated/) and [OpenShift Online](https://www.openshift.com/products/online/) | | [OpenShift Container Platform](https://www.openshift.com/products/container-platform/) | | [OpenShift Container Platform](https://www.openshift.com/products/container-platform/) |[OpenShift Container Platform](https://www.openshift.com/products/container-platform/)
|
||||
| [Oracle Cloud Infrastructure Container Engine for Kubernetes (OKE)](https://docs.cloud.oracle.com/iaas/Content/ContEng/Concepts/contengoverview.htm) | ✔ | ✔ | | | |
|
||||
| [oVirt](https://www.ovirt.org/) | | | | | ✔ |
|
||||
| [Pivotal](https://pivotal.io/) | | [Enterprise Pivotal Container Service (PKS)](https://pivotal.io/platform/pivotal-container-service) | [Enterprise Pivotal Container Service (PKS)](https://pivotal.io/platform/pivotal-container-service) | | |
|
||||
| [Platform9](https://platform9.com/) | [Platform9 Managed Kubernetes](https://platform9.com/managed-kubernetes/) | | [Platform9 Managed Kubernetes](https://platform9.com/managed-kubernetes/) | ✔ | ✔ | ✔
|
||||
| [Rancher](https://rancher.com/) | | [Rancher 2.x](https://rancher.com/docs/rancher/v2.x/en/) | | [Rancher Kubernetes Engine (RKE)](https://rancher.com/docs/rke/latest/en/) | | [k3s](https://k3s.io/)
|
||||
| [Supergiant](https://supergiant.io/) | |✔ | | | |
|
||||
| [SUSE](https://www.suse.com/) | | ✔ | | | |
|
||||
| [SysEleven](https://www.syseleven.io/) | ✔ | | | | |
|
||||
| [Tencent Cloud](https://intl.cloud.tencent.com/) | [Tencent Kubernetes Engine](https://intl.cloud.tencent.com/product/tke) | ✔ | ✔ | | | ✔ |
|
||||
| [VEXXHOST](https://vexxhost.com/) | ✔ | ✔ | | | |
|
||||
| [VMware](https://cloud.vmware.com/) | [VMware Cloud PKS](https://cloud.vmware.com/vmware-cloud-pks) |[VMware Enterprise PKS](https://cloud.vmware.com/vmware-enterprise-pks) | [VMware Enterprise PKS](https://cloud.vmware.com/vmware-enterprise-pks) | [VMware Essential PKS](https://cloud.vmware.com/vmware-essential-pks) | |[VMware Essential PKS](https://cloud.vmware.com/vmware-essential-pks)
|
||||
| [Z.A.R.V.I.S.](https://zarvis.ai/) | ✔ | | | | | |
|
||||
For a list of [Certified Kubernetes](https://github.com/cncf/k8s-conformance/#certified-kubernetes) providers, see "[Partners](https://kubernetes.io/partners/#conformance)".
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
@@ -188,7 +188,7 @@ kubernetes-minion-wf8i Ready <none> 2m v1.13.0
|
||||
|
||||
Create a volume using the dynamic volume creation (only PersistentVolumes are supported for zone affinity):
|
||||
|
||||
```json
|
||||
```bash
|
||||
kubectl apply -f - <<EOF
|
||||
{
|
||||
"apiVersion": "v1",
|
||||
|
||||
@@ -24,7 +24,7 @@ Minikube supports the following Kubernetes features:
|
||||
* NodePorts
|
||||
* ConfigMaps and Secrets
|
||||
* Dashboards
|
||||
* Container Runtime: Docker, [CRI-O](https://cri-o.io/), and [containerd](https://github.com/containerd/containerd)
|
||||
* Container Runtime: [Docker](https://www.docker.com/), [CRI-O](https://cri-o.io/), and [containerd](https://github.com/containerd/containerd)
|
||||
* Enabling CNI (Container Network Interface)
|
||||
* Ingress
|
||||
|
||||
|
||||
@@ -49,14 +49,12 @@ Download the latest release with the command:
|
||||
curl -LO https://github.com/kubernetes/kops/releases/download/$(curl -s https://api.github.com/repos/kubernetes/kops/releases/latest | grep tag_name | cut -d '"' -f 4)/kops-darwin-amd64
|
||||
```
|
||||
|
||||
To download a specific version, replace the
|
||||
To download a specific version, replace the following portion of the command with the specific kops version.
|
||||
|
||||
```shell
|
||||
$(curl -s https://api.github.com/repos/kubernetes/kops/releases/latest | grep tag_name | cut -d '"' -f 4)
|
||||
```
|
||||
|
||||
portion of the command with the specific version.
|
||||
|
||||
For example, to download kops version v1.15.0 type:
|
||||
|
||||
```shell
|
||||
|
||||
@@ -344,10 +344,6 @@ Please refer to this installation guide: [Contiv-VPP Manual Installation](https:
|
||||
|
||||
For `flannel` to work correctly, you must pass `--pod-network-cidr=10.244.0.0/16` to `kubeadm init`.
|
||||
|
||||
Set `/proc/sys/net/bridge/bridge-nf-call-iptables` to `1` by running `sysctl net.bridge.bridge-nf-call-iptables=1`
|
||||
to pass bridged IPv4 traffic to iptables' chains. This is a requirement for some CNI plugins to work, for more information
|
||||
please see [Network Plugin Requirements](/docs/concepts/cluster-administration/network-plugins/#network-plugin-requirements).
|
||||
|
||||
Make sure that your firewall rules allow UDP ports 8285 and 8472 traffic for all hosts participating in the overlay network. The [Firewall](https://coreos.com/flannel/docs/latest/troubleshooting.html#firewalls) section of Flannel's troubleshooting guide explains about this in more detail.
|
||||
|
||||
Flannel works on `amd64`, `arm`, `arm64`, `ppc64le` and `s390x` architectures under Linux.
|
||||
@@ -362,9 +358,6 @@ For more information about `flannel`, see [the CoreOS flannel repository on GitH
|
||||
{{% /tab %}}
|
||||
|
||||
{{% tab name="Kube-router" %}}
|
||||
Set `/proc/sys/net/bridge/bridge-nf-call-iptables` to `1` by running `sysctl net.bridge.bridge-nf-call-iptables=1`
|
||||
to pass bridged IPv4 traffic to iptables' chains. This is a requirement for some CNI plugins to work, for more information
|
||||
please see [Network Plugin Requirements](/docs/concepts/cluster-administration/network-plugins/#network-plugin-requirements).
|
||||
|
||||
Kube-router relies on kube-controller-manager to allocate Pod CIDR for the nodes. Therefore, use `kubeadm init` with the `--pod-network-cidr` flag.
|
||||
|
||||
@@ -374,9 +367,6 @@ For information on using the `kubeadm` tool to set up a Kubernetes cluster with
|
||||
{{% /tab %}}
|
||||
|
||||
{{% tab name="Weave Net" %}}
|
||||
Set `/proc/sys/net/bridge/bridge-nf-call-iptables` to `1` by running `sysctl net.bridge.bridge-nf-call-iptables=1`
|
||||
to pass bridged IPv4 traffic to iptables' chains. This is a requirement for some CNI plugins to work, for more information
|
||||
please see [Network Plugin Requirements](/docs/concepts/cluster-administration/network-plugins/#network-plugin-requirements).
|
||||
|
||||
For more information on setting up your Kubernetes cluster with Weave Net, please see [Integrating Kubernetes via the Addon]((https://www.weave.works/docs/net/latest/kube-addon/).
|
||||
|
||||
|
||||
@@ -51,6 +51,23 @@ may [fail](https://github.com/kubernetes/kubeadm/issues/31).
|
||||
If you have more than one network adapter, and your Kubernetes components are not reachable on the default
|
||||
route, we recommend you add IP route(s) so Kubernetes cluster addresses go via the appropriate adapter.
|
||||
|
||||
## Letting iptables see bridged traffic
|
||||
|
||||
As a requirement for your Linux Node's iptables to correctly see bridged traffic, you should ensure `net.bridge.bridge-nf-call-iptables` is set to 1 in your `sysctl` config, e.g.
|
||||
|
||||
```bash
|
||||
cat <<EOF > /etc/sysctl.d/k8s.conf
|
||||
net.bridge.bridge-nf-call-ip6tables = 1
|
||||
net.bridge.bridge-nf-call-iptables = 1
|
||||
EOF
|
||||
sysctl --system
|
||||
```
|
||||
|
||||
Make sure that the `br_netfilter` module is loaded before this step. This can be done by running `lsmod | grep br_netfilter`. To load it explicitly call `modprobe br_netfilter`.
|
||||
|
||||
For more details please see the [Network Plugin Requirements](https://kubernetes.io/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins/#network-plugin-requirements) page.
|
||||
|
||||
|
||||
## Ensure iptables tooling does not use the nftables backend
|
||||
|
||||
In Linux, nftables is available as a modern replacement for the kernel's iptables subsystem. The
|
||||
@@ -230,17 +247,7 @@ systemctl enable --now kubelet
|
||||
- Setting SELinux in permissive mode by running `setenforce 0` and `sed ...` effectively disables it.
|
||||
This is required to allow containers to access the host filesystem, which is needed by pod networks for example.
|
||||
You have to do this until SELinux support is improved in the kubelet.
|
||||
- Some users on RHEL/CentOS 7 have reported issues with traffic being routed incorrectly due to iptables being bypassed. You should ensure
|
||||
`net.bridge.bridge-nf-call-iptables` is set to 1 in your `sysctl` config, e.g.
|
||||
|
||||
```bash
|
||||
cat <<EOF > /etc/sysctl.d/k8s.conf
|
||||
net.bridge.bridge-nf-call-ip6tables = 1
|
||||
net.bridge.bridge-nf-call-iptables = 1
|
||||
EOF
|
||||
sysctl --system
|
||||
```
|
||||
- Make sure that the `br_netfilter` module is loaded before this step. This can be done by running `lsmod | grep br_netfilter`. To load it explicitly call `modprobe br_netfilter`.
|
||||
|
||||
{{% /tab %}}
|
||||
{{% tab name="Container Linux" %}}
|
||||
Install CNI plugins (required for most pod network):
|
||||
|
||||
+3
-3
@@ -307,17 +307,17 @@ The tracking issue for this problem is [here](https://github.com/kubernetes/kube
|
||||
|
||||
*Note: This [issue](https://github.com/kubernetes/kubeadm/issues/1358) only applies to tools that marshal kubeadm types (e.g. to a YAML configuration file). It will be fixed in kubeadm API v1beta2.*
|
||||
|
||||
By default, kubeadm applies the `role.kubernetes.io/master:NoSchedule` taint to control-plane nodes.
|
||||
By default, kubeadm applies the `node-role.kubernetes.io/master:NoSchedule` taint to control-plane nodes.
|
||||
If you prefer kubeadm to not taint the control-plane node, and set `InitConfiguration.NodeRegistration.Taints` to an empty slice,
|
||||
the field will be omitted when marshalling. When the field is omitted, kubeadm applies the default taint.
|
||||
|
||||
There are at least two workarounds:
|
||||
|
||||
1. Use the `role.kubernetes.io/master:PreferNoSchedule` taint instead of an empty slice. [Pods will get scheduled on masters](/docs/concepts/configuration/taint-and-toleration/), unless other nodes have capacity.
|
||||
1. Use the `node-role.kubernetes.io/master:PreferNoSchedule` taint instead of an empty slice. [Pods will get scheduled on masters](/docs/concepts/configuration/taint-and-toleration/), unless other nodes have capacity.
|
||||
|
||||
2. Remove the taint after kubeadm init exits:
|
||||
```bash
|
||||
kubectl taint nodes NODE_NAME role.kubernetes.io/master:NoSchedule-
|
||||
kubectl taint nodes NODE_NAME node-role.kubernetes.io/master:NoSchedule-
|
||||
```
|
||||
|
||||
## `/usr` is mounted read-only on nodes {#usr-mounted-read-only}
|
||||
|
||||
@@ -107,6 +107,14 @@ Port mapping is also supported, but for simplicity in this example the container
|
||||
Windows container hosts are not able to access the IP of services scheduled on them due to current platform limitations of the Windows networking stack. Only Windows pods are able to access service IPs.
|
||||
{{< /note >}}
|
||||
|
||||
## Observability
|
||||
|
||||
### Capturing logs from workloads
|
||||
|
||||
Logs are an important element of observability; they enable users to gain insights into the operational aspect of workloads and are a key ingredient to troubleshooting issues. Because Windows containers and workloads inside Windows containers behave differently from Linux containers, users had a hard time collecting logs, limiting operational visibility. Windows workloads for example are usually configured to log to ETW (Event Tracing for Windows) or push entries to the application event log. [LogMonitor](https://github.com/microsoft/windows-container-tools/tree/master/LogMonitor), an open source tool by Microsoft, is the recommended way to monitor configured log sources inside a Windows container. LogMonitor supports monitoring event logs, ETW providers, and custom application logs, piping them to STDOUT for consumption by `kubectl logs <pod>`.
|
||||
|
||||
Follow the instructions in the LogMonitor GitHub page to copy its binaries and configuration files to all your containers and add the necessary entrypoints for LogMonitor to push your logs to STDOUT.
|
||||
|
||||
## Using configurable Container usernames
|
||||
|
||||
Starting with Kubernetes v1.16, Windows containers can be configured to run their entrypoints and processes with different usernames than the image defaults. The way this is achieved is a bit different from the way it is done for Linux containers. Learn more about it [here](/docs/tasks/configure-pod-container/configure-runasusername/).
|
||||
|
||||
Reference in New Issue
Block a user