Merge remote-tracking branch 'upstream/master' into dev-1.19

This commit is contained in:
Savitha Raghunathan
2020-05-26 18:03:50 -04:00
97 changed files with 3552 additions and 867 deletions
@@ -0,0 +1,589 @@
---
layout: blog
title: "WSL+Docker: Kubernetes on the Windows Desktop"
date: 2020-05-21
slug: wsl-docker-kubernetes-on-the-windows-desktop
---
**Authors**: [Nuno do Carmo](https://twitter.com/nunixtech) Docker Captain and WSL Corsair; [Ihor Dvoretskyi](https://twitter.com/idvoretskyi), Developer Advocate, Cloud Native Computing Foundation
# Introduction
New to Windows 10 and WSL2, or new to Docker and Kubernetes? Welcome to this blog post where we will install from scratch Kubernetes in Docker [KinD](https://kind.sigs.k8s.io/) and [Minikube](https://minikube.sigs.k8s.io/docs/).
# Why Kubernetes on Windows?
For the last few years, Kubernetes became a de-facto standard platform for running containerized services and applications in distributed environments. While a wide variety of distributions and installers exist to deploy Kubernetes in the cloud environments (public, private or hybrid), or within the bare metal environments, there is still a need to deploy and run Kubernetes locally, for example, on the developer's workstation.
Kubernetes has been originally designed to be deployed and used in the Linux environments. However, a good number of users (and not only application developers) use Windows OS as their daily driver. When Microsoft revealed WSL - [the Windows Subsystem for Linux](https://docs.microsoft.com/en-us/windows/wsl/), the line between Windows and Linux environments became even less visible.
Also, WSL brought an ability to run Kubernetes on Windows almost seamlessly!
Below, we will cover in brief how to install and use various solutions to run Kubernetes locally.
# Prerequisites
Since we will explain how to install KinD, we won't go into too much detail around the installation of KinD's dependencies.
However, here is the list of the prerequisites needed and their version/lane:
- OS: Windows 10 version 2004, Build 19041
- [WSL2 enabled](https://docs.microsoft.com/en-us/windows/wsl/wsl2-install)
- In order to install the distros as WSL2 by default, once WSL2 installed, run the command `wsl.exe --set-default-version 2` in Powershell
- WSL2 distro installed from the Windows Store - the distro used is Ubuntu-18.04
- [Docker Desktop for Windows](https://hub.docker.com/editions/community/docker-ce-desktop-windows), stable channel - the version used is 2.2.0.4
- [Optional] Microsoft Terminal installed from the Windows Store
- Open the Windows store and type "Terminal" in the search, it will be (normally) the first option
![Windows Store Terminal](/images/blog/2020-05-21-wsl2-dockerdesktop-k8s/wsl2-windows-store-terminal.png)
And that's actually it. For Docker Desktop for Windows, no need to configure anything yet as we will explain it in the next section.
# WSL2: First contact
Once everything is installed, we can launch the WSL2 terminal from the Start menu, and type "Ubuntu" for searching the applications and documents:
![Start Menu Search](/images/blog/2020-05-21-wsl2-dockerdesktop-k8s/wsl2-start-menu-search.png)
Once found, click on the name and it will launch the default Windows console with the Ubuntu bash shell running.
Like for any normal Linux distro, you need to create a user and set a password:
![User-Password](/images/blog/2020-05-21-wsl2-dockerdesktop-k8s/wsl2-user-password.png)
## [Optional] Update the `sudoers`
As we are working, normally, on our local computer, it might be nice to update the `sudoers` and set the group `%sudo` to be password-less:
```bash
# Edit the sudoers with the visudo command
sudo visudo
# Change the %sudo group to be password-less
%sudo ALL=(ALL:ALL) NOPASSWD: ALL
# Press CTRL+X to exit
# Press Y to save
# Press Enter to confirm
```
![visudo](/images/blog/2020-05-21-wsl2-dockerdesktop-k8s/wsl2-visudo.png)
## Update Ubuntu
Before we move to the Docker Desktop settings, let's update our system and ensure we start in the best conditions:
```bash
# Update the repositories and list of the packages available
sudo apt update
# Update the system based on the packages installed > the "-y" will approve the change automatically
sudo apt upgrade -y
```
![apt-update-upgrade](/images/blog/2020-05-21-wsl2-dockerdesktop-k8s/wsl2-apt-update-upgrade.png)
# Docker Desktop: faster with WSL2
Before we move into the settings, let's do a small test, it will display really how cool the new integration with Docker Desktop is:
```bash
# Try to see if the docker cli and daemon are installed
docker version
# Same for kubectl
kubectl version
```
![kubectl-error](/images/blog/2020-05-21-wsl2-dockerdesktop-k8s/wsl2-docker-kubectl-error.png)
You got an error? Perfect! It's actually good news, so let's now move on to the settings.
## Docker Desktop settings: enable WSL2 integration
First let's start Docker Desktop for Windows if it's not still the case. Open the Windows start menu and type "docker", click on the name to start the application:
![docker-start](/images/blog/2020-05-21-wsl2-dockerdesktop-k8s/wsl2-docker-start.png)
You should now see the Docker icon with the other taskbar icons near the clock:
![docker-taskbar](/images/blog/2020-05-21-wsl2-dockerdesktop-k8s/wsl2-docker-taskbar.png)
Now click on the Docker icon and choose settings. A new window will appear:
![docker-settings-general](/images/blog/2020-05-21-wsl2-dockerdesktop-k8s/wsl2-docker-settings-general.png)
By default, the WSL2 integration is not active, so click the "Enable the experimental WSL 2 based engine" and click "Apply & Restart":
![docker-settings-wsl2](/images/blog/2020-05-21-wsl2-dockerdesktop-k8s/wsl2-docker-settings-wsl2-activated.png)
What this feature did behind the scenes was to create two new distros in WSL2, containing and running all the needed backend sockets, daemons and also the CLI tools (read: docker and kubectl command).
Still, this first setting is still not enough to run the commands inside our distro. If we try, we will have the same error as before.
In order to fix it, and finally be able to use the commands, we need to tell the Docker Desktop to "attach" itself to our distro also:
![docker-resources-wsl](/images/blog/2020-05-21-wsl2-dockerdesktop-k8s/wsl2-docker-resources-wsl-integration.png)
Let's now switch back to our WSL2 terminal and see if we can (finally) launch the commands:
```bash
# Try to see if the docker cli and daemon are installed
docker version
# Same for kubectl
kubectl version
```
![docker-kubectl-success](/images/blog/2020-05-21-wsl2-dockerdesktop-k8s/wsl2-docker-kubectl-success.png)
> Tip: if nothing happens, restart Docker Desktop and restart the WSL process in Powershell: `Restart-Service LxssManager` and launch a new Ubuntu session
And success! The basic settings are now done and we move to the installation of KinD.
# KinD: Kubernetes made easy in a container
Right now, we have Docker that is installed, configured and the last test worked fine.
However, if we look carefully at the `kubectl` command, it found the "Client Version" (1.15.5), but it didn't find any server.
This is normal as we didn't enable the Docker Kubernetes cluster. So let's install KinD and create our first cluster.
And as sources are always important to mention, we will follow (partially) the how-to on the [official KinD website](https://kind.sigs.k8s.io/docs/user/quick-start/):
```bash
# Download the latest version of KinD
curl -Lo ./kind https://github.com/kubernetes-sigs/kind/releases/download/v0.7.0/kind-$(uname)-amd64
# Make the binary executable
chmod +x ./kind
# Move the binary to your executable path
sudo mv ./kind /usr/local/bin/
```
![kind-install](/images/blog/2020-05-21-wsl2-dockerdesktop-k8s/wsl2-kind-install.png)
## KinD: the first cluster
We are ready to create our first cluster:
```bash
# Check if the KUBECONFIG is not set
echo $KUBECONFIG
# Check if the .kube directory is created > if not, no need to create it
ls $HOME/.kube
# Create the cluster and give it a name (optional)
kind create cluster --name wslkind
# Check if the .kube has been created and populated with files
ls $HOME/.kube
```
![kind-cluster-create](/images/blog/2020-05-21-wsl2-dockerdesktop-k8s/wsl2-kind-cluster-create.png)
> Tip: as you can see, the Terminal was changed so the nice icons are all displayed
The cluster has been successfully created, and because we are using Docker Desktop, the network is all set for us to use "as is".
So we can open the `Kubernetes master` URL in our Windows browser:
![kind-browser-k8s-master](/images/blog/2020-05-21-wsl2-dockerdesktop-k8s/wsl2-kind-browse-k8s-master.png)
And this is the real strength from Docker Desktop for Windows with the WSL2 backend. Docker really did an amazing integration.
## KinD: counting 1 - 2 - 3
Our first cluster was created and it's the "normal" one node cluster:
```bash
# Check how many nodes it created
kubectl get nodes
# Check the services for the whole cluster
kubectl get all --all-namespaces
```
![kind-list-nodes-services](/images/blog/2020-05-21-wsl2-dockerdesktop-k8s/wsl2-kind-list-nodes-services.png)
While this will be enough for most people, let's leverage one of the coolest feature, multi-node clustering:
```bash
# Delete the existing cluster
kind delete cluster --name wslkind
# Create a config file for a 3 nodes cluster
cat << EOF > kind-3nodes.yaml
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
- role: worker
- role: worker
EOF
# Create a new cluster with the config file
kind create cluster --name wslkindmultinodes --config ./kind-3nodes.yaml
# Check how many nodes it created
kubectl get nodes
```
![kind-cluster-create-multinodes](/images/blog/2020-05-21-wsl2-dockerdesktop-k8s/wsl2-kind-cluster-create-multinodes.png)
> Tip: depending on how fast we run the "get nodes" command, it can be that not all the nodes are ready, wait few seconds and run it again, everything should be ready
And that's it, we have created a three-node cluster, and if we look at the services one more time, we will see several that have now three replicas:
```bash
# Check the services for the whole cluster
kubectl get all --all-namespaces
```
![wsl2-kind-list-services-multinodes](/images/blog/2020-05-21-wsl2-dockerdesktop-k8s/wsl2-kind-list-services-multinodes.png)
## KinD: can I see a nice dashboard?
Working on the command line is always good and very insightful. However, when dealing with Kubernetes we might want, at some point, to have a visual overview.
For that, the [Kubernetes Dashboard](https://github.com/kubernetes/dashboard) project has been created. The installation and first connection test is quite fast, so let's do it:
```bash
# Install the Dashboard application into our cluster
kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.0-rc6/aio/deploy/recommended.yaml
# Check the resources it created based on the new namespace created
kubectl get all -n kubernetes-dashboard
```
![kind-install-dashboard](/images/blog/2020-05-21-wsl2-dockerdesktop-k8s/wsl2-kind-install-dashboard.png)
As it created a service with a ClusterIP (read: internal network address), we cannot reach it if we type the URL in our Windows browser:
![kind-browse-dashboard-error](/images/blog/2020-05-21-wsl2-dockerdesktop-k8s/wsl2-kind-browse-dashboard-error.png)
That's because we need to create a temporary proxy:
```bash
# Start a kubectl proxy
kubectl proxy
# Enter the URL on your browser: http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/
```
![kind-browse-dashboard-success](/images/blog/2020-05-21-wsl2-dockerdesktop-k8s/wsl2-kind-browse-dashboard-success.png)
Finally to login, we can either enter a Token, which we didn't create, or enter the `kubeconfig` file from our Cluster.
If we try to login with the `kubeconfig`, we will get the error "Internal error (500): Not enough data to create auth info structure". This is due to the lack of credentials in the `kubeconfig` file.
So to avoid you ending with the same error, let's follow the [recommended RBAC approach](https://github.com/kubernetes/dashboard/blob/master/docs/user/access-control/creating-sample-user.md).
Let's open a new WSL2 session:
```bash
# Create a new ServiceAccount
kubectl apply -f - <<EOF
apiVersion: v1
kind: ServiceAccount
metadata:
name: admin-user
namespace: kubernetes-dashboard
EOF
# Create a ClusterRoleBinding for the ServiceAccount
kubectl apply -f - <<EOF
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: admin-user
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: admin-user
namespace: kubernetes-dashboard
EOF
```
![kind-browse-dashboard-rbac-serviceaccount](/images/blog/2020-05-21-wsl2-dockerdesktop-k8s/wsl2-kind-browse-dashboard-rbac-serviceaccount.png)
```bash
# Get the Token for the ServiceAccount
kubectl -n kubernetes-dashboard describe secret $(kubectl -n kubernetes-dashboard get secret | grep admin-user | awk '{print $1}')
# Copy the token and copy it into the Dashboard login and press "Sign in"
```
![kind-browse-dashboard-login-success](/images/blog/2020-05-21-wsl2-dockerdesktop-k8s/wsl2-kind-browse-dashboard-login-success.png)
Success! And let's see our nodes listed also:
![kind-browse-dashboard-browse-nodes](/images/blog/2020-05-21-wsl2-dockerdesktop-k8s/wsl2-kind-browse-dashboard-browse-nodes.png)
A nice and shiny three nodes appear.
# Minikube: Kubernetes from everywhere
Right now, we have Docker that is installed, configured and the last test worked fine.
However, if we look carefully at the `kubectl` command, it found the "Client Version" (1.15.5), but it didn't find any server.
This is normal as we didn't enable the Docker Kubernetes cluster. So let's install Minikube and create our first cluster.
And as sources are always important to mention, we will follow (partially) the how-to from the [Kubernetes.io website](https://kubernetes.io/docs/tasks/tools/install-minikube/):
```bash
# Download the latest version of Minikube
curl -Lo minikube https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
# Make the binary executable
chmod +x ./minikube
# Move the binary to your executable path
sudo mv ./minikube /usr/local/bin/
```
![minikube-install](/images/blog/2020-05-21-wsl2-dockerdesktop-k8s/wsl2-minikube-install.png)
## Minikube: updating the host
If we follow the how-to, it states that we should use the `--driver=none` flag in order to run Minikube directly on the host and Docker.
Unfortunately, we will get an error about "conntrack" being required to run Kubernetes v 1.18:
```bash
# Create a minikube one node cluster
minikube start --driver=none
```
![minikube-start-error](/images/blog/2020-05-21-wsl2-dockerdesktop-k8s/wsl2-minikube-start-error.png)
> Tip: as you can see, the Terminal was changed so the nice icons are all displayed
So let's fix the issue by installing the missing package:
```bash
# Install the conntrack package
sudo apt install -y conntrack
```
![minikube-install-conntrack](/images/blog/2020-05-21-wsl2-dockerdesktop-k8s/wsl2-minikube-install conntrack.png)
Let's try to launch it again:
```bash
# Create a minikube one node cluster
minikube start --driver=none
# We got a permissions error > try again with sudo
sudo minikube start --driver=none
```
![minikube-start-error-systemd](/images/blog/2020-05-21-wsl2-dockerdesktop-k8s/wsl2-minikube-start-error-systemd.png)
Ok, this error cloud be problematic ... in the past. Luckily for us, there's a solution
## Minikube: enabling SystemD
In order to enable SystemD on WSL2, we will apply the [scripts](https://forum.snapcraft.io/t/running-snaps-on-wsl2-insiders-only-for-now/13033) from [Daniel Llewellyn](https://twitter.com/diddledan).
I invite you to read the full blog post and how he came to the solution, and the various iterations he did to fix several issues.
So in a nutshell, here are the commands:
```bash
# Install the needed packages
sudo apt install -yqq daemonize dbus-user-session fontconfig
```
![minikube-systemd-packages](/images/blog/2020-05-21-wsl2-dockerdesktop-k8s/wsl2-minikube-systemd-packages.png)
```bash
# Create the start-systemd-namespace script
sudo vi /usr/sbin/start-systemd-namespace
#!/bin/bash
SYSTEMD_PID=$(ps -ef | grep '/lib/systemd/systemd --system-unit=basic.target$' | grep -v unshare | awk '{print $2}')
if [ -z "$SYSTEMD_PID" ] || [ "$SYSTEMD_PID" != "1" ]; then
export PRE_NAMESPACE_PATH="$PATH"
(set -o posix; set) | \
grep -v "^BASH" | \
grep -v "^DIRSTACK=" | \
grep -v "^EUID=" | \
grep -v "^GROUPS=" | \
grep -v "^HOME=" | \
grep -v "^HOSTNAME=" | \
grep -v "^HOSTTYPE=" | \
grep -v "^IFS='.*"$'\n'"'" | \
grep -v "^LANG=" | \
grep -v "^LOGNAME=" | \
grep -v "^MACHTYPE=" | \
grep -v "^NAME=" | \
grep -v "^OPTERR=" | \
grep -v "^OPTIND=" | \
grep -v "^OSTYPE=" | \
grep -v "^PIPESTATUS=" | \
grep -v "^POSIXLY_CORRECT=" | \
grep -v "^PPID=" | \
grep -v "^PS1=" | \
grep -v "^PS4=" | \
grep -v "^SHELL=" | \
grep -v "^SHELLOPTS=" | \
grep -v "^SHLVL=" | \
grep -v "^SYSTEMD_PID=" | \
grep -v "^UID=" | \
grep -v "^USER=" | \
grep -v "^_=" | \
cat - > "$HOME/.systemd-env"
echo "PATH='$PATH'" >> "$HOME/.systemd-env"
exec sudo /usr/sbin/enter-systemd-namespace "$BASH_EXECUTION_STRING"
fi
if [ -n "$PRE_NAMESPACE_PATH" ]; then
export PATH="$PRE_NAMESPACE_PATH"
fi
```
```bash
# Create the enter-systemd-namespace
sudo vi /usr/sbin/enter-systemd-namespace
#!/bin/bash
if [ "$UID" != 0 ]; then
echo "You need to run $0 through sudo"
exit 1
fi
SYSTEMD_PID="$(ps -ef | grep '/lib/systemd/systemd --system-unit=basic.target$' | grep -v unshare | awk '{print $2}')"
if [ -z "$SYSTEMD_PID" ]; then
/usr/sbin/daemonize /usr/bin/unshare --fork --pid --mount-proc /lib/systemd/systemd --system-unit=basic.target
while [ -z "$SYSTEMD_PID" ]; do
SYSTEMD_PID="$(ps -ef | grep '/lib/systemd/systemd --system-unit=basic.target$' | grep -v unshare | awk '{print $2}')"
done
fi
if [ -n "$SYSTEMD_PID" ] && [ "$SYSTEMD_PID" != "1" ]; then
if [ -n "$1" ] && [ "$1" != "bash --login" ] && [ "$1" != "/bin/bash --login" ]; then
exec /usr/bin/nsenter -t "$SYSTEMD_PID" -a \
/usr/bin/sudo -H -u "$SUDO_USER" \
/bin/bash -c 'set -a; source "$HOME/.systemd-env"; set +a; exec bash -c '"$(printf "%q" "$@")"
else
exec /usr/bin/nsenter -t "$SYSTEMD_PID" -a \
/bin/login -p -f "$SUDO_USER" \
$(/bin/cat "$HOME/.systemd-env" | grep -v "^PATH=")
fi
echo "Existential crisis"
fi
```
```bash
# Edit the permissions of the enter-systemd-namespace script
sudo chmod +x /usr/sbin/enter-systemd-namespace
# Edit the bash.bashrc file
sudo sed -i 2a"# Start or enter a PID namespace in WSL2\nsource /usr/sbin/start-systemd-namespace\n" /etc/bash.bashrc
```
![minikube-systemd-files](/images/blog/2020-05-21-wsl2-dockerdesktop-k8s/wsl2-minikube-systemd-files.png)
Finally, exit and launch a new session. You **do not** need to stop WSL2, a new session is enough:
![minikube-systemd-enabled](/images/blog/2020-05-21-wsl2-dockerdesktop-k8s/wsl2-minikube-systemd-enabled.png)
## Minikube: the first cluster
We are ready to create our first cluster:
```bash
# Check if the KUBECONFIG is not set
echo $KUBECONFIG
# Check if the .kube directory is created > if not, no need to create it
ls $HOME/.kube
# Check if the .minikube directory is created > if yes, delete it
ls $HOME/.minikube
# Create the cluster with sudo
sudo minikube start --driver=none
```
In order to be able to use `kubectl` with our user, and not `sudo`, Minikube recommends running the `chown` command:
```bash
# Change the owner of the .kube and .minikube directories
sudo chown -R $USER $HOME/.kube $HOME/.minikube
# Check the access and if the cluster is running
kubectl cluster-info
# Check the resources created
kubectl get all --all-namespaces
```
![minikube-start-fixed](/images/blog/2020-05-21-wsl2-dockerdesktop-k8s/wsl2-minikube-start-fixed.png)
The cluster has been successfully created, and Minikube used the WSL2 IP, which is great for several reasons, and one of them is that we can open the `Kubernetes master` URL in our Windows browser:
![minikube-browse-k8s-master](/images/blog/2020-05-21-wsl2-dockerdesktop-k8s/wsl2-minikube-browse-k8s-master.png)
And the real strength of WSL2 integration, the port `8443` once open on WSL2 distro, it actually forwards it to Windows, so instead of the need to remind the IP address, we can also reach the `Kubernetes master` URL via `localhost`:
![minikube-browse-k8s-master-localhost](/images/blog/2020-05-21-wsl2-dockerdesktop-k8s/wsl2-minikube-browse-k8s-master-localhost.png)
## Minikube: can I see a nice dashboard?
Working on the command line is always good and very insightful. However, when dealing with Kubernetes we might want, at some point, to have a visual overview.
For that, Minikube embeded the [Kubernetes Dashboard](https://github.com/kubernetes/dashboard). Thanks to it, running and accessing the Dashboard is very simple:
```bash
# Enable the Dashboard service
sudo minikube dashboard
# Access the Dashboard from a browser on Windows side
```
![minikube-browse-dashboard](/images/blog/2020-05-21-wsl2-dockerdesktop-k8s/wsl2-minikube-browse-dashboard.png)
The command creates also a proxy, which means that once we end the command, by pressing `CTRL+C`, the Dashboard will no more be accessible.
Still, if we look at the namespace `kubernetes-dashboard`, we will see that the service is still created:
```bash
# Get all the services from the dashboard namespace
kubectl get all --namespace kubernetes-dashboard
```
![minikube-dashboard-get-all](/images/blog/2020-05-21-wsl2-dockerdesktop-k8s/wsl2-minikube-dashboard-get-all.png)
Let's edit the service and change it's type to `LoadBalancer`:
```bash
# Edit the Dashoard service
kubectl edit service/kubernetes-dashboard --namespace kubernetes-dashboard
# Go to the very end and remove the last 2 lines
status:
loadBalancer: {}
# Change the type from ClusterIO to LoadBalancer
type: LoadBalancer
# Save the file
```
![minikube-dashboard-type-loadbalancer](/images/blog/2020-05-21-wsl2-dockerdesktop-k8s/wsl2-minikube-dashboard-type-loadbalancer.png)
Check again the Dashboard service and let's access the Dashboard via the LoadBalancer:
```bash
# Get all the services from the dashboard namespace
kubectl get all --namespace kubernetes-dashboard
# Access the Dashboard from a browser on Windows side with the URL: localhost:<port exposed>
```
![minikube-browse-dashboard-loadbalancer](/images/blog/2020-05-21-wsl2-dockerdesktop-k8s/wsl2-minikube-browse-dashboard-loadbalancer.png)
# Conclusion
It's clear that we are far from done as we could have some LoadBalancing implemented and/or other services (storage, ingress, registry, etc...).
Concerning Minikube on WSL2, as it needed to enable SystemD, we can consider it as an intermediate level to be implemented.
So with two solutions, what could be the "best for you"? Both bring their own advantages and inconveniences, so here an overview from our point of view solely:
| Criteria | KinD | Minikube |
| -------------------- | ----------------------------- | -------- |
| Installation on WSL2 | Very Easy | Medium |
| Multi-node | Yes | No |
| Plugins | Manual install | Yes |
| Persistence | Yes, however not designed for | Yes |
| Alternatives | K3d | Microk8s |
We hope you could have a real taste of the integration between the different components: WSL2 - Docker Desktop - KinD/Minikube. And that gave you some ideas or, even better, some answers to your Kubernetes workflows with KinD and/or Minikube on Windows and WSL2.
See you soon for other adventures in the Kubernetes ocean.
[Nuno](https://twitter.com/nunixtech) & [Ihor](https://twitter.com/idvoretskyi)
@@ -308,7 +308,7 @@ Node objects track information about the Node's resource capacity (for example:
of memory available, and the number of CPUs).
Nodes that [self register](#self-registration-of-nodes) report their capacity during
registration. If you [manually](#manual-node-administration) add a Node, then
you need to set the node's capacity informaton when you add it.
you need to set the node's capacity information when you add it.
The Kubernetes {{< glossary_tooltip text="scheduler" term_id="kube-scheduler" >}} ensures that
there are enough resources for all the Pods on a Node. The scheduler checks that the sum
@@ -77,7 +77,7 @@ The [imagePullPolicy](/docs/concepts/containers/images/#updating-images) and the
- `imagePullPolicy: IfNotPresent`: the image is pulled only if it is not already present locally.
- `imagePullPolicy: Always`: the image is pulled every time the pod is started.
- `imagePullPolicy: Always`: every time the kubelet launches a container, the kubelet queries the container image registry to resolve the name to an image digest. If the kubelet has a container image with that exact digest cached locally, the kubelet uses its cached image; otherwise, the kubelet downloads (pulls) the image with the resolved digest, and uses that image to launch the container.
- `imagePullPolicy` is omitted and either the image tag is `:latest` or it is omitted: `Always` is applied.
@@ -580,7 +580,7 @@ spec:
- name: foo
secret:
secretName: mysecret
defaultMode: 256
defaultMode: 0400
```
Then, the secret will be mounted on `/etc/foo` and all the files created by the
@@ -590,6 +590,38 @@ Note that the JSON spec doesn't support octal notation, so use the value 256 for
0400 permissions. If you use YAML instead of JSON for the Pod, you can use octal
notation to specify permissions in a more natural way.
Note if you `kubectl exec` into the Pod, you need to follow the symlink to find
the expected file mode. For example,
Check the secrets file mode on the pod.
```
kubectl exec mypod -it sh
cd /etc/foo
ls -l
```
The output is similar to this:
```
total 0
lrwxrwxrwx 1 root root 15 May 18 00:18 password -> ..data/password
lrwxrwxrwx 1 root root 15 May 18 00:18 username -> ..data/username
```
Follow the symlink to find the correct file mode.
```
cd /etc/foo/..data
ls -l
```
The output is similar to this:
```
total 8
-r-------- 1 root root 12 May 18 00:18 password
-r-------- 1 root root 5 May 18 00:18 username
```
You can also use mapping, as in the previous example, and specify different
permissions for different files like this:
@@ -612,12 +644,12 @@ spec:
items:
- key: username
path: my-group/my-username
mode: 511
mode: 0777
```
In this case, the file resulting in `/etc/foo/my-group/my-username` will have
permission value of `0777`. Owing to JSON limitations, you must specify the mode
in decimal notation.
permission value of `0777`. If you use JSON, owing to JSON limitations, you
must specify the mode in decimal notation, `511`.
Note that this permission value might be displayed in decimal notation if you
read it later.
@@ -83,11 +83,13 @@ For example:
#### Support traffic shaping
**Experimental Feature**
The CNI networking plugin also supports pod ingress and egress traffic shaping. You can use the official [bandwidth](https://github.com/containernetworking/plugins/tree/master/plugins/meta/bandwidth)
plugin offered by the CNI plugin team or use your own plugin with bandwidth control functionality.
If you want to enable traffic shaping support, you must add a `bandwidth` plugin to your CNI configuration file
(default `/etc/cni/net.d`).
If you want to enable traffic shaping support, you must add the `bandwidth` plugin to your CNI configuration file
(default `/etc/cni/net.d`) and ensure that the binary is included in your CNI bin dir (default `/opt/cni/bin`).
```json
{
@@ -51,11 +51,11 @@ You can list the current namespaces in a cluster using:
kubectl get namespace
```
```
NAME STATUS AGE
default Active 1d
kube-system Active 1d
kube-public Active 1d
kube-node-lease Active 1d
NAME STATUS AGE
default Active 1d
kube-node-lease Active 1d
kube-public Active 1d
kube-system Active 1d
```
Kubernetes starts with three initial namespaces:
@@ -374,6 +374,8 @@ several security mechanisms.
{{< codenew file="policy/restricted-psp.yaml" >}}
See [Pod Security Standards](/docs/concepts/security/pod-security-standards/#policy-instantiation) for more examples.
## Policy Reference
### Privileged
@@ -633,6 +635,8 @@ Refer to the [Sysctl documentation](
{{% capture whatsnext %}}
See [Pod Security Standards](/docs/concepts/security/pod-security-standards/) for policy recommendations.
Refer to [Pod Security Policy Reference](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#podsecuritypolicy-v1beta1-policy) for the api details.
{{% /capture %}}
@@ -77,21 +77,10 @@ A toleration "matches" a taint if the keys are the same and the effects are the
There are two special cases:
* An empty `key` with operator `Exists` matches all keys, values and effects which means this
An empty `key` with operator `Exists` matches all keys, values and effects which means this
will tolerate everything.
```yaml
tolerations:
- operator: "Exists"
```
* An empty `effect` matches all effects with key `key`.
```yaml
tolerations:
- key: "key"
operator: "Exists"
```
An empty `effect` matches all effects with key `key`.
{{< /note >}}
@@ -0,0 +1,300 @@
---
reviewers:
- tallclair
title: Pod Security Standards
content_template: templates/concept
weight: 10
---
{{% capture overview %}}
Security settings for Pods are typically applied by using [security
contexts](/docs/tasks/configure-pod-container/security-context/). Security Contexts allow for the
definition of privilege and access controls on a per-Pod basis.
The enforcement and policy-based definition of cluster requirements of security contexts has
previously been achieved using [Pod Security Policy](/docs/concepts/policy/pod-security-policy/). A
_Pod Security Policy_ is a cluster-level resource that controls security sensitive aspects of the
Pod specification.
However, numerous means of policy enforcement have arisen that augment or replace the use of
PodSecurityPolicy. The intent of this page is to detail recommended Pod security profiles, decoupled
from any specific instantiation.
{{% /capture %}}
{{% capture body %}}
## Policy Types
There is an immediate need for base policy definitions to broadly cover the security spectrum. These
should range from highly restricted to highly flexible:
- **_Privileged_** - Unrestricted policy, providing the widest possible level of permissions. This
policy allows for known privilege escalations.
- **_Baseline/Default_** - Minimally restrictive policy while preventing known privilege
escalations. Allows the default (minimally specified) Pod configuration.
- **_Restricted_** - Heavily restricted policy, following current Pod hardening best practices.
## Policies
### Privileged
The Privileged policy is purposely-open, and entirely unrestricted. This type of policy is typically
aimed at system- and infrastructure-level workloads managed by privileged, trusted users.
The privileged policy is defined by an absence of restrictions. For blacklist-oriented enforcement
mechanisms (such as gatekeeper), the privileged profile may be an absence of applied constraints
rather than an instantiated policy. In contrast, for a whitelist oriented mechanism (such as Pod
Security Policy) the privileged policy should enable all controls (disable all restrictions).
### Baseline/Default
The Baseline/Default policy is aimed at ease of adoption for common containerized workloads while
preventing known privilege escalations. This policy is targeted at application operators and
developers of non-critical applications. The following listed controls should be
enforced/disallowed:
<table>
<caption style="display:none">Baseline policy specification</caption>
<tbody>
<tr>
<td><strong>Control</strong></td>
<td><strong>Policy</strong></td>
</tr>
<tr>
<td>Host Namespaces</td>
<td>
Sharing the host namespaces must be disallowed.<br>
<br><b>Restricted Fields:</b><br>
spec.hostNetwork<br>
spec.hostPID<br>
spec.hostIPC<br>
<br><b>Allowed Values:</b> false<br>
</td>
</tr>
<tr>
<td>Privileged Containers</td>
<td>
Privileged Pods disable most security mechanisms and must be disallowed.<br>
<br><b>Restricted Fields:</b><br>
spec.containers[*].securityContext.privileged<br>
spec.initContainers[*].securityContext.privileged<br>
<br><b>Allowed Values:</b> false, undefined/nil<br>
</td>
</tr>
<tr>
<td>Capabilities</td>
<td>
Adding additional capabilities beyond the <a href="https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities">default set</a> must be disallowed.<br>
<br><b>Restricted Fields:</b><br>
spec.containers[*].securityContext.capabilities.add<br>
spec.initContainers[*].securityContext.capabilities.add<br>
<br><b>Allowed Values:</b> empty (optionally whitelisted defaults)<br>
</td>
</tr>
<tr>
<td>HostPath Volumes</td>
<td>
HostPath volumes must be forbidden.<br>
<br><b>Restricted Fields:</b><br>
spec.volumes[*].hostPath<br>
<br><b>Allowed Values:</b> undefined/nil<br>
</td>
</tr>
<tr>
<td>Host Ports</td>
<td>
HostPorts should be disallowed, or at minimum restricted to a whitelist.<br>
<br><b>Restricted Fields:</b><br>
spec.containers[*].ports[*].hostPort<br>
spec.initContainers[*].ports[*].hostPort<br>
<br><b>Allowed Values:</b> 0, undefined, (whitelisted)<br>
</td>
</tr>
<tr>
<td>AppArmor <em>(optional)</em></td>
<td>
On supported hosts, the `runtime/default` AppArmor profile is applied by default. The default policy should prevent overriding or disabling the policy, or restrict overrides to a whitelisted set of profiles.<br>
<br><b>Restricted Fields:</b><br>
metadata.annotations['container.apparmor.security.beta.kubernetes.io/*']<br>
<br><b>Allowed Values:</b> runtime/default, undefined<br>
</td>
</tr>
<tr>
<td>SELinux <em>(optional)</em></td>
<td>
Setting custom SELinux options should be disallowed.<br>
<br><b>Restricted Fields:</b><br>
spec.securityContext.seLinuxOptions<br>
spec.containers[*].securityContext.seLinuxOptions<br>
spec.initContainers[*].securityContext.seLinuxOptions<br>
<br><b>Allowed Values:</b> undefined/nil<br>
</td>
</tr>
</tbody>
</table>
### Restricted
The Restricted policy is aimed at enforcing current Pod hardening best practices, at the expense of
some compatibility. It is targeted at operators and developers of security-critical applications, as
well as lower-trust users.The following listed controls should be enforced/disallowed:
<table>
<caption style="display:none">Restricted policy specification</caption>
<tbody>
<tr>
<td><strong>Control</strong></td>
<td><strong>Policy</strong></td>
</tr>
<tr>
<td colspan="2"><em>Everything from the default profile.</em></td>
</tr>
<tr>
<td>Volume Types</td>
<td>
In addition to restricting HostPath volumes, the restricted profile limits usage of non-core volume types to those defined through PersistentVolumes.<br>
<br><b>Restricted Fields:</b><br>
spec.volumes[*].hostPath<br>
spec.volumes[*].gcePersistentDisk<br>
spec.volumes[*].awsElasticBlockStore<br>
spec.volumes[*].gitRepo<br>
spec.volumes[*].nfs<br>
spec.volumes[*].iscsi<br>
spec.volumes[*].glusterfs<br>
spec.volumes[*].rbd<br>
spec.volumes[*].flexVolume<br>
spec.volumes[*].cinder<br>
spec.volumes[*].cephFS<br>
spec.volumes[*].flocker<br>
spec.volumes[*].fc<br>
spec.volumes[*].azureFile<br>
spec.volumes[*].vsphereVolume<br>
spec.volumes[*].quobyte<br>
spec.volumes[*].azureDisk<br>
spec.volumes[*].portworxVolume<br>
spec.volumes[*].scaleIO<br>
spec.volumes[*].storageos<br>
spec.volumes[*].csi<br>
<br><b>Allowed Values:</b> undefined/nil<br>
</td>
</tr>
<tr>
<td>Privilege Escalation</td>
<td>
Privilege escalation to root should not be allowed.<br>
<br><b>Restricted Fields:</b><br>
spec.containers[*].securityContext.privileged<br>
spec.initContainers[*].securityContext.privileged<br>
<br><b>Allowed Values:</b> false, undefined/nil<br>
</td>
</tr>
<tr>
<td>Running as Non-root</td>
<td>
Containers must be required to run as non-root users.<br>
<br><b>Restricted Fields:</b><br>
spec.securityContext.runAsNonRoot<br>
spec.containers[*].securityContext.runAsNonRoot<br>
spec.initContainers[*].securityContext.runAsNonRoot<br>
<br><b>Allowed Values:</b> true<br>
</td>
</tr>
<tr>
<td>Non-root groups <em>(optional)</em></td>
<td>
Containers should be forbidden from running with a root primary or supplementary GID.<br>
<br><b>Restricted Fields:</b><br>
spec.securityContext.runAsGroup<br>
spec.securityContext.supplementalGroups[*]<br>
spec.securityContext.fsGroup<br>
spec.containers[*].securityContext.runAsGroup<br>
spec.containers[*].securityContext.supplementalGroups[*]<br>
spec.containers[*].securityContext.fsGroup<br>
spec.initContainers[*].securityContext.runAsGroup<br>
spec.initContainers[*].securityContext.supplementalGroups[*]<br>
spec.initContainers[*].securityContext.fsGroup<br>
<br><b>Allowed Values:</b><br>
non-zero<br>
undefined / nil (except for `*.runAsGroup`)<br>
</td>
</tr>
<tr>
<td>Seccomp</td>
<td>
The runtime/default seccomp profile must be required, or allow additional whitelisted values.<br>
<br><b>Restricted Fields:</b><br>
metadata.annotations['seccomp.security.alpha.kubernetes.io/pod']<br>
metadata.annotations['container.seccomp.security.alpha.kubernetes.io/*']<br>
<br><b>Allowed Values:</b><br>
runtime/default<br>
undefined (container annotation)<br>
</td>
</tr>
</tbody>
</table>
## Policy Instantiation
Decoupling policy definition from policy instantiation allows for a common understanding and
consistent language of policies across clusters, independent of the underlying enforcement
mechanism.
As mechanisms mature, they will be defined below on a per-policy basis. The methods of enforcement
of individual policies are not defined here.
[**PodSecurityPolicy**](/docs/concepts/policy/pod-security-policy/)
- [Privileged](https://raw.githubusercontent.com/kubernetes/website/master/content/en/examples/policy/privileged-psp.yaml)
- [Baseline](https://raw.githubusercontent.com/kubernetes/website/master/content/en/examples/policy/baseline-psp.yaml)
- [Restricted](https://raw.githubusercontent.com/kubernetes/website/master/content/en/examples/policy/restricted-psp.yaml)
## FAQ
### Why isn't there a profile between privileged and default?
The three profiles defined here have a clear linear progression from most secure (restricted) to least
secure (privileged), and cover a broad set of workloads. Privileges required above the baseline
policy are typically very application specific, so we do not offer a standard profile in this
niche. This is not to say that the privileged profile should always be used in this case, but that
policies in this space need to be defined on a case-by-case basis.
SIG Auth may reconsider this position in the future, should a clear need for other profiles arise.
### What's the difference between a security policy and a security context?
[Security Contexts](/docs/tasks/configure-pod-container/security-context/) configure Pods and
Containers at runtime. Security contexts are defined as part of the Pod and container specifications
in the Pod manifest, and represent parameters to the container runtime.
Security policies are control plane mechanisms to enforce specific settings in the Security Context,
as well as other parameters outside the Security Contex. As of February 2020, the current native
solution for enforcing these security policies is [Pod Security
Policy](/docs/concepts/policy/pod-security-policy/) - a mechanism for centrally enforcing security
policy on Pods across a cluster. Other alternatives for enforcing security policy are being
developed in the Kubernetes ecosystem, such as [OPA
Gatekeeper](https://github.com/open-policy-agent/gatekeeper).
### What profiles should I apply to my Windows Pods?
Windows in Kubernetes has some limitations and differentiators from standard Linux-based
workloads. Specifically, the Pod SecurityContext fields [have no effect on
Windows](/docs/setup/production-environment/windows/intro-windows-in-kubernetes/#v1-podsecuritycontext). As
such, no standardized Pod Security profiles currently exists.
### What about sandboxed Pods?
There is not currently an API standard that controls whether a Pod is considered sandboxed or
not. Sandbox Pods may be identified by the use of a sandboxed runtime (such as gVisor or Kata
Containers), but there is no standard definition of what a sandboxed runtime is.
The protections necessary for sandboxed workloads can differ from others. For example, the need to
restrict privileged permissions is lessened when the workload is isolated from the underlying
kernel. This allows for workloads requiring heightened permissions to still be isolated.
Additionally, the protection of sandboxed workloads is highly dependent on the method of
sandboxing. As such, no single recommended policy is recommended for all sandboxed workloads.
{{% /capture %}}
@@ -19,7 +19,7 @@ By default, Docker uses host-private networking, so containers can talk to other
Coordinating port allocations across multiple developers or teams that provide containers is very difficult to do at scale, and exposes users to cluster-level issues outside of their control. Kubernetes assumes that pods can communicate with other pods, regardless of which host they land on. Kubernetes gives every pod its own cluster-private IP address, so you do not need to explicitly create links between pods or map container ports to host ports. This means that containers within a Pod can all reach each other's ports on localhost, and all pods in a cluster can see each other without NAT. The rest of this document elaborates on how you can run reliable services on such a networking model.
This guide uses a simple nginx server to demonstrate proof of concept. The same principles are embodied in a more complete [Jenkins CI application](https://kubernetes.io/blog/2015/07/strong-simple-ssl-for-kubernetes).
This guide uses a simple nginx server to demonstrate proof of concept.
{{% /capture %}}
@@ -254,7 +254,7 @@ options ndots:5
### Feature availability
The availability of Pod DNS Config and DNS Policy "`None`"" is shown as below.
The availability of Pod DNS Config and DNS Policy "`None`" is shown as below.
| k8s version | Feature support |
| :---------: |:-----------:|
@@ -125,7 +125,7 @@ That introduces the following issues:
scheduler instead of the DaemonSet controller, by adding the `NodeAffinity` term
to the DaemonSet pods, instead of the `.spec.nodeName` term. The default
scheduler is then used to bind the pod to the target host. If node affinity of
the DaemonSet pod already exists, it is replaced. The DaemonSet controller only
the DaemonSet pod already exists, it is replaced (the original node affinity was taken into account before selecting the target host). The DaemonSet controller only
performs these operations when creating or modifying DaemonSet pods, and no
changes are made to the `spec.template` of the DaemonSet.
@@ -472,7 +472,7 @@ starts a Spark master controller (see [spark example](https://github.com/kuberne
driver, and then cleans up.
An advantage of this approach is that the overall process gets the completion guarantee of a Job
object, but complete control over what Pods are created and how work is assigned to them.
object, but maintains complete control over what Pods are created and how work is assigned to them.
## Cron Jobs {#cron-jobs}
@@ -1079,37 +1079,37 @@ In order from most secure to least secure, the approaches are:
2. Grant a role to the "default" service account in a namespace
If an application does not specify a `serviceAccountName`, it uses the "default" service account.
If an application does not specify a `serviceAccountName`, it uses the "default" service account.
{{< note >}}
Permissions given to the "default" service account are available to any pod
in the namespace that does not specify a `serviceAccountName`.
{{< /note >}}
{{< note >}}
Permissions given to the "default" service account are available to any pod
in the namespace that does not specify a `serviceAccountName`.
{{< /note >}}
For example, grant read-only permission within "my-namespace" to the "default" service account:
For example, grant read-only permission within "my-namespace" to the "default" service account:
```shell
kubectl create rolebinding default-view \
--clusterrole=view \
--serviceaccount=my-namespace:default \
--namespace=my-namespace
```
```shell
kubectl create rolebinding default-view \
--clusterrole=view \
--serviceaccount=my-namespace:default \
--namespace=my-namespace
```
Many [add-ons](/docs/concepts/cluster-administration/addons/) run as the
"default" service account in the `kube-system` namespace.
To allow those add-ons to run with super-user access, grant cluster-admin
permissions to the "default" service account in the `kube-system` namespace.
Many [add-ons](/docs/concepts/cluster-administration/addons/) run as the
"default" service account in the `kube-system` namespace.
To allow those add-ons to run with super-user access, grant cluster-admin
permissions to the "default" service account in the `kube-system` namespace.
{{< caution >}}
Enabling this means the `kube-system` namespace contains Secrets
that grant super-user access to your cluster's API.
{{< /caution >}}
{{< caution >}}
Enabling this means the `kube-system` namespace contains Secrets
that grant super-user access to your cluster's API.
{{< /caution >}}
```shell
kubectl create clusterrolebinding add-on-cluster-admin \
--clusterrole=cluster-admin \
--serviceaccount=kube-system:default
```
```shell
kubectl create clusterrolebinding add-on-cluster-admin \
--clusterrole=cluster-admin \
--serviceaccount=kube-system:default
```
3. Grant a role to all service accounts in a namespace
@@ -119,7 +119,7 @@ track=stable
- **CPU requirement (cores)** and **Memory requirement (MiB)**: You can specify the minimum [resource limits](/docs/tasks/configure-pod-container/limit-range/) for the container. By default, Pods run with unbounded CPU and memory limits.
- **Run command** and **Run command arguments**: By default, your containers run the specified Docker image's default [entrypoint command](/docs/user-guide/containers/#containers-and-commands). You can use the command options and arguments to override the default.
- **Run command** and **Run command arguments**: By default, your containers run the specified Docker image's default [entrypoint command](/docs/tasks/inject-data-application/define-command-argument-container/). You can use the command options and arguments to override the default.
- **Run as privileged**: This setting determines whether processes in [privileged containers](/docs/user-guide/pods/#privileged-mode-for-pod-containers) are equivalent to processes running as root on the host. Privileged containers can make use of capabilities like manipulating the network stack and accessing devices.
+6 -2
View File
@@ -47,7 +47,9 @@ This tutorial provides a container image that uses NGINX to echo back all the re
{{< kat-button >}}
{{< note >}}If you installed Minikube locally, run `minikube start`.{{< /note >}}
{{< note >}}
If you installed Minikube locally, run `minikube start`.
{{< /note >}}
2. Open the Kubernetes dashboard in a browser:
@@ -113,7 +115,9 @@ Pod runs a Container based on the provided Docker image.
kubectl config view
```
{{< note >}}For more information about `kubectl`commands, see the [kubectl overview](/docs/user-guide/kubectl-overview/).{{< /note >}}
{{< note >}}
For more information about `kubectl`commands, see the [kubectl overview](/docs/user-guide/kubectl-overview/).
{{< /note >}}
## Create a Service
@@ -0,0 +1,74 @@
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
name: baseline
annotations:
# Optional: Allow the default AppArmor profile, requires setting the default.
apparmor.security.beta.kubernetes.io/allowedProfileNames: 'runtime/default'
apparmor.security.beta.kubernetes.io/defaultProfileName: 'runtime/default'
# Optional: Allow the default seccomp profile, requires setting the default.
seccomp.security.alpha.kubernetes.io/allowedProfileNames: 'docker/default,runtime/default,unconfined'
seccomp.security.alpha.kubernetes.io/defaultProfileName: 'unconfined'
spec:
privileged: false
# The moby default capability set, defined here:
# https://github.com/moby/moby/blob/0a5cec2833f82a6ad797d70acbf9cbbaf8956017/oci/caps/defaults.go#L6-L19
allowedCapabilities:
- 'CHOWN'
- 'DAC_OVERRIDE'
- 'FSETID'
- 'FOWNER'
- 'MKNOD'
- 'NET_RAW'
- 'SETGID'
- 'SETUID'
- 'SETFCAP'
- 'SETPCAP'
- 'NET_BIND_SERVICE'
- 'SYS_CHROOT'
- 'KILL'
- 'AUDIT_WRITE'
# Allow all volume types except hostpath
volumes:
# 'core' volume types
- 'configMap'
- 'emptyDir'
- 'projected'
- 'secret'
- 'downwardAPI'
# Assume that persistentVolumes set up by the cluster admin are safe to use.
- 'persistentVolumeClaim'
# Allow all other non-hostpath volume types.
- 'awsElasticBlockStore'
- 'azureDisk'
- 'azureFile'
- 'cephFS'
- 'cinder'
- 'csi'
- 'fc'
- 'flexVolume'
- 'flocker'
- 'gcePersistentDisk'
- 'gitRepo'
- 'glusterfs'
- 'iscsi'
- 'nfs'
- 'photonPersistentDisk'
- 'portworxVolume'
- 'quobyte'
- 'rbd'
- 'scaleIO'
- 'storageos'
- 'vsphereVolume'
hostNetwork: false
hostIPC: false
hostPID: false
readOnlyRootFilesystem: false
runAsUser:
rule: 'RunAsAny'
seLinux:
rule: 'RunAsAny'
supplementalGroups:
rule: 'RunAsAny'
fsGroup:
rule: 'RunAsAny'
+1 -1
View File
@@ -97,7 +97,7 @@ class: training
</h5>
<p>The Certified Kubernetes Administrator (CKA) program provides assurance that CKAs have the skills, knowledge, and competency to perform the responsibilities of Kubernetes administrators.</p>
<br>
<a href=https://training.linuxfoundation.org/certification/certified-kubernetes-administrator-cka/" target="_blank" class="button">Go to Certification</a>
<a href="https://training.linuxfoundation.org/certification/certified-kubernetes-administrator-cka/" target="_blank" class="button">Go to Certification</a>
</center>
</div>
</div>