whentimeslows/website
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #30422 from PushkarJ/psa-tutorial

Browse Source 
Tutorials for Pod Security Admission
This commit is contained in:
Kubernetes Prow Robot
2021-12-08 16:30:10 -08:00
committed by GitHub
parent 2740d85747 d1e25451d3
commit aa848abc62
6 changed files with 591 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
content/en/docs/tutorials/_index.md
+5
View File
@@ -57,6 +57,11 @@ Before walking through each tutorial, you may want to bookmark the
* [Using Source IP](/docs/tutorials/services/source-ip/)
## Security
* [Apply Pod Security Standards at Cluster level](/docs/tutorials/security/cluster-level-pss/)
* [Apply Pod Security Standards at Namespace level](/docs/tutorials/security/ns-level-pss/)
## {{% heading "whatsnext" %}}
If you would like to write a tutorial, see
content/en/docs/tutorials/security/_index.md
+5
View File
@@ -0,0 +1,5 @@
---
title: "Security"
weight: 40
---
content/en/docs/tutorials/security/cluster-level-pss.md
+324
View File
@@ -0,0 +1,324 @@
---
title: Apply Pod Security Standards at the Cluster Level
content_type: tutorial
weight: 10
---
{{% alert title="Note" %}}
This tutorial applies only for new clusters.
{{% /alert %}}
Pod Security admission (PSA) is enabled by default in v1.23 and later, as it has
[graduated to beta](/blog/2021/12/09/pod-security-admission-beta/).
Pod Security
is an admission controller that carries out checks against the Kubernetes
[Pod Security Standards](docs/concepts/security/pod-security-standards/) when new pods are
created. This tutorial shows you how to enforce the `baseline` Pod Security
Standard at the cluster level which applies a standard configuration
to all namespaces in a cluster.
To apply Pod Security Standards to specific namespaces, refer to [Apply Pod Security Standards at the namespace level](/docs/tutorials/security/ns-level-pss).
## {{% heading "prerequisites" %}}
Install the following on your workstation:
- [KinD](https://kind.sigs.k8s.io/docs/user/quick-start/#installation)
- [kubectl](https://kubernetes.io/docs/tasks/tools/)
## Choose the right Pod Security Standard to apply
[Pod Security Admission](/docs/concepts/security/pod-security-admission/)
lets you apply built-in [Pod Security Standards](/docs/concepts/security/pod-security-standards/)
with the following modes: `enforce`, `audit`, and `warn`.
To gather information that helps you to choose the Pod Security Standards
that are most appropriate for your configuration, do the following:
1. Create a cluster with no Pod Security Standards applied:
```shell
kind create cluster --name psa-wo-cluster-pss --image kindest/node:v1.23.0
```
The output is similar to this:
```
Creating cluster "psa-wo-cluster-pss" ...
✓ Ensuring node image (kindest/node:v1.23.0) 🖼
✓ Preparing nodes 📦
✓ Writing configuration 📜
✓ Starting control-plane 🕹️
✓ Installing CNI 🔌
✓ Installing StorageClass 💾
Set kubectl context to "kind-psa-wo-cluster-pss"
You can now use your cluster with:
kubectl cluster-info --context kind-psa-wo-cluster-pss
Thanks for using kind! 😊
```
1. Set the kubectl context to the new cluster:
```shell
kubectl cluster-info --context kind-psa-wo-cluster-pss
```
The output is similar to this:
```
Kubernetes control plane is running at https://127.0.0.1:61350
CoreDNS is running at https://127.0.0.1:61350/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
```
1. Get a list of namespaces in the cluster:
```shell
kubectl get ns
```
The output is similar to this:
```
NAME STATUS AGE
default Active 9m30s
kube-node-lease Active 9m32s
kube-public Active 9m32s
kube-system Active 9m32s
local-path-storage Active 9m26s
```
1. Use `--dry-run=server` to understand what happens when different Pod Security Standards
are applied:
1. Privileged
```shell
kubectl label --dry-run=server --overwrite ns --all \
pod-security.kubernetes.io/enforce=privileged
```
The output is similar to this:
```
namespace/default labeled
namespace/kube-node-lease labeled
namespace/kube-public labeled
namespace/kube-system labeled
namespace/local-path-storage labeled
```
2. Baseline
```shell
kubectl label --dry-run=server --overwrite ns --all \
pod-security.kubernetes.io/enforce=baseline
```
The output is similar to this:
```
namespace/default labeled
namespace/kube-node-lease labeled
namespace/kube-public labeled
Warning: existing pods in namespace "kube-system" violate the new PodSecurity enforce level "baseline:latest"
Warning: etcd-psa-wo-cluster-pss-control-plane (and 3 other pods): host namespaces, hostPath volumes
Warning: kindnet-vzj42: non-default capabilities, host namespaces, hostPath volumes
Warning: kube-proxy-m6hwf: host namespaces, hostPath volumes, privileged
namespace/kube-system labeled
namespace/local-path-storage labeled
```
3. Restricted
```shell
kubectl label --dry-run=server --overwrite ns --all \
pod-security.kubernetes.io/enforce=restricted
```
The output is similar to this:
```
namespace/default labeled
namespace/kube-node-lease labeled
namespace/kube-public labeled
Warning: existing pods in namespace "kube-system" violate the new PodSecurity enforce level "restricted:latest"
Warning: coredns-7bb9c7b568-hsptc (and 1 other pod): unrestricted capabilities, runAsNonRoot != true, seccompProfile
Warning: etcd-psa-wo-cluster-pss-control-plane (and 3 other pods): host namespaces, hostPath volumes, allowPrivilegeEscalation != false, unrestricted capabilities, restricted volume types, runAsNonRoot != true
Warning: kindnet-vzj42: non-default capabilities, host namespaces, hostPath volumes, allowPrivilegeEscalation != false, unrestricted capabilities, restricted volume types, runAsNonRoot != true, seccompProfile
Warning: kube-proxy-m6hwf: host namespaces, hostPath volumes, privileged, allowPrivilegeEscalation != false, unrestricted capabilities, restricted volume types, runAsNonRoot != true, seccompProfile
namespace/kube-system labeled
Warning: existing pods in namespace "local-path-storage" violate the new PodSecurity enforce level "restricted:latest"
Warning: local-path-provisioner-d6d9f7ffc-lw9lh: allowPrivilegeEscalation != false, unrestricted capabilities, runAsNonRoot != true, seccompProfile
namespace/local-path-storage labeled
```
From the previous output, you'll notice that applying the `privileged` Pod Security Standard shows no warnings
for any namespaces. However, `baseline` and `restricted` standards both have
warnings, specifically in the `kube-system` namespace.
## Set modes, versions and standards
In this section, you apply the following Pod Security Standards to the `latest` version:
* `baseline` standard in `enforce` mode.
* `restricted` standard in `warn` and `audit` mode.
The `baseline` Pod Security Standard provides a convenient
middle ground that allows keeping the exemption list short and prevents known
privilege escalations.
Additionally, to prevent pods from failing in `kube-system`, you'll exempt the namespace
from having Pod Security Standards applied.
When you implement Pod Security Admission in your own environment, consider the
following:
1. Based on the risk posture applied to a cluster, a stricter Pod Security
Standard like `restricted` might be a better choice.
1. Exempting the `kube-system` namespace allows pods to run as
`privileged` in this namespace. For real world use, the Kubernetes project
strongly recommends that you apply strict RBAC
policies that limit access to `kube-system`, following the principle of least
privilege.
To implement the preceding standards, do the following:
1. Create a configuration file that can be consumed by the Pod Security
Admission Controller to implement these Pod Security Standards:
```
mkdir -p /tmp/pss
cat <<EOF > /tmp/pss/cluster-level-pss.yaml
apiVersion: apiserver.config.k8s.io/v1
kind: AdmissionConfiguration
plugins:
- name: PodSecurity
configuration:
apiVersion: pod-security.admission.config.k8s.io/v1beta1
kind: PodSecurityConfiguration
defaults:
enforce: "baseline"
enforce-version: "latest"
audit: "restricted"
audit-version: "latest"
warn: "restricted"
warn-version: "latest"
exemptions:
usernames: []
runtimeClasses: []
namespaces: [kube-system]
EOF
```
1. Configure the API server to consume this file during cluster creation:
```
cat <<EOF > /tmp/pss/cluster-config.yaml
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
kubeadmConfigPatches:
- |
kind: ClusterConfiguration
apiServer:
extraArgs:
admission-control-config-file: /etc/config/cluster-level-pss.yaml
extraVolumes:
- name: accf
hostPath: /etc/config
mountPath: /etc/config
readOnly: false
pathType: "DirectoryOrCreate"
extraMounts:
- hostPath: /tmp/pss
containerPath: /etc/config
# optional: if set, the mount is read-only.
# default false
readOnly: false
# optional: if set, the mount needs SELinux relabeling.
# default false
selinuxRelabel: false
# optional: set propagation mode (None, HostToContainer or Bidirectional)
# see https://kubernetes.io/docs/concepts/storage/volumes/#mount-propagation
# default None
propagation: None
EOF
```
{{<note>}}
If you use Docker Desktop with KinD on macOS, you can
add `/tmp` as a Shared Directory under the menu item
**Preferences > Resources > File Sharing**.
{{</note>}}
1. Create a cluster that uses Pod Security Admission to apply
these Pod Security Standards:
```shell
kind create cluster --name psa-with-cluster-pss --image kindest/node:v1.23.0 --config /tmp/pss/cluster-config.yaml
```
The output is similar to this:
```
Creating cluster "psa-with-cluster-pss" ...
✓ Ensuring node image (kindest/node:v1.23.0) 🖼
✓ Preparing nodes 📦
✓ Writing configuration 📜
✓ Starting control-plane 🕹️
✓ Installing CNI 🔌
✓ Installing StorageClass 💾
Set kubectl context to "kind-psa-with-cluster-pss"
You can now use your cluster with:
kubectl cluster-info --context kind-psa-with-cluster-pss
Have a question, bug, or feature request? Let us know! https://kind.sigs.k8s.io/#community 🙂
```
1. Point kubectl to the cluster
```shell
kubectl cluster-info --context kind-psa-with-cluster-pss
```
The output is similar to this:
```
Kubernetes control plane is running at https://127.0.0.1:63855
CoreDNS is running at https://127.0.0.1:63855/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
```
1. Create the following Pod specification for a minimal configuration in the default namespace:
```
cat <<EOF > /tmp/pss/nginx-pod.yaml
apiVersion: v1
kind: Pod
metadata:
name: nginx
spec:
containers:
- image: nginx
name: nginx
ports:
- containerPort: 80
EOF
```
1. Create the Pod in the cluster:
```shell
kubectl apply -f /tmp/pss/nginx-pod.yaml
```
The output is similar to this:
```
Warning: would violate PodSecurity "restricted:latest": allowPrivilegeEscalation != false (container "nginx" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "nginx" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or container "nginx" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container "nginx" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")
pod/nginx created
```
## Clean up
Run `kind delete cluster -name psa-with-cluster-pss` and
`kind delete cluster -name psa-wo-cluster-pss` to delete the clusters you
created.
## {{% heading "whatsnext" %}}
- Run a
[shell script](/examples/security/kind-with-cluster-level-baseline-pod-security.sh)
to perform all the preceding steps at once:
1. Create a Pod Security Standards based cluster level Configuration
2. Create a file to let API server consumes this configuration
3. Create a cluster that creates an API server with this configuration
4. Set kubectl context to this new cluster
5. Create a minimal pod yaml file
6. Apply this file to create a Pod in the new cluster
- [Pod Security Admission](/docs/concepts/security/pod-security-admission/)
- [Pod Security Standards](/docs/concepts/security/pod-security-standards/)
- [Apply Pod Security Standards at the namespace level](/docs/tutorials/security/ns-level-pss/)
content/en/docs/tutorials/security/ns-level-pss.md
+159
View File
@@ -0,0 +1,159 @@
---
title: Apply Pod Security Standards at the Namespace Level
content_type: tutorial
weight: 10
---
{{% alert title="Note" %}}
This tutorial applies only for new clusters.
{{% /alert %}}
Pod Security admission (PSA) is enabled by default in v1.23 and later, as it [graduated
to beta](/blog/2021/12/09/pod-security-admission-beta/). Pod Security Admission
is an admission controller that applies
[Pod Security Standards](docs/concepts/security/pod-security-standards/)
when pods are created. In this tutorial, you will enforce the `baseline` Pod Security Standard,
one namespace at a time.
You can also apply Pod Security Standards to multiple namespaces at once at the cluster
level. For instructions, refer to [Apply Pod Security Standards at the cluster level](/docs/tutorials/security/cluster-level-pss).
## {{% heading "prerequisites" %}}
Install the following on your workstation:
- [KinD](https://kind.sigs.k8s.io/docs/user/quick-start/#installation)
- [kubectl](https://kubernetes.io/docs/tasks/tools/)
## Create cluster
1. Create a `KinD` cluster as follows:
```shell
kind create cluster --name psa-ns-level --image kindest/node:v1.23.0
```
The output is similar to this:
```
Creating cluster "psa-ns-level" ...
✓ Ensuring node image (kindest/node:v1.23.0) 🖼
✓ Preparing nodes 📦
✓ Writing configuration 📜
✓ Starting control-plane 🕹️
✓ Installing CNI 🔌
✓ Installing StorageClass 💾
Set kubectl context to "kind-psa-ns-level"
You can now use your cluster with:
kubectl cluster-info --context kind-psa-ns-level
Not sure what to do next? 😅 Check out https://kind.sigs.k8s.io/docs/user/quick-start/
```
1. Set the kubectl context to the new cluster:
```shell
kubectl cluster-info --context kind-psa-ns-level
```
The output is similar to this:
```
Kubernetes control plane is running at https://127.0.0.1:50996
CoreDNS is running at https://127.0.0.1:50996/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
```
## Create a namespace
Create a new namespace called `example`:
```shell
kubectl create ns example
```
The output is similar to this:
```
namespace/example created
```
## Apply Pod Security Standards
1. Enable Pod Security Standards on this namespace using labels supported by
built-in Pod Security Admission. In this step we will warn on baseline pod
security standard as per the latest version (default value)
```shell
kubectl label --overwrite ns example \
pod-security.kubernetes.io/warn=baseline \
pod-security.kubernetes.io/warn-version=latest
```
2. Multiple pod security standards can be enabled on any namespace, using labels.
Following command will `enforce` the `baseline` Pod Security Standard, but
`warn` and `audit` for `restricted` Pod Security Standards as per the latest
version (default value)
```
kubectl label --overwrite ns example \
pod-security.kubernetes.io/enforce=baseline \
pod-security.kubernetes.io/enforce-version=latest \
pod-security.kubernetes.io/warn=restricted \
pod-security.kubernetes.io/warn-version=latest \
pod-security.kubernetes.io/audit=restricted \
pod-security.kubernetes.io/audit-version=latest
```
## Verify the Pod Security Standards
1. Create a minimal pod in `example` namespace:
```shell
cat <<EOF > /tmp/pss/nginx-pod.yaml
apiVersion: v1
kind: Pod
metadata:
name: nginx
spec:
containers:
- image: nginx
name: nginx
ports:
- containerPort: 80
EOF
```
1. Apply the pod spec to the cluster in `example` namespace:
```shell
kubectl apply -n example -f /tmp/pss/nginx-pod.yaml
```
The output is similar to this:
```
Warning: would violate PodSecurity "restricted:latest": allowPrivilegeEscalation != false (container "nginx" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "nginx" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or container "nginx" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container "nginx" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")
pod/nginx created
```
1. Apply the pod spec to the cluster in `default` namespace:
```shell
kubectl apply -n default -f /tmp/pss/nginx-pod.yaml
```
Output is similar to this:
```
pod/nginx created
```
The Pod Security Standards were applied only to the `example`
namespace. You could create the same Pod in the `default` namespace
with no warnings.
## Clean up
Run `kind delete cluster -name psa-ns-level` to delete the cluster created.
## {{% heading "whatsnext" %}}
- Run a
[shell script](/examples/security/kind-with-namespace-level-baseline-pod-security.sh)
to perform all the preceding steps all at once.
1. Create KinD cluster
2. Create new namespace
3. Apply `baseline` Pod Security Standard in `enforce` mode while applying
`restricted` Pod Security Standard also in `warn` and `audit` mode.
4. Create a new pod with the following pod security standards applied
- [Pod Security Admission](/docs/concepts/security/pod-security-admission/)
- [Pod Security Standards](/docs/concepts/security/pod-security-standards/)
- [Apply Pod Security Standards at the cluster level](/docs/tutorials/security/cluster-level-pss/)
content/en/examples/security/kind-with-cluster-level-baseline-pod-security.sh
+70
View File
@@ -0,0 +1,70 @@
#!/bin/sh
mkdir -p /tmp/pss
cat <<EOF > /tmp/pss/cluster-level-pss.yaml
apiVersion: apiserver.config.k8s.io/v1
kind: AdmissionConfiguration
plugins:
- name: PodSecurity
configuration:
apiVersion: pod-security.admission.config.k8s.io/v1beta1
kind: PodSecurityConfiguration
defaults:
enforce: "baseline"
enforce-version: "latest"
audit: "restricted"
audit-version: "latest"
warn: "restricted"
warn-version: "latest"
exemptions:
usernames: []
runtimeClasses: []
namespaces: [kube-system]
EOF
cat <<EOF > /tmp/pss/cluster-config.yaml
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
kubeadmConfigPatches:
- |
kind: ClusterConfiguration
apiServer:
extraArgs:
admission-control-config-file: /etc/config/cluster-level-pss.yaml
extraVolumes:
- name: accf
hostPath: /etc/config
mountPath: /etc/config
readOnly: false
pathType: "DirectoryOrCreate"
extraMounts:
- hostPath: /tmp/pss
containerPath: /etc/config
# optional: if set, the mount is read-only.
# default false
readOnly: false
# optional: if set, the mount needs SELinux relabeling.
# default false
selinuxRelabel: false
# optional: set propagation mode (None, HostToContainer or Bidirectional)
# see https://kubernetes.io/docs/concepts/storage/volumes/#mount-propagation
# default None
propagation: None
EOF
kind create cluster --name psa-with-cluster-pss --image kindest/node:v1.23.0 --config /tmp/pss/cluster-config.yaml
kubectl cluster-info --context kind-psa-with-cluster-pss
# Wait for 15 seconds (arbitrary) ServiceAccount Admission Controller to be available
sleep 15
cat <<EOF > /tmp/pss/nginx-pod.yaml
apiVersion: v1
kind: Pod
metadata:
name: nginx
spec:
containers:
- image: nginx
name: nginx
ports:
- containerPort: 80
EOF
kubectl apply -f /tmp/pss/nginx-pod.yaml
content/en/examples/security/kind-with-namespace-level-baseline-pod-security.sh
+28
View File
@@ -0,0 +1,28 @@
#!/bin/sh
# Until v1.23 is released, kind node image needs to be built from k/k master branch
# Ref: https://kind.sigs.k8s.io/docs/user/quick-start/#building-images
kind create cluster --name psa-ns-level --image kindest/node:v1.23.0
kubectl cluster-info --context kind-psa-ns-level
# Wait for 15 seconds (arbitrary) ServiceAccount Admission Controller to be available
sleep 15
kubectl create ns example
kubectl label --overwrite ns example \
pod-security.kubernetes.io/enforce=baseline \
pod-security.kubernetes.io/enforce-version=latest \
pod-security.kubernetes.io/warn=restricted \
pod-security.kubernetes.io/warn-version=latest \
pod-security.kubernetes.io/audit=restricted \
pod-security.kubernetes.io/audit-version=latest
cat <<EOF > /tmp/pss/nginx-pod.yaml
apiVersion: v1
kind: Pod
metadata:
name: nginx
spec:
containers:
- image: nginx
name: nginx
ports:
- containerPort: 80
EOF
kubectl apply -n example -f /tmp/pss/nginx-pod.yaml