Merge remote-tracking branch 'upstream/main' into dev-1.24
This commit is contained in:
Nate W
@@ -154,7 +154,7 @@ deployment.apps/my-deployment created
|
||||
persistentvolumeclaim/my-pvc created
|
||||
```
|
||||
|
||||
If you're interested in learning more about `kubectl`, go ahead and read [kubectl Overview](/docs/reference/kubectl/overview/).
|
||||
If you're interested in learning more about `kubectl`, go ahead and read [Command line tool (kubectl)](/docs/reference/kubectl/).
|
||||
|
||||
## Using labels effectively
|
||||
|
||||
|
||||
@@ -79,7 +79,7 @@ addressing, and it can be used in combination with other CNI plugins.
|
||||
|
||||
### CNI-Genie from Huawei
|
||||
|
||||
[CNI-Genie](https://github.com/cni-genie/CNI-Genie) is a CNI plugin that enables Kubernetes to [simultaneously have access to different implementations](https://github.com/cni-genie/CNI-Genie/blob/master/docs/multiple-cni-plugins/README.md#what-cni-genie-feature-1-multiple-cni-plugins-enables) of the [Kubernetes network model](/docs/concepts/cluster-administration/networking/#the-kubernetes-network-model) in runtime. This includes any implementation that runs as a [CNI plugin](https://github.com/containernetworking/cni#3rd-party-plugins), such as [Flannel](https://github.com/flannel-io/flannel#flannel), [Calico](https://projectcalico.docs.tigera.io/about/about-calico/), [Weave-net](https://www.weave.works/oss/net/).
|
||||
[CNI-Genie](https://github.com/cni-genie/CNI-Genie) is a CNI plugin that enables Kubernetes to [simultaneously have access to different implementations](https://github.com/cni-genie/CNI-Genie/blob/master/docs/multiple-cni-plugins/README.md#what-cni-genie-feature-1-multiple-cni-plugins-enables) of the [Kubernetes network model](/docs/concepts/cluster-administration/networking/#how-to-implement-the-kubernetes-networking-model) in runtime. This includes any implementation that runs as a [CNI plugin](https://github.com/containernetworking/cni#3rd-party-plugins), such as [Flannel](https://github.com/flannel-io/flannel#flannel), [Calico](https://projectcalico.docs.tigera.io/about/about-calico/), [Weave-net](https://www.weave.works/oss/net/).
|
||||
|
||||
CNI-Genie also supports [assigning multiple IP addresses to a pod](https://github.com/cni-genie/CNI-Genie/blob/master/docs/multiple-ips/README.md#feature-2-extension-cni-genie-multiple-ip-addresses-per-pod), each from a different CNI plugin.
|
||||
|
||||
|
||||
@@ -26,7 +26,7 @@ many core Kubernetes functions are now built using custom resources, making Kube
|
||||
Custom resources can appear and disappear in a running cluster through dynamic registration,
|
||||
and cluster admins can update custom resources independently of the cluster itself.
|
||||
Once a custom resource is installed, users can create and access its objects using
|
||||
[kubectl](/docs/reference/kubectl/overview/), just as they do for built-in resources like
|
||||
[kubectl](/docs/reference/kubectl/), just as they do for built-in resources like
|
||||
*Pods*.
|
||||
|
||||
## Custom controllers
|
||||
|
||||
@@ -23,7 +23,7 @@ The Kubernetes API lets you query and manipulate the state of API objects in Kub
|
||||
(for example: Pods, Namespaces, ConfigMaps, and Events).
|
||||
|
||||
Most operations can be performed through the
|
||||
[kubectl](/docs/reference/kubectl/overview/) command-line interface or other
|
||||
[kubectl](/docs/reference/kubectl/) command-line interface or other
|
||||
command-line tools, such as
|
||||
[kubeadm](/docs/reference/setup-tools/kubeadm/), which in turn use the
|
||||
API. However, you can also access the API directly using REST calls.
|
||||
|
||||
@@ -83,10 +83,19 @@ In the `.yaml` file for the Kubernetes object you want to create, you'll need to
|
||||
|
||||
The precise format of the object `spec` is different for every Kubernetes object, and contains nested fields specific to that object. The [Kubernetes API Reference](https://kubernetes.io/docs/reference/kubernetes-api/) can help you find the spec format for all of the objects you can create using Kubernetes.
|
||||
|
||||
For example, the reference for Pod details the [`spec` field](/docs/reference/kubernetes-api/workload-resources/pod-v1/#PodSpec)
|
||||
for a Pod in the API, and the reference for Deployment details the [`spec` field](/docs/reference/kubernetes-api/workload-resources/deployment-v1/#DeploymentSpec) for Deployments.
|
||||
In those API reference pages you'll see mention of PodSpec and DeploymentSpec. These names are implementation details of the Golang code that Kubernetes uses to implement its API.
|
||||
|
||||
For example, see the [`spec` field](/docs/reference/kubernetes-api/workload-resources/pod-v1/#PodSpec)
|
||||
for the Pod API reference.
|
||||
For each Pod, the `.spec` field specifies the pod and its desired state (such as the container image name for
|
||||
each container within that pod).
|
||||
Another example of an object specification is the
|
||||
[`spec` field](/docs/reference/kubernetes-api/workload-resources/stateful-set-v1/#StatefulSetSpec)
|
||||
for the StatefulSet API. For StatefulSet, the `.spec` field specifies the StatefulSet and
|
||||
its desired state.
|
||||
Within the `.spec` of a StatefulSet is a [template](/docs/concepts/workloads/pods/#pod-templates)
|
||||
for Pod objects. That template describes Pods that the StatefulSet controller will create in order to
|
||||
satisfy the StatefulSet specification.
|
||||
Different kinds of object can also have different `.status`; again, the API reference pages
|
||||
detail the structure of that `.status` field, and its content for each different type of object.
|
||||
|
||||
## {{% heading "whatsnext" %}}
|
||||
|
||||
|
||||
@@ -27,7 +27,8 @@ updates.
|
||||
## What is a Pod Security Policy?
|
||||
|
||||
A _Pod Security Policy_ is a cluster-level resource that controls security
|
||||
sensitive aspects of the pod specification. The [PodSecurityPolicy](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#podsecuritypolicy-v1beta1-policy) objects
|
||||
sensitive aspects of the pod specification. The
|
||||
[PodSecurityPolicy](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#podsecuritypolicy-v1beta1-policy) objects
|
||||
define a set of conditions that a pod must run with in order to be accepted into
|
||||
the system, as well as defaults for the related fields. They allow an
|
||||
administrator to control the following:
|
||||
@@ -54,10 +55,10 @@ administrator to control the following:
|
||||
|
||||
## Enabling Pod Security Policies
|
||||
|
||||
Pod security policy control is implemented as an optional [admission
|
||||
controller](/docs/reference/access-authn-authz/admission-controllers/#podsecuritypolicy).
|
||||
PodSecurityPolicies are enforced by [enabling the admission
|
||||
controller](/docs/reference/access-authn-authz/admission-controllers/#how-do-i-turn-on-an-admission-control-plug-in),
|
||||
Pod security policy control is implemented as an optional
|
||||
[admission controller](/docs/reference/access-authn-authz/admission-controllers/#podsecuritypolicy).
|
||||
PodSecurityPolicies are enforced by
|
||||
[enabling the admission controller](/docs/reference/access-authn-authz/admission-controllers/#how-do-i-turn-on-an-admission-control-plug-in),
|
||||
but doing so without authorizing any policies **will prevent any pods from being created** in the
|
||||
cluster.
|
||||
|
||||
@@ -69,9 +70,9 @@ controller.
|
||||
## Authorizing Policies
|
||||
|
||||
When a PodSecurityPolicy resource is created, it does nothing. In order to use
|
||||
it, the requesting user or target pod's [service
|
||||
account](/docs/tasks/configure-pod-container/configure-service-account/) must be
|
||||
authorized to use the policy, by allowing the `use` verb on the policy.
|
||||
it, the requesting user or target pod's
|
||||
[service account](/docs/tasks/configure-pod-container/configure-service-account/)
|
||||
must be authorized to use the policy, by allowing the `use` verb on the policy.
|
||||
|
||||
Most Kubernetes pods are not created directly by users. Instead, they are
|
||||
typically created indirectly as part of a
|
||||
@@ -132,6 +133,7 @@ subjects:
|
||||
If a `RoleBinding` (not a `ClusterRoleBinding`) is used, it will only grant
|
||||
usage for pods being run in the same namespace as the binding. This can be
|
||||
paired with system groups to grant access to all pods run in the namespace:
|
||||
|
||||
```yaml
|
||||
# Authorize all service accounts in a namespace:
|
||||
- kind: Group
|
||||
@@ -143,45 +145,47 @@ paired with system groups to grant access to all pods run in the namespace:
|
||||
name: system:authenticated
|
||||
```
|
||||
|
||||
For more examples of RBAC bindings, see [Role Binding
|
||||
Examples](/docs/reference/access-authn-authz/rbac#role-binding-examples).
|
||||
For a complete example of authorizing a PodSecurityPolicy, see
|
||||
[below](#example).
|
||||
For more examples of RBAC bindings, see
|
||||
[RoleBinding examples](/docs/reference/access-authn-authz/rbac#role-binding-examples).
|
||||
For a complete example of authorizing a PodSecurityPolicy, see [below](#example).
|
||||
|
||||
### Recommended Practice
|
||||
|
||||
PodSecurityPolicy is being replaced by a new, simplified `PodSecurity` {{< glossary_tooltip
|
||||
text="admission controller" term_id="admission-controller" >}}. For more details on this change, see
|
||||
[PodSecurityPolicy Deprecation: Past, Present, and
|
||||
Future](/blog/2021/04/06/podsecuritypolicy-deprecation-past-present-and-future/). Follow these
|
||||
guidelines to simplify migration from PodSecurityPolicy to the new admission controller:
|
||||
PodSecurityPolicy is being replaced by a new, simplified `PodSecurity`
|
||||
{{< glossary_tooltip text="admission controller" term_id="admission-controller" >}}.
|
||||
For more details on this change, see
|
||||
[PodSecurityPolicy Deprecation: Past, Present, and Future](/blog/2021/04/06/podsecuritypolicy-deprecation-past-present-and-future/).
|
||||
Follow these guidelines to simplify migration from PodSecurityPolicy to the
|
||||
new admission controller:
|
||||
|
||||
1. Limit your PodSecurityPolicies to the policies defined by the [Pod Security Standards](/docs/concepts/security/pod-security-standards):
|
||||
- {{< example file="policy/privileged-psp.yaml" >}}Privileged{{< /example >}}
|
||||
- {{< example file="policy/baseline-psp.yaml" >}}Baseline{{< /example >}}
|
||||
- {{< example file="policy/restricted-psp.yaml" >}}Restricted{{< /example >}}
|
||||
1. Limit your PodSecurityPolicies to the policies defined by the
|
||||
[Pod Security Standards](/docs/concepts/security/pod-security-standards):
|
||||
|
||||
2. Only bind PSPs to entire namespaces, by using the `system:serviceaccounts:<namespace>` group
|
||||
- {{< example file="policy/privileged-psp.yaml" >}}Privileged{{< /example >}}
|
||||
- {{< example file="policy/baseline-psp.yaml" >}}Baseline{{< /example >}}
|
||||
- {{< example file="policy/restricted-psp.yaml" >}}Restricted{{< /example >}}
|
||||
|
||||
1. Only bind PSPs to entire namespaces, by using the `system:serviceaccounts:<namespace>` group
|
||||
(where `<namespace>` is the target namespace). For example:
|
||||
|
||||
```yaml
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
# This cluster role binding allows all pods in the "development" namespace to use the baseline PSP.
|
||||
kind: ClusterRoleBinding
|
||||
metadata:
|
||||
name: psp-baseline-namespaces
|
||||
roleRef:
|
||||
kind: ClusterRole
|
||||
name: psp-baseline
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
subjects:
|
||||
- kind: Group
|
||||
name: system:serviceaccounts:development
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
- kind: Group
|
||||
name: system:serviceaccounts:canary
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
```
|
||||
```yaml
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
# This cluster role binding allows all pods in the "development" namespace to use the baseline PSP.
|
||||
kind: ClusterRoleBinding
|
||||
metadata:
|
||||
name: psp-baseline-namespaces
|
||||
roleRef:
|
||||
kind: ClusterRole
|
||||
name: psp-baseline
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
subjects:
|
||||
- kind: Group
|
||||
name: system:serviceaccounts:development
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
- kind: Group
|
||||
name: system:serviceaccounts:canary
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
```
|
||||
|
||||
### Troubleshooting
|
||||
|
||||
@@ -217,8 +221,8 @@ only non-mutating PodSecurityPolicies are used to validate the pod.
|
||||
|
||||
## Example
|
||||
|
||||
_This example assumes you have a running cluster with the PodSecurityPolicy
|
||||
admission controller enabled and you have cluster admin privileges._
|
||||
This example assumes you have a running cluster with the PodSecurityPolicy
|
||||
admission controller enabled and you have cluster admin privileges.
|
||||
|
||||
### Set up
|
||||
|
||||
@@ -364,12 +368,24 @@ Let's try that again, slightly differently:
|
||||
|
||||
```shell
|
||||
kubectl-user create deployment pause --image=k8s.gcr.io/pause
|
||||
```
|
||||
|
||||
```none
|
||||
deployment "pause" created
|
||||
|
||||
```
|
||||
```shell
|
||||
kubectl-user get pods
|
||||
No resources found.
|
||||
```
|
||||
|
||||
```
|
||||
No resources found.
|
||||
```
|
||||
|
||||
```shell
|
||||
kubectl-user get events | head -n 2
|
||||
```
|
||||
|
||||
```
|
||||
LASTSEEN FIRSTSEEN COUNT NAME KIND SUBOBJECT TYPE REASON SOURCE MESSAGE
|
||||
1m 2m 15 pause-7774d79b5 ReplicaSet Warning FailedCreate replicaset-controller Error creating: pods "pause-7774d79b5-" is forbidden: no providers available to validate pod request
|
||||
```
|
||||
@@ -390,6 +406,9 @@ is `default`:
|
||||
kubectl-admin create rolebinding default:psp:unprivileged \
|
||||
--role=psp:unprivileged \
|
||||
--serviceaccount=psp-example:default
|
||||
```
|
||||
|
||||
```none
|
||||
rolebinding "default:psp:unprivileged" created
|
||||
```
|
||||
|
||||
@@ -398,6 +417,9 @@ eventually succeed in creating the pod:
|
||||
|
||||
```shell
|
||||
kubectl-user get pods --watch
|
||||
```
|
||||
|
||||
```none
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
pause-7774d79b5-qrgcb 0/1 Pending 0 1s
|
||||
pause-7774d79b5-qrgcb 0/1 Pending 0 1s
|
||||
@@ -411,6 +433,9 @@ Delete the namespace to clean up most of the example resources:
|
||||
|
||||
```shell
|
||||
kubectl-admin delete ns psp-example
|
||||
```
|
||||
|
||||
```
|
||||
namespace "psp-example" deleted
|
||||
```
|
||||
|
||||
@@ -419,6 +444,9 @@ up separately:
|
||||
|
||||
```shell
|
||||
kubectl-admin delete psp example
|
||||
```
|
||||
|
||||
```
|
||||
podsecuritypolicy "example" deleted
|
||||
```
|
||||
|
||||
@@ -435,7 +463,8 @@ several security mechanisms.
|
||||
|
||||
{{< codenew file="policy/restricted-psp.yaml" >}}
|
||||
|
||||
See [Pod Security Standards](/docs/concepts/security/pod-security-standards/#policy-instantiation) for more examples.
|
||||
See [Pod Security Standards](/docs/concepts/security/pod-security-standards/#policy-instantiation)
|
||||
for more examples.
|
||||
|
||||
## Policy Reference
|
||||
|
||||
@@ -471,17 +500,17 @@ and `max`(inclusive). Defaults to no allowed host ports.
|
||||
**Volumes** - Provides a list of allowed volume types. The allowable values
|
||||
correspond to the volume sources that are defined when creating a volume. For
|
||||
the complete list of volume types, see [Types of
|
||||
Volumes](/docs/concepts/storage/volumes/#types-of-volumes). Additionally, `*`
|
||||
may be used to allow all volume types.
|
||||
Volumes](/docs/concepts/storage/volumes/#types-of-volumes). Additionally,
|
||||
`*` may be used to allow all volume types.
|
||||
|
||||
The **recommended minimum set** of allowed volumes for new PSPs are:
|
||||
|
||||
- configMap
|
||||
- downwardAPI
|
||||
- emptyDir
|
||||
- persistentVolumeClaim
|
||||
- secret
|
||||
- projected
|
||||
- `configMap`
|
||||
- `downwardAPI`
|
||||
- `emptyDir`
|
||||
- `persistentVolumeClaim`
|
||||
- `secret`
|
||||
- `projected`
|
||||
|
||||
{{< warning >}}
|
||||
PodSecurityPolicy does not limit the types of `PersistentVolume` objects that
|
||||
@@ -493,10 +522,10 @@ should be granted permission to create `PersistentVolume` objects.
|
||||
**FSGroup** - Controls the supplemental group applied to some volumes.
|
||||
|
||||
- *MustRunAs* - Requires at least one `range` to be specified. Uses the
|
||||
minimum value of the first range as the default. Validates against all ranges.
|
||||
minimum value of the first range as the default. Validates against all ranges.
|
||||
- *MayRunAs* - Requires at least one `range` to be specified. Allows
|
||||
`FSGroups` to be left unset without providing a default. Validates against
|
||||
all ranges if `FSGroups` is set.
|
||||
`FSGroups` to be left unset without providing a default. Validates against
|
||||
all ranges if `FSGroups` is set.
|
||||
- *RunAsAny* - No default provided. Allows any `fsGroup` ID to be specified.
|
||||
|
||||
**AllowedHostPaths** - This specifies a list of host paths that are allowed
|
||||
@@ -515,7 +544,8 @@ For example:
|
||||
readOnly: true # only allow read-only mounts
|
||||
```
|
||||
|
||||
{{< warning >}}There are many ways a container with unrestricted access to the host
|
||||
{{< warning >}}
|
||||
There are many ways a container with unrestricted access to the host
|
||||
filesystem can escalate privileges, including reading data from other
|
||||
containers, and abusing the credentials of system services, such as Kubelet.
|
||||
|
||||
@@ -556,33 +586,33 @@ spec:
|
||||
**RunAsUser** - Controls which user ID the containers are run with.
|
||||
|
||||
- *MustRunAs* - Requires at least one `range` to be specified. Uses the
|
||||
minimum value of the first range as the default. Validates against all ranges.
|
||||
minimum value of the first range as the default. Validates against all ranges.
|
||||
- *MustRunAsNonRoot* - Requires that the pod be submitted with a non-zero
|
||||
`runAsUser` or have the `USER` directive defined (using a numeric UID) in the
|
||||
image. Pods which have specified neither `runAsNonRoot` nor `runAsUser` settings
|
||||
will be mutated to set `runAsNonRoot=true`, thus requiring a defined non-zero
|
||||
numeric `USER` directive in the container. No default provided. Setting
|
||||
`allowPrivilegeEscalation=false` is strongly recommended with this strategy.
|
||||
`runAsUser` or have the `USER` directive defined (using a numeric UID) in the
|
||||
image. Pods which have specified neither `runAsNonRoot` nor `runAsUser` settings
|
||||
will be mutated to set `runAsNonRoot=true`, thus requiring a defined non-zero
|
||||
numeric `USER` directive in the container. No default provided. Setting
|
||||
`allowPrivilegeEscalation=false` is strongly recommended with this strategy.
|
||||
- *RunAsAny* - No default provided. Allows any `runAsUser` to be specified.
|
||||
|
||||
**RunAsGroup** - Controls which primary group ID the containers are run with.
|
||||
|
||||
- *MustRunAs* - Requires at least one `range` to be specified. Uses the
|
||||
minimum value of the first range as the default. Validates against all ranges.
|
||||
minimum value of the first range as the default. Validates against all ranges.
|
||||
- *MayRunAs* - Does not require that RunAsGroup be specified. However, when RunAsGroup
|
||||
is specified, they have to fall in the defined range.
|
||||
is specified, they have to fall in the defined range.
|
||||
- *RunAsAny* - No default provided. Allows any `runAsGroup` to be specified.
|
||||
|
||||
|
||||
**SupplementalGroups** - Controls which group IDs containers add.
|
||||
|
||||
- *MustRunAs* - Requires at least one `range` to be specified. Uses the
|
||||
minimum value of the first range as the default. Validates against all ranges.
|
||||
minimum value of the first range as the default. Validates against all ranges.
|
||||
- *MayRunAs* - Requires at least one `range` to be specified. Allows
|
||||
`supplementalGroups` to be left unset without providing a default.
|
||||
Validates against all ranges if `supplementalGroups` is set.
|
||||
`supplementalGroups` to be left unset without providing a default.
|
||||
Validates against all ranges if `supplementalGroups` is set.
|
||||
- *RunAsAny* - No default provided. Allows any `supplementalGroups` to be
|
||||
specified.
|
||||
specified.
|
||||
|
||||
### Privilege Escalation
|
||||
|
||||
@@ -627,8 +657,8 @@ added. Capabilities listed in `RequiredDropCapabilities` must not be included in
|
||||
`AllowedCapabilities` or `DefaultAddCapabilities`.
|
||||
|
||||
**DefaultAddCapabilities** - The capabilities which are added to containers by
|
||||
default, in addition to the runtime defaults. See the [Docker
|
||||
documentation](https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities)
|
||||
default, in addition to the runtime defaults. See the
|
||||
[Docker documentation](https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities)
|
||||
for the default list of capabilities when using the Docker runtime.
|
||||
|
||||
### SELinux
|
||||
@@ -655,16 +685,17 @@ denoted as the string `Unmasked`.
|
||||
|
||||
### AppArmor
|
||||
|
||||
Controlled via annotations on the PodSecurityPolicy. Refer to the [AppArmor
|
||||
documentation](/docs/tutorials/clusters/apparmor/#podsecuritypolicy-annotations).
|
||||
Controlled via annotations on the PodSecurityPolicy. Refer to the
|
||||
[AppArmor documentation](/docs/tutorials/security/apparmor/#podsecuritypolicy-annotations).
|
||||
|
||||
### Seccomp
|
||||
|
||||
As of Kubernetes v1.19, you can use the `seccompProfile` field in the
|
||||
`securityContext` of Pods or containers to [control use of seccomp
|
||||
profiles](/docs/tutorials/clusters/seccomp). In prior versions, seccomp was
|
||||
controlled by adding annotations to a Pod. The same PodSecurityPolicies can be
|
||||
used with either version to enforce how these fields or annotations are applied.
|
||||
`securityContext` of Pods or containers to
|
||||
[control use of seccomp profiles](/docs/tutorials/security/seccomp/).
|
||||
In prior versions, seccomp was controlled by adding annotations to a Pod. The
|
||||
same PodSecurityPolicies can be used with either version to enforce how these
|
||||
fields or annotations are applied.
|
||||
|
||||
**seccomp.security.alpha.kubernetes.io/defaultProfileName** - Annotation that
|
||||
specifies the default seccomp profile to apply to containers. Possible values
|
||||
@@ -681,10 +712,10 @@ are:
|
||||
flag is not defined, the default path will be used, which is
|
||||
`<root-dir>/seccomp` where `<root-dir>` is specified by the `--root-dir` flag.
|
||||
|
||||
{{< note >}}
|
||||
{{< note >}}
|
||||
The `--seccomp-profile-root` flag is deprecated since Kubernetes
|
||||
v1.19. Users are encouraged to use the default path.
|
||||
{{< /note >}}
|
||||
{{< /note >}}
|
||||
|
||||
**seccomp.security.alpha.kubernetes.io/allowedProfileNames** - Annotation that
|
||||
specifies which values are allowed for the pod seccomp annotations. Specified as
|
||||
@@ -696,18 +727,22 @@ default cannot be changed.
|
||||
|
||||
By default, all safe sysctls are allowed.
|
||||
|
||||
- `forbiddenSysctls` - excludes specific sysctls. You can forbid a combination of safe and unsafe sysctls in the list. To forbid setting any sysctls, use `*` on its own.
|
||||
- `allowedUnsafeSysctls` - allows specific sysctls that had been disallowed by the default list, so long as these are not listed in `forbiddenSysctls`.
|
||||
- `forbiddenSysctls` - excludes specific sysctls. You can forbid a combination
|
||||
of safe and unsafe sysctls in the list. To forbid setting any sysctls, use
|
||||
`*` on its own.
|
||||
- `allowedUnsafeSysctls` - allows specific sysctls that had been disallowed by
|
||||
the default list, so long as these are not listed in `forbiddenSysctls`.
|
||||
|
||||
Refer to the [Sysctl documentation](
|
||||
/docs/tasks/administer-cluster/sysctl-cluster/#podsecuritypolicy).
|
||||
Refer to the [Sysctl documentation](/docs/tasks/administer-cluster/sysctl-cluster/#podsecuritypolicy).
|
||||
|
||||
## {{% heading "whatsnext" %}}
|
||||
|
||||
- See [PodSecurityPolicy Deprecation: Past, Present, and
|
||||
Future](/blog/2021/04/06/podsecuritypolicy-deprecation-past-present-and-future/) to learn about
|
||||
the future of pod security policy.
|
||||
- See [PodSecurityPolicy Deprecation: Past, Present, and Future](/blog/2021/04/06/podsecuritypolicy-deprecation-past-present-and-future/)
|
||||
to learn about the future of pod security policy.
|
||||
|
||||
- See [Pod Security Standards](/docs/concepts/security/pod-security-standards/) for policy recommendations.
|
||||
- See [Pod Security Standards](/docs/concepts/security/pod-security-standards/)
|
||||
for policy recommendations.
|
||||
|
||||
- Refer to [PodSecurityPolicy reference](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#podsecuritypolicy-v1beta1-policy)
|
||||
for the API details.
|
||||
|
||||
- Refer to [Pod Security Policy Reference](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#podsecuritypolicy-v1beta1-policy) for the api details.
|
||||
|
||||
@@ -6,14 +6,117 @@ weight: 70
|
||||
|
||||
{{< glossary_definition term_id="api-eviction" length="short" >}} </br>
|
||||
|
||||
You can request eviction by directly calling the Eviction API
|
||||
using a client of the kube-apiserver, like the `kubectl drain` command.
|
||||
This creates an `Eviction` object, which causes the API server to terminate the Pod.
|
||||
You can request eviction by calling the Eviction API directly, or programmatically
|
||||
using a client of the {{<glossary_tooltip term_id="kube-apiserver" text="API server">}}, like the `kubectl drain` command. This
|
||||
creates an `Eviction` object, which causes the API server to terminate the Pod.
|
||||
|
||||
API-initiated evictions respect your configured [`PodDisruptionBudgets`](/docs/tasks/run-application/configure-pdb/)
|
||||
and [`terminationGracePeriodSeconds`](/docs/concepts/workloads/pods/pod-lifecycle#pod-termination).
|
||||
|
||||
Using the API to create an Eviction object for a Pod is like performing a
|
||||
policy-controlled [`DELETE` operation](/docs/reference/kubernetes-api/workload-resources/pod-v1/#delete-delete-a-pod)
|
||||
on the Pod.
|
||||
|
||||
## Calling the Eviction API
|
||||
|
||||
You can use a [Kubernetes language client](/docs/tasks/administer-cluster/access-cluster-api/#programmatic-access-to-the-api)
|
||||
to access the Kubernetes API and create an `Eviction` object. To do this, you
|
||||
POST the attempted operation, similar to the following example:
|
||||
|
||||
{{< tabs name="Eviction_example" >}}
|
||||
{{% tab name="policy/v1" %}}
|
||||
{{< note >}}
|
||||
`policy/v1` Eviction is available in v1.22+. Use `policy/v1beta1` with prior releases.
|
||||
{{< /note >}}
|
||||
|
||||
```json
|
||||
{
|
||||
"apiVersion": "policy/v1",
|
||||
"kind": "Eviction",
|
||||
"metadata": {
|
||||
"name": "quux",
|
||||
"namespace": "default"
|
||||
}
|
||||
}
|
||||
```
|
||||
{{% /tab %}}
|
||||
{{% tab name="policy/v1beta1" %}}
|
||||
{{< note >}}
|
||||
Deprecated in v1.22 in favor of `policy/v1`
|
||||
{{< /note >}}
|
||||
|
||||
```json
|
||||
{
|
||||
"apiVersion": "policy/v1beta1",
|
||||
"kind": "Eviction",
|
||||
"metadata": {
|
||||
"name": "quux",
|
||||
"namespace": "default"
|
||||
}
|
||||
}
|
||||
```
|
||||
{{% /tab %}}
|
||||
{{< /tabs >}}
|
||||
|
||||
Alternatively, you can attempt an eviction operation by accessing the API using
|
||||
`curl` or `wget`, similar to the following example:
|
||||
|
||||
```bash
|
||||
curl -v -H 'Content-type: application/json' https://your-cluster-api-endpoint.example/api/v1/namespaces/default/pods/quux/eviction -d @eviction.json
|
||||
```
|
||||
|
||||
## How API-initiated eviction works
|
||||
|
||||
When you request an eviction using the API, the API server performs admission
|
||||
checks and responds in one of the following ways:
|
||||
|
||||
* `200 OK`: the eviction is allowed, the `Eviction` subresource is created, and
|
||||
the Pod is deleted, similar to sending a `DELETE` request to the Pod URL.
|
||||
* `429 Too Many Requests`: the eviction is not currently allowed because of the
|
||||
configured {{<glossary_tooltip term_id="pod-disruption-budget" text="PodDisruptionBudget">}}.
|
||||
You may be able to attempt the eviction again later. You might also see this
|
||||
response because of API rate limiting.
|
||||
* `500 Internal Server Error`: the eviction is not allowed because there is a
|
||||
misconfiguration, like if multiple PodDisruptionBudgets reference the same Pod.
|
||||
|
||||
If the Pod you want to evict isn't part of a workload that has a
|
||||
PodDisruptionBudget, the API server always returns `200 OK` and allows the
|
||||
eviction.
|
||||
|
||||
If the API server allows the eviction, the Pod is deleted as follows:
|
||||
|
||||
1. The `Pod` resource in the API server is updated with a deletion timestamp,
|
||||
after which the API server considers the `Pod` resource to be terminated. The
|
||||
`Pod` resource is also marked with the configured grace period.
|
||||
1. The {{<glossary_tooltip term_id="kubelet" text="kubelet">}} on the node where the local Pod is running notices that the `Pod`
|
||||
resource is marked for termination and starts to gracefully shut down the
|
||||
local Pod.
|
||||
1. While the kubelet is shutting the Pod down, the control plane removes the Pod
|
||||
from {{<glossary_tooltip term_id="endpoint" text="Endpoint">}} and
|
||||
{{<glossary_tooltip term_id="endpoint-slice" text="EndpointSlice">}}
|
||||
objects. As a result, controllers no longer consider the Pod as a valid object.
|
||||
1. After the grace period for the Pod expires, the kubelet forcefully terminates
|
||||
the local Pod.
|
||||
1. The kubelet tells the API server to remove the `Pod` resource.
|
||||
1. The API server deletes the `Pod` resource.
|
||||
|
||||
## Troubleshooting stuck evictions
|
||||
|
||||
In some cases, your applications may enter a broken state, where the Eviction
|
||||
API will only return `429` or `500` responses until you intervene. This can
|
||||
happen if, for example, a ReplicaSet creates pods for your application but new
|
||||
pods do not enter a `Ready` state. You may also notice this behavior in cases
|
||||
where the last evicted Pod had a long termination grace period.
|
||||
|
||||
If you notice stuck evictions, try one of the following solutions:
|
||||
|
||||
* Abort or pause the automated operation causing the issue. Investigate the stuck
|
||||
application before you restart the operation.
|
||||
* Wait a while, then directly delete the Pod from your cluster control plane
|
||||
instead of using the Eviction API.
|
||||
|
||||
## {{% heading "whatsnext" %}}
|
||||
|
||||
* Learn about [Node-pressure Eviction](/docs/concepts/scheduling-eviction/node-pressure-eviction/)
|
||||
* Learn about [Pod Priority and Preemption](/docs/concepts/scheduling-eviction/pod-priority-preemption/)
|
||||
* Learn how to protect your applications with a [Pod Disruption Budget](/docs/tasks/run-application/configure-pdb/).
|
||||
* Learn about [Node-pressure Eviction](/docs/concepts/scheduling-eviction/node-pressure-eviction/).
|
||||
* Learn about [Pod Priority and Preemption](/docs/concepts/scheduling-eviction/pod-priority-preemption/).
|
||||
|
||||
@@ -167,7 +167,7 @@ To do so, add an `addedAffinity` to the args of the [`NodeAffinity` plugin](/doc
|
||||
in the [scheduler configuration](/docs/reference/scheduling/config/). For example:
|
||||
|
||||
```yaml
|
||||
apiVersion: kubescheduler.config.k8s.io/v1beta1
|
||||
apiVersion: kubescheduler.config.k8s.io/v1beta3
|
||||
kind: KubeSchedulerConfiguration
|
||||
|
||||
profiles:
|
||||
|
||||
@@ -38,7 +38,7 @@ Below is an example configuration that sets
|
||||
resources `intel.com/foo` and `intel.com/bar`.
|
||||
|
||||
```yaml
|
||||
apiVersion: kubescheduler.config.k8s.io/v1beta1
|
||||
apiVersion: kubescheduler.config.k8s.io/v1beta3
|
||||
kind: KubeSchedulerConfiguration
|
||||
profiles:
|
||||
# ...
|
||||
|
||||
@@ -255,10 +255,11 @@ up to 3 replicas, as well as scaling down the old ReplicaSet to 0 replicas.
|
||||
Deployment also ensures that only a certain number of Pods are created above the desired number of Pods.
|
||||
By default, it ensures that at most 125% of the desired number of Pods are up (25% max surge).
|
||||
|
||||
For example, if you look at the above Deployment closely, you will see that it first created a new Pod,
|
||||
then deleted some old Pods, and created new ones. It does not kill old Pods until a sufficient number of
|
||||
For example, if you look at the above Deployment closely, you will see that it first creates a new Pod,
|
||||
then deletes an old Pod, and creates another new one. It does not kill old Pods until a sufficient number of
|
||||
new Pods have come up, and does not create new Pods until a sufficient number of old Pods have been killed.
|
||||
It makes sure that at least 2 Pods are available and that at max 4 Pods in total are available.
|
||||
It makes sure that at least 3 Pods are available and that at max 4 Pods in total are available. In case of
|
||||
a Deployment with 4 replicas, the number of Pods would be between 3 and 5.
|
||||
|
||||
* Get details of your Deployment:
|
||||
```shell
|
||||
@@ -305,10 +306,17 @@ up to 3 replicas, as well as scaling down the old ReplicaSet to 0 replicas.
|
||||
```
|
||||
Here you see that when you first created the Deployment, it created a ReplicaSet (nginx-deployment-2035384211)
|
||||
and scaled it up to 3 replicas directly. When you updated the Deployment, it created a new ReplicaSet
|
||||
(nginx-deployment-1564180365) and scaled it up to 1 and then scaled down the old ReplicaSet to 2, so that at
|
||||
least 2 Pods were available and at most 4 Pods were created at all times. It then continued scaling up and down
|
||||
the new and the old ReplicaSet, with the same rolling update strategy. Finally, you'll have 3 available replicas
|
||||
in the new ReplicaSet, and the old ReplicaSet is scaled down to 0.
|
||||
(nginx-deployment-1564180365) and scaled it up to 1 and waited for it to come up. Then it scaled down the old ReplicaSet
|
||||
to 2 and scaled up the new ReplicaSet to 2 so that at least 3 Pods were available and at most 4 Pods were created at all times.
|
||||
It then continued scaling up and down the new and the old ReplicaSet, with the same rolling update strategy.
|
||||
Finally, you'll have 3 available replicas in the new ReplicaSet, and the old ReplicaSet is scaled down to 0.
|
||||
|
||||
{{< note >}}
|
||||
Kubernetes doesn't count terminating Pods when calculating the number of `availableReplicas`, which must be between
|
||||
`replicas - maxUnavailable` and `replicas + maxSurge`. As a result, you might notice that there are more Pods than
|
||||
expected during a rollout, and that the total resources consumed by the Deployment is more than `replicas + maxSurge`
|
||||
until the `terminationGracePeriodSeconds` of the terminating Pods expires.
|
||||
{{< /note >}}
|
||||
|
||||
### Rollover (aka multiple updates in-flight)
|
||||
|
||||
|
||||
@@ -266,7 +266,7 @@ Note that we recommend using Deployments instead of directly using Replica Sets,
|
||||
|
||||
### Deployment (Recommended)
|
||||
|
||||
[`Deployment`](/docs/concepts/workloads/controllers/deployment/) is a higher-level API object that updates its underlying Replica Sets and their Pods. Deployments are recommended if you want the rolling update functionality because, they are declarative, server-side, and have additional features.
|
||||
[`Deployment`](/docs/concepts/workloads/controllers/deployment/) is a higher-level API object that updates its underlying Replica Sets and their Pods. Deployments are recommended if you want the rolling update functionality, because they are declarative, server-side, and have additional features.
|
||||
|
||||
### Bare Pods
|
||||
|
||||
|
||||
@@ -296,7 +296,7 @@ replication controllers, replica sets or stateful sets that the Pod belongs to.
|
||||
An example configuration might look like follows:
|
||||
|
||||
```yaml
|
||||
apiVersion: kubescheduler.config.k8s.io/v1beta1
|
||||
apiVersion: kubescheduler.config.k8s.io/v1beta3
|
||||
kind: KubeSchedulerConfiguration
|
||||
|
||||
profiles:
|
||||
@@ -352,7 +352,7 @@ you can disable those defaults by setting `defaultingType` to `List` and leaving
|
||||
empty `defaultConstraints` in the `PodTopologySpread` plugin configuration:
|
||||
|
||||
```yaml
|
||||
apiVersion: kubescheduler.config.k8s.io/v1beta1
|
||||
apiVersion: kubescheduler.config.k8s.io/v1beta3
|
||||
kind: KubeSchedulerConfiguration
|
||||
|
||||
profiles:
|
||||
|
||||
Reference in New Issue
Block a user