Merge pull request #22218 from savitharaghunathan/merged-master-dev-1.19

Branch sync - Merge master with dev 1.19
2020-07-07 10:45:58 -07:00
parent 7825b79270 220d754880
commit c7cf16fbb6
373 changed files with 15355 additions and 4779 deletions
@@ -99,7 +99,9 @@ NamespaceLifecycle, LimitRanger, ServiceAccount, TaintNodesByCondition, Priority

 ## What does each admission controller do?

-### AlwaysAdmit {#alwaysadmit} {{< feature-state for_k8s_version="v1.13" state="deprecated" >}}
+### AlwaysAdmit {#alwaysadmit}
+
+{{< feature-state for_k8s_version="v1.13" state="deprecated" >}}

 This admission controller allows all pods into the cluster. It is deprecated because its behavior is the same as if there were no admission controller at all.

@@ -113,7 +115,9 @@ scheduled onto the right node), without any authorization check against the imag
 is enabled, images are always pulled prior to starting containers, which means valid credentials are
 required.

-### AlwaysDeny {#alwaysdeny} {{< feature-state for_k8s_version="v1.13" state="deprecated" >}}
+### AlwaysDeny {#alwaysdeny}
+
+{{< feature-state for_k8s_version="v1.13" state="deprecated" >}}

 Rejects all requests. AlwaysDeny is DEPRECATED as no real meaning.

@@ -164,7 +168,9 @@ if the pods don't already have toleration for taints
 `node.kubernetes.io/not-ready:NoExecute` or
 `node.alpha.kubernetes.io/unreachable:NoExecute`.

-### DenyExecOnPrivileged {#denyexeconprivileged} {{< feature-state for_k8s_version="v1.13" state="deprecated" >}}
+### DenyExecOnPrivileged {#denyexeconprivileged}
+
+{{< feature-state for_k8s_version="v1.13" state="deprecated" >}}

 This admission controller will intercept all requests to exec a command in a pod if that pod has a privileged container.

@@ -175,7 +181,9 @@ Use of a policy-based admission plugin (like [PodSecurityPolicy](#podsecuritypol
 which can be targeted at specific users or Namespaces and also protects against creation of overly privileged Pods
 is recommended instead.

-### DenyEscalatingExec {#denyescalatingexec} {{< feature-state for_k8s_version="v1.13" state="deprecated" >}}
+### DenyEscalatingExec {#denyescalatingexec}
+
+{{< feature-state for_k8s_version="v1.13" state="deprecated" >}}

 This admission controller will deny exec and attach commands to pods that run with escalated privileges that
 allow host access.  This includes pods that run as privileged, have access to the host IPC namespace, and
@@ -187,7 +195,9 @@ Use of a policy-based admission plugin (like [PodSecurityPolicy](#podsecuritypol
 which can be targeted at specific users or Namespaces and also protects against creation of overly privileged Pods
 is recommended instead.

-### EventRateLimit {#eventratelimit} {{< feature-state for_k8s_version="v1.13" state="alpha" >}}
+### EventRateLimit {#eventratelimit}
+
+{{< feature-state for_k8s_version="v1.13" state="alpha" >}}

 This admission controller mitigates the problem where the API server gets flooded by
 event requests. The cluster admin can specify event rate limits by:
@@ -446,7 +456,9 @@ applies a 0.1 CPU requirement to all Pods in the `default` namespace.

 See the [limitRange design doc](https://git.k8s.io/community/contributors/design-proposals/resource-management/admission_control_limit_range.md) and the [example of Limit Range](/docs/tasks/configure-pod-container/limit-range/) for more details.

-### MutatingAdmissionWebhook {#mutatingadmissionwebhook} {{< feature-state for_k8s_version="v1.13" state="beta" >}}
+### MutatingAdmissionWebhook {#mutatingadmissionwebhook}
+
+{{< feature-state for_k8s_version="v1.13" state="beta" >}}

 This admission controller calls any mutating webhooks which match the request. Matching
 webhooks are called in serial; each one may modify the object if it desires.
@@ -537,7 +549,9 @@ This admission controller also protects the access to `metadata.ownerReferences[
 of an object, so that only users with "update" permission to the `finalizers`
 subresource of the referenced *owner* can change it.

-### PersistentVolumeLabel {#persistentvolumelabel} {{< feature-state for_k8s_version="v1.13" state="deprecated" >}}
+### PersistentVolumeLabel {#persistentvolumelabel}
+
+{{< feature-state for_k8s_version="v1.13" state="deprecated" >}}

 This admission controller automatically attaches region or zone labels to PersistentVolumes
 as defined by the cloud provider (for example, GCE or AWS).
@@ -708,7 +722,9 @@ objects in your Kubernetes deployment, you MUST use this admission controller to

 See the [resourceQuota design doc](https://git.k8s.io/community/contributors/design-proposals/resource-management/admission_control_resource_quota.md) and the [example of Resource Quota](/docs/concepts/policy/resource-quotas/) for more details.

-### RuntimeClass {#runtimeclass} {{< feature-state for_k8s_version="v1.16" state="alpha" >}}
+### RuntimeClass {#runtimeclass}
+
+{{< feature-state for_k8s_version="v1.16" state="alpha" >}}

 For [RuntimeClass](/docs/concepts/containers/runtime-class/) definitions which describe an overhead associated with running a pod,
 this admission controller will set the pod.Spec.Overhead field accordingly.
@@ -729,11 +745,15 @@ We strongly recommend using this admission controller if you intend to make use

 The `StorageObjectInUseProtection` plugin adds the `kubernetes.io/pvc-protection` or `kubernetes.io/pv-protection` finalizers to newly created Persistent Volume Claims (PVCs) or Persistent Volumes (PV). In case a user deletes a PVC or PV the PVC or PV is not removed until the finalizer is removed from the PVC or PV by PVC or PV Protection Controller. Refer to the [Storage Object in Use Protection](/docs/concepts/storage/persistent-volumes/#storage-object-in-use-protection) for more detailed information.

-### TaintNodesByCondition {#taintnodesbycondition} {{< feature-state for_k8s_version="v1.12" state="beta" >}}
+### TaintNodesByCondition {#taintnodesbycondition}
+
+{{< feature-state for_k8s_version="v1.12" state="beta" >}}

 This admission controller {{< glossary_tooltip text="taints" term_id="taint" >}} newly created Nodes as `NotReady` and `NoSchedule`. That tainting avoids a race condition that could cause Pods to be scheduled on new Nodes before their taints were updated to accurately reflect their reported conditions.

-### ValidatingAdmissionWebhook {#validatingadmissionwebhook} {{< feature-state for_k8s_version="v1.13" state="beta" >}}
+### ValidatingAdmissionWebhook {#validatingadmissionwebhook}
+
+{{< feature-state for_k8s_version="v1.13" state="beta" >}}

 This admission controller calls any validating webhooks which match the request. Matching
 webhooks are called in parallel; if any of them rejects the request, the request
@@ -773,6 +793,4 @@ phase, and therefore is the last admission controller to run.
 in the mutating phase.

    For earlier versions, there was no concept of validating versus mutating and the
-admission controllers ran in the exact order specified.
-
-
+admission controllers ran in the exact order specified.
@@ -26,6 +26,8 @@ even a file with a list of usernames and passwords. In this regard, _Kubernetes
 does not have objects which represent normal user accounts._ Normal users
 cannot be added to a cluster through an API call.

+Even though normal user cannot be added via an API call, but any user that presents a valid certificate signed by the cluster’s certificate authority (CA) is considered authenticated. In this configuration, Kubernetes determines the username from the common name field in the ‘subject’ of the cert (e.g., “/CN=bob”). From there, the role based access control (RBAC) sub-system would determine whether the user is authorized to perform a specific operation a resource. You can refer to [creating user certificate request](/docs/reference/access-authn-authz/certificate-signing-requests/#user-csr) for more details about this.
+
 In contrast, service accounts are users managed by the Kubernetes API. They are
 bound to specific namespaces, and created automatically by the API server or
 manually through API calls. Service accounts are tied to a set of credentials
@@ -48,7 +48,7 @@ The CertificateSigningRequest `status.certificate` field is empty until the sign

 Once the `status.certificate` field has been populated, the request has been completed and clients can now
 fetch the signed certificate PEM data from the CertificateSigningRequest resource.
-Signers can instead deny certificate signing if the approval conditions are not met.
+The signers can instead deny certificate signing if the approval conditions are not met.

 In order to reduce the number of old CertificateSigningRequest resources left in a cluster, a garbage collection
 controller runs periodically. The garbage collection removes CertificateSigningRequests that have not changed
@@ -67,10 +67,10 @@ This includes:
 1. **Permitted subjects**: any restrictions on and behavior when a disallowed subject is requested.
 1. **Permitted x509 extensions**: including IP subjectAltNames, DNS subjectAltNames, Email subjectAltNames, URI subjectAltNames etc, and behavior when a disallowed extension is requested.
 1. **Permitted key usages / extended key usages**: any restrictions on and behavior when usages different than the signer-determined usages are specified in the CSR.
-1. **Expiration/certificate lifetime**: whether it is fixed by the signer, configurable by the admin, determined by the CSR object etc and behavior if an expiration different than the signer-determined expiration is specified in the CSR.
+1. **Expiration/certificate lifetime**: whether it is fixed by the signer, configurable by the admin, determined by the CSR object etc and the behavior when an expiration is different than the signer-determined expiration that is specified in the CSR.
 1. **CA bit allowed/disallowed**: and behavior if a CSR contains a request a for a CA certificate when the signer does not permit it.

-Commonly, the `status.certificate` field contains a single PEM-encoded X.509 certificate once the CSR is approved and the certificate is issued. Some signers store multiple certificates into the `status.certificate` field. In that case, the documentation for the signer should specify the meaning of additional certificates; for example, this might be certificate plus intermediates to be presented during TLS handshakes.
+Commonly, the `status.certificate` field contains a single PEM-encoded X.509 certificate once the CSR is approved and the certificate is issued. Some signers store multiple certificates into the `status.certificate` field. In that case, the documentation for the signer should specify the meaning of additional certificates; for example, this might be the certificate plus intermediates to be presented during TLS handshakes.

 ### Kubernetes signers

@@ -88,19 +88,18 @@ Kubernetes provides built-in signers that each have a well-known `signerName`:
 1. `kubernetes.io/kube-apiserver-client-kubelet`: signs client certificates that will be honored as client-certs by the
  kube-apiserver.
  May be auto-approved by {{< glossary_tooltip term_id="kube-controller-manager" >}}.
-    1. Trust distribution: signed certificates must be honored as client-certificates by the kube-apiserver.  The CA bundle
+    1. Trust distribution: signed certificates must be honored as client-certificates by the kube-apiserver. The CA bundle
       is not distributed by any other means.
    1. Permitted subjects - organizations are exactly `[]string{"system:nodes"}`, common name starts with `"system:node:"`
-    1. Permitted x509 extensions - honors key usage extensions, forbids subjectAltName extensions, drops other extensions.
+    1. Permitted x509 extensions - honors key usage extensions, forbids subjectAltName extensions and drops other extensions.
    1. Permitted key usages - exactly `[]string{"key encipherment", "digital signature", "client auth"}`
-    1. Expiration/certificate lifetime - minimum of CSR signer or request.  Sanity of the time is the concern of the signer.
+    1. Expiration/certificate lifetime - minimum of CSR signer or request. The signer is responsible for checking that the certificate lifetime is valid and permissible.
    1. CA bit allowed/disallowed - not allowed.

 1. `kubernetes.io/kubelet-serving`: signs serving certificates that are honored as a valid kubelet serving certificate
  by the kube-apiserver, but has no other guarantees.
  Never auto-approved by {{< glossary_tooltip term_id="kube-controller-manager" >}}.
-    1. Trust distribution: signed certificates must be honored by the kube-apiserver as valid to terminate connections to a kubelet.
-       The CA bundle is not distributed by any other means.
+    1. Trust distribution: signed certificates must be honored by the kube-apiserver as valid to terminate connections to a kubelet. The CA bundle is not distributed by any other means.
    1. Permitted subjects - organizations are exactly `[]string{"system:nodes"}`, common name starts with `"system:node:"`
    1. Permitted x509 extensions - honors key usage and DNSName/IPAddress subjectAltName extensions, forbids EmailAddress and URI subjectAltName extensions, drops other extensions. At least one DNS or IP subjectAltName must be present.
    1. Permitted key usages - exactly `[]string{"key encipherment", "digital signature", "server auth"}`
@@ -108,13 +107,13 @@ Kubernetes provides built-in signers that each have a well-known `signerName`:
    1. CA bit allowed/disallowed - not allowed.

 1. `kubernetes.io/legacy-unknown`:  has no guarantees for trust at all. Some distributions may honor these as client
-  certs, but that behavior is not standard Kubernetes behavior.
+  certs, but that behavior is non-standard Kubernetes behavior.
  Never auto-approved by {{< glossary_tooltip term_id="kube-controller-manager" >}}.
    1. Trust distribution: None.  There is no standard trust or distribution for this signer in a Kubernetes cluster.
    1. Permitted subjects - any
    1. Permitted x509 extensions - honors subjectAltName and key usage extensions and discards other extensions.
    1. Permitted key usages - any
-    1. Expiration/certificate lifetime - minimum of CSR signer or request.  Sanity of the time is the concern of the signer.
+    1. Expiration/certificate lifetime - minimum of CSR signer or request. The signer is responsible for checking that the certificate lifetime is valid and permissible.
    1. CA bit allowed/disallowed - not allowed.

 {{< note >}}
@@ -226,6 +225,101 @@ rules:
  - sign
 ```

+## Normal User
+
+There are a few steps are required in order to get normal user to be able to authenticate and invoke API. First, this user must have certificate issued by the Kubernetes Cluster, and then present that Certificate into the API call as the Certificate Header, or through the kubectl.
+
+### Create Private Key
+
+The following scripts show how to generate PKI private key and CSR. It is important to set CN and O attribute of the CSR. CN is the name of the user and O is the group that this user will belong to. You can refer to [RBAC](/docs/reference/access-authn-authz/rbac/) for standard groups.
+
+```
+openssl genrsa -out john.key 2048
+openssl req -new -key john.key -out john.csr
+```
+
+### Create Certificate Request Kubernetes Object
+
+Create a CertificateSigningRequest and submit it to a Kubernetes Cluster via kubectl. Below is a script to generate the CertificateSigningRequest.
+
+```
+cat <<EOF | kubectl apply -f -
+apiVersion: certificates.k8s.io/v1beta1
+kind: CertificateSigningRequest
+metadata:
+  name: john
+spec:
+  groups:
+  - system:authenticated
+  request: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURSBSRVFVRVNULS0tLS0KTUlJQ1ZqQ0NBVDRDQVFBd0VURVBNQTBHQTFVRUF3d0dZVzVuWld4aE1JSUJJakFOQmdrcWhraUc5dzBCQVFFRgpBQU9DQVE4QU1JSUJDZ0tDQVFFQTByczhJTHRHdTYxakx2dHhWTTJSVlRWMDNHWlJTWWw0dWluVWo4RElaWjBOCnR2MUZtRVFSd3VoaUZsOFEzcWl0Qm0wMUFSMkNJVXBGd2ZzSjZ4MXF3ckJzVkhZbGlBNVhwRVpZM3ExcGswSDQKM3Z3aGJlK1o2MVNrVHF5SVBYUUwrTWM5T1Nsbm0xb0R2N0NtSkZNMUlMRVI3QTVGZnZKOEdFRjJ6dHBoaUlFMwpub1dtdHNZb3JuT2wzc2lHQ2ZGZzR4Zmd4eW8ybmlneFNVekl1bXNnVm9PM2ttT0x1RVF6cXpkakJ3TFJXbWlECklmMXBMWnoyalVnald4UkhCM1gyWnVVV1d1T09PZnpXM01LaE8ybHEvZi9DdS8wYk83c0x0MCt3U2ZMSU91TFcKcW90blZtRmxMMytqTy82WDNDKzBERHk5aUtwbXJjVDBnWGZLemE1dHJRSURBUUFCb0FBd0RRWUpLb1pJaHZjTgpBUUVMQlFBRGdnRUJBR05WdmVIOGR4ZzNvK21VeVRkbmFjVmQ1N24zSkExdnZEU1JWREkyQTZ1eXN3ZFp1L1BVCkkwZXpZWFV0RVNnSk1IRmQycVVNMjNuNVJsSXJ3R0xuUXFISUh5VStWWHhsdnZsRnpNOVpEWllSTmU3QlJvYXgKQVlEdUI5STZXT3FYbkFvczFqRmxNUG5NbFpqdU5kSGxpT1BjTU1oNndLaTZzZFhpVStHYTJ2RUVLY01jSVUyRgpvU2djUWdMYTk0aEpacGk3ZnNMdm1OQUxoT045UHdNMGM1dVJVejV4T0dGMUtCbWRSeEgvbUNOS2JKYjFRQm1HCkkwYitEUEdaTktXTU0xMzhIQXdoV0tkNjVoVHdYOWl4V3ZHMkh4TG1WQzg0L1BHT0tWQW9FNkpsYWFHdTlQVmkKdjlOSjVaZlZrcXdCd0hKbzZXdk9xVlA3SVFjZmg3d0drWm89Ci0tLS0tRU5EIENFUlRJRklDQVRFIFJFUVVFU1QtLS0tLQo=
+  usages:
+  - client auth
+EOF
+```
+
+Some points to note:
+
+- usage has to be 'client auth'
+- request is the base64 encoded value of the CSR file content. You can use this command to get that ```cat john.csr | base64 | tr -d "\n"```
+
+### Approve Certificate Request
+
+Use kubeadmin to create a CSR and approve it.
+
+Get the list of CSRs
+```
+kubectl get csr
+```
+
+Approve the CSR
+```
+kubectl certificate approve john
+```
+
+### Get the Certificate
+
+Retrieve the Certificate from the CSR.
+
+```
+kubectl get csr/john -o yaml
+```
+
+The Certificate value is in Base64-encoded format under status.certificate.
+
+### Create Role and Role Binding
+
+You get the Certificate already. Now it is time to define the Role and Role Binding for this user to access Kubernetes Cluster resources.
+
+This is a sample script to create role for this new user
+```
+kubectl create role developer --verb=create --verb=get --verb=list --verb=update --verb=delete --resource=pods
+```
+
+This is a sample script to create role binding for this new user
+```
+kubectl create rolebinding developer-binding-john --role=developer --user=john
+```
+
+### Add to KubeConfig
+
+The last step is to add this user into the KubeConfig. We assume the key and crt files are located here "/home/vagrant/work/".
+
+First, we need to add new credentials
+```
+kubectl config set-credentials john --client-key=/home/vagrant/work/john.key --client-certificate=/home/vagrant/work/john.crt --embed-certs=true
+
+```
+
+Then, we need to add the context
+```
+kubectl config set-context john --cluster=kubernetes --user=john
+```
+
+To test it, change kubecontext to john
+```
+kubectl config use-context john
+```
+
 ## Approval & rejection

 ### Control plane automated approval {#approval-rejection-control-plane}
@@ -137,7 +137,8 @@ different Kubernetes components.
 | `ServerSideApply` | `true` | Beta | 1.16 | |
 | `ServiceNodeExclusion` | `false` | Alpha | 1.8 | |
 | `ServiceTopology` | `false` | Alpha | 1.17 | |
-| `StartupProbe` | `false` | Alpha | 1.16 | |
+| `StartupProbe` | `false` | Alpha | 1.16 | 1.17 |
+| `StartupProbe` | `true` | Beta | 1.18 | |
 | `StorageVersionHash` | `false` | Alpha | 1.14 | 1.14 |
 | `StorageVersionHash` | `true` | Beta | 1.15 | |
 | `SupportNodePidsLimit` | `false` | Alpha | 1.14 | 1.14 |
@@ -290,10 +290,10 @@ kubectl logs -f my-pod                              # stream pod logs (stdout)
 kubectl logs -f my-pod -c my-container              # stream pod container logs (stdout, multi-container case)
 kubectl logs -f -l name=myLabel --all-containers    # stream all pods logs with label name=myLabel (stdout)
 kubectl run -i --tty busybox --image=busybox -- sh  # Run pod as interactive shell
-kubectl run nginx --image=nginx --restart=Never -n 
+kubectl run nginx --image=nginx -n 
 mynamespace                                         # Run pod nginx in a specific namespace
-kubectl run nginx --image=nginx --restart=Never     # Run pod nginx and write its spec into a file called pod.yaml
--dry-run -o yaml > pod.yaml
+kubectl run nginx --image=nginx                     # Run pod nginx and write its spec into a file called pod.yaml
+--dry-run=client -o yaml > pod.yaml

 kubectl attach my-pod -i                            # Attach to Running Container
 kubectl port-forward my-pod 5000:6000               # Listen on port 5000 on the local machine and forward to port 6000 on my-pod
@@ -76,9 +76,6 @@ kubectl run [-i] [--tty] --attach <name> --image=<image>
 Unlike `docker run ...`, if you specify `--attach`, then you attach `stdin`, `stdout` and `stderr`. You cannot control which streams are attached (`docker -a ...`).
 To detach from the container, you can type the escape sequence Ctrl+P followed by Ctrl+Q.

-Because the kubectl run command starts a Deployment for the container, the Deployment restarts if you terminate the attached process by using Ctrl+C, unlike `docker run -it`.
-To destroy the Deployment and its pods you need to run `kubectl delete deployment <name>`.
-
 ## docker ps

 To list what is currently running, see [kubectl get](/docs/reference/generated/kubectl/kubectl-commands/#get).
@@ -191,7 +188,7 @@ docker exec -ti 55c103fa1296 /bin/sh
 kubectl:

 ```shell
-kubectl exec -ti nginx-app-5jyvm -- /bin/sh      
+kubectl exec -ti nginx-app-5jyvm -- /bin/sh
 # exit
 ```

@@ -59,7 +59,7 @@ kubeadm join phase kubelet-start --help
 ```

 Similar to the [kubeadm init phase](/docs/reference/setup-tools/kubeadm/kubeadm-init/#init-phases)
-command, `kubadm join phase` allows you to skip a list of phases using the `--skip-phases` flag.
+command, `kubeadm join phase` allows you to skip a list of phases using the `--skip-phases` flag.

 For example: