Merge remote-tracking branch 'upstream/main' into dev-1.23
This commit is contained in:
Jesse Butler
@@ -190,9 +190,9 @@ kubectl get configmap
|
||||
No resources found in default namespace.
|
||||
```
|
||||
|
||||
To sum things up, when there's an override owner reference from a child to a parent, deleting the parent deletes the children automatically. This is called `cascade`. The default for cascade is `true`, however, you can use the --cascade=false option for `kubectl delete` to delete an object and orphan its children.
|
||||
To sum things up, when there's an override owner reference from a child to a parent, deleting the parent deletes the children automatically. This is called `cascade`. The default for cascade is `true`, however, you can use the --cascade=orphan option for `kubectl delete` to delete an object and orphan its children.
|
||||
|
||||
In the following example, there is a parent and a child. Notice the owner references are still included. If I delete the parent using --cascade=false, the parent is deleted but the child still exists:
|
||||
In the following example, there is a parent and a child. Notice the owner references are still included. If I delete the parent using --cascade=orphan, the parent is deleted but the child still exists:
|
||||
|
||||
```
|
||||
kubectl get configmap
|
||||
@@ -200,7 +200,7 @@ NAME DATA AGE
|
||||
mymap-child 0 13m8s
|
||||
mymap-parent 0 13m8s
|
||||
|
||||
kubectl delete --cascade=false configmap/mymap-parent
|
||||
kubectl delete --cascade=orphan configmap/mymap-parent
|
||||
configmap "mymap-parent" deleted
|
||||
|
||||
kubectl get configmap
|
||||
|
||||
@@ -0,0 +1,287 @@
|
||||
---
|
||||
layout: blog
|
||||
title: "Introducing Single Pod Access Mode for PersistentVolumes"
|
||||
date: 2021-09-13
|
||||
slug: read-write-once-pod-access-mode-alpha
|
||||
---
|
||||
|
||||
**Author:** Chris Henzie (Google)
|
||||
|
||||
Last month's release of Kubernetes v1.22 introduced a new ReadWriteOncePod access mode for [PersistentVolumes](/docs/concepts/storage/persistent-volumes/#persistent-volumes) and [PersistentVolumeClaims](/docs/concepts/storage/persistent-volumes/#persistentvolumeclaims).
|
||||
With this alpha feature, Kubernetes allows you to restrict volume access to a single pod in the cluster.
|
||||
|
||||
## What are access modes and why are they important?
|
||||
|
||||
When using storage, there are different ways to model how that storage is consumed.
|
||||
|
||||
For example, a storage system like a network file share can have many users all reading and writing data simultaneously.
|
||||
In other cases maybe everyone is allowed to read data but not write it.
|
||||
For highly sensitive data, maybe only one user is allowed to read and write data but nobody else.
|
||||
|
||||
In the world of Kubernetes, [access modes](/docs/concepts/storage/persistent-volumes/#access-modes) are the way you can define how durable storage is consumed.
|
||||
These access modes are a part of the spec for PersistentVolumes (PVs) and PersistentVolumeClaims (PVCs).
|
||||
|
||||
```yaml
|
||||
kind: PersistentVolumeClaim
|
||||
apiVersion: v1
|
||||
metadata:
|
||||
name: shared-cache
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteMany # Allow many pods to access shared-cache simultaneously.
|
||||
resources:
|
||||
requests:
|
||||
storage: 1Gi
|
||||
```
|
||||
|
||||
Before v1.22, Kubernetes offered three access modes for PVs and PVCs:
|
||||
|
||||
- ReadWriteOnce – the volume can be mounted as read-write by a single node
|
||||
- ReadOnlyMany – the volume can be mounted read-only by many nodes
|
||||
- ReadWriteMany – the volume can be mounted as read-write by many nodes
|
||||
|
||||
These access modes are enforced by Kubernetes components like the `kube-controller-manager` and `kubelet` to ensure only certain pods are allowed to access a given PersistentVolume.
|
||||
|
||||
## What is this new access mode and how does it work?
|
||||
|
||||
Kubernetes v1.22 introduced a fourth access mode for PVs and PVCs, that you can use for CSI volumes:
|
||||
|
||||
- ReadWriteOncePod – the volume can be mounted as read-write by a single pod
|
||||
|
||||
If you create a pod with a PVC that uses the ReadWriteOncePod access mode, Kubernetes ensures that pod is the only pod across your whole cluster that can read that PVC or write to it.
|
||||
|
||||
If you create another pod that references the same PVC with this access mode, the pod will fail to start because the PVC is already in use by another pod.
|
||||
For example:
|
||||
|
||||
```
|
||||
Events:
|
||||
Type Reason Age From Message
|
||||
---- ------ ---- ---- -------
|
||||
Warning FailedScheduling 1s default-scheduler 0/1 nodes are available: 1 node has pod using PersistentVolumeClaim with the same name and ReadWriteOncePod access mode.
|
||||
```
|
||||
|
||||
### How is this different than the ReadWriteOnce access mode?
|
||||
|
||||
The ReadWriteOnce access mode restricts volume access to a single *node*, which means it is possible for multiple pods on the same node to read from and write to the same volume.
|
||||
This could potentially be a major problem for some applications, especially if they require at most one writer for data safety guarantees.
|
||||
|
||||
With ReadWriteOncePod these issues go away.
|
||||
Set the access mode on your PVC, and Kubernetes guarantees that only a single pod has access.
|
||||
|
||||
## How do I use it?
|
||||
|
||||
The ReadWriteOncePod access mode is in alpha for Kubernetes v1.22 and is only supported for CSI volumes.
|
||||
As a first step you need to enable the ReadWriteOncePod [feature gate](/docs/reference/command-line-tools-reference/feature-gates) for `kube-apiserver`, `kube-scheduler`, and `kubelet`.
|
||||
You can enable the feature by setting command line arguments:
|
||||
|
||||
```
|
||||
--feature-gates="...,ReadWriteOncePod=true"
|
||||
```
|
||||
|
||||
You also need to update the following CSI sidecars to these versions or greater:
|
||||
|
||||
- [csi-provisioner:v3.0.0+](https://github.com/kubernetes-csi/external-provisioner/releases/tag/v3.0.0)
|
||||
- [csi-attacher:v3.3.0+](https://github.com/kubernetes-csi/external-attacher/releases/tag/v3.3.0)
|
||||
- [csi-resizer:v1.3.0+](https://github.com/kubernetes-csi/external-resizer/releases/tag/v1.3.0)
|
||||
|
||||
### Creating a PersistentVolumeClaim
|
||||
|
||||
In order to use the ReadWriteOncePod access mode for your PVs and PVCs, you will need to create a new PVC with the access mode:
|
||||
|
||||
```yaml
|
||||
kind: PersistentVolumeClaim
|
||||
apiVersion: v1
|
||||
metadata:
|
||||
name: single-writer-only
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOncePod # Allow only a single pod to access single-writer-only.
|
||||
resources:
|
||||
requests:
|
||||
storage: 1Gi
|
||||
```
|
||||
|
||||
If your storage plugin supports [dynamic provisioning](/docs/concepts/storage/dynamic-provisioning/), new PersistentVolumes will be created with the ReadWriteOncePod access mode applied.
|
||||
|
||||
#### Migrating existing PersistentVolumes
|
||||
|
||||
If you have existing PersistentVolumes, they can be migrated to use ReadWriteOncePod.
|
||||
|
||||
In this example, we already have a "cat-pictures-pvc" PersistentVolumeClaim that is bound to a "cat-pictures-pv" PersistentVolume, and a "cat-pictures-writer" Deployment that uses this PersistentVolumeClaim.
|
||||
|
||||
As a first step, you need to edit your PersistentVolume's `spec.persistentVolumeReclaimPolicy` and set it to `Retain`.
|
||||
This ensures your PersistentVolume will not be deleted when we delete the corresponding PersistentVolumeClaim:
|
||||
|
||||
```shell
|
||||
kubectl patch pv cat-pictures-pv -p '{"spec":{"persistentVolumeReclaimPolicy":"Retain"}}'
|
||||
```
|
||||
|
||||
Next you need to stop any workloads that are using the PersistentVolumeClaim bound to the PersistentVolume you want to migrate, and then delete the PersistentVolumeClaim.
|
||||
|
||||
Once that is done, you need to clear your PersistentVolume's `spec.claimRef.uid` to ensure PersistentVolumeClaims can bind to it upon recreation:
|
||||
|
||||
```shell
|
||||
kubectl scale --replicas=0 deployment cat-pictures-writer
|
||||
kubectl delete pvc cat-pictures-pvc
|
||||
kubectl patch pv cat-pictures-pv -p '{"spec":{"claimRef":{"uid":""}}}'
|
||||
```
|
||||
|
||||
After that you need to replace the PersistentVolume's access modes with ReadWriteOncePod:
|
||||
|
||||
```shell
|
||||
kubectl patch pv cat-pictures-pv -p '{"spec":{"accessModes":["ReadWriteOncePod"]}}'
|
||||
```
|
||||
|
||||
{{< note >}}
|
||||
The ReadWriteOncePod access mode cannot be combined with other access modes.
|
||||
Make sure ReadWriteOncePod is the only access mode on the PersistentVolume when updating, otherwise the request will fail.
|
||||
{{< /note >}}
|
||||
|
||||
Next you need to modify your PersistentVolumeClaim to set ReadWriteOncePod as the only access mode.
|
||||
You should also set your PersistentVolumeClaim's `spec.volumeName` to the name of your PersistentVolume.
|
||||
|
||||
Once this is done, you can recreate your PersistentVolumeClaim and start up your workloads:
|
||||
|
||||
```shell
|
||||
# IMPORTANT: Make sure to edit your PVC in cat-pictures-pvc.yaml before applying. You need to:
|
||||
# - Set ReadWriteOncePod as the only access mode
|
||||
# - Set spec.volumeName to "cat-pictures-pv"
|
||||
|
||||
kubectl apply -f cat-pictures-pvc.yaml
|
||||
kubectl apply -f cat-pictures-writer-deployment.yaml
|
||||
```
|
||||
|
||||
Lastly you may edit your PersistentVolume's `spec.persistentVolumeReclaimPolicy` and set to it back to `Delete` if you previously changed it.
|
||||
|
||||
```shell
|
||||
kubectl patch pv cat-pictures-pv -p '{"spec":{"persistentVolumeReclaimPolicy":"Delete"}}'
|
||||
```
|
||||
|
||||
You can read [Configure a Pod to Use a PersistentVolume for Storage](/docs/tasks/configure-pod-container/configure-persistent-volume-storage/) for more details on working with PersistentVolumes and PersistentVolumeClaims.
|
||||
|
||||
## What volume plugins support this?
|
||||
|
||||
The only volume plugins that support this are CSI drivers.
|
||||
SIG Storage does not plan to support this for in-tree plugins because they are being deprecated as part of [CSI migration](/blog/2019/12/09/kubernetes-1-17-feature-csi-migration-beta/#what-is-the-timeline-status).
|
||||
Support may be considered for beta for users that prefer to use the legacy in-tree volume APIs with CSI migration enabled.
|
||||
|
||||
## As a storage vendor, how do I add support for this access mode to my CSI driver?
|
||||
|
||||
The ReadWriteOncePod access mode will work out of the box without any required updates to CSI drivers, but [does require updates to CSI sidecars](#update-your-csi-sidecars).
|
||||
With that being said, if you would like to stay up to date with the latest changes to the CSI specification (v1.5.0+), read on.
|
||||
|
||||
Two new access modes were introduced to the CSI specification in order to disambiguate the legacy [`SINGLE_NODE_WRITER`](https://github.com/container-storage-interface/spec/blob/v1.5.0/csi.proto#L418-L420) access mode.
|
||||
They are [`SINGLE_NODE_SINGLE_WRITER` and `SINGLE_NODE_MULTI_WRITER`](https://github.com/container-storage-interface/spec/blob/v1.5.0/csi.proto#L437-L447).
|
||||
In order to communicate to sidecars (like the [external-provisioner](https://github.com/kubernetes-csi/external-provisioner)) that your driver understands and accepts these two new CSI access modes, your driver will also need to advertise the `SINGLE_NODE_MULTI_WRITER` capability for the [controller service](https://github.com/container-storage-interface/spec/blob/v1.5.0/csi.proto#L1073-L1081) and [node service](https://github.com/container-storage-interface/spec/blob/v1.5.0/csi.proto#L1515-L1524).
|
||||
|
||||
If you'd like to read up on the motivation for these access modes and capability bits, you can also read the [CSI Specification Changes, Volume Capabilities](https://github.com/kubernetes/enhancements/blob/master/keps/sig-storage/2485-read-write-once-pod-pv-access-mode/README.md#csi-specification-changes-volume-capabilities) section of KEP-2485 (ReadWriteOncePod PersistentVolume Access Mode).
|
||||
|
||||
### Update your CSI driver to use the new interface
|
||||
|
||||
As a first step you will need to update your driver's `container-storage-interface` dependency to v1.5.0+, which contains support for these new access modes and capabilities.
|
||||
|
||||
### Accept new CSI access modes
|
||||
|
||||
If your CSI driver contains logic for validating CSI access modes for requests , it may need updating.
|
||||
If it currently accepts `SINGLE_NODE_WRITER`, it should be updated to also accept `SINGLE_NODE_SINGLE_WRITER` and `SINGLE_NODE_MULTI_WRITER`.
|
||||
|
||||
Using the [GCP PD CSI driver validation logic](https://github.com/kubernetes-sigs/gcp-compute-persistent-disk-csi-driver/blob/v1.2.2/pkg/gce-pd-csi-driver/utils.go#L116-L130) as an example, here is how it can be extended:
|
||||
|
||||
```diff
|
||||
diff --git a/pkg/gce-pd-csi-driver/utils.go b/pkg/gce-pd-csi-driver/utils.go
|
||||
index 281242c..b6c5229 100644
|
||||
--- a/pkg/gce-pd-csi-driver/utils.go
|
||||
+++ b/pkg/gce-pd-csi-driver/utils.go
|
||||
@@ -123,6 +123,8 @@ func validateAccessMode(am *csi.VolumeCapability_AccessMode) error {
|
||||
case csi.VolumeCapability_AccessMode_SINGLE_NODE_READER_ONLY:
|
||||
case csi.VolumeCapability_AccessMode_MULTI_NODE_READER_ONLY:
|
||||
case csi.VolumeCapability_AccessMode_MULTI_NODE_MULTI_WRITER:
|
||||
+ case csi.VolumeCapability_AccessMode_SINGLE_NODE_SINGLE_WRITER:
|
||||
+ case csi.VolumeCapability_AccessMode_SINGLE_NODE_MULTI_WRITER:
|
||||
default:
|
||||
return fmt.Errorf("%v access mode is not supported for for PD", am.GetMode())
|
||||
}
|
||||
```
|
||||
|
||||
### Advertise new CSI controller and node service capabilities
|
||||
|
||||
Your CSI driver will also need to return the new `SINGLE_NODE_MULTI_WRITER` capability as part of the `ControllerGetCapabilities` and `NodeGetCapabilities` RPCs.
|
||||
|
||||
Using the [GCP PD CSI driver capability advertisement logic](https://github.com/kubernetes-sigs/gcp-compute-persistent-disk-csi-driver/blob/v1.2.2/pkg/gce-pd-csi-driver/gce-pd-driver.go#L54-L77) as an example, here is how it can be extended:
|
||||
|
||||
```diff
|
||||
diff --git a/pkg/gce-pd-csi-driver/gce-pd-driver.go b/pkg/gce-pd-csi-driver/gce-pd-driver.go
|
||||
index 45903f3..0d7ea26 100644
|
||||
--- a/pkg/gce-pd-csi-driver/gce-pd-driver.go
|
||||
+++ b/pkg/gce-pd-csi-driver/gce-pd-driver.go
|
||||
@@ -56,6 +56,8 @@ func (gceDriver *GCEDriver) SetupGCEDriver(name, vendorVersion string, extraVolu
|
||||
csi.VolumeCapability_AccessMode_SINGLE_NODE_WRITER,
|
||||
csi.VolumeCapability_AccessMode_MULTI_NODE_READER_ONLY,
|
||||
csi.VolumeCapability_AccessMode_MULTI_NODE_MULTI_WRITER,
|
||||
+ csi.VolumeCapability_AccessMode_SINGLE_NODE_SINGLE_WRITER,
|
||||
+ csi.VolumeCapability_AccessMode_SINGLE_NODE_MULTI_WRITER,
|
||||
}
|
||||
gceDriver.AddVolumeCapabilityAccessModes(vcam)
|
||||
csc := []csi.ControllerServiceCapability_RPC_Type{
|
||||
@@ -67,12 +69,14 @@ func (gceDriver *GCEDriver) SetupGCEDriver(name, vendorVersion string, extraVolu
|
||||
csi.ControllerServiceCapability_RPC_EXPAND_VOLUME,
|
||||
csi.ControllerServiceCapability_RPC_LIST_VOLUMES,
|
||||
csi.ControllerServiceCapability_RPC_LIST_VOLUMES_PUBLISHED_NODES,
|
||||
+ csi.ControllerServiceCapability_RPC_SINGLE_NODE_MULTI_WRITER,
|
||||
}
|
||||
gceDriver.AddControllerServiceCapabilities(csc)
|
||||
ns := []csi.NodeServiceCapability_RPC_Type{
|
||||
csi.NodeServiceCapability_RPC_STAGE_UNSTAGE_VOLUME,
|
||||
csi.NodeServiceCapability_RPC_EXPAND_VOLUME,
|
||||
csi.NodeServiceCapability_RPC_GET_VOLUME_STATS,
|
||||
+ csi.NodeServiceCapability_RPC_SINGLE_NODE_MULTI_WRITER,
|
||||
}
|
||||
gceDriver.AddNodeServiceCapabilities(ns)
|
||||
```
|
||||
|
||||
### Implement `NodePublishVolume` behavior
|
||||
|
||||
The CSI spec outlines expected behavior for the `NodePublishVolume` RPC when called more than once for the same volume but with different arguments (like the target path).
|
||||
Please refer to [the second table in the NodePublishVolume section of the CSI spec](https://github.com/container-storage-interface/spec/blob/v1.5.0/spec.md#nodepublishvolume) for more details on expected behavior when implementing in your driver.
|
||||
|
||||
### Update your CSI sidecars
|
||||
|
||||
When deploying your CSI drivers, you must update the following CSI sidecars to versions that depend on CSI spec v1.5.0+ and the Kubernetes v1.22 API.
|
||||
The minimum required versions are:
|
||||
|
||||
- [csi-provisioner:v3.0.0+](https://github.com/kubernetes-csi/external-provisioner/releases/tag/v3.0.0)
|
||||
- [csi-attacher:v3.3.0+](https://github.com/kubernetes-csi/external-attacher/releases/tag/v3.3.0)
|
||||
- [csi-resizer:v1.3.0+](https://github.com/kubernetes-csi/external-resizer/releases/tag/v1.3.0)
|
||||
|
||||
## What’s next?
|
||||
|
||||
As part of the beta graduation for this feature, SIG Storage plans to update the Kubenetes scheduler to support pod preemption in relation to ReadWriteOncePod storage.
|
||||
This means if two pods request a PersistentVolumeClaim with ReadWriteOncePod, the pod with highest priority will gain access to the PersistentVolumeClaim and any pod with lower priority will be preempted from the node and be unable to access the PersistentVolumeClaim.
|
||||
|
||||
## How can I learn more?
|
||||
|
||||
Please see [KEP-2485](https://github.com/kubernetes/enhancements/blob/master/keps/sig-storage/2485-read-write-once-pod-pv-access-mode/README.md) for more details on the ReadWriteOncePod access mode and motivations for CSI spec changes.
|
||||
|
||||
## How do I get involved?
|
||||
|
||||
The [Kubernetes #csi Slack channel](https://kubernetes.slack.com/messages/csi) and any of the [standard SIG Storage communication channels](https://github.com/kubernetes/community/blob/master/sig-storage/README.md#contact) are great mediums to reach out to the SIG Storage and the CSI teams.
|
||||
|
||||
Special thanks to the following people for their insightful reviews and design considerations:
|
||||
|
||||
* Abdullah Gharaibeh (ahg-g)
|
||||
* Aldo Culquicondor (alculquicondor)
|
||||
* Ben Swartzlander (bswartz)
|
||||
* Deep Debroy (ddebroy)
|
||||
* Hemant Kumar (gnufied)
|
||||
* Humble Devassy Chirammal (humblec)
|
||||
* James DeFelice (jdef)
|
||||
* Jan Šafránek (jsafrane)
|
||||
* Jing Xu (jingxu97)
|
||||
* Jordan Liggitt (liggitt)
|
||||
* Michelle Au (msau42)
|
||||
* Saad Ali (saad-ali)
|
||||
* Tim Hockin (thockin)
|
||||
* Xing Yang (xing-yang)
|
||||
|
||||
If you’re interested in getting involved with the design and development of CSI or any part of the Kubernetes storage system, join the [Kubernetes Storage Special Interest Group](https://github.com/kubernetes/community/tree/master/sig-storage) (SIG).
|
||||
We’re rapidly growing and always welcome new contributors.
|
||||
@@ -91,18 +91,6 @@ imply any preferential status.
|
||||
Project [Antrea](https://github.com/vmware-tanzu/antrea) is an opensource Kubernetes networking solution intended to be Kubernetes native. It leverages Open vSwitch as the networking data plane. Open vSwitch is a high-performance programmable virtual switch that supports both Linux and Windows. Open vSwitch enables Antrea to implement Kubernetes Network Policies in a high-performance and efficient manner.
|
||||
Thanks to the "programmable" characteristic of Open vSwitch, Antrea is able to implement an extensive set of networking and security features and services on top of Open vSwitch.
|
||||
|
||||
### AOS from Apstra
|
||||
|
||||
[AOS](https://www.apstra.com/products/aos/) is an Intent-Based Networking system that creates and manages complex datacenter environments from a simple integrated platform. AOS leverages a highly scalable distributed design to eliminate network outages while minimizing costs.
|
||||
|
||||
The AOS Reference Design currently supports Layer-3 connected hosts that eliminate legacy Layer-2 switching problems. These Layer-3 hosts can be Linux servers (Debian, Ubuntu, CentOS) that create BGP neighbor relationships directly with the top of rack switches (TORs). AOS automates the routing adjacencies and then provides fine grained control over the route health injections (RHI) that are common in a Kubernetes deployment.
|
||||
|
||||
AOS has a rich set of REST API endpoints that enable Kubernetes to quickly change the network policy based on application requirements. Further enhancements will integrate the AOS Graph model used for the network design with the workload provisioning, enabling an end to end management system for both private and public clouds.
|
||||
|
||||
AOS supports the use of common vendor equipment from manufacturers including Cisco, Arista, Dell, Mellanox, HPE, and a large number of white-box systems and open network operating systems like Microsoft SONiC, Dell OPX, and Cumulus Linux.
|
||||
|
||||
Details on how the AOS system works can be accessed here: https://www.apstra.com/products/how-it-works/
|
||||
|
||||
### AWS VPC CNI for Kubernetes
|
||||
|
||||
The [AWS VPC CNI](https://github.com/aws/amazon-vpc-cni-k8s) offers integrated AWS Virtual Private Cloud (VPC) networking for Kubernetes clusters. This CNI plugin offers high throughput and availability, low latency, and minimal network jitter. Additionally, users can apply existing AWS VPC networking and security best practices for building Kubernetes clusters. This includes the ability to use VPC flow logs, VPC routing policies, and security groups for network traffic isolation.
|
||||
@@ -116,15 +104,6 @@ Additionally, the CNI can be run alongside [Calico for network policy enforcemen
|
||||
|
||||
Azure CNI is available natively in the [Azure Kubernetes Service (AKS)](https://docs.microsoft.com/en-us/azure/aks/configure-azure-cni).
|
||||
|
||||
|
||||
### Big Cloud Fabric from Big Switch Networks
|
||||
|
||||
[Big Cloud Fabric](https://www.bigswitch.com/container-network-automation) is a cloud native networking architecture, designed to run Kubernetes in private cloud/on-premises environments. Using unified physical & virtual SDN, Big Cloud Fabric tackles inherent container networking problems such as load balancing, visibility, troubleshooting, security policies & container traffic monitoring.
|
||||
|
||||
With the help of the Big Cloud Fabric's virtual pod multi-tenant architecture, container orchestration systems such as Kubernetes, RedHat OpenShift, Mesosphere DC/OS & Docker Swarm will be natively integrated alongside with VM orchestration systems such as VMware, OpenStack & Nutanix. Customers will be able to securely inter-connect any number of these clusters and enable inter-tenant communication between them if needed.
|
||||
|
||||
BCF was recognized by Gartner as a visionary in the latest [Magic Quadrant](https://go.bigswitch.com/17GatedDocuments-MagicQuadrantforDataCenterNetworking_Reg.html). One of the BCF Kubernetes on-premises deployments (which includes Kubernetes, DC/OS & VMware running on multiple DCs across different geographic regions) is also referenced [here](https://portworx.com/architects-corner-kubernetes-satya-komala-nio/).
|
||||
|
||||
### Calico
|
||||
|
||||
[Calico](https://docs.projectcalico.org/) is an open source networking and network security solution for containers, virtual machines, and native host-based workloads. Calico supports multiple data planes including: a pure Linux eBPF dataplane, a standard Linux networking dataplane, and a Windows HNS dataplane. Calico provides a full networking stack but can also be used in conjunction with [cloud provider CNIs](https://docs.projectcalico.org/networking/determine-best-networking#calico-compatible-cni-plugins-and-cloud-provider-integrations) to provide network policy enforcement.
|
||||
|
||||
@@ -11,7 +11,7 @@ weight: 20
|
||||
<!-- overview -->
|
||||
|
||||
The aggregation layer allows Kubernetes to be extended with additional APIs, beyond what is offered by the core Kubernetes APIs.
|
||||
The additional APIs can either be ready-made solutions such as [service-catalog](/docs/concepts/extend-kubernetes/service-catalog/), or APIs that you develop yourself.
|
||||
The additional APIs can either be ready-made solutions such as a [metrics server](https://github.com/kubernetes-sigs/metrics-server), or APIs that you develop yourself.
|
||||
|
||||
The aggregation layer is different from [Custom Resources](/docs/concepts/extend-kubernetes/api-extension/custom-resources/), which are a way to make the {{< glossary_tooltip term_id="kube-apiserver" text="kube-apiserver" >}} recognise new kinds of object.
|
||||
|
||||
|
||||
@@ -32,7 +32,7 @@ The application can access the message queue as a service.
|
||||
|
||||
Service Catalog uses the [Open service broker API](https://github.com/openservicebrokerapi/servicebroker) to communicate with service brokers, acting as an intermediary for the Kubernetes API Server to negotiate the initial provisioning and retrieve the credentials necessary for the application to use a managed service.
|
||||
|
||||
It is implemented as an extension API server and a controller, using etcd for storage. It also uses the [aggregation layer](/docs/concepts/extend-kubernetes/api-extension/apiserver-aggregation/) available in Kubernetes 1.7+ to present its API.
|
||||
It is implemented using a [CRDs-based](/docs/concepts/extend-kubernetes/api-extension/custom-resources/#custom-resources) architecture.
|
||||
|
||||
<br>
|
||||
|
||||
|
||||
@@ -444,8 +444,7 @@ variables and DNS.
|
||||
|
||||
When a Pod is run on a Node, the kubelet adds a set of environment variables
|
||||
for each active Service. It supports both [Docker links
|
||||
compatible](https://docs.docker.com/userguide/dockerlinks/) variables (see
|
||||
[makeLinkVariables](https://releases.k8s.io/{{< param "fullversion" >}}/pkg/kubelet/envvars/envvars.go#L49))
|
||||
compatible](https://docs.docker.com/userguide/dockerlinks/) variables (see [makeLinkVariables](https://github.com/kubernetes/kubernetes/blob/dd2d12f6dc0e654c15d5db57a5f9f6ba61192726/pkg/kubelet/envvars/envvars.go#L72))
|
||||
and simpler `{SVCNAME}_SERVICE_HOST` and `{SVCNAME}_SERVICE_PORT` variables,
|
||||
where the Service name is upper-cased and dashes are converted to underscores.
|
||||
|
||||
|
||||
@@ -185,7 +185,7 @@ You can modify the Pods that a DaemonSet creates. However, Pods do not allow al
|
||||
fields to be updated. Also, the DaemonSet controller will use the original template the next
|
||||
time a node (even with the same name) is created.
|
||||
|
||||
You can delete a DaemonSet. If you specify `--cascade=false` with `kubectl`, then the Pods
|
||||
You can delete a DaemonSet. If you specify `--cascade=orphan` with `kubectl`, then the Pods
|
||||
will be left on the nodes. If you subsequently create a new DaemonSet with the same selector,
|
||||
the new DaemonSet adopts the existing Pods. If any Pods need replacing the DaemonSet replaces
|
||||
them according to its `updateStrategy`.
|
||||
|
||||
@@ -523,7 +523,7 @@ to keep running, but you want the rest of the Pods it creates
|
||||
to use a different pod template and for the Job to have a new name.
|
||||
You cannot update the Job because these fields are not updatable.
|
||||
Therefore, you delete Job `old` but _leave its pods
|
||||
running_, using `kubectl delete jobs/old --cascade=false`.
|
||||
running_, using `kubectl delete jobs/old --cascade=orphan`.
|
||||
Before deleting it, you make a note of what selector it uses:
|
||||
|
||||
```shell
|
||||
|
||||
@@ -192,7 +192,7 @@ When using the REST API or Go client library, you need to do the steps explicitl
|
||||
|
||||
You can delete a ReplicationController without affecting any of its pods.
|
||||
|
||||
Using kubectl, specify the `--cascade=false` option to [`kubectl delete`](/docs/reference/generated/kubectl/kubectl-commands#delete).
|
||||
Using kubectl, specify the `--cascade=orphan` option to [`kubectl delete`](/docs/reference/generated/kubectl/kubectl-commands#delete).
|
||||
|
||||
When using the REST API or Go client library, you can delete the ReplicationController object.
|
||||
|
||||
|
||||
@@ -173,7 +173,7 @@ Cluster Domain will be set to `cluster.local` unless
|
||||
|
||||
### Stable Storage
|
||||
|
||||
For each VolumeClaimTemplate entry defined in a StatefulSet, each Pod receives one PersistentVolumeClaim. In the nginx example above, each Podreceives a single PersistentVolume with a StorageClass of `my-storage-class` and 1 Gib of provisioned storage. If no StorageClass
|
||||
For each VolumeClaimTemplate entry defined in a StatefulSet, each Pod receives one PersistentVolumeClaim. In the nginx example above, each Pod receives a single PersistentVolume with a StorageClass of `my-storage-class` and 1 Gib of provisioned storage. If no StorageClass
|
||||
is specified, then the default StorageClass will be used. When a Pod is (re)scheduled
|
||||
onto a node, its `volumeMounts` mount the PersistentVolumes associated with its
|
||||
PersistentVolume Claims. Note that, the PersistentVolumes associated with the
|
||||
@@ -301,4 +301,3 @@ Please note that this field only works if you enable the `StatefulSetMinReadySec
|
||||
* Follow an example of [deploying a stateful application](/docs/tutorials/stateful-application/basic-stateful-set/).
|
||||
* Follow an example of [deploying Cassandra with Stateful Sets](/docs/tutorials/stateful-application/cassandra/).
|
||||
* Follow an example of [running a replicated stateful application](/docs/tasks/run-application/run-replicated-stateful-application/).
|
||||
|
||||
|
||||
@@ -230,20 +230,9 @@ If you apply "two-constraints.yaml" to this cluster, you will notice "mypod" sta
|
||||
|
||||
To overcome this situation, you can either increase the `maxSkew` or modify one of the constraints to use `whenUnsatisfiable: ScheduleAnyway`.
|
||||
|
||||
### Conventions
|
||||
### Interaction With Node Affinity and Node Selectors
|
||||
|
||||
There are some implicit conventions worth noting here:
|
||||
|
||||
- Only the Pods holding the same namespace as the incoming Pod can be matching candidates.
|
||||
|
||||
- Nodes without `topologySpreadConstraints[*].topologyKey` present will be bypassed. It implies that:
|
||||
|
||||
1. the Pods located on those nodes do not impact `maxSkew` calculation - in the above example, suppose "node1" does not have label "zone", then the 2 Pods will be disregarded, hence the incoming Pod will be scheduled into "zoneA".
|
||||
2. the incoming Pod has no chances to be scheduled onto this kind of nodes - in the above example, suppose a "node5" carrying label `{zone-typo: zoneC}` joins the cluster, it will be bypassed due to the absence of label key "zone".
|
||||
|
||||
- Be aware of what will happen if the incomingPod's `topologySpreadConstraints[*].labelSelector` doesn't match its own labels. In the above example, if we remove the incoming Pod's labels, it can still be placed onto "zoneB" since the constraints are still satisfied. However, after the placement, the degree of imbalance of the cluster remains unchanged - it's still zoneA having 2 Pods which hold label {foo:bar}, and zoneB having 1 Pod which holds label {foo:bar}. So if this is not what you expect, we recommend the workload's `topologySpreadConstraints[*].labelSelector` to match its own labels.
|
||||
|
||||
- If the incoming Pod has `spec.nodeSelector` or `spec.affinity.nodeAffinity` defined, nodes not matching them will be bypassed.
|
||||
The scheduler will skip the non-matching nodes from the skew calculations if the incoming Pod has `spec.nodeSelector` or `spec.affinity.nodeAffinity` defined.
|
||||
|
||||
Suppose you have a 5-node cluster ranging from zoneA to zoneC:
|
||||
|
||||
@@ -283,6 +272,21 @@ There are some implicit conventions worth noting here:
|
||||
|
||||
{{< codenew file="pods/topology-spread-constraints/one-constraint-with-nodeaffinity.yaml" >}}
|
||||
|
||||
The scheduler doesn't have prior knowledge of all the zones or other topology domains that a cluster has. They are determined from the existing nodes in the cluster. This could lead to a problem in autoscaled clusters, when a node pool (or node group) is scaled to zero nodes and the user is expecting them to scale up, because, in this case, those topology domains won't be considered until there is at least one node in them.
|
||||
|
||||
### Other Noticeable Semantics
|
||||
|
||||
There are some implicit conventions worth noting here:
|
||||
|
||||
- Only the Pods holding the same namespace as the incoming Pod can be matching candidates.
|
||||
|
||||
- The scheduler will bypass the nodes without `topologySpreadConstraints[*].topologyKey` present. This implies that:
|
||||
|
||||
1. the Pods located on those nodes do not impact `maxSkew` calculation - in the above example, suppose "node1" does not have label "zone", then the 2 Pods will be disregarded, hence the incoming Pod will be scheduled into "zoneA".
|
||||
2. the incoming Pod has no chances to be scheduled onto this kind of nodes - in the above example, suppose a "node5" carrying label `{zone-typo: zoneC}` joins the cluster, it will be bypassed due to the absence of label key "zone".
|
||||
|
||||
- Be aware of what will happen if the incomingPod's `topologySpreadConstraints[*].labelSelector` doesn't match its own labels. In the above example, if we remove the incoming Pod's labels, it can still be placed onto "zoneB" since the constraints are still satisfied. However, after the placement, the degree of imbalance of the cluster remains unchanged - it's still zoneA having 2 Pods which hold label {foo:bar}, and zoneB having 1 Pod which holds label {foo:bar}. So if this is not what you expect, we recommend the workload's `topologySpreadConstraints[*].labelSelector` to match its own labels.
|
||||
|
||||
### Cluster-level default constraints
|
||||
|
||||
It is possible to set default topology spread constraints for a cluster. Default
|
||||
|
||||
@@ -83,6 +83,33 @@ You can also include a full definition:
|
||||
which renders as:
|
||||
{{< glossary_definition term_id="cluster" length="all" >}}
|
||||
|
||||
## Links to API Reference
|
||||
|
||||
You can link to a page of the Kubernetes API reference using the `api-reference` shortcode, for example to the {{< api-reference page="workload-resources/pod-v1" >}} reference:
|
||||
|
||||
```
|
||||
{{</* api-reference page="workload-resources/pod-v1" */>}}
|
||||
```
|
||||
|
||||
The content of the `page` parameter is the suffix of the URL of the API reference page.
|
||||
|
||||
|
||||
You can link to a specific place into a page by specifying an `anchor` parameter, for example to the {{< api-reference page="workload-resources/pod-v1" anchor="PodSpec" >}} reference or the {{< api-reference page="workload-resources/pod-v1" anchor="environment-variables" >}} section of the page:
|
||||
|
||||
```
|
||||
{{</* api-reference page="workload-resources/pod-v1" anchor="PodSpec" */>}}
|
||||
{{</* api-reference page="workload-resources/pod-v1" anchor="environment-variables" */>}}
|
||||
```
|
||||
|
||||
|
||||
You can change the text of the link by specifying a `text` parameter, for example by linking to the {{< api-reference page="workload-resources/pod-v1" anchor="environment-variables" text="Environment Variables">}} section of the page:
|
||||
|
||||
```
|
||||
{{</* api-reference page="workload-resources/pod-v1" anchor="environment-variables" text="Environment Variable" */>}}
|
||||
```
|
||||
|
||||
|
||||
|
||||
## Table captions
|
||||
|
||||
You can make tables more accessible to screen readers by adding a table caption. To add a [caption](https://www.w3schools.com/tags/tag_caption.asp) to a table, enclose the table with a `table` shortcode and specify the caption with the `caption` parameter.
|
||||
|
||||
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 175 KiB |
@@ -64,6 +64,8 @@ client libraries:
|
||||
* [Scheduler Policies](/docs/reference/scheduling/policies)
|
||||
* [Scheduler Profiles](/docs/reference/scheduling/config#profiles)
|
||||
|
||||
* List of [ports and protocols](/docs/reference/ports-and-protocols/) that
|
||||
should be open on control plane and worker nodes
|
||||
## Config APIs
|
||||
|
||||
This section hosts the documentation for "unpublished" APIs which are used to
|
||||
|
||||
@@ -72,7 +72,7 @@ The ServiceAccount admission controller will add the following projected volume
|
||||
defaultMode: 420 # 0644
|
||||
sources:
|
||||
- serviceAccountToken:
|
||||
expirationSeconds: 3600
|
||||
expirationSeconds: 3607
|
||||
path: token
|
||||
- configMap:
|
||||
items:
|
||||
|
||||
@@ -75,7 +75,7 @@ If you need help, run `kubectl help` from the terminal window.
|
||||
|
||||
By default `kubectl` will first determine if it is running within a pod, and thus in a cluster. It starts by checking for the `KUBERNETES_SERVICE_HOST` and `KUBERNETES_SERVICE_PORT` environment variables and the existence of a service account token file at `/var/run/secrets/kubernetes.io/serviceaccount/token`. If all three are found in-cluster authentication is assumed.
|
||||
|
||||
To maintain backwards compatibility, if the `POD_NAMESPACE` environment variable is set during in-cluster authentication it will override the default namespace from the from the service account token. Any manifests or tools relying on namespace defaulting will be affected by this.
|
||||
To maintain backwards compatibility, if the `POD_NAMESPACE` environment variable is set during in-cluster authentication it will override the default namespace from the service account token. Any manifests or tools relying on namespace defaulting will be affected by this.
|
||||
|
||||
**`POD_NAMESPACE` environment variable**
|
||||
|
||||
|
||||
@@ -0,0 +1,40 @@
|
||||
---
|
||||
title: Ports and Protocols
|
||||
content_type: reference
|
||||
weight: 50
|
||||
---
|
||||
|
||||
When running Kubernetes in an environment with strict network boundaries, such
|
||||
as on-premises datacenter with physical network firewalls or Virtual
|
||||
Networks in Public Cloud, it is useful to be aware of the ports and protocols
|
||||
used by Kubernetes components
|
||||
|
||||
## Control plane
|
||||
|
||||
| Protocol | Direction | Port Range | Purpose | Used By |
|
||||
|----------|-----------|------------|-------------------------|---------------------------|
|
||||
| TCP | Inbound | 6443 | Kubernetes API server | All |
|
||||
| TCP | Inbound | 2379-2380 | etcd server client API | kube-apiserver, etcd |
|
||||
| TCP | Inbound | 10250 | Kubelet API | Self, Control plane |
|
||||
| TCP | Inbound | 10259 | kube-scheduler | Self |
|
||||
| TCP | Inbound | 10257 | kube-controller-manager | Self |
|
||||
|
||||
Although etcd ports are included in control plane section, you can also host your own
|
||||
etcd cluster externally or on custom ports.
|
||||
|
||||
## Worker node(s) {#node}
|
||||
|
||||
| Protocol | Direction | Port Range | Purpose | Used By |
|
||||
|----------|-----------|-------------|-----------------------|-------------------------|
|
||||
| TCP | Inbound | 10250 | Kubelet API | Self, Control plane |
|
||||
| TCP | Inbound | 30000-32767 | NodePort Services† | All |
|
||||
|
||||
† Default port range for [NodePort Services](/docs/concepts/services-networking/service/).
|
||||
|
||||
All default port numbers can be overridden. When custom ports are used those
|
||||
ports need to be open instead of defaults mentioned here.
|
||||
|
||||
One common example is API server port that is sometimes switched
|
||||
to 443. Alternatively, the default port is kept as is and API server is put
|
||||
behind a load balancer that listens on 443 and routes the requests to API server
|
||||
on the default port.
|
||||
@@ -67,31 +67,9 @@ sudo sysctl --system
|
||||
For more details please see the [Network Plugin Requirements](/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins/#network-plugin-requirements) page.
|
||||
|
||||
## Check required ports
|
||||
|
||||
### Control-plane node(s)
|
||||
|
||||
| Protocol | Direction | Port Range | Purpose | Used By |
|
||||
|----------|-----------|------------|-------------------------|---------------------------|
|
||||
| TCP | Inbound | 6443\* | Kubernetes API server | All |
|
||||
| TCP | Inbound | 2379-2380 | etcd server client API | kube-apiserver, etcd |
|
||||
| TCP | Inbound | 10250 | kubelet API | Self, Control plane |
|
||||
| TCP | Inbound | 10251 | kube-scheduler | Self |
|
||||
| TCP | Inbound | 10252 | kube-controller-manager | Self |
|
||||
|
||||
### Worker node(s)
|
||||
|
||||
| Protocol | Direction | Port Range | Purpose | Used By |
|
||||
|----------|-----------|-------------|-----------------------|-------------------------|
|
||||
| TCP | Inbound | 10250 | kubelet API | Self, Control plane |
|
||||
| TCP | Inbound | 30000-32767 | NodePort Services† | All |
|
||||
|
||||
† Default port range for [NodePort Services](/docs/concepts/services-networking/service/).
|
||||
|
||||
Any port numbers marked with * are overridable, so you will need to ensure any
|
||||
custom ports you provide are also open.
|
||||
|
||||
Although etcd ports are included in control-plane nodes, you can also host your own
|
||||
etcd cluster externally or on custom ports.
|
||||
These
|
||||
[required ports](/docs/reference/ports-and-protocols/)
|
||||
need to be open in order for Kubernetes components to communicate with each other.
|
||||
|
||||
The pod network plugin you use (see below) may also require certain ports to be
|
||||
open. Since this differs with each pod network plugin, please see the
|
||||
|
||||
@@ -44,6 +44,46 @@ This page shows you how to set up a simple Ingress which routes requests to Serv
|
||||
|
||||
1. Verify that the NGINX Ingress controller is running
|
||||
|
||||
|
||||
{{< tabs name="tab_with_md" >}}
|
||||
{{% tab name="minikube v1.19 or later" %}}
|
||||
```shell
|
||||
kubectl get pods -n ingress-nginx
|
||||
```
|
||||
{{< note >}}This can take up to a minute.{{< /note >}}
|
||||
|
||||
Output:
|
||||
|
||||
```
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
ingress-nginx-admission-create-g9g49 0/1 Completed 0 11m
|
||||
ingress-nginx-admission-patch-rqp78 0/1 Completed 1 11m
|
||||
ingress-nginx-controller-59b45fb494-26npt 1/1 Running 0 11m
|
||||
```
|
||||
{{% /tab %}}
|
||||
|
||||
{{% tab name="minikube v1.18.1 or earlier" %}}
|
||||
```shell
|
||||
kubectl get pods -n kube-system
|
||||
```
|
||||
{{< note >}}This can take up to a minute.{{< /note >}}
|
||||
|
||||
Output:
|
||||
|
||||
```
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
default-http-backend-59868b7dd6-xb8tq 1/1 Running 0 1m
|
||||
kube-addon-manager-minikube 1/1 Running 0 3m
|
||||
kube-dns-6dcb57bcc8-n4xd4 3/3 Running 0 2m
|
||||
kubernetes-dashboard-5498ccf677-b8p5h 1/1 Running 0 2m
|
||||
nginx-ingress-controller-5984b97644-rnkrg 1/1 Running 0 1m
|
||||
storage-provisioner 1/1 Running 0 2m
|
||||
```
|
||||
{{% /tab %}}
|
||||
{{< /tabs >}}
|
||||
|
||||
|
||||
|
||||
```shell
|
||||
kubectl get pods -n ingress-nginx
|
||||
```
|
||||
@@ -59,6 +99,7 @@ This page shows you how to set up a simple Ingress which routes requests to Serv
|
||||
ingress-nginx-controller-59b45fb494-lzmw2 1/1 Running 0 3m28s
|
||||
```
|
||||
|
||||
|
||||
## Deploy a hello, world app
|
||||
|
||||
1. Create a Deployment using the following command:
|
||||
|
||||
@@ -133,10 +133,18 @@ the [etcd administration guide](https://etcd.io/docs/v2.3/admin_guide/#member-mi
|
||||
|
||||
## Implementation notes
|
||||
|
||||
|
||||
|
||||
|
||||
### Overview
|
||||
The figure above illustrates three control plane nodes and their components in a highly available cluster. The control plane node’s components employ the following methods:
|
||||
|
||||
- etcd: instances are clustered together using consensus.
|
||||
|
||||
- Controllers, scheduler and cluster auto-scaler: only one instance of each will be active in a cluster using a lease mechanism.
|
||||
|
||||
- Add-on manager: each works independently to keep add-ons in sync.
|
||||
|
||||
In addition, a load balancer operating in front of the API servers routes external and internal traffic to the control plane nodes.
|
||||
Each of the control plane nodes will run the following components in the following mode:
|
||||
|
||||
* etcd instance: all instances will be clustered together using consensus;
|
||||
@@ -215,4 +223,3 @@ server coordination (for example, the `StorageVersionAPI` feature gate).
|
||||
|
||||
[Automated HA master deployment - design doc](https://git.k8s.io/community/contributors/design-proposals/cluster-lifecycle/ha_master.md)
|
||||
|
||||
|
||||
|
||||
@@ -9,17 +9,17 @@ weight: 20
|
||||
<!-- overview -->
|
||||
|
||||
This page explains how to upgrade a Kubernetes cluster created with kubeadm from version
|
||||
{{< skew latestVersionAddMinor -1 >}}.x to version {{< skew latestVersion >}}.x, and from version
|
||||
{{< skew latestVersion >}}.x to {{< skew latestVersion >}}.y (where `y > x`). Skipping MINOR versions
|
||||
{{< skew currentVersionAddMinor -1 >}}.x to version {{< skew currentVersion >}}.x, and from version
|
||||
{{< skew currentVersion >}}.x to {{< skew currentVersion >}}.y (where `y > x`). Skipping MINOR versions
|
||||
when upgrading is unsupported.
|
||||
|
||||
To see information about upgrading clusters created using older versions of kubeadm,
|
||||
please refer to following pages instead:
|
||||
|
||||
- [Upgrading a kubeadm cluster from {{< skew latestVersionAddMinor -2 >}} to {{< skew latestVersionAddMinor -1 >}}](https://v{{< skew latestVersionAddMinor -1 "-" >}}.docs.kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/)
|
||||
- [Upgrading a kubeadm cluster from {{< skew latestVersionAddMinor -3 >}} to {{< skew latestVersionAddMinor -2 >}}](https://v{{< skew latestVersionAddMinor -2 "-" >}}.docs.kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/)
|
||||
- [Upgrading a kubeadm cluster from {{< skew latestVersionAddMinor -4 >}} to {{< skew latestVersionAddMinor -3 >}}](https://v{{< skew latestVersionAddMinor -3 "-" >}}.docs.kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/)
|
||||
- [Upgrading a kubeadm cluster from {{< skew latestVersionAddMinor -5 >}} to {{< skew latestVersionAddMinor -4 >}}](https://v{{< skew latestVersionAddMinor -4 "-" >}}.docs.kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/)
|
||||
- [Upgrading a kubeadm cluster from {{< skew currentVersionAddMinor -2 >}} to {{< skew currentVersionAddMinor -1 >}}](https://v{{< skew currentVersionAddMinor -1 "-" >}}.docs.kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/)
|
||||
- [Upgrading a kubeadm cluster from {{< skew currentVersionAddMinor -3 >}} to {{< skew currentVersionAddMinor -2 >}}](https://v{{< skew currentVersionAddMinor -2 "-" >}}.docs.kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/)
|
||||
- [Upgrading a kubeadm cluster from {{< skew currentVersionAddMinor -4 >}} to {{< skew currentVersionAddMinor -3 >}}](https://v{{< skew currentVersionAddMinor -3 "-" >}}.docs.kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/)
|
||||
- [Upgrading a kubeadm cluster from {{< skew currentVersionAddMinor -5 >}} to {{< skew currentVersionAddMinor -4 >}}](https://v{{< skew currentVersionAddMinor -4 "-" >}}.docs.kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/)
|
||||
|
||||
The upgrade workflow at high level is the following:
|
||||
|
||||
@@ -45,19 +45,19 @@ The upgrade workflow at high level is the following:
|
||||
|
||||
## Determine which version to upgrade to
|
||||
|
||||
Find the latest stable {{< skew latestVersion >}} version using the OS package manager:
|
||||
Find the latest patch release for Kubernetes {{< skew currentVersion >}} using the OS package manager:
|
||||
|
||||
{{< tabs name="k8s_install_versions" >}}
|
||||
{{% tab name="Ubuntu, Debian or HypriotOS" %}}
|
||||
apt update
|
||||
apt-cache madison kubeadm
|
||||
# find the latest {{< skew latestVersion >}} version in the list
|
||||
# it should look like {{< skew latestVersion >}}.x-00, where x is the latest patch
|
||||
# find the latest {{< skew currentVersion >}} version in the list
|
||||
# it should look like {{< skew currentVersion >}}.x-00, where x is the latest patch
|
||||
{{% /tab %}}
|
||||
{{% tab name="CentOS, RHEL or Fedora" %}}
|
||||
yum list --showduplicates kubeadm --disableexcludes=kubernetes
|
||||
# find the latest {{< skew latestVersion >}} version in the list
|
||||
# it should look like {{< skew latestVersion >}}.x-0, where x is the latest patch
|
||||
# find the latest {{< skew currentVersion >}} version in the list
|
||||
# it should look like {{< skew currentVersion >}}.x-0, where x is the latest patch
|
||||
{{% /tab %}}
|
||||
{{< /tabs >}}
|
||||
|
||||
@@ -74,18 +74,18 @@ Pick a control plane node that you wish to upgrade first. It must have the `/etc
|
||||
|
||||
{{< tabs name="k8s_install_kubeadm_first_cp" >}}
|
||||
{{% tab name="Ubuntu, Debian or HypriotOS" %}}
|
||||
# replace x in {{< skew latestVersion >}}.x-00 with the latest patch version
|
||||
# replace x in {{< skew currentVersion >}}.x-00 with the latest patch version
|
||||
apt-mark unhold kubeadm && \
|
||||
apt-get update && apt-get install -y kubeadm={{< skew latestVersion >}}.x-00 && \
|
||||
apt-get update && apt-get install -y kubeadm={{< skew currentVersion >}}.x-00 && \
|
||||
apt-mark hold kubeadm
|
||||
-
|
||||
# since apt-get version 1.1 you can also use the following method
|
||||
apt-get update && \
|
||||
apt-get install -y --allow-change-held-packages kubeadm={{< skew latestVersion >}}.x-00
|
||||
apt-get install -y --allow-change-held-packages kubeadm={{< skew currentVersion >}}.x-00
|
||||
{{% /tab %}}
|
||||
{{% tab name="CentOS, RHEL or Fedora" %}}
|
||||
# replace x in {{< skew latestVersion >}}.x-0 with the latest patch version
|
||||
yum install -y kubeadm-{{< skew latestVersion >}}.x-0 --disableexcludes=kubernetes
|
||||
# replace x in {{< skew currentVersion >}}.x-0 with the latest patch version
|
||||
yum install -y kubeadm-{{< skew currentVersion >}}.x-0 --disableexcludes=kubernetes
|
||||
{{% /tab %}}
|
||||
{{< /tabs >}}
|
||||
|
||||
@@ -120,13 +120,13 @@ Failing to do so will cause `kubeadm upgrade apply` to exit with an error and no
|
||||
|
||||
```shell
|
||||
# replace x with the patch version you picked for this upgrade
|
||||
sudo kubeadm upgrade apply v{{< skew latestVersion >}}.x
|
||||
sudo kubeadm upgrade apply v{{< skew currentVersion >}}.x
|
||||
```
|
||||
|
||||
Once the command finishes you should see:
|
||||
|
||||
```
|
||||
[upgrade/successful] SUCCESS! Your cluster was upgraded to "v{{< skew latestVersion >}}.x". Enjoy!
|
||||
[upgrade/successful] SUCCESS! Your cluster was upgraded to "v{{< skew currentVersion >}}.x". Enjoy!
|
||||
|
||||
[upgrade/kubelet] Now that your control plane is upgraded, please proceed with upgrading your kubelets if you haven't already done so.
|
||||
```
|
||||
@@ -171,20 +171,20 @@ Also calling `kubeadm upgrade plan` and upgrading the CNI provider plugin is no
|
||||
{{< tabs name="k8s_install_kubelet" >}}
|
||||
{{< tab name="Ubuntu, Debian or HypriotOS" >}}
|
||||
<pre>
|
||||
# replace x in {{< skew latestVersion >}}.x-00 with the latest patch version
|
||||
# replace x in {{< skew currentVersion >}}.x-00 with the latest patch version
|
||||
apt-mark unhold kubelet kubectl && \
|
||||
apt-get update && apt-get install -y kubelet={{< skew latestVersion >}}.x-00 kubectl={{< skew latestVersion >}}.x-00 && \
|
||||
apt-get update && apt-get install -y kubelet={{< skew currentVersion >}}.x-00 kubectl={{< skew currentVersion >}}.x-00 && \
|
||||
apt-mark hold kubelet kubectl
|
||||
-
|
||||
# since apt-get version 1.1 you can also use the following method
|
||||
apt-get update && \
|
||||
apt-get install -y --allow-change-held-packages kubelet={{< skew latestVersion >}}.x-00 kubectl={{< skew latestVersion >}}.x-00
|
||||
apt-get install -y --allow-change-held-packages kubelet={{< skew currentVersion >}}.x-00 kubectl={{< skew currentVersion >}}.x-00
|
||||
</pre>
|
||||
{{< /tab >}}
|
||||
{{< tab name="CentOS, RHEL or Fedora" >}}
|
||||
<pre>
|
||||
# replace x in {{< skew latestVersion >}}.x-0 with the latest patch version
|
||||
yum install -y kubelet-{{< skew latestVersion >}}.x-0 kubectl-{{< skew latestVersion >}}.x-0 --disableexcludes=kubernetes
|
||||
# replace x in {{< skew currentVersion >}}.x-0 with the latest patch version
|
||||
yum install -y kubelet-{{< skew currentVersion >}}.x-0 kubectl-{{< skew currentVersion >}}.x-0 --disableexcludes=kubernetes
|
||||
</pre>
|
||||
{{< /tab >}}
|
||||
{{< /tabs >}}
|
||||
@@ -216,18 +216,18 @@ without compromising the minimum required capacity for running your workloads.
|
||||
|
||||
{{< tabs name="k8s_install_kubeadm_worker_nodes" >}}
|
||||
{{% tab name="Ubuntu, Debian or HypriotOS" %}}
|
||||
# replace x in {{< skew latestVersion >}}.x-00 with the latest patch version
|
||||
# replace x in {{< skew currentVersion >}}.x-00 with the latest patch version
|
||||
apt-mark unhold kubeadm && \
|
||||
apt-get update && apt-get install -y kubeadm={{< skew latestVersion >}}.x-00 && \
|
||||
apt-get update && apt-get install -y kubeadm={{< skew currentVersion >}}.x-00 && \
|
||||
apt-mark hold kubeadm
|
||||
-
|
||||
# since apt-get version 1.1 you can also use the following method
|
||||
apt-get update && \
|
||||
apt-get install -y --allow-change-held-packages kubeadm={{< skew latestVersion >}}.x-00
|
||||
apt-get install -y --allow-change-held-packages kubeadm={{< skew currentVersion >}}.x-00
|
||||
{{% /tab %}}
|
||||
{{% tab name="CentOS, RHEL or Fedora" %}}
|
||||
# replace x in {{< skew latestVersion >}}.x-0 with the latest patch version
|
||||
yum install -y kubeadm-{{< skew latestVersion >}}.x-0 --disableexcludes=kubernetes
|
||||
# replace x in {{< skew currentVersion >}}.x-0 with the latest patch version
|
||||
yum install -y kubeadm-{{< skew currentVersion >}}.x-0 --disableexcludes=kubernetes
|
||||
{{% /tab %}}
|
||||
{{< /tabs >}}
|
||||
|
||||
@@ -254,18 +254,18 @@ without compromising the minimum required capacity for running your workloads.
|
||||
|
||||
{{< tabs name="k8s_kubelet_and_kubectl" >}}
|
||||
{{% tab name="Ubuntu, Debian or HypriotOS" %}}
|
||||
# replace x in {{< skew latestVersion >}}.x-00 with the latest patch version
|
||||
# replace x in {{< skew currentVersion >}}.x-00 with the latest patch version
|
||||
apt-mark unhold kubelet kubectl && \
|
||||
apt-get update && apt-get install -y kubelet={{< skew latestVersion >}}.x-00 kubectl={{< skew latestVersion >}}.x-00 && \
|
||||
apt-get update && apt-get install -y kubelet={{< skew currentVersion >}}.x-00 kubectl={{< skew currentVersion >}}.x-00 && \
|
||||
apt-mark hold kubelet kubectl
|
||||
-
|
||||
# since apt-get version 1.1 you can also use the following method
|
||||
apt-get update && \
|
||||
apt-get install -y --allow-change-held-packages kubelet={{< skew latestVersion >}}.x-00 kubectl={{< skew latestVersion >}}.x-00
|
||||
apt-get install -y --allow-change-held-packages kubelet={{< skew currentVersion >}}.x-00 kubectl={{< skew currentVersion >}}.x-00
|
||||
{{% /tab %}}
|
||||
{{% tab name="CentOS, RHEL or Fedora" %}}
|
||||
# replace x in {{< skew latestVersion >}}.x-0 with the latest patch version
|
||||
yum install -y kubelet-{{< skew latestVersion >}}.x-0 kubectl-{{< skew latestVersion >}}.x-0 --disableexcludes=kubernetes
|
||||
# replace x in {{< skew currentVersion >}}.x-0 with the latest patch version
|
||||
yum install -y kubelet-{{< skew currentVersion >}}.x-0 kubectl-{{< skew currentVersion >}}.x-0 --disableexcludes=kubernetes
|
||||
{{% /tab %}}
|
||||
{{< /tabs >}}
|
||||
|
||||
|
||||
@@ -34,14 +34,21 @@ If you are just looking for how to run a pod as a non-root user, see [SecurityCo
|
||||
|
||||
## Running Kubernetes inside Rootless Docker/Podman
|
||||
|
||||
[kind](https://kind.sigs.k8s.io/) supports running Kubernetes inside a Rootless Docker or Rootless Podman.
|
||||
### kind
|
||||
|
||||
[kind](https://kind.sigs.k8s.io/) supports running Kubernetes inside Rootless Docker or Rootless Podman.
|
||||
|
||||
See [Running kind with Rootless Docker](https://kind.sigs.k8s.io/docs/user/rootless/).
|
||||
|
||||
<!--
|
||||
[minikube](https://minikube.sigs.k8s.io/docs/) also plans to support Rootless Docker/Podman drivers.
|
||||
See [minikube issue #10836](https://github.com/kubernetes/minikube/issues/10836) to track the progress.
|
||||
-->
|
||||
### minikube
|
||||
|
||||
[minikube](https://minikube.sigs.k8s.io/) also supports running Kubernetes inside Rootless Docker.
|
||||
|
||||
See the page about the [docker](https://minikube.sigs.k8s.io/docs/drivers/docker/) driver in the Minikube documentation.
|
||||
|
||||
Rootless Podman is not supported.
|
||||
|
||||
<!-- Supporting rootless podman is discussed in https://github.com/kubernetes/minikube/issues/8719 -->
|
||||
|
||||
## Running Rootless Kubernetes directly on a host
|
||||
|
||||
|
||||
@@ -302,7 +302,7 @@ For details, read the [documentation for your Kubernetes version](/docs/home/sup
|
||||
Run the following command:
|
||||
|
||||
```shell
|
||||
kubectl delete deployment nginx-deployment --cascade=false
|
||||
kubectl delete deployment nginx-deployment --cascade=orphan
|
||||
```
|
||||
|
||||
**Using the Kubernetes API**
|
||||
|
||||
+1
-1
@@ -9,7 +9,7 @@ min-kubernetes-server-version: v1.22
|
||||
|
||||
As of v1.22, Kubernetes provides a built-in [admission controller](/docs/reference/access-authn-authz/admission-controllers/#podsecurity)
|
||||
to enforce the [Pod Security Standards](/docs/concepts/security/pod-security-standards).
|
||||
You can configure this admission controller to set cluster-wide defaults and [exemptions](#exemptions).
|
||||
You can configure this admission controller to set cluster-wide defaults and [exemptions](/docs/concepts/security/pod-security-admission/#exemptions).
|
||||
|
||||
## {{% heading "prerequisites" %}}
|
||||
|
||||
|
||||
@@ -88,10 +88,10 @@ GitHub Mentions: [@kubernetes/release-engineering](https://github.com/orgs/kuber
|
||||
|
||||
- Adolfo García Veytia ([@puerco](https://github.com/puerco))
|
||||
- Carlos Panato ([@cpanato](https://github.com/cpanato))
|
||||
- Daniel Mangum ([@hasheddan](https://github.com/hasheddan))
|
||||
- Marko Mudrinić ([@xmudrii](https://github.com/xmudrii))
|
||||
- Sascha Grunert ([@saschagrunert](https://github.com/saschagrunert))
|
||||
- Stephen Augustus ([@justaugustus](https://github.com/justaugustus))
|
||||
- Verónica López ([@verolop](https://github.com/verolop))
|
||||
|
||||
### Becoming a Release Manager
|
||||
|
||||
@@ -138,7 +138,6 @@ GitHub Mentions: @kubernetes/release-engineering
|
||||
- Nabarun Pal ([@palnabarun](https://github.com/palnabarun))
|
||||
- Seth McCombs ([@sethmccombs](https://github.com/sethmccombs))
|
||||
- Taylor Dolezal ([@onlydole](https://github.com/onlydole))
|
||||
- Verónica López ([@verolop](https://github.com/verolop))
|
||||
- Wilson Husin ([@wilsonehusin](https://github.com/wilsonehusin))
|
||||
|
||||
### Becoming a Release Manager Associate
|
||||
@@ -199,7 +198,6 @@ GitHub team: [@kubernetes/sig-release-leads](https://github.com/orgs/kubernetes/
|
||||
|
||||
- Adolfo García Veytia ([@puerco](https://github.com/puerco))
|
||||
- Carlos Panato ([@cpanato](https://github.com/cpanato))
|
||||
- Daniel Mangum ([@hasheddan](https://github.com/hasheddan))
|
||||
- Jeremy Rickard ([@jeremyrickard](https://github.com/jeremyrickard))
|
||||
|
||||
---
|
||||
|
||||
Reference in New Issue
Block a user