From ff6b8edc5b2075fb3d554b6b47f01fbfe703c6ff Mon Sep 17 00:00:00 2001 From: Qiming Teng Date: Mon, 14 Sep 2020 09:08:28 +0800 Subject: [PATCH] Move Server Side Apply into a separate reference page "Server Side Apply" is a big topic that warrants a dedicated page on it. Its current content is 400+ lines in the `api-concepts` page, kind of hijacking the "api-concepts" for a standalone feature. This PR proposes a separation for maintainability. --- .../feature-gates.md | 2 +- .../docs/reference/using-api/api-concepts.md | 446 +---------------- .../reference/using-api/server-side-apply.md | 462 ++++++++++++++++++ 3 files changed, 471 insertions(+), 439 deletions(-) create mode 100644 content/en/docs/reference/using-api/server-side-apply.md diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates.md b/content/en/docs/reference/command-line-tools-reference/feature-gates.md index eca4a65b87..6b81839774 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates.md @@ -510,7 +510,7 @@ Each feature gate is designed for enabling/disabling a specific feature: - `RuntimeClass`: Enable the [RuntimeClass](/docs/concepts/containers/runtime-class/) feature for selecting container runtime configurations. - `ScheduleDaemonSetPods`: Enable DaemonSet Pods to be scheduled by the default scheduler instead of the DaemonSet controller. - `SCTPSupport`: Enables the _SCTP_ `protocol` value in Pod, Service, Endpoints, EndpointSlice, and NetworkPolicy definitions. -- `ServerSideApply`: Enables the [Sever Side Apply (SSA)](/docs/reference/using-api/api-concepts/#server-side-apply) path at the API Server. +- `ServerSideApply`: Enables the [Sever Side Apply (SSA)](/docs/reference/using-api/server-side-apply/) path at the API Server. - `ServiceAccountIssuerDiscovery`: Enable OIDC discovery endpoints (issuer and JWKS URLs) for the service account issuer in the API server. See [Configure Service Accounts for Pods](/docs/tasks/configure-pod-container/configure-service-account/#service-account-issuer-discovery) for more details. - `ServiceAppProtocol`: Enables the `AppProtocol` field on Services and Endpoints. - `ServiceLoadBalancerFinalizer`: Enable finalizer protection for Service load balancers. diff --git a/content/en/docs/reference/using-api/api-concepts.md b/content/en/docs/reference/using-api/api-concepts.md index c251e228ee..5f8f8337d1 100644 --- a/content/en/docs/reference/using-api/api-concepts.md +++ b/content/en/docs/reference/using-api/api-concepts.md @@ -431,444 +431,14 @@ Some values of an object are typically generated before the object is persisted. ## Server Side Apply -{{< feature-state for_k8s_version="v1.16" state="beta" >}} - -Starting from Kubernetes v1.18, if you have Server Side Apply enabled then the control plane tracks managed fields for all newly created objects. - -### Introduction - -Server Side Apply helps users and controllers manage their resources via -declarative configurations. It allows them to create and/or modify their -objects declaratively, simply by sending their fully specified intent. - -A fully specified intent is a partial object that only includes the fields and -values for which the user has an opinion. That intent either creates a new -object or is [combined](#merge-strategy), by the server, with the existing object. - -The system supports multiple appliers collaborating on a single object. - -Changes to an object's fields are tracked through a "[field management](#field-management)" -mechanism. When a field's value changes, ownership moves from its current -manager to the manager making the change. When trying to apply an object, fields -that have a different value and are owned by another manager will result in a -[conflict](#conflicts). This is done in order to signal that the operation might undo another -collaborator's changes. Conflicts can be forced, in which case the value will be -overridden, and the ownership will be transferred. - -If you remove a field from a configuration and apply the configuration, server side apply checks -if there are any other field managers that also own the field. If the field is -not owned by any other field managers, it is either deleted from the live -object or reset to its default value, if it has one. The same rule applies to associative list or -map items. - -Server side apply is meant both as a replacement for the original `kubectl -apply` and as a simpler mechanism for controllers to enact their changes. - -### Field Management - -Compared to the `last-applied` annotation managed by `kubectl`, Server Side -Apply uses a more declarative approach, which tracks a user's field management, -rather than a user's last applied state. This means that as a side effect of -using Server Side Apply, information about which field manager manages each -field in an object also becomes available. - -For a user to manage a field, in the Server Side Apply sense, means that the -user relies on and expects the value of the field not to change. The user who -last made an assertion about the value of a field will be recorded as the -current field manager. This can be done either by changing the value with -`POST`, `PUT`, or non-apply `PATCH`, or by including the field in a config sent -to the Server Side Apply endpoint. When using Server-Side Apply, trying to -change a field which is managed by someone else will result in a rejected -request (if not forced, see [Conflicts](#conflicts)). - -When two or more appliers set a field to the same value, they share ownership of -that field. Any subsequent attempt to change the value of the shared field, by any of -the appliers, results in a conflict. Shared field owners may give up ownership -of a field by removing it from their configuration. - -Field management is stored in a`managedFields` field that is part of an object's -[`metadata`](/docs/reference/generated/kubernetes-api/{{< latest-version >}}/#objectmeta-v1-meta). - -A simple example of an object created by Server Side Apply could look like this: - -```yaml -apiVersion: v1 -kind: ConfigMap -metadata: - name: test-cm - namespace: default - labels: - test-label: test - managedFields: - - manager: kubectl - operation: Apply - apiVersion: v1 - time: "2010-10-10T0:00:00Z" - fieldsType: FieldsV1 - fieldsV1: - f:metadata: - f:labels: - f:test-label: {} - f:data: - f:key: {} -data: - key: some value -``` - -The above object contains a single manager in `metadata.managedFields`. The -manager consists of basic information about the managing entity itself, like -operation type, API version, and the fields managed by it. - -{{< note >}} -This field is managed by the API server and should not be changed by -the user. -{{< /note >}} - -Nevertheless it is possible to change `metadata.managedFields` through an -`Update` operation. Doing so is highly discouraged, but might be a reasonable -option to try if, for example, the `managedFields` get into an inconsistent -state (which clearly should not happen). - -The format of the `managedFields` is described in the [API](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#fieldsv1-v1-meta). - -### Conflicts - -A conflict is a special status error that occurs when an `Apply` operation tries -to change a field, which another user also claims to manage. This prevents an -applier from unintentionally overwriting the value set by another user. When -this occurs, the applier has 3 options to resolve the conflicts: - -* **Overwrite value, become sole manager:** If overwriting the value was - intentional (or if the applier is an automated process like a controller) the - applier should set the `force` query parameter to true and make the request - again. This forces the operation to succeed, changes the value of the field, - and removes the field from all other managers' entries in managedFields. -* **Don't overwrite value, give up management claim:** If the applier doesn't - care about the value of the field anymore, they can remove it from their - config and make the request again. This leaves the value unchanged, and causes - the field to be removed from the applier's entry in managedFields. -* **Don't overwrite value, become shared manager:** If the applier still cares - about the value of the field, but doesn't want to overwrite it, they can - change the value of the field in their config to match the value of the object - on the server, and make the request again. This leaves the value unchanged, - and causes the field's management to be shared by the applier and all other - field managers that already claimed to manage it. - - -### Managers - -Managers identify distinct workflows that are modifying the object (especially -useful on conflicts!), and can be specified through the `fieldManager` query -parameter as part of a modifying request. It is required for the apply endpoint, -though kubectl will default it to `kubectl`. For other updates, its default is -computed from the user-agent. - -### Apply and Update - -The two operation types considered by this feature are `Apply` (`PATCH` with -content type `application/apply-patch+yaml`) and `Update` (all other operations -which modify the object). Both operations update the `managedFields`, but behave -a little differently. - -{{< note >}} -Whether you are submitting JSON data or YAML data, use `application/apply-patch+yaml` as the -`Content-Type` header value. - -All JSON documents are valid YAML. -{{< /note >}} - -For instance, only the apply operation fails on conflicts while update does -not. Also, apply operations are required to identify themselves by providing a -`fieldManager` query parameter, while the query parameter is optional for update -operations. Finally, when using the apply operation you cannot have `managedFields` in the object that is being applied. - -An example object with multiple managers could look like this: - -```yaml -apiVersion: v1 -kind: ConfigMap -metadata: - name: test-cm - namespace: default - labels: - test-label: test - managedFields: - - manager: kubectl - operation: Apply - apiVersion: v1 - fields: - f:metadata: - f:labels: - f:test-label: {} - - manager: kube-controller-manager - operation: Update - apiVersion: v1 - time: '2019-03-30T16:00:00.000Z' - fields: - f:data: - f:key: {} -data: - key: new value -``` - -In this example, a second operation was run as an `Update` by the manager called -`kube-controller-manager`. The update changed a value in the data field which -caused the field's management to change to the `kube-controller-manager`. - -If this update would have been an `Apply` operation, the operation -would have failed due to conflicting ownership. - -### Merge strategy - -The merging strategy, implemented with Server Side Apply, provides a generally -more stable object lifecycle. Server Side Apply tries to merge fields based on -the fact who manages them instead of overruling just based on values. This way -it is intended to make it easier and more stable for multiple actors updating -the same object by causing less unexpected interference. - -When a user sends a "fully-specified intent" object to the Server Side Apply -endpoint, the server merges it with the live object favoring the value in the -applied config if it is specified in both places. If the set of items present in -the applied config is not a superset of the items applied by the same user last -time, each missing item not managed by any other appliers is removed. For -more information about how an object's schema is used to make decisions when -merging, see -[sigs.k8s.io/structured-merge-diff](https://sigs.k8s.io/structured-merge-diff). - -A number of markers were added in Kubernetes 1.16 and 1.17, to allow API -developers to describe the merge strategy supported by lists, maps, and -structs. These markers can be applied to objects of the respective type, -in Go files or in the -[OpenAPI schema definition of the CRD](/docs/reference/generated/kubernetes-api/{{< param "version" >}}#jsonschemaprops-v1-apiextensions-k8s-io): - -| Golang marker | OpenAPI extension | Accepted values | Description | Introduced in | -|---|---|---|---|---| -| `//+listType` | `x-kubernetes-list-type` | `atomic`/`set`/`map` | Applicable to lists. `atomic` and `set` apply to lists with scalar elements only. `map` applies to lists of nested types only. If configured as `atomic`, the entire list is replaced during merge; a single manager manages the list as a whole at any one time. If `granular`, different managers can manage entries separately. | 1.16 | -| `//+listMapKey` | `x-kubernetes-list-map-keys` | Slice of map keys that uniquely identify entries for example `["port", "protocol"]` | Only applicable when `+listType=map`. A slice of strings whose values in combination must uniquely identify list entries. While there can be multiple keys, `listMapKey` is singular because keys need to be specified individually in the Go type. | 1.16 | -| `//+mapType` | `x-kubernetes-map-type` | `atomic`/`granular` | Applicable to maps. `atomic` means that the map can only be entirely replaced by a single manager. `granular` means that the map supports separate managers updating individual fields. | 1.17 | -| `//+structType` | `x-kubernetes-map-type` | `atomic`/`granular` | Applicable to structs; otherwise same usage and OpenAPI annotation as `//+mapType`.| 1.17 | - -### Custom Resources - -By default, Server Side Apply treats custom resources as unstructured data. All -keys are treated the same as struct fields, and all lists are considered atomic. - -If the Custom Resource Definition defines a -[schema](/docs/reference/generated/kubernetes-api/{{< param "version" >}}#jsonschemaprops-v1-apiextensions-k8s-io) -that contains annotations as defined in the previous "Merge Strategy" -section, these annotations will be used when merging objects of this -type. - -### Using Server-Side Apply in a controller - -As a developer of a controller, you can use server-side apply as a way to -simplify the update logic of your controller. The main differences with a -read-modify-write and/or patch are the following: - -* the applied object must contain all the fields that the controller cares about. -* there are no way to remove fields that haven't been applied by the controller - before (controller can still send a PATCH/UPDATE for these use-cases). -* the object doesn't have to be read beforehand, `resourceVersion` doesn't have - to be specified. - -It is strongly recommended for controllers to always "force" conflicts, since they -might not be able to resolve or act on these conflicts. - -### Transferring Ownership - -In addition to the concurrency controls provided by [conflict resolution](#conflicts), -Server Side Apply provides ways to perform coordinated -field ownership transfers from users to controllers. - -This is best explained by example. Let's look at how to safely transfer -ownership of the `replicas` field from a user to a controller while enabling -automatic horizontal scaling for a Deployment, using the HorizontalPodAutoscaler -resource and its accompanying controller. - -Say a user has defined deployment with `replicas` set to the desired value: - -{{< codenew file="application/ssa/nginx-deployment.yaml" >}} - -And the user has created the deployment using server side apply like so: - -```shell -kubectl apply -f https://k8s.io/examples/application/ssa/nginx-deployment.yaml --server-side -``` - -Then later, HPA is enabled for the deployment, e.g.: - -```shell -kubectl autoscale deployment nginx-deployment --cpu-percent=50 --min=1 --max=10 -``` - -Now, the user would like to remove `replicas` from their configuration, so they -don't accidentally fight with the HPA controller. However, there is a race: it -might take some time before HPA feels the need to adjust `replicas`, and if -the user removes `replicas` before the HPA writes to the field and becomes -its owner, then apiserver will set `replicas` to 1, its default value. This -is not what the user wants to happen, even temporarily. - -There are two solutions: - -- (easy) Leave `replicas` in the configuration; when HPA eventually writes to that - field, the system gives the user a conflict over it. At that point, it is safe - to remove from the configuration. - -- (more advanced) If, however, the user doesn't want to wait, for example - because they want to keep the cluster legible to coworkers, then they can take - the following steps to make it safe to remove `replicas` from their - configuration: - -First, the user defines a new configuration containing only the `replicas` field: - -{{< codenew file="application/ssa/nginx-deployment-replicas-only.yaml" >}} - -The user applies that configuration using the field manager name `handover-to-hpa`: - -```shell -kubectl apply -f https://k8s.io/examples/application/ssa/nginx-deployment-replicas-only.yaml --server-side --field-manager=handover-to-hpa --validate=false -``` - -If the apply results in a conflict with the HPA controller, then do nothing. The -conflict just indicates the controller has claimed the field earlier in the -process than it sometimes does. - -At this point the user may remove the `replicas` field from their configuration. - -{{< codenew file="application/ssa/nginx-deployment-no-replicas.yaml" >}} - -Note that whenever the HPA controller sets the `replicas` field to a new value, -the temporary field manager will no longer own any fields and will be -automatically deleted. No clean up is required. - -### Transferring Ownership Between Users - -Users can transfer ownership of a field between each other by setting the field -to the same value in both of their applied configs, causing them to share -ownership of the field. Once the users share ownership of the field, one of them -can remove the field from their applied configuration to give up ownership and -complete the transfer to the other user. - -### Comparison with Client Side Apply - -A consequence of the conflict detection and resolution implemented by Server -Side Apply is that an applier always has up to date field values in their local -state. If they don't, they get a conflict the next time they apply. Any of the -three options to resolve conflicts results in the applied configuration being an -up to date subset of the object on the server's fields. - -This is different from Client Side Apply, where outdated values which have been -overwritten by other users are left in an applier's local config. These values -only become accurate when the user updates that specific field, if ever, and an -applier has no way of knowing whether their next apply will overwrite other -users' changes. - -Another difference is that an applier using Client Side Apply is unable to -change the API version they are using, but Server Side Apply supports this use -case. - -### Upgrading from client-side apply to server-side apply - -Client-side apply users who manage a resource with `kubectl apply` can start -using server-side apply with the following flag. - -```shell -kubectl apply --server-side [--dry-run=server] -``` - -By default, field management of the object transfers from client-side apply -to kubectl server-side apply without encountering conflicts. - -{{< caution >}} -Keep the `last-applied-configuration` annotation up to date. -The annotation infers client-side apply's managed fields. -Any fields not managed by client-side apply raise conflicts. - -For example, if you used `kubectl scale` to update the replicas field after client-side apply, -then this field is not owned by client-side apply and creates conflicts on `kubectl apply --server-side`. -{{< /caution >}} - -This behavior applies to server-side apply with the `kubectl` field manager. As -an exception, you can opt-out of this behavior by specifying a different, -non-default field manager, as seen in the following example. The default field manager for kubectl -server-side apply is `kubectl`. - -```shell -kubectl apply --server-side --field-manager=my-manager [--dry-run=server] -``` - -### Downgrading from server-side apply to client-side apply - -If you manage a resource with `kubectl apply --server-side`, -you can downgrade to client-side apply directly with `kubectl apply`. - -Downgrading works because kubectl server-side apply keeps the -`last-applied-configuration` annotation up-to-date if you use -`kubectl apply`. - -This behavior applies to server-side apply with the `kubectl` field manager. As -an exception, you can opt-out of this behavior by specifying a different, -non-default field manager, as seen in the following example. The default field manager for kubectl -server-side apply is `kubectl`. - -```shell -kubectl apply --server-side --field-manager=my-manager [--dry-run=server] -``` - -### API Endpoint - -With the Server Side Apply feature enabled, the `PATCH` endpoint accepts the -additional `application/apply-patch+yaml` content type. Users of Server Side -Apply can send partially specified objects as YAML to this endpoint. -When applying a configuration, one should always include all the fields -that they have an opinion about. - -### Clearing ManagedFields - -It is possible to strip all managedFields from an object by overwriting them -using `MergePatch`, `StrategicMergePatch`, `JSONPatch` or `Update`, so every -non-apply operation. This can be done by overwriting the managedFields field -with an empty entry. Two examples are: - -```console -PATCH /api/v1/namespaces/default/configmaps/example-cm -Content-Type: application/merge-patch+json -Accept: application/json -Data: {"metadata":{"managedFields": [{}]}} -``` - -```console -PATCH /api/v1/namespaces/default/configmaps/example-cm -Content-Type: application/json-patch+json -Accept: application/json -Data: [{"op": "replace", "path": "/metadata/managedFields", "value": [{}]}] -``` - -This will overwrite the managedFields with a list containing a single empty -entry that then results in the managedFields being stripped entirely from the -object. Note that just setting the managedFields to an empty list will not reset -the field. This is on purpose, so managedFields never get stripped by clients -not aware of the field. - -In cases where the reset operation is combined with changes to other fields than -the managedFields, this will result in the managedFields being reset first and -the other changes being processed afterwards. As a result the applier takes -ownership of any fields updated in the same request. - -{{< caution >}} -Server Side Apply does not correctly track ownership on -sub-resources that don't receive the resource object type. If you are -using Server Side Apply with such a sub-resource, the changed fields -won't be tracked. -{{< /caution >}} - -### Disabling the feature - -Server Side Apply is a beta feature, so it is enabled by default. To turn this -[feature gate](/docs/reference/command-line-tools-reference/feature-gates) off, -you need to include the `--feature-gates ServerSideApply=false` flag when -starting `kube-apiserver`. If you have multiple `kube-apiserver` replicas, all -should have the same flag setting. +Starting from Kubernetes v1.18, you can enable the +[Server Side Apply](/docs/reference/using-api/server-side-apply/) +feature so that the control plane tracks managed fields for all newly created objects. +Server Side Apply provides a clear pattern for managing field conflicts, +offers server-side `Apply` and `Update` operations, and replaces the +client-side functionality of `kubectl apply`. For more details about this +feature, see the section on +[Server Side Apply](/docs/reference/using-api/server-side-apply/). ## Resource Versions diff --git a/content/en/docs/reference/using-api/server-side-apply.md b/content/en/docs/reference/using-api/server-side-apply.md new file mode 100644 index 0000000000..5f37b25682 --- /dev/null +++ b/content/en/docs/reference/using-api/server-side-apply.md @@ -0,0 +1,462 @@ +--- +title: Server-Side Apply +reviewers: +- smarterclayton +- apelisse +- lavalamp +- liggitt +content_type: concept +weight: 25 +min-kubernetes-server-version: 1.16 +--- + + + +{{< feature-state for_k8s_version="v1.16" state="beta" >}} + +## Introduction + +Server Side Apply helps users and controllers manage their resources via +declarative configurations. It allows them to create and/or modify their +[objects](/docs/concepts/overview/working-with-objects/kubernetes-objects/) +declaratively, simply by sending their fully specified intent. + +A fully specified intent is a partial object that only includes the fields and +values for which the user has an opinion. That intent either creates a new +object or is [combined](#merge-strategy), by the server, with the existing object. + +The system supports multiple appliers collaborating on a single object. + +Changes to an object's fields are tracked through a "[field management](#field-management)" +mechanism. When a field's value changes, ownership moves from its current +manager to the manager making the change. When trying to apply an object, +fields that have a different value and are owned by another manager will +result in a [conflict](#conflicts). This is done in order to signal that the +operation might undo another collaborator's changes. Conflicts can be forced, +in which case the value will be overridden, and the ownership will be +transferred. + +If you remove a field from a configuration and apply the configuration, server +side apply checks if there are any other field managers that also own the +field. If the field is not owned by any other field managers, it is either +deleted from the live object or reset to its default value, if it has one. The +same rule applies to associative list or map items. + +Server side apply is meant both as a replacement for the original `kubectl +apply` and as a simpler mechanism for controllers to enact their changes. + +If you have Server Side Apply enabled, the control plane tracks managed fields +for all newlly created objects. + +## Field Management + +Compared to the `last-applied` annotation managed by `kubectl`, Server Side +Apply uses a more declarative approach, which tracks a user's field management, +rather than a user's last applied state. This means that as a side effect of +using Server Side Apply, information about which field manager manages each +field in an object also becomes available. + +For a user to manage a field, in the Server Side Apply sense, means that the +user relies on and expects the value of the field not to change. The user who +last made an assertion about the value of a field will be recorded as the +current field manager. This can be done either by changing the value with +`POST`, `PUT`, or non-apply `PATCH`, or by including the field in a config sent +to the Server Side Apply endpoint. When using Server-Side Apply, trying to +change a field which is managed by someone else will result in a rejected +request (if not forced, see [Conflicts](#conflicts)). + +When two or more appliers set a field to the same value, they share ownership of +that field. Any subsequent attempt to change the value of the shared field, by any of +the appliers, results in a conflict. Shared field owners may give up ownership +of a field by removing it from their configuration. + +Field management is stored in a`managedFields` field that is part of an object's +[`metadata`](/docs/reference/generated/kubernetes-api/{{< latest-version >}}/#objectmeta-v1-meta). + +A simple example of an object created by Server Side Apply could look like this: + +```yaml +apiVersion: v1 +kind: ConfigMap +metadata: + name: test-cm + namespace: default + labels: + test-label: test + managedFields: + - manager: kubectl + operation: Apply + apiVersion: v1 + time: "2010-10-10T0:00:00Z" + fieldsType: FieldsV1 + fieldsV1: + f:metadata: + f:labels: + f:test-label: {} + f:data: + f:key: {} +data: + key: some value +``` + +The above object contains a single manager in `metadata.managedFields`. The +manager consists of basic information about the managing entity itself, like +operation type, API version, and the fields managed by it. + +{{< note >}} +This field is managed by the API server and should not be changed by +the user. +{{< /note >}} + +Nevertheless it is possible to change `metadata.managedFields` through an +`Update` operation. Doing so is highly discouraged, but might be a reasonable +option to try if, for example, the `managedFields` get into an inconsistent +state (which clearly should not happen). + +The format of the `managedFields` is described in the +[API](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#fieldsv1-v1-meta). + +## Conflicts + +A conflict is a special status error that occurs when an `Apply` operation tries +to change a field, which another user also claims to manage. This prevents an +applier from unintentionally overwriting the value set by another user. When +this occurs, the applier has 3 options to resolve the conflicts: + +* **Overwrite value, become sole manager:** If overwriting the value was + intentional (or if the applier is an automated process like a controller) the + applier should set the `force` query parameter to true and make the request + again. This forces the operation to succeed, changes the value of the field, + and removes the field from all other managers' entries in managedFields. + +* **Don't overwrite value, give up management claim:** If the applier doesn't + care about the value of the field anymore, they can remove it from their + config and make the request again. This leaves the value unchanged, and causes + the field to be removed from the applier's entry in managedFields. + +* **Don't overwrite value, become shared manager:** If the applier still cares + about the value of the field, but doesn't want to overwrite it, they can + change the value of the field in their config to match the value of the object + on the server, and make the request again. This leaves the value unchanged, + and causes the field's management to be shared by the applier and all other + field managers that already claimed to manage it. + +## Managers + +Managers identify distinct workflows that are modifying the object (especially +useful on conflicts!), and can be specified through the `fieldManager` query +parameter as part of a modifying request. It is required for the apply endpoint, +though kubectl will default it to `kubectl`. For other updates, its default is +computed from the user-agent. + +## Apply and Update + +The two operation types considered by this feature are `Apply` (`PATCH` with +content type `application/apply-patch+yaml`) and `Update` (all other operations +which modify the object). Both operations update the `managedFields`, but behave +a little differently. + +{{< note >}} +Whether you are submitting JSON data or YAML data, use +`application/apply-patch+yaml` as the `Content-Type` header value. + +All JSON documents are valid YAML. +{{< /note >}} + +For instance, only the apply operation fails on conflicts while update does +not. Also, apply operations are required to identify themselves by providing a +`fieldManager` query parameter, while the query parameter is optional for update +operations. Finally, when using the apply operation you cannot have +`managedFields` in the object that is being applied. + +An example object with multiple managers could look like this: + +```yaml +apiVersion: v1 +kind: ConfigMap +metadata: + name: test-cm + namespace: default + labels: + test-label: test + managedFields: + - manager: kubectl + operation: Apply + apiVersion: v1 + fields: + f:metadata: + f:labels: + f:test-label: {} + - manager: kube-controller-manager + operation: Update + apiVersion: v1 + time: '2019-03-30T16:00:00.000Z' + fields: + f:data: + f:key: {} +data: + key: new value +``` + +In this example, a second operation was run as an `Update` by the manager called +`kube-controller-manager`. The update changed a value in the data field which +caused the field's management to change to the `kube-controller-manager`. + +If this update would have been an `Apply` operation, the operation +would have failed due to conflicting ownership. + +## Merge strategy + +The merging strategy, implemented with Server Side Apply, provides a generally +more stable object lifecycle. Server Side Apply tries to merge fields based on +the fact who manages them instead of overruling just based on values. This way +it is intended to make it easier and more stable for multiple actors updating +the same object by causing less unexpected interference. + +When a user sends a "fully-specified intent" object to the Server Side Apply +endpoint, the server merges it with the live object favoring the value in the +applied config if it is specified in both places. If the set of items present in +the applied config is not a superset of the items applied by the same user last +time, each missing item not managed by any other appliers is removed. For +more information about how an object's schema is used to make decisions when +merging, see +[sigs.k8s.io/structured-merge-diff](https://sigs.k8s.io/structured-merge-diff). + +A number of markers were added in Kubernetes 1.16 and 1.17, to allow API +developers to describe the merge strategy supported by lists, maps, and +structs. These markers can be applied to objects of the respective type, +in Go files or in the OpenAPI schema definition of the +[CRD](/docs/reference/generated/kubernetes-api/{{< param "version" >}}#jsonschemaprops-v1-apiextensions-k8s-io): + +| Golang marker | OpenAPI extension | Accepted values | Description | Introduced in | +|---|---|---|---|---| +| `//+listType` | `x-kubernetes-list-type` | `atomic`/`set`/`map` | Applicable to lists. `atomic` and `set` apply to lists with scalar elements only. `map` applies to lists of nested types only. If configured as `atomic`, the entire list is replaced during merge; a single manager manages the list as a whole at any one time. If `granular`, different managers can manage entries separately. | 1.16 | +| `//+listMapKey` | `x-kubernetes-list-map-keys` | Slice of map keys that uniquely identify entries for example `["port", "protocol"]` | Only applicable when `+listType=map`. A slice of strings whose values in combination must uniquely identify list entries. While there can be multiple keys, `listMapKey` is singular because keys need to be specified individually in the Go type. | 1.16 | +| `//+mapType` | `x-kubernetes-map-type` | `atomic`/`granular` | Applicable to maps. `atomic` means that the map can only be entirely replaced by a single manager. `granular` means that the map supports separate managers updating individual fields. | 1.17 | +| `//+structType` | `x-kubernetes-map-type` | `atomic`/`granular` | Applicable to structs; otherwise same usage and OpenAPI annotation as `//+mapType`.| 1.17 | + +### Custom Resources + +By default, Server Side Apply treats custom resources as unstructured data. All +keys are treated the same as struct fields, and all lists are considered atomic. + +If the Custom Resource Definition defines a +[schema](/docs/reference/generated/kubernetes-api/{{< param "version" >}}#jsonschemaprops-v1-apiextensions-k8s-io) +that contains annotations as defined in the previous "Merge Strategy" +section, these annotations will be used when merging objects of this +type. + +### Using Server-Side Apply in a controller + +As a developer of a controller, you can use server-side apply as a way to +simplify the update logic of your controller. The main differences with a +read-modify-write and/or patch are the following: + +* the applied object must contain all the fields that the controller cares about. +* there are no way to remove fields that haven't been applied by the controller + before (controller can still send a PATCH/UPDATE for these use-cases). +* the object doesn't have to be read beforehand, `resourceVersion` doesn't have + to be specified. + +It is strongly recommended for controllers to always "force" conflicts, since they +might not be able to resolve or act on these conflicts. + +### Transferring Ownership + +In addition to the concurrency controls provided by [conflict resolution](#conflicts), +Server Side Apply provides ways to perform coordinated +field ownership transfers from users to controllers. + +This is best explained by example. Let's look at how to safely transfer +ownership of the `replicas` field from a user to a controller while enabling +automatic horizontal scaling for a Deployment, using the HorizontalPodAutoscaler +resource and its accompanying controller. + +Say a user has defined deployment with `replicas` set to the desired value: + +{{< codenew file="application/ssa/nginx-deployment.yaml" >}} + +And the user has created the deployment using server side apply like so: + +```shell +kubectl apply -f https://k8s.io/examples/application/ssa/nginx-deployment.yaml --server-side +``` + +Then later, HPA is enabled for the deployment, e.g.: + +```shell +kubectl autoscale deployment nginx-deployment --cpu-percent=50 --min=1 --max=10 +``` + +Now, the user would like to remove `replicas` from their configuration, so they +don't accidentally fight with the HPA controller. However, there is a race: it +might take some time before HPA feels the need to adjust `replicas`, and if +the user removes `replicas` before the HPA writes to the field and becomes +its owner, then apiserver will set `replicas` to 1, its default value. This +is not what the user wants to happen, even temporarily. + +There are two solutions: + +- (easy) Leave `replicas` in the configuration; when HPA eventually writes to that + field, the system gives the user a conflict over it. At that point, it is safe + to remove from the configuration. + +- (more advanced) If, however, the user doesn't want to wait, for example + because they want to keep the cluster legible to coworkers, then they can take + the following steps to make it safe to remove `replicas` from their + configuration: + +First, the user defines a new configuration containing only the `replicas` field: + +{{< codenew file="application/ssa/nginx-deployment-replicas-only.yaml" >}} + +The user applies that configuration using the field manager name `handover-to-hpa`: + +```shell +kubectl apply -f https://k8s.io/examples/application/ssa/nginx-deployment-replicas-only.yaml \ + --server-side --field-manager=handover-to-hpa \ + --validate=false +``` + +If the apply results in a conflict with the HPA controller, then do nothing. The +conflict just indicates the controller has claimed the field earlier in the +process than it sometimes does. + +At this point the user may remove the `replicas` field from their configuration. + +{{< codenew file="application/ssa/nginx-deployment-no-replicas.yaml" >}} + +Note that whenever the HPA controller sets the `replicas` field to a new value, +the temporary field manager will no longer own any fields and will be +automatically deleted. No clean up is required. + +## Transferring Ownership Between Users + +Users can transfer ownership of a field between each other by setting the field +to the same value in both of their applied configs, causing them to share +ownership of the field. Once the users share ownership of the field, one of them +can remove the field from their applied configuration to give up ownership and +complete the transfer to the other user. + +## Comparison with Client Side Apply + +A consequence of the conflict detection and resolution implemented by Server +Side Apply is that an applier always has up to date field values in their local +state. If they don't, they get a conflict the next time they apply. Any of the +three options to resolve conflicts results in the applied configuration being an +up to date subset of the object on the server's fields. + +This is different from Client Side Apply, where outdated values which have been +overwritten by other users are left in an applier's local config. These values +only become accurate when the user updates that specific field, if ever, and an +applier has no way of knowing whether their next apply will overwrite other +users' changes. + +Another difference is that an applier using Client Side Apply is unable to +change the API version they are using, but Server Side Apply supports this use +case. + +## Upgrading from client-side apply to server-side apply + +Client-side apply users who manage a resource with `kubectl apply` can start +using server-side apply with the following flag. + +```shell +kubectl apply --server-side [--dry-run=server] +``` + +By default, field management of the object transfers from client-side apply to +kubectl server-side apply without encountering conflicts. + +{{< caution >}} +Keep the `last-applied-configuration` annotation up to date. +The annotation infers client-side apply's managed fields. +Any fields not managed by client-side apply raise conflicts. + +For example, if you used `kubectl scale` to update the replicas field after +client-side apply, then this field is not owned by client-side apply and +creates conflicts on `kubectl apply --server-side`. +{{< /caution >}} + +This behavior applies to server-side apply with the `kubectl` field manager. +As an exception, you can opt-out of this behavior by specifying a different, +non-default field manager, as seen in the following example. The default field +manager for kubectl server-side apply is `kubectl`. + +```shell +kubectl apply --server-side --field-manager=my-manager [--dry-run=server] +``` + +## Downgrading from server-side apply to client-side apply + +If you manage a resource with `kubectl apply --server-side`, +you can downgrade to client-side apply directly with `kubectl apply`. + +Downgrading works because kubectl server-side apply keeps the +`last-applied-configuration` annotation up-to-date if you use +`kubectl apply`. + +This behavior applies to server-side apply with the `kubectl` field manager. +As an exception, you can opt-out of this behavior by specifying a different, +non-default field manager, as seen in the following example. The default field +manager for kubectl server-side apply is `kubectl`. + +```shell +kubectl apply --server-side --field-manager=my-manager [--dry-run=server] +``` + +## API Endpoint + +With the Server Side Apply feature enabled, the `PATCH` endpoint accepts the +additional `application/apply-patch+yaml` content type. Users of Server Side +Apply can send partially specified objects as YAML to this endpoint. When +applying a configuration, one should always include all the fields that they +have an opinion about. + +## Clearing ManagedFields + +It is possible to strip all managedFields from an object by overwriting them +using `MergePatch`, `StrategicMergePatch`, `JSONPatch` or `Update`, so every +non-apply operation. This can be done by overwriting the managedFields field +with an empty entry. Two examples are: + +```console +PATCH /api/v1/namespaces/default/configmaps/example-cm +Content-Type: application/merge-patch+json +Accept: application/json +Data: {"metadata":{"managedFields": [{}]}} +``` + +```console +PATCH /api/v1/namespaces/default/configmaps/example-cm +Content-Type: application/json-patch+json +Accept: application/json +Data: [{"op": "replace", "path": "/metadata/managedFields", "value": [{}]}] +``` + +This will overwrite the managedFields with a list containing a single empty +entry that then results in the managedFields being stripped entirely from the +object. Note that just setting the managedFields to an empty list will not +reset the field. This is on purpose, so managedFields never get stripped by +clients not aware of the field. + +In cases where the reset operation is combined with changes to other fields +than the managedFields, this will result in the managedFields being reset +first and the other changes being processed afterwards. As a result the +applier takes ownership of any fields updated in the same request. + +{{< caution >}} +Server Side Apply does not correctly track ownership on +sub-resources that don't receive the resource object type. If you are +using Server Side Apply with such a sub-resource, the changed fields +won't be tracked. +{{< /caution >}} + +## Disabling the feature + +Server Side Apply is a beta feature, so it is enabled by default. To turn this +[feature gate](/docs/reference/command-line-tools-reference/feature-gates) off, +you need to include the `--feature-gates ServerSideApply=false` flag when +starting `kube-apiserver`. If you have multiple `kube-apiserver` replicas, all +should have the same flag setting. +