Convert site to Hugo (#8316)
This commit converts content and layout to use Hugo.
This commit is contained in:
committed by
k8s-ci-robot
parent
7745f0e0c5
commit
7f3b633aa0
@@ -0,0 +1,5 @@
|
||||
---
|
||||
title: "Extend Kubernetes"
|
||||
weight: 90
|
||||
---
|
||||
|
||||
@@ -0,0 +1,53 @@
|
||||
---
|
||||
title: Configure the aggregation layer
|
||||
reviewers:
|
||||
- lavalamp
|
||||
- cheftako
|
||||
- chenopis
|
||||
content_template: templates/task
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
Configuring the [aggregation layer](/docs/concepts/api-extension/apiserver-aggregation/) allows the Kubernetes apiserver to be extended with additional APIs, which are not part of the core Kubernetes APIs.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture prerequisites %}}
|
||||
|
||||
{{< include "task-tutorial-prereqs.md" >}} {{< version-check >}}
|
||||
|
||||
**Note:** There are a few setup requirements for getting the aggregation layer working in your environment to support mutual TLS auth between the proxy and extension apiservers. Kubernetes and the kube-apiserver have multiple CAs, so make sure that the proxy is signed by the aggregation layer CA and not by something else, like the master CA.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture steps %}}
|
||||
|
||||
## Enable apiserver flags
|
||||
|
||||
Enable the aggregation layer via the following kube-apiserver flags. They may have already been taken care of by your provider.
|
||||
|
||||
--requestheader-client-ca-file=<path to aggregator CA cert>
|
||||
--requestheader-allowed-names=aggregator
|
||||
--requestheader-extra-headers-prefix=X-Remote-Extra-
|
||||
--requestheader-group-headers=X-Remote-Group
|
||||
--requestheader-username-headers=X-Remote-User
|
||||
--proxy-client-cert-file=<path to aggregator proxy cert>
|
||||
--proxy-client-key-file=<path to aggregator proxy key>
|
||||
|
||||
If you are not running kube-proxy on a host running the API server, then you must make sure that the system is enabled with the following apiserver flag:
|
||||
|
||||
--enable-aggregator-routing=true
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture whatsnext %}}
|
||||
|
||||
* [Setup an extension api-server](/docs/tasks/access-kubernetes-api/setup-extension-api-server/) to work with the aggregation layer.
|
||||
* For a high level overview, see [Extending the Kubernetes API with the aggregation layer](/docs/concepts/api-extension/apiserver-aggregation/).
|
||||
* Learn how to [Extend the Kubernetes API Using Custom Resource Definitions](/docs/tasks/access-kubernetes-api/extend-api-custom-resource-definitions/).
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
|
||||
|
||||
+551
@@ -0,0 +1,551 @@
|
||||
---
|
||||
title: Extend the Kubernetes API with CustomResourceDefinitions
|
||||
reviewers:
|
||||
- deads2k
|
||||
- enisoc
|
||||
content_template: templates/task
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
This page shows how to install a
|
||||
[custom resource](/docs/concepts/api-extension/custom-resources/)
|
||||
into the Kubernetes API by creating a
|
||||
[CustomResourceDefinition](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#customresourcedefinition-v1beta1-apiextensions).
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture prerequisites %}}
|
||||
|
||||
{{< include "task-tutorial-prereqs.md" >}} {{< version-check >}}
|
||||
|
||||
* Make sure your Kubernetes cluster has a master version of 1.7.0 or higher.
|
||||
|
||||
* Read about [custom resources](/docs/concepts/api-extension/custom-resources/).
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture steps %}}
|
||||
## Create a CustomResourceDefinition
|
||||
|
||||
When you create a new CustomResourceDefinition (CRD), the Kubernetes API Server
|
||||
reacts by creating a new RESTful resource path, either namespaced or cluster-scoped,
|
||||
as specified in the CRD's `scope` field. As with existing built-in objects, deleting a
|
||||
namespace deletes all custom objects in that namespace.
|
||||
CustomResourceDefinitions themselves are non-namespaced and are available to all namespaces.
|
||||
|
||||
For example, if you save the following CustomResourceDefinition to `resourcedefinition.yaml`:
|
||||
|
||||
```yaml
|
||||
apiVersion: apiextensions.k8s.io/v1beta1
|
||||
kind: CustomResourceDefinition
|
||||
metadata:
|
||||
# name must match the spec fields below, and be in the form: <plural>.<group>
|
||||
name: crontabs.stable.example.com
|
||||
spec:
|
||||
# group name to use for REST API: /apis/<group>/<version>
|
||||
group: stable.example.com
|
||||
# version name to use for REST API: /apis/<group>/<version>
|
||||
version: v1
|
||||
# either Namespaced or Cluster
|
||||
scope: Namespaced
|
||||
names:
|
||||
# plural name to be used in the URL: /apis/<group>/<version>/<plural>
|
||||
plural: crontabs
|
||||
# singular name to be used as an alias on the CLI and for display
|
||||
singular: crontab
|
||||
# kind is normally the CamelCased singular type. Your resource manifests use this.
|
||||
kind: CronTab
|
||||
# shortNames allow shorter string to match your resource on the CLI
|
||||
shortNames:
|
||||
- ct
|
||||
```
|
||||
|
||||
And create it:
|
||||
|
||||
```shell
|
||||
kubectl create -f resourcedefinition.yaml
|
||||
```
|
||||
|
||||
Then a new namespaced RESTful API endpoint is created at:
|
||||
|
||||
```
|
||||
/apis/stable.example.com/v1/namespaces/*/crontabs/...
|
||||
```
|
||||
|
||||
This endpoint URL can then be used to create and manage custom objects.
|
||||
The `kind` of these objects will be `CronTab` from the spec of the
|
||||
CustomResourceDefinition object you created above.
|
||||
|
||||
Please note that it might take a few seconds for the endpoint to be created.
|
||||
You can watch the `Established` condition of your CustomResourceDefinition
|
||||
to be true or watch the discovery information of the API server for your
|
||||
resource to show up.
|
||||
|
||||
## Create custom objects
|
||||
|
||||
After the CustomResourceDefinition object has been created, you can create
|
||||
custom objects. Custom objects can contain custom fields. These fields can
|
||||
contain arbitrary JSON.
|
||||
In the following example, the `cronSpec` and `image` custom fields are set in a
|
||||
custom object of kind `CronTab`. The kind `CronTab` comes from the spec of the
|
||||
CustomResourceDefinition object you created above.
|
||||
|
||||
If you save the following YAML to `my-crontab.yaml`:
|
||||
|
||||
```yaml
|
||||
apiVersion: "stable.example.com/v1"
|
||||
kind: CronTab
|
||||
metadata:
|
||||
name: my-new-cron-object
|
||||
spec:
|
||||
cronSpec: "* * * * */5"
|
||||
image: my-awesome-cron-image
|
||||
```
|
||||
|
||||
and create it:
|
||||
|
||||
```shell
|
||||
kubectl create -f my-crontab.yaml
|
||||
```
|
||||
|
||||
You can then manage your CronTab objects using kubectl. For example:
|
||||
|
||||
```shell
|
||||
kubectl get crontab
|
||||
```
|
||||
|
||||
Should print a list like this:
|
||||
|
||||
```console
|
||||
NAME KIND
|
||||
my-new-cron-object CronTab.v1.stable.example.com
|
||||
```
|
||||
|
||||
Note that resource names are not case-sensitive when using kubectl,
|
||||
and you can use either the singular or plural forms defined in the CRD,
|
||||
as well as any short names.
|
||||
|
||||
You can also view the raw YAML data:
|
||||
|
||||
```shell
|
||||
kubectl get ct -o yaml
|
||||
```
|
||||
|
||||
You should see that it contains the custom `cronSpec` and `image` fields
|
||||
from the yaml you used to create it:
|
||||
|
||||
```console
|
||||
apiVersion: v1
|
||||
items:
|
||||
- apiVersion: stable.example.com/v1
|
||||
kind: CronTab
|
||||
metadata:
|
||||
clusterName: ""
|
||||
creationTimestamp: 2017-05-31T12:56:35Z
|
||||
deletionGracePeriodSeconds: null
|
||||
deletionTimestamp: null
|
||||
name: my-new-cron-object
|
||||
namespace: default
|
||||
resourceVersion: "285"
|
||||
selfLink: /apis/stable.example.com/v1/namespaces/default/crontabs/my-new-cron-object
|
||||
uid: 9423255b-4600-11e7-af6a-28d2447dc82b
|
||||
spec:
|
||||
cronSpec: '* * * * */5'
|
||||
image: my-awesome-cron-image
|
||||
kind: List
|
||||
metadata:
|
||||
resourceVersion: ""
|
||||
selfLink: ""
|
||||
```
|
||||
|
||||
## Delete a CustomResourceDefinition
|
||||
|
||||
When you delete a CustomResourceDefinition, the server will uninstall the RESTful API endpoint
|
||||
and **delete all custom objects stored in it**.
|
||||
|
||||
```shell
|
||||
kubectl delete -f resourcedefinition.yaml
|
||||
kubectl get crontabs
|
||||
```
|
||||
|
||||
```console
|
||||
Error from server (NotFound): Unable to list "crontabs": the server could not find the requested resource (get crontabs.stable.example.com)
|
||||
```
|
||||
|
||||
If you later recreate the same CustomResourceDefinition, it will start out empty.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture discussion %}}
|
||||
## Advanced topics
|
||||
|
||||
### Finalizers
|
||||
|
||||
*Finalizers* allow controllers to implement asynchronous pre-delete hooks.
|
||||
Custom objects support finalizers just like built-in objects.
|
||||
|
||||
You can add a finalizer to a custom object like this:
|
||||
|
||||
```yaml
|
||||
apiVersion: "stable.example.com/v1"
|
||||
kind: CronTab
|
||||
metadata:
|
||||
finalizers:
|
||||
- finalizer.stable.example.com
|
||||
```
|
||||
|
||||
Finalizers are arbitrary string values, that when present ensure that a hard delete
|
||||
of a resource is not possible while they exist.
|
||||
|
||||
The first delete request on an object with finalizers merely sets a value for the
|
||||
`metadata.deletionTimestamp` field instead of deleting it. Once this value is set,
|
||||
entries in the `finalizer` list can only be removed.
|
||||
|
||||
This triggers controllers watching the object to execute any finalizers they handle.
|
||||
This will be represented via polling update requests for that
|
||||
object, until all finalizers have been removed and the resource is deleted.
|
||||
|
||||
The time period of polling update can be controlled by `metadata.deletionGracePeriodSeconds`.
|
||||
|
||||
It is the responsibility of each controller to removes its finalizer from the list.
|
||||
|
||||
Kubernetes will only finally delete the object if the list of finalizers is empty,
|
||||
meaning all finalizers are done.
|
||||
|
||||
### Validation
|
||||
|
||||
Validation of custom objects is possible via
|
||||
[OpenAPI v3 schema](https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.0.md#schemaObject).
|
||||
Additionally, the following restrictions are applied to the schema:
|
||||
|
||||
- The fields `default`, `nullable`, `discriminator`, `readOnly`, `writeOnly`, `xml`,
|
||||
`deprecated` and `$ref` cannot be set.
|
||||
- The field `uniqueItems` cannot be set to true.
|
||||
- The field `additionalProperties` cannot be set to false.
|
||||
|
||||
This feature is __beta__ in v1.9.
|
||||
You can disable this feature using the `CustomResourceValidation` feature gate on
|
||||
the [kube-apiserver](/docs/admin/kube-apiserver):
|
||||
|
||||
```
|
||||
--feature-gates=CustomResourceValidation=false
|
||||
```
|
||||
|
||||
The schema is defined in the CustomResourceDefinition. In the following example, the
|
||||
CustomResourceDefinition applies the following validations on the custom object:
|
||||
|
||||
- `spec.cronSpec` must be a string and must be of the form described by the regular expression.
|
||||
- `spec.replicas` must be an integer and must have a minimum value of 1 and a maximum value of 10.
|
||||
|
||||
Save the CustomResourceDefinition to `resourcedefinition.yaml`:
|
||||
|
||||
```yaml
|
||||
apiVersion: apiextensions.k8s.io/v1beta1
|
||||
kind: CustomResourceDefinition
|
||||
metadata:
|
||||
name: crontabs.stable.example.com
|
||||
spec:
|
||||
group: stable.example.com
|
||||
version: v1
|
||||
scope: Namespaced
|
||||
names:
|
||||
plural: crontabs
|
||||
singular: crontab
|
||||
kind: CronTab
|
||||
shortNames:
|
||||
- ct
|
||||
validation:
|
||||
# openAPIV3Schema is the schema for validating custom objects.
|
||||
openAPIV3Schema:
|
||||
properties:
|
||||
spec:
|
||||
properties:
|
||||
cronSpec:
|
||||
type: string
|
||||
pattern: '^(\d+|\*)(/\d+)?(\s+(\d+|\*)(/\d+)?){4}$'
|
||||
replicas:
|
||||
type: integer
|
||||
minimum: 1
|
||||
maximum: 10
|
||||
```
|
||||
|
||||
And create it:
|
||||
|
||||
```shell
|
||||
kubectl create -f resourcedefinition.yaml
|
||||
```
|
||||
|
||||
A request to create a custom object of kind `CronTab` will be rejected if there are invalid values in its fields.
|
||||
In the following example, the custom object contains fields with invalid values:
|
||||
|
||||
- `spec.cronSpec` does not match the regular expression.
|
||||
- `spec.replicas` is greater than 10.
|
||||
|
||||
If you save the following YAML to `my-crontab.yaml`:
|
||||
|
||||
```yaml
|
||||
apiVersion: "stable.example.com/v1"
|
||||
kind: CronTab
|
||||
metadata:
|
||||
name: my-new-cron-object
|
||||
spec:
|
||||
cronSpec: "* * * *"
|
||||
image: my-awesome-cron-image
|
||||
replicas: 15
|
||||
```
|
||||
|
||||
and create it:
|
||||
|
||||
```shell
|
||||
kubectl create -f my-crontab.yaml
|
||||
```
|
||||
|
||||
you will get an error:
|
||||
|
||||
```console
|
||||
The CronTab "my-new-cron-object" is invalid: []: Invalid value: map[string]interface {}{"apiVersion":"stable.example.com/v1", "kind":"CronTab", "metadata":map[string]interface {}{"name":"my-new-cron-object", "namespace":"default", "deletionTimestamp":interface {}(nil), "deletionGracePeriodSeconds":(*int64)(nil), "creationTimestamp":"2017-09-05T05:20:07Z", "uid":"e14d79e7-91f9-11e7-a598-f0761cb232d1", "selfLink":"", "clusterName":""}, "spec":map[string]interface {}{"cronSpec":"* * * *", "image":"my-awesome-cron-image", "replicas":15}}:
|
||||
validation failure list:
|
||||
spec.cronSpec in body should match '^(\d+|\*)(/\d+)?(\s+(\d+|\*)(/\d+)?){4}$'
|
||||
spec.replicas in body should be less than or equal to 10
|
||||
```
|
||||
|
||||
If the fields contain valid values, the object creation request is accepted.
|
||||
|
||||
Save the following YAML to `my-crontab.yaml`:
|
||||
|
||||
```yaml
|
||||
apiVersion: "stable.example.com/v1"
|
||||
kind: CronTab
|
||||
metadata:
|
||||
name: my-new-cron-object
|
||||
spec:
|
||||
cronSpec: "* * * * */5"
|
||||
image: my-awesome-cron-image
|
||||
replicas: 5
|
||||
```
|
||||
|
||||
And create it:
|
||||
|
||||
```shell
|
||||
kubectl create -f my-crontab.yaml
|
||||
crontab "my-new-cron-object" created
|
||||
```
|
||||
|
||||
### Subresources
|
||||
|
||||
Custom resources support `/status` and `/scale` subresources.
|
||||
This feature is __alpha__ in v1.10 and may change in backward incompatible ways.
|
||||
|
||||
Enable this feature using the `CustomResourceSubresources` feature gate on
|
||||
the [kube-apiserver](/docs/admin/kube-apiserver):
|
||||
|
||||
```
|
||||
--feature-gates=CustomResourceSubresources=true
|
||||
```
|
||||
|
||||
When the `CustomResourceSubresources` feature gate is enabled, only the `properties` construct
|
||||
is allowed in the root schema for custom resource validation.
|
||||
|
||||
The status and scale subresources can be optionally enabled by
|
||||
defining them in the CustomResourceDefinition.
|
||||
|
||||
#### Status subresource
|
||||
|
||||
When the status subresource is enabled, the `/status` subresource for the custom resource is exposed.
|
||||
|
||||
- The status and the spec stanzas are represented by the `.status` and `.spec` JSONPaths respectively inside of a custom resource.
|
||||
- `PUT` requests to the `/status` subresource take a custom resource object and ignore changes to anything except the status stanza.
|
||||
- `PUT` requests to the `/status` subresource only validate the status stanza of the custom resource.
|
||||
- `PUT`/`POST`/`PATCH` requests to the custom resource ignore changes to the status stanza.
|
||||
- Any changes to the spec stanza increments the value at `.metadata.generation`.
|
||||
|
||||
#### Scale subresource
|
||||
|
||||
When the scale subresource is enabled, the `/scale` subresource for the custom resource is exposed.
|
||||
The `autoscaling/v1.Scale` object is sent as the payload for `/scale`.
|
||||
|
||||
To enable the scale subresource, the following values are defined in the CustomResourceDefinition.
|
||||
|
||||
- `SpecReplicasPath` defines the JSONPath inside of a custom resource that corresponds to `Scale.Spec.Replicas`.
|
||||
|
||||
- It is a required value.
|
||||
- Only JSONPaths under `.spec` and with the dot notation are allowed.
|
||||
- If there is no value under the `SpecReplicasPath` in the custom resource,
|
||||
the `/scale` subresource will return an error on GET.
|
||||
|
||||
- `StatusReplicasPath` defines the JSONPath inside of a custom resource that corresponds to `Scale.Status.Replicas`.
|
||||
|
||||
- It is a required value.
|
||||
- Only JSONPaths under `.status` and with the dotation are allowed.
|
||||
- If there is no value under the `StatusReplicasPath` in the custom resource,
|
||||
the status replica value in the `/scale` subresource will default to 0.
|
||||
|
||||
- `LabelSelectorPath` defines the JSONPath inside of a custom resource that corresponds to `Scale.Status.Selector`.
|
||||
|
||||
- It is an optional value.
|
||||
- It must be set to work with HPA.
|
||||
- Only JSONPaths under `.status` and with the dotation are allowed.
|
||||
- If there is no value under the `LabelSelectorPath` in the custom resource,
|
||||
the status selector value in the `/scale` subresource will default to the empty string.
|
||||
|
||||
In the following example, both status and scale subresources are enabled.
|
||||
|
||||
Save the CustomResourceDefinition to `resourcedefinition.yaml`:
|
||||
|
||||
```yaml
|
||||
apiVersion: apiextensions.k8s.io/v1beta1
|
||||
kind: CustomResourceDefinition
|
||||
metadata:
|
||||
name: crontabs.stable.example.com
|
||||
spec:
|
||||
group: stable.example.com
|
||||
version: v1
|
||||
scope: Namespaced
|
||||
names:
|
||||
plural: crontabs
|
||||
singular: crontab
|
||||
kind: CronTab
|
||||
shortNames:
|
||||
- ct
|
||||
# subresources describes the subresources for custom resources.
|
||||
subresources:
|
||||
# status enables the status subresource.
|
||||
status: {}
|
||||
# scale enables the scale subresource.
|
||||
scale:
|
||||
# specReplicasPath defines the JSONPath inside of a custom resource that corresponds to Scale.Spec.Replicas.
|
||||
specReplicasPath: .spec.replicas
|
||||
# statusReplicasPath defines the JSONPath inside of a custom resource that corresponds to Scale.Status.Replicas.
|
||||
statusReplicasPath: .status.replicas
|
||||
# labelSelectorPath defines the JSONPath inside of a custom resource that corresponds to Scale.Status.Selector.
|
||||
labelSelectorPath: .status.labelSelector
|
||||
```
|
||||
|
||||
And create it:
|
||||
|
||||
```shell
|
||||
kubectl create -f resourcedefinition.yaml
|
||||
```
|
||||
|
||||
After the CustomResourceDefinition object has been created, you can create custom objects.
|
||||
|
||||
If you save the following YAML to `my-crontab.yaml`:
|
||||
|
||||
```yaml
|
||||
apiVersion: "stable.example.com/v1"
|
||||
kind: CronTab
|
||||
metadata:
|
||||
name: my-new-cron-object
|
||||
spec:
|
||||
cronSpec: "* * * * */5"
|
||||
image: my-awesome-cron-image
|
||||
replicas: 3
|
||||
```
|
||||
|
||||
and create it:
|
||||
|
||||
```shell
|
||||
kubectl create -f my-crontab.yaml
|
||||
```
|
||||
|
||||
Then new namespaced RESTful API endpoints are created at:
|
||||
|
||||
```
|
||||
/apis/stable.example.com/v1/namespaces/*/crontabs/status
|
||||
```
|
||||
|
||||
and
|
||||
|
||||
```
|
||||
/apis/stable.example.com/v1/namespaces/*/crontabs/scale
|
||||
```
|
||||
|
||||
A custom resource can be scaled using the `kubectl scale` command.
|
||||
For example, the following command sets `.spec.replicas` of the
|
||||
custom resource created above to 5:
|
||||
|
||||
```shell
|
||||
kubectl scale --replicas=5 crontabs/my-new-cron-object
|
||||
crontabs "my-new-cron-object" scaled
|
||||
|
||||
kubectl get crontabs my-new-cron-object -o jsonpath='{.spec.replicas}'
|
||||
5
|
||||
```
|
||||
|
||||
### Categories
|
||||
|
||||
Categories is a list of grouped resources the custom resource belongs to (eg. `all`).
|
||||
You can use `kubectl get <category-name>` to list the resources belonging to the category.
|
||||
This feature is __beta__ and available for custom resources from v1.10.
|
||||
|
||||
The following example adds `all` in the list of categories in the CustomResourceDefinition
|
||||
and illustrates how to output the custom resource using `kubectl get all`.
|
||||
|
||||
Save the following CustomResourceDefinition to `resourcedefinition.yaml`:
|
||||
|
||||
```yaml
|
||||
apiVersion: apiextensions.k8s.io/v1beta1
|
||||
kind: CustomResourceDefinition
|
||||
metadata:
|
||||
name: crontabs.stable.example.com
|
||||
spec:
|
||||
group: stable.example.com
|
||||
version: v1
|
||||
scope: Namespaced
|
||||
names:
|
||||
plural: crontabs
|
||||
singular: crontab
|
||||
kind: CronTab
|
||||
shortNames:
|
||||
- ct
|
||||
# categories is a list of grouped resources the custom resource belongs to.
|
||||
categories:
|
||||
- all
|
||||
```
|
||||
|
||||
And create it:
|
||||
|
||||
```shell
|
||||
kubectl create -f resourcedefinition.yaml
|
||||
```
|
||||
|
||||
After the CustomResourceDefinition object has been created, you can create custom objects.
|
||||
|
||||
Save the following YAML to `my-crontab.yaml`:
|
||||
|
||||
```yaml
|
||||
apiVersion: "stable.example.com/v1"
|
||||
kind: CronTab
|
||||
metadata:
|
||||
name: my-new-cron-object
|
||||
spec:
|
||||
cronSpec: "* * * * */5"
|
||||
image: my-awesome-cron-image
|
||||
```
|
||||
|
||||
and create it:
|
||||
|
||||
```shell
|
||||
kubectl create -f my-crontab.yaml
|
||||
```
|
||||
|
||||
You can specify the category using `kubectl get`:
|
||||
|
||||
```
|
||||
kubectl get all
|
||||
```
|
||||
|
||||
and it will include the custom resources of kind `CronTab`:
|
||||
|
||||
```console
|
||||
NAME AGE
|
||||
crontabs/my-new-cron-object 3s
|
||||
```
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture whatsnext %}}
|
||||
* Learn how to [Migrate a ThirdPartyResource to CustomResourceDefinition](/docs/tasks/access-kubernetes-api/migrate-third-party-resource/).
|
||||
* See [CustomResourceDefinition](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#customresourcedefinition-v1beta1-apiextensions).
|
||||
{{% /capture %}}
|
||||
|
||||
|
||||
@@ -0,0 +1,141 @@
|
||||
---
|
||||
reviewers:
|
||||
- enisoc
|
||||
- IanLewis
|
||||
title: Extend the Kubernetes API with ThirdPartyResources
|
||||
---
|
||||
|
||||
{{< feature-state for_k8s_version="1.7" state="deprecated" >}}
|
||||
|
||||
{{< toc >}}
|
||||
|
||||
## What is ThirdPartyResource?
|
||||
|
||||
**ThirdPartyResource is deprecated as of Kubernetes 1.7 and has been removed in version 1.8 in
|
||||
accordance with the [deprecation policy](/docs/reference/deprecation-policy) for beta features.**
|
||||
|
||||
**To avoid losing data stored in ThirdPartyResources, you must
|
||||
[migrate to CustomResourceDefinition](/docs/tasks/access-kubernetes-api/migrate-third-party-resource/)
|
||||
before upgrading to Kubernetes 1.8 or higher.**
|
||||
|
||||
Kubernetes comes with many built-in API objects. However, there are often times when you might need to extend Kubernetes with your own API objects in order to do custom automation.
|
||||
|
||||
`ThirdPartyResource` objects are a way to extend the Kubernetes API with a new API object type. The new API object type will be given an API endpoint URL and support CRUD operations, and watch API. You can then create custom objects using this API endpoint. You can think of `ThirdPartyResources` as being much like the schema for a database table. Once you have created the table, you can then start storing rows in the table. Once created, `ThirdPartyResources` can act as the data model behind custom controllers or automation programs.
|
||||
|
||||
## Structure of a ThirdPartyResource
|
||||
|
||||
Each `ThirdPartyResource` has the following:
|
||||
|
||||
* `metadata` - Standard Kubernetes object metadata.
|
||||
* `kind` - The kind of the resources described by this third party resource.
|
||||
* `description` - A free text description of the resource.
|
||||
* `versions` - A list of the versions of the resource.
|
||||
|
||||
The `kind` for a `ThirdPartyResource` takes the form `<kind name>.<domain>`. You are expected to provide a unique kind and domain name in order to avoid conflicts with other `ThirdPartyResource` objects. Kind names will be converted to CamelCase when creating instances of the `ThirdPartyResource`. Hyphens in the `kind` are assumed to be word breaks. For instance the kind `camel-case` would be converted to `CamelCase` but `camelcase` would be converted to `Camelcase`.
|
||||
|
||||
Other fields on the `ThirdPartyResource` are treated as custom data fields. These fields can hold arbitrary JSON data and have any structure.
|
||||
|
||||
You can view the full documentation about `ThirdPartyResources` using the `explain` command in kubectl.
|
||||
|
||||
```
|
||||
$ kubectl explain thirdpartyresource
|
||||
```
|
||||
|
||||
## Creating a ThirdPartyResource
|
||||
|
||||
When you create a new `ThirdPartyResource`, the Kubernetes API Server reacts by creating a new, namespaced RESTful resource path. For now, non-namespaced objects are not supported. As with existing built-in objects, deleting a namespace deletes all custom objects in that namespace. `ThirdPartyResources` themselves are non-namespaced and are available to all namespaces.
|
||||
|
||||
For example, if you save the following `ThirdPartyResource` to `resource.yaml`:
|
||||
|
||||
```yaml
|
||||
apiVersion: extensions/v1beta1
|
||||
kind: ThirdPartyResource
|
||||
metadata:
|
||||
name: cron-tab.stable.example.com
|
||||
description: "A specification of a Pod to run on a cron style schedule"
|
||||
versions:
|
||||
- name: v1
|
||||
```
|
||||
|
||||
And create it:
|
||||
|
||||
```shell
|
||||
$ kubectl create -f resource.yaml
|
||||
thirdpartyresource "cron-tab.stable.example.com" created
|
||||
```
|
||||
|
||||
Then a new RESTful API endpoint is created at:
|
||||
|
||||
`/apis/stable.example.com/v1/namespaces/<namespace>/crontabs/...`
|
||||
|
||||
This endpoint URL can then be used to create and manage custom objects.
|
||||
The `kind` of these objects will be `CronTab` following the camel case
|
||||
rules applied to the `metadata.name` of this `ThirdPartyResource`
|
||||
(`cron-tab.stable.example.com`)
|
||||
|
||||
## Creating Custom Objects
|
||||
|
||||
After the `ThirdPartyResource` object has been created you can create custom objects. Custom objects can contain custom fields. These fields can contain arbitrary JSON.
|
||||
In the following example, a `cronSpec` and `image` custom fields are set to the custom object of kind `CronTab`. The kind `CronTab` is derived from the
|
||||
`metadata.name` of the `ThirdPartyResource` object we created above.
|
||||
|
||||
If you save the following YAML to `my-crontab.yaml`:
|
||||
|
||||
```yaml
|
||||
apiVersion: "stable.example.com/v1"
|
||||
kind: CronTab
|
||||
metadata:
|
||||
name: my-new-cron-object
|
||||
cronSpec: "* * * * /5"
|
||||
image: my-awesome-cron-image
|
||||
```
|
||||
|
||||
and create it:
|
||||
|
||||
```shell
|
||||
$ kubectl create -f my-crontab.yaml
|
||||
crontab "my-new-cron-object" created
|
||||
```
|
||||
|
||||
You can then manage our `CronTab` objects using kubectl. Note that resource names are not case-sensitive when using kubectl:
|
||||
|
||||
```shell
|
||||
$ kubectl get crontab
|
||||
NAME KIND
|
||||
my-new-cron-object CronTab.v1.stable.example.com
|
||||
```
|
||||
|
||||
You can also view the raw JSON data. Here you can see that it contains the custom `cronSpec` and `image` fields from the yaml you used to create it:
|
||||
|
||||
```yaml
|
||||
$ kubectl get crontab -o json
|
||||
{
|
||||
"apiVersion": "v1",
|
||||
"items": [
|
||||
{
|
||||
"apiVersion": "stable.example.com/v1",
|
||||
"cronSpec": "* * * * /5",
|
||||
"image": "my-awesome-cron-image",
|
||||
"kind": "CronTab",
|
||||
"metadata": {
|
||||
"creationTimestamp": "2016-09-29T04:59:00Z",
|
||||
"name": "my-new-cron-object",
|
||||
"namespace": "default",
|
||||
"resourceVersion": "12601503",
|
||||
"selfLink": "/apis/stable.example.com/v1/namespaces/default/crontabs/my-new-cron-object",
|
||||
"uid": "6f65e7a3-8601-11e6-a23e-42010af0000c"
|
||||
}
|
||||
}
|
||||
],
|
||||
"kind": "List",
|
||||
"metadata": {},
|
||||
"resourceVersion": "",
|
||||
"selfLink": ""
|
||||
}
|
||||
```
|
||||
|
||||
## What's next
|
||||
|
||||
* [Migrate a ThirdPartyResource to a CustomResourceDefinition](/docs/tasks/access-kubernetes-api/migrate-third-party-resource/)
|
||||
* [Extend the Kubernetes API with CustomResourceDefinitions](/docs/tasks/access-kubernetes-api/extend-api-custom-resource-definitions/)
|
||||
* [ThirdPartyResource](https://v1-7.docs.kubernetes.io/docs/reference/v1.7/#thirdpartyresource-v1beta1-extensions)
|
||||
@@ -0,0 +1,85 @@
|
||||
---
|
||||
title: Use an HTTP Proxy to Access the Kubernetes API
|
||||
content_template: templates/task
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
This page shows how to use an HTTP proxy to access the Kubernetes API.
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture prerequisites %}}
|
||||
|
||||
* {{< include "task-tutorial-prereqs.md" >}} {{< version-check >}}
|
||||
|
||||
* If you do not already have an application running in your cluster, start
|
||||
a Hello world application by entering this command:
|
||||
|
||||
kubectl run node-hello --image=gcr.io/google-samples/node-hello:1.0 --port=8080
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture steps %}}
|
||||
|
||||
## Using kubectl to start a proxy server
|
||||
|
||||
This command starts a proxy to the Kubernetes API server:
|
||||
|
||||
kubectl proxy --port=8080
|
||||
|
||||
## Exploring the Kubernetes API
|
||||
|
||||
When the proxy server is running, you can explore the API using `curl`, `wget`,
|
||||
or a browser.
|
||||
|
||||
Get the API versions:
|
||||
|
||||
curl http://localhost:8080/api/
|
||||
|
||||
{
|
||||
"kind": "APIVersions",
|
||||
"versions": [
|
||||
"v1"
|
||||
],
|
||||
"serverAddressByClientCIDRs": [
|
||||
{
|
||||
"clientCIDR": "0.0.0.0/0",
|
||||
"serverAddress": "10.0.2.15:8443"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
Get a list of pods:
|
||||
|
||||
curl http://localhost:8080/api/v1/namespaces/default/pods
|
||||
|
||||
{
|
||||
"kind": "PodList",
|
||||
"apiVersion": "v1",
|
||||
"metadata": {
|
||||
"selfLink": "/api/v1/namespaces/default/pods",
|
||||
"resourceVersion": "33074"
|
||||
},
|
||||
"items": [
|
||||
{
|
||||
"metadata": {
|
||||
"name": "kubernetes-bootcamp-2321272333-ix8pt",
|
||||
"generateName": "kubernetes-bootcamp-2321272333-",
|
||||
"namespace": "default",
|
||||
"selfLink": "/api/v1/namespaces/default/pods/kubernetes-bootcamp-2321272333-ix8pt",
|
||||
"uid": "ba21457c-6b1d-11e6-85f7-1ef9f1dab92b",
|
||||
"resourceVersion": "33003",
|
||||
"creationTimestamp": "2016-08-25T23:43:30Z",
|
||||
"labels": {
|
||||
"pod-template-hash": "2321272333",
|
||||
"run": "kubernetes-bootcamp"
|
||||
},
|
||||
...
|
||||
}
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture whatsnext %}}
|
||||
Learn more about [kubectl proxy](/docs/reference/generated/kubectl/kubectl-commands#proxy).
|
||||
{{% /capture %}}
|
||||
|
||||
|
||||
@@ -0,0 +1,170 @@
|
||||
---
|
||||
title: Migrate a ThirdPartyResource to CustomResourceDefinition
|
||||
reviewers:
|
||||
- enisoc
|
||||
- deads2k
|
||||
content_template: templates/task
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
This page shows how to migrate data stored in a ThirdPartyResource (TPR) to a
|
||||
[CustomResourceDefinition](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#customresourcedefinition-v1beta1-apiextensions) (CRD).
|
||||
|
||||
Kubernetes does not automatically migrate existing TPRs.
|
||||
This is due to API changes introduced as part of
|
||||
[graduating to beta](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/api-machinery/thirdpartyresources.md)
|
||||
under a new name and API group.
|
||||
Instead, both TPR and CRD are available and operate independently in Kubernetes 1.7.
|
||||
Users must migrate each TPR one by one to preserve their data before upgrading to Kubernetes 1.8.
|
||||
|
||||
The simplest way to migrate is to stop all clients that use a given TPR, then delete the TPR and
|
||||
start from scratch with a CRD.
|
||||
This page describes an optional process that eases the transition by migrating existing TPR data for
|
||||
you **on a best-effort basis**.
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture prerequisites %}}
|
||||
{{< include "task-tutorial-prereqs.md" >}} {{< version-check >}}
|
||||
|
||||
* Make sure your Kubernetes cluster has a **master version of exactly 1.7.x** (any patch release),
|
||||
as this is the only version that supports both TPR and CRD.
|
||||
* If you use a TPR-based custom controller, check with the author of the controller first.
|
||||
Some or all of these steps may be unnecessary if the custom controller handles the migration for
|
||||
you.
|
||||
* Be familiar with the concept of [custom resources](/docs/concepts/api-extension/custom-resources/),
|
||||
which were known as *third-party resources* until Kubernetes 1.7.
|
||||
* Be familiar with [CustomResourceDefinitions](/docs/concepts/api-extension/custom-resources/#customresourcedefinitions),
|
||||
which are a simple way to implement custom resources.
|
||||
* **Before performing a migration on real data, conduct a dry run by going through these steps in a test cluster.**
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture steps %}}
|
||||
## Migrate TPR data
|
||||
|
||||
1. **Rewrite the TPR definition**
|
||||
|
||||
Clients that access the REST API for your custom resource should not need any changes.
|
||||
However, you will need to rewrite your TPR definition as a CRD.
|
||||
|
||||
Make sure you specify values for the CRD fields that match what the server used to fill in for
|
||||
you with TPR.
|
||||
|
||||
For example, if your ThirdPartyResource looks like this:
|
||||
|
||||
```yaml
|
||||
apiVersion: extensions/v1beta1
|
||||
kind: ThirdPartyResource
|
||||
metadata:
|
||||
name: cron-tab.stable.example.com
|
||||
description: "A specification of a Pod to run on a cron style schedule"
|
||||
versions:
|
||||
- name: v1
|
||||
```
|
||||
|
||||
A matching CustomResourceDefinition could look like this:
|
||||
|
||||
```yaml
|
||||
apiVersion: apiextensions.k8s.io/v1beta1
|
||||
kind: CustomResourceDefinition
|
||||
metadata:
|
||||
name: crontabs.stable.example.com
|
||||
spec:
|
||||
scope: Namespaced
|
||||
group: stable.example.com
|
||||
version: v1
|
||||
names:
|
||||
kind: CronTab
|
||||
plural: crontabs
|
||||
singular: crontab
|
||||
```
|
||||
|
||||
1. **Install the CustomResourceDefinition**
|
||||
|
||||
While the source TPR is still active, install the matching CRD with `kubectl create`.
|
||||
Existing TPR data remains accessible because TPRs take precedence over CRDs when both try
|
||||
to serve the same resource.
|
||||
|
||||
After you create the CRD, make sure the *Established* condition goes to True.
|
||||
You can check it with a command like this:
|
||||
|
||||
```shell
|
||||
kubectl get crd -o 'custom-columns=NAME:{.metadata.name},ESTABLISHED:{.status.conditions[?(@.type=="Established")].status}'
|
||||
```
|
||||
|
||||
The output should look like this:
|
||||
|
||||
```console
|
||||
NAME ESTABLISHED
|
||||
crontabs.stable.example.com True
|
||||
```
|
||||
|
||||
1. **Stop all clients that use the TPR**
|
||||
|
||||
The API server attempts to prevent TPR data for the resource from changing while it
|
||||
copies objects to the CRD, but it can't guarantee consistency in all cases, such as with
|
||||
[multiple masters](/docs/admin/high-availability/).
|
||||
Stopping clients, such as TPR-based custom controllers, helps to avoid inconsistencies in
|
||||
the copied data.
|
||||
|
||||
In addition, clients that watch TPR data do not receive any more events once the migration
|
||||
begins.
|
||||
You must restart them after the migration completes so they start watching CRD data instead.
|
||||
|
||||
1. **Back up TPR data**
|
||||
|
||||
In case the data migration fails, save a copy of existing data for the resource:
|
||||
|
||||
```shell
|
||||
kubectl get crontabs --all-namespaces -o yaml > crontabs.yaml
|
||||
```
|
||||
|
||||
You should also save a copy of the TPR definition if you don't have one already:
|
||||
|
||||
```shell
|
||||
kubectl get thirdpartyresource cron-tab.stable.example.com -o yaml --export > tpr.yaml
|
||||
```
|
||||
|
||||
1. **Delete the TPR definition**
|
||||
|
||||
Normally, when you delete a TPR definition, the API server tries to clean up any objects stored
|
||||
in that resource.
|
||||
Because a matching CRD exists, the server copies objects to the CRD instead of deleting them.
|
||||
|
||||
```shell
|
||||
kubectl delete thirdpartyresource cron-tab.stable.example.com
|
||||
```
|
||||
|
||||
1. **Verify the new CRD data**
|
||||
|
||||
It can take up to 10 seconds for the TPR controller to notice when you delete the TPR definition
|
||||
and to initiate the migration. The TPR data remains accessible during this time.
|
||||
|
||||
Once the migration completes, the resource begins serving through the CRD.
|
||||
Check that all your objects were correctly copied:
|
||||
|
||||
```shell
|
||||
kubectl get crontabs --all-namespaces -o yaml
|
||||
```
|
||||
|
||||
If the copy failed, you can quickly revert to the set of objects that existed just before the
|
||||
migration by recreating the TPR definition:
|
||||
|
||||
```shell
|
||||
kubectl create -f tpr.yaml
|
||||
```
|
||||
|
||||
1. **Restart clients**
|
||||
|
||||
After verifying the CRD data, restart any clients you stopped before the migration, such as
|
||||
custom controllers and other watchers.
|
||||
These clients now access CRD data when they make requests on the same API endpoints
|
||||
that the TPR previously served.
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture whatsnext %}}
|
||||
* Learn more about [custom resources](/docs/concepts/api-extension/custom-resources/).
|
||||
* Learn more about [using CustomResourceDefinitions](/docs/tasks/access-kubernetes-api/extend-api-custom-resource-definitions/).
|
||||
* See [CustomResourceDefinition](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#customresourcedefinition-v1beta1-apiextensions).
|
||||
{{% /capture %}}
|
||||
|
||||
|
||||
@@ -0,0 +1,58 @@
|
||||
---
|
||||
title: Setup an extension API server
|
||||
reviewers:
|
||||
- lavalamp
|
||||
- cheftako
|
||||
- chenopis
|
||||
content_template: templates/task
|
||||
---
|
||||
|
||||
{{% capture overview %}}
|
||||
|
||||
Setting up an extension API server to work the aggregation layer allows the Kubernetes apiserver to be extended with additional APIs, which are not part of the core Kubernetes APIs.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture prerequisites %}}
|
||||
|
||||
* You need to have a Kubernetes cluster running.
|
||||
* You must [configure the aggregation layer](/docs/tasks/access-kubernetes-api/configure-aggregation-layer/) and enable the apiserver flags.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture steps %}}
|
||||
|
||||
## Setup an extension api-server to work with the aggregation layer
|
||||
|
||||
The following steps describe how to set up an extension-apiserver *at a high level*. These steps apply regardless if you're using YAML configs or using APIs. An attempt is made to specifically identify any differences between the two. For a concrete example of how they can be implemented using YAML configs, you can look at the [sample-apiserver](https://github.com/kubernetes/sample-apiserver/blob/master/README.md) in the Kubernetes repo.
|
||||
|
||||
Alternatively, you can use an existing 3rd party solution, such as [apiserver-builder](https://github.com/Kubernetes-incubator/apiserver-builder/blob/master/README.md), which should generate a skeleton and automate all of the following steps for you.
|
||||
|
||||
1. Make sure the APIService API is enabled (check `--runtime-config`). It should be on by default, unless it's been deliberately turned off in your cluster.
|
||||
1. You may need to make an RBAC rule allowing you to add APIService objects, or get your cluster administrator to make one. (Since API extensions affect the entire cluster, it is not recommended to do testing/development/debug of an API extension in a live cluster.)
|
||||
1. Create the Kubernetes namespace you want to run your extension api-service in.
|
||||
1. Create/get a CA cert to be used to sign the server cert the extension api-server uses for HTTPS.
|
||||
1. Create a server cert/key for the api-server to use for HTTPS. This cert should be signed by the above CA. It should also have a CN of the Kube DNS name. This is derived from the Kubernetes service and be of the form `<service name>.<service name namespace>.svc`
|
||||
1. Create a Kubernetes secret with the server cert/key in your namespace.
|
||||
1. Create a Kubernetes deployment for the extension api-server and make sure you are loading the secret as a volume. It should contain a reference to a working image of your extension api-server. The deployment should also be in your namespace.
|
||||
1. Make sure that your extension-apiserver loads those certs from that volume and that they are used in the HTTPS handshake.
|
||||
1. Create a Kubernetes service account in your namespace.
|
||||
1. Create a Kubernetes cluster role for the operations you want to allow on your resources.
|
||||
1. Create a Kubernetes cluster role binding from the service account in your namespace to the cluster role you just created.
|
||||
1. Create a Kubernetes cluster role binding from the service account in your namespace to the `system:auth-delegator` cluster role to delegate auth decisions to the Kubernetes core API server.
|
||||
1. Create a Kubernetes role binding from the service account in your namespace to the `extension-apiserver-authentication-reader` role. This allows your extension api-server to access the `extension-apiserver-authentication` configmap.
|
||||
1. Create a Kubernetes apiservice. The CA cert above should be base64 encoded, stripped of new lines and used as the spec.caBundle in the apiservice. This should not be namespaced. If using the [kube-aggregator API](https://github.com/kubernetes/kube-aggregator/), only pass in the PEM encoded CA bundle because the base 64 encoding is done for you.
|
||||
1. Use kubectl to get your resource. It should return "No resources found." Which means that everything worked but you currently have no objects of that resource type created yet.
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
{{% capture whatsnext %}}
|
||||
|
||||
* If you haven't already, [configure the aggregation layer](/docs/tasks/access-kubernetes-api/configure-aggregation-layer/) and enable the apiserver flags.
|
||||
* For a high level overview, see [Extending the Kubernetes API with the aggregation layer](/docs/concepts/api-extension/apiserver-aggregation).
|
||||
* Learn how to [Extend the Kubernetes API Using Custom Resource Definitions](/docs/tasks/access-kubernetes-api/extend-api-custom-resource-definitions/).
|
||||
|
||||
{{% /capture %}}
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user