whentimeslows/website
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial checkin of v1.1 -- does not build

Browse Source
This commit is contained in:
John Mulhausen
2016-02-10 16:55:31 -08:00
parent d1da8c3615
commit a0fb30a6fb
420 changed files with 49449 additions and 158 deletions
Download Patch File Download Diff File Expand all files Collapse all files
v1.1/docs/devel/README.md
+96
View File
@@ -0,0 +1,96 @@
---
layout: docwithnav
title: "Kubernetes Developer Guide"
---
<!-- BEGIN MUNGE: UNVERSIONED_WARNING -->
<!-- END MUNGE: UNVERSIONED_WARNING -->
# Kubernetes Developer Guide
The developer guide is for anyone wanting to either write code which directly accesses the
Kubernetes API, or to contribute directly to the Kubernetes project.
It assumes some familiarity with concepts in the [User Guide](../user-guide/README.html) and the [Cluster Admin
Guide](../admin/README.html).
## The process of developing and contributing code to the Kubernetes project
* **On Collaborative Development** ([collab.md](collab.html)): Info on pull requests and code reviews.
* **GitHub Issues** ([issues.md](issues.html)): How incoming issues are reviewed and prioritized.
* **Pull Request Process** ([pull-requests.md](pull-requests.html)): When and why pull requests are closed.
* **Faster PR reviews** ([faster_reviews.md](faster_reviews.html)): How to get faster PR reviews.
* **Getting Recent Builds** ([getting-builds.md](getting-builds.html)): How to get recent builds including the latest builds that pass CI.
* **Automated Tools** ([automation.md](automation.html)): Descriptions of the automation that is running on our github repository.
## Setting up your dev environment, coding, and debugging
* **Development Guide** ([development.md](development.html)): Setting up your development environment.
* **Hunting flaky tests** ([flaky-tests.md](flaky-tests.html)): We have a goal of 99.9% flake free tests.
Here's how to run your tests many times.
* **Logging Conventions** ([logging.md](logging.html)]: Glog levels.
* **Profiling Kubernetes** ([profiling.md](profiling.html)): How to plug in go pprof profiler to Kubernetes.
* **Instrumenting Kubernetes with a new metric**
([instrumentation.md](instrumentation.html)): How to add a new metrics to the
Kubernetes code base.
* **Coding Conventions** ([coding-conventions.md](coding-conventions.html)):
Coding style advice for contributors.
## Developing against the Kubernetes API
* API objects are explained at [http://kubernetes.io/third_party/swagger-ui/](http://kubernetes.io/third_party/swagger-ui/).
* **Annotations** ([docs/user-guide/annotations.md](../user-guide/annotations.html)): are for attaching arbitrary non-identifying metadata to objects.
Programs that automate Kubernetes objects may use annotations to store small amounts of their state.
* **API Conventions** ([api-conventions.md](api-conventions.html)):
Defining the verbs and resources used in the Kubernetes API.
* **API Client Libraries** ([client-libraries.md](client-libraries.html)):
A list of existing client libraries, both supported and user-contributed.
## Writing plugins
* **Authentication Plugins** ([docs/admin/authentication.md](../admin/authentication.html)):
The current and planned states of authentication tokens.
* **Authorization Plugins** ([docs/admin/authorization.md](../admin/authorization.html)):
Authorization applies to all HTTP requests on the main apiserver port.
This doc explains the available authorization implementations.
* **Admission Control Plugins** ([admission_control](../design/admission_control.html))
## Building releases
* **Making release notes** ([making-release-notes.md](making-release-notes.html)): Generating release nodes for a new release.
* **Releasing Kubernetes** ([releasing.md](releasing.html)): How to create a Kubernetes release (as in version)
and how the version information gets embedded into the built binaries.
<!-- BEGIN MUNGE: IS_VERSIONED -->
<!-- TAG IS_VERSIONED -->
<!-- END MUNGE: IS_VERSIONED -->
<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/devel/README.md?pixel)]()
<!-- END MUNGE: GENERATED_ANALYTICS -->
v1.1/docs/devel/api-conventions.md
+733
View File
@@ -0,0 +1,733 @@
---
layout: docwithnav
title: "API Conventions"
---
<!-- BEGIN MUNGE: UNVERSIONED_WARNING -->
<!-- END MUNGE: UNVERSIONED_WARNING -->
API Conventions
===============
Updated: 9/20/2015
*This document is oriented at users who want a deeper understanding of the Kubernetes
API structure, and developers wanting to extend the Kubernetes API. An introduction to
using resources with kubectl can be found in [Working with resources](../user-guide/working-with-resources.html).*
**Table of Contents**
<!-- BEGIN MUNGE: GENERATED_TOC -->
- [Types (Kinds)](#types-kinds)
- [Resources](#resources)
- [Objects](#objects)
- [Metadata](#metadata)
- [Spec and Status](#spec-and-status)
- [Typical status properties](#typical-status-properties)
- [References to related objects](#references-to-related-objects)
- [Lists of named subobjects preferred over maps](#lists-of-named-subobjects-preferred-over-maps)
- [Constants](#constants)
- [Lists and Simple kinds](#lists-and-simple-kinds)
- [Differing Representations](#differing-representations)
- [Verbs on Resources](#verbs-on-resources)
- [PATCH operations](#patch-operations)
- [Strategic Merge Patch](#strategic-merge-patch)
- [List Operations](#list-operations)
- [Map Operations](#map-operations)
- [Idempotency](#idempotency)
- [Defaulting](#defaulting)
- [Late Initialization](#late-initialization)
- [Concurrency Control and Consistency](#concurrency-control-and-consistency)
- [Serialization Format](#serialization-format)
- [Units](#units)
- [Selecting Fields](#selecting-fields)
- [Object references](#object-references)
- [HTTP Status codes](#http-status-codes)
- [Success codes](#success-codes)
- [Error codes](#error-codes)
- [Response Status Kind](#response-status-kind)
- [Events](#events)
- [Naming conventions](#naming-conventions)
- [Label, selector, and annotation conventions](#label-selector-and-annotation-conventions)
<!-- END MUNGE: GENERATED_TOC -->
The conventions of the [Kubernetes API](../api.html) (and related APIs in the ecosystem) are intended to ease client development and ensure that configuration mechanisms can be implemented that work across a diverse set of use cases consistently.
The general style of the Kubernetes API is RESTful - clients create, update, delete, or retrieve a description of an object via the standard HTTP verbs (POST, PUT, DELETE, and GET) - and those APIs preferentially accept and return JSON. Kubernetes also exposes additional endpoints for non-standard verbs and allows alternative content types. All of the JSON accepted and returned by the server has a schema, identified by the "kind" and "apiVersion" fields. Where relevant HTTP header fields exist, they should mirror the content of JSON fields, but the information should not be represented only in the HTTP header.
The following terms are defined:
* **Kind** the name of a particular object schema (e.g. the "Cat" and "Dog" kinds would have different attributes and properties)
* **Resource** a representation of a system entity, sent or retrieved as JSON via HTTP to the server. Resources are exposed via:
* Collections - a list of resources of the same type, which may be queryable
* Elements - an individual resource, addressable via a URL
Each resource typically accepts and returns data of a single kind. A kind may be accepted or returned by multiple resources that reflect specific use cases. For instance, the kind "Pod" is exposed as a "pods" resource that allows end users to create, update, and delete pods, while a separate "pod status" resource (that acts on "Pod" kind) allows automated processes to update a subset of the fields in that resource.
Resource collections should be all lowercase and plural, whereas kinds are CamelCase and singular.
## Types (Kinds)
Kinds are grouped into three categories:
1. **Objects** represent a persistent entity in the system.
Creating an API object is a record of intent - once created, the system will work to ensure that resource exists. All API objects have common metadata.
An object may have multiple resources that clients can use to perform specific actions that create, update, delete, or get.
Examples: `Pod`, `ReplicationController`, `Service`, `Namespace`, `Node`.
2. **Lists** are collections of **resources** of one (usually) or more (occasionally) kinds.
Lists have a limited set of common metadata. All lists use the "items" field to contain the array of objects they return.
Most objects defined in the system should have an endpoint that returns the full set of resources, as well as zero or more endpoints that return subsets of the full list. Some objects may be singletons (the current user, the system defaults) and may not have lists.
In addition, all lists that return objects with labels should support label filtering (see [docs/user-guide/labels.md](../user-guide/labels.html), and most lists should support filtering by fields.
Examples: PodLists, ServiceLists, NodeLists
TODO: Describe field filtering below or in a separate doc.
3. **Simple** kinds are used for specific actions on objects and for non-persistent entities.
Given their limited scope, they have the same set of limited common metadata as lists.
For instance, the "Status" kind is returned when errors occur and is not persisted in the system.
Many simple resources are "subresources", which are rooted at API paths of specific resources. When resources wish to expose alternative actions or views that are closely coupled to a single resource, they should do so using new sub-resources. Common subresources include:
* `/binding`: Used to bind a resource representing a user request (e.g., Pod, PersistentVolumeClaim) to a cluster infrastructure resource (e.g., Node, PersistentVolume).
* `/status`: Used to write just the status portion of a resource. For example, the `/pods` endpoint only allows updates to `metadata` and `spec`, since those reflect end-user intent. An automated process should be able to modify status for users to see by sending an updated Pod kind to the server to the "/pods/&lt;name&gt;/status" endpoint - the alternate endpoint allows different rules to be applied to the update, and access to be appropriately restricted.
* `/scale`: Used to read and write the count of a resource in a manner that is independent of the specific resource schema.
Two additional subresources, `proxy` and `portforward`, provide access to cluster resources as described in [docs/user-guide/accessing-the-cluster.md](../user-guide/accessing-the-cluster.html).
The standard REST verbs (defined below) MUST return singular JSON objects. Some API endpoints may deviate from the strict REST pattern and return resources that are not singular JSON objects, such as streams of JSON objects or unstructured text log data.
The term "kind" is reserved for these "top-level" API types. The term "type" should be used for distinguishing sub-categories within objects or subobjects.
### Resources
All JSON objects returned by an API MUST have the following fields:
* kind: a string that identifies the schema this object should have
* apiVersion: a string that identifies the version of the schema the object should have
These fields are required for proper decoding of the object. They may be populated by the server by default from the specified URL path, but the client likely needs to know the values in order to construct the URL path.
### Objects
#### Metadata
Every object kind MUST have the following metadata in a nested object field called "metadata":
* namespace: a namespace is a DNS compatible subdomain that objects are subdivided into. The default namespace is 'default'. See [docs/user-guide/namespaces.md](../user-guide/namespaces.html) for more.
* name: a string that uniquely identifies this object within the current namespace (see [docs/user-guide/identifiers.md](../user-guide/identifiers.html)). This value is used in the path when retrieving an individual object.
* uid: a unique in time and space value (typically an RFC 4122 generated identifier, see [docs/user-guide/identifiers.md](../user-guide/identifiers.html)) used to distinguish between objects with the same name that have been deleted and recreated
Every object SHOULD have the following metadata in a nested object field called "metadata":
* resourceVersion: a string that identifies the internal version of this object that can be used by clients to determine when objects have changed. This value MUST be treated as opaque by clients and passed unmodified back to the server. Clients should not assume that the resource version has meaning across namespaces, different kinds of resources, or different servers. (see [concurrency control](#concurrency-control-and-consistency), below, for more details)
* generation: a sequence number representing a specific generation of the desired state. Set by the system and monotonically increasing, per-resource. May be compared, such as for RAW and WAW consistency.
* creationTimestamp: a string representing an RFC 3339 date of the date and time an object was created
* deletionTimestamp: a string representing an RFC 3339 date of the date and time after which this resource will be deleted. This field is set by the server when a graceful deletion is requested by the user, and is not directly settable by a client. The resource will be deleted (no longer visible from resource lists, and not reachable by name) after the time in this field. Once set, this value may not be unset or be set further into the future, although it may be shortened or the resource may be deleted prior to this time.
* labels: a map of string keys and values that can be used to organize and categorize objects (see [docs/user-guide/labels.md](../user-guide/labels.html))
* annotations: a map of string keys and values that can be used by external tooling to store and retrieve arbitrary metadata about this object (see [docs/user-guide/annotations.md](../user-guide/annotations.html))
Labels are intended for organizational purposes by end users (select the pods that match this label query). Annotations enable third-party automation and tooling to decorate objects with additional metadata for their own use.
#### Spec and Status
By convention, the Kubernetes API makes a distinction between the specification of the desired state of an object (a nested object field called "spec") and the status of the object at the current time (a nested object field called "status"). The specification is a complete description of the desired state, including configuration settings provided by the user, [default values](#defaulting) expanded by the system, and properties initialized or otherwise changed after creation by other ecosystem components (e.g., schedulers, auto-scalers), and is persisted in stable storage with the API object. If the specification is deleted, the object will be purged from the system. The status summarizes the current state of the object in the system, and is usually persisted with the object by an automated processes but may be generated on the fly. At some cost and perhaps some temporary degradation in behavior, the status could be reconstructed by observation if it were lost.
When a new version of an object is POSTed or PUT, the "spec" is updated and available immediately. Over time the system will work to bring the "status" into line with the "spec". The system will drive toward the most recent "spec" regardless of previous versions of that stanza. In other words, if a value is changed from 2 to 5 in one PUT and then back down to 3 in another PUT the system is not required to 'touch base' at 5 before changing the "status" to 3. In other words, the system's behavior is *level-based* rather than *edge-based*. This enables robust behavior in the presence of missed intermediate state changes.
The Kubernetes API also serves as the foundation for the declarative configuration schema for the system. In order to facilitate level-based operation and expression of declarative configuration, fields in the specification should have declarative rather than imperative names and semantics -- they represent the desired state, not actions intended to yield the desired state.
The PUT and POST verbs on objects will ignore the "status" values. A `/status` subresource is provided to enable system components to update statuses of resources they manage.
Otherwise, PUT expects the whole object to be specified. Therefore, if a field is omitted it is assumed that the client wants to clear that field's value. The PUT verb does not accept partial updates. Modification of just part of an object may be achieved by GETting the resource, modifying part of the spec, labels, or annotations, and then PUTting it back. See [concurrency control](#concurrency-control-and-consistency), below, regarding read-modify-write consistency when using this pattern. Some objects may expose alternative resource representations that allow mutation of the status, or performing custom actions on the object.
All objects that represent a physical resource whose state may vary from the user's desired intent SHOULD have a "spec" and a "status". Objects whose state cannot vary from the user's desired intent MAY have only "spec", and MAY rename "spec" to a more appropriate name.
Objects that contain both spec and status should not contain additional top-level fields other than the standard metadata fields.
##### Typical status properties
**Conditions** represent the latest available observations of an object's current state. Objects may report multiple conditions, and new types of conditions may be added in the future. Therefore, conditions are represented using a list/slice, where all have similar structure.
The `FooCondition` type for some resource type `Foo` may include a subset of the following fields, but must contain at least `type` and `status` fields:
{% highlight go %}
{% raw %}
Type FooConditionType `json:"type" description:"type of Foo condition"`
Status ConditionStatus `json:"status" description:"status of the condition, one of True, False, Unknown"`
LastHeartbeatTime unversioned.Time `json:"lastHeartbeatTime,omitempty" description:"last time we got an update on a given condition"`
LastTransitionTime unversioned.Time `json:"lastTransitionTime,omitempty" description:"last time the condition transit from one status to another"`
Reason string `json:"reason,omitempty" description:"one-word CamelCase reason for the condition's last transition"`
Message string `json:"message,omitempty" description:"human-readable message indicating details about last transition"`
{% endraw %}
{% endhighlight %}
Additional fields may be added in the future.
Conditions should be added to explicitly convey properties that users and components care about rather than requiring those properties to be inferred from other observations.
Condition status values may be `True`, `False`, or `Unknown`. The absence of a condition should be interpreted the same as `Unknown`.
In general, condition values may change back and forth, but some condition transitions may be monotonic, depending on the resource and condition type. However, conditions are observations and not, themselves, state machines, nor do we define comprehensive state machines for objects, nor behaviors associated with state transitions. The system is level-based rather than edge-triggered, and should assume an Open World.
A typical oscillating condition type is `Ready`, which indicates the object was believed to be fully operational at the time it was last probed. A possible monotonic condition could be `Succeeded`. A `False` status for `Succeeded` would imply failure. An object that was still active would not have a `Succeeded` condition, or its status would be `Unknown`.
Some resources in the v1 API contain fields called **`phase`**, and associated `message`, `reason`, and other status fields. The pattern of using `phase` is deprecated. Newer API types should use conditions instead. Phase was essentially a state-machine enumeration field, that contradicted [system-design principles](../design/principles.html#control-logic) and hampered evolution, since [adding new enum values breaks backward compatibility](api_changes.html). Rather than encouraging clients to infer implicit properties from phases, we intend to explicitly expose the conditions that clients need to monitor. Conditions also have the benefit that it is possible to create some conditions with uniform meaning across all resource types, while still exposing others that are unique to specific resource types. See [#7856](http://issues.k8s.io/7856) for more details and discussion.
In condition types, and everywhere else they appear in the API, **`Reason`** is intended to be a one-word, CamelCase representation of the category of cause of the current status, and **`Message`** is intended to be a human-readable phrase or sentence, which may contain specific details of the individual occurrence. `Reason` is intended to be used in concise output, such as one-line `kubectl get` output, and in summarizing occurrences of causes, whereas `Message` is intended to be presented to users in detailed status explanations, such as `kubectl describe` output.
Historical information status (e.g., last transition time, failure counts) is only provided with reasonable effort, and is not guaranteed to not be lost.
Status information that may be large (especially proportional in size to collections of other resources, such as lists of references to other objects -- see below) and/or rapidly changing, such as [resource usage](../design/resources.html#usage-data), should be put into separate objects, with possibly a reference from the original object. This helps to ensure that GETs and watch remain reasonably efficient for the majority of clients, which may not need that data.
Some resources report the `observedGeneration`, which is the `generation` most recently observed by the component responsible for acting upon changes to the desired state of the resource. This can be used, for instance, to ensure that the reported status reflects the most recent desired status.
#### References to related objects
References to loosely coupled sets of objects, such as [pods](../user-guide/pods.html) overseen by a [replication controller](../user-guide/replication-controller.html), are usually best referred to using a [label selector](../user-guide/labels.html). In order to ensure that GETs of individual objects remain bounded in time and space, these sets may be queried via separate API queries, but will not be expanded in the referring object's status.
References to specific objects, especially specific resource versions and/or specific fields of those objects, are specified using the `ObjectReference` type (or other types representing strict subsets of it). Unlike partial URLs, the ObjectReference type facilitates flexible defaulting of fields from the referring object or other contextual information.
References in the status of the referee to the referrer may be permitted, when the references are one-to-one and do not need to be frequently updated, particularly in an edge-based manner.
#### Lists of named subobjects preferred over maps
Discussed in [#2004](http://issue.k8s.io/2004) and elsewhere. There are no maps of subobjects in any API objects. Instead, the convention is to use a list of subobjects containing name fields.
For example:
{% highlight yaml %}
{% raw %}
ports:
- name: www
containerPort: 80
{% endraw %}
{% endhighlight %}
vs.
{% highlight yaml %}
{% raw %}
ports:
www:
containerPort: 80
{% endraw %}
{% endhighlight %}
This rule maintains the invariant that all JSON/YAML keys are fields in API objects. The only exceptions are pure maps in the API (currently, labels, selectors, annotations, data), as opposed to sets of subobjects.
#### Constants
Some fields will have a list of allowed values (enumerations). These values will be strings, and they will be in CamelCase, with an initial uppercase letter. Examples: "ClusterFirst", "Pending", "ClientIP".
### Lists and Simple kinds
Every list or simple kind SHOULD have the following metadata in a nested object field called "metadata":
* resourceVersion: a string that identifies the common version of the objects returned by in a list. This value MUST be treated as opaque by clients and passed unmodified back to the server. A resource version is only valid within a single namespace on a single kind of resource.
Every simple kind returned by the server, and any simple kind sent to the server that must support idempotency or optimistic concurrency should return this value.Since simple resources are often used as input alternate actions that modify objects, the resource version of the simple resource should correspond to the resource version of the object.
## Differing Representations
An API may represent a single entity in different ways for different clients, or transform an object after certain transitions in the system occur. In these cases, one request object may have two representations available as different resources, or different kinds.
An example is a Service, which represents the intent of the user to group a set of pods with common behavior on common ports. When Kubernetes detects a pod matches the service selector, the IP address and port of the pod are added to an Endpoints resource for that Service. The Endpoints resource exists only if the Service exists, but exposes only the IPs and ports of the selected pods. The full service is represented by two distinct resources - under the original Service resource the user created, as well as in the Endpoints resource.
As another example, a "pod status" resource may accept a PUT with the "pod" kind, with different rules about what fields may be changed.
Future versions of Kubernetes may allow alternative encodings of objects beyond JSON.
## Verbs on Resources
API resources should use the traditional REST pattern:
* GET /&lt;resourceNamePlural&gt; - Retrieve a list of type &lt;resourceName&gt;, e.g. GET /pods returns a list of Pods.
* POST /&lt;resourceNamePlural&gt; - Create a new resource from the JSON object provided by the client.
* GET /&lt;resourceNamePlural&gt;/&lt;name&gt; - Retrieves a single resource with the given name, e.g. GET /pods/first returns a Pod named 'first'. Should be constant time, and the resource should be bounded in size.
* DELETE /&lt;resourceNamePlural&gt;/&lt;name&gt; - Delete the single resource with the given name. DeleteOptions may specify gracePeriodSeconds, the optional duration in seconds before the object should be deleted. Individual kinds may declare fields which provide a default grace period, and different kinds may have differing kind-wide default grace periods. A user provided grace period overrides a default grace period, including the zero grace period ("now").
* PUT /&lt;resourceNamePlural&gt;/&lt;name&gt; - Update or create the resource with the given name with the JSON object provided by the client.
* PATCH /&lt;resourceNamePlural&gt;/&lt;name&gt; - Selectively modify the specified fields of the resource. See more information [below](#patch).
* GET /&lt;resourceNamePlural&gt;&amp;watch=true - Receive a stream of JSON objects corresponding to changes made to any resource of the given kind over time.
### PATCH operations
The API supports three different PATCH operations, determined by their corresponding Content-Type header:
* JSON Patch, `Content-Type: application/json-patch+json`
* As defined in [RFC6902](https://tools.ietf.org/html/rfc6902), a JSON Patch is a sequence of operations that are executed on the resource, e.g. `{"op": "add", "path": "/a/b/c", "value": [ "foo", "bar" ]}`. For more details on how to use JSON Patch, see the RFC.
* Merge Patch, `Content-Type: application/merge-patch+json`
* As defined in [RFC7386](https://tools.ietf.org/html/rfc7386), a Merge Patch is essentially a partial representation of the resource. The submitted JSON is "merged" with the current resource to create a new one, then the new one is saved. For more details on how to use Merge Patch, see the RFC.
* Strategic Merge Patch, `Content-Type: application/strategic-merge-patch+json`
* Strategic Merge Patch is a custom implementation of Merge Patch. For a detailed explanation of how it works and why it needed to be introduced, see below.
#### Strategic Merge Patch
In the standard JSON merge patch, JSON objects are always merged but lists are always replaced. Often that isn't what we want. Let's say we start with the following Pod:
{% highlight yaml %}
{% raw %}
spec:
containers:
- name: nginx
image: nginx-1.0
{% endraw %}
{% endhighlight %}
...and we POST that to the server (as JSON). Then let's say we want to *add* a container to this Pod.
{% highlight yaml %}
{% raw %}
PATCH /api/v1/namespaces/default/pods/pod-name
spec:
containers:
- name: log-tailer
image: log-tailer-1.0
{% endraw %}
{% endhighlight %}
If we were to use standard Merge Patch, the entire container list would be replaced with the single log-tailer container. However, our intent is for the container lists to merge together based on the `name` field.
To solve this problem, Strategic Merge Patch uses metadata attached to the API objects to determine what lists should be merged and which ones should not. Currently the metadata is available as struct tags on the API objects themselves, but will become available to clients as Swagger annotations in the future. In the above example, the `patchStrategy` metadata for the `containers` field would be `merge` and the `patchMergeKey` would be `name`.
Note: If the patch results in merging two lists of scalars, the scalars are first deduplicated and then merged.
Strategic Merge Patch also supports special operations as listed below.
### List Operations
To override the container list to be strictly replaced, regardless of the default:
{% highlight yaml %}
{% raw %}
containers:
- name: nginx
image: nginx-1.0
- $patch: replace # any further $patch operations nested in this list will be ignored
{% endraw %}
{% endhighlight %}
To delete an element of a list that should be merged:
{% highlight yaml %}
{% raw %}
containers:
- name: nginx
image: nginx-1.0
- $patch: delete
name: log-tailer # merge key and value goes here
{% endraw %}
{% endhighlight %}
### Map Operations
To indicate that a map should not be merged and instead should be taken literally:
{% highlight yaml %}
{% raw %}
$patch: replace # recursive and applies to all fields of the map it's in
containers:
- name: nginx
image: nginx-1.0
{% endraw %}
{% endhighlight %}
To delete a field of a map:
{% highlight yaml %}
{% raw %}
name: nginx
image: nginx-1.0
labels:
live: null # set the value of the map key to null
{% endraw %}
{% endhighlight %}
## Idempotency
All compatible Kubernetes APIs MUST support "name idempotency" and respond with an HTTP status code 409 when a request is made to POST an object that has the same name as an existing object in the system. See [docs/user-guide/identifiers.md](../user-guide/identifiers.html) for details.
Names generated by the system may be requested using `metadata.generateName`. GenerateName indicates that the name should be made unique by the server prior to persisting it. A non-empty value for the field indicates the name will be made unique (and the name returned to the client will be different than the name passed). The value of this field will be combined with a unique suffix on the server if the Name field has not been provided. The provided value must be valid within the rules for Name, and may be truncated by the length of the suffix required to make the value unique on the server. If this field is specified, and Name is not present, the server will NOT return a 409 if the generated name exists - instead, it will either return 201 Created or 504 with Reason `ServerTimeout` indicating a unique name could not be found in the time allotted, and the client should retry (optionally after the time indicated in the Retry-After header).
## Defaulting
Default resource values are API version-specific, and they are applied during
the conversion from API-versioned declarative configuration to internal objects
representing the desired state (`Spec`) of the resource. Subsequent GETs of the
resource will include the default values explicitly.
Incorporating the default values into the `Spec` ensures that `Spec` depicts the
full desired state so that it is easier for the system to determine how to
achieve the state, and for the user to know what to anticipate.
API version-specific default values are set by the API server.
## Late Initialization
Late initialization is when resource fields are set by a system controller
after an object is created/updated.
For example, the scheduler sets the `pod.spec.nodeName` field after the pod is created.
Late-initializers should only make the following types of modifications:
- Setting previously unset fields
- Adding keys to maps
- Adding values to arrays which have mergeable semantics (`patchStrategy:"merge"` attribute in
the type definition).
These conventions:
1. allow a user (with sufficient privilege) to override any system-default behaviors by setting
the fields that would otherwise have been defaulted.
1. enables updates from users to be merged with changes made during late initialization, using
strategic merge patch, as opposed to clobbering the change.
1. allow the component which does the late-initialization to use strategic merge patch, which
facilitates composition and concurrency of such components.
Although the apiserver Admission Control stage acts prior to object creation,
Admission Control plugins should follow the Late Initialization conventions
too, to allow their implementation to be later moved to a 'controller', or to client libraries.
## Concurrency Control and Consistency
Kubernetes leverages the concept of *resource versions* to achieve optimistic concurrency. All Kubernetes resources have a "resourceVersion" field as part of their metadata. This resourceVersion is a string that identifies the internal version of an object that can be used by clients to determine when objects have changed. When a record is about to be updated, it's version is checked against a pre-saved value, and if it doesn't match, the update fails with a StatusConflict (HTTP status code 409).
The resourceVersion is changed by the server every time an object is modified. If resourceVersion is included with the PUT operation the system will verify that there have not been other successful mutations to the resource during a read/modify/write cycle, by verifying that the current value of resourceVersion matches the specified value.
The resourceVersion is currently backed by [etcd's modifiedIndex](https://coreos.com/docs/distributed-configuration/etcd-api/). However, it's important to note that the application should *not* rely on the implementation details of the versioning system maintained by Kubernetes. We may change the implementation of resourceVersion in the future, such as to change it to a timestamp or per-object counter.
The only way for a client to know the expected value of resourceVersion is to have received it from the server in response to a prior operation, typically a GET. This value MUST be treated as opaque by clients and passed unmodified back to the server. Clients should not assume that the resource version has meaning across namespaces, different kinds of resources, or different servers. Currently, the value of resourceVersion is set to match etcd's sequencer. You could think of it as a logical clock the API server can use to order requests. However, we expect the implementation of resourceVersion to change in the future, such as in the case we shard the state by kind and/or namespace, or port to another storage system.
In the case of a conflict, the correct client action at this point is to GET the resource again, apply the changes afresh, and try submitting again. This mechanism can be used to prevent races like the following:
```
{% raw %}
Client #1 Client #2
GET Foo GET Foo
Set Foo.Bar = "one" Set Foo.Baz = "two"
PUT Foo PUT Foo
{% endraw %}
```
When these sequences occur in parallel, either the change to Foo.Bar or the change to Foo.Baz can be lost.
On the other hand, when specifying the resourceVersion, one of the PUTs will fail, since whichever write succeeds changes the resourceVersion for Foo.
resourceVersion may be used as a precondition for other operations (e.g., GET, DELETE) in the future, such as for read-after-write consistency in the presence of caching.
"Watch" operations specify resourceVersion using a query parameter. It is used to specify the point at which to begin watching the specified resources. This may be used to ensure that no mutations are missed between a GET of a resource (or list of resources) and a subsequent Watch, even if the current version of the resource is more recent. This is currently the main reason that list operations (GET on a collection) return resourceVersion.
## Serialization Format
APIs may return alternative representations of any resource in response to an Accept header or under alternative endpoints, but the default serialization for input and output of API responses MUST be JSON.
All dates should be serialized as RFC3339 strings.
## Units
Units must either be explicit in the field name (e.g., `timeoutSeconds`), or must be specified as part of the value (e.g., `resource.Quantity`). Which approach is preferred is TBD, though currently we use the `fooSeconds` convention for durations.
## Selecting Fields
Some APIs may need to identify which field in a JSON object is invalid, or to reference a value to extract from a separate resource. The current recommendation is to use standard JavaScript syntax for accessing that field, assuming the JSON object was transformed into a JavaScript object, without the leading dot, such as `metadata.name`.
Examples:
* Find the field "current" in the object "state" in the second item in the array "fields": `fields[1].state.current`
## Object references
Object references should either be called `fooName` if referring to an object of kind `Foo` by just the name (within the current namespace, if a namespaced resource), or should be called `fooRef`, and should contain a subset of the fields of the `ObjectReference` type.
TODO: Plugins, extensions, nested kinds, headers
## HTTP Status codes
The server will respond with HTTP status codes that match the HTTP spec. See the section below for a breakdown of the types of status codes the server will send.
The following HTTP status codes may be returned by the API.
#### Success codes
* `200 StatusOK`
* Indicates that the request completed successfully.
* `201 StatusCreated`
* Indicates that the request to create kind completed successfully.
* `204 StatusNoContent`
* Indicates that the request completed successfully, and the response contains no body.
* Returned in response to HTTP OPTIONS requests.
#### Error codes
* `307 StatusTemporaryRedirect`
* Indicates that the address for the requested resource has changed.
* Suggested client recovery behavior
* Follow the redirect.
* `400 StatusBadRequest`
* Indicates the requested is invalid.
* Suggested client recovery behavior:
* Do not retry. Fix the request.
* `401 StatusUnauthorized`
* Indicates that the server can be reached and understood the request, but refuses to take any further action, because the client must provide authorization. If the client has provided authorization, the server is indicating the provided authorization is unsuitable or invalid.
* Suggested client recovery behavior
* If the user has not supplied authorization information, prompt them for the appropriate credentials
* If the user has supplied authorization information, inform them their credentials were rejected and optionally prompt them again.
* `403 StatusForbidden`
* Indicates that the server can be reached and understood the request, but refuses to take any further action, because it is configured to deny access for some reason to the requested resource by the client.
* Suggested client recovery behavior
* Do not retry. Fix the request.
* `404 StatusNotFound`
* Indicates that the requested resource does not exist.
* Suggested client recovery behavior
* Do not retry. Fix the request.
* `405 StatusMethodNotAllowed`
* Indicates that the action the client attempted to perform on the resource was not supported by the code.
* Suggested client recovery behavior
* Do not retry. Fix the request.
* `409 StatusConflict`
* Indicates that either the resource the client attempted to create already exists or the requested update operation cannot be completed due to a conflict.
* Suggested client recovery behavior
* * If creating a new resource
* * Either change the identifier and try again, or GET and compare the fields in the pre-existing object and issue a PUT/update to modify the existing object.
* * If updating an existing resource:
* See `Conflict` from the `status` response section below on how to retrieve more information about the nature of the conflict.
* GET and compare the fields in the pre-existing object, merge changes (if still valid according to preconditions), and retry with the updated request (including `ResourceVersion`).
* `422 StatusUnprocessableEntity`
* Indicates that the requested create or update operation cannot be completed due to invalid data provided as part of the request.
* Suggested client recovery behavior
* Do not retry. Fix the request.
* `429 StatusTooManyRequests`
* Indicates that the either the client rate limit has been exceeded or the server has received more requests then it can process.
* Suggested client recovery behavior:
* Read the `Retry-After` HTTP header from the response, and wait at least that long before retrying.
* `500 StatusInternalServerError`
* Indicates that the server can be reached and understood the request, but either an unexpected internal error occurred and the outcome of the call is unknown, or the server cannot complete the action in a reasonable time (this maybe due to temporary server load or a transient communication issue with another server).
* Suggested client recovery behavior:
* Retry with exponential backoff.
* `503 StatusServiceUnavailable`
* Indicates that required service is unavailable.
* Suggested client recovery behavior:
* Retry with exponential backoff.
* `504 StatusServerTimeout`
* Indicates that the request could not be completed within the given time. Clients can get this response ONLY when they specified a timeout param in the request.
* Suggested client recovery behavior:
* Increase the value of the timeout param and retry with exponential backoff
## Response Status Kind
Kubernetes will always return the `Status` kind from any API endpoint when an error occurs.
Clients SHOULD handle these types of objects when appropriate.
A `Status` kind will be returned by the API in two cases:
* When an operation is not successful (i.e. when the server would return a non 2xx HTTP status code).
* When a HTTP `DELETE` call is successful.
The status object is encoded as JSON and provided as the body of the response. The status object contains fields for humans and machine consumers of the API to get more detailed information for the cause of the failure. The information in the status object supplements, but does not override, the HTTP status code's meaning. When fields in the status object have the same meaning as generally defined HTTP headers and that header is returned with the response, the header should be considered as having higher priority.
**Example:**
{% highlight console %}
{% raw %}
$ curl -v -k -H "Authorization: Bearer WhCDvq4VPpYhrcfmF6ei7V9qlbqTubUc" https://10.240.122.184:443/api/v1/namespaces/default/pods/grafana
> GET /api/v1/namespaces/default/pods/grafana HTTP/1.1
> User-Agent: curl/7.26.0
> Host: 10.240.122.184
> Accept: */*
> Authorization: Bearer WhCDvq4VPpYhrcfmF6ei7V9qlbqTubUc
>
< HTTP/1.1 404 Not Found
< Content-Type: application/json
< Date: Wed, 20 May 2015 18:10:42 GMT
< Content-Length: 232
<
{
"kind": "Status",
"apiVersion": "v1",
"metadata": {},
"status": "Failure",
"message": "pods \"grafana\" not found",
"reason": "NotFound",
"details": {
"name": "grafana",
"kind": "pods"
},
"code": 404
}
{% endraw %}
{% endhighlight %}
`status` field contains one of two possible values:
* `Success`
* `Failure`
`message` may contain human-readable description of the error
`reason` may contain a machine-readable, one-word, CamelCase description of why this operation is in the `Failure` status. If this value is empty there is no information available. The `reason` clarifies an HTTP status code but does not override it.
`details` may contain extended data associated with the reason. Each reason may define its own extended details. This field is optional and the data returned is not guaranteed to conform to any schema except that defined by the reason type.
Possible values for the `reason` and `details` fields:
* `BadRequest`
* Indicates that the request itself was invalid, because the request doesn't make any sense, for example deleting a read-only object.
* This is different than `status reason` `Invalid` above which indicates that the API call could possibly succeed, but the data was invalid.
* API calls that return BadRequest can never succeed.
* Http status code: `400 StatusBadRequest`
* `Unauthorized`
* Indicates that the server can be reached and understood the request, but refuses to take any further action without the client providing appropriate authorization. If the client has provided authorization, this error indicates the provided credentials are insufficient or invalid.
* Details (optional):
* `kind string`
* The kind attribute of the unauthorized resource (on some operations may differ from the requested resource).
* `name string`
* The identifier of the unauthorized resource.
* HTTP status code: `401 StatusUnauthorized`
* `Forbidden`
* Indicates that the server can be reached and understood the request, but refuses to take any further action, because it is configured to deny access for some reason to the requested resource by the client.
* Details (optional):
* `kind string`
* The kind attribute of the forbidden resource (on some operations may differ from the requested resource).
* `name string`
* The identifier of the forbidden resource.
* HTTP status code: `403 StatusForbidden`
* `NotFound`
* Indicates that one or more resources required for this operation could not be found.
* Details (optional):
* `kind string`
* The kind attribute of the missing resource (on some operations may differ from the requested resource).
* `name string`
* The identifier of the missing resource.
* HTTP status code: `404 StatusNotFound`
* `AlreadyExists`
* Indicates that the resource you are creating already exists.
* Details (optional):
* `kind string`
* The kind attribute of the conflicting resource.
* `name string`
* The identifier of the conflicting resource.
* HTTP status code: `409 StatusConflict`
* `Conflict`
* Indicates that the requested update operation cannot be completed due to a conflict. The client may need to alter the request. Each resource may define custom details that indicate the nature of the conflict.
* HTTP status code: `409 StatusConflict`
* `Invalid`
* Indicates that the requested create or update operation cannot be completed due to invalid data provided as part of the request.
* Details (optional):
* `kind string`
* the kind attribute of the invalid resource
* `name string`
* the identifier of the invalid resource
* `causes`
* One or more `StatusCause` entries indicating the data in the provided resource that was invalid. The `reason`, `message`, and `field` attributes will be set.
* HTTP status code: `422 StatusUnprocessableEntity`
* `Timeout`
* Indicates that the request could not be completed within the given time. Clients may receive this response if the server has decided to rate limit the client, or if the server is overloaded and cannot process the request at this time.
* Http status code: `429 TooManyRequests`
* The server should set the `Retry-After` HTTP header and return `retryAfterSeconds` in the details field of the object. A value of `0` is the default.
* `ServerTimeout`
* Indicates that the server can be reached and understood the request, but cannot complete the action in a reasonable time. This maybe due to temporary server load or a transient communication issue with another server.
* Details (optional):
* `kind string`
* The kind attribute of the resource being acted on.
* `name string`
* The operation that is being attempted.
* The server should set the `Retry-After` HTTP header and return `retryAfterSeconds` in the details field of the object. A value of `0` is the default.
* Http status code: `504 StatusServerTimeout`
* `MethodNotAllowed`
* Indicates that the action the client attempted to perform on the resource was not supported by the code.
* For instance, attempting to delete a resource that can only be created.
* API calls that return MethodNotAllowed can never succeed.
* Http status code: `405 StatusMethodNotAllowed`
* `InternalError`
* Indicates that an internal error occurred, it is unexpected and the outcome of the call is unknown.
* Details (optional):
* `causes`
* The original error.
* Http status code: `500 StatusInternalServerError`
`code` may contain the suggested HTTP return code for this status.
## Events
Events are complementary to status information, since they can provide some historical information about status and occurrences in addition to current or previous status. Generate events for situations users or administrators should be alerted about.
Choose a unique, specific, short, CamelCase reason for each event category. For example, `FreeDiskSpaceInvalid` is a good event reason because it is likely to refer to just one situation, but `Started` is not a good reason because it doesn't sufficiently indicate what started, even when combined with other event fields.
`Error creating foo` or `Error creating foo %s` would be appropriate for an event message, with the latter being preferable, since it is more informational.
Accumulate repeated events in the client, especially for frequent events, to reduce data volume, load on the system, and noise exposed to users.
## Naming conventions
* Go field names must be CamelCase. JSON field names must be camelCase. Other than capitalization of the initial letter, the two should almost always match. No underscores nor dashes in either.
* Field and resource names should be declarative, not imperative (DoSomething, SomethingDoer, DoneBy, DoneAt).
* `Minion` has been deprecated in favor of `Node`. Use `Node` where referring to the node resource in the context of the cluster. Use `Host` where referring to properties of the individual physical/virtual system, such as `hostname`, `hostPath`, `hostNetwork`, etc.
* `FooController` is a deprecated kind naming convention. Name the kind after the thing being controlled instead (e.g., `Job` rather than `JobController`).
* The name of a field that specifies the time at which `something` occurs should be called `somethingTime`. Do not use `stamp` (e.g., `creationTimestamp`).
* We use the `fooSeconds` convention for durations, as discussed in the [units subsection](#units).
* `fooPeriodSeconds` is preferred for periodic intervals and other waiting periods (e.g., over `fooIntervalSeconds`).
* `fooTimeoutSeconds` is preferred for inactivity/unresponsiveness deadlines.
* `fooDeadlineSeconds` is preferred for activity completion deadlines.
* Do not use abbreviations in the API, except where they are extremely commonly used, such as "id", "args", or "stdin".
* Acronyms should similarly only be used when extremely commonly known. All letters in the acronym should have the same case, using the appropriate case for the situation. For example, at the beginning of a field name, the acronym should be all lowercase, such as "httpGet". Where used as a constant, all letters should be uppercase, such as "TCP" or "UDP".
* The name of a field referring to another resource of kind `Foo` by name should be called `fooName`. The name of a field referring to another resource of kind `Foo` by ObjectReference (or subset thereof) should be called `fooRef`.
* More generally, include the units and/or type in the field name if they could be ambiguous and they are not specified by the value or value type.
## Label, selector, and annotation conventions
Labels are the domain of users. They are intended to facilitate organization and management of API resources using attributes that are meaningful to users, as opposed to meaningful to the system. Think of them as user-created mp3 or email inbox labels, as opposed to the directory structure used by a program to store its data. The former is enables the user to apply an arbitrary ontology, whereas the latter is implementation-centric and inflexible. Users will use labels to select resources to operate on, display label values in CLI/UI columns, etc. Users should always retain full power and flexibility over the label schemas they apply to labels in their namespaces.
However, we should support conveniences for common cases by default. For example, what we now do in ReplicationController is automatically set the RC's selector and labels to the labels in the pod template by default, if they are not already set. That ensures that the selector will match the template, and that the RC can be managed using the same labels as the pods it creates. Note that once we generalize selectors, it won't necessarily be possible to unambiguously generate labels that match an arbitrary selector.
If the user wants to apply additional labels to the pods that it doesn't select upon, such as to facilitate adoption of pods or in the expectation that some label values will change, they can set the selector to a subset of the pod labels. Similarly, the RC's labels could be initialized to a subset of the pod template's labels, or could include additional/different labels.
For disciplined users managing resources within their own namespaces, it's not that hard to consistently apply schemas that ensure uniqueness. One just needs to ensure that at least one value of some label key in common differs compared to all other comparable resources. We could/should provide a verification tool to check that. However, development of conventions similar to the examples in [Labels](../user-guide/labels.html) make uniqueness straightforward. Furthermore, relatively narrowly used namespaces (e.g., per environment, per application) can be used to reduce the set of resources that could potentially cause overlap.
In cases where users could be running misc. examples with inconsistent schemas, or where tooling or components need to programmatically generate new objects to be selected, there needs to be a straightforward way to generate unique label sets. A simple way to ensure uniqueness of the set is to ensure uniqueness of a single label value, such as by using a resource name, uid, resource hash, or generation number.
Problems with uids and hashes, however, include that they have no semantic meaning to the user, are not memorable nor readily recognizable, and are not predictable. Lack of predictability obstructs use cases such as creation of a replication controller from a pod, such as people want to do when exploring the system, bootstrapping a self-hosted cluster, or deletion and re-creation of a new RC that adopts the pods of the previous one, such as to rename it. Generation numbers are more predictable and much clearer, assuming there is a logical sequence. Fortunately, for deployments that's the case. For jobs, use of creation timestamps is common internally. Users should always be able to turn off auto-generation, in order to permit some of the scenarios described above. Note that auto-generated labels will also become one more field that needs to be stripped out when cloning a resource, within a namespace, in a new namespace, in a new cluster, etc., and will need to be ignored around when updating a resource via patch or read-modify-write sequence.
Inclusion of a system prefix in a label key is fairly hostile to UX. A prefix is only necessary in the case that the user cannot choose the label key, in order to avoid collisions with user-defined labels. However, I firmly believe that the user should always be allowed to select the label keys to use on their resources, so it should always be possible to override default label keys.
Therefore, resources supporting auto-generation of unique labels should have a `uniqueLabelKey` field, so that the user could specify the key if they wanted to, but if unspecified, it could be set by default, such as to the resource type, like job, deployment, or replicationController. The value would need to be at least spatially unique, and perhaps temporally unique in the case of job.
Annotations have very different intended usage from labels. We expect them to be primarily generated and consumed by tooling and system extensions. I'm inclined to generalize annotations to permit them to directly store arbitrary json. Rigid names and name prefixes make sense, since they are analogous to API fields.
In fact, in-development API fields, including those used to represent fields of newer alpha/beta API versions in the older stable storage version, may be represented as annotations with the form `something.alpha.kubernetes.io/name` or `something.beta.kubernetes.io/name` (depending on our confidence in it). For example `net.alpha.kubernetes.io/policy` might represent an experimental network policy field.
Other advice regarding use of labels, annotations, and other generic map keys by Kubernetes components and tools:
- Key names should be all lowercase, with words separated by dashes, such as `desired-replicas`
- Prefix the key with `kubernetes.io/` or `foo.kubernetes.io/`, preferably the latter if the label/annotation is specific to `foo`
- For instance, prefer `service-account.kubernetes.io/name` over `kubernetes.io/service-account.name`
- Use annotations to store API extensions that the controller responsible for the resource doesn't need to know about, experimental fields that aren't intended to be generally used API fields, etc. Beware that annotations aren't automatically handled by the API conversion machinery.
<!-- BEGIN MUNGE: IS_VERSIONED -->
<!-- TAG IS_VERSIONED -->
<!-- END MUNGE: IS_VERSIONED -->
<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/devel/api-conventions.md?pixel)]()
<!-- END MUNGE: GENERATED_ANALYTICS -->
v1.1/docs/devel/api_changes.md
+628
View File
@@ -0,0 +1,628 @@
---
layout: docwithnav
title: "So you want to change the API?"
---
<!-- BEGIN MUNGE: UNVERSIONED_WARNING -->
<!-- END MUNGE: UNVERSIONED_WARNING -->
# So you want to change the API?
Before attempting a change to the API, you should familiarize yourself
with a number of existing API types and with the [API
conventions](api-conventions.html). If creating a new API
type/resource, we also recommend that you first send a PR containing
just a proposal for the new API types, and that you initially target
the extensions API (pkg/apis/extensions).
The Kubernetes API has two major components - the internal structures and
the versioned APIs. The versioned APIs are intended to be stable, while the
internal structures are implemented to best reflect the needs of the Kubernetes
code itself.
What this means for API changes is that you have to be somewhat thoughtful in
how you approach changes, and that you have to touch a number of pieces to make
a complete change. This document aims to guide you through the process, though
not all API changes will need all of these steps.
## Operational overview
It is important to have a high level understanding of the API system used in
Kubernetes in order to navigate the rest of this document.
As mentioned above, the internal representation of an API object is decoupled
from any one API version. This provides a lot of freedom to evolve the code,
but it requires robust infrastructure to convert between representations. There
are multiple steps in processing an API operation - even something as simple as
a GET involves a great deal of machinery.
The conversion process is logically a "star" with the internal form at the
center. Every versioned API can be converted to the internal form (and
vice-versa), but versioned APIs do not convert to other versioned APIs directly.
This sounds like a heavy process, but in reality we do not intend to keep more
than a small number of versions alive at once. While all of the Kubernetes code
operates on the internal structures, they are always converted to a versioned
form before being written to storage (disk or etcd) or being sent over a wire.
Clients should consume and operate on the versioned APIs exclusively.
To demonstrate the general process, here is a (hypothetical) example:
1. A user POSTs a `Pod` object to `/api/v7beta1/...`
2. The JSON is unmarshalled into a `v7beta1.Pod` structure
3. Default values are applied to the `v7beta1.Pod`
4. The `v7beta1.Pod` is converted to an `api.Pod` structure
5. The `api.Pod` is validated, and any errors are returned to the user
6. The `api.Pod` is converted to a `v6.Pod` (because v6 is the latest stable
version)
7. The `v6.Pod` is marshalled into JSON and written to etcd
Now that we have the `Pod` object stored, a user can GET that object in any
supported api version. For example:
1. A user GETs the `Pod` from `/api/v5/...`
2. The JSON is read from etcd and unmarshalled into a `v6.Pod` structure
3. Default values are applied to the `v6.Pod`
4. The `v6.Pod` is converted to an `api.Pod` structure
5. The `api.Pod` is converted to a `v5.Pod` structure
6. The `v5.Pod` is marshalled into JSON and sent to the user
The implication of this process is that API changes must be done carefully and
backward-compatibly.
## On compatibility
Before talking about how to make API changes, it is worthwhile to clarify what
we mean by API compatibility. An API change is considered backward-compatible
if it:
* adds new functionality that is not required for correct behavior (e.g.,
does not add a new required field)
* does not change existing semantics, including:
* default values and behavior
* interpretation of existing API types, fields, and values
* which fields are required and which are not
Put another way:
1. Any API call (e.g. a structure POSTed to a REST endpoint) that worked before
your change must work the same after your change.
2. Any API call that uses your change must not cause problems (e.g. crash or
degrade behavior) when issued against servers that do not include your change.
3. It must be possible to round-trip your change (convert to different API
versions and back) with no loss of information.
4. Existing clients need not be aware of your change in order for them to continue
to function as they did previously, even when your change is utilized
If your change does not meet these criteria, it is not considered strictly
compatible.
Let's consider some examples. In a hypothetical API (assume we're at version
v6), the `Frobber` struct looks something like this:
{% highlight go %}
{% raw %}
// API v6.
type Frobber struct {
Height int `json:"height"`
Param string `json:"param"`
}
{% endraw %}
{% endhighlight %}
You want to add a new `Width` field. It is generally safe to add new fields
without changing the API version, so you can simply change it to:
{% highlight go %}
{% raw %}
// Still API v6.
type Frobber struct {
Height int `json:"height"`
Width int `json:"width"`
Param string `json:"param"`
}
{% endraw %}
{% endhighlight %}
The onus is on you to define a sane default value for `Width` such that rule #1
above is true - API calls and stored objects that used to work must continue to
work.
For your next change you want to allow multiple `Param` values. You can not
simply change `Param string` to `Params []string` (without creating a whole new
API version) - that fails rules #1 and #2. You can instead do something like:
{% highlight go %}
{% raw %}
// Still API v6, but kind of clumsy.
type Frobber struct {
Height int `json:"height"`
Width int `json:"width"`
Param string `json:"param"` // the first param
ExtraParams []string `json:"params"` // additional params
}
{% endraw %}
{% endhighlight %}
Now you can satisfy the rules: API calls that provide the old style `Param`
will still work, while servers that don't understand `ExtraParams` can ignore
it. This is somewhat unsatisfying as an API, but it is strictly compatible.
Part of the reason for versioning APIs and for using internal structs that are
distinct from any one version is to handle growth like this. The internal
representation can be implemented as:
{% highlight go %}
{% raw %}
// Internal, soon to be v7beta1.
type Frobber struct {
Height int
Width int
Params []string
}
{% endraw %}
{% endhighlight %}
The code that converts to/from versioned APIs can decode this into the somewhat
uglier (but compatible!) structures. Eventually, a new API version, let's call
it v7beta1, will be forked and it can use the clean internal structure.
We've seen how to satisfy rules #1 and #2. Rule #3 means that you can not
extend one versioned API without also extending the others. For example, an
API call might POST an object in API v7beta1 format, which uses the cleaner
`Params` field, but the API server might store that object in trusty old v6
form (since v7beta1 is "beta"). When the user reads the object back in the
v7beta1 API it would be unacceptable to have lost all but `Params[0]`. This
means that, even though it is ugly, a compatible change must be made to the v6
API.
However, this is very challenging to do correctly. It often requires
multiple representations of the same information in the same API resource, which
need to be kept in sync in the event that either is changed. For example,
let's say you decide to rename a field within the same API version. In this case,
you add units to `height` and `width`. You implement this by adding duplicate
fields:
{% highlight go %}
{% raw %}
type Frobber struct {
Height *int `json:"height"`
Width *int `json:"width"`
HeightInInches *int `json:"heightInInches"`
WidthInInches *int `json:"widthInInches"`
}
{% endraw %}
{% endhighlight %}
You convert all of the fields to pointers in order to distinguish between unset and
set to 0, and then set each corresponding field from the other in the defaulting
pass (e.g., `heightInInches` from `height`, and vice versa), which runs just prior
to conversion. That works fine when the user creates a resource from a hand-written
configuration -- clients can write either field and read either field, but what about
creation or update from the output of GET, or update via PATCH (see
[In-place updates](../user-guide/managing-deployments.html#in-place-updates-of-resources))?
In this case, the two fields will conflict, because only one field would be updated
in the case of an old client that was only aware of the old field (e.g., `height`).
Say the client creates:
{% highlight json %}
{% raw %}
{
"height": 10,
"width": 5
}
{% endraw %}
{% endhighlight %}
and GETs:
{% highlight json %}
{% raw %}
{
"height": 10,
"heightInInches": 10,
"width": 5,
"widthInInches": 5
}
{% endraw %}
{% endhighlight %}
then PUTs back:
{% highlight json %}
{% raw %}
{
"height": 13,
"heightInInches": 10,
"width": 5,
"widthInInches": 5
}
{% endraw %}
{% endhighlight %}
The update should not fail, because it would have worked before `heightInInches` was added.
Therefore, when there are duplicate fields, the old field MUST take precedence
over the new, and the new field should be set to match by the server upon write.
A new client would be aware of the old field as well as the new, and so can ensure
that the old field is either unset or is set consistently with the new field. However,
older clients would be unaware of the new field. Please avoid introducing duplicate
fields due to the complexity they incur in the API.
A new representation, even in a new API version, that is more expressive than an old one
breaks backward compatibility, since clients that only understood the old representation
would not be aware of the new representation nor its semantics. Examples of
proposals that have run into this challenge include [generalized label
selectors](http://issues.k8s.io/341) and [pod-level security
context](http://prs.k8s.io/12823).
As another interesting example, enumerated values cause similar challenges.
Adding a new value to an enumerated set is *not* a compatible change. Clients
which assume they know how to handle all possible values of a given field will
not be able to handle the new values. However, removing value from an
enumerated set *can* be a compatible change, if handled properly (treat the
removed value as deprecated but allowed). This is actually a special case of
a new representation, discussed above.
## Incompatible API changes
There are times when this might be OK, but mostly we want changes that
meet this definition. If you think you need to break compatibility,
you should talk to the Kubernetes team first.
Breaking compatibility of a beta or stable API version, such as v1, is unacceptable.
Compatibility for experimental or alpha APIs is not strictly required, but
breaking compatibility should not be done lightly, as it disrupts all users of the
feature. Experimental APIs may be removed. Alpha and beta API versions may be deprecated
and eventually removed wholesale, as described in the [versioning document](../design/versioning.html).
Document incompatible changes across API versions under the [conversion tips](../api.html).
If your change is going to be backward incompatible or might be a breaking change for API
consumers, please send an announcement to `kubernetes-dev@googlegroups.com` before
the change gets in. If you are unsure, ask. Also make sure that the change gets documented in
the release notes for the next release by labeling the PR with the "release-note" github label.
If you found that your change accidentally broke clients, it should be reverted.
In short, the expected API evolution is as follows:
* `extensions/v1alpha1` ->
* `newapigroup/v1alpha1` -> ... -> `newapigroup/v1alphaN` ->
* `newapigroup/v1beta1` -> ... -> `newapigroup/v1betaN` ->
* `newapigroup/v1` ->
* `newapigroup/v2alpha1` -> ...
While in extensions we have no obligation to move forward with the API at all and may delete or break it at any time.
While in alpha we expect to move forward with it, but may break it.
Once in beta we will preserve forward compatibility, but may introduce new versions and delete old ones.
v1 must be backward-compatible for an extended length of time.
## Changing versioned APIs
For most changes, you will probably find it easiest to change the versioned
APIs first. This forces you to think about how to make your change in a
compatible way. Rather than doing each step in every version, it's usually
easier to do each versioned API one at a time, or to do all of one version
before starting "all the rest".
### Edit types.go
The struct definitions for each API are in `pkg/api/<version>/types.go`. Edit
those files to reflect the change you want to make. Note that all types and non-inline
fields in versioned APIs must be preceded by descriptive comments - these are used to generate
documentation.
Optional fields should have the `,omitempty` json tag; fields are interpreted as being
required otherwise.
### Edit defaults.go
If your change includes new fields for which you will need default values, you
need to add cases to `pkg/api/<version>/defaults.go`. Of course, since you
have added code, you have to add a test: `pkg/api/<version>/defaults_test.go`.
Do use pointers to scalars when you need to distinguish between an unset value
and an automatic zero value. For example,
`PodSpec.TerminationGracePeriodSeconds` is defined as `*int64` the go type
definition. A zero value means 0 seconds, and a nil value asks the system to
pick a default.
Don't forget to run the tests!
### Edit conversion.go
Given that you have not yet changed the internal structs, this might feel
premature, and that's because it is. You don't yet have anything to convert to
or from. We will revisit this in the "internal" section. If you're doing this
all in a different order (i.e. you started with the internal structs), then you
should jump to that topic below. In the very rare case that you are making an
incompatible change you might or might not want to do this now, but you will
have to do more later. The files you want are
`pkg/api/<version>/conversion.go` and `pkg/api/<version>/conversion_test.go`.
Note that the conversion machinery doesn't generically handle conversion of values,
such as various kinds of field references and API constants. [The client
library](https://releases.k8s.io/release-1.1/pkg/client/unversioned/request.go) has custom conversion code for
field references. You also need to add a call to api.Scheme.AddFieldLabelConversionFunc
with a mapping function that understands supported translations.
## Changing the internal structures
Now it is time to change the internal structs so your versioned changes can be
used.
### Edit types.go
Similar to the versioned APIs, the definitions for the internal structs are in
`pkg/api/types.go`. Edit those files to reflect the change you want to make.
Keep in mind that the internal structs must be able to express *all* of the
versioned APIs.
## Edit validation.go
Most changes made to the internal structs need some form of input validation.
Validation is currently done on internal objects in
`pkg/api/validation/validation.go`. This validation is the one of the first
opportunities we have to make a great user experience - good error messages and
thorough validation help ensure that users are giving you what you expect and,
when they don't, that they know why and how to fix it. Think hard about the
contents of `string` fields, the bounds of `int` fields and the
requiredness/optionalness of fields.
Of course, code needs tests - `pkg/api/validation/validation_test.go`.
## Edit version conversions
At this point you have both the versioned API changes and the internal
structure changes done. If there are any notable differences - field names,
types, structural change in particular - you must add some logic to convert
versioned APIs to and from the internal representation. If you see errors from
the `serialization_test`, it may indicate the need for explicit conversions.
Performance of conversions very heavily influence performance of apiserver.
Thus, we are auto-generating conversion functions that are much more efficient
than the generic ones (which are based on reflections and thus are highly
inefficient).
The conversion code resides with each versioned API. There are two files:
- `pkg/api/<version>/conversion.go` containing manually written conversion
functions
- `pkg/api/<version>/conversion_generated.go` containing auto-generated
conversion functions
- `pkg/apis/extensions/<version>/conversion.go` containing manually written
conversion functions
- `pkg/apis/extensions/<version>/conversion_generated.go` containing
auto-generated conversion functions
Since auto-generated conversion functions are using manually written ones,
those manually written should be named with a defined convention, i.e. a function
converting type X in pkg a to type Y in pkg b, should be named:
`convert_a_X_To_b_Y`.
Also note that you can (and for efficiency reasons should) use auto-generated
conversion functions when writing your conversion functions.
Once all the necessary manually written conversions are added, you need to
regenerate auto-generated ones. To regenerate them:
- run
{% highlight sh %}
{% raw %}
hack/update-generated-conversions.sh
{% endraw %}
{% endhighlight %}
If running the above script is impossible due to compile errors, the easiest
workaround is to comment out the code causing errors and let the script to
regenerate it. If the auto-generated conversion methods are not used by the
manually-written ones, it's fine to just remove the whole file and let the
generator to create it from scratch.
Unsurprisingly, adding manually written conversion also requires you to add tests to
`pkg/api/<version>/conversion_test.go`.
## Edit deep copy files
At this point you have both the versioned API changes and the internal
structure changes done. You now need to generate code to handle deep copy
of your versioned api objects.
The deep copy code resides with each versioned API:
- `pkg/api/<version>/deep_copy_generated.go` containing auto-generated copy functions
- `pkg/apis/extensions/<version>/deep_copy_generated.go` containing auto-generated copy functions
To regenerate them:
- run
{% highlight sh %}
{% raw %}
hack/update-generated-deep-copies.sh
{% endraw %}
{% endhighlight %}
## Edit json (un)marshaling code
We are auto-generating code for marshaling and unmarshaling json representation
of api objects - this is to improve the overall system performance.
The auto-generated code resides with each versioned API:
- `pkg/api/<version>/types.generated.go`
- `pkg/apis/extensions/<version>/types.generated.go`
To regenerate them:
- run
{% highlight sh %}
{% raw %}
hack/update-codecgen.sh
{% endraw %}
{% endhighlight %}
## Making a new API Group
This section is under construction, as we make the tooling completely generic.
At the moment, you'll have to make a new directory under pkg/apis/; copy the
directory structure from pkg/apis/extensions. Add the new group/version to all
of the hack/{verify,update}-generated-{deep-copy,conversions,swagger}.sh files
in the appropriate places--it should just require adding your new group/version
to a bash array. You will also need to make sure your new types are imported by
the generation commands (cmd/gendeepcopy/ & cmd/genconversion). These
instructions may not be complete and will be updated as we gain experience.
Adding API groups outside of the pkg/apis/ directory is not currently supported,
but is clearly desirable. The deep copy & conversion generators need to work by
parsing go files instead of by reflection; then they will be easy to point at
arbitrary directories: see issue [#13775](http://issue.k8s.io/13775).
## Update the fuzzer
Part of our testing regimen for APIs is to "fuzz" (fill with random values) API
objects and then convert them to and from the different API versions. This is
a great way of exposing places where you lost information or made bad
assumptions. If you have added any fields which need very careful formatting
(the test does not run validation) or if you have made assumptions such as
"this slice will always have at least 1 element", you may get an error or even
a panic from the `serialization_test`. If so, look at the diff it produces (or
the backtrace in case of a panic) and figure out what you forgot. Encode that
into the fuzzer's custom fuzz functions. Hint: if you added defaults for a field,
that field will need to have a custom fuzz function that ensures that the field is
fuzzed to a non-empty value.
The fuzzer can be found in `pkg/api/testing/fuzzer.go`.
## Update the semantic comparisons
VERY VERY rarely is this needed, but when it hits, it hurts. In some rare
cases we end up with objects (e.g. resource quantities) that have morally
equivalent values with different bitwise representations (e.g. value 10 with a
base-2 formatter is the same as value 0 with a base-10 formatter). The only way
Go knows how to do deep-equality is through field-by-field bitwise comparisons.
This is a problem for us.
The first thing you should do is try not to do that. If you really can't avoid
this, I'd like to introduce you to our semantic DeepEqual routine. It supports
custom overrides for specific types - you can find that in `pkg/api/helpers.go`.
There's one other time when you might have to touch this: unexported fields.
You see, while Go's `reflect` package is allowed to touch unexported fields, us
mere mortals are not - this includes semantic DeepEqual. Fortunately, most of
our API objects are "dumb structs" all the way down - all fields are exported
(start with a capital letter) and there are no unexported fields. But sometimes
you want to include an object in our API that does have unexported fields
somewhere in it (for example, `time.Time` has unexported fields). If this hits
you, you may have to touch the semantic DeepEqual customization functions.
## Implement your change
Now you have the API all changed - go implement whatever it is that you're
doing!
## Write end-to-end tests
Check out the [E2E docs](e2e-tests.html) for detailed information about how to write end-to-end
tests for your feature.
## Examples and docs
At last, your change is done, all unit tests pass, e2e passes, you're done,
right? Actually, no. You just changed the API. If you are touching an
existing facet of the API, you have to try *really* hard to make sure that
*all* the examples and docs are updated. There's no easy way to do this, due
in part to JSON and YAML silently dropping unknown fields. You're clever -
you'll figure it out. Put `grep` or `ack` to good use.
If you added functionality, you should consider documenting it and/or writing
an example to illustrate your change.
Make sure you update the swagger API spec by running:
{% highlight sh %}
{% raw %}
hack/update-swagger-spec.sh
{% endraw %}
{% endhighlight %}
The API spec changes should be in a commit separate from your other changes.
## Adding new REST objects
TODO(smarterclayton): write this.
## Alpha, Beta, and Stable Versions
New feature development proceeds through a series of stages of increasing maturity:
- Development level
- Object Versioning: no convention
- Availability: not commited to main kubernetes repo, and thus not available in offical releases
- Audience: other developers closely collaborating on a feature or proof-of-concept
- Upgradeability, Reliability, Completeness, and Support: no requirements or guarantees
- Alpha level
- Object Versioning: API version name contains `alpha` (e.g. `v1alpha1`)
- Availability: committed to main kubernetes repo; appears in an official release; feature is
disabled by default, but may be enabled by flag
- Audience: developers and expert users interested in giving early feedback on features
- Completeness: some API operations, CLI commands, or UI support may not be implemented; the API
need not have had an *API review* (an intensive and targeted review of the API, on top of a normal
code review)
- Upgradeability: the object schema and semantics may change in a later software release, without
any provision for preserving objects in an existing cluster;
removing the upgradability concern allows developers to make rapid progress; in particular,
API versions can increment faster than the minor release cadence and the developer need not
maintain multiple versions; developers should still increment the API version when object schema
or semantics change in an [incompatible way](#on-compatibility)
- Cluster Reliability: because the feature is relatively new, and may lack complete end-to-end
tests, enabling the feature via a flag might expose bugs with destabilize the cluster (e.g. a
bug in a control loop might rapidly create excessive numbers of object, exhausting API storage).
- Support: there is *no commitment* from the project to complete the feature; the feature may be
dropped entirely in a later software release
- Recommended Use Cases: only in short-lived testing clusters, due to complexity of upgradeability
and lack of long-term support and lack of upgradability.
- Beta level:
- Object Versioning: API version name contains `beta` (e.g. `v2beta3`)
- Availability: in official Kubernetes releases, and enabled by default
- Audience: users interested in providing feedback on features
- Completeness: all API operations, CLI commands, and UI support should be implemented; end-to-end
tests complete; the API has had a thorough API review and is thought to be complete, though use
during beta may frequently turn up API issues not thought of during review
- Upgradeability: the object schema and semantics may change in a later software release; when
this happens, an upgrade path will be documentedr; in some cases, objects will be automatically
converted to the new version; in other cases, a manual upgrade may be necessary; a manual
upgrade may require downtime for anything relying on the new feature, and may require
manual conversion of objects to the new version; when manual conversion is necessary, the
project will provide documentation on the process (for an example, see [v1 conversion
tips](../api.html))
- Cluster Reliability: since the feature has e2e tests, enabling the feature via a flag should not
create new bugs in unrelated features; because the feature is new, it may have minor bugs
- Support: the project commits to complete the feature, in some form, in a subsequent Stable
version; typically this will happen within 3 months, but sometimes longer; releases should
simultaneously support two consecutive versions (e.g. `v1beta1` and `v1beta2`; or `v1beta2` and
`v1`) for at least one minor release cycle (typically 3 months) so that users have enough time
to upgrade and migrate objects
- Recommended Use Cases: in short-lived testing clusters; in production clusters as part of a
short-lived evaluation of the feature in order to provide feedback
- Stable level:
- Object Versioning: API version `vX` where `X` is an integer (e.g. `v1`)
- Availability: in official Kubernetes releases, and enabled by default
- Audience: all users
- Completeness: same as beta
- Upgradeability: only [strictly compatible](#on-compatibility) changes allowed in subsequent
software releases
- Cluster Reliability: high
- Support: API version will continue to be present for many subsequent software releases;
- Recommended Use Cases: any
<!-- BEGIN MUNGE: IS_VERSIONED -->
<!-- TAG IS_VERSIONED -->
<!-- END MUNGE: IS_VERSIONED -->
<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/devel/api_changes.md?pixel)]()
<!-- END MUNGE: GENERATED_ANALYTICS -->
v1.1/docs/devel/automation.md
+124
View File
@@ -0,0 +1,124 @@
---
layout: docwithnav
title: "Kubernetes Development Automation"
---
<!-- BEGIN MUNGE: UNVERSIONED_WARNING -->
<!-- END MUNGE: UNVERSIONED_WARNING -->
# Kubernetes Development Automation
## Overview
Kubernetes uses a variety of automated tools in an attempt to relieve developers of repeptitive, low
brain power work. This document attempts to describe these processes.
## Submit Queue
In an effort to
* reduce load on core developers
* maintain e2e stability
* load test githubs label feature
We have added an automated [submit-queue](https://github.com/kubernetes/contrib/tree/master/submit-queue)
for kubernetes.
The submit-queue does the following:
{% highlight go %}
{% raw %}
for _, pr := range readyToMergePRs() {
if testsAreStable() {
mergePR(pr)
}
}
{% endraw %}
{% endhighlight %}
The status of the submit-queue is [online.](http://submit-queue.k8s.io/)
### Ready to merge status
A PR is considered "ready for merging" if it matches the following:
* it has the `lgtm` label, and that `lgtm` is newer than the latest commit
* it has passed the cla pre-submit and has the `cla:yes` label
* it has passed the travis and shippable pre-submit tests
* one (or all) of
* its author is in kubernetes/contrib/submit-queue/whitelist.txt
* its author is in contributors.txt via the github API.
* the PR has the `ok-to-merge` label
* One (or both of)
* it has passed the Jenkins e2e test
* it has the `e2e-not-required` label
Note that the combined whitelist/committer list is available at [submit-queue.k8s.io](http://submit-queue.k8s.io)
### Merge process
Merges _only_ occur when the `critical builds` (Jenkins e2e for gce, gke, scalability, upgrade) are passing.
We're open to including more builds here, let us know...
Merges are serialized, so only a single PR is merged at a time, to ensure against races.
If the PR has the `e2e-not-required` label, it is simply merged.
If the PR does not have this label, e2e tests are re-run, if these new tests pass, the PR is merged.
If e2e flakes or is currently buggy, the PR will not be merged, but it will be re-run on the following
pass.
## Github Munger
We also run a [github "munger"](https://github.com/kubernetes/contrib/tree/master/mungegithub)
This runs repeatedly over github pulls and issues and runs modular "mungers" similar to "mungedocs"
Currently this runs:
* blunderbuss - Tries to automatically find an owner for a PR without an owner, uses mapping file here:
https://github.com/kubernetes/contrib/blob/master/mungegithub/blunderbuss.yml
* needs-rebase - Adds `needs-rebase` to PRs that aren't currently mergeable, and removes it from those that are.
* size - Adds `size/xs` - `size/xxl` labels to PRs
* ok-to-test - Adds the `ok-to-test` message to PRs that have an `lgtm` but the e2e-builder would otherwise not test due to whitelist
* ping-ci - Attempts to ping the ci systems (Travis/Shippable) if they are missing from a PR.
* lgtm-after-commit - Removes the `lgtm` label from PRs where there are commits that are newer than the `lgtm` label
In the works:
* issue-detector - machine learning for determining if an issue that has been filed is a `support` issue, `bug` or `feature`
Please feel free to unleash your creativity on this tool, send us new mungers that you think will help support the Kubernetes development process.
## PR builder
We also run a robotic PR builder that attempts to run e2e tests for each PR.
Before a PR from an unknown user is run, the PR builder bot (`k8s-bot`) asks to a message from a
contributor that a PR is "ok to test", the contributor replies with that message. Contributors can also
add users to the whitelist by replying with the message "add to whitelist" ("please" is optional, but
remember to treat your robots with kindness...)
If a PR is approved for testing, and tests either haven't run, or need to be re-run, you can ask the
PR builder to re-run the tests. To do this, reply to the PR with a message that begins with `@k8s-bot test this`, this should trigger a re-build/re-test.
## FAQ:
#### How can I ask my PR to be tested again for Jenkins failures?
Right now you have to ask a contributor (this may be you!) to re-run the test with "@k8s-bot test this"
### How can I kick Shippable to re-test on a failure?
Right now the easiest way is to close and then immediately re-open the PR.
<!-- BEGIN MUNGE: IS_VERSIONED -->
<!-- TAG IS_VERSIONED -->
<!-- END MUNGE: IS_VERSIONED -->
<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/devel/automation.md?pixel)]()
<!-- END MUNGE: GENERATED_ANALYTICS -->
v1.1/docs/devel/cherry-picks.md
+56
View File
@@ -0,0 +1,56 @@
---
layout: docwithnav
title: "Overview"
---
<!-- BEGIN MUNGE: UNVERSIONED_WARNING -->
<!-- END MUNGE: UNVERSIONED_WARNING -->
# Overview
This document explains cherry picks are managed on release branches within the
Kubernetes projects.
## Propose a Cherry Pick
Any contributor can propose a cherry pick of any pull request, like so:
{% highlight sh %}
{% raw %}
hack/cherry_pick_pull.sh upstream/release-3.14 98765
{% endraw %}
{% endhighlight %}
This will walk you through the steps to propose an automated cherry pick of pull
#98765 for remote branch `upstream/release-3.14`.
## Cherry Pick Review
Cherry pick pull requests are reviewed differently than normal pull requests. In
particular, they may be self-merged by the release branch owner without fanfare,
in the case the release branch owner knows the cherry pick was already
requested - this should not be the norm, but it may happen.
[Contributor License Agreements](http://releases.k8s.io/release-1.1/CONTRIBUTING.md) is considered implicit
for all code within cherry-pick pull requests, ***unless there is a large
conflict***.
## Searching for Cherry Picks
Now that we've structured cherry picks as PRs, searching for all cherry-picks
against a release is a GitHub query: For example,
[this query is all of the v0.21.x cherry-picks](https://github.com/kubernetes/kubernetes/pulls?utf8=%E2%9C%93&q=is%3Apr+%22automated+cherry+pick%22+base%3Arelease-0.21)
<!-- BEGIN MUNGE: IS_VERSIONED -->
<!-- TAG IS_VERSIONED -->
<!-- END MUNGE: IS_VERSIONED -->
<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/devel/cherry-picks.md?pixel)]()
<!-- END MUNGE: GENERATED_ANALYTICS -->
v1.1/docs/devel/cli-roadmap.md
+28
View File
@@ -0,0 +1,28 @@
---
layout: docwithnav
title: "Kubernetes CLI/Configuration Roadmap"
---
<!-- BEGIN MUNGE: UNVERSIONED_WARNING -->
<!-- END MUNGE: UNVERSIONED_WARNING -->
# Kubernetes CLI/Configuration Roadmap
See github issues with the following labels:
* [area/app-config-deployment](https://github.com/kubernetes/kubernetes/labels/area/app-config-deployment)
* [component/kubectl](https://github.com/kubernetes/kubernetes/labels/component/kubectl)
* [component/clientlib](https://github.com/kubernetes/kubernetes/labels/component/clientlib)
<!-- BEGIN MUNGE: IS_VERSIONED -->
<!-- TAG IS_VERSIONED -->
<!-- END MUNGE: IS_VERSIONED -->
<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/devel/cli-roadmap.md?pixel)]()
<!-- END MUNGE: GENERATED_ANALYTICS -->
v1.1/docs/devel/client-libraries.md
+40
View File
@@ -0,0 +1,40 @@
---
layout: docwithnav
title: "Kubernetes API client libraries"
---
<!-- BEGIN MUNGE: UNVERSIONED_WARNING -->
<!-- END MUNGE: UNVERSIONED_WARNING -->
## Kubernetes API client libraries
### Supported
* [Go](http://releases.k8s.io/release-1.1/pkg/client/)
### User Contributed
*Note: Libraries provided by outside parties are supported by their authors, not the core Kubernetes team*
* [Java (OSGi)](https://bitbucket.org/amdatulabs/amdatu-kubernetes)
* [Java (Fabric8, OSGi)](https://github.com/fabric8io/kubernetes-client)
* [Ruby](https://github.com/Ch00k/kuber)
* [Ruby](https://github.com/abonas/kubeclient)
* [PHP](https://github.com/devstub/kubernetes-api-php-client)
* [PHP](https://github.com/maclof/kubernetes-client)
* [Node.js](https://github.com/tenxcloud/node-kubernetes-client)
* [Perl](https://metacpan.org/pod/Net::Kubernetes)
<!-- BEGIN MUNGE: IS_VERSIONED -->
<!-- TAG IS_VERSIONED -->
<!-- END MUNGE: IS_VERSIONED -->
<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/devel/client-libraries.md?pixel)]()
<!-- END MUNGE: GENERATED_ANALYTICS -->
v1.1/docs/devel/coding-conventions.md
+73
View File
@@ -0,0 +1,73 @@
---
layout: docwithnav
title: "devel/coding-conventions"
---
<!-- BEGIN MUNGE: UNVERSIONED_WARNING -->
<!-- END MUNGE: UNVERSIONED_WARNING -->
Code conventions
- Bash
- https://google-styleguide.googlecode.com/svn/trunk/shell.xml
- Ensure that build, release, test, and cluster-management scripts run on OS X
- Go
- Ensure your code passes the [presubmit checks](development.html#hooks)
- [Go Code Review Comments](https://github.com/golang/go/wiki/CodeReviewComments)
- [Effective Go](https://golang.org/doc/effective_go.html)
- Comment your code.
- [Go's commenting conventions](http://blog.golang.org/godoc-documenting-go-code)
- If reviewers ask questions about why the code is the way it is, that's a sign that comments might be helpful.
- Command-line flags should use dashes, not underscores
- Naming
- Please consider package name when selecting an interface name, and avoid redundancy.
- e.g.: `storage.Interface` is better than `storage.StorageInterface`.
- Do not use uppercase characters, underscores, or dashes in package names.
- Please consider parent directory name when choosing a package name.
- so pkg/controllers/autoscaler/foo.go should say `package autoscaler` not `package autoscalercontroller`.
- Unless there's a good reason, the `package foo` line should match the name of the directory in which the .go file exists.
- Importers can use a different name if they need to disambiguate.
- Locks should be called `lock` and should never be embedded (always `lock sync.Mutex`). When multiple locks are present, give each lock a distinct name following Go conventions - `stateLock`, `mapLock` etc.
- API conventions
- [API changes](api_changes.html)
- [API conventions](api-conventions.html)
- [Kubectl conventions](kubectl-conventions.html)
- [Logging conventions](logging.html)
Testing conventions
- All new packages and most new significant functionality must come with unit tests
- Table-driven tests are preferred for testing multiple scenarios/inputs; for example, see [TestNamespaceAuthorization](https://releases.k8s.io/release-1.1/test/integration/auth_test.go)
- Significant features should come with integration (test/integration) and/or end-to-end (test/e2e) tests
- Including new kubectl commands and major features of existing commands
- Unit tests must pass on OS X and Windows platforms - if you use Linux specific features, your test case must either be skipped on windows or compiled out (skipped is better when running Linux specific commands, compiled out is required when your code does not compile on Windows).
Directory and file conventions
- Avoid package sprawl. Find an appropriate subdirectory for new packages. (See [#4851](http://issues.k8s.io/4851) for discussion.)
- Libraries with no more appropriate home belong in new package subdirectories of pkg/util
- Avoid general utility packages. Packages called "util" are suspect. Instead, derive a name that describes your desired function. For example, the utility functions dealing with waiting for operations are in the "wait" package and include functionality like Poll. So the full name is wait.Poll
- Go source files and directories use underscores, not dashes
- Package directories should generally avoid using separators as much as possible (when packages are multiple words, they usually should be in nested subdirectories).
- Document directories and filenames should use dashes rather than underscores
- Contrived examples that illustrate system features belong in /docs/user-guide or /docs/admin, depending on whether it is a feature primarily intended for users that deploy applications or cluster administrators, respectively. Actual application examples belong in /examples.
- Examples should also illustrate [best practices for using the system](../user-guide/config-best-practices.html)
- Third-party code
- Third-party Go code is managed using Godeps
- Other third-party code belongs in /third_party
- Third-party code must include licenses
- This includes modified third-party code and excerpts, as well
Coding advice
- Go
- [Go landmines](https://gist.github.com/lavalamp/4bd23295a9f32706a48f)
<!-- BEGIN MUNGE: IS_VERSIONED -->
<!-- TAG IS_VERSIONED -->
<!-- END MUNGE: IS_VERSIONED -->
<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/devel/coding-conventions.md?pixel)]()
<!-- END MUNGE: GENERATED_ANALYTICS -->
v1.1/docs/devel/collab.md
+62
View File
@@ -0,0 +1,62 @@
---
layout: docwithnav
title: "On Collaborative Development"
---
<!-- BEGIN MUNGE: UNVERSIONED_WARNING -->
<!-- END MUNGE: UNVERSIONED_WARNING -->
# On Collaborative Development
Kubernetes is open source, but many of the people working on it do so as their day job. In order to avoid forcing people to be "at work" effectively 24/7, we want to establish some semi-formal protocols around development. Hopefully these rules make things go more smoothly. If you find that this is not the case, please complain loudly.
## Patches welcome
First and foremost: as a potential contributor, your changes and ideas are welcome at any hour of the day or night, weekdays, weekends, and holidays. Please do not ever hesitate to ask a question or send a PR.
## Code reviews
All changes must be code reviewed. For non-maintainers this is obvious, since you can't commit anyway. But even for maintainers, we want all changes to get at least one review, preferably (for non-trivial changes obligatorily) from someone who knows the areas the change touches. For non-trivial changes we may want two reviewers. The primary reviewer will make this decision and nominate a second reviewer, if needed. Except for trivial changes, PRs should not be committed until relevant parties (e.g. owners of the subsystem affected by the PR) have had a reasonable chance to look at PR in their local business hours.
Most PRs will find reviewers organically. If a maintainer intends to be the primary reviewer of a PR they should set themselves as the assignee on GitHub and say so in a reply to the PR. Only the primary reviewer of a change should actually do the merge, except in rare cases (e.g. they are unavailable in a reasonable timeframe).
If a PR has gone 2 work days without an owner emerging, please poke the PR thread and ask for a reviewer to be assigned.
Except for rare cases, such as trivial changes (e.g. typos, comments) or emergencies (e.g. broken builds), maintainers should not merge their own changes.
Expect reviewers to request that you avoid [common go style mistakes](https://github.com/golang/go/wiki/CodeReviewComments) in your PRs.
## Assigned reviews
Maintainers can assign reviews to other maintainers, when appropriate. The assignee becomes the shepherd for that PR and is responsible for merging the PR once they are satisfied with it or else closing it. The assignee might request reviews from non-maintainers.
## Merge hours
Maintainers will do merges of appropriately reviewed-and-approved changes during their local "business hours" (typically 7:00 am Monday to 5:00 pm (17:00h) Friday). PRs that arrive over the weekend or on holidays will only be merged if there is a very good reason for it and if the code review requirements have been met. Concretely this means that nobody should merge changes immediately before going to bed for the night.
There may be discussion an even approvals granted outside of the above hours, but merges will generally be deferred.
If a PR is considered complex or controversial, the merge of that PR should be delayed to give all interested parties in all timezones the opportunity to provide feedback. Concretely, this means that such PRs should be held for 24
hours before merging. Of course "complex" and "controversial" are left to the judgment of the people involved, but we trust that part of being a committer is the judgment required to evaluate such things honestly, and not be
motivated by your desire (or your cube-mate's desire) to get their code merged. Also see "Holds" below, any reviewer can issue a "hold" to indicate that the PR is in fact complicated or complex and deserves further review.
PRs that are incorrectly judged to be merge-able, may be reverted and subject to re-review, if subsequent reviewers believe that they in fact are controversial or complex.
## Holds
Any maintainer or core contributor who wants to review a PR but does not have time immediately may put a hold on a PR simply by saying so on the PR discussion and offering an ETA measured in single-digit days at most. Any PR that has a hold shall not be merged until the person who requested the hold acks the review, withdraws their hold, or is overruled by a preponderance of maintainers.
<!-- BEGIN MUNGE: IS_VERSIONED -->
<!-- TAG IS_VERSIONED -->
<!-- END MUNGE: IS_VERSIONED -->
<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/devel/collab.md?pixel)]()
<!-- END MUNGE: GENERATED_ANALYTICS -->
v1.1/docs/devel/developer-guides/vagrant.md
+430
View File
@@ -0,0 +1,430 @@
---
layout: docwithnav
title: "Getting started with Vagrant"
---
<!-- BEGIN MUNGE: UNVERSIONED_WARNING -->
<!-- END MUNGE: UNVERSIONED_WARNING -->
## Getting started with Vagrant
Running kubernetes with Vagrant (and VirtualBox) is an easy way to run/test/develop on your local machine (Linux, Mac OS X).
### Prerequisites
1. Install latest version >= 1.6.2 of vagrant from http://www.vagrantup.com/downloads.html
2. Install one of:
1. The latest version of Virtual Box from https://www.virtualbox.org/wiki/Downloads
2. [VMWare Fusion](https://www.vmware.com/products/fusion/) version 5 or greater as well as the appropriate [Vagrant VMWare Fusion provider](https://www.vagrantup.com/vmware)
3. [VMWare Workstation](https://www.vmware.com/products/workstation/) version 9 or greater as well as the [Vagrant VMWare Workstation provider](https://www.vagrantup.com/vmware)
4. [Parallels Desktop](https://www.parallels.com/products/desktop/) version 9 or greater as well as the [Vagrant Parallels provider](https://parallels.github.io/vagrant-parallels/)
3. Get or build a [binary release](../../../docs/getting-started-guides/binary_release.html)
### Setup
By default, the Vagrant setup will create a single master VM (called kubernetes-master) and one node (called kubernetes-minion-1). Each VM will take 1 GB, so make sure you have at least 2GB to 4GB of free memory (plus appropriate free disk space). To start your local cluster, open a shell and run:
{% highlight sh %}
{% raw %}
cd kubernetes
export KUBERNETES_PROVIDER=vagrant
./cluster/kube-up.sh
{% endraw %}
{% endhighlight %}
The `KUBERNETES_PROVIDER` environment variable tells all of the various cluster management scripts which variant to use. If you forget to set this, the assumption is you are running on Google Compute Engine.
If you installed more than one Vagrant provider, Kubernetes will usually pick the appropriate one. However, you can override which one Kubernetes will use by setting the [`VAGRANT_DEFAULT_PROVIDER`](https://docs.vagrantup.com/v2/providers/default.html) environment variable:
{% highlight sh %}
{% raw %}
export VAGRANT_DEFAULT_PROVIDER=parallels
export KUBERNETES_PROVIDER=vagrant
./cluster/kube-up.sh
{% endraw %}
{% endhighlight %}
Vagrant will provision each machine in the cluster with all the necessary components to run Kubernetes. The initial setup can take a few minutes to complete on each machine.
By default, each VM in the cluster is running Fedora, and all of the Kubernetes services are installed into systemd.
To access the master or any node:
{% highlight sh %}
{% raw %}
vagrant ssh master
vagrant ssh minion-1
{% endraw %}
{% endhighlight %}
If you are running more than one nodes, you can access the others by:
{% highlight sh %}
{% raw %}
vagrant ssh minion-2
vagrant ssh minion-3
{% endraw %}
{% endhighlight %}
To view the service status and/or logs on the kubernetes-master:
{% highlight console %}
{% raw %}
$ vagrant ssh master
[vagrant@kubernetes-master ~] $ sudo systemctl status kube-apiserver
[vagrant@kubernetes-master ~] $ sudo journalctl -r -u kube-apiserver
[vagrant@kubernetes-master ~] $ sudo systemctl status kube-controller-manager
[vagrant@kubernetes-master ~] $ sudo journalctl -r -u kube-controller-manager
[vagrant@kubernetes-master ~] $ sudo systemctl status etcd
[vagrant@kubernetes-master ~] $ sudo systemctl status nginx
{% endraw %}
{% endhighlight %}
To view the services on any of the nodes:
{% highlight console %}
{% raw %}
$ vagrant ssh minion-1
[vagrant@kubernetes-minion-1] $ sudo systemctl status docker
[vagrant@kubernetes-minion-1] $ sudo journalctl -r -u docker
[vagrant@kubernetes-minion-1] $ sudo systemctl status kubelet
[vagrant@kubernetes-minion-1] $ sudo journalctl -r -u kubelet
{% endraw %}
{% endhighlight %}
### Interacting with your Kubernetes cluster with Vagrant.
With your Kubernetes cluster up, you can manage the nodes in your cluster with the regular Vagrant commands.
To push updates to new Kubernetes code after making source changes:
{% highlight sh %}
{% raw %}
./cluster/kube-push.sh
{% endraw %}
{% endhighlight %}
To stop and then restart the cluster:
{% highlight sh %}
{% raw %}
vagrant halt
./cluster/kube-up.sh
{% endraw %}
{% endhighlight %}
To destroy the cluster:
{% highlight sh %}
{% raw %}
vagrant destroy
{% endraw %}
{% endhighlight %}
Once your Vagrant machines are up and provisioned, the first thing to do is to check that you can use the `kubectl.sh` script.
You may need to build the binaries first, you can do this with `make`
{% highlight console %}
{% raw %}
$ ./cluster/kubectl.sh get nodes
NAME LABELS STATUS
kubernetes-minion-0whl kubernetes.io/hostname=kubernetes-minion-0whl Ready
kubernetes-minion-4jdf kubernetes.io/hostname=kubernetes-minion-4jdf Ready
kubernetes-minion-epbe kubernetes.io/hostname=kubernetes-minion-epbe Ready
{% endraw %}
{% endhighlight %}
### Interacting with your Kubernetes cluster with the `kube-*` scripts.
Alternatively to using the vagrant commands, you can also use the `cluster/kube-*.sh` scripts to interact with the vagrant based provider just like any other hosting platform for kubernetes.
All of these commands assume you have set `KUBERNETES_PROVIDER` appropriately:
{% highlight sh %}
{% raw %}
export KUBERNETES_PROVIDER=vagrant
{% endraw %}
{% endhighlight %}
Bring up a vagrant cluster
{% highlight sh %}
{% raw %}
./cluster/kube-up.sh
{% endraw %}
{% endhighlight %}
Destroy the vagrant cluster
{% highlight sh %}
{% raw %}
./cluster/kube-down.sh
{% endraw %}
{% endhighlight %}
Update the vagrant cluster after you make changes (only works when building your own releases locally):
{% highlight sh %}
{% raw %}
./cluster/kube-push.sh
{% endraw %}
{% endhighlight %}
Interact with the cluster
{% highlight sh %}
{% raw %}
./cluster/kubectl.sh
{% endraw %}
{% endhighlight %}
### Authenticating with your master
When using the vagrant provider in Kubernetes, the `cluster/kubectl.sh` script will cache your credentials in a `~/.kubernetes_vagrant_auth` file so you will not be prompted for them in the future.
{% highlight console %}
{% raw %}
$ cat ~/.kubernetes_vagrant_auth
{ "User": "vagrant",
"Password": "vagrant"
"CAFile": "/home/k8s_user/.kubernetes.vagrant.ca.crt",
"CertFile": "/home/k8s_user/.kubecfg.vagrant.crt",
"KeyFile": "/home/k8s_user/.kubecfg.vagrant.key"
}
{% endraw %}
{% endhighlight %}
You should now be set to use the `cluster/kubectl.sh` script. For example try to list the nodes that you have started with:
{% highlight sh %}
{% raw %}
./cluster/kubectl.sh get nodes
{% endraw %}
{% endhighlight %}
### Running containers
Your cluster is running, you can list the nodes in your cluster:
{% highlight console %}
{% raw %}
$ ./cluster/kubectl.sh get nodes
NAME LABELS STATUS
kubernetes-minion-0whl kubernetes.io/hostname=kubernetes-minion-0whl Ready
kubernetes-minion-4jdf kubernetes.io/hostname=kubernetes-minion-4jdf Ready
kubernetes-minion-epbe kubernetes.io/hostname=kubernetes-minion-epbe Ready
{% endraw %}
{% endhighlight %}
Now start running some containers!
You can now use any of the cluster/kube-*.sh commands to interact with your VM machines.
Before starting a container there will be no pods, services and replication controllers.
{% highlight console %}
{% raw %}
$ cluster/kubectl.sh get pods
NAME READY STATUS RESTARTS AGE
$ cluster/kubectl.sh get services
NAME LABELS SELECTOR IP(S) PORT(S)
$ cluster/kubectl.sh get rc
CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS
{% endraw %}
{% endhighlight %}
Start a container running nginx with a replication controller and three replicas
{% highlight console %}
{% raw %}
$ cluster/kubectl.sh run my-nginx --image=nginx --replicas=3 --port=80
CONTROLLER CONTAINER(S) IMAGE(S) SELECTOR REPLICAS
my-nginx my-nginx nginx run=my-nginx 3
{% endraw %}
{% endhighlight %}
When listing the pods, you will see that three containers have been started and are in Waiting state:
{% highlight console %}
{% raw %}
$ cluster/kubectl.sh get pods
NAME READY STATUS RESTARTS AGE
my-nginx-389da 1/1 Waiting 0 33s
my-nginx-kqdjk 1/1 Waiting 0 33s
my-nginx-nyj3x 1/1 Waiting 0 33s
{% endraw %}
{% endhighlight %}
You need to wait for the provisioning to complete, you can monitor the minions by doing:
{% highlight console %}
{% raw %}
$ sudo salt '*minion-1' cmd.run 'docker images'
kubernetes-minion-1:
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
<none> <none> 96864a7d2df3 26 hours ago 204.4 MB
kubernetes/pause latest 6c4579af347b 8 weeks ago 239.8 kB
{% endraw %}
{% endhighlight %}
Once the docker image for nginx has been downloaded, the container will start and you can list it:
{% highlight console %}
{% raw %}
$ sudo salt '*minion-1' cmd.run 'docker ps'
kubernetes-minion-1:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
dbe79bf6e25b nginx:latest "nginx" 21 seconds ago Up 19 seconds k8s--mynginx.8c5b8a3a--7813c8bd_-_3ffe_-_11e4_-_9036_-_0800279696e1.etcd--7813c8bd_-_3ffe_-_11e4_-_9036_-_0800279696e1--fcfa837f
fa0e29c94501 kubernetes/pause:latest "/pause" 8 minutes ago Up 8 minutes 0.0.0.0:8080->80/tcp k8s--net.a90e7ce4--7813c8bd_-_3ffe_-_11e4_-_9036_-_0800279696e1.etcd--7813c8bd_-_3ffe_-_11e4_-_9036_-_0800279696e1--baf5b21b
{% endraw %}
{% endhighlight %}
Going back to listing the pods, services and replicationcontrollers, you now have:
{% highlight console %}
{% raw %}
$ cluster/kubectl.sh get pods
NAME READY STATUS RESTARTS AGE
my-nginx-389da 1/1 Running 0 33s
my-nginx-kqdjk 1/1 Running 0 33s
my-nginx-nyj3x 1/1 Running 0 33s
$ cluster/kubectl.sh get services
NAME LABELS SELECTOR IP(S) PORT(S)
$ cluster/kubectl.sh get rc
NAME IMAGE(S) SELECTOR REPLICAS
my-nginx nginx run=my-nginx 3
{% endraw %}
{% endhighlight %}
We did not start any services, hence there are none listed. But we see three replicas displayed properly.
Check the [guestbook](../../../examples/guestbook/README.html) application to learn how to create a service.
You can already play with scaling the replicas with:
{% highlight console %}
{% raw %}
$ ./cluster/kubectl.sh scale rc my-nginx --replicas=2
$ ./cluster/kubectl.sh get pods
NAME READY STATUS RESTARTS AGE
my-nginx-kqdjk 1/1 Running 0 13m
my-nginx-nyj3x 1/1 Running 0 13m
{% endraw %}
{% endhighlight %}
Congratulations!
### Testing
The following will run all of the end-to-end testing scenarios assuming you set your environment in `cluster/kube-env.sh`:
{% highlight sh %}
{% raw %}
NUM_MINIONS=3 hack/e2e-test.sh
{% endraw %}
{% endhighlight %}
### Troubleshooting
#### I keep downloading the same (large) box all the time!
By default the Vagrantfile will download the box from S3. You can change this (and cache the box locally) by providing a name and an alternate URL when calling `kube-up.sh`
{% highlight sh %}
{% raw %}
export KUBERNETES_BOX_NAME=choose_your_own_name_for_your_kuber_box
export KUBERNETES_BOX_URL=path_of_your_kuber_box
export KUBERNETES_PROVIDER=vagrant
./cluster/kube-up.sh
{% endraw %}
{% endhighlight %}
#### I just created the cluster, but I am getting authorization errors!
You probably have an incorrect ~/.kubernetes_vagrant_auth file for the cluster you are attempting to contact.
{% highlight sh %}
{% raw %}
rm ~/.kubernetes_vagrant_auth
{% endraw %}
{% endhighlight %}
After using kubectl.sh make sure that the correct credentials are set:
{% highlight console %}
{% raw %}
$ cat ~/.kubernetes_vagrant_auth
{
"User": "vagrant",
"Password": "vagrant"
}
{% endraw %}
{% endhighlight %}
#### I just created the cluster, but I do not see my container running!
If this is your first time creating the cluster, the kubelet on each node schedules a number of docker pull requests to fetch prerequisite images. This can take some time and as a result may delay your initial pod getting provisioned.
#### I changed Kubernetes code, but it's not running!
Are you sure there was no build error? After running `$ vagrant provision`, scroll up and ensure that each Salt state was completed successfully on each box in the cluster.
It's very likely you see a build error due to an error in your source files!
#### I have brought Vagrant up but the nodes won't validate!
Are you sure you built a release first? Did you install `net-tools`? For more clues, login to one of the nodes (`vagrant ssh minion-1`) and inspect the salt minion log (`sudo cat /var/log/salt/minion`).
#### I want to change the number of nodes!
You can control the number of nodes that are instantiated via the environment variable `NUM_MINIONS` on your host machine. If you plan to work with replicas, we strongly encourage you to work with enough nodes to satisfy your largest intended replica size. If you do not plan to work with replicas, you can save some system resources by running with a single node. You do this, by setting `NUM_MINIONS` to 1 like so:
{% highlight sh %}
{% raw %}
export NUM_MINIONS=1
{% endraw %}
{% endhighlight %}
#### I want my VMs to have more memory!
You can control the memory allotted to virtual machines with the `KUBERNETES_MEMORY` environment variable.
Just set it to the number of megabytes you would like the machines to have. For example:
{% highlight sh %}
{% raw %}
export KUBERNETES_MEMORY=2048
{% endraw %}
{% endhighlight %}
If you need more granular control, you can set the amount of memory for the master and nodes independently. For example:
{% highlight sh %}
{% raw %}
export KUBERNETES_MASTER_MEMORY=1536
export KUBERNETES_MINION_MEMORY=2048
{% endraw %}
{% endhighlight %}
#### I ran vagrant suspend and nothing works!
`vagrant suspend` seems to mess up the network. It's not supported at this time.
<!-- BEGIN MUNGE: IS_VERSIONED -->
<!-- TAG IS_VERSIONED -->
<!-- END MUNGE: IS_VERSIONED -->
<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/devel/developer-guides/vagrant.md?pixel)]()
<!-- END MUNGE: GENERATED_ANALYTICS -->
v1.1/docs/devel/development.md
+395
View File
@@ -0,0 +1,395 @@
---
layout: docwithnav
title: "Development Guide"
---
<!-- BEGIN MUNGE: UNVERSIONED_WARNING -->
<!-- END MUNGE: UNVERSIONED_WARNING -->
# Development Guide
# Releases and Official Builds
Official releases are built in Docker containers. Details are [here](http://releases.k8s.io/release-1.1/build/README.md). You can do simple builds and development with just a local Docker installation. If want to build go locally outside of docker, please continue below.
## Go development environment
Kubernetes is written in [Go](http://golang.org) programming language. If you haven't set up Go development environment, please follow [this instruction](http://golang.org/doc/code.html) to install go tool and set up GOPATH. Ensure your version of Go is at least 1.3.
## Git Setup
Below, we outline one of the more common git workflows that core developers use. Other git workflows are also valid.
### Visual overview
![Git workflow](git_workflow.png)
### Fork the main repository
1. Go to https://github.com/kubernetes/kubernetes
2. Click the "Fork" button (at the top right)
### Clone your fork
The commands below require that you have $GOPATH set ([$GOPATH docs](https://golang.org/doc/code.html#GOPATH)). We highly recommend you put Kubernetes' code into your GOPATH. Note: the commands below will not work if there is more than one directory in your `$GOPATH`.
{% highlight sh %}
{% raw %}
mkdir -p $GOPATH/src/k8s.io
cd $GOPATH/src/k8s.io
# Replace "$YOUR_GITHUB_USERNAME" below with your github username
git clone https://github.com/$YOUR_GITHUB_USERNAME/kubernetes.git
cd kubernetes
git remote add upstream 'https://github.com/kubernetes/kubernetes.git'
{% endraw %}
{% endhighlight %}
### Create a branch and make changes
{% highlight sh %}
{% raw %}
git checkout -b myfeature
# Make your code changes
{% endraw %}
{% endhighlight %}
### Keeping your development fork in sync
{% highlight sh %}
{% raw %}
git fetch upstream
git rebase upstream/master
{% endraw %}
{% endhighlight %}
Note: If you have write access to the main repository at github.com/kubernetes/kubernetes, you should modify your git configuration so that you can't accidentally push to upstream:
{% highlight sh %}
{% raw %}
git remote set-url --push upstream no_push
{% endraw %}
{% endhighlight %}
### Committing changes to your fork
{% highlight sh %}
{% raw %}
git commit
git push -f origin myfeature
{% endraw %}
{% endhighlight %}
### Creating a pull request
1. Visit https://github.com/$YOUR_GITHUB_USERNAME/kubernetes
2. Click the "Compare and pull request" button next to your "myfeature" branch.
3. Check out the pull request [process](pull-requests.html) for more details
### When to retain commits and when to squash
Upon merge, all git commits should represent meaningful milestones or units of
work. Use commits to add clarity to the development and review process.
Before merging a PR, squash any "fix review feedback", "typo", and "rebased"
sorts of commits. It is not imperative that every commit in a PR compile and
pass tests independently, but it is worth striving for. For mass automated
fixups (e.g. automated doc formatting), use one or more commits for the
changes to tooling and a final commit to apply the fixup en masse. This makes
reviews much easier.
See [Faster Reviews](faster_reviews.html) for more details.
## godep and dependency management
Kubernetes uses [godep](https://github.com/tools/godep) to manage dependencies. It is not strictly required for building Kubernetes but it is required when managing dependencies under the Godeps/ tree, and is required by a number of the build and test scripts. Please make sure that ``godep`` is installed and in your ``$PATH``.
### Installing godep
There are many ways to build and host go binaries. Here is an easy way to get utilities like `godep` installed:
1) Ensure that [mercurial](http://mercurial.selenic.com/wiki/Download) is installed on your system. (some of godep's dependencies use the mercurial
source control system). Use `apt-get install mercurial` or `yum install mercurial` on Linux, or [brew.sh](http://brew.sh) on OS X, or download
directly from mercurial.
2) Create a new GOPATH for your tools and install godep:
{% highlight sh %}
{% raw %}
export GOPATH=$HOME/go-tools
mkdir -p $GOPATH
go get github.com/tools/godep
{% endraw %}
{% endhighlight %}
3) Add $GOPATH/bin to your path. Typically you'd add this to your ~/.profile:
{% highlight sh %}
{% raw %}
export GOPATH=$HOME/go-tools
export PATH=$PATH:$GOPATH/bin
{% endraw %}
{% endhighlight %}
### Using godep
Here's a quick walkthrough of one way to use godeps to add or update a Kubernetes dependency into Godeps/_workspace. For more details, please see the instructions in [godep's documentation](https://github.com/tools/godep).
1) Devote a directory to this endeavor:
_Devoting a separate directory is not required, but it is helpful to separate dependency updates from other changes._
{% highlight sh %}
{% raw %}
export KPATH=$HOME/code/kubernetes
mkdir -p $KPATH/src/k8s.io/kubernetes
cd $KPATH/src/k8s.io/kubernetes
git clone https://path/to/your/fork .
# Or copy your existing local repo here. IMPORTANT: making a symlink doesn't work.
{% endraw %}
{% endhighlight %}
2) Set up your GOPATH.
{% highlight sh %}
{% raw %}
# Option A: this will let your builds see packages that exist elsewhere on your system.
export GOPATH=$KPATH:$GOPATH
# Option B: This will *not* let your local builds see packages that exist elsewhere on your system.
export GOPATH=$KPATH
# Option B is recommended if you're going to mess with the dependencies.
{% endraw %}
{% endhighlight %}
3) Populate your new GOPATH.
{% highlight sh %}
{% raw %}
cd $KPATH/src/k8s.io/kubernetes
godep restore
{% endraw %}
{% endhighlight %}
4) Next, you can either add a new dependency or update an existing one.
{% highlight sh %}
{% raw %}
# To add a new dependency, do:
cd $KPATH/src/k8s.io/kubernetes
go get path/to/dependency
# Change code in Kubernetes to use the dependency.
godep save ./...
# To update an existing dependency, do:
cd $KPATH/src/k8s.io/kubernetes
go get -u path/to/dependency
# Change code in Kubernetes accordingly if necessary.
godep update path/to/dependency/...
{% endraw %}
{% endhighlight %}
_If `go get -u path/to/dependency` fails with compilation errors, instead try `go get -d -u path/to/dependency`
to fetch the dependencies without compiling them. This can happen when updating the cadvisor dependency._
5) Before sending your PR, it's a good idea to sanity check that your Godeps.json file is ok by running hack/verify-godeps.sh
_If hack/verify-godeps.sh fails after a `godep update`, it is possible that a transitive dependency was added or removed but not
updated by godeps. It then may be necessary to perform a `godep save ./...` to pick up the transitive dependency changes._
It is sometimes expedient to manually fix the /Godeps/godeps.json file to minimize the changes.
Please send dependency updates in separate commits within your PR, for easier reviewing.
## Hooks
Before committing any changes, please link/copy these hooks into your .git
directory. This will keep you from accidentally committing non-gofmt'd go code.
{% highlight sh %}
{% raw %}
cd kubernetes/.git/hooks/
ln -s ../../hooks/pre-commit .
{% endraw %}
{% endhighlight %}
## Unit tests
{% highlight sh %}
{% raw %}
cd kubernetes
hack/test-go.sh
{% endraw %}
{% endhighlight %}
Alternatively, you could also run:
{% highlight sh %}
{% raw %}
cd kubernetes
godep go test ./...
{% endraw %}
{% endhighlight %}
If you only want to run unit tests in one package, you could run ``godep go test`` under the package directory. For example, the following commands will run all unit tests in package kubelet:
{% highlight console %}
{% raw %}
$ cd kubernetes # step into the kubernetes directory.
$ cd pkg/kubelet
$ godep go test
# some output from unit tests
PASS
ok k8s.io/kubernetes/pkg/kubelet 0.317s
{% endraw %}
{% endhighlight %}
## Coverage
Currently, collecting coverage is only supported for the Go unit tests.
To run all unit tests and generate an HTML coverage report, run the following:
{% highlight sh %}
{% raw %}
cd kubernetes
KUBE_COVER=y hack/test-go.sh
{% endraw %}
{% endhighlight %}
At the end of the run, an the HTML report will be generated with the path printed to stdout.
To run tests and collect coverage in only one package, pass its relative path under the `kubernetes` directory as an argument, for example:
{% highlight sh %}
{% raw %}
cd kubernetes
KUBE_COVER=y hack/test-go.sh pkg/kubectl
{% endraw %}
{% endhighlight %}
Multiple arguments can be passed, in which case the coverage results will be combined for all tests run.
Coverage results for the project can also be viewed on [Coveralls](https://coveralls.io/r/kubernetes/kubernetes), and are continuously updated as commits are merged. Additionally, all pull requests which spawn a Travis build will report unit test coverage results to Coveralls. Coverage reports from before the Kubernetes Github organization was created can be found [here](https://coveralls.io/r/GoogleCloudPlatform/kubernetes).
## Integration tests
You need an [etcd](https://github.com/coreos/etcd/releases/tag/v2.0.0) in your path, please make sure it is installed and in your ``$PATH``.
{% highlight sh %}
{% raw %}
cd kubernetes
hack/test-integration.sh
{% endraw %}
{% endhighlight %}
## End-to-End tests
You can run an end-to-end test which will bring up a master and two nodes, perform some tests, and then tear everything down. Make sure you have followed the getting started steps for your chosen cloud platform (which might involve changing the `KUBERNETES_PROVIDER` environment variable to something other than "gce".
{% highlight sh %}
{% raw %}
cd kubernetes
hack/e2e-test.sh
{% endraw %}
{% endhighlight %}
Pressing control-C should result in an orderly shutdown but if something goes wrong and you still have some VMs running you can force a cleanup with this command:
{% highlight sh %}
{% raw %}
go run hack/e2e.go --down
{% endraw %}
{% endhighlight %}
### Flag options
See the flag definitions in `hack/e2e.go` for more options, such as reusing an existing cluster, here is an overview:
{% highlight sh %}
{% raw %}
# Build binaries for testing
go run hack/e2e.go --build
# Create a fresh cluster. Deletes a cluster first, if it exists
go run hack/e2e.go --up
# Create a fresh cluster at a specific release version.
go run hack/e2e.go --up --version=0.7.0
# Test if a cluster is up.
go run hack/e2e.go --isup
# Push code to an existing cluster
go run hack/e2e.go --push
# Push to an existing cluster, or bring up a cluster if it's down.
go run hack/e2e.go --pushup
# Run all tests
go run hack/e2e.go --test
# Run tests matching the regex "Pods.*env"
go run hack/e2e.go -v -test --test_args="--ginkgo.focus=Pods.*env"
# Alternately, if you have the e2e cluster up and no desire to see the event stream, you can run ginkgo-e2e.sh directly:
hack/ginkgo-e2e.sh --ginkgo.focus=Pods.*env
{% endraw %}
{% endhighlight %}
### Combining flags
{% highlight sh %}
{% raw %}
# Flags can be combined, and their actions will take place in this order:
# -build, -push|-up|-pushup, -test|-tests=..., -down
# e.g.:
go run hack/e2e.go -build -pushup -test -down
# -v (verbose) can be added if you want streaming output instead of only
# seeing the output of failed commands.
# -ctl can be used to quickly call kubectl against your e2e cluster. Useful for
# cleaning up after a failed test or viewing logs. Use -v to avoid suppressing
# kubectl output.
go run hack/e2e.go -v -ctl='get events'
go run hack/e2e.go -v -ctl='delete pod foobar'
{% endraw %}
{% endhighlight %}
## Conformance testing
End-to-end testing, as described above, is for [development
distributions](writing-a-getting-started-guide.html). A conformance test is used on
a [versioned distro](writing-a-getting-started-guide.html).
The conformance test runs a subset of the e2e-tests against a manually-created cluster. It does not
require support for up/push/down and other operations. To run a conformance test, you need to know the
IP of the master for your cluster and the authorization arguments to use. The conformance test is
intended to run against a cluster at a specific binary release of Kubernetes.
See [conformance-test.sh](http://releases.k8s.io/release-1.1/hack/conformance-test.sh).
## Testing out flaky tests
[Instructions here](flaky-tests.html)
## Regenerating the CLI documentation
{% highlight sh %}
{% raw %}
hack/update-generated-docs.sh
{% endraw %}
{% endhighlight %}
<!-- BEGIN MUNGE: IS_VERSIONED -->
<!-- TAG IS_VERSIONED -->
<!-- END MUNGE: IS_VERSIONED -->
<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/devel/development.md?pixel)]()
<!-- END MUNGE: GENERATED_ANALYTICS -->
v1.1/docs/devel/e2e-tests.md
+135
View File
@@ -0,0 +1,135 @@
---
layout: docwithnav
title: "End-2-End Testing in Kubernetes"
---
<!-- BEGIN MUNGE: UNVERSIONED_WARNING -->
<!-- END MUNGE: UNVERSIONED_WARNING -->
# End-2-End Testing in Kubernetes
## Overview
The end-2-end tests for kubernetes provide a mechanism to test behavior of the system, and to ensure end user operations match developer specifications. In distributed systems it is not uncommon that a minor change may pass all unit tests, but cause unforseen changes at the system level. Thus, the primary objectives of the end-2-end tests are to ensure a consistent and reliable behavior of the kubernetes code base, and to catch bugs early.
The end-2-end tests in kubernetes are built atop of [ginkgo] (http://onsi.github.io/ginkgo/) and [gomega] (http://onsi.github.io/gomega/). There are a host of features that this BDD testing framework provides, and it is recommended that the developer read the documentation prior to diving into the tests.
The purpose of *this* document is to serve as a primer for developers who are looking to execute, or add tests, using a local development environment.
## Building and Running the Tests
**NOTE:** The tests have an array of options. For simplicity, the examples will focus on leveraging the tests on a local cluster using `sudo ./hack/local-up-cluster.sh`
### Building the Tests
The tests are built into a single binary which can be run against any deployed kubernetes system. To build the tests, navigate to your source directory and execute:
`$ make all`
The output for the end-2-end tests will be a single binary called `e2e.test` under the default output directory, which is typically `_output/local/bin/linux/amd64/`. Within the repository there are scripts that are provided under the `./hack` directory that are helpful for automation, but may not apply for a local development purposes. Instead, we recommend familiarizing yourself with the executable options. To obtain the full list of options, run the following:
`$ ./e2e.test --help`
### Running the Tests
For the purposes of brevity, we will look at a subset of the options, which are listed below:
```
{% raw %}
-ginkgo.dryRun=false: If set, ginkgo will walk the test hierarchy without actually running anything. Best paired with -v.
-ginkgo.failFast=false: If set, ginkgo will stop running a test suite after a failure occurs.
-ginkgo.failOnPending=false: If set, ginkgo will mark the test suite as failed if any specs are pending.
-ginkgo.focus="": If set, ginkgo will only run specs that match this regular expression.
-ginkgo.skip="": If set, ginkgo will only run specs that do not match this regular expression.
-ginkgo.trace=false: If set, default reporter prints out the full stack trace when a failure occurs
-ginkgo.v=false: If set, default reporter print out all specs as they begin.
-host="": The host, or api-server, to connect to
-kubeconfig="": Path to kubeconfig containing embedded authinfo.
-prom-push-gateway="": The URL to prometheus gateway, so that metrics can be pushed during e2es and scraped by prometheus. Typically something like 127.0.0.1:9091.
-provider="": The name of the Kubernetes provider (gce, gke, local, vagrant, etc.)
-repo-root="../../": Root directory of kubernetes repository, for finding test files.
{% endraw %}
```
Prior to running the tests, it is recommended that you first create a simple auth file in your home directory, e.g. `$HOME/.kubernetes_auth` , with the following:
```
{% raw %}
{
"User": "root",
"Password": ""
}
{% endraw %}
```
Next, you will need a cluster that you can test against. As mentioned earlier, you will want to execute `sudo ./hack/local-up-cluster.sh`. To get a sense of what tests exist, you may want to run:
`e2e.test --host="127.0.0.1:8080" --provider="local" --ginkgo.v=true -ginkgo.dryRun=true --kubeconfig="$HOME/.kubernetes_auth" --repo-root="$KUBERNETES_SRC_PATH"`
If you wish to execute a specific set of tests you can use the `-ginkgo.focus=` regex, e.g.:
`e2e.test ... --ginkgo.focus="DNS|(?i)nodeport(?-i)|kubectl guestbook"`
Conversely, if you wish to exclude a set of tests, you can run:
`e2e.test ... --ginkgo.skip="Density|Scale"`
As mentioned earlier there are a host of other options that are available, but are left to the developer
**NOTE:** If you are running tests on a local cluster repeatedly, you may need to periodically perform some manual cleanup.
- `rm -rf /var/run/kubernetes`, clear kube generated credentials, sometimes stale permissions can cause problems.
- `sudo iptables -F`, clear ip tables rules left by the kube-proxy.
## Adding a New Test
As mentioned above, prior to adding a new test, it is a good idea to perform a `-ginkgo.dryRun=true` on the system, in order to see if a behavior is already being tested, or to determine if it may be possible to augment an existing set of tests for a specific use case.
If a behavior does not currently have coverage and a developer wishes to add a new e2e test, navigate to the ./test/e2e directory and create a new test using the existing suite as a guide.
**TODO:** Create a self-documented example which has been disabled, but can be copied to create new tests and outlines the capabilities and libraries used.
## Performance Evaluation
Another benefit of the end-2-end tests is the ability to create reproducible loads on the system, which can then be used to determine the responsiveness, or analyze other characteristics of the system. For example, the density tests load the system to 30,50,100 pods per/node and measures the different characteristics of the system, such as throughput, api-latency, etc.
For a good overview of how we analyze performance data, please read the following [post](http://blog.kubernetes.io/2015/09/kubernetes-performance-measurements-and.html)
For developers who are interested in doing their own performance analysis, we recommend setting up [prometheus](http://prometheus.io/) for data collection, and using [promdash](http://prometheus.io/docs/visualization/promdash/) to visualize the data. There also exists the option of pushing your own metrics in from the tests using a [prom-push-gateway](http://prometheus.io/docs/instrumenting/pushing/). Containers for all of these components can be found [here](https://hub.docker.com/u/prom/).
For more accurate measurements, you may wish to set up prometheus external to kubernetes in an environment where it can access the major system components (api-server, controller-manager, scheduler). This is especially useful when attempting to gather metrics in a load-balanced api-server environment, because all api-servers can be analyzed independently as well as collectively. On startup, configuration file is passed to prometheus that specifies the endpoints that prometheus will scrape, as well as the sampling interval.
```
{% raw %}
#prometheus.conf
job: {
name: "kubernetes"
scrape_interval: "1s"
target_group: {
# apiserver(s)
target: "http://localhost:8080/metrics"
# scheduler
target: "http://localhost:10251/metrics"
# controller-manager
target: "http://localhost:10252/metrics"
}
{% endraw %}
```
Once prometheus is scraping the kubernetes endpoints, that data can then be plotted using promdash, and alerts can be created against the assortment of metrics that kubernetes provides.
**HAPPY TESTING!**
<!-- BEGIN MUNGE: IS_VERSIONED -->
<!-- TAG IS_VERSIONED -->
<!-- END MUNGE: IS_VERSIONED -->
<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/devel/e2e-tests.md?pixel)]()
<!-- END MUNGE: GENERATED_ANALYTICS -->
v1.1/docs/devel/faster_reviews.md
+223
View File
@@ -0,0 +1,223 @@
---
layout: docwithnav
title: "How to get faster PR reviews"
---
<!-- BEGIN MUNGE: UNVERSIONED_WARNING -->
<!-- END MUNGE: UNVERSIONED_WARNING -->
# How to get faster PR reviews
Most of what is written here is not at all specific to Kubernetes, but it bears
being written down in the hope that it will occasionally remind people of "best
practices" around code reviews.
You've just had a brilliant idea on how to make Kubernetes better. Let's call
that idea "FeatureX". Feature X is not even that complicated. You have a
pretty good idea of how to implement it. You jump in and implement it, fixing a
bunch of stuff along the way. You send your PR - this is awesome! And it sits.
And sits. A week goes by and nobody reviews it. Finally someone offers a few
comments, which you fix up and wait for more review. And you wait. Another
week or two goes by. This is horrible.
What went wrong? One particular problem that comes up frequently is this - your
PR is too big to review. You've touched 39 files and have 8657 insertions.
When your would-be reviewers pull up the diffs they run away - this PR is going
to take 4 hours to review and they don't have 4 hours right now. They'll get to it
later, just as soon as they have more free time (ha!).
Let's talk about how to avoid this.
## 0. Familiarize yourself with project conventions
* [Development guide](development.html)
* [Coding conventions](coding-conventions.html)
* [API conventions](api-conventions.html)
* [Kubectl conventions](kubectl-conventions.html)
## 1. Don't build a cathedral in one PR
Are you sure FeatureX is something the Kubernetes team wants or will accept, or
that it is implemented to fit with other changes in flight? Are you willing to
bet a few days or weeks of work on it? If you have any doubt at all about the
usefulness of your feature or the design - make a proposal doc (in docs/proposals;
for example [the QoS proposal](http://prs.k8s.io/11713)) or a sketch PR (e.g., just
the API or Go interface) or both. Write or code up just enough to express the idea
and the design and why you made those choices, then get feedback on this. Be clear
about what type of feedback you are asking for. Now, if we ask you to change a
bunch of facets of the design, you won't have to re-write it all.
## 2. Smaller diffs are exponentially better
Small PRs get reviewed faster and are more likely to be correct than big ones.
Let's face it - attention wanes over time. If your PR takes 60 minutes to
review, I almost guarantee that the reviewer's eye for details is not as keen in
the last 30 minutes as it was in the first. This leads to multiple rounds of
review when one might have sufficed. In some cases the review is delayed in its
entirety by the need for a large contiguous block of time to sit and read your
code.
Whenever possible, break up your PRs into multiple commits. Making a series of
discrete commits is a powerful way to express the evolution of an idea or the
different ideas that make up a single feature. There's a balance to be struck,
obviously. If your commits are too small they become more cumbersome to deal
with. Strive to group logically distinct ideas into commits.
For example, if you found that FeatureX needed some "prefactoring" to fit in,
make a commit that JUST does that prefactoring. Then make a new commit for
FeatureX. Don't lump unrelated things together just because you didn't think
about prefactoring. If you need to, fork a new branch, do the prefactoring
there and send a PR for that. If you can explain why you are doing seemingly
no-op work ("it makes the FeatureX change easier, I promise") we'll probably be
OK with it.
Obviously, a PR with 25 commits is still very cumbersome to review, so use
common sense.
## 3. Multiple small PRs are often better than multiple commits
If you can extract whole ideas from your PR and send those as PRs of their own,
you can avoid the painful problem of continually rebasing. Kubernetes is a
fast-moving codebase - lock in your changes ASAP, and make merges be someone
else's problem.
Obviously, we want every PR to be useful on its own, so you'll have to use
common sense in deciding what can be a PR vs what should be a commit in a larger
PR. Rule of thumb - if this commit or set of commits is directly related to
FeatureX and nothing else, it should probably be part of the FeatureX PR. If
you can plausibly imagine someone finding value in this commit outside of
FeatureX, try it as a PR.
Don't worry about flooding us with PRs. We'd rather have 100 small, obvious PRs
than 10 unreviewable monoliths.
## 4. Don't rename, reformat, comment, etc in the same PR
Often, as you are implementing FeatureX, you find things that are just wrong.
Bad comments, poorly named functions, bad structure, weak type-safety. You
should absolutely fix those things (or at least file issues, please) - but not
in this PR. See the above points - break unrelated changes out into different
PRs or commits. Otherwise your diff will have WAY too many changes, and your
reviewer won't see the forest because of all the trees.
## 5. Comments matter
Read up on GoDoc - follow those general rules. If you're writing code and you
think there is any possible chance that someone might not understand why you did
something (or that you won't remember what you yourself did), comment it. If
you think there's something pretty obvious that we could follow up on, add a
TODO. Many code-review comments are about this exact issue.
## 5. Tests are almost always required
Nothing is more frustrating than doing a review, only to find that the tests are
inadequate or even entirely absent. Very few PRs can touch code and NOT touch
tests. If you don't know how to test FeatureX - ask! We'll be happy to help
you design things for easy testing or to suggest appropriate test cases.
## 6. Look for opportunities to generify
If you find yourself writing something that touches a lot of modules, think hard
about the dependencies you are introducing between packages. Can some of what
you're doing be made more generic and moved up and out of the FeatureX package?
Do you need to use a function or type from an otherwise unrelated package? If
so, promote! We have places specifically for hosting more generic code.
Likewise if FeatureX is similar in form to FeatureW which was checked in last
month and it happens to exactly duplicate some tricky stuff from FeatureW,
consider prefactoring core logic out and using it in both FeatureW and FeatureX.
But do that in a different commit or PR, please.
## 7. Fix feedback in a new commit
Your reviewer has finally sent you some feedback on FeatureX. You make a bunch
of changes and ... what? You could patch those into your commits with git
"squash" or "fixup" logic. But that makes your changes hard to verify. Unless
your whole PR is pretty trivial, you should instead put your fixups into a new
commit and re-push. Your reviewer can then look at that commit on its own - so
much faster to review than starting over.
We might still ask you to clean up your commits at the very end, for the sake
of a more readable history, but don't do this until asked, typically at the point
where the PR would otherwise be tagged LGTM.
General squashing guidelines:
* Sausage => squash
When there are several commits to fix bugs in the original commit(s), address reviewer feedback, etc. Really we only want to see the end state and commit message for the whole PR.
* Layers => don't squash
When there are independent changes layered upon each other to achieve a single goal. For instance, writing a code munger could be one commit, applying it could be another, and adding a precommit check could be a third. One could argue they should be separate PRs, but there's really no way to test/review the munger without seeing it applied, and there needs to be a precommit check to ensure the munged output doesn't immediately get out of date.
A commit, as much as possible, should be a single logical change. Each commit should always have a good title line (<70 characters) and include an additional description paragraph describing in more detail the change intended. Do not link pull requests by `#` in a commit description, because GitHub creates lots of spam. Instead, reference other PRs via the PR your commit is in.
## 8. KISS, YAGNI, MVP, etc
Sometimes we need to remind each other of core tenets of software design - Keep
It Simple, You Aren't Gonna Need It, Minimum Viable Product, and so on. Adding
features "because we might need it later" is antithetical to software that
ships. Add the things you need NOW and (ideally) leave room for things you
might need later - but don't implement them now.
## 9. Push back
We understand that it is hard to imagine, but sometimes we make mistakes. It's
OK to push back on changes requested during a review. If you have a good reason
for doing something a certain way, you are absolutely allowed to debate the
merits of a requested change. You might be overruled, but you might also
prevail. We're mostly pretty reasonable people. Mostly.
## 10. I'm still getting stalled - help?!
So, you've done all that and you still aren't getting any PR love? Here's some
things you can do that might help kick a stalled process along:
* Make sure that your PR has an assigned reviewer (assignee in GitHub). If
this is not the case, reply to the PR comment stream asking for one to be
assigned.
* Ping the assignee (@username) on the PR comment stream asking for an
estimate of when they can get to it.
* Ping the assignee by email (many of us have email addresses that are well
published or are the same as our GitHub handle @google.com or @redhat.com).
* Ping the [team](https://github.com/orgs/kubernetes/teams) (via @team-name)
that works in the area you're submitting code.
If you think you have fixed all the issues in a round of review, and you haven't
heard back, you should ping the reviewer (assignee) on the comment stream with a
"please take another look" (PTAL) or similar comment indicating you are done and
you think it is ready for re-review. In fact, this is probably a good habit for
all PRs.
One phenomenon of open-source projects (where anyone can comment on any issue)
is the dog-pile - your PR gets so many comments from so many people it becomes
hard to follow. In this situation you can ask the primary reviewer
(assignee) whether they want you to fork a new PR to clear out all the comments.
Remember: you don't HAVE to fix every issue raised by every person who feels
like commenting, but you should at least answer reasonable comments with an
explanation.
## Final: Use common sense
Obviously, none of these points are hard rules. There is no document that can
take the place of common sense and good taste. Use your best judgment, but put
a bit of thought into how your work can be made easier to review. If you do
these things your PRs will flow much more easily.
<!-- BEGIN MUNGE: IS_VERSIONED -->
<!-- TAG IS_VERSIONED -->
<!-- END MUNGE: IS_VERSIONED -->
<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/devel/faster_reviews.md?pixel)]()
<!-- END MUNGE: GENERATED_ANALYTICS -->
v1.1/docs/devel/flaky-tests.md
+95
View File
@@ -0,0 +1,95 @@
---
layout: docwithnav
title: "Hunting flaky tests in Kubernetes"
---
<!-- BEGIN MUNGE: UNVERSIONED_WARNING -->
<!-- END MUNGE: UNVERSIONED_WARNING -->
# Hunting flaky tests in Kubernetes
Sometimes unit tests are flaky. This means that due to (usually) race conditions, they will occasionally fail, even though most of the time they pass.
We have a goal of 99.9% flake free tests. This means that there is only one flake in one thousand runs of a test.
Running a test 1000 times on your own machine can be tedious and time consuming. Fortunately, there is a better way to achieve this using Kubernetes.
_Note: these instructions are mildly hacky for now, as we get run once semantics and logging they will get better_
There is a testing image `brendanburns/flake` up on the docker hub. We will use this image to test our fix.
Create a replication controller with the following config:
{% highlight yaml %}
{% raw %}
apiVersion: v1
kind: ReplicationController
metadata:
name: flakecontroller
spec:
replicas: 24
template:
metadata:
labels:
name: flake
spec:
containers:
- name: flake
image: brendanburns/flake
env:
- name: TEST_PACKAGE
value: pkg/tools
- name: REPO_SPEC
value: https://github.com/kubernetes/kubernetes
{% endraw %}
{% endhighlight %}
Note that we omit the labels and the selector fields of the replication controller, because they will be populated from the labels field of the pod template by default.
{% highlight sh %}
{% raw %}
kubectl create -f ./controller.yaml
{% endraw %}
{% endhighlight %}
This will spin up 24 instances of the test. They will run to completion, then exit, and the kubelet will restart them, accumulating more and more runs of the test.
You can examine the recent runs of the test by calling `docker ps -a` and looking for tasks that exited with non-zero exit codes. Unfortunately, docker ps -a only keeps around the exit status of the last 15-20 containers with the same image, so you have to check them frequently.
You can use this script to automate checking for failures, assuming your cluster is running on GCE and has four nodes:
{% highlight sh %}
{% raw %}
echo "" > output.txt
for i in {1..4}; do
echo "Checking kubernetes-minion-${i}"
echo "kubernetes-minion-${i}:" >> output.txt
gcloud compute ssh "kubernetes-minion-${i}" --command="sudo docker ps -a" >> output.txt
done
grep "Exited ([^0])" output.txt
{% endraw %}
{% endhighlight %}
Eventually you will have sufficient runs for your purposes. At that point you can stop and delete the replication controller by running:
{% highlight sh %}
{% raw %}
kubectl stop replicationcontroller flakecontroller
{% endraw %}
{% endhighlight %}
If you do a final check for flakes with `docker ps -a`, ignore tasks that exited -1, since that's what happens when you stop the replication controller.
Happy flake hunting!
<!-- BEGIN MUNGE: IS_VERSIONED -->
<!-- TAG IS_VERSIONED -->
<!-- END MUNGE: IS_VERSIONED -->
<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/devel/flaky-tests.md?pixel)]()
<!-- END MUNGE: GENERATED_ANALYTICS -->
v1.1/docs/devel/getting-builds.md
+62
View File
@@ -0,0 +1,62 @@
---
layout: docwithnav
title: "Getting Kubernetes Builds"
---
<!-- BEGIN MUNGE: UNVERSIONED_WARNING -->
<!-- END MUNGE: UNVERSIONED_WARNING -->
# Getting Kubernetes Builds
You can use [hack/get-build.sh](http://releases.k8s.io/release-1.1/hack/get-build.sh) to or use as a reference on how to get the most recent builds with curl. With `get-build.sh` you can grab the most recent stable build, the most recent release candidate, or the most recent build to pass our ci and gce e2e tests (essentially a nightly build).
Run `./hack/get-build.sh -h` for its usage.
For example, to get a build at a specific version (v1.0.2):
{% highlight console %}
{% raw %}
./hack/get-build.sh v1.0.2
{% endraw %}
{% endhighlight %}
Alternatively, to get the latest stable release:
{% highlight console %}
{% raw %}
./hack/get-build.sh release/stable
{% endraw %}
{% endhighlight %}
Finally, you can just print the latest or stable version:
{% highlight console %}
{% raw %}
./hack/get-build.sh -v ci/latest
{% endraw %}
{% endhighlight %}
You can also use the gsutil tool to explore the Google Cloud Storage release buckets. Here are some examples:
{% highlight sh %}
{% raw %}
gsutil cat gs://kubernetes-release/ci/latest.txt # output the latest ci version number
gsutil cat gs://kubernetes-release/ci/latest-green.txt # output the latest ci version number that passed gce e2e
gsutil ls gs://kubernetes-release/ci/v0.20.0-29-g29a55cc/ # list the contents of a ci release
gsutil ls gs://kubernetes-release/release # list all official releases and rcs
{% endraw %}
{% endhighlight %}
<!-- BEGIN MUNGE: IS_VERSIONED -->
<!-- TAG IS_VERSIONED -->
<!-- END MUNGE: IS_VERSIONED -->
<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/devel/getting-builds.md?pixel)]()
<!-- END MUNGE: GENERATED_ANALYTICS -->
v1.1/docs/devel/git_workflow.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 112 KiB

v1.1/docs/devel/index.md
+96
View File
@@ -0,0 +1,96 @@
---
layout: docwithnav
title: "Kubernetes Developer Guide"
---
<!-- BEGIN MUNGE: UNVERSIONED_WARNING -->
<!-- END MUNGE: UNVERSIONED_WARNING -->
# Kubernetes Developer Guide
The developer guide is for anyone wanting to either write code which directly accesses the
Kubernetes API, or to contribute directly to the Kubernetes project.
It assumes some familiarity with concepts in the [User Guide](../user-guide/README.html) and the [Cluster Admin
Guide](../admin/README.html).
## The process of developing and contributing code to the Kubernetes project
* **On Collaborative Development** ([collab.md](collab.html)): Info on pull requests and code reviews.
* **GitHub Issues** ([issues.md](issues.html)): How incoming issues are reviewed and prioritized.
* **Pull Request Process** ([pull-requests.md](pull-requests.html)): When and why pull requests are closed.
* **Faster PR reviews** ([faster_reviews.md](faster_reviews.html)): How to get faster PR reviews.
* **Getting Recent Builds** ([getting-builds.md](getting-builds.html)): How to get recent builds including the latest builds that pass CI.
* **Automated Tools** ([automation.md](automation.html)): Descriptions of the automation that is running on our github repository.
## Setting up your dev environment, coding, and debugging
* **Development Guide** ([development.md](development.html)): Setting up your development environment.
* **Hunting flaky tests** ([flaky-tests.md](flaky-tests.html)): We have a goal of 99.9% flake free tests.
Here's how to run your tests many times.
* **Logging Conventions** ([logging.md](logging.html)]: Glog levels.
* **Profiling Kubernetes** ([profiling.md](profiling.html)): How to plug in go pprof profiler to Kubernetes.
* **Instrumenting Kubernetes with a new metric**
([instrumentation.md](instrumentation.html)): How to add a new metrics to the
Kubernetes code base.
* **Coding Conventions** ([coding-conventions.md](coding-conventions.html)):
Coding style advice for contributors.
## Developing against the Kubernetes API
* API objects are explained at [http://kubernetes.io/third_party/swagger-ui/](http://kubernetes.io/third_party/swagger-ui/).
* **Annotations** ([docs/user-guide/annotations.md](../user-guide/annotations.html)): are for attaching arbitrary non-identifying metadata to objects.
Programs that automate Kubernetes objects may use annotations to store small amounts of their state.
* **API Conventions** ([api-conventions.md](api-conventions.html)):
Defining the verbs and resources used in the Kubernetes API.
* **API Client Libraries** ([client-libraries.md](client-libraries.html)):
A list of existing client libraries, both supported and user-contributed.
## Writing plugins
* **Authentication Plugins** ([docs/admin/authentication.md](../admin/authentication.html)):
The current and planned states of authentication tokens.
* **Authorization Plugins** ([docs/admin/authorization.md](../admin/authorization.html)):
Authorization applies to all HTTP requests on the main apiserver port.
This doc explains the available authorization implementations.
* **Admission Control Plugins** ([admission_control](../design/admission_control.html))
## Building releases
* **Making release notes** ([making-release-notes.md](making-release-notes.html)): Generating release nodes for a new release.
* **Releasing Kubernetes** ([releasing.md](releasing.html)): How to create a Kubernetes release (as in version)
and how the version information gets embedded into the built binaries.
<!-- BEGIN MUNGE: IS_VERSIONED -->
<!-- TAG IS_VERSIONED -->
<!-- END MUNGE: IS_VERSIONED -->
<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/devel/README.md?pixel)]()
<!-- END MUNGE: GENERATED_ANALYTICS -->
v1.1/docs/devel/instrumentation.md
+54
View File
@@ -0,0 +1,54 @@
---
layout: docwithnav
title: "Instrumenting Kubernetes with a new metric"
---
<!-- BEGIN MUNGE: UNVERSIONED_WARNING -->
<!-- END MUNGE: UNVERSIONED_WARNING -->
Instrumenting Kubernetes with a new metric
===================
The following is a step-by-step guide for adding a new metric to the Kubernetes code base.
We use the Prometheus monitoring system's golang client library for instrumenting our code. Once you've picked out a file that you want to add a metric to, you should:
1. Import "github.com/prometheus/client_golang/prometheus".
2. Create a top-level var to define the metric. For this, you have to:
1. Pick the type of metric. Use a Gauge for things you want to set to a particular value, a Counter for things you want to increment, or a Histogram or Summary for histograms/distributions of values (typically for latency). Histograms are better if you're going to aggregate the values across jobs, while summaries are better if you just want the job to give you a useful summary of the values.
2. Give the metric a name and description.
3. Pick whether you want to distinguish different categories of things using labels on the metric. If so, add "Vec" to the name of the type of metric you want and add a slice of the label names to the definition.
https://github.com/kubernetes/kubernetes/blob/cd3299307d44665564e1a5c77d0daa0286603ff5/pkg/apiserver/apiserver.go#L53
https://github.com/kubernetes/kubernetes/blob/cd3299307d44665564e1a5c77d0daa0286603ff5/pkg/kubelet/metrics/metrics.go#L31
3. Register the metric so that prometheus will know to export it.
https://github.com/kubernetes/kubernetes/blob/cd3299307d44665564e1a5c77d0daa0286603ff5/pkg/kubelet/metrics/metrics.go#L74
https://github.com/kubernetes/kubernetes/blob/cd3299307d44665564e1a5c77d0daa0286603ff5/pkg/apiserver/apiserver.go#L78
4. Use the metric by calling the appropriate method for your metric type (Set, Inc/Add, or Observe, respectively for Gauge, Counter, or Histogram/Summary), first calling WithLabelValues if your metric has any labels
https://github.com/kubernetes/kubernetes/blob/3ce7fe8310ff081dbbd3d95490193e1d5250d2c9/pkg/kubelet/kubelet.go#L1384
https://github.com/kubernetes/kubernetes/blob/cd3299307d44665564e1a5c77d0daa0286603ff5/pkg/apiserver/apiserver.go#L87
These are the metric type definitions if you're curious to learn about them or need more information:
https://github.com/prometheus/client_golang/blob/master/prometheus/gauge.go
https://github.com/prometheus/client_golang/blob/master/prometheus/counter.go
https://github.com/prometheus/client_golang/blob/master/prometheus/histogram.go
https://github.com/prometheus/client_golang/blob/master/prometheus/summary.go
<!-- BEGIN MUNGE: IS_VERSIONED -->
<!-- TAG IS_VERSIONED -->
<!-- END MUNGE: IS_VERSIONED -->
<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/devel/instrumentation.md?pixel)]()
<!-- END MUNGE: GENERATED_ANALYTICS -->
v1.1/docs/devel/issues.md
+40
View File
@@ -0,0 +1,40 @@
---
layout: docwithnav
title: "GitHub Issues for the Kubernetes Project"
---
<!-- BEGIN MUNGE: UNVERSIONED_WARNING -->
<!-- END MUNGE: UNVERSIONED_WARNING -->
GitHub Issues for the Kubernetes Project
========================================
A list quick overview of how we will review and prioritize incoming issues at https://github.com/kubernetes/kubernetes/issues
Priorities
----------
We will use GitHub issue labels for prioritization. The absence of a priority label means the bug has not been reviewed and prioritized yet.
Definitions
-----------
* P0 - something broken for users, build broken, or critical security issue. Someone must drop everything and work on it.
* P1 - must fix for earliest possible binary release (every two weeks)
* P2 - should be fixed in next major release version
* P3 - default priority for lower importance bugs that we still want to track and plan to fix at some point
* design - priority/design is for issues that are used to track design discussions
* support - priority/support is used for issues tracking user support requests
* untriaged - anything without a priority/X label will be considered untriaged
<!-- BEGIN MUNGE: IS_VERSIONED -->
<!-- TAG IS_VERSIONED -->
<!-- END MUNGE: IS_VERSIONED -->
<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/devel/issues.md?pixel)]()
<!-- END MUNGE: GENERATED_ANALYTICS -->
v1.1/docs/devel/kubectl-conventions.md
+124
View File
@@ -0,0 +1,124 @@
---
layout: docwithnav
title: "Kubectl Conventions"
---
<!-- BEGIN MUNGE: UNVERSIONED_WARNING -->
<!-- END MUNGE: UNVERSIONED_WARNING -->
Kubectl Conventions
===================
Updated: 8/27/2015
**Table of Contents**
<!-- BEGIN MUNGE: GENERATED_TOC -->
- [Principles](#principles)
- [Command conventions](#command-conventions)
- [Flag conventions](#flag-conventions)
- [Output conventions](#output-conventions)
- [Documentation conventions](#documentation-conventions)
<!-- END MUNGE: GENERATED_TOC -->
## Principles
* Strive for consistency across commands
* Explicit should always override implicit
* Environment variables should override default values
* Command-line flags should override default values and environment variables
* --namespace should also override the value specified in a specified resource
## Command conventions
* Command names are all lowercase, and hyphenated if multiple words.
* kubectl VERB NOUNs for commands that apply to multiple resource types
* NOUNs may be specified as TYPE name1 name2 ... or TYPE/name1 TYPE/name2; TYPE is omitted when only a single type is expected
* Resource types are all lowercase, with no hyphens; both singular and plural forms are accepted
* NOUNs may also be specified by one or more file arguments: -f file1 -f file2 ...
* Resource types may have 2- or 3-letter aliases.
* Business logic should be decoupled from the command framework, so that it can be reused independently of kubectl, cobra, etc.
* Ideally, commonly needed functionality would be implemented server-side in order to avoid problems typical of "fat" clients and to make it readily available to non-Go clients
* Commands that generate resources, such as `run` or `expose`, should obey the following conventions:
* Flags should be converted to a parameter Go map or json map prior to invoking the generator
* The generator must be versioned so that users depending on a specific behavior may pin to that version, via `--generator=`
* Generation should be decoupled from creation
* `--dry-run` should output the resource that would be created, without creating it
* A command group (e.g., `kubectl config`) may be used to group related non-standard commands, such as custom generators, mutations, and computations
## Flag conventions
* Flags are all lowercase, with words separated by hyphens
* Flag names and single-character aliases should have the same meaning across all commands
* Command-line flags corresponding to API fields should accept API enums exactly (e.g., --restart=Always)
* Do not reuse flags for different semantic purposes, and do not use different flag names for the same semantic purpose -- grep for `"Flags()"` before adding a new flag
* Use short flags sparingly, only for the most frequently used options, prefer lowercase over uppercase for the most common cases, try to stick to well known conventions for UNIX commands and/or Docker, where they exist, and update this list when adding new short flags
* `-f`: Resource file
* also used for `--follow` in `logs`, but should be deprecated in favor of `-F`
* `-l`: Label selector
* also used for `--labels` in `expose`, but should be deprecated
* `-L`: Label columns
* `-c`: Container
* also used for `--client` in `version`, but should be deprecated
* `-i`: Attach stdin
* `-t`: Allocate TTY
* also used for `--template`, but deprecated
* `-w`: Watch (currently also used for `--www` in `proxy`, but should be deprecated)
* `-p`: Previous
* also used for `--pod` in `exec`, but deprecated
* also used for `--patch` in `patch`, but should be deprecated
* also used for `--port` in `proxy`, but should be deprecated
* `-P`: Static file prefix in `proxy`, but should be deprecated
* `-r`: Replicas
* `-u`: Unix socket
* `-v`: Verbose logging level
* `--dry-run`: Don't modify the live state; simulate the mutation and display the output
* `--local`: Don't contact the server; just do local read, transformation, generation, etc. and display the output
* `--output-version=...`: Convert the output to a different API group/version
* `--validate`: Validate the resource schema
## Output conventions
* By default, output is intended for humans rather than programs
* However, affordances are made for simple parsing of `get` output
* Only errors should be directed to stderr
* `get` commands should output one row per resource, and one resource per row
* Column titles and values should not contain spaces in order to facilitate commands that break lines into fields: cut, awk, etc.
* By default, `get` output should fit within about 80 columns
* Eventually we could perhaps auto-detect width
* `-o wide` may be used to display additional columns
* The first column should be the resource name, titled `NAME` (may change this to an abbreviation of resource type)
* NAMESPACE should be displayed as the first column when --all-namespaces is specified
* The last default column should be time since creation, titled `AGE`
* `-Lkey` should append a column containing the value of label with key `key`, with `<none>` if not present
* json, yaml, Go template, and jsonpath template formats should be supported and encouraged for subsequent processing
* Users should use --api-version or --output-version to ensure the output uses the version they expect
* `describe` commands may output on multiple lines and may include information from related resources, such as events. Describe should add additional information from related resources that a normal user may need to know - if a user would always run "describe resource1" and the immediately want to run a "get type2" or "describe resource2", consider including that info. Examples, persistent volume claims for pods that reference claims, events for most resources, nodes and the pods scheduled on them. When fetching related resources, a targeted field selector should be used in favor of client side filtering of related resources.
* Mutations should output TYPE/name verbed by default, where TYPE is singular; `-o name` may be used to just display TYPE/name, which may be used to specify resources in other commands
## Documentation conventions
* Commands are documented using Cobra; docs are then auto-generated by `hack/update-generated-docs.sh`.
* Use should contain a short usage string for the most common use case(s), not an exhaustive specification
* Short should contain a one-line explanation of what the command does
* Long may contain multiple lines, including additional information about input, output, commonly used flags, etc.
* Example should contain examples
* Start commands with `$`
* A comment should precede each example command, and should begin with `#`
* Use "FILENAME" for filenames
* Use "TYPE" for the particular flavor of resource type accepted by kubectl, rather than "RESOURCE" or "KIND"
* Use "NAME" for resource names
<!-- BEGIN MUNGE: IS_VERSIONED -->
<!-- TAG IS_VERSIONED -->
<!-- END MUNGE: IS_VERSIONED -->
<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/devel/kubectl-conventions.md?pixel)]()
<!-- END MUNGE: GENERATED_ANALYTICS -->
v1.1/docs/devel/logging.md
+47
View File
@@ -0,0 +1,47 @@
---
layout: docwithnav
title: "Logging Conventions"
---
<!-- BEGIN MUNGE: UNVERSIONED_WARNING -->
<!-- END MUNGE: UNVERSIONED_WARNING -->
Logging Conventions
===================
The following conventions for the glog levels to use. [glog](http://godoc.org/github.com/golang/glog) is globally preferred to [log](http://golang.org/pkg/log/) for better runtime control.
* glog.Errorf() - Always an error
* glog.Warningf() - Something unexpected, but probably not an error
* glog.Infof() has multiple levels:
* glog.V(0) - Generally useful for this to ALWAYS be visible to an operator
* Programmer errors
* Logging extra info about a panic
* CLI argument handling
* glog.V(1) - A reasonable default log level if you don't want verbosity.
* Information about config (listening on X, watching Y)
* Errors that repeat frequently that relate to conditions that can be corrected (pod detected as unhealthy)
* glog.V(2) - Useful steady state information about the service and important log messages that may correlate to significant changes in the system. This is the recommended default log level for most systems.
* Logging HTTP requests and their exit code
* System state changing (killing pod)
* Controller state change events (starting pods)
* Scheduler log messages
* glog.V(3) - Extended information about changes
* More info about system state changes
* glog.V(4) - Debug level verbosity (for now)
* Logging in particularly thorny parts of code where you may want to come back later and check it
As per the comments, the practical default level is V(2). Developers and QE environments may wish to run at V(3) or V(4). If you wish to change the log level, you can pass in `-v=X` where X is the desired maximum level to log.
<!-- BEGIN MUNGE: IS_VERSIONED -->
<!-- TAG IS_VERSIONED -->
<!-- END MUNGE: IS_VERSIONED -->
<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/devel/logging.md?pixel)]()
<!-- END MUNGE: GENERATED_ANALYTICS -->
v1.1/docs/devel/making-release-notes.md
+63
View File
@@ -0,0 +1,63 @@
---
layout: docwithnav
title: "Making release notes"
---
<!-- BEGIN MUNGE: UNVERSIONED_WARNING -->
<!-- END MUNGE: UNVERSIONED_WARNING -->
## Making release notes
This documents the process for making release notes for a release.
### 1) Note the PR number of the previous release
Find the most-recent PR that was merged with the previous .0 release. Remember this as $LASTPR.
_TODO_: Figure out a way to record this somewhere to save the next release engineer time.
Find the most-recent PR that was merged with the current .0 release. Remember this as $CURRENTPR.
### 2) Run the release-notes tool
{% highlight bash %}
{% raw %}
${KUBERNETES_ROOT}/build/make-release-notes.sh $LASTPR $CURRENTPR
{% endraw %}
{% endhighlight %}
### 3) Trim the release notes
This generates a list of the entire set of PRs merged since the last minor
release. It is likely long and many PRs aren't worth mentioning. If any of the
PRs were cherrypicked into patches on the last minor release, you should exclude
them from the current release's notes.
Open up `candidate-notes.md` in your favorite editor.
Remove, regroup, organize to your hearts content.
### 4) Update CHANGELOG.md
With the final markdown all set, cut and paste it to the top of `CHANGELOG.md`
### 5) Update the Release page
* Switch to the [releases](https://github.com/kubernetes/kubernetes/releases) page.
* Open up the release you are working on.
* Cut and paste the final markdown from above into the release notes
* Press Save.
<!-- BEGIN MUNGE: IS_VERSIONED -->
<!-- TAG IS_VERSIONED -->
<!-- END MUNGE: IS_VERSIONED -->
<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/devel/making-release-notes.md?pixel)]()
<!-- END MUNGE: GENERATED_ANALYTICS -->
v1.1/docs/devel/profiling.md
+69
View File
@@ -0,0 +1,69 @@
---
layout: docwithnav
title: "Profiling Kubernetes"
---
<!-- BEGIN MUNGE: UNVERSIONED_WARNING -->
<!-- END MUNGE: UNVERSIONED_WARNING -->
# Profiling Kubernetes
This document explain how to plug in profiler and how to profile Kubernetes services.
## Profiling library
Go comes with inbuilt 'net/http/pprof' profiling library and profiling web service. The way service works is binding debug/pprof/ subtree on a running webserver to the profiler. Reading from subpages of debug/pprof returns pprof-formatted profiles of the running binary. The output can be processed offline by the tool of choice, or used as an input to handy 'go tool pprof', which can graphically represent the result.
## Adding profiling to services to APIserver.
TL;DR: Add lines:
{% highlight go %}
{% raw %}
m.mux.HandleFunc("/debug/pprof/", pprof.Index)
m.mux.HandleFunc("/debug/pprof/profile", pprof.Profile)
m.mux.HandleFunc("/debug/pprof/symbol", pprof.Symbol)
{% endraw %}
{% endhighlight %}
to the init(c *Config) method in 'pkg/master/master.go' and import 'net/http/pprof' package.
In most use cases to use profiler service it's enough to do 'import _ net/http/pprof', which automatically registers a handler in the default http.Server. Slight inconvenience is that APIserver uses default server for intra-cluster communication, so plugging profiler to it is not really useful. In 'pkg/master/server/server.go' more servers are created and started as separate goroutines. The one that is usually serving external traffic is secureServer. The handler for this traffic is defined in 'pkg/master/master.go' and stored in Handler variable. It is created from HTTP multiplexer, so the only thing that needs to be done is adding profiler handler functions to this multiplexer. This is exactly what lines after TL;DR do.
## Connecting to the profiler
Even when running profiler I found not really straightforward to use 'go tool pprof' with it. The problem is that at least for dev purposes certificates generated for APIserver are not signed by anyone trusted and because secureServer serves only secure traffic it isn't straightforward to connect to the service. The best workaround I found is by creating an ssh tunnel from the kubernetes_master open unsecured port to some external server, and use this server as a proxy. To save everyone looking for correct ssh flags, it is done by running:
{% highlight sh %}
{% raw %}
ssh kubernetes_master -L<local_port>:localhost:8080
{% endraw %}
{% endhighlight %}
or analogous one for you Cloud provider. Afterwards you can e.g. run
{% highlight sh %}
{% raw %}
go tool pprof http://localhost:<local_port>/debug/pprof/profile
{% endraw %}
{% endhighlight %}
to get 30 sec. CPU profile.
## Contention profiling
To enable contention profiling you need to add line `rt.SetBlockProfileRate(1)` in addition to `m.mux.HandleFunc(...)` added before (`rt` stands for `runtime` in `master.go`). This enables 'debug/pprof/block' subpage, which can be used as an input to `go tool pprof`.
<!-- BEGIN MUNGE: IS_VERSIONED -->
<!-- TAG IS_VERSIONED -->
<!-- END MUNGE: IS_VERSIONED -->
<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/devel/profiling.md?pixel)]()
<!-- END MUNGE: GENERATED_ANALYTICS -->
v1.1/docs/devel/pull-requests.md
+57
View File
@@ -0,0 +1,57 @@
---
layout: docwithnav
title: "Pull Request Process"
---
<!-- BEGIN MUNGE: UNVERSIONED_WARNING -->
<!-- END MUNGE: UNVERSIONED_WARNING -->
Pull Request Process
====================
An overview of how we will manage old or out-of-date pull requests.
Process
-------
We will close any pull requests older than two weeks.
Exceptions can be made for PRs that have active review comments, or that are awaiting other dependent PRs. Closed pull requests are easy to recreate, and little work is lost by closing a pull request that subsequently needs to be reopened.
We want to limit the total number of PRs in flight to:
* Maintain a clean project
* Remove old PRs that would be difficult to rebase as the underlying code has changed over time
* Encourage code velocity
Life of a Pull Request
----------------------
Unless in the last few weeks of a milestone when we need to reduce churn and stabilize, we aim to be always accepting pull requests.
Either the [on call](https://github.com/kubernetes/kubernetes/wiki/Kubernetes-on-call-rotation) manually or the [submit queue](https://github.com/kubernetes/contrib/tree/master/submit-queue) automatically will manage merging PRs.
There are several requirements for the submit queue to work:
* Author must have signed CLA ("cla: yes" label added to PR)
* No changes can be made since last lgtm label was applied
* k8s-bot must have reported the GCE E2E build and test steps passed (Travis, Shippable and Jenkins build)
Additionally, for infrequent or new contributors, we require the on call to apply the "ok-to-merge" label manually. This is gated by the [whitelist](https://github.com/kubernetes/contrib/tree/master/submit-queue/whitelist.txt).
Automation
----------
We use a variety of automation to manage pull requests. This automation is described in detail
[elsewhere.](automation.html)
<!-- BEGIN MUNGE: IS_VERSIONED -->
<!-- TAG IS_VERSIONED -->
<!-- END MUNGE: IS_VERSIONED -->
<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/devel/pull-requests.md?pixel)]()
<!-- END MUNGE: GENERATED_ANALYTICS -->
v1.1/docs/devel/releasing.dot
+113
View File
@@ -0,0 +1,113 @@
// Build it with:
// $ dot -Tsvg releasing.dot >releasing.svg
digraph tagged_release {
size = "5,5"
// Arrows go up.
rankdir = BT
subgraph left {
// Group the left nodes together.
ci012abc -> pr101 -> ci345cde -> pr102
style = invis
}
subgraph right {
// Group the right nodes together.
version_commit -> dev_commit
style = invis
}
{ // Align the version commit and the info about it.
rank = same
// Align them with pr101
pr101
version_commit
// release_info shows the change in the commit.
release_info
}
{ // Align the dev commit and the info about it.
rank = same
// Align them with 345cde
ci345cde
dev_commit
dev_info
}
// Join the nodes from subgraph left.
pr99 -> ci012abc
pr102 -> pr100
// Do the version node.
pr99 -> version_commit
dev_commit -> pr100
tag -> version_commit
pr99 [
label = "Merge PR #99"
shape = box
fillcolor = "#ccccff"
style = "filled"
fontname = "Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif"
];
ci012abc [
label = "012abc"
shape = circle
fillcolor = "#ffffcc"
style = "filled"
fontname = "Consolas, Liberation Mono, Menlo, Courier, monospace"
];
pr101 [
label = "Merge PR #101"
shape = box
fillcolor = "#ccccff"
style = "filled"
fontname = "Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif"
];
ci345cde [
label = "345cde"
shape = circle
fillcolor = "#ffffcc"
style = "filled"
fontname = "Consolas, Liberation Mono, Menlo, Courier, monospace"
];
pr102 [
label = "Merge PR #102"
shape = box
fillcolor = "#ccccff"
style = "filled"
fontname = "Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif"
];
version_commit [
label = "678fed"
shape = circle
fillcolor = "#ccffcc"
style = "filled"
fontname = "Consolas, Liberation Mono, Menlo, Courier, monospace"
];
dev_commit [
label = "456dcb"
shape = circle
fillcolor = "#ffffcc"
style = "filled"
fontname = "Consolas, Liberation Mono, Menlo, Courier, monospace"
];
pr100 [
label = "Merge PR #100"
shape = box
fillcolor = "#ccccff"
style = "filled"
fontname = "Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif"
];
release_info [
label = "pkg/version/base.go:\ngitVersion = \"v0.5\";"
shape = none
fontname = "Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif"
];
dev_info [
label = "pkg/version/base.go:\ngitVersion = \"v0.5-dev\";"
shape = none
fontname = "Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif"
];
tag [
label = "$ git tag -a v0.5"
fillcolor = "#ffcccc"
style = "filled"
fontname = "Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif"
];
}
v1.1/docs/devel/releasing.md
+335
View File
@@ -0,0 +1,335 @@
---
layout: docwithnav
title: "Releasing Kubernetes"
---
<!-- BEGIN MUNGE: UNVERSIONED_WARNING -->
<!-- END MUNGE: UNVERSIONED_WARNING -->
# Releasing Kubernetes
This document explains how to cut a release, and the theory behind it. If you
just want to cut a release and move on with your life, you can stop reading
after the first section.
## How to cut a Kubernetes release
Regardless of whether you are cutting a major or minor version, cutting a
release breaks down into four pieces:
1. Selecting release components.
1. Tagging and merging the release in Git.
1. Building and pushing the binaries.
1. Writing release notes.
You should progress in this strict order.
### Building a New Major/Minor Version (`vX.Y.0`)
#### Selecting Release Components
When cutting a major/minor release, your first job is to find the branch
point. We cut `vX.Y.0` releases directly from `master`, which is also the
branch that we have most continuous validation on. Go first to [the main GCE
Jenkins end-to-end job](http://go/k8s-test/job/kubernetes-e2e-gce) and next to [the
Critical Builds page](http://go/k8s-test/view/Critical%20Builds) and hopefully find a
recent Git hash that looks stable across at least `kubernetes-e2e-gce` and
`kubernetes-e2e-gke-ci`. First glance through builds and look for nice solid
rows of green builds, and then check temporally with the other Critical Builds
to make sure they're solid around then as well. Once you find some greens, you
can find the Git hash for a build by looking at the "Console Log", then look for
`githash=`. You should see a line line:
{% highlight console %}
{% raw %}
+ githash=v0.20.2-322-g974377b
{% endraw %}
{% endhighlight %}
Because Jenkins builds frequently, if you're looking between jobs
(e.g. `kubernetes-e2e-gke-ci` and `kubernetes-e2e-gce`), there may be no single
`githash` that's been run on both jobs. In that case, take the a green
`kubernetes-e2e-gce` build (but please check that it corresponds to a temporally
similar build that's green on `kubernetes-e2e-gke-ci`). Lastly, if you're having
trouble understanding why the GKE continuous integration clusters are failing
and you're trying to cut a release, don't hesitate to contact the GKE
oncall.
Before proceeding to the next step:
{% highlight sh %}
{% raw %}
export BRANCHPOINT=v0.20.2-322-g974377b
{% endraw %}
{% endhighlight %}
Where `v0.20.2-322-g974377b` is the git hash you decided on. This will become
our (retroactive) branch point.
#### Branching, Tagging and Merging
Do the following:
1. `export VER=x.y` (e.g. `0.20` for v0.20)
1. cd to the base of the repo
1. `git fetch upstream && git checkout -b release-${VER} ${BRANCHPOINT}` (you did set `${BRANCHPOINT}`, right?)
1. Make sure you don't have any files you care about littering your repo (they
better be checked in or outside the repo, or the next step will delete them).
1. `make clean && git reset --hard HEAD && git clean -xdf`
1. `make` (TBD: you really shouldn't have to do this, but the swagger output step requires it right now)
1. `./build/mark-new-version.sh v${VER}.0` to mark the new release and get further
instructions. This creates a series of commits on the branch you're working
on (`release-${VER}`), including forking our documentation for the release,
the release version commit (which is then tagged), and the post-release
version commit.
1. Follow the instructions given to you by that script. They are canon for the
remainder of the Git process. If you don't understand something in that
process, please ask!
**TODO**: how to fix tags, etc., if you have to shift the release branchpoint.
#### Building and Pushing Binaries
In your git repo (you still have `${VER}` set from above right?):
1. `git checkout upstream/master && build/build-official-release.sh v${VER}.0` (the `build-official-release.sh` script is version agnostic, so it's best to run it off `master` directly).
1. Follow the instructions given to you by that script.
1. At this point, you've done all the Git bits, you've got all the binary bits pushed, and you've got the template for the release started on GitHub.
#### Writing Release Notes
[This helpful guide](making-release-notes.html) describes how to write release
notes for a major/minor release. In the release template on GitHub, leave the
last PR number that the tool finds for the `.0` release, so the next releaser
doesn't have to hunt.
### Building a New Patch Release (`vX.Y.Z` for `Z > 0`)
#### Selecting Release Components
We cut `vX.Y.Z` releases from the `release-vX.Y` branch after all cherry picks
to the branch have been resolved. You should ensure all outstanding cherry picks
have been reviewed and merged and the branch validated on Jenkins (validation
TBD). See the [Cherry Picks](cherry-picks.html) for more information on how to
manage cherry picks prior to cutting the release.
#### Tagging and Merging
1. `export VER=x.y` (e.g. `0.20` for v0.20)
1. `export PATCH=Z` where `Z` is the patch level of `vX.Y.Z`
1. cd to the base of the repo
1. `git fetch upstream && git checkout -b upstream/release-${VER} release-${VER}`
1. Make sure you don't have any files you care about littering your repo (they
better be checked in or outside the repo, or the next step will delete them).
1. `make clean && git reset --hard HEAD && git clean -xdf`
1. `make` (TBD: you really shouldn't have to do this, but the swagger output step requires it right now)
1. `./build/mark-new-version.sh v${VER}.${PATCH}` to mark the new release and get further
instructions. This creates a series of commits on the branch you're working
on (`release-${VER}`), including forking our documentation for the release,
the release version commit (which is then tagged), and the post-release
version commit.
1. Follow the instructions given to you by that script. They are canon for the
remainder of the Git process. If you don't understand something in that
process, please ask! When proposing PRs, you can pre-fill the body with
`hack/cherry_pick_list.sh upstream/release-${VER}` to inform people of what
is already on the branch.
**TODO**: how to fix tags, etc., if the release is changed.
#### Building and Pushing Binaries
In your git repo (you still have `${VER}` and `${PATCH}` set from above right?):
1. `git checkout upstream/master && build/build-official-release.sh
v${VER}.${PATCH}` (the `build-official-release.sh` script is version
agnostic, so it's best to run it off `master` directly).
1. Follow the instructions given to you by that script. At this point, you've
done all the Git bits, you've got all the binary bits pushed, and you've got
the template for the release started on GitHub.
#### Writing Release Notes
Run `hack/cherry_pick_list.sh ${VER}.${PATCH}~1` to get the release notes for
the patch release you just created. Feel free to prune anything internal, like
you would for a major release, but typically for patch releases we tend to
include everything in the release notes.
## Origin of the Sources
Kubernetes may be built from either a git tree (using `hack/build-go.sh`) or
from a tarball (using either `hack/build-go.sh` or `go install`) or directly by
the Go native build system (using `go get`).
When building from git, we want to be able to insert specific information about
the build tree at build time. In particular, we want to use the output of `git
describe` to generate the version of Kubernetes and the status of the build
tree (add a `-dirty` prefix if the tree was modified.)
When building from a tarball or using the Go build system, we will not have
access to the information about the git tree, but we still want to be able to
tell whether this build corresponds to an exact release (e.g. v0.3) or is
between releases (e.g. at some point in development between v0.3 and v0.4).
## Version Number Format
In order to account for these use cases, there are some specific formats that
may end up representing the Kubernetes version. Here are a few examples:
- **v0.5**: This is official version 0.5 and this version will only be used
when building from a clean git tree at the v0.5 git tag, or from a tree
extracted from the tarball corresponding to that specific release.
- **v0.5-15-g0123abcd4567**: This is the `git describe` output and it indicates
that we are 15 commits past the v0.5 release and that the SHA1 of the commit
where the binaries were built was `0123abcd4567`. It is only possible to have
this level of detail in the version information when building from git, not
when building from a tarball.
- **v0.5-15-g0123abcd4567-dirty** or **v0.5-dirty**: The extra `-dirty` prefix
means that the tree had local modifications or untracked files at the time of
the build, so there's no guarantee that the source code matches exactly the
state of the tree at the `0123abcd4567` commit or at the `v0.5` git tag
(resp.)
- **v0.5-dev**: This means we are building from a tarball or using `go get` or,
if we have a git tree, we are using `go install` directly, so it is not
possible to inject the git version into the build information. Additionally,
this is not an official release, so the `-dev` prefix indicates that the
version we are building is after `v0.5` but before `v0.6`. (There is actually
an exception where a commit with `v0.5-dev` is not present on `v0.6`, see
later for details.)
## Injecting Version into Binaries
In order to cover the different build cases, we start by providing information
that can be used when using only Go build tools or when we do not have the git
version information available.
To be able to provide a meaningful version in those cases, we set the contents
of variables in a Go source file that will be used when no overrides are
present.
We are using `pkg/version/base.go` as the source of versioning in absence of
information from git. Here is a sample of that file's contents:
{% highlight go %}
{% raw %}
var (
gitVersion string = "v0.4-dev" // version from git, output of $(git describe)
gitCommit string = "" // sha1 from git, output of $(git rev-parse HEAD)
)
{% endraw %}
{% endhighlight %}
This means a build with `go install` or `go get` or a build from a tarball will
yield binaries that will identify themselves as `v0.4-dev` and will not be able
to provide you with a SHA1.
To add the extra versioning information when building from git, the
`hack/build-go.sh` script will gather that information (using `git describe` and
`git rev-parse`) and then create a `-ldflags` string to pass to `go install` and
tell the Go linker to override the contents of those variables at build time. It
can, for instance, tell it to override `gitVersion` and set it to
`v0.4-13-g4567bcdef6789-dirty` and set `gitCommit` to `4567bcdef6789...` which
is the complete SHA1 of the (dirty) tree used at build time.
## Handling Official Versions
Handling official versions from git is easy, as long as there is an annotated
git tag pointing to a specific version then `git describe` will return that tag
exactly which will match the idea of an official version (e.g. `v0.5`).
Handling it on tarballs is a bit harder since the exact version string must be
present in `pkg/version/base.go` for it to get embedded into the binaries. But
simply creating a commit with `v0.5` on its own would mean that the commits
coming after it would also get the `v0.5` version when built from tarball or `go
get` while in fact they do not match `v0.5` (the one that was tagged) exactly.
To handle that case, creating a new release should involve creating two adjacent
commits where the first of them will set the version to `v0.5` and the second
will set it to `v0.5-dev`. In that case, even in the presence of merges, there
will be a single commit where the exact `v0.5` version will be used and all
others around it will either have `v0.4-dev` or `v0.5-dev`.
The diagram below illustrates it.
![Diagram of git commits involved in the release](releasing.png)
After working on `v0.4-dev` and merging PR 99 we decide it is time to release
`v0.5`. So we start a new branch, create one commit to update
`pkg/version/base.go` to include `gitVersion = "v0.5"` and `git commit` it.
We test it and make sure everything is working as expected.
Before sending a PR for it, we create a second commit on that same branch,
updating `pkg/version/base.go` to include `gitVersion = "v0.5-dev"`. That will
ensure that further builds (from tarball or `go install`) on that tree will
always include the `-dev` prefix and will not have a `v0.5` version (since they
do not match the official `v0.5` exactly.)
We then send PR 100 with both commits in it.
Once the PR is accepted, we can use `git tag -a` to create an annotated tag
*pointing to the one commit* that has `v0.5` in `pkg/version/base.go` and push
it to GitHub. (Unfortunately GitHub tags/releases are not annotated tags, so
this needs to be done from a git client and pushed to GitHub using SSH or
HTTPS.)
## Parallel Commits
While we are working on releasing `v0.5`, other development takes place and
other PRs get merged. For instance, in the example above, PRs 101 and 102 get
merged to the master branch before the versioning PR gets merged.
This is not a problem, it is only slightly inaccurate that checking out the tree
at commit `012abc` or commit `345cde` or at the commit of the merges of PR 101
or 102 will yield a version of `v0.4-dev` *but* those commits are not present in
`v0.5`.
In that sense, there is a small window in which commits will get a
`v0.4-dev` or `v0.4-N-gXXX` label and while they're indeed later than `v0.4`
but they are not really before `v0.5` in that `v0.5` does not contain those
commits.
Unfortunately, there is not much we can do about it. On the other hand, other
projects seem to live with that and it does not really become a large problem.
As an example, Docker commit a327d9b91edf has a `v1.1.1-N-gXXX` label but it is
not present in Docker `v1.2.0`:
{% highlight console %}
{% raw %}
$ git describe a327d9b91edf
v1.1.1-822-ga327d9b91edf
$ git log --oneline v1.2.0..a327d9b91edf
a327d9b91edf Fix data space reporting from Kb/Mb to KB/MB
(Non-empty output here means the commit is not present on v1.2.0.)
{% endraw %}
{% endhighlight %}
## Release Notes
No official release should be made final without properly matching release notes.
There should be made available, per release, a small summary, preamble, of the
major changes, both in terms of feature improvements/bug fixes and notes about
functional feature changes (if any) regarding the previous released version so
that the BOM regarding updating to it gets as obvious and trouble free as possible.
After this summary, preamble, all the relevant PRs/issues that got in that
version should be listed and linked together with a small summary understandable
by plain mortals (in a perfect world PR/issue's title would be enough but often
it is just too cryptic/geeky/domain-specific that it isn't).
<!-- BEGIN MUNGE: IS_VERSIONED -->
<!-- TAG IS_VERSIONED -->
<!-- END MUNGE: IS_VERSIONED -->
<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/devel/releasing.md?pixel)]()
<!-- END MUNGE: GENERATED_ANALYTICS -->
v1.1/docs/devel/releasing.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 30 KiB

v1.1/docs/devel/releasing.svg
+113
View File
@@ -0,0 +1,113 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN"
"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<!-- Generated by graphviz version 2.36.0 (20140111.2315)
-->
<!-- Title: tagged_release Pages: 1 -->
<svg width="257pt" height="360pt"
viewBox="0.00 0.00 257.33 360.00" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<g id="graph0" class="graph" transform="scale(0.649819 0.649819) rotate(0) translate(4 550)">
<title>tagged_release</title>
<polygon fill="white" stroke="none" points="-4,4 -4,-550 392,-550 392,4 -4,4"/>
<!-- ci012abc -->
<g id="node1" class="node"><title>ci012abc</title>
<ellipse fill="#ffffcc" stroke="black" cx="56" cy="-115" rx="42.7926" ry="42.7926"/>
<text text-anchor="middle" x="56" y="-111.3" font-family="Consolas, Liberation Mono, Menlo, Courier, monospace" font-size="14.00">012abc</text>
</g>
<!-- pr101 -->
<g id="node2" class="node"><title>pr101</title>
<polygon fill="#ccccff" stroke="black" points="112,-255 0,-255 0,-219 112,-219 112,-255"/>
<text text-anchor="middle" x="56" y="-233.3" font-family="Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif" font-size="14.00">Merge PR #101</text>
</g>
<!-- ci012abc&#45;&gt;pr101 -->
<g id="edge1" class="edge"><title>ci012abc&#45;&gt;pr101</title>
<path fill="none" stroke="black" d="M56,-157.97C56,-174.83 56,-193.784 56,-208.789"/>
<polygon fill="black" stroke="black" points="52.5001,-208.93 56,-218.93 59.5001,-208.93 52.5001,-208.93"/>
</g>
<!-- ci345cde -->
<g id="node3" class="node"><title>ci345cde</title>
<ellipse fill="#ffffcc" stroke="black" cx="62" cy="-359" rx="42.7926" ry="42.7926"/>
<text text-anchor="middle" x="62" y="-355.3" font-family="Consolas, Liberation Mono, Menlo, Courier, monospace" font-size="14.00">345cde</text>
</g>
<!-- pr101&#45;&gt;ci345cde -->
<g id="edge2" class="edge"><title>pr101&#45;&gt;ci345cde</title>
<path fill="none" stroke="black" d="M56.8597,-255.193C57.5237,-268.473 58.4796,-287.592 59.3874,-305.748"/>
<polygon fill="black" stroke="black" points="55.904,-306.17 59.8991,-315.982 62.8953,-305.82 55.904,-306.17"/>
</g>
<!-- pr102 -->
<g id="node4" class="node"><title>pr102</title>
<polygon fill="#ccccff" stroke="black" points="129,-474 17,-474 17,-438 129,-438 129,-474"/>
<text text-anchor="middle" x="73" y="-452.3" font-family="Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif" font-size="14.00">Merge PR #102</text>
</g>
<!-- ci345cde&#45;&gt;pr102 -->
<g id="edge3" class="edge"><title>ci345cde&#45;&gt;pr102</title>
<path fill="none" stroke="black" d="M66.8248,-401.668C67.8523,-410.542 68.9117,-419.692 69.8567,-427.853"/>
<polygon fill="black" stroke="black" points="66.3936,-428.375 71.0207,-437.906 73.3472,-427.57 66.3936,-428.375"/>
</g>
<!-- pr100 -->
<g id="node10" class="node"><title>pr100</title>
<polygon fill="#ccccff" stroke="black" points="174,-546 62,-546 62,-510 174,-510 174,-546"/>
<text text-anchor="middle" x="118" y="-524.3" font-family="Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif" font-size="14.00">Merge PR #100</text>
</g>
<!-- pr102&#45;&gt;pr100 -->
<g id="edge6" class="edge"><title>pr102&#45;&gt;pr100</title>
<path fill="none" stroke="black" d="M84.1236,-474.303C89.355,-482.441 95.6999,-492.311 101.478,-501.299"/>
<polygon fill="black" stroke="black" points="98.6526,-503.377 107.004,-509.896 104.541,-499.591 98.6526,-503.377"/>
</g>
<!-- version_commit -->
<g id="node5" class="node"><title>version_commit</title>
<ellipse fill="#ccffcc" stroke="black" cx="173" cy="-237" rx="42.7926" ry="42.7926"/>
<text text-anchor="middle" x="173" y="-233.3" font-family="Consolas, Liberation Mono, Menlo, Courier, monospace" font-size="14.00">678fed</text>
</g>
<!-- dev_commit -->
<g id="node6" class="node"><title>dev_commit</title>
<ellipse fill="#ffffcc" stroke="black" cx="169" cy="-359" rx="42.7926" ry="42.7926"/>
<text text-anchor="middle" x="169" y="-355.3" font-family="Consolas, Liberation Mono, Menlo, Courier, monospace" font-size="14.00">456dcb</text>
</g>
<!-- version_commit&#45;&gt;dev_commit -->
<g id="edge4" class="edge"><title>version_commit&#45;&gt;dev_commit</title>
<path fill="none" stroke="black" d="M171.601,-279.97C171.322,-288.326 171.027,-297.195 170.739,-305.844"/>
<polygon fill="black" stroke="black" points="167.24,-305.74 170.405,-315.851 174.236,-305.973 167.24,-305.74"/>
</g>
<!-- dev_commit&#45;&gt;pr100 -->
<g id="edge8" class="edge"><title>dev_commit&#45;&gt;pr100</title>
<path fill="none" stroke="black" d="M158.36,-400.568C152.438,-422.433 144.719,-449.815 137,-474 134.253,-482.606 131.013,-491.906 127.994,-500.278"/>
<polygon fill="black" stroke="black" points="124.616,-499.325 124.47,-509.919 131.191,-501.729 124.616,-499.325"/>
</g>
<!-- release_info -->
<g id="node7" class="node"><title>release_info</title>
<text text-anchor="middle" x="304" y="-240.8" font-family="Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif" font-size="14.00">pkg/version/base.go:</text>
<text text-anchor="middle" x="304" y="-225.8" font-family="Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif" font-size="14.00">gitVersion = &quot;v0.5&quot;;</text>
</g>
<!-- dev_info -->
<g id="node8" class="node"><title>dev_info</title>
<text text-anchor="middle" x="309" y="-362.8" font-family="Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif" font-size="14.00">pkg/version/base.go:</text>
<text text-anchor="middle" x="309" y="-347.8" font-family="Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif" font-size="14.00">gitVersion = &quot;v0.5&#45;dev&quot;;</text>
</g>
<!-- pr99 -->
<g id="node9" class="node"><title>pr99</title>
<polygon fill="#ccccff" stroke="black" points="143,-36 39,-36 39,-0 143,-0 143,-36"/>
<text text-anchor="middle" x="91" y="-14.3" font-family="Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif" font-size="14.00">Merge PR #99</text>
</g>
<!-- pr99&#45;&gt;ci012abc -->
<g id="edge5" class="edge"><title>pr99&#45;&gt;ci012abc</title>
<path fill="none" stroke="black" d="M84.5805,-36.4245C81.5586,-44.6267 77.7879,-54.8615 73.9865,-65.1795"/>
<polygon fill="black" stroke="black" points="70.6804,-64.0292 70.5075,-74.6226 77.2488,-66.4492 70.6804,-64.0292"/>
</g>
<!-- pr99&#45;&gt;version_commit -->
<g id="edge7" class="edge"><title>pr99&#45;&gt;version_commit</title>
<path fill="none" stroke="black" d="M97.4344,-36.0276C109.585,-68.1826 136.317,-138.924 154.498,-187.038"/>
<polygon fill="black" stroke="black" points="151.318,-188.523 158.127,-196.64 157.866,-186.048 151.318,-188.523"/>
</g>
<!-- tag -->
<g id="node11" class="node"><title>tag</title>
<ellipse fill="#ffcccc" stroke="black" cx="226" cy="-115" rx="71.4873" ry="18"/>
<text text-anchor="middle" x="226" y="-111.3" font-family="Helvetica Neue, Helvetica, Segoe UI, Arial, freesans, sans-serif" font-size="14.00">$ git tag &#45;a v0.5</text>
</g>
<!-- tag&#45;&gt;version_commit -->
<g id="edge9" class="edge"><title>tag&#45;&gt;version_commit</title>
<path fill="none" stroke="black" d="M218.519,-132.939C212.168,-147.318 202.736,-168.673 194.103,-188.22"/>
<polygon fill="black" stroke="black" points="190.784,-187.071 189.946,-197.632 197.188,-189.899 190.784,-187.071"/>
</g>
</g>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 7.1 KiB

v1.1/docs/devel/scheduler.md
Executable
+68
View File
@@ -0,0 +1,68 @@
---
layout: docwithnav
title: "The Kubernetes Scheduler"
---
<!-- BEGIN MUNGE: UNVERSIONED_WARNING -->
<!-- END MUNGE: UNVERSIONED_WARNING -->
# The Kubernetes Scheduler
The Kubernetes scheduler runs as a process alongside the other master
components such as the API server. Its interface to the API server is to watch
for Pods with an empty PodSpec.NodeName, and for each Pod, it posts a Binding
indicating where the Pod should be scheduled.
## The scheduling process
The scheduler tries to find a node for each Pod, one at a time, as it notices
these Pods via watch. There are three steps. First it applies a set of "predicates" that filter out
inappropriate nodes. For example, if the PodSpec specifies resource requests, then the scheduler
will filter out nodes that don't have at least that much resources available (computed
as the capacity of the node minus the sum of the resource requests of the containers that
are already running on the node). Second, it applies a set of "priority functions"
that rank the nodes that weren't filtered out by the predicate check. For example,
it tries to spread Pods across nodes while at the same time favoring the least-loaded
nodes (where "load" here is sum of the resource requests of the containers running on the node,
divided by the node's capacity).
Finally, the node with the highest priority is chosen
(or, if there are multiple such nodes, then one of them is chosen at random). The code
for this main scheduling loop is in the function `Schedule()` in
[plugin/pkg/scheduler/generic_scheduler.go](http://releases.k8s.io/release-1.1/plugin/pkg/scheduler/generic_scheduler.go)
## Scheduler extensibility
The scheduler is extensible: the cluster administrator can choose which of the pre-defined
scheduling policies to apply, and can add new ones. The built-in predicates and priorities are
defined in [plugin/pkg/scheduler/algorithm/predicates/predicates.go](http://releases.k8s.io/release-1.1/plugin/pkg/scheduler/algorithm/predicates/predicates.go) and
[plugin/pkg/scheduler/algorithm/priorities/priorities.go](http://releases.k8s.io/release-1.1/plugin/pkg/scheduler/algorithm/priorities/priorities.go), respectively.
The policies that are applied when scheduling can be chosen in one of two ways. Normally,
the policies used are selected by the functions `defaultPredicates()` and `defaultPriorities()` in
[plugin/pkg/scheduler/algorithmprovider/defaults/defaults.go](http://releases.k8s.io/release-1.1/plugin/pkg/scheduler/algorithmprovider/defaults/defaults.go).
However, the choice of policies
can be overridden by passing the command-line flag `--policy-config-file` to the scheduler, pointing to a JSON
file specifying which scheduling policies to use. See
[examples/scheduler-policy-config.json](../../examples/scheduler-policy-config.json) for an example
config file. (Note that the config file format is versioned; the API is defined in
[plugin/pkg/scheduler/api](http://releases.k8s.io/release-1.1/plugin/pkg/scheduler/api/)).
Thus to add a new scheduling policy, you should modify predicates.go or priorities.go,
and either register the policy in `defaultPredicates()` or `defaultPriorities()`, or use a policy config file.
## Exploring the code
If you want to get a global picture of how the scheduler works, you can start in
[plugin/cmd/kube-scheduler/app/server.go](http://releases.k8s.io/release-1.1/plugin/cmd/kube-scheduler/app/server.go)
<!-- BEGIN MUNGE: IS_VERSIONED -->
<!-- TAG IS_VERSIONED -->
<!-- END MUNGE: IS_VERSIONED -->
<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/devel/scheduler.md?pixel)]()
<!-- END MUNGE: GENERATED_ANALYTICS -->
v1.1/docs/devel/scheduler_algorithm.md
Executable
+56
View File
@@ -0,0 +1,56 @@
---
layout: docwithnav
title: "Scheduler Algorithm in Kubernetes"
---
<!-- BEGIN MUNGE: UNVERSIONED_WARNING -->
<!-- END MUNGE: UNVERSIONED_WARNING -->
# Scheduler Algorithm in Kubernetes
For each unscheduled Pod, the Kubernetes scheduler tries to find a node across the cluster according to a set of rules. A general introduction to the Kubernetes scheduler can be found at [scheduler.md](scheduler.html). In this document, the algorithm of how to select a node for the Pod is explained. There are two steps before a destination node of a Pod is chosen. The first step is filtering all the nodes and the second is ranking the remaining nodes to find a best fit for the Pod.
## Filtering the nodes
The purpose of filtering the nodes is to filter out the nodes that do not meet certain requirements of the Pod. For example, if the free resource on a node (measured by the capacity minus the sum of the resource requests of all the Pods that already run on the node) is less than the Pod's required resource, the node should not be considered in the ranking phase so it is filtered out. Currently, there are several "predicates" implementing different filtering policies, including:
- `NoDiskConflict`: Evaluate if a pod can fit due to the volumes it requests, and those that are already mounted.
- `PodFitsResources`: Check if the free resource (CPU and Memory) meets the requirement of the Pod. The free resource is measured by the capacity minus the sum of requests of all Pods on the node. To learn more about the resource QoS in Kubernetes, please check [QoS proposal](../proposals/resource-qos.html).
- `PodFitsHostPorts`: Check if any HostPort required by the Pod is already occupied on the node.
- `PodFitsHost`: Filter out all nodes except the one specified in the PodSpec's NodeName field.
- `PodSelectorMatches`: Check if the labels of the node match the labels specified in the Pod's `nodeSelector` field ([Here](../user-guide/node-selection/) is an example of how to use `nodeSelector` field).
- `CheckNodeLabelPresence`: Check if all the specified labels exist on a node or not, regardless of the value.
The details of the above predicates can be found in [plugin/pkg/scheduler/algorithm/predicates/predicates.go](http://releases.k8s.io/release-1.1/plugin/pkg/scheduler/algorithm/predicates/predicates.go). All predicates mentioned above can be used in combination to perform a sophisticated filtering policy. Kubernetes uses some, but not all, of these predicates by default. You can see which ones are used by default in [plugin/pkg/scheduler/algorithmprovider/defaults/defaults.go](http://releases.k8s.io/release-1.1/plugin/pkg/scheduler/algorithmprovider/defaults/defaults.go).
## Ranking the nodes
The filtered nodes are considered suitable to host the Pod, and it is often that there are more than one nodes remaining. Kubernetes prioritizes the remaining nodes to find the "best" one for the Pod. The prioritization is performed by a set of priority functions. For each remaining node, a priority function gives a score which scales from 0-10 with 10 representing for "most preferred" and 0 for "least preferred". Each priority function is weighted by a positive number and the final score of each node is calculated by adding up all the weighted scores. For example, suppose there are two priority functions, `priorityFunc1` and `priorityFunc2` with weighting factors `weight1` and `weight2` respectively, the final score of some NodeA is:
finalScoreNodeA = (weight1 * priorityFunc1) + (weight2 * priorityFunc2)
After the scores of all nodes are calculated, the node with highest score is chosen as the host of the Pod. If there are more than one nodes with equal highest scores, a random one among them is chosen.
Currently, Kubernetes scheduler provides some practical priority functions, including:
- `LeastRequestedPriority`: The node is prioritized based on the fraction of the node that would be free if the new Pod were scheduled onto the node. (In other words, (capacity - sum of requests of all Pods already on the node - request of Pod that is being scheduled) / capacity). CPU and memory are equally weighted. The node with the highest free fraction is the most preferred. Note that this priority function has the effect of spreading Pods across the nodes with respect to resource consumption.
- `CalculateNodeLabelPriority`: Prefer nodes that have the specified label.
- `BalancedResourceAllocation`: This priority function tries to put the Pod on a node such that the CPU and Memory utilization rate is balanced after the Pod is deployed.
- `CalculateSpreadPriority`: Spread Pods by minimizing the number of Pods belonging to the same service on the same node.
- `CalculateAntiAffinityPriority`: Spread Pods by minimizing the number of Pods belonging to the same service on nodes with the same value for a particular label.
The details of the above priority functions can be found in [plugin/pkg/scheduler/algorithm/priorities](http://releases.k8s.io/release-1.1/plugin/pkg/scheduler/algorithm/priorities/). Kubernetes uses some, but not all, of these priority functions by default. You can see which ones are used by default in [plugin/pkg/scheduler/algorithmprovider/defaults/defaults.go](http://releases.k8s.io/release-1.1/plugin/pkg/scheduler/algorithmprovider/defaults/defaults.go). Similar as predicates, you can combine the above priority functions and assign weight factors (positive number) to them as you want (check [scheduler.md](scheduler.html) for how to customize).
<!-- BEGIN MUNGE: IS_VERSIONED -->
<!-- TAG IS_VERSIONED -->
<!-- END MUNGE: IS_VERSIONED -->
<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/devel/scheduler_algorithm.md?pixel)]()
<!-- END MUNGE: GENERATED_ANALYTICS -->
v1.1/docs/devel/writing-a-getting-started-guide.md
+118
View File
@@ -0,0 +1,118 @@
---
layout: docwithnav
title: "Writing a Getting Started Guide"
---
<!-- BEGIN MUNGE: UNVERSIONED_WARNING -->
<!-- END MUNGE: UNVERSIONED_WARNING -->
# Writing a Getting Started Guide
This page gives some advice for anyone planning to write or update a Getting Started Guide for Kubernetes.
It also gives some guidelines which reviewers should follow when reviewing a pull request for a
guide.
A Getting Started Guide is instructions on how to create a Kubernetes cluster on top of a particular
type(s) of infrastructure. Infrastructure includes: the IaaS provider for VMs;
the node OS; inter-node networking; and node Configuration Management system.
A guide refers to scripts, Configuration Management files, and/or binary assets such as RPMs. We call
the combination of all these things needed to run on a particular type of infrastructure a
**distro**.
[The Matrix](../../docs/getting-started-guides/README.html) lists the distros. If there is already a guide
which is similar to the one you have planned, consider improving that one.
Distros fall into two categories:
- **versioned distros** are tested to work with a particular binary release of Kubernetes. These
come in a wide variety, reflecting a wide range of ideas and preferences in how to run a cluster.
- **development distros** are tested work with the latest Kubernetes source code. But, there are
relatively few of these and the bar is much higher for creating one. They must support
fully automated cluster creation, deletion, and upgrade.
There are different guidelines for each.
## Versioned Distro Guidelines
These guidelines say *what* to do. See the Rationale section for *why*.
- Send us a PR.
- Put the instructions in `docs/getting-started-guides/...`. Scripts go there too. This helps devs easily
search for uses of flags by guides.
- We may ask that you host binary assets or large amounts of code in our `contrib` directory or on your
own repo.
- Add or update a row in [The Matrix](../../docs/getting-started-guides/README.html).
- State the binary version of Kubernetes that you tested clearly in your Guide doc.
- Setup a cluster and run the [conformance test](development.html#conformance-testing) against it, and report the
results in your PR.
- Versioned distros should typically not modify or add code in `cluster/`. That is just scripts for developer
distros.
- When a new major or minor release of Kubernetes comes out, we may also release a new
conformance test, and require a new conformance test run to earn a conformance checkmark.
If you have a cluster partially working, but doing all the above steps seems like too much work,
we still want to hear from you. We suggest you write a blog post or a Gist, and we will link to it on our wiki page.
Just file an issue or chat us on [Slack](../troubleshooting.html#slack) and one of the committers will link to it from the wiki.
## Development Distro Guidelines
These guidelines say *what* to do. See the Rationale section for *why*.
- the main reason to add a new development distro is to support a new IaaS provider (VM and
network management). This means implementing a new `pkg/cloudprovider/providers/$IAAS_NAME`.
- Development distros should use Saltstack for Configuration Management.
- development distros need to support automated cluster creation, deletion, upgrading, etc.
This mean writing scripts in `cluster/$IAAS_NAME`.
- all commits to the tip of this repo need to not break any of the development distros
- the author of the change is responsible for making changes necessary on all the cloud-providers if the
change affects any of them, and reverting the change if it breaks any of the CIs.
- a development distro needs to have an organization which owns it. This organization needs to:
- Setting up and maintaining Continuous Integration that runs e2e frequently (multiple times per day) against the
Distro at head, and which notifies all devs of breakage.
- being reasonably available for questions and assisting with
refactoring and feature additions that affect code for their IaaS.
## Rationale
- We want people to create Kubernetes clusters with whatever IaaS, Node OS,
configuration management tools, and so on, which they are familiar with. The
guidelines for **versioned distros** are designed for flexibility.
- We want developers to be able to work without understanding all the permutations of
IaaS, NodeOS, and configuration management. The guidelines for **developer distros** are designed
for consistency.
- We want users to have a uniform experience with Kubernetes whenever they follow instructions anywhere
in our Github repository. So, we ask that versioned distros pass a **conformance test** to make sure
really work.
- We want to **limit the number of development distros** for several reasons. Developers should
only have to change a limited number of places to add a new feature. Also, since we will
gate commits on passing CI for all distros, and since end-to-end tests are typically somewhat
flaky, it would be highly likely for there to be false positives and CI backlogs with many CI pipelines.
- We do not require versioned distros to do **CI** for several reasons. It is a steep
learning curve to understand our automated testing scripts. And it is considerable effort
to fully automate setup and teardown of a cluster, which is needed for CI. And, not everyone
has the time and money to run CI. We do not want to
discourage people from writing and sharing guides because of this.
- Versioned distro authors are free to run their own CI and let us know if there is breakage, but we
will not include them as commit hooks -- there cannot be so many commit checks that it is impossible
to pass them all.
- We prefer a single Configuration Management tool for development distros. If there were more
than one, the core developers would have to learn multiple tools and update config in multiple
places. **Saltstack** happens to be the one we picked when we started the project. We
welcome versioned distros that use any tool; there are already examples of
CoreOS Fleet, Ansible, and others.
- You can still run code from head or your own branch
if you use another Configuration Management tool -- you just have to do some manual steps
during testing and deployment.
<!-- BEGIN MUNGE: IS_VERSIONED -->
<!-- TAG IS_VERSIONED -->
<!-- END MUNGE: IS_VERSIONED -->
<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/devel/writing-a-getting-started-guide.md?pixel)]()
<!-- END MUNGE: GENERATED_ANALYTICS -->