Events
diff --git a/docs/admin/accessing-the-api.md b/docs/admin/accessing-the-api.md
index cb3f3d4ce4..92de1772ab 100644
--- a/docs/admin/accessing-the-api.md
+++ b/docs/admin/accessing-the-api.md
@@ -3,7 +3,7 @@ assignees:
- bgrant0607
- erictune
- lavalamp
-
+title: Overview
---
This document describes how access to the Kubernetes API is controlled.
@@ -24,7 +24,7 @@ following diagram:
In a typical Kubernetes cluster, the API served on port 443. A TLS connection is
established. The API server presents a certificate. This certificate is
often self-signed, so `$USER/.kube/config` on the user's machine typically
-contains the root certficate for the API server's certificate, which when specified
+contains the root certificate for the API server's certificate, which when specified
is used in place of the system default root certificates. This certificate is typically
automatically written into your `$USER/.kube/config` when you create a cluster yourself
using `kube-up.sh`. If the cluster has multiple users, then the creator needs to share
@@ -86,7 +86,7 @@ For version 1.2, clusters created by `kube-up.sh` are configured so that no auth
required for any request.
As of version 1.3, clusters created by `kube-up.sh` are configured so that the ABAC authorization
-modules is enabled. However, its input file is initially set to allow all users to do all
+modules are enabled. However, its input file is initially set to allow all users to do all
operations. The cluster administrator needs to edit that file, or configure a different authorizer
to restrict what users can do.
@@ -94,7 +94,7 @@ to restrict what users can do.
The Authorization step is designed to operate on attributes that are likely to be common to most
REST requests, such as object name, kind, etc. This is intended to facilitate interation with
existing organization-wide or cloud-provider-wide access control systems (which may handle
-other APIs besides the Kubernetes API.
+other APIs besides the Kubernetes API).
Access controls and policies that depend on specific fields of specific Kinds of objects
are handled by Admission Controllers.
@@ -127,7 +127,7 @@ for the corresponding API object, and then written to the object store (shown as
The previous discussion applies to requests sent to the secure port of the API server
(the typical case). The API server can actually serve on 2 ports:
-By default the Kubernetes APIserver serves HTTP on 2 ports:
+By default the Kubernetes API server serves HTTP on 2 ports:
1. `Localhost Port`:
@@ -148,7 +148,7 @@ By default the Kubernetes APIserver serves HTTP on 2 ports:
- default IP is first non-localhost network interface, change with `--bind-address` flag.
- request handled by authentication and authorization modules.
- request handled by admission control module(s).
- - authentication and authoriation modules run.
+ - authentication and authorisation modules run.
When the cluster is created by `kube-up.sh`, on Google Compute Engine (GCE),
and on several other cloud providers, the API server serves on port 443. On
diff --git a/docs/admin/addons.md b/docs/admin/addons.md
index 1555f8263c..aeee68cc30 100644
--- a/docs/admin/addons.md
+++ b/docs/admin/addons.md
@@ -1,4 +1,5 @@
---
+title: Installing Addons
---
## Overview
@@ -11,9 +12,9 @@ Add-ons in each section are sorted alphabetically - the ordering does not imply
## Networking and Network Policy
-* [Calico](http://docs.projectcalico.org/v1.6/getting-started/kubernetes/installation/hosted/) is a secure L3 networking and network policy provider.
+* [Calico](http://docs.projectcalico.org/v2.0/getting-started/kubernetes/installation/hosted/) is a secure L3 networking and network policy provider.
* [Canal](https://github.com/tigera/canal/tree/master/k8s-install/kubeadm) unites Flannel and Calico, providing networking and network policy.
-* [Flannel](https://github.com/coreos/flannel/blob/master/Documentation/kube-flannel.yml) is a overlay network provider that can be used with Kubernetes.
+* [Flannel](https://github.com/coreos/flannel/blob/master/Documentation/kube-flannel.yml) is an overlay network provider that can be used with Kubernetes.
* [Romana](http://romana.io) is a Layer 3 networking solution for pod networks that also supports the [NetworkPolicy API](/docs/user-guide/networkpolicies/). Kubeadm add-on installation details available [here](https://github.com/romana/romana/tree/master/containerize).
* [Weave Net](https://www.weave.works/docs/net/latest/kube-addon/) provides networking and network policy, will carry on working on both sides of a network partition, and does not require an external database.
diff --git a/docs/admin/admission-controllers.md b/docs/admin/admission-controllers.md
index 24da796163..089dce2605 100644
--- a/docs/admin/admission-controllers.md
+++ b/docs/admin/admission-controllers.md
@@ -6,7 +6,7 @@ assignees:
- erictune
- janetkuo
- thockin
-
+title: Using Admission Controllers
---
* TOC
@@ -126,7 +126,7 @@ For additional HTTP configuration, refer to the [kubeconfig](/docs/user-guide/ku
When faced with an admission decision, the API Server POSTs a JSON serialized api.imagepolicy.v1alpha1.ImageReview object describing the action. This object contains fields describing the containers being admitted, as well as any pod annotations that match `*.image-policy.k8s.io/*`.
-Note that webhook API objects are subject to the same versioning compatibility rules as other Kubernetes API objects. Implementers should be aware of looser compatibility promises for alpha objects and check the “apiVersion” field of the request to ensure correct deserialization. Additionally, the API Server must enable the imagepolicy.k8s.io/v1alpha1 API extensions group (`--runtime-config=imagepolicy.k8s.io/v1alpha1=true`).
+Note that webhook API objects are subject to the same versioning compatibility rules as other Kubernetes API objects. Implementers should be aware of looser compatibility promises for alpha objects and check the "apiVersion" field of the request to ensure correct deserialization. Additionally, the API Server must enable the imagepolicy.k8s.io/v1alpha1 API extensions group (`--runtime-config=imagepolicy.k8s.io/v1alpha1=true`).
An example request body:
@@ -151,7 +151,7 @@ An example request body:
}
```
-The remote service is expected to fill the ImageReviewStatus field of the request and respond to either allow or disallow access. The response body’s “spec” field is ignored and may be omitted. A permissive response would return:
+The remote service is expected to fill the ImageReviewStatus field of the request and respond to either allow or disallow access. The response body's "spec" field is ignored and may be omitted. A permissive response would return:
```
{
diff --git a/docs/admin/apparmor/index.md b/docs/admin/apparmor/index.md
index 9730c07953..224f0bbdeb 100644
--- a/docs/admin/apparmor/index.md
+++ b/docs/admin/apparmor/index.md
@@ -1,7 +1,7 @@
---
assignees:
- stclair
-
+title: AppArmor
---
AppArmor is a Linux kernel enhancement that can reduce the potential attack surface of an
@@ -384,7 +384,7 @@ Specifying the default profile to apply to containers when none is provided:
- **key**: `apparmor.security.beta.kubernetes.io/defaultProfileName`
- **value**: a profile reference, described above
-Specifying the list of profiles Pod containers are allowed to specify:
+Specifying the list of profiles Pod containers is allowed to specify:
- **key**: `apparmor.security.beta.kubernetes.io/allowedProfileNames`
- **value**: a comma-separated list of profile references (described above)
diff --git a/docs/admin/audit.md b/docs/admin/audit.md
index c3a6fda6da..ec4f7b2e95 100644
--- a/docs/admin/audit.md
+++ b/docs/admin/audit.md
@@ -2,14 +2,12 @@
assignees:
- soltysh
- sttts
-
+title: Audit in Kubernetes
---
* TOC
{:toc}
-## Audit in Kubernetes
-
Kubernetes Audit provides a security-relevant chronological set of records documenting
the sequence of activities that have affected system by individual users, administrators
or other components of the system. It allows cluster administrator to
@@ -23,7 +21,7 @@ answer the following questions:
- to where was it going?
NOTE: Currently, Kubernetes provides only basic audit capabilities, there is still a lot
-of work going on to provide fully featured auditing capabilities (see https://github.com/kubernetes/features/issues/22).
+of work going on to provide fully featured auditing capabilities (see [this issue](https://github.com/kubernetes/features/issues/22)).
Kubernetes audit is part of [kube-apiserver](/docs/admin/kube-apiserver) logging all requests
coming to the server. Each audit log contains two entries:
diff --git a/docs/admin/authentication.md b/docs/admin/authentication.md
index 3d73f527bf..4bbd0a4fee 100644
--- a/docs/admin/authentication.md
+++ b/docs/admin/authentication.md
@@ -4,8 +4,10 @@ assignees:
- lavalamp
- ericchiang
- deads2k
-
+- liggitt
+title: Authenticating
---
+
* TOC
{:toc}
@@ -29,14 +31,14 @@ to talk to the Kubernetes API.
API requests are tied to either a normal user or a service account, or are treated
as anonymous requests. This means every process inside or outside the cluster, from
a human user typing `kubectl` on a workstation, to `kubelets` on nodes, to members
-of the control plane, must authenticate when making requests to the the API server,
+of the control plane, must authenticate when making requests to the API server,
or be treated as an anonymous user.
## Authentication strategies
Kubernetes uses client certificates, bearer tokens, an authenticating proxy, or HTTP basic auth to
-authenticate API requests through authentication plugins. As HTTP request are
-made to the API server plugins attempts to associate the following attributes
+authenticate API requests through authentication plugins. As HTTP requests are
+made to the API server, plugins attempt to associate the following attributes
with the request:
* Username: a string which identifies the end user. Common values might be `kube-admin` or `jane@example.com`.
@@ -382,6 +384,13 @@ option to the API server during startup. The plugin is implemented in
`plugin/pkg/auth/authenticator/password/keystone/keystone.go` and currently uses
basic auth to verify used by username and password.
+If you have configured self-signed certificates for the Keystone server,
+you may need to set the `--experimental-keystone-ca-file=SOMEFILE` option when
+starting the Kubernetes API server. If you set the option, the Keystone
+server's certificate is verified by one of the authorities in the
+`experimental-keystone-ca-file`. Otherwise, the certificate is verified by
+the host's root Certificate Authority.
+
For details on how to use keystone to manage projects and users, refer to the
[Keystone documentation](http://docs.openstack.org/developer/keystone/). Please
note that this plugin is still experimental, under active development, and likely
@@ -420,7 +429,7 @@ enterprise directory, kerberos, etc.)
### Creating Certificates
When using client certificate authentication, you can generate certificates
-using an existing deployment script or manually through `easyrsa` or `openssl.``
+using an existing deployment script or manually through `easyrsa` or `openssl.`
#### Using an Existing Deployment Script
@@ -435,7 +444,7 @@ The script will generate three files: `ca.crt`, `server.crt`, and `server.key`.
Finally, add the following parameters into API server start parameters:
- `--client-ca-file=/srv/kubernetes/ca.crt`
-- `--tls-cert-file=/srv/kubernetes/server.cert`
+- `--tls-cert-file=/srv/kubernetes/server.crt`
- `--tls-private-key-file=/srv/kubernetes/server.key`
#### easyrsa
@@ -459,7 +468,7 @@ Finally, add the following parameters into API server start parameters:
1. Fill in and add the following parameters into the API server start parameters:
--client-ca-file=/yourdirectory/ca.crt
- --tls-cert-file=/yourdirectory/server.cert
+ --tls-cert-file=/yourdirectory/server.crt
--tls-private-key-file=/yourdirectory/server.key
#### openssl
diff --git a/docs/admin/authorization.md b/docs/admin/authorization.md
index 1a86359a92..c9dd662449 100644
--- a/docs/admin/authorization.md
+++ b/docs/admin/authorization.md
@@ -2,7 +2,9 @@
assignees:
- erictune
- lavalamp
-
+- deads2k
+- liggitt
+title: Using Authorization Plugins
---
In Kubernetes, authorization happens as a separate step from authentication.
@@ -53,7 +55,7 @@ A request has the following attributes that can be considered for authorization:
- what resource is being accessed (for resource requests only)
- what subresource is being accessed (for resource requests only)
- the namespace of the object being accessed (for namespaced resource requests only)
- - the API group being accessed (for resource requests only); an empty string designates the [core API group](../api.md#api-groups)
+ - the API group being accessed (for resource requests only); an empty string designates the [core API group](/docs/api/)
The request verb for a resource API endpoint can be determined by the HTTP verb used and whether or not the request acts on an individual resource or a collection of resources:
@@ -297,9 +299,8 @@ subjects:
name: jane
roleRef:
kind: Role
- namespace: default
name: pod-reader
- apiVersion: rbac.authorization.k8s.io/v1alpha1
+ apiGroup: rbac.authorization.k8s.io
```
`RoleBindings` may also refer to a `ClusterRole`. However, a `RoleBinding` that
@@ -324,26 +325,26 @@ subjects:
roleRef:
kind: ClusterRole
name: secret-reader
- apiVersion: rbac.authorization.k8s.io/v1alpha1
+ apiGroup: rbac.authorization.k8s.io
```
Finally a `ClusterRoleBinding` may be used to grant permissions in all
namespaces. The following `ClusterRoleBinding` allows any user in the group
-"manager" to read secrets in any namepsace.
+"manager" to read secrets in any namespace.
```yaml
# This cluster role binding allows anyone in the "manager" group to read secrets in any namespace.
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1alpha1
metadata:
- name: read-secrets
+ name: read-secrets-global
subjects:
- kind: Group # May be "User", "Group" or "ServiceAccount"
name: manager
roleRef:
kind: ClusterRole
- name: secret-reader
- apiVersion: rbac.authorization.k8s.io/v1alpha1
+ name: secret-reader
+ apiGroup: rbac.authorization.k8s.io
```
### Referring to Resources
@@ -380,7 +381,7 @@ Subjects can be groups, users or service accounts.
Users are represented by strings. These can be plain usernames, like
"alice", or email style names, like "bob@example.com", or numeric ids
as string. It is up to the Kubernetes admin to configure
-the [authentication modules](/doc/admin/authentication/) to produce
+the [authentication modules](/docs/admin/authentication/) to produce
usernames in the desired format. The RBAC authorization system does
not require any particular format. However, the prefix `system:` is
reserved for Kubernetes system use, and so the admin should ensure
@@ -565,10 +566,10 @@ Access to non-resource paths are sent as:
Non-resource paths include: `/api`, `/apis`, `/metrics`, `/resetMetrics`,
`/logs`, `/debug`, `/healthz`, `/swagger-ui/`, `/swaggerapi/`, `/ui`, and
-`/version.` Clients require access to `/api`, `/api/*/`, `/apis/`, `/apis/*`,
-`/apis/*/*`, and `/version` to discover what resources and versions are present
-on the server. Access to other non-resource paths can be disallowed without
-restricting access to the REST api.
+`/version.` Clients require access to `/api`, `/api/*`, `/apis`, `/apis/*`,
+and `/version` to discover what resources and versions are present on the server.
+Access to other non-resource paths can be disallowed without restricting access
+to the REST api.
For further documentation refer to the authorization.v1beta1 API objects and
plugin/pkg/auth/authorizer/webhook/webhook.go.
diff --git a/docs/admin/cluster-components.md b/docs/admin/cluster-components.md
index c1bcae8577..8519a199f4 100644
--- a/docs/admin/cluster-components.md
+++ b/docs/admin/cluster-components.md
@@ -1,7 +1,7 @@
---
assignees:
- lavalamp
-
+title: Kubernetes Components
---
This document outlines the various binary components that need to run to
@@ -61,12 +61,12 @@ selects a node for them to run on.
### addons
-Addons are pods and services that implement cluster features. They don't run on
-the master VM, but currently the default setup scripts that make the API calls
-to create these pods and services does run on the master VM. See:
-[kube-master-addons](http://releases.k8s.io/HEAD/cluster/saltbase/salt/kube-master-addons/kube-master-addons.sh)
+Addons are pods and services that implement cluster features. The pods may be managed
+by Deployments, ReplicationContollers, etc. Namespaced addon objects are created in
+the "kube-system" namespace.
-Addon objects are created in the "kube-system" namespace.
+Addon manager takes the responsibility for creating and maintaining addon resources.
+See [here](http://releases.k8s.io/HEAD/cluster/addons) for more details.
#### DNS
@@ -91,15 +91,8 @@ about containers in a central database, and provides a UI for browsing that data
#### Cluster-level Logging
-[Container Logging](/docs/user-guide/monitoring) saves container logs
-to a central log store with search/browsing interface. There are two
-implementations:
-
-* [Cluster-level logging to Google Cloud Logging](
-/docs/user-guide/logging/#cluster-level-logging-to-google-cloud-logging)
-
-* [Cluster-level Logging with Elasticsearch and Kibana](
-/docs/getting-started-guides/logging-elasticsearch/)
+A [Cluster-level logging](/docs/user-guide/logging/overview) mechanism is responsible for
+saving container logs to a central log store with search/browsing interface.
## Node components
diff --git a/docs/admin/cluster-large.md b/docs/admin/cluster-large.md
index d2285c3346..08d6d6c0d8 100644
--- a/docs/admin/cluster-large.md
+++ b/docs/admin/cluster-large.md
@@ -1,126 +1,127 @@
----
-assignees:
-- davidopp
-- lavalamp
-
----
-
-
-## Support
-
-At {{page.version}}, Kubernetes supports clusters with up to 1000 nodes. More specifically, we support configurations that meet *all* of the following criteria:
-
-* No more than 2000 nodes
-* No more than 60000 total pods
-* No more than 120000 total containers
-* No more than 100 pods per node
-
-* TOC
-{:toc}
-
-## Setup
-
-A cluster is a set of nodes (physical or virtual machines) running Kubernetes agents, managed by a "master" (the cluster-level control plane).
-
-Normally the number of nodes in a cluster is controlled by the the value `NUM_NODES` in the platform-specific `config-default.sh` file (for example, see [GCE's `config-default.sh`](http://releases.k8s.io/{{page.githubbranch}}/cluster/gce/config-default.sh)).
-
-Simply changing that value to something very large, however, may cause the setup script to fail for many cloud providers. A GCE deployment, for example, will run in to quota issues and fail to bring the cluster up.
-
-When setting up a large Kubernetes cluster, the following issues must be considered.
-
-### Quota Issues
-
-To avoid running into cloud provider quota issues, when creating a cluster with many nodes, consider:
-
-* Increase the quota for things like CPU, IPs, etc.
- * In [GCE, for example,](https://cloud.google.com/compute/docs/resource-quotas) you'll want to increase the quota for:
- * CPUs
- * VM instances
- * Total persistent disk reserved
- * In-use IP addresses
- * Firewall Rules
- * Forwarding rules
- * Routes
- * Target pools
-* Gating the setup script so that it brings up new node VMs in smaller batches with waits in between, because some cloud providers rate limit the creation of VMs.
-
-### Etcd storage
-
-To improve performance of large clusters, we store events in a separate dedicated etcd instance.
-
-When creating a cluster, existing salt scripts:
-
-* start and configure additional etcd instance
-* configure api-server to use it for storing events
-
-### Size of master and master components
-
-On GCE/GKE and AWS, `kube-up` automatically configures the proper VM size for your master depending on the number of nodes
-in your cluster. On other providers, you will need to configure it manually. For reference, the sizes we use on GCE are
-
-* 1-5 nodes: n1-standard-1
-* 6-10 nodes: n1-standard-2
-* 11-100 nodes: n1-standard-4
-* 101-250 nodes: n1-standard-8
-* 251-500 nodes: n1-standard-16
-* more than 500 nodes: n1-standard-32
-
-And the sizes we use on AWS are
-
-* 1-5 nodes: m3.medium
-* 6-10 nodes: m3.large
-* 11-100 nodes: m3.xlarge
-* 101-250 nodes: m3.2xlarge
-* 251-500 nodes: c4.4xlarge
-* more than 500 nodes: c4.8xlarge
-
-Note that these master node sizes are currently only set at cluster startup time, and are not adjusted if you later scale your cluster up or down (e.g. manually removing or adding nodes, or using a cluster autoscaler).
-
-### Addon Resources
-
-To prevent memory leaks or other resource issues in [cluster addons](https://releases.k8s.io/{{page.githubbranch}}/cluster/addons) from consuming all the resources available on a node, Kubernetes sets resource limits on addon containers to limit the CPU and Memory resources they can consume (See PR [#10653](http://pr.k8s.io/10653/files) and [#10778](http://pr.k8s.io/10778/files)).
-
-For [example](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/cluster/saltbase/salt/fluentd-gcp/fluentd-gcp.yaml):
-
-```yaml
- containers:
- - name: fluentd-cloud-logging
- image: gcr.io/google_containers/fluentd-gcp:1.16
- resources:
- limits:
- cpu: 100m
- memory: 200Mi
-```
-
-Except for Heapster, these limits are static and are based on data we collected from addons running on 4-node clusters (see [#10335](http://issue.k8s.io/10335#issuecomment-117861225)). The addons consume a lot more resources when running on large deployment clusters (see [#5880](http://issue.k8s.io/5880#issuecomment-113984085)). So, if a large cluster is deployed without adjusting these values, the addons may continuously get killed because they keep hitting the limits.
-
-To avoid running into cluster addon resource issues, when creating a cluster with many nodes, consider the following:
-
-* Scale memory and CPU limits for each of the following addons, if used, as you scale up the size of cluster (there is one replica of each handling the entire cluster so memory and CPU usage tends to grow proportionally with size/load on cluster):
- * [InfluxDB and Grafana](http://releases.k8s.io/{{page.githubbranch}}/cluster/addons/cluster-monitoring/influxdb/influxdb-grafana-controller.yaml)
- * [skydns, kube2sky, and dns etcd](http://releases.k8s.io/{{page.githubbranch}}/cluster/addons/dns/skydns-rc.yaml.in)
- * [Kibana](http://releases.k8s.io/{{page.githubbranch}}/cluster/addons/fluentd-elasticsearch/kibana-controller.yaml)
-* Scale number of replicas for the following addons, if used, along with the size of cluster (there are multiple replicas of each so increasing replicas should help handle increased load, but, since load per replica also increases slightly, also consider increasing CPU/memory limits):
- * [elasticsearch](http://releases.k8s.io/{{page.githubbranch}}/cluster/addons/fluentd-elasticsearch/es-controller.yaml)
-* Increase memory and CPU limits slightly for each of the following addons, if used, along with the size of cluster (there is one replica per node but CPU/memory usage increases slightly along with cluster load/size as well):
- * [FluentD with ElasticSearch Plugin](http://releases.k8s.io/{{page.githubbranch}}/cluster/saltbase/salt/fluentd-es/fluentd-es.yaml)
- * [FluentD with GCP Plugin](http://releases.k8s.io/{{page.githubbranch}}/cluster/saltbase/salt/fluentd-gcp/fluentd-gcp.yaml)
-
-Heapster's resource limits are set dynamically based on the initial size of your cluster (see [#16185](http://issue.k8s.io/16185)
-and [#22940](http://issue.k8s.io/22940)). If you find that Heapster is running
-out of resources, you should adjust the formulas that compute heapster memory request (see those PRs for details).
-
-For directions on how to detect if addon containers are hitting resource limits, see the [Troubleshooting section of Compute Resources](/docs/user-guide/compute-resources/#troubleshooting).
-
-In the [future](http://issue.k8s.io/13048), we anticipate to set all cluster addon resource limits based on cluster size, and to dynamically adjust them if you grow or shrink your cluster.
-We welcome PRs that implement those features.
-
-### Allowing minor node failure at startup
-
-For various reasons (see [#18969](https://github.com/kubernetes/kubernetes/issues/18969) for more details) running
-`kube-up.sh` with a very large `NUM_NODES` may fail due to a very small number of nodes not coming up properly.
-Currently you have two choices: restart the cluster (`kube-down.sh` and then `kube-up.sh` again), or before
-running `kube-up.sh` set the environment variable `ALLOWED_NOTREADY_NODES` to whatever value you feel comfortable
-with. This will allow `kube-up.sh` to succeed with fewer than `NUM_NODES` coming up. Depending on the
-reason for the failure, those additional nodes may join later or the cluster may remain at a size of
-`NUM_NODES - ALLOWED_NOTREADY_NODES`.
+---
+assignees:
+- davidopp
+- lavalamp
+title: Building Large Clusters
+---
+
+## Support
+
+At {{page.version}}, Kubernetes supports clusters with up to 1000 nodes. More specifically, we support configurations that meet *all* of the following criteria:
+
+* No more than 2000 nodes
+* No more than 60000 total pods
+* No more than 120000 total containers
+* No more than 100 pods per node
+
+
+
+* TOC
+{:toc}
+
+## Setup
+
+A cluster is a set of nodes (physical or virtual machines) running Kubernetes agents, managed by a "master" (the cluster-level control plane).
+
+Normally the number of nodes in a cluster is controlled by the the value `NUM_NODES` in the platform-specific `config-default.sh` file (for example, see [GCE's `config-default.sh`](http://releases.k8s.io/{{page.githubbranch}}/cluster/gce/config-default.sh)).
+
+Simply changing that value to something very large, however, may cause the setup script to fail for many cloud providers. A GCE deployment, for example, will run in to quota issues and fail to bring the cluster up.
+
+When setting up a large Kubernetes cluster, the following issues must be considered.
+
+### Quota Issues
+
+To avoid running into cloud provider quota issues, when creating a cluster with many nodes, consider:
+
+* Increase the quota for things like CPU, IPs, etc.
+ * In [GCE, for example,](https://cloud.google.com/compute/docs/resource-quotas) you'll want to increase the quota for:
+ * CPUs
+ * VM instances
+ * Total persistent disk reserved
+ * In-use IP addresses
+ * Firewall Rules
+ * Forwarding rules
+ * Routes
+ * Target pools
+* Gating the setup script so that it brings up new node VMs in smaller batches with waits in between, because some cloud providers rate limit the creation of VMs.
+
+### Etcd storage
+
+To improve performance of large clusters, we store events in a separate dedicated etcd instance.
+
+When creating a cluster, existing salt scripts:
+
+* start and configure additional etcd instance
+* configure api-server to use it for storing events
+
+### Size of master and master components
+
+On GCE/GKE and AWS, `kube-up` automatically configures the proper VM size for your master depending on the number of nodes
+in your cluster. On other providers, you will need to configure it manually. For reference, the sizes we use on GCE are
+
+* 1-5 nodes: n1-standard-1
+* 6-10 nodes: n1-standard-2
+* 11-100 nodes: n1-standard-4
+* 101-250 nodes: n1-standard-8
+* 251-500 nodes: n1-standard-16
+* more than 500 nodes: n1-standard-32
+
+And the sizes we use on AWS are
+
+* 1-5 nodes: m3.medium
+* 6-10 nodes: m3.large
+* 11-100 nodes: m3.xlarge
+* 101-250 nodes: m3.2xlarge
+* 251-500 nodes: c4.4xlarge
+* more than 500 nodes: c4.8xlarge
+
+Note that these master node sizes are currently only set at cluster startup time, and are not adjusted if you later scale your cluster up or down (e.g. manually removing or adding nodes, or using a cluster autoscaler).
+
+### Addon Resources
+
+To prevent memory leaks or other resource issues in [cluster addons](https://releases.k8s.io/{{page.githubbranch}}/cluster/addons) from consuming all the resources available on a node, Kubernetes sets resource limits on addon containers to limit the CPU and Memory resources they can consume (See PR [#10653](http://pr.k8s.io/10653/files) and [#10778](http://pr.k8s.io/10778/files)).
+
+For [example](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/cluster/saltbase/salt/fluentd-gcp/fluentd-gcp.yaml):
+
+```yaml
+ containers:
+ - name: fluentd-cloud-logging
+ image: gcr.io/google_containers/fluentd-gcp:1.16
+ resources:
+ limits:
+ cpu: 100m
+ memory: 200Mi
+```
+
+Except for Heapster, these limits are static and are based on data we collected from addons running on 4-node clusters (see [#10335](http://issue.k8s.io/10335#issuecomment-117861225)). The addons consume a lot more resources when running on large deployment clusters (see [#5880](http://issue.k8s.io/5880#issuecomment-113984085)). So, if a large cluster is deployed without adjusting these values, the addons may continuously get killed because they keep hitting the limits.
+
+To avoid running into cluster addon resource issues, when creating a cluster with many nodes, consider the following:
+
+* Scale memory and CPU limits for each of the following addons, if used, as you scale up the size of cluster (there is one replica of each handling the entire cluster so memory and CPU usage tends to grow proportionally with size/load on cluster):
+ * [InfluxDB and Grafana](http://releases.k8s.io/{{page.githubbranch}}/cluster/addons/cluster-monitoring/influxdb/influxdb-grafana-controller.yaml)
+ * [skydns, kube2sky, and dns etcd](http://releases.k8s.io/{{page.githubbranch}}/cluster/addons/dns/skydns-rc.yaml.in)
+ * [Kibana](http://releases.k8s.io/{{page.githubbranch}}/cluster/addons/fluentd-elasticsearch/kibana-controller.yaml)
+* Scale number of replicas for the following addons, if used, along with the size of cluster (there are multiple replicas of each so increasing replicas should help handle increased load, but, since load per replica also increases slightly, also consider increasing CPU/memory limits):
+ * [elasticsearch](http://releases.k8s.io/{{page.githubbranch}}/cluster/addons/fluentd-elasticsearch/es-controller.yaml)
+* Increase memory and CPU limits slightly for each of the following addons, if used, along with the size of cluster (there is one replica per node but CPU/memory usage increases slightly along with cluster load/size as well):
+ * [FluentD with ElasticSearch Plugin](http://releases.k8s.io/{{page.githubbranch}}/cluster/saltbase/salt/fluentd-es/fluentd-es.yaml)
+ * [FluentD with GCP Plugin](http://releases.k8s.io/{{page.githubbranch}}/cluster/saltbase/salt/fluentd-gcp/fluentd-gcp.yaml)
+
+Heapster's resource limits are set dynamically based on the initial size of your cluster (see [#16185](http://issue.k8s.io/16185)
+and [#22940](http://issue.k8s.io/22940)). If you find that Heapster is running
+out of resources, you should adjust the formulas that compute heapster memory request (see those PRs for details).
+
+For directions on how to detect if addon containers are hitting resource limits, see the [Troubleshooting section of Compute Resources](/docs/user-guide/compute-resources/#troubleshooting).
+
+In the [future](http://issue.k8s.io/13048), we anticipate to set all cluster addon resource limits based on cluster size, and to dynamically adjust them if you grow or shrink your cluster.
+We welcome PRs that implement those features.
+
+### Allowing minor node failure at startup
+
+For various reasons (see [#18969](https://github.com/kubernetes/kubernetes/issues/18969) for more details) running
+`kube-up.sh` with a very large `NUM_NODES` may fail due to a very small number of nodes not coming up properly.
+Currently you have two choices: restart the cluster (`kube-down.sh` and then `kube-up.sh` again), or before
+running `kube-up.sh` set the environment variable `ALLOWED_NOTREADY_NODES` to whatever value you feel comfortable
+with. This will allow `kube-up.sh` to succeed with fewer than `NUM_NODES` coming up. Depending on the
+reason for the failure, those additional nodes may join later or the cluster may remain at a size of
+`NUM_NODES - ALLOWED_NOTREADY_NODES`.
diff --git a/docs/admin/cluster-management.md b/docs/admin/cluster-management.md
index 97362c4bab..5bcdb8b8f3 100644
--- a/docs/admin/cluster-management.md
+++ b/docs/admin/cluster-management.md
@@ -2,7 +2,7 @@
assignees:
- lavalamp
- thockin
-
+title: Cluster Management Guide
---
* TOC
@@ -92,7 +92,7 @@ an extended period of time (10min but it may change in the future).
Cluster autoscaler is configured per instance group (GCE) or node pool (GKE).
If you are using GCE then you can either enable it while creating a cluster with kube-up.sh script.
-To configure cluser autoscaler you have to set 3 environment variables:
+To configure cluster autoscaler you have to set 3 environment variables:
* `KUBE_ENABLE_CLUSTER_AUTOSCALER` - it enables cluster autoscaler if set to true.
* `KUBE_AUTOSCALER_MIN_NODES` - minimum number of nodes in the cluster.
@@ -180,7 +180,7 @@ For the purposes of these flags, _legacy_ APIs are those APIs which have been ex
The objects that are stored to disk for a cluster's internal representation of the Kubernetes resources active in the cluster are written using a particular version of the API.
When the supported API changes, these objects may need to be rewritten in the newer API. Failure to do this will eventually result in resources that are no longer decodable or usable
-by the kubernetes API server.
+by the Kubernetes API server.
`KUBE_API_VERSIONS` environment variable for the `kube-apiserver` binary which controls the API versions that are supported in the cluster. The first version in the list is used as the cluster's storage version. Hence, to set a specific version as the storage version, bring it to the front of list of versions in the value of `KUBE_API_VERSIONS`. You need to restart the `kube-apiserver` binary
for changes to this variable to take effect.
diff --git a/docs/admin/cluster-troubleshooting.md b/docs/admin/cluster-troubleshooting.md
index 8bab089ce6..ff8358a7a9 100644
--- a/docs/admin/cluster-troubleshooting.md
+++ b/docs/admin/cluster-troubleshooting.md
@@ -1,7 +1,7 @@
---
assignees:
- davidopp
-
+title: Troubleshooting Clusters
---
This doc is about cluster troubleshooting; we assume you have already ruled out your application as the root cause of the
@@ -89,7 +89,7 @@ Mitigations:
- Mitigates: Apiserver VM shutdown or apiserver crashing
- Mitigates: Supporting services VM shutdown or crashes
-- Action use IaaS providers reliable storage (e.g GCE PD or AWS EBS volume) for VMs with apiserver+etcd
+- Action use IaaS providers reliable storage (e.g. GCE PD or AWS EBS volume) for VMs with apiserver+etcd
- Mitigates: Apiserver backing storage lost
- Action: Use (experimental) [high-availability](/docs/admin/high-availability) configuration
diff --git a/docs/admin/daemons.md b/docs/admin/daemons.md
index be3137bc93..4682a62b71 100644
--- a/docs/admin/daemons.md
+++ b/docs/admin/daemons.md
@@ -1,26 +1,26 @@
---
assignees:
- erictune
-
+title: Daemon Sets
---
* TOC
{:toc}
-## What is a _Daemon Set_?
+## What is a DaemonSet?
-A _Daemon Set_ ensures that all (or some) nodes run a copy of a pod. As nodes are added to the
+A _DaemonSet_ ensures that all (or some) nodes run a copy of a pod. As nodes are added to the
cluster, pods are added to them. As nodes are removed from the cluster, those pods are garbage
-collected. Deleting a Daemon Set will clean up the pods it created.
+collected. Deleting a DaemonSet will clean up the pods it created.
-Some typical uses of a Daemon Set are:
+Some typical uses of a DaemonSet are:
- running a cluster storage daemon, such as `glusterd`, `ceph`, on each node.
- running a logs collection daemon on every node, such as `fluentd` or `logstash`.
- running a node monitoring daemon on every node, such as [Prometheus Node Exporter](
https://github.com/prometheus/node_exporter), `collectd`, New Relic agent, or Ganglia `gmond`.
-In a simple case, one Daemon Set, covering all nodes, would be used for each type of daemon.
+In a simple case, one DaemonSet, covering all nodes, would be used for each type of daemon.
A more complex setup might use multiple DaemonSets would be used for a single type of daemon,
but with different flags and/or different memory and cpu requests for different hardware types.
@@ -74,7 +74,7 @@ a node for testing.
If you specify a `.spec.template.spec.nodeSelector`, then the DaemonSet controller will
create pods on nodes which match that [node
-selector](https://github.com/kubernetes/kubernetes.github.io/tree/{{page.docsbranch}}/docs/user-guide/node-selection).
+selector](/docs/user-guide/node-selection/).
If you specify a `scheduler.alpha.kubernetes.io/affinity` annotation in `.spec.template.metadata.annotations`,
then DaemonSet controller will create pods on nodes which match that [node affinity](../../user-guide/node-selection/#alpha-feature-in-kubernetes-v12-node-affinity).
@@ -88,18 +88,17 @@ created by the Daemon controller have the machine already selected (`.spec.nodeN
when the pod is created, so it is ignored by the scheduler). Therefore:
- the [`unschedulable`](/docs/admin/node/#manual-node-administration) field of a node is not respected
- by the daemon set controller.
- - daemon set controller can make pods even when the scheduler has not been started, which can help cluster
+ by the DaemonSet controller.
+ - DaemonSet controller can make pods even when the scheduler has not been started, which can help cluster
bootstrap.
## Communicating with DaemonSet Pods
Some possible patterns for communicating with pods in a DaemonSet are:
-- **Push**: Pods in the Daemon Set are configured to send updates to another service, such
+- **Push**: Pods in the DaemonSet are configured to send updates to another service, such
as a stats database. They do not have clients.
-- **NodeIP and Known Port**: Pods in the Daemon Set use a `hostPort`, so that the pods are reachable
- via the node IPs. Clients knows the the list of nodes ips somehow, and know the port by convention.
+- **NodeIP and Known Port**: Pods in the DaemonSet use a `hostPort`, so that the pods are reachable via the node IPs. Clients know the list of nodes ips somehow, and know the port by convention.
- **DNS**: Create a [headless service](/docs/user-guide/services/#headless-services) with the same pod selector,
and then discover DaemonSets using the `endpoints` resource or retrieve multiple A records from
DNS.
@@ -126,11 +125,11 @@ You cannot update a DaemonSet.
Support for updating DaemonSets and controlled updating of nodes is planned.
-## Alternatives to Daemon Set
+## Alternatives to DaemonSet
### Init Scripts
-It is certainly possible to run daemon processes by directly starting them on a node (e.g using
+It is certainly possible to run daemon processes by directly starting them on a node (e.g. using
`init`, `upstartd`, or `systemd`). This is perfectly fine. However, there are several advantages to
running such processes via a DaemonSet:
@@ -145,9 +144,9 @@ running such processes via a DaemonSet:
### Bare Pods
It is possible to create pods directly which specify a particular node to run on. However,
-a Daemon Set replaces pods that are deleted or terminated for any reason, such as in the case of
+a DaemonSet replaces pods that are deleted or terminated for any reason, such as in the case of
node failure or disruptive node maintenance, such as a kernel upgrade. For this reason, you should
-use a Daemon Set rather than creating individual pods.
+use a DaemonSet rather than creating individual pods.
### Static Pods
@@ -159,7 +158,7 @@ in cluster bootstrapping cases. Also, static pods may be deprecated in the futu
### Replication Controller
-Daemon Set are similar to [Replication Controllers](/docs/user-guide/replication-controller) in that
+DaemonSet are similar to [Replication Controllers](/docs/user-guide/replication-controller) in that
they both create pods, and those pods have processes which are not expected to terminate (e.g. web servers,
storage servers).
diff --git a/docs/admin/disruptions.md b/docs/admin/disruptions.md
index 0743b565e6..6c4ee7df4a 100644
--- a/docs/admin/disruptions.md
+++ b/docs/admin/disruptions.md
@@ -1,7 +1,7 @@
---
assignees:
-- mml
-
+- davidopp
+title: Pod Disruption Budget
---
This guide is for anyone wishing to specify safety constraints on pods or anyone
wishing to write software (typically automation software) that respects those
@@ -59,7 +59,7 @@ itself. To attempt an eviction (perhaps more REST-precisely, to attempt to
```json
{
- "apiVersion": "policy/v1alpha1",
+ "apiVersion": "policy/v1beta1",
"kind": "Eviction",
"metadata": {
"name": "quux",
diff --git a/docs/admin/dns.md b/docs/admin/dns.md
index d75acfa093..407c439b73 100644
--- a/docs/admin/dns.md
+++ b/docs/admin/dns.md
@@ -1,9 +1,8 @@
---
assignees:
-- ArtfulCoder
- davidopp
-- lavalamp
-
+- thockin
+title: Using DNS Pods and Services
---
## Introduction
@@ -60,7 +59,7 @@ of the form `auto-generated-name.my-svc.my-namespace.svc.cluster.local`.
### Backwards compatibility
-Previous versions of kube-dns made names of the for
+Previous versions of kube-dns made names of the form
`my-svc.my-namespace.cluster.local` (the 'svc' level was added later). This
is no longer supported.
@@ -70,14 +69,14 @@ is no longer supported.
When enabled, pods are assigned a DNS A record in the form of `pod-ip-address.my-namespace.pod.cluster.local`.
-For example, a pod with ip `1.2.3.4` in the namespace `default` with a dns name of `cluster.local` would have an entry: `1-2-3-4.default.pod.cluster.local`.
+For example, a pod with ip `1.2.3.4` in the namespace `default` with a DNS name of `cluster.local` would have an entry: `1-2-3-4.default.pod.cluster.local`.
#### A Records and hostname based on Pod's hostname and subdomain fields
Currently when a pod is created, its hostname is the Pod's `metadata.name` value.
With v1.2, users can specify a Pod annotation, `pod.beta.kubernetes.io/hostname`, to specify what the Pod's hostname should be.
-The Pod annotation, if specified, takes precendence over the Pod's name, to be the hostname of the pod.
+The Pod annotation, if specified, takes precedence over the Pod's name, to be the hostname of the pod.
For example, given a Pod with annotation `pod.beta.kubernetes.io/hostname: my-pod-name`, the Pod will have its hostname set to "my-pod-name".
With v1.3, the PodSpec has a `hostname` field, which can be used to specify the Pod's hostname. This field value takes precedence over the
@@ -94,13 +93,43 @@ Example:
```yaml
apiVersion: v1
+kind: Service
+metadata:
+ name: default-subdomain
+spec:
+ selector:
+ name: busybox
+ clusterIP: None
+ ports:
+ - name: foo # Actually, no port is needed.
+ port: 1234
+ targetPort: 1234
+---
+apiVersion: v1
kind: Pod
metadata:
- name: busybox
- namespace: default
+ name: busybox1
+ labels:
+ name: busybox
spec:
hostname: busybox-1
- subdomain: default
+ subdomain: default-subdomain
+ containers:
+ - image: busybox
+ command:
+ - sleep
+ - "3600"
+ name: busybox
+---
+apiVersion: v1
+kind: Pod
+metadata:
+ name: busybox2
+ labels:
+ name: busybox
+spec:
+ hostname: busybox-2
+ subdomain: default-subdomain
containers:
- image: busybox
command:
@@ -110,11 +139,11 @@ spec:
```
If there exists a headless service in the same namespace as the pod and with the same name as the subdomain, the cluster's KubeDNS Server also returns an A record for the Pod's fully qualified hostname.
-Given a Pod with the hostname set to "foo" and the subdomain set to "bar", and a headless Service named "bar" in the same namespace, the pod will see it's own FQDN as "foo.bar.my-namespace.svc.cluster.local". DNS serves an A record at that name, pointing to the Pod's IP.
+Given a Pod with the hostname set to "busybox-1" and the subdomain set to "default-subdomain", and a headless Service named "default-subdomain" in the same namespace, the pod will see it's own FQDN as "busybox-1.default-subdomain.my-namespace.svc.cluster.local". DNS serves an A record at that name, pointing to the Pod's IP. Both pods "busybox1" and "busybox2" can have their distinct A records.
-With v1.2, the Endpoints object also has a new annotation `endpoints.beta.kubernetes.io/hostnames-map`. Its value is the json representation of map[string(IP)][endpoints.HostRecord], for example: '{"10.245.1.6":{HostName: "my-webserver"}}'.
+As of Kubernetes v1.2, the Endpoints object also has the annotation `endpoints.beta.kubernetes.io/hostnames-map`. Its value is the json representation of map[string(IP)][endpoints.HostRecord], for example: '{"10.245.1.6":{HostName: "my-webserver"}}'.
If the Endpoints are for a headless service, an A record is created with the format
...svc.
-For the example json, if endpoints are for a headless service named "bar", and one of the endpoints has IP "10.245.1.6", an A is created with the name "my-webserver.bar.my-namespace.svc.cluster.local" and the A record lookup would return "10.245.1.6".
+For the example json, if endpoints are for a headless service named "bar", and one of the endpoints has IP "10.245.1.6", an A record is created with the name "my-webserver.bar.my-namespace.svc.cluster.local" and the A record lookup would return "10.245.1.6".
This endpoints annotation generally does not need to be specified by end-users, but can used by the internal service controller to deliver the aforementioned feature.
With v1.3, The Endpoints object can specify the `hostname` for any endpoint, along with its IP. The hostname field takes precedence over the hostname value
@@ -171,7 +200,7 @@ busybox 1/1 Running 0
Once that pod is running, you can exec nslookup in that environment:
```
-kubectl exec busybox -- nslookup kubernetes.default
+kubectl exec -ti busybox -- nslookup kubernetes.default
```
You should see something like:
@@ -194,10 +223,10 @@ If the nslookup command fails, check the following:
Take a look inside the resolv.conf file. (See "Inheriting DNS from the node" and "Known issues" below for more information)
```
-cat /etc/resolv.conf
+kubectl exec busybox cat /etc/resolv.conf
```
-Verify that the search path and name server are set up like the following (note that seach path may vary for different cloud providers):
+Verify that the search path and name server are set up like the following (note that search path may vary for different cloud providers):
```
search default.svc.cluster.local svc.cluster.local cluster.local google.internal c.gce_project_id.internal
@@ -210,7 +239,7 @@ options ndots:5
Errors such as the following indicate a problem with the kube-dns add-on or associated Services:
```
-$ kubectl exec busybox -- nslookup kubernetes.default
+$ kubectl exec -ti busybox -- nslookup kubernetes.default
Server: 10.0.0.10
Address 1: 10.0.0.10
@@ -220,7 +249,7 @@ nslookup: can't resolve 'kubernetes.default'
or
```
-$ kubectl exec busybox -- nslookup kubernetes.default
+$ kubectl exec -ti busybox -- nslookup kubernetes.default
Server: 10.0.0.10
Address 1: 10.0.0.10 kube-dns.kube-system.svc.cluster.local
@@ -244,21 +273,21 @@ kube-dns-v19-ezo1y 3/3 Running 0
...
```
-If you see that no pod is running or that the pod has failed/completed, the dns add-on may not be deployed by default in your current environment and you will have to deploy it manually.
+If you see that no pod is running or that the pod has failed/completed, the DNS add-on may not be deployed by default in your current environment and you will have to deploy it manually.
#### Check for Errors in the DNS pod
Use `kubectl logs` command to see logs for the DNS daemons.
```
-kubectl logs --namespace=kube-system $(kubectl get pods --namespace=kube-system -l k8s-app=kube-dns -o name) -c kubedns
+kubectl logs --namespace=kube-system $(kubectl get pods --namespace=kube-system -l k8s-app=kube-dns -o name) -c kube-dns
kubectl logs --namespace=kube-system $(kubectl get pods --namespace=kube-system -l k8s-app=kube-dns -o name) -c dnsmasq
kubectl logs --namespace=kube-system $(kubectl get pods --namespace=kube-system -l k8s-app=kube-dns -o name) -c healthz
```
See if there is any suspicious log. W, E, F letter at the beginning represent Warning, Error and Failure. Please search for entries that have these as the logging level and use [kubernetes issues](https://github.com/kubernetes/kubernetes/issues) to report unexpected errors.
-#### Is dns service up?
+#### Is DNS service up?
Verify that the DNS service is up by using the `kubectl get service` command.
@@ -277,7 +306,7 @@ kube-dns 10.0.0.10 53/UDP,53/TCP 1h
If you have created the service or in the case it should be created by default but it does not appear, see this [debugging services page](http://kubernetes.io/docs/user-guide/debugging-services/) for more information.
-#### Are dns endpoints exposed?
+#### Are DNS endpoints exposed?
You can verify that dns endpoints are exposed by using the `kubectl get endpoints` command.
@@ -348,11 +377,13 @@ some of those settings will be lost. As a partial workaround, the node can run
`dnsmasq` which will provide more `nameserver` entries, but not more `search`
entries. You can also use kubelet's `--resolv-conf` flag.
-If you are using Alpine version 3.3 or earlier as your base image, dns may not
+If you are using Alpine version 3.3 or earlier as your base image, DNS may not
work properly owing to a known issue with Alpine. Check [here](https://github.com/kubernetes/kubernetes/issues/30215)
for more information.
## References
-- [Docs for the DNS cluster addon](http://releases.k8s.io/{{page.githubbranch}}/build-tools/kube-dns/README.md)
+- [Docs for the DNS cluster addon](http://releases.k8s.io/{{page.githubbranch}}/cluster/addons/dns/README.md)
+## What's next
+- [Autoscaling the DNS Service in a Cluster](/docs/tasks/administer-cluster/dns-horizontal-autoscaling/).
diff --git a/docs/admin/etcd.md b/docs/admin/etcd.md
index 14b36a33be..41a3e35ddb 100644
--- a/docs/admin/etcd.md
+++ b/docs/admin/etcd.md
@@ -1,11 +1,10 @@
---
assignees:
- lavalamp
-
+title: Configuring Kubernetes Use of etcd
---
-
-[etcd](https://coreos.com/etcd/docs/2.2.1/) is a highly-available key value
+[etcd](https://coreos.com/etcd/docs/latest/) is a highly-available key value
store which Kubernetes uses for persistent storage of all of its REST API
objects.
diff --git a/docs/admin/federation-apiserver.md b/docs/admin/federation-apiserver.md
index 00fb08c1a0..9eb760d087 100644
--- a/docs/admin/federation-apiserver.md
+++ b/docs/admin/federation-apiserver.md
@@ -1,4 +1,6 @@
---
+title: federation-apiserver
+notitle: true
---
## federation-apiserver
@@ -20,10 +22,11 @@ federation-apiserver
### Options
```
- --admission-control string Ordered list of plug-ins to do admission control of resources into cluster. Comma-delimited list of: AlwaysAdmit, AlwaysDeny, NamespaceLifecycle. (default "AlwaysAdmit")
+ --admission-control string Ordered list of plug-ins to do admission control of resources into cluster. Comma-delimited list of: AlwaysAdmit, AlwaysDeny, NamespaceLifecycle, OwnerReferencesPermissionEnforcement. (default "AlwaysAdmit")
--admission-control-config-file string File with admission control configuration.
- --advertise-address value The IP address on which to advertise the apiserver to members of the cluster. This address must be reachable by the rest of the cluster. If blank, the --bind-address will be used. If --bind-address is unspecified, the host's default interface will be used.
- --apiserver-count int The number of apiservers running in the cluster. (default 1)
+ --advertise-address ip The IP address on which to advertise the apiserver to members of the cluster. This address must be reachable by the rest of the cluster. If blank, the --bind-address will be used. If --bind-address is unspecified, the host's default interface will be used.
+ --anonymous-auth Enables anonymous requests to the secure port of the API server. Requests that are not rejected by another authentication method are treated as anonymous requests. Anonymous requests have a username of system:anonymous, and a group name of system:unauthenticated. (default true)
+ --apiserver-count int The number of apiservers running in the cluster. Must be a positive number. (default 1)
--audit-log-maxage int The maximum number of days to retain old audit log files based on the timestamp encoded in their filename.
--audit-log-maxbackup int The maximum number of old audit log files to retain.
--audit-log-maxsize int The maximum size in megabytes of the audit log file before it gets rotated. Defaults to 100MB.
@@ -35,64 +38,71 @@ federation-apiserver
--authorization-webhook-cache-unauthorized-ttl duration The duration to cache 'unauthorized' responses from the webhook authorizer. Default is 30s. (default 30s)
--authorization-webhook-config-file string File with webhook configuration in kubeconfig format, used with --authorization-mode=Webhook. The API server will query the remote service to determine access on the API server's secure port.
--basic-auth-file string If set, the file that will be used to admit requests to the secure port of the API server via http basic authentication.
- --bind-address value The IP address on which to listen for the --secure-port port. The associated interface(s) must be reachable by the rest of the cluster, and by CLI/web clients. If blank, all interfaces will be used (0.0.0.0). (default 0.0.0.0)
+ --bind-address ip The IP address on which to listen for the --secure-port port. The associated interface(s) must be reachable by the rest of the cluster, and by CLI/web clients. If blank, all interfaces will be used (0.0.0.0). (default 0.0.0.0)
--cert-dir string The directory where the TLS certs are located (by default /var/run/kubernetes). If --tls-cert-file and --tls-private-key-file are provided, this flag will be ignored. (default "/var/run/kubernetes")
--client-ca-file string If set, any request presenting a client certificate signed by one of the authorities in the client-ca-file is authenticated with an identity corresponding to the CommonName of the client certificate.
--cloud-config string The path to the cloud provider configuration file. Empty string for no configuration file.
--cloud-provider string The provider for cloud services. Empty string for no provider.
- --cors-allowed-origins value List of allowed origins for CORS, comma separated. An allowed origin can be a regular expression to support subdomain matching. If this list is empty CORS will not be enabled. (default [])
+ --contention-profiling Enable contention profiling. Requires --profiling to be set to work.
+ --cors-allowed-origins stringSlice List of allowed origins for CORS, comma separated. An allowed origin can be a regular expression to support subdomain matching. If this list is empty CORS will not be enabled.
--delete-collection-workers int Number of workers spawned for DeleteCollection call. These are used to speed up namespace cleanup. (default 1)
--deserialization-cache-size int Number of deserialized json objects to cache in memory.
+ --enable-garbage-collector Enables the generic garbage collector. MUST be synced with the corresponding flag of the kube-controller-manager. (default true)
--enable-swagger-ui Enables swagger ui on the apiserver at /swagger-ui
--etcd-cafile string SSL Certificate Authority file used to secure etcd communication.
--etcd-certfile string SSL certification file used to secure etcd communication.
--etcd-keyfile string SSL key file used to secure etcd communication.
--etcd-prefix string The prefix for all resource paths in etcd. (default "/registry")
--etcd-quorum-read If true, enable quorum read.
- --etcd-servers value List of etcd servers to connect with (http://ip:port), comma separated. (default [])
- --etcd-servers-overrides value Per-resource etcd servers overrides, comma separated. The individual override format: group/resource#servers, where servers are http://ip:port, semicolon separated. (default [])
+ --etcd-servers stringSlice List of etcd servers to connect with (scheme://ip:port), comma separated.
+ --etcd-servers-overrides stringSlice Per-resource etcd servers overrides, comma separated. The individual override format: group/resource#servers, where servers are http://ip:port, semicolon separated.
--event-ttl duration Amount of time to retain events. Default is 1h. (default 1h0m0s)
+ --experimental-keystone-ca-file string If set, the Keystone server's certificate will be verified by one of the authorities in the experimental-keystone-ca-file, otherwise the host's root CA set will be used.
--experimental-keystone-url string If passed, activates the keystone authentication plugin.
--external-hostname string The hostname to use when generating externalized URLs for this master (e.g. Swagger API Docs).
- --feature-gates value A set of key=value pairs that describe feature gates for alpha/experimental features. Options are:
+ --feature-gates mapStringBool A set of key=value pairs that describe feature gates for alpha/experimental features. Options are:
AllAlpha=true|false (ALPHA - default=false)
-AllowExtTrafficLocalEndpoints=true|false (ALPHA - default=false)
+AllowExtTrafficLocalEndpoints=true|false (BETA - default=true)
AppArmor=true|false (BETA - default=true)
DynamicKubeletConfig=true|false (ALPHA - default=false)
DynamicVolumeProvisioning=true|false (ALPHA - default=true)
- --insecure-bind-address value The IP address on which to serve the --insecure-port (set to 0.0.0.0 for all interfaces). Defaults to localhost. (default 127.0.0.1)
+ExperimentalHostUserNamespaceDefaulting=true|false (ALPHA - default=false)
+StreamingProxyRedirects=true|false (ALPHA - default=false)
+ --insecure-allow-any-token username/group1,group2 If set, your server will be INSECURE. Any token will be allowed and user information will be parsed from the token as username/group1,group2
+ --insecure-bind-address ip The IP address on which to serve the --insecure-port (set to 0.0.0.0 for all interfaces). Defaults to localhost. (default 127.0.0.1)
--insecure-port int The port on which to serve unsecured, unauthenticated access. Default 8080. It is assumed that firewall rules are set up such that this port is not reachable from outside of the cluster and that port 443 on the cluster's public address is proxied to this port. This is performed by nginx in the default setup. (default 8080)
--kubernetes-service-node-port int If non-zero, the Kubernetes master service (which apiserver creates/maintains) will be of type NodePort, using this as the value of the port. If zero, the Kubernetes master service will be of type ClusterIP.
--long-running-request-regexp string A regular expression matching long running requests which should be excluded from maximum inflight request handling. (default "(/|^)((watch|proxy)(/|$)|(logs?|portforward|exec|attach)/?$)")
- --master-service-namespace string The namespace from which the kubernetes master services should be injected into pods. (default "default")
+ --master-service-namespace string DEPRECATED: the namespace from which the kubernetes master services should be injected into pods. (default "default")
--max-requests-inflight int The maximum number of requests in flight at a given time. When the server exceeds this, it rejects requests. Zero for no limit. (default 400)
--min-request-timeout int An optional field indicating the minimum number of seconds a handler must keep a request open before timing it out. Currently only honored by the watch request handler, which picks a randomized value above this number as the connection timeout, to spread out load. (default 1800)
--oidc-ca-file string If set, the OpenID server's certificate will be verified by one of the authorities in the oidc-ca-file, otherwise the host's root CA set will be used.
--oidc-client-id string The client ID for the OpenID Connect client, must be set if oidc-issuer-url is set.
- --oidc-groups-claim string If provided, the name of a custom OpenID Connect claim for specifying user groups. The claim value is expected to be an array of strings. This flag is experimental, please see the authentication documentation for further details.
+ --oidc-groups-claim string If provided, the name of a custom OpenID Connect claim for specifying user groups. The claim value is expected to be a string or array of strings. This flag is experimental, please see the authentication documentation for further details.
--oidc-issuer-url string The URL of the OpenID issuer, only HTTPS scheme will be accepted. If set, it will be used to verify the OIDC JSON Web Token (JWT).
--oidc-username-claim string The OpenID claim to use as the user name. Note that claims other than the default ('sub') is not guaranteed to be unique and immutable. This flag is experimental, please see the authentication documentation for further details. (default "sub")
--profiling Enable profiling via web interface host:port/debug/pprof/ (default true)
- --runtime-config value A set of key=value pairs that describe runtime configuration that may be passed to apiserver. apis/ key can be used to turn on/off specific api versions. apis// can be used to turn on/off specific resources. api/all and api/legacy are special keys to control all and legacy api versions respectively.
+ --requestheader-allowed-names stringSlice List of client certificate common names to allow to provide usernames in headers specified by --requestheader-username-headers. If empty, any client certificate validated by the authorities in --requestheader-client-ca-file is allowed.
+ --requestheader-client-ca-file string Root certificate bundle to use to verify client certificates on incoming requests before trusting usernames in headers specified by --requestheader-username-headers
+ --requestheader-username-headers stringSlice List of request headers to inspect for usernames. X-Remote-User is common.
+ --runtime-config mapStringString A set of key=value pairs that describe runtime configuration that may be passed to apiserver. apis/ key can be used to turn on/off specific api versions. apis// can be used to turn on/off specific resources. api/all and api/legacy are special keys to control all and legacy api versions respectively.
--secure-port int The port on which to serve HTTPS with authentication and authorization. If 0, don't serve HTTPS at all. (default 6443)
- --service-cluster-ip-range value A CIDR notation IP range from which to assign service cluster IPs. This must not overlap with any IP ranges assigned to nodes for pods.
- --service-node-port-range value A port range to reserve for services with NodePort visibility. Example: '30000-32767'. Inclusive at both ends of the range. (default 30000-32767)
+ --service-cluster-ip-range ipNet A CIDR notation IP range from which to assign service cluster IPs. This must not overlap with any IP ranges assigned to nodes for pods.
+ --service-node-port-range portRange A port range to reserve for services with NodePort visibility. Example: '30000-32767'. Inclusive at both ends of the range. (default 30000-32767)
--storage-backend string The storage backend for persistence. Options: 'etcd2' (default), 'etcd3'.
--storage-media-type string The media type to use to store objects in storage. Defaults to application/json. Some resources may only support a specific media type and will ignore this setting. (default "application/json")
- --storage-versions string The per-group version to store resources in. Specified in the format "group1/version1,group2/version2,...". In the case where objects are moved from one group to the other, you may specify the format "group1=group2/v1beta1,group3/v1beta1,...". You only need to pass the groups you wish to change from the defaults. It defaults to a list of preferred versions of all registered groups, which is derived from the KUBE_API_VERSIONS environment variable. (default "apps/v1alpha1,authentication.k8s.io/v1beta1,authorization.k8s.io/v1beta1,autoscaling/v1,batch/v1,certificates.k8s.io/v1alpha1,componentconfig/v1alpha1,extensions/v1beta1,federation/v1beta1,policy/v1alpha1,rbac.authorization.k8s.io/v1alpha1,storage.k8s.io/v1beta1,v1")
+ --storage-versions string The per-group version to store resources in. Specified in the format "group1/version1,group2/version2,...". In the case where objects are moved from one group to the other, you may specify the format "group1=group2/v1beta1,group3/v1beta1,...". You only need to pass the groups you wish to change from the defaults. It defaults to a list of preferred versions of all registered groups, which is derived from the KUBE_API_VERSIONS environment variable. (default "apps/v1beta1,authentication.k8s.io/v1beta1,authorization.k8s.io/v1beta1,autoscaling/v1,batch/v1,certificates.k8s.io/v1alpha1,componentconfig/v1alpha1,extensions/v1beta1,federation/v1beta1,policy/v1beta1,rbac.authorization.k8s.io/v1alpha1,storage.k8s.io/v1beta1,v1")
--target-ram-mb int Memory limit for apiserver in MB (used to configure sizes of caches, etc.)
- --tls-cert-file string File containing x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If HTTPS serving is enabled, and --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to /var/run/kubernetes.
- --tls-private-key-file string File containing x509 private key matching --tls-cert-file.
+ --tls-ca-file string If set, this certificate authority will used for secure access from Admission Controllers. This must be a valid PEM-encoded CA bundle.
+ --tls-cert-file string File containing the default x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If HTTPS serving is enabled, and --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to /var/run/kubernetes.
+ --tls-private-key-file string File containing the default x509 private key matching --tls-cert-file.
+ --tls-sni-cert-key namedCertKey A pair of x509 certificate and private key file paths, optionally suffixed with a list of domain patterns which are fully qualified domain names, possibly with prefixed wildcard segments. If no domain patterns are provided, the names of the certificate are extracted. Non-wildcard matches trump over wildcard matches, explicit domain patterns trump over extracted names. For multiple key/certificate pairs, use the --tls-sni-cert-key multiple times. Examples: "example.key,example.crt" or "*.foo.com,foo.com:foo.key,foo.crt". (default [])
--token-auth-file string If set, the file that will be used to secure the secure port of the API server via token authentication.
--watch-cache Enable watch caching in the apiserver (default true)
- --watch-cache-sizes value List of watch cache sizes for every resource (pods, nodes, etc.), comma separated. The individual override format: resource#size, where size is a number. It takes effect when watch-cache is enabled. (default [])
+ --watch-cache-sizes stringSlice List of watch cache sizes for every resource (pods, nodes, etc.), comma separated. The individual override format: resource#size, where size is a number. It takes effect when watch-cache is enabled.
```
-###### Auto generated by spf13/cobra on 24-Oct-2016
-
-
-
-
+###### Auto generated by spf13/cobra on 13-Dec-2016
diff --git a/docs/admin/federation-controller-manager.md b/docs/admin/federation-controller-manager.md
index d73dde0b9e..d3dca5bf06 100644
--- a/docs/admin/federation-controller-manager.md
+++ b/docs/admin/federation-controller-manager.md
@@ -1,4 +1,6 @@
---
+title: federation-controller-mananger
+notitle: true
---
## federation-controller-manager
@@ -23,14 +25,14 @@ federation-controller-manager
### Options
```
- --address value The IP address to serve on (set to 0.0.0.0 for all interfaces) (default 0.0.0.0)
+ --address ip The IP address to serve on (set to 0.0.0.0 for all interfaces) (default 0.0.0.0)
--cluster-monitor-period duration The period for syncing ClusterStatus in ClusterController. (default 40s)
--concurrent-replicaset-syncs int The number of ReplicaSets syncing operations that will be done concurrently. Larger number = faster endpoint updating, but more CPU (and network) load (default 10)
--concurrent-service-syncs int The number of service syncing operations that will be done concurrently. Larger number = faster endpoint updating, but more CPU (and network) load (default 10)
- --dns-provider string DNS provider. Valid values are: ["aws-route53" "google-clouddns"]
+ --dns-provider string DNS provider. Valid values are: ["google-clouddns" "aws-route53"]
--dns-provider-config string Path to config file for configuring DNS provider.
--federated-api-burst int Burst to use while talking with federation apiserver (default 30)
- --federated-api-qps value QPS to use while talking with federation apiserver (default 20)
+ --federated-api-qps float32 QPS to use while talking with federation apiserver (default 20)
--federation-name string Federation name.
--kube-api-content-type string ContentType of requests sent to apiserver. Passing application/vnd.kubernetes.protobuf is an experimental feature now.
--kubeconfig string Path to kubeconfig file with authorization and master location information.
@@ -41,14 +43,12 @@ federation-controller-manager
--master string The address of the federation API server (overrides any value in kubeconfig)
--port int The port that the controller-manager's http service runs on (default 10253)
--profiling Enable profiling via web interface host:port/debug/pprof/ (default true)
+ --service-dns-suffix string DNS Suffix to use when publishing federated service names. Defaults to zone-name
+ --zone-id string Zone ID, needed if the zone name is not unique.
--zone-name string Zone name, like example.com.
```
-###### Auto generated by spf13/cobra on 24-Oct-2016
-
-
-
-
+###### Auto generated by spf13/cobra on 13-Dec-2016
diff --git a/docs/admin/federation/index.md b/docs/admin/federation/index.md
index 41d43b9572..59a0199d7b 100644
--- a/docs/admin/federation/index.md
+++ b/docs/admin/federation/index.md
@@ -3,8 +3,9 @@ assignees:
- madhusudancs
- mml
- nikhiljindal
-
+title: Using `federation-up` and `deploy.sh`
---
+
This guide explains how to set up cluster federation that lets us control multiple Kubernetes clusters.
@@ -14,11 +15,11 @@ This guide explains how to set up cluster federation that lets us control multip
## Prerequisites
This guide assumes that you have a running Kubernetes cluster.
-If not, then head over to the [getting started guides](/docs/getting-started-guides/) to bring up a cluster.
+If you need to start a new cluster, see the [getting started guides](/docs/getting-started-guides/) for instructions on bringing a cluster up.
-This guide also assumes that you have a Kubernetes release
-[downloaded from here](/docs/getting-started-guides/binary_release/),
-extracted into a directory and all the commands in this guide are run from
+To use the commands in this guide, you must download a Kubernetes release from the
+[getting started binary releases](/docs/getting-started-guides/binary_release/) and
+extract into a directory; all the commands in this guide are run from
that directory.
```shell
@@ -26,8 +27,8 @@ $ curl -L https://github.com/kubernetes/kubernetes/releases/download/v1.4.0/kube
$ cd kubernetes
```
-This guide also assumes that you have an installation of Docker running
-locally, i.e. on the machine where you run the commands described in this
+You must also have a Docker installation running
+locally--meaning on the machine where you run the commands described in this
guide.
## Setting up a federation control plane
@@ -109,7 +110,7 @@ $ KUBE_REGISTRY="gcr.io/myrepository" federation/develop/develop.sh build_image
$ KUBE_REGISTRY="gcr.io/myrepository" federation/develop/develop.sh push
```
-Note: This is going to overwite the values you might have set for
+Note: This is going to overwrite the values you might have set for
`apiserverRegistry`, `apiserverVersion`, `controllerManagerRegistry` and
`controllerManagerVersion` in your `${FEDERATION_OUTPUT_ROOT}/values.yaml`
file. Hence, it is not recommend to customize these values in
@@ -212,47 +213,81 @@ cluster1 Ready 3m
## Updating KubeDNS
-Once the cluster is registered with the federation, you are all set to use it.
-But for the cluster to be able to route federation service requests, you need to restart
-KubeDNS and pass it a `--federations` flag which tells it about valid federation DNS hostnames.
-Format of the flag is like this:
+Once you've registered your cluster with the federation, you'll need to update KubeDNS so that your cluster can route federation service requests. The update method varies depending on your Kubernetes version; on Kubernetes 1.5 or later, you must pass the
+`--federations` flag to kube-dns via the kube-dns config map. In version 1.4 or earlier, you must set the `--federations` flag directly on kube-dns-rc on other clusters.
+
+### Kubernetes 1.5+: Passing federations flag via config map to kube-dns
+
+For Kubernetes clusters of version 1.5+, you can pass the
+`--federations` flag to kube-dns via the kube-dns config map.
+The flag uses the following format:
```
--federations=${FEDERATION_NAME}=${DNS_DOMAIN_NAME}
```
-To update KubeDNS with federations flag, you can edit the existing kubedns replication controller to
-include that flag in pod template spec and then delete the existing pod. Replication controller will
-recreate the pod with updated template.
+To pass this flag to KubeDNS, create a config-map with name `kube-dns` in
+namespace `kube-system`. The configmap should look like the following:
-To find the name of existing kubedns replication controller, run
+```yaml
+apiVersion: v1
+kind: ConfigMap
+metadata:
+ name: kube-dns
+ namespace: kube-system
+data:
+ federations: =
+```
+
+where `` should be replaced by the name you want to give to your
+federation, and
+`federation-domain-name` should be replaced by the domain name you want to use
+in your federation DNS.
+
+You can find more details about config maps in general at
+http://kubernetes.io/docs/user-guide/configmap/.
+
+### Kubernetes 1.4 and earlier: Setting federations flag on kube-dns-rc
+
+If your cluster is running Kubernetes version 1.4 or earlier, you must to restart
+KubeDNS and pass it a `--federations` flag, which tells it about valid federation DNS hostnames.
+The flag uses the following format:
+
+```
+--federations=${FEDERATION_NAME}=${DNS_DOMAIN_NAME}
+```
+
+To update KubeDNS with the `--federations` flag, you can edit the existing kubedns replication controller to
+include that flag in pod template spec, and then delete the existing pod. The replication controller then
+recreates the pod with updated template.
+
+To find the name of existing kubedns replication controller, run the following command:
```shell
$ kubectl get rc --namespace=kube-system
```
-This will list all the replication controllers. Name of the kube-dns replication
-controller will look like `kube-dns-v18`. You can then edit it by running:
+You should see a list of all the replication controllers on the cluster. The kube-dns replication
+controller should have a name similar to `kube-dns-v18`. To edit the replication controller, specify it by name as follows:
```shell
$ kubectl edit rc --namespace=kube-system
```
-Add the `--federations` flag as args to kube-dns container in the YAML file that
-pops up after running the above command.
+In the resulting YAML file for the kube-dns replication controller, add the `--federations` flag as an argument to kube-dns container.
-To delete the existing kube dns pod, you can first find it by running:
+Then, you must delete the existing kube dns pod. You can find the pod by running:
```shell
$ kubectl get pods --namespace=kube-system
```
-And then delete it by running:
+And then delete the appropriate pod by running:
```shell
$ kubectl delete pods --namespace=kube-system
```
-You are now all set to start using federation.
+Once you've completed the kube-dns configuration, your federation is ready for use.
## Turn down
@@ -317,7 +352,7 @@ $ KUBERNETES_PROVIDER=gce FEDERATION_DNS_PROVIDER=google-clouddns FEDERATION_NAM
set appropriately if it is missing and `KUBERNETES_PROVIDER` is one of `gce`, `gke` and `aws`.
This is used to resolve DNS requests for federation services. The service
controller keeps DNS records with the provider updated as services/pods are
-updated in underlying kubernetes clusters.
+updated in underlying Kubernetes clusters.
`FEDERATION_NAME` is a name you can choose for your federation. This is the name that will appear in DNS routes.
diff --git a/docs/admin/federation/kubefed.md b/docs/admin/federation/kubefed.md
new file mode 100644
index 0000000000..1551c74213
--- /dev/null
+++ b/docs/admin/federation/kubefed.md
@@ -0,0 +1,194 @@
+---
+assignees:
+- madhusudancs
+title: Setting up Cluster Federation with Kubefed
+---
+
+* TOC
+{:toc}
+
+Kubernetes version 1.5 includes a new command line tool called
+`kubefed` to help you administrate your federated clusters.
+`kubefed` helps you to deploy a new Kubernetes cluster federation
+control plane, and to add clusters to or remove clusters from an
+existing federation control plane.
+
+This guide explains how to administer a Kubernetes Cluster Federation
+using `kubefed`.
+
+> Note: `kubefed` is an alpha feature in Kubernetes 1.5.
+
+## Prerequisites
+
+This guide assumes that you have a running Kubernetes cluster. Please
+see one of the [getting started](/docs/getting-started-guides/) guides
+for installation instructions for your platform.
+
+
+## Getting `kubefed`
+
+Download the client tarball corresponding to Kubernetes version 1.5
+or later
+[from the release page](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG.md),
+extract the binaries in the tarball to one of the directories
+in your `$PATH` and set the executable permission on those binaries.
+
+```shell
+curl -O https://storage.googleapis.com/kubernetes-release/release/v1.5.0/kubernetes-client-linux-amd64.tar.gz
+tar -xzvf kubernetes-client-linux-amd64.tar.gz
+sudo cp kubernetes/client/bin/kubefed /usr/local/bin
+sudo chmod +x /usr/local/bin/kubefed
+sudo cp kubernetes/client/bin/kubectl /usr/local/bin
+sudo chmod +x /usr/local/bin/kubectl
+```
+
+
+## Choosing a host cluster.
+
+You'll need to choose one of your Kubernetes clusters to be the
+*host cluster*. The host cluster hosts the components that make up
+your federation control plane. Ensure that you have a `kubeconfig`
+entry in your local `kubeconfig` that corresponds to the host cluster.
+You can verify that you have the required `kubeconfig` entry by
+running:
+
+```shell
+kubectl config get-contexts
+```
+
+The output should contain an entry corresponding to your host cluster,
+similar to the following:
+
+```
+CURRENT NAME CLUSTER AUTHINFO NAMESPACE
+ gke_myproject_asia-east1-b_gce-asia-east1 gke_myproject_asia-east1-b_gce-asia-east1 gke_myproject_asia-east1-b_gce-asia-east1
+```
+
+
+You'll need to provide the `kubeconfig` context (called name in the
+entry above) for your host cluster when you deploy your federation
+control plane.
+
+
+## Deploying a federation control plane.
+
+"To deploy a federation control plane on your host cluster, run
+`kubefed init` command. When you use `kubefed init`, you must provide
+the following:
+
+* Federation name
+* `--host-cluster-context`, the `kubeconfig` context for the host cluster
+* `--dns-zone-name`, a domain name suffix for your federated services
+
+The following example command deploys a federation control plane with
+the name `fellowship`, a host cluster context `rivendell`, and the
+domain suffix `example.com`:
+
+```shell
+kubefed init fellowship --host-cluster-context=rivendell --dns-zone-name="example.com"
+```
+
+The domain suffix you specify in `--dns-zone-name` must be an existing
+domain that you control, and that is programmable by your DNS provider.
+
+`kubefed init` sets up the federation control plane in the host
+cluster and also adds an entry for the federation API server in your
+local kubeconfig. Note that in the alpha release in Kubernetes 1.5,
+`kubefed init` does not automatically set the current context to the
+newly deployed federation. You can set the current context manually by
+running:
+
+```shell
+kubectl config use-context fellowship
+```
+
+where `fellowship` is the name of your federation.
+
+
+## Adding a cluster to a federation
+
+Once you've deployed a federation control plane, you'll need to make
+that control plane aware of the clusters it should manage. You can add
+a cluster to your federation by using the `kubefed join` command.
+
+To use `kubefed join`, you'll need to provide the name of the cluster
+you want to add to the federation, and the `--host-cluster-context`
+for the federation control plane's host cluster.
+
+The following example command adds the cluster `gondor` to the
+federation with host cluster `rivendell`:
+
+```
+kubefed join gondor --host-cluster-context=rivendell
+```
+
+> Note: Kubernetes requires that you manually join clusters to a
+federation because the federation control plane manages only those
+clusters that it is responsible for managing. Adding a cluster tells
+the federation control plane that it is responsible for managing that
+cluster.
+
+### Naming rules and customization
+
+The cluster name you supply to `kubefed join` must be a valid RFC 1035
+label.
+
+Furthermore, federation control plane requires credentials of the
+joined clusters to operate on them. These credentials are obtained
+from the local kubeconfig. `kubefed join` uses the cluster name
+specified as the argument to look for the cluster's context in the
+local kubeconfig. If it fails to find a matching context, it exits
+with an error.
+
+This might cause issues in cases where context names for each cluster
+in the federation don't follow RFC 1035 label naming rules. In such
+cases, you can specify a cluster name that conforms to the RFC 1035
+label naming rules and specify the cluster context using the
+`--cluster-context` flag. For example, if context of the cluster your
+are joining is `gondor_needs-no_king`, then you can
+join the cluster by running:
+
+```shell
+kubefed join gondor --host-cluster-context=rivendell --cluster-context=gondor_needs-no_king
+```
+
+#### Secret name
+
+Cluster credentials required by the federation control plane as
+described above are stored as a secret in the host cluster. The name
+of the secret is also derived from the cluster name.
+
+However, the name of a secret object in Kubernetes should conform
+to the subdomain name specification described in RFC 1123. If this
+isn't case, you can pass the secret name to `kubefed join` using the
+`--secret-name` flag. For example, if the cluster name is `noldor` and
+the secret name is `11kingdom`, you can join the cluster by
+running:
+
+```shell
+kubefed join noldor --host-cluster-context=rivendell --secret-name=11kingdom
+```
+
+## Removing a cluster from a federation
+
+To remove a cluster from a federation, run the `kubefed unjoin`
+command with the cluster name and the federation's
+`--host-cluster-context`:
+
+```
+kubefed unjoin gondor --host-cluster-context=rivendell
+```
+
+
+## Turning down the federation control plane:
+
+Proper cleanup of federation control plane is not fully implemented in
+this alpha release of `kubefed`. However, for the time being, deleting
+the federation system namespace should remove all the resources except
+the persistent storage volume dynamically provisioned for the
+federation control plane's etcd. You can delete the federation
+namespace by running the following command:
+
+```
+$ kubectl delete ns federation-system
+```
diff --git a/docs/admin/garbage-collection.md b/docs/admin/garbage-collection.md
index a3112a07f1..3a8ecda475 100644
--- a/docs/admin/garbage-collection.md
+++ b/docs/admin/garbage-collection.md
@@ -1,7 +1,7 @@
---
assignees:
- mikedanese
-
+title: Configuring kubelet Garbage Collection
---
* TOC
@@ -13,11 +13,11 @@ External garbage collection tools are not recommended as these tools can potenti
### Image Collection
-kubernetes manages lifecycle of all images through imageManager, with the cooperation
+Kubernetes manages lifecycle of all images through imageManager, with the cooperation
of cadvisor.
The policy for garbage collecting images takes two factors into consideration:
-`HighThresholdPercent` and `LowThresholdPercent`. Disk usage above the the high threshold
+`HighThresholdPercent` and `LowThresholdPercent`. Disk usage above the high threshold
will trigger garbage collection. The garbage collection will delete least recently used images until the low
threshold has been met.
diff --git a/docs/admin/ha-master-gce.md b/docs/admin/ha-master-gce.md
new file mode 100644
index 0000000000..91e0019272
--- /dev/null
+++ b/docs/admin/ha-master-gce.md
@@ -0,0 +1,158 @@
+---
+assignees:
+- jszczepkowski
+title: Setting up High-Availability Kubernetes Masters
+---
+
+* TOC
+{:toc}
+
+Kubernetes version 1.5 adds alpha support for replicating Kubernetes masters in `kube-up` or `kube-down` scripts for Google Compute Engine.
+This document describes how to use kube-up/down scripts to manage highly available (HA) masters and how HA masters are implemented for use with GCE.
+
+## Starting an HA-compatible cluster
+
+To create a new HA-compatible cluster, you must set the following flags in your `kube-up` script:
+
+* `MULTIZONE=true` - to prevent removal of master replicas kubelets from zones different than server's default zone.
+Required if you want to run master replicas in different zones, which is recommended.
+
+* `ENABLE_ETCD_QUORUM_READS=true` - to ensure that reads from all API servers will return most up-to-date data.
+If true, reads will be directed to leader etcd replica.
+Setting this value to true is optional: reads will be more reliable but will also be slower.
+
+Optionally, you can specify a GCE zone where the first master replica is to be created.
+Set the following flag:
+
+* `KUBE_GCE_ZONE=zone` - zone where the first master replica will run.
+
+The following sample command sets up a HA-compatible cluster in the GCE zone europe-west1-b:
+
+```shell
+$ MULTIZONE=true KUBE_GCE_ZONE=europe-west1-b ENABLE_ETCD_QUORUM_READS=true ./cluster/kube-up.sh
+```
+
+Note that the commands above create a cluster with one master;
+however, you can add new master replicas to the cluster with subsequent commands.
+
+## Adding a new master replica
+
+After you have created an HA-compatible cluster, you can add master replicas to it.
+You add master replicas by using a `kube-up` script with the following flags:
+
+* `KUBE_REPLICATE_EXISTING_MASTER=true` - to create a replica of an existing
+master.
+
+* `KUBE_GCE_ZONE=zone` - zone where the master replica will run.
+Must be in the same region as other replicas' zones.
+
+You don't need to set the `MULTIZONE` or `ENABLE_ETCD_QUORUM_READS` flags,
+as those are inherited from when you started your HA-compatible cluster.
+
+The following sample command replicates the master on an existing HA-compatible cluster:
+
+```shell
+$ KUBE_GCE_ZONE=europe-west1-c KUBE_REPLICATE_EXISTING_MASTER=true ./cluster/kube-up.sh
+```
+
+## Removing a master replica
+
+You can remove a master replica from an HA cluster by using a `kube-down` script with the following flags:
+
+* `KUBE_DELETE_NODES=false` - to restrain deletion of kubelets.
+
+* `KUBE_GCE_ZONE=zone` - the zone from where master replica will be removed.
+
+* `KUBE_REPLICA_NAME=replica_name` - (optional) the name of master replica to remove.
+If empty: any replica from the given zone will be removed.
+
+The following sample command removes a master replica from an existing HA cluster:
+
+```shell
+$ KUBE_DELETE_NODES=false KUBE_GCE_ZONE=europe-west1-c ./cluster/kube-down.sh
+```
+
+## Handling master replica failures
+
+If one of the master replicas in your HA cluster fails,
+the best practice is to remove the replica from your cluster and add a new replica in the same zone.
+The following sample commands demonstrate this process:
+
+1. Remove the broken replica:
+
+```shell
+$ KUBE_DELETE_NODES=false KUBE_GCE_ZONE=replica_zone KUBE_REPLICA_NAME=replica_name ./cluster/kube-down.sh
+```
+
+2. Add a new replica in place of the old one:
+
+```shell
+$ KUBE_GCE_ZONE=replica-zone KUBE_REPLICATE_EXISTING_MASTER=true ./cluster/kube-up.sh
+```
+
+## Best practices for replicating masters for HA clusters
+
+* Try to place masters replicas in different zones. During a zone failure, all master placed inside the zone will fail.
+To survive zone failure, also place nodes in multiple zones
+(see [multiple-zones](http://kubernetes.io/docs/admin/multiple-zones/) for details).
+
+* Do not use a cluster with two master replicas. Consensus on a two replica cluster requires both replicas running when changing persistent state.
+As a result, both replicas are needed and a failure of any replica turns cluster into majority failure state.
+A two-replica cluster is thus inferior, in terms of HA, to a single replica cluster.
+
+* When you add a master replica, cluster state (etcd) is copied to a new instance.
+If the cluster is large, it may take a long time to duplicate its state.
+This operation may be speed up by migrating etcd data directory, as described [here](https://coreos.com/etcd/docs/latest/admin_guide.html#member-migration) here
+(we are considering adding support for etcd data dir migration in future).
+
+## Implementation notes
+
+
+
+### Overview
+
+Each of master replicas will run the following components in the following mode:
+
+* etcd instance: all instances will be clustered together using consensus;
+
+* API server: each server will talk to local etcd - all API servers in the cluster will be available;
+
+* controllers, scheduler, and cluster auto-scaler: will use lease mechanism - only one instance of each of them will be active in the cluster;
+
+* add-on manager: each manager will work independently trying to keep add-ons in sync.
+
+In addition, there will be a load balancer in front of API servers that will route external and internal traffic to them.
+
+### Load balancing
+
+When starting the second master replica, a load balancer containing the two replicas will be created
+and the IP address of the first replica will be promoted to IP address of load balancer.
+Similarly, after removal of the penultimate master replica, the load balancer will be removed and its IP address will be assigned to the last remaining replica.
+Please note that creation and removal of load balancer are complex operations and it may take some time (~20 minutes) for them to propagate.
+
+### Master service & kubelets
+
+Instead of trying to keep an up-to-date list of Kubernetes apiserver in the Kubernetes service,
+the system directs all traffic to the external IP:
+
+* in one master cluster the IP points to the single master,
+
+* in multi-master cluster the IP points to the load balancer in-front of the masters.
+
+Similarly, the external IP will be used by kubelets to communicate with master.
+
+### Master certificates
+
+Kubernetes generates Master TLS certificates for the external public IP and local IP for each replica.
+There are no certificates for the ephemeral public IP for replicas;
+to access a replica via its ephemeral public IP, you must skip TLS verification.
+
+### Clustering etcd
+
+To allow etcd clustering, ports needed to communicate between etcd instances will be opened (for inside cluster communication).
+To make such deployment secure, communication between etcd instances is authorized using SSL.
+
+## Additional reading
+
+[Automated HA master deployment - design doc](https://github.com/kubernetes/kubernetes/blob/master/docs/design/ha_master.md)
+
diff --git a/docs/admin/ha-master-gce.png b/docs/admin/ha-master-gce.png
new file mode 100644
index 0000000000..a05e04a187
Binary files /dev/null and b/docs/admin/ha-master-gce.png differ
diff --git a/docs/admin/high-availability/index.md b/docs/admin/high-availability/index.md
index ad78270e4a..42e51d3d51 100644
--- a/docs/admin/high-availability/index.md
+++ b/docs/admin/high-availability/index.md
@@ -1,7 +1,7 @@
----
-
----
-
+---
+title: Building High-Availability Clusters
+---
+
## Introduction
This document describes how to build a high-availability (HA) Kubernetes cluster. This is a fairly advanced topic.
diff --git a/docs/admin/index.md b/docs/admin/index.md
index 47f47f8116..d4111691c6 100644
--- a/docs/admin/index.md
+++ b/docs/admin/index.md
@@ -2,7 +2,7 @@
assignees:
- davidopp
- lavalamp
-
+title: Admin Guide
---
The cluster admin guide is for anyone creating or administering a Kubernetes cluster.
@@ -13,7 +13,7 @@ It assumes some familiarity with concepts in the [User Guide](/docs/user-guide/)
## Planning a cluster
-There are many different examples of how to setup a kubernetes cluster. Many of them are listed in this
+There are many different examples of how to setup a Kubernetes cluster. Many of them are listed in this
[matrix](/docs/getting-started-guides/). We call each of the combinations in this matrix a *distro*.
Before choosing a particular guide, here are some things to consider:
@@ -25,12 +25,12 @@ Before choosing a particular guide, here are some things to consider:
- Will your cluster be on-premises, or in the cloud (IaaS)? Kubernetes does not directly support hybrid clusters. We
recommend setting up multiple clusters rather than spanning distant locations.
- Will you be running Kubernetes on "bare metal" or virtual machines? Kubernetes supports both, via different distros.
- - Do you just want to run a cluster, or do you expect to do active development of kubernetes project code? If the
+ - Do you just want to run a cluster, or do you expect to do active development of Kubernetes project code? If the
latter, it is better to pick a distro actively used by other developers. Some distros only use binary releases, but
offer is a greater variety of choices.
- Not all distros are maintained as actively. Prefer ones which are listed as tested on a more recent version of
Kubernetes.
- - If you are configuring kubernetes on-premises, you will need to consider what [networking
+ - If you are configuring Kubernetes on-premises, you will need to consider what [networking
model](/docs/admin/networking) fits best.
- If you are designing for very high-availability, you may want [clusters in multiple zones](/docs/admin/multi-cluster).
- You may want to familiarize yourself with the various
@@ -61,7 +61,8 @@ project](/docs/admin/salt).
* **DNS Integration with SkyDNS** ([dns.md](/docs/admin/dns)):
Resolving a DNS name directly to a Kubernetes service.
-* **Logging** with [Kibana](/docs/user-guide/logging)
+* [**Cluster-level logging**](/docs/user-guide/logging/overview)
+ Saving container logs to a central log store with search/browsing interface.
## Multi-tenant support
@@ -84,3 +85,8 @@ project](/docs/admin/salt).
* **Sysctls** [sysctls](/docs/admin/sysctls.md)
* **Audit** [audit](/docs/admin/audit)
+
+* **Securing the kubelet**
+ * [Master-Node communication](/docs/admin/master-node-communication/)
+ * [TLS bootstrapping](/docs/admin/kubelet-tls-bootstrapping/)
+ * [Kubelet authentication/authorization](/docs/admin/kubelet-authentication-authorization/)
diff --git a/docs/admin/kube-apiserver.md b/docs/admin/kube-apiserver.md
index 4e24cd2d89..1e2c8a602e 100644
--- a/docs/admin/kube-apiserver.md
+++ b/docs/admin/kube-apiserver.md
@@ -1,4 +1,6 @@
---
+title: kube-apiserver
+notitle: true
---
## kube-apiserver
@@ -20,11 +22,12 @@ kube-apiserver
### Options
```
- --admission-control string Ordered list of plug-ins to do admission control of resources into cluster. Comma-delimited list of: AlwaysAdmit, AlwaysDeny, AlwaysPullImages, DefaultStorageClass, DenyEscalatingExec, DenyExecOnPrivileged, ImagePolicyWebhook, InitialResources, LimitPodHardAntiAffinityTopology, LimitRanger, NamespaceAutoProvision, NamespaceExists, NamespaceLifecycle, PersistentVolumeLabel, PodSecurityPolicy, ResourceQuota, SecurityContextDeny, ServiceAccount. (default "AlwaysAdmit")
+ --admission-control string Ordered list of plug-ins to do admission control of resources into cluster. Comma-delimited list of: AlwaysAdmit, AlwaysDeny, AlwaysPullImages, DefaultStorageClass, DenyEscalatingExec, DenyExecOnPrivileged, ImagePolicyWebhook, InitialResources, LimitPodHardAntiAffinityTopology, LimitRanger, NamespaceAutoProvision, NamespaceExists, NamespaceLifecycle, OwnerReferencesPermissionEnforcement, PersistentVolumeLabel, PodNodeSelector, PodSecurityPolicy, ResourceQuota, SecurityContextDeny, ServiceAccount. (default "AlwaysAdmit")
--admission-control-config-file string File with admission control configuration.
- --advertise-address value The IP address on which to advertise the apiserver to members of the cluster. This address must be reachable by the rest of the cluster. If blank, the --bind-address will be used. If --bind-address is unspecified, the host's default interface will be used.
+ --advertise-address ip The IP address on which to advertise the apiserver to members of the cluster. This address must be reachable by the rest of the cluster. If blank, the --bind-address will be used. If --bind-address is unspecified, the host's default interface will be used.
--allow-privileged If true, allow privileged containers.
- --apiserver-count int The number of apiservers running in the cluster. (default 1)
+ --anonymous-auth Enables anonymous requests to the secure port of the API server. Requests that are not rejected by another authentication method are treated as anonymous requests. Anonymous requests have a username of system:anonymous, and a group name of system:unauthenticated. (default true)
+ --apiserver-count int The number of apiservers running in the cluster. Must be a positive number. (default 1)
--audit-log-maxage int The maximum number of days to retain old audit log files based on the timestamp encoded in their filename.
--audit-log-maxbackup int The maximum number of old audit log files to retain.
--audit-log-maxsize int The maximum size in megabytes of the audit log file before it gets rotated. Defaults to 100MB.
@@ -38,12 +41,13 @@ kube-apiserver
--authorization-webhook-cache-unauthorized-ttl duration The duration to cache 'unauthorized' responses from the webhook authorizer. Default is 30s. (default 30s)
--authorization-webhook-config-file string File with webhook configuration in kubeconfig format, used with --authorization-mode=Webhook. The API server will query the remote service to determine access on the API server's secure port.
--basic-auth-file string If set, the file that will be used to admit requests to the secure port of the API server via http basic authentication.
- --bind-address value The IP address on which to listen for the --secure-port port. The associated interface(s) must be reachable by the rest of the cluster, and by CLI/web clients. If blank, all interfaces will be used (0.0.0.0). (default 0.0.0.0)
+ --bind-address ip The IP address on which to listen for the --secure-port port. The associated interface(s) must be reachable by the rest of the cluster, and by CLI/web clients. If blank, all interfaces will be used (0.0.0.0). (default 0.0.0.0)
--cert-dir string The directory where the TLS certs are located (by default /var/run/kubernetes). If --tls-cert-file and --tls-private-key-file are provided, this flag will be ignored. (default "/var/run/kubernetes")
--client-ca-file string If set, any request presenting a client certificate signed by one of the authorities in the client-ca-file is authenticated with an identity corresponding to the CommonName of the client certificate.
--cloud-config string The path to the cloud provider configuration file. Empty string for no configuration file.
--cloud-provider string The provider for cloud services. Empty string for no provider.
- --cors-allowed-origins value List of allowed origins for CORS, comma separated. An allowed origin can be a regular expression to support subdomain matching. If this list is empty CORS will not be enabled. (default [])
+ --contention-profiling Enable contention profiling. Requires --profiling to be set to work.
+ --cors-allowed-origins stringSlice List of allowed origins for CORS, comma separated. An allowed origin can be a regular expression to support subdomain matching. If this list is empty CORS will not be enabled.
--delete-collection-workers int Number of workers spawned for DeleteCollection call. These are used to speed up namespace cleanup. (default 1)
--deserialization-cache-size int Number of deserialized json objects to cache in memory.
--enable-garbage-collector Enables the generic garbage collector. MUST be synced with the corresponding flag of the kube-controller-manager. (default true)
@@ -53,62 +57,68 @@ kube-apiserver
--etcd-keyfile string SSL key file used to secure etcd communication.
--etcd-prefix string The prefix for all resource paths in etcd. (default "/registry")
--etcd-quorum-read If true, enable quorum read.
- --etcd-servers value List of etcd servers to connect with (http://ip:port), comma separated. (default [])
- --etcd-servers-overrides value Per-resource etcd servers overrides, comma separated. The individual override format: group/resource#servers, where servers are http://ip:port, semicolon separated. (default [])
+ --etcd-servers stringSlice List of etcd servers to connect with (scheme://ip:port), comma separated.
+ --etcd-servers-overrides stringSlice Per-resource etcd servers overrides, comma separated. The individual override format: group/resource#servers, where servers are http://ip:port, semicolon separated.
--event-ttl duration Amount of time to retain events. Default is 1h. (default 1h0m0s)
+ --experimental-keystone-ca-file string If set, the Keystone server's certificate will be verified by one of the authorities in the experimental-keystone-ca-file, otherwise the host's root CA set will be used.
--experimental-keystone-url string If passed, activates the keystone authentication plugin.
--external-hostname string The hostname to use when generating externalized URLs for this master (e.g. Swagger API Docs).
- --feature-gates value A set of key=value pairs that describe feature gates for alpha/experimental features. Options are:
+ --feature-gates mapStringBool A set of key=value pairs that describe feature gates for alpha/experimental features. Options are:
AllAlpha=true|false (ALPHA - default=false)
-AllowExtTrafficLocalEndpoints=true|false (ALPHA - default=false)
+AllowExtTrafficLocalEndpoints=true|false (BETA - default=true)
AppArmor=true|false (BETA - default=true)
DynamicKubeletConfig=true|false (ALPHA - default=false)
DynamicVolumeProvisioning=true|false (ALPHA - default=true)
+ExperimentalHostUserNamespaceDefaulting=true|false (ALPHA - default=false)
+StreamingProxyRedirects=true|false (ALPHA - default=false)
--google-json-key string The Google Cloud Platform Service Account JSON Key to use for authentication.
- --insecure-bind-address value The IP address on which to serve the --insecure-port (set to 0.0.0.0 for all interfaces). Defaults to localhost. (default 127.0.0.1)
+ --insecure-allow-any-token username/group1,group2 If set, your server will be INSECURE. Any token will be allowed and user information will be parsed from the token as username/group1,group2
+ --insecure-bind-address ip The IP address on which to serve the --insecure-port (set to 0.0.0.0 for all interfaces). Defaults to localhost. (default 127.0.0.1)
--insecure-port int The port on which to serve unsecured, unauthenticated access. Default 8080. It is assumed that firewall rules are set up such that this port is not reachable from outside of the cluster and that port 443 on the cluster's public address is proxied to this port. This is performed by nginx in the default setup. (default 8080)
--kubelet-certificate-authority string Path to a cert file for the certificate authority.
--kubelet-client-certificate string Path to a client cert file for TLS.
--kubelet-client-key string Path to a client key file for TLS.
--kubelet-https Use https for kubelet connections. (default true)
+ --kubelet-preferred-address-types stringSlice List of the preferred NodeAddressTypes to use for kubelet connections. (default [Hostname,InternalIP,ExternalIP,LegacyHostIP])
--kubelet-timeout duration Timeout for kubelet operations. (default 5s)
--kubernetes-service-node-port int If non-zero, the Kubernetes master service (which apiserver creates/maintains) will be of type NodePort, using this as the value of the port. If zero, the Kubernetes master service will be of type ClusterIP.
--long-running-request-regexp string A regular expression matching long running requests which should be excluded from maximum inflight request handling. (default "(/|^)((watch|proxy)(/|$)|(logs?|portforward|exec|attach)/?$)")
- --master-service-namespace string The namespace from which the kubernetes master services should be injected into pods. (default "default")
+ --master-service-namespace string DEPRECATED: the namespace from which the kubernetes master services should be injected into pods. (default "default")
--max-connection-bytes-per-sec int If non-zero, throttle each user connection to this number of bytes/sec. Currently only applies to long-running requests.
--max-requests-inflight int The maximum number of requests in flight at a given time. When the server exceeds this, it rejects requests. Zero for no limit. (default 400)
--min-request-timeout int An optional field indicating the minimum number of seconds a handler must keep a request open before timing it out. Currently only honored by the watch request handler, which picks a randomized value above this number as the connection timeout, to spread out load. (default 1800)
--oidc-ca-file string If set, the OpenID server's certificate will be verified by one of the authorities in the oidc-ca-file, otherwise the host's root CA set will be used.
--oidc-client-id string The client ID for the OpenID Connect client, must be set if oidc-issuer-url is set.
- --oidc-groups-claim string If provided, the name of a custom OpenID Connect claim for specifying user groups. The claim value is expected to be an array of strings. This flag is experimental, please see the authentication documentation for further details.
+ --oidc-groups-claim string If provided, the name of a custom OpenID Connect claim for specifying user groups. The claim value is expected to be a string or array of strings. This flag is experimental, please see the authentication documentation for further details.
--oidc-issuer-url string The URL of the OpenID issuer, only HTTPS scheme will be accepted. If set, it will be used to verify the OIDC JSON Web Token (JWT).
--oidc-username-claim string The OpenID claim to use as the user name. Note that claims other than the default ('sub') is not guaranteed to be unique and immutable. This flag is experimental, please see the authentication documentation for further details. (default "sub")
--profiling Enable profiling via web interface host:port/debug/pprof/ (default true)
--repair-malformed-updates If true, server will do its best to fix the update request to pass the validation, e.g., setting empty UID in update request to its existing value. This flag can be turned off after we fix all the clients that send malformed updates. (default true)
- --runtime-config value A set of key=value pairs that describe runtime configuration that may be passed to apiserver. apis/ key can be used to turn on/off specific api versions. apis// can be used to turn on/off specific resources. api/all and api/legacy are special keys to control all and legacy api versions respectively.
+ --requestheader-allowed-names stringSlice List of client certificate common names to allow to provide usernames in headers specified by --requestheader-username-headers. If empty, any client certificate validated by the authorities in --requestheader-client-ca-file is allowed.
+ --requestheader-client-ca-file string Root certificate bundle to use to verify client certificates on incoming requests before trusting usernames in headers specified by --requestheader-username-headers
+ --requestheader-username-headers stringSlice List of request headers to inspect for usernames. X-Remote-User is common.
+ --runtime-config mapStringString A set of key=value pairs that describe runtime configuration that may be passed to apiserver. apis/ key can be used to turn on/off specific api versions. apis// can be used to turn on/off specific resources. api/all and api/legacy are special keys to control all and legacy api versions respectively.
--secure-port int The port on which to serve HTTPS with authentication and authorization. If 0, don't serve HTTPS at all. (default 6443)
- --service-account-key-file string File containing PEM-encoded x509 RSA private or public key, used to verify ServiceAccount tokens. If unspecified, --tls-private-key-file is used.
+ --service-account-key-file stringArray File containing PEM-encoded x509 RSA or ECDSA private or public keys, used to verify ServiceAccount tokens. If unspecified, --tls-private-key-file is used. The specified file can contain multiple keys, and the flag can be specified multiple times with different files.
--service-account-lookup If true, validate ServiceAccount tokens exist in etcd as part of authentication.
- --service-cluster-ip-range value A CIDR notation IP range from which to assign service cluster IPs. This must not overlap with any IP ranges assigned to nodes for pods.
- --service-node-port-range value A port range to reserve for services with NodePort visibility. Example: '30000-32767'. Inclusive at both ends of the range. (default 30000-32767)
+ --service-cluster-ip-range ipNet A CIDR notation IP range from which to assign service cluster IPs. This must not overlap with any IP ranges assigned to nodes for pods.
+ --service-node-port-range portRange A port range to reserve for services with NodePort visibility. Example: '30000-32767'. Inclusive at both ends of the range. (default 30000-32767)
--ssh-keyfile string If non-empty, use secure SSH proxy to the nodes, using this user keyfile
--ssh-user string If non-empty, use secure SSH proxy to the nodes, using this user name
--storage-backend string The storage backend for persistence. Options: 'etcd2' (default), 'etcd3'.
--storage-media-type string The media type to use to store objects in storage. Defaults to application/json. Some resources may only support a specific media type and will ignore this setting. (default "application/json")
- --storage-versions string The per-group version to store resources in. Specified in the format "group1/version1,group2/version2,...". In the case where objects are moved from one group to the other, you may specify the format "group1=group2/v1beta1,group3/v1beta1,...". You only need to pass the groups you wish to change from the defaults. It defaults to a list of preferred versions of all registered groups, which is derived from the KUBE_API_VERSIONS environment variable. (default "apps/v1alpha1,authentication.k8s.io/v1beta1,authorization.k8s.io/v1beta1,autoscaling/v1,batch/v1,certificates.k8s.io/v1alpha1,componentconfig/v1alpha1,extensions/v1beta1,imagepolicy.k8s.io/v1alpha1,policy/v1alpha1,rbac.authorization.k8s.io/v1alpha1,storage.k8s.io/v1beta1,v1")
+ --storage-versions string The per-group version to store resources in. Specified in the format "group1/version1,group2/version2,...". In the case where objects are moved from one group to the other, you may specify the format "group1=group2/v1beta1,group3/v1beta1,...". You only need to pass the groups you wish to change from the defaults. It defaults to a list of preferred versions of all registered groups, which is derived from the KUBE_API_VERSIONS environment variable. (default "apps/v1beta1,authentication.k8s.io/v1beta1,authorization.k8s.io/v1beta1,autoscaling/v1,batch/v1,certificates.k8s.io/v1alpha1,componentconfig/v1alpha1,extensions/v1beta1,imagepolicy.k8s.io/v1alpha1,policy/v1beta1,rbac.authorization.k8s.io/v1alpha1,storage.k8s.io/v1beta1,v1")
--target-ram-mb int Memory limit for apiserver in MB (used to configure sizes of caches, etc.)
- --tls-cert-file string File containing x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If HTTPS serving is enabled, and --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to /var/run/kubernetes.
- --tls-private-key-file string File containing x509 private key matching --tls-cert-file.
+ --tls-ca-file string If set, this certificate authority will used for secure access from Admission Controllers. This must be a valid PEM-encoded CA bundle.
+ --tls-cert-file string File containing the default x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If HTTPS serving is enabled, and --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to /var/run/kubernetes.
+ --tls-private-key-file string File containing the default x509 private key matching --tls-cert-file.
+ --tls-sni-cert-key namedCertKey A pair of x509 certificate and private key file paths, optionally suffixed with a list of domain patterns which are fully qualified domain names, possibly with prefixed wildcard segments. If no domain patterns are provided, the names of the certificate are extracted. Non-wildcard matches trump over wildcard matches, explicit domain patterns trump over extracted names. For multiple key/certificate pairs, use the --tls-sni-cert-key multiple times. Examples: "example.key,example.crt" or "*.foo.com,foo.com:foo.key,foo.crt". (default [])
--token-auth-file string If set, the file that will be used to secure the secure port of the API server via token authentication.
--watch-cache Enable watch caching in the apiserver (default true)
- --watch-cache-sizes value List of watch cache sizes for every resource (pods, nodes, etc.), comma separated. The individual override format: resource#size, where size is a number. It takes effect when watch-cache is enabled. (default [])
+ --watch-cache-sizes stringSlice List of watch cache sizes for every resource (pods, nodes, etc.), comma separated. The individual override format: resource#size, where size is a number. It takes effect when watch-cache is enabled.
```
-###### Auto generated by spf13/cobra on 24-Oct-2016
-
-
-
-
+###### Auto generated by spf13/cobra on 13-Dec-2016
diff --git a/docs/admin/kube-controller-manager.md b/docs/admin/kube-controller-manager.md
index c6db5ea858..82dd43cbcd 100644
--- a/docs/admin/kube-controller-manager.md
+++ b/docs/admin/kube-controller-manager.md
@@ -1,4 +1,6 @@
---
+title: kube-controller-manager
+notitle: true
---
## kube-controller-manager
@@ -24,7 +26,7 @@ kube-controller-manager
### Options
```
- --address value The IP address to serve on (set to 0.0.0.0 for all interfaces) (default 0.0.0.0)
+ --address ip The IP address to serve on (set to 0.0.0.0 for all interfaces) (default 0.0.0.0)
--allocate-node-cidrs Should CIDRs for Pods be allocated and set on the cloud provider.
--cloud-config string The path to the cloud provider configuration file. Empty string for no configuration file.
--cloud-provider string The provider for cloud services. Empty string for no provider.
@@ -32,37 +34,39 @@ kube-controller-manager
--cluster-name string The instance prefix for the cluster (default "kubernetes")
--cluster-signing-cert-file string Filename containing a PEM-encoded X509 CA certificate used to issue cluster-scoped certificates (default "/etc/kubernetes/ca/ca.pem")
--cluster-signing-key-file string Filename containing a PEM-encoded RSA or ECDSA private key used to sign cluster-scoped certificates (default "/etc/kubernetes/ca/ca.key")
- --concurrent-deployment-syncs value The number of deployment objects that are allowed to sync concurrently. Larger number = more responsive deployments, but more CPU (and network) load (default 5)
- --concurrent-endpoint-syncs value The number of endpoint syncing operations that will be done concurrently. Larger number = faster endpoint updating, but more CPU (and network) load (default 5)
- --concurrent-gc-syncs value The number of garbage collector workers that are allowed to sync concurrently. (default 20)
- --concurrent-namespace-syncs value The number of namespace objects that are allowed to sync concurrently. Larger number = more responsive namespace termination, but more CPU (and network) load (default 2)
- --concurrent-replicaset-syncs value The number of replica sets that are allowed to sync concurrently. Larger number = more responsive replica management, but more CPU (and network) load (default 5)
- --concurrent-resource-quota-syncs value The number of resource quotas that are allowed to sync concurrently. Larger number = more responsive quota management, but more CPU (and network) load (default 5)
- --concurrent-service-syncs value The number of services that are allowed to sync concurrently. Larger number = more responsive service management, but more CPU (and network) load (default 1)
- --concurrent-serviceaccount-token-syncs value The number of service account token objects that are allowed to sync concurrently. Larger number = more responsive token generation, but more CPU (and network) load (default 5)
- --concurrent_rc_syncs value The number of replication controllers that are allowed to sync concurrently. Larger number = more responsive replica management, but more CPU (and network) load (default 5)
+ --concurrent-deployment-syncs int32 The number of deployment objects that are allowed to sync concurrently. Larger number = more responsive deployments, but more CPU (and network) load (default 5)
+ --concurrent-endpoint-syncs int32 The number of endpoint syncing operations that will be done concurrently. Larger number = faster endpoint updating, but more CPU (and network) load (default 5)
+ --concurrent-gc-syncs int32 The number of garbage collector workers that are allowed to sync concurrently. (default 20)
+ --concurrent-namespace-syncs int32 The number of namespace objects that are allowed to sync concurrently. Larger number = more responsive namespace termination, but more CPU (and network) load (default 2)
+ --concurrent-replicaset-syncs int32 The number of replica sets that are allowed to sync concurrently. Larger number = more responsive replica management, but more CPU (and network) load (default 5)
+ --concurrent-resource-quota-syncs int32 The number of resource quotas that are allowed to sync concurrently. Larger number = more responsive quota management, but more CPU (and network) load (default 5)
+ --concurrent-service-syncs int32 The number of services that are allowed to sync concurrently. Larger number = more responsive service management, but more CPU (and network) load (default 1)
+ --concurrent-serviceaccount-token-syncs int32 The number of service account token objects that are allowed to sync concurrently. Larger number = more responsive token generation, but more CPU (and network) load (default 5)
+ --concurrent_rc_syncs int32 The number of replication controllers that are allowed to sync concurrently. Larger number = more responsive replica management, but more CPU (and network) load (default 5)
--configure-cloud-routes Should CIDRs allocated by allocate-node-cidrs be configured on the cloud provider. (default true)
- --controller-start-interval duration Interval between starting controller managers. (default 0s)
- --daemonset-lookup-cache-size value The the size of lookup cache for daemonsets. Larger number = more responsive daemonsets, but more MEM load. (default 1024)
+ --controller-start-interval duration Interval between starting controller managers.
+ --daemonset-lookup-cache-size int32 The size of lookup cache for daemonsets. Larger number = more responsive daemonsets, but more MEM load. (default 1024)
--deployment-controller-sync-period duration Period for syncing the deployments. (default 30s)
--enable-dynamic-provisioning Enable dynamic provisioning for environments that support it. (default true)
--enable-garbage-collector Enables the generic garbage collector. MUST be synced with the corresponding flag of the kube-apiserver. (default true)
--enable-hostpath-provisioner Enable HostPath PV provisioning when running without a cloud provider. This allows testing and development of provisioning features. HostPath provisioning is not supported in any way, won't work in a multi-node cluster, and should not be used for anything other than testing or development.
- --feature-gates value A set of key=value pairs that describe feature gates for alpha/experimental features. Options are:
+ --feature-gates mapStringBool A set of key=value pairs that describe feature gates for alpha/experimental features. Options are:
AllAlpha=true|false (ALPHA - default=false)
-AllowExtTrafficLocalEndpoints=true|false (ALPHA - default=false)
+AllowExtTrafficLocalEndpoints=true|false (BETA - default=true)
AppArmor=true|false (BETA - default=true)
DynamicKubeletConfig=true|false (ALPHA - default=false)
DynamicVolumeProvisioning=true|false (ALPHA - default=true)
+ExperimentalHostUserNamespaceDefaulting=true|false (ALPHA - default=false)
+StreamingProxyRedirects=true|false (ALPHA - default=false)
--flex-volume-plugin-dir string Full path of the directory in which the flex volume plugin should search for additional third party volume plugins. (default "/usr/libexec/kubernetes/kubelet-plugins/volume/exec/")
--google-json-key string The Google Cloud Platform Service Account JSON Key to use for authentication.
--horizontal-pod-autoscaler-sync-period duration The period for syncing the number of pods in horizontal pod autoscaler. (default 30s)
--insecure-experimental-approve-all-kubelet-csrs-for-group string The group for which the controller-manager will auto approve all CSRs for kubelet client certificates.
- --kube-api-burst value Burst to use while talking with kubernetes apiserver (default 30)
+ --kube-api-burst int32 Burst to use while talking with Kubernetes apiserver (default 30)
--kube-api-content-type string Content type of requests sent to apiserver. (default "application/vnd.kubernetes.protobuf")
- --kube-api-qps value QPS to use while talking with kubernetes apiserver (default 20)
+ --kube-api-qps float32 QPS to use while talking with Kubernetes apiserver (default 20)
--kubeconfig string Path to kubeconfig file with authorization and master location information.
- --large-cluster-size-threshold value Number of nodes from which NodeController treats the cluster as large for the eviction logic purposes. --secondary-node-eviction-rate is implicitly overridden to 0 for clusters this size or smaller. (default 50)
+ --large-cluster-size-threshold int32 Number of nodes from which NodeController treats the cluster as large for the eviction logic purposes. --secondary-node-eviction-rate is implicitly overridden to 0 for clusters this size or smaller. (default 50)
--leader-elect Start a leader election client and gain leadership before executing the main loop. Enable this when running replicated components for high availability. (default true)
--leader-elect-lease-duration duration The duration that non-leader candidates will wait after observing a leadership renewal until attempting to acquire leadership of a led but unrenewed leader slot. This is effectively the maximum duration that a leader can be stopped before it is replaced by another candidate. This is only applicable if leader election is enabled. (default 15s)
--leader-elect-renew-deadline duration The interval between attempts by the acting master to renew a leadership slot before it stops leading. This must be less than or equal to the lease duration. This is only applicable if leader election is enabled. (default 10s)
@@ -70,39 +74,36 @@ DynamicVolumeProvisioning=true|false (ALPHA - default=true)
--master string The address of the Kubernetes API server (overrides any value in kubeconfig)
--min-resync-period duration The resync period in reflectors will be random between MinResyncPeriod and 2*MinResyncPeriod (default 12h0m0s)
--namespace-sync-period duration The period for syncing namespace life-cycle updates (default 5m0s)
- --node-cidr-mask-size value Mask size for node cidr in cluster. (default 24)
- --node-eviction-rate value Number of nodes per second on which pods are deleted in case of node failure when a zone is healthy (see --unhealthy-zone-threshold for definition of healthy/unhealthy). Zone refers to entire cluster in non-multizone clusters. (default 0.1)
+ --node-cidr-mask-size int32 Mask size for node cidr in cluster. (default 24)
+ --node-eviction-rate float32 Number of nodes per second on which pods are deleted in case of node failure when a zone is healthy (see --unhealthy-zone-threshold for definition of healthy/unhealthy). Zone refers to entire cluster in non-multizone clusters. (default 0.1)
--node-monitor-grace-period duration Amount of time which we allow running Node to be unresponsive before marking it unhealthy. Must be N times more than kubelet's nodeStatusUpdateFrequency, where N means number of retries allowed for kubelet to post node status. (default 40s)
--node-monitor-period duration The period for syncing NodeStatus in NodeController. (default 5s)
--node-startup-grace-period duration Amount of time which we allow starting Node to be unresponsive before marking it unhealthy. (default 1m0s)
- --node-sync-period duration The period for syncing nodes from cloudprovider. Longer periods will result in fewer calls to cloud provider, but may delay addition of new nodes to cluster. (default 10s)
--pod-eviction-timeout duration The grace period for deleting pods on failed nodes. (default 5m0s)
- --port value The port that the controller-manager's http service runs on (default 10252)
+ --port int32 The port that the controller-manager's http service runs on (default 10252)
--profiling Enable profiling via web interface host:port/debug/pprof/ (default true)
- --pv-recycler-increment-timeout-nfs value the increment of time added per Gi to ActiveDeadlineSeconds for an NFS scrubber pod (default 30)
- --pv-recycler-minimum-timeout-hostpath value The minimum ActiveDeadlineSeconds to use for a HostPath Recycler pod. This is for development and testing only and will not work in a multi-node cluster. (default 60)
- --pv-recycler-minimum-timeout-nfs value The minimum ActiveDeadlineSeconds to use for an NFS Recycler pod (default 300)
+ --pv-recycler-increment-timeout-nfs int32 the increment of time added per Gi to ActiveDeadlineSeconds for an NFS scrubber pod (default 30)
+ --pv-recycler-minimum-timeout-hostpath int32 The minimum ActiveDeadlineSeconds to use for a HostPath Recycler pod. This is for development and testing only and will not work in a multi-node cluster. (default 60)
+ --pv-recycler-minimum-timeout-nfs int32 The minimum ActiveDeadlineSeconds to use for an NFS Recycler pod (default 300)
--pv-recycler-pod-template-filepath-hostpath string The file path to a pod definition used as a template for HostPath persistent volume recycling. This is for development and testing only and will not work in a multi-node cluster.
--pv-recycler-pod-template-filepath-nfs string The file path to a pod definition used as a template for NFS persistent volume recycling
- --pv-recycler-timeout-increment-hostpath value the increment of time added per Gi to ActiveDeadlineSeconds for a HostPath scrubber pod. This is for development and testing only and will not work in a multi-node cluster. (default 30)
+ --pv-recycler-timeout-increment-hostpath int32 the increment of time added per Gi to ActiveDeadlineSeconds for a HostPath scrubber pod. This is for development and testing only and will not work in a multi-node cluster. (default 30)
--pvclaimbinder-sync-period duration The period for syncing persistent volumes and persistent volume claims (default 15s)
- --replicaset-lookup-cache-size value The the size of lookup cache for replicatsets. Larger number = more responsive replica management, but more MEM load. (default 4096)
- --replication-controller-lookup-cache-size value The the size of lookup cache for replication controllers. Larger number = more responsive replica management, but more MEM load. (default 4096)
+ --replicaset-lookup-cache-size int32 The size of lookup cache for replicatsets. Larger number = more responsive replica management, but more MEM load. (default 4096)
+ --replication-controller-lookup-cache-size int32 The size of lookup cache for replication controllers. Larger number = more responsive replica management, but more MEM load. (default 4096)
--resource-quota-sync-period duration The period for syncing quota usage status in the system (default 5m0s)
--root-ca-file string If set, this root certificate authority will be included in service account's token secret. This must be a valid PEM-encoded CA bundle.
- --secondary-node-eviction-rate value Number of nodes per second on which pods are deleted in case of node failure when a zone is unhealthy (see --unhealthy-zone-threshold for definition of healthy/unhealthy). Zone refers to entire cluster in non-multizone clusters. This value is implicitly overridden to 0 if the cluster size is smaller than --large-cluster-size-threshold. (default 0.01)
- --service-account-private-key-file string Filename containing a PEM-encoded private RSA key used to sign service account tokens.
+ --route-reconciliation-period duration The period for reconciling routes created for Nodes by cloud provider. (default 10s)
+ --secondary-node-eviction-rate float32 Number of nodes per second on which pods are deleted in case of node failure when a zone is unhealthy (see --unhealthy-zone-threshold for definition of healthy/unhealthy). Zone refers to entire cluster in non-multizone clusters. This value is implicitly overridden to 0 if the cluster size is smaller than --large-cluster-size-threshold. (default 0.01)
+ --service-account-private-key-file string Filename containing a PEM-encoded private RSA or ECDSA key used to sign service account tokens.
--service-cluster-ip-range string CIDR Range for Services in cluster.
--service-sync-period duration The period for syncing services with their external load balancers (default 5m0s)
- --terminated-pod-gc-threshold value Number of terminated pods that can exist before the terminated pod garbage collector starts deleting terminated pods. If <= 0, the terminated pod garbage collector is disabled. (default 12500)
- --unhealthy-zone-threshold value Fraction of Nodes in a zone which needs to be not Ready (minimum 3) for zone to be treated as unhealthy. (default 0.55)
+ --terminated-pod-gc-threshold int32 Number of terminated pods that can exist before the terminated pod garbage collector starts deleting terminated pods. If <= 0, the terminated pod garbage collector is disabled. (default 12500)
+ --unhealthy-zone-threshold float32 Fraction of Nodes in a zone which needs to be not Ready (minimum 3) for zone to be treated as unhealthy. (default 0.55)
+ --use-service-account-credentials If true, use individual service account credentials for each controller.
```
-###### Auto generated by spf13/cobra on 24-Oct-2016
-
-
-
-
+###### Auto generated by spf13/cobra on 13-Dec-2016
diff --git a/docs/admin/kube-proxy.md b/docs/admin/kube-proxy.md
index 98480612cb..ea13d528e3 100644
--- a/docs/admin/kube-proxy.md
+++ b/docs/admin/kube-proxy.md
@@ -1,4 +1,6 @@
---
+title: kube-proxy
+notitle: true
---
## kube-proxy
@@ -23,42 +25,42 @@ kube-proxy
### Options
```
- --bind-address value The IP address for the proxy server to serve on (set to 0.0.0.0 for all interfaces) (default 0.0.0.0)
+ --bind-address ip The IP address for the proxy server to serve on (set to 0.0.0.0 for all interfaces) (default 0.0.0.0)
--cleanup-iptables If true cleanup iptables rules and exit.
--cluster-cidr string The CIDR range of pods in the cluster. It is used to bridge traffic coming from outside of the cluster. If not provided, no off-cluster bridging will be performed.
--config-sync-period duration How often configuration from the apiserver is refreshed. Must be greater than 0. (default 15m0s)
- --conntrack-max-per-core value Maximum number of NAT connections to track per CPU core (0 to leave the limit as-is and ignore conntrack-min). (default 32768)
- --conntrack-min value Minimum number of conntrack entries to allocate, regardless of conntrack-max-per-core (set conntrack-max-per-core=0 to leave the limit as-is). (default 131072)
+ --conntrack-max-per-core int32 Maximum number of NAT connections to track per CPU core (0 to leave the limit as-is and ignore conntrack-min). (default 32768)
+ --conntrack-min int32 Minimum number of conntrack entries to allocate, regardless of conntrack-max-per-core (set conntrack-max-per-core=0 to leave the limit as-is). (default 131072)
+ --conntrack-tcp-timeout-close-wait duration NAT timeout for TCP connections in the CLOSE_WAIT state (default 1h0m0s)
--conntrack-tcp-timeout-established duration Idle timeout for established TCP connections (0 to leave as-is) (default 24h0m0s)
- --feature-gates value A set of key=value pairs that describe feature gates for alpha/experimental features. Options are:
+ --feature-gates mapStringBool A set of key=value pairs that describe feature gates for alpha/experimental features. Options are:
AllAlpha=true|false (ALPHA - default=false)
-AllowExtTrafficLocalEndpoints=true|false (ALPHA - default=false)
+AllowExtTrafficLocalEndpoints=true|false (BETA - default=true)
AppArmor=true|false (BETA - default=true)
DynamicKubeletConfig=true|false (ALPHA - default=false)
DynamicVolumeProvisioning=true|false (ALPHA - default=true)
+ExperimentalHostUserNamespaceDefaulting=true|false (ALPHA - default=false)
+StreamingProxyRedirects=true|false (ALPHA - default=false)
--google-json-key string The Google Cloud Platform Service Account JSON Key to use for authentication.
- --healthz-bind-address value The IP address for the health check server to serve on, defaulting to 127.0.0.1 (set to 0.0.0.0 for all interfaces) (default 127.0.0.1)
- --healthz-port value The port to bind the health check server. Use 0 to disable. (default 10249)
+ --healthz-bind-address ip The IP address for the health check server to serve on, defaulting to 127.0.0.1 (set to 0.0.0.0 for all interfaces) (default 127.0.0.1)
+ --healthz-port int32 The port to bind the health check server. Use 0 to disable. (default 10249)
--hostname-override string If non-empty, will use this string as identification instead of the actual hostname.
- --iptables-masquerade-bit value If using the pure iptables proxy, the bit of the fwmark space to mark packets requiring SNAT with. Must be within the range [0, 31]. (default 14)
- --iptables-sync-period duration How often iptables rules are refreshed (e.g. '5s', '1m', '2h22m'). Must be greater than 0. (default 30s)
- --kube-api-burst value Burst to use while talking with kubernetes apiserver (default 10)
+ --iptables-masquerade-bit int32 If using the pure iptables proxy, the bit of the fwmark space to mark packets requiring SNAT with. Must be within the range [0, 31]. (default 14)
+ --iptables-min-sync-period duration The minimum interval of how often the iptables rules can be refreshed as endpoints and services change (e.g. '5s', '1m', '2h22m').
+ --iptables-sync-period duration The maximum interval of how often iptables rules are refreshed (e.g. '5s', '1m', '2h22m'). Must be greater than 0. (default 30s)
+ --kube-api-burst int32 Burst to use while talking with Kubernetes apiserver (default 10)
--kube-api-content-type string Content type of requests sent to apiserver. (default "application/vnd.kubernetes.protobuf")
- --kube-api-qps value QPS to use while talking with kubernetes apiserver (default 5)
+ --kube-api-qps float32 QPS to use while talking with Kubernetes apiserver (default 5)
--kubeconfig string Path to kubeconfig file with authorization information (the master location is set by the master flag).
--masquerade-all If using the pure iptables proxy, SNAT everything
--master string The address of the Kubernetes API server (overrides any value in kubeconfig)
- --oom-score-adj value The oom-score-adj value for kube-proxy process. Values must be within the range [-1000, 1000] (default -999)
- --proxy-mode value Which proxy mode to use: 'userspace' (older) or 'iptables' (faster). If blank, look at the Node object on the Kubernetes API and respect the 'net.experimental.kubernetes.io/proxy-mode' annotation if provided. Otherwise use the best-available proxy (currently iptables). If the iptables proxy is selected, regardless of how, but the system's kernel or iptables versions are insufficient, this always falls back to the userspace proxy.
- --proxy-port-range value Range of host ports (beginPort-endPort, inclusive) that may be consumed in order to proxy service traffic. If unspecified (0-0) then ports will be randomly chosen.
+ --oom-score-adj int32 The oom-score-adj value for kube-proxy process. Values must be within the range [-1000, 1000] (default -999)
+ --proxy-mode ProxyMode Which proxy mode to use: 'userspace' (older) or 'iptables' (faster). If blank, look at the Node object on the Kubernetes API and respect the 'net.experimental.kubernetes.io/proxy-mode' annotation if provided. Otherwise use the best-available proxy (currently iptables). If the iptables proxy is selected, regardless of how, but the system's kernel or iptables versions are insufficient, this always falls back to the userspace proxy.
+ --proxy-port-range port-range Range of host ports (beginPort-endPort, inclusive) that may be consumed in order to proxy service traffic. If unspecified (0-0) then ports will be randomly chosen.
--udp-timeout duration How long an idle UDP connection will be kept open (e.g. '250ms', '2s'). Must be greater than 0. Only applicable for proxy-mode=userspace (default 250ms)
```
-###### Auto generated by spf13/cobra on 24-Oct-2016
-
-
-
-
+###### Auto generated by spf13/cobra on 13-Dec-2016
diff --git a/docs/admin/kube-scheduler.md b/docs/admin/kube-scheduler.md
index 3316d7e10d..15e47d2f46 100644
--- a/docs/admin/kube-scheduler.md
+++ b/docs/admin/kube-scheduler.md
@@ -1,4 +1,6 @@
---
+title: kube-scheduler
+notitle: true
---
## kube-scheduler
@@ -24,19 +26,21 @@ kube-scheduler
```
--address string The IP address to serve on (set to 0.0.0.0 for all interfaces) (default "0.0.0.0")
- --algorithm-provider string The scheduling algorithm provider to use, one of: DefaultProvider | ClusterAutoscalerProvider (default "DefaultProvider")
+ --algorithm-provider string The scheduling algorithm provider to use, one of: ClusterAutoscalerProvider | DefaultProvider (default "DefaultProvider")
--failure-domains string Indicate the "all topologies" set for an empty topologyKey when it's used for PreferredDuringScheduling pod anti-affinity. (default "kubernetes.io/hostname,failure-domain.beta.kubernetes.io/zone,failure-domain.beta.kubernetes.io/region")
- --feature-gates value A set of key=value pairs that describe feature gates for alpha/experimental features. Options are:
+ --feature-gates mapStringBool A set of key=value pairs that describe feature gates for alpha/experimental features. Options are:
AllAlpha=true|false (ALPHA - default=false)
-AllowExtTrafficLocalEndpoints=true|false (ALPHA - default=false)
+AllowExtTrafficLocalEndpoints=true|false (BETA - default=true)
AppArmor=true|false (BETA - default=true)
DynamicKubeletConfig=true|false (ALPHA - default=false)
DynamicVolumeProvisioning=true|false (ALPHA - default=true)
+ExperimentalHostUserNamespaceDefaulting=true|false (ALPHA - default=false)
+StreamingProxyRedirects=true|false (ALPHA - default=false)
--google-json-key string The Google Cloud Platform Service Account JSON Key to use for authentication.
--hard-pod-affinity-symmetric-weight int RequiredDuringScheduling affinity is not symmetric, but there is an implicit PreferredDuringScheduling affinity rule corresponding to every RequiredDuringScheduling affinity rule. --hard-pod-affinity-symmetric-weight represents the weight of implicit PreferredDuringScheduling affinity rule. (default 1)
- --kube-api-burst value Burst to use while talking with kubernetes apiserver (default 100)
+ --kube-api-burst int32 Burst to use while talking with Kubernetes apiserver (default 100)
--kube-api-content-type string Content type of requests sent to apiserver. (default "application/vnd.kubernetes.protobuf")
- --kube-api-qps value QPS to use while talking with kubernetes apiserver (default 50)
+ --kube-api-qps float32 QPS to use while talking with Kubernetes apiserver (default 50)
--kubeconfig string Path to kubeconfig file with authorization and master location information.
--leader-elect Start a leader election client and gain leadership before executing the main loop. Enable this when running replicated components for high availability. (default true)
--leader-elect-lease-duration duration The duration that non-leader candidates will wait after observing a leadership renewal until attempting to acquire leadership of a led but unrenewed leader slot. This is effectively the maximum duration that a leader can be stopped before it is replaced by another candidate. This is only applicable if leader election is enabled. (default 15s)
@@ -44,16 +48,12 @@ DynamicVolumeProvisioning=true|false (ALPHA - default=true)
--leader-elect-retry-period duration The duration the clients should wait between attempting acquisition and renewal of a leadership. This is only applicable if leader election is enabled. (default 2s)
--master string The address of the Kubernetes API server (overrides any value in kubeconfig)
--policy-config-file string File with scheduler policy configuration
- --port value The port that the scheduler's http service runs on (default 10251)
+ --port int32 The port that the scheduler's http service runs on (default 10251)
--profiling Enable profiling via web interface host:port/debug/pprof/ (default true)
--scheduler-name string Name of the scheduler, used to select which pods will be processed by this scheduler, based on pod's annotation with key 'scheduler.alpha.kubernetes.io/name' (default "default-scheduler")
```
-###### Auto generated by spf13/cobra on 24-Oct-2016
-
-
-
-
+###### Auto generated by spf13/cobra on 13-Dec-2016
diff --git a/docs/admin/kubeadm.md b/docs/admin/kubeadm.md
index e1c8537149..1014ee1ab6 100644
--- a/docs/admin/kubeadm.md
+++ b/docs/admin/kubeadm.md
@@ -4,10 +4,9 @@ assignees:
- luxas
- errordeveloper
- jbeda
-
+title: kubeadm reference
---
-
This document provides information on how to use kubeadm's advanced options.
Running `kubeadm init` bootstraps a Kubernetes cluster. This consists of the
@@ -82,9 +81,12 @@ of the box. You can specify a cloud provider using `--cloud-provider`.
Valid values are the ones supported by `controller-manager`, namely `"aws"`,
`"azure"`, `"cloudstack"`, `"gce"`, `"mesos"`, `"openstack"`, `"ovirt"`,
`"rackspace"`, `"vsphere"`. In order to provide additional configuration for
-the cloud provider, you should create a `/etc/kubernetes/cloud-config.json`
+the cloud provider, you should create a `/etc/kubernetes/cloud-config`
file manually, before running `kubeadm init`. `kubeadm` automatically
picks those settings up and ensures other nodes are configured correctly.
+The exact format and content of the file `/etc/kubernetes/cloud-config` depends
+on the type you specified for `--cloud-provider`; see the appropriate documentation
+for your cloud provider for details.
You must also set the `--cloud-provider` and `--cloud-config` parameters
yourself by editing the `/etc/systemd/system/kubelet.service.d/10-kubeadm.conf`
file appropriately.
@@ -141,10 +143,10 @@ By default, `kubeadm init` automatically generates the token used to initialise
each new node. If you would like to manually specify this token, you can use the
`--token` flag. The token must be of the format `<6 character string>.<16 character string>`.
-- `--use-kubernetes-version` (default 'v1.4.4') the kubernetes version to initialise
+- `--use-kubernetes-version` (default 'v1.5.1') the kubernetes version to initialise
`kubeadm` was originally built for Kubernetes version **v1.4.0**, older versions are not
-supported. With this flag you can try any future version, e.g. **v1.5.0-beta.1**
+supported. With this flag you can try any future version, e.g. **v1.6.0-beta.1**
whenever it comes out (check [releases page](https://github.com/kubernetes/kubernetes/releases)
for a full list of available versions).
@@ -168,6 +170,59 @@ necessary.
By default, when `kubeadm init` runs, a token is generated and revealed in the output.
That's the token you should use here.
+
+## Using kubeadm with a configuration file
+
+WARNING: kubeadm is in alpha and the configuration API syntax will likely change before GA.
+
+It's possible to configure kubeadm with a configuration file instead of command line flags, and some more advanced features may only be
+available as configuration file options.
+
+### Sample Master Configuration
+
+ ```yaml
+ apiVersion: kubeadm.k8s.io/v1alpha1
+ kind: MasterConfiguration
+ api:
+ advertiseAddresses:
+ -
+ -
+ bindPort:
+ externalDNSNames:
+ -
+ -
+ cloudProvider:
+ discovery:
+ bindPort:
+ etcd:
+ endpoints:
+ -
+ -
+ caFile:
+ certFile:
+ keyFile:
+ kubernetesVersion:
+ networking:
+ dnsDomain:
+ serviceSubnet:
+ podSubnet:
+ secrets:
+ givenToken:
+ ```
+
+### Sample Node Configuration
+
+ ```yaml
+ apiVersion: kubeadm.k8s.io/v1alpha1
+ kind: NodeConfiguration
+ apiPort:
+ discoveryPort:
+ masterAddresses:
+ -
+ secrets:
+ givenToken:
+ ```
+
## Automating kubeadm
Rather than copying the token you obtained from `kubeadm init` to each node, as
@@ -175,13 +230,12 @@ in the basic `kubeadm` tutorials, you can parallelize the token distribution for
easier automation. To implement this automation, you must know the IP address
that the master will have after it is started.
-1. Generate a token. This token must have the form `<6 character string>.<16
-character string>`
+1. Generate a token. This token must have the form `<6 character string>.<16 character string>`.
- Here is a simple python one-liner for this:
+ Kubeadm can pre-generate a token for you:
- ```
- python -c 'import random; print "%0x.%0x" % (random.SystemRandom().getrandbits(3*8), random.SystemRandom().getrandbits(8*8))'
+ ```console
+ $ kubeadm token generate
```
1. Start both the master node and the worker nodes concurrently with this token. As they come up they should find each other and form the cluster.
@@ -191,6 +245,7 @@ Once the cluster is up, you can grab the admin credentials from the master node
## Environment variables
There are some environment variables that modify the way that `kubeadm` works. Most users will have no need to set these.
+These environment variables are a short-term solution, eventually they will be integrated in the kubeadm configuration file.
| Variable | Default | Description |
| --- | --- | --- |
@@ -200,36 +255,10 @@ There are some environment variables that modify the way that `kubeadm` works.
| `KUBE_HYPERKUBE_IMAGE` | `` | If set, use a single hyperkube image with this name. If not set, individual images per server component will be used. |
| `KUBE_DISCOVERY_IMAGE` | `gcr.io/google_containers/kube-discovery-:1.0` | The bootstrap discovery helper image to use. |
| `KUBE_ETCD_IMAGE` | `gcr.io/google_containers/etcd-:2.2.5` | The etcd container image to use. |
-| `KUBE_COMPONENT_LOGLEVEL` | `--v=4` | Logging configuration for all Kubernetes components |
-
+| `KUBE_REPO_PREFIX` | `gcr.io/google_containers` | The image prefix for all images that are used. |
## Releases and release notes
If you already have kubeadm installed and want to upgrade, run `apt-get update && apt-get upgrade` or `yum update` to get the latest version of kubeadm.
- - Second release between v1.4 and v1.5: `v1.5.0-alpha.2.421+a6bea3d79b8bba`
- - Switch to the 10.96.0.0/12 subnet: [#35290](https://github.com/kubernetes/kubernetes/pull/35290)
- - Fix kubeadm on AWS by including /etc/ssl/certs in the controller-manager [#33681](https://github.com/kubernetes/kubernetes/pull/33681)
- - The API was refactored and is now componentconfig: [#33728](https://github.com/kubernetes/kubernetes/pull/33728), [#34147](https://github.com/kubernetes/kubernetes/pull/34147) and [#34555](https://github.com/kubernetes/kubernetes/pull/34555)
- - Allow kubeadm to get config options from a file: [#34501](https://github.com/kubernetes/kubernetes/pull/34501), [#34885](https://github.com/kubernetes/kubernetes/pull/34885) and [#34891](https://github.com/kubernetes/kubernetes/pull/34891)
- - Implement preflight checks: [#34341](https://github.com/kubernetes/kubernetes/pull/34341) and [#35843](https://github.com/kubernetes/kubernetes/pull/35843)
- - Using kubernetes v1.4.4 by default: [#34419](https://github.com/kubernetes/kubernetes/pull/34419) and [#35270](https://github.com/kubernetes/kubernetes/pull/35270)
- - Make api and discovery ports configurable and default to 6443: [#34719](https://github.com/kubernetes/kubernetes/pull/34719)
- - Implement kubeadm reset: [#34807](https://github.com/kubernetes/kubernetes/pull/34807)
- - Make kubeadm poll/wait for endpoints instead of directly fail when the master isn't available [#34703](https://github.com/kubernetes/kubernetes/pull/34703) and [#34718](https://github.com/kubernetes/kubernetes/pull/34718)
- - Allow empty directories in the directory preflight check: [#35632](https://github.com/kubernetes/kubernetes/pull/35632)
- - Started adding unit tests: [#35231](https://github.com/kubernetes/kubernetes/pull/35231), [#35326](https://github.com/kubernetes/kubernetes/pull/35326) and [#35332](https://github.com/kubernetes/kubernetes/pull/35332)
- - Various enhancements: [#35075](https://github.com/kubernetes/kubernetes/pull/35075), [#35111](https://github.com/kubernetes/kubernetes/pull/35111), [#35119](https://github.com/kubernetes/kubernetes/pull/35119), [#35124](https://github.com/kubernetes/kubernetes/pull/35124), [#35265](https://github.com/kubernetes/kubernetes/pull/35265) and [#35777](https://github.com/kubernetes/kubernetes/pull/35777)
- - Bug fixes: [#34352](https://github.com/kubernetes/kubernetes/pull/34352), [#34558](https://github.com/kubernetes/kubernetes/pull/34558), [#34573](https://github.com/kubernetes/kubernetes/pull/34573), [#34834](https://github.com/kubernetes/kubernetes/pull/34834), [#34607](https://github.com/kubernetes/kubernetes/pull/34607), [#34907](https://github.com/kubernetes/kubernetes/pull/34907) and [#35796](https://github.com/kubernetes/kubernetes/pull/35796)
- - Initial v1.4 release: `v1.5.0-alpha.0.1534+cf7301f16c0363`
-
-
-## Troubleshooting
-
-* Some users on RHEL/CentOS 7 have reported issues with traffic being routed incorrectly due to iptables being bypassed. You should ensure `net.bridge.bridge-nf-call-iptables` is set to 1 in your sysctl config, eg.
-
-```
-# cat /etc/sysctl.d/k8s.conf
-net.bridge.bridge-nf-call-ip6tables = 1
-net.bridge.bridge-nf-call-iptables = 1
-```
+Refer to the [CHANGELOG.md](https://github.com/kubernetes/kubeadm/blob/master/CHANGELOG.md) for more information.
diff --git a/docs/admin/kubelet-authentication-authorization.md b/docs/admin/kubelet-authentication-authorization.md
new file mode 100644
index 0000000000..aa8abf85df
--- /dev/null
+++ b/docs/admin/kubelet-authentication-authorization.md
@@ -0,0 +1,87 @@
+---
+assignees:
+- liggitt
+title: Kubelet authentication/authorization
+---
+
+* TOC
+{:toc}
+
+## Overview
+
+A kubelet's HTTPS endpoint exposes APIs which give access to data of varying sensitivity,
+and allow you to perform operations with varying levels of power on the node and within containers.
+
+This document describes how to authenticate and authorize access to the kubelet's HTTPS endpoint.
+
+## Kubelet authentication
+
+By default, requests to the kubelet's HTTPS endpoint that are not rejected by other configured
+authentication methods are treated as anonymous requests, and given a username of `system:anonymous`
+and a group of `system:unauthenticated`.
+
+To disable anonymous access and send `401 Unauthorized` responses to unauthenticated requests:
+
+* start the kubelet with the `--anonymous-auth=false` flag
+
+To enable X509 client certificate authentication to the kubelet's HTTPS endpoint:
+
+* start the kubelet with the `--client-ca-file` flag, providing a CA bundle to verify client certificates with
+* start the apiserver with `--kubelet-client-certificate` and `--kubelet-client-key` flags
+* see the [apiserver authentication documentation](/docs/admin/authentication/#x509-client-certs) for more details
+
+To enable API bearer tokens (including service account tokens) to be used to authenticate to the kubelet's HTTPS endpoint:
+
+* ensure the `authentication.k8s.io/v1beta1` API group is enabled in the API server
+* start the kubelet with the `--authentication-token-webhook`, `--kubeconfig`, and `--require-kubeconfig` flags
+* the kubelet calls the `TokenReview` API on the configured API server to determine user information from bearer tokens
+
+## Kubelet authorization
+
+Any request that is successfully authenticated (including an anonymous request) is then authorized. The default authorization mode is `AlwaysAllow`, which allows all requests.
+
+There are many possible reasons to subdivide access to the kubelet API:
+
+* anonymous auth is enabled, but anonymous users' ability to call the kubelet API should be limited
+* bearer token auth is enabled, but arbitrary API users' (like service accounts) ability to call the kubelet API should be limited
+* client certificate auth is enabled, but only some of the client certificates signed by the configured CA should be allowed to use the kubelet API
+
+To subdivide access to the kubelet API, delegate authorization to the API server:
+
+* ensure the `authorization.k8s.io/v1beta1` API group is enabled in the API server
+* start the kubelet with the `--authorization-mode=Webhook`, `--kubeconfig`, and `--require-kubeconfig` flags
+* the kubelet calls the `SubjectAccessReview` API on the configured API server to determine whether each request is authorized
+
+The kubelet authorizes API requests using the same [request attributes](/docs/admin/authorization/#request-attributes) approach as the apiserver.
+
+The verb is determined from the incoming request's HTTP verb:
+
+HTTP verb | request verb
+----------|---------------
+POST | create
+GET, HEAD | get
+PUT | update
+PATCH | patch
+DELETE | delete
+
+The resource and subresource is determined from the incoming request's path:
+
+Kubelet API | resource | subresource
+-------------|----------|------------
+/stats/\* | nodes | stats
+/metrics/\* | nodes | metrics
+/logs/\* | nodes | log
+/spec/\* | nodes | spec
+*all others* | nodes | proxy
+
+The namespace and API group attributes are always an empty string, and
+the resource name is always the name of the kubelet's `Node` API object.
+
+When running in this mode, ensure the user identified by the `--kubelet-client-certificate` and `--kubelet-client-key`
+flags passed to the apiserver is authorized for the following attributes:
+
+* verb=\*, resource=nodes, subresource=proxy
+* verb=\*, resource=nodes, subresource=stats
+* verb=\*, resource=nodes, subresource=log
+* verb=\*, resource=nodes, subresource=spec
+* verb=\*, resource=nodes, subresource=metrics
diff --git a/docs/admin/kubelet-tls-bootstrapping.md b/docs/admin/kubelet-tls-bootstrapping.md
new file mode 100644
index 0000000000..0dfc4bbf55
--- /dev/null
+++ b/docs/admin/kubelet-tls-bootstrapping.md
@@ -0,0 +1,96 @@
+---
+assignees:
+- mikedanese
+title: TLS bootstrapping
+---
+
+* TOC
+{:toc}
+
+## Overview
+
+This document describes how to set up TLS client certificate bootstrapping for kubelets.
+Kubernetes 1.4 introduces an experimental API for requesting certificates from a cluster-level
+Certificate Authority (CA). The first supported use of this API is the provisioning of TLS client
+certificates for kubelets. The proposal can be found [here](https://github.com/kubernetes/kubernetes/pull/20439)
+and progress on the feature is being tracked as [feature #43](https://github.com/kubernetes/features/issues/43).
+
+## apiserver configuration
+
+You must provide a token file which specifies at least one "bootstrap token" assigned to a kubelet bootstrap-specific group.
+This group will later be used in the controller-manager configuration to scope approvals in the default approval
+controller. As this feature matures, you should ensure tokens are bound to an RBAC policy which limits requests
+using the bootstrap token to only be able to make requests related to certificate provisioning. When RBAC policy
+is in place, scoping the tokens to a group will allow great flexibility (e.g. you could disable a particular
+bootstrap group's access when you are done provisioning the nodes).
+
+### Token auth file
+Tokens are arbitrary but should represent at least 128 bits of entropy derived from a secure random number
+generator (such as /dev/urandom on most modern systems). There are multiple ways you can generate a token. For example:
+
+`head -c 16 /dev/urandom | od -An -t x | tr -d ' '`
+
+will generate tokens that look like `02b50b05283e98dd0fd71db496ef01e8`
+
+The token file will look like the following example, where the first three values can be anything and the quoted group
+name should be as depicted:
+
+```
+02b50b05283e98dd0fd71db496ef01e8,kubelet-bootstrap,10001,"system:kubelet-bootstrap"
+```
+
+Add the `--token-auth-file=FILENAME` flag to the apiserver command to enable the token file.
+See docs at http://kubernetes.io/docs/admin/authentication/#static-token-file for further details.
+
+### Client certificate CA bundle
+
+Add the `--client-ca-file=FILENAME` flag to the apiserver command to enable client certificate authentication,
+referencing a certificate authority bundle containing the signing certificate.
+
+## controller-manager configuration
+The API for requesting certificates adds a certificate-issuing control loop to the KCM. This takes the form of a
+[cfssl](https://blog.cloudflare.com/introducing-cfssl/) local signer using assets on disk.
+Currently, all certificates issued have one year validity and a default set of key usages.
+
+### Signing assets
+You must provide a Certificate Authority in order to provide the cryptographic materials necessary to issue certificates.
+This CA should be trusted by the apiserver for authentication with the `--client-ca-file=SOMEFILE` flag. The management
+of the CA is beyond the scope of this document but it is recommended that you generate a dedicated CA for Kubernetes.
+Both certificate and key are assumed to be PEM-encoded.
+
+The new controller-manager flags are:
+```
+--cluster-signing-cert-file="/etc/path/to/kubernetes/ca/ca.crt" --cluster-signing-key-file="/etc/path/to/kubernetes/ca/ca.key"
+```
+
+### Auto-approval
+To ease deployment and testing, the alpha version of the certificate request API includes a flag to approve all certificate
+requests made by users in a certain group. The intended use of this is to whitelist only the group corresponding to the bootstrap
+token in the token file above. Use of this flag circumvents makes the "approval" process described below and is not recommended
+for production use.
+
+The flag is:
+```
+--insecure-experimental-approve-all-kubelet-csrs-for-group="system:kubelet-bootstrap"
+```
+
+## kubelet configuration
+To use request a client cert from the certificate request API, the kubelet needs a path to a kubeconfig file that contains the
+bootstrap auth token. If the file specified by `--kubeconfig` does not exist, the bootstrap kubeconfig is used to request a
+client certificate from the API server. On success, a kubeconfig file referencing the generated key and obtained certificate
+is written to the path specified by `--kubeconfig`. The certificate and key file will be stored in the directory pointed
+by `--cert-dir`. The new flag is:
+
+```
+--experimental-bootstrap-kubeconfig="/path/to/bootstrap/kubeconfig"
+```
+
+## kubectl approval
+The signing controller does not immediately sign all certificate requests. Instead, it waits until they have been flagged with an
+"Approved" status by an appropriately-privileged user. This is intended to eventually be an automated process handled by an external
+approval controller, but for the alpha version of the API it can be done manually by a cluster administrator using kubectl.
+An administrator can list CSRs with `kubectl get csr`, describe one in detail with `kubectl describe `. There are
+[currently no direct approve/deny commands](https://github.com/kubernetes/kubernetes/issues/30163) so an approver will need to update
+the Status field directly. A rough example of how to do this in bash which should only be used until the porcelain merges is available
+at [https://github.com/gtank/csrctl](https://github.com/gtank/csrctl).
+
diff --git a/docs/admin/kubelet.md b/docs/admin/kubelet.md
index 88842eab14..258db1e3aa 100644
--- a/docs/admin/kubelet.md
+++ b/docs/admin/kubelet.md
@@ -1,4 +1,6 @@
---
+title: Overview
+notitle: true
---
## kubelet
@@ -15,7 +17,7 @@ various mechanisms (primarily through the apiserver) and ensures that the contai
described in those PodSpecs are running and healthy. The kubelet doesn't manage
containers which were not created by Kubernetes.
-Other than from an PodSpec from the apiserver, there are three ways that a container
+Other than from a PodSpec from the apiserver, there are three ways that a container
manifest can be provided to the Kubelet.
File: Path passed as a flag on the command line. This file is rechecked every 20
@@ -34,123 +36,134 @@ kubelet
### Options
```
- --address value The IP address for the Kubelet to serve on (set to 0.0.0.0 for all interfaces) (default 0.0.0.0)
- --allow-privileged If true, allow containers to request privileged mode. [default=false]
- --cadvisor-port value The port of the localhost cAdvisor endpoint (default 4194)
- --cert-dir string The directory where the TLS certs are located (by default /var/run/kubernetes). If --tls-cert-file and --tls-private-key-file are provided, this flag will be ignored. (default "/var/run/kubernetes")
- --cgroup-root string Optional root cgroup to use for pods. This is handled by the container runtime on a best effort basis. Default: '', which means use the container runtime default.
- --chaos-chance float If > 0.0, introduce random client errors and latency. Intended for testing. [default=0.0]
- --cloud-config string The path to the cloud provider configuration file. Empty string for no configuration file.
- --cloud-provider string The provider for cloud services. By default, kubelet will attempt to auto-detect the cloud provider. Specify empty string for running with no cloud provider. [default=auto-detect] (default "auto-detect")
- --cluster-dns string IP address for a cluster DNS server. This value is used for containers' DNS server in case of Pods with "dnsPolicy=ClusterFirst"
- --cluster-domain string Domain for this cluster. If set, kubelet will configure all containers to search this domain in addition to the host's search domains
- --cni-bin-dir string The full path of the directory in which to search for CNI plugin binaries. Default: /opt/cni/bin
- --cni-conf-dir string The full path of the directory in which to search for CNI config files. Default: /etc/cni/net.d
- --container-runtime string The container runtime to use. Possible values: 'docker', 'rkt'. Default: 'docker'. (default "docker")
- --container-runtime-endpoint string The unix socket endpoint of remote runtime service. If not empty, this option will override --container-runtime. This is an experimental feature. Intended for testing only.
- --containerized Experimental support for running kubelet in a container. Intended for testing. [default=false]
- --cpu-cfs-quota Enable CPU CFS quota enforcement for containers that specify CPU limits (default true)
- --docker-endpoint string Use this for the docker endpoint to communicate with (default "unix:///var/run/docker.sock")
- --docker-exec-handler string Handler to use when executing a command in a container. Valid values are 'native' and 'nsenter'. Defaults to 'native'. (default "native")
- --enable-controller-attach-detach Enables the Attach/Detach controller to manage attachment/detachment of volumes scheduled to this node, and disables kubelet from executing any attach/detach operations (default true)
- --enable-custom-metrics Support for gathering custom metrics.
- --enable-debugging-handlers Enables server endpoints for log collection and local running of containers and commands (default true)
- --enable-server Enable the Kubelet's server (default true)
- --event-burst value Maximum size of a bursty event records, temporarily allows event records to burst to this number, while still not exceeding event-qps. Only used if --event-qps > 0 (default 10)
- --event-qps value If > 0, limit event creations per second to this value. If 0, unlimited. (default 5)
- --eviction-hard string A set of eviction thresholds (e.g. memory.available<1Gi) that if met would trigger a pod eviction. (default "memory.available<100Mi")
- --eviction-max-pod-grace-period value Maximum allowed grace period (in seconds) to use when terminating pods in response to a soft eviction threshold being met. If negative, defer to pod specified value.
- --eviction-minimum-reclaim string A set of minimum reclaims (e.g. imagefs.available=2Gi) that describes the minimum amount of resource the kubelet will reclaim when performing a pod eviction if that resource is under pressure.
- --eviction-pressure-transition-period duration Duration for which the kubelet has to wait before transitioning out of an eviction pressure condition. (default 5m0s)
- --eviction-soft string A set of eviction thresholds (e.g. memory.available<1.5Gi) that if met over a corresponding grace period would trigger a pod eviction.
- --eviction-soft-grace-period string A set of eviction grace periods (e.g. memory.available=1m30s) that correspond to how long a soft eviction threshold must hold before triggering a pod eviction.
- --exit-on-lock-contention Whether kubelet should exit upon lock-file contention.
- --experimental-allowed-unsafe-sysctls value Comma-separated whitelist of unsafe sysctls or unsafe sysctl patterns (ending in *). Use these at your own risk. (default [])
- --experimental-bootstrap-kubeconfig string Path to a kubeconfig file that will be used to get client certificate for kubelet. If the file specified by --kubeconfig does not exist, the bootstrap kubeconfig is used to request a client certificate from the API server. On success, a kubeconfig file referencing the generated key and obtained certificate is written to the path specified by --kubeconfig. The certificate and key file will be stored in the directory pointed by --cert-dir.
- --experimental-nvidia-gpus value Number of NVIDIA GPU devices on this node. Only 0 (default) and 1 are currently supported.
- --feature-gates value A set of key=value pairs that describe feature gates for alpha/experimental features. Options are:
+ --address ip The IP address for the Kubelet to serve on (set to 0.0.0.0 for all interfaces) (default 0.0.0.0)
+ --allow-privileged If true, allow containers to request privileged mode. [default=false]
+ --anonymous-auth Enables anonymous requests to the Kubelet server. Requests that are not rejected by another authentication method are treated as anonymous requests. Anonymous requests have a username of system:anonymous, and a group name of system:unauthenticated. (default true)
+ --authentication-token-webhook Use the TokenReview API to determine authentication for bearer tokens.
+ --authentication-token-webhook-cache-ttl duration The duration to cache responses from the webhook token authenticator. (default 2m0s)
+ --authorization-mode string Authorization mode for Kubelet server. Valid options are AlwaysAllow or Webhook. Webhook mode uses the SubjectAccessReview API to determine authorization. (default "AlwaysAllow")
+ --authorization-webhook-cache-authorized-ttl duration The duration to cache 'authorized' responses from the webhook authorizer. (default 5m0s)
+ --authorization-webhook-cache-unauthorized-ttl duration The duration to cache 'unauthorized' responses from the webhook authorizer. (default 30s)
+ --cadvisor-port int32 The port of the localhost cAdvisor endpoint (default 4194)
+ --cert-dir string The directory where the TLS certs are located (by default /var/run/kubernetes). If --tls-cert-file and --tls-private-key-file are provided, this flag will be ignored. (default "/var/run/kubernetes")
+ --cgroup-driver string Driver that the kubelet uses to manipulate cgroups on the host. Possible values: 'cgroupfs', 'systemd' (default "cgroupfs")
+ --cgroup-root string Optional root cgroup to use for pods. This is handled by the container runtime on a best effort basis. Default: '', which means use the container runtime default.
+ --chaos-chance float If > 0.0, introduce random client errors and latency. Intended for testing. [default=0.0]
+ --client-ca-file string If set, any request presenting a client certificate signed by one of the authorities in the client-ca-file is authenticated with an identity corresponding to the CommonName of the client certificate.
+ --cloud-config string The path to the cloud provider configuration file. Empty string for no configuration file.
+ --cloud-provider string The provider for cloud services. By default, kubelet will attempt to auto-detect the cloud provider. Specify empty string for running with no cloud provider. [default=auto-detect] (default "auto-detect")
+ --cluster-dns string IP address for a cluster DNS server. This value is used for containers' DNS server in case of Pods with "dnsPolicy=ClusterFirst"
+ --cluster-domain string Domain for this cluster. If set, kubelet will configure all containers to search this domain in addition to the host's search domains
+ --cni-bin-dir string The full path of the directory in which to search for CNI plugin binaries. Default: /opt/cni/bin
+ --cni-conf-dir string The full path of the directory in which to search for CNI config files. Default: /etc/cni/net.d
+ --container-runtime string The container runtime to use. Possible values: 'docker', 'rkt'. Default: 'docker'. (default "docker")
+ --container-runtime-endpoint string [Experimental] The unix socket endpoint of remote runtime service. The endpoint is used only when CRI integration is enabled (--experimental-cri)
+ --containerized Experimental support for running kubelet in a container. Intended for testing. [default=false]
+ --cpu-cfs-quota Enable CPU CFS quota enforcement for containers that specify CPU limits (default true)
+ --docker-endpoint string Use this for the docker endpoint to communicate with (default "unix:///var/run/docker.sock")
+ --docker-exec-handler string Handler to use when executing a command in a container. Valid values are 'native' and 'nsenter'. Defaults to 'native'. (default "native")
+ --enable-controller-attach-detach Enables the Attach/Detach controller to manage attachment/detachment of volumes scheduled to this node, and disables kubelet from executing any attach/detach operations (default true)
+ --enable-custom-metrics Support for gathering custom metrics.
+ --enable-debugging-handlers Enables server endpoints for log collection and local running of containers and commands (default true)
+ --enable-server Enable the Kubelet's server (default true)
+ --event-burst int32 Maximum size of a bursty event records, temporarily allows event records to burst to this number, while still not exceeding event-qps. Only used if --event-qps > 0 (default 10)
+ --event-qps int32 If > 0, limit event creations per second to this value. If 0, unlimited. (default 5)
+ --eviction-hard string A set of eviction thresholds (e.g. memory.available<1Gi) that if met would trigger a pod eviction. (default "memory.available<100Mi")
+ --eviction-max-pod-grace-period int32 Maximum allowed grace period (in seconds) to use when terminating pods in response to a soft eviction threshold being met. If negative, defer to pod specified value.
+ --eviction-minimum-reclaim string A set of minimum reclaims (e.g. imagefs.available=2Gi) that describes the minimum amount of resource the kubelet will reclaim when performing a pod eviction if that resource is under pressure.
+ --eviction-pressure-transition-period duration Duration for which the kubelet has to wait before transitioning out of an eviction pressure condition. (default 5m0s)
+ --eviction-soft string A set of eviction thresholds (e.g. memory.available<1.5Gi) that if met over a corresponding grace period would trigger a pod eviction.
+ --eviction-soft-grace-period string A set of eviction grace periods (e.g. memory.available=1m30s) that correspond to how long a soft eviction threshold must hold before triggering a pod eviction.
+ --exit-on-lock-contention Whether kubelet should exit upon lock-file contention.
+ --experimental-allowed-unsafe-sysctls stringSlice Comma-separated whitelist of unsafe sysctls or unsafe sysctl patterns (ending in *). Use these at your own risk.
+ --experimental-bootstrap-kubeconfig string Path to a kubeconfig file that will be used to get client certificate for kubelet. If the file specified by --kubeconfig does not exist, the bootstrap kubeconfig is used to request a client certificate from the API server. On success, a kubeconfig file referencing the generated key and obtained certificate is written to the path specified by --kubeconfig. The certificate and key file will be stored in the directory pointed by --cert-dir.
+ --experimental-cgroups-per-qos Enable creation of QoS cgroup hierarchy, if true top level QoS and pod cgroups are created.
+ --experimental-check-node-capabilities-before-mount [Experimental] if set true, the kubelet will check the underlying node for required components (binaries, etc.) before performing the mount
+ --experimental-cri [Experimental] Enable the Container Runtime Interface (CRI) integration. If --container-runtime is set to "remote", Kubelet will communicate with the runtime/image CRI server listening on the endpoint specified by --remote-runtime-endpoint/--remote-image-endpoint. If --container-runtime is set to "docker", Kubelet will launch a in-process CRI server on behalf of docker, and communicate over a default endpoint.
+ --experimental-fail-swap-on Makes the Kubelet fail to start if swap is enabled on the node. This is a temporary option to maintain legacy behavior, failing due to swap enabled will happen by default in v1.6.
+ --experimental-kernel-memcg-notification If enabled, the kubelet will integrate with the kernel memcg notification to determine if memory eviction thresholds are crossed rather than polling.
+ --experimental-mounter-path string [Experimental] Path of mounter binary. Leave empty to use the default mount.
+ --experimental-nvidia-gpus int32 Number of NVIDIA GPU devices on this node. Only 0 (default) and 1 are currently supported.
+ --feature-gates string A set of key=value pairs that describe feature gates for alpha/experimental features. Options are:
AllAlpha=true|false (ALPHA - default=false)
-AllowExtTrafficLocalEndpoints=true|false (ALPHA - default=false)
+AllowExtTrafficLocalEndpoints=true|false (BETA - default=true)
AppArmor=true|false (BETA - default=true)
DynamicKubeletConfig=true|false (ALPHA - default=false)
DynamicVolumeProvisioning=true|false (ALPHA - default=true)
- --file-check-frequency duration Duration between checking config files for new data (default 20s)
- --google-json-key string The Google Cloud Platform Service Account JSON Key to use for authentication.
- --hairpin-mode string How should the kubelet setup hairpin NAT. This allows endpoints of a Service to loadbalance back to themselves if they should try to access their own Service. Valid values are "promiscuous-bridge", "hairpin-veth" and "none". (default "promiscuous-bridge")
- --healthz-bind-address value The IP address for the healthz server to serve on, defaulting to 127.0.0.1 (set to 0.0.0.0 for all interfaces) (default 127.0.0.1)
- --healthz-port value The port of the localhost healthz endpoint (default 10248)
- --host-ipc-sources value Comma-separated list of sources from which the Kubelet allows pods to use the host ipc namespace. [default="*"] (default [*])
- --host-network-sources value Comma-separated list of sources from which the Kubelet allows pods to use of host network. [default="*"] (default [*])
- --host-pid-sources value Comma-separated list of sources from which the Kubelet allows pods to use the host pid namespace. [default="*"] (default [*])
- --hostname-override string If non-empty, will use this string as identification instead of the actual hostname.
- --http-check-frequency duration Duration between checking http for new data (default 20s)
- --image-gc-high-threshold value The percent of disk usage after which image garbage collection is always run. Default: 90% (default 90)
- --image-gc-low-threshold value The percent of disk usage before which image garbage collection is never run. Lowest disk usage to garbage collect to. Default: 80% (default 80)
- --image-service-endpoint string The unix socket endpoint of remote image service. If not specified, it will be the same with container-runtime-endpoint by default. This is an experimental feature. Intended for testing only.
- --iptables-drop-bit value The bit of the fwmark space to mark packets for dropping. Must be within the range [0, 31]. (default 15)
- --iptables-masquerade-bit value The bit of the fwmark space to mark packets for SNAT. Must be within the range [0, 31]. Please match this parameter with corresponding parameter in kube-proxy. (default 14)
- --kube-api-burst value Burst to use while talking with kubernetes apiserver (default 10)
- --kube-api-content-type string Content type of requests sent to apiserver. (default "application/vnd.kubernetes.protobuf")
- --kube-api-qps value QPS to use while talking with kubernetes apiserver (default 5)
- --kube-reserved value A set of ResourceName=ResourceQuantity (e.g. cpu=200m,memory=150G) pairs that describe resources reserved for kubernetes system components. Currently only cpu and memory are supported. See http://releases.k8s.io/release-1.4/docs/user-guide/compute-resources.md for more detail. [default=none]
- --kubeconfig value Path to a kubeconfig file, specifying how to connect to the API server. --api-servers will be used for the location unless --require-kubeconfig is set. (default "/var/lib/kubelet/kubeconfig")
- --kubelet-cgroups string Optional absolute name of cgroups to create and run the Kubelet in.
- --lock-file string The path to file for kubelet to use as a lock file.
- --low-diskspace-threshold-mb value The absolute free disk space, in MB, to maintain. When disk space falls below this threshold, new pods would be rejected. Default: 256 (default 256)
- --make-iptables-util-chains If true, kubelet will ensure iptables utility rules are present on host. (default true)
- --manifest-url string URL for accessing the container manifest
- --manifest-url-header string HTTP header to use when accessing the manifest URL, with the key separated from the value with a ':', as in 'key:value'
- --master-service-namespace string The namespace from which the kubernetes master services should be injected into pods (default "default")
- --max-open-files int Number of files that can be opened by Kubelet process. [default=1000000] (default 1000000)
- --max-pods value Number of Pods that can run on this Kubelet. (default 110)
- --minimum-image-ttl-duration duration Minimum age for an unused image before it is garbage collected. Examples: '300ms', '10s' or '2h45m'. Default: '2m' (default 2m0s)
- --network-plugin string The name of the network plugin to be invoked for various events in kubelet/pod lifecycle
- --network-plugin-dir string The full path of the directory in which to search for network plugins or CNI config
- --network-plugin-mtu value The MTU to be passed to the network plugin, to override the default. Set to 0 to use the default 1460 MTU.
- --node-ip string IP address of the node. If set, kubelet will use this IP address for the node
- --node-labels value Labels to add when registering the node in the cluster. Labels must be key=value pairs separated by ','.
- --node-status-update-frequency duration Specifies how often kubelet posts node status to master. Note: be cautious when changing the constant, it must work with nodeMonitorGracePeriod in nodecontroller. Default: 10s (default 10s)
- --non-masquerade-cidr string Traffic to IPs outside this range will use IP masquerade. (default "10.0.0.0/8")
- --oom-score-adj value The oom-score-adj value for kubelet process. Values must be within the range [-1000, 1000] (default -999)
- --outofdisk-transition-frequency duration Duration for which the kubelet has to wait before transitioning out of out-of-disk node condition status. Default: 5m0s (default 5m0s)
- --pod-cidr string The CIDR to use for pod IP addresses, only used in standalone mode. In cluster mode, this is obtained from the master.
- --pod-infra-container-image string The image whose network/ipc namespaces containers in each pod will use. (default "gcr.io/google_containers/pause-amd64:3.0")
- --pod-manifest-path string Path to to the directory containing pod manifest files to run, or the path to a single pod manifest file.
- --pods-per-core value Number of Pods per core that can run on this Kubelet. The total number of Pods on this Kubelet cannot exceed max-pods, so max-pods will be used if this calculation results in a larger number of Pods allowed on the Kubelet. A value of 0 disables this limit.
- --port value The port for the Kubelet to serve on. (default 10250)
- --protect-kernel-defaults Default kubelet behaviour for kernel tuning. If set, kubelet errors if any of kernel tunables is different than kubelet defaults.
- --read-only-port value The read-only port for the Kubelet to serve on with no authentication/authorization (set to 0 to disable) (default 10255)
- --really-crash-for-testing If true, when panics occur crash. Intended for testing.
- --reconcile-cidr Reconcile node CIDR with the CIDR specified by the API server. No-op if register-node or configure-cbr0 is false. [default=true] (default true)
- --register-node Register the node with the apiserver (defaults to true if --api-servers is set) (default true)
- --register-schedulable Register the node as schedulable. No-op if register-node is false. [default=true] (default true)
- --registry-burst value Maximum size of a bursty pulls, temporarily allows pulls to burst to this number, while still not exceeding registry-qps. Only used if --registry-qps > 0 (default 10)
- --registry-qps value If > 0, limit registry pull QPS to this value. If 0, unlimited. [default=5.0] (default 5)
- --require-kubeconfig If true the Kubelet will exit if there are configuration errors, and will ignore the value of --api-servers in favor of the server defined in the kubeconfig file.
- --resolv-conf string Resolver configuration file used as the basis for the container DNS resolution configuration. (default "/etc/resolv.conf")
- --rkt-api-endpoint string The endpoint of the rkt API service to communicate with. Only used if --container-runtime='rkt'. (default "localhost:15441")
- --rkt-path string Path of rkt binary. Leave empty to use the first rkt in $PATH. Only used if --container-runtime='rkt'.
- --root-dir string Directory path for managing kubelet files (volume mounts,etc). (default "/var/lib/kubelet")
- --runonce If true, exit after spawning pods from local manifests or remote urls. Exclusive with --api-servers, and --enable-server
- --runtime-cgroups string Optional absolute name of cgroups to create and run the runtime in.
- --runtime-request-timeout duration Timeout of all runtime requests except long running request - pull, logs, exec and attach. When timeout exceeded, kubelet will cancel the request, throw out an error and retry later. Default: 2m0s (default 2m0s)
- --seccomp-profile-root string Directory path for seccomp profiles.
- --serialize-image-pulls Pull images one at a time. We recommend *not* changing the default value on nodes that run docker daemon with version < 1.9 or an Aufs storage backend. Issue #10959 has more details. [default=true] (default true)
- --streaming-connection-idle-timeout duration Maximum time a streaming connection can be idle before the connection is automatically closed. 0 indicates no timeout. Example: '5m' (default 4h0m0s)
- --sync-frequency duration Max period between synchronizing running containers and config (default 1m0s)
- --system-cgroups / Optional absolute name of cgroups in which to place all non-kernel processes that are not already inside a cgroup under /. Empty for no container. Rolling back the flag requires a reboot. (Default: "").
- --system-reserved value A set of ResourceName=ResourceQuantity (e.g. cpu=200m,memory=150G) pairs that describe resources reserved for non-kubernetes components. Currently only cpu and memory are supported. See http://releases.k8s.io/release-1.4/docs/user-guide/compute-resources.md for more detail. [default=none]
- --tls-cert-file string File containing x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to the directory passed to --cert-dir.
- --tls-private-key-file string File containing x509 private key matching --tls-cert-file.
- --volume-plugin-dir string The full path of the directory in which to search for additional third party volume plugins (default "/usr/libexec/kubernetes/kubelet-plugins/volume/exec/")
- --volume-stats-agg-period duration Specifies interval for kubelet to calculate and cache the volume disk usage for all pods and volumes. To disable volume calculations, set to 0. Default: '1m' (default 1m0s)
+ExperimentalHostUserNamespaceDefaulting=true|false (ALPHA - default=false)
+StreamingProxyRedirects=true|false (ALPHA - default=false)
+ --file-check-frequency duration Duration between checking config files for new data (default 20s)
+ --google-json-key string The Google Cloud Platform Service Account JSON Key to use for authentication.
+ --hairpin-mode string How should the kubelet setup hairpin NAT. This allows endpoints of a Service to loadbalance back to themselves if they should try to access their own Service. Valid values are "promiscuous-bridge", "hairpin-veth" and "none". (default "promiscuous-bridge")
+ --healthz-bind-address ip The IP address for the healthz server to serve on, defaulting to 127.0.0.1 (set to 0.0.0.0 for all interfaces) (default 127.0.0.1)
+ --healthz-port int32 The port of the localhost healthz endpoint (default 10248)
+ --host-ipc-sources stringSlice Comma-separated list of sources from which the Kubelet allows pods to use the host ipc namespace. [default="*"] (default [*])
+ --host-network-sources stringSlice Comma-separated list of sources from which the Kubelet allows pods to use of host network. [default="*"] (default [*])
+ --host-pid-sources stringSlice Comma-separated list of sources from which the Kubelet allows pods to use the host pid namespace. [default="*"] (default [*])
+ --hostname-override string If non-empty, will use this string as identification instead of the actual hostname.
+ --http-check-frequency duration Duration between checking http for new data (default 20s)
+ --image-gc-high-threshold int32 The percent of disk usage after which image garbage collection is always run. Default: 90% (default 90)
+ --image-gc-low-threshold int32 The percent of disk usage before which image garbage collection is never run. Lowest disk usage to garbage collect to. Default: 80% (default 80)
+ --image-service-endpoint string [Experimental] The unix socket endpoint of remote image service. If not specified, it will be the same with container-runtime-endpoint by default. The endpoint is used only when CRI integration is enabled (--experimental-cri)
+ --iptables-drop-bit int32 The bit of the fwmark space to mark packets for dropping. Must be within the range [0, 31]. (default 15)
+ --iptables-masquerade-bit int32 The bit of the fwmark space to mark packets for SNAT. Must be within the range [0, 31]. Please match this parameter with corresponding parameter in kube-proxy. (default 14)
+ --kube-api-burst int32 Burst to use while talking with Kubernetes apiserver (default 10)
+ --kube-api-content-type string Content type of requests sent to apiserver. (default "application/vnd.kubernetes.protobuf")
+ --kube-api-qps int32 QPS to use while talking with Kubernetes apiserver (default 5)
+ --kube-reserved mapStringString A set of ResourceName=ResourceQuantity (e.g. cpu=200m,memory=150G) pairs that describe resources reserved for kubernetes system components. Currently only cpu and memory are supported. See http://kubernetes.io/docs/user-guide/compute-resources for more detail. [default=none]
+ --kubeconfig string Path to a kubeconfig file, specifying how to connect to the API server. --api-servers will be used for the location unless --require-kubeconfig is set. (default "/var/lib/kubelet/kubeconfig")
+ --kubelet-cgroups string Optional absolute name of cgroups to create and run the Kubelet in.
+ --lock-file string The path to file for kubelet to use as a lock file.
+ --low-diskspace-threshold-mb int32 The absolute free disk space, in MB, to maintain. When disk space falls below this threshold, new pods would be rejected. Default: 256 (default 256)
+ --make-iptables-util-chains If true, kubelet will ensure iptables utility rules are present on host. (default true)
+ --manifest-url string URL for accessing the container manifest
+ --manifest-url-header string HTTP header to use when accessing the manifest URL, with the key separated from the value with a ':', as in 'key:value'
+ --master-service-namespace string The namespace from which the Kubernetes master services should be injected into pods (default "default")
+ --max-open-files int Number of files that can be opened by Kubelet process. [default=1000000] (default 1000000)
+ --max-pods int32 Number of Pods that can run on this Kubelet. (default 110)
+ --minimum-image-ttl-duration duration Minimum age for an unused image before it is garbage collected. Examples: '300ms', '10s' or '2h45m'. Default: '2m' (default 2m0s)
+ --network-plugin string The name of the network plugin to be invoked for various events in kubelet/pod lifecycle
+ --network-plugin-dir string The full path of the directory in which to search for network plugins or CNI config
+ --network-plugin-mtu int32 The MTU to be passed to the network plugin, to override the default. Set to 0 to use the default 1460 MTU.
+ --node-ip string IP address of the node. If set, kubelet will use this IP address for the node
+ --node-labels mapStringString Labels to add when registering the node in the cluster. Labels must be key=value pairs separated by ','.
+ --node-status-update-frequency duration Specifies how often kubelet posts node status to master. Note: be cautious when changing the constant, it must work with nodeMonitorGracePeriod in nodecontroller. Default: 10s (default 10s)
+ --non-masquerade-cidr string Traffic to IPs outside this range will use IP masquerade. (default "10.0.0.0/8")
+ --oom-score-adj int32 The oom-score-adj value for kubelet process. Values must be within the range [-1000, 1000] (default -999)
+ --outofdisk-transition-frequency duration Duration for which the kubelet has to wait before transitioning out of out-of-disk node condition status. Default: 5m0s (default 5m0s)
+ --pod-cidr string The CIDR to use for pod IP addresses, only used in standalone mode. In cluster mode, this is obtained from the master.
+ --pod-infra-container-image string The image whose network/ipc namespaces containers in each pod will use. (default "gcr.io/google_containers/pause-amd64:3.0")
+ --pod-manifest-path string Path to to the directory containing pod manifest files to run, or the path to a single pod manifest file.
+ --pods-per-core int32 Number of Pods per core that can run on this Kubelet. The total number of Pods on this Kubelet cannot exceed max-pods, so max-pods will be used if this calculation results in a larger number of Pods allowed on the Kubelet. A value of 0 disables this limit.
+ --port int32 The port for the Kubelet to serve on. (default 10250)
+ --protect-kernel-defaults Default kubelet behaviour for kernel tuning. If set, kubelet errors if any of kernel tunables is different than kubelet defaults.
+ --read-only-port int32 The read-only port for the Kubelet to serve on with no authentication/authorization (set to 0 to disable) (default 10255)
+ --really-crash-for-testing If true, when panics occur crash. Intended for testing.
+ --register-node Register the node with the apiserver (defaults to true if --api-servers is set) (default true)
+ --register-schedulable Register the node as schedulable. Won't have any effect if register-node is false. [default=true] (default true)
+ --registry-burst int32 Maximum size of a bursty pulls, temporarily allows pulls to burst to this number, while still not exceeding registry-qps. Only used if --registry-qps > 0 (default 10)
+ --registry-qps int32 If > 0, limit registry pull QPS to this value. If 0, unlimited. [default=5.0] (default 5)
+ --require-kubeconfig If true the Kubelet will exit if there are configuration errors, and will ignore the value of --api-servers in favor of the server defined in the kubeconfig file.
+ --resolv-conf string Resolver configuration file used as the basis for the container DNS resolution configuration. (default "/etc/resolv.conf")
+ --rkt-api-endpoint string The endpoint of the rkt API service to communicate with. Only used if --container-runtime='rkt'. (default "localhost:15441")
+ --rkt-path string Path of rkt binary. Leave empty to use the first rkt in $PATH. Only used if --container-runtime='rkt'.
+ --root-dir string Directory path for managing kubelet files (volume mounts,etc). (default "/var/lib/kubelet")
+ --runonce If true, exit after spawning pods from local manifests or remote urls. Exclusive with --api-servers, and --enable-server
+ --runtime-cgroups string Optional absolute name of cgroups to create and run the runtime in.
+ --runtime-request-timeout duration Timeout of all runtime requests except long running request - pull, logs, exec and attach. When timeout exceeded, kubelet will cancel the request, throw out an error and retry later. Default: 2m0s (default 2m0s)
+ --seccomp-profile-root string Directory path for seccomp profiles. (default "/var/lib/kubelet/seccomp")
+ --serialize-image-pulls Pull images one at a time. We recommend *not* changing the default value on nodes that run docker daemon with version < 1.9 or an Aufs storage backend. Issue #10959 has more details. [default=true] (default true)
+ --streaming-connection-idle-timeout duration Maximum time a streaming connection can be idle before the connection is automatically closed. 0 indicates no timeout. Example: '5m' (default 4h0m0s)
+ --sync-frequency duration Max period between synchronizing running containers and config (default 1m0s)
+ --system-cgroups / Optional absolute name of cgroups in which to place all non-kernel processes that are not already inside a cgroup under /. Empty for no container. Rolling back the flag requires a reboot. (Default: "").
+ --system-reserved mapStringString A set of ResourceName=ResourceQuantity (e.g. cpu=200m,memory=150G) pairs that describe resources reserved for non-kubernetes components. Currently only cpu and memory are supported. See http://kubernetes.io/docs/user-guide/compute-resources for more detail. [default=none]
+ --tls-cert-file string File containing x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to the directory passed to --cert-dir.
+ --tls-private-key-file string File containing x509 private key matching --tls-cert-file.
+ --volume-plugin-dir string The full path of the directory in which to search for additional third party volume plugins (default "/usr/libexec/kubernetes/kubelet-plugins/volume/exec/")
+ --volume-stats-agg-period duration Specifies interval for kubelet to calculate and cache the volume disk usage for all pods and volumes. To disable volume calculations, set to 0. Default: '1m' (default 1m0s)
```
-###### Auto generated by spf13/cobra on 24-Oct-2016
-
-
-
-
+###### Auto generated by spf13/cobra on 13-Dec-2016
diff --git a/docs/admin/limitrange/index.md b/docs/admin/limitrange/index.md
index 0336264bc3..2241cbb140 100644
--- a/docs/admin/limitrange/index.md
+++ b/docs/admin/limitrange/index.md
@@ -2,7 +2,7 @@
assignees:
- derekwaynecarr
- janetkuo
-
+title: Setting Pod CPU and Memory Limits
---
By default, pods run with unbounded CPU and memory limits. This means that any pod in the
@@ -184,7 +184,7 @@ Note that this pod specifies explicit resource *limits* and *requests* so it did
default values.
Note: The *limits* for CPU resource are enforced in the default Kubernetes setup on the physical node
-that runs the container unless the administrator deploys the kubelet with the folllowing flag:
+that runs the container unless the administrator deploys the kubelet with the following flag:
```shell
$ kubelet --help
diff --git a/docs/admin/master-node-communication.md b/docs/admin/master-node-communication.md
index 3860c33475..91ecff7ef9 100644
--- a/docs/admin/master-node-communication.md
+++ b/docs/admin/master-node-communication.md
@@ -2,13 +2,14 @@
assignees:
- dchen1107
- roberthbailey
-
+- liggitt
+title: Master-Node communication
---
* TOC
{:toc}
-## Summary
+## Overview
This document catalogs the communication paths between the master (really the
apiserver) and the Kubernetes cluster. The intent is to allow users to
@@ -22,14 +23,21 @@ All communication paths from the cluster to the master terminate at the
apiserver (none of the other master components are designed to expose remote
services). In a typical deployment, the apiserver is configured to listen for
remote connections on a secure HTTPS port (443) with one or more forms of
-client [authentication](/docs/admin/authentication/) enabled.
+client [authentication](/docs/admin/authentication/) enabled. One or more forms
+of [authorization](/docs/admin/authorization/) should be enabled, especially
+if [anonymous requests](/docs/admin/authentication/#anonymous-requests) or
+[service account tokens](/docs/admin/authentication/#service-account-tokens)
+are allowed.
Nodes should be provisioned with the public root certificate for the cluster
such that they can connect securely to the apiserver along with valid client
credentials. For example, on a default GCE deployment, the client credentials
-provided to the kubelet are in the form of a client certificate. Pods that
-wish to connect to the apiserver can do so securely by leveraging a service
-account so that Kubernetes will automatically inject the public root
+provided to the kubelet are in the form of a client certificate. See
+[kubelet TLS bootstrapping](/docs/admin/kubelet-tls-bootstrapping/) for
+automated provisioning of kubelet client certificates.
+
+Pods that wish to connect to the apiserver can do so securely by leveraging a
+service account so that Kubernetes will automatically inject the public root
certificate and a valid bearer token into the pod when it is instantiated.
The `kubernetes` service (in all namespaces) is configured with a virtual IP
address that is redirected (via kube-proxy) to the HTTPS endpoint on the
@@ -54,16 +62,29 @@ cluster. The first is from the apiserver to the kubelet process which runs on
each node in the cluster. The second is from the apiserver to any node, pod,
or service through the apiserver's proxy functionality.
+### apiserver -> kubelet
+
The connections from the apiserver to the kubelet are used for fetching logs
for pods, attaching (through kubectl) to running pods, and using the kubelet's
-port-forwarding functionality. These connections terminate at the kubelet's
-HTTPS endpoint, which is typically using a self-signed certificate, and
-ignore the certificate presented by the kubelet (although you can override this
-behavior by specifying the `--kubelet-certificate-authority`,
-`--kubelet-client-certificate`, and `--kubelet-client-key` flags when starting
-the cluster apiserver). By default, these connections **are not currently safe**
-to run over untrusted and/or public networks as they are subject to
-man-in-the-middle attacks.
+port-forwarding functionality. These connections terminate at the kubelet's
+HTTPS endpoint.
+
+By default, the apiserver does not verify the kubelet's serving certificate,
+which makes the connection subject to man-in-the-middle attacks, and
+**unsafe** to run over untrusted and/or public networks.
+
+To verify this connection, use the `--kubelet-certificate-authority` flag to
+provide the apiserver with a root certificates bundle to use to verify the
+kubelet's serving certificate.
+
+If that is not possible, use [SSH tunneling](/docs/admin/master-node-communication/#ssh-tunnels)
+between the apiserver and kubelet if required to avoid connecting over an
+untrusted or public network.
+
+Finally, [Kubelet authentication and/or authorization](/docs/admin/kubelet-authentication-authorization/)
+should be enabled to secure the kubelet API.
+
+### apiserver -> nodes, pods, and services
The connections from the apiserver to a node, pod, or service default to plain
HTTP connections and are therefore neither authenticated nor encrypted. They
@@ -83,83 +104,3 @@ cluster (connecting to the ssh server listening on port 22) and passes all
traffic destined for a kubelet, node, pod, or service through the tunnel.
This tunnel ensures that the traffic is not exposed outside of the private
GCE network in which the cluster is running.
-
-### Kubelet TLS Bootstrap
-
-Kubernetes 1.4 introduces an experimental API for requesting certificates from a cluster-level
-Certificate Authority (CA). The first supported use of this API is the provisioning of TLS client
-certificates for kubelets. The proposal can be found [here](https://github.com/kubernetes/kubernetes/pull/20439)
-and progress on the feature is being tracked as [feature #43](https://github.com/kubernetes/features/issues/43).
-
-##### apiserver configuration
-You must provide a token file which specifies at least one "bootstrap token" assigned to a kubelet boostrap-specific group.
-This group will later be used in the controller-manager configuration to scope approvals in the default approval
-controller. As this feature matures, you should ensure tokens are bound to an RBAC policy which limits requests
-using the bootstrap token to only be able to make requests related to certificate provisioning. When RBAC policy
-is in place, scoping the tokens to a group will allow great flexibility (e.g. you could disable a particular
-bootstrap group's access when you are done provisioning the nodes).
-
-##### Token auth file
-Tokens are arbitrary but should represent at least 128 bits of entropy derived from a secure random number
-generator (such as /dev/urandom on most modern systems). There are multiple ways you can generate a token. For example:
-
-`head -c 16 /dev/urandom | od -An -t x | tr -d ' '`
-
-will generate tokens that look like `02b50b05283e98dd0fd71db496ef01e8`
-
-The token file will look like the following example, where the first three values can be anything and the quoted group
-name should be as depicted:
-
-```
-02b50b05283e98dd0fd71db496ef01e8,kubelet-bootstrap,10001,"system:kubelet-bootstrap"
-```
-
-Add the `--token-auth-file=FILENAME` flag to the apiserver command to enable the token file.
-See docs at http://kubernetes.io/docs/admin/authentication/#static-token-file for further details.
-
-#### controller-manager configuration
-The API for requesting certificates adds a certificate-issuing control loop to the KCM. This takes the form of a
-[cfssl](https://blog.cloudflare.com/introducing-cfssl/) local signer using assets on disk.
-Currently, all certificates issued have one year validity and a default set of key usages.
-
-##### Signing assets
-You must provide a Certificate Authority in order to provide the cryptographic materials necessary to issue certificates.
-This CA should be trusted by the apiserver for authentication with the `--client-ca-file=SOMEFILE` flag. The management
-of the CA is beyond the scope of this document but it is recommended that you generate a dedicated CA for Kubernetes.
-Both certificate and key are assumed to be PEM-encoded.
-
-The new controller-manager flags are:
-```
---cluster-signing-cert-file="/etc/path/to/kubernetes/ca/ca.crt" --cluster-signing-key-file="/etc/path/to/kubernetes/ca/ca.key"
-```
-
-##### Auto-approval
-To ease deployment and testing, the alpha version of the certificate request API includes a flag to approve all certificate
-requests made by users in a certain group. The intended use of this is to whitelist only the group corresponding to the bootstrap
-token in the token file above. Use of this flag circumvents makes the "approval" process described below and is not recommended
-for production use.
-
-The flag is:
-```
---insecure-experimental-approve-all-kubelet-csrs-for-group="system:kubelet-bootstrap"
-```
-
-#### kubelet configuration
-To use request a client cert from the certificate request API, the kubelet needs a path to a kubeconfig file that contains the
-bootstrap auth token. If the file specified by `--kubeconfig` does not exist, the bootstrap kubeconfig is used to request a
-client certificate from the API server. On success, a kubeconfig file referencing the generated key and obtained certificate
-is written to the path specified by `--kubeconfig`. The certificate and key file will be stored in the directory pointed
-by `--cert-dir`. The new flag is:
-
-```
---experimental-bootstrap-kubeconfig="/path/to/bootstrap/kubeconfig"
-```
-
-#### kubectl approval
-The signing controller does not immediately sign all certificate requests. Instead, it waits until they have been flagged with an
-"Approved" status by an appropriately-privileged user. This is intended to eventually be an automated process handled by an external
-approval controller, but for the alpha version of the API it can be done manually by a cluster administrator using kubectl.
-An administrator can list CSRs with `kubectl get csr`, describe one in detail with `kubectl describe `. There are
-[currently no direct approve/deny commands](https://github.com/kubernetes/kubernetes/issues/30163) so an approver will need to update
-the Status field directly. A rough example of how to do this in bash which should only be used until the porcelain merges is available
-at https://github.com/gtank/csrctl.
diff --git a/docs/admin/multi-cluster.md b/docs/admin/multi-cluster.md
index 6359782409..67a2589d40 100644
--- a/docs/admin/multi-cluster.md
+++ b/docs/admin/multi-cluster.md
@@ -1,14 +1,14 @@
---
assignees:
- davidopp
-
+title: Using Multiple Clusters
---
You may want to set up multiple Kubernetes clusters, both to
have clusters in different regions to be nearer to your users, and to tolerate failures and/or invasive maintenance.
This document describes some of the issues to consider when making a decision about doing so.
-If you decide to have multiple clusters, kubernetes provides a way to [federate them](/docs/admin/federation/)
+If you decide to have multiple clusters, Kubernetes provides a way to [federate them](/docs/admin/federation/)
## Scope of a single cluster
@@ -52,7 +52,7 @@ Second, decide how many clusters should be able to be unavailable at the same ti
the number that can be unavailable `U`. If you are not sure, then 1 is a fine choice.
If it is allowable for load-balancing to direct traffic to any region in the event of a cluster failure, then
-you need at least the larger of `R` or `U + 1` clusters. If it is not (e.g you want to ensure low latency for all
+you need at least the larger of `R` or `U + 1` clusters. If it is not (e.g. you want to ensure low latency for all
users in the event of a cluster failure), then you need to have `R * (U + 1)` clusters
(`U + 1` in each of `R` regions). In any case, try to put each cluster in a different zone.
diff --git a/docs/admin/multiple-schedulers.md b/docs/admin/multiple-schedulers.md
index 8ba152ac04..eb1c4c44f9 100644
--- a/docs/admin/multiple-schedulers.md
+++ b/docs/admin/multiple-schedulers.md
@@ -2,7 +2,7 @@
assignees:
- davidopp
- madhusudancs
-
+title: Configuring Multiple Schedulers
---
Kubernetes ships with a default scheduler that is described [here](/docs/admin/kube-scheduler/).
diff --git a/docs/admin/multiple-zones.md b/docs/admin/multiple-zones.md
index bfde54213e..e215b31716 100644
--- a/docs/admin/multiple-zones.md
+++ b/docs/admin/multiple-zones.md
@@ -3,7 +3,7 @@ assignees:
- jlowdermilk
- justinsb
- quinton-hoole
-
+title: Running in Multiple Zones
---
## Introduction
diff --git a/docs/admin/namespaces/index.md b/docs/admin/namespaces/index.md
index 574f41b10a..b723a9c361 100644
--- a/docs/admin/namespaces/index.md
+++ b/docs/admin/namespaces/index.md
@@ -2,7 +2,7 @@
assignees:
- derekwaynecarr
- janetkuo
-
+title: Sharing a Cluster with Namespaces
---
A Namespace is a mechanism to partition resources created by users into
diff --git a/docs/admin/namespaces/walkthrough.md b/docs/admin/namespaces/walkthrough.md
index 2a3e6298ea..b9c509697c 100644
--- a/docs/admin/namespaces/walkthrough.md
+++ b/docs/admin/namespaces/walkthrough.md
@@ -2,7 +2,7 @@
assignees:
- derekwaynecarr
- janetkuo
-
+title: Namespaces Walkthrough
---
Kubernetes _namespaces_ help different projects, teams, or customers to share a Kubernetes cluster.
@@ -151,7 +151,7 @@ Let's create some content.
$ kubectl run snowflake --image=kubernetes/serve_hostname --replicas=2
```
We have just created a deployment whose replica size is 2 that is running the pod called snowflake with a basic container that just serves the hostname.
-Note that `kubectl run` creates deployments only on kubernetes cluster >= v1.2. If you are running older versions, it creates replication controllers instead.
+Note that `kubectl run` creates deployments only on Kubernetes cluster >= v1.2. If you are running older versions, it creates replication controllers instead.
If you want to obtain the old behavior, use `--generator=run/v1` to create replication controllers. See [`kubectl run`](/docs/user-guide/kubectl/kubectl_run/) for more details.
```shell
diff --git a/docs/admin/network-plugins.md b/docs/admin/network-plugins.md
index 6c5f354423..dd51a11b86 100644
--- a/docs/admin/network-plugins.md
+++ b/docs/admin/network-plugins.md
@@ -3,7 +3,7 @@ assignees:
- dcbw
- freehan
- thockin
-
+title: Network Plugins
---
* TOC
@@ -26,13 +26,13 @@ The kubelet has a single default network plugin, and a default network common to
## Network Plugin Requirements
-Besides providing the [`NetworkPlugin` interface](https://github.com/kubernetes/kubernetes/tree/{{page.version}}.0/pkg/kubelet/network/plugins.go) to configure and clean up pod networking, the plugin may also need specific support for kube-proxy. The iptables proxy obviously depends on iptables, and the plugin may need to ensure that container traffic is made available to iptables. For example, if the plugin connects containers to a Linux bridge, the plugin must set the `net/bridge/bridge-nf-call-iptables` sysctl to `1` to ensure that the iptables proxy functions correctly. If the plugin does not use a Linux bridge (but instead something like Open vSwitch or some other mechanism) it should ensure container traffic is appropriately routed for the proxy.
+Besides providing the [`NetworkPlugin` interface](https://github.com/kubernetes/kubernetes/tree/{{page.version}}/pkg/kubelet/network/plugins.go) to configure and clean up pod networking, the plugin may also need specific support for kube-proxy. The iptables proxy obviously depends on iptables, and the plugin may need to ensure that container traffic is made available to iptables. For example, if the plugin connects containers to a Linux bridge, the plugin must set the `net/bridge/bridge-nf-call-iptables` sysctl to `1` to ensure that the iptables proxy functions correctly. If the plugin does not use a Linux bridge (but instead something like Open vSwitch or some other mechanism) it should ensure container traffic is appropriately routed for the proxy.
By default if no kubelet network plugin is specified, the `noop` plugin is used, which sets `net/bridge/bridge-nf-call-iptables=1` to ensure simple configurations (like docker with a bridge) work correctly with the iptables proxy.
### Exec
-Place plugins in `network-plugin-dir/plugin-name/plugin-name`, i.e if you have a bridge plugin and `network-plugin-dir` is `/usr/lib/kubernetes`, you'd place the bridge plugin executable at `/usr/lib/kubernetes/bridge/bridge`. See [this comment](https://github.com/kubernetes/kubernetes/tree/{{page.version}}.0/pkg/kubelet/network/exec/exec.go) for more details.
+Place plugins in `network-plugin-dir/plugin-name/plugin-name`, i.e. if you have a bridge plugin and `network-plugin-dir` is `/usr/lib/kubernetes`, you'd place the bridge plugin executable at `/usr/lib/kubernetes/bridge/bridge`. See [this comment](https://github.com/kubernetes/kubernetes/tree/{{page.version}}/pkg/kubelet/network/exec/exec.go) for more details.
### CNI
@@ -50,13 +50,11 @@ Kubenet is a very basic, simple network plugin, on Linux only. It does not, of
Kubenet creates a Linux bridge named `cbr0` and creates a veth pair for each pod with the host end of each pair connected to `cbr0`. The pod end of the pair is assigned an IP address allocated from a range assigned to the node either through configuration or by the controller-manager. `cbr0` is assigned an MTU matching the smallest MTU of an enabled normal interface on the host.
-The kubenet plugin is mutually exclusive with the --configure-cbr0 option.
-
The plugin requires a few things:
* The standard CNI `bridge`, `lo` and `host-local` plugins are required, at minimum version 0.2.0. Kubenet will first search for them in `/opt/cni/bin`. Specify `network-plugin-dir` to supply additional search path. The first found match will take effect.
* Kubelet must be run with the `--network-plugin=kubenet` argument to enable the plugin
-* Kubelet must also be run with the `--reconcile-cidr` argument to ensure the IP subnet assigned to the node by configuration or the controller-manager is propagated to the plugin
+* Kubelet should also be run with the `--non-masquerade-cidr=` argumment to ensure traffic to IPs outside this range will use IP masquerade.
* The node must be assigned an IP subnet through either the `--pod-cidr` kubelet command-line option or the `--allocate-node-cidrs=true --cluster-cidr=` controller-manager command-line options.
### Customizing the MTU (with kubenet)
diff --git a/docs/admin/networking.md b/docs/admin/networking.md
index 28c259a5f7..08b3d1442e 100644
--- a/docs/admin/networking.md
+++ b/docs/admin/networking.md
@@ -1,8 +1,7 @@
---
assignees:
-- lavalamp
- thockin
-
+title: Networking in Kubernetes
---
Kubernetes approaches networking somewhat differently than Docker does by
@@ -169,12 +168,26 @@ Follow the "With Linux Bridge devices" section of [this very nice
tutorial](http://blog.oddbit.com/2014/08/11/four-ways-to-connect-a-docker/) from
Lars Kellogg-Stedman.
+### Nuage Networks VCS (Virtualized Cloud Services)
+
+[Nuage](http://www.nuagenetworks.net) provides a highly scalable policy-based Software-Defined Networking (SDN) platform. Nuage uses the open source Open vSwitch for the data plane along with a feature rich SDN Controller built on open standards.
+
+The Nuage platform uses overlays to provide seamless policy-based networking between Kubernetes Pods and non-Kubernetes environments (VMs and bare metal servers). Nuage's policy abstraction model is designed with applications in mind and makes it easy to declare fine-grained policies for applications.The platform's real-time analytics engine enables visibility and security monitoring for Kubernetes applications.
+
### OpenVSwitch
[OpenVSwitch](/docs/admin/ovs-networking) is a somewhat more mature but also
complicated way to build an overlay network. This is endorsed by several of the
"Big Shops" for networking.
+### OVN (Open Virtual Networking)
+
+OVN is an opensource network virtualization solution developed by the
+Open vSwitch community. It lets one create logical switches, logical routers,
+stateful ACLs, load-balancers etc to build different virtual networking
+topologies. The project has a specific Kubernetes plugin and documentation
+at [ovn-kubernetes](https://github.com/openvswitch/ovn-kubernetes).
+
### Project Calico
[Project Calico](http://docs.projectcalico.org/) is an open source container networking provider and network policy engine.
diff --git a/docs/admin/node-conformance.md b/docs/admin/node-conformance.md
index 2adb159c5c..f53ba858b1 100644
--- a/docs/admin/node-conformance.md
+++ b/docs/admin/node-conformance.md
@@ -1,7 +1,7 @@
---
assignees:
- Random-Liu
-
+title: Validate Node Setup
---
* TOC
@@ -9,54 +9,52 @@ assignees:
## Node Conformance Test
-*Node conformance test* is a test framework validating whether a node meets the
-minimum requirement of Kubernetes with a set of system verification and
-functionality test. A node which passes the tests is qualified to join a
-Kubernetes cluster.
+*Node conformance test* is a containerized test framework that provides a system
+verification and functionality test for a node. The test validates whether the
+node meets the minimum requirements for Kubernetes; a node that passes the test
+is qualified to join a Kubernetes cluster.
## Limitations
-There are following limitations in the current implementation of node
-conformance test. They'll be improved in future version.
+In Kubernetes version 1.5, node conformance test has the following limitations:
* Node conformance test only supports Docker as the container runtime.
-* Node conformance test doesn't validate network related system configurations
- and functionalities.
-## Prerequisite
+## Node Prerequisite
-Node conformance test is used to test whether a node is ready to join a
-Kubernetes cluster, so the prerequisite is the same with a standard Kubernetes
-node. At least, the node should have properly installed:
+To run node conformance test, a node must satisfy the same prerequisites as a
+standard Kubernetes node. At a minimum, the node should have the following
+daemons installed:
* Container Runtime (Docker)
* Kubelet
-Node conformance test validates kernel configurations. If the kenrel module
-`configs` is built as module in your environment, it must be loaded before the
-test. (See [Caveats #3](#caveats) for more information)
+## Running Node Conformance Test
-## Usage
+To run the node conformance test, perform the following steps:
-### Run Node Conformance Test
+1. Point your Kubelet to localhost `--api-servers="http://localhost:8080"`,
+because the test framework starts a local master to test Kubelet. There are some
+other Kubelet flags you may care:
+ * `--pod-cidr`: If you are using `kubenet`, you should specify an arbitrary CIDR
+ to Kubelet, for example `--pod-cidr=10.180.0.0/24`.
+ * `--cloud-provider`: If you are using `--cloud-provider=gce`, you should
+ remove the flag to run the test.
-* **Step 1:** Point your Kubelet to localhost `--api-servers="http://localhost:8080"`,
-because the test framework starts a local master to test Kubelet.
-
-* **Step 2:** Run the node conformance test with command:
+2. Run the node conformance test with command:
```shell
-# $CONFIG_DIR is the pod manifest path of your kubelet.
+# $CONFIG_DIR is the pod manifest path of your Kubelet.
# $LOG_DIR is the test output path.
sudo docker run -it --rm --privileged --net=host \
- -v /:/rootfs:ro -v /var/run:/var/run \
- -v $CONFIG_DIR:/etc/manifest -v $LOG_DIR:/var/result \
- gcr.io/google_containers/node-test-amd64:v0.1
+ -v /:/rootfs -v $CONFIG_DIR:$CONFIG_DIR -v $LOG_DIR:/var/result \
+ gcr.io/google_containers/node-test:0.2
```
-### Run Node Conformance Test for Other Architectures
+## Running Node Conformance Test for Other Architectures
-We also build node conformance test docker images for other architectures:
+Kubernetes also provides node conformance test docker images for other
+architectures:
Arch | Image |
--------|:-----------------:|
@@ -64,25 +62,16 @@ We also build node conformance test docker images for other architectures:
arm | node-test-arm |
arm64 | node-test-arm64 |
-### Run Selected Test
-
-In fact, Node conformance test is a containerized version of [node e2e
-test](https://github.com/kubernetes/kubernetes/blob/release-1.4/docs/devel/e2e-node-tests.md).
-By default, it runs all conformance test.
-
-Theoretically, you can run any node e2e test if you configure the container and
-mount required volumes properly. But **it is strongly recommended to only run conformance
-test**, because the non-conformance test needs much more complex framework configuration.
+## Running Selected Test
To run specific tests, overwrite the environment variable `FOCUS` with the
regular expression of tests you want to run.
```shell
sudo docker run -it --rm --privileged --net=host \
- -v /:/rootfs:ro -v /var/run:/var/run \
- -v $CONFIG_DIR:/etc/manifest -v $LOG_DIR:/var/result \
+ -v /:/rootfs:ro -v $CONFIG_DIR:$CONFIG_DIR -v $LOG_DIR:/var/result \
-e FOCUS=MirrorPod \ # Only run MirrorPod test
- gcr.io/google_containers/node-test-amd64:v0.1
+ gcr.io/google_containers/node-test:0.2
```
To skip specific tests, overwrite the environment variable `SKIP` with the
@@ -90,25 +79,22 @@ regular expression of tests you want to skip.
```shell
sudo docker run -it --rm --privileged --net=host \
- -v /:/rootfs:ro -v /var/run:/var/run \
- -v $CONFIG_DIR:/etc/manifest -v $LOG_DIR:/var/result \
- -e SKIP=MirrorPod \ # Run all conformance test and skip MirrorPod test
- gcr.io/google_containers/node-test-amd64:v0.1
+ -v /:/rootfs:ro -v $CONFIG_DIR:$CONFIG_DIR -v $LOG_DIR:/var/result \
+ -e SKIP=MirrorPod \ # Run all conformance tests but skip MirrorPod test
+ gcr.io/google_containers/node-test:0.2
```
-### Caveats
+Node conformance test is a containerized version of [node e2e test](https://github.com/kubernetes/kubernetes/blob/release-1.5/docs/devel/e2e-node-tests.md).
+By default, it runs all conformance tests.
-* The test will leave some docker images on the node, including the node
- conformance test image and images of containers used in the functionality
+Theoretically, you can run any node e2e test if you configure the container and
+mount required volumes properly. But **it is strongly recommended to only run conformance
+test**, because it requires much more complex configuration to run non-conformance test.
+
+## Caveats
+
+* The test leaves some docker images on the node, including the node conformance
+ test image and images of containers used in the functionality
test.
-* The test will leave dead containers on the node, these containers are created
+* The test leaves dead containers on the node. These containers are created
during the functionality test.
-* Node conformance test validates kernel configuration. However, in some os
- distro the kernel module `configs` may not be loaded by default, and you will get
- the error `no config path in [POSSIBLE KERNEL CONFIG FILE PATHS] is
- available`. In that case please do either of the followings:
- * Manually load/unload `configs` kernel module: run `sudo modprobe configs` to
- load the kernel module, and `sudo modprobe -r configs` to unload it after the test.
- * Mount `modprobe` into the container: Add option `-v /bin/kmod:/bin/kmod
- -v /sbin/modprobe:/sbin/modprobe -v /lib/modules:/lib/modules` when starting
- the test container.
diff --git a/docs/admin/node-problem.md b/docs/admin/node-problem.md
index b6926ba15b..b4f3e6ee31 100644
--- a/docs/admin/node-problem.md
+++ b/docs/admin/node-problem.md
@@ -2,7 +2,7 @@
assignees:
- Random-Liu
- dchen1107
-
+title: Monitoring Node Health
---
* TOC
@@ -49,7 +49,7 @@ either `kubectl` or addon pod.
### Kubectl
-This is the recommanded way to start node problem detector outside of GCE. It
+This is the recommended way to start node problem detector outside of GCE. It
provides more flexible management, such as overwriting the default
configuration to fit it into your environment or detect
customized node problems.
@@ -238,7 +238,7 @@ implement a new translator for a new log format.
## Caveats
-It is recommanded to run the node problem detector in your cluster to monitor
+It is recommended to run the node problem detector in your cluster to monitor
the node health. However, you should be aware that this will introduce extra
resource overhead on each node. Usually this is fine, because:
diff --git a/docs/admin/node.md b/docs/admin/node.md
index 78ef3c4b2e..0923169256 100644
--- a/docs/admin/node.md
+++ b/docs/admin/node.md
@@ -2,8 +2,7 @@
assignees:
- caesarxuchao
- dchen1107
-- lavalamp
-
+title: Nodes
---
* TOC
@@ -11,44 +10,47 @@ assignees:
## What is a node?
-`Node` is a worker machine in Kubernetes, previously known as `Minion`. Node
+A `node` is a worker machine in Kubernetes, previously known as a `minion`. A node
may be a VM or physical machine, depending on the cluster. Each node has
-the services necessary to run [Pods](/docs/user-guide/pods) and is managed by the master
-components. The services on a node include docker, kubelet and network proxy. See
+the services necessary to run [pods](/docs/user-guide/pods) and is managed by the master
+components. The services on a node include Docker, kubelet and kube-proxy. See
[The Kubernetes Node](https://github.com/kubernetes/kubernetes/blob/{{page.githubbranch}}/docs/design/architecture.md#the-kubernetes-node) section in the
architecture design doc for more details.
## Node Status
-Node status describes current status of a node. For now, there are the following
-pieces of information:
+A node's status contains the following information:
-### Node Addresses
+* [Addresses](#Addresses)
+* ~~[Phase](#Phase)~~ **deprecated**
+* [Condition](#Condition)
+* [Capacity](#Capacity)
+* [Info](#Info)
+
+Each section is described in detail below.
+
+### Addresses
The usage of these fields varies depending on your cloud provider or bare metal configuration.
* HostName: The hostname as reported by the node's kernel. Can be overridden via the kubelet `--hostname-override` parameter.
+* ExternalIP: Typically the IP address of the node that is externally routable (available from outside the cluster).
+* InternalIP: Typically the IP address of the node that is routable only within the cluster.
-* ExternalIP: Generally the IP address of the node that is externally routable (available from outside the cluster)
+### Phase
-* InternalIP: Generally the IP address of the node that is routable only within the cluster
+Deprecated: node phase is no longer used.
-
-### Node Phase
-
-Deprecated: Node Phase is no longer used
-
-### Node Condition
+### Condition
The `conditions` field describes the status of all `Running` nodes.
| Node Condition | Description |
|----------------|-------------|
-| `OutOfDisk` | `True` if insufficient free space on the node for adding new pods, otherwise `False` |
-| `Ready` | `True` if the node is healthy ready to accept pods, `False` if the node is not healthy and is not accepting pods, and `Unknown` if the Node Controller has not heard from the node in the last 40 seconds |
+| `OutOfDisk` | `True` if there is insufficient free space on the node for adding new pods, otherwise `False` |
+| `Ready` | `True` if the node is healthy and ready to accept pods, `False` if the node is not healthy and is not accepting pods, and `Unknown` if the node controller has not heard from the node in the last 40 seconds |
-Node condition is represented as a JSON object. For example, the following response describes a healthy node:
-conditions mean the node is in sane state:
+The node condition is represented as a JSON object. For example, the following response describes a healthy node.
```json
"conditions": [
@@ -59,28 +61,30 @@ conditions mean the node is in sane state:
]
```
-If the Status of the Ready condition
-is Unknown or False for more than five minutes, then all of the Pods on the node are terminated by the Node Controller.
+If the Status of the Ready condition is "Unknown" or "False" for longer than the `pod-eviction-timeout`, an argument passed to the [kube-controller-manager](docs/admin/kube-controller-manager/), all of the Pods on the node are scheduled for deletion by the Node Controller. The default eviction timeout duration is **five minutes**. In some cases when the node is unreachable, the apiserver is unable to communicate with the kubelet on it. The decision to delete the pods cannot be communicated to the kubelet until it re-establishes communication with the apiserver. In the meantime, the pods which are scheduled for deletion may continue to run on the partitioned node.
-### Node Capacity
+In versions of Kubernetes prior to 1.5, the node controller would [force delete](/docs/user-guide/pods/#force-deletion-of-pods) these unreachable pods from the apiserver. However, in 1.5 and higher, the node controller does not force delete pods until it is confirmed that they have stopped running in the cluster. One can see these pods which may be running on an unreachable node as being in the "Terminating" or "Unknown" states. In cases where Kubernetes cannot deduce from the underlying infrastructure if a node has permanently left a cluster, the cluster administrator may need to delete the node object by hand. Deleting the node object from Kubernetes causes all the Pod objects running on it to be deleted from the apiserver, freeing up their names.
-Describes the resources available on the node: CPUs, memory and the maximum
+### Capacity
+
+Describes the resources available on the node: CPU, memory and the maximum
number of pods that can be scheduled onto the node.
-### Node Info
+### Info
-General information about the node, for instance kernel version, Kubernetes version
-(kubelet version, kube-proxy version), docker version (if used), OS name.
+General information about the node, such as kernel version, Kubernetes version
+(kubelet and kube-proxy version), Docker version (if used), OS name.
The information is gathered by Kubelet from the node.
-## Node Management
+## Management
-Unlike [Pods](/docs/user-guide/pods) and [Services](/docs/user-guide/services), a Node is not inherently
-created by Kubernetes: it is either taken from cloud providers like Google Compute Engine,
-or from your pool of physical or virtual machines. What this means is that when
-Kubernetes creates a node, it is really just creating an object that represents the node in its internal state.
-After creation, Kubernetes will check whether the node is valid or not.
-For example, if you try to create a node from the following content:
+Unlike [pods](/docs/user-guide/pods) and [services](/docs/user-guide/services),
+a node is not inherently created by Kubernetes: it is created externally by cloud
+providers like Google Compute Engine, or exists in your pool of physical or virtual
+machines. What this means is that when Kubernetes creates a node, it is really
+just creating an object that represents the node. After creation, Kubernetes
+will check whether the node is valid or not. For example, if you try to create
+a node from the following content:
```json
{
@@ -95,117 +99,127 @@ For example, if you try to create a node from the following content:
}
```
-Kubernetes will create a Node object internally (the representation), and
-validate the node by health checking based on the `metadata.name` field: we
-assume `metadata.name` can be resolved. If the node is valid, i.e. all necessary
-services are running, it is eligible to run a Pod; otherwise, it will be
-ignored for any cluster activity, until it becomes valid. Note that Kubernetes
-will keep the object for the invalid node unless it is explicitly deleted by the client, and it will keep
-checking to see if it becomes valid.
+Kubernetes will create a node object internally (the representation), and
+validate the node by health checking based on the `metadata.name` field (we
+assume `metadata.name` can be resolved). If the node is valid, i.e. all necessary
+services are running, it is eligible to run a pod; otherwise, it will be
+ignored for any cluster activity until it becomes valid. Note that Kubernetes
+will keep the object for the invalid node unless it is explicitly deleted by
+the client, and it will keep checking to see if it becomes valid.
-Currently, there are three components that interact with the Kubernetes node interface: Node Controller, Kubelet, and kubectl.
+Currently, there are three components that interact with the Kubernetes node
+interface: node controller, kubelet, and kubectl.
### Node Controller
-Node controller is a component in Kubernetes master which manages Node
-objects.
+The node controller is a Kubernetes master component which manages various
+aspects of nodes.
-Node controller has mutliple roles in Node's life. First is assigning a CIDR block to
-the Node when it is registered (if CIDR assignment is turned on). Second is keeping the
-node controller's list of nodes up to date with the cloud provider's list of available
-machines. When running in cloud environment whenever a node is unhealthy node controller
-asks cloud provider if the VM for that node is still available. If not, the node
+The node controller has multiple roles in a node's life. The first is assigning a
+CIDR block to the node when it is registered (if CIDR assignment is turned on).
+
+The second is keeping the node controller's internal list of nodes up to date with
+the cloud provider's list of available machines. When running in a cloud
+environment, whenever a node is unhealthy the node controller asks the cloud
+provider if the VM for that node is still available. If not, the node
controller deletes the node from its list of nodes.
-Third responsibiliy is monitoring Node's health. Node controller is responsible for updating
-the NodeReady condition of NodeStatus to ConditionUnknown when a node becomes unreachable
-(i.e. node controller stops receiving heartbeats e.g. due to the node being down), and then
-later evicting all the pods from the node (using graceful termination) if the node continues
-to be unreachable (the current timeouts are 40s to start reporting ConditionUnknown and 5m
-after that to start evicting pods). Node controller checks the state of each node every
-`--node-monitor-period` seconds.
+The third is monitoring the nodes' health. The node controller is
+responsible for updating the NodeReady condition of NodeStatus to
+ConditionUnknown when a node becomes unreachable (i.e. the node controller stops
+receiving heartbeats for some reason, e.g. due to the node being down), and then later evicting
+all the pods from the node (using graceful termination) if the node continues
+to be unreachable. (The default timeouts are 40s to start reporting
+ConditionUnknown and 5m after that to start evicting pods.) The node controller
+checks the state of each node every `--node-monitor-period` seconds.
-In 1.4 release we updated the logic of node controller to better handle cases when a
-big number of Nodes have problems with reaching the master machine (e.g. because
-master machine has networking problem). Starting with 1.4 node controller will look at the
-state of all Nodes in the cluster when making a decision about pod eviction.
+In Kubernetes 1.4, we updated the logic of the node controller to better handle
+cases when a big number of nodes have problems with reaching the master
+(e.g. because the master has networking problem). Starting with 1.4, the node
+controller will look at the state of all nodes in the cluster when making a
+decision about pod eviction.
-In most cases, node controller limits the eviction rate to `--node-eviction-rate` (default 0.1)
-per second, meaning it won't evict pods from more than 1 node per 10 seconds.
+In most cases, node controller limits the eviction rate to
+`--node-eviction-rate` (default 0.1) per second, meaning it won't evict pods
+from more than 1 node per 10 seconds.
-The node eviction behavior changes when a node in a given availability zone becomes unhealthy,
-node controller checks what percentage of nodes in the zone are unhealthy (NodeReady condition
-is ConditionUnknown or ConditionFalse) at the same time. If the fraction of unhealthy nodes is
-at least `--unhealthy-zone-threshold` (default 0.55) then the eviction rate is reduced: if
-the cluster is small (i.e. has less than or equal to `--large-cluster-size-threshold`
-nodes - default 50) then evictions are stopped, otherwise the eviction rate is reduced to
-`--secondary-node-eviction-rate` (default 0.01) per second. The reason these policies are
-implemented per availability zone is because one availability zone might become partitioned
-from the master while the others remain connected. If your cluster does not span multiple cloud
-provider availability zones, then there is only one availability zone, namely the whole cluster.
+The node eviction behavior changes when a node in a given availability zone
+becomes unhealthy. The node controller checks what percentage of nodes in the zone
+are unhealthy (NodeReady condition is ConditionUnknown or ConditionFalse) at
+the same time. If the fraction of unhealthy nodes is at least
+`--unhealthy-zone-threshold` (default 0.55) then the eviction rate is reduced:
+if the cluster is small (i.e. has less than or equal to
+`--large-cluster-size-threshold` nodes - default 50) then evictions are
+stopped, otherwise the eviction rate is reduced to
+`--secondary-node-eviction-rate` (default 0.01) per second. The reason these
+policies are implemented per availability zone is because one availability zone
+might become partitioned from the master while the others remain connected. If
+your cluster does not span multiple cloud provider availability zones, then
+there is only one availability zone (the whole cluster).
-A key reason for spreading your nodes across availability zones is so that workload can be
-shifted to healthy zones when one entire zone goes down. To enable this behavior, if all
-nodes in a zone are unhealthy then node controller evicts at the normal rate `--node-eviction-rate`.
-The corner case for that is when all zones are completely unhealthy (i.e. there's no healthy node in
-the cluster). In such case node controller assumes that there's some problem with master machine
-connectivity and stops all evictions until any connectivity is restored.
+A key reason for spreading your nodes across availability zones is so that the
+workload can be shifted to healthy zones when one entire zone goes down.
+Therefore, if all nodes in a zone are unhealthy then node controller evicts at
+the normal rate `--node-eviction-rate`. The corner case is when all zones are
+completely unhealthy (i.e. there are no healthy nodes in the cluster). In such
+case, the node controller assumes that there's some problem with master
+connectivity and stops all evictions until some connectivity is restored.
### Self-Registration of Nodes
-When kubelet flag `--register-node` is true (the default), the kubelet will attempt to
+When the kubelet flag `--register-node` is true (the default), the kubelet will attempt to
register itself with the API server. This is the preferred pattern, used by most distros.
For self-registration, the kubelet is started with the following options:
- - `--api-servers=` tells the kubelet the location of the apiserver.
- - `--kubeconfig` tells kubelet where to find credentials to authenticate itself to the apiserver.
- - `--cloud-provider=` tells the kubelet how to talk to a cloud provider to read metadata about itself.
- - `--register-node` tells the kubelet to create its own node resource.
+ - `--api-servers=` - Location of the apiservers.
+ - `--kubeconfig=` - Path to credentials to authenticate itself to the apiserver.
+ - `--cloud-provider=` - How to talk to a cloud provider to read metadata about itself.
+ - `--register-node` - Automatically register with the API server.
Currently, any kubelet is authorized to create/modify any node resource, but in practice it only creates/modifies
-its own. (In the future, we plan to limit authorization to only allow a kubelet to modify its own Node resource.)
+its own. (In the future, we plan to only allow a kubelet to modify its own node resource.)
#### Manual Node Administration
-A cluster administrator can create and modify Node objects.
+A cluster administrator can create and modify node objects.
-If the administrator wishes to create node objects manually, set kubelet flag
+If the administrator wishes to create node objects manually, set the kubelet flag
`--register-node=false`.
-The administrator can modify Node resources (regardless of the setting of `--register-node`).
-Modifications include setting labels on the Node, and marking it unschedulable.
+The administrator can modify node resources (regardless of the setting of `--register-node`).
+Modifications include setting labels on the node and marking it unschedulable.
Labels on nodes can be used in conjunction with node selectors on pods to control scheduling,
-e.g. to constrain a Pod to only be eligible to run on a subset of the nodes.
+e.g. to constrain a pod to only be eligible to run on a subset of the nodes.
-Making a node unscheduleable will prevent new pods from being scheduled to that
-node, but will not affect any existing pods on the node. This is useful as a
-preparatory step before a node reboot, etc. For example, to mark a node
+Marking a node as unschedulable will prevent new pods from being scheduled to that
+node, but will not affect any existing pods on the node. This is useful as a
+preparatory step before a node reboot, etc. For example, to mark a node
unschedulable, run this command:
```shell
-kubectl patch nodes $NODENAME -p '{"spec": {"unschedulable": true}}'
+kubectl cordon $NODENAME
```
Note that pods which are created by a daemonSet controller bypass the Kubernetes scheduler,
-and do not respect the unschedulable attribute on a node. The assumption is that daemons belong on
+and do not respect the unschedulable attribute on a node. The assumption is that daemons belong on
the machine even if it is being drained of applications in preparation for a reboot.
### Node capacity
-The capacity of the node (number of cpus and amount of memory) is part of the node resource.
-Normally, nodes register themselves and report their capacity when creating the node resource. If
+The capacity of the node (number of cpus and amount of memory) is part of the node object.
+Normally, nodes register themselves and report their capacity when creating the node object. If
you are doing [manual node administration](#manual-node-administration), then you need to set node
capacity when adding a node.
The Kubernetes scheduler ensures that there are enough resources for all the pods on a node. It
checks that the sum of the limits of containers on the node is no greater than the node capacity. It
-includes all containers started by kubelet, but not containers started directly by docker, nor
+includes all containers started by the kubelet, but not containers started directly by Docker nor
processes not in containers.
-If you want to explicitly reserve resources for non-Pod processes, you can create a placeholder
-pod. Use the following template:
+If you want to explicitly reserve resources for non-pod processes, you can create a placeholder
+pod. Use the following template:
```yaml
apiVersion: v1
@@ -229,6 +243,6 @@ on each kubelet where you want to reserve resources.
## API Object
-Node is a top-level resource in the kubernetes REST API. More details about the
+Node is a top-level resource in the Kubernetes REST API. More details about the
API object can be found at: [Node API
object](/docs/api-reference/v1/definitions/#_v1_node).
diff --git a/docs/admin/out-of-resource.md b/docs/admin/out-of-resource.md
index 8af7114ed6..30b8744624 100644
--- a/docs/admin/out-of-resource.md
+++ b/docs/admin/out-of-resource.md
@@ -3,7 +3,7 @@ assignees:
- derekwaynecarr
- vishh
- timstclair
-
+title: Configuring Out Of Resource Handling
---
* TOC
@@ -330,7 +330,7 @@ for eviction. Instead `DaemonSet` should ideally launch `Guaranteed` pods.
`kubelet` has been freeing up disk space on demand to keep the node stable.
As disk based eviction matures, the following `kubelet` flags will be marked for deprecation
-in favor of the simpler configuation supported around eviction.
+in favor of the simpler configuration supported around eviction.
| Existing Flag | New Flag |
| ------------- | -------- |
@@ -349,7 +349,7 @@ in favor of the simpler configuation supported around eviction.
The `kubelet` currently polls `cAdvisor` to collect memory usage stats at a regular interval. If memory usage
increases within that window rapidly, the `kubelet` may not observe `MemoryPressure` fast enough, and the `OOMKiller`
will still be invoked. We intend to integrate with the `memcg` notification API in a future release to reduce this
-latency, and instead have the kernel tell us when a threshold has been crossed immmediately.
+latency, and instead have the kernel tell us when a threshold has been crossed immediately.
If you are not trying to achieve extreme utilization, but a sensible measure of overcommit, a viable workaround for
this issue is to set eviction thresholds at approximately 75% capacity. This increases the ability of this feature
diff --git a/docs/admin/ovs-networking.md b/docs/admin/ovs-networking.md
index 7a8f89506c..8ae71cc186 100644
--- a/docs/admin/ovs-networking.md
+++ b/docs/admin/ovs-networking.md
@@ -1,8 +1,7 @@
---
assignees:
-- lavalamp
- thockin
-
+title: Kubernetes OpenVSwitch GRE/VxLAN networking
---
This document describes how OpenVSwitch is used to setup networking between pods across nodes.
diff --git a/docs/admin/rescheduler.md b/docs/admin/rescheduler.md
index f32c185c81..1d81ac63f2 100644
--- a/docs/admin/rescheduler.md
+++ b/docs/admin/rescheduler.md
@@ -3,15 +3,13 @@ assignees:
- davidopp
- filipg
- piosz
-
+title: Guaranteed Scheduling For Critical Add-On Pods
---
* TOC
{:toc}
-# "Guaranteed" scheduling of critical add-on pods
-
-## Critical add-ons
+## Overview
In addition to Kubernetes core components like api-server, scheduler, controller-manager running on a master machine
there are a number of add-ons which, for various reasons, must run on a regular cluster node (rather than the Kubernetes master).
@@ -30,29 +28,30 @@ given the pods that are already running in the cluster
the rescheduler tries to free up space for the add-on by evicting some pods; then the scheduler will schedule the add-on pod.
To avoid situation when another pod is scheduled into the space prepared for the critical add-on,
-the chosen node gets a temporary taint “CriticalAddonsOnly” before the eviction(s)
+the chosen node gets a temporary taint "CriticalAddonsOnly" before the eviction(s)
(see [more details](https://github.com/kubernetes/kubernetes/blob/master/docs/design/taint-toleration-dedicated.md)).
Each critical add-on has to tolerate it,
the other pods shouldn't tolerate the taint. The tain is removed once the add-on is successfully scheduled.
*Warning:* currently there is no guarantee which node is chosen and which pods are being killed
-in order to schedule crical pod, so if rescheduler is enabled you pods might be occasionally
+in order to schedule critical pods, so if rescheduler is enabled you pods might be occasionally
killed for this purpose.
## Config
Rescheduler doesn't have any user facing configuration (component config) or API.
It's enabled by default. It can be disabled:
+
* during cluster setup by setting `ENABLE_RESCHEDULER` flag to `false`
* on running cluster by deleting its manifest from master node
(default path `/etc/kubernetes/manifests/rescheduler.manifest`)
### Marking add-on as critical
-To be critical an add-on has to run in `kube-system` namespace (cofigurable via flag)
+To be critical an add-on has to run in `kube-system` namespace (configurable via flag)
and have the following annotations specified:
+
* `scheduler.alpha.kubernetes.io/critical-pod` set to empty string
* `scheduler.alpha.kubernetes.io/tolerations` set to `[{"key":"CriticalAddonsOnly", "operator":"Exists"}]`
The first one marks a pod a critical. The second one is required by Rescheduler algorithm.
-
diff --git a/docs/admin/resourcequota/index.md b/docs/admin/resourcequota/index.md
index ff76942702..24d7a09246 100644
--- a/docs/admin/resourcequota/index.md
+++ b/docs/admin/resourcequota/index.md
@@ -1,7 +1,7 @@
---
assignees:
- derekwaynecarr
-
+title: Resource Quotas
---
When several users or teams share a cluster with a fixed number of nodes,
@@ -52,8 +52,7 @@ Resource Quota is enforced in a particular namespace when there is a
## Compute Resource Quota
-You can limit the total sum of [compute resources](/docs/user-guide/compute-resources) and [storage resources](/docs/user-guide/persistent-volumes)
-that can be requested in a given namespace.
+You can limit the total sum of [compute resources](/docs/user-guide/compute-resources) that can be requested in a given namespace.
The following resource types are supported:
@@ -65,7 +64,25 @@ The following resource types are supported:
| `memory` | Across all pods in a non-terminal state, the sum of memory requests cannot exceed this value. |
| `requests.cpu` | Across all pods in a non-terminal state, the sum of CPU requests cannot exceed this value. |
| `requests.memory` | Across all pods in a non-terminal state, the sum of memory requests cannot exceed this value. |
+
+## Storage Resource Quota
+
+You can limit the total sum of [storage resources](/docs/user-guide/persistent-volumes) that can be requested in a given namespace.
+
+In addition, you can limit consumption of storage resources based on associated storage-class.
+
+| Resource Name | Description |
+| --------------------- | ----------------------------------------------------------- |
| `requests.storage` | Across all persistent volume claims, the sum of storage requests cannot exceed this value. |
+| `persistentvolumeclaims` | The total number of [persistent volume claims](/docs/user-guide/persistent-volumes/#persistentvolumeclaims) that can exist in the namespace. |
+| `.storageclass.storage.k8s.io/requests.storage` | Across all persistent volume claims associated with the storage-class-name, the sum of storage requests cannot exceed this value. |
+| `.storageclass.storage.k8s.io/persistentvolumeclaims` | Across all persistent volume claims associated with the storage-class-name, the total number of [persistent volume claims](/docs/user-guide/persistent-volumes/#persistentvolumeclaims) that can exist in the namespace. |
+
+For example, if an operator wants to quota storage with `gold` storage class separate from `bronze` storage class, the operator can
+define a quota as follows:
+
+* `gold.storageclass.storage.k8s.io/requests.storage: 500Gi`
+* `bronze.storageclass.storage.k8s.io/requests.storage: 100Gi`
## Object Count Quota
@@ -125,7 +142,7 @@ The quota can be configured to quota either value.
If the quota has a value specified for `requests.cpu` or `requests.memory`, then it requires that every incoming
container makes an explicit request for those resources. If the quota has a value specified for `limits.cpu` or `limits.memory`,
-then it requires that every incoming container specifies an explict limit for those resources.
+then it requires that every incoming container specifies an explicit limit for those resources.
## Viewing and Setting Quotas
diff --git a/docs/admin/resourcequota/walkthrough.md b/docs/admin/resourcequota/walkthrough.md
index 7422f2abcf..1120e7550d 100644
--- a/docs/admin/resourcequota/walkthrough.md
+++ b/docs/admin/resourcequota/walkthrough.md
@@ -2,7 +2,7 @@
assignees:
- derekwaynecarr
- janetkuo
-
+title: Applying Resource Quotas and Limits
---
This example demonstrates a typical setup to control for resource usage in a namespace.
@@ -232,7 +232,7 @@ services.loadbalancers 0 2
services.nodeports 0 0
```
-As you can see, the pod that was created is consuming explict amounts of compute resources, and the usage is being
+As you can see, the pod that was created is consuming explicit amounts of compute resources, and the usage is being
tracked by Kubernetes properly.
## Step 5: Advanced quota scopes
diff --git a/docs/admin/salt.md b/docs/admin/salt.md
index 5d82b54d39..d2479ffff2 100644
--- a/docs/admin/salt.md
+++ b/docs/admin/salt.md
@@ -1,8 +1,7 @@
---
assignees:
- davidopp
-- lavalamp
-
+title: Configuring Kubernetes with Salt
---
The Kubernetes cluster can be configured using Salt.
diff --git a/docs/admin/service-accounts-admin.md b/docs/admin/service-accounts-admin.md
index 810f4d7515..4a31fbeced 100644
--- a/docs/admin/service-accounts-admin.md
+++ b/docs/admin/service-accounts-admin.md
@@ -4,7 +4,7 @@ assignees:
- davidopp
- lavalamp
- liggitt
-
+title: Managing Service Accounts
---
*This is a Cluster Administrator guide to service accounts. It assumes knowledge of
diff --git a/docs/admin/static-pods.md b/docs/admin/static-pods.md
index 531494fb04..4766002917 100644
--- a/docs/admin/static-pods.md
+++ b/docs/admin/static-pods.md
@@ -1,7 +1,7 @@
---
assignees:
- jsafrane
-
+title: Static Pods
---
**If you are running clustered Kubernetes and are using static pods to run a pod on every node, you should probably be using a [DaemonSet](/docs/admin/daemons/)!**
@@ -16,7 +16,7 @@ Static pod can be created in two ways: either by using configuration file(s) or
### Configuration files
-The configuration files are just standard pod definition in json or yaml format in specific directory. Use `kubelet --config=` to start kubelet daemon, which periodically scans the directory and creates/deletes static pods as yaml/json files appear/disappear there.
+The configuration files are just standard pod definition in json or yaml format in specific directory. Use `kubelet --pod-manifest-path=` to start kubelet daemon, which periodically scans the directory and creates/deletes static pods as yaml/json files appear/disappear there.
For example, this is how to start a simple web server as a static pod:
@@ -48,10 +48,10 @@ For example, this is how to start a simple web server as a static pod:
EOF
```
-2. Configure your kubelet daemon on the node to use this directory by running it with `--config=/etc/kubelet.d/` argument. On Fedora edit `/etc/kubernetes/kubelet` to include this line:
+2. Configure your kubelet daemon on the node to use this directory by running it with `--pod-manifest-path=/etc/kubelet.d/` argument. On Fedora edit `/etc/kubernetes/kubelet` to include this line:
```conf
- KUBELET_ARGS="--cluster-dns=10.254.0.10 --cluster-domain=kube.local --config=/etc/kubelet.d/"
+ KUBELET_ARGS="--cluster-dns=10.254.0.10 --cluster-domain=kube.local --pod-manifest-path=/etc/kubelet.d/"
```
Instructions for other distributions or Kubernetes installations may vary.
@@ -64,11 +64,11 @@ For example, this is how to start a simple web server as a static pod:
## Pods created via HTTP
-Kubelet periodically downloads a file specified by `--manifest-url=` argument and interprets it as a json/yaml file with a pod definition. It works the same as `--config=`, i.e. it's reloaded every now and then and changes are applied to running static pods (see below).
+Kubelet periodically downloads a file specified by `--manifest-url=` argument and interprets it as a json/yaml file with a pod definition. It works the same as `--pod-manifest-path=`, i.e. it's reloaded every now and then and changes are applied to running static pods (see below).
## Behavior of static pods
-When kubelet starts, it automatically starts all pods defined in directory specified in `--config=` or `--manifest-url=` arguments, i.e. our static-web. (It may take some time to pull nginx image, be patient…):
+When kubelet starts, it automatically starts all pods defined in directory specified in `--pod-manifest-path=` or `--manifest-url=` arguments, i.e. our static-web. (It may take some time to pull nginx image, be patient…):
```shell
[joe@my-node1 ~] $ docker ps
diff --git a/docs/admin/sysctls.md b/docs/admin/sysctls.md
index dc62b8c3d1..aa75c4df2a 100644
--- a/docs/admin/sysctls.md
+++ b/docs/admin/sysctls.md
@@ -1,7 +1,7 @@
---
assignees:
- sttts
-
+title: Using Sysctls in a Kubernetes Cluster
---
* TOC
@@ -9,7 +9,7 @@ assignees:
This document describes how sysctls are used within a Kubernetes cluster.
-## What is a _Sysctl_?
+## What is a Sysctl?
In Linux, the sysctl interface allows an administrator to modify kernel
parameters at runtime. Parameters are available via the `/proc/sys/` virtual
diff --git a/docs/api-reference/README.md b/docs/api-reference/README.md
deleted file mode 100644
index 905b267947..0000000000
--- a/docs/api-reference/README.md
+++ /dev/null
@@ -1,20 +0,0 @@
----
----
-
-# API Reference
-
-Use the following reference docs to understand the kubernetes REST API for various API group versions:
-
-* v1: [operations](/docs/api-reference/v1/operations.html), [model definitions](/docs/api-reference/v1/definitions.html)
-* extensions/v1beta1: [operations](/docs/api-reference/extensions/v1beta1/operations.html), [model definitions](/docs/api-reference/extensions/v1beta1/definitions.html)
-* batch/v1: [operations](/docs/api-reference/batch/v1/operations.html), [model definitions](/docs/api-reference/batch/v1/definitions.html)
-* autoscaling/v1: [operations](/docs/api-reference/autoscaling/v1/operations.html), [model definitions](/docs/api-reference/autoscaling/v1/definitions.html)
-
-
-
-
-
-
-
-[]()
-
diff --git a/docs/api-reference/apps/v1alpha1/definitions.html b/docs/api-reference/apps/v1beta1/definitions.html
similarity index 89%
rename from docs/api-reference/apps/v1alpha1/definitions.html
rename to docs/api-reference/apps/v1beta1/definitions.html
index 0dfdf27a56..77ef25e10c 100755
--- a/docs/api-reference/apps/v1alpha1/definitions.html
+++ b/docs/api-reference/apps/v1beta1/definitions.html
@@ -18,10 +18,10 @@
@@ -31,6 +31,85 @@
Definitions
+
v1.PhotonPersistentDiskVolumeSource
+
+
Represents a Photon Controller persistent disk resource.
+
+
+
+
+
+
+
+
+
+
+
+| Name |
+Description |
+Required |
+Schema |
+Default |
+
+
+
+
+pdID |
+ID that identifies Photon Controller persistent disk |
+true |
+string |
+ |
+
+
+fsType |
+Filesystem type to mount. Must be a filesystem type supported by the host operating system. Ex. "ext4", "xfs", "ntfs". Implicitly inferred to be "ext4" if unspecified. |
+false |
+string |
+ |
+
+
+
+
+
+
+
versioned.Event
+
+
+
+
+
+
+
+
+
+
+| Name |
+Description |
+Required |
+Schema |
+Default |
+
+
+
+
+type |
+ |
+true |
+string |
+ |
+
+
+object |
+ |
+true |
+string |
+ |
+
+
+
+
+
+
v1.Preconditions
Preconditions must be fulfilled before an operation (update, delete, etc.) is carried out.
@@ -41,7 +120,7 @@
-
+
@@ -75,7 +154,7 @@
-
+
@@ -130,7 +209,7 @@
-
+
@@ -171,7 +250,7 @@
-
+
@@ -226,7 +305,7 @@
-
+
@@ -240,21 +319,21 @@
server |
-Server is the hostname or IP address of the NFS server. More info: http://releases.k8s.io/release-1.4/docs/user-guide/volumes.md#nfs |
+Server is the hostname or IP address of the NFS server. More info: http://kubernetes.io/docs/user-guide/volumes#nfs |
true |
string |
|
path |
-Path that is exported by the NFS server. More info: http://releases.k8s.io/release-1.4/docs/user-guide/volumes.md#nfs |
+Path that is exported by the NFS server. More info: http://kubernetes.io/docs/user-guide/volumes#nfs |
true |
string |
|
readOnly |
-ReadOnly here will force the NFS export to be mounted with read-only permissions. Defaults to false. More info: http://releases.k8s.io/release-1.4/docs/user-guide/volumes.md#nfs |
+ReadOnly here will force the NFS export to be mounted with read-only permissions. Defaults to false. More info: http://kubernetes.io/docs/user-guide/volumes#nfs |
false |
boolean |
false |
@@ -274,7 +353,7 @@
-
+
@@ -288,7 +367,7 @@
accessModes |
-AccessModes contains the desired access modes the volume should have. More info: http://releases.k8s.io/release-1.4/docs/user-guide/persistent-volumes.md#access-modes-1 |
+AccessModes contains the desired access modes the volume should have. More info: http://kubernetes.io/docs/user-guide/persistent-volumes#access-modes-1 |
false |
v1.PersistentVolumeAccessMode array |
|
@@ -302,7 +381,7 @@
resources |
-Resources represents the minimum resources the volume should have. More info: http://releases.k8s.io/release-1.4/docs/user-guide/persistent-volumes.md#resources |
+Resources represents the minimum resources the volume should have. More info: http://kubernetes.io/docs/user-guide/persistent-volumes#resources |
false |
v1.ResourceRequirements |
|
@@ -329,7 +408,7 @@
-
+
@@ -343,7 +422,7 @@
monitors |
-Required: Monitors is a collection of Ceph monitors More info: http://releases.k8s.io/release-1.4/examples/volumes/cephfs/README.md#how-to-use-it |
+Required: Monitors is a collection of Ceph monitors More info: http://releases.k8s.io/HEAD/examples/volumes/cephfs/README.md#how-to-use-it |
true |
string array |
|
@@ -357,28 +436,28 @@
user |
-Optional: User is the rados user name, default is admin More info: http://releases.k8s.io/release-1.4/examples/volumes/cephfs/README.md#how-to-use-it |
+Optional: User is the rados user name, default is admin More info: http://releases.k8s.io/HEAD/examples/volumes/cephfs/README.md#how-to-use-it |
false |
string |
|
secretFile |
-Optional: SecretFile is the path to key ring for User, default is /etc/ceph/user.secret More info: http://releases.k8s.io/release-1.4/examples/volumes/cephfs/README.md#how-to-use-it |
+Optional: SecretFile is the path to key ring for User, default is /etc/ceph/user.secret More info: http://releases.k8s.io/HEAD/examples/volumes/cephfs/README.md#how-to-use-it |
false |
string |
|
secretRef |
-Optional: SecretRef is reference to the authentication secret for User, default is empty. More info: http://releases.k8s.io/release-1.4/examples/volumes/cephfs/README.md#how-to-use-it |
+Optional: SecretRef is reference to the authentication secret for User, default is empty. More info: http://releases.k8s.io/HEAD/examples/volumes/cephfs/README.md#how-to-use-it |
false |
v1.LocalObjectReference |
|
readOnly |
-Optional: Defaults to false (read/write). ReadOnly here will force the ReadOnly setting in VolumeMounts. More info: http://releases.k8s.io/release-1.4/examples/volumes/cephfs/README.md#how-to-use-it |
+Optional: Defaults to false (read/write). ReadOnly here will force the ReadOnly setting in VolumeMounts. More info: http://releases.k8s.io/HEAD/examples/volumes/cephfs/README.md#how-to-use-it |
false |
boolean |
false |
@@ -398,7 +477,7 @@
-
+
@@ -439,7 +518,7 @@
-
+
@@ -494,7 +573,7 @@
-
+
@@ -535,7 +614,7 @@
-
+
@@ -575,6 +654,47 @@ Examples:
- 
- 
- 
+ 
+ 
+ 
+ 
+ 
diff --git a/case-studies/pearson.html b/case-studies/pearson.html
index bf871789b9..50f16ce7ae 100644
--- a/case-studies/pearson.html
+++ b/case-studies/pearson.html
@@ -13,13 +13,13 @@ title: Pearson Case Study
