merge master to 1.10, with fixes (#7682)

2018-03-08 14:03:55 -05:00
parent bb8c59a640
commit 44b51d6056
548 changed files with 11634 additions and 318622 deletions
@@ -16,7 +16,7 @@ manually through `easyrsa`, `openssl` or `cfssl`.

 1.  Download, unpack, and initialize the patched version of easyrsa3.

-        curl -L -O https://storage.googleapis.com/kubernetes-release/easy-rsa/easy-rsa.tar.gz
+        curl -LO https://storage.googleapis.com/kubernetes-release/easy-rsa/easy-rsa.tar.gz
        tar xzf easy-rsa.tar.gz
        cd easy-rsa-master/easyrsa3
        ./easyrsa init-pki
@@ -1,5 +1,5 @@
 ---
-approvers:
+reviewers:
 - davidopp
 - lavalamp
 title: Cluster Administration Overview
@@ -21,10 +21,10 @@ Before choosing a guide, here are some considerations:
 - **If you are designing for high-availability**, learn about configuring [clusters in multiple zones](/docs/concepts/cluster-administration/federation/).
 - Will you be using **a hosted Kubernetes cluster**, such as [Google Kubernetes Engine](https://cloud.google.com/kubernetes-engine/), or **hosting your own cluster**?
 - Will your cluster be **on-premises**, or **in the cloud (IaaS)**? Kubernetes does not directly support hybrid clusters. Instead, you can set up multiple clusters.
- - **If you are configuring Kubernetes on-premises**, consider which [networking model](/docs/concepts/cluster-administration/networking/) fits best. One option for custom networking is [*OpenVSwitch GRE/VxLAN networking*](/docs/admin/ovs-networking/), which uses OpenVSwitch to set up networking between pods across Kubernetes nodes.
+ - **If you are configuring Kubernetes on-premises**, consider which [networking model](/docs/concepts/cluster-administration/networking/) fits best.
 - Will you be running Kubernetes on **"bare metal" hardware** or on **virtual machines (VMs)**?
 - Do you **just want to run a cluster**, or do you expect to do **active development of Kubernetes project code**? If the
-   latter, choose a actively-developed distro. Some distros only use binary releases, but
+   latter, choose an actively-developed distro. Some distros only use binary releases, but
   offer a greater variety of choices.
 - Familiarize yourself with the [components](/docs/admin/cluster-components/) needed to run a cluster.

@@ -1,5 +1,5 @@
 ---
-approvers:
+reviewers:
 title: Device Plugins
 description: Use the Kubernetes device plugin framework to implement plugins for GPUs, NICs, FPGAs, InfiniBand, and similar resources that require vendor-specific setup.
 ---
@@ -138,6 +138,7 @@ For examples of device plugin implementations, see:
 * The official [NVIDIA GPU device plugin](https://github.com/NVIDIA/k8s-device-plugin)
    * it requires using [nvidia-docker 2.0](https://github.com/NVIDIA/nvidia-docker) which allows you to run GPU enabled docker containers
 * The [NVIDIA GPU device plugin for COS base OS](https://github.com/GoogleCloudPlatform/container-engine-accelerators/tree/master/cmd/nvidia_gpu).
+* The [RDMA device plugin](https://github.com/hustcat/k8s-rdma-device-plugin)

 {% endcapture %}

@@ -34,7 +34,6 @@ why you might want multiple clusters are:
 * Fault isolation: It might be better to have multiple small clusters rather
  than a single large  cluster for fault isolation (for example: multiple
  clusters in different availability zones of a cloud provider).
-  See [Multi cluster guide](/docs/concepts/cluster-administration/federation/) for details.
 * Scalability: There are scalability limits to a single kubernetes cluster (this
  should not be the case for most users. For more details:
  [Kubernetes Scaling and Performance Goals](https://git.k8s.io/community/sig-scalability/goals.md)).
@@ -57,18 +56,14 @@ some caveats:
  on the side of safety and avoiding multi-cluster outage.
 * Maturity: The federation project is relatively new and is not very mature.
  Not all resources are available and many are still alpha. [Issue
-  38893](https://github.com/kubernetes/kubernetes/issues/38893) enumerates
+  88](https://github.com/kubernetes/federation/issues/88) enumerates
  known issues with the system that the team is busy solving.

 ### Hybrid cloud capabilities

 Federations of Kubernetes Clusters can include clusters running in
 different cloud providers (e.g. Google Cloud, AWS), and on-premises
-(e.g. on OpenStack). Simply create all of the clusters that you
-require, in the appropriate cloud providers and/or locations, and
-register each cluster's API endpoint and credentials with your
-Federation API Server (See the
-[federation admin guide](/docs/admin/federation/) for details).
+(e.g. on OpenStack). [Kubefed](https://kubernetes.io/docs/tasks/federation/set-up-cluster-federation-kubefed/) is the recommended way to deploy federated clusters.

 Thereafter, your [API resources](#api-resources) can span different clusters
 and cloud providers.
@@ -132,8 +127,9 @@ We suggest that all the VMs in a Kubernetes cluster should be in the same availa
  - when the Kubernetes developers are designing the system (e.g. making assumptions about latency, bandwidth, or
    correlated failures) they are assuming all the machines are in a single data center, or otherwise closely connected.

-It is okay to have multiple clusters per availability zone, though on balance we think fewer is better.
-Reasons to prefer fewer clusters are:
+It is recommended to run fewer clusters with more VMs per availability zone; but it is possible to run multiple clusters per availability zones.
+
+Reasons to prefer fewer clusters per availability zone are:

  - improved bin packing of Pods in some cases with more nodes in one cluster (less resource fragmentation).
  - reduced operational overhead (though the advantage is diminished as ops tooling and processes mature).
@@ -176,6 +172,7 @@ clusters up to 5000 nodes. See [Building Large Clusters](/docs/admin/cluster-lar
  proposal](https://github.com/kubernetes/community/blob/{{page.githubbranch}}/contributors/design-proposals/multicluster/federation.md).
 * See this [setup guide](/docs/tutorials/federation/set-up-cluster-federation-kubefed/) for cluster federation.
 * See this [Kubecon2016 talk on federation](https://www.youtube.com/watch?v=pq9lbkmxpS8)
+* See this [Kubecon2017 Europe update on federation](https://www.youtube.com/watch?v=kwOvOLnFYck)
 {% endcapture %}

 {% include templates/concept.md %}
@@ -1,5 +1,5 @@
 ---
-approvers:
+reviewers:
 - mikedanese
 title: Configuring kubelet Garbage Collection
 ---
@@ -1,5 +1,5 @@
 ---
-approvers:
+reviewers:
 - crassirostris
 - piosz
 title: Logging Architecture
@@ -1,5 +1,5 @@
 ---
-approvers:
+reviewers:
 - bgrant0607
 - janetkuo
 - mikedanese
@@ -8,9 +8,6 @@ title: Managing Resources

 You've deployed your application and exposed it via a service. Now what? Kubernetes provides a number of tools to help you manage your application deployment, including scaling and updating. Among the features that we will discuss in more depth are [configuration files](/docs/concepts/configuration/overview/) and [labels](/docs/concepts/overview/working-with-objects/labels/).

-You can find all the files for this example [in our docs
-repo here](https://github.com/kubernetes/website/tree/{{page.docsbranch}}/docs/user-guide/).
-
 * TOC
 {:toc}

@@ -18,12 +15,12 @@ repo here](https://github.com/kubernetes/website/tree/{{page.docsbranch}}/docs/u

 Many applications require multiple resources to be created, such as a Deployment and a Service. Management of multiple resources can be simplified by grouping them together in the same file (separated by `---` in YAML). For example:

-{% include code.html language="yaml" file="nginx-app.yaml" ghlink="/docs/user-guide/nginx-app.yaml" %}
+{% include code.html language="yaml" file="nginx-app.yaml" ghlink="/docs/concepts/cluster-administration/nginx-app.yaml" %}

 Multiple resources can be created the same way as a single resource:

 ```shell
-$ kubectl create -f docs/user-guide/nginx-app.yaml
+$ kubectl create -f https://k8s.io/docs/concepts/cluster-administration/nginx-app.yaml
 service "my-nginx-svc" created
 deployment "my-nginx" created
 ```
@@ -33,13 +30,13 @@ The resources will be created in the order they appear in the file. Therefore, i
 `kubectl create` also accepts multiple `-f` arguments:

 ```shell
-$ kubectl create -f docs/user-guide/nginx/nginx-svc.yaml -f docs/user-guide/nginx/nginx-deployment.yaml
+$ kubectl create -f https://k8s.io/docs/concepts/cluster-administration/nginx/nginx-svc.yaml -f https://k8s.io/docs/concepts/cluster-administration/nginx/nginx-deployment.yaml
 ```

 And a directory can be specified rather than or in addition to individual files:

 ```shell
-$ kubectl create -f docs/user-guide/nginx/
+$ kubectl create -f https://k8s.io/docs/concepts/cluster-administration/nginx/
 ```

 `kubectl` will read any files with suffixes `.yaml`, `.yml`, or `.json`.
@@ -49,7 +46,7 @@ It is a recommended practice to put resources related to the same microservice o
 A URL can also be specified as a configuration source, which is handy for deploying directly from configuration files checked into github:

 ```shell
-$ kubectl create -f https://raw.githubusercontent.com/kubernetes/kubernetes/master/docs/user-guide/nginx-deployment.yaml
+$ kubectl create -f https://raw.githubusercontent.com/kubernetes/website/master/docs/concepts/cluster-administration/nginx-deployment.yaml
 deployment "nginx-deployment" created
 ```

@@ -58,7 +55,7 @@ deployment "nginx-deployment" created
 Resource creation isn't the only operation that `kubectl` can perform in bulk. It can also extract resource names from configuration files in order to perform other operations, in particular to delete the same resources you created:

 ```shell
-$ kubectl delete -f docs/user-guide/nginx/
+$ kubectl delete -f https://k8s.io/docs/concepts/cluster-administration/nginx/
 deployment "my-nginx" deleted
 service "my-nginx-svc" deleted
 ```
@@ -80,12 +77,12 @@ service "my-nginx-svc" deleted
 Because `kubectl` outputs resource names in the same syntax it accepts, it's easy to chain operations using `$()` or `xargs`:

 ```shell
-$ kubectl get $(kubectl create -f docs/user-guide/nginx/ -o name | grep service)
+$ kubectl get $(kubectl create -f docs/concepts/cluster-administration/nginx/ -o name | grep service)
 NAME           CLUSTER-IP   EXTERNAL-IP   PORT(S)      AGE
 my-nginx-svc   10.0.0.208   <pending>     80/TCP       0s
 ```

-With the above commands, we first create resources under docs/user-guide/nginx/ and print the resources created with `-o name` output format
+With the above commands, we first create resources under `docs/concepts/cluster-administration/nginx/` and print the resources created with `-o name` output format
 (print each resource as resource/name). Then we `grep` only the "service", and then print it with `kubectl get`.

 If you happen to organize your resources across several subdirectories within a particular directory, you can recursively perform the operations on the subdirectories also, by specifying `--recursive` or `-R` alongside the `--filename,-f` flag.
@@ -131,7 +128,7 @@ deployment "my-deployment" created
 persistentvolumeclaim "my-pvc" created
 ```

-If you're interested in learning more about `kubectl`, go ahead and read [kubectl Overview](/docs/user-guide/kubectl-overview).
+If you're interested in learning more about `kubectl`, go ahead and read [kubectl Overview](/docs/reference/kubectl/overview/).

 ## Using labels effectively

@@ -256,7 +253,7 @@ my-nginx-2035384211-u3t6x   1/1       Running   0          23m       fe

 This outputs all "app=nginx" pods, with an additional label column of pods' tier (specified with `-L` or `--label-columns`).

-For more information, please see [labels](/docs/concepts/overview/working-with-objects/labels/) and [kubectl label](/docs/user-guide/kubectl/{{page.version}}/#label) document.
+For more information, please see [labels](/docs/concepts/overview/working-with-objects/labels/) and [kubectl label](/docs/reference/generated/kubectl/kubectl-commands/#label).

 ## Updating annotations

@@ -273,7 +270,7 @@ metadata:
 ...
 ```

-For more information, please see [annotations](/docs/concepts/overview/working-with-objects/annotations/) and [kubectl annotate](/docs/user-guide/kubectl/{{page.version}}/#annotate) document.
+For more information, please see [annotations](/docs/concepts/overview/working-with-objects/annotations/) and [kubectl annotate](/docs/reference/generated/kubectl/kubectl-commands/#annotate) document.

 ## Scaling your application

@@ -301,7 +298,7 @@ deployment "my-nginx" autoscaled

 Now your nginx replicas will be scaled up and down as needed, automatically.

-For more information, please see [kubectl scale](/docs/user-guide/kubectl/{{page.version}}/#scale), [kubectl autoscale](/docs/user-guide/kubectl/{{page.version}}/#autoscale) and [horizontal pod autoscaler](/docs/tasks/run-application/horizontal-pod-autoscale/) document.
+For more information, please see [kubectl scale](/docs/reference/generated/kubectl/kubectl-commands/#scale), [kubectl autoscale](/docs/reference/generated/kubectl/kubectl-commands/#autoscale) and [horizontal pod autoscaler](/docs/tasks/run-application/horizontal-pod-autoscale/) document.


 ## In-place updates of resources
@@ -312,12 +309,12 @@ Sometimes it's necessary to make narrow, non-disruptive updates to resources you

 It is suggested to maintain a set of configuration files in source control (see [configuration as code](http://martinfowler.com/bliki/InfrastructureAsCode.html)),
 so that they can be maintained and versioned along with the code for the resources they configure.
-Then, you can use [`kubectl apply`](/docs/user-guide/kubectl/{{page.version}}/#apply) to push your configuration changes to the cluster.
+Then, you can use [`kubectl apply`](/docs/reference/generated/kubectl/kubectl-commands/#apply) to push your configuration changes to the cluster.

 This command will compare the version of the configuration that you're pushing with the previous version and apply the changes you've made, without overwriting any automated changes to properties you haven't specified.

 ```shell
-$ kubectl apply -f docs/user-guide/nginx/nginx-deployment.yaml
+$ kubectl apply -f docs/concepts/cluster-administration/nginx/nginx-deployment.yaml
 deployment "my-nginx" configured
 ```

@@ -328,6 +325,7 @@ Currently, resources are created without this annotation, so the first invocatio
 All subsequent calls to `kubectl apply`, and other commands that modify the configuration, such as `kubectl replace` and `kubectl edit`, will update the annotation, allowing subsequent calls to `kubectl apply` to detect and perform deletions using a three-way diff.

 **Note:** To use apply, always create resource initially with either `kubectl apply` or `kubectl create --save-config`.
+{: .note}

 ### kubectl edit

@@ -350,7 +348,7 @@ $ rm /tmp/nginx.yaml

 This allows you to do more significant changes more easily. Note that you can specify the editor with your `EDITOR` or `KUBE_EDITOR` environment variables.

-For more information, please see [kubectl edit](/docs/user-guide/kubectl/{{page.version}}/#edit) document.
+For more information, please see [kubectl edit](/docs/reference/generated/kubectl/kubectl-commands/#edit) document.

 ### kubectl patch

@@ -358,14 +356,14 @@ You can use `kubectl patch` to update API objects in place. This command support
 JSON merge patch, and strategic merge patch. See
 [Update API Objects in Place Using kubectl patch](/docs/tasks/run-application/update-api-object-kubectl-patch/)
 and
-[kubectl patch](/docs/user-guide/kubectl/{{page.version}}/#patch).
+[kubectl patch](/docs/reference/generated/kubectl/kubectl-commands/#patch).

 ## Disruptive updates

 In some cases, you may need to update resource fields that cannot be updated once initialized, or you may just want to make a recursive change immediately, such as to fix broken pods created by a Deployment. To change such fields, use `replace --force`, which deletes and re-creates the resource. In this case, you can simply modify your original configuration file:

 ```shell
-$ kubectl replace -f docs/user-guide/nginx/nginx-deployment.yaml --force
+$ kubectl replace -f docs/concepts/cluster-administration/nginx/nginx-deployment.yaml --force
 deployment "my-nginx" deleted
 deployment "my-nginx" replaced
 ```
@@ -1,5 +1,5 @@
 ---
-approvers:
+reviewers:
 - dcbw
 - freehan
 - thockin
@@ -18,16 +18,16 @@ Network plugins in Kubernetes come in a few flavors:

 ## Installation

-The kubelet has a single default network plugin, and a default network common to the entire cluster. It probes for plugins when it starts up, remembers what it found, and executes the selected plugin at appropriate times in the pod lifecycle (this is only true for docker, as rkt manages its own CNI plugins). There are two Kubelet command line parameters to keep in mind when using plugins:
+The kubelet has a single default network plugin, and a default network common to the entire cluster. It probes for plugins when it starts up, remembers what it found, and executes the selected plugin at appropriate times in the pod lifecycle (this is only true for Docker, as rkt manages its own CNI plugins). There are two Kubelet command line parameters to keep in mind when using plugins:

-* `network-plugin-dir`: Kubelet probes this directory for plugins on startup
-* `network-plugin`: The network plugin to use from `network-plugin-dir`.  It must match the name reported by a plugin probed from the plugin directory.  For CNI plugins, this is simply "cni".
+* `cni-bin-dir`: Kubelet probes this directory for plugins on startup
+* `network-plugin`: The network plugin to use from `cni-bin-dir`.  It must match the name reported by a plugin probed from the plugin directory.  For CNI plugins, this is simply "cni".

 ## Network Plugin Requirements

 Besides providing the [`NetworkPlugin` interface](https://github.com/kubernetes/kubernetes/tree/{{page.fullversion}}/pkg/kubelet/network/plugins.go) to configure and clean up pod networking, the plugin may also need specific support for kube-proxy.  The iptables proxy obviously depends on iptables, and the plugin may need to ensure that container traffic is made available to iptables.  For example, if the plugin connects containers to a Linux bridge, the plugin must set the `net/bridge/bridge-nf-call-iptables` sysctl to `1` to ensure that the iptables proxy functions correctly.  If the plugin does not use a Linux bridge (but instead something like Open vSwitch or some other mechanism) it should ensure container traffic is appropriately routed for the proxy.

-By default if no kubelet network plugin is specified, the `noop` plugin is used, which sets `net/bridge/bridge-nf-call-iptables=1` to ensure simple configurations (like docker with a bridge) work correctly with the iptables proxy.
+By default if no kubelet network plugin is specified, the `noop` plugin is used, which sets `net/bridge/bridge-nf-call-iptables=1` to ensure simple configurations (like Docker with a bridge) work correctly with the iptables proxy.

 ### CNI

@@ -47,7 +47,7 @@ Kubenet creates a Linux bridge named `cbr0` and creates a veth pair for each pod

 The plugin requires a few things:

-* The standard CNI `bridge`, `lo` and `host-local` plugins are required, at minimum version 0.2.0. Kubenet will first search for them in `/opt/cni/bin`. Specify `network-plugin-dir` to supply additional search path. The first found match will take effect.
+* The standard CNI `bridge`, `lo` and `host-local` plugins are required, at minimum version 0.2.0. Kubenet will first search for them in `/opt/cni/bin`. Specify `cni-bin-dir` to supply additional search path. The first found match will take effect.
 * Kubelet must be run with the `--network-plugin=kubenet` argument to enable the plugin
 * Kubelet should also be run with the `--non-masquerade-cidr=<clusterCidr>` argument to ensure traffic to IPs outside this range will use IP masquerade.
 * The node must be assigned an IP subnet through either the `--pod-cidr` kubelet command-line option or the `--allocate-node-cidrs=true --cluster-cidr=<cidr>` controller-manager command-line options.
@@ -69,5 +69,5 @@ This option is provided to the network-plugin; currently **only kubenet supports
 ## Usage Summary

 * `--network-plugin=cni` specifies that we use the `cni` network plugin with actual CNI plugin binaries located in `--cni-bin-dir` (default `/opt/cni/bin`) and CNI plugin configuration located in `--cni-conf-dir` (default `/etc/cni/net.d`).
-* `--network-plugin=kubenet` specifies that we use the `kubenet` network plugin with CNI `bridge` and `host-local` plugins placed in `/opt/cni/bin` or `network-plugin-dir`.
+* `--network-plugin=kubenet` specifies that we use the `kubenet` network plugin with CNI `bridge` and `host-local` plugins placed in `/opt/cni/bin` or `cni-bin-dir`.
 * `--network-plugin-mtu=9001` specifies the MTU to use, currently only used by the `kubenet` network plugin.
@@ -1,5 +1,5 @@
 ---
-approvers:
+reviewers:
 - thockin
 title: Cluster Networking
 ---
@@ -20,15 +20,15 @@ default.  There are 4 distinct networking problems to solve:
 ## Summary

 Kubernetes assumes that pods can communicate with other pods, regardless of
-which host they land on.  We give every pod its own IP address so you do not
+which host they land on.  Every pod gets its own IP address so you do not
 need to explicitly create links between pods and you almost never need to deal
 with mapping container ports to host ports.  This creates a clean,
 backwards-compatible model where pods can be treated much like VMs or physical
 hosts from the perspectives of port allocation, naming, service discovery, load
 balancing, application configuration, and migration.

-To achieve this we must impose some requirements on how you set up your cluster
-networking.
+There are requirements imposed on how you set up your cluster networking to
+achieve this.

 ## Docker model

@@ -47,11 +47,11 @@ are on the same machine (and thus the same virtual bridge).  Containers on
 different machines can not reach each other - in fact they may end up with the
 exact same network ranges and IP addresses.

-In order for Docker containers to communicate across nodes, they must be
-allocated ports on the machine's own IP address, which are then forwarded or
-proxied to the containers.  This obviously means that containers must either
-coordinate which ports they use very carefully or else be allocated ports
-dynamically.
+In order for Docker containers to communicate across nodes, there must
+be allocated ports on the machine’s own IP address, which are then
+forwarded or proxied to the containers. This obviously means that
+containers must either coordinate which ports they use very carefully
+or ports must be allocated dynamically.

 ## Kubernetes model

@@ -84,8 +84,8 @@ applies IP addresses at the `Pod` scope - containers within a `Pod` share their
 network namespaces - including their IP address.  This means that containers
 within a `Pod` can all reach each other's ports on `localhost`. This does imply
 that containers within a `Pod` must coordinate port usage, but this is no
-different than processes in a VM.  We call this the "IP-per-pod" model.  This
-is implemented in Docker as a "pod container" which holds the network namespace
+different than processes in a VM.  This is called the "IP-per-pod" model.  This
+is implemented, using Docker, as a "pod container" which holds the network namespace
 open while "app containers" (the things the user specified) join that namespace
 with Docker's `--net=container:<id>` function.

@@ -106,7 +106,14 @@ imply any preferential status.
 ### ACI

 [Cisco Application Centric Infrastructure](https://www.cisco.com/c/en/us/solutions/data-center-virtualization/application-centric-infrastructure/index.html) offers an integrated overlay and underlay SDN solution that supports containers, virtual machines, and bare metal servers. [ACI](https://www.github.com/noironetworks/aci-containers) provides container networking integration for ACI. An overview of the integration is provided [here](https://www.cisco.com/c/dam/en/us/solutions/collateral/data-center-virtualization/application-centric-infrastructure/solution-overview-c22-739493.pdf).
+
+### Big Cloud Fabric from Big Switch Networks
 
+[Big Cloud Fabric](https://www.bigswitch.com/container-network-automation) is a cloud native networking architecture, designed to run Kubernetes in private cloud/on-premise environments. Using unified physical & virtual SDN, Big Cloud Fabric tackles inherent container networking problems such as load balancing, visibility, troubleshooting, security policies & container traffic monitoring. 
+
+With the help of the Big Cloud Fabric's virtual pod multi-tenant architecture, container orchestration systems such as Kubernetes, RedHat Openshift, Mesosphere DC/OS & Docker Swarm will be natively integrated along side with VM orchestration systems such as VMware, OpenStack & Nutanix. Customers will be able to securely inter-connect any number of these clusters and enable inter-tenant communication between them if needed. 
+
+BCF was recognized by Gartner as a visionary in the latest [Magic Quadrant](http://go.bigswitch.com/17GatedDocuments-MagicQuadrantforDataCenterNetworking_Reg.html). One of the BCF Kubernetes on premise deployments (which includes Kubernetes, DC/OS & VMware running on multiple DCs across different geographic regions) is also referenced [here](https://portworx.com/architects-corner-kubernetes-satya-komala-nio/).

 ### Cilium

@@ -132,15 +139,15 @@ people have reported success with Flannel and Kubernetes.

 ### Google Compute Engine (GCE)

-For the Google Compute Engine cluster configuration scripts, we use [advanced
-routing](https://cloud.google.com/vpc/docs/routes) to
+For the Google Compute Engine cluster configuration scripts, [advanced
+routing](https://cloud.google.com/vpc/docs/routes) is used to
 assign each VM a subnet (default is `/24` - 254 IPs).  Any traffic bound for that
 subnet will be routed directly to the VM by the GCE network fabric.  This is in
 addition to the "main" IP address assigned to the VM, which is NAT'ed for
 outbound internet access.  A linux bridge (called `cbr0`) is configured to exist
 on that subnet, and is passed to docker's `--bridge` flag.

-We start Docker with:
+Docker is started with:

 ```shell
 DOCKER_OPTS="--bridge=cbr0 --iptables=false --ip-masq=false"
@@ -154,8 +161,8 @@ each other and `Nodes` over the `cbr0` bridge.  Those IPs are all routable
 within the GCE project network.

 GCE itself does not know anything about these IPs, though, so it will not NAT
-them for outbound internet traffic.  To achieve that we use an iptables rule to
-masquerade (aka SNAT - to make it seem as if packets came from the `Node`
+them for outbound internet traffic.  To achieve that an iptables rule is used 
+to masquerade (aka SNAT - to make it seem as if packets came from the `Node`
 itself) traffic that is bound for IPs outside the GCE project network
 (10.0.0.0/8).

@@ -163,7 +170,7 @@ itself) traffic that is bound for IPs outside the GCE project network
 iptables -t nat -A POSTROUTING ! -d 10.0.0.0/8 -o eth0 -j MASQUERADE
 ```

-Lastly we enable IP forwarding in the kernel (so the kernel will process
+Lastly IP forwarding is enabled in the kernel (so the kernel will process
 packets for bridged containers):

 ```shell
@@ -11,12 +11,17 @@ spec:
  selector:
    app: nginx
 ---
-apiVersion: apps/v1beta1
+apiVersion: apps/v1
 kind: Deployment
 metadata:
  name: my-nginx
+  labels:
+    app: nginx
 spec:
  replicas: 3
+  selector:
+    matchLabels:
+      app: nginx
  template:
    metadata:
      labels:
@@ -0,0 +1,19 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: nginx-deployment
+spec:
+  selector:
+    matchLabels:
+      app: nginx
+  replicas: 3
+  template:
+    metadata:
+      labels:
+        app: nginx
+    spec:
+      containers:
+      - name: nginx
+        image: nginx:1.7.9
+        ports:
+        - containerPort: 80
@@ -0,0 +1,19 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: my-nginx
+spec:
+  selector:
+    matchLabels:
+      app: nginx
+  replicas: 3
+  template:
+    metadata:
+      labels:
+        app: nginx
+    spec:
+      containers:
+      - name: nginx
+        image: nginx:1.7.9
+        ports:
+        - containerPort: 80
@@ -0,0 +1,12 @@
+apiVersion: v1
+kind: Service
+metadata:
+  name: my-nginx-svc
+  labels:
+    app: nginx
+spec:
+  type: LoadBalancer
+  ports:
+  - port: 80
+  selector:
+    app: nginx
@@ -1,5 +1,5 @@
 ---
-approvers:
+reviewers:
 - sttts
 title: Using Sysctls in a Kubernetes Cluster
 ---
@@ -70,6 +70,9 @@ For Kubernetes 1.4, the following sysctls are supported in the _safe_ set:
 - `net.ipv4.ip_local_port_range`,
 - `net.ipv4.tcp_syncookies`.

+**Note**: The example `net.ipv4.tcp_syncookies` is not namespaced on Linux kernel version 4.4 or lower.
+{: .note}
+
 This list will be extended in future Kubernetes versions when the kubelet
 supports better isolation mechanisms.

@@ -124,3 +127,22 @@ any node which has not enabled those two _unsafe_ sysctls explicitly. As with
 _node-level_ sysctls it is recommended to use [_taints and toleration_
 feature](/docs/user-guide/kubectl/{{page.version}}/#taint) or [taints on nodes](/docs/concepts/configuration/taint-and-toleration/)
 to schedule those pods onto the right nodes.
+
+## PodSecurityPolicy Annotations
+
+The use of sysctl in pods can be controlled via annotations on the PodSecurityPolicy.
+
+Here is an example, it authorizes binding user creating pod with corresponding
+_safe_ and _unsafe_ sysctls.
+
+```yaml
+apiVersion: extensions/v1beta1
+kind: PodSecurityPolicy
+metadata:
+  name: sysctl-psp
+  annotations:
+    security.alpha.kubernetes.io/sysctls: 'kernel.shm_rmid_forced'
+    security.alpha.kubernetes.io/unsafe-sysctls: 'net.ipv4.route.*,kernel.msg*'
+spec:
+ ...
+```