Release 1.7 (#4094)
* Minor fixes in the Deployment doc Signed-off-by: Michail Kargakis <mkargaki@redhat.com> * add NodeRestriction to admission-controllers (#3842) * Admins Can Configure Zones in Storage Class The PR #38505 (https://github.com/kubernetes/kubernetes/pull/38505) added zones optional parameter to Storage Class for AWS and GCE provisioners. That's why documentation needs to be updated accordingly. * document custom resource definitions * add host paths to psp (#3971) * add host paths to psp * add italics * Update ConfigMap doc to explain TTL-based cache updates (#3989) * Update ConfigMap doc to explain TTL-based cache updates * swap word order Change "When a ConfigMap being already consumed..." to "When a ConfigMap already being consumed..." * Update NetworkPolicy docs for v1 * StorageOS Volume plugin * Update GPU docs * docs: HPA autoscaling/v2alpha1 status conditions This commit documents the new status conditions feature for HPA autoscaling/v2alpha1. It demonstrates how to get the status conditions using `kubectl describe`, and how to interpret them. * Update description about NodeRestriction kubelet node can alse create mirror pods for their own static pods. * adding storage as a supported resource to node allocatable Signed-off-by: Vishnu kannan <vishnuk@google.com> * Add documentation for podpreset opt-out annotation This adds the annotation for having the podpreset admission controller to skip (opt-out) manipulating the pod spec. Also, the annotation format for what presets have acted on a pod has been modified to add a prefix of "podpreset-". The new naming makes it such that there is no chance of collision with the newly introduced opt-out annotation (or future ones yet to be added). Opt-out annotation PR: kubernetes/kubernetes#44965 * Update PDB documentation to explain new field (#3885) * update-docs-pdb * Addressed erictune@'s comments * Fix title and add a TOC to the logging concept page * Patch #4118 for typos * Describe setting coredns server in nameserver resolv chain * Address comments in PR #3997. Comment is in https://github.com/kubernetes/kubernetes.github.io/pull/3997/files/f6eb59c67e28efc298c87b1ef49a96bc6adacd1e#diff-7a14981f3dd8eb203f897ce6c11d9828 * Update task for DaemonSet history and rollback (#4098) * Update task for DaemonSet history and rollback Also remove mentions of templateGeneration field because it's deprecated * Address comments * removed lt and gt as operators (#4152) * removed lt and gt as operators * replace lt and gt for node-affinfity * updated based on bsalamat review * Initial draft of upgrade guide for kubeadm clusters. In-place upgrades are supported between 1.6 and 1.7 releases. Rollback instructions to come in a separate commit. Fixes https://github.com/kubernetes/kubeadm/issues/278 * Add local volume documentation (#4050) * Add local volume documentation * Add PV local volume example * Patch PR #3999 * Add documentation for Stackdriver event exporter * Add documentation about controller metrics * Federation: Add task for setting up placement policies (#4075) * Add task for setting up placement policies * Update version of management sidecar in policy engine deployment * Address @nikhiljindal's comments - Lower case filenames - Comments in policy - Typo fixes - Removed type LoadBalancer from OPA Service * Add example that sets cluster selector Per-@nikhiljindal's suggestion * Fix wording and templating per @chenopis * PodDisruptionBudget documentation Improvements (#4140) * Changes from #3885 Title: Update PDB documentation to explain new field Author: foxish * Added Placeholder Disruptions Concept Guide New file: docs/concepts/workloads/pods/disruptions.md Intented contents: concept for Pod Disruption Budget, cross reference to Eviction and Preemption docs. Linked from: concepts > workloads > pods * Added placeholder Configuring PDB Task New file: docs/tasks/run-application/configure-pdb.md Intented contents: task for writing a Pod Disruption Budget. Linked from: tasks > configuring-applications > configure pdb. * Add refs to the "drain a node" task. * Refactor PDB docs. Move the "Requesting an eviction" section from: docs/tasks/administer-cluster/configure-pod-disruption-budget.md -- which is going away -- to: docs/tasks/administer-cluster/safely-drain-node.md The move is verbatim, except for an introductory sentence. Also added assignees. * Refactor of PDB docs Moved the section: Specifying a PodDisruptionBudget from: docs/tasks/administer-cluster/configure-pod-disruption-budget.md to: docs/tasks/run-application/configure-pdb.md because that former file is going away. Move is verbatim. * Explain how Eviction tools should handle failures * Refactor PDB docs Move text from: docs/tasks/administer-cluster/configure-pod-disruption-budget.md to: docs/concepts/workloads/pods/disruptions.md Delete the now empty: docs/tasks/administer-cluster/configure-pod-disruption-budget.md Added a redirects_from section to the new doc, containing the path of the now-deleted doc, plus all the redirects from the deleted doc. * Expand PDB Concept guide Building on a little content from the old task, greatly expanded the Disruptions concept guide, including an abstract example. * Update creating a pdb Task. * Address review comments. * Fixed for all cody-clark's review comments * Address review comments from mml * Address review comments from maisem * Fix missing backtick * Api and Kubectl reference docs updates for 1.7 (#4193) * Fix includes groups * Generated kubectl docs for 1.7 * Generated references docs for 1.7 api * Document node authorization mode * API Aggregator (#4173) * API Aggregator * Additional bullet points * incorporated feedback for apiserver-aggregation.md * split setup-api-aggregator.md into two docs and address feedback * fix link * addressed docs feedback * incorporate feedback * integrate feedback * Add documentation for DNS stub domains (#4063) * Add documentation for DNS stub domains * add additional prereq * fix image path * review feedback * minor grammar and style nits * documentation for using hostAliases to manage hosts file (#4080) * documentation for using hostAliases to manage hosts file * add to table of contents * review comments * update the right command to see hosts file * reformat doc based on suggestion and change some wording * Fix typo for #4080 * Patch PR #4063 * Fix wording in placement policy task introduction * Add update to statefulset concepts and basic tutorial (#4174) * Add update to statefulset concpets and basic tutorial * Address tech comments. * Update ESIPP docs for new added API fields * Custom resource docs * update audit document with advanced audit features added in 1.7 * kubeadm v1.7 documentation updates (#4018) * v1.7 updates for kubeadm * Address review comments * Address Luke's comments * Encrypting secrets at rest and cluster security guide * Edits for Custom DNS Documentation (#4207) * reorganize custom dns doc * format fixes * Update version numbers to 1.7 * Patch PR #4140 (#4215) * Patch PR #4140 * fix link and typos * Update PR template * Update TLS bootstrapping with 1.7 features This includes documenting the new CSR approver built into the controller manager and the kubelet alpha features for certificate rotation. Since the CSR approver changed over the 1.7 release cycle we need to call out the migration steps for those using the alpha feature. This document as a whole could probably use some updates, but the main focus of this PR is just to get these features minimally documented before the release. * Federated ClusterSelector formatting updates from review * complete PR #4181 (#4223) * complete PR #4181 * fix security link * Extensible admission controller (#4092) * extensible-admission-controllers * Update extensible-admission-controllers.md * more on initializers * fixes * Expand external admission webhooks documentation * wrap at 80 chars * more * add reference * Use correct apigroup for network policy * Docs changes to PR #4092 (#4224) * Docs changes to PR #4092 * address feedback * add doc for --as-group in cli Add doc for this pr: https://github.com/kubernetes/kubernetes/pull/43696
This commit is contained in:
zhangchaolei
@@ -0,0 +1,34 @@
|
||||
apiVersion: extensions/v1beta1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
labels:
|
||||
app: opa
|
||||
name: opa
|
||||
namespace: federation-system
|
||||
spec:
|
||||
replicas: 1
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: opa
|
||||
name: opa
|
||||
spec:
|
||||
containers:
|
||||
- name: opa
|
||||
image: openpolicyagent/opa:0.4.10
|
||||
args:
|
||||
- "run"
|
||||
- "--server"
|
||||
- name: kube-mgmt
|
||||
image: openpolicyagent/kube-mgmt:0.2
|
||||
args:
|
||||
- "-kubeconfig=/srv/kubernetes/kubeconfig"
|
||||
- "-cluster=federation/v1beta1/clusters"
|
||||
volumeMounts:
|
||||
- name: federation-kubeconfig
|
||||
mountPath: /srv/kubernetes
|
||||
readOnly: true
|
||||
volumes:
|
||||
- name: federation-kubeconfig
|
||||
secret:
|
||||
secretName: federation-controller-manager-kubeconfig
|
||||
@@ -0,0 +1,13 @@
|
||||
kind: Service
|
||||
apiVersion: v1
|
||||
metadata:
|
||||
name: opa
|
||||
namespace: federation-system
|
||||
spec:
|
||||
selector:
|
||||
app: opa
|
||||
ports:
|
||||
- name: http
|
||||
protocol: TCP
|
||||
port: 8181
|
||||
targetPort: 8181
|
||||
@@ -0,0 +1,74 @@
|
||||
# OPA supports a high-level declarative language named Rego for authoring and
|
||||
# enforcing policies. For more infomration on Rego, visit
|
||||
# http://openpolicyagent.org.
|
||||
|
||||
# Rego policies are namespaced by the "package" directive.
|
||||
package kubernetes.placement
|
||||
|
||||
# Imports provide aliases for data inside the policy engine. In this case, the
|
||||
# policy simply refers to "clusters" below.
|
||||
import data.kubernetes.clusters
|
||||
|
||||
# The "annotations" rule generates a JSON object containing the key
|
||||
# "federation.kubernetes.io/replica-set-preferences" mapped to <preferences>.
|
||||
# The preferences values is generated dynamically by OPA when it evaluates the
|
||||
# rule.
|
||||
#
|
||||
# The SchedulingPolicy Admission Controller running inside the Federation API
|
||||
# server will merge these annotatiosn into incoming Federated resources. By
|
||||
# setting replica-set-preferences, we can control the placement of Federated
|
||||
# ReplicaSets.
|
||||
#
|
||||
# Rules are defined to generate JSON values (booleans, strings, objects, etc.)
|
||||
# When OPA evaluates a rule, it generates a value IF all of the expressions in
|
||||
# the body evaluate successfully. All rules can be understood intuitively as
|
||||
# <head> if <body> where <body> is true if <expr-1> AND <expr-2> AND ...
|
||||
# <expr-N> is true (for some set of data.)
|
||||
annotations["federation.kubernetes.io/replica-set-preferences"] = preferences {
|
||||
input.kind = "ReplicaSet"
|
||||
value = {"clusters": cluster_map, "rebalance": true}
|
||||
json.marshal(value, preferences)
|
||||
}
|
||||
|
||||
# This "annotations" rule generates a value for the "federation.alpha.kubernetes.io/cluster-selector"
|
||||
# annotation.
|
||||
#
|
||||
# In English, the policy asserts that resources in the "production" namespace
|
||||
# that are not annotated with "criticality=low" MUST be placed on clusters
|
||||
# labelled with "on-premise=true".
|
||||
annotations["federation.alpha.kubernetes.io/cluster-selector"] = selector {
|
||||
input.metadata.namespace = "production"
|
||||
not input.metadata.annotations.criticality = "low"
|
||||
json.marshal([{
|
||||
"operator": "=",
|
||||
"key": "on-premise",
|
||||
"values": "[true]",
|
||||
}], selector)
|
||||
}
|
||||
|
||||
# Generates a set of cluster names that satisfy the incoming Federated
|
||||
# ReplicaSet's requirements. In this case, just PCI compliance.
|
||||
replica_set_clusters[cluster_name] {
|
||||
clusters[cluster_name]
|
||||
not insufficient_pci[cluster_name]
|
||||
}
|
||||
|
||||
# Generates a set of clusters that must not be used for Federated ReplicaSets
|
||||
# that request PCI compliance.
|
||||
insufficient_pci[cluster_name] {
|
||||
clusters[cluster_name]
|
||||
input.metadata.annotations["requires-pci"] = "true"
|
||||
not pci_clusters[cluster_name]
|
||||
}
|
||||
|
||||
# Generates a set of clusters that are PCI certified. In this case, we assume
|
||||
# clusters are annotated to indicate if they have passed PCI compliance audits.
|
||||
pci_clusters[cluster_name] {
|
||||
clusters[cluster_name].metadata.annotations["pci-certified"] = "true"
|
||||
}
|
||||
|
||||
# Helper rule to generate a mapping of desired clusters to weights. In this
|
||||
# case, weights are static.
|
||||
cluster_map[cluster_name] = {"weight": 1} {
|
||||
replica_set_clusters[cluster_name]
|
||||
}
|
||||
@@ -0,0 +1,21 @@
|
||||
apiVersion: extensions/v1beta1
|
||||
kind: ReplicaSet
|
||||
metadata:
|
||||
labels:
|
||||
app: nginx-pci
|
||||
name: nginx-pci
|
||||
annotations:
|
||||
requires-pci: "true"
|
||||
spec:
|
||||
replicas: 3
|
||||
selector:
|
||||
matchLabels:
|
||||
app: nginx-pci
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: nginx-pci
|
||||
spec:
|
||||
containers:
|
||||
- image: nginx
|
||||
name: nginx-pci
|
||||
@@ -0,0 +1,29 @@
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: admission
|
||||
namespace: federation-system
|
||||
data:
|
||||
config.yml: |
|
||||
apiVersion: apiserver.k8s.io/v1alpha1
|
||||
kind: AdmissionConfiguration
|
||||
plugins:
|
||||
- name: SchedulingPolicy
|
||||
path: /etc/kubernetes/admission/scheduling-policy-config.yml
|
||||
scheduling-policy-config.yml: |
|
||||
kubeconfig: /etc/kubernetes/admission/opa-kubeconfig
|
||||
opa-kubeconfig: |
|
||||
clusters:
|
||||
- name: opa-api
|
||||
cluster:
|
||||
server: http://opa.federation-system.svc.cluster.local:8181/v0/data/kubernetes/placement
|
||||
users:
|
||||
- name: scheduling-policy
|
||||
user:
|
||||
token: deadbeefsecret
|
||||
contexts:
|
||||
- name: default
|
||||
context:
|
||||
cluster: opa-api
|
||||
user: scheduling-policy
|
||||
current-context: default
|
||||
@@ -97,9 +97,11 @@ coredns-provider.conf has below format:
|
||||
[Global]
|
||||
etcd-endpoints = http://etcd-cluster.my-namespace:2379
|
||||
zones = example.com.
|
||||
coredns-endpoints = <coredns-server-ip>:<port>
|
||||
|
||||
- `etcd-endpoints` is the endpoint to access etcd.
|
||||
- `zones` is the federation domain for which CoreDNS is authoritative and is same as --dns-zone-name flag of `kubefed init`.
|
||||
- `coredns-endpoints` is the endpoint to access coredns server. This is an optional parameter introduced from v1.7 onwards.
|
||||
|
||||
*Note: middleware.etcd.zones in CoreDNS configuration and --dns-zone-name
|
||||
flag to kubefed init should match.*
|
||||
@@ -107,6 +109,11 @@ flag to kubefed init should match.*
|
||||
|
||||
## Setup CoreDNS server in nameserver resolv.conf chain
|
||||
|
||||
*Note: The following section applies only to versions prior to v1.7
|
||||
and will be automatically taken care of if the `coredns-endpoints`
|
||||
parameter is configured in `coredns-provider.conf` as described in
|
||||
section above.*
|
||||
|
||||
Once the federation control plane is deployed and federated clusters
|
||||
are joined to the federation, you need to add the CoreDNS server to the
|
||||
pod's nameserver resolv.conf chain in all the federated clusters as this
|
||||
@@ -118,9 +125,6 @@ achieved by adding the below line to `dnsmasq` container's arg in
|
||||
|
||||
Replace `example.com` above with federation domain.
|
||||
|
||||
*Note: Adding CoreDNS server to the pod's nameserver resolv.conf chain will be
|
||||
automated in subsequent releases.*
|
||||
|
||||
|
||||
Now the federated cluster is ready for cross-cluster service discovery!
|
||||
|
||||
|
||||
@@ -0,0 +1,148 @@
|
||||
---
|
||||
title: Set up placement policies in Federation
|
||||
redirect_from:
|
||||
- "/docs/tutorials/federation/set-up-placement-policies-federation/"
|
||||
- "/docs/tutorials/federation/set-up-placement-policies-federation.html"
|
||||
---
|
||||
|
||||
{% capture overview %}
|
||||
|
||||
This page shows how to enforce policy-based placement decisions over Federated
|
||||
resources using an external policy engine.
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
{% capture prerequisites %}
|
||||
|
||||
You need to have a running Kubernetes cluster (which is referenced as host
|
||||
cluster). Please see one of the [getting started](/docs/getting-started-guides/)
|
||||
guides for installation instructions for your platform.
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
{% capture steps %}
|
||||
|
||||
## Deploying Federation and configuring an external policy engine
|
||||
|
||||
The Federation control plane can be deployed using `kubefed init`.
|
||||
|
||||
After deploying the Federation control plane, you must configure an Admission
|
||||
Controller in the Federation API server that enforces placement decisions
|
||||
received from the external policy engine.
|
||||
|
||||
kubectl create -f scheduling-policy-admission.yaml
|
||||
|
||||
Shown below is an example ConfigMap for the Admission Controller:
|
||||
|
||||
{% include code.html language="yaml" file="scheduling-policy-admission.yaml"
|
||||
ghlink="/docs/tutorials/federation/scheduling-policy-admission.yaml" %}
|
||||
|
||||
The ConfigMap contains three files:
|
||||
|
||||
* `config.yml` specifies the location of the `SchedulingPolicy` Admission
|
||||
Controller config file.
|
||||
* `scheduling-policy-config.yml` specifies the location of the kubeconfig file
|
||||
required to contact the external policy engine. This file can also include a
|
||||
`retryBackoff` value that controls the initial retry backoff delay in
|
||||
milliseconds.
|
||||
* `opa-kubeconfig` is a standard kubeconfig containing the URL and credentials
|
||||
needed to contact the external policy engine.
|
||||
|
||||
Edit the Federation API server deployment to enable the `SchedulingPolicy`
|
||||
Admission Controller.
|
||||
|
||||
kubectl -n federation-system edit deployment federation-apiserver
|
||||
|
||||
Update the Federation API server command line arguments to enable the Admission
|
||||
Controller and mount the ConfigMap into the container. If there's an existing
|
||||
`--admission-control` flag, append `,SchedulingPolicy` instead of adding
|
||||
another line.
|
||||
|
||||
--admission-control=SchedulingPolicy
|
||||
--admission-control-config-file=/etc/kubernetes/admission/config.yml
|
||||
|
||||
Add the following volume to the Federation API server pod:
|
||||
|
||||
- name: admission-config
|
||||
configMap:
|
||||
name: admission
|
||||
|
||||
Add the following volume mount the Federation API server `apiserver` container:
|
||||
|
||||
volumeMounts:
|
||||
- name: admission-config
|
||||
mountPath: /etc/kubernetes/admission
|
||||
|
||||
## Deploying an external policy engine
|
||||
|
||||
The [Open Policy Agent (OPA)](http://openpolicyagent.org) is an open source,
|
||||
general-purpose policy engine that you can use to enforce policy-based placement
|
||||
decisions in the Federation control plane.
|
||||
|
||||
Create a Service in the host cluster to contact the external policy engine:
|
||||
|
||||
kubectl create -f policy-engine-service.yaml
|
||||
|
||||
Shown below is an example Service for OPA.
|
||||
|
||||
{% include code.html language="yaml" file="policy-engine-service.yaml"
|
||||
ghlink="/docs/tutorials/federation/policy-engine-service.yaml" %}
|
||||
|
||||
Create a Deployment in the host cluster with the Federation control plane:
|
||||
|
||||
kubectl create -f policy-engine-deployment.yaml
|
||||
|
||||
Shown below is an example Deployment for OPA.
|
||||
|
||||
{% include code.html language="yaml" file="policy-engine-deployment.yaml"
|
||||
ghlink="/docs/tutorials/federation/policy-engine-deployment.yaml" %}
|
||||
|
||||
## Configuring placement policies via ConfigMaps
|
||||
|
||||
The external policy engine will discover placement policies created in the
|
||||
`kube-federation-scheduling-policy` namespace in the Federation API server.
|
||||
|
||||
Create the namespace if it does not already exist:
|
||||
|
||||
kubectl --context=federation create namespace kube-federation-scheduling-policy
|
||||
|
||||
Configure a sample policy to test the external policy engine:
|
||||
|
||||
{% include code.html language="yaml" file="policy.rego"
|
||||
ghlink="/docs/tutorials/federation/policy.rego" %}
|
||||
|
||||
Shown below is the command to create the sample policy:
|
||||
|
||||
kubectl --context=federation -n kube-federation-scheduling-policy create configmap scheduling-policy --from-file=policy.rego
|
||||
|
||||
This sample policy illustrates a few key ideas:
|
||||
|
||||
* Placement policies can refer to any field in Federated resources.
|
||||
* Placement policies can leverage external context (for example, Cluster
|
||||
metadata) to make decisions.
|
||||
* Administrative policy can be managed centrally.
|
||||
* Policies can define simple interfaces (such as the `requires-pci` annotation) to
|
||||
avoid duplicating logic in manifests.
|
||||
|
||||
## Testing placement policies
|
||||
|
||||
Annotate one of the clusters to indicate that it is PCI certified.
|
||||
|
||||
kubectl --context=federation annotate clusters cluster-name-1 pci-certified=true
|
||||
|
||||
Deploy a Federated ReplicaSet to test the placement policy.
|
||||
|
||||
{% include code.html language="yaml" file="replicaset-example-policy.yaml"
|
||||
ghlink="/docs/tutorials/federation/replicaset-example-policy.yaml" %}
|
||||
|
||||
Shown below is the command to deploy a ReplicaSet that *does* match the policy.
|
||||
|
||||
kubectl --context=federation create -f replicaset-example-policy.yaml
|
||||
|
||||
Inspect the ReplicaSet to confirm the appropriate annotations have been applied:
|
||||
|
||||
kubectl --context=federation get rs nginx-pci -o jsonpath='{.metadata.annotations}'
|
||||
|
||||
{% endcapture %}
|
||||
|
||||
{% include templates/task.md %}
|
||||
Reference in New Issue
Block a user