The example does not offer other aspects about the hostPath, it's some kind of duplicate to the preceding one. So, maybe only the preceding one is enough.
Manually uploaded a :1.1 tag for the staging container, now it's alpine based
(1/4th in image size) and no longer has the warning mentioned in #2837.
Signed-off-by: Ahmet Alp Balkan <ahmetb@google.com>
* Fix the number of processes on Kubernetes master
According to https://kubernetes.io/docs/getting-started-guides/centos/centos_manual_config/, we can know that there are only three processes of Kubernetes running on the master. etcd and flannel are three-party softwares. These are kube-apiserver, kube-controller-manager, and kube-scheduler. If we want to deploy pod on master, we need to start other two processes: kube-proxy and kubelet. So Three or Five is correct. In this case, I perfer Three. Pure master without Pod is fine.
* fix the pr
fix the pr
* Fix the pr
Fix the pr
The link now is a page of kubernetes.io, not a github's .md file any more.
So I think use the link page's title( Building High-Availability Clusters) is fine.
"high-availability.md" may be the older edition's link icon.
* [Federation] Add a note about DNS API scopes to the Federation tutorial
Not sure this is the right place for this, but I wanted to get the text out somewhere before I forgot about it.
* Update set-up-cluster-federation-kubefed.md
* Update set-up-cluster-federation-kubefed.md
* Fix formatting of code blocks.
* hitting the `ClusterIP` from a pod
when client pod and server pod in the same node,client_address is the client pod ip address
when client pod and server pod in different node,client_address is the client pod's node flannel ip address
* Update source-ip.md
* Update source-ip.md
* Fix incorrect sentence
The changes removes an unnecessary word to correct doc sentence.
* Fix incorrect sentence
Remove an extra incorrect word from the sentence.
A sentence in the "What is a DaemonSet" section had an extra "would be
used" which made the sentence grammatically incorrect.
Signed-off-by: Steven Erenst <stevenerenst@gmail.com>
* update index.md
add one word "be".
* update index.md
change "to help users familiar with Docker Compose move to Kubernetes" to "to help Docker Compose users move to Kubernetes".
This is much better. Thank you! At first i misunderstand the meaning of this sentence.
Adding a "View docs/bla-bla.md on GitHub" button next to the
"Edit docs/bla-bla.md" button so that people can view the file
first without clicking the Edit button (which does not work without
forking the repository).
I did not need this because I was trying to do something without
forking. I just found it to be bit difficult to view source of a page
on GitHub.
I'm open to ideas, perhaps we can instead add an article footer button
named "View on GitHub" next to the "Edit this Page".
Signed-off-by: Ahmet Alp Balkan <ahmetb@google.com>
Provide reference of Minikube along with kube-up.sh on creating cluster. This
will give convey message to first time users of the Minikube that they should
make sure Minikube is installed before running this step of
$ kubectl cluster-info
fixes#2863
* modify one word
examples to example
* doc-walkthrough-content
modify
* limitrange-update
I think it's redundant that the second to ,in order to do sth and do
sth!thanks!
* cluster-components-could be running
could be doing sth
* update cluster-components.md
In theory,Master components can be run....
* update cluster-components--add a space
add a space after the comma: "In theory, Master components..."
* Highlighted command --kubectl describe
In web page ,there is no space ,need some empty space,thanks!
* Revert "Highlighted command --kubectl describe"
This reverts commit a70d0a3e3537c3f91b197b23f949e1e506c951d1.
* kubeadm reference--change any to some
kubeadm can install any add-on ?
Ubuntu default install not include apt-transport-https. so if you want to download package from https repo, you need install apt-transport-https package first
- Proposing a different way to run curl without proxy, without touching current environment
- There's a formatting problem and the whole thing is being displayed in a single line

It not necessary to mention where the functions are implemented in
document. Further more, the directories recorded here is out of date.
Now they are in:
staging/src/k8s.io/apiserver/plugin/pkg/authenticator/...
Actually according to the command in example:
kubectl create secret generic ssh-key-secret --from-file=ssh-privatekey=/path/to/.ssh/id_rsa --from-file=ssh-publickey=/path/to/.ssh/id_rsa.pub
The keys will be available with names ssh-publickey and ssh-privatekey.
When I test this command, if the configmap has a label, it just delete the configmaps which has the same label key and that are not in the file.
kubectl apply --prune -f manifest.yaml --all --prune-whitelist=core/v1/ConfigMap
So I advise change the description like this:
"Apply the configuration in manifest.yaml and delete all the other configmaps with the same label key that are not in the file."
The file 'blob/master/docs/devel/api-conventions.md' has moved to
'community/blob/master/contributors/devel/api-conventions.md'
Updates the links associated with these file.
The key-value "optional:true" doesn't work in 1.5.2 yet.
So should we add a note here? "The key-value "optional:true" will be work after kubernetes 1.5.2."
So that it is above "Accessing the Cluster" article which describes
more advanced topics and does not look coherent when it is read
linearly.
Signed-off-by: Ahmet Alp Balkan <ahmetb@google.com>
Increase the font-size and line-height CSS
properties for the paragraph and list items
within the documentation body content.
Adjusting these properties will make the
information a little easier to read.
* Initial commit for Init Containers migration to Concepts section
* Add 1.5 beta include file
* Change include to renamed user-guide-content-moved.md file
* Fix Concepts/KO/Pods ToC
* reformat examples to bullet points
* fix formatting
* Add back Detailed Behavior, Support and compatibility
* Adjust formatting issues
* revised based on feedback from Steve
* complete sentence rewrite
When the namespace name only contains number, such as "10", the pod can't be created in this kind of namespace, the pod will be put in default namespace.
1. fix unreachable federated resources links
2. federation proposal doc has been relocated, so i fix this link
3. make federation resources supporting cascading deletion clickable
4. make link to `Kubernetes Scaling and Performance Goals` clickable
5. other typo fixes
Former: The namespace name may contain a maximum of 63 alphanumeric characters and dashes (-) , can not contain capital letters.
Now: The namespace name may contain a maximum of 63 alphanumeric characters and dashes (-) but can not contain capital letters.
1. Namespace can not contain capital letter.
2. For service, internal endpoints for cluster connections and external endpoints for external users. Here, it lost "external".
- Proposing a different way to run curl without proxy, without touching current environment
- There's a formatting problem and the whole thing is being displayed in a single line

It not necessary to mention where the functions are implemented in
document. Further more, the directories recorded here is out of date.
Now they are in:
staging/src/k8s.io/apiserver/plugin/pkg/authenticator/...
Actually according to the command in example:
kubectl create secret generic ssh-key-secret --from-file=ssh-privatekey=/path/to/.ssh/id_rsa --from-file=ssh-publickey=/path/to/.ssh/id_rsa.pub
The keys will be available with names ssh-publickey and ssh-privatekey.
When I test this command, if the configmap has a label, it just delete the configmaps which has the same label key and that are not in the file.
kubectl apply --prune -f manifest.yaml --all --prune-whitelist=core/v1/ConfigMap
So I advise change the description like this:
"Apply the configuration in manifest.yaml and delete all the other configmaps with the same label key that are not in the file."
The file 'blob/master/docs/devel/api-conventions.md' has moved to
'community/blob/master/contributors/devel/api-conventions.md'
Updates the links associated with these file.
The key-value "optional:true" doesn't work in 1.5.2 yet.
So should we add a note here? "The key-value "optional:true" will be work after kubernetes 1.5.2."
Some of the serial numbers of the ordered list in the "Static pod creation" section are incorrect because items in the list are segmented by code blocks. I think this can be solved in at least the following ways:
1.Remove the ordered list style and just use plain text each beginning with "1", "2",...,or "step 1", "step 2",... (My modofication adopts this one)
2.Replace the ordered list with an unordered list in which each item beginning with "step 1", "step 2",...
3.Use HTML tags, such as <ol>,<li> to give the lists involved
Are there any other better ways for this problem? I would be appreciated to know.
So that it is above "Accessing the Cluster" article which describes
more advanced topics and does not look coherent when it is read
linearly.
Signed-off-by: Ahmet Alp Balkan <ahmetb@google.com>
Increase the font-size and line-height CSS
properties for the paragraph and list items
within the documentation body content.
Adjusting these properties will make the
information a little easier to read.
* Initial commit for Init Containers migration to Concepts section
* Add 1.5 beta include file
* Change include to renamed user-guide-content-moved.md file
* Fix Concepts/KO/Pods ToC
* reformat examples to bullet points
* fix formatting
* Add back Detailed Behavior, Support and compatibility
* Adjust formatting issues
* revised based on feedback from Steve
* complete sentence rewrite
When the namespace name only contains number, such as "10", the pod can't be created in this kind of namespace, the pod will be put in default namespace.
1. fix unreachable federated resources links
2. federation proposal doc has been relocated, so i fix this link
3. make federation resources supporting cascading deletion clickable
4. make link to `Kubernetes Scaling and Performance Goals` clickable
5. other typo fixes
Former: The namespace name may contain a maximum of 63 alphanumeric characters and dashes (-) , can not contain capital letters.
Now: The namespace name may contain a maximum of 63 alphanumeric characters and dashes (-) but can not contain capital letters.
1. Namespace can not contain capital letter.
2. For service, internal endpoints for cluster connections and external endpoints for external users. Here, it lost "external".
Netlify will redirect with a 301 status to the right places without the
need of a custom javascript file.
The current redirects.js is not removed so production can keep working
as it is for now.
Signed-off-by: David Calavera <david.calavera@gmail.com>
* adding Operational Considerations section
modified: docs/getting-started-guides/ubuntu/logging.md
* Moving logging section from operational-considerations.md to logging.md
renamed: docs/getting-started-guides/ubuntu/operations.md -> docs/getting-started-guides/ubuntu/operational-considerations.md
* mapping to the proposed name
* ditched CDK naming in favor of k8s or cluster
* add whitelines for titles
Beginners don't know the details for troubleshooting, better instruction will save time for beginners to trouble shoot logs in minikube complaining gcr.io not reachable and etc.
I installed gcr.io/google_containers/kubernetes-dashboard-amd64:v1.5.0. When I deploy a containerized app through dashboad ui, the appname can be "dxltest1" . The "dxltest1" can be deployed successfully. so the appname can be end with a number.
The data dir should not be at the root of the volume, because many
volume types create files or directories there automatically.
The database initialization script requires the data dir to be empty,
and we shouldn't try to guess which files may exist.
In particular, by retaining lost+found, it also becomes possible
to recover files that end up there.
This page previously included instructions to use curl to download and install
Kubectl via http, rather than with https. This exposed the end user to
man-in-the-middle attacks.
Additionally, the instructions given here are slightly different from the
authoritative page at https://kubernetes.io/docs/getting-started-guides/kubectl/
- which covers useful things like getting the latest release and setting up tab
autocompletion.
Rather than just change 'http' to 'https' in the URLs here, I've updated the
minikube setup page to include a reference to the kubectl setup page.
"Action use IaaS providers reliable storage (e.g. GCE PD or AWS EBS volume) for VMs with apiserver+etcd" did not follow the same format as other actions in this section.
1. Update the script to skip checking a whole folder
2. Skip checking auto-generated kubectl docs since they already work without title
3. Remove some skipped files in skip_toc_check.txt since we can now skip a folder
- This supercedes PR #1770
- This breaks up the section from one huge installation page into:
- A new index page.
- New support grid with commercial and community support options.
- Move to lifecycle-based pages based on operational tasks.
- Add support for local deployments via LXD.
- Added screenshots to images/docs/ubuntu
- Add backup page.
- Add decommissioning page.
- Add a glossary of terms page.
- Rewritten installation page.
- Add logging page.
- Add monitoring page.
- Add networking page, flannel only for now, calico in progress.
- Add a scaling page.
- Add a security page.
- Add a storage page.
- Add a troubleshooting page.
- Add an upgrade page.
- Add a cluster validation page.
- Add new ubuntu content to the index page.
- Add Ubuntu to _data TOC.
- Add warning about choosing ipv6 with LXD, thanks Ed Baluf.
- Template-ize all the pages per jaredb's review.
* Example of `/etc/sysconfig/flanneld` is not valid
Configuration keys were different, I got those errors when followed steps from docs:
```
[ailyin@elite ~]$ systemctl status flanneld.service
● flanneld.service - Flanneld overlay address etcd agent
Loaded: loaded (/usr/lib/systemd/system/flanneld.service; enabled; vendor preset: disabled)
Active: activating (start) since Wed 2017-01-04 19:00:21 EST; 34s ago
Main PID: 4668 (flanneld)
Memory: 3.5M
CGroup: /system.slice/flanneld.service
└─4668 /usr/bin/flanneld -etcd-endpoints= -etcd-prefix=
Jan 04 19:00:46 elite.dev flanneld[4668]: E0104 19:00:46.395562 04668 network.go:53] Failed to retrieve network config: client: etcd cluster is unavailable or misconfigured
Jan 04 19:00:47 elite.dev flanneld[4668]: E0104 19:00:47.395788 04668 network.go:53] Failed to retrieve network config: client: etcd cluster is unavailable or misconfigured
Jan 04 19:00:48 elite.dev flanneld[4668]: E0104 19:00:48.395990 04668 network.go:53] Failed to retrieve network config: client: etcd cluster is unavailable or misconfigured
Jan 04 19:00:49 elite.dev flanneld[4668]: E0104 19:00:49.396141 04668 network.go:53] Failed to retrieve network config: client: etcd cluster is unavailable or misconfigured
Jan 04 19:00:50 elite.dev flanneld[4668]: E0104 19:00:50.396306 04668 network.go:53] Failed to retrieve network config: client: etcd cluster is unavailable or misconfigured
Jan 04 19:00:51 elite.dev flanneld[4668]: E0104 19:00:51.396451 04668 network.go:53] Failed to retrieve network config: client: etcd cluster is unavailable or misconfigured
Jan 04 19:00:52 elite.dev flanneld[4668]: E0104 19:00:52.396572 04668 network.go:53] Failed to retrieve network config: client: etcd cluster is unavailable or misconfigured
Jan 04 19:00:53 elite.dev flanneld[4668]: E0104 19:00:53.396701 04668 network.go:53] Failed to retrieve network config: client: etcd cluster is unavailable or misconfigured
Jan 04 19:00:54 elite.dev flanneld[4668]: E0104 19:00:54.396855 04668 network.go:53] Failed to retrieve network config: client: etcd cluster is unavailable or misconfigured
Jan 04 19:00:55 elite.dev flanneld[4668]: E0104 19:00:55.396992 04668 network.go:53] Failed to retrieve network config: client: etcd cluster is unavailable or misconfigured
```
* Updated Flanneld configuration example for node (same as for master);
Create additional files for Kompose documentation including
architecture, development, a more extensive index and a user guide.
This also bumps Kompose up to the top on tools page to be symmetrical to
the what's listed on the side navbar.
I have assigned myself to each doc in order to get updates / any
questions regarding the updating of documentation.
The way that the document currently flows and how we typically perceive credentials, we usually set our username first followed by the password.
Seeing the configuration first set password then username seems counter-intuitive. There is nothing wrong with the docs, it is just clearer to view username=>password.
* WIP: New Docs Landing Page and ToC Organization.
* More changes to Landing Page/TOC nav.
* Debugging fixes to TOC.
* More debugging and wording changes to site nav and TOCs.
* More debugging fixes for site nav and TOC.
* Yet more debugging fixes to site nav and TOC.
* Initial edit to Docs landing page.
* Debug edit.
* Attempt at removing left-nav TOC from Docs Landing Page.
* Incremental fix for docs landing page.
* More incremental layout fixes for landing page.
* Incremental change.
* CSS fixes; fills out landing page with additional information.
* More layout fixes.
* More layout and text edits for the docs landing page.
* More tinkering with CSS and layouts; fixed a hanging <code> tag.
* Last tweak for now.
* Modified text in response to review comments.
* Removed Samples and Contribution Evangelism from docs landing page.
* Removed intro text on Docs landing page.
* Reorganized landing page to drop weird CSS formatting and reorganize "Quickstarts" into setup guidance; separated out tutorial. Corrected spelling error in main page header.
* Fixed links to major subsections (Guides/Tutorials/Tasks/Concepts).
Clarified /current/ usage of each--subject to change as Guides get deprecated and Tutorials get filled out.
Changed description for kubeadm/kops links to mention that they are newer/experimental ways to set up Kubernetes.
* Corrected Guides link.
The headless service spec example requires the attribute clusterIP: None. Without this, the service that is created is not headless and the two example pods will not be able to nslookup one another.
In Managing the ZooKeeper Process : Handling Process Failure : paragraph 4 :
"The command used as the container's entry point has PID 1, and the the ZooKeeper process, a child of the entry point, has PID 23." =>
"The command used as the container's entry point has PID 1, and the ZooKeeper process, a child of the entry point, has PID 23."
1. Make plural nouns into same style. See it is `Services` in first sentence and `Pods` in the line 286 , but others are `Service`s and `Pod`s .So change style for these plural nouns for consistency and beauty.
2. Fix minor grammar.
Provide valid --cloud-providers, vagrant and openshift were listed but is not actually a valid cloud provider in this context in the current code base while a number of other valid options were omitted including azure, cloudstack, openstack, photon and vsphere.
The original one is: "kops update cluster useast1.dev.awsdata.com --yes"
However, the previous example used useast1.dev.example.com to create configuration file.
So, the "awsdata" should be changed into example.
This commit attempts to make more clear the differences between service
types by ensuring definitions are consistent and concise. It also aims
to fix a few typos and grammatical issues.
OVN is an opensource network virtualization solution
developed by the Open vSwitch community. It lets one
create logical switches, logical routers, stateful
ACLs, load-balancers etc to build different virtual
networking topologies. The project has a specific
Kubernetes plugin and documentation at
https://github.com/openvswitch/ovn-kubernetes.
* Initial commit
* Adds section for cleanup
Corrects some spelling errors
decapitalizes liveness and readiness
* Adds test for zookeeper example
* Address enisoc review
* Remove space between shell and raw annotation
* Remove extranous inserted text
* Remove fencing statement
* Modify sentence for grammer
* refocus to zookeeper with some loss of generality
* Spelling, Grammar, DNS link
* update to address foxish comments
This commit moves the site to use the official Jekyll SEO Tag
plugin (https://github.com/jekyll/jekyll-seo-tag) to generate the
search-engine meta in the HEAD of each page. The title and canonical
URL output should be largely the same as before, but with additional
metadata, such as JSON-LD (for richer indexing), Twitter card
metadat, etc.
Like the other plugins, this should largely work out-of-the-box,
with no additional day-to-day configuration, but is customizable
where desired.
Starting with Jekyll v3.3, Jekyll ships with `relative_url` and
`absolute_url` filters. Rather than manual concatenating strings,
we can rely on Jekyll to handle the logic in Ruby land, which
takes into account the site's URL (locally and in production), and
handles things like double "/"s.
Remove baseurl from the config for three reasons:
1. It's not used anyplace within the site
2. The site’s base URL is only to be used when the resulting site
lives at a subpath of the domain. Otherwise, the subpath should be
nil, not /.
3. When the Pages Gem is loaded as part of the :jekyll_plugins
group, baseurl and url are set automatically.
Although the title already has a "name" entry, this commit replicates
the name as "title", and adds a "description" entry, both of which
are shared across common, official Jekyll plugins like jekyll-feed
and jekyll-sitemap.
This allows the feed and sitemap to properly output the title and
description in various places.
Jekyll Sitemap (https://github.com/jekyll/jekyll-sitemap) is an
official Jekyll plugin, which should serve as a drop-in replacement
for the existing sitemap.xml. The resulting sitemap should be largely
similar to the existing sitemap, but with a shared, battle-tested
template that accounts for all sorts of edge cases, handles collections
and static files, etc.
Jekyll Feed (https://github.com/jekyll/jekyll-feed) is an official
Jekyll plugin which should replicate the existing feed nearly
identically, but with a shared, battle-tested template that
accounts for all sorts of edge cases like relative links in feed
entries.
It should be a drop in replacement and "just work" without any
additional configuration.
The GitHub Pages Gem (https://github.com/github/pages-gem)
is a meta-gem that does two things:
1. It locks dependencies to the same version used by GitHub Pages
ensuring that when you build the site locally, you're using the same
version of plugins and other dependencies used in production.
2. When loaded as part of the :jekyll_plugins group, it allows
the Gem to set certain configuration defaults and overrides
(such as activating default plugins) to ensure, once again
that your local preview replicates the production version
as closely as possible.
Things looked odd due to the padding on code blocks; they sat 2px above the text surrounding them. This PR will fix that by aligning inline code blocks, and the text surrounding them, on their baseline.
There are too many horizontal lines in the documentation, which makes knowing which things are headings (h2) and which are sub-headings (h3) almost impossible.
I've tidied up the weights a little to make it more obvious and trimmed some excess whitespace. Ultimately I think a typographer's eye on the single choice of font-face (perhaps a serif for headings?) would be worthwhile.
`-d` is used for the Linux version of the command but in the Mac/BSD version they use `-D`.
Using `--decode` we are sure that the flag is compatible with both.
Although the Fedora install docs are under the "Bare Metal"
section in the docs, the instructions provided work fine
for Fedora installs in virtual machines
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
The Fedora multi-node doc need to tell users to install
flannel, since previous setup won't have brought it in.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
The install of kubernetes pulls in docker, which has a direct
dependancy on iptables. So there is no need to tell people to
yum install iptables manually.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
The /var/run/kubernetes directory is already created by
installation of the kubernetes RPM with correct user
and group ownership and permissions.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
The docs still refer to etcd using port 4001, but in all
currently released Fedora versions it will be using 2379
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
The KUBE_LOGTOSTDERR, KUBE_LOG_LEVEL and KUBE_ALLOW_PRIV
settings will already have correct values out of the box,
so no need to tell people to set them.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
There is no reason to tell people to enable updates-testing
repo for Fedora by default. The latest stable packages are
already in the main repos and updates-testing should only be
used by people wishing to test & report quality of pre-released
software, not general Kubnetes users.
Also remove the link to Koji builds - any Koji build which
is working well enough to user will be submitted as an update.
Telling users to install packages straight from Koji will
lead to them using potentially unstable / buggy builds.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
dnf has replaced yum as the standard package install tool
in all versions of Fedora that are currently supported.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
The content is in
docs/tutorials/stateful-applications/basic-stateful-set.md
The example application, named web, is in
docs/tutorials/stateful-applications/web.yaml
The tutorials TOC and index have been udpated to include a Stateful
Applications section of which this example is a subsection
Removes deprecation from comment wrt PetSets
Creates separate section in concepts/index.md for Controllers
Fixes grammer "an integer"
Removes claim about persistent storage being auto-deleted
Clarifies stability
Adds call out tick around attributes
The two affected lines:
a) Were both represented entirely in a code block due to indentation,
which does not appear to be the intent.
b) Look like a mess if you don't know they are formatted as a URL query
string.
The two affected lines:
a) Were both represented entirely in a code block due to indentation,
which does not appear to be the intent.
b) Look like a mess if you don't know they are formatted as a URL query
string.
This makes the image a lot smaller and it's using an official image
as the base to make sure we get security updates etc.
The image size went from:
```
REPOSITORY SIZE
gcr.io/google-samples/k8sdocs 837 MB
```
To:
```
REPOSITORY SIZE
gcr.io/google-samples/k8sdocs 227 MB
```
Signed-off-by: Jess Frazelle <acidburn@google.com>
This is the new disclosure process as discussed here: https://github.com/kubernetes/kubernetes/issues/35462
This relies on a doc to be merged into docs/devel/security-release-process.md but this doc can be reviewed in parallel.
You can find a draft of the content of that doc on #35462
"What's next" was listed underneath "Patterns to avoid" because the
template hard-codes the heading depth. Increase depth of all other
headings by 1 to match the template's expectations.
Create an easy to remember and locate URL for security disclosure
process. This URL will need to be placed in lots of templates and tools
so make it easy.
The content is in
docs/tutorials/stateful-applications/basic-stateful-set.md
The example application, named web, is in
docs/tutorials/stateful-applications/web.yaml
The tutorials TOC and index have been udpated to include a Stateful
Applications section of which this example is a subsection
For kubenet, users should run kubelet with argument
--non-masquerade-cidr, or pods may have problem when access
"10.0.0.0/8" cidr. Also, if --pod-cidr is set outside the scope
of --non-masquerade-cidr, node will do ip masquerade for ingress
packets, which is not excepted.
- Change "Kubernetes", which is not a thing you can assign copyright to, to "The Kubernetes Authors", in line with comments in source files. This may or may not be more appropriate to assign copyright to CNCF.
- Add link to LICENSE with CC BY SA text, which will only be true when #1763 is merged.
- I don't think you'll like the CSS, but without the 'light-text' tag you get a blue/purple link.
moved text under command and changed wording based on some feedback. With regards to mentioning cloud providers. I think a lot of this documentation in general assumes a public cloud like Google so it may be worthwhile to mention a specific private cloud where this isn't implemented.
Removes deprecation from comment wrt PetSets
Creates separate section in concepts/index.md for Controllers
Fixes grammer "an integer"
Removes claim about persistent storage being auto-deleted
Clarifies stability
Adds call out tick around attributes
RestartPolicy section gives an example of what kind of controller
should be used for what type of workload. However it only lists
rc for web-server type workloads. Adding Deployment next to rc
as well.
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
This way users don't have to read to the bottom to realize they need to add
--authorization-rbac-super-user=admin to their cluster when bootstrapping for the first time.
🐑
* Fix broken link by removing invalid characters....
The link to this page is broken because of invalid characters introduced with PR #1549.
* Update networking.md
This removes the stars rating that appears on the bottom of each page.
Doing so also required the update the "a.issue" css that correctly
aligns the "Create Issue" button and "Edit this Page" button on the
bottom of the page.
This change provide a href to users, so that users knowns where to
download this loopback cin plugin. And a more accurate link about
cni configuration file is provided.
assuming that kubernetes is almost always used with docker. In case other container solutions as lxc are integrated this section
will need further revision. Just using the bare etcl command is a bit too less, at least there should be a hint that you log in into the container.
Quick getting started guide for kops.
Also try to provide some guidance as to when to use kops and when
kubeadm; based on discussions with sig-cluster-lifecycle that kubeadm is
a building block and not a provisioning tool.
I used "kubeadm" as a simplifying concept for "kubeadm and the other
work done by sig-cluster-lifecycle that kubeadm is a part of", in that
tools that don't (yet) use kubeadm are still leveraging the kubeadm
stream of work.
Docker for Windows 1.12/1.12.1 does not work as described when pushing images to the google container image registry - it stops with an error message. It takes a long time (6h to several days) to find a solution and since Docker is becoming more popular on windows this seems worth mentioning.
The old text said that the authorizer is expected to determine group
memberships when the authenticator does not. This not true. It is
allowed, but not expected --- and none of the standard authorizers do
it. I tried composing a brief correct statement about this, but the
reviews were mainly aghast that internal details of some non-standard
authorizers were being injected into the discussion of authentictors.
I decided that the better part of valor is simply to delete the whole
topic from here. Besides, it is a conclusion that any reader would
normally draw --- since there is no statement forbidding it (nor
indeed any indication that there might be a reason to forbid it), any
reader would naturally conclude that an authorizer is free to derive
additional intermediate information of any sort and in any way it
likes.
Noted this where the relevance of api group is introduced, and
corrected the reference to what the empty string means ("core" api
group, which is the terminology used in the page that introduces api
group, rather than the old text "default").
- fix list of valid resource types
- fix showing template
It was fainling with following error message:
Liquid Warning: Liquid syntax error (line 68): [:dot, "."] is not a valid expression in "{{.status.phase}}" in docs/user-guide/kubectl/kubectl_get.md
1) The example given in the section "Using Secrets as Environment
Variables" incorrectly stated the secret was using a volume.
2) The commands used to produce secrets in the section "Use-Case: Pods
with prod / test credentials" has been modified such that the literal
used matches the text for what expected files should be on the
filesystem (changed user to username).
In a corporate environment you need to pass proxy information, and just setting the appropriate environment variables does not work.
The code snippet here works for me
1. Update script to:
a) Rename kubectl.md to index.md
b) Remove "See Also" section from kubectl subcommands md files
2. Update the _date/reference.yml link to kubectl
Noted that every bearer token, in any of the four authentication
strategies that use bearer tokens, appears in an HTTP header
value without additional quotation/encoding (beyond that supported
by HTTP). Included a fully concrete example. Wrote this down once,
where the issue first arises, and referenced it from the other relevant
strategies.
This constraint was elicited in #sig-auth discussion
on Sep 21, and not previously stated explicitly and in a way that
clearly applied to all four kinds of bearer token --- leaving the
reader to wonder if some other encoding is expected.
It seems docs were moved from kubernetes repo and now
live in the website repo. I've updated the update-demo docs
to work again with actual references.
I'm not sure on what the best wording for this should be, but `Node Phase` is no longer populated, and so the docs should say so.
@random-liu on slack/#sig-node told me that they believed node phase was unused.
cc @dawnchen
2016-05-20 15:19:35 -07:00
3161 changed files with 549518 additions and 32206 deletions
**First off, thanks for taking the time to contribute!**
The following is a set of guidelines for contributing to Kubernetes documentation, hosted at [Kubernetes.io](http://kubernetes.io/).
These are just guidelines, not rules. Use your best judgment, and feel free to propose changes to this document in a pull request.
## Before you get started
### Code of Conduct
Kubernetes follows the [Cloud Native Computing Foundation (CNCF) Code of Conduct](https://github.com/cncf/foundation/blob/master/code-of-conduct.md). By participating, you are expected to uphold this code. Please report unacceptable behavior to Sarah Novotny[sarahnovotny@google.com](mailto:sarahnovotny@google.com) and/or Dan Kohn[dan@linuxfoundation.org](mailto:dan@linuxfoundation.org).
### Documentation and Site Decisions
The [Kubernetes SIG Docs Discussion Group](https://groups.google.com/forum/#!forum/kubernetes-sig-docs) is the discussion group for doc releases, suggested site improvements, and improving the doc contribution experience. If you are planning to be a regular contributor, join this group to stay informed and involved.
### Style Guides and Templates
Before submitting a pull request to create new content, please review the [Kubernetes.io style guide](http://kubernetes.io/docs/contribute/style-guide/) and follow the [instructions for using page templates](http://kubernetes.io/docs/contribute/page-templates/).
## Contributing to Documentation
### Reporting Documentation Issues
Kubernetes.io uses github issues to track documentation issues and requests. If you see a documentation issue, submit an issue using the following steps:
1. Check the [kubernetes.io issues list](https://github.com/kubernetes/kubernetes.github.io/issues) as you might find out the issue is a duplicate.
2. Use the [included template for every new issue](https://github.com/kubernetes/kubernetes.github.io/issues/new). When you create a bug report, include as many details as possible and include suggested fixes to the issue.
Note that code issues should be filed against the main kubernetes repository, while documentation issues should go in the kubernetes.io repository.
### Submitting Documentation Pull Requests
If you're fixing an issue in the existing documentation, you should submit a PR against the master branch. Follow [these instructions to create a documentation pull request against the kubernetes.io repository](http://kubernetes.io/docs/contribute/create-pull-request/).
## Instructions for Contributing to the Docs/Website
## Instructions for Contributing to the Kubernetes Documentation
Welcome! We are very pleased you want to contribute to the documentation and/or website for Kubernetes.
Welcome! We are very pleased you want to contribute to the Kubernetes documentation.
You can click the "Fork" button in the upper-right area of the screen to create a copy of our site on your GitHub account called a "fork." Make any changes you want in your fork, and when you are ready to send those changes to us, go to the index page for your fork and click "New Pull Request" to let us know about it.
You can click the **Fork** button in the upper-right area of the screen to create a copy of this repository in your GitHub account called a *fork*. Make any changes you want in your fork, and when you are ready to send those changes to us, go to your fork and create a new pull request to let us know about it.
## Staging the site on GitHub Pages
For more information about contributing to the Kubernetes documentation, see:
If you want to see your changes staged without having to install anything locally, remove the CNAME file in this directory and
change the name of the fork to be:
YOUR_GITHUB_USERNAME.github.io
Then make your changes.
When you visit [http://YOUR_GITHUB_USERNAME.github.io](http://YOUR_GITHUB_USERNAME.github.io) you should see a special-to-you version of the site that contains the changes you just made.
## Staging the site locally (using Docker)
Don't like installing stuff? Download and run a local staging server with a single `docker run` command.
docker run -ti --rm -v "$PWD":/k8sdocs -p 4000:4000 gcr.io/google-samples/k8sdocs:1.0
Then visit [http://localhost:4000](http://localhost:4000) to see our site. Any changes you make on your local machine will be automatically staged.
If you're interested you can view [the Dockerfile for this image](https://github.com/kubernetes/kubernetes.github.io/blob/master/staging-container/Dockerfile).
## Staging the site locally (from scratch setup)
The below commands to setup your environment for running GitHub pages locally. Then, any edits you make will be viewable
on a lightweight webserver that runs on your local machine.
This will typically be the fastest way (by far) to iterate on docs changes and see them staged, once you get this set up, but it does involve several install steps that take awhile to complete, and makes system-wide modifications.
Install Ruby 2.2 or higher. If you're on Linux, run these commands:
apt-get install software-properties-common
apt-add-repository ppa:brightbox/ruby-ng
apt-get install ruby2.2
apt-get install ruby2.2-dev
* If you're on a Mac, follow [these instructions](https://gorails.com/setup/osx/).
* If you're on a Windows machine you can use the [Ruby Installer](http://rubyinstaller.org/downloads/). During the installation make sure to check the option for *Add Ruby executables to your PATH*.
The remainder of the steps should work the same across operating systems.
To confirm you've installed Ruby correctly, at the command prompt run `gem --version` and you should get a response with your version number. Likewise you can confirm you have Git installed properly by running `git --version`, which will respond with your version of Git.
Install the GitHub Pages package, which includes Jekyll:
Make any changes you want. Then, to see your changes locally:
cd kubernetes.github.io
jekyll serve
Your copy of the site will then be viewable at: [http://localhost:4000](http://localhost:4000)
(or wherever Jekyll tells you).
## GitHub help
If you're a bit rusty with git/GitHub, you might wanna read
[this](http://readwrite.com/2013/10/02/github-for-beginners-part-2) for a refresher.
## Common Tasks
### Edit Page Titles or Change the Left Navigation
Edit the yaml files in `/_data/` for the Guides, Reference, Samples, or Support areas.
You may have to exit and `jekyll clean` before restarting the `jekyll serve` to
get changes to files in `/_data/` to show up.
### Add Images
Put the new image in `/images/docs/` if it's for the documentation, and just `/images/` if it's for the website.
**For diagrams, we greatly prefer SVG files!**
### Include code from another file
To include a file that is hosted on this GitHub repo, insert this code:
<pre>{% include code.html language="<LEXERVALUE>" file="<RELATIVEPATH>" ghlink="<PATHFROMROOT>" %}</pre>
*`LEXERVALUE`: The language in which the file was written; must be [a value supported by Rouge](https://github.com/jneen/rouge/wiki/list-of-supported-languages-and-lexers).
*`RELATIVEPATH`: The path to the file you're including, relative to the current file.
*`PATHFROMROOT`: The path to the file relative to root, e.g. `/docs/admin/foo.yaml`
To include a file that is hosted in the external, main Kubernetes repo, make sure it's added to [/update-imported-docs.sh](https://github.com/kubernetes/kubernetes.github.io/blob/master/update-imported-docs.sh), and run it so that the file gets downloaded, then enter:
<pre>{% include code.html language="<LEXERVALUE>" file="<RELATIVEPATH>" k8slink="<PATHFROMK8SROOT>" %}</pre>
*`PATHFROMK8SROOT`: The path to the file relative to the root of [the Kubernetes repo](https://github.com/kubernetes/kubernetes/tree/release-1.2), e.g. `/examples/rbd/foo.yaml`
## Using tabs for multi-language examples
By specifying some inline CSV in a varable called `tabspec`, you can include a file
called `tabs.html` that generates tabs showing code examples in multiple langauges.
In English, this would read: "Create a set of tabs with the alias `servicesample`,
and have tabs visually labeled "JSON" and "YAML" that use `json` and `yaml` Rouge syntax highlighting, which display the contents of
`service-sample.{extension}` on the page, and link to the file in GitHub at (full path)."
Example file: [Pods: Multi-Container](http://kubernetes.io/docs/user-guide/pods/multi-container/).
## Use a global variable
The `/_config.yml` file defines some useful variables you can use when editing docs.
*`page.githubbranch`: The name of the GitHub branch on the Kubernetes repo that is associated with this branch of the docs. e.g. `release-1.2`
*`page.version` The version of Kubernetes associated with this branch of the docs. e.g. `v1.2`
*`page.docsbranch` The name of the GitHub branch on the Docs/Website repo that you are currently using. e.g. `release-1.1` or `master`
This keeps the docs you're editing aligned with the Kubernetes version you're talking about. For example, if you define a link like so, you'll never have to worry about it going stale in future doc branches:
<pre>View the README [here](http://releases.k8s.io/{{page.githubbranch}}/cluster/addons/README.md).</pre>
(Or whatever Kubernetes release that docs branch is associated with.)
## Branch structure
The current version of the website is served out of the `master` branch.
All versions of the site that relate to past and future versions will be named after their Kubernetes release number. For example, [the old branch for the 1.1 docs is called `release-1.1`](https://github.com/kubernetes/kubernetes.github.io/tree/release-1.1).
Changes in the "docsv2" branch (where we are testing a revamp of the docs) are automatically staged here:
http://k8sdocs.github.io/docs/tutorials/
Changes in the "release-1.1" branch (for k8s v1.1 docs) are automatically staged here:
http://kubernetes-v1-1.github.io/
Changes in the "release-1.3" branch (for k8s v1.3 docs) are automatically staged here:
http://kubernetes-v1-3.github.io/
Editing of these branches will kick off a build using Travis CI that auto-updates these URLs; you can monitor the build progress at [https://travis-ci.org/kubernetes/kubernetes.github.io](https://travis-ci.org/kubernetes/kubernetes.github.io).
## Config yaml guidelines
Guidelines for config yamls that are included in the site docs. These
are the yaml or json files that contain Kubernetes object
configuration to be used with `kubectl create -f` Config yamls should
be:
* Separate deployable files, not embedded in the document, unless very
small variations of a full config.
* Included in the doc with the include code
[above.](#include-code-from-another-file)
* In the same directory as the doc that they are being used in
* If you are re-using a yaml from another doc, that is OK, just
leave it there, don't move it up to a higher level directory.
section on "Docker images" from the Kubernetes repository.
## Partners
Kubernetes partners refers to the companies who contribute to the Kubernetes core codebase and/or extend their platform to support Kubernetes. Partners can get their logos added to the partner section of the [community page](http://k8s.io/community) by following the below steps and meeting the below logo specifications. Partners will also need to have a URL that is specific to integrating with Kubernetes ready; this URL will be the destination when the logo is clicked.
* The partner product logo should be a transparent png image centered in a 215x125 px frame. (look at the existing logos for reference)
* The logo must link to a URL that is specific to integrating with Kubernetes, hosted on the partner's site.
* The logo should be named *product-name*_logo.png and placed in the `/images/community_logos` folder.
* The image reference (including the link to the partner URL) should be added in `community.html` under `<div class="partner-logos" > ...</div>`.
* Please do not change the order of the existing partner images. Append your logo to the end of the list.
* Once completed and tested the look and feel, submit the pull request.
* [Contributing to the Kubernetes Documentation](http://kubernetes.io/editdocs/)
* [Creating a Documentation Pull Request](http://kubernetes.io/docs/contribute/create-pull-request/)
* [Writing a New Topic](http://kubernetes.io/docs/contribute/write-new-topic/)
* [Staging Your Documentation Changes](http://kubernetes.io/docs/contribute/stage-documentation-changes/)
<!-- Commenting out AnswerDash for now; we need to work on our list of questions/answers/design first
<!-- Start of AnswerDash script <script>var AnswerDash;!function(e,t,n,s,a){if(!t.getElementById(s)){var i,r=t.createElement(n),c=t.getElementsByTagName(n)[0];e[a]||(i=e[a]=function(){i.__oninit.push(arguments)},i.__oninit=[]),r.type="text/javascript",r.async=!0,r.src="https://p1.answerdash.com/answerdash.min.js?siteid=756",r.setAttribute("id",s),c.parentNode.insertBefore(r,c)}}(window,document,"script","answerdash-script","AnswerDash");</script> <!-- End of AnswerDash script -->
<p>Ready to get your hands dirty? Build a simple Kubernetes cluster that runs "Hello World" for Node.js.</p>
</div>
<divclass="nav-box">
<h3><ahref="/docs/">Documentation</a></h3>
<p>Learn how to use Kubernetes with the use of walkthroughs, samples, and reference documentation. You can even <ahref="/editdocs/"data-auto-burger-exclude>help contribute to the docs</a>!</p>
</div>
<divclass="nav-box">
<h3><ahref="/community/">Community</a></h3>
<p>If you need help, you can connect with other Kubernetes users and the Kubernetes authors, attend community events, and watch video presentations from around the web.</p>
<p>Ready to get your hands dirty? Build a simple Kubernetes cluster that runs "Hello World" for Node.js.</p>
</div>
<divclass="nav-box">
<h3><ahref="/docs/">Documentation</a></h3>
<p>Learn how to use Kubernetes with the use of walkthroughs, samples, and reference documentation. You can even <ahref="/editdocs/"data-auto-burger-exclude>help contribute to the docs</a>!</p>
</div>
<divclass="nav-box">
<h3><ahref="/community/">Community</a></h3>
<p>If you need help, you can connect with other Kubernetes users and the Kubernetes authors, attend community events, and watch video presentations from around the web.</p>
blurb:'Tectonic is the enterprise-ready Kubernetes product, by CoreOS. It adds key features to allow you to manage, update, and control clusters in production.'
},
{
type:0,
name:'Deis',
logo:'deis',
link:'https://deis.com',
blurb:'Deis the creators of Helm, Workflow, and Steward, helps developers and operators build, deploy, manage and scale their applications on top of Kubernetes.'
},
{
type:0,
name:'StackPointCloud',
logo:'stackpoint',
link:'https://stackpoint.io',
blurb:'StackPointCloud builds Stackpoint.io, the universal control plane for Kubernetes Anywhere -- compose and build your own infrastructure as easily as a DigitalOcean droplet at any public cloud provider.'
blurb:'The Puppet module for Kubernetes makes it easy to manage Pods, Replication Controllers, Services and more in Kubernetes, and to build domain-specific interfaces to one\'s Kubernetes configuration.'
blurb:'CockroachDB is a distributed SQL database whose built-in replication and survivability model pair with Kubernetes to truly make data easy.'
},
{
type:0,
name:'Skippbox',
logo:'skippbox',
link:'http://www.skippbox.com/tag/products/',
blurb:'Creator of Cabin the first mobile application for Kubernetes, and kompose. Skippbox’s solutions distill all the power of k8s in simple easy to use interfaces.'
},
{
type:0,
name:'Weave Works',
logo:'weave_works',
link:' https://weave.works/kubernetes',
blurb:'Weaveworks enables Developers and Dev/Ops teams to easily connect, deploy, secure, manage, and troubleshoot microservices in Kubernetes.'
blurb:'Wercker automates your build, test and deploy pipelines for launching containers and triggering rolling updates on your Kubernetes cluster. '
},
{
type:0,
name:'Rancher',
logo:'rancher',
link:'http://rancher.com/kubernetes/',
blurb:'Rancher is an open-source, production-ready container management platform that makes it easy to deploy and leverage Kubernetes in the enterprise.'
},
{
type:0,
name:'Red Hat',
logo:'redhat',
link:'https://www.openshift.com/',
blurb:'Leverage an enterprise Kubernetes platform to orchestrate complex, multi-container apps.'
blurb:'AppFormix is a cloud infrastructure performance optimization service helping enterprise operators streamline their cloud operations on any Kubernetes cloud. '
blurb:'The Canonical Distribution of Kubernetes enables you to operate Kubernetes clusters on demand on any major public cloud and private infrastructure.'
},
{
type:0,
name:'Distelli',
logo:'distelli',
link:'https://www.distelli.com/',
blurb:'Pipelines from your source repositories to your Kubernetes Clusters on any cloud.'
blurb:'The Nuage SDN platform provides policy-based networking between Kubernetes Pods and non-Kubernetes environments with visibility and security monitoring.'
},
{
type:0,
name:'Sematext',
logo:'sematext',
link:'https://sematext.com/kubernetes/',
blurb:'Logging & Monitoring: Automatic collection and processing of Metrics, Events and Logs for auto-discovered pods and Kubernetes nodes.'
},
{
type:0,
name:'Diamanti',
logo:'diamanti',
link:'https://www.diamanti.com/products/',
blurb:'Diamanti deploys containers with guaranteed performance using Kubernetes in the first hyperconverged appliance purpose built for containerized applications.'
},
{
type:0,
name:'Apprenda',
logo:'apprenda',
link:'https://apprenda.com/kubernetes-support/',
blurb:'Apprenda creates and supports modern, enterprise-ready application platforms for both cloud native and traditional application workloads.'
},
{
type:0,
name:'Aporeto',
logo:'aporeto',
link:'https://aporeto.com/trireme',
blurb:'Aporeto makes cloud-native applications secure by default without impacting developer velocity and works at any scale, on any cloud.'
},
{
type:0,
name:'Giant Swarm',
logo:'giant_swarm',
link:'https://giantswarm.io',
blurb:'Giant Swarm provides fully-managed Kubernetes Clusters in your location of choice, so you can focus on your product.'
blurb:'Mirantis builds and manages private clouds with open source software such as OpenStack, deployed as containers orchestrated by Kubernetes.'
},
{
type:0,
name:'Kubernetic',
logo:'kubernetic',
link:'https://kubernetic.com/',
blurb:'Kubernetic is a Kubernetes Desktop client that simplifies and democratizes cluster management for DevOps.'
},
{
type:1,
name:'Apprenda',
logo:'apprenda',
link:'https://apprenda.com/kubernetes-support/',
blurb:'Apprenda offers flexible and wide range of support plans for pure play Kubernetes on your choice of infrastructure, cloud provider and operating system.'
},
{
type:1,
name:'Reactive Ops',
logo:'reactive_ops',
link:'https://www.reactiveops.com/kubernetes/',
blurb:'ReactiveOps has written automation on best practices for infrastructure as code on GCP & AWS using Kubernetes, helping you build and maintain a world-class infrastructure at a fraction of the price of an internal hire.'
blurb:'Kubernetes experts that on-board applications and empower IT teams to get the most out of containerised technology.'
},
{
type:1,
name:'Deis',
logo:'deis',
link:'https://deis.com/services/',
blurb:'Deis provides professional services and 24x7 operational support for any Kubernetes cluster managed by our global cluster operations team.'
},
{
type:1,
name:'StackPointCloud',
logo:'stackpoint',
link:'https://stackpoint.io',
blurb:'StackPointCloud offers a wide range of support plans for managed Kubernetes clusters built through its universal control plane for Kubernetes Anywhere.'
blurb:'Samsung SDS’s Cloud Native Computing Team offers expert consulting across the range of technical aspects involved in building services targeted at a Kubernetes cluster.'
blurb:'Container Solutions is a premium software consultancy that focuses on programmable infrastructure, offering our expertise in software development, strategy and operations to help you innovate at speed and scale.'
},
{
type:1,
name:'Jetstack',
logo:'jetstack',
link:'https://www.jetstack.io/',
blurb:'Jetstack is an organisation focused entirely on Kubernetes. They will help you to get the most out of Kubernetes through expert professional services and open source tooling. Get in touch, and accelerate your project.'
blurb:'Tigera builds high performance, policy driven, cloud native networking solutions for Kubernetes.'
},
{
type:1,
name:'Skippbox',
logo:'skippbox',
link:'http://www.skippbox.com/services/',
blurb:'Skippbox brings its Kubernetes expertise to help companies embrace Kubernetes on their way to digital transformation. Skippbox offers both professional services and expert training.'
},
{
type:1,
name:'Harbur',
logo:'harbur',
link:'https://harbur.io/',
blurb:'Based in Barcelona, Harbur is a consulting firm that helps companies deploy self-healing solutions empowered by Container technologies'
},
{
type:1,
name:'Endocode',
logo:'endocode',
link:'https://endocode.com/kubernetes/',
blurb:'Endocode practices and teaches the open source way. Kernel to cluster - Dev to Ops. We offer Kubernetes trainings, services and support.'
blurb:'ETA helps companies architect, implement, and manage scalable applications using Kubernetes on on public or private cloud.'
},
{
type:0,
name:'CloudPlex.io',
logo:'cloudplex',
link:'http://www.cloudplex.io',
blurb:'CloudPlex enables operations teams to visually deploy, orchestrate, manage, and monitor infrastructure, applications, and services in public or private cloud.'
},
{
type:1,
name:'Kumina',
logo:'kumina',
link:'https://www.kumina.nl/managed_kubernetes',
blurb:'Kumina creates Kubernetes solutions on your choice of infrastructure with around-the-clock management and unlimited support.'
{% include templates/_errorthrower.md missing_block='purpose' heading='Purpose' purpose='states, in one sentence, what the purpose of this document is, so that the user will know what they are able to achieve if they follow the provided steps.' %}
{% endif %}
{% if recommended_background %}
### Recommended background
{{ recommended_background }}
{% else %}
{% include templates/_errorthrower.md missing_block='recommended_background' heading='Recommended background' purpose='lists assumptions of baseline knowledge that you expect the user to have before reading ahead.' %}
{% include templates/_errorthrower.md missing_block='overview' purpose='states, in one or two sentences, the purpose of this document' %}
{% endif %}
{% if step_by_step %}
* TOC
{: toc}
### Step by step
{{ step_by_step }}
{% if prerequisites %}
## Before you begin
{{ prerequisites }}
{% else %}
{% include templates/_errorthrower.md missing_block='step_by_step' heading='Step by step' purpose='lists a series of linear, numbered steps that accomplish the described task.' %}
{% include templates/_errorthrower.md missing_block='prerequisites' heading='Before you begin' purpose='lists action prerequisites and knowledge prerequisites' %}
{% endif %}
{% endif %}
{% if steps %}
{{ steps }}
{% else %}
{% include templates/_errorthrower.md missing_block='steps' purpose='lists a sequence of numbered steps that accomplish the task.' %}
{% include templates/_errorthrower.md missing_block='overview' purpose='states, in one or two sentences, the purpose of this document' %}
{% endif %}
* TOC
{: toc}
{% if objectives %}
## Objectives
{{ objectives }}
{% else %}
{% include templates/_errorthrower.md missing_block='objectives' heading='Objectives' purpose='lists the objectives for this tutorial.' %}
{% endif %}
{% if prerequisites %}
## Before you begin
{{ prerequisites }}
{% else %}
{% include templates/_errorthrower.md missing_block='prerequisites' heading='Before you begin' purpose='lists action prerequisites and knowledge prerequisites' %}
{% endif %}
{% if lessoncontent %}
{{ lessoncontent }}
{% else %}
{% include templates/_errorthrower.md missing_block='lessoncontent' purpose='provides the lesson content for this tutorial.' %}
<p>As of March 14, 2017, the Kubernetes SIG-Docs-Maintainers group have begun migration of the User Guide content as announced previously to the <a href="https://git.k8s.io/community/sig-docs">SIG Docs community</a> through the <a href="https://groups.google.com/forum/#!forum/kubernetes-sig-docs">kubernetes-sig-docs</a> group and <a href="https://kubernetes.slack.com/messages/sig-docs/">kubernetes.slack.com #sig-docs</a> channel.</p>
<p>The user guides within this section are being refactored into topics within Tutorials, Tasks, and Concepts. Anything that has been moved will have a notice placed in its previous location as well as a link to its new location. The reorganization implements a new table of contents and should improve the documentation's findability and readability for a wider range of audiences.</p>
<p>For any questions, please contact: <a href="mailto:kubernetes-sig-docs@googlegroups.com">kubernetes-sig-docs@googlegroups.com</a></p>
<p>FlexVolume represents a generic volume resource that is provisioned/attached using a exec based plugin. This is an alpha feature and may change in future.</p>
<p>FlexVolume represents a generic volume resource that is provisioned/attached using an exec based plugin. This is an alpha feature and may change in future.</p>
<tdclass="tableblock halign-left valign-top"><pclass="tableblock">FlexVolume represents a generic volume resource that is provisioned/attached using a exec based plugin. This is an alpha feature and may change in future.</p></td>
<tdclass="tableblock halign-left valign-top"><pclass="tableblock">FlexVolume represents a generic volume resource that is provisioned/attached using an exec based plugin. This is an alpha feature and may change in future.</p></td>
<tdclass="tableblock halign-left valign-top"><pclass="tableblock">Host is the fully qualified domain name of a network host, as defined by RFC 3986. Note the following deviations from the "host" part of the URI as defined in the RFC: 1. IPs are not allowed. Currently an IngressRuleValue can only apply to the<br>
IP in the Spec of the parent Ingress.<br>
IP in the Spec of the parent Ingress.<br>
2. The <code>:</code> delimiter is not respected because ports are not allowed.<br>
Currently the port of an Ingress is implicitly :80 for http and<br>
:443 for https.<br>
Currently the port of an Ingress is implicitly :80 for http and<br>
:443 for https.<br>
Both these may change in the future. Incoming requests are matched against the host before the IngressRuleValue. If the host is unspecified, the Ingress routes all traffic based on the specified IngressRuleValue.</p></td>
<tdclass="tableblock halign-left valign-top"><pclass="tableblock">Path is a extended POSIX regex as defined by IEEE Std 1003.1, (i.e this follows the egrep/unix syntax, not the perl syntax) matched against the path of an incoming request. Currently it can contain characters disallowed from the conventional "path" part of a URL as defined by RFC 3986. Paths must begin with a <em>/</em>. If unspecified, the path defaults to a catch all sending traffic to the backend.</p></td>
<tdclass="tableblock halign-left valign-top"><pclass="tableblock">Path is an extended POSIX regex as defined by IEEE Std 1003.1, (i.e this follows the egrep/unix syntax, not the perl syntax) matched against the path of an incoming request. Currently it can contain characters disallowed from the conventional "path" part of a URL as defined by RFC 3986. Paths must begin with a <em>/</em>. If unspecified, the path defaults to a catch all sending traffic to the backend.</p></td>
<p>FlexVolume represents a generic volume resource that is provisioned/attached using a exec based plugin. This is an alpha feature and may change in future.</p>
<p>FlexVolume represents a generic volume resource that is provisioned/attached using an exec based plugin. This is an alpha feature and may change in future.</p>
<tdclass="tableblock halign-left valign-top"><pclass="tableblock">FlexVolume represents a generic volume resource that is provisioned/attached using a exec based plugin. This is an alpha feature and may change in future.</p></td>
<tdclass="tableblock halign-left valign-top"><pclass="tableblock">FlexVolume represents a generic volume resource that is provisioned/attached using an exec based plugin. This is an alpha feature and may change in future.</p></td>
<tdclass="tableblock halign-left valign-top"><pclass="tableblock">FlexVolume represents a generic volume resource that is provisioned/attached using a exec based plugin. This is an alpha feature and may change in future.</p></td>
<tdclass="tableblock halign-left valign-top"><pclass="tableblock">FlexVolume represents a generic volume resource that is provisioned/attached using an exec based plugin. This is an alpha feature and may change in future.</p></td>
<tdclass="tableblock halign-left valign-top"><pclass="tableblock">Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: <ahref="http://releases.k8s.io/release-1.4/docs/devel/api-conventions.md#types-kinds">http://releases.k8s.io/release-1.4/docs/devel/api-conventions.md#types-kinds</a></p></td>
<tdclass="tableblock halign-left valign-top"><pclass="tableblock">APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: <ahref="http://releases.k8s.io/release-1.4/docs/devel/api-conventions.md#resources">http://releases.k8s.io/release-1.4/docs/devel/api-conventions.md#resources</a></p></td>
<tdclass="tableblock halign-left valign-top"><pclass="tableblock">Parameters holds the parameters for the provisioner that should create volumes of this storage class.</p></td>
<tdclass="tableblock halign-left valign-top"><pclass="tableblock">Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: <ahref="http://releases.k8s.io/release-1.4/docs/devel/api-conventions.md#types-kinds">http://releases.k8s.io/release-1.4/docs/devel/api-conventions.md#types-kinds</a></p></td>
<tdclass="tableblock halign-left valign-top"><pclass="tableblock">APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: <ahref="http://releases.k8s.io/release-1.4/docs/devel/api-conventions.md#resources">http://releases.k8s.io/release-1.4/docs/devel/api-conventions.md#resources</a></p></td>
<tdclass="tableblock halign-left valign-top"><pclass="tableblock">Standard list metadata More info: <ahref="http://releases.k8s.io/release-1.4/docs/devel/api-conventions.md#metadata">http://releases.k8s.io/release-1.4/docs/devel/api-conventions.md#metadata</a></p></td>
<tdclass="tableblock halign-left valign-top"><pclass="tableblock">Host is the fully qualified domain name of a network host, as defined by RFC 3986. Note the following deviations from the "host" part of the URI as defined in the RFC: 1. IPs are not allowed. Currently an IngressRuleValue can only apply to the<br>
IP in the Spec of the parent Ingress.<br>
IP in the Spec of the parent Ingress.<br>
2. The <code>:</code> delimiter is not respected because ports are not allowed.<br>
Currently the port of an Ingress is implicitly :80 for http and<br>
:443 for https.<br>
Currently the port of an Ingress is implicitly :80 for http and<br>
:443 for https.<br>
Both these may change in the future. Incoming requests are matched against the host before the IngressRuleValue. If the host is unspecified, the Ingress routes all traffic based on the specified IngressRuleValue.</p></td>
<tdclass="tableblock halign-left valign-top"><pclass="tableblock">Path is an extended POSIX regex as defined by IEEE Std 1003.1, (i.e this follows the egrep/unix syntax, not the perl syntax) matched against the path of an incoming request. Currently it can contain characters disallowed from the conventional "path" part of a URL as defined by RFC 3986. Paths must begin with a <em>/</em>. If unspecified, the path defaults to a catch all sending traffic to the backend.</p></td>
<tdclass="tableblock halign-left valign-top"><pclass="tableblock">Path is an extended POSIX regex as defined by IEEE Std 1003.1, (i.e. this follows the egrep/unix syntax, not the perl syntax) matched against the path of an incoming request. Currently it can contain characters disallowed from the conventional "path" part of a URL as defined by RFC 3986. Paths must begin with a <em>/</em>. If unspecified, the path defaults to a catch all sending traffic to the backend.</p></td>
@@ -6391,7 +6391,7 @@ The resulting set of endpoints can be viewed as:<br>
<divclass="sect2">
<h3id="_v1alpha1_eviction">v1alpha1.Eviction</h3>
<divclass="paragraph">
<p>Eviction evicts a pod from its node subject to certain policies and safety constraints. This is a subresource of Pod. A request to cause such an eviction is created by POSTing to …/pods/foo/evictions.</p>
<p>Eviction evicts a pod from its node subject to certain policies and safety constraints. This is a subresource of Pod. A request to cause such an eviction is created by POSTing to …/pods/<pod name>/evictions.</p>
{% for thistoc in site.data.globals.tocs %}{% if foundTOC %}{% break %}{% else %}{% assign tree = site.data[thistoc].toc %}{% include tocsearch.html %}{% endif %}{% endfor %}
{% for override in site.data.overrides.overrides %}{% if page.path contains override.path %}{% assign notitle = "true" %}{% endif %}{% endfor %}
<!-- Commenting out AnswerDash for now; we need to work on our list of questions/answers/design first
<!-- Start of AnswerDash script <script>var AnswerDash;!function(e,t,n,s,a){if(!t.getElementById(s)){var i,r=t.createElement(n),c=t.getElementsByTagName(n)[0];e[a]||(i=e[a]=function(){i.__oninit.push(arguments)},i.__oninit=[]),r.type="text/javascript",r.async=!0,r.src="https://p1.answerdash.com/answerdash.min.js?siteid=756",r.setAttribute("id",s),c.parentNode.insertBefore(r,c)}}(window,document,"script","answerdash-script","AnswerDash");</script> <!-- End of AnswerDash script -->
<pclass="quote">"With Kubernetes, we're simplifying our environment and making it easier for developers to build the tools that make wikis run better."</p>
<atarget="_blank"href="https://docs.google.com/a/google.com/forms/d/e/1FAIpQLScuI7Ye3VQHQTwBASrgkjQDSS5TP0g3AXfFhwSM9YpHgxRKFA/viewform"class="tell-your-story"><imgsrc="/images/case_studies/story.png"alt="Tell your story"></a>
<h3id="caseStudyTitle">Using Kubernetes to reinvent the world's largest educational company</h3>
<p>
Pearson, the world's education company, serving 75 million learners worldwide, set a goal to more than double that number to 200 million by 2025. A key part of this growth is in digital learning experiences, and that requires an infrastructure platform that is able to scale quickly and deliver products to market faster. So Pearson's Cloud Technology team chose Kubernetes to help build a platform to meet the business requirements. </p>
"To transform our infrastructure, we had to think beyond simply enabling automated provisioning, we realized we had to build a platform that would allow Pearson developers to build manage and deploy applications in a completely different way. We chose Kubernetes because of its flexibility, ease of management and the way it would improve our engineers' productivity." </p>
<pclass="attrib">— Chris Jackson, Director for Cloud Product Engineering, Pearson</p>
</div>
</div>
</main>
</section>
<sectionclass="bullets">
<main>
<divclass="content">
<divclass="bullet">
<h4>Challenges:</h4>
<ul>
<li>Pearson had difficulty in scaling and adapting to the growing online audience. They wanted to build and deliver content primarily over the web.</li>
</ul>
</div>
<divclass="bullet">
<h4>Why Kubernetes:</h4>
<ul>
<li>Kubernetes will allow Pearson's teams to develop their apps in a consistent manner, saving time and minimizing complexity.</li>
</ul>
</div>
<divclass="bullet">
<h4>Approach:</h4>
<ul>
<li>Build a centralized platform for use across the entire enterprise</li>
<li>Use container technology as the core of the platform</li>
<li>Deploy Kubernetes to manage the platform</li>
</ul>
</div>
<divclass="bullet">
<h4>Results:</h4>
<ul>
<li>Pearson is building an enterprise-wide platform for delivering innovative, web-based educational content. They expect engineers' productivity to increase by up to 20 percent.</li>
</ul>
</div>
</div>
</main>
</section>
<sectionclass="details">
<main>
<divclass="content">
<h4>Kubernetes powers a comprehensive developer experience</h4>
<p>Pearson wanted to use as much open source technology as possible for the platform given that it provides both technical and commercial benefits over the duration of the project. Jackson says, "Building an infrastructure platform based on open source technology in Pearson was a no-brainer, the sharing of technical challenges and advanced use cases in a community of people with talent far beyond what we could hire independently allows us to innovate at a level we could not reach on our own. Our engineers enjoy returning code to the community and participating in talks, blogs and meetings, it's a great way for us to allow our team to express themselves and share the pride they have in their work."</p>
<p>It also wanted to use a container-focused platform. Pearson has 400 development groups and diverse brands with varying business and technical needs. With containers, each brand could experiment with building new types of content using their preferred technologies, and then deliver it using containers. Pearson chose Kubernetes because it believes that is the best technology for managing containers, has the widest community support and offers the most flexible and powerful tools."</p>
<p>Kubernetes is at the core of the platform we've built for developers. After we get our big spike in back-to-school in traffic, much of Pearson's traffic will interact with Kubernetes. It is proving to be as effective as we had hoped," Jackson says.</p>
<p>With the new platform, Pearson will increase stability and performance, and to bring products to market more quickly. The company says its engineers will also get a productivity boost because they won't spend time managing infrastructure. Jackson estimates 15 to 20 percent in productivity savings.</p>
<p>Beyond that, Pearson says the platform will encourage innovation because of the ease with which new applications can be developed, and because applications will be deployed far more quickly than in the past. It expects that will help the company meet its goal of reaching 200 million learners within the next 10 years.</p>
<p>"We're already seeing tremendous benefits with Kubernetes — improved engineering productivity, faster delivery of applications and a simplified infrastructure. But this is just the beginning. Kubernetes will help transform the way that educational content is delivered online," says Jackson.</p>
<h3id="caseStudyTitle">Using Kubernetes to Build Tools to Improve the World's Wikis</h3>
<p>
The non-profit Wikimedia Foundation operates some of the largest collaboratively edited reference projects in the world, including Wikipedia. To help users maintain and use wikis, it runs Wikimedia Tool Labs, a hosting environment for community developers working on tools and bots to help editors and other volunteers do their work, including reducing vandalism. The community around Wikimedia Tool Labs began forming nearly 10 years ago.
"Wikimedia Tool Labs is vital for making sure wikis all around the world work as well as they possibly can. Because it's grown organically for almost 10 years, it has become an extremely challenging environment and difficult to maintain. It's like a big ball of mud — you really can't see through it. With Kubernetes, we're simplifying the environment and making it easier for developers to build the tools that make wikis run better."
</p>
<pclass="attrib">— Yuvi Panda, operations engineer at Wikimedia Foundation and Wikimedia Tool Labs</p>
</div>
</div>
</main>
</section>
<sectionclass="bullets">
<main>
<divclass="content">
<divclass="bullet">
<h4>Challenges:</h4>
<ul>
<li>Simplify a complex, difficult-to-manage infrastructure</li>
<li>Allow developers to continue writing tools and bots using existing techniques</li>
</ul>
</div>
<divclass="bullet">
<h4>Why Kubernetes:</h4>
<ul>
<li>Wikimedia Tool Labs chose Kubernetes because it can mimic existing workflows, while reducing complexity</li>
</ul>
</div>
<divclass="bullet">
<h4>Approach:</h4>
<ul>
<li>Migrate old systems and a complex infrastructure to Kubernetes</li>
</ul>
</div>
<divclass="bullet">
<h4>Results:</h4>
<ul>
<li>20 percent of web tools that account for more than 40 percent of web traffic now run on Kubernetes</li>
<li>A 25-node cluster that keeps up with each new Kubernetes release</li>
<li>Thousands of lines of old code have been deleted, thanks to Kubernetes</li>
</ul>
</div>
</div>
</main>
</section>
<sectionclass="details">
<main>
<divclass="content">
<h4>Using Kubernetes to provide tools for maintaining wikis</h4>
<p>
Wikimedia Tool Labs is run by a staff of four-and-a-half paid employees and two volunteers. The infrastructure didn't make it easy or intuitive for developers to build bots and other tools to make wikis work more easily. Yuvi says, "It's incredibly chaotic. We have lots of Perl and Bash duct tape on top of it. Everything is super fragile."
</p>
<p>
To solve the problem, Wikimedia Tool Labs migrated parts of its infrastructure to Kubernetes, in preparation for eventually moving its entire system. Yuvi said Kubernetes greatly simplifies maintenance. The goal is to allow developers creating bots and other tools to use whatever development methods they want, but make it easier for the Wikimedia Tool Labs to maintain the required infrastructure for hosting and sharing them.
</p>
<p>
"With Kubernetes, I've been able to remove a lot of our custom-made code, which makes everything easier to maintain. Our users' code also runs in a more stable way than previously," says Yuvi.
</p>
</div>
</main>
</section>
<sectionclass="details">
<main>
<divclass="content">
<h4>Simplifying infrastructure and keeping wikis running better</h4>
<p>
Wikimedia Tool Labs has seen great success with the initial Kubernetes deployment. Old code is being simplified and eliminated, contributing developers don't have to change the way they write their tools and bots, and those tools and bots run in a more stable fashion than they have in the past. The paid staff and volunteers are able to better keep up with fixing issues.
</p>
<p>
In the future, with a more complete migration to Kubernetes, Wikimedia Tool Labs expects to make it even easier to host and maintain the bots and tools that help run wikis across the world. The tool labs already host approximately 1,300 tools and bots from 800 volunteers, with many more being submitted every day. Twenty percent of the tool labs' web tools that account for more than 60 percent of web traffic now run on Kubernetes. The tool labs has a 25-node cluster that keeps up with each new Kubernetes release. Many existing web tools are migrating to Kubernetes.
</p>
<p>
"Our goal is to make sure that people all over the world can share knowledge as easily as possible. Kubernetes helps with that, by making it easier for wikis everywhere to have the tools they need to thrive," says Yuvi.
<p>We are working with a broad group of partners who contribute to the kubernetes core codebase, making it stronger and richer, as well as help in growing the kubernetes ecosystem supporting
a sprectrum of compelmenting platforms, from open source solutions to market-leading technologies.</p>
@@ -7,16 +8,21 @@ Add-ons extend the functionality of Kubernetes.
This page lists some of the available add-ons and links to their respective installation instructions.
Add-ons in each section are sorted alphabetically - the ordering does not imply any preferential status.
## Networking and Network Policy
* [Weave Net](https://github.com/weaveworks/weave-kube) provides networking and network policy, will carry on working on both sides of a network partition, and does not require an external database.
* [Calico](https://github.com/projectcalico/calico-containers/tree/master/docs/cni/kubernetes/manifests/kubeadm) is a secure L3 networking and network policy provider.
* [Calico](http://docs.projectcalico.org/v2.0/getting-started/kubernetes/installation/hosted/) is a secure L3 networking and network policy provider.
* [Canal](https://github.com/tigera/canal/tree/master/k8s-install/kubeadm) unites Flannel and Calico, providing networking and network policy.
* [Contiv](http://contiv.github.io) provides configurable networking (native L3 using BGP, overlay using vxlan, classic L2, and Cisco-SDN/ACI) for various use cases and a rich policy framework. Contiv project is fully [open sourced](http://github.com/contiv). The [installer](http://github.com/contiv/install) provides both kubeadm and non-kubeadm based installation options.
* [Flannel](https://github.com/coreos/flannel/blob/master/Documentation/kube-flannel.yml) is an overlay network provider that can be used with Kubernetes.
* [Romana](http://romana.io) is a Layer 3 networking solution for pod networks that also supports the [NetworkPolicy API](/docs/user-guide/networkpolicies/). Kubeadm add-on installation details available [here](https://github.com/romana/romana/tree/master/containerize).
* [Weave Net](https://www.weave.works/docs/net/latest/kube-addon/) provides networking and network policy, will carry on working on both sides of a network partition, and does not require an external database.
## Visualization & Control
* [Weave Scope](https://www.weave.works/documentation/scope-latest-installing/#k8s) is a tool for graphically visualizing your containers, pods, services etc. Use it in conjunction with a [Weave Cloud account](https://cloud.weave.works/) or host the UI yourself.
* [Dashboard](https://github.com/kubernetes/dashboard#kubernetes-dashboard) is a dashboard web interface for Kubernetes.
* [Weave Scope](https://www.weave.works/documentation/scope-latest-installing/#k8s) is a tool for graphically visualizing your containers, pods, services etc. Use it in conjunction with a [Weave Cloud account](https://cloud.weave.works/) or host the UI yourself.
@@ -87,7 +87,7 @@ The ImagePolicyWebhook plug-in allows a backend webhook to make admission decisi
```
#### Configuration File Format
ImagePolicyWebhook uses the admission controller config file (`--admission-controller-config-file`) to set configuration options for the behavior of the backend. This file may be json or yaml and has the following format:
ImagePolicyWebhook uses the admission config file `--admission-controller-config-file` to set configuration options for the behavior of the backend. This file may be json or yaml and has the following format:
The config file must reference a [kubeconfig](/docs/user-guide/kubeconfig-file/) formatted file which sets up the connection to the backend. It is required that the backend communicate over TLS.
The config file must reference a [kubeconfig](/docs/concepts/cluster-administration/authenticate-across-clusters-kubeconfig/) formatted file which sets up the connection to the backend. It is required that the backend communicate over TLS.
The kubeconfig file's cluster field must point to the remote service, and the user field must contain the returned authorizer.
@@ -120,13 +120,13 @@ users:
client-certificate:/path/to/cert.pem# cert for the webhook plugin to use
client-key:/path/to/key.pem # key matching the cert
```
For additional HTTP configuration, refer to the [kubeconfig](/docs/user-guide/kubeconfig-file/) documentation.
For additional HTTP configuration, refer to the [kubeconfig](/docs/concepts/cluster-administration/authenticate-across-clusters-kubeconfig/) documentation.
#### Request Payloads
When faced with an admission decision, the API Server POSTs a JSON serialized api.imagepolicy.v1alpha1.ImageReview object describing the action. This object contains fields describing the containers being admitted, as well as any pod annotations that match `*.image-policy.k8s.io/*`.
Note that webhook API objects are subject to the same versioning compatibility rules as other Kubernetes API objects. Implementers should be aware of looser compatibility promises for alpha objects and check the “apiVersion” field of the request to ensure correct deserialization. Additionally, the API Server must enable the imagepolicy.k8s.io/v1alpha1 API extensions group (`--runtime-config=imagepolicy.k8s.io/v1alpha1=true`).
Note that webhook API objects are subject to the same versioning compatibility rules as other Kubernetes API objects. Implementers should be aware of looser compatibility promises for alpha objects and check the "apiVersion" field of the request to ensure correct deserialization. Additionally, the API Server must enable the imagepolicy.k8s.io/v1alpha1 API extensions group (`--runtime-config=imagepolicy.k8s.io/v1alpha1=true`).
An example request body:
@@ -151,7 +151,7 @@ An example request body:
}
```
The remote service is expected to fill the ImageReviewStatus field of the request and respond to either allow or disallow access. The response body’s “spec” field is ignored and may be omitted. A permissive response would return:
The remote service is expected to fill the ImageReviewStatus field of the request and respond to either allow or disallow access. The response body's "spec" field is ignored and may be omitted. A permissive response would return:
```
{
@@ -252,6 +252,11 @@ This plugin ignores any `PersistentVolumeClaim` updates, it acts only on creatio
See [persistent volume](/docs/user-guide/persistent-volumes) documentation about persistent volume claims and
storage classes and how to mark a storage class as default.
### DefaultTolerationSeconds
This plug-in sets the default forgiveness toleration for pods, which have no forgiveness tolerations, to tolerate
the taints `notready:NoExecute` and `unreachable:NoExecute` for 5 minutes.
Where `<container_name>` is the name of the container to apply the profile to, and `<profile_ref>`
specifies the profile to apply. The `profile_ref` can be one of:
-`runtime/default` to apply the runtime's default profile.
-`localhost/<profile_name>` to apply the profile loaded on the host with the name `<profile_name>`
See the [API Reference](#api-reference) for the full details on the annotation and profile name formats.
The Kubernetes AppArmor enforcement works by first checking that all the prerequisites have been
met, and then forwarding the profile selection to the container runtime for enforcement. If the
prerequisites have not been met, the Pod will be rejected, and will not run.
To verify that the profile was applied, you can expect to see the AppArmor security option listed in the container created event:
$ kubectl get events | grep Created
22s 22s 1 hello-apparmor Pod spec.containers{hello} Normal Created {kubelet e2e-test-stclair-minion-group-31nt} Created container with docker id 269a53b202d3; Security:[seccomp=unconfined apparmor=k8s-apparmor-example-deny-write]
You can also verify directly that the container's root process is running with the correct profile by checking its proc attr:
If we look at the pod events, we can see that the Pod container was created with the AppArmor
profile "k8s-apparmor-example-deny-write":
$ kubectl get events | grep hello-apparmor
14s 14s 1 hello-apparmor Pod Normal Scheduled {default-scheduler } Successfully assigned hello-apparmor to gke-test-default-pool-239f5d02-gyn2
14s 14s 1 hello-apparmor Pod spec.containers{hello} Normal Pulling {kubelet gke-test-default-pool-239f5d02-gyn2} pulling image "busybox"
13s 13s 1 hello-apparmor Pod spec.containers{hello} Normal Pulled {kubelet gke-test-default-pool-239f5d02-gyn2} Successfully pulled image "busybox"
13s 13s 1 hello-apparmor Pod spec.containers{hello} Normal Created {kubelet gke-test-default-pool-239f5d02-gyn2} Created container with docker id 06b6cd1c0989; Security:[seccomp=unconfined apparmor=k8s-apparmor-example-deny-write]
13s 13s 1 hello-apparmor Pod spec.containers{hello} Normal Started {kubelet gke-test-default-pool-239f5d02-gyn2} Started container with docker id 06b6cd1c0989
We can verify that the container is actually running with that profile by checking its proc attr:
This would create a CSR for the username "jbeda", belonging to two groups, "app1" and "app2".
See [APPENDIX](#appendix) for how to generate a client cert.
### Static Token File
Token file is enabled by passing the `--token-auth-file=SOMEFILE` option to the
API server. Currently, tokens last indefinitely, and the token list cannot be
The API server reads bearer tokens from a file when given the `--token-auth-file=SOMEFILE` option on the command line. Currently, tokens last indefinitely, and the token list cannot be
changed without restarting API server.
The token file format is implemented in `plugin/pkg/auth/authenticator/token/tokenfile/...`
and is a csv file with a minimum of 3 columns: token, user name, user uid, followed by
optional group names. Note, if you have more than one group the column must be double quoted e.g.
The token file is a csv file with a minimum of 3 columns: token, user name, user uid,
followed by optional group names. Note, if you have more than one group the column must be
double quoted e.g.
```conf
token,user,uid,"group1,group2,group3"
```
When using token authentication from an http client the API server expects an `Authorization`
header with a value of `Bearer SOMETOKEN`.
#### Putting a Bearer Token in a Request
When using bearer token authentication from an http client, the API
server expects an `Authorization` header with a value of `Bearer
THETOKEN`. The bearer token must be a character sequence that can be
put in an HTTP header value using no more than the encoding and
quoting facilities of HTTP. For example: if the bearer token is
`31ada4fd-adec-460c-809a-9e56ceb75269` then it would appear in an HTTP
2. Your identity provider will provide you with an `access_token`, `id_token` and a `refresh_token`
3. When using `kubectl`, use your `id_token` with the `--token` flag or add it directly to your `kubeconfig`
4. `kubectl` sends your `id_token` in a header called Authorization to the API server
5. The API server will make sure the JWT signature is valid by checking against the certificate named in the configuration
6. Check to make sure the `id_token` hasn't expired
7. Make sure the user is authorized
8. Once authorized the API server returns a response to `kubectl`
9. `kubectl` provides feedback to the user
Since all of the data needed to validate who you are is in the `id_token`, Kubernetes doesn't need to
"phone home" to the identity provider. In a model where every request is stateless this provides a very scalable
solution for authentication. It does offer a few challenges:
1. Kubernetes has no "web interface" to trigger the authentication process. There is no browser or interface to collect credentials which is why you need to authenticate to your identity provider first.
2. The `id_token` can't be revoked, its like a certificate so it should be short-lived (only a few minutes) so it can be very annoying to have to get a new token every few minutes
3. There's no easy way to authenticate to the Kubernetes dashboard without using the `kubectl -proxy` command or a reverse proxy that injects the `id_token`
#### Configuring the API Server
To enable the plugin, configure the following flags on the API server:
| Parameter | Description | Example | Required |
| --------- | ----------- | ------- | ------- |
| `--oidc-issuer-url` | URL of the provider which allows the API server to discover public signing keys. Only URLs which use the `https://` scheme are accepted. This is typically the provider's discovery URL without a path, for example "https://accounts.google.com" or "https://login.salesforce.com". This URL should point to the level below .well-known/openid-configuration | If the discovery URL is https://accounts.google.com/.well-known/openid-configuration the value should be https://accounts.google.com | Yes |
| `--oidc-client-id` | A client id that all tokens must be issued for. | kubernetes | Yes |
| `--oidc-username-claim` | JWT claim to use as the user name. By default `sub`, which is expected to be a unique identifier of the end user. Admins can choose other claims, such as `email` or `name`, depending on their provider. However, claims other than `email` will be prefixed with the issuer URL to prevent naming clashes with other plugins. | sub | No |
| `--oidc-groups-claim` | JWT claim to use as the user's group. If the claim is present it must be an array of strings. | groups | No |
| `--oidc-ca-file` | The path to the certificate for the CA that signed your identity provider's web certificate. Defaults to the host's root CAs. | `/etc/kubernetes/ssl/kc-ca.pem` | No |
If a claim other than `email` is chosen for `--oidc-username-claim`, the value
will be prefixed with the `--oidc-issuer-url` to prevent clashes with existing
Kubernetes names (such as the `system:` users). For example, if the provider
URL is `https://accounts.google.com` and the username claim maps to `jane`, the
plugin will authenticate the user as:
```
Authentication: Bearer (id_token)
https://accounts.google.com#jane
```
To enable the plugin, pass the following required flags:
*`--oidc-issuer-url` URL of the provider which allows the API server to discover
public signing keys. Only URLs which use the `https://` scheme are accepted. This is typically
the provider's URL without a path, for example "https://accounts.google.com" or "https://login.salesforce.com".
*`--oidc-client-id` A client id that all tokens must be issued for.
Importantly, the API server is not an OAuth2 client, rather it can only be
configured to trust a single client. This allows the use of public providers,
configured to trust a single issuer. This allows the use of public providers,
such as Google, without trusting credentials issued to third parties. Admins who
wish utilize multiple OAuth clients should explore providers which support the
wish to utilize multiple OAuth clients should explore providers which support the
`azp` (authorized party) claim, a mechanism for allowing one client to issue
tokens on behalf of another.
The plugin also accepts the following optional flags:
*`--oidc-ca-file` Used by the API server to establish and verify the secure
connection to the issuer. Defaults to the host's root CAs.
And experimental flags:
*`--oidc-username-claim` JWT claim to use as the user name. By default `sub`,
which is expected to be a unique identifier of the end user. Admins can choose
other claims, such as `email`, depending on their provider.
*`--oidc-groups-claim` JWT claim to use as the user's group. If the claim is present
it must be an array of strings.
Kubernetes does not provide an OpenID Connect Identity Provider.
You can use an existing public OpenID Connect Identity Provider (such as Google, or [others](http://connect2id.com/products/nimbus-oauth-openid-connect-sdk/openid-connect-providers)).
Or, you can run your own Identity Provider, such as CoreOS [dex](https://github.com/coreos/dex), [Keycloak](https://github.com/keycloak/keycloak) or CloudFoundary [UAA](https://github.com/cloudfoundry/uaa).
Or, you can run your own Identity Provider, such as CoreOS [dex](https://github.com/coreos/dex), [Keycloak](https://github.com/keycloak/keycloak), CloudFoundry [UAA](https://github.com/cloudfoundry/uaa), or Tremolo Security's [OpenUnison](https://github.com/tremolosecurity/openunison).
The provider needs to support [OpenID connect discovery](https://openid.net/specs/openid-connect-discovery-1_0.html); not all do.
For an identity provider to work with Kubernetes it must:
1. Support [OpenID connect discovery](https://openid.net/specs/openid-connect-discovery-1_0.html); not all do.
2. Run in TLS with non-obsolete ciphers
3. Have a CA signed certificate (even if the CA is not a commercial CA or is self signed)
A note about requirement #3 above, requiring a CA signed certificate. If you deploy your own identity provider (as opposed to one of the cloud providers like Google or Microsoft) you MUST have your identity provider's web server certificate signed by a certificate with the `CA` flag set to `TRUE`, even if it is self signed. This is due to GoLang's TLS client implementation being very strict to the standards around certificate validation. If you don't have a CA handy, you can use this script from the CoreOS team to create a simple CA and a signed certificate and key pair - https://github.com/coreos/dex/blob/1ee5920c54f5926d6468d2607c728b71cfe98092/examples/k8s/gencert.sh or this script based on it that will generate SHA256 certs with a longer life and larger key size https://raw.githubusercontent.com/TremoloSecurity/openunison-qs-kubernetes/master/makecerts.sh.
The first option is to use the `oidc` authenticator. This authenticator takes your `id_token`, `refresh_token` and your OIDC `client_secret` and will refresh your token automatically. Once you have authenticated to your identity provider:
Once your `id_token` expires, `kubectl` will attempt to refresh your `id_token` using your `refresh_token` and `client_secret` storing the new values for the `refresh_token` and `id_token` in your `kube/.config`.
##### Option 2 - Use the `--token` Option
The `kubectl` command lets you pass in a token using the `--token` option. Simply copy and paste the `id_token` into this option:
```
kubectl --token=eyJhbGciOiJSUzI1NiJ9.eyJpc3MiOiJodHRwczovL21sYi50cmVtb2xvLmxhbjo4MDQzL2F1dGgvaWRwL29pZGMiLCJhdWQiOiJrdWJlcm5ldGVzIiwiZXhwIjoxNDc0NTk2NjY5LCJqdGkiOiI2RDUzNXoxUEpFNjJOR3QxaWVyYm9RIiwiaWF0IjoxNDc0NTk2MzY5LCJuYmYiOjE0NzQ1OTYyNDksInN1YiI6Im13aW5kdSIsInVzZXJfcm9sZSI6WyJ1c2VycyIsIm5ldy1uYW1lc3BhY2Utdmlld2VyIl0sImVtYWlsIjoibXdpbmR1QG5vbW9yZWplZGkuY29tIn0.f2As579n9VNoaKzoF-dOQGmXkFKf1FMyNV0-va_B63jn-_n9LGSCca_6IVMP8pO-Zb4KvRqGyTP0r3HkHxYy5c81AnIh8ijarruczl-TK_yF5akjSTHFZD-0gRzlevBDiH8Q79NAr-ky0P4iIXS8lY9Vnjch5MF74Zx0c3alKJHJUnnpjIACByfF2SCaYzbWFMUNat-K1PaUk5-ujMBG7yYnr95xD-63n8CO8teGUAAEMx6zRjzfhnhbzX-ajwZLGwGUBT4WqjMs70-6a7_8gZmLZb2az1cZynkFRj2BaCkVT3A2RrjeEwZEtGXlMqKJ1_I2ulrOVsYx01_yD35-rw get nodes
```
### Webhook Token Authentication
@@ -244,7 +359,7 @@ Webhook authentication is a hook for verifying bearer tokens.
* `--authentication-token-webhook-config-file` a kubeconfig file describing how to access the remote webhook service.
* `--authentication-token-webhook-cache-ttl` how long to cache authentication decisions. Defaults to two minutes.
The configuration file uses the [kubeconfig](/docs/user-guide/kubeconfig-file/)
The configuration file uses the [kubeconfig](/docs/concepts/cluster-administration/authenticate-across-clusters-kubeconfig/)
file format. Within the file "users" refers to the API server webhook and
"clusters" refers to the remote service. An example would be:
@@ -272,10 +387,11 @@ contexts:
name: webhook
```
When a client attempts to authenticate with the API server using a bearer token,
using the `Authorization: Bearer (TOKEN)` HTTP header the authentication webhook
When a client attempts to authenticate with the API server using a bearer token
as discussed [above](#putting-a-bearer-token-in-a-request),
the authentication webhook
queries the remote service with a review object containing the token. Kubernetes
will not challenge request that lack such a header.
will not challenge a request that lacks such a header.
Note that webhook API objects are subject to the same [versioning compatibility rules](/docs/api/)
as other Kubernetes API objects. Implementers should be aware of looser
@@ -338,6 +454,20 @@ An unsuccessful request would return:
HTTP status codes can be used to supply additional error context.
### Authenticating Proxy
The API server can be configured to identify users from request header values, such as `X-Remote-User`.
It is designed for use in combination with an authenticating proxy, which sets the request header value.
In order to prevent header spoofing, the authenticating proxy is required to present a valid client
certificate to the API server for validation against the specified CA before the request headers are
checked.
* `--requestheader-username-headers` Required, case-insensitive. Header names to check, in order, for the user identity. The first header containing a value is used as the identity.
* `--requestheader-client-ca-file` Required. PEM-encoded certificate bundle. A valid client certificate must be presented and validated against the certificate authorities in the specified file before the request headers are checked for user names.
* `--requestheader-allowed-names` Optional. List of common names (cn). If set, a valid client certificate with a Common Name (cn) in the specified list must be presented before the request headers are checked for user names. If empty, any Common Name is allowed.
### Keystone Password
Keystone authentication is enabled by passing the `--experimental-keystone-url=<AuthURL>`
@@ -345,6 +475,13 @@ option to the API server during startup. The plugin is implemented in
`plugin/pkg/auth/authenticator/password/keystone/keystone.go` and currently uses
basic auth to verify used by username and password.
If you have configured self-signed certificates for the Keystone server,
you may need to set the `--experimental-keystone-ca-file=SOMEFILE` option when
starting the Kubernetes API server. If you set the option, the Keystone
server's certificate is verified by one of the authorities in the
`experimental-keystone-ca-file`. Otherwise, the certificate is verified by
the host's root Certificate Authority.
For details on how to use keystone to manage projects and users, refer to the
In Kubernetes, authorization happens as a separate step from authentication.
@@ -22,10 +24,9 @@ The following implementations are available, and are selected by flag:
need authorization.
-`--authorization-mode=ABAC` allows for a simple local-file-based user-configured
authorization policy. ABAC stands for Attribute-Based Access Control.
authorization policy.
-`--authorization-mode=RBAC` is an experimental implementation which allows
for authorization to be driven by the Kubernetes API.
RBAC stands for Roles-Based Access Control.
RBAC stands for Role-Based Access Control.
-`--authorization-mode=Webhook` allows for authorization to be driven by a
remote service using REST.
@@ -53,7 +54,7 @@ A request has the following attributes that can be considered for authorization:
- what resource is being accessed (for resource requests only)
- what subresource is being accessed (for resource requests only)
- the namespace of the object being accessed (for namespaced resource requests only)
- the API group being accessed (for resource requests only)
- the API group being accessed (for resource requests only); an empty string designates the [core API group](/docs/api/)
The request verb for a resource API endpoint can be determined by the HTTP verb used and whether or not the request acts on an individual resource or a collection of resources:
@@ -83,17 +84,27 @@ properties:
-`kind`, type string: valid values are "Policy". Allows versioning and conversion of the policy format.
-`spec` property set to a map with the following properties:
- Subject-matching properties:
-`user`, type string; the user-string from `--token-auth-file`. If you specify `user`, it must match the username of the authenticated user.`*` matches all requests.
-`group`, type string; if you specify `group`, it must match one of the groups of the authenticated user. `*` matches all requests.
-`readonly`, type boolean, when true, means that the policy only applies to get, list, and watch operations.
-`user`, type string; the user-string from `--token-auth-file`. If you specify `user`, it must match the username of the authenticated user.
-`group`, type string; if you specify `group`, it must match one of the groups of the authenticated user. `system:authenticated` matches all authenticated requests. `system:unauthenticated` matches all unauthenticated requests.
- Resource-matching properties:
-`apiGroup`, type string; an API group, such as `extensions`. `*` matches all API groups.
-`namespace`, type string; a namespace string. `*` matches all resource requests.
-`resource`, type string; a resource, such as `pods`.`*` matches all resource requests.
-`apiGroup`, type string; an API group.
- Ex: `extensions`
- Wildcard:`*` matches all API groups.
-`namespace`, type string; a namespace.
- Ex: `kube-system`
- Wildcard: `*` matches all resource requests.
-`resource`, type string; a resource type
- Ex: `pods`
- Wildcard: `*` matches all resource requests.
- Non-resource-matching properties:
-`nonResourcePath`, type string; matches the non-resource request paths (like `/version` and `/apis`). `*` matches all non-resource requests. `/foo/*` matches `/foo/` and all of its subpaths.
-`nonResourcePath`, type string; non-resource request paths.
- Ex: `/version` or `/apis`
- Wildcard:
-`*` matches all non-resource requests.
-`/foo/*` matches `/foo/` and all of its subpaths.
-`readonly`, type boolean, when true, means that the policy only applies to get, list, and watch operations.
An unset property is the same as a property set to the zero value for its type
**NOTES:**An unset property is the same as a property set to the zero value for its type
(e.g. empty string, 0, false). However, unset should be preferred for
readability.
@@ -113,8 +124,11 @@ The tuple of attributes is checked for a match against every policy in the
policy file. If at least one line matches the request attributes, then the
request is authorized (but may fail later validation).
To permit any user to do something, write a policy with the user property set to
`"*"`.
To permit any authenticated user to do something, write a policy with the
group property set to `"system:authenticated"`.
To permit any unauthenticated user to do something, write a policy with the
group property set to `"system:unauthenticated"`.
To permit a user to do anything, write a policy with the apiGroup, namespace,
resource, and nonResourcePath properties set to `"*"`.
@@ -163,7 +177,8 @@ up the verbosity:
5. Anyone can make read-only requests to all non-resource paths:
At {{page.version}}, Kubernetes supports clusters with up to 1000 nodes. More specifically, we support configurations that meet *all* of the following criteria:
* No more than 2000 nodes
* No more than 60000 total pods
* No more than 120000 total containers
* No more than 100 pods per node
* TOC
{:toc}
## Setup
A cluster is a set of nodes (physical or virtual machines) running Kubernetes agents, managed by a "master" (the cluster-level control plane).
Normally the number of nodes in a cluster is controlled by the the value `NUM_NODES` in the platform-specific `config-default.sh` file (for example, see [GCE's `config-default.sh`](http://releases.k8s.io/{{page.githubbranch}}/cluster/gce/config-default.sh)).
Simply changing that value to something very large, however, may cause the setup script to fail for many cloud providers. A GCE deployment, for example, will run in to quota issues and fail to bring the cluster up.
When setting up a large Kubernetes cluster, the following issues must be considered.
### Quota Issues
To avoid running into cloud provider quota issues, when creating a cluster with many nodes, consider:
* Increase the quota for things like CPU, IPs, etc.
* In [GCE, for example,](https://cloud.google.com/compute/docs/resource-quotas) you'll want to increase the quota for:
*CPUs
*VM instances
*Total persistent disk reserved
*In-use IP addresses
*Firewall Rules
* Forwarding rules
*Routes
*Target pools
* Gating the setup script so that it brings up new node VMs in smaller batches with waits in between, because some cloud providers rate limit the creation of VMs.
### Etcd storage
To improve performance of large clusters, we store events in a separate dedicated etcd instance.
When creating a cluster, existing salt scripts:
* start and configure additional etcd instance
* configure api-server to use it for storing events
### Size of master and master components
On GCE/GKE and AWS, `kube-up` automatically configures the proper VM size for your master depending on the number of nodes
in your cluster. On other providers, you will need to configure it manually. For reference, the sizes we use on GCE are
* 1-5 nodes: n1-standard-1
*6-10 nodes: n1-standard-2
*11-100 nodes: n1-standard-4
* 101-250 nodes: n1-standard-8
*251-500 nodes: n1-standard-16
*more than 500 nodes: n1-standard-32
And the sizes we use on AWS are
* 1-5 nodes: m3.medium
*6-10 nodes: m3.large
*11-100 nodes: m3.xlarge
* 101-250 nodes: m3.2xlarge
*251-500 nodes: c4.4xlarge
*more than 500 nodes: c4.8xlarge
Note that these master node sizes are currently only set at cluster startup time, and are not adjusted if you later scale your cluster up or down (e.g. manually removing or adding nodes, or using a cluster autoscaler).
### Addon Resources
To prevent memory leaks or other resource issues in [cluster addons](https://releases.k8s.io/{{page.githubbranch}}/cluster/addons) from consuming all the resources available on a node, Kubernetes sets resource limits on addon containers to limit the CPU and Memory resources they can consume (See PR [#10653](http://pr.k8s.io/10653/files) and [#10778](http://pr.k8s.io/10778/files)).
For [example](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/cluster/saltbase/salt/fluentd-gcp/fluentd-gcp.yaml):
```yaml
containers:
- name:fluentd-cloud-logging
image:gcr.io/google_containers/fluentd-gcp:1.16
resources:
limits:
cpu:100m
memory:200Mi
```
Except for Heapster, these limits are static and are based on data we collected from addons running on 4-node clusters (see [#10335](http://issue.k8s.io/10335#issuecomment-117861225)). The addons consume a lot more resources when running on large deployment clusters (see [#5880](http://issue.k8s.io/5880#issuecomment-113984085)). So, if a large cluster is deployed without adjusting these values, the addons may continuously get killed because they keep hitting the limits.
To avoid running into cluster addon resource issues, when creating a cluster with many nodes, consider the following:
* Scale memory and CPU limits for each of the following addons, if used, as you scale up the size of cluster (there is one replica of each handling the entire cluster so memory and CPU usage tends to grow proportionally with size/load on cluster):
*[InfluxDB and Grafana](http://releases.k8s.io/{{page.githubbranch}}/cluster/addons/cluster-monitoring/influxdb/influxdb-grafana-controller.yaml)
* [skydns, kube2sky, and dns etcd](http://releases.k8s.io/{{page.githubbranch}}/cluster/addons/dns/skydns-rc.yaml.in)
*Scale number of replicas for the following addons, if used, along with the size of cluster (there are multiple replicas of each so increasing replicas should help handle increased load, but, since load per replica also increases slightly, also consider increasing CPU/memory limits):
*Increase memory and CPU limits slightly for each of the following addons, if used, along with the size of cluster (there is one replica per node but CPU/memory usage increases slightly along with cluster load/size as well):
*[FluentD with ElasticSearch Plugin](http://releases.k8s.io/{{page.githubbranch}}/cluster/saltbase/salt/fluentd-es/fluentd-es.yaml)
* [FluentD with GCP Plugin](http://releases.k8s.io/{{page.githubbranch}}/cluster/saltbase/salt/fluentd-gcp/fluentd-gcp.yaml)
Heapster's resource limits are set dynamically based on the initial size of your cluster (see [#16185](http://issue.k8s.io/16185)
and [#22940](http://issue.k8s.io/22940)). If you find that Heapster is running
out of resources, you should adjust the formulas that compute heapster memory request (see those PRs for details).
For directions on how to detect if addon containers are hitting resource limits, see the [Troubleshooting section of Compute Resources](/docs/user-guide/compute-resources/#troubleshooting).
In the [future](http://issue.k8s.io/13048), we anticipate to set all cluster addon resource limits based on cluster size, and to dynamically adjust them if you grow or shrink your cluster.
We welcome PRs that implement those features.
### Allowing minor node failure at startup
For various reasons (see [#18969](https://github.com/kubernetes/kubernetes/issues/18969) for more details) running
`kube-up.sh` with a very large `NUM_NODES` may fail due to a very small number of nodes not coming up properly.
Currently you have two choices: restart the cluster (`kube-down.sh` and then `kube-up.sh` again), or before
running `kube-up.sh` set the environment variable `ALLOWED_NOTREADY_NODES` to whatever value you feel comfortable
with. This will allow `kube-up.sh` to succeed with fewer than `NUM_NODES` coming up. Depending on the
reason for the failure, those additional nodes may join later or the cluster may remain at a size of
`NUM_NODES - ALLOWED_NOTREADY_NODES`.
---
assignees:
- davidopp
- lavalamp
title: Building Large Clusters
---
## Support
At {{page.version}}, Kubernetes supports clusters with up to 1000 nodes. More specifically, we support configurations that meet *all* of the following criteria:
* No more than 2000 nodes
* No more than 60000 total pods
* No more than 120000 total containers
* No more than 100 pods per node
<br>
* TOC
{:toc}
## Setup
A cluster is a set of nodes (physical or virtual machines) running Kubernetes agents, managed by a "master" (the cluster-level control plane).
Normally the number of nodes in a cluster is controlled by the the value `NUM_NODES` in the platform-specific `config-default.sh` file (for example, see [GCE's `config-default.sh`](http://releases.k8s.io/{{page.githubbranch}}/cluster/gce/config-default.sh)).
Simply changing that value to something very large, however, may cause the setup script to fail for many cloud providers. A GCE deployment, for example, will run in to quota issues and fail to bring the cluster up.
When setting up a large Kubernetes cluster, the following issues must be considered.
### Quota Issues
To avoid running into cloud provider quota issues, when creating a cluster with many nodes, consider:
* Increase the quota for things like CPU, IPs, etc.
*In [GCE, for example,](https://cloud.google.com/compute/docs/resource-quotas) you'll want to increase the quota for:
*CPUs
*VM instances
*Total persistent disk reserved
*In-use IP addresses
* Firewall Rules
*Forwarding rules
*Routes
* Target pools
* Gating the setup script so that it brings up new node VMs in smaller batches with waits in between, because some cloud providers rate limit the creation of VMs.
### Etcd storage
To improve performance of large clusters, we store events in a separate dedicated etcd instance.
When creating a cluster, existing salt scripts:
* start and configure additional etcd instance
* configure api-server to use it for storing events
### Size of master and master components
On GCE/GKE and AWS, `kube-up` automatically configures the proper VM size for your master depending on the number of nodes
in your cluster. On other providers, you will need to configure it manually. For reference, the sizes we use on GCE are
*1-5 nodes: n1-standard-1
*6-10 nodes: n1-standard-2
* 11-100 nodes: n1-standard-4
*101-250 nodes: n1-standard-8
*251-500 nodes: n1-standard-16
* more than 500 nodes: n1-standard-32
And the sizes we use on AWS are
*1-5 nodes: m3.medium
*6-10 nodes: m3.large
* 11-100 nodes: m3.xlarge
*101-250 nodes: m3.2xlarge
*251-500 nodes: c4.4xlarge
* more than 500 nodes: c4.8xlarge
Note that these master node sizes are currently only set at cluster startup time, and are not adjusted if you later scale your cluster up or down (e.g. manually removing or adding nodes, or using a cluster autoscaler).
### Addon Resources
To prevent memory leaks or other resource issues in [cluster addons](https://releases.k8s.io/{{page.githubbranch}}/cluster/addons) from consuming all the resources available on a node, Kubernetes sets resource limits on addon containers to limit the CPU and Memory resources they can consume (See PR [#10653](http://pr.k8s.io/10653/files) and [#10778](http://pr.k8s.io/10778/files)).
For [example](https://github.com/kubernetes/kubernetes/tree/{{page.githubbranch}}/cluster/saltbase/salt/fluentd-gcp/fluentd-gcp.yaml):
```yaml
containers:
- name:fluentd-cloud-logging
image:gcr.io/google_containers/fluentd-gcp:1.16
resources:
limits:
cpu:100m
memory:200Mi
```
Except for Heapster, these limits are static and are based on data we collected from addons running on 4-node clusters (see [#10335](http://issue.k8s.io/10335#issuecomment-117861225)). The addons consume a lot more resources when running on large deployment clusters (see [#5880](http://issue.k8s.io/5880#issuecomment-113984085)). So, if a large cluster is deployed without adjusting these values, the addons may continuously get killed because they keep hitting the limits.
To avoid running into cluster addon resource issues, when creating a cluster with many nodes, consider the following:
*Scale memory and CPU limits for each of the following addons, if used, as you scale up the size of cluster (there is one replica of each handling the entire cluster so memory and CPU usage tends to grow proportionally with size/load on cluster):
* [InfluxDB and Grafana](http://releases.k8s.io/{{page.githubbranch}}/cluster/addons/cluster-monitoring/influxdb/influxdb-grafana-controller.yaml)
* [kubedns, dnsmasq, and sidecar](http://releases.k8s.io/{{page.githubbranch}}/cluster/addons/dns/kubedns-controller.yaml.in)
*Scale number of replicas for the following addons, if used, along with the size of cluster (there are multiple replicas of each so increasing replicas should help handle increased load, but, since load per replica also increases slightly, also consider increasing CPU/memory limits):
*Increase memory and CPU limits slightly for each of the following addons, if used, along with the size of cluster (there is one replica per node but CPU/memory usage increases slightly along with cluster load/size as well):
* [FluentD with ElasticSearch Plugin](http://releases.k8s.io/{{page.githubbranch}}/cluster/addons/fluentd-elasticsearch/fluentd-es-ds.yaml)
* [FluentD with GCP Plugin](http://releases.k8s.io/{{page.githubbranch}}/cluster/addons/fluentd-gcp/fluentd-gcp-ds.yaml)
Heapster's resource limits are set dynamically based on the initial size of your cluster (see [#16185](http://issue.k8s.io/16185)
and [#22940](http://issue.k8s.io/22940)). If you find that Heapster is running
out of resources, you should adjust the formulas that compute heapster memory request (see those PRs for details).
For directions on how to detect if addon containers are hitting resource limits, see the [Troubleshooting section of Compute Resources](/docs/user-guide/compute-resources/#troubleshooting).
In the [future](http://issue.k8s.io/13048), we anticipate to set all cluster addon resource limits based on cluster size, and to dynamically adjust them if you grow or shrink your cluster.
We welcome PRs that implement those features.
### Allowing minor node failure at startup
For various reasons (see [#18969](https://github.com/kubernetes/kubernetes/issues/18969) for more details) running
`kube-up.sh` with a very large `NUM_NODES` may fail due to a very small number of nodes not coming up properly.
Currently you have two choices: restart the cluster (`kube-down.sh` and then `kube-up.sh` again), or before
running `kube-up.sh` set the environment variable `ALLOWED_NOTREADY_NODES` to whatever value you feel comfortable
with. This will allow `kube-up.sh` to succeed with fewer than `NUM_NODES` coming up. Depending on the
reason for the failure, those additional nodes may join later or the cluster may remain at a size of
@@ -19,7 +19,9 @@ To install Kubernetes on a set of machines, consult one of the existing [Getting
## Upgrading a cluster
The current state of cluster upgrades is provider dependent.
The current state of cluster upgrades is provider dependent, and some releases may require special care when upgrading. It is recommended that administrators consult both the [release notes](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG.md), as well as the version specific upgrade notes prior to upgrading their clusters.
* [Upgrading to 1.6](/docs/admin/upgrade-1-6)
### Upgrading Google Compute Engine clusters
@@ -56,8 +58,12 @@ The node upgrade process is user-initiated and is described in the [GKE document
### Upgrading clusters on other platforms
The `cluster/kube-push.sh` script will do a rudimentary update. This process is still quite experimental, we
recommend testing the upgrade on an experimental cluster before performing the update on a production cluster.
Different providers, and tools, will manage upgrades differently. It is recommended that you consult their main documentation regarding upgrades.
On GKE you configure cluster autoscaler either on cluster creation or update or when creating a particular node pool
@@ -159,11 +164,11 @@ node discovery; currently this is only Google Compute Engine, not including Core
### Upgrading to a different API version
When a new API version is released, you may need to upgrade a cluster to support the new API version (e.g. switching from 'v1' to 'v2' when 'v2' is launched)
When a new API version is released, you may need to upgrade a cluster to support the new API version (e.g. switching from 'v1' to 'v2' when 'v2' is launched).
This is an infrequent event, but it requires careful management. There is a sequence of steps to upgrade to a new API version.
1. Turn on the new api version.
1. Turn on the new API version.
1. Upgrade the cluster's storage to use the new version.
1. Upgrade all config files. Identify users of the old API version endpoints.
1. Update existing objects in the storage to new version by running `cluster/update-storage-objects.sh`.
@@ -171,16 +176,16 @@ This is an infrequent event, but it requires careful management. There is a sequ
### Turn on or off an API version for your cluster
Specific API versions can be turned on or off by passing --runtime-config=api/<version> flag while bringing up the API server. For example: to turn off v1 API, pass `--runtime-config=api/v1=false`.
Specific API versions can be turned on or off by passing `--runtime-config=api/<version>` flag while bringing up the API server. For example: to turn off v1 API, pass `--runtime-config=api/v1=false`.
runtime-config also supports 2 special keys: api/all and api/legacy to control all and legacy APIs respectively.
For example, for turning off all api versions except v1, pass `--runtime-config=api/all=false,api/v1=true`.
For example, for turning off all API versions except v1, pass `--runtime-config=api/all=false,api/v1=true`.
For the purposes of these flags, _legacy_ APIs are those APIs which have been explicitly deprecated (e.g. `v1beta3`).
### Switching your cluster's storage API version
The objects that are stored to disk for a cluster's internal representation of the Kubernetes resources active in the cluster are written using a particular version of the API.
When the supported API changes, these objects may need to be rewritten in the newer API. Failure to do this will eventually result in resources that are no longer decodable or usable
by the kubernetes API server.
by the Kubernetes API server.
`KUBE_API_VERSIONS` environment variable for the `kube-apiserver` binary which controls the API versions that are supported in the cluster. The first version in the list is used as the cluster's storage version. Hence, to set a specific version as the storage version, bring it to the front of list of versions in the value of `KUBE_API_VERSIONS`. You need to restart the `kube-apiserver` binary
for changes to this variable to take effect.
@@ -190,7 +195,7 @@ for changes to this variable to take effect.
You can use `kubectl convert` command to convert config files between different API versions.
```shell
$ kubectl convert -f pod.yaml --output-version v1
kubectl convert -f pod.yaml --output-version v1
```
For more options, please refer to the usage of [kubectl convert](/docs/user-guide/kubectl/kubectl_convert/) command.
If you specify a `scheduler.alpha.kubernetes.io/affinity` annotation in `.spec.template.metadata.annotations`,
then DaemonSet controller will create pods on nodes which match that [node affinity](../../user-guide/node-selection/#alpha-feature-in-kubernetes-v12-node-affinity).
@@ -88,18 +88,17 @@ created by the Daemon controller have the machine already selected (`.spec.nodeN
when the pod is created, so it is ignored by the scheduler). Therefore:
- the [`unschedulable`](/docs/admin/node/#manual-node-administration) field of a node is not respected
by the daemon set controller.
- daemon set controller can make pods even when the scheduler has not been started, which can help cluster
by the DaemonSet controller.
- DaemonSet controller can make pods even when the scheduler has not been started, which can help cluster
bootstrap.
## Communicating with DaemonSet Pods
Some possible patterns for communicating with pods in a DaemonSet are:
- **Push**: Pods in the DaemonSet are configured to send updates to another service, such
- **Push**: Pods in the DaemonSet are configured to send updates to another service, such
as a stats database. They do not have clients.
- **NodeIP and Known Port**: Pods in the DaemonSet use a `hostPort`, so that the pods are reachable
via the node IPs. Clients knows the the list of nodes ips somehow, and know the port by convention.
- **NodeIP and Known Port**: Pods in the DaemonSet use a `hostPort`, so that the pods are reachable via the node IPs. Clients know the list of nodes ips somehow, and know the port by convention.
- **DNS**: Create a [headless service](/docs/user-guide/services/#headless-services) with the same pod selector,
and then discover DaemonSets using the `endpoints` resource or retrieve multiple A records from
DNS.
@@ -126,11 +125,11 @@ You cannot update a DaemonSet.
Support for updating DaemonSets and controlled updating of nodes is planned.
## Alternatives to DaemonSet
## Alternatives to DaemonSet
### Init Scripts
It is certainly possible to run daemon processes by directly starting them on a node (e.g using
It is certainly possible to run daemon processes by directly starting them on a node (e.g. using
`init`, `upstartd`, or `systemd`). This is perfectly fine. However, there are several advantages to
running such processes via a DaemonSet:
@@ -145,9 +144,9 @@ running such processes via a DaemonSet:
### Bare Pods
It is possible to create pods directly which specify a particular node to run on. However,
a DaemonSet replaces pods that are deleted or terminated for any reason, such as in the case of
a DaemonSet replaces pods that are deleted or terminated for any reason, such as in the case of
node failure or disruptive node maintenance, such as a kernel upgrade. For this reason, you should
use a DaemonSet rather than creating individual pods.
use a DaemonSet rather than creating individual pods.
### Static Pods
@@ -159,7 +158,7 @@ in cluster bootstrapping cases. Also, static pods may be deprecated in the futu
### Replication Controller
DaemonSet are similar to [Replication Controllers](/docs/user-guide/replication-controller) in that
DaemonSet are similar to [Replication Controllers](/docs/user-guide/replication-controller) in that
they both create pods, and those pods have processes which are not expected to terminate (e.g. web servers,
As of Kubernetes 1.3, DNS is a built-in service launched automatically using the addon manager [cluster add-on](http://releases.k8s.io/{{page.githubbranch}}/cluster/addons/README.md).
A DNS Pod and Service will be scheduled on the cluster, and the kubelets will be
configured to tell individual containers to use the DNS Service's IP to resolve DNS names.
Every Service defined in the cluster (including the DNS server itself) will be
assigned a DNS name. By default, a client Pod's DNS search list will
include the Pod's own namespace and the cluster's default domain. This is best
illustrated by example:
Assume a Service named `foo` in the Kubernetes namespace `bar`. A Pod running
in namespace `bar` can look up this service by simply doing a DNS query for
`foo`. A Pod running in namespace `quux` can look up this service by doing a
DNS query for `foo.bar`.
The Kubernetes cluster DNS server (based off the [SkyDNS](https://github.com/skynetservices/skydns) library)
supports forward lookups (A records), service lookups (SRV records) and reverse IP address lookups (PTR records).
## How it Works
The running Kubernetes DNS pod holds 3 containers - kubedns, dnsmasq and a health check called healthz.
The kubedns process watches the Kubernetes master for changes in Services and Endpoints, and maintains
in-memory lookup structures to service DNS requests. The dnsmasq container adds DNS caching to improve
performance. The healthz container provides a single health check endpoint while performing dual healthchecks
(for dnsmasq and kubedns).
## Kubernetes Federation (Multiple Zone support)
Release 1.3 introduced Cluster Federation support for multi-site
Kubernetes installations. This required some minor
(backward-compatible) changes to the way
the Kubernetes cluster DNS server processes DNS queries, to facilitate
the lookup of federated services (which span multiple Kubernetes clusters).
See the [Cluster Federation Administrators' Guide](/docs/admin/federation) for more
details on Cluster Federation and multi-site support.
## References
- [Docs for the DNS cluster addon](http://releases.k8s.io/{{page.githubbranch}}/build/kube-dns/README.md)
{% include user-guide-content-moved.md %}
[DNS Pods and Services](/docs/concepts/services-networking/dns-pod-service/)
--admission-control string Ordered list of plug-ins to do admission control of resources into cluster. Comma-delimited list of: AlwaysAdmit, AlwaysDeny, NamespaceLifecycle. (default "AlwaysAdmit")
--admission-control string Ordered list of plug-ins to do admission control of resources into cluster. Comma-delimited list of: AlwaysAdmit, AlwaysDeny, NamespaceLifecycle, OwnerReferencesPermissionEnforcement. (default "AlwaysAdmit")
--admission-control-config-file string File with admission control configuration.
--advertise-address value The IP address on which to advertise the apiserver to members of the cluster. This address must be reachable by the rest of the cluster. If blank, the --bind-address will be used. If --bind-address is unspecified, the host's default interface will be used.
--apiserver-count intThe number of apiservers running in the cluster. (default 1)
--advertise-address ip The IP address on which to advertise the apiserver to members of the cluster. This address must be reachable by the rest of the cluster. If blank, the --bind-address will be used. If --bind-address is unspecified, the host's default interface will be used.
--anonymous-auth Enables anonymous requests to the secure port of the API server. Requests that are not rejected by another authentication method are treated as anonymous requests. Anonymous requests have a username of system:anonymous, and a group name of system:unauthenticated. (default true)
--apiserver-count int The number of apiservers running in the cluster. Must be a positive number. (default 1)
--audit-log-maxage int The maximum number of days to retain old audit log files based on the timestamp encoded in their filename.
--audit-log-maxbackup int The maximum number of old audit log files to retain.
--audit-log-maxsize int The maximum size in megabytes of the audit log file before it gets rotated. Defaults to 100MB.
@@ -35,64 +38,71 @@ federation-apiserver
--authorization-webhook-cache-unauthorized-ttl duration The duration to cache 'unauthorized' responses from the webhook authorizer. Default is 30s. (default 30s)
--authorization-webhook-config-file string File with webhook configuration in kubeconfig format, used with --authorization-mode=Webhook. The API server will query the remote service to determine access on the API server's secure port.
--basic-auth-file string If set, the file that will be used to admit requests to the secure port of the API server via http basic authentication.
--bind-address value The IP address on which to listen for the --secure-port port. The associated interface(s) must be reachable by the rest of the cluster, and by CLI/web clients. If blank, all interfaces will be used (0.0.0.0). (default 0.0.0.0)
--bind-address ip The IP address on which to listen for the --secure-port port. The associated interface(s) must be reachable by the rest of the cluster, and by CLI/web clients. If blank, all interfaces will be used (0.0.0.0). (default 0.0.0.0)
--cert-dir string The directory where the TLS certs are located (by default /var/run/kubernetes). If --tls-cert-file and --tls-private-key-file are provided, this flag will be ignored. (default "/var/run/kubernetes")
--client-ca-file string If set, any request presenting a client certificate signed by one of the authorities in the client-ca-file is authenticated with an identity corresponding to the CommonName of the client certificate.
--cloud-config string The path to the cloud provider configuration file. Empty string for no configuration file.
--cloud-provider string The provider for cloud services. Empty string for no provider.
--cors-allowed-origins valueList of allowed origins for CORS, comma separated. An allowed origin can be a regular expression to support subdomain matching. If this list is empty CORS will not be enabled. (default [])
--contention-profiling Enable contention profiling. Requires --profiling to be set to work.
--cors-allowed-origins stringSlice List of allowed origins for CORS, comma separated. An allowed origin can be a regular expression to support subdomain matching. If this list is empty CORS will not be enabled.
--delete-collection-workers int Number of workers spawned for DeleteCollection call. These are used to speed up namespace cleanup. (default 1)
--deserialization-cache-size int Number of deserialized json objects to cache in memory. (default 50000)
--deserialization-cache-size int Number of deserialized json objects to cache in memory.
--enable-garbage-collector Enables the generic garbage collector. MUST be synced with the corresponding flag of the kube-controller-manager. (default true)
--enable-swagger-ui Enables swagger ui on the apiserver at /swagger-ui
--etcd-cafile string SSL Certificate Authority file used to secure etcd communication.
--etcd-certfile string SSL certification file used to secure etcd communication.
--etcd-keyfile string SSL key file used to secure etcd communication.
--etcd-prefix string The prefix for all resource paths in etcd. (default "/registry")
--etcd-quorum-read If true, enable quorum read.
--etcd-servers value List of etcd servers to connect with (http://ip:port), comma separated. (default [])
--etcd-servers-overrides value Per-resource etcd servers overrides, comma separated. The individual override format: group/resource#servers, where servers are http://ip:port, semicolon separated. (default [])
--etcd-servers stringSlice List of etcd servers to connect with (scheme://ip:port), comma separated.
--etcd-servers-overrides stringSlice Per-resource etcd servers overrides, comma separated. The individual override format: group/resource#servers, where servers are http://ip:port, semicolon separated.
--event-ttl duration Amount of time to retain events. Default is 1h. (default 1h0m0s)
--experimental-keystone-ca-file string If set, the Keystone server's certificate will be verified by one of the authorities in the experimental-keystone-ca-file, otherwise the host's root CA set will be used.
--experimental-keystone-url string If passed, activates the keystone authentication plugin.
--external-hostname string The hostname to use when generating externalized URLs for this master (e.g. Swagger API Docs).
--feature-gates value A set of key=value pairs that describe feature gates for alpha/experimental features. Options are:
--feature-gates mapStringBool A set of key=value pairs that describe feature gates for alpha/experimental features. Options are:
--insecure-bind-address value The IP address on which to serve the --insecure-port (set to 0.0.0.0 for all interfaces). Defaults to localhost. (default 127.0.0.1)
--insecure-allow-any-token username/group1,group2 If set, your server will be INSECURE. Any token will be allowed and user information will be parsed from the token as username/group1,group2
--insecure-bind-address ip The IP address on which to serve the --insecure-port (set to 0.0.0.0 for all interfaces). Defaults to localhost. (default 127.0.0.1)
--insecure-port int The port on which to serve unsecured, unauthenticated access. Default 8080. It is assumed that firewall rules are set up such that this port is not reachable from outside of the cluster and that port 443 on the cluster's public address is proxied to this port. This is performed by nginx in the default setup. (default 8080)
--kubernetes-service-node-port int If non-zero, the Kubernetes master service (which apiserver creates/maintains) will be of type NodePort, using this as the value of the port. If zero, the Kubernetes master service will be of type ClusterIP.
--long-running-request-regexp string A regular expression matching long running requests which should be excluded from maximum inflight request handling. (default "(/|^)((watch|proxy)(/|$)|(logs?|portforward|exec|attach)/?$)")
--master-service-namespace string The namespace from which the kubernetes master services should be injected into pods. (default "default")
--master-service-namespace string DEPRECATED: the namespace from which the kubernetes master services should be injected into pods. (default "default")
--max-requests-inflight int The maximum number of requests in flight at a given time. When the server exceeds this, it rejects requests. Zero for no limit. (default 400)
--min-request-timeout int An optional field indicating the minimum number of seconds a handler must keep a request open before timing it out. Currently only honored by the watch request handler, which picks a randomized value above this number as the connection timeout, to spread out load. (default 1800)
--oidc-ca-file string If set, the OpenID server's certificate will be verified by one of the authorities in the oidc-ca-file, otherwise the host's root CA set will be used.
--oidc-client-id string The client ID for the OpenID Connect client, must be set if oidc-issuer-url is set.
--oidc-groups-claim string If provided, the name of a custom OpenID Connect claim for specifying user groups. The claim value is expected to be an array of strings. This flag is experimental, please see the authentication documentation for further details.
--oidc-groups-claim string If provided, the name of a custom OpenID Connect claim for specifying user groups. The claim value is expected to be a string or array of strings. This flag is experimental, please see the authentication documentation for further details.
--oidc-issuer-url string The URL of the OpenID issuer, only HTTPS scheme will be accepted. If set, it will be used to verify the OIDC JSON Web Token (JWT).
--oidc-username-claim string The OpenID claim to use as the user name. Note that claims other than the default ('sub') is not guaranteed to be unique and immutable. This flag is experimental, please see the authentication documentation for further details. (default "sub")
--profiling Enable profiling via web interface host:port/debug/pprof/ (default true)
--runtime-config value A set of key=value pairs that describe runtime configuration that may be passed to apiserver. apis/<groupVersion> key can be used to turn on/off specific api versions. apis/<groupVersion>/<resource> can be used to turn on/off specific resources. api/all and api/legacy are special keys to control all and legacy api versions respectively.
--requestheader-allowed-names stringSlice List of client certificate common names to allow to provide usernames in headers specified by --requestheader-username-headers. If empty, any client certificate validated by the authorities in --requestheader-client-ca-file is allowed.
--requestheader-client-ca-file string Root certificate bundle to use to verify client certificates on incoming requests before trusting usernames in headers specified by --requestheader-username-headers
--requestheader-username-headers stringSlice List of request headers to inspect for usernames. X-Remote-User is common.
--runtime-config mapStringString A set of key=value pairs that describe runtime configuration that may be passed to apiserver. apis/<groupVersion> key can be used to turn on/off specific api versions. apis/<groupVersion>/<resource> can be used to turn on/off specific resources. api/all and api/legacy are special keys to control all and legacy api versions respectively.
--secure-port int The port on which to serve HTTPS with authentication and authorization. If 0, don't serve HTTPS at all. (default 6443)
--service-cluster-ip-range value A CIDR notation IP range from which to assign service cluster IPs. This must not overlap with any IP ranges assigned to nodes for pods.
--service-node-port-range value A port range to reserve for services with NodePort visibility. Example: '30000-32767'. Inclusive at both ends of the range. (default 30000-32767)
--service-cluster-ip-range ipNet A CIDR notation IP range from which to assign service cluster IPs. This must not overlap with any IP ranges assigned to nodes for pods.
--service-node-port-range portRange A port range to reserve for services with NodePort visibility. Example: '30000-32767'. Inclusive at both ends of the range. (default 30000-32767)
--storage-backend string The storage backend for persistence. Options: 'etcd2' (default), 'etcd3'.
--storage-media-type string The media type to use to store objects in storage. Defaults to application/json. Some resources may only support a specific media type and will ignore this setting. (default "application/json")
--storage-versions string The per-group version to store resources in. Specified in the format "group1/version1,group2/version2,...". In the case where objects are moved from one group to the other, you may specify the format "group1=group2/v1beta1,group3/v1beta1,...". You only need to pass the groups you wish to change from the defaults. It defaults to a list of preferred versions of all registered groups, which is derived from the KUBE_API_VERSIONS environment variable. (default "apps/v1alpha1,authentication.k8s.io/v1beta1,authorization.k8s.io/v1beta1,autoscaling/v1,batch/v1,certificates/v1alpha1,componentconfig/v1alpha1,extensions/v1beta1,federation/v1beta1,policy/v1alpha1,rbac.authorization.k8s.io/v1alpha1,v1")
--storage-versions string The per-group version to store resources in. Specified in the format "group1/version1,group2/version2,...". In the case where objects are moved from one group to the other, you may specify the format "group1=group2/v1beta1,group3/v1beta1,...". You only need to pass the groups you wish to change from the defaults. It defaults to a list of preferred versions of all registered groups, which is derived from the KUBE_API_VERSIONS environment variable. (default "apps/v1beta1,authentication.k8s.io/v1beta1,authorization.k8s.io/v1beta1,autoscaling/v1,batch/v1,certificates.k8s.io/v1alpha1,componentconfig/v1alpha1,extensions/v1beta1,federation/v1beta1,policy/v1beta1,rbac.authorization.k8s.io/v1alpha1,storage.k8s.io/v1beta1,v1")
--target-ram-mb int Memory limit for apiserver in MB (used to configure sizes of caches, etc.)
--tls-cert-file string File containing x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If HTTPS serving is enabled, and --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to /var/run/kubernetes.
--tls-ca-file string If set, this certificate authority will used for secure access from Admission Controllers. This must be a valid PEM-encoded CA bundle.
--tls-cert-file string File containing the default x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If HTTPS serving is enabled, and --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to /var/run/kubernetes.
--tls-sni-cert-key namedCertKey A pair of x509 certificate and private key file paths, optionally suffixed with a list of domain patterns which are fully qualified domain names, possibly with prefixed wildcard segments. If no domain patterns are provided, the names of the certificate are extracted. Non-wildcard matches trump over wildcard matches, explicit domain patterns trump over extracted names. For multiple key/certificate pairs, use the --tls-sni-cert-key multiple times. Examples: "example.key,example.crt" or "*.foo.com,foo.com:foo.key,foo.crt". (default [])
--token-auth-file string If set, the file that will be used to secure the secure port of the API server via token authentication.
--watch-cache Enable watch caching in the apiserver (default true)
--watch-cache-sizes value List of watch cache sizes for every resource (pods, nodes, etc.), comma separated. The individual override format: resource#size, where size is a number. It takes effect when watch-cache is enabled. (default [])
--watch-cache-sizes stringSlice List of watch cache sizes for every resource (pods, nodes, etc.), comma separated. The individual override format: resource#size, where size is a number. It takes effect when watch-cache is enabled.
```
###### Auto generated by spf13/cobra on 2-Sep-2016
###### Auto generated by spf13/cobra on 13-Dec-2016
--address value The IP address to serve on (set to 0.0.0.0 for all interfaces) (default 0.0.0.0)
--address ip The IP address to serve on (set to 0.0.0.0 for all interfaces) (default 0.0.0.0)
--cluster-monitor-period duration The period for syncing ClusterStatus in ClusterController. (default 40s)
--concurrent-replicaset-syncs int The number of ReplicaSets syncing operations that will be done concurrently. Larger number = faster endpoint updating, but more CPU (and network) load (default 10)
--concurrent-service-syncs int The number of service syncing operations that will be done concurrently. Larger number = faster endpoint updating, but more CPU (and network) load (default 10)
--dns-provider string DNS provider. Valid values are: ["aws-route53" "google-clouddns"]
--dns-provider string DNS provider. Valid values are: ["google-clouddns" "aws-route53"]
--dns-provider-config string Path to config file for configuring DNS provider.
--federated-api-burst int Burst to use while talking with federation apiserver (default 30)
--federated-api-qps value QPS to use while talking with federation apiserver (default 20)
--federated-api-qps float32 QPS to use while talking with federation apiserver (default 20)
--federation-name string Federation name.
--kube-api-content-type string ContentType of requests sent to apiserver. Passing application/vnd.kubernetes.protobuf is an experimental feature now.
--kubeconfig string Path to kubeconfig file with authorization and master location information.
@@ -41,14 +43,12 @@ federation-controller-manager
--master string The address of the federation API server (overrides any value in kubeconfig)
--port int The port that the controller-manager's http service runs on (default 10253)
--profiling Enable profiling via web interface host:port/debug/pprof/ (default true)
--service-dns-suffix string DNS Suffix to use when publishing federated service names. Defaults to zone-name
--zone-id string Zone ID, needed if the zone name is not unique.
--zone-name string Zone name, like example.com.
```
###### Auto generated by spf13/cobra on 2-Sep-2016
###### Auto generated by spf13/cobra on 13-Dec-2016
Note: This is going to overwite the values you might have set for
Note: This is going to overwrite the values you might have set for
`apiserverRegistry`, `apiserverVersion`, `controllerManagerRegistry` and
`controllerManagerVersion` in your `${FEDERATION_OUTPUT_ROOT}/values.yaml`
file. Hence, it is not recommend to customize these values in
@@ -212,47 +213,81 @@ cluster1 Ready 3m
## Updating KubeDNS
Once the cluster is registered with the federation, you are all set to use it.
But for the cluster to be able to route federation service requests, you need to restart
KubeDNS and pass it a `--federations` flag which tells it about valid federation DNS hostnames.
Format of the flag is like this:
Once you've registered your cluster with the federation, you'll need to update KubeDNS so that your cluster can route federation service requests. The update method varies depending on your Kubernetes version; on Kubernetes 1.5 or later, you must pass the
`--federations` flag to kube-dns via the kube-dns config map. In version 1.4 or earlier, you must set the `--federations` flag directly on kube-dns-rc on other clusters.
### Kubernetes 1.5+: Passing federations flag via config map to kube-dns
For Kubernetes clusters of version 1.5+, you can pass the
`--federations` flag to kube-dns via the kube-dns config map.
Garbage collection is a helpful function of kubelet that will clean up unused images and unused containers. kubelet will perform garbage collection for containers every minute and garbage collection for images every five minutes.
Garbage collection is a helpful function of kubelet that will clean up unused images and unused containers. Kubelet will perform garbage collection for containers every minute and garbage collection for images every five minutes.
External garbage collection tools are not recommended as these tools can potentially break the behavior of kubelet by removing containers expected to exist.
### Image Collection
kubernetes manages lifecycle of all images through imageManager, with the cooperation
Kubernetes manages lifecycle of all images through imageManager, with the cooperation
of cadvisor.
The policy for garbage collecting images takes two factors into consideration:
`HighThresholdPercent` and `LowThresholdPercent`. Disk usage above the the high threshold
`HighThresholdPercent` and `LowThresholdPercent`. Disk usage above the high threshold
will trigger garbage collection. The garbage collection will delete least recently used images until the low
threshold has been met.
### Container Collection
The policy for garbage collecting containers considers three user-defined variables. `MinAge` is the minimum age at which a container can be garbage collected. `MaxPerPodContainer` is the maximum number of dead containers any single
pod (UID, container name) pair is allowed to have. `MaxContainers` is the maximum number of total dead containers. These variables can be individually disabled by setting 'MinAge' to zero and setting 'MaxPerPodContainer' and 'MaxContainers' respectively to less than zero.
The policy for garbage collecting containers considers three user-defined variables. `MinAge` is the minimum age at which a container can be garbage collected. `MaxPerPodContainer` is the maximum number of dead containers every single
pod (UID, container name) pair is allowed to have. `MaxContainers` is the maximum number of total dead containers. These variables can be individually disabled by setting `MinAge` to zero and setting `MaxPerPodContainer` and `MaxContainers` respectively to less than zero.
Kubelet will act on containers that are unidentified, deleted, or outside of the boundaries set by the previously mentioned flags. The oldest containers will generally be removed first. 'MaxPerPodContainer' and 'MaxContainer' may potentially conflict with each other in situations where retaining the maximum number of containers per pod ('MaxPerPodContainer') would go outside the allowable range of global dead containers ('MaxContainers'). 'MaxPerPodContainer' would be adjusted in this situation: A worst case scenario would be to downgrade 'MaxPerPodContainer' to 1 and evict the oldest containers. Additionally, containers owned by pods that have been deleted are removed once they are older than `MinAge`.
Kubelet will act on containers that are unidentified, deleted, or outside of the boundaries set by the previously mentioned flags. The oldest containers will generally be removed first. `MaxPerPodContainer` and `MaxContainer` may potentially conflict with each other in situations where retaining the maximum number of containers per pod (`MaxPerPodContainer`) would go outside the allowable range of global dead containers (`MaxContainers`). `MaxPerPodContainer` would be adjusted in this situation: A worst case scenario would be to downgrade `MaxPerPodContainer` to 1 and evict the oldest containers. Additionally, containers owned by pods that have been deleted are removed once they are older than `MinAge`.
Containers that are not managed by kubelet are not subject to container garbage collection.
@@ -42,15 +42,34 @@ to free. Default is 80%.
We also allow users to customize garbage collection policy through the following kubelet flags:
1. `minimum-container-ttl-duration`, minimum age for a finished container before it is
garbage collected. Default is 1 minute.
2. `maximum-dead-containers-per-container`, maximum number of old instances to retain
per container. Default is 2.
garbage collected. Default is 0 minute, which means every finished container will be garbaged collected.
2. `maximum-dead-containers-per-container`, maximum number of old instances to be retained
per container. Default is 1.
3. `maximum-dead-containers`, maximum number of old instances of containers to retain globally.
Default is 100.
Default is -1, which means there is no global limit.
Containers can potentially be garbage collected before their usefulness has expired. These containers
can contain logs and other data that can be useful for troubleshooting. A sufficiently large value for
`maximum-dead-containers-per-container` is highly recommended to allow at least 2 dead containers to be
retained per expected container. A higher value for `maximum-dead-containers` is also recommended for a
similiar reason.
`maximum-dead-containers-per-container` is highly recommended to allow at least 1 dead container to be
retained per expected container. A larger value for `maximum-dead-containers` is also recommended for a
similar reason.
See [this issue](https://github.com/kubernetes/kubernetes/issues/13287) for more details.
### Deprecation
Some kubelet Garbage Collection features in this doc will be replaced by kubelet eviction in the future.
Including:
| Existing Flag | New Flag | Rationale |
| ------------- | -------- | --------- |
| `--image-gc-high-threshold` | `--eviction-hard` or `--eviction-soft` | existing eviction signals can trigger image garbage collection |
| `--image-gc-low-threshold` | `--eviction-minimum-reclaim` | eviction reclaims achieve the same behavior |
| `--maximum-dead-containers` | | deprecated once old logs are stored outside of container's context |
| `--maximum-dead-containers-per-container` | | deprecated once old logs are stored outside of container's context |
| `--minimum-container-ttl-duration` | | deprecated once old logs are stored outside of container's context |
| `--low-diskspace-threshold-mb` | `--eviction-hard` or `eviction-soft` | eviction generalizes disk thresholds to other resources |
| `--outofdisk-transition-frequency` | `--eviction-pressure-transition-period` | eviction generalizes disk pressure transition to other resources |
See [Configuring Out Of Resource Handling](/docs/admin/out-of-resource/) for more details.
## Best practices for replicating masters for HA clusters
* Try to place masters replicas in different zones. During a zone failure, all master placed inside the zone will fail.
To survive zone failure, also place nodes in multiple zones
(see [multiple-zones](http://kubernetes.io/docs/admin/multiple-zones/) for details).
* Do not use a cluster with two master replicas. Consensus on a two replica cluster requires both replicas running when changing persistent state.
As a result, both replicas are needed and a failure of any replica turns cluster into majority failure state.
A two-replica cluster is thus inferior, in terms of HA, to a single replica cluster.
* When you add a master replica, cluster state (etcd) is copied to a new instance.
If the cluster is large, it may take a long time to duplicate its state.
This operation may be sped up by migrating etcd data directory, as described [here](https://coreos.com/etcd/docs/latest/admin_guide.html#member-migration)
(we are considering adding support for etcd data dir migration in future).
## Implementation notes

### Overview
Each of master replicas will run the following components in the following mode:
* etcd instance: all instances will be clustered together using consensus;
* API server: each server will talk to local etcd - all API servers in the cluster will be available;
* controllers, scheduler, and cluster auto-scaler: will use lease mechanism - only one instance of each of them will be active in the cluster;
* add-on manager: each manager will work independently trying to keep add-ons in sync.
In addition, there will be a load balancer in front of API servers that will route external and internal traffic to them.
### Load balancing
When starting the second master replica, a load balancer containing the two replicas will be created
and the IP address of the first replica will be promoted to IP address of load balancer.
Similarly, after removal of the penultimate master replica, the load balancer will be removed and its IP address will be assigned to the last remaining replica.
Please note that creation and removal of load balancer are complex operations and it may take some time (~20 minutes) for them to propagate.
### Master service & kubelets
Instead of trying to keep an up-to-date list of Kubernetes apiserver in the Kubernetes service,
the system directs all traffic to the external IP:
* in one master cluster the IP points to the single master,
* in multi-master cluster the IP points to the load balancer in-front of the masters.
Similarly, the external IP will be used by kubelets to communicate with master.
### Master certificates
Kubernetes generates Master TLS certificates for the external public IP and local IP for each replica.
There are no certificates for the ephemeral public IP for replicas;
to access a replica via its ephemeral public IP, you must skip TLS verification.
### Clustering etcd
To allow etcd clustering, ports needed to communicate between etcd instances will be opened (for inside cluster communication).
To make such deployment secure, communication between etcd instances is authorized using SSL.
## Additional reading
[Automated HA master deployment - design doc](https://github.com/kubernetes/kubernetes/blob/master/docs/design/ha_master.md)
--admission-control string Ordered list of plug-ins to do admission control of resources into cluster. Comma-delimited list of: AlwaysAdmit, AlwaysDeny, AlwaysPullImages, DefaultStorageClass, DenyEscalatingExec, DenyExecOnPrivileged, ImagePolicyWebhook, InitialResources, LimitPodHardAntiAffinityTopology, LimitRanger, NamespaceAutoProvision, NamespaceExists, NamespaceLifecycle, PersistentVolumeLabel, PodSecurityPolicy, ResourceQuota, SecurityContextDeny, ServiceAccount. (default "AlwaysAdmit")
--admission-control string Ordered list of plug-ins to do admission control of resources into cluster. Comma-delimited list of: AlwaysAdmit, AlwaysDeny, AlwaysPullImages, DefaultStorageClass, DenyEscalatingExec, DenyExecOnPrivileged, ImagePolicyWebhook, InitialResources, LimitPodHardAntiAffinityTopology, LimitRanger, NamespaceAutoProvision, NamespaceExists, NamespaceLifecycle, OwnerReferencesPermissionEnforcement, PersistentVolumeLabel, PodNodeSelector, PodSecurityPolicy, ResourceQuota, SecurityContextDeny, ServiceAccount. (default "AlwaysAdmit")
--admission-control-config-file string File with admission control configuration.
--advertise-address value The IP address on which to advertise the apiserver to members of the cluster. This address must be reachable by the rest of the cluster. If blank, the --bind-address will be used. If --bind-address is unspecified, the host's default interface will be used.
--advertise-address ip The IP address on which to advertise the apiserver to members of the cluster. This address must be reachable by the rest of the cluster. If blank, the --bind-address will be used. If --bind-address is unspecified, the host's default interface will be used.
--allow-privileged If true, allow privileged containers.
--apiserver-count intThe number of apiservers running in the cluster. (default 1)
--anonymous-auth Enables anonymous requests to the secure port of the API server. Requests that are not rejected by another authentication method are treated as anonymous requests. Anonymous requests have a username of system:anonymous, and a group name of system:unauthenticated. (default true)
--apiserver-count int The number of apiservers running in the cluster. Must be a positive number. (default 1)
--audit-log-maxage int The maximum number of days to retain old audit log files based on the timestamp encoded in their filename.
--audit-log-maxbackup int The maximum number of old audit log files to retain.
--audit-log-maxsize int The maximum size in megabytes of the audit log file before it gets rotated. Defaults to 100MB.
@@ -38,14 +41,15 @@ kube-apiserver
--authorization-webhook-cache-unauthorized-ttl duration The duration to cache 'unauthorized' responses from the webhook authorizer. Default is 30s. (default 30s)
--authorization-webhook-config-file string File with webhook configuration in kubeconfig format, used with --authorization-mode=Webhook. The API server will query the remote service to determine access on the API server's secure port.
--basic-auth-file string If set, the file that will be used to admit requests to the secure port of the API server via http basic authentication.
--bind-address value The IP address on which to listen for the --secure-port port. The associated interface(s) must be reachable by the rest of the cluster, and by CLI/web clients. If blank, all interfaces will be used (0.0.0.0). (default 0.0.0.0)
--bind-address ip The IP address on which to listen for the --secure-port port. The associated interface(s) must be reachable by the rest of the cluster, and by CLI/web clients. If blank, all interfaces will be used (0.0.0.0). (default 0.0.0.0)
--cert-dir string The directory where the TLS certs are located (by default /var/run/kubernetes). If --tls-cert-file and --tls-private-key-file are provided, this flag will be ignored. (default "/var/run/kubernetes")
--client-ca-file string If set, any request presenting a client certificate signed by one of the authorities in the client-ca-file is authenticated with an identity corresponding to the CommonName of the client certificate.
--cloud-config string The path to the cloud provider configuration file. Empty string for no configuration file.
--cloud-provider string The provider for cloud services. Empty string for no provider.
--cors-allowed-origins valueList of allowed origins for CORS, comma separated. An allowed origin can be a regular expression to support subdomain matching. If this list is empty CORS will not be enabled. (default [])
--contention-profiling Enable contention profiling. Requires --profiling to be set to work.
--cors-allowed-origins stringSlice List of allowed origins for CORS, comma separated. An allowed origin can be a regular expression to support subdomain matching. If this list is empty CORS will not be enabled.
--delete-collection-workers int Number of workers spawned for DeleteCollection call. These are used to speed up namespace cleanup. (default 1)
--deserialization-cache-size int Number of deserialized json objects to cache in memory. (default 50000)
--deserialization-cache-size int Number of deserialized json objects to cache in memory.
--enable-garbage-collector Enables the generic garbage collector. MUST be synced with the corresponding flag of the kube-controller-manager. (default true)
--enable-swagger-ui Enables swagger ui on the apiserver at /swagger-ui
--etcd-cafile string SSL Certificate Authority file used to secure etcd communication.
@@ -53,62 +57,75 @@ kube-apiserver
--etcd-keyfile string SSL key file used to secure etcd communication.
--etcd-prefix string The prefix for all resource paths in etcd. (default "/registry")
--etcd-quorum-read If true, enable quorum read.
--etcd-servers value List of etcd servers to connect with (http://ip:port), comma separated. (default [])
--etcd-servers-overrides value Per-resource etcd servers overrides, comma separated. The individual override format: group/resource#servers, where servers are http://ip:port, semicolon separated. (default [])
--etcd-servers stringSlice List of etcd servers to connect with (scheme://ip:port), comma separated.
--etcd-servers-overrides stringSlice Per-resource etcd servers overrides, comma separated. The individual override format: group/resource#servers, where servers are http://ip:port, semicolon separated.
--event-ttl duration Amount of time to retain events. Default is 1h. (default 1h0m0s)
--experimental-keystone-ca-file string If set, the Keystone server's certificate will be verified by one of the authorities in the experimental-keystone-ca-file, otherwise the host's root CA set will be used.
--experimental-keystone-url string If passed, activates the keystone authentication plugin.
--external-hostname string The hostname to use when generating externalized URLs for this master (e.g. Swagger API Docs).
--feature-gates value A set of key=value pairs that describe feature gates for alpha/experimental features. Options are:
--feature-gates mapStringBool A set of key=value pairs that describe feature gates for alpha/experimental features. Options are:
--google-json-key string The Google Cloud Platform Service Account JSON Key to use for authentication.
--insecure-bind-address value The IP address on which to serve the --insecure-port (set to 0.0.0.0 for all interfaces). Defaults to localhost. (default 127.0.0.1)
--insecure-allow-any-token username/group1,group2 If set, your server will be INSECURE. Any token will be allowed and user information will be parsed from the token as username/group1,group2
--insecure-bind-address ip The IP address on which to serve the --insecure-port (set to 0.0.0.0 for all interfaces). Defaults to localhost. (default 127.0.0.1)
--insecure-port int The port on which to serve unsecured, unauthenticated access. Default 8080. It is assumed that firewall rules are set up such that this port is not reachable from outside of the cluster and that port 443 on the cluster's public address is proxied to this port. This is performed by nginx in the default setup. (default 8080)
--ir-data-source string Data source used by InitialResources. Supported options: influxdb, gcm. (default "influxdb")
--ir-dbname string InfluxDB database name which contains metrics required by InitialResources. (default "k8s")
--ir-hawkular string Hawkular configuration URL.
--ir-influxdb-host string Address of InfluxDB which contains metrics required by InitialResources. (default "localhost:8080/api/v1/proxy/namespaces/kube-system/services/monitoring-influxdb:api")
--ir-namespace-only Whether the estimation should be made only based on data from the same namespace.
--ir-password string Password used for connecting to InfluxDB. (default "root")
--ir-percentile int Which percentile of samples should InitialResources use when estimating resources. For experiment purposes. (default 90)
--ir-user string User used for connecting to InfluxDB. (default "root")
--kubelet-certificate-authority string Path to a cert file for the certificate authority.
--kubelet-client-certificate string Path to a client cert file for TLS.
--kubelet-client-key string Path to a client key file for TLS.
--kubelet-https Use https for kubelet connections. (default true)
--kubelet-preferred-address-types stringSlice List of the preferred NodeAddressTypes to use for kubelet connections. (default [Hostname,InternalIP,ExternalIP,LegacyHostIP])
--kubelet-timeout duration Timeout for kubelet operations. (default 5s)
--kubernetes-service-node-port int If non-zero, the Kubernetes master service (which apiserver creates/maintains) will be of type NodePort, using this as the value of the port. If zero, the Kubernetes master service will be of type ClusterIP.
--long-running-request-regexp string A regular expression matching long running requests which should be excluded from maximum inflight request handling. (default "(/|^)((watch|proxy)(/|$)|(logs?|portforward|exec|attach)/?$)")
--master-service-namespace string The namespace from which the kubernetes master services should be injected into pods. (default "default")
--master-service-namespace string DEPRECATED: the namespace from which the kubernetes master services should be injected into pods. (default "default")
--max-connection-bytes-per-sec int If non-zero, throttle each user connection to this number of bytes/sec. Currently only applies to long-running requests.
--max-requests-inflight int The maximum number of requests in flight at a given time. When the server exceeds this, it rejects requests. Zero for no limit. (default 400)
--min-request-timeout int An optional field indicating the minimum number of seconds a handler must keep a request open before timing it out. Currently only honored by the watch request handler, which picks a randomized value above this number as the connection timeout, to spread out load. (default 1800)
--oidc-ca-file string If set, the OpenID server's certificate will be verified by one of the authorities in the oidc-ca-file, otherwise the host's root CA set will be used.
--oidc-client-id string The client ID for the OpenID Connect client, must be set if oidc-issuer-url is set.
--oidc-groups-claim string If provided, the name of a custom OpenID Connect claim for specifying user groups. The claim value is expected to be an array of strings. This flag is experimental, please see the authentication documentation for further details.
--oidc-groups-claim string If provided, the name of a custom OpenID Connect claim for specifying user groups. The claim value is expected to be a string or array of strings. This flag is experimental, please see the authentication documentation for further details.
--oidc-issuer-url string The URL of the OpenID issuer, only HTTPS scheme will be accepted. If set, it will be used to verify the OIDC JSON Web Token (JWT).
--oidc-username-claim string The OpenID claim to use as the user name. Note that claims other than the default ('sub') is not guaranteed to be unique and immutable. This flag is experimental, please see the authentication documentation for further details. (default "sub")
--profiling Enable profiling via web interface host:port/debug/pprof/ (default true)
--repair-malformed-updates If true, server will do its best to fix the update request to pass the validation, e.g., setting empty UID in update request to its existing value. This flag can be turned off after we fix all the clients that send malformed updates. (default true)
--runtime-config value A set of key=value pairs that describe runtime configuration that may be passed to apiserver. apis/<groupVersion> key can be used to turn on/off specific api versions. apis/<groupVersion>/<resource> can be used to turn on/off specific resources. api/all and api/legacy are special keys to control all and legacy api versions respectively.
--requestheader-allowed-names stringSlice List of client certificate common names to allow to provide usernames in headers specified by --requestheader-username-headers. If empty, any client certificate validated by the authorities in --requestheader-client-ca-file is allowed.
--requestheader-client-ca-file string Root certificate bundle to use to verify client certificates on incoming requests before trusting usernames in headers specified by --requestheader-username-headers
--requestheader-username-headers stringSlice List of request headers to inspect for usernames. X-Remote-User is common.
--runtime-config mapStringString A set of key=value pairs that describe runtime configuration that may be passed to apiserver. apis/<groupVersion> key can be used to turn on/off specific api versions. apis/<groupVersion>/<resource> can be used to turn on/off specific resources. api/all and api/legacy are special keys to control all and legacy api versions respectively.
--secure-port int The port on which to serve HTTPS with authentication and authorization. If 0, don't serve HTTPS at all. (default 6443)
--service-account-key-file string File containing PEM-encoded x509 RSA private or public key, used to verify ServiceAccount tokens. If unspecified, --tls-private-key-file is used.
--service-account-key-file stringArray File containing PEM-encoded x509 RSA or ECDSA private or public keys, used to verify ServiceAccount tokens. If unspecified, --tls-private-key-file is used. The specified file can contain multiple keys, and the flag can be specified multiple times with different files.
--service-account-lookup If true, validate ServiceAccount tokens exist in etcd as part of authentication.
--service-cluster-ip-range value A CIDR notation IP range from which to assign service cluster IPs. This must not overlap with any IP ranges assigned to nodes for pods.
--service-node-port-range value A port range to reserve for services with NodePort visibility. Example: '30000-32767'. Inclusive at both ends of the range. (default 30000-32767)
--service-cluster-ip-range ipNet A CIDR notation IP range from which to assign service cluster IPs. This must not overlap with any IP ranges assigned to nodes for pods.
--service-node-port-range portRange A port range to reserve for services with NodePort visibility. Example: '30000-32767'. Inclusive at both ends of the range. (default 30000-32767)
--ssh-keyfile string If non-empty, use secure SSH proxy to the nodes, using this user keyfile
--ssh-user string If non-empty, use secure SSH proxy to the nodes, using this user name
--storage-backend string The storage backend for persistence. Options: 'etcd2' (default), 'etcd3'.
--storage-media-type string The media type to use to store objects in storage. Defaults to application/json. Some resources may only support a specific media type and will ignore this setting. (default "application/json")
--storage-versions string The per-group version to store resources in. Specified in the format "group1/version1,group2/version2,...". In the case where objects are moved from one group to the other, you may specify the format "group1=group2/v1beta1,group3/v1beta1,...". You only need to pass the groups you wish to change from the defaults. It defaults to a list of preferred versions of all registered groups, which is derived from the KUBE_API_VERSIONS environment variable. (default "apps/v1alpha1,authentication.k8s.io/v1beta1,authorization.k8s.io/v1beta1,autoscaling/v1,batch/v1,certificates/v1alpha1,componentconfig/v1alpha1,extensions/v1beta1,imagepolicy.k8s.io/v1alpha1,policy/v1alpha1,rbac.authorization.k8s.io/v1alpha1,v1")
--storage-versions string The per-group version to store resources in. Specified in the format "group1/version1,group2/version2,...". In the case where objects are moved from one group to the other, you may specify the format "group1=group2/v1beta1,group3/v1beta1,...". You only need to pass the groups you wish to change from the defaults. It defaults to a list of preferred versions of all registered groups, which is derived from the KUBE_API_VERSIONS environment variable. (default "apps/v1beta1,authentication.k8s.io/v1beta1,authorization.k8s.io/v1beta1,autoscaling/v1,batch/v1,certificates.k8s.io/v1alpha1,componentconfig/v1alpha1,extensions/v1beta1,imagepolicy.k8s.io/v1alpha1,policy/v1beta1,rbac.authorization.k8s.io/v1alpha1,storage.k8s.io/v1beta1,v1")
--target-ram-mb int Memory limit for apiserver in MB (used to configure sizes of caches, etc.)
--tls-cert-file string File containing x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If HTTPS serving is enabled, and --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to /var/run/kubernetes.
--tls-ca-file string If set, this certificate authority will used for secure access from Admission Controllers. This must be a valid PEM-encoded CA bundle.
--tls-cert-file string File containing the default x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If HTTPS serving is enabled, and --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to /var/run/kubernetes.
--tls-sni-cert-key namedCertKey A pair of x509 certificate and private key file paths, optionally suffixed with a list of domain patterns which are fully qualified domain names, possibly with prefixed wildcard segments. If no domain patterns are provided, the names of the certificate are extracted. Non-wildcard matches trump over wildcard matches, explicit domain patterns trump over extracted names. For multiple key/certificate pairs, use the --tls-sni-cert-key multiple times. Examples: "example.key,example.crt" or "*.foo.com,foo.com:foo.key,foo.crt". (default [])
--token-auth-file string If set, the file that will be used to secure the secure port of the API server via token authentication.
--watch-cache Enable watch caching in the apiserver (default true)
--watch-cache-sizes value List of watch cache sizes for every resource (pods, nodes, etc.), comma separated. The individual override format: resource#size, where size is a number. It takes effect when watch-cache is enabled. (default [])
--watch-cache-sizes stringSlice List of watch cache sizes for every resource (pods, nodes, etc.), comma separated. The individual override format: resource#size, where size is a number. It takes effect when watch-cache is enabled.
```
###### Auto generated by spf13/cobra on 2-Sep-2016
###### Auto generated by spf13/cobra on 13-Dec-2016
--address value The IP address to serve on (set to 0.0.0.0 for all interfaces) (default 0.0.0.0)
--address ip The IP address to serve on (set to 0.0.0.0 for all interfaces) (default 0.0.0.0)
--allocate-node-cidrs Should CIDRs for Pods be allocated and set on the cloud provider.
--allow-verification-with-non-compliant-keys Allow a SignatureVerifier to use keys which are technically non-compliant with RFC6962.
--cloud-config string The path to the cloud provider configuration file. Empty string for no configuration file.
--cloud-provider string The provider for cloud services. Empty string for no provider.
--cluster-cidr string CIDR Range for Pods in cluster.
--cluster-name string The instance prefix for the cluster (default "kubernetes")
--cluster-signing-cert-file string Filename containing a PEM-encoded X509 CA certificate used to issue cluster-scoped certificates (default "/etc/kubernetes/ca/ca.pem")
--cluster-signing-key-file string Filename containing a PEM-encoded RSA or ECDSA private key used to sign cluster-scoped certificates (default "/etc/kubernetes/ca/ca.key")
--concurrent-deployment-syncs value The number of deployment objects that are allowed to sync concurrently. Larger number = more responsive deployments, but more CPU (and network) load (default 5)
--concurrent-endpoint-syncs value The number of endpoint syncing operations that will be done concurrently. Larger number = faster endpoint updating, but more CPU (and network) load (default 5)
--concurrent-gc-syncs value The number of garbage collector workers that are allowed to sync concurrently. (default 20)
--concurrent-namespace-syncs value The number of namespace objects that are allowed to sync concurrently. Larger number = more responsive namespace termination, but more CPU (and network) load (default 2)
--concurrent-replicaset-syncs value The number of replica sets that are allowed to sync concurrently. Larger number = more responsive replica management, but more CPU (and network) load (default 5)
--concurrent-resource-quota-syncs value The number of resource quotas that are allowed to sync concurrently. Larger number = more responsive quota management, but more CPU (and network) load (default 5)
--concurrent-service-syncs value The number of services that are allowed to sync concurrently. Larger number = more responsive service management, but more CPU (and network) load (default 1)
--concurrent-serviceaccount-token-syncs value The number of service account token objects that are allowed to sync concurrently. Larger number = more responsive token generation, but more CPU (and network) load (default 5)
--concurrent_rc_syncs value The number of replication controllers that are allowed to sync concurrently. Larger number = more responsive replica management, but more CPU (and network) load (default 5)
--concurrent-deployment-syncs int32 The number of deployment objects that are allowed to sync concurrently. Larger number = more responsive deployments, but more CPU (and network) load (default 5)
--concurrent-endpoint-syncs int32 The number of endpoint syncing operations that will be done concurrently. Larger number = faster endpoint updating, but more CPU (and network) load (default 5)
--concurrent-gc-syncs int32 The number of garbage collector workers that are allowed to sync concurrently. (default 20)
--concurrent-namespace-syncs int32 The number of namespace objects that are allowed to sync concurrently. Larger number = more responsive namespace termination, but more CPU (and network) load (default 2)
--concurrent-replicaset-syncs int32 The number of replica sets that are allowed to sync concurrently. Larger number = more responsive replica management, but more CPU (and network) load (default 5)
--concurrent-resource-quota-syncs int32 The number of resource quotas that are allowed to sync concurrently. Larger number = more responsive quota management, but more CPU (and network) load (default 5)
--concurrent-service-syncs int32 The number of services that are allowed to sync concurrently. Larger number = more responsive service management, but more CPU (and network) load (default 1)
--concurrent-serviceaccount-token-syncs int32 The number of service account token objects that are allowed to sync concurrently. Larger number = more responsive token generation, but more CPU (and network) load (default 5)
--concurrent_rc_syncs int32 The number of replication controllers that are allowed to sync concurrently. Larger number = more responsive replica management, but more CPU (and network) load (default 5)
--configure-cloud-routes Should CIDRs allocated by allocate-node-cidrs be configured on the cloud provider. (default true)
--controller-start-interval duration Interval between starting controller managers.
--daemonset-lookup-cache-size value The the size of lookup cache for daemonsets. Larger number = more responsive daemonsets, but more MEM load. (default 1024)
--daemonset-lookup-cache-size int32 The size of lookup cache for daemonsets. Larger number = more responsive daemonsets, but more MEM load. (default 1024)
--deployment-controller-sync-period duration Period for syncing the deployments. (default 30s)
--enable-dynamic-provisioning Enable dynamic provisioning for environments that support it. (default true)
--enable-garbage-collector Enables the generic garbage collector. MUST be synced with the corresponding flag of the kube-apiserver. (default true)
--enable-hostpath-provisioner Enable HostPath PV provisioning when running without a cloud provider. This allows testing and development of provisioning features. HostPath provisioning is not supported in any way, won't work in a multi-node cluster, and should not be used for anything other than testing or development.
--feature-gates value A set of key=value pairs that describe feature gates for alpha/experimental features. Options are:
--feature-gates mapStringBool A set of key=value pairs that describe feature gates for alpha/experimental features. Options are:
--flex-volume-plugin-dir string Full path of the directory in which the flex volume plugin should search for additional third party volume plugins. (default "/usr/libexec/kubernetes/kubelet-plugins/volume/exec/")
--google-json-key string The Google Cloud Platform Service Account JSON Key to use for authentication.
--horizontal-pod-autoscaler-sync-period duration The period for syncing the number of pods in horizontal pod autoscaler. (default 30s)
--insecure-experimental-approve-all-kubelet-csrs-for-group string The group for which the controller-manager will auto approve all CSRs for kubelet client certificates.
--kube-api-burst value Burst to use while talking with kubernetes apiserver (default 30)
--kube-api-burst int32 Burst to use while talking with Kubernetes apiserver (default 30)
--kube-api-content-type string Content type of requests sent to apiserver. (default "application/vnd.kubernetes.protobuf")
--kube-api-qps value QPS to use while talking with kubernetes apiserver (default 20)
--kube-api-qps float32 QPS to use while talking with Kubernetes apiserver (default 20)
--kubeconfig string Path to kubeconfig file with authorization and master location information.
--large-cluster-size-threshold value Number of nodes from which NodeController treats the cluster as large for the eviction logic purposes. --secondary-node-eviction-rate is implicitly overridden to 0 for clusters this size or smaller. (default 50)
--large-cluster-size-threshold int32 Number of nodes from which NodeController treats the cluster as large for the eviction logic purposes. --secondary-node-eviction-rate is implicitly overridden to 0 for clusters this size or smaller. (default 50)
--leader-elect Start a leader election client and gain leadership before executing the main loop. Enable this when running replicated components for high availability. (default true)
--leader-elect-lease-duration duration The duration that non-leader candidates will wait after observing a leadership renewal until attempting to acquire leadership of a led but unrenewed leader slot. This is effectively the maximum duration that a leader can be stopped before it is replaced by another candidate. This is only applicable if leader election is enabled. (default 15s)
--leader-elect-renew-deadline duration The interval between attempts by the acting master to renew a leadership slot before it stops leading. This must be less than or equal to the lease duration. This is only applicable if leader election is enabled. (default 10s)
--master string The address of the Kubernetes API server (overrides any value in kubeconfig)
--min-resync-period duration The resync period in reflectors will be random between MinResyncPeriod and 2*MinResyncPeriod (default 12h0m0s)
--namespace-sync-period duration The period for syncing namespace life-cycle updates (default 5m0s)
--node-cidr-mask-size value Mask size for node cidr in cluster. (default 24)
--node-eviction-rate value Number of nodes per second on which pods are deleted in case of node failure when a zone is healthy (see --unhealthy-zone-threshold for definition of healthy/unhealthy). Zone refers to entire cluster in non-multizone clusters. (default 0.1)
--node-cidr-mask-size int32 Mask size for node cidr in cluster. (default 24)
--node-eviction-rate float32 Number of nodes per second on which pods are deleted in case of node failure when a zone is healthy (see --unhealthy-zone-threshold for definition of healthy/unhealthy). Zone refers to entire cluster in non-multizone clusters. (default 0.1)
--node-monitor-grace-period duration Amount of time which we allow running Node to be unresponsive before marking it unhealthy. Must be N times more than kubelet's nodeStatusUpdateFrequency, where N means number of retries allowed for kubelet to post node status. (default 40s)
--node-monitor-period duration The period for syncing NodeStatus in NodeController. (default 5s)
--node-startup-grace-period duration Amount of time which we allow starting Node to be unresponsive before marking it unhealthy. (default 1m0s)
--node-sync-period duration The period for syncing nodes from cloudprovider. Longer periods will result in fewer calls to cloud provider, but may delay addition of new nodes to cluster. (default 10s)
--pod-eviction-timeout duration The grace period for deleting pods on failed nodes. (default 5m0s)
--port value The port that the controller-manager's http service runs on (default 10252)
--port int32 The port that the controller-manager's http service runs on (default 10252)
--profiling Enable profiling via web interface host:port/debug/pprof/ (default true)
--pv-recycler-increment-timeout-nfs value the increment of time added per Gi to ActiveDeadlineSeconds for an NFS scrubber pod (default 30)
--pv-recycler-minimum-timeout-hostpath value The minimum ActiveDeadlineSeconds to use for a HostPath Recycler pod. This is for development and testing only and will not work in a multi-node cluster. (default 60)
--pv-recycler-minimum-timeout-nfs value The minimum ActiveDeadlineSeconds to use for an NFS Recycler pod (default 300)
--pv-recycler-increment-timeout-nfs int32 the increment of time added per Gi to ActiveDeadlineSeconds for an NFS scrubber pod (default 30)
--pv-recycler-minimum-timeout-hostpath int32 The minimum ActiveDeadlineSeconds to use for a HostPath Recycler pod. This is for development and testing only and will not work in a multi-node cluster. (default 60)
--pv-recycler-minimum-timeout-nfs int32 The minimum ActiveDeadlineSeconds to use for an NFS Recycler pod (default 300)
--pv-recycler-pod-template-filepath-hostpath string The file path to a pod definition used as a template for HostPath persistent volume recycling. This is for development and testing only and will not work in a multi-node cluster.
--pv-recycler-pod-template-filepath-nfs string The file path to a pod definition used as a template for NFS persistent volume recycling
--pv-recycler-timeout-increment-hostpath value the increment of time added per Gi to ActiveDeadlineSeconds for a HostPath scrubber pod. This is for development and testing only and will not work in a multi-node cluster. (default 30)
--pv-recycler-timeout-increment-hostpath int32 the increment of time added per Gi to ActiveDeadlineSeconds for a HostPath scrubber pod. This is for development and testing only and will not work in a multi-node cluster. (default 30)
--pvclaimbinder-sync-period duration The period for syncing persistent volumes and persistent volume claims (default 15s)
--replicaset-lookup-cache-size value The the size of lookup cache for replicatsets. Larger number = more responsive replica management, but more MEM load. (default 4096)
--replication-controller-lookup-cache-size value The the size of lookup cache for replication controllers. Larger number = more responsive replica management, but more MEM load. (default 4096)
--replicaset-lookup-cache-size int32 The size of lookup cache for replicatsets. Larger number = more responsive replica management, but more MEM load. (default 4096)
--replication-controller-lookup-cache-size int32 The size of lookup cache for replication controllers. Larger number = more responsive replica management, but more MEM load. (default 4096)
--resource-quota-sync-period duration The period for syncing quota usage status in the system (default 5m0s)
--root-ca-file string If set, this root certificate authority will be included in service account's token secret. This must be a valid PEM-encoded CA bundle.
--secondary-node-eviction-rate value Number of nodes per second on which pods are deleted in case of node failure when a zone is unhealthy (see --unhealthy-zone-threshold for definition of healthy/unhealthy). Zone refers to entire cluster in non-multizone clusters. This value is implicitly overridden to 0 if the cluster size is smaller than --large-cluster-size-threshold. (default 0.01)
--service-account-private-key-file string Filename containing a PEM-encoded private RSA key used to sign service account tokens.
--route-reconciliation-period duration The period for reconciling routes created for Nodes by cloud provider. (default 10s)
--secondary-node-eviction-rate float32 Number of nodes per second on which pods are deleted in case of node failure when a zone is unhealthy (see --unhealthy-zone-threshold for definition of healthy/unhealthy). Zone refers to entire cluster in non-multizone clusters. This value is implicitly overridden to 0 if the cluster size is smaller than --large-cluster-size-threshold. (default 0.01)
--service-account-private-key-file string Filename containing a PEM-encoded private RSA or ECDSA key used to sign service account tokens.
--service-cluster-ip-range string CIDR Range for Services in cluster.
--service-sync-period duration The period for syncing services with their external load balancers (default 5m0s)
--terminated-pod-gc-threshold value Number of terminated pods that can exist before the terminated pod garbage collector starts deleting terminated pods. If <= 0, the terminated pod garbage collector is disabled. (default 12500)
--unhealthy-zone-threshold value Fraction of Nodes in a zone which needs to be not Ready (minimum 3) for zone to be treated as unhealthy. (default 0.55)
--terminated-pod-gc-threshold int32 Number of terminated pods that can exist before the terminated pod garbage collector starts deleting terminated pods. If <= 0, the terminated pod garbage collector is disabled. (default 12500)
--unhealthy-zone-threshold float32 Fraction of Nodes in a zone which needs to be not Ready (minimum 3) for zone to be treated as unhealthy. (default 0.55)
--use-service-account-credentials If true, use individual service account credentials for each controller.
```
###### Auto generated by spf13/cobra on 2-Sep-2016
###### Auto generated by spf13/cobra on 13-Dec-2016
--bind-address value The IP address for the proxy server to serve on (set to 0.0.0.0 for all interfaces) (default 0.0.0.0)
--bind-address ip The IP address for the proxy server to serve on (set to 0.0.0.0 for all interfaces) (default 0.0.0.0)
--cleanup-iptables If true cleanup iptables rules and exit.
--cluster-cidr string The CIDR range of pods in the cluster. It is used to bridge traffic coming from outside of the cluster. If not provided, no off-cluster bridging will be performed.
--config-sync-period duration How often configuration from the apiserver is refreshed. Must be greater than 0. (default 15m0s)
--conntrack-max value Maximum number of NAT connections to track (0 to leave as-is).
--conntrack-max-per-core value Maximum number of NAT connections to track per CPU core (0 to leave as-is). This is only considered if conntrack-max is 0. (default 32768)
--conntrack-max-per-core int32 Maximum number of NAT connections to track per CPU core (0 to leave the limit as-is and ignore conntrack-min). (default 32768)
--conntrack-min int32 Minimum number of conntrack entries to allocate, regardless of conntrack-max-per-core (set conntrack-max-per-core=0 to leave the limit as-is). (default 131072)
--conntrack-tcp-timeout-close-wait duration NAT timeout for TCP connections in the CLOSE_WAIT state (default 1h0m0s)
--conntrack-tcp-timeout-established duration Idle timeout for established TCP connections (0 to leave as-is) (default 24h0m0s)
--feature-gates value A set of key=value pairs that describe feature gates for alpha/experimental features. Options are:
--feature-gates mapStringBool A set of key=value pairs that describe feature gates for alpha/experimental features. Options are:
--google-json-key string The Google Cloud Platform Service Account JSON Key to use for authentication.
--healthz-bind-address value The IP address for the health check server to serve on, defaulting to 127.0.0.1 (set to 0.0.0.0 for all interfaces) (default 127.0.0.1)
--healthz-port value The port to bind the health check server. Use 0 to disable. (default 10249)
--healthz-bind-address ip The IP address for the health check server to serve on, defaulting to 127.0.0.1 (set to 0.0.0.0 for all interfaces) (default 127.0.0.1)
--healthz-port int32 The port to bind the health check server. Use 0 to disable. (default 10249)
--hostname-override string If non-empty, will use this string as identification instead of the actual hostname.
--iptables-masquerade-bit value If using the pure iptables proxy, the bit of the fwmark space to mark packets requiring SNAT with. Must be within the range [0, 31]. (default 14)
--iptables-sync-period duration How often iptables rules are refreshed (e.g. '5s', '1m', '2h22m'). Must be greater than 0. (default 30s)
--kube-api-burst value Burst to use while talking with kubernetes apiserver (default 10)
--iptables-masquerade-bit int32 If using the pure iptables proxy, the bit of the fwmark space to mark packets requiring SNAT with. Must be within the range [0, 31]. (default 14)
--iptables-min-sync-period duration The minimum interval of how often the iptables rules can be refreshed as endpoints and services change (e.g. '5s', '1m', '2h22m').
--iptables-sync-period duration The maximum interval of how often iptables rules are refreshed (e.g. '5s', '1m', '2h22m'). Must be greater than 0. (default 30s)
--kube-api-burst int32 Burst to use while talking with Kubernetes apiserver (default 10)
--kube-api-content-type string Content type of requests sent to apiserver. (default "application/vnd.kubernetes.protobuf")
--kube-api-qps value QPS to use while talking with kubernetes apiserver (default 5)
--kube-api-qps float32 QPS to use while talking with Kubernetes apiserver (default 5)
--kubeconfig string Path to kubeconfig file with authorization information (the master location is set by the master flag).
--masquerade-all If using the pure iptables proxy, SNAT everything
--master string The address of the Kubernetes API server (overrides any value in kubeconfig)
--oom-score-adj value The oom-score-adj value for kube-proxy process. Values must be within the range [-1000, 1000] (default -999)
--proxy-mode value Which proxy mode to use: 'userspace' (older) or 'iptables' (faster). If blank, look at the Node object on the Kubernetes API and respect the 'net.experimental.kubernetes.io/proxy-mode' annotation if provided. Otherwise use the best-available proxy (currently iptables). If the iptables proxy is selected, regardless of how, but the system's kernel or iptables versions are insufficient, this always falls back to the userspace proxy.
--proxy-port-range value Range of host ports (beginPort-endPort, inclusive) that may be consumed in order to proxy service traffic. If unspecified (0-0) then ports will be randomly chosen.
--oom-score-adj int32 The oom-score-adj value for kube-proxy process. Values must be within the range [-1000, 1000] (default -999)
--proxy-mode ProxyMode Which proxy mode to use: 'userspace' (older) or 'iptables' (faster). If blank, look at the Node object on the Kubernetes API and respect the 'net.experimental.kubernetes.io/proxy-mode' annotation if provided. Otherwise use the best-available proxy (currently iptables). If the iptables proxy is selected, regardless of how, but the system's kernel or iptables versions are insufficient, this always falls back to the userspace proxy.
--proxy-port-range port-range Range of host ports (beginPort-endPort, inclusive) that may be consumed in order to proxy service traffic. If unspecified (0-0) then ports will be randomly chosen.
--udp-timeout duration How long an idle UDP connection will be kept open (e.g. '250ms', '2s'). Must be greater than 0. Only applicable for proxy-mode=userspace (default 250ms)
```
###### Auto generated by spf13/cobra on 2-Sep-2016
###### Auto generated by spf13/cobra on 13-Dec-2016
--address string The IP address to serve on (set to 0.0.0.0 for all interfaces) (default "0.0.0.0")
--algorithm-provider string The scheduling algorithm provider to use, one of: DefaultProvider | ClusterAutoscalerProvider (default "DefaultProvider")
--algorithm-provider string The scheduling algorithm provider to use, one of: ClusterAutoscalerProvider | DefaultProvider (default "DefaultProvider")
--failure-domains string Indicate the "all topologies" set for an empty topologyKey when it's used for PreferredDuringScheduling pod anti-affinity. (default "kubernetes.io/hostname,failure-domain.beta.kubernetes.io/zone,failure-domain.beta.kubernetes.io/region")
--feature-gates value A set of key=value pairs that describe feature gates for alpha/experimental features. Options are:
--feature-gates mapStringBool A set of key=value pairs that describe feature gates for alpha/experimental features. Options are:
--hard-pod-affinity-symmetric-weight int RequiredDuringScheduling affinity is not symmetric, but there is an implicit PreferredDuringScheduling affinity rule corresponding to every RequiredDuringScheduling affinity rule. --hard-pod-affinity-symmetric-weight represents the weight of implicit PreferredDuringScheduling affinity rule. (default 1)
--kube-api-burst value Burst to use while talking with kubernetes apiserver (default 100)
--kube-api-burst int32 Burst to use while talking with Kubernetes apiserver (default 100)
--kube-api-content-type string Content type of requests sent to apiserver. (default "application/vnd.kubernetes.protobuf")
--kube-api-qps value QPS to use while talking with kubernetes apiserver (default 50)
--kube-api-qps float32 QPS to use while talking with Kubernetes apiserver (default 50)
--kubeconfig string Path to kubeconfig file with authorization and master location information.
--leader-elect Start a leader election client and gain leadership before executing the main loop. Enable this when running replicated components for high availability. (default true)
--leader-elect-lease-duration duration The duration that non-leader candidates will wait after observing a leadership renewal until attempting to acquire leadership of a led but unrenewed leader slot. This is effectively the maximum duration that a leader can be stopped before it is replaced by another candidate. This is only applicable if leader election is enabled. (default 15s)
--leader-elect-retry-period duration The duration the clients should wait between attempting acquisition and renewal of a leadership. This is only applicable if leader election is enabled. (default 2s)
--master string The address of the Kubernetes API server (overrides any value in kubeconfig)
--policy-config-file string File with scheduler policy configuration
--port value The port that the scheduler's http service runs on (default 10251)
--port int32 The port that the scheduler's http service runs on (default 10251)
--profiling Enable profiling via web interface host:port/debug/pprof/ (default true)
--scheduler-name string Name of the scheduler, used to select which pods will be processed by this scheduler, based on pod's annotation with key 'scheduler.alpha.kubernetes.io/name' (default "default-scheduler")
```
###### Auto generated by spf13/cobra on 2-Sep-2016
###### Auto generated by spf13/cobra on 13-Dec-2016
| `KUBE_HOST_ETCD_PATH` | `/var/lib/etcd` | Local etcd state for Kubernetes cluster |
| `KUBE_HYPERKUBE_IMAGE` | `` | If set, use a single hyperkube image with this name. If not set, individual images per server component will be used. |
| `KUBE_DISCOVERY_IMAGE` | `gcr.io/google_containers/kube-discovery-<arch>:1.0` | The bootstrap discovery helper image to use. |
| `KUBE_ETCD_IMAGE` | `gcr.io/google_containers/etcd-<arch>:2.2.5` | The etcd container image to use. |
| `KUBE_REPO_PREFIX` | `gcr.io/google_containers` | The image prefix for all images that are used. |
If you want to use kubeadm with an http proxy, you may need to configure it to support http_proxy, https_proxy, or no_proxy.
For example, if your kube master node IP address is 10.18.17.16 and you have proxy support both http/https on 10.18.17.16 port 8080, you can use the following command:
Remember to change ```proxy_ip``` and add a kube master node IP address to ```no_proxy```.
## Releases and release notes
If you already have kubeadm installed and want to upgrade, run `apt-get update && apt-get upgrade` or `yum update` to get the latest version of kubeadm.
Refer to the [CHANGELOG.md](https://github.com/kubernetes/kubeadm/blob/master/CHANGELOG.md) for more information.
A kubelet's HTTPS endpoint exposes APIs which give access to data of varying sensitivity,
and allow you to perform operations with varying levels of power on the node and within containers.
This document describes how to authenticate and authorize access to the kubelet's HTTPS endpoint.
## Kubelet authentication
By default, requests to the kubelet's HTTPS endpoint that are not rejected by other configured
authentication methods are treated as anonymous requests, and given a username of `system:anonymous`
and a group of `system:unauthenticated`.
To disable anonymous access and send `401 Unauthorized` responses to unauthenticated requests:
* start the kubelet with the `--anonymous-auth=false` flag
To enable X509 client certificate authentication to the kubelet's HTTPS endpoint:
* start the kubelet with the `--client-ca-file` flag, providing a CA bundle to verify client certificates with
* start the apiserver with `--kubelet-client-certificate` and `--kubelet-client-key` flags
* see the [apiserver authentication documentation](/docs/admin/authentication/#x509-client-certs) for more details
To enable API bearer tokens (including service account tokens) to be used to authenticate to the kubelet's HTTPS endpoint:
* ensure the `authentication.k8s.io/v1beta1` API group is enabled in the API server
* start the kubelet with the `--authentication-token-webhook`, `--kubeconfig`, and `--require-kubeconfig` flags
* the kubelet calls the `TokenReview` API on the configured API server to determine user information from bearer tokens
## Kubelet authorization
Any request that is successfully authenticated (including an anonymous request) is then authorized. The default authorization mode is `AlwaysAllow`, which allows all requests.
There are many possible reasons to subdivide access to the kubelet API:
* anonymous auth is enabled, but anonymous users' ability to call the kubelet API should be limited
* bearer token auth is enabled, but arbitrary API users' (like service accounts) ability to call the kubelet API should be limited
* client certificate auth is enabled, but only some of the client certificates signed by the configured CA should be allowed to use the kubelet API
To subdivide access to the kubelet API, delegate authorization to the API server:
* ensure the `authorization.k8s.io/v1beta1` API group is enabled in the API server
* start the kubelet with the `--authorization-mode=Webhook`, `--kubeconfig`, and `--require-kubeconfig` flags
* the kubelet calls the `SubjectAccessReview` API on the configured API server to determine whether each request is authorized
The kubelet authorizes API requests using the same [request attributes](/docs/admin/authorization/#request-attributes) approach as the apiserver.
The verb is determined from the incoming request's HTTP verb:
HTTP verb | request verb
----------|---------------
POST | create
GET, HEAD | get
PUT | update
PATCH | patch
DELETE | delete
The resource and subresource is determined from the incoming request's path:
Kubelet API | resource | subresource
-------------|----------|------------
/stats/\* | nodes | stats
/metrics/\* | nodes | metrics
/logs/\* | nodes | log
/spec/\* | nodes | spec
*all others* | nodes | proxy
The namespace and API group attributes are always an empty string, and
the resource name is always the name of the kubelet's `Node` API object.
When running in this mode, ensure the user identified by the `--kubelet-client-certificate` and `--kubelet-client-key`
flags passed to the apiserver is authorized for the following attributes:
* verb=\*, resource=nodes, subresource=proxy
* verb=\*, resource=nodes, subresource=stats
* verb=\*, resource=nodes, subresource=log
* verb=\*, resource=nodes, subresource=spec
* verb=\*, resource=nodes, subresource=metrics
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.